2022-06-17T12:30:36.478 INFO:root:teuthology version: 1.1.1.dev374+g31a356e0 2022-06-17T12:30:36.482 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821 --description fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} --name teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821/orig.config.yaml 2022-06-17T12:30:36.545 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2022-06-17T12:30:36.647 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821 branch: octopus description: fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} email: ceph-qa@ceph.io first_in_suite: false job_id: '6883821' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 30 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: octopus ceph: cephfs_ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 15m rados osd op timeout: 15m mds: debug mds: 20 debug ms: 1 mds bal frag: true mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 15m rados osd op timeout: 15m mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - evicting unresponsive client - POOL_APP_NOT_ENABLED - has not responded to cap revoke by MDS for over - MDS_CLIENT_LATE_RELEASE - responding to mclientcaps sha1: b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: default sha1: b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: octopus sha1: b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6 owner: scheduled_teuthology@teuthology priority: 1000 repo: git://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 - - client.0 - - client.1 - - client.2 sha1: b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6 sleep_before_teardown: 0 suite: fs suite_branch: octopus suite_path: /home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph.git suite_sha1: b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6 targets: smithi036.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC57MPEtIquGAy9QJplKQzWpsUlNg8CKOuWx+xjj9hH7r3J/8ftanBScqsF8v/E0hm/ThvVNgk3vM5KyRDnJFa3Oq+Z6Eyri4LFUtZh2ty2TMBVkM1ZYd4a59iuMh1HvlVbw+f7J3c5esl3tukfF/4aDeqk566Rp1vgQCt5n7mjqgbHMV9aW37WvSHkeDSH6qhGiLcnyYTb8g1oNyNU5TzaonLKMDj5COEFZKgZUNqC+HbweQzZteAospxQx0fIX80VFKmGEOKbmZPUHK89tiAJzyeaGalRXpopyzcKBZ+8RtKchLS5pRCqT+cTEsfEjFstW9OJM0eo7MKge4Rhm7BYtPYJ7DaqETsuHOghiAZY30hVyG1Yl4hJCEZKigyHEDuNFN0IU5htrAgzZs1ufpQBx2/lpQLExQiUfFOUH4RF6W/TFis4uhzMwxJlFHhRSwxTtTHr+ARBV5X+YsJRm2VzSpFTbxhV6/IjyTocTyLeBFsqiAg7vni3S5sAAOliEGU= smithi070.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCr9BmpKJlgwMIaIyD/z9fXBgHOkzXl5yVZppmk5TezmA1ybxEzwlWxo+qNajL8vh28omeB7pyVVER98fuU3mgqLQ0MzOxaMM7Ow4/nmQHDy/jBbIfo/nm6XqcMtY8dk486t3rmmjfbm+MKE2BOn45c7Zy9L/BzHW1ubTW3wl9KWm3VKqtYM3jzUgPXkXZiqGI1gzSmfs7vkWtUENHu2UHgNGC0AFDLVGiAgCpAKJ9/hO30/8m/9F1wlERPi7XCvTrVQ8QydKYEjOUEyy2zY2jkeZKKABbnEpTvV02YSSGy0g3g+zW+lm2pgTiN6GRsklsHJ6PTk8ZP11fp6cmInw5RVCIkEubp1cAIZS4teFnL6QlBvXYfl7azhbGqLT5BoT/80zrGpMGnSWMcIap3Kbl291HDCWTehs0JiaZ+jlVXTMRjKPL5DdA09M+zGCOj6ffDO+YmeFnmBpgalswEneMaL5pratARdtxifeJDMbV4FTjMSwJrnNHA/7criBd1Mvk= smithi107.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCWY+bhZkRud/tllPmLGstGhb34LtSQQ1kHy82yEkc20qdlIW1ret48XHHIke0JtuKjqSu9vkZF6PwyvJhvd1T8N/wV82vPjrUwV3vmcmErSQzpnOhbmGWiFlPyhf1LAMwCfYTbeUgpOrSI/44t5bBAqoo5BzjR8rh+r1QbnZ+7DJ6Wd1fwmQGke3tjHKbD9+G41SB9bAK7oSzpibvr5cKKie49Ap1FQxBdBuDhv5aUVPDNttT+inD6ZU9e6fQD3B5RPe6FgSUUGSc7vYqqYZfDSOQb5fu+lHhSWnPZ9jc4XKXZHnsgJXRGKRZG2cRPt1HWI7RIndhRuIvtRPjw8mluWlv6qdOXWC49Xt79LUOeyYRBctMolbNUvRBV7kCfhdW7LVAAGPuhS5adYC1pT4QRLdiSb4VBI+nAl/5hVpBAJvEcldSG1FRSNWQsErUjpqUm7f0Z4Z9oWXAF+cjkLAlrryQTveST37axt8NAmIFftQzVrhCWOU/CNuU737WTtFM= smithi136.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCdl7wPIRbjjnibXe7F9vDsAsyQuGVjBhySCFB6BPeliMm5v5XT+W/J7jOh58MslaePsEVYQBJ6sNGQxlkt6X/lDyv+dz7t/jp+IUnE38urxjhjBrE25AIW+wcUJmwLlPD0SGrEP5wZFTuDgRgcNvtZY/Ykl0x24M3TBDjfXtX4bMmuqLg5rEkcudliDL+zITfc7Xd1v1iS6IDJE6ZDZyyUmTesAMbT5auJfzHU3p9KcsNMPhfemLgQ01pUzPOq80sDOqBP1qfMyNkJ2mFGdCbtZGhMzeU7+fLLynA2Z5Lwiv9wZavnvhmyNehfno9NcMqlTO/5gRCXn0nK4e4BtXQGOXhzSzssQb+g++bGbVp6IUuJokAIEMIMmoPFCS/ruBWtfnYH1VayVc2K+J5uzatWVUOBDKhDGR8sFbIXjEstX/raG+h+ByPOxR/8CKYT3Q9duMNz5snnfAkIqCmUqXFA+Uh9c+gnrXvMPNF3d1FB7Km8iLEYxCisud+M17A0JbE= smithi143.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCSQ9FUvkLUVb56bxWPRq0gzjNMepOXMWM7aOuObcI6zVGFGsw/KDlFx6laxwYbTqmYSjE+QN8C1M8uKi1kN4orrjTS7uaGpMqa+vH0GI77OaGFlgzMrDB4WjStiqYP/UBYv1zA5lokyp+fvfwmYIfu/HZPfF+/GyAxJFUrb9cXpxZ071u5Zs/769G1RU/mss17w6FtQIEr1ehDcS1aLY6mQxO/kRbOLnmvfCdHm4LFJLnI+WEQPEWwoJAS9Qi2P3PMwsokOvXrRMqW1iW9hwo91VRU+M9cONYFKM3iDl6n7ge0fJRjopRdvmXAWGZjsi5SlFdyfMD3S95RluC5JT8dk7ENRzh6v2+6cldf0V10LOwvUpPCbCawgfLgfA5sVOMj5hwImJ80F2VmL800zbLwfz4sV4iWbPLdTDrYRbq1fwZzLN1AJbPtajGnPXNJKzPk8ZfKfNKLLQsh9FO+BBlNQrgsuBepK2s9hiQRlvc2nmx2jd2ZO3JD6kPDCLytqoc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell rpm: - python3-cephfs extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_misc teuthology_branch: main teuthology_sha1: 31a356e0c737f1781079045bb8dc366982be8713 timestamp: 2022-06-17_04:10:03 tube: smithi user: teuthology verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1713 2022-06-17T12:30:36.648 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa; will attempt to use it 2022-06-17T12:30:36.649 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks 2022-06-17T12:30:36.650 INFO:teuthology.run_tasks:Running task internal.check_packages... 2022-06-17T12:30:36.652 INFO:teuthology.task.internal:Checking packages... 2022-06-17T12:30:36.677 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6' 2022-06-17T12:30:36.677 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-06-17T12:30:36.678 INFO:teuthology.packaging:ref: None 2022-06-17T12:30:36.678 INFO:teuthology.packaging:tag: None 2022-06-17T12:30:36.678 INFO:teuthology.packaging:branch: octopus 2022-06-17T12:30:36.679 INFO:teuthology.packaging:sha1: b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6 2022-06-17T12:30:36.680 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=octopus 2022-06-17T12:30:37.187 INFO:teuthology.task.internal:Found packages for ceph version 15.2.16-260-gb3641a55-1focal 2022-06-17T12:30:37.188 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2022-06-17T12:30:37.200 INFO:teuthology.task.internal:no buildpackages task found 2022-06-17T12:30:37.201 INFO:teuthology.run_tasks:Running task internal.save_config... 2022-06-17T12:30:37.207 INFO:teuthology.task.internal:Saving configuration 2022-06-17T12:30:37.229 INFO:teuthology.run_tasks:Running task internal.check_lock... 2022-06-17T12:30:37.239 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-06-17T12:30:37.267 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi036.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-06-17 12:24:53.634989', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOWs20qWRFsOROE7uajwVBPEAfsTXCdpBHfdrhqYhVJ8'} 2022-06-17T12:30:37.296 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-06-17 12:24:53.637091', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICQT4RJNOWi/PRV7GUavaZMo8QVtK4uVRLKZ+dFIcSEE'} 2022-06-17T12:30:37.324 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-06-17 12:24:53.638145', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIISgmXtlrgQVW+ZF427+XWYLPDdi4UiA18MEtBTyZfnm'} 2022-06-17T12:30:37.355 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-06-17 12:24:53.636045', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIw1AtZVfJVS/qwtu9dv6hUtQFst0xFlJjXOw+1l0brD'} 2022-06-17T12:30:37.382 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi143.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-06-17 12:24:53.633600', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBVrqPq+6vPFTbuzn2xP9MAWvkBqSLg8rvYVYBpIwTJS'} 2022-06-17T12:30:37.382 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2022-06-17T12:30:37.388 INFO:teuthology.task.internal:roles: ubuntu@smithi036.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2022-06-17T12:30:37.389 INFO:teuthology.task.internal:roles: ubuntu@smithi070.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-06-17T12:30:37.389 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['client.0'] 2022-06-17T12:30:37.389 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['client.1'] 2022-06-17T12:30:37.389 INFO:teuthology.task.internal:roles: ubuntu@smithi143.front.sepia.ceph.com - ['client.2'] 2022-06-17T12:30:37.390 INFO:teuthology.run_tasks:Running task console_log... 2022-06-17T12:30:37.651 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2ccf215840>, signals=[15]) 2022-06-17T12:30:37.652 INFO:teuthology.run_tasks:Running task internal.connect... 2022-06-17T12:30:37.666 INFO:teuthology.task.internal:Opening connections... 2022-06-17T12:30:37.666 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi036.front.sepia.ceph.com 2022-06-17T12:30:37.669 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-17T12:30:37.798 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi070.front.sepia.ceph.com 2022-06-17T12:30:37.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-17T12:30:37.910 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2022-06-17T12:30:37.912 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-17T12:30:38.015 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi136.front.sepia.ceph.com 2022-06-17T12:30:38.016 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-17T12:30:38.149 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi143.front.sepia.ceph.com 2022-06-17T12:30:38.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-17T12:30:38.295 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2022-06-17T12:30:38.303 DEBUG:teuthology.orchestra.run.smithi036:> uname -m 2022-06-17T12:30:38.312 INFO:teuthology.orchestra.run.smithi036.stdout:x86_64 2022-06-17T12:30:38.312 DEBUG:teuthology.orchestra.run.smithi036:> cat /etc/os-release 2022-06-17T12:30:38.365 INFO:teuthology.orchestra.run.smithi036.stdout:NAME="Ubuntu" 2022-06-17T12:30:38.365 INFO:teuthology.orchestra.run.smithi036.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-06-17T12:30:38.366 INFO:teuthology.orchestra.run.smithi036.stdout:ID=ubuntu 2022-06-17T12:30:38.366 INFO:teuthology.orchestra.run.smithi036.stdout:ID_LIKE=debian 2022-06-17T12:30:38.367 INFO:teuthology.orchestra.run.smithi036.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-06-17T12:30:38.367 INFO:teuthology.orchestra.run.smithi036.stdout:VERSION_ID="20.04" 2022-06-17T12:30:38.368 INFO:teuthology.orchestra.run.smithi036.stdout:HOME_URL="https://www.ubuntu.com/" 2022-06-17T12:30:38.368 INFO:teuthology.orchestra.run.smithi036.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-06-17T12:30:38.368 INFO:teuthology.orchestra.run.smithi036.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-06-17T12:30:38.369 INFO:teuthology.orchestra.run.smithi036.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-06-17T12:30:38.369 INFO:teuthology.orchestra.run.smithi036.stdout:VERSION_CODENAME=focal 2022-06-17T12:30:38.370 INFO:teuthology.orchestra.run.smithi036.stdout:UBUNTU_CODENAME=focal 2022-06-17T12:30:38.371 INFO:teuthology.lock.ops:Updating smithi036.front.sepia.ceph.com on lock server 2022-06-17T12:30:38.402 DEBUG:teuthology.orchestra.run.smithi070:> uname -m 2022-06-17T12:30:38.414 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64 2022-06-17T12:30:38.415 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release 2022-06-17T12:30:38.462 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="Ubuntu" 2022-06-17T12:30:38.462 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-06-17T12:30:38.463 INFO:teuthology.orchestra.run.smithi070.stdout:ID=ubuntu 2022-06-17T12:30:38.463 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE=debian 2022-06-17T12:30:38.463 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-06-17T12:30:38.463 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="20.04" 2022-06-17T12:30:38.464 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://www.ubuntu.com/" 2022-06-17T12:30:38.464 INFO:teuthology.orchestra.run.smithi070.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-06-17T12:30:38.464 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-06-17T12:30:38.465 INFO:teuthology.orchestra.run.smithi070.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-06-17T12:30:38.465 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_CODENAME=focal 2022-06-17T12:30:38.465 INFO:teuthology.orchestra.run.smithi070.stdout:UBUNTU_CODENAME=focal 2022-06-17T12:30:38.466 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2022-06-17T12:30:38.492 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2022-06-17T12:30:38.501 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2022-06-17T12:30:38.502 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2022-06-17T12:30:38.550 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="Ubuntu" 2022-06-17T12:30:38.550 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-06-17T12:30:38.551 INFO:teuthology.orchestra.run.smithi107.stdout:ID=ubuntu 2022-06-17T12:30:38.551 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE=debian 2022-06-17T12:30:38.551 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-06-17T12:30:38.552 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="20.04" 2022-06-17T12:30:38.552 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://www.ubuntu.com/" 2022-06-17T12:30:38.552 INFO:teuthology.orchestra.run.smithi107.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-06-17T12:30:38.552 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-06-17T12:30:38.552 INFO:teuthology.orchestra.run.smithi107.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-06-17T12:30:38.553 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_CODENAME=focal 2022-06-17T12:30:38.553 INFO:teuthology.orchestra.run.smithi107.stdout:UBUNTU_CODENAME=focal 2022-06-17T12:30:38.554 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2022-06-17T12:30:38.575 DEBUG:teuthology.orchestra.run.smithi136:> uname -m 2022-06-17T12:30:38.583 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2022-06-17T12:30:38.584 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2022-06-17T12:30:38.633 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="Ubuntu" 2022-06-17T12:30:38.633 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-06-17T12:30:38.633 INFO:teuthology.orchestra.run.smithi136.stdout:ID=ubuntu 2022-06-17T12:30:38.633 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE=debian 2022-06-17T12:30:38.634 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-06-17T12:30:38.634 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="20.04" 2022-06-17T12:30:38.634 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://www.ubuntu.com/" 2022-06-17T12:30:38.634 INFO:teuthology.orchestra.run.smithi136.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-06-17T12:30:38.635 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-06-17T12:30:38.635 INFO:teuthology.orchestra.run.smithi136.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-06-17T12:30:38.635 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_CODENAME=focal 2022-06-17T12:30:38.635 INFO:teuthology.orchestra.run.smithi136.stdout:UBUNTU_CODENAME=focal 2022-06-17T12:30:38.636 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2022-06-17T12:30:38.655 DEBUG:teuthology.orchestra.run.smithi143:> uname -m 2022-06-17T12:30:38.663 INFO:teuthology.orchestra.run.smithi143.stdout:x86_64 2022-06-17T12:30:38.663 DEBUG:teuthology.orchestra.run.smithi143:> cat /etc/os-release 2022-06-17T12:30:38.711 INFO:teuthology.orchestra.run.smithi143.stdout:NAME="Ubuntu" 2022-06-17T12:30:38.711 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-06-17T12:30:38.712 INFO:teuthology.orchestra.run.smithi143.stdout:ID=ubuntu 2022-06-17T12:30:38.712 INFO:teuthology.orchestra.run.smithi143.stdout:ID_LIKE=debian 2022-06-17T12:30:38.713 INFO:teuthology.orchestra.run.smithi143.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-06-17T12:30:38.713 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_ID="20.04" 2022-06-17T12:30:38.713 INFO:teuthology.orchestra.run.smithi143.stdout:HOME_URL="https://www.ubuntu.com/" 2022-06-17T12:30:38.714 INFO:teuthology.orchestra.run.smithi143.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-06-17T12:30:38.715 INFO:teuthology.orchestra.run.smithi143.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-06-17T12:30:38.715 INFO:teuthology.orchestra.run.smithi143.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-06-17T12:30:38.716 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_CODENAME=focal 2022-06-17T12:30:38.716 INFO:teuthology.orchestra.run.smithi143.stdout:UBUNTU_CODENAME=focal 2022-06-17T12:30:38.717 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server 2022-06-17T12:30:38.746 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2022-06-17T12:30:38.772 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2022-06-17T12:30:38.782 INFO:teuthology.task.internal:Checking for old test directory... 2022-06-17T12:30:38.783 DEBUG:teuthology.orchestra.run.smithi036:> test '!' -e /home/ubuntu/cephtest 2022-06-17T12:30:38.786 DEBUG:teuthology.orchestra.run.smithi070:> test '!' -e /home/ubuntu/cephtest 2022-06-17T12:30:38.790 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2022-06-17T12:30:38.797 DEBUG:teuthology.orchestra.run.smithi136:> test '!' -e /home/ubuntu/cephtest 2022-06-17T12:30:38.803 DEBUG:teuthology.orchestra.run.smithi143:> test '!' -e /home/ubuntu/cephtest 2022-06-17T12:30:38.811 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2022-06-17T12:30:38.822 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2022-06-17T12:30:38.823 DEBUG:teuthology.orchestra.run.smithi036:> test -z $(ls -A /var/lib/ceph) 2022-06-17T12:30:38.835 DEBUG:teuthology.orchestra.run.smithi070:> test -z $(ls -A /var/lib/ceph) 2022-06-17T12:30:38.838 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2022-06-17T12:30:38.851 DEBUG:teuthology.orchestra.run.smithi136:> test -z $(ls -A /var/lib/ceph) 2022-06-17T12:30:38.856 DEBUG:teuthology.orchestra.run.smithi143:> test -z $(ls -A /var/lib/ceph) 2022-06-17T12:30:38.876 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2022-06-17T12:30:39.011 INFO:teuthology.run_tasks:Running task kernel... 2022-06-17T12:30:39.043 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2022-06-17T12:30:39.045 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}, 'client.2': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2022-06-17T12:30:39.046 DEBUG:teuthology.orchestra.run.smithi036:> test -f /run/.containerenv -o -f /.dockerenv 2022-06-17T12:30:39.048 DEBUG:teuthology.orchestra.run.smithi070:> test -f /run/.containerenv -o -f /.dockerenv 2022-06-17T12:30:39.049 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2022-06-17T12:30:39.050 DEBUG:teuthology.orchestra.run.smithi136:> test -f /run/.containerenv -o -f /.dockerenv 2022-06-17T12:30:39.051 DEBUG:teuthology.orchestra.run.smithi143:> test -f /run/.containerenv -o -f /.dockerenv 2022-06-17T12:30:39.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:30:39.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:30:39.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:30:39.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:30:39.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:30:39.068 DEBUG:teuthology.orchestra.run.smithi036:> uname -r 2022-06-17T12:30:39.069 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2022-06-17T12:30:39.070 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2022-06-17T12:30:39.070 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2022-06-17T12:30:39.071 DEBUG:teuthology.orchestra.run.smithi143:> uname -r 2022-06-17T12:30:39.077 INFO:teuthology.orchestra.run.smithi136.stdout:5.4.0-117-generic 2022-06-17T12:30:39.078 INFO:teuthology.task.kernel:Running kernel on smithi136: 5.4.0-117-generic 2022-06-17T12:30:39.078 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get clean 2022-06-17T12:30:39.080 INFO:teuthology.orchestra.run.smithi070.stdout:5.4.0-117-generic 2022-06-17T12:30:39.080 INFO:teuthology.task.kernel:Running kernel on smithi070: 5.4.0-117-generic 2022-06-17T12:30:39.080 DEBUG:teuthology.orchestra.run.smithi070:> sudo apt-get clean 2022-06-17T12:30:39.114 INFO:teuthology.orchestra.run.smithi036.stdout:5.4.0-117-generic 2022-06-17T12:30:39.114 INFO:teuthology.task.kernel:Running kernel on smithi036: 5.4.0-117-generic 2022-06-17T12:30:39.116 DEBUG:teuthology.orchestra.run.smithi036:> sudo apt-get clean 2022-06-17T12:30:39.119 INFO:teuthology.orchestra.run.smithi107.stdout:5.4.0-117-generic 2022-06-17T12:30:39.119 INFO:teuthology.task.kernel:Running kernel on smithi107: 5.4.0-117-generic 2022-06-17T12:30:39.120 DEBUG:teuthology.orchestra.run.smithi107:> sudo apt-get clean 2022-06-17T12:30:39.121 INFO:teuthology.orchestra.run.smithi143.stdout:5.4.0-117-generic 2022-06-17T12:30:39.122 INFO:teuthology.task.kernel:Running kernel on smithi143: 5.4.0-117-generic 2022-06-17T12:30:39.122 DEBUG:teuthology.orchestra.run.smithi143:> sudo apt-get clean 2022-06-17T12:30:39.175 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get update 2022-06-17T12:30:39.199 DEBUG:teuthology.orchestra.run.smithi070:> sudo apt-get update 2022-06-17T12:30:39.206 DEBUG:teuthology.orchestra.run.smithi036:> sudo apt-get update 2022-06-17T12:30:39.219 DEBUG:teuthology.orchestra.run.smithi107:> sudo apt-get update 2022-06-17T12:30:39.227 DEBUG:teuthology.orchestra.run.smithi143:> sudo apt-get update 2022-06-17T12:30:39.372 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-17T12:30:39.376 INFO:teuthology.orchestra.run.smithi136.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-06-17T12:30:39.392 INFO:teuthology.orchestra.run.smithi136.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-06-17T12:30:39.492 INFO:teuthology.orchestra.run.smithi070.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-17T12:30:39.492 INFO:teuthology.orchestra.run.smithi070.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-06-17T12:30:39.501 INFO:teuthology.orchestra.run.smithi036.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-17T12:30:39.501 INFO:teuthology.orchestra.run.smithi036.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-06-17T12:30:39.512 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-17T12:30:39.512 INFO:teuthology.orchestra.run.smithi107.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-06-17T12:30:39.535 INFO:teuthology.orchestra.run.smithi136.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-06-17T12:30:39.536 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-17T12:30:39.536 INFO:teuthology.orchestra.run.smithi143.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-06-17T12:30:39.541 INFO:teuthology.orchestra.run.smithi070.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-06-17T12:30:39.556 INFO:teuthology.orchestra.run.smithi036.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-06-17T12:30:39.559 INFO:teuthology.orchestra.run.smithi107.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-06-17T12:30:39.572 INFO:teuthology.orchestra.run.smithi143.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-06-17T12:30:39.758 INFO:teuthology.orchestra.run.smithi136.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,579 kB] 2022-06-17T12:30:39.879 INFO:teuthology.orchestra.run.smithi070.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-06-17T12:30:39.892 INFO:teuthology.orchestra.run.smithi036.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-06-17T12:30:39.899 INFO:teuthology.orchestra.run.smithi107.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-06-17T12:30:39.913 INFO:teuthology.orchestra.run.smithi136.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,924 kB] 2022-06-17T12:30:39.920 INFO:teuthology.orchestra.run.smithi143.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-06-17T12:30:39.953 INFO:teuthology.orchestra.run.smithi070.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [458 kB] 2022-06-17T12:30:39.979 INFO:teuthology.orchestra.run.smithi107.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [458 kB] 2022-06-17T12:30:39.987 INFO:teuthology.orchestra.run.smithi143.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [458 kB] 2022-06-17T12:30:40.001 INFO:teuthology.orchestra.run.smithi136.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [458 kB] 2022-06-17T12:30:40.009 INFO:teuthology.orchestra.run.smithi136.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [268 kB] 2022-06-17T12:30:40.012 INFO:teuthology.orchestra.run.smithi136.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [10.6 kB] 2022-06-17T12:30:40.013 INFO:teuthology.orchestra.run.smithi136.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,031 kB] 2022-06-17T12:30:40.083 INFO:teuthology.orchestra.run.smithi036.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [458 kB] 2022-06-17T12:30:40.095 INFO:teuthology.orchestra.run.smithi070.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [678 kB] 2022-06-17T12:30:40.107 INFO:teuthology.orchestra.run.smithi136.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [146 kB] 2022-06-17T12:30:40.110 INFO:teuthology.orchestra.run.smithi107.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [678 kB] 2022-06-17T12:30:40.111 INFO:teuthology.orchestra.run.smithi136.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [708 kB] 2022-06-17T12:30:40.120 INFO:teuthology.orchestra.run.smithi136.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [554 kB] 2022-06-17T12:30:40.123 INFO:teuthology.orchestra.run.smithi136.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [127 kB] 2022-06-17T12:30:40.124 INFO:teuthology.orchestra.run.smithi136.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [14.6 kB] 2022-06-17T12:30:40.125 INFO:teuthology.orchestra.run.smithi136.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [22.2 kB] 2022-06-17T12:30:40.125 INFO:teuthology.orchestra.run.smithi136.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [512 B] 2022-06-17T12:30:40.134 INFO:teuthology.orchestra.run.smithi143.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,924 kB] 2022-06-17T12:30:40.166 INFO:teuthology.orchestra.run.smithi136.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [678 kB] 2022-06-17T12:30:40.192 INFO:teuthology.orchestra.run.smithi136.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [348 kB] 2022-06-17T12:30:40.203 INFO:teuthology.orchestra.run.smithi136.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [15.6 kB] 2022-06-17T12:30:40.204 INFO:teuthology.orchestra.run.smithi136.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,093 kB] 2022-06-17T12:30:40.222 INFO:teuthology.orchestra.run.smithi036.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,924 kB] 2022-06-17T12:30:40.240 INFO:teuthology.orchestra.run.smithi136.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [156 kB] 2022-06-17T12:30:40.243 INFO:teuthology.orchestra.run.smithi136.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [684 kB] 2022-06-17T12:30:40.247 INFO:teuthology.orchestra.run.smithi070.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,579 kB] 2022-06-17T12:30:40.259 INFO:teuthology.orchestra.run.smithi136.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [931 kB] 2022-06-17T12:30:40.269 INFO:teuthology.orchestra.run.smithi107.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,579 kB] 2022-06-17T12:30:40.276 INFO:teuthology.orchestra.run.smithi136.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [209 kB] 2022-06-17T12:30:40.280 INFO:teuthology.orchestra.run.smithi136.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [20.9 kB] 2022-06-17T12:30:40.280 INFO:teuthology.orchestra.run.smithi136.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [24.4 kB] 2022-06-17T12:30:40.281 INFO:teuthology.orchestra.run.smithi136.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [596 B] 2022-06-17T12:30:40.282 INFO:teuthology.orchestra.run.smithi143.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,579 kB] 2022-06-17T12:30:40.282 INFO:teuthology.orchestra.run.smithi136.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [35.2 kB] 2022-06-17T12:30:40.283 INFO:teuthology.orchestra.run.smithi136.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [44.8 kB] 2022-06-17T12:30:40.313 INFO:teuthology.orchestra.run.smithi136.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-backports/main Translation-en [11.3 kB] 2022-06-17T12:30:40.314 INFO:teuthology.orchestra.run.smithi136.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [976 B] 2022-06-17T12:30:40.379 INFO:teuthology.orchestra.run.smithi036.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,579 kB] 2022-06-17T12:30:40.383 INFO:teuthology.orchestra.run.smithi070.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,924 kB] 2022-06-17T12:30:40.399 INFO:teuthology.orchestra.run.smithi107.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,924 kB] 2022-06-17T12:30:40.421 INFO:teuthology.orchestra.run.smithi070.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [268 kB] 2022-06-17T12:30:40.436 INFO:teuthology.orchestra.run.smithi070.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [10.6 kB] 2022-06-17T12:30:40.437 INFO:teuthology.orchestra.run.smithi070.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,031 kB] 2022-06-17T12:30:40.443 INFO:teuthology.orchestra.run.smithi107.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [268 kB] 2022-06-17T12:30:40.454 INFO:teuthology.orchestra.run.smithi143.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [268 kB] 2022-06-17T12:30:40.454 INFO:teuthology.orchestra.run.smithi107.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [10.6 kB] 2022-06-17T12:30:40.455 INFO:teuthology.orchestra.run.smithi107.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,031 kB] 2022-06-17T12:30:40.467 INFO:teuthology.orchestra.run.smithi143.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [10.6 kB] 2022-06-17T12:30:40.468 INFO:teuthology.orchestra.run.smithi143.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,031 kB] 2022-06-17T12:30:40.486 INFO:teuthology.orchestra.run.smithi070.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [146 kB] 2022-06-17T12:30:40.492 INFO:teuthology.orchestra.run.smithi070.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [708 kB] 2022-06-17T12:30:40.507 INFO:teuthology.orchestra.run.smithi107.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [146 kB] 2022-06-17T12:30:40.514 INFO:teuthology.orchestra.run.smithi070.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [554 kB] 2022-06-17T12:30:40.516 INFO:teuthology.orchestra.run.smithi107.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [554 kB] 2022-06-17T12:30:40.518 INFO:teuthology.orchestra.run.smithi143.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [146 kB] 2022-06-17T12:30:40.524 INFO:teuthology.orchestra.run.smithi143.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [708 kB] 2022-06-17T12:30:40.529 INFO:teuthology.orchestra.run.smithi070.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [127 kB] 2022-06-17T12:30:40.530 INFO:teuthology.orchestra.run.smithi107.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [708 kB] 2022-06-17T12:30:40.532 INFO:teuthology.orchestra.run.smithi070.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [14.6 kB] 2022-06-17T12:30:40.532 INFO:teuthology.orchestra.run.smithi070.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [22.2 kB] 2022-06-17T12:30:40.533 INFO:teuthology.orchestra.run.smithi070.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [512 B] 2022-06-17T12:30:40.541 INFO:teuthology.orchestra.run.smithi070.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [348 kB] 2022-06-17T12:30:40.543 INFO:teuthology.orchestra.run.smithi143.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [678 kB] 2022-06-17T12:30:40.546 INFO:teuthology.orchestra.run.smithi143.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [554 kB] 2022-06-17T12:30:40.549 INFO:teuthology.orchestra.run.smithi107.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [127 kB] 2022-06-17T12:30:40.552 INFO:teuthology.orchestra.run.smithi107.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [14.6 kB] 2022-06-17T12:30:40.552 INFO:teuthology.orchestra.run.smithi107.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [22.2 kB] 2022-06-17T12:30:40.553 INFO:teuthology.orchestra.run.smithi036.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [268 kB] 2022-06-17T12:30:40.554 INFO:teuthology.orchestra.run.smithi107.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [512 B] 2022-06-17T12:30:40.555 INFO:teuthology.orchestra.run.smithi070.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [15.6 kB] 2022-06-17T12:30:40.556 INFO:teuthology.orchestra.run.smithi070.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,093 kB] 2022-06-17T12:30:40.558 INFO:teuthology.orchestra.run.smithi107.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [348 kB] 2022-06-17T12:30:40.560 INFO:teuthology.orchestra.run.smithi143.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [127 kB] 2022-06-17T12:30:40.561 INFO:teuthology.orchestra.run.smithi143.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [348 kB] 2022-06-17T12:30:40.563 INFO:teuthology.orchestra.run.smithi143.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [14.6 kB] 2022-06-17T12:30:40.564 INFO:teuthology.orchestra.run.smithi143.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [22.2 kB] 2022-06-17T12:30:40.565 INFO:teuthology.orchestra.run.smithi143.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [512 B] 2022-06-17T12:30:40.565 INFO:teuthology.orchestra.run.smithi036.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [10.6 kB] 2022-06-17T12:30:40.568 INFO:teuthology.orchestra.run.smithi036.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,031 kB] 2022-06-17T12:30:40.570 INFO:teuthology.orchestra.run.smithi107.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [15.6 kB] 2022-06-17T12:30:40.571 INFO:teuthology.orchestra.run.smithi143.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [15.6 kB] 2022-06-17T12:30:40.572 INFO:teuthology.orchestra.run.smithi143.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,093 kB] 2022-06-17T12:30:40.573 INFO:teuthology.orchestra.run.smithi107.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,093 kB] 2022-06-17T12:30:40.594 INFO:teuthology.orchestra.run.smithi070.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [156 kB] 2022-06-17T12:30:40.599 INFO:teuthology.orchestra.run.smithi070.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [931 kB] 2022-06-17T12:30:40.611 INFO:teuthology.orchestra.run.smithi107.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [156 kB] 2022-06-17T12:30:40.617 INFO:teuthology.orchestra.run.smithi107.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [684 kB] 2022-06-17T12:30:40.622 INFO:teuthology.orchestra.run.smithi036.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [146 kB] 2022-06-17T12:30:40.626 INFO:teuthology.orchestra.run.smithi036.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [554 kB] 2022-06-17T12:30:40.627 INFO:teuthology.orchestra.run.smithi143.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [156 kB] 2022-06-17T12:30:40.629 INFO:teuthology.orchestra.run.smithi036.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [678 kB] 2022-06-17T12:30:40.630 INFO:teuthology.orchestra.run.smithi070.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [684 kB] 2022-06-17T12:30:40.631 INFO:teuthology.orchestra.run.smithi143.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [931 kB] 2022-06-17T12:30:40.642 INFO:teuthology.orchestra.run.smithi036.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [708 kB] 2022-06-17T12:30:40.643 INFO:teuthology.orchestra.run.smithi070.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [209 kB] 2022-06-17T12:30:40.644 INFO:teuthology.orchestra.run.smithi107.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [931 kB] 2022-06-17T12:30:40.646 INFO:teuthology.orchestra.run.smithi070.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [20.9 kB] 2022-06-17T12:30:40.647 INFO:teuthology.orchestra.run.smithi070.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [24.4 kB] 2022-06-17T12:30:40.648 INFO:teuthology.orchestra.run.smithi070.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [596 B] 2022-06-17T12:30:40.648 INFO:teuthology.orchestra.run.smithi036.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [348 kB] 2022-06-17T12:30:40.656 INFO:teuthology.orchestra.run.smithi143.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [684 kB] 2022-06-17T12:30:40.659 INFO:teuthology.orchestra.run.smithi036.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [15.6 kB] 2022-06-17T12:30:40.659 INFO:teuthology.orchestra.run.smithi036.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,093 kB] 2022-06-17T12:30:40.663 INFO:teuthology.orchestra.run.smithi107.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [209 kB] 2022-06-17T12:30:40.664 INFO:teuthology.orchestra.run.smithi036.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [127 kB] 2022-06-17T12:30:40.665 INFO:teuthology.orchestra.run.smithi036.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [14.6 kB] 2022-06-17T12:30:40.665 INFO:teuthology.orchestra.run.smithi036.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [22.2 kB] 2022-06-17T12:30:40.666 INFO:teuthology.orchestra.run.smithi036.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [512 B] 2022-06-17T12:30:40.667 INFO:teuthology.orchestra.run.smithi107.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [20.9 kB] 2022-06-17T12:30:40.668 INFO:teuthology.orchestra.run.smithi107.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [24.4 kB] 2022-06-17T12:30:40.668 INFO:teuthology.orchestra.run.smithi107.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [596 B] 2022-06-17T12:30:40.674 INFO:teuthology.orchestra.run.smithi143.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [209 kB] 2022-06-17T12:30:40.683 INFO:teuthology.orchestra.run.smithi143.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [20.9 kB] 2022-06-17T12:30:40.690 INFO:teuthology.orchestra.run.smithi143.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [24.4 kB] 2022-06-17T12:30:40.691 INFO:teuthology.orchestra.run.smithi143.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [596 B] 2022-06-17T12:30:40.723 INFO:teuthology.orchestra.run.smithi107.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [44.8 kB] 2022-06-17T12:30:40.723 INFO:teuthology.orchestra.run.smithi107.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [35.2 kB] 2022-06-17T12:30:40.724 INFO:teuthology.orchestra.run.smithi036.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [156 kB] 2022-06-17T12:30:40.725 INFO:teuthology.orchestra.run.smithi036.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [684 kB] 2022-06-17T12:30:40.725 INFO:teuthology.orchestra.run.smithi070.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [44.8 kB] 2022-06-17T12:30:40.726 INFO:teuthology.orchestra.run.smithi070.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [35.2 kB] 2022-06-17T12:30:40.726 INFO:teuthology.orchestra.run.smithi070.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-backports/main Translation-en [11.3 kB] 2022-06-17T12:30:40.727 INFO:teuthology.orchestra.run.smithi070.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [976 B] 2022-06-17T12:30:40.729 INFO:teuthology.orchestra.run.smithi107.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-backports/main Translation-en [11.3 kB] 2022-06-17T12:30:40.729 INFO:teuthology.orchestra.run.smithi107.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [976 B] 2022-06-17T12:30:40.730 INFO:teuthology.orchestra.run.smithi143.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [44.8 kB] 2022-06-17T12:30:40.730 INFO:teuthology.orchestra.run.smithi143.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [35.2 kB] 2022-06-17T12:30:40.730 INFO:teuthology.orchestra.run.smithi143.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-backports/main Translation-en [11.3 kB] 2022-06-17T12:30:40.731 INFO:teuthology.orchestra.run.smithi143.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [976 B] 2022-06-17T12:30:40.735 INFO:teuthology.orchestra.run.smithi036.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [931 kB] 2022-06-17T12:30:40.764 INFO:teuthology.orchestra.run.smithi036.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [209 kB] 2022-06-17T12:30:40.766 INFO:teuthology.orchestra.run.smithi036.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [20.9 kB] 2022-06-17T12:30:40.767 INFO:teuthology.orchestra.run.smithi036.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [24.4 kB] 2022-06-17T12:30:40.767 INFO:teuthology.orchestra.run.smithi036.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [596 B] 2022-06-17T12:30:40.810 INFO:teuthology.orchestra.run.smithi036.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [44.8 kB] 2022-06-17T12:30:40.811 INFO:teuthology.orchestra.run.smithi036.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [35.2 kB] 2022-06-17T12:30:40.812 INFO:teuthology.orchestra.run.smithi036.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-backports/main Translation-en [11.3 kB] 2022-06-17T12:30:40.812 INFO:teuthology.orchestra.run.smithi036.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [976 B] 2022-06-17T12:30:44.516 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 11.4 MB in 2s (5,839 kB/s) 2022-06-17T12:30:44.761 INFO:teuthology.orchestra.run.smithi107.stdout:Fetched 11.4 MB in 2s (4,920 kB/s) 2022-06-17T12:30:44.797 INFO:teuthology.orchestra.run.smithi070.stdout:Fetched 11.4 MB in 2s (5,006 kB/s) 2022-06-17T12:30:44.858 INFO:teuthology.orchestra.run.smithi143.stdout:Fetched 11.4 MB in 2s (4,856 kB/s) 2022-06-17T12:30:45.108 INFO:teuthology.orchestra.run.smithi036.stdout:Fetched 11.4 MB in 3s (4,537 kB/s) 2022-06-17T12:30:45.837 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2022-06-17T12:30:45.860 DEBUG:teuthology.orchestra.run.smithi136:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-06-17T12:30:45.944 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2022-06-17T12:30:46.038 INFO:teuthology.orchestra.run.smithi107.stdout:Reading package lists... 2022-06-17T12:30:46.063 DEBUG:teuthology.orchestra.run.smithi107:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-06-17T12:30:46.090 INFO:teuthology.orchestra.run.smithi143.stdout:Reading package lists... 2022-06-17T12:30:46.112 DEBUG:teuthology.orchestra.run.smithi143:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-06-17T12:30:46.119 INFO:teuthology.orchestra.run.smithi136.stdout:Building dependency tree... 2022-06-17T12:30:46.120 INFO:teuthology.orchestra.run.smithi136.stdout:Reading state information... 2022-06-17T12:30:46.121 INFO:teuthology.orchestra.run.smithi070.stdout:Reading package lists... 2022-06-17T12:30:46.132 INFO:teuthology.orchestra.run.smithi107.stdout:Reading package lists... 2022-06-17T12:30:46.147 DEBUG:teuthology.orchestra.run.smithi070:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-06-17T12:30:46.189 INFO:teuthology.orchestra.run.smithi143.stdout:Reading package lists... 2022-06-17T12:30:46.224 INFO:teuthology.orchestra.run.smithi070.stdout:Reading package lists... 2022-06-17T12:30:46.277 INFO:teuthology.orchestra.run.smithi136.stdout:The following additional packages will be installed: 2022-06-17T12:30:46.277 INFO:teuthology.orchestra.run.smithi136.stdout: linux-generic linux-headers-5.4.0-120 linux-headers-5.4.0-120-generic 2022-06-17T12:30:46.278 INFO:teuthology.orchestra.run.smithi136.stdout: linux-headers-generic linux-image-5.4.0-120-generic 2022-06-17T12:30:46.278 INFO:teuthology.orchestra.run.smithi136.stdout: linux-modules-5.4.0-120-generic linux-modules-extra-5.4.0-120-generic 2022-06-17T12:30:46.279 INFO:teuthology.orchestra.run.smithi136.stdout:Suggested packages: 2022-06-17T12:30:46.280 INFO:teuthology.orchestra.run.smithi136.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2022-06-17T12:30:46.280 INFO:teuthology.orchestra.run.smithi136.stdout:Recommended packages: 2022-06-17T12:30:46.280 INFO:teuthology.orchestra.run.smithi136.stdout: thermald 2022-06-17T12:30:46.306 INFO:teuthology.orchestra.run.smithi107.stdout:Building dependency tree... 2022-06-17T12:30:46.307 INFO:teuthology.orchestra.run.smithi107.stdout:Reading state information... 2022-06-17T12:30:46.337 INFO:teuthology.orchestra.run.smithi136.stdout:The following NEW packages will be installed: 2022-06-17T12:30:46.338 INFO:teuthology.orchestra.run.smithi136.stdout: linux-headers-5.4.0-120 linux-headers-5.4.0-120-generic 2022-06-17T12:30:46.338 INFO:teuthology.orchestra.run.smithi136.stdout: linux-image-5.4.0-120-generic linux-modules-5.4.0-120-generic 2022-06-17T12:30:46.339 INFO:teuthology.orchestra.run.smithi136.stdout: linux-modules-extra-5.4.0-120-generic 2022-06-17T12:30:46.340 INFO:teuthology.orchestra.run.smithi136.stdout:The following packages will be upgraded: 2022-06-17T12:30:46.340 INFO:teuthology.orchestra.run.smithi136.stdout: linux-generic linux-headers-generic linux-image-generic 2022-06-17T12:30:46.364 INFO:teuthology.orchestra.run.smithi143.stdout:Building dependency tree... 2022-06-17T12:30:46.365 INFO:teuthology.orchestra.run.smithi143.stdout:Reading state information... 2022-06-17T12:30:46.388 INFO:teuthology.orchestra.run.smithi036.stdout:Reading package lists... 2022-06-17T12:30:46.406 INFO:teuthology.orchestra.run.smithi070.stdout:Building dependency tree... 2022-06-17T12:30:46.409 INFO:teuthology.orchestra.run.smithi070.stdout:Reading state information... 2022-06-17T12:30:46.411 DEBUG:teuthology.orchestra.run.smithi036:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-06-17T12:30:46.464 INFO:teuthology.orchestra.run.smithi107.stdout:The following additional packages will be installed: 2022-06-17T12:30:46.465 INFO:teuthology.orchestra.run.smithi107.stdout: linux-generic linux-headers-5.4.0-120 linux-headers-5.4.0-120-generic 2022-06-17T12:30:46.465 INFO:teuthology.orchestra.run.smithi107.stdout: linux-headers-generic linux-image-5.4.0-120-generic 2022-06-17T12:30:46.466 INFO:teuthology.orchestra.run.smithi107.stdout: linux-modules-5.4.0-120-generic linux-modules-extra-5.4.0-120-generic 2022-06-17T12:30:46.466 INFO:teuthology.orchestra.run.smithi107.stdout:Suggested packages: 2022-06-17T12:30:46.467 INFO:teuthology.orchestra.run.smithi107.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2022-06-17T12:30:46.467 INFO:teuthology.orchestra.run.smithi107.stdout:Recommended packages: 2022-06-17T12:30:46.467 INFO:teuthology.orchestra.run.smithi107.stdout: thermald 2022-06-17T12:30:46.490 INFO:teuthology.orchestra.run.smithi036.stdout:Reading package lists... 2022-06-17T12:30:46.513 INFO:teuthology.orchestra.run.smithi107.stdout:The following NEW packages will be installed: 2022-06-17T12:30:46.514 INFO:teuthology.orchestra.run.smithi107.stdout: linux-headers-5.4.0-120 linux-headers-5.4.0-120-generic 2022-06-17T12:30:46.514 INFO:teuthology.orchestra.run.smithi107.stdout: linux-image-5.4.0-120-generic linux-modules-5.4.0-120-generic 2022-06-17T12:30:46.514 INFO:teuthology.orchestra.run.smithi107.stdout: linux-modules-extra-5.4.0-120-generic 2022-06-17T12:30:46.516 INFO:teuthology.orchestra.run.smithi136.stdout:3 upgraded, 5 newly installed, 0 to remove and 5 not upgraded. 2022-06-17T12:30:46.516 INFO:teuthology.orchestra.run.smithi136.stdout:Need to get 77.1 MB of archives. 2022-06-17T12:30:46.517 INFO:teuthology.orchestra.run.smithi136.stdout:After this operation, 380 MB of additional disk space will be used. 2022-06-17T12:30:46.517 INFO:teuthology.orchestra.run.smithi136.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-120-generic amd64 5.4.0-120.136 [15.0 MB] 2022-06-17T12:30:46.517 INFO:teuthology.orchestra.run.smithi107.stdout:The following packages will be upgraded: 2022-06-17T12:30:46.517 INFO:teuthology.orchestra.run.smithi107.stdout: linux-generic linux-headers-generic linux-image-generic 2022-06-17T12:30:46.524 INFO:teuthology.orchestra.run.smithi143.stdout:The following additional packages will be installed: 2022-06-17T12:30:46.525 INFO:teuthology.orchestra.run.smithi143.stdout: linux-generic linux-headers-5.4.0-120 linux-headers-5.4.0-120-generic 2022-06-17T12:30:46.525 INFO:teuthology.orchestra.run.smithi143.stdout: linux-headers-generic linux-image-5.4.0-120-generic 2022-06-17T12:30:46.526 INFO:teuthology.orchestra.run.smithi143.stdout: linux-modules-5.4.0-120-generic linux-modules-extra-5.4.0-120-generic 2022-06-17T12:30:46.526 INFO:teuthology.orchestra.run.smithi143.stdout:Suggested packages: 2022-06-17T12:30:46.527 INFO:teuthology.orchestra.run.smithi143.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2022-06-17T12:30:46.527 INFO:teuthology.orchestra.run.smithi143.stdout:Recommended packages: 2022-06-17T12:30:46.528 INFO:teuthology.orchestra.run.smithi143.stdout: thermald 2022-06-17T12:30:46.561 INFO:teuthology.orchestra.run.smithi070.stdout:The following additional packages will be installed: 2022-06-17T12:30:46.562 INFO:teuthology.orchestra.run.smithi070.stdout: linux-generic linux-headers-5.4.0-120 linux-headers-5.4.0-120-generic 2022-06-17T12:30:46.562 INFO:teuthology.orchestra.run.smithi070.stdout: linux-headers-generic linux-image-5.4.0-120-generic 2022-06-17T12:30:46.562 INFO:teuthology.orchestra.run.smithi070.stdout: linux-modules-5.4.0-120-generic linux-modules-extra-5.4.0-120-generic 2022-06-17T12:30:46.563 INFO:teuthology.orchestra.run.smithi070.stdout:Suggested packages: 2022-06-17T12:30:46.563 INFO:teuthology.orchestra.run.smithi070.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2022-06-17T12:30:46.563 INFO:teuthology.orchestra.run.smithi070.stdout:Recommended packages: 2022-06-17T12:30:46.564 INFO:teuthology.orchestra.run.smithi070.stdout: thermald 2022-06-17T12:30:46.576 INFO:teuthology.orchestra.run.smithi143.stdout:The following NEW packages will be installed: 2022-06-17T12:30:46.576 INFO:teuthology.orchestra.run.smithi143.stdout: linux-headers-5.4.0-120 linux-headers-5.4.0-120-generic 2022-06-17T12:30:46.577 INFO:teuthology.orchestra.run.smithi143.stdout: linux-image-5.4.0-120-generic linux-modules-5.4.0-120-generic 2022-06-17T12:30:46.577 INFO:teuthology.orchestra.run.smithi143.stdout: linux-modules-extra-5.4.0-120-generic 2022-06-17T12:30:46.587 INFO:teuthology.orchestra.run.smithi143.stdout:The following packages will be upgraded: 2022-06-17T12:30:46.587 INFO:teuthology.orchestra.run.smithi143.stdout: linux-generic linux-headers-generic linux-image-generic 2022-06-17T12:30:46.615 INFO:teuthology.orchestra.run.smithi070.stdout:The following NEW packages will be installed: 2022-06-17T12:30:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: linux-headers-5.4.0-120 linux-headers-5.4.0-120-generic 2022-06-17T12:30:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: linux-image-5.4.0-120-generic linux-modules-5.4.0-120-generic 2022-06-17T12:30:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: linux-modules-extra-5.4.0-120-generic 2022-06-17T12:30:46.618 INFO:teuthology.orchestra.run.smithi070.stdout:The following packages will be upgraded: 2022-06-17T12:30:46.618 INFO:teuthology.orchestra.run.smithi070.stdout: linux-generic linux-headers-generic linux-image-generic 2022-06-17T12:30:46.627 INFO:teuthology.orchestra.run.smithi143.stdout:3 upgraded, 5 newly installed, 0 to remove and 5 not upgraded. 2022-06-17T12:30:46.628 INFO:teuthology.orchestra.run.smithi143.stdout:Need to get 77.1 MB of archives. 2022-06-17T12:30:46.628 INFO:teuthology.orchestra.run.smithi143.stdout:After this operation, 380 MB of additional disk space will be used. 2022-06-17T12:30:46.628 INFO:teuthology.orchestra.run.smithi143.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-120-generic amd64 5.4.0-120.136 [15.0 MB] 2022-06-17T12:30:46.661 INFO:teuthology.orchestra.run.smithi036.stdout:Building dependency tree... 2022-06-17T12:30:46.661 INFO:teuthology.orchestra.run.smithi036.stdout:Reading state information... 2022-06-17T12:30:46.691 INFO:teuthology.orchestra.run.smithi107.stdout:3 upgraded, 5 newly installed, 0 to remove and 5 not upgraded. 2022-06-17T12:30:46.692 INFO:teuthology.orchestra.run.smithi107.stdout:Need to get 77.1 MB of archives. 2022-06-17T12:30:46.692 INFO:teuthology.orchestra.run.smithi107.stdout:After this operation, 380 MB of additional disk space will be used. 2022-06-17T12:30:46.693 INFO:teuthology.orchestra.run.smithi107.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-120-generic amd64 5.4.0-120.136 [15.0 MB] 2022-06-17T12:30:46.794 INFO:teuthology.orchestra.run.smithi070.stdout:3 upgraded, 5 newly installed, 0 to remove and 5 not upgraded. 2022-06-17T12:30:46.795 INFO:teuthology.orchestra.run.smithi070.stdout:Need to get 77.1 MB of archives. 2022-06-17T12:30:46.795 INFO:teuthology.orchestra.run.smithi070.stdout:After this operation, 380 MB of additional disk space will be used. 2022-06-17T12:30:46.795 INFO:teuthology.orchestra.run.smithi070.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-120-generic amd64 5.4.0-120.136 [15.0 MB] 2022-06-17T12:30:46.821 INFO:teuthology.orchestra.run.smithi036.stdout:The following additional packages will be installed: 2022-06-17T12:30:46.822 INFO:teuthology.orchestra.run.smithi036.stdout: linux-generic linux-headers-5.4.0-120 linux-headers-5.4.0-120-generic 2022-06-17T12:30:46.822 INFO:teuthology.orchestra.run.smithi036.stdout: linux-headers-generic linux-image-5.4.0-120-generic 2022-06-17T12:30:46.823 INFO:teuthology.orchestra.run.smithi036.stdout: linux-modules-5.4.0-120-generic linux-modules-extra-5.4.0-120-generic 2022-06-17T12:30:46.823 INFO:teuthology.orchestra.run.smithi036.stdout:Suggested packages: 2022-06-17T12:30:46.824 INFO:teuthology.orchestra.run.smithi036.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2022-06-17T12:30:46.824 INFO:teuthology.orchestra.run.smithi036.stdout:Recommended packages: 2022-06-17T12:30:46.824 INFO:teuthology.orchestra.run.smithi036.stdout: thermald 2022-06-17T12:30:46.893 INFO:teuthology.orchestra.run.smithi036.stdout:The following NEW packages will be installed: 2022-06-17T12:30:46.894 INFO:teuthology.orchestra.run.smithi036.stdout: linux-headers-5.4.0-120 linux-headers-5.4.0-120-generic 2022-06-17T12:30:46.894 INFO:teuthology.orchestra.run.smithi036.stdout: linux-image-5.4.0-120-generic linux-modules-5.4.0-120-generic 2022-06-17T12:30:46.895 INFO:teuthology.orchestra.run.smithi036.stdout: linux-modules-extra-5.4.0-120-generic 2022-06-17T12:30:46.895 INFO:teuthology.orchestra.run.smithi036.stdout:The following packages will be upgraded: 2022-06-17T12:30:46.896 INFO:teuthology.orchestra.run.smithi036.stdout: linux-generic linux-headers-generic linux-image-generic 2022-06-17T12:30:46.955 INFO:teuthology.orchestra.run.smithi143.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-120-generic amd64 5.4.0-120.136 [10.5 MB] 2022-06-17T12:30:47.053 INFO:teuthology.orchestra.run.smithi143.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-120-generic amd64 5.4.0-120.136 [39.3 MB] 2022-06-17T12:30:47.073 INFO:teuthology.orchestra.run.smithi036.stdout:3 upgraded, 5 newly installed, 0 to remove and 5 not upgraded. 2022-06-17T12:30:47.073 INFO:teuthology.orchestra.run.smithi036.stdout:Need to get 77.1 MB of archives. 2022-06-17T12:30:47.074 INFO:teuthology.orchestra.run.smithi036.stdout:After this operation, 380 MB of additional disk space will be used. 2022-06-17T12:30:47.074 INFO:teuthology.orchestra.run.smithi036.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-120-generic amd64 5.4.0-120.136 [15.0 MB] 2022-06-17T12:30:47.593 INFO:teuthology.orchestra.run.smithi143.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.120.121 [1,900 B] 2022-06-17T12:30:47.594 INFO:teuthology.orchestra.run.smithi143.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.120.121 [2,548 B] 2022-06-17T12:30:47.595 INFO:teuthology.orchestra.run.smithi143.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-120 all 5.4.0-120.136 [11.0 MB] 2022-06-17T12:30:47.660 INFO:teuthology.orchestra.run.smithi136.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-120-generic amd64 5.4.0-120.136 [10.5 MB] 2022-06-17T12:30:47.688 INFO:teuthology.orchestra.run.smithi143.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-120-generic amd64 5.4.0-120.136 [1,393 kB] 2022-06-17T12:30:47.699 INFO:teuthology.orchestra.run.smithi143.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.120.121 [2,416 B] 2022-06-17T12:30:47.842 INFO:teuthology.orchestra.run.smithi107.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-120-generic amd64 5.4.0-120.136 [10.5 MB] 2022-06-17T12:30:47.928 INFO:teuthology.orchestra.run.smithi136.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-120-generic amd64 5.4.0-120.136 [39.3 MB] 2022-06-17T12:30:47.938 INFO:teuthology.orchestra.run.smithi070.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-120-generic amd64 5.4.0-120.136 [10.5 MB] 2022-06-17T12:30:48.110 INFO:teuthology.orchestra.run.smithi107.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-120-generic amd64 5.4.0-120.136 [39.3 MB] 2022-06-17T12:30:48.129 INFO:teuthology.orchestra.run.smithi143.stdout:Fetched 77.1 MB in 1s (69.1 MB/s) 2022-06-17T12:30:48.203 INFO:teuthology.orchestra.run.smithi070.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-120-generic amd64 5.4.0-120.136 [39.3 MB] 2022-06-17T12:30:48.215 INFO:teuthology.orchestra.run.smithi036.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-120-generic amd64 5.4.0-120.136 [10.5 MB] 2022-06-17T12:30:48.326 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package linux-modules-5.4.0-120-generic. 2022-06-17T12:30:48.482 INFO:teuthology.orchestra.run.smithi036.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-120-generic amd64 5.4.0-120.136 [39.3 MB] 2022-06-17T12:30:49.151 INFO:teuthology.orchestra.run.smithi136.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.120.121 [1,900 B] 2022-06-17T12:30:49.152 INFO:teuthology.orchestra.run.smithi136.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.120.121 [2,548 B] 2022-06-17T12:30:49.152 INFO:teuthology.orchestra.run.smithi136.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-120 all 5.4.0-120.136 [11.0 MB] 2022-06-17T12:30:49.321 INFO:teuthology.orchestra.run.smithi107.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.120.121 [1,900 B] 2022-06-17T12:30:49.321 INFO:teuthology.orchestra.run.smithi107.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.120.121 [2,548 B] 2022-06-17T12:30:49.322 INFO:teuthology.orchestra.run.smithi107.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-120 all 5.4.0-120.136 [11.0 MB] 2022-06-17T12:30:49.433 INFO:teuthology.orchestra.run.smithi136.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-120-generic amd64 5.4.0-120.136 [1,393 kB] 2022-06-17T12:30:49.460 INFO:teuthology.orchestra.run.smithi136.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.120.121 [2,416 B] 2022-06-17T12:30:49.494 INFO:teuthology.orchestra.run.smithi070.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.120.121 [1,900 B] 2022-06-17T12:30:49.495 INFO:teuthology.orchestra.run.smithi070.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.120.121 [2,548 B] 2022-06-17T12:30:49.495 INFO:teuthology.orchestra.run.smithi070.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-120 all 5.4.0-120.136 [11.0 MB] 2022-06-17T12:30:49.657 INFO:teuthology.orchestra.run.smithi107.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-120-generic amd64 5.4.0-120.136 [1,393 kB] 2022-06-17T12:30:49.676 INFO:teuthology.orchestra.run.smithi143.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102337 files and directories currently installed.) 2022-06-17T12:30:49.680 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../0-linux-modules-5.4.0-120-generic_5.4.0-120.136_amd64.deb ... 2022-06-17T12:30:49.696 INFO:teuthology.orchestra.run.smithi107.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.120.121 [2,416 B] 2022-06-17T12:30:49.733 INFO:teuthology.orchestra.run.smithi036.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.120.121 [1,900 B] 2022-06-17T12:30:49.734 INFO:teuthology.orchestra.run.smithi036.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.120.121 [2,548 B] 2022-06-17T12:30:49.735 INFO:teuthology.orchestra.run.smithi036.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-120 all 5.4.0-120.136 [11.0 MB] 2022-06-17T12:30:49.757 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking linux-modules-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:30:49.854 INFO:teuthology.orchestra.run.smithi070.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-120-generic amd64 5.4.0-120.136 [1,393 kB] 2022-06-17T12:30:49.874 INFO:teuthology.orchestra.run.smithi070.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.120.121 [2,416 B] 2022-06-17T12:30:49.911 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 77.1 MB in 3s (24.8 MB/s) 2022-06-17T12:30:50.021 INFO:teuthology.orchestra.run.smithi036.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-120-generic amd64 5.4.0-120.136 [1,393 kB] 2022-06-17T12:30:50.056 INFO:teuthology.orchestra.run.smithi036.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.120.121 [2,416 B] 2022-06-17T12:30:50.102 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-modules-5.4.0-120-generic. 2022-06-17T12:30:50.135 INFO:teuthology.orchestra.run.smithi107.stdout:Fetched 77.1 MB in 3s (24.3 MB/s) 2022-06-17T12:30:50.313 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package linux-modules-5.4.0-120-generic. 2022-06-17T12:30:50.314 INFO:teuthology.orchestra.run.smithi070.stdout:Fetched 77.1 MB in 3s (23.7 MB/s) 2022-06-17T12:30:50.489 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package linux-modules-5.4.0-120-generic. 2022-06-17T12:30:50.555 INFO:teuthology.orchestra.run.smithi036.stdout:Fetched 77.1 MB in 3s (24.1 MB/s) 2022-06-17T12:30:50.715 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package linux-modules-5.4.0-120-generic. 2022-06-17T12:30:51.459 INFO:teuthology.orchestra.run.smithi136.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102337 files and directories currently installed.) 2022-06-17T12:30:51.464 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../0-linux-modules-5.4.0-120-generic_5.4.0-120.136_amd64.deb ... 2022-06-17T12:30:51.548 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-modules-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:30:51.584 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package linux-image-5.4.0-120-generic. 2022-06-17T12:30:51.599 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../1-linux-image-5.4.0-120-generic_5.4.0-120.136_amd64.deb ... 2022-06-17T12:30:51.640 INFO:teuthology.orchestra.run.smithi107.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102337 files and directories currently installed.) 2022-06-17T12:30:51.649 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../0-linux-modules-5.4.0-120-generic_5.4.0-120.136_amd64.deb ... 2022-06-17T12:30:51.700 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking linux-image-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:30:51.718 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking linux-modules-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:30:51.826 INFO:teuthology.orchestra.run.smithi070.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102337 files and directories currently installed.) 2022-06-17T12:30:51.829 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../0-linux-modules-5.4.0-120-generic_5.4.0-120.136_amd64.deb ... 2022-06-17T12:30:51.976 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking linux-modules-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:30:52.036 INFO:teuthology.orchestra.run.smithi036.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102337 files and directories currently installed.) 2022-06-17T12:30:52.040 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../0-linux-modules-5.4.0-120-generic_5.4.0-120.136_amd64.deb ... 2022-06-17T12:30:52.136 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking linux-modules-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:30:52.774 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-120-generic. 2022-06-17T12:30:52.782 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-120-generic_5.4.0-120.136_amd64.deb ... 2022-06-17T12:30:52.820 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking linux-modules-extra-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:30:53.419 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-image-5.4.0-120-generic. 2022-06-17T12:30:53.428 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../1-linux-image-5.4.0-120-generic_5.4.0-120.136_amd64.deb ... 2022-06-17T12:30:53.520 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-image-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:30:53.533 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package linux-image-5.4.0-120-generic. 2022-06-17T12:30:53.549 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../1-linux-image-5.4.0-120-generic_5.4.0-120.136_amd64.deb ... 2022-06-17T12:30:53.644 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking linux-image-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:30:53.820 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package linux-image-5.4.0-120-generic. 2022-06-17T12:30:53.834 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../1-linux-image-5.4.0-120-generic_5.4.0-120.136_amd64.deb ... 2022-06-17T12:30:53.957 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking linux-image-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:30:54.000 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package linux-image-5.4.0-120-generic. 2022-06-17T12:30:54.012 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../1-linux-image-5.4.0-120-generic_5.4.0-120.136_amd64.deb ... 2022-06-17T12:30:54.103 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking linux-image-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:30:54.617 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-120-generic. 2022-06-17T12:30:54.631 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-120-generic_5.4.0-120.136_amd64.deb ... 2022-06-17T12:30:54.696 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-modules-extra-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:30:54.702 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-120-generic. 2022-06-17T12:30:54.718 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-120-generic_5.4.0-120.136_amd64.deb ... 2022-06-17T12:30:54.761 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking linux-modules-extra-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:30:55.079 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-120-generic. 2022-06-17T12:30:55.094 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-120-generic_5.4.0-120.136_amd64.deb ... 2022-06-17T12:30:55.137 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking linux-modules-extra-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:30:55.312 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-120-generic. 2022-06-17T12:30:55.328 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-120-generic_5.4.0-120.136_amd64.deb ... 2022-06-17T12:30:55.362 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking linux-modules-extra-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:30:57.668 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../3-linux-generic_5.4.0.120.121_amd64.deb ... 2022-06-17T12:30:57.795 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking linux-generic (5.4.0.120.121) over (5.4.0.117.120) ... 2022-06-17T12:30:58.123 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.120.121_amd64.deb ... 2022-06-17T12:30:58.249 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking linux-image-generic (5.4.0.120.121) over (5.4.0.117.120) ... 2022-06-17T12:30:58.552 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package linux-headers-5.4.0-120. 2022-06-17T12:30:58.568 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../5-linux-headers-5.4.0-120_5.4.0-120.136_all.deb ... 2022-06-17T12:30:58.617 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking linux-headers-5.4.0-120 (5.4.0-120.136) ... 2022-06-17T12:30:59.508 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../3-linux-generic_5.4.0.120.121_amd64.deb ... 2022-06-17T12:30:59.592 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../3-linux-generic_5.4.0.120.121_amd64.deb ... 2022-06-17T12:30:59.644 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-generic (5.4.0.120.121) over (5.4.0.117.120) ... 2022-06-17T12:30:59.719 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking linux-generic (5.4.0.120.121) over (5.4.0.117.120) ... 2022-06-17T12:30:59.789 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../3-linux-generic_5.4.0.120.121_amd64.deb ... 2022-06-17T12:30:59.899 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking linux-generic (5.4.0.120.121) over (5.4.0.117.120) ... 2022-06-17T12:30:59.942 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.120.121_amd64.deb ... 2022-06-17T12:31:00.046 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.120.121_amd64.deb ... 2022-06-17T12:31:00.081 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-image-generic (5.4.0.120.121) over (5.4.0.117.120) ... 2022-06-17T12:31:00.084 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../3-linux-generic_5.4.0.120.121_amd64.deb ... 2022-06-17T12:31:00.173 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking linux-image-generic (5.4.0.120.121) over (5.4.0.117.120) ... 2022-06-17T12:31:00.207 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking linux-generic (5.4.0.120.121) over (5.4.0.117.120) ... 2022-06-17T12:31:00.284 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.120.121_amd64.deb ... 2022-06-17T12:31:00.391 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-headers-5.4.0-120. 2022-06-17T12:31:00.408 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../5-linux-headers-5.4.0-120_5.4.0-120.136_all.deb ... 2022-06-17T12:31:00.415 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking linux-image-generic (5.4.0.120.121) over (5.4.0.117.120) ... 2022-06-17T12:31:00.467 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-headers-5.4.0-120 (5.4.0-120.136) ... 2022-06-17T12:31:00.517 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package linux-headers-5.4.0-120. 2022-06-17T12:31:00.531 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../5-linux-headers-5.4.0-120_5.4.0-120.136_all.deb ... 2022-06-17T12:31:00.552 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.120.121_amd64.deb ... 2022-06-17T12:31:00.579 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking linux-headers-5.4.0-120 (5.4.0-120.136) ... 2022-06-17T12:31:00.660 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking linux-image-generic (5.4.0.120.121) over (5.4.0.117.120) ... 2022-06-17T12:31:00.771 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package linux-headers-5.4.0-120. 2022-06-17T12:31:00.788 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../5-linux-headers-5.4.0-120_5.4.0-120.136_all.deb ... 2022-06-17T12:31:00.847 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking linux-headers-5.4.0-120 (5.4.0-120.136) ... 2022-06-17T12:31:00.992 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package linux-headers-5.4.0-120. 2022-06-17T12:31:01.004 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../5-linux-headers-5.4.0-120_5.4.0-120.136_all.deb ... 2022-06-17T12:31:01.046 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking linux-headers-5.4.0-120 (5.4.0-120.136) ... 2022-06-17T12:31:03.833 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package linux-headers-5.4.0-120-generic. 2022-06-17T12:31:03.857 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../6-linux-headers-5.4.0-120-generic_5.4.0-120.136_amd64.deb ... 2022-06-17T12:31:03.903 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking linux-headers-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:05.914 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-headers-5.4.0-120-generic. 2022-06-17T12:31:05.935 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../6-linux-headers-5.4.0-120-generic_5.4.0-120.136_amd64.deb ... 2022-06-17T12:31:05.972 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-headers-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:06.029 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package linux-headers-5.4.0-120-generic. 2022-06-17T12:31:06.051 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../6-linux-headers-5.4.0-120-generic_5.4.0-120.136_amd64.deb ... 2022-06-17T12:31:06.097 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking linux-headers-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:06.282 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package linux-headers-5.4.0-120-generic. 2022-06-17T12:31:06.312 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../6-linux-headers-5.4.0-120-generic_5.4.0-120.136_amd64.deb ... 2022-06-17T12:31:06.350 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking linux-headers-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:06.572 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package linux-headers-5.4.0-120-generic. 2022-06-17T12:31:06.594 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../6-linux-headers-5.4.0-120-generic_5.4.0-120.136_amd64.deb ... 2022-06-17T12:31:06.616 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.120.121_amd64.deb ... 2022-06-17T12:31:06.630 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking linux-headers-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:06.718 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking linux-headers-generic (5.4.0.120.121) over (5.4.0.117.120) ... 2022-06-17T12:31:07.003 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up linux-modules-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:08.205 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up linux-headers-5.4.0-120 (5.4.0-120.136) ... 2022-06-17T12:31:08.324 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up linux-image-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:08.578 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.120.121_amd64.deb ... 2022-06-17T12:31:08.604 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.120.121_amd64.deb ... 2022-06-17T12:31:08.702 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-headers-generic (5.4.0.120.121) over (5.4.0.117.120) ... 2022-06-17T12:31:08.717 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking linux-headers-generic (5.4.0.120.121) over (5.4.0.117.120) ... 2022-06-17T12:31:09.034 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.120.121_amd64.deb ... 2022-06-17T12:31:09.055 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-modules-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:09.060 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up linux-modules-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:09.187 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking linux-headers-generic (5.4.0.120.121) over (5.4.0.117.120) ... 2022-06-17T12:31:09.449 INFO:teuthology.orchestra.run.smithi143.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-120-generic 2022-06-17T12:31:09.449 INFO:teuthology.orchestra.run.smithi143.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-120-generic 2022-06-17T12:31:09.527 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up linux-modules-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:09.584 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up linux-headers-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:09.710 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up linux-headers-generic (5.4.0.120.121) ... 2022-06-17T12:31:09.839 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up linux-modules-extra-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:10.264 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.120.121_amd64.deb ... 2022-06-17T12:31:10.271 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-headers-5.4.0-120 (5.4.0-120.136) ... 2022-06-17T12:31:10.273 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up linux-headers-5.4.0-120 (5.4.0-120.136) ... 2022-06-17T12:31:10.359 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking linux-headers-generic (5.4.0.120.121) over (5.4.0.117.120) ... 2022-06-17T12:31:10.416 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up linux-image-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:10.422 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-image-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:10.702 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up linux-modules-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:10.734 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up linux-headers-5.4.0-120 (5.4.0-120.136) ... 2022-06-17T12:31:10.869 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up linux-image-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:11.446 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up linux-image-generic (5.4.0.120.121) ... 2022-06-17T12:31:11.565 INFO:teuthology.orchestra.run.smithi107.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-120-generic 2022-06-17T12:31:11.566 INFO:teuthology.orchestra.run.smithi107.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-120-generic 2022-06-17T12:31:11.567 INFO:teuthology.orchestra.run.smithi136.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-120-generic 2022-06-17T12:31:11.567 INFO:teuthology.orchestra.run.smithi136.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-120-generic 2022-06-17T12:31:11.572 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up linux-generic (5.4.0.120.121) ... 2022-06-17T12:31:11.673 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for linux-image-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:11.703 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up linux-headers-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:11.706 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-headers-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:11.721 INFO:teuthology.orchestra.run.smithi143.stdout:/etc/kernel/postinst.d/initramfs-tools: 2022-06-17T12:31:11.722 INFO:teuthology.orchestra.run.smithi143.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-120-generic 2022-06-17T12:31:11.787 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-headers-generic (5.4.0.120.121) ... 2022-06-17T12:31:11.844 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up linux-headers-generic (5.4.0.120.121) ... 2022-06-17T12:31:11.905 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-modules-extra-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:11.958 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up linux-headers-5.4.0-120 (5.4.0-120.136) ... 2022-06-17T12:31:11.962 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up linux-modules-extra-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:12.192 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up linux-image-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:12.209 INFO:teuthology.orchestra.run.smithi070.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-120-generic 2022-06-17T12:31:12.209 INFO:teuthology.orchestra.run.smithi070.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-120-generic 2022-06-17T12:31:12.538 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up linux-headers-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:12.672 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up linux-headers-generic (5.4.0.120.121) ... 2022-06-17T12:31:12.824 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up linux-modules-extra-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:13.122 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up linux-image-generic (5.4.0.120.121) ... 2022-06-17T12:31:13.273 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up linux-generic (5.4.0.120.121) ... 2022-06-17T12:31:13.358 INFO:teuthology.orchestra.run.smithi036.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-120-generic 2022-06-17T12:31:13.359 INFO:teuthology.orchestra.run.smithi036.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-120-generic 2022-06-17T12:31:13.400 INFO:teuthology.orchestra.run.smithi107.stdout:Processing triggers for linux-image-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:13.460 INFO:teuthology.orchestra.run.smithi107.stdout:/etc/kernel/postinst.d/initramfs-tools: 2022-06-17T12:31:13.461 INFO:teuthology.orchestra.run.smithi107.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-120-generic 2022-06-17T12:31:13.494 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up linux-headers-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:13.645 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up linux-headers-generic (5.4.0.120.121) ... 2022-06-17T12:31:13.737 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-image-generic (5.4.0.120.121) ... 2022-06-17T12:31:13.771 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up linux-modules-extra-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:13.991 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up linux-image-generic (5.4.0.120.121) ... 2022-06-17T12:31:14.092 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up linux-generic (5.4.0.120.121) ... 2022-06-17T12:31:14.223 INFO:teuthology.orchestra.run.smithi070.stdout:Processing triggers for linux-image-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:14.271 INFO:teuthology.orchestra.run.smithi070.stdout:/etc/kernel/postinst.d/initramfs-tools: 2022-06-17T12:31:14.272 INFO:teuthology.orchestra.run.smithi070.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-120-generic 2022-06-17T12:31:14.617 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-generic (5.4.0.120.121) ... 2022-06-17T12:31:14.718 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for linux-image-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:14.779 INFO:teuthology.orchestra.run.smithi136.stdout:/etc/kernel/postinst.d/initramfs-tools: 2022-06-17T12:31:14.779 INFO:teuthology.orchestra.run.smithi136.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-120-generic 2022-06-17T12:31:15.340 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up linux-image-generic (5.4.0.120.121) ... 2022-06-17T12:31:15.466 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up linux-generic (5.4.0.120.121) ... 2022-06-17T12:31:15.601 INFO:teuthology.orchestra.run.smithi036.stdout:Processing triggers for linux-image-5.4.0-120-generic (5.4.0-120.136) ... 2022-06-17T12:31:15.677 INFO:teuthology.orchestra.run.smithi036.stdout:/etc/kernel/postinst.d/initramfs-tools: 2022-06-17T12:31:15.678 INFO:teuthology.orchestra.run.smithi036.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-120-generic 2022-06-17T12:31:33.421 INFO:teuthology.orchestra.run.smithi143.stdout:/etc/kernel/postinst.d/zz-update-grub: 2022-06-17T12:31:33.421 INFO:teuthology.orchestra.run.smithi143.stdout:Sourcing file `/etc/default/grub' 2022-06-17T12:31:33.422 INFO:teuthology.orchestra.run.smithi143.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-06-17T12:31:33.435 INFO:teuthology.orchestra.run.smithi143.stdout:Generating grub configuration file ... 2022-06-17T12:31:33.735 INFO:teuthology.orchestra.run.smithi143.stdout:Found linux image: /boot/vmlinuz-5.4.0-120-generic 2022-06-17T12:31:33.765 INFO:teuthology.orchestra.run.smithi143.stdout:Found initrd image: /boot/initrd.img-5.4.0-120-generic 2022-06-17T12:31:34.169 INFO:teuthology.orchestra.run.smithi143.stdout:Found linux image: /boot/vmlinuz-5.4.0-117-generic 2022-06-17T12:31:34.180 INFO:teuthology.orchestra.run.smithi143.stdout:Found initrd image: /boot/initrd.img-5.4.0-117-generic 2022-06-17T12:31:36.084 INFO:teuthology.orchestra.run.smithi070.stdout:/etc/kernel/postinst.d/zz-update-grub: 2022-06-17T12:31:36.084 INFO:teuthology.orchestra.run.smithi070.stdout:Sourcing file `/etc/default/grub' 2022-06-17T12:31:36.095 INFO:teuthology.orchestra.run.smithi070.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-06-17T12:31:36.106 INFO:teuthology.orchestra.run.smithi070.stdout:Generating grub configuration file ... 2022-06-17T12:31:36.394 INFO:teuthology.orchestra.run.smithi107.stdout:/etc/kernel/postinst.d/zz-update-grub: 2022-06-17T12:31:36.394 INFO:teuthology.orchestra.run.smithi107.stdout:Sourcing file `/etc/default/grub' 2022-06-17T12:31:36.405 INFO:teuthology.orchestra.run.smithi107.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-06-17T12:31:36.416 INFO:teuthology.orchestra.run.smithi107.stdout:Generating grub configuration file ... 2022-06-17T12:31:36.418 INFO:teuthology.orchestra.run.smithi070.stdout:Found linux image: /boot/vmlinuz-5.4.0-120-generic 2022-06-17T12:31:36.460 INFO:teuthology.orchestra.run.smithi070.stdout:Found initrd image: /boot/initrd.img-5.4.0-120-generic 2022-06-17T12:31:36.562 INFO:teuthology.orchestra.run.smithi136.stdout:/etc/kernel/postinst.d/zz-update-grub: 2022-06-17T12:31:36.562 INFO:teuthology.orchestra.run.smithi136.stdout:Sourcing file `/etc/default/grub' 2022-06-17T12:31:36.573 INFO:teuthology.orchestra.run.smithi136.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-06-17T12:31:36.577 INFO:teuthology.orchestra.run.smithi136.stdout:Generating grub configuration file ... 2022-06-17T12:31:36.704 INFO:teuthology.orchestra.run.smithi107.stdout:Found linux image: /boot/vmlinuz-5.4.0-120-generic 2022-06-17T12:31:36.746 INFO:teuthology.orchestra.run.smithi107.stdout:Found initrd image: /boot/initrd.img-5.4.0-120-generic 2022-06-17T12:31:36.857 INFO:teuthology.orchestra.run.smithi143.stdout:done 2022-06-17T12:31:36.864 INFO:teuthology.orchestra.run.smithi070.stdout:Found linux image: /boot/vmlinuz-5.4.0-117-generic 2022-06-17T12:31:36.876 INFO:teuthology.orchestra.run.smithi070.stdout:Found initrd image: /boot/initrd.img-5.4.0-117-generic 2022-06-17T12:31:36.886 INFO:teuthology.orchestra.run.smithi136.stdout:Found linux image: /boot/vmlinuz-5.4.0-120-generic 2022-06-17T12:31:36.913 INFO:teuthology.orchestra.run.smithi136.stdout:Found initrd image: /boot/initrd.img-5.4.0-120-generic 2022-06-17T12:31:37.142 INFO:teuthology.orchestra.run.smithi107.stdout:Found linux image: /boot/vmlinuz-5.4.0-117-generic 2022-06-17T12:31:37.154 INFO:teuthology.orchestra.run.smithi107.stdout:Found initrd image: /boot/initrd.img-5.4.0-117-generic 2022-06-17T12:31:37.246 DEBUG:teuthology.orchestra.run.smithi143:> dpkg -s linux-image-generic 2022-06-17T12:31:37.278 INFO:teuthology.orchestra.run.smithi143.stdout:Package: linux-image-generic 2022-06-17T12:31:37.279 INFO:teuthology.orchestra.run.smithi143.stdout:Status: install ok installed 2022-06-17T12:31:37.279 INFO:teuthology.orchestra.run.smithi143.stdout:Priority: optional 2022-06-17T12:31:37.279 INFO:teuthology.orchestra.run.smithi143.stdout:Section: kernel 2022-06-17T12:31:37.280 INFO:teuthology.orchestra.run.smithi143.stdout:Installed-Size: 19 2022-06-17T12:31:37.280 INFO:teuthology.orchestra.run.smithi143.stdout:Maintainer: Ubuntu Kernel Team 2022-06-17T12:31:37.280 INFO:teuthology.orchestra.run.smithi143.stdout:Architecture: amd64 2022-06-17T12:31:37.280 INFO:teuthology.orchestra.run.smithi143.stdout:Source: linux-meta 2022-06-17T12:31:37.280 INFO:teuthology.orchestra.run.smithi143.stdout:Version: 5.4.0.120.121 2022-06-17T12:31:37.281 INFO:teuthology.orchestra.run.smithi143.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.13) 2022-06-17T12:31:37.281 INFO:teuthology.orchestra.run.smithi143.stdout:Depends: linux-image-5.4.0-120-generic, linux-modules-extra-5.4.0-120-generic, linux-firmware, intel-microcode, amd64-microcode 2022-06-17T12:31:37.281 INFO:teuthology.orchestra.run.smithi143.stdout:Recommends: thermald 2022-06-17T12:31:37.281 INFO:teuthology.orchestra.run.smithi143.stdout:Description: Generic Linux kernel image 2022-06-17T12:31:37.282 INFO:teuthology.orchestra.run.smithi143.stdout: This package will always depend on the latest generic kernel image 2022-06-17T12:31:37.282 INFO:teuthology.orchestra.run.smithi143.stdout: available. 2022-06-17T12:31:37.282 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-117-generic Expected: 5.4.0-120-generic 2022-06-17T12:31:37.283 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2022-06-17T12:31:37.283 DEBUG:teuthology.task.kernel:src is distro, skipping download 2022-06-17T12:31:37.283 INFO:teuthology.task.kernel:Installing distro kernel on client.2... 2022-06-17T12:31:37.283 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi143.front.sepia.ceph.com, path=None, version=distro) 2022-06-17T12:31:37.284 DEBUG:teuthology.orchestra.run.smithi143:> sudo apt-get clean 2022-06-17T12:31:37.330 INFO:teuthology.orchestra.run.smithi136.stdout:Found linux image: /boot/vmlinuz-5.4.0-117-generic 2022-06-17T12:31:37.340 INFO:teuthology.orchestra.run.smithi136.stdout:Found initrd image: /boot/initrd.img-5.4.0-117-generic 2022-06-17T12:31:37.371 DEBUG:teuthology.orchestra.run.smithi143:> sudo apt-get update 2022-06-17T12:31:37.634 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-17T12:31:37.702 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-06-17T12:31:37.784 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-06-17T12:31:38.259 INFO:teuthology.orchestra.run.smithi036.stdout:/etc/kernel/postinst.d/zz-update-grub: 2022-06-17T12:31:38.260 INFO:teuthology.orchestra.run.smithi036.stdout:Sourcing file `/etc/default/grub' 2022-06-17T12:31:38.270 INFO:teuthology.orchestra.run.smithi036.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-06-17T12:31:38.283 INFO:teuthology.orchestra.run.smithi036.stdout:Generating grub configuration file ... 2022-06-17T12:31:38.417 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-06-17T12:31:38.584 INFO:teuthology.orchestra.run.smithi036.stdout:Found linux image: /boot/vmlinuz-5.4.0-120-generic 2022-06-17T12:31:38.634 INFO:teuthology.orchestra.run.smithi036.stdout:Found initrd image: /boot/initrd.img-5.4.0-120-generic 2022-06-17T12:31:38.998 INFO:teuthology.orchestra.run.smithi036.stdout:Found linux image: /boot/vmlinuz-5.4.0-117-generic 2022-06-17T12:31:39.009 INFO:teuthology.orchestra.run.smithi036.stdout:Found initrd image: /boot/initrd.img-5.4.0-117-generic 2022-06-17T12:31:39.536 INFO:teuthology.orchestra.run.smithi070.stdout:done 2022-06-17T12:31:39.575 INFO:teuthology.orchestra.run.smithi143.stdout:Reading package lists... 2022-06-17T12:31:39.599 DEBUG:teuthology.orchestra.run.smithi143:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-06-17T12:31:39.667 INFO:teuthology.orchestra.run.smithi143.stdout:Reading package lists... 2022-06-17T12:31:39.845 INFO:teuthology.orchestra.run.smithi143.stdout:Building dependency tree... 2022-06-17T12:31:39.846 INFO:teuthology.orchestra.run.smithi143.stdout:Reading state information... 2022-06-17T12:31:39.855 INFO:teuthology.orchestra.run.smithi107.stdout:done 2022-06-17T12:31:39.925 DEBUG:teuthology.orchestra.run.smithi070:> dpkg -s linux-image-generic 2022-06-17T12:31:39.952 INFO:teuthology.orchestra.run.smithi070.stdout:Package: linux-image-generic 2022-06-17T12:31:39.953 INFO:teuthology.orchestra.run.smithi070.stdout:Status: install ok installed 2022-06-17T12:31:39.953 INFO:teuthology.orchestra.run.smithi070.stdout:Priority: optional 2022-06-17T12:31:39.953 INFO:teuthology.orchestra.run.smithi070.stdout:Section: kernel 2022-06-17T12:31:39.953 INFO:teuthology.orchestra.run.smithi070.stdout:Installed-Size: 19 2022-06-17T12:31:39.954 INFO:teuthology.orchestra.run.smithi070.stdout:Maintainer: Ubuntu Kernel Team 2022-06-17T12:31:39.955 INFO:teuthology.orchestra.run.smithi070.stdout:Architecture: amd64 2022-06-17T12:31:39.955 INFO:teuthology.orchestra.run.smithi070.stdout:Source: linux-meta 2022-06-17T12:31:39.955 INFO:teuthology.orchestra.run.smithi070.stdout:Version: 5.4.0.120.121 2022-06-17T12:31:39.955 INFO:teuthology.orchestra.run.smithi070.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.13) 2022-06-17T12:31:39.956 INFO:teuthology.orchestra.run.smithi070.stdout:Depends: linux-image-5.4.0-120-generic, linux-modules-extra-5.4.0-120-generic, linux-firmware, intel-microcode, amd64-microcode 2022-06-17T12:31:39.956 INFO:teuthology.orchestra.run.smithi070.stdout:Recommends: thermald 2022-06-17T12:31:39.956 INFO:teuthology.orchestra.run.smithi070.stdout:Description: Generic Linux kernel image 2022-06-17T12:31:39.956 INFO:teuthology.orchestra.run.smithi070.stdout: This package will always depend on the latest generic kernel image 2022-06-17T12:31:39.957 INFO:teuthology.orchestra.run.smithi070.stdout: available. 2022-06-17T12:31:39.957 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-117-generic Expected: 5.4.0-120-generic 2022-06-17T12:31:39.957 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2022-06-17T12:31:39.958 DEBUG:teuthology.task.kernel:src is distro, skipping download 2022-06-17T12:31:39.959 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2022-06-17T12:31:39.959 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi070.front.sepia.ceph.com, path=None, version=distro) 2022-06-17T12:31:39.959 DEBUG:teuthology.orchestra.run.smithi070:> sudo apt-get clean 2022-06-17T12:31:40.012 DEBUG:teuthology.orchestra.run.smithi070:> sudo apt-get update 2022-06-17T12:31:40.028 INFO:teuthology.orchestra.run.smithi136.stdout:done 2022-06-17T12:31:40.035 INFO:teuthology.orchestra.run.smithi143.stdout:linux-image-generic is already the newest version (5.4.0.120.121). 2022-06-17T12:31:40.035 INFO:teuthology.orchestra.run.smithi143.stdout:0 upgraded, 0 newly installed, 0 to remove and 5 not upgraded. 2022-06-17T12:31:40.037 DEBUG:teuthology.orchestra.run.smithi143:> dpkg -s linux-image-generic 2022-06-17T12:31:40.064 INFO:teuthology.orchestra.run.smithi143.stdout:Package: linux-image-generic 2022-06-17T12:31:40.064 INFO:teuthology.orchestra.run.smithi143.stdout:Status: install ok installed 2022-06-17T12:31:40.065 INFO:teuthology.orchestra.run.smithi143.stdout:Priority: optional 2022-06-17T12:31:40.065 INFO:teuthology.orchestra.run.smithi143.stdout:Section: kernel 2022-06-17T12:31:40.065 INFO:teuthology.orchestra.run.smithi143.stdout:Installed-Size: 19 2022-06-17T12:31:40.065 INFO:teuthology.orchestra.run.smithi143.stdout:Maintainer: Ubuntu Kernel Team 2022-06-17T12:31:40.066 INFO:teuthology.orchestra.run.smithi143.stdout:Architecture: amd64 2022-06-17T12:31:40.066 INFO:teuthology.orchestra.run.smithi143.stdout:Source: linux-meta 2022-06-17T12:31:40.066 INFO:teuthology.orchestra.run.smithi143.stdout:Version: 5.4.0.120.121 2022-06-17T12:31:40.066 INFO:teuthology.orchestra.run.smithi143.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.13) 2022-06-17T12:31:40.067 INFO:teuthology.orchestra.run.smithi143.stdout:Depends: linux-image-5.4.0-120-generic, linux-modules-extra-5.4.0-120-generic, linux-firmware, intel-microcode, amd64-microcode 2022-06-17T12:31:40.067 INFO:teuthology.orchestra.run.smithi143.stdout:Recommends: thermald 2022-06-17T12:31:40.067 INFO:teuthology.orchestra.run.smithi143.stdout:Description: Generic Linux kernel image 2022-06-17T12:31:40.067 INFO:teuthology.orchestra.run.smithi143.stdout: This package will always depend on the latest generic kernel image 2022-06-17T12:31:40.067 INFO:teuthology.orchestra.run.smithi143.stdout: available. 2022-06-17T12:31:40.068 DEBUG:teuthology.orchestra.run.smithi143:> mktemp 2022-06-17T12:31:40.114 INFO:teuthology.orchestra.run.smithi143.stdout:/tmp/tmp.JAuQxxYBGy 2022-06-17T12:31:40.116 DEBUG:teuthology.orchestra.run.smithi143:> sudo cp /boot/grub/grub.cfg /tmp/tmp.JAuQxxYBGy 2022-06-17T12:31:40.133 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0666 /tmp/tmp.JAuQxxYBGy 2022-06-17T12:31:40.188 INFO:teuthology.orchestra.run.smithi070.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-06-17T12:31:40.217 DEBUG:teuthology.orchestra.remote:smithi143:/tmp/tmp.JAuQxxYBGy is 9KB 2022-06-17T12:31:40.252 DEBUG:teuthology.orchestra.run.smithi107:> dpkg -s linux-image-generic 2022-06-17T12:31:40.273 INFO:teuthology.orchestra.run.smithi070.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-17T12:31:40.276 DEBUG:teuthology.orchestra.run.smithi143:> rm -fr /tmp/tmp.JAuQxxYBGy 2022-06-17T12:31:40.281 INFO:teuthology.orchestra.run.smithi107.stdout:Package: linux-image-generic 2022-06-17T12:31:40.282 INFO:teuthology.orchestra.run.smithi107.stdout:Status: install ok installed 2022-06-17T12:31:40.282 INFO:teuthology.orchestra.run.smithi107.stdout:Priority: optional 2022-06-17T12:31:40.282 INFO:teuthology.orchestra.run.smithi107.stdout:Section: kernel 2022-06-17T12:31:40.282 INFO:teuthology.orchestra.run.smithi107.stdout:Installed-Size: 19 2022-06-17T12:31:40.283 INFO:teuthology.orchestra.run.smithi107.stdout:Maintainer: Ubuntu Kernel Team 2022-06-17T12:31:40.283 INFO:teuthology.orchestra.run.smithi107.stdout:Architecture: amd64 2022-06-17T12:31:40.283 INFO:teuthology.orchestra.run.smithi107.stdout:Source: linux-meta 2022-06-17T12:31:40.283 INFO:teuthology.orchestra.run.smithi107.stdout:Version: 5.4.0.120.121 2022-06-17T12:31:40.284 INFO:teuthology.orchestra.run.smithi107.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.13) 2022-06-17T12:31:40.284 INFO:teuthology.orchestra.run.smithi107.stdout:Depends: linux-image-5.4.0-120-generic, linux-modules-extra-5.4.0-120-generic, linux-firmware, intel-microcode, amd64-microcode 2022-06-17T12:31:40.284 INFO:teuthology.orchestra.run.smithi107.stdout:Recommends: thermald 2022-06-17T12:31:40.285 INFO:teuthology.orchestra.run.smithi107.stdout:Description: Generic Linux kernel image 2022-06-17T12:31:40.285 INFO:teuthology.orchestra.run.smithi107.stdout: This package will always depend on the latest generic kernel image 2022-06-17T12:31:40.285 INFO:teuthology.orchestra.run.smithi107.stdout: available. 2022-06-17T12:31:40.286 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-117-generic Expected: 5.4.0-120-generic 2022-06-17T12:31:40.287 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2022-06-17T12:31:40.287 DEBUG:teuthology.task.kernel:src is distro, skipping download 2022-06-17T12:31:40.287 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2022-06-17T12:31:40.288 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi107.front.sepia.ceph.com, path=None, version=distro) 2022-06-17T12:31:40.289 DEBUG:teuthology.orchestra.run.smithi107:> sudo apt-get clean 2022-06-17T12:31:40.292 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2022-06-17T12:31:40.351 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2022-06-17T12:31:40.352 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/grub.d/01_ceph_kernel 2022-06-17T12:31:40.353 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2022-06-17T12:31:40.355 INFO:teuthology.orchestra.run.smithi070.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-06-17T12:31:40.371 DEBUG:teuthology.orchestra.run.smithi107:> sudo apt-get update 2022-06-17T12:31:40.414 DEBUG:teuthology.orchestra.run.smithi136:> dpkg -s linux-image-generic 2022-06-17T12:31:40.418 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-120-generic 2022-06-17T12:31:40.418 DEBUG:teuthology.orchestra.run.smithi143:> sudo update-grub 2022-06-17T12:31:40.432 INFO:teuthology.orchestra.run.smithi070.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-06-17T12:31:40.446 INFO:teuthology.orchestra.run.smithi136.stdout:Package: linux-image-generic 2022-06-17T12:31:40.447 INFO:teuthology.orchestra.run.smithi136.stdout:Status: install ok installed 2022-06-17T12:31:40.447 INFO:teuthology.orchestra.run.smithi136.stdout:Priority: optional 2022-06-17T12:31:40.447 INFO:teuthology.orchestra.run.smithi136.stdout:Section: kernel 2022-06-17T12:31:40.447 INFO:teuthology.orchestra.run.smithi136.stdout:Installed-Size: 19 2022-06-17T12:31:40.448 INFO:teuthology.orchestra.run.smithi136.stdout:Maintainer: Ubuntu Kernel Team 2022-06-17T12:31:40.449 INFO:teuthology.orchestra.run.smithi136.stdout:Architecture: amd64 2022-06-17T12:31:40.449 INFO:teuthology.orchestra.run.smithi136.stdout:Source: linux-meta 2022-06-17T12:31:40.449 INFO:teuthology.orchestra.run.smithi136.stdout:Version: 5.4.0.120.121 2022-06-17T12:31:40.450 INFO:teuthology.orchestra.run.smithi136.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.13) 2022-06-17T12:31:40.450 INFO:teuthology.orchestra.run.smithi136.stdout:Depends: linux-image-5.4.0-120-generic, linux-modules-extra-5.4.0-120-generic, linux-firmware, intel-microcode, amd64-microcode 2022-06-17T12:31:40.451 INFO:teuthology.orchestra.run.smithi136.stdout:Recommends: thermald 2022-06-17T12:31:40.451 INFO:teuthology.orchestra.run.smithi136.stdout:Description: Generic Linux kernel image 2022-06-17T12:31:40.452 INFO:teuthology.orchestra.run.smithi136.stdout: This package will always depend on the latest generic kernel image 2022-06-17T12:31:40.452 INFO:teuthology.orchestra.run.smithi136.stdout: available. 2022-06-17T12:31:40.453 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-117-generic Expected: 5.4.0-120-generic 2022-06-17T12:31:40.453 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2022-06-17T12:31:40.454 DEBUG:teuthology.task.kernel:src is distro, skipping download 2022-06-17T12:31:40.454 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2022-06-17T12:31:40.454 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi136.front.sepia.ceph.com, path=None, version=distro) 2022-06-17T12:31:40.455 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get clean 2022-06-17T12:31:40.503 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get update 2022-06-17T12:31:40.560 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-17T12:31:40.589 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-06-17T12:31:40.628 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-06-17T12:31:40.630 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-06-17T12:31:40.700 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-17T12:31:40.707 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-06-17T12:31:40.726 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-06-17T12:31:40.816 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-06-17T12:31:41.596 INFO:teuthology.orchestra.run.smithi036.stdout:done 2022-06-17T12:31:41.603 INFO:teuthology.orchestra.run.smithi143.stderr:Sourcing file `/etc/default/grub' 2022-06-17T12:31:41.605 INFO:teuthology.orchestra.run.smithi143.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-06-17T12:31:41.610 INFO:teuthology.orchestra.run.smithi143.stderr:Generating grub configuration file ... 2022-06-17T12:31:41.753 INFO:teuthology.orchestra.run.smithi070.stdout:Reading package lists... 2022-06-17T12:31:41.775 DEBUG:teuthology.orchestra.run.smithi070:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-06-17T12:31:41.823 INFO:teuthology.orchestra.run.smithi143.stderr:Found linux image: /boot/vmlinuz-5.4.0-120-generic 2022-06-17T12:31:41.835 INFO:teuthology.orchestra.run.smithi143.stderr:Found initrd image: /boot/initrd.img-5.4.0-120-generic 2022-06-17T12:31:41.844 INFO:teuthology.orchestra.run.smithi070.stdout:Reading package lists... 2022-06-17T12:31:41.979 DEBUG:teuthology.orchestra.run.smithi036:> dpkg -s linux-image-generic 2022-06-17T12:31:42.000 INFO:teuthology.orchestra.run.smithi070.stdout:Building dependency tree... 2022-06-17T12:31:42.001 INFO:teuthology.orchestra.run.smithi070.stdout:Reading state information... 2022-06-17T12:31:42.027 INFO:teuthology.orchestra.run.smithi036.stdout:Package: linux-image-generic 2022-06-17T12:31:42.027 INFO:teuthology.orchestra.run.smithi036.stdout:Status: install ok installed 2022-06-17T12:31:42.028 INFO:teuthology.orchestra.run.smithi036.stdout:Priority: optional 2022-06-17T12:31:42.028 INFO:teuthology.orchestra.run.smithi036.stdout:Section: kernel 2022-06-17T12:31:42.028 INFO:teuthology.orchestra.run.smithi036.stdout:Installed-Size: 19 2022-06-17T12:31:42.028 INFO:teuthology.orchestra.run.smithi036.stdout:Maintainer: Ubuntu Kernel Team 2022-06-17T12:31:42.029 INFO:teuthology.orchestra.run.smithi036.stdout:Architecture: amd64 2022-06-17T12:31:42.029 INFO:teuthology.orchestra.run.smithi036.stdout:Source: linux-meta 2022-06-17T12:31:42.029 INFO:teuthology.orchestra.run.smithi036.stdout:Version: 5.4.0.120.121 2022-06-17T12:31:42.029 INFO:teuthology.orchestra.run.smithi036.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.13) 2022-06-17T12:31:42.030 INFO:teuthology.orchestra.run.smithi036.stdout:Depends: linux-image-5.4.0-120-generic, linux-modules-extra-5.4.0-120-generic, linux-firmware, intel-microcode, amd64-microcode 2022-06-17T12:31:42.030 INFO:teuthology.orchestra.run.smithi036.stdout:Recommends: thermald 2022-06-17T12:31:42.030 INFO:teuthology.orchestra.run.smithi036.stdout:Description: Generic Linux kernel image 2022-06-17T12:31:42.030 INFO:teuthology.orchestra.run.smithi036.stdout: This package will always depend on the latest generic kernel image 2022-06-17T12:31:42.031 INFO:teuthology.orchestra.run.smithi036.stdout: available. 2022-06-17T12:31:42.031 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-117-generic Expected: 5.4.0-120-generic 2022-06-17T12:31:42.031 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2022-06-17T12:31:42.031 DEBUG:teuthology.task.kernel:src is distro, skipping download 2022-06-17T12:31:42.032 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2022-06-17T12:31:42.033 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi036.front.sepia.ceph.com, path=None, version=distro) 2022-06-17T12:31:42.033 DEBUG:teuthology.orchestra.run.smithi036:> sudo apt-get clean 2022-06-17T12:31:42.104 INFO:teuthology.orchestra.run.smithi107.stdout:Reading package lists... 2022-06-17T12:31:42.126 DEBUG:teuthology.orchestra.run.smithi107:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-06-17T12:31:42.129 DEBUG:teuthology.orchestra.run.smithi036:> sudo apt-get update 2022-06-17T12:31:42.174 INFO:teuthology.orchestra.run.smithi070.stdout:linux-image-generic is already the newest version (5.4.0.120.121). 2022-06-17T12:31:42.174 INFO:teuthology.orchestra.run.smithi070.stdout:0 upgraded, 0 newly installed, 0 to remove and 5 not upgraded. 2022-06-17T12:31:42.176 DEBUG:teuthology.orchestra.run.smithi070:> dpkg -s linux-image-generic 2022-06-17T12:31:42.194 INFO:teuthology.orchestra.run.smithi070.stdout:Package: linux-image-generic 2022-06-17T12:31:42.195 INFO:teuthology.orchestra.run.smithi070.stdout:Status: install ok installed 2022-06-17T12:31:42.195 INFO:teuthology.orchestra.run.smithi070.stdout:Priority: optional 2022-06-17T12:31:42.195 INFO:teuthology.orchestra.run.smithi070.stdout:Section: kernel 2022-06-17T12:31:42.196 INFO:teuthology.orchestra.run.smithi070.stdout:Installed-Size: 19 2022-06-17T12:31:42.197 INFO:teuthology.orchestra.run.smithi070.stdout:Maintainer: Ubuntu Kernel Team 2022-06-17T12:31:42.198 INFO:teuthology.orchestra.run.smithi070.stdout:Architecture: amd64 2022-06-17T12:31:42.198 INFO:teuthology.orchestra.run.smithi070.stdout:Source: linux-meta 2022-06-17T12:31:42.198 INFO:teuthology.orchestra.run.smithi070.stdout:Version: 5.4.0.120.121 2022-06-17T12:31:42.199 INFO:teuthology.orchestra.run.smithi070.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.13) 2022-06-17T12:31:42.199 INFO:teuthology.orchestra.run.smithi070.stdout:Depends: linux-image-5.4.0-120-generic, linux-modules-extra-5.4.0-120-generic, linux-firmware, intel-microcode, amd64-microcode 2022-06-17T12:31:42.199 INFO:teuthology.orchestra.run.smithi070.stdout:Recommends: thermald 2022-06-17T12:31:42.199 INFO:teuthology.orchestra.run.smithi070.stdout:Description: Generic Linux kernel image 2022-06-17T12:31:42.199 INFO:teuthology.orchestra.run.smithi070.stdout: This package will always depend on the latest generic kernel image 2022-06-17T12:31:42.200 INFO:teuthology.orchestra.run.smithi070.stdout: available. 2022-06-17T12:31:42.200 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2022-06-17T12:31:42.201 INFO:teuthology.orchestra.run.smithi107.stdout:Reading package lists... 2022-06-17T12:31:42.208 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.4IaDzjhb6z 2022-06-17T12:31:42.208 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /boot/grub/grub.cfg /tmp/tmp.4IaDzjhb6z 2022-06-17T12:31:42.211 INFO:teuthology.orchestra.run.smithi143.stderr:Found linux image: /boot/vmlinuz-5.4.0-117-generic 2022-06-17T12:31:42.223 INFO:teuthology.orchestra.run.smithi143.stderr:Found initrd image: /boot/initrd.img-5.4.0-117-generic 2022-06-17T12:31:42.249 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2022-06-17T12:31:42.264 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.4IaDzjhb6z 2022-06-17T12:31:42.268 INFO:teuthology.orchestra.run.smithi036.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-06-17T12:31:42.272 DEBUG:teuthology.orchestra.run.smithi136:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-06-17T12:31:42.298 INFO:teuthology.orchestra.run.smithi036.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-17T12:31:42.303 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.4IaDzjhb6z is 9KB 2022-06-17T12:31:42.321 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.4IaDzjhb6z 2022-06-17T12:31:42.339 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2022-06-17T12:31:42.340 INFO:teuthology.orchestra.run.smithi036.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-06-17T12:31:42.345 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2022-06-17T12:31:42.372 INFO:teuthology.orchestra.run.smithi107.stdout:Building dependency tree... 2022-06-17T12:31:42.373 INFO:teuthology.orchestra.run.smithi107.stdout:Reading state information... 2022-06-17T12:31:42.383 INFO:teuthology.orchestra.run.smithi036.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-06-17T12:31:42.399 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2022-06-17T12:31:42.400 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/grub.d/01_ceph_kernel 2022-06-17T12:31:42.400 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2022-06-17T12:31:42.443 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-120-generic 2022-06-17T12:31:42.444 DEBUG:teuthology.orchestra.run.smithi070:> sudo update-grub 2022-06-17T12:31:42.520 INFO:teuthology.orchestra.run.smithi136.stdout:Building dependency tree... 2022-06-17T12:31:42.520 INFO:teuthology.orchestra.run.smithi136.stdout:Reading state information... 2022-06-17T12:31:42.545 INFO:teuthology.orchestra.run.smithi107.stdout:linux-image-generic is already the newest version (5.4.0.120.121). 2022-06-17T12:31:42.545 INFO:teuthology.orchestra.run.smithi107.stdout:0 upgraded, 0 newly installed, 0 to remove and 5 not upgraded. 2022-06-17T12:31:42.547 DEBUG:teuthology.orchestra.run.smithi107:> dpkg -s linux-image-generic 2022-06-17T12:31:42.570 INFO:teuthology.orchestra.run.smithi107.stdout:Package: linux-image-generic 2022-06-17T12:31:42.570 INFO:teuthology.orchestra.run.smithi107.stdout:Status: install ok installed 2022-06-17T12:31:42.571 INFO:teuthology.orchestra.run.smithi107.stdout:Priority: optional 2022-06-17T12:31:42.571 INFO:teuthology.orchestra.run.smithi107.stdout:Section: kernel 2022-06-17T12:31:42.571 INFO:teuthology.orchestra.run.smithi107.stdout:Installed-Size: 19 2022-06-17T12:31:42.571 INFO:teuthology.orchestra.run.smithi107.stdout:Maintainer: Ubuntu Kernel Team 2022-06-17T12:31:42.571 INFO:teuthology.orchestra.run.smithi107.stdout:Architecture: amd64 2022-06-17T12:31:42.572 INFO:teuthology.orchestra.run.smithi107.stdout:Source: linux-meta 2022-06-17T12:31:42.572 INFO:teuthology.orchestra.run.smithi107.stdout:Version: 5.4.0.120.121 2022-06-17T12:31:42.572 INFO:teuthology.orchestra.run.smithi107.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.13) 2022-06-17T12:31:42.572 INFO:teuthology.orchestra.run.smithi107.stdout:Depends: linux-image-5.4.0-120-generic, linux-modules-extra-5.4.0-120-generic, linux-firmware, intel-microcode, amd64-microcode 2022-06-17T12:31:42.573 INFO:teuthology.orchestra.run.smithi107.stdout:Recommends: thermald 2022-06-17T12:31:42.573 INFO:teuthology.orchestra.run.smithi107.stdout:Description: Generic Linux kernel image 2022-06-17T12:31:42.573 INFO:teuthology.orchestra.run.smithi107.stdout: This package will always depend on the latest generic kernel image 2022-06-17T12:31:42.573 INFO:teuthology.orchestra.run.smithi107.stdout: available. 2022-06-17T12:31:42.574 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2022-06-17T12:31:42.579 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.DAIUhdvovi 2022-06-17T12:31:42.579 DEBUG:teuthology.orchestra.run.smithi107:> sudo cp /boot/grub/grub.cfg /tmp/tmp.DAIUhdvovi 2022-06-17T12:31:42.594 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.DAIUhdvovi 2022-06-17T12:31:42.694 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.DAIUhdvovi is 9KB 2022-06-17T12:31:42.708 INFO:teuthology.orchestra.run.smithi136.stdout:linux-image-generic is already the newest version (5.4.0.120.121). 2022-06-17T12:31:42.709 INFO:teuthology.orchestra.run.smithi136.stdout:0 upgraded, 0 newly installed, 0 to remove and 5 not upgraded. 2022-06-17T12:31:42.711 DEBUG:teuthology.orchestra.run.smithi136:> dpkg -s linux-image-generic 2022-06-17T12:31:42.735 INFO:teuthology.orchestra.run.smithi136.stdout:Package: linux-image-generic 2022-06-17T12:31:42.736 INFO:teuthology.orchestra.run.smithi136.stdout:Status: install ok installed 2022-06-17T12:31:42.736 INFO:teuthology.orchestra.run.smithi136.stdout:Priority: optional 2022-06-17T12:31:42.737 INFO:teuthology.orchestra.run.smithi136.stdout:Section: kernel 2022-06-17T12:31:42.737 INFO:teuthology.orchestra.run.smithi136.stdout:Installed-Size: 19 2022-06-17T12:31:42.738 INFO:teuthology.orchestra.run.smithi136.stdout:Maintainer: Ubuntu Kernel Team 2022-06-17T12:31:42.738 INFO:teuthology.orchestra.run.smithi136.stdout:Architecture: amd64 2022-06-17T12:31:42.739 INFO:teuthology.orchestra.run.smithi136.stdout:Source: linux-meta 2022-06-17T12:31:42.740 INFO:teuthology.orchestra.run.smithi136.stdout:Version: 5.4.0.120.121 2022-06-17T12:31:42.740 INFO:teuthology.orchestra.run.smithi136.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.13) 2022-06-17T12:31:42.741 INFO:teuthology.orchestra.run.smithi136.stdout:Depends: linux-image-5.4.0-120-generic, linux-modules-extra-5.4.0-120-generic, linux-firmware, intel-microcode, amd64-microcode 2022-06-17T12:31:42.741 INFO:teuthology.orchestra.run.smithi136.stdout:Recommends: thermald 2022-06-17T12:31:42.741 INFO:teuthology.orchestra.run.smithi136.stdout:Description: Generic Linux kernel image 2022-06-17T12:31:42.742 INFO:teuthology.orchestra.run.smithi136.stdout: This package will always depend on the latest generic kernel image 2022-06-17T12:31:42.742 INFO:teuthology.orchestra.run.smithi136.stdout: available. 2022-06-17T12:31:42.743 DEBUG:teuthology.orchestra.run.smithi136:> mktemp 2022-06-17T12:31:42.750 INFO:teuthology.orchestra.run.smithi136.stdout:/tmp/tmp.JCHjB9UzyC 2022-06-17T12:31:42.751 DEBUG:teuthology.orchestra.run.smithi136:> sudo cp /boot/grub/grub.cfg /tmp/tmp.JCHjB9UzyC 2022-06-17T12:31:42.762 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.DAIUhdvovi 2022-06-17T12:31:42.772 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2022-06-17T12:31:42.834 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2022-06-17T12:31:42.834 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/grub.d/01_ceph_kernel 2022-06-17T12:31:42.835 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2022-06-17T12:31:42.836 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0666 /tmp/tmp.JCHjB9UzyC 2022-06-17T12:31:42.900 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-120-generic 2022-06-17T12:31:42.900 DEBUG:teuthology.orchestra.run.smithi107:> sudo update-grub 2022-06-17T12:31:42.921 DEBUG:teuthology.orchestra.remote:smithi136:/tmp/tmp.JCHjB9UzyC is 9KB 2022-06-17T12:31:42.936 DEBUG:teuthology.orchestra.run.smithi136:> rm -fr /tmp/tmp.JCHjB9UzyC 2022-06-17T12:31:42.945 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2022-06-17T12:31:43.007 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2022-06-17T12:31:43.008 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/grub.d/01_ceph_kernel 2022-06-17T12:31:43.008 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2022-06-17T12:31:43.063 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-120-generic 2022-06-17T12:31:43.068 DEBUG:teuthology.orchestra.run.smithi136:> sudo update-grub 2022-06-17T12:31:43.755 INFO:teuthology.orchestra.run.smithi070.stderr:Sourcing file `/etc/default/grub' 2022-06-17T12:31:43.758 INFO:teuthology.orchestra.run.smithi070.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-06-17T12:31:43.762 INFO:teuthology.orchestra.run.smithi070.stderr:Generating grub configuration file ... 2022-06-17T12:31:43.854 INFO:teuthology.orchestra.run.smithi143.stderr:done 2022-06-17T12:31:43.855 DEBUG:teuthology.orchestra.run.smithi143:> sudo shutdown -r now 2022-06-17T12:31:43.916 INFO:teuthology.orchestra.run.smithi036.stdout:Reading package lists... 2022-06-17T12:31:43.944 DEBUG:teuthology.orchestra.run.smithi036:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-06-17T12:31:44.015 INFO:teuthology.orchestra.run.smithi070.stderr:Found linux image: /boot/vmlinuz-5.4.0-120-generic 2022-06-17T12:31:44.022 INFO:teuthology.orchestra.run.smithi036.stdout:Reading package lists... 2022-06-17T12:31:44.027 INFO:teuthology.orchestra.run.smithi070.stderr:Found initrd image: /boot/initrd.img-5.4.0-120-generic 2022-06-17T12:31:44.170 INFO:teuthology.orchestra.run.smithi107.stderr:Sourcing file `/etc/default/grub' 2022-06-17T12:31:44.180 INFO:teuthology.orchestra.run.smithi107.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-06-17T12:31:44.180 INFO:teuthology.orchestra.run.smithi107.stderr:Generating grub configuration file ... 2022-06-17T12:31:44.199 INFO:teuthology.orchestra.run.smithi036.stdout:Building dependency tree... 2022-06-17T12:31:44.200 INFO:teuthology.orchestra.run.smithi036.stdout:Reading state information... 2022-06-17T12:31:44.374 INFO:teuthology.orchestra.run.smithi136.stderr:Sourcing file `/etc/default/grub' 2022-06-17T12:31:44.380 INFO:teuthology.orchestra.run.smithi136.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-06-17T12:31:44.385 INFO:teuthology.orchestra.run.smithi136.stderr:Generating grub configuration file ... 2022-06-17T12:31:44.397 INFO:teuthology.orchestra.run.smithi036.stdout:linux-image-generic is already the newest version (5.4.0.120.121). 2022-06-17T12:31:44.397 INFO:teuthology.orchestra.run.smithi036.stdout:0 upgraded, 0 newly installed, 0 to remove and 5 not upgraded. 2022-06-17T12:31:44.399 DEBUG:teuthology.orchestra.run.smithi036:> dpkg -s linux-image-generic 2022-06-17T12:31:44.400 INFO:teuthology.orchestra.run.smithi070.stderr:Found linux image: /boot/vmlinuz-5.4.0-117-generic 2022-06-17T12:31:44.410 INFO:teuthology.orchestra.run.smithi070.stderr:Found initrd image: /boot/initrd.img-5.4.0-117-generic 2022-06-17T12:31:44.417 INFO:teuthology.orchestra.run.smithi107.stderr:Found linux image: /boot/vmlinuz-5.4.0-120-generic 2022-06-17T12:31:44.422 INFO:teuthology.orchestra.run.smithi036.stdout:Package: linux-image-generic 2022-06-17T12:31:44.422 INFO:teuthology.orchestra.run.smithi036.stdout:Status: install ok installed 2022-06-17T12:31:44.423 INFO:teuthology.orchestra.run.smithi036.stdout:Priority: optional 2022-06-17T12:31:44.423 INFO:teuthology.orchestra.run.smithi036.stdout:Section: kernel 2022-06-17T12:31:44.423 INFO:teuthology.orchestra.run.smithi036.stdout:Installed-Size: 19 2022-06-17T12:31:44.423 INFO:teuthology.orchestra.run.smithi036.stdout:Maintainer: Ubuntu Kernel Team 2022-06-17T12:31:44.424 INFO:teuthology.orchestra.run.smithi036.stdout:Architecture: amd64 2022-06-17T12:31:44.424 INFO:teuthology.orchestra.run.smithi036.stdout:Source: linux-meta 2022-06-17T12:31:44.424 INFO:teuthology.orchestra.run.smithi036.stdout:Version: 5.4.0.120.121 2022-06-17T12:31:44.424 INFO:teuthology.orchestra.run.smithi036.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.13) 2022-06-17T12:31:44.424 INFO:teuthology.orchestra.run.smithi036.stdout:Depends: linux-image-5.4.0-120-generic, linux-modules-extra-5.4.0-120-generic, linux-firmware, intel-microcode, amd64-microcode 2022-06-17T12:31:44.425 INFO:teuthology.orchestra.run.smithi036.stdout:Recommends: thermald 2022-06-17T12:31:44.425 INFO:teuthology.orchestra.run.smithi036.stdout:Description: Generic Linux kernel image 2022-06-17T12:31:44.425 INFO:teuthology.orchestra.run.smithi036.stdout: This package will always depend on the latest generic kernel image 2022-06-17T12:31:44.425 INFO:teuthology.orchestra.run.smithi036.stdout: available. 2022-06-17T12:31:44.426 DEBUG:teuthology.orchestra.run.smithi036:> mktemp 2022-06-17T12:31:44.428 INFO:teuthology.orchestra.run.smithi107.stderr:Found initrd image: /boot/initrd.img-5.4.0-120-generic 2022-06-17T12:31:44.471 INFO:teuthology.orchestra.run.smithi036.stdout:/tmp/tmp.rZIUnU6TNP 2022-06-17T12:31:44.472 DEBUG:teuthology.orchestra.run.smithi036:> sudo cp /boot/grub/grub.cfg /tmp/tmp.rZIUnU6TNP 2022-06-17T12:31:44.489 DEBUG:teuthology.orchestra.run.smithi036:> sudo chmod 0666 /tmp/tmp.rZIUnU6TNP 2022-06-17T12:31:44.570 DEBUG:teuthology.orchestra.remote:smithi036:/tmp/tmp.rZIUnU6TNP is 9KB 2022-06-17T12:31:44.587 DEBUG:teuthology.orchestra.run.smithi036:> rm -fr /tmp/tmp.rZIUnU6TNP 2022-06-17T12:31:44.598 DEBUG:teuthology.orchestra.run.smithi036:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2022-06-17T12:31:44.642 INFO:teuthology.orchestra.run.smithi136.stderr:Found linux image: /boot/vmlinuz-5.4.0-120-generic 2022-06-17T12:31:44.651 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-06-17T12:31:44.651 DEBUG:teuthology.orchestra.run.smithi036:> sudo dd of=/etc/grub.d/01_ceph_kernel 2022-06-17T12:31:44.652 DEBUG:teuthology.orchestra.run.smithi036:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2022-06-17T12:31:44.654 INFO:teuthology.orchestra.run.smithi136.stderr:Found initrd image: /boot/initrd.img-5.4.0-120-generic 2022-06-17T12:31:44.684 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-120-generic 2022-06-17T12:31:44.685 DEBUG:teuthology.orchestra.run.smithi036:> sudo update-grub 2022-06-17T12:31:44.778 INFO:teuthology.orchestra.run.smithi107.stderr:Found linux image: /boot/vmlinuz-5.4.0-117-generic 2022-06-17T12:31:44.789 INFO:teuthology.orchestra.run.smithi107.stderr:Found initrd image: /boot/initrd.img-5.4.0-117-generic 2022-06-17T12:31:45.046 INFO:teuthology.orchestra.run.smithi136.stderr:Found linux image: /boot/vmlinuz-5.4.0-117-generic 2022-06-17T12:31:45.053 INFO:teuthology.orchestra.run.smithi136.stderr:Found initrd image: /boot/initrd.img-5.4.0-117-generic 2022-06-17T12:31:45.961 INFO:teuthology.orchestra.run.smithi070.stderr:done 2022-06-17T12:31:45.963 DEBUG:teuthology.orchestra.run.smithi070:> sudo shutdown -r now 2022-06-17T12:31:46.121 INFO:teuthology.orchestra.run.smithi036.stderr:Sourcing file `/etc/default/grub' 2022-06-17T12:31:46.135 INFO:teuthology.orchestra.run.smithi036.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-06-17T12:31:46.151 INFO:teuthology.orchestra.run.smithi036.stderr:Generating grub configuration file ... 2022-06-17T12:31:46.397 INFO:teuthology.orchestra.run.smithi107.stderr:done 2022-06-17T12:31:46.399 DEBUG:teuthology.orchestra.run.smithi107:> sudo shutdown -r now 2022-06-17T12:31:46.409 INFO:teuthology.orchestra.run.smithi036.stderr:Found linux image: /boot/vmlinuz-5.4.0-120-generic 2022-06-17T12:31:46.422 INFO:teuthology.orchestra.run.smithi036.stderr:Found initrd image: /boot/initrd.img-5.4.0-120-generic 2022-06-17T12:31:46.708 INFO:teuthology.orchestra.run.smithi136.stderr:done 2022-06-17T12:31:46.711 DEBUG:teuthology.orchestra.run.smithi136:> sudo shutdown -r now 2022-06-17T12:31:46.809 INFO:teuthology.orchestra.run.smithi036.stderr:Found linux image: /boot/vmlinuz-5.4.0-117-generic 2022-06-17T12:31:46.830 INFO:teuthology.orchestra.run.smithi036.stderr:Found initrd image: /boot/initrd.img-5.4.0-117-generic 2022-06-17T12:31:48.441 INFO:teuthology.orchestra.run.smithi036.stderr:done 2022-06-17T12:31:48.443 DEBUG:teuthology.orchestra.run.smithi036:> sudo shutdown -r now 2022-06-17T12:32:13.882 INFO:teuthology.task.kernel:Checking client client.2 for new kernel version... 2022-06-17T12:32:13.883 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2022-06-17T12:32:13.884 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-17T12:32:15.973 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2022-06-17T12:32:15.974 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2022-06-17T12:32:15.975 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-17T12:32:16.405 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2022-06-17T12:32:16.406 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2022-06-17T12:32:16.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-17T12:32:16.731 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2022-06-17T12:32:16.732 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2022-06-17T12:32:16.733 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-17T12:32:18.449 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2022-06-17T12:32:18.449 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi036.front.sepia.ceph.com' 2022-06-17T12:32:18.450 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-17T12:32:32.255 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.143 2022-06-17T12:32:34.302 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.70 2022-06-17T12:32:34.810 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2022-06-17T12:32:35.066 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.136 2022-06-17T12:32:36.862 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.36 2022-06-17T12:33:02.287 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2022-06-17T12:33:02.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-17T12:33:04.306 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2022-06-17T12:33:04.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-17T12:33:04.812 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2022-06-17T12:33:04.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-17T12:33:05.069 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2022-06-17T12:33:05.070 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-17T12:33:05.371 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.143 2022-06-17T12:33:06.865 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi036.front.sepia.ceph.com' 2022-06-17T12:33:06.866 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-17T12:33:07.387 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.70 2022-06-17T12:33:07.866 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2022-06-17T12:33:08.154 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.136 2022-06-17T12:33:22.515 DEBUG:teuthology.orchestra.run.smithi036:> true 2022-06-17T12:33:23.013 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi036.front.sepia.ceph.com' 2022-06-17T12:33:23.014 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-120-generic"... 2022-06-17T12:33:23.015 DEBUG:teuthology.orchestra.run.smithi036:> uname -r 2022-06-17T12:33:23.083 INFO:teuthology.orchestra.run.smithi036.stdout:5.4.0-120-generic 2022-06-17T12:33:23.084 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-120-generic vs 5.4.0-120-generic 2022-06-17T12:33:23.084 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2022-06-17T12:33:23.085 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2022-06-17T12:33:24.088 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2022-06-17T12:33:24.089 DEBUG:teuthology.orchestra.run.smithi036:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-06-17T12:33:24.195 INFO:teuthology.orchestra.run.smithi036.stdout:ttyS1 2022-06-17T12:33:24.219 DEBUG:teuthology.parallel:result is None 2022-06-17T12:33:35.374 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2022-06-17T12:33:35.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-17T12:33:35.668 DEBUG:teuthology.orchestra.run.smithi143:> true 2022-06-17T12:33:36.171 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2022-06-17T12:33:36.172 INFO:teuthology.task.kernel:Checking kernel version of client.2, want "5.4.0-120-generic"... 2022-06-17T12:33:36.172 DEBUG:teuthology.orchestra.run.smithi143:> uname -r 2022-06-17T12:33:36.182 INFO:teuthology.orchestra.run.smithi143.stdout:5.4.0-120-generic 2022-06-17T12:33:36.182 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-120-generic vs 5.4.0-120-generic 2022-06-17T12:33:36.183 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2022-06-17T12:33:36.183 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2022-06-17T12:33:37.185 INFO:teuthology.task.kernel:Enabling kdb on client.2... 2022-06-17T12:33:37.185 DEBUG:teuthology.orchestra.run.smithi143:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-06-17T12:33:37.288 INFO:teuthology.orchestra.run.smithi143.stdout:ttyS1 2022-06-17T12:33:37.302 DEBUG:teuthology.parallel:result is None 2022-06-17T12:33:37.389 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2022-06-17T12:33:37.390 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-17T12:33:37.652 DEBUG:teuthology.orchestra.run.smithi070:> true 2022-06-17T12:33:37.868 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2022-06-17T12:33:37.869 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-17T12:33:38.106 DEBUG:teuthology.orchestra.run.smithi107:> true 2022-06-17T12:33:38.150 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2022-06-17T12:33:38.151 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-120-generic"... 2022-06-17T12:33:38.152 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2022-06-17T12:33:38.155 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2022-06-17T12:33:38.156 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-17T12:33:38.200 INFO:teuthology.orchestra.run.smithi070.stdout:5.4.0-120-generic 2022-06-17T12:33:38.200 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-120-generic vs 5.4.0-120-generic 2022-06-17T12:33:38.201 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2022-06-17T12:33:38.201 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2022-06-17T12:33:38.426 DEBUG:teuthology.orchestra.run.smithi136:> true 2022-06-17T12:33:38.604 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2022-06-17T12:33:38.606 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.4.0-120-generic"... 2022-06-17T12:33:38.607 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2022-06-17T12:33:38.655 INFO:teuthology.orchestra.run.smithi107.stdout:5.4.0-120-generic 2022-06-17T12:33:38.656 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-120-generic vs 5.4.0-120-generic 2022-06-17T12:33:38.657 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2022-06-17T12:33:38.657 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2022-06-17T12:33:38.917 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2022-06-17T12:33:38.917 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.4.0-120-generic"... 2022-06-17T12:33:38.918 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2022-06-17T12:33:38.925 INFO:teuthology.orchestra.run.smithi136.stdout:5.4.0-120-generic 2022-06-17T12:33:38.926 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-120-generic vs 5.4.0-120-generic 2022-06-17T12:33:38.926 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2022-06-17T12:33:38.927 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2022-06-17T12:33:39.204 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2022-06-17T12:33:39.205 DEBUG:teuthology.orchestra.run.smithi070:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-06-17T12:33:39.301 INFO:teuthology.orchestra.run.smithi070.stdout:ttyS1 2022-06-17T12:33:39.326 DEBUG:teuthology.parallel:result is None 2022-06-17T12:33:39.658 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2022-06-17T12:33:39.659 DEBUG:teuthology.orchestra.run.smithi107:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-06-17T12:33:39.743 INFO:teuthology.orchestra.run.smithi107.stdout:ttyS1 2022-06-17T12:33:39.779 DEBUG:teuthology.parallel:result is None 2022-06-17T12:33:39.927 INFO:teuthology.task.kernel:Enabling kdb on client.1... 2022-06-17T12:33:39.928 DEBUG:teuthology.orchestra.run.smithi136:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-06-17T12:33:40.022 INFO:teuthology.orchestra.run.smithi136.stdout:ttyS1 2022-06-17T12:33:40.050 DEBUG:teuthology.parallel:result is None 2022-06-17T12:33:40.051 INFO:teuthology.run_tasks:Running task internal.base... 2022-06-17T12:33:40.092 INFO:teuthology.task.internal:Creating test directory... 2022-06-17T12:33:40.094 DEBUG:teuthology.orchestra.run.smithi036:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-06-17T12:33:40.098 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-06-17T12:33:40.105 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-06-17T12:33:40.109 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-06-17T12:33:40.115 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-06-17T12:33:40.125 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2022-06-17T12:33:40.158 INFO:teuthology.run_tasks:Running task internal.archive... 2022-06-17T12:33:40.172 INFO:teuthology.task.internal:Creating archive directory... 2022-06-17T12:33:40.173 DEBUG:teuthology.orchestra.run.smithi036:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-06-17T12:33:40.184 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-06-17T12:33:40.188 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-06-17T12:33:40.192 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-06-17T12:33:40.198 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-06-17T12:33:40.476 INFO:teuthology.run_tasks:Running task internal.coredump... 2022-06-17T12:33:40.487 INFO:teuthology.task.internal:Enabling coredump saving... 2022-06-17T12:33:40.487 DEBUG:teuthology.orchestra.run.smithi036:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-06-17T12:33:40.491 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-06-17T12:33:40.495 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-06-17T12:33:40.500 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-06-17T12:33:40.504 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-06-17T12:33:40.518 INFO:teuthology.orchestra.run.smithi036.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-06-17T12:33:40.523 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-06-17T12:33:40.524 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-06-17T12:33:40.526 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-06-17T12:33:40.527 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-06-17T12:33:40.529 INFO:teuthology.orchestra.run.smithi036.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-06-17T12:33:40.535 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-06-17T12:33:40.543 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-06-17T12:33:40.546 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-06-17T12:33:40.547 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-06-17T12:33:40.549 INFO:teuthology.run_tasks:Running task internal.sudo... 2022-06-17T12:33:40.564 INFO:teuthology.task.internal:Configuring sudo... 2022-06-17T12:33:40.565 DEBUG:teuthology.orchestra.run.smithi036:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-06-17T12:33:40.584 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-06-17T12:33:40.589 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-06-17T12:33:40.602 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-06-17T12:33:40.606 DEBUG:teuthology.orchestra.run.smithi143:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-06-17T12:33:40.624 INFO:teuthology.run_tasks:Running task internal.syslog... 2022-06-17T12:33:40.640 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2022-06-17T12:33:40.641 DEBUG:teuthology.orchestra.run.smithi036:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-06-17T12:33:40.661 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-06-17T12:33:40.668 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-06-17T12:33:40.692 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-06-17T12:33:40.699 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-06-17T12:33:40.707 DEBUG:teuthology.orchestra.run.smithi036:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-06-17T12:33:40.716 DEBUG:teuthology.orchestra.run.smithi036:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-06-17T12:33:40.768 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-06-17T12:33:40.769 DEBUG:teuthology.orchestra.run.smithi036:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-06-17T12:33:40.810 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-06-17T12:33:40.822 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-06-17T12:33:40.829 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2022-06-17T12:33:40.830 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-06-17T12:33:40.892 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-06-17T12:33:40.903 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-06-17T12:33:40.955 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2022-06-17T12:33:40.956 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-06-17T12:33:40.987 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-06-17T12:33:40.996 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-06-17T12:33:41.057 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2022-06-17T12:33:41.057 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-06-17T12:33:41.087 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-06-17T12:33:41.097 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-06-17T12:33:41.113 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2022-06-17T12:33:41.113 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-06-17T12:33:41.182 DEBUG:teuthology.orchestra.run.smithi036:> sudo service rsyslog restart 2022-06-17T12:33:41.188 DEBUG:teuthology.orchestra.run.smithi070:> sudo service rsyslog restart 2022-06-17T12:33:41.193 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2022-06-17T12:33:41.199 DEBUG:teuthology.orchestra.run.smithi136:> sudo service rsyslog restart 2022-06-17T12:33:41.203 DEBUG:teuthology.orchestra.run.smithi143:> sudo service rsyslog restart 2022-06-17T12:33:41.287 INFO:teuthology.run_tasks:Running task internal.timer... 2022-06-17T12:33:41.312 INFO:teuthology.task.internal:Starting timer... 2022-06-17T12:33:41.313 INFO:teuthology.run_tasks:Running task pcp... 2022-06-17T12:33:41.335 INFO:teuthology.run_tasks:Running task selinux... 2022-06-17T12:33:41.368 DEBUG:teuthology.task.selinux:Excluding smithi036: OS 'ubuntu' does not support SELinux 2022-06-17T12:33:41.378 DEBUG:teuthology.task.selinux:Excluding smithi070: OS 'ubuntu' does not support SELinux 2022-06-17T12:33:41.379 DEBUG:teuthology.task.selinux:Excluding smithi107: OS 'ubuntu' does not support SELinux 2022-06-17T12:33:41.379 DEBUG:teuthology.task.selinux:Excluding smithi136: OS 'ubuntu' does not support SELinux 2022-06-17T12:33:41.380 DEBUG:teuthology.task.selinux:Excluding smithi143: OS 'ubuntu' does not support SELinux 2022-06-17T12:33:41.380 DEBUG:teuthology.task.selinux:Getting current SELinux state 2022-06-17T12:33:41.381 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2022-06-17T12:33:41.381 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2022-06-17T12:33:41.382 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2022-06-17T12:33:41.407 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2022-06-17T12:33:41.409 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_main to origin/main 2022-06-17T12:33:41.444 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2022-06-17T12:33:41.446 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi036.front.sepia.ceph.com,smithi070.front.sepia.ceph.com,smithi107.front.sepia.ceph.com,smithi136.front.sepia.ceph.com,smithi143.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_main/cephlab.yml 2022-06-17T12:43:14.769 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi036.front.sepia.ceph.com'), Remote(name='ubuntu@smithi070.front.sepia.ceph.com'), Remote(name='ubuntu@smithi107.front.sepia.ceph.com'), Remote(name='ubuntu@smithi136.front.sepia.ceph.com'), Remote(name='ubuntu@smithi143.front.sepia.ceph.com')] 2022-06-17T12:43:14.779 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi036.front.sepia.ceph.com' 2022-06-17T12:43:14.782 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-17T12:43:15.047 DEBUG:teuthology.orchestra.run.smithi036:> true 2022-06-17T12:43:15.293 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2022-06-17T12:43:15.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-17T12:43:15.590 DEBUG:teuthology.orchestra.run.smithi070:> true 2022-06-17T12:43:15.714 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2022-06-17T12:43:15.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-17T12:43:16.228 DEBUG:teuthology.orchestra.run.smithi107:> true 2022-06-17T12:43:16.375 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2022-06-17T12:43:16.376 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-17T12:43:16.664 DEBUG:teuthology.orchestra.run.smithi136:> true 2022-06-17T12:43:16.799 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2022-06-17T12:43:16.834 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-17T12:43:17.372 DEBUG:teuthology.orchestra.run.smithi143:> true 2022-06-17T12:43:17.474 INFO:teuthology.run_tasks:Running task clock... 2022-06-17T12:43:17.508 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2022-06-17T12:43:17.529 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-06-17T12:43:17.546 DEBUG:teuthology.orchestra.run.smithi036:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-06-17T12:43:17.551 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-06-17T12:43:17.556 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-06-17T12:43:17.593 INFO:teuthology.orchestra.run.smithi036.stdout:17 Jun 12:43:17 ntpd[17513]: ntpd 4.2.8p12@1.3728-o (1): Starting 2022-06-17T12:43:17.616 INFO:teuthology.orchestra.run.smithi036.stdout:17 Jun 12:43:17 ntpd[17513]: Command line: ntpd -gq 2022-06-17T12:43:17.616 INFO:teuthology.orchestra.run.smithi036.stdout:17 Jun 12:43:17 ntpd[17513]: proto: precision = 0.070 usec (-24) 2022-06-17T12:43:17.617 INFO:teuthology.orchestra.run.smithi036.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-06-17T12:43:17.618 INFO:teuthology.orchestra.run.smithi036.stdout:17 Jun 12:43:17 ntpd[17513]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-06-17T12:43:17.624 INFO:teuthology.orchestra.run.smithi036.stderr:restrict ::: KOD does nothing without LIMITED. 2022-06-17T12:43:17.630 INFO:teuthology.orchestra.run.smithi036.stdout:17 Jun 12:43:17 ntpd[17513]: restrict ::: KOD does nothing without LIMITED. 2022-06-17T12:43:17.631 INFO:teuthology.orchestra.run.smithi036.stdout:17 Jun 12:43:17 ntpd[17513]: Listen and drop on 0 v6wildcard [::]:123 2022-06-17T12:43:17.631 INFO:teuthology.orchestra.run.smithi036.stdout:17 Jun 12:43:17 ntpd[17513]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2022-06-17T12:43:17.632 INFO:teuthology.orchestra.run.smithi036.stdout:17 Jun 12:43:17 ntpd[17513]: Listen normally on 2 lo 127.0.0.1:123 2022-06-17T12:43:17.632 INFO:teuthology.orchestra.run.smithi036.stdout:17 Jun 12:43:17 ntpd[17513]: Listen normally on 3 ens1f0 172.21.15.36:123 2022-06-17T12:43:17.633 INFO:teuthology.orchestra.run.smithi036.stdout:17 Jun 12:43:17 ntpd[17513]: Listen normally on 4 lo [::1]:123 2022-06-17T12:43:17.672 INFO:teuthology.orchestra.run.smithi036.stdout:17 Jun 12:43:17 ntpd[17513]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:2542%4]:123 2022-06-17T12:43:17.687 INFO:teuthology.orchestra.run.smithi036.stdout:17 Jun 12:43:17 ntpd[17513]: Listening on routing socket on fd #22 for interface updates 2022-06-17T12:43:17.691 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-06-17T12:43:17.692 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-06-17T12:43:17.708 INFO:teuthology.orchestra.run.smithi070.stdout:17 Jun 12:43:17 ntpd[17566]: ntpd 4.2.8p12@1.3728-o (1): Starting 2022-06-17T12:43:17.708 INFO:teuthology.orchestra.run.smithi070.stdout:17 Jun 12:43:17 ntpd[17566]: Command line: ntpd -gq 2022-06-17T12:43:17.717 INFO:teuthology.orchestra.run.smithi070.stdout:17 Jun 12:43:17 ntpd[17566]: proto: precision = 0.157 usec (-23) 2022-06-17T12:43:17.734 INFO:teuthology.orchestra.run.smithi070.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-06-17T12:43:17.746 INFO:teuthology.orchestra.run.smithi070.stdout:17 Jun 12:43:17 ntpd[17566]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-06-17T12:43:17.746 INFO:teuthology.orchestra.run.smithi070.stdout:17 Jun 12:43:17 ntpd[17566]: restrict ::: KOD does nothing without LIMITED. 2022-06-17T12:43:17.747 INFO:teuthology.orchestra.run.smithi070.stderr:restrict ::: KOD does nothing without LIMITED. 2022-06-17T12:43:17.755 INFO:teuthology.orchestra.run.smithi070.stdout:17 Jun 12:43:17 ntpd[17566]: Listen and drop on 0 v6wildcard [::]:123 2022-06-17T12:43:17.756 INFO:teuthology.orchestra.run.smithi070.stdout:17 Jun 12:43:17 ntpd[17566]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2022-06-17T12:43:17.756 INFO:teuthology.orchestra.run.smithi070.stdout:17 Jun 12:43:17 ntpd[17566]: Listen normally on 2 lo 127.0.0.1:123 2022-06-17T12:43:17.757 INFO:teuthology.orchestra.run.smithi070.stdout:17 Jun 12:43:17 ntpd[17566]: Listen normally on 3 enp3s0f1 172.21.15.70:123 2022-06-17T12:43:17.758 INFO:teuthology.orchestra.run.smithi070.stdout:17 Jun 12:43:17 ntpd[17566]: Listen normally on 4 lo [::1]:123 2022-06-17T12:43:17.758 INFO:teuthology.orchestra.run.smithi070.stdout:17 Jun 12:43:17 ntpd[17566]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d3e7%5]:123 2022-06-17T12:43:17.759 INFO:teuthology.orchestra.run.smithi070.stdout:17 Jun 12:43:17 ntpd[17566]: Listening on routing socket on fd #22 for interface updates 2022-06-17T12:43:17.775 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-06-17T12:43:17.776 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-06-17T12:43:17.791 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-06-17T12:43:17.792 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-06-17T12:43:17.839 INFO:teuthology.orchestra.run.smithi136.stdout:17 Jun 12:43:17 ntpd[17571]: ntpd 4.2.8p12@1.3728-o (1): Starting 2022-06-17T12:43:17.839 INFO:teuthology.orchestra.run.smithi136.stdout:17 Jun 12:43:17 ntpd[17571]: Command line: ntpd -gq 2022-06-17T12:43:17.840 INFO:teuthology.orchestra.run.smithi136.stdout:17 Jun 12:43:17 ntpd[17571]: proto: precision = 0.184 usec (-22) 2022-06-17T12:43:17.841 INFO:teuthology.orchestra.run.smithi107.stdout:17 Jun 12:43:17 ntpd[17601]: ntpd 4.2.8p12@1.3728-o (1): Starting 2022-06-17T12:43:17.841 INFO:teuthology.orchestra.run.smithi107.stdout:17 Jun 12:43:17 ntpd[17601]: Command line: ntpd -gq 2022-06-17T12:43:17.841 INFO:teuthology.orchestra.run.smithi107.stdout:17 Jun 12:43:17 ntpd[17601]: proto: precision = 0.191 usec (-22) 2022-06-17T12:43:17.843 INFO:teuthology.orchestra.run.smithi136.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-06-17T12:43:17.843 INFO:teuthology.orchestra.run.smithi107.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-06-17T12:43:17.844 INFO:teuthology.orchestra.run.smithi143.stdout:17 Jun 12:43:17 ntpd[17641]: ntpd 4.2.8p12@1.3728-o (1): Starting 2022-06-17T12:43:17.844 INFO:teuthology.orchestra.run.smithi143.stdout:17 Jun 12:43:17 ntpd[17641]: Command line: ntpd -gq 2022-06-17T12:43:17.845 INFO:teuthology.orchestra.run.smithi143.stdout:17 Jun 12:43:17 ntpd[17641]: proto: precision = 0.114 usec (-23) 2022-06-17T12:43:17.846 INFO:teuthology.orchestra.run.smithi136.stdout:17 Jun 12:43:17 ntpd[17571]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-06-17T12:43:17.855 INFO:teuthology.orchestra.run.smithi143.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-06-17T12:43:17.856 INFO:teuthology.orchestra.run.smithi136.stderr:restrict ::: KOD does nothing without LIMITED. 2022-06-17T12:43:17.857 INFO:teuthology.orchestra.run.smithi107.stdout:17 Jun 12:43:17 ntpd[17601]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-06-17T12:43:17.857 INFO:teuthology.orchestra.run.smithi107.stdout:17 Jun 12:43:17 ntpd[17601]: restrict ::: KOD does nothing without LIMITED. 2022-06-17T12:43:17.859 INFO:teuthology.orchestra.run.smithi143.stdout:17 Jun 12:43:17 ntpd[17641]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-06-17T12:43:17.863 INFO:teuthology.orchestra.run.smithi107.stderr:restrict ::: KOD does nothing without LIMITED. 2022-06-17T12:43:17.865 INFO:teuthology.orchestra.run.smithi143.stderr:restrict ::: KOD does nothing without LIMITED. 2022-06-17T12:43:17.880 INFO:teuthology.orchestra.run.smithi136.stdout:17 Jun 12:43:17 ntpd[17571]: restrict ::: KOD does nothing without LIMITED. 2022-06-17T12:43:17.880 INFO:teuthology.orchestra.run.smithi136.stdout:17 Jun 12:43:17 ntpd[17571]: Listen and drop on 0 v6wildcard [::]:123 2022-06-17T12:43:17.881 INFO:teuthology.orchestra.run.smithi136.stdout:17 Jun 12:43:17 ntpd[17571]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2022-06-17T12:43:17.881 INFO:teuthology.orchestra.run.smithi136.stdout:17 Jun 12:43:17 ntpd[17571]: Listen normally on 2 lo 127.0.0.1:123 2022-06-17T12:43:17.882 INFO:teuthology.orchestra.run.smithi136.stdout:17 Jun 12:43:17 ntpd[17571]: Listen normally on 3 enp3s0f1 172.21.15.136:123 2022-06-17T12:43:17.883 INFO:teuthology.orchestra.run.smithi136.stdout:17 Jun 12:43:17 ntpd[17571]: Listen normally on 4 lo [::1]:123 2022-06-17T12:43:17.883 INFO:teuthology.orchestra.run.smithi136.stdout:17 Jun 12:43:17 ntpd[17571]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6e43%5]:123 2022-06-17T12:43:17.884 INFO:teuthology.orchestra.run.smithi136.stdout:17 Jun 12:43:17 ntpd[17571]: Listening on routing socket on fd #22 for interface updates 2022-06-17T12:43:17.884 INFO:teuthology.orchestra.run.smithi107.stdout:17 Jun 12:43:17 ntpd[17601]: Listen and drop on 0 v6wildcard [::]:123 2022-06-17T12:43:17.884 INFO:teuthology.orchestra.run.smithi107.stdout:17 Jun 12:43:17 ntpd[17601]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2022-06-17T12:43:17.885 INFO:teuthology.orchestra.run.smithi107.stdout:17 Jun 12:43:17 ntpd[17601]: Listen normally on 2 lo 127.0.0.1:123 2022-06-17T12:43:17.885 INFO:teuthology.orchestra.run.smithi107.stdout:17 Jun 12:43:17 ntpd[17601]: Listen normally on 3 enp3s0f1 172.21.15.107:123 2022-06-17T12:43:17.885 INFO:teuthology.orchestra.run.smithi107.stdout:17 Jun 12:43:17 ntpd[17601]: Listen normally on 4 lo [::1]:123 2022-06-17T12:43:17.886 INFO:teuthology.orchestra.run.smithi107.stdout:17 Jun 12:43:17 ntpd[17601]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:47eb%5]:123 2022-06-17T12:43:17.886 INFO:teuthology.orchestra.run.smithi107.stdout:17 Jun 12:43:17 ntpd[17601]: Listening on routing socket on fd #22 for interface updates 2022-06-17T12:43:17.887 INFO:teuthology.orchestra.run.smithi143.stdout:17 Jun 12:43:17 ntpd[17641]: restrict ::: KOD does nothing without LIMITED. 2022-06-17T12:43:17.887 INFO:teuthology.orchestra.run.smithi143.stdout:17 Jun 12:43:17 ntpd[17641]: Listen and drop on 0 v6wildcard [::]:123 2022-06-17T12:43:17.887 INFO:teuthology.orchestra.run.smithi143.stdout:17 Jun 12:43:17 ntpd[17641]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2022-06-17T12:43:17.888 INFO:teuthology.orchestra.run.smithi143.stdout:17 Jun 12:43:17 ntpd[17641]: Listen normally on 2 lo 127.0.0.1:123 2022-06-17T12:43:17.888 INFO:teuthology.orchestra.run.smithi143.stdout:17 Jun 12:43:17 ntpd[17641]: Listen normally on 3 enp3s0f1 172.21.15.143:123 2022-06-17T12:43:17.889 INFO:teuthology.orchestra.run.smithi143.stdout:17 Jun 12:43:17 ntpd[17641]: Listen normally on 4 lo [::1]:123 2022-06-17T12:43:17.889 INFO:teuthology.orchestra.run.smithi143.stdout:17 Jun 12:43:17 ntpd[17641]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7783%5]:123 2022-06-17T12:43:17.936 INFO:teuthology.orchestra.run.smithi143.stdout:17 Jun 12:43:17 ntpd[17641]: Listening on routing socket on fd #22 for interface updates 2022-06-17T12:43:18.584 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:18 ntpd[17513]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:18.584 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:18 ntpd[17513]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:18.585 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:18 ntpd[17513]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:18.585 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:18 ntpd[17513]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:18.586 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:18 ntpd[17513]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:18.586 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:18 ntpd[17513]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:18.631 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:18 ntpd[17566]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:18.632 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:18 ntpd[17566]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:18.632 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:18 ntpd[17566]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:18.633 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:18 ntpd[17566]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:18.633 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:18 ntpd[17566]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:18.634 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:18 ntpd[17566]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:18.805 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:18 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:18.806 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:18 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:18.806 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:18 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:18.806 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:18 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:18.806 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:18 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:18.807 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:18 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:18.807 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:18 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:18.807 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:18 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:18.807 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:18 ntpd[17601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:18.808 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:18 ntpd[17601]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:18.808 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:18 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:18.808 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:18 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:18.808 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:18 ntpd[17601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:18.809 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:18 ntpd[17601]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:18.810 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:18 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:18.810 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:18 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:18.810 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:18 ntpd[17601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:18.810 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:18 ntpd[17601]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:18.825 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:18 ntpd[17571]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:18.826 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:18 ntpd[17571]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:18.826 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:18 ntpd[17571]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:18.826 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:18 ntpd[17571]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:18.827 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:18 ntpd[17571]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:18.827 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:18 ntpd[17571]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:18.827 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:18 ntpd[17571]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:18.827 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:18 ntpd[17571]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:18.830 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:18 ntpd[17641]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:18.831 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:18 ntpd[17641]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:18.831 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:18 ntpd[17641]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:18.832 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:18 ntpd[17641]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:18.832 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:18 ntpd[17641]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:18.833 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:18 ntpd[17641]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:18.833 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:18 ntpd[17641]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:18.833 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:18 ntpd[17641]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:18.834 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:18 ntpd[17641]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:18.834 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:18 ntpd[17641]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:18.835 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:18 ntpd[17641]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:18.835 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:18 ntpd[17641]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:18.835 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:18 ntpd[17641]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:18.836 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:18 ntpd[17641]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:19.635 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:19 ntpd[17566]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:19.635 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:19 ntpd[17566]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:19.636 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:19 ntpd[17566]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:19.636 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:19 ntpd[17566]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:19.636 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:19 ntpd[17566]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:19.636 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:19 ntpd[17566]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:19.836 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:19 ntpd[17571]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:19.836 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:19 ntpd[17571]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:19.837 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:19 ntpd[17571]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:19.879 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:19 ntpd[17571]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:19.879 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:19 ntpd[17571]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:19.879 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:19 ntpd[17571]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:20.586 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:20 ntpd[17513]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:20.587 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:20 ntpd[17513]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:20.587 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:20 ntpd[17513]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:20.588 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:20 ntpd[17513]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:20.588 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:20 ntpd[17513]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:20.588 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:20 ntpd[17513]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:20.589 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:20 ntpd[17513]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:20.589 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:20 ntpd[17513]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:20.590 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:20 ntpd[17513]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:20.590 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:20 ntpd[17513]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:20.591 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:20 ntpd[17513]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:20.591 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:20 ntpd[17513]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:20.634 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:20 ntpd[17566]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:20.635 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:20 ntpd[17566]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:20.636 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:20 ntpd[17566]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:20.636 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:20 ntpd[17566]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:20.636 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:20 ntpd[17566]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:20.637 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:20 ntpd[17566]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:20.847 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:20 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:20.848 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:20 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:20.848 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:20 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:20.848 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:20 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:20.849 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:20 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:20.850 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:20 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:20.851 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:20 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:20.851 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:20 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:20.852 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:20 ntpd[17601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:20.852 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:20 ntpd[17601]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:20.853 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:20 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:20.853 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:20 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:20.854 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:20 ntpd[17601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:20.854 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:20 ntpd[17601]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:20.854 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:20 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:20.855 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:20 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:20.855 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:20 ntpd[17601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:20.856 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:20 ntpd[17601]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:20.871 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:20 ntpd[17571]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:20.871 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:20 ntpd[17571]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:20.873 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:20 ntpd[17571]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:20.874 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:20 ntpd[17571]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:20.874 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:20 ntpd[17571]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:20.874 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:20 ntpd[17571]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:20.880 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:20 ntpd[17641]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:20.881 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:20 ntpd[17641]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:20.881 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:20 ntpd[17641]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:20.882 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:20 ntpd[17641]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:20.882 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:20 ntpd[17641]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:20.883 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:20 ntpd[17641]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:20.918 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:20 ntpd[17641]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:20.918 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:20 ntpd[17641]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:20.918 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:20 ntpd[17641]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:20.921 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:20 ntpd[17641]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:20.922 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:20 ntpd[17641]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:20.922 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:20 ntpd[17641]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:21.586 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:21 ntpd[17513]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:21.596 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:21 ntpd[17513]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:21.614 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:21 ntpd[17513]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:21.614 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:21 ntpd[17513]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:21.615 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:21 ntpd[17513]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:21.615 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:21 ntpd[17513]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:21.633 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:21 ntpd[17566]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:21.633 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:21 ntpd[17566]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:21.633 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:21 ntpd[17566]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:21.634 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:21 ntpd[17566]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:21.634 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:21 ntpd[17566]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:21.636 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:21 ntpd[17566]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:21.637 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:21 ntpd[17566]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:21.637 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:21 ntpd[17566]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:21.811 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:21 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:21.811 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:21 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:21.823 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:21 ntpd[17571]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:21.824 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:21 ntpd[17571]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:21.827 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:21 ntpd[17571]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:21.827 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:21 ntpd[17571]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:21.828 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:21 ntpd[17571]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:21.828 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:21 ntpd[17571]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:21.829 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:21 ntpd[17571]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:21.829 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:21 ntpd[17571]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:21.829 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:21 ntpd[17571]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:21.830 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:21 ntpd[17571]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:21.842 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:21 ntpd[17571]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:21.843 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:21 ntpd[17571]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:21.844 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:21 ntpd[17641]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:21.845 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:21 ntpd[17641]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:21.845 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:21 ntpd[17641]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:21.846 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:21 ntpd[17641]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:21.846 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:21 ntpd[17641]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:21.846 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:21 ntpd[17641]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:22.590 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:22 ntpd[17513]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:22.591 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:22 ntpd[17513]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:22.591 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:22 ntpd[17513]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:22.591 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:22 ntpd[17513]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:22.592 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:22 ntpd[17513]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:22.592 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:22 ntpd[17513]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:22.592 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:22 ntpd[17513]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:22.592 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:22 ntpd[17513]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:22.593 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:22 ntpd[17513]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:22.593 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:22 ntpd[17513]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:22.593 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:22 ntpd[17513]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:22.593 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:22 ntpd[17513]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:22.594 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:22 ntpd[17513]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:22.594 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:22 ntpd[17513]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:22.637 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:22 ntpd[17566]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:22.637 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:22 ntpd[17566]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:22.638 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:22 ntpd[17566]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:22.638 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:22 ntpd[17566]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:22.639 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:22 ntpd[17566]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:22.639 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:22 ntpd[17566]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:22.640 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:22 ntpd[17566]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:22.641 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:22 ntpd[17566]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:22.642 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:22 ntpd[17566]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:22.642 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:22 ntpd[17566]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:22.642 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:22 ntpd[17566]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:22.642 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:22 ntpd[17566]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:22.825 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:22 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:22.829 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:22 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:22.830 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:22 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:22.830 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:22 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:22.831 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:22 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:22.831 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:22 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:22.831 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:22 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:22.832 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:22 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:22.833 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:22 ntpd[17601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:22.833 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:22 ntpd[17601]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:22.834 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:22 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:22.835 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:22 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:22.835 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:22 ntpd[17601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:22.836 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:22 ntpd[17601]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:22.836 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:22 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:22.836 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:22 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:22.839 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:22 ntpd[17601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:22.840 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:22 ntpd[17601]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:22.843 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:22 ntpd[17571]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:22.844 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:22 ntpd[17571]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:22.845 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:22 ntpd[17571]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:22.845 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:22 ntpd[17571]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:22.845 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:22 ntpd[17571]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:22.867 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:22 ntpd[17571]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:22.869 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:22 ntpd[17641]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:22.870 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:22 ntpd[17641]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:22.871 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:22 ntpd[17641]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:22.871 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:22 ntpd[17641]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:22.872 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:22 ntpd[17641]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:22.884 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:22 ntpd[17641]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:22.885 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:22 ntpd[17641]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:22.885 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:22 ntpd[17641]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:22.886 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:22 ntpd[17641]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:22.886 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:22 ntpd[17641]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:22.887 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:22 ntpd[17641]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:22.887 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:22 ntpd[17641]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:23.587 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:23 ntpd[17513]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:23.588 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:23 ntpd[17513]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:23.588 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:23 ntpd[17513]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:23.588 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:23 ntpd[17513]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:23.589 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:23 ntpd[17513]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:23.589 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:23 ntpd[17513]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:23.629 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:23 ntpd[17566]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:23.630 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:23 ntpd[17566]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:23.630 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:23 ntpd[17566]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:23.630 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:23 ntpd[17566]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:23.631 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:23 ntpd[17566]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:23.631 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:23 ntpd[17566]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:23.825 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:23 ntpd[17571]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:23.832 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:23 ntpd[17571]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:23.832 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:23 ntpd[17571]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:23.833 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:23 ntpd[17571]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:23.833 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:23 ntpd[17571]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:23.833 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:23 ntpd[17571]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:23.834 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:23 ntpd[17571]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:23.834 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:23 ntpd[17571]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:23.835 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:23 ntpd[17571]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:23.835 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:23 ntpd[17571]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:23.835 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:23 ntpd[17571]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:23.839 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:23 ntpd[17571]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:23.840 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:23 ntpd[17641]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:23.840 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:23 ntpd[17641]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:23.841 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:23 ntpd[17641]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:23.842 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:23 ntpd[17641]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:23.842 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:23 ntpd[17641]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:23.842 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:23 ntpd[17641]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:24.583 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:24 ntpd[17513]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:24.583 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:24 ntpd[17513]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:24.584 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:24 ntpd[17513]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:24.584 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:24 ntpd[17513]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:24.587 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:24 ntpd[17513]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:24.588 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:24 ntpd[17513]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:24.588 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:24 ntpd[17513]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:24.588 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:24 ntpd[17513]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:24.594 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:24 ntpd[17513]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:24.595 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:24 ntpd[17513]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:24.595 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:24 ntpd[17513]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:24.596 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:24 ntpd[17513]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:24.596 INFO:teuthology.orchestra.run.smithi036.stdout:17 Jun 12:43:24 ntpd[17513]: ntpd: time slew +0.005752 s 2022-06-17T12:43:24.597 INFO:teuthology.orchestra.run.smithi036.stdout:ntpd: time slew +0.005752s 2022-06-17T12:43:24.605 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:24 ntpd[17513]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2022-06-17T12:43:24.605 INFO:teuthology.orchestra.run.smithi036.stderr:17 Jun 12:43:24 ntpd[17513]: can't open /var/log/ntpstats/loopstats.20220617: Permission denied 2022-06-17T12:43:24.655 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:24 ntpd[17566]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:24.657 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:24 ntpd[17566]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:24.658 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:24 ntpd[17566]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:24.659 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:24 ntpd[17566]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:24.660 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:24 ntpd[17566]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:24.660 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:24 ntpd[17566]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:24.660 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:24 ntpd[17566]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:24.660 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:24 ntpd[17566]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:24.660 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:24 ntpd[17566]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:24.661 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:24 ntpd[17566]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:24.661 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:24 ntpd[17566]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:24.661 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:24 ntpd[17566]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:24.662 INFO:teuthology.orchestra.run.smithi070.stdout:17 Jun 12:43:24 ntpd[17566]: ntpd: time slew +0.003348 s 2022-06-17T12:43:24.662 INFO:teuthology.orchestra.run.smithi070.stdout:ntpd: time slew +0.003348s 2022-06-17T12:43:24.663 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:24 ntpd[17566]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2022-06-17T12:43:24.663 INFO:teuthology.orchestra.run.smithi070.stderr:17 Jun 12:43:24 ntpd[17566]: can't open /var/log/ntpstats/loopstats.20220617: Permission denied 2022-06-17T12:43:24.665 INFO:teuthology.orchestra.run.smithi036.stdout: remote refid st t when poll reach delay offset jitter 2022-06-17T12:43:24.684 INFO:teuthology.orchestra.run.smithi036.stdout:============================================================================== 2022-06-17T12:43:24.684 INFO:teuthology.orchestra.run.smithi036.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-17T12:43:24.685 INFO:teuthology.orchestra.run.smithi036.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-17T12:43:24.685 INFO:teuthology.orchestra.run.smithi036.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-17T12:43:24.686 INFO:teuthology.orchestra.run.smithi036.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-17T12:43:24.693 INFO:teuthology.orchestra.run.smithi070.stdout: remote refid st t when poll reach delay offset jitter 2022-06-17T12:43:24.694 INFO:teuthology.orchestra.run.smithi070.stdout:============================================================================== 2022-06-17T12:43:24.695 INFO:teuthology.orchestra.run.smithi070.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-17T12:43:24.695 INFO:teuthology.orchestra.run.smithi070.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-17T12:43:24.695 INFO:teuthology.orchestra.run.smithi070.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-17T12:43:24.701 INFO:teuthology.orchestra.run.smithi070.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-17T12:43:24.811 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:24 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:24.811 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:24 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:24.812 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:24 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:24.812 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:24 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:24.812 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:24 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:24.813 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:24 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:24.813 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:24 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:24.824 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:24 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:24.825 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:24 ntpd[17601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:24.840 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:24 ntpd[17601]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:24.844 INFO:teuthology.orchestra.run.smithi107.stdout:17 Jun 12:43:24 ntpd[17601]: ntpd: time slew -0.000692 s 2022-06-17T12:43:24.845 INFO:teuthology.orchestra.run.smithi107.stdout:ntpd: time slew -0.000692s 2022-06-17T12:43:24.846 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:24 ntpd[17601]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2022-06-17T12:43:24.847 INFO:teuthology.orchestra.run.smithi107.stderr:17 Jun 12:43:24 ntpd[17601]: can't open /var/log/ntpstats/loopstats.20220617: Permission denied 2022-06-17T12:43:24.863 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:24 ntpd[17571]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:24.871 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:24 ntpd[17571]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:24.881 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:24 ntpd[17571]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:24.882 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:24 ntpd[17571]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:24.882 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:24 ntpd[17571]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:24.884 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:24 ntpd[17571]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:24.930 INFO:teuthology.orchestra.run.smithi136.stdout:17 Jun 12:43:24 ntpd[17571]: ntpd: time slew -0.002204 s 2022-06-17T12:43:24.974 INFO:teuthology.orchestra.run.smithi136.stdout:ntpd: time slew -0.002204s 2022-06-17T12:43:24.976 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:24 ntpd[17571]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2022-06-17T12:43:25.010 INFO:teuthology.orchestra.run.smithi136.stderr:17 Jun 12:43:24 ntpd[17571]: can't open /var/log/ntpstats/loopstats.20220617: Permission denied 2022-06-17T12:43:25.012 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:24 ntpd[17641]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:25.012 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:24 ntpd[17641]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:25.013 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:24 ntpd[17641]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:25.029 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:24 ntpd[17641]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:25.029 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:24 ntpd[17641]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-17T12:43:25.030 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:24 ntpd[17641]: can't open /var/log/ntpstats/rawstats.20220617: Permission denied 2022-06-17T12:43:25.030 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:24 ntpd[17641]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-17T12:43:25.036 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:24 ntpd[17641]: can't open /var/log/ntpstats/peerstats.20220617: Permission denied 2022-06-17T12:43:25.040 INFO:teuthology.orchestra.run.smithi143.stdout:17 Jun 12:43:24 ntpd[17641]: ntpd: time slew -0.001554 s 2022-06-17T12:43:25.041 INFO:teuthology.orchestra.run.smithi143.stdout:ntpd: time slew -0.001554s 2022-06-17T12:43:25.043 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:24 ntpd[17641]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2022-06-17T12:43:25.043 INFO:teuthology.orchestra.run.smithi143.stderr:17 Jun 12:43:24 ntpd[17641]: can't open /var/log/ntpstats/loopstats.20220617: Permission denied 2022-06-17T12:43:25.044 INFO:teuthology.orchestra.run.smithi136.stdout: remote refid st t when poll reach delay offset jitter 2022-06-17T12:43:25.044 INFO:teuthology.orchestra.run.smithi136.stdout:============================================================================== 2022-06-17T12:43:25.045 INFO:teuthology.orchestra.run.smithi136.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-17T12:43:25.046 INFO:teuthology.orchestra.run.smithi136.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-17T12:43:25.046 INFO:teuthology.orchestra.run.smithi136.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-17T12:43:25.046 INFO:teuthology.orchestra.run.smithi136.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-17T12:43:25.049 INFO:teuthology.orchestra.run.smithi107.stdout: remote refid st t when poll reach delay offset jitter 2022-06-17T12:43:25.049 INFO:teuthology.orchestra.run.smithi107.stdout:============================================================================== 2022-06-17T12:43:25.049 INFO:teuthology.orchestra.run.smithi107.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-17T12:43:25.050 INFO:teuthology.orchestra.run.smithi107.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-17T12:43:25.051 INFO:teuthology.orchestra.run.smithi107.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-17T12:43:25.051 INFO:teuthology.orchestra.run.smithi107.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-17T12:43:25.055 INFO:teuthology.orchestra.run.smithi143.stdout: remote refid st t when poll reach delay offset jitter 2022-06-17T12:43:25.056 INFO:teuthology.orchestra.run.smithi143.stdout:============================================================================== 2022-06-17T12:43:25.056 INFO:teuthology.orchestra.run.smithi143.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-17T12:43:25.057 INFO:teuthology.orchestra.run.smithi143.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-17T12:43:25.057 INFO:teuthology.orchestra.run.smithi143.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-17T12:43:25.057 INFO:teuthology.orchestra.run.smithi143.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-17T12:43:25.070 INFO:teuthology.run_tasks:Running task install... 2022-06-17T12:43:25.089 DEBUG:teuthology.task.install:project ceph 2022-06-17T12:43:25.091 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6'}} 2022-06-17T12:43:25.092 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': 'b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6'} 2022-06-17T12:43:25.093 INFO:teuthology.task.install:Using flavor: default 2022-06-17T12:43:25.243 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2022-06-17T12:43:25.244 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']} 2022-06-17T12:43:25.244 DEBUG:teuthology.orchestra.run.smithi036:> sudo lsb_release -is 2022-06-17T12:43:25.307 INFO:teuthology.orchestra.run.smithi036.stdout:Ubuntu 2022-06-17T12:43:25.308 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-17T12:43:25.339 DEBUG:teuthology.orchestra.run.smithi070:> sudo lsb_release -is 2022-06-17T12:43:25.341 DEBUG:teuthology.orchestra.run.smithi036:> sudo apt-key list | grep Ceph 2022-06-17T12:43:25.418 INFO:teuthology.orchestra.run.smithi070.stdout:Ubuntu 2022-06-17T12:43:25.419 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-17T12:43:25.420 DEBUG:teuthology.orchestra.run.smithi107:> sudo lsb_release -is 2022-06-17T12:43:25.421 DEBUG:teuthology.orchestra.run.smithi070:> sudo apt-key list | grep Ceph 2022-06-17T12:43:25.490 INFO:teuthology.orchestra.run.smithi036.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2022-06-17T12:43:25.494 INFO:teuthology.orchestra.run.smithi107.stdout:Ubuntu 2022-06-17T12:43:25.495 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-17T12:43:25.497 DEBUG:teuthology.orchestra.run.smithi136:> sudo lsb_release -is 2022-06-17T12:43:25.498 DEBUG:teuthology.orchestra.run.smithi107:> sudo apt-key list | grep Ceph 2022-06-17T12:43:25.547 INFO:teuthology.orchestra.run.smithi036.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2022-06-17T12:43:25.548 INFO:teuthology.orchestra.run.smithi036.stdout:uid [ unknown] Ceph.com (release key) 2022-06-17T12:43:25.554 DEBUG:teuthology.orchestra.run.smithi036:> sudo lsb_release -is 2022-06-17T12:43:25.558 INFO:teuthology.orchestra.run.smithi136.stdout:Ubuntu 2022-06-17T12:43:25.559 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-17T12:43:25.570 DEBUG:teuthology.orchestra.run.smithi143:> sudo lsb_release -is 2022-06-17T12:43:25.580 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-key list | grep Ceph 2022-06-17T12:43:25.599 INFO:teuthology.orchestra.run.smithi070.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2022-06-17T12:43:25.608 INFO:teuthology.orchestra.run.smithi107.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2022-06-17T12:43:25.645 INFO:teuthology.orchestra.run.smithi143.stdout:Ubuntu 2022-06-17T12:43:25.645 INFO:teuthology.orchestra.run.smithi036.stdout:Ubuntu 2022-06-17T12:43:25.654 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-17T12:43:25.655 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-17T12:43:25.656 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2022-06-17T12:43:25.657 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2022-06-17T12:43:25.658 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6 2022-06-17T12:43:25.676 DEBUG:teuthology.orchestra.run.smithi143:> sudo apt-key list | grep Ceph 2022-06-17T12:43:25.685 INFO:teuthology.orchestra.run.smithi107.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2022-06-17T12:43:25.686 INFO:teuthology.orchestra.run.smithi107.stdout:uid [ unknown] Ceph.com (release key) 2022-06-17T12:43:25.686 INFO:teuthology.orchestra.run.smithi070.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2022-06-17T12:43:25.687 INFO:teuthology.orchestra.run.smithi070.stdout:uid [ unknown] Ceph.com (release key) 2022-06-17T12:43:25.687 DEBUG:teuthology.orchestra.run.smithi107:> sudo lsb_release -is 2022-06-17T12:43:25.690 DEBUG:teuthology.orchestra.run.smithi070:> sudo lsb_release -is 2022-06-17T12:43:25.692 INFO:teuthology.orchestra.run.smithi136.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2022-06-17T12:43:25.763 INFO:teuthology.orchestra.run.smithi136.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2022-06-17T12:43:25.763 INFO:teuthology.orchestra.run.smithi136.stdout:uid [ unknown] Ceph.com (release key) 2022-06-17T12:43:25.764 INFO:teuthology.orchestra.run.smithi070.stdout:Ubuntu 2022-06-17T12:43:25.764 INFO:teuthology.orchestra.run.smithi107.stdout:Ubuntu 2022-06-17T12:43:25.765 DEBUG:teuthology.orchestra.run.smithi136:> sudo lsb_release -is 2022-06-17T12:43:25.767 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-17T12:43:25.768 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2022-06-17T12:43:25.779 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2022-06-17T12:43:25.780 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6 2022-06-17T12:43:25.788 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-17T12:43:25.789 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2022-06-17T12:43:25.790 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2022-06-17T12:43:25.791 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6 2022-06-17T12:43:25.808 INFO:teuthology.orchestra.run.smithi143.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2022-06-17T12:43:25.919 INFO:teuthology.orchestra.run.smithi136.stdout:Ubuntu 2022-06-17T12:43:25.920 INFO:teuthology.orchestra.run.smithi143.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2022-06-17T12:43:25.920 INFO:teuthology.orchestra.run.smithi143.stdout:uid [ unknown] Ceph.com (release key) 2022-06-17T12:43:25.920 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-17T12:43:25.921 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2022-06-17T12:43:25.921 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2022-06-17T12:43:25.922 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6 2022-06-17T12:43:25.933 DEBUG:teuthology.orchestra.run.smithi143:> sudo lsb_release -is 2022-06-17T12:43:26.032 INFO:teuthology.orchestra.run.smithi143.stdout:Ubuntu 2022-06-17T12:43:26.032 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-17T12:43:26.033 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2022-06-17T12:43:26.034 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2022-06-17T12:43:26.035 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6 2022-06-17T12:43:26.057 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default/ 2022-06-17T12:43:26.058 INFO:teuthology.task.install.deb:Package version is 15.2.16-260-gb3641a55-1focal 2022-06-17T12:43:26.068 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default/ 2022-06-17T12:43:26.069 INFO:teuthology.task.install.deb:Package version is 15.2.16-260-gb3641a55-1focal 2022-06-17T12:43:26.218 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default/ 2022-06-17T12:43:26.219 INFO:teuthology.task.install.deb:Package version is 15.2.16-260-gb3641a55-1focal 2022-06-17T12:43:26.300 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default/ 2022-06-17T12:43:26.301 INFO:teuthology.task.install.deb:Package version is 15.2.16-260-gb3641a55-1focal 2022-06-17T12:43:26.371 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-06-17T12:43:26.373 DEBUG:teuthology.orchestra.run.smithi036:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2022-06-17T12:43:26.374 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2022-06-17T12:43:26.375 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2022-06-17T12:43:26.401 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2022-06-17T12:43:26.402 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2022-06-17T12:43:26.426 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default/ 2022-06-17T12:43:26.427 INFO:teuthology.task.install.deb:Package version is 15.2.16-260-gb3641a55-1focal 2022-06-17T12:43:26.469 DEBUG:teuthology.orchestra.run.smithi107:> sudo apt-get update 2022-06-17T12:43:26.469 DEBUG:teuthology.orchestra.run.smithi036:> sudo apt-get update 2022-06-17T12:43:26.472 DEBUG:teuthology.orchestra.run.smithi070:> sudo apt-get update 2022-06-17T12:43:26.552 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2022-06-17T12:43:26.555 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2022-06-17T12:43:26.583 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get update 2022-06-17T12:43:26.615 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-17T12:43:26.616 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-06-17T12:43:26.640 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-06-17T12:43:26.700 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2022-06-17T12:43:26.702 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2022-06-17T12:43:26.704 INFO:teuthology.orchestra.run.smithi036.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-17T12:43:26.705 INFO:teuthology.orchestra.run.smithi070.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-17T12:43:26.706 INFO:teuthology.orchestra.run.smithi107.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal InRelease 2022-06-17T12:43:26.707 INFO:teuthology.orchestra.run.smithi070.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-06-17T12:43:26.724 INFO:teuthology.orchestra.run.smithi036.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-06-17T12:43:26.725 INFO:teuthology.orchestra.run.smithi107.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal Release [7,680 B] 2022-06-17T12:43:26.730 INFO:teuthology.orchestra.run.smithi070.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-06-17T12:43:26.731 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:6 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-06-17T12:43:26.733 DEBUG:teuthology.orchestra.run.smithi143:> sudo apt-get update 2022-06-17T12:43:26.735 INFO:teuthology.orchestra.run.smithi107.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal Release.gpg 2022-06-17T12:43:26.744 INFO:teuthology.orchestra.run.smithi036.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-06-17T12:43:26.757 INFO:teuthology.orchestra.run.smithi036.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal InRelease 2022-06-17T12:43:26.759 INFO:teuthology.orchestra.run.smithi107.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main i386 Packages [3,067 B] 2022-06-17T12:43:26.763 INFO:teuthology.orchestra.run.smithi070.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal InRelease 2022-06-17T12:43:26.784 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-17T12:43:26.784 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-06-17T12:43:26.786 INFO:teuthology.orchestra.run.smithi070.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal Release [7,680 B] 2022-06-17T12:43:26.788 INFO:teuthology.orchestra.run.smithi036.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal Release [7,680 B] 2022-06-17T12:43:26.791 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-06-17T12:43:26.802 INFO:teuthology.orchestra.run.smithi107.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2022-06-17T12:43:26.806 INFO:teuthology.orchestra.run.smithi036.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal Release.gpg 2022-06-17T12:43:26.808 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-06-17T12:43:26.811 INFO:teuthology.orchestra.run.smithi070.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal Release.gpg 2022-06-17T12:43:26.815 INFO:teuthology.orchestra.run.smithi070.stdout:Hit:7 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-06-17T12:43:26.827 INFO:teuthology.orchestra.run.smithi036.stdout:Hit:7 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-06-17T12:43:26.839 INFO:teuthology.orchestra.run.smithi070.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main i386 Packages [3,067 B] 2022-06-17T12:43:26.840 INFO:teuthology.orchestra.run.smithi036.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main i386 Packages [3,067 B] 2022-06-17T12:43:26.850 INFO:teuthology.orchestra.run.smithi136.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal InRelease 2022-06-17T12:43:26.867 INFO:teuthology.orchestra.run.smithi136.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal Release [7,680 B] 2022-06-17T12:43:26.868 INFO:teuthology.orchestra.run.smithi036.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2022-06-17T12:43:26.877 INFO:teuthology.orchestra.run.smithi070.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2022-06-17T12:43:26.891 INFO:teuthology.orchestra.run.smithi136.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal Release.gpg 2022-06-17T12:43:26.919 INFO:teuthology.orchestra.run.smithi136.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2022-06-17T12:43:26.949 INFO:teuthology.orchestra.run.smithi136.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main i386 Packages [3,067 B] 2022-06-17T12:43:26.997 INFO:teuthology.orchestra.run.smithi143.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal InRelease 2022-06-17T12:43:27.024 INFO:teuthology.orchestra.run.smithi143.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal Release [7,680 B] 2022-06-17T12:43:27.054 INFO:teuthology.orchestra.run.smithi143.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal Release.gpg 2022-06-17T12:43:27.071 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-06-17T12:43:27.071 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-17T12:43:27.090 INFO:teuthology.orchestra.run.smithi143.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main i386 Packages [3,067 B] 2022-06-17T12:43:27.114 INFO:teuthology.orchestra.run.smithi143.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2022-06-17T12:43:27.135 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-06-17T12:43:27.212 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-06-17T12:43:27.282 INFO:teuthology.orchestra.run.smithi107.stdout:Fetched 26.5 kB in 1s (41.3 kB/s) 2022-06-17T12:43:27.346 INFO:teuthology.orchestra.run.smithi070.stdout:Fetched 26.5 kB in 1s (42.7 kB/s) 2022-06-17T12:43:27.358 INFO:teuthology.orchestra.run.smithi036.stdout:Fetched 26.5 kB in 1s (41.3 kB/s) 2022-06-17T12:43:27.441 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 26.5 kB in 1s (42.0 kB/s) 2022-06-17T12:43:27.711 INFO:teuthology.orchestra.run.smithi143.stdout:Fetched 26.5 kB in 1s (34.9 kB/s) 2022-06-17T12:43:28.156 INFO:teuthology.orchestra.run.smithi107.stdout:Reading package lists... 2022-06-17T12:43:28.188 DEBUG:teuthology.orchestra.run.smithi107:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.16-260-gb3641a55-1focal cephadm=15.2.16-260-gb3641a55-1focal ceph-mds=15.2.16-260-gb3641a55-1focal ceph-mgr=15.2.16-260-gb3641a55-1focal ceph-common=15.2.16-260-gb3641a55-1focal ceph-fuse=15.2.16-260-gb3641a55-1focal ceph-test=15.2.16-260-gb3641a55-1focal radosgw=15.2.16-260-gb3641a55-1focal python3-rados=15.2.16-260-gb3641a55-1focal python3-rgw=15.2.16-260-gb3641a55-1focal python3-cephfs=15.2.16-260-gb3641a55-1focal python3-rbd=15.2.16-260-gb3641a55-1focal libcephfs2=15.2.16-260-gb3641a55-1focal libcephfs-dev=15.2.16-260-gb3641a55-1focal librados2=15.2.16-260-gb3641a55-1focal librbd1=15.2.16-260-gb3641a55-1focal rbd-fuse=15.2.16-260-gb3641a55-1focal python3-cephfs=15.2.16-260-gb3641a55-1focal cephfs-shell=15.2.16-260-gb3641a55-1focal 2022-06-17T12:43:28.220 INFO:teuthology.orchestra.run.smithi070.stdout:Reading package lists... 2022-06-17T12:43:28.247 DEBUG:teuthology.orchestra.run.smithi070:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.16-260-gb3641a55-1focal cephadm=15.2.16-260-gb3641a55-1focal ceph-mds=15.2.16-260-gb3641a55-1focal ceph-mgr=15.2.16-260-gb3641a55-1focal ceph-common=15.2.16-260-gb3641a55-1focal ceph-fuse=15.2.16-260-gb3641a55-1focal ceph-test=15.2.16-260-gb3641a55-1focal radosgw=15.2.16-260-gb3641a55-1focal python3-rados=15.2.16-260-gb3641a55-1focal python3-rgw=15.2.16-260-gb3641a55-1focal python3-cephfs=15.2.16-260-gb3641a55-1focal python3-rbd=15.2.16-260-gb3641a55-1focal libcephfs2=15.2.16-260-gb3641a55-1focal libcephfs-dev=15.2.16-260-gb3641a55-1focal librados2=15.2.16-260-gb3641a55-1focal librbd1=15.2.16-260-gb3641a55-1focal rbd-fuse=15.2.16-260-gb3641a55-1focal python3-cephfs=15.2.16-260-gb3641a55-1focal cephfs-shell=15.2.16-260-gb3641a55-1focal 2022-06-17T12:43:28.280 INFO:teuthology.orchestra.run.smithi107.stdout:Reading package lists... 2022-06-17T12:43:28.282 INFO:teuthology.orchestra.run.smithi036.stdout:Reading package lists... 2022-06-17T12:43:28.295 DEBUG:teuthology.orchestra.run.smithi036:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.16-260-gb3641a55-1focal cephadm=15.2.16-260-gb3641a55-1focal ceph-mds=15.2.16-260-gb3641a55-1focal ceph-mgr=15.2.16-260-gb3641a55-1focal ceph-common=15.2.16-260-gb3641a55-1focal ceph-fuse=15.2.16-260-gb3641a55-1focal ceph-test=15.2.16-260-gb3641a55-1focal radosgw=15.2.16-260-gb3641a55-1focal python3-rados=15.2.16-260-gb3641a55-1focal python3-rgw=15.2.16-260-gb3641a55-1focal python3-cephfs=15.2.16-260-gb3641a55-1focal python3-rbd=15.2.16-260-gb3641a55-1focal libcephfs2=15.2.16-260-gb3641a55-1focal libcephfs-dev=15.2.16-260-gb3641a55-1focal librados2=15.2.16-260-gb3641a55-1focal librbd1=15.2.16-260-gb3641a55-1focal rbd-fuse=15.2.16-260-gb3641a55-1focal python3-cephfs=15.2.16-260-gb3641a55-1focal cephfs-shell=15.2.16-260-gb3641a55-1focal 2022-06-17T12:43:28.313 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2022-06-17T12:43:28.329 INFO:teuthology.orchestra.run.smithi070.stdout:Reading package lists... 2022-06-17T12:43:28.342 DEBUG:teuthology.orchestra.run.smithi136:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.16-260-gb3641a55-1focal cephadm=15.2.16-260-gb3641a55-1focal ceph-mds=15.2.16-260-gb3641a55-1focal ceph-mgr=15.2.16-260-gb3641a55-1focal ceph-common=15.2.16-260-gb3641a55-1focal ceph-fuse=15.2.16-260-gb3641a55-1focal ceph-test=15.2.16-260-gb3641a55-1focal radosgw=15.2.16-260-gb3641a55-1focal python3-rados=15.2.16-260-gb3641a55-1focal python3-rgw=15.2.16-260-gb3641a55-1focal python3-cephfs=15.2.16-260-gb3641a55-1focal python3-rbd=15.2.16-260-gb3641a55-1focal libcephfs2=15.2.16-260-gb3641a55-1focal libcephfs-dev=15.2.16-260-gb3641a55-1focal librados2=15.2.16-260-gb3641a55-1focal librbd1=15.2.16-260-gb3641a55-1focal rbd-fuse=15.2.16-260-gb3641a55-1focal python3-cephfs=15.2.16-260-gb3641a55-1focal cephfs-shell=15.2.16-260-gb3641a55-1focal 2022-06-17T12:43:28.369 INFO:teuthology.orchestra.run.smithi036.stdout:Reading package lists... 2022-06-17T12:43:28.469 INFO:teuthology.orchestra.run.smithi107.stdout:Building dependency tree... 2022-06-17T12:43:28.470 INFO:teuthology.orchestra.run.smithi107.stdout:Reading state information... 2022-06-17T12:43:28.471 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2022-06-17T12:43:28.493 INFO:teuthology.orchestra.run.smithi070.stdout:Building dependency tree... 2022-06-17T12:43:28.494 INFO:teuthology.orchestra.run.smithi070.stdout:Reading state information... 2022-06-17T12:43:28.549 INFO:teuthology.orchestra.run.smithi036.stdout:Building dependency tree... 2022-06-17T12:43:28.550 INFO:teuthology.orchestra.run.smithi036.stdout:Reading state information... 2022-06-17T12:43:28.592 INFO:teuthology.orchestra.run.smithi143.stdout:Reading package lists... 2022-06-17T12:43:28.612 INFO:teuthology.orchestra.run.smithi107.stdout:The following packages were automatically installed and are no longer required: 2022-06-17T12:43:28.612 INFO:teuthology.orchestra.run.smithi107.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-06-17T12:43:28.640 INFO:teuthology.orchestra.run.smithi107.stdout:Use 'sudo apt autoremove' to remove them. 2022-06-17T12:43:28.640 INFO:teuthology.orchestra.run.smithi107.stdout:The following additional packages will be installed: 2022-06-17T12:43:28.641 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-06-17T12:43:28.641 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-06-17T12:43:28.641 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2022-06-17T12:43:28.642 INFO:teuthology.orchestra.run.smithi107.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2022-06-17T12:43:28.642 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2022-06-17T12:43:28.643 INFO:teuthology.orchestra.run.smithi107.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2022-06-17T12:43:28.643 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2022-06-17T12:43:28.644 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2022-06-17T12:43:28.644 INFO:teuthology.orchestra.run.smithi107.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2022-06-17T12:43:28.645 INFO:teuthology.orchestra.run.smithi107.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2022-06-17T12:43:28.645 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2022-06-17T12:43:28.645 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2022-06-17T12:43:28.646 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2022-06-17T12:43:28.647 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2022-06-17T12:43:28.647 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2022-06-17T12:43:28.648 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2022-06-17T12:43:28.648 INFO:teuthology.orchestra.run.smithi107.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2022-06-17T12:43:28.648 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2022-06-17T12:43:28.649 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2022-06-17T12:43:28.649 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet 2022-06-17T12:43:28.650 INFO:teuthology.orchestra.run.smithi107.stdout:Suggested packages: 2022-06-17T12:43:28.650 INFO:teuthology.orchestra.run.smithi107.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2022-06-17T12:43:28.651 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2022-06-17T12:43:28.651 INFO:teuthology.orchestra.run.smithi107.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2022-06-17T12:43:28.651 INFO:teuthology.orchestra.run.smithi107.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2022-06-17T12:43:28.652 INFO:teuthology.orchestra.run.smithi107.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2022-06-17T12:43:28.652 INFO:teuthology.orchestra.run.smithi107.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2022-06-17T12:43:28.653 INFO:teuthology.orchestra.run.smithi107.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2022-06-17T12:43:28.653 INFO:teuthology.orchestra.run.smithi107.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2022-06-17T12:43:28.654 INFO:teuthology.orchestra.run.smithi107.stdout:Recommended packages: 2022-06-17T12:43:28.654 INFO:teuthology.orchestra.run.smithi107.stdout: btrfs-tools 2022-06-17T12:43:28.658 INFO:teuthology.orchestra.run.smithi136.stdout:Building dependency tree... 2022-06-17T12:43:28.658 INFO:teuthology.orchestra.run.smithi136.stdout:Reading state information... 2022-06-17T12:43:28.661 DEBUG:teuthology.orchestra.run.smithi143:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.16-260-gb3641a55-1focal cephadm=15.2.16-260-gb3641a55-1focal ceph-mds=15.2.16-260-gb3641a55-1focal ceph-mgr=15.2.16-260-gb3641a55-1focal ceph-common=15.2.16-260-gb3641a55-1focal ceph-fuse=15.2.16-260-gb3641a55-1focal ceph-test=15.2.16-260-gb3641a55-1focal radosgw=15.2.16-260-gb3641a55-1focal python3-rados=15.2.16-260-gb3641a55-1focal python3-rgw=15.2.16-260-gb3641a55-1focal python3-cephfs=15.2.16-260-gb3641a55-1focal python3-rbd=15.2.16-260-gb3641a55-1focal libcephfs2=15.2.16-260-gb3641a55-1focal libcephfs-dev=15.2.16-260-gb3641a55-1focal librados2=15.2.16-260-gb3641a55-1focal librbd1=15.2.16-260-gb3641a55-1focal rbd-fuse=15.2.16-260-gb3641a55-1focal python3-cephfs=15.2.16-260-gb3641a55-1focal cephfs-shell=15.2.16-260-gb3641a55-1focal 2022-06-17T12:43:28.665 INFO:teuthology.orchestra.run.smithi070.stdout:The following packages were automatically installed and are no longer required: 2022-06-17T12:43:28.666 INFO:teuthology.orchestra.run.smithi070.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-06-17T12:43:28.666 INFO:teuthology.orchestra.run.smithi070.stdout:Use 'sudo apt autoremove' to remove them. 2022-06-17T12:43:28.667 INFO:teuthology.orchestra.run.smithi070.stdout:The following additional packages will be installed: 2022-06-17T12:43:28.667 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-06-17T12:43:28.668 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-06-17T12:43:28.668 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2022-06-17T12:43:28.668 INFO:teuthology.orchestra.run.smithi070.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2022-06-17T12:43:28.669 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2022-06-17T12:43:28.669 INFO:teuthology.orchestra.run.smithi070.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2022-06-17T12:43:28.672 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2022-06-17T12:43:28.673 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2022-06-17T12:43:28.673 INFO:teuthology.orchestra.run.smithi070.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2022-06-17T12:43:28.673 INFO:teuthology.orchestra.run.smithi070.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2022-06-17T12:43:28.694 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2022-06-17T12:43:28.703 INFO:teuthology.orchestra.run.smithi070.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2022-06-17T12:43:28.703 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2022-06-17T12:43:28.704 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2022-06-17T12:43:28.704 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2022-06-17T12:43:28.704 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2022-06-17T12:43:28.705 INFO:teuthology.orchestra.run.smithi070.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2022-06-17T12:43:28.705 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2022-06-17T12:43:28.714 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2022-06-17T12:43:28.715 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet 2022-06-17T12:43:28.715 INFO:teuthology.orchestra.run.smithi070.stdout:Suggested packages: 2022-06-17T12:43:28.722 INFO:teuthology.orchestra.run.smithi070.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2022-06-17T12:43:28.739 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2022-06-17T12:43:28.740 INFO:teuthology.orchestra.run.smithi070.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2022-06-17T12:43:28.740 INFO:teuthology.orchestra.run.smithi070.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2022-06-17T12:43:28.740 INFO:teuthology.orchestra.run.smithi070.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2022-06-17T12:43:28.741 INFO:teuthology.orchestra.run.smithi070.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2022-06-17T12:43:28.741 INFO:teuthology.orchestra.run.smithi070.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2022-06-17T12:43:28.742 INFO:teuthology.orchestra.run.smithi070.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2022-06-17T12:43:28.742 INFO:teuthology.orchestra.run.smithi070.stdout:Recommended packages: 2022-06-17T12:43:28.742 INFO:teuthology.orchestra.run.smithi070.stdout: btrfs-tools 2022-06-17T12:43:28.747 INFO:teuthology.orchestra.run.smithi143.stdout:Reading package lists... 2022-06-17T12:43:28.758 INFO:teuthology.orchestra.run.smithi036.stdout:The following packages were automatically installed and are no longer required: 2022-06-17T12:43:28.760 INFO:teuthology.orchestra.run.smithi036.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-06-17T12:43:28.771 INFO:teuthology.orchestra.run.smithi036.stdout:Use 'sudo apt autoremove' to remove them. 2022-06-17T12:43:28.771 INFO:teuthology.orchestra.run.smithi036.stdout:The following additional packages will be installed: 2022-06-17T12:43:28.772 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-06-17T12:43:28.772 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-06-17T12:43:28.772 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2022-06-17T12:43:28.773 INFO:teuthology.orchestra.run.smithi036.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2022-06-17T12:43:28.773 INFO:teuthology.orchestra.run.smithi036.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2022-06-17T12:43:28.773 INFO:teuthology.orchestra.run.smithi036.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2022-06-17T12:43:28.774 INFO:teuthology.orchestra.run.smithi036.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2022-06-17T12:43:28.774 INFO:teuthology.orchestra.run.smithi036.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2022-06-17T12:43:28.775 INFO:teuthology.orchestra.run.smithi036.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2022-06-17T12:43:28.775 INFO:teuthology.orchestra.run.smithi036.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2022-06-17T12:43:28.776 INFO:teuthology.orchestra.run.smithi036.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2022-06-17T12:43:28.776 INFO:teuthology.orchestra.run.smithi036.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2022-06-17T12:43:28.776 INFO:teuthology.orchestra.run.smithi036.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2022-06-17T12:43:28.777 INFO:teuthology.orchestra.run.smithi036.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2022-06-17T12:43:28.777 INFO:teuthology.orchestra.run.smithi036.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2022-06-17T12:43:28.778 INFO:teuthology.orchestra.run.smithi036.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2022-06-17T12:43:28.779 INFO:teuthology.orchestra.run.smithi036.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2022-06-17T12:43:28.779 INFO:teuthology.orchestra.run.smithi036.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2022-06-17T12:43:28.780 INFO:teuthology.orchestra.run.smithi036.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2022-06-17T12:43:28.780 INFO:teuthology.orchestra.run.smithi036.stdout: xmlstarlet 2022-06-17T12:43:28.782 INFO:teuthology.orchestra.run.smithi036.stdout:Suggested packages: 2022-06-17T12:43:28.783 INFO:teuthology.orchestra.run.smithi036.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2022-06-17T12:43:28.783 INFO:teuthology.orchestra.run.smithi036.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2022-06-17T12:43:28.784 INFO:teuthology.orchestra.run.smithi036.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2022-06-17T12:43:28.784 INFO:teuthology.orchestra.run.smithi036.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2022-06-17T12:43:28.785 INFO:teuthology.orchestra.run.smithi036.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2022-06-17T12:43:28.785 INFO:teuthology.orchestra.run.smithi036.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2022-06-17T12:43:28.786 INFO:teuthology.orchestra.run.smithi036.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2022-06-17T12:43:28.786 INFO:teuthology.orchestra.run.smithi036.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2022-06-17T12:43:28.786 INFO:teuthology.orchestra.run.smithi036.stdout:Recommended packages: 2022-06-17T12:43:28.787 INFO:teuthology.orchestra.run.smithi036.stdout: btrfs-tools 2022-06-17T12:43:28.792 INFO:teuthology.orchestra.run.smithi107.stdout:The following NEW packages will be installed: 2022-06-17T12:43:28.796 INFO:teuthology.orchestra.run.smithi107.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2022-06-17T12:43:28.796 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-06-17T12:43:28.796 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-06-17T12:43:28.797 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2022-06-17T12:43:28.797 INFO:teuthology.orchestra.run.smithi107.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2022-06-17T12:43:28.797 INFO:teuthology.orchestra.run.smithi107.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2022-06-17T12:43:28.797 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2022-06-17T12:43:28.798 INFO:teuthology.orchestra.run.smithi107.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2022-06-17T12:43:28.798 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2022-06-17T12:43:28.798 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2022-06-17T12:43:28.798 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2022-06-17T12:43:28.799 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2022-06-17T12:43:28.799 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2022-06-17T12:43:28.799 INFO:teuthology.orchestra.run.smithi107.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2022-06-17T12:43:28.799 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2022-06-17T12:43:28.800 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2022-06-17T12:43:28.800 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2022-06-17T12:43:28.800 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2022-06-17T12:43:28.800 INFO:teuthology.orchestra.run.smithi107.stdout: python3-simplejson python3-singledispatch python3-sklearn 2022-06-17T12:43:28.823 INFO:teuthology.orchestra.run.smithi107.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2022-06-17T12:43:28.824 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2022-06-17T12:43:28.824 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2022-06-17T12:43:28.825 INFO:teuthology.orchestra.run.smithi107.stdout:The following packages will be upgraded: 2022-06-17T12:43:28.825 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 librbd1 2022-06-17T12:43:28.833 INFO:teuthology.orchestra.run.smithi036.stdout:The following NEW packages will be installed: 2022-06-17T12:43:28.834 INFO:teuthology.orchestra.run.smithi036.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2022-06-17T12:43:28.834 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-06-17T12:43:28.835 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-06-17T12:43:28.835 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2022-06-17T12:43:28.836 INFO:teuthology.orchestra.run.smithi036.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2022-06-17T12:43:28.836 INFO:teuthology.orchestra.run.smithi036.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2022-06-17T12:43:28.836 INFO:teuthology.orchestra.run.smithi036.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2022-06-17T12:43:28.837 INFO:teuthology.orchestra.run.smithi036.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2022-06-17T12:43:28.837 INFO:teuthology.orchestra.run.smithi036.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2022-06-17T12:43:28.838 INFO:teuthology.orchestra.run.smithi036.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2022-06-17T12:43:28.838 INFO:teuthology.orchestra.run.smithi036.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2022-06-17T12:43:28.838 INFO:teuthology.orchestra.run.smithi036.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2022-06-17T12:43:28.839 INFO:teuthology.orchestra.run.smithi036.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2022-06-17T12:43:28.839 INFO:teuthology.orchestra.run.smithi036.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2022-06-17T12:43:28.840 INFO:teuthology.orchestra.run.smithi036.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2022-06-17T12:43:28.840 INFO:teuthology.orchestra.run.smithi036.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2022-06-17T12:43:28.841 INFO:teuthology.orchestra.run.smithi036.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2022-06-17T12:43:28.841 INFO:teuthology.orchestra.run.smithi036.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2022-06-17T12:43:28.841 INFO:teuthology.orchestra.run.smithi036.stdout: python3-simplejson python3-singledispatch python3-sklearn 2022-06-17T12:43:28.842 INFO:teuthology.orchestra.run.smithi036.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2022-06-17T12:43:28.842 INFO:teuthology.orchestra.run.smithi036.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2022-06-17T12:43:28.842 INFO:teuthology.orchestra.run.smithi036.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2022-06-17T12:43:28.843 INFO:teuthology.orchestra.run.smithi036.stdout:The following packages will be upgraded: 2022-06-17T12:43:28.858 INFO:teuthology.orchestra.run.smithi036.stdout: librados2 librbd1 2022-06-17T12:43:28.859 INFO:teuthology.orchestra.run.smithi070.stdout:The following NEW packages will be installed: 2022-06-17T12:43:28.872 INFO:teuthology.orchestra.run.smithi070.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2022-06-17T12:43:28.873 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-06-17T12:43:28.873 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-06-17T12:43:28.877 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2022-06-17T12:43:28.878 INFO:teuthology.orchestra.run.smithi070.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2022-06-17T12:43:28.878 INFO:teuthology.orchestra.run.smithi070.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2022-06-17T12:43:28.878 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2022-06-17T12:43:28.878 INFO:teuthology.orchestra.run.smithi070.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2022-06-17T12:43:28.879 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2022-06-17T12:43:28.879 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2022-06-17T12:43:28.879 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2022-06-17T12:43:28.879 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2022-06-17T12:43:28.880 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2022-06-17T12:43:28.880 INFO:teuthology.orchestra.run.smithi070.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2022-06-17T12:43:28.880 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2022-06-17T12:43:28.881 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2022-06-17T12:43:28.881 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2022-06-17T12:43:28.881 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2022-06-17T12:43:28.881 INFO:teuthology.orchestra.run.smithi070.stdout: python3-simplejson python3-singledispatch python3-sklearn 2022-06-17T12:43:28.881 INFO:teuthology.orchestra.run.smithi070.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2022-06-17T12:43:28.882 INFO:teuthology.orchestra.run.smithi070.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2022-06-17T12:43:28.882 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2022-06-17T12:43:28.882 INFO:teuthology.orchestra.run.smithi070.stdout:The following packages will be upgraded: 2022-06-17T12:43:28.882 INFO:teuthology.orchestra.run.smithi070.stdout: librados2 librbd1 2022-06-17T12:43:28.885 INFO:teuthology.orchestra.run.smithi136.stdout:The following packages were automatically installed and are no longer required: 2022-06-17T12:43:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-06-17T12:43:28.886 INFO:teuthology.orchestra.run.smithi136.stdout:Use 'sudo apt autoremove' to remove them. 2022-06-17T12:43:28.886 INFO:teuthology.orchestra.run.smithi136.stdout:The following additional packages will be installed: 2022-06-17T12:43:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-06-17T12:43:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-06-17T12:43:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2022-06-17T12:43:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2022-06-17T12:43:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2022-06-17T12:43:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2022-06-17T12:43:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2022-06-17T12:43:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2022-06-17T12:43:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2022-06-17T12:43:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2022-06-17T12:43:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2022-06-17T12:43:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2022-06-17T12:43:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2022-06-17T12:43:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2022-06-17T12:43:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2022-06-17T12:43:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2022-06-17T12:43:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2022-06-17T12:43:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2022-06-17T12:43:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2022-06-17T12:43:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet 2022-06-17T12:43:28.897 INFO:teuthology.orchestra.run.smithi136.stdout:Suggested packages: 2022-06-17T12:43:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2022-06-17T12:43:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2022-06-17T12:43:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2022-06-17T12:43:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2022-06-17T12:43:28.900 INFO:teuthology.orchestra.run.smithi136.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2022-06-17T12:43:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2022-06-17T12:43:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2022-06-17T12:43:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2022-06-17T12:43:28.909 INFO:teuthology.orchestra.run.smithi136.stdout:Recommended packages: 2022-06-17T12:43:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: btrfs-tools 2022-06-17T12:43:28.909 INFO:teuthology.orchestra.run.smithi136.stdout:The following NEW packages will be installed: 2022-06-17T12:43:28.910 INFO:teuthology.orchestra.run.smithi136.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2022-06-17T12:43:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-06-17T12:43:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-06-17T12:43:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2022-06-17T12:43:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2022-06-17T12:43:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2022-06-17T12:43:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2022-06-17T12:43:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2022-06-17T12:43:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2022-06-17T12:43:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2022-06-17T12:43:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2022-06-17T12:43:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2022-06-17T12:43:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2022-06-17T12:43:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2022-06-17T12:43:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2022-06-17T12:43:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2022-06-17T12:43:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2022-06-17T12:43:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2022-06-17T12:43:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: python3-simplejson python3-singledispatch python3-sklearn 2022-06-17T12:43:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2022-06-17T12:43:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2022-06-17T12:43:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2022-06-17T12:43:28.917 INFO:teuthology.orchestra.run.smithi136.stdout:The following packages will be upgraded: 2022-06-17T12:43:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: librados2 librbd1 2022-06-17T12:43:28.918 INFO:teuthology.orchestra.run.smithi070.stdout:2 upgraded, 101 newly installed, 0 to remove and 2 not upgraded. 2022-06-17T12:43:28.919 INFO:teuthology.orchestra.run.smithi070.stdout:Need to get 153 MB of archives. 2022-06-17T12:43:28.919 INFO:teuthology.orchestra.run.smithi070.stdout:After this operation, 699 MB of additional disk space will be used. 2022-06-17T12:43:28.919 INFO:teuthology.orchestra.run.smithi070.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.16-260-gb3641a55-1focal [1,698 kB] 2022-06-17T12:43:28.920 INFO:teuthology.orchestra.run.smithi107.stdout:2 upgraded, 101 newly installed, 0 to remove and 2 not upgraded. 2022-06-17T12:43:28.920 INFO:teuthology.orchestra.run.smithi107.stdout:Need to get 153 MB of archives. 2022-06-17T12:43:28.920 INFO:teuthology.orchestra.run.smithi107.stdout:After this operation, 699 MB of additional disk space will be used. 2022-06-17T12:43:28.920 INFO:teuthology.orchestra.run.smithi107.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.16-260-gb3641a55-1focal [1,698 kB] 2022-06-17T12:43:28.921 INFO:teuthology.orchestra.run.smithi107.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2022-06-17T12:43:28.921 INFO:teuthology.orchestra.run.smithi143.stdout:Building dependency tree... 2022-06-17T12:43:28.922 INFO:teuthology.orchestra.run.smithi143.stdout:Reading state information... 2022-06-17T12:43:28.929 INFO:teuthology.orchestra.run.smithi070.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2022-06-17T12:43:28.969 INFO:teuthology.orchestra.run.smithi036.stdout:2 upgraded, 101 newly installed, 0 to remove and 2 not upgraded. 2022-06-17T12:43:28.969 INFO:teuthology.orchestra.run.smithi036.stdout:Need to get 153 MB of archives. 2022-06-17T12:43:28.970 INFO:teuthology.orchestra.run.smithi036.stdout:After this operation, 699 MB of additional disk space will be used. 2022-06-17T12:43:28.970 INFO:teuthology.orchestra.run.smithi036.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.16-260-gb3641a55-1focal [1,698 kB] 2022-06-17T12:43:28.980 INFO:teuthology.orchestra.run.smithi107.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2022-06-17T12:43:29.010 INFO:teuthology.orchestra.run.smithi036.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2022-06-17T12:43:29.011 INFO:teuthology.orchestra.run.smithi070.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2022-06-17T12:43:29.012 INFO:teuthology.orchestra.run.smithi107.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2022-06-17T12:43:29.013 INFO:teuthology.orchestra.run.smithi136.stdout:2 upgraded, 101 newly installed, 0 to remove and 2 not upgraded. 2022-06-17T12:43:29.013 INFO:teuthology.orchestra.run.smithi136.stdout:Need to get 153 MB of archives. 2022-06-17T12:43:29.031 INFO:teuthology.orchestra.run.smithi136.stdout:After this operation, 699 MB of additional disk space will be used. 2022-06-17T12:43:29.032 INFO:teuthology.orchestra.run.smithi136.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.16-260-gb3641a55-1focal [1,698 kB] 2022-06-17T12:43:29.047 INFO:teuthology.orchestra.run.smithi136.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2022-06-17T12:43:29.048 INFO:teuthology.orchestra.run.smithi070.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2022-06-17T12:43:29.054 INFO:teuthology.orchestra.run.smithi143.stdout:The following packages were automatically installed and are no longer required: 2022-06-17T12:43:29.055 INFO:teuthology.orchestra.run.smithi143.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-06-17T12:43:29.056 INFO:teuthology.orchestra.run.smithi143.stdout:Use 'sudo apt autoremove' to remove them. 2022-06-17T12:43:29.056 INFO:teuthology.orchestra.run.smithi143.stdout:The following additional packages will be installed: 2022-06-17T12:43:29.057 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-06-17T12:43:29.057 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-06-17T12:43:29.076 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2022-06-17T12:43:29.077 INFO:teuthology.orchestra.run.smithi143.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2022-06-17T12:43:29.077 INFO:teuthology.orchestra.run.smithi143.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2022-06-17T12:43:29.079 INFO:teuthology.orchestra.run.smithi143.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2022-06-17T12:43:29.080 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2022-06-17T12:43:29.080 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2022-06-17T12:43:29.081 INFO:teuthology.orchestra.run.smithi143.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2022-06-17T12:43:29.081 INFO:teuthology.orchestra.run.smithi143.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2022-06-17T12:43:29.082 INFO:teuthology.orchestra.run.smithi143.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2022-06-17T12:43:29.082 INFO:teuthology.orchestra.run.smithi143.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2022-06-17T12:43:29.083 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2022-06-17T12:43:29.083 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2022-06-17T12:43:29.083 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2022-06-17T12:43:29.084 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2022-06-17T12:43:29.084 INFO:teuthology.orchestra.run.smithi143.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2022-06-17T12:43:29.107 INFO:teuthology.orchestra.run.smithi143.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2022-06-17T12:43:29.108 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2022-06-17T12:43:29.108 INFO:teuthology.orchestra.run.smithi143.stdout: xmlstarlet 2022-06-17T12:43:29.109 INFO:teuthology.orchestra.run.smithi143.stdout:Suggested packages: 2022-06-17T12:43:29.109 INFO:teuthology.orchestra.run.smithi143.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2022-06-17T12:43:29.109 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2022-06-17T12:43:29.110 INFO:teuthology.orchestra.run.smithi143.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2022-06-17T12:43:29.111 INFO:teuthology.orchestra.run.smithi143.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2022-06-17T12:43:29.111 INFO:teuthology.orchestra.run.smithi143.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2022-06-17T12:43:29.112 INFO:teuthology.orchestra.run.smithi143.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2022-06-17T12:43:29.112 INFO:teuthology.orchestra.run.smithi143.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2022-06-17T12:43:29.113 INFO:teuthology.orchestra.run.smithi143.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2022-06-17T12:43:29.113 INFO:teuthology.orchestra.run.smithi143.stdout:Recommended packages: 2022-06-17T12:43:29.113 INFO:teuthology.orchestra.run.smithi143.stdout: btrfs-tools 2022-06-17T12:43:29.115 INFO:teuthology.orchestra.run.smithi107.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.16-260-gb3641a55-1focal [3,257 kB] 2022-06-17T12:43:29.115 INFO:teuthology.orchestra.run.smithi036.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2022-06-17T12:43:29.120 INFO:teuthology.orchestra.run.smithi036.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2022-06-17T12:43:29.157 INFO:teuthology.orchestra.run.smithi136.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2022-06-17T12:43:29.158 INFO:teuthology.orchestra.run.smithi070.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.16-260-gb3641a55-1focal [3,257 kB] 2022-06-17T12:43:29.182 INFO:teuthology.orchestra.run.smithi107.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2022-06-17T12:43:29.182 INFO:teuthology.orchestra.run.smithi136.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2022-06-17T12:43:29.203 INFO:teuthology.orchestra.run.smithi143.stdout:The following NEW packages will be installed: 2022-06-17T12:43:29.204 INFO:teuthology.orchestra.run.smithi143.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2022-06-17T12:43:29.204 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-06-17T12:43:29.205 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-06-17T12:43:29.207 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2022-06-17T12:43:29.207 INFO:teuthology.orchestra.run.smithi143.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2022-06-17T12:43:29.208 INFO:teuthology.orchestra.run.smithi143.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2022-06-17T12:43:29.208 INFO:teuthology.orchestra.run.smithi143.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2022-06-17T12:43:29.209 INFO:teuthology.orchestra.run.smithi143.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2022-06-17T12:43:29.211 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2022-06-17T12:43:29.216 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2022-06-17T12:43:29.216 INFO:teuthology.orchestra.run.smithi143.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2022-06-17T12:43:29.218 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2022-06-17T12:43:29.219 INFO:teuthology.orchestra.run.smithi143.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2022-06-17T12:43:29.220 INFO:teuthology.orchestra.run.smithi143.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2022-06-17T12:43:29.220 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2022-06-17T12:43:29.221 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2022-06-17T12:43:29.225 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2022-06-17T12:43:29.226 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2022-06-17T12:43:29.226 INFO:teuthology.orchestra.run.smithi143.stdout: python3-simplejson python3-singledispatch python3-sklearn 2022-06-17T12:43:29.227 INFO:teuthology.orchestra.run.smithi143.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2022-06-17T12:43:29.227 INFO:teuthology.orchestra.run.smithi143.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2022-06-17T12:43:29.231 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2022-06-17T12:43:29.232 INFO:teuthology.orchestra.run.smithi143.stdout:The following packages will be upgraded: 2022-06-17T12:43:29.232 INFO:teuthology.orchestra.run.smithi143.stdout: librados2 librbd1 2022-06-17T12:43:29.233 INFO:teuthology.orchestra.run.smithi070.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2022-06-17T12:43:29.249 INFO:teuthology.orchestra.run.smithi143.stdout:2 upgraded, 101 newly installed, 0 to remove and 2 not upgraded. 2022-06-17T12:43:29.253 INFO:teuthology.orchestra.run.smithi143.stdout:Need to get 153 MB of archives. 2022-06-17T12:43:29.253 INFO:teuthology.orchestra.run.smithi143.stdout:After this operation, 699 MB of additional disk space will be used. 2022-06-17T12:43:29.253 INFO:teuthology.orchestra.run.smithi143.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2022-06-17T12:43:29.254 INFO:teuthology.orchestra.run.smithi036.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.16-260-gb3641a55-1focal [3,257 kB] 2022-06-17T12:43:29.257 INFO:teuthology.orchestra.run.smithi143.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2022-06-17T12:43:29.258 INFO:teuthology.orchestra.run.smithi143.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2022-06-17T12:43:29.282 INFO:teuthology.orchestra.run.smithi107.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2022-06-17T12:43:29.282 INFO:teuthology.orchestra.run.smithi107.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2022-06-17T12:43:29.283 INFO:teuthology.orchestra.run.smithi036.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2022-06-17T12:43:29.297 INFO:teuthology.orchestra.run.smithi143.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2022-06-17T12:43:29.303 INFO:teuthology.orchestra.run.smithi136.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2022-06-17T12:43:29.344 INFO:teuthology.orchestra.run.smithi070.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2022-06-17T12:43:29.346 INFO:teuthology.orchestra.run.smithi070.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2022-06-17T12:43:29.348 INFO:teuthology.orchestra.run.smithi070.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2022-06-17T12:43:29.350 INFO:teuthology.orchestra.run.smithi136.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.16-260-gb3641a55-1focal [3,257 kB] 2022-06-17T12:43:29.350 INFO:teuthology.orchestra.run.smithi107.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2022-06-17T12:43:29.351 INFO:teuthology.orchestra.run.smithi143.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2022-06-17T12:43:29.352 INFO:teuthology.orchestra.run.smithi143.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2022-06-17T12:43:29.353 INFO:teuthology.orchestra.run.smithi143.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2022-06-17T12:43:29.354 INFO:teuthology.orchestra.run.smithi143.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2022-06-17T12:43:29.364 INFO:teuthology.orchestra.run.smithi143.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2022-06-17T12:43:29.365 INFO:teuthology.orchestra.run.smithi143.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2022-06-17T12:43:29.378 INFO:teuthology.orchestra.run.smithi143.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2022-06-17T12:43:29.378 INFO:teuthology.orchestra.run.smithi143.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2022-06-17T12:43:29.379 INFO:teuthology.orchestra.run.smithi143.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2022-06-17T12:43:29.379 INFO:teuthology.orchestra.run.smithi143.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jwt all 1.7.1-2ubuntu2 [17.4 kB] 2022-06-17T12:43:29.384 INFO:teuthology.orchestra.run.smithi143.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2022-06-17T12:43:29.384 INFO:teuthology.orchestra.run.smithi143.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2022-06-17T12:43:29.387 INFO:teuthology.orchestra.run.smithi143.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2022-06-17T12:43:29.388 INFO:teuthology.orchestra.run.smithi143.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2022-06-17T12:43:29.388 INFO:teuthology.orchestra.run.smithi143.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2022-06-17T12:43:29.389 INFO:teuthology.orchestra.run.smithi143.stdout:Get:20 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.16-260-gb3641a55-1focal [1,698 kB] 2022-06-17T12:43:29.391 INFO:teuthology.orchestra.run.smithi036.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2022-06-17T12:43:29.402 INFO:teuthology.orchestra.run.smithi036.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2022-06-17T12:43:29.408 INFO:teuthology.orchestra.run.smithi107.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2022-06-17T12:43:29.409 INFO:teuthology.orchestra.run.smithi143.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2022-06-17T12:43:29.425 INFO:teuthology.orchestra.run.smithi143.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2022-06-17T12:43:29.426 INFO:teuthology.orchestra.run.smithi036.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2022-06-17T12:43:29.427 INFO:teuthology.orchestra.run.smithi143.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2022-06-17T12:43:29.427 INFO:teuthology.orchestra.run.smithi143.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2022-06-17T12:43:29.427 INFO:teuthology.orchestra.run.smithi143.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2022-06-17T12:43:29.427 INFO:teuthology.orchestra.run.smithi143.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2022-06-17T12:43:29.428 INFO:teuthology.orchestra.run.smithi143.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2022-06-17T12:43:29.429 INFO:teuthology.orchestra.run.smithi107.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2022-06-17T12:43:29.429 INFO:teuthology.orchestra.run.smithi107.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2022-06-17T12:43:29.430 INFO:teuthology.orchestra.run.smithi143.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2022-06-17T12:43:29.430 INFO:teuthology.orchestra.run.smithi143.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2022-06-17T12:43:29.430 INFO:teuthology.orchestra.run.smithi143.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2022-06-17T12:43:29.439 INFO:teuthology.orchestra.run.smithi136.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2022-06-17T12:43:29.462 INFO:teuthology.orchestra.run.smithi070.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2022-06-17T12:43:29.465 INFO:teuthology.orchestra.run.smithi136.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2022-06-17T12:43:29.466 INFO:teuthology.orchestra.run.smithi136.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2022-06-17T12:43:29.467 INFO:teuthology.orchestra.run.smithi107.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2022-06-17T12:43:29.468 INFO:teuthology.orchestra.run.smithi107.stdout:Get:14 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.16-260-gb3641a55-1focal [463 kB] 2022-06-17T12:43:29.469 INFO:teuthology.orchestra.run.smithi107.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2022-06-17T12:43:29.469 INFO:teuthology.orchestra.run.smithi107.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2022-06-17T12:43:29.471 INFO:teuthology.orchestra.run.smithi143.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2022-06-17T12:43:29.471 INFO:teuthology.orchestra.run.smithi143.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2022-06-17T12:43:29.471 INFO:teuthology.orchestra.run.smithi143.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2022-06-17T12:43:29.472 INFO:teuthology.orchestra.run.smithi143.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2022-06-17T12:43:29.472 INFO:teuthology.orchestra.run.smithi143.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2022-06-17T12:43:29.473 INFO:teuthology.orchestra.run.smithi143.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2022-06-17T12:43:29.473 INFO:teuthology.orchestra.run.smithi143.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2022-06-17T12:43:29.475 INFO:teuthology.orchestra.run.smithi070.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2022-06-17T12:43:29.475 INFO:teuthology.orchestra.run.smithi070.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2022-06-17T12:43:29.476 INFO:teuthology.orchestra.run.smithi143.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2022-06-17T12:43:29.481 INFO:teuthology.orchestra.run.smithi143.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2022-06-17T12:43:29.484 INFO:teuthology.orchestra.run.smithi107.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jwt all 1.7.1-2ubuntu2 [17.4 kB] 2022-06-17T12:43:29.485 INFO:teuthology.orchestra.run.smithi107.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2022-06-17T12:43:29.504 INFO:teuthology.orchestra.run.smithi107.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2022-06-17T12:43:29.505 INFO:teuthology.orchestra.run.smithi070.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2022-06-17T12:43:29.506 INFO:teuthology.orchestra.run.smithi070.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2022-06-17T12:43:29.506 INFO:teuthology.orchestra.run.smithi070.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2022-06-17T12:43:29.515 INFO:teuthology.orchestra.run.smithi107.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2022-06-17T12:43:29.516 INFO:teuthology.orchestra.run.smithi036.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2022-06-17T12:43:29.518 INFO:teuthology.orchestra.run.smithi107.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2022-06-17T12:43:29.518 INFO:teuthology.orchestra.run.smithi070.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jwt all 1.7.1-2ubuntu2 [17.4 kB] 2022-06-17T12:43:29.519 INFO:teuthology.orchestra.run.smithi070.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2022-06-17T12:43:29.519 INFO:teuthology.orchestra.run.smithi107.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2022-06-17T12:43:29.520 INFO:teuthology.orchestra.run.smithi070.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2022-06-17T12:43:29.522 INFO:teuthology.orchestra.run.smithi107.stdout:Get:23 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.16-260-gb3641a55-1focal [365 kB] 2022-06-17T12:43:29.538 INFO:teuthology.orchestra.run.smithi036.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2022-06-17T12:43:29.559 INFO:teuthology.orchestra.run.smithi036.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2022-06-17T12:43:29.567 INFO:teuthology.orchestra.run.smithi070.stdout:Get:19 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.16-260-gb3641a55-1focal [463 kB] 2022-06-17T12:43:29.591 INFO:teuthology.orchestra.run.smithi070.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2022-06-17T12:43:29.592 INFO:teuthology.orchestra.run.smithi070.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2022-06-17T12:43:29.593 INFO:teuthology.orchestra.run.smithi070.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2022-06-17T12:43:29.602 INFO:teuthology.orchestra.run.smithi136.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2022-06-17T12:43:29.603 INFO:teuthology.orchestra.run.smithi136.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2022-06-17T12:43:29.611 INFO:teuthology.orchestra.run.smithi136.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2022-06-17T12:43:29.613 INFO:teuthology.orchestra.run.smithi036.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2022-06-17T12:43:29.615 INFO:teuthology.orchestra.run.smithi036.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2022-06-17T12:43:29.616 INFO:teuthology.orchestra.run.smithi036.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2022-06-17T12:43:29.616 INFO:teuthology.orchestra.run.smithi036.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jwt all 1.7.1-2ubuntu2 [17.4 kB] 2022-06-17T12:43:29.617 INFO:teuthology.orchestra.run.smithi036.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2022-06-17T12:43:29.617 INFO:teuthology.orchestra.run.smithi036.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2022-06-17T12:43:29.619 INFO:teuthology.orchestra.run.smithi107.stdout:Get:24 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.16-260-gb3641a55-1focal [20.9 kB] 2022-06-17T12:43:29.620 INFO:teuthology.orchestra.run.smithi107.stdout:Get:25 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.16-260-gb3641a55-1focal [156 kB] 2022-06-17T12:43:29.620 INFO:teuthology.orchestra.run.smithi107.stdout:Get:26 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.16-260-gb3641a55-1focal [29.4 kB] 2022-06-17T12:43:29.622 INFO:teuthology.orchestra.run.smithi107.stdout:Get:27 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.16-260-gb3641a55-1focal [363 kB] 2022-06-17T12:43:29.622 INFO:teuthology.orchestra.run.smithi107.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2022-06-17T12:43:29.623 INFO:teuthology.orchestra.run.smithi107.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2022-06-17T12:43:29.624 INFO:teuthology.orchestra.run.smithi107.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2022-06-17T12:43:29.625 INFO:teuthology.orchestra.run.smithi107.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2022-06-17T12:43:29.625 INFO:teuthology.orchestra.run.smithi107.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2022-06-17T12:43:29.626 INFO:teuthology.orchestra.run.smithi107.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2022-06-17T12:43:29.628 INFO:teuthology.orchestra.run.smithi143.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2022-06-17T12:43:29.629 INFO:teuthology.orchestra.run.smithi070.stdout:Get:23 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.16-260-gb3641a55-1focal [365 kB] 2022-06-17T12:43:29.629 INFO:teuthology.orchestra.run.smithi107.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2022-06-17T12:43:29.630 INFO:teuthology.orchestra.run.smithi136.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2022-06-17T12:43:29.631 INFO:teuthology.orchestra.run.smithi136.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2022-06-17T12:43:29.632 INFO:teuthology.orchestra.run.smithi136.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2022-06-17T12:43:29.634 INFO:teuthology.orchestra.run.smithi036.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2022-06-17T12:43:29.635 INFO:teuthology.orchestra.run.smithi107.stdout:Get:35 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.16-260-gb3641a55-1focal [3,145 kB] 2022-06-17T12:43:29.635 INFO:teuthology.orchestra.run.smithi143.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2022-06-17T12:43:29.636 INFO:teuthology.orchestra.run.smithi143.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2022-06-17T12:43:29.640 INFO:teuthology.orchestra.run.smithi070.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2022-06-17T12:43:29.641 INFO:teuthology.orchestra.run.smithi070.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2022-06-17T12:43:29.641 INFO:teuthology.orchestra.run.smithi036.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2022-06-17T12:43:29.642 INFO:teuthology.orchestra.run.smithi036.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2022-06-17T12:43:29.677 INFO:teuthology.orchestra.run.smithi136.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jwt all 1.7.1-2ubuntu2 [17.4 kB] 2022-06-17T12:43:29.678 INFO:teuthology.orchestra.run.smithi136.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2022-06-17T12:43:29.678 INFO:teuthology.orchestra.run.smithi136.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2022-06-17T12:43:29.679 INFO:teuthology.orchestra.run.smithi136.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2022-06-17T12:43:29.680 INFO:teuthology.orchestra.run.smithi143.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2022-06-17T12:43:29.680 INFO:teuthology.orchestra.run.smithi143.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2022-06-17T12:43:29.681 INFO:teuthology.orchestra.run.smithi143.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2022-06-17T12:43:29.681 INFO:teuthology.orchestra.run.smithi143.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2022-06-17T12:43:29.681 INFO:teuthology.orchestra.run.smithi143.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2022-06-17T12:43:29.684 INFO:teuthology.orchestra.run.smithi070.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2022-06-17T12:43:29.684 INFO:teuthology.orchestra.run.smithi070.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2022-06-17T12:43:29.685 INFO:teuthology.orchestra.run.smithi070.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2022-06-17T12:43:29.685 INFO:teuthology.orchestra.run.smithi070.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2022-06-17T12:43:29.686 INFO:teuthology.orchestra.run.smithi070.stdout:Get:30 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.16-260-gb3641a55-1focal [20.9 kB] 2022-06-17T12:43:29.686 INFO:teuthology.orchestra.run.smithi070.stdout:Get:31 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.16-260-gb3641a55-1focal [156 kB] 2022-06-17T12:43:29.687 INFO:teuthology.orchestra.run.smithi070.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2022-06-17T12:43:29.687 INFO:teuthology.orchestra.run.smithi070.stdout:Get:33 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.16-260-gb3641a55-1focal [29.4 kB] 2022-06-17T12:43:29.688 INFO:teuthology.orchestra.run.smithi070.stdout:Get:34 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.16-260-gb3641a55-1focal [363 kB] 2022-06-17T12:43:29.694 INFO:teuthology.orchestra.run.smithi136.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2022-06-17T12:43:29.696 INFO:teuthology.orchestra.run.smithi136.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2022-06-17T12:43:29.696 INFO:teuthology.orchestra.run.smithi107.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2022-06-17T12:43:29.705 INFO:teuthology.orchestra.run.smithi107.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2022-06-17T12:43:29.706 INFO:teuthology.orchestra.run.smithi107.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2022-06-17T12:43:29.708 INFO:teuthology.orchestra.run.smithi036.stdout:Get:22 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.16-260-gb3641a55-1focal [463 kB] 2022-06-17T12:43:29.710 INFO:teuthology.orchestra.run.smithi143.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2022-06-17T12:43:29.711 INFO:teuthology.orchestra.run.smithi143.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2022-06-17T12:43:29.711 INFO:teuthology.orchestra.run.smithi143.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2022-06-17T12:43:29.711 INFO:teuthology.orchestra.run.smithi143.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2022-06-17T12:43:29.712 INFO:teuthology.orchestra.run.smithi143.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2022-06-17T12:43:29.713 INFO:teuthology.orchestra.run.smithi143.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2022-06-17T12:43:29.713 INFO:teuthology.orchestra.run.smithi143.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2022-06-17T12:43:29.714 INFO:teuthology.orchestra.run.smithi143.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2022-06-17T12:43:29.716 INFO:teuthology.orchestra.run.smithi143.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2022-06-17T12:43:29.716 INFO:teuthology.orchestra.run.smithi036.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2022-06-17T12:43:29.717 INFO:teuthology.orchestra.run.smithi036.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2022-06-17T12:43:29.717 INFO:teuthology.orchestra.run.smithi036.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2022-06-17T12:43:29.719 INFO:teuthology.orchestra.run.smithi070.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2022-06-17T12:43:29.719 INFO:teuthology.orchestra.run.smithi070.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2022-06-17T12:43:29.720 INFO:teuthology.orchestra.run.smithi070.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2022-06-17T12:43:29.720 INFO:teuthology.orchestra.run.smithi036.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2022-06-17T12:43:29.737 INFO:teuthology.orchestra.run.smithi070.stdout:Get:38 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.16-260-gb3641a55-1focal [3,145 kB] 2022-06-17T12:43:29.748 INFO:teuthology.orchestra.run.smithi036.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2022-06-17T12:43:29.748 INFO:teuthology.orchestra.run.smithi036.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2022-06-17T12:43:29.750 INFO:teuthology.orchestra.run.smithi143.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2022-06-17T12:43:29.751 INFO:teuthology.orchestra.run.smithi143.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2022-06-17T12:43:29.751 INFO:teuthology.orchestra.run.smithi143.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2022-06-17T12:43:29.751 INFO:teuthology.orchestra.run.smithi143.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2022-06-17T12:43:29.752 INFO:teuthology.orchestra.run.smithi143.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2022-06-17T12:43:29.752 INFO:teuthology.orchestra.run.smithi143.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2022-06-17T12:43:29.753 INFO:teuthology.orchestra.run.smithi143.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2022-06-17T12:43:29.753 INFO:teuthology.orchestra.run.smithi143.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2022-06-17T12:43:29.754 INFO:teuthology.orchestra.run.smithi143.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2022-06-17T12:43:29.755 INFO:teuthology.orchestra.run.smithi107.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2022-06-17T12:43:29.756 INFO:teuthology.orchestra.run.smithi036.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2022-06-17T12:43:29.757 INFO:teuthology.orchestra.run.smithi136.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2022-06-17T12:43:29.759 INFO:teuthology.orchestra.run.smithi143.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2022-06-17T12:43:29.761 INFO:teuthology.orchestra.run.smithi107.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2022-06-17T12:43:29.761 INFO:teuthology.orchestra.run.smithi143.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2022-06-17T12:43:29.762 INFO:teuthology.orchestra.run.smithi143.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2022-06-17T12:43:29.762 INFO:teuthology.orchestra.run.smithi143.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2022-06-17T12:43:29.763 INFO:teuthology.orchestra.run.smithi036.stdout:Get:30 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.16-260-gb3641a55-1focal [365 kB] 2022-06-17T12:43:29.764 INFO:teuthology.orchestra.run.smithi107.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2022-06-17T12:43:29.765 INFO:teuthology.orchestra.run.smithi107.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2022-06-17T12:43:29.769 INFO:teuthology.orchestra.run.smithi107.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2022-06-17T12:43:29.779 INFO:teuthology.orchestra.run.smithi136.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2022-06-17T12:43:29.780 INFO:teuthology.orchestra.run.smithi136.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2022-06-17T12:43:29.780 INFO:teuthology.orchestra.run.smithi136.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2022-06-17T12:43:29.781 INFO:teuthology.orchestra.run.smithi136.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2022-06-17T12:43:29.781 INFO:teuthology.orchestra.run.smithi136.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2022-06-17T12:43:29.783 INFO:teuthology.orchestra.run.smithi107.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2022-06-17T12:43:29.784 INFO:teuthology.orchestra.run.smithi107.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2022-06-17T12:43:29.784 INFO:teuthology.orchestra.run.smithi143.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2022-06-17T12:43:29.785 INFO:teuthology.orchestra.run.smithi143.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2022-06-17T12:43:29.785 INFO:teuthology.orchestra.run.smithi143.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2022-06-17T12:43:29.786 INFO:teuthology.orchestra.run.smithi143.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2022-06-17T12:43:29.793 INFO:teuthology.orchestra.run.smithi136.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2022-06-17T12:43:29.794 INFO:teuthology.orchestra.run.smithi036.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2022-06-17T12:43:29.794 INFO:teuthology.orchestra.run.smithi036.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2022-06-17T12:43:29.795 INFO:teuthology.orchestra.run.smithi036.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2022-06-17T12:43:29.803 INFO:teuthology.orchestra.run.smithi070.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2022-06-17T12:43:29.804 INFO:teuthology.orchestra.run.smithi070.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2022-06-17T12:43:29.812 INFO:teuthology.orchestra.run.smithi070.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2022-06-17T12:43:29.812 INFO:teuthology.orchestra.run.smithi070.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2022-06-17T12:43:29.813 INFO:teuthology.orchestra.run.smithi070.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2022-06-17T12:43:29.814 INFO:teuthology.orchestra.run.smithi070.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2022-06-17T12:43:29.823 INFO:teuthology.orchestra.run.smithi070.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2022-06-17T12:43:29.825 INFO:teuthology.orchestra.run.smithi036.stdout:Get:34 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.16-260-gb3641a55-1focal [20.9 kB] 2022-06-17T12:43:29.825 INFO:teuthology.orchestra.run.smithi036.stdout:Get:35 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.16-260-gb3641a55-1focal [156 kB] 2022-06-17T12:43:29.826 INFO:teuthology.orchestra.run.smithi036.stdout:Get:36 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.16-260-gb3641a55-1focal [29.4 kB] 2022-06-17T12:43:29.826 INFO:teuthology.orchestra.run.smithi036.stdout:Get:37 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.16-260-gb3641a55-1focal [363 kB] 2022-06-17T12:43:29.845 INFO:teuthology.orchestra.run.smithi107.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2022-06-17T12:43:29.845 INFO:teuthology.orchestra.run.smithi107.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2022-06-17T12:43:29.847 INFO:teuthology.orchestra.run.smithi136.stdout:Get:29 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.16-260-gb3641a55-1focal [463 kB] 2022-06-17T12:43:29.848 INFO:teuthology.orchestra.run.smithi136.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2022-06-17T12:43:29.848 INFO:teuthology.orchestra.run.smithi136.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2022-06-17T12:43:29.848 INFO:teuthology.orchestra.run.smithi136.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2022-06-17T12:43:29.872 INFO:teuthology.orchestra.run.smithi036.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2022-06-17T12:43:29.876 INFO:teuthology.orchestra.run.smithi070.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2022-06-17T12:43:29.877 INFO:teuthology.orchestra.run.smithi070.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2022-06-17T12:43:29.878 INFO:teuthology.orchestra.run.smithi036.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2022-06-17T12:43:29.881 INFO:teuthology.orchestra.run.smithi036.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2022-06-17T12:43:29.901 INFO:teuthology.orchestra.run.smithi036.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2022-06-17T12:43:29.903 INFO:teuthology.orchestra.run.smithi036.stdout:Get:42 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.16-260-gb3641a55-1focal [3,145 kB] 2022-06-17T12:43:29.904 INFO:teuthology.orchestra.run.smithi036.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2022-06-17T12:43:29.904 INFO:teuthology.orchestra.run.smithi036.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2022-06-17T12:43:29.905 INFO:teuthology.orchestra.run.smithi036.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2022-06-17T12:43:29.905 INFO:teuthology.orchestra.run.smithi136.stdout:Get:33 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.16-260-gb3641a55-1focal [365 kB] 2022-06-17T12:43:29.916 INFO:teuthology.orchestra.run.smithi136.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2022-06-17T12:43:29.936 INFO:teuthology.orchestra.run.smithi136.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2022-06-17T12:43:29.937 INFO:teuthology.orchestra.run.smithi136.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2022-06-17T12:43:29.937 INFO:teuthology.orchestra.run.smithi136.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2022-06-17T12:43:29.938 INFO:teuthology.orchestra.run.smithi136.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2022-06-17T12:43:29.938 INFO:teuthology.orchestra.run.smithi136.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2022-06-17T12:43:29.938 INFO:teuthology.orchestra.run.smithi136.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2022-06-17T12:43:29.953 INFO:teuthology.orchestra.run.smithi036.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2022-06-17T12:43:29.953 INFO:teuthology.orchestra.run.smithi036.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2022-06-17T12:43:29.972 INFO:teuthology.orchestra.run.smithi136.stdout:Get:41 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.16-260-gb3641a55-1focal [20.9 kB] 2022-06-17T12:43:29.973 INFO:teuthology.orchestra.run.smithi136.stdout:Get:42 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.16-260-gb3641a55-1focal [156 kB] 2022-06-17T12:43:29.981 INFO:teuthology.orchestra.run.smithi107.stdout:Get:48 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.16-260-gb3641a55-1focal [116 kB] 2022-06-17T12:43:29.987 INFO:teuthology.orchestra.run.smithi107.stdout:Get:49 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.16-260-gb3641a55-1focal [360 kB] 2022-06-17T12:43:29.996 INFO:teuthology.orchestra.run.smithi136.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2022-06-17T12:43:29.997 INFO:teuthology.orchestra.run.smithi136.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2022-06-17T12:43:30.043 INFO:teuthology.orchestra.run.smithi107.stdout:Get:50 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.16-260-gb3641a55-1focal [20.1 MB] 2022-06-17T12:43:30.044 INFO:teuthology.orchestra.run.smithi136.stdout:Get:45 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.16-260-gb3641a55-1focal [29.4 kB] 2022-06-17T12:43:30.044 INFO:teuthology.orchestra.run.smithi136.stdout:Get:46 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.16-260-gb3641a55-1focal [363 kB] 2022-06-17T12:43:30.051 INFO:teuthology.orchestra.run.smithi136.stdout:Get:47 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.16-260-gb3641a55-1focal [3,145 kB] 2022-06-17T12:43:30.095 INFO:teuthology.orchestra.run.smithi070.stdout:Get:48 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.16-260-gb3641a55-1focal [116 kB] 2022-06-17T12:43:30.151 INFO:teuthology.orchestra.run.smithi070.stdout:Get:49 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.16-260-gb3641a55-1focal [360 kB] 2022-06-17T12:43:30.182 INFO:teuthology.orchestra.run.smithi070.stdout:Get:50 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.16-260-gb3641a55-1focal [20.1 MB] 2022-06-17T12:43:30.183 INFO:teuthology.orchestra.run.smithi107.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2022-06-17T12:43:30.224 INFO:teuthology.orchestra.run.smithi070.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2022-06-17T12:43:30.247 INFO:teuthology.orchestra.run.smithi107.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2022-06-17T12:43:30.257 INFO:teuthology.orchestra.run.smithi107.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2022-06-17T12:43:30.270 INFO:teuthology.orchestra.run.smithi107.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2022-06-17T12:43:30.271 INFO:teuthology.orchestra.run.smithi107.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2022-06-17T12:43:30.271 INFO:teuthology.orchestra.run.smithi107.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2022-06-17T12:43:30.276 INFO:teuthology.orchestra.run.smithi070.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2022-06-17T12:43:30.276 INFO:teuthology.orchestra.run.smithi070.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2022-06-17T12:43:30.284 INFO:teuthology.orchestra.run.smithi036.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2022-06-17T12:43:30.351 INFO:teuthology.orchestra.run.smithi136.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2022-06-17T12:43:30.352 INFO:teuthology.orchestra.run.smithi036.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2022-06-17T12:43:30.352 INFO:teuthology.orchestra.run.smithi036.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2022-06-17T12:43:30.353 INFO:teuthology.orchestra.run.smithi107.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2022-06-17T12:43:30.353 INFO:teuthology.orchestra.run.smithi107.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2022-06-17T12:43:30.363 INFO:teuthology.orchestra.run.smithi107.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2022-06-17T12:43:30.364 INFO:teuthology.orchestra.run.smithi107.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2022-06-17T12:43:30.364 INFO:teuthology.orchestra.run.smithi107.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2022-06-17T12:43:30.364 INFO:teuthology.orchestra.run.smithi107.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2022-06-17T12:43:30.365 INFO:teuthology.orchestra.run.smithi070.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2022-06-17T12:43:30.365 INFO:teuthology.orchestra.run.smithi070.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2022-06-17T12:43:30.365 INFO:teuthology.orchestra.run.smithi070.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2022-06-17T12:43:30.365 INFO:teuthology.orchestra.run.smithi070.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2022-06-17T12:43:30.369 INFO:teuthology.orchestra.run.smithi070.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2022-06-17T12:43:30.369 INFO:teuthology.orchestra.run.smithi070.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2022-06-17T12:43:30.370 INFO:teuthology.orchestra.run.smithi070.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2022-06-17T12:43:30.371 INFO:teuthology.orchestra.run.smithi070.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2022-06-17T12:43:30.372 INFO:teuthology.orchestra.run.smithi070.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2022-06-17T12:43:30.381 INFO:teuthology.orchestra.run.smithi036.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2022-06-17T12:43:30.382 INFO:teuthology.orchestra.run.smithi036.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2022-06-17T12:43:30.382 INFO:teuthology.orchestra.run.smithi036.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2022-06-17T12:43:30.383 INFO:teuthology.orchestra.run.smithi036.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2022-06-17T12:43:30.384 INFO:teuthology.orchestra.run.smithi036.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2022-06-17T12:43:30.386 INFO:teuthology.orchestra.run.smithi036.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2022-06-17T12:43:30.386 INFO:teuthology.orchestra.run.smithi036.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2022-06-17T12:43:30.388 INFO:teuthology.orchestra.run.smithi136.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2022-06-17T12:43:30.390 INFO:teuthology.orchestra.run.smithi136.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2022-06-17T12:43:30.411 INFO:teuthology.orchestra.run.smithi036.stdout:Get:58 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.16-260-gb3641a55-1focal [116 kB] 2022-06-17T12:43:30.411 INFO:teuthology.orchestra.run.smithi107.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2022-06-17T12:43:30.412 INFO:teuthology.orchestra.run.smithi036.stdout:Get:59 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.16-260-gb3641a55-1focal [360 kB] 2022-06-17T12:43:30.412 INFO:teuthology.orchestra.run.smithi107.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2022-06-17T12:43:30.436 INFO:teuthology.orchestra.run.smithi036.stdout:Get:60 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.16-260-gb3641a55-1focal [20.1 MB] 2022-06-17T12:43:30.439 INFO:teuthology.orchestra.run.smithi136.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2022-06-17T12:43:30.439 INFO:teuthology.orchestra.run.smithi136.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2022-06-17T12:43:30.443 INFO:teuthology.orchestra.run.smithi136.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2022-06-17T12:43:30.458 INFO:teuthology.orchestra.run.smithi136.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2022-06-17T12:43:30.472 INFO:teuthology.orchestra.run.smithi136.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2022-06-17T12:43:30.478 INFO:teuthology.orchestra.run.smithi136.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2022-06-17T12:43:30.478 INFO:teuthology.orchestra.run.smithi107.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2022-06-17T12:43:30.479 INFO:teuthology.orchestra.run.smithi107.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2022-06-17T12:43:30.479 INFO:teuthology.orchestra.run.smithi107.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2022-06-17T12:43:30.480 INFO:teuthology.orchestra.run.smithi107.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2022-06-17T12:43:30.480 INFO:teuthology.orchestra.run.smithi107.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2022-06-17T12:43:30.480 INFO:teuthology.orchestra.run.smithi107.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2022-06-17T12:43:30.482 INFO:teuthology.orchestra.run.smithi136.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2022-06-17T12:43:30.482 INFO:teuthology.orchestra.run.smithi036.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2022-06-17T12:43:30.483 INFO:teuthology.orchestra.run.smithi036.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2022-06-17T12:43:30.484 INFO:teuthology.orchestra.run.smithi070.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2022-06-17T12:43:30.484 INFO:teuthology.orchestra.run.smithi070.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2022-06-17T12:43:30.485 INFO:teuthology.orchestra.run.smithi070.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2022-06-17T12:43:30.485 INFO:teuthology.orchestra.run.smithi070.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2022-06-17T12:43:30.485 INFO:teuthology.orchestra.run.smithi070.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2022-06-17T12:43:30.485 INFO:teuthology.orchestra.run.smithi070.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2022-06-17T12:43:30.486 INFO:teuthology.orchestra.run.smithi070.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2022-06-17T12:43:30.486 INFO:teuthology.orchestra.run.smithi070.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2022-06-17T12:43:30.486 INFO:teuthology.orchestra.run.smithi107.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2022-06-17T12:43:30.494 INFO:teuthology.orchestra.run.smithi107.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2022-06-17T12:43:30.495 INFO:teuthology.orchestra.run.smithi136.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2022-06-17T12:43:30.496 INFO:teuthology.orchestra.run.smithi136.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2022-06-17T12:43:30.524 INFO:teuthology.orchestra.run.smithi036.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2022-06-17T12:43:30.540 INFO:teuthology.orchestra.run.smithi070.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2022-06-17T12:43:30.540 INFO:teuthology.orchestra.run.smithi070.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2022-06-17T12:43:30.560 INFO:teuthology.orchestra.run.smithi036.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2022-06-17T12:43:30.561 INFO:teuthology.orchestra.run.smithi036.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2022-06-17T12:43:30.561 INFO:teuthology.orchestra.run.smithi036.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2022-06-17T12:43:30.562 INFO:teuthology.orchestra.run.smithi036.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2022-06-17T12:43:30.562 INFO:teuthology.orchestra.run.smithi036.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2022-06-17T12:43:30.563 INFO:teuthology.orchestra.run.smithi036.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2022-06-17T12:43:30.564 INFO:teuthology.orchestra.run.smithi036.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2022-06-17T12:43:30.604 INFO:teuthology.orchestra.run.smithi136.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2022-06-17T12:43:30.605 INFO:teuthology.orchestra.run.smithi136.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2022-06-17T12:43:30.609 INFO:teuthology.orchestra.run.smithi107.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2022-06-17T12:43:30.610 INFO:teuthology.orchestra.run.smithi107.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2022-06-17T12:43:30.610 INFO:teuthology.orchestra.run.smithi107.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2022-06-17T12:43:30.611 INFO:teuthology.orchestra.run.smithi107.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2022-06-17T12:43:30.611 INFO:teuthology.orchestra.run.smithi107.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2022-06-17T12:43:30.612 INFO:teuthology.orchestra.run.smithi107.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2022-06-17T12:43:30.612 INFO:teuthology.orchestra.run.smithi107.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2022-06-17T12:43:30.613 INFO:teuthology.orchestra.run.smithi107.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2022-06-17T12:43:30.668 INFO:teuthology.orchestra.run.smithi036.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2022-06-17T12:43:30.669 INFO:teuthology.orchestra.run.smithi036.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2022-06-17T12:43:30.689 INFO:teuthology.orchestra.run.smithi070.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2022-06-17T12:43:30.710 INFO:teuthology.orchestra.run.smithi070.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2022-06-17T12:43:30.711 INFO:teuthology.orchestra.run.smithi070.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2022-06-17T12:43:30.711 INFO:teuthology.orchestra.run.smithi070.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2022-06-17T12:43:30.715 INFO:teuthology.orchestra.run.smithi070.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2022-06-17T12:43:30.716 INFO:teuthology.orchestra.run.smithi070.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2022-06-17T12:43:30.716 INFO:teuthology.orchestra.run.smithi070.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2022-06-17T12:43:30.746 INFO:teuthology.orchestra.run.smithi070.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2022-06-17T12:43:30.746 INFO:teuthology.orchestra.run.smithi070.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2022-06-17T12:43:30.747 INFO:teuthology.orchestra.run.smithi070.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2022-06-17T12:43:30.769 INFO:teuthology.orchestra.run.smithi070.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2022-06-17T12:43:30.775 INFO:teuthology.orchestra.run.smithi107.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2022-06-17T12:43:30.776 INFO:teuthology.orchestra.run.smithi107.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2022-06-17T12:43:30.776 INFO:teuthology.orchestra.run.smithi107.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2022-06-17T12:43:30.776 INFO:teuthology.orchestra.run.smithi107.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2022-06-17T12:43:30.777 INFO:teuthology.orchestra.run.smithi136.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2022-06-17T12:43:30.777 INFO:teuthology.orchestra.run.smithi136.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2022-06-17T12:43:30.778 INFO:teuthology.orchestra.run.smithi136.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2022-06-17T12:43:30.778 INFO:teuthology.orchestra.run.smithi136.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2022-06-17T12:43:30.778 INFO:teuthology.orchestra.run.smithi136.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2022-06-17T12:43:30.779 INFO:teuthology.orchestra.run.smithi136.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2022-06-17T12:43:30.779 INFO:teuthology.orchestra.run.smithi136.stdout:Get:68 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.16-260-gb3641a55-1focal [116 kB] 2022-06-17T12:43:30.779 INFO:teuthology.orchestra.run.smithi136.stdout:Get:69 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.16-260-gb3641a55-1focal [360 kB] 2022-06-17T12:43:30.780 INFO:teuthology.orchestra.run.smithi136.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2022-06-17T12:43:30.780 INFO:teuthology.orchestra.run.smithi136.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2022-06-17T12:43:30.780 INFO:teuthology.orchestra.run.smithi136.stdout:Get:72 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.16-260-gb3641a55-1focal [20.1 MB] 2022-06-17T12:43:30.781 INFO:teuthology.orchestra.run.smithi136.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2022-06-17T12:43:30.781 INFO:teuthology.orchestra.run.smithi136.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2022-06-17T12:43:30.781 INFO:teuthology.orchestra.run.smithi136.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2022-06-17T12:43:30.781 INFO:teuthology.orchestra.run.smithi136.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2022-06-17T12:43:30.782 INFO:teuthology.orchestra.run.smithi136.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2022-06-17T12:43:30.782 INFO:teuthology.orchestra.run.smithi136.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2022-06-17T12:43:30.783 INFO:teuthology.orchestra.run.smithi136.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2022-06-17T12:43:30.783 INFO:teuthology.orchestra.run.smithi136.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2022-06-17T12:43:30.784 INFO:teuthology.orchestra.run.smithi036.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2022-06-17T12:43:30.785 INFO:teuthology.orchestra.run.smithi036.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2022-06-17T12:43:30.785 INFO:teuthology.orchestra.run.smithi036.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2022-06-17T12:43:30.786 INFO:teuthology.orchestra.run.smithi036.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2022-06-17T12:43:30.786 INFO:teuthology.orchestra.run.smithi036.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2022-06-17T12:43:30.787 INFO:teuthology.orchestra.run.smithi036.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2022-06-17T12:43:30.787 INFO:teuthology.orchestra.run.smithi036.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2022-06-17T12:43:30.788 INFO:teuthology.orchestra.run.smithi036.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2022-06-17T12:43:30.825 INFO:teuthology.orchestra.run.smithi036.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2022-06-17T12:43:30.826 INFO:teuthology.orchestra.run.smithi036.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2022-06-17T12:43:30.829 INFO:teuthology.orchestra.run.smithi070.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2022-06-17T12:43:30.831 INFO:teuthology.orchestra.run.smithi136.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2022-06-17T12:43:30.849 INFO:teuthology.orchestra.run.smithi136.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2022-06-17T12:43:30.852 INFO:teuthology.orchestra.run.smithi036.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2022-06-17T12:43:30.852 INFO:teuthology.orchestra.run.smithi036.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2022-06-17T12:43:30.903 INFO:teuthology.orchestra.run.smithi136.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2022-06-17T12:43:30.903 INFO:teuthology.orchestra.run.smithi136.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2022-06-17T12:43:31.370 INFO:teuthology.orchestra.run.smithi143.stdout:Get:74 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.16-260-gb3641a55-1focal [3,257 kB] 2022-06-17T12:43:32.378 INFO:teuthology.orchestra.run.smithi107.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.16-260-gb3641a55-1focal [5,344 kB] 2022-06-17T12:43:32.787 INFO:teuthology.orchestra.run.smithi070.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.16-260-gb3641a55-1focal [5,344 kB] 2022-06-17T12:43:32.931 INFO:teuthology.orchestra.run.smithi107.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.16-260-gb3641a55-1focal [163 kB] 2022-06-17T12:43:32.993 INFO:teuthology.orchestra.run.smithi107.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.16-260-gb3641a55-1focal [1,403 kB] 2022-06-17T12:43:33.155 INFO:teuthology.orchestra.run.smithi107.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.16-260-gb3641a55-1focal [6,165 kB] 2022-06-17T12:43:33.419 INFO:teuthology.orchestra.run.smithi070.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.16-260-gb3641a55-1focal [163 kB] 2022-06-17T12:43:33.419 INFO:teuthology.orchestra.run.smithi070.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.16-260-gb3641a55-1focal [1,403 kB] 2022-06-17T12:43:33.552 INFO:teuthology.orchestra.run.smithi036.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.16-260-gb3641a55-1focal [5,344 kB] 2022-06-17T12:43:33.597 INFO:teuthology.orchestra.run.smithi070.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.16-260-gb3641a55-1focal [6,165 kB] 2022-06-17T12:43:33.802 INFO:teuthology.orchestra.run.smithi107.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.16-260-gb3641a55-1focal [24.0 MB] 2022-06-17T12:43:33.985 INFO:teuthology.orchestra.run.smithi136.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.16-260-gb3641a55-1focal [5,344 kB] 2022-06-17T12:43:34.387 INFO:teuthology.orchestra.run.smithi036.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.16-260-gb3641a55-1focal [163 kB] 2022-06-17T12:43:34.388 INFO:teuthology.orchestra.run.smithi036.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.16-260-gb3641a55-1focal [1,403 kB] 2022-06-17T12:43:34.474 INFO:teuthology.orchestra.run.smithi143.stdout:Get:75 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.16-260-gb3641a55-1focal [463 kB] 2022-06-17T12:43:34.638 INFO:teuthology.orchestra.run.smithi136.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.16-260-gb3641a55-1focal [163 kB] 2022-06-17T12:43:34.653 INFO:teuthology.orchestra.run.smithi136.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.16-260-gb3641a55-1focal [1,403 kB] 2022-06-17T12:43:34.702 INFO:teuthology.orchestra.run.smithi036.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.16-260-gb3641a55-1focal [6,165 kB] 2022-06-17T12:43:34.702 INFO:teuthology.orchestra.run.smithi070.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.16-260-gb3641a55-1focal [24.0 MB] 2022-06-17T12:43:34.751 INFO:teuthology.orchestra.run.smithi143.stdout:Get:76 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.16-260-gb3641a55-1focal [365 kB] 2022-06-17T12:43:34.850 INFO:teuthology.orchestra.run.smithi136.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.16-260-gb3641a55-1focal [6,165 kB] 2022-06-17T12:43:34.973 INFO:teuthology.orchestra.run.smithi143.stdout:Get:77 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.16-260-gb3641a55-1focal [20.9 kB] 2022-06-17T12:43:34.989 INFO:teuthology.orchestra.run.smithi143.stdout:Get:78 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.16-260-gb3641a55-1focal [156 kB] 2022-06-17T12:43:35.056 INFO:teuthology.orchestra.run.smithi143.stdout:Get:79 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.16-260-gb3641a55-1focal [29.4 kB] 2022-06-17T12:43:35.079 INFO:teuthology.orchestra.run.smithi143.stdout:Get:80 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.16-260-gb3641a55-1focal [363 kB] 2022-06-17T12:43:35.294 INFO:teuthology.orchestra.run.smithi143.stdout:Get:81 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.16-260-gb3641a55-1focal [3,145 kB] 2022-06-17T12:43:35.814 INFO:teuthology.orchestra.run.smithi136.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.16-260-gb3641a55-1focal [24.0 MB] 2022-06-17T12:43:35.815 INFO:teuthology.orchestra.run.smithi036.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.16-260-gb3641a55-1focal [24.0 MB] 2022-06-17T12:43:35.940 INFO:teuthology.orchestra.run.smithi107.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.16-260-gb3641a55-1focal [4,032 B] 2022-06-17T12:43:35.941 INFO:teuthology.orchestra.run.smithi107.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.16-260-gb3641a55-1focal [538 kB] 2022-06-17T12:43:35.952 INFO:teuthology.orchestra.run.smithi107.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.16-260-gb3641a55-1focal [2,007 kB] 2022-06-17T12:43:36.053 INFO:teuthology.orchestra.run.smithi107.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.16-260-gb3641a55-1focal [53.2 kB] 2022-06-17T12:43:36.054 INFO:teuthology.orchestra.run.smithi107.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.16-260-gb3641a55-1focal [58.3 kB] 2022-06-17T12:43:36.054 INFO:teuthology.orchestra.run.smithi107.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.16-260-gb3641a55-1focal [3,218 kB] 2022-06-17T12:43:36.314 INFO:teuthology.orchestra.run.smithi107.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.16-260-gb3641a55-1focal [34.4 kB] 2022-06-17T12:43:36.314 INFO:teuthology.orchestra.run.smithi107.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.16-260-gb3641a55-1focal [7,889 kB] 2022-06-17T12:43:36.953 INFO:teuthology.orchestra.run.smithi107.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.16-260-gb3641a55-1focal [4,148 B] 2022-06-17T12:43:36.972 INFO:teuthology.orchestra.run.smithi107.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.16-260-gb3641a55-1focal [40.2 MB] 2022-06-17T12:43:37.026 INFO:teuthology.orchestra.run.smithi143.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.16-260-gb3641a55-1focal [116 kB] 2022-06-17T12:43:37.096 INFO:teuthology.orchestra.run.smithi143.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.16-260-gb3641a55-1focal [360 kB] 2022-06-17T12:43:37.282 INFO:teuthology.orchestra.run.smithi143.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.16-260-gb3641a55-1focal [20.1 MB] 2022-06-17T12:43:39.449 INFO:teuthology.orchestra.run.smithi136.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.16-260-gb3641a55-1focal [4,032 B] 2022-06-17T12:43:39.450 INFO:teuthology.orchestra.run.smithi136.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.16-260-gb3641a55-1focal [538 kB] 2022-06-17T12:43:39.499 INFO:teuthology.orchestra.run.smithi136.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.16-260-gb3641a55-1focal [2,007 kB] 2022-06-17T12:43:39.781 INFO:teuthology.orchestra.run.smithi136.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.16-260-gb3641a55-1focal [53.2 kB] 2022-06-17T12:43:39.797 INFO:teuthology.orchestra.run.smithi136.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.16-260-gb3641a55-1focal [58.3 kB] 2022-06-17T12:43:39.823 INFO:teuthology.orchestra.run.smithi136.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.16-260-gb3641a55-1focal [3,218 kB] 2022-06-17T12:43:40.043 INFO:teuthology.orchestra.run.smithi070.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.16-260-gb3641a55-1focal [4,032 B] 2022-06-17T12:43:40.044 INFO:teuthology.orchestra.run.smithi070.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.16-260-gb3641a55-1focal [538 kB] 2022-06-17T12:43:40.111 INFO:teuthology.orchestra.run.smithi070.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.16-260-gb3641a55-1focal [2,007 kB] 2022-06-17T12:43:40.297 INFO:teuthology.orchestra.run.smithi036.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.16-260-gb3641a55-1focal [4,032 B] 2022-06-17T12:43:40.298 INFO:teuthology.orchestra.run.smithi036.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.16-260-gb3641a55-1focal [538 kB] 2022-06-17T12:43:40.322 INFO:teuthology.orchestra.run.smithi136.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.16-260-gb3641a55-1focal [34.4 kB] 2022-06-17T12:43:40.323 INFO:teuthology.orchestra.run.smithi136.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.16-260-gb3641a55-1focal [7,889 kB] 2022-06-17T12:43:40.363 INFO:teuthology.orchestra.run.smithi036.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.16-260-gb3641a55-1focal [2,007 kB] 2022-06-17T12:43:40.544 INFO:teuthology.orchestra.run.smithi070.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.16-260-gb3641a55-1focal [53.2 kB] 2022-06-17T12:43:40.546 INFO:teuthology.orchestra.run.smithi070.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.16-260-gb3641a55-1focal [58.3 kB] 2022-06-17T12:43:40.549 INFO:teuthology.orchestra.run.smithi070.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.16-260-gb3641a55-1focal [3,218 kB] 2022-06-17T12:43:40.601 INFO:teuthology.orchestra.run.smithi107.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.16-260-gb3641a55-1focal [15.9 kB] 2022-06-17T12:43:40.601 INFO:teuthology.orchestra.run.smithi107.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.16-260-gb3641a55-1focal [18.1 kB] 2022-06-17T12:43:40.602 INFO:teuthology.orchestra.run.smithi107.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.16-260-gb3641a55-1focal [9,670 kB] 2022-06-17T12:43:40.721 INFO:teuthology.orchestra.run.smithi036.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.16-260-gb3641a55-1focal [53.2 kB] 2022-06-17T12:43:40.738 INFO:teuthology.orchestra.run.smithi036.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.16-260-gb3641a55-1focal [58.3 kB] 2022-06-17T12:43:40.739 INFO:teuthology.orchestra.run.smithi036.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.16-260-gb3641a55-1focal [3,218 kB] 2022-06-17T12:43:41.263 INFO:teuthology.orchestra.run.smithi070.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.16-260-gb3641a55-1focal [34.4 kB] 2022-06-17T12:43:41.299 INFO:teuthology.orchestra.run.smithi070.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.16-260-gb3641a55-1focal [7,889 kB] 2022-06-17T12:43:41.331 INFO:teuthology.orchestra.run.smithi107.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.16-260-gb3641a55-1focal [61.0 kB] 2022-06-17T12:43:41.372 INFO:teuthology.orchestra.run.smithi036.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.16-260-gb3641a55-1focal [34.4 kB] 2022-06-17T12:43:41.372 INFO:teuthology.orchestra.run.smithi036.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.16-260-gb3641a55-1focal [7,889 kB] 2022-06-17T12:43:41.507 INFO:teuthology.orchestra.run.smithi136.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.16-260-gb3641a55-1focal [4,148 B] 2022-06-17T12:43:41.508 INFO:teuthology.orchestra.run.smithi136.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.16-260-gb3641a55-1focal [40.2 MB] 2022-06-17T12:43:41.944 INFO:teuthology.orchestra.run.smithi107.stdout:Fetched 153 MB in 13s (12.1 MB/s) 2022-06-17T12:43:42.069 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pyperclip. 2022-06-17T12:43:42.134 INFO:teuthology.orchestra.run.smithi107.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 138767 files and directories currently installed.) 2022-06-17T12:43:42.146 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2022-06-17T12:43:42.177 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2022-06-17T12:43:42.373 INFO:teuthology.orchestra.run.smithi036.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.16-260-gb3641a55-1focal [4,148 B] 2022-06-17T12:43:42.389 INFO:teuthology.orchestra.run.smithi036.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.16-260-gb3641a55-1focal [40.2 MB] 2022-06-17T12:43:42.458 INFO:teuthology.orchestra.run.smithi070.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.16-260-gb3641a55-1focal [4,148 B] 2022-06-17T12:43:42.460 INFO:teuthology.orchestra.run.smithi070.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.16-260-gb3641a55-1focal [40.2 MB] 2022-06-17T12:43:42.496 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-wcwidth. 2022-06-17T12:43:42.515 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2022-06-17T12:43:42.568 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2022-06-17T12:43:42.720 INFO:teuthology.orchestra.run.smithi143.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.16-260-gb3641a55-1focal [5,344 kB] 2022-06-17T12:43:42.790 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-cmd2. 2022-06-17T12:43:42.818 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2022-06-17T12:43:42.844 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2022-06-17T12:43:43.235 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2022-06-17T12:43:43.236 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2022-06-17T12:43:43.270 INFO:teuthology.orchestra.run.smithi143.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.16-260-gb3641a55-1focal [163 kB] 2022-06-17T12:43:43.311 INFO:teuthology.orchestra.run.smithi143.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.16-260-gb3641a55-1focal [1,403 kB] 2022-06-17T12:43:43.318 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2022-06-17T12:43:43.467 INFO:teuthology.orchestra.run.smithi143.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.16-260-gb3641a55-1focal [6,165 kB] 2022-06-17T12:43:43.683 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../004-librbd1_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:43.814 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking librbd1 (15.2.16-260-gb3641a55-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-06-17T12:43:44.085 INFO:teuthology.orchestra.run.smithi143.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.16-260-gb3641a55-1focal [24.0 MB] 2022-06-17T12:43:44.355 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../005-librados2_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:44.480 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking librados2 (15.2.16-260-gb3641a55-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-06-17T12:43:45.123 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libcephfs2. 2022-06-17T12:43:45.139 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../006-libcephfs2_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:45.183 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libcephfs2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:45.467 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-rados. 2022-06-17T12:43:45.482 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../007-python3-rados_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:45.518 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-rados (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:45.903 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-ceph-argparse. 2022-06-17T12:43:45.922 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:43:45.970 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-ceph-argparse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:46.271 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-cephfs. 2022-06-17T12:43:46.272 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../009-python3-cephfs_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:46.307 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-cephfs (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:46.338 INFO:teuthology.orchestra.run.smithi143.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.16-260-gb3641a55-1focal [4,032 B] 2022-06-17T12:43:46.338 INFO:teuthology.orchestra.run.smithi143.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.16-260-gb3641a55-1focal [538 kB] 2022-06-17T12:43:46.339 INFO:teuthology.orchestra.run.smithi143.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.16-260-gb3641a55-1focal [2,007 kB] 2022-06-17T12:43:46.428 INFO:teuthology.orchestra.run.smithi143.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.16-260-gb3641a55-1focal [53.2 kB] 2022-06-17T12:43:46.429 INFO:teuthology.orchestra.run.smithi143.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.16-260-gb3641a55-1focal [58.3 kB] 2022-06-17T12:43:46.430 INFO:teuthology.orchestra.run.smithi143.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.16-260-gb3641a55-1focal [3,218 kB] 2022-06-17T12:43:46.647 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-ceph-common. 2022-06-17T12:43:46.649 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:43:46.694 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-ceph-common (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:46.790 INFO:teuthology.orchestra.run.smithi143.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.16-260-gb3641a55-1focal [34.4 kB] 2022-06-17T12:43:46.813 INFO:teuthology.orchestra.run.smithi143.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.16-260-gb3641a55-1focal [7,889 kB] 2022-06-17T12:43:46.886 INFO:teuthology.orchestra.run.smithi136.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.16-260-gb3641a55-1focal [15.9 kB] 2022-06-17T12:43:46.887 INFO:teuthology.orchestra.run.smithi136.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.16-260-gb3641a55-1focal [18.1 kB] 2022-06-17T12:43:46.887 INFO:teuthology.orchestra.run.smithi136.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.16-260-gb3641a55-1focal [9,670 kB] 2022-06-17T12:43:46.981 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-prettytable. 2022-06-17T12:43:46.998 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2022-06-17T12:43:47.037 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2022-06-17T12:43:47.297 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-rbd. 2022-06-17T12:43:47.309 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../012-python3-rbd_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:47.354 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-rbd (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:47.354 INFO:teuthology.orchestra.run.smithi143.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.16-260-gb3641a55-1focal [4,148 B] 2022-06-17T12:43:47.355 INFO:teuthology.orchestra.run.smithi143.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.16-260-gb3641a55-1focal [40.2 MB] 2022-06-17T12:43:47.780 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package librabbitmq4:amd64. 2022-06-17T12:43:47.785 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2022-06-17T12:43:47.829 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2022-06-17T12:43:48.182 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package librdkafka1:amd64. 2022-06-17T12:43:48.197 INFO:teuthology.orchestra.run.smithi136.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.16-260-gb3641a55-1focal [61.0 kB] 2022-06-17T12:43:48.198 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2022-06-17T12:43:48.230 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-06-17T12:43:48.536 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package librgw2. 2022-06-17T12:43:48.555 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../015-librgw2_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:48.585 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking librgw2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:48.690 INFO:teuthology.orchestra.run.smithi036.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.16-260-gb3641a55-1focal [15.9 kB] 2022-06-17T12:43:48.691 INFO:teuthology.orchestra.run.smithi036.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.16-260-gb3641a55-1focal [18.1 kB] 2022-06-17T12:43:48.691 INFO:teuthology.orchestra.run.smithi036.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.16-260-gb3641a55-1focal [9,670 kB] 2022-06-17T12:43:48.792 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 153 MB in 19s (7,929 kB/s) 2022-06-17T12:43:48.910 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pyperclip. 2022-06-17T12:43:48.966 INFO:teuthology.orchestra.run.smithi136.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 138767 files and directories currently installed.) 2022-06-17T12:43:48.967 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2022-06-17T12:43:49.022 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2022-06-17T12:43:49.136 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-rgw. 2022-06-17T12:43:49.142 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../016-python3-rgw_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:49.186 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-rgw (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:49.251 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-wcwidth. 2022-06-17T12:43:49.275 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2022-06-17T12:43:49.300 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2022-06-17T12:43:49.435 INFO:teuthology.orchestra.run.smithi070.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.16-260-gb3641a55-1focal [15.9 kB] 2022-06-17T12:43:49.436 INFO:teuthology.orchestra.run.smithi070.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.16-260-gb3641a55-1focal [18.1 kB] 2022-06-17T12:43:49.436 INFO:teuthology.orchestra.run.smithi070.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.16-260-gb3641a55-1focal [9,670 kB] 2022-06-17T12:43:49.540 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libleveldb1d:amd64. 2022-06-17T12:43:49.555 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2022-06-17T12:43:49.604 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-06-17T12:43:49.633 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cmd2. 2022-06-17T12:43:49.664 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2022-06-17T12:43:49.721 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2022-06-17T12:43:49.750 INFO:teuthology.orchestra.run.smithi036.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.16-260-gb3641a55-1focal [61.0 kB] 2022-06-17T12:43:49.955 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package liboath0. 2022-06-17T12:43:49.966 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2022-06-17T12:43:49.984 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2022-06-17T12:43:50.088 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2022-06-17T12:43:50.115 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2022-06-17T12:43:50.156 INFO:teuthology.orchestra.run.smithi143.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.16-260-gb3641a55-1focal [15.9 kB] 2022-06-17T12:43:50.157 INFO:teuthology.orchestra.run.smithi143.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.16-260-gb3641a55-1focal [18.1 kB] 2022-06-17T12:43:50.157 INFO:teuthology.orchestra.run.smithi143.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.16-260-gb3641a55-1focal [9,670 kB] 2022-06-17T12:43:50.192 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2022-06-17T12:43:50.252 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libradosstriper1. 2022-06-17T12:43:50.271 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../019-libradosstriper1_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:50.306 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libradosstriper1 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:50.359 INFO:teuthology.orchestra.run.smithi036.stdout:Fetched 153 MB in 21s (7,324 kB/s) 2022-06-17T12:43:50.398 INFO:teuthology.orchestra.run.smithi070.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.16-260-gb3641a55-1focal [61.0 kB] 2022-06-17T12:43:50.518 INFO:teuthology.orchestra.run.smithi143.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/octopus/b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.16-260-gb3641a55-1focal [61.0 kB] 2022-06-17T12:43:50.536 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-pyperclip. 2022-06-17T12:43:50.537 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../004-librbd1_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:50.623 INFO:teuthology.orchestra.run.smithi036.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 138767 files and directories currently installed.) 2022-06-17T12:43:50.624 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2022-06-17T12:43:50.627 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librbd1 (15.2.16-260-gb3641a55-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-06-17T12:43:50.648 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2022-06-17T12:43:50.808 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-common. 2022-06-17T12:43:50.809 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../020-ceph-common_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:50.852 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-common (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:50.890 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-wcwidth. 2022-06-17T12:43:50.915 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2022-06-17T12:43:50.969 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2022-06-17T12:43:51.033 INFO:teuthology.orchestra.run.smithi070.stdout:Fetched 153 MB in 22s (7,067 kB/s) 2022-06-17T12:43:51.097 INFO:teuthology.orchestra.run.smithi143.stdout:Fetched 153 MB in 21s (7,182 kB/s) 2022-06-17T12:43:51.171 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-pyperclip. 2022-06-17T12:43:51.172 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../005-librados2_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:51.203 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pyperclip. 2022-06-17T12:43:51.214 INFO:teuthology.orchestra.run.smithi070.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 138767 files and directories currently installed.) 2022-06-17T12:43:51.219 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2022-06-17T12:43:51.270 INFO:teuthology.orchestra.run.smithi143.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 138767 files and directories currently installed.) 2022-06-17T12:43:51.270 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2022-06-17T12:43:51.271 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2022-06-17T12:43:51.272 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-cmd2. 2022-06-17T12:43:51.298 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2022-06-17T12:43:51.299 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librados2 (15.2.16-260-gb3641a55-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-06-17T12:43:51.313 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2022-06-17T12:43:51.328 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2022-06-17T12:43:51.623 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-wcwidth. 2022-06-17T12:43:51.624 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-wcwidth. 2022-06-17T12:43:51.635 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2022-06-17T12:43:51.645 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2022-06-17T12:43:51.695 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2022-06-17T12:43:51.695 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2022-06-17T12:43:51.712 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2022-06-17T12:43:51.713 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2022-06-17T12:43:51.790 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2022-06-17T12:43:51.978 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libcephfs2. 2022-06-17T12:43:51.978 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-cmd2. 2022-06-17T12:43:51.986 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2022-06-17T12:43:51.999 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../006-libcephfs2_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:52.029 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2022-06-17T12:43:52.031 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libcephfs2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:52.041 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-cmd2. 2022-06-17T12:43:52.048 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2022-06-17T12:43:52.090 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2022-06-17T12:43:52.180 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../004-librbd1_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:52.279 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking librbd1 (15.2.16-260-gb3641a55-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-06-17T12:43:52.302 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-rados. 2022-06-17T12:43:52.303 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2022-06-17T12:43:52.320 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2022-06-17T12:43:52.320 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../007-python3-rados_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:52.397 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2022-06-17T12:43:52.398 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-rados (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:52.411 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2022-06-17T12:43:52.420 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2022-06-17T12:43:52.511 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2022-06-17T12:43:52.726 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-ceph-argparse. 2022-06-17T12:43:52.741 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:43:52.751 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../004-librbd1_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:52.821 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-ceph-argparse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:52.839 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../004-librbd1_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:52.855 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../005-librados2_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:52.866 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking librbd1 (15.2.16-260-gb3641a55-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-06-17T12:43:52.887 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-base. 2022-06-17T12:43:52.903 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../021-ceph-base_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:52.947 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-base (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:52.978 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking librados2 (15.2.16-260-gb3641a55-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-06-17T12:43:52.979 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking librbd1 (15.2.16-260-gb3641a55-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-06-17T12:43:53.109 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cephfs. 2022-06-17T12:43:53.127 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../009-python3-cephfs_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:53.170 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cephfs (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:53.409 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../005-librados2_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:53.507 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-ceph-common. 2022-06-17T12:43:53.517 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:43:53.533 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking librados2 (15.2.16-260-gb3641a55-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-06-17T12:43:53.555 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../005-librados2_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:53.574 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-ceph-common (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:53.617 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libcephfs2. 2022-06-17T12:43:53.657 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../006-libcephfs2_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:53.698 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking librados2 (15.2.16-260-gb3641a55-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-06-17T12:43:53.699 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libcephfs2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:53.823 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-prettytable. 2022-06-17T12:43:53.850 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2022-06-17T12:43:53.870 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2022-06-17T12:43:53.882 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-cryptography. 2022-06-17T12:43:53.892 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2022-06-17T12:43:53.933 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2022-06-17T12:43:53.961 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-rados. 2022-06-17T12:43:53.978 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../007-python3-rados_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:54.021 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-rados (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:54.124 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-rbd. 2022-06-17T12:43:54.150 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../012-python3-rbd_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:54.166 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libcephfs2. 2022-06-17T12:43:54.181 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-rbd (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:54.182 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../006-libcephfs2_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:54.227 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libcephfs2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:54.370 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package libcephfs2. 2022-06-17T12:43:54.406 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../006-libcephfs2_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:54.407 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-ceph-argparse. 2022-06-17T12:43:54.408 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-openssl. 2022-06-17T12:43:54.413 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2022-06-17T12:43:54.423 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:43:54.429 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking libcephfs2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:54.456 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-ceph-argparse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:54.457 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2022-06-17T12:43:54.535 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-rados. 2022-06-17T12:43:54.563 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../007-python3-rados_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:54.615 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package librabbitmq4:amd64. 2022-06-17T12:43:54.616 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-rados (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:54.617 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2022-06-17T12:43:54.663 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2022-06-17T12:43:54.782 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2022-06-17T12:43:54.782 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-rados. 2022-06-17T12:43:54.804 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-cephfs. 2022-06-17T12:43:54.806 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:43:54.807 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../007-python3-rados_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:54.809 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../009-python3-cephfs_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:54.851 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mgr-modules-core (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:54.867 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-rados (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:54.868 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-cephfs (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:54.959 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-ceph-argparse. 2022-06-17T12:43:54.980 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:43:54.997 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package librdkafka1:amd64. 2022-06-17T12:43:55.053 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-ceph-argparse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:55.054 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2022-06-17T12:43:55.055 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-06-17T12:43:55.175 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-ceph-common. 2022-06-17T12:43:55.190 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:43:55.207 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-ceph-argparse. 2022-06-17T12:43:55.211 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-bcrypt. 2022-06-17T12:43:55.220 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-ceph-common (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:55.265 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2022-06-17T12:43:55.268 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:43:55.270 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-ceph-argparse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:55.272 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2022-06-17T12:43:55.272 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-cephfs. 2022-06-17T12:43:55.273 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../009-python3-cephfs_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:55.299 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-cephfs (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:55.373 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package librgw2. 2022-06-17T12:43:55.391 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../015-librgw2_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:55.424 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librgw2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:55.527 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-cherrypy3. 2022-06-17T12:43:55.539 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-cephfs. 2022-06-17T12:43:55.540 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2022-06-17T12:43:55.556 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-prettytable. 2022-06-17T12:43:55.557 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../009-python3-cephfs_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:55.565 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2022-06-17T12:43:55.572 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2022-06-17T12:43:55.593 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-cephfs (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:55.602 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-ceph-common. 2022-06-17T12:43:55.605 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2022-06-17T12:43:55.618 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:43:55.661 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-ceph-common (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:55.846 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-rbd. 2022-06-17T12:43:55.863 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../012-python3-rbd_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:55.886 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-jwt. 2022-06-17T12:43:55.907 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-ceph-common. 2022-06-17T12:43:55.908 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2_all.deb ... 2022-06-17T12:43:55.915 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:43:55.925 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-rbd (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:55.941 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2) ... 2022-06-17T12:43:55.946 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-ceph-common (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:55.974 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-prettytable. 2022-06-17T12:43:55.975 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-rgw. 2022-06-17T12:43:55.991 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2022-06-17T12:43:55.994 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../016-python3-rgw_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:56.032 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2022-06-17T12:43:56.092 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-rgw (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:56.213 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-logutils. 2022-06-17T12:43:56.238 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2022-06-17T12:43:56.255 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-prettytable. 2022-06-17T12:43:56.266 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2022-06-17T12:43:56.304 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package librabbitmq4:amd64. 2022-06-17T12:43:56.305 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-rbd. 2022-06-17T12:43:56.307 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2022-06-17T12:43:56.321 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libleveldb1d:amd64. 2022-06-17T12:43:56.321 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2022-06-17T12:43:56.322 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../012-python3-rbd_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:56.365 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-logutils (0.3.3-7) ... 2022-06-17T12:43:56.366 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-rbd (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:56.367 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2022-06-17T12:43:56.409 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-06-17T12:43:56.429 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2022-06-17T12:43:56.588 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-rbd. 2022-06-17T12:43:56.624 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../012-python3-rbd_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:56.643 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-rbd (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:56.650 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package librdkafka1:amd64. 2022-06-17T12:43:56.652 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-markupsafe. 2022-06-17T12:43:56.664 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2022-06-17T12:43:56.665 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2022-06-17T12:43:56.698 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-06-17T12:43:56.727 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2022-06-17T12:43:56.728 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package librabbitmq4:amd64. 2022-06-17T12:43:56.729 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2022-06-17T12:43:56.769 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2022-06-17T12:43:56.791 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package liboath0. 2022-06-17T12:43:56.817 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2022-06-17T12:43:56.873 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2022-06-17T12:43:57.011 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-mako. 2022-06-17T12:43:57.035 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package librabbitmq4:amd64. 2022-06-17T12:43:57.042 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2022-06-17T12:43:57.072 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2022-06-17T12:43:57.074 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package librgw2. 2022-06-17T12:43:57.075 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-06-17T12:43:57.102 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../015-librgw2_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:57.106 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package librdkafka1:amd64. 2022-06-17T12:43:57.127 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking librgw2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:57.127 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2022-06-17T12:43:57.142 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2022-06-17T12:43:57.164 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-06-17T12:43:57.185 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libradosstriper1. 2022-06-17T12:43:57.211 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../019-libradosstriper1_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:57.270 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libradosstriper1 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:57.376 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-simplegeneric. 2022-06-17T12:43:57.403 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package librdkafka1:amd64. 2022-06-17T12:43:57.403 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2022-06-17T12:43:57.438 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2022-06-17T12:43:57.447 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2022-06-17T12:43:57.457 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-06-17T12:43:57.534 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package librgw2. 2022-06-17T12:43:57.547 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../015-librgw2_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:57.584 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking librgw2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:57.632 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-common. 2022-06-17T12:43:57.649 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../020-ceph-common_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:57.680 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-common (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:57.728 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-rgw. 2022-06-17T12:43:57.746 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../016-python3-rgw_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:57.758 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-singledispatch. 2022-06-17T12:43:57.777 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2022-06-17T12:43:57.795 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-rgw (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:57.796 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package librgw2. 2022-06-17T12:43:57.811 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../015-librgw2_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:57.818 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2022-06-17T12:43:57.857 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking librgw2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:58.115 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-rgw. 2022-06-17T12:43:58.128 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../016-python3-rgw_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:58.129 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libleveldb1d:amd64. 2022-06-17T12:43:58.171 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2022-06-17T12:43:58.172 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-webob. 2022-06-17T12:43:58.172 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2022-06-17T12:43:58.195 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-rgw (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:58.221 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-06-17T12:43:58.222 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2022-06-17T12:43:58.427 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-rgw. 2022-06-17T12:43:58.449 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../016-python3-rgw_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:58.487 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-rgw (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:58.489 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libleveldb1d:amd64. 2022-06-17T12:43:58.509 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2022-06-17T12:43:58.553 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package liboath0. 2022-06-17T12:43:58.556 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-06-17T12:43:58.557 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-soupsieve. 2022-06-17T12:43:58.558 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2022-06-17T12:43:58.576 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2022-06-17T12:43:58.651 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2022-06-17T12:43:58.652 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2022-06-17T12:43:58.850 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package libleveldb1d:amd64. 2022-06-17T12:43:58.870 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2022-06-17T12:43:58.906 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-06-17T12:43:58.938 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libradosstriper1. 2022-06-17T12:43:58.940 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../019-libradosstriper1_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:58.940 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package liboath0. 2022-06-17T12:43:58.941 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2022-06-17T12:43:58.948 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-bs4. 2022-06-17T12:43:58.948 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libradosstriper1 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:58.970 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2022-06-17T12:43:58.989 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2022-06-17T12:43:59.000 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2022-06-17T12:43:59.185 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package liboath0. 2022-06-17T12:43:59.212 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2022-06-17T12:43:59.247 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libradosstriper1. 2022-06-17T12:43:59.247 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2022-06-17T12:43:59.263 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../019-libradosstriper1_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:59.304 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libradosstriper1 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:59.384 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-common. 2022-06-17T12:43:59.394 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../020-ceph-common_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:59.396 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-waitress. 2022-06-17T12:43:59.412 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2022-06-17T12:43:59.421 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-common (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:59.457 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2022-06-17T12:43:59.595 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package libradosstriper1. 2022-06-17T12:43:59.623 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../019-libradosstriper1_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:59.687 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking libradosstriper1 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:59.750 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-common. 2022-06-17T12:43:59.776 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../020-ceph-common_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:59.820 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-tempita. 2022-06-17T12:43:59.820 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-common (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:43:59.872 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2022-06-17T12:43:59.873 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-base. 2022-06-17T12:43:59.873 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../021-ceph-base_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:43:59.886 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-tempita (0.5.2-6) ... 2022-06-17T12:43:59.913 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-base (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:00.175 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package ceph-common. 2022-06-17T12:44:00.190 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-paste. 2022-06-17T12:44:00.191 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../020-ceph-common_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:00.208 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2022-06-17T12:44:00.234 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking ceph-common (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:00.249 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2022-06-17T12:44:00.603 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2022-06-17T12:44:00.621 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2022-06-17T12:44:00.655 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2022-06-17T12:44:00.866 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cryptography. 2022-06-17T12:44:00.890 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2022-06-17T12:44:00.931 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2022-06-17T12:44:00.986 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pastedeploy. 2022-06-17T12:44:01.003 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2022-06-17T12:44:01.053 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2022-06-17T12:44:01.370 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-webtest. 2022-06-17T12:44:01.397 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2022-06-17T12:44:01.956 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-base. 2022-06-17T12:44:01.962 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../021-ceph-base_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:02.011 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-webtest (2.0.32-2) ... 2022-06-17T12:44:02.013 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-base (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:02.354 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pecan. 2022-06-17T12:44:02.364 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2022-06-17T12:44:02.405 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2022-06-17T12:44:02.451 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-base. 2022-06-17T12:44:02.451 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../021-ceph-base_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:02.490 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-base (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:02.510 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-openssl. 2022-06-17T12:44:02.523 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2022-06-17T12:44:02.568 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2022-06-17T12:44:02.657 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package ceph-base. 2022-06-17T12:44:02.676 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../021-ceph-base_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:02.728 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking ceph-base (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:02.792 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-werkzeug. 2022-06-17T12:44:02.809 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2022-06-17T12:44:02.849 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2022-06-17T12:44:02.856 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2022-06-17T12:44:02.872 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:02.913 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-modules-core (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:02.915 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-cryptography. 2022-06-17T12:44:02.934 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2022-06-17T12:44:02.966 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2022-06-17T12:44:03.218 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mgr. 2022-06-17T12:44:03.234 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../044-ceph-mgr_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:03.294 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-bcrypt. 2022-06-17T12:44:03.297 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mgr (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:03.313 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2022-06-17T12:44:03.353 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2022-06-17T12:44:03.360 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-cryptography. 2022-06-17T12:44:03.382 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2022-06-17T12:44:03.401 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-openssl. 2022-06-17T12:44:03.424 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2022-06-17T12:44:03.424 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2022-06-17T12:44:03.451 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2022-06-17T12:44:03.674 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-cryptography. 2022-06-17T12:44:03.679 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2022-06-17T12:44:03.696 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cherrypy3. 2022-06-17T12:44:03.731 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2022-06-17T12:44:03.737 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2022-06-17T12:44:03.737 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mon. 2022-06-17T12:44:03.754 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2022-06-17T12:44:03.756 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../045-ceph-mon_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:03.777 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:03.778 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2022-06-17T12:44:03.794 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mon (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:03.826 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-mgr-modules-core (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:03.854 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-openssl. 2022-06-17T12:44:03.873 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2022-06-17T12:44:03.902 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2022-06-17T12:44:04.154 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2022-06-17T12:44:04.169 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-jwt. 2022-06-17T12:44:04.176 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:04.178 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-openssl. 2022-06-17T12:44:04.184 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2_all.deb ... 2022-06-17T12:44:04.185 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-bcrypt. 2022-06-17T12:44:04.186 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2022-06-17T12:44:04.201 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2022-06-17T12:44:04.204 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr-modules-core (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:04.234 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2) ... 2022-06-17T12:44:04.243 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2022-06-17T12:44:04.255 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2022-06-17T12:44:04.548 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-bcrypt. 2022-06-17T12:44:04.551 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-cherrypy3. 2022-06-17T12:44:04.568 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2022-06-17T12:44:04.570 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-logutils. 2022-06-17T12:44:04.571 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2022-06-17T12:44:04.586 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2022-06-17T12:44:04.591 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2022-06-17T12:44:04.611 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:04.628 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-logutils (0.3.3-7) ... 2022-06-17T12:44:04.629 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2022-06-17T12:44:04.638 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2022-06-17T12:44:04.653 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking ceph-mgr-modules-core (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:04.756 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-osd. 2022-06-17T12:44:04.783 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../046-ceph-osd_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:04.816 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-osd (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:04.967 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-cherrypy3. 2022-06-17T12:44:04.967 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-markupsafe. 2022-06-17T12:44:04.972 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-jwt. 2022-06-17T12:44:04.979 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2022-06-17T12:44:04.981 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2022-06-17T12:44:04.997 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2_all.deb ... 2022-06-17T12:44:05.026 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2022-06-17T12:44:05.036 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2022-06-17T12:44:05.074 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-bcrypt. 2022-06-17T12:44:05.075 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2) ... 2022-06-17T12:44:05.076 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2022-06-17T12:44:05.125 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2022-06-17T12:44:05.351 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-mako. 2022-06-17T12:44:05.362 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2022-06-17T12:44:05.402 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-jwt. 2022-06-17T12:44:05.403 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-logutils. 2022-06-17T12:44:05.405 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-06-17T12:44:05.405 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2_all.deb ... 2022-06-17T12:44:05.406 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2022-06-17T12:44:05.446 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2) ... 2022-06-17T12:44:05.449 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-logutils (0.3.3-7) ... 2022-06-17T12:44:05.456 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-cherrypy3. 2022-06-17T12:44:05.479 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2022-06-17T12:44:05.553 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2022-06-17T12:44:05.765 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-logutils. 2022-06-17T12:44:05.783 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2022-06-17T12:44:05.785 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-markupsafe. 2022-06-17T12:44:05.807 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2022-06-17T12:44:05.807 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-simplegeneric. 2022-06-17T12:44:05.822 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2022-06-17T12:44:05.826 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-logutils (0.3.3-7) ... 2022-06-17T12:44:05.845 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2022-06-17T12:44:05.893 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2022-06-17T12:44:05.958 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-jwt. 2022-06-17T12:44:05.966 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2_all.deb ... 2022-06-17T12:44:06.003 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2) ... 2022-06-17T12:44:06.146 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-markupsafe. 2022-06-17T12:44:06.147 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2022-06-17T12:44:06.167 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-mako. 2022-06-17T12:44:06.183 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2022-06-17T12:44:06.195 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2022-06-17T12:44:06.234 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-06-17T12:44:06.241 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-singledispatch. 2022-06-17T12:44:06.275 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2022-06-17T12:44:06.298 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2022-06-17T12:44:06.355 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-logutils. 2022-06-17T12:44:06.370 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2022-06-17T12:44:06.414 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-logutils (0.3.3-7) ... 2022-06-17T12:44:06.522 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-mako. 2022-06-17T12:44:06.546 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2022-06-17T12:44:06.579 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-simplegeneric. 2022-06-17T12:44:06.591 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-06-17T12:44:06.606 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2022-06-17T12:44:06.639 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-webob. 2022-06-17T12:44:06.640 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2022-06-17T12:44:06.666 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2022-06-17T12:44:06.694 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2022-06-17T12:44:06.732 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-markupsafe. 2022-06-17T12:44:06.751 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2022-06-17T12:44:06.815 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2022-06-17T12:44:06.958 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-simplegeneric. 2022-06-17T12:44:06.993 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2022-06-17T12:44:07.010 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-singledispatch. 2022-06-17T12:44:07.016 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2022-06-17T12:44:07.048 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2022-06-17T12:44:07.056 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-soupsieve. 2022-06-17T12:44:07.078 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2022-06-17T12:44:07.079 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2022-06-17T12:44:07.124 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2022-06-17T12:44:07.197 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph. 2022-06-17T12:44:07.254 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../047-ceph_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:07.255 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-mako. 2022-06-17T12:44:07.255 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2022-06-17T12:44:07.256 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:07.320 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-singledispatch. 2022-06-17T12:44:07.334 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2022-06-17T12:44:07.376 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2022-06-17T12:44:07.398 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-webob. 2022-06-17T12:44:07.412 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2022-06-17T12:44:07.459 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2022-06-17T12:44:07.472 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-bs4. 2022-06-17T12:44:07.491 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2022-06-17T12:44:07.516 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-06-17T12:44:07.547 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2022-06-17T12:44:07.595 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-fuse. 2022-06-17T12:44:07.623 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../048-ceph-fuse_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:07.646 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-fuse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:07.674 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-webob. 2022-06-17T12:44:07.714 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2022-06-17T12:44:07.734 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2022-06-17T12:44:07.809 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-soupsieve. 2022-06-17T12:44:07.824 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2022-06-17T12:44:07.865 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2022-06-17T12:44:07.877 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-simplegeneric. 2022-06-17T12:44:07.886 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2022-06-17T12:44:07.933 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2022-06-17T12:44:07.976 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-waitress. 2022-06-17T12:44:07.995 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2022-06-17T12:44:08.033 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mds. 2022-06-17T12:44:08.054 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2022-06-17T12:44:08.055 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../049-ceph-mds_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:08.066 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-soupsieve. 2022-06-17T12:44:08.076 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mds (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:08.084 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2022-06-17T12:44:08.122 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2022-06-17T12:44:08.206 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-bs4. 2022-06-17T12:44:08.231 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2022-06-17T12:44:08.240 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-singledispatch. 2022-06-17T12:44:08.258 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2022-06-17T12:44:08.269 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2022-06-17T12:44:08.290 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2022-06-17T12:44:08.381 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-tempita. 2022-06-17T12:44:08.397 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2022-06-17T12:44:08.425 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-bs4. 2022-06-17T12:44:08.450 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2022-06-17T12:44:08.451 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-tempita (0.5.2-6) ... 2022-06-17T12:44:08.486 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2022-06-17T12:44:08.592 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-webob. 2022-06-17T12:44:08.595 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package cephadm. 2022-06-17T12:44:08.606 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2022-06-17T12:44:08.615 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../050-cephadm_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:08.642 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2022-06-17T12:44:08.663 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-waitress. 2022-06-17T12:44:08.664 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking cephadm (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:08.682 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2022-06-17T12:44:08.721 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2022-06-17T12:44:08.822 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-paste. 2022-06-17T12:44:08.822 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2022-06-17T12:44:08.842 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2022-06-17T12:44:08.874 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-waitress. 2022-06-17T12:44:08.886 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2022-06-17T12:44:08.890 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-jinja2. 2022-06-17T12:44:08.916 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2022-06-17T12:44:08.941 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2022-06-17T12:44:08.942 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2022-06-17T12:44:08.979 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-soupsieve. 2022-06-17T12:44:08.996 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2022-06-17T12:44:09.042 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2022-06-17T12:44:09.093 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-tempita. 2022-06-17T12:44:09.104 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2022-06-17T12:44:09.146 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-tempita (0.5.2-6) ... 2022-06-17T12:44:09.196 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2022-06-17T12:44:09.234 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2022-06-17T12:44:09.243 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2022-06-17T12:44:09.243 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:09.256 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2022-06-17T12:44:09.278 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mgr-cephadm (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:09.302 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-tempita. 2022-06-17T12:44:09.312 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2022-06-17T12:44:09.351 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-tempita (0.5.2-6) ... 2022-06-17T12:44:09.388 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-bs4. 2022-06-17T12:44:09.401 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2022-06-17T12:44:09.469 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-paste. 2022-06-17T12:44:09.483 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2022-06-17T12:44:09.505 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2022-06-17T12:44:09.554 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2022-06-17T12:44:09.583 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pastedeploy. 2022-06-17T12:44:09.602 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2022-06-17T12:44:09.615 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-repoze.lru. 2022-06-17T12:44:09.624 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2022-06-17T12:44:09.629 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2022-06-17T12:44:09.668 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-paste. 2022-06-17T12:44:09.676 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2022-06-17T12:44:09.690 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2022-06-17T12:44:09.741 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2022-06-17T12:44:09.818 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-waitress. 2022-06-17T12:44:09.844 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2022-06-17T12:44:09.871 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2022-06-17T12:44:09.882 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2022-06-17T12:44:09.883 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2022-06-17T12:44:09.931 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2022-06-17T12:44:09.932 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libjs-underscore. 2022-06-17T12:44:09.951 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2022-06-17T12:44:09.960 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-webtest. 2022-06-17T12:44:09.978 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2022-06-17T12:44:09.982 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-06-17T12:44:10.022 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-webtest (2.0.32-2) ... 2022-06-17T12:44:10.058 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2022-06-17T12:44:10.077 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2022-06-17T12:44:10.112 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2022-06-17T12:44:10.247 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-tempita. 2022-06-17T12:44:10.271 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2022-06-17T12:44:10.298 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-pastedeploy. 2022-06-17T12:44:10.299 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libjs-sphinxdoc. 2022-06-17T12:44:10.317 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2022-06-17T12:44:10.318 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-tempita (0.5.2-6) ... 2022-06-17T12:44:10.319 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2022-06-17T12:44:10.356 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-06-17T12:44:10.357 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pecan. 2022-06-17T12:44:10.357 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2022-06-17T12:44:10.358 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2022-06-17T12:44:10.415 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2022-06-17T12:44:10.459 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pastedeploy. 2022-06-17T12:44:10.473 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2022-06-17T12:44:10.521 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2022-06-17T12:44:10.599 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-paste. 2022-06-17T12:44:10.619 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2022-06-17T12:44:10.648 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2022-06-17T12:44:10.678 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-routes. 2022-06-17T12:44:10.686 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2022-06-17T12:44:10.730 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-routes (2.4.1-2) ... 2022-06-17T12:44:10.730 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-webtest. 2022-06-17T12:44:10.736 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2022-06-17T12:44:10.790 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-webtest (2.0.32-2) ... 2022-06-17T12:44:10.839 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-werkzeug. 2022-06-17T12:44:10.845 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-webtest. 2022-06-17T12:44:10.875 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2022-06-17T12:44:10.875 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2022-06-17T12:44:10.897 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2022-06-17T12:44:10.905 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-webtest (2.0.32-2) ... 2022-06-17T12:44:11.024 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2022-06-17T12:44:11.068 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2022-06-17T12:44:11.092 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2022-06-17T12:44:11.119 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2022-06-17T12:44:11.132 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:11.143 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-pecan. 2022-06-17T12:44:11.155 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2022-06-17T12:44:11.183 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mgr-dashboard (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:11.202 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2022-06-17T12:44:11.202 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pecan. 2022-06-17T12:44:11.222 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2022-06-17T12:44:11.252 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2022-06-17T12:44:11.285 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr. 2022-06-17T12:44:11.303 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../044-ceph-mgr_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:11.343 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:11.432 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-pastedeploy. 2022-06-17T12:44:11.448 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2022-06-17T12:44:11.482 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2022-06-17T12:44:11.567 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-werkzeug. 2022-06-17T12:44:11.584 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-werkzeug. 2022-06-17T12:44:11.599 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2022-06-17T12:44:11.614 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2022-06-17T12:44:11.635 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2022-06-17T12:44:11.665 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2022-06-17T12:44:11.807 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-webtest. 2022-06-17T12:44:11.826 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2022-06-17T12:44:11.874 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-webtest (2.0.32-2) ... 2022-06-17T12:44:11.880 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mon. 2022-06-17T12:44:11.896 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../045-ceph-mon_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:11.941 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mon (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:12.075 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr. 2022-06-17T12:44:12.103 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-mgr. 2022-06-17T12:44:12.104 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../044-ceph-mgr_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:12.109 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../044-ceph-mgr_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:12.135 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:12.135 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2022-06-17T12:44:12.159 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:12.160 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-mgr (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:12.235 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-pecan. 2022-06-17T12:44:12.252 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2022-06-17T12:44:12.335 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2022-06-17T12:44:12.533 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:12.618 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-mon. 2022-06-17T12:44:12.638 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../045-ceph-mon_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:12.700 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-mon (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:12.749 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-werkzeug. 2022-06-17T12:44:12.757 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2022-06-17T12:44:12.798 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2022-06-17T12:44:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-decorator. 2022-06-17T12:44:12.938 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2022-06-17T12:44:12.968 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mon. 2022-06-17T12:44:12.971 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../045-ceph-mon_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:12.978 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2022-06-17T12:44:12.992 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-osd. 2022-06-17T12:44:13.012 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../046-ceph-osd_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:13.013 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mon (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:13.055 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-osd (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:13.185 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package ceph-mgr. 2022-06-17T12:44:13.204 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../044-ceph-mgr_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:13.289 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking ceph-mgr (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:13.324 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2022-06-17T12:44:13.347 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2022-06-17T12:44:13.396 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-06-17T12:44:13.684 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-scipy. 2022-06-17T12:44:13.711 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2022-06-17T12:44:13.738 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-osd. 2022-06-17T12:44:13.760 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../046-ceph-osd_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:13.776 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2022-06-17T12:44:13.779 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package ceph-mon. 2022-06-17T12:44:13.809 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-osd (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:13.810 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../045-ceph-mon_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:13.844 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking ceph-mon (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:14.047 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-osd. 2022-06-17T12:44:14.062 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../046-ceph-osd_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:14.103 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-osd (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:14.854 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package ceph-osd. 2022-06-17T12:44:14.855 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../046-ceph-osd_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:14.912 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking ceph-osd (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:15.380 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-sklearn-lib. 2022-06-17T12:44:15.397 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2022-06-17T12:44:15.455 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-06-17T12:44:15.456 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph. 2022-06-17T12:44:15.479 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../047-ceph_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:15.515 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:15.875 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-fuse. 2022-06-17T12:44:15.892 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../048-ceph-fuse_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:15.933 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-fuse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:15.986 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-joblib. 2022-06-17T12:44:15.998 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2022-06-17T12:44:16.061 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-joblib (0.14.0-3) ... 2022-06-17T12:44:16.154 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph. 2022-06-17T12:44:16.168 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../047-ceph_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:16.206 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:16.414 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mds. 2022-06-17T12:44:16.419 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../049-ceph-mds_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:16.434 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-sklearn. 2022-06-17T12:44:16.446 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2022-06-17T12:44:16.463 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mds (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:16.464 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph. 2022-06-17T12:44:16.484 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../047-ceph_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:16.486 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-06-17T12:44:16.526 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:16.562 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-fuse. 2022-06-17T12:44:16.580 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../048-ceph-fuse_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:16.617 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-fuse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:16.851 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-fuse. 2022-06-17T12:44:16.866 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../048-ceph-fuse_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:16.919 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-fuse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:17.002 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package cephadm. 2022-06-17T12:44:17.017 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../050-cephadm_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:17.062 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking cephadm (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:17.133 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-mds. 2022-06-17T12:44:17.147 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../049-ceph-mds_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:17.196 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2022-06-17T12:44:17.208 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:17.209 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-mds (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:17.209 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package ceph. 2022-06-17T12:44:17.233 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../047-ceph_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:17.254 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking ceph (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:17.272 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:17.304 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mds. 2022-06-17T12:44:17.319 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../049-ceph-mds_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:17.359 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mds (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:17.410 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-jinja2. 2022-06-17T12:44:17.434 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2022-06-17T12:44:17.467 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2022-06-17T12:44:17.599 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package ceph-fuse. 2022-06-17T12:44:17.622 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../048-ceph-fuse_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:17.659 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking ceph-fuse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:17.723 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package cephadm. 2022-06-17T12:44:17.741 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../050-cephadm_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:17.787 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package cephadm. 2022-06-17T12:44:17.788 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking cephadm (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:17.803 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../050-cephadm_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:17.804 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2022-06-17T12:44:17.822 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:17.843 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking cephadm (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:17.853 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-cephadm (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:18.109 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package ceph-mds. 2022-06-17T12:44:18.127 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../049-ceph-mds_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:18.160 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-jinja2. 2022-06-17T12:44:18.161 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2022-06-17T12:44:18.170 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking ceph-mds (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:18.171 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-repoze.lru. 2022-06-17T12:44:18.197 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2022-06-17T12:44:18.199 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2022-06-17T12:44:18.200 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-jinja2. 2022-06-17T12:44:18.202 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2022-06-17T12:44:18.218 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2022-06-17T12:44:18.251 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2022-06-17T12:44:18.365 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-cachetools. 2022-06-17T12:44:18.389 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2022-06-17T12:44:18.456 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2022-06-17T12:44:18.498 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libjs-underscore. 2022-06-17T12:44:18.509 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2022-06-17T12:44:18.553 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-06-17T12:44:18.574 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2022-06-17T12:44:18.575 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2022-06-17T12:44:18.575 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:18.585 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:18.617 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr-cephadm (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:18.620 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-mgr-cephadm (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:18.674 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package cephadm. 2022-06-17T12:44:18.675 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../050-cephadm_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:18.741 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking cephadm (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:18.769 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pyasn1. 2022-06-17T12:44:18.781 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2022-06-17T12:44:18.809 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2022-06-17T12:44:18.846 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libjs-sphinxdoc. 2022-06-17T12:44:18.875 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2022-06-17T12:44:18.903 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-06-17T12:44:18.947 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-repoze.lru. 2022-06-17T12:44:18.962 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2022-06-17T12:44:18.974 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-repoze.lru. 2022-06-17T12:44:18.994 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2022-06-17T12:44:19.005 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2022-06-17T12:44:19.039 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2022-06-17T12:44:19.041 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-jinja2. 2022-06-17T12:44:19.059 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2022-06-17T12:44:19.099 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2022-06-17T12:44:19.178 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pyasn1-modules. 2022-06-17T12:44:19.180 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2022-06-17T12:44:19.222 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-06-17T12:44:19.263 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libjs-underscore. 2022-06-17T12:44:19.284 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2022-06-17T12:44:19.289 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-routes. 2022-06-17T12:44:19.290 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2022-06-17T12:44:19.295 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libjs-underscore. 2022-06-17T12:44:19.311 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2022-06-17T12:44:19.350 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-routes (2.4.1-2) ... 2022-06-17T12:44:19.351 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-06-17T12:44:19.365 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-06-17T12:44:19.477 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2022-06-17T12:44:19.492 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:19.539 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking ceph-mgr-cephadm (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:19.562 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-rsa. 2022-06-17T12:44:19.579 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2022-06-17T12:44:19.598 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libjs-sphinxdoc. 2022-06-17T12:44:19.610 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2022-06-17T12:44:19.627 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-rsa (4.0-3) ... 2022-06-17T12:44:19.651 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-06-17T12:44:19.674 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libjs-sphinxdoc. 2022-06-17T12:44:19.687 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2022-06-17T12:44:19.715 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2022-06-17T12:44:19.740 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-06-17T12:44:19.741 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:19.773 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-dashboard (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:19.899 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-repoze.lru. 2022-06-17T12:44:19.914 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2022-06-17T12:44:19.915 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-routes. 2022-06-17T12:44:19.927 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2022-06-17T12:44:19.944 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-google-auth. 2022-06-17T12:44:19.948 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2022-06-17T12:44:19.962 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-routes (2.4.1-2) ... 2022-06-17T12:44:19.967 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2022-06-17T12:44:20.010 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2022-06-17T12:44:20.100 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-routes. 2022-06-17T12:44:20.101 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2022-06-17T12:44:20.167 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-routes (2.4.1-2) ... 2022-06-17T12:44:20.243 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package libjs-underscore. 2022-06-17T12:44:20.258 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2022-06-17T12:44:20.275 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2022-06-17T12:44:20.307 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:20.324 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr-dashboard (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:20.342 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-06-17T12:44:20.347 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-blinker. 2022-06-17T12:44:20.366 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2022-06-17T12:44:20.405 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-06-17T12:44:20.492 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2022-06-17T12:44:20.527 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:20.556 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-mgr-dashboard (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:20.593 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package libjs-sphinxdoc. 2022-06-17T12:44:20.622 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2022-06-17T12:44:20.671 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-06-17T12:44:20.734 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-oauthlib. 2022-06-17T12:44:20.750 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2022-06-17T12:44:20.790 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2022-06-17T12:44:20.791 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2022-06-17T12:44:20.808 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:20.866 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:20.998 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-routes. 2022-06-17T12:44:21.020 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2022-06-17T12:44:21.054 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-routes (2.4.1-2) ... 2022-06-17T12:44:21.134 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-requests-oauthlib. 2022-06-17T12:44:21.145 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2022-06-17T12:44:21.172 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-decorator. 2022-06-17T12:44:21.191 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2022-06-17T12:44:21.192 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-06-17T12:44:21.222 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2022-06-17T12:44:21.233 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2022-06-17T12:44:21.241 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:21.294 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:21.434 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2022-06-17T12:44:21.448 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:21.492 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking ceph-mgr-dashboard (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:21.507 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-websocket. 2022-06-17T12:44:21.529 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2022-06-17T12:44:21.555 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-decorator. 2022-06-17T12:44:21.566 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2022-06-17T12:44:21.574 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2022-06-17T12:44:21.576 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2022-06-17T12:44:21.593 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:21.605 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2022-06-17T12:44:21.634 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2022-06-17T12:44:21.650 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2022-06-17T12:44:21.691 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-06-17T12:44:21.895 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2022-06-17T12:44:21.910 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-kubernetes. 2022-06-17T12:44:21.913 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2022-06-17T12:44:21.938 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2022-06-17T12:44:21.962 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-06-17T12:44:21.973 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2022-06-17T12:44:21.983 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:22.020 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-scipy. 2022-06-17T12:44:22.039 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2022-06-17T12:44:22.094 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2022-06-17T12:44:22.261 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-scipy. 2022-06-17T12:44:22.280 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2022-06-17T12:44:22.338 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2022-06-17T12:44:22.361 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-decorator. 2022-06-17T12:44:22.391 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2022-06-17T12:44:22.419 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2022-06-17T12:44:22.487 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2022-06-17T12:44:22.494 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2022-06-17T12:44:22.497 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:22.518 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:22.562 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mgr-k8sevents (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:22.582 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:22.760 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2022-06-17T12:44:22.783 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2022-06-17T12:44:22.825 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-06-17T12:44:22.886 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libonig5:amd64. 2022-06-17T12:44:22.901 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2022-06-17T12:44:22.950 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2022-06-17T12:44:22.959 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-decorator. 2022-06-17T12:44:22.968 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2022-06-17T12:44:23.013 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2022-06-17T12:44:23.120 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-scipy. 2022-06-17T12:44:23.138 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2022-06-17T12:44:23.207 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2022-06-17T12:44:23.322 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libjq1:amd64. 2022-06-17T12:44:23.337 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-06-17T12:44:23.372 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2022-06-17T12:44:23.379 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-06-17T12:44:23.388 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2022-06-17T12:44:23.429 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-06-17T12:44:23.625 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-sklearn-lib. 2022-06-17T12:44:23.643 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2022-06-17T12:44:23.669 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package jq. 2022-06-17T12:44:23.682 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-06-17T12:44:23.683 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-06-17T12:44:23.728 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2022-06-17T12:44:23.824 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-scipy. 2022-06-17T12:44:23.834 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-sklearn-lib. 2022-06-17T12:44:23.843 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2022-06-17T12:44:23.865 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2022-06-17T12:44:23.894 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2022-06-17T12:44:23.896 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-06-17T12:44:23.960 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package socat. 2022-06-17T12:44:23.977 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2022-06-17T12:44:24.011 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking socat (1.7.3.3-2) ... 2022-06-17T12:44:24.253 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-joblib. 2022-06-17T12:44:24.271 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2022-06-17T12:44:24.340 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-joblib (0.14.0-3) ... 2022-06-17T12:44:24.481 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-joblib. 2022-06-17T12:44:24.499 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2022-06-17T12:44:24.500 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libxslt1.1:amd64. 2022-06-17T12:44:24.516 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2022-06-17T12:44:24.539 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-joblib (0.14.0-3) ... 2022-06-17T12:44:24.560 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2022-06-17T12:44:24.653 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-sklearn-lib. 2022-06-17T12:44:24.674 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2022-06-17T12:44:24.709 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-06-17T12:44:24.744 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-sklearn. 2022-06-17T12:44:24.761 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2022-06-17T12:44:24.801 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-06-17T12:44:24.847 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package xmlstarlet. 2022-06-17T12:44:24.864 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2022-06-17T12:44:24.906 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2022-06-17T12:44:24.910 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-sklearn. 2022-06-17T12:44:24.930 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2022-06-17T12:44:24.968 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-06-17T12:44:25.234 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-test. 2022-06-17T12:44:25.260 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../083-ceph-test_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:25.290 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-test (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:25.313 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-joblib. 2022-06-17T12:44:25.326 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-sklearn-lib. 2022-06-17T12:44:25.337 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2022-06-17T12:44:25.348 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2022-06-17T12:44:25.380 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-joblib (0.14.0-3) ... 2022-06-17T12:44:25.383 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-06-17T12:44:25.613 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2022-06-17T12:44:25.628 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:25.652 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2022-06-17T12:44:25.671 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:25.672 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:25.743 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:25.757 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-sklearn. 2022-06-17T12:44:25.776 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2022-06-17T12:44:25.815 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-06-17T12:44:25.994 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-joblib. 2022-06-17T12:44:26.014 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2022-06-17T12:44:26.090 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-joblib (0.14.0-3) ... 2022-06-17T12:44:26.502 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-sklearn. 2022-06-17T12:44:26.522 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2022-06-17T12:44:26.549 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2022-06-17T12:44:26.567 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:26.568 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-06-17T12:44:26.607 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:26.746 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-cachetools. 2022-06-17T12:44:26.763 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2022-06-17T12:44:26.805 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2022-06-17T12:44:26.930 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cachetools. 2022-06-17T12:44:26.963 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2022-06-17T12:44:26.988 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2022-06-17T12:44:27.122 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pyasn1. 2022-06-17T12:44:27.148 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2022-06-17T12:44:27.182 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2022-06-17T12:44:27.298 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2022-06-17T12:44:27.317 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:27.335 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pyasn1. 2022-06-17T12:44:27.351 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2022-06-17T12:44:27.380 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:27.401 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2022-06-17T12:44:27.534 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pyasn1-modules. 2022-06-17T12:44:27.563 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2022-06-17T12:44:27.606 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-06-17T12:44:27.771 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-cachetools. 2022-06-17T12:44:27.783 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2022-06-17T12:44:27.788 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pyasn1-modules. 2022-06-17T12:44:27.805 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2022-06-17T12:44:27.821 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2022-06-17T12:44:27.878 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-06-17T12:44:27.929 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-rsa. 2022-06-17T12:44:27.945 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2022-06-17T12:44:27.999 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-rsa (4.0-3) ... 2022-06-17T12:44:28.184 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-pyasn1. 2022-06-17T12:44:28.188 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2022-06-17T12:44:28.229 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2022-06-17T12:44:28.256 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-rsa. 2022-06-17T12:44:28.273 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2022-06-17T12:44:28.323 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-rsa (4.0-3) ... 2022-06-17T12:44:28.332 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-google-auth. 2022-06-17T12:44:28.361 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2022-06-17T12:44:28.405 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2022-06-17T12:44:28.583 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-pyasn1-modules. 2022-06-17T12:44:28.603 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-cachetools. 2022-06-17T12:44:28.604 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2022-06-17T12:44:28.612 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2022-06-17T12:44:28.644 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-google-auth. 2022-06-17T12:44:28.645 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-06-17T12:44:28.659 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2022-06-17T12:44:28.683 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2022-06-17T12:44:28.700 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2022-06-17T12:44:28.712 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-blinker. 2022-06-17T12:44:28.730 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-colorama. 2022-06-17T12:44:28.731 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2022-06-17T12:44:28.763 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2022-06-17T12:44:28.770 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-06-17T12:44:28.790 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2022-06-17T12:44:28.977 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-rsa. 2022-06-17T12:44:28.978 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2022-06-17T12:44:28.998 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-pyasn1. 2022-06-17T12:44:29.011 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2022-06-17T12:44:29.046 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-blinker. 2022-06-17T12:44:29.047 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-rsa (4.0-3) ... 2022-06-17T12:44:29.062 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2022-06-17T12:44:29.071 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-oauthlib. 2022-06-17T12:44:29.071 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2022-06-17T12:44:29.078 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package cephfs-shell. 2022-06-17T12:44:29.091 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../085-cephfs-shell_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:29.093 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2022-06-17T12:44:29.114 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-06-17T12:44:29.136 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking cephfs-shell (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:29.136 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2022-06-17T12:44:29.350 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libcephfs-dev. 2022-06-17T12:44:29.371 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-google-auth. 2022-06-17T12:44:29.372 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:29.373 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2022-06-17T12:44:29.414 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libcephfs-dev (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:29.415 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2022-06-17T12:44:29.442 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-oauthlib. 2022-06-17T12:44:29.469 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2022-06-17T12:44:29.470 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-pyasn1-modules. 2022-06-17T12:44:29.473 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-requests-oauthlib. 2022-06-17T12:44:29.474 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2022-06-17T12:44:29.475 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2022-06-17T12:44:29.506 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2022-06-17T12:44:29.514 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-06-17T12:44:29.518 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-06-17T12:44:29.771 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-blinker. 2022-06-17T12:44:29.795 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2022-06-17T12:44:29.813 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package nvme-cli. 2022-06-17T12:44:29.831 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2022-06-17T12:44:29.832 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-06-17T12:44:29.856 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-websocket. 2022-06-17T12:44:29.857 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2022-06-17T12:44:29.897 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-requests-oauthlib. 2022-06-17T12:44:29.919 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2022-06-17T12:44:29.919 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2022-06-17T12:44:29.920 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-rsa. 2022-06-17T12:44:29.921 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2022-06-17T12:44:29.935 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2022-06-17T12:44:29.976 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-rsa (4.0-3) ... 2022-06-17T12:44:29.977 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-06-17T12:44:30.176 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-oauthlib. 2022-06-17T12:44:30.187 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2022-06-17T12:44:30.235 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-kubernetes. 2022-06-17T12:44:30.236 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2022-06-17T12:44:30.237 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2022-06-17T12:44:30.280 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2022-06-17T12:44:30.288 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-websocket. 2022-06-17T12:44:30.308 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2022-06-17T12:44:30.334 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-atomicwrites. 2022-06-17T12:44:30.335 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-google-auth. 2022-06-17T12:44:30.343 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2022-06-17T12:44:30.344 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2022-06-17T12:44:30.381 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2022-06-17T12:44:30.382 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2022-06-17T12:44:30.385 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2022-06-17T12:44:30.617 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-requests-oauthlib. 2022-06-17T12:44:30.640 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2022-06-17T12:44:30.659 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-attr. 2022-06-17T12:44:30.672 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-06-17T12:44:30.684 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2022-06-17T12:44:30.720 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-kubernetes. 2022-06-17T12:44:30.735 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2022-06-17T12:44:30.735 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-blinker. 2022-06-17T12:44:30.736 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-attr (19.3.0-2) ... 2022-06-17T12:44:30.742 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2022-06-17T12:44:30.778 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2022-06-17T12:44:30.782 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2022-06-17T12:44:30.783 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-06-17T12:44:30.801 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:30.857 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr-k8sevents (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:30.975 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-webencodings. 2022-06-17T12:44:31.000 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2022-06-17T12:44:31.038 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2022-06-17T12:44:31.043 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-websocket. 2022-06-17T12:44:31.059 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2022-06-17T12:44:31.079 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2022-06-17T12:44:31.116 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-oauthlib. 2022-06-17T12:44:31.156 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2022-06-17T12:44:31.192 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2022-06-17T12:44:31.207 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libonig5:amd64. 2022-06-17T12:44:31.207 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2022-06-17T12:44:31.234 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2022-06-17T12:44:31.339 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-html5lib. 2022-06-17T12:44:31.365 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2022-06-17T12:44:31.380 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2022-06-17T12:44:31.399 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2022-06-17T12:44:31.399 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:31.418 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-kubernetes. 2022-06-17T12:44:31.439 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2022-06-17T12:44:31.452 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-k8sevents (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:31.481 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2022-06-17T12:44:31.565 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-requests-oauthlib. 2022-06-17T12:44:31.583 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2022-06-17T12:44:31.614 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-06-17T12:44:31.690 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libjq1:amd64. 2022-06-17T12:44:31.714 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-06-17T12:44:31.770 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-06-17T12:44:31.772 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-lxml:amd64. 2022-06-17T12:44:31.780 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2022-06-17T12:44:31.798 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libonig5:amd64. 2022-06-17T12:44:31.818 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2022-06-17T12:44:31.831 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-06-17T12:44:31.856 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2022-06-17T12:44:31.961 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-websocket. 2022-06-17T12:44:31.972 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2022-06-17T12:44:32.012 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2022-06-17T12:44:32.029 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package jq. 2022-06-17T12:44:32.030 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2022-06-17T12:44:32.043 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-06-17T12:44:32.053 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:32.084 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2022-06-17T12:44:32.104 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-mgr-k8sevents (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:32.261 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libjq1:amd64. 2022-06-17T12:44:32.272 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-packaging. 2022-06-17T12:44:32.280 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-06-17T12:44:32.286 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2022-06-17T12:44:32.323 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-06-17T12:44:32.366 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-kubernetes. 2022-06-17T12:44:32.366 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-packaging (20.3-1) ... 2022-06-17T12:44:32.367 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package socat. 2022-06-17T12:44:32.378 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2022-06-17T12:44:32.385 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2022-06-17T12:44:32.421 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2022-06-17T12:44:32.425 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking socat (1.7.3.3-2) ... 2022-06-17T12:44:32.460 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libonig5:amd64. 2022-06-17T12:44:32.469 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2022-06-17T12:44:32.542 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2022-06-17T12:44:32.631 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package jq. 2022-06-17T12:44:32.652 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-06-17T12:44:32.661 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pastescript. 2022-06-17T12:44:32.681 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2022-06-17T12:44:32.687 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2022-06-17T12:44:32.721 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2022-06-17T12:44:32.905 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libjq1:amd64. 2022-06-17T12:44:32.919 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libxslt1.1:amd64. 2022-06-17T12:44:32.931 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-06-17T12:44:32.933 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2022-06-17T12:44:32.966 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-06-17T12:44:32.974 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2022-06-17T12:44:32.974 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2022-06-17T12:44:32.998 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:33.040 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pluggy. 2022-06-17T12:44:33.041 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking ceph-mgr-k8sevents (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:33.060 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2022-06-17T12:44:33.102 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2022-06-17T12:44:33.251 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package xmlstarlet. 2022-06-17T12:44:33.257 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package jq. 2022-06-17T12:44:33.269 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2022-06-17T12:44:33.280 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-06-17T12:44:33.314 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2022-06-17T12:44:33.322 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2022-06-17T12:44:33.358 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package socat. 2022-06-17T12:44:33.375 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2022-06-17T12:44:33.402 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package libonig5:amd64. 2022-06-17T12:44:33.420 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-psutil. 2022-06-17T12:44:33.421 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2022-06-17T12:44:33.422 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking socat (1.7.3.3-2) ... 2022-06-17T12:44:33.432 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2022-06-17T12:44:33.502 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2022-06-17T12:44:33.503 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2022-06-17T12:44:33.631 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package socat. 2022-06-17T12:44:33.632 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2022-06-17T12:44:33.676 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking socat (1.7.3.3-2) ... 2022-06-17T12:44:33.698 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-test. 2022-06-17T12:44:33.757 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../083-ceph-test_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:33.766 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-test (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:33.831 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-py. 2022-06-17T12:44:33.841 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2022-06-17T12:44:33.887 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package libjq1:amd64. 2022-06-17T12:44:33.888 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-06-17T12:44:33.890 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2022-06-17T12:44:33.934 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-06-17T12:44:33.946 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libxslt1.1:amd64. 2022-06-17T12:44:33.970 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2022-06-17T12:44:34.009 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2022-06-17T12:44:34.174 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libxslt1.1:amd64. 2022-06-17T12:44:34.209 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2022-06-17T12:44:34.232 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package jq. 2022-06-17T12:44:34.233 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2022-06-17T12:44:34.259 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pyinotify. 2022-06-17T12:44:34.260 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2022-06-17T12:44:34.261 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-06-17T12:44:34.290 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2022-06-17T12:44:34.312 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-06-17T12:44:34.313 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package xmlstarlet. 2022-06-17T12:44:34.326 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2022-06-17T12:44:34.370 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2022-06-17T12:44:34.548 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package xmlstarlet. 2022-06-17T12:44:34.558 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2022-06-17T12:44:34.600 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2022-06-17T12:44:34.601 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package socat. 2022-06-17T12:44:34.602 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2022-06-17T12:44:34.620 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pytest. 2022-06-17T12:44:34.637 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2022-06-17T12:44:34.644 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking socat (1.7.3.3-2) ... 2022-06-17T12:44:34.689 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pytest (4.6.9-1) ... 2022-06-17T12:44:34.745 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-test. 2022-06-17T12:44:34.755 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../083-ceph-test_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:34.795 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-test (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:34.940 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package ceph-test. 2022-06-17T12:44:34.959 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../083-ceph-test_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:35.007 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking ceph-test (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:35.023 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-simplejson. 2022-06-17T12:44:35.038 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2022-06-17T12:44:35.078 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2022-06-17T12:44:35.182 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package libxslt1.1:amd64. 2022-06-17T12:44:35.202 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2022-06-17T12:44:35.241 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2022-06-17T12:44:35.544 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package xmlstarlet. 2022-06-17T12:44:35.546 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package radosgw. 2022-06-17T12:44:35.561 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2022-06-17T12:44:35.562 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../101-radosgw_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:35.603 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking radosgw (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:35.604 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2022-06-17T12:44:35.965 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package ceph-test. 2022-06-17T12:44:35.980 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../083-ceph-test_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:36.068 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking ceph-test (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:36.689 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package rbd-fuse. 2022-06-17T12:44:36.709 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../102-rbd-fuse_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:36.755 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking rbd-fuse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:37.070 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-attr (19.3.0-2) ... 2022-06-17T12:44:37.352 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-colorama. 2022-06-17T12:44:37.360 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2022-06-17T12:44:37.412 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-06-17T12:44:37.413 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2022-06-17T12:44:37.545 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2022-06-17T12:44:37.671 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package cephfs-shell. 2022-06-17T12:44:37.686 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../085-cephfs-shell_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:37.693 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2022-06-17T12:44:37.729 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking cephfs-shell (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:37.905 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up cephadm (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:37.978 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libcephfs-dev. 2022-06-17T12:44:37.996 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:38.032 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libcephfs-dev (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:38.080 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-colorama. 2022-06-17T12:44:38.096 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2022-06-17T12:44:38.124 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2022-06-17T12:44:38.415 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package cephfs-shell. 2022-06-17T12:44:38.430 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../085-cephfs-shell_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:38.470 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package nvme-cli. 2022-06-17T12:44:38.483 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking cephfs-shell (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:38.484 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2022-06-17T12:44:38.501 INFO:teuthology.orchestra.run.smithi107.stdout:Adding system user cephadm....done 2022-06-17T12:44:38.535 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-colorama. 2022-06-17T12:44:38.546 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2022-06-17T12:44:38.572 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2022-06-17T12:44:38.593 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2022-06-17T12:44:38.666 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2022-06-17T12:44:38.749 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libcephfs-dev. 2022-06-17T12:44:38.769 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:38.812 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libcephfs-dev (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:38.833 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package cephfs-shell. 2022-06-17T12:44:38.855 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../085-cephfs-shell_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:38.927 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking cephfs-shell (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:38.948 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-atomicwrites. 2022-06-17T12:44:38.956 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2022-06-17T12:44:39.001 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2022-06-17T12:44:39.059 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-repoze.lru (0.7-2) ... 2022-06-17T12:44:39.155 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package nvme-cli. 2022-06-17T12:44:39.156 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libcephfs-dev. 2022-06-17T12:44:39.156 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:39.184 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2022-06-17T12:44:39.201 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libcephfs-dev (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:39.228 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2022-06-17T12:44:39.304 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-attr. 2022-06-17T12:44:39.316 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2022-06-17T12:44:39.369 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-attr (19.3.0-2) ... 2022-06-17T12:44:39.371 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up liboath0 (2.6.1-1.3) ... 2022-06-17T12:44:39.492 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2022-06-17T12:44:39.533 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-colorama. 2022-06-17T12:44:39.544 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2022-06-17T12:44:39.596 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package nvme-cli. 2022-06-17T12:44:39.607 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-atomicwrites. 2022-06-17T12:44:39.614 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2022-06-17T12:44:39.623 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2022-06-17T12:44:39.626 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2022-06-17T12:44:39.651 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-webencodings. 2022-06-17T12:44:39.663 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2022-06-17T12:44:39.668 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2022-06-17T12:44:39.670 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2022-06-17T12:44:39.712 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2022-06-17T12:44:39.846 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-joblib (0.14.0-3) ... 2022-06-17T12:44:39.947 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package cephfs-shell. 2022-06-17T12:44:39.962 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../085-cephfs-shell_15.2.16-260-gb3641a55-1focal_all.deb ... 2022-06-17T12:44:40.017 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-attr. 2022-06-17T12:44:40.017 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2022-06-17T12:44:40.034 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking cephfs-shell (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:40.035 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-html5lib. 2022-06-17T12:44:40.037 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2022-06-17T12:44:40.043 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-atomicwrites. 2022-06-17T12:44:40.054 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2022-06-17T12:44:40.061 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2022-06-17T12:44:40.089 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-attr (19.3.0-2) ... 2022-06-17T12:44:40.095 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2022-06-17T12:44:40.270 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-jwt (1.7.1-2ubuntu2) ... 2022-06-17T12:44:40.314 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package libcephfs-dev. 2022-06-17T12:44:40.314 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:40.348 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking libcephfs-dev (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:40.376 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-lxml:amd64. 2022-06-17T12:44:40.379 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-attr. 2022-06-17T12:44:40.402 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2022-06-17T12:44:40.426 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-webencodings. 2022-06-17T12:44:40.427 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2022-06-17T12:44:40.452 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2022-06-17T12:44:40.453 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-attr (19.3.0-2) ... 2022-06-17T12:44:40.453 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-06-17T12:44:40.484 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2022-06-17T12:44:40.566 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-cachetools (4.0.0-1) ... 2022-06-17T12:44:40.774 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package nvme-cli. 2022-06-17T12:44:40.775 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-html5lib. 2022-06-17T12:44:40.803 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2022-06-17T12:44:40.803 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-webencodings. 2022-06-17T12:44:40.804 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2022-06-17T12:44:40.808 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2022-06-17T12:44:40.832 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2022-06-17T12:44:40.836 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-packaging. 2022-06-17T12:44:40.852 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2022-06-17T12:44:40.852 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2022-06-17T12:44:40.874 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2022-06-17T12:44:40.876 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2022-06-17T12:44:40.895 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-packaging (20.3-1) ... 2022-06-17T12:44:41.158 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-html5lib. 2022-06-17T12:44:41.185 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2022-06-17T12:44:41.187 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2022-06-17T12:44:41.195 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pastescript. 2022-06-17T12:44:41.206 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-lxml:amd64. 2022-06-17T12:44:41.206 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2022-06-17T12:44:41.215 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2022-06-17T12:44:41.231 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2022-06-17T12:44:41.257 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2022-06-17T12:44:41.258 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-atomicwrites. 2022-06-17T12:44:41.258 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2022-06-17T12:44:41.259 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-06-17T12:44:41.300 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2022-06-17T12:44:41.459 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-06-17T12:44:41.514 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-lxml:amd64. 2022-06-17T12:44:41.539 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2022-06-17T12:44:41.569 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pluggy. 2022-06-17T12:44:41.590 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-06-17T12:44:41.591 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2022-06-17T12:44:41.634 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2022-06-17T12:44:41.645 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-attr. 2022-06-17T12:44:41.662 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2022-06-17T12:44:41.706 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-attr (19.3.0-2) ... 2022-06-17T12:44:41.773 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-ceph-argparse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:41.774 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-packaging. 2022-06-17T12:44:41.797 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2022-06-17T12:44:41.871 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-psutil. 2022-06-17T12:44:41.886 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-packaging (20.3-1) ... 2022-06-17T12:44:41.887 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2022-06-17T12:44:41.930 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2022-06-17T12:44:42.069 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-webencodings. 2022-06-17T12:44:42.069 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-06-17T12:44:42.081 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2022-06-17T12:44:42.125 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-packaging. 2022-06-17T12:44:42.163 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2022-06-17T12:44:42.163 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2022-06-17T12:44:42.183 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-packaging (20.3-1) ... 2022-06-17T12:44:42.197 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pastescript. 2022-06-17T12:44:42.210 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2022-06-17T12:44:42.212 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-py. 2022-06-17T12:44:42.216 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2022-06-17T12:44:42.236 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2022-06-17T12:44:42.272 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2022-06-17T12:44:42.272 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2022-06-17T12:44:42.419 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-html5lib. 2022-06-17T12:44:42.447 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2022-06-17T12:44:42.483 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2022-06-17T12:44:42.497 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2022-06-17T12:44:42.536 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-pastescript. 2022-06-17T12:44:42.552 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2022-06-17T12:44:42.569 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pyinotify. 2022-06-17T12:44:42.585 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2022-06-17T12:44:42.598 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2022-06-17T12:44:42.625 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pluggy. 2022-06-17T12:44:42.626 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-06-17T12:44:42.643 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2022-06-17T12:44:42.682 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2022-06-17T12:44:42.813 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2022-06-17T12:44:42.818 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-lxml:amd64. 2022-06-17T12:44:42.841 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2022-06-17T12:44:42.859 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-06-17T12:44:42.894 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pytest. 2022-06-17T12:44:42.912 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2022-06-17T12:44:42.951 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pytest (4.6.9-1) ... 2022-06-17T12:44:42.956 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-pluggy. 2022-06-17T12:44:42.987 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2022-06-17T12:44:43.031 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2022-06-17T12:44:43.040 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-psutil. 2022-06-17T12:44:43.062 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2022-06-17T12:44:43.106 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2022-06-17T12:44:43.116 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2022-06-17T12:44:43.326 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-simplejson. 2022-06-17T12:44:43.343 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2022-06-17T12:44:43.344 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-packaging. 2022-06-17T12:44:43.348 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2022-06-17T12:44:43.380 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2022-06-17T12:44:43.417 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-packaging (20.3-1) ... 2022-06-17T12:44:43.417 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-psutil. 2022-06-17T12:44:43.424 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2022-06-17T12:44:43.468 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2022-06-17T12:44:43.469 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-py. 2022-06-17T12:44:43.489 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2022-06-17T12:44:43.534 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2022-06-17T12:44:43.743 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-pastescript. 2022-06-17T12:44:43.752 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2022-06-17T12:44:43.790 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2022-06-17T12:44:43.819 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2022-06-17T12:44:43.835 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-py. 2022-06-17T12:44:43.862 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2022-06-17T12:44:43.894 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pyinotify. 2022-06-17T12:44:43.897 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2022-06-17T12:44:43.906 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2022-06-17T12:44:43.990 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-06-17T12:44:44.094 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2022-06-17T12:44:44.162 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-pluggy. 2022-06-17T12:44:44.169 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2022-06-17T12:44:44.222 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2022-06-17T12:44:44.300 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-pyinotify. 2022-06-17T12:44:44.301 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package radosgw. 2022-06-17T12:44:44.307 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../101-radosgw_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:44.311 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2022-06-17T12:44:44.345 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pytest. 2022-06-17T12:44:44.346 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2022-06-17T12:44:44.349 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking radosgw (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:44.350 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-06-17T12:44:44.389 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pytest (4.6.9-1) ... 2022-06-17T12:44:44.453 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2022-06-17T12:44:44.554 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-psutil. 2022-06-17T12:44:44.571 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2022-06-17T12:44:44.615 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2022-06-17T12:44:44.663 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-pytest. 2022-06-17T12:44:44.680 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2022-06-17T12:44:44.747 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-simplejson. 2022-06-17T12:44:44.757 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-pytest (4.6.9-1) ... 2022-06-17T12:44:44.771 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2022-06-17T12:44:44.784 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2022-06-17T12:44:44.809 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2022-06-17T12:44:44.999 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-py. 2022-06-17T12:44:45.026 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2022-06-17T12:44:45.090 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2022-06-17T12:44:45.108 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package python3-simplejson. 2022-06-17T12:44:45.123 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-tempita (0.5.2-6) ... 2022-06-17T12:44:45.128 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2022-06-17T12:44:45.202 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2022-06-17T12:44:45.215 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package radosgw. 2022-06-17T12:44:45.233 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../101-radosgw_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:45.261 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking radosgw (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:45.402 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package rbd-fuse. 2022-06-17T12:44:45.434 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-packaging (20.3-1) ... 2022-06-17T12:44:45.435 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../102-rbd-fuse_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:45.455 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-pyinotify. 2022-06-17T12:44:45.462 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking rbd-fuse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:45.480 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2022-06-17T12:44:45.522 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-06-17T12:44:45.687 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package radosgw. 2022-06-17T12:44:45.688 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../101-radosgw_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:45.728 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking radosgw (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:45.761 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2022-06-17T12:44:45.780 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-attr (19.3.0-2) ... 2022-06-17T12:44:45.852 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-pytest. 2022-06-17T12:44:45.854 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2022-06-17T12:44:45.866 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2022-06-17T12:44:45.912 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-pytest (4.6.9-1) ... 2022-06-17T12:44:46.108 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-06-17T12:44:46.181 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2022-06-17T12:44:46.227 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2022-06-17T12:44:46.353 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package python3-simplejson. 2022-06-17T12:44:46.382 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package rbd-fuse. 2022-06-17T12:44:46.382 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2022-06-17T12:44:46.400 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2022-06-17T12:44:46.401 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../102-rbd-fuse_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:46.426 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2022-06-17T12:44:46.470 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking rbd-fuse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:46.583 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up cephadm (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:46.611 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2022-06-17T12:44:46.773 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-attr (19.3.0-2) ... 2022-06-17T12:44:46.897 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package rbd-fuse. 2022-06-17T12:44:46.898 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package radosgw. 2022-06-17T12:44:46.916 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2022-06-17T12:44:46.932 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../102-rbd-fuse_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:46.933 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../101-radosgw_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:46.988 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking radosgw (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:47.003 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking rbd-fuse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:47.084 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-06-17T12:44:47.181 INFO:teuthology.orchestra.run.smithi143.stdout:Adding system user cephadm....done 2022-06-17T12:44:47.220 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2022-06-17T12:44:47.289 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2022-06-17T12:44:47.300 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-attr (19.3.0-2) ... 2022-06-17T12:44:47.350 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2022-06-17T12:44:47.351 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2022-06-17T12:44:47.575 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up cephadm (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:47.656 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up socat (1.7.3.3-2) ... 2022-06-17T12:44:47.685 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-06-17T12:44:47.686 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-repoze.lru (0.7-2) ... 2022-06-17T12:44:47.780 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-ceph-common (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:47.811 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2022-06-17T12:44:47.974 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2022-06-17T12:44:47.975 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up liboath0 (2.6.1-1.3) ... 2022-06-17T12:44:48.079 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-html5lib (1.0.1-2) ... 2022-06-17T12:44:48.097 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2022-06-17T12:44:48.098 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package rbd-fuse. 2022-06-17T12:44:48.114 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../102-rbd-fuse_15.2.16-260-gb3641a55-1focal_amd64.deb ... 2022-06-17T12:44:48.163 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking rbd-fuse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:48.164 INFO:teuthology.orchestra.run.smithi136.stdout:Adding system user cephadm....done 2022-06-17T12:44:48.181 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up cephadm (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:48.323 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2022-06-17T12:44:48.427 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-joblib (0.14.0-3) ... 2022-06-17T12:44:48.431 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-06-17T12:44:48.659 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-attr (19.3.0-2) ... 2022-06-17T12:44:48.672 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-repoze.lru (0.7-2) ... 2022-06-17T12:44:48.736 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pluggy (0.13.0-2) ... 2022-06-17T12:44:48.831 INFO:teuthology.orchestra.run.smithi036.stdout:Adding system user cephadm....done 2022-06-17T12:44:48.832 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-jwt (1.7.1-2ubuntu2) ... 2022-06-17T12:44:48.959 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up liboath0 (2.6.1-1.3) ... 2022-06-17T12:44:48.982 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-06-17T12:44:48.992 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2022-06-17T12:44:49.025 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2022-06-17T12:44:49.092 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2022-06-17T12:44:49.106 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2022-06-17T12:44:49.185 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2022-06-17T12:44:49.185 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2022-06-17T12:44:49.335 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-cachetools (4.0.0-1) ... 2022-06-17T12:44:49.353 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-repoze.lru (0.7-2) ... 2022-06-17T12:44:49.403 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up cephadm (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:49.423 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-joblib (0.14.0-3) ... 2022-06-17T12:44:49.518 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2022-06-17T12:44:49.614 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2022-06-17T12:44:49.674 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up liboath0 (2.6.1-1.3) ... 2022-06-17T12:44:49.799 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-jwt (1.7.1-2ubuntu2) ... 2022-06-17T12:44:49.814 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2022-06-17T12:44:49.815 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-logutils (0.3.3-7) ... 2022-06-17T12:44:49.905 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2022-06-17T12:44:50.046 INFO:teuthology.orchestra.run.smithi070.stdout:Adding system user cephadm....done 2022-06-17T12:44:50.059 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cachetools (4.0.0-1) ... 2022-06-17T12:44:50.116 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2022-06-17T12:44:50.156 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-joblib (0.14.0-3) ... 2022-06-17T12:44:50.199 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2022-06-17T12:44:50.199 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-06-17T12:44:50.249 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2022-06-17T12:44:50.368 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2022-06-17T12:44:50.502 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-ceph-argparse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:50.512 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-jwt (1.7.1-2ubuntu2) ... 2022-06-17T12:44:50.541 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-repoze.lru (0.7-2) ... 2022-06-17T12:44:50.549 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-prettytable (0.7.2-5) ... 2022-06-17T12:44:50.674 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2022-06-17T12:44:50.792 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-06-17T12:44:50.833 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-cachetools (4.0.0-1) ... 2022-06-17T12:44:50.905 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up liboath0 (2.6.1-1.3) ... 2022-06-17T12:44:50.905 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2022-06-17T12:44:50.929 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2022-06-17T12:44:50.973 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-06-17T12:44:50.984 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2022-06-17T12:44:51.124 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2022-06-17T12:44:51.190 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2022-06-17T12:44:51.213 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2022-06-17T12:44:51.305 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-ceph-argparse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:51.377 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-joblib (0.14.0-3) ... 2022-06-17T12:44:51.438 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2022-06-17T12:44:51.500 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2022-06-17T12:44:51.538 INFO:teuthology.orchestra.run.smithi107.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2022-06-17T12:44:51.635 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-06-17T12:44:51.726 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-06-17T12:44:51.776 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2022-06-17T12:44:51.784 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2022-06-17T12:44:51.820 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2022-06-17T12:44:51.833 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-jwt (1.7.1-2ubuntu2) ... 2022-06-17T12:44:51.945 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-06-17T12:44:52.018 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-ceph-argparse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:52.025 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2022-06-17T12:44:52.132 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-cachetools (4.0.0-1) ... 2022-06-17T12:44:52.164 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2022-06-17T12:44:52.305 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-06-17T12:44:52.313 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2022-06-17T12:44:52.335 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2022-06-17T12:44:52.384 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2022-06-17T12:44:52.436 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2022-06-17T12:44:52.462 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2022-06-17T12:44:52.665 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-06-17T12:44:52.666 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2022-06-17T12:44:52.703 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2022-06-17T12:44:52.822 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-06-17T12:44:52.823 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2022-06-17T12:44:52.921 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-webob (1:1.8.5-2) ... 2022-06-17T12:44:53.025 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-06-17T12:44:53.041 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2022-06-17T12:44:53.046 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2022-06-17T12:44:53.061 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2022-06-17T12:44:53.267 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-06-17T12:44:53.340 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2022-06-17T12:44:53.343 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-ceph-argparse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:53.356 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2022-06-17T12:44:53.411 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-tempita (0.5.2-6) ... 2022-06-17T12:44:53.575 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up librados2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:53.644 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2022-06-17T12:44:53.682 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-packaging (20.3-1) ... 2022-06-17T12:44:53.683 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-06-17T12:44:53.683 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2022-06-17T12:44:53.706 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2022-06-17T12:44:53.857 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2022-06-17T12:44:53.933 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2022-06-17T12:44:54.000 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2022-06-17T12:44:54.017 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2022-06-17T12:44:54.121 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2022-06-17T12:44:54.143 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2022-06-17T12:44:54.251 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-tempita (0.5.2-6) ... 2022-06-17T12:44:54.311 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2022-06-17T12:44:54.394 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2022-06-17T12:44:54.419 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2022-06-17T12:44:54.556 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-packaging (20.3-1) ... 2022-06-17T12:44:54.697 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2022-06-17T12:44:54.712 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2022-06-17T12:44:54.804 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2022-06-17T12:44:54.870 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2022-06-17T12:44:55.031 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2022-06-17T12:44:55.047 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2022-06-17T12:44:55.075 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2022-06-17T12:44:55.099 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2022-06-17T12:44:55.176 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up librgw2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:55.326 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-06-17T12:44:55.327 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2022-06-17T12:44:55.372 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2022-06-17T12:44:55.402 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-tempita (0.5.2-6) ... 2022-06-17T12:44:55.461 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2022-06-17T12:44:55.461 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pytest (4.6.9-1) ... 2022-06-17T12:44:55.716 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2022-06-17T12:44:55.738 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2022-06-17T12:44:55.771 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-packaging (20.3-1) ... 2022-06-17T12:44:55.822 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up socat (1.7.3.3-2) ... 2022-06-17T12:44:55.873 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2022-06-17T12:44:55.948 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-ceph-common (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:56.034 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2022-06-17T12:44:56.037 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2022-06-17T12:44:56.053 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2022-06-17T12:44:56.180 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2022-06-17T12:44:56.206 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2022-06-17T12:44:56.225 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-html5lib (1.0.1-2) ... 2022-06-17T12:44:56.348 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-tempita (0.5.2-6) ... 2022-06-17T12:44:56.390 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2022-06-17T12:44:56.482 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-bs4 (4.8.2-1) ... 2022-06-17T12:44:56.493 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2022-06-17T12:44:56.560 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-06-17T12:44:56.615 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-packaging (20.3-1) ... 2022-06-17T12:44:56.703 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pluggy (0.13.0-2) ... 2022-06-17T12:44:56.775 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up socat (1.7.3.3-2) ... 2022-06-17T12:44:56.868 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libcephfs2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:56.894 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-ceph-common (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:56.914 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2022-06-17T12:44:56.949 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2022-06-17T12:44:56.989 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2022-06-17T12:44:56.996 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2022-06-17T12:44:57.087 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2022-06-17T12:44:57.111 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2022-06-17T12:44:57.212 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2022-06-17T12:44:57.223 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-html5lib (1.0.1-2) ... 2022-06-17T12:44:57.315 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libradosstriper1 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:57.362 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2022-06-17T12:44:57.399 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2022-06-17T12:44:57.437 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-06-17T12:44:57.560 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-06-17T12:44:57.581 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2022-06-17T12:44:57.686 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pluggy (0.13.0-2) ... 2022-06-17T12:44:57.687 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-logutils (0.3.3-7) ... 2022-06-17T12:44:57.782 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2022-06-17T12:44:57.804 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-06-17T12:44:57.980 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2022-06-17T12:44:58.016 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up socat (1.7.3.3-2) ... 2022-06-17T12:44:58.017 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2022-06-17T12:44:58.094 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2022-06-17T12:44:58.106 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2022-06-17T12:44:58.134 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-ceph-common (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:58.170 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2022-06-17T12:44:58.374 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-prettytable (0.7.2-5) ... 2022-06-17T12:44:58.454 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-html5lib (1.0.1-2) ... 2022-06-17T12:44:58.499 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2022-06-17T12:44:58.534 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2022-06-17T12:44:58.691 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2022-06-17T12:44:58.796 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-06-17T12:44:58.807 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-logutils (0.3.3-7) ... 2022-06-17T12:44:58.862 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up librbd1 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:58.865 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up socat (1.7.3.3-2) ... 2022-06-17T12:44:58.929 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2022-06-17T12:44:58.956 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-pluggy (0.13.0-2) ... 2022-06-17T12:44:58.994 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-ceph-common (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:44:59.033 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-06-17T12:44:59.087 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2022-06-17T12:44:59.153 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2022-06-17T12:44:59.249 INFO:teuthology.orchestra.run.smithi143.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2022-06-17T12:44:59.254 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2022-06-17T12:44:59.255 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2022-06-17T12:44:59.279 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-06-17T12:44:59.309 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-html5lib (1.0.1-2) ... 2022-06-17T12:44:59.394 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2022-06-17T12:44:59.541 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-prettytable (0.7.2-5) ... 2022-06-17T12:44:59.543 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2022-06-17T12:44:59.587 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2022-06-17T12:44:59.656 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-06-17T12:44:59.674 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-06-17T12:44:59.683 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-rsa (4.0-3) ... 2022-06-17T12:44:59.718 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2022-06-17T12:44:59.800 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-pluggy (0.13.0-2) ... 2022-06-17T12:44:59.871 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2022-06-17T12:44:59.987 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2022-06-17T12:44:59.988 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-webtest (2.0.32-2) ... 2022-06-17T12:45:00.027 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-logutils (0.3.3-7) ... 2022-06-17T12:45:00.075 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2022-06-17T12:45:00.173 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2022-06-17T12:45:00.227 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2022-06-17T12:45:00.283 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mgr-modules-core (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:00.325 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2022-06-17T12:45:00.355 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-06-17T12:45:00.421 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-fuse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:00.488 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-06-17T12:45:00.507 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2022-06-17T12:45:00.508 INFO:teuthology.orchestra.run.smithi136.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2022-06-17T12:45:00.529 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2022-06-17T12:45:00.589 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-webob (1:1.8.5-2) ... 2022-06-17T12:45:00.652 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-06-17T12:45:00.652 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-06-17T12:45:00.733 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-prettytable (0.7.2-5) ... 2022-06-17T12:45:00.789 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2022-06-17T12:45:00.792 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-logutils (0.3.3-7) ... 2022-06-17T12:45:00.876 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-06-17T12:45:00.949 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-06-17T12:45:01.013 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2022-06-17T12:45:01.101 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2022-06-17T12:45:01.133 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libcephfs-dev (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:01.196 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up librados2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:01.243 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2022-06-17T12:45:01.286 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2022-06-17T12:45:01.304 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2022-06-17T12:45:01.305 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-06-17T12:45:01.347 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2022-06-17T12:45:01.535 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-prettytable (0.7.2-5) ... 2022-06-17T12:45:01.574 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pastescript (2.0.2-4) ... 2022-06-17T12:45:01.616 INFO:teuthology.orchestra.run.smithi036.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2022-06-17T12:45:01.645 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-06-17T12:45:01.781 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-06-17T12:45:01.812 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2022-06-17T12:45:01.903 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2022-06-17T12:45:01.904 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2022-06-17T12:45:01.955 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-webob (1:1.8.5-2) ... 2022-06-17T12:45:02.049 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-06-17T12:45:02.090 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2022-06-17T12:45:02.254 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-rados (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:02.317 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-06-17T12:45:02.355 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2022-06-17T12:45:02.389 INFO:teuthology.orchestra.run.smithi070.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2022-06-17T12:45:02.390 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-rbd (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:02.559 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up rbd-fuse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:02.580 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librados2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:02.705 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-routes (2.4.1-2) ... 2022-06-17T12:45:02.708 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2022-06-17T12:45:02.708 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2022-06-17T12:45:02.748 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-06-17T12:45:02.779 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up librgw2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:02.835 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-06-17T12:45:02.893 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-06-17T12:45:02.906 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-06-17T12:45:03.034 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-webob (1:1.8.5-2) ... 2022-06-17T12:45:03.044 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pytest (4.6.9-1) ... 2022-06-17T12:45:03.065 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-google-auth (1.5.1-2) ... 2022-06-17T12:45:03.151 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2022-06-17T12:45:03.368 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-rgw (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:03.384 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-06-17T12:45:03.425 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2022-06-17T12:45:03.516 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-cephfs (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:03.534 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-06-17T12:45:03.645 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up librados2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:03.646 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-06-17T12:45:03.714 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2022-06-17T12:45:03.775 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-webob (1:1.8.5-2) ... 2022-06-17T12:45:03.804 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2022-06-17T12:45:03.817 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up cephfs-shell (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:03.944 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2022-06-17T12:45:03.967 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-bs4 (4.8.2-1) ... 2022-06-17T12:45:04.098 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-06-17T12:45:04.305 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librgw2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:04.314 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libcephfs2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:04.386 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up librados2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:04.437 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2022-06-17T12:45:04.437 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-06-17T12:45:04.519 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2022-06-17T12:45:04.524 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pytest (4.6.9-1) ... 2022-06-17T12:45:04.741 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libradosstriper1 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:04.862 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-06-17T12:45:04.878 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2022-06-17T12:45:04.897 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-common (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:05.186 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2022-06-17T12:45:05.188 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-06-17T12:45:05.323 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up librgw2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:05.454 INFO:teuthology.orchestra.run.smithi107.stdout:Adding group ceph....done 2022-06-17T12:45:05.469 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-06-17T12:45:05.545 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-bs4 (4.8.2-1) ... 2022-06-17T12:45:05.621 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-pytest (4.6.9-1) ... 2022-06-17T12:45:05.799 INFO:teuthology.orchestra.run.smithi107.stdout:Adding system user ceph....done 2022-06-17T12:45:05.882 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libcephfs2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:05.980 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2022-06-17T12:45:06.027 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up librgw2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:06.027 INFO:teuthology.orchestra.run.smithi107.stdout:Setting system user ceph properties....done 2022-06-17T12:45:06.028 INFO:teuthology.orchestra.run.smithi107.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2022-06-17T12:45:06.077 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2022-06-17T12:45:06.134 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-06-17T12:45:06.174 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2022-06-17T12:45:06.222 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up librbd1 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:06.274 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-pytest (4.6.9-1) ... 2022-06-17T12:45:06.282 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libradosstriper1 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:06.342 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2022-06-17T12:45:06.371 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-06-17T12:45:06.406 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-06-17T12:45:06.465 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2022-06-17T12:45:06.506 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2022-06-17T12:45:06.568 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-06-17T12:45:06.632 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-bs4 (4.8.2-1) ... 2022-06-17T12:45:06.643 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2022-06-17T12:45:06.744 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-06-17T12:45:06.868 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2022-06-17T12:45:06.978 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-test (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:06.979 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2022-06-17T12:45:07.002 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libcephfs2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:07.003 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-rsa (4.0-3) ... 2022-06-17T12:45:07.102 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up radosgw (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:07.140 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2022-06-17T12:45:07.231 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-bs4 (4.8.2-1) ... 2022-06-17T12:45:07.278 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-webtest (2.0.32-2) ... 2022-06-17T12:45:07.367 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-06-17T12:45:07.368 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-06-17T12:45:07.483 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libradosstriper1 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:07.579 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up libcephfs2 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:07.597 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr-modules-core (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:07.619 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-06-17T12:45:07.715 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2022-06-17T12:45:07.716 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-fuse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:07.758 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librbd1 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:07.849 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-06-17T12:45:07.943 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-06-17T12:45:07.944 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-06-17T12:45:07.944 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-06-17T12:45:08.001 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2022-06-17T12:45:08.063 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up libradosstriper1 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:08.076 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-base (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:08.154 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-06-17T12:45:08.222 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-06-17T12:45:08.418 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libcephfs-dev (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:08.454 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2022-06-17T12:45:08.473 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2022-06-17T12:45:08.521 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-06-17T12:45:08.552 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-06-17T12:45:08.577 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-rsa (4.0-3) ... 2022-06-17T12:45:08.787 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pastescript (2.0.2-4) ... 2022-06-17T12:45:08.923 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-webtest (2.0.32-2) ... 2022-06-17T12:45:08.961 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mds (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:09.009 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up librbd1 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:09.115 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2022-06-17T12:45:09.143 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-06-17T12:45:09.175 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-06-17T12:45:09.176 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-06-17T12:45:09.232 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-modules-core (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:09.296 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2022-06-17T12:45:09.383 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-fuse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:09.427 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-06-17T12:45:09.486 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-rados (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:09.571 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-rbd (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:09.579 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-06-17T12:45:09.579 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-06-17T12:45:09.591 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up librbd1 (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:09.646 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mgr (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:09.691 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up rbd-fuse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:09.746 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2022-06-17T12:45:09.767 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-06-17T12:45:09.818 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-routes (2.4.1-2) ... 2022-06-17T12:45:09.891 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-rsa (4.0-3) ... 2022-06-17T12:45:09.891 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2022-06-17T12:45:09.892 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-06-17T12:45:09.907 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-06-17T12:45:10.047 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-06-17T12:45:10.087 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libcephfs-dev (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:10.143 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-google-auth (1.5.1-2) ... 2022-06-17T12:45:10.200 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-webtest (2.0.32-2) ... 2022-06-17T12:45:10.212 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-06-17T12:45:10.366 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2022-06-17T12:45:10.383 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:10.444 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-rgw (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:10.496 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-rsa (4.0-3) ... 2022-06-17T12:45:10.502 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pastescript (2.0.2-4) ... 2022-06-17T12:45:10.529 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-osd (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:10.531 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-mgr-modules-core (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:10.582 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-cephfs (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:10.659 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-fuse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:10.809 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-webtest (2.0.32-2) ... 2022-06-17T12:45:10.834 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2022-06-17T12:45:10.884 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-06-17T12:45:10.885 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-06-17T12:45:10.895 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up cephfs-shell (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:10.998 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2022-06-17T12:45:11.029 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-06-17T12:45:11.029 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-06-17T12:45:11.099 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up ceph-mgr-modules-core (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:11.198 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-rados (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:11.225 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up ceph-fuse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:11.315 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-rbd (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:11.375 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libcephfs-dev (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:11.467 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up rbd-fuse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:11.481 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-06-17T12:45:11.482 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-06-17T12:45:11.522 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mgr-k8sevents (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:11.523 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-06-17T12:45:11.592 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-routes (2.4.1-2) ... 2022-06-17T12:45:11.666 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mgr-diskprediction-local (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:11.809 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mon (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:11.825 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-pastescript (2.0.2-4) ... 2022-06-17T12:45:11.908 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-common (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:11.961 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-google-auth (1.5.1-2) ... 2022-06-17T12:45:11.988 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up libcephfs-dev (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:12.039 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-06-17T12:45:12.040 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-06-17T12:45:12.111 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-06-17T12:45:12.173 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2022-06-17T12:45:12.259 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-rgw (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:12.382 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cephfs (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:12.398 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-pastescript (2.0.2-4) ... 2022-06-17T12:45:12.457 INFO:teuthology.orchestra.run.smithi143.stdout:Adding group ceph....done 2022-06-17T12:45:12.499 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mgr-cephadm (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:12.531 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-rados (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:12.638 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:12.682 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2022-06-17T12:45:12.682 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-rbd (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:12.707 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up cephfs-shell (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:12.763 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mgr-dashboard (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:12.827 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2022-06-17T12:45:12.828 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up rbd-fuse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:12.831 INFO:teuthology.orchestra.run.smithi143.stdout:Adding system user ceph....done 2022-06-17T12:45:12.958 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-routes (2.4.1-2) ... 2022-06-17T12:45:12.958 INFO:teuthology.orchestra.run.smithi107.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2022-06-17T12:45:13.018 INFO:teuthology.orchestra.run.smithi143.stdout:Setting system user ceph properties....done 2022-06-17T12:45:13.019 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-rados (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:13.032 INFO:teuthology.orchestra.run.smithi143.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2022-06-17T12:45:13.132 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-rbd (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:13.190 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2022-06-17T12:45:13.284 INFO:teuthology.orchestra.run.smithi107.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-06-17T12:45:13.285 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up rbd-fuse (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:13.340 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-google-auth (1.5.1-2) ... 2022-06-17T12:45:13.401 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-routes (2.4.1-2) ... 2022-06-17T12:45:13.530 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2022-06-17T12:45:13.665 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-rgw (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:13.762 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-common (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:13.792 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-cephfs (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:13.820 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-google-auth (1.5.1-2) ... 2022-06-17T12:45:13.999 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-test (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:14.116 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up cephfs-shell (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:14.130 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up radosgw (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:14.157 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-rgw (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:14.243 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2022-06-17T12:45:14.375 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-06-17T12:45:14.376 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-06-17T12:45:14.386 INFO:teuthology.orchestra.run.smithi136.stdout:Adding group ceph....done 2022-06-17T12:45:14.762 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-cephfs (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:14.806 INFO:teuthology.orchestra.run.smithi136.stdout:Adding system user ceph....done 2022-06-17T12:45:15.022 INFO:teuthology.orchestra.run.smithi136.stdout:Setting system user ceph properties....done 2022-06-17T12:45:15.029 INFO:teuthology.orchestra.run.smithi136.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2022-06-17T12:45:15.054 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up cephfs-shell (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:15.191 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2022-06-17T12:45:15.192 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2022-06-17T12:45:15.197 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-common (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:15.224 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-base (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:15.523 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2022-06-17T12:45:15.603 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2022-06-17T12:45:15.835 INFO:teuthology.orchestra.run.smithi036.stdout:Adding group ceph....done 2022-06-17T12:45:16.039 INFO:teuthology.orchestra.run.smithi107.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-06-17T12:45:16.082 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-test (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:16.085 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mds (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:16.122 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up ceph-common (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:16.203 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up radosgw (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:16.268 INFO:teuthology.orchestra.run.smithi036.stdout:Adding system user ceph....done 2022-06-17T12:45:16.330 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-06-17T12:45:16.331 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-06-17T12:45:16.432 INFO:teuthology.orchestra.run.smithi107.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-06-17T12:45:16.441 DEBUG:teuthology.orchestra.run.smithi107:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2022-06-17T12:45:16.470 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-06-17T12:45:16.473 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-06-17T12:45:16.495 INFO:teuthology.orchestra.run.smithi036.stdout:Setting system user ceph properties....done 2022-06-17T12:45:16.508 INFO:teuthology.orchestra.run.smithi036.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2022-06-17T12:45:16.547 INFO:teuthology.orchestra.run.smithi107.stdout:Reading package lists... 2022-06-17T12:45:16.661 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2022-06-17T12:45:16.735 INFO:teuthology.orchestra.run.smithi107.stdout:Building dependency tree... 2022-06-17T12:45:16.736 INFO:teuthology.orchestra.run.smithi107.stdout:Reading state information... 2022-06-17T12:45:16.741 INFO:teuthology.orchestra.run.smithi070.stdout:Adding group ceph....done 2022-06-17T12:45:16.779 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:16.888 INFO:teuthology.orchestra.run.smithi107.stdout:libtool-bin is already the newest version (2.4.6-14). 2022-06-17T12:45:16.890 INFO:teuthology.orchestra.run.smithi107.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2022-06-17T12:45:16.891 INFO:teuthology.orchestra.run.smithi107.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.3). 2022-06-17T12:45:16.891 INFO:teuthology.orchestra.run.smithi107.stdout:The following packages were automatically installed and are no longer required: 2022-06-17T12:45:16.892 INFO:teuthology.orchestra.run.smithi107.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-06-17T12:45:16.892 INFO:teuthology.orchestra.run.smithi107.stdout:Use 'sudo apt autoremove' to remove them. 2022-06-17T12:45:16.893 INFO:teuthology.orchestra.run.smithi107.stdout:Suggested packages: 2022-06-17T12:45:16.893 INFO:teuthology.orchestra.run.smithi107.stdout: bison-doc flex-doc indent-doc libssl-doc 2022-06-17T12:45:16.937 INFO:teuthology.orchestra.run.smithi107.stdout:The following NEW packages will be installed: 2022-06-17T12:45:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2022-06-17T12:45:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: libssl-dev 2022-06-17T12:45:17.008 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-06-17T12:45:17.009 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-06-17T12:45:17.021 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2022-06-17T12:45:17.099 INFO:teuthology.orchestra.run.smithi070.stdout:Adding system user ceph....done 2022-06-17T12:45:17.103 INFO:teuthology.orchestra.run.smithi107.stdout:0 upgraded, 10 newly installed, 0 to remove and 2 not upgraded. 2022-06-17T12:45:17.103 INFO:teuthology.orchestra.run.smithi107.stdout:Need to get 2,914 kB of archives. 2022-06-17T12:45:17.108 INFO:teuthology.orchestra.run.smithi107.stdout:After this operation, 12.4 MB of additional disk space will be used. 2022-06-17T12:45:17.109 INFO:teuthology.orchestra.run.smithi107.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2022-06-17T12:45:17.199 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-base (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:17.351 INFO:teuthology.orchestra.run.smithi070.stdout:Setting system user ceph properties....done 2022-06-17T12:45:17.369 INFO:teuthology.orchestra.run.smithi070.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2022-06-17T12:45:17.467 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:17.512 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2022-06-17T12:45:17.528 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-test (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:17.589 INFO:teuthology.orchestra.run.smithi107.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2022-06-17T12:45:17.590 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2022-06-17T12:45:17.610 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-osd (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:17.637 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up radosgw (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:17.731 INFO:teuthology.orchestra.run.smithi107.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2022-06-17T12:45:17.731 INFO:teuthology.orchestra.run.smithi107.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2022-06-17T12:45:17.748 INFO:teuthology.orchestra.run.smithi107.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2022-06-17T12:45:17.748 INFO:teuthology.orchestra.run.smithi107.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2022-06-17T12:45:17.748 INFO:teuthology.orchestra.run.smithi107.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2022-06-17T12:45:17.749 INFO:teuthology.orchestra.run.smithi107.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2022-06-17T12:45:17.752 INFO:teuthology.orchestra.run.smithi107.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2022-06-17T12:45:17.752 INFO:teuthology.orchestra.run.smithi107.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.13 [1,584 kB] 2022-06-17T12:45:17.857 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2022-06-17T12:45:17.937 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-06-17T12:45:17.938 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-06-17T12:45:18.046 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mds (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:18.068 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-06-17T12:45:18.068 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-06-17T12:45:18.205 INFO:teuthology.orchestra.run.smithi107.stdout:Fetched 2,914 kB in 1s (3,132 kB/s) 2022-06-17T12:45:18.300 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-06-17T12:45:18.300 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-06-17T12:45:18.326 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up ceph-test (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:18.348 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package flex. 2022-06-17T12:45:18.425 INFO:teuthology.orchestra.run.smithi107.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 146191 files and directories currently installed.) 2022-06-17T12:45:18.427 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../0-flex_2.6.4-6.2_amd64.deb ... 2022-06-17T12:45:18.448 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up radosgw (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:18.456 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking flex (2.6.4-6.2) ... 2022-06-17T12:45:18.533 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr-k8sevents (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:18.663 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr-diskprediction-local (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:18.679 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-base (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:18.744 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-06-17T12:45:18.745 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-06-17T12:45:18.754 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:18.801 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mon (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:19.001 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package bison. 2022-06-17T12:45:19.002 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../1-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2022-06-17T12:45:19.011 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-06-17T12:45:19.011 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-06-17T12:45:19.035 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-06-17T12:45:19.035 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-06-17T12:45:19.040 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2022-06-17T12:45:19.058 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2022-06-17T12:45:19.470 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up ceph-base (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:19.480 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:19.543 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package liblzo2-2:amd64. 2022-06-17T12:45:19.543 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr-cephadm (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:19.553 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-mds (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:19.565 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../2-liblzo2-2_2.10-2_amd64.deb ... 2022-06-17T12:45:19.640 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2022-06-17T12:45:19.659 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-osd (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:19.687 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:19.781 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-06-17T12:45:19.781 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-06-17T12:45:19.826 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr-dashboard (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:19.849 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2022-06-17T12:45:19.971 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2022-06-17T12:45:20.125 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package dump. 2022-06-17T12:45:20.135 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../3-dump_0.4b46-6_amd64.deb ... 2022-06-17T12:45:20.176 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking dump (0.4b46-6) ... 2022-06-17T12:45:20.259 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-mgr (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:20.260 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-06-17T12:45:20.260 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-06-17T12:45:20.270 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-06-17T12:45:20.346 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up ceph-mds (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:20.485 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-06-17T12:45:20.486 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-06-17T12:45:20.491 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package indent. 2022-06-17T12:45:20.503 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../4-indent_2.2.12-1_amd64.deb ... 2022-06-17T12:45:20.550 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking indent (2.2.12-1) ... 2022-06-17T12:45:20.610 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-06-17T12:45:20.611 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-06-17T12:45:20.726 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-k8sevents (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:20.845 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libaio-dev:amd64. 2022-06-17T12:45:20.852 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../5-libaio-dev_0.3.112-5_amd64.deb ... 2022-06-17T12:45:20.862 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-diskprediction-local (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:20.899 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2022-06-17T12:45:20.980 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:21.033 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mon (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:21.086 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up ceph-mgr (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:21.108 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-osd (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:21.209 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libelf-dev:amd64. 2022-06-17T12:45:21.239 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../6-libelf-dev_0.176-1.1build1_amd64.deb ... 2022-06-17T12:45:21.256 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-06-17T12:45:21.257 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-06-17T12:45:21.270 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2022-06-17T12:45:21.322 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-06-17T12:45:21.323 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-06-17T12:45:21.590 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-06-17T12:45:21.591 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-06-17T12:45:21.637 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libfl2:amd64. 2022-06-17T12:45:21.654 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../7-libfl2_2.6.4-6.2_amd64.deb ... 2022-06-17T12:45:21.696 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2022-06-17T12:45:21.736 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-cephadm (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:21.823 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:21.890 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:21.935 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libfl-dev:amd64. 2022-06-17T12:45:21.956 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../8-libfl-dev_2.6.4-6.2_amd64.deb ... 2022-06-17T12:45:21.988 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up ceph-osd (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:21.989 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2022-06-17T12:45:22.033 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-dashboard (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:22.080 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-mgr-k8sevents (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:22.196 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2022-06-17T12:45:22.242 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-mgr-diskprediction-local (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:22.271 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libssl-dev:amd64. 2022-06-17T12:45:22.285 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../9-libssl-dev_1.1.1f-1ubuntu2.13_amd64.deb ... 2022-06-17T12:45:22.346 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.13) ... 2022-06-17T12:45:22.414 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-mon (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:22.489 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-06-17T12:45:22.520 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-06-17T12:45:22.538 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-06-17T12:45:22.612 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-06-17T12:45:22.613 INFO:teuthology.orchestra.run.smithi036.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-06-17T12:45:22.811 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up flex (2.6.4-6.2) ... 2022-06-17T12:45:22.930 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2022-06-17T12:45:22.970 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-06-17T12:45:23.004 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up ceph-mgr-k8sevents (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:23.056 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2022-06-17T12:45:23.105 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-mgr-cephadm (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:23.133 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up ceph-mgr-diskprediction-local (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:23.181 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.13) ... 2022-06-17T12:45:23.250 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:23.319 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up ceph-mon (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:23.340 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2022-06-17T12:45:23.444 INFO:teuthology.orchestra.run.smithi143.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-06-17T12:45:23.445 DEBUG:teuthology.orchestra.run.smithi143:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2022-06-17T12:45:23.446 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2022-06-17T12:45:23.471 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up ceph-mgr-dashboard (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:23.540 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up dump (0.4b46-6) ... 2022-06-17T12:45:23.554 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-06-17T12:45:23.555 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-06-17T12:45:23.609 INFO:teuthology.orchestra.run.smithi143.stdout:Reading package lists... 2022-06-17T12:45:23.613 INFO:teuthology.orchestra.run.smithi107.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2022-06-17T12:45:23.673 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2022-06-17T12:45:23.758 INFO:teuthology.orchestra.run.smithi107.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2022-06-17T12:45:23.779 INFO:teuthology.orchestra.run.smithi143.stdout:Building dependency tree... 2022-06-17T12:45:23.780 INFO:teuthology.orchestra.run.smithi143.stdout:Reading state information... 2022-06-17T12:45:23.862 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up indent (2.2.12-1) ... 2022-06-17T12:45:23.939 INFO:teuthology.orchestra.run.smithi143.stdout:libtool-bin is already the newest version (2.4.6-14). 2022-06-17T12:45:23.939 INFO:teuthology.orchestra.run.smithi143.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2022-06-17T12:45:23.940 INFO:teuthology.orchestra.run.smithi143.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.3). 2022-06-17T12:45:23.940 INFO:teuthology.orchestra.run.smithi143.stdout:The following packages were automatically installed and are no longer required: 2022-06-17T12:45:23.940 INFO:teuthology.orchestra.run.smithi143.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-06-17T12:45:23.940 INFO:teuthology.orchestra.run.smithi143.stdout:Use 'sudo apt autoremove' to remove them. 2022-06-17T12:45:23.941 INFO:teuthology.orchestra.run.smithi143.stdout:Suggested packages: 2022-06-17T12:45:23.941 INFO:teuthology.orchestra.run.smithi143.stdout: bison-doc flex-doc indent-doc libssl-doc 2022-06-17T12:45:23.963 INFO:teuthology.orchestra.run.smithi143.stdout:The following NEW packages will be installed: 2022-06-17T12:45:23.963 INFO:teuthology.orchestra.run.smithi143.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2022-06-17T12:45:23.963 INFO:teuthology.orchestra.run.smithi143.stdout: libssl-dev 2022-06-17T12:45:23.995 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2022-06-17T12:45:24.027 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up ceph-mgr-cephadm (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:24.088 INFO:teuthology.orchestra.run.smithi143.stdout:0 upgraded, 10 newly installed, 0 to remove and 2 not upgraded. 2022-06-17T12:45:24.089 INFO:teuthology.orchestra.run.smithi143.stdout:Need to get 2,914 kB of archives. 2022-06-17T12:45:24.090 INFO:teuthology.orchestra.run.smithi143.stdout:After this operation, 12.4 MB of additional disk space will be used. 2022-06-17T12:45:24.091 INFO:teuthology.orchestra.run.smithi143.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2022-06-17T12:45:24.116 INFO:teuthology.orchestra.run.smithi107.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-06-17T12:45:24.169 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up ceph (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:24.242 INFO:teuthology.orchestra.run.smithi036.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2022-06-17T12:45:24.266 INFO:teuthology.orchestra.run.smithi107.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-06-17T12:45:24.330 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up ceph-mgr-dashboard (15.2.16-260-gb3641a55-1focal) ... 2022-06-17T12:45:24.365 INFO:teuthology.orchestra.run.smithi143.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2022-06-17T12:45:24.447 INFO:teuthology.orchestra.run.smithi143.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2022-06-17T12:45:24.448 INFO:teuthology.orchestra.run.smithi143.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2022-06-17T12:45:24.513 INFO:teuthology.orchestra.run.smithi143.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2022-06-17T12:45:24.513 INFO:teuthology.orchestra.run.smithi143.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2022-06-17T12:45:24.514 INFO:teuthology.orchestra.run.smithi143.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2022-06-17T12:45:24.515 INFO:teuthology.orchestra.run.smithi143.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2022-06-17T12:45:24.515 INFO:teuthology.orchestra.run.smithi143.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2022-06-17T12:45:24.516 INFO:teuthology.orchestra.run.smithi143.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.13 [1,584 kB] 2022-06-17T12:45:24.517 INFO:teuthology.orchestra.run.smithi070.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2022-06-17T12:45:24.582 INFO:teuthology.orchestra.run.smithi036.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-06-17T12:45:24.832 INFO:teuthology.orchestra.run.smithi070.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-06-17T12:45:24.881 INFO:teuthology.orchestra.run.smithi143.stdout:Fetched 2,914 kB in 1s (5,209 kB/s) 2022-06-17T12:45:25.030 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package flex. 2022-06-17T12:45:25.095 INFO:teuthology.orchestra.run.smithi143.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 146191 files and directories currently installed.) 2022-06-17T12:45:25.095 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../0-flex_2.6.4-6.2_amd64.deb ... 2022-06-17T12:45:25.124 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking flex (2.6.4-6.2) ... 2022-06-17T12:45:25.330 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-06-17T12:45:25.722 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package bison. 2022-06-17T12:45:25.739 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../1-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2022-06-17T12:45:25.752 INFO:teuthology.orchestra.run.smithi136.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-06-17T12:45:25.755 DEBUG:teuthology.orchestra.run.smithi136:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2022-06-17T12:45:25.786 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2022-06-17T12:45:25.834 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2022-06-17T12:45:26.009 INFO:teuthology.orchestra.run.smithi136.stdout:Building dependency tree... 2022-06-17T12:45:26.010 INFO:teuthology.orchestra.run.smithi136.stdout:Reading state information... 2022-06-17T12:45:26.174 INFO:teuthology.orchestra.run.smithi136.stdout:libtool-bin is already the newest version (2.4.6-14). 2022-06-17T12:45:26.176 INFO:teuthology.orchestra.run.smithi136.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2022-06-17T12:45:26.176 INFO:teuthology.orchestra.run.smithi136.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.3). 2022-06-17T12:45:26.177 INFO:teuthology.orchestra.run.smithi136.stdout:The following packages were automatically installed and are no longer required: 2022-06-17T12:45:26.177 INFO:teuthology.orchestra.run.smithi136.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-06-17T12:45:26.178 INFO:teuthology.orchestra.run.smithi136.stdout:Use 'sudo apt autoremove' to remove them. 2022-06-17T12:45:26.179 INFO:teuthology.orchestra.run.smithi136.stdout:Suggested packages: 2022-06-17T12:45:26.179 INFO:teuthology.orchestra.run.smithi136.stdout: bison-doc flex-doc indent-doc libssl-doc 2022-06-17T12:45:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2022-06-17T12:45:26.208 INFO:teuthology.orchestra.run.smithi136.stdout:The following NEW packages will be installed: 2022-06-17T12:45:26.208 INFO:teuthology.orchestra.run.smithi136.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2022-06-17T12:45:26.209 INFO:teuthology.orchestra.run.smithi136.stdout: libssl-dev 2022-06-17T12:45:26.275 INFO:teuthology.orchestra.run.smithi136.stdout:0 upgraded, 10 newly installed, 0 to remove and 2 not upgraded. 2022-06-17T12:45:26.275 INFO:teuthology.orchestra.run.smithi136.stdout:Need to get 2,914 kB of archives. 2022-06-17T12:45:26.276 INFO:teuthology.orchestra.run.smithi136.stdout:After this operation, 12.4 MB of additional disk space will be used. 2022-06-17T12:45:26.276 INFO:teuthology.orchestra.run.smithi136.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2022-06-17T12:45:26.286 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package liblzo2-2:amd64. 2022-06-17T12:45:26.305 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../2-liblzo2-2_2.10-2_amd64.deb ... 2022-06-17T12:45:26.385 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2022-06-17T12:45:26.410 INFO:teuthology.orchestra.run.smithi136.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2022-06-17T12:45:26.450 INFO:teuthology.orchestra.run.smithi136.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2022-06-17T12:45:26.454 INFO:teuthology.orchestra.run.smithi136.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2022-06-17T12:45:26.459 INFO:teuthology.orchestra.run.smithi136.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2022-06-17T12:45:26.475 INFO:teuthology.orchestra.run.smithi136.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2022-06-17T12:45:26.476 INFO:teuthology.orchestra.run.smithi136.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2022-06-17T12:45:26.476 INFO:teuthology.orchestra.run.smithi136.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2022-06-17T12:45:26.476 INFO:teuthology.orchestra.run.smithi136.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2022-06-17T12:45:26.477 INFO:teuthology.orchestra.run.smithi136.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.13 [1,584 kB] 2022-06-17T12:45:26.724 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package dump. 2022-06-17T12:45:26.748 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../3-dump_0.4b46-6_amd64.deb ... 2022-06-17T12:45:26.780 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking dump (0.4b46-6) ... 2022-06-17T12:45:26.863 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 2,914 kB in 0s (9,683 kB/s) 2022-06-17T12:45:26.904 INFO:teuthology.orchestra.run.smithi107.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-06-17T12:45:26.905 DEBUG:teuthology.parallel:result is None 2022-06-17T12:45:27.026 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package flex. 2022-06-17T12:45:27.086 INFO:teuthology.orchestra.run.smithi136.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 146191 files and directories currently installed.) 2022-06-17T12:45:27.092 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package indent. 2022-06-17T12:45:27.110 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../0-flex_2.6.4-6.2_amd64.deb ... 2022-06-17T12:45:27.114 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../4-indent_2.2.12-1_amd64.deb ... 2022-06-17T12:45:27.118 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking flex (2.6.4-6.2) ... 2022-06-17T12:45:27.150 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking indent (2.2.12-1) ... 2022-06-17T12:45:27.396 INFO:teuthology.orchestra.run.smithi036.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-06-17T12:45:27.456 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libaio-dev:amd64. 2022-06-17T12:45:27.472 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../5-libaio-dev_0.3.112-5_amd64.deb ... 2022-06-17T12:45:27.512 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2022-06-17T12:45:27.549 INFO:teuthology.orchestra.run.smithi070.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-06-17T12:45:27.644 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package bison. 2022-06-17T12:45:27.658 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../1-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2022-06-17T12:45:27.736 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2022-06-17T12:45:27.785 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libelf-dev:amd64. 2022-06-17T12:45:27.798 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../6-libelf-dev_0.176-1.1build1_amd64.deb ... 2022-06-17T12:45:27.813 INFO:teuthology.orchestra.run.smithi036.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-06-17T12:45:27.816 DEBUG:teuthology.orchestra.run.smithi036:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2022-06-17T12:45:27.846 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2022-06-17T12:45:27.893 INFO:teuthology.orchestra.run.smithi036.stdout:Reading package lists... 2022-06-17T12:45:27.973 INFO:teuthology.orchestra.run.smithi070.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-06-17T12:45:27.976 DEBUG:teuthology.orchestra.run.smithi070:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2022-06-17T12:45:28.049 INFO:teuthology.orchestra.run.smithi070.stdout:Reading package lists... 2022-06-17T12:45:28.072 INFO:teuthology.orchestra.run.smithi036.stdout:Building dependency tree... 2022-06-17T12:45:28.074 INFO:teuthology.orchestra.run.smithi036.stdout:Reading state information... 2022-06-17T12:45:28.176 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libfl2:amd64. 2022-06-17T12:45:28.199 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../7-libfl2_2.6.4-6.2_amd64.deb ... 2022-06-17T12:45:28.200 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package liblzo2-2:amd64. 2022-06-17T12:45:28.213 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../2-liblzo2-2_2.10-2_amd64.deb ... 2022-06-17T12:45:28.226 INFO:teuthology.orchestra.run.smithi070.stdout:Building dependency tree... 2022-06-17T12:45:28.227 INFO:teuthology.orchestra.run.smithi070.stdout:Reading state information... 2022-06-17T12:45:28.235 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2022-06-17T12:45:28.236 INFO:teuthology.orchestra.run.smithi036.stdout:libtool-bin is already the newest version (2.4.6-14). 2022-06-17T12:45:28.236 INFO:teuthology.orchestra.run.smithi036.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2022-06-17T12:45:28.236 INFO:teuthology.orchestra.run.smithi036.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.3). 2022-06-17T12:45:28.237 INFO:teuthology.orchestra.run.smithi036.stdout:The following packages were automatically installed and are no longer required: 2022-06-17T12:45:28.237 INFO:teuthology.orchestra.run.smithi036.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-06-17T12:45:28.242 INFO:teuthology.orchestra.run.smithi036.stdout:Use 'sudo apt autoremove' to remove them. 2022-06-17T12:45:28.242 INFO:teuthology.orchestra.run.smithi036.stdout:Suggested packages: 2022-06-17T12:45:28.242 INFO:teuthology.orchestra.run.smithi036.stdout: bison-doc flex-doc indent-doc libssl-doc 2022-06-17T12:45:28.275 INFO:teuthology.orchestra.run.smithi036.stdout:The following NEW packages will be installed: 2022-06-17T12:45:28.275 INFO:teuthology.orchestra.run.smithi036.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2022-06-17T12:45:28.276 INFO:teuthology.orchestra.run.smithi036.stdout: libssl-dev 2022-06-17T12:45:28.302 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2022-06-17T12:45:28.382 INFO:teuthology.orchestra.run.smithi070.stdout:libtool-bin is already the newest version (2.4.6-14). 2022-06-17T12:45:28.400 INFO:teuthology.orchestra.run.smithi070.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2022-06-17T12:45:28.401 INFO:teuthology.orchestra.run.smithi070.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.3). 2022-06-17T12:45:28.401 INFO:teuthology.orchestra.run.smithi070.stdout:The following packages were automatically installed and are no longer required: 2022-06-17T12:45:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-06-17T12:45:28.408 INFO:teuthology.orchestra.run.smithi070.stdout:Use 'sudo apt autoremove' to remove them. 2022-06-17T12:45:28.409 INFO:teuthology.orchestra.run.smithi070.stdout:Suggested packages: 2022-06-17T12:45:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: bison-doc flex-doc indent-doc libssl-doc 2022-06-17T12:45:28.416 INFO:teuthology.orchestra.run.smithi070.stdout:The following NEW packages will be installed: 2022-06-17T12:45:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2022-06-17T12:45:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: libssl-dev 2022-06-17T12:45:28.454 INFO:teuthology.orchestra.run.smithi036.stdout:0 upgraded, 10 newly installed, 0 to remove and 2 not upgraded. 2022-06-17T12:45:28.454 INFO:teuthology.orchestra.run.smithi036.stdout:Need to get 2,914 kB of archives. 2022-06-17T12:45:28.455 INFO:teuthology.orchestra.run.smithi036.stdout:After this operation, 12.4 MB of additional disk space will be used. 2022-06-17T12:45:28.455 INFO:teuthology.orchestra.run.smithi036.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2022-06-17T12:45:28.468 INFO:teuthology.orchestra.run.smithi070.stdout:0 upgraded, 10 newly installed, 0 to remove and 2 not upgraded. 2022-06-17T12:45:28.470 INFO:teuthology.orchestra.run.smithi070.stdout:Need to get 2,914 kB of archives. 2022-06-17T12:45:28.470 INFO:teuthology.orchestra.run.smithi070.stdout:After this operation, 12.4 MB of additional disk space will be used. 2022-06-17T12:45:28.471 INFO:teuthology.orchestra.run.smithi070.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2022-06-17T12:45:28.522 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libfl-dev:amd64. 2022-06-17T12:45:28.528 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../8-libfl-dev_2.6.4-6.2_amd64.deb ... 2022-06-17T12:45:28.567 INFO:teuthology.orchestra.run.smithi070.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2022-06-17T12:45:28.596 INFO:teuthology.orchestra.run.smithi070.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2022-06-17T12:45:28.597 INFO:teuthology.orchestra.run.smithi070.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2022-06-17T12:45:28.598 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2022-06-17T12:45:28.616 INFO:teuthology.orchestra.run.smithi070.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2022-06-17T12:45:28.617 INFO:teuthology.orchestra.run.smithi070.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2022-06-17T12:45:28.617 INFO:teuthology.orchestra.run.smithi070.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2022-06-17T12:45:28.617 INFO:teuthology.orchestra.run.smithi070.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2022-06-17T12:45:28.618 INFO:teuthology.orchestra.run.smithi070.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2022-06-17T12:45:28.618 INFO:teuthology.orchestra.run.smithi070.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.13 [1,584 kB] 2022-06-17T12:45:28.657 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package dump. 2022-06-17T12:45:28.672 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../3-dump_0.4b46-6_amd64.deb ... 2022-06-17T12:45:28.714 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking dump (0.4b46-6) ... 2022-06-17T12:45:28.843 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libssl-dev:amd64. 2022-06-17T12:45:28.867 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../9-libssl-dev_1.1.1f-1ubuntu2.13_amd64.deb ... 2022-06-17T12:45:28.899 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.13) ... 2022-06-17T12:45:28.942 INFO:teuthology.orchestra.run.smithi036.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2022-06-17T12:45:28.972 INFO:teuthology.orchestra.run.smithi070.stdout:Fetched 2,914 kB in 0s (14.5 MB/s) 2022-06-17T12:45:29.008 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package indent. 2022-06-17T12:45:29.044 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../4-indent_2.2.12-1_amd64.deb ... 2022-06-17T12:45:29.065 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking indent (2.2.12-1) ... 2022-06-17T12:45:29.083 INFO:teuthology.orchestra.run.smithi036.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2022-06-17T12:45:29.084 INFO:teuthology.orchestra.run.smithi036.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2022-06-17T12:45:29.119 INFO:teuthology.orchestra.run.smithi036.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2022-06-17T12:45:29.120 INFO:teuthology.orchestra.run.smithi036.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2022-06-17T12:45:29.120 INFO:teuthology.orchestra.run.smithi036.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2022-06-17T12:45:29.121 INFO:teuthology.orchestra.run.smithi036.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2022-06-17T12:45:29.121 INFO:teuthology.orchestra.run.smithi036.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2022-06-17T12:45:29.122 INFO:teuthology.orchestra.run.smithi036.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.13 [1,584 kB] 2022-06-17T12:45:29.141 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package flex. 2022-06-17T12:45:29.195 INFO:teuthology.orchestra.run.smithi070.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 146191 files and directories currently installed.) 2022-06-17T12:45:29.196 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../0-flex_2.6.4-6.2_amd64.deb ... 2022-06-17T12:45:29.246 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking flex (2.6.4-6.2) ... 2022-06-17T12:45:29.361 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libaio-dev:amd64. 2022-06-17T12:45:29.385 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up flex (2.6.4-6.2) ... 2022-06-17T12:45:29.385 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../5-libaio-dev_0.3.112-5_amd64.deb ... 2022-06-17T12:45:29.503 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2022-06-17T12:45:29.518 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2022-06-17T12:45:29.586 INFO:teuthology.orchestra.run.smithi036.stdout:Fetched 2,914 kB in 1s (3,113 kB/s) 2022-06-17T12:45:29.638 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2022-06-17T12:45:29.743 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package flex. 2022-06-17T12:45:29.747 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.13) ... 2022-06-17T12:45:29.788 INFO:teuthology.orchestra.run.smithi036.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 146191 files and directories currently installed.) 2022-06-17T12:45:29.790 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package bison. 2022-06-17T12:45:29.791 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../0-flex_2.6.4-6.2_amd64.deb ... 2022-06-17T12:45:29.804 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../1-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2022-06-17T12:45:29.839 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking flex (2.6.4-6.2) ... 2022-06-17T12:45:29.863 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libelf-dev:amd64. 2022-06-17T12:45:29.863 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2022-06-17T12:45:29.875 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../6-libelf-dev_0.176-1.1build1_amd64.deb ... 2022-06-17T12:45:29.887 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2022-06-17T12:45:29.914 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2022-06-17T12:45:29.988 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2022-06-17T12:45:30.114 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up dump (0.4b46-6) ... 2022-06-17T12:45:30.211 INFO:teuthology.orchestra.run.smithi143.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2022-06-17T12:45:30.271 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2022-06-17T12:45:30.332 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libfl2:amd64. 2022-06-17T12:45:30.347 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../7-libfl2_2.6.4-6.2_amd64.deb ... 2022-06-17T12:45:30.374 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package bison. 2022-06-17T12:45:30.384 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../1-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2022-06-17T12:45:30.399 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2022-06-17T12:45:30.419 INFO:teuthology.orchestra.run.smithi143.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2022-06-17T12:45:30.433 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2022-06-17T12:45:30.482 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package liblzo2-2:amd64. 2022-06-17T12:45:30.489 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up indent (2.2.12-1) ... 2022-06-17T12:45:30.514 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../2-liblzo2-2_2.10-2_amd64.deb ... 2022-06-17T12:45:30.590 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2022-06-17T12:45:30.616 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2022-06-17T12:45:30.667 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libfl-dev:amd64. 2022-06-17T12:45:30.685 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../8-libfl-dev_2.6.4-6.2_amd64.deb ... 2022-06-17T12:45:30.722 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2022-06-17T12:45:30.745 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-06-17T12:45:30.899 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-06-17T12:45:30.900 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package liblzo2-2:amd64. 2022-06-17T12:45:30.913 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../2-liblzo2-2_2.10-2_amd64.deb ... 2022-06-17T12:45:30.978 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package dump. 2022-06-17T12:45:30.996 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../3-dump_0.4b46-6_amd64.deb ... 2022-06-17T12:45:31.015 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libssl-dev:amd64. 2022-06-17T12:45:31.031 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2022-06-17T12:45:31.032 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../9-libssl-dev_1.1.1f-1ubuntu2.13_amd64.deb ... 2022-06-17T12:45:31.060 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking dump (0.4b46-6) ... 2022-06-17T12:45:31.073 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.13) ... 2022-06-17T12:45:31.402 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package indent. 2022-06-17T12:45:31.412 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../4-indent_2.2.12-1_amd64.deb ... 2022-06-17T12:45:31.430 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package dump. 2022-06-17T12:45:31.433 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../3-dump_0.4b46-6_amd64.deb ... 2022-06-17T12:45:31.459 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking indent (2.2.12-1) ... 2022-06-17T12:45:31.482 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking dump (0.4b46-6) ... 2022-06-17T12:45:31.578 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up flex (2.6.4-6.2) ... 2022-06-17T12:45:31.706 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2022-06-17T12:45:31.757 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package libaio-dev:amd64. 2022-06-17T12:45:31.773 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../5-libaio-dev_0.3.112-5_amd64.deb ... 2022-06-17T12:45:31.781 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package indent. 2022-06-17T12:45:31.803 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../4-indent_2.2.12-1_amd64.deb ... 2022-06-17T12:45:31.815 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2022-06-17T12:45:31.829 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2022-06-17T12:45:31.864 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking indent (2.2.12-1) ... 2022-06-17T12:45:31.988 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.13) ... 2022-06-17T12:45:32.114 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libaio-dev:amd64. 2022-06-17T12:45:32.133 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2022-06-17T12:45:32.133 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../5-libaio-dev_0.3.112-5_amd64.deb ... 2022-06-17T12:45:32.134 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package libelf-dev:amd64. 2022-06-17T12:45:32.173 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2022-06-17T12:45:32.174 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../6-libelf-dev_0.176-1.1build1_amd64.deb ... 2022-06-17T12:45:32.193 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2022-06-17T12:45:32.293 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2022-06-17T12:45:32.427 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up dump (0.4b46-6) ... 2022-06-17T12:45:32.481 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libelf-dev:amd64. 2022-06-17T12:45:32.482 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../6-libelf-dev_0.176-1.1build1_amd64.deb ... 2022-06-17T12:45:32.522 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2022-06-17T12:45:32.523 INFO:teuthology.orchestra.run.smithi136.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2022-06-17T12:45:32.579 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2022-06-17T12:45:32.587 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package libfl2:amd64. 2022-06-17T12:45:32.606 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../7-libfl2_2.6.4-6.2_amd64.deb ... 2022-06-17T12:45:32.667 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2022-06-17T12:45:32.700 INFO:teuthology.orchestra.run.smithi136.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2022-06-17T12:45:32.773 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up indent (2.2.12-1) ... 2022-06-17T12:45:32.870 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2022-06-17T12:45:32.871 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libfl2:amd64. 2022-06-17T12:45:32.891 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../7-libfl2_2.6.4-6.2_amd64.deb ... 2022-06-17T12:45:32.900 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2022-06-17T12:45:32.930 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2022-06-17T12:45:32.960 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package libfl-dev:amd64. 2022-06-17T12:45:32.973 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../8-libfl-dev_2.6.4-6.2_amd64.deb ... 2022-06-17T12:45:33.030 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2022-06-17T12:45:33.064 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-06-17T12:45:33.190 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libfl-dev:amd64. 2022-06-17T12:45:33.202 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-06-17T12:45:33.210 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../8-libfl-dev_2.6.4-6.2_amd64.deb ... 2022-06-17T12:45:33.243 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2022-06-17T12:45:33.298 INFO:teuthology.orchestra.run.smithi070.stdout:Selecting previously unselected package libssl-dev:amd64. 2022-06-17T12:45:33.323 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing to unpack .../9-libssl-dev_1.1.1f-1ubuntu2.13_amd64.deb ... 2022-06-17T12:45:33.354 INFO:teuthology.orchestra.run.smithi070.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.13) ... 2022-06-17T12:45:33.485 INFO:teuthology.orchestra.run.smithi036.stdout:Selecting previously unselected package libssl-dev:amd64. 2022-06-17T12:45:33.502 INFO:teuthology.orchestra.run.smithi036.stdout:Preparing to unpack .../9-libssl-dev_1.1.1f-1ubuntu2.13_amd64.deb ... 2022-06-17T12:45:33.544 INFO:teuthology.orchestra.run.smithi036.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.13) ... 2022-06-17T12:45:33.573 INFO:teuthology.orchestra.run.smithi143.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-06-17T12:45:33.583 DEBUG:teuthology.parallel:result is None 2022-06-17T12:45:33.837 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up flex (2.6.4-6.2) ... 2022-06-17T12:45:33.998 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2022-06-17T12:45:34.006 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up flex (2.6.4-6.2) ... 2022-06-17T12:45:34.123 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2022-06-17T12:45:34.249 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2022-06-17T12:45:34.279 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.13) ... 2022-06-17T12:45:34.417 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2022-06-17T12:45:34.436 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2022-06-17T12:45:34.603 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.13) ... 2022-06-17T12:45:34.605 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2022-06-17T12:45:34.727 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up dump (0.4b46-6) ... 2022-06-17T12:45:34.740 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2022-06-17T12:45:34.806 INFO:teuthology.orchestra.run.smithi070.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2022-06-17T12:45:34.879 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2022-06-17T12:45:34.907 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2022-06-17T12:45:34.995 INFO:teuthology.orchestra.run.smithi070.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2022-06-17T12:45:35.026 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up dump (0.4b46-6) ... 2022-06-17T12:45:35.065 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up indent (2.2.12-1) ... 2022-06-17T12:45:35.118 INFO:teuthology.orchestra.run.smithi036.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2022-06-17T12:45:35.181 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2022-06-17T12:45:35.216 INFO:teuthology.orchestra.run.smithi070.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2022-06-17T12:45:35.279 INFO:teuthology.orchestra.run.smithi036.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2022-06-17T12:45:35.348 INFO:teuthology.orchestra.run.smithi070.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-06-17T12:45:35.360 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up indent (2.2.12-1) ... 2022-06-17T12:45:35.518 INFO:teuthology.orchestra.run.smithi036.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2022-06-17T12:45:35.627 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2022-06-17T12:45:35.645 INFO:teuthology.orchestra.run.smithi036.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-06-17T12:45:35.683 INFO:teuthology.orchestra.run.smithi070.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-06-17T12:45:35.797 INFO:teuthology.orchestra.run.smithi036.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-06-17T12:45:36.332 INFO:teuthology.orchestra.run.smithi136.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-06-17T12:45:36.338 DEBUG:teuthology.parallel:result is None 2022-06-17T12:45:37.682 INFO:teuthology.orchestra.run.smithi070.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2022-06-17T12:45:37.773 INFO:teuthology.orchestra.run.smithi036.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2022-06-17T12:45:38.416 INFO:teuthology.orchestra.run.smithi070.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-06-17T12:45:38.418 DEBUG:teuthology.parallel:result is None 2022-06-17T12:45:38.503 INFO:teuthology.orchestra.run.smithi036.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-06-17T12:45:38.504 DEBUG:teuthology.parallel:result is None 2022-06-17T12:45:38.505 DEBUG:teuthology.orchestra.run.smithi036:> sudo lsb_release -is 2022-06-17T12:45:38.567 INFO:teuthology.orchestra.run.smithi036.stdout:Ubuntu 2022-06-17T12:45:38.579 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-17T12:45:38.584 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6 2022-06-17T12:45:38.815 DEBUG:teuthology.orchestra.run.smithi036:> dpkg-query -W -f '${Version}' ceph 2022-06-17T12:45:38.858 INFO:teuthology.orchestra.run.smithi036.stdout:15.2.16-260-gb3641a55-1focal 2022-06-17T12:45:38.860 INFO:teuthology.packaging:The installed version of ceph is 15.2.16-260-gb3641a55-1focal 2022-06-17T12:45:38.861 INFO:teuthology.task.install:The correct ceph version 15.2.16-260-gb3641a55-1focal is installed. 2022-06-17T12:45:38.862 DEBUG:teuthology.orchestra.run.smithi070:> sudo lsb_release -is 2022-06-17T12:45:38.940 INFO:teuthology.orchestra.run.smithi070.stdout:Ubuntu 2022-06-17T12:45:38.950 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-17T12:45:38.961 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6 2022-06-17T12:45:39.158 DEBUG:teuthology.orchestra.run.smithi070:> dpkg-query -W -f '${Version}' ceph 2022-06-17T12:45:39.195 INFO:teuthology.orchestra.run.smithi070.stdout:15.2.16-260-gb3641a55-1focal 2022-06-17T12:45:39.195 INFO:teuthology.packaging:The installed version of ceph is 15.2.16-260-gb3641a55-1focal 2022-06-17T12:45:39.196 INFO:teuthology.task.install:The correct ceph version 15.2.16-260-gb3641a55-1focal is installed. 2022-06-17T12:45:39.197 DEBUG:teuthology.orchestra.run.smithi107:> sudo lsb_release -is 2022-06-17T12:45:39.254 INFO:teuthology.orchestra.run.smithi107.stdout:Ubuntu 2022-06-17T12:45:39.259 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-17T12:45:39.260 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6 2022-06-17T12:45:39.526 DEBUG:teuthology.orchestra.run.smithi107:> dpkg-query -W -f '${Version}' ceph 2022-06-17T12:45:39.566 INFO:teuthology.orchestra.run.smithi107.stdout:15.2.16-260-gb3641a55-1focal 2022-06-17T12:45:39.567 INFO:teuthology.packaging:The installed version of ceph is 15.2.16-260-gb3641a55-1focal 2022-06-17T12:45:39.567 INFO:teuthology.task.install:The correct ceph version 15.2.16-260-gb3641a55-1focal is installed. 2022-06-17T12:45:39.568 DEBUG:teuthology.orchestra.run.smithi136:> sudo lsb_release -is 2022-06-17T12:45:39.635 INFO:teuthology.orchestra.run.smithi136.stdout:Ubuntu 2022-06-17T12:45:39.640 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-17T12:45:39.641 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6 2022-06-17T12:45:39.852 DEBUG:teuthology.orchestra.run.smithi136:> dpkg-query -W -f '${Version}' ceph 2022-06-17T12:45:39.891 INFO:teuthology.orchestra.run.smithi136.stdout:15.2.16-260-gb3641a55-1focal 2022-06-17T12:45:39.892 INFO:teuthology.packaging:The installed version of ceph is 15.2.16-260-gb3641a55-1focal 2022-06-17T12:45:39.893 INFO:teuthology.task.install:The correct ceph version 15.2.16-260-gb3641a55-1focal is installed. 2022-06-17T12:45:39.894 DEBUG:teuthology.orchestra.run.smithi143:> sudo lsb_release -is 2022-06-17T12:45:39.948 INFO:teuthology.orchestra.run.smithi143.stdout:Ubuntu 2022-06-17T12:45:39.974 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-17T12:45:39.975 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6 2022-06-17T12:45:40.249 DEBUG:teuthology.orchestra.run.smithi143:> dpkg-query -W -f '${Version}' ceph 2022-06-17T12:45:40.287 INFO:teuthology.orchestra.run.smithi143.stdout:15.2.16-260-gb3641a55-1focal 2022-06-17T12:45:40.288 INFO:teuthology.packaging:The installed version of ceph is 15.2.16-260-gb3641a55-1focal 2022-06-17T12:45:40.288 INFO:teuthology.task.install:The correct ceph version 15.2.16-260-gb3641a55-1focal is installed. 2022-06-17T12:45:40.306 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2022-06-17T12:45:40.307 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-06-17T12:45:40.308 DEBUG:teuthology.orchestra.run.smithi036:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-06-17T12:45:40.347 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2022-06-17T12:45:40.351 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-06-17T12:45:40.376 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2022-06-17T12:45:40.379 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-06-17T12:45:40.415 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2022-06-17T12:45:40.416 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-06-17T12:45:40.444 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2022-06-17T12:45:40.445 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-06-17T12:45:40.483 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2022-06-17T12:45:40.497 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-06-17T12:45:40.497 DEBUG:teuthology.orchestra.run.smithi036:> sudo dd of=/usr/bin/daemon-helper 2022-06-17T12:45:40.559 DEBUG:teuthology.orchestra.run.smithi036:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-06-17T12:45:40.622 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2022-06-17T12:45:40.622 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/daemon-helper 2022-06-17T12:45:40.642 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-06-17T12:45:40.696 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2022-06-17T12:45:40.697 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2022-06-17T12:45:40.763 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-06-17T12:45:40.872 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2022-06-17T12:45:40.873 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/daemon-helper 2022-06-17T12:45:40.923 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-06-17T12:45:40.997 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2022-06-17T12:45:41.008 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/daemon-helper 2022-06-17T12:45:41.041 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-06-17T12:45:41.108 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2022-06-17T12:45:41.109 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-06-17T12:45:41.117 DEBUG:teuthology.orchestra.run.smithi036:> sudo dd of=/usr/bin/adjust-ulimits 2022-06-17T12:45:41.176 DEBUG:teuthology.orchestra.run.smithi036:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-06-17T12:45:41.219 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2022-06-17T12:45:41.222 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/adjust-ulimits 2022-06-17T12:45:41.358 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-06-17T12:45:41.431 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2022-06-17T12:45:41.431 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2022-06-17T12:45:41.508 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-06-17T12:45:41.578 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2022-06-17T12:45:41.579 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/adjust-ulimits 2022-06-17T12:45:41.611 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-06-17T12:45:41.637 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2022-06-17T12:45:41.638 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/adjust-ulimits 2022-06-17T12:45:41.707 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-06-17T12:45:41.723 INFO:teuthology.run_tasks:Running task ceph... 2022-06-17T12:45:41.914 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2022-06-17T12:45:41.916 DEBUG:teuthology.orchestra.run.smithi036:> sudo chmod 777 /var/log/ceph 2022-06-17T12:45:41.920 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 777 /var/log/ceph 2022-06-17T12:45:41.927 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /var/log/ceph 2022-06-17T12:45:41.971 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 777 /var/log/ceph 2022-06-17T12:45:41.987 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 777 /var/log/ceph 2022-06-17T12:45:42.035 INFO:tasks.ceph:Disabling ceph logrotate... 2022-06-17T12:45:42.036 DEBUG:teuthology.orchestra.run.smithi036:> sudo rm -f -- /etc/logrotate.d/ceph 2022-06-17T12:45:42.043 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/logrotate.d/ceph 2022-06-17T12:45:42.072 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/logrotate.d/ceph 2022-06-17T12:45:42.077 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/logrotate.d/ceph 2022-06-17T12:45:42.081 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /etc/logrotate.d/ceph 2022-06-17T12:45:42.099 INFO:tasks.ceph:Creating extra log directories... 2022-06-17T12:45:42.100 DEBUG:teuthology.orchestra.run.smithi036:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-06-17T12:45:42.118 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-06-17T12:45:42.121 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-06-17T12:45:42.144 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-06-17T12:45:42.151 DEBUG:teuthology.orchestra.run.smithi143:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-06-17T12:45:42.172 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2022-06-17T12:45:42.173 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2022-06-17T12:45:42.174 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2022-06-17T12:45:42.174 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-06-17T12:45:42.175 DEBUG:teuthology.orchestra.run.smithi036:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-06-17T12:45:42.190 DEBUG:teuthology.orchestra.run.smithi036:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-06-17T12:45:42.271 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2022-06-17T12:45:42.272 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-06-17T12:45:42.289 DEBUG:teuthology.orchestra.run.smithi070:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-06-17T12:45:42.383 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2022-06-17T12:45:42.394 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-06-17T12:45:42.480 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-06-17T12:45:42.525 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2022-06-17T12:45:42.528 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-06-17T12:45:42.625 DEBUG:teuthology.orchestra.run.smithi136:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-06-17T12:45:42.744 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2022-06-17T12:45:42.744 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-06-17T12:45:42.791 DEBUG:teuthology.orchestra.run.smithi143:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-06-17T12:45:42.881 INFO:tasks.ceph:Creating ceph cluster ceph... 2022-06-17T12:45:42.881 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'POOL_APP_NOT_ENABLED', 'has not responded to cap revoke by MDS for over', 'MDS_CLIENT_LATE_RELEASE', 'responding to mclientcaps'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2022-06-17T12:45:42.883 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821', 'branch': 'octopus', 'description': 'fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests}', 'email': 'ceph-qa@ceph.io', 'first_in_suite': False, 'job_id': '6883821', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 30}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'octopus'}, 'ceph': {'cephfs_ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'POOL_APP_NOT_ENABLED', 'has not responded to cap revoke by MDS for over', 'MDS_CLIENT_LATE_RELEASE', 'responding to mclientcaps'], 'sha1': 'b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'octopus', 'sha1': 'b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 1000, 'repo': 'git://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'], ['client.0'], ['client.1'], ['client.2']], 'sha1': 'b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'octopus', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa', 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph.git', 'suite_sha1': 'b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6', 'targets': {'smithi036.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC57MPEtIquGAy9QJplKQzWpsUlNg8CKOuWx+xjj9hH7r3J/8ftanBScqsF8v/E0hm/ThvVNgk3vM5KyRDnJFa3Oq+Z6Eyri4LFUtZh2ty2TMBVkM1ZYd4a59iuMh1HvlVbw+f7J3c5esl3tukfF/4aDeqk566Rp1vgQCt5n7mjqgbHMV9aW37WvSHkeDSH6qhGiLcnyYTb8g1oNyNU5TzaonLKMDj5COEFZKgZUNqC+HbweQzZteAospxQx0fIX80VFKmGEOKbmZPUHK89tiAJzyeaGalRXpopyzcKBZ+8RtKchLS5pRCqT+cTEsfEjFstW9OJM0eo7MKge4Rhm7BYtPYJ7DaqETsuHOghiAZY30hVyG1Yl4hJCEZKigyHEDuNFN0IU5htrAgzZs1ufpQBx2/lpQLExQiUfFOUH4RF6W/TFis4uhzMwxJlFHhRSwxTtTHr+ARBV5X+YsJRm2VzSpFTbxhV6/IjyTocTyLeBFsqiAg7vni3S5sAAOliEGU=', 'smithi070.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCr9BmpKJlgwMIaIyD/z9fXBgHOkzXl5yVZppmk5TezmA1ybxEzwlWxo+qNajL8vh28omeB7pyVVER98fuU3mgqLQ0MzOxaMM7Ow4/nmQHDy/jBbIfo/nm6XqcMtY8dk486t3rmmjfbm+MKE2BOn45c7Zy9L/BzHW1ubTW3wl9KWm3VKqtYM3jzUgPXkXZiqGI1gzSmfs7vkWtUENHu2UHgNGC0AFDLVGiAgCpAKJ9/hO30/8m/9F1wlERPi7XCvTrVQ8QydKYEjOUEyy2zY2jkeZKKABbnEpTvV02YSSGy0g3g+zW+lm2pgTiN6GRsklsHJ6PTk8ZP11fp6cmInw5RVCIkEubp1cAIZS4teFnL6QlBvXYfl7azhbGqLT5BoT/80zrGpMGnSWMcIap3Kbl291HDCWTehs0JiaZ+jlVXTMRjKPL5DdA09M+zGCOj6ffDO+YmeFnmBpgalswEneMaL5pratARdtxifeJDMbV4FTjMSwJrnNHA/7criBd1Mvk=', 'smithi107.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCWY+bhZkRud/tllPmLGstGhb34LtSQQ1kHy82yEkc20qdlIW1ret48XHHIke0JtuKjqSu9vkZF6PwyvJhvd1T8N/wV82vPjrUwV3vmcmErSQzpnOhbmGWiFlPyhf1LAMwCfYTbeUgpOrSI/44t5bBAqoo5BzjR8rh+r1QbnZ+7DJ6Wd1fwmQGke3tjHKbD9+G41SB9bAK7oSzpibvr5cKKie49Ap1FQxBdBuDhv5aUVPDNttT+inD6ZU9e6fQD3B5RPe6FgSUUGSc7vYqqYZfDSOQb5fu+lHhSWnPZ9jc4XKXZHnsgJXRGKRZG2cRPt1HWI7RIndhRuIvtRPjw8mluWlv6qdOXWC49Xt79LUOeyYRBctMolbNUvRBV7kCfhdW7LVAAGPuhS5adYC1pT4QRLdiSb4VBI+nAl/5hVpBAJvEcldSG1FRSNWQsErUjpqUm7f0Z4Z9oWXAF+cjkLAlrryQTveST37axt8NAmIFftQzVrhCWOU/CNuU737WTtFM=', 'smithi136.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCdl7wPIRbjjnibXe7F9vDsAsyQuGVjBhySCFB6BPeliMm5v5XT+W/J7jOh58MslaePsEVYQBJ6sNGQxlkt6X/lDyv+dz7t/jp+IUnE38urxjhjBrE25AIW+wcUJmwLlPD0SGrEP5wZFTuDgRgcNvtZY/Ykl0x24M3TBDjfXtX4bMmuqLg5rEkcudliDL+zITfc7Xd1v1iS6IDJE6ZDZyyUmTesAMbT5auJfzHU3p9KcsNMPhfemLgQ01pUzPOq80sDOqBP1qfMyNkJ2mFGdCbtZGhMzeU7+fLLynA2Z5Lwiv9wZavnvhmyNehfno9NcMqlTO/5gRCXn0nK4e4BtXQGOXhzSzssQb+g++bGbVp6IUuJokAIEMIMmoPFCS/ruBWtfnYH1VayVc2K+J5uzatWVUOBDKhDGR8sFbIXjEstX/raG+h+ByPOxR/8CKYT3Q9duMNz5snnfAkIqCmUqXFA+Uh9c+gnrXvMPNF3d1FB7Km8iLEYxCisud+M17A0JbE=', 'smithi143.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCSQ9FUvkLUVb56bxWPRq0gzjNMepOXMWM7aOuObcI6zVGFGsw/KDlFx6laxwYbTqmYSjE+QN8C1M8uKi1kN4orrjTS7uaGpMqa+vH0GI77OaGFlgzMrDB4WjStiqYP/UBYv1zA5lokyp+fvfwmYIfu/HZPfF+/GyAxJFUrb9cXpxZ071u5Zs/769G1RU/mss17w6FtQIEr1ehDcS1aLY6mQxO/kRbOLnmvfCdHm4LFJLnI+WEQPEWwoJAS9Qi2P3PMwsokOvXrRMqW1iW9hwo91VRU+M9cONYFKM3iDl6n7ge0fJRjopRdvmXAWGZjsi5SlFdyfMD3S95RluC5JT8dk7ENRzh6v2+6cldf0V10LOwvUpPCbCawgfLgfA5sVOMj5hwImJ80F2VmL800zbLwfz4sV4iWbPLdTDrYRbq1fwZzLN1AJbPtajGnPXNJKzPk8ZfKfNKLLQsh9FO+BBlNQrgsuBepK2s9hiQRlvc2nmx2jd2ZO3JD6kPDCLytqoc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': 'b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_misc']}}], 'teuthology_branch': 'main', 'teuthology_sha1': '31a356e0c737f1781079045bb8dc366982be8713', 'timestamp': '2022-06-17_04:10:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.1713'} 2022-06-17T12:45:42.885 DEBUG:teuthology.orchestra.run.smithi036:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-06-17T12:45:42.898 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-06-17T12:45:42.911 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-06-17T12:45:42.965 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-06-17T12:45:43.030 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-06-17T12:45:43.060 DEBUG:teuthology.orchestra.run.smithi036:> sudo install -d -m0777 -- /var/run/ceph 2022-06-17T12:45:43.105 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m0777 -- /var/run/ceph 2022-06-17T12:45:43.111 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m0777 -- /var/run/ceph 2022-06-17T12:45:43.134 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m0777 -- /var/run/ceph 2022-06-17T12:45:43.161 DEBUG:teuthology.orchestra.run.smithi143:> sudo install -d -m0777 -- /var/run/ceph 2022-06-17T12:45:43.214 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-06-17T12:45:43.215 DEBUG:teuthology.orchestra.run.smithi036:> dd if=/scratch_devs of=/dev/stdout 2022-06-17T12:45:43.340 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-06-17T12:45:43.340 DEBUG:teuthology.orchestra.run.smithi036:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2022-06-17T12:45:43.374 INFO:teuthology.orchestra.run.smithi036.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-06-17T12:45:43.375 INFO:teuthology.orchestra.run.smithi036.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-06-17T12:45:43.375 INFO:teuthology.orchestra.run.smithi036.stdout:Device: 6h/6d Inode: 362 Links: 1 2022-06-17T12:45:43.376 INFO:teuthology.orchestra.run.smithi036.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-06-17T12:45:43.394 INFO:teuthology.orchestra.run.smithi036.stdout:Access: 2022-06-17 12:41:49.320698302 +0000 2022-06-17T12:45:43.394 INFO:teuthology.orchestra.run.smithi036.stdout:Modify: 2022-06-17 12:41:48.512698276 +0000 2022-06-17T12:45:43.395 INFO:teuthology.orchestra.run.smithi036.stdout:Change: 2022-06-17 12:41:48.512698276 +0000 2022-06-17T12:45:43.395 INFO:teuthology.orchestra.run.smithi036.stdout: Birth: - 2022-06-17T12:45:43.445 INFO:teuthology.orchestra.run.smithi036.stderr:1+0 records in 2022-06-17T12:45:43.455 INFO:teuthology.orchestra.run.smithi036.stderr:1+0 records out 2022-06-17T12:45:43.456 INFO:teuthology.orchestra.run.smithi036.stderr:512 bytes copied, 0.000419258 s, 1.2 MB/s 2022-06-17T12:45:43.457 DEBUG:teuthology.orchestra.run.smithi036:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2022-06-17T12:45:43.532 INFO:teuthology.orchestra.run.smithi036.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-06-17T12:45:43.533 INFO:teuthology.orchestra.run.smithi036.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-06-17T12:45:43.533 INFO:teuthology.orchestra.run.smithi036.stdout:Device: 6h/6d Inode: 368 Links: 1 2022-06-17T12:45:43.534 INFO:teuthology.orchestra.run.smithi036.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-06-17T12:45:43.534 INFO:teuthology.orchestra.run.smithi036.stdout:Access: 2022-06-17 12:41:50.404698337 +0000 2022-06-17T12:45:43.535 INFO:teuthology.orchestra.run.smithi036.stdout:Modify: 2022-06-17 12:41:49.548698310 +0000 2022-06-17T12:45:43.535 INFO:teuthology.orchestra.run.smithi036.stdout:Change: 2022-06-17 12:41:49.548698310 +0000 2022-06-17T12:45:43.536 INFO:teuthology.orchestra.run.smithi036.stdout: Birth: - 2022-06-17T12:45:43.540 INFO:teuthology.orchestra.run.smithi036.stderr:1+0 records in 2022-06-17T12:45:43.540 INFO:teuthology.orchestra.run.smithi036.stderr:1+0 records out 2022-06-17T12:45:43.540 INFO:teuthology.orchestra.run.smithi036.stderr:512 bytes copied, 0.000409203 s, 1.3 MB/s 2022-06-17T12:45:43.541 DEBUG:teuthology.orchestra.run.smithi036:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2022-06-17T12:45:43.608 INFO:teuthology.orchestra.run.smithi036.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-06-17T12:45:43.609 INFO:teuthology.orchestra.run.smithi036.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-06-17T12:45:43.609 INFO:teuthology.orchestra.run.smithi036.stdout:Device: 6h/6d Inode: 375 Links: 1 2022-06-17T12:45:43.609 INFO:teuthology.orchestra.run.smithi036.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-06-17T12:45:43.610 INFO:teuthology.orchestra.run.smithi036.stdout:Access: 2022-06-17 12:41:51.428698370 +0000 2022-06-17T12:45:43.610 INFO:teuthology.orchestra.run.smithi036.stdout:Modify: 2022-06-17 12:41:50.620698344 +0000 2022-06-17T12:45:43.610 INFO:teuthology.orchestra.run.smithi036.stdout:Change: 2022-06-17 12:41:50.620698344 +0000 2022-06-17T12:45:43.610 INFO:teuthology.orchestra.run.smithi036.stdout: Birth: - 2022-06-17T12:45:43.611 INFO:teuthology.orchestra.run.smithi036.stderr:1+0 records in 2022-06-17T12:45:43.611 INFO:teuthology.orchestra.run.smithi036.stderr:1+0 records out 2022-06-17T12:45:43.611 INFO:teuthology.orchestra.run.smithi036.stderr:512 bytes copied, 0.000417841 s, 1.2 MB/s 2022-06-17T12:45:43.613 DEBUG:teuthology.orchestra.run.smithi036:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2022-06-17T12:45:43.640 INFO:teuthology.orchestra.run.smithi036.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-06-17T12:45:43.641 INFO:teuthology.orchestra.run.smithi036.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-06-17T12:45:43.641 INFO:teuthology.orchestra.run.smithi036.stdout:Device: 6h/6d Inode: 381 Links: 1 2022-06-17T12:45:43.642 INFO:teuthology.orchestra.run.smithi036.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-06-17T12:45:43.642 INFO:teuthology.orchestra.run.smithi036.stdout:Access: 2022-06-17 12:41:51.644698377 +0000 2022-06-17T12:45:43.642 INFO:teuthology.orchestra.run.smithi036.stdout:Modify: 2022-06-17 12:41:51.640698377 +0000 2022-06-17T12:45:43.642 INFO:teuthology.orchestra.run.smithi036.stdout:Change: 2022-06-17 12:41:51.640698377 +0000 2022-06-17T12:45:43.643 INFO:teuthology.orchestra.run.smithi036.stdout: Birth: - 2022-06-17T12:45:43.649 INFO:teuthology.orchestra.run.smithi036.stderr:1+0 records in 2022-06-17T12:45:43.649 INFO:teuthology.orchestra.run.smithi036.stderr:1+0 records out 2022-06-17T12:45:43.650 INFO:teuthology.orchestra.run.smithi036.stderr:512 bytes copied, 0.000412501 s, 1.2 MB/s 2022-06-17T12:45:43.653 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-06-17T12:45:43.654 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2022-06-17T12:45:43.654 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/scratch_devs of=/dev/stdout 2022-06-17T12:45:43.669 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-06-17T12:45:43.669 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2022-06-17T12:45:43.718 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-06-17T12:45:43.718 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-06-17T12:45:43.718 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 6h/6d Inode: 364 Links: 1 2022-06-17T12:45:43.720 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-06-17T12:45:43.720 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2022-06-17 12:41:55.206595618 +0000 2022-06-17T12:45:43.721 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2022-06-17 12:41:54.338595600 +0000 2022-06-17T12:45:43.721 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2022-06-17 12:41:54.338595600 +0000 2022-06-17T12:45:43.722 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: - 2022-06-17T12:45:43.772 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2022-06-17T12:45:43.773 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2022-06-17T12:45:43.773 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000299041 s, 1.7 MB/s 2022-06-17T12:45:43.778 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2022-06-17T12:45:43.809 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-06-17T12:45:43.810 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-06-17T12:45:43.810 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 6h/6d Inode: 369 Links: 1 2022-06-17T12:45:43.811 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-06-17T12:45:43.811 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2022-06-17 12:41:56.090595637 +0000 2022-06-17T12:45:43.813 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2022-06-17 12:41:55.430595623 +0000 2022-06-17T12:45:43.814 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2022-06-17 12:41:55.430595623 +0000 2022-06-17T12:45:43.814 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: - 2022-06-17T12:45:43.886 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2022-06-17T12:45:43.888 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2022-06-17T12:45:43.888 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000303084 s, 1.7 MB/s 2022-06-17T12:45:43.888 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2022-06-17T12:45:43.911 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-06-17T12:45:43.912 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-06-17T12:45:43.936 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 6h/6d Inode: 374 Links: 1 2022-06-17T12:45:43.937 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-06-17T12:45:43.937 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2022-06-17 12:41:57.258595661 +0000 2022-06-17T12:45:43.946 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2022-06-17 12:41:56.338595642 +0000 2022-06-17T12:45:43.949 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2022-06-17 12:41:56.338595642 +0000 2022-06-17T12:45:43.949 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: - 2022-06-17T12:45:43.968 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2022-06-17T12:45:43.969 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2022-06-17T12:45:43.970 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000292991 s, 1.7 MB/s 2022-06-17T12:45:43.985 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2022-06-17T12:45:44.029 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-06-17T12:45:44.030 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-06-17T12:45:44.030 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 6h/6d Inode: 383 Links: 1 2022-06-17T12:45:44.031 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-06-17T12:45:44.031 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2022-06-17 12:41:58.390595685 +0000 2022-06-17T12:45:44.032 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2022-06-17 12:41:57.478595666 +0000 2022-06-17T12:45:44.032 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2022-06-17 12:41:57.478595666 +0000 2022-06-17T12:45:44.032 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: - 2022-06-17T12:45:44.035 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2022-06-17T12:45:44.040 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2022-06-17T12:45:44.041 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000295634 s, 1.7 MB/s 2022-06-17T12:45:44.041 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-06-17T12:45:44.042 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi036.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi070.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-06-17T12:45:44.043 INFO:tasks.ceph:Generating config... 2022-06-17T12:45:44.046 INFO:tasks.ceph:[client] client mount timeout = 600 2022-06-17T12:45:44.065 INFO:tasks.ceph:[client] debug client = 20 2022-06-17T12:45:44.065 INFO:tasks.ceph:[client] debug ms = 1 2022-06-17T12:45:44.066 INFO:tasks.ceph:[client] rados mon op timeout = 15m 2022-06-17T12:45:44.066 INFO:tasks.ceph:[client] rados osd op timeout = 15m 2022-06-17T12:45:44.067 INFO:tasks.ceph:[mds] debug mds = 20 2022-06-17T12:45:44.067 INFO:tasks.ceph:[mds] debug ms = 1 2022-06-17T12:45:44.068 INFO:tasks.ceph:[mds] mds bal frag = True 2022-06-17T12:45:44.069 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2022-06-17T12:45:44.070 INFO:tasks.ceph:[mds] mds bal merge size = 5 2022-06-17T12:45:44.071 INFO:tasks.ceph:[mds] mds bal split bits = 3 2022-06-17T12:45:44.074 INFO:tasks.ceph:[mds] mds bal split size = 100 2022-06-17T12:45:44.075 INFO:tasks.ceph:[mds] mds debug frag = True 2022-06-17T12:45:44.076 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2022-06-17T12:45:44.076 INFO:tasks.ceph:[mds] mds op complaint time = 180 2022-06-17T12:45:44.076 INFO:tasks.ceph:[mds] mds verify scatter = True 2022-06-17T12:45:44.077 INFO:tasks.ceph:[mds] osd op complaint time = 180 2022-06-17T12:45:44.077 INFO:tasks.ceph:[mds] rados mon op timeout = 15m 2022-06-17T12:45:44.078 INFO:tasks.ceph:[mds] rados osd op timeout = 15m 2022-06-17T12:45:44.078 INFO:tasks.ceph:[mgr] debug mgr = 20 2022-06-17T12:45:44.079 INFO:tasks.ceph:[mgr] debug ms = 1 2022-06-17T12:45:44.080 INFO:tasks.ceph:[mon] debug mon = 20 2022-06-17T12:45:44.080 INFO:tasks.ceph:[mon] debug ms = 1 2022-06-17T12:45:44.081 INFO:tasks.ceph:[mon] debug paxos = 20 2022-06-17T12:45:44.081 INFO:tasks.ceph:[mon] mon op complaint time = 120 2022-06-17T12:45:44.081 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2022-06-17T12:45:44.086 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2022-06-17T12:45:44.087 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2022-06-17T12:45:44.087 INFO:tasks.ceph:[osd] debug bluefs = 20 2022-06-17T12:45:44.088 INFO:tasks.ceph:[osd] debug bluestore = 20 2022-06-17T12:45:44.088 INFO:tasks.ceph:[osd] debug ms = 1 2022-06-17T12:45:44.089 INFO:tasks.ceph:[osd] debug osd = 20 2022-06-17T12:45:44.089 INFO:tasks.ceph:[osd] debug rocksdb = 10 2022-06-17T12:45:44.090 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2022-06-17T12:45:44.090 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2022-06-17T12:45:44.091 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2022-06-17T12:45:44.092 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2022-06-17T12:45:44.092 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2022-06-17T12:45:44.097 INFO:tasks.ceph:[osd] osd op complaint time = 180 2022-06-17T12:45:44.098 INFO:tasks.ceph:Setting up mon.a... 2022-06-17T12:45:44.098 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2022-06-17T12:45:44.202 INFO:teuthology.orchestra.run.smithi036.stdout:creating /etc/ceph/ceph.keyring 2022-06-17T12:45:44.206 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2022-06-17T12:45:44.257 DEBUG:teuthology.orchestra.run.smithi036:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-06-17T12:45:44.361 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.36'), ('mon.b', '172.21.15.70'), ('mon.c', '[v2:172.21.15.70:3301,v1:172.21.15.70:6790]')] 2022-06-17T12:45:44.401 DEBUG:teuthology.orchestra.run.smithi036:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.36 --add b 172.21.15.70 --addv c '[v2:172.21.15.70:3301,v1:172.21.15.70:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2022-06-17T12:45:44.456 INFO:teuthology.orchestra.run.smithi036.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2022-06-17T12:45:44.457 INFO:teuthology.orchestra.run.smithi036.stdout:monmaptool: generated fsid b68b9700-0c85-42ca-94d7-6cd53db91c14 2022-06-17T12:45:44.460 INFO:teuthology.orchestra.run.smithi036.stdout:epoch 0 2022-06-17T12:45:44.461 INFO:teuthology.orchestra.run.smithi036.stdout:fsid b68b9700-0c85-42ca-94d7-6cd53db91c14 2022-06-17T12:45:44.461 INFO:teuthology.orchestra.run.smithi036.stdout:last_changed 2022-06-17T12:45:44.466620+0000 2022-06-17T12:45:44.476 INFO:teuthology.orchestra.run.smithi036.stdout:created 2022-06-17T12:45:44.466620+0000 2022-06-17T12:45:44.476 INFO:teuthology.orchestra.run.smithi036.stdout:min_mon_release 0 (unknown) 2022-06-17T12:45:44.477 INFO:teuthology.orchestra.run.smithi036.stdout:0: [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] mon.a 2022-06-17T12:45:44.477 INFO:teuthology.orchestra.run.smithi036.stdout:1: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.b 2022-06-17T12:45:44.505 INFO:teuthology.orchestra.run.smithi036.stdout:2: [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] mon.c 2022-06-17T12:45:44.505 INFO:teuthology.orchestra.run.smithi036.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2022-06-17T12:45:44.508 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b68b9700-0c85-42ca-94d7-6cd53db91c14... 2022-06-17T12:45:44.527 DEBUG:teuthology.orchestra.run.smithi036:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-17T12:45:44.556 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-17T12:45:44.562 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-17T12:45:44.585 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-17T12:45:44.590 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-17T12:45:44.663 INFO:teuthology.orchestra.run.smithi070.stdout:[global] 2022-06-17T12:45:44.680 INFO:teuthology.orchestra.run.smithi070.stdout: chdir = "" 2022-06-17T12:45:44.680 INFO:teuthology.orchestra.run.smithi070.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-17T12:45:44.681 INFO:teuthology.orchestra.run.smithi070.stdout: auth supported = cephx 2022-06-17T12:45:44.681 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.682 INFO:teuthology.orchestra.run.smithi070.stdout: filestore xattr use omap = true 2022-06-17T12:45:44.683 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.683 INFO:teuthology.orchestra.run.smithi070.stdout: mon clock drift allowed = 1.000 2022-06-17T12:45:44.683 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.684 INFO:teuthology.orchestra.run.smithi070.stdout: osd crush chooseleaf type = 0 2022-06-17T12:45:44.684 INFO:teuthology.orchestra.run.smithi070.stdout: auth debug = true 2022-06-17T12:45:44.692 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.692 INFO:teuthology.orchestra.run.smithi070.stdout: ms die on old message = true 2022-06-17T12:45:44.693 INFO:teuthology.orchestra.run.smithi070.stdout: ms die on bug = true 2022-06-17T12:45:44.734 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.735 INFO:teuthology.orchestra.run.smithi070.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-17T12:45:44.735 INFO:teuthology.orchestra.run.smithi070.stdout: mon pg warn max object skew = 0 2022-06-17T12:45:44.736 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.736 INFO:teuthology.orchestra.run.smithi070.stdout: # disable pg_autoscaler by default for new pools 2022-06-17T12:45:44.737 INFO:teuthology.orchestra.run.smithi070.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-17T12:45:44.737 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.757 INFO:teuthology.orchestra.run.smithi070.stdout: osd pool default size = 2 2022-06-17T12:45:44.758 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.758 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd allow primary affinity = true 2022-06-17T12:45:44.758 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd allow pg remap = true 2022-06-17T12:45:44.759 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on legacy crush tunables = false 2022-06-17T12:45:44.759 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on crush straw calc version zero = false 2022-06-17T12:45:44.760 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on no sortbitwise = false 2022-06-17T12:45:44.760 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on osd down out interval zero = false 2022-06-17T12:45:44.760 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on too few osds = false 2022-06-17T12:45:44.761 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-17T12:45:44.762 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-17T12:45:44.762 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.763 INFO:teuthology.orchestra.run.smithi070.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-17T12:45:44.763 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.764 INFO:teuthology.orchestra.run.smithi070.stdout: osd default data pool replay window = 5 2022-06-17T12:45:44.764 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.764 INFO:teuthology.orchestra.run.smithi070.stdout: mon allow pool delete = true 2022-06-17T12:45:44.765 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.765 INFO:teuthology.orchestra.run.smithi070.stdout: mon cluster log file level = debug 2022-06-17T12:45:44.766 INFO:teuthology.orchestra.run.smithi070.stdout: debug asserts on shutdown = true 2022-06-17T12:45:44.766 INFO:teuthology.orchestra.run.smithi070.stdout: mon health detail to clog = false 2022-06-17T12:45:44.767 INFO:teuthology.orchestra.run.smithi070.stdout: mon host = "172.21.15.36,172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790]" 2022-06-17T12:45:44.767 INFO:teuthology.orchestra.run.smithi070.stdout: fsid = b68b9700-0c85-42ca-94d7-6cd53db91c14 2022-06-17T12:45:44.768 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.768 INFO:teuthology.orchestra.run.smithi070.stdout:[osd] 2022-06-17T12:45:44.768 INFO:teuthology.orchestra.run.smithi070.stdout: osd journal size = 100 2022-06-17T12:45:44.769 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.769 INFO:teuthology.orchestra.run.smithi070.stdout: osd scrub load threshold = 5.0 2022-06-17T12:45:44.770 INFO:teuthology.orchestra.run.smithi070.stdout: osd scrub max interval = 600 2022-06-17T12:45:44.770 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.771 INFO:teuthology.orchestra.run.smithi070.stdout: osd recover clone overlap = true 2022-06-17T12:45:44.771 INFO:teuthology.orchestra.run.smithi070.stdout: osd recovery max chunk = 1048576 2022-06-17T12:45:44.772 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.772 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug shutdown = true 2022-06-17T12:45:44.773 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug op order = true 2022-06-17T12:45:44.773 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug verify stray on activate = true 2022-06-17T12:45:44.773 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.783 INFO:teuthology.orchestra.run.smithi070.stdout: osd open classes on start = true 2022-06-17T12:45:44.784 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug pg log writeout = true 2022-06-17T12:45:44.784 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.785 INFO:teuthology.orchestra.run.smithi070.stdout: osd deep scrub update digest min age = 30 2022-06-17T12:45:44.785 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.785 INFO:teuthology.orchestra.run.smithi070.stdout: osd map max advance = 10 2022-06-17T12:45:44.786 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.786 INFO:teuthology.orchestra.run.smithi070.stdout: journal zero on create = true 2022-06-17T12:45:44.787 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.787 INFO:teuthology.orchestra.run.smithi070.stdout: filestore ondisk finisher threads = 3 2022-06-17T12:45:44.787 INFO:teuthology.orchestra.run.smithi070.stdout: filestore apply finisher threads = 3 2022-06-17T12:45:44.788 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.788 INFO:teuthology.orchestra.run.smithi070.stdout: bdev debug aio = true 2022-06-17T12:45:44.788 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug misdirected ops = true 2022-06-17T12:45:44.789 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore block size = 96636764160 2022-06-17T12:45:44.789 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore compression mode = aggressive 2022-06-17T12:45:44.790 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore fsck on mount = True 2022-06-17T12:45:44.791 INFO:teuthology.orchestra.run.smithi070.stdout: debug bluefs = 20 2022-06-17T12:45:44.791 INFO:teuthology.orchestra.run.smithi070.stdout: debug bluestore = 20 2022-06-17T12:45:44.792 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2022-06-17T12:45:44.793 INFO:teuthology.orchestra.run.smithi070.stdout: debug osd = 20 2022-06-17T12:45:44.794 INFO:teuthology.orchestra.run.smithi070.stdout: debug rocksdb = 10 2022-06-17T12:45:44.794 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd backfillfull_ratio = 0.85 2022-06-17T12:45:44.794 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd full ratio = 0.9 2022-06-17T12:45:44.795 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd nearfull ratio = 0.8 2022-06-17T12:45:44.795 INFO:teuthology.orchestra.run.smithi070.stdout: osd failsafe full ratio = 0.95 2022-06-17T12:45:44.796 INFO:teuthology.orchestra.run.smithi070.stdout: osd objectstore = bluestore 2022-06-17T12:45:44.796 INFO:teuthology.orchestra.run.smithi070.stdout: osd op complaint time = 180 2022-06-17T12:45:44.797 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.797 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr] 2022-06-17T12:45:44.798 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2022-06-17T12:45:44.798 INFO:teuthology.orchestra.run.smithi070.stdout: debug mgr = 20 2022-06-17T12:45:44.799 INFO:teuthology.orchestra.run.smithi070.stdout: debug mon = 20 2022-06-17T12:45:44.799 INFO:teuthology.orchestra.run.smithi070.stdout: debug auth = 20 2022-06-17T12:45:44.799 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:45:44.800 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:45:44.801 INFO:teuthology.orchestra.run.smithi070.stdout: mgr/telemetry/nag = false 2022-06-17T12:45:44.801 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.801 INFO:teuthology.orchestra.run.smithi070.stdout:[mon] 2022-06-17T12:45:44.802 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2022-06-17T12:45:44.802 INFO:teuthology.orchestra.run.smithi070.stdout: debug mon = 20 2022-06-17T12:45:44.803 INFO:teuthology.orchestra.run.smithi070.stdout: debug paxos = 20 2022-06-17T12:45:44.803 INFO:teuthology.orchestra.run.smithi070.stdout: debug auth = 20 2022-06-17T12:45:44.803 INFO:teuthology.orchestra.run.smithi070.stdout: mon data avail warn = 5 2022-06-17T12:45:44.804 INFO:teuthology.orchestra.run.smithi070.stdout: mon mgr mkfs grace = 240 2022-06-17T12:45:44.804 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:45:44.805 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd reporter subtree level = osd 2022-06-17T12:45:44.805 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd prime pg temp = true 2022-06-17T12:45:44.805 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:45:44.806 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.813 INFO:teuthology.orchestra.run.smithi070.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-17T12:45:44.813 INFO:teuthology.orchestra.run.smithi070.stdout: auth mon ticket ttl = 660 # 11m 2022-06-17T12:45:44.813 INFO:teuthology.orchestra.run.smithi070.stdout: auth service ticket ttl = 240 # 4m 2022-06-17T12:45:44.814 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.814 INFO:teuthology.orchestra.run.smithi070.stdout: # don't complain about insecure global_id in the test suite 2022-06-17T12:45:44.815 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-17T12:45:44.815 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-17T12:45:44.816 INFO:teuthology.orchestra.run.smithi070.stdout: mon op complaint time = 120 2022-06-17T12:45:44.816 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:45:44.817 INFO:teuthology.orchestra.run.smithi070.stdout:[client] 2022-06-17T12:45:44.817 INFO:teuthology.orchestra.run.smithi070.stdout: rgw cache enabled = true 2022-06-17T12:45:44.817 INFO:teuthology.orchestra.run.smithi070.stdout: rgw enable ops log = true 2022-06-17T12:45:44.825 INFO:teuthology.orchestra.run.smithi070.stdout: rgw enable usage log = true 2022-06-17T12:45:44.826 INFO:teuthology.orchestra.run.smithi070.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-17T12:45:44.828 INFO:teuthology.orchestra.run.smithi070.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-17T12:45:44.829 INFO:teuthology.orchestra.run.smithi070.stdout: client mount timeout = 600 2022-06-17T12:45:44.829 INFO:teuthology.orchestra.run.smithi070.stdout: debug client = 20 2022-06-17T12:45:44.829 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2022-06-17T12:45:44.830 INFO:teuthology.orchestra.run.smithi070.stdout: rados mon op timeout = 15m 2022-06-17T12:45:44.830 INFO:teuthology.orchestra.run.smithi070.stdout: rados osd op timeout = 15m 2022-06-17T12:45:44.831 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.a] 2022-06-17T12:45:44.831 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.b] 2022-06-17T12:45:44.831 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.c] 2022-06-17T12:45:44.832 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.a] 2022-06-17T12:45:44.834 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.b] 2022-06-17T12:45:44.835 INFO:teuthology.orchestra.run.smithi070.stdout:[mds] 2022-06-17T12:45:44.835 INFO:teuthology.orchestra.run.smithi070.stdout: debug mds = 20 2022-06-17T12:45:44.835 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2022-06-17T12:45:44.836 INFO:teuthology.orchestra.run.smithi070.stdout: mds bal frag = True 2022-06-17T12:45:44.840 INFO:teuthology.orchestra.run.smithi070.stdout: mds bal fragment size max = 10000 2022-06-17T12:45:44.842 INFO:teuthology.orchestra.run.smithi070.stdout: mds bal merge size = 5 2022-06-17T12:45:44.843 INFO:teuthology.orchestra.run.smithi070.stdout: mds bal split bits = 3 2022-06-17T12:45:44.844 INFO:teuthology.orchestra.run.smithi070.stdout: mds bal split size = 100 2022-06-17T12:45:44.844 INFO:teuthology.orchestra.run.smithi070.stdout: mds debug frag = True 2022-06-17T12:45:44.845 INFO:teuthology.orchestra.run.smithi070.stdout: mds debug scatterstat = True 2022-06-17T12:45:44.845 INFO:teuthology.orchestra.run.smithi070.stdout: mds op complaint time = 180 2022-06-17T12:45:44.858 INFO:teuthology.orchestra.run.smithi070.stdout: mds verify scatter = True 2022-06-17T12:45:44.859 INFO:teuthology.orchestra.run.smithi070.stdout: osd op complaint time = 180 2022-06-17T12:45:44.859 INFO:teuthology.orchestra.run.smithi070.stdout: rados mon op timeout = 15m 2022-06-17T12:45:44.859 INFO:teuthology.orchestra.run.smithi070.stdout: rados osd op timeout = 15m 2022-06-17T12:45:44.859 INFO:teuthology.orchestra.run.smithi036.stdout:[global] 2022-06-17T12:45:44.860 INFO:teuthology.orchestra.run.smithi036.stdout: chdir = "" 2022-06-17T12:45:44.860 INFO:teuthology.orchestra.run.smithi036.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-17T12:45:44.860 INFO:teuthology.orchestra.run.smithi036.stdout: auth supported = cephx 2022-06-17T12:45:44.860 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.860 INFO:teuthology.orchestra.run.smithi036.stdout: filestore xattr use omap = true 2022-06-17T12:45:44.861 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.861 INFO:teuthology.orchestra.run.smithi036.stdout: mon clock drift allowed = 1.000 2022-06-17T12:45:44.861 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.861 INFO:teuthology.orchestra.run.smithi036.stdout: osd crush chooseleaf type = 0 2022-06-17T12:45:44.887 INFO:teuthology.orchestra.run.smithi036.stdout: auth debug = true 2022-06-17T12:45:44.888 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.888 INFO:teuthology.orchestra.run.smithi036.stdout: ms die on old message = true 2022-06-17T12:45:44.888 INFO:teuthology.orchestra.run.smithi036.stdout: ms die on bug = true 2022-06-17T12:45:44.888 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.889 INFO:teuthology.orchestra.run.smithi036.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-17T12:45:44.889 INFO:teuthology.orchestra.run.smithi036.stdout: mon pg warn max object skew = 0 2022-06-17T12:45:44.889 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.889 INFO:teuthology.orchestra.run.smithi036.stdout: # disable pg_autoscaler by default for new pools 2022-06-17T12:45:44.889 INFO:teuthology.orchestra.run.smithi036.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-17T12:45:44.890 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.890 INFO:teuthology.orchestra.run.smithi036.stdout: osd pool default size = 2 2022-06-17T12:45:44.890 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.890 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd allow primary affinity = true 2022-06-17T12:45:44.891 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd allow pg remap = true 2022-06-17T12:45:44.891 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on legacy crush tunables = false 2022-06-17T12:45:44.891 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on crush straw calc version zero = false 2022-06-17T12:45:44.891 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on no sortbitwise = false 2022-06-17T12:45:44.891 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on osd down out interval zero = false 2022-06-17T12:45:44.892 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on too few osds = false 2022-06-17T12:45:44.892 INFO:teuthology.orchestra.run.smithi036.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-17T12:45:44.892 INFO:teuthology.orchestra.run.smithi036.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-17T12:45:44.892 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.893 INFO:teuthology.orchestra.run.smithi036.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-17T12:45:44.893 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.893 INFO:teuthology.orchestra.run.smithi036.stdout: osd default data pool replay window = 5 2022-06-17T12:45:44.893 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.893 INFO:teuthology.orchestra.run.smithi036.stdout: mon allow pool delete = true 2022-06-17T12:45:44.894 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.894 INFO:teuthology.orchestra.run.smithi036.stdout: mon cluster log file level = debug 2022-06-17T12:45:44.894 INFO:teuthology.orchestra.run.smithi036.stdout: debug asserts on shutdown = true 2022-06-17T12:45:44.894 INFO:teuthology.orchestra.run.smithi036.stdout: mon health detail to clog = false 2022-06-17T12:45:44.895 INFO:teuthology.orchestra.run.smithi036.stdout: mon host = "172.21.15.36,172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790]" 2022-06-17T12:45:44.895 INFO:teuthology.orchestra.run.smithi036.stdout: fsid = b68b9700-0c85-42ca-94d7-6cd53db91c14 2022-06-17T12:45:44.895 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.896 INFO:teuthology.orchestra.run.smithi036.stdout:[osd] 2022-06-17T12:45:44.896 INFO:teuthology.orchestra.run.smithi036.stdout: osd journal size = 100 2022-06-17T12:45:44.896 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.896 INFO:teuthology.orchestra.run.smithi036.stdout: osd scrub load threshold = 5.0 2022-06-17T12:45:44.897 INFO:teuthology.orchestra.run.smithi036.stdout: osd scrub max interval = 600 2022-06-17T12:45:44.897 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.897 INFO:teuthology.orchestra.run.smithi036.stdout: osd recover clone overlap = true 2022-06-17T12:45:44.897 INFO:teuthology.orchestra.run.smithi036.stdout: osd recovery max chunk = 1048576 2022-06-17T12:45:44.897 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.898 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug shutdown = true 2022-06-17T12:45:44.898 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug op order = true 2022-06-17T12:45:44.898 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug verify stray on activate = true 2022-06-17T12:45:44.899 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.899 INFO:teuthology.orchestra.run.smithi036.stdout: osd open classes on start = true 2022-06-17T12:45:44.899 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug pg log writeout = true 2022-06-17T12:45:44.899 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.899 INFO:teuthology.orchestra.run.smithi036.stdout: osd deep scrub update digest min age = 30 2022-06-17T12:45:44.900 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.908 INFO:teuthology.orchestra.run.smithi036.stdout: osd map max advance = 10 2022-06-17T12:45:44.908 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.909 INFO:teuthology.orchestra.run.smithi036.stdout: journal zero on create = true 2022-06-17T12:45:44.909 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.909 INFO:teuthology.orchestra.run.smithi036.stdout: filestore ondisk finisher threads = 3 2022-06-17T12:45:44.909 INFO:teuthology.orchestra.run.smithi036.stdout: filestore apply finisher threads = 3 2022-06-17T12:45:44.910 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.910 INFO:teuthology.orchestra.run.smithi036.stdout: bdev debug aio = true 2022-06-17T12:45:44.910 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug misdirected ops = true 2022-06-17T12:45:44.910 INFO:teuthology.orchestra.run.smithi036.stdout: bluestore block size = 96636764160 2022-06-17T12:45:44.911 INFO:teuthology.orchestra.run.smithi036.stdout: bluestore compression mode = aggressive 2022-06-17T12:45:44.911 INFO:teuthology.orchestra.run.smithi036.stdout: bluestore fsck on mount = True 2022-06-17T12:45:44.911 INFO:teuthology.orchestra.run.smithi036.stdout: debug bluefs = 20 2022-06-17T12:45:44.911 INFO:teuthology.orchestra.run.smithi036.stdout: debug bluestore = 20 2022-06-17T12:45:44.911 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2022-06-17T12:45:44.912 INFO:teuthology.orchestra.run.smithi036.stdout: debug osd = 20 2022-06-17T12:45:44.912 INFO:teuthology.orchestra.run.smithi036.stdout: debug rocksdb = 10 2022-06-17T12:45:44.912 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd backfillfull_ratio = 0.85 2022-06-17T12:45:44.912 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd full ratio = 0.9 2022-06-17T12:45:44.913 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd nearfull ratio = 0.8 2022-06-17T12:45:44.913 INFO:teuthology.orchestra.run.smithi036.stdout: osd failsafe full ratio = 0.95 2022-06-17T12:45:44.913 INFO:teuthology.orchestra.run.smithi036.stdout: osd objectstore = bluestore 2022-06-17T12:45:44.913 INFO:teuthology.orchestra.run.smithi036.stdout: osd op complaint time = 180 2022-06-17T12:45:44.913 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.915 INFO:teuthology.orchestra.run.smithi036.stdout:[mgr] 2022-06-17T12:45:44.916 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2022-06-17T12:45:44.916 INFO:teuthology.orchestra.run.smithi036.stdout: debug mgr = 20 2022-06-17T12:45:44.916 INFO:teuthology.orchestra.run.smithi036.stdout: debug mon = 20 2022-06-17T12:45:44.916 INFO:teuthology.orchestra.run.smithi036.stdout: debug auth = 20 2022-06-17T12:45:44.917 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:45:44.917 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:45:44.917 INFO:teuthology.orchestra.run.smithi036.stdout: mgr/telemetry/nag = false 2022-06-17T12:45:44.917 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.918 INFO:teuthology.orchestra.run.smithi036.stdout:[mon] 2022-06-17T12:45:44.918 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2022-06-17T12:45:44.918 INFO:teuthology.orchestra.run.smithi036.stdout: debug mon = 20 2022-06-17T12:45:44.918 INFO:teuthology.orchestra.run.smithi036.stdout: debug paxos = 20 2022-06-17T12:45:44.919 INFO:teuthology.orchestra.run.smithi036.stdout: debug auth = 20 2022-06-17T12:45:44.919 INFO:teuthology.orchestra.run.smithi036.stdout: mon data avail warn = 5 2022-06-17T12:45:44.919 INFO:teuthology.orchestra.run.smithi036.stdout: mon mgr mkfs grace = 240 2022-06-17T12:45:44.920 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:45:44.920 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd reporter subtree level = osd 2022-06-17T12:45:44.920 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd prime pg temp = true 2022-06-17T12:45:44.920 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:45:44.921 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.921 INFO:teuthology.orchestra.run.smithi036.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-17T12:45:44.921 INFO:teuthology.orchestra.run.smithi036.stdout: auth mon ticket ttl = 660 # 11m 2022-06-17T12:45:44.921 INFO:teuthology.orchestra.run.smithi036.stdout: auth service ticket ttl = 240 # 4m 2022-06-17T12:45:44.921 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.922 INFO:teuthology.orchestra.run.smithi036.stdout: # don't complain about insecure global_id in the test suite 2022-06-17T12:45:44.922 INFO:teuthology.orchestra.run.smithi036.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-17T12:45:44.943 INFO:teuthology.orchestra.run.smithi036.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-17T12:45:44.943 INFO:teuthology.orchestra.run.smithi036.stdout: mon op complaint time = 120 2022-06-17T12:45:44.944 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:45:44.944 INFO:teuthology.orchestra.run.smithi036.stdout:[client] 2022-06-17T12:45:44.944 INFO:teuthology.orchestra.run.smithi036.stdout: rgw cache enabled = true 2022-06-17T12:45:44.945 INFO:teuthology.orchestra.run.smithi036.stdout: rgw enable ops log = true 2022-06-17T12:45:44.945 INFO:teuthology.orchestra.run.smithi036.stdout: rgw enable usage log = true 2022-06-17T12:45:44.946 INFO:teuthology.orchestra.run.smithi036.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-17T12:45:44.946 INFO:teuthology.orchestra.run.smithi036.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-17T12:45:44.947 INFO:teuthology.orchestra.run.smithi036.stdout: client mount timeout = 600 2022-06-17T12:45:44.947 INFO:teuthology.orchestra.run.smithi036.stdout: debug client = 20 2022-06-17T12:45:44.948 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2022-06-17T12:45:44.948 INFO:teuthology.orchestra.run.smithi036.stdout: rados mon op timeout = 15m 2022-06-17T12:45:44.949 INFO:teuthology.orchestra.run.smithi036.stdout: rados osd op timeout = 15m 2022-06-17T12:45:44.949 INFO:teuthology.orchestra.run.smithi036.stdout:[mon.a] 2022-06-17T12:45:44.949 INFO:teuthology.orchestra.run.smithi036.stdout:[mon.b] 2022-06-17T12:45:44.950 INFO:teuthology.orchestra.run.smithi036.stdout:[mon.c] 2022-06-17T12:45:44.950 INFO:teuthology.orchestra.run.smithi036.stdout:[mds.a] 2022-06-17T12:45:44.951 INFO:teuthology.orchestra.run.smithi036.stdout:[mds.b] 2022-06-17T12:45:44.951 INFO:teuthology.orchestra.run.smithi036.stdout:[mds] 2022-06-17T12:45:44.952 INFO:teuthology.orchestra.run.smithi036.stdout: debug mds = 20 2022-06-17T12:45:44.959 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2022-06-17T12:45:44.959 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal frag = True 2022-06-17T12:45:44.960 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal fragment size max = 10000 2022-06-17T12:45:44.960 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal merge size = 5 2022-06-17T12:45:44.960 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal split bits = 3 2022-06-17T12:45:44.961 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal split size = 100 2022-06-17T12:45:44.961 INFO:teuthology.orchestra.run.smithi036.stdout: mds debug frag = True 2022-06-17T12:45:44.962 INFO:teuthology.orchestra.run.smithi036.stdout: mds debug scatterstat = True 2022-06-17T12:45:44.962 INFO:teuthology.orchestra.run.smithi036.stdout: mds op complaint time = 180 2022-06-17T12:45:44.963 INFO:teuthology.orchestra.run.smithi036.stdout: mds verify scatter = True 2022-06-17T12:45:44.964 INFO:teuthology.orchestra.run.smithi036.stdout: osd op complaint time = 180 2022-06-17T12:45:44.964 INFO:teuthology.orchestra.run.smithi036.stdout: rados mon op timeout = 15m 2022-06-17T12:45:44.965 INFO:teuthology.orchestra.run.smithi036.stdout: rados osd op timeout = 15m 2022-06-17T12:45:44.970 INFO:teuthology.orchestra.run.smithi143.stdout:[global] 2022-06-17T12:45:44.971 INFO:teuthology.orchestra.run.smithi143.stdout: chdir = "" 2022-06-17T12:45:44.972 INFO:teuthology.orchestra.run.smithi143.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-17T12:45:44.973 INFO:teuthology.orchestra.run.smithi143.stdout: auth supported = cephx 2022-06-17T12:45:44.973 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:44.974 INFO:teuthology.orchestra.run.smithi143.stdout: filestore xattr use omap = true 2022-06-17T12:45:45.011 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.011 INFO:teuthology.orchestra.run.smithi143.stdout: mon clock drift allowed = 1.000 2022-06-17T12:45:45.012 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.012 INFO:teuthology.orchestra.run.smithi143.stdout: osd crush chooseleaf type = 0 2022-06-17T12:45:45.013 INFO:teuthology.orchestra.run.smithi143.stdout: auth debug = true 2022-06-17T12:45:45.013 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.013 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on old message = true 2022-06-17T12:45:45.019 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on bug = true 2022-06-17T12:45:45.020 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.020 INFO:teuthology.orchestra.run.smithi143.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-17T12:45:45.021 INFO:teuthology.orchestra.run.smithi143.stdout: mon pg warn max object skew = 0 2022-06-17T12:45:45.021 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.027 INFO:teuthology.orchestra.run.smithi143.stdout: # disable pg_autoscaler by default for new pools 2022-06-17T12:45:45.027 INFO:teuthology.orchestra.run.smithi143.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-17T12:45:45.028 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.028 INFO:teuthology.orchestra.run.smithi143.stdout: osd pool default size = 2 2022-06-17T12:45:45.029 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.029 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd allow primary affinity = true 2022-06-17T12:45:45.029 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd allow pg remap = true 2022-06-17T12:45:45.030 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on legacy crush tunables = false 2022-06-17T12:45:45.031 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on crush straw calc version zero = false 2022-06-17T12:45:45.031 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on no sortbitwise = false 2022-06-17T12:45:45.032 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on osd down out interval zero = false 2022-06-17T12:45:45.033 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on too few osds = false 2022-06-17T12:45:45.034 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-17T12:45:45.034 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-17T12:45:45.035 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.035 INFO:teuthology.orchestra.run.smithi143.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-17T12:45:45.036 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.036 INFO:teuthology.orchestra.run.smithi143.stdout: osd default data pool replay window = 5 2022-06-17T12:45:45.046 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.047 INFO:teuthology.orchestra.run.smithi143.stdout: mon allow pool delete = true 2022-06-17T12:45:45.047 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.048 INFO:teuthology.orchestra.run.smithi143.stdout: mon cluster log file level = debug 2022-06-17T12:45:45.048 INFO:teuthology.orchestra.run.smithi143.stdout: debug asserts on shutdown = true 2022-06-17T12:45:45.048 INFO:teuthology.orchestra.run.smithi143.stdout: mon health detail to clog = false 2022-06-17T12:45:45.049 INFO:teuthology.orchestra.run.smithi143.stdout: mon host = "172.21.15.36,172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790]" 2022-06-17T12:45:45.049 INFO:teuthology.orchestra.run.smithi143.stdout: fsid = b68b9700-0c85-42ca-94d7-6cd53db91c14 2022-06-17T12:45:45.049 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.082 INFO:teuthology.orchestra.run.smithi143.stdout:[osd] 2022-06-17T12:45:45.083 INFO:teuthology.orchestra.run.smithi143.stdout: osd journal size = 100 2022-06-17T12:45:45.083 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.083 INFO:teuthology.orchestra.run.smithi143.stdout: osd scrub load threshold = 5.0 2022-06-17T12:45:45.084 INFO:teuthology.orchestra.run.smithi143.stdout: osd scrub max interval = 600 2022-06-17T12:45:45.084 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.085 INFO:teuthology.orchestra.run.smithi143.stdout: osd recover clone overlap = true 2022-06-17T12:45:45.085 INFO:teuthology.orchestra.run.smithi143.stdout: osd recovery max chunk = 1048576 2022-06-17T12:45:45.092 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.093 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug shutdown = true 2022-06-17T12:45:45.093 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug op order = true 2022-06-17T12:45:45.094 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug verify stray on activate = true 2022-06-17T12:45:45.094 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.095 INFO:teuthology.orchestra.run.smithi143.stdout: osd open classes on start = true 2022-06-17T12:45:45.095 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug pg log writeout = true 2022-06-17T12:45:45.118 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.125 INFO:teuthology.orchestra.run.smithi143.stdout: osd deep scrub update digest min age = 30 2022-06-17T12:45:45.126 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.126 INFO:teuthology.orchestra.run.smithi143.stdout: osd map max advance = 10 2022-06-17T12:45:45.127 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.142 INFO:teuthology.orchestra.run.smithi143.stdout: journal zero on create = true 2022-06-17T12:45:45.142 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.143 INFO:teuthology.orchestra.run.smithi143.stdout: filestore ondisk finisher threads = 3 2022-06-17T12:45:45.144 INFO:teuthology.orchestra.run.smithi143.stdout: filestore apply finisher threads = 3 2022-06-17T12:45:45.144 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.145 INFO:teuthology.orchestra.run.smithi143.stdout: bdev debug aio = true 2022-06-17T12:45:45.146 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug misdirected ops = true 2022-06-17T12:45:45.146 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore block size = 96636764160 2022-06-17T12:45:45.147 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore compression mode = aggressive 2022-06-17T12:45:45.147 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore fsck on mount = True 2022-06-17T12:45:45.147 INFO:teuthology.orchestra.run.smithi143.stdout: debug bluefs = 20 2022-06-17T12:45:45.148 INFO:teuthology.orchestra.run.smithi143.stdout: debug bluestore = 20 2022-06-17T12:45:45.148 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2022-06-17T12:45:45.148 INFO:teuthology.orchestra.run.smithi143.stdout: debug osd = 20 2022-06-17T12:45:45.149 INFO:teuthology.orchestra.run.smithi143.stdout: debug rocksdb = 10 2022-06-17T12:45:45.149 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd backfillfull_ratio = 0.85 2022-06-17T12:45:45.149 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd full ratio = 0.9 2022-06-17T12:45:45.151 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd nearfull ratio = 0.8 2022-06-17T12:45:45.152 INFO:teuthology.orchestra.run.smithi143.stdout: osd failsafe full ratio = 0.95 2022-06-17T12:45:45.152 INFO:teuthology.orchestra.run.smithi143.stdout: osd objectstore = bluestore 2022-06-17T12:45:45.152 INFO:teuthology.orchestra.run.smithi143.stdout: osd op complaint time = 180 2022-06-17T12:45:45.153 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.162 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr] 2022-06-17T12:45:45.163 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2022-06-17T12:45:45.163 INFO:teuthology.orchestra.run.smithi143.stdout: debug mgr = 20 2022-06-17T12:45:45.164 INFO:teuthology.orchestra.run.smithi143.stdout: debug mon = 20 2022-06-17T12:45:45.164 INFO:teuthology.orchestra.run.smithi143.stdout: debug auth = 20 2022-06-17T12:45:45.164 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:45:45.165 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:45:45.165 INFO:teuthology.orchestra.run.smithi143.stdout: mgr/telemetry/nag = false 2022-06-17T12:45:45.165 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.170 INFO:teuthology.orchestra.run.smithi143.stdout:[mon] 2022-06-17T12:45:45.171 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2022-06-17T12:45:45.171 INFO:teuthology.orchestra.run.smithi143.stdout: debug mon = 20 2022-06-17T12:45:45.172 INFO:teuthology.orchestra.run.smithi143.stdout: debug paxos = 20 2022-06-17T12:45:45.173 INFO:teuthology.orchestra.run.smithi143.stdout: debug auth = 20 2022-06-17T12:45:45.174 INFO:teuthology.orchestra.run.smithi143.stdout: mon data avail warn = 5 2022-06-17T12:45:45.174 INFO:teuthology.orchestra.run.smithi143.stdout: mon mgr mkfs grace = 240 2022-06-17T12:45:45.175 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:45:45.175 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd reporter subtree level = osd 2022-06-17T12:45:45.175 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd prime pg temp = true 2022-06-17T12:45:45.176 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:45:45.176 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.177 INFO:teuthology.orchestra.run.smithi143.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-17T12:45:45.177 INFO:teuthology.orchestra.run.smithi143.stdout: auth mon ticket ttl = 660 # 11m 2022-06-17T12:45:45.177 INFO:teuthology.orchestra.run.smithi143.stdout: auth service ticket ttl = 240 # 4m 2022-06-17T12:45:45.179 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.179 INFO:teuthology.orchestra.run.smithi143.stdout: # don't complain about insecure global_id in the test suite 2022-06-17T12:45:45.180 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-17T12:45:45.220 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-17T12:45:45.220 INFO:teuthology.orchestra.run.smithi143.stdout: mon op complaint time = 120 2022-06-17T12:45:45.221 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:45:45.221 INFO:teuthology.orchestra.run.smithi143.stdout:[client] 2022-06-17T12:45:45.221 INFO:teuthology.orchestra.run.smithi143.stdout: rgw cache enabled = true 2022-06-17T12:45:45.222 INFO:teuthology.orchestra.run.smithi143.stdout: rgw enable ops log = true 2022-06-17T12:45:45.222 INFO:teuthology.orchestra.run.smithi143.stdout: rgw enable usage log = true 2022-06-17T12:45:45.223 INFO:teuthology.orchestra.run.smithi143.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-17T12:45:45.223 INFO:teuthology.orchestra.run.smithi143.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-17T12:45:45.224 INFO:teuthology.orchestra.run.smithi143.stdout: client mount timeout = 600 2022-06-17T12:45:45.224 INFO:teuthology.orchestra.run.smithi143.stdout: debug client = 20 2022-06-17T12:45:45.224 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2022-06-17T12:45:45.225 INFO:teuthology.orchestra.run.smithi143.stdout: rados mon op timeout = 15m 2022-06-17T12:45:45.229 INFO:teuthology.orchestra.run.smithi143.stdout: rados osd op timeout = 15m 2022-06-17T12:45:45.230 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.a] 2022-06-17T12:45:45.230 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.b] 2022-06-17T12:45:45.231 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.c] 2022-06-17T12:45:45.231 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.a] 2022-06-17T12:45:45.232 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.b] 2022-06-17T12:45:45.232 INFO:teuthology.orchestra.run.smithi143.stdout:[mds] 2022-06-17T12:45:45.232 INFO:teuthology.orchestra.run.smithi143.stdout: debug mds = 20 2022-06-17T12:45:45.233 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2022-06-17T12:45:45.233 INFO:teuthology.orchestra.run.smithi143.stdout: mds bal frag = True 2022-06-17T12:45:45.234 INFO:teuthology.orchestra.run.smithi143.stdout: mds bal fragment size max = 10000 2022-06-17T12:45:45.234 INFO:teuthology.orchestra.run.smithi143.stdout: mds bal merge size = 5 2022-06-17T12:45:45.235 INFO:teuthology.orchestra.run.smithi143.stdout: mds bal split bits = 3 2022-06-17T12:45:45.236 INFO:teuthology.orchestra.run.smithi143.stdout: mds bal split size = 100 2022-06-17T12:45:45.236 INFO:teuthology.orchestra.run.smithi143.stdout: mds debug frag = True 2022-06-17T12:45:45.236 INFO:teuthology.orchestra.run.smithi143.stdout: mds debug scatterstat = True 2022-06-17T12:45:45.237 INFO:teuthology.orchestra.run.smithi143.stdout: mds op complaint time = 180 2022-06-17T12:45:45.248 INFO:teuthology.orchestra.run.smithi143.stdout: mds verify scatter = True 2022-06-17T12:45:45.248 INFO:teuthology.orchestra.run.smithi143.stdout: osd op complaint time = 180 2022-06-17T12:45:45.248 INFO:teuthology.orchestra.run.smithi143.stdout: rados mon op timeout = 15m 2022-06-17T12:45:45.266 INFO:teuthology.orchestra.run.smithi143.stdout: rados osd op timeout = 15m 2022-06-17T12:45:45.266 INFO:teuthology.orchestra.run.smithi136.stdout:[global] 2022-06-17T12:45:45.267 INFO:teuthology.orchestra.run.smithi136.stdout: chdir = "" 2022-06-17T12:45:45.267 INFO:teuthology.orchestra.run.smithi136.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-17T12:45:45.268 INFO:teuthology.orchestra.run.smithi136.stdout: auth supported = cephx 2022-06-17T12:45:45.268 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.269 INFO:teuthology.orchestra.run.smithi136.stdout: filestore xattr use omap = true 2022-06-17T12:45:45.269 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.269 INFO:teuthology.orchestra.run.smithi136.stdout: mon clock drift allowed = 1.000 2022-06-17T12:45:45.270 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.270 INFO:teuthology.orchestra.run.smithi136.stdout: osd crush chooseleaf type = 0 2022-06-17T12:45:45.271 INFO:teuthology.orchestra.run.smithi136.stdout: auth debug = true 2022-06-17T12:45:45.271 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.272 INFO:teuthology.orchestra.run.smithi136.stdout: ms die on old message = true 2022-06-17T12:45:45.272 INFO:teuthology.orchestra.run.smithi136.stdout: ms die on bug = true 2022-06-17T12:45:45.273 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.273 INFO:teuthology.orchestra.run.smithi136.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-17T12:45:45.274 INFO:teuthology.orchestra.run.smithi136.stdout: mon pg warn max object skew = 0 2022-06-17T12:45:45.274 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.275 INFO:teuthology.orchestra.run.smithi136.stdout: # disable pg_autoscaler by default for new pools 2022-06-17T12:45:45.275 INFO:teuthology.orchestra.run.smithi136.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-17T12:45:45.276 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.277 INFO:teuthology.orchestra.run.smithi136.stdout: osd pool default size = 2 2022-06-17T12:45:45.277 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.277 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd allow primary affinity = true 2022-06-17T12:45:45.278 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd allow pg remap = true 2022-06-17T12:45:45.278 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on legacy crush tunables = false 2022-06-17T12:45:45.279 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on crush straw calc version zero = false 2022-06-17T12:45:45.279 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on no sortbitwise = false 2022-06-17T12:45:45.280 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on osd down out interval zero = false 2022-06-17T12:45:45.280 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on too few osds = false 2022-06-17T12:45:45.281 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-17T12:45:45.281 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-17T12:45:45.281 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.292 INFO:teuthology.orchestra.run.smithi136.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-17T12:45:45.292 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.293 INFO:teuthology.orchestra.run.smithi136.stdout: osd default data pool replay window = 5 2022-06-17T12:45:45.293 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.294 INFO:teuthology.orchestra.run.smithi136.stdout: mon allow pool delete = true 2022-06-17T12:45:45.294 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.295 INFO:teuthology.orchestra.run.smithi136.stdout: mon cluster log file level = debug 2022-06-17T12:45:45.295 INFO:teuthology.orchestra.run.smithi136.stdout: debug asserts on shutdown = true 2022-06-17T12:45:45.295 INFO:teuthology.orchestra.run.smithi136.stdout: mon health detail to clog = false 2022-06-17T12:45:45.296 INFO:teuthology.orchestra.run.smithi136.stdout: mon host = "172.21.15.36,172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790]" 2022-06-17T12:45:45.297 INFO:teuthology.orchestra.run.smithi136.stdout: fsid = b68b9700-0c85-42ca-94d7-6cd53db91c14 2022-06-17T12:45:45.297 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.297 INFO:teuthology.orchestra.run.smithi136.stdout:[osd] 2022-06-17T12:45:45.298 INFO:teuthology.orchestra.run.smithi136.stdout: osd journal size = 100 2022-06-17T12:45:45.298 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.299 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub load threshold = 5.0 2022-06-17T12:45:45.299 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub max interval = 600 2022-06-17T12:45:45.300 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.300 INFO:teuthology.orchestra.run.smithi136.stdout: osd recover clone overlap = true 2022-06-17T12:45:45.301 INFO:teuthology.orchestra.run.smithi136.stdout: osd recovery max chunk = 1048576 2022-06-17T12:45:45.301 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.301 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug shutdown = true 2022-06-17T12:45:45.302 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug op order = true 2022-06-17T12:45:45.302 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug verify stray on activate = true 2022-06-17T12:45:45.303 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.303 INFO:teuthology.orchestra.run.smithi136.stdout: osd open classes on start = true 2022-06-17T12:45:45.303 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug pg log writeout = true 2022-06-17T12:45:45.304 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.304 INFO:teuthology.orchestra.run.smithi136.stdout: osd deep scrub update digest min age = 30 2022-06-17T12:45:45.305 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.305 INFO:teuthology.orchestra.run.smithi136.stdout: osd map max advance = 10 2022-06-17T12:45:45.305 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.306 INFO:teuthology.orchestra.run.smithi136.stdout: journal zero on create = true 2022-06-17T12:45:45.308 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.308 INFO:teuthology.orchestra.run.smithi136.stdout: filestore ondisk finisher threads = 3 2022-06-17T12:45:45.309 INFO:teuthology.orchestra.run.smithi136.stdout: filestore apply finisher threads = 3 2022-06-17T12:45:45.309 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.309 INFO:teuthology.orchestra.run.smithi136.stdout: bdev debug aio = true 2022-06-17T12:45:45.310 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug misdirected ops = true 2022-06-17T12:45:45.310 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore block size = 96636764160 2022-06-17T12:45:45.311 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore compression mode = aggressive 2022-06-17T12:45:45.311 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore fsck on mount = True 2022-06-17T12:45:45.312 INFO:teuthology.orchestra.run.smithi136.stdout: debug bluefs = 20 2022-06-17T12:45:45.312 INFO:teuthology.orchestra.run.smithi136.stdout: debug bluestore = 20 2022-06-17T12:45:45.313 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2022-06-17T12:45:45.313 INFO:teuthology.orchestra.run.smithi136.stdout: debug osd = 20 2022-06-17T12:45:45.314 INFO:teuthology.orchestra.run.smithi136.stdout: debug rocksdb = 10 2022-06-17T12:45:45.315 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd backfillfull_ratio = 0.85 2022-06-17T12:45:45.315 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd full ratio = 0.9 2022-06-17T12:45:45.316 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd nearfull ratio = 0.8 2022-06-17T12:45:45.316 INFO:teuthology.orchestra.run.smithi136.stdout: osd failsafe full ratio = 0.95 2022-06-17T12:45:45.316 INFO:teuthology.orchestra.run.smithi136.stdout: osd objectstore = bluestore 2022-06-17T12:45:45.317 INFO:teuthology.orchestra.run.smithi136.stdout: osd op complaint time = 180 2022-06-17T12:45:45.317 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.328 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr] 2022-06-17T12:45:45.329 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2022-06-17T12:45:45.332 INFO:teuthology.orchestra.run.smithi136.stdout: debug mgr = 20 2022-06-17T12:45:45.333 INFO:teuthology.orchestra.run.smithi136.stdout: debug mon = 20 2022-06-17T12:45:45.344 INFO:teuthology.orchestra.run.smithi136.stdout: debug auth = 20 2022-06-17T12:45:45.344 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:45:45.344 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:45:45.345 INFO:teuthology.orchestra.run.smithi136.stdout: mgr/telemetry/nag = false 2022-06-17T12:45:45.345 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.363 INFO:teuthology.orchestra.run.smithi136.stdout:[mon] 2022-06-17T12:45:45.363 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2022-06-17T12:45:45.366 INFO:teuthology.orchestra.run.smithi136.stdout: debug mon = 20 2022-06-17T12:45:45.367 INFO:teuthology.orchestra.run.smithi136.stdout: debug paxos = 20 2022-06-17T12:45:45.367 INFO:teuthology.orchestra.run.smithi136.stdout: debug auth = 20 2022-06-17T12:45:45.367 INFO:teuthology.orchestra.run.smithi136.stdout: mon data avail warn = 5 2022-06-17T12:45:45.374 INFO:teuthology.orchestra.run.smithi136.stdout: mon mgr mkfs grace = 240 2022-06-17T12:45:45.375 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:45:45.375 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd reporter subtree level = osd 2022-06-17T12:45:45.375 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd prime pg temp = true 2022-06-17T12:45:45.375 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:45:45.376 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.376 INFO:teuthology.orchestra.run.smithi136.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-17T12:45:45.376 INFO:teuthology.orchestra.run.smithi136.stdout: auth mon ticket ttl = 660 # 11m 2022-06-17T12:45:45.376 INFO:teuthology.orchestra.run.smithi136.stdout: auth service ticket ttl = 240 # 4m 2022-06-17T12:45:45.377 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.377 INFO:teuthology.orchestra.run.smithi136.stdout: # don't complain about insecure global_id in the test suite 2022-06-17T12:45:45.377 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-17T12:45:45.377 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-17T12:45:45.377 INFO:teuthology.orchestra.run.smithi136.stdout: mon op complaint time = 120 2022-06-17T12:45:45.380 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:45:45.380 INFO:teuthology.orchestra.run.smithi136.stdout:[client] 2022-06-17T12:45:45.380 INFO:teuthology.orchestra.run.smithi136.stdout: rgw cache enabled = true 2022-06-17T12:45:45.381 INFO:teuthology.orchestra.run.smithi136.stdout: rgw enable ops log = true 2022-06-17T12:45:45.382 INFO:teuthology.orchestra.run.smithi136.stdout: rgw enable usage log = true 2022-06-17T12:45:45.382 INFO:teuthology.orchestra.run.smithi136.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-17T12:45:45.382 INFO:teuthology.orchestra.run.smithi136.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-17T12:45:45.382 INFO:teuthology.orchestra.run.smithi136.stdout: client mount timeout = 600 2022-06-17T12:45:45.383 INFO:teuthology.orchestra.run.smithi136.stdout: debug client = 20 2022-06-17T12:45:45.383 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2022-06-17T12:45:45.383 INFO:teuthology.orchestra.run.smithi136.stdout: rados mon op timeout = 15m 2022-06-17T12:45:45.383 INFO:teuthology.orchestra.run.smithi136.stdout: rados osd op timeout = 15m 2022-06-17T12:45:45.384 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.a] 2022-06-17T12:45:45.384 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.b] 2022-06-17T12:45:45.384 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.c] 2022-06-17T12:45:45.395 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.a] 2022-06-17T12:45:45.395 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.b] 2022-06-17T12:45:45.395 INFO:teuthology.orchestra.run.smithi136.stdout:[mds] 2022-06-17T12:45:45.397 INFO:teuthology.orchestra.run.smithi136.stdout: debug mds = 20 2022-06-17T12:45:45.397 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2022-06-17T12:45:45.397 INFO:teuthology.orchestra.run.smithi136.stdout: mds bal frag = True 2022-06-17T12:45:45.398 INFO:teuthology.orchestra.run.smithi136.stdout: mds bal fragment size max = 10000 2022-06-17T12:45:45.398 INFO:teuthology.orchestra.run.smithi136.stdout: mds bal merge size = 5 2022-06-17T12:45:45.398 INFO:teuthology.orchestra.run.smithi136.stdout: mds bal split bits = 3 2022-06-17T12:45:45.398 INFO:teuthology.orchestra.run.smithi136.stdout: mds bal split size = 100 2022-06-17T12:45:45.399 INFO:teuthology.orchestra.run.smithi136.stdout: mds debug frag = True 2022-06-17T12:45:45.399 INFO:teuthology.orchestra.run.smithi136.stdout: mds debug scatterstat = True 2022-06-17T12:45:45.399 INFO:teuthology.orchestra.run.smithi136.stdout: mds op complaint time = 180 2022-06-17T12:45:45.399 INFO:teuthology.orchestra.run.smithi136.stdout: mds verify scatter = True 2022-06-17T12:45:45.400 INFO:teuthology.orchestra.run.smithi136.stdout: osd op complaint time = 180 2022-06-17T12:45:45.400 INFO:teuthology.orchestra.run.smithi136.stdout: rados mon op timeout = 15m 2022-06-17T12:45:45.400 INFO:teuthology.orchestra.run.smithi136.stdout: rados osd op timeout = 15m 2022-06-17T12:45:45.400 INFO:teuthology.orchestra.run.smithi107.stdout:[global] 2022-06-17T12:45:45.401 INFO:teuthology.orchestra.run.smithi107.stdout: chdir = "" 2022-06-17T12:45:45.401 INFO:teuthology.orchestra.run.smithi107.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-17T12:45:45.401 INFO:teuthology.orchestra.run.smithi107.stdout: auth supported = cephx 2022-06-17T12:45:45.401 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.402 INFO:teuthology.orchestra.run.smithi107.stdout: filestore xattr use omap = true 2022-06-17T12:45:45.402 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.403 INFO:teuthology.orchestra.run.smithi107.stdout: mon clock drift allowed = 1.000 2022-06-17T12:45:45.403 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.403 INFO:teuthology.orchestra.run.smithi107.stdout: osd crush chooseleaf type = 0 2022-06-17T12:45:45.403 INFO:teuthology.orchestra.run.smithi107.stdout: auth debug = true 2022-06-17T12:45:45.404 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.404 INFO:teuthology.orchestra.run.smithi107.stdout: ms die on old message = true 2022-06-17T12:45:45.404 INFO:teuthology.orchestra.run.smithi107.stdout: ms die on bug = true 2022-06-17T12:45:45.404 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.404 INFO:teuthology.orchestra.run.smithi107.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-17T12:45:45.405 INFO:teuthology.orchestra.run.smithi107.stdout: mon pg warn max object skew = 0 2022-06-17T12:45:45.405 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.405 INFO:teuthology.orchestra.run.smithi107.stdout: # disable pg_autoscaler by default for new pools 2022-06-17T12:45:45.406 INFO:teuthology.orchestra.run.smithi107.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-17T12:45:45.406 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.406 INFO:teuthology.orchestra.run.smithi107.stdout: osd pool default size = 2 2022-06-17T12:45:45.406 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.407 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd allow primary affinity = true 2022-06-17T12:45:45.407 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd allow pg remap = true 2022-06-17T12:45:45.407 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on legacy crush tunables = false 2022-06-17T12:45:45.407 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on crush straw calc version zero = false 2022-06-17T12:45:45.427 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on no sortbitwise = false 2022-06-17T12:45:45.428 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on osd down out interval zero = false 2022-06-17T12:45:45.428 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on too few osds = false 2022-06-17T12:45:45.428 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-17T12:45:45.434 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-17T12:45:45.435 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.435 INFO:teuthology.orchestra.run.smithi107.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-17T12:45:45.435 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.436 INFO:teuthology.orchestra.run.smithi107.stdout: osd default data pool replay window = 5 2022-06-17T12:45:45.436 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.437 INFO:teuthology.orchestra.run.smithi107.stdout: mon allow pool delete = true 2022-06-17T12:45:45.437 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.438 INFO:teuthology.orchestra.run.smithi107.stdout: mon cluster log file level = debug 2022-06-17T12:45:45.438 INFO:teuthology.orchestra.run.smithi107.stdout: debug asserts on shutdown = true 2022-06-17T12:45:45.439 INFO:teuthology.orchestra.run.smithi107.stdout: mon health detail to clog = false 2022-06-17T12:45:45.439 INFO:teuthology.orchestra.run.smithi107.stdout: mon host = "172.21.15.36,172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790]" 2022-06-17T12:45:45.440 INFO:teuthology.orchestra.run.smithi107.stdout: fsid = b68b9700-0c85-42ca-94d7-6cd53db91c14 2022-06-17T12:45:45.440 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.441 INFO:teuthology.orchestra.run.smithi107.stdout:[osd] 2022-06-17T12:45:45.441 INFO:teuthology.orchestra.run.smithi107.stdout: osd journal size = 100 2022-06-17T12:45:45.442 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.442 INFO:teuthology.orchestra.run.smithi107.stdout: osd scrub load threshold = 5.0 2022-06-17T12:45:45.443 INFO:teuthology.orchestra.run.smithi107.stdout: osd scrub max interval = 600 2022-06-17T12:45:45.443 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.444 INFO:teuthology.orchestra.run.smithi107.stdout: osd recover clone overlap = true 2022-06-17T12:45:45.444 INFO:teuthology.orchestra.run.smithi107.stdout: osd recovery max chunk = 1048576 2022-06-17T12:45:45.444 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.445 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug shutdown = true 2022-06-17T12:45:45.445 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug op order = true 2022-06-17T12:45:45.446 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug verify stray on activate = true 2022-06-17T12:45:45.446 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.447 INFO:teuthology.orchestra.run.smithi107.stdout: osd open classes on start = true 2022-06-17T12:45:45.447 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug pg log writeout = true 2022-06-17T12:45:45.448 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.448 INFO:teuthology.orchestra.run.smithi107.stdout: osd deep scrub update digest min age = 30 2022-06-17T12:45:45.449 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.450 INFO:teuthology.orchestra.run.smithi107.stdout: osd map max advance = 10 2022-06-17T12:45:45.451 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.459 INFO:teuthology.orchestra.run.smithi107.stdout: journal zero on create = true 2022-06-17T12:45:45.460 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.461 INFO:teuthology.orchestra.run.smithi107.stdout: filestore ondisk finisher threads = 3 2022-06-17T12:45:45.462 INFO:teuthology.orchestra.run.smithi107.stdout: filestore apply finisher threads = 3 2022-06-17T12:45:45.462 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.463 INFO:teuthology.orchestra.run.smithi107.stdout: bdev debug aio = true 2022-06-17T12:45:45.463 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug misdirected ops = true 2022-06-17T12:45:45.464 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore block size = 96636764160 2022-06-17T12:45:45.464 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore compression mode = aggressive 2022-06-17T12:45:45.465 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore fsck on mount = True 2022-06-17T12:45:45.465 INFO:teuthology.orchestra.run.smithi107.stdout: debug bluefs = 20 2022-06-17T12:45:45.465 INFO:teuthology.orchestra.run.smithi107.stdout: debug bluestore = 20 2022-06-17T12:45:45.480 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2022-06-17T12:45:45.480 INFO:teuthology.orchestra.run.smithi107.stdout: debug osd = 20 2022-06-17T12:45:45.481 INFO:teuthology.orchestra.run.smithi107.stdout: debug rocksdb = 10 2022-06-17T12:45:45.481 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd backfillfull_ratio = 0.85 2022-06-17T12:45:45.481 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd full ratio = 0.9 2022-06-17T12:45:45.482 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd nearfull ratio = 0.8 2022-06-17T12:45:45.482 INFO:teuthology.orchestra.run.smithi107.stdout: osd failsafe full ratio = 0.95 2022-06-17T12:45:45.483 INFO:teuthology.orchestra.run.smithi107.stdout: osd objectstore = bluestore 2022-06-17T12:45:45.484 INFO:teuthology.orchestra.run.smithi107.stdout: osd op complaint time = 180 2022-06-17T12:45:45.484 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.485 INFO:teuthology.orchestra.run.smithi107.stdout:[mgr] 2022-06-17T12:45:45.485 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2022-06-17T12:45:45.485 INFO:teuthology.orchestra.run.smithi107.stdout: debug mgr = 20 2022-06-17T12:45:45.486 INFO:teuthology.orchestra.run.smithi107.stdout: debug mon = 20 2022-06-17T12:45:45.510 INFO:teuthology.orchestra.run.smithi107.stdout: debug auth = 20 2022-06-17T12:45:45.511 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:45:45.511 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:45:45.511 INFO:teuthology.orchestra.run.smithi107.stdout: mgr/telemetry/nag = false 2022-06-17T12:45:45.512 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.512 INFO:teuthology.orchestra.run.smithi107.stdout:[mon] 2022-06-17T12:45:45.512 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2022-06-17T12:45:45.513 INFO:teuthology.orchestra.run.smithi107.stdout: debug mon = 20 2022-06-17T12:45:45.513 INFO:teuthology.orchestra.run.smithi107.stdout: debug paxos = 20 2022-06-17T12:45:45.513 INFO:teuthology.orchestra.run.smithi107.stdout: debug auth = 20 2022-06-17T12:45:45.549 INFO:teuthology.orchestra.run.smithi107.stdout: mon data avail warn = 5 2022-06-17T12:45:45.550 INFO:teuthology.orchestra.run.smithi107.stdout: mon mgr mkfs grace = 240 2022-06-17T12:45:45.550 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:45:45.551 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd reporter subtree level = osd 2022-06-17T12:45:45.552 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd prime pg temp = true 2022-06-17T12:45:45.553 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:45:45.554 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.555 INFO:teuthology.orchestra.run.smithi107.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-17T12:45:45.587 INFO:teuthology.orchestra.run.smithi107.stdout: auth mon ticket ttl = 660 # 11m 2022-06-17T12:45:45.587 INFO:teuthology.orchestra.run.smithi107.stdout: auth service ticket ttl = 240 # 4m 2022-06-17T12:45:45.588 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.588 INFO:teuthology.orchestra.run.smithi107.stdout: # don't complain about insecure global_id in the test suite 2022-06-17T12:45:45.589 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-17T12:45:45.590 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-17T12:45:45.590 INFO:teuthology.orchestra.run.smithi107.stdout: mon op complaint time = 120 2022-06-17T12:45:45.590 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:45:45.591 INFO:teuthology.orchestra.run.smithi107.stdout:[client] 2022-06-17T12:45:45.600 INFO:teuthology.orchestra.run.smithi107.stdout: rgw cache enabled = true 2022-06-17T12:45:45.600 INFO:teuthology.orchestra.run.smithi107.stdout: rgw enable ops log = true 2022-06-17T12:45:45.600 INFO:teuthology.orchestra.run.smithi107.stdout: rgw enable usage log = true 2022-06-17T12:45:45.601 INFO:teuthology.orchestra.run.smithi107.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-17T12:45:45.601 INFO:teuthology.orchestra.run.smithi107.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-17T12:45:45.602 INFO:teuthology.orchestra.run.smithi107.stdout: client mount timeout = 600 2022-06-17T12:45:45.602 INFO:teuthology.orchestra.run.smithi107.stdout: debug client = 20 2022-06-17T12:45:45.602 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2022-06-17T12:45:45.603 INFO:teuthology.orchestra.run.smithi107.stdout: rados mon op timeout = 15m 2022-06-17T12:45:45.603 INFO:teuthology.orchestra.run.smithi107.stdout: rados osd op timeout = 15m 2022-06-17T12:45:45.603 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.a] 2022-06-17T12:45:45.604 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.b] 2022-06-17T12:45:45.604 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.c] 2022-06-17T12:45:45.605 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.a] 2022-06-17T12:45:45.605 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.b] 2022-06-17T12:45:45.606 INFO:teuthology.orchestra.run.smithi107.stdout:[mds] 2022-06-17T12:45:45.620 INFO:teuthology.orchestra.run.smithi107.stdout: debug mds = 20 2022-06-17T12:45:45.620 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2022-06-17T12:45:45.620 INFO:teuthology.orchestra.run.smithi107.stdout: mds bal frag = True 2022-06-17T12:45:45.621 INFO:teuthology.orchestra.run.smithi107.stdout: mds bal fragment size max = 10000 2022-06-17T12:45:45.621 INFO:teuthology.orchestra.run.smithi107.stdout: mds bal merge size = 5 2022-06-17T12:45:45.622 INFO:teuthology.orchestra.run.smithi107.stdout: mds bal split bits = 3 2022-06-17T12:45:45.622 INFO:teuthology.orchestra.run.smithi107.stdout: mds bal split size = 100 2022-06-17T12:45:45.623 INFO:teuthology.orchestra.run.smithi107.stdout: mds debug frag = True 2022-06-17T12:45:45.623 INFO:teuthology.orchestra.run.smithi107.stdout: mds debug scatterstat = True 2022-06-17T12:45:45.624 INFO:teuthology.orchestra.run.smithi107.stdout: mds op complaint time = 180 2022-06-17T12:45:45.625 INFO:teuthology.orchestra.run.smithi107.stdout: mds verify scatter = True 2022-06-17T12:45:45.625 INFO:teuthology.orchestra.run.smithi107.stdout: osd op complaint time = 180 2022-06-17T12:45:45.626 INFO:teuthology.orchestra.run.smithi107.stdout: rados mon op timeout = 15m 2022-06-17T12:45:45.626 INFO:teuthology.orchestra.run.smithi107.stdout: rados osd op timeout = 15m 2022-06-17T12:45:45.628 INFO:tasks.ceph:Creating admin key on mon.a... 2022-06-17T12:45:45.629 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2022-06-17T12:45:45.716 INFO:tasks.ceph:Copying monmap to all nodes... 2022-06-17T12:45:45.839 DEBUG:teuthology.orchestra.remote:smithi036:/etc/ceph/ceph.keyring is 206B 2022-06-17T12:45:46.058 DEBUG:teuthology.orchestra.remote:smithi036:/home/ubuntu/cephtest/ceph.monmap is 379B 2022-06-17T12:45:46.155 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi036.front.sepia.ceph.com 2022-06-17T12:45:46.158 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-06-17T12:45:46.159 DEBUG:teuthology.orchestra.run.smithi036:> sudo dd of=/etc/ceph/ceph.keyring 2022-06-17T12:45:46.160 DEBUG:teuthology.orchestra.run.smithi036:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-06-17T12:45:46.242 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-06-17T12:45:46.243 DEBUG:teuthology.orchestra.run.smithi036:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-06-17T12:45:46.282 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi070.front.sepia.ceph.com 2022-06-17T12:45:46.291 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2022-06-17T12:45:46.291 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.keyring 2022-06-17T12:45:46.292 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-06-17T12:45:46.377 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2022-06-17T12:45:46.398 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-06-17T12:45:46.471 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi107.front.sepia.ceph.com 2022-06-17T12:45:46.471 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2022-06-17T12:45:46.472 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.keyring 2022-06-17T12:45:46.473 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-06-17T12:45:46.555 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2022-06-17T12:45:46.556 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-06-17T12:45:46.622 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi136.front.sepia.ceph.com 2022-06-17T12:45:46.623 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2022-06-17T12:45:46.624 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.keyring 2022-06-17T12:45:46.624 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-06-17T12:45:46.663 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2022-06-17T12:45:46.664 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-06-17T12:45:46.680 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi143.front.sepia.ceph.com 2022-06-17T12:45:46.681 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2022-06-17T12:45:46.683 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/ceph/ceph.keyring 2022-06-17T12:45:46.684 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-06-17T12:45:46.723 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2022-06-17T12:45:46.724 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-06-17T12:45:46.787 INFO:tasks.ceph:Setting up mon nodes... 2022-06-17T12:45:46.789 INFO:tasks.ceph:Setting up mgr nodes... 2022-06-17T12:45:46.790 DEBUG:teuthology.orchestra.run.smithi036:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2022-06-17T12:45:46.851 INFO:teuthology.orchestra.run.smithi036.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2022-06-17T12:45:46.854 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2022-06-17T12:45:46.949 INFO:teuthology.orchestra.run.smithi070.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2022-06-17T12:45:46.949 INFO:tasks.ceph:Setting up mds nodes... 2022-06-17T12:45:46.950 DEBUG:teuthology.orchestra.run.smithi036:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2022-06-17T12:45:47.001 INFO:teuthology.orchestra.run.smithi036.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2022-06-17T12:45:47.008 DEBUG:teuthology.orchestra.run.smithi036:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2022-06-17T12:45:47.070 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2022-06-17T12:45:47.134 INFO:teuthology.orchestra.run.smithi070.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2022-06-17T12:45:47.153 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2022-06-17T12:45:47.248 INFO:tasks.ceph_client:Setting up client nodes... 2022-06-17T12:45:47.249 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2022-06-17T12:45:47.337 INFO:teuthology.orchestra.run.smithi107.stdout:creating /etc/ceph/ceph.client.0.keyring 2022-06-17T12:45:47.346 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2022-06-17T12:45:47.403 INFO:teuthology.orchestra.run.smithi136.stdout:creating /etc/ceph/ceph.client.1.keyring 2022-06-17T12:45:47.414 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2022-06-17T12:45:47.467 INFO:teuthology.orchestra.run.smithi143.stdout:creating /etc/ceph/ceph.client.2.keyring 2022-06-17T12:45:47.483 INFO:tasks.ceph:Running mkfs on osd nodes... 2022-06-17T12:45:47.484 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi036.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi070.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-06-17T12:45:47.485 DEBUG:teuthology.orchestra.run.smithi036:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2022-06-17T12:45:47.575 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-06-17T12:45:47.576 INFO:tasks.ceph:role: osd.0 2022-06-17T12:45:47.586 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi036.front.sepia.ceph.com 2022-06-17T12:45:47.587 DEBUG:teuthology.orchestra.run.smithi036:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-06-17T12:45:48.187 INFO:teuthology.orchestra.run.smithi036.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-06-17T12:45:48.188 INFO:teuthology.orchestra.run.smithi036.stdout: = sectsz=512 attr=2, projid32bit=1 2022-06-17T12:45:48.188 INFO:teuthology.orchestra.run.smithi036.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-06-17T12:45:48.188 INFO:teuthology.orchestra.run.smithi036.stdout: = reflink=1 2022-06-17T12:45:48.189 INFO:teuthology.orchestra.run.smithi036.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-06-17T12:45:48.189 INFO:teuthology.orchestra.run.smithi036.stdout: = sunit=0 swidth=0 blks 2022-06-17T12:45:48.189 INFO:teuthology.orchestra.run.smithi036.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-06-17T12:45:48.189 INFO:teuthology.orchestra.run.smithi036.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-06-17T12:45:48.190 INFO:teuthology.orchestra.run.smithi036.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-06-17T12:45:48.190 INFO:teuthology.orchestra.run.smithi036.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-06-17T12:45:48.191 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi036.front.sepia.ceph.com -o noatime 2022-06-17T12:45:48.191 DEBUG:teuthology.orchestra.run.smithi036:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2022-06-17T12:45:48.351 DEBUG:teuthology.orchestra.run.smithi036:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2022-06-17T12:45:48.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:45:48.465 INFO:teuthology.orchestra.run.smithi036.stderr:sudo: /sbin/restorecon: command not found 2022-06-17T12:45:48.469 DEBUG:teuthology.orchestra.run.smithi036:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2022-06-17T12:45:48.498 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-06-17T12:45:48.499 INFO:tasks.ceph:role: osd.1 2022-06-17T12:45:48.500 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi036.front.sepia.ceph.com 2022-06-17T12:45:48.500 DEBUG:teuthology.orchestra.run.smithi036:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-06-17T12:45:49.057 INFO:teuthology.orchestra.run.smithi036.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-06-17T12:45:49.079 INFO:teuthology.orchestra.run.smithi036.stdout: = sectsz=512 attr=2, projid32bit=1 2022-06-17T12:45:49.080 INFO:teuthology.orchestra.run.smithi036.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-06-17T12:45:49.112 INFO:teuthology.orchestra.run.smithi036.stdout: = reflink=1 2022-06-17T12:45:49.113 INFO:teuthology.orchestra.run.smithi036.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-06-17T12:45:49.113 INFO:teuthology.orchestra.run.smithi036.stdout: = sunit=0 swidth=0 blks 2022-06-17T12:45:49.127 INFO:teuthology.orchestra.run.smithi036.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-06-17T12:45:49.127 INFO:teuthology.orchestra.run.smithi036.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-06-17T12:45:49.128 INFO:teuthology.orchestra.run.smithi036.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-06-17T12:45:49.128 INFO:teuthology.orchestra.run.smithi036.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-06-17T12:45:49.160 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi036.front.sepia.ceph.com -o noatime 2022-06-17T12:45:49.161 DEBUG:teuthology.orchestra.run.smithi036:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2022-06-17T12:45:49.242 DEBUG:teuthology.orchestra.run.smithi036:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2022-06-17T12:45:49.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:45:49.323 INFO:teuthology.orchestra.run.smithi036.stderr:sudo: /sbin/restorecon: command not found 2022-06-17T12:45:49.324 DEBUG:teuthology.orchestra.run.smithi036:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2022-06-17T12:45:49.354 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-06-17T12:45:49.355 INFO:tasks.ceph:role: osd.2 2022-06-17T12:45:49.355 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi036.front.sepia.ceph.com 2022-06-17T12:45:49.356 DEBUG:teuthology.orchestra.run.smithi036:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-06-17T12:45:49.911 INFO:teuthology.orchestra.run.smithi036.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-06-17T12:45:49.911 INFO:teuthology.orchestra.run.smithi036.stdout: = sectsz=512 attr=2, projid32bit=1 2022-06-17T12:45:49.912 INFO:teuthology.orchestra.run.smithi036.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-06-17T12:45:49.912 INFO:teuthology.orchestra.run.smithi036.stdout: = reflink=1 2022-06-17T12:45:49.912 INFO:teuthology.orchestra.run.smithi036.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-06-17T12:45:49.913 INFO:teuthology.orchestra.run.smithi036.stdout: = sunit=0 swidth=0 blks 2022-06-17T12:45:49.913 INFO:teuthology.orchestra.run.smithi036.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-06-17T12:45:49.913 INFO:teuthology.orchestra.run.smithi036.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-06-17T12:45:49.913 INFO:teuthology.orchestra.run.smithi036.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-06-17T12:45:49.914 INFO:teuthology.orchestra.run.smithi036.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-06-17T12:45:49.914 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi036.front.sepia.ceph.com -o noatime 2022-06-17T12:45:49.914 DEBUG:teuthology.orchestra.run.smithi036:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2022-06-17T12:45:50.005 DEBUG:teuthology.orchestra.run.smithi036:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2022-06-17T12:45:50.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:45:50.019 INFO:teuthology.orchestra.run.smithi036.stderr:sudo: /sbin/restorecon: command not found 2022-06-17T12:45:50.020 DEBUG:teuthology.orchestra.run.smithi036:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2022-06-17T12:45:50.079 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-06-17T12:45:50.080 INFO:tasks.ceph:role: osd.3 2022-06-17T12:45:50.080 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi036.front.sepia.ceph.com 2022-06-17T12:45:50.081 DEBUG:teuthology.orchestra.run.smithi036:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-06-17T12:45:50.624 INFO:teuthology.orchestra.run.smithi036.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-06-17T12:45:50.625 INFO:teuthology.orchestra.run.smithi036.stdout: = sectsz=512 attr=2, projid32bit=1 2022-06-17T12:45:50.625 INFO:teuthology.orchestra.run.smithi036.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-06-17T12:45:50.625 INFO:teuthology.orchestra.run.smithi036.stdout: = reflink=1 2022-06-17T12:45:50.625 INFO:teuthology.orchestra.run.smithi036.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-06-17T12:45:50.626 INFO:teuthology.orchestra.run.smithi036.stdout: = sunit=0 swidth=0 blks 2022-06-17T12:45:50.626 INFO:teuthology.orchestra.run.smithi036.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-06-17T12:45:50.626 INFO:teuthology.orchestra.run.smithi036.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-06-17T12:45:50.626 INFO:teuthology.orchestra.run.smithi036.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-06-17T12:45:50.627 INFO:teuthology.orchestra.run.smithi036.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-06-17T12:45:50.627 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi036.front.sepia.ceph.com -o noatime 2022-06-17T12:45:50.627 DEBUG:teuthology.orchestra.run.smithi036:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2022-06-17T12:45:50.729 DEBUG:teuthology.orchestra.run.smithi036:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2022-06-17T12:45:50.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:45:50.747 INFO:teuthology.orchestra.run.smithi036.stderr:sudo: /sbin/restorecon: command not found 2022-06-17T12:45:50.747 DEBUG:teuthology.orchestra.run.smithi036:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-06-17T12:45:50.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:50.881+0000 7fe8e218dd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2022-06-17T12:45:50.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:50.881+0000 7fe8e218dd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2022-06-17T12:45:50.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:50.881+0000 7fe8e218dd80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-06-17T12:45:50.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:50.881+0000 7fe8e218dd80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-06-17T12:45:50.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:50.881+0000 7fe8e218dd80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-06-17T12:45:50.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:50.881+0000 7fe8e218dd80 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2022-06-17T12:45:50.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:50.901+0000 7fe8e218dd80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-06-17T12:45:53.327 DEBUG:teuthology.orchestra.run.smithi036:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2022-06-17T12:45:53.360 DEBUG:teuthology.orchestra.run.smithi036:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-06-17T12:45:53.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:53.465+0000 7f11b4a1ed80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2022-06-17T12:45:53.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:53.465+0000 7f11b4a1ed80 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2022-06-17T12:45:53.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:53.465+0000 7f11b4a1ed80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-06-17T12:45:53.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:53.465+0000 7f11b4a1ed80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-06-17T12:45:53.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:53.465+0000 7f11b4a1ed80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-06-17T12:45:53.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:53.465+0000 7f11b4a1ed80 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2022-06-17T12:45:53.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:53.485+0000 7f11b4a1ed80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-06-17T12:45:55.917 DEBUG:teuthology.orchestra.run.smithi036:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2022-06-17T12:45:55.937 DEBUG:teuthology.orchestra.run.smithi036:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-06-17T12:45:56.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:56.045+0000 7fce9e250d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2022-06-17T12:45:56.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:56.045+0000 7fce9e250d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2022-06-17T12:45:56.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:56.045+0000 7fce9e250d80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-06-17T12:45:56.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:56.045+0000 7fce9e250d80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-06-17T12:45:56.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:56.045+0000 7fce9e250d80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-06-17T12:45:56.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:56.049+0000 7fce9e250d80 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2022-06-17T12:45:56.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:56.065+0000 7fce9e250d80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-06-17T12:45:58.506 DEBUG:teuthology.orchestra.run.smithi036:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2022-06-17T12:45:58.540 DEBUG:teuthology.orchestra.run.smithi036:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-06-17T12:45:58.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:58.657+0000 7f6c5835bd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2022-06-17T12:45:58.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:58.657+0000 7f6c5835bd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2022-06-17T12:45:58.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:58.657+0000 7f6c5835bd80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-06-17T12:45:58.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:58.657+0000 7f6c5835bd80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-06-17T12:45:58.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:58.657+0000 7f6c5835bd80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-06-17T12:45:58.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:58.657+0000 7f6c5835bd80 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2022-06-17T12:45:58.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:45:58.677+0000 7f6c5835bd80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-06-17T12:46:01.109 DEBUG:teuthology.orchestra.run.smithi036:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2022-06-17T12:46:01.173 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2022-06-17T12:46:01.198 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-06-17T12:46:01.198 INFO:tasks.ceph:role: osd.4 2022-06-17T12:46:01.198 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi070.front.sepia.ceph.com 2022-06-17T12:46:01.199 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-06-17T12:46:01.291 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-06-17T12:46:01.291 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2022-06-17T12:46:01.291 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-06-17T12:46:01.292 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 2022-06-17T12:46:01.292 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-06-17T12:46:01.292 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2022-06-17T12:46:01.292 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-06-17T12:46:01.293 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-06-17T12:46:01.293 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-06-17T12:46:01.293 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-06-17T12:46:01.293 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2022-06-17T12:46:01.311 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2022-06-17T12:46:01.360 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2022-06-17T12:46:01.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:46:01.374 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: /sbin/restorecon: command not found 2022-06-17T12:46:01.374 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2022-06-17T12:46:01.444 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-06-17T12:46:01.445 INFO:tasks.ceph:role: osd.5 2022-06-17T12:46:01.446 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi070.front.sepia.ceph.com 2022-06-17T12:46:01.447 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-06-17T12:46:01.547 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-06-17T12:46:01.548 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2022-06-17T12:46:01.548 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-06-17T12:46:01.548 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 2022-06-17T12:46:01.549 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-06-17T12:46:01.550 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2022-06-17T12:46:01.551 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-06-17T12:46:01.551 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-06-17T12:46:01.551 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-06-17T12:46:01.552 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-06-17T12:46:01.552 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2022-06-17T12:46:01.553 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2022-06-17T12:46:01.583 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2022-06-17T12:46:01.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:46:01.646 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: /sbin/restorecon: command not found 2022-06-17T12:46:01.647 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2022-06-17T12:46:01.680 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-06-17T12:46:01.681 INFO:tasks.ceph:role: osd.6 2022-06-17T12:46:01.681 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi070.front.sepia.ceph.com 2022-06-17T12:46:01.682 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-06-17T12:46:01.766 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-06-17T12:46:01.767 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2022-06-17T12:46:01.767 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-06-17T12:46:01.774 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 2022-06-17T12:46:01.775 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-06-17T12:46:01.775 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2022-06-17T12:46:01.776 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-06-17T12:46:01.776 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-06-17T12:46:01.776 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-06-17T12:46:01.777 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-06-17T12:46:01.779 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2022-06-17T12:46:01.780 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2022-06-17T12:46:01.813 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2022-06-17T12:46:01.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:46:01.873 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: /sbin/restorecon: command not found 2022-06-17T12:46:01.878 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2022-06-17T12:46:01.901 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-06-17T12:46:01.902 INFO:tasks.ceph:role: osd.7 2022-06-17T12:46:01.903 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi070.front.sepia.ceph.com 2022-06-17T12:46:01.914 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-06-17T12:46:01.988 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-06-17T12:46:01.988 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2022-06-17T12:46:01.989 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-06-17T12:46:01.989 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 2022-06-17T12:46:01.990 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-06-17T12:46:01.990 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2022-06-17T12:46:01.990 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-06-17T12:46:01.990 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-06-17T12:46:01.991 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-06-17T12:46:01.991 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-06-17T12:46:01.993 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2022-06-17T12:46:01.993 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2022-06-17T12:46:02.015 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2022-06-17T12:46:02.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:46:02.084 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: /sbin/restorecon: command not found 2022-06-17T12:46:02.085 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-06-17T12:46:02.163 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:02.179+0000 7f09d720ad80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2022-06-17T12:46:02.164 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:02.179+0000 7f09d720ad80 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2022-06-17T12:46:02.164 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:02.179+0000 7f09d720ad80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-06-17T12:46:02.166 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:02.179+0000 7f09d720ad80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-06-17T12:46:02.166 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:02.179+0000 7f09d720ad80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-06-17T12:46:02.167 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:02.179+0000 7f09d720ad80 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2022-06-17T12:46:02.173 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:02.195+0000 7f09d720ad80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-06-17T12:46:04.626 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2022-06-17T12:46:04.658 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-06-17T12:46:04.762 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:04.779+0000 7f6a810e5d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2022-06-17T12:46:04.763 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:04.779+0000 7f6a810e5d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2022-06-17T12:46:04.776 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:04.779+0000 7f6a810e5d80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-06-17T12:46:04.776 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:04.779+0000 7f6a810e5d80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-06-17T12:46:04.777 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:04.779+0000 7f6a810e5d80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-06-17T12:46:04.778 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:04.779+0000 7f6a810e5d80 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2022-06-17T12:46:04.779 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:04.795+0000 7f6a810e5d80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-06-17T12:46:07.260 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2022-06-17T12:46:07.338 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-06-17T12:46:07.440 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:07.451+0000 7fbd3385fd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2022-06-17T12:46:07.441 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:07.451+0000 7fbd3385fd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2022-06-17T12:46:07.441 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:07.451+0000 7fbd3385fd80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-06-17T12:46:07.441 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:07.451+0000 7fbd3385fd80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-06-17T12:46:07.441 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:07.451+0000 7fbd3385fd80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-06-17T12:46:07.459 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:07.451+0000 7fbd3385fd80 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2022-06-17T12:46:07.468 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:07.467+0000 7fbd3385fd80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-06-17T12:46:09.908 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2022-06-17T12:46:09.976 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-06-17T12:46:10.017 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:10.039+0000 7f3237ef4d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2022-06-17T12:46:10.018 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:10.039+0000 7f3237ef4d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2022-06-17T12:46:10.018 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:10.039+0000 7f3237ef4d80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-06-17T12:46:10.018 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:10.039+0000 7f3237ef4d80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-06-17T12:46:10.019 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:10.039+0000 7f3237ef4d80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-06-17T12:46:10.019 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:10.039+0000 7f3237ef4d80 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2022-06-17T12:46:10.031 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:10.055+0000 7f3237ef4d80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-06-17T12:46:12.500 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2022-06-17T12:46:12.563 INFO:tasks.ceph:Reading keys from all nodes... 2022-06-17T12:46:12.564 DEBUG:teuthology.orchestra.run.smithi036:> mktemp 2022-06-17T12:46:12.576 INFO:teuthology.orchestra.run.smithi036.stdout:/tmp/tmp.ZSMI96eCtO 2022-06-17T12:46:12.610 DEBUG:teuthology.orchestra.run.smithi036:> sudo cp /var/lib/ceph/mgr/ceph-y/keyring /tmp/tmp.ZSMI96eCtO 2022-06-17T12:46:12.646 DEBUG:teuthology.orchestra.run.smithi036:> sudo chmod 0666 /tmp/tmp.ZSMI96eCtO 2022-06-17T12:46:12.740 DEBUG:teuthology.orchestra.remote:smithi036:/tmp/tmp.ZSMI96eCtO is 56B 2022-06-17T12:46:12.817 DEBUG:teuthology.orchestra.run.smithi036:> rm -fr /tmp/tmp.ZSMI96eCtO 2022-06-17T12:46:12.830 DEBUG:teuthology.orchestra.run.smithi036:> mktemp 2022-06-17T12:46:12.892 INFO:teuthology.orchestra.run.smithi036.stdout:/tmp/tmp.DG5MG363AL 2022-06-17T12:46:12.901 DEBUG:teuthology.orchestra.run.smithi036:> sudo cp /var/lib/ceph/mds/ceph-a/keyring /tmp/tmp.DG5MG363AL 2022-06-17T12:46:12.903 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:46:12.994 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:46:13.025 DEBUG:teuthology.orchestra.run.smithi036:> sudo chmod 0666 /tmp/tmp.DG5MG363AL 2022-06-17T12:46:13.050 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:46:13.095 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:46:13.135 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:46:13.218 DEBUG:teuthology.orchestra.remote:smithi036:/tmp/tmp.DG5MG363AL is 56B 2022-06-17T12:46:13.351 DEBUG:teuthology.orchestra.run.smithi036:> rm -fr /tmp/tmp.DG5MG363AL 2022-06-17T12:46:13.364 DEBUG:teuthology.orchestra.run.smithi036:> mktemp 2022-06-17T12:46:13.467 INFO:teuthology.orchestra.run.smithi036.stdout:/tmp/tmp.cfFtTk13gu 2022-06-17T12:46:13.468 DEBUG:teuthology.orchestra.run.smithi036:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.cfFtTk13gu 2022-06-17T12:46:13.492 DEBUG:teuthology.orchestra.run.smithi036:> sudo chmod 0666 /tmp/tmp.cfFtTk13gu 2022-06-17T12:46:13.624 DEBUG:teuthology.orchestra.remote:smithi036:/tmp/tmp.cfFtTk13gu is 56B 2022-06-17T12:46:13.780 DEBUG:teuthology.orchestra.run.smithi036:> rm -fr /tmp/tmp.cfFtTk13gu 2022-06-17T12:46:13.868 DEBUG:teuthology.orchestra.run.smithi036:> mktemp 2022-06-17T12:46:13.894 INFO:teuthology.orchestra.run.smithi036.stdout:/tmp/tmp.iL7hzWZvz5 2022-06-17T12:46:13.895 DEBUG:teuthology.orchestra.run.smithi036:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.iL7hzWZvz5 2022-06-17T12:46:13.937 DEBUG:teuthology.orchestra.run.smithi036:> sudo chmod 0666 /tmp/tmp.iL7hzWZvz5 2022-06-17T12:46:14.071 DEBUG:teuthology.orchestra.remote:smithi036:/tmp/tmp.iL7hzWZvz5 is 56B 2022-06-17T12:46:14.162 DEBUG:teuthology.orchestra.run.smithi036:> rm -fr /tmp/tmp.iL7hzWZvz5 2022-06-17T12:46:14.183 DEBUG:teuthology.orchestra.run.smithi036:> mktemp 2022-06-17T12:46:14.233 INFO:teuthology.orchestra.run.smithi036.stdout:/tmp/tmp.gAg30407Ry 2022-06-17T12:46:14.234 DEBUG:teuthology.orchestra.run.smithi036:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.gAg30407Ry 2022-06-17T12:46:14.296 DEBUG:teuthology.orchestra.run.smithi036:> sudo chmod 0666 /tmp/tmp.gAg30407Ry 2022-06-17T12:46:14.425 DEBUG:teuthology.orchestra.remote:smithi036:/tmp/tmp.gAg30407Ry is 56B 2022-06-17T12:46:14.477 DEBUG:teuthology.orchestra.run.smithi036:> rm -fr /tmp/tmp.gAg30407Ry 2022-06-17T12:46:14.502 DEBUG:teuthology.orchestra.run.smithi036:> mktemp 2022-06-17T12:46:14.570 INFO:teuthology.orchestra.run.smithi036.stdout:/tmp/tmp.JORIUJ4Gwt 2022-06-17T12:46:14.572 DEBUG:teuthology.orchestra.run.smithi036:> sudo cp /var/lib/ceph/osd/ceph-3/keyring /tmp/tmp.JORIUJ4Gwt 2022-06-17T12:46:14.593 DEBUG:teuthology.orchestra.run.smithi036:> sudo chmod 0666 /tmp/tmp.JORIUJ4Gwt 2022-06-17T12:46:14.713 DEBUG:teuthology.orchestra.remote:smithi036:/tmp/tmp.JORIUJ4Gwt is 56B 2022-06-17T12:46:14.861 DEBUG:teuthology.orchestra.run.smithi036:> rm -fr /tmp/tmp.JORIUJ4Gwt 2022-06-17T12:46:14.874 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2022-06-17T12:46:14.893 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.60BbTyLPAZ 2022-06-17T12:46:14.894 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.60BbTyLPAZ 2022-06-17T12:46:14.934 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.60BbTyLPAZ 2022-06-17T12:46:15.101 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.60BbTyLPAZ is 56B 2022-06-17T12:46:15.157 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.60BbTyLPAZ 2022-06-17T12:46:15.172 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2022-06-17T12:46:15.237 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.agBIrcDulo 2022-06-17T12:46:15.238 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /var/lib/ceph/mds/ceph-b/keyring /tmp/tmp.agBIrcDulo 2022-06-17T12:46:15.274 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.agBIrcDulo 2022-06-17T12:46:15.365 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.agBIrcDulo is 56B 2022-06-17T12:46:15.427 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.agBIrcDulo 2022-06-17T12:46:15.447 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2022-06-17T12:46:15.492 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.kT0hESu9eW 2022-06-17T12:46:15.493 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /var/lib/ceph/osd/ceph-4/keyring /tmp/tmp.kT0hESu9eW 2022-06-17T12:46:15.517 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.kT0hESu9eW 2022-06-17T12:46:15.747 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.kT0hESu9eW is 56B 2022-06-17T12:46:15.910 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.kT0hESu9eW 2022-06-17T12:46:15.921 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2022-06-17T12:46:16.006 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.hTWfkKAQ0B 2022-06-17T12:46:16.007 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /var/lib/ceph/osd/ceph-5/keyring /tmp/tmp.hTWfkKAQ0B 2022-06-17T12:46:16.088 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.hTWfkKAQ0B 2022-06-17T12:46:16.345 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.hTWfkKAQ0B is 56B 2022-06-17T12:46:16.396 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.hTWfkKAQ0B 2022-06-17T12:46:16.416 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2022-06-17T12:46:16.472 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.nOEbw2nPjh 2022-06-17T12:46:16.473 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /var/lib/ceph/osd/ceph-6/keyring /tmp/tmp.nOEbw2nPjh 2022-06-17T12:46:16.536 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.nOEbw2nPjh 2022-06-17T12:46:16.714 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.nOEbw2nPjh is 56B 2022-06-17T12:46:16.807 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.nOEbw2nPjh 2022-06-17T12:46:16.849 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2022-06-17T12:46:16.892 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.rb8QilUvh4 2022-06-17T12:46:16.892 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /var/lib/ceph/osd/ceph-7/keyring /tmp/tmp.rb8QilUvh4 2022-06-17T12:46:16.915 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.rb8QilUvh4 2022-06-17T12:46:17.024 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.rb8QilUvh4 is 56B 2022-06-17T12:46:17.134 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.rb8QilUvh4 2022-06-17T12:46:17.250 DEBUG:teuthology.orchestra.remote:smithi107:/etc/ceph/ceph.client.0.keyring is 59B 2022-06-17T12:46:17.407 DEBUG:teuthology.orchestra.remote:smithi136:/etc/ceph/ceph.client.1.keyring is 59B 2022-06-17T12:46:17.549 DEBUG:teuthology.orchestra.remote:smithi143:/etc/ceph/ceph.client.2.keyring is 59B 2022-06-17T12:46:17.630 INFO:tasks.ceph:Adding keys to all mons... 2022-06-17T12:46:17.652 DEBUG:teuthology.orchestra.run.smithi036:> sudo tee -a /etc/ceph/ceph.keyring 2022-06-17T12:46:17.669 DEBUG:teuthology.orchestra.run.smithi070:> sudo tee -a /etc/ceph/ceph.keyring 2022-06-17T12:46:17.745 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr.y] 2022-06-17T12:46:17.746 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQB6d6xiHXghMxAAzpsVRCbLWHWZPtrPk1jFlA== 2022-06-17T12:46:17.747 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.a] 2022-06-17T12:46:17.747 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQB7d6xiyaUgARAAxp7bzOl4uVo2EsaoRBeMRw== 2022-06-17T12:46:17.748 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.0] 2022-06-17T12:46:17.748 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQB+d6xibhXGNBAA6AuUT9ITWAZjDFpNsgA4Wg== 2022-06-17T12:46:17.748 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.1] 2022-06-17T12:46:17.749 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCBd6xiOIPvGxAADkrBoJCy9x91zan6TWU0tQ== 2022-06-17T12:46:17.749 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.2] 2022-06-17T12:46:17.750 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCEd6xiP4/9AhAAaP3cWXV6vbfGBPfqSLKqUg== 2022-06-17T12:46:17.751 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.3] 2022-06-17T12:46:17.751 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCGd6xi+LZgJxAAJjGwP5qW6TFDZfR+2rz6ig== 2022-06-17T12:46:17.752 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr.x] 2022-06-17T12:46:17.752 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQB6d6xiyH9uORAATJb0chHE1CH0X02w7edVtw== 2022-06-17T12:46:17.753 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.b] 2022-06-17T12:46:17.753 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQB7d6xifyxLCRAABI389jmz1zboeDanzraT8Q== 2022-06-17T12:46:17.753 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.4] 2022-06-17T12:46:17.754 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCKd6xikvrTChAAXIyWSETswDv+tb3zsGyYpA== 2022-06-17T12:46:17.754 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.5] 2022-06-17T12:46:17.754 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCMd6xiqr2QLhAArhimHuNsVWrDGGreFYxj6A== 2022-06-17T12:46:17.755 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.6] 2022-06-17T12:46:17.755 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCPd6xiUXcXGxAAdrYz8txdBWMIqF5DF4Aimw== 2022-06-17T12:46:17.756 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.7] 2022-06-17T12:46:17.756 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCSd6xiBJ2IAhAAUkNfgmlxxWY7+VouPLAvyw== 2022-06-17T12:46:17.757 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2022-06-17T12:46:17.757 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQB7d6xiXO3QFBAAsSLtQ0yKKZxhqUT/eYKdtg== 2022-06-17T12:46:17.757 INFO:teuthology.orchestra.run.smithi070.stdout:[client.1] 2022-06-17T12:46:17.758 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQB7d6xi51KdGBAAi7fGZ+Qepg6j3DAhlzoWxw== 2022-06-17T12:46:17.780 INFO:teuthology.orchestra.run.smithi070.stdout:[client.2] 2022-06-17T12:46:17.781 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQB7d6xiug2hHBAAqHpLgwMdthSWtJ8Vwu+m1Q== 2022-06-17T12:46:17.781 INFO:teuthology.orchestra.run.smithi036.stdout:[mgr.y] 2022-06-17T12:46:17.788 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQB6d6xiHXghMxAAzpsVRCbLWHWZPtrPk1jFlA== 2022-06-17T12:46:17.788 INFO:teuthology.orchestra.run.smithi036.stdout:[mds.a] 2022-06-17T12:46:17.789 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQB7d6xiyaUgARAAxp7bzOl4uVo2EsaoRBeMRw== 2022-06-17T12:46:17.789 INFO:teuthology.orchestra.run.smithi036.stdout:[osd.0] 2022-06-17T12:46:17.790 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQB+d6xibhXGNBAA6AuUT9ITWAZjDFpNsgA4Wg== 2022-06-17T12:46:17.790 INFO:teuthology.orchestra.run.smithi036.stdout:[osd.1] 2022-06-17T12:46:17.790 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQCBd6xiOIPvGxAADkrBoJCy9x91zan6TWU0tQ== 2022-06-17T12:46:17.795 INFO:teuthology.orchestra.run.smithi036.stdout:[osd.2] 2022-06-17T12:46:17.796 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQCEd6xiP4/9AhAAaP3cWXV6vbfGBPfqSLKqUg== 2022-06-17T12:46:17.796 INFO:teuthology.orchestra.run.smithi036.stdout:[osd.3] 2022-06-17T12:46:17.797 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQCGd6xi+LZgJxAAJjGwP5qW6TFDZfR+2rz6ig== 2022-06-17T12:46:17.798 INFO:teuthology.orchestra.run.smithi036.stdout:[mgr.x] 2022-06-17T12:46:17.798 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQB6d6xiyH9uORAATJb0chHE1CH0X02w7edVtw== 2022-06-17T12:46:17.799 INFO:teuthology.orchestra.run.smithi036.stdout:[mds.b] 2022-06-17T12:46:17.799 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQB7d6xifyxLCRAABI389jmz1zboeDanzraT8Q== 2022-06-17T12:46:17.800 INFO:teuthology.orchestra.run.smithi036.stdout:[osd.4] 2022-06-17T12:46:17.800 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQCKd6xikvrTChAAXIyWSETswDv+tb3zsGyYpA== 2022-06-17T12:46:17.800 INFO:teuthology.orchestra.run.smithi036.stdout:[osd.5] 2022-06-17T12:46:17.801 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQCMd6xiqr2QLhAArhimHuNsVWrDGGreFYxj6A== 2022-06-17T12:46:17.801 INFO:teuthology.orchestra.run.smithi036.stdout:[osd.6] 2022-06-17T12:46:17.802 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQCPd6xiUXcXGxAAdrYz8txdBWMIqF5DF4Aimw== 2022-06-17T12:46:17.802 INFO:teuthology.orchestra.run.smithi036.stdout:[osd.7] 2022-06-17T12:46:17.803 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQCSd6xiBJ2IAhAAUkNfgmlxxWY7+VouPLAvyw== 2022-06-17T12:46:17.803 INFO:teuthology.orchestra.run.smithi036.stdout:[client.0] 2022-06-17T12:46:17.804 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQB7d6xiXO3QFBAAsSLtQ0yKKZxhqUT/eYKdtg== 2022-06-17T12:46:17.804 INFO:teuthology.orchestra.run.smithi036.stdout:[client.1] 2022-06-17T12:46:17.804 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQB7d6xi51KdGBAAi7fGZ+Qepg6j3DAhlzoWxw== 2022-06-17T12:46:17.805 INFO:teuthology.orchestra.run.smithi036.stdout:[client.2] 2022-06-17T12:46:17.805 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQB7d6xiug2hHBAAqHpLgwMdthSWtJ8Vwu+m1Q== 2022-06-17T12:46:17.814 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-06-17T12:46:17.827 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-06-17T12:46:17.881 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-06-17T12:46:17.935 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-06-17T12:46:17.997 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-17T12:46:18.023 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-17T12:46:18.075 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-17T12:46:18.113 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-17T12:46:18.171 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-17T12:46:18.205 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-17T12:46:18.254 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-17T12:46:18.280 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-17T12:46:18.348 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-06-17T12:46:18.415 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-06-17T12:46:18.506 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-06-17T12:46:18.514 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-06-17T12:46:18.622 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-17T12:46:18.654 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-17T12:46:18.732 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-17T12:46:18.829 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-17T12:46:18.893 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-17T12:46:18.902 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-17T12:46:18.962 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-17T12:46:19.038 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-17T12:46:19.156 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-06-17T12:46:19.185 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-06-17T12:46:19.310 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-06-17T12:46:19.376 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-06-17T12:46:19.449 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-06-17T12:46:19.523 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-06-17T12:46:19.601 INFO:tasks.ceph:Running mkfs on mon nodes... 2022-06-17T12:46:19.601 DEBUG:teuthology.orchestra.run.smithi036:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2022-06-17T12:46:19.667 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-06-17T12:46:19.759 DEBUG:teuthology.orchestra.run.smithi036:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2022-06-17T12:46:19.787 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2022-06-17T12:46:19.828 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-06-17T12:46:19.940 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2022-06-17T12:46:19.975 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2022-06-17T12:46:20.053 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-06-17T12:46:20.175 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2022-06-17T12:46:20.194 DEBUG:teuthology.orchestra.run.smithi036:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-06-17T12:46:20.197 DEBUG:teuthology.orchestra.run.smithi070:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-06-17T12:46:20.246 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2022-06-17T12:46:20.252 INFO:tasks.ceph.mon.a:Restarting daemon 2022-06-17T12:46:20.253 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2022-06-17T12:46:20.256 INFO:tasks.ceph.mon.a:Started 2022-06-17T12:46:20.256 INFO:tasks.ceph.mon.b:Restarting daemon 2022-06-17T12:46:20.257 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2022-06-17T12:46:20.260 INFO:tasks.ceph.mon.b:Started 2022-06-17T12:46:20.260 INFO:tasks.ceph.mon.c:Restarting daemon 2022-06-17T12:46:20.261 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2022-06-17T12:46:20.277 INFO:tasks.ceph.mon.c:Started 2022-06-17T12:46:20.279 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2022-06-17T12:46:20.279 INFO:tasks.ceph.mgr.y:Restarting daemon 2022-06-17T12:46:20.280 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2022-06-17T12:46:20.286 INFO:tasks.ceph.mgr.y:Started 2022-06-17T12:46:20.286 INFO:tasks.ceph.mgr.x:Restarting daemon 2022-06-17T12:46:20.287 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2022-06-17T12:46:20.315 INFO:tasks.ceph.mgr.x:Started 2022-06-17T12:46:20.316 INFO:tasks.ceph:Setting crush tunables to default 2022-06-17T12:46:20.320 DEBUG:teuthology.orchestra.run.smithi036:> sudo ceph --cluster ceph osd crush tunables default 2022-06-17T12:46:20.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.441+0000 7f36d5798700 1 Processor -- start 2022-06-17T12:46:20.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.441+0000 7f36d5798700 1 -- start start 2022-06-17T12:46:20.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.441+0000 7f36d5798700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36d00b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:20.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.441+0000 7f36d5798700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d01429c0 0x7f36d013f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:20.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.441+0000 7f36d5798700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36d013fb70 con 0x7f36d0139160 2022-06-17T12:46:20.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.441+0000 7f36d5798700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36d00aff70 con 0x7f36d00b6440 2022-06-17T12:46:20.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.441+0000 7f36d5798700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d013fe10 con 0x7f36d01429c0 2022-06-17T12:46:20.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.441+0000 7f36cf7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f36d0139160 0x7f36d005f770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:48108/0 (socket says 172.21.15.36:48108) 2022-06-17T12:46:20.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.441+0000 7f36cf7fe700 1 -- 172.21.15.36:0/290302583 learned_addr learned my addr 172.21.15.36:0/290302583 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:46:20.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.441+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36d00b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:20.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.441+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d01429c0 0x7f36d013f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:20.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.441+0000 7f36ceffd700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 msgr2=0x7f36d00b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-06-17T12:46:20.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.441+0000 7f36ceffd700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 msgr2=0x7f36d00b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-17T12:46:20.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.441+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36d00b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-17T12:46:20.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.441+0000 7f36ce7fc700 1 -- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d01429c0 msgr2=0x7f36d013f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-06-17T12:46:20.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.441+0000 7f36ce7fc700 1 -- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d01429c0 msgr2=0x7f36d013f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-17T12:46:20.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.441+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d01429c0 0x7f36d013f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-17T12:46:20.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.441+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36d00b6820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-17T12:46:20.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.441+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d01429c0 0x7f36d013f3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-17T12:46:20.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.641+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d01429c0 0x7f36d013f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:20.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.641+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36d00b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:20.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.641+0000 7f36ce7fc700 1 -- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d01429c0 msgr2=0x7f36d013f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-06-17T12:46:20.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.641+0000 7f36ce7fc700 1 -- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d01429c0 msgr2=0x7f36d013f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-17T12:46:20.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.641+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d01429c0 0x7f36d013f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-17T12:46:20.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.641+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d01429c0 0x7f36d013f3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2022-06-17T12:46:20.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.641+0000 7f36ceffd700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 msgr2=0x7f36d00b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-06-17T12:46:20.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.641+0000 7f36ceffd700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 msgr2=0x7f36d00b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-17T12:46:20.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.641+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36d00b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-17T12:46:20.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:20.641+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36d00b6820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2022-06-17T12:46:21.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:21.045+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d01429c0 0x7f36d013f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:21.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:21.045+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36d00b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:21.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:21.045+0000 7f36ce7fc700 1 -- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d01429c0 msgr2=0x7f36d013f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-06-17T12:46:21.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:21.045+0000 7f36ce7fc700 1 -- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d01429c0 msgr2=0x7f36d013f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-17T12:46:21.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:21.045+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d01429c0 0x7f36d013f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-17T12:46:21.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:21.045+0000 7f36ceffd700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 msgr2=0x7f36d00b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-06-17T12:46:21.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:21.045+0000 7f36ceffd700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 msgr2=0x7f36d00b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-17T12:46:21.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:21.045+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36d00b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-17T12:46:21.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:21.045+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d01429c0 0x7f36d013f3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2022-06-17T12:46:21.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:21.045+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36d00b6820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2022-06-17T12:46:21.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:21.845+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d01429c0 0x7f36d013f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:21.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:21.845+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36d00b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:21.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:21.845+0000 7f36ce7fc700 1 -- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d01429c0 msgr2=0x7f36d013f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-06-17T12:46:21.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:21.845+0000 7f36ce7fc700 1 -- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d01429c0 msgr2=0x7f36d013f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-17T12:46:21.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:21.845+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d01429c0 0x7f36d013f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-17T12:46:21.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:21.845+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d01429c0 0x7f36d013f3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2022-06-17T12:46:21.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:21.845+0000 7f36ceffd700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 msgr2=0x7f36d00b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-06-17T12:46:21.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:21.845+0000 7f36ceffd700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 msgr2=0x7f36d00b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-17T12:46:21.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:21.845+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36d00b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-17T12:46:21.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:21.845+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36d00b6820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2022-06-17T12:46:23.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:23.441+0000 7f36ccff9700 1 -- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d01429c0 msgr2=0x7f36d013f3c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-06-17T12:46:23.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:23.441+0000 7f36ccff9700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d01429c0 0x7f36d013f3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:23.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:23.441+0000 7f36ccff9700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 msgr2=0x7f36d00b6820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-06-17T12:46:23.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:23.441+0000 7f36ccff9700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36d00b6820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:23.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:23.441+0000 7f36ccff9700 1 -- 172.21.15.36:0/290302583 >> v1:172.21.15.36:6789/0 conn(0x7f36d0139160 legacy=0x7f36d005f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:23.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:23.441+0000 7f36ccff9700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b40033b0 0x7f36b4005830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:23.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:23.441+0000 7f36ccff9700 1 -- 172.21.15.36:0/290302583 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36d01400b0 con 0x7f36b4005e30 2022-06-17T12:46:23.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:23.441+0000 7f36ccff9700 1 -- 172.21.15.36:0/290302583 --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36d00aff70 con 0x7f36b40033b0 2022-06-17T12:46:23.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:23.441+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b40033b0 0x7f36b4005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:23.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:23.441+0000 7f36ceffd700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b40033b0 msgr2=0x7f36b4005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-06-17T12:46:23.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:23.441+0000 7f36ceffd700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b40033b0 msgr2=0x7f36b4005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-17T12:46:23.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:23.441+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b40033b0 0x7f36b4005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-17T12:46:23.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:23.441+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b40033b0 0x7f36b4005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-17T12:46:23.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:23.641+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b40033b0 0x7f36b4005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:23.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:23.641+0000 7f36ceffd700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b40033b0 msgr2=0x7f36b4005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-06-17T12:46:23.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:23.641+0000 7f36ceffd700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b40033b0 msgr2=0x7f36b4005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-17T12:46:23.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:23.641+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b40033b0 0x7f36b4005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-17T12:46:23.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:23.641+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b40033b0 0x7f36b4005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2022-06-17T12:46:24.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:24.045+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b40033b0 0x7f36b4005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:24.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:24.045+0000 7f36ceffd700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b40033b0 msgr2=0x7f36b4005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-06-17T12:46:24.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:24.045+0000 7f36ceffd700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b40033b0 msgr2=0x7f36b4005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-17T12:46:24.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:24.045+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b40033b0 0x7f36b4005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-17T12:46:24.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:24.045+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b40033b0 0x7f36b4005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2022-06-17T12:46:24.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:24.845+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b40033b0 0x7f36b4005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:24.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:24.845+0000 7f36ceffd700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b40033b0 msgr2=0x7f36b4005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-06-17T12:46:24.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:24.845+0000 7f36ceffd700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b40033b0 msgr2=0x7f36b4005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-17T12:46:24.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:24.845+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b40033b0 0x7f36b4005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-17T12:46:24.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:24.845+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b40033b0 0x7f36b4005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2022-06-17T12:46:26.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36ccff9700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b40033b0 msgr2=0x7f36b4005830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-06-17T12:46:26.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36ccff9700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b40033b0 0x7f36b4005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:26.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36ccff9700 1 -- 172.21.15.36:0/290302583 >> v1:172.21.15.70:6789/0 conn(0x7f36b4005e30 legacy=0x7f36b40082d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:26.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36cf7fe700 1 -- 172.21.15.36:0/290302583 reap_dead start 2022-06-17T12:46:26.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36ccff9700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36b4009c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:26.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36ccff9700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 0x7f36b4005830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:26.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36ccff9700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b400a690 0x7f36b40082d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:26.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36ccff9700 1 -- 172.21.15.36:0/290302583 --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36d00aff70 con 0x7f36b400a690 2022-06-17T12:46:26.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36ccff9700 1 -- 172.21.15.36:0/290302583 --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36d013fe10 con 0x7f36d00b6440 2022-06-17T12:46:26.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36ccff9700 1 -- 172.21.15.36:0/290302583 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b4008a00 con 0x7f36b400a150 2022-06-17T12:46:26.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36b4009c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:26.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 0x7f36b4005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:26.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36cf7fe700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b400a690 0x7f36b40082d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:26.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36ceffd700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 msgr2=0x7f36b4009c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-06-17T12:46:26.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36ceffd700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 msgr2=0x7f36b4009c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-17T12:46:26.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36b4009c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-17T12:46:26.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36b4009c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-17T12:46:26.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36ce7fc700 1 -- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 msgr2=0x7f36b4005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-06-17T12:46:26.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36ce7fc700 1 -- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 msgr2=0x7f36b4005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-17T12:46:26.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 0x7f36b4005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-17T12:46:26.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36cf7fe700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b400a690 msgr2=0x7f36b40082d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-06-17T12:46:26.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 0x7f36b4005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-17T12:46:26.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36cf7fe700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b400a690 msgr2=0x7f36b40082d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-17T12:46:26.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36cf7fe700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b400a690 0x7f36b40082d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-17T12:46:26.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.441+0000 7f36cf7fe700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b400a690 0x7f36b40082d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-17T12:46:26.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.641+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36b4009c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:26.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.641+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 0x7f36b4005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:26.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.641+0000 7f36cf7fe700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b400a690 0x7f36b40082d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:26.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.645+0000 7f36ceffd700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 msgr2=0x7f36b4009c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-06-17T12:46:26.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.645+0000 7f36ceffd700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 msgr2=0x7f36b4009c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-17T12:46:26.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.645+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36b4009c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-17T12:46:26.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.645+0000 7f36ce7fc700 1 -- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 msgr2=0x7f36b4005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-06-17T12:46:26.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.645+0000 7f36ce7fc700 1 -- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 msgr2=0x7f36b4005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-17T12:46:26.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.645+0000 7f36cf7fe700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b400a690 msgr2=0x7f36b40082d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-06-17T12:46:26.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.645+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 0x7f36b4005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-17T12:46:26.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.645+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36b4009c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2022-06-17T12:46:26.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.645+0000 7f36cf7fe700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b400a690 msgr2=0x7f36b40082d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-17T12:46:26.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.645+0000 7f36cf7fe700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b400a690 0x7f36b40082d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-17T12:46:26.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.645+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 0x7f36b4005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2022-06-17T12:46:26.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:26.645+0000 7f36cf7fe700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b400a690 0x7f36b40082d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2022-06-17T12:46:27.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.045+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 0x7f36b4005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:27.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.045+0000 7f36cf7fe700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b400a690 0x7f36b40082d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:27.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.045+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36b4009c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:27.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.045+0000 7f36ce7fc700 1 -- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 msgr2=0x7f36b4005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-06-17T12:46:27.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.045+0000 7f36ce7fc700 1 -- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 msgr2=0x7f36b4005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-17T12:46:27.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.045+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 0x7f36b4005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-17T12:46:27.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.045+0000 7f36cf7fe700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b400a690 msgr2=0x7f36b40082d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-06-17T12:46:27.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.045+0000 7f36ceffd700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 msgr2=0x7f36b4009c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-06-17T12:46:27.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.045+0000 7f36cf7fe700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b400a690 msgr2=0x7f36b40082d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-17T12:46:27.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.045+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 0x7f36b4005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2022-06-17T12:46:27.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.045+0000 7f36ceffd700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 msgr2=0x7f36b4009c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-17T12:46:27.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.045+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36b4009c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-17T12:46:27.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.045+0000 7f36cf7fe700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b400a690 0x7f36b40082d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-17T12:46:27.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.045+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36b4009c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2022-06-17T12:46:27.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.045+0000 7f36cf7fe700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b400a690 0x7f36b40082d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2022-06-17T12:46:27.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.845+0000 7f36cf7fe700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b400a690 0x7f36b40082d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:27.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.845+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 0x7f36b4005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:27.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.845+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36b4009c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:27.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.845+0000 7f36ce7fc700 1 -- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 msgr2=0x7f36b4005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-06-17T12:46:27.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.845+0000 7f36ce7fc700 1 -- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 msgr2=0x7f36b4005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-17T12:46:27.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.845+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 0x7f36b4005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-17T12:46:27.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.845+0000 7f36cf7fe700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b400a690 msgr2=0x7f36b40082d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-06-17T12:46:27.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.845+0000 7f36cf7fe700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b400a690 msgr2=0x7f36b40082d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-17T12:46:27.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.845+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 0x7f36b4005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2022-06-17T12:46:27.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.845+0000 7f36cf7fe700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b400a690 0x7f36b40082d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-17T12:46:27.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.845+0000 7f36ceffd700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 msgr2=0x7f36b4009c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-06-17T12:46:27.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.845+0000 7f36ceffd700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 msgr2=0x7f36b4009c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-17T12:46:27.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.845+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36b4009c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-17T12:46:27.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.845+0000 7f36cf7fe700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b400a690 0x7f36b40082d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2022-06-17T12:46:27.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:27.845+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36b4009c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2022-06-17T12:46:29.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:29.441+0000 7f36ccff9700 1 -- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 msgr2=0x7f36b4005830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-06-17T12:46:29.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:29.441+0000 7f36ccff9700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 0x7f36b4005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:29.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:29.441+0000 7f36ccff9700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 msgr2=0x7f36b4009c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-06-17T12:46:29.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:29.441+0000 7f36ccff9700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.70:3300/0 conn(0x7f36d00b6440 0x7f36b4009c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:29.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:29.441+0000 7f36ccff9700 1 -- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b400a690 msgr2=0x7f36b40082d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-06-17T12:46:29.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:29.441+0000 7f36ccff9700 1 --2- 172.21.15.36:0/290302583 >> v2:172.21.15.36:3300/0 conn(0x7f36b400a690 0x7f36b40082d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:29.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:29.441+0000 7f36ccff9700 1 -- 172.21.15.36:0/290302583 --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36b400ac10 con 0x7f36b4008da0 2022-06-17T12:46:29.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:29.441+0000 7f36ccff9700 1 -- 172.21.15.36:0/290302583 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36b4014a90 con 0x7f36b400ff30 2022-06-17T12:46:30.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.693+0000 7f36cdffb700 1 -- 172.21.15.36:0/290302583 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2315186940 0 0) 0x7f36b400ac10 con 0x7f36b4008da0 2022-06-17T12:46:30.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.693+0000 7f36cdffb700 1 -- 172.21.15.36:0/290302583 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36b0003260 con 0x7f36b4008da0 2022-06-17T12:46:30.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.693+0000 7f36cdffb700 1 -- 172.21.15.36:0/290302583 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2077556189 0 0) 0x7f36b4014a90 con 0x7f36b400ff30 2022-06-17T12:46:30.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.693+0000 7f36cdffb700 1 -- 172.21.15.36:0/290302583 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36b400ac10 con 0x7f36b400ff30 2022-06-17T12:46:30.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36cdffb700 1 -- 172.21.15.36:0/290302583 <== mon.1 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3521313123 0 0) 0x7f36b400ac10 con 0x7f36b400ff30 2022-06-17T12:46:30.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36cdffb700 1 -- 172.21.15.36:0/290302583 >> v1:172.21.15.36:6789/0 conn(0x7f36b4008da0 legacy=0x7f36b40091b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:30.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36cdffb700 1 -- 172.21.15.36:0/290302583 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b4015c70 con 0x7f36b400ff30 2022-06-17T12:46:30.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36cdffb700 1 -- 172.21.15.36:0/290302583 <== mon.1 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f36b80156f0 con 0x7f36b400ff30 2022-06-17T12:46:30.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36cdffb700 1 -- 172.21.15.36:0/290302583 >> v1:172.21.15.70:6789/0 conn(0x7f36b400ff30 legacy=0x7f36b4012320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:30.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36cf7fe700 1 -- 172.21.15.36:0/290302583 reap_dead start 2022-06-17T12:46:30.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36cdffb700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 0x7f36b00038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:30.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36cdffb700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f36b0003dd0 0x7f36b0004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:30.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36cdffb700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36b0004a60 0x7f36b0004ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:30.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36cdffb700 1 -- 172.21.15.36:0/290302583 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b000b8e0 con 0x7f36b0003dd0 2022-06-17T12:46:30.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36cdffb700 1 -- 172.21.15.36:0/290302583 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b000ba50 con 0x7f36b0004a60 2022-06-17T12:46:30.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36cdffb700 1 -- 172.21.15.36:0/290302583 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b000bc10 con 0x7f36b400a150 2022-06-17T12:46:30.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f36b0003dd0 0x7f36b0004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:30.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36cf7fe700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36b0004a60 0x7f36b0004ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:30.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 0x7f36b00038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:30.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36ce7fc700 1 -- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 msgr2=0x7f36b00038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:30.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 0x7f36b00038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:30.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36ce7fc700 1 -- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36b0004a60 msgr2=0x7f36b0004ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:30.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36b0004a60 0x7f36b0004ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:30.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36ce7fc700 1 -- 172.21.15.36:0/290302583 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b400ac10 con 0x7f36b0003dd0 2022-06-17T12:46:30.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36ceffd700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 0x7f36b00038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:46:30.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f36b0003dd0 0x7f36b0004500 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f36b8002f70 tx=0x7f36b8008040).ready entity=mon.0 client_cookie=fb4c07038b94a56a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:30.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36cdffb700 1 -- 172.21.15.36:0/290302583 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36b801d070 con 0x7f36b0003dd0 2022-06-17T12:46:30.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36cdffb700 1 -- 172.21.15.36:0/290302583 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b80176a0 con 0x7f36b0003dd0 2022-06-17T12:46:30.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.697+0000 7f36cdffb700 1 -- 172.21.15.36:0/290302583 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36b8017a00 con 0x7f36b0003dd0 2022-06-17T12:46:30.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.701+0000 7f36d5798700 1 -- 172.21.15.36:0/290302583 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f36b0003dd0 msgr2=0x7f36b0004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:30.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.701+0000 7f36d5798700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f36b0003dd0 0x7f36b0004500 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f36b8002f70 tx=0x7f36b8008040).stop 2022-06-17T12:46:30.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.701+0000 7f36d5798700 1 -- 172.21.15.36:0/290302583 shutdown_connections 2022-06-17T12:46:30.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.701+0000 7f36d5798700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36b0004a60 0x7f36b0004ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:30.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.701+0000 7f36d5798700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f36b0003dd0 0x7f36b0004500 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:30.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.701+0000 7f36d5798700 1 --2- 172.21.15.36:0/290302583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b400a150 0x7f36b00038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:30.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.701+0000 7f36d5798700 1 -- 172.21.15.36:0/290302583 >> 172.21.15.36:0/290302583 conn(0x7f36d00638d0 msgr2=0x7f36d00b5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:30.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.701+0000 7f36d5798700 1 -- 172.21.15.36:0/290302583 shutdown_connections 2022-06-17T12:46:30.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.701+0000 7f36d5798700 1 -- 172.21.15.36:0/290302583 wait complete. 2022-06-17T12:46:30.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.705+0000 7f36d5798700 1 Processor -- start 2022-06-17T12:46:30.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.705+0000 7f36d5798700 1 -- start start 2022-06-17T12:46:30.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.705+0000 7f36d5798700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b0003dd0 0x7f36d00b1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:30.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.705+0000 7f36d5798700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36b0004a60 0x7f36d0151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:30.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.705+0000 7f36d5798700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f36b400a150 0x7f36d0066390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:30.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.705+0000 7f36d5798700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d013f3d0 con 0x7f36b400a150 2022-06-17T12:46:30.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.705+0000 7f36d5798700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d0151200 con 0x7f36b0004a60 2022-06-17T12:46:30.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.705+0000 7f36d5798700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d01514a0 con 0x7f36b0003dd0 2022-06-17T12:46:30.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.705+0000 7f36cf7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f36b400a150 0x7f36d0066390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:30.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.705+0000 7f36cf7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f36b400a150 0x7f36d0066390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:58940/0 (socket says 172.21.15.36:58940) 2022-06-17T12:46:30.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.705+0000 7f36cf7fe700 1 -- 172.21.15.36:0/538062190 learned_addr learned my addr 172.21.15.36:0/538062190 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:30.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.705+0000 7f36ce7fc700 1 --2- 172.21.15.36:0/538062190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36b0004a60 0x7f36d0151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:30.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.705+0000 7f36ceffd700 1 --2- 172.21.15.36:0/538062190 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b0003dd0 0x7f36d00b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:30.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.705+0000 7f36cf7fe700 1 -- 172.21.15.36:0/538062190 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b0003dd0 msgr2=0x7f36d00b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:30.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.705+0000 7f36cf7fe700 1 --2- 172.21.15.36:0/538062190 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b0003dd0 0x7f36d00b1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:30.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.705+0000 7f36cf7fe700 1 -- 172.21.15.36:0/538062190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36b0004a60 msgr2=0x7f36d0151940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:30.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.705+0000 7f36cf7fe700 1 --2- 172.21.15.36:0/538062190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36b0004a60 0x7f36d0151940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:30.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.705+0000 7f36cf7fe700 1 -- 172.21.15.36:0/538062190 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d0065a20 con 0x7f36b400a150 2022-06-17T12:46:30.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.705+0000 7f36cf7fe700 1 --2- 172.21.15.36:0/538062190 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f36b400a150 0x7f36d0066390 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f36d00b0aa0 tx=0x7f36c8007260).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:30.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.709+0000 7f36c7fff700 1 -- 172.21.15.36:0/538062190 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36c8012cb0 con 0x7f36b400a150 2022-06-17T12:46:30.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.709+0000 7f36c7fff700 1 -- 172.21.15.36:0/538062190 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c800d040 con 0x7f36b400a150 2022-06-17T12:46:30.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.709+0000 7f36d5798700 1 -- 172.21.15.36:0/538062190 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d0151e30 con 0x7f36b400a150 2022-06-17T12:46:30.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.709+0000 7f36c7fff700 1 -- 172.21.15.36:0/538062190 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36c8014400 con 0x7f36b400a150 2022-06-17T12:46:30.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.709+0000 7f36d5798700 1 -- 172.21.15.36:0/538062190 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d0066a20 con 0x7f36b400a150 2022-06-17T12:46:30.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.709+0000 7f36c7fff700 1 -- 172.21.15.36:0/538062190 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f36c801d030 con 0x7f36b400a150 2022-06-17T12:46:30.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.709+0000 7f36c7fff700 1 -- 172.21.15.36:0/538062190 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f36c801f3b0 con 0x7f36b400a150 2022-06-17T12:46:30.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.709+0000 7f36d5798700 1 -- 172.21.15.36:0/538062190 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d0151660 con 0x7f36b400a150 2022-06-17T12:46:30.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.713+0000 7f36c7fff700 1 -- 172.21.15.36:0/538062190 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f36c801b070 con 0x7f36b400a150 2022-06-17T12:46:30.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.777+0000 7f36d5798700 1 -- 172.21.15.36:0/538062190 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f36d0057410 con 0x7f36b400a150 2022-06-17T12:46:30.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.781+0000 7f36c7fff700 1 -- 172.21.15.36:0/538062190 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f36c801f570 con 0x7f36b400a150 2022-06-17T12:46:30.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.781+0000 7f36d5798700 1 -- 172.21.15.36:0/538062190 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f36b400a150 msgr2=0x7f36d0066390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:30.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.781+0000 7f36d5798700 1 --2- 172.21.15.36:0/538062190 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f36b400a150 0x7f36d0066390 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f36d00b0aa0 tx=0x7f36c8007260).stop 2022-06-17T12:46:30.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.781+0000 7f36d5798700 1 -- 172.21.15.36:0/538062190 shutdown_connections 2022-06-17T12:46:30.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.781+0000 7f36d5798700 1 --2- 172.21.15.36:0/538062190 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f36b400a150 0x7f36d0066390 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:30.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.781+0000 7f36d5798700 1 --2- 172.21.15.36:0/538062190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36b0004a60 0x7f36d0151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:30.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.781+0000 7f36d5798700 1 --2- 172.21.15.36:0/538062190 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36b0003dd0 0x7f36d00b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:30.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.781+0000 7f36d5798700 1 -- 172.21.15.36:0/538062190 >> 172.21.15.36:0/538062190 conn(0x7f36d00638d0 msgr2=0x7f36d0137f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:30.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.781+0000 7f36d5798700 1 -- 172.21.15.36:0/538062190 shutdown_connections 2022-06-17T12:46:30.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:30.781+0000 7f36d5798700 1 -- 172.21.15.36:0/538062190 wait complete. 2022-06-17T12:46:30.773 INFO:teuthology.orchestra.run.smithi036.stderr:adjusted tunables profile to default 2022-06-17T12:46:30.776 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2022-06-17T12:46:30.777 DEBUG:teuthology.orchestra.run.smithi036:> mktemp 2022-06-17T12:46:30.806 INFO:teuthology.orchestra.run.smithi036.stdout:/tmp/tmp.C86VxP7FqP 2022-06-17T12:46:30.807 DEBUG:teuthology.orchestra.run.smithi036:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.C86VxP7FqP 2022-06-17T12:46:30.916 DEBUG:teuthology.orchestra.run.smithi036:> sudo chmod 0666 /tmp/tmp.C86VxP7FqP 2022-06-17T12:46:31.114 DEBUG:teuthology.orchestra.remote:smithi036:/tmp/tmp.C86VxP7FqP is 37B 2022-06-17T12:46:31.170 DEBUG:teuthology.orchestra.run.smithi036:> rm -fr /tmp/tmp.C86VxP7FqP 2022-06-17T12:46:31.187 DEBUG:teuthology.orchestra.run.smithi036:> mktemp 2022-06-17T12:46:31.243 INFO:teuthology.orchestra.run.smithi036.stdout:/tmp/tmp.Wjwglvy6L6 2022-06-17T12:46:31.245 DEBUG:teuthology.orchestra.run.smithi036:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.Wjwglvy6L6 2022-06-17T12:46:31.264 DEBUG:teuthology.orchestra.run.smithi036:> sudo chmod 0666 /tmp/tmp.Wjwglvy6L6 2022-06-17T12:46:31.337 DEBUG:teuthology.orchestra.remote:smithi036:/tmp/tmp.Wjwglvy6L6 is 37B 2022-06-17T12:46:31.446 DEBUG:teuthology.orchestra.run.smithi036:> rm -fr /tmp/tmp.Wjwglvy6L6 2022-06-17T12:46:31.469 DEBUG:teuthology.orchestra.run.smithi036:> mktemp 2022-06-17T12:46:31.607 INFO:teuthology.orchestra.run.smithi036.stdout:/tmp/tmp.gcoJQn8std 2022-06-17T12:46:31.608 DEBUG:teuthology.orchestra.run.smithi036:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.gcoJQn8std 2022-06-17T12:46:31.644 DEBUG:teuthology.orchestra.run.smithi036:> sudo chmod 0666 /tmp/tmp.gcoJQn8std 2022-06-17T12:46:31.852 DEBUG:teuthology.orchestra.remote:smithi036:/tmp/tmp.gcoJQn8std is 37B 2022-06-17T12:46:31.876 DEBUG:teuthology.orchestra.run.smithi036:> rm -fr /tmp/tmp.gcoJQn8std 2022-06-17T12:46:31.889 DEBUG:teuthology.orchestra.run.smithi036:> mktemp 2022-06-17T12:46:31.950 INFO:teuthology.orchestra.run.smithi036.stdout:/tmp/tmp.jbkT42Jz4j 2022-06-17T12:46:31.951 DEBUG:teuthology.orchestra.run.smithi036:> sudo cp /var/lib/ceph/osd/ceph-3/fsid /tmp/tmp.jbkT42Jz4j 2022-06-17T12:46:32.013 DEBUG:teuthology.orchestra.run.smithi036:> sudo chmod 0666 /tmp/tmp.jbkT42Jz4j 2022-06-17T12:46:32.131 DEBUG:teuthology.orchestra.remote:smithi036:/tmp/tmp.jbkT42Jz4j is 37B 2022-06-17T12:46:32.222 DEBUG:teuthology.orchestra.run.smithi036:> rm -fr /tmp/tmp.jbkT42Jz4j 2022-06-17T12:46:32.269 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2022-06-17T12:46:32.293 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.MxWu5K9zue 2022-06-17T12:46:32.294 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /var/lib/ceph/osd/ceph-4/fsid /tmp/tmp.MxWu5K9zue 2022-06-17T12:46:32.360 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.MxWu5K9zue 2022-06-17T12:46:32.465 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.MxWu5K9zue is 37B 2022-06-17T12:46:32.639 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.MxWu5K9zue 2022-06-17T12:46:32.670 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2022-06-17T12:46:32.731 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.PjpqqLj5qW 2022-06-17T12:46:32.731 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /var/lib/ceph/osd/ceph-5/fsid /tmp/tmp.PjpqqLj5qW 2022-06-17T12:46:32.756 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.PjpqqLj5qW 2022-06-17T12:46:32.850 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.PjpqqLj5qW is 37B 2022-06-17T12:46:32.900 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.PjpqqLj5qW 2022-06-17T12:46:32.920 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2022-06-17T12:46:33.004 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.rCN6LQsiea 2022-06-17T12:46:33.006 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /var/lib/ceph/osd/ceph-6/fsid /tmp/tmp.rCN6LQsiea 2022-06-17T12:46:33.070 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.rCN6LQsiea 2022-06-17T12:46:33.173 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.rCN6LQsiea is 37B 2022-06-17T12:46:33.254 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.rCN6LQsiea 2022-06-17T12:46:33.286 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2022-06-17T12:46:33.312 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.3pMza9PIec 2022-06-17T12:46:33.313 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /var/lib/ceph/osd/ceph-7/fsid /tmp/tmp.3pMza9PIec 2022-06-17T12:46:33.346 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.3pMza9PIec 2022-06-17T12:46:33.506 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.3pMza9PIec is 37B 2022-06-17T12:46:33.568 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.3pMza9PIec 2022-06-17T12:46:33.594 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new eb8ddf14-4f6d-4045-9340-acf01d2935ad 0 2022-06-17T12:46:33.757 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd22c84700 1 Processor -- start 2022-06-17T12:46:33.783 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd22c84700 1 -- start start 2022-06-17T12:46:33.783 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd22c84700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efd1c063730 0x7efd1c0b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:33.784 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd22c84700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efd1c140610 0x7efd1c0b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:33.784 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd22c84700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd1c063b10 con 0x7efd1c064eb0 2022-06-17T12:46:33.784 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd22c84700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd1c147430 con 0x7efd1c063730 2022-06-17T12:46:33.785 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd22c84700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c0b1ca0 con 0x7efd1c140610 2022-06-17T12:46:33.785 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd21c82700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efd1c063730 0x7efd1c0b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:33.786 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd22483700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efd1c140610 0x7efd1c0b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:33.787 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd21c82700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efd1c063730 0x7efd1c0b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44882/0 (socket says 172.21.15.70:44882) 2022-06-17T12:46:33.787 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd21c82700 1 -- 172.21.15.70:0/2713760282 learned_addr learned my addr 172.21.15.70:0/2713760282 (peer_addr_for_me v2:172.21.15.70:0/0) 2022-06-17T12:46:33.787 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd21c82700 1 -- 172.21.15.70:0/2713760282 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efd1c140610 msgr2=0x7efd1c0b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:33.788 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd21c82700 1 --2- 172.21.15.70:0/2713760282 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efd1c140610 0x7efd1c0b33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:33.788 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd21c82700 1 -- 172.21.15.70:0/2713760282 >> v1:172.21.15.36:6789/0 conn(0x7efd1c064eb0 legacy=0x7efd1c13ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:33.789 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd21c82700 1 -- 172.21.15.70:0/2713760282 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1c1464d0 con 0x7efd1c063730 2022-06-17T12:46:33.789 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd21c82700 1 --2- 172.21.15.70:0/2713760282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd1c063730 0x7efd1c0b2e10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7efd1c063ca0 tx=0x7efd14009290).ready entity=mon.1 client_cookie=783bdd9de3b7ad3f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:33.790 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd20c80700 1 -- 172.21.15.70:0/2713760282 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd1400b860 con 0x7efd1c063730 2022-06-17T12:46:33.791 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd20c80700 1 -- 172.21.15.70:0/2713760282 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1400b9c0 con 0x7efd1c063730 2022-06-17T12:46:33.791 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd20c80700 1 -- 172.21.15.70:0/2713760282 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd1400bb30 con 0x7efd1c063730 2022-06-17T12:46:33.792 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd22c84700 1 -- 172.21.15.70:0/2713760282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd1c063730 msgr2=0x7efd1c0b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:33.792 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd22c84700 1 --2- 172.21.15.70:0/2713760282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd1c063730 0x7efd1c0b2e10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7efd1c063ca0 tx=0x7efd14009290).stop 2022-06-17T12:46:33.793 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd22c84700 1 -- 172.21.15.70:0/2713760282 shutdown_connections 2022-06-17T12:46:33.793 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd22c84700 1 --2- 172.21.15.70:0/2713760282 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efd1c140610 0x7efd1c0b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:33.793 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd22c84700 1 --2- 172.21.15.70:0/2713760282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd1c063730 0x7efd1c0b2e10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:33.794 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd22c84700 1 -- 172.21.15.70:0/2713760282 >> 172.21.15.70:0/2713760282 conn(0x7efd1c0614b0 msgr2=0x7efd1c0618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:33.794 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd22c84700 1 -- 172.21.15.70:0/2713760282 shutdown_connections 2022-06-17T12:46:33.795 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.759+0000 7efd22c84700 1 -- 172.21.15.70:0/2713760282 wait complete. 2022-06-17T12:46:33.795 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd22c84700 1 Processor -- start 2022-06-17T12:46:33.796 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd22c84700 1 -- start start 2022-06-17T12:46:33.796 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd22c84700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd1c063730 0x7efd1c0aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:33.797 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd22c84700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efd1c064eb0 0x7efd1c0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:33.797 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd22c84700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd1c140610 0x7efd1c152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:33.798 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd22c84700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c152550 con 0x7efd1c140610 2022-06-17T12:46:33.798 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd22c84700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c1527f0 con 0x7efd1c063730 2022-06-17T12:46:33.798 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd22c84700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c0af0e0 con 0x7efd1c064eb0 2022-06-17T12:46:33.799 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd21c82700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd1c063730 0x7efd1c0aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:33.799 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd21c82700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd1c063730 0x7efd1c0aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44888/0 (socket says 172.21.15.70:44888) 2022-06-17T12:46:33.800 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd21c82700 1 -- 172.21.15.70:0/4142572593 learned_addr learned my addr 172.21.15.70:0/4142572593 (peer_addr_for_me v2:172.21.15.70:0/0) 2022-06-17T12:46:33.804 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd22483700 1 --2- 172.21.15.70:0/4142572593 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd1c140610 0x7efd1c152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:33.806 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd21481700 1 --2- 172.21.15.70:0/4142572593 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efd1c064eb0 0x7efd1c0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:33.806 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd21c82700 1 -- 172.21.15.70:0/4142572593 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efd1c064eb0 msgr2=0x7efd1c0b6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:33.807 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd21c82700 1 --2- 172.21.15.70:0/4142572593 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efd1c064eb0 0x7efd1c0b6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:33.808 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd21c82700 1 -- 172.21.15.70:0/4142572593 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd1c140610 msgr2=0x7efd1c152c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:33.808 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd21c82700 1 --2- 172.21.15.70:0/4142572593 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd1c140610 0x7efd1c152c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:33.809 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd21c82700 1 -- 172.21.15.70:0/4142572593 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd14009530 con 0x7efd1c063730 2022-06-17T12:46:33.815 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd21481700 1 --2- 172.21.15.70:0/4142572593 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efd1c064eb0 0x7efd1c0b6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:46:33.816 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd21c82700 1 --2- 172.21.15.70:0/4142572593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd1c063730 0x7efd1c0aebf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7efd1c063ca0 tx=0x7efd14016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:33.817 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd0affd700 1 -- 172.21.15.70:0/4142572593 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd14021480 con 0x7efd1c063730 2022-06-17T12:46:33.818 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd0affd700 1 -- 172.21.15.70:0/4142572593 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd140215e0 con 0x7efd1c063730 2022-06-17T12:46:33.818 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd22c84700 1 -- 172.21.15.70:0/4142572593 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1c0b5760 con 0x7efd1c063730 2022-06-17T12:46:33.819 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd0affd700 1 -- 172.21.15.70:0/4142572593 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd14021940 con 0x7efd1c063730 2022-06-17T12:46:33.820 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.763+0000 7efd22c84700 1 -- 172.21.15.70:0/4142572593 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd1c114e40 con 0x7efd1c063730 2022-06-17T12:46:33.820 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.767+0000 7efd0affd700 1 -- 172.21.15.70:0/4142572593 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 2) v1 ==== 32894+0+0 (secure 0 0 0) 0x7efd1400f070 con 0x7efd1c063730 2022-06-17T12:46:33.820 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.767+0000 7efd0affd700 1 -- 172.21.15.70:0/4142572593 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7efd1403b780 con 0x7efd1c063730 2022-06-17T12:46:33.821 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.767+0000 7efd22c84700 1 -- 172.21.15.70:0/4142572593 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd1c057410 con 0x7efd1c063730 2022-06-17T12:46:33.853 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.767+0000 7efd0affd700 1 -- 172.21.15.70:0/4142572593 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7efd14015030 con 0x7efd1c063730 2022-06-17T12:46:33.860 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.831+0000 7efd22c84700 1 -- 172.21.15.70:0/4142572593 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd new", "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", "id": 0} v 0) v1 -- 0x7efd1c05fb30 con 0x7efd1c063730 2022-06-17T12:46:33.861 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.843+0000 7efd0affd700 1 -- 172.21.15.70:0/4142572593 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7efd140103a0 con 0x7efd1c063730 2022-06-17T12:46:33.862 INFO:teuthology.orchestra.run.smithi070.stdout:0 2022-06-17T12:46:33.875 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.843+0000 7efd22c84700 1 -- 172.21.15.70:0/4142572593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd1c063730 msgr2=0x7efd1c0aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:33.875 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.843+0000 7efd22c84700 1 --2- 172.21.15.70:0/4142572593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd1c063730 0x7efd1c0aebf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7efd1c063ca0 tx=0x7efd14016000).stop 2022-06-17T12:46:33.876 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.843+0000 7efd22c84700 1 -- 172.21.15.70:0/4142572593 shutdown_connections 2022-06-17T12:46:33.876 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.843+0000 7efd22c84700 1 --2- 172.21.15.70:0/4142572593 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd1c140610 0x7efd1c152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:33.877 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.843+0000 7efd22c84700 1 --2- 172.21.15.70:0/4142572593 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efd1c064eb0 0x7efd1c0b6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:33.877 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.843+0000 7efd22c84700 1 --2- 172.21.15.70:0/4142572593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd1c063730 0x7efd1c0aebf0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:33.877 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.843+0000 7efd22c84700 1 -- 172.21.15.70:0/4142572593 >> 172.21.15.70:0/4142572593 conn(0x7efd1c0614b0 msgr2=0x7efd1c1443d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:33.878 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.843+0000 7efd22c84700 1 -- 172.21.15.70:0/4142572593 shutdown_connections 2022-06-17T12:46:33.879 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:33.843+0000 7efd22c84700 1 -- 172.21.15.70:0/4142572593 wait complete. 2022-06-17T12:46:33.880 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new 07042677-5771-4de0-bfef-a3e2a4540735 1 2022-06-17T12:46:34.016 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.015+0000 7f0767149700 1 Processor -- start 2022-06-17T12:46:34.016 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.015+0000 7f0767149700 1 -- start start 2022-06-17T12:46:34.017 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.015+0000 7f0767149700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0760144c00 0x7f0760062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.017 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.015+0000 7f0767149700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07600b0ba0 con 0x7f0760062770 2022-06-17T12:46:34.019 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.015+0000 7f0767149700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0760147e90 con 0x7f07600ad330 2022-06-17T12:46:34.020 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.015+0000 7f0767149700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07600ae820 con 0x7f0760144c00 2022-06-17T12:46:34.020 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.015+0000 7f07656e6700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f07600ad330 0x7f07600ad780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36272/0 (socket says 172.21.15.70:36272) 2022-06-17T12:46:34.021 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.015+0000 7f07656e6700 1 -- 172.21.15.70:0/3538932727 learned_addr learned my addr 172.21.15.70:0/3538932727 (peer_addr_for_me v1:172.21.15.70:0/0) 2022-06-17T12:46:34.021 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.015+0000 7f075ffff700 1 --2- 172.21.15.70:0/3538932727 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0760144c00 0x7f0760062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.022 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.015+0000 7f075f7fe700 1 -- 172.21.15.70:0/3538932727 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3385119818 0 0) 0x7f0760147e90 con 0x7f07600ad330 2022-06-17T12:46:34.022 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.015+0000 7f075f7fe700 1 -- 172.21.15.70:0/3538932727 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0744003260 con 0x7f07600ad330 2022-06-17T12:46:34.023 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.015+0000 7f075ffff700 1 -- 172.21.15.70:0/3538932727 >> v1:172.21.15.70:6789/0 conn(0x7f07600ad330 legacy=0x7f07600ad780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.023 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.015+0000 7f075ffff700 1 -- 172.21.15.70:0/3538932727 >> v1:172.21.15.36:6789/0 conn(0x7f0760062770 legacy=0x7f07601444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.024 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.015+0000 7f075ffff700 1 -- 172.21.15.70:0/3538932727 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07600b0dd0 con 0x7f0760144c00 2022-06-17T12:46:34.024 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.015+0000 7f075ffff700 1 --2- 172.21.15.70:0/3538932727 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0760144c00 0x7f0760062b50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f0748002f70 tx=0x7f0748006df0).ready entity=mon.2 client_cookie=d2b50e4d0eaaf1e1 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:34.025 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.015+0000 7f075f7fe700 1 -- 172.21.15.70:0/3538932727 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f074800d570 con 0x7f0760144c00 2022-06-17T12:46:34.025 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.015+0000 7f075f7fe700 1 -- 172.21.15.70:0/3538932727 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0748000e90 con 0x7f0760144c00 2022-06-17T12:46:34.025 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.015+0000 7f075f7fe700 1 -- 172.21.15.70:0/3538932727 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f074800d8c0 con 0x7f0760144c00 2022-06-17T12:46:34.026 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f0767149700 1 -- 172.21.15.70:0/3538932727 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0760144c00 msgr2=0x7f0760062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.026 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f0767149700 1 --2- 172.21.15.70:0/3538932727 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0760144c00 0x7f0760062b50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f0748002f70 tx=0x7f0748006df0).stop 2022-06-17T12:46:34.027 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f0767149700 1 -- 172.21.15.70:0/3538932727 shutdown_connections 2022-06-17T12:46:34.027 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f0767149700 1 --2- 172.21.15.70:0/3538932727 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0760144c00 0x7f0760062b50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.027 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f0767149700 1 -- 172.21.15.70:0/3538932727 >> 172.21.15.70:0/3538932727 conn(0x7f07600b53e0 msgr2=0x7f07600b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:34.028 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f0767149700 1 -- 172.21.15.70:0/3538932727 shutdown_connections 2022-06-17T12:46:34.038 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f0767149700 1 -- 172.21.15.70:0/3538932727 wait complete. 2022-06-17T12:46:34.039 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f0767149700 1 Processor -- start 2022-06-17T12:46:34.039 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f0767149700 1 -- start start 2022-06-17T12:46:34.040 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f0767149700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0760062770 0x7f07600636d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.040 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f0767149700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f07600ad330 0x7f0760065ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.041 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f0767149700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0760144c00 0x7f0760159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.041 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f0767149700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0760131310 con 0x7f0760062770 2022-06-17T12:46:34.041 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f0767149700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07601315b0 con 0x7f0760144c00 2022-06-17T12:46:34.042 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f0767149700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0760131850 con 0x7f07600ad330 2022-06-17T12:46:34.042 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f075ffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f07600ad330 0x7f0760065ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.043 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f075ffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f07600ad330 0x7f0760065ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:59976/0 (socket says 172.21.15.70:59976) 2022-06-17T12:46:34.043 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f075ffff700 1 -- 172.21.15.70:0/2920139422 learned_addr learned my addr 172.21.15.70:0/2920139422 (peer_addr_for_me v2:172.21.15.70:0/0) 2022-06-17T12:46:34.044 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f07656e6700 1 --2- 172.21.15.70:0/2920139422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0760144c00 0x7f0760159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.044 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f0764ee5700 1 --2- 172.21.15.70:0/2920139422 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0760062770 0x7f07600636d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.045 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f075ffff700 1 -- 172.21.15.70:0/2920139422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0760144c00 msgr2=0x7f0760159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.045 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f075ffff700 1 --2- 172.21.15.70:0/2920139422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0760144c00 0x7f0760159eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.046 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f075ffff700 1 -- 172.21.15.70:0/2920139422 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0760062770 msgr2=0x7f07600636d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.046 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f075ffff700 1 --2- 172.21.15.70:0/2920139422 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0760062770 0x7f07600636d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.047 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f075ffff700 1 -- 172.21.15.70:0/2920139422 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074800b180 con 0x7f07600ad330 2022-06-17T12:46:34.047 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f0764ee5700 1 --2- 172.21.15.70:0/2920139422 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0760062770 0x7f07600636d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:46:34.048 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.019+0000 7f075ffff700 1 --2- 172.21.15.70:0/2920139422 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f07600ad330 0x7f0760065ee0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f0748002f70 tx=0x7f0748014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:34.049 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.023+0000 7f075dffb700 1 -- 172.21.15.70:0/2920139422 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f074800dea0 con 0x7f07600ad330 2022-06-17T12:46:34.049 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.023+0000 7f075dffb700 1 -- 172.21.15.70:0/2920139422 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0748013e90 con 0x7f07600ad330 2022-06-17T12:46:34.049 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.023+0000 7f0767149700 1 -- 172.21.15.70:0/2920139422 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0760066860 con 0x7f07600ad330 2022-06-17T12:46:34.050 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.023+0000 7f075dffb700 1 -- 172.21.15.70:0/2920139422 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f074801e5f0 con 0x7f07600ad330 2022-06-17T12:46:34.050 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.023+0000 7f0767149700 1 -- 172.21.15.70:0/2920139422 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0760131cf0 con 0x7f07600ad330 2022-06-17T12:46:34.051 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.023+0000 7f0767149700 1 -- 172.21.15.70:0/2920139422 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0760131a10 con 0x7f07600ad330 2022-06-17T12:46:34.051 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.023+0000 7f075dffb700 1 -- 172.21.15.70:0/2920139422 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 2) v1 ==== 32894+0+0 (secure 0 0 0) 0x7f0748025030 con 0x7f07600ad330 2022-06-17T12:46:34.052 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.023+0000 7f075dffb700 1 -- 172.21.15.70:0/2920139422 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7f074803ace0 con 0x7f07600ad330 2022-06-17T12:46:34.052 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.027+0000 7f075dffb700 1 -- 172.21.15.70:0/2920139422 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f074801c770 con 0x7f07600ad330 2022-06-17T12:46:34.075 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.095+0000 7f0767149700 1 -- 172.21.15.70:0/2920139422 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd new", "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", "id": 1} v 0) v1 -- 0x7f07600b1c20 con 0x7f07600ad330 2022-06-17T12:46:34.087 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.107+0000 7f075dffb700 1 -- 172.21.15.70:0/2920139422 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0748010e40 con 0x7f07600ad330 2022-06-17T12:46:34.088 INFO:teuthology.orchestra.run.smithi070.stdout:1 2022-06-17T12:46:34.092 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.107+0000 7f0767149700 1 -- 172.21.15.70:0/2920139422 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f07600ad330 msgr2=0x7f0760065ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.093 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.107+0000 7f0767149700 1 --2- 172.21.15.70:0/2920139422 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f07600ad330 0x7f0760065ee0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f0748002f70 tx=0x7f0748014000).stop 2022-06-17T12:46:34.093 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.107+0000 7f0767149700 1 -- 172.21.15.70:0/2920139422 shutdown_connections 2022-06-17T12:46:34.094 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.107+0000 7f0767149700 1 --2- 172.21.15.70:0/2920139422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0760144c00 0x7f0760159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.094 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.107+0000 7f0767149700 1 --2- 172.21.15.70:0/2920139422 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f07600ad330 0x7f0760065ee0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.095 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.107+0000 7f0767149700 1 --2- 172.21.15.70:0/2920139422 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0760062770 0x7f07600636d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.095 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.107+0000 7f0767149700 1 -- 172.21.15.70:0/2920139422 >> 172.21.15.70:0/2920139422 conn(0x7f07600b53e0 msgr2=0x7f0760130850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:34.096 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.107+0000 7f0767149700 1 -- 172.21.15.70:0/2920139422 shutdown_connections 2022-06-17T12:46:34.096 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.107+0000 7f0767149700 1 -- 172.21.15.70:0/2920139422 wait complete. 2022-06-17T12:46:34.098 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new 97d0097b-fa05-483c-808c-44feaa212a1c 2 2022-06-17T12:46:34.212 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.227+0000 7ff87dc3e700 1 Processor -- start 2022-06-17T12:46:34.214 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.227+0000 7ff87dc3e700 1 -- start start 2022-06-17T12:46:34.214 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.227+0000 7ff87dc3e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff87813cb70 0x7ff878143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.215 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.227+0000 7ff87dc3e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff8780aef60 0x7ff8780b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.215 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.227+0000 7ff87dc3e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8780af4b0 con 0x7ff8780ba2e0 2022-06-17T12:46:34.215 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.227+0000 7ff87dc3e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff878142f50 con 0x7ff87813cb70 2022-06-17T12:46:34.216 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.227+0000 7ff87dc3e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8781431f0 con 0x7ff8780aef60 2022-06-17T12:46:34.216 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.227+0000 7ff877fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff8780aef60 0x7ff8780b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.216 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.227+0000 7ff877fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff8780aef60 0x7ff8780b9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44908/0 (socket says 172.21.15.70:44908) 2022-06-17T12:46:34.216 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.227+0000 7ff877fff700 1 -- 172.21.15.70:0/880349376 learned_addr learned my addr 172.21.15.70:0/880349376 (peer_addr_for_me v2:172.21.15.70:0/0) 2022-06-17T12:46:34.217 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.227+0000 7ff87cc3c700 1 --2- 172.21.15.70:0/880349376 >> v2:172.21.15.36:3300/0 conn(0x7ff87813cb70 0x7ff878143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.217 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.227+0000 7ff877fff700 1 -- 172.21.15.70:0/880349376 >> v2:172.21.15.36:3300/0 conn(0x7ff87813cb70 msgr2=0x7ff878143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:34.217 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.227+0000 7ff877fff700 1 --2- 172.21.15.70:0/880349376 >> v2:172.21.15.36:3300/0 conn(0x7ff87813cb70 0x7ff878143690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.217 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.227+0000 7ff877fff700 1 -- 172.21.15.70:0/880349376 >> v1:172.21.15.36:6789/0 conn(0x7ff8780ba2e0 legacy=0x7ff8781442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.218 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.227+0000 7ff877fff700 1 -- 172.21.15.70:0/880349376 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8781449b0 con 0x7ff8780aef60 2022-06-17T12:46:34.218 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.227+0000 7ff877fff700 1 --2- 172.21.15.70:0/880349376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8780aef60 0x7ff8780b9df0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7ff860002f70 tx=0x7ff860006eb0).ready entity=mon.1 client_cookie=65716a045c0a6174 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:34.218 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.227+0000 7ff8777fe700 1 -- 172.21.15.70:0/880349376 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff86000d5c0 con 0x7ff8780aef60 2022-06-17T12:46:34.219 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.227+0000 7ff8777fe700 1 -- 172.21.15.70:0/880349376 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff860000e90 con 0x7ff8780aef60 2022-06-17T12:46:34.219 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.227+0000 7ff8777fe700 1 -- 172.21.15.70:0/880349376 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff86000d950 con 0x7ff8780aef60 2022-06-17T12:46:34.219 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.231+0000 7ff87dc3e700 1 -- 172.21.15.70:0/880349376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8780aef60 msgr2=0x7ff8780b9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.219 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.231+0000 7ff87dc3e700 1 --2- 172.21.15.70:0/880349376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8780aef60 0x7ff8780b9df0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7ff860002f70 tx=0x7ff860006eb0).stop 2022-06-17T12:46:34.219 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.231+0000 7ff87dc3e700 1 -- 172.21.15.70:0/880349376 shutdown_connections 2022-06-17T12:46:34.220 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.231+0000 7ff87dc3e700 1 --2- 172.21.15.70:0/880349376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8780aef60 0x7ff8780b9df0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.220 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.231+0000 7ff87dc3e700 1 --2- 172.21.15.70:0/880349376 >> v2:172.21.15.36:3300/0 conn(0x7ff87813cb70 0x7ff878143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.220 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.231+0000 7ff87dc3e700 1 -- 172.21.15.70:0/880349376 >> 172.21.15.70:0/880349376 conn(0x7ff87805c550 msgr2=0x7ff8780b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:34.220 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.231+0000 7ff87dc3e700 1 -- 172.21.15.70:0/880349376 shutdown_connections 2022-06-17T12:46:34.221 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.231+0000 7ff87dc3e700 1 -- 172.21.15.70:0/880349376 wait complete. 2022-06-17T12:46:34.221 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.231+0000 7ff87dc3e700 1 Processor -- start 2022-06-17T12:46:34.226 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.231+0000 7ff87dc3e700 1 -- start start 2022-06-17T12:46:34.226 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.231+0000 7ff87dc3e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8780aef60 0x7ff8780b5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.226 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.231+0000 7ff87dc3e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8780ba2e0 0x7ff878155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.227 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.231+0000 7ff87dc3e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff87813cb70 0x7ff878156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.227 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.231+0000 7ff87dc3e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff878155930 con 0x7ff87813cb70 2022-06-17T12:46:34.227 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.231+0000 7ff87dc3e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff878156740 con 0x7ff8780aef60 2022-06-17T12:46:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.231+0000 7ff87dc3e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8780b00b0 con 0x7ff8780ba2e0 2022-06-17T12:46:34.229 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.231+0000 7ff87cc3c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8780aef60 0x7ff8780b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.229 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.231+0000 7ff877fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8780ba2e0 0x7ff878155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.230 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.231+0000 7ff87cc3c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8780aef60 0x7ff8780b5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44914/0 (socket says 172.21.15.70:44914) 2022-06-17T12:46:34.230 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.231+0000 7ff877fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8780ba2e0 0x7ff878155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:59986/0 (socket says 172.21.15.70:59986) 2022-06-17T12:46:34.231 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.231+0000 7ff87cc3c700 1 -- 172.21.15.70:0/683333474 learned_addr learned my addr 172.21.15.70:0/683333474 (peer_addr_for_me v2:172.21.15.70:0/0) 2022-06-17T12:46:34.231 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.231+0000 7ff87d43d700 1 --2- 172.21.15.70:0/683333474 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff87813cb70 0x7ff878156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.231 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.235+0000 7ff87cc3c700 1 -- 172.21.15.70:0/683333474 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8780ba2e0 msgr2=0x7ff878155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.231 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.235+0000 7ff87cc3c700 1 --2- 172.21.15.70:0/683333474 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8780ba2e0 0x7ff878155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.232 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.235+0000 7ff87cc3c700 1 -- 172.21.15.70:0/683333474 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff87813cb70 msgr2=0x7ff878156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.232 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.235+0000 7ff87cc3c700 1 --2- 172.21.15.70:0/683333474 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff87813cb70 0x7ff878156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.232 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.235+0000 7ff87cc3c700 1 -- 172.21.15.70:0/683333474 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff86000b1e0 con 0x7ff8780aef60 2022-06-17T12:46:34.233 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.235+0000 7ff877fff700 1 --2- 172.21.15.70:0/683333474 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8780ba2e0 0x7ff878155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:46:34.233 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.235+0000 7ff87cc3c700 1 --2- 172.21.15.70:0/683333474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8780aef60 0x7ff8780b5540 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7ff86800c000 tx=0x7ff868006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:34.233 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.235+0000 7ff875ffb700 1 -- 172.21.15.70:0/683333474 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff868011750 con 0x7ff8780aef60 2022-06-17T12:46:34.233 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.235+0000 7ff875ffb700 1 -- 172.21.15.70:0/683333474 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8680118b0 con 0x7ff8780aef60 2022-06-17T12:46:34.234 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.235+0000 7ff87dc3e700 1 -- 172.21.15.70:0/683333474 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8780b5a30 con 0x7ff8780aef60 2022-06-17T12:46:34.234 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.235+0000 7ff875ffb700 1 -- 172.21.15.70:0/683333474 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff868011c10 con 0x7ff8780aef60 2022-06-17T12:46:34.235 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.235+0000 7ff87dc3e700 1 -- 172.21.15.70:0/683333474 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff878140430 con 0x7ff8780aef60 2022-06-17T12:46:34.236 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.235+0000 7ff875ffb700 1 -- 172.21.15.70:0/683333474 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 2) v1 ==== 32894+0+0 (secure 0 0 0) 0x7ff868011d70 con 0x7ff8780aef60 2022-06-17T12:46:34.236 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.235+0000 7ff87dc3e700 1 -- 172.21.15.70:0/683333474 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8780afed0 con 0x7ff8780aef60 2022-06-17T12:46:34.236 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.235+0000 7ff875ffb700 1 -- 172.21.15.70:0/683333474 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1360+0+0 (secure 0 0 0) 0x7ff86803a890 con 0x7ff8780aef60 2022-06-17T12:46:34.237 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.239+0000 7ff875ffb700 1 -- 172.21.15.70:0/683333474 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7ff86801a360 con 0x7ff8780aef60 2022-06-17T12:46:34.290 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.307+0000 7ff87dc3e700 1 -- 172.21.15.70:0/683333474 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd new", "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", "id": 2} v 0) v1 -- 0x7ff878155a70 con 0x7ff8780aef60 2022-06-17T12:46:34.302 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.319+0000 7ff875ffb700 1 -- 172.21.15.70:0/683333474 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff868010aa0 con 0x7ff8780aef60 2022-06-17T12:46:34.303 INFO:teuthology.orchestra.run.smithi070.stdout:2 2022-06-17T12:46:34.316 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.319+0000 7ff87dc3e700 1 -- 172.21.15.70:0/683333474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8780aef60 msgr2=0x7ff8780b5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.317 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.319+0000 7ff87dc3e700 1 --2- 172.21.15.70:0/683333474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8780aef60 0x7ff8780b5540 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7ff86800c000 tx=0x7ff868006df0).stop 2022-06-17T12:46:34.317 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.319+0000 7ff87dc3e700 1 -- 172.21.15.70:0/683333474 shutdown_connections 2022-06-17T12:46:34.318 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.319+0000 7ff87dc3e700 1 --2- 172.21.15.70:0/683333474 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff87813cb70 0x7ff878156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.319 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.319+0000 7ff87dc3e700 1 --2- 172.21.15.70:0/683333474 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8780ba2e0 0x7ff878155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.319 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.319+0000 7ff87dc3e700 1 --2- 172.21.15.70:0/683333474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8780aef60 0x7ff8780b5540 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.320 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.319+0000 7ff87dc3e700 1 -- 172.21.15.70:0/683333474 >> 172.21.15.70:0/683333474 conn(0x7ff87805c550 msgr2=0x7ff8780b7920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:34.320 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.319+0000 7ff87dc3e700 1 -- 172.21.15.70:0/683333474 shutdown_connections 2022-06-17T12:46:34.321 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.319+0000 7ff87dc3e700 1 -- 172.21.15.70:0/683333474 wait complete. 2022-06-17T12:46:34.321 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new fb471fe2-5427-441e-b6d7-47c004564c69 3 2022-06-17T12:46:34.436 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.451+0000 7f0e86b10700 1 Processor -- start 2022-06-17T12:46:34.436 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.451+0000 7f0e86b10700 1 -- start start 2022-06-17T12:46:34.437 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.451+0000 7f0e86b10700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e800b2200 0x7f0e800af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.437 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.451+0000 7f0e86b10700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0e800b2cd0 0x7f0e80147600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.437 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.451+0000 7f0e86b10700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e800b25e0 con 0x7f0e800af8d0 2022-06-17T12:46:34.437 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.451+0000 7f0e86b10700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e800ad790 con 0x7f0e800b2cd0 2022-06-17T12:46:34.438 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.451+0000 7f0e86b10700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e800aef40 con 0x7f0e800b2200 2022-06-17T12:46:34.438 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.451+0000 7f0e85b0e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e800b2200 0x7f0e800af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.438 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.451+0000 7f0e8630f700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0e800b2cd0 0x7f0e80147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.439 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.451+0000 7f0e85b0e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e800b2200 0x7f0e800af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:59992/0 (socket says 172.21.15.70:59992) 2022-06-17T12:46:34.439 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.451+0000 7f0e8530d700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0e800af8d0 0x7f0e8014a8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36296/0 (socket says 172.21.15.70:36296) 2022-06-17T12:46:34.439 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.451+0000 7f0e85b0e700 1 -- 172.21.15.70:0/3251624684 learned_addr learned my addr 172.21.15.70:0/3251624684 (peer_addr_for_me v2:172.21.15.70:0/0) 2022-06-17T12:46:34.440 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.451+0000 7f0e85b0e700 1 -- 172.21.15.70:0/3251624684 >> v2:172.21.15.70:3300/0 conn(0x7f0e800b2cd0 msgr2=0x7f0e80147600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:34.440 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.451+0000 7f0e85b0e700 1 --2- 172.21.15.70:0/3251624684 >> v2:172.21.15.70:3300/0 conn(0x7f0e800b2cd0 0x7f0e80147600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.440 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.451+0000 7f0e85b0e700 1 -- 172.21.15.70:0/3251624684 >> v1:172.21.15.70:6789/0 conn(0x7f0e800af8d0 legacy=0x7f0e8014a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.440 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.451+0000 7f0e85b0e700 1 -- 172.21.15.70:0/3251624684 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e800b1950 con 0x7f0e800b2200 2022-06-17T12:46:34.440 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e85b0e700 1 --2- 172.21.15.70:0/3251624684 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e800b2200 0x7f0e800af3e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f0e70002f70 tx=0x7f0e70006eb0).ready entity=mon.2 client_cookie=7626b50075d15515 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:34.441 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e84b0c700 1 -- 172.21.15.70:0/3251624684 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e7000d570 con 0x7f0e800b2200 2022-06-17T12:46:34.441 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e84b0c700 1 -- 172.21.15.70:0/3251624684 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e70000e90 con 0x7f0e800b2200 2022-06-17T12:46:34.441 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e84b0c700 1 -- 172.21.15.70:0/3251624684 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e7000d900 con 0x7f0e800b2200 2022-06-17T12:46:34.442 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e86b10700 1 -- 172.21.15.70:0/3251624684 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e800b2200 msgr2=0x7f0e800af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.442 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e86b10700 1 --2- 172.21.15.70:0/3251624684 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e800b2200 0x7f0e800af3e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f0e70002f70 tx=0x7f0e70006eb0).stop 2022-06-17T12:46:34.443 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e86b10700 1 -- 172.21.15.70:0/3251624684 shutdown_connections 2022-06-17T12:46:34.443 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e86b10700 1 --2- 172.21.15.70:0/3251624684 >> v2:172.21.15.70:3300/0 conn(0x7f0e800b2cd0 0x7f0e80147600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.444 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e86b10700 1 --2- 172.21.15.70:0/3251624684 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e800b2200 0x7f0e800af3e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.446 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e86b10700 1 -- 172.21.15.70:0/3251624684 >> 172.21.15.70:0/3251624684 conn(0x7f0e8005f770 msgr2=0x7f0e8005fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:34.446 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e86b10700 1 -- 172.21.15.70:0/3251624684 shutdown_connections 2022-06-17T12:46:34.447 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e86b10700 1 -- 172.21.15.70:0/3251624684 wait complete. 2022-06-17T12:46:34.447 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e86b10700 1 Processor -- start 2022-06-17T12:46:34.447 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e86b10700 1 -- start start 2022-06-17T12:46:34.448 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e86b10700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e800af8d0 0x7f0e800aaf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.448 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e86b10700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e800b2200 0x7f0e8014d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.449 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e86b10700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e800b2cd0 0x7f0e800a8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.449 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e86b10700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e800ab8f0 con 0x7f0e800b2cd0 2022-06-17T12:46:34.449 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e86b10700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e800a95f0 con 0x7f0e800af8d0 2022-06-17T12:46:34.450 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e86b10700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e800a90c0 con 0x7f0e800b2200 2022-06-17T12:46:34.450 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e8530d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e800b2200 0x7f0e8014d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.450 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e8530d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e800b2200 0x7f0e8014d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:60000/0 (socket says 172.21.15.70:60000) 2022-06-17T12:46:34.451 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e85b0e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e800af8d0 0x7f0e800aaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.451 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e8530d700 1 -- 172.21.15.70:0/1598430358 learned_addr learned my addr 172.21.15.70:0/1598430358 (peer_addr_for_me v2:172.21.15.70:0/0) 2022-06-17T12:46:34.452 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e8630f700 1 --2- 172.21.15.70:0/1598430358 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e800b2cd0 0x7f0e800a8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.452 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e8530d700 1 -- 172.21.15.70:0/1598430358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e800af8d0 msgr2=0x7f0e800aaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.453 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e8530d700 1 --2- 172.21.15.70:0/1598430358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e800af8d0 0x7f0e800aaf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.453 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e8530d700 1 -- 172.21.15.70:0/1598430358 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e800b2cd0 msgr2=0x7f0e800a8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.453 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e8530d700 1 --2- 172.21.15.70:0/1598430358 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e800b2cd0 0x7f0e800a8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.454 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.455+0000 7f0e8530d700 1 -- 172.21.15.70:0/1598430358 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7000b1e0 con 0x7f0e800b2200 2022-06-17T12:46:34.455 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.459+0000 7f0e8530d700 1 --2- 172.21.15.70:0/1598430358 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e800b2200 0x7f0e8014d140 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f0e800afcd0 tx=0x7f0e74007bd0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:34.455 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.459+0000 7f0e7effd700 1 -- 172.21.15.70:0/1598430358 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e74007170 con 0x7f0e800b2200 2022-06-17T12:46:34.456 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.459+0000 7f0e7effd700 1 -- 172.21.15.70:0/1598430358 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e740072d0 con 0x7f0e800b2200 2022-06-17T12:46:34.456 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.459+0000 7f0e86b10700 1 -- 172.21.15.70:0/1598430358 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e800a9350 con 0x7f0e800b2200 2022-06-17T12:46:34.457 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.459+0000 7f0e7effd700 1 -- 172.21.15.70:0/1598430358 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e74004710 con 0x7f0e800b2200 2022-06-17T12:46:34.457 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.459+0000 7f0e86b10700 1 -- 172.21.15.70:0/1598430358 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e800a7300 con 0x7f0e800b2200 2022-06-17T12:46:34.457 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.459+0000 7f0e7effd700 1 -- 172.21.15.70:0/1598430358 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 2) v1 ==== 32894+0+0 (secure 0 0 0) 0x7f0e74004870 con 0x7f0e800b2200 2022-06-17T12:46:34.458 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.459+0000 7f0e7effd700 1 -- 172.21.15.70:0/1598430358 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1499+0+0 (secure 0 0 0) 0x7f0e74039ce0 con 0x7f0e800b2200 2022-06-17T12:46:34.465 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.459+0000 7f0e86b10700 1 -- 172.21.15.70:0/1598430358 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e800ad330 con 0x7f0e800b2200 2022-06-17T12:46:34.467 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.463+0000 7f0e7effd700 1 -- 172.21.15.70:0/1598430358 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f0e74004b00 con 0x7f0e800b2200 2022-06-17T12:46:34.484 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.499+0000 7f0e7effd700 1 -- 172.21.15.70:0/1598430358 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mgrmap(e 3) v1 ==== 65884+0+0 (secure 0 0 0) 0x7f0e7401d910 con 0x7f0e800b2200 2022-06-17T12:46:34.485 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.499+0000 7f0e7effd700 1 --2- 172.21.15.70:0/1598430358 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0e600540d0 0x7f0e600564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.485 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.499+0000 7f0e85b0e700 1 --2- 172.21.15.70:0/1598430358 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0e600540d0 0x7f0e600564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.486 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.503+0000 7f0e85b0e700 1 --2- 172.21.15.70:0/1598430358 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0e600540d0 0x7f0e600564c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f0e7001c000 tx=0x7f0e70015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:34.503 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.527+0000 7f0e86b10700 1 -- 172.21.15.70:0/1598430358 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd new", "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", "id": 3} v 0) v1 -- 0x7f0e80057410 con 0x7f0e800b2200 2022-06-17T12:46:34.509 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.535+0000 7f0e7effd700 1 -- 172.21.15.70:0/1598430358 <== mon.2 v2:172.21.15.70:3301/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0e74044ca0 con 0x7f0e800b2200 2022-06-17T12:46:34.510 INFO:teuthology.orchestra.run.smithi070.stdout:3 2022-06-17T12:46:34.515 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.535+0000 7f0e86b10700 1 -- 172.21.15.70:0/1598430358 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0e600540d0 msgr2=0x7f0e600564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.516 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.535+0000 7f0e86b10700 1 --2- 172.21.15.70:0/1598430358 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0e600540d0 0x7f0e600564c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f0e7001c000 tx=0x7f0e70015000).stop 2022-06-17T12:46:34.516 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.535+0000 7f0e86b10700 1 -- 172.21.15.70:0/1598430358 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e800b2200 msgr2=0x7f0e8014d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.517 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.535+0000 7f0e86b10700 1 --2- 172.21.15.70:0/1598430358 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e800b2200 0x7f0e8014d140 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f0e800afcd0 tx=0x7f0e74007bd0).stop 2022-06-17T12:46:34.520 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.535+0000 7f0e86b10700 1 -- 172.21.15.70:0/1598430358 shutdown_connections 2022-06-17T12:46:34.521 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.535+0000 7f0e86b10700 1 --2- 172.21.15.70:0/1598430358 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0e600540d0 0x7f0e600564c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.522 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.535+0000 7f0e86b10700 1 --2- 172.21.15.70:0/1598430358 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e800b2cd0 0x7f0e800a8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.522 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.535+0000 7f0e86b10700 1 --2- 172.21.15.70:0/1598430358 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e800b2200 0x7f0e8014d140 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.523 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.535+0000 7f0e86b10700 1 --2- 172.21.15.70:0/1598430358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e800af8d0 0x7f0e800aaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.524 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.535+0000 7f0e86b10700 1 -- 172.21.15.70:0/1598430358 >> 172.21.15.70:0/1598430358 conn(0x7f0e8005f770 msgr2=0x7f0e801487d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:34.525 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.535+0000 7f0e86b10700 1 -- 172.21.15.70:0/1598430358 shutdown_connections 2022-06-17T12:46:34.525 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.535+0000 7f0e86b10700 1 -- 172.21.15.70:0/1598430358 wait complete. 2022-06-17T12:46:34.527 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new 4894aebf-e4b3-4436-a3ff-00a1241dbfcc 4 2022-06-17T12:46:34.633 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa649d2700 1 Processor -- start 2022-06-17T12:46:34.634 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa649d2700 1 -- start start 2022-06-17T12:46:34.634 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa649d2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa600b2cd0 0x7faa601465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.634 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa649d2700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7faa60146ae0 0x7faa60146f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.634 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa649d2700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa600b25e0 con 0x7faa600b2200 2022-06-17T12:46:34.635 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa649d2700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa600ad790 con 0x7faa60146ae0 2022-06-17T12:46:34.635 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa649d2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa600aef40 con 0x7faa600b2cd0 2022-06-17T12:46:34.635 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa5effd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa600b2cd0 0x7faa601465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.635 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa5effd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa600b2cd0 0x7faa601465f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:60014/0 (socket says 172.21.15.70:60014) 2022-06-17T12:46:34.636 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa5effd700 1 -- 172.21.15.70:0/1215826654 learned_addr learned my addr 172.21.15.70:0/1215826654 (peer_addr_for_me v2:172.21.15.70:0/0) 2022-06-17T12:46:34.636 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa5ffff700 1 --2- 172.21.15.70:0/1215826654 >> v2:172.21.15.36:3300/0 conn(0x7faa60146ae0 0x7faa60146f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.636 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa5effd700 1 -- 172.21.15.70:0/1215826654 >> v2:172.21.15.36:3300/0 conn(0x7faa60146ae0 msgr2=0x7faa60146f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:34.636 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa5effd700 1 --2- 172.21.15.70:0/1215826654 >> v2:172.21.15.36:3300/0 conn(0x7faa60146ae0 0x7faa60146f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.636 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa5effd700 1 -- 172.21.15.70:0/1215826654 >> v1:172.21.15.36:6789/0 conn(0x7faa600b2200 legacy=0x7faa600af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.637 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa5effd700 1 -- 172.21.15.70:0/1215826654 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa600afaf0 con 0x7faa600b2cd0 2022-06-17T12:46:34.637 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa5effd700 1 --2- 172.21.15.70:0/1215826654 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa600b2cd0 0x7faa601465f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7faa54002f70 tx=0x7faa54006eb0).ready entity=mon.2 client_cookie=715beeb44fbb1a33 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:34.638 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa5e7fc700 1 -- 172.21.15.70:0/1215826654 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa5400d570 con 0x7faa600b2cd0 2022-06-17T12:46:34.638 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa5e7fc700 1 -- 172.21.15.70:0/1215826654 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa54000e90 con 0x7faa600b2cd0 2022-06-17T12:46:34.638 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa5e7fc700 1 -- 172.21.15.70:0/1215826654 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa5400d900 con 0x7faa600b2cd0 2022-06-17T12:46:34.638 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa649d2700 1 -- 172.21.15.70:0/1215826654 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa600b2cd0 msgr2=0x7faa601465f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.639 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa649d2700 1 --2- 172.21.15.70:0/1215826654 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa600b2cd0 0x7faa601465f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7faa54002f70 tx=0x7faa54006eb0).stop 2022-06-17T12:46:34.640 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa649d2700 1 -- 172.21.15.70:0/1215826654 shutdown_connections 2022-06-17T12:46:34.640 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa649d2700 1 --2- 172.21.15.70:0/1215826654 >> v2:172.21.15.36:3300/0 conn(0x7faa60146ae0 0x7faa60146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.641 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa649d2700 1 --2- 172.21.15.70:0/1215826654 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa600b2cd0 0x7faa601465f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.641 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa649d2700 1 -- 172.21.15.70:0/1215826654 >> 172.21.15.70:0/1215826654 conn(0x7faa6005f770 msgr2=0x7faa6005fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:34.641 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa649d2700 1 -- 172.21.15.70:0/1215826654 shutdown_connections 2022-06-17T12:46:34.642 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.655+0000 7faa649d2700 1 -- 172.21.15.70:0/1215826654 wait complete. 2022-06-17T12:46:34.645 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.659+0000 7faa649d2700 1 Processor -- start 2022-06-17T12:46:34.646 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.659+0000 7faa649d2700 1 -- start start 2022-06-17T12:46:34.660 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.659+0000 7faa649d2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa600b2200 0x7faa601403d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.666 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.659+0000 7faa649d2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa600b2cd0 0x7faa600b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.667 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.659+0000 7faa649d2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa60146ae0 0x7faa60156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.668 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.659+0000 7faa649d2700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa601408c0 con 0x7faa600b2cd0 2022-06-17T12:46:34.668 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.659+0000 7faa649d2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa60144740 con 0x7faa60146ae0 2022-06-17T12:46:34.668 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.659+0000 7faa649d2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa601449e0 con 0x7faa600b2200 2022-06-17T12:46:34.669 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.659+0000 7faa5f7fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa600b2200 0x7faa601403d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.670 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.659+0000 7faa5f7fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa600b2200 0x7faa601403d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:60018/0 (socket says 172.21.15.70:60018) 2022-06-17T12:46:34.671 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.659+0000 7faa5f7fe700 1 -- 172.21.15.70:0/2376782970 learned_addr learned my addr 172.21.15.70:0/2376782970 (peer_addr_for_me v2:172.21.15.70:0/0) 2022-06-17T12:46:34.671 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.659+0000 7faa5ffff700 1 --2- 172.21.15.70:0/2376782970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa60146ae0 0x7faa60156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.671 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.663+0000 7faa5effd700 1 --2- 172.21.15.70:0/2376782970 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa600b2cd0 0x7faa600b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.672 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.663+0000 7faa5f7fe700 1 -- 172.21.15.70:0/2376782970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa60146ae0 msgr2=0x7faa60156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.672 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.663+0000 7faa5f7fe700 1 --2- 172.21.15.70:0/2376782970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa60146ae0 0x7faa60156da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.673 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.663+0000 7faa5f7fe700 1 -- 172.21.15.70:0/2376782970 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa600b2cd0 msgr2=0x7faa600b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.673 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.663+0000 7faa5f7fe700 1 --2- 172.21.15.70:0/2376782970 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa600b2cd0 0x7faa600b0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.675 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.663+0000 7faa5f7fe700 1 -- 172.21.15.70:0/2376782970 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5400b1e0 con 0x7faa600b2200 2022-06-17T12:46:34.675 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.663+0000 7faa5f7fe700 1 --2- 172.21.15.70:0/2376782970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa600b2200 0x7faa601403d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7faa600af2c0 tx=0x7faa50007c20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:34.676 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.663+0000 7faa5cff9700 1 -- 172.21.15.70:0/2376782970 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa50013450 con 0x7faa600b2200 2022-06-17T12:46:34.676 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.663+0000 7faa5cff9700 1 -- 172.21.15.70:0/2376782970 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa500135b0 con 0x7faa600b2200 2022-06-17T12:46:34.677 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.663+0000 7faa649d2700 1 -- 172.21.15.70:0/2376782970 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa60144c80 con 0x7faa600b2200 2022-06-17T12:46:34.678 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.663+0000 7faa5cff9700 1 -- 172.21.15.70:0/2376782970 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa50013910 con 0x7faa600b2200 2022-06-17T12:46:34.679 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.663+0000 7faa649d2700 1 -- 172.21.15.70:0/2376782970 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa60142ac0 con 0x7faa600b2200 2022-06-17T12:46:34.679 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.663+0000 7faa649d2700 1 -- 172.21.15.70:0/2376782970 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa60156af0 con 0x7faa600b2200 2022-06-17T12:46:34.680 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.663+0000 7faa5cff9700 1 -- 172.21.15.70:0/2376782970 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 3) v1 ==== 65884+0+0 (secure 0 0 0) 0x7faa5001a060 con 0x7faa600b2200 2022-06-17T12:46:34.680 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.663+0000 7faa5cff9700 1 --2- 172.21.15.70:0/2376782970 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa40053a30 0x7faa40055ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.681 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.663+0000 7faa5cff9700 1 -- 172.21.15.70:0/2376782970 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1614+0+0 (secure 0 0 0) 0x7faa5006a2e0 con 0x7faa600b2200 2022-06-17T12:46:34.681 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.663+0000 7faa5effd700 1 --2- 172.21.15.70:0/2376782970 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa40053a30 0x7faa40055ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.681 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.667+0000 7faa5effd700 1 --2- 172.21.15.70:0/2376782970 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa40053a30 0x7faa40055ef0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7faa5401c000 tx=0x7faa54015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:34.683 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.675+0000 7faa5cff9700 1 -- 172.21.15.70:0/2376782970 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faa50042450 con 0x7faa600b2200 2022-06-17T12:46:34.781 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.799+0000 7faa649d2700 1 -- 172.21.15.70:0/2376782970 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", "id": 4} v 0) v1 -- 0x7faa600b1950 con 0x7faa600b2200 2022-06-17T12:46:34.799 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.807+0000 7faa5cff9700 1 -- 172.21.15.70:0/2376782970 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7faa50018020 con 0x7faa600b2200 2022-06-17T12:46:34.800 INFO:teuthology.orchestra.run.smithi070.stdout:4 2022-06-17T12:46:34.807 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.811+0000 7faa649d2700 1 -- 172.21.15.70:0/2376782970 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa40053a30 msgr2=0x7faa40055ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.807 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.811+0000 7faa649d2700 1 --2- 172.21.15.70:0/2376782970 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa40053a30 0x7faa40055ef0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7faa5401c000 tx=0x7faa54015000).stop 2022-06-17T12:46:34.808 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.811+0000 7faa649d2700 1 -- 172.21.15.70:0/2376782970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa600b2200 msgr2=0x7faa601403d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.808 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.811+0000 7faa649d2700 1 --2- 172.21.15.70:0/2376782970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa600b2200 0x7faa601403d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7faa600af2c0 tx=0x7faa50007c20).stop 2022-06-17T12:46:34.808 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.811+0000 7faa649d2700 1 -- 172.21.15.70:0/2376782970 shutdown_connections 2022-06-17T12:46:34.811 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.811+0000 7faa649d2700 1 --2- 172.21.15.70:0/2376782970 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa40053a30 0x7faa40055ef0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.812 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.811+0000 7faa649d2700 1 --2- 172.21.15.70:0/2376782970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa60146ae0 0x7faa60156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.813 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.811+0000 7faa649d2700 1 --2- 172.21.15.70:0/2376782970 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa600b2cd0 0x7faa600b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.813 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.811+0000 7faa649d2700 1 --2- 172.21.15.70:0/2376782970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa600b2200 0x7faa601403d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.814 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.811+0000 7faa649d2700 1 -- 172.21.15.70:0/2376782970 >> 172.21.15.70:0/2376782970 conn(0x7faa6005f770 msgr2=0x7faa6005fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:34.814 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.811+0000 7faa649d2700 1 -- 172.21.15.70:0/2376782970 shutdown_connections 2022-06-17T12:46:34.815 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.811+0000 7faa649d2700 1 -- 172.21.15.70:0/2376782970 wait complete. 2022-06-17T12:46:34.815 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new 74e2041b-4756-47c2-bd3c-defaa046e936 5 2022-06-17T12:46:34.935 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b883d700 1 Processor -- start 2022-06-17T12:46:34.936 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b883d700 1 -- start start 2022-06-17T12:46:34.936 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b883d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f97b40b2200 0x7f97b40af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.937 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b883d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f97b40af8d0 0x7f97b414a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.937 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b883d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97b40b2cd0 0x7f97b414adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.938 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b883d700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b40b25e0 con 0x7f97b40b2200 2022-06-17T12:46:34.938 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b883d700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b40ad790 con 0x7f97b40af8d0 2022-06-17T12:46:34.939 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b883d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b40aef40 con 0x7f97b40b2cd0 2022-06-17T12:46:34.940 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b259c700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f97b40af8d0 0x7f97b414a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.941 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b359e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97b40b2cd0 0x7f97b414adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.941 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b359e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97b40b2cd0 0x7f97b414adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:60030/0 (socket says 172.21.15.70:60030) 2022-06-17T12:46:34.942 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b359e700 1 -- 172.21.15.70:0/1565205158 learned_addr learned my addr 172.21.15.70:0/1565205158 (peer_addr_for_me v2:172.21.15.70:0/0) 2022-06-17T12:46:34.942 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b2d9d700 1 --2- 172.21.15.70:0/1565205158 >> v2:172.21.15.36:3300/0 conn(0x7f97b40b2200 0x7f97b40af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.943 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b359e700 1 -- 172.21.15.70:0/1565205158 >> v2:172.21.15.70:3300/0 conn(0x7f97b40af8d0 msgr2=0x7f97b414a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:34.948 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b359e700 1 --2- 172.21.15.70:0/1565205158 >> v2:172.21.15.70:3300/0 conn(0x7f97b40af8d0 0x7f97b414a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.949 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b359e700 1 -- 172.21.15.70:0/1565205158 >> v2:172.21.15.36:3300/0 conn(0x7f97b40b2200 msgr2=0x7f97b40af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:34.949 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b359e700 1 --2- 172.21.15.70:0/1565205158 >> v2:172.21.15.36:3300/0 conn(0x7f97b40b2200 0x7f97b40af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.949 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b359e700 1 -- 172.21.15.70:0/1565205158 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b40b1950 con 0x7f97b40b2cd0 2022-06-17T12:46:34.949 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b359e700 1 --2- 172.21.15.70:0/1565205158 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97b40b2cd0 0x7f97b414adc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f979c002f70 tx=0x7f979c006eb0).ready entity=mon.2 client_cookie=c26b98ae423e8436 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:34.949 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b1d9b700 1 -- 172.21.15.70:0/1565205158 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f979c00d570 con 0x7f97b40b2cd0 2022-06-17T12:46:34.950 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b1d9b700 1 -- 172.21.15.70:0/1565205158 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979c000e90 con 0x7f97b40b2cd0 2022-06-17T12:46:34.950 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b1d9b700 1 -- 172.21.15.70:0/1565205158 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f979c00d900 con 0x7f97b40b2cd0 2022-06-17T12:46:34.950 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b883d700 1 -- 172.21.15.70:0/1565205158 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97b40b2cd0 msgr2=0x7f97b414adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.950 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b883d700 1 --2- 172.21.15.70:0/1565205158 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97b40b2cd0 0x7f97b414adc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f979c002f70 tx=0x7f979c006eb0).stop 2022-06-17T12:46:34.953 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b883d700 1 -- 172.21.15.70:0/1565205158 shutdown_connections 2022-06-17T12:46:34.953 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b883d700 1 --2- 172.21.15.70:0/1565205158 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97b40b2cd0 0x7f97b414adc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.954 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b883d700 1 --2- 172.21.15.70:0/1565205158 >> v2:172.21.15.70:3300/0 conn(0x7f97b40af8d0 0x7f97b414a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.954 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b883d700 1 --2- 172.21.15.70:0/1565205158 >> v2:172.21.15.36:3300/0 conn(0x7f97b40b2200 0x7f97b40af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.954 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b883d700 1 -- 172.21.15.70:0/1565205158 >> 172.21.15.70:0/1565205158 conn(0x7f97b405f770 msgr2=0x7f97b405fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:34.954 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.943+0000 7f97b883d700 1 -- 172.21.15.70:0/1565205158 shutdown_connections 2022-06-17T12:46:34.955 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.947+0000 7f97b883d700 1 -- 172.21.15.70:0/1565205158 wait complete. 2022-06-17T12:46:34.955 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.947+0000 7f97b883d700 1 Processor -- start 2022-06-17T12:46:34.955 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.947+0000 7f97b883d700 1 -- start start 2022-06-17T12:46:34.955 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.947+0000 7f97b883d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97b40af8d0 0x7f97b41565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.955 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.947+0000 7f97b883d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97b40b2200 0x7f97b414e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.956 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.947+0000 7f97b883d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97b40b2cd0 0x7f97b41455e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.956 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.947+0000 7f97b883d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b40aa140 con 0x7f97b40af8d0 2022-06-17T12:46:34.956 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.947+0000 7f97b883d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b40a7260 con 0x7f97b40b2200 2022-06-17T12:46:34.957 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.947+0000 7f97b883d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b40aaff0 con 0x7f97b40b2cd0 2022-06-17T12:46:34.957 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.947+0000 7f97b259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97b40b2200 0x7f97b414e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.957 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.947+0000 7f97b359e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97b40b2cd0 0x7f97b41455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.957 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.947+0000 7f97b359e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97b40b2cd0 0x7f97b41455e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:60036/0 (socket says 172.21.15.70:60036) 2022-06-17T12:46:34.963 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.947+0000 7f97b359e700 1 -- 172.21.15.70:0/4056701208 learned_addr learned my addr 172.21.15.70:0/4056701208 (peer_addr_for_me v2:172.21.15.70:0/0) 2022-06-17T12:46:34.963 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.947+0000 7f97b259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97b40b2200 0x7f97b414e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44960/0 (socket says 172.21.15.70:44960) 2022-06-17T12:46:34.963 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.947+0000 7f97b2d9d700 1 --2- 172.21.15.70:0/4056701208 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97b40af8d0 0x7f97b41565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.964 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.947+0000 7f97b259c700 1 -- 172.21.15.70:0/4056701208 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97b40b2cd0 msgr2=0x7f97b41455e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.965 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.947+0000 7f97b259c700 1 --2- 172.21.15.70:0/4056701208 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97b40b2cd0 0x7f97b41455e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.965 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.947+0000 7f97b259c700 1 -- 172.21.15.70:0/4056701208 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97b40af8d0 msgr2=0x7f97b41565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:34.965 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.947+0000 7f97b259c700 1 --2- 172.21.15.70:0/4056701208 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97b40af8d0 0x7f97b41565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:34.966 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.947+0000 7f97b259c700 1 -- 172.21.15.70:0/4056701208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c00b1e0 con 0x7f97b40b2200 2022-06-17T12:46:34.966 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.947+0000 7f97b259c700 1 --2- 172.21.15.70:0/4056701208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97b40b2200 0x7f97b414e2a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f97ac00c000 tx=0x7f97ac0091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:34.966 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.951+0000 7f97abfff700 1 -- 172.21.15.70:0/4056701208 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97ac00c740 con 0x7f97b40b2200 2022-06-17T12:46:34.966 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.951+0000 7f97b883d700 1 -- 172.21.15.70:0/4056701208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b40a8c40 con 0x7f97b40b2200 2022-06-17T12:46:34.967 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.951+0000 7f97b883d700 1 -- 172.21.15.70:0/4056701208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b40a91a0 con 0x7f97b40b2200 2022-06-17T12:46:34.967 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.951+0000 7f97abfff700 1 -- 172.21.15.70:0/4056701208 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ac00c8a0 con 0x7f97b40b2200 2022-06-17T12:46:34.979 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.951+0000 7f97abfff700 1 -- 172.21.15.70:0/4056701208 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97ac00cc00 con 0x7f97b40b2200 2022-06-17T12:46:34.979 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.951+0000 7f97b883d700 1 -- 172.21.15.70:0/4056701208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b40aa2b0 con 0x7f97b40b2200 2022-06-17T12:46:34.980 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.951+0000 7f97abfff700 1 -- 172.21.15.70:0/4056701208 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 3) v1 ==== 65884+0+0 (secure 0 0 0) 0x7f97ac00cd60 con 0x7f97b40b2200 2022-06-17T12:46:34.980 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.951+0000 7f97abfff700 1 --2- 172.21.15.70:0/4056701208 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9798053a30 0x7f9798055ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:34.981 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.951+0000 7f97abfff700 1 -- 172.21.15.70:0/4056701208 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1729+0+0 (secure 0 0 0) 0x7f97ac06b2f0 con 0x7f97b40b2200 2022-06-17T12:46:34.981 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.951+0000 7f97b2d9d700 1 --2- 172.21.15.70:0/4056701208 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9798053a30 0x7f9798055ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:34.981 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.951+0000 7f97b2d9d700 1 --2- 172.21.15.70:0/4056701208 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9798053a30 0x7f9798055ef0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f97a400b000 tx=0x7f97a4006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:34.983 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:34.959+0000 7f97abfff700 1 -- 172.21.15.70:0/4056701208 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f97b40aa2b0 con 0x7f97b40b2200 2022-06-17T12:46:35.070 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.095+0000 7f97b883d700 1 -- 172.21.15.70:0/4056701208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd new", "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", "id": 5} v 0) v1 -- 0x7f97b40aa2b0 con 0x7f97b40b2200 2022-06-17T12:46:35.093 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.103+0000 7f97abfff700 1 -- 172.21.15.70:0/4056701208 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f97b40aa2b0 con 0x7f97b40b2200 2022-06-17T12:46:35.095 INFO:teuthology.orchestra.run.smithi070.stdout:5 2022-06-17T12:46:35.098 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.103+0000 7f97b883d700 1 -- 172.21.15.70:0/4056701208 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9798053a30 msgr2=0x7f9798055ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:35.099 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.103+0000 7f97b883d700 1 --2- 172.21.15.70:0/4056701208 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9798053a30 0x7f9798055ef0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f97a400b000 tx=0x7f97a4006df0).stop 2022-06-17T12:46:35.099 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.103+0000 7f97b883d700 1 -- 172.21.15.70:0/4056701208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97b40b2200 msgr2=0x7f97b414e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:35.099 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.103+0000 7f97b883d700 1 --2- 172.21.15.70:0/4056701208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97b40b2200 0x7f97b414e2a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f97ac00c000 tx=0x7f97ac0091d0).stop 2022-06-17T12:46:35.100 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.103+0000 7f97b883d700 1 -- 172.21.15.70:0/4056701208 shutdown_connections 2022-06-17T12:46:35.100 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.103+0000 7f97b883d700 1 --2- 172.21.15.70:0/4056701208 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9798053a30 0x7f9798055ef0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:35.101 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.103+0000 7f97b883d700 1 --2- 172.21.15.70:0/4056701208 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97b40b2cd0 0x7f97b41455e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:35.101 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.103+0000 7f97b883d700 1 --2- 172.21.15.70:0/4056701208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97b40b2200 0x7f97b414e2a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:35.102 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.103+0000 7f97b883d700 1 --2- 172.21.15.70:0/4056701208 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97b40af8d0 0x7f97b41565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:35.102 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.103+0000 7f97b883d700 1 -- 172.21.15.70:0/4056701208 >> 172.21.15.70:0/4056701208 conn(0x7f97b405f770 msgr2=0x7f97b40b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:35.103 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.103+0000 7f97b883d700 1 -- 172.21.15.70:0/4056701208 shutdown_connections 2022-06-17T12:46:35.103 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.103+0000 7f97b883d700 1 -- 172.21.15.70:0/4056701208 wait complete. 2022-06-17T12:46:35.104 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new f55cd87d-673d-4717-bde0-103a9204f75b 6 2022-06-17T12:46:35.239 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.243+0000 7fb299784700 1 Processor -- start 2022-06-17T12:46:35.241 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.243+0000 7fb299784700 1 -- start start 2022-06-17T12:46:35.241 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.243+0000 7fb299784700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb294062770 0x7fb294062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:35.241 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.243+0000 7fb299784700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb2940ad330 0x7fb2940ad780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:35.242 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.247+0000 7fb299784700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2940b0ba0 con 0x7fb2941444f0 2022-06-17T12:46:35.242 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.247+0000 7fb299784700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2940ae820 con 0x7fb2940ad330 2022-06-17T12:46:35.242 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.247+0000 7fb299784700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb294061030 con 0x7fb294062770 2022-06-17T12:46:35.243 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.247+0000 7fb2927fc700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb2941444f0 0x7fb294144930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36340/0 (socket says 172.21.15.70:36340) 2022-06-17T12:46:35.243 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.247+0000 7fb2927fc700 1 -- 172.21.15.70:0/2923831745 learned_addr learned my addr 172.21.15.70:0/2923831745 (peer_addr_for_me v1:172.21.15.70:0/0) 2022-06-17T12:46:35.244 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.247+0000 7fb2937fe700 1 --2- 172.21.15.70:0/2923831745 >> v2:172.21.15.70:3300/0 conn(0x7fb2940ad330 0x7fb2940ad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:35.244 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.247+0000 7fb292ffd700 1 --2- 172.21.15.70:0/2923831745 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb294062770 0x7fb294062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:35.245 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.247+0000 7fb2937fe700 1 -- 172.21.15.70:0/2923831745 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb294062770 msgr2=0x7fb294062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:35.245 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.247+0000 7fb2937fe700 1 --2- 172.21.15.70:0/2923831745 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb294062770 0x7fb294062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:35.245 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.247+0000 7fb2937fe700 1 -- 172.21.15.70:0/2923831745 >> v1:172.21.15.70:6789/0 conn(0x7fb2941444f0 legacy=0x7fb294144930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:35.246 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.247+0000 7fb291ffb700 1 -- 172.21.15.70:0/2923831745 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4094570014 0 0) 0x7fb2940b0ba0 con 0x7fb2941444f0 2022-06-17T12:46:35.246 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.247+0000 7fb2937fe700 1 -- 172.21.15.70:0/2923831745 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb294147e90 con 0x7fb2940ad330 2022-06-17T12:46:35.246 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.247+0000 7fb2937fe700 1 --2- 172.21.15.70:0/2923831745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2940ad330 0x7fb2940ad780 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fb284002f70 tx=0x7fb284006df0).ready entity=mon.1 client_cookie=591edc9d8dd1c00a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:35.247 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.247+0000 7fb291ffb700 1 -- 172.21.15.70:0/2923831745 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb28400d570 con 0x7fb2940ad330 2022-06-17T12:46:35.248 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.247+0000 7fb291ffb700 1 -- 172.21.15.70:0/2923831745 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb284000e90 con 0x7fb2940ad330 2022-06-17T12:46:35.248 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.247+0000 7fb291ffb700 1 -- 172.21.15.70:0/2923831745 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb28400d8c0 con 0x7fb2940ad330 2022-06-17T12:46:35.249 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.247+0000 7fb299784700 1 -- 172.21.15.70:0/2923831745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2940ad330 msgr2=0x7fb2940ad780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:35.250 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.247+0000 7fb299784700 1 --2- 172.21.15.70:0/2923831745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2940ad330 0x7fb2940ad780 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fb284002f70 tx=0x7fb284006df0).stop 2022-06-17T12:46:35.250 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.247+0000 7fb299784700 1 -- 172.21.15.70:0/2923831745 shutdown_connections 2022-06-17T12:46:35.251 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.247+0000 7fb299784700 1 --2- 172.21.15.70:0/2923831745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2940ad330 0x7fb2940ad780 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:35.251 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.247+0000 7fb299784700 1 --2- 172.21.15.70:0/2923831745 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb294062770 0x7fb294062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:35.251 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.247+0000 7fb299784700 1 -- 172.21.15.70:0/2923831745 >> 172.21.15.70:0/2923831745 conn(0x7fb2940b53e0 msgr2=0x7fb2940b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:35.252 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.247+0000 7fb299784700 1 -- 172.21.15.70:0/2923831745 shutdown_connections 2022-06-17T12:46:35.252 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.247+0000 7fb299784700 1 -- 172.21.15.70:0/2923831745 wait complete. 2022-06-17T12:46:35.255 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.251+0000 7fb299784700 1 Processor -- start 2022-06-17T12:46:35.256 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.251+0000 7fb299784700 1 -- start start 2022-06-17T12:46:35.257 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.251+0000 7fb299784700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb294062770 0x7fb294064830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:35.257 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.251+0000 7fb299784700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2940ad330 0x7fb29405fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:35.258 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.251+0000 7fb299784700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb2941444f0 0x7fb2941564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:35.260 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.251+0000 7fb299784700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb294159b60 con 0x7fb2941444f0 2022-06-17T12:46:35.261 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.251+0000 7fb299784700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2941569d0 con 0x7fb2940ad330 2022-06-17T12:46:35.261 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.251+0000 7fb299784700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb294063c30 con 0x7fb294062770 2022-06-17T12:46:35.261 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.251+0000 7fb292ffd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb294062770 0x7fb294064830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:35.261 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.251+0000 7fb2927fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2940ad330 0x7fb29405fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:35.262 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.251+0000 7fb292ffd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb294062770 0x7fb294064830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:60046/0 (socket says 172.21.15.70:60046) 2022-06-17T12:46:35.262 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.251+0000 7fb292ffd700 1 -- 172.21.15.70:0/935345043 learned_addr learned my addr 172.21.15.70:0/935345043 (peer_addr_for_me v2:172.21.15.70:0/0) 2022-06-17T12:46:35.263 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.251+0000 7fb2937fe700 1 --2- 172.21.15.70:0/935345043 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb2941444f0 0x7fb2941564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:35.263 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.251+0000 7fb292ffd700 1 -- 172.21.15.70:0/935345043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2940ad330 msgr2=0x7fb29405fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:35.263 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.251+0000 7fb292ffd700 1 --2- 172.21.15.70:0/935345043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2940ad330 0x7fb29405fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:35.263 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.251+0000 7fb292ffd700 1 -- 172.21.15.70:0/935345043 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb2941444f0 msgr2=0x7fb2941564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:35.264 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.251+0000 7fb292ffd700 1 --2- 172.21.15.70:0/935345043 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb2941444f0 0x7fb2941564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:35.264 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.251+0000 7fb292ffd700 1 -- 172.21.15.70:0/935345043 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28400b180 con 0x7fb294062770 2022-06-17T12:46:35.264 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.251+0000 7fb2937fe700 1 --2- 172.21.15.70:0/935345043 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb2941444f0 0x7fb2941564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:46:35.264 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.251+0000 7fb292ffd700 1 --2- 172.21.15.70:0/935345043 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb294062770 0x7fb294064830 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fb28c00c000 tx=0x7fb28c006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:35.265 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.255+0000 7fb28bfff700 1 -- 172.21.15.70:0/935345043 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb28c010780 con 0x7fb294062770 2022-06-17T12:46:35.265 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.255+0000 7fb28bfff700 1 -- 172.21.15.70:0/935345043 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28c0108e0 con 0x7fb294062770 2022-06-17T12:46:35.265 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.255+0000 7fb299784700 1 -- 172.21.15.70:0/935345043 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2940636f0 con 0x7fb294062770 2022-06-17T12:46:35.265 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.255+0000 7fb28bfff700 1 -- 172.21.15.70:0/935345043 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb28c010c40 con 0x7fb294062770 2022-06-17T12:46:35.266 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.255+0000 7fb299784700 1 -- 172.21.15.70:0/935345043 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2941460e0 con 0x7fb294062770 2022-06-17T12:46:35.266 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.255+0000 7fb299784700 1 -- 172.21.15.70:0/935345043 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb294159cd0 con 0x7fb294062770 2022-06-17T12:46:35.267 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.255+0000 7fb28bfff700 1 -- 172.21.15.70:0/935345043 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 3) v1 ==== 65884+0+0 (secure 0 0 0) 0x7fb28c01c460 con 0x7fb294062770 2022-06-17T12:46:35.267 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.255+0000 7fb28bfff700 1 --2- 172.21.15.70:0/935345043 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb2780539a0 0x7fb278055e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:35.267 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.255+0000 7fb28bfff700 1 -- 172.21.15.70:0/935345043 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1844+0+0 (secure 0 0 0) 0x7fb28c06b340 con 0x7fb294062770 2022-06-17T12:46:35.267 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.255+0000 7fb2927fc700 1 --2- 172.21.15.70:0/935345043 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb2780539a0 0x7fb278055e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:35.268 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.259+0000 7fb2927fc700 1 --2- 172.21.15.70:0/935345043 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb2780539a0 0x7fb278055e60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fb27c00d000 tx=0x7fb27c001360).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:35.268 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.263+0000 7fb28bfff700 1 -- 172.21.15.70:0/935345043 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb28c0431d0 con 0x7fb294062770 2022-06-17T12:46:35.380 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.399+0000 7fb299784700 1 -- 172.21.15.70:0/935345043 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", "id": 6} v 0) v1 -- 0x7fb2940b1c20 con 0x7fb294062770 2022-06-17T12:46:35.383 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.407+0000 7fb28bfff700 1 -- 172.21.15.70:0/935345043 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb28c0467f0 con 0x7fb294062770 2022-06-17T12:46:35.385 INFO:teuthology.orchestra.run.smithi070.stdout:6 2022-06-17T12:46:35.386 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.411+0000 7fb299784700 1 -- 172.21.15.70:0/935345043 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb2780539a0 msgr2=0x7fb278055e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:35.387 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.411+0000 7fb299784700 1 --2- 172.21.15.70:0/935345043 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb2780539a0 0x7fb278055e60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fb27c00d000 tx=0x7fb27c001360).stop 2022-06-17T12:46:35.387 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.411+0000 7fb299784700 1 -- 172.21.15.70:0/935345043 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb294062770 msgr2=0x7fb294064830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:35.387 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.411+0000 7fb299784700 1 --2- 172.21.15.70:0/935345043 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb294062770 0x7fb294064830 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fb28c00c000 tx=0x7fb28c006df0).stop 2022-06-17T12:46:35.392 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.411+0000 7fb299784700 1 -- 172.21.15.70:0/935345043 shutdown_connections 2022-06-17T12:46:35.393 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.411+0000 7fb299784700 1 --2- 172.21.15.70:0/935345043 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb2780539a0 0x7fb278055e60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:35.394 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.411+0000 7fb299784700 1 --2- 172.21.15.70:0/935345043 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb2941444f0 0x7fb2941564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:35.394 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.411+0000 7fb299784700 1 --2- 172.21.15.70:0/935345043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2940ad330 0x7fb29405fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:35.394 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.411+0000 7fb299784700 1 --2- 172.21.15.70:0/935345043 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb294062770 0x7fb294064830 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:35.395 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.411+0000 7fb299784700 1 -- 172.21.15.70:0/935345043 >> 172.21.15.70:0/935345043 conn(0x7fb2940b53e0 msgr2=0x7fb2940af400 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:35.395 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.411+0000 7fb299784700 1 -- 172.21.15.70:0/935345043 shutdown_connections 2022-06-17T12:46:35.395 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.411+0000 7fb299784700 1 -- 172.21.15.70:0/935345043 wait complete. 2022-06-17T12:46:35.398 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new cbf2a5fd-464a-4b16-872b-d86fbe56f84c 7 2022-06-17T12:46:35.507 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.527+0000 7f8f2a13c700 1 Processor -- start 2022-06-17T12:46:35.509 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.527+0000 7f8f2a13c700 1 -- start start 2022-06-17T12:46:35.543 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.527+0000 7f8f2a13c700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8f240b2cd0 0x7f8f241465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:35.544 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.527+0000 7f8f2a13c700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f240b25e0 con 0x7f8f240b2200 2022-06-17T12:46:35.544 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.527+0000 7f8f2a13c700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f240ad790 con 0x7f8f24146ae0 2022-06-17T12:46:35.553 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.527+0000 7f8f2a13c700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f240aef40 con 0x7f8f240b2cd0 2022-06-17T12:46:35.554 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.527+0000 7f8f2993b700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8f24146ae0 0x7f8f24146f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36356/0 (socket says 172.21.15.70:36356) 2022-06-17T12:46:35.554 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.527+0000 7f8f28939700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8f240b2cd0 0x7f8f241465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:35.554 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.527+0000 7f8f2993b700 1 -- 172.21.15.70:0/1548132064 learned_addr learned my addr 172.21.15.70:0/1548132064 (peer_addr_for_me v1:172.21.15.70:0/0) 2022-06-17T12:46:35.555 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.531+0000 7f8f1bfff700 1 -- 172.21.15.70:0/1548132064 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 349851373 0 0) 0x7f8f240ad790 con 0x7f8f24146ae0 2022-06-17T12:46:35.556 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.531+0000 7f8f1bfff700 1 -- 172.21.15.70:0/1548132064 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f08003260 con 0x7f8f24146ae0 2022-06-17T12:46:35.556 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.531+0000 7f8f1bfff700 1 -- 172.21.15.70:0/1548132064 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3293347911 0 0) 0x7f8f240b25e0 con 0x7f8f240b2200 2022-06-17T12:46:35.557 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.531+0000 7f8f28939700 1 -- 172.21.15.70:0/1548132064 >> v1:172.21.15.70:6789/0 conn(0x7f8f24146ae0 legacy=0x7f8f24146f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:35.557 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.531+0000 7f8f28939700 1 -- 172.21.15.70:0/1548132064 >> v1:172.21.15.36:6789/0 conn(0x7f8f240b2200 legacy=0x7f8f240af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:35.557 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.531+0000 7f8f28939700 1 -- 172.21.15.70:0/1548132064 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f240afaf0 con 0x7f8f240b2cd0 2022-06-17T12:46:35.558 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.531+0000 7f8f28939700 1 --2- 172.21.15.70:0/1548132064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f240b2cd0 0x7f8f241465f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f8f14002f70 tx=0x7f8f14006df0).ready entity=mon.1 client_cookie=3e24bf264470b7ac server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:35.558 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.531+0000 7f8f1bfff700 1 -- 172.21.15.70:0/1548132064 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f1400d610 con 0x7f8f240b2cd0 2022-06-17T12:46:35.558 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.531+0000 7f8f1bfff700 1 -- 172.21.15.70:0/1548132064 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f14000e90 con 0x7f8f240b2cd0 2022-06-17T12:46:35.559 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.531+0000 7f8f1bfff700 1 -- 172.21.15.70:0/1548132064 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f1400d960 con 0x7f8f240b2cd0 2022-06-17T12:46:35.559 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.531+0000 7f8f2a13c700 1 -- 172.21.15.70:0/1548132064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f240b2cd0 msgr2=0x7f8f241465f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:35.559 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.531+0000 7f8f2a13c700 1 --2- 172.21.15.70:0/1548132064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f240b2cd0 0x7f8f241465f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f8f14002f70 tx=0x7f8f14006df0).stop 2022-06-17T12:46:35.560 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.531+0000 7f8f2a13c700 1 -- 172.21.15.70:0/1548132064 shutdown_connections 2022-06-17T12:46:35.560 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.531+0000 7f8f2a13c700 1 --2- 172.21.15.70:0/1548132064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f240b2cd0 0x7f8f241465f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:35.560 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.531+0000 7f8f2a13c700 1 -- 172.21.15.70:0/1548132064 >> 172.21.15.70:0/1548132064 conn(0x7f8f2405f770 msgr2=0x7f8f2405fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:35.561 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.531+0000 7f8f2a13c700 1 -- 172.21.15.70:0/1548132064 shutdown_connections 2022-06-17T12:46:35.561 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.531+0000 7f8f2a13c700 1 -- 172.21.15.70:0/1548132064 wait complete. 2022-06-17T12:46:35.562 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.531+0000 7f8f2a13c700 1 Processor -- start 2022-06-17T12:46:35.562 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.531+0000 7f8f2a13c700 1 -- start start 2022-06-17T12:46:35.562 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.535+0000 7f8f2a13c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f240b2200 0x7f8f24145630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:35.562 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.535+0000 7f8f2a13c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f240b2cd0 0x7f8f240a8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:35.562 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.535+0000 7f8f2a13c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f24146ae0 0x7f8f24157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:35.563 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.535+0000 7f8f2a13c700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f24063660 con 0x7f8f240b2cd0 2022-06-17T12:46:35.569 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.535+0000 7f8f2a13c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f24156930 con 0x7f8f240b2200 2022-06-17T12:46:35.570 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.535+0000 7f8f2a13c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f24156bd0 con 0x7f8f24146ae0 2022-06-17T12:46:35.570 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.535+0000 7f8f2913a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f240b2200 0x7f8f24145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:35.571 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.535+0000 7f8f2913a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f240b2200 0x7f8f24145630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44986/0 (socket says 172.21.15.70:44986) 2022-06-17T12:46:35.576 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.535+0000 7f8f2913a700 1 -- 172.21.15.70:0/3238343408 learned_addr learned my addr 172.21.15.70:0/3238343408 (peer_addr_for_me v2:172.21.15.70:0/0) 2022-06-17T12:46:35.576 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.535+0000 7f8f2993b700 1 --2- 172.21.15.70:0/3238343408 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f24146ae0 0x7f8f24157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:35.577 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.535+0000 7f8f28939700 1 --2- 172.21.15.70:0/3238343408 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f240b2cd0 0x7f8f240a8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:35.577 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.535+0000 7f8f2913a700 1 -- 172.21.15.70:0/3238343408 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f24146ae0 msgr2=0x7f8f24157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:35.578 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.535+0000 7f8f2913a700 1 --2- 172.21.15.70:0/3238343408 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f24146ae0 0x7f8f24157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:35.578 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.535+0000 7f8f2913a700 1 -- 172.21.15.70:0/3238343408 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f240b2cd0 msgr2=0x7f8f240a8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:35.579 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.535+0000 7f8f2913a700 1 --2- 172.21.15.70:0/3238343408 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f240b2cd0 0x7f8f240a8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:35.579 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.535+0000 7f8f2913a700 1 -- 172.21.15.70:0/3238343408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1400b180 con 0x7f8f240b2200 2022-06-17T12:46:35.579 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.535+0000 7f8f2913a700 1 --2- 172.21.15.70:0/3238343408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f240b2200 0x7f8f24145630 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f8f20000bc0 tx=0x7f8f20007a70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:35.580 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.535+0000 7f8f1a7fc700 1 -- 172.21.15.70:0/3238343408 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f20012400 con 0x7f8f240b2200 2022-06-17T12:46:35.580 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.535+0000 7f8f1a7fc700 1 -- 172.21.15.70:0/3238343408 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f20012560 con 0x7f8f240b2200 2022-06-17T12:46:35.580 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.535+0000 7f8f2a13c700 1 -- 172.21.15.70:0/3238343408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f24145b20 con 0x7f8f240b2200 2022-06-17T12:46:35.581 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.535+0000 7f8f1a7fc700 1 -- 172.21.15.70:0/3238343408 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f200128c0 con 0x7f8f240b2200 2022-06-17T12:46:35.581 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.535+0000 7f8f2a13c700 1 -- 172.21.15.70:0/3238343408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f240a8560 con 0x7f8f240b2200 2022-06-17T12:46:35.581 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.535+0000 7f8f2a13c700 1 -- 172.21.15.70:0/3238343408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f240634d0 con 0x7f8f240b2200 2022-06-17T12:46:35.582 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.539+0000 7f8f1a7fc700 1 -- 172.21.15.70:0/3238343408 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 3) v1 ==== 65884+0+0 (secure 0 0 0) 0x7f8f20019030 con 0x7f8f240b2200 2022-06-17T12:46:35.582 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.539+0000 7f8f1a7fc700 1 --2- 172.21.15.70:0/3238343408 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8f0c0539d0 0x7f8f0c055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:35.583 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.539+0000 7f8f1a7fc700 1 -- 172.21.15.70:0/3238343408 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1959+0+0 (secure 0 0 0) 0x7f8f20069f00 con 0x7f8f240b2200 2022-06-17T12:46:35.583 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.539+0000 7f8f28939700 1 --2- 172.21.15.70:0/3238343408 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8f0c0539d0 0x7f8f0c055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:35.583 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.539+0000 7f8f28939700 1 --2- 172.21.15.70:0/3238343408 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8f0c0539d0 0x7f8f0c055e90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f8f14018000 tx=0x7f8f14014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:35.584 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.543+0000 7f8f1a7fc700 1 -- 172.21.15.70:0/3238343408 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8f20041fc0 con 0x7f8f240b2200 2022-06-17T12:46:35.658 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.679+0000 7f8f2a13c700 1 -- 172.21.15.70:0/3238343408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", "id": 7} v 0) v1 -- 0x7f8f24156d90 con 0x7f8f240b2200 2022-06-17T12:46:35.668 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.691+0000 7f8f1a7fc700 1 -- 172.21.15.70:0/3238343408 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8f200455e0 con 0x7f8f240b2200 2022-06-17T12:46:35.669 INFO:teuthology.orchestra.run.smithi070.stdout:7 2022-06-17T12:46:35.671 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.695+0000 7f8f2a13c700 1 -- 172.21.15.70:0/3238343408 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8f0c0539d0 msgr2=0x7f8f0c055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:35.674 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.695+0000 7f8f2a13c700 1 --2- 172.21.15.70:0/3238343408 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8f0c0539d0 0x7f8f0c055e90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f8f14018000 tx=0x7f8f14014000).stop 2022-06-17T12:46:35.674 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.695+0000 7f8f2a13c700 1 -- 172.21.15.70:0/3238343408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f240b2200 msgr2=0x7f8f24145630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:35.675 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.695+0000 7f8f2a13c700 1 --2- 172.21.15.70:0/3238343408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f240b2200 0x7f8f24145630 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f8f20000bc0 tx=0x7f8f20007a70).stop 2022-06-17T12:46:35.675 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.695+0000 7f8f2a13c700 1 -- 172.21.15.70:0/3238343408 shutdown_connections 2022-06-17T12:46:35.675 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.695+0000 7f8f2a13c700 1 --2- 172.21.15.70:0/3238343408 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8f0c0539d0 0x7f8f0c055e90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:35.675 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.695+0000 7f8f2a13c700 1 --2- 172.21.15.70:0/3238343408 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f24146ae0 0x7f8f24157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:35.676 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.695+0000 7f8f2a13c700 1 --2- 172.21.15.70:0/3238343408 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f240b2cd0 0x7f8f240a8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:35.677 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.695+0000 7f8f2a13c700 1 --2- 172.21.15.70:0/3238343408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f240b2200 0x7f8f24145630 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:35.677 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.695+0000 7f8f2a13c700 1 -- 172.21.15.70:0/3238343408 >> 172.21.15.70:0/3238343408 conn(0x7f8f2405f770 msgr2=0x7f8f240b0200 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:35.677 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.695+0000 7f8f2a13c700 1 -- 172.21.15.70:0/3238343408 shutdown_connections 2022-06-17T12:46:35.678 INFO:teuthology.orchestra.run.smithi070.stderr:2022-06-17T12:46:35.695+0000 7f8f2a13c700 1 -- 172.21.15.70:0/3238343408 wait complete. 2022-06-17T12:46:35.683 INFO:tasks.ceph.osd.0:Restarting daemon 2022-06-17T12:46:35.683 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2022-06-17T12:46:35.688 INFO:tasks.ceph.osd.0:Started 2022-06-17T12:46:35.689 INFO:tasks.ceph.osd.1:Restarting daemon 2022-06-17T12:46:35.690 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2022-06-17T12:46:35.712 INFO:tasks.ceph.osd.1:Started 2022-06-17T12:46:35.713 INFO:tasks.ceph.osd.2:Restarting daemon 2022-06-17T12:46:35.713 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2022-06-17T12:46:35.718 INFO:tasks.ceph.osd.2:Started 2022-06-17T12:46:35.719 INFO:tasks.ceph.osd.3:Restarting daemon 2022-06-17T12:46:35.719 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2022-06-17T12:46:35.745 INFO:tasks.ceph.osd.3:Started 2022-06-17T12:46:35.745 INFO:tasks.ceph.osd.4:Restarting daemon 2022-06-17T12:46:35.748 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2022-06-17T12:46:35.755 INFO:tasks.ceph.osd.4:Started 2022-06-17T12:46:35.759 INFO:tasks.ceph.osd.5:Restarting daemon 2022-06-17T12:46:35.760 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2022-06-17T12:46:35.763 INFO:tasks.ceph.osd.5:Started 2022-06-17T12:46:35.764 INFO:tasks.ceph.osd.6:Restarting daemon 2022-06-17T12:46:35.765 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2022-06-17T12:46:35.769 INFO:tasks.ceph.osd.6:Started 2022-06-17T12:46:35.772 INFO:tasks.ceph.osd.7:Restarting daemon 2022-06-17T12:46:35.772 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2022-06-17T12:46:35.776 INFO:tasks.ceph.osd.7:Started 2022-06-17T12:46:35.777 INFO:tasks.ceph:Waiting for OSDs to come up 2022-06-17T12:46:36.072 INFO:tasks.ceph.osd.0.smithi036.stderr:2022-06-17T12:46:36.085+0000 7fa502bbdd80 -1 Falling back to public interface 2022-06-17T12:46:36.119 INFO:tasks.ceph.osd.5.smithi070.stderr:2022-06-17T12:46:36.143+0000 7fdd21783d80 -1 Falling back to public interface 2022-06-17T12:46:36.146 INFO:tasks.ceph.osd.4.smithi070.stderr:2022-06-17T12:46:36.163+0000 7f05aae2ad80 -1 Falling back to public interface 2022-06-17T12:46:36.333 INFO:tasks.ceph.osd.1.smithi036.stderr:2022-06-17T12:46:36.349+0000 7fb2c6e5bd80 -1 Falling back to public interface 2022-06-17T12:46:36.341 INFO:tasks.ceph.osd.3.smithi036.stderr:2022-06-17T12:46:36.357+0000 7fe040331d80 -1 Falling back to public interface 2022-06-17T12:46:36.363 INFO:tasks.ceph.osd.2.smithi036.stderr:2022-06-17T12:46:36.369+0000 7f4cd9ce6d80 -1 Falling back to public interface 2022-06-17T12:46:36.396 INFO:tasks.ceph.osd.6.smithi070.stderr:2022-06-17T12:46:36.407+0000 7f1cdba9dd80 -1 Falling back to public interface 2022-06-17T12:46:36.397 INFO:tasks.ceph.osd.7.smithi070.stderr:2022-06-17T12:46:36.415+0000 7facc50d2d80 -1 Falling back to public interface 2022-06-17T12:46:36.598 DEBUG:teuthology.orchestra.run.smithi036:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-06-17T12:46:36.734 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.745+0000 7f92da2cb700 1 Processor -- start 2022-06-17T12:46:36.735 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.745+0000 7f92da2cb700 1 -- start start 2022-06-17T12:46:36.735 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92da2cb700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92d40af5e0 0x7f92d4140350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:36.736 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92da2cb700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f92d40b2220 0x7f92d4140840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:36.737 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92da2cb700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92d40af9c0 con 0x7f92d40b1250 2022-06-17T12:46:36.739 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92da2cb700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92d4063080 con 0x7f92d40b2220 2022-06-17T12:46:36.739 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92da2cb700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d4064080 con 0x7f92d40af5e0 2022-06-17T12:46:36.741 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92d37fe700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f92d40b2220 0x7f92d4140840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:36.741 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92d8868700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f92d40b1250 0x7f92d4144460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:44040/0 (socket says 172.21.15.36:44040) 2022-06-17T12:46:36.742 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92d8868700 1 -- 172.21.15.36:0/578955563 learned_addr learned my addr 172.21.15.36:0/578955563 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:46:36.742 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92d3fff700 1 --2- 172.21.15.36:0/578955563 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92d40af5e0 0x7f92d4140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:36.753 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92d37fe700 1 -- 172.21.15.36:0/578955563 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92d40af5e0 msgr2=0x7f92d4140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:36.754 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92d37fe700 1 --2- 172.21.15.36:0/578955563 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92d40af5e0 0x7f92d4140350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:36.754 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92d37fe700 1 -- 172.21.15.36:0/578955563 >> v1:172.21.15.70:6789/0 conn(0x7f92d40b1250 legacy=0x7f92d4144460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:36.755 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92d37fe700 1 -- 172.21.15.36:0/578955563 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d4064320 con 0x7f92d40b2220 2022-06-17T12:46:36.755 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92d37fe700 1 --2- 172.21.15.36:0/578955563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92d40b2220 0x7f92d4140840 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f92cc002f70 tx=0x7f92cc006df0).ready entity=mon.1 client_cookie=9a55664650080dbb server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:36.755 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92d2ffd700 1 -- 172.21.15.36:0/578955563 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92cc00d570 con 0x7f92d40b2220 2022-06-17T12:46:36.756 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92d2ffd700 1 -- 172.21.15.36:0/578955563 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92cc000e90 con 0x7f92d40b2220 2022-06-17T12:46:36.756 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92d2ffd700 1 -- 172.21.15.36:0/578955563 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92cc00d8c0 con 0x7f92d40b2220 2022-06-17T12:46:36.757 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92da2cb700 1 -- 172.21.15.36:0/578955563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92d40b2220 msgr2=0x7f92d4140840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:36.757 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92da2cb700 1 --2- 172.21.15.36:0/578955563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92d40b2220 0x7f92d4140840 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f92cc002f70 tx=0x7f92cc006df0).stop 2022-06-17T12:46:36.762 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92da2cb700 1 -- 172.21.15.36:0/578955563 shutdown_connections 2022-06-17T12:46:36.762 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92da2cb700 1 --2- 172.21.15.36:0/578955563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92d40b2220 0x7f92d4140840 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:36.764 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92da2cb700 1 --2- 172.21.15.36:0/578955563 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92d40af5e0 0x7f92d4140350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:36.764 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92da2cb700 1 -- 172.21.15.36:0/578955563 >> 172.21.15.36:0/578955563 conn(0x7f92d40b5c50 msgr2=0x7f92d40b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:36.765 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92da2cb700 1 -- 172.21.15.36:0/578955563 shutdown_connections 2022-06-17T12:46:36.765 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.749+0000 7f92da2cb700 1 -- 172.21.15.36:0/578955563 wait complete. 2022-06-17T12:46:36.766 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.753+0000 7f92da2cb700 1 Processor -- start 2022-06-17T12:46:36.766 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.753+0000 7f92da2cb700 1 -- start start 2022-06-17T12:46:36.767 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.753+0000 7f92da2cb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92d40af5e0 0x7f92d409f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:36.767 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.753+0000 7f92da2cb700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92d40b1250 0x7f92d4142790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:36.768 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.753+0000 7f92da2cb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f92d40b2220 0x7f92d41501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:36.768 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.753+0000 7f92da2cb700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d40acdd0 con 0x7f92d40b2220 2022-06-17T12:46:36.769 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.753+0000 7f92da2cb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d409eb90 con 0x7f92d40af5e0 2022-06-17T12:46:36.769 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.753+0000 7f92da2cb700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d409ee30 con 0x7f92d40b1250 2022-06-17T12:46:36.770 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.753+0000 7f92d8868700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f92d40b2220 0x7f92d41501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:36.770 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.753+0000 7f92d3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92d40af5e0 0x7f92d409f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:36.771 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.753+0000 7f92d37fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92d40b1250 0x7f92d4142790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:36.771 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.753+0000 7f92d3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92d40af5e0 0x7f92d409f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:51160/0 (socket says 172.21.15.36:51160) 2022-06-17T12:46:36.772 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.753+0000 7f92d3fff700 1 -- 172.21.15.36:0/4212539444 learned_addr learned my addr 172.21.15.36:0/4212539444 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:36.772 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.753+0000 7f92d3fff700 1 -- 172.21.15.36:0/4212539444 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92d40b1250 msgr2=0x7f92d4142790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:36.774 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.753+0000 7f92d3fff700 1 --2- 172.21.15.36:0/4212539444 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92d40b1250 0x7f92d4142790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:36.775 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.753+0000 7f92d3fff700 1 -- 172.21.15.36:0/4212539444 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f92d40b2220 msgr2=0x7f92d41501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:36.775 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.753+0000 7f92d3fff700 1 --2- 172.21.15.36:0/4212539444 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f92d40b2220 0x7f92d41501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:36.775 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.753+0000 7f92d3fff700 1 -- 172.21.15.36:0/4212539444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92cc00b180 con 0x7f92d40af5e0 2022-06-17T12:46:36.776 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.753+0000 7f92d3fff700 1 --2- 172.21.15.36:0/4212539444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92d40af5e0 0x7f92d409f570 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f92c800c000 tx=0x7f92c8006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:36.776 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.757+0000 7f92d17fa700 1 -- 172.21.15.36:0/4212539444 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92c8010780 con 0x7f92d40af5e0 2022-06-17T12:46:36.777 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.757+0000 7f92d17fa700 1 -- 172.21.15.36:0/4212539444 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c80108e0 con 0x7f92d40af5e0 2022-06-17T12:46:36.777 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.757+0000 7f92d17fa700 1 -- 172.21.15.36:0/4212539444 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92c8010c40 con 0x7f92d40af5e0 2022-06-17T12:46:36.777 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.757+0000 7f92da2cb700 1 -- 172.21.15.36:0/4212539444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d409f0d0 con 0x7f92d40af5e0 2022-06-17T12:46:36.778 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.757+0000 7f92da2cb700 1 -- 172.21.15.36:0/4212539444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d4142c80 con 0x7f92d40af5e0 2022-06-17T12:46:36.778 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.757+0000 7f92d17fa700 1 -- 172.21.15.36:0/4212539444 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f92c801c460 con 0x7f92d40af5e0 2022-06-17T12:46:36.779 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.757+0000 7f92d17fa700 1 --2- 172.21.15.36:0/4212539444 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f92b4053af0 0x7f92b4055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:36.779 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.757+0000 7f92d17fa700 1 -- 172.21.15.36:0/4212539444 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 2074+0+0 (secure 0 0 0) 0x7f92c806b390 con 0x7f92d40af5e0 2022-06-17T12:46:36.779 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.757+0000 7f92d37fe700 1 --2- 172.21.15.36:0/4212539444 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f92b4053af0 0x7f92b4055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:36.780 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.757+0000 7f92d37fe700 1 --2- 172.21.15.36:0/4212539444 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f92b4053af0 0x7f92b4055fb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f92cc01a000 tx=0x7f92cc014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:36.780 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.761+0000 7f92da2cb700 1 -- 172.21.15.36:0/4212539444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92d409f260 con 0x7f92d40af5e0 2022-06-17T12:46:36.784 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.769+0000 7f92d17fa700 1 -- 172.21.15.36:0/4212539444 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f92c8043120 con 0x7f92d40af5e0 2022-06-17T12:46:36.931 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.945+0000 7f92da2cb700 1 -- 172.21.15.36:0/4212539444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92d40645c0 con 0x7f92d40af5e0 2022-06-17T12:46:36.937 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.949+0000 7f92d17fa700 1 -- 172.21.15.36:0/4212539444 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+8274 (secure 0 0 0) 0x7f92c8046740 con 0x7f92d40af5e0 2022-06-17T12:46:36.938 INFO:teuthology.misc.health.smithi036.stdout: 2022-06-17T12:46:36.938 INFO:teuthology.misc.health.smithi036.stdout:{"epoch":10,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:46:35.681900+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"5","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:46:36.943 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.949+0000 7f92da2cb700 1 -- 172.21.15.36:0/4212539444 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f92b4053af0 msgr2=0x7f92b4055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:36.943 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.949+0000 7f92da2cb700 1 --2- 172.21.15.36:0/4212539444 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f92b4053af0 0x7f92b4055fb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f92cc01a000 tx=0x7f92cc014000).stop 2022-06-17T12:46:36.944 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.949+0000 7f92da2cb700 1 -- 172.21.15.36:0/4212539444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92d40af5e0 msgr2=0x7f92d409f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:36.944 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.949+0000 7f92da2cb700 1 --2- 172.21.15.36:0/4212539444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92d40af5e0 0x7f92d409f570 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f92c800c000 tx=0x7f92c8006df0).stop 2022-06-17T12:46:36.944 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.949+0000 7f92da2cb700 1 -- 172.21.15.36:0/4212539444 shutdown_connections 2022-06-17T12:46:36.945 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.949+0000 7f92da2cb700 1 --2- 172.21.15.36:0/4212539444 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f92b4053af0 0x7f92b4055fb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:36.945 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.949+0000 7f92da2cb700 1 --2- 172.21.15.36:0/4212539444 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f92d40b2220 0x7f92d41501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:36.946 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.949+0000 7f92da2cb700 1 --2- 172.21.15.36:0/4212539444 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92d40b1250 0x7f92d4142790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:36.946 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.949+0000 7f92da2cb700 1 --2- 172.21.15.36:0/4212539444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92d40af5e0 0x7f92d409f570 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:36.946 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.949+0000 7f92da2cb700 1 -- 172.21.15.36:0/4212539444 >> 172.21.15.36:0/4212539444 conn(0x7f92d40b5c50 msgr2=0x7f92d4140b30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:36.946 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.949+0000 7f92da2cb700 1 -- 172.21.15.36:0/4212539444 shutdown_connections 2022-06-17T12:46:36.947 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:36.949+0000 7f92da2cb700 1 -- 172.21.15.36:0/4212539444 wait complete. 2022-06-17T12:46:36.948 DEBUG:teuthology.misc:0 of 8 OSDs are up 2022-06-17T12:46:37.249 INFO:tasks.ceph.osd.0.smithi036.stderr:2022-06-17T12:46:37.257+0000 7fa502bbdd80 -1 osd.0 0 log_to_monitors {default=true} 2022-06-17T12:46:37.316 INFO:tasks.ceph.osd.5.smithi070.stderr:2022-06-17T12:46:37.339+0000 7fdd21783d80 -1 osd.5 0 log_to_monitors {default=true} 2022-06-17T12:46:37.387 INFO:tasks.ceph.osd.4.smithi070.stderr:2022-06-17T12:46:37.403+0000 7f05aae2ad80 -1 osd.4 0 log_to_monitors {default=true} 2022-06-17T12:46:37.497 INFO:tasks.ceph.osd.1.smithi036.stderr:2022-06-17T12:46:37.517+0000 7fb2c6e5bd80 -1 osd.1 0 log_to_monitors {default=true} 2022-06-17T12:46:37.523 INFO:tasks.ceph.osd.3.smithi036.stderr:2022-06-17T12:46:37.545+0000 7fe040331d80 -1 osd.3 0 log_to_monitors {default=true} 2022-06-17T12:46:37.558 INFO:tasks.ceph.osd.6.smithi070.stderr:2022-06-17T12:46:37.575+0000 7f1cdba9dd80 -1 osd.6 0 log_to_monitors {default=true} 2022-06-17T12:46:37.567 INFO:tasks.ceph.osd.2.smithi036.stderr:2022-06-17T12:46:37.581+0000 7f4cd9ce6d80 -1 osd.2 0 log_to_monitors {default=true} 2022-06-17T12:46:37.569 INFO:tasks.ceph.osd.7.smithi070.stderr:2022-06-17T12:46:37.587+0000 7facc50d2d80 -1 osd.7 0 log_to_monitors {default=true} 2022-06-17T12:46:38.552 INFO:tasks.ceph.osd.5.smithi070.stderr:2022-06-17T12:46:38.575+0000 7fdd1382b700 -1 osd.5 0 waiting for initial osdmap 2022-06-17T12:46:38.554 INFO:tasks.ceph.osd.1.smithi036.stderr:2022-06-17T12:46:38.573+0000 7fb2b8f03700 -1 osd.1 0 waiting for initial osdmap 2022-06-17T12:46:38.558 INFO:tasks.ceph.osd.0.smithi036.stderr:2022-06-17T12:46:38.577+0000 7fa4f4c65700 -1 osd.0 0 waiting for initial osdmap 2022-06-17T12:46:38.560 INFO:tasks.ceph.osd.4.smithi070.stderr:2022-06-17T12:46:38.583+0000 7f059ced2700 -1 osd.4 0 waiting for initial osdmap 2022-06-17T12:46:38.561 INFO:tasks.ceph.osd.5.smithi070.stderr:2022-06-17T12:46:38.587+0000 7fdd1aca7700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-06-17T12:46:38.569 INFO:tasks.ceph.osd.1.smithi036.stderr:2022-06-17T12:46:38.581+0000 7fb2c037f700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-06-17T12:46:38.570 INFO:tasks.ceph.osd.0.smithi036.stderr:2022-06-17T12:46:38.585+0000 7fa4fc0e1700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-06-17T12:46:38.584 INFO:tasks.ceph.osd.4.smithi070.stderr:2022-06-17T12:46:38.591+0000 7f05a434e700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-06-17T12:46:39.586 INFO:tasks.ceph.osd.2.smithi036.stderr:2022-06-17T12:46:39.585+0000 7f4ccbd8e700 -1 osd.2 0 waiting for initial osdmap 2022-06-17T12:46:39.587 INFO:tasks.ceph.osd.6.smithi070.stderr:2022-06-17T12:46:39.591+0000 7f1ccdb45700 -1 osd.6 0 waiting for initial osdmap 2022-06-17T12:46:39.588 INFO:tasks.ceph.osd.3.smithi036.stderr:2022-06-17T12:46:39.589+0000 7fe0323d9700 -1 osd.3 0 waiting for initial osdmap 2022-06-17T12:46:39.589 INFO:tasks.ceph.osd.7.smithi070.stderr:2022-06-17T12:46:39.591+0000 7facb717a700 -1 osd.7 0 waiting for initial osdmap 2022-06-17T12:46:39.593 INFO:tasks.ceph.osd.2.smithi036.stderr:2022-06-17T12:46:39.593+0000 7f4cd320a700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-06-17T12:46:39.600 INFO:tasks.ceph.osd.6.smithi070.stderr:2022-06-17T12:46:39.599+0000 7f1cd4fc1700 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-06-17T12:46:39.601 INFO:tasks.ceph.osd.3.smithi036.stderr:2022-06-17T12:46:39.597+0000 7fe039855700 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-06-17T12:46:39.602 INFO:tasks.ceph.osd.7.smithi070.stderr:2022-06-17T12:46:39.599+0000 7facbe5f6700 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-06-17T12:46:43.238 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:46:43.308 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:46:43.317 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:46:43.356 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:46:43.418 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:46:43.802 DEBUG:teuthology.orchestra.run.smithi036:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-06-17T12:46:43.920 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.920+0000 7fb6d7fff700 1 Processor -- start 2022-06-17T12:46:43.921 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.920+0000 7fb6d7fff700 1 -- start start 2022-06-17T12:46:43.927 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.920+0000 7fb6d7fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb6d80b4200 0x7fb6d80b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:43.928 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.920+0000 7fb6d7fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb6d80b2a30 0x7fb6d8142f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:43.928 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.920+0000 7fb6d7fff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d8143410 con 0x7fb6d80b25a0 2022-06-17T12:46:43.928 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.920+0000 7fb6d7fff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d8060a70 con 0x7fb6d80b2a30 2022-06-17T12:46:43.929 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.920+0000 7fb6d7fff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d8146f10 con 0x7fb6d80b4200 2022-06-17T12:46:43.929 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.920+0000 7fb6d77fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb6d80b2a30 0x7fb6d8142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:43.930 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.920+0000 7fb6d77fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb6d80b2a30 0x7fb6d8142f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59236/0 (socket says 172.21.15.36:59236) 2022-06-17T12:46:43.930 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.920+0000 7fb6d77fe700 1 -- 172.21.15.36:0/2291488906 learned_addr learned my addr 172.21.15.36:0/2291488906 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:43.931 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.920+0000 7fb6d67fc700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fb6d80b25a0 0x7fb6d80ad340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:48532/0 (socket says 172.21.15.36:48532) 2022-06-17T12:46:43.938 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.920+0000 7fb6d6ffd700 1 --2- 172.21.15.36:0/2291488906 >> v2:172.21.15.70:3300/0 conn(0x7fb6d80b4200 0x7fb6d80b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:43.974 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.920+0000 7fb6d77fe700 1 -- 172.21.15.36:0/2291488906 >> v2:172.21.15.70:3300/0 conn(0x7fb6d80b4200 msgr2=0x7fb6d80b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:43.974 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.920+0000 7fb6d77fe700 1 --2- 172.21.15.36:0/2291488906 >> v2:172.21.15.70:3300/0 conn(0x7fb6d80b4200 0x7fb6d80b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:43.976 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.920+0000 7fb6d77fe700 1 -- 172.21.15.36:0/2291488906 >> v1:172.21.15.36:6789/0 conn(0x7fb6d80b25a0 legacy=0x7fb6d80ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:43.976 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.920+0000 7fb6d5ffb700 1 -- 172.21.15.36:0/2291488906 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4152747781 0 0) 0x7fb6d8143410 con 0x7fb6d80b25a0 2022-06-17T12:46:43.976 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.920+0000 7fb6d77fe700 1 -- 172.21.15.36:0/2291488906 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d8147130 con 0x7fb6d80b2a30 2022-06-17T12:46:43.977 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.924+0000 7fb6d77fe700 1 --2- 172.21.15.36:0/2291488906 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb6d80b2a30 0x7fb6d8142f20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fb6c8002f70 tx=0x7fb6c8006eb0).ready entity=mon.0 client_cookie=9bef74b59bf34a1b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:43.977 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.924+0000 7fb6d5ffb700 1 -- 172.21.15.36:0/2291488906 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6c800d5c0 con 0x7fb6d80b2a30 2022-06-17T12:46:43.978 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.924+0000 7fb6d5ffb700 1 -- 172.21.15.36:0/2291488906 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c8000e90 con 0x7fb6d80b2a30 2022-06-17T12:46:43.978 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.924+0000 7fb6d5ffb700 1 -- 172.21.15.36:0/2291488906 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6c800d950 con 0x7fb6d80b2a30 2022-06-17T12:46:43.978 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.924+0000 7fb6d7fff700 1 -- 172.21.15.36:0/2291488906 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb6d80b2a30 msgr2=0x7fb6d8142f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:43.979 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.924+0000 7fb6d7fff700 1 --2- 172.21.15.36:0/2291488906 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb6d80b2a30 0x7fb6d8142f20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fb6c8002f70 tx=0x7fb6c8006eb0).stop 2022-06-17T12:46:43.995 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.924+0000 7fb6d7fff700 1 -- 172.21.15.36:0/2291488906 shutdown_connections 2022-06-17T12:46:43.996 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.924+0000 7fb6d7fff700 1 --2- 172.21.15.36:0/2291488906 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb6d80b2a30 0x7fb6d8142f20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:43.997 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.924+0000 7fb6d7fff700 1 --2- 172.21.15.36:0/2291488906 >> v2:172.21.15.70:3300/0 conn(0x7fb6d80b4200 0x7fb6d80b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:43.997 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.924+0000 7fb6d7fff700 1 -- 172.21.15.36:0/2291488906 >> 172.21.15.36:0/2291488906 conn(0x7fb6d805f6b0 msgr2=0x7fb6d805fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:43.997 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.924+0000 7fb6d7fff700 1 -- 172.21.15.36:0/2291488906 shutdown_connections 2022-06-17T12:46:43.998 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.924+0000 7fb6d7fff700 1 -- 172.21.15.36:0/2291488906 wait complete. 2022-06-17T12:46:43.998 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.928+0000 7fb6d7fff700 1 Processor -- start 2022-06-17T12:46:43.999 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.928+0000 7fb6d7fff700 1 -- start start 2022-06-17T12:46:44.000 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.928+0000 7fb6d7fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6d80b25a0 0x7fb6d8065780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:44.000 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.928+0000 7fb6d7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6d80b2a30 0x7fb6d813ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:44.000 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.928+0000 7fb6d7fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb6d80b4200 0x7fb6d813f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:44.001 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.928+0000 7fb6d7fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d8156040 con 0x7fb6d80b4200 2022-06-17T12:46:44.001 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.928+0000 7fb6d7fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d813f730 con 0x7fb6d80b2a30 2022-06-17T12:46:44.002 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.928+0000 7fb6d7fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d80b6a60 con 0x7fb6d80b25a0 2022-06-17T12:46:44.007 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.928+0000 7fb6d77fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb6d80b4200 0x7fb6d813f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:44.007 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.928+0000 7fb6d77fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb6d80b4200 0x7fb6d813f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59242/0 (socket says 172.21.15.36:59242) 2022-06-17T12:46:44.008 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.928+0000 7fb6d77fe700 1 -- 172.21.15.36:0/1697359003 learned_addr learned my addr 172.21.15.36:0/1697359003 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:44.008 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.928+0000 7fb6d6ffd700 1 --2- 172.21.15.36:0/1697359003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6d80b25a0 0x7fb6d8065780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:44.009 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.928+0000 7fb6d67fc700 1 --2- 172.21.15.36:0/1697359003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6d80b2a30 0x7fb6d813ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:44.009 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.928+0000 7fb6d77fe700 1 -- 172.21.15.36:0/1697359003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6d80b25a0 msgr2=0x7fb6d8065780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:44.009 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.928+0000 7fb6d77fe700 1 --2- 172.21.15.36:0/1697359003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6d80b25a0 0x7fb6d8065780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:44.010 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.928+0000 7fb6d77fe700 1 -- 172.21.15.36:0/1697359003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6d80b2a30 msgr2=0x7fb6d813ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:44.010 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.928+0000 7fb6d77fe700 1 --2- 172.21.15.36:0/1697359003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6d80b2a30 0x7fb6d813ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:44.011 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.928+0000 7fb6d77fe700 1 -- 172.21.15.36:0/1697359003 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c800b1e0 con 0x7fb6d80b4200 2022-06-17T12:46:44.011 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.928+0000 7fb6d77fe700 1 --2- 172.21.15.36:0/1697359003 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb6d80b4200 0x7fb6d813f240 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fb6c8002f70 tx=0x7fb6c8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:44.012 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.928+0000 7fb6c7fff700 1 -- 172.21.15.36:0/1697359003 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6c8011610 con 0x7fb6d80b4200 2022-06-17T12:46:44.012 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.928+0000 7fb6c7fff700 1 -- 172.21.15.36:0/1697359003 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c8011770 con 0x7fb6d80b4200 2022-06-17T12:46:44.012 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.928+0000 7fb6d7fff700 1 -- 172.21.15.36:0/1697359003 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d8156470 con 0x7fb6d80b4200 2022-06-17T12:46:44.013 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.928+0000 7fb6c7fff700 1 -- 172.21.15.36:0/1697359003 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6c8011ad0 con 0x7fb6d80b4200 2022-06-17T12:46:44.013 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.928+0000 7fb6d7fff700 1 -- 172.21.15.36:0/1697359003 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d8065fd0 con 0x7fb6d80b4200 2022-06-17T12:46:44.014 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.932+0000 7fb6d7fff700 1 -- 172.21.15.36:0/1697359003 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d80aef60 con 0x7fb6d80b4200 2022-06-17T12:46:44.031 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.932+0000 7fb6c7fff700 1 -- 172.21.15.36:0/1697359003 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fb6c8025030 con 0x7fb6d80b4200 2022-06-17T12:46:44.031 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.932+0000 7fb6c7fff700 1 --2- 172.21.15.36:0/1697359003 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb6b4053a40 0x7fb6b4055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:44.032 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.932+0000 7fb6c7fff700 1 -- 172.21.15.36:0/1697359003 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4900+0+0 (secure 0 0 0) 0x7fb6c806cbd0 con 0x7fb6d80b4200 2022-06-17T12:46:44.032 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.932+0000 7fb6d6ffd700 1 --2- 172.21.15.36:0/1697359003 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb6b4053a40 0x7fb6b4055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:44.032 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.932+0000 7fb6d6ffd700 1 --2- 172.21.15.36:0/1697359003 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb6b4053a40 0x7fb6b4055f00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fb6d000b000 tx=0x7fb6d00091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:44.033 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:43.940+0000 7fb6c7fff700 1 -- 172.21.15.36:0/1697359003 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb6c8043eb0 con 0x7fb6d80b4200 2022-06-17T12:46:44.070 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:44.068+0000 7fb6d7fff700 1 -- 172.21.15.36:0/1697359003 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6d8065e10 con 0x7fb6d80b4200 2022-06-17T12:46:44.070 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:44.072+0000 7fb6c7fff700 1 -- 172.21.15.36:0/1697359003 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+11806 (secure 0 0 0) 0x7fb6c80474d0 con 0x7fb6d80b4200 2022-06-17T12:46:44.071 INFO:teuthology.misc.health.smithi036.stdout: 2022-06-17T12:46:44.071 INFO:teuthology.misc.health.smithi036.stdout:{"epoch":16,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:46:42.609562+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:46:44.121 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:44.076+0000 7fb6d7fff700 1 -- 172.21.15.36:0/1697359003 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb6b4053a40 msgr2=0x7fb6b4055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:44.122 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:44.076+0000 7fb6d7fff700 1 --2- 172.21.15.36:0/1697359003 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb6b4053a40 0x7fb6b4055f00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fb6d000b000 tx=0x7fb6d00091d0).stop 2022-06-17T12:46:44.123 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:44.076+0000 7fb6d7fff700 1 -- 172.21.15.36:0/1697359003 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb6d80b4200 msgr2=0x7fb6d813f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:44.123 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:44.076+0000 7fb6d7fff700 1 --2- 172.21.15.36:0/1697359003 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb6d80b4200 0x7fb6d813f240 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fb6c8002f70 tx=0x7fb6c8015000).stop 2022-06-17T12:46:44.124 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:44.076+0000 7fb6d7fff700 1 -- 172.21.15.36:0/1697359003 shutdown_connections 2022-06-17T12:46:44.124 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:44.076+0000 7fb6d7fff700 1 --2- 172.21.15.36:0/1697359003 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb6b4053a40 0x7fb6b4055f00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:44.124 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:44.076+0000 7fb6d7fff700 1 --2- 172.21.15.36:0/1697359003 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb6d80b4200 0x7fb6d813f240 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:44.125 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:44.076+0000 7fb6d7fff700 1 --2- 172.21.15.36:0/1697359003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6d80b2a30 0x7fb6d813ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:44.126 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:44.076+0000 7fb6d7fff700 1 --2- 172.21.15.36:0/1697359003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6d80b25a0 0x7fb6d8065780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:44.126 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:44.076+0000 7fb6d7fff700 1 -- 172.21.15.36:0/1697359003 >> 172.21.15.36:0/1697359003 conn(0x7fb6d805f6b0 msgr2=0x7fb6d8146440 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:44.126 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:44.076+0000 7fb6d7fff700 1 -- 172.21.15.36:0/1697359003 shutdown_connections 2022-06-17T12:46:44.127 INFO:teuthology.misc.health.smithi036.stderr:2022-06-17T12:46:44.076+0000 7fb6d7fff700 1 -- 172.21.15.36:0/1697359003 wait complete. 2022-06-17T12:46:44.128 DEBUG:teuthology.misc:8 of 8 OSDs are up 2022-06-17T12:46:44.129 INFO:tasks.ceph:Creating RBD pool 2022-06-17T12:46:44.129 DEBUG:teuthology.orchestra.run.smithi036:> sudo ceph --cluster ceph osd pool create rbd 8 2022-06-17T12:46:44.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.284+0000 7faa43081700 1 Processor -- start 2022-06-17T12:46:44.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.284+0000 7faa43081700 1 -- start start 2022-06-17T12:46:44.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.284+0000 7faa43081700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7faa3c0b0fd0 0x7faa3c0aca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:44.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.284+0000 7faa43081700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faa3c060df0 0x7faa3c140200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:44.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.284+0000 7faa43081700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa3c143d50 con 0x7faa3c0adbd0 2022-06-17T12:46:44.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.284+0000 7faa43081700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa3c0acf50 con 0x7faa3c0b0fd0 2022-06-17T12:46:44.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.284+0000 7faa43081700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa3c143f80 con 0x7faa3c060df0 2022-06-17T12:46:44.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.284+0000 7faa40e1d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7faa3c0b0fd0 0x7faa3c0aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:44.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.284+0000 7faa40e1d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7faa3c0b0fd0 0x7faa3c0aca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59246/0 (socket says 172.21.15.36:59246) 2022-06-17T12:46:44.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.284+0000 7faa40e1d700 1 -- 172.21.15.36:0/405690487 learned_addr learned my addr 172.21.15.36:0/405690487 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:44.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.284+0000 7faa3bfff700 1 --1- 172.21.15.36:0/405690487 >> v1:172.21.15.36:6789/0 conn(0x7faa3c0adbd0 0x7faa3c05f770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:48548/0 (socket says 172.21.15.36:48548) 2022-06-17T12:46:44.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.284+0000 7faa4161e700 1 --2- 172.21.15.36:0/405690487 >> v2:172.21.15.70:3300/0 conn(0x7faa3c060df0 0x7faa3c140200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:44.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.284+0000 7faa3b7fe700 1 -- 172.21.15.36:0/405690487 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3842338201 0 0) 0x7faa3c143d50 con 0x7faa3c0adbd0 2022-06-17T12:46:44.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.284+0000 7faa3b7fe700 1 -- 172.21.15.36:0/405690487 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa20003260 con 0x7faa3c0adbd0 2022-06-17T12:46:44.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.284+0000 7faa40e1d700 1 -- 172.21.15.36:0/405690487 >> v2:172.21.15.70:3300/0 conn(0x7faa3c060df0 msgr2=0x7faa3c140200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:44.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.284+0000 7faa40e1d700 1 --2- 172.21.15.36:0/405690487 >> v2:172.21.15.70:3300/0 conn(0x7faa3c060df0 0x7faa3c140200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:44.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.284+0000 7faa40e1d700 1 -- 172.21.15.36:0/405690487 >> v1:172.21.15.36:6789/0 conn(0x7faa3c0adbd0 legacy=0x7faa3c05f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:44.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.284+0000 7faa40e1d700 1 -- 172.21.15.36:0/405690487 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3c144220 con 0x7faa3c0b0fd0 2022-06-17T12:46:44.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.284+0000 7faa4161e700 1 --2- 172.21.15.36:0/405690487 >> v2:172.21.15.70:3300/0 conn(0x7faa3c060df0 0x7faa3c140200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:46:44.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.284+0000 7faa40e1d700 1 --2- 172.21.15.36:0/405690487 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa3c0b0fd0 0x7faa3c0aca60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7faa30002f70 tx=0x7faa30006eb0).ready entity=mon.0 client_cookie=c62f17c262c87fda server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:44.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.284+0000 7faa3b7fe700 1 -- 172.21.15.36:0/405690487 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa3000d5c0 con 0x7faa3c0b0fd0 2022-06-17T12:46:44.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.284+0000 7faa3b7fe700 1 -- 172.21.15.36:0/405690487 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa30000e90 con 0x7faa3c0b0fd0 2022-06-17T12:46:44.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.284+0000 7faa3b7fe700 1 -- 172.21.15.36:0/405690487 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa3000d950 con 0x7faa3c0b0fd0 2022-06-17T12:46:44.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.288+0000 7faa43081700 1 -- 172.21.15.36:0/405690487 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa3c0b0fd0 msgr2=0x7faa3c0aca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:44.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.288+0000 7faa43081700 1 --2- 172.21.15.36:0/405690487 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa3c0b0fd0 0x7faa3c0aca60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7faa30002f70 tx=0x7faa30006eb0).stop 2022-06-17T12:46:44.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.288+0000 7faa43081700 1 -- 172.21.15.36:0/405690487 shutdown_connections 2022-06-17T12:46:44.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.288+0000 7faa43081700 1 --2- 172.21.15.36:0/405690487 >> v2:172.21.15.70:3300/0 conn(0x7faa3c060df0 0x7faa3c140200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:44.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.288+0000 7faa43081700 1 --2- 172.21.15.36:0/405690487 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa3c0b0fd0 0x7faa3c0aca60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:44.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.288+0000 7faa43081700 1 -- 172.21.15.36:0/405690487 >> 172.21.15.36:0/405690487 conn(0x7faa3c05c550 msgr2=0x7faa3c0b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:44.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.288+0000 7faa43081700 1 -- 172.21.15.36:0/405690487 shutdown_connections 2022-06-17T12:46:44.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.288+0000 7faa43081700 1 -- 172.21.15.36:0/405690487 wait complete. 2022-06-17T12:46:44.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.288+0000 7faa43081700 1 Processor -- start 2022-06-17T12:46:44.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.288+0000 7faa43081700 1 -- start start 2022-06-17T12:46:44.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.288+0000 7faa43081700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa3c060df0 0x7faa3c065da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:44.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.288+0000 7faa43081700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa3c0adbd0 0x7faa3c14e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:44.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.288+0000 7faa43081700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa3c0b0fd0 0x7faa3c066600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:44.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.288+0000 7faa43081700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c14dd90 con 0x7faa3c060df0 2022-06-17T12:46:44.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.288+0000 7faa43081700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c14efe0 con 0x7faa3c0adbd0 2022-06-17T12:46:44.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.288+0000 7faa43081700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c14dfc0 con 0x7faa3c0b0fd0 2022-06-17T12:46:44.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.288+0000 7faa40e1d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa3c060df0 0x7faa3c065da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:44.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.288+0000 7faa40e1d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa3c060df0 0x7faa3c065da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59252/0 (socket says 172.21.15.36:59252) 2022-06-17T12:46:44.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.288+0000 7faa40e1d700 1 -- 172.21.15.36:0/827832662 learned_addr learned my addr 172.21.15.36:0/827832662 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:44.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.288+0000 7faa4161e700 1 --2- 172.21.15.36:0/827832662 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa3c0b0fd0 0x7faa3c066600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:44.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.288+0000 7faa3bfff700 1 --2- 172.21.15.36:0/827832662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa3c0adbd0 0x7faa3c14e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:44.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.292+0000 7faa40e1d700 1 -- 172.21.15.36:0/827832662 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa3c0b0fd0 msgr2=0x7faa3c066600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:44.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.292+0000 7faa40e1d700 1 --2- 172.21.15.36:0/827832662 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa3c0b0fd0 0x7faa3c066600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:44.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.292+0000 7faa40e1d700 1 -- 172.21.15.36:0/827832662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa3c0adbd0 msgr2=0x7faa3c14e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:44.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.292+0000 7faa40e1d700 1 --2- 172.21.15.36:0/827832662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa3c0adbd0 0x7faa3c14e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:44.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.292+0000 7faa40e1d700 1 -- 172.21.15.36:0/827832662 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3000b1e0 con 0x7faa3c060df0 2022-06-17T12:46:44.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.292+0000 7faa40e1d700 1 --2- 172.21.15.36:0/827832662 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa3c060df0 0x7faa3c065da0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7faa30002f70 tx=0x7faa30015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:44.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.292+0000 7faa39ffb700 1 -- 172.21.15.36:0/827832662 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa30011760 con 0x7faa3c060df0 2022-06-17T12:46:44.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.292+0000 7faa39ffb700 1 -- 172.21.15.36:0/827832662 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa300118c0 con 0x7faa3c060df0 2022-06-17T12:46:44.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.292+0000 7faa43081700 1 -- 172.21.15.36:0/827832662 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa3c14e260 con 0x7faa3c060df0 2022-06-17T12:46:44.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.292+0000 7faa39ffb700 1 -- 172.21.15.36:0/827832662 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa30011c20 con 0x7faa3c060df0 2022-06-17T12:46:44.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.292+0000 7faa43081700 1 -- 172.21.15.36:0/827832662 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa3c14ebf0 con 0x7faa3c060df0 2022-06-17T12:46:44.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.292+0000 7faa43081700 1 -- 172.21.15.36:0/827832662 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa3c1406f0 con 0x7faa3c060df0 2022-06-17T12:46:44.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.292+0000 7faa39ffb700 1 -- 172.21.15.36:0/827832662 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7faa30025030 con 0x7faa3c060df0 2022-06-17T12:46:44.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.292+0000 7faa39ffb700 1 --2- 172.21.15.36:0/827832662 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa1c053a40 0x7faa1c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:44.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.292+0000 7faa39ffb700 1 -- 172.21.15.36:0/827832662 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4900+0+0 (secure 0 0 0) 0x7faa3006cb40 con 0x7faa3c060df0 2022-06-17T12:46:44.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.296+0000 7faa3bfff700 1 --2- 172.21.15.36:0/827832662 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa1c053a40 0x7faa1c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:44.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.296+0000 7faa3bfff700 1 --2- 172.21.15.36:0/827832662 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa1c053a40 0x7faa1c055f00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7faa2800f000 tx=0x7faa28009c20).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:44.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.300+0000 7faa39ffb700 1 -- 172.21.15.36:0/827832662 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faa30043e20 con 0x7faa3c060df0 2022-06-17T12:46:44.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.428+0000 7faa43081700 1 -- 172.21.15.36:0/827832662 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7faa3c140f50 con 0x7faa3c060df0 2022-06-17T12:46:44.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.644+0000 7faa39ffb700 1 -- 172.21.15.36:0/827832662 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7faa30023070 con 0x7faa3c060df0 2022-06-17T12:46:44.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.648+0000 7faa43081700 1 -- 172.21.15.36:0/827832662 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa1c053a40 msgr2=0x7faa1c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:44.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.648+0000 7faa43081700 1 --2- 172.21.15.36:0/827832662 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa1c053a40 0x7faa1c055f00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7faa2800f000 tx=0x7faa28009c20).stop 2022-06-17T12:46:44.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.648+0000 7faa43081700 1 -- 172.21.15.36:0/827832662 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa3c060df0 msgr2=0x7faa3c065da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:44.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.648+0000 7faa43081700 1 --2- 172.21.15.36:0/827832662 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa3c060df0 0x7faa3c065da0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7faa30002f70 tx=0x7faa30015000).stop 2022-06-17T12:46:44.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.648+0000 7faa43081700 1 -- 172.21.15.36:0/827832662 shutdown_connections 2022-06-17T12:46:44.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.648+0000 7faa43081700 1 --2- 172.21.15.36:0/827832662 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa1c053a40 0x7faa1c055f00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:44.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.648+0000 7faa43081700 1 --2- 172.21.15.36:0/827832662 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa3c0b0fd0 0x7faa3c066600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:44.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.648+0000 7faa43081700 1 --2- 172.21.15.36:0/827832662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa3c0adbd0 0x7faa3c14e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:44.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.648+0000 7faa43081700 1 --2- 172.21.15.36:0/827832662 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa3c060df0 0x7faa3c065da0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:44.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.648+0000 7faa43081700 1 -- 172.21.15.36:0/827832662 >> 172.21.15.36:0/827832662 conn(0x7faa3c05c550 msgr2=0x7faa3c13c730 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:44.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.648+0000 7faa43081700 1 -- 172.21.15.36:0/827832662 shutdown_connections 2022-06-17T12:46:44.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.648+0000 7faa43081700 1 -- 172.21.15.36:0/827832662 wait complete. 2022-06-17T12:46:44.681 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'rbd' created 2022-06-17T12:46:44.683 DEBUG:teuthology.orchestra.run.smithi036:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2022-06-17T12:46:44.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.819+0000 7f27713da700 1 Processor -- start 2022-06-17T12:46:44.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.819+0000 7f27713da700 1 -- start start 2022-06-17T12:46:44.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f27713da700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f276c0b4e10 0x7f276c0b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:44.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f27713da700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f276c0b31c0 0x7f276c0b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:44.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f27713da700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f276c0b21d0 con 0x7f276c0620d0 2022-06-17T12:46:44.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f27713da700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f276c0b2430 con 0x7f276c0b4e10 2022-06-17T12:46:44.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f27713da700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f276c0b26d0 con 0x7f276c0b31c0 2022-06-17T12:46:44.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f276a7fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f276c0b4e10 0x7f276c0b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:44.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f276a7fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f276c0b4e10 0x7f276c0b5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59262/0 (socket says 172.21.15.36:59262) 2022-06-17T12:46:44.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f276a7fc700 1 -- 172.21.15.36:0/2416482905 learned_addr learned my addr 172.21.15.36:0/2416482905 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:44.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f276affd700 1 --1- 172.21.15.36:0/2416482905 >> v1:172.21.15.36:6789/0 conn(0x7f276c0620d0 0x7f276c0b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:48558/0 (socket says 172.21.15.36:48558) 2022-06-17T12:46:44.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f276b7fe700 1 --2- 172.21.15.36:0/2416482905 >> v2:172.21.15.70:3300/0 conn(0x7f276c0b31c0 0x7f276c0b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:44.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f276a7fc700 1 -- 172.21.15.36:0/2416482905 >> v2:172.21.15.70:3300/0 conn(0x7f276c0b31c0 msgr2=0x7f276c0b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:44.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f276a7fc700 1 --2- 172.21.15.36:0/2416482905 >> v2:172.21.15.70:3300/0 conn(0x7f276c0b31c0 0x7f276c0b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:44.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f276a7fc700 1 -- 172.21.15.36:0/2416482905 >> v1:172.21.15.36:6789/0 conn(0x7f276c0620d0 legacy=0x7f276c0b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:44.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f276a7fc700 1 -- 172.21.15.36:0/2416482905 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f276c0b43f0 con 0x7f276c0b4e10 2022-06-17T12:46:44.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f276a7fc700 1 --2- 172.21.15.36:0/2416482905 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f276c0b4e10 0x7f276c0b5bc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f275c002f70 tx=0x7f275c006eb0).ready entity=mon.0 client_cookie=489057645672bfef server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:44.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f2769ffb700 1 -- 172.21.15.36:0/2416482905 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f275c00d610 con 0x7f276c0b4e10 2022-06-17T12:46:44.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f2769ffb700 1 -- 172.21.15.36:0/2416482905 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275c000e90 con 0x7f276c0b4e10 2022-06-17T12:46:44.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f2769ffb700 1 -- 172.21.15.36:0/2416482905 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f275c00d9a0 con 0x7f276c0b4e10 2022-06-17T12:46:44.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f27713da700 1 -- 172.21.15.36:0/2416482905 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f276c0b4e10 msgr2=0x7f276c0b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:44.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f27713da700 1 --2- 172.21.15.36:0/2416482905 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f276c0b4e10 0x7f276c0b5bc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f275c002f70 tx=0x7f275c006eb0).stop 2022-06-17T12:46:44.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f27713da700 1 -- 172.21.15.36:0/2416482905 shutdown_connections 2022-06-17T12:46:44.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f27713da700 1 --2- 172.21.15.36:0/2416482905 >> v2:172.21.15.70:3300/0 conn(0x7f276c0b31c0 0x7f276c0b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:44.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f27713da700 1 --2- 172.21.15.36:0/2416482905 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f276c0b4e10 0x7f276c0b5bc0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:44.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f27713da700 1 -- 172.21.15.36:0/2416482905 >> 172.21.15.36:0/2416482905 conn(0x7f276c0b9850 msgr2=0x7f276c0b9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:44.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f27713da700 1 -- 172.21.15.36:0/2416482905 shutdown_connections 2022-06-17T12:46:44.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.823+0000 7f27713da700 1 -- 172.21.15.36:0/2416482905 wait complete. 2022-06-17T12:46:44.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.827+0000 7f27713da700 1 Processor -- start 2022-06-17T12:46:44.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.827+0000 7f27713da700 1 -- start start 2022-06-17T12:46:44.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.827+0000 7f27713da700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f276c0620d0 0x7f276c1471c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:44.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.827+0000 7f27713da700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f276c0b31c0 0x7f276c144560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:44.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.827+0000 7f27713da700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f276c0b4e10 0x7f276c1469e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:44.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.827+0000 7f27713da700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c13ece0 con 0x7f276c0620d0 2022-06-17T12:46:44.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.827+0000 7f27713da700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c1132b0 con 0x7f276c0b4e10 2022-06-17T12:46:44.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.827+0000 7f27713da700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f276c112bc0 con 0x7f276c0b31c0 2022-06-17T12:46:44.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.827+0000 7f276affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f276c0620d0 0x7f276c1471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:44.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.827+0000 7f276a7fc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f276c0b31c0 0x7f276c144560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:44.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.827+0000 7f276affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f276c0620d0 0x7f276c1471c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59264/0 (socket says 172.21.15.36:59264) 2022-06-17T12:46:44.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.827+0000 7f276affd700 1 -- 172.21.15.36:0/3698133531 learned_addr learned my addr 172.21.15.36:0/3698133531 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:45.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.827+0000 7f276b7fe700 1 --2- 172.21.15.36:0/3698133531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f276c0b4e10 0x7f276c1469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:45.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.827+0000 7f276affd700 1 -- 172.21.15.36:0/3698133531 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f276c0b31c0 msgr2=0x7f276c144560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:45.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.827+0000 7f276affd700 1 --2- 172.21.15.36:0/3698133531 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f276c0b31c0 0x7f276c144560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:45.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.827+0000 7f276affd700 1 -- 172.21.15.36:0/3698133531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f276c0b4e10 msgr2=0x7f276c1469e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:45.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.827+0000 7f276affd700 1 --2- 172.21.15.36:0/3698133531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f276c0b4e10 0x7f276c1469e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:45.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.827+0000 7f276affd700 1 -- 172.21.15.36:0/3698133531 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275c00b1e0 con 0x7f276c0620d0 2022-06-17T12:46:45.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.827+0000 7f276affd700 1 --2- 172.21.15.36:0/3698133531 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f276c0620d0 0x7f276c1471c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f276c0b2070 tx=0x7f2754007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:45.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.827+0000 7f2763fff700 1 -- 172.21.15.36:0/3698133531 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27540070d0 con 0x7f276c0620d0 2022-06-17T12:46:45.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.827+0000 7f2763fff700 1 -- 172.21.15.36:0/3698133531 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2754007230 con 0x7f276c0620d0 2022-06-17T12:46:45.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.827+0000 7f2763fff700 1 -- 172.21.15.36:0/3698133531 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2754005550 con 0x7f276c0620d0 2022-06-17T12:46:45.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.827+0000 7f27713da700 1 -- 172.21.15.36:0/3698133531 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f276c112e60 con 0x7f276c0620d0 2022-06-17T12:46:45.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.827+0000 7f27713da700 1 -- 172.21.15.36:0/3698133531 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f276c114ec0 con 0x7f276c0620d0 2022-06-17T12:46:45.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.831+0000 7f27713da700 1 -- 172.21.15.36:0/3698133531 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f276c145690 con 0x7f276c0620d0 2022-06-17T12:46:45.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.831+0000 7f2763fff700 1 -- 172.21.15.36:0/3698133531 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f275401a070 con 0x7f276c0620d0 2022-06-17T12:46:45.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.831+0000 7f2763fff700 1 --2- 172.21.15.36:0/3698133531 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2750053a60 0x7f2750055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:45.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.831+0000 7f2763fff700 1 -- 172.21.15.36:0/3698133531 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5242+0+0 (secure 0 0 0) 0x7f275406ae30 con 0x7f276c0620d0 2022-06-17T12:46:45.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.831+0000 7f276a7fc700 1 --2- 172.21.15.36:0/3698133531 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2750053a60 0x7f2750055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:45.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.831+0000 7f276a7fc700 1 --2- 172.21.15.36:0/3698133531 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2750053a60 0x7f2750055f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f275c01c000 tx=0x7f275c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:45.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.839+0000 7f2763fff700 1 -- 172.21.15.36:0/3698133531 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2754041fc0 con 0x7f276c0620d0 2022-06-17T12:46:45.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:44.967+0000 7f27713da700 1 -- 172.21.15.36:0/3698133531 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f276c0624b0 con 0x7f276c0620d0 2022-06-17T12:46:45.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.643+0000 7f2763fff700 1 -- 172.21.15.36:0/3698133531 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7f2754018030 con 0x7f276c0620d0 2022-06-17T12:46:45.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.651+0000 7f27713da700 1 -- 172.21.15.36:0/3698133531 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2750053a60 msgr2=0x7f2750055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:45.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.651+0000 7f27713da700 1 --2- 172.21.15.36:0/3698133531 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2750053a60 0x7f2750055f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f275c01c000 tx=0x7f275c015000).stop 2022-06-17T12:46:45.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.651+0000 7f27713da700 1 -- 172.21.15.36:0/3698133531 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f276c0620d0 msgr2=0x7f276c1471c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:45.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.651+0000 7f27713da700 1 --2- 172.21.15.36:0/3698133531 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f276c0620d0 0x7f276c1471c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f276c0b2070 tx=0x7f2754007c40).stop 2022-06-17T12:46:45.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.651+0000 7f27713da700 1 -- 172.21.15.36:0/3698133531 shutdown_connections 2022-06-17T12:46:45.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.651+0000 7f27713da700 1 --2- 172.21.15.36:0/3698133531 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2750053a60 0x7f2750055f20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:45.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.651+0000 7f27713da700 1 --2- 172.21.15.36:0/3698133531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f276c0b4e10 0x7f276c1469e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:45.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.651+0000 7f27713da700 1 --2- 172.21.15.36:0/3698133531 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f276c0b31c0 0x7f276c144560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:45.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.651+0000 7f27713da700 1 --2- 172.21.15.36:0/3698133531 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f276c0620d0 0x7f276c1471c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:45.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.651+0000 7f27713da700 1 -- 172.21.15.36:0/3698133531 >> 172.21.15.36:0/3698133531 conn(0x7f276c0b9850 msgr2=0x7f276c0b4680 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:45.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.651+0000 7f27713da700 1 -- 172.21.15.36:0/3698133531 shutdown_connections 2022-06-17T12:46:45.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.651+0000 7f27713da700 1 -- 172.21.15.36:0/3698133531 wait complete. 2022-06-17T12:46:45.682 INFO:teuthology.orchestra.run.smithi036.stderr:enabled application 'rbd' on pool 'rbd' 2022-06-17T12:46:45.684 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2022-06-17T12:46:45.685 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-17T12:46:45.685 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-17T12:46:45.707 INFO:tasks.ceph.mds.a:Started 2022-06-17T12:46:45.707 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-17T12:46:45.708 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-17T12:46:45.715 INFO:tasks.ceph.mds.b:Started 2022-06-17T12:46:45.721 INFO:tasks.ceph:Setting up CephFS filesystem... 2022-06-17T12:46:45.724 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:46:45.807 INFO:tasks.ceph.mds.a.smithi036.stdout:starting mds.a at 2022-06-17T12:46:45.809 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2022-06-17T12:46:45.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.883+0000 7f2f3e8d3700 1 Processor -- start 2022-06-17T12:46:45.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.883+0000 7f2f3e8d3700 1 -- start start 2022-06-17T12:46:45.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.883+0000 7f2f3e8d3700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2f38060ac0 0x7f2f380afe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:45.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.883+0000 7f2f3e8d3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f380aee30 0x7f2f3813c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:45.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.883+0000 7f2f3e8d3700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f38060ea0 con 0x7f2f380b20f0 2022-06-17T12:46:45.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.883+0000 7f2f3e8d3700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f380b3060 con 0x7f2f38060ac0 2022-06-17T12:46:45.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.883+0000 7f2f3e8d3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f38067a20 con 0x7f2f380aee30 2022-06-17T12:46:45.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.883+0000 7f2f3d8d1700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2f38060ac0 0x7f2f380afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:45.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.883+0000 7f2f3d8d1700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2f38060ac0 0x7f2f380afe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59286/0 (socket says 172.21.15.36:59286) 2022-06-17T12:46:45.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.883+0000 7f2f3d8d1700 1 -- 172.21.15.36:0/1266133644 learned_addr learned my addr 172.21.15.36:0/1266133644 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:45.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.883+0000 7f2f3e0d2700 1 --2- 172.21.15.36:0/1266133644 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f380aee30 0x7f2f3813c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:45.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.883+0000 7f2f3d8d1700 1 -- 172.21.15.36:0/1266133644 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f380aee30 msgr2=0x7f2f3813c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:45.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.883+0000 7f2f3d8d1700 1 --2- 172.21.15.36:0/1266133644 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f380aee30 0x7f2f3813c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:45.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.883+0000 7f2f3d8d1700 1 -- 172.21.15.36:0/1266133644 >> v1:172.21.15.70:6789/0 conn(0x7f2f380b20f0 legacy=0x7f2f380ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:45.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.883+0000 7f2f3d8d1700 1 -- 172.21.15.36:0/1266133644 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f38143fb0 con 0x7f2f38060ac0 2022-06-17T12:46:45.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.883+0000 7f2f3d8d1700 1 --2- 172.21.15.36:0/1266133644 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f38060ac0 0x7f2f380afe50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f2f34002f70 tx=0x7f2f34006eb0).ready entity=mon.0 client_cookie=3ef3a3485efc1054 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:46.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.887+0000 7f2f3c8cf700 1 -- 172.21.15.36:0/1266133644 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f3400d570 con 0x7f2f38060ac0 2022-06-17T12:46:46.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.887+0000 7f2f3c8cf700 1 -- 172.21.15.36:0/1266133644 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f34000e90 con 0x7f2f38060ac0 2022-06-17T12:46:46.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.887+0000 7f2f3c8cf700 1 -- 172.21.15.36:0/1266133644 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f3400d900 con 0x7f2f38060ac0 2022-06-17T12:46:46.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.887+0000 7f2f3e8d3700 1 -- 172.21.15.36:0/1266133644 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f38060ac0 msgr2=0x7f2f380afe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:46.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.887+0000 7f2f3e8d3700 1 --2- 172.21.15.36:0/1266133644 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f38060ac0 0x7f2f380afe50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f2f34002f70 tx=0x7f2f34006eb0).stop 2022-06-17T12:46:46.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.887+0000 7f2f3e8d3700 1 -- 172.21.15.36:0/1266133644 shutdown_connections 2022-06-17T12:46:46.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.887+0000 7f2f3e8d3700 1 --2- 172.21.15.36:0/1266133644 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f380aee30 0x7f2f3813c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.887+0000 7f2f3e8d3700 1 --2- 172.21.15.36:0/1266133644 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f38060ac0 0x7f2f380afe50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.887+0000 7f2f3e8d3700 1 -- 172.21.15.36:0/1266133644 >> 172.21.15.36:0/1266133644 conn(0x7f2f380b65f0 msgr2=0x7f2f380b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:46.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.887+0000 7f2f3e8d3700 1 -- 172.21.15.36:0/1266133644 shutdown_connections 2022-06-17T12:46:46.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.887+0000 7f2f3e8d3700 1 -- 172.21.15.36:0/1266133644 wait complete. 2022-06-17T12:46:46.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.891+0000 7f2f3e8d3700 1 Processor -- start 2022-06-17T12:46:46.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.891+0000 7f2f3e8d3700 1 -- start start 2022-06-17T12:46:46.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.891+0000 7f2f3e8d3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f38060ac0 0x7f2f381409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:46.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.891+0000 7f2f3e8d3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f380aee30 0x7f2f38145a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:46.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.891+0000 7f2f3e8d3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f380b20f0 0x7f2f38155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:46.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.891+0000 7f2f3e8d3700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3813db30 con 0x7f2f38060ac0 2022-06-17T12:46:46.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.891+0000 7f2f3e8d3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f38152d60 con 0x7f2f380b20f0 2022-06-17T12:46:46.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.891+0000 7f2f3e8d3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f38153000 con 0x7f2f380aee30 2022-06-17T12:46:46.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.891+0000 7f2f3d8d1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f38060ac0 0x7f2f381409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:46.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.891+0000 7f2f3d8d1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f38060ac0 0x7f2f381409f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59292/0 (socket says 172.21.15.36:59292) 2022-06-17T12:46:46.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.891+0000 7f2f3d8d1700 1 -- 172.21.15.36:0/2659797845 learned_addr learned my addr 172.21.15.36:0/2659797845 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:46.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.891+0000 7f2f3d0d0700 1 --2- 172.21.15.36:0/2659797845 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f380aee30 0x7f2f38145a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:46.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.891+0000 7f2f3e0d2700 1 --2- 172.21.15.36:0/2659797845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f380b20f0 0x7f2f38155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:46.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.891+0000 7f2f3d8d1700 1 -- 172.21.15.36:0/2659797845 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f380aee30 msgr2=0x7f2f38145a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:46.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.891+0000 7f2f3d8d1700 1 --2- 172.21.15.36:0/2659797845 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f380aee30 0x7f2f38145a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.891+0000 7f2f3d8d1700 1 -- 172.21.15.36:0/2659797845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f380b20f0 msgr2=0x7f2f38155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:46.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.891+0000 7f2f3d8d1700 1 --2- 172.21.15.36:0/2659797845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f380b20f0 0x7f2f38155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.891+0000 7f2f3d8d1700 1 -- 172.21.15.36:0/2659797845 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3400b1e0 con 0x7f2f38060ac0 2022-06-17T12:46:46.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.891+0000 7f2f3d8d1700 1 --2- 172.21.15.36:0/2659797845 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f38060ac0 0x7f2f381409f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f2f34002f70 tx=0x7f2f34015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:46.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.891+0000 7f2f2affd700 1 -- 172.21.15.36:0/2659797845 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f3400d8f0 con 0x7f2f38060ac0 2022-06-17T12:46:46.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.891+0000 7f2f2affd700 1 -- 172.21.15.36:0/2659797845 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f34011b80 con 0x7f2f38060ac0 2022-06-17T12:46:46.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.891+0000 7f2f3e8d3700 1 -- 172.21.15.36:0/2659797845 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f381557b0 con 0x7f2f38060ac0 2022-06-17T12:46:46.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.891+0000 7f2f2affd700 1 -- 172.21.15.36:0/2659797845 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f3401e460 con 0x7f2f38060ac0 2022-06-17T12:46:46.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.891+0000 7f2f3e8d3700 1 -- 172.21.15.36:0/2659797845 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3813f650 con 0x7f2f38060ac0 2022-06-17T12:46:46.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.895+0000 7f2f3e8d3700 1 -- 172.21.15.36:0/2659797845 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f38155970 con 0x7f2f38060ac0 2022-06-17T12:46:46.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.895+0000 7f2f2affd700 1 -- 172.21.15.36:0/2659797845 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2f34025030 con 0x7f2f38060ac0 2022-06-17T12:46:46.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.895+0000 7f2f2affd700 1 --2- 172.21.15.36:0/2659797845 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2f18053a40 0x7f2f18055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:46.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.895+0000 7f2f2affd700 1 -- 172.21.15.36:0/2659797845 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f2f3406cda0 con 0x7f2f38060ac0 2022-06-17T12:46:46.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.895+0000 7f2f3d0d0700 1 --2- 172.21.15.36:0/2659797845 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2f18053a40 0x7f2f18055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:46.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.895+0000 7f2f3d0d0700 1 --2- 172.21.15.36:0/2659797845 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2f18053a40 0x7f2f18055f00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f2f2c00d000 tx=0x7f2f2c007b70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:46.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:45.903+0000 7f2f2affd700 1 -- 172.21.15.36:0/2659797845 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2f34043f20 con 0x7f2f38060ac0 2022-06-17T12:46:46.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.031+0000 7f2f3e8d3700 1 -- 172.21.15.36:0/2659797845 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f38057410 con 0x7f2f38060ac0 2022-06-17T12:46:46.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.031+0000 7f2f2affd700 1 -- 172.21.15.36:0/2659797845 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13192 (secure 0 0 0) 0x7f2f34023070 con 0x7f2f38060ac0 2022-06-17T12:46:46.059 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:46:46.060 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":18,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:46:45.641376+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:46:46.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.035+0000 7f2f3e8d3700 1 -- 172.21.15.36:0/2659797845 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2f18053a40 msgr2=0x7f2f18055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:46.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.035+0000 7f2f3e8d3700 1 --2- 172.21.15.36:0/2659797845 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2f18053a40 0x7f2f18055f00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f2f2c00d000 tx=0x7f2f2c007b70).stop 2022-06-17T12:46:46.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.035+0000 7f2f3e8d3700 1 -- 172.21.15.36:0/2659797845 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f38060ac0 msgr2=0x7f2f381409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:46.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.035+0000 7f2f3e8d3700 1 --2- 172.21.15.36:0/2659797845 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f38060ac0 0x7f2f381409f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f2f34002f70 tx=0x7f2f34015000).stop 2022-06-17T12:46:46.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.035+0000 7f2f3e8d3700 1 -- 172.21.15.36:0/2659797845 shutdown_connections 2022-06-17T12:46:46.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.035+0000 7f2f3e8d3700 1 --2- 172.21.15.36:0/2659797845 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2f18053a40 0x7f2f18055f00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.035+0000 7f2f3e8d3700 1 --2- 172.21.15.36:0/2659797845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f380b20f0 0x7f2f38155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.035+0000 7f2f3e8d3700 1 --2- 172.21.15.36:0/2659797845 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f380aee30 0x7f2f38145a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.035+0000 7f2f3e8d3700 1 --2- 172.21.15.36:0/2659797845 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f38060ac0 0x7f2f381409f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.035+0000 7f2f3e8d3700 1 -- 172.21.15.36:0/2659797845 >> 172.21.15.36:0/2659797845 conn(0x7f2f380b65f0 msgr2=0x7f2f380b04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:46.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.035+0000 7f2f3e8d3700 1 -- 172.21.15.36:0/2659797845 shutdown_connections 2022-06-17T12:46:46.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.035+0000 7f2f3e8d3700 1 -- 172.21.15.36:0/2659797845 wait complete. 2022-06-17T12:46:46.159 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-17T12:46:46.160 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:46:46.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.299+0000 7fb8ed767700 1 Processor -- start 2022-06-17T12:46:46.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.299+0000 7fb8ed767700 1 -- start start 2022-06-17T12:46:46.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.299+0000 7fb8ed767700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb8e80601c0 0x7fb8e81443d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:46.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.299+0000 7fb8ed767700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8e80ae360 0x7fb8e81448c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:46.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.299+0000 7fb8ed767700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8e80b2b60 con 0x7fb8e80b1db0 2022-06-17T12:46:46.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.299+0000 7fb8ed767700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e80b3230 con 0x7fb8e80601c0 2022-06-17T12:46:46.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.299+0000 7fb8ed767700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e80605a0 con 0x7fb8e80ae360 2022-06-17T12:46:46.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.299+0000 7fb8e77fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8e80ae360 0x7fb8e81448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:46.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.299+0000 7fb8e7fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb8e80601c0 0x7fb8e81443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:46.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.299+0000 7fb8ecf66700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb8e80b1db0 0x7fb8e81499f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:44336/0 (socket says 172.21.15.36:44336) 2022-06-17T12:46:46.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.299+0000 7fb8ecf66700 1 -- 172.21.15.36:0/3038475305 learned_addr learned my addr 172.21.15.36:0/3038475305 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:46:46.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.299+0000 7fb8e77fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8e80ae360 0x7fb8e81448c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:45860/0 (socket says 172.21.15.36:45860) 2022-06-17T12:46:46.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.303+0000 7fb8e6ffd700 1 -- 172.21.15.36:0/3038475305 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228114253 0 0) 0x7fb8e80b2b60 con 0x7fb8e80b1db0 2022-06-17T12:46:46.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.303+0000 7fb8e6ffd700 1 -- 172.21.15.36:0/3038475305 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8d4003260 con 0x7fb8e80b1db0 2022-06-17T12:46:46.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.303+0000 7fb8e77fe700 1 -- 172.21.15.36:0/3038475305 >> v2:172.21.15.70:3300/0 conn(0x7fb8e80601c0 msgr2=0x7fb8e81443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:46.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.303+0000 7fb8e77fe700 1 --2- 172.21.15.36:0/3038475305 >> v2:172.21.15.70:3300/0 conn(0x7fb8e80601c0 0x7fb8e81443d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.303+0000 7fb8e77fe700 1 -- 172.21.15.36:0/3038475305 >> v1:172.21.15.70:6789/0 conn(0x7fb8e80b1db0 legacy=0x7fb8e81499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:46.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.303+0000 7fb8e77fe700 1 -- 172.21.15.36:0/3038475305 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e80662f0 con 0x7fb8e80ae360 2022-06-17T12:46:46.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.303+0000 7fb8e77fe700 1 --2- 172.21.15.36:0/3038475305 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8e80ae360 0x7fb8e81448c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fb8dc002f70 tx=0x7fb8dc006df0).ready entity=mon.2 client_cookie=44307c8805bee3db server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:46.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.303+0000 7fb8e6ffd700 1 -- 172.21.15.36:0/3038475305 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8dc00d4a0 con 0x7fb8e80ae360 2022-06-17T12:46:46.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.303+0000 7fb8e6ffd700 1 -- 172.21.15.36:0/3038475305 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8dc000e90 con 0x7fb8e80ae360 2022-06-17T12:46:46.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.303+0000 7fb8e6ffd700 1 -- 172.21.15.36:0/3038475305 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8dc00d7f0 con 0x7fb8e80ae360 2022-06-17T12:46:46.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.303+0000 7fb8ed767700 1 -- 172.21.15.36:0/3038475305 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8e80ae360 msgr2=0x7fb8e81448c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:46.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.303+0000 7fb8ed767700 1 --2- 172.21.15.36:0/3038475305 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8e80ae360 0x7fb8e81448c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fb8dc002f70 tx=0x7fb8dc006df0).stop 2022-06-17T12:46:46.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.303+0000 7fb8ed767700 1 -- 172.21.15.36:0/3038475305 shutdown_connections 2022-06-17T12:46:46.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.303+0000 7fb8ed767700 1 --2- 172.21.15.36:0/3038475305 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8e80ae360 0x7fb8e81448c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.303+0000 7fb8ed767700 1 --2- 172.21.15.36:0/3038475305 >> v2:172.21.15.70:3300/0 conn(0x7fb8e80601c0 0x7fb8e81443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.303+0000 7fb8ed767700 1 -- 172.21.15.36:0/3038475305 >> 172.21.15.36:0/3038475305 conn(0x7fb8e8060990 msgr2=0x7fb8e805f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:46.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.303+0000 7fb8ed767700 1 -- 172.21.15.36:0/3038475305 shutdown_connections 2022-06-17T12:46:46.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.303+0000 7fb8ed767700 1 -- 172.21.15.36:0/3038475305 wait complete. 2022-06-17T12:46:46.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.307+0000 7fb8ed767700 1 Processor -- start 2022-06-17T12:46:46.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.307+0000 7fb8ed767700 1 -- start start 2022-06-17T12:46:46.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.307+0000 7fb8ed767700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb8e80601c0 0x7fb8e81536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:46.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.307+0000 7fb8ed767700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8e80ae360 0x7fb8e81435a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:46.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.307+0000 7fb8ed767700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8e80b1db0 0x7fb8e813cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:46.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.307+0000 7fb8ed767700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e80afa50 con 0x7fb8e80601c0 2022-06-17T12:46:46.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.307+0000 7fb8ed767700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e8140860 con 0x7fb8e80b1db0 2022-06-17T12:46:46.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.307+0000 7fb8ed767700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e8140b00 con 0x7fb8e80ae360 2022-06-17T12:46:46.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.307+0000 7fb8e7fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb8e80601c0 0x7fb8e81536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:46.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.307+0000 7fb8e7fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb8e80601c0 0x7fb8e81536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59306/0 (socket says 172.21.15.36:59306) 2022-06-17T12:46:46.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.307+0000 7fb8e7fff700 1 -- 172.21.15.36:0/2754244248 learned_addr learned my addr 172.21.15.36:0/2754244248 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:46.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.307+0000 7fb8e77fe700 1 --2- 172.21.15.36:0/2754244248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8e80ae360 0x7fb8e81435a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:46.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.307+0000 7fb8ecf66700 1 --2- 172.21.15.36:0/2754244248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8e80b1db0 0x7fb8e813cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:46.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.307+0000 7fb8e7fff700 1 -- 172.21.15.36:0/2754244248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8e80ae360 msgr2=0x7fb8e81435a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:46.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.307+0000 7fb8e7fff700 1 --2- 172.21.15.36:0/2754244248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8e80ae360 0x7fb8e81435a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.307+0000 7fb8e7fff700 1 -- 172.21.15.36:0/2754244248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8e80b1db0 msgr2=0x7fb8e813cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:46.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.307+0000 7fb8e7fff700 1 --2- 172.21.15.36:0/2754244248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8e80b1db0 0x7fb8e813cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.307+0000 7fb8e7fff700 1 -- 172.21.15.36:0/2754244248 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8dc00b180 con 0x7fb8e80601c0 2022-06-17T12:46:46.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.307+0000 7fb8e77fe700 1 --2- 172.21.15.36:0/2754244248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8e80ae360 0x7fb8e81435a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:46:46.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.307+0000 7fb8e7fff700 1 --2- 172.21.15.36:0/2754244248 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb8e80601c0 0x7fb8e81536f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fb8d800c000 tx=0x7fb8d8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:46.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.307+0000 7fb8e57fa700 1 -- 172.21.15.36:0/2754244248 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8d8010780 con 0x7fb8e80601c0 2022-06-17T12:46:46.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.307+0000 7fb8e57fa700 1 -- 172.21.15.36:0/2754244248 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d80108e0 con 0x7fb8e80601c0 2022-06-17T12:46:46.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.311+0000 7fb8ed767700 1 -- 172.21.15.36:0/2754244248 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e80af2a0 con 0x7fb8e80601c0 2022-06-17T12:46:46.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.311+0000 7fb8e57fa700 1 -- 172.21.15.36:0/2754244248 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8d8010c40 con 0x7fb8e80601c0 2022-06-17T12:46:46.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.311+0000 7fb8ed767700 1 -- 172.21.15.36:0/2754244248 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e8141550 con 0x7fb8e80601c0 2022-06-17T12:46:46.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.311+0000 7fb8ed767700 1 -- 172.21.15.36:0/2754244248 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e80af430 con 0x7fb8e80601c0 2022-06-17T12:46:46.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.311+0000 7fb8e57fa700 1 -- 172.21.15.36:0/2754244248 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fb8d801c460 con 0x7fb8e80601c0 2022-06-17T12:46:46.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.311+0000 7fb8e57fa700 1 --2- 172.21.15.36:0/2754244248 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb8c8053920 0x7fb8c8055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:46.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.311+0000 7fb8e57fa700 1 -- 172.21.15.36:0/2754244248 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fb8d806c180 con 0x7fb8e80601c0 2022-06-17T12:46:46.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.311+0000 7fb8e77fe700 1 --2- 172.21.15.36:0/2754244248 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb8c8053920 0x7fb8c8055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:46.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.315+0000 7fb8e77fe700 1 --2- 172.21.15.36:0/2754244248 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb8c8053920 0x7fb8c8055de0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fb8dc018000 tx=0x7fb8dc014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:46.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.319+0000 7fb8e57fa700 1 -- 172.21.15.36:0/2754244248 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb8d8043220 con 0x7fb8e80601c0 2022-06-17T12:46:46.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.447+0000 7fb8ed767700 1 -- 172.21.15.36:0/2754244248 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb8e8066590 con 0x7fb8e80601c0 2022-06-17T12:46:46.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.451+0000 7fb8e57fa700 1 -- 172.21.15.36:0/2754244248 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v18) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb8d8046840 con 0x7fb8e80601c0 2022-06-17T12:46:46.476 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:46:46.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.451+0000 7fb8ed767700 1 -- 172.21.15.36:0/2754244248 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb8c8053920 msgr2=0x7fb8c8055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:46.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.451+0000 7fb8ed767700 1 --2- 172.21.15.36:0/2754244248 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb8c8053920 0x7fb8c8055de0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fb8dc018000 tx=0x7fb8dc014000).stop 2022-06-17T12:46:46.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.451+0000 7fb8ed767700 1 -- 172.21.15.36:0/2754244248 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb8e80601c0 msgr2=0x7fb8e81536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:46.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.451+0000 7fb8ed767700 1 --2- 172.21.15.36:0/2754244248 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb8e80601c0 0x7fb8e81536f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fb8d800c000 tx=0x7fb8d8006df0).stop 2022-06-17T12:46:46.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.451+0000 7fb8ed767700 1 -- 172.21.15.36:0/2754244248 shutdown_connections 2022-06-17T12:46:46.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.451+0000 7fb8ed767700 1 --2- 172.21.15.36:0/2754244248 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb8c8053920 0x7fb8c8055de0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.451+0000 7fb8ed767700 1 --2- 172.21.15.36:0/2754244248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8e80b1db0 0x7fb8e813cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.451+0000 7fb8ed767700 1 --2- 172.21.15.36:0/2754244248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8e80ae360 0x7fb8e81435a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.451+0000 7fb8ed767700 1 --2- 172.21.15.36:0/2754244248 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb8e80601c0 0x7fb8e81536f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.451+0000 7fb8ed767700 1 -- 172.21.15.36:0/2754244248 >> 172.21.15.36:0/2754244248 conn(0x7fb8e8060990 msgr2=0x7fb8e80b6000 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:46.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.451+0000 7fb8ed767700 1 -- 172.21.15.36:0/2754244248 shutdown_connections 2022-06-17T12:46:46.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.451+0000 7fb8ed767700 1 -- 172.21.15.36:0/2754244248 wait complete. 2022-06-17T12:46:46.493 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:46:46.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.619+0000 7ff3fffff700 1 Processor -- start 2022-06-17T12:46:46.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.619+0000 7ff3fffff700 1 -- start start 2022-06-17T12:46:46.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3fffff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff4000b6860 0x7ff400146d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:46.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3fffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4000b18e0 0x7ff400143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:46.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3fffff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4000b2b70 con 0x7ff4000b39a0 2022-06-17T12:46:46.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3fffff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4000b6c40 con 0x7ff4000b6860 2022-06-17T12:46:46.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3fffff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff400142d30 con 0x7ff4000b18e0 2022-06-17T12:46:46.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3f67fc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4000b18e0 0x7ff400143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:46.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3feffd700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff4000b6860 0x7ff400146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:46.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3ff7fe700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff4000b39a0 0x7ff400143640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:44350/0 (socket says 172.21.15.36:44350) 2022-06-17T12:46:46.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3ff7fe700 1 -- 172.21.15.36:0/3708700623 learned_addr learned my addr 172.21.15.36:0/3708700623 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:46:46.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3fe7fc700 1 -- 172.21.15.36:0/3708700623 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3407330171 0 0) 0x7ff4000b2b70 con 0x7ff4000b39a0 2022-06-17T12:46:46.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3fe7fc700 1 -- 172.21.15.36:0/3708700623 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3e0003260 con 0x7ff4000b39a0 2022-06-17T12:46:46.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3f67fc700 1 -- 172.21.15.36:0/3708700623 >> v2:172.21.15.70:3300/0 conn(0x7ff4000b6860 msgr2=0x7ff400146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:46.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3f67fc700 1 --2- 172.21.15.36:0/3708700623 >> v2:172.21.15.70:3300/0 conn(0x7ff4000b6860 0x7ff400146d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3f67fc700 1 -- 172.21.15.36:0/3708700623 >> v1:172.21.15.70:6789/0 conn(0x7ff4000b39a0 legacy=0x7ff400143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:46.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3f67fc700 1 -- 172.21.15.36:0/3708700623 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4000b0d30 con 0x7ff4000b18e0 2022-06-17T12:46:46.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3f67fc700 1 --2- 172.21.15.36:0/3708700623 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4000b18e0 0x7ff400143150 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7ff3e8002f70 tx=0x7ff3e8006d90).ready entity=mon.2 client_cookie=f5182933b6155293 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:46.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3fe7fc700 1 -- 172.21.15.36:0/3708700623 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3e800e590 con 0x7ff4000b18e0 2022-06-17T12:46:46.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3fe7fc700 1 -- 172.21.15.36:0/3708700623 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e8000e90 con 0x7ff4000b18e0 2022-06-17T12:46:46.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3fe7fc700 1 -- 172.21.15.36:0/3708700623 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3e800e8e0 con 0x7ff4000b18e0 2022-06-17T12:46:46.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3fffff700 1 -- 172.21.15.36:0/3708700623 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4000b18e0 msgr2=0x7ff400143150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:46.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3fffff700 1 --2- 172.21.15.36:0/3708700623 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4000b18e0 0x7ff400143150 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7ff3e8002f70 tx=0x7ff3e8006d90).stop 2022-06-17T12:46:46.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3fffff700 1 -- 172.21.15.36:0/3708700623 shutdown_connections 2022-06-17T12:46:46.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3fffff700 1 --2- 172.21.15.36:0/3708700623 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4000b18e0 0x7ff400143150 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3fffff700 1 --2- 172.21.15.36:0/3708700623 >> v2:172.21.15.70:3300/0 conn(0x7ff4000b6860 0x7ff400146d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3fffff700 1 -- 172.21.15.36:0/3708700623 >> 172.21.15.36:0/3708700623 conn(0x7ff4000614b0 msgr2=0x7ff400061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:46.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3fffff700 1 -- 172.21.15.36:0/3708700623 shutdown_connections 2022-06-17T12:46:46.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.623+0000 7ff3fffff700 1 -- 172.21.15.36:0/3708700623 wait complete. 2022-06-17T12:46:46.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.627+0000 7ff3fffff700 1 Processor -- start 2022-06-17T12:46:46.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.627+0000 7ff3fffff700 1 -- start start 2022-06-17T12:46:46.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.627+0000 7ff3fffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4000b18e0 0x7ff4000660a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:46.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.627+0000 7ff3fffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4000b39a0 0x7ff400066590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:46.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.627+0000 7ff3fffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff4000b6860 0x7ff400155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:46.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.627+0000 7ff3fffff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff400115fa0 con 0x7ff4000b6860 2022-06-17T12:46:46.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.627+0000 7ff3fffff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff400115d00 con 0x7ff4000b18e0 2022-06-17T12:46:46.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.627+0000 7ff3fffff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff400065750 con 0x7ff4000b39a0 2022-06-17T12:46:46.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.627+0000 7ff3ff7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff4000b6860 0x7ff400155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:46.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.627+0000 7ff3ff7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff4000b6860 0x7ff400155770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59324/0 (socket says 172.21.15.36:59324) 2022-06-17T12:46:46.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.627+0000 7ff3feffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4000b18e0 0x7ff4000660a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:46.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.627+0000 7ff3ff7fe700 1 -- 172.21.15.36:0/165679025 learned_addr learned my addr 172.21.15.36:0/165679025 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:46.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.627+0000 7ff3f67fc700 1 --2- 172.21.15.36:0/165679025 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4000b39a0 0x7ff400066590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:46.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.627+0000 7ff3ff7fe700 1 -- 172.21.15.36:0/165679025 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4000b39a0 msgr2=0x7ff400066590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:46.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.627+0000 7ff3ff7fe700 1 --2- 172.21.15.36:0/165679025 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4000b39a0 0x7ff400066590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.627+0000 7ff3ff7fe700 1 -- 172.21.15.36:0/165679025 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4000b18e0 msgr2=0x7ff4000660a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:46.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.627+0000 7ff3ff7fe700 1 --2- 172.21.15.36:0/165679025 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4000b18e0 0x7ff4000660a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.627+0000 7ff3ff7fe700 1 -- 172.21.15.36:0/165679025 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e800b120 con 0x7ff4000b6860 2022-06-17T12:46:46.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.631+0000 7ff3ff7fe700 1 --2- 172.21.15.36:0/165679025 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff4000b6860 0x7ff400155770 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7ff3f8001490 tx=0x7ff3f8007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:46.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.631+0000 7ff3fcff9700 1 -- 172.21.15.36:0/165679025 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3f8005ba0 con 0x7ff4000b6860 2022-06-17T12:46:46.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.631+0000 7ff3fcff9700 1 -- 172.21.15.36:0/165679025 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f8005d00 con 0x7ff4000b6860 2022-06-17T12:46:46.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.631+0000 7ff3fcff9700 1 -- 172.21.15.36:0/165679025 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3f80072c0 con 0x7ff4000b6860 2022-06-17T12:46:46.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.631+0000 7ff3fffff700 1 -- 172.21.15.36:0/165679025 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff400066a80 con 0x7ff4000b6860 2022-06-17T12:46:46.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.631+0000 7ff3fffff700 1 -- 172.21.15.36:0/165679025 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff400113ed0 con 0x7ff4000b6860 2022-06-17T12:46:46.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.631+0000 7ff3fffff700 1 -- 172.21.15.36:0/165679025 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff400065910 con 0x7ff4000b6860 2022-06-17T12:46:46.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.631+0000 7ff3fcff9700 1 -- 172.21.15.36:0/165679025 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff3f8010070 con 0x7ff4000b6860 2022-06-17T12:46:46.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.631+0000 7ff3fcff9700 1 --2- 172.21.15.36:0/165679025 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff3e40539d0 0x7ff3e4055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:46.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.631+0000 7ff3fcff9700 1 -- 172.21.15.36:0/165679025 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7ff3f806b3e0 con 0x7ff4000b6860 2022-06-17T12:46:46.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.631+0000 7ff3feffd700 1 --2- 172.21.15.36:0/165679025 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff3e40539d0 0x7ff3e4055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:46.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.635+0000 7ff3feffd700 1 --2- 172.21.15.36:0/165679025 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff3e40539d0 0x7ff3e4055e90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7ff3f000b000 tx=0x7ff3f0006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:46.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.639+0000 7ff3fcff9700 1 -- 172.21.15.36:0/165679025 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff3f8042670 con 0x7ff4000b6860 2022-06-17T12:46:46.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.770+0000 7ff3fffff700 1 -- 172.21.15.36:0/165679025 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff400066c70 con 0x7ff4000b6860 2022-06-17T12:46:46.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.774+0000 7ff3fcff9700 1 -- 172.21.15.36:0/165679025 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff3f8018070 con 0x7ff4000b6860 2022-06-17T12:46:46.758 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:46:46.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.774+0000 7ff3fffff700 1 -- 172.21.15.36:0/165679025 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff3e40539d0 msgr2=0x7ff3e4055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:46.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.774+0000 7ff3fffff700 1 --2- 172.21.15.36:0/165679025 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff3e40539d0 0x7ff3e4055e90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7ff3f000b000 tx=0x7ff3f0006df0).stop 2022-06-17T12:46:46.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.774+0000 7ff3fffff700 1 -- 172.21.15.36:0/165679025 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff4000b6860 msgr2=0x7ff400155770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:46.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.774+0000 7ff3fffff700 1 --2- 172.21.15.36:0/165679025 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff4000b6860 0x7ff400155770 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7ff3f8001490 tx=0x7ff3f8007e00).stop 2022-06-17T12:46:46.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.774+0000 7ff3fffff700 1 -- 172.21.15.36:0/165679025 shutdown_connections 2022-06-17T12:46:46.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.774+0000 7ff3fffff700 1 --2- 172.21.15.36:0/165679025 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff3e40539d0 0x7ff3e4055e90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.774+0000 7ff3fffff700 1 --2- 172.21.15.36:0/165679025 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff4000b6860 0x7ff400155770 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.774+0000 7ff3fffff700 1 --2- 172.21.15.36:0/165679025 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4000b39a0 0x7ff400066590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.774+0000 7ff3fffff700 1 --2- 172.21.15.36:0/165679025 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4000b18e0 0x7ff4000660a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.774+0000 7ff3fffff700 1 -- 172.21.15.36:0/165679025 >> 172.21.15.36:0/165679025 conn(0x7ff4000614b0 msgr2=0x7ff400140120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:46.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.774+0000 7ff3fffff700 1 -- 172.21.15.36:0/165679025 shutdown_connections 2022-06-17T12:46:46.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.774+0000 7ff3fffff700 1 -- 172.21.15.36:0/165679025 wait complete. 2022-06-17T12:46:46.788 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-17T12:46:46.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f491eb14700 1 Processor -- start 2022-06-17T12:46:46.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f491eb14700 1 -- start start 2022-06-17T12:46:46.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f491eb14700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4918062770 0x7f4918062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:46.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f491eb14700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f49181444f0 0x7f4918144930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:46.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f491eb14700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49180b0ba0 con 0x7f49180ad330 2022-06-17T12:46:46.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f491eb14700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49180ae820 con 0x7f49181444f0 2022-06-17T12:46:46.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f491eb14700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4918061030 con 0x7f4918062770 2022-06-17T12:46:46.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f4917fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f49181444f0 0x7f4918144930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:46.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f4917fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f49181444f0 0x7f4918144930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59332/0 (socket says 172.21.15.36:59332) 2022-06-17T12:46:46.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f4917fff700 1 -- 172.21.15.36:0/2085151443 learned_addr learned my addr 172.21.15.36:0/2085151443 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:46.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f491c8b0700 1 --2- 172.21.15.36:0/2085151443 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4918062770 0x7f4918062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:46.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f4917fff700 1 -- 172.21.15.36:0/2085151443 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4918062770 msgr2=0x7f4918062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:46.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f4917fff700 1 --2- 172.21.15.36:0/2085151443 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4918062770 0x7f4918062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:46.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f4917fff700 1 -- 172.21.15.36:0/2085151443 >> v1:172.21.15.70:6789/0 conn(0x7f49180ad330 legacy=0x7f49180ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:47.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f4917fff700 1 -- 172.21.15.36:0/2085151443 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4918147e90 con 0x7f49181444f0 2022-06-17T12:46:47.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f4917fff700 1 --2- 172.21.15.36:0/2085151443 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f49181444f0 0x7f4918144930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f4918061170 tx=0x7f490c006eb0).ready entity=mon.0 client_cookie=ab0e97d269eb5700 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:47.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f49177fe700 1 -- 172.21.15.36:0/2085151443 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f490c00d5c0 con 0x7f49181444f0 2022-06-17T12:46:47.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f49177fe700 1 -- 172.21.15.36:0/2085151443 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490c000e90 con 0x7f49181444f0 2022-06-17T12:46:47.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f49177fe700 1 -- 172.21.15.36:0/2085151443 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f490c00d950 con 0x7f49181444f0 2022-06-17T12:46:47.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f491eb14700 1 -- 172.21.15.36:0/2085151443 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f49181444f0 msgr2=0x7f4918144930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:47.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f491eb14700 1 --2- 172.21.15.36:0/2085151443 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f49181444f0 0x7f4918144930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f4918061170 tx=0x7f490c006eb0).stop 2022-06-17T12:46:47.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f491eb14700 1 -- 172.21.15.36:0/2085151443 shutdown_connections 2022-06-17T12:46:47.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f491eb14700 1 --2- 172.21.15.36:0/2085151443 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f49181444f0 0x7f4918144930 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:47.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f491eb14700 1 --2- 172.21.15.36:0/2085151443 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4918062770 0x7f4918062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:47.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f491eb14700 1 -- 172.21.15.36:0/2085151443 >> 172.21.15.36:0/2085151443 conn(0x7f49180b53e0 msgr2=0x7f49180b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:47.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f491eb14700 1 -- 172.21.15.36:0/2085151443 shutdown_connections 2022-06-17T12:46:47.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.910+0000 7f491eb14700 1 -- 172.21.15.36:0/2085151443 wait complete. 2022-06-17T12:46:47.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f491eb14700 1 Processor -- start 2022-06-17T12:46:47.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f491eb14700 1 -- start start 2022-06-17T12:46:47.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f491eb14700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4918062770 0x7f4918157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:47.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f491eb14700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49180ad330 0x7f491814af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:47.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f491eb14700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f49181444f0 0x7f4918149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:47.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f491eb14700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4918149140 con 0x7f4918062770 2022-06-17T12:46:47.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f491eb14700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49181493e0 con 0x7f49180ad330 2022-06-17T12:46:47.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f491eb14700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491814a5d0 con 0x7f49181444f0 2022-06-17T12:46:47.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f491c8b0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4918062770 0x7f4918157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:47.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f491c8b0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4918062770 0x7f4918157480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59334/0 (socket says 172.21.15.36:59334) 2022-06-17T12:46:47.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f491c8b0700 1 -- 172.21.15.36:0/3881735099 learned_addr learned my addr 172.21.15.36:0/3881735099 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:47.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f4917fff700 1 --2- 172.21.15.36:0/3881735099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49180ad330 0x7f491814af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:47.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f491d0b1700 1 --2- 172.21.15.36:0/3881735099 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f49181444f0 0x7f4918149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:47.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f491c8b0700 1 -- 172.21.15.36:0/3881735099 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f49181444f0 msgr2=0x7f4918149780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:47.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f491c8b0700 1 --2- 172.21.15.36:0/3881735099 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f49181444f0 0x7f4918149780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:47.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f491c8b0700 1 -- 172.21.15.36:0/3881735099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49180ad330 msgr2=0x7f491814af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:47.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f491c8b0700 1 --2- 172.21.15.36:0/3881735099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49180ad330 0x7f491814af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:47.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f491c8b0700 1 -- 172.21.15.36:0/3881735099 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c00b1e0 con 0x7f4918062770 2022-06-17T12:46:47.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f4917fff700 1 --2- 172.21.15.36:0/3881735099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49180ad330 0x7f491814af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:46:47.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f491c8b0700 1 --2- 172.21.15.36:0/3881735099 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4918062770 0x7f4918157480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f490800c000 tx=0x7f4908006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:47.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f4915ffb700 1 -- 172.21.15.36:0/3881735099 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4908010780 con 0x7f4918062770 2022-06-17T12:46:47.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f4915ffb700 1 -- 172.21.15.36:0/3881735099 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49080108e0 con 0x7f4918062770 2022-06-17T12:46:47.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f491eb14700 1 -- 172.21.15.36:0/3881735099 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f491814a8b0 con 0x7f4918062770 2022-06-17T12:46:47.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f4915ffb700 1 -- 172.21.15.36:0/3881735099 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4908010c40 con 0x7f4918062770 2022-06-17T12:46:47.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.914+0000 7f491eb14700 1 -- 172.21.15.36:0/3881735099 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4918145550 con 0x7f4918062770 2022-06-17T12:46:47.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.918+0000 7f491eb14700 1 -- 172.21.15.36:0/3881735099 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49180b1c20 con 0x7f4918062770 2022-06-17T12:46:47.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.918+0000 7f4915ffb700 1 -- 172.21.15.36:0/3881735099 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f490801c460 con 0x7f4918062770 2022-06-17T12:46:47.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.918+0000 7f4915ffb700 1 --2- 172.21.15.36:0/3881735099 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f48f8053ad0 0x7f48f8055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:47.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.918+0000 7f4915ffb700 1 -- 172.21.15.36:0/3881735099 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f490806c090 con 0x7f4918062770 2022-06-17T12:46:47.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.918+0000 7f4917fff700 1 --2- 172.21.15.36:0/3881735099 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f48f8053ad0 0x7f48f8055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:47.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.918+0000 7f4917fff700 1 --2- 172.21.15.36:0/3881735099 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f48f8053ad0 0x7f48f8055f90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f490c019000 tx=0x7f490c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:47.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:46.922+0000 7f4915ffb700 1 -- 172.21.15.36:0/3881735099 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f49080432c0 con 0x7f4918062770 2022-06-17T12:46:47.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.046+0000 7f491eb14700 1 -- 172.21.15.36:0/3881735099 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4918149550 con 0x7f4918062770 2022-06-17T12:46:47.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.046+0000 7f4915ffb700 1 -- 172.21.15.36:0/3881735099 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+143 (secure 0 0 0) 0x7f49080468e0 con 0x7f4918062770 2022-06-17T12:46:47.101 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:46:47.101 INFO:teuthology.orchestra.run.smithi036.stdout:[ 2022-06-17T12:46:47.102 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:46:47.102 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 1, 2022-06-17T12:46:47.102 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "device_health_metrics" 2022-06-17T12:46:47.102 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:46:47.103 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:46:47.103 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 2, 2022-06-17T12:46:47.103 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "rbd" 2022-06-17T12:46:47.103 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:46:47.104 INFO:teuthology.orchestra.run.smithi036.stdout:] 2022-06-17T12:46:47.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.046+0000 7f491eb14700 1 -- 172.21.15.36:0/3881735099 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f48f8053ad0 msgr2=0x7f48f8055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:47.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.046+0000 7f491eb14700 1 --2- 172.21.15.36:0/3881735099 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f48f8053ad0 0x7f48f8055f90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f490c019000 tx=0x7f490c015000).stop 2022-06-17T12:46:47.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.046+0000 7f491eb14700 1 -- 172.21.15.36:0/3881735099 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4918062770 msgr2=0x7f4918157480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:47.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.046+0000 7f491eb14700 1 --2- 172.21.15.36:0/3881735099 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4918062770 0x7f4918157480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f490800c000 tx=0x7f4908006df0).stop 2022-06-17T12:46:47.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.050+0000 7f491eb14700 1 -- 172.21.15.36:0/3881735099 shutdown_connections 2022-06-17T12:46:47.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.050+0000 7f491eb14700 1 --2- 172.21.15.36:0/3881735099 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f48f8053ad0 0x7f48f8055f90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:47.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.050+0000 7f491eb14700 1 --2- 172.21.15.36:0/3881735099 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f49181444f0 0x7f4918149780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:47.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.050+0000 7f491eb14700 1 --2- 172.21.15.36:0/3881735099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49180ad330 0x7f491814af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:47.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.050+0000 7f491eb14700 1 --2- 172.21.15.36:0/3881735099 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4918062770 0x7f4918157480 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:47.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.050+0000 7f491eb14700 1 -- 172.21.15.36:0/3881735099 >> 172.21.15.36:0/3881735099 conn(0x7f49180b53e0 msgr2=0x7f49180b4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:47.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.050+0000 7f491eb14700 1 -- 172.21.15.36:0/3881735099 shutdown_connections 2022-06-17T12:46:47.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.050+0000 7f491eb14700 1 -- 172.21.15.36:0/3881735099 wait complete. 2022-06-17T12:46:47.119 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-17T12:46:47.120 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-17T12:46:47.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.270+0000 7f37d9f5e700 1 Processor -- start 2022-06-17T12:46:47.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d9f5e700 1 -- start start 2022-06-17T12:46:47.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d9f5e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f37d40b64f0 0x7f37d4143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:47.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d9f5e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f37d40b4120 0x7f37d4146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:47.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d9f5e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37d4147270 0x7f37d4140100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:47.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d9f5e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d41476e0 con 0x7f37d40b4120 2022-06-17T12:46:47.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d9f5e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d40b68d0 con 0x7f37d40b64f0 2022-06-17T12:46:47.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d9f5e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d40b6e20 con 0x7f37d4147270 2022-06-17T12:46:47.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37caffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f37d40b4120 0x7f37d4146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:47.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37caffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f37d40b4120 0x7f37d4146d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59342/0 (socket says 172.21.15.36:59342) 2022-06-17T12:46:47.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37caffd700 1 -- 172.21.15.36:0/2772784989 learned_addr learned my addr 172.21.15.36:0/2772784989 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:47.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d37fe700 1 --2- 172.21.15.36:0/2772784989 >> v2:172.21.15.70:3300/0 conn(0x7f37d40b64f0 0x7f37d4143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:47.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d3fff700 1 --2- 172.21.15.36:0/2772784989 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37d4147270 0x7f37d4140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:47.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d3fff700 1 -- 172.21.15.36:0/2772784989 >> v2:172.21.15.70:3300/0 conn(0x7f37d40b64f0 msgr2=0x7f37d4143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:47.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d3fff700 1 --2- 172.21.15.36:0/2772784989 >> v2:172.21.15.70:3300/0 conn(0x7f37d40b64f0 0x7f37d4143150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:47.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d3fff700 1 -- 172.21.15.36:0/2772784989 >> v2:172.21.15.36:3300/0 conn(0x7f37d40b4120 msgr2=0x7f37d4146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:47.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d3fff700 1 --2- 172.21.15.36:0/2772784989 >> v2:172.21.15.36:3300/0 conn(0x7f37d40b4120 0x7f37d4146d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:47.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d3fff700 1 -- 172.21.15.36:0/2772784989 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d41468e0 con 0x7f37d4147270 2022-06-17T12:46:47.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d3fff700 1 --2- 172.21.15.36:0/2772784989 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37d4147270 0x7f37d4140100 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f37cc0032f0 tx=0x7f37cc0091d0).ready entity=mon.2 client_cookie=c6e136d3bc2b7020 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:47.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d2ffd700 1 -- 172.21.15.36:0/2772784989 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37cc00b600 con 0x7f37d4147270 2022-06-17T12:46:47.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d2ffd700 1 -- 172.21.15.36:0/2772784989 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37cc00b760 con 0x7f37d4147270 2022-06-17T12:46:47.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d2ffd700 1 -- 172.21.15.36:0/2772784989 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37cc00b8d0 con 0x7f37d4147270 2022-06-17T12:46:47.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d9f5e700 1 -- 172.21.15.36:0/2772784989 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37d4147270 msgr2=0x7f37d4140100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:47.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d9f5e700 1 --2- 172.21.15.36:0/2772784989 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37d4147270 0x7f37d4140100 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f37cc0032f0 tx=0x7f37cc0091d0).stop 2022-06-17T12:46:47.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d9f5e700 1 -- 172.21.15.36:0/2772784989 shutdown_connections 2022-06-17T12:46:47.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d9f5e700 1 --2- 172.21.15.36:0/2772784989 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37d4147270 0x7f37d4140100 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:47.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d9f5e700 1 --2- 172.21.15.36:0/2772784989 >> v2:172.21.15.36:3300/0 conn(0x7f37d40b4120 0x7f37d4146d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:47.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d9f5e700 1 --2- 172.21.15.36:0/2772784989 >> v2:172.21.15.70:3300/0 conn(0x7f37d40b64f0 0x7f37d4143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:47.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.274+0000 7f37d9f5e700 1 -- 172.21.15.36:0/2772784989 >> 172.21.15.36:0/2772784989 conn(0x7f37d40620f0 msgr2=0x7f37d4062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:47.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.278+0000 7f37d9f5e700 1 -- 172.21.15.36:0/2772784989 shutdown_connections 2022-06-17T12:46:47.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.278+0000 7f37d9f5e700 1 -- 172.21.15.36:0/2772784989 wait complete. 2022-06-17T12:46:47.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.278+0000 7f37d9f5e700 1 Processor -- start 2022-06-17T12:46:47.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.278+0000 7f37d9f5e700 1 -- start start 2022-06-17T12:46:47.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.278+0000 7f37d9f5e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37d40b4120 0x7f37d4149530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:47.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.278+0000 7f37d9f5e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37d40b64f0 0x7f37d4155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:47.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.278+0000 7f37d9f5e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37d4147270 0x7f37d41564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:47.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.278+0000 7f37d9f5e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d40b2960 con 0x7f37d40b64f0 2022-06-17T12:46:47.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.278+0000 7f37d9f5e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d40b16c0 con 0x7f37d40b4120 2022-06-17T12:46:47.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.278+0000 7f37d9f5e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d40b0660 con 0x7f37d4147270 2022-06-17T12:46:47.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.278+0000 7f37caffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37d40b64f0 0x7f37d4155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:47.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.278+0000 7f37caffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37d40b64f0 0x7f37d4155fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59350/0 (socket says 172.21.15.36:59350) 2022-06-17T12:46:47.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.278+0000 7f37caffd700 1 -- 172.21.15.36:0/3702435116 learned_addr learned my addr 172.21.15.36:0/3702435116 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:47.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.278+0000 7f37d37fe700 1 --2- 172.21.15.36:0/3702435116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37d40b4120 0x7f37d4149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:47.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.278+0000 7f37d3fff700 1 --2- 172.21.15.36:0/3702435116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37d4147270 0x7f37d41564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:47.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.278+0000 7f37caffd700 1 -- 172.21.15.36:0/3702435116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37d4147270 msgr2=0x7f37d41564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:47.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.278+0000 7f37caffd700 1 --2- 172.21.15.36:0/3702435116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37d4147270 0x7f37d41564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:47.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.278+0000 7f37caffd700 1 -- 172.21.15.36:0/3702435116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37d40b4120 msgr2=0x7f37d4149530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:47.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.278+0000 7f37caffd700 1 --2- 172.21.15.36:0/3702435116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37d40b4120 0x7f37d4149530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:47.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.278+0000 7f37caffd700 1 -- 172.21.15.36:0/3702435116 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37cc009470 con 0x7f37d40b64f0 2022-06-17T12:46:47.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.278+0000 7f37d3fff700 1 --2- 172.21.15.36:0/3702435116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37d4147270 0x7f37d41564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:46:47.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.278+0000 7f37caffd700 1 --2- 172.21.15.36:0/3702435116 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37d40b64f0 0x7f37d4155fb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f37bc00c000 tx=0x7f37bc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:47.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.282+0000 7f37d17fa700 1 -- 172.21.15.36:0/3702435116 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37bc013610 con 0x7f37d40b64f0 2022-06-17T12:46:47.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.282+0000 7f37d17fa700 1 -- 172.21.15.36:0/3702435116 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37bc013770 con 0x7f37d40b64f0 2022-06-17T12:46:47.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.282+0000 7f37d9f5e700 1 -- 172.21.15.36:0/3702435116 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d4155970 con 0x7f37d40b64f0 2022-06-17T12:46:47.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.282+0000 7f37d17fa700 1 -- 172.21.15.36:0/3702435116 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37bc013ad0 con 0x7f37d40b64f0 2022-06-17T12:46:47.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.282+0000 7f37d9f5e700 1 -- 172.21.15.36:0/3702435116 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d40b0aa0 con 0x7f37d40b64f0 2022-06-17T12:46:47.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.282+0000 7f37d9f5e700 1 -- 172.21.15.36:0/3702435116 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d40b0820 con 0x7f37d40b64f0 2022-06-17T12:46:47.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.282+0000 7f37d17fa700 1 -- 172.21.15.36:0/3702435116 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f37bc013c30 con 0x7f37d40b64f0 2022-06-17T12:46:47.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.282+0000 7f37d17fa700 1 --2- 172.21.15.36:0/3702435116 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f37b80539b0 0x7f37b8055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:47.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.282+0000 7f37d17fa700 1 -- 172.21.15.36:0/3702435116 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f37bc06bbb0 con 0x7f37d40b64f0 2022-06-17T12:46:47.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.282+0000 7f37d37fe700 1 --2- 172.21.15.36:0/3702435116 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f37b80539b0 0x7f37b8055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:47.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.286+0000 7f37d37fe700 1 --2- 172.21.15.36:0/3702435116 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f37b80539b0 0x7f37b8055e70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f37c400b000 tx=0x7f37c4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:47.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.286+0000 7f37d17fa700 1 -- 172.21.15.36:0/3702435116 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f37bc072080 con 0x7f37d40b64f0 2022-06-17T12:46:47.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.414+0000 7f37d9f5e700 1 -- 172.21.15.36:0/3702435116 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f37d40b04b0 con 0x7f37d40b64f0 2022-06-17T12:46:47.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.838+0000 7f37d17fa700 1 -- 172.21.15.36:0/3702435116 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v20) v1 ==== 133+0+0 (secure 0 0 0) 0x7f37bc042e40 con 0x7f37d40b64f0 2022-06-17T12:46:47.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.842+0000 7f37d9f5e700 1 -- 172.21.15.36:0/3702435116 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f37b80539b0 msgr2=0x7f37b8055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:47.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.842+0000 7f37d9f5e700 1 --2- 172.21.15.36:0/3702435116 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f37b80539b0 0x7f37b8055e70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f37c400b000 tx=0x7f37c4008040).stop 2022-06-17T12:46:47.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.842+0000 7f37d9f5e700 1 -- 172.21.15.36:0/3702435116 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37d40b64f0 msgr2=0x7f37d4155fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:47.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.842+0000 7f37d9f5e700 1 --2- 172.21.15.36:0/3702435116 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37d40b64f0 0x7f37d4155fb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f37bc00c000 tx=0x7f37bc008040).stop 2022-06-17T12:46:47.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.846+0000 7f37d9f5e700 1 -- 172.21.15.36:0/3702435116 shutdown_connections 2022-06-17T12:46:47.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.846+0000 7f37d9f5e700 1 --2- 172.21.15.36:0/3702435116 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f37b80539b0 0x7f37b8055e70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:47.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.846+0000 7f37d9f5e700 1 --2- 172.21.15.36:0/3702435116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37d4147270 0x7f37d41564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:47.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.846+0000 7f37d9f5e700 1 --2- 172.21.15.36:0/3702435116 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37d40b64f0 0x7f37d4155fb0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:47.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.846+0000 7f37d9f5e700 1 --2- 172.21.15.36:0/3702435116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37d40b4120 0x7f37d4149530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:47.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.846+0000 7f37d9f5e700 1 -- 172.21.15.36:0/3702435116 >> 172.21.15.36:0/3702435116 conn(0x7f37d40620f0 msgr2=0x7f37d413f230 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:47.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.846+0000 7f37d9f5e700 1 -- 172.21.15.36:0/3702435116 shutdown_connections 2022-06-17T12:46:47.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:47.846+0000 7f37d9f5e700 1 -- 172.21.15.36:0/3702435116 wait complete. 2022-06-17T12:46:47.859 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' created 2022-06-17T12:46:47.860 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2022-06-17T12:46:47.861 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data m=2 k=2 crush-failure-domain=osd 2022-06-17T12:46:48.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.006+0000 7fd84ef79700 1 Processor -- start 2022-06-17T12:46:48.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.006+0000 7fd84ef79700 1 -- start start 2022-06-17T12:46:48.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd84ef79700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8480b4e10 0x7fd8480b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:48.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd84ef79700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd8480b31c0 0x7fd8480b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:48.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd84ef79700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8480b21d0 con 0x7fd8480620d0 2022-06-17T12:46:48.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd84ef79700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8480b2430 con 0x7fd8480b31c0 2022-06-17T12:46:48.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd84ef79700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8480b26d0 con 0x7fd8480b4e10 2022-06-17T12:46:48.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd84cd15700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fd8480620d0 0x7fd8480b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:48654/0 (socket says 172.21.15.36:48654) 2022-06-17T12:46:48.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd84cd15700 1 -- 172.21.15.36:0/783607969 learned_addr learned my addr 172.21.15.36:0/783607969 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:46:48.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd847fff700 1 --2- 172.21.15.36:0/783607969 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8480b4e10 0x7fd8480b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:48.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd84d516700 1 --2- 172.21.15.36:0/783607969 >> v2:172.21.15.70:3300/0 conn(0x7fd8480b31c0 0x7fd8480b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:48.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd8477fe700 1 -- 172.21.15.36:0/783607969 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1566014469 0 0) 0x7fd8480b21d0 con 0x7fd8480620d0 2022-06-17T12:46:48.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd8477fe700 1 -- 172.21.15.36:0/783607969 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd828003260 con 0x7fd8480620d0 2022-06-17T12:46:48.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd8477fe700 1 -- 172.21.15.36:0/783607969 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1233530591 0 0) 0x7fd828003260 con 0x7fd8480620d0 2022-06-17T12:46:48.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd8477fe700 1 -- 172.21.15.36:0/783607969 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8480b4e10 msgr2=0x7fd8480b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:48.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd8477fe700 1 --2- 172.21.15.36:0/783607969 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8480b4e10 0x7fd8480b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:48.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd8477fe700 1 -- 172.21.15.36:0/783607969 >> v2:172.21.15.70:3300/0 conn(0x7fd8480b31c0 msgr2=0x7fd8480b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:48.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd8477fe700 1 --2- 172.21.15.36:0/783607969 >> v2:172.21.15.70:3300/0 conn(0x7fd8480b31c0 0x7fd8480b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:48.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd8477fe700 1 -- 172.21.15.36:0/783607969 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8480b43f0 con 0x7fd8480620d0 2022-06-17T12:46:48.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd8477fe700 1 -- 172.21.15.36:0/783607969 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7fd830001680 con 0x7fd8480620d0 2022-06-17T12:46:48.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd8477fe700 1 -- 172.21.15.36:0/783607969 >> v1:172.21.15.36:6789/0 conn(0x7fd8480620d0 legacy=0x7fd8480b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:48.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd8477fe700 1 --2- 172.21.15.36:0/783607969 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd828003750 0x7fd828003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:48.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd8477fe700 1 --2- 172.21.15.36:0/783607969 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8480b4e10 0x7fd828004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:48.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd8477fe700 1 --2- 172.21.15.36:0/783607969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8280049f0 0x7fd828004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:48.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd8477fe700 1 -- 172.21.15.36:0/783607969 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8480b2430 con 0x7fd828003750 2022-06-17T12:46:48.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd8477fe700 1 -- 172.21.15.36:0/783607969 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8480b26d0 con 0x7fd8280049f0 2022-06-17T12:46:48.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd8477fe700 1 -- 172.21.15.36:0/783607969 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd82800b750 con 0x7fd8480b4e10 2022-06-17T12:46:48.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd84cd15700 1 --2- 172.21.15.36:0/783607969 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd828003750 0x7fd828003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:48.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd84d516700 1 --2- 172.21.15.36:0/783607969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8280049f0 0x7fd828004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:48.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd847fff700 1 --2- 172.21.15.36:0/783607969 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8480b4e10 0x7fd828004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:48.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd84cd15700 1 -- 172.21.15.36:0/783607969 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8480b4e10 msgr2=0x7fd828004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:48.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd84cd15700 1 --2- 172.21.15.36:0/783607969 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8480b4e10 0x7fd828004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:48.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd84cd15700 1 -- 172.21.15.36:0/783607969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8280049f0 msgr2=0x7fd828004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:48.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd84cd15700 1 --2- 172.21.15.36:0/783607969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8280049f0 0x7fd828004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:48.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd84cd15700 1 -- 172.21.15.36:0/783607969 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd828003260 con 0x7fd828003750 2022-06-17T12:46:48.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd84cd15700 1 --2- 172.21.15.36:0/783607969 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd828003750 0x7fd828003b70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fd830001390 tx=0x7fd830001e70).ready entity=mon.0 client_cookie=d5574b15d6ab61af server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:48.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd8477fe700 1 -- 172.21.15.36:0/783607969 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd830007170 con 0x7fd828003750 2022-06-17T12:46:48.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd8477fe700 1 -- 172.21.15.36:0/783607969 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8300072d0 con 0x7fd828003750 2022-06-17T12:46:48.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.010+0000 7fd8477fe700 1 -- 172.21.15.36:0/783607969 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd83000c8f0 con 0x7fd828003750 2022-06-17T12:46:48.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.014+0000 7fd84ef79700 1 -- 172.21.15.36:0/783607969 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd828003750 msgr2=0x7fd828003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:48.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.014+0000 7fd84ef79700 1 --2- 172.21.15.36:0/783607969 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd828003750 0x7fd828003b70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fd830001390 tx=0x7fd830001e70).stop 2022-06-17T12:46:48.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.014+0000 7fd84d516700 1 -- 172.21.15.36:0/783607969 reap_dead start 2022-06-17T12:46:48.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.014+0000 7fd84ef79700 1 -- 172.21.15.36:0/783607969 shutdown_connections 2022-06-17T12:46:48.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.014+0000 7fd84ef79700 1 -- 172.21.15.36:0/783607969 >> 172.21.15.36:0/783607969 conn(0x7fd8480b9850 msgr2=0x7fd8480b9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:48.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.014+0000 7fd84ef79700 1 -- 172.21.15.36:0/783607969 shutdown_connections 2022-06-17T12:46:48.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.014+0000 7fd84ef79700 1 -- 172.21.15.36:0/783607969 wait complete. 2022-06-17T12:46:48.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.014+0000 7fd84ef79700 1 Processor -- start 2022-06-17T12:46:48.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.014+0000 7fd84ef79700 1 -- start start 2022-06-17T12:46:48.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.018+0000 7fd84ef79700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd828003750 0x7fd8481142e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:48.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.018+0000 7fd84ef79700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8481136f0 0x7fd848155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:48.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.018+0000 7fd84ef79700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8480afe70 0x7fd848156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:48.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.018+0000 7fd84ef79700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd848116720 con 0x7fd828003750 2022-06-17T12:46:48.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.018+0000 7fd84ef79700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd848116980 con 0x7fd8480afe70 2022-06-17T12:46:48.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.018+0000 7fd84ef79700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd848112bc0 con 0x7fd8481136f0 2022-06-17T12:46:48.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.018+0000 7fd84cd15700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd828003750 0x7fd8481142e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:48.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.018+0000 7fd84cd15700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd828003750 0x7fd8481142e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59368/0 (socket says 172.21.15.36:59368) 2022-06-17T12:46:48.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.018+0000 7fd84cd15700 1 -- 172.21.15.36:0/2962732794 learned_addr learned my addr 172.21.15.36:0/2962732794 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:48.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.018+0000 7fd84d516700 1 --2- 172.21.15.36:0/2962732794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8480afe70 0x7fd848156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:48.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.018+0000 7fd847fff700 1 --2- 172.21.15.36:0/2962732794 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8481136f0 0x7fd848155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:48.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.018+0000 7fd84cd15700 1 -- 172.21.15.36:0/2962732794 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8481136f0 msgr2=0x7fd848155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:48.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.018+0000 7fd84cd15700 1 --2- 172.21.15.36:0/2962732794 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8481136f0 0x7fd848155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:48.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.018+0000 7fd84cd15700 1 -- 172.21.15.36:0/2962732794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8480afe70 msgr2=0x7fd848156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:48.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.018+0000 7fd84cd15700 1 --2- 172.21.15.36:0/2962732794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8480afe70 0x7fd848156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:48.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.018+0000 7fd84cd15700 1 -- 172.21.15.36:0/2962732794 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd848156880 con 0x7fd828003750 2022-06-17T12:46:48.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.018+0000 7fd84cd15700 1 --2- 172.21.15.36:0/2962732794 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd828003750 0x7fd8481142e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fd830001390 tx=0x7fd830015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:48.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.018+0000 7fd845ffb700 1 -- 172.21.15.36:0/2962732794 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd830007170 con 0x7fd828003750 2022-06-17T12:46:48.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.018+0000 7fd845ffb700 1 -- 172.21.15.36:0/2962732794 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd830004520 con 0x7fd828003750 2022-06-17T12:46:48.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.018+0000 7fd84ef79700 1 -- 172.21.15.36:0/2962732794 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd848156ad0 con 0x7fd828003750 2022-06-17T12:46:48.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.018+0000 7fd845ffb700 1 -- 172.21.15.36:0/2962732794 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd830004880 con 0x7fd828003750 2022-06-17T12:46:48.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.018+0000 7fd84ef79700 1 -- 172.21.15.36:0/2962732794 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd84813f3d0 con 0x7fd828003750 2022-06-17T12:46:48.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.022+0000 7fd84ef79700 1 -- 172.21.15.36:0/2962732794 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd848112d80 con 0x7fd828003750 2022-06-17T12:46:48.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.022+0000 7fd845ffb700 1 -- 172.21.15.36:0/2962732794 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd830004bf0 con 0x7fd828003750 2022-06-17T12:46:48.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.022+0000 7fd845ffb700 1 --2- 172.21.15.36:0/2962732794 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd82c053a40 0x7fd82c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:48.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.022+0000 7fd845ffb700 1 -- 172.21.15.36:0/2962732794 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fd83001c040 con 0x7fd828003750 2022-06-17T12:46:48.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.022+0000 7fd847fff700 1 --2- 172.21.15.36:0/2962732794 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd82c053a40 0x7fd82c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:48.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.022+0000 7fd847fff700 1 --2- 172.21.15.36:0/2962732794 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd82c053a40 0x7fd82c055f00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fd83800b000 tx=0x7fd838006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:48.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.026+0000 7fd845ffb700 1 -- 172.21.15.36:0/2962732794 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd8300445a0 con 0x7fd828003750 2022-06-17T12:46:48.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.162+0000 7fd84ef79700 1 -- 172.21.15.36:0/2962732794 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fd8480624b0 con 0x7fd828003750 2022-06-17T12:46:48.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.841+0000 7fd845ffb700 1 -- 172.21.15.36:0/2962732794 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 154+0+0 (secure 0 0 0) 0x7fd830047bc0 con 0x7fd828003750 2022-06-17T12:46:48.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.845+0000 7fd84ef79700 1 -- 172.21.15.36:0/2962732794 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd82c053a40 msgr2=0x7fd82c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:48.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.845+0000 7fd84ef79700 1 --2- 172.21.15.36:0/2962732794 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd82c053a40 0x7fd82c055f00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fd83800b000 tx=0x7fd838006df0).stop 2022-06-17T12:46:48.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.845+0000 7fd84ef79700 1 -- 172.21.15.36:0/2962732794 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd828003750 msgr2=0x7fd8481142e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:48.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.845+0000 7fd84ef79700 1 --2- 172.21.15.36:0/2962732794 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd828003750 0x7fd8481142e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fd830001390 tx=0x7fd830015000).stop 2022-06-17T12:46:48.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.849+0000 7fd84ef79700 1 -- 172.21.15.36:0/2962732794 shutdown_connections 2022-06-17T12:46:48.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.849+0000 7fd84ef79700 1 --2- 172.21.15.36:0/2962732794 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd82c053a40 0x7fd82c055f00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:48.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.849+0000 7fd84ef79700 1 --2- 172.21.15.36:0/2962732794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8480afe70 0x7fd848156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:48.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.849+0000 7fd84ef79700 1 --2- 172.21.15.36:0/2962732794 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8481136f0 0x7fd848155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:48.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.849+0000 7fd84ef79700 1 --2- 172.21.15.36:0/2962732794 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd828003750 0x7fd8481142e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:48.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.849+0000 7fd84ef79700 1 -- 172.21.15.36:0/2962732794 >> 172.21.15.36:0/2962732794 conn(0x7fd8480b9850 msgr2=0x7fd8480b9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:48.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.849+0000 7fd84ef79700 1 -- 172.21.15.36:0/2962732794 shutdown_connections 2022-06-17T12:46:48.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.849+0000 7fd84ef79700 1 -- 172.21.15.36:0/2962732794 wait complete. 2022-06-17T12:46:48.850 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 erasure cephfs_data 2022-06-17T12:46:48.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.985+0000 7fb32085b700 1 Processor -- start 2022-06-17T12:46:48.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.985+0000 7fb32085b700 1 -- start start 2022-06-17T12:46:48.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.989+0000 7fb32085b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb31c0b6780 0x7fb31c144300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:48.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.989+0000 7fb32085b700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb31c05faa0 0x7fb31c1447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:48.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.989+0000 7fb32085b700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb31c0b6b60 con 0x7fb31c144ce0 2022-06-17T12:46:48.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.989+0000 7fb32085b700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb31c060900 con 0x7fb31c0b6780 2022-06-17T12:46:48.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.989+0000 7fb32085b700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb31c067710 con 0x7fb31c05faa0 2022-06-17T12:46:48.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.989+0000 7fb31a59c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb31c0b6780 0x7fb31c144300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:48.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.989+0000 7fb31a59c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb31c0b6780 0x7fb31c144300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59376/0 (socket says 172.21.15.36:59376) 2022-06-17T12:46:48.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.989+0000 7fb31a59c700 1 -- 172.21.15.36:0/773616801 learned_addr learned my addr 172.21.15.36:0/773616801 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:48.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.989+0000 7fb319d9b700 1 --2- 172.21.15.36:0/773616801 >> v2:172.21.15.70:3300/0 conn(0x7fb31c05faa0 0x7fb31c1447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:48.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.989+0000 7fb31a59c700 1 -- 172.21.15.36:0/773616801 >> v2:172.21.15.70:3300/0 conn(0x7fb31c05faa0 msgr2=0x7fb31c1447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:48.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.989+0000 7fb31a59c700 1 --2- 172.21.15.36:0/773616801 >> v2:172.21.15.70:3300/0 conn(0x7fb31c05faa0 0x7fb31c1447f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:48.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.989+0000 7fb31a59c700 1 -- 172.21.15.36:0/773616801 >> v1:172.21.15.70:6789/0 conn(0x7fb31c144ce0 legacy=0x7fb31c0b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:48.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.989+0000 7fb31a59c700 1 -- 172.21.15.36:0/773616801 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31c060580 con 0x7fb31c0b6780 2022-06-17T12:46:49.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.989+0000 7fb31a59c700 1 --2- 172.21.15.36:0/773616801 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb31c0b6780 0x7fb31c144300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fb30c002f70 tx=0x7fb30c006eb0).ready entity=mon.0 client_cookie=45af1bff3ff2e2c9 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:49.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.989+0000 7fb31959a700 1 -- 172.21.15.36:0/773616801 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb30c00d5c0 con 0x7fb31c0b6780 2022-06-17T12:46:49.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.989+0000 7fb31959a700 1 -- 172.21.15.36:0/773616801 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c000e90 con 0x7fb31c0b6780 2022-06-17T12:46:49.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.989+0000 7fb31959a700 1 -- 172.21.15.36:0/773616801 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb30c00d950 con 0x7fb31c0b6780 2022-06-17T12:46:49.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.989+0000 7fb32085b700 1 -- 172.21.15.36:0/773616801 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb31c0b6780 msgr2=0x7fb31c144300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:49.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.989+0000 7fb32085b700 1 --2- 172.21.15.36:0/773616801 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb31c0b6780 0x7fb31c144300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fb30c002f70 tx=0x7fb30c006eb0).stop 2022-06-17T12:46:49.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.989+0000 7fb32085b700 1 -- 172.21.15.36:0/773616801 shutdown_connections 2022-06-17T12:46:49.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.989+0000 7fb32085b700 1 --2- 172.21.15.36:0/773616801 >> v2:172.21.15.70:3300/0 conn(0x7fb31c05faa0 0x7fb31c1447f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:49.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.989+0000 7fb32085b700 1 --2- 172.21.15.36:0/773616801 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb31c0b6780 0x7fb31c144300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:49.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.993+0000 7fb32085b700 1 -- 172.21.15.36:0/773616801 >> 172.21.15.36:0/773616801 conn(0x7fb31c063890 msgr2=0x7fb31c0b5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:49.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.993+0000 7fb32085b700 1 -- 172.21.15.36:0/773616801 shutdown_connections 2022-06-17T12:46:49.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.993+0000 7fb32085b700 1 -- 172.21.15.36:0/773616801 wait complete. 2022-06-17T12:46:49.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.993+0000 7fb32085b700 1 Processor -- start 2022-06-17T12:46:49.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.993+0000 7fb32085b700 1 -- start start 2022-06-17T12:46:49.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.993+0000 7fb32085b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb31c05faa0 0x7fb31c1149c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:49.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.993+0000 7fb32085b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb31c0b6780 0x7fb31c152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:49.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.993+0000 7fb32085b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb31c0649b0 0x7fb31c143ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:49.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.993+0000 7fb32085b700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c114eb0 con 0x7fb31c0649b0 2022-06-17T12:46:49.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.993+0000 7fb32085b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c0651d0 con 0x7fb31c0b6780 2022-06-17T12:46:49.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.993+0000 7fb32085b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c152940 con 0x7fb31c05faa0 2022-06-17T12:46:49.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.993+0000 7fb31ad9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb31c0649b0 0x7fb31c143ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:49.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.993+0000 7fb31ad9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb31c0649b0 0x7fb31c143ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59386/0 (socket says 172.21.15.36:59386) 2022-06-17T12:46:49.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.993+0000 7fb31ad9d700 1 -- 172.21.15.36:0/1806068272 learned_addr learned my addr 172.21.15.36:0/1806068272 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:49.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.993+0000 7fb319d9b700 1 --2- 172.21.15.36:0/1806068272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb31c0b6780 0x7fb31c152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:49.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.997+0000 7fb31a59c700 1 --2- 172.21.15.36:0/1806068272 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb31c05faa0 0x7fb31c1149c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:49.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.997+0000 7fb31ad9d700 1 -- 172.21.15.36:0/1806068272 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb31c05faa0 msgr2=0x7fb31c1149c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:49.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.997+0000 7fb31ad9d700 1 --2- 172.21.15.36:0/1806068272 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb31c05faa0 0x7fb31c1149c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:49.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.997+0000 7fb31ad9d700 1 -- 172.21.15.36:0/1806068272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb31c0b6780 msgr2=0x7fb31c152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:49.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.997+0000 7fb31ad9d700 1 --2- 172.21.15.36:0/1806068272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb31c0b6780 0x7fb31c152450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:49.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.997+0000 7fb31ad9d700 1 -- 172.21.15.36:0/1806068272 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb30c00b1e0 con 0x7fb31c0649b0 2022-06-17T12:46:49.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.997+0000 7fb319d9b700 1 --2- 172.21.15.36:0/1806068272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb31c0b6780 0x7fb31c152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:46:49.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.997+0000 7fb31a59c700 1 --2- 172.21.15.36:0/1806068272 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb31c05faa0 0x7fb31c1149c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:46:49.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.997+0000 7fb31ad9d700 1 --2- 172.21.15.36:0/1806068272 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb31c0649b0 0x7fb31c143ae0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fb31c1137c0 tx=0x7fb3140013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:49.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.997+0000 7fb30b7fe700 1 -- 172.21.15.36:0/1806068272 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb314006aa0 con 0x7fb31c0649b0 2022-06-17T12:46:49.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.997+0000 7fb30b7fe700 1 -- 172.21.15.36:0/1806068272 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb314006c00 con 0x7fb31c0649b0 2022-06-17T12:46:49.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.997+0000 7fb32085b700 1 -- 172.21.15.36:0/1806068272 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb31c064f30 con 0x7fb31c0649b0 2022-06-17T12:46:49.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.997+0000 7fb30b7fe700 1 -- 172.21.15.36:0/1806068272 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb314007030 con 0x7fb31c0649b0 2022-06-17T12:46:49.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.997+0000 7fb32085b700 1 -- 172.21.15.36:0/1806068272 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31c143fd0 con 0x7fb31c0649b0 2022-06-17T12:46:49.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.997+0000 7fb32085b700 1 -- 172.21.15.36:0/1806068272 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb31c057410 con 0x7fb31c0649b0 2022-06-17T12:46:49.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.997+0000 7fb30b7fe700 1 -- 172.21.15.36:0/1806068272 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fb314010030 con 0x7fb31c0649b0 2022-06-17T12:46:49.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.997+0000 7fb30b7fe700 1 --2- 172.21.15.36:0/1806068272 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb2f8053a60 0x7fb2f8055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:49.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:48.997+0000 7fb30b7fe700 1 -- 172.21.15.36:0/1806068272 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5863+0+0 (secure 0 0 0) 0x7fb31406bcc0 con 0x7fb31c0649b0 2022-06-17T12:46:49.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:49.001+0000 7fb31a59c700 1 --2- 172.21.15.36:0/1806068272 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb2f8053a60 0x7fb2f8055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:49.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:49.001+0000 7fb31a59c700 1 --2- 172.21.15.36:0/1806068272 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb2f8053a60 0x7fb2f8055f20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fb30c01a000 tx=0x7fb30c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:49.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:49.005+0000 7fb30b7fe700 1 -- 172.21.15.36:0/1806068272 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb314042cf0 con 0x7fb31c0649b0 2022-06-17T12:46:49.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:49.133+0000 7fb32085b700 1 -- 172.21.15.36:0/1806068272 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8, "pool_type": "erasure", "erasure_code_profile": "cephfs_data"} v 0) v1 -- 0x7fb31c05d820 con 0x7fb31c0649b0 2022-06-17T12:46:50.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:50.860+0000 7fb30b7fe700 1 -- 172.21.15.36:0/1806068272 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8, "pool_type": "erasure", "erasure_code_profile": "cephfs_data"}]=0 pool 'cephfs_data' created v23) v1 ==== 188+0+0 (secure 0 0 0) 0x7fb314046310 con 0x7fb31c0649b0 2022-06-17T12:46:50.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:50.864+0000 7fb32085b700 1 -- 172.21.15.36:0/1806068272 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb2f8053a60 msgr2=0x7fb2f8055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:50.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:50.864+0000 7fb32085b700 1 --2- 172.21.15.36:0/1806068272 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb2f8053a60 0x7fb2f8055f20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fb30c01a000 tx=0x7fb30c015000).stop 2022-06-17T12:46:50.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:50.864+0000 7fb32085b700 1 -- 172.21.15.36:0/1806068272 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb31c0649b0 msgr2=0x7fb31c143ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:50.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:50.864+0000 7fb32085b700 1 --2- 172.21.15.36:0/1806068272 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb31c0649b0 0x7fb31c143ae0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fb31c1137c0 tx=0x7fb3140013a0).stop 2022-06-17T12:46:50.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:50.864+0000 7fb32085b700 1 -- 172.21.15.36:0/1806068272 shutdown_connections 2022-06-17T12:46:50.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:50.864+0000 7fb32085b700 1 --2- 172.21.15.36:0/1806068272 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb2f8053a60 0x7fb2f8055f20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:50.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:50.864+0000 7fb32085b700 1 --2- 172.21.15.36:0/1806068272 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb31c0649b0 0x7fb31c143ae0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:50.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:50.864+0000 7fb32085b700 1 --2- 172.21.15.36:0/1806068272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb31c0b6780 0x7fb31c152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:50.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:50.864+0000 7fb32085b700 1 --2- 172.21.15.36:0/1806068272 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb31c05faa0 0x7fb31c1149c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:50.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:50.864+0000 7fb32085b700 1 -- 172.21.15.36:0/1806068272 >> 172.21.15.36:0/1806068272 conn(0x7fb31c063890 msgr2=0x7fb31c1416e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:50.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:50.864+0000 7fb32085b700 1 -- 172.21.15.36:0/1806068272 shutdown_connections 2022-06-17T12:46:50.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:50.868+0000 7fb32085b700 1 -- 172.21.15.36:0/1806068272 wait complete. 2022-06-17T12:46:50.889 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' created 2022-06-17T12:46:50.891 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data allow_ec_overwrites true 2022-06-17T12:46:51.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.032+0000 7fa5aea9b700 1 Processor -- start 2022-06-17T12:46:51.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.032+0000 7fa5aea9b700 1 -- start start 2022-06-17T12:46:51.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.032+0000 7fa5aea9b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5a80b0cb0 0x7fa5a8142f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:51.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.032+0000 7fa5aea9b700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a80b44b0 con 0x7fa5a8143e80 2022-06-17T12:46:51.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.032+0000 7fa5aea9b700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a8062650 con 0x7fa5a8062270 2022-06-17T12:46:51.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.032+0000 7fa5aea9b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a80b6650 con 0x7fa5a80b0cb0 2022-06-17T12:46:51.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.032+0000 7fa5ad038700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fa5a8143e80 0x7fa5a8146970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:48692/0 (socket says 172.21.15.36:48692) 2022-06-17T12:46:51.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.032+0000 7fa5ad038700 1 -- 172.21.15.36:0/2035160054 learned_addr learned my addr 172.21.15.36:0/2035160054 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:46:51.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.036+0000 7fa5a7fff700 1 --2- 172.21.15.36:0/2035160054 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5a80b0cb0 0x7fa5a8142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:51.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.036+0000 7fa5a77fe700 1 -- 172.21.15.36:0/2035160054 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2019757426 0 0) 0x7fa5a80b44b0 con 0x7fa5a8143e80 2022-06-17T12:46:51.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.036+0000 7fa5a77fe700 1 -- 172.21.15.36:0/2035160054 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa594003260 con 0x7fa5a8143e80 2022-06-17T12:46:51.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.036+0000 7fa5a7fff700 1 -- 172.21.15.36:0/2035160054 >> v1:172.21.15.70:6789/0 conn(0x7fa5a8062270 legacy=0x7fa5a8143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:51.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.036+0000 7fa5a7fff700 1 -- 172.21.15.36:0/2035160054 >> v1:172.21.15.36:6789/0 conn(0x7fa5a8143e80 legacy=0x7fa5a8146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:51.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.036+0000 7fa5a77fe700 1 -- 172.21.15.36:0/2035160054 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1699167142 0 0) 0x7fa5a8062650 con 0x7fa5a8062270 2022-06-17T12:46:51.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.036+0000 7fa5a7fff700 1 -- 172.21.15.36:0/2035160054 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a80b4210 con 0x7fa5a80b0cb0 2022-06-17T12:46:51.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.036+0000 7fa5a7fff700 1 --2- 172.21.15.36:0/2035160054 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5a80b0cb0 0x7fa5a8142f20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fa598002f70 tx=0x7fa598006df0).ready entity=mon.2 client_cookie=94226b0f6ebe4479 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:51.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.036+0000 7fa5a77fe700 1 -- 172.21.15.36:0/2035160054 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa59800d570 con 0x7fa5a80b0cb0 2022-06-17T12:46:51.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.036+0000 7fa5a77fe700 1 -- 172.21.15.36:0/2035160054 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa598000e90 con 0x7fa5a80b0cb0 2022-06-17T12:46:51.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.036+0000 7fa5a77fe700 1 -- 172.21.15.36:0/2035160054 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa59800d8c0 con 0x7fa5a80b0cb0 2022-06-17T12:46:51.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.036+0000 7fa5aea9b700 1 -- 172.21.15.36:0/2035160054 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5a80b0cb0 msgr2=0x7fa5a8142f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:51.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.036+0000 7fa5aea9b700 1 --2- 172.21.15.36:0/2035160054 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5a80b0cb0 0x7fa5a8142f20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fa598002f70 tx=0x7fa598006df0).stop 2022-06-17T12:46:51.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.036+0000 7fa5aea9b700 1 -- 172.21.15.36:0/2035160054 shutdown_connections 2022-06-17T12:46:51.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.036+0000 7fa5aea9b700 1 --2- 172.21.15.36:0/2035160054 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5a80b0cb0 0x7fa5a8142f20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:51.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.036+0000 7fa5aea9b700 1 -- 172.21.15.36:0/2035160054 >> 172.21.15.36:0/2035160054 conn(0x7fa5a80b9870 msgr2=0x7fa5a80b9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:51.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.036+0000 7fa5aea9b700 1 -- 172.21.15.36:0/2035160054 shutdown_connections 2022-06-17T12:46:51.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.036+0000 7fa5aea9b700 1 -- 172.21.15.36:0/2035160054 wait complete. 2022-06-17T12:46:51.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5aea9b700 1 Processor -- start 2022-06-17T12:46:51.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5aea9b700 1 -- start start 2022-06-17T12:46:51.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5aea9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5a8062270 0x7fa5a80b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:51.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5aea9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa5a8143e80 0x7fa5a80ad610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:51.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5aea9b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5a8145f40 0x7fa5a80adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:51.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5aea9b700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a8147450 con 0x7fa5a8143e80 2022-06-17T12:46:51.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5aea9b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a8147640 con 0x7fa5a8062270 2022-06-17T12:46:51.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5aea9b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a81478e0 con 0x7fa5a8145f40 2022-06-17T12:46:51.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5a7fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa5a8143e80 0x7fa5a80ad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:51.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5a7fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa5a8143e80 0x7fa5a80ad610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59396/0 (socket says 172.21.15.36:59396) 2022-06-17T12:46:51.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5a7fff700 1 -- 172.21.15.36:0/2582219834 learned_addr learned my addr 172.21.15.36:0/2582219834 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:51.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5ac837700 1 --2- 172.21.15.36:0/2582219834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5a8062270 0x7fa5a80b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:51.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5ad038700 1 --2- 172.21.15.36:0/2582219834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5a8145f40 0x7fa5a80adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:51.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5a7fff700 1 -- 172.21.15.36:0/2582219834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5a8145f40 msgr2=0x7fa5a80adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:51.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5a7fff700 1 --2- 172.21.15.36:0/2582219834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5a8145f40 0x7fa5a80adb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:51.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5a7fff700 1 -- 172.21.15.36:0/2582219834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5a8062270 msgr2=0x7fa5a80b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:51.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5a7fff700 1 --2- 172.21.15.36:0/2582219834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5a8062270 0x7fa5a80b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:51.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5a7fff700 1 -- 172.21.15.36:0/2582219834 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59800b180 con 0x7fa5a8143e80 2022-06-17T12:46:51.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5ad038700 1 --2- 172.21.15.36:0/2582219834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5a8145f40 0x7fa5a80adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:46:51.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5a7fff700 1 --2- 172.21.15.36:0/2582219834 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa5a8143e80 0x7fa5a80ad610 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fa598002f70 tx=0x7fa598014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:51.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5a5ffb700 1 -- 172.21.15.36:0/2582219834 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa598016070 con 0x7fa5a8143e80 2022-06-17T12:46:51.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5a5ffb700 1 -- 172.21.15.36:0/2582219834 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5980139c0 con 0x7fa5a8143e80 2022-06-17T12:46:51.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5aea9b700 1 -- 172.21.15.36:0/2582219834 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a8144530 con 0x7fa5a8143e80 2022-06-17T12:46:51.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5a5ffb700 1 -- 172.21.15.36:0/2582219834 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa598013d20 con 0x7fa5a8143e80 2022-06-17T12:46:51.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.040+0000 7fa5aea9b700 1 -- 172.21.15.36:0/2582219834 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a8146710 con 0x7fa5a8143e80 2022-06-17T12:46:51.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.044+0000 7fa5a5ffb700 1 -- 172.21.15.36:0/2582219834 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fa598025030 con 0x7fa5a8143e80 2022-06-17T12:46:51.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.044+0000 7fa5aea9b700 1 -- 172.21.15.36:0/2582219834 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a80ae330 con 0x7fa5a8143e80 2022-06-17T12:46:51.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.044+0000 7fa5a5ffb700 1 --2- 172.21.15.36:0/2582219834 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa588053ad0 0x7fa588055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:51.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.044+0000 7fa5ac837700 1 --2- 172.21.15.36:0/2582219834 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa588053ad0 0x7fa588055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:51.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.044+0000 7fa5a5ffb700 1 -- 172.21.15.36:0/2582219834 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6315+0+0 (secure 0 0 0) 0x7fa59806d070 con 0x7fa5a8143e80 2022-06-17T12:46:51.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.044+0000 7fa5ac837700 1 --2- 172.21.15.36:0/2582219834 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa588053ad0 0x7fa588055f90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fa59c00b000 tx=0x7fa59c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:51.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.052+0000 7fa5a5ffb700 1 -- 172.21.15.36:0/2582219834 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa598043dd0 con 0x7fa5a8143e80 2022-06-17T12:46:51.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.180+0000 7fa5aea9b700 1 -- 172.21.15.36:0/2582219834 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fa5a8147aa0 con 0x7fa5a8143e80 2022-06-17T12:46:51.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.864+0000 7fa5a5ffb700 1 -- 172.21.15.36:0/2582219834 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) v1 ==== 166+0+0 (secure 0 0 0) 0x7fa598023020 con 0x7fa5a8143e80 2022-06-17T12:46:51.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.872+0000 7fa5aea9b700 1 -- 172.21.15.36:0/2582219834 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa588053ad0 msgr2=0x7fa588055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:51.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.872+0000 7fa5aea9b700 1 --2- 172.21.15.36:0/2582219834 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa588053ad0 0x7fa588055f90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fa59c00b000 tx=0x7fa59c009040).stop 2022-06-17T12:46:51.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.872+0000 7fa5aea9b700 1 -- 172.21.15.36:0/2582219834 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa5a8143e80 msgr2=0x7fa5a80ad610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:51.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.872+0000 7fa5aea9b700 1 --2- 172.21.15.36:0/2582219834 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa5a8143e80 0x7fa5a80ad610 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fa598002f70 tx=0x7fa598014000).stop 2022-06-17T12:46:51.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.872+0000 7fa5aea9b700 1 -- 172.21.15.36:0/2582219834 shutdown_connections 2022-06-17T12:46:51.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.872+0000 7fa5aea9b700 1 --2- 172.21.15.36:0/2582219834 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa588053ad0 0x7fa588055f90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:51.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.872+0000 7fa5aea9b700 1 --2- 172.21.15.36:0/2582219834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5a8145f40 0x7fa5a80adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:51.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.872+0000 7fa5aea9b700 1 --2- 172.21.15.36:0/2582219834 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa5a8143e80 0x7fa5a80ad610 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:51.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.872+0000 7fa5aea9b700 1 --2- 172.21.15.36:0/2582219834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5a8062270 0x7fa5a80b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:51.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.872+0000 7fa5aea9b700 1 -- 172.21.15.36:0/2582219834 >> 172.21.15.36:0/2582219834 conn(0x7fa5a80b9870 msgr2=0x7fa5a805f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:51.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.872+0000 7fa5aea9b700 1 -- 172.21.15.36:0/2582219834 shutdown_connections 2022-06-17T12:46:51.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:51.872+0000 7fa5aea9b700 1 -- 172.21.15.36:0/2582219834 wait complete. 2022-06-17T12:46:51.929 INFO:teuthology.orchestra.run.smithi036.stderr:set pool 4 allow_ec_overwrites to true 2022-06-17T12:46:51.930 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-17T12:46:52.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.064+0000 7fe9ec8dd700 1 Processor -- start 2022-06-17T12:46:52.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.064+0000 7fe9ec8dd700 1 -- start start 2022-06-17T12:46:52.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.068+0000 7fe9ec8dd700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe9e8063730 0x7fe9e80b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:52.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.068+0000 7fe9ec8dd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9e80644e0 0x7fe9e8064940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:52.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.068+0000 7fe9ec8dd700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9e8060390 con 0x7fe9e8064e30 2022-06-17T12:46:52.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.068+0000 7fe9ec8dd700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9e8065280 con 0x7fe9e8063730 2022-06-17T12:46:52.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.068+0000 7fe9ec8dd700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e80b2230 con 0x7fe9e80644e0 2022-06-17T12:46:52.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.068+0000 7fe9e759e700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fe9e8064e30 0x7fe9e80b63a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:48706/0 (socket says 172.21.15.36:48706) 2022-06-17T12:46:52.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.068+0000 7fe9e759e700 1 -- 172.21.15.36:0/2928532418 learned_addr learned my addr 172.21.15.36:0/2928532418 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:46:52.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.068+0000 7fe9dffff700 1 --2- 172.21.15.36:0/2928532418 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9e80644e0 0x7fe9e8064940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:52.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.068+0000 7fe9e6d9d700 1 --2- 172.21.15.36:0/2928532418 >> v2:172.21.15.70:3300/0 conn(0x7fe9e8063730 0x7fe9e80b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:52.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.068+0000 7fe9e659c700 1 -- 172.21.15.36:0/2928532418 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930626537 0 0) 0x7fe9e8060390 con 0x7fe9e8064e30 2022-06-17T12:46:52.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.068+0000 7fe9e659c700 1 -- 172.21.15.36:0/2928532418 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9c8003260 con 0x7fe9e8064e30 2022-06-17T12:46:52.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.068+0000 7fe9dffff700 1 -- 172.21.15.36:0/2928532418 >> v2:172.21.15.70:3300/0 conn(0x7fe9e8063730 msgr2=0x7fe9e80b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:52.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.068+0000 7fe9dffff700 1 --2- 172.21.15.36:0/2928532418 >> v2:172.21.15.70:3300/0 conn(0x7fe9e8063730 0x7fe9e80b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.068+0000 7fe9dffff700 1 -- 172.21.15.36:0/2928532418 >> v1:172.21.15.36:6789/0 conn(0x7fe9e8064e30 legacy=0x7fe9e80b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:52.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.068+0000 7fe9dffff700 1 -- 172.21.15.36:0/2928532418 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e8060130 con 0x7fe9e80644e0 2022-06-17T12:46:52.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.068+0000 7fe9dffff700 1 --2- 172.21.15.36:0/2928532418 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9e80644e0 0x7fe9e8064940 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fe9d0002f70 tx=0x7fe9d0006df0).ready entity=mon.2 client_cookie=9ecda288a19f850a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:52.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.068+0000 7fe9e659c700 1 -- 172.21.15.36:0/2928532418 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9d000d4a0 con 0x7fe9e80644e0 2022-06-17T12:46:52.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.068+0000 7fe9e659c700 1 -- 172.21.15.36:0/2928532418 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d0000e90 con 0x7fe9e80644e0 2022-06-17T12:46:52.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.068+0000 7fe9e659c700 1 -- 172.21.15.36:0/2928532418 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9d000d7f0 con 0x7fe9e80644e0 2022-06-17T12:46:52.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.072+0000 7fe9ec8dd700 1 -- 172.21.15.36:0/2928532418 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9e80644e0 msgr2=0x7fe9e8064940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:52.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.072+0000 7fe9ec8dd700 1 --2- 172.21.15.36:0/2928532418 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9e80644e0 0x7fe9e8064940 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fe9d0002f70 tx=0x7fe9d0006df0).stop 2022-06-17T12:46:52.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.072+0000 7fe9ec8dd700 1 -- 172.21.15.36:0/2928532418 shutdown_connections 2022-06-17T12:46:52.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.072+0000 7fe9ec8dd700 1 --2- 172.21.15.36:0/2928532418 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9e80644e0 0x7fe9e8064940 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.072+0000 7fe9ec8dd700 1 --2- 172.21.15.36:0/2928532418 >> v2:172.21.15.70:3300/0 conn(0x7fe9e8063730 0x7fe9e80b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.072+0000 7fe9ec8dd700 1 -- 172.21.15.36:0/2928532418 >> 172.21.15.36:0/2928532418 conn(0x7fe9e80614b0 msgr2=0x7fe9e80618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:52.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.072+0000 7fe9ec8dd700 1 -- 172.21.15.36:0/2928532418 shutdown_connections 2022-06-17T12:46:52.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.072+0000 7fe9ec8dd700 1 -- 172.21.15.36:0/2928532418 wait complete. 2022-06-17T12:46:52.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.072+0000 7fe9ec8dd700 1 Processor -- start 2022-06-17T12:46:52.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.072+0000 7fe9ec8dd700 1 -- start start 2022-06-17T12:46:52.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.072+0000 7fe9ec8dd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9e8063730 0x7fe9e813fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:52.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.072+0000 7fe9ec8dd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe9e80644e0 0x7fe9e81522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:52.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.072+0000 7fe9ec8dd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9e8064e30 0x7fe9e81527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:52.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.072+0000 7fe9ec8dd700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e8144bc0 con 0x7fe9e80644e0 2022-06-17T12:46:52.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.072+0000 7fe9ec8dd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e8065ca0 con 0x7fe9e8063730 2022-06-17T12:46:52.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.072+0000 7fe9ec8dd700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e8144df0 con 0x7fe9e8064e30 2022-06-17T12:46:52.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.072+0000 7fe9dffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe9e80644e0 0x7fe9e81522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:52.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.076+0000 7fe9dffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe9e80644e0 0x7fe9e81522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59412/0 (socket says 172.21.15.36:59412) 2022-06-17T12:46:52.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.076+0000 7fe9dffff700 1 -- 172.21.15.36:0/4187230582 learned_addr learned my addr 172.21.15.36:0/4187230582 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:52.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.076+0000 7fe9e6d9d700 1 --2- 172.21.15.36:0/4187230582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9e8063730 0x7fe9e813fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:52.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.076+0000 7fe9e759e700 1 --2- 172.21.15.36:0/4187230582 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9e8064e30 0x7fe9e81527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:52.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.076+0000 7fe9dffff700 1 -- 172.21.15.36:0/4187230582 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9e8064e30 msgr2=0x7fe9e81527a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:52.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.076+0000 7fe9dffff700 1 --2- 172.21.15.36:0/4187230582 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9e8064e30 0x7fe9e81527a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.076+0000 7fe9dffff700 1 -- 172.21.15.36:0/4187230582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9e8063730 msgr2=0x7fe9e813fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:52.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.076+0000 7fe9dffff700 1 --2- 172.21.15.36:0/4187230582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9e8063730 0x7fe9e813fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.076+0000 7fe9dffff700 1 -- 172.21.15.36:0/4187230582 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d000b180 con 0x7fe9e80644e0 2022-06-17T12:46:52.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.076+0000 7fe9dffff700 1 --2- 172.21.15.36:0/4187230582 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe9e80644e0 0x7fe9e81522b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fe9e8065e30 tx=0x7fe9d0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:52.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.076+0000 7fe9e4d99700 1 -- 172.21.15.36:0/4187230582 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9d0016070 con 0x7fe9e80644e0 2022-06-17T12:46:52.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.076+0000 7fe9e4d99700 1 -- 172.21.15.36:0/4187230582 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d0013a00 con 0x7fe9e80644e0 2022-06-17T12:46:52.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.076+0000 7fe9e4d99700 1 -- 172.21.15.36:0/4187230582 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9d0013d60 con 0x7fe9e80644e0 2022-06-17T12:46:52.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.076+0000 7fe9ec8dd700 1 -- 172.21.15.36:0/4187230582 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e8065a00 con 0x7fe9e80644e0 2022-06-17T12:46:52.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.076+0000 7fe9ec8dd700 1 -- 172.21.15.36:0/4187230582 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e8152c90 con 0x7fe9e80644e0 2022-06-17T12:46:52.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.076+0000 7fe9ec8dd700 1 -- 172.21.15.36:0/4187230582 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e80b2e10 con 0x7fe9e80644e0 2022-06-17T12:46:52.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.076+0000 7fe9e4d99700 1 -- 172.21.15.36:0/4187230582 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe9d0025030 con 0x7fe9e80644e0 2022-06-17T12:46:52.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.080+0000 7fe9e4d99700 1 --2- 172.21.15.36:0/4187230582 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe9cc053ad0 0x7fe9cc055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:52.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.080+0000 7fe9e4d99700 1 -- 172.21.15.36:0/4187230582 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6315+0+0 (secure 0 0 0) 0x7fe9d006cfa0 con 0x7fe9e80644e0 2022-06-17T12:46:52.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.080+0000 7fe9e6d9d700 1 --2- 172.21.15.36:0/4187230582 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe9cc053ad0 0x7fe9cc055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:52.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.080+0000 7fe9e6d9d700 1 --2- 172.21.15.36:0/4187230582 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe9cc053ad0 0x7fe9cc055f90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fe9d800b000 tx=0x7fe9d8006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:52.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.084+0000 7fe9e4d99700 1 -- 172.21.15.36:0/4187230582 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe9d0043e10 con 0x7fe9e80644e0 2022-06-17T12:46:52.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.248+0000 7fe9ec8dd700 1 -- 172.21.15.36:0/4187230582 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fe9e8057410 con 0x7fe9e80644e0 2022-06-17T12:46:52.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.256+0000 7fe9e4d99700 1 -- 172.21.15.36:0/4187230582 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 187+0+0 (secure 0 0 0) 0x7fe9d0023070 con 0x7fe9e80644e0 2022-06-17T12:46:52.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.260+0000 7fe9ec8dd700 1 -- 172.21.15.36:0/4187230582 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe9cc053ad0 msgr2=0x7fe9cc055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:52.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.260+0000 7fe9ec8dd700 1 --2- 172.21.15.36:0/4187230582 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe9cc053ad0 0x7fe9cc055f90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fe9d800b000 tx=0x7fe9d8006df0).stop 2022-06-17T12:46:52.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.260+0000 7fe9ec8dd700 1 -- 172.21.15.36:0/4187230582 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe9e80644e0 msgr2=0x7fe9e81522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:52.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.260+0000 7fe9ec8dd700 1 --2- 172.21.15.36:0/4187230582 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe9e80644e0 0x7fe9e81522b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fe9e8065e30 tx=0x7fe9d0014000).stop 2022-06-17T12:46:52.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.260+0000 7fe9ec8dd700 1 -- 172.21.15.36:0/4187230582 shutdown_connections 2022-06-17T12:46:52.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.260+0000 7fe9ec8dd700 1 --2- 172.21.15.36:0/4187230582 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe9cc053ad0 0x7fe9cc055f90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.260+0000 7fe9ec8dd700 1 --2- 172.21.15.36:0/4187230582 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9e8064e30 0x7fe9e81527a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.260+0000 7fe9ec8dd700 1 --2- 172.21.15.36:0/4187230582 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe9e80644e0 0x7fe9e81522b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.260+0000 7fe9ec8dd700 1 --2- 172.21.15.36:0/4187230582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9e8063730 0x7fe9e813fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.260+0000 7fe9ec8dd700 1 -- 172.21.15.36:0/4187230582 >> 172.21.15.36:0/4187230582 conn(0x7fe9e80614b0 msgr2=0x7fe9e805bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:52.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.260+0000 7fe9ec8dd700 1 -- 172.21.15.36:0/4187230582 shutdown_connections 2022-06-17T12:46:52.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.260+0000 7fe9ec8dd700 1 -- 172.21.15.36:0/4187230582 wait complete. 2022-06-17T12:46:52.258 INFO:teuthology.orchestra.run.smithi036.stderr:new fs with metadata pool 3 and data pool 4 2022-06-17T12:46:52.292 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:46:52.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.412+0000 7f880c4fe700 1 Processor -- start 2022-06-17T12:46:52.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.412+0000 7f880c4fe700 1 -- start start 2022-06-17T12:46:52.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.412+0000 7f880c4fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f880405fab0 0x7f8804143280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:52.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.412+0000 7f880c4fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8804140480 0x7f88040afb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:52.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.412+0000 7f880c4fe700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f880405fe90 con 0x7f88040aea60 2022-06-17T12:46:52.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.412+0000 7f880c4fe700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8804143770 con 0x7f8804140480 2022-06-17T12:46:52.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.412+0000 7f880c4fe700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8804143990 con 0x7f880405fab0 2022-06-17T12:46:52.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.416+0000 7f880aa9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8804140480 0x7f88040afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:52.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.416+0000 7f880a29a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f880405fab0 0x7f8804143280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:52.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.416+0000 7f880aa9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8804140480 0x7f88040afb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59422/0 (socket says 172.21.15.36:59422) 2022-06-17T12:46:52.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.416+0000 7f880aa9b700 1 -- 172.21.15.36:0/209132929 learned_addr learned my addr 172.21.15.36:0/209132929 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:52.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.416+0000 7f880aa9b700 1 -- 172.21.15.36:0/209132929 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f880405fab0 msgr2=0x7f8804143280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:52.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.416+0000 7f880aa9b700 1 --2- 172.21.15.36:0/209132929 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f880405fab0 0x7f8804143280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.416+0000 7f880aa9b700 1 -- 172.21.15.36:0/209132929 >> v1:172.21.15.70:6789/0 conn(0x7f88040aea60 legacy=0x7f880413fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:52.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.416+0000 7f880aa9b700 1 -- 172.21.15.36:0/209132929 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8804143c30 con 0x7f8804140480 2022-06-17T12:46:52.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.416+0000 7f880aa9b700 1 --2- 172.21.15.36:0/209132929 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8804140480 0x7f88040afb50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f87fc0032f0 tx=0x7f87fc009290).ready entity=mon.0 client_cookie=23f762e9fd238e6 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:52.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.416+0000 7f8809298700 1 -- 172.21.15.36:0/209132929 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87fc00b8a0 con 0x7f8804140480 2022-06-17T12:46:52.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.416+0000 7f8809298700 1 -- 172.21.15.36:0/209132929 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87fc00ba00 con 0x7f8804140480 2022-06-17T12:46:52.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.416+0000 7f8809298700 1 -- 172.21.15.36:0/209132929 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87fc00bb70 con 0x7f8804140480 2022-06-17T12:46:52.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.416+0000 7f880c4fe700 1 -- 172.21.15.36:0/209132929 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8804140480 msgr2=0x7f88040afb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:52.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.416+0000 7f880c4fe700 1 --2- 172.21.15.36:0/209132929 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8804140480 0x7f88040afb50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f87fc0032f0 tx=0x7f87fc009290).stop 2022-06-17T12:46:52.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.416+0000 7f880c4fe700 1 -- 172.21.15.36:0/209132929 shutdown_connections 2022-06-17T12:46:52.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.416+0000 7f880c4fe700 1 --2- 172.21.15.36:0/209132929 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8804140480 0x7f88040afb50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.416+0000 7f880c4fe700 1 --2- 172.21.15.36:0/209132929 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f880405fab0 0x7f8804143280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.416+0000 7f880c4fe700 1 -- 172.21.15.36:0/209132929 >> 172.21.15.36:0/209132929 conn(0x7f880405c550 msgr2=0x7f880405c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:52.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.416+0000 7f880c4fe700 1 -- 172.21.15.36:0/209132929 shutdown_connections 2022-06-17T12:46:52.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.416+0000 7f880c4fe700 1 -- 172.21.15.36:0/209132929 wait complete. 2022-06-17T12:46:52.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.416+0000 7f880c4fe700 1 Processor -- start 2022-06-17T12:46:52.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.416+0000 7f880c4fe700 1 -- start start 2022-06-17T12:46:52.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.420+0000 7f880c4fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f880405fab0 0x7f8804142a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:52.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.420+0000 7f880c4fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88040aea60 0x7f8804152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:52.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.420+0000 7f880c4fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8804140480 0x7f88041526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:52.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.420+0000 7f880c4fe700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8804129570 con 0x7f88040aea60 2022-06-17T12:46:52.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.420+0000 7f880c4fe700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880412a2f0 con 0x7f8804140480 2022-06-17T12:46:52.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.420+0000 7f880c4fe700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f880412a590 con 0x7f880405fab0 2022-06-17T12:46:52.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.420+0000 7f8809a99700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88040aea60 0x7f8804152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:52.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.420+0000 7f8809a99700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88040aea60 0x7f8804152200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59426/0 (socket says 172.21.15.36:59426) 2022-06-17T12:46:52.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.420+0000 7f8809a99700 1 -- 172.21.15.36:0/3527419676 learned_addr learned my addr 172.21.15.36:0/3527419676 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:52.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.420+0000 7f880a29a700 1 --2- 172.21.15.36:0/3527419676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f880405fab0 0x7f8804142a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:52.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.420+0000 7f880aa9b700 1 --2- 172.21.15.36:0/3527419676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8804140480 0x7f88041526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:52.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.420+0000 7f8809a99700 1 -- 172.21.15.36:0/3527419676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f880405fab0 msgr2=0x7f8804142a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:52.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.420+0000 7f8809a99700 1 --2- 172.21.15.36:0/3527419676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f880405fab0 0x7f8804142a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.420+0000 7f8809a99700 1 -- 172.21.15.36:0/3527419676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8804140480 msgr2=0x7f88041526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:52.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.420+0000 7f8809a99700 1 --2- 172.21.15.36:0/3527419676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8804140480 0x7f88041526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.420+0000 7f8809a99700 1 -- 172.21.15.36:0/3527419676 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87fc009530 con 0x7f88040aea60 2022-06-17T12:46:52.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.420+0000 7f8809a99700 1 --2- 172.21.15.36:0/3527419676 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88040aea60 0x7f8804152200 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f880406a4e0 tx=0x7f87f4007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:52.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.420+0000 7f87f37fe700 1 -- 172.21.15.36:0/3527419676 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87f4012400 con 0x7f88040aea60 2022-06-17T12:46:52.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.420+0000 7f87f37fe700 1 -- 172.21.15.36:0/3527419676 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f4012560 con 0x7f88040aea60 2022-06-17T12:46:52.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.420+0000 7f880c4fe700 1 -- 172.21.15.36:0/3527419676 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f880412a830 con 0x7f88040aea60 2022-06-17T12:46:52.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.420+0000 7f87f37fe700 1 -- 172.21.15.36:0/3527419676 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87f40128c0 con 0x7f88040aea60 2022-06-17T12:46:52.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.420+0000 7f880c4fe700 1 -- 172.21.15.36:0/3527419676 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880412acd0 con 0x7f88040aea60 2022-06-17T12:46:52.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.420+0000 7f880c4fe700 1 -- 172.21.15.36:0/3527419676 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8804141f40 con 0x7f88040aea60 2022-06-17T12:46:52.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.424+0000 7f87f37fe700 1 -- 172.21.15.36:0/3527419676 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f87f4019030 con 0x7f88040aea60 2022-06-17T12:46:52.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.424+0000 7f87f37fe700 1 --2- 172.21.15.36:0/3527419676 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f87e4053ad0 0x7f87e4055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:52.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.424+0000 7f87f37fe700 1 -- 172.21.15.36:0/3527419676 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f87f406b3c0 con 0x7f88040aea60 2022-06-17T12:46:52.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.424+0000 7f880a29a700 1 --2- 172.21.15.36:0/3527419676 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f87e4053ad0 0x7f87e4055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:52.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.424+0000 7f880a29a700 1 --2- 172.21.15.36:0/3527419676 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f87e4053ad0 0x7f87e4055f90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f880000b000 tx=0x7f8800006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:52.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.428+0000 7f87f37fe700 1 -- 172.21.15.36:0/3527419676 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8804141f40 con 0x7f88040aea60 2022-06-17T12:46:52.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.560+0000 7f880c4fe700 1 -- 172.21.15.36:0/3527419676 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8804141f40 con 0x7f88040aea60 2022-06-17T12:46:52.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.564+0000 7f87f37fe700 1 -- 172.21.15.36:0/3527419676 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16312 (secure 0 0 0) 0x7f8804141f40 con 0x7f88040aea60 2022-06-17T12:46:52.582 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:46:52.583 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":25,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:46:52.253433+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:46:47.465328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-17T12:46:49.893338+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:46:52.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.564+0000 7f880c4fe700 1 -- 172.21.15.36:0/3527419676 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f87e4053ad0 msgr2=0x7f87e4055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:52.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.564+0000 7f880c4fe700 1 --2- 172.21.15.36:0/3527419676 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f87e4053ad0 0x7f87e4055f90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f880000b000 tx=0x7f8800006df0).stop 2022-06-17T12:46:52.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.564+0000 7f880c4fe700 1 -- 172.21.15.36:0/3527419676 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88040aea60 msgr2=0x7f8804152200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:52.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.564+0000 7f880c4fe700 1 --2- 172.21.15.36:0/3527419676 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88040aea60 0x7f8804152200 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f880406a4e0 tx=0x7f87f4007c50).stop 2022-06-17T12:46:52.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.564+0000 7f880c4fe700 1 -- 172.21.15.36:0/3527419676 shutdown_connections 2022-06-17T12:46:52.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.564+0000 7f880c4fe700 1 --2- 172.21.15.36:0/3527419676 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f87e4053ad0 0x7f87e4055f90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.564+0000 7f880c4fe700 1 --2- 172.21.15.36:0/3527419676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8804140480 0x7f88041526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.564+0000 7f880c4fe700 1 --2- 172.21.15.36:0/3527419676 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88040aea60 0x7f8804152200 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.564+0000 7f880c4fe700 1 --2- 172.21.15.36:0/3527419676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f880405fab0 0x7f8804142a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.564+0000 7f880c4fe700 1 -- 172.21.15.36:0/3527419676 >> 172.21.15.36:0/3527419676 conn(0x7f880405c550 msgr2=0x7f88040b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:52.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.568+0000 7f880c4fe700 1 -- 172.21.15.36:0/3527419676 shutdown_connections 2022-06-17T12:46:52.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.568+0000 7f880c4fe700 1 -- 172.21.15.36:0/3527419676 wait complete. 2022-06-17T12:46:52.602 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:46:52.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfef219700 1 Processor -- start 2022-06-17T12:46:52.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfef219700 1 -- start start 2022-06-17T12:46:52.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfef219700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbfe8062770 0x7fbfe8062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:52.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfef219700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe80b0ba0 con 0x7fbfe81444f0 2022-06-17T12:46:52.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfef219700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe8147e90 con 0x7fbfe80ad330 2022-06-17T12:46:52.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfef219700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe80ae820 con 0x7fbfe8062770 2022-06-17T12:46:52.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfe7fff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fbfe81444f0 0x7fbfe8144960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:48732/0 (socket says 172.21.15.36:48732) 2022-06-17T12:46:52.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfe7fff700 1 -- 172.21.15.36:0/1115934725 learned_addr learned my addr 172.21.15.36:0/1115934725 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:46:52.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfecfb5700 1 --2- 172.21.15.36:0/1115934725 >> v2:172.21.15.70:3300/0 conn(0x7fbfe8062770 0x7fbfe8062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:52.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfe77fe700 1 -- 172.21.15.36:0/1115934725 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1008559682 0 0) 0x7fbfe80b0ba0 con 0x7fbfe81444f0 2022-06-17T12:46:52.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfe77fe700 1 -- 172.21.15.36:0/1115934725 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfc8003260 con 0x7fbfe81444f0 2022-06-17T12:46:52.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfecfb5700 1 -- 172.21.15.36:0/1115934725 >> v1:172.21.15.70:6789/0 conn(0x7fbfe80ad330 legacy=0x7fbfe80ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:52.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfecfb5700 1 -- 172.21.15.36:0/1115934725 >> v1:172.21.15.36:6789/0 conn(0x7fbfe81444f0 legacy=0x7fbfe8144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:52.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfecfb5700 1 -- 172.21.15.36:0/1115934725 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe80b0dd0 con 0x7fbfe8062770 2022-06-17T12:46:52.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfecfb5700 1 --2- 172.21.15.36:0/1115934725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfe8062770 0x7fbfe8062b50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fbfd0002f70 tx=0x7fbfd0006df0).ready entity=mon.1 client_cookie=4dc707fcd4a01c83 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:52.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfe77fe700 1 -- 172.21.15.36:0/1115934725 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfd000d610 con 0x7fbfe8062770 2022-06-17T12:46:52.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfe77fe700 1 -- 172.21.15.36:0/1115934725 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd0000e90 con 0x7fbfe8062770 2022-06-17T12:46:52.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfe77fe700 1 -- 172.21.15.36:0/1115934725 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfd000d960 con 0x7fbfe8062770 2022-06-17T12:46:52.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfef219700 1 -- 172.21.15.36:0/1115934725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfe8062770 msgr2=0x7fbfe8062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:52.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfef219700 1 --2- 172.21.15.36:0/1115934725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfe8062770 0x7fbfe8062b50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fbfd0002f70 tx=0x7fbfd0006df0).stop 2022-06-17T12:46:52.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfef219700 1 -- 172.21.15.36:0/1115934725 shutdown_connections 2022-06-17T12:46:52.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfef219700 1 --2- 172.21.15.36:0/1115934725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfe8062770 0x7fbfe8062b50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfef219700 1 -- 172.21.15.36:0/1115934725 >> 172.21.15.36:0/1115934725 conn(0x7fbfe80b53e0 msgr2=0x7fbfe80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:52.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfef219700 1 -- 172.21.15.36:0/1115934725 shutdown_connections 2022-06-17T12:46:52.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.748+0000 7fbfef219700 1 -- 172.21.15.36:0/1115934725 wait complete. 2022-06-17T12:46:52.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfef219700 1 Processor -- start 2022-06-17T12:46:52.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfef219700 1 -- start start 2022-06-17T12:46:52.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfef219700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfe8062770 0x7fbfe805fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:52.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfef219700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfe80ad330 0x7fbfe815a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:52.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfef219700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbfe81444f0 0x7fbfe8156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:52.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfef219700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe815aac0 con 0x7fbfe80ad330 2022-06-17T12:46:52.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfef219700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe8159bf0 con 0x7fbfe8062770 2022-06-17T12:46:52.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfef219700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe8159e90 con 0x7fbfe81444f0 2022-06-17T12:46:52.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfe7fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfe80ad330 0x7fbfe815a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:52.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfe7fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfe80ad330 0x7fbfe815a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59440/0 (socket says 172.21.15.36:59440) 2022-06-17T12:46:52.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfe7fff700 1 -- 172.21.15.36:0/1929769329 learned_addr learned my addr 172.21.15.36:0/1929769329 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:52.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfecfb5700 1 --2- 172.21.15.36:0/1929769329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfe8062770 0x7fbfe805fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:52.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfed7b6700 1 --2- 172.21.15.36:0/1929769329 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbfe81444f0 0x7fbfe8156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:52.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfe7fff700 1 -- 172.21.15.36:0/1929769329 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbfe81444f0 msgr2=0x7fbfe8156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:52.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfe7fff700 1 --2- 172.21.15.36:0/1929769329 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbfe81444f0 0x7fbfe8156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfe7fff700 1 -- 172.21.15.36:0/1929769329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfe8062770 msgr2=0x7fbfe805fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:52.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfe7fff700 1 --2- 172.21.15.36:0/1929769329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfe8062770 0x7fbfe805fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfe7fff700 1 -- 172.21.15.36:0/1929769329 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd000b180 con 0x7fbfe80ad330 2022-06-17T12:46:52.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfecfb5700 1 --2- 172.21.15.36:0/1929769329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfe8062770 0x7fbfe805fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:46:52.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfe7fff700 1 --2- 172.21.15.36:0/1929769329 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfe80ad330 0x7fbfe815a5d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fbfd8001490 tx=0x7fbfd8007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:52.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfe5ffb700 1 -- 172.21.15.36:0/1929769329 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfd8005ba0 con 0x7fbfe80ad330 2022-06-17T12:46:52.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfe5ffb700 1 -- 172.21.15.36:0/1929769329 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd8005d00 con 0x7fbfe80ad330 2022-06-17T12:46:52.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfef219700 1 -- 172.21.15.36:0/1929769329 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe815a130 con 0x7fbfe80ad330 2022-06-17T12:46:52.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfe5ffb700 1 -- 172.21.15.36:0/1929769329 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfd80072c0 con 0x7fbfe80ad330 2022-06-17T12:46:52.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.752+0000 7fbfef219700 1 -- 172.21.15.36:0/1929769329 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe8063090 con 0x7fbfe80ad330 2022-06-17T12:46:52.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.756+0000 7fbfef219700 1 -- 172.21.15.36:0/1929769329 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe80b1c20 con 0x7fbfe80ad330 2022-06-17T12:46:52.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.756+0000 7fbfe5ffb700 1 -- 172.21.15.36:0/1929769329 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fbfd8010070 con 0x7fbfe80ad330 2022-06-17T12:46:52.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.756+0000 7fbfe5ffb700 1 --2- 172.21.15.36:0/1929769329 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbfcc053af0 0x7fbfcc055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:52.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.756+0000 7fbfecfb5700 1 --2- 172.21.15.36:0/1929769329 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbfcc053af0 0x7fbfcc055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:52.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.756+0000 7fbfe5ffb700 1 -- 172.21.15.36:0/1929769329 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fbfd806bb30 con 0x7fbfe80ad330 2022-06-17T12:46:52.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.756+0000 7fbfecfb5700 1 --2- 172.21.15.36:0/1929769329 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbfcc053af0 0x7fbfcc055fb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fbfd0018000 tx=0x7fbfd0014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:52.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.759+0000 7fbfe5ffb700 1 -- 172.21.15.36:0/1929769329 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbfd8042820 con 0x7fbfe80ad330 2022-06-17T12:46:52.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.887+0000 7fbfef219700 1 -- 172.21.15.36:0/1929769329 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbfe8057410 con 0x7fbfe80ad330 2022-06-17T12:46:52.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.887+0000 7fbfe5ffb700 1 -- 172.21.15.36:0/1929769329 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16312 (secure 0 0 0) 0x7fbfd8045e40 con 0x7fbfe80ad330 2022-06-17T12:46:52.910 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:46:52.911 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":25,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:46:52.253433+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:46:47.465328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-17T12:46:49.893338+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:46:52.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.891+0000 7fbfef219700 1 -- 172.21.15.36:0/1929769329 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbfcc053af0 msgr2=0x7fbfcc055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:52.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.891+0000 7fbfef219700 1 --2- 172.21.15.36:0/1929769329 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbfcc053af0 0x7fbfcc055fb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fbfd0018000 tx=0x7fbfd0014000).stop 2022-06-17T12:46:52.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.891+0000 7fbfef219700 1 -- 172.21.15.36:0/1929769329 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfe80ad330 msgr2=0x7fbfe815a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:52.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.891+0000 7fbfef219700 1 --2- 172.21.15.36:0/1929769329 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfe80ad330 0x7fbfe815a5d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fbfd8001490 tx=0x7fbfd8007e50).stop 2022-06-17T12:46:52.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.891+0000 7fbfef219700 1 -- 172.21.15.36:0/1929769329 shutdown_connections 2022-06-17T12:46:52.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.891+0000 7fbfef219700 1 --2- 172.21.15.36:0/1929769329 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbfcc053af0 0x7fbfcc055fb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.891+0000 7fbfef219700 1 --2- 172.21.15.36:0/1929769329 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbfe81444f0 0x7fbfe8156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.891+0000 7fbfef219700 1 --2- 172.21.15.36:0/1929769329 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfe80ad330 0x7fbfe815a5d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.891+0000 7fbfef219700 1 --2- 172.21.15.36:0/1929769329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfe8062770 0x7fbfe805fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:52.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.891+0000 7fbfef219700 1 -- 172.21.15.36:0/1929769329 >> 172.21.15.36:0/1929769329 conn(0x7fbfe80b53e0 msgr2=0x7fbfe80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:52.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.891+0000 7fbfef219700 1 -- 172.21.15.36:0/1929769329 shutdown_connections 2022-06-17T12:46:52.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:52.891+0000 7fbfef219700 1 -- 172.21.15.36:0/1929769329 wait complete. 2022-06-17T12:46:52.933 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-17T12:46:53.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.087+0000 7fab8fb17700 1 Processor -- start 2022-06-17T12:46:53.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.087+0000 7fab8fb17700 1 -- start start 2022-06-17T12:46:53.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8fb17700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab88062270 0x7fab88143d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:53.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8fb17700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fab880b4190 0x7fab881471d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:53.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8fb17700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab88143130 con 0x7fab880b48b0 2022-06-17T12:46:53.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8fb17700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab880b6080 con 0x7fab880b4190 2022-06-17T12:46:53.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8fb17700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab88062650 con 0x7fab88062270 2022-06-17T12:46:53.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8d0b2700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fab880b4190 0x7fab881471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:53.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8d0b2700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fab880b4190 0x7fab881471d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59450/0 (socket says 172.21.15.36:59450) 2022-06-17T12:46:53.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8e0b4700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fab880b48b0 0x7fab880b4d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:48746/0 (socket says 172.21.15.36:48746) 2022-06-17T12:46:53.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8d0b2700 1 -- 172.21.15.36:0/3451529473 learned_addr learned my addr 172.21.15.36:0/3451529473 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:53.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8d8b3700 1 --2- 172.21.15.36:0/3451529473 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab88062270 0x7fab88143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:53.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8d0b2700 1 -- 172.21.15.36:0/3451529473 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab88062270 msgr2=0x7fab88143d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:53.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8d0b2700 1 --2- 172.21.15.36:0/3451529473 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab88062270 0x7fab88143d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:53.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8d0b2700 1 -- 172.21.15.36:0/3451529473 >> v1:172.21.15.36:6789/0 conn(0x7fab880b48b0 legacy=0x7fab880b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:53.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8d0b2700 1 -- 172.21.15.36:0/3451529473 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab88143360 con 0x7fab880b4190 2022-06-17T12:46:53.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8c8b1700 1 -- 172.21.15.36:0/3451529473 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3686214042 0 0) 0x7fab88143130 con 0x7fab880b48b0 2022-06-17T12:46:53.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8d8b3700 1 --2- 172.21.15.36:0/3451529473 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab88062270 0x7fab88143d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:46:53.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8d0b2700 1 --2- 172.21.15.36:0/3451529473 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fab880b4190 0x7fab881471d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fab84002f70 tx=0x7fab84006eb0).ready entity=mon.0 client_cookie=8c4918f274e7a90a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:53.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8c8b1700 1 -- 172.21.15.36:0/3451529473 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab8400d5c0 con 0x7fab880b4190 2022-06-17T12:46:53.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8c8b1700 1 -- 172.21.15.36:0/3451529473 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab84000e90 con 0x7fab880b4190 2022-06-17T12:46:53.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8c8b1700 1 -- 172.21.15.36:0/3451529473 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab8400d910 con 0x7fab880b4190 2022-06-17T12:46:53.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8fb17700 1 -- 172.21.15.36:0/3451529473 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fab880b4190 msgr2=0x7fab881471d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:53.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8fb17700 1 --2- 172.21.15.36:0/3451529473 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fab880b4190 0x7fab881471d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fab84002f70 tx=0x7fab84006eb0).stop 2022-06-17T12:46:53.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8fb17700 1 -- 172.21.15.36:0/3451529473 shutdown_connections 2022-06-17T12:46:53.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8fb17700 1 --2- 172.21.15.36:0/3451529473 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fab880b4190 0x7fab881471d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:53.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8fb17700 1 --2- 172.21.15.36:0/3451529473 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab88062270 0x7fab88143d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:53.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8fb17700 1 -- 172.21.15.36:0/3451529473 >> 172.21.15.36:0/3451529473 conn(0x7fab880b97f0 msgr2=0x7fab880b9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:53.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8fb17700 1 -- 172.21.15.36:0/3451529473 shutdown_connections 2022-06-17T12:46:53.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.091+0000 7fab8fb17700 1 -- 172.21.15.36:0/3451529473 wait complete. 2022-06-17T12:46:53.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab8fb17700 1 Processor -- start 2022-06-17T12:46:53.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab8fb17700 1 -- start start 2022-06-17T12:46:53.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab8fb17700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fab88062270 0x7fab880b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:53.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab8fb17700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab880b4190 0x7fab88146870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:53.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab8fb17700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab880b48b0 0x7fab8812f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:53.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab8fb17700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab880b1c50 con 0x7fab88062270 2022-06-17T12:46:53.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab8fb17700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8812efb0 con 0x7fab880b48b0 2022-06-17T12:46:53.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab8fb17700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab880b2530 con 0x7fab880b4190 2022-06-17T12:46:53.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab8d8b3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fab88062270 0x7fab880b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:53.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab8d8b3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fab88062270 0x7fab880b0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59452/0 (socket says 172.21.15.36:59452) 2022-06-17T12:46:53.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab8d8b3700 1 -- 172.21.15.36:0/2148788454 learned_addr learned my addr 172.21.15.36:0/2148788454 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:53.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab8d0b2700 1 --2- 172.21.15.36:0/2148788454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab880b4190 0x7fab88146870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:53.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab8e0b4700 1 --2- 172.21.15.36:0/2148788454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab880b48b0 0x7fab8812f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:53.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab8d8b3700 1 -- 172.21.15.36:0/2148788454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab880b4190 msgr2=0x7fab88146870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:53.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab8d8b3700 1 --2- 172.21.15.36:0/2148788454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab880b4190 0x7fab88146870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:53.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab8d8b3700 1 -- 172.21.15.36:0/2148788454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab880b48b0 msgr2=0x7fab8812f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:53.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab8d8b3700 1 --2- 172.21.15.36:0/2148788454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab880b48b0 0x7fab8812f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:53.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab8d8b3700 1 -- 172.21.15.36:0/2148788454 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8400b1e0 con 0x7fab88062270 2022-06-17T12:46:53.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab8e0b4700 1 --2- 172.21.15.36:0/2148788454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab880b48b0 0x7fab8812f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:46:53.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab8d8b3700 1 --2- 172.21.15.36:0/2148788454 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fab88062270 0x7fab880b0ba0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fab78006940 tx=0x7fab78008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:53.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab76ffd700 1 -- 172.21.15.36:0/2148788454 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab78011850 con 0x7fab88062270 2022-06-17T12:46:53.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab76ffd700 1 -- 172.21.15.36:0/2148788454 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab780119b0 con 0x7fab88062270 2022-06-17T12:46:53.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab8fb17700 1 -- 172.21.15.36:0/2148788454 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab880b27d0 con 0x7fab88062270 2022-06-17T12:46:53.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab76ffd700 1 -- 172.21.15.36:0/2148788454 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab78011d10 con 0x7fab88062270 2022-06-17T12:46:53.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.095+0000 7fab8fb17700 1 -- 172.21.15.36:0/2148788454 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab8812d8a0 con 0x7fab88062270 2022-06-17T12:46:53.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.099+0000 7fab76ffd700 1 -- 172.21.15.36:0/2148788454 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fab78012420 con 0x7fab88062270 2022-06-17T12:46:53.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.099+0000 7fab8fb17700 1 -- 172.21.15.36:0/2148788454 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab88143600 con 0x7fab88062270 2022-06-17T12:46:53.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.099+0000 7fab76ffd700 1 --2- 172.21.15.36:0/2148788454 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fab680539b0 0x7fab68055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:53.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.099+0000 7fab76ffd700 1 -- 172.21.15.36:0/2148788454 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fab7806bf80 con 0x7fab88062270 2022-06-17T12:46:53.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.099+0000 7fab8d0b2700 1 --2- 172.21.15.36:0/2148788454 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fab680539b0 0x7fab68055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:53.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.099+0000 7fab8d0b2700 1 --2- 172.21.15.36:0/2148788454 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fab680539b0 0x7fab68055e70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fab8401c000 tx=0x7fab84015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:53.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.103+0000 7fab76ffd700 1 -- 172.21.15.36:0/2148788454 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fab78042d20 con 0x7fab88062270 2022-06-17T12:46:53.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:53.271+0000 7fab8fb17700 1 -- 172.21.15.36:0/2148788454 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fab88057410 con 0x7fab88062270 2022-06-17T12:46:54.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.271+0000 7fab76ffd700 1 -- 172.21.15.36:0/2148788454 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7fab78046340 con 0x7fab88062270 2022-06-17T12:46:54.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.271+0000 7fab8fb17700 1 -- 172.21.15.36:0/2148788454 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fab680539b0 msgr2=0x7fab68055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:54.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.271+0000 7fab8fb17700 1 --2- 172.21.15.36:0/2148788454 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fab680539b0 0x7fab68055e70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fab8401c000 tx=0x7fab84015000).stop 2022-06-17T12:46:54.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.271+0000 7fab8fb17700 1 -- 172.21.15.36:0/2148788454 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fab88062270 msgr2=0x7fab880b0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:54.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.271+0000 7fab8fb17700 1 --2- 172.21.15.36:0/2148788454 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fab88062270 0x7fab880b0ba0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fab78006940 tx=0x7fab78008040).stop 2022-06-17T12:46:54.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.275+0000 7fab8fb17700 1 -- 172.21.15.36:0/2148788454 shutdown_connections 2022-06-17T12:46:54.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.275+0000 7fab8fb17700 1 --2- 172.21.15.36:0/2148788454 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fab680539b0 0x7fab68055e70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:54.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.275+0000 7fab8fb17700 1 --2- 172.21.15.36:0/2148788454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab880b48b0 0x7fab8812f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:54.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.275+0000 7fab8fb17700 1 --2- 172.21.15.36:0/2148788454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab880b4190 0x7fab88146870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:54.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.275+0000 7fab8fb17700 1 --2- 172.21.15.36:0/2148788454 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fab88062270 0x7fab880b0ba0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:54.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.275+0000 7fab8fb17700 1 -- 172.21.15.36:0/2148788454 >> 172.21.15.36:0/2148788454 conn(0x7fab880b97f0 msgr2=0x7fab880afd00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:54.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.275+0000 7fab8fb17700 1 -- 172.21.15.36:0/2148788454 shutdown_connections 2022-06-17T12:46:54.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.275+0000 7fab8fb17700 1 -- 172.21.15.36:0/2148788454 wait complete. 2022-06-17T12:46:54.320 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:46:54.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.443+0000 7f7e6789f700 1 Processor -- start 2022-06-17T12:46:54.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.443+0000 7f7e6789f700 1 -- start start 2022-06-17T12:46:54.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e6789f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7e601469c0 0x7f7e60146da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:54.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e6789f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e600b6600 0x7f7e600b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:54.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e6789f700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e600b48e0 con 0x7f7e600b54d0 2022-06-17T12:46:54.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e6789f700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e60147290 con 0x7f7e601469c0 2022-06-17T12:46:54.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e6789f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e600b4b40 con 0x7f7e600b6600 2022-06-17T12:46:54.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e65e3c700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f7e600b54d0 0x7f7e60062080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:48762/0 (socket says 172.21.15.36:48762) 2022-06-17T12:46:54.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e65e3c700 1 -- 172.21.15.36:0/3702481638 learned_addr learned my addr 172.21.15.36:0/3702481638 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:46:54.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e6563b700 1 --2- 172.21.15.36:0/3702481638 >> v2:172.21.15.36:3300/0 conn(0x7f7e601469c0 0x7f7e60146da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:54.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e64e3a700 1 --2- 172.21.15.36:0/3702481638 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e600b6600 0x7f7e600b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:54.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e4bfff700 1 -- 172.21.15.36:0/3702481638 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1151304775 0 0) 0x7f7e600b48e0 con 0x7f7e600b54d0 2022-06-17T12:46:54.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e4bfff700 1 -- 172.21.15.36:0/3702481638 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e44003260 con 0x7f7e600b54d0 2022-06-17T12:46:54.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e6563b700 1 -- 172.21.15.36:0/3702481638 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e600b6600 msgr2=0x7f7e600b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:54.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e6563b700 1 --2- 172.21.15.36:0/3702481638 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e600b6600 0x7f7e600b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:54.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e6563b700 1 -- 172.21.15.36:0/3702481638 >> v1:172.21.15.36:6789/0 conn(0x7f7e600b54d0 legacy=0x7f7e60062080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:54.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e6563b700 1 -- 172.21.15.36:0/3702481638 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e600b6aa0 con 0x7f7e601469c0 2022-06-17T12:46:54.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e64e3a700 1 --2- 172.21.15.36:0/3702481638 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e600b6600 0x7f7e600b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:46:54.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e6563b700 1 --2- 172.21.15.36:0/3702481638 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7e601469c0 0x7f7e60146da0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f7e5c002f70 tx=0x7f7e5c006df0).ready entity=mon.0 client_cookie=eea67bb2ad6fe007 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:54.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e4bfff700 1 -- 172.21.15.36:0/3702481638 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e5c00d4a0 con 0x7f7e601469c0 2022-06-17T12:46:54.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e4bfff700 1 -- 172.21.15.36:0/3702481638 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5c000e90 con 0x7f7e601469c0 2022-06-17T12:46:54.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e4bfff700 1 -- 172.21.15.36:0/3702481638 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e5c00d7f0 con 0x7f7e601469c0 2022-06-17T12:46:54.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e6789f700 1 -- 172.21.15.36:0/3702481638 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7e601469c0 msgr2=0x7f7e60146da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:54.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e6789f700 1 --2- 172.21.15.36:0/3702481638 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7e601469c0 0x7f7e60146da0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f7e5c002f70 tx=0x7f7e5c006df0).stop 2022-06-17T12:46:54.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e6789f700 1 -- 172.21.15.36:0/3702481638 shutdown_connections 2022-06-17T12:46:54.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e6789f700 1 --2- 172.21.15.36:0/3702481638 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e600b6600 0x7f7e600b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:54.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e6789f700 1 --2- 172.21.15.36:0/3702481638 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7e601469c0 0x7f7e60146da0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:54.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e6789f700 1 -- 172.21.15.36:0/3702481638 >> 172.21.15.36:0/3702481638 conn(0x7f7e6005f770 msgr2=0x7f7e6005fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:54.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e6789f700 1 -- 172.21.15.36:0/3702481638 shutdown_connections 2022-06-17T12:46:54.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.447+0000 7f7e6789f700 1 -- 172.21.15.36:0/3702481638 wait complete. 2022-06-17T12:46:54.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.451+0000 7f7e6789f700 1 Processor -- start 2022-06-17T12:46:54.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.451+0000 7f7e6789f700 1 -- start start 2022-06-17T12:46:54.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.451+0000 7f7e6789f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e600b54d0 0x7f7e600b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:54.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.451+0000 7f7e6789f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7e600b6600 0x7f7e60141f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:54.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.451+0000 7f7e6789f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e601469c0 0x7f7e6013eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:54.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.451+0000 7f7e6789f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e60112ba0 con 0x7f7e600b6600 2022-06-17T12:46:54.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.451+0000 7f7e6789f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e60143000 con 0x7f7e600b54d0 2022-06-17T12:46:54.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.451+0000 7f7e6789f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e60143c30 con 0x7f7e601469c0 2022-06-17T12:46:54.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.451+0000 7f7e64e3a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7e600b6600 0x7f7e60141f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:54.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.451+0000 7f7e64e3a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7e600b6600 0x7f7e60141f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59468/0 (socket says 172.21.15.36:59468) 2022-06-17T12:46:54.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.451+0000 7f7e64e3a700 1 -- 172.21.15.36:0/3726675294 learned_addr learned my addr 172.21.15.36:0/3726675294 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:54.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.451+0000 7f7e6563b700 1 --2- 172.21.15.36:0/3726675294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e600b54d0 0x7f7e600b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:54.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.451+0000 7f7e65e3c700 1 --2- 172.21.15.36:0/3726675294 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e601469c0 0x7f7e6013eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:54.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.451+0000 7f7e64e3a700 1 -- 172.21.15.36:0/3726675294 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e601469c0 msgr2=0x7f7e6013eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:54.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.451+0000 7f7e64e3a700 1 --2- 172.21.15.36:0/3726675294 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e601469c0 0x7f7e6013eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:54.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.451+0000 7f7e64e3a700 1 -- 172.21.15.36:0/3726675294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e600b54d0 msgr2=0x7f7e600b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:54.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.451+0000 7f7e64e3a700 1 --2- 172.21.15.36:0/3726675294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e600b54d0 0x7f7e600b35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:54.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.451+0000 7f7e64e3a700 1 -- 172.21.15.36:0/3726675294 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5400b180 con 0x7f7e600b6600 2022-06-17T12:46:54.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.451+0000 7f7e64e3a700 1 --2- 172.21.15.36:0/3726675294 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7e600b6600 0x7f7e60141f00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f7e5400c000 tx=0x7f7e54006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:54.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.451+0000 7f7e52ffd700 1 -- 172.21.15.36:0/3726675294 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e5400c7e0 con 0x7f7e600b6600 2022-06-17T12:46:54.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.451+0000 7f7e52ffd700 1 -- 172.21.15.36:0/3726675294 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5400c940 con 0x7f7e600b6600 2022-06-17T12:46:54.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.451+0000 7f7e6789f700 1 -- 172.21.15.36:0/3726675294 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e5c00b180 con 0x7f7e600b6600 2022-06-17T12:46:54.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.451+0000 7f7e52ffd700 1 -- 172.21.15.36:0/3726675294 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e5400cca0 con 0x7f7e600b6600 2022-06-17T12:46:54.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.451+0000 7f7e6789f700 1 -- 172.21.15.36:0/3726675294 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e601438b0 con 0x7f7e600b6600 2022-06-17T12:46:54.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.455+0000 7f7e6789f700 1 -- 172.21.15.36:0/3726675294 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e60112d10 con 0x7f7e600b6600 2022-06-17T12:46:54.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.455+0000 7f7e52ffd700 1 -- 172.21.15.36:0/3726675294 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f7e5401c460 con 0x7f7e600b6600 2022-06-17T12:46:54.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.455+0000 7f7e52ffd700 1 --2- 172.21.15.36:0/3726675294 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7e3c053a60 0x7f7e3c055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:54.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.455+0000 7f7e52ffd700 1 -- 172.21.15.36:0/3726675294 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f7e5406c580 con 0x7f7e600b6600 2022-06-17T12:46:54.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.455+0000 7f7e6563b700 1 --2- 172.21.15.36:0/3726675294 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7e3c053a60 0x7f7e3c055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:54.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.455+0000 7f7e6563b700 1 --2- 172.21.15.36:0/3726675294 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7e3c053a60 0x7f7e3c055f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f7e5c01a000 tx=0x7f7e5c014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:54.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.463+0000 7f7e52ffd700 1 -- 172.21.15.36:0/3726675294 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7e54043320 con 0x7f7e600b6600 2022-06-17T12:46:54.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.627+0000 7f7e6789f700 1 -- 172.21.15.36:0/3726675294 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e600b5950 con 0x7f7e600b6600 2022-06-17T12:46:54.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.627+0000 7f7e52ffd700 1 -- 172.21.15.36:0/3726675294 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2134 (secure 0 0 0) 0x7f7e54046940 con 0x7f7e600b6600 2022-06-17T12:46:54.654 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:46:54.654 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/4243909763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":4243909763},{"type":"v1","addr":"172.21.15.36:6835","nonce":4243909763}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:46:52.253609+0000","modified":"2022-06-17T12:46:54.268223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/3425584732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3425584732},{"type":"v1","addr":"172.21.15.70:6833","nonce":3425584732}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-06-17T12:46:54.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.631+0000 7f7e6789f700 1 -- 172.21.15.36:0/3726675294 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7e3c053a60 msgr2=0x7f7e3c055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:54.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.631+0000 7f7e6789f700 1 --2- 172.21.15.36:0/3726675294 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7e3c053a60 0x7f7e3c055f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f7e5c01a000 tx=0x7f7e5c014000).stop 2022-06-17T12:46:54.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.631+0000 7f7e6789f700 1 -- 172.21.15.36:0/3726675294 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7e600b6600 msgr2=0x7f7e60141f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:54.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.631+0000 7f7e6789f700 1 --2- 172.21.15.36:0/3726675294 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7e600b6600 0x7f7e60141f00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f7e5400c000 tx=0x7f7e54006df0).stop 2022-06-17T12:46:54.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.631+0000 7f7e6789f700 1 -- 172.21.15.36:0/3726675294 shutdown_connections 2022-06-17T12:46:54.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.631+0000 7f7e6789f700 1 --2- 172.21.15.36:0/3726675294 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7e3c053a60 0x7f7e3c055f20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:54.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.631+0000 7f7e6789f700 1 --2- 172.21.15.36:0/3726675294 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e601469c0 0x7f7e6013eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:54.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.631+0000 7f7e6789f700 1 --2- 172.21.15.36:0/3726675294 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7e600b6600 0x7f7e60141f00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:54.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.631+0000 7f7e6789f700 1 --2- 172.21.15.36:0/3726675294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e600b54d0 0x7f7e600b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:54.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.631+0000 7f7e6789f700 1 -- 172.21.15.36:0/3726675294 >> 172.21.15.36:0/3726675294 conn(0x7f7e6005f770 msgr2=0x7f7e600b76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:54.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.631+0000 7f7e6789f700 1 -- 172.21.15.36:0/3726675294 shutdown_connections 2022-06-17T12:46:54.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.631+0000 7f7e6789f700 1 -- 172.21.15.36:0/3726675294 wait complete. 2022-06-17T12:46:54.759 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 7 2022-06-17T12:46:54.826 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:46:54.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.986+0000 7f0fab646700 1 Processor -- start 2022-06-17T12:46:54.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.986+0000 7f0fab646700 1 -- start start 2022-06-17T12:46:55.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0fab646700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0fa4062770 0x7f0fa4062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:55.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0fab646700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0fa41444f0 0x7f0fa4144960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:55.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0fab646700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fa40b0ba0 con 0x7f0fa40ad330 2022-06-17T12:46:55.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0fab646700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fa40ae820 con 0x7f0fa4062770 2022-06-17T12:46:55.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0fab646700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fa4061030 con 0x7f0fa41444f0 2022-06-17T12:46:55.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0fa93e2700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0fa4062770 0x7f0fa4062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:55.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0fa9be3700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f0fa40ad330 0x7f0fa40ad7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:48776/0 (socket says 172.21.15.36:48776) 2022-06-17T12:46:55.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0fa9be3700 1 -- 172.21.15.36:0/3782012057 learned_addr learned my addr 172.21.15.36:0/3782012057 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:46:55.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0fa8be1700 1 --2- 172.21.15.36:0/3782012057 >> v2:172.21.15.70:3300/0 conn(0x7f0fa41444f0 0x7f0fa4144960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:55.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0f97fff700 1 -- 172.21.15.36:0/3782012057 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1925241140 0 0) 0x7f0fa40b0ba0 con 0x7f0fa40ad330 2022-06-17T12:46:55.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0f97fff700 1 -- 172.21.15.36:0/3782012057 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f88003260 con 0x7f0fa40ad330 2022-06-17T12:46:55.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0fa8be1700 1 -- 172.21.15.36:0/3782012057 >> v2:172.21.15.36:3300/0 conn(0x7f0fa4062770 msgr2=0x7f0fa4062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:55.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0fa8be1700 1 --2- 172.21.15.36:0/3782012057 >> v2:172.21.15.36:3300/0 conn(0x7f0fa4062770 0x7f0fa4062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:55.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0fa8be1700 1 -- 172.21.15.36:0/3782012057 >> v1:172.21.15.36:6789/0 conn(0x7f0fa40ad330 legacy=0x7f0fa40ad7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:55.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0fa8be1700 1 -- 172.21.15.36:0/3782012057 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa4147e90 con 0x7f0fa41444f0 2022-06-17T12:46:55.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0fa8be1700 1 --2- 172.21.15.36:0/3782012057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fa41444f0 0x7f0fa4144960 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f0f98002f70 tx=0x7f0f98006df0).ready entity=mon.1 client_cookie=b384cd48597235ca server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:55.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0f97fff700 1 -- 172.21.15.36:0/3782012057 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f9800d4f0 con 0x7f0fa41444f0 2022-06-17T12:46:55.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0f97fff700 1 -- 172.21.15.36:0/3782012057 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f98000e90 con 0x7f0fa41444f0 2022-06-17T12:46:55.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0f97fff700 1 -- 172.21.15.36:0/3782012057 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f9800d840 con 0x7f0fa41444f0 2022-06-17T12:46:55.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0fab646700 1 -- 172.21.15.36:0/3782012057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fa41444f0 msgr2=0x7f0fa4144960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:55.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0fab646700 1 --2- 172.21.15.36:0/3782012057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fa41444f0 0x7f0fa4144960 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f0f98002f70 tx=0x7f0f98006df0).stop 2022-06-17T12:46:55.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0fab646700 1 -- 172.21.15.36:0/3782012057 shutdown_connections 2022-06-17T12:46:55.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0fab646700 1 --2- 172.21.15.36:0/3782012057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fa41444f0 0x7f0fa4144960 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:55.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0fab646700 1 --2- 172.21.15.36:0/3782012057 >> v2:172.21.15.36:3300/0 conn(0x7f0fa4062770 0x7f0fa4062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:55.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0fab646700 1 -- 172.21.15.36:0/3782012057 >> 172.21.15.36:0/3782012057 conn(0x7f0fa40b53e0 msgr2=0x7f0fa40b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:55.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0fab646700 1 -- 172.21.15.36:0/3782012057 shutdown_connections 2022-06-17T12:46:55.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.990+0000 7f0fab646700 1 -- 172.21.15.36:0/3782012057 wait complete. 2022-06-17T12:46:55.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0fab646700 1 Processor -- start 2022-06-17T12:46:55.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0fab646700 1 -- start start 2022-06-17T12:46:55.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0fab646700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fa4062770 0x7f0fa414d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:55.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0fab646700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0fa40ad330 0x7f0fa40657b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:55.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0fab646700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0fa41444f0 0x7f0fa4066c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:55.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0fab646700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa4066860 con 0x7f0fa40ad330 2022-06-17T12:46:55.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0fab646700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa414aaa0 con 0x7f0fa4062770 2022-06-17T12:46:55.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0fab646700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa4065ca0 con 0x7f0fa41444f0 2022-06-17T12:46:55.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0fa8be1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0fa40ad330 0x7f0fa40657b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:55.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0fa8be1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0fa40ad330 0x7f0fa40657b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59482/0 (socket says 172.21.15.36:59482) 2022-06-17T12:46:55.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0fa8be1700 1 -- 172.21.15.36:0/3978351074 learned_addr learned my addr 172.21.15.36:0/3978351074 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:55.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0fa93e2700 1 --2- 172.21.15.36:0/3978351074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fa4062770 0x7f0fa414d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:55.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0fa9be3700 1 --2- 172.21.15.36:0/3978351074 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0fa41444f0 0x7f0fa4066c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:55.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0fa8be1700 1 -- 172.21.15.36:0/3978351074 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0fa41444f0 msgr2=0x7f0fa4066c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:55.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0fa8be1700 1 --2- 172.21.15.36:0/3978351074 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0fa41444f0 0x7f0fa4066c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:55.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0fa8be1700 1 -- 172.21.15.36:0/3978351074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fa4062770 msgr2=0x7f0fa414d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:55.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0fa8be1700 1 --2- 172.21.15.36:0/3978351074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fa4062770 0x7f0fa414d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:55.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0fa8be1700 1 -- 172.21.15.36:0/3978351074 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f9800b180 con 0x7f0fa40ad330 2022-06-17T12:46:55.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0fa93e2700 1 --2- 172.21.15.36:0/3978351074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fa4062770 0x7f0fa414d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:46:55.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0fa8be1700 1 --2- 172.21.15.36:0/3978351074 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0fa40ad330 0x7f0fa40657b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f0f98002f70 tx=0x7f0f98014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:55.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0f967fc700 1 -- 172.21.15.36:0/3978351074 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f98013e90 con 0x7f0fa40ad330 2022-06-17T12:46:55.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0f967fc700 1 -- 172.21.15.36:0/3978351074 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f9800de60 con 0x7f0fa40ad330 2022-06-17T12:46:55.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0fab646700 1 -- 172.21.15.36:0/3978351074 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa414ad40 con 0x7f0fa40ad330 2022-06-17T12:46:55.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0f967fc700 1 -- 172.21.15.36:0/3978351074 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f9801e5f0 con 0x7f0fa40ad330 2022-06-17T12:46:55.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.994+0000 7f0fab646700 1 -- 172.21.15.36:0/3978351074 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa414b410 con 0x7f0fa40ad330 2022-06-17T12:46:55.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.998+0000 7f0fab646700 1 -- 172.21.15.36:0/3978351074 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fa40669d0 con 0x7f0fa40ad330 2022-06-17T12:46:55.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.998+0000 7f0f967fc700 1 -- 172.21.15.36:0/3978351074 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f0f98025030 con 0x7f0fa40ad330 2022-06-17T12:46:55.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.998+0000 7f0f967fc700 1 --2- 172.21.15.36:0/3978351074 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0f8c053ad0 0x7f0f8c055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:55.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.998+0000 7f0f967fc700 1 -- 172.21.15.36:0/3978351074 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0f9806ccd0 con 0x7f0fa40ad330 2022-06-17T12:46:55.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.998+0000 7f0fa93e2700 1 --2- 172.21.15.36:0/3978351074 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0f8c053ad0 0x7f0f8c055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:55.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:54.998+0000 7f0fa93e2700 1 --2- 172.21.15.36:0/3978351074 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0f8c053ad0 0x7f0f8c055f90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f0fa000b000 tx=0x7f0fa0006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:55.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.006+0000 7f0f967fc700 1 -- 172.21.15.36:0/3978351074 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0fa40669d0 con 0x7f0fa40ad330 2022-06-17T12:46:55.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.130+0000 7f0fab646700 1 -- 172.21.15.36:0/3978351074 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fa40669a0 con 0x7f0fa40ad330 2022-06-17T12:46:55.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.134+0000 7f0f967fc700 1 -- 172.21.15.36:0/3978351074 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f0f98023070 con 0x7f0fa40ad330 2022-06-17T12:46:55.135 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:46:55.135 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":26,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:46:53.258075+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:46:47.465328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-17T12:46:49.893338+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:46:55.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.134+0000 7f0fab646700 1 -- 172.21.15.36:0/3978351074 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0f8c053ad0 msgr2=0x7f0f8c055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:55.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.134+0000 7f0fab646700 1 --2- 172.21.15.36:0/3978351074 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0f8c053ad0 0x7f0f8c055f90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f0fa000b000 tx=0x7f0fa0006df0).stop 2022-06-17T12:46:55.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.134+0000 7f0fab646700 1 -- 172.21.15.36:0/3978351074 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0fa40ad330 msgr2=0x7f0fa40657b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:55.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.134+0000 7f0fab646700 1 --2- 172.21.15.36:0/3978351074 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0fa40ad330 0x7f0fa40657b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f0f98002f70 tx=0x7f0f98014000).stop 2022-06-17T12:46:55.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.134+0000 7f0fab646700 1 -- 172.21.15.36:0/3978351074 shutdown_connections 2022-06-17T12:46:55.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.134+0000 7f0fab646700 1 --2- 172.21.15.36:0/3978351074 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0f8c053ad0 0x7f0f8c055f90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:55.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.134+0000 7f0fab646700 1 --2- 172.21.15.36:0/3978351074 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0fa41444f0 0x7f0fa4066c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:55.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.134+0000 7f0fab646700 1 --2- 172.21.15.36:0/3978351074 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0fa40ad330 0x7f0fa40657b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:55.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.134+0000 7f0fab646700 1 --2- 172.21.15.36:0/3978351074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fa4062770 0x7f0fa414d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:55.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.134+0000 7f0fab646700 1 -- 172.21.15.36:0/3978351074 >> 172.21.15.36:0/3978351074 conn(0x7f0fa40b53e0 msgr2=0x7f0fa4145540 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:55.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.134+0000 7f0fab646700 1 -- 172.21.15.36:0/3978351074 shutdown_connections 2022-06-17T12:46:55.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.134+0000 7f0fab646700 1 -- 172.21.15.36:0/3978351074 wait complete. 2022-06-17T12:46:55.338 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:46:55.355 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2022-06-17T12:46:55.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.482+0000 7f3390c6c700 1 Processor -- start 2022-06-17T12:46:55.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.482+0000 7f3390c6c700 1 -- start start 2022-06-17T12:46:55.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.482+0000 7f3390c6c700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f338c062770 0x7f338c062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:55.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.482+0000 7f3390c6c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f338c0ad330 0x7f338c0ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:55.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.482+0000 7f3390c6c700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f338c0b0ba0 con 0x7f338c1444f0 2022-06-17T12:46:55.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.482+0000 7f3390c6c700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f338c0ae820 con 0x7f338c0ad330 2022-06-17T12:46:55.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.482+0000 7f3390c6c700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f338c061030 con 0x7f338c062770 2022-06-17T12:46:55.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.482+0000 7f338ad9d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f338c0ad330 0x7f338c0ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:55.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.482+0000 7f3389d9b700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f338c1444f0 0x7f338c144960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:48788/0 (socket says 172.21.15.36:48788) 2022-06-17T12:46:55.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.482+0000 7f3389d9b700 1 -- 172.21.15.36:0/3778737476 learned_addr learned my addr 172.21.15.36:0/3778737476 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:46:55.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.482+0000 7f338a59c700 1 --2- 172.21.15.36:0/3778737476 >> v2:172.21.15.70:3300/0 conn(0x7f338c062770 0x7f338c062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:55.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.482+0000 7f338ad9d700 1 -- 172.21.15.36:0/3778737476 >> v2:172.21.15.70:3300/0 conn(0x7f338c062770 msgr2=0x7f338c062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:55.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.482+0000 7f338ad9d700 1 --2- 172.21.15.36:0/3778737476 >> v2:172.21.15.70:3300/0 conn(0x7f338c062770 0x7f338c062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:55.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.482+0000 7f338ad9d700 1 -- 172.21.15.36:0/3778737476 >> v1:172.21.15.36:6789/0 conn(0x7f338c1444f0 legacy=0x7f338c144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:55.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.482+0000 7f338ad9d700 1 -- 172.21.15.36:0/3778737476 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338c147e90 con 0x7f338c0ad330 2022-06-17T12:46:55.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.482+0000 7f338ad9d700 1 --2- 172.21.15.36:0/3778737476 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f338c0ad330 0x7f338c0ad7c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f338c061170 tx=0x7f33840091d0).ready entity=mon.0 client_cookie=bbe652442cafa33b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:55.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.482+0000 7f338959a700 1 -- 172.21.15.36:0/3778737476 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f338400b680 con 0x7f338c0ad330 2022-06-17T12:46:55.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.482+0000 7f338959a700 1 -- 172.21.15.36:0/3778737476 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338400b7e0 con 0x7f338c0ad330 2022-06-17T12:46:55.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.482+0000 7f338959a700 1 -- 172.21.15.36:0/3778737476 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f338400b950 con 0x7f338c0ad330 2022-06-17T12:46:55.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.482+0000 7f3390c6c700 1 -- 172.21.15.36:0/3778737476 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f338c0ad330 msgr2=0x7f338c0ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:55.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.482+0000 7f3390c6c700 1 --2- 172.21.15.36:0/3778737476 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f338c0ad330 0x7f338c0ad7c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f338c061170 tx=0x7f33840091d0).stop 2022-06-17T12:46:55.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.482+0000 7f3390c6c700 1 -- 172.21.15.36:0/3778737476 shutdown_connections 2022-06-17T12:46:55.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.482+0000 7f3390c6c700 1 --2- 172.21.15.36:0/3778737476 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f338c0ad330 0x7f338c0ad7c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:55.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.482+0000 7f3390c6c700 1 --2- 172.21.15.36:0/3778737476 >> v2:172.21.15.70:3300/0 conn(0x7f338c062770 0x7f338c062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:55.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.486+0000 7f3390c6c700 1 -- 172.21.15.36:0/3778737476 >> 172.21.15.36:0/3778737476 conn(0x7f338c0b53e0 msgr2=0x7f338c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:55.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.486+0000 7f3390c6c700 1 -- 172.21.15.36:0/3778737476 shutdown_connections 2022-06-17T12:46:55.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.486+0000 7f3390c6c700 1 -- 172.21.15.36:0/3778737476 wait complete. 2022-06-17T12:46:55.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.486+0000 7f3390c6c700 1 Processor -- start 2022-06-17T12:46:55.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.486+0000 7f3390c6c700 1 -- start start 2022-06-17T12:46:55.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.490+0000 7f3390c6c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f338c062770 0x7f338c146ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:55.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.490+0000 7f3390c6c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f338c0ad330 0x7f338c14a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:55.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.490+0000 7f3390c6c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f338c1444f0 0x7f338c159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:55.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.490+0000 7f3390c6c700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c15a220 con 0x7f338c062770 2022-06-17T12:46:55.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.490+0000 7f3390c6c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c0648d0 con 0x7f338c0ad330 2022-06-17T12:46:55.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.490+0000 7f3390c6c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338c14ac80 con 0x7f338c1444f0 2022-06-17T12:46:55.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.490+0000 7f338a59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f338c062770 0x7f338c146ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:55.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.490+0000 7f338a59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f338c062770 0x7f338c146ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59494/0 (socket says 172.21.15.36:59494) 2022-06-17T12:46:55.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.490+0000 7f338a59c700 1 -- 172.21.15.36:0/3815242372 learned_addr learned my addr 172.21.15.36:0/3815242372 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:55.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.490+0000 7f3389d9b700 1 --2- 172.21.15.36:0/3815242372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f338c0ad330 0x7f338c14a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:55.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.490+0000 7f338ad9d700 1 --2- 172.21.15.36:0/3815242372 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f338c1444f0 0x7f338c159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:55.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.490+0000 7f338a59c700 1 -- 172.21.15.36:0/3815242372 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f338c1444f0 msgr2=0x7f338c159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:55.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.490+0000 7f338a59c700 1 --2- 172.21.15.36:0/3815242372 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f338c1444f0 0x7f338c159d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:55.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.490+0000 7f338a59c700 1 -- 172.21.15.36:0/3815242372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f338c0ad330 msgr2=0x7f338c14a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:55.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.490+0000 7f338a59c700 1 --2- 172.21.15.36:0/3815242372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f338c0ad330 0x7f338c14a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:55.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.490+0000 7f338a59c700 1 -- 172.21.15.36:0/3815242372 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3384009470 con 0x7f338c062770 2022-06-17T12:46:55.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.490+0000 7f338a59c700 1 --2- 172.21.15.36:0/3815242372 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f338c062770 0x7f338c146ba0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f337c00c000 tx=0x7f337c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:55.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.490+0000 7f337b7fe700 1 -- 172.21.15.36:0/3815242372 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f337c010780 con 0x7f338c062770 2022-06-17T12:46:55.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.490+0000 7f337b7fe700 1 -- 172.21.15.36:0/3815242372 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337c0108e0 con 0x7f338c062770 2022-06-17T12:46:55.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.490+0000 7f3390c6c700 1 -- 172.21.15.36:0/3815242372 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f338c14af20 con 0x7f338c062770 2022-06-17T12:46:55.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.490+0000 7f337b7fe700 1 -- 172.21.15.36:0/3815242372 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f337c010c40 con 0x7f338c062770 2022-06-17T12:46:55.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.490+0000 7f3390c6c700 1 -- 172.21.15.36:0/3815242372 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f338c0636f0 con 0x7f338c062770 2022-06-17T12:46:55.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.490+0000 7f3390c6c700 1 -- 172.21.15.36:0/3815242372 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f338c064a90 con 0x7f338c062770 2022-06-17T12:46:55.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.490+0000 7f337b7fe700 1 -- 172.21.15.36:0/3815242372 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f337c01c460 con 0x7f338c062770 2022-06-17T12:46:55.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.494+0000 7f337b7fe700 1 --2- 172.21.15.36:0/3815242372 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3368053ad0 0x7f3368055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:55.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.494+0000 7f337b7fe700 1 -- 172.21.15.36:0/3815242372 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f337c06c620 con 0x7f338c062770 2022-06-17T12:46:55.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.494+0000 7f3389d9b700 1 --2- 172.21.15.36:0/3815242372 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3368053ad0 0x7f3368055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:55.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.494+0000 7f3389d9b700 1 --2- 172.21.15.36:0/3815242372 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3368053ad0 0x7f3368055f90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f338000d000 tx=0x7f3380000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:55.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.498+0000 7f337b7fe700 1 -- 172.21.15.36:0/3815242372 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f338c064a90 con 0x7f338c062770 2022-06-17T12:46:55.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.630+0000 7f3390c6c700 1 -- 172.21.15.36:0/3815242372 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f338c064a90 con 0x7f338c062770 2022-06-17T12:46:55.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.630+0000 7f337b7fe700 1 -- 172.21.15.36:0/3815242372 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f338c064a90 con 0x7f338c062770 2022-06-17T12:46:55.621 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:46:55.622 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":26,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:46:53.258075+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:46:47.465328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-17T12:46:49.893338+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:46:55.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.634+0000 7f3390c6c700 1 -- 172.21.15.36:0/3815242372 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3368053ad0 msgr2=0x7f3368055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:55.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.634+0000 7f3390c6c700 1 --2- 172.21.15.36:0/3815242372 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3368053ad0 0x7f3368055f90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f338000d000 tx=0x7f3380000bc0).stop 2022-06-17T12:46:55.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.634+0000 7f3390c6c700 1 -- 172.21.15.36:0/3815242372 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f338c062770 msgr2=0x7f338c146ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:55.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.634+0000 7f3390c6c700 1 --2- 172.21.15.36:0/3815242372 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f338c062770 0x7f338c146ba0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f337c00c000 tx=0x7f337c006df0).stop 2022-06-17T12:46:55.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.634+0000 7f3390c6c700 1 -- 172.21.15.36:0/3815242372 shutdown_connections 2022-06-17T12:46:55.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.634+0000 7f3390c6c700 1 --2- 172.21.15.36:0/3815242372 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3368053ad0 0x7f3368055f90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:55.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.634+0000 7f3390c6c700 1 --2- 172.21.15.36:0/3815242372 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f338c1444f0 0x7f338c159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:55.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.634+0000 7f3390c6c700 1 --2- 172.21.15.36:0/3815242372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f338c0ad330 0x7f338c14a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:55.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.634+0000 7f3390c6c700 1 --2- 172.21.15.36:0/3815242372 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f338c062770 0x7f338c146ba0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:55.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.634+0000 7f3390c6c700 1 -- 172.21.15.36:0/3815242372 >> 172.21.15.36:0/3815242372 conn(0x7f338c0b53e0 msgr2=0x7f338c0b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:55.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.634+0000 7f3390c6c700 1 -- 172.21.15.36:0/3815242372 shutdown_connections 2022-06-17T12:46:55.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.634+0000 7f3390c6c700 1 -- 172.21.15.36:0/3815242372 wait complete. 2022-06-17T12:46:55.683 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:46:47.465328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:46:49.893338+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:46:55.685 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:46:55.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.830+0000 7fe87499d700 1 Processor -- start 2022-06-17T12:46:55.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.830+0000 7fe87499d700 1 -- start start 2022-06-17T12:46:55.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.830+0000 7fe87499d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe8700b64f0 0x7fe870143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:55.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.830+0000 7fe87499d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8700b4120 0x7fe870146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:55.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.830+0000 7fe87499d700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8700b68d0 con 0x7fe870147270 2022-06-17T12:46:55.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.830+0000 7fe87499d700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8701476b0 con 0x7fe8700b64f0 2022-06-17T12:46:55.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.830+0000 7fe87499d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8700b6e20 con 0x7fe8700b4120 2022-06-17T12:46:55.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.830+0000 7fe86e59c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe8700b64f0 0x7fe870143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:55.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.830+0000 7fe86e59c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe8700b64f0 0x7fe870143150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59502/0 (socket says 172.21.15.36:59502) 2022-06-17T12:46:55.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.830+0000 7fe86e59c700 1 -- 172.21.15.36:0/3603791295 learned_addr learned my addr 172.21.15.36:0/3603791295 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:55.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.830+0000 7fe865d9b700 1 --2- 172.21.15.36:0/3603791295 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8700b4120 0x7fe870146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:55.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.830+0000 7fe86dd9b700 1 -- 172.21.15.36:0/3603791295 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 545334492 0 0) 0x7fe8700b68d0 con 0x7fe870147270 2022-06-17T12:46:55.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.830+0000 7fe86e59c700 1 -- 172.21.15.36:0/3603791295 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8700b4120 msgr2=0x7fe870146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:55.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.830+0000 7fe86e59c700 1 --2- 172.21.15.36:0/3603791295 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8700b4120 0x7fe870146d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:55.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.830+0000 7fe86e59c700 1 -- 172.21.15.36:0/3603791295 >> v1:172.21.15.70:6789/0 conn(0x7fe870147270 legacy=0x7fe870140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:55.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.830+0000 7fe86e59c700 1 -- 172.21.15.36:0/3603791295 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8701468e0 con 0x7fe8700b64f0 2022-06-17T12:46:55.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.830+0000 7fe86e59c700 1 --2- 172.21.15.36:0/3603791295 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8700b64f0 0x7fe870143150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fe860002f70 tx=0x7fe860006eb0).ready entity=mon.0 client_cookie=3470420e9a413cc7 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:55.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.834+0000 7fe86dd9b700 1 -- 172.21.15.36:0/3603791295 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe86000d570 con 0x7fe8700b64f0 2022-06-17T12:46:55.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.834+0000 7fe86dd9b700 1 -- 172.21.15.36:0/3603791295 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe860000e90 con 0x7fe8700b64f0 2022-06-17T12:46:55.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.834+0000 7fe86dd9b700 1 -- 172.21.15.36:0/3603791295 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe86000d8c0 con 0x7fe8700b64f0 2022-06-17T12:46:55.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.834+0000 7fe87499d700 1 -- 172.21.15.36:0/3603791295 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8700b64f0 msgr2=0x7fe870143150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:55.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.834+0000 7fe87499d700 1 --2- 172.21.15.36:0/3603791295 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8700b64f0 0x7fe870143150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fe860002f70 tx=0x7fe860006eb0).stop 2022-06-17T12:46:55.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.834+0000 7fe87499d700 1 -- 172.21.15.36:0/3603791295 shutdown_connections 2022-06-17T12:46:55.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.834+0000 7fe87499d700 1 --2- 172.21.15.36:0/3603791295 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8700b4120 0x7fe870146d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:55.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.834+0000 7fe87499d700 1 --2- 172.21.15.36:0/3603791295 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8700b64f0 0x7fe870143150 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:55.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.834+0000 7fe87499d700 1 -- 172.21.15.36:0/3603791295 >> 172.21.15.36:0/3603791295 conn(0x7fe8700620f0 msgr2=0x7fe870062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:56.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.834+0000 7fe87499d700 1 -- 172.21.15.36:0/3603791295 shutdown_connections 2022-06-17T12:46:56.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.834+0000 7fe87499d700 1 -- 172.21.15.36:0/3603791295 wait complete. 2022-06-17T12:46:56.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.834+0000 7fe87499d700 1 Processor -- start 2022-06-17T12:46:56.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.834+0000 7fe87499d700 1 -- start start 2022-06-17T12:46:56.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.838+0000 7fe87499d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8700b4120 0x7fe870152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:56.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.838+0000 7fe87499d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8700b64f0 0x7fe8701408f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:56.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.838+0000 7fe87499d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe870147270 0x7fe87013d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:56.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.838+0000 7fe87499d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8700b32d0 con 0x7fe870147270 2022-06-17T12:46:56.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.838+0000 7fe87499d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe870152300 con 0x7fe8700b64f0 2022-06-17T12:46:56.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.838+0000 7fe87499d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8701525a0 con 0x7fe8700b4120 2022-06-17T12:46:56.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.838+0000 7fe86ed9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe870147270 0x7fe87013d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:56.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.838+0000 7fe86ed9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe870147270 0x7fe87013d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59512/0 (socket says 172.21.15.36:59512) 2022-06-17T12:46:56.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.838+0000 7fe86ed9d700 1 -- 172.21.15.36:0/115893082 learned_addr learned my addr 172.21.15.36:0/115893082 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:56.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.838+0000 7fe86e59c700 1 --2- 172.21.15.36:0/115893082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8700b4120 0x7fe870152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:56.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.838+0000 7fe865d9b700 1 --2- 172.21.15.36:0/115893082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8700b64f0 0x7fe8701408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:56.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.838+0000 7fe86ed9d700 1 -- 172.21.15.36:0/115893082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8700b4120 msgr2=0x7fe870152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:56.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.838+0000 7fe86ed9d700 1 --2- 172.21.15.36:0/115893082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8700b4120 0x7fe870152f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:56.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.838+0000 7fe86ed9d700 1 -- 172.21.15.36:0/115893082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8700b64f0 msgr2=0x7fe8701408f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:56.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.838+0000 7fe86ed9d700 1 --2- 172.21.15.36:0/115893082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8700b64f0 0x7fe8701408f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:56.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.838+0000 7fe86ed9d700 1 -- 172.21.15.36:0/115893082 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86000b1e0 con 0x7fe870147270 2022-06-17T12:46:56.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.838+0000 7fe86e59c700 1 --2- 172.21.15.36:0/115893082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8700b4120 0x7fe870152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:46:56.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.838+0000 7fe86ed9d700 1 --2- 172.21.15.36:0/115893082 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe870147270 0x7fe87013d580 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fe868000f40 tx=0x7fe868007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:56.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.838+0000 7fe867fff700 1 -- 172.21.15.36:0/115893082 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe868006aa0 con 0x7fe870147270 2022-06-17T12:46:56.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.838+0000 7fe867fff700 1 -- 172.21.15.36:0/115893082 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe868006c00 con 0x7fe870147270 2022-06-17T12:46:56.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.838+0000 7fe867fff700 1 -- 172.21.15.36:0/115893082 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe868007030 con 0x7fe870147270 2022-06-17T12:46:56.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.838+0000 7fe87499d700 1 -- 172.21.15.36:0/115893082 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe870152840 con 0x7fe870147270 2022-06-17T12:46:56.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.838+0000 7fe87499d700 1 -- 172.21.15.36:0/115893082 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe870140590 con 0x7fe870147270 2022-06-17T12:46:56.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.842+0000 7fe87499d700 1 -- 172.21.15.36:0/115893082 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe870143640 con 0x7fe870147270 2022-06-17T12:46:56.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.842+0000 7fe867fff700 1 -- 172.21.15.36:0/115893082 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe868010030 con 0x7fe870147270 2022-06-17T12:46:56.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.842+0000 7fe867fff700 1 --2- 172.21.15.36:0/115893082 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe854053a40 0x7fe854055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:56.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.842+0000 7fe867fff700 1 -- 172.21.15.36:0/115893082 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe86806be70 con 0x7fe870147270 2022-06-17T12:46:56.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.842+0000 7fe86e59c700 1 --2- 172.21.15.36:0/115893082 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe854053a40 0x7fe854055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:56.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.842+0000 7fe86e59c700 1 --2- 172.21.15.36:0/115893082 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe854053a40 0x7fe854055f00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fe86001c000 tx=0x7fe860015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:56.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.850+0000 7fe867fff700 1 -- 172.21.15.36:0/115893082 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe868042c70 con 0x7fe870147270 2022-06-17T12:46:56.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.982+0000 7fe87499d700 1 -- 172.21.15.36:0/115893082 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe870057410 con 0x7fe870147270 2022-06-17T12:46:56.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.982+0000 7fe867fff700 1 -- 172.21.15.36:0/115893082 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe868046290 con 0x7fe870147270 2022-06-17T12:46:56.155 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:46:56.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.982+0000 7fe87499d700 1 -- 172.21.15.36:0/115893082 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe854053a40 msgr2=0x7fe854055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:56.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.982+0000 7fe87499d700 1 --2- 172.21.15.36:0/115893082 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe854053a40 0x7fe854055f00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fe86001c000 tx=0x7fe860015000).stop 2022-06-17T12:46:56.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.982+0000 7fe87499d700 1 -- 172.21.15.36:0/115893082 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe870147270 msgr2=0x7fe87013d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:56.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.982+0000 7fe87499d700 1 --2- 172.21.15.36:0/115893082 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe870147270 0x7fe87013d580 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fe868000f40 tx=0x7fe868007f90).stop 2022-06-17T12:46:56.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.982+0000 7fe87499d700 1 -- 172.21.15.36:0/115893082 shutdown_connections 2022-06-17T12:46:56.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.982+0000 7fe87499d700 1 --2- 172.21.15.36:0/115893082 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe854053a40 0x7fe854055f00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:56.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.982+0000 7fe87499d700 1 --2- 172.21.15.36:0/115893082 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe870147270 0x7fe87013d580 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:56.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.982+0000 7fe87499d700 1 --2- 172.21.15.36:0/115893082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8700b64f0 0x7fe8701408f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:56.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.982+0000 7fe87499d700 1 --2- 172.21.15.36:0/115893082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8700b4120 0x7fe870152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:56.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.982+0000 7fe87499d700 1 -- 172.21.15.36:0/115893082 >> 172.21.15.36:0/115893082 conn(0x7fe8700620f0 msgr2=0x7fe87013e820 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:56.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.982+0000 7fe87499d700 1 -- 172.21.15.36:0/115893082 shutdown_connections 2022-06-17T12:46:56.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:55.982+0000 7fe87499d700 1 -- 172.21.15.36:0/115893082 wait complete. 2022-06-17T12:46:56.172 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:46:56.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.302+0000 7fc86ee71700 1 Processor -- start 2022-06-17T12:46:56.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.302+0000 7fc86ee71700 1 -- start start 2022-06-17T12:46:56.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.302+0000 7fc86ee71700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc8680b45c0 0x7fc8680b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:56.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.302+0000 7fc86ee71700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8680b4160 con 0x7fc8680b9860 2022-06-17T12:46:56.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.306+0000 7fc86ee71700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8680b9c40 con 0x7fc8680b4f00 2022-06-17T12:46:56.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.306+0000 7fc86ee71700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8680627d0 con 0x7fc8680b45c0 2022-06-17T12:46:56.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.306+0000 7fc86d66e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc8680b45c0 0x7fc8680b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:56.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.306+0000 7fc86d66e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc8680b45c0 0x7fc8680b4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59518/0 (socket says 172.21.15.36:59518) 2022-06-17T12:46:56.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.306+0000 7fc86d66e700 1 -- 172.21.15.36:0/76909930 learned_addr learned my addr 172.21.15.36:0/76909930 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:56.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.306+0000 7fc86de6f700 1 --1- 172.21.15.36:0/76909930 >> v1:172.21.15.36:6789/0 conn(0x7fc8680b9860 0x7fc868143840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:48814/0 (socket says 172.21.15.36:48814) 2022-06-17T12:46:56.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.306+0000 7fc86d66e700 1 -- 172.21.15.36:0/76909930 >> v1:172.21.15.70:6789/0 conn(0x7fc8680b4f00 legacy=0x7fc8680b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:56.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.306+0000 7fc86d66e700 1 -- 172.21.15.36:0/76909930 >> v1:172.21.15.36:6789/0 conn(0x7fc8680b9860 legacy=0x7fc868143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:56.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.306+0000 7fc86d66e700 1 -- 172.21.15.36:0/76909930 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc868062a70 con 0x7fc8680b45c0 2022-06-17T12:46:56.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.306+0000 7fc86d66e700 1 --2- 172.21.15.36:0/76909930 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8680b45c0 0x7fc8680b4a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fc8680ba5d0 tx=0x7fc85c006eb0).ready entity=mon.0 client_cookie=4e43c7644ea1676e server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:56.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.306+0000 7fc86ce6d700 1 -- 172.21.15.36:0/76909930 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc85c00d660 con 0x7fc8680b45c0 2022-06-17T12:46:56.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.306+0000 7fc86ce6d700 1 -- 172.21.15.36:0/76909930 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85c000e90 con 0x7fc8680b45c0 2022-06-17T12:46:56.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.306+0000 7fc86ce6d700 1 -- 172.21.15.36:0/76909930 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc85c00d9f0 con 0x7fc8680b45c0 2022-06-17T12:46:56.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.306+0000 7fc86ee71700 1 -- 172.21.15.36:0/76909930 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8680b45c0 msgr2=0x7fc8680b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:56.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.306+0000 7fc86ee71700 1 --2- 172.21.15.36:0/76909930 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8680b45c0 0x7fc8680b4a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fc8680ba5d0 tx=0x7fc85c006eb0).stop 2022-06-17T12:46:56.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.306+0000 7fc86ee71700 1 -- 172.21.15.36:0/76909930 shutdown_connections 2022-06-17T12:46:56.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.306+0000 7fc86ee71700 1 --2- 172.21.15.36:0/76909930 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8680b45c0 0x7fc8680b4a10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:56.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.306+0000 7fc86ee71700 1 -- 172.21.15.36:0/76909930 >> 172.21.15.36:0/76909930 conn(0x7fc86805c550 msgr2=0x7fc868060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:56.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.306+0000 7fc86ee71700 1 -- 172.21.15.36:0/76909930 shutdown_connections 2022-06-17T12:46:56.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.306+0000 7fc86ee71700 1 -- 172.21.15.36:0/76909930 wait complete. 2022-06-17T12:46:56.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.306+0000 7fc86ee71700 1 Processor -- start 2022-06-17T12:46:56.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc86ee71700 1 -- start start 2022-06-17T12:46:56.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc86ee71700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8680b4f00 0x7fc868140470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:56.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc86ee71700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8680b9860 0x7fc8680b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:56.366 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc86ee71700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc86813d8c0 0x7fc868155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:56.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc86ee71700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8680af850 con 0x7fc86813d8c0 2022-06-17T12:46:56.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc86ee71700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8680afaf0 con 0x7fc8680b9860 2022-06-17T12:46:56.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc86ee71700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8680afd90 con 0x7fc8680b4f00 2022-06-17T12:46:56.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc86e670700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc86813d8c0 0x7fc868155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:56.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc86d66e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8680b9860 0x7fc8680b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:56.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc86de6f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8680b4f00 0x7fc868140470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:56.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc86d66e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8680b9860 0x7fc8680b8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:51672/0 (socket says 172.21.15.36:51672) 2022-06-17T12:46:56.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc86d66e700 1 -- 172.21.15.36:0/4088842193 learned_addr learned my addr 172.21.15.36:0/4088842193 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:56.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc86e670700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc86813d8c0 0x7fc868155e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59526/0 (socket says 172.21.15.36:59526) 2022-06-17T12:46:56.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc86d66e700 1 -- 172.21.15.36:0/4088842193 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8680b4f00 msgr2=0x7fc868140470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:56.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc86d66e700 1 --2- 172.21.15.36:0/4088842193 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8680b4f00 0x7fc868140470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:56.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc86d66e700 1 -- 172.21.15.36:0/4088842193 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc86813d8c0 msgr2=0x7fc868155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:56.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc86d66e700 1 --2- 172.21.15.36:0/4088842193 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc86813d8c0 0x7fc868155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:56.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc86d66e700 1 -- 172.21.15.36:0/4088842193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85c00b1e0 con 0x7fc8680b9860 2022-06-17T12:46:56.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc86e670700 1 --2- 172.21.15.36:0/4088842193 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc86813d8c0 0x7fc868155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:46:56.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc86d66e700 1 --2- 172.21.15.36:0/4088842193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8680b9860 0x7fc8680b8c00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fc85c000b60 tx=0x7fc85c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:56.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc85affd700 1 -- 172.21.15.36:0/4088842193 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc85c0068b0 con 0x7fc8680b9860 2022-06-17T12:46:56.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc85affd700 1 -- 172.21.15.36:0/4088842193 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85c002f50 con 0x7fc8680b9860 2022-06-17T12:46:56.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc86ee71700 1 -- 172.21.15.36:0/4088842193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86813ffd0 con 0x7fc8680b9860 2022-06-17T12:46:56.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc85affd700 1 -- 172.21.15.36:0/4088842193 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc85c011d10 con 0x7fc8680b9860 2022-06-17T12:46:56.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.310+0000 7fc86ee71700 1 -- 172.21.15.36:0/4088842193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8681562f0 con 0x7fc8680b9860 2022-06-17T12:46:56.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.314+0000 7fc86ee71700 1 -- 172.21.15.36:0/4088842193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc868140160 con 0x7fc8680b9860 2022-06-17T12:46:56.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.314+0000 7fc85affd700 1 -- 172.21.15.36:0/4088842193 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fc85c00f070 con 0x7fc8680b9860 2022-06-17T12:46:56.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.314+0000 7fc85affd700 1 --2- 172.21.15.36:0/4088842193 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc848053a40 0x7fc848055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:56.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.314+0000 7fc85affd700 1 -- 172.21.15.36:0/4088842193 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc85c06cc90 con 0x7fc8680b9860 2022-06-17T12:46:56.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.314+0000 7fc86de6f700 1 --2- 172.21.15.36:0/4088842193 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc848053a40 0x7fc848055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:56.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.314+0000 7fc86de6f700 1 --2- 172.21.15.36:0/4088842193 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc848053a40 0x7fc848055f00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fc86400d000 tx=0x7fc864009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:56.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.322+0000 7fc85affd700 1 -- 172.21.15.36:0/4088842193 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc85c043aa0 con 0x7fc8680b9860 2022-06-17T12:46:56.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.454+0000 7fc86ee71700 1 -- 172.21.15.36:0/4088842193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc868057410 con 0x7fc8680b9860 2022-06-17T12:46:56.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.454+0000 7fc85affd700 1 -- 172.21.15.36:0/4088842193 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc85c0470c0 con 0x7fc8680b9860 2022-06-17T12:46:56.494 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:46:56.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.458+0000 7fc86ee71700 1 -- 172.21.15.36:0/4088842193 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc848053a40 msgr2=0x7fc848055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:56.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.458+0000 7fc86ee71700 1 --2- 172.21.15.36:0/4088842193 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc848053a40 0x7fc848055f00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fc86400d000 tx=0x7fc864009040).stop 2022-06-17T12:46:56.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.458+0000 7fc86ee71700 1 -- 172.21.15.36:0/4088842193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8680b9860 msgr2=0x7fc8680b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:56.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.458+0000 7fc86ee71700 1 --2- 172.21.15.36:0/4088842193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8680b9860 0x7fc8680b8c00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fc85c000b60 tx=0x7fc85c015000).stop 2022-06-17T12:46:56.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.458+0000 7fc86ee71700 1 -- 172.21.15.36:0/4088842193 shutdown_connections 2022-06-17T12:46:56.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.458+0000 7fc86ee71700 1 --2- 172.21.15.36:0/4088842193 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc848053a40 0x7fc848055f00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:56.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.458+0000 7fc86ee71700 1 --2- 172.21.15.36:0/4088842193 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc86813d8c0 0x7fc868155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:56.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.458+0000 7fc86ee71700 1 --2- 172.21.15.36:0/4088842193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8680b9860 0x7fc8680b8c00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:56.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.458+0000 7fc86ee71700 1 --2- 172.21.15.36:0/4088842193 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8680b4f00 0x7fc868140470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:56.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.458+0000 7fc86ee71700 1 -- 172.21.15.36:0/4088842193 >> 172.21.15.36:0/4088842193 conn(0x7fc86805c550 msgr2=0x7fc868143b10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:56.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.458+0000 7fc86ee71700 1 -- 172.21.15.36:0/4088842193 shutdown_connections 2022-06-17T12:46:56.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.458+0000 7fc86ee71700 1 -- 172.21.15.36:0/4088842193 wait complete. 2022-06-17T12:46:56.508 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:46:56.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.642+0000 7ff97c564700 1 Processor -- start 2022-06-17T12:46:56.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.642+0000 7ff97c564700 1 -- start start 2022-06-17T12:46:56.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff97c564700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9740b2d80 0x7ff9740b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:56.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff97c564700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff974143e60 0x7ff974140300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:56.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff97c564700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9740aa170 con 0x7ff9740b2350 2022-06-17T12:46:56.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff97c564700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9740a9c30 con 0x7ff974143e60 2022-06-17T12:46:56.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff97c564700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9740a9ed0 con 0x7ff9740b2d80 2022-06-17T12:46:56.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff97ab01700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff974143e60 0x7ff974140300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:56.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff979aff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7ff9740b2350 0x7ff974143750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:48830/0 (socket says 172.21.15.36:48830) 2022-06-17T12:46:56.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff979aff700 1 -- 172.21.15.36:0/1070302874 learned_addr learned my addr 172.21.15.36:0/1070302874 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:46:56.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff97a300700 1 --2- 172.21.15.36:0/1070302874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9740b2d80 0x7ff9740b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:56.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff97ab01700 1 -- 172.21.15.36:0/1070302874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9740b2d80 msgr2=0x7ff9740b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:56.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff97ab01700 1 --2- 172.21.15.36:0/1070302874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9740b2d80 0x7ff9740b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:56.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff97ab01700 1 -- 172.21.15.36:0/1070302874 >> v1:172.21.15.36:6789/0 conn(0x7ff9740b2350 legacy=0x7ff974143750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:56.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff9792fe700 1 -- 172.21.15.36:0/1070302874 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4205514514 0 0) 0x7ff9740aa170 con 0x7ff9740b2350 2022-06-17T12:46:56.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff97ab01700 1 -- 172.21.15.36:0/1070302874 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9740b08c0 con 0x7ff974143e60 2022-06-17T12:46:56.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff97ab01700 1 --2- 172.21.15.36:0/1070302874 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff974143e60 0x7ff974140300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7ff96c0032f0 tx=0x7ff96c0091d0).ready entity=mon.0 client_cookie=3e18cec2a4fa505d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:56.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff9792fe700 1 -- 172.21.15.36:0/1070302874 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff96c00b6c0 con 0x7ff974143e60 2022-06-17T12:46:56.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff9792fe700 1 -- 172.21.15.36:0/1070302874 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96c00b820 con 0x7ff974143e60 2022-06-17T12:46:56.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff9792fe700 1 -- 172.21.15.36:0/1070302874 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff96c00b990 con 0x7ff974143e60 2022-06-17T12:46:56.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff97c564700 1 -- 172.21.15.36:0/1070302874 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff974143e60 msgr2=0x7ff974140300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:56.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff97c564700 1 --2- 172.21.15.36:0/1070302874 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff974143e60 0x7ff974140300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7ff96c0032f0 tx=0x7ff96c0091d0).stop 2022-06-17T12:46:56.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff97c564700 1 -- 172.21.15.36:0/1070302874 shutdown_connections 2022-06-17T12:46:56.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff97c564700 1 --2- 172.21.15.36:0/1070302874 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff974143e60 0x7ff974140300 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:56.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff97c564700 1 --2- 172.21.15.36:0/1070302874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9740b2d80 0x7ff9740b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:56.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff97c564700 1 -- 172.21.15.36:0/1070302874 >> 172.21.15.36:0/1070302874 conn(0x7ff97405c550 msgr2=0x7ff97405c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:56.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff97c564700 1 -- 172.21.15.36:0/1070302874 shutdown_connections 2022-06-17T12:46:56.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.646+0000 7ff97c564700 1 -- 172.21.15.36:0/1070302874 wait complete. 2022-06-17T12:46:56.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.650+0000 7ff97c564700 1 Processor -- start 2022-06-17T12:46:56.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.650+0000 7ff97c564700 1 -- start start 2022-06-17T12:46:56.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.650+0000 7ff97c564700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9740b2350 0x7ff974144ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:56.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.650+0000 7ff97c564700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff9740b2d80 0x7ff97413f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:56.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.650+0000 7ff97c564700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff974143e60 0x7ff97413fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:56.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.650+0000 7ff97c564700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9740acfb0 con 0x7ff9740b2d80 2022-06-17T12:46:56.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.650+0000 7ff97c564700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9740648c0 con 0x7ff974143e60 2022-06-17T12:46:56.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.650+0000 7ff97c564700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff974064620 con 0x7ff9740b2350 2022-06-17T12:46:56.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.650+0000 7ff979aff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff9740b2d80 0x7ff97413f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:56.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.650+0000 7ff979aff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff9740b2d80 0x7ff97413f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59540/0 (socket says 172.21.15.36:59540) 2022-06-17T12:46:56.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.650+0000 7ff979aff700 1 -- 172.21.15.36:0/856817706 learned_addr learned my addr 172.21.15.36:0/856817706 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:56.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.650+0000 7ff97a300700 1 --2- 172.21.15.36:0/856817706 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9740b2350 0x7ff974144ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:57.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.650+0000 7ff97ab01700 1 --2- 172.21.15.36:0/856817706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff974143e60 0x7ff97413fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:57.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.650+0000 7ff979aff700 1 -- 172.21.15.36:0/856817706 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9740b2350 msgr2=0x7ff974144ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:57.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.650+0000 7ff979aff700 1 --2- 172.21.15.36:0/856817706 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9740b2350 0x7ff974144ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:57.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.650+0000 7ff979aff700 1 -- 172.21.15.36:0/856817706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff974143e60 msgr2=0x7ff97413fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:57.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.650+0000 7ff979aff700 1 --2- 172.21.15.36:0/856817706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff974143e60 0x7ff97413fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:57.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.650+0000 7ff979aff700 1 -- 172.21.15.36:0/856817706 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96c009470 con 0x7ff9740b2d80 2022-06-17T12:46:57.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.650+0000 7ff979aff700 1 --2- 172.21.15.36:0/856817706 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff9740b2d80 0x7ff97413f910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7ff964000ed0 tx=0x7ff964007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:57.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.650+0000 7ff9637fe700 1 -- 172.21.15.36:0/856817706 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9640070d0 con 0x7ff9740b2d80 2022-06-17T12:46:57.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.650+0000 7ff9637fe700 1 -- 172.21.15.36:0/856817706 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff964007230 con 0x7ff9740b2d80 2022-06-17T12:46:57.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.650+0000 7ff97c564700 1 -- 172.21.15.36:0/856817706 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9740ab400 con 0x7ff9740b2d80 2022-06-17T12:46:57.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.650+0000 7ff9637fe700 1 -- 172.21.15.36:0/856817706 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff964005610 con 0x7ff9740b2d80 2022-06-17T12:46:57.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.650+0000 7ff97c564700 1 -- 172.21.15.36:0/856817706 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9740aa410 con 0x7ff9740b2d80 2022-06-17T12:46:57.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.654+0000 7ff9637fe700 1 -- 172.21.15.36:0/856817706 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff96401a060 con 0x7ff9740b2d80 2022-06-17T12:46:57.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.654+0000 7ff97c564700 1 -- 172.21.15.36:0/856817706 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9740ab140 con 0x7ff9740b2d80 2022-06-17T12:46:57.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.654+0000 7ff9637fe700 1 --2- 172.21.15.36:0/856817706 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff954053ad0 0x7ff954055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:57.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.654+0000 7ff9637fe700 1 -- 172.21.15.36:0/856817706 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff96406b7d0 con 0x7ff9740b2d80 2022-06-17T12:46:57.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.654+0000 7ff97a300700 1 --2- 172.21.15.36:0/856817706 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff954053ad0 0x7ff954055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:57.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.654+0000 7ff97a300700 1 --2- 172.21.15.36:0/856817706 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff954053ad0 0x7ff954055f90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7ff97000b000 tx=0x7ff970006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:57.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.658+0000 7ff9637fe700 1 -- 172.21.15.36:0/856817706 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff9740ab140 con 0x7ff9740b2d80 2022-06-17T12:46:57.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.781+0000 7ff97c564700 1 -- 172.21.15.36:0/856817706 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff9740ab140 con 0x7ff9740b2d80 2022-06-17T12:46:57.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.781+0000 7ff9637fe700 1 -- 172.21.15.36:0/856817706 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+10 (secure 0 0 0) 0x7ff9740ab140 con 0x7ff9740b2d80 2022-06-17T12:46:57.162 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:46:57.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.785+0000 7ff97c564700 1 -- 172.21.15.36:0/856817706 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff954053ad0 msgr2=0x7ff954055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:57.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.785+0000 7ff97c564700 1 --2- 172.21.15.36:0/856817706 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff954053ad0 0x7ff954055f90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7ff97000b000 tx=0x7ff970006df0).stop 2022-06-17T12:46:57.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.785+0000 7ff97c564700 1 -- 172.21.15.36:0/856817706 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff9740b2d80 msgr2=0x7ff97413f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:57.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.785+0000 7ff97c564700 1 --2- 172.21.15.36:0/856817706 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff9740b2d80 0x7ff97413f910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7ff964000ed0 tx=0x7ff964007ac0).stop 2022-06-17T12:46:57.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.785+0000 7ff97c564700 1 -- 172.21.15.36:0/856817706 shutdown_connections 2022-06-17T12:46:57.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.785+0000 7ff97c564700 1 --2- 172.21.15.36:0/856817706 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff954053ad0 0x7ff954055f90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:57.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.785+0000 7ff97c564700 1 --2- 172.21.15.36:0/856817706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff974143e60 0x7ff97413fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:57.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.785+0000 7ff97c564700 1 --2- 172.21.15.36:0/856817706 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff9740b2d80 0x7ff97413f910 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:57.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.785+0000 7ff97c564700 1 --2- 172.21.15.36:0/856817706 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9740b2350 0x7ff974144ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:57.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.785+0000 7ff97c564700 1 -- 172.21.15.36:0/856817706 >> 172.21.15.36:0/856817706 conn(0x7ff97405c550 msgr2=0x7ff974141380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:57.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.785+0000 7ff97c564700 1 -- 172.21.15.36:0/856817706 shutdown_connections 2022-06-17T12:46:57.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:56.785+0000 7ff97c564700 1 -- 172.21.15.36:0/856817706 wait complete. 2022-06-17T12:46:57.276 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:46:57.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.449+0000 7f89bac09700 1 Processor -- start 2022-06-17T12:46:57.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.449+0000 7f89bac09700 1 -- start start 2022-06-17T12:46:57.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.449+0000 7f89bac09700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f89b40b6860 0x7f89b4146d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:57.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.449+0000 7f89bac09700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89b40b18e0 0x7f89b4143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:57.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.449+0000 7f89bac09700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89b40b2b70 con 0x7f89b40b39a0 2022-06-17T12:46:57.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.449+0000 7f89bac09700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b40b6c40 con 0x7f89b40b6860 2022-06-17T12:46:57.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.449+0000 7f89bac09700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b4142d30 con 0x7f89b40b18e0 2022-06-17T12:46:57.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89ba408700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f89b40b39a0 0x7f89b4143640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:48846/0 (socket says 172.21.15.36:48846) 2022-06-17T12:46:57.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89ba408700 1 -- 172.21.15.36:0/474607434 learned_addr learned my addr 172.21.15.36:0/474607434 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:46:57.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89b9406700 1 --2- 172.21.15.36:0/474607434 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89b40b18e0 0x7f89b4143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:57.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89b9c07700 1 --2- 172.21.15.36:0/474607434 >> v2:172.21.15.70:3300/0 conn(0x7f89b40b6860 0x7f89b4146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:57.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89b8c05700 1 -- 172.21.15.36:0/474607434 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4267693205 0 0) 0x7f89b40b2b70 con 0x7f89b40b39a0 2022-06-17T12:46:57.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89b8c05700 1 -- 172.21.15.36:0/474607434 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f899c003260 con 0x7f89b40b39a0 2022-06-17T12:46:57.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89b8c05700 1 -- 172.21.15.36:0/474607434 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2914504864 0 0) 0x7f899c003260 con 0x7f89b40b39a0 2022-06-17T12:46:57.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89b8c05700 1 -- 172.21.15.36:0/474607434 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89b40b18e0 msgr2=0x7f89b4143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:57.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89b8c05700 1 --2- 172.21.15.36:0/474607434 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89b40b18e0 0x7f89b4143150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:57.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89b8c05700 1 -- 172.21.15.36:0/474607434 >> v2:172.21.15.70:3300/0 conn(0x7f89b40b6860 msgr2=0x7f89b4146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:57.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89b8c05700 1 --2- 172.21.15.36:0/474607434 >> v2:172.21.15.70:3300/0 conn(0x7f89b40b6860 0x7f89b4146d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:57.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89b8c05700 1 -- 172.21.15.36:0/474607434 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b40b0d30 con 0x7f89b40b39a0 2022-06-17T12:46:57.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89b8c05700 1 -- 172.21.15.36:0/474607434 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f89ac0019e0 con 0x7f89b40b39a0 2022-06-17T12:46:57.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89b8c05700 1 -- 172.21.15.36:0/474607434 >> v1:172.21.15.36:6789/0 conn(0x7f89b40b39a0 legacy=0x7f89b4143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:57.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89b8c05700 1 --2- 172.21.15.36:0/474607434 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89b40b18e0 0x7f899c003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:57.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89b8c05700 1 --2- 172.21.15.36:0/474607434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f899c003c80 0x7f899c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:57.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89b8c05700 1 --2- 172.21.15.36:0/474607434 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f899c0049f0 0x7f899c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:57.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89b8c05700 1 -- 172.21.15.36:0/474607434 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b40b6c40 con 0x7f899c0049f0 2022-06-17T12:46:57.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89b8c05700 1 -- 172.21.15.36:0/474607434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b4142d30 con 0x7f899c003c80 2022-06-17T12:46:57.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89b8c05700 1 -- 172.21.15.36:0/474607434 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899c00b750 con 0x7f89b40b18e0 2022-06-17T12:46:57.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89ba408700 1 --2- 172.21.15.36:0/474607434 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f899c0049f0 0x7f899c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:57.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89b9c07700 1 --2- 172.21.15.36:0/474607434 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89b40b18e0 0x7f899c003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:57.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89b9406700 1 --2- 172.21.15.36:0/474607434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f899c003c80 0x7f899c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:57.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89ba408700 1 -- 172.21.15.36:0/474607434 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89b40b18e0 msgr2=0x7f899c003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:57.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89ba408700 1 --2- 172.21.15.36:0/474607434 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89b40b18e0 0x7f899c003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:57.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89ba408700 1 -- 172.21.15.36:0/474607434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f899c003c80 msgr2=0x7f899c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:57.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89ba408700 1 --2- 172.21.15.36:0/474607434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f899c003c80 0x7f899c004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:57.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89ba408700 1 -- 172.21.15.36:0/474607434 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899c003260 con 0x7f899c0049f0 2022-06-17T12:46:57.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89ba408700 1 --2- 172.21.15.36:0/474607434 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f899c0049f0 0x7f899c004e70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f899c0054d0 tx=0x7f89b40b0ec0).ready entity=mon.0 client_cookie=4df740da0fa9a396 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:57.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89b8c05700 1 -- 172.21.15.36:0/474607434 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89ac00cd90 con 0x7f899c0049f0 2022-06-17T12:46:57.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89b8c05700 1 -- 172.21.15.36:0/474607434 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ac005990 con 0x7f899c0049f0 2022-06-17T12:46:57.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.453+0000 7f89b8c05700 1 -- 172.21.15.36:0/474607434 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89ac005cf0 con 0x7f899c0049f0 2022-06-17T12:46:57.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.457+0000 7f89bac09700 1 -- 172.21.15.36:0/474607434 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f899c0049f0 msgr2=0x7f899c004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:57.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.457+0000 7f89bac09700 1 --2- 172.21.15.36:0/474607434 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f899c0049f0 0x7f899c004e70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f899c0054d0 tx=0x7f89b40b0ec0).stop 2022-06-17T12:46:57.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.457+0000 7f89ba408700 1 -- 172.21.15.36:0/474607434 reap_dead start 2022-06-17T12:46:57.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.457+0000 7f89bac09700 1 -- 172.21.15.36:0/474607434 shutdown_connections 2022-06-17T12:46:57.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.457+0000 7f89bac09700 1 -- 172.21.15.36:0/474607434 >> 172.21.15.36:0/474607434 conn(0x7f89b40614b0 msgr2=0x7f89b4061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:57.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.457+0000 7f89bac09700 1 -- 172.21.15.36:0/474607434 shutdown_connections 2022-06-17T12:46:57.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.457+0000 7f89bac09700 1 -- 172.21.15.36:0/474607434 wait complete. 2022-06-17T12:46:57.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.457+0000 7f89bac09700 1 Processor -- start 2022-06-17T12:46:57.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.457+0000 7f89bac09700 1 -- start start 2022-06-17T12:46:57.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.457+0000 7f89bac09700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f89b40639c0 0x7f89b41558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:57.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.457+0000 7f89bac09700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89b4113cc0 0x7f89b41517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:57.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.457+0000 7f89bac09700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89b4151ca0 0x7f89b4116670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:57.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.457+0000 7f89bac09700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b4066380 con 0x7f89b40639c0 2022-06-17T12:46:57.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.457+0000 7f89bac09700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b4064740 con 0x7f89b4151ca0 2022-06-17T12:46:57.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.457+0000 7f89bac09700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b40665b0 con 0x7f89b4113cc0 2022-06-17T12:46:57.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.457+0000 7f89b9c07700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f89b40639c0 0x7f89b41558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:57.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.461+0000 7f89b9c07700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f89b40639c0 0x7f89b41558c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59558/0 (socket says 172.21.15.36:59558) 2022-06-17T12:46:57.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.461+0000 7f89b9c07700 1 -- 172.21.15.36:0/1725765643 learned_addr learned my addr 172.21.15.36:0/1725765643 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:57.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.461+0000 7f89b9406700 1 --2- 172.21.15.36:0/1725765643 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89b4113cc0 0x7f89b41517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:57.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.461+0000 7f89ba408700 1 --2- 172.21.15.36:0/1725765643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89b4151ca0 0x7f89b4116670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:57.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.461+0000 7f89b9c07700 1 -- 172.21.15.36:0/1725765643 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89b4113cc0 msgr2=0x7f89b41517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:57.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.461+0000 7f89b9c07700 1 --2- 172.21.15.36:0/1725765643 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89b4113cc0 0x7f89b41517b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:57.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.461+0000 7f89b9c07700 1 -- 172.21.15.36:0/1725765643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89b4151ca0 msgr2=0x7f89b4116670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:57.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.461+0000 7f89b9c07700 1 --2- 172.21.15.36:0/1725765643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89b4151ca0 0x7f89b4116670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:57.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.461+0000 7f89b9c07700 1 -- 172.21.15.36:0/1725765643 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b4066850 con 0x7f89b40639c0 2022-06-17T12:46:57.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.461+0000 7f89b9406700 1 --2- 172.21.15.36:0/1725765643 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89b4113cc0 0x7f89b41517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:46:57.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.461+0000 7f89ba408700 1 --2- 172.21.15.36:0/1725765643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89b4151ca0 0x7f89b4116670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:46:57.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.461+0000 7f89b9c07700 1 --2- 172.21.15.36:0/1725765643 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f89b40639c0 0x7f89b41558c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f89b000b000 tx=0x7f89b0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:57.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.461+0000 7f89a2ffd700 1 -- 172.21.15.36:0/1725765643 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89b000f730 con 0x7f89b40639c0 2022-06-17T12:46:57.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.461+0000 7f89a2ffd700 1 -- 172.21.15.36:0/1725765643 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b000f890 con 0x7f89b40639c0 2022-06-17T12:46:57.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.461+0000 7f89a2ffd700 1 -- 172.21.15.36:0/1725765643 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89b000fbf0 con 0x7f89b40639c0 2022-06-17T12:46:57.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.461+0000 7f89bac09700 1 -- 172.21.15.36:0/1725765643 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b4152120 con 0x7f89b40639c0 2022-06-17T12:46:57.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.461+0000 7f89bac09700 1 -- 172.21.15.36:0/1725765643 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b4063da0 con 0x7f89b40639c0 2022-06-17T12:46:57.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.461+0000 7f89a2ffd700 1 -- 172.21.15.36:0/1725765643 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f89b000fd50 con 0x7f89b40639c0 2022-06-17T12:46:57.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.461+0000 7f89bac09700 1 -- 172.21.15.36:0/1725765643 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b4057410 con 0x7f89b40639c0 2022-06-17T12:46:57.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.461+0000 7f89a2ffd700 1 --2- 172.21.15.36:0/1725765643 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8994053af0 0x7f8994055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:57.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.461+0000 7f89b9406700 1 --2- 172.21.15.36:0/1725765643 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8994053af0 0x7f8994055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:57.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.461+0000 7f89a2ffd700 1 -- 172.21.15.36:0/1725765643 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f89b006c9d0 con 0x7f89b40639c0 2022-06-17T12:46:57.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.465+0000 7f89b9406700 1 --2- 172.21.15.36:0/1725765643 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8994053af0 0x7f8994055fb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f89a4006000 tx=0x7f89a4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:57.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.469+0000 7f89a2ffd700 1 -- 172.21.15.36:0/1725765643 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f89b0043660 con 0x7f89b40639c0 2022-06-17T12:46:57.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.601+0000 7f89bac09700 1 -- 172.21.15.36:0/1725765643 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f89b40b0960 con 0x7f89b40639c0 2022-06-17T12:46:57.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.601+0000 7f89a2ffd700 1 -- 172.21.15.36:0/1725765643 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+10 (secure 0 0 0) 0x7f89b0018020 con 0x7f89b40639c0 2022-06-17T12:46:57.615 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:46:57.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.601+0000 7f89bac09700 1 -- 172.21.15.36:0/1725765643 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8994053af0 msgr2=0x7f8994055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:57.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.601+0000 7f89bac09700 1 --2- 172.21.15.36:0/1725765643 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8994053af0 0x7f8994055fb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f89a4006000 tx=0x7f89a4008040).stop 2022-06-17T12:46:57.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.601+0000 7f89bac09700 1 -- 172.21.15.36:0/1725765643 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f89b40639c0 msgr2=0x7f89b41558c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:57.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.601+0000 7f89bac09700 1 --2- 172.21.15.36:0/1725765643 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f89b40639c0 0x7f89b41558c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f89b000b000 tx=0x7f89b0006df0).stop 2022-06-17T12:46:57.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.601+0000 7f89bac09700 1 -- 172.21.15.36:0/1725765643 shutdown_connections 2022-06-17T12:46:57.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.601+0000 7f89bac09700 1 --2- 172.21.15.36:0/1725765643 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8994053af0 0x7f8994055fb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:57.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.601+0000 7f89bac09700 1 --2- 172.21.15.36:0/1725765643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89b4151ca0 0x7f89b4116670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:57.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.601+0000 7f89bac09700 1 --2- 172.21.15.36:0/1725765643 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89b4113cc0 0x7f89b41517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:57.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.601+0000 7f89bac09700 1 --2- 172.21.15.36:0/1725765643 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f89b40639c0 0x7f89b41558c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:57.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.601+0000 7f89bac09700 1 -- 172.21.15.36:0/1725765643 >> 172.21.15.36:0/1725765643 conn(0x7f89b40614b0 msgr2=0x7f89b40ae510 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:57.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.601+0000 7f89bac09700 1 -- 172.21.15.36:0/1725765643 shutdown_connections 2022-06-17T12:46:57.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.601+0000 7f89bac09700 1 -- 172.21.15.36:0/1725765643 wait complete. 2022-06-17T12:46:57.687 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2022-06-17T12:46:57.688 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2022-06-17T12:46:57.689 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2022-06-17T12:46:57.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.821+0000 7f95b2f8a700 1 Processor -- start 2022-06-17T12:46:57.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.821+0000 7f95b2f8a700 1 -- start start 2022-06-17T12:46:57.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.821+0000 7f95b2f8a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f95ac13cb70 0x7f95ac143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:57.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.821+0000 7f95b2f8a700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ac0af4b0 con 0x7f95ac13cf50 2022-06-17T12:46:57.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.821+0000 7f95b2f8a700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ac0ba500 con 0x7f95ac0aef60 2022-06-17T12:46:57.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.821+0000 7f95b2f8a700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ac142f50 con 0x7f95ac13cb70 2022-06-17T12:46:57.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.821+0000 7f95b1f88700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f95ac13cb70 0x7f95ac143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:57.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.821+0000 7f95b2789700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f95ac13cf50 0x7f95ac1442a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:48864/0 (socket says 172.21.15.36:48864) 2022-06-17T12:46:57.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.821+0000 7f95b2789700 1 -- 172.21.15.36:0/4103435808 learned_addr learned my addr 172.21.15.36:0/4103435808 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:46:57.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.821+0000 7f95b0f86700 1 -- 172.21.15.36:0/4103435808 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 921453312 0 0) 0x7f95ac0af4b0 con 0x7f95ac13cf50 2022-06-17T12:46:57.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.825+0000 7f95b1f88700 1 -- 172.21.15.36:0/4103435808 >> v1:172.21.15.70:6789/0 conn(0x7f95ac0aef60 legacy=0x7f95ac0b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:57.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.825+0000 7f95b1f88700 1 -- 172.21.15.36:0/4103435808 >> v1:172.21.15.36:6789/0 conn(0x7f95ac13cf50 legacy=0x7f95ac1442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:57.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.825+0000 7f95b1f88700 1 -- 172.21.15.36:0/4103435808 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac1431f0 con 0x7f95ac13cb70 2022-06-17T12:46:57.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.825+0000 7f95b1f88700 1 --2- 172.21.15.36:0/4103435808 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95ac13cb70 0x7f95ac143690 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f95a8002f70 tx=0x7f95a8006df0).ready entity=mon.0 client_cookie=87c2e92a7baa7c96 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:57.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.825+0000 7f95b0f86700 1 -- 172.21.15.36:0/4103435808 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95a800d610 con 0x7f95ac13cb70 2022-06-17T12:46:57.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.825+0000 7f95b0f86700 1 -- 172.21.15.36:0/4103435808 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a8000e90 con 0x7f95ac13cb70 2022-06-17T12:46:57.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.825+0000 7f95b0f86700 1 -- 172.21.15.36:0/4103435808 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95a800d960 con 0x7f95ac13cb70 2022-06-17T12:46:57.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.825+0000 7f95b2f8a700 1 -- 172.21.15.36:0/4103435808 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95ac13cb70 msgr2=0x7f95ac143690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:57.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.825+0000 7f95b2f8a700 1 --2- 172.21.15.36:0/4103435808 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95ac13cb70 0x7f95ac143690 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f95a8002f70 tx=0x7f95a8006df0).stop 2022-06-17T12:46:57.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.825+0000 7f95b2f8a700 1 -- 172.21.15.36:0/4103435808 shutdown_connections 2022-06-17T12:46:57.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.825+0000 7f95b2f8a700 1 --2- 172.21.15.36:0/4103435808 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95ac13cb70 0x7f95ac143690 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:57.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.825+0000 7f95b2f8a700 1 -- 172.21.15.36:0/4103435808 >> 172.21.15.36:0/4103435808 conn(0x7f95ac05c550 msgr2=0x7f95ac0b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:57.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.825+0000 7f95b2f8a700 1 -- 172.21.15.36:0/4103435808 shutdown_connections 2022-06-17T12:46:58.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.825+0000 7f95b2f8a700 1 -- 172.21.15.36:0/4103435808 wait complete. 2022-06-17T12:46:58.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.829+0000 7f95b2f8a700 1 Processor -- start 2022-06-17T12:46:58.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.829+0000 7f95b2f8a700 1 -- start start 2022-06-17T12:46:58.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.829+0000 7f95b2f8a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95ac0aef60 0x7f95ac0b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:58.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.829+0000 7f95b2f8a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ac13cb70 0x7f95ac13db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:58.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.829+0000 7f95b2f8a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95ac13cf50 0x7f95ac0b0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:58.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.829+0000 7f95b2f8a700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac13ec90 con 0x7f95ac13cf50 2022-06-17T12:46:58.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.829+0000 7f95b2f8a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac13ef30 con 0x7f95ac13cb70 2022-06-17T12:46:58.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.829+0000 7f95b2f8a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac13e060 con 0x7f95ac0aef60 2022-06-17T12:46:58.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.829+0000 7f95b2789700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95ac13cf50 0x7f95ac0b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:58.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.829+0000 7f95b2789700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95ac13cf50 0x7f95ac0b0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59572/0 (socket says 172.21.15.36:59572) 2022-06-17T12:46:58.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.829+0000 7f95b2789700 1 -- 172.21.15.36:0/1023054062 learned_addr learned my addr 172.21.15.36:0/1023054062 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:58.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.829+0000 7f95b1787700 1 --2- 172.21.15.36:0/1023054062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ac13cb70 0x7f95ac13db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:58.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.829+0000 7f95b1f88700 1 --2- 172.21.15.36:0/1023054062 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95ac0aef60 0x7f95ac0b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:58.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.829+0000 7f95b2789700 1 -- 172.21.15.36:0/1023054062 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95ac0aef60 msgr2=0x7f95ac0b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:58.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.829+0000 7f95b2789700 1 --2- 172.21.15.36:0/1023054062 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95ac0aef60 0x7f95ac0b51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:58.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.829+0000 7f95b2789700 1 -- 172.21.15.36:0/1023054062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ac13cb70 msgr2=0x7f95ac13db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:58.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.829+0000 7f95b2789700 1 --2- 172.21.15.36:0/1023054062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ac13cb70 0x7f95ac13db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:58.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.829+0000 7f95b2789700 1 -- 172.21.15.36:0/1023054062 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a800b180 con 0x7f95ac13cf50 2022-06-17T12:46:58.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.829+0000 7f95b2789700 1 --2- 172.21.15.36:0/1023054062 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95ac13cf50 0x7f95ac0b0990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f95a4001250 tx=0x7f95a4001360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:58.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.829+0000 7f959affd700 1 -- 172.21.15.36:0/1023054062 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95a4005ba0 con 0x7f95ac13cf50 2022-06-17T12:46:58.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.829+0000 7f959affd700 1 -- 172.21.15.36:0/1023054062 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a4005d00 con 0x7f95ac13cf50 2022-06-17T12:46:58.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.829+0000 7f95b2f8a700 1 -- 172.21.15.36:0/1023054062 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ac0b5c70 con 0x7f95ac13cf50 2022-06-17T12:46:58.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.829+0000 7f959affd700 1 -- 172.21.15.36:0/1023054062 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95a4004a90 con 0x7f95ac13cf50 2022-06-17T12:46:58.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.829+0000 7f95b2f8a700 1 -- 172.21.15.36:0/1023054062 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac0b1020 con 0x7f95ac13cf50 2022-06-17T12:46:58.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.833+0000 7f95b2f8a700 1 -- 172.21.15.36:0/1023054062 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ac13f2b0 con 0x7f95ac13cf50 2022-06-17T12:46:58.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.833+0000 7f959affd700 1 -- 172.21.15.36:0/1023054062 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f95a4004bf0 con 0x7f95ac13cf50 2022-06-17T12:46:58.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.833+0000 7f959affd700 1 --2- 172.21.15.36:0/1023054062 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f958c053af0 0x7f958c055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:58.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.833+0000 7f959affd700 1 -- 172.21.15.36:0/1023054062 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f95a406bab0 con 0x7f95ac13cf50 2022-06-17T12:46:58.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.833+0000 7f95b1f88700 1 --2- 172.21.15.36:0/1023054062 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f958c053af0 0x7f958c055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:58.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.833+0000 7f95b1f88700 1 --2- 172.21.15.36:0/1023054062 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f958c053af0 0x7f958c055fb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f95a801c000 tx=0x7f95a8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:58.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.841+0000 7f959affd700 1 -- 172.21.15.36:0/1023054062 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f95a40428b0 con 0x7f95ac13cf50 2022-06-17T12:46:58.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.989+0000 7f95b2f8a700 1 -- 172.21.15.36:0/1023054062 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f95ac1449b0 con 0x7f95ac13cf50 2022-06-17T12:46:58.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.993+0000 7f959affd700 1 -- 172.21.15.36:0/1023054062 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+131705 (secure 0 0 0) 0x7f95a4045ed0 con 0x7f95ac13cf50 2022-06-17T12:46:58.213 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:46:58.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.997+0000 7f95b2f8a700 1 -- 172.21.15.36:0/1023054062 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f958c053af0 msgr2=0x7f958c055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:58.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.997+0000 7f95b2f8a700 1 --2- 172.21.15.36:0/1023054062 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f958c053af0 0x7f958c055fb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f95a801c000 tx=0x7f95a8015000).stop 2022-06-17T12:46:58.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.997+0000 7f95b2f8a700 1 -- 172.21.15.36:0/1023054062 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95ac13cf50 msgr2=0x7f95ac0b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:58.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.997+0000 7f95b2f8a700 1 --2- 172.21.15.36:0/1023054062 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95ac13cf50 0x7f95ac0b0990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f95a4001250 tx=0x7f95a4001360).stop 2022-06-17T12:46:58.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.997+0000 7f95b2f8a700 1 -- 172.21.15.36:0/1023054062 shutdown_connections 2022-06-17T12:46:58.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.997+0000 7f95b2f8a700 1 --2- 172.21.15.36:0/1023054062 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f958c053af0 0x7f958c055fb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:58.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.997+0000 7f95b2f8a700 1 --2- 172.21.15.36:0/1023054062 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95ac13cf50 0x7f95ac0b0990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:58.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.997+0000 7f95b2f8a700 1 --2- 172.21.15.36:0/1023054062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ac13cb70 0x7f95ac13db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:58.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.997+0000 7f95b2f8a700 1 --2- 172.21.15.36:0/1023054062 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95ac0aef60 0x7f95ac0b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:58.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.997+0000 7f95b2f8a700 1 -- 172.21.15.36:0/1023054062 >> 172.21.15.36:0/1023054062 conn(0x7f95ac05c550 msgr2=0x7f95ac13a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:58.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.997+0000 7f95b2f8a700 1 -- 172.21.15.36:0/1023054062 shutdown_connections 2022-06-17T12:46:58.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:57.997+0000 7f95b2f8a700 1 -- 172.21.15.36:0/1023054062 wait complete. 2022-06-17T12:46:58.310 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":4,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6800","nonce":22419},{"type":"v1","addr":"172.21.15.36:6801","nonce":22419}]},"active_addr":"172.21.15.36:6801/22419","active_change":"2022-06-17T12:46:33.480535+0000","active_mgr_features":4540138292840890367,"available":true,"standbys":[{"gid":4103,"name":"x","mgr_features":4540138292840890367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.36:0","nonce":1164310211}]},{"addrvec":[{"type":"v2","addr":"172.21.15.36:0","nonce":4263516711}]}]}} 2022-06-17T12:46:58.352 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2022-06-17T12:46:58.353 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2022-06-17T12:46:58.353 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:46:58.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.505+0000 7f5a7defb700 1 Processor -- start 2022-06-17T12:46:58.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.505+0000 7f5a7defb700 1 -- start start 2022-06-17T12:46:58.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.505+0000 7f5a7defb700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5a780ad9c0 0x7f5a780b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:58.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.505+0000 7f5a7defb700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a780afc30 con 0x7f5a7813fde0 2022-06-17T12:46:58.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.505+0000 7f5a7defb700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a780ae650 con 0x7f5a780ae270 2022-06-17T12:46:58.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.505+0000 7f5a7defb700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a780b0cb0 con 0x7f5a780ad9c0 2022-06-17T12:46:58.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.505+0000 7f5a76ffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5a780ad9c0 0x7f5a780b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:58.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.505+0000 7f5a76ffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5a780ad9c0 0x7f5a780b39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59576/0 (socket says 172.21.15.36:59576) 2022-06-17T12:46:58.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.505+0000 7f5a76ffd700 1 -- 172.21.15.36:0/1199789792 learned_addr learned my addr 172.21.15.36:0/1199789792 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:58.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.505+0000 7f5a77fff700 1 --1- 172.21.15.36:0/1199789792 >> v1:172.21.15.36:6789/0 conn(0x7f5a7813fde0 0x7f5a7813edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:48878/0 (socket says 172.21.15.36:48878) 2022-06-17T12:46:58.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.505+0000 7f5a76ffd700 1 -- 172.21.15.36:0/1199789792 >> v1:172.21.15.70:6789/0 conn(0x7f5a780ae270 legacy=0x7f5a780b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:58.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.505+0000 7f5a76ffd700 1 -- 172.21.15.36:0/1199789792 >> v1:172.21.15.36:6789/0 conn(0x7f5a7813fde0 legacy=0x7f5a7813edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:58.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.505+0000 7f5a76ffd700 1 -- 172.21.15.36:0/1199789792 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a780afe60 con 0x7f5a780ad9c0 2022-06-17T12:46:58.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.505+0000 7f5a767fc700 1 -- 172.21.15.36:0/1199789792 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 209438143 0 0) 0x7f5a780afc30 con 0x7f5a7813fde0 2022-06-17T12:46:58.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.505+0000 7f5a76ffd700 1 --2- 172.21.15.36:0/1199789792 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a780ad9c0 0x7f5a780b39c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f5a78062470 tx=0x7f5a60006eb0).ready entity=mon.0 client_cookie=f4f518423a46d820 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:58.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.505+0000 7f5a767fc700 1 -- 172.21.15.36:0/1199789792 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a6000d610 con 0x7f5a780ad9c0 2022-06-17T12:46:58.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.505+0000 7f5a767fc700 1 -- 172.21.15.36:0/1199789792 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a60000e90 con 0x7f5a780ad9c0 2022-06-17T12:46:58.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.505+0000 7f5a767fc700 1 -- 172.21.15.36:0/1199789792 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a6000d960 con 0x7f5a780ad9c0 2022-06-17T12:46:58.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.509+0000 7f5a7defb700 1 -- 172.21.15.36:0/1199789792 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a780ad9c0 msgr2=0x7f5a780b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:58.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.509+0000 7f5a7defb700 1 --2- 172.21.15.36:0/1199789792 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a780ad9c0 0x7f5a780b39c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f5a78062470 tx=0x7f5a60006eb0).stop 2022-06-17T12:46:58.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.509+0000 7f5a7defb700 1 -- 172.21.15.36:0/1199789792 shutdown_connections 2022-06-17T12:46:58.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.509+0000 7f5a7defb700 1 --2- 172.21.15.36:0/1199789792 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a780ad9c0 0x7f5a780b39c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:58.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.509+0000 7f5a7defb700 1 -- 172.21.15.36:0/1199789792 >> 172.21.15.36:0/1199789792 conn(0x7f5a7805c550 msgr2=0x7f5a7805c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:58.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.509+0000 7f5a7defb700 1 -- 172.21.15.36:0/1199789792 shutdown_connections 2022-06-17T12:46:58.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.509+0000 7f5a7defb700 1 -- 172.21.15.36:0/1199789792 wait complete. 2022-06-17T12:46:58.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.509+0000 7f5a7defb700 1 Processor -- start 2022-06-17T12:46:58.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.509+0000 7f5a7defb700 1 -- start start 2022-06-17T12:46:58.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.509+0000 7f5a7defb700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5a780ad9c0 0x7f5a780663e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:58.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.509+0000 7f5a7defb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a780ae270 0x7f5a78145430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:58.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.509+0000 7f5a7defb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a7813fde0 0x7f5a78152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:58.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.509+0000 7f5a7defb700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a78143cc0 con 0x7f5a7813fde0 2022-06-17T12:46:58.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.509+0000 7f5a7defb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7813b520 con 0x7f5a780ae270 2022-06-17T12:46:58.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.509+0000 7f5a7defb700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a78143ef0 con 0x7f5a780ad9c0 2022-06-17T12:46:58.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.513+0000 7f5a77fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a7813fde0 0x7f5a78152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:58.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.513+0000 7f5a77fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a7813fde0 0x7f5a78152970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59586/0 (socket says 172.21.15.36:59586) 2022-06-17T12:46:58.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.513+0000 7f5a77fff700 1 -- 172.21.15.36:0/1802299830 learned_addr learned my addr 172.21.15.36:0/1802299830 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:58.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.513+0000 7f5a76ffd700 1 --2- 172.21.15.36:0/1802299830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a780ae270 0x7f5a78145430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:58.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.513+0000 7f5a777fe700 1 --2- 172.21.15.36:0/1802299830 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5a780ad9c0 0x7f5a780663e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:58.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.513+0000 7f5a77fff700 1 -- 172.21.15.36:0/1802299830 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5a780ad9c0 msgr2=0x7f5a780663e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:58.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.513+0000 7f5a77fff700 1 --2- 172.21.15.36:0/1802299830 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5a780ad9c0 0x7f5a780663e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:58.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.513+0000 7f5a77fff700 1 -- 172.21.15.36:0/1802299830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a780ae270 msgr2=0x7f5a78145430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:58.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.513+0000 7f5a77fff700 1 --2- 172.21.15.36:0/1802299830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a780ae270 0x7f5a78145430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:58.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.513+0000 7f5a77fff700 1 -- 172.21.15.36:0/1802299830 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6000b1e0 con 0x7f5a7813fde0 2022-06-17T12:46:58.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.513+0000 7f5a77fff700 1 --2- 172.21.15.36:0/1802299830 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a7813fde0 0x7f5a78152970 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f5a70001250 tx=0x7f5a70001360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:58.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.513+0000 7f5a74ff9700 1 -- 172.21.15.36:0/1802299830 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a70005ae0 con 0x7f5a7813fde0 2022-06-17T12:46:58.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.513+0000 7f5a74ff9700 1 -- 172.21.15.36:0/1802299830 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a70005c40 con 0x7f5a7813fde0 2022-06-17T12:46:58.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.513+0000 7f5a74ff9700 1 -- 172.21.15.36:0/1802299830 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a700070d0 con 0x7f5a7813fde0 2022-06-17T12:46:58.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.513+0000 7f5a7defb700 1 -- 172.21.15.36:0/1802299830 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a78144190 con 0x7f5a7813fde0 2022-06-17T12:46:58.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.513+0000 7f5a7defb700 1 -- 172.21.15.36:0/1802299830 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a78152260 con 0x7f5a7813fde0 2022-06-17T12:46:58.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.513+0000 7f5a7defb700 1 -- 172.21.15.36:0/1802299830 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a780b0100 con 0x7f5a7813fde0 2022-06-17T12:46:58.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.513+0000 7f5a74ff9700 1 -- 172.21.15.36:0/1802299830 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f5a70010070 con 0x7f5a7813fde0 2022-06-17T12:46:58.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.517+0000 7f5a74ff9700 1 --2- 172.21.15.36:0/1802299830 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5a5c053af0 0x7f5a5c055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:58.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.517+0000 7f5a74ff9700 1 -- 172.21.15.36:0/1802299830 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5a7006b890 con 0x7f5a7813fde0 2022-06-17T12:46:58.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.517+0000 7f5a777fe700 1 --2- 172.21.15.36:0/1802299830 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5a5c053af0 0x7f5a5c055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:58.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.517+0000 7f5a777fe700 1 --2- 172.21.15.36:0/1802299830 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5a5c053af0 0x7f5a5c055fb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f5a6800d000 tx=0x7f5a68007b70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:58.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.525+0000 7f5a74ff9700 1 -- 172.21.15.36:0/1802299830 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5a70042690 con 0x7f5a7813fde0 2022-06-17T12:46:58.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.657+0000 7f5a7defb700 1 -- 172.21.15.36:0/1802299830 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a7805f220 con 0x7f5a7813fde0 2022-06-17T12:46:58.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.657+0000 7f5a74ff9700 1 -- 172.21.15.36:0/1802299830 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f5a70045cb0 con 0x7f5a7813fde0 2022-06-17T12:46:58.704 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:46:58.705 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":26,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:46:53.258075+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:46:47.465328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-17T12:46:49.893338+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:46:58.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.661+0000 7f5a7defb700 1 -- 172.21.15.36:0/1802299830 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5a5c053af0 msgr2=0x7f5a5c055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:58.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.661+0000 7f5a7defb700 1 --2- 172.21.15.36:0/1802299830 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5a5c053af0 0x7f5a5c055fb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f5a6800d000 tx=0x7f5a68007b70).stop 2022-06-17T12:46:58.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.661+0000 7f5a7defb700 1 -- 172.21.15.36:0/1802299830 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a7813fde0 msgr2=0x7f5a78152970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:58.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.661+0000 7f5a7defb700 1 --2- 172.21.15.36:0/1802299830 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a7813fde0 0x7f5a78152970 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f5a70001250 tx=0x7f5a70001360).stop 2022-06-17T12:46:58.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.661+0000 7f5a7defb700 1 -- 172.21.15.36:0/1802299830 shutdown_connections 2022-06-17T12:46:58.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.661+0000 7f5a7defb700 1 --2- 172.21.15.36:0/1802299830 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5a5c053af0 0x7f5a5c055fb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:58.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.661+0000 7f5a7defb700 1 --2- 172.21.15.36:0/1802299830 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a7813fde0 0x7f5a78152970 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:58.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.661+0000 7f5a7defb700 1 --2- 172.21.15.36:0/1802299830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a780ae270 0x7f5a78145430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:58.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.661+0000 7f5a7defb700 1 --2- 172.21.15.36:0/1802299830 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5a780ad9c0 0x7f5a780663e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:58.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.661+0000 7f5a7defb700 1 -- 172.21.15.36:0/1802299830 >> 172.21.15.36:0/1802299830 conn(0x7f5a7805c550 msgr2=0x7f5a780b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:58.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.661+0000 7f5a7defb700 1 -- 172.21.15.36:0/1802299830 shutdown_connections 2022-06-17T12:46:58.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.661+0000 7f5a7defb700 1 -- 172.21.15.36:0/1802299830 wait complete. 2022-06-17T12:46:58.721 INFO:tasks.ceph.ceph_manager.ceph:all up! 2022-06-17T12:46:58.730 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:46:58.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.877+0000 7feacffd0700 1 Processor -- start 2022-06-17T12:46:58.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.877+0000 7feacffd0700 1 -- start start 2022-06-17T12:46:58.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.877+0000 7feacffd0700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feac80b7b60 0x7feac80b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:58.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.877+0000 7feacffd0700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feac80616c0 0x7feac813cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:58.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.877+0000 7feacffd0700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac80b7f40 con 0x7feac8061eb0 2022-06-17T12:46:58.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.877+0000 7feacffd0700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac80b3670 con 0x7feac80b7b60 2022-06-17T12:46:58.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.877+0000 7feacffd0700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac81433b0 con 0x7feac80616c0 2022-06-17T12:46:58.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.877+0000 7feace56d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feac80616c0 0x7feac813cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:58.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.877+0000 7feacd56b700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7feac8061eb0 0x7feac8143af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:44622/0 (socket says 172.21.15.36:44622) 2022-06-17T12:46:58.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.877+0000 7feacd56b700 1 -- 172.21.15.36:0/1487611198 learned_addr learned my addr 172.21.15.36:0/1487611198 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:46:58.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.877+0000 7feacdd6c700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feac80b7b60 0x7feac80b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:58.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.877+0000 7feaccd6a700 1 -- 172.21.15.36:0/1487611198 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2509389775 0 0) 0x7feac80b7f40 con 0x7feac8061eb0 2022-06-17T12:46:58.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.877+0000 7feace56d700 1 -- 172.21.15.36:0/1487611198 >> v2:172.21.15.70:3300/0 conn(0x7feac80b7b60 msgr2=0x7feac80b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:58.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.877+0000 7feace56d700 1 --2- 172.21.15.36:0/1487611198 >> v2:172.21.15.70:3300/0 conn(0x7feac80b7b60 0x7feac80b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:58.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.877+0000 7feace56d700 1 -- 172.21.15.36:0/1487611198 >> v1:172.21.15.70:6789/0 conn(0x7feac8061eb0 legacy=0x7feac8143af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:58.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.877+0000 7feace56d700 1 -- 172.21.15.36:0/1487611198 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac8143650 con 0x7feac80616c0 2022-06-17T12:46:58.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.881+0000 7feace56d700 1 --2- 172.21.15.36:0/1487611198 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feac80616c0 0x7feac813cd30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7feac00032f0 tx=0x7feac00091d0).ready entity=mon.2 client_cookie=b9080b259a0a3453 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:58.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.881+0000 7feaccd6a700 1 -- 172.21.15.36:0/1487611198 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feac000b6c0 con 0x7feac80616c0 2022-06-17T12:46:58.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.881+0000 7feaccd6a700 1 -- 172.21.15.36:0/1487611198 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac000b820 con 0x7feac80616c0 2022-06-17T12:46:58.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.881+0000 7feaccd6a700 1 -- 172.21.15.36:0/1487611198 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feac000b990 con 0x7feac80616c0 2022-06-17T12:46:58.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.881+0000 7feacffd0700 1 -- 172.21.15.36:0/1487611198 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feac80616c0 msgr2=0x7feac813cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:58.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.881+0000 7feacffd0700 1 --2- 172.21.15.36:0/1487611198 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feac80616c0 0x7feac813cd30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7feac00032f0 tx=0x7feac00091d0).stop 2022-06-17T12:46:58.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.881+0000 7feacffd0700 1 -- 172.21.15.36:0/1487611198 shutdown_connections 2022-06-17T12:46:58.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.881+0000 7feacffd0700 1 --2- 172.21.15.36:0/1487611198 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feac80616c0 0x7feac813cd30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:58.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.881+0000 7feacffd0700 1 --2- 172.21.15.36:0/1487611198 >> v2:172.21.15.70:3300/0 conn(0x7feac80b7b60 0x7feac80b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:58.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.881+0000 7feacffd0700 1 -- 172.21.15.36:0/1487611198 >> 172.21.15.36:0/1487611198 conn(0x7feac8060c60 msgr2=0x7feac805c550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:58.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.881+0000 7feacffd0700 1 -- 172.21.15.36:0/1487611198 shutdown_connections 2022-06-17T12:46:58.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.881+0000 7feacffd0700 1 -- 172.21.15.36:0/1487611198 wait complete. 2022-06-17T12:46:58.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.881+0000 7feacffd0700 1 Processor -- start 2022-06-17T12:46:58.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.881+0000 7feacffd0700 1 -- start start 2022-06-17T12:46:58.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.885+0000 7feacffd0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feac80616c0 0x7feac813ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:58.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.885+0000 7feacffd0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feac8061eb0 0x7feac8145a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:58.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.885+0000 7feacffd0700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feac80b7b60 0x7feac8140070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:58.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.885+0000 7feacffd0700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac80664d0 con 0x7feac80616c0 2022-06-17T12:46:58.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.885+0000 7feacffd0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac8065120 con 0x7feac8061eb0 2022-06-17T12:46:58.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.885+0000 7feacffd0700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac80655c0 con 0x7feac80b7b60 2022-06-17T12:46:58.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.885+0000 7feacdd6c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feac80616c0 0x7feac813ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:58.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.885+0000 7feacd56b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feac8061eb0 0x7feac8145a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:58.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.885+0000 7feace56d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feac80b7b60 0x7feac8140070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:58.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.885+0000 7feacdd6c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feac80616c0 0x7feac813ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59598/0 (socket says 172.21.15.36:59598) 2022-06-17T12:46:58.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.885+0000 7feacd56b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feac8061eb0 0x7feac8145a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:51746/0 (socket says 172.21.15.36:51746) 2022-06-17T12:46:58.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.885+0000 7feacdd6c700 1 -- 172.21.15.36:0/1595280092 learned_addr learned my addr 172.21.15.36:0/1595280092 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:58.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.885+0000 7feacdd6c700 1 -- 172.21.15.36:0/1595280092 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feac80b7b60 msgr2=0x7feac8140070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:58.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.885+0000 7feacdd6c700 1 --2- 172.21.15.36:0/1595280092 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feac80b7b60 0x7feac8140070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:58.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.885+0000 7feacdd6c700 1 -- 172.21.15.36:0/1595280092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feac8061eb0 msgr2=0x7feac8145a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:58.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.885+0000 7feacdd6c700 1 --2- 172.21.15.36:0/1595280092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feac8061eb0 0x7feac8145a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:58.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.885+0000 7feacdd6c700 1 -- 172.21.15.36:0/1595280092 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac0009470 con 0x7feac80616c0 2022-06-17T12:46:58.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.885+0000 7feacd56b700 1 --2- 172.21.15.36:0/1595280092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feac8061eb0 0x7feac8145a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:46:58.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.885+0000 7feacdd6c700 1 --2- 172.21.15.36:0/1595280092 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feac80616c0 0x7feac813ed00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7feab800c000 tx=0x7feab8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:58.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.885+0000 7feac6ffd700 1 -- 172.21.15.36:0/1595280092 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feab8011750 con 0x7feac80616c0 2022-06-17T12:46:58.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.885+0000 7feac6ffd700 1 -- 172.21.15.36:0/1595280092 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab80118b0 con 0x7feac80616c0 2022-06-17T12:46:58.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.885+0000 7feacffd0700 1 -- 172.21.15.36:0/1595280092 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac80658b0 con 0x7feac80616c0 2022-06-17T12:46:58.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.885+0000 7feac6ffd700 1 -- 172.21.15.36:0/1595280092 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feab8011c10 con 0x7feac80616c0 2022-06-17T12:46:58.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.885+0000 7feacffd0700 1 -- 172.21.15.36:0/1595280092 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac8067240 con 0x7feac80616c0 2022-06-17T12:46:58.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.889+0000 7feacffd0700 1 -- 172.21.15.36:0/1595280092 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac8057410 con 0x7feac80616c0 2022-06-17T12:46:58.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.889+0000 7feac6ffd700 1 -- 172.21.15.36:0/1595280092 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7feab8011d70 con 0x7feac80616c0 2022-06-17T12:46:58.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.889+0000 7feac6ffd700 1 --2- 172.21.15.36:0/1595280092 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7feaa8053a40 0x7feaa8055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:58.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.889+0000 7feacd56b700 1 --2- 172.21.15.36:0/1595280092 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7feaa8053a40 0x7feaa8055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:58.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.889+0000 7feac6ffd700 1 -- 172.21.15.36:0/1595280092 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7feab806c4f0 con 0x7feac80616c0 2022-06-17T12:46:58.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.889+0000 7feacd56b700 1 --2- 172.21.15.36:0/1595280092 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7feaa8053a40 0x7feaa8055f00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7feabc00d000 tx=0x7feabc007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:58.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:58.897+0000 7feac6ffd700 1 -- 172.21.15.36:0/1595280092 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7feab8043180 con 0x7feac80616c0 2022-06-17T12:46:59.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.021+0000 7feacffd0700 1 -- 172.21.15.36:0/1595280092 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feac8066610 con 0x7feac80616c0 2022-06-17T12:46:59.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.021+0000 7feac6ffd700 1 -- 172.21.15.36:0/1595280092 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+16299 (secure 0 0 0) 0x7feab80467a0 con 0x7feac80616c0 2022-06-17T12:46:59.018 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:46:59.019 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":26,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:46:53.258075+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:46:47.465328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-17T12:46:49.893338+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:46:59.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.025+0000 7feacffd0700 1 -- 172.21.15.36:0/1595280092 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7feaa8053a40 msgr2=0x7feaa8055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:59.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.025+0000 7feacffd0700 1 --2- 172.21.15.36:0/1595280092 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7feaa8053a40 0x7feaa8055f00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7feabc00d000 tx=0x7feabc007a00).stop 2022-06-17T12:46:59.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.025+0000 7feacffd0700 1 -- 172.21.15.36:0/1595280092 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feac80616c0 msgr2=0x7feac813ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:59.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.025+0000 7feacffd0700 1 --2- 172.21.15.36:0/1595280092 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feac80616c0 0x7feac813ed00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7feab800c000 tx=0x7feab8006df0).stop 2022-06-17T12:46:59.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.025+0000 7feacffd0700 1 -- 172.21.15.36:0/1595280092 shutdown_connections 2022-06-17T12:46:59.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.025+0000 7feacffd0700 1 --2- 172.21.15.36:0/1595280092 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7feaa8053a40 0x7feaa8055f00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:59.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.025+0000 7feacffd0700 1 --2- 172.21.15.36:0/1595280092 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feac80b7b60 0x7feac8140070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:59.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.025+0000 7feacffd0700 1 --2- 172.21.15.36:0/1595280092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feac8061eb0 0x7feac8145a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:59.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.025+0000 7feacffd0700 1 --2- 172.21.15.36:0/1595280092 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feac80616c0 0x7feac813ed00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:59.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.025+0000 7feacffd0700 1 -- 172.21.15.36:0/1595280092 >> 172.21.15.36:0/1595280092 conn(0x7feac8060c60 msgr2=0x7feac8140ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:59.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.025+0000 7feacffd0700 1 -- 172.21.15.36:0/1595280092 shutdown_connections 2022-06-17T12:46:59.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.025+0000 7feacffd0700 1 -- 172.21.15.36:0/1595280092 wait complete. 2022-06-17T12:46:59.037 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2022-06-17T12:46:59.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.173+0000 7f7b98d37700 1 Processor -- start 2022-06-17T12:46:59.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.173+0000 7f7b98d37700 1 -- start start 2022-06-17T12:46:59.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.173+0000 7f7b98d37700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7b940ac340 0x7f7b9414b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:59.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.173+0000 7f7b98d37700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b940ac870 con 0x7f7b940af7f0 2022-06-17T12:46:59.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.173+0000 7f7b98d37700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b940acaa0 con 0x7f7b940ad140 2022-06-17T12:46:59.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.173+0000 7f7b98d37700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b94148960 con 0x7f7b940ac340 2022-06-17T12:46:59.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.173+0000 7f7b92d9d700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f7b940af7f0 0x7f7b94147b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:48906/0 (socket says 172.21.15.36:48906) 2022-06-17T12:46:59.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.173+0000 7f7b92d9d700 1 -- 172.21.15.36:0/2641776247 learned_addr learned my addr 172.21.15.36:0/2641776247 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:46:59.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.173+0000 7f7b91d9b700 1 --2- 172.21.15.36:0/2641776247 >> v2:172.21.15.70:3300/0 conn(0x7f7b940ac340 0x7f7b9414b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:59.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.173+0000 7f7b9159a700 1 -- 172.21.15.36:0/2641776247 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2177220233 0 0) 0x7f7b940ac870 con 0x7f7b940af7f0 2022-06-17T12:46:59.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.173+0000 7f7b9159a700 1 -- 172.21.15.36:0/2641776247 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b78003260 con 0x7f7b940af7f0 2022-06-17T12:46:59.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.173+0000 7f7b91d9b700 1 -- 172.21.15.36:0/2641776247 >> v1:172.21.15.70:6789/0 conn(0x7f7b940ad140 legacy=0x7f7b9414aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:59.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.173+0000 7f7b91d9b700 1 -- 172.21.15.36:0/2641776247 >> v1:172.21.15.36:6789/0 conn(0x7f7b940af7f0 legacy=0x7f7b94147b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:59.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.173+0000 7f7b91d9b700 1 -- 172.21.15.36:0/2641776247 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b940b0bf0 con 0x7f7b940ac340 2022-06-17T12:46:59.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.173+0000 7f7b91d9b700 1 --2- 172.21.15.36:0/2641776247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b940ac340 0x7f7b9414b3b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f7b84002f70 tx=0x7f7b84006df0).ready entity=mon.1 client_cookie=2b5d8370e34b8172 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:59.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.173+0000 7f7b9159a700 1 -- 172.21.15.36:0/2641776247 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b8400d610 con 0x7f7b940ac340 2022-06-17T12:46:59.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.177+0000 7f7b9159a700 1 -- 172.21.15.36:0/2641776247 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b84000e90 con 0x7f7b940ac340 2022-06-17T12:46:59.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.177+0000 7f7b9159a700 1 -- 172.21.15.36:0/2641776247 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b8400d960 con 0x7f7b940ac340 2022-06-17T12:46:59.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.177+0000 7f7b98d37700 1 -- 172.21.15.36:0/2641776247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b940ac340 msgr2=0x7f7b9414b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:59.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.177+0000 7f7b98d37700 1 --2- 172.21.15.36:0/2641776247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b940ac340 0x7f7b9414b3b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f7b84002f70 tx=0x7f7b84006df0).stop 2022-06-17T12:46:59.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.177+0000 7f7b98d37700 1 -- 172.21.15.36:0/2641776247 shutdown_connections 2022-06-17T12:46:59.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.177+0000 7f7b98d37700 1 --2- 172.21.15.36:0/2641776247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b940ac340 0x7f7b9414b3b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:59.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.177+0000 7f7b98d37700 1 -- 172.21.15.36:0/2641776247 >> 172.21.15.36:0/2641776247 conn(0x7f7b940b5430 msgr2=0x7f7b940b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:59.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.177+0000 7f7b98d37700 1 -- 172.21.15.36:0/2641776247 shutdown_connections 2022-06-17T12:46:59.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.177+0000 7f7b98d37700 1 -- 172.21.15.36:0/2641776247 wait complete. 2022-06-17T12:46:59.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.177+0000 7f7b98d37700 1 Processor -- start 2022-06-17T12:46:59.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.177+0000 7f7b98d37700 1 -- start start 2022-06-17T12:46:59.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.177+0000 7f7b98d37700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b940ac340 0x7f7b94148ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:59.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.177+0000 7f7b98d37700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7b940ad140 0x7f7b940aae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:59.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.177+0000 7f7b98d37700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b940af7f0 0x7f7b94149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:59.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.177+0000 7f7b98d37700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b940a97b0 con 0x7f7b940ad140 2022-06-17T12:46:59.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.177+0000 7f7b98d37700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b940aa520 con 0x7f7b940ac340 2022-06-17T12:46:59.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.177+0000 7f7b98d37700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b940aa750 con 0x7f7b940af7f0 2022-06-17T12:46:59.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.177+0000 7f7b91d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7b940ad140 0x7f7b940aae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:59.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.177+0000 7f7b91d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7b940ad140 0x7f7b940aae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59612/0 (socket says 172.21.15.36:59612) 2022-06-17T12:46:59.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.177+0000 7f7b91d9b700 1 -- 172.21.15.36:0/2131127181 learned_addr learned my addr 172.21.15.36:0/2131127181 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:59.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.177+0000 7f7b9259c700 1 --2- 172.21.15.36:0/2131127181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b940ac340 0x7f7b94148ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:59.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.177+0000 7f7b92d9d700 1 --2- 172.21.15.36:0/2131127181 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b940af7f0 0x7f7b94149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:59.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.181+0000 7f7b91d9b700 1 -- 172.21.15.36:0/2131127181 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b940af7f0 msgr2=0x7f7b94149dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:59.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.181+0000 7f7b91d9b700 1 --2- 172.21.15.36:0/2131127181 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b940af7f0 0x7f7b94149dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:59.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.181+0000 7f7b91d9b700 1 -- 172.21.15.36:0/2131127181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b940ac340 msgr2=0x7f7b94148ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:59.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.181+0000 7f7b91d9b700 1 --2- 172.21.15.36:0/2131127181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b940ac340 0x7f7b94148ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:59.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.181+0000 7f7b91d9b700 1 -- 172.21.15.36:0/2131127181 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8400b180 con 0x7f7b940ad140 2022-06-17T12:46:59.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.181+0000 7f7b91d9b700 1 --2- 172.21.15.36:0/2131127181 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7b940ad140 0x7f7b940aae90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f7b84002f70 tx=0x7f7b84014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:59.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.181+0000 7f7b837fe700 1 -- 172.21.15.36:0/2131127181 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b8400d290 con 0x7f7b940ad140 2022-06-17T12:46:59.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.181+0000 7f7b837fe700 1 -- 172.21.15.36:0/2131127181 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b84013e90 con 0x7f7b940ad140 2022-06-17T12:46:59.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.181+0000 7f7b98d37700 1 -- 172.21.15.36:0/2131127181 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b940aa9f0 con 0x7f7b940ad140 2022-06-17T12:46:59.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.181+0000 7f7b837fe700 1 -- 172.21.15.36:0/2131127181 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b8401e5f0 con 0x7f7b940ad140 2022-06-17T12:46:59.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.181+0000 7f7b98d37700 1 -- 172.21.15.36:0/2131127181 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b940abb90 con 0x7f7b940ad140 2022-06-17T12:46:59.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.181+0000 7f7b98d37700 1 -- 172.21.15.36:0/2131127181 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7b940aab80 con 0x7f7b940ad140 2022-06-17T12:46:59.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.181+0000 7f7b837fe700 1 -- 172.21.15.36:0/2131127181 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f7b84025030 con 0x7f7b940ad140 2022-06-17T12:46:59.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.181+0000 7f7b837fe700 1 --2- 172.21.15.36:0/2131127181 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7b70053a40 0x7f7b70055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:59.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.181+0000 7f7b837fe700 1 -- 172.21.15.36:0/2131127181 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f7b8406cf10 con 0x7f7b940ad140 2022-06-17T12:46:59.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.181+0000 7f7b9259c700 1 --2- 172.21.15.36:0/2131127181 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7b70053a40 0x7f7b70055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:59.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.185+0000 7f7b837fe700 1 --2- 172.21.15.36:0/2131127181 >> [v2:172.21.15.36:6802/22619,v1:172.21.15.36:6803/22619] conn(0x7f7b7005b250 0x7f7b7005d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:59.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.185+0000 7f7b837fe700 1 -- 172.21.15.36:0/2131127181 --> [v2:172.21.15.36:6802/22619,v1:172.21.15.36:6803/22619] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b7005dcd0 con 0x7f7b7005b250 2022-06-17T12:46:59.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.185+0000 7f7b837fe700 1 -- 172.21.15.36:0/2131127181 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b8406f490 con 0x7f7b940ad140 2022-06-17T12:46:59.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.185+0000 7f7b92d9d700 1 --2- 172.21.15.36:0/2131127181 >> [v2:172.21.15.36:6802/22619,v1:172.21.15.36:6803/22619] conn(0x7f7b7005b250 0x7f7b7005d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:59.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.185+0000 7f7b9259c700 1 --2- 172.21.15.36:0/2131127181 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7b70053a40 0x7f7b70055f00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f7b8800d000 tx=0x7f7b88007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:59.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.185+0000 7f7b92d9d700 1 --2- 172.21.15.36:0/2131127181 >> [v2:172.21.15.36:6802/22619,v1:172.21.15.36:6803/22619] conn(0x7f7b7005b250 0x7f7b7005d6f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:59.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.185+0000 7f7b837fe700 1 -- 172.21.15.36:0/2131127181 <== osd.0 v2:172.21.15.36:6802/22619 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f7b7005dcd0 con 0x7f7b7005b250 2022-06-17T12:46:59.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.201+0000 7f7b98d37700 1 -- 172.21.15.36:0/2131127181 --> [v2:172.21.15.36:6802/22619,v1:172.21.15.36:6803/22619] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7b94057410 con 0x7f7b7005b250 2022-06-17T12:46:59.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.201+0000 7f7b837fe700 1 -- 172.21.15.36:0/2131127181 <== osd.0 v2:172.21.15.36:6802/22619 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7b94057410 con 0x7f7b7005b250 2022-06-17T12:46:59.279 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.201+0000 7f7b98d37700 1 -- 172.21.15.36:0/2131127181 >> [v2:172.21.15.36:6802/22619,v1:172.21.15.36:6803/22619] conn(0x7f7b7005b250 msgr2=0x7f7b7005d6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:59.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.201+0000 7f7b98d37700 1 --2- 172.21.15.36:0/2131127181 >> [v2:172.21.15.36:6802/22619,v1:172.21.15.36:6803/22619] conn(0x7f7b7005b250 0x7f7b7005d6f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:59.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.201+0000 7f7b98d37700 1 -- 172.21.15.36:0/2131127181 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7b70053a40 msgr2=0x7f7b70055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:59.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.201+0000 7f7b98d37700 1 --2- 172.21.15.36:0/2131127181 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7b70053a40 0x7f7b70055f00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f7b8800d000 tx=0x7f7b88007b90).stop 2022-06-17T12:46:59.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.201+0000 7f7b98d37700 1 -- 172.21.15.36:0/2131127181 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7b940ad140 msgr2=0x7f7b940aae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:59.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.201+0000 7f7b98d37700 1 --2- 172.21.15.36:0/2131127181 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7b940ad140 0x7f7b940aae90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f7b84002f70 tx=0x7f7b84014000).stop 2022-06-17T12:46:59.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.201+0000 7f7b92d9d700 1 -- 172.21.15.36:0/2131127181 reap_dead start 2022-06-17T12:46:59.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.201+0000 7f7b98d37700 1 -- 172.21.15.36:0/2131127181 shutdown_connections 2022-06-17T12:46:59.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.201+0000 7f7b98d37700 1 -- 172.21.15.36:0/2131127181 >> 172.21.15.36:0/2131127181 conn(0x7f7b940b5430 msgr2=0x7f7b9405f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:59.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.201+0000 7f7b98d37700 1 -- 172.21.15.36:0/2131127181 shutdown_connections 2022-06-17T12:46:59.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.201+0000 7f7b98d37700 1 -- 172.21.15.36:0/2131127181 wait complete. 2022-06-17T12:46:59.284 INFO:teuthology.orchestra.run.smithi036.stdout:55834574854 2022-06-17T12:46:59.285 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2022-06-17T12:46:59.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26f75da700 1 Processor -- start 2022-06-17T12:46:59.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26f75da700 1 -- start start 2022-06-17T12:46:59.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26f75da700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26f00aedc0 0x7f26f00b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:59.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26f75da700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f00af1a0 con 0x7f26f0064340 2022-06-17T12:46:59.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26f75da700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f0063960 con 0x7f26f0067290 2022-06-17T12:46:59.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26f75da700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f0063c00 con 0x7f26f00aedc0 2022-06-17T12:46:59.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26f4b75700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f26f0064340 0x7f26f0066b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:48918/0 (socket says 172.21.15.36:48918) 2022-06-17T12:46:59.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26f4b75700 1 -- 172.21.15.36:0/2095782364 learned_addr learned my addr 172.21.15.36:0/2095782364 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:46:59.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26f5376700 1 --2- 172.21.15.36:0/2095782364 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26f00aedc0 0x7f26f00b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:59.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26effff700 1 -- 172.21.15.36:0/2095782364 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256532108 0 0) 0x7f26f00af1a0 con 0x7f26f0064340 2022-06-17T12:46:59.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26effff700 1 -- 172.21.15.36:0/2095782364 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26dc003260 con 0x7f26f0064340 2022-06-17T12:46:59.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26effff700 1 -- 172.21.15.36:0/2095782364 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1060341179 0 0) 0x7f26dc003260 con 0x7f26f0064340 2022-06-17T12:46:59.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26effff700 1 -- 172.21.15.36:0/2095782364 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26f00aedc0 msgr2=0x7f26f00b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:59.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26effff700 1 --2- 172.21.15.36:0/2095782364 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26f00aedc0 0x7f26f00b0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:59.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26effff700 1 -- 172.21.15.36:0/2095782364 >> v1:172.21.15.70:6789/0 conn(0x7f26f0067290 legacy=0x7f26f005f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:59.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26effff700 1 -- 172.21.15.36:0/2095782364 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f0063ea0 con 0x7f26f0064340 2022-06-17T12:46:59.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26effff700 1 -- 172.21.15.36:0/2095782364 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f26e00019e0 con 0x7f26f0064340 2022-06-17T12:46:59.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26effff700 1 -- 172.21.15.36:0/2095782364 >> v1:172.21.15.36:6789/0 conn(0x7f26f0064340 legacy=0x7f26f0066b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:59.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26effff700 1 --2- 172.21.15.36:0/2095782364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26dc003780 0x7f26dc003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:59.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26effff700 1 --2- 172.21.15.36:0/2095782364 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26f00aedc0 0x7f26dc004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:59.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26effff700 1 --2- 172.21.15.36:0/2095782364 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26dc0049f0 0x7f26dc004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:59.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26effff700 1 -- 172.21.15.36:0/2095782364 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f0063c00 con 0x7f26dc0049f0 2022-06-17T12:46:59.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26effff700 1 -- 172.21.15.36:0/2095782364 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc00b780 con 0x7f26dc003780 2022-06-17T12:46:59.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26effff700 1 -- 172.21.15.36:0/2095782364 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc00b940 con 0x7f26f00aedc0 2022-06-17T12:46:59.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26f5b77700 1 --2- 172.21.15.36:0/2095782364 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26dc0049f0 0x7f26dc004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:59.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.413+0000 7f26f5376700 1 --2- 172.21.15.36:0/2095782364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26dc003780 0x7f26dc003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:59.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.417+0000 7f26f4b75700 1 --2- 172.21.15.36:0/2095782364 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26f00aedc0 0x7f26dc004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:59.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.417+0000 7f26f5376700 1 -- 172.21.15.36:0/2095782364 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26f00aedc0 msgr2=0x7f26dc004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:59.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.417+0000 7f26f5376700 1 --2- 172.21.15.36:0/2095782364 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26f00aedc0 0x7f26dc004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:59.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.417+0000 7f26f5376700 1 -- 172.21.15.36:0/2095782364 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26dc0049f0 msgr2=0x7f26dc004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:59.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.417+0000 7f26f5376700 1 --2- 172.21.15.36:0/2095782364 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26dc0049f0 0x7f26dc004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:59.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.417+0000 7f26f5376700 1 -- 172.21.15.36:0/2095782364 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26dc003260 con 0x7f26dc003780 2022-06-17T12:46:59.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.417+0000 7f26f5b77700 1 --2- 172.21.15.36:0/2095782364 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26dc0049f0 0x7f26dc004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:46:59.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.417+0000 7f26f5376700 1 --2- 172.21.15.36:0/2095782364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26dc003780 0x7f26dc003ba0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f26e400c000 tx=0x7f26e4006df0).ready entity=mon.1 client_cookie=84ccb13b21df84d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:59.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.417+0000 7f26effff700 1 -- 172.21.15.36:0/2095782364 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26e400e830 con 0x7f26dc003780 2022-06-17T12:46:59.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.417+0000 7f26effff700 1 -- 172.21.15.36:0/2095782364 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e400e990 con 0x7f26dc003780 2022-06-17T12:46:59.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.417+0000 7f26effff700 1 -- 172.21.15.36:0/2095782364 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26e400ecf0 con 0x7f26dc003780 2022-06-17T12:46:59.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.417+0000 7f26f75da700 1 -- 172.21.15.36:0/2095782364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26dc003780 msgr2=0x7f26dc003ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:59.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.417+0000 7f26f75da700 1 --2- 172.21.15.36:0/2095782364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26dc003780 0x7f26dc003ba0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f26e400c000 tx=0x7f26e4006df0).stop 2022-06-17T12:46:59.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.417+0000 7f26f5b77700 1 -- 172.21.15.36:0/2095782364 reap_dead start 2022-06-17T12:46:59.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.417+0000 7f26f75da700 1 -- 172.21.15.36:0/2095782364 shutdown_connections 2022-06-17T12:46:59.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.417+0000 7f26f75da700 1 -- 172.21.15.36:0/2095782364 >> 172.21.15.36:0/2095782364 conn(0x7f26f00b5430 msgr2=0x7f26f00b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:59.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.417+0000 7f26f75da700 1 -- 172.21.15.36:0/2095782364 shutdown_connections 2022-06-17T12:46:59.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.417+0000 7f26f75da700 1 -- 172.21.15.36:0/2095782364 wait complete. 2022-06-17T12:46:59.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.417+0000 7f26f75da700 1 Processor -- start 2022-06-17T12:46:59.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.417+0000 7f26f75da700 1 -- start start 2022-06-17T12:46:59.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.421+0000 7f26f75da700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26dc003780 0x7f26f00b4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:59.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.421+0000 7f26f75da700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26f00a03f0 0x7f26f0142910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:59.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.421+0000 7f26f75da700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f009f620 0x7f26f00a2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:59.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.421+0000 7f26f75da700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f00a12b0 con 0x7f26dc003780 2022-06-17T12:46:59.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.421+0000 7f26f75da700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f009faa0 con 0x7f26f009f620 2022-06-17T12:46:59.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.421+0000 7f26f75da700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f009fd30 con 0x7f26f00a03f0 2022-06-17T12:46:59.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.421+0000 7f26f5376700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26dc003780 0x7f26f00b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:59.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.421+0000 7f26f5376700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26dc003780 0x7f26f00b4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59630/0 (socket says 172.21.15.36:59630) 2022-06-17T12:46:59.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.421+0000 7f26f5376700 1 -- 172.21.15.36:0/2387252681 learned_addr learned my addr 172.21.15.36:0/2387252681 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:59.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.421+0000 7f26f4b75700 1 --2- 172.21.15.36:0/2387252681 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26f00a03f0 0x7f26f0142910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:59.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.421+0000 7f26f5b77700 1 --2- 172.21.15.36:0/2387252681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f009f620 0x7f26f00a2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:59.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.421+0000 7f26f5376700 1 -- 172.21.15.36:0/2387252681 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26f00a03f0 msgr2=0x7f26f0142910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:59.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.421+0000 7f26f5376700 1 --2- 172.21.15.36:0/2387252681 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26f00a03f0 0x7f26f0142910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:59.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.421+0000 7f26f5376700 1 -- 172.21.15.36:0/2387252681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f009f620 msgr2=0x7f26f00a2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:59.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.421+0000 7f26f5376700 1 --2- 172.21.15.36:0/2387252681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f009f620 0x7f26f00a2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:59.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.421+0000 7f26f5376700 1 -- 172.21.15.36:0/2387252681 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f009ffd0 con 0x7f26dc003780 2022-06-17T12:46:59.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.421+0000 7f26f5376700 1 --2- 172.21.15.36:0/2387252681 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26dc003780 0x7f26f00b4530 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f26e400c000 tx=0x7f26e4019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:59.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.421+0000 7f26ee7fc700 1 -- 172.21.15.36:0/2387252681 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26e401b820 con 0x7f26dc003780 2022-06-17T12:46:59.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.421+0000 7f26ee7fc700 1 -- 172.21.15.36:0/2387252681 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e4000e90 con 0x7f26dc003780 2022-06-17T12:46:59.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.421+0000 7f26f75da700 1 -- 172.21.15.36:0/2387252681 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f00a1d60 con 0x7f26dc003780 2022-06-17T12:46:59.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.421+0000 7f26ee7fc700 1 -- 172.21.15.36:0/2387252681 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26e401bb70 con 0x7f26dc003780 2022-06-17T12:46:59.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.421+0000 7f26f75da700 1 -- 172.21.15.36:0/2387252681 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f00a2910 con 0x7f26dc003780 2022-06-17T12:46:59.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.425+0000 7f26f75da700 1 -- 172.21.15.36:0/2387252681 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f26f00a1ef0 con 0x7f26dc003780 2022-06-17T12:46:59.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.425+0000 7f26ee7fc700 1 -- 172.21.15.36:0/2387252681 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f26e401bcd0 con 0x7f26dc003780 2022-06-17T12:46:59.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.425+0000 7f26ee7fc700 1 --2- 172.21.15.36:0/2387252681 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26d0053940 0x7f26d0055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:59.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.425+0000 7f26ee7fc700 1 -- 172.21.15.36:0/2387252681 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f26e406d210 con 0x7f26dc003780 2022-06-17T12:46:59.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.425+0000 7f26f4b75700 1 --2- 172.21.15.36:0/2387252681 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26d0053940 0x7f26d0055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:59.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.425+0000 7f26ee7fc700 1 --2- 172.21.15.36:0/2387252681 >> [v2:172.21.15.36:6810/22620,v1:172.21.15.36:6811/22620] conn(0x7f26d005b360 0x7f26d005d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:59.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.425+0000 7f26ee7fc700 1 -- 172.21.15.36:0/2387252681 --> [v2:172.21.15.36:6810/22620,v1:172.21.15.36:6811/22620] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f26d005de80 con 0x7f26d005b360 2022-06-17T12:46:59.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.425+0000 7f26ee7fc700 1 -- 172.21.15.36:0/2387252681 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26e406f8a0 con 0x7f26dc003780 2022-06-17T12:46:59.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.425+0000 7f26f5b77700 1 --2- 172.21.15.36:0/2387252681 >> [v2:172.21.15.36:6810/22620,v1:172.21.15.36:6811/22620] conn(0x7f26d005b360 0x7f26d005d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:59.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.425+0000 7f26f4b75700 1 --2- 172.21.15.36:0/2387252681 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26d0053940 0x7f26d0055e00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f26e0005000 tx=0x7f26e0002190).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:59.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.425+0000 7f26f5b77700 1 --2- 172.21.15.36:0/2387252681 >> [v2:172.21.15.36:6810/22620,v1:172.21.15.36:6811/22620] conn(0x7f26d005b360 0x7f26d005d800 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:59.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.425+0000 7f26ee7fc700 1 -- 172.21.15.36:0/2387252681 <== osd.1 v2:172.21.15.36:6810/22620 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f26d005de80 con 0x7f26d005b360 2022-06-17T12:46:59.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.445+0000 7f26f75da700 1 -- 172.21.15.36:0/2387252681 --> [v2:172.21.15.36:6810/22620,v1:172.21.15.36:6811/22620] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f26f0057410 con 0x7f26d005b360 2022-06-17T12:46:59.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.445+0000 7f26ee7fc700 1 -- 172.21.15.36:0/2387252681 <== osd.1 v2:172.21.15.36:6810/22620 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f26f0057410 con 0x7f26d005b360 2022-06-17T12:46:59.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.445+0000 7f26f75da700 1 -- 172.21.15.36:0/2387252681 >> [v2:172.21.15.36:6810/22620,v1:172.21.15.36:6811/22620] conn(0x7f26d005b360 msgr2=0x7f26d005d800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:59.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.445+0000 7f26f75da700 1 --2- 172.21.15.36:0/2387252681 >> [v2:172.21.15.36:6810/22620,v1:172.21.15.36:6811/22620] conn(0x7f26d005b360 0x7f26d005d800 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:59.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.445+0000 7f26f75da700 1 -- 172.21.15.36:0/2387252681 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26d0053940 msgr2=0x7f26d0055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:59.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.445+0000 7f26f75da700 1 --2- 172.21.15.36:0/2387252681 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26d0053940 0x7f26d0055e00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f26e0005000 tx=0x7f26e0002190).stop 2022-06-17T12:46:59.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.445+0000 7f26f75da700 1 -- 172.21.15.36:0/2387252681 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26dc003780 msgr2=0x7f26f00b4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:59.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.445+0000 7f26f75da700 1 --2- 172.21.15.36:0/2387252681 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26dc003780 0x7f26f00b4530 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f26e400c000 tx=0x7f26e4019000).stop 2022-06-17T12:46:59.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.445+0000 7f26f5b77700 1 -- 172.21.15.36:0/2387252681 reap_dead start 2022-06-17T12:46:59.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.445+0000 7f26f75da700 1 -- 172.21.15.36:0/2387252681 shutdown_connections 2022-06-17T12:46:59.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.445+0000 7f26f75da700 1 -- 172.21.15.36:0/2387252681 >> 172.21.15.36:0/2387252681 conn(0x7f26f00b5430 msgr2=0x7f26f00b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:59.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.445+0000 7f26f75da700 1 -- 172.21.15.36:0/2387252681 shutdown_connections 2022-06-17T12:46:59.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.445+0000 7f26f75da700 1 -- 172.21.15.36:0/2387252681 wait complete. 2022-06-17T12:46:59.637 INFO:teuthology.orchestra.run.smithi036.stdout:55834574854 2022-06-17T12:46:59.638 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2022-06-17T12:46:59.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.765+0000 7f65e33b4700 1 Processor -- start 2022-06-17T12:46:59.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.765+0000 7f65e33b4700 1 -- start start 2022-06-17T12:46:59.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65e33b4700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65dc0620d0 0x7f65dc0b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:59.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65e33b4700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f65dc0b4e10 0x7f65dc0b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:59.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65e33b4700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65dc0b21d0 con 0x7f65dc0b31c0 2022-06-17T12:46:59.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65e33b4700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65dc0b2430 con 0x7f65dc0b4e10 2022-06-17T12:46:59.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65e33b4700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc0b26d0 con 0x7f65dc0620d0 2022-06-17T12:46:59.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65e1951700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f65dc0b4e10 0x7f65dc0b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:59.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65e1951700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f65dc0b4e10 0x7f65dc0b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59644/0 (socket says 172.21.15.36:59644) 2022-06-17T12:46:59.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65e1951700 1 -- 172.21.15.36:0/2082865960 learned_addr learned my addr 172.21.15.36:0/2082865960 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:59.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65e094f700 1 --1- 172.21.15.36:0/2082865960 >> v1:172.21.15.36:6789/0 conn(0x7f65dc0b31c0 0x7f65dc0b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:48938/0 (socket says 172.21.15.36:48938) 2022-06-17T12:46:59.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65e1150700 1 --2- 172.21.15.36:0/2082865960 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65dc0620d0 0x7f65dc0b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:59.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65e1951700 1 -- 172.21.15.36:0/2082865960 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65dc0620d0 msgr2=0x7f65dc0b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:46:59.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65e1951700 1 --2- 172.21.15.36:0/2082865960 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65dc0620d0 0x7f65dc0b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:59.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65e1951700 1 -- 172.21.15.36:0/2082865960 >> v1:172.21.15.36:6789/0 conn(0x7f65dc0b31c0 legacy=0x7f65dc0b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:59.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65e1951700 1 -- 172.21.15.36:0/2082865960 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc0b43f0 con 0x7f65dc0b4e10 2022-06-17T12:46:59.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65e1951700 1 --2- 172.21.15.36:0/2082865960 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f65dc0b4e10 0x7f65dc0b2b70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f65d40032f0 tx=0x7f65d4009290).ready entity=mon.0 client_cookie=513b2aca420157f5 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:59.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65dbfff700 1 -- 172.21.15.36:0/2082865960 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65d400b8a0 con 0x7f65dc0b4e10 2022-06-17T12:46:59.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65dbfff700 1 -- 172.21.15.36:0/2082865960 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d400ba00 con 0x7f65dc0b4e10 2022-06-17T12:46:59.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65dbfff700 1 -- 172.21.15.36:0/2082865960 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65d400bb70 con 0x7f65dc0b4e10 2022-06-17T12:46:59.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65e33b4700 1 -- 172.21.15.36:0/2082865960 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f65dc0b4e10 msgr2=0x7f65dc0b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:59.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65e33b4700 1 --2- 172.21.15.36:0/2082865960 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f65dc0b4e10 0x7f65dc0b2b70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f65d40032f0 tx=0x7f65d4009290).stop 2022-06-17T12:46:59.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65e33b4700 1 -- 172.21.15.36:0/2082865960 shutdown_connections 2022-06-17T12:46:59.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65e33b4700 1 --2- 172.21.15.36:0/2082865960 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f65dc0b4e10 0x7f65dc0b2b70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:59.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65e33b4700 1 --2- 172.21.15.36:0/2082865960 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65dc0620d0 0x7f65dc0b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:59.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65e33b4700 1 -- 172.21.15.36:0/2082865960 >> 172.21.15.36:0/2082865960 conn(0x7f65dc0b9850 msgr2=0x7f65dc0b9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:46:59.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65e33b4700 1 -- 172.21.15.36:0/2082865960 shutdown_connections 2022-06-17T12:46:59.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.769+0000 7f65e33b4700 1 -- 172.21.15.36:0/2082865960 wait complete. 2022-06-17T12:46:59.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.773+0000 7f65e33b4700 1 Processor -- start 2022-06-17T12:46:59.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.773+0000 7f65e33b4700 1 -- start start 2022-06-17T12:46:59.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.773+0000 7f65e33b4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f65dc0620d0 0x7f65dc0af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:59.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.773+0000 7f65e33b4700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65dc0b31c0 0x7f65dc146ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:59.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.773+0000 7f65e33b4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65dc0b4e10 0x7f65dc1152b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:59.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.773+0000 7f65e33b4700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc0af6c0 con 0x7f65dc0620d0 2022-06-17T12:46:59.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.773+0000 7f65e33b4700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc114e10 con 0x7f65dc0b4e10 2022-06-17T12:46:59.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.773+0000 7f65e33b4700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc114b20 con 0x7f65dc0b31c0 2022-06-17T12:46:59.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.773+0000 7f65e1150700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f65dc0620d0 0x7f65dc0af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:59.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.773+0000 7f65e1150700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f65dc0620d0 0x7f65dc0af090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59646/0 (socket says 172.21.15.36:59646) 2022-06-17T12:46:59.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.773+0000 7f65e1150700 1 -- 172.21.15.36:0/464699750 learned_addr learned my addr 172.21.15.36:0/464699750 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:46:59.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.773+0000 7f65e094f700 1 --2- 172.21.15.36:0/464699750 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65dc0b31c0 0x7f65dc146ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:59.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.773+0000 7f65e1951700 1 --2- 172.21.15.36:0/464699750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65dc0b4e10 0x7f65dc1152b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:46:59.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.777+0000 7f65e1150700 1 -- 172.21.15.36:0/464699750 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65dc0b31c0 msgr2=0x7f65dc146ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:59.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.777+0000 7f65e1150700 1 --2- 172.21.15.36:0/464699750 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65dc0b31c0 0x7f65dc146ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:59.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.777+0000 7f65e1150700 1 -- 172.21.15.36:0/464699750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65dc0b4e10 msgr2=0x7f65dc1152b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:46:59.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.777+0000 7f65e1150700 1 --2- 172.21.15.36:0/464699750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65dc0b4e10 0x7f65dc1152b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:46:59.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.777+0000 7f65e1150700 1 -- 172.21.15.36:0/464699750 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d4009530 con 0x7f65dc0620d0 2022-06-17T12:46:59.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.777+0000 7f65e1951700 1 --2- 172.21.15.36:0/464699750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65dc0b4e10 0x7f65dc1152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:46:59.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.777+0000 7f65e094f700 1 --2- 172.21.15.36:0/464699750 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65dc0b31c0 0x7f65dc146ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:46:59.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.777+0000 7f65e1150700 1 --2- 172.21.15.36:0/464699750 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f65dc0620d0 0x7f65dc0af090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f65cc0030b0 tx=0x7f65cc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:46:59.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.777+0000 7f65da7fc700 1 -- 172.21.15.36:0/464699750 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65cc011780 con 0x7f65dc0620d0 2022-06-17T12:46:59.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.777+0000 7f65da7fc700 1 -- 172.21.15.36:0/464699750 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65cc0118e0 con 0x7f65dc0620d0 2022-06-17T12:46:59.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.777+0000 7f65da7fc700 1 -- 172.21.15.36:0/464699750 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65cc011c40 con 0x7f65dc0620d0 2022-06-17T12:46:59.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.777+0000 7f65e33b4700 1 -- 172.21.15.36:0/464699750 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65dc0aedb0 con 0x7f65dc0620d0 2022-06-17T12:46:59.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.777+0000 7f65e33b4700 1 -- 172.21.15.36:0/464699750 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65dc1157a0 con 0x7f65dc0620d0 2022-06-17T12:46:59.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.777+0000 7f65e33b4700 1 -- 172.21.15.36:0/464699750 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f65dc114fd0 con 0x7f65dc0620d0 2022-06-17T12:46:59.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.777+0000 7f65da7fc700 1 -- 172.21.15.36:0/464699750 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f65cc01c460 con 0x7f65dc0620d0 2022-06-17T12:46:59.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.777+0000 7f65da7fc700 1 --2- 172.21.15.36:0/464699750 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f65bc0539b0 0x7f65bc055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:46:59.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.777+0000 7f65da7fc700 1 -- 172.21.15.36:0/464699750 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f65cc06c3e0 con 0x7f65dc0620d0 2022-06-17T12:46:59.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.781+0000 7f65e094f700 1 --2- 172.21.15.36:0/464699750 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f65bc0539b0 0x7f65bc055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:00.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.781+0000 7f65da7fc700 1 --2- 172.21.15.36:0/464699750 >> [v2:172.21.15.36:6826/22643,v1:172.21.15.36:6827/22643] conn(0x7f65bc05b2b0 0x7f65bc05d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:00.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.781+0000 7f65da7fc700 1 -- 172.21.15.36:0/464699750 --> [v2:172.21.15.36:6826/22643,v1:172.21.15.36:6827/22643] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f65bc05ddd0 con 0x7f65bc05b2b0 2022-06-17T12:47:00.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.781+0000 7f65da7fc700 1 -- 172.21.15.36:0/464699750 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65cc06e960 con 0x7f65dc0620d0 2022-06-17T12:47:00.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.781+0000 7f65e1951700 1 --2- 172.21.15.36:0/464699750 >> [v2:172.21.15.36:6826/22643,v1:172.21.15.36:6827/22643] conn(0x7f65bc05b2b0 0x7f65bc05d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:00.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.781+0000 7f65e094f700 1 --2- 172.21.15.36:0/464699750 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f65bc0539b0 0x7f65bc055e70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f65d000b000 tx=0x7f65d00070e0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:00.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.781+0000 7f65e1951700 1 --2- 172.21.15.36:0/464699750 >> [v2:172.21.15.36:6826/22643,v1:172.21.15.36:6827/22643] conn(0x7f65bc05b2b0 0x7f65bc05d750 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:00.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.781+0000 7f65da7fc700 1 -- 172.21.15.36:0/464699750 <== osd.2 v2:172.21.15.36:6826/22643 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f65bc05ddd0 con 0x7f65bc05b2b0 2022-06-17T12:47:00.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.801+0000 7f65e33b4700 1 -- 172.21.15.36:0/464699750 --> [v2:172.21.15.36:6826/22643,v1:172.21.15.36:6827/22643] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f65dc0ad940 con 0x7f65bc05b2b0 2022-06-17T12:47:00.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.801+0000 7f65da7fc700 1 -- 172.21.15.36:0/464699750 <== osd.2 v2:172.21.15.36:6826/22643 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f65dc0ad940 con 0x7f65bc05b2b0 2022-06-17T12:47:00.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.801+0000 7f65e33b4700 1 -- 172.21.15.36:0/464699750 >> [v2:172.21.15.36:6826/22643,v1:172.21.15.36:6827/22643] conn(0x7f65bc05b2b0 msgr2=0x7f65bc05d750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:00.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.801+0000 7f65e33b4700 1 --2- 172.21.15.36:0/464699750 >> [v2:172.21.15.36:6826/22643,v1:172.21.15.36:6827/22643] conn(0x7f65bc05b2b0 0x7f65bc05d750 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:00.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.801+0000 7f65e33b4700 1 -- 172.21.15.36:0/464699750 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f65bc0539b0 msgr2=0x7f65bc055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:00.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.801+0000 7f65e33b4700 1 --2- 172.21.15.36:0/464699750 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f65bc0539b0 0x7f65bc055e70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f65d000b000 tx=0x7f65d00070e0).stop 2022-06-17T12:47:00.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.801+0000 7f65e33b4700 1 -- 172.21.15.36:0/464699750 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f65dc0620d0 msgr2=0x7f65dc0af090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:00.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.801+0000 7f65e33b4700 1 --2- 172.21.15.36:0/464699750 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f65dc0620d0 0x7f65dc0af090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f65cc0030b0 tx=0x7f65cc008040).stop 2022-06-17T12:47:00.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.801+0000 7f65e1951700 1 -- 172.21.15.36:0/464699750 reap_dead start 2022-06-17T12:47:00.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.801+0000 7f65e33b4700 1 -- 172.21.15.36:0/464699750 shutdown_connections 2022-06-17T12:47:00.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.801+0000 7f65e33b4700 1 -- 172.21.15.36:0/464699750 >> 172.21.15.36:0/464699750 conn(0x7f65dc0b9850 msgr2=0x7f65dc143110 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:00.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.801+0000 7f65e33b4700 1 -- 172.21.15.36:0/464699750 shutdown_connections 2022-06-17T12:47:00.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:46:59.801+0000 7f65e33b4700 1 -- 172.21.15.36:0/464699750 wait complete. 2022-06-17T12:47:00.084 INFO:teuthology.orchestra.run.smithi036.stdout:60129542150 2022-06-17T12:47:00.103 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2022-06-17T12:47:00.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.237+0000 7f4d4571e700 1 Processor -- start 2022-06-17T12:47:00.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.237+0000 7f4d4571e700 1 -- start start 2022-06-17T12:47:00.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.237+0000 7f4d4571e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d400b6660 0x7f4d400b2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:00.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.237+0000 7f4d4571e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4d400af540 0x7f4d4013a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:00.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.237+0000 7f4d4571e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d400b3cf0 con 0x7f4d400b1190 2022-06-17T12:47:00.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.237+0000 7f4d4571e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d400670a0 con 0x7f4d400af540 2022-06-17T12:47:00.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.237+0000 7f4d4571e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d40144bf0 con 0x7f4d400b6660 2022-06-17T12:47:00.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.237+0000 7f4d3f7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4d400af540 0x7f4d4013a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:00.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.237+0000 7f4d44f1d700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f4d400b1190 0x7f4d40141010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:48958/0 (socket says 172.21.15.36:48958) 2022-06-17T12:47:00.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.237+0000 7f4d44f1d700 1 -- 172.21.15.36:0/958404513 learned_addr learned my addr 172.21.15.36:0/958404513 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:00.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.237+0000 7f4d3ffff700 1 --2- 172.21.15.36:0/958404513 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d400b6660 0x7f4d400b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:00.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.237+0000 7f4d3f7fe700 1 -- 172.21.15.36:0/958404513 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d400b6660 msgr2=0x7f4d400b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:00.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.237+0000 7f4d3f7fe700 1 --2- 172.21.15.36:0/958404513 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d400b6660 0x7f4d400b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:00.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.237+0000 7f4d3f7fe700 1 -- 172.21.15.36:0/958404513 >> v1:172.21.15.36:6789/0 conn(0x7f4d400b1190 legacy=0x7f4d40141010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:00.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.237+0000 7f4d3f7fe700 1 -- 172.21.15.36:0/958404513 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d40144e90 con 0x7f4d400af540 2022-06-17T12:47:00.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.237+0000 7f4d3effd700 1 -- 172.21.15.36:0/958404513 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1156304296 0 0) 0x7f4d38001490 con 0x7f4d400b1190 2022-06-17T12:47:00.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.241+0000 7f4d3f7fe700 1 --2- 172.21.15.36:0/958404513 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d400af540 0x7f4d4013a790 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f4d28002f70 tx=0x7f4d28006d90).ready entity=mon.0 client_cookie=abf9d13c0c4aecf0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:00.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.241+0000 7f4d3effd700 1 -- 172.21.15.36:0/958404513 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d2800d4a0 con 0x7f4d400af540 2022-06-17T12:47:00.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.241+0000 7f4d3effd700 1 -- 172.21.15.36:0/958404513 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d28000e90 con 0x7f4d400af540 2022-06-17T12:47:00.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.241+0000 7f4d3effd700 1 -- 172.21.15.36:0/958404513 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d2800d7f0 con 0x7f4d400af540 2022-06-17T12:47:00.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.241+0000 7f4d4571e700 1 -- 172.21.15.36:0/958404513 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d400af540 msgr2=0x7f4d4013a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:00.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.241+0000 7f4d4571e700 1 --2- 172.21.15.36:0/958404513 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d400af540 0x7f4d4013a790 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f4d28002f70 tx=0x7f4d28006d90).stop 2022-06-17T12:47:00.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.241+0000 7f4d4571e700 1 -- 172.21.15.36:0/958404513 shutdown_connections 2022-06-17T12:47:00.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.241+0000 7f4d4571e700 1 --2- 172.21.15.36:0/958404513 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d400af540 0x7f4d4013a790 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:00.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.241+0000 7f4d4571e700 1 --2- 172.21.15.36:0/958404513 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d400b6660 0x7f4d400b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:00.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.241+0000 7f4d4571e700 1 -- 172.21.15.36:0/958404513 >> 172.21.15.36:0/958404513 conn(0x7f4d400614b0 msgr2=0x7f4d40061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:00.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.241+0000 7f4d4571e700 1 -- 172.21.15.36:0/958404513 shutdown_connections 2022-06-17T12:47:00.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.241+0000 7f4d4571e700 1 -- 172.21.15.36:0/958404513 wait complete. 2022-06-17T12:47:00.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.241+0000 7f4d4571e700 1 Processor -- start 2022-06-17T12:47:00.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.241+0000 7f4d4571e700 1 -- start start 2022-06-17T12:47:00.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.245+0000 7f4d4571e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d400af540 0x7f4d40150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:00.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.245+0000 7f4d4571e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d400b1190 0x7f4d40065530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:00.278 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.245+0000 7f4d4571e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d400b6660 0x7f4d40153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:00.279 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.245+0000 7f4d4571e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d40153750 con 0x7f4d400b6660 2022-06-17T12:47:00.279 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.245+0000 7f4d4571e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4013d3f0 con 0x7f4d400b1190 2022-06-17T12:47:00.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.245+0000 7f4d4571e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d40153980 con 0x7f4d400af540 2022-06-17T12:47:00.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.245+0000 7f4d44f1d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d400b6660 0x7f4d40153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:00.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.245+0000 7f4d44f1d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d400b6660 0x7f4d40153e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59666/0 (socket says 172.21.15.36:59666) 2022-06-17T12:47:00.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.245+0000 7f4d44f1d700 1 -- 172.21.15.36:0/654782762 learned_addr learned my addr 172.21.15.36:0/654782762 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:00.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.245+0000 7f4d3ffff700 1 --2- 172.21.15.36:0/654782762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d400af540 0x7f4d40150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:00.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.245+0000 7f4d3f7fe700 1 --2- 172.21.15.36:0/654782762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d400b1190 0x7f4d40065530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:00.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.245+0000 7f4d44f1d700 1 -- 172.21.15.36:0/654782762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d400af540 msgr2=0x7f4d40150140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:00.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.245+0000 7f4d44f1d700 1 --2- 172.21.15.36:0/654782762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d400af540 0x7f4d40150140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:00.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.245+0000 7f4d44f1d700 1 -- 172.21.15.36:0/654782762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d400b1190 msgr2=0x7f4d40065530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:00.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.245+0000 7f4d44f1d700 1 --2- 172.21.15.36:0/654782762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d400b1190 0x7f4d40065530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:00.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.245+0000 7f4d44f1d700 1 -- 172.21.15.36:0/654782762 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2800b120 con 0x7f4d400b6660 2022-06-17T12:47:00.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.245+0000 7f4d44f1d700 1 --2- 172.21.15.36:0/654782762 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d400b6660 0x7f4d40153e20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f4d38000f40 tx=0x7f4d38007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:00.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.245+0000 7f4d3d7fa700 1 -- 172.21.15.36:0/654782762 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d38005ba0 con 0x7f4d400b6660 2022-06-17T12:47:00.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.245+0000 7f4d3d7fa700 1 -- 172.21.15.36:0/654782762 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d38005d00 con 0x7f4d400b6660 2022-06-17T12:47:00.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.245+0000 7f4d4571e700 1 -- 172.21.15.36:0/654782762 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d4013d780 con 0x7f4d400b6660 2022-06-17T12:47:00.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.245+0000 7f4d3d7fa700 1 -- 172.21.15.36:0/654782762 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d380072c0 con 0x7f4d400b6660 2022-06-17T12:47:00.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.245+0000 7f4d4571e700 1 -- 172.21.15.36:0/654782762 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d40065a20 con 0x7f4d400b6660 2022-06-17T12:47:00.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.245+0000 7f4d4571e700 1 -- 172.21.15.36:0/654782762 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4d40153b40 con 0x7f4d400b6660 2022-06-17T12:47:00.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.249+0000 7f4d3d7fa700 1 -- 172.21.15.36:0/654782762 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f4d3801a060 con 0x7f4d400b6660 2022-06-17T12:47:00.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.249+0000 7f4d3d7fa700 1 --2- 172.21.15.36:0/654782762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d24053af0 0x7f4d24055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:00.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.249+0000 7f4d3d7fa700 1 -- 172.21.15.36:0/654782762 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f4d3806b840 con 0x7f4d400b6660 2022-06-17T12:47:00.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.249+0000 7f4d3ffff700 1 --2- 172.21.15.36:0/654782762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d24053af0 0x7f4d24055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:00.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.249+0000 7f4d3d7fa700 1 --2- 172.21.15.36:0/654782762 >> [v2:172.21.15.36:6818/22646,v1:172.21.15.36:6819/22646] conn(0x7f4d2405b470 0x7f4d2405d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:00.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.249+0000 7f4d3d7fa700 1 -- 172.21.15.36:0/654782762 --> [v2:172.21.15.36:6818/22646,v1:172.21.15.36:6819/22646] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4d2405def0 con 0x7f4d2405b470 2022-06-17T12:47:00.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.249+0000 7f4d3d7fa700 1 -- 172.21.15.36:0/654782762 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d38070240 con 0x7f4d400b6660 2022-06-17T12:47:00.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.249+0000 7f4d3f7fe700 1 --2- 172.21.15.36:0/654782762 >> [v2:172.21.15.36:6818/22646,v1:172.21.15.36:6819/22646] conn(0x7f4d2405b470 0x7f4d2405d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:00.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.249+0000 7f4d3ffff700 1 --2- 172.21.15.36:0/654782762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d24053af0 0x7f4d24055fb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f4d3000b000 tx=0x7f4d30006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:00.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.249+0000 7f4d3f7fe700 1 --2- 172.21.15.36:0/654782762 >> [v2:172.21.15.36:6818/22646,v1:172.21.15.36:6819/22646] conn(0x7f4d2405b470 0x7f4d2405d910 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:00.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.249+0000 7f4d3d7fa700 1 -- 172.21.15.36:0/654782762 <== osd.3 v2:172.21.15.36:6818/22646 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f4d2405def0 con 0x7f4d2405b470 2022-06-17T12:47:00.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.269+0000 7f4d4571e700 1 -- 172.21.15.36:0/654782762 --> [v2:172.21.15.36:6818/22646,v1:172.21.15.36:6819/22646] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4d400b3cf0 con 0x7f4d2405b470 2022-06-17T12:47:00.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.269+0000 7f4d3d7fa700 1 -- 172.21.15.36:0/654782762 <== osd.3 v2:172.21.15.36:6818/22646 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4d400b3cf0 con 0x7f4d2405b470 2022-06-17T12:47:00.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.269+0000 7f4d4571e700 1 -- 172.21.15.36:0/654782762 >> [v2:172.21.15.36:6818/22646,v1:172.21.15.36:6819/22646] conn(0x7f4d2405b470 msgr2=0x7f4d2405d910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:00.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.269+0000 7f4d4571e700 1 --2- 172.21.15.36:0/654782762 >> [v2:172.21.15.36:6818/22646,v1:172.21.15.36:6819/22646] conn(0x7f4d2405b470 0x7f4d2405d910 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:00.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.269+0000 7f4d4571e700 1 -- 172.21.15.36:0/654782762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d24053af0 msgr2=0x7f4d24055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:00.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.269+0000 7f4d4571e700 1 --2- 172.21.15.36:0/654782762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d24053af0 0x7f4d24055fb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f4d3000b000 tx=0x7f4d30006df0).stop 2022-06-17T12:47:00.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.269+0000 7f4d4571e700 1 -- 172.21.15.36:0/654782762 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d400b6660 msgr2=0x7f4d40153e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:00.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.269+0000 7f4d4571e700 1 --2- 172.21.15.36:0/654782762 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d400b6660 0x7f4d40153e20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f4d38000f40 tx=0x7f4d38007be0).stop 2022-06-17T12:47:00.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.269+0000 7f4d44f1d700 1 -- 172.21.15.36:0/654782762 reap_dead start 2022-06-17T12:47:00.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.269+0000 7f4d4571e700 1 -- 172.21.15.36:0/654782762 shutdown_connections 2022-06-17T12:47:00.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.269+0000 7f4d4571e700 1 -- 172.21.15.36:0/654782762 >> 172.21.15.36:0/654782762 conn(0x7f4d400614b0 msgr2=0x7f4d400b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:00.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.269+0000 7f4d4571e700 1 -- 172.21.15.36:0/654782762 shutdown_connections 2022-06-17T12:47:00.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.269+0000 7f4d4571e700 1 -- 172.21.15.36:0/654782762 wait complete. 2022-06-17T12:47:00.330 INFO:teuthology.orchestra.run.smithi036.stdout:60129542151 2022-06-17T12:47:00.332 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2022-06-17T12:47:00.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.465+0000 7fd1f5757700 1 Processor -- start 2022-06-17T12:47:00.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.465+0000 7fd1f5757700 1 -- start start 2022-06-17T12:47:00.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.465+0000 7fd1f5757700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd1f00b15b0 0x7fd1f00b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:00.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.465+0000 7fd1f5757700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f013fdf0 con 0x7fd1f00b65f0 2022-06-17T12:47:00.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.465+0000 7fd1f5757700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f0140090 con 0x7fd1f00adf10 2022-06-17T12:47:00.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.465+0000 7fd1f5757700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1f00ae8e0 con 0x7fd1f00b15b0 2022-06-17T12:47:00.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.465+0000 7fd1f4f56700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd1f00b15b0 0x7fd1f00b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:00.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.465+0000 7fd1effff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fd1f00b65f0 0x7fd1f0140530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:48972/0 (socket says 172.21.15.36:48972) 2022-06-17T12:47:00.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.465+0000 7fd1f4f56700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd1f00b15b0 0x7fd1f00b69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59676/0 (socket says 172.21.15.36:59676) 2022-06-17T12:47:00.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.465+0000 7fd1effff700 1 -- 172.21.15.36:0/3242505835 learned_addr learned my addr 172.21.15.36:0/3242505835 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:00.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.465+0000 7fd1f4f56700 1 -- 172.21.15.36:0/3242505835 >> v1:172.21.15.70:6789/0 conn(0x7fd1f00adf10 legacy=0x7fd1f00af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:00.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.465+0000 7fd1f4f56700 1 -- 172.21.15.36:0/3242505835 >> v1:172.21.15.36:6789/0 conn(0x7fd1f00b65f0 legacy=0x7fd1f0140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:00.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.465+0000 7fd1f4f56700 1 -- 172.21.15.36:0/3242505835 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f00b1be0 con 0x7fd1f00b15b0 2022-06-17T12:47:00.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.465+0000 7fd1eeffd700 1 -- 172.21.15.36:0/3242505835 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048298058 0 0) 0x7fd1f013fdf0 con 0x7fd1f00b65f0 2022-06-17T12:47:00.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.465+0000 7fd1f4f56700 1 --2- 172.21.15.36:0/3242505835 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1f00b15b0 0x7fd1f00b69d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fd1e8002f70 tx=0x7fd1e8006df0).ready entity=mon.0 client_cookie=c466856abbeb6a3b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:00.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.465+0000 7fd1eeffd700 1 -- 172.21.15.36:0/3242505835 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1e800d610 con 0x7fd1f00b15b0 2022-06-17T12:47:00.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.465+0000 7fd1eeffd700 1 -- 172.21.15.36:0/3242505835 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e8000e90 con 0x7fd1f00b15b0 2022-06-17T12:47:00.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.465+0000 7fd1eeffd700 1 -- 172.21.15.36:0/3242505835 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1e800d960 con 0x7fd1f00b15b0 2022-06-17T12:47:00.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.469+0000 7fd1f5757700 1 -- 172.21.15.36:0/3242505835 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1f00b15b0 msgr2=0x7fd1f00b69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:00.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.469+0000 7fd1f5757700 1 --2- 172.21.15.36:0/3242505835 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1f00b15b0 0x7fd1f00b69d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fd1e8002f70 tx=0x7fd1e8006df0).stop 2022-06-17T12:47:00.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.469+0000 7fd1f5757700 1 -- 172.21.15.36:0/3242505835 shutdown_connections 2022-06-17T12:47:00.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.469+0000 7fd1f5757700 1 --2- 172.21.15.36:0/3242505835 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1f00b15b0 0x7fd1f00b69d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:00.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.469+0000 7fd1f5757700 1 -- 172.21.15.36:0/3242505835 >> 172.21.15.36:0/3242505835 conn(0x7fd1f00614b0 msgr2=0x7fd1f0061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:00.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.469+0000 7fd1f5757700 1 -- 172.21.15.36:0/3242505835 shutdown_connections 2022-06-17T12:47:00.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.469+0000 7fd1f5757700 1 -- 172.21.15.36:0/3242505835 wait complete. 2022-06-17T12:47:00.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.469+0000 7fd1f5757700 1 Processor -- start 2022-06-17T12:47:00.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.469+0000 7fd1f5757700 1 -- start start 2022-06-17T12:47:00.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.469+0000 7fd1f5757700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd1f00adf10 0x7fd1f010f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:00.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.469+0000 7fd1f5757700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1f00b15b0 0x7fd1f0145aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:00.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.469+0000 7fd1f5757700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1f00b65f0 0x7fd1f01390c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:00.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.469+0000 7fd1f5757700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f0066eb0 con 0x7fd1f00b15b0 2022-06-17T12:47:00.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.469+0000 7fd1f5757700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f0063d30 con 0x7fd1f00b65f0 2022-06-17T12:47:00.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.469+0000 7fd1f5757700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f013f5e0 con 0x7fd1f00adf10 2022-06-17T12:47:00.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.469+0000 7fd1ef7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1f00b15b0 0x7fd1f0145aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:00.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.469+0000 7fd1ef7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1f00b15b0 0x7fd1f0145aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59680/0 (socket says 172.21.15.36:59680) 2022-06-17T12:47:00.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.469+0000 7fd1ef7fe700 1 -- 172.21.15.36:0/1424296203 learned_addr learned my addr 172.21.15.36:0/1424296203 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:00.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.469+0000 7fd1effff700 1 --2- 172.21.15.36:0/1424296203 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd1f00adf10 0x7fd1f010f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:00.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.473+0000 7fd1f4f56700 1 --2- 172.21.15.36:0/1424296203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1f00b65f0 0x7fd1f01390c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:00.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.473+0000 7fd1ef7fe700 1 -- 172.21.15.36:0/1424296203 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd1f00adf10 msgr2=0x7fd1f010f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:00.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.473+0000 7fd1ef7fe700 1 --2- 172.21.15.36:0/1424296203 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd1f00adf10 0x7fd1f010f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:00.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.473+0000 7fd1ef7fe700 1 -- 172.21.15.36:0/1424296203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1f00b65f0 msgr2=0x7fd1f01390c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:00.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.473+0000 7fd1ef7fe700 1 --2- 172.21.15.36:0/1424296203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1f00b65f0 0x7fd1f01390c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:00.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.473+0000 7fd1ef7fe700 1 -- 172.21.15.36:0/1424296203 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e800b180 con 0x7fd1f00b15b0 2022-06-17T12:47:00.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.473+0000 7fd1ef7fe700 1 --2- 172.21.15.36:0/1424296203 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1f00b15b0 0x7fd1f0145aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fd1f00b4010 tx=0x7fd1d8007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:00.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.473+0000 7fd1ed7fa700 1 -- 172.21.15.36:0/1424296203 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1d80068e0 con 0x7fd1f00b15b0 2022-06-17T12:47:00.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.473+0000 7fd1ed7fa700 1 -- 172.21.15.36:0/1424296203 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d8006a40 con 0x7fd1f00b15b0 2022-06-17T12:47:00.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.473+0000 7fd1ed7fa700 1 -- 172.21.15.36:0/1424296203 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1d8006da0 con 0x7fd1f00b15b0 2022-06-17T12:47:00.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.473+0000 7fd1f5757700 1 -- 172.21.15.36:0/1424296203 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f0066c10 con 0x7fd1f00b15b0 2022-06-17T12:47:00.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.473+0000 7fd1f5757700 1 -- 172.21.15.36:0/1424296203 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f013ea20 con 0x7fd1f00b15b0 2022-06-17T12:47:00.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.473+0000 7fd1f5757700 1 -- 172.21.15.36:0/1424296203 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd1f0156270 con 0x7fd1f00b15b0 2022-06-17T12:47:00.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.473+0000 7fd1ed7fa700 1 -- 172.21.15.36:0/1424296203 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd1d8010070 con 0x7fd1f00b15b0 2022-06-17T12:47:00.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.477+0000 7fd1ed7fa700 1 --2- 172.21.15.36:0/1424296203 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd1d4053ad0 0x7fd1d4055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:00.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.477+0000 7fd1ed7fa700 1 -- 172.21.15.36:0/1424296203 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd1d806b5d0 con 0x7fd1f00b15b0 2022-06-17T12:47:00.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.477+0000 7fd1effff700 1 --2- 172.21.15.36:0/1424296203 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd1d4053ad0 0x7fd1d4055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:00.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.477+0000 7fd1ed7fa700 1 --2- 172.21.15.36:0/1424296203 >> [v2:172.21.15.70:6808/22812,v1:172.21.15.70:6809/22812] conn(0x7fd1d405b450 0x7fd1d405d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:00.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.477+0000 7fd1ed7fa700 1 -- 172.21.15.36:0/1424296203 --> [v2:172.21.15.70:6808/22812,v1:172.21.15.70:6809/22812] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd1d405ded0 con 0x7fd1d405b450 2022-06-17T12:47:00.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.477+0000 7fd1ed7fa700 1 -- 172.21.15.36:0/1424296203 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1d806dc40 con 0x7fd1f00b15b0 2022-06-17T12:47:00.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.477+0000 7fd1f4f56700 1 --2- 172.21.15.36:0/1424296203 >> [v2:172.21.15.70:6808/22812,v1:172.21.15.70:6809/22812] conn(0x7fd1d405b450 0x7fd1d405d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:00.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.477+0000 7fd1effff700 1 --2- 172.21.15.36:0/1424296203 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd1d4053ad0 0x7fd1d4055f90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fd1e000d000 tx=0x7fd1e0007c00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:00.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.477+0000 7fd1f4f56700 1 --2- 172.21.15.36:0/1424296203 >> [v2:172.21.15.70:6808/22812,v1:172.21.15.70:6809/22812] conn(0x7fd1d405b450 0x7fd1d405d8f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:00.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.477+0000 7fd1ed7fa700 1 -- 172.21.15.36:0/1424296203 <== osd.4 v2:172.21.15.70:6808/22812 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7fd1d405ded0 con 0x7fd1d405b450 2022-06-17T12:47:00.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.497+0000 7fd1f5757700 1 -- 172.21.15.36:0/1424296203 --> [v2:172.21.15.70:6808/22812,v1:172.21.15.70:6809/22812] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd1f013fb20 con 0x7fd1d405b450 2022-06-17T12:47:00.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.497+0000 7fd1ed7fa700 1 -- 172.21.15.36:0/1424296203 <== osd.4 v2:172.21.15.70:6808/22812 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd1f013fb20 con 0x7fd1d405b450 2022-06-17T12:47:00.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.497+0000 7fd1f5757700 1 -- 172.21.15.36:0/1424296203 >> [v2:172.21.15.70:6808/22812,v1:172.21.15.70:6809/22812] conn(0x7fd1d405b450 msgr2=0x7fd1d405d8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:00.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.497+0000 7fd1f5757700 1 --2- 172.21.15.36:0/1424296203 >> [v2:172.21.15.70:6808/22812,v1:172.21.15.70:6809/22812] conn(0x7fd1d405b450 0x7fd1d405d8f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:00.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.497+0000 7fd1f5757700 1 -- 172.21.15.36:0/1424296203 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd1d4053ad0 msgr2=0x7fd1d4055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:00.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.497+0000 7fd1f5757700 1 --2- 172.21.15.36:0/1424296203 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd1d4053ad0 0x7fd1d4055f90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fd1e000d000 tx=0x7fd1e0007c00).stop 2022-06-17T12:47:00.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.497+0000 7fd1f5757700 1 -- 172.21.15.36:0/1424296203 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1f00b15b0 msgr2=0x7fd1f0145aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:00.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.497+0000 7fd1f5757700 1 --2- 172.21.15.36:0/1424296203 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1f00b15b0 0x7fd1f0145aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fd1f00b4010 tx=0x7fd1d8007f90).stop 2022-06-17T12:47:00.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.497+0000 7fd1f4f56700 1 -- 172.21.15.36:0/1424296203 reap_dead start 2022-06-17T12:47:00.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.497+0000 7fd1f5757700 1 -- 172.21.15.36:0/1424296203 shutdown_connections 2022-06-17T12:47:00.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.497+0000 7fd1f5757700 1 -- 172.21.15.36:0/1424296203 >> 172.21.15.36:0/1424296203 conn(0x7fd1f00614b0 msgr2=0x7fd1f00b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:00.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.497+0000 7fd1f5757700 1 -- 172.21.15.36:0/1424296203 shutdown_connections 2022-06-17T12:47:00.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.497+0000 7fd1f5757700 1 -- 172.21.15.36:0/1424296203 wait complete. 2022-06-17T12:47:00.767 INFO:teuthology.orchestra.run.smithi036.stdout:55834574854 2022-06-17T12:47:00.768 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2022-06-17T12:47:00.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.897+0000 7f5b6ee95700 1 Processor -- start 2022-06-17T12:47:00.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.897+0000 7f5b6ee95700 1 -- start start 2022-06-17T12:47:00.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.897+0000 7f5b6ee95700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5b680ad140 0x7f5b680af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:00.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.897+0000 7f5b6ee95700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b680ac340 0x7f5b680ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:00.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.897+0000 7f5b6ee95700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b680ac870 con 0x7f5b6814aca0 2022-06-17T12:47:00.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.897+0000 7f5b6ee95700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b68148960 con 0x7f5b680ad140 2022-06-17T12:47:00.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.897+0000 7f5b6ee95700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b680acaa0 con 0x7f5b680ac340 2022-06-17T12:47:00.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.897+0000 7f5b6cc31700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5b680ad140 0x7f5b680af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:00.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.897+0000 7f5b6cc31700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5b680ad140 0x7f5b680af7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59688/0 (socket says 172.21.15.36:59688) 2022-06-17T12:47:00.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.897+0000 7f5b6cc31700 1 -- 172.21.15.36:0/1436785322 learned_addr learned my addr 172.21.15.36:0/1436785322 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:00.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.901+0000 7f5b67fff700 1 --2- 172.21.15.36:0/1436785322 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b680ac340 0x7f5b680ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:00.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.901+0000 7f5b6cc31700 1 -- 172.21.15.36:0/1436785322 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b680ac340 msgr2=0x7f5b680ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:00.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.901+0000 7f5b6cc31700 1 --2- 172.21.15.36:0/1436785322 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b680ac340 0x7f5b680ad520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:00.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.901+0000 7f5b6cc31700 1 -- 172.21.15.36:0/1436785322 >> v1:172.21.15.70:6789/0 conn(0x7f5b6814aca0 legacy=0x7f5b6814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:00.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.901+0000 7f5b6cc31700 1 -- 172.21.15.36:0/1436785322 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b680b0bf0 con 0x7f5b680ad140 2022-06-17T12:47:00.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.901+0000 7f5b6cc31700 1 --2- 172.21.15.36:0/1436785322 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5b680ad140 0x7f5b680af7f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f5b5c002f70 tx=0x7f5b5c006eb0).ready entity=mon.0 client_cookie=a90cc71c6797de55 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:00.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.901+0000 7f5b677fe700 1 -- 172.21.15.36:0/1436785322 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b5c00d570 con 0x7f5b680ad140 2022-06-17T12:47:00.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.901+0000 7f5b677fe700 1 -- 172.21.15.36:0/1436785322 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5c000e90 con 0x7f5b680ad140 2022-06-17T12:47:00.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.901+0000 7f5b677fe700 1 -- 172.21.15.36:0/1436785322 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b5c00d900 con 0x7f5b680ad140 2022-06-17T12:47:00.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.901+0000 7f5b6ee95700 1 -- 172.21.15.36:0/1436785322 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5b680ad140 msgr2=0x7f5b680af7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:00.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.901+0000 7f5b6ee95700 1 --2- 172.21.15.36:0/1436785322 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5b680ad140 0x7f5b680af7f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f5b5c002f70 tx=0x7f5b5c006eb0).stop 2022-06-17T12:47:01.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.901+0000 7f5b6ee95700 1 -- 172.21.15.36:0/1436785322 shutdown_connections 2022-06-17T12:47:01.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.901+0000 7f5b6ee95700 1 --2- 172.21.15.36:0/1436785322 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b680ac340 0x7f5b680ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:01.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.901+0000 7f5b6ee95700 1 --2- 172.21.15.36:0/1436785322 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5b680ad140 0x7f5b680af7f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:01.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.901+0000 7f5b6ee95700 1 -- 172.21.15.36:0/1436785322 >> 172.21.15.36:0/1436785322 conn(0x7f5b680b5430 msgr2=0x7f5b680b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:01.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.901+0000 7f5b6ee95700 1 -- 172.21.15.36:0/1436785322 shutdown_connections 2022-06-17T12:47:01.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.901+0000 7f5b6ee95700 1 -- 172.21.15.36:0/1436785322 wait complete. 2022-06-17T12:47:01.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.905+0000 7f5b6ee95700 1 Processor -- start 2022-06-17T12:47:01.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.905+0000 7f5b6ee95700 1 -- start start 2022-06-17T12:47:01.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.905+0000 7f5b6ee95700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b680ac340 0x7f5b680a8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:01.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.905+0000 7f5b6ee95700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5b680ad140 0x7f5b680aba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:01.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.905+0000 7f5b6ee95700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b6814aca0 0x7f5b680a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:01.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.905+0000 7f5b6ee95700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b680ab330 con 0x7f5b680ad140 2022-06-17T12:47:01.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.905+0000 7f5b6ee95700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b680ab5d0 con 0x7f5b6814aca0 2022-06-17T12:47:01.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.905+0000 7f5b6ee95700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b680ab060 con 0x7f5b680ac340 2022-06-17T12:47:01.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.905+0000 7f5b67fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5b680ad140 0x7f5b680aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:01.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.905+0000 7f5b67fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5b680ad140 0x7f5b680aba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59694/0 (socket says 172.21.15.36:59694) 2022-06-17T12:47:01.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.905+0000 7f5b67fff700 1 -- 172.21.15.36:0/4054639385 learned_addr learned my addr 172.21.15.36:0/4054639385 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:01.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.905+0000 7f5b6cc31700 1 --2- 172.21.15.36:0/4054639385 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b680ac340 0x7f5b680a8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:01.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.905+0000 7f5b6d432700 1 --2- 172.21.15.36:0/4054639385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b6814aca0 0x7f5b680a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:01.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.905+0000 7f5b67fff700 1 -- 172.21.15.36:0/4054639385 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b680ac340 msgr2=0x7f5b680a8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:01.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.905+0000 7f5b67fff700 1 --2- 172.21.15.36:0/4054639385 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b680ac340 0x7f5b680a8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:01.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.905+0000 7f5b67fff700 1 -- 172.21.15.36:0/4054639385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b6814aca0 msgr2=0x7f5b680a75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:01.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.905+0000 7f5b67fff700 1 --2- 172.21.15.36:0/4054639385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b6814aca0 0x7f5b680a75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:01.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.905+0000 7f5b67fff700 1 -- 172.21.15.36:0/4054639385 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5c00b1e0 con 0x7f5b680ad140 2022-06-17T12:47:01.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.905+0000 7f5b6cc31700 1 --2- 172.21.15.36:0/4054639385 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b680ac340 0x7f5b680a8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:01.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.905+0000 7f5b67fff700 1 --2- 172.21.15.36:0/4054639385 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5b680ad140 0x7f5b680aba70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f5b5800c000 tx=0x7f5b58006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:01.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.905+0000 7f5b65ffb700 1 -- 172.21.15.36:0/4054639385 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b58010780 con 0x7f5b680ad140 2022-06-17T12:47:01.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.905+0000 7f5b65ffb700 1 -- 172.21.15.36:0/4054639385 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b580108e0 con 0x7f5b680ad140 2022-06-17T12:47:01.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.905+0000 7f5b6ee95700 1 -- 172.21.15.36:0/4054639385 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b680a7ab0 con 0x7f5b680ad140 2022-06-17T12:47:01.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.905+0000 7f5b65ffb700 1 -- 172.21.15.36:0/4054639385 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b58010c40 con 0x7f5b680ad140 2022-06-17T12:47:01.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.909+0000 7f5b6ee95700 1 -- 172.21.15.36:0/4054639385 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b6815a300 con 0x7f5b680ad140 2022-06-17T12:47:01.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.909+0000 7f5b6ee95700 1 -- 172.21.15.36:0/4054639385 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5b680ab790 con 0x7f5b680ad140 2022-06-17T12:47:01.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.909+0000 7f5b65ffb700 1 -- 172.21.15.36:0/4054639385 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f5b5801c460 con 0x7f5b680ad140 2022-06-17T12:47:01.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.909+0000 7f5b65ffb700 1 --2- 172.21.15.36:0/4054639385 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5b48053af0 0x7f5b48055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:01.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.909+0000 7f5b65ffb700 1 -- 172.21.15.36:0/4054639385 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5b5806c4a0 con 0x7f5b680ad140 2022-06-17T12:47:01.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.909+0000 7f5b6cc31700 1 --2- 172.21.15.36:0/4054639385 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5b48053af0 0x7f5b48055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:01.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.909+0000 7f5b65ffb700 1 --2- 172.21.15.36:0/4054639385 >> [v2:172.21.15.70:6800/22813,v1:172.21.15.70:6801/22813] conn(0x7f5b4805b470 0x7f5b4805d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:01.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.909+0000 7f5b65ffb700 1 -- 172.21.15.36:0/4054639385 --> [v2:172.21.15.70:6800/22813,v1:172.21.15.70:6801/22813] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5b4805def0 con 0x7f5b4805b470 2022-06-17T12:47:01.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.909+0000 7f5b65ffb700 1 -- 172.21.15.36:0/4054639385 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b5806eb10 con 0x7f5b680ad140 2022-06-17T12:47:01.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.909+0000 7f5b6d432700 1 --2- 172.21.15.36:0/4054639385 >> [v2:172.21.15.70:6800/22813,v1:172.21.15.70:6801/22813] conn(0x7f5b4805b470 0x7f5b4805d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:01.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.909+0000 7f5b6cc31700 1 --2- 172.21.15.36:0/4054639385 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5b48053af0 0x7f5b48055fb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f5b5c01a000 tx=0x7f5b5c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:01.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.909+0000 7f5b6d432700 1 --2- 172.21.15.36:0/4054639385 >> [v2:172.21.15.70:6800/22813,v1:172.21.15.70:6801/22813] conn(0x7f5b4805b470 0x7f5b4805d910 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:01.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.913+0000 7f5b65ffb700 1 -- 172.21.15.36:0/4054639385 <== osd.5 v2:172.21.15.70:6800/22813 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f5b4805def0 con 0x7f5b4805b470 2022-06-17T12:47:01.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.933+0000 7f5b6ee95700 1 -- 172.21.15.36:0/4054639385 --> [v2:172.21.15.70:6800/22813,v1:172.21.15.70:6801/22813] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5b680b1500 con 0x7f5b4805b470 2022-06-17T12:47:01.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.933+0000 7f5b65ffb700 1 -- 172.21.15.36:0/4054639385 <== osd.5 v2:172.21.15.70:6800/22813 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5b680b1500 con 0x7f5b4805b470 2022-06-17T12:47:01.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.933+0000 7f5b6ee95700 1 -- 172.21.15.36:0/4054639385 >> [v2:172.21.15.70:6800/22813,v1:172.21.15.70:6801/22813] conn(0x7f5b4805b470 msgr2=0x7f5b4805d910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:01.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.933+0000 7f5b6ee95700 1 --2- 172.21.15.36:0/4054639385 >> [v2:172.21.15.70:6800/22813,v1:172.21.15.70:6801/22813] conn(0x7f5b4805b470 0x7f5b4805d910 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:01.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.933+0000 7f5b6ee95700 1 -- 172.21.15.36:0/4054639385 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5b48053af0 msgr2=0x7f5b48055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:01.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.933+0000 7f5b6ee95700 1 --2- 172.21.15.36:0/4054639385 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5b48053af0 0x7f5b48055fb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f5b5c01a000 tx=0x7f5b5c015000).stop 2022-06-17T12:47:01.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.933+0000 7f5b6ee95700 1 -- 172.21.15.36:0/4054639385 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5b680ad140 msgr2=0x7f5b680aba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:01.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.933+0000 7f5b6ee95700 1 --2- 172.21.15.36:0/4054639385 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5b680ad140 0x7f5b680aba70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f5b5800c000 tx=0x7f5b58006df0).stop 2022-06-17T12:47:01.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.933+0000 7f5b6d432700 1 -- 172.21.15.36:0/4054639385 reap_dead start 2022-06-17T12:47:01.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.933+0000 7f5b6ee95700 1 -- 172.21.15.36:0/4054639385 shutdown_connections 2022-06-17T12:47:01.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.933+0000 7f5b6ee95700 1 -- 172.21.15.36:0/4054639385 >> 172.21.15.36:0/4054639385 conn(0x7f5b680b5430 msgr2=0x7f5b68147dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:01.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.933+0000 7f5b6ee95700 1 -- 172.21.15.36:0/4054639385 shutdown_connections 2022-06-17T12:47:01.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:00.933+0000 7f5b6ee95700 1 -- 172.21.15.36:0/4054639385 wait complete. 2022-06-17T12:47:01.273 INFO:teuthology.orchestra.run.smithi036.stdout:55834574855 2022-06-17T12:47:01.281 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2022-06-17T12:47:01.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.429+0000 7f3f107e2700 1 Processor -- start 2022-06-17T12:47:01.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.429+0000 7f3f107e2700 1 -- start start 2022-06-17T12:47:01.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.429+0000 7f3f107e2700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f080ad140 0x7f3f080af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:01.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.429+0000 7f3f107e2700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f3f080ac340 0x7f3f080ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:01.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.429+0000 7f3f107e2700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f080ac870 con 0x7f3f0814aca0 2022-06-17T12:47:01.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.429+0000 7f3f107e2700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f08148960 con 0x7f3f080ac340 2022-06-17T12:47:01.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.429+0000 7f3f107e2700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f080acaa0 con 0x7f3f080ad140 2022-06-17T12:47:01.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.429+0000 7f3f0dd7d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f3f080ac340 0x7f3f080ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:01.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.429+0000 7f3f0dd7d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f3f080ac340 0x7f3f080ad520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59702/0 (socket says 172.21.15.36:59702) 2022-06-17T12:47:01.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.429+0000 7f3f0dd7d700 1 -- 172.21.15.36:0/1634161375 learned_addr learned my addr 172.21.15.36:0/1634161375 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:01.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.429+0000 7f3f0e57e700 1 --2- 172.21.15.36:0/1634161375 >> v2:172.21.15.70:3300/0 conn(0x7f3f080ad140 0x7f3f080af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:01.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.433+0000 7f3f0dd7d700 1 -- 172.21.15.36:0/1634161375 >> v2:172.21.15.70:3300/0 conn(0x7f3f080ad140 msgr2=0x7f3f080af7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:01.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.433+0000 7f3f0dd7d700 1 --2- 172.21.15.36:0/1634161375 >> v2:172.21.15.70:3300/0 conn(0x7f3f080ad140 0x7f3f080af7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:01.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.433+0000 7f3f0dd7d700 1 -- 172.21.15.36:0/1634161375 >> v1:172.21.15.70:6789/0 conn(0x7f3f0814aca0 legacy=0x7f3f0814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:01.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.433+0000 7f3f0dd7d700 1 -- 172.21.15.36:0/1634161375 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f080b0bf0 con 0x7f3f080ac340 2022-06-17T12:47:01.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.433+0000 7f3f0dd7d700 1 --2- 172.21.15.36:0/1634161375 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3f080ac340 0x7f3f080ad520 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f3f04002f70 tx=0x7f3f04006eb0).ready entity=mon.0 client_cookie=7c7220e25fd330de server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:01.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.433+0000 7f3f0d57c700 1 -- 172.21.15.36:0/1634161375 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f0400d5c0 con 0x7f3f080ac340 2022-06-17T12:47:01.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.433+0000 7f3f0d57c700 1 -- 172.21.15.36:0/1634161375 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f04000e90 con 0x7f3f080ac340 2022-06-17T12:47:01.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.433+0000 7f3f0d57c700 1 -- 172.21.15.36:0/1634161375 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f0400d950 con 0x7f3f080ac340 2022-06-17T12:47:01.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.433+0000 7f3f107e2700 1 -- 172.21.15.36:0/1634161375 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3f080ac340 msgr2=0x7f3f080ad520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:01.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.433+0000 7f3f107e2700 1 --2- 172.21.15.36:0/1634161375 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3f080ac340 0x7f3f080ad520 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f3f04002f70 tx=0x7f3f04006eb0).stop 2022-06-17T12:47:01.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.433+0000 7f3f107e2700 1 -- 172.21.15.36:0/1634161375 shutdown_connections 2022-06-17T12:47:01.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.433+0000 7f3f107e2700 1 --2- 172.21.15.36:0/1634161375 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3f080ac340 0x7f3f080ad520 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:01.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.433+0000 7f3f107e2700 1 --2- 172.21.15.36:0/1634161375 >> v2:172.21.15.70:3300/0 conn(0x7f3f080ad140 0x7f3f080af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:01.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.433+0000 7f3f107e2700 1 -- 172.21.15.36:0/1634161375 >> 172.21.15.36:0/1634161375 conn(0x7f3f080b5430 msgr2=0x7f3f080b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:01.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.433+0000 7f3f107e2700 1 -- 172.21.15.36:0/1634161375 shutdown_connections 2022-06-17T12:47:01.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.433+0000 7f3f107e2700 1 -- 172.21.15.36:0/1634161375 wait complete. 2022-06-17T12:47:01.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.437+0000 7f3f107e2700 1 Processor -- start 2022-06-17T12:47:01.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.437+0000 7f3f107e2700 1 -- start start 2022-06-17T12:47:01.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.437+0000 7f3f107e2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f080ac340 0x7f3f080a9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:01.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.437+0000 7f3f107e2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3f080ad140 0x7f3f08156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:01.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.437+0000 7f3f107e2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f0814aca0 0x7f3f08063610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:01.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.437+0000 7f3f107e2700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f080ab6c0 con 0x7f3f080ad140 2022-06-17T12:47:01.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.437+0000 7f3f107e2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f080ab420 con 0x7f3f080ac340 2022-06-17T12:47:01.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.437+0000 7f3f107e2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f08063170 con 0x7f3f0814aca0 2022-06-17T12:47:01.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.437+0000 7f3f0dd7d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3f080ad140 0x7f3f08156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:01.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.437+0000 7f3f0dd7d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3f080ad140 0x7f3f08156a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59710/0 (socket says 172.21.15.36:59710) 2022-06-17T12:47:01.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.437+0000 7f3f0dd7d700 1 -- 172.21.15.36:0/1116580250 learned_addr learned my addr 172.21.15.36:0/1116580250 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:01.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.437+0000 7f3f0e57e700 1 --2- 172.21.15.36:0/1116580250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f080ac340 0x7f3f080a9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:01.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.437+0000 7f3f0ed7f700 1 --2- 172.21.15.36:0/1116580250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f0814aca0 0x7f3f08063610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:01.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.437+0000 7f3f0dd7d700 1 -- 172.21.15.36:0/1116580250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f0814aca0 msgr2=0x7f3f08063610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:01.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.437+0000 7f3f0dd7d700 1 --2- 172.21.15.36:0/1116580250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f0814aca0 0x7f3f08063610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:01.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.437+0000 7f3f0dd7d700 1 -- 172.21.15.36:0/1116580250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f080ac340 msgr2=0x7f3f080a9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:01.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.437+0000 7f3f0dd7d700 1 --2- 172.21.15.36:0/1116580250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f080ac340 0x7f3f080a9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:01.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.437+0000 7f3f0dd7d700 1 -- 172.21.15.36:0/1116580250 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0400b1e0 con 0x7f3f080ad140 2022-06-17T12:47:01.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.441+0000 7f3f0dd7d700 1 --2- 172.21.15.36:0/1116580250 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3f080ad140 0x7f3f08156a90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f3f04002f70 tx=0x7f3f04015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:01.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.441+0000 7f3ef77fe700 1 -- 172.21.15.36:0/1116580250 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f04011860 con 0x7f3f080ad140 2022-06-17T12:47:01.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.441+0000 7f3ef77fe700 1 -- 172.21.15.36:0/1116580250 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f040119c0 con 0x7f3f080ad140 2022-06-17T12:47:01.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.441+0000 7f3f107e2700 1 -- 172.21.15.36:0/1116580250 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f080af310 con 0x7f3f080ad140 2022-06-17T12:47:01.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.441+0000 7f3ef77fe700 1 -- 172.21.15.36:0/1116580250 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f04011d20 con 0x7f3f080ad140 2022-06-17T12:47:01.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.441+0000 7f3f107e2700 1 -- 172.21.15.36:0/1116580250 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f080a8fe0 con 0x7f3f080ad140 2022-06-17T12:47:01.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.441+0000 7f3f107e2700 1 -- 172.21.15.36:0/1116580250 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3f08063330 con 0x7f3f080ad140 2022-06-17T12:47:01.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.441+0000 7f3ef77fe700 1 -- 172.21.15.36:0/1116580250 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f3f04025030 con 0x7f3f080ad140 2022-06-17T12:47:01.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.441+0000 7f3ef77fe700 1 --2- 172.21.15.36:0/1116580250 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3ee8053940 0x7f3ee8055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:01.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.441+0000 7f3ef77fe700 1 -- 172.21.15.36:0/1116580250 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f3f0406d0c0 con 0x7f3f080ad140 2022-06-17T12:47:01.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.441+0000 7f3f0e57e700 1 --2- 172.21.15.36:0/1116580250 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3ee8053940 0x7f3ee8055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:01.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.441+0000 7f3ef77fe700 1 --2- 172.21.15.36:0/1116580250 >> [v2:172.21.15.70:6816/22822,v1:172.21.15.70:6817/22822] conn(0x7f3ee805b360 0x7f3ee805d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:01.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.441+0000 7f3ef77fe700 1 -- 172.21.15.36:0/1116580250 --> [v2:172.21.15.70:6816/22822,v1:172.21.15.70:6817/22822] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3ee805de80 con 0x7f3ee805b360 2022-06-17T12:47:01.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.441+0000 7f3ef77fe700 1 -- 172.21.15.36:0/1116580250 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f0406f750 con 0x7f3f080ad140 2022-06-17T12:47:01.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.441+0000 7f3f0ed7f700 1 --2- 172.21.15.36:0/1116580250 >> [v2:172.21.15.70:6816/22822,v1:172.21.15.70:6817/22822] conn(0x7f3ee805b360 0x7f3ee805d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:01.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.441+0000 7f3f0e57e700 1 --2- 172.21.15.36:0/1116580250 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3ee8053940 0x7f3ee8055e00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f3ef800b000 tx=0x7f3ef8006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:01.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.445+0000 7f3f0ed7f700 1 --2- 172.21.15.36:0/1116580250 >> [v2:172.21.15.70:6816/22822,v1:172.21.15.70:6817/22822] conn(0x7f3ee805b360 0x7f3ee805d800 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:01.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.445+0000 7f3ef77fe700 1 -- 172.21.15.36:0/1116580250 <== osd.6 v2:172.21.15.70:6816/22822 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f3ee805de80 con 0x7f3ee805b360 2022-06-17T12:47:01.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.465+0000 7f3f107e2700 1 -- 172.21.15.36:0/1116580250 --> [v2:172.21.15.70:6816/22822,v1:172.21.15.70:6817/22822] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3f080ab250 con 0x7f3ee805b360 2022-06-17T12:47:01.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.465+0000 7f3ef77fe700 1 -- 172.21.15.36:0/1116580250 <== osd.6 v2:172.21.15.70:6816/22822 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3f080ab250 con 0x7f3ee805b360 2022-06-17T12:47:01.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.465+0000 7f3f107e2700 1 -- 172.21.15.36:0/1116580250 >> [v2:172.21.15.70:6816/22822,v1:172.21.15.70:6817/22822] conn(0x7f3ee805b360 msgr2=0x7f3ee805d800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:01.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.465+0000 7f3f107e2700 1 --2- 172.21.15.36:0/1116580250 >> [v2:172.21.15.70:6816/22822,v1:172.21.15.70:6817/22822] conn(0x7f3ee805b360 0x7f3ee805d800 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:01.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.465+0000 7f3f107e2700 1 -- 172.21.15.36:0/1116580250 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3ee8053940 msgr2=0x7f3ee8055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:01.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.465+0000 7f3f107e2700 1 --2- 172.21.15.36:0/1116580250 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3ee8053940 0x7f3ee8055e00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f3ef800b000 tx=0x7f3ef8006df0).stop 2022-06-17T12:47:01.602 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.465+0000 7f3f107e2700 1 -- 172.21.15.36:0/1116580250 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3f080ad140 msgr2=0x7f3f08156a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:01.602 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.465+0000 7f3f107e2700 1 --2- 172.21.15.36:0/1116580250 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3f080ad140 0x7f3f08156a90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f3f04002f70 tx=0x7f3f04015000).stop 2022-06-17T12:47:01.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.465+0000 7f3f0ed7f700 1 -- 172.21.15.36:0/1116580250 reap_dead start 2022-06-17T12:47:01.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.465+0000 7f3f107e2700 1 -- 172.21.15.36:0/1116580250 shutdown_connections 2022-06-17T12:47:01.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.465+0000 7f3f107e2700 1 -- 172.21.15.36:0/1116580250 >> 172.21.15.36:0/1116580250 conn(0x7f3f080b5430 msgr2=0x7f3f08143c50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:01.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.465+0000 7f3f107e2700 1 -- 172.21.15.36:0/1116580250 shutdown_connections 2022-06-17T12:47:01.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.465+0000 7f3f107e2700 1 -- 172.21.15.36:0/1116580250 wait complete. 2022-06-17T12:47:01.605 INFO:teuthology.orchestra.run.smithi036.stdout:60129542150 2022-06-17T12:47:01.615 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2022-06-17T12:47:01.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.777+0000 7f97e048d700 1 Processor -- start 2022-06-17T12:47:01.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.777+0000 7f97e048d700 1 -- start start 2022-06-17T12:47:01.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97e048d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f97d80b1220 0x7f97d80626e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:01.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97e048d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97d80ac870 0x7f97d8062bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:01.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97e048d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f97d80acd80 0x7f97d80630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:01.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97e048d700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97d80ad490 con 0x7f97d80acd80 2022-06-17T12:47:01.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97e048d700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97d80b1600 con 0x7f97d80b1220 2022-06-17T12:47:01.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97e048d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d80abb70 con 0x7f97d80ac870 2022-06-17T12:47:01.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97dea2a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f97d80acd80 0x7f97d80630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:01.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97dea2a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f97d80acd80 0x7f97d80630c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59722/0 (socket says 172.21.15.36:59722) 2022-06-17T12:47:01.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97dea2a700 1 -- 172.21.15.36:0/1865572344 learned_addr learned my addr 172.21.15.36:0/1865572344 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:01.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97dda28700 1 --2- 172.21.15.36:0/1865572344 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97d80ac870 0x7f97d8062bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:01.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97de229700 1 --2- 172.21.15.36:0/1865572344 >> v2:172.21.15.70:3300/0 conn(0x7f97d80b1220 0x7f97d80626e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:01.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97dda28700 1 -- 172.21.15.36:0/1865572344 >> v2:172.21.15.70:3300/0 conn(0x7f97d80b1220 msgr2=0x7f97d80626e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:01.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97dda28700 1 --2- 172.21.15.36:0/1865572344 >> v2:172.21.15.70:3300/0 conn(0x7f97d80b1220 0x7f97d80626e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:01.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97dda28700 1 -- 172.21.15.36:0/1865572344 >> v2:172.21.15.36:3300/0 conn(0x7f97d80acd80 msgr2=0x7f97d80630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:01.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97dda28700 1 --2- 172.21.15.36:0/1865572344 >> v2:172.21.15.36:3300/0 conn(0x7f97d80acd80 0x7f97d80630c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:01.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97dda28700 1 -- 172.21.15.36:0/1865572344 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d80abe10 con 0x7f97d80ac870 2022-06-17T12:47:01.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97dea2a700 1 --2- 172.21.15.36:0/1865572344 >> v2:172.21.15.36:3300/0 conn(0x7f97d80acd80 0x7f97d80630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:01.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97dda28700 1 --2- 172.21.15.36:0/1865572344 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97d80ac870 0x7f97d8062bd0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f97c8002f70 tx=0x7f97c8006df0).ready entity=mon.2 client_cookie=fb866840fc3739ed server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:01.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97dd227700 1 -- 172.21.15.36:0/1865572344 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97c800d4a0 con 0x7f97d80ac870 2022-06-17T12:47:01.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97dd227700 1 -- 172.21.15.36:0/1865572344 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c8000e90 con 0x7f97d80ac870 2022-06-17T12:47:01.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97dd227700 1 -- 172.21.15.36:0/1865572344 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97c800d7f0 con 0x7f97d80ac870 2022-06-17T12:47:01.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97e048d700 1 -- 172.21.15.36:0/1865572344 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97d80ac870 msgr2=0x7f97d8062bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:01.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97e048d700 1 --2- 172.21.15.36:0/1865572344 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97d80ac870 0x7f97d8062bd0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f97c8002f70 tx=0x7f97c8006df0).stop 2022-06-17T12:47:01.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97e048d700 1 -- 172.21.15.36:0/1865572344 shutdown_connections 2022-06-17T12:47:01.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97e048d700 1 --2- 172.21.15.36:0/1865572344 >> v2:172.21.15.36:3300/0 conn(0x7f97d80acd80 0x7f97d80630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:01.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97e048d700 1 --2- 172.21.15.36:0/1865572344 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97d80ac870 0x7f97d8062bd0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:01.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97e048d700 1 --2- 172.21.15.36:0/1865572344 >> v2:172.21.15.70:3300/0 conn(0x7f97d80b1220 0x7f97d80626e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:01.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97e048d700 1 -- 172.21.15.36:0/1865572344 >> 172.21.15.36:0/1865572344 conn(0x7f97d80b5430 msgr2=0x7f97d80b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:01.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97e048d700 1 -- 172.21.15.36:0/1865572344 shutdown_connections 2022-06-17T12:47:01.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.781+0000 7f97e048d700 1 -- 172.21.15.36:0/1865572344 wait complete. 2022-06-17T12:47:01.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.785+0000 7f97e048d700 1 Processor -- start 2022-06-17T12:47:01.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.785+0000 7f97e048d700 1 -- start start 2022-06-17T12:47:01.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.785+0000 7f97e048d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97d80ac870 0x7f97d813a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:01.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.785+0000 7f97e048d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97d80acd80 0x7f97d81397d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:01.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.785+0000 7f97e048d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97d80b1220 0x7f97d813d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:01.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.785+0000 7f97e048d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d81426c0 con 0x7f97d80b1220 2022-06-17T12:47:01.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.785+0000 7f97e048d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d813ac00 con 0x7f97d80ac870 2022-06-17T12:47:01.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.785+0000 7f97e048d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d8142920 con 0x7f97d80acd80 2022-06-17T12:47:01.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.785+0000 7f97dea2a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97d80b1220 0x7f97d813d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:01.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.785+0000 7f97dda28700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97d80acd80 0x7f97d81397d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:01.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.785+0000 7f97dea2a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97d80b1220 0x7f97d813d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59728/0 (socket says 172.21.15.36:59728) 2022-06-17T12:47:01.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.785+0000 7f97de229700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97d80ac870 0x7f97d813a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:01.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.785+0000 7f97dea2a700 1 -- 172.21.15.36:0/3243151592 learned_addr learned my addr 172.21.15.36:0/3243151592 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:01.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.785+0000 7f97dea2a700 1 -- 172.21.15.36:0/3243151592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97d80acd80 msgr2=0x7f97d81397d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:01.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.785+0000 7f97dea2a700 1 --2- 172.21.15.36:0/3243151592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97d80acd80 0x7f97d81397d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:01.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.785+0000 7f97dea2a700 1 -- 172.21.15.36:0/3243151592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97d80ac870 msgr2=0x7f97d813a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:01.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.785+0000 7f97dea2a700 1 --2- 172.21.15.36:0/3243151592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97d80ac870 0x7f97d813a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:01.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.785+0000 7f97dea2a700 1 -- 172.21.15.36:0/3243151592 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d0009530 con 0x7f97d80b1220 2022-06-17T12:47:01.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.785+0000 7f97dea2a700 1 --2- 172.21.15.36:0/3243151592 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97d80b1220 0x7f97d813d180 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f97d000c000 tx=0x7f97d0009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:01.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.785+0000 7f97c77fe700 1 -- 172.21.15.36:0/3243151592 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97d000c850 con 0x7f97d80b1220 2022-06-17T12:47:01.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.785+0000 7f97c77fe700 1 -- 172.21.15.36:0/3243151592 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d000c9b0 con 0x7f97d80b1220 2022-06-17T12:47:01.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.785+0000 7f97c77fe700 1 -- 172.21.15.36:0/3243151592 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97d000cd10 con 0x7f97d80b1220 2022-06-17T12:47:01.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.785+0000 7f97e048d700 1 -- 172.21.15.36:0/3243151592 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c800b180 con 0x7f97d80b1220 2022-06-17T12:47:01.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.785+0000 7f97e048d700 1 -- 172.21.15.36:0/3243151592 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d813d670 con 0x7f97d80b1220 2022-06-17T12:47:01.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.789+0000 7f97e048d700 1 -- 172.21.15.36:0/3243151592 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f97d8142ae0 con 0x7f97d80b1220 2022-06-17T12:47:01.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.789+0000 7f97c77fe700 1 -- 172.21.15.36:0/3243151592 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f97d0022070 con 0x7f97d80b1220 2022-06-17T12:47:01.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.789+0000 7f97c77fe700 1 --2- 172.21.15.36:0/3243151592 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f97b80539d0 0x7f97b8055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:01.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.789+0000 7f97c77fe700 1 -- 172.21.15.36:0/3243151592 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f97d006c8d0 con 0x7f97d80b1220 2022-06-17T12:47:01.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.789+0000 7f97de229700 1 --2- 172.21.15.36:0/3243151592 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f97b80539d0 0x7f97b8055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:01.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.789+0000 7f97c77fe700 1 --2- 172.21.15.36:0/3243151592 >> [v2:172.21.15.70:6824/22837,v1:172.21.15.70:6825/22837] conn(0x7f97b805b350 0x7f97b805d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:01.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.789+0000 7f97c77fe700 1 -- 172.21.15.36:0/3243151592 --> [v2:172.21.15.70:6824/22837,v1:172.21.15.70:6825/22837] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f97b805de70 con 0x7f97b805b350 2022-06-17T12:47:01.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.789+0000 7f97c77fe700 1 -- 172.21.15.36:0/3243151592 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97d0071240 con 0x7f97d80b1220 2022-06-17T12:47:01.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.789+0000 7f97dda28700 1 --2- 172.21.15.36:0/3243151592 >> [v2:172.21.15.70:6824/22837,v1:172.21.15.70:6825/22837] conn(0x7f97b805b350 0x7f97b805d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:01.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.789+0000 7f97de229700 1 --2- 172.21.15.36:0/3243151592 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f97b80539d0 0x7f97b8055e90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f97d400b000 tx=0x7f97d4006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:01.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.793+0000 7f97dda28700 1 --2- 172.21.15.36:0/3243151592 >> [v2:172.21.15.70:6824/22837,v1:172.21.15.70:6825/22837] conn(0x7f97b805b350 0x7f97b805d7f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:01.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.793+0000 7f97c77fe700 1 -- 172.21.15.36:0/3243151592 <== osd.7 v2:172.21.15.70:6824/22837 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f97b805de70 con 0x7f97b805b350 2022-06-17T12:47:01.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.813+0000 7f97e048d700 1 -- 172.21.15.36:0/3243151592 --> [v2:172.21.15.70:6824/22837,v1:172.21.15.70:6825/22837] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f97d80ac0b0 con 0x7f97b805b350 2022-06-17T12:47:01.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.813+0000 7f97c77fe700 1 -- 172.21.15.36:0/3243151592 <== osd.7 v2:172.21.15.70:6824/22837 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f97d80ac0b0 con 0x7f97b805b350 2022-06-17T12:47:01.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.813+0000 7f97e048d700 1 -- 172.21.15.36:0/3243151592 >> [v2:172.21.15.70:6824/22837,v1:172.21.15.70:6825/22837] conn(0x7f97b805b350 msgr2=0x7f97b805d7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:01.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.813+0000 7f97e048d700 1 --2- 172.21.15.36:0/3243151592 >> [v2:172.21.15.70:6824/22837,v1:172.21.15.70:6825/22837] conn(0x7f97b805b350 0x7f97b805d7f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:01.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.813+0000 7f97e048d700 1 -- 172.21.15.36:0/3243151592 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f97b80539d0 msgr2=0x7f97b8055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:01.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.813+0000 7f97e048d700 1 --2- 172.21.15.36:0/3243151592 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f97b80539d0 0x7f97b8055e90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f97d400b000 tx=0x7f97d4006df0).stop 2022-06-17T12:47:01.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.813+0000 7f97e048d700 1 -- 172.21.15.36:0/3243151592 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97d80b1220 msgr2=0x7f97d813d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:01.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.813+0000 7f97e048d700 1 --2- 172.21.15.36:0/3243151592 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97d80b1220 0x7f97d813d180 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f97d000c000 tx=0x7f97d0009290).stop 2022-06-17T12:47:01.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.813+0000 7f97dea2a700 1 -- 172.21.15.36:0/3243151592 reap_dead start 2022-06-17T12:47:01.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.813+0000 7f97e048d700 1 -- 172.21.15.36:0/3243151592 shutdown_connections 2022-06-17T12:47:01.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.813+0000 7f97e048d700 1 -- 172.21.15.36:0/3243151592 >> 172.21.15.36:0/3243151592 conn(0x7f97d80b5430 msgr2=0x7f97d80afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:01.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.813+0000 7f97e048d700 1 -- 172.21.15.36:0/3243151592 shutdown_connections 2022-06-17T12:47:01.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.813+0000 7f97e048d700 1 -- 172.21.15.36:0/3243151592 wait complete. 2022-06-17T12:47:01.841 INFO:teuthology.orchestra.run.smithi036.stdout:60129542150 2022-06-17T12:47:01.841 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2022-06-17T12:47:01.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.981+0000 7f52fee7b700 1 Processor -- start 2022-06-17T12:47:01.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.981+0000 7f52fee7b700 1 -- start start 2022-06-17T12:47:01.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.981+0000 7f52fee7b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52f813cb70 0x7f52f8143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:01.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.981+0000 7f52fee7b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f52f813cf50 0x7f52f81442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:01.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.981+0000 7f52fee7b700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f80af4b0 con 0x7f52f80aef60 2022-06-17T12:47:01.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.981+0000 7f52fee7b700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f80ba500 con 0x7f52f813cf50 2022-06-17T12:47:01.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.981+0000 7f52fee7b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f8142f50 con 0x7f52f813cb70 2022-06-17T12:47:01.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.981+0000 7f52fe67a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f52f813cf50 0x7f52f81442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:01.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.981+0000 7f52fe67a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f52f813cf50 0x7f52f81442a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59738/0 (socket says 172.21.15.36:59738) 2022-06-17T12:47:01.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.981+0000 7f52fe67a700 1 -- 172.21.15.36:0/2405549284 learned_addr learned my addr 172.21.15.36:0/2405549284 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:01.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.981+0000 7f52fde79700 1 --2- 172.21.15.36:0/2405549284 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52f813cb70 0x7f52f8143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:01.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.981+0000 7f52fe67a700 1 -- 172.21.15.36:0/2405549284 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52f813cb70 msgr2=0x7f52f8143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:01.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.981+0000 7f52fe67a700 1 --2- 172.21.15.36:0/2405549284 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52f813cb70 0x7f52f8143690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:01.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.981+0000 7f52fe67a700 1 -- 172.21.15.36:0/2405549284 >> v1:172.21.15.70:6789/0 conn(0x7f52f80aef60 legacy=0x7f52f80b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:01.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.981+0000 7f52fe67a700 1 -- 172.21.15.36:0/2405549284 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f81431f0 con 0x7f52f813cf50 2022-06-17T12:47:01.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.981+0000 7f52fe67a700 1 --2- 172.21.15.36:0/2405549284 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52f813cf50 0x7f52f81442a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f52f00032f0 tx=0x7f52f0009290).ready entity=mon.0 client_cookie=298ac8038678496c server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:01.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.981+0000 7f52fce77700 1 -- 172.21.15.36:0/2405549284 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52f000b8a0 con 0x7f52f813cf50 2022-06-17T12:47:01.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.981+0000 7f52fce77700 1 -- 172.21.15.36:0/2405549284 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f000ba00 con 0x7f52f813cf50 2022-06-17T12:47:01.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.981+0000 7f52fce77700 1 -- 172.21.15.36:0/2405549284 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52f000bb70 con 0x7f52f813cf50 2022-06-17T12:47:02.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.985+0000 7f52fee7b700 1 -- 172.21.15.36:0/2405549284 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52f813cf50 msgr2=0x7f52f81442a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:02.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.985+0000 7f52fee7b700 1 --2- 172.21.15.36:0/2405549284 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52f813cf50 0x7f52f81442a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f52f00032f0 tx=0x7f52f0009290).stop 2022-06-17T12:47:02.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.985+0000 7f52fee7b700 1 -- 172.21.15.36:0/2405549284 shutdown_connections 2022-06-17T12:47:02.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.985+0000 7f52fee7b700 1 --2- 172.21.15.36:0/2405549284 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52f813cf50 0x7f52f81442a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.985+0000 7f52fee7b700 1 --2- 172.21.15.36:0/2405549284 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52f813cb70 0x7f52f8143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.985+0000 7f52fee7b700 1 -- 172.21.15.36:0/2405549284 >> 172.21.15.36:0/2405549284 conn(0x7f52f805c550 msgr2=0x7f52f80b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:02.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.985+0000 7f52fee7b700 1 -- 172.21.15.36:0/2405549284 shutdown_connections 2022-06-17T12:47:02.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.985+0000 7f52fee7b700 1 -- 172.21.15.36:0/2405549284 wait complete. 2022-06-17T12:47:02.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.985+0000 7f52fee7b700 1 Processor -- start 2022-06-17T12:47:02.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.985+0000 7f52fee7b700 1 -- start start 2022-06-17T12:47:02.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52fee7b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f80aef60 0x7f52f813f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:02.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52fee7b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52f813cb70 0x7f52f8155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:02.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52fee7b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52f813cf50 0x7f52f81562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:02.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52fee7b700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f80b0eb0 con 0x7f52f813cb70 2022-06-17T12:47:02.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52fee7b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f80b0100 con 0x7f52f80aef60 2022-06-17T12:47:02.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52fee7b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f80b6210 con 0x7f52f813cf50 2022-06-17T12:47:02.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52fd678700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52f813cb70 0x7f52f8155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:02.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52fd678700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52f813cb70 0x7f52f8155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59742/0 (socket says 172.21.15.36:59742) 2022-06-17T12:47:02.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52fd678700 1 -- 172.21.15.36:0/3705671213 learned_addr learned my addr 172.21.15.36:0/3705671213 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:02.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52fde79700 1 --2- 172.21.15.36:0/3705671213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f80aef60 0x7f52f813f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:02.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52fe67a700 1 --2- 172.21.15.36:0/3705671213 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52f813cf50 0x7f52f81562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:02.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52fd678700 1 -- 172.21.15.36:0/3705671213 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52f813cf50 msgr2=0x7f52f81562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:02.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52fd678700 1 --2- 172.21.15.36:0/3705671213 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52f813cf50 0x7f52f81562e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52fd678700 1 -- 172.21.15.36:0/3705671213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f80aef60 msgr2=0x7f52f813f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:02.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52fd678700 1 --2- 172.21.15.36:0/3705671213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f80aef60 0x7f52f813f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52fd678700 1 -- 172.21.15.36:0/3705671213 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f0009530 con 0x7f52f813cb70 2022-06-17T12:47:02.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52fe67a700 1 --2- 172.21.15.36:0/3705671213 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52f813cf50 0x7f52f81562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:02.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52fde79700 1 --2- 172.21.15.36:0/3705671213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f80aef60 0x7f52f813f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:02.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52fd678700 1 --2- 172.21.15.36:0/3705671213 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52f813cb70 0x7f52f8155df0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f52f80b7280 tx=0x7f52e8007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:02.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52e6ffd700 1 -- 172.21.15.36:0/3705671213 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52e800ce30 con 0x7f52f813cb70 2022-06-17T12:47:02.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52e6ffd700 1 -- 172.21.15.36:0/3705671213 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e800f470 con 0x7f52f813cb70 2022-06-17T12:47:02.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52fee7b700 1 -- 172.21.15.36:0/3705671213 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f80b6510 con 0x7f52f813cb70 2022-06-17T12:47:02.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52e6ffd700 1 -- 172.21.15.36:0/3705671213 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52e800f7d0 con 0x7f52f813cb70 2022-06-17T12:47:02.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52fee7b700 1 -- 172.21.15.36:0/3705671213 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f813f880 con 0x7f52f813cb70 2022-06-17T12:47:02.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.989+0000 7f52fee7b700 1 -- 172.21.15.36:0/3705671213 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f8057410 con 0x7f52f813cb70 2022-06-17T12:47:02.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.993+0000 7f52e6ffd700 1 -- 172.21.15.36:0/3705671213 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f52e8019030 con 0x7f52f813cb70 2022-06-17T12:47:02.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.993+0000 7f52e6ffd700 1 --2- 172.21.15.36:0/3705671213 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f52d8053ad0 0x7f52d8055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:02.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.993+0000 7f52e6ffd700 1 -- 172.21.15.36:0/3705671213 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f52e806b3b0 con 0x7f52f813cb70 2022-06-17T12:47:02.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.993+0000 7f52fde79700 1 --2- 172.21.15.36:0/3705671213 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f52d8053ad0 0x7f52d8055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:02.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.993+0000 7f52fde79700 1 --2- 172.21.15.36:0/3705671213 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f52d8053ad0 0x7f52d8055f90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f52f400b000 tx=0x7f52f4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:02.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:01.997+0000 7f52e6ffd700 1 -- 172.21.15.36:0/3705671213 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f52e80420a0 con 0x7f52f813cb70 2022-06-17T12:47:02.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.125+0000 7f52fee7b700 1 -- 172.21.15.36:0/3705671213 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f52f80b0320 con 0x7f52f813cb70 2022-06-17T12:47:02.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.125+0000 7f52e6ffd700 1 -- 172.21.15.36:0/3705671213 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f52e80456c0 con 0x7f52f813cb70 2022-06-17T12:47:02.122 INFO:teuthology.orchestra.run.smithi036.stdout:55834574855 2022-06-17T12:47:02.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.129+0000 7f52fee7b700 1 -- 172.21.15.36:0/3705671213 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f52d8053ad0 msgr2=0x7f52d8055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:02.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.129+0000 7f52fee7b700 1 --2- 172.21.15.36:0/3705671213 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f52d8053ad0 0x7f52d8055f90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f52f400b000 tx=0x7f52f4008040).stop 2022-06-17T12:47:02.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.129+0000 7f52fee7b700 1 -- 172.21.15.36:0/3705671213 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52f813cb70 msgr2=0x7f52f8155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:02.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.129+0000 7f52fee7b700 1 --2- 172.21.15.36:0/3705671213 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52f813cb70 0x7f52f8155df0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f52f80b7280 tx=0x7f52e8007c50).stop 2022-06-17T12:47:02.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.129+0000 7f52fee7b700 1 -- 172.21.15.36:0/3705671213 shutdown_connections 2022-06-17T12:47:02.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.129+0000 7f52fee7b700 1 --2- 172.21.15.36:0/3705671213 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f52d8053ad0 0x7f52d8055f90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.129+0000 7f52fee7b700 1 --2- 172.21.15.36:0/3705671213 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52f813cf50 0x7f52f81562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.129+0000 7f52fee7b700 1 --2- 172.21.15.36:0/3705671213 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52f813cb70 0x7f52f8155df0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.129+0000 7f52fee7b700 1 --2- 172.21.15.36:0/3705671213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f80aef60 0x7f52f813f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.129+0000 7f52fee7b700 1 -- 172.21.15.36:0/3705671213 >> 172.21.15.36:0/3705671213 conn(0x7f52f805c550 msgr2=0x7f52f80b88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:02.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.129+0000 7f52fee7b700 1 -- 172.21.15.36:0/3705671213 shutdown_connections 2022-06-17T12:47:02.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.129+0000 7f52fee7b700 1 -- 172.21.15.36:0/3705671213 wait complete. 2022-06-17T12:47:02.134 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.0 2022-06-17T12:47:02.135 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2022-06-17T12:47:02.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.289+0000 7f25a1a38700 1 Processor -- start 2022-06-17T12:47:02.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.289+0000 7f25a1a38700 1 -- start start 2022-06-17T12:47:02.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.289+0000 7f25a1a38700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f259c0ae270 0x7f259c0b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:02.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.289+0000 7f25a1a38700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f259c0ad9c0 0x7f259c0b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:02.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.289+0000 7f25a1a38700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f259c0afc30 con 0x7f259c13fde0 2022-06-17T12:47:02.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.289+0000 7f25a1a38700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f259c0b3eb0 con 0x7f259c0ae270 2022-06-17T12:47:02.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.289+0000 7f25a1a38700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f259c0ae650 con 0x7f259c0ad9c0 2022-06-17T12:47:02.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.293+0000 7f259affd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f259c0ae270 0x7f259c0b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:02.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.293+0000 7f259affd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f259c0ae270 0x7f259c0b39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59748/0 (socket says 172.21.15.36:59748) 2022-06-17T12:47:02.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.293+0000 7f259affd700 1 -- 172.21.15.36:0/2114386864 learned_addr learned my addr 172.21.15.36:0/2114386864 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:02.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.293+0000 7f259a7fc700 1 --2- 172.21.15.36:0/2114386864 >> v2:172.21.15.70:3300/0 conn(0x7f259c0ad9c0 0x7f259c0b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:02.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.293+0000 7f259affd700 1 -- 172.21.15.36:0/2114386864 >> v2:172.21.15.70:3300/0 conn(0x7f259c0ad9c0 msgr2=0x7f259c0b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:02.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.293+0000 7f259affd700 1 --2- 172.21.15.36:0/2114386864 >> v2:172.21.15.70:3300/0 conn(0x7f259c0ad9c0 0x7f259c0b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.293+0000 7f259affd700 1 -- 172.21.15.36:0/2114386864 >> v1:172.21.15.70:6789/0 conn(0x7f259c13fde0 legacy=0x7f259c1433b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:02.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.293+0000 7f2599ffb700 1 -- 172.21.15.36:0/2114386864 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4164008509 0 0) 0x7f259c0afc30 con 0x7f259c13fde0 2022-06-17T12:47:02.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.293+0000 7f259affd700 1 -- 172.21.15.36:0/2114386864 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259c0afe60 con 0x7f259c0ae270 2022-06-17T12:47:02.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.293+0000 7f259a7fc700 1 --2- 172.21.15.36:0/2114386864 >> v2:172.21.15.70:3300/0 conn(0x7f259c0ad9c0 0x7f259c0b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:02.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.293+0000 7f259affd700 1 --2- 172.21.15.36:0/2114386864 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f259c0ae270 0x7f259c0b39c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f258c002f70 tx=0x7f258c006eb0).ready entity=mon.0 client_cookie=42a4cb159790416 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:02.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.293+0000 7f2599ffb700 1 -- 172.21.15.36:0/2114386864 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f258c00d5c0 con 0x7f259c0ae270 2022-06-17T12:47:02.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.293+0000 7f2599ffb700 1 -- 172.21.15.36:0/2114386864 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c000e90 con 0x7f259c0ae270 2022-06-17T12:47:02.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.293+0000 7f2599ffb700 1 -- 172.21.15.36:0/2114386864 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f258c00d910 con 0x7f259c0ae270 2022-06-17T12:47:02.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.293+0000 7f25a1a38700 1 -- 172.21.15.36:0/2114386864 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f259c0ae270 msgr2=0x7f259c0b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:02.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.293+0000 7f25a1a38700 1 --2- 172.21.15.36:0/2114386864 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f259c0ae270 0x7f259c0b39c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f258c002f70 tx=0x7f258c006eb0).stop 2022-06-17T12:47:02.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.293+0000 7f25a1a38700 1 -- 172.21.15.36:0/2114386864 shutdown_connections 2022-06-17T12:47:02.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.293+0000 7f25a1a38700 1 --2- 172.21.15.36:0/2114386864 >> v2:172.21.15.70:3300/0 conn(0x7f259c0ad9c0 0x7f259c0b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.293+0000 7f25a1a38700 1 --2- 172.21.15.36:0/2114386864 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f259c0ae270 0x7f259c0b39c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.293+0000 7f25a1a38700 1 -- 172.21.15.36:0/2114386864 >> 172.21.15.36:0/2114386864 conn(0x7f259c05c550 msgr2=0x7f259c05c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:02.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.293+0000 7f25a1a38700 1 -- 172.21.15.36:0/2114386864 shutdown_connections 2022-06-17T12:47:02.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.293+0000 7f25a1a38700 1 -- 172.21.15.36:0/2114386864 wait complete. 2022-06-17T12:47:02.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.297+0000 7f25a1a38700 1 Processor -- start 2022-06-17T12:47:02.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.297+0000 7f25a1a38700 1 -- start start 2022-06-17T12:47:02.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.297+0000 7f25a1a38700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f259c0ad9c0 0x7f259c13e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:02.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.297+0000 7f25a1a38700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f259c0ae270 0x7f259c138dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:02.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.297+0000 7f25a1a38700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f259c13fde0 0x7f259c1392c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:02.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.297+0000 7f25a1a38700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c13bab0 con 0x7f259c0ad9c0 2022-06-17T12:47:02.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.297+0000 7f25a1a38700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c142f50 con 0x7f259c13fde0 2022-06-17T12:47:02.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.297+0000 7f25a1a38700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259c143180 con 0x7f259c0ae270 2022-06-17T12:47:02.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.297+0000 7f259affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f259c0ad9c0 0x7f259c13e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:02.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.297+0000 7f259affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f259c0ad9c0 0x7f259c13e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59754/0 (socket says 172.21.15.36:59754) 2022-06-17T12:47:02.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.297+0000 7f259affd700 1 -- 172.21.15.36:0/608996450 learned_addr learned my addr 172.21.15.36:0/608996450 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:02.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.297+0000 7f259a7fc700 1 --2- 172.21.15.36:0/608996450 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f259c0ae270 0x7f259c138dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:02.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.297+0000 7f259b7fe700 1 --2- 172.21.15.36:0/608996450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f259c13fde0 0x7f259c1392c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:02.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.297+0000 7f259affd700 1 -- 172.21.15.36:0/608996450 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f259c0ae270 msgr2=0x7f259c138dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:02.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.297+0000 7f259affd700 1 --2- 172.21.15.36:0/608996450 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f259c0ae270 0x7f259c138dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.297+0000 7f259affd700 1 -- 172.21.15.36:0/608996450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f259c13fde0 msgr2=0x7f259c1392c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:02.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.297+0000 7f259affd700 1 --2- 172.21.15.36:0/608996450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f259c13fde0 0x7f259c1392c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.297+0000 7f259affd700 1 -- 172.21.15.36:0/608996450 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258400b180 con 0x7f259c0ad9c0 2022-06-17T12:47:02.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.297+0000 7f259affd700 1 --2- 172.21.15.36:0/608996450 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f259c0ad9c0 0x7f259c13e220 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f259c13dcc0 tx=0x7f258c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:02.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.301+0000 7f2593fff700 1 -- 172.21.15.36:0/608996450 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f258c0116e0 con 0x7f259c0ad9c0 2022-06-17T12:47:02.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.301+0000 7f2593fff700 1 -- 172.21.15.36:0/608996450 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c011840 con 0x7f259c0ad9c0 2022-06-17T12:47:02.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.301+0000 7f25a1a38700 1 -- 172.21.15.36:0/608996450 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f258c00b1e0 con 0x7f259c0ad9c0 2022-06-17T12:47:02.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.301+0000 7f2593fff700 1 -- 172.21.15.36:0/608996450 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f258c011ba0 con 0x7f259c0ad9c0 2022-06-17T12:47:02.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.301+0000 7f25a1a38700 1 -- 172.21.15.36:0/608996450 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259c13c460 con 0x7f259c0ad9c0 2022-06-17T12:47:02.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.301+0000 7f25a1a38700 1 -- 172.21.15.36:0/608996450 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f259c0b0100 con 0x7f259c0ad9c0 2022-06-17T12:47:02.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.301+0000 7f2593fff700 1 -- 172.21.15.36:0/608996450 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f258c025030 con 0x7f259c0ad9c0 2022-06-17T12:47:02.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.301+0000 7f2593fff700 1 --2- 172.21.15.36:0/608996450 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2580053a40 0x7f2580055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:02.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.301+0000 7f2593fff700 1 -- 172.21.15.36:0/608996450 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f258c06d160 con 0x7f259c0ad9c0 2022-06-17T12:47:02.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.301+0000 7f259a7fc700 1 --2- 172.21.15.36:0/608996450 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2580053a40 0x7f2580055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:02.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.301+0000 7f259a7fc700 1 --2- 172.21.15.36:0/608996450 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2580053a40 0x7f2580055f00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f258400b000 tx=0x7f2584006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:02.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.309+0000 7f2593fff700 1 -- 172.21.15.36:0/608996450 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f258c043e50 con 0x7f259c0ad9c0 2022-06-17T12:47:02.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.433+0000 7f25a1a38700 1 -- 172.21.15.36:0/608996450 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f259c13db30 con 0x7f259c0ad9c0 2022-06-17T12:47:02.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.433+0000 7f2593fff700 1 -- 172.21.15.36:0/608996450 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f258c023090 con 0x7f259c0ad9c0 2022-06-17T12:47:02.436 INFO:teuthology.orchestra.run.smithi036.stdout:55834574855 2022-06-17T12:47:02.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.437+0000 7f25a1a38700 1 -- 172.21.15.36:0/608996450 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2580053a40 msgr2=0x7f2580055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:02.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.437+0000 7f25a1a38700 1 --2- 172.21.15.36:0/608996450 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2580053a40 0x7f2580055f00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f258400b000 tx=0x7f2584006df0).stop 2022-06-17T12:47:02.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.437+0000 7f25a1a38700 1 -- 172.21.15.36:0/608996450 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f259c0ad9c0 msgr2=0x7f259c13e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:02.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.437+0000 7f25a1a38700 1 --2- 172.21.15.36:0/608996450 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f259c0ad9c0 0x7f259c13e220 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f259c13dcc0 tx=0x7f258c015000).stop 2022-06-17T12:47:02.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.437+0000 7f25a1a38700 1 -- 172.21.15.36:0/608996450 shutdown_connections 2022-06-17T12:47:02.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.437+0000 7f25a1a38700 1 --2- 172.21.15.36:0/608996450 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2580053a40 0x7f2580055f00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.437+0000 7f25a1a38700 1 --2- 172.21.15.36:0/608996450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f259c13fde0 0x7f259c1392c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.437+0000 7f25a1a38700 1 --2- 172.21.15.36:0/608996450 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f259c0ae270 0x7f259c138dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.437+0000 7f25a1a38700 1 --2- 172.21.15.36:0/608996450 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f259c0ad9c0 0x7f259c13e220 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.437+0000 7f25a1a38700 1 -- 172.21.15.36:0/608996450 >> 172.21.15.36:0/608996450 conn(0x7f259c05c550 msgr2=0x7f259c0b4790 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:02.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.437+0000 7f25a1a38700 1 -- 172.21.15.36:0/608996450 shutdown_connections 2022-06-17T12:47:02.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.437+0000 7f25a1a38700 1 -- 172.21.15.36:0/608996450 wait complete. 2022-06-17T12:47:02.447 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.1 2022-06-17T12:47:02.447 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2022-06-17T12:47:02.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff61baab700 1 Processor -- start 2022-06-17T12:47:02.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff61baab700 1 -- start start 2022-06-17T12:47:02.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff61baab700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff614062270 0x7ff614142f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:02.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff61baab700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff614143e80 0x7ff614146970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:02.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff61baab700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6140b44b0 con 0x7ff6140b0cb0 2022-06-17T12:47:02.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff61baab700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6140b6650 con 0x7ff614062270 2022-06-17T12:47:02.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff61baab700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff614062650 con 0x7ff614143e80 2022-06-17T12:47:02.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff619847700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff614062270 0x7ff614142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:02.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff619046700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7ff6140b0cb0 0x7ff614143770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49060/0 (socket says 172.21.15.36:49060) 2022-06-17T12:47:02.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff619046700 1 -- 172.21.15.36:0/2427168700 learned_addr learned my addr 172.21.15.36:0/2427168700 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:02.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff61a048700 1 --2- 172.21.15.36:0/2427168700 >> v2:172.21.15.70:3300/0 conn(0x7ff614143e80 0x7ff614146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:02.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff619847700 1 -- 172.21.15.36:0/2427168700 >> v2:172.21.15.70:3300/0 conn(0x7ff614143e80 msgr2=0x7ff614146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:02.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff619847700 1 --2- 172.21.15.36:0/2427168700 >> v2:172.21.15.70:3300/0 conn(0x7ff614143e80 0x7ff614146970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff619847700 1 -- 172.21.15.36:0/2427168700 >> v1:172.21.15.36:6789/0 conn(0x7ff6140b0cb0 legacy=0x7ff614143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:02.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff619847700 1 -- 172.21.15.36:0/2427168700 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6140b4210 con 0x7ff614062270 2022-06-17T12:47:02.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff619847700 1 --2- 172.21.15.36:0/2427168700 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff614062270 0x7ff614142f20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7ff610002f70 tx=0x7ff610006df0).ready entity=mon.0 client_cookie=e3090e714305baae server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:02.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff618845700 1 -- 172.21.15.36:0/2427168700 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff61000d5c0 con 0x7ff614062270 2022-06-17T12:47:02.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff618845700 1 -- 172.21.15.36:0/2427168700 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff610000e90 con 0x7ff614062270 2022-06-17T12:47:02.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff618845700 1 -- 172.21.15.36:0/2427168700 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff61000d910 con 0x7ff614062270 2022-06-17T12:47:02.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff61baab700 1 -- 172.21.15.36:0/2427168700 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff614062270 msgr2=0x7ff614142f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:02.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff61baab700 1 --2- 172.21.15.36:0/2427168700 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff614062270 0x7ff614142f20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7ff610002f70 tx=0x7ff610006df0).stop 2022-06-17T12:47:02.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff61baab700 1 -- 172.21.15.36:0/2427168700 shutdown_connections 2022-06-17T12:47:02.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff61baab700 1 --2- 172.21.15.36:0/2427168700 >> v2:172.21.15.70:3300/0 conn(0x7ff614143e80 0x7ff614146970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff61baab700 1 --2- 172.21.15.36:0/2427168700 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff614062270 0x7ff614142f20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff61baab700 1 -- 172.21.15.36:0/2427168700 >> 172.21.15.36:0/2427168700 conn(0x7ff6140b9870 msgr2=0x7ff6140b9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:02.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff61baab700 1 -- 172.21.15.36:0/2427168700 shutdown_connections 2022-06-17T12:47:02.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.577+0000 7ff61baab700 1 -- 172.21.15.36:0/2427168700 wait complete. 2022-06-17T12:47:02.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.581+0000 7ff61baab700 1 Processor -- start 2022-06-17T12:47:02.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.581+0000 7ff61baab700 1 -- start start 2022-06-17T12:47:02.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.581+0000 7ff61baab700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff614062270 0x7ff614142550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:02.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.581+0000 7ff61baab700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff614143e80 0x7ff614148ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:02.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.581+0000 7ff61baab700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff6140ad330 0x7ff6141491d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:02.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.581+0000 7ff61baab700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6140ae630 con 0x7ff6140ad330 2022-06-17T12:47:02.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.581+0000 7ff61baab700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6140af360 con 0x7ff614143e80 2022-06-17T12:47:02.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.581+0000 7ff61baab700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6140ae2b0 con 0x7ff614062270 2022-06-17T12:47:02.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.581+0000 7ff61a048700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff6140ad330 0x7ff6141491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:02.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.581+0000 7ff61a048700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff6140ad330 0x7ff6141491d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59772/0 (socket says 172.21.15.36:59772) 2022-06-17T12:47:02.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.581+0000 7ff61a048700 1 -- 172.21.15.36:0/587425694 learned_addr learned my addr 172.21.15.36:0/587425694 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:02.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.581+0000 7ff619046700 1 --2- 172.21.15.36:0/587425694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff614143e80 0x7ff614148ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:02.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.581+0000 7ff619847700 1 --2- 172.21.15.36:0/587425694 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff614062270 0x7ff614142550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:02.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.581+0000 7ff61a048700 1 -- 172.21.15.36:0/587425694 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff614062270 msgr2=0x7ff614142550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:02.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.581+0000 7ff61a048700 1 --2- 172.21.15.36:0/587425694 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff614062270 0x7ff614142550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.581+0000 7ff61a048700 1 -- 172.21.15.36:0/587425694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff614143e80 msgr2=0x7ff614148ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:02.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.581+0000 7ff61a048700 1 --2- 172.21.15.36:0/587425694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff614143e80 0x7ff614148ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.581+0000 7ff61a048700 1 -- 172.21.15.36:0/587425694 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61000b180 con 0x7ff6140ad330 2022-06-17T12:47:02.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.581+0000 7ff61a048700 1 --2- 172.21.15.36:0/587425694 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff6140ad330 0x7ff6141491d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7ff60c00a000 tx=0x7ff60c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:02.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.581+0000 7ff606ffd700 1 -- 172.21.15.36:0/587425694 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff60c0127e0 con 0x7ff6140ad330 2022-06-17T12:47:02.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.581+0000 7ff606ffd700 1 -- 172.21.15.36:0/587425694 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60c012940 con 0x7ff6140ad330 2022-06-17T12:47:02.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.581+0000 7ff61baab700 1 -- 172.21.15.36:0/587425694 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6141496c0 con 0x7ff6140ad330 2022-06-17T12:47:02.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.581+0000 7ff606ffd700 1 -- 172.21.15.36:0/587425694 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff60c012ca0 con 0x7ff6140ad330 2022-06-17T12:47:02.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.581+0000 7ff61baab700 1 -- 172.21.15.36:0/587425694 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6141463e0 con 0x7ff6140ad330 2022-06-17T12:47:02.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.585+0000 7ff61baab700 1 -- 172.21.15.36:0/587425694 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6140af560 con 0x7ff6140ad330 2022-06-17T12:47:02.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.585+0000 7ff606ffd700 1 -- 172.21.15.36:0/587425694 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff60c013480 con 0x7ff6140ad330 2022-06-17T12:47:02.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.585+0000 7ff606ffd700 1 --2- 172.21.15.36:0/587425694 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff5fc053a60 0x7ff5fc055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:02.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.585+0000 7ff606ffd700 1 -- 172.21.15.36:0/587425694 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff60c06c300 con 0x7ff6140ad330 2022-06-17T12:47:02.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.585+0000 7ff619847700 1 --2- 172.21.15.36:0/587425694 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff5fc053a60 0x7ff5fc055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:02.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.589+0000 7ff619847700 1 --2- 172.21.15.36:0/587425694 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff5fc053a60 0x7ff5fc055f20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7ff61001a000 tx=0x7ff610014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:02.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.593+0000 7ff606ffd700 1 -- 172.21.15.36:0/587425694 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff60c0430a0 con 0x7ff6140ad330 2022-06-17T12:47:02.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.717+0000 7ff61baab700 1 -- 172.21.15.36:0/587425694 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff614146d10 con 0x7ff6140ad330 2022-06-17T12:47:02.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.717+0000 7ff606ffd700 1 -- 172.21.15.36:0/587425694 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff60c012e00 con 0x7ff6140ad330 2022-06-17T12:47:02.778 INFO:teuthology.orchestra.run.smithi036.stdout:60129542150 2022-06-17T12:47:02.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.721+0000 7ff61baab700 1 -- 172.21.15.36:0/587425694 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff5fc053a60 msgr2=0x7ff5fc055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:02.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.721+0000 7ff61baab700 1 --2- 172.21.15.36:0/587425694 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff5fc053a60 0x7ff5fc055f20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7ff61001a000 tx=0x7ff610014000).stop 2022-06-17T12:47:02.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.721+0000 7ff61baab700 1 -- 172.21.15.36:0/587425694 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff6140ad330 msgr2=0x7ff6141491d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:02.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.721+0000 7ff61baab700 1 --2- 172.21.15.36:0/587425694 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff6140ad330 0x7ff6141491d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7ff60c00a000 tx=0x7ff60c008040).stop 2022-06-17T12:47:02.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.721+0000 7ff61baab700 1 -- 172.21.15.36:0/587425694 shutdown_connections 2022-06-17T12:47:02.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.721+0000 7ff61baab700 1 --2- 172.21.15.36:0/587425694 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff5fc053a60 0x7ff5fc055f20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.721+0000 7ff61baab700 1 --2- 172.21.15.36:0/587425694 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff6140ad330 0x7ff6141491d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.721+0000 7ff61baab700 1 --2- 172.21.15.36:0/587425694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff614143e80 0x7ff614148ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.721+0000 7ff61baab700 1 --2- 172.21.15.36:0/587425694 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff614062270 0x7ff614142550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:02.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.721+0000 7ff61baab700 1 -- 172.21.15.36:0/587425694 >> 172.21.15.36:0/587425694 conn(0x7ff6140b9870 msgr2=0x7ff61405f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:02.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.721+0000 7ff61baab700 1 -- 172.21.15.36:0/587425694 shutdown_connections 2022-06-17T12:47:02.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.721+0000 7ff61baab700 1 -- 172.21.15.36:0/587425694 wait complete. 2022-06-17T12:47:02.788 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.2 2022-06-17T12:47:02.789 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2022-06-17T12:47:03.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.913+0000 7fe3ba5f7700 1 Processor -- start 2022-06-17T12:47:03.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.913+0000 7fe3ba5f7700 1 -- start start 2022-06-17T12:47:03.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.913+0000 7fe3ba5f7700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe3b4062770 0x7fe3b4062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:03.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.913+0000 7fe3ba5f7700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe3b40ad330 0x7fe3b40ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:03.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.913+0000 7fe3ba5f7700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b40b0ba0 con 0x7fe3b41444f0 2022-06-17T12:47:03.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.913+0000 7fe3ba5f7700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b40ae820 con 0x7fe3b40ad330 2022-06-17T12:47:03.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.913+0000 7fe3ba5f7700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b4061030 con 0x7fe3b4062770 2022-06-17T12:47:03.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.917+0000 7fe3b8b94700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe3b40ad330 0x7fe3b40ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:03.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.917+0000 7fe3b8b94700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe3b40ad330 0x7fe3b40ad7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59780/0 (socket says 172.21.15.36:59780) 2022-06-17T12:47:03.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.917+0000 7fe3b8b94700 1 -- 172.21.15.36:0/3949153946 learned_addr learned my addr 172.21.15.36:0/3949153946 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:03.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.917+0000 7fe3b3fff700 1 --2- 172.21.15.36:0/3949153946 >> v2:172.21.15.70:3300/0 conn(0x7fe3b4062770 0x7fe3b4062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:03.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.917+0000 7fe3b8b94700 1 -- 172.21.15.36:0/3949153946 >> v2:172.21.15.70:3300/0 conn(0x7fe3b4062770 msgr2=0x7fe3b4062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:03.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.917+0000 7fe3b8b94700 1 --2- 172.21.15.36:0/3949153946 >> v2:172.21.15.70:3300/0 conn(0x7fe3b4062770 0x7fe3b4062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.917+0000 7fe3b8b94700 1 -- 172.21.15.36:0/3949153946 >> v1:172.21.15.70:6789/0 conn(0x7fe3b41444f0 legacy=0x7fe3b4144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:03.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.917+0000 7fe3b8b94700 1 -- 172.21.15.36:0/3949153946 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b4147e90 con 0x7fe3b40ad330 2022-06-17T12:47:03.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.917+0000 7fe3b3fff700 1 --2- 172.21.15.36:0/3949153946 >> v2:172.21.15.70:3300/0 conn(0x7fe3b4062770 0x7fe3b4062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:03.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.917+0000 7fe3b8b94700 1 --2- 172.21.15.36:0/3949153946 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe3b40ad330 0x7fe3b40ad7c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fe3b4061170 tx=0x7fe3ac009290).ready entity=mon.0 client_cookie=b298d90cd7432925 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:03.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.917+0000 7fe3b2ffd700 1 -- 172.21.15.36:0/3949153946 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3ac00b830 con 0x7fe3b40ad330 2022-06-17T12:47:03.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.917+0000 7fe3b2ffd700 1 -- 172.21.15.36:0/3949153946 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ac00b990 con 0x7fe3b40ad330 2022-06-17T12:47:03.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.917+0000 7fe3b2ffd700 1 -- 172.21.15.36:0/3949153946 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3ac00bb00 con 0x7fe3b40ad330 2022-06-17T12:47:03.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.917+0000 7fe3ba5f7700 1 -- 172.21.15.36:0/3949153946 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe3b40ad330 msgr2=0x7fe3b40ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:03.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.917+0000 7fe3ba5f7700 1 --2- 172.21.15.36:0/3949153946 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe3b40ad330 0x7fe3b40ad7c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fe3b4061170 tx=0x7fe3ac009290).stop 2022-06-17T12:47:03.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.917+0000 7fe3ba5f7700 1 -- 172.21.15.36:0/3949153946 shutdown_connections 2022-06-17T12:47:03.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.917+0000 7fe3ba5f7700 1 --2- 172.21.15.36:0/3949153946 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe3b40ad330 0x7fe3b40ad7c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.917+0000 7fe3ba5f7700 1 --2- 172.21.15.36:0/3949153946 >> v2:172.21.15.70:3300/0 conn(0x7fe3b4062770 0x7fe3b4062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.917+0000 7fe3ba5f7700 1 -- 172.21.15.36:0/3949153946 >> 172.21.15.36:0/3949153946 conn(0x7fe3b40b53e0 msgr2=0x7fe3b40b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:03.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.917+0000 7fe3ba5f7700 1 -- 172.21.15.36:0/3949153946 shutdown_connections 2022-06-17T12:47:03.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.917+0000 7fe3ba5f7700 1 -- 172.21.15.36:0/3949153946 wait complete. 2022-06-17T12:47:03.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3ba5f7700 1 Processor -- start 2022-06-17T12:47:03.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3ba5f7700 1 -- start start 2022-06-17T12:47:03.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3ba5f7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe3b4062770 0x7fe3b4063090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:03.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3ba5f7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe3b40ad330 0x7fe3b4066980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:03.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3ba5f7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3b41444f0 0x7fe3b4066e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:03.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3ba5f7700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b40636c0 con 0x7fe3b40ad330 2022-06-17T12:47:03.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3ba5f7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b41571c0 con 0x7fe3b41444f0 2022-06-17T12:47:03.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3ba5f7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b4157460 con 0x7fe3b4062770 2022-06-17T12:47:03.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3b37fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe3b40ad330 0x7fe3b4066980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:03.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3b37fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe3b40ad330 0x7fe3b4066980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59784/0 (socket says 172.21.15.36:59784) 2022-06-17T12:47:03.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3b37fe700 1 -- 172.21.15.36:0/1546777225 learned_addr learned my addr 172.21.15.36:0/1546777225 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:03.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3b8b94700 1 --2- 172.21.15.36:0/1546777225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3b41444f0 0x7fe3b4066e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:03.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3b3fff700 1 --2- 172.21.15.36:0/1546777225 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe3b4062770 0x7fe3b4063090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:03.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3b37fe700 1 -- 172.21.15.36:0/1546777225 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe3b4062770 msgr2=0x7fe3b4063090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:03.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3b37fe700 1 --2- 172.21.15.36:0/1546777225 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe3b4062770 0x7fe3b4063090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3b37fe700 1 -- 172.21.15.36:0/1546777225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3b41444f0 msgr2=0x7fe3b4066e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:03.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3b37fe700 1 --2- 172.21.15.36:0/1546777225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3b41444f0 0x7fe3b4066e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3b37fe700 1 -- 172.21.15.36:0/1546777225 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ac009530 con 0x7fe3b40ad330 2022-06-17T12:47:03.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3b3fff700 1 --2- 172.21.15.36:0/1546777225 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe3b4062770 0x7fe3b4063090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:03.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3b37fe700 1 --2- 172.21.15.36:0/1546777225 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe3b40ad330 0x7fe3b4066980 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fe3b405d270 tx=0x7fe39c007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:03.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3b17fa700 1 -- 172.21.15.36:0/1546777225 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe39c012400 con 0x7fe3b40ad330 2022-06-17T12:47:03.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3b17fa700 1 -- 172.21.15.36:0/1546777225 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39c012560 con 0x7fe3b40ad330 2022-06-17T12:47:03.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3ba5f7700 1 -- 172.21.15.36:0/1546777225 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b4143710 con 0x7fe3b40ad330 2022-06-17T12:47:03.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3b17fa700 1 -- 172.21.15.36:0/1546777225 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe39c0128c0 con 0x7fe3b40ad330 2022-06-17T12:47:03.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.921+0000 7fe3ba5f7700 1 -- 172.21.15.36:0/1546777225 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b4143be0 con 0x7fe3b40ad330 2022-06-17T12:47:03.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.925+0000 7fe3ba5f7700 1 -- 172.21.15.36:0/1546777225 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b40b1c20 con 0x7fe3b40ad330 2022-06-17T12:47:03.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.925+0000 7fe3b17fa700 1 -- 172.21.15.36:0/1546777225 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe39c019030 con 0x7fe3b40ad330 2022-06-17T12:47:03.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.925+0000 7fe3b17fa700 1 --2- 172.21.15.36:0/1546777225 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe398053a60 0x7fe398055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:03.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.925+0000 7fe3b17fa700 1 -- 172.21.15.36:0/1546777225 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe39c06b400 con 0x7fe3b40ad330 2022-06-17T12:47:03.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.925+0000 7fe3b3fff700 1 --2- 172.21.15.36:0/1546777225 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe398053a60 0x7fe398055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:03.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.925+0000 7fe3b3fff700 1 --2- 172.21.15.36:0/1546777225 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe398053a60 0x7fe398055f20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fe3a400c000 tx=0x7fe3a4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:03.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:02.929+0000 7fe3b17fa700 1 -- 172.21.15.36:0/1546777225 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe39c0420f0 con 0x7fe3b40ad330 2022-06-17T12:47:03.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.057+0000 7fe3ba5f7700 1 -- 172.21.15.36:0/1546777225 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe3b4157630 con 0x7fe3b40ad330 2022-06-17T12:47:03.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.057+0000 7fe3b17fa700 1 -- 172.21.15.36:0/1546777225 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe39c045710 con 0x7fe3b40ad330 2022-06-17T12:47:03.169 INFO:teuthology.orchestra.run.smithi036.stdout:60129542151 2022-06-17T12:47:03.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.061+0000 7fe3ba5f7700 1 -- 172.21.15.36:0/1546777225 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe398053a60 msgr2=0x7fe398055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:03.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.061+0000 7fe3ba5f7700 1 --2- 172.21.15.36:0/1546777225 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe398053a60 0x7fe398055f20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fe3a400c000 tx=0x7fe3a4008040).stop 2022-06-17T12:47:03.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.061+0000 7fe3ba5f7700 1 -- 172.21.15.36:0/1546777225 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe3b40ad330 msgr2=0x7fe3b4066980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:03.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.061+0000 7fe3ba5f7700 1 --2- 172.21.15.36:0/1546777225 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe3b40ad330 0x7fe3b4066980 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fe3b405d270 tx=0x7fe39c007c50).stop 2022-06-17T12:47:03.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.061+0000 7fe3ba5f7700 1 -- 172.21.15.36:0/1546777225 shutdown_connections 2022-06-17T12:47:03.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.061+0000 7fe3ba5f7700 1 --2- 172.21.15.36:0/1546777225 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe398053a60 0x7fe398055f20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.061+0000 7fe3ba5f7700 1 --2- 172.21.15.36:0/1546777225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3b41444f0 0x7fe3b4066e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.061+0000 7fe3ba5f7700 1 --2- 172.21.15.36:0/1546777225 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe3b40ad330 0x7fe3b4066980 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.061+0000 7fe3ba5f7700 1 --2- 172.21.15.36:0/1546777225 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe3b4062770 0x7fe3b4063090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.061+0000 7fe3ba5f7700 1 -- 172.21.15.36:0/1546777225 >> 172.21.15.36:0/1546777225 conn(0x7fe3b40b53e0 msgr2=0x7fe3b4144960 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:03.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.061+0000 7fe3ba5f7700 1 -- 172.21.15.36:0/1546777225 shutdown_connections 2022-06-17T12:47:03.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.061+0000 7fe3ba5f7700 1 -- 172.21.15.36:0/1546777225 wait complete. 2022-06-17T12:47:03.277 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.3 2022-06-17T12:47:03.278 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2022-06-17T12:47:03.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.401+0000 7f1a164f4700 1 Processor -- start 2022-06-17T12:47:03.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.401+0000 7f1a164f4700 1 -- start start 2022-06-17T12:47:03.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a164f4700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a10062770 0x7f1a10062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:03.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a164f4700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f1a100ad330 0x7f1a100ad780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:03.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a164f4700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a100b0ba0 con 0x7f1a101444f0 2022-06-17T12:47:03.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a164f4700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a100ae820 con 0x7f1a100ad330 2022-06-17T12:47:03.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a164f4700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a10061030 con 0x7f1a10062770 2022-06-17T12:47:03.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a14a91700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f1a100ad330 0x7f1a100ad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:03.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a14a91700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f1a100ad330 0x7f1a100ad780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59794/0 (socket says 172.21.15.36:59794) 2022-06-17T12:47:03.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a0f7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f1a101444f0 0x7f1a10144930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49088/0 (socket says 172.21.15.36:49088) 2022-06-17T12:47:03.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a14a91700 1 -- 172.21.15.36:0/421729616 learned_addr learned my addr 172.21.15.36:0/421729616 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:03.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a0ffff700 1 --2- 172.21.15.36:0/421729616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a10062770 0x7f1a10062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:03.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a14a91700 1 -- 172.21.15.36:0/421729616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a10062770 msgr2=0x7f1a10062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:03.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a14a91700 1 --2- 172.21.15.36:0/421729616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a10062770 0x7f1a10062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a14a91700 1 -- 172.21.15.36:0/421729616 >> v1:172.21.15.36:6789/0 conn(0x7f1a101444f0 legacy=0x7f1a10144930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:03.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a14a91700 1 -- 172.21.15.36:0/421729616 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a10147e90 con 0x7f1a100ad330 2022-06-17T12:47:03.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a0effd700 1 -- 172.21.15.36:0/421729616 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1437014786 0 0) 0x7f1a100b0ba0 con 0x7f1a101444f0 2022-06-17T12:47:03.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a14a91700 1 --2- 172.21.15.36:0/421729616 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a100ad330 0x7f1a100ad780 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f1a080032f0 tx=0x7f1a08009290).ready entity=mon.0 client_cookie=99de8df6dcfcd7c6 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:03.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a0effd700 1 -- 172.21.15.36:0/421729616 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a0800b8a0 con 0x7f1a100ad330 2022-06-17T12:47:03.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a0effd700 1 -- 172.21.15.36:0/421729616 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0800ba00 con 0x7f1a100ad330 2022-06-17T12:47:03.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a0effd700 1 -- 172.21.15.36:0/421729616 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a0800bb70 con 0x7f1a100ad330 2022-06-17T12:47:03.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a164f4700 1 -- 172.21.15.36:0/421729616 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a100ad330 msgr2=0x7f1a100ad780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:03.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a164f4700 1 --2- 172.21.15.36:0/421729616 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a100ad330 0x7f1a100ad780 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f1a080032f0 tx=0x7f1a08009290).stop 2022-06-17T12:47:03.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a164f4700 1 -- 172.21.15.36:0/421729616 shutdown_connections 2022-06-17T12:47:03.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a164f4700 1 --2- 172.21.15.36:0/421729616 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a100ad330 0x7f1a100ad780 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a164f4700 1 --2- 172.21.15.36:0/421729616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a10062770 0x7f1a10062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a164f4700 1 -- 172.21.15.36:0/421729616 >> 172.21.15.36:0/421729616 conn(0x7f1a100b53e0 msgr2=0x7f1a100b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:03.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a164f4700 1 -- 172.21.15.36:0/421729616 shutdown_connections 2022-06-17T12:47:03.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a164f4700 1 -- 172.21.15.36:0/421729616 wait complete. 2022-06-17T12:47:03.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a164f4700 1 Processor -- start 2022-06-17T12:47:03.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.405+0000 7f1a164f4700 1 -- start start 2022-06-17T12:47:03.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.409+0000 7f1a164f4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a10062770 0x7f1a100671d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:03.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.409+0000 7f1a164f4700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a100ad330 0x7f1a10130750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:03.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.409+0000 7f1a164f4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a101444f0 0x7f1a10065850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:03.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.409+0000 7f1a164f4700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a10066a90 con 0x7f1a10062770 2022-06-17T12:47:03.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.409+0000 7f1a164f4700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a10066d30 con 0x7f1a101444f0 2022-06-17T12:47:03.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.409+0000 7f1a164f4700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a100667e0 con 0x7f1a100ad330 2022-06-17T12:47:03.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.409+0000 7f1a0ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a10062770 0x7f1a100671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:03.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.409+0000 7f1a0ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a10062770 0x7f1a100671d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59796/0 (socket says 172.21.15.36:59796) 2022-06-17T12:47:03.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.409+0000 7f1a0ffff700 1 -- 172.21.15.36:0/1972351466 learned_addr learned my addr 172.21.15.36:0/1972351466 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:03.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.409+0000 7f1a14a91700 1 --2- 172.21.15.36:0/1972351466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a101444f0 0x7f1a10065850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:03.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.409+0000 7f1a0f7fe700 1 --2- 172.21.15.36:0/1972351466 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a100ad330 0x7f1a10130750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:03.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.409+0000 7f1a14a91700 1 -- 172.21.15.36:0/1972351466 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a100ad330 msgr2=0x7f1a10130750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:03.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.409+0000 7f1a14a91700 1 --2- 172.21.15.36:0/1972351466 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a100ad330 0x7f1a10130750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.409+0000 7f1a14a91700 1 -- 172.21.15.36:0/1972351466 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a10062770 msgr2=0x7f1a100671d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:03.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.409+0000 7f1a14a91700 1 --2- 172.21.15.36:0/1972351466 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a10062770 0x7f1a100671d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.409+0000 7f1a14a91700 1 -- 172.21.15.36:0/1972351466 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a08009530 con 0x7f1a101444f0 2022-06-17T12:47:03.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.409+0000 7f1a0ffff700 1 --2- 172.21.15.36:0/1972351466 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a10062770 0x7f1a100671d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:03.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.409+0000 7f1a14a91700 1 --2- 172.21.15.36:0/1972351466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a101444f0 0x7f1a10065850 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f1a080032f0 tx=0x7f1a08016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:03.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.409+0000 7f1a0d7fa700 1 -- 172.21.15.36:0/1972351466 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a0800b8a0 con 0x7f1a101444f0 2022-06-17T12:47:03.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.409+0000 7f1a164f4700 1 -- 172.21.15.36:0/1972351466 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a1014d610 con 0x7f1a101444f0 2022-06-17T12:47:03.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.409+0000 7f1a164f4700 1 -- 172.21.15.36:0/1972351466 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a10063440 con 0x7f1a101444f0 2022-06-17T12:47:03.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.409+0000 7f1a0d7fa700 1 -- 172.21.15.36:0/1972351466 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0800bb70 con 0x7f1a101444f0 2022-06-17T12:47:03.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.409+0000 7f1a0d7fa700 1 -- 172.21.15.36:0/1972351466 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a08021820 con 0x7f1a101444f0 2022-06-17T12:47:03.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.413+0000 7f1a164f4700 1 -- 172.21.15.36:0/1972351466 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e8003410 con 0x7f1a101444f0 2022-06-17T12:47:03.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.413+0000 7f1a0d7fa700 1 -- 172.21.15.36:0/1972351466 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f1a08029070 con 0x7f1a101444f0 2022-06-17T12:47:03.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.413+0000 7f1a0d7fa700 1 --2- 172.21.15.36:0/1972351466 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f19f00539b0 0x7f19f0055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:03.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.413+0000 7f1a0d7fa700 1 -- 172.21.15.36:0/1972351466 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f1a0806cf20 con 0x7f1a101444f0 2022-06-17T12:47:03.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.413+0000 7f1a0ffff700 1 --2- 172.21.15.36:0/1972351466 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f19f00539b0 0x7f19f0055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:03.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.413+0000 7f1a0ffff700 1 --2- 172.21.15.36:0/1972351466 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f19f00539b0 0x7f19f0055e70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f1a0400b000 tx=0x7f1a04006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:03.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.425+0000 7f1a0d7fa700 1 -- 172.21.15.36:0/1972351466 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1a08043b90 con 0x7f1a101444f0 2022-06-17T12:47:03.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.553+0000 7f1a164f4700 1 -- 172.21.15.36:0/1972351466 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f19e80036c0 con 0x7f1a101444f0 2022-06-17T12:47:03.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.557+0000 7f1a0d7fa700 1 -- 172.21.15.36:0/1972351466 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1a08015030 con 0x7f1a101444f0 2022-06-17T12:47:03.626 INFO:teuthology.orchestra.run.smithi036.stdout:55834574855 2022-06-17T12:47:03.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.561+0000 7f1a164f4700 1 -- 172.21.15.36:0/1972351466 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f19f00539b0 msgr2=0x7f19f0055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:03.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.561+0000 7f1a164f4700 1 --2- 172.21.15.36:0/1972351466 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f19f00539b0 0x7f19f0055e70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f1a0400b000 tx=0x7f1a04006df0).stop 2022-06-17T12:47:03.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.561+0000 7f1a164f4700 1 -- 172.21.15.36:0/1972351466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a101444f0 msgr2=0x7f1a10065850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:03.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.561+0000 7f1a164f4700 1 --2- 172.21.15.36:0/1972351466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a101444f0 0x7f1a10065850 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f1a080032f0 tx=0x7f1a08016000).stop 2022-06-17T12:47:03.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.561+0000 7f1a164f4700 1 -- 172.21.15.36:0/1972351466 shutdown_connections 2022-06-17T12:47:03.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.561+0000 7f1a164f4700 1 --2- 172.21.15.36:0/1972351466 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f19f00539b0 0x7f19f0055e70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.561+0000 7f1a164f4700 1 --2- 172.21.15.36:0/1972351466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a101444f0 0x7f1a10065850 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.561+0000 7f1a164f4700 1 --2- 172.21.15.36:0/1972351466 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a100ad330 0x7f1a10130750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.561+0000 7f1a164f4700 1 --2- 172.21.15.36:0/1972351466 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a10062770 0x7f1a100671d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.561+0000 7f1a164f4700 1 -- 172.21.15.36:0/1972351466 >> 172.21.15.36:0/1972351466 conn(0x7f1a100b53e0 msgr2=0x7f1a100b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:03.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.561+0000 7f1a164f4700 1 -- 172.21.15.36:0/1972351466 shutdown_connections 2022-06-17T12:47:03.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.561+0000 7f1a164f4700 1 -- 172.21.15.36:0/1972351466 wait complete. 2022-06-17T12:47:03.636 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.4 2022-06-17T12:47:03.638 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2022-06-17T12:47:03.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.793+0000 7fe03b6c3700 1 Processor -- start 2022-06-17T12:47:03.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.793+0000 7fe03b6c3700 1 -- start start 2022-06-17T12:47:03.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.793+0000 7fe03b6c3700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe034066c60 0x7fe0340ad880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:03.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.793+0000 7fe03b6c3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0340afd80 0x7fe034146640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:03.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.793+0000 7fe03b6c3700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe0340ad330 0x7fe034141fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:03.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.793+0000 7fe03b6c3700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0340add70 con 0x7fe0340ad330 2022-06-17T12:47:03.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.793+0000 7fe03b6c3700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe034067040 con 0x7fe034066c60 2022-06-17T12:47:03.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.793+0000 7fe03b6c3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0340b0940 con 0x7fe0340afd80 2022-06-17T12:47:03.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.793+0000 7fe039c60700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe0340ad330 0x7fe034141fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:03.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.793+0000 7fe039c60700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe0340ad330 0x7fe034141fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59808/0 (socket says 172.21.15.36:59808) 2022-06-17T12:47:03.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.793+0000 7fe039c60700 1 -- 172.21.15.36:0/2399575054 learned_addr learned my addr 172.21.15.36:0/2399575054 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:03.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.793+0000 7fe038c5e700 1 --2- 172.21.15.36:0/2399575054 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0340afd80 0x7fe034146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:03.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.793+0000 7fe03945f700 1 --2- 172.21.15.36:0/2399575054 >> v2:172.21.15.70:3300/0 conn(0x7fe034066c60 0x7fe0340ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:03.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.797+0000 7fe039c60700 1 -- 172.21.15.36:0/2399575054 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0340afd80 msgr2=0x7fe034146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:03.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.797+0000 7fe039c60700 1 --2- 172.21.15.36:0/2399575054 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0340afd80 0x7fe034146640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.797+0000 7fe039c60700 1 -- 172.21.15.36:0/2399575054 >> v2:172.21.15.70:3300/0 conn(0x7fe034066c60 msgr2=0x7fe0340ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:03.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.797+0000 7fe039c60700 1 --2- 172.21.15.36:0/2399575054 >> v2:172.21.15.70:3300/0 conn(0x7fe034066c60 0x7fe0340ad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.797+0000 7fe039c60700 1 -- 172.21.15.36:0/2399575054 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0340b1300 con 0x7fe0340ad330 2022-06-17T12:47:03.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.797+0000 7fe039c60700 1 --2- 172.21.15.36:0/2399575054 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0340ad330 0x7fe034141fc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fe02c0032f0 tx=0x7fe02c009290).ready entity=mon.0 client_cookie=7480ad6105ec80c9 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:03.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.797+0000 7fe02bfff700 1 -- 172.21.15.36:0/2399575054 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe02c00b870 con 0x7fe0340ad330 2022-06-17T12:47:03.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.797+0000 7fe02bfff700 1 -- 172.21.15.36:0/2399575054 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02c00b9d0 con 0x7fe0340ad330 2022-06-17T12:47:03.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.797+0000 7fe02bfff700 1 -- 172.21.15.36:0/2399575054 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe02c00bb40 con 0x7fe0340ad330 2022-06-17T12:47:03.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.797+0000 7fe03b6c3700 1 -- 172.21.15.36:0/2399575054 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0340ad330 msgr2=0x7fe034141fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:03.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.797+0000 7fe03b6c3700 1 --2- 172.21.15.36:0/2399575054 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0340ad330 0x7fe034141fc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fe02c0032f0 tx=0x7fe02c009290).stop 2022-06-17T12:47:03.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.797+0000 7fe03b6c3700 1 -- 172.21.15.36:0/2399575054 shutdown_connections 2022-06-17T12:47:03.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.797+0000 7fe03b6c3700 1 --2- 172.21.15.36:0/2399575054 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0340ad330 0x7fe034141fc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.797+0000 7fe03b6c3700 1 --2- 172.21.15.36:0/2399575054 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0340afd80 0x7fe034146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.797+0000 7fe03b6c3700 1 --2- 172.21.15.36:0/2399575054 >> v2:172.21.15.70:3300/0 conn(0x7fe034066c60 0x7fe0340ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.797+0000 7fe03b6c3700 1 -- 172.21.15.36:0/2399575054 >> 172.21.15.36:0/2399575054 conn(0x7fe0340b53e0 msgr2=0x7fe0340b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:03.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.797+0000 7fe03b6c3700 1 -- 172.21.15.36:0/2399575054 shutdown_connections 2022-06-17T12:47:03.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.797+0000 7fe03b6c3700 1 -- 172.21.15.36:0/2399575054 wait complete. 2022-06-17T12:47:03.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.801+0000 7fe03b6c3700 1 Processor -- start 2022-06-17T12:47:03.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.801+0000 7fe03b6c3700 1 -- start start 2022-06-17T12:47:03.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.801+0000 7fe03b6c3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe034066c60 0x7fe0341433b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:03.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.801+0000 7fe03b6c3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0340ad330 0x7fe034148fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:03.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.801+0000 7fe03b6c3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0340afd80 0x7fe0341450d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:03.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.801+0000 7fe03b6c3700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe034144f20 con 0x7fe034066c60 2022-06-17T12:47:03.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.801+0000 7fe03b6c3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe034141cd0 con 0x7fe0340ad330 2022-06-17T12:47:03.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.801+0000 7fe03b6c3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe034144c80 con 0x7fe0340afd80 2022-06-17T12:47:03.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.801+0000 7fe03945f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe034066c60 0x7fe0341433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:03.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.801+0000 7fe03945f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe034066c60 0x7fe0341433b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59810/0 (socket says 172.21.15.36:59810) 2022-06-17T12:47:03.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.801+0000 7fe03945f700 1 -- 172.21.15.36:0/542843147 learned_addr learned my addr 172.21.15.36:0/542843147 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:03.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.801+0000 7fe038c5e700 1 --2- 172.21.15.36:0/542843147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0340ad330 0x7fe034148fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:03.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.801+0000 7fe039c60700 1 --2- 172.21.15.36:0/542843147 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0340afd80 0x7fe0341450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:03.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.801+0000 7fe03945f700 1 -- 172.21.15.36:0/542843147 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0340afd80 msgr2=0x7fe0341450d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:03.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.801+0000 7fe03945f700 1 --2- 172.21.15.36:0/542843147 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0340afd80 0x7fe0341450d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.801+0000 7fe03945f700 1 -- 172.21.15.36:0/542843147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0340ad330 msgr2=0x7fe034148fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:03.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.801+0000 7fe03945f700 1 --2- 172.21.15.36:0/542843147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0340ad330 0x7fe034148fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.801+0000 7fe03945f700 1 -- 172.21.15.36:0/542843147 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02c009530 con 0x7fe034066c60 2022-06-17T12:47:03.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.801+0000 7fe039c60700 1 --2- 172.21.15.36:0/542843147 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0340afd80 0x7fe0341450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:03.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.801+0000 7fe03945f700 1 --2- 172.21.15.36:0/542843147 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe034066c60 0x7fe0341433b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fe0240030b0 tx=0x7fe024008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:03.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.801+0000 7fe02a7fc700 1 -- 172.21.15.36:0/542843147 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe024011780 con 0x7fe034066c60 2022-06-17T12:47:03.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.805+0000 7fe02a7fc700 1 -- 172.21.15.36:0/542843147 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0240118e0 con 0x7fe034066c60 2022-06-17T12:47:03.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.805+0000 7fe03b6c3700 1 -- 172.21.15.36:0/542843147 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe034148b00 con 0x7fe034066c60 2022-06-17T12:47:03.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.805+0000 7fe02a7fc700 1 -- 172.21.15.36:0/542843147 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe024011c40 con 0x7fe034066c60 2022-06-17T12:47:03.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.805+0000 7fe03b6c3700 1 -- 172.21.15.36:0/542843147 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0341447e0 con 0x7fe034066c60 2022-06-17T12:47:03.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.805+0000 7fe03b6c3700 1 -- 172.21.15.36:0/542843147 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe034147450 con 0x7fe034066c60 2022-06-17T12:47:03.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.805+0000 7fe02a7fc700 1 -- 172.21.15.36:0/542843147 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe024012420 con 0x7fe034066c60 2022-06-17T12:47:03.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.805+0000 7fe02a7fc700 1 --2- 172.21.15.36:0/542843147 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe01c053a40 0x7fe01c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:03.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.805+0000 7fe02a7fc700 1 -- 172.21.15.36:0/542843147 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe02406be40 con 0x7fe034066c60 2022-06-17T12:47:03.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.805+0000 7fe038c5e700 1 --2- 172.21.15.36:0/542843147 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe01c053a40 0x7fe01c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:03.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.809+0000 7fe038c5e700 1 --2- 172.21.15.36:0/542843147 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe01c053a40 0x7fe01c055f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fe03000c000 tx=0x7fe030008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:03.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.813+0000 7fe02a7fc700 1 -- 172.21.15.36:0/542843147 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe034147450 con 0x7fe034066c60 2022-06-17T12:47:03.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.941+0000 7fe03b6c3700 1 -- 172.21.15.36:0/542843147 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe034147450 con 0x7fe034066c60 2022-06-17T12:47:03.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.941+0000 7fe02a7fc700 1 -- 172.21.15.36:0/542843147 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe034147450 con 0x7fe034066c60 2022-06-17T12:47:03.931 INFO:teuthology.orchestra.run.smithi036.stdout:55834574855 2022-06-17T12:47:03.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.945+0000 7fe03b6c3700 1 -- 172.21.15.36:0/542843147 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe01c053a40 msgr2=0x7fe01c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:03.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.945+0000 7fe03b6c3700 1 --2- 172.21.15.36:0/542843147 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe01c053a40 0x7fe01c055f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fe03000c000 tx=0x7fe030008040).stop 2022-06-17T12:47:03.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.945+0000 7fe03b6c3700 1 -- 172.21.15.36:0/542843147 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe034066c60 msgr2=0x7fe0341433b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:03.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.945+0000 7fe03b6c3700 1 --2- 172.21.15.36:0/542843147 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe034066c60 0x7fe0341433b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fe0240030b0 tx=0x7fe024008040).stop 2022-06-17T12:47:03.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.945+0000 7fe03b6c3700 1 -- 172.21.15.36:0/542843147 shutdown_connections 2022-06-17T12:47:03.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.945+0000 7fe03b6c3700 1 --2- 172.21.15.36:0/542843147 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe01c053a40 0x7fe01c055f00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.945+0000 7fe03b6c3700 1 --2- 172.21.15.36:0/542843147 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0340afd80 0x7fe0341450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.945+0000 7fe03b6c3700 1 --2- 172.21.15.36:0/542843147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0340ad330 0x7fe034148fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.945+0000 7fe03b6c3700 1 --2- 172.21.15.36:0/542843147 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe034066c60 0x7fe0341433b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:03.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.945+0000 7fe03b6c3700 1 -- 172.21.15.36:0/542843147 >> 172.21.15.36:0/542843147 conn(0x7fe0340b53e0 msgr2=0x7fe0340b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:03.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.945+0000 7fe03b6c3700 1 -- 172.21.15.36:0/542843147 shutdown_connections 2022-06-17T12:47:03.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:03.945+0000 7fe03b6c3700 1 -- 172.21.15.36:0/542843147 wait complete. 2022-06-17T12:47:03.948 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2022-06-17T12:47:03.948 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2022-06-17T12:47:04.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.069+0000 7fb3aeeeb700 1 Processor -- start 2022-06-17T12:47:04.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.069+0000 7fb3aeeeb700 1 -- start start 2022-06-17T12:47:04.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.069+0000 7fb3aeeeb700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb3a8146d00 0x7fb3a8147150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:04.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.069+0000 7fb3aeeeb700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3a80b25e0 con 0x7fb3a80b2200 2022-06-17T12:47:04.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.069+0000 7fb3aeeeb700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3a80ad790 con 0x7fb3a80b2cd0 2022-06-17T12:47:04.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.069+0000 7fb3aeeeb700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3a80aef40 con 0x7fb3a8146d00 2022-06-17T12:47:04.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.069+0000 7fb3ae6ea700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb3a8146d00 0x7fb3a8147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:04.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.069+0000 7fb3adee9700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fb3a80b2200 0x7fb3a80af3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49116/0 (socket says 172.21.15.36:49116) 2022-06-17T12:47:04.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.069+0000 7fb3adee9700 1 -- 172.21.15.36:0/447620222 learned_addr learned my addr 172.21.15.36:0/447620222 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:04.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.069+0000 7fb3acee7700 1 -- 172.21.15.36:0/447620222 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2453568637 0 0) 0x7fb3a80b25e0 con 0x7fb3a80b2200 2022-06-17T12:47:04.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.069+0000 7fb3acee7700 1 -- 172.21.15.36:0/447620222 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb390003260 con 0x7fb3a80b2200 2022-06-17T12:47:04.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.069+0000 7fb3ae6ea700 1 -- 172.21.15.36:0/447620222 >> v1:172.21.15.70:6789/0 conn(0x7fb3a80b2cd0 legacy=0x7fb3a81465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:04.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.069+0000 7fb3ae6ea700 1 -- 172.21.15.36:0/447620222 >> v1:172.21.15.36:6789/0 conn(0x7fb3a80b2200 legacy=0x7fb3a80af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:04.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.069+0000 7fb3acee7700 1 -- 172.21.15.36:0/447620222 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3998079705 0 0) 0x7fb3a80ad790 con 0x7fb3a80b2cd0 2022-06-17T12:47:04.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.069+0000 7fb3ae6ea700 1 -- 172.21.15.36:0/447620222 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a80afaf0 con 0x7fb3a8146d00 2022-06-17T12:47:04.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.069+0000 7fb3ae6ea700 1 --2- 172.21.15.36:0/447620222 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3a8146d00 0x7fb3a8147150 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fb3a0002f70 tx=0x7fb3a0006df0).ready entity=mon.0 client_cookie=b657cd1623565296 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:04.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.069+0000 7fb3acee7700 1 -- 172.21.15.36:0/447620222 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3a000d610 con 0x7fb3a8146d00 2022-06-17T12:47:04.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.069+0000 7fb3acee7700 1 -- 172.21.15.36:0/447620222 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a0000e90 con 0x7fb3a8146d00 2022-06-17T12:47:04.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.069+0000 7fb3acee7700 1 -- 172.21.15.36:0/447620222 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3a000d960 con 0x7fb3a8146d00 2022-06-17T12:47:04.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3aeeeb700 1 -- 172.21.15.36:0/447620222 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3a8146d00 msgr2=0x7fb3a8147150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:04.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3aeeeb700 1 --2- 172.21.15.36:0/447620222 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3a8146d00 0x7fb3a8147150 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fb3a0002f70 tx=0x7fb3a0006df0).stop 2022-06-17T12:47:04.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3aeeeb700 1 -- 172.21.15.36:0/447620222 shutdown_connections 2022-06-17T12:47:04.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3aeeeb700 1 --2- 172.21.15.36:0/447620222 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3a8146d00 0x7fb3a8147150 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:04.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3aeeeb700 1 -- 172.21.15.36:0/447620222 >> 172.21.15.36:0/447620222 conn(0x7fb3a805f770 msgr2=0x7fb3a805fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:04.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3aeeeb700 1 -- 172.21.15.36:0/447620222 shutdown_connections 2022-06-17T12:47:04.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3aeeeb700 1 -- 172.21.15.36:0/447620222 wait complete. 2022-06-17T12:47:04.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3aeeeb700 1 Processor -- start 2022-06-17T12:47:04.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3aeeeb700 1 -- start start 2022-06-17T12:47:04.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3aeeeb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3a80b2200 0x7fb3a814c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:04.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3aeeeb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3a80b2cd0 0x7fb3a80b5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:04.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3aeeeb700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3a8146d00 0x7fb3a815a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:04.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3aeeeb700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a814a7e0 con 0x7fb3a80b2cd0 2022-06-17T12:47:04.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3aeeeb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a8064200 con 0x7fb3a80b2200 2022-06-17T12:47:04.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3aeeeb700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a80644a0 con 0x7fb3a8146d00 2022-06-17T12:47:04.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3ad6e8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3a80b2cd0 0x7fb3a80b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:04.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3ad6e8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3a80b2cd0 0x7fb3a80b5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59826/0 (socket says 172.21.15.36:59826) 2022-06-17T12:47:04.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3ad6e8700 1 -- 172.21.15.36:0/2188774222 learned_addr learned my addr 172.21.15.36:0/2188774222 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:04.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3ae6ea700 1 --2- 172.21.15.36:0/2188774222 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3a8146d00 0x7fb3a815a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:04.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3adee9700 1 --2- 172.21.15.36:0/2188774222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3a80b2200 0x7fb3a814c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:04.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3ad6e8700 1 -- 172.21.15.36:0/2188774222 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3a8146d00 msgr2=0x7fb3a815a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:04.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3ad6e8700 1 --2- 172.21.15.36:0/2188774222 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3a8146d00 0x7fb3a815a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:04.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3ad6e8700 1 -- 172.21.15.36:0/2188774222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3a80b2200 msgr2=0x7fb3a814c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:04.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3ad6e8700 1 --2- 172.21.15.36:0/2188774222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3a80b2200 0x7fb3a814c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:04.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.073+0000 7fb3ad6e8700 1 -- 172.21.15.36:0/2188774222 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a000b180 con 0x7fb3a80b2cd0 2022-06-17T12:47:04.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.077+0000 7fb3ad6e8700 1 --2- 172.21.15.36:0/2188774222 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3a80b2cd0 0x7fb3a80b5780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fb3a4001250 tx=0x7fb3a4007e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:04.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.077+0000 7fb396ffd700 1 -- 172.21.15.36:0/2188774222 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3a400e4a0 con 0x7fb3a80b2cd0 2022-06-17T12:47:04.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.077+0000 7fb396ffd700 1 -- 172.21.15.36:0/2188774222 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a400e600 con 0x7fb3a80b2cd0 2022-06-17T12:47:04.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.077+0000 7fb396ffd700 1 -- 172.21.15.36:0/2188774222 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3a400e960 con 0x7fb3a80b2cd0 2022-06-17T12:47:04.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.077+0000 7fb3aeeeb700 1 -- 172.21.15.36:0/2188774222 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a8064740 con 0x7fb3a80b2cd0 2022-06-17T12:47:04.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.077+0000 7fb3aeeeb700 1 -- 172.21.15.36:0/2188774222 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a80b5e90 con 0x7fb3a80b2cd0 2022-06-17T12:47:04.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.077+0000 7fb396ffd700 1 -- 172.21.15.36:0/2188774222 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fb3a4019030 con 0x7fb3a80b2cd0 2022-06-17T12:47:04.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.077+0000 7fb3aeeeb700 1 -- 172.21.15.36:0/2188774222 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3a80648d0 con 0x7fb3a80b2cd0 2022-06-17T12:47:04.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.077+0000 7fb396ffd700 1 --2- 172.21.15.36:0/2188774222 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb3880539b0 0x7fb388055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:04.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.077+0000 7fb396ffd700 1 -- 172.21.15.36:0/2188774222 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb3a406b9a0 con 0x7fb3a80b2cd0 2022-06-17T12:47:04.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.077+0000 7fb3adee9700 1 --2- 172.21.15.36:0/2188774222 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb3880539b0 0x7fb388055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:04.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.077+0000 7fb3adee9700 1 --2- 172.21.15.36:0/2188774222 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb3880539b0 0x7fb388055e70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fb39800d000 tx=0x7fb398009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:04.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.085+0000 7fb396ffd700 1 -- 172.21.15.36:0/2188774222 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb3a40427a0 con 0x7fb3a80b2cd0 2022-06-17T12:47:04.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.209+0000 7fb3aeeeb700 1 -- 172.21.15.36:0/2188774222 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fb3a8064000 con 0x7fb3a80b2cd0 2022-06-17T12:47:04.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.209+0000 7fb396ffd700 1 -- 172.21.15.36:0/2188774222 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb3a4017020 con 0x7fb3a80b2cd0 2022-06-17T12:47:04.217 INFO:teuthology.orchestra.run.smithi036.stdout:60129542151 2022-06-17T12:47:04.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.213+0000 7fb3aeeeb700 1 -- 172.21.15.36:0/2188774222 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb3880539b0 msgr2=0x7fb388055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:04.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.213+0000 7fb3aeeeb700 1 --2- 172.21.15.36:0/2188774222 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb3880539b0 0x7fb388055e70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fb39800d000 tx=0x7fb398009040).stop 2022-06-17T12:47:04.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.213+0000 7fb3aeeeb700 1 -- 172.21.15.36:0/2188774222 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3a80b2cd0 msgr2=0x7fb3a80b5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:04.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.213+0000 7fb3aeeeb700 1 --2- 172.21.15.36:0/2188774222 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3a80b2cd0 0x7fb3a80b5780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fb3a4001250 tx=0x7fb3a4007e40).stop 2022-06-17T12:47:04.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.213+0000 7fb3aeeeb700 1 -- 172.21.15.36:0/2188774222 shutdown_connections 2022-06-17T12:47:04.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.213+0000 7fb3aeeeb700 1 --2- 172.21.15.36:0/2188774222 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb3880539b0 0x7fb388055e70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:04.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.213+0000 7fb3aeeeb700 1 --2- 172.21.15.36:0/2188774222 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3a8146d00 0x7fb3a815a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:04.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.213+0000 7fb3aeeeb700 1 --2- 172.21.15.36:0/2188774222 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3a80b2cd0 0x7fb3a80b5780 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:04.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.213+0000 7fb3aeeeb700 1 --2- 172.21.15.36:0/2188774222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3a80b2200 0x7fb3a814c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:04.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.213+0000 7fb3aeeeb700 1 -- 172.21.15.36:0/2188774222 >> 172.21.15.36:0/2188774222 conn(0x7fb3a805f770 msgr2=0x7fb3a814cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:04.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.213+0000 7fb3aeeeb700 1 -- 172.21.15.36:0/2188774222 shutdown_connections 2022-06-17T12:47:04.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.213+0000 7fb3aeeeb700 1 -- 172.21.15.36:0/2188774222 wait complete. 2022-06-17T12:47:04.240 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.6 2022-06-17T12:47:04.251 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2022-06-17T12:47:04.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.401+0000 7fbb96c71700 1 Processor -- start 2022-06-17T12:47:04.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.401+0000 7fbb96c71700 1 -- start start 2022-06-17T12:47:04.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.401+0000 7fbb96c71700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbb90062b50 0x7fbb900ada40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:04.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.401+0000 7fbb96c71700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb900b0ba0 con 0x7fbb90062770 2022-06-17T12:47:04.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.401+0000 7fbb96c71700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb90147e90 con 0x7fbb90144c00 2022-06-17T12:47:04.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.401+0000 7fbb96c71700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb900ae820 con 0x7fbb90062b50 2022-06-17T12:47:04.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.401+0000 7fbb9520e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbb90062b50 0x7fbb900ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:04.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.401+0000 7fbb9520e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbb90062b50 0x7fbb900ada40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59836/0 (socket says 172.21.15.36:59836) 2022-06-17T12:47:04.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.401+0000 7fbb9520e700 1 -- 172.21.15.36:0/208737360 learned_addr learned my addr 172.21.15.36:0/208737360 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:04.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.401+0000 7fbb94a0d700 1 --1- 172.21.15.36:0/208737360 >> v1:172.21.15.36:6789/0 conn(0x7fbb90062770 0x7fbb901444f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49132/0 (socket says 172.21.15.36:49132) 2022-06-17T12:47:04.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.401+0000 7fbb9520e700 1 -- 172.21.15.36:0/208737360 >> v1:172.21.15.70:6789/0 conn(0x7fbb90144c00 legacy=0x7fbb900ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:04.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.401+0000 7fbb9520e700 1 -- 172.21.15.36:0/208737360 >> v1:172.21.15.36:6789/0 conn(0x7fbb90062770 legacy=0x7fbb901444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:04.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.401+0000 7fbb9520e700 1 -- 172.21.15.36:0/208737360 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb900b0dd0 con 0x7fbb90062b50 2022-06-17T12:47:04.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.401+0000 7fbb8f7fe700 1 -- 172.21.15.36:0/208737360 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550480645 0 0) 0x7fbb900b0ba0 con 0x7fbb90062770 2022-06-17T12:47:04.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.401+0000 7fbb9520e700 1 --2- 172.21.15.36:0/208737360 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb90062b50 0x7fbb900ada40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fbb880032f0 tx=0x7fbb88009290).ready entity=mon.0 client_cookie=1a130514451951d3 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:04.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.401+0000 7fbb8f7fe700 1 -- 172.21.15.36:0/208737360 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb8800b8a0 con 0x7fbb90062b50 2022-06-17T12:47:04.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.401+0000 7fbb8f7fe700 1 -- 172.21.15.36:0/208737360 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8800ba00 con 0x7fbb90062b50 2022-06-17T12:47:04.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.401+0000 7fbb8f7fe700 1 -- 172.21.15.36:0/208737360 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb8800bb70 con 0x7fbb90062b50 2022-06-17T12:47:04.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.405+0000 7fbb96c71700 1 -- 172.21.15.36:0/208737360 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb90062b50 msgr2=0x7fbb900ada40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:04.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.405+0000 7fbb96c71700 1 --2- 172.21.15.36:0/208737360 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb90062b50 0x7fbb900ada40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fbb880032f0 tx=0x7fbb88009290).stop 2022-06-17T12:47:04.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.405+0000 7fbb96c71700 1 -- 172.21.15.36:0/208737360 shutdown_connections 2022-06-17T12:47:04.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.405+0000 7fbb96c71700 1 --2- 172.21.15.36:0/208737360 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb90062b50 0x7fbb900ada40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:04.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.405+0000 7fbb96c71700 1 -- 172.21.15.36:0/208737360 >> 172.21.15.36:0/208737360 conn(0x7fbb900b53e0 msgr2=0x7fbb900b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:04.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.405+0000 7fbb96c71700 1 -- 172.21.15.36:0/208737360 shutdown_connections 2022-06-17T12:47:04.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.405+0000 7fbb96c71700 1 -- 172.21.15.36:0/208737360 wait complete. 2022-06-17T12:47:04.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.405+0000 7fbb96c71700 1 Processor -- start 2022-06-17T12:47:04.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.405+0000 7fbb96c71700 1 -- start start 2022-06-17T12:47:04.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.409+0000 7fbb96c71700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb90062770 0x7fbb900635a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:04.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.409+0000 7fbb96c71700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb90062b50 0x7fbb900ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:04.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.409+0000 7fbb96c71700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb90144c00 0x7fbb901453b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:04.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.409+0000 7fbb96c71700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb90064be0 con 0x7fbb90144c00 2022-06-17T12:47:04.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.409+0000 7fbb96c71700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb901458a0 con 0x7fbb90062b50 2022-06-17T12:47:04.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.409+0000 7fbb96c71700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb900647f0 con 0x7fbb90062770 2022-06-17T12:47:04.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.409+0000 7fbb9520e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb90144c00 0x7fbb901453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:04.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.409+0000 7fbb9520e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb90144c00 0x7fbb901453b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59842/0 (socket says 172.21.15.36:59842) 2022-06-17T12:47:04.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.409+0000 7fbb9520e700 1 -- 172.21.15.36:0/3437879556 learned_addr learned my addr 172.21.15.36:0/3437879556 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:04.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.409+0000 7fbb94a0d700 1 --2- 172.21.15.36:0/3437879556 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb90062770 0x7fbb900635a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:04.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.409+0000 7fbb8ffff700 1 --2- 172.21.15.36:0/3437879556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb90062b50 0x7fbb900ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:04.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.409+0000 7fbb9520e700 1 -- 172.21.15.36:0/3437879556 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb90062770 msgr2=0x7fbb900635a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:04.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.409+0000 7fbb9520e700 1 --2- 172.21.15.36:0/3437879556 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb90062770 0x7fbb900635a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:04.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.409+0000 7fbb9520e700 1 -- 172.21.15.36:0/3437879556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb90062b50 msgr2=0x7fbb900ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:04.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.409+0000 7fbb9520e700 1 --2- 172.21.15.36:0/3437879556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb90062b50 0x7fbb900ad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:04.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.409+0000 7fbb9520e700 1 -- 172.21.15.36:0/3437879556 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb88009530 con 0x7fbb90144c00 2022-06-17T12:47:04.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.409+0000 7fbb9520e700 1 --2- 172.21.15.36:0/3437879556 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb90144c00 0x7fbb901453b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fbb880032f0 tx=0x7fbb88016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:04.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.409+0000 7fbb8dffb700 1 -- 172.21.15.36:0/3437879556 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb8800bb70 con 0x7fbb90144c00 2022-06-17T12:47:04.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.409+0000 7fbb8dffb700 1 -- 172.21.15.36:0/3437879556 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8801f3f0 con 0x7fbb90144c00 2022-06-17T12:47:04.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.409+0000 7fbb96c71700 1 -- 172.21.15.36:0/3437879556 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb90064e40 con 0x7fbb90144c00 2022-06-17T12:47:04.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.409+0000 7fbb8dffb700 1 -- 172.21.15.36:0/3437879556 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb8801f750 con 0x7fbb90144c00 2022-06-17T12:47:04.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.409+0000 7fbb96c71700 1 -- 172.21.15.36:0/3437879556 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb900632b0 con 0x7fbb90144c00 2022-06-17T12:47:04.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.409+0000 7fbb96c71700 1 -- 172.21.15.36:0/3437879556 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb900b1c20 con 0x7fbb90144c00 2022-06-17T12:47:04.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.409+0000 7fbb8dffb700 1 -- 172.21.15.36:0/3437879556 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fbb88026030 con 0x7fbb90144c00 2022-06-17T12:47:04.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.413+0000 7fbb8dffb700 1 --2- 172.21.15.36:0/3437879556 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb68053ad0 0x7fbb68055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:04.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.413+0000 7fbb94a0d700 1 --2- 172.21.15.36:0/3437879556 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb68053ad0 0x7fbb68055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:04.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.413+0000 7fbb8dffb700 1 -- 172.21.15.36:0/3437879556 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fbb8806ce60 con 0x7fbb90144c00 2022-06-17T12:47:04.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.413+0000 7fbb94a0d700 1 --2- 172.21.15.36:0/3437879556 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb68053ad0 0x7fbb68055f90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fbb8000d000 tx=0x7fbb80007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:04.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.417+0000 7fbb8dffb700 1 -- 172.21.15.36:0/3437879556 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbb88043b50 con 0x7fbb90144c00 2022-06-17T12:47:04.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.541+0000 7fbb96c71700 1 -- 172.21.15.36:0/3437879556 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fbb90057410 con 0x7fbb90144c00 2022-06-17T12:47:04.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.541+0000 7fbb8dffb700 1 -- 172.21.15.36:0/3437879556 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbb88024090 con 0x7fbb90144c00 2022-06-17T12:47:04.532 INFO:teuthology.orchestra.run.smithi036.stdout:60129542150 2022-06-17T12:47:04.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.545+0000 7fbb96c71700 1 -- 172.21.15.36:0/3437879556 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb68053ad0 msgr2=0x7fbb68055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:04.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.545+0000 7fbb96c71700 1 --2- 172.21.15.36:0/3437879556 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb68053ad0 0x7fbb68055f90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fbb8000d000 tx=0x7fbb80007a00).stop 2022-06-17T12:47:04.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.545+0000 7fbb96c71700 1 -- 172.21.15.36:0/3437879556 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb90144c00 msgr2=0x7fbb901453b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:04.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.545+0000 7fbb96c71700 1 --2- 172.21.15.36:0/3437879556 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb90144c00 0x7fbb901453b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fbb880032f0 tx=0x7fbb88016000).stop 2022-06-17T12:47:04.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.545+0000 7fbb96c71700 1 -- 172.21.15.36:0/3437879556 shutdown_connections 2022-06-17T12:47:04.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.545+0000 7fbb96c71700 1 --2- 172.21.15.36:0/3437879556 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb68053ad0 0x7fbb68055f90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:04.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.545+0000 7fbb96c71700 1 --2- 172.21.15.36:0/3437879556 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb90144c00 0x7fbb901453b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:04.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.545+0000 7fbb96c71700 1 --2- 172.21.15.36:0/3437879556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb90062b50 0x7fbb900ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:04.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.545+0000 7fbb96c71700 1 --2- 172.21.15.36:0/3437879556 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb90062770 0x7fbb900635a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:04.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.545+0000 7fbb96c71700 1 -- 172.21.15.36:0/3437879556 >> 172.21.15.36:0/3437879556 conn(0x7fbb900b53e0 msgr2=0x7fbb900aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:04.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.545+0000 7fbb96c71700 1 -- 172.21.15.36:0/3437879556 shutdown_connections 2022-06-17T12:47:04.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.545+0000 7fbb96c71700 1 -- 172.21.15.36:0/3437879556 wait complete. 2022-06-17T12:47:04.596 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.7 2022-06-17T12:47:04.596 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2022-06-17T12:47:04.597 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-06-17T12:47:04.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.717+0000 7f9410aaf700 1 Processor -- start 2022-06-17T12:47:04.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.717+0000 7f9410aaf700 1 -- start start 2022-06-17T12:47:04.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.717+0000 7f9410aaf700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f940c063730 0x7f940c0b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:04.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.717+0000 7f9410aaf700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f940c140610 0x7f940c0b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:04.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.717+0000 7f9410aaf700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f940c063b10 con 0x7f940c064eb0 2022-06-17T12:47:04.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.717+0000 7f9410aaf700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f940c147430 con 0x7f940c063730 2022-06-17T12:47:04.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.717+0000 7f9410aaf700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f940c0b1ca0 con 0x7f940c140610 2022-06-17T12:47:04.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.717+0000 7f940affd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f940c064eb0 0x7f940c13ff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49144/0 (socket says 172.21.15.36:49144) 2022-06-17T12:47:04.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.717+0000 7f940affd700 1 -- 172.21.15.36:0/292517641 learned_addr learned my addr 172.21.15.36:0/292517641 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:04.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.717+0000 7f940bfff700 1 --2- 172.21.15.36:0/292517641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f940c140610 0x7f940c0b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:04.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940b7fe700 1 --2- 172.21.15.36:0/292517641 >> v2:172.21.15.70:3300/0 conn(0x7f940c063730 0x7f940c0b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:04.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940a7fc700 1 -- 172.21.15.36:0/292517641 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2106687933 0 0) 0x7f940c063b10 con 0x7f940c064eb0 2022-06-17T12:47:04.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940a7fc700 1 -- 172.21.15.36:0/292517641 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93ec003260 con 0x7f940c064eb0 2022-06-17T12:47:04.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940a7fc700 1 -- 172.21.15.36:0/292517641 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2922201314 0 0) 0x7f93ec003260 con 0x7f940c064eb0 2022-06-17T12:47:04.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940a7fc700 1 -- 172.21.15.36:0/292517641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f940c140610 msgr2=0x7f940c0b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:04.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940a7fc700 1 --2- 172.21.15.36:0/292517641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f940c140610 0x7f940c0b33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:04.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940a7fc700 1 -- 172.21.15.36:0/292517641 >> v2:172.21.15.70:3300/0 conn(0x7f940c063730 msgr2=0x7f940c0b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:04.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940a7fc700 1 --2- 172.21.15.36:0/292517641 >> v2:172.21.15.70:3300/0 conn(0x7f940c063730 0x7f940c0b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:04.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940a7fc700 1 -- 172.21.15.36:0/292517641 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f940c1464d0 con 0x7f940c064eb0 2022-06-17T12:47:04.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940a7fc700 1 -- 172.21.15.36:0/292517641 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f93f4001660 con 0x7f940c064eb0 2022-06-17T12:47:04.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940a7fc700 1 -- 172.21.15.36:0/292517641 >> v1:172.21.15.36:6789/0 conn(0x7f940c064eb0 legacy=0x7f940c13ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:04.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940a7fc700 1 --2- 172.21.15.36:0/292517641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93ec003780 0x7f93ec003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:04.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940a7fc700 1 --2- 172.21.15.36:0/292517641 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93ec004500 0x7f93ec004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:04.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940a7fc700 1 --2- 172.21.15.36:0/292517641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f940c140610 0x7f93ec004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:04.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940a7fc700 1 -- 172.21.15.36:0/292517641 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c147430 con 0x7f93ec004500 2022-06-17T12:47:04.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940a7fc700 1 -- 172.21.15.36:0/292517641 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c0b1ca0 con 0x7f93ec003780 2022-06-17T12:47:04.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940a7fc700 1 -- 172.21.15.36:0/292517641 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec00b750 con 0x7f940c140610 2022-06-17T12:47:04.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940affd700 1 --2- 172.21.15.36:0/292517641 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93ec004500 0x7f93ec004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:04.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940b7fe700 1 --2- 172.21.15.36:0/292517641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93ec003780 0x7f93ec003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:04.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940bfff700 1 --2- 172.21.15.36:0/292517641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f940c140610 0x7f93ec004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:04.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940affd700 1 -- 172.21.15.36:0/292517641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f940c140610 msgr2=0x7f93ec004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:04.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940affd700 1 --2- 172.21.15.36:0/292517641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f940c140610 0x7f93ec004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:04.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940affd700 1 -- 172.21.15.36:0/292517641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93ec003780 msgr2=0x7f93ec003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:04.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940affd700 1 --2- 172.21.15.36:0/292517641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93ec003780 0x7f93ec003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:04.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940affd700 1 -- 172.21.15.36:0/292517641 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ec003260 con 0x7f93ec004500 2022-06-17T12:47:04.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940bfff700 1 --2- 172.21.15.36:0/292517641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f940c140610 0x7f93ec004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:47:04.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940affd700 1 --2- 172.21.15.36:0/292517641 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93ec004500 0x7f93ec004980 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f940c063ca0 tx=0x7f940c05f8c0).ready entity=mon.0 client_cookie=bf8dffe50904900 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:04.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940a7fc700 1 -- 172.21.15.36:0/292517641 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93f40070d0 con 0x7f93ec004500 2022-06-17T12:47:04.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940a7fc700 1 -- 172.21.15.36:0/292517641 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f4007230 con 0x7f93ec004500 2022-06-17T12:47:04.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940a7fc700 1 -- 172.21.15.36:0/292517641 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93f400c8c0 con 0x7f93ec004500 2022-06-17T12:47:04.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f9410aaf700 1 -- 172.21.15.36:0/292517641 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93ec004500 msgr2=0x7f93ec004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:04.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f9410aaf700 1 --2- 172.21.15.36:0/292517641 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93ec004500 0x7f93ec004980 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f940c063ca0 tx=0x7f940c05f8c0).stop 2022-06-17T12:47:04.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f940bfff700 1 -- 172.21.15.36:0/292517641 reap_dead start 2022-06-17T12:47:04.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f9410aaf700 1 -- 172.21.15.36:0/292517641 shutdown_connections 2022-06-17T12:47:04.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f9410aaf700 1 -- 172.21.15.36:0/292517641 >> 172.21.15.36:0/292517641 conn(0x7f940c0614b0 msgr2=0x7f940c0618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:04.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f9410aaf700 1 -- 172.21.15.36:0/292517641 shutdown_connections 2022-06-17T12:47:04.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.721+0000 7f9410aaf700 1 -- 172.21.15.36:0/292517641 wait complete. 2022-06-17T12:47:04.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f9410aaf700 1 Processor -- start 2022-06-17T12:47:04.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f9410aaf700 1 -- start start 2022-06-17T12:47:04.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f9410aaf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f940c0b4440 0x7f940c0b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:04.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f9410aaf700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f940c0b5a20 0x7f940c0afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:04.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f9410aaf700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f940c0b0050 0x7f940c1557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:04.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f9410aaf700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c115200 con 0x7f940c0b0050 2022-06-17T12:47:04.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f9410aaf700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c114820 con 0x7f940c0b4440 2022-06-17T12:47:04.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f9410aaf700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f940c114ac0 con 0x7f940c0b5a20 2022-06-17T12:47:04.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f940bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f940c0b0050 0x7f940c1557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:04.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f940affd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f940c0b5a20 0x7f940c0afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:04.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f940bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f940c0b0050 0x7f940c1557e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59862/0 (socket says 172.21.15.36:59862) 2022-06-17T12:47:04.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f940bfff700 1 -- 172.21.15.36:0/2380958280 learned_addr learned my addr 172.21.15.36:0/2380958280 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:04.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f940b7fe700 1 --2- 172.21.15.36:0/2380958280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f940c0b4440 0x7f940c0b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:04.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f940bfff700 1 -- 172.21.15.36:0/2380958280 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f940c0b5a20 msgr2=0x7f940c0afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:04.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f940bfff700 1 --2- 172.21.15.36:0/2380958280 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f940c0b5a20 0x7f940c0afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:04.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f940bfff700 1 -- 172.21.15.36:0/2380958280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f940c0b4440 msgr2=0x7f940c0b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:04.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f940bfff700 1 --2- 172.21.15.36:0/2380958280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f940c0b4440 0x7f940c0b4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:04.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f940bfff700 1 -- 172.21.15.36:0/2380958280 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f940c114d60 con 0x7f940c0b0050 2022-06-17T12:47:04.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f940affd700 1 --2- 172.21.15.36:0/2380958280 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f940c0b5a20 0x7f940c0afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:04.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f940bfff700 1 --2- 172.21.15.36:0/2380958280 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f940c0b0050 0x7f940c1557e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f940400b000 tx=0x7f9404008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:04.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f9408ff9700 1 -- 172.21.15.36:0/2380958280 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9404018070 con 0x7f940c0b0050 2022-06-17T12:47:04.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f9408ff9700 1 -- 172.21.15.36:0/2380958280 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9404013770 con 0x7f940c0b0050 2022-06-17T12:47:04.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f9410aaf700 1 -- 172.21.15.36:0/2380958280 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f940c0b5e80 con 0x7f940c0b0050 2022-06-17T12:47:04.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f9408ff9700 1 -- 172.21.15.36:0/2380958280 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9404013ad0 con 0x7f940c0b0050 2022-06-17T12:47:04.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.725+0000 7f9410aaf700 1 -- 172.21.15.36:0/2380958280 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f940c155cd0 con 0x7f940c0b0050 2022-06-17T12:47:04.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.729+0000 7f9410aaf700 1 -- 172.21.15.36:0/2380958280 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f940c13edc0 con 0x7f940c0b0050 2022-06-17T12:47:04.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.729+0000 7f9408ff9700 1 -- 172.21.15.36:0/2380958280 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f9404012040 con 0x7f940c0b0050 2022-06-17T12:47:04.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.729+0000 7f9408ff9700 1 --2- 172.21.15.36:0/2380958280 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f93f0053a60 0x7f93f0055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:04.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.729+0000 7f940b7fe700 1 --2- 172.21.15.36:0/2380958280 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f93f0053a60 0x7f93f0055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:04.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.729+0000 7f9408ff9700 1 -- 172.21.15.36:0/2380958280 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f940406bf90 con 0x7f940c0b0050 2022-06-17T12:47:04.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.729+0000 7f940b7fe700 1 --2- 172.21.15.36:0/2380958280 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f93f0053a60 0x7f93f0055f20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f93fc00b000 tx=0x7f93fc006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:04.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.733+0000 7f9408ff9700 1 -- 172.21.15.36:0/2380958280 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f940c13edc0 con 0x7f940c0b0050 2022-06-17T12:47:04.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.861+0000 7f9410aaf700 1 -- 172.21.15.36:0/2380958280 --> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f940c13edc0 con 0x7f93f0053a60 2022-06-17T12:47:04.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.869+0000 7f9408ff9700 1 -- 172.21.15.36:0/2380958280 <== mgr.4104 v2:172.21.15.36:6800/22419 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82373 (secure 0 0 0) 0x7f940c13edc0 con 0x7f93f0053a60 2022-06-17T12:47:04.937 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:04.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.869+0000 7f9410aaf700 1 -- 172.21.15.36:0/2380958280 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f93f0053a60 msgr2=0x7f93f0055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:04.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.869+0000 7f9410aaf700 1 --2- 172.21.15.36:0/2380958280 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f93f0053a60 0x7f93f0055f20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f93fc00b000 tx=0x7f93fc006df0).stop 2022-06-17T12:47:04.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.869+0000 7f9410aaf700 1 -- 172.21.15.36:0/2380958280 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f940c0b0050 msgr2=0x7f940c1557e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:04.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.869+0000 7f9410aaf700 1 --2- 172.21.15.36:0/2380958280 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f940c0b0050 0x7f940c1557e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f940400b000 tx=0x7f9404008040).stop 2022-06-17T12:47:04.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.869+0000 7f9410aaf700 1 -- 172.21.15.36:0/2380958280 shutdown_connections 2022-06-17T12:47:04.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.869+0000 7f9410aaf700 1 --2- 172.21.15.36:0/2380958280 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f93f0053a60 0x7f93f0055f20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:04.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.869+0000 7f9410aaf700 1 --2- 172.21.15.36:0/2380958280 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f940c0b0050 0x7f940c1557e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:04.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.869+0000 7f9410aaf700 1 --2- 172.21.15.36:0/2380958280 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f940c0b5a20 0x7f940c0afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:04.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.869+0000 7f9410aaf700 1 --2- 172.21.15.36:0/2380958280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f940c0b4440 0x7f940c0b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:04.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.869+0000 7f9410aaf700 1 -- 172.21.15.36:0/2380958280 >> 172.21.15.36:0/2380958280 conn(0x7f940c0614b0 msgr2=0x7f940c0b3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:04.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.869+0000 7f9410aaf700 1 -- 172.21.15.36:0/2380958280 shutdown_connections 2022-06-17T12:47:04.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:04.869+0000 7f9410aaf700 1 -- 172.21.15.36:0/2380958280 wait complete. 2022-06-17T12:47:04.969 INFO:teuthology.orchestra.run.smithi036.stderr:dumped all 2022-06-17T12:47:04.969 INFO:teuthology.orchestra.run.smithi036.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2022-06-17T12:47:03.496542+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390880,"kb_used_data":2208,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746583840,"statfs":{"total":773094113280,"available":764501852160,"internally_reserved":8589934592,"allocated":2260992,"data_stored":926048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002088"},"pg_stats":[{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.628424+0000","last_change":"2022-06-17T12:46:52.269670+0000","last_active":"2022-06-17T12:46:52.628424+0000","last_peered":"2022-06-17T12:46:52.628424+0000","last_clean":"2022-06-17T12:46:52.628424+0000","last_became_active":"2022-06-17T12:46:52.267094+0000","last_became_peered":"2022-06-17T12:46:52.267094+0000","last_unstale":"2022-06-17T12:46:52.628424+0000","last_undegraded":"2022-06-17T12:46:52.628424+0000","last_fullsized":"2022-06-17T12:46:52.628424+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_clean_scrub_stamp":"2022-06-17T12:46:50.849792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.277769+0000","last_change":"2022-06-17T12:46:41.622624+0000","last_active":"2022-06-17T12:46:52.277769+0000","last_peered":"2022-06-17T12:46:52.277769+0000","last_clean":"2022-06-17T12:46:52.277769+0000","last_became_active":"2022-06-17T12:46:41.622138+0000","last_became_peered":"2022-06-17T12:46:41.622138+0000","last_unstale":"2022-06-17T12:46:52.277769+0000","last_undegraded":"2022-06-17T12:46:52.277769+0000","last_fullsized":"2022-06-17T12:46:52.277769+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":4,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:34.095313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:34.095313+0000","last_clean_scrub_stamp":"2022-06-17T12:46:34.095313+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.271351+0000","last_change":"2022-06-17T12:46:45.658148+0000","last_active":"2022-06-17T12:46:52.271351+0000","last_peered":"2022-06-17T12:46:52.271351+0000","last_clean":"2022-06-17T12:46:52.271351+0000","last_became_active":"2022-06-17T12:46:45.657682+0000","last_became_peered":"2022-06-17T12:46:45.657682+0000","last_unstale":"2022-06-17T12:46:52.271351+0000","last_undegraded":"2022-06-17T12:46:52.271351+0000","last_fullsized":"2022-06-17T12:46:52.271351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"25'1","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.273316+0000","last_change":"2022-06-17T12:46:48.848511+0000","last_active":"2022-06-17T12:46:52.273316+0000","last_peered":"2022-06-17T12:46:52.273316+0000","last_clean":"2022-06-17T12:46:52.273316+0000","last_became_active":"2022-06-17T12:46:48.848184+0000","last_became_peered":"2022-06-17T12:46:48.848184+0000","last_unstale":"2022-06-17T12:46:52.273316+0000","last_undegraded":"2022-06-17T12:46:52.273316+0000","last_fullsized":"2022-06-17T12:46:52.273316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_clean_scrub_stamp":"2022-06-17T12:46:47.827822+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.274485+0000","last_change":"2022-06-17T12:46:52.263152+0000","last_active":"2022-06-17T12:46:52.274485+0000","last_peered":"2022-06-17T12:46:52.274485+0000","last_clean":"2022-06-17T12:46:52.274485+0000","last_became_active":"2022-06-17T12:46:52.262694+0000","last_became_peered":"2022-06-17T12:46:52.262694+0000","last_unstale":"2022-06-17T12:46:52.274485+0000","last_undegraded":"2022-06-17T12:46:52.274485+0000","last_fullsized":"2022-06-17T12:46:52.274485+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_clean_scrub_stamp":"2022-06-17T12:46:50.849792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.277662+0000","last_change":"2022-06-17T12:46:45.662621+0000","last_active":"2022-06-17T12:46:52.277662+0000","last_peered":"2022-06-17T12:46:52.277662+0000","last_clean":"2022-06-17T12:46:52.277662+0000","last_became_active":"2022-06-17T12:46:45.662293+0000","last_became_peered":"2022-06-17T12:46:45.662293+0000","last_unstale":"2022-06-17T12:46:52.277662+0000","last_undegraded":"2022-06-17T12:46:52.277662+0000","last_fullsized":"2022-06-17T12:46:52.277662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.274079+0000","last_change":"2022-06-17T12:46:48.851317+0000","last_active":"2022-06-17T12:46:52.274079+0000","last_peered":"2022-06-17T12:46:52.274079+0000","last_clean":"2022-06-17T12:46:52.274079+0000","last_became_active":"2022-06-17T12:46:48.850976+0000","last_became_peered":"2022-06-17T12:46:48.850976+0000","last_unstale":"2022-06-17T12:46:52.274079+0000","last_undegraded":"2022-06-17T12:46:52.274079+0000","last_fullsized":"2022-06-17T12:46:52.274079+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_clean_scrub_stamp":"2022-06-17T12:46:47.827822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.272725+0000","last_change":"2022-06-17T12:46:52.264340+0000","last_active":"2022-06-17T12:46:52.272725+0000","last_peered":"2022-06-17T12:46:52.272725+0000","last_clean":"2022-06-17T12:46:52.272725+0000","last_became_active":"2022-06-17T12:46:52.262659+0000","last_became_peered":"2022-06-17T12:46:52.262659+0000","last_unstale":"2022-06-17T12:46:52.272725+0000","last_undegraded":"2022-06-17T12:46:52.272725+0000","last_fullsized":"2022-06-17T12:46:52.272725+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_clean_scrub_stamp":"2022-06-17T12:46:50.849792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.278375+0000","last_change":"2022-06-17T12:46:46.357931+0000","last_active":"2022-06-17T12:46:52.278375+0000","last_peered":"2022-06-17T12:46:52.278375+0000","last_clean":"2022-06-17T12:46:52.278375+0000","last_became_active":"2022-06-17T12:46:46.356789+0000","last_became_peered":"2022-06-17T12:46:46.356789+0000","last_unstale":"2022-06-17T12:46:52.278375+0000","last_undegraded":"2022-06-17T12:46:52.278375+0000","last_fullsized":"2022-06-17T12:46:52.278375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"25'8","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.280532+0000","last_change":"2022-06-17T12:46:48.853511+0000","last_active":"2022-06-17T12:46:52.280532+0000","last_peered":"2022-06-17T12:46:52.280532+0000","last_clean":"2022-06-17T12:46:52.280532+0000","last_became_active":"2022-06-17T12:46:48.853120+0000","last_became_peered":"2022-06-17T12:46:48.853120+0000","last_unstale":"2022-06-17T12:46:52.280532+0000","last_undegraded":"2022-06-17T12:46:52.280532+0000","last_fullsized":"2022-06-17T12:46:52.280532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_clean_scrub_stamp":"2022-06-17T12:46:47.827822+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.274277+0000","last_change":"2022-06-17T12:46:52.259111+0000","last_active":"2022-06-17T12:46:52.274277+0000","last_peered":"2022-06-17T12:46:52.274277+0000","last_clean":"2022-06-17T12:46:52.274277+0000","last_became_active":"2022-06-17T12:46:52.257942+0000","last_became_peered":"2022-06-17T12:46:52.257942+0000","last_unstale":"2022-06-17T12:46:52.274277+0000","last_undegraded":"2022-06-17T12:46:52.274277+0000","last_fullsized":"2022-06-17T12:46:52.274277+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_clean_scrub_stamp":"2022-06-17T12:46:50.849792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.271354+0000","last_change":"2022-06-17T12:46:45.662903+0000","last_active":"2022-06-17T12:46:52.271354+0000","last_peered":"2022-06-17T12:46:52.271354+0000","last_clean":"2022-06-17T12:46:52.271354+0000","last_became_active":"2022-06-17T12:46:45.662460+0000","last_became_peered":"2022-06-17T12:46:45.662460+0000","last_unstale":"2022-06-17T12:46:52.271354+0000","last_undegraded":"2022-06-17T12:46:52.271354+0000","last_fullsized":"2022-06-17T12:46:52.271354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"25'5","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.284343+0000","last_change":"2022-06-17T12:46:48.856578+0000","last_active":"2022-06-17T12:46:52.284343+0000","last_peered":"2022-06-17T12:46:52.284343+0000","last_clean":"2022-06-17T12:46:52.284343+0000","last_became_active":"2022-06-17T12:46:48.856155+0000","last_became_peered":"2022-06-17T12:46:48.856155+0000","last_unstale":"2022-06-17T12:46:52.284343+0000","last_undegraded":"2022-06-17T12:46:52.284343+0000","last_fullsized":"2022-06-17T12:46:52.284343+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_clean_scrub_stamp":"2022-06-17T12:46:47.827822+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.272515+0000","last_change":"2022-06-17T12:46:52.263338+0000","last_active":"2022-06-17T12:46:52.272515+0000","last_peered":"2022-06-17T12:46:52.272515+0000","last_clean":"2022-06-17T12:46:52.272515+0000","last_became_active":"2022-06-17T12:46:52.262706+0000","last_became_peered":"2022-06-17T12:46:52.262706+0000","last_unstale":"2022-06-17T12:46:52.272515+0000","last_undegraded":"2022-06-17T12:46:52.272515+0000","last_fullsized":"2022-06-17T12:46:52.272515+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_clean_scrub_stamp":"2022-06-17T12:46:50.849792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"25'6","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.282758+0000","last_change":"2022-06-17T12:46:48.855213+0000","last_active":"2022-06-17T12:46:52.282758+0000","last_peered":"2022-06-17T12:46:52.282758+0000","last_clean":"2022-06-17T12:46:52.282758+0000","last_became_active":"2022-06-17T12:46:48.854686+0000","last_became_peered":"2022-06-17T12:46:48.854686+0000","last_unstale":"2022-06-17T12:46:52.282758+0000","last_undegraded":"2022-06-17T12:46:52.282758+0000","last_fullsized":"2022-06-17T12:46:52.282758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_clean_scrub_stamp":"2022-06-17T12:46:47.827822+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.274561+0000","last_change":"2022-06-17T12:46:46.261176+0000","last_active":"2022-06-17T12:46:52.274561+0000","last_peered":"2022-06-17T12:46:52.274561+0000","last_clean":"2022-06-17T12:46:52.274561+0000","last_became_active":"2022-06-17T12:46:46.260143+0000","last_became_peered":"2022-06-17T12:46:46.260143+0000","last_unstale":"2022-06-17T12:46:52.274561+0000","last_undegraded":"2022-06-17T12:46:52.274561+0000","last_fullsized":"2022-06-17T12:46:52.274561+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.274554+0000","last_change":"2022-06-17T12:46:52.263813+0000","last_active":"2022-06-17T12:46:52.274554+0000","last_peered":"2022-06-17T12:46:52.274554+0000","last_clean":"2022-06-17T12:46:52.274554+0000","last_became_active":"2022-06-17T12:46:52.262614+0000","last_became_peered":"2022-06-17T12:46:52.262614+0000","last_unstale":"2022-06-17T12:46:52.274554+0000","last_undegraded":"2022-06-17T12:46:52.274554+0000","last_fullsized":"2022-06-17T12:46:52.274554+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_clean_scrub_stamp":"2022-06-17T12:46:50.849792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"25'8","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.288159+0000","last_change":"2022-06-17T12:46:48.852628+0000","last_active":"2022-06-17T12:46:52.288159+0000","last_peered":"2022-06-17T12:46:52.288159+0000","last_clean":"2022-06-17T12:46:52.288159+0000","last_became_active":"2022-06-17T12:46:48.852216+0000","last_became_peered":"2022-06-17T12:46:48.852216+0000","last_unstale":"2022-06-17T12:46:52.288159+0000","last_undegraded":"2022-06-17T12:46:52.288159+0000","last_fullsized":"2022-06-17T12:46:52.288159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_clean_scrub_stamp":"2022-06-17T12:46:47.827822+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.277410+0000","last_change":"2022-06-17T12:46:46.265378+0000","last_active":"2022-06-17T12:46:52.277410+0000","last_peered":"2022-06-17T12:46:52.277410+0000","last_clean":"2022-06-17T12:46:52.277410+0000","last_became_active":"2022-06-17T12:46:46.264499+0000","last_became_peered":"2022-06-17T12:46:46.264499+0000","last_unstale":"2022-06-17T12:46:52.277410+0000","last_undegraded":"2022-06-17T12:46:52.277410+0000","last_fullsized":"2022-06-17T12:46:52.277410+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.266946+0000","last_change":"2022-06-17T12:46:52.265747+0000","last_active":"2022-06-17T12:46:52.266946+0000","last_peered":"2022-06-17T12:46:52.266946+0000","last_clean":"2022-06-17T12:46:52.266946+0000","last_became_active":"2022-06-17T12:46:52.265143+0000","last_became_peered":"2022-06-17T12:46:52.265143+0000","last_unstale":"2022-06-17T12:46:52.266946+0000","last_undegraded":"2022-06-17T12:46:52.266946+0000","last_fullsized":"2022-06-17T12:46:52.266946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_clean_scrub_stamp":"2022-06-17T12:46:50.849792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"25'7","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.278014+0000","last_change":"2022-06-17T12:46:49.297977+0000","last_active":"2022-06-17T12:46:52.278014+0000","last_peered":"2022-06-17T12:46:52.278014+0000","last_clean":"2022-06-17T12:46:52.278014+0000","last_became_active":"2022-06-17T12:46:49.297174+0000","last_became_peered":"2022-06-17T12:46:49.297174+0000","last_unstale":"2022-06-17T12:46:52.278014+0000","last_undegraded":"2022-06-17T12:46:52.278014+0000","last_fullsized":"2022-06-17T12:46:52.278014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_clean_scrub_stamp":"2022-06-17T12:46:47.827822+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.272385+0000","last_change":"2022-06-17T12:46:45.658907+0000","last_active":"2022-06-17T12:46:52.272385+0000","last_peered":"2022-06-17T12:46:52.272385+0000","last_clean":"2022-06-17T12:46:52.272385+0000","last_became_active":"2022-06-17T12:46:45.658569+0000","last_became_peered":"2022-06-17T12:46:45.658569+0000","last_unstale":"2022-06-17T12:46:52.272385+0000","last_undegraded":"2022-06-17T12:46:52.272385+0000","last_fullsized":"2022-06-17T12:46:52.272385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.272444+0000","last_change":"2022-06-17T12:46:52.271397+0000","last_active":"2022-06-17T12:46:52.272444+0000","last_peered":"2022-06-17T12:46:52.272444+0000","last_clean":"2022-06-17T12:46:52.272444+0000","last_became_active":"2022-06-17T12:46:52.267136+0000","last_became_peered":"2022-06-17T12:46:52.267136+0000","last_unstale":"2022-06-17T12:46:52.272444+0000","last_undegraded":"2022-06-17T12:46:52.272444+0000","last_fullsized":"2022-06-17T12:46:52.272444+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_clean_scrub_stamp":"2022-06-17T12:46:50.849792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"26'4","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2022-06-17T12:46:54.496698+0000","last_change":"2022-06-17T12:46:48.853616+0000","last_active":"2022-06-17T12:46:54.496698+0000","last_peered":"2022-06-17T12:46:54.496698+0000","last_clean":"2022-06-17T12:46:54.496698+0000","last_became_active":"2022-06-17T12:46:48.853263+0000","last_became_peered":"2022-06-17T12:46:48.853263+0000","last_unstale":"2022-06-17T12:46:54.496698+0000","last_undegraded":"2022-06-17T12:46:54.496698+0000","last_fullsized":"2022-06-17T12:46:54.496698+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_clean_scrub_stamp":"2022-06-17T12:46:47.827822+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.628542+0000","last_change":"2022-06-17T12:46:45.660735+0000","last_active":"2022-06-17T12:46:52.628542+0000","last_peered":"2022-06-17T12:46:52.628542+0000","last_clean":"2022-06-17T12:46:52.628542+0000","last_became_active":"2022-06-17T12:46:45.660343+0000","last_became_peered":"2022-06-17T12:46:45.660343+0000","last_unstale":"2022-06-17T12:46:52.628542+0000","last_undegraded":"2022-06-17T12:46:52.628542+0000","last_fullsized":"2022-06-17T12:46:52.628542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048860,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322980,"statfs":{"total":96636764160,"available":95562731520,"internally_reserved":1073741824,"allocated":282624,"data_stored":115755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]}]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048852,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322988,"statfs":{"total":96636764160,"available":95562739712,"internally_reserved":1073741824,"allocated":274432,"data_stored":115180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048868,"kb_used_data":284,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322972,"statfs":{"total":96636764160,"available":95562723328,"internally_reserved":1073741824,"allocated":290816,"data_stored":115915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]}]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048864,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322976,"statfs":{"total":96636764160,"available":95562727424,"internally_reserved":1073741824,"allocated":286720,"data_stored":116715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2490000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.377}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048860,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322980,"statfs":{"total":96636764160,"available":95562731520,"internally_reserved":1073741824,"allocated":282624,"data_stored":115340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.119}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048856,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322984,"statfs":{"total":96636764160,"available":95562735616,"internally_reserved":1073741824,"allocated":278528,"data_stored":115214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048864,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322976,"statfs":{"total":96636764160,"available":95562727424,"internally_reserved":1073741824,"allocated":286720,"data_stored":116715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048856,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322984,"statfs":{"total":96636764160,"available":95562735616,"internally_reserved":1073741824,"allocated":278528,"data_stored":115214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]}]}]}} 2022-06-17T12:47:04.980 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-06-17T12:47:05.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.109+0000 7f0117fff700 1 Processor -- start 2022-06-17T12:47:05.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.113+0000 7f0117fff700 1 -- start start 2022-06-17T12:47:05.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.113+0000 7f0117fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0118060ac0 0x7f01180afe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:05.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.113+0000 7f0117fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f01180aee30 0x7f011813c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:05.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.113+0000 7f0117fff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0118060ea0 con 0x7f01180b20f0 2022-06-17T12:47:05.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.113+0000 7f0117fff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01180b3060 con 0x7f01180aee30 2022-06-17T12:47:05.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.113+0000 7f0117fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0118067a20 con 0x7f0118060ac0 2022-06-17T12:47:05.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.113+0000 7f010ffff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f01180b20f0 0x7f01180ae720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49164/0 (socket says 172.21.15.36:49164) 2022-06-17T12:47:05.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.113+0000 7f010ffff700 1 -- 172.21.15.36:0/759467012 learned_addr learned my addr 172.21.15.36:0/759467012 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:05.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.113+0000 7f01177fe700 1 --2- 172.21.15.36:0/759467012 >> v2:172.21.15.70:3300/0 conn(0x7f01180aee30 0x7f011813c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:05.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.113+0000 7f0116ffd700 1 --2- 172.21.15.36:0/759467012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0118060ac0 0x7f01180afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:05.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.113+0000 7f01167fc700 1 -- 172.21.15.36:0/759467012 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283348766 0 0) 0x7f0118060ea0 con 0x7f01180b20f0 2022-06-17T12:47:05.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.113+0000 7f01167fc700 1 -- 172.21.15.36:0/759467012 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00f8003260 con 0x7f01180b20f0 2022-06-17T12:47:05.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.113+0000 7f01177fe700 1 -- 172.21.15.36:0/759467012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0118060ac0 msgr2=0x7f01180afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:05.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.113+0000 7f01177fe700 1 --2- 172.21.15.36:0/759467012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0118060ac0 0x7f01180afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:05.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.113+0000 7f01177fe700 1 -- 172.21.15.36:0/759467012 >> v1:172.21.15.36:6789/0 conn(0x7f01180b20f0 legacy=0x7f01180ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:05.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.113+0000 7f01177fe700 1 -- 172.21.15.36:0/759467012 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0118143fb0 con 0x7f01180aee30 2022-06-17T12:47:05.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.113+0000 7f0116ffd700 1 --2- 172.21.15.36:0/759467012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0118060ac0 0x7f01180afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:05.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.113+0000 7f01177fe700 1 --2- 172.21.15.36:0/759467012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01180aee30 0x7f011813c7e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f01100032f0 tx=0x7f01100091d0).ready entity=mon.1 client_cookie=51036b86ab044213 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:05.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.113+0000 7f01167fc700 1 -- 172.21.15.36:0/759467012 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f011000b600 con 0x7f01180aee30 2022-06-17T12:47:05.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.113+0000 7f01167fc700 1 -- 172.21.15.36:0/759467012 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f011000b760 con 0x7f01180aee30 2022-06-17T12:47:05.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.113+0000 7f01167fc700 1 -- 172.21.15.36:0/759467012 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f011000b8d0 con 0x7f01180aee30 2022-06-17T12:47:05.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.117+0000 7f0117fff700 1 -- 172.21.15.36:0/759467012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01180aee30 msgr2=0x7f011813c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:05.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.117+0000 7f0117fff700 1 --2- 172.21.15.36:0/759467012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01180aee30 0x7f011813c7e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f01100032f0 tx=0x7f01100091d0).stop 2022-06-17T12:47:05.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.117+0000 7f0117fff700 1 -- 172.21.15.36:0/759467012 shutdown_connections 2022-06-17T12:47:05.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.117+0000 7f0117fff700 1 --2- 172.21.15.36:0/759467012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01180aee30 0x7f011813c7e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:05.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.117+0000 7f0117fff700 1 --2- 172.21.15.36:0/759467012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0118060ac0 0x7f01180afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:05.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.117+0000 7f0117fff700 1 -- 172.21.15.36:0/759467012 >> 172.21.15.36:0/759467012 conn(0x7f01180b65f0 msgr2=0x7f01180b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:05.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.117+0000 7f0117fff700 1 -- 172.21.15.36:0/759467012 shutdown_connections 2022-06-17T12:47:05.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.117+0000 7f0117fff700 1 -- 172.21.15.36:0/759467012 wait complete. 2022-06-17T12:47:05.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.117+0000 7f0117fff700 1 Processor -- start 2022-06-17T12:47:05.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.117+0000 7f0117fff700 1 -- start start 2022-06-17T12:47:05.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.117+0000 7f0117fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0118060ac0 0x7f0118142020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:05.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.117+0000 7f0117fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01180aee30 0x7f0118148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:05.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.121+0000 7f0117fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01180b20f0 0x7f0118146080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:05.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.121+0000 7f0117fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0118147a80 con 0x7f0118060ac0 2022-06-17T12:47:05.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.121+0000 7f0117fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0118156460 con 0x7f01180aee30 2022-06-17T12:47:05.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.121+0000 7f0117fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0118155f60 con 0x7f01180b20f0 2022-06-17T12:47:05.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.121+0000 7f0116ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0118060ac0 0x7f0118142020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:05.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.121+0000 7f0116ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0118060ac0 0x7f0118142020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59872/0 (socket says 172.21.15.36:59872) 2022-06-17T12:47:05.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.121+0000 7f0116ffd700 1 -- 172.21.15.36:0/1643098223 learned_addr learned my addr 172.21.15.36:0/1643098223 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:05.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.121+0000 7f01177fe700 1 --2- 172.21.15.36:0/1643098223 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01180b20f0 0x7f0118146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:05.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.121+0000 7f010ffff700 1 --2- 172.21.15.36:0/1643098223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01180aee30 0x7f0118148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:05.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.121+0000 7f0116ffd700 1 -- 172.21.15.36:0/1643098223 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01180b20f0 msgr2=0x7f0118146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:05.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.121+0000 7f0116ffd700 1 --2- 172.21.15.36:0/1643098223 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01180b20f0 0x7f0118146080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:05.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.121+0000 7f0116ffd700 1 -- 172.21.15.36:0/1643098223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01180aee30 msgr2=0x7f0118148ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:05.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.121+0000 7f0116ffd700 1 --2- 172.21.15.36:0/1643098223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01180aee30 0x7f0118148ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:05.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.121+0000 7f0116ffd700 1 -- 172.21.15.36:0/1643098223 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010800b180 con 0x7f0118060ac0 2022-06-17T12:47:05.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.121+0000 7f0116ffd700 1 --2- 172.21.15.36:0/1643098223 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0118060ac0 0x7f0118142020 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f010800c000 tx=0x7f0108006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:05.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.121+0000 7f0114ff9700 1 -- 172.21.15.36:0/1643098223 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f010800c7e0 con 0x7f0118060ac0 2022-06-17T12:47:05.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.121+0000 7f0114ff9700 1 -- 172.21.15.36:0/1643098223 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010800c940 con 0x7f0118060ac0 2022-06-17T12:47:05.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.121+0000 7f0114ff9700 1 -- 172.21.15.36:0/1643098223 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f010800cca0 con 0x7f0118060ac0 2022-06-17T12:47:05.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.121+0000 7f0117fff700 1 -- 172.21.15.36:0/1643098223 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0110009470 con 0x7f0118060ac0 2022-06-17T12:47:05.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.121+0000 7f0117fff700 1 -- 172.21.15.36:0/1643098223 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0118143a30 con 0x7f0118060ac0 2022-06-17T12:47:05.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.121+0000 7f0117fff700 1 -- 172.21.15.36:0/1643098223 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0118057410 con 0x7f0118060ac0 2022-06-17T12:47:05.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.121+0000 7f0114ff9700 1 -- 172.21.15.36:0/1643098223 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f010801c460 con 0x7f0118060ac0 2022-06-17T12:47:05.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.125+0000 7f0114ff9700 1 --2- 172.21.15.36:0/1643098223 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f00fc053a40 0x7f00fc055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:05.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.125+0000 7f0114ff9700 1 -- 172.21.15.36:0/1643098223 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f010806c6a0 con 0x7f0118060ac0 2022-06-17T12:47:05.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.125+0000 7f010ffff700 1 --2- 172.21.15.36:0/1643098223 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f00fc053a40 0x7f00fc055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:05.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.125+0000 7f010ffff700 1 --2- 172.21.15.36:0/1643098223 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f00fc053a40 0x7f00fc055f00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f010000d000 tx=0x7f0100009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:05.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.129+0000 7f0114ff9700 1 -- 172.21.15.36:0/1643098223 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0108043330 con 0x7f0118060ac0 2022-06-17T12:47:05.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.253+0000 7f0117fff700 1 -- 172.21.15.36:0/1643098223 --> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0118148840 con 0x7f00fc053a40 2022-06-17T12:47:05.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.261+0000 7f0114ff9700 1 -- 172.21.15.36:0/1643098223 <== mgr.4104 v2:172.21.15.36:6800/22419 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82373 (secure 0 0 0) 0x7f0118148840 con 0x7f00fc053a40 2022-06-17T12:47:05.249 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:05.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.261+0000 7f0117fff700 1 -- 172.21.15.36:0/1643098223 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f00fc053a40 msgr2=0x7f00fc055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:05.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.261+0000 7f0117fff700 1 --2- 172.21.15.36:0/1643098223 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f00fc053a40 0x7f00fc055f00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f010000d000 tx=0x7f0100009040).stop 2022-06-17T12:47:05.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.261+0000 7f0117fff700 1 -- 172.21.15.36:0/1643098223 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0118060ac0 msgr2=0x7f0118142020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:05.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.261+0000 7f0117fff700 1 --2- 172.21.15.36:0/1643098223 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0118060ac0 0x7f0118142020 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f010800c000 tx=0x7f0108006df0).stop 2022-06-17T12:47:05.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.261+0000 7f0117fff700 1 -- 172.21.15.36:0/1643098223 shutdown_connections 2022-06-17T12:47:05.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.261+0000 7f0117fff700 1 --2- 172.21.15.36:0/1643098223 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f00fc053a40 0x7f00fc055f00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:05.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.261+0000 7f0117fff700 1 --2- 172.21.15.36:0/1643098223 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01180b20f0 0x7f0118146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:05.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.261+0000 7f0117fff700 1 --2- 172.21.15.36:0/1643098223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01180aee30 0x7f0118148ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:05.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.261+0000 7f0117fff700 1 --2- 172.21.15.36:0/1643098223 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0118060ac0 0x7f0118142020 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:05.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.261+0000 7f0117fff700 1 -- 172.21.15.36:0/1643098223 >> 172.21.15.36:0/1643098223 conn(0x7f01180b65f0 msgr2=0x7f01180b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:05.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.261+0000 7f0117fff700 1 -- 172.21.15.36:0/1643098223 shutdown_connections 2022-06-17T12:47:05.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.261+0000 7f0117fff700 1 -- 172.21.15.36:0/1643098223 wait complete. 2022-06-17T12:47:05.297 INFO:teuthology.orchestra.run.smithi036.stderr:dumped all 2022-06-17T12:47:05.298 INFO:teuthology.orchestra.run.smithi036.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2022-06-17T12:47:03.496542+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390880,"kb_used_data":2208,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746583840,"statfs":{"total":773094113280,"available":764501852160,"internally_reserved":8589934592,"allocated":2260992,"data_stored":926048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002088"},"pg_stats":[{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.628424+0000","last_change":"2022-06-17T12:46:52.269670+0000","last_active":"2022-06-17T12:46:52.628424+0000","last_peered":"2022-06-17T12:46:52.628424+0000","last_clean":"2022-06-17T12:46:52.628424+0000","last_became_active":"2022-06-17T12:46:52.267094+0000","last_became_peered":"2022-06-17T12:46:52.267094+0000","last_unstale":"2022-06-17T12:46:52.628424+0000","last_undegraded":"2022-06-17T12:46:52.628424+0000","last_fullsized":"2022-06-17T12:46:52.628424+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_clean_scrub_stamp":"2022-06-17T12:46:50.849792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.277769+0000","last_change":"2022-06-17T12:46:41.622624+0000","last_active":"2022-06-17T12:46:52.277769+0000","last_peered":"2022-06-17T12:46:52.277769+0000","last_clean":"2022-06-17T12:46:52.277769+0000","last_became_active":"2022-06-17T12:46:41.622138+0000","last_became_peered":"2022-06-17T12:46:41.622138+0000","last_unstale":"2022-06-17T12:46:52.277769+0000","last_undegraded":"2022-06-17T12:46:52.277769+0000","last_fullsized":"2022-06-17T12:46:52.277769+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":4,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:34.095313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:34.095313+0000","last_clean_scrub_stamp":"2022-06-17T12:46:34.095313+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.271351+0000","last_change":"2022-06-17T12:46:45.658148+0000","last_active":"2022-06-17T12:46:52.271351+0000","last_peered":"2022-06-17T12:46:52.271351+0000","last_clean":"2022-06-17T12:46:52.271351+0000","last_became_active":"2022-06-17T12:46:45.657682+0000","last_became_peered":"2022-06-17T12:46:45.657682+0000","last_unstale":"2022-06-17T12:46:52.271351+0000","last_undegraded":"2022-06-17T12:46:52.271351+0000","last_fullsized":"2022-06-17T12:46:52.271351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"25'1","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.273316+0000","last_change":"2022-06-17T12:46:48.848511+0000","last_active":"2022-06-17T12:46:52.273316+0000","last_peered":"2022-06-17T12:46:52.273316+0000","last_clean":"2022-06-17T12:46:52.273316+0000","last_became_active":"2022-06-17T12:46:48.848184+0000","last_became_peered":"2022-06-17T12:46:48.848184+0000","last_unstale":"2022-06-17T12:46:52.273316+0000","last_undegraded":"2022-06-17T12:46:52.273316+0000","last_fullsized":"2022-06-17T12:46:52.273316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_clean_scrub_stamp":"2022-06-17T12:46:47.827822+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.274485+0000","last_change":"2022-06-17T12:46:52.263152+0000","last_active":"2022-06-17T12:46:52.274485+0000","last_peered":"2022-06-17T12:46:52.274485+0000","last_clean":"2022-06-17T12:46:52.274485+0000","last_became_active":"2022-06-17T12:46:52.262694+0000","last_became_peered":"2022-06-17T12:46:52.262694+0000","last_unstale":"2022-06-17T12:46:52.274485+0000","last_undegraded":"2022-06-17T12:46:52.274485+0000","last_fullsized":"2022-06-17T12:46:52.274485+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_clean_scrub_stamp":"2022-06-17T12:46:50.849792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.277662+0000","last_change":"2022-06-17T12:46:45.662621+0000","last_active":"2022-06-17T12:46:52.277662+0000","last_peered":"2022-06-17T12:46:52.277662+0000","last_clean":"2022-06-17T12:46:52.277662+0000","last_became_active":"2022-06-17T12:46:45.662293+0000","last_became_peered":"2022-06-17T12:46:45.662293+0000","last_unstale":"2022-06-17T12:46:52.277662+0000","last_undegraded":"2022-06-17T12:46:52.277662+0000","last_fullsized":"2022-06-17T12:46:52.277662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.274079+0000","last_change":"2022-06-17T12:46:48.851317+0000","last_active":"2022-06-17T12:46:52.274079+0000","last_peered":"2022-06-17T12:46:52.274079+0000","last_clean":"2022-06-17T12:46:52.274079+0000","last_became_active":"2022-06-17T12:46:48.850976+0000","last_became_peered":"2022-06-17T12:46:48.850976+0000","last_unstale":"2022-06-17T12:46:52.274079+0000","last_undegraded":"2022-06-17T12:46:52.274079+0000","last_fullsized":"2022-06-17T12:46:52.274079+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_clean_scrub_stamp":"2022-06-17T12:46:47.827822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.272725+0000","last_change":"2022-06-17T12:46:52.264340+0000","last_active":"2022-06-17T12:46:52.272725+0000","last_peered":"2022-06-17T12:46:52.272725+0000","last_clean":"2022-06-17T12:46:52.272725+0000","last_became_active":"2022-06-17T12:46:52.262659+0000","last_became_peered":"2022-06-17T12:46:52.262659+0000","last_unstale":"2022-06-17T12:46:52.272725+0000","last_undegraded":"2022-06-17T12:46:52.272725+0000","last_fullsized":"2022-06-17T12:46:52.272725+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_clean_scrub_stamp":"2022-06-17T12:46:50.849792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.278375+0000","last_change":"2022-06-17T12:46:46.357931+0000","last_active":"2022-06-17T12:46:52.278375+0000","last_peered":"2022-06-17T12:46:52.278375+0000","last_clean":"2022-06-17T12:46:52.278375+0000","last_became_active":"2022-06-17T12:46:46.356789+0000","last_became_peered":"2022-06-17T12:46:46.356789+0000","last_unstale":"2022-06-17T12:46:52.278375+0000","last_undegraded":"2022-06-17T12:46:52.278375+0000","last_fullsized":"2022-06-17T12:46:52.278375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"25'8","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.280532+0000","last_change":"2022-06-17T12:46:48.853511+0000","last_active":"2022-06-17T12:46:52.280532+0000","last_peered":"2022-06-17T12:46:52.280532+0000","last_clean":"2022-06-17T12:46:52.280532+0000","last_became_active":"2022-06-17T12:46:48.853120+0000","last_became_peered":"2022-06-17T12:46:48.853120+0000","last_unstale":"2022-06-17T12:46:52.280532+0000","last_undegraded":"2022-06-17T12:46:52.280532+0000","last_fullsized":"2022-06-17T12:46:52.280532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_clean_scrub_stamp":"2022-06-17T12:46:47.827822+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.274277+0000","last_change":"2022-06-17T12:46:52.259111+0000","last_active":"2022-06-17T12:46:52.274277+0000","last_peered":"2022-06-17T12:46:52.274277+0000","last_clean":"2022-06-17T12:46:52.274277+0000","last_became_active":"2022-06-17T12:46:52.257942+0000","last_became_peered":"2022-06-17T12:46:52.257942+0000","last_unstale":"2022-06-17T12:46:52.274277+0000","last_undegraded":"2022-06-17T12:46:52.274277+0000","last_fullsized":"2022-06-17T12:46:52.274277+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_clean_scrub_stamp":"2022-06-17T12:46:50.849792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.271354+0000","last_change":"2022-06-17T12:46:45.662903+0000","last_active":"2022-06-17T12:46:52.271354+0000","last_peered":"2022-06-17T12:46:52.271354+0000","last_clean":"2022-06-17T12:46:52.271354+0000","last_became_active":"2022-06-17T12:46:45.662460+0000","last_became_peered":"2022-06-17T12:46:45.662460+0000","last_unstale":"2022-06-17T12:46:52.271354+0000","last_undegraded":"2022-06-17T12:46:52.271354+0000","last_fullsized":"2022-06-17T12:46:52.271354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"25'5","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.284343+0000","last_change":"2022-06-17T12:46:48.856578+0000","last_active":"2022-06-17T12:46:52.284343+0000","last_peered":"2022-06-17T12:46:52.284343+0000","last_clean":"2022-06-17T12:46:52.284343+0000","last_became_active":"2022-06-17T12:46:48.856155+0000","last_became_peered":"2022-06-17T12:46:48.856155+0000","last_unstale":"2022-06-17T12:46:52.284343+0000","last_undegraded":"2022-06-17T12:46:52.284343+0000","last_fullsized":"2022-06-17T12:46:52.284343+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_clean_scrub_stamp":"2022-06-17T12:46:47.827822+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.272515+0000","last_change":"2022-06-17T12:46:52.263338+0000","last_active":"2022-06-17T12:46:52.272515+0000","last_peered":"2022-06-17T12:46:52.272515+0000","last_clean":"2022-06-17T12:46:52.272515+0000","last_became_active":"2022-06-17T12:46:52.262706+0000","last_became_peered":"2022-06-17T12:46:52.262706+0000","last_unstale":"2022-06-17T12:46:52.272515+0000","last_undegraded":"2022-06-17T12:46:52.272515+0000","last_fullsized":"2022-06-17T12:46:52.272515+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_clean_scrub_stamp":"2022-06-17T12:46:50.849792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"25'6","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.282758+0000","last_change":"2022-06-17T12:46:48.855213+0000","last_active":"2022-06-17T12:46:52.282758+0000","last_peered":"2022-06-17T12:46:52.282758+0000","last_clean":"2022-06-17T12:46:52.282758+0000","last_became_active":"2022-06-17T12:46:48.854686+0000","last_became_peered":"2022-06-17T12:46:48.854686+0000","last_unstale":"2022-06-17T12:46:52.282758+0000","last_undegraded":"2022-06-17T12:46:52.282758+0000","last_fullsized":"2022-06-17T12:46:52.282758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_clean_scrub_stamp":"2022-06-17T12:46:47.827822+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.274561+0000","last_change":"2022-06-17T12:46:46.261176+0000","last_active":"2022-06-17T12:46:52.274561+0000","last_peered":"2022-06-17T12:46:52.274561+0000","last_clean":"2022-06-17T12:46:52.274561+0000","last_became_active":"2022-06-17T12:46:46.260143+0000","last_became_peered":"2022-06-17T12:46:46.260143+0000","last_unstale":"2022-06-17T12:46:52.274561+0000","last_undegraded":"2022-06-17T12:46:52.274561+0000","last_fullsized":"2022-06-17T12:46:52.274561+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.274554+0000","last_change":"2022-06-17T12:46:52.263813+0000","last_active":"2022-06-17T12:46:52.274554+0000","last_peered":"2022-06-17T12:46:52.274554+0000","last_clean":"2022-06-17T12:46:52.274554+0000","last_became_active":"2022-06-17T12:46:52.262614+0000","last_became_peered":"2022-06-17T12:46:52.262614+0000","last_unstale":"2022-06-17T12:46:52.274554+0000","last_undegraded":"2022-06-17T12:46:52.274554+0000","last_fullsized":"2022-06-17T12:46:52.274554+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_clean_scrub_stamp":"2022-06-17T12:46:50.849792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"25'8","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.288159+0000","last_change":"2022-06-17T12:46:48.852628+0000","last_active":"2022-06-17T12:46:52.288159+0000","last_peered":"2022-06-17T12:46:52.288159+0000","last_clean":"2022-06-17T12:46:52.288159+0000","last_became_active":"2022-06-17T12:46:48.852216+0000","last_became_peered":"2022-06-17T12:46:48.852216+0000","last_unstale":"2022-06-17T12:46:52.288159+0000","last_undegraded":"2022-06-17T12:46:52.288159+0000","last_fullsized":"2022-06-17T12:46:52.288159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_clean_scrub_stamp":"2022-06-17T12:46:47.827822+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.277410+0000","last_change":"2022-06-17T12:46:46.265378+0000","last_active":"2022-06-17T12:46:52.277410+0000","last_peered":"2022-06-17T12:46:52.277410+0000","last_clean":"2022-06-17T12:46:52.277410+0000","last_became_active":"2022-06-17T12:46:46.264499+0000","last_became_peered":"2022-06-17T12:46:46.264499+0000","last_unstale":"2022-06-17T12:46:52.277410+0000","last_undegraded":"2022-06-17T12:46:52.277410+0000","last_fullsized":"2022-06-17T12:46:52.277410+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.266946+0000","last_change":"2022-06-17T12:46:52.265747+0000","last_active":"2022-06-17T12:46:52.266946+0000","last_peered":"2022-06-17T12:46:52.266946+0000","last_clean":"2022-06-17T12:46:52.266946+0000","last_became_active":"2022-06-17T12:46:52.265143+0000","last_became_peered":"2022-06-17T12:46:52.265143+0000","last_unstale":"2022-06-17T12:46:52.266946+0000","last_undegraded":"2022-06-17T12:46:52.266946+0000","last_fullsized":"2022-06-17T12:46:52.266946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_clean_scrub_stamp":"2022-06-17T12:46:50.849792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"25'7","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.278014+0000","last_change":"2022-06-17T12:46:49.297977+0000","last_active":"2022-06-17T12:46:52.278014+0000","last_peered":"2022-06-17T12:46:52.278014+0000","last_clean":"2022-06-17T12:46:52.278014+0000","last_became_active":"2022-06-17T12:46:49.297174+0000","last_became_peered":"2022-06-17T12:46:49.297174+0000","last_unstale":"2022-06-17T12:46:52.278014+0000","last_undegraded":"2022-06-17T12:46:52.278014+0000","last_fullsized":"2022-06-17T12:46:52.278014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_clean_scrub_stamp":"2022-06-17T12:46:47.827822+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.272385+0000","last_change":"2022-06-17T12:46:45.658907+0000","last_active":"2022-06-17T12:46:52.272385+0000","last_peered":"2022-06-17T12:46:52.272385+0000","last_clean":"2022-06-17T12:46:52.272385+0000","last_became_active":"2022-06-17T12:46:45.658569+0000","last_became_peered":"2022-06-17T12:46:45.658569+0000","last_unstale":"2022-06-17T12:46:52.272385+0000","last_undegraded":"2022-06-17T12:46:52.272385+0000","last_fullsized":"2022-06-17T12:46:52.272385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.272444+0000","last_change":"2022-06-17T12:46:52.271397+0000","last_active":"2022-06-17T12:46:52.272444+0000","last_peered":"2022-06-17T12:46:52.272444+0000","last_clean":"2022-06-17T12:46:52.272444+0000","last_became_active":"2022-06-17T12:46:52.267136+0000","last_became_peered":"2022-06-17T12:46:52.267136+0000","last_unstale":"2022-06-17T12:46:52.272444+0000","last_undegraded":"2022-06-17T12:46:52.272444+0000","last_fullsized":"2022-06-17T12:46:52.272444+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:50.849792+0000","last_clean_scrub_stamp":"2022-06-17T12:46:50.849792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"26'4","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2022-06-17T12:46:54.496698+0000","last_change":"2022-06-17T12:46:48.853616+0000","last_active":"2022-06-17T12:46:54.496698+0000","last_peered":"2022-06-17T12:46:54.496698+0000","last_clean":"2022-06-17T12:46:54.496698+0000","last_became_active":"2022-06-17T12:46:48.853263+0000","last_became_peered":"2022-06-17T12:46:48.853263+0000","last_unstale":"2022-06-17T12:46:54.496698+0000","last_undegraded":"2022-06-17T12:46:54.496698+0000","last_fullsized":"2022-06-17T12:46:54.496698+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:47.827822+0000","last_clean_scrub_stamp":"2022-06-17T12:46:47.827822+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-17T12:46:52.628542+0000","last_change":"2022-06-17T12:46:45.660735+0000","last_active":"2022-06-17T12:46:52.628542+0000","last_peered":"2022-06-17T12:46:52.628542+0000","last_clean":"2022-06-17T12:46:52.628542+0000","last_became_active":"2022-06-17T12:46:45.660343+0000","last_became_peered":"2022-06-17T12:46:45.660343+0000","last_unstale":"2022-06-17T12:46:52.628542+0000","last_undegraded":"2022-06-17T12:46:52.628542+0000","last_fullsized":"2022-06-17T12:46:52.628542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048860,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322980,"statfs":{"total":96636764160,"available":95562731520,"internally_reserved":1073741824,"allocated":282624,"data_stored":115755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]}]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048852,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322988,"statfs":{"total":96636764160,"available":95562739712,"internally_reserved":1073741824,"allocated":274432,"data_stored":115180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048868,"kb_used_data":284,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322972,"statfs":{"total":96636764160,"available":95562723328,"internally_reserved":1073741824,"allocated":290816,"data_stored":115915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]}]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048864,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322976,"statfs":{"total":96636764160,"available":95562727424,"internally_reserved":1073741824,"allocated":286720,"data_stored":116715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2490000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.377}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048860,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322980,"statfs":{"total":96636764160,"available":95562731520,"internally_reserved":1073741824,"allocated":282624,"data_stored":115340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.119}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048856,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322984,"statfs":{"total":96636764160,"available":95562735616,"internally_reserved":1073741824,"allocated":278528,"data_stored":115214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048864,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322976,"statfs":{"total":96636764160,"available":95562727424,"internally_reserved":1073741824,"allocated":286720,"data_stored":116715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048856,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322984,"statfs":{"total":96636764160,"available":95562735616,"internally_reserved":1073741824,"allocated":278528,"data_stored":115214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]}]}]}} 2022-06-17T12:47:05.333 INFO:tasks.ceph.ceph_manager.ceph:clean! 2022-06-17T12:47:05.334 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2022-06-17T12:47:05.334 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2022-06-17T12:47:05.384 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2022-06-17T12:47:05.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.509+0000 7f61a14d7700 1 Processor -- start 2022-06-17T12:47:05.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.509+0000 7f61a14d7700 1 -- start start 2022-06-17T12:47:05.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.509+0000 7f61a14d7700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f619c1451c0 0x7f619c0b7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:05.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.509+0000 7f61a14d7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f619c0ae150 0x7f619c140e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:05.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.509+0000 7f61a14d7700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f619c066110 con 0x7f619c0b7880 2022-06-17T12:47:05.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.509+0000 7f61a14d7700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619c066390 con 0x7f619c1451c0 2022-06-17T12:47:05.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.509+0000 7f61a14d7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619c144610 con 0x7f619c0ae150 2022-06-17T12:47:05.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.513+0000 7f6193fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f619c1451c0 0x7f619c0b7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:05.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.513+0000 7f6193fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f619c1451c0 0x7f619c0b7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59880/0 (socket says 172.21.15.36:59880) 2022-06-17T12:47:05.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.513+0000 7f6193fff700 1 -- 172.21.15.36:0/150884517 learned_addr learned my addr 172.21.15.36:0/150884517 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:05.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.513+0000 7f619b7fe700 1 --2- 172.21.15.36:0/150884517 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f619c0ae150 0x7f619c140e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:05.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.513+0000 7f6193fff700 1 -- 172.21.15.36:0/150884517 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f619c0ae150 msgr2=0x7f619c140e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:05.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.513+0000 7f6193fff700 1 --2- 172.21.15.36:0/150884517 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f619c0ae150 0x7f619c140e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:05.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.513+0000 7f6193fff700 1 -- 172.21.15.36:0/150884517 >> v1:172.21.15.70:6789/0 conn(0x7f619c0b7880 legacy=0x7f619c144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:05.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.513+0000 7f6193fff700 1 -- 172.21.15.36:0/150884517 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619c141380 con 0x7f619c1451c0 2022-06-17T12:47:05.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.513+0000 7f6193fff700 1 --2- 172.21.15.36:0/150884517 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f619c1451c0 0x7f619c0b7c60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f619c064de0 tx=0x7f6184006eb0).ready entity=mon.0 client_cookie=720688b24dda224 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:05.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.513+0000 7f619a7fc700 1 -- 172.21.15.36:0/150884517 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f618400d5c0 con 0x7f619c1451c0 2022-06-17T12:47:05.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.513+0000 7f619a7fc700 1 -- 172.21.15.36:0/150884517 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6184000e90 con 0x7f619c1451c0 2022-06-17T12:47:05.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.513+0000 7f619a7fc700 1 -- 172.21.15.36:0/150884517 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f618400d950 con 0x7f619c1451c0 2022-06-17T12:47:05.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.513+0000 7f61a14d7700 1 -- 172.21.15.36:0/150884517 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f619c1451c0 msgr2=0x7f619c0b7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:05.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.513+0000 7f61a14d7700 1 --2- 172.21.15.36:0/150884517 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f619c1451c0 0x7f619c0b7c60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f619c064de0 tx=0x7f6184006eb0).stop 2022-06-17T12:47:05.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.513+0000 7f61a14d7700 1 -- 172.21.15.36:0/150884517 shutdown_connections 2022-06-17T12:47:05.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.513+0000 7f61a14d7700 1 --2- 172.21.15.36:0/150884517 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f619c0ae150 0x7f619c140e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:05.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.513+0000 7f61a14d7700 1 --2- 172.21.15.36:0/150884517 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f619c1451c0 0x7f619c0b7c60 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:05.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.513+0000 7f61a14d7700 1 -- 172.21.15.36:0/150884517 >> 172.21.15.36:0/150884517 conn(0x7f619c060990 msgr2=0x7f619c05c550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:05.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.513+0000 7f61a14d7700 1 -- 172.21.15.36:0/150884517 shutdown_connections 2022-06-17T12:47:05.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.513+0000 7f61a14d7700 1 -- 172.21.15.36:0/150884517 wait complete. 2022-06-17T12:47:05.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.513+0000 7f61a14d7700 1 Processor -- start 2022-06-17T12:47:05.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.513+0000 7f61a14d7700 1 -- start start 2022-06-17T12:47:05.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.517+0000 7f61a14d7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f619c0ae150 0x7f619c0af100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:05.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.517+0000 7f61a14d7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f619c0b7880 0x7f619c0b0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:05.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.517+0000 7f61a14d7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f619c1451c0 0x7f619c0b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:05.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.517+0000 7f61a14d7700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c0b6b30 con 0x7f619c0ae150 2022-06-17T12:47:05.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.517+0000 7f61a14d7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c152af0 con 0x7f619c0b7880 2022-06-17T12:47:05.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.517+0000 7f61a14d7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619c152d90 con 0x7f619c1451c0 2022-06-17T12:47:05.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.517+0000 7f619affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f619c0ae150 0x7f619c0af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:05.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.517+0000 7f6193fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f619c0b7880 0x7f619c0b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:05.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.517+0000 7f619b7fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f619c1451c0 0x7f619c0b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:05.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.517+0000 7f6193fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f619c0b7880 0x7f619c0b0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:52036/0 (socket says 172.21.15.36:52036) 2022-06-17T12:47:05.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.517+0000 7f6193fff700 1 -- 172.21.15.36:0/3373302379 learned_addr learned my addr 172.21.15.36:0/3373302379 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:05.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.517+0000 7f6193fff700 1 -- 172.21.15.36:0/3373302379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f619c1451c0 msgr2=0x7f619c0b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:05.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.517+0000 7f6193fff700 1 --2- 172.21.15.36:0/3373302379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f619c1451c0 0x7f619c0b0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:05.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.517+0000 7f6193fff700 1 -- 172.21.15.36:0/3373302379 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f619c0ae150 msgr2=0x7f619c0af100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:05.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.517+0000 7f6193fff700 1 --2- 172.21.15.36:0/3373302379 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f619c0ae150 0x7f619c0af100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:05.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.517+0000 7f6193fff700 1 -- 172.21.15.36:0/3373302379 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618400b1e0 con 0x7f619c0b7880 2022-06-17T12:47:05.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.517+0000 7f619b7fe700 1 --2- 172.21.15.36:0/3373302379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f619c1451c0 0x7f619c0b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:05.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.517+0000 7f6193fff700 1 --2- 172.21.15.36:0/3373302379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f619c0b7880 0x7f619c0b0270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f619c064de0 tx=0x7f6184015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:05.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.517+0000 7f6198ff9700 1 -- 172.21.15.36:0/3373302379 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6184021070 con 0x7f619c0b7880 2022-06-17T12:47:05.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.517+0000 7f6198ff9700 1 -- 172.21.15.36:0/3373302379 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6184006b40 con 0x7f619c0b7880 2022-06-17T12:47:05.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.517+0000 7f61a14d7700 1 -- 172.21.15.36:0/3373302379 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f619c153030 con 0x7f619c0b7880 2022-06-17T12:47:05.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.517+0000 7f6198ff9700 1 -- 172.21.15.36:0/3373302379 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6184011d70 con 0x7f619c0b7880 2022-06-17T12:47:05.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.517+0000 7f61a14d7700 1 -- 172.21.15.36:0/3373302379 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619c0b6fa0 con 0x7f619c0b7880 2022-06-17T12:47:05.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.517+0000 7f61a14d7700 1 -- 172.21.15.36:0/3373302379 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f619c1531f0 con 0x7f619c0b7880 2022-06-17T12:47:05.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.521+0000 7f6198ff9700 1 -- 172.21.15.36:0/3373302379 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f618400f070 con 0x7f619c0b7880 2022-06-17T12:47:05.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.521+0000 7f6198ff9700 1 --2- 172.21.15.36:0/3373302379 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6180053a60 0x7f6180055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:05.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.521+0000 7f6198ff9700 1 -- 172.21.15.36:0/3373302379 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f618406d190 con 0x7f619c0b7880 2022-06-17T12:47:05.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.521+0000 7f619affd700 1 --2- 172.21.15.36:0/3373302379 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6180053a60 0x7f6180055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:05.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.521+0000 7f619affd700 1 --2- 172.21.15.36:0/3373302379 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6180053a60 0x7f6180055f20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f618c00d000 tx=0x7f618c000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:05.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.529+0000 7f6198ff9700 1 -- 172.21.15.36:0/3373302379 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6184043ef0 con 0x7f619c0b7880 2022-06-17T12:47:05.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.693+0000 7f61a14d7700 1 -- 172.21.15.36:0/3373302379 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f619c057410 con 0x7f619c0b7880 2022-06-17T12:47:05.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.697+0000 7f6198ff9700 1 -- 172.21.15.36:0/3373302379 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7f6184047510 con 0x7f619c0b7880 2022-06-17T12:47:05.684 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:05.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.697+0000 7f61a14d7700 1 -- 172.21.15.36:0/3373302379 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6180053a60 msgr2=0x7f6180055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:05.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.697+0000 7f61a14d7700 1 --2- 172.21.15.36:0/3373302379 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6180053a60 0x7f6180055f20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f618c00d000 tx=0x7f618c000bc0).stop 2022-06-17T12:47:05.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.697+0000 7f61a14d7700 1 -- 172.21.15.36:0/3373302379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f619c0b7880 msgr2=0x7f619c0b0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:05.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.697+0000 7f61a14d7700 1 --2- 172.21.15.36:0/3373302379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f619c0b7880 0x7f619c0b0270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f619c064de0 tx=0x7f6184015000).stop 2022-06-17T12:47:05.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.697+0000 7f61a14d7700 1 -- 172.21.15.36:0/3373302379 shutdown_connections 2022-06-17T12:47:05.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.697+0000 7f61a14d7700 1 --2- 172.21.15.36:0/3373302379 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6180053a60 0x7f6180055f20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:05.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.697+0000 7f61a14d7700 1 --2- 172.21.15.36:0/3373302379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f619c1451c0 0x7f619c0b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:05.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.697+0000 7f61a14d7700 1 --2- 172.21.15.36:0/3373302379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f619c0b7880 0x7f619c0b0270 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:05.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.697+0000 7f61a14d7700 1 --2- 172.21.15.36:0/3373302379 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f619c0ae150 0x7f619c0af100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:05.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.697+0000 7f61a14d7700 1 -- 172.21.15.36:0/3373302379 >> 172.21.15.36:0/3373302379 conn(0x7f619c060990 msgr2=0x7f619c0b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:05.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.697+0000 7f61a14d7700 1 -- 172.21.15.36:0/3373302379 shutdown_connections 2022-06-17T12:47:05.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.697+0000 7f61a14d7700 1 -- 172.21.15.36:0/3373302379 wait complete. 2022-06-17T12:47:05.723 INFO:teuthology.orchestra.run.smithi036.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2022-06-17T12:47:05.724 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2022-06-17T12:47:05.725 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:47:05.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.845+0000 7f96df59e700 1 Processor -- start 2022-06-17T12:47:05.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.845+0000 7f96df59e700 1 -- start start 2022-06-17T12:47:05.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.845+0000 7f96df59e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f96e00b4080 0x7f96e00673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:05.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.845+0000 7f96df59e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96e0065830 con 0x7f96e00ba130 2022-06-17T12:47:05.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.845+0000 7f96df59e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96e00ba510 con 0x7f96e00b56a0 2022-06-17T12:47:05.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.845+0000 7f96df59e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e00678c0 con 0x7f96e00b4080 2022-06-17T12:47:05.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.845+0000 7f96de59c700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f96e00ba130 0x7f96e0066cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49192/0 (socket says 172.21.15.36:49192) 2022-06-17T12:47:05.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.845+0000 7f96de59c700 1 -- 172.21.15.36:0/789665892 learned_addr learned my addr 172.21.15.36:0/789665892 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:05.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.845+0000 7f96ded9d700 1 --2- 172.21.15.36:0/789665892 >> v2:172.21.15.36:3300/0 conn(0x7f96e00b4080 0x7f96e00673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:05.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.845+0000 7f96dd59a700 1 -- 172.21.15.36:0/789665892 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1291193707 0 0) 0x7f96e0065830 con 0x7f96e00ba130 2022-06-17T12:47:05.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.845+0000 7f96dd59a700 1 -- 172.21.15.36:0/789665892 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96c4003260 con 0x7f96e00ba130 2022-06-17T12:47:05.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.845+0000 7f96dd59a700 1 -- 172.21.15.36:0/789665892 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1274984331 0 0) 0x7f96e00ba510 con 0x7f96e00b56a0 2022-06-17T12:47:05.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.845+0000 7f96dd59a700 1 -- 172.21.15.36:0/789665892 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96e0065830 con 0x7f96e00b56a0 2022-06-17T12:47:05.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.845+0000 7f96ded9d700 1 -- 172.21.15.36:0/789665892 >> v1:172.21.15.70:6789/0 conn(0x7f96e00b56a0 legacy=0x7f96e00b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:05.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.845+0000 7f96ded9d700 1 -- 172.21.15.36:0/789665892 >> v1:172.21.15.36:6789/0 conn(0x7f96e00ba130 legacy=0x7f96e0066cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:05.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.845+0000 7f96ded9d700 1 -- 172.21.15.36:0/789665892 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e0065590 con 0x7f96e00b4080 2022-06-17T12:47:05.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.845+0000 7f96ded9d700 1 --2- 172.21.15.36:0/789665892 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96e00b4080 0x7f96e00673d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f96d8002f70 tx=0x7f96d8006df0).ready entity=mon.0 client_cookie=242765fb1c9393f0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:05.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.845+0000 7f96dd59a700 1 -- 172.21.15.36:0/789665892 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96d800d540 con 0x7f96e00b4080 2022-06-17T12:47:05.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.845+0000 7f96dd59a700 1 -- 172.21.15.36:0/789665892 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d8000e90 con 0x7f96e00b4080 2022-06-17T12:47:05.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.845+0000 7f96dd59a700 1 -- 172.21.15.36:0/789665892 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96d800d890 con 0x7f96e00b4080 2022-06-17T12:47:05.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.849+0000 7f96df59e700 1 -- 172.21.15.36:0/789665892 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96e00b4080 msgr2=0x7f96e00673d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:05.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.849+0000 7f96df59e700 1 --2- 172.21.15.36:0/789665892 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96e00b4080 0x7f96e00673d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f96d8002f70 tx=0x7f96d8006df0).stop 2022-06-17T12:47:05.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.849+0000 7f96df59e700 1 -- 172.21.15.36:0/789665892 shutdown_connections 2022-06-17T12:47:05.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.849+0000 7f96df59e700 1 --2- 172.21.15.36:0/789665892 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96e00b4080 0x7f96e00673d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:05.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.849+0000 7f96df59e700 1 -- 172.21.15.36:0/789665892 >> 172.21.15.36:0/789665892 conn(0x7f96e005c550 msgr2=0x7f96e00b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:05.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.849+0000 7f96df59e700 1 -- 172.21.15.36:0/789665892 shutdown_connections 2022-06-17T12:47:05.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.849+0000 7f96df59e700 1 -- 172.21.15.36:0/789665892 wait complete. 2022-06-17T12:47:05.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.849+0000 7f96df59e700 1 Processor -- start 2022-06-17T12:47:05.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.849+0000 7f96df59e700 1 -- start start 2022-06-17T12:47:05.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.853+0000 7f96df59e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96e00b4080 0x7f96e00b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:05.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.853+0000 7f96df59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96e00b56a0 0x7f96e0146c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:05.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.853+0000 7f96df59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96e00ba130 0x7f96e01470f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:05.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.853+0000 7f96df59e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e00b2150 con 0x7f96e00ba130 2022-06-17T12:47:05.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.853+0000 7f96df59e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e00af7c0 con 0x7f96e00b4080 2022-06-17T12:47:05.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.853+0000 7f96df59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e00b0ab0 con 0x7f96e00b56a0 2022-06-17T12:47:05.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.853+0000 7f96ded9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96e00ba130 0x7f96e01470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:05.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.853+0000 7f96ded9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96e00ba130 0x7f96e01470f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59904/0 (socket says 172.21.15.36:59904) 2022-06-17T12:47:05.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.853+0000 7f96ded9d700 1 -- 172.21.15.36:0/1993335402 learned_addr learned my addr 172.21.15.36:0/1993335402 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:05.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.853+0000 7f96ddd9b700 1 --2- 172.21.15.36:0/1993335402 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96e00b56a0 0x7f96e0146c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:05.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.853+0000 7f96de59c700 1 --2- 172.21.15.36:0/1993335402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96e00b4080 0x7f96e00b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:05.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.853+0000 7f96ded9d700 1 -- 172.21.15.36:0/1993335402 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96e00b56a0 msgr2=0x7f96e0146c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:05.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.853+0000 7f96ded9d700 1 --2- 172.21.15.36:0/1993335402 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96e00b56a0 0x7f96e0146c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:05.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.853+0000 7f96ded9d700 1 -- 172.21.15.36:0/1993335402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96e00b4080 msgr2=0x7f96e00b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:05.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.853+0000 7f96ded9d700 1 --2- 172.21.15.36:0/1993335402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96e00b4080 0x7f96e00b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:05.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.853+0000 7f96ded9d700 1 -- 172.21.15.36:0/1993335402 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d800b180 con 0x7f96e00ba130 2022-06-17T12:47:05.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.853+0000 7f96ddd9b700 1 --2- 172.21.15.36:0/1993335402 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96e00b56a0 0x7f96e0146c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:05.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.853+0000 7f96ded9d700 1 --2- 172.21.15.36:0/1993335402 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96e00ba130 0x7f96e01470f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f96e011b110 tx=0x7f96d8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:05.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.853+0000 7f96cf7fe700 1 -- 172.21.15.36:0/1993335402 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96d800d1c0 con 0x7f96e00ba130 2022-06-17T12:47:05.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.853+0000 7f96cf7fe700 1 -- 172.21.15.36:0/1993335402 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d8006780 con 0x7f96e00ba130 2022-06-17T12:47:05.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.853+0000 7f96df59e700 1 -- 172.21.15.36:0/1993335402 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e00b0810 con 0x7f96e00ba130 2022-06-17T12:47:05.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.853+0000 7f96cf7fe700 1 -- 172.21.15.36:0/1993335402 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96d8013e70 con 0x7f96e00ba130 2022-06-17T12:47:05.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.853+0000 7f96df59e700 1 -- 172.21.15.36:0/1993335402 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e00674b0 con 0x7f96e00ba130 2022-06-17T12:47:05.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.857+0000 7f96df59e700 1 -- 172.21.15.36:0/1993335402 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e011b150 con 0x7f96e00ba130 2022-06-17T12:47:05.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.857+0000 7f96cf7fe700 1 -- 172.21.15.36:0/1993335402 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f96d8025030 con 0x7f96e00ba130 2022-06-17T12:47:05.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.857+0000 7f96cf7fe700 1 --2- 172.21.15.36:0/1993335402 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f96bc053ad0 0x7f96bc055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:05.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.857+0000 7f96de59c700 1 --2- 172.21.15.36:0/1993335402 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f96bc053ad0 0x7f96bc055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:05.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.857+0000 7f96cf7fe700 1 -- 172.21.15.36:0/1993335402 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f96d806cd30 con 0x7f96e00ba130 2022-06-17T12:47:05.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.857+0000 7f96de59c700 1 --2- 172.21.15.36:0/1993335402 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f96bc053ad0 0x7f96bc055f90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f96d000d000 tx=0x7f96d0009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:05.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.861+0000 7f96cf7fe700 1 -- 172.21.15.36:0/1993335402 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f96d8043a60 con 0x7f96e00ba130 2022-06-17T12:47:05.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.981+0000 7f96df59e700 1 -- 172.21.15.36:0/1993335402 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96e00b49c0 con 0x7f96e00ba130 2022-06-17T12:47:05.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.981+0000 7f96cf7fe700 1 -- 172.21.15.36:0/1993335402 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f96d8023070 con 0x7f96e00ba130 2022-06-17T12:47:05.973 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:05.974 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":26,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:46:53.258075+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:46:47.465328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-17T12:46:49.893338+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:47:05.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.985+0000 7f96df59e700 1 -- 172.21.15.36:0/1993335402 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f96bc053ad0 msgr2=0x7f96bc055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:05.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.985+0000 7f96df59e700 1 --2- 172.21.15.36:0/1993335402 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f96bc053ad0 0x7f96bc055f90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f96d000d000 tx=0x7f96d0009040).stop 2022-06-17T12:47:05.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.985+0000 7f96df59e700 1 -- 172.21.15.36:0/1993335402 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96e00ba130 msgr2=0x7f96e01470f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:05.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.985+0000 7f96df59e700 1 --2- 172.21.15.36:0/1993335402 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96e00ba130 0x7f96e01470f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f96e011b110 tx=0x7f96d8014000).stop 2022-06-17T12:47:05.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.985+0000 7f96df59e700 1 -- 172.21.15.36:0/1993335402 shutdown_connections 2022-06-17T12:47:06.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.985+0000 7f96df59e700 1 --2- 172.21.15.36:0/1993335402 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f96bc053ad0 0x7f96bc055f90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:06.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.985+0000 7f96df59e700 1 --2- 172.21.15.36:0/1993335402 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96e00ba130 0x7f96e01470f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:06.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.985+0000 7f96df59e700 1 --2- 172.21.15.36:0/1993335402 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96e00b56a0 0x7f96e0146c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:06.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.985+0000 7f96df59e700 1 --2- 172.21.15.36:0/1993335402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96e00b4080 0x7f96e00b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:06.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.985+0000 7f96df59e700 1 -- 172.21.15.36:0/1993335402 >> 172.21.15.36:0/1993335402 conn(0x7f96e005c550 msgr2=0x7f96e014be60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:06.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.985+0000 7f96df59e700 1 -- 172.21.15.36:0/1993335402 shutdown_connections 2022-06-17T12:47:06.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:05.985+0000 7f96df59e700 1 -- 172.21.15.36:0/1993335402 wait complete. 2022-06-17T12:47:06.025 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:46:47.465328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:46:49.893338+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:47:06.039 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:47:06.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.205+0000 7fee69bf9700 1 Processor -- start 2022-06-17T12:47:06.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.205+0000 7fee69bf9700 1 -- start start 2022-06-17T12:47:06.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.209+0000 7fee69bf9700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fee64140c00 0x7fee640b3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:06.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.209+0000 7fee69bf9700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee640b2120 con 0x7fee64060020 2022-06-17T12:47:06.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.209+0000 7fee69bf9700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee640b23c0 con 0x7fee640b3e20 2022-06-17T12:47:06.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.209+0000 7fee69bf9700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee64141040 con 0x7fee64140c00 2022-06-17T12:47:06.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.209+0000 7fee68bf7700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fee64060020 0x7fee640b2b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49206/0 (socket says 172.21.15.36:49206) 2022-06-17T12:47:06.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.209+0000 7fee68bf7700 1 -- 172.21.15.36:0/2247064821 learned_addr learned my addr 172.21.15.36:0/2247064821 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:06.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.209+0000 7fee63fff700 1 --2- 172.21.15.36:0/2247064821 >> v2:172.21.15.70:3300/0 conn(0x7fee64140c00 0x7fee640b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:06.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.209+0000 7fee637fe700 1 -- 172.21.15.36:0/2247064821 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3467920393 0 0) 0x7fee640b2120 con 0x7fee64060020 2022-06-17T12:47:06.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.209+0000 7fee637fe700 1 -- 172.21.15.36:0/2247064821 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee50003260 con 0x7fee64060020 2022-06-17T12:47:06.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.209+0000 7fee63fff700 1 -- 172.21.15.36:0/2247064821 >> v1:172.21.15.70:6789/0 conn(0x7fee640b3e20 legacy=0x7fee6413a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:06.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.209+0000 7fee63fff700 1 -- 172.21.15.36:0/2247064821 >> v1:172.21.15.36:6789/0 conn(0x7fee64060020 legacy=0x7fee640b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:06.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.209+0000 7fee63fff700 1 -- 172.21.15.36:0/2247064821 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee640b2660 con 0x7fee64140c00 2022-06-17T12:47:06.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.209+0000 7fee63fff700 1 --2- 172.21.15.36:0/2247064821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee64140c00 0x7fee640b3930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fee54002f70 tx=0x7fee54006df0).ready entity=mon.1 client_cookie=ece8c8c29a0954de server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:06.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.209+0000 7fee637fe700 1 -- 172.21.15.36:0/2247064821 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee5400d610 con 0x7fee64140c00 2022-06-17T12:47:06.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.209+0000 7fee637fe700 1 -- 172.21.15.36:0/2247064821 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee54000e90 con 0x7fee64140c00 2022-06-17T12:47:06.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.209+0000 7fee637fe700 1 -- 172.21.15.36:0/2247064821 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee5400d960 con 0x7fee64140c00 2022-06-17T12:47:06.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.209+0000 7fee69bf9700 1 -- 172.21.15.36:0/2247064821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee64140c00 msgr2=0x7fee640b3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:06.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.209+0000 7fee69bf9700 1 --2- 172.21.15.36:0/2247064821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee64140c00 0x7fee640b3930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fee54002f70 tx=0x7fee54006df0).stop 2022-06-17T12:47:06.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.209+0000 7fee69bf9700 1 -- 172.21.15.36:0/2247064821 shutdown_connections 2022-06-17T12:47:06.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.209+0000 7fee69bf9700 1 --2- 172.21.15.36:0/2247064821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee64140c00 0x7fee640b3930 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:06.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.209+0000 7fee69bf9700 1 -- 172.21.15.36:0/2247064821 >> 172.21.15.36:0/2247064821 conn(0x7fee64060990 msgr2=0x7fee6405f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:06.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.209+0000 7fee69bf9700 1 -- 172.21.15.36:0/2247064821 shutdown_connections 2022-06-17T12:47:06.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.209+0000 7fee69bf9700 1 -- 172.21.15.36:0/2247064821 wait complete. 2022-06-17T12:47:06.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.213+0000 7fee69bf9700 1 Processor -- start 2022-06-17T12:47:06.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.213+0000 7fee69bf9700 1 -- start start 2022-06-17T12:47:06.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.213+0000 7fee69bf9700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fee64060020 0x7fee640b6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:06.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.213+0000 7fee69bf9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee640b3e20 0x7fee64061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:06.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.213+0000 7fee69bf9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fee64140c00 0x7fee64061b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:06.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.213+0000 7fee69bf9700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee64062e60 con 0x7fee64140c00 2022-06-17T12:47:06.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.213+0000 7fee69bf9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee64063100 con 0x7fee640b3e20 2022-06-17T12:47:06.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.213+0000 7fee69bf9700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee640af250 con 0x7fee64060020 2022-06-17T12:47:06.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.213+0000 7fee693f8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fee64140c00 0x7fee64061b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:06.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.213+0000 7fee63fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee640b3e20 0x7fee64061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:06.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.213+0000 7fee693f8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fee64140c00 0x7fee64061b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59918/0 (socket says 172.21.15.36:59918) 2022-06-17T12:47:06.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.213+0000 7fee693f8700 1 -- 172.21.15.36:0/3813055668 learned_addr learned my addr 172.21.15.36:0/3813055668 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:06.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.213+0000 7fee68bf7700 1 --2- 172.21.15.36:0/3813055668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fee64060020 0x7fee640b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:06.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.213+0000 7fee63fff700 1 -- 172.21.15.36:0/3813055668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fee64060020 msgr2=0x7fee640b6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:06.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.213+0000 7fee63fff700 1 --2- 172.21.15.36:0/3813055668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fee64060020 0x7fee640b6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:06.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.213+0000 7fee63fff700 1 -- 172.21.15.36:0/3813055668 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fee64140c00 msgr2=0x7fee64061b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:06.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.213+0000 7fee63fff700 1 --2- 172.21.15.36:0/3813055668 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fee64140c00 0x7fee64061b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:06.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.213+0000 7fee63fff700 1 -- 172.21.15.36:0/3813055668 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5400b180 con 0x7fee640b3e20 2022-06-17T12:47:06.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.217+0000 7fee63fff700 1 --2- 172.21.15.36:0/3813055668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee640b3e20 0x7fee64061640 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fee54002f70 tx=0x7fee54014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:06.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.217+0000 7fee61ffb700 1 -- 172.21.15.36:0/3813055668 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee54016070 con 0x7fee640b3e20 2022-06-17T12:47:06.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.217+0000 7fee61ffb700 1 -- 172.21.15.36:0/3813055668 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5400de90 con 0x7fee640b3e20 2022-06-17T12:47:06.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.217+0000 7fee69bf9700 1 -- 172.21.15.36:0/3813055668 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee641522a0 con 0x7fee640b3e20 2022-06-17T12:47:06.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.217+0000 7fee61ffb700 1 -- 172.21.15.36:0/3813055668 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee54013da0 con 0x7fee640b3e20 2022-06-17T12:47:06.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.217+0000 7fee69bf9700 1 -- 172.21.15.36:0/3813055668 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee6413cdb0 con 0x7fee640b3e20 2022-06-17T12:47:06.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.217+0000 7fee69bf9700 1 -- 172.21.15.36:0/3813055668 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee64063480 con 0x7fee640b3e20 2022-06-17T12:47:06.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.217+0000 7fee61ffb700 1 -- 172.21.15.36:0/3813055668 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fee54025030 con 0x7fee640b3e20 2022-06-17T12:47:06.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.217+0000 7fee61ffb700 1 --2- 172.21.15.36:0/3813055668 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fee44053a40 0x7fee44055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:06.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.217+0000 7fee68bf7700 1 --2- 172.21.15.36:0/3813055668 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fee44053a40 0x7fee44055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:06.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.217+0000 7fee61ffb700 1 -- 172.21.15.36:0/3813055668 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fee5406cf50 con 0x7fee640b3e20 2022-06-17T12:47:06.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.221+0000 7fee68bf7700 1 --2- 172.21.15.36:0/3813055668 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fee44053a40 0x7fee44055f00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fee5800d000 tx=0x7fee58009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:06.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.225+0000 7fee61ffb700 1 -- 172.21.15.36:0/3813055668 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fee54043c40 con 0x7fee640b3e20 2022-06-17T12:47:06.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.353+0000 7fee69bf9700 1 -- 172.21.15.36:0/3813055668 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fee640b1da0 con 0x7fee640b3e20 2022-06-17T12:47:06.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.357+0000 7fee61ffb700 1 -- 172.21.15.36:0/3813055668 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7fee54023020 con 0x7fee640b3e20 2022-06-17T12:47:06.352 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:47:06.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.357+0000 7fee69bf9700 1 -- 172.21.15.36:0/3813055668 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fee44053a40 msgr2=0x7fee44055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:06.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.357+0000 7fee69bf9700 1 --2- 172.21.15.36:0/3813055668 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fee44053a40 0x7fee44055f00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fee5800d000 tx=0x7fee58009040).stop 2022-06-17T12:47:06.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.357+0000 7fee69bf9700 1 -- 172.21.15.36:0/3813055668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee640b3e20 msgr2=0x7fee64061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:06.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.357+0000 7fee69bf9700 1 --2- 172.21.15.36:0/3813055668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee640b3e20 0x7fee64061640 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fee54002f70 tx=0x7fee54014000).stop 2022-06-17T12:47:06.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.357+0000 7fee69bf9700 1 -- 172.21.15.36:0/3813055668 shutdown_connections 2022-06-17T12:47:06.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.357+0000 7fee69bf9700 1 --2- 172.21.15.36:0/3813055668 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fee44053a40 0x7fee44055f00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:06.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.357+0000 7fee69bf9700 1 --2- 172.21.15.36:0/3813055668 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fee64140c00 0x7fee64061b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:06.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.357+0000 7fee69bf9700 1 --2- 172.21.15.36:0/3813055668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee640b3e20 0x7fee64061640 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:06.366 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.357+0000 7fee69bf9700 1 --2- 172.21.15.36:0/3813055668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fee64060020 0x7fee640b6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:06.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.357+0000 7fee69bf9700 1 -- 172.21.15.36:0/3813055668 >> 172.21.15.36:0/3813055668 conn(0x7fee64060990 msgr2=0x7fee6413e800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:06.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.357+0000 7fee69bf9700 1 -- 172.21.15.36:0/3813055668 shutdown_connections 2022-06-17T12:47:06.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.357+0000 7fee69bf9700 1 -- 172.21.15.36:0/3813055668 wait complete. 2022-06-17T12:47:06.370 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:47:06.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.493+0000 7f529ccb4700 1 Processor -- start 2022-06-17T12:47:06.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.493+0000 7f529ccb4700 1 -- start start 2022-06-17T12:47:06.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.493+0000 7f529ccb4700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f52980b5be0 0x7f5298144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:06.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.493+0000 7f529ccb4700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5298147270 0x7f529814ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:06.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.493+0000 7f529ccb4700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52980b2c80 con 0x7f52980601c0 2022-06-17T12:47:06.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.493+0000 7f529ccb4700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52980b72d0 con 0x7f52980b5be0 2022-06-17T12:47:06.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.493+0000 7f529ccb4700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52980b0e50 con 0x7f5298147270 2022-06-17T12:47:06.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.493+0000 7f5296ffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f52980b5be0 0x7f5298144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:06.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.493+0000 7f52977fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f52980601c0 0x7f5298144270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49222/0 (socket says 172.21.15.36:49222) 2022-06-17T12:47:06.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.493+0000 7f52977fe700 1 -- 172.21.15.36:0/4106256195 learned_addr learned my addr 172.21.15.36:0/4106256195 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:06.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.497+0000 7f5297fff700 1 --2- 172.21.15.36:0/4106256195 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5298147270 0x7f529814ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:06.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.497+0000 7f5296ffd700 1 -- 172.21.15.36:0/4106256195 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5298147270 msgr2=0x7f529814ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:06.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.497+0000 7f5296ffd700 1 --2- 172.21.15.36:0/4106256195 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5298147270 0x7f529814ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:06.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.497+0000 7f5296ffd700 1 -- 172.21.15.36:0/4106256195 >> v1:172.21.15.36:6789/0 conn(0x7f52980601c0 legacy=0x7f5298144270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:06.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.497+0000 7f5296ffd700 1 -- 172.21.15.36:0/4106256195 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52980605a0 con 0x7f52980b5be0 2022-06-17T12:47:06.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.497+0000 7f52967fc700 1 -- 172.21.15.36:0/4106256195 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2010555387 0 0) 0x7f52980b2c80 con 0x7f52980601c0 2022-06-17T12:47:06.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.497+0000 7f5296ffd700 1 --2- 172.21.15.36:0/4106256195 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52980b5be0 0x7f5298144980 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f5288002f70 tx=0x7f5288006df0).ready entity=mon.0 client_cookie=2b28cc92da7b0b10 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:06.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.497+0000 7f52967fc700 1 -- 172.21.15.36:0/4106256195 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f528800d570 con 0x7f52980b5be0 2022-06-17T12:47:06.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.497+0000 7f52967fc700 1 -- 172.21.15.36:0/4106256195 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5288000e90 con 0x7f52980b5be0 2022-06-17T12:47:06.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.497+0000 7f52967fc700 1 -- 172.21.15.36:0/4106256195 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f528800d8c0 con 0x7f52980b5be0 2022-06-17T12:47:06.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.497+0000 7f529ccb4700 1 -- 172.21.15.36:0/4106256195 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52980b5be0 msgr2=0x7f5298144980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:06.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.497+0000 7f529ccb4700 1 --2- 172.21.15.36:0/4106256195 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52980b5be0 0x7f5298144980 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f5288002f70 tx=0x7f5288006df0).stop 2022-06-17T12:47:06.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.497+0000 7f529ccb4700 1 -- 172.21.15.36:0/4106256195 shutdown_connections 2022-06-17T12:47:06.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.497+0000 7f529ccb4700 1 --2- 172.21.15.36:0/4106256195 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5298147270 0x7f529814ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:06.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.497+0000 7f529ccb4700 1 --2- 172.21.15.36:0/4106256195 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52980b5be0 0x7f5298144980 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:06.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.497+0000 7f529ccb4700 1 -- 172.21.15.36:0/4106256195 >> 172.21.15.36:0/4106256195 conn(0x7f529805c550 msgr2=0x7f529805f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:06.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.497+0000 7f529ccb4700 1 -- 172.21.15.36:0/4106256195 shutdown_connections 2022-06-17T12:47:06.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.497+0000 7f529ccb4700 1 -- 172.21.15.36:0/4106256195 wait complete. 2022-06-17T12:47:06.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.501+0000 7f529ccb4700 1 Processor -- start 2022-06-17T12:47:06.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.501+0000 7f529ccb4700 1 -- start start 2022-06-17T12:47:06.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.501+0000 7f529ccb4700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52980601c0 0x7f5298148cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:06.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.501+0000 7f529ccb4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52980b5be0 0x7f5298159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:06.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.501+0000 7f529ccb4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5298147270 0x7f5298159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:06.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.501+0000 7f529ccb4700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5298065c10 con 0x7f5298147270 2022-06-17T12:47:06.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.501+0000 7f529ccb4700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52980aec80 con 0x7f52980b5be0 2022-06-17T12:47:06.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.501+0000 7f529ccb4700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52980aef20 con 0x7f52980601c0 2022-06-17T12:47:06.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.501+0000 7f5297fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5298147270 0x7f5298159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:06.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.501+0000 7f5297fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5298147270 0x7f5298159f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59932/0 (socket says 172.21.15.36:59932) 2022-06-17T12:47:06.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.501+0000 7f5296ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52980b5be0 0x7f5298159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:06.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.501+0000 7f5297fff700 1 -- 172.21.15.36:0/4137487265 learned_addr learned my addr 172.21.15.36:0/4137487265 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:06.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.501+0000 7f52977fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52980601c0 0x7f5298148cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:06.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.501+0000 7f5297fff700 1 -- 172.21.15.36:0/4137487265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52980601c0 msgr2=0x7f5298148cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:06.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.501+0000 7f5297fff700 1 --2- 172.21.15.36:0/4137487265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52980601c0 0x7f5298148cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:06.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.501+0000 7f5297fff700 1 -- 172.21.15.36:0/4137487265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52980b5be0 msgr2=0x7f5298159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:06.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.501+0000 7f5297fff700 1 --2- 172.21.15.36:0/4137487265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52980b5be0 0x7f5298159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:06.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.501+0000 7f5297fff700 1 -- 172.21.15.36:0/4137487265 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528800b180 con 0x7f5298147270 2022-06-17T12:47:06.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.501+0000 7f5297fff700 1 --2- 172.21.15.36:0/4137487265 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5298147270 0x7f5298159f90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f529000a000 tx=0x7f5290008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:06.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.505+0000 7f5294ff9700 1 -- 172.21.15.36:0/4137487265 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52900127e0 con 0x7f5298147270 2022-06-17T12:47:06.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.505+0000 7f5294ff9700 1 -- 172.21.15.36:0/4137487265 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5290012940 con 0x7f5298147270 2022-06-17T12:47:06.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.505+0000 7f529ccb4700 1 -- 172.21.15.36:0/4137487265 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52980af1c0 con 0x7f5298147270 2022-06-17T12:47:06.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.505+0000 7f5294ff9700 1 -- 172.21.15.36:0/4137487265 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5290012ca0 con 0x7f5298147270 2022-06-17T12:47:06.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.505+0000 7f529ccb4700 1 -- 172.21.15.36:0/4137487265 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52980ae850 con 0x7f5298147270 2022-06-17T12:47:06.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.505+0000 7f529ccb4700 1 -- 172.21.15.36:0/4137487265 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5298067400 con 0x7f5298147270 2022-06-17T12:47:06.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.505+0000 7f5294ff9700 1 -- 172.21.15.36:0/4137487265 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f5290013480 con 0x7f5298147270 2022-06-17T12:47:06.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.505+0000 7f5294ff9700 1 --2- 172.21.15.36:0/4137487265 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5278053a60 0x7f5278055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:06.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.505+0000 7f5294ff9700 1 -- 172.21.15.36:0/4137487265 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f529006c2c0 con 0x7f5298147270 2022-06-17T12:47:06.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.505+0000 7f52977fe700 1 --2- 172.21.15.36:0/4137487265 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5278053a60 0x7f5278055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:06.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.509+0000 7f52977fe700 1 --2- 172.21.15.36:0/4137487265 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5278053a60 0x7f5278055f20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f528c00d000 tx=0x7f528c007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:06.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.513+0000 7f5294ff9700 1 -- 172.21.15.36:0/4137487265 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5298067400 con 0x7f5298147270 2022-06-17T12:47:06.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.637+0000 7f529ccb4700 1 -- 172.21.15.36:0/4137487265 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5298067400 con 0x7f5298147270 2022-06-17T12:47:06.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.641+0000 7f5294ff9700 1 -- 172.21.15.36:0/4137487265 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5298067400 con 0x7f5298147270 2022-06-17T12:47:06.736 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:06.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.641+0000 7f529ccb4700 1 -- 172.21.15.36:0/4137487265 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5278053a60 msgr2=0x7f5278055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:06.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.641+0000 7f529ccb4700 1 --2- 172.21.15.36:0/4137487265 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5278053a60 0x7f5278055f20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f528c00d000 tx=0x7f528c007a00).stop 2022-06-17T12:47:06.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.641+0000 7f529ccb4700 1 -- 172.21.15.36:0/4137487265 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5298147270 msgr2=0x7f5298159f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:06.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.641+0000 7f529ccb4700 1 --2- 172.21.15.36:0/4137487265 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5298147270 0x7f5298159f90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f529000a000 tx=0x7f5290008040).stop 2022-06-17T12:47:06.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.641+0000 7f529ccb4700 1 -- 172.21.15.36:0/4137487265 shutdown_connections 2022-06-17T12:47:06.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.641+0000 7f529ccb4700 1 --2- 172.21.15.36:0/4137487265 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5278053a60 0x7f5278055f20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:06.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.641+0000 7f529ccb4700 1 --2- 172.21.15.36:0/4137487265 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5298147270 0x7f5298159f90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:06.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.641+0000 7f529ccb4700 1 --2- 172.21.15.36:0/4137487265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52980b5be0 0x7f5298159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:06.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.641+0000 7f529ccb4700 1 --2- 172.21.15.36:0/4137487265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52980601c0 0x7f5298148cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:06.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.641+0000 7f529ccb4700 1 -- 172.21.15.36:0/4137487265 >> 172.21.15.36:0/4137487265 conn(0x7f529805c550 msgr2=0x7f52981468a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:06.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.641+0000 7f529ccb4700 1 -- 172.21.15.36:0/4137487265 shutdown_connections 2022-06-17T12:47:06.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.641+0000 7f529ccb4700 1 -- 172.21.15.36:0/4137487265 wait complete. 2022-06-17T12:47:06.749 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:47:06.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.929+0000 7fa9a3fff700 1 Processor -- start 2022-06-17T12:47:06.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.929+0000 7fa9a3fff700 1 -- start start 2022-06-17T12:47:06.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.929+0000 7fa9a3fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa9a413ca70 0x7fa9a413ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:06.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.929+0000 7fa9a3fff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a41432c0 con 0x7fa9a40b3f80 2022-06-17T12:47:06.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.929+0000 7fa9a3fff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a4143df0 con 0x7fa9a4060020 2022-06-17T12:47:06.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.929+0000 7fa9a3fff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a4062db0 con 0x7fa9a413ca70 2022-06-17T12:47:06.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.933+0000 7fa9a37fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa9a413ca70 0x7fa9a413ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:06.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.933+0000 7fa9a37fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa9a413ca70 0x7fa9a413ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59940/0 (socket says 172.21.15.36:59940) 2022-06-17T12:47:06.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.933+0000 7fa9a27fc700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fa9a40b3f80 0x7fa9a40b46a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49234/0 (socket says 172.21.15.36:49234) 2022-06-17T12:47:06.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.933+0000 7fa9a37fe700 1 -- 172.21.15.36:0/415349532 learned_addr learned my addr 172.21.15.36:0/415349532 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:06.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.933+0000 7fa9a37fe700 1 -- 172.21.15.36:0/415349532 >> v1:172.21.15.70:6789/0 conn(0x7fa9a4060020 legacy=0x7fa9a40626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:06.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.933+0000 7fa9a37fe700 1 -- 172.21.15.36:0/415349532 >> v1:172.21.15.36:6789/0 conn(0x7fa9a40b3f80 legacy=0x7fa9a40b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:06.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.933+0000 7fa9a1ffb700 1 -- 172.21.15.36:0/415349532 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1310096911 0 0) 0x7fa9a41432c0 con 0x7fa9a40b3f80 2022-06-17T12:47:06.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.933+0000 7fa9a37fe700 1 -- 172.21.15.36:0/415349532 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a4061f60 con 0x7fa9a413ca70 2022-06-17T12:47:06.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.933+0000 7fa9a37fe700 1 --2- 172.21.15.36:0/415349532 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa9a413ca70 0x7fa9a413ceb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fa99c0032f0 tx=0x7fa99c009290).ready entity=mon.0 client_cookie=46ea98ea0c1140fb server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:06.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.933+0000 7fa9a1ffb700 1 -- 172.21.15.36:0/415349532 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa99c00b8a0 con 0x7fa9a413ca70 2022-06-17T12:47:06.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.933+0000 7fa9a1ffb700 1 -- 172.21.15.36:0/415349532 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99c00ba00 con 0x7fa9a413ca70 2022-06-17T12:47:06.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.933+0000 7fa9a1ffb700 1 -- 172.21.15.36:0/415349532 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa99c00bb70 con 0x7fa9a413ca70 2022-06-17T12:47:06.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.933+0000 7fa9a3fff700 1 -- 172.21.15.36:0/415349532 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa9a413ca70 msgr2=0x7fa9a413ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:06.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.933+0000 7fa9a3fff700 1 --2- 172.21.15.36:0/415349532 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa9a413ca70 0x7fa9a413ceb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fa99c0032f0 tx=0x7fa99c009290).stop 2022-06-17T12:47:06.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.933+0000 7fa9a3fff700 1 -- 172.21.15.36:0/415349532 shutdown_connections 2022-06-17T12:47:06.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.933+0000 7fa9a3fff700 1 --2- 172.21.15.36:0/415349532 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa9a413ca70 0x7fa9a413ceb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:06.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.933+0000 7fa9a3fff700 1 -- 172.21.15.36:0/415349532 >> 172.21.15.36:0/415349532 conn(0x7fa9a405c550 msgr2=0x7fa9a405f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:06.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.933+0000 7fa9a3fff700 1 -- 172.21.15.36:0/415349532 shutdown_connections 2022-06-17T12:47:06.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.933+0000 7fa9a3fff700 1 -- 172.21.15.36:0/415349532 wait complete. 2022-06-17T12:47:06.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.937+0000 7fa9a3fff700 1 Processor -- start 2022-06-17T12:47:06.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.937+0000 7fa9a3fff700 1 -- start start 2022-06-17T12:47:06.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.937+0000 7fa9a3fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9a4060020 0x7fa9a4145670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:07.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.937+0000 7fa9a3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9a40b3f80 0x7fa9a4149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:07.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.937+0000 7fa9a3fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa9a413ca70 0x7fa9a4064d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:07.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.937+0000 7fa9a3fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a413c930 con 0x7fa9a413ca70 2022-06-17T12:47:07.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.937+0000 7fa9a3fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a41486d0 con 0x7fa9a40b3f80 2022-06-17T12:47:07.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.937+0000 7fa9a3fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a4148900 con 0x7fa9a4060020 2022-06-17T12:47:07.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.937+0000 7fa9a37fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa9a413ca70 0x7fa9a4064d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:07.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.937+0000 7fa9a37fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa9a413ca70 0x7fa9a4064d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59946/0 (socket says 172.21.15.36:59946) 2022-06-17T12:47:07.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.937+0000 7fa9a27fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9a40b3f80 0x7fa9a4149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:07.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.937+0000 7fa9a37fe700 1 -- 172.21.15.36:0/1573179566 learned_addr learned my addr 172.21.15.36:0/1573179566 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:07.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.937+0000 7fa9a2ffd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9a4060020 0x7fa9a4145670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:07.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.941+0000 7fa9a37fe700 1 -- 172.21.15.36:0/1573179566 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9a4060020 msgr2=0x7fa9a4145670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:07.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.941+0000 7fa9a37fe700 1 --2- 172.21.15.36:0/1573179566 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9a4060020 0x7fa9a4145670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:07.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.941+0000 7fa9a37fe700 1 -- 172.21.15.36:0/1573179566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9a40b3f80 msgr2=0x7fa9a4149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:07.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.941+0000 7fa9a37fe700 1 --2- 172.21.15.36:0/1573179566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9a40b3f80 0x7fa9a4149890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:07.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.941+0000 7fa9a37fe700 1 -- 172.21.15.36:0/1573179566 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99c009530 con 0x7fa9a413ca70 2022-06-17T12:47:07.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.941+0000 7fa9a37fe700 1 --2- 172.21.15.36:0/1573179566 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa9a413ca70 0x7fa9a4064d80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fa99c0032f0 tx=0x7fa99c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:07.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.941+0000 7fa993fff700 1 -- 172.21.15.36:0/1573179566 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa99c00bb70 con 0x7fa9a413ca70 2022-06-17T12:47:07.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.941+0000 7fa993fff700 1 -- 172.21.15.36:0/1573179566 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99c01f3f0 con 0x7fa9a413ca70 2022-06-17T12:47:07.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.941+0000 7fa9a3fff700 1 -- 172.21.15.36:0/1573179566 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a4148ba0 con 0x7fa9a413ca70 2022-06-17T12:47:07.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.941+0000 7fa993fff700 1 -- 172.21.15.36:0/1573179566 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa99c01f750 con 0x7fa9a413ca70 2022-06-17T12:47:07.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.941+0000 7fa9a3fff700 1 -- 172.21.15.36:0/1573179566 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a4149d80 con 0x7fa9a413ca70 2022-06-17T12:47:07.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.941+0000 7fa9a3fff700 1 -- 172.21.15.36:0/1573179566 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a4062200 con 0x7fa9a413ca70 2022-06-17T12:47:07.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.941+0000 7fa993fff700 1 -- 172.21.15.36:0/1573179566 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fa99c026030 con 0x7fa9a413ca70 2022-06-17T12:47:07.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.945+0000 7fa993fff700 1 --2- 172.21.15.36:0/1573179566 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa980053920 0x7fa980055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:07.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.945+0000 7fa993fff700 1 -- 172.21.15.36:0/1573179566 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa99c06cd60 con 0x7fa9a413ca70 2022-06-17T12:47:07.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.945+0000 7fa9a2ffd700 1 --2- 172.21.15.36:0/1573179566 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa980053920 0x7fa980055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:07.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.945+0000 7fa9a2ffd700 1 --2- 172.21.15.36:0/1573179566 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa980053920 0x7fa980055de0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fa99400d000 tx=0x7fa994007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:07.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:06.949+0000 7fa993fff700 1 -- 172.21.15.36:0/1573179566 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa99c043a50 con 0x7fa9a413ca70 2022-06-17T12:47:07.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.081+0000 7fa9a3fff700 1 -- 172.21.15.36:0/1573179566 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa9a405f220 con 0x7fa9a413ca70 2022-06-17T12:47:07.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.081+0000 7fa993fff700 1 -- 172.21.15.36:0/1573179566 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+10 (secure 0 0 0) 0x7fa99c0240a0 con 0x7fa9a413ca70 2022-06-17T12:47:07.114 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:07.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.085+0000 7fa9a3fff700 1 -- 172.21.15.36:0/1573179566 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa980053920 msgr2=0x7fa980055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:07.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.085+0000 7fa9a3fff700 1 --2- 172.21.15.36:0/1573179566 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa980053920 0x7fa980055de0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fa99400d000 tx=0x7fa994007b90).stop 2022-06-17T12:47:07.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.085+0000 7fa9a3fff700 1 -- 172.21.15.36:0/1573179566 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa9a413ca70 msgr2=0x7fa9a4064d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:07.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.085+0000 7fa9a3fff700 1 --2- 172.21.15.36:0/1573179566 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa9a413ca70 0x7fa9a4064d80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fa99c0032f0 tx=0x7fa99c016000).stop 2022-06-17T12:47:07.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.085+0000 7fa9a3fff700 1 -- 172.21.15.36:0/1573179566 shutdown_connections 2022-06-17T12:47:07.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.085+0000 7fa9a3fff700 1 --2- 172.21.15.36:0/1573179566 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa980053920 0x7fa980055de0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:07.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.085+0000 7fa9a3fff700 1 --2- 172.21.15.36:0/1573179566 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa9a413ca70 0x7fa9a4064d80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:07.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.085+0000 7fa9a3fff700 1 --2- 172.21.15.36:0/1573179566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9a40b3f80 0x7fa9a4149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:07.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.085+0000 7fa9a3fff700 1 --2- 172.21.15.36:0/1573179566 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9a4060020 0x7fa9a4145670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:07.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.085+0000 7fa9a3fff700 1 -- 172.21.15.36:0/1573179566 >> 172.21.15.36:0/1573179566 conn(0x7fa9a405c550 msgr2=0x7fa9a405f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:07.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.085+0000 7fa9a3fff700 1 -- 172.21.15.36:0/1573179566 shutdown_connections 2022-06-17T12:47:07.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.085+0000 7fa9a3fff700 1 -- 172.21.15.36:0/1573179566 wait complete. 2022-06-17T12:47:07.127 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:47:07.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.257+0000 7fedaa07a700 1 Processor -- start 2022-06-17T12:47:07.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.257+0000 7fedaa07a700 1 -- start start 2022-06-17T12:47:07.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7fedaa07a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7feda40b27b0 0x7feda414b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:07.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7fedaa07a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feda4147bd0 0x7feda41442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:07.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7fedaa07a700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feda414b870 con 0x7feda4060ca0 2022-06-17T12:47:07.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7fedaa07a700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feda40ac500 con 0x7feda40b27b0 2022-06-17T12:47:07.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7fedaa07a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda40ace90 con 0x7feda4147bd0 2022-06-17T12:47:07.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7feda9078700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7feda40b27b0 0x7feda414b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:07.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7feda8877700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7feda4060ca0 0x7feda40ac780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49248/0 (socket says 172.21.15.36:49248) 2022-06-17T12:47:07.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7feda8877700 1 -- 172.21.15.36:0/1301846213 learned_addr learned my addr 172.21.15.36:0/1301846213 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:07.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7feda9879700 1 --2- 172.21.15.36:0/1301846213 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feda4147bd0 0x7feda41442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:07.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7fed9bfff700 1 -- 172.21.15.36:0/1301846213 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4116660446 0 0) 0x7feda414b870 con 0x7feda4060ca0 2022-06-17T12:47:07.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7fed9bfff700 1 -- 172.21.15.36:0/1301846213 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed88003260 con 0x7feda4060ca0 2022-06-17T12:47:07.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7feda9078700 1 -- 172.21.15.36:0/1301846213 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feda4147bd0 msgr2=0x7feda41442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:07.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7feda9078700 1 --2- 172.21.15.36:0/1301846213 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feda4147bd0 0x7feda41442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:07.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7feda9078700 1 -- 172.21.15.36:0/1301846213 >> v1:172.21.15.36:6789/0 conn(0x7feda4060ca0 legacy=0x7feda40ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:07.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7feda9078700 1 -- 172.21.15.36:0/1301846213 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda40b2b90 con 0x7feda40b27b0 2022-06-17T12:47:07.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7feda9078700 1 --2- 172.21.15.36:0/1301846213 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feda40b27b0 0x7feda414b2b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fed94002f70 tx=0x7fed94006df0).ready entity=mon.0 client_cookie=88368df3ca6afa26 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:07.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7fed9bfff700 1 -- 172.21.15.36:0/1301846213 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed9400d4a0 con 0x7feda40b27b0 2022-06-17T12:47:07.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7fed9bfff700 1 -- 172.21.15.36:0/1301846213 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed94000e90 con 0x7feda40b27b0 2022-06-17T12:47:07.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7fed9bfff700 1 -- 172.21.15.36:0/1301846213 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed9400d7f0 con 0x7feda40b27b0 2022-06-17T12:47:07.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7fedaa07a700 1 -- 172.21.15.36:0/1301846213 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feda40b27b0 msgr2=0x7feda414b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:07.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7fedaa07a700 1 --2- 172.21.15.36:0/1301846213 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feda40b27b0 0x7feda414b2b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fed94002f70 tx=0x7fed94006df0).stop 2022-06-17T12:47:07.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7fedaa07a700 1 -- 172.21.15.36:0/1301846213 shutdown_connections 2022-06-17T12:47:07.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7fedaa07a700 1 --2- 172.21.15.36:0/1301846213 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feda4147bd0 0x7feda41442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:07.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7fedaa07a700 1 --2- 172.21.15.36:0/1301846213 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feda40b27b0 0x7feda414b2b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:07.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7fedaa07a700 1 -- 172.21.15.36:0/1301846213 >> 172.21.15.36:0/1301846213 conn(0x7feda40b5d10 msgr2=0x7feda40b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:07.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7fedaa07a700 1 -- 172.21.15.36:0/1301846213 shutdown_connections 2022-06-17T12:47:07.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.261+0000 7fedaa07a700 1 -- 172.21.15.36:0/1301846213 wait complete. 2022-06-17T12:47:07.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.265+0000 7fedaa07a700 1 Processor -- start 2022-06-17T12:47:07.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.265+0000 7fedaa07a700 1 -- start start 2022-06-17T12:47:07.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.265+0000 7fedaa07a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feda4060ca0 0x7feda40a7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:07.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.265+0000 7fedaa07a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feda40b27b0 0x7feda41599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:07.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.265+0000 7fedaa07a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feda4147bd0 0x7feda4159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:07.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.265+0000 7fedaa07a700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda4064490 con 0x7feda4147bd0 2022-06-17T12:47:07.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.265+0000 7fedaa07a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda40a86f0 con 0x7feda4060ca0 2022-06-17T12:47:07.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.265+0000 7fedaa07a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda40a8940 con 0x7feda40b27b0 2022-06-17T12:47:07.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.265+0000 7feda9879700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feda4147bd0 0x7feda4159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:07.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.265+0000 7feda9879700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feda4147bd0 0x7feda4159ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59960/0 (socket says 172.21.15.36:59960) 2022-06-17T12:47:07.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.265+0000 7feda9078700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feda4060ca0 0x7feda40a7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:07.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.265+0000 7feda8877700 1 --2- 172.21.15.36:0/4199260152 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feda40b27b0 0x7feda41599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:07.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.265+0000 7feda9879700 1 -- 172.21.15.36:0/4199260152 learned_addr learned my addr 172.21.15.36:0/4199260152 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:07.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.265+0000 7feda9879700 1 -- 172.21.15.36:0/4199260152 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feda40b27b0 msgr2=0x7feda41599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:07.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.265+0000 7feda9879700 1 --2- 172.21.15.36:0/4199260152 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feda40b27b0 0x7feda41599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:07.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.265+0000 7feda9879700 1 -- 172.21.15.36:0/4199260152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feda4060ca0 msgr2=0x7feda40a7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:07.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.265+0000 7feda9879700 1 --2- 172.21.15.36:0/4199260152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feda4060ca0 0x7feda40a7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:07.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.265+0000 7feda9879700 1 -- 172.21.15.36:0/4199260152 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9400b180 con 0x7feda4147bd0 2022-06-17T12:47:07.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.265+0000 7feda9879700 1 --2- 172.21.15.36:0/4199260152 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feda4147bd0 0x7feda4159ec0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fed9c006e00 tx=0x7fed9c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:07.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.265+0000 7fed9a7fc700 1 -- 172.21.15.36:0/4199260152 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed9c00c640 con 0x7feda4147bd0 2022-06-17T12:47:07.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.265+0000 7fedaa07a700 1 -- 172.21.15.36:0/4199260152 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda40a8be0 con 0x7feda4147bd0 2022-06-17T12:47:07.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.265+0000 7fed9a7fc700 1 -- 172.21.15.36:0/4199260152 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9c00c7a0 con 0x7feda4147bd0 2022-06-17T12:47:07.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.265+0000 7fed9a7fc700 1 -- 172.21.15.36:0/4199260152 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed9c00cb00 con 0x7feda4147bd0 2022-06-17T12:47:07.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.265+0000 7fedaa07a700 1 -- 172.21.15.36:0/4199260152 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda40670c0 con 0x7feda4147bd0 2022-06-17T12:47:07.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.269+0000 7fedaa07a700 1 -- 172.21.15.36:0/4199260152 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda4057410 con 0x7feda4147bd0 2022-06-17T12:47:07.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.269+0000 7fed9a7fc700 1 -- 172.21.15.36:0/4199260152 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fed9c00cc60 con 0x7feda4147bd0 2022-06-17T12:47:07.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.269+0000 7fed9a7fc700 1 --2- 172.21.15.36:0/4199260152 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fed8c0539d0 0x7fed8c055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:07.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.269+0000 7fed9a7fc700 1 -- 172.21.15.36:0/4199260152 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fed9c06c220 con 0x7feda4147bd0 2022-06-17T12:47:07.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.269+0000 7feda9078700 1 --2- 172.21.15.36:0/4199260152 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fed8c0539d0 0x7fed8c055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:07.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.269+0000 7feda9078700 1 --2- 172.21.15.36:0/4199260152 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fed8c0539d0 0x7fed8c055e90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fed9401a000 tx=0x7fed94014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:07.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.277+0000 7fed9a7fc700 1 -- 172.21.15.36:0/4199260152 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fed9c042f80 con 0x7feda4147bd0 2022-06-17T12:47:07.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.405+0000 7fedaa07a700 1 -- 172.21.15.36:0/4199260152 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feda40a9080 con 0x7feda4147bd0 2022-06-17T12:47:07.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.405+0000 7fed9a7fc700 1 -- 172.21.15.36:0/4199260152 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+10 (secure 0 0 0) 0x7fed9c0465a0 con 0x7feda4147bd0 2022-06-17T12:47:07.401 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:07.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.409+0000 7fedaa07a700 1 -- 172.21.15.36:0/4199260152 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fed8c0539d0 msgr2=0x7fed8c055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:07.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.409+0000 7fedaa07a700 1 --2- 172.21.15.36:0/4199260152 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fed8c0539d0 0x7fed8c055e90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fed9401a000 tx=0x7fed94014000).stop 2022-06-17T12:47:07.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.409+0000 7fedaa07a700 1 -- 172.21.15.36:0/4199260152 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feda4147bd0 msgr2=0x7feda4159ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:07.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.409+0000 7fedaa07a700 1 --2- 172.21.15.36:0/4199260152 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feda4147bd0 0x7feda4159ec0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fed9c006e00 tx=0x7fed9c0091d0).stop 2022-06-17T12:47:07.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.409+0000 7fedaa07a700 1 -- 172.21.15.36:0/4199260152 shutdown_connections 2022-06-17T12:47:07.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.409+0000 7fedaa07a700 1 --2- 172.21.15.36:0/4199260152 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fed8c0539d0 0x7fed8c055e90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:07.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.409+0000 7fedaa07a700 1 --2- 172.21.15.36:0/4199260152 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feda4147bd0 0x7feda4159ec0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:07.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.409+0000 7fedaa07a700 1 --2- 172.21.15.36:0/4199260152 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feda40b27b0 0x7feda41599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:07.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.409+0000 7fedaa07a700 1 --2- 172.21.15.36:0/4199260152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feda4060ca0 0x7feda40a7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:07.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.409+0000 7fedaa07a700 1 -- 172.21.15.36:0/4199260152 >> 172.21.15.36:0/4199260152 conn(0x7feda40b5d10 msgr2=0x7feda40b3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:07.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.409+0000 7fedaa07a700 1 -- 172.21.15.36:0/4199260152 shutdown_connections 2022-06-17T12:47:07.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.409+0000 7fedaa07a700 1 -- 172.21.15.36:0/4199260152 wait complete. 2022-06-17T12:47:07.453 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:47:07.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc338889700 1 Processor -- start 2022-06-17T12:47:07.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc338889700 1 -- start start 2022-06-17T12:47:07.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc338889700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc3340ae270 0x7fc3340b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:07.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc338889700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc3340ad9c0 0x7fc3340b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:07.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc338889700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3340afc30 con 0x7fc33413fde0 2022-06-17T12:47:07.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc338889700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3340b3eb0 con 0x7fc3340ae270 2022-06-17T12:47:07.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc338889700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3340ae650 con 0x7fc3340ad9c0 2022-06-17T12:47:07.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc33259c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc3340ae270 0x7fc3340b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:07.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc33259c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc3340ae270 0x7fc3340b39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59964/0 (socket says 172.21.15.36:59964) 2022-06-17T12:47:07.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc33259c700 1 -- 172.21.15.36:0/738284016 learned_addr learned my addr 172.21.15.36:0/738284016 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:07.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc331d9b700 1 --2- 172.21.15.36:0/738284016 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc3340ad9c0 0x7fc3340b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:07.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc33259c700 1 -- 172.21.15.36:0/738284016 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc3340ad9c0 msgr2=0x7fc3340b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:07.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc33259c700 1 --2- 172.21.15.36:0/738284016 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc3340ad9c0 0x7fc3340b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:07.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc33259c700 1 -- 172.21.15.36:0/738284016 >> v1:172.21.15.70:6789/0 conn(0x7fc33413fde0 legacy=0x7fc3341433b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:07.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc33259c700 1 -- 172.21.15.36:0/738284016 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3340afe60 con 0x7fc3340ae270 2022-06-17T12:47:07.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc331d9b700 1 --2- 172.21.15.36:0/738284016 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc3340ad9c0 0x7fc3340b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:07.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc33259c700 1 --2- 172.21.15.36:0/738284016 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc3340ae270 0x7fc3340b39c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fc31c002f70 tx=0x7fc31c006f40).ready entity=mon.0 client_cookie=c96f6d06bf03367d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:07.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc33159a700 1 -- 172.21.15.36:0/738284016 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc31c00dc70 con 0x7fc3340ae270 2022-06-17T12:47:07.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc33159a700 1 -- 172.21.15.36:0/738284016 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31c000e90 con 0x7fc3340ae270 2022-06-17T12:47:07.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc33159a700 1 -- 172.21.15.36:0/738284016 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc31c00be90 con 0x7fc3340ae270 2022-06-17T12:47:07.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc338889700 1 -- 172.21.15.36:0/738284016 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc3340ae270 msgr2=0x7fc3340b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:07.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc338889700 1 --2- 172.21.15.36:0/738284016 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc3340ae270 0x7fc3340b39c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fc31c002f70 tx=0x7fc31c006f40).stop 2022-06-17T12:47:07.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc338889700 1 -- 172.21.15.36:0/738284016 shutdown_connections 2022-06-17T12:47:07.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc338889700 1 --2- 172.21.15.36:0/738284016 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc3340ad9c0 0x7fc3340b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:07.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc338889700 1 --2- 172.21.15.36:0/738284016 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc3340ae270 0x7fc3340b39c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:07.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc338889700 1 -- 172.21.15.36:0/738284016 >> 172.21.15.36:0/738284016 conn(0x7fc33405c550 msgr2=0x7fc33405c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:07.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc338889700 1 -- 172.21.15.36:0/738284016 shutdown_connections 2022-06-17T12:47:07.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.597+0000 7fc338889700 1 -- 172.21.15.36:0/738284016 wait complete. 2022-06-17T12:47:07.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc338889700 1 Processor -- start 2022-06-17T12:47:07.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc338889700 1 -- start start 2022-06-17T12:47:07.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc338889700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc3340ad9c0 0x7fc3340670c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:07.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc338889700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3340ae270 0x7fc334152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:07.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc338889700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc33413fde0 0x7fc3341447d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:07.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc338889700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc334153310 con 0x7fc3340ad9c0 2022-06-17T12:47:07.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc338889700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33413e510 con 0x7fc3340ae270 2022-06-17T12:47:07.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc338889700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc33413e7b0 con 0x7fc33413fde0 2022-06-17T12:47:07.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc33259c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc3340ad9c0 0x7fc3340670c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:07.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc33259c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc3340ad9c0 0x7fc3340670c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59970/0 (socket says 172.21.15.36:59970) 2022-06-17T12:47:07.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc33259c700 1 -- 172.21.15.36:0/54798939 learned_addr learned my addr 172.21.15.36:0/54798939 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:07.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc332d9d700 1 --2- 172.21.15.36:0/54798939 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc33413fde0 0x7fc3341447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:07.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc331d9b700 1 --2- 172.21.15.36:0/54798939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3340ae270 0x7fc334152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:07.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc33259c700 1 -- 172.21.15.36:0/54798939 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc33413fde0 msgr2=0x7fc3341447d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:07.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc33259c700 1 --2- 172.21.15.36:0/54798939 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc33413fde0 0x7fc3341447d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:07.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc33259c700 1 -- 172.21.15.36:0/54798939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3340ae270 msgr2=0x7fc334152330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:07.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc33259c700 1 --2- 172.21.15.36:0/54798939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3340ae270 0x7fc334152330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:07.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc33259c700 1 -- 172.21.15.36:0/54798939 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31c00b460 con 0x7fc3340ad9c0 2022-06-17T12:47:07.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc332d9d700 1 --2- 172.21.15.36:0/54798939 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc33413fde0 0x7fc3341447d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:07.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc33259c700 1 --2- 172.21.15.36:0/54798939 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc3340ad9c0 0x7fc3340670c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fc31c002f70 tx=0x7fc33406a320).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:07.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc32b7fe700 1 -- 172.21.15.36:0/54798939 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc31c01d070 con 0x7fc3340ad9c0 2022-06-17T12:47:07.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc32b7fe700 1 -- 172.21.15.36:0/54798939 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31c022410 con 0x7fc3340ad9c0 2022-06-17T12:47:07.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc338889700 1 -- 172.21.15.36:0/54798939 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc33413e190 con 0x7fc3340ad9c0 2022-06-17T12:47:07.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc32b7fe700 1 -- 172.21.15.36:0/54798939 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc31c022770 con 0x7fc3340ad9c0 2022-06-17T12:47:07.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.601+0000 7fc338889700 1 -- 172.21.15.36:0/54798939 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc33413be00 con 0x7fc3340ad9c0 2022-06-17T12:47:07.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.605+0000 7fc338889700 1 -- 172.21.15.36:0/54798939 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc33413e970 con 0x7fc3340ad9c0 2022-06-17T12:47:07.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.605+0000 7fc32b7fe700 1 -- 172.21.15.36:0/54798939 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fc31c029070 con 0x7fc3340ad9c0 2022-06-17T12:47:07.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.605+0000 7fc32b7fe700 1 --2- 172.21.15.36:0/54798939 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc318053ad0 0x7fc318055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:07.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.605+0000 7fc32b7fe700 1 -- 172.21.15.36:0/54798939 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc31c06d3b0 con 0x7fc3340ad9c0 2022-06-17T12:47:07.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.605+0000 7fc331d9b700 1 --2- 172.21.15.36:0/54798939 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc318053ad0 0x7fc318055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:07.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.605+0000 7fc331d9b700 1 --2- 172.21.15.36:0/54798939 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc318053ad0 0x7fc318055f90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fc32400c000 tx=0x7fc324008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:07.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.609+0000 7fc32b7fe700 1 -- 172.21.15.36:0/54798939 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc31c0441f0 con 0x7fc3340ad9c0 2022-06-17T12:47:07.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.777+0000 7fc338889700 1 -- 172.21.15.36:0/54798939 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc33413e380 con 0x7fc3340ad9c0 2022-06-17T12:47:07.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.781+0000 7fc32b7fe700 1 -- 172.21.15.36:0/54798939 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2134 (secure 0 0 0) 0x7fc31c0477d0 con 0x7fc3340ad9c0 2022-06-17T12:47:07.775 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:07.776 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/4243909763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":4243909763},{"type":"v1","addr":"172.21.15.36:6835","nonce":4243909763}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:46:52.253609+0000","modified":"2022-06-17T12:46:54.268223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/3425584732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3425584732},{"type":"v1","addr":"172.21.15.70:6833","nonce":3425584732}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-06-17T12:47:07.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.781+0000 7fc338889700 1 -- 172.21.15.36:0/54798939 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc318053ad0 msgr2=0x7fc318055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:07.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.781+0000 7fc338889700 1 --2- 172.21.15.36:0/54798939 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc318053ad0 0x7fc318055f90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fc32400c000 tx=0x7fc324008040).stop 2022-06-17T12:47:07.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.781+0000 7fc338889700 1 -- 172.21.15.36:0/54798939 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc3340ad9c0 msgr2=0x7fc3340670c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:07.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.781+0000 7fc338889700 1 --2- 172.21.15.36:0/54798939 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc3340ad9c0 0x7fc3340670c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fc31c002f70 tx=0x7fc33406a320).stop 2022-06-17T12:47:07.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.781+0000 7fc338889700 1 -- 172.21.15.36:0/54798939 shutdown_connections 2022-06-17T12:47:07.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.781+0000 7fc338889700 1 --2- 172.21.15.36:0/54798939 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc318053ad0 0x7fc318055f90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:07.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.781+0000 7fc338889700 1 --2- 172.21.15.36:0/54798939 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc33413fde0 0x7fc3341447d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:07.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.781+0000 7fc338889700 1 --2- 172.21.15.36:0/54798939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3340ae270 0x7fc334152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:07.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.781+0000 7fc338889700 1 --2- 172.21.15.36:0/54798939 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc3340ad9c0 0x7fc3340670c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:07.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.781+0000 7fc338889700 1 -- 172.21.15.36:0/54798939 >> 172.21.15.36:0/54798939 conn(0x7fc33405c550 msgr2=0x7fc3340b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:07.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.781+0000 7fc338889700 1 -- 172.21.15.36:0/54798939 shutdown_connections 2022-06-17T12:47:07.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:07.781+0000 7fc338889700 1 -- 172.21.15.36:0/54798939 wait complete. 2022-06-17T12:47:07.823 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 7 2022-06-17T12:47:08.050 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:46:52.253609+0000', 'modified': '2022-06-17T12:46:54.268223+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4223}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4223': {'gid': 4223, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6833/3425584732', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 3425584732}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 3425584732}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:47:08.050 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:47:08.051 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-17T12:47:08.165 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":4223,"want_state":"up:active","state":"up:active","rank_uptime":15.911089882000001,"mdsmap_epoch":7,"osdmap_epoch":26,"osdmap_epoch_barrier":26,"uptime":22.359144971999999} 2022-06-17T12:47:08.166 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 4223, "mdsmap_epoch": 7, "osdmap_epoch": 26, "osdmap_epoch_barrier": 26, "rank_uptime": 15.911089882, "state": "up:active", "uptime": 22.359144972, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:47:08.166 INFO:teuthology.run_tasks:Running task ceph-fuse... 2022-06-17T12:47:08.250 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2022-06-17T12:47:08.251 INFO:tasks.ceph_fuse:config is None 2022-06-17T12:47:08.251 INFO:tasks.ceph_fuse:new config is {'client.0': None, 'client.1': None, 'client.2': None} 2022-06-17T12:47:08.252 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2022-06-17T12:47:08.253 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:47:08.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.413+0000 7f112f6ce700 1 Processor -- start 2022-06-17T12:47:08.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.413+0000 7f112f6ce700 1 -- start start 2022-06-17T12:47:08.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112f6ce700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1128062770 0x7f1128062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:08.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112f6ce700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f11281444f0 0x7f1128144960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:08.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112f6ce700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11280ad330 0x7f11280ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:08.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112f6ce700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11280ae820 con 0x7f11281444f0 2022-06-17T12:47:08.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112f6ce700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1128061030 con 0x7f1128062770 2022-06-17T12:47:08.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112f6ce700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11280b0ba0 con 0x7f11280ad330 2022-06-17T12:47:08.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112cc69700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f11281444f0 0x7f1128144960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:08.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112cc69700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f11281444f0 0x7f1128144960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59982/0 (socket says 172.21.15.36:59982) 2022-06-17T12:47:08.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112cc69700 1 -- 172.21.15.36:0/4014962936 learned_addr learned my addr 172.21.15.36:0/4014962936 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:08.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112dc6b700 1 --2- 172.21.15.36:0/4014962936 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11280ad330 0x7f11280ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:08.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112d46a700 1 --2- 172.21.15.36:0/4014962936 >> v2:172.21.15.70:3300/0 conn(0x7f1128062770 0x7f1128062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:08.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112cc69700 1 -- 172.21.15.36:0/4014962936 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11280ad330 msgr2=0x7f11280ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:08.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112cc69700 1 --2- 172.21.15.36:0/4014962936 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11280ad330 0x7f11280ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:08.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112cc69700 1 -- 172.21.15.36:0/4014962936 >> v2:172.21.15.70:3300/0 conn(0x7f1128062770 msgr2=0x7f1128062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:08.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112cc69700 1 --2- 172.21.15.36:0/4014962936 >> v2:172.21.15.70:3300/0 conn(0x7f1128062770 0x7f1128062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:08.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112cc69700 1 -- 172.21.15.36:0/4014962936 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1128147e90 con 0x7f11281444f0 2022-06-17T12:47:08.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112cc69700 1 --2- 172.21.15.36:0/4014962936 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11281444f0 0x7f1128144960 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f1128147470 tx=0x7f111c006eb0).ready entity=mon.0 client_cookie=b7aa50303e46c91b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:08.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f1127fff700 1 -- 172.21.15.36:0/4014962936 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f111c00d570 con 0x7f11281444f0 2022-06-17T12:47:08.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f1127fff700 1 -- 172.21.15.36:0/4014962936 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111c000e90 con 0x7f11281444f0 2022-06-17T12:47:08.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f1127fff700 1 -- 172.21.15.36:0/4014962936 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f111c00d900 con 0x7f11281444f0 2022-06-17T12:47:08.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112f6ce700 1 -- 172.21.15.36:0/4014962936 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11281444f0 msgr2=0x7f1128144960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:08.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112f6ce700 1 --2- 172.21.15.36:0/4014962936 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11281444f0 0x7f1128144960 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f1128147470 tx=0x7f111c006eb0).stop 2022-06-17T12:47:08.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112f6ce700 1 -- 172.21.15.36:0/4014962936 shutdown_connections 2022-06-17T12:47:08.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112f6ce700 1 --2- 172.21.15.36:0/4014962936 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11280ad330 0x7f11280ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:08.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112f6ce700 1 --2- 172.21.15.36:0/4014962936 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11281444f0 0x7f1128144960 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:08.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112f6ce700 1 --2- 172.21.15.36:0/4014962936 >> v2:172.21.15.70:3300/0 conn(0x7f1128062770 0x7f1128062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:08.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112f6ce700 1 -- 172.21.15.36:0/4014962936 >> 172.21.15.36:0/4014962936 conn(0x7f11280b53e0 msgr2=0x7f11280b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:08.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112f6ce700 1 -- 172.21.15.36:0/4014962936 shutdown_connections 2022-06-17T12:47:08.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.417+0000 7f112f6ce700 1 -- 172.21.15.36:0/4014962936 wait complete. 2022-06-17T12:47:08.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f112f6ce700 1 Processor -- start 2022-06-17T12:47:08.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f112f6ce700 1 -- start start 2022-06-17T12:47:08.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f112f6ce700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1128062770 0x7f1128066c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:08.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f112f6ce700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11280ad330 0x7f1128067120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:08.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f112f6ce700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11281444f0 0x7f1128159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:08.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f112f6ce700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112814b8b0 con 0x7f11280ad330 2022-06-17T12:47:08.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f112f6ce700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112814a7e0 con 0x7f1128062770 2022-06-17T12:47:08.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f112f6ce700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112814aa80 con 0x7f11281444f0 2022-06-17T12:47:08.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f112cc69700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11280ad330 0x7f1128067120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:08.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f112dc6b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11281444f0 0x7f1128159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:08.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f112d46a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1128062770 0x7f1128066c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:08.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f112dc6b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11281444f0 0x7f1128159ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:46546/0 (socket says 172.21.15.36:46546) 2022-06-17T12:47:08.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f112dc6b700 1 -- 172.21.15.36:0/3796131359 learned_addr learned my addr 172.21.15.36:0/3796131359 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:08.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f112cc69700 1 -- 172.21.15.36:0/3796131359 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11281444f0 msgr2=0x7f1128159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:08.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f112cc69700 1 --2- 172.21.15.36:0/3796131359 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11281444f0 0x7f1128159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:08.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f112cc69700 1 -- 172.21.15.36:0/3796131359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1128062770 msgr2=0x7f1128066c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:08.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f112cc69700 1 --2- 172.21.15.36:0/3796131359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1128062770 0x7f1128066c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:08.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f112cc69700 1 -- 172.21.15.36:0/3796131359 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c00b1e0 con 0x7f11280ad330 2022-06-17T12:47:08.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f112dc6b700 1 --2- 172.21.15.36:0/3796131359 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11281444f0 0x7f1128159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:08.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f112cc69700 1 --2- 172.21.15.36:0/3796131359 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11280ad330 0x7f1128067120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f1128147470 tx=0x7f111c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:08.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f11267fc700 1 -- 172.21.15.36:0/3796131359 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f111c021070 con 0x7f11280ad330 2022-06-17T12:47:08.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f11267fc700 1 -- 172.21.15.36:0/3796131359 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111c002f70 con 0x7f11280ad330 2022-06-17T12:47:08.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f112f6ce700 1 -- 172.21.15.36:0/3796131359 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f112814ad20 con 0x7f11280ad330 2022-06-17T12:47:08.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f11267fc700 1 -- 172.21.15.36:0/3796131359 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f111c00dea0 con 0x7f11280ad330 2022-06-17T12:47:08.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.421+0000 7f112f6ce700 1 -- 172.21.15.36:0/3796131359 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1128066510 con 0x7f11280ad330 2022-06-17T12:47:08.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.425+0000 7f112f6ce700 1 -- 172.21.15.36:0/3796131359 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1128144e50 con 0x7f11280ad330 2022-06-17T12:47:08.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.425+0000 7f11267fc700 1 -- 172.21.15.36:0/3796131359 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f111c00f070 con 0x7f11280ad330 2022-06-17T12:47:08.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.425+0000 7f11267fc700 1 --2- 172.21.15.36:0/3796131359 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1108053a40 0x7f1108055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:08.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.425+0000 7f11267fc700 1 -- 172.21.15.36:0/3796131359 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f111c06d010 con 0x7f11280ad330 2022-06-17T12:47:08.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.425+0000 7f112d46a700 1 --2- 172.21.15.36:0/3796131359 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1108053a40 0x7f1108055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:08.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.425+0000 7f112d46a700 1 --2- 172.21.15.36:0/3796131359 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1108053a40 0x7f1108055f00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f111800d000 tx=0x7f1118008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:08.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.433+0000 7f11267fc700 1 -- 172.21.15.36:0/3796131359 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f111c043e20 con 0x7f11280ad330 2022-06-17T12:47:08.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.557+0000 7f112f6ce700 1 -- 172.21.15.36:0/3796131359 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1128057410 con 0x7f11280ad330 2022-06-17T12:47:08.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.561+0000 7f11267fc700 1 -- 172.21.15.36:0/3796131359 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f111c047440 con 0x7f11280ad330 2022-06-17T12:47:08.702 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:08.703 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":26,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:46:53.258075+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:46:47.465328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-17T12:46:49.893338+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:47:08.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.561+0000 7f112f6ce700 1 -- 172.21.15.36:0/3796131359 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1108053a40 msgr2=0x7f1108055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:08.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.561+0000 7f112f6ce700 1 --2- 172.21.15.36:0/3796131359 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1108053a40 0x7f1108055f00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f111800d000 tx=0x7f1118008040).stop 2022-06-17T12:47:08.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.561+0000 7f112f6ce700 1 -- 172.21.15.36:0/3796131359 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11280ad330 msgr2=0x7f1128067120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:08.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.561+0000 7f112f6ce700 1 --2- 172.21.15.36:0/3796131359 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11280ad330 0x7f1128067120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f1128147470 tx=0x7f111c015000).stop 2022-06-17T12:47:08.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.561+0000 7f112f6ce700 1 -- 172.21.15.36:0/3796131359 shutdown_connections 2022-06-17T12:47:08.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.561+0000 7f112f6ce700 1 --2- 172.21.15.36:0/3796131359 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1108053a40 0x7f1108055f00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:08.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.561+0000 7f112f6ce700 1 --2- 172.21.15.36:0/3796131359 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11281444f0 0x7f1128159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:08.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.561+0000 7f112f6ce700 1 --2- 172.21.15.36:0/3796131359 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11280ad330 0x7f1128067120 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:08.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.561+0000 7f112f6ce700 1 --2- 172.21.15.36:0/3796131359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1128062770 0x7f1128066c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:08.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.561+0000 7f112f6ce700 1 -- 172.21.15.36:0/3796131359 >> 172.21.15.36:0/3796131359 conn(0x7f11280b53e0 msgr2=0x7f11280b4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:08.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.561+0000 7f112f6ce700 1 -- 172.21.15.36:0/3796131359 shutdown_connections 2022-06-17T12:47:08.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.561+0000 7f112f6ce700 1 -- 172.21.15.36:0/3796131359 wait complete. 2022-06-17T12:47:08.786 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:46:47.465328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:46:49.893338+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:47:08.788 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:47:09.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.921+0000 7f267c950700 1 Processor -- start 2022-06-17T12:47:09.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.921+0000 7f267c950700 1 -- start start 2022-06-17T12:47:09.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.921+0000 7f267c950700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2678060020 0x7f26780626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:09.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.921+0000 7f267c950700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f26780b3f80 0x7f26780b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:09.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.921+0000 7f267c950700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26781432c0 con 0x7f26780b4b90 2022-06-17T12:47:09.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.921+0000 7f267c950700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2678143df0 con 0x7f26780b3f80 2022-06-17T12:47:09.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.921+0000 7f267c950700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2678061f60 con 0x7f2678060020 2022-06-17T12:47:09.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.921+0000 7f267759e700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f26780b4b90 0x7f267813ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49294/0 (socket says 172.21.15.36:49294) 2022-06-17T12:47:09.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.921+0000 7f267659c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f26780b3f80 0x7f26780b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:09.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.921+0000 7f267759e700 1 -- 172.21.15.36:0/1654412878 learned_addr learned my addr 172.21.15.36:0/1654412878 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:09.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.925+0000 7f2676d9d700 1 --2- 172.21.15.36:0/1654412878 >> v2:172.21.15.70:3300/0 conn(0x7f2678060020 0x7f26780626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:09.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.925+0000 7f2675d9b700 1 -- 172.21.15.36:0/1654412878 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 447872613 0 0) 0x7f26781432c0 con 0x7f26780b4b90 2022-06-17T12:47:09.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.925+0000 7f2675d9b700 1 -- 172.21.15.36:0/1654412878 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2658003260 con 0x7f26780b4b90 2022-06-17T12:47:09.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.925+0000 7f267659c700 1 -- 172.21.15.36:0/1654412878 >> v2:172.21.15.70:3300/0 conn(0x7f2678060020 msgr2=0x7f26780626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:09.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.925+0000 7f267659c700 1 --2- 172.21.15.36:0/1654412878 >> v2:172.21.15.70:3300/0 conn(0x7f2678060020 0x7f26780626a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:09.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.925+0000 7f267659c700 1 -- 172.21.15.36:0/1654412878 >> v1:172.21.15.36:6789/0 conn(0x7f26780b4b90 legacy=0x7f267813ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:09.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.925+0000 7f267659c700 1 -- 172.21.15.36:0/1654412878 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2678062200 con 0x7f26780b3f80 2022-06-17T12:47:09.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.925+0000 7f267659c700 1 --2- 172.21.15.36:0/1654412878 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26780b3f80 0x7f26780b46a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f2660002f70 tx=0x7f2660006df0).ready entity=mon.0 client_cookie=e0f70f44f43e3031 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:09.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.925+0000 7f2675d9b700 1 -- 172.21.15.36:0/1654412878 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f266000d570 con 0x7f26780b3f80 2022-06-17T12:47:09.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.925+0000 7f2675d9b700 1 -- 172.21.15.36:0/1654412878 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2660000e90 con 0x7f26780b3f80 2022-06-17T12:47:09.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.925+0000 7f2675d9b700 1 -- 172.21.15.36:0/1654412878 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f266000d8c0 con 0x7f26780b3f80 2022-06-17T12:47:09.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.925+0000 7f267c950700 1 -- 172.21.15.36:0/1654412878 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26780b3f80 msgr2=0x7f26780b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:09.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.925+0000 7f267c950700 1 --2- 172.21.15.36:0/1654412878 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26780b3f80 0x7f26780b46a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f2660002f70 tx=0x7f2660006df0).stop 2022-06-17T12:47:09.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.925+0000 7f267c950700 1 -- 172.21.15.36:0/1654412878 shutdown_connections 2022-06-17T12:47:09.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.925+0000 7f267c950700 1 --2- 172.21.15.36:0/1654412878 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26780b3f80 0x7f26780b46a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:09.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.925+0000 7f267c950700 1 --2- 172.21.15.36:0/1654412878 >> v2:172.21.15.70:3300/0 conn(0x7f2678060020 0x7f26780626a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:09.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.925+0000 7f267c950700 1 -- 172.21.15.36:0/1654412878 >> 172.21.15.36:0/1654412878 conn(0x7f267805c550 msgr2=0x7f267805f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:09.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.925+0000 7f267c950700 1 -- 172.21.15.36:0/1654412878 shutdown_connections 2022-06-17T12:47:09.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.925+0000 7f267c950700 1 -- 172.21.15.36:0/1654412878 wait complete. 2022-06-17T12:47:09.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.925+0000 7f267c950700 1 Processor -- start 2022-06-17T12:47:09.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.925+0000 7f267c950700 1 -- start start 2022-06-17T12:47:09.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f267c950700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2678060020 0x7f2678146f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:09.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f267c950700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26780b3f80 0x7f2678141d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:09.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f267c950700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26780b4b90 0x7f26780ad330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:09.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f267c950700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26780ae3d0 con 0x7f26780b4b90 2022-06-17T12:47:09.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f267c950700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2678144440 con 0x7f26780b3f80 2022-06-17T12:47:09.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f267c950700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26781446e0 con 0x7f2678060020 2022-06-17T12:47:09.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f267759e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26780b4b90 0x7f26780ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:09.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f267759e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26780b4b90 0x7f26780ad330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:59998/0 (socket says 172.21.15.36:59998) 2022-06-17T12:47:09.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f267759e700 1 -- 172.21.15.36:0/410535022 learned_addr learned my addr 172.21.15.36:0/410535022 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:09.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f267759e700 1 -- 172.21.15.36:0/410535022 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2678060020 msgr2=0x7f2678146f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:09.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f2676d9d700 1 --2- 172.21.15.36:0/410535022 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2678060020 0x7f2678146f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:09.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f267659c700 1 --2- 172.21.15.36:0/410535022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26780b3f80 0x7f2678141d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:09.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f267759e700 1 --2- 172.21.15.36:0/410535022 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2678060020 0x7f2678146f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:09.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f267759e700 1 -- 172.21.15.36:0/410535022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26780b3f80 msgr2=0x7f2678141d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:09.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f267759e700 1 --2- 172.21.15.36:0/410535022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26780b3f80 0x7f2678141d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:09.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f267759e700 1 -- 172.21.15.36:0/410535022 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266000b180 con 0x7f26780b4b90 2022-06-17T12:47:09.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f2676d9d700 1 --2- 172.21.15.36:0/410535022 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2678060020 0x7f2678146f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:47:09.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f267659c700 1 --2- 172.21.15.36:0/410535022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26780b3f80 0x7f2678141d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:47:09.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f267759e700 1 --2- 172.21.15.36:0/410535022 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26780b4b90 0x7f26780ad330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f2670001490 tx=0x7f2670007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:09.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f266ffff700 1 -- 172.21.15.36:0/410535022 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2670005ba0 con 0x7f26780b4b90 2022-06-17T12:47:09.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f266ffff700 1 -- 172.21.15.36:0/410535022 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2670005d00 con 0x7f26780b4b90 2022-06-17T12:47:09.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f267c950700 1 -- 172.21.15.36:0/410535022 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2678144f50 con 0x7f26780b4b90 2022-06-17T12:47:09.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f266ffff700 1 -- 172.21.15.36:0/410535022 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26700072c0 con 0x7f26780b4b90 2022-06-17T12:47:09.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f267c950700 1 -- 172.21.15.36:0/410535022 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26780aff90 con 0x7f26780b4b90 2022-06-17T12:47:09.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f267c950700 1 -- 172.21.15.36:0/410535022 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26781448a0 con 0x7f26780b4b90 2022-06-17T12:47:09.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.929+0000 7f266ffff700 1 -- 172.21.15.36:0/410535022 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f267001a060 con 0x7f26780b4b90 2022-06-17T12:47:09.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.933+0000 7f266ffff700 1 --2- 172.21.15.36:0/410535022 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f265c053a40 0x7f265c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:09.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.933+0000 7f2676d9d700 1 --2- 172.21.15.36:0/410535022 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f265c053a40 0x7f265c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:09.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.933+0000 7f266ffff700 1 -- 172.21.15.36:0/410535022 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f267006bab0 con 0x7f26780b4b90 2022-06-17T12:47:09.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.933+0000 7f2676d9d700 1 --2- 172.21.15.36:0/410535022 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f265c053a40 0x7f265c055f00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f2668008000 tx=0x7f2668008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:09.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:08.937+0000 7f266ffff700 1 -- 172.21.15.36:0/410535022 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f26700428b0 con 0x7f26780b4b90 2022-06-17T12:47:09.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.065+0000 7f267c950700 1 -- 172.21.15.36:0/410535022 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f2678057410 con 0x7f26780b4b90 2022-06-17T12:47:09.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.065+0000 7f266ffff700 1 -- 172.21.15.36:0/410535022 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2670018030 con 0x7f26780b4b90 2022-06-17T12:47:09.095 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:47:09.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.069+0000 7f267c950700 1 -- 172.21.15.36:0/410535022 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f265c053a40 msgr2=0x7f265c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:09.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.069+0000 7f267c950700 1 --2- 172.21.15.36:0/410535022 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f265c053a40 0x7f265c055f00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f2668008000 tx=0x7f2668008040).stop 2022-06-17T12:47:09.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.069+0000 7f267c950700 1 -- 172.21.15.36:0/410535022 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26780b4b90 msgr2=0x7f26780ad330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:09.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.069+0000 7f267c950700 1 --2- 172.21.15.36:0/410535022 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26780b4b90 0x7f26780ad330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f2670001490 tx=0x7f2670007e00).stop 2022-06-17T12:47:09.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.069+0000 7f267c950700 1 -- 172.21.15.36:0/410535022 shutdown_connections 2022-06-17T12:47:09.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.069+0000 7f267c950700 1 --2- 172.21.15.36:0/410535022 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f265c053a40 0x7f265c055f00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:09.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.069+0000 7f267c950700 1 --2- 172.21.15.36:0/410535022 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26780b4b90 0x7f26780ad330 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:09.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.069+0000 7f267c950700 1 --2- 172.21.15.36:0/410535022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26780b3f80 0x7f2678141d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:09.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.069+0000 7f267c950700 1 --2- 172.21.15.36:0/410535022 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2678060020 0x7f2678146f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:09.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.069+0000 7f267c950700 1 -- 172.21.15.36:0/410535022 >> 172.21.15.36:0/410535022 conn(0x7f267805c550 msgr2=0x7f26780b26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:09.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.069+0000 7f267c950700 1 -- 172.21.15.36:0/410535022 shutdown_connections 2022-06-17T12:47:09.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.069+0000 7f267c950700 1 -- 172.21.15.36:0/410535022 wait complete. 2022-06-17T12:47:09.126 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:47:09.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.257+0000 7f8b45c24700 1 Processor -- start 2022-06-17T12:47:09.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.257+0000 7f8b45c24700 1 -- start start 2022-06-17T12:47:09.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.257+0000 7f8b45c24700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b400b4190 0x7f8b400b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:09.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.257+0000 7f8b45c24700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b40143130 con 0x7f8b40062270 2022-06-17T12:47:09.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.257+0000 7f8b45c24700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b40062650 con 0x7f8b40143d40 2022-06-17T12:47:09.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.257+0000 7f8b45c24700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b400b6080 con 0x7f8b400b4190 2022-06-17T12:47:09.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.257+0000 7f8b3f7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f8b40062270 0x7f8b400b48b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49304/0 (socket says 172.21.15.36:49304) 2022-06-17T12:47:09.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.257+0000 7f8b3f7fe700 1 -- 172.21.15.36:0/4034362983 learned_addr learned my addr 172.21.15.36:0/4034362983 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:09.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.261+0000 7f8b3effd700 1 --2- 172.21.15.36:0/4034362983 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b400b4190 0x7f8b400b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:09.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.261+0000 7f8b3e7fc700 1 -- 172.21.15.36:0/4034362983 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1754561861 0 0) 0x7f8b40062650 con 0x7f8b40143d40 2022-06-17T12:47:09.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.261+0000 7f8b3e7fc700 1 -- 172.21.15.36:0/4034362983 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b2c003260 con 0x7f8b40143d40 2022-06-17T12:47:09.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.261+0000 7f8b3e7fc700 1 -- 172.21.15.36:0/4034362983 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 622757580 0 0) 0x7f8b40143130 con 0x7f8b40062270 2022-06-17T12:47:09.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.261+0000 7f8b3effd700 1 -- 172.21.15.36:0/4034362983 >> v1:172.21.15.70:6789/0 conn(0x7f8b40143d40 legacy=0x7f8b401400f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:09.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.261+0000 7f8b3effd700 1 -- 172.21.15.36:0/4034362983 >> v1:172.21.15.36:6789/0 conn(0x7f8b40062270 legacy=0x7f8b400b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:09.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.261+0000 7f8b3effd700 1 -- 172.21.15.36:0/4034362983 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b40143360 con 0x7f8b400b4190 2022-06-17T12:47:09.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.261+0000 7f8b3effd700 1 --2- 172.21.15.36:0/4034362983 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b400b4190 0x7f8b400b4fc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f8b34002f70 tx=0x7f8b34006df0).ready entity=mon.2 client_cookie=66bb320f7cbf2c7 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:09.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.261+0000 7f8b3e7fc700 1 -- 172.21.15.36:0/4034362983 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b3400d570 con 0x7f8b400b4190 2022-06-17T12:47:09.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.261+0000 7f8b3e7fc700 1 -- 172.21.15.36:0/4034362983 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b34000e90 con 0x7f8b400b4190 2022-06-17T12:47:09.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.261+0000 7f8b3e7fc700 1 -- 172.21.15.36:0/4034362983 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b3400d8c0 con 0x7f8b400b4190 2022-06-17T12:47:09.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.261+0000 7f8b45c24700 1 -- 172.21.15.36:0/4034362983 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b400b4190 msgr2=0x7f8b400b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:09.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.261+0000 7f8b45c24700 1 --2- 172.21.15.36:0/4034362983 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b400b4190 0x7f8b400b4fc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f8b34002f70 tx=0x7f8b34006df0).stop 2022-06-17T12:47:09.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.261+0000 7f8b45c24700 1 -- 172.21.15.36:0/4034362983 shutdown_connections 2022-06-17T12:47:09.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.261+0000 7f8b45c24700 1 --2- 172.21.15.36:0/4034362983 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b400b4190 0x7f8b400b4fc0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:09.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.261+0000 7f8b45c24700 1 -- 172.21.15.36:0/4034362983 >> 172.21.15.36:0/4034362983 conn(0x7f8b400b97f0 msgr2=0x7f8b400b9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:09.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.261+0000 7f8b45c24700 1 -- 172.21.15.36:0/4034362983 shutdown_connections 2022-06-17T12:47:09.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.261+0000 7f8b45c24700 1 -- 172.21.15.36:0/4034362983 wait complete. 2022-06-17T12:47:09.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.265+0000 7f8b45c24700 1 Processor -- start 2022-06-17T12:47:09.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.265+0000 7f8b45c24700 1 -- start start 2022-06-17T12:47:09.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.265+0000 7f8b45c24700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b40062270 0x7f8b401280e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:09.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.265+0000 7f8b45c24700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b400b4190 0x7f8b4012c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:09.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.265+0000 7f8b45c24700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b40143d40 0x7f8b40126380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:09.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.265+0000 7f8b45c24700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b40127650 con 0x7f8b400b4190 2022-06-17T12:47:09.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.265+0000 7f8b45c24700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b40128fc0 con 0x7f8b40143d40 2022-06-17T12:47:09.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.265+0000 7f8b45c24700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b40155920 con 0x7f8b40062270 2022-06-17T12:47:09.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.265+0000 7f8b3effd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b400b4190 0x7f8b4012c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:09.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.265+0000 7f8b3effd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b400b4190 0x7f8b4012c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60014/0 (socket says 172.21.15.36:60014) 2022-06-17T12:47:09.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.265+0000 7f8b3effd700 1 -- 172.21.15.36:0/666554465 learned_addr learned my addr 172.21.15.36:0/666554465 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:09.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.265+0000 7f8b3f7fe700 1 --2- 172.21.15.36:0/666554465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b40062270 0x7f8b401280e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:09.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.265+0000 7f8b3ffff700 1 --2- 172.21.15.36:0/666554465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b40143d40 0x7f8b40126380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:09.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.265+0000 7f8b3effd700 1 -- 172.21.15.36:0/666554465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b40062270 msgr2=0x7f8b401280e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:09.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.265+0000 7f8b3effd700 1 --2- 172.21.15.36:0/666554465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b40062270 0x7f8b401280e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:09.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.265+0000 7f8b3effd700 1 -- 172.21.15.36:0/666554465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b40143d40 msgr2=0x7f8b40126380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:09.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.265+0000 7f8b3effd700 1 --2- 172.21.15.36:0/666554465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b40143d40 0x7f8b40126380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:09.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.265+0000 7f8b3effd700 1 -- 172.21.15.36:0/666554465 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3400b180 con 0x7f8b400b4190 2022-06-17T12:47:09.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.265+0000 7f8b3effd700 1 --2- 172.21.15.36:0/666554465 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b400b4190 0x7f8b4012c260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f8b34002f70 tx=0x7f8b34014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:09.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.265+0000 7f8b3cff9700 1 -- 172.21.15.36:0/666554465 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b34013da0 con 0x7f8b400b4190 2022-06-17T12:47:09.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.265+0000 7f8b3cff9700 1 -- 172.21.15.36:0/666554465 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b3401e400 con 0x7f8b400b4190 2022-06-17T12:47:09.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.265+0000 7f8b45c24700 1 -- 172.21.15.36:0/666554465 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b40155e60 con 0x7f8b400b4190 2022-06-17T12:47:09.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.265+0000 7f8b3cff9700 1 -- 172.21.15.36:0/666554465 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b3401e760 con 0x7f8b400b4190 2022-06-17T12:47:09.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.265+0000 7f8b45c24700 1 -- 172.21.15.36:0/666554465 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b40156620 con 0x7f8b400b4190 2022-06-17T12:47:09.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.269+0000 7f8b45c24700 1 -- 172.21.15.36:0/666554465 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b40143600 con 0x7f8b400b4190 2022-06-17T12:47:09.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.269+0000 7f8b3cff9700 1 -- 172.21.15.36:0/666554465 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8b34025030 con 0x7f8b400b4190 2022-06-17T12:47:09.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.269+0000 7f8b3cff9700 1 --2- 172.21.15.36:0/666554465 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b20053920 0x7f8b20055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:09.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.269+0000 7f8b3cff9700 1 -- 172.21.15.36:0/666554465 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f8b3406cc90 con 0x7f8b400b4190 2022-06-17T12:47:09.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.269+0000 7f8b3f7fe700 1 --2- 172.21.15.36:0/666554465 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b20053920 0x7f8b20055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:09.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.269+0000 7f8b3f7fe700 1 --2- 172.21.15.36:0/666554465 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b20053920 0x7f8b20055de0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f8b3000d000 tx=0x7f8b30007a20).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:09.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.277+0000 7f8b3cff9700 1 -- 172.21.15.36:0/666554465 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8b34043a90 con 0x7f8b400b4190 2022-06-17T12:47:09.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.401+0000 7f8b45c24700 1 -- 172.21.15.36:0/666554465 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8b40057410 con 0x7f8b400b4190 2022-06-17T12:47:09.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.401+0000 7f8b3cff9700 1 -- 172.21.15.36:0/666554465 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8b340230a0 con 0x7f8b400b4190 2022-06-17T12:47:09.537 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:09.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.405+0000 7f8b45c24700 1 -- 172.21.15.36:0/666554465 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b20053920 msgr2=0x7f8b20055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:09.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.405+0000 7f8b45c24700 1 --2- 172.21.15.36:0/666554465 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b20053920 0x7f8b20055de0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f8b3000d000 tx=0x7f8b30007a20).stop 2022-06-17T12:47:09.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.405+0000 7f8b45c24700 1 -- 172.21.15.36:0/666554465 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b400b4190 msgr2=0x7f8b4012c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:09.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.405+0000 7f8b45c24700 1 --2- 172.21.15.36:0/666554465 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b400b4190 0x7f8b4012c260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f8b34002f70 tx=0x7f8b34014000).stop 2022-06-17T12:47:09.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.405+0000 7f8b45c24700 1 -- 172.21.15.36:0/666554465 shutdown_connections 2022-06-17T12:47:09.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.405+0000 7f8b45c24700 1 --2- 172.21.15.36:0/666554465 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b20053920 0x7f8b20055de0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:09.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.405+0000 7f8b45c24700 1 --2- 172.21.15.36:0/666554465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b40143d40 0x7f8b40126380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:09.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.405+0000 7f8b45c24700 1 --2- 172.21.15.36:0/666554465 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b400b4190 0x7f8b4012c260 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:09.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.405+0000 7f8b45c24700 1 --2- 172.21.15.36:0/666554465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b40062270 0x7f8b401280e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:09.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.405+0000 7f8b45c24700 1 -- 172.21.15.36:0/666554465 >> 172.21.15.36:0/666554465 conn(0x7f8b400b97f0 msgr2=0x7f8b400af8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:09.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.405+0000 7f8b45c24700 1 -- 172.21.15.36:0/666554465 shutdown_connections 2022-06-17T12:47:09.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.405+0000 7f8b45c24700 1 -- 172.21.15.36:0/666554465 wait complete. 2022-06-17T12:47:09.570 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:47:09.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.717+0000 7f6203d3b700 1 Processor -- start 2022-06-17T12:47:09.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.717+0000 7f6203d3b700 1 -- start start 2022-06-17T12:47:09.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.717+0000 7f6203d3b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f61fc0b4190 0x7f61fc0b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:09.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.717+0000 7f6203d3b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61fc14b780 0x7f61fc0b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:09.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.717+0000 7f6203d3b700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61fc062650 con 0x7f61fc062270 2022-06-17T12:47:09.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.717+0000 7f6203d3b700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61fc147810 con 0x7f61fc0b4190 2022-06-17T12:47:09.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.717+0000 7f6203d3b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc147ab0 con 0x7f61fc14b780 2022-06-17T12:47:09.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.717+0000 7f62012d6700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f61fc0b4190 0x7f61fc0b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:09.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.717+0000 7f62012d6700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f61fc0b4190 0x7f61fc0b48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60022/0 (socket says 172.21.15.36:60022) 2022-06-17T12:47:09.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.717+0000 7f62012d6700 1 -- 172.21.15.36:0/1611124964 learned_addr learned my addr 172.21.15.36:0/1611124964 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:09.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.717+0000 7f6201ad7700 1 --1- 172.21.15.36:0/1611124964 >> v1:172.21.15.36:6789/0 conn(0x7f61fc062270 0x7f61fc1481f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49318/0 (socket says 172.21.15.36:49318) 2022-06-17T12:47:09.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.717+0000 7f62022d8700 1 --2- 172.21.15.36:0/1611124964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61fc14b780 0x7f61fc0b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:09.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.717+0000 7f62012d6700 1 -- 172.21.15.36:0/1611124964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61fc14b780 msgr2=0x7f61fc0b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:09.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.717+0000 7f62012d6700 1 --2- 172.21.15.36:0/1611124964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61fc14b780 0x7f61fc0b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:09.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.717+0000 7f62012d6700 1 -- 172.21.15.36:0/1611124964 >> v1:172.21.15.36:6789/0 conn(0x7f61fc062270 legacy=0x7f61fc1481f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:09.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.717+0000 7f62012d6700 1 -- 172.21.15.36:0/1611124964 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc147d50 con 0x7f61fc0b4190 2022-06-17T12:47:09.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.717+0000 7f62012d6700 1 --2- 172.21.15.36:0/1611124964 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61fc0b4190 0x7f61fc0b48b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f61f0002f70 tx=0x7f61f0006eb0).ready entity=mon.0 client_cookie=14284d7ff34f3d54 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:09.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.717+0000 7f6200ad5700 1 -- 172.21.15.36:0/1611124964 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61f000d5c0 con 0x7f61fc0b4190 2022-06-17T12:47:09.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.717+0000 7f6200ad5700 1 -- 172.21.15.36:0/1611124964 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f0000e90 con 0x7f61fc0b4190 2022-06-17T12:47:09.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.717+0000 7f6200ad5700 1 -- 172.21.15.36:0/1611124964 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61f000d950 con 0x7f61fc0b4190 2022-06-17T12:47:09.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f6203d3b700 1 -- 172.21.15.36:0/1611124964 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61fc0b4190 msgr2=0x7f61fc0b48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:09.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f6203d3b700 1 --2- 172.21.15.36:0/1611124964 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61fc0b4190 0x7f61fc0b48b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f61f0002f70 tx=0x7f61f0006eb0).stop 2022-06-17T12:47:09.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f6203d3b700 1 -- 172.21.15.36:0/1611124964 shutdown_connections 2022-06-17T12:47:09.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f6203d3b700 1 --2- 172.21.15.36:0/1611124964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61fc14b780 0x7f61fc0b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:09.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f6203d3b700 1 --2- 172.21.15.36:0/1611124964 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61fc0b4190 0x7f61fc0b48b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:09.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f6203d3b700 1 -- 172.21.15.36:0/1611124964 >> 172.21.15.36:0/1611124964 conn(0x7f61fc0b97f0 msgr2=0x7f61fc0b9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:09.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f6203d3b700 1 -- 172.21.15.36:0/1611124964 shutdown_connections 2022-06-17T12:47:09.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f6203d3b700 1 -- 172.21.15.36:0/1611124964 wait complete. 2022-06-17T12:47:09.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f6203d3b700 1 Processor -- start 2022-06-17T12:47:09.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f6203d3b700 1 -- start start 2022-06-17T12:47:09.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f6203d3b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61fc062270 0x7f61fc0b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:09.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f6203d3b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61fc0b4190 0x7f61fc142850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:09.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f6203d3b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61fc14b780 0x7f61fc142d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:09.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f6203d3b700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc0b7910 con 0x7f61fc0b4190 2022-06-17T12:47:09.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f6203d3b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc0b7bb0 con 0x7f61fc062270 2022-06-17T12:47:09.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f6203d3b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc0b8620 con 0x7f61fc14b780 2022-06-17T12:47:09.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f62012d6700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61fc0b4190 0x7f61fc142850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:09.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f62012d6700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61fc0b4190 0x7f61fc142850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60028/0 (socket says 172.21.15.36:60028) 2022-06-17T12:47:09.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f62012d6700 1 -- 172.21.15.36:0/3730864718 learned_addr learned my addr 172.21.15.36:0/3730864718 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:09.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f62022d8700 1 --2- 172.21.15.36:0/3730864718 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61fc14b780 0x7f61fc142d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:09.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f6201ad7700 1 --2- 172.21.15.36:0/3730864718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61fc062270 0x7f61fc0b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:09.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f62012d6700 1 -- 172.21.15.36:0/3730864718 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61fc14b780 msgr2=0x7f61fc142d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:09.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f62012d6700 1 --2- 172.21.15.36:0/3730864718 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61fc14b780 0x7f61fc142d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:09.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f62012d6700 1 -- 172.21.15.36:0/3730864718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61fc062270 msgr2=0x7f61fc0b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:09.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f62012d6700 1 --2- 172.21.15.36:0/3730864718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61fc062270 0x7f61fc0b5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:09.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f62012d6700 1 -- 172.21.15.36:0/3730864718 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f000b1e0 con 0x7f61fc0b4190 2022-06-17T12:47:09.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f62012d6700 1 --2- 172.21.15.36:0/3730864718 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61fc0b4190 0x7f61fc142850 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f61f0002f70 tx=0x7f61f0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:09.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f61eeffd700 1 -- 172.21.15.36:0/3730864718 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61f00116e0 con 0x7f61fc0b4190 2022-06-17T12:47:09.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f61eeffd700 1 -- 172.21.15.36:0/3730864718 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f0011840 con 0x7f61fc0b4190 2022-06-17T12:47:09.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f6203d3b700 1 -- 172.21.15.36:0/3730864718 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61fc143230 con 0x7f61fc0b4190 2022-06-17T12:47:09.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f61eeffd700 1 -- 172.21.15.36:0/3730864718 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61f0011ba0 con 0x7f61fc0b4190 2022-06-17T12:47:09.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.721+0000 7f6203d3b700 1 -- 172.21.15.36:0/3730864718 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61fc0b8ac0 con 0x7f61fc0b4190 2022-06-17T12:47:09.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.725+0000 7f6203d3b700 1 -- 172.21.15.36:0/3730864718 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61fc0b87e0 con 0x7f61fc0b4190 2022-06-17T12:47:09.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.725+0000 7f61eeffd700 1 -- 172.21.15.36:0/3730864718 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f61f0025030 con 0x7f61fc0b4190 2022-06-17T12:47:09.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.725+0000 7f61eeffd700 1 --2- 172.21.15.36:0/3730864718 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f61dc053940 0x7f61dc055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:09.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.725+0000 7f61eeffd700 1 -- 172.21.15.36:0/3730864718 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f61f006d1f0 con 0x7f61fc0b4190 2022-06-17T12:47:09.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.725+0000 7f6201ad7700 1 --2- 172.21.15.36:0/3730864718 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f61dc053940 0x7f61dc055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:09.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.725+0000 7f6201ad7700 1 --2- 172.21.15.36:0/3730864718 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f61dc053940 0x7f61dc055e00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f61f800d000 tx=0x7f61f8007b80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:09.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.733+0000 7f61eeffd700 1 -- 172.21.15.36:0/3730864718 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f61f0043ff0 con 0x7f61fc0b4190 2022-06-17T12:47:09.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.861+0000 7f6203d3b700 1 -- 172.21.15.36:0/3730864718 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f61fc057410 con 0x7f61fc0b4190 2022-06-17T12:47:09.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.861+0000 7f61eeffd700 1 -- 172.21.15.36:0/3730864718 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+10 (secure 0 0 0) 0x7f61f0023020 con 0x7f61fc0b4190 2022-06-17T12:47:09.998 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:10.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.861+0000 7f6203d3b700 1 -- 172.21.15.36:0/3730864718 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f61dc053940 msgr2=0x7f61dc055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:10.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.861+0000 7f6203d3b700 1 --2- 172.21.15.36:0/3730864718 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f61dc053940 0x7f61dc055e00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f61f800d000 tx=0x7f61f8007b80).stop 2022-06-17T12:47:10.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.861+0000 7f6203d3b700 1 -- 172.21.15.36:0/3730864718 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61fc0b4190 msgr2=0x7f61fc142850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:10.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.861+0000 7f6203d3b700 1 --2- 172.21.15.36:0/3730864718 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61fc0b4190 0x7f61fc142850 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f61f0002f70 tx=0x7f61f0015000).stop 2022-06-17T12:47:10.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.861+0000 7f6203d3b700 1 -- 172.21.15.36:0/3730864718 shutdown_connections 2022-06-17T12:47:10.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.861+0000 7f6203d3b700 1 --2- 172.21.15.36:0/3730864718 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f61dc053940 0x7f61dc055e00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:10.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.861+0000 7f6203d3b700 1 --2- 172.21.15.36:0/3730864718 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61fc14b780 0x7f61fc142d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:10.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.861+0000 7f6203d3b700 1 --2- 172.21.15.36:0/3730864718 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61fc0b4190 0x7f61fc142850 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:10.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.861+0000 7f6203d3b700 1 --2- 172.21.15.36:0/3730864718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61fc062270 0x7f61fc0b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:10.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.861+0000 7f6203d3b700 1 -- 172.21.15.36:0/3730864718 >> 172.21.15.36:0/3730864718 conn(0x7f61fc0b97f0 msgr2=0x7f61fc148a40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:10.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.861+0000 7f6203d3b700 1 -- 172.21.15.36:0/3730864718 shutdown_connections 2022-06-17T12:47:10.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:09.861+0000 7f6203d3b700 1 -- 172.21.15.36:0/3730864718 wait complete. 2022-06-17T12:47:10.012 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:47:10.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.181+0000 7fbe92d39700 1 Processor -- start 2022-06-17T12:47:10.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.181+0000 7fbe92d39700 1 -- start start 2022-06-17T12:47:10.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe92d39700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe8c062270 0x7fbe8c143d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:10.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe92d39700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe8c143130 con 0x7fbe8c0b4190 2022-06-17T12:47:10.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe92d39700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe8c062650 con 0x7fbe8c0b4fc0 2022-06-17T12:47:10.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe92d39700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c0b6080 con 0x7fbe8c062270 2022-06-17T12:47:10.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe83fff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fbe8c0b4190 0x7fbe8c0b48b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49334/0 (socket says 172.21.15.36:49334) 2022-06-17T12:47:10.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe83fff700 1 -- 172.21.15.36:0/118678985 learned_addr learned my addr 172.21.15.36:0/118678985 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:10.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe90ad5700 1 --2- 172.21.15.36:0/118678985 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe8c062270 0x7fbe8c143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:10.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe837fe700 1 -- 172.21.15.36:0/118678985 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4023077749 0 0) 0x7fbe8c143130 con 0x7fbe8c0b4190 2022-06-17T12:47:10.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe837fe700 1 -- 172.21.15.36:0/118678985 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe78003260 con 0x7fbe8c0b4190 2022-06-17T12:47:10.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe837fe700 1 -- 172.21.15.36:0/118678985 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 694182167 0 0) 0x7fbe8c062650 con 0x7fbe8c0b4fc0 2022-06-17T12:47:10.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe837fe700 1 -- 172.21.15.36:0/118678985 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe8c143130 con 0x7fbe8c0b4fc0 2022-06-17T12:47:10.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe837fe700 1 -- 172.21.15.36:0/118678985 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1875803956 0 0) 0x7fbe78003260 con 0x7fbe8c0b4190 2022-06-17T12:47:10.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe90ad5700 1 -- 172.21.15.36:0/118678985 >> v1:172.21.15.70:6789/0 conn(0x7fbe8c0b4fc0 legacy=0x7fbe8c1400f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:10.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe90ad5700 1 -- 172.21.15.36:0/118678985 >> v1:172.21.15.36:6789/0 conn(0x7fbe8c0b4190 legacy=0x7fbe8c0b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:10.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe90ad5700 1 -- 172.21.15.36:0/118678985 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8c143360 con 0x7fbe8c062270 2022-06-17T12:47:10.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe90ad5700 1 --2- 172.21.15.36:0/118678985 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe8c062270 0x7fbe8c143d40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fbe7c001110 tx=0x7fbe88006df0).ready entity=mon.2 client_cookie=58d67ad76edd2a84 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:10.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe837fe700 1 -- 172.21.15.36:0/118678985 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe7c001660 con 0x7fbe8c062270 2022-06-17T12:47:10.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe837fe700 1 -- 172.21.15.36:0/118678985 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe88000e90 con 0x7fbe8c062270 2022-06-17T12:47:10.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe837fe700 1 -- 172.21.15.36:0/118678985 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe88012460 con 0x7fbe8c062270 2022-06-17T12:47:10.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe92d39700 1 -- 172.21.15.36:0/118678985 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe8c062270 msgr2=0x7fbe8c143d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:10.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe92d39700 1 --2- 172.21.15.36:0/118678985 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe8c062270 0x7fbe8c143d40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fbe7c001110 tx=0x7fbe88006df0).stop 2022-06-17T12:47:10.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe92d39700 1 -- 172.21.15.36:0/118678985 shutdown_connections 2022-06-17T12:47:10.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe92d39700 1 --2- 172.21.15.36:0/118678985 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe8c062270 0x7fbe8c143d40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:10.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe92d39700 1 -- 172.21.15.36:0/118678985 >> 172.21.15.36:0/118678985 conn(0x7fbe8c0b97f0 msgr2=0x7fbe8c0b9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:10.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe92d39700 1 -- 172.21.15.36:0/118678985 shutdown_connections 2022-06-17T12:47:10.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.185+0000 7fbe92d39700 1 -- 172.21.15.36:0/118678985 wait complete. 2022-06-17T12:47:10.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.189+0000 7fbe92d39700 1 Processor -- start 2022-06-17T12:47:10.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.189+0000 7fbe92d39700 1 -- start start 2022-06-17T12:47:10.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.189+0000 7fbe92d39700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe8c062270 0x7fbe8c0aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:10.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.189+0000 7fbe92d39700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe8c0b4190 0x7fbe8c0b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:10.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.189+0000 7fbe92d39700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbe8c0b4fc0 0x7fbe8c0b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:10.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.189+0000 7fbe92d39700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c1475a0 con 0x7fbe8c0b4fc0 2022-06-17T12:47:10.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.189+0000 7fbe92d39700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c146920 con 0x7fbe8c0b4190 2022-06-17T12:47:10.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.189+0000 7fbe92d39700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c146bc0 con 0x7fbe8c062270 2022-06-17T12:47:10.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.189+0000 7fbe912d6700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbe8c0b4fc0 0x7fbe8c0b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:10.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.189+0000 7fbe90ad5700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe8c062270 0x7fbe8c0aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:10.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.189+0000 7fbe912d6700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbe8c0b4fc0 0x7fbe8c0b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60044/0 (socket says 172.21.15.36:60044) 2022-06-17T12:47:10.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.189+0000 7fbe912d6700 1 -- 172.21.15.36:0/1977140759 learned_addr learned my addr 172.21.15.36:0/1977140759 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:10.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.189+0000 7fbe83fff700 1 --2- 172.21.15.36:0/1977140759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe8c0b4190 0x7fbe8c0b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:10.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.189+0000 7fbe912d6700 1 -- 172.21.15.36:0/1977140759 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe8c062270 msgr2=0x7fbe8c0aff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:10.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.189+0000 7fbe912d6700 1 --2- 172.21.15.36:0/1977140759 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe8c062270 0x7fbe8c0aff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:10.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.189+0000 7fbe912d6700 1 -- 172.21.15.36:0/1977140759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe8c0b4190 msgr2=0x7fbe8c0b2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:10.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.189+0000 7fbe912d6700 1 --2- 172.21.15.36:0/1977140759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe8c0b4190 0x7fbe8c0b2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:10.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.189+0000 7fbe912d6700 1 -- 172.21.15.36:0/1977140759 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8800b180 con 0x7fbe8c0b4fc0 2022-06-17T12:47:10.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.189+0000 7fbe912d6700 1 --2- 172.21.15.36:0/1977140759 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbe8c0b4fc0 0x7fbe8c0b2f10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fbe84001490 tx=0x7fbe84007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:10.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.189+0000 7fbe81ffb700 1 -- 172.21.15.36:0/1977140759 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe84002310 con 0x7fbe8c0b4fc0 2022-06-17T12:47:10.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.193+0000 7fbe81ffb700 1 -- 172.21.15.36:0/1977140759 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe84002470 con 0x7fbe8c0b4fc0 2022-06-17T12:47:10.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.193+0000 7fbe92d39700 1 -- 172.21.15.36:0/1977140759 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe8c146e60 con 0x7fbe8c0b4fc0 2022-06-17T12:47:10.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.193+0000 7fbe81ffb700 1 -- 172.21.15.36:0/1977140759 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe840017a0 con 0x7fbe8c0b4fc0 2022-06-17T12:47:10.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.193+0000 7fbe92d39700 1 -- 172.21.15.36:0/1977140759 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8c0b3a60 con 0x7fbe8c0b4fc0 2022-06-17T12:47:10.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.193+0000 7fbe81ffb700 1 -- 172.21.15.36:0/1977140759 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fbe84010030 con 0x7fbe8c0b4fc0 2022-06-17T12:47:10.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.193+0000 7fbe92d39700 1 -- 172.21.15.36:0/1977140759 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe8c146200 con 0x7fbe8c0b4fc0 2022-06-17T12:47:10.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.193+0000 7fbe81ffb700 1 --2- 172.21.15.36:0/1977140759 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbe6c053a40 0x7fbe6c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:10.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.193+0000 7fbe81ffb700 1 -- 172.21.15.36:0/1977140759 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fbe8406bf10 con 0x7fbe8c0b4fc0 2022-06-17T12:47:10.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.193+0000 7fbe90ad5700 1 --2- 172.21.15.36:0/1977140759 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbe6c053a40 0x7fbe6c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:10.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.193+0000 7fbe90ad5700 1 --2- 172.21.15.36:0/1977140759 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbe6c053a40 0x7fbe6c055f00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fbe8801a000 tx=0x7fbe88014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:10.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.197+0000 7fbe81ffb700 1 -- 172.21.15.36:0/1977140759 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbe8c146200 con 0x7fbe8c0b4fc0 2022-06-17T12:47:10.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.325+0000 7fbe92d39700 1 -- 172.21.15.36:0/1977140759 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbe8c147300 con 0x7fbe8c0b4fc0 2022-06-17T12:47:10.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.325+0000 7fbe81ffb700 1 -- 172.21.15.36:0/1977140759 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+10 (secure 0 0 0) 0x7fbe8c147300 con 0x7fbe8c0b4fc0 2022-06-17T12:47:10.331 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:10.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.325+0000 7fbe92d39700 1 -- 172.21.15.36:0/1977140759 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbe6c053a40 msgr2=0x7fbe6c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:10.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.325+0000 7fbe92d39700 1 --2- 172.21.15.36:0/1977140759 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbe6c053a40 0x7fbe6c055f00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fbe8801a000 tx=0x7fbe88014000).stop 2022-06-17T12:47:10.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.325+0000 7fbe92d39700 1 -- 172.21.15.36:0/1977140759 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbe8c0b4fc0 msgr2=0x7fbe8c0b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:10.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.325+0000 7fbe92d39700 1 --2- 172.21.15.36:0/1977140759 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbe8c0b4fc0 0x7fbe8c0b2f10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fbe84001490 tx=0x7fbe84007dd0).stop 2022-06-17T12:47:10.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.329+0000 7fbe92d39700 1 -- 172.21.15.36:0/1977140759 shutdown_connections 2022-06-17T12:47:10.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.329+0000 7fbe92d39700 1 --2- 172.21.15.36:0/1977140759 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbe6c053a40 0x7fbe6c055f00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:10.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.329+0000 7fbe92d39700 1 --2- 172.21.15.36:0/1977140759 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbe8c0b4fc0 0x7fbe8c0b2f10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:10.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.329+0000 7fbe92d39700 1 --2- 172.21.15.36:0/1977140759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe8c0b4190 0x7fbe8c0b2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:10.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.329+0000 7fbe92d39700 1 --2- 172.21.15.36:0/1977140759 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe8c062270 0x7fbe8c0aff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:10.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.329+0000 7fbe92d39700 1 -- 172.21.15.36:0/1977140759 >> 172.21.15.36:0/1977140759 conn(0x7fbe8c0b97f0 msgr2=0x7fbe8c140080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:10.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.329+0000 7fbe92d39700 1 -- 172.21.15.36:0/1977140759 shutdown_connections 2022-06-17T12:47:10.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.329+0000 7fbe92d39700 1 -- 172.21.15.36:0/1977140759 wait complete. 2022-06-17T12:47:10.417 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:47:10.418 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:47:10.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.545+0000 7f5ba0f11700 1 Processor -- start 2022-06-17T12:47:10.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.545+0000 7f5ba0f11700 1 -- start start 2022-06-17T12:47:10.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.545+0000 7f5ba0f11700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5b9c0b16d0 0x7f5b9c141540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:10.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.545+0000 7f5ba0f11700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b9c065090 con 0x7f5b9c0b12f0 2022-06-17T12:47:10.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.545+0000 7f5ba0f11700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b9c0b0b40 con 0x7f5b9c062bd0 2022-06-17T12:47:10.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.545+0000 7f5ba0f11700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b9c063510 con 0x7f5b9c0b16d0 2022-06-17T12:47:10.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.545+0000 7f5b9a59c700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f5b9c0b12f0 0x7f5b9c13aac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49348/0 (socket says 172.21.15.36:49348) 2022-06-17T12:47:10.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.545+0000 7f5b9a59c700 1 -- 172.21.15.36:0/2362913856 learned_addr learned my addr 172.21.15.36:0/2362913856 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:10.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.545+0000 7f5b9ad9d700 1 --2- 172.21.15.36:0/2362913856 >> v2:172.21.15.70:3300/0 conn(0x7f5b9c0b16d0 0x7f5b9c141540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:10.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.545+0000 7f5b9959a700 1 -- 172.21.15.36:0/2362913856 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4134486297 0 0) 0x7f5b9c065090 con 0x7f5b9c0b12f0 2022-06-17T12:47:10.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.545+0000 7f5b9959a700 1 -- 172.21.15.36:0/2362913856 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b80003260 con 0x7f5b9c0b12f0 2022-06-17T12:47:10.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.545+0000 7f5b9959a700 1 -- 172.21.15.36:0/2362913856 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3065680789 0 0) 0x7f5b9c0b0b40 con 0x7f5b9c062bd0 2022-06-17T12:47:10.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.545+0000 7f5b9959a700 1 -- 172.21.15.36:0/2362913856 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b9c065090 con 0x7f5b9c062bd0 2022-06-17T12:47:10.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.545+0000 7f5b9ad9d700 1 -- 172.21.15.36:0/2362913856 >> v1:172.21.15.70:6789/0 conn(0x7f5b9c062bd0 legacy=0x7f5b9c140e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:10.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.545+0000 7f5b9ad9d700 1 -- 172.21.15.36:0/2362913856 >> v1:172.21.15.36:6789/0 conn(0x7f5b9c0b12f0 legacy=0x7f5b9c13aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:10.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.545+0000 7f5b9ad9d700 1 -- 172.21.15.36:0/2362913856 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9c064df0 con 0x7f5b9c0b16d0 2022-06-17T12:47:10.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.545+0000 7f5b9ad9d700 1 --2- 172.21.15.36:0/2362913856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b9c0b16d0 0x7f5b9c141540 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f5b9c063060 tx=0x7f5b940091d0).ready entity=mon.1 client_cookie=b781a28961a821ab server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:10.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.549+0000 7f5b9959a700 1 -- 172.21.15.36:0/2362913856 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b9400b600 con 0x7f5b9c0b16d0 2022-06-17T12:47:10.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.549+0000 7f5b9959a700 1 -- 172.21.15.36:0/2362913856 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9400b760 con 0x7f5b9c0b16d0 2022-06-17T12:47:10.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.549+0000 7f5b9959a700 1 -- 172.21.15.36:0/2362913856 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b9400b8d0 con 0x7f5b9c0b16d0 2022-06-17T12:47:10.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.549+0000 7f5ba0f11700 1 -- 172.21.15.36:0/2362913856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b9c0b16d0 msgr2=0x7f5b9c141540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:10.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.549+0000 7f5ba0f11700 1 --2- 172.21.15.36:0/2362913856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b9c0b16d0 0x7f5b9c141540 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f5b9c063060 tx=0x7f5b940091d0).stop 2022-06-17T12:47:10.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.549+0000 7f5ba0f11700 1 -- 172.21.15.36:0/2362913856 shutdown_connections 2022-06-17T12:47:10.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.549+0000 7f5ba0f11700 1 --2- 172.21.15.36:0/2362913856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b9c0b16d0 0x7f5b9c141540 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:10.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.549+0000 7f5ba0f11700 1 -- 172.21.15.36:0/2362913856 >> 172.21.15.36:0/2362913856 conn(0x7f5b9c0b53e0 msgr2=0x7f5b9c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:10.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.549+0000 7f5ba0f11700 1 -- 172.21.15.36:0/2362913856 shutdown_connections 2022-06-17T12:47:10.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.549+0000 7f5ba0f11700 1 -- 172.21.15.36:0/2362913856 wait complete. 2022-06-17T12:47:10.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.549+0000 7f5ba0f11700 1 Processor -- start 2022-06-17T12:47:10.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.553+0000 7f5ba0f11700 1 -- start start 2022-06-17T12:47:10.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.553+0000 7f5ba0f11700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b9c062bd0 0x7f5b9c13e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:10.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.553+0000 7f5ba0f11700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b9c0b12f0 0x7f5b9c13ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:10.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.553+0000 7f5ba0f11700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5b9c0b16d0 0x7f5b9c140340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:10.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.553+0000 7f5ba0f11700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c13c9f0 con 0x7f5b9c0b16d0 2022-06-17T12:47:10.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.553+0000 7f5ba0f11700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c13cc90 con 0x7f5b9c062bd0 2022-06-17T12:47:10.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.553+0000 7f5ba0f11700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c141300 con 0x7f5b9c0b12f0 2022-06-17T12:47:10.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.553+0000 7f5b9ad9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5b9c0b16d0 0x7f5b9c140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:10.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.553+0000 7f5b9ad9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5b9c0b16d0 0x7f5b9c140340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60058/0 (socket says 172.21.15.36:60058) 2022-06-17T12:47:10.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.553+0000 7f5b9ad9d700 1 -- 172.21.15.36:0/413929826 learned_addr learned my addr 172.21.15.36:0/413929826 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:10.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.553+0000 7f5b9a59c700 1 --2- 172.21.15.36:0/413929826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b9c062bd0 0x7f5b9c13e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:10.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.553+0000 7f5b99d9b700 1 --2- 172.21.15.36:0/413929826 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b9c0b12f0 0x7f5b9c13ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:10.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.553+0000 7f5b9ad9d700 1 -- 172.21.15.36:0/413929826 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b9c0b12f0 msgr2=0x7f5b9c13ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:10.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.553+0000 7f5b9ad9d700 1 --2- 172.21.15.36:0/413929826 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b9c0b12f0 0x7f5b9c13ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:10.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.553+0000 7f5b9ad9d700 1 -- 172.21.15.36:0/413929826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b9c062bd0 msgr2=0x7f5b9c13e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:10.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.553+0000 7f5b9ad9d700 1 --2- 172.21.15.36:0/413929826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b9c062bd0 0x7f5b9c13e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:10.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.553+0000 7f5b9ad9d700 1 -- 172.21.15.36:0/413929826 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b94009470 con 0x7f5b9c0b16d0 2022-06-17T12:47:10.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.553+0000 7f5b9ad9d700 1 --2- 172.21.15.36:0/413929826 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5b9c0b16d0 0x7f5b9c140340 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f5b9c063060 tx=0x7f5b94016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:10.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.553+0000 7f5b8b7fe700 1 -- 172.21.15.36:0/413929826 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b9400b600 con 0x7f5b9c0b16d0 2022-06-17T12:47:10.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.553+0000 7f5b8b7fe700 1 -- 172.21.15.36:0/413929826 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b940032f0 con 0x7f5b9c0b16d0 2022-06-17T12:47:10.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.553+0000 7f5ba0f11700 1 -- 172.21.15.36:0/413929826 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b9c140830 con 0x7f5b9c0b16d0 2022-06-17T12:47:10.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.553+0000 7f5b8b7fe700 1 -- 172.21.15.36:0/413929826 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b9401f5e0 con 0x7f5b9c0b16d0 2022-06-17T12:47:10.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.553+0000 7f5ba0f11700 1 -- 172.21.15.36:0/413929826 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b9c13dc50 con 0x7f5b9c0b16d0 2022-06-17T12:47:10.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.553+0000 7f5ba0f11700 1 -- 172.21.15.36:0/413929826 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b9c141520 con 0x7f5b9c0b16d0 2022-06-17T12:47:10.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.557+0000 7f5b8b7fe700 1 -- 172.21.15.36:0/413929826 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f5b9400f070 con 0x7f5b9c0b16d0 2022-06-17T12:47:10.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.557+0000 7f5b8b7fe700 1 --2- 172.21.15.36:0/413929826 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5b78053a40 0x7f5b78055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:10.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.557+0000 7f5b8b7fe700 1 -- 172.21.15.36:0/413929826 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5b9406c8e0 con 0x7f5b9c0b16d0 2022-06-17T12:47:10.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.557+0000 7f5b9a59c700 1 --2- 172.21.15.36:0/413929826 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5b78053a40 0x7f5b78055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:10.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.557+0000 7f5b9a59c700 1 --2- 172.21.15.36:0/413929826 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5b78053a40 0x7f5b78055f00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f5b9000b000 tx=0x7f5b90009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:10.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.565+0000 7f5b8b7fe700 1 -- 172.21.15.36:0/413929826 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5b940436c0 con 0x7f5b9c0b16d0 2022-06-17T12:47:10.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.729+0000 7f5ba0f11700 1 -- 172.21.15.36:0/413929826 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b9c0b03c0 con 0x7f5b9c0b16d0 2022-06-17T12:47:10.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.729+0000 7f5b8b7fe700 1 -- 172.21.15.36:0/413929826 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2134 (secure 0 0 0) 0x7f5b94024870 con 0x7f5b9c0b16d0 2022-06-17T12:47:10.723 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:10.723 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/4243909763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":4243909763},{"type":"v1","addr":"172.21.15.36:6835","nonce":4243909763}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:46:52.253609+0000","modified":"2022-06-17T12:46:54.268223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/3425584732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3425584732},{"type":"v1","addr":"172.21.15.70:6833","nonce":3425584732}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-06-17T12:47:10.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.733+0000 7f5ba0f11700 1 -- 172.21.15.36:0/413929826 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5b78053a40 msgr2=0x7f5b78055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:10.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.733+0000 7f5ba0f11700 1 --2- 172.21.15.36:0/413929826 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5b78053a40 0x7f5b78055f00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f5b9000b000 tx=0x7f5b90009040).stop 2022-06-17T12:47:10.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.733+0000 7f5ba0f11700 1 -- 172.21.15.36:0/413929826 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5b9c0b16d0 msgr2=0x7f5b9c140340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:10.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.733+0000 7f5ba0f11700 1 --2- 172.21.15.36:0/413929826 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5b9c0b16d0 0x7f5b9c140340 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f5b9c063060 tx=0x7f5b94016000).stop 2022-06-17T12:47:10.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.733+0000 7f5ba0f11700 1 -- 172.21.15.36:0/413929826 shutdown_connections 2022-06-17T12:47:10.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.733+0000 7f5ba0f11700 1 --2- 172.21.15.36:0/413929826 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5b78053a40 0x7f5b78055f00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:10.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.733+0000 7f5ba0f11700 1 --2- 172.21.15.36:0/413929826 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5b9c0b16d0 0x7f5b9c140340 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:10.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.733+0000 7f5ba0f11700 1 --2- 172.21.15.36:0/413929826 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b9c0b12f0 0x7f5b9c13ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:10.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.733+0000 7f5ba0f11700 1 --2- 172.21.15.36:0/413929826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b9c062bd0 0x7f5b9c13e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:10.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.733+0000 7f5ba0f11700 1 -- 172.21.15.36:0/413929826 >> 172.21.15.36:0/413929826 conn(0x7f5b9c0b53e0 msgr2=0x7f5b9c0b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:10.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.733+0000 7f5ba0f11700 1 -- 172.21.15.36:0/413929826 shutdown_connections 2022-06-17T12:47:10.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:10.733+0000 7f5ba0f11700 1 -- 172.21.15.36:0/413929826 wait complete. 2022-06-17T12:47:10.740 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 7 2022-06-17T12:47:10.952 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:46:52.253609+0000', 'modified': '2022-06-17T12:46:54.268223+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4223}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4223': {'gid': 4223, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6833/3425584732', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 3425584732}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 3425584732}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:47:10.953 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:47:10.954 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-17T12:47:11.113 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":4223,"want_state":"up:active","state":"up:active","rank_uptime":18.848282922999999,"mdsmap_epoch":7,"osdmap_epoch":26,"osdmap_epoch_barrier":26,"uptime":25.29634235} 2022-06-17T12:47:11.114 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 4223, "mdsmap_epoch": 7, "osdmap_epoch": 26, "osdmap_epoch_barrier": 26, "rank_uptime": 18.848282923, "state": "up:active", "uptime": 25.29634235, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:47:11.114 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:47:11.115 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-17T12:47:11.115 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi107.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-17T12:47:11.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:11.116 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:47:11.171 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:47:11.250 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:47:11.255 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:47:11.366 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:47:11.394 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:47:11.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:11.397 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:47:11.432 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:47:11.433 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:47:11.447 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:47:11.560 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:47:11.560 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:47:11.562 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:2022-06-17T12:47:11.567+0000 7fb999f47080 -1 init, newargv = 0x55ea55229340 newargc=9 2022-06-17T12:47:11.562 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[21734]: starting ceph client 2022-06-17T12:47:11.568 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[21734]: starting fuse 2022-06-17T12:47:11.586 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:47:11.587 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:47:11.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:11.602 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:47:11.647 INFO:teuthology.orchestra.run.smithi107.stdout:52 2022-06-17T12:47:11.698 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:47:11.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:11.700 DEBUG:teuthology.orchestra.run.smithi107:> sudo python3 -c ' 2022-06-17T12:47:11.701 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2022-06-17T12:47:11.701 DEBUG:teuthology.orchestra.run.smithi107:> import re 2022-06-17T12:47:11.720 DEBUG:teuthology.orchestra.run.smithi107:> import os 2022-06-17T12:47:11.722 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2022-06-17T12:47:11.722 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:47:11.723 DEBUG:teuthology.orchestra.run.smithi107:> def find_socket(client_name): 2022-06-17T12:47:11.723 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-17T12:47:11.723 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2022-06-17T12:47:11.724 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:47:11.724 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2022-06-17T12:47:11.724 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2022-06-17T12:47:11.725 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2022-06-17T12:47:11.725 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2022-06-17T12:47:11.725 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:47:11.726 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2022-06-17T12:47:11.726 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:47:11.726 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:47:11.727 DEBUG:teuthology.orchestra.run.smithi107:> return f 2022-06-17T12:47:11.727 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:47:11.727 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:47:11.728 DEBUG:teuthology.orchestra.run.smithi107:> print(find_socket("client.0")) 2022-06-17T12:47:11.731 DEBUG:teuthology.orchestra.run.smithi107:> ' 2022-06-17T12:47:11.789 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.0.21734.asok 2022-06-17T12:47:11.820 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.21734.asok 2022-06-17T12:47:11.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:11.822 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.21734.asok status 2022-06-17T12:47:11.965 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2022-06-17T12:47:11.966 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2022-06-17T12:47:11.966 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:47:11.966 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:47:11.967 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "0", 2022-06-17T12:47:11.967 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2022-06-17T12:47:11.967 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-17T12:47:11.967 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "21734", 2022-06-17T12:47:11.968 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/" 2022-06-17T12:47:11.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:47:11.968 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2022-06-17T12:47:11.968 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2022-06-17T12:47:11.968 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 4635, 2022-06-17T12:47:11.969 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2022-06-17T12:47:11.969 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2022-06-17T12:47:11.969 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2022-06-17T12:47:11.969 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 4635 2022-06-17T12:47:11.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:47:11.976 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:47:11.977 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:47:11.977 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:47:11.984 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1369185213 2022-06-17T12:47:11.984 INFO:teuthology.orchestra.run.smithi107.stdout: } 2022-06-17T12:47:11.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:47:11.985 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:47:11.985 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:47:11.985 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:47:11.986 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1369185213 2022-06-17T12:47:11.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:47:11.986 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.4635 172.21.15.107:0/1369185213", 2022-06-17T12:47:11.986 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "172.21.15.107:0/1369185213", 2022-06-17T12:47:11.988 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 1, 2022-06-17T12:47:11.988 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 7, 2022-06-17T12:47:11.988 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 26, 2022-06-17T12:47:11.989 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:47:11.989 INFO:teuthology.orchestra.run.smithi107.stdout: "blacklisted": false, 2022-06-17T12:47:11.989 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2022-06-17T12:47:11.989 INFO:teuthology.orchestra.run.smithi107.stdout:} 2022-06-17T12:47:11.991 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:47:12.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.125+0000 7fe7380b2700 1 Processor -- start 2022-06-17T12:47:12.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.125+0000 7fe7380b2700 1 -- start start 2022-06-17T12:47:12.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.129+0000 7fe7380b2700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe7300b0cb0 0x7fe730142f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:12.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.129+0000 7fe7380b2700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7300b44b0 con 0x7fe730062270 2022-06-17T12:47:12.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.129+0000 7fe7380b2700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe730062650 con 0x7fe730143e80 2022-06-17T12:47:12.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.129+0000 7fe7380b2700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7300b6650 con 0x7fe7300b0cb0 2022-06-17T12:47:12.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.129+0000 7fe735e4e700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fe730062270 0x7fe730143770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49362/0 (socket says 172.21.15.36:49362) 2022-06-17T12:47:12.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.129+0000 7fe73564d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe7300b0cb0 0x7fe730142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:12.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.129+0000 7fe735e4e700 1 -- 172.21.15.36:0/3167260888 learned_addr learned my addr 172.21.15.36:0/3167260888 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:12.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.129+0000 7fe73564d700 1 -- 172.21.15.36:0/3167260888 >> v1:172.21.15.70:6789/0 conn(0x7fe730143e80 legacy=0x7fe730146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:12.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.129+0000 7fe73564d700 1 -- 172.21.15.36:0/3167260888 >> v1:172.21.15.36:6789/0 conn(0x7fe730062270 legacy=0x7fe730143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:12.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.129+0000 7fe734e4c700 1 -- 172.21.15.36:0/3167260888 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1439684824 0 0) 0x7fe7300b44b0 con 0x7fe730062270 2022-06-17T12:47:12.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.129+0000 7fe73564d700 1 -- 172.21.15.36:0/3167260888 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7300b4210 con 0x7fe7300b0cb0 2022-06-17T12:47:12.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.129+0000 7fe73564d700 1 --2- 172.21.15.36:0/3167260888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7300b0cb0 0x7fe730142f20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fe720002f70 tx=0x7fe720006df0).ready entity=mon.1 client_cookie=2fe94f123ce7c5a5 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:12.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.129+0000 7fe734e4c700 1 -- 172.21.15.36:0/3167260888 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe72000d610 con 0x7fe7300b0cb0 2022-06-17T12:47:12.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.129+0000 7fe734e4c700 1 -- 172.21.15.36:0/3167260888 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe720000e90 con 0x7fe7300b0cb0 2022-06-17T12:47:12.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.129+0000 7fe734e4c700 1 -- 172.21.15.36:0/3167260888 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe72000d960 con 0x7fe7300b0cb0 2022-06-17T12:47:12.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.129+0000 7fe7380b2700 1 -- 172.21.15.36:0/3167260888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7300b0cb0 msgr2=0x7fe730142f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:12.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.129+0000 7fe7380b2700 1 --2- 172.21.15.36:0/3167260888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7300b0cb0 0x7fe730142f20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fe720002f70 tx=0x7fe720006df0).stop 2022-06-17T12:47:12.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.129+0000 7fe7380b2700 1 -- 172.21.15.36:0/3167260888 shutdown_connections 2022-06-17T12:47:12.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.129+0000 7fe7380b2700 1 --2- 172.21.15.36:0/3167260888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7300b0cb0 0x7fe730142f20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:12.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.129+0000 7fe7380b2700 1 -- 172.21.15.36:0/3167260888 >> 172.21.15.36:0/3167260888 conn(0x7fe7300b9870 msgr2=0x7fe7300b9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:12.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.129+0000 7fe7380b2700 1 -- 172.21.15.36:0/3167260888 shutdown_connections 2022-06-17T12:47:12.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.129+0000 7fe7380b2700 1 -- 172.21.15.36:0/3167260888 wait complete. 2022-06-17T12:47:12.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe7380b2700 1 Processor -- start 2022-06-17T12:47:12.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe7380b2700 1 -- start start 2022-06-17T12:47:12.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe7380b2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe730062270 0x7fe7300b3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:12.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe7380b2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe730143e80 0x7fe730156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:12.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe7380b2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe730146710 0x7fe7301522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:12.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe7380b2700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7300ae720 con 0x7fe730062270 2022-06-17T12:47:12.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe7380b2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe730155940 con 0x7fe730143e80 2022-06-17T12:47:12.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe7380b2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe730155be0 con 0x7fe730146710 2022-06-17T12:47:12.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe735e4e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe730062270 0x7fe7300b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:12.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe73564d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe730143e80 0x7fe730156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:12.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe73564d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe730143e80 0x7fe730156080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:52220/0 (socket says 172.21.15.36:52220) 2022-06-17T12:47:12.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe735e4e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe730062270 0x7fe7300b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60068/0 (socket says 172.21.15.36:60068) 2022-06-17T12:47:12.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe73564d700 1 -- 172.21.15.36:0/1044001503 learned_addr learned my addr 172.21.15.36:0/1044001503 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:12.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe73664f700 1 --2- 172.21.15.36:0/1044001503 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe730146710 0x7fe7301522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:12.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe735e4e700 1 -- 172.21.15.36:0/1044001503 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe730146710 msgr2=0x7fe7301522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:12.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe735e4e700 1 --2- 172.21.15.36:0/1044001503 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe730146710 0x7fe7301522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:12.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe735e4e700 1 -- 172.21.15.36:0/1044001503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe730143e80 msgr2=0x7fe730156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:12.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe735e4e700 1 --2- 172.21.15.36:0/1044001503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe730143e80 0x7fe730156080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:12.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe735e4e700 1 -- 172.21.15.36:0/1044001503 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72000b180 con 0x7fe730062270 2022-06-17T12:47:12.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe735e4e700 1 --2- 172.21.15.36:0/1044001503 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe730062270 0x7fe7300b3190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fe72c000ed0 tx=0x7fe72c007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:12.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe71effd700 1 -- 172.21.15.36:0/1044001503 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe72c0070d0 con 0x7fe730062270 2022-06-17T12:47:12.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe71effd700 1 -- 172.21.15.36:0/1044001503 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72c007230 con 0x7fe730062270 2022-06-17T12:47:12.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe7380b2700 1 -- 172.21.15.36:0/1044001503 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe730146c90 con 0x7fe730062270 2022-06-17T12:47:12.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe71effd700 1 -- 172.21.15.36:0/1044001503 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe72c005610 con 0x7fe730062270 2022-06-17T12:47:12.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.133+0000 7fe7380b2700 1 -- 172.21.15.36:0/1044001503 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7300ae2f0 con 0x7fe730062270 2022-06-17T12:47:12.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.137+0000 7fe7380b2700 1 -- 172.21.15.36:0/1044001503 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe730155da0 con 0x7fe730062270 2022-06-17T12:47:12.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.137+0000 7fe71effd700 1 -- 172.21.15.36:0/1044001503 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe72c01a060 con 0x7fe730062270 2022-06-17T12:47:12.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.137+0000 7fe71effd700 1 --2- 172.21.15.36:0/1044001503 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe710053920 0x7fe710055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:12.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.137+0000 7fe71effd700 1 -- 172.21.15.36:0/1044001503 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe72c06b6b0 con 0x7fe730062270 2022-06-17T12:47:12.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.137+0000 7fe73564d700 1 --2- 172.21.15.36:0/1044001503 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe710053920 0x7fe710055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:12.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.137+0000 7fe73564d700 1 --2- 172.21.15.36:0/1044001503 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe710053920 0x7fe710055de0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fe72001c000 tx=0x7fe720015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:12.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.141+0000 7fe71effd700 1 -- 172.21.15.36:0/1044001503 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe72c0424b0 con 0x7fe730062270 2022-06-17T12:47:12.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.273+0000 7fe7380b2700 1 -- 172.21.15.36:0/1044001503 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7300b63b0 con 0x7fe730062270 2022-06-17T12:47:12.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.277+0000 7fe71effd700 1 -- 172.21.15.36:0/1044001503 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+16299 (secure 0 0 0) 0x7fe72c018070 con 0x7fe730062270 2022-06-17T12:47:12.399 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:12.400 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":26,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:46:53.258075+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:46:47.465328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-17T12:46:49.893338+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:47:12.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.277+0000 7fe7380b2700 1 -- 172.21.15.36:0/1044001503 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe710053920 msgr2=0x7fe710055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:12.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.277+0000 7fe7380b2700 1 --2- 172.21.15.36:0/1044001503 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe710053920 0x7fe710055de0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fe72001c000 tx=0x7fe720015000).stop 2022-06-17T12:47:12.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.277+0000 7fe7380b2700 1 -- 172.21.15.36:0/1044001503 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe730062270 msgr2=0x7fe7300b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:12.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.277+0000 7fe7380b2700 1 --2- 172.21.15.36:0/1044001503 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe730062270 0x7fe7300b3190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fe72c000ed0 tx=0x7fe72c007a00).stop 2022-06-17T12:47:12.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.277+0000 7fe7380b2700 1 -- 172.21.15.36:0/1044001503 shutdown_connections 2022-06-17T12:47:12.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.277+0000 7fe7380b2700 1 --2- 172.21.15.36:0/1044001503 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe710053920 0x7fe710055de0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:12.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.277+0000 7fe7380b2700 1 --2- 172.21.15.36:0/1044001503 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe730146710 0x7fe7301522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:12.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.277+0000 7fe7380b2700 1 --2- 172.21.15.36:0/1044001503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe730143e80 0x7fe730156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:12.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.277+0000 7fe7380b2700 1 --2- 172.21.15.36:0/1044001503 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe730062270 0x7fe7300b3190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:12.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.277+0000 7fe7380b2700 1 -- 172.21.15.36:0/1044001503 >> 172.21.15.36:0/1044001503 conn(0x7fe7300b9870 msgr2=0x7fe730141700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:12.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.277+0000 7fe7380b2700 1 -- 172.21.15.36:0/1044001503 shutdown_connections 2022-06-17T12:47:12.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.277+0000 7fe7380b2700 1 -- 172.21.15.36:0/1044001503 wait complete. 2022-06-17T12:47:12.439 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:46:47.465328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:46:49.893338+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:47:12.440 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:47:12.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.577+0000 7fda5b334700 1 Processor -- start 2022-06-17T12:47:12.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.577+0000 7fda5b334700 1 -- start start 2022-06-17T12:47:12.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.577+0000 7fda5b334700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fda540af540 0x7fda540b2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:12.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.577+0000 7fda5b334700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fda540b1190 0x7fda54141010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:12.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.577+0000 7fda5b334700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda540b3cf0 con 0x7fda540b6660 2022-06-17T12:47:12.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.577+0000 7fda5b334700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda540670a0 con 0x7fda540af540 2022-06-17T12:47:12.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.577+0000 7fda5b334700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda54144bf0 con 0x7fda540b1190 2022-06-17T12:47:12.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.577+0000 7fda59b31700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fda540af540 0x7fda540b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:12.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.581+0000 7fda59b31700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fda540af540 0x7fda540b2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60078/0 (socket says 172.21.15.36:60078) 2022-06-17T12:47:12.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.581+0000 7fda59b31700 1 -- 172.21.15.36:0/2219575450 learned_addr learned my addr 172.21.15.36:0/2219575450 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:12.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.581+0000 7fda5a332700 1 --1- 172.21.15.36:0/2219575450 >> v1:172.21.15.36:6789/0 conn(0x7fda540b6660 0x7fda5413a790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49374/0 (socket says 172.21.15.36:49374) 2022-06-17T12:47:12.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.581+0000 7fda5ab33700 1 --2- 172.21.15.36:0/2219575450 >> v2:172.21.15.70:3300/0 conn(0x7fda540b1190 0x7fda54141010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:12.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.581+0000 7fda59b31700 1 -- 172.21.15.36:0/2219575450 >> v2:172.21.15.70:3300/0 conn(0x7fda540b1190 msgr2=0x7fda54141010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:12.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.581+0000 7fda59b31700 1 --2- 172.21.15.36:0/2219575450 >> v2:172.21.15.70:3300/0 conn(0x7fda540b1190 0x7fda54141010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:12.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.581+0000 7fda59b31700 1 -- 172.21.15.36:0/2219575450 >> v1:172.21.15.36:6789/0 conn(0x7fda540b6660 legacy=0x7fda5413a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:12.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.581+0000 7fda59330700 1 -- 172.21.15.36:0/2219575450 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443297461 0 0) 0x7fda50001110 con 0x7fda540b6660 2022-06-17T12:47:12.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.581+0000 7fda59b31700 1 -- 172.21.15.36:0/2219575450 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda54144e90 con 0x7fda540af540 2022-06-17T12:47:12.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.581+0000 7fda59b31700 1 --2- 172.21.15.36:0/2219575450 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda540af540 0x7fda540b2860 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fda44002f70 tx=0x7fda44006e50).ready entity=mon.0 client_cookie=d4a32f44f85f638c server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:12.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.581+0000 7fda59330700 1 -- 172.21.15.36:0/2219575450 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda4400d610 con 0x7fda540af540 2022-06-17T12:47:12.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.581+0000 7fda59330700 1 -- 172.21.15.36:0/2219575450 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda44000e90 con 0x7fda540af540 2022-06-17T12:47:12.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.581+0000 7fda59330700 1 -- 172.21.15.36:0/2219575450 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda4400d960 con 0x7fda540af540 2022-06-17T12:47:12.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.581+0000 7fda5b334700 1 -- 172.21.15.36:0/2219575450 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda540af540 msgr2=0x7fda540b2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:12.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.581+0000 7fda5b334700 1 --2- 172.21.15.36:0/2219575450 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda540af540 0x7fda540b2860 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fda44002f70 tx=0x7fda44006e50).stop 2022-06-17T12:47:12.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.581+0000 7fda5b334700 1 -- 172.21.15.36:0/2219575450 shutdown_connections 2022-06-17T12:47:12.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.581+0000 7fda5b334700 1 --2- 172.21.15.36:0/2219575450 >> v2:172.21.15.70:3300/0 conn(0x7fda540b1190 0x7fda54141010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:12.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.581+0000 7fda5b334700 1 --2- 172.21.15.36:0/2219575450 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda540af540 0x7fda540b2860 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:12.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.581+0000 7fda5b334700 1 -- 172.21.15.36:0/2219575450 >> 172.21.15.36:0/2219575450 conn(0x7fda540614b0 msgr2=0x7fda54061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:12.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.581+0000 7fda5b334700 1 -- 172.21.15.36:0/2219575450 shutdown_connections 2022-06-17T12:47:12.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.581+0000 7fda5b334700 1 -- 172.21.15.36:0/2219575450 wait complete. 2022-06-17T12:47:12.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.581+0000 7fda5b334700 1 Processor -- start 2022-06-17T12:47:12.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.581+0000 7fda5b334700 1 -- start start 2022-06-17T12:47:12.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda5b334700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda540af540 0x7fda5413b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:12.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda5b334700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda540b1190 0x7fda540656a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:12.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda5b334700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda540b6660 0x7fda5410f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:12.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda5b334700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5413d460 con 0x7fda540b1190 2022-06-17T12:47:12.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda5b334700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5410ff10 con 0x7fda540af540 2022-06-17T12:47:12.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda5b334700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda54150460 con 0x7fda540b6660 2022-06-17T12:47:12.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda59b31700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda540b1190 0x7fda540656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:12.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda59b31700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda540b1190 0x7fda540656a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60082/0 (socket says 172.21.15.36:60082) 2022-06-17T12:47:12.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda59b31700 1 -- 172.21.15.36:0/4002765047 learned_addr learned my addr 172.21.15.36:0/4002765047 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:12.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda5ab33700 1 --2- 172.21.15.36:0/4002765047 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda540b6660 0x7fda5410f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:12.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda5a332700 1 --2- 172.21.15.36:0/4002765047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda540af540 0x7fda5413b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:12.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda59b31700 1 -- 172.21.15.36:0/4002765047 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda540b6660 msgr2=0x7fda5410f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:12.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda59b31700 1 --2- 172.21.15.36:0/4002765047 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda540b6660 0x7fda5410f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:12.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda59b31700 1 -- 172.21.15.36:0/4002765047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda540af540 msgr2=0x7fda5413b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:12.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda59b31700 1 --2- 172.21.15.36:0/4002765047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda540af540 0x7fda5413b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:12.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda59b31700 1 -- 172.21.15.36:0/4002765047 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4400b1e0 con 0x7fda540b1190 2022-06-17T12:47:12.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda59b31700 1 --2- 172.21.15.36:0/4002765047 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda540b1190 0x7fda540656a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fda44002f70 tx=0x7fda44014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:12.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda437fe700 1 -- 172.21.15.36:0/4002765047 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda4400d290 con 0x7fda540b1190 2022-06-17T12:47:12.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda437fe700 1 -- 172.21.15.36:0/4002765047 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda44013e90 con 0x7fda540b1190 2022-06-17T12:47:12.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda5b334700 1 -- 172.21.15.36:0/4002765047 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda54150190 con 0x7fda540b1190 2022-06-17T12:47:12.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda437fe700 1 -- 172.21.15.36:0/4002765047 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda4401e5f0 con 0x7fda540b1190 2022-06-17T12:47:12.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda5b334700 1 -- 172.21.15.36:0/4002765047 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda54065b90 con 0x7fda540b1190 2022-06-17T12:47:12.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda437fe700 1 -- 172.21.15.36:0/4002765047 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fda4401c070 con 0x7fda540b1190 2022-06-17T12:47:12.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda437fe700 1 --2- 172.21.15.36:0/4002765047 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fda34053920 0x7fda34055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:12.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda437fe700 1 -- 172.21.15.36:0/4002765047 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fda4406cac0 con 0x7fda540b1190 2022-06-17T12:47:12.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda5b334700 1 -- 172.21.15.36:0/4002765047 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda5410fd10 con 0x7fda540b1190 2022-06-17T12:47:12.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.585+0000 7fda5a332700 1 --2- 172.21.15.36:0/4002765047 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fda34053920 0x7fda34055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:12.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.589+0000 7fda5a332700 1 --2- 172.21.15.36:0/4002765047 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fda34053920 0x7fda34055de0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fda5000b000 tx=0x7fda50007160).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:12.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.593+0000 7fda437fe700 1 -- 172.21.15.36:0/4002765047 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fda44023020 con 0x7fda540b1190 2022-06-17T12:47:12.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.721+0000 7fda5b334700 1 -- 172.21.15.36:0/4002765047 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fda54140b70 con 0x7fda540b1190 2022-06-17T12:47:12.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.721+0000 7fda437fe700 1 -- 172.21.15.36:0/4002765047 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7fda440438c0 con 0x7fda540b1190 2022-06-17T12:47:12.747 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:47:12.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.725+0000 7fda5b334700 1 -- 172.21.15.36:0/4002765047 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fda34053920 msgr2=0x7fda34055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:12.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.725+0000 7fda5b334700 1 --2- 172.21.15.36:0/4002765047 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fda34053920 0x7fda34055de0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fda5000b000 tx=0x7fda50007160).stop 2022-06-17T12:47:12.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.725+0000 7fda5b334700 1 -- 172.21.15.36:0/4002765047 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda540b1190 msgr2=0x7fda540656a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:12.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.725+0000 7fda5b334700 1 --2- 172.21.15.36:0/4002765047 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda540b1190 0x7fda540656a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fda44002f70 tx=0x7fda44014000).stop 2022-06-17T12:47:12.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.725+0000 7fda5b334700 1 -- 172.21.15.36:0/4002765047 shutdown_connections 2022-06-17T12:47:12.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.725+0000 7fda5b334700 1 --2- 172.21.15.36:0/4002765047 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fda34053920 0x7fda34055de0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:12.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.725+0000 7fda5b334700 1 --2- 172.21.15.36:0/4002765047 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda540b6660 0x7fda5410f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:12.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.725+0000 7fda5b334700 1 --2- 172.21.15.36:0/4002765047 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda540b1190 0x7fda540656a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:12.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.725+0000 7fda5b334700 1 --2- 172.21.15.36:0/4002765047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda540af540 0x7fda5413b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:12.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.725+0000 7fda5b334700 1 -- 172.21.15.36:0/4002765047 >> 172.21.15.36:0/4002765047 conn(0x7fda540614b0 msgr2=0x7fda54144250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:12.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.725+0000 7fda5b334700 1 -- 172.21.15.36:0/4002765047 shutdown_connections 2022-06-17T12:47:12.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.725+0000 7fda5b334700 1 -- 172.21.15.36:0/4002765047 wait complete. 2022-06-17T12:47:12.783 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:47:12.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.913+0000 7f312ed04700 1 Processor -- start 2022-06-17T12:47:12.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.913+0000 7f312ed04700 1 -- start start 2022-06-17T12:47:12.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.913+0000 7f312ed04700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31280b18e0 0x7f3128143860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:12.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.913+0000 7f312ed04700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f31280b39a0 0x7f3128146d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:12.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.913+0000 7f312ed04700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31280b2b70 con 0x7f31280b6860 2022-06-17T12:47:12.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.913+0000 7f312ed04700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31280b6c40 con 0x7f31280b39a0 2022-06-17T12:47:12.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.913+0000 7f312ed04700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3128142d30 con 0x7f31280b18e0 2022-06-17T12:47:12.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.917+0000 7f312d501700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31280b18e0 0x7f3128143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:12.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.917+0000 7f312e503700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f31280b39a0 0x7f3128146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:12.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.917+0000 7f312d501700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31280b18e0 0x7f3128143860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:46648/0 (socket says 172.21.15.36:46648) 2022-06-17T12:47:12.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.917+0000 7f312d501700 1 -- 172.21.15.36:0/2472695472 learned_addr learned my addr 172.21.15.36:0/2472695472 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:12.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.917+0000 7f312e503700 1 -- 172.21.15.36:0/2472695472 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31280b18e0 msgr2=0x7f3128143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:12.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.917+0000 7f312e503700 1 --2- 172.21.15.36:0/2472695472 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31280b18e0 0x7f3128143860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:12.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.917+0000 7f312e503700 1 -- 172.21.15.36:0/2472695472 >> v1:172.21.15.70:6789/0 conn(0x7f31280b6860 legacy=0x7f3128143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:12.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.917+0000 7f312e503700 1 -- 172.21.15.36:0/2472695472 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31280b0d30 con 0x7f31280b39a0 2022-06-17T12:47:12.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.917+0000 7f312d501700 1 --2- 172.21.15.36:0/2472695472 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31280b18e0 0x7f3128143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:12.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.917+0000 7f312e503700 1 --2- 172.21.15.36:0/2472695472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31280b39a0 0x7f3128146d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f31200032f0 tx=0x7f31200091d0).ready entity=mon.1 client_cookie=93092b9db1fa9664 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:12.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.917+0000 7f312cd00700 1 -- 172.21.15.36:0/2472695472 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f312000b6c0 con 0x7f31280b39a0 2022-06-17T12:47:12.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.917+0000 7f312cd00700 1 -- 172.21.15.36:0/2472695472 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312000b820 con 0x7f31280b39a0 2022-06-17T12:47:12.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.917+0000 7f312cd00700 1 -- 172.21.15.36:0/2472695472 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f312000b990 con 0x7f31280b39a0 2022-06-17T12:47:12.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.917+0000 7f312ed04700 1 -- 172.21.15.36:0/2472695472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31280b39a0 msgr2=0x7f3128146d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:12.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.917+0000 7f312ed04700 1 --2- 172.21.15.36:0/2472695472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31280b39a0 0x7f3128146d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f31200032f0 tx=0x7f31200091d0).stop 2022-06-17T12:47:12.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.917+0000 7f312ed04700 1 -- 172.21.15.36:0/2472695472 shutdown_connections 2022-06-17T12:47:12.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.917+0000 7f312ed04700 1 --2- 172.21.15.36:0/2472695472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31280b39a0 0x7f3128146d40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:12.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.917+0000 7f312ed04700 1 --2- 172.21.15.36:0/2472695472 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31280b18e0 0x7f3128143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:12.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.917+0000 7f312ed04700 1 -- 172.21.15.36:0/2472695472 >> 172.21.15.36:0/2472695472 conn(0x7f31280614b0 msgr2=0x7f3128061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:12.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.917+0000 7f312ed04700 1 -- 172.21.15.36:0/2472695472 shutdown_connections 2022-06-17T12:47:12.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.917+0000 7f312ed04700 1 -- 172.21.15.36:0/2472695472 wait complete. 2022-06-17T12:47:12.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.921+0000 7f312ed04700 1 Processor -- start 2022-06-17T12:47:12.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.921+0000 7f312ed04700 1 -- start start 2022-06-17T12:47:12.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.921+0000 7f312ed04700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f31280b18e0 0x7f31281146f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:12.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.921+0000 7f312ed04700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31280b39a0 0x7f3128114be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:12.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.921+0000 7f312ed04700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31280b6860 0x7f31281400b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:12.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.921+0000 7f312ed04700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3128066810 con 0x7f31280b18e0 2022-06-17T12:47:12.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.921+0000 7f312ed04700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31281173c0 con 0x7f31280b6860 2022-06-17T12:47:12.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.921+0000 7f312ed04700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3128117120 con 0x7f31280b39a0 2022-06-17T12:47:12.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.921+0000 7f312dd02700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f31280b18e0 0x7f31281146f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:12.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.921+0000 7f312d501700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31280b39a0 0x7f3128114be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:12.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.921+0000 7f312dd02700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f31280b18e0 0x7f31281146f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60096/0 (socket says 172.21.15.36:60096) 2022-06-17T12:47:12.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.921+0000 7f312dd02700 1 -- 172.21.15.36:0/546978135 learned_addr learned my addr 172.21.15.36:0/546978135 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:12.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.921+0000 7f312e503700 1 --2- 172.21.15.36:0/546978135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31280b6860 0x7f31281400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:12.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.921+0000 7f312dd02700 1 -- 172.21.15.36:0/546978135 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31280b39a0 msgr2=0x7f3128114be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:12.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.921+0000 7f312dd02700 1 --2- 172.21.15.36:0/546978135 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31280b39a0 0x7f3128114be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:12.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.921+0000 7f312dd02700 1 -- 172.21.15.36:0/546978135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31280b6860 msgr2=0x7f31281400b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:12.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.921+0000 7f312dd02700 1 --2- 172.21.15.36:0/546978135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31280b6860 0x7f31281400b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:12.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.921+0000 7f312dd02700 1 -- 172.21.15.36:0/546978135 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c00b1e0 con 0x7f31280b18e0 2022-06-17T12:47:12.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.921+0000 7f312dd02700 1 --2- 172.21.15.36:0/546978135 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f31280b18e0 0x7f31281146f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f31280ae3c0 tx=0x7f3124007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:12.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.921+0000 7f311affd700 1 -- 172.21.15.36:0/546978135 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3124012400 con 0x7f31280b18e0 2022-06-17T12:47:12.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.921+0000 7f311affd700 1 -- 172.21.15.36:0/546978135 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3124012560 con 0x7f31280b18e0 2022-06-17T12:47:12.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.921+0000 7f312ed04700 1 -- 172.21.15.36:0/546978135 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3120009470 con 0x7f31280b18e0 2022-06-17T12:47:12.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.925+0000 7f311affd700 1 -- 172.21.15.36:0/546978135 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31240128c0 con 0x7f31280b18e0 2022-06-17T12:47:12.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.925+0000 7f312ed04700 1 -- 172.21.15.36:0/546978135 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3128113f20 con 0x7f31280b18e0 2022-06-17T12:47:12.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.925+0000 7f312ed04700 1 -- 172.21.15.36:0/546978135 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3128116f70 con 0x7f31280b18e0 2022-06-17T12:47:12.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.925+0000 7f311affd700 1 -- 172.21.15.36:0/546978135 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f3124019030 con 0x7f31280b18e0 2022-06-17T12:47:12.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.925+0000 7f311affd700 1 --2- 172.21.15.36:0/546978135 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f31080539b0 0x7f3108055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:12.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.925+0000 7f311affd700 1 -- 172.21.15.36:0/546978135 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f312406b2b0 con 0x7f31280b18e0 2022-06-17T12:47:12.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.925+0000 7f312d501700 1 --2- 172.21.15.36:0/546978135 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f31080539b0 0x7f3108055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:12.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.929+0000 7f312d501700 1 --2- 172.21.15.36:0/546978135 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f31080539b0 0x7f3108055e70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f311c00b000 tx=0x7f311c006e50).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:12.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:12.933+0000 7f311affd700 1 -- 172.21.15.36:0/546978135 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f31240420b0 con 0x7f31280b18e0 2022-06-17T12:47:13.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.057+0000 7f312ed04700 1 -- 172.21.15.36:0/546978135 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3128057410 con 0x7f31280b18e0 2022-06-17T12:47:13.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.057+0000 7f311affd700 1 -- 172.21.15.36:0/546978135 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f31240456d0 con 0x7f31280b18e0 2022-06-17T12:47:13.045 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:13.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.057+0000 7f312ed04700 1 -- 172.21.15.36:0/546978135 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f31080539b0 msgr2=0x7f3108055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:13.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.057+0000 7f312ed04700 1 --2- 172.21.15.36:0/546978135 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f31080539b0 0x7f3108055e70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f311c00b000 tx=0x7f311c006e50).stop 2022-06-17T12:47:13.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.057+0000 7f312ed04700 1 -- 172.21.15.36:0/546978135 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f31280b18e0 msgr2=0x7f31281146f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:13.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.057+0000 7f312ed04700 1 --2- 172.21.15.36:0/546978135 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f31280b18e0 0x7f31281146f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f31280ae3c0 tx=0x7f3124007c50).stop 2022-06-17T12:47:13.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.057+0000 7f312ed04700 1 -- 172.21.15.36:0/546978135 shutdown_connections 2022-06-17T12:47:13.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.057+0000 7f312ed04700 1 --2- 172.21.15.36:0/546978135 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f31080539b0 0x7f3108055e70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:13.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.057+0000 7f312ed04700 1 --2- 172.21.15.36:0/546978135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31280b6860 0x7f31281400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:13.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.057+0000 7f312ed04700 1 --2- 172.21.15.36:0/546978135 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31280b39a0 0x7f3128114be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:13.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.057+0000 7f312ed04700 1 --2- 172.21.15.36:0/546978135 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f31280b18e0 0x7f31281146f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:13.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.057+0000 7f312ed04700 1 -- 172.21.15.36:0/546978135 >> 172.21.15.36:0/546978135 conn(0x7f31280614b0 msgr2=0x7f3128141600 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:13.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.057+0000 7f312ed04700 1 -- 172.21.15.36:0/546978135 shutdown_connections 2022-06-17T12:47:13.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.057+0000 7f312ed04700 1 -- 172.21.15.36:0/546978135 wait complete. 2022-06-17T12:47:13.059 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:47:13.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.193+0000 7f5e2c8cb700 1 Processor -- start 2022-06-17T12:47:13.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.193+0000 7f5e2c8cb700 1 -- start start 2022-06-17T12:47:13.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.193+0000 7f5e2c8cb700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5e280b1e00 0x7f5e28067610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:13.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.193+0000 7f5e2c8cb700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e28067b00 con 0x7f5e28060920 2022-06-17T12:47:13.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.193+0000 7f5e2c8cb700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e280ad020 con 0x7f5e2813e8c0 2022-06-17T12:47:13.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.193+0000 7f5e2c8cb700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e280aff80 con 0x7f5e280b1e00 2022-06-17T12:47:13.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.193+0000 7f5e26d9d700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f5e28060920 0x7f5e280ad760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49402/0 (socket says 172.21.15.36:49402) 2022-06-17T12:47:13.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.193+0000 7f5e26d9d700 1 -- 172.21.15.36:0/482679117 learned_addr learned my addr 172.21.15.36:0/482679117 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:13.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e2659c700 1 --2- 172.21.15.36:0/482679117 >> v2:172.21.15.70:3300/0 conn(0x7f5e280b1e00 0x7f5e28067610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:13.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e25d9b700 1 -- 172.21.15.36:0/482679117 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2019609648 0 0) 0x7f5e28067b00 con 0x7f5e28060920 2022-06-17T12:47:13.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e25d9b700 1 -- 172.21.15.36:0/482679117 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e0c003260 con 0x7f5e28060920 2022-06-17T12:47:13.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e25d9b700 1 -- 172.21.15.36:0/482679117 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2590417479 0 0) 0x7f5e280ad020 con 0x7f5e2813e8c0 2022-06-17T12:47:13.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e25d9b700 1 -- 172.21.15.36:0/482679117 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e28067b00 con 0x7f5e2813e8c0 2022-06-17T12:47:13.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e25d9b700 1 -- 172.21.15.36:0/482679117 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3610151611 0 0) 0x7f5e0c003260 con 0x7f5e28060920 2022-06-17T12:47:13.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e25d9b700 1 -- 172.21.15.36:0/482679117 >> v2:172.21.15.70:3300/0 conn(0x7f5e280b1e00 msgr2=0x7f5e28067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:13.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e25d9b700 1 --2- 172.21.15.36:0/482679117 >> v2:172.21.15.70:3300/0 conn(0x7f5e280b1e00 0x7f5e28067610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:13.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e25d9b700 1 -- 172.21.15.36:0/482679117 >> v1:172.21.15.70:6789/0 conn(0x7f5e2813e8c0 legacy=0x7f5e2813ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:13.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e25d9b700 1 -- 172.21.15.36:0/482679117 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e280ad2c0 con 0x7f5e28060920 2022-06-17T12:47:13.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e25d9b700 1 -- 172.21.15.36:0/482679117 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f5e18001660 con 0x7f5e28060920 2022-06-17T12:47:13.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e25d9b700 1 -- 172.21.15.36:0/482679117 >> v1:172.21.15.36:6789/0 conn(0x7f5e28060920 legacy=0x7f5e280ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:13.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e25d9b700 1 --2- 172.21.15.36:0/482679117 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e0c0038e0 0x7f5e0c003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:13.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e25d9b700 1 --2- 172.21.15.36:0/482679117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e0c004500 0x7f5e0c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:13.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e25d9b700 1 --2- 172.21.15.36:0/482679117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5e0c004e70 0x7f5e0c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:13.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e25d9b700 1 -- 172.21.15.36:0/482679117 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e280aff80 con 0x7f5e0c0038e0 2022-06-17T12:47:13.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e25d9b700 1 -- 172.21.15.36:0/482679117 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c00bcf0 con 0x7f5e0c004500 2022-06-17T12:47:13.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e25d9b700 1 -- 172.21.15.36:0/482679117 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c00beb0 con 0x7f5e0c004e70 2022-06-17T12:47:13.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e26d9d700 1 --2- 172.21.15.36:0/482679117 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e0c0038e0 0x7f5e0c003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:13.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e2659c700 1 --2- 172.21.15.36:0/482679117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e0c004500 0x7f5e0c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:13.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e2759e700 1 --2- 172.21.15.36:0/482679117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5e0c004e70 0x7f5e0c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:13.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e26d9d700 1 -- 172.21.15.36:0/482679117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5e0c004e70 msgr2=0x7f5e0c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:13.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e26d9d700 1 --2- 172.21.15.36:0/482679117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5e0c004e70 0x7f5e0c00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:13.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e26d9d700 1 -- 172.21.15.36:0/482679117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e0c004500 msgr2=0x7f5e0c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:13.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e26d9d700 1 --2- 172.21.15.36:0/482679117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e0c004500 0x7f5e0c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:13.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e2759e700 1 -- 172.21.15.36:0/482679117 reap_dead start 2022-06-17T12:47:13.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e26d9d700 1 -- 172.21.15.36:0/482679117 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e0c003260 con 0x7f5e0c0038e0 2022-06-17T12:47:13.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e26d9d700 1 --2- 172.21.15.36:0/482679117 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e0c0038e0 0x7f5e0c003ce0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f5e18001430 tx=0x7f5e280b6a50).ready entity=mon.0 client_cookie=d8a0d313ea83ae7a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:13.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e25d9b700 1 -- 172.21.15.36:0/482679117 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e180071a0 con 0x7f5e0c0038e0 2022-06-17T12:47:13.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e25d9b700 1 -- 172.21.15.36:0/482679117 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e18007300 con 0x7f5e0c0038e0 2022-06-17T12:47:13.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e25d9b700 1 -- 172.21.15.36:0/482679117 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e1800c8f0 con 0x7f5e0c0038e0 2022-06-17T12:47:13.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e2c8cb700 1 -- 172.21.15.36:0/482679117 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e0c0038e0 msgr2=0x7f5e0c003ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:13.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e2c8cb700 1 --2- 172.21.15.36:0/482679117 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e0c0038e0 0x7f5e0c003ce0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f5e18001430 tx=0x7f5e280b6a50).stop 2022-06-17T12:47:13.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e2c8cb700 1 -- 172.21.15.36:0/482679117 shutdown_connections 2022-06-17T12:47:13.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e2c8cb700 1 --2- 172.21.15.36:0/482679117 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e0c0038e0 0x7f5e0c003ce0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:13.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.197+0000 7f5e2c8cb700 1 -- 172.21.15.36:0/482679117 >> 172.21.15.36:0/482679117 conn(0x7f5e280b64f0 msgr2=0x7f5e280b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:13.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.201+0000 7f5e2c8cb700 1 -- 172.21.15.36:0/482679117 shutdown_connections 2022-06-17T12:47:13.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.201+0000 7f5e2c8cb700 1 -- 172.21.15.36:0/482679117 wait complete. 2022-06-17T12:47:13.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.201+0000 7f5e2c8cb700 1 Processor -- start 2022-06-17T12:47:13.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.201+0000 7f5e2c8cb700 1 -- start start 2022-06-17T12:47:13.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.201+0000 7f5e2c8cb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e0c0038e0 0x7f5e28150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:13.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.205+0000 7f5e2c8cb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e28138100 0x7f5e281537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:13.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.205+0000 7f5e2c8cb700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5e28064d80 0x7f5e28065230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:13.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.205+0000 7f5e2c8cb700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e28154030 con 0x7f5e28138100 2022-06-17T12:47:13.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.205+0000 7f5e2659c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e28138100 0x7f5e281537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:13.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.205+0000 7f5e2659c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e28138100 0x7f5e281537e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60118/0 (socket says 172.21.15.36:60118) 2022-06-17T12:47:13.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.205+0000 7f5e2659c700 1 -- 172.21.15.36:0/2590699568 learned_addr learned my addr 172.21.15.36:0/2590699568 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:13.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.205+0000 7f5e2c8cb700 1 -- 172.21.15.36:0/2590699568 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e281542d0 con 0x7f5e0c0038e0 2022-06-17T12:47:13.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.205+0000 7f5e2c8cb700 1 -- 172.21.15.36:0/2590699568 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e28151030 con 0x7f5e28064d80 2022-06-17T12:47:13.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.205+0000 7f5e2759e700 1 --2- 172.21.15.36:0/2590699568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5e28064d80 0x7f5e28065230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:13.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.205+0000 7f5e26d9d700 1 --2- 172.21.15.36:0/2590699568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e0c0038e0 0x7f5e28150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:13.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.205+0000 7f5e2659c700 1 -- 172.21.15.36:0/2590699568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5e28064d80 msgr2=0x7f5e28065230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:13.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.205+0000 7f5e2659c700 1 --2- 172.21.15.36:0/2590699568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5e28064d80 0x7f5e28065230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:13.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.205+0000 7f5e2659c700 1 -- 172.21.15.36:0/2590699568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e0c0038e0 msgr2=0x7f5e28150b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:13.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.205+0000 7f5e2659c700 1 --2- 172.21.15.36:0/2590699568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e0c0038e0 0x7f5e28150b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:13.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.205+0000 7f5e2659c700 1 -- 172.21.15.36:0/2590699568 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e28153cd0 con 0x7f5e28138100 2022-06-17T12:47:13.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.205+0000 7f5e2659c700 1 --2- 172.21.15.36:0/2590699568 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e28138100 0x7f5e281537e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f5e1c00b000 tx=0x7f5e1c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:13.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.205+0000 7f5e17fff700 1 -- 172.21.15.36:0/2590699568 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e1c013680 con 0x7f5e28138100 2022-06-17T12:47:13.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.205+0000 7f5e2c8cb700 1 -- 172.21.15.36:0/2590699568 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e281377f0 con 0x7f5e28138100 2022-06-17T12:47:13.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.205+0000 7f5e17fff700 1 -- 172.21.15.36:0/2590699568 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1c000e90 con 0x7f5e28138100 2022-06-17T12:47:13.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.205+0000 7f5e2c8cb700 1 -- 172.21.15.36:0/2590699568 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e28137d90 con 0x7f5e28138100 2022-06-17T12:47:13.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.205+0000 7f5e17fff700 1 -- 172.21.15.36:0/2590699568 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e1c0139d0 con 0x7f5e28138100 2022-06-17T12:47:13.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.205+0000 7f5e2c8cb700 1 -- 172.21.15.36:0/2590699568 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e28060d00 con 0x7f5e28138100 2022-06-17T12:47:13.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.209+0000 7f5e17fff700 1 -- 172.21.15.36:0/2590699568 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f5e1c013b30 con 0x7f5e28138100 2022-06-17T12:47:13.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.209+0000 7f5e17fff700 1 --2- 172.21.15.36:0/2590699568 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5e040539d0 0x7f5e04055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:13.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.209+0000 7f5e17fff700 1 -- 172.21.15.36:0/2590699568 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5e1c06c6d0 con 0x7f5e28138100 2022-06-17T12:47:13.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.209+0000 7f5e26d9d700 1 --2- 172.21.15.36:0/2590699568 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5e040539d0 0x7f5e04055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:13.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.209+0000 7f5e26d9d700 1 --2- 172.21.15.36:0/2590699568 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5e040539d0 0x7f5e04055e90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f5e18006000 tx=0x7f5e18013000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:13.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.213+0000 7f5e17fff700 1 -- 172.21.15.36:0/2590699568 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5e1c043430 con 0x7f5e28138100 2022-06-17T12:47:13.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.341+0000 7f5e2c8cb700 1 -- 172.21.15.36:0/2590699568 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5e28057410 con 0x7f5e28138100 2022-06-17T12:47:13.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.341+0000 7f5e17fff700 1 -- 172.21.15.36:0/2590699568 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+10 (secure 0 0 0) 0x7f5e1c0180a0 con 0x7f5e28138100 2022-06-17T12:47:13.333 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:13.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.345+0000 7f5e2c8cb700 1 -- 172.21.15.36:0/2590699568 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5e040539d0 msgr2=0x7f5e04055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:13.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.345+0000 7f5e2c8cb700 1 --2- 172.21.15.36:0/2590699568 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5e040539d0 0x7f5e04055e90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f5e18006000 tx=0x7f5e18013000).stop 2022-06-17T12:47:13.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.345+0000 7f5e2c8cb700 1 -- 172.21.15.36:0/2590699568 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e28138100 msgr2=0x7f5e281537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:13.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.345+0000 7f5e2c8cb700 1 --2- 172.21.15.36:0/2590699568 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e28138100 0x7f5e281537e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f5e1c00b000 tx=0x7f5e1c006df0).stop 2022-06-17T12:47:13.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.345+0000 7f5e2c8cb700 1 -- 172.21.15.36:0/2590699568 shutdown_connections 2022-06-17T12:47:13.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.345+0000 7f5e2c8cb700 1 --2- 172.21.15.36:0/2590699568 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5e040539d0 0x7f5e04055e90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:13.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.345+0000 7f5e2c8cb700 1 --2- 172.21.15.36:0/2590699568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5e28064d80 0x7f5e28065230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:13.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.345+0000 7f5e2c8cb700 1 --2- 172.21.15.36:0/2590699568 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e28138100 0x7f5e281537e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:13.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.345+0000 7f5e2c8cb700 1 --2- 172.21.15.36:0/2590699568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e0c0038e0 0x7f5e28150b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:13.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.345+0000 7f5e2c8cb700 1 -- 172.21.15.36:0/2590699568 >> 172.21.15.36:0/2590699568 conn(0x7f5e280b64f0 msgr2=0x7f5e281328e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:13.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.345+0000 7f5e2c8cb700 1 -- 172.21.15.36:0/2590699568 shutdown_connections 2022-06-17T12:47:13.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.345+0000 7f5e2c8cb700 1 -- 172.21.15.36:0/2590699568 wait complete. 2022-06-17T12:47:13.342 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:47:13.504 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:47:13.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.461+0000 7feaa55eb700 1 Processor -- start 2022-06-17T12:47:13.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.461+0000 7feaa55eb700 1 -- start start 2022-06-17T12:47:13.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.465+0000 7feaa55eb700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7feaa00b39a0 0x7feaa0143860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:13.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.465+0000 7feaa55eb700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaa00b2b70 con 0x7feaa00b18e0 2022-06-17T12:47:13.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.465+0000 7feaa55eb700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaa00b0d30 con 0x7feaa00b6860 2022-06-17T12:47:13.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.465+0000 7feaa55eb700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaa00b6c40 con 0x7feaa00b39a0 2022-06-17T12:47:13.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.465+0000 7feaa4dea700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7feaa00b39a0 0x7feaa0143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:13.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.465+0000 7fea9f7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7feaa00b18e0 0x7feaa013fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49424/0 (socket says 172.21.15.36:49424) 2022-06-17T12:47:13.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.465+0000 7fea9f7fe700 1 -- 172.21.15.36:0/3545005758 learned_addr learned my addr 172.21.15.36:0/3545005758 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:13.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.465+0000 7fea9effd700 1 -- 172.21.15.36:0/3545005758 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3109956451 0 0) 0x7feaa00b2b70 con 0x7feaa00b18e0 2022-06-17T12:47:13.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.465+0000 7fea9effd700 1 -- 172.21.15.36:0/3545005758 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea80003260 con 0x7feaa00b18e0 2022-06-17T12:47:13.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.465+0000 7feaa4dea700 1 -- 172.21.15.36:0/3545005758 >> v1:172.21.15.70:6789/0 conn(0x7feaa00b6860 legacy=0x7feaa0143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:13.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.465+0000 7feaa4dea700 1 -- 172.21.15.36:0/3545005758 >> v1:172.21.15.36:6789/0 conn(0x7feaa00b18e0 legacy=0x7feaa013fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:13.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.465+0000 7feaa4dea700 1 -- 172.21.15.36:0/3545005758 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa00b0fd0 con 0x7feaa00b39a0 2022-06-17T12:47:13.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.465+0000 7feaa4dea700 1 --2- 172.21.15.36:0/3545005758 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaa00b39a0 0x7feaa0143860 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7fea980032f0 tx=0x7fea980091d0).ready entity=mon.0 client_cookie=d4e133c346996bdd server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:13.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.465+0000 7fea9effd700 1 -- 172.21.15.36:0/3545005758 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea9800b710 con 0x7feaa00b39a0 2022-06-17T12:47:13.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.465+0000 7fea9effd700 1 -- 172.21.15.36:0/3545005758 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9800b870 con 0x7feaa00b39a0 2022-06-17T12:47:13.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.465+0000 7fea9effd700 1 -- 172.21.15.36:0/3545005758 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea9800b9e0 con 0x7feaa00b39a0 2022-06-17T12:47:13.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.465+0000 7feaa55eb700 1 -- 172.21.15.36:0/3545005758 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaa00b39a0 msgr2=0x7feaa0143860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:13.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.465+0000 7feaa55eb700 1 --2- 172.21.15.36:0/3545005758 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaa00b39a0 0x7feaa0143860 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7fea980032f0 tx=0x7fea980091d0).stop 2022-06-17T12:47:13.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.465+0000 7feaa55eb700 1 -- 172.21.15.36:0/3545005758 shutdown_connections 2022-06-17T12:47:13.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.465+0000 7feaa55eb700 1 --2- 172.21.15.36:0/3545005758 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaa00b39a0 0x7feaa0143860 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:13.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.465+0000 7feaa55eb700 1 -- 172.21.15.36:0/3545005758 >> 172.21.15.36:0/3545005758 conn(0x7feaa00614b0 msgr2=0x7feaa0061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:13.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.465+0000 7feaa55eb700 1 -- 172.21.15.36:0/3545005758 shutdown_connections 2022-06-17T12:47:13.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.465+0000 7feaa55eb700 1 -- 172.21.15.36:0/3545005758 wait complete. 2022-06-17T12:47:13.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7feaa55eb700 1 Processor -- start 2022-06-17T12:47:13.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7feaa55eb700 1 -- start start 2022-06-17T12:47:13.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7feaa55eb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaa00b18e0 0x7feaa0066ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:13.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7feaa55eb700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaa00b39a0 0x7feaa0065910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:13.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7feaa55eb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaa00b6860 0x7feaa0148ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:13.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7feaa55eb700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa014a5b0 con 0x7feaa00b18e0 2022-06-17T12:47:13.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7feaa55eb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa0149390 con 0x7feaa00b6860 2022-06-17T12:47:13.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7feaa55eb700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa0149590 con 0x7feaa00b39a0 2022-06-17T12:47:13.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7fea9ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaa00b18e0 0x7feaa0066ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:13.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7fea9ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaa00b18e0 0x7feaa0066ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60130/0 (socket says 172.21.15.36:60130) 2022-06-17T12:47:13.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7fea9ffff700 1 -- 172.21.15.36:0/4009680182 learned_addr learned my addr 172.21.15.36:0/4009680182 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:13.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7fea9f7fe700 1 --2- 172.21.15.36:0/4009680182 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaa00b39a0 0x7feaa0065910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:13.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7feaa4dea700 1 --2- 172.21.15.36:0/4009680182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaa00b6860 0x7feaa0148ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:13.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7fea9ffff700 1 -- 172.21.15.36:0/4009680182 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaa00b39a0 msgr2=0x7feaa0065910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:13.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7fea9ffff700 1 --2- 172.21.15.36:0/4009680182 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaa00b39a0 0x7feaa0065910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:13.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7fea9ffff700 1 -- 172.21.15.36:0/4009680182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaa00b6860 msgr2=0x7feaa0148ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:13.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7fea9ffff700 1 --2- 172.21.15.36:0/4009680182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaa00b6860 0x7feaa0148ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:13.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7fea9ffff700 1 -- 172.21.15.36:0/4009680182 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea98009470 con 0x7feaa00b18e0 2022-06-17T12:47:13.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7feaa4dea700 1 --2- 172.21.15.36:0/4009680182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaa00b6860 0x7feaa0148ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:47:13.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7fea9ffff700 1 --2- 172.21.15.36:0/4009680182 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaa00b18e0 0x7feaa0066ba0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7feaa0065220 tx=0x7fea90007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:13.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7fea9d7fa700 1 -- 172.21.15.36:0/4009680182 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea9000ce20 con 0x7feaa00b18e0 2022-06-17T12:47:13.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7fea9d7fa700 1 -- 172.21.15.36:0/4009680182 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9000f470 con 0x7feaa00b18e0 2022-06-17T12:47:13.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7feaa55eb700 1 -- 172.21.15.36:0/4009680182 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa0149830 con 0x7feaa00b18e0 2022-06-17T12:47:13.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7fea9d7fa700 1 -- 172.21.15.36:0/4009680182 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea9000f7d0 con 0x7feaa00b18e0 2022-06-17T12:47:13.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.469+0000 7feaa55eb700 1 -- 172.21.15.36:0/4009680182 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa0146bb0 con 0x7feaa00b18e0 2022-06-17T12:47:13.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.473+0000 7feaa55eb700 1 -- 172.21.15.36:0/4009680182 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaa01499c0 con 0x7feaa00b18e0 2022-06-17T12:47:13.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.473+0000 7fea9d7fa700 1 -- 172.21.15.36:0/4009680182 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fea90019030 con 0x7feaa00b18e0 2022-06-17T12:47:13.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.473+0000 7fea9d7fa700 1 --2- 172.21.15.36:0/4009680182 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fea84053ad0 0x7fea84055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:13.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.473+0000 7fea9d7fa700 1 -- 172.21.15.36:0/4009680182 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fea9006b3c0 con 0x7feaa00b18e0 2022-06-17T12:47:13.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.473+0000 7fea9f7fe700 1 --2- 172.21.15.36:0/4009680182 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fea84053ad0 0x7fea84055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:13.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.473+0000 7fea9f7fe700 1 --2- 172.21.15.36:0/4009680182 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fea84053ad0 0x7fea84055f90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fea8800b000 tx=0x7fea88009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:13.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.477+0000 7fea9d7fa700 1 -- 172.21.15.36:0/4009680182 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fea900421c0 con 0x7feaa00b18e0 2022-06-17T12:47:13.602 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.601+0000 7feaa55eb700 1 -- 172.21.15.36:0/4009680182 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feaa0057410 con 0x7feaa00b18e0 2022-06-17T12:47:13.602 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.601+0000 7fea9d7fa700 1 -- 172.21.15.36:0/4009680182 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+10 (secure 0 0 0) 0x7fea900457e0 con 0x7feaa00b18e0 2022-06-17T12:47:13.602 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:13.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.605+0000 7feaa55eb700 1 -- 172.21.15.36:0/4009680182 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fea84053ad0 msgr2=0x7fea84055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:13.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.605+0000 7feaa55eb700 1 --2- 172.21.15.36:0/4009680182 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fea84053ad0 0x7fea84055f90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fea8800b000 tx=0x7fea88009040).stop 2022-06-17T12:47:13.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.605+0000 7feaa55eb700 1 -- 172.21.15.36:0/4009680182 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaa00b18e0 msgr2=0x7feaa0066ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:13.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.605+0000 7feaa55eb700 1 --2- 172.21.15.36:0/4009680182 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaa00b18e0 0x7feaa0066ba0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7feaa0065220 tx=0x7fea90007c50).stop 2022-06-17T12:47:13.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.605+0000 7feaa55eb700 1 -- 172.21.15.36:0/4009680182 shutdown_connections 2022-06-17T12:47:13.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.605+0000 7feaa55eb700 1 --2- 172.21.15.36:0/4009680182 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fea84053ad0 0x7fea84055f90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:13.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.605+0000 7feaa55eb700 1 --2- 172.21.15.36:0/4009680182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaa00b6860 0x7feaa0148ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:13.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.605+0000 7feaa55eb700 1 --2- 172.21.15.36:0/4009680182 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaa00b39a0 0x7feaa0065910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:13.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.605+0000 7feaa55eb700 1 --2- 172.21.15.36:0/4009680182 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaa00b18e0 0x7feaa0066ba0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:13.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.605+0000 7feaa55eb700 1 -- 172.21.15.36:0/4009680182 >> 172.21.15.36:0/4009680182 conn(0x7feaa00614b0 msgr2=0x7feaa005c550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:13.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.605+0000 7feaa55eb700 1 -- 172.21.15.36:0/4009680182 shutdown_connections 2022-06-17T12:47:13.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.605+0000 7feaa55eb700 1 -- 172.21.15.36:0/4009680182 wait complete. 2022-06-17T12:47:13.615 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:47:13.619 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:47:13.619 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:47:13.627 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:47:13.678 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:47:13.695 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:47:13.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f6336c5d700 1 Processor -- start 2022-06-17T12:47:13.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f6336c5d700 1 -- start start 2022-06-17T12:47:13.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f6336c5d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6330144c00 0x7f6330062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:13.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f6336c5d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f63300ad330 0x7f63300ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:13.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f6336c5d700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63300b0ba0 con 0x7f6330062770 2022-06-17T12:47:13.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f6336c5d700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63300ae820 con 0x7f63300ad330 2022-06-17T12:47:13.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f6336c5d700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6330061030 con 0x7f6330144c00 2022-06-17T12:47:13.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f63351fa700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f63300ad330 0x7f63300ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:13.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f63351fa700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f63300ad330 0x7f63300ad7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60142/0 (socket says 172.21.15.36:60142) 2022-06-17T12:47:13.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f63351fa700 1 -- 172.21.15.36:0/934843332 learned_addr learned my addr 172.21.15.36:0/934843332 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:13.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f632ffff700 1 --2- 172.21.15.36:0/934843332 >> v2:172.21.15.70:3300/0 conn(0x7f6330144c00 0x7f6330062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:13.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f63351fa700 1 -- 172.21.15.36:0/934843332 >> v2:172.21.15.70:3300/0 conn(0x7f6330144c00 msgr2=0x7f6330062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:13.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f63351fa700 1 --2- 172.21.15.36:0/934843332 >> v2:172.21.15.70:3300/0 conn(0x7f6330144c00 0x7f6330062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:13.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f63351fa700 1 -- 172.21.15.36:0/934843332 >> v1:172.21.15.70:6789/0 conn(0x7f6330062770 legacy=0x7f63301444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:13.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f63351fa700 1 -- 172.21.15.36:0/934843332 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6330147e90 con 0x7f63300ad330 2022-06-17T12:47:13.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f63351fa700 1 --2- 172.21.15.36:0/934843332 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f63300ad330 0x7f63300ad7c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f6330061170 tx=0x7f6328006eb0).ready entity=mon.0 client_cookie=6992d7d628e69a65 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:13.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f632f7fe700 1 -- 172.21.15.36:0/934843332 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f632800d610 con 0x7f63300ad330 2022-06-17T12:47:13.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f632f7fe700 1 -- 172.21.15.36:0/934843332 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6328000e90 con 0x7f63300ad330 2022-06-17T12:47:13.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f632f7fe700 1 -- 172.21.15.36:0/934843332 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f632800d9a0 con 0x7f63300ad330 2022-06-17T12:47:13.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f6336c5d700 1 -- 172.21.15.36:0/934843332 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f63300ad330 msgr2=0x7f63300ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:13.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f6336c5d700 1 --2- 172.21.15.36:0/934843332 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f63300ad330 0x7f63300ad7c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f6330061170 tx=0x7f6328006eb0).stop 2022-06-17T12:47:13.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f6336c5d700 1 -- 172.21.15.36:0/934843332 shutdown_connections 2022-06-17T12:47:13.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f6336c5d700 1 --2- 172.21.15.36:0/934843332 >> v2:172.21.15.70:3300/0 conn(0x7f6330144c00 0x7f6330062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:13.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f6336c5d700 1 --2- 172.21.15.36:0/934843332 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f63300ad330 0x7f63300ad7c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:13.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f6336c5d700 1 -- 172.21.15.36:0/934843332 >> 172.21.15.36:0/934843332 conn(0x7f63300b53e0 msgr2=0x7f63300b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:13.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f6336c5d700 1 -- 172.21.15.36:0/934843332 shutdown_connections 2022-06-17T12:47:13.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.849+0000 7f6336c5d700 1 -- 172.21.15.36:0/934843332 wait complete. 2022-06-17T12:47:13.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.853+0000 7f6336c5d700 1 Processor -- start 2022-06-17T12:47:13.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.853+0000 7f6336c5d700 1 -- start start 2022-06-17T12:47:13.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.853+0000 7f6336c5d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6330062770 0x7f63300662c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:13.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.853+0000 7f6336c5d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63300ad330 0x7f6330159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:13.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.853+0000 7f6336c5d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6330144c00 0x7f633015a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:13.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.853+0000 7f6336c5d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6330131860 con 0x7f6330062770 2022-06-17T12:47:13.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.853+0000 7f6336c5d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6330065b90 con 0x7f63300ad330 2022-06-17T12:47:13.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.853+0000 7f6336c5d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6330065e30 con 0x7f6330144c00 2022-06-17T12:47:13.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.853+0000 7f63349f9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6330062770 0x7f63300662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:13.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.853+0000 7f63349f9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6330062770 0x7f63300662c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60144/0 (socket says 172.21.15.36:60144) 2022-06-17T12:47:13.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.853+0000 7f632ffff700 1 --2- 172.21.15.36:0/2590346119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63300ad330 0x7f6330159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:13.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.853+0000 7f63349f9700 1 -- 172.21.15.36:0/2590346119 learned_addr learned my addr 172.21.15.36:0/2590346119 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:14.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.853+0000 7f63351fa700 1 --2- 172.21.15.36:0/2590346119 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6330144c00 0x7f633015a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:14.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.853+0000 7f63349f9700 1 -- 172.21.15.36:0/2590346119 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6330144c00 msgr2=0x7f633015a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:14.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.853+0000 7f63349f9700 1 --2- 172.21.15.36:0/2590346119 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6330144c00 0x7f633015a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:14.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.853+0000 7f63349f9700 1 -- 172.21.15.36:0/2590346119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63300ad330 msgr2=0x7f6330159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:14.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.853+0000 7f63349f9700 1 --2- 172.21.15.36:0/2590346119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63300ad330 0x7f6330159e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:14.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.853+0000 7f63349f9700 1 -- 172.21.15.36:0/2590346119 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632800b1e0 con 0x7f6330062770 2022-06-17T12:47:14.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.853+0000 7f63349f9700 1 --2- 172.21.15.36:0/2590346119 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6330062770 0x7f63300662c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f633005d270 tx=0x7f6318007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:14.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.853+0000 7f632dffb700 1 -- 172.21.15.36:0/2590346119 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6318012400 con 0x7f6330062770 2022-06-17T12:47:14.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.857+0000 7f632dffb700 1 -- 172.21.15.36:0/2590346119 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6318012560 con 0x7f6330062770 2022-06-17T12:47:14.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.857+0000 7f6336c5d700 1 -- 172.21.15.36:0/2590346119 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6330132d20 con 0x7f6330062770 2022-06-17T12:47:14.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.857+0000 7f632dffb700 1 -- 172.21.15.36:0/2590346119 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63180128c0 con 0x7f6330062770 2022-06-17T12:47:14.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.857+0000 7f6336c5d700 1 -- 172.21.15.36:0/2590346119 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63300667b0 con 0x7f6330062770 2022-06-17T12:47:14.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.857+0000 7f6336c5d700 1 -- 172.21.15.36:0/2590346119 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6330066990 con 0x7f6330062770 2022-06-17T12:47:14.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.857+0000 7f632dffb700 1 -- 172.21.15.36:0/2590346119 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6318019030 con 0x7f6330062770 2022-06-17T12:47:14.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.857+0000 7f632dffb700 1 --2- 172.21.15.36:0/2590346119 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6308053ad0 0x7f6308055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:14.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.857+0000 7f632ffff700 1 --2- 172.21.15.36:0/2590346119 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6308053ad0 0x7f6308055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:14.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.857+0000 7f632dffb700 1 -- 172.21.15.36:0/2590346119 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f631806b3e0 con 0x7f6330062770 2022-06-17T12:47:14.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.857+0000 7f632ffff700 1 --2- 172.21.15.36:0/2590346119 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6308053ad0 0x7f6308055f90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f632000b000 tx=0x7f63200091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:14.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:13.865+0000 7f632dffb700 1 -- 172.21.15.36:0/2590346119 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6330066990 con 0x7f6330062770 2022-06-17T12:47:14.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:14.025+0000 7f6336c5d700 1 -- 172.21.15.36:0/2590346119 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6330132eb0 con 0x7f6330062770 2022-06-17T12:47:14.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:14.025+0000 7f632dffb700 1 -- 172.21.15.36:0/2590346119 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2134 (secure 0 0 0) 0x7f63180420d0 con 0x7f6330062770 2022-06-17T12:47:14.040 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:14.041 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/4243909763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":4243909763},{"type":"v1","addr":"172.21.15.36:6835","nonce":4243909763}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:46:52.253609+0000","modified":"2022-06-17T12:46:54.268223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/3425584732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3425584732},{"type":"v1","addr":"172.21.15.70:6833","nonce":3425584732}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-06-17T12:47:14.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:14.029+0000 7f6336c5d700 1 -- 172.21.15.36:0/2590346119 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6308053ad0 msgr2=0x7f6308055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:14.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:14.029+0000 7f6336c5d700 1 --2- 172.21.15.36:0/2590346119 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6308053ad0 0x7f6308055f90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f632000b000 tx=0x7f63200091d0).stop 2022-06-17T12:47:14.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:14.029+0000 7f6336c5d700 1 -- 172.21.15.36:0/2590346119 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6330062770 msgr2=0x7f63300662c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:14.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:14.029+0000 7f6336c5d700 1 --2- 172.21.15.36:0/2590346119 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6330062770 0x7f63300662c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f633005d270 tx=0x7f6318007c50).stop 2022-06-17T12:47:14.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:14.029+0000 7f6336c5d700 1 -- 172.21.15.36:0/2590346119 shutdown_connections 2022-06-17T12:47:14.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:14.029+0000 7f6336c5d700 1 --2- 172.21.15.36:0/2590346119 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6308053ad0 0x7f6308055f90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:14.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:14.029+0000 7f6336c5d700 1 --2- 172.21.15.36:0/2590346119 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6330144c00 0x7f633015a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:14.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:14.029+0000 7f6336c5d700 1 --2- 172.21.15.36:0/2590346119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63300ad330 0x7f6330159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:14.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:14.029+0000 7f6336c5d700 1 --2- 172.21.15.36:0/2590346119 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6330062770 0x7f63300662c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:14.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:14.029+0000 7f6336c5d700 1 -- 172.21.15.36:0/2590346119 >> 172.21.15.36:0/2590346119 conn(0x7f63300b53e0 msgr2=0x7f63300b1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:14.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:14.029+0000 7f6336c5d700 1 -- 172.21.15.36:0/2590346119 shutdown_connections 2022-06-17T12:47:14.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:14.029+0000 7f6336c5d700 1 -- 172.21.15.36:0/2590346119 wait complete. 2022-06-17T12:47:14.065 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 7 2022-06-17T12:47:14.298 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:46:52.253609+0000', 'modified': '2022-06-17T12:46:54.268223+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4223}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4223': {'gid': 4223, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6833/3425584732', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 3425584732}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 3425584732}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:47:14.299 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:47:14.300 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-17T12:47:14.420 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":4223,"want_state":"up:active","state":"up:active","rank_uptime":22.166071196000001,"mdsmap_epoch":7,"osdmap_epoch":26,"osdmap_epoch_barrier":26,"uptime":28.614132865999999} 2022-06-17T12:47:14.421 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 4223, "mdsmap_epoch": 7, "osdmap_epoch": 26, "osdmap_epoch_barrier": 26, "rank_uptime": 22.166071196, "state": "up:active", "uptime": 28.614132866, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:47:14.422 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:47:14.423 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-06-17T12:47:14.424 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi136.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-06-17T12:47:14.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:14.425 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:47:14.449 DEBUG:teuthology.orchestra.run.smithi136:> sudo modprobe fuse 2022-06-17T12:47:14.613 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:47:14.614 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:47:14.664 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:47:14.666 INFO:teuthology.orchestra.run.smithi136.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:47:14.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:14.667 DEBUG:teuthology.orchestra.run.smithi136:> ls /sys/fs/fuse/connections 2022-06-17T12:47:14.780 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:47:14.785 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:47:14.819 DEBUG:teuthology.orchestra.run.smithi136:> sudo modprobe fuse 2022-06-17T12:47:14.843 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:47:14.844 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:47:14.934 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:2022-06-17T12:47:14.918+0000 7f9d7d7a1080 -1 init, newargv = 0x56057801b6f0 newargc=9ceph-fuse[21691]: starting ceph client 2022-06-17T12:47:14.936 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr: 2022-06-17T12:47:14.937 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:ceph-fuse[21691]: starting fuse 2022-06-17T12:47:14.951 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:47:14.953 INFO:teuthology.orchestra.run.smithi136.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:47:14.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:14.957 DEBUG:teuthology.orchestra.run.smithi136:> ls /sys/fs/fuse/connections 2022-06-17T12:47:14.969 INFO:teuthology.orchestra.run.smithi136.stdout:52 2022-06-17T12:47:14.970 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:47:14.971 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:14.972 DEBUG:teuthology.orchestra.run.smithi136:> sudo python3 -c ' 2022-06-17T12:47:14.972 DEBUG:teuthology.orchestra.run.smithi136:> import glob 2022-06-17T12:47:14.973 DEBUG:teuthology.orchestra.run.smithi136:> import re 2022-06-17T12:47:15.001 DEBUG:teuthology.orchestra.run.smithi136:> import os 2022-06-17T12:47:15.002 DEBUG:teuthology.orchestra.run.smithi136:> import subprocess 2022-06-17T12:47:15.002 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:47:15.002 DEBUG:teuthology.orchestra.run.smithi136:> def find_socket(client_name): 2022-06-17T12:47:15.003 DEBUG:teuthology.orchestra.run.smithi136:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-06-17T12:47:15.003 DEBUG:teuthology.orchestra.run.smithi136:> files = glob.glob(asok_path) 2022-06-17T12:47:15.003 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:47:15.004 DEBUG:teuthology.orchestra.run.smithi136:> # Given a non-glob path, it better be there 2022-06-17T12:47:15.004 DEBUG:teuthology.orchestra.run.smithi136:> if "*" not in asok_path: 2022-06-17T12:47:15.004 DEBUG:teuthology.orchestra.run.smithi136:> assert(len(files) == 1) 2022-06-17T12:47:15.005 DEBUG:teuthology.orchestra.run.smithi136:> return files[0] 2022-06-17T12:47:15.005 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:47:15.005 DEBUG:teuthology.orchestra.run.smithi136:> for f in files: 2022-06-17T12:47:15.006 DEBUG:teuthology.orchestra.run.smithi136:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:47:15.015 DEBUG:teuthology.orchestra.run.smithi136:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:47:15.015 DEBUG:teuthology.orchestra.run.smithi136:> return f 2022-06-17T12:47:15.016 DEBUG:teuthology.orchestra.run.smithi136:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:47:15.016 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:47:15.016 DEBUG:teuthology.orchestra.run.smithi136:> print(find_socket("client.1")) 2022-06-17T12:47:15.017 DEBUG:teuthology.orchestra.run.smithi136:> ' 2022-06-17T12:47:15.089 INFO:teuthology.orchestra.run.smithi136.stdout:/var/run/ceph/ceph-client.1.21691.asok 2022-06-17T12:47:15.108 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.21691.asok 2022-06-17T12:47:15.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:15.109 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.21691.asok status 2022-06-17T12:47:15.246 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2022-06-17T12:47:15.248 INFO:teuthology.orchestra.run.smithi136.stdout: "metadata": { 2022-06-17T12:47:15.248 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:47:15.248 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:47:15.249 INFO:teuthology.orchestra.run.smithi136.stdout: "entity_id": "1", 2022-06-17T12:47:15.249 INFO:teuthology.orchestra.run.smithi136.stdout: "hostname": "smithi136", 2022-06-17T12:47:15.252 INFO:teuthology.orchestra.run.smithi136.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-06-17T12:47:15.252 INFO:teuthology.orchestra.run.smithi136.stdout: "pid": "21691", 2022-06-17T12:47:15.252 INFO:teuthology.orchestra.run.smithi136.stdout: "root": "/" 2022-06-17T12:47:15.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:47:15.253 INFO:teuthology.orchestra.run.smithi136.stdout: "dentry_count": 0, 2022-06-17T12:47:15.253 INFO:teuthology.orchestra.run.smithi136.stdout: "dentry_pinned_count": 0, 2022-06-17T12:47:15.253 INFO:teuthology.orchestra.run.smithi136.stdout: "id": 4683, 2022-06-17T12:47:15.253 INFO:teuthology.orchestra.run.smithi136.stdout: "inst": { 2022-06-17T12:47:15.253 INFO:teuthology.orchestra.run.smithi136.stdout: "name": { 2022-06-17T12:47:15.254 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "client", 2022-06-17T12:47:15.254 INFO:teuthology.orchestra.run.smithi136.stdout: "num": 4683 2022-06-17T12:47:15.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:47:15.255 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": { 2022-06-17T12:47:15.255 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "v1", 2022-06-17T12:47:15.255 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": "172.21.15.136:0", 2022-06-17T12:47:15.260 INFO:teuthology.orchestra.run.smithi136.stdout: "nonce": 649038136 2022-06-17T12:47:15.260 INFO:teuthology.orchestra.run.smithi136.stdout: } 2022-06-17T12:47:15.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:47:15.260 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": { 2022-06-17T12:47:15.261 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "v1", 2022-06-17T12:47:15.261 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": "172.21.15.136:0", 2022-06-17T12:47:15.261 INFO:teuthology.orchestra.run.smithi136.stdout: "nonce": 649038136 2022-06-17T12:47:15.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:47:15.261 INFO:teuthology.orchestra.run.smithi136.stdout: "inst_str": "client.4683 172.21.15.136:0/649038136", 2022-06-17T12:47:15.262 INFO:teuthology.orchestra.run.smithi136.stdout: "addr_str": "172.21.15.136:0/649038136", 2022-06-17T12:47:15.262 INFO:teuthology.orchestra.run.smithi136.stdout: "inode_count": 1, 2022-06-17T12:47:15.262 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_epoch": 7, 2022-06-17T12:47:15.262 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_epoch": 26, 2022-06-17T12:47:15.263 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:47:15.263 INFO:teuthology.orchestra.run.smithi136.stdout: "blacklisted": false, 2022-06-17T12:47:15.263 INFO:teuthology.orchestra.run.smithi136.stdout: "fs_name": "cephfs" 2022-06-17T12:47:15.263 INFO:teuthology.orchestra.run.smithi136.stdout:} 2022-06-17T12:47:15.283 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:47:15.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.417+0000 7f70788b9700 1 Processor -- start 2022-06-17T12:47:15.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.417+0000 7f70788b9700 1 -- start start 2022-06-17T12:47:15.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.417+0000 7f70788b9700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7074066bc0 0x7f7074143880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:15.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.417+0000 7f70788b9700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70740b1570 con 0x7f70740afe30 2022-06-17T12:47:15.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.417+0000 7f70788b9700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7074066fa0 con 0x7f7074067510 2022-06-17T12:47:15.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.417+0000 7f70788b9700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70740acb10 con 0x7f7074066bc0 2022-06-17T12:47:15.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.417+0000 7f7072ffd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f70740afe30 0x7f7074143d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49450/0 (socket says 172.21.15.36:49450) 2022-06-17T12:47:15.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.417+0000 7f7072ffd700 1 -- 172.21.15.36:0/4219869910 learned_addr learned my addr 172.21.15.36:0/4219869910 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:15.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.421+0000 7f70737fe700 1 --2- 172.21.15.36:0/4219869910 >> v2:172.21.15.36:3300/0 conn(0x7f7074066bc0 0x7f7074143880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:15.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.421+0000 7f70727fc700 1 -- 172.21.15.36:0/4219869910 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1621245046 0 0) 0x7f70740b1570 con 0x7f70740afe30 2022-06-17T12:47:15.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.421+0000 7f70727fc700 1 -- 172.21.15.36:0/4219869910 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7060003260 con 0x7f70740afe30 2022-06-17T12:47:15.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.421+0000 7f70727fc700 1 -- 172.21.15.36:0/4219869910 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1070221280 0 0) 0x7f7060003260 con 0x7f70740afe30 2022-06-17T12:47:15.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.421+0000 7f70737fe700 1 -- 172.21.15.36:0/4219869910 >> v1:172.21.15.70:6789/0 conn(0x7f7074067510 legacy=0x7f70740addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:15.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.421+0000 7f70737fe700 1 -- 172.21.15.36:0/4219869910 >> v1:172.21.15.36:6789/0 conn(0x7f70740afe30 legacy=0x7f7074143d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:15.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.421+0000 7f70737fe700 1 -- 172.21.15.36:0/4219869910 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70740b17a0 con 0x7f7074066bc0 2022-06-17T12:47:15.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.421+0000 7f70737fe700 1 --2- 172.21.15.36:0/4219869910 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7074066bc0 0x7f7074143880 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f7064001800 tx=0x7f7068006df0).ready entity=mon.0 client_cookie=398e66ed01d5c5d8 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:15.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.421+0000 7f70727fc700 1 -- 172.21.15.36:0/4219869910 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7064001660 con 0x7f7074066bc0 2022-06-17T12:47:15.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.421+0000 7f70727fc700 1 -- 172.21.15.36:0/4219869910 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7068000e90 con 0x7f7074066bc0 2022-06-17T12:47:15.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.421+0000 7f70727fc700 1 -- 172.21.15.36:0/4219869910 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7068011400 con 0x7f7074066bc0 2022-06-17T12:47:15.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.421+0000 7f70788b9700 1 -- 172.21.15.36:0/4219869910 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7074066bc0 msgr2=0x7f7074143880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:15.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.421+0000 7f70788b9700 1 --2- 172.21.15.36:0/4219869910 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7074066bc0 0x7f7074143880 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f7064001800 tx=0x7f7068006df0).stop 2022-06-17T12:47:15.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.421+0000 7f70788b9700 1 -- 172.21.15.36:0/4219869910 shutdown_connections 2022-06-17T12:47:15.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.421+0000 7f70788b9700 1 --2- 172.21.15.36:0/4219869910 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7074066bc0 0x7f7074143880 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:15.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.421+0000 7f70788b9700 1 -- 172.21.15.36:0/4219869910 >> 172.21.15.36:0/4219869910 conn(0x7f707405f770 msgr2=0x7f707405fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:15.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.421+0000 7f70788b9700 1 -- 172.21.15.36:0/4219869910 shutdown_connections 2022-06-17T12:47:15.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.421+0000 7f70788b9700 1 -- 172.21.15.36:0/4219869910 wait complete. 2022-06-17T12:47:15.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.421+0000 7f70788b9700 1 Processor -- start 2022-06-17T12:47:15.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.421+0000 7f70788b9700 1 -- start start 2022-06-17T12:47:15.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.425+0000 7f70788b9700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7074066bc0 0x7f70740a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:15.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.425+0000 7f70788b9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7074067510 0x7f7074141a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:15.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.425+0000 7f70788b9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f70740afe30 0x7f70740ab710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:15.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.425+0000 7f70788b9700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70740a0310 con 0x7f70740afe30 2022-06-17T12:47:15.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.425+0000 7f70788b9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707409f170 con 0x7f7074067510 2022-06-17T12:47:15.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.425+0000 7f70788b9700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707409f410 con 0x7f7074066bc0 2022-06-17T12:47:15.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.425+0000 7f7073fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f70740afe30 0x7f70740ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:15.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.425+0000 7f7072ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7074067510 0x7f7074141a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:15.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.425+0000 7f70737fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7074066bc0 0x7f70740a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:15.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.425+0000 7f7072ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7074067510 0x7f7074141a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:52310/0 (socket says 172.21.15.36:52310) 2022-06-17T12:47:15.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.425+0000 7f7072ffd700 1 -- 172.21.15.36:0/1502937692 learned_addr learned my addr 172.21.15.36:0/1502937692 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:15.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.425+0000 7f7072ffd700 1 -- 172.21.15.36:0/1502937692 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7074066bc0 msgr2=0x7f70740a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:15.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.425+0000 7f7072ffd700 1 --2- 172.21.15.36:0/1502937692 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7074066bc0 0x7f70740a0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:15.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.425+0000 7f7072ffd700 1 -- 172.21.15.36:0/1502937692 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f70740afe30 msgr2=0x7f70740ab710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:15.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.425+0000 7f7072ffd700 1 --2- 172.21.15.36:0/1502937692 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f70740afe30 0x7f70740ab710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:15.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.425+0000 7f7072ffd700 1 -- 172.21.15.36:0/1502937692 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706800b180 con 0x7f7074067510 2022-06-17T12:47:15.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.425+0000 7f7072ffd700 1 --2- 172.21.15.36:0/1502937692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7074067510 0x7f7074141a40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f7064001840 tx=0x7f7064001720).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:15.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.425+0000 7f7070ff9700 1 -- 172.21.15.36:0/1502937692 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7064007170 con 0x7f7074067510 2022-06-17T12:47:15.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.425+0000 7f7070ff9700 1 -- 172.21.15.36:0/1502937692 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70640072d0 con 0x7f7074067510 2022-06-17T12:47:15.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.425+0000 7f70788b9700 1 -- 172.21.15.36:0/1502937692 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7074155aa0 con 0x7f7074067510 2022-06-17T12:47:15.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.425+0000 7f7070ff9700 1 -- 172.21.15.36:0/1502937692 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7064004710 con 0x7f7074067510 2022-06-17T12:47:15.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.425+0000 7f70788b9700 1 -- 172.21.15.36:0/1502937692 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f707409ebf0 con 0x7f7074067510 2022-06-17T12:47:15.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.429+0000 7f70788b9700 1 -- 172.21.15.36:0/1502937692 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70740a2680 con 0x7f7074067510 2022-06-17T12:47:15.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.429+0000 7f7070ff9700 1 -- 172.21.15.36:0/1502937692 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f706401b070 con 0x7f7074067510 2022-06-17T12:47:15.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.429+0000 7f7070ff9700 1 --2- 172.21.15.36:0/1502937692 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7054053a60 0x7f7054055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:15.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.429+0000 7f7070ff9700 1 -- 172.21.15.36:0/1502937692 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f706406b580 con 0x7f7074067510 2022-06-17T12:47:15.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.429+0000 7f70737fe700 1 --2- 172.21.15.36:0/1502937692 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7054053a60 0x7f7054055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:15.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.429+0000 7f70737fe700 1 --2- 172.21.15.36:0/1502937692 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7054053a60 0x7f7054055f20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f7068019000 tx=0x7f70740adc20).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:15.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.437+0000 7f7070ff9700 1 -- 172.21.15.36:0/1502937692 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f70640422e0 con 0x7f7074067510 2022-06-17T12:47:15.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.565+0000 7f70788b9700 1 -- 172.21.15.36:0/1502937692 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f707409f5d0 con 0x7f7074067510 2022-06-17T12:47:15.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.569+0000 7f7070ff9700 1 -- 172.21.15.36:0/1502937692 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f7064019020 con 0x7f7074067510 2022-06-17T12:47:15.561 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:15.561 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":26,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:46:53.258075+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:46:47.465328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-17T12:46:49.893338+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:47:15.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.573+0000 7f70788b9700 1 -- 172.21.15.36:0/1502937692 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7054053a60 msgr2=0x7f7054055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:15.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.573+0000 7f70788b9700 1 --2- 172.21.15.36:0/1502937692 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7054053a60 0x7f7054055f20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f7068019000 tx=0x7f70740adc20).stop 2022-06-17T12:47:15.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.573+0000 7f70788b9700 1 -- 172.21.15.36:0/1502937692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7074067510 msgr2=0x7f7074141a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:15.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.573+0000 7f70788b9700 1 --2- 172.21.15.36:0/1502937692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7074067510 0x7f7074141a40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f7064001840 tx=0x7f7064001720).stop 2022-06-17T12:47:15.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.573+0000 7f70788b9700 1 -- 172.21.15.36:0/1502937692 shutdown_connections 2022-06-17T12:47:15.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.573+0000 7f70788b9700 1 --2- 172.21.15.36:0/1502937692 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7054053a60 0x7f7054055f20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:15.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.573+0000 7f70788b9700 1 --2- 172.21.15.36:0/1502937692 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f70740afe30 0x7f70740ab710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:15.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.573+0000 7f70788b9700 1 --2- 172.21.15.36:0/1502937692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7074067510 0x7f7074141a40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:15.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.573+0000 7f70788b9700 1 --2- 172.21.15.36:0/1502937692 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7074066bc0 0x7f70740a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:15.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.573+0000 7f70788b9700 1 -- 172.21.15.36:0/1502937692 >> 172.21.15.36:0/1502937692 conn(0x7f707405f770 msgr2=0x7f70740b3670 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:15.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.573+0000 7f70788b9700 1 -- 172.21.15.36:0/1502937692 shutdown_connections 2022-06-17T12:47:15.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.573+0000 7f70788b9700 1 -- 172.21.15.36:0/1502937692 wait complete. 2022-06-17T12:47:15.597 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:46:47.465328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:46:49.893338+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:47:15.671 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:47:16.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6fb0dc9700 1 Processor -- start 2022-06-17T12:47:16.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6fb0dc9700 1 -- start start 2022-06-17T12:47:16.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6fb0dc9700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6fac0b27b0 0x7f6fac14b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:16.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6fb0dc9700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fac14b870 con 0x7f6fac147bd0 2022-06-17T12:47:16.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6fb0dc9700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fac0ac500 con 0x7f6fac060ca0 2022-06-17T12:47:16.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6fb0dc9700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fac0ace90 con 0x7f6fac0b27b0 2022-06-17T12:47:16.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6fabfff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f6fac147bd0 0x7f6fac1442d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49464/0 (socket says 172.21.15.36:49464) 2022-06-17T12:47:16.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6fabfff700 1 -- 172.21.15.36:0/2321682138 learned_addr learned my addr 172.21.15.36:0/2321682138 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:16.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6fab7fe700 1 --2- 172.21.15.36:0/2321682138 >> v2:172.21.15.70:3300/0 conn(0x7f6fac0b27b0 0x7f6fac14b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:16.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6faa7fc700 1 -- 172.21.15.36:0/2321682138 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968098711 0 0) 0x7f6fac14b870 con 0x7f6fac147bd0 2022-06-17T12:47:16.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6faa7fc700 1 -- 172.21.15.36:0/2321682138 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f8c003260 con 0x7f6fac147bd0 2022-06-17T12:47:16.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6faa7fc700 1 -- 172.21.15.36:0/2321682138 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2186937887 0 0) 0x7f6f8c003260 con 0x7f6fac147bd0 2022-06-17T12:47:16.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6faa7fc700 1 -- 172.21.15.36:0/2321682138 >> v2:172.21.15.70:3300/0 conn(0x7f6fac0b27b0 msgr2=0x7f6fac14b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:16.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6faa7fc700 1 --2- 172.21.15.36:0/2321682138 >> v2:172.21.15.70:3300/0 conn(0x7f6fac0b27b0 0x7f6fac14b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6faa7fc700 1 -- 172.21.15.36:0/2321682138 >> v1:172.21.15.70:6789/0 conn(0x7f6fac060ca0 legacy=0x7f6fac0ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:16.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6faa7fc700 1 -- 172.21.15.36:0/2321682138 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fac0b2b90 con 0x7f6fac147bd0 2022-06-17T12:47:16.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6faa7fc700 1 -- 172.21.15.36:0/2321682138 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f6fa40019e0 con 0x7f6fac147bd0 2022-06-17T12:47:16.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6faa7fc700 1 -- 172.21.15.36:0/2321682138 >> v1:172.21.15.36:6789/0 conn(0x7f6fac147bd0 legacy=0x7f6fac1442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:16.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6faa7fc700 1 --2- 172.21.15.36:0/2321682138 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f8c003750 0x7f6f8c003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:16.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6faa7fc700 1 --2- 172.21.15.36:0/2321682138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f8c004500 0x7f6f8c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:16.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6faa7fc700 1 --2- 172.21.15.36:0/2321682138 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6f8c004e70 0x7f6f8c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:16.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6faa7fc700 1 -- 172.21.15.36:0/2321682138 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac0ace90 con 0x7f6f8c004e70 2022-06-17T12:47:16.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6faa7fc700 1 -- 172.21.15.36:0/2321682138 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c00bc70 con 0x7f6f8c004500 2022-06-17T12:47:16.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6faa7fc700 1 -- 172.21.15.36:0/2321682138 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c00be30 con 0x7f6f8c003750 2022-06-17T12:47:16.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6fabfff700 1 --2- 172.21.15.36:0/2321682138 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6f8c004e70 0x7f6f8c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:16.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6faaffd700 1 --2- 172.21.15.36:0/2321682138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f8c004500 0x7f6f8c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:16.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6fab7fe700 1 --2- 172.21.15.36:0/2321682138 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f8c003750 0x7f6f8c003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:16.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6fabfff700 1 -- 172.21.15.36:0/2321682138 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f8c003750 msgr2=0x7f6f8c003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:16.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6fabfff700 1 --2- 172.21.15.36:0/2321682138 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f8c003750 0x7f6f8c003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6fabfff700 1 -- 172.21.15.36:0/2321682138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f8c004500 msgr2=0x7f6f8c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:16.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6fabfff700 1 --2- 172.21.15.36:0/2321682138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f8c004500 0x7f6f8c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6fabfff700 1 -- 172.21.15.36:0/2321682138 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8c003260 con 0x7f6f8c004e70 2022-06-17T12:47:16.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6fabfff700 1 -- 172.21.15.36:0/2321682138 reap_dead start 2022-06-17T12:47:16.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6fabfff700 1 --2- 172.21.15.36:0/2321682138 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6f8c004e70 0x7f6f8c00b750 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f6fa40016f0 tx=0x7f6fac0acfd0).ready entity=mon.0 client_cookie=9e83c0493d7d9fe3 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:16.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6faa7fc700 1 -- 172.21.15.36:0/2321682138 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fa400ccc0 con 0x7f6f8c004e70 2022-06-17T12:47:16.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6faa7fc700 1 -- 172.21.15.36:0/2321682138 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa400ce20 con 0x7f6f8c004e70 2022-06-17T12:47:16.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.805+0000 7f6faa7fc700 1 -- 172.21.15.36:0/2321682138 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fa4005b80 con 0x7f6f8c004e70 2022-06-17T12:47:16.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.809+0000 7f6fb0dc9700 1 -- 172.21.15.36:0/2321682138 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6f8c004e70 msgr2=0x7f6f8c00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:16.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.809+0000 7f6fb0dc9700 1 --2- 172.21.15.36:0/2321682138 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6f8c004e70 0x7f6f8c00b750 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f6fa40016f0 tx=0x7f6fac0acfd0).stop 2022-06-17T12:47:16.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.809+0000 7f6fb0dc9700 1 -- 172.21.15.36:0/2321682138 shutdown_connections 2022-06-17T12:47:16.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.809+0000 7f6fb0dc9700 1 --2- 172.21.15.36:0/2321682138 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6f8c004e70 0x7f6f8c00b750 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.809+0000 7f6fb0dc9700 1 -- 172.21.15.36:0/2321682138 >> 172.21.15.36:0/2321682138 conn(0x7f6fac0b5d10 msgr2=0x7f6fac0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:16.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.809+0000 7f6fb0dc9700 1 -- 172.21.15.36:0/2321682138 shutdown_connections 2022-06-17T12:47:16.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.809+0000 7f6fb0dc9700 1 -- 172.21.15.36:0/2321682138 wait complete. 2022-06-17T12:47:16.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.809+0000 7f6fb0dc9700 1 Processor -- start 2022-06-17T12:47:16.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.809+0000 7f6fb0dc9700 1 -- start start 2022-06-17T12:47:16.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.809+0000 7f6fb0dc9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f8c004e70 0x7f6fac064340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:16.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.809+0000 7f6fb0dc9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6fac0ac140 0x7f6fac065850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:16.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.809+0000 7f6fb0dc9700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6fac065d40 0x7f6fac0a7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:16.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6fb0dc9700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac067020 con 0x7f6fac0ac140 2022-06-17T12:47:16.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6fb0dc9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac0a8610 con 0x7f6f8c004e70 2022-06-17T12:47:16.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6fb0dc9700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac156f00 con 0x7f6fac065d40 2022-06-17T12:47:16.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6faaffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6fac0ac140 0x7f6fac065850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:16.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6faaffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6fac0ac140 0x7f6fac065850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60180/0 (socket says 172.21.15.36:60180) 2022-06-17T12:47:16.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6faaffd700 1 -- 172.21.15.36:0/2876487966 learned_addr learned my addr 172.21.15.36:0/2876487966 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:16.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6fab7fe700 1 --2- 172.21.15.36:0/2876487966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f8c004e70 0x7f6fac064340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:16.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6fabfff700 1 --2- 172.21.15.36:0/2876487966 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6fac065d40 0x7f6fac0a7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:16.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6faaffd700 1 -- 172.21.15.36:0/2876487966 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6fac065d40 msgr2=0x7f6fac0a7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:16.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6faaffd700 1 --2- 172.21.15.36:0/2876487966 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6fac065d40 0x7f6fac0a7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6faaffd700 1 -- 172.21.15.36:0/2876487966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f8c004e70 msgr2=0x7f6fac064340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:16.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6faaffd700 1 --2- 172.21.15.36:0/2876487966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f8c004e70 0x7f6fac064340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6faaffd700 1 -- 172.21.15.36:0/2876487966 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fac156c20 con 0x7f6fac0ac140 2022-06-17T12:47:16.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6faaffd700 1 --2- 172.21.15.36:0/2876487966 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6fac0ac140 0x7f6fac065850 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f6f9c000f50 tx=0x7f6f9c0079e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:16.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6fa8ff9700 1 -- 172.21.15.36:0/2876487966 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f9c00f7e0 con 0x7f6fac0ac140 2022-06-17T12:47:16.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6fa8ff9700 1 -- 172.21.15.36:0/2876487966 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9c00f940 con 0x7f6fac0ac140 2022-06-17T12:47:16.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6fb0dc9700 1 -- 172.21.15.36:0/2876487966 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fac064830 con 0x7f6fac0ac140 2022-06-17T12:47:16.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6fa8ff9700 1 -- 172.21.15.36:0/2876487966 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f9c00fca0 con 0x7f6fac0ac140 2022-06-17T12:47:16.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6fb0dc9700 1 -- 172.21.15.36:0/2876487966 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fac0a8120 con 0x7f6fac0ac140 2022-06-17T12:47:16.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6fb0dc9700 1 -- 172.21.15.36:0/2876487966 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fac1570c0 con 0x7f6fac0ac140 2022-06-17T12:47:16.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6fa8ff9700 1 -- 172.21.15.36:0/2876487966 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6f9c01b030 con 0x7f6fac0ac140 2022-06-17T12:47:16.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6fa8ff9700 1 --2- 172.21.15.36:0/2876487966 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6f90053a40 0x7f6f90055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:16.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6fa8ff9700 1 -- 172.21.15.36:0/2876487966 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f6f9c06c1d0 con 0x7f6fac0ac140 2022-06-17T12:47:16.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6fab7fe700 1 --2- 172.21.15.36:0/2876487966 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6f90053a40 0x7f6f90055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:16.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.813+0000 7f6fab7fe700 1 --2- 172.21.15.36:0/2876487966 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6f90053a40 0x7f6f90055f00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f6f9400b000 tx=0x7f6f94008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:16.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.821+0000 7f6fa8ff9700 1 -- 172.21.15.36:0/2876487966 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6f9c042ed0 con 0x7f6fac0ac140 2022-06-17T12:47:16.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.953+0000 7f6fb0dc9700 1 -- 172.21.15.36:0/2876487966 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6fac0a8430 con 0x7f6fac0ac140 2022-06-17T12:47:16.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.953+0000 7f6fa8ff9700 1 -- 172.21.15.36:0/2876487966 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6f9c0464f0 con 0x7f6fac0ac140 2022-06-17T12:47:16.304 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:47:16.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.953+0000 7f6fb0dc9700 1 -- 172.21.15.36:0/2876487966 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6f90053a40 msgr2=0x7f6f90055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:16.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.953+0000 7f6fb0dc9700 1 --2- 172.21.15.36:0/2876487966 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6f90053a40 0x7f6f90055f00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f6f9400b000 tx=0x7f6f94008040).stop 2022-06-17T12:47:16.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.953+0000 7f6fb0dc9700 1 -- 172.21.15.36:0/2876487966 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6fac0ac140 msgr2=0x7f6fac065850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:16.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.953+0000 7f6fb0dc9700 1 --2- 172.21.15.36:0/2876487966 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6fac0ac140 0x7f6fac065850 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f6f9c000f50 tx=0x7f6f9c0079e0).stop 2022-06-17T12:47:16.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.957+0000 7f6fb0dc9700 1 -- 172.21.15.36:0/2876487966 shutdown_connections 2022-06-17T12:47:16.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.957+0000 7f6fb0dc9700 1 --2- 172.21.15.36:0/2876487966 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6f90053a40 0x7f6f90055f00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.957+0000 7f6fb0dc9700 1 --2- 172.21.15.36:0/2876487966 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6fac065d40 0x7f6fac0a7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.957+0000 7f6fb0dc9700 1 --2- 172.21.15.36:0/2876487966 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6fac0ac140 0x7f6fac065850 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.957+0000 7f6fb0dc9700 1 --2- 172.21.15.36:0/2876487966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f8c004e70 0x7f6fac064340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.957+0000 7f6fb0dc9700 1 -- 172.21.15.36:0/2876487966 >> 172.21.15.36:0/2876487966 conn(0x7f6fac0b5d10 msgr2=0x7f6fac0acfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:16.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.957+0000 7f6fb0dc9700 1 -- 172.21.15.36:0/2876487966 shutdown_connections 2022-06-17T12:47:16.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:15.957+0000 7f6fb0dc9700 1 -- 172.21.15.36:0/2876487966 wait complete. 2022-06-17T12:47:16.309 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:47:16.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64f5ee7700 1 Processor -- start 2022-06-17T12:47:16.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64f5ee7700 1 -- start start 2022-06-17T12:47:16.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64f5ee7700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f64f00b17e0 0x7f64f0066df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:16.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64f5ee7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64f01436e0 0x7f64f00ace80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:16.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64f5ee7700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f00af780 con 0x7f64f00605f0 2022-06-17T12:47:16.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64f5ee7700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64f0143b70 con 0x7f64f00b17e0 2022-06-17T12:47:16.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64f5ee7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f00af240 con 0x7f64f01436e0 2022-06-17T12:47:16.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64eeffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f64f00b17e0 0x7f64f0066df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:16.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64eeffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f64f00b17e0 0x7f64f0066df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60188/0 (socket says 172.21.15.36:60188) 2022-06-17T12:47:16.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64eeffd700 1 -- 172.21.15.36:0/4256843577 learned_addr learned my addr 172.21.15.36:0/4256843577 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:16.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64effff700 1 --2- 172.21.15.36:0/4256843577 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64f01436e0 0x7f64f00ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:16.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64eeffd700 1 -- 172.21.15.36:0/4256843577 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64f01436e0 msgr2=0x7f64f00ace80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:16.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64eeffd700 1 --2- 172.21.15.36:0/4256843577 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64f01436e0 0x7f64f00ace80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64eeffd700 1 -- 172.21.15.36:0/4256843577 >> v1:172.21.15.70:6789/0 conn(0x7f64f00605f0 legacy=0x7f64f00666e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:16.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64eeffd700 1 -- 172.21.15.36:0/4256843577 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f00af4e0 con 0x7f64f00b17e0 2022-06-17T12:47:16.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64eeffd700 1 --2- 172.21.15.36:0/4256843577 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64f00b17e0 0x7f64f0066df0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f64e4002f70 tx=0x7f64e4006e40).ready entity=mon.0 client_cookie=4703677b44f129fe server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:16.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64ee7fc700 1 -- 172.21.15.36:0/4256843577 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64e400bd20 con 0x7f64f00b17e0 2022-06-17T12:47:16.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64ee7fc700 1 -- 172.21.15.36:0/4256843577 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e4000e90 con 0x7f64f00b17e0 2022-06-17T12:47:16.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64ee7fc700 1 -- 172.21.15.36:0/4256843577 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64e40155e0 con 0x7f64f00b17e0 2022-06-17T12:47:16.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64f5ee7700 1 -- 172.21.15.36:0/4256843577 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64f00b17e0 msgr2=0x7f64f0066df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:16.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64f5ee7700 1 --2- 172.21.15.36:0/4256843577 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64f00b17e0 0x7f64f0066df0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f64e4002f70 tx=0x7f64e4006e40).stop 2022-06-17T12:47:16.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64f5ee7700 1 -- 172.21.15.36:0/4256843577 shutdown_connections 2022-06-17T12:47:16.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64f5ee7700 1 --2- 172.21.15.36:0/4256843577 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64f01436e0 0x7f64f00ace80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64f5ee7700 1 --2- 172.21.15.36:0/4256843577 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64f00b17e0 0x7f64f0066df0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64f5ee7700 1 -- 172.21.15.36:0/4256843577 >> 172.21.15.36:0/4256843577 conn(0x7f64f005c550 msgr2=0x7f64f00b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:16.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64f5ee7700 1 -- 172.21.15.36:0/4256843577 shutdown_connections 2022-06-17T12:47:16.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.473+0000 7f64f5ee7700 1 -- 172.21.15.36:0/4256843577 wait complete. 2022-06-17T12:47:16.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.477+0000 7f64f5ee7700 1 Processor -- start 2022-06-17T12:47:16.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.477+0000 7f64f5ee7700 1 -- start start 2022-06-17T12:47:16.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.477+0000 7f64f5ee7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64f00605f0 0x7f64f0142340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:16.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.477+0000 7f64f5ee7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f00b17e0 0x7f64f014e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:16.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.477+0000 7f64f5ee7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64f01436e0 0x7f64f009ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:16.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.477+0000 7f64f5ee7700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f0143190 con 0x7f64f00605f0 2022-06-17T12:47:16.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.477+0000 7f64f5ee7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f009eac0 con 0x7f64f00b17e0 2022-06-17T12:47:16.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.477+0000 7f64f5ee7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f005fc10 con 0x7f64f01436e0 2022-06-17T12:47:16.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.477+0000 7f64ef7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64f00605f0 0x7f64f0142340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:16.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.477+0000 7f64eeffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f00b17e0 0x7f64f014e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:16.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.477+0000 7f64ef7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64f00605f0 0x7f64f0142340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60192/0 (socket says 172.21.15.36:60192) 2022-06-17T12:47:16.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.477+0000 7f64ef7fe700 1 -- 172.21.15.36:0/3147521416 learned_addr learned my addr 172.21.15.36:0/3147521416 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:16.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.477+0000 7f64effff700 1 --2- 172.21.15.36:0/3147521416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64f01436e0 0x7f64f009ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:16.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.477+0000 7f64eeffd700 1 -- 172.21.15.36:0/3147521416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64f01436e0 msgr2=0x7f64f009ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:16.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.477+0000 7f64eeffd700 1 --2- 172.21.15.36:0/3147521416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64f01436e0 0x7f64f009ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.477+0000 7f64eeffd700 1 -- 172.21.15.36:0/3147521416 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64f00605f0 msgr2=0x7f64f0142340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:16.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.477+0000 7f64eeffd700 1 --2- 172.21.15.36:0/3147521416 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64f00605f0 0x7f64f0142340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.477+0000 7f64eeffd700 1 -- 172.21.15.36:0/3147521416 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e400d4d0 con 0x7f64f00b17e0 2022-06-17T12:47:16.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.477+0000 7f64eeffd700 1 --2- 172.21.15.36:0/3147521416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f00b17e0 0x7f64f014e150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f64e4006fc0 tx=0x7f64e4018000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:16.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.481+0000 7f64ecff9700 1 -- 172.21.15.36:0/3147521416 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64e4022070 con 0x7f64f00b17e0 2022-06-17T12:47:16.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.481+0000 7f64ecff9700 1 -- 172.21.15.36:0/3147521416 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e4015a00 con 0x7f64f00b17e0 2022-06-17T12:47:16.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.481+0000 7f64f5ee7700 1 -- 172.21.15.36:0/3147521416 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f014e640 con 0x7f64f00b17e0 2022-06-17T12:47:16.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.481+0000 7f64ecff9700 1 -- 172.21.15.36:0/3147521416 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64e4015d60 con 0x7f64f00b17e0 2022-06-17T12:47:16.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.481+0000 7f64f5ee7700 1 -- 172.21.15.36:0/3147521416 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f0142f10 con 0x7f64f00b17e0 2022-06-17T12:47:16.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.481+0000 7f64f5ee7700 1 -- 172.21.15.36:0/3147521416 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f009f790 con 0x7f64f00b17e0 2022-06-17T12:47:16.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.481+0000 7f64ecff9700 1 -- 172.21.15.36:0/3147521416 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f64e4027410 con 0x7f64f00b17e0 2022-06-17T12:47:16.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.481+0000 7f64ecff9700 1 --2- 172.21.15.36:0/3147521416 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f64d0053a60 0x7f64d0055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:16.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.481+0000 7f64ecff9700 1 -- 172.21.15.36:0/3147521416 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f64e406f050 con 0x7f64f00b17e0 2022-06-17T12:47:16.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.481+0000 7f64ef7fe700 1 --2- 172.21.15.36:0/3147521416 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f64d0053a60 0x7f64d0055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:16.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.481+0000 7f64ef7fe700 1 --2- 172.21.15.36:0/3147521416 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f64d0053a60 0x7f64d0055f20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f64e000d000 tx=0x7f64e0007c50).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:16.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.489+0000 7f64ecff9700 1 -- 172.21.15.36:0/3147521416 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f64f009f790 con 0x7f64f00b17e0 2022-06-17T12:47:16.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.621+0000 7f64f5ee7700 1 -- 172.21.15.36:0/3147521416 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f64f009f790 con 0x7f64f00b17e0 2022-06-17T12:47:16.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.621+0000 7f64ecff9700 1 -- 172.21.15.36:0/3147521416 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f64f009f790 con 0x7f64f00b17e0 2022-06-17T12:47:16.625 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:16.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.625+0000 7f64f5ee7700 1 -- 172.21.15.36:0/3147521416 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f64d0053a60 msgr2=0x7f64d0055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:16.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.625+0000 7f64f5ee7700 1 --2- 172.21.15.36:0/3147521416 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f64d0053a60 0x7f64d0055f20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f64e000d000 tx=0x7f64e0007c50).stop 2022-06-17T12:47:16.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.625+0000 7f64f5ee7700 1 -- 172.21.15.36:0/3147521416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f00b17e0 msgr2=0x7f64f014e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:16.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.625+0000 7f64f5ee7700 1 --2- 172.21.15.36:0/3147521416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f00b17e0 0x7f64f014e150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f64e4006fc0 tx=0x7f64e4018000).stop 2022-06-17T12:47:16.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.625+0000 7f64f5ee7700 1 -- 172.21.15.36:0/3147521416 shutdown_connections 2022-06-17T12:47:16.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.625+0000 7f64f5ee7700 1 --2- 172.21.15.36:0/3147521416 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f64d0053a60 0x7f64d0055f20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.625+0000 7f64f5ee7700 1 --2- 172.21.15.36:0/3147521416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64f01436e0 0x7f64f009ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.625+0000 7f64f5ee7700 1 --2- 172.21.15.36:0/3147521416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f00b17e0 0x7f64f014e150 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.625+0000 7f64f5ee7700 1 --2- 172.21.15.36:0/3147521416 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64f00605f0 0x7f64f0142340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.625+0000 7f64f5ee7700 1 -- 172.21.15.36:0/3147521416 >> 172.21.15.36:0/3147521416 conn(0x7f64f005c550 msgr2=0x7f64f00b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:16.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.625+0000 7f64f5ee7700 1 -- 172.21.15.36:0/3147521416 shutdown_connections 2022-06-17T12:47:16.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.625+0000 7f64f5ee7700 1 -- 172.21.15.36:0/3147521416 wait complete. 2022-06-17T12:47:16.630 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:47:16.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.765+0000 7f76dffff700 1 Processor -- start 2022-06-17T12:47:16.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.765+0000 7f76dffff700 1 -- start start 2022-06-17T12:47:16.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.765+0000 7f76dffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76e0060ac0 0x7f76e0140260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:16.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.765+0000 7f76dffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f76e00b2c80 0x7f76e0140750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:16.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.765+0000 7f76dffff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f76e0067260 0x7f76e00aee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:16.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.765+0000 7f76dffff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e00b2210 con 0x7f76e00b2c80 2022-06-17T12:47:16.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.765+0000 7f76dffff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e0060ea0 con 0x7f76e0067260 2022-06-17T12:47:16.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.765+0000 7f76dffff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e00aebb0 con 0x7f76e0060ac0 2022-06-17T12:47:16.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.769+0000 7f76d67fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f76e00b2c80 0x7f76e0140750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:16.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.769+0000 7f76d67fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f76e00b2c80 0x7f76e0140750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60202/0 (socket says 172.21.15.36:60202) 2022-06-17T12:47:16.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.769+0000 7f76d67fc700 1 -- 172.21.15.36:0/1230998755 learned_addr learned my addr 172.21.15.36:0/1230998755 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:16.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.769+0000 7f76deffd700 1 --2- 172.21.15.36:0/1230998755 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76e0060ac0 0x7f76e0140260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:16.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.769+0000 7f76df7fe700 1 --2- 172.21.15.36:0/1230998755 >> v2:172.21.15.70:3300/0 conn(0x7f76e0067260 0x7f76e00aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:16.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.769+0000 7f76d67fc700 1 -- 172.21.15.36:0/1230998755 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76e0060ac0 msgr2=0x7f76e0140260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:16.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.769+0000 7f76d67fc700 1 --2- 172.21.15.36:0/1230998755 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76e0060ac0 0x7f76e0140260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.769+0000 7f76d67fc700 1 -- 172.21.15.36:0/1230998755 >> v2:172.21.15.70:3300/0 conn(0x7f76e0067260 msgr2=0x7f76e00aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:16.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.769+0000 7f76d67fc700 1 --2- 172.21.15.36:0/1230998755 >> v2:172.21.15.70:3300/0 conn(0x7f76e0067260 0x7f76e00aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.769+0000 7f76d67fc700 1 -- 172.21.15.36:0/1230998755 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e00b1ad0 con 0x7f76e00b2c80 2022-06-17T12:47:16.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.769+0000 7f76df7fe700 1 --2- 172.21.15.36:0/1230998755 >> v2:172.21.15.70:3300/0 conn(0x7f76e0067260 0x7f76e00aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:16.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.769+0000 7f76d67fc700 1 --2- 172.21.15.36:0/1230998755 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f76e00b2c80 0x7f76e0140750 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f76c8002f70 tx=0x7f76c8006eb0).ready entity=mon.0 client_cookie=3e4b5dca92c8c0e1 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:16.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.769+0000 7f76de7fc700 1 -- 172.21.15.36:0/1230998755 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76c800d570 con 0x7f76e00b2c80 2022-06-17T12:47:16.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.769+0000 7f76de7fc700 1 -- 172.21.15.36:0/1230998755 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c8000e90 con 0x7f76e00b2c80 2022-06-17T12:47:16.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.769+0000 7f76de7fc700 1 -- 172.21.15.36:0/1230998755 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76c800d900 con 0x7f76e00b2c80 2022-06-17T12:47:16.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.769+0000 7f76dffff700 1 -- 172.21.15.36:0/1230998755 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f76e00b2c80 msgr2=0x7f76e0140750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:16.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.769+0000 7f76dffff700 1 --2- 172.21.15.36:0/1230998755 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f76e00b2c80 0x7f76e0140750 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f76c8002f70 tx=0x7f76c8006eb0).stop 2022-06-17T12:47:16.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.769+0000 7f76dffff700 1 -- 172.21.15.36:0/1230998755 shutdown_connections 2022-06-17T12:47:16.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.769+0000 7f76dffff700 1 --2- 172.21.15.36:0/1230998755 >> v2:172.21.15.70:3300/0 conn(0x7f76e0067260 0x7f76e00aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.769+0000 7f76dffff700 1 --2- 172.21.15.36:0/1230998755 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f76e00b2c80 0x7f76e0140750 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.769+0000 7f76dffff700 1 --2- 172.21.15.36:0/1230998755 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76e0060ac0 0x7f76e0140260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.769+0000 7f76dffff700 1 -- 172.21.15.36:0/1230998755 >> 172.21.15.36:0/1230998755 conn(0x7f76e00b64f0 msgr2=0x7f76e00b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:16.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.769+0000 7f76dffff700 1 -- 172.21.15.36:0/1230998755 shutdown_connections 2022-06-17T12:47:16.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.769+0000 7f76dffff700 1 -- 172.21.15.36:0/1230998755 wait complete. 2022-06-17T12:47:16.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76dffff700 1 Processor -- start 2022-06-17T12:47:16.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76dffff700 1 -- start start 2022-06-17T12:47:16.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76dffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f76e0060ac0 0x7f76e01445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:16.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76dffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76e0067260 0x7f76e0146ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:16.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76dffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76e00b2c80 0x7f76e01421e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:16.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76dffff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e0140e40 con 0x7f76e0060ac0 2022-06-17T12:47:16.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76dffff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e0147c80 con 0x7f76e0067260 2022-06-17T12:47:16.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76dffff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e0147f20 con 0x7f76e00b2c80 2022-06-17T12:47:16.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76deffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f76e0060ac0 0x7f76e01445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:16.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76deffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f76e0060ac0 0x7f76e01445b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60206/0 (socket says 172.21.15.36:60206) 2022-06-17T12:47:16.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76deffd700 1 -- 172.21.15.36:0/3348742920 learned_addr learned my addr 172.21.15.36:0/3348742920 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:16.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76d67fc700 1 --2- 172.21.15.36:0/3348742920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76e0067260 0x7f76e0146ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:16.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76df7fe700 1 --2- 172.21.15.36:0/3348742920 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76e00b2c80 0x7f76e01421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:16.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76deffd700 1 -- 172.21.15.36:0/3348742920 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76e00b2c80 msgr2=0x7f76e01421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:16.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76deffd700 1 --2- 172.21.15.36:0/3348742920 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76e00b2c80 0x7f76e01421e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76deffd700 1 -- 172.21.15.36:0/3348742920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76e0067260 msgr2=0x7f76e0146ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:16.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76deffd700 1 --2- 172.21.15.36:0/3348742920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76e0067260 0x7f76e0146ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:16.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76deffd700 1 -- 172.21.15.36:0/3348742920 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c800b1e0 con 0x7f76e0060ac0 2022-06-17T12:47:16.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76d67fc700 1 --2- 172.21.15.36:0/3348742920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76e0067260 0x7f76e0146ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:16.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76df7fe700 1 --2- 172.21.15.36:0/3348742920 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76e00b2c80 0x7f76e01421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:16.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76deffd700 1 --2- 172.21.15.36:0/3348742920 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f76e0060ac0 0x7f76e01445b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f76d00030b0 tx=0x7f76d0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:16.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76dcff9700 1 -- 172.21.15.36:0/3348742920 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76d0011780 con 0x7f76e0060ac0 2022-06-17T12:47:16.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76dcff9700 1 -- 172.21.15.36:0/3348742920 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d00118e0 con 0x7f76e0060ac0 2022-06-17T12:47:16.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76dffff700 1 -- 172.21.15.36:0/3348742920 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e01481c0 con 0x7f76e0060ac0 2022-06-17T12:47:16.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.773+0000 7f76dcff9700 1 -- 172.21.15.36:0/3348742920 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76d0011c40 con 0x7f76e0060ac0 2022-06-17T12:47:16.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.777+0000 7f76dffff700 1 -- 172.21.15.36:0/3348742920 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e01428f0 con 0x7f76e0060ac0 2022-06-17T12:47:16.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.777+0000 7f76dffff700 1 -- 172.21.15.36:0/3348742920 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e0148350 con 0x7f76e0060ac0 2022-06-17T12:47:17.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.777+0000 7f76dcff9700 1 -- 172.21.15.36:0/3348742920 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f76d0012420 con 0x7f76e0060ac0 2022-06-17T12:47:17.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.777+0000 7f76dcff9700 1 --2- 172.21.15.36:0/3348742920 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f76c4053920 0x7f76c4055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:17.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.777+0000 7f76dcff9700 1 -- 172.21.15.36:0/3348742920 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f76d006bf00 con 0x7f76e0060ac0 2022-06-17T12:47:17.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.777+0000 7f76d67fc700 1 --2- 172.21.15.36:0/3348742920 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f76c4053920 0x7f76c4055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:17.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.781+0000 7f76d67fc700 1 --2- 172.21.15.36:0/3348742920 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f76c4053920 0x7f76c4055de0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f76c801c000 tx=0x7f76c8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:17.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.785+0000 7f76dcff9700 1 -- 172.21.15.36:0/3348742920 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f76d0042b90 con 0x7f76e0060ac0 2022-06-17T12:47:17.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.909+0000 7f76dffff700 1 -- 172.21.15.36:0/3348742920 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f76e0057410 con 0x7f76e0060ac0 2022-06-17T12:47:17.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.909+0000 7f76dcff9700 1 -- 172.21.15.36:0/3348742920 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+10 (secure 0 0 0) 0x7f76d00461b0 con 0x7f76e0060ac0 2022-06-17T12:47:17.059 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:17.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.909+0000 7f76dffff700 1 -- 172.21.15.36:0/3348742920 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f76c4053920 msgr2=0x7f76c4055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:17.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.909+0000 7f76dffff700 1 --2- 172.21.15.36:0/3348742920 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f76c4053920 0x7f76c4055de0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f76c801c000 tx=0x7f76c8015000).stop 2022-06-17T12:47:17.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.909+0000 7f76dffff700 1 -- 172.21.15.36:0/3348742920 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f76e0060ac0 msgr2=0x7f76e01445b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:17.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.909+0000 7f76dffff700 1 --2- 172.21.15.36:0/3348742920 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f76e0060ac0 0x7f76e01445b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f76d00030b0 tx=0x7f76d0008040).stop 2022-06-17T12:47:17.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.909+0000 7f76dffff700 1 -- 172.21.15.36:0/3348742920 shutdown_connections 2022-06-17T12:47:17.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.909+0000 7f76dffff700 1 --2- 172.21.15.36:0/3348742920 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f76c4053920 0x7f76c4055de0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:17.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.909+0000 7f76dffff700 1 --2- 172.21.15.36:0/3348742920 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76e00b2c80 0x7f76e01421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:17.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.909+0000 7f76dffff700 1 --2- 172.21.15.36:0/3348742920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76e0067260 0x7f76e0146ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:17.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.909+0000 7f76dffff700 1 --2- 172.21.15.36:0/3348742920 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f76e0060ac0 0x7f76e01445b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:17.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.909+0000 7f76dffff700 1 -- 172.21.15.36:0/3348742920 >> 172.21.15.36:0/3348742920 conn(0x7f76e00b64f0 msgr2=0x7f76e00b42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:17.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.909+0000 7f76dffff700 1 -- 172.21.15.36:0/3348742920 shutdown_connections 2022-06-17T12:47:17.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:16.909+0000 7f76dffff700 1 -- 172.21.15.36:0/3348742920 wait complete. 2022-06-17T12:47:17.107 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:47:17.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.257+0000 7fc7e8b85700 1 Processor -- start 2022-06-17T12:47:17.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.257+0000 7fc7e8b85700 1 -- start start 2022-06-17T12:47:17.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.257+0000 7fc7e8b85700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7e405f910 0x7fc7e40b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:17.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.257+0000 7fc7e8b85700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc7e40b0e30 0x7fc7e4146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:17.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.257+0000 7fc7e8b85700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7e40a9ea0 con 0x7fc7e40b0680 2022-06-17T12:47:17.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.257+0000 7fc7e8b85700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7e4143cc0 con 0x7fc7e40b0e30 2022-06-17T12:47:17.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.257+0000 7fc7e8b85700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e4143f60 con 0x7fc7e405f910 2022-06-17T12:47:17.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.257+0000 7fc7e1d9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc7e40b0e30 0x7fc7e4146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:17.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.257+0000 7fc7e1d9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc7e40b0e30 0x7fc7e4146770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60214/0 (socket says 172.21.15.36:60214) 2022-06-17T12:47:17.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.257+0000 7fc7e1d9b700 1 -- 172.21.15.36:0/3649547799 learned_addr learned my addr 172.21.15.36:0/3649547799 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:17.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.257+0000 7fc7e259c700 1 --2- 172.21.15.36:0/3649547799 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7e405f910 0x7fc7e40b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:17.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.257+0000 7fc7e1d9b700 1 -- 172.21.15.36:0/3649547799 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7e405f910 msgr2=0x7fc7e40b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:17.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.257+0000 7fc7e1d9b700 1 --2- 172.21.15.36:0/3649547799 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7e405f910 0x7fc7e40b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:17.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.257+0000 7fc7e1d9b700 1 -- 172.21.15.36:0/3649547799 >> v1:172.21.15.70:6789/0 conn(0x7fc7e40b0680 legacy=0x7fc7e4146c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:17.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.257+0000 7fc7e1d9b700 1 -- 172.21.15.36:0/3649547799 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e40a9b90 con 0x7fc7e40b0e30 2022-06-17T12:47:17.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.257+0000 7fc7e259c700 1 --2- 172.21.15.36:0/3649547799 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7e405f910 0x7fc7e40b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:17.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.261+0000 7fc7e1d9b700 1 --2- 172.21.15.36:0/3649547799 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc7e40b0e30 0x7fc7e4146770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fc7d4002f70 tx=0x7fc7d4006eb0).ready entity=mon.0 client_cookie=e717f3cf0eb32604 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:17.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.261+0000 7fc7e159a700 1 -- 172.21.15.36:0/3649547799 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7d400d5c0 con 0x7fc7e40b0e30 2022-06-17T12:47:17.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.261+0000 7fc7e159a700 1 -- 172.21.15.36:0/3649547799 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d4000e90 con 0x7fc7e40b0e30 2022-06-17T12:47:17.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.261+0000 7fc7e159a700 1 -- 172.21.15.36:0/3649547799 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7d400d950 con 0x7fc7e40b0e30 2022-06-17T12:47:17.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.261+0000 7fc7e8b85700 1 -- 172.21.15.36:0/3649547799 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc7e40b0e30 msgr2=0x7fc7e4146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:17.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.261+0000 7fc7e8b85700 1 --2- 172.21.15.36:0/3649547799 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc7e40b0e30 0x7fc7e4146770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fc7d4002f70 tx=0x7fc7d4006eb0).stop 2022-06-17T12:47:17.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.261+0000 7fc7e8b85700 1 -- 172.21.15.36:0/3649547799 shutdown_connections 2022-06-17T12:47:17.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.261+0000 7fc7e8b85700 1 --2- 172.21.15.36:0/3649547799 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc7e40b0e30 0x7fc7e4146770 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:17.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.261+0000 7fc7e8b85700 1 --2- 172.21.15.36:0/3649547799 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7e405f910 0x7fc7e40b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:17.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.261+0000 7fc7e8b85700 1 -- 172.21.15.36:0/3649547799 >> 172.21.15.36:0/3649547799 conn(0x7fc7e405c550 msgr2=0x7fc7e405c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:17.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.261+0000 7fc7e8b85700 1 -- 172.21.15.36:0/3649547799 shutdown_connections 2022-06-17T12:47:17.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.261+0000 7fc7e8b85700 1 -- 172.21.15.36:0/3649547799 wait complete. 2022-06-17T12:47:17.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.261+0000 7fc7e8b85700 1 Processor -- start 2022-06-17T12:47:17.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.261+0000 7fc7e8b85700 1 -- start start 2022-06-17T12:47:17.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.265+0000 7fc7e8b85700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7e405f910 0x7fc7e41451f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:17.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.265+0000 7fc7e8b85700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7e40b0680 0x7fc7e41456e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:17.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.265+0000 7fc7e8b85700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc7e40b0e30 0x7fc7e4152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:17.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.265+0000 7fc7e8b85700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e40b71d0 con 0x7fc7e40b0e30 2022-06-17T12:47:17.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.265+0000 7fc7e8b85700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e40ac430 con 0x7fc7e405f910 2022-06-17T12:47:17.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.265+0000 7fc7e8b85700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e40ac190 con 0x7fc7e40b0680 2022-06-17T12:47:17.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.265+0000 7fc7e2d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc7e40b0e30 0x7fc7e4152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:17.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.265+0000 7fc7e2d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc7e40b0e30 0x7fc7e4152ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60224/0 (socket says 172.21.15.36:60224) 2022-06-17T12:47:17.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.265+0000 7fc7e2d9d700 1 -- 172.21.15.36:0/2711742273 learned_addr learned my addr 172.21.15.36:0/2711742273 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:17.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.265+0000 7fc7e259c700 1 --2- 172.21.15.36:0/2711742273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7e405f910 0x7fc7e41451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:17.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.265+0000 7fc7e1d9b700 1 --2- 172.21.15.36:0/2711742273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7e40b0680 0x7fc7e41456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:17.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.265+0000 7fc7e2d9d700 1 -- 172.21.15.36:0/2711742273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7e40b0680 msgr2=0x7fc7e41456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:17.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.265+0000 7fc7e2d9d700 1 --2- 172.21.15.36:0/2711742273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7e40b0680 0x7fc7e41456e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:17.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.265+0000 7fc7e2d9d700 1 -- 172.21.15.36:0/2711742273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7e405f910 msgr2=0x7fc7e41451f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:17.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.265+0000 7fc7e2d9d700 1 --2- 172.21.15.36:0/2711742273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7e405f910 0x7fc7e41451f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:17.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.265+0000 7fc7e2d9d700 1 -- 172.21.15.36:0/2711742273 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d400b1e0 con 0x7fc7e40b0e30 2022-06-17T12:47:17.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.265+0000 7fc7e2d9d700 1 --2- 172.21.15.36:0/2711742273 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc7e40b0e30 0x7fc7e4152ec0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fc7e40b2830 tx=0x7fc7dc007f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:17.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.265+0000 7fc7db7fe700 1 -- 172.21.15.36:0/2711742273 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7dc00f470 con 0x7fc7e40b0e30 2022-06-17T12:47:17.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.265+0000 7fc7db7fe700 1 -- 172.21.15.36:0/2711742273 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7dc00f5d0 con 0x7fc7e40b0e30 2022-06-17T12:47:17.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.265+0000 7fc7e8b85700 1 -- 172.21.15.36:0/2711742273 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7e40acd80 con 0x7fc7e40b0e30 2022-06-17T12:47:17.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.265+0000 7fc7db7fe700 1 -- 172.21.15.36:0/2711742273 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7dc00f930 con 0x7fc7e40b0e30 2022-06-17T12:47:17.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.265+0000 7fc7e8b85700 1 -- 172.21.15.36:0/2711742273 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e40ac610 con 0x7fc7e40b0e30 2022-06-17T12:47:17.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.265+0000 7fc7e8b85700 1 -- 172.21.15.36:0/2711742273 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7e4057410 con 0x7fc7e40b0e30 2022-06-17T12:47:17.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.265+0000 7fc7db7fe700 1 -- 172.21.15.36:0/2711742273 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fc7dc019030 con 0x7fc7e40b0e30 2022-06-17T12:47:17.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.269+0000 7fc7db7fe700 1 --2- 172.21.15.36:0/2711742273 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc7c8053a40 0x7fc7c8055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:17.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.269+0000 7fc7db7fe700 1 -- 172.21.15.36:0/2711742273 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc7dc06b6f0 con 0x7fc7e40b0e30 2022-06-17T12:47:17.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.269+0000 7fc7e259c700 1 --2- 172.21.15.36:0/2711742273 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc7c8053a40 0x7fc7c8055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:17.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.269+0000 7fc7e259c700 1 --2- 172.21.15.36:0/2711742273 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc7c8053a40 0x7fc7c8055f00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fc7cc00c000 tx=0x7fc7cc008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:17.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.273+0000 7fc7db7fe700 1 -- 172.21.15.36:0/2711742273 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc7dc042450 con 0x7fc7e40b0e30 2022-06-17T12:47:17.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.401+0000 7fc7e8b85700 1 -- 172.21.15.36:0/2711742273 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc7e405d2c0 con 0x7fc7e40b0e30 2022-06-17T12:47:17.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.401+0000 7fc7db7fe700 1 -- 172.21.15.36:0/2711742273 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc7dc017020 con 0x7fc7e40b0e30 2022-06-17T12:47:17.735 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:17.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.401+0000 7fc7e8b85700 1 -- 172.21.15.36:0/2711742273 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc7c8053a40 msgr2=0x7fc7c8055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:17.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.401+0000 7fc7e8b85700 1 --2- 172.21.15.36:0/2711742273 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc7c8053a40 0x7fc7c8055f00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fc7cc00c000 tx=0x7fc7cc008040).stop 2022-06-17T12:47:17.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.401+0000 7fc7e8b85700 1 -- 172.21.15.36:0/2711742273 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc7e40b0e30 msgr2=0x7fc7e4152ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:17.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.401+0000 7fc7e8b85700 1 --2- 172.21.15.36:0/2711742273 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc7e40b0e30 0x7fc7e4152ec0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fc7e40b2830 tx=0x7fc7dc007f00).stop 2022-06-17T12:47:17.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.401+0000 7fc7e8b85700 1 -- 172.21.15.36:0/2711742273 shutdown_connections 2022-06-17T12:47:17.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.401+0000 7fc7e8b85700 1 --2- 172.21.15.36:0/2711742273 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc7c8053a40 0x7fc7c8055f00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:17.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.401+0000 7fc7e8b85700 1 --2- 172.21.15.36:0/2711742273 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc7e40b0e30 0x7fc7e4152ec0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:17.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.401+0000 7fc7e8b85700 1 --2- 172.21.15.36:0/2711742273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7e40b0680 0x7fc7e41456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:17.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.401+0000 7fc7e8b85700 1 --2- 172.21.15.36:0/2711742273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7e405f910 0x7fc7e41451f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:17.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.401+0000 7fc7e8b85700 1 -- 172.21.15.36:0/2711742273 >> 172.21.15.36:0/2711742273 conn(0x7fc7e405c550 msgr2=0x7fc7e413f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:17.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.401+0000 7fc7e8b85700 1 -- 172.21.15.36:0/2711742273 shutdown_connections 2022-06-17T12:47:17.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.401+0000 7fc7e8b85700 1 -- 172.21.15.36:0/2711742273 wait complete. 2022-06-17T12:47:17.753 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:47:17.753 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:47:17.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.881+0000 7f192934f700 1 Processor -- start 2022-06-17T12:47:17.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.881+0000 7f192934f700 1 -- start start 2022-06-17T12:47:17.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.881+0000 7f192934f700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f19240b80c0 0x7f1924140c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:17.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.881+0000 7f192934f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1924142d70 0x7f1924143220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:17.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.881+0000 7f192934f700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1924142160 con 0x7f19240b7ce0 2022-06-17T12:47:17.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.881+0000 7f192934f700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19240b4f30 con 0x7f19240b80c0 2022-06-17T12:47:17.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.881+0000 7f192934f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1924142390 con 0x7f1924142d70 2022-06-17T12:47:17.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.881+0000 7f1923fff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f19240b7ce0 0x7f192413aa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49526/0 (socket says 172.21.15.36:49526) 2022-06-17T12:47:17.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.881+0000 7f1923fff700 1 -- 172.21.15.36:0/570004077 learned_addr learned my addr 172.21.15.36:0/570004077 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:17.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.881+0000 7f1928b4e700 1 --2- 172.21.15.36:0/570004077 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1924142d70 0x7f1924143220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:17.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.881+0000 7f19237fe700 1 --2- 172.21.15.36:0/570004077 >> v2:172.21.15.70:3300/0 conn(0x7f19240b80c0 0x7f1924140c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:17.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.885+0000 7f1922ffd700 1 -- 172.21.15.36:0/570004077 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455361644 0 0) 0x7f1924142160 con 0x7f19240b7ce0 2022-06-17T12:47:17.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.885+0000 7f1922ffd700 1 -- 172.21.15.36:0/570004077 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1910003260 con 0x7f19240b7ce0 2022-06-17T12:47:17.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.885+0000 7f19237fe700 1 -- 172.21.15.36:0/570004077 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1924142d70 msgr2=0x7f1924143220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:17.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.885+0000 7f19237fe700 1 --2- 172.21.15.36:0/570004077 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1924142d70 0x7f1924143220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:17.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.885+0000 7f19237fe700 1 -- 172.21.15.36:0/570004077 >> v1:172.21.15.36:6789/0 conn(0x7f19240b7ce0 legacy=0x7f192413aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:17.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.885+0000 7f19237fe700 1 -- 172.21.15.36:0/570004077 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1924142630 con 0x7f19240b80c0 2022-06-17T12:47:17.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.885+0000 7f19237fe700 1 --2- 172.21.15.36:0/570004077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19240b80c0 0x7f1924140c90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f1914002f70 tx=0x7f1914006df0).ready entity=mon.1 client_cookie=f7a6fa350851427c server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:17.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.885+0000 7f1922ffd700 1 -- 172.21.15.36:0/570004077 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f191400d4a0 con 0x7f19240b80c0 2022-06-17T12:47:17.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.885+0000 7f1922ffd700 1 -- 172.21.15.36:0/570004077 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1914000e90 con 0x7f19240b80c0 2022-06-17T12:47:17.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.885+0000 7f1922ffd700 1 -- 172.21.15.36:0/570004077 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f191400d7f0 con 0x7f19240b80c0 2022-06-17T12:47:17.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.885+0000 7f192934f700 1 -- 172.21.15.36:0/570004077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19240b80c0 msgr2=0x7f1924140c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:17.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.885+0000 7f192934f700 1 --2- 172.21.15.36:0/570004077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19240b80c0 0x7f1924140c90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f1914002f70 tx=0x7f1914006df0).stop 2022-06-17T12:47:17.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.885+0000 7f192934f700 1 -- 172.21.15.36:0/570004077 shutdown_connections 2022-06-17T12:47:17.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.885+0000 7f192934f700 1 --2- 172.21.15.36:0/570004077 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1924142d70 0x7f1924143220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:18.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.885+0000 7f192934f700 1 --2- 172.21.15.36:0/570004077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19240b80c0 0x7f1924140c90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:18.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.885+0000 7f192934f700 1 -- 172.21.15.36:0/570004077 >> 172.21.15.36:0/570004077 conn(0x7f192405c550 msgr2=0x7f19240b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:18.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.885+0000 7f192934f700 1 -- 172.21.15.36:0/570004077 shutdown_connections 2022-06-17T12:47:18.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.885+0000 7f192934f700 1 -- 172.21.15.36:0/570004077 wait complete. 2022-06-17T12:47:18.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.885+0000 7f192934f700 1 Processor -- start 2022-06-17T12:47:18.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.885+0000 7f192934f700 1 -- start start 2022-06-17T12:47:18.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.889+0000 7f192934f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19240b7ce0 0x7f1924139760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:18.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.889+0000 7f192934f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19240b80c0 0x7f19240b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:18.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.889+0000 7f192934f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1924142d70 0x7f1924135d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:18.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.889+0000 7f192934f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19240b2de0 con 0x7f19240b7ce0 2022-06-17T12:47:18.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.889+0000 7f192934f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1924141a00 con 0x7f1924142d70 2022-06-17T12:47:18.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.889+0000 7f192934f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19240b3830 con 0x7f19240b80c0 2022-06-17T12:47:18.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.889+0000 7f1923fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19240b7ce0 0x7f1924139760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:18.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.889+0000 7f1923fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19240b7ce0 0x7f1924139760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60234/0 (socket says 172.21.15.36:60234) 2022-06-17T12:47:18.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.889+0000 7f1923fff700 1 -- 172.21.15.36:0/1975658542 learned_addr learned my addr 172.21.15.36:0/1975658542 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:18.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.889+0000 7f1928b4e700 1 --2- 172.21.15.36:0/1975658542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1924142d70 0x7f1924135d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:18.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.889+0000 7f19237fe700 1 --2- 172.21.15.36:0/1975658542 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19240b80c0 0x7f19240b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:18.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.889+0000 7f1923fff700 1 -- 172.21.15.36:0/1975658542 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19240b80c0 msgr2=0x7f19240b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:18.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.889+0000 7f1923fff700 1 --2- 172.21.15.36:0/1975658542 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19240b80c0 0x7f19240b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:18.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.889+0000 7f1923fff700 1 -- 172.21.15.36:0/1975658542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1924142d70 msgr2=0x7f1924135d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:18.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.889+0000 7f1923fff700 1 --2- 172.21.15.36:0/1975658542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1924142d70 0x7f1924135d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:18.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.889+0000 7f1923fff700 1 -- 172.21.15.36:0/1975658542 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191400b180 con 0x7f19240b7ce0 2022-06-17T12:47:18.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.889+0000 7f1928b4e700 1 --2- 172.21.15.36:0/1975658542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1924142d70 0x7f1924135d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:18.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.889+0000 7f19237fe700 1 --2- 172.21.15.36:0/1975658542 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19240b80c0 0x7f19240b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:18.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.889+0000 7f1923fff700 1 --2- 172.21.15.36:0/1975658542 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19240b7ce0 0x7f1924139760 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f1918001110 tx=0x7f1918007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:18.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.889+0000 7f19217fa700 1 -- 172.21.15.36:0/1975658542 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19180070d0 con 0x7f19240b7ce0 2022-06-17T12:47:18.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.889+0000 7f19217fa700 1 -- 172.21.15.36:0/1975658542 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1918007230 con 0x7f19240b7ce0 2022-06-17T12:47:18.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.889+0000 7f192934f700 1 -- 172.21.15.36:0/1975658542 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19240b3ad0 con 0x7f19240b7ce0 2022-06-17T12:47:18.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.889+0000 7f19217fa700 1 -- 172.21.15.36:0/1975658542 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1918005610 con 0x7f19240b7ce0 2022-06-17T12:47:18.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.889+0000 7f192934f700 1 -- 172.21.15.36:0/1975658542 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1924067190 con 0x7f19240b7ce0 2022-06-17T12:47:18.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.893+0000 7f192934f700 1 -- 172.21.15.36:0/1975658542 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19240b3c60 con 0x7f19240b7ce0 2022-06-17T12:47:18.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.893+0000 7f19217fa700 1 -- 172.21.15.36:0/1975658542 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f1918010070 con 0x7f19240b7ce0 2022-06-17T12:47:18.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.893+0000 7f19217fa700 1 --2- 172.21.15.36:0/1975658542 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1904053a40 0x7f1904055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:18.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.893+0000 7f19217fa700 1 -- 172.21.15.36:0/1975658542 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f191806b6c0 con 0x7f19240b7ce0 2022-06-17T12:47:18.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.893+0000 7f19237fe700 1 --2- 172.21.15.36:0/1975658542 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1904053a40 0x7f1904055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:18.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.893+0000 7f19237fe700 1 --2- 172.21.15.36:0/1975658542 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1904053a40 0x7f1904055f00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f1914018000 tx=0x7f1914014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:18.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:17.901+0000 7f19217fa700 1 -- 172.21.15.36:0/1975658542 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f19180423b0 con 0x7f19240b7ce0 2022-06-17T12:47:18.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:18.065+0000 7f192934f700 1 -- 172.21.15.36:0/1975658542 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19241428d0 con 0x7f19240b7ce0 2022-06-17T12:47:18.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:18.065+0000 7f19217fa700 1 -- 172.21.15.36:0/1975658542 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2134 (secure 0 0 0) 0x7f19180459d0 con 0x7f19240b7ce0 2022-06-17T12:47:18.110 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:18.116 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/4243909763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":4243909763},{"type":"v1","addr":"172.21.15.36:6835","nonce":4243909763}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:46:52.253609+0000","modified":"2022-06-17T12:46:54.268223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/3425584732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3425584732},{"type":"v1","addr":"172.21.15.70:6833","nonce":3425584732}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-06-17T12:47:18.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:18.069+0000 7f192934f700 1 -- 172.21.15.36:0/1975658542 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1904053a40 msgr2=0x7f1904055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:18.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:18.069+0000 7f192934f700 1 --2- 172.21.15.36:0/1975658542 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1904053a40 0x7f1904055f00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f1914018000 tx=0x7f1914014000).stop 2022-06-17T12:47:18.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:18.069+0000 7f192934f700 1 -- 172.21.15.36:0/1975658542 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19240b7ce0 msgr2=0x7f1924139760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:18.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:18.069+0000 7f192934f700 1 --2- 172.21.15.36:0/1975658542 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19240b7ce0 0x7f1924139760 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f1918001110 tx=0x7f1918007aa0).stop 2022-06-17T12:47:18.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:18.069+0000 7f192934f700 1 -- 172.21.15.36:0/1975658542 shutdown_connections 2022-06-17T12:47:18.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:18.069+0000 7f192934f700 1 --2- 172.21.15.36:0/1975658542 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1904053a40 0x7f1904055f00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:18.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:18.069+0000 7f192934f700 1 --2- 172.21.15.36:0/1975658542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1924142d70 0x7f1924135d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:18.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:18.069+0000 7f192934f700 1 --2- 172.21.15.36:0/1975658542 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19240b80c0 0x7f19240b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:18.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:18.069+0000 7f192934f700 1 --2- 172.21.15.36:0/1975658542 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19240b7ce0 0x7f1924139760 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:18.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:18.069+0000 7f192934f700 1 -- 172.21.15.36:0/1975658542 >> 172.21.15.36:0/1975658542 conn(0x7f192405c550 msgr2=0x7f19240b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:18.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:18.069+0000 7f192934f700 1 -- 172.21.15.36:0/1975658542 shutdown_connections 2022-06-17T12:47:18.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:18.069+0000 7f192934f700 1 -- 172.21.15.36:0/1975658542 wait complete. 2022-06-17T12:47:18.225 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 7 2022-06-17T12:47:18.434 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:46:52.253609+0000', 'modified': '2022-06-17T12:46:54.268223+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4223}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4223': {'gid': 4223, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6833/3425584732', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 3425584732}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 3425584732}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:47:18.435 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:47:18.442 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-17T12:47:18.570 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":4223,"want_state":"up:active","state":"up:active","rank_uptime":26.316263912,"mdsmap_epoch":7,"osdmap_epoch":26,"osdmap_epoch_barrier":26,"uptime":32.764324629000001} 2022-06-17T12:47:18.573 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 4223, "mdsmap_epoch": 7, "osdmap_epoch": 26, "osdmap_epoch_barrier": 26, "rank_uptime": 26.316263912, "state": "up:active", "uptime": 32.764324629, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:47:18.574 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:47:18.575 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2022-06-17T12:47:18.576 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.2 at ubuntu@smithi143.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.2... 2022-06-17T12:47:18.576 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:18.577 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:47:18.625 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2022-06-17T12:47:18.675 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:47:18.683 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:47:18.747 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:47:18.748 INFO:teuthology.orchestra.run.smithi143.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:47:18.750 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:18.751 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2022-06-17T12:47:18.798 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:47:18.798 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.2 /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:47:18.809 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2022-06-17T12:47:18.827 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:47:18.828 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:47:18.891 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi143.stderr:2022-06-17T12:47:18.900+0000 7fa612992080 -1 init, newargv = 0x556de8f373d0 newargc=9ceph-fuse[21779]: starting ceph client 2022-06-17T12:47:18.892 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi143.stderr: 2022-06-17T12:47:18.907 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi143.stderr:ceph-fuse[21779]: starting fuse 2022-06-17T12:47:18.913 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:47:18.915 INFO:teuthology.orchestra.run.smithi143.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:47:18.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:18.926 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2022-06-17T12:47:18.973 INFO:teuthology.orchestra.run.smithi143.stdout:52 2022-06-17T12:47:18.996 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:47:18.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:18.999 DEBUG:teuthology.orchestra.run.smithi143:> sudo python3 -c ' 2022-06-17T12:47:19.000 DEBUG:teuthology.orchestra.run.smithi143:> import glob 2022-06-17T12:47:19.001 DEBUG:teuthology.orchestra.run.smithi143:> import re 2022-06-17T12:47:19.001 DEBUG:teuthology.orchestra.run.smithi143:> import os 2022-06-17T12:47:19.002 DEBUG:teuthology.orchestra.run.smithi143:> import subprocess 2022-06-17T12:47:19.002 DEBUG:teuthology.orchestra.run.smithi143:> 2022-06-17T12:47:19.003 DEBUG:teuthology.orchestra.run.smithi143:> def find_socket(client_name): 2022-06-17T12:47:19.004 DEBUG:teuthology.orchestra.run.smithi143:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2022-06-17T12:47:19.012 DEBUG:teuthology.orchestra.run.smithi143:> files = glob.glob(asok_path) 2022-06-17T12:47:19.013 DEBUG:teuthology.orchestra.run.smithi143:> 2022-06-17T12:47:19.038 DEBUG:teuthology.orchestra.run.smithi143:> # Given a non-glob path, it better be there 2022-06-17T12:47:19.039 DEBUG:teuthology.orchestra.run.smithi143:> if "*" not in asok_path: 2022-06-17T12:47:19.040 DEBUG:teuthology.orchestra.run.smithi143:> assert(len(files) == 1) 2022-06-17T12:47:19.065 DEBUG:teuthology.orchestra.run.smithi143:> return files[0] 2022-06-17T12:47:19.065 DEBUG:teuthology.orchestra.run.smithi143:> 2022-06-17T12:47:19.066 DEBUG:teuthology.orchestra.run.smithi143:> for f in files: 2022-06-17T12:47:19.067 DEBUG:teuthology.orchestra.run.smithi143:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:47:19.068 DEBUG:teuthology.orchestra.run.smithi143:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:47:19.068 DEBUG:teuthology.orchestra.run.smithi143:> return f 2022-06-17T12:47:19.069 DEBUG:teuthology.orchestra.run.smithi143:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:47:19.069 DEBUG:teuthology.orchestra.run.smithi143:> 2022-06-17T12:47:19.090 DEBUG:teuthology.orchestra.run.smithi143:> print(find_socket("client.2")) 2022-06-17T12:47:19.091 DEBUG:teuthology.orchestra.run.smithi143:> ' 2022-06-17T12:47:19.181 INFO:teuthology.orchestra.run.smithi143.stdout:/var/run/ceph/ceph-client.2.21779.asok 2022-06-17T12:47:19.181 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.21779.asok 2022-06-17T12:47:19.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:19.182 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.21779.asok status 2022-06-17T12:47:19.482 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2022-06-17T12:47:19.483 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": { 2022-06-17T12:47:19.483 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:47:19.484 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:47:19.484 INFO:teuthology.orchestra.run.smithi143.stdout: "entity_id": "2", 2022-06-17T12:47:19.493 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": "smithi143", 2022-06-17T12:47:19.494 INFO:teuthology.orchestra.run.smithi143.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2022-06-17T12:47:19.495 INFO:teuthology.orchestra.run.smithi143.stdout: "pid": "21779", 2022-06-17T12:47:19.495 INFO:teuthology.orchestra.run.smithi143.stdout: "root": "/" 2022-06-17T12:47:19.496 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2022-06-17T12:47:19.496 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_count": 0, 2022-06-17T12:47:19.497 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_pinned_count": 0, 2022-06-17T12:47:19.497 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 4684, 2022-06-17T12:47:19.497 INFO:teuthology.orchestra.run.smithi143.stdout: "inst": { 2022-06-17T12:47:19.498 INFO:teuthology.orchestra.run.smithi143.stdout: "name": { 2022-06-17T12:47:19.499 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "client", 2022-06-17T12:47:19.499 INFO:teuthology.orchestra.run.smithi143.stdout: "num": 4684 2022-06-17T12:47:19.499 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2022-06-17T12:47:19.500 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2022-06-17T12:47:19.500 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2022-06-17T12:47:19.501 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "172.21.15.143:0", 2022-06-17T12:47:19.501 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 1144709035 2022-06-17T12:47:19.502 INFO:teuthology.orchestra.run.smithi143.stdout: } 2022-06-17T12:47:19.503 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2022-06-17T12:47:19.503 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2022-06-17T12:47:19.504 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2022-06-17T12:47:19.504 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "172.21.15.143:0", 2022-06-17T12:47:19.505 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 1144709035 2022-06-17T12:47:19.505 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2022-06-17T12:47:19.506 INFO:teuthology.orchestra.run.smithi143.stdout: "inst_str": "client.4684 172.21.15.143:0/1144709035", 2022-06-17T12:47:19.506 INFO:teuthology.orchestra.run.smithi143.stdout: "addr_str": "172.21.15.143:0/1144709035", 2022-06-17T12:47:19.507 INFO:teuthology.orchestra.run.smithi143.stdout: "inode_count": 1, 2022-06-17T12:47:19.507 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_epoch": 7, 2022-06-17T12:47:19.507 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch": 26, 2022-06-17T12:47:19.508 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:47:19.508 INFO:teuthology.orchestra.run.smithi143.stdout: "blacklisted": false, 2022-06-17T12:47:19.509 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs" 2022-06-17T12:47:19.509 INFO:teuthology.orchestra.run.smithi143.stdout:} 2022-06-17T12:47:19.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:19.566 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:47:19.567 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:47:19.639 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:47:19.639 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:47:19.639 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:19.649 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:47:19.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:19.714 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:47:19.715 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:47:19.765 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:47:19.766 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:47:19.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:19.767 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:47:19.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:19.801 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:47:19.802 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:47:19.980 INFO:teuthology.orchestra.run.smithi143.stdout:fuseblk 2022-06-17T12:47:19.981 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2022-06-17T12:47:19.981 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:19.982 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:47:20.045 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2022-06-17T12:47:20.445 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:47:20.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.645+0000 7f05ed732700 1 Processor -- start 2022-06-17T12:47:20.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.645+0000 7f05ed732700 1 -- start start 2022-06-17T12:47:20.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.645+0000 7f05ed732700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f05e80b2200 0x7f05e80af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:20.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.645+0000 7f05ed732700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f05e80b2cd0 0x7f05e8147600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:20.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.645+0000 7f05ed732700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e80b25e0 con 0x7f05e80af8d0 2022-06-17T12:47:20.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.645+0000 7f05ed732700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e80ad790 con 0x7f05e80b2cd0 2022-06-17T12:47:20.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.645+0000 7f05ed732700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e80aef40 con 0x7f05e80b2200 2022-06-17T12:47:20.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.649+0000 7f05ecf31700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f05e80b2cd0 0x7f05e8147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:20.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.649+0000 7f05ecf31700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f05e80b2cd0 0x7f05e8147600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60246/0 (socket says 172.21.15.36:60246) 2022-06-17T12:47:20.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.649+0000 7f05ecf31700 1 -- 172.21.15.36:0/2263423993 learned_addr learned my addr 172.21.15.36:0/2263423993 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:20.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.649+0000 7f05ecf31700 1 -- 172.21.15.36:0/2263423993 >> v2:172.21.15.70:3300/0 conn(0x7f05e80b2200 msgr2=0x7f05e80af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:20.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.649+0000 7f05e7fff700 1 --2- 172.21.15.36:0/2263423993 >> v2:172.21.15.70:3300/0 conn(0x7f05e80b2200 0x7f05e80af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:20.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.649+0000 7f05ecf31700 1 --2- 172.21.15.36:0/2263423993 >> v2:172.21.15.70:3300/0 conn(0x7f05e80b2200 0x7f05e80af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:20.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.649+0000 7f05ecf31700 1 -- 172.21.15.36:0/2263423993 >> v1:172.21.15.70:6789/0 conn(0x7f05e80af8d0 legacy=0x7f05e814a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:20.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.649+0000 7f05ecf31700 1 -- 172.21.15.36:0/2263423993 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e80b1950 con 0x7f05e80b2cd0 2022-06-17T12:47:20.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.649+0000 7f05e7fff700 1 --2- 172.21.15.36:0/2263423993 >> v2:172.21.15.70:3300/0 conn(0x7f05e80b2200 0x7f05e80af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:47:20.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.649+0000 7f05ecf31700 1 --2- 172.21.15.36:0/2263423993 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05e80b2cd0 0x7f05e8147600 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f05e00032f0 tx=0x7f05e0009290).ready entity=mon.0 client_cookie=38da9f584ebd16b3 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:20.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.649+0000 7f05e6ffd700 1 -- 172.21.15.36:0/2263423993 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05e000b870 con 0x7f05e80b2cd0 2022-06-17T12:47:20.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.649+0000 7f05e6ffd700 1 -- 172.21.15.36:0/2263423993 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e000b9d0 con 0x7f05e80b2cd0 2022-06-17T12:47:20.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.649+0000 7f05e6ffd700 1 -- 172.21.15.36:0/2263423993 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05e000bb40 con 0x7f05e80b2cd0 2022-06-17T12:47:20.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.649+0000 7f05ed732700 1 -- 172.21.15.36:0/2263423993 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05e80b2cd0 msgr2=0x7f05e8147600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:20.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.649+0000 7f05ed732700 1 --2- 172.21.15.36:0/2263423993 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05e80b2cd0 0x7f05e8147600 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f05e00032f0 tx=0x7f05e0009290).stop 2022-06-17T12:47:20.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.649+0000 7f05ed732700 1 -- 172.21.15.36:0/2263423993 shutdown_connections 2022-06-17T12:47:20.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.649+0000 7f05ed732700 1 --2- 172.21.15.36:0/2263423993 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05e80b2cd0 0x7f05e8147600 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:20.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.649+0000 7f05ed732700 1 --2- 172.21.15.36:0/2263423993 >> v2:172.21.15.70:3300/0 conn(0x7f05e80b2200 0x7f05e80af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:20.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.649+0000 7f05ed732700 1 -- 172.21.15.36:0/2263423993 >> 172.21.15.36:0/2263423993 conn(0x7f05e805f770 msgr2=0x7f05e805fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:20.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.649+0000 7f05ed732700 1 -- 172.21.15.36:0/2263423993 shutdown_connections 2022-06-17T12:47:20.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.649+0000 7f05ed732700 1 -- 172.21.15.36:0/2263423993 wait complete. 2022-06-17T12:47:20.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.649+0000 7f05ed732700 1 Processor -- start 2022-06-17T12:47:20.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.649+0000 7f05ed732700 1 -- start start 2022-06-17T12:47:20.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.653+0000 7f05ed732700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05e80af8d0 0x7f05e80a7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:20.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.653+0000 7f05ed732700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f05e80b2200 0x7f05e815a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:20.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.653+0000 7f05ed732700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05e80b2cd0 0x7f05e815a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:20.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.653+0000 7f05ed732700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e8144dd0 con 0x7f05e80b2cd0 2022-06-17T12:47:20.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.653+0000 7f05ed732700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e8145da0 con 0x7f05e80af8d0 2022-06-17T12:47:20.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.653+0000 7f05ed732700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e8145fd0 con 0x7f05e80b2200 2022-06-17T12:47:20.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.653+0000 7f05ecf31700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05e80b2cd0 0x7f05e815a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:20.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.653+0000 7f05ecf31700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05e80b2cd0 0x7f05e815a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60252/0 (socket says 172.21.15.36:60252) 2022-06-17T12:47:21.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.653+0000 7f05ecf31700 1 -- 172.21.15.36:0/1967651490 learned_addr learned my addr 172.21.15.36:0/1967651490 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:21.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.653+0000 7f05e7fff700 1 --2- 172.21.15.36:0/1967651490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05e80af8d0 0x7f05e80a7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:21.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.653+0000 7f05e77fe700 1 --2- 172.21.15.36:0/1967651490 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f05e80b2200 0x7f05e815a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:21.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.653+0000 7f05ecf31700 1 -- 172.21.15.36:0/1967651490 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f05e80b2200 msgr2=0x7f05e815a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:21.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.653+0000 7f05ecf31700 1 --2- 172.21.15.36:0/1967651490 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f05e80b2200 0x7f05e815a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:21.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.653+0000 7f05ecf31700 1 -- 172.21.15.36:0/1967651490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05e80af8d0 msgr2=0x7f05e80a7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:21.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.653+0000 7f05ecf31700 1 --2- 172.21.15.36:0/1967651490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05e80af8d0 0x7f05e80a7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:21.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.653+0000 7f05ecf31700 1 -- 172.21.15.36:0/1967651490 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e0009530 con 0x7f05e80b2cd0 2022-06-17T12:47:21.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.653+0000 7f05ecf31700 1 --2- 172.21.15.36:0/1967651490 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05e80b2cd0 0x7f05e815a8d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f05e00032f0 tx=0x7f05e0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:21.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.653+0000 7f05e57fa700 1 -- 172.21.15.36:0/1967651490 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05e0021630 con 0x7f05e80b2cd0 2022-06-17T12:47:21.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.653+0000 7f05e57fa700 1 -- 172.21.15.36:0/1967651490 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e0021790 con 0x7f05e80b2cd0 2022-06-17T12:47:21.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.653+0000 7f05e57fa700 1 -- 172.21.15.36:0/1967651490 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05e0021af0 con 0x7f05e80b2cd0 2022-06-17T12:47:21.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.653+0000 7f05ed732700 1 -- 172.21.15.36:0/1967651490 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e80a7a70 con 0x7f05e80b2cd0 2022-06-17T12:47:21.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.653+0000 7f05ed732700 1 -- 172.21.15.36:0/1967651490 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e81493b0 con 0x7f05e80b2cd0 2022-06-17T12:47:21.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.657+0000 7f05ed732700 1 -- 172.21.15.36:0/1967651490 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e8144b10 con 0x7f05e80b2cd0 2022-06-17T12:47:21.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.657+0000 7f05e57fa700 1 -- 172.21.15.36:0/1967651490 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f05e0027020 con 0x7f05e80b2cd0 2022-06-17T12:47:21.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.657+0000 7f05e57fa700 1 --2- 172.21.15.36:0/1967651490 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f05c8053a40 0x7f05c8055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:21.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.657+0000 7f05e57fa700 1 -- 172.21.15.36:0/1967651490 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f05e006d900 con 0x7f05e80b2cd0 2022-06-17T12:47:21.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.657+0000 7f05e7fff700 1 --2- 172.21.15.36:0/1967651490 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f05c8053a40 0x7f05c8055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:21.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.657+0000 7f05e7fff700 1 --2- 172.21.15.36:0/1967651490 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f05c8053a40 0x7f05c8055f00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f05dc00d000 tx=0x7f05dc008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:21.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.665+0000 7f05e57fa700 1 -- 172.21.15.36:0/1967651490 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f05e8144b10 con 0x7f05e80b2cd0 2022-06-17T12:47:21.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.797+0000 7f05ed732700 1 -- 172.21.15.36:0/1967651490 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05e8144b10 con 0x7f05e80b2cd0 2022-06-17T12:47:21.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.797+0000 7f05e57fa700 1 -- 172.21.15.36:0/1967651490 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f05e8144b10 con 0x7f05e80b2cd0 2022-06-17T12:47:21.139 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:21.140 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":26,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:46:53.258075+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:46:47.465328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-17T12:46:49.893338+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:47:21.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.801+0000 7f05ed732700 1 -- 172.21.15.36:0/1967651490 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f05c8053a40 msgr2=0x7f05c8055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:21.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.801+0000 7f05ed732700 1 --2- 172.21.15.36:0/1967651490 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f05c8053a40 0x7f05c8055f00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f05dc00d000 tx=0x7f05dc008040).stop 2022-06-17T12:47:21.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.801+0000 7f05ed732700 1 -- 172.21.15.36:0/1967651490 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05e80b2cd0 msgr2=0x7f05e815a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:21.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.801+0000 7f05ed732700 1 --2- 172.21.15.36:0/1967651490 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05e80b2cd0 0x7f05e815a8d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f05e00032f0 tx=0x7f05e0016000).stop 2022-06-17T12:47:21.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.801+0000 7f05ed732700 1 -- 172.21.15.36:0/1967651490 shutdown_connections 2022-06-17T12:47:21.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.801+0000 7f05ed732700 1 --2- 172.21.15.36:0/1967651490 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f05c8053a40 0x7f05c8055f00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:21.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.801+0000 7f05ed732700 1 --2- 172.21.15.36:0/1967651490 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05e80b2cd0 0x7f05e815a8d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:21.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.801+0000 7f05ed732700 1 --2- 172.21.15.36:0/1967651490 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f05e80b2200 0x7f05e815a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:21.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.801+0000 7f05ed732700 1 --2- 172.21.15.36:0/1967651490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05e80af8d0 0x7f05e80a7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:21.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.801+0000 7f05ed732700 1 -- 172.21.15.36:0/1967651490 >> 172.21.15.36:0/1967651490 conn(0x7f05e805f770 msgr2=0x7f05e80b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:21.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.801+0000 7f05ed732700 1 -- 172.21.15.36:0/1967651490 shutdown_connections 2022-06-17T12:47:21.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:20.801+0000 7f05ed732700 1 -- 172.21.15.36:0/1967651490 wait complete. 2022-06-17T12:47:21.205 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:46:47.465328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:46:49.893338+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:47:21.206 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:47:21.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.337+0000 7fbf126bd700 1 Processor -- start 2022-06-17T12:47:21.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.337+0000 7fbf126bd700 1 -- start start 2022-06-17T12:47:21.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.337+0000 7fbf126bd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbf0c060ca0 0x7fbf0c146c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:21.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.337+0000 7fbf126bd700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf0c0b2b90 con 0x7fbf0c0aeac0 2022-06-17T12:47:21.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.337+0000 7fbf126bd700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf0c0af610 con 0x7fbf0c0b27b0 2022-06-17T12:47:21.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.337+0000 7fbf126bd700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf0c066b90 con 0x7fbf0c060ca0 2022-06-17T12:47:21.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.337+0000 7fbf11ebc700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fbf0c0aeac0 0x7fbf0c143360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49556/0 (socket says 172.21.15.36:49556) 2022-06-17T12:47:21.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.337+0000 7fbf10eba700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbf0c060ca0 0x7fbf0c146c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:21.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.337+0000 7fbf11ebc700 1 -- 172.21.15.36:0/2740279934 learned_addr learned my addr 172.21.15.36:0/2740279934 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:21.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.337+0000 7fbf03fff700 1 -- 172.21.15.36:0/2740279934 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 914777847 0 0) 0x7fbf0c0b2b90 con 0x7fbf0c0aeac0 2022-06-17T12:47:21.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.337+0000 7fbf03fff700 1 -- 172.21.15.36:0/2740279934 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbef0003260 con 0x7fbf0c0aeac0 2022-06-17T12:47:21.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.337+0000 7fbf10eba700 1 -- 172.21.15.36:0/2740279934 >> v1:172.21.15.70:6789/0 conn(0x7fbf0c0b27b0 legacy=0x7fbf0c146520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:21.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.337+0000 7fbf10eba700 1 -- 172.21.15.36:0/2740279934 >> v1:172.21.15.36:6789/0 conn(0x7fbf0c0aeac0 legacy=0x7fbf0c143360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:21.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.341+0000 7fbf10eba700 1 -- 172.21.15.36:0/2740279934 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0c0620f0 con 0x7fbf0c060ca0 2022-06-17T12:47:21.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.341+0000 7fbf10eba700 1 --2- 172.21.15.36:0/2740279934 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbf0c060ca0 0x7fbf0c146c30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7fbf08002f70 tx=0x7fbf08006df0).ready entity=mon.0 client_cookie=e99a876e99fc7332 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:21.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.341+0000 7fbf03fff700 1 -- 172.21.15.36:0/2740279934 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf0800d610 con 0x7fbf0c060ca0 2022-06-17T12:47:21.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.341+0000 7fbf03fff700 1 -- 172.21.15.36:0/2740279934 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf08000e90 con 0x7fbf0c060ca0 2022-06-17T12:47:21.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.341+0000 7fbf03fff700 1 -- 172.21.15.36:0/2740279934 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf0800d960 con 0x7fbf0c060ca0 2022-06-17T12:47:21.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.341+0000 7fbf126bd700 1 -- 172.21.15.36:0/2740279934 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbf0c060ca0 msgr2=0x7fbf0c146c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:21.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.341+0000 7fbf126bd700 1 --2- 172.21.15.36:0/2740279934 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbf0c060ca0 0x7fbf0c146c30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7fbf08002f70 tx=0x7fbf08006df0).stop 2022-06-17T12:47:21.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.341+0000 7fbf126bd700 1 -- 172.21.15.36:0/2740279934 shutdown_connections 2022-06-17T12:47:21.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.341+0000 7fbf126bd700 1 --2- 172.21.15.36:0/2740279934 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbf0c060ca0 0x7fbf0c146c30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:21.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.341+0000 7fbf126bd700 1 -- 172.21.15.36:0/2740279934 >> 172.21.15.36:0/2740279934 conn(0x7fbf0c0b5d10 msgr2=0x7fbf0c0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:21.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.341+0000 7fbf126bd700 1 -- 172.21.15.36:0/2740279934 shutdown_connections 2022-06-17T12:47:21.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.341+0000 7fbf126bd700 1 -- 172.21.15.36:0/2740279934 wait complete. 2022-06-17T12:47:21.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.341+0000 7fbf126bd700 1 Processor -- start 2022-06-17T12:47:21.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.341+0000 7fbf126bd700 1 -- start start 2022-06-17T12:47:21.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf126bd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf0c060ca0 0x7fbf0c144850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:21.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf126bd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf0c0b27b0 0x7fbf0c09f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:21.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf126bd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbf0c09e8d0 0x7fbf0c155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:21.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf126bd700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c0a0980 con 0x7fbf0c09e8d0 2022-06-17T12:47:21.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf126bd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c155810 con 0x7fbf0c060ca0 2022-06-17T12:47:21.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf126bd700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c155a40 con 0x7fbf0c0b27b0 2022-06-17T12:47:21.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf11ebc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbf0c09e8d0 0x7fbf0c155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:21.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf11ebc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbf0c09e8d0 0x7fbf0c155ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60266/0 (socket says 172.21.15.36:60266) 2022-06-17T12:47:21.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf11ebc700 1 -- 172.21.15.36:0/4082414636 learned_addr learned my addr 172.21.15.36:0/4082414636 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:21.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf116bb700 1 --2- 172.21.15.36:0/4082414636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf0c060ca0 0x7fbf0c144850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:21.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf10eba700 1 --2- 172.21.15.36:0/4082414636 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf0c0b27b0 0x7fbf0c09f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:21.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf11ebc700 1 -- 172.21.15.36:0/4082414636 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf0c0b27b0 msgr2=0x7fbf0c09f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:21.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf11ebc700 1 --2- 172.21.15.36:0/4082414636 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf0c0b27b0 0x7fbf0c09f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:21.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf11ebc700 1 -- 172.21.15.36:0/4082414636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf0c060ca0 msgr2=0x7fbf0c144850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:21.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf11ebc700 1 --2- 172.21.15.36:0/4082414636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf0c060ca0 0x7fbf0c144850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:21.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf11ebc700 1 -- 172.21.15.36:0/4082414636 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0800b180 con 0x7fbf0c09e8d0 2022-06-17T12:47:21.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf116bb700 1 --2- 172.21.15.36:0/4082414636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf0c060ca0 0x7fbf0c144850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:21.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf11ebc700 1 --2- 172.21.15.36:0/4082414636 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbf0c09e8d0 0x7fbf0c155ee0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fbf04001490 tx=0x7fbf04007da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:21.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf027fc700 1 -- 172.21.15.36:0/4082414636 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf04005c70 con 0x7fbf0c09e8d0 2022-06-17T12:47:21.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf027fc700 1 -- 172.21.15.36:0/4082414636 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf04005dd0 con 0x7fbf0c09e8d0 2022-06-17T12:47:21.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf126bd700 1 -- 172.21.15.36:0/4082414636 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf0c0a1ee0 con 0x7fbf0c09e8d0 2022-06-17T12:47:21.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf027fc700 1 -- 172.21.15.36:0/4082414636 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf040072c0 con 0x7fbf0c09e8d0 2022-06-17T12:47:21.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf126bd700 1 -- 172.21.15.36:0/4082414636 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf0c09fcd0 con 0x7fbf0c09e8d0 2022-06-17T12:47:21.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf027fc700 1 -- 172.21.15.36:0/4082414636 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fbf0401a060 con 0x7fbf0c09e8d0 2022-06-17T12:47:21.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.345+0000 7fbf126bd700 1 -- 172.21.15.36:0/4082414636 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf0c066880 con 0x7fbf0c09e8d0 2022-06-17T12:47:21.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.349+0000 7fbf027fc700 1 --2- 172.21.15.36:0/4082414636 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbef40539b0 0x7fbef4055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:21.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.349+0000 7fbf027fc700 1 -- 172.21.15.36:0/4082414636 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fbf0406b9f0 con 0x7fbf0c09e8d0 2022-06-17T12:47:21.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.349+0000 7fbf116bb700 1 --2- 172.21.15.36:0/4082414636 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbef40539b0 0x7fbef4055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:21.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.349+0000 7fbf116bb700 1 --2- 172.21.15.36:0/4082414636 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbef40539b0 0x7fbef4055e70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fbefc00b000 tx=0x7fbefc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:21.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.353+0000 7fbf027fc700 1 -- 172.21.15.36:0/4082414636 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbf0406de60 con 0x7fbf0c09e8d0 2022-06-17T12:47:21.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.481+0000 7fbf126bd700 1 -- 172.21.15.36:0/4082414636 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fbf0c057410 con 0x7fbf0c09e8d0 2022-06-17T12:47:21.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.481+0000 7fbf027fc700 1 -- 172.21.15.36:0/4082414636 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7fbf04018070 con 0x7fbf0c09e8d0 2022-06-17T12:47:21.670 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:47:21.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.481+0000 7fbf126bd700 1 -- 172.21.15.36:0/4082414636 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbef40539b0 msgr2=0x7fbef4055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:21.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.481+0000 7fbf126bd700 1 --2- 172.21.15.36:0/4082414636 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbef40539b0 0x7fbef4055e70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fbefc00b000 tx=0x7fbefc009040).stop 2022-06-17T12:47:21.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.481+0000 7fbf126bd700 1 -- 172.21.15.36:0/4082414636 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbf0c09e8d0 msgr2=0x7fbf0c155ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:21.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.481+0000 7fbf126bd700 1 --2- 172.21.15.36:0/4082414636 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbf0c09e8d0 0x7fbf0c155ee0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fbf04001490 tx=0x7fbf04007da0).stop 2022-06-17T12:47:21.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.481+0000 7fbf126bd700 1 -- 172.21.15.36:0/4082414636 shutdown_connections 2022-06-17T12:47:21.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.481+0000 7fbf126bd700 1 --2- 172.21.15.36:0/4082414636 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbef40539b0 0x7fbef4055e70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:21.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.481+0000 7fbf126bd700 1 --2- 172.21.15.36:0/4082414636 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbf0c09e8d0 0x7fbf0c155ee0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:21.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.481+0000 7fbf126bd700 1 --2- 172.21.15.36:0/4082414636 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf0c0b27b0 0x7fbf0c09f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:21.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.481+0000 7fbf126bd700 1 --2- 172.21.15.36:0/4082414636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf0c060ca0 0x7fbf0c144850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:21.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.481+0000 7fbf126bd700 1 -- 172.21.15.36:0/4082414636 >> 172.21.15.36:0/4082414636 conn(0x7fbf0c0b5d10 msgr2=0x7fbf0c1417e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:21.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.481+0000 7fbf126bd700 1 -- 172.21.15.36:0/4082414636 shutdown_connections 2022-06-17T12:47:21.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.485+0000 7fbf126bd700 1 -- 172.21.15.36:0/4082414636 wait complete. 2022-06-17T12:47:21.690 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:47:21.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.885+0000 7f7a3ac18700 1 Processor -- start 2022-06-17T12:47:21.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.885+0000 7f7a3ac18700 1 -- start start 2022-06-17T12:47:21.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.885+0000 7f7a3ac18700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7a340b4190 0x7f7a340b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:21.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.885+0000 7f7a3ac18700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a34143d40 0x7f7a341471d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:21.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.885+0000 7f7a3ac18700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a34143130 con 0x7f7a34062270 2022-06-17T12:47:21.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.885+0000 7f7a3ac18700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a340b6080 con 0x7f7a340b4190 2022-06-17T12:47:21.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.885+0000 7f7a3ac18700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a34062650 con 0x7f7a34143d40 2022-06-17T12:47:21.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.885+0000 7f7a33fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7a340b4190 0x7f7a340b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:21.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.885+0000 7f7a389b4700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f7a34062270 0x7f7a340b48b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49568/0 (socket says 172.21.15.36:49568) 2022-06-17T12:47:21.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.889+0000 7f7a389b4700 1 -- 172.21.15.36:0/456802199 learned_addr learned my addr 172.21.15.36:0/456802199 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:21.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.889+0000 7f7a391b5700 1 --2- 172.21.15.36:0/456802199 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a34143d40 0x7f7a341471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:21.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.889+0000 7f7a33fff700 1 -- 172.21.15.36:0/456802199 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a34143d40 msgr2=0x7f7a341471d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:21.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.889+0000 7f7a33fff700 1 --2- 172.21.15.36:0/456802199 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a34143d40 0x7f7a341471d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:21.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.889+0000 7f7a33fff700 1 -- 172.21.15.36:0/456802199 >> v1:172.21.15.36:6789/0 conn(0x7f7a34062270 legacy=0x7f7a340b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:21.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.889+0000 7f7a33fff700 1 -- 172.21.15.36:0/456802199 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a34143360 con 0x7f7a340b4190 2022-06-17T12:47:21.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.889+0000 7f7a337fe700 1 -- 172.21.15.36:0/456802199 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3370921178 0 0) 0x7f7a34143130 con 0x7f7a34062270 2022-06-17T12:47:21.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.889+0000 7f7a33fff700 1 --2- 172.21.15.36:0/456802199 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7a340b4190 0x7f7a340b4fc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f7a1c002f70 tx=0x7f7a1c006df0).ready entity=mon.0 client_cookie=9abf8188baa6f6e1 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:21.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.889+0000 7f7a337fe700 1 -- 172.21.15.36:0/456802199 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a1c00d570 con 0x7f7a340b4190 2022-06-17T12:47:21.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.889+0000 7f7a337fe700 1 -- 172.21.15.36:0/456802199 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a1c000e90 con 0x7f7a340b4190 2022-06-17T12:47:21.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.889+0000 7f7a337fe700 1 -- 172.21.15.36:0/456802199 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a1c00d8c0 con 0x7f7a340b4190 2022-06-17T12:47:21.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.889+0000 7f7a3ac18700 1 -- 172.21.15.36:0/456802199 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7a340b4190 msgr2=0x7f7a340b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:21.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.889+0000 7f7a3ac18700 1 --2- 172.21.15.36:0/456802199 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7a340b4190 0x7f7a340b4fc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f7a1c002f70 tx=0x7f7a1c006df0).stop 2022-06-17T12:47:21.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.889+0000 7f7a3ac18700 1 -- 172.21.15.36:0/456802199 shutdown_connections 2022-06-17T12:47:21.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.889+0000 7f7a3ac18700 1 --2- 172.21.15.36:0/456802199 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a34143d40 0x7f7a341471d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:21.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.889+0000 7f7a3ac18700 1 --2- 172.21.15.36:0/456802199 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7a340b4190 0x7f7a340b4fc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:21.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.889+0000 7f7a3ac18700 1 -- 172.21.15.36:0/456802199 >> 172.21.15.36:0/456802199 conn(0x7f7a340b97f0 msgr2=0x7f7a340b9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:21.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.889+0000 7f7a3ac18700 1 -- 172.21.15.36:0/456802199 shutdown_connections 2022-06-17T12:47:21.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.889+0000 7f7a3ac18700 1 -- 172.21.15.36:0/456802199 wait complete. 2022-06-17T12:47:21.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a3ac18700 1 Processor -- start 2022-06-17T12:47:21.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a3ac18700 1 -- start start 2022-06-17T12:47:21.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a3ac18700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a34062270 0x7f7a3412c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:21.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a3ac18700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7a340b4190 0x7f7a3412f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:21.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a3ac18700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a34143d40 0x7f7a3412fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:21.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a3ac18700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a340b3910 con 0x7f7a340b4190 2022-06-17T12:47:21.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a3ac18700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a340b3bb0 con 0x7f7a34143d40 2022-06-17T12:47:21.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a3ac18700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3412ff10 con 0x7f7a34062270 2022-06-17T12:47:21.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a33fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7a340b4190 0x7f7a3412f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:21.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a33fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7a340b4190 0x7f7a3412f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60276/0 (socket says 172.21.15.36:60276) 2022-06-17T12:47:21.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a33fff700 1 -- 172.21.15.36:0/2049444117 learned_addr learned my addr 172.21.15.36:0/2049444117 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:21.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a389b4700 1 --2- 172.21.15.36:0/2049444117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a34062270 0x7f7a3412c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:21.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a391b5700 1 --2- 172.21.15.36:0/2049444117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a34143d40 0x7f7a3412fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:21.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a33fff700 1 -- 172.21.15.36:0/2049444117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a34062270 msgr2=0x7f7a3412c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:21.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a33fff700 1 --2- 172.21.15.36:0/2049444117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a34062270 0x7f7a3412c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:21.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a33fff700 1 -- 172.21.15.36:0/2049444117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a34143d40 msgr2=0x7f7a3412fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:21.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a33fff700 1 --2- 172.21.15.36:0/2049444117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a34143d40 0x7f7a3412fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:21.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a33fff700 1 -- 172.21.15.36:0/2049444117 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1c00b180 con 0x7f7a340b4190 2022-06-17T12:47:21.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a391b5700 1 --2- 172.21.15.36:0/2049444117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a34143d40 0x7f7a3412fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:21.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a33fff700 1 --2- 172.21.15.36:0/2049444117 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7a340b4190 0x7f7a3412f530 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f7a1c002f70 tx=0x7f7a1c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:21.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a31ffb700 1 -- 172.21.15.36:0/2049444117 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a1c00dea0 con 0x7f7a340b4190 2022-06-17T12:47:21.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a31ffb700 1 -- 172.21.15.36:0/2049444117 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a1c013e90 con 0x7f7a340b4190 2022-06-17T12:47:21.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a3ac18700 1 -- 172.21.15.36:0/2049444117 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a340b2eb0 con 0x7f7a340b4190 2022-06-17T12:47:21.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a31ffb700 1 -- 172.21.15.36:0/2049444117 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a1c01e5f0 con 0x7f7a340b4190 2022-06-17T12:47:21.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.893+0000 7f7a3ac18700 1 -- 172.21.15.36:0/2049444117 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a34142c40 con 0x7f7a340b4190 2022-06-17T12:47:21.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.897+0000 7f7a31ffb700 1 -- 172.21.15.36:0/2049444117 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f7a1c025030 con 0x7f7a340b4190 2022-06-17T12:47:21.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.897+0000 7f7a3ac18700 1 -- 172.21.15.36:0/2049444117 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a3412cd90 con 0x7f7a340b4190 2022-06-17T12:47:21.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.897+0000 7f7a31ffb700 1 --2- 172.21.15.36:0/2049444117 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7a18053a40 0x7f7a18055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:21.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.897+0000 7f7a31ffb700 1 -- 172.21.15.36:0/2049444117 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f7a1c06cd40 con 0x7f7a340b4190 2022-06-17T12:47:21.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.897+0000 7f7a389b4700 1 --2- 172.21.15.36:0/2049444117 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7a18053a40 0x7f7a18055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:21.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.897+0000 7f7a389b4700 1 --2- 172.21.15.36:0/2049444117 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7a18053a40 0x7f7a18055f00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f7a2400d000 tx=0x7f7a24007a20).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:21.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:21.901+0000 7f7a31ffb700 1 -- 172.21.15.36:0/2049444117 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7a1c043b40 con 0x7f7a340b4190 2022-06-17T12:47:22.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.029+0000 7f7a3ac18700 1 -- 172.21.15.36:0/2049444117 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7a34143600 con 0x7f7a340b4190 2022-06-17T12:47:22.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.029+0000 7f7a31ffb700 1 -- 172.21.15.36:0/2049444117 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7a1c023070 con 0x7f7a340b4190 2022-06-17T12:47:22.034 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:22.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.033+0000 7f7a3ac18700 1 -- 172.21.15.36:0/2049444117 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7a18053a40 msgr2=0x7f7a18055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:22.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.033+0000 7f7a3ac18700 1 --2- 172.21.15.36:0/2049444117 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7a18053a40 0x7f7a18055f00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f7a2400d000 tx=0x7f7a24007a20).stop 2022-06-17T12:47:22.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.033+0000 7f7a3ac18700 1 -- 172.21.15.36:0/2049444117 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7a340b4190 msgr2=0x7f7a3412f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:22.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.033+0000 7f7a3ac18700 1 --2- 172.21.15.36:0/2049444117 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7a340b4190 0x7f7a3412f530 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f7a1c002f70 tx=0x7f7a1c014000).stop 2022-06-17T12:47:22.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.033+0000 7f7a3ac18700 1 -- 172.21.15.36:0/2049444117 shutdown_connections 2022-06-17T12:47:22.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.033+0000 7f7a3ac18700 1 --2- 172.21.15.36:0/2049444117 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7a18053a40 0x7f7a18055f00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:22.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.033+0000 7f7a3ac18700 1 --2- 172.21.15.36:0/2049444117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a34143d40 0x7f7a3412fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:22.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.033+0000 7f7a3ac18700 1 --2- 172.21.15.36:0/2049444117 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7a340b4190 0x7f7a3412f530 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:22.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.033+0000 7f7a3ac18700 1 --2- 172.21.15.36:0/2049444117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a34062270 0x7f7a3412c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:22.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.033+0000 7f7a3ac18700 1 -- 172.21.15.36:0/2049444117 >> 172.21.15.36:0/2049444117 conn(0x7f7a340b97f0 msgr2=0x7f7a340b7620 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:22.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.033+0000 7f7a3ac18700 1 -- 172.21.15.36:0/2049444117 shutdown_connections 2022-06-17T12:47:22.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.033+0000 7f7a3ac18700 1 -- 172.21.15.36:0/2049444117 wait complete. 2022-06-17T12:47:22.124 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:47:22.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.253+0000 7fa0b5794700 1 Processor -- start 2022-06-17T12:47:22.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.253+0000 7fa0b5794700 1 -- start start 2022-06-17T12:47:22.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.257+0000 7fa0b5794700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa0b00b27b0 0x7fa0b00b1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:22.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.257+0000 7fa0b5794700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b00ac0b0 con 0x7fa0b0144d20 2022-06-17T12:47:22.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.257+0000 7fa0b5794700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b00b2b90 con 0x7fa0b0140940 2022-06-17T12:47:22.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.257+0000 7fa0b5794700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b0140d90 con 0x7fa0b00b27b0 2022-06-17T12:47:22.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.257+0000 7fa0affff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa0b00b27b0 0x7fa0b00b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:22.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.257+0000 7fa0af7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fa0b0144d20 0x7fa0b0140230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49584/0 (socket says 172.21.15.36:49584) 2022-06-17T12:47:22.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.257+0000 7fa0af7fe700 1 -- 172.21.15.36:0/1139967807 learned_addr learned my addr 172.21.15.36:0/1139967807 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:22.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.257+0000 7fa0affff700 1 -- 172.21.15.36:0/1139967807 >> v1:172.21.15.70:6789/0 conn(0x7fa0b0140940 legacy=0x7fa0b0141180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:22.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.257+0000 7fa0affff700 1 -- 172.21.15.36:0/1139967807 >> v1:172.21.15.36:6789/0 conn(0x7fa0b0144d20 legacy=0x7fa0b0140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:22.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.257+0000 7fa0aeffd700 1 -- 172.21.15.36:0/1139967807 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 400625568 0 0) 0x7fa0b00b2b90 con 0x7fa0b0140940 2022-06-17T12:47:22.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.257+0000 7fa0affff700 1 -- 172.21.15.36:0/1139967807 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b00afc60 con 0x7fa0b00b27b0 2022-06-17T12:47:22.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.257+0000 7fa0affff700 1 --2- 172.21.15.36:0/1139967807 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa0b00b27b0 0x7fa0b00b1830 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fa0b00b1d20 tx=0x7fa098006df0).ready entity=mon.0 client_cookie=4a2c9b30c4500033 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:22.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.257+0000 7fa0aeffd700 1 -- 172.21.15.36:0/1139967807 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa09800d5c0 con 0x7fa0b00b27b0 2022-06-17T12:47:22.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.257+0000 7fa0aeffd700 1 -- 172.21.15.36:0/1139967807 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa098000e90 con 0x7fa0b00b27b0 2022-06-17T12:47:22.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.257+0000 7fa0aeffd700 1 -- 172.21.15.36:0/1139967807 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa09800d910 con 0x7fa0b00b27b0 2022-06-17T12:47:22.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.257+0000 7fa0b5794700 1 -- 172.21.15.36:0/1139967807 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa0b00b27b0 msgr2=0x7fa0b00b1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:22.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.257+0000 7fa0b5794700 1 --2- 172.21.15.36:0/1139967807 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa0b00b27b0 0x7fa0b00b1830 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fa0b00b1d20 tx=0x7fa098006df0).stop 2022-06-17T12:47:22.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.257+0000 7fa0b5794700 1 -- 172.21.15.36:0/1139967807 shutdown_connections 2022-06-17T12:47:22.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.257+0000 7fa0b5794700 1 --2- 172.21.15.36:0/1139967807 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa0b00b27b0 0x7fa0b00b1830 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:22.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.257+0000 7fa0b5794700 1 -- 172.21.15.36:0/1139967807 >> 172.21.15.36:0/1139967807 conn(0x7fa0b00b5d10 msgr2=0x7fa0b00b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:22.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.257+0000 7fa0b5794700 1 -- 172.21.15.36:0/1139967807 shutdown_connections 2022-06-17T12:47:22.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.257+0000 7fa0b5794700 1 -- 172.21.15.36:0/1139967807 wait complete. 2022-06-17T12:47:22.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0b5794700 1 Processor -- start 2022-06-17T12:47:22.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0b5794700 1 -- start start 2022-06-17T12:47:22.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0b5794700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa0b00b27b0 0x7fa0b00adb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:22.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0b5794700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b0140940 0x7fa0b00acac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:22.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0b5794700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0b0144d20 0x7fa0b00b3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:22.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0b5794700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b013e410 con 0x7fa0b00b27b0 2022-06-17T12:47:22.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0b5794700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b00b3710 con 0x7fa0b0140940 2022-06-17T12:47:22.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0b5794700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b00ad400 con 0x7fa0b0144d20 2022-06-17T12:47:22.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0affff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa0b00b27b0 0x7fa0b00adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:22.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0affff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa0b00b27b0 0x7fa0b00adb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60290/0 (socket says 172.21.15.36:60290) 2022-06-17T12:47:22.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0affff700 1 -- 172.21.15.36:0/3937163212 learned_addr learned my addr 172.21.15.36:0/3937163212 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:22.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0af7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b0140940 0x7fa0b00acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:22.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0b4f93700 1 --2- 172.21.15.36:0/3937163212 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0b0144d20 0x7fa0b00b3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:22.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0affff700 1 -- 172.21.15.36:0/3937163212 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0b0144d20 msgr2=0x7fa0b00b3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:22.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0affff700 1 --2- 172.21.15.36:0/3937163212 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0b0144d20 0x7fa0b00b3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:22.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0affff700 1 -- 172.21.15.36:0/3937163212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b0140940 msgr2=0x7fa0b00acac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:22.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0affff700 1 --2- 172.21.15.36:0/3937163212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b0140940 0x7fa0b00acac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:22.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0affff700 1 -- 172.21.15.36:0/3937163212 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09800b180 con 0x7fa0b00b27b0 2022-06-17T12:47:22.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0af7fe700 1 --2- 172.21.15.36:0/3937163212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b0140940 0x7fa0b00acac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:22.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0affff700 1 --2- 172.21.15.36:0/3937163212 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa0b00b27b0 0x7fa0b00adb40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fa0b00b1d20 tx=0x7fa098014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:22.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0ad7fa700 1 -- 172.21.15.36:0/3937163212 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0980067e0 con 0x7fa0b00b27b0 2022-06-17T12:47:22.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0ad7fa700 1 -- 172.21.15.36:0/3937163212 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa098013de0 con 0x7fa0b00b27b0 2022-06-17T12:47:22.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0b5794700 1 -- 172.21.15.36:0/3937163212 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b00ad6a0 con 0x7fa0b00b27b0 2022-06-17T12:47:22.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0ad7fa700 1 -- 172.21.15.36:0/3937163212 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa09801e5f0 con 0x7fa0b00b27b0 2022-06-17T12:47:22.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.261+0000 7fa0b5794700 1 -- 172.21.15.36:0/3937163212 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b0067040 con 0x7fa0b00b27b0 2022-06-17T12:47:22.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.265+0000 7fa0ad7fa700 1 -- 172.21.15.36:0/3937163212 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fa098025030 con 0x7fa0b00b27b0 2022-06-17T12:47:22.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.265+0000 7fa0b5794700 1 -- 172.21.15.36:0/3937163212 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b00ac310 con 0x7fa0b00b27b0 2022-06-17T12:47:22.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.265+0000 7fa0ad7fa700 1 --2- 172.21.15.36:0/3937163212 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa094053920 0x7fa094055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:22.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.265+0000 7fa0ad7fa700 1 -- 172.21.15.36:0/3937163212 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa09806cf20 con 0x7fa0b00b27b0 2022-06-17T12:47:22.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.265+0000 7fa0af7fe700 1 --2- 172.21.15.36:0/3937163212 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa094053920 0x7fa094055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:22.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.265+0000 7fa0af7fe700 1 --2- 172.21.15.36:0/3937163212 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa094053920 0x7fa094055de0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fa0a000d000 tx=0x7fa0a0009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:22.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.269+0000 7fa0ad7fa700 1 -- 172.21.15.36:0/3937163212 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa098043b90 con 0x7fa0b00b27b0 2022-06-17T12:47:22.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.397+0000 7fa0b5794700 1 -- 172.21.15.36:0/3937163212 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa0b0057410 con 0x7fa0b00b27b0 2022-06-17T12:47:22.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.397+0000 7fa0ad7fa700 1 -- 172.21.15.36:0/3937163212 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+10 (secure 0 0 0) 0x7fa098023020 con 0x7fa0b00b27b0 2022-06-17T12:47:22.499 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:22.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.401+0000 7fa0b5794700 1 -- 172.21.15.36:0/3937163212 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa094053920 msgr2=0x7fa094055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:22.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.401+0000 7fa0b5794700 1 --2- 172.21.15.36:0/3937163212 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa094053920 0x7fa094055de0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fa0a000d000 tx=0x7fa0a0009040).stop 2022-06-17T12:47:22.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.401+0000 7fa0b5794700 1 -- 172.21.15.36:0/3937163212 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa0b00b27b0 msgr2=0x7fa0b00adb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:22.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.401+0000 7fa0b5794700 1 --2- 172.21.15.36:0/3937163212 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa0b00b27b0 0x7fa0b00adb40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fa0b00b1d20 tx=0x7fa098014000).stop 2022-06-17T12:47:22.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.401+0000 7fa0b5794700 1 -- 172.21.15.36:0/3937163212 shutdown_connections 2022-06-17T12:47:22.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.401+0000 7fa0b5794700 1 --2- 172.21.15.36:0/3937163212 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa094053920 0x7fa094055de0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:22.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.401+0000 7fa0b5794700 1 --2- 172.21.15.36:0/3937163212 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0b0144d20 0x7fa0b00b3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:22.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.401+0000 7fa0b5794700 1 --2- 172.21.15.36:0/3937163212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b0140940 0x7fa0b00acac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:22.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.401+0000 7fa0b5794700 1 --2- 172.21.15.36:0/3937163212 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa0b00b27b0 0x7fa0b00adb40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:22.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.401+0000 7fa0b5794700 1 -- 172.21.15.36:0/3937163212 >> 172.21.15.36:0/3937163212 conn(0x7fa0b00b5d10 msgr2=0x7fa0b01441f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:22.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.401+0000 7fa0b5794700 1 -- 172.21.15.36:0/3937163212 shutdown_connections 2022-06-17T12:47:22.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.401+0000 7fa0b5794700 1 -- 172.21.15.36:0/3937163212 wait complete. 2022-06-17T12:47:22.520 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:47:22.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.653+0000 7f7432248700 1 Processor -- start 2022-06-17T12:47:22.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.653+0000 7f7432248700 1 -- start start 2022-06-17T12:47:22.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f7432248700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f742c1473d0 0x7f742c05fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:22.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f7432248700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f742c060370 0x7f742c143200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:22.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f7432248700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f742c1467c0 con 0x7f742c0b2df0 2022-06-17T12:47:22.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f7432248700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f742c0b31d0 con 0x7f742c1473d0 2022-06-17T12:47:22.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f7432248700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742c1469f0 con 0x7f742c060370 2022-06-17T12:47:22.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f7423fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f742c1473d0 0x7f742c05fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:22.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f742bfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f742c060370 0x7f742c143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:22.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f742b7fe700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f742c0b2df0 0x7f742c05f770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:45330/0 (socket says 172.21.15.36:45330) 2022-06-17T12:47:22.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f742b7fe700 1 -- 172.21.15.36:0/1080581801 learned_addr learned my addr 172.21.15.36:0/1080581801 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:22.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f742affd700 1 -- 172.21.15.36:0/1080581801 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1368296917 0 0) 0x7f742c1467c0 con 0x7f742c0b2df0 2022-06-17T12:47:22.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f742affd700 1 -- 172.21.15.36:0/1080581801 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f740c003260 con 0x7f742c0b2df0 2022-06-17T12:47:22.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f7423fff700 1 -- 172.21.15.36:0/1080581801 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f742c060370 msgr2=0x7f742c143200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:22.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f7423fff700 1 --2- 172.21.15.36:0/1080581801 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f742c060370 0x7f742c143200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:22.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f7423fff700 1 -- 172.21.15.36:0/1080581801 >> v1:172.21.15.70:6789/0 conn(0x7f742c0b2df0 legacy=0x7f742c05f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:22.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f7423fff700 1 -- 172.21.15.36:0/1080581801 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742c146c90 con 0x7f742c1473d0 2022-06-17T12:47:22.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f742bfff700 1 --2- 172.21.15.36:0/1080581801 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f742c060370 0x7f742c143200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:22.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f7423fff700 1 --2- 172.21.15.36:0/1080581801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f742c1473d0 0x7f742c05fe80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f7414002f70 tx=0x7f7414006df0).ready entity=mon.1 client_cookie=faeab849242ab543 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:22.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f742affd700 1 -- 172.21.15.36:0/1080581801 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f741400d4a0 con 0x7f742c1473d0 2022-06-17T12:47:22.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f742affd700 1 -- 172.21.15.36:0/1080581801 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7414000e90 con 0x7f742c1473d0 2022-06-17T12:47:22.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f742affd700 1 -- 172.21.15.36:0/1080581801 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f741400d7f0 con 0x7f742c1473d0 2022-06-17T12:47:22.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f7432248700 1 -- 172.21.15.36:0/1080581801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f742c1473d0 msgr2=0x7f742c05fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:22.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f7432248700 1 --2- 172.21.15.36:0/1080581801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f742c1473d0 0x7f742c05fe80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f7414002f70 tx=0x7f7414006df0).stop 2022-06-17T12:47:22.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f7432248700 1 -- 172.21.15.36:0/1080581801 shutdown_connections 2022-06-17T12:47:22.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f7432248700 1 --2- 172.21.15.36:0/1080581801 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f742c060370 0x7f742c143200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:22.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f7432248700 1 --2- 172.21.15.36:0/1080581801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f742c1473d0 0x7f742c05fe80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:22.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f7432248700 1 -- 172.21.15.36:0/1080581801 >> 172.21.15.36:0/1080581801 conn(0x7f742c063020 msgr2=0x7f742c063430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:22.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f7432248700 1 -- 172.21.15.36:0/1080581801 shutdown_connections 2022-06-17T12:47:22.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.657+0000 7f7432248700 1 -- 172.21.15.36:0/1080581801 wait complete. 2022-06-17T12:47:22.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.661+0000 7f7432248700 1 Processor -- start 2022-06-17T12:47:22.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.661+0000 7f7432248700 1 -- start start 2022-06-17T12:47:22.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.661+0000 7f7432248700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f742c060370 0x7f742c143d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:22.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.661+0000 7f7432248700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f742c0b2df0 0x7f742c156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:22.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.661+0000 7f7432248700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f742c1473d0 0x7f742c152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:22.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.661+0000 7f7432248700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742c155720 con 0x7f742c1473d0 2022-06-17T12:47:22.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.661+0000 7f7432248700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742c155910 con 0x7f742c0b2df0 2022-06-17T12:47:22.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.661+0000 7f7432248700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742c155bb0 con 0x7f742c060370 2022-06-17T12:47:22.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.661+0000 7f742bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f742c1473d0 0x7f742c152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:22.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.665+0000 7f7423fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f742c0b2df0 0x7f742c156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:22.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.665+0000 7f742b7fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f742c060370 0x7f742c143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:22.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.665+0000 7f742bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f742c1473d0 0x7f742c152180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60306/0 (socket says 172.21.15.36:60306) 2022-06-17T12:47:22.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.665+0000 7f7423fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f742c0b2df0 0x7f742c156050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:52454/0 (socket says 172.21.15.36:52454) 2022-06-17T12:47:22.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.665+0000 7f742bfff700 1 -- 172.21.15.36:0/2367056079 learned_addr learned my addr 172.21.15.36:0/2367056079 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:22.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.665+0000 7f7423fff700 1 -- 172.21.15.36:0/2367056079 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f742c060370 msgr2=0x7f742c143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:22.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.665+0000 7f7423fff700 1 --2- 172.21.15.36:0/2367056079 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f742c060370 0x7f742c143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:22.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.665+0000 7f7423fff700 1 -- 172.21.15.36:0/2367056079 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f742c1473d0 msgr2=0x7f742c152180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:22.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.665+0000 7f7423fff700 1 --2- 172.21.15.36:0/2367056079 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f742c1473d0 0x7f742c152180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:22.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.665+0000 7f7423fff700 1 -- 172.21.15.36:0/2367056079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7424009470 con 0x7f742c0b2df0 2022-06-17T12:47:22.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.665+0000 7f742bfff700 1 --2- 172.21.15.36:0/2367056079 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f742c1473d0 0x7f742c152180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:22.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.665+0000 7f7423fff700 1 --2- 172.21.15.36:0/2367056079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f742c0b2df0 0x7f742c156050 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f7414002f70 tx=0x7f7414014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:22.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.665+0000 7f74297fa700 1 -- 172.21.15.36:0/2367056079 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7414012930 con 0x7f742c0b2df0 2022-06-17T12:47:22.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.665+0000 7f74297fa700 1 -- 172.21.15.36:0/2367056079 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7414012a90 con 0x7f742c0b2df0 2022-06-17T12:47:22.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.665+0000 7f7432248700 1 -- 172.21.15.36:0/2367056079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f741400b180 con 0x7f742c0b2df0 2022-06-17T12:47:22.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.665+0000 7f74297fa700 1 -- 172.21.15.36:0/2367056079 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7414012df0 con 0x7f742c0b2df0 2022-06-17T12:47:22.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.665+0000 7f7432248700 1 -- 172.21.15.36:0/2367056079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f742c156540 con 0x7f742c0b2df0 2022-06-17T12:47:22.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.665+0000 7f7432248700 1 -- 172.21.15.36:0/2367056079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f742c146f30 con 0x7f742c0b2df0 2022-06-17T12:47:22.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.669+0000 7f74297fa700 1 -- 172.21.15.36:0/2367056079 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f741401c070 con 0x7f742c0b2df0 2022-06-17T12:47:22.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.669+0000 7f74297fa700 1 --2- 172.21.15.36:0/2367056079 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f74100539d0 0x7f7410055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:22.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.669+0000 7f742b7fe700 1 --2- 172.21.15.36:0/2367056079 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f74100539d0 0x7f7410055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:22.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.669+0000 7f74297fa700 1 -- 172.21.15.36:0/2367056079 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f741406cb90 con 0x7f742c0b2df0 2022-06-17T12:47:22.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.669+0000 7f742b7fe700 1 --2- 172.21.15.36:0/2367056079 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f74100539d0 0x7f7410055e90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f741c00d000 tx=0x7f741c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:22.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.677+0000 7f74297fa700 1 -- 172.21.15.36:0/2367056079 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f74140438b0 con 0x7f742c0b2df0 2022-06-17T12:47:22.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.805+0000 7f7432248700 1 -- 172.21.15.36:0/2367056079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f742c057410 con 0x7f742c0b2df0 2022-06-17T12:47:22.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.809+0000 7f74297fa700 1 -- 172.21.15.36:0/2367056079 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7414019070 con 0x7f742c0b2df0 2022-06-17T12:47:22.796 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:22.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.809+0000 7f7432248700 1 -- 172.21.15.36:0/2367056079 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f74100539d0 msgr2=0x7f7410055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:22.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.809+0000 7f7432248700 1 --2- 172.21.15.36:0/2367056079 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f74100539d0 0x7f7410055e90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f741c00d000 tx=0x7f741c009040).stop 2022-06-17T12:47:22.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.809+0000 7f7432248700 1 -- 172.21.15.36:0/2367056079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f742c0b2df0 msgr2=0x7f742c156050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:22.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.809+0000 7f7432248700 1 --2- 172.21.15.36:0/2367056079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f742c0b2df0 0x7f742c156050 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f7414002f70 tx=0x7f7414014000).stop 2022-06-17T12:47:22.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.809+0000 7f7432248700 1 -- 172.21.15.36:0/2367056079 shutdown_connections 2022-06-17T12:47:22.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.809+0000 7f7432248700 1 --2- 172.21.15.36:0/2367056079 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f74100539d0 0x7f7410055e90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:22.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.809+0000 7f7432248700 1 --2- 172.21.15.36:0/2367056079 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f742c1473d0 0x7f742c152180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:22.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.809+0000 7f7432248700 1 --2- 172.21.15.36:0/2367056079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f742c0b2df0 0x7f742c156050 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:22.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.809+0000 7f7432248700 1 --2- 172.21.15.36:0/2367056079 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f742c060370 0x7f742c143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:22.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.809+0000 7f7432248700 1 -- 172.21.15.36:0/2367056079 >> 172.21.15.36:0/2367056079 conn(0x7f742c063020 msgr2=0x7f742c147840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:22.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.809+0000 7f7432248700 1 -- 172.21.15.36:0/2367056079 shutdown_connections 2022-06-17T12:47:22.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.809+0000 7f7432248700 1 -- 172.21.15.36:0/2367056079 wait complete. 2022-06-17T12:47:22.845 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:47:22.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.965+0000 7f67a8dd5700 1 Processor -- start 2022-06-17T12:47:22.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.965+0000 7f67a8dd5700 1 -- start start 2022-06-17T12:47:22.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a8dd5700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f67a40b1280 0x7f67a413fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:22.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a8dd5700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67a40abe30 0x7f67a4062ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:22.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a8dd5700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f67a4140410 0x7f67a41436d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:22.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a8dd5700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a40aef10 con 0x7f67a40b1280 2022-06-17T12:47:22.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a8dd5700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a4140080 con 0x7f67a4140410 2022-06-17T12:47:22.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a8dd5700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a40b1660 con 0x7f67a40abe30 2022-06-17T12:47:22.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a37fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f67a40b1280 0x7f67a413fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:22.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a37fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f67a40b1280 0x7f67a413fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60314/0 (socket says 172.21.15.36:60314) 2022-06-17T12:47:22.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a37fe700 1 -- 172.21.15.36:0/167579517 learned_addr learned my addr 172.21.15.36:0/167579517 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:22.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a3fff700 1 --2- 172.21.15.36:0/167579517 >> v2:172.21.15.70:3300/0 conn(0x7f67a4140410 0x7f67a41436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:22.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a2ffd700 1 --2- 172.21.15.36:0/167579517 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67a40abe30 0x7f67a4062ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:22.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a37fe700 1 -- 172.21.15.36:0/167579517 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67a40abe30 msgr2=0x7f67a4062ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:22.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a37fe700 1 --2- 172.21.15.36:0/167579517 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67a40abe30 0x7f67a4062ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:22.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a37fe700 1 -- 172.21.15.36:0/167579517 >> v2:172.21.15.70:3300/0 conn(0x7f67a4140410 msgr2=0x7f67a41436d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:22.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a37fe700 1 --2- 172.21.15.36:0/167579517 >> v2:172.21.15.70:3300/0 conn(0x7f67a4140410 0x7f67a41436d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:22.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a37fe700 1 -- 172.21.15.36:0/167579517 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a40ac3b0 con 0x7f67a40b1280 2022-06-17T12:47:22.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a2ffd700 1 --2- 172.21.15.36:0/167579517 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67a40abe30 0x7f67a4062ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:22.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a37fe700 1 --2- 172.21.15.36:0/167579517 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67a40b1280 0x7f67a413fb90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f67a4062510 tx=0x7f6794006eb0).ready entity=mon.0 client_cookie=4adc8711fcde9497 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:22.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a27fc700 1 -- 172.21.15.36:0/167579517 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f679400d570 con 0x7f67a40b1280 2022-06-17T12:47:22.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a27fc700 1 -- 172.21.15.36:0/167579517 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6794000e90 con 0x7f67a40b1280 2022-06-17T12:47:22.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a27fc700 1 -- 172.21.15.36:0/167579517 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f679400d900 con 0x7f67a40b1280 2022-06-17T12:47:22.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a8dd5700 1 -- 172.21.15.36:0/167579517 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67a40b1280 msgr2=0x7f67a413fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:23.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a8dd5700 1 --2- 172.21.15.36:0/167579517 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67a40b1280 0x7f67a413fb90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f67a4062510 tx=0x7f6794006eb0).stop 2022-06-17T12:47:23.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a8dd5700 1 -- 172.21.15.36:0/167579517 shutdown_connections 2022-06-17T12:47:23.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a8dd5700 1 --2- 172.21.15.36:0/167579517 >> v2:172.21.15.70:3300/0 conn(0x7f67a4140410 0x7f67a41436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a8dd5700 1 --2- 172.21.15.36:0/167579517 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67a40abe30 0x7f67a4062ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a8dd5700 1 --2- 172.21.15.36:0/167579517 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67a40b1280 0x7f67a413fb90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a8dd5700 1 -- 172.21.15.36:0/167579517 >> 172.21.15.36:0/167579517 conn(0x7f67a405f770 msgr2=0x7f67a405fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:23.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a8dd5700 1 -- 172.21.15.36:0/167579517 shutdown_connections 2022-06-17T12:47:23.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.969+0000 7f67a8dd5700 1 -- 172.21.15.36:0/167579517 wait complete. 2022-06-17T12:47:23.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a8dd5700 1 Processor -- start 2022-06-17T12:47:23.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a8dd5700 1 -- start start 2022-06-17T12:47:23.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a8dd5700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67a40abe30 0x7f67a413e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:23.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a8dd5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67a40b1280 0x7f67a40b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:23.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a8dd5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67a4140410 0x7f67a41561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:23.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a8dd5700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a4155d60 con 0x7f67a4140410 2022-06-17T12:47:23.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a8dd5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a40b3ee0 con 0x7f67a40b1280 2022-06-17T12:47:23.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a8dd5700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a40b4180 con 0x7f67a40abe30 2022-06-17T12:47:23.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a3fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67a4140410 0x7f67a41561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:23.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a3fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67a4140410 0x7f67a41561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60320/0 (socket says 172.21.15.36:60320) 2022-06-17T12:47:23.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a3fff700 1 -- 172.21.15.36:0/3011719363 learned_addr learned my addr 172.21.15.36:0/3011719363 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:23.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a37fe700 1 --2- 172.21.15.36:0/3011719363 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67a40abe30 0x7f67a413e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:23.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a2ffd700 1 --2- 172.21.15.36:0/3011719363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67a40b1280 0x7f67a40b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:23.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a3fff700 1 -- 172.21.15.36:0/3011719363 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67a40abe30 msgr2=0x7f67a413e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:23.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a3fff700 1 --2- 172.21.15.36:0/3011719363 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67a40abe30 0x7f67a413e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a3fff700 1 -- 172.21.15.36:0/3011719363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67a40b1280 msgr2=0x7f67a40b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:23.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a3fff700 1 --2- 172.21.15.36:0/3011719363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67a40b1280 0x7f67a40b5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a3fff700 1 -- 172.21.15.36:0/3011719363 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679400b1e0 con 0x7f67a4140410 2022-06-17T12:47:23.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a37fe700 1 --2- 172.21.15.36:0/3011719363 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67a40abe30 0x7f67a413e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:47:23.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a3fff700 1 --2- 172.21.15.36:0/3011719363 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67a4140410 0x7f67a41561c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f679c006e00 tx=0x7f679c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:23.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a0ff9700 1 -- 172.21.15.36:0/3011719363 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f679c00c7f0 con 0x7f67a4140410 2022-06-17T12:47:23.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a0ff9700 1 -- 172.21.15.36:0/3011719363 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679c00c950 con 0x7f67a4140410 2022-06-17T12:47:23.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a8dd5700 1 -- 172.21.15.36:0/3011719363 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a40b3b60 con 0x7f67a4140410 2022-06-17T12:47:23.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a0ff9700 1 -- 172.21.15.36:0/3011719363 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f679c00ccb0 con 0x7f67a4140410 2022-06-17T12:47:23.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.973+0000 7f67a8dd5700 1 -- 172.21.15.36:0/3011719363 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a413c180 con 0x7f67a4140410 2022-06-17T12:47:23.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.977+0000 7f67a8dd5700 1 -- 172.21.15.36:0/3011719363 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a4155b70 con 0x7f67a4140410 2022-06-17T12:47:23.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.977+0000 7f67a0ff9700 1 -- 172.21.15.36:0/3011719363 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f679c022070 con 0x7f67a4140410 2022-06-17T12:47:23.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.977+0000 7f67a0ff9700 1 --2- 172.21.15.36:0/3011719363 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6788053ad0 0x7f6788055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:23.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.977+0000 7f67a0ff9700 1 -- 172.21.15.36:0/3011719363 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f679c06c6b0 con 0x7f67a4140410 2022-06-17T12:47:23.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.977+0000 7f67a37fe700 1 --2- 172.21.15.36:0/3011719363 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6788053ad0 0x7f6788055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:23.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.977+0000 7f67a37fe700 1 --2- 172.21.15.36:0/3011719363 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6788053ad0 0x7f6788055f90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f6794016000 tx=0x7f6794015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:23.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:22.981+0000 7f67a0ff9700 1 -- 172.21.15.36:0/3011719363 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f679c043410 con 0x7f67a4140410 2022-06-17T12:47:23.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.109+0000 7f67a8dd5700 1 -- 172.21.15.36:0/3011719363 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67a40ac650 con 0x7f67a4140410 2022-06-17T12:47:23.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.113+0000 7f67a0ff9700 1 -- 172.21.15.36:0/3011719363 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f679c016090 con 0x7f67a4140410 2022-06-17T12:47:23.100 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:23.101 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":26,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:46:53.258075+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:46:47.465328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-17T12:46:49.893338+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:47:23.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.113+0000 7f67a8dd5700 1 -- 172.21.15.36:0/3011719363 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6788053ad0 msgr2=0x7f6788055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:23.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.113+0000 7f67a8dd5700 1 --2- 172.21.15.36:0/3011719363 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6788053ad0 0x7f6788055f90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f6794016000 tx=0x7f6794015000).stop 2022-06-17T12:47:23.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.113+0000 7f67a8dd5700 1 -- 172.21.15.36:0/3011719363 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67a4140410 msgr2=0x7f67a41561c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:23.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.113+0000 7f67a8dd5700 1 --2- 172.21.15.36:0/3011719363 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67a4140410 0x7f67a41561c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f679c006e00 tx=0x7f679c0091d0).stop 2022-06-17T12:47:23.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.113+0000 7f67a8dd5700 1 -- 172.21.15.36:0/3011719363 shutdown_connections 2022-06-17T12:47:23.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.113+0000 7f67a8dd5700 1 --2- 172.21.15.36:0/3011719363 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6788053ad0 0x7f6788055f90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.113+0000 7f67a8dd5700 1 --2- 172.21.15.36:0/3011719363 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67a4140410 0x7f67a41561c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.113+0000 7f67a8dd5700 1 --2- 172.21.15.36:0/3011719363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67a40b1280 0x7f67a40b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.113+0000 7f67a8dd5700 1 --2- 172.21.15.36:0/3011719363 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67a40abe30 0x7f67a413e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.113+0000 7f67a8dd5700 1 -- 172.21.15.36:0/3011719363 >> 172.21.15.36:0/3011719363 conn(0x7f67a405f770 msgr2=0x7f67a4130f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:23.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.113+0000 7f67a8dd5700 1 -- 172.21.15.36:0/3011719363 shutdown_connections 2022-06-17T12:47:23.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.113+0000 7f67a8dd5700 1 -- 172.21.15.36:0/3011719363 wait complete. 2022-06-17T12:47:23.131 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:46:47.465328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:46:49.893338+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:47:23.132 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:47:23.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f730ecbb700 1 Processor -- start 2022-06-17T12:47:23.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f730ecbb700 1 -- start start 2022-06-17T12:47:23.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f730ecbb700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f73080b2ee0 0x7f73080b1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:23.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f730ecbb700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f730813eb10 0x7f730813ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:23.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f730ecbb700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73080ad4c0 con 0x7f730805f770 2022-06-17T12:47:23.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f730ecbb700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73080b32c0 con 0x7f73080b2ee0 2022-06-17T12:47:23.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f730ecbb700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73080ad720 con 0x7f730813eb10 2022-06-17T12:47:23.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f730d258700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f730805f770 0x7f730805fbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49624/0 (socket says 172.21.15.36:49624) 2022-06-17T12:47:23.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f730d258700 1 -- 172.21.15.36:0/3584942360 learned_addr learned my addr 172.21.15.36:0/3584942360 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:23.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f730ca57700 1 --2- 172.21.15.36:0/3584942360 >> v2:172.21.15.70:3300/0 conn(0x7f73080b2ee0 0x7f73080b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:23.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f7307fff700 1 --2- 172.21.15.36:0/3584942360 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f730813eb10 0x7f730813ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:23.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f73077fe700 1 -- 172.21.15.36:0/3584942360 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2895744199 0 0) 0x7f73080ad4c0 con 0x7f730805f770 2022-06-17T12:47:23.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f73077fe700 1 -- 172.21.15.36:0/3584942360 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72ec003260 con 0x7f730805f770 2022-06-17T12:47:23.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f73077fe700 1 -- 172.21.15.36:0/3584942360 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 202946134 0 0) 0x7f72ec003260 con 0x7f730805f770 2022-06-17T12:47:23.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f73077fe700 1 -- 172.21.15.36:0/3584942360 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f730813eb10 msgr2=0x7f730813ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:23.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f73077fe700 1 --2- 172.21.15.36:0/3584942360 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f730813eb10 0x7f730813ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f73077fe700 1 -- 172.21.15.36:0/3584942360 >> v2:172.21.15.70:3300/0 conn(0x7f73080b2ee0 msgr2=0x7f73080b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:23.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f73077fe700 1 --2- 172.21.15.36:0/3584942360 >> v2:172.21.15.70:3300/0 conn(0x7f73080b2ee0 0x7f73080b1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f73077fe700 1 -- 172.21.15.36:0/3584942360 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73080ad9c0 con 0x7f730805f770 2022-06-17T12:47:23.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f73077fe700 1 -- 172.21.15.36:0/3584942360 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f73000019e0 con 0x7f730805f770 2022-06-17T12:47:23.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f730ca57700 1 --2- 172.21.15.36:0/3584942360 >> v2:172.21.15.70:3300/0 conn(0x7f73080b2ee0 0x7f73080b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:23.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f73077fe700 1 -- 172.21.15.36:0/3584942360 >> v1:172.21.15.36:6789/0 conn(0x7f730805f770 legacy=0x7f730805fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:23.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f73077fe700 1 --2- 172.21.15.36:0/3584942360 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f72ec0037d0 0x7f72ec003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:23.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f73077fe700 1 --2- 172.21.15.36:0/3584942360 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f730813eb10 0x7f72ec004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:23.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f73077fe700 1 --2- 172.21.15.36:0/3584942360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72ec0049f0 0x7f72ec004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:23.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f73077fe700 1 -- 172.21.15.36:0/3584942360 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73080b32c0 con 0x7f72ec0037d0 2022-06-17T12:47:23.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f73077fe700 1 -- 172.21.15.36:0/3584942360 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73080ad720 con 0x7f72ec0049f0 2022-06-17T12:47:23.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f73077fe700 1 -- 172.21.15.36:0/3584942360 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec00b750 con 0x7f730813eb10 2022-06-17T12:47:23.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f730ca57700 1 --2- 172.21.15.36:0/3584942360 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f72ec0037d0 0x7f72ec003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:23.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f730d258700 1 --2- 172.21.15.36:0/3584942360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72ec0049f0 0x7f72ec004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:23.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f7307fff700 1 --2- 172.21.15.36:0/3584942360 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f730813eb10 0x7f72ec004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:23.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f730ca57700 1 -- 172.21.15.36:0/3584942360 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f730813eb10 msgr2=0x7f72ec004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:23.366 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f730ca57700 1 --2- 172.21.15.36:0/3584942360 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f730813eb10 0x7f72ec004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f730ca57700 1 -- 172.21.15.36:0/3584942360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72ec0049f0 msgr2=0x7f72ec004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:23.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f730ca57700 1 --2- 172.21.15.36:0/3584942360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72ec0049f0 0x7f72ec004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f730ca57700 1 -- 172.21.15.36:0/3584942360 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ec003260 con 0x7f72ec0037d0 2022-06-17T12:47:23.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.265+0000 7f730d258700 1 --2- 172.21.15.36:0/3584942360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72ec0049f0 0x7f72ec004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:23.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730ca57700 1 --2- 172.21.15.36:0/3584942360 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f72ec0037d0 0x7f72ec003bf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f72fc00c000 tx=0x7f72fc006df0).ready entity=mon.0 client_cookie=bd2f888d02a06f01 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:23.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f73077fe700 1 -- 172.21.15.36:0/3584942360 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72fc0098b0 con 0x7f72ec0037d0 2022-06-17T12:47:23.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f73077fe700 1 -- 172.21.15.36:0/3584942360 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72fc009a10 con 0x7f72ec0037d0 2022-06-17T12:47:23.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f73077fe700 1 -- 172.21.15.36:0/3584942360 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72fc009d70 con 0x7f72ec0037d0 2022-06-17T12:47:23.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730ecbb700 1 -- 172.21.15.36:0/3584942360 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f72ec0037d0 msgr2=0x7f72ec003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:23.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730ecbb700 1 --2- 172.21.15.36:0/3584942360 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f72ec0037d0 0x7f72ec003bf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f72fc00c000 tx=0x7f72fc006df0).stop 2022-06-17T12:47:23.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730d258700 1 -- 172.21.15.36:0/3584942360 reap_dead start 2022-06-17T12:47:23.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730ecbb700 1 -- 172.21.15.36:0/3584942360 shutdown_connections 2022-06-17T12:47:23.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730ecbb700 1 -- 172.21.15.36:0/3584942360 >> 172.21.15.36:0/3584942360 conn(0x7f7308063020 msgr2=0x7f7308063430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:23.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730ecbb700 1 -- 172.21.15.36:0/3584942360 shutdown_connections 2022-06-17T12:47:23.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730ecbb700 1 -- 172.21.15.36:0/3584942360 wait complete. 2022-06-17T12:47:23.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730ecbb700 1 Processor -- start 2022-06-17T12:47:23.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730ecbb700 1 -- start start 2022-06-17T12:47:23.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730ecbb700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f730813e180 0x7f7308145940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:23.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730ecbb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f730813da20 0x7f7308136b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:23.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730ecbb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7308146910 0x7f7308137010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:23.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730ecbb700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7308145e30 con 0x7f7308146910 2022-06-17T12:47:23.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730ecbb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730813e560 con 0x7f730813da20 2022-06-17T12:47:23.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730ecbb700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730813ce10 con 0x7f730813e180 2022-06-17T12:47:23.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730d258700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7308146910 0x7f7308137010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:23.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730ca57700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f730813e180 0x7f7308145940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:23.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f7307fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f730813da20 0x7f7308136b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:23.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730d258700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7308146910 0x7f7308137010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60340/0 (socket says 172.21.15.36:60340) 2022-06-17T12:47:23.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730d258700 1 -- 172.21.15.36:0/1908549565 learned_addr learned my addr 172.21.15.36:0/1908549565 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:23.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730d258700 1 -- 172.21.15.36:0/1908549565 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f730813e180 msgr2=0x7f7308145940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:23.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730d258700 1 --2- 172.21.15.36:0/1908549565 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f730813e180 0x7f7308145940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730d258700 1 -- 172.21.15.36:0/1908549565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f730813da20 msgr2=0x7f7308136b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:23.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730d258700 1 --2- 172.21.15.36:0/1908549565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f730813da20 0x7f7308136b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730d258700 1 -- 172.21.15.36:0/1908549565 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7308137ee0 con 0x7f7308146910 2022-06-17T12:47:23.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.269+0000 7f730d258700 1 --2- 172.21.15.36:0/1908549565 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7308146910 0x7f7308137010 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f7300000b60 tx=0x7f7300002100).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:23.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.273+0000 7f7305ffb700 1 -- 172.21.15.36:0/1908549565 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f730000fb60 con 0x7f7308146910 2022-06-17T12:47:23.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.273+0000 7f7305ffb700 1 -- 172.21.15.36:0/1908549565 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730000fcc0 con 0x7f7308146910 2022-06-17T12:47:23.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.273+0000 7f7305ffb700 1 -- 172.21.15.36:0/1908549565 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7300007e30 con 0x7f7308146910 2022-06-17T12:47:23.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.273+0000 7f730ecbb700 1 -- 172.21.15.36:0/1908549565 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73080a9520 con 0x7f7308146910 2022-06-17T12:47:23.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.273+0000 7f730ecbb700 1 -- 172.21.15.36:0/1908549565 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73081376a0 con 0x7f7308146910 2022-06-17T12:47:23.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.273+0000 7f730ecbb700 1 -- 172.21.15.36:0/1908549565 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7308137d20 con 0x7f7308146910 2022-06-17T12:47:23.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.273+0000 7f7305ffb700 1 -- 172.21.15.36:0/1908549565 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f730001b030 con 0x7f7308146910 2022-06-17T12:47:23.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.273+0000 7f7305ffb700 1 --2- 172.21.15.36:0/1908549565 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f72e80539b0 0x7f72e8055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:23.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.273+0000 7f7305ffb700 1 -- 172.21.15.36:0/1908549565 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f730006bbb0 con 0x7f7308146910 2022-06-17T12:47:23.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.273+0000 7f730ca57700 1 --2- 172.21.15.36:0/1908549565 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f72e80539b0 0x7f72e8055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:23.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.277+0000 7f730ca57700 1 --2- 172.21.15.36:0/1908549565 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f72e80539b0 0x7f72e8055e70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f72fc01b000 tx=0x7f72fc006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:23.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.281+0000 7f7305ffb700 1 -- 172.21.15.36:0/1908549565 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7300072080 con 0x7f7308146910 2022-06-17T12:47:23.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.409+0000 7f730ecbb700 1 -- 172.21.15.36:0/1908549565 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f73080b2120 con 0x7f7308146910 2022-06-17T12:47:23.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.409+0000 7f7305ffb700 1 -- 172.21.15.36:0/1908549565 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7300019020 con 0x7f7308146910 2022-06-17T12:47:23.451 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:47:23.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.413+0000 7f730ecbb700 1 -- 172.21.15.36:0/1908549565 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f72e80539b0 msgr2=0x7f72e8055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:23.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.413+0000 7f730ecbb700 1 --2- 172.21.15.36:0/1908549565 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f72e80539b0 0x7f72e8055e70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f72fc01b000 tx=0x7f72fc006df0).stop 2022-06-17T12:47:23.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.413+0000 7f730ecbb700 1 -- 172.21.15.36:0/1908549565 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7308146910 msgr2=0x7f7308137010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:23.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.413+0000 7f730ecbb700 1 --2- 172.21.15.36:0/1908549565 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7308146910 0x7f7308137010 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f7300000b60 tx=0x7f7300002100).stop 2022-06-17T12:47:23.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.413+0000 7f730ecbb700 1 -- 172.21.15.36:0/1908549565 shutdown_connections 2022-06-17T12:47:23.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.413+0000 7f730ecbb700 1 --2- 172.21.15.36:0/1908549565 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f72e80539b0 0x7f72e8055e70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.413+0000 7f730ecbb700 1 --2- 172.21.15.36:0/1908549565 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7308146910 0x7f7308137010 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.413+0000 7f730ecbb700 1 --2- 172.21.15.36:0/1908549565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f730813da20 0x7f7308136b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.413+0000 7f730ecbb700 1 --2- 172.21.15.36:0/1908549565 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f730813e180 0x7f7308145940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.413+0000 7f730ecbb700 1 -- 172.21.15.36:0/1908549565 >> 172.21.15.36:0/1908549565 conn(0x7f7308063020 msgr2=0x7f7308060770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:23.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.413+0000 7f730ecbb700 1 -- 172.21.15.36:0/1908549565 shutdown_connections 2022-06-17T12:47:23.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.413+0000 7f730ecbb700 1 -- 172.21.15.36:0/1908549565 wait complete. 2022-06-17T12:47:23.492 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:47:23.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.657+0000 7f8b0e183700 1 Processor -- start 2022-06-17T12:47:23.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.657+0000 7f8b0e183700 1 -- start start 2022-06-17T12:47:23.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.657+0000 7f8b0e183700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8b080b27b0 0x7f8b0814b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:23.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.657+0000 7f8b0e183700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8b08060ca0 0x7f8b080ac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:23.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.657+0000 7f8b0e183700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b08147bd0 0x7f8b081442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:23.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.657+0000 7f8b0e183700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b0814b870 con 0x7f8b08060ca0 2022-06-17T12:47:23.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.657+0000 7f8b0e183700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b080ac500 con 0x7f8b080b27b0 2022-06-17T12:47:23.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.657+0000 7f8b0e183700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b080b2b90 con 0x7f8b08147bd0 2022-06-17T12:47:23.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.657+0000 7f8b0c980700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8b08060ca0 0x7f8b080ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:23.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.657+0000 7f8b0c980700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8b08060ca0 0x7f8b080ac780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60346/0 (socket says 172.21.15.36:60346) 2022-06-17T12:47:23.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.657+0000 7f8b0c980700 1 -- 172.21.15.36:0/3428663196 learned_addr learned my addr 172.21.15.36:0/3428663196 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:23.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.657+0000 7f8b0d181700 1 --2- 172.21.15.36:0/3428663196 >> v2:172.21.15.70:3300/0 conn(0x7f8b080b27b0 0x7f8b0814b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:23.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.657+0000 7f8b0d982700 1 --2- 172.21.15.36:0/3428663196 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b08147bd0 0x7f8b081442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:23.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.657+0000 7f8b0c980700 1 -- 172.21.15.36:0/3428663196 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b08147bd0 msgr2=0x7f8b081442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:23.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.657+0000 7f8b0c980700 1 --2- 172.21.15.36:0/3428663196 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b08147bd0 0x7f8b081442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.657+0000 7f8b0c980700 1 -- 172.21.15.36:0/3428663196 >> v2:172.21.15.70:3300/0 conn(0x7f8b080b27b0 msgr2=0x7f8b0814b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:23.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.657+0000 7f8b0c980700 1 --2- 172.21.15.36:0/3428663196 >> v2:172.21.15.70:3300/0 conn(0x7f8b080b27b0 0x7f8b0814b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.657+0000 7f8b0c980700 1 -- 172.21.15.36:0/3428663196 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b08147490 con 0x7f8b08060ca0 2022-06-17T12:47:23.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.657+0000 7f8b0d181700 1 --2- 172.21.15.36:0/3428663196 >> v2:172.21.15.70:3300/0 conn(0x7f8b080b27b0 0x7f8b0814b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:23.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.657+0000 7f8b0c980700 1 --2- 172.21.15.36:0/3428663196 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b08060ca0 0x7f8b080ac780 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f8af8002f70 tx=0x7f8af8006eb0).ready entity=mon.0 client_cookie=48766dfe0c861659 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:23.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.657+0000 7f8afffff700 1 -- 172.21.15.36:0/3428663196 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8af800d570 con 0x7f8b08060ca0 2022-06-17T12:47:23.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.657+0000 7f8afffff700 1 -- 172.21.15.36:0/3428663196 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af8000e90 con 0x7f8b08060ca0 2022-06-17T12:47:23.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.657+0000 7f8afffff700 1 -- 172.21.15.36:0/3428663196 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8af800d900 con 0x7f8b08060ca0 2022-06-17T12:47:23.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.661+0000 7f8b0e183700 1 -- 172.21.15.36:0/3428663196 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b08060ca0 msgr2=0x7f8b080ac780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:23.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.661+0000 7f8b0e183700 1 --2- 172.21.15.36:0/3428663196 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b08060ca0 0x7f8b080ac780 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f8af8002f70 tx=0x7f8af8006eb0).stop 2022-06-17T12:47:23.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.661+0000 7f8b0e183700 1 -- 172.21.15.36:0/3428663196 shutdown_connections 2022-06-17T12:47:23.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.661+0000 7f8b0e183700 1 --2- 172.21.15.36:0/3428663196 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b08147bd0 0x7f8b081442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.661+0000 7f8b0e183700 1 --2- 172.21.15.36:0/3428663196 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b08060ca0 0x7f8b080ac780 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.661+0000 7f8b0e183700 1 --2- 172.21.15.36:0/3428663196 >> v2:172.21.15.70:3300/0 conn(0x7f8b080b27b0 0x7f8b0814b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.661+0000 7f8b0e183700 1 -- 172.21.15.36:0/3428663196 >> 172.21.15.36:0/3428663196 conn(0x7f8b080b5d10 msgr2=0x7f8b080b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:23.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.661+0000 7f8b0e183700 1 -- 172.21.15.36:0/3428663196 shutdown_connections 2022-06-17T12:47:23.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.661+0000 7f8b0e183700 1 -- 172.21.15.36:0/3428663196 wait complete. 2022-06-17T12:47:23.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.661+0000 7f8b0e183700 1 Processor -- start 2022-06-17T12:47:23.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.661+0000 7f8b0e183700 1 -- start start 2022-06-17T12:47:23.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.661+0000 7f8b0e183700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b08060ca0 0x7f8b081598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:23.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.661+0000 7f8b0e183700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b080b27b0 0x7f8b0814dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:23.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.661+0000 7f8b0e183700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b08147bd0 0x7f8b08063e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:23.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.661+0000 7f8b0e183700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b08066aa0 con 0x7f8b080b27b0 2022-06-17T12:47:23.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.661+0000 7f8b0e183700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b08064c80 con 0x7f8b08147bd0 2022-06-17T12:47:23.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.661+0000 7f8b0e183700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b08064300 con 0x7f8b08060ca0 2022-06-17T12:47:23.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.661+0000 7f8b0c980700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b080b27b0 0x7f8b0814dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:23.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.661+0000 7f8b0c980700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b080b27b0 0x7f8b0814dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60352/0 (socket says 172.21.15.36:60352) 2022-06-17T12:47:23.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.661+0000 7f8b0c980700 1 -- 172.21.15.36:0/4057479386 learned_addr learned my addr 172.21.15.36:0/4057479386 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:23.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.665+0000 7f8b0d181700 1 --2- 172.21.15.36:0/4057479386 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b08060ca0 0x7f8b081598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:23.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.665+0000 7f8b0d982700 1 --2- 172.21.15.36:0/4057479386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b08147bd0 0x7f8b08063e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:23.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.665+0000 7f8b0c980700 1 -- 172.21.15.36:0/4057479386 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b08060ca0 msgr2=0x7f8b081598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:23.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.665+0000 7f8b0c980700 1 --2- 172.21.15.36:0/4057479386 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b08060ca0 0x7f8b081598e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.665+0000 7f8b0c980700 1 -- 172.21.15.36:0/4057479386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b08147bd0 msgr2=0x7f8b08063e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:23.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.665+0000 7f8b0c980700 1 --2- 172.21.15.36:0/4057479386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b08147bd0 0x7f8b08063e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.665+0000 7f8b0c980700 1 -- 172.21.15.36:0/4057479386 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af800b1e0 con 0x7f8b080b27b0 2022-06-17T12:47:23.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.665+0000 7f8b0c980700 1 --2- 172.21.15.36:0/4057479386 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b080b27b0 0x7f8b0814dfd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f8b08066900 tx=0x7f8af8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:23.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.665+0000 7f8afe7fc700 1 -- 172.21.15.36:0/4057479386 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8af80116e0 con 0x7f8b080b27b0 2022-06-17T12:47:23.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.665+0000 7f8afe7fc700 1 -- 172.21.15.36:0/4057479386 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af8011840 con 0x7f8b080b27b0 2022-06-17T12:47:23.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.665+0000 7f8afe7fc700 1 -- 172.21.15.36:0/4057479386 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8af8011ba0 con 0x7f8b080b27b0 2022-06-17T12:47:23.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.665+0000 7f8b0e183700 1 -- 172.21.15.36:0/4057479386 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b08064590 con 0x7f8b080b27b0 2022-06-17T12:47:23.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.665+0000 7f8b0e183700 1 -- 172.21.15.36:0/4057479386 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b080a8040 con 0x7f8b080b27b0 2022-06-17T12:47:23.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.665+0000 7f8b0e183700 1 -- 172.21.15.36:0/4057479386 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b08057410 con 0x7f8b080b27b0 2022-06-17T12:47:23.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.665+0000 7f8afe7fc700 1 -- 172.21.15.36:0/4057479386 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8af8025030 con 0x7f8b080b27b0 2022-06-17T12:47:23.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.665+0000 7f8afe7fc700 1 --2- 172.21.15.36:0/4057479386 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8af0053a40 0x7f8af0055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:23.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.665+0000 7f8afe7fc700 1 -- 172.21.15.36:0/4057479386 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f8af806d030 con 0x7f8b080b27b0 2022-06-17T12:47:23.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.669+0000 7f8b0d181700 1 --2- 172.21.15.36:0/4057479386 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8af0053a40 0x7f8af0055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:23.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.669+0000 7f8b0d181700 1 --2- 172.21.15.36:0/4057479386 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8af0053a40 0x7f8af0055f00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f8b0400c000 tx=0x7f8b04008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:23.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.673+0000 7f8afe7fc700 1 -- 172.21.15.36:0/4057479386 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8af8043e30 con 0x7f8b080b27b0 2022-06-17T12:47:23.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.805+0000 7f8b0e183700 1 -- 172.21.15.36:0/4057479386 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8b0805d9b0 con 0x7f8b080b27b0 2022-06-17T12:47:23.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.805+0000 7f8afe7fc700 1 -- 172.21.15.36:0/4057479386 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8af8023020 con 0x7f8b080b27b0 2022-06-17T12:47:23.807 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:23.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.809+0000 7f8b0e183700 1 -- 172.21.15.36:0/4057479386 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8af0053a40 msgr2=0x7f8af0055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:23.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.809+0000 7f8b0e183700 1 --2- 172.21.15.36:0/4057479386 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8af0053a40 0x7f8af0055f00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f8b0400c000 tx=0x7f8b04008040).stop 2022-06-17T12:47:23.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.809+0000 7f8b0e183700 1 -- 172.21.15.36:0/4057479386 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b080b27b0 msgr2=0x7f8b0814dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:23.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.809+0000 7f8b0e183700 1 --2- 172.21.15.36:0/4057479386 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b080b27b0 0x7f8b0814dfd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f8b08066900 tx=0x7f8af8015000).stop 2022-06-17T12:47:23.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.809+0000 7f8b0e183700 1 -- 172.21.15.36:0/4057479386 shutdown_connections 2022-06-17T12:47:23.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.809+0000 7f8b0e183700 1 --2- 172.21.15.36:0/4057479386 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8af0053a40 0x7f8af0055f00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.809+0000 7f8b0e183700 1 --2- 172.21.15.36:0/4057479386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b08147bd0 0x7f8b08063e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.809+0000 7f8b0e183700 1 --2- 172.21.15.36:0/4057479386 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b080b27b0 0x7f8b0814dfd0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.809+0000 7f8b0e183700 1 --2- 172.21.15.36:0/4057479386 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b08060ca0 0x7f8b081598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:23.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.809+0000 7f8b0e183700 1 -- 172.21.15.36:0/4057479386 >> 172.21.15.36:0/4057479386 conn(0x7f8b080b5d10 msgr2=0x7f8b080b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:23.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.809+0000 7f8b0e183700 1 -- 172.21.15.36:0/4057479386 shutdown_connections 2022-06-17T12:47:23.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.809+0000 7f8b0e183700 1 -- 172.21.15.36:0/4057479386 wait complete. 2022-06-17T12:47:23.834 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:47:24.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.977+0000 7f19a2262700 1 Processor -- start 2022-06-17T12:47:24.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.977+0000 7f19a2262700 1 -- start start 2022-06-17T12:47:24.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f19a2262700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f199c0acee0 0x7f199c0b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:24.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f19a2262700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f199c0b1900 con 0x7f199c144240 2022-06-17T12:47:24.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f19a2262700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f199c0af370 con 0x7f199c0acb00 2022-06-17T12:47:24.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f19a2262700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199c0ad360 con 0x7f199c0acee0 2022-06-17T12:47:24.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199bfff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f199c144240 0x7f199c1446d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49660/0 (socket says 172.21.15.36:49660) 2022-06-17T12:47:24.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199bfff700 1 -- 172.21.15.36:0/2732554868 learned_addr learned my addr 172.21.15.36:0/2732554868 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:24.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199affd700 1 --2- 172.21.15.36:0/2732554868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f199c0acee0 0x7f199c0b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:24.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199a7fc700 1 -- 172.21.15.36:0/2732554868 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2692888126 0 0) 0x7f199c0b1900 con 0x7f199c144240 2022-06-17T12:47:24.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199a7fc700 1 -- 172.21.15.36:0/2732554868 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1988003260 con 0x7f199c144240 2022-06-17T12:47:24.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199a7fc700 1 -- 172.21.15.36:0/2732554868 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2114250507 0 0) 0x7f1988003260 con 0x7f199c144240 2022-06-17T12:47:24.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199a7fc700 1 -- 172.21.15.36:0/2732554868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f199c0acee0 msgr2=0x7f199c0b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:24.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199a7fc700 1 --2- 172.21.15.36:0/2732554868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f199c0acee0 0x7f199c0b05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:24.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199a7fc700 1 -- 172.21.15.36:0/2732554868 >> v1:172.21.15.70:6789/0 conn(0x7f199c0acb00 legacy=0x7f199c147480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:24.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199a7fc700 1 -- 172.21.15.36:0/2732554868 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199c0b1b30 con 0x7f199c144240 2022-06-17T12:47:24.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199a7fc700 1 -- 172.21.15.36:0/2732554868 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f19940019e0 con 0x7f199c144240 2022-06-17T12:47:24.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199a7fc700 1 -- 172.21.15.36:0/2732554868 >> v1:172.21.15.36:6789/0 conn(0x7f199c144240 legacy=0x7f199c1446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:24.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199a7fc700 1 --2- 172.21.15.36:0/2732554868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1988003780 0x7f1988003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:24.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199a7fc700 1 --2- 172.21.15.36:0/2732554868 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1988004500 0x7f1988004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:24.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199a7fc700 1 --2- 172.21.15.36:0/2732554868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1988004e70 0x7f198800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:24.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199a7fc700 1 -- 172.21.15.36:0/2732554868 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198800bc70 con 0x7f1988004500 2022-06-17T12:47:24.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199a7fc700 1 -- 172.21.15.36:0/2732554868 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198800bde0 con 0x7f1988004e70 2022-06-17T12:47:24.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199a7fc700 1 -- 172.21.15.36:0/2732554868 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198800bfa0 con 0x7f1988003780 2022-06-17T12:47:24.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199affd700 1 --2- 172.21.15.36:0/2732554868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f199c0acee0 0x7f199c0b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:24.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199affd700 1 --2- 172.21.15.36:0/2732554868 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1988004500 0x7f1988004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:24.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199bfff700 1 --2- 172.21.15.36:0/2732554868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1988004e70 0x7f198800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:24.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199b7fe700 1 --2- 172.21.15.36:0/2732554868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1988003780 0x7f1988003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:24.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199affd700 1 -- 172.21.15.36:0/2732554868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1988003780 msgr2=0x7f1988003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:24.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199affd700 1 --2- 172.21.15.36:0/2732554868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1988003780 0x7f1988003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:24.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199affd700 1 -- 172.21.15.36:0/2732554868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1988004e70 msgr2=0x7f198800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:24.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199affd700 1 --2- 172.21.15.36:0/2732554868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1988004e70 0x7f198800b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:24.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199affd700 1 -- 172.21.15.36:0/2732554868 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1988003260 con 0x7f1988004500 2022-06-17T12:47:24.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199affd700 1 --2- 172.21.15.36:0/2732554868 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1988004500 0x7f1988004980 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f198c00c000 tx=0x7f198c006df0).ready entity=mon.0 client_cookie=a1661d33e8787e21 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:24.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199a7fc700 1 -- 172.21.15.36:0/2732554868 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f198c00e650 con 0x7f1988004500 2022-06-17T12:47:24.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199a7fc700 1 -- 172.21.15.36:0/2732554868 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198c00e7b0 con 0x7f1988004500 2022-06-17T12:47:24.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199a7fc700 1 -- 172.21.15.36:0/2732554868 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f198c00eb10 con 0x7f1988004500 2022-06-17T12:47:24.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f19a2262700 1 -- 172.21.15.36:0/2732554868 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1988004500 msgr2=0x7f1988004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:24.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f19a2262700 1 --2- 172.21.15.36:0/2732554868 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1988004500 0x7f1988004980 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f198c00c000 tx=0x7f198c006df0).stop 2022-06-17T12:47:24.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f199bfff700 1 -- 172.21.15.36:0/2732554868 reap_dead start 2022-06-17T12:47:24.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f19a2262700 1 -- 172.21.15.36:0/2732554868 shutdown_connections 2022-06-17T12:47:24.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f19a2262700 1 -- 172.21.15.36:0/2732554868 >> 172.21.15.36:0/2732554868 conn(0x7f199c05c550 msgr2=0x7f199c0b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:24.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f19a2262700 1 -- 172.21.15.36:0/2732554868 shutdown_connections 2022-06-17T12:47:24.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.981+0000 7f19a2262700 1 -- 172.21.15.36:0/2732554868 wait complete. 2022-06-17T12:47:24.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f19a2262700 1 Processor -- start 2022-06-17T12:47:24.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f19a2262700 1 -- start start 2022-06-17T12:47:24.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f19a2262700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1988004500 0x7f199c062ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:24.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f19a2262700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f199c159cd0 0x7f199c14abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:24.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f19a2262700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f199c156c80 0x7f199c157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:24.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f19a2262700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199c15a130 con 0x7f199c156c80 2022-06-17T12:47:24.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f19a2262700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199c142580 con 0x7f199c159cd0 2022-06-17T12:47:24.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f19a2262700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199c156850 con 0x7f1988004500 2022-06-17T12:47:24.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f199bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f199c156c80 0x7f199c157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:24.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f199bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f199c156c80 0x7f199c157100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60374/0 (socket says 172.21.15.36:60374) 2022-06-17T12:47:24.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f199bfff700 1 -- 172.21.15.36:0/346838455 learned_addr learned my addr 172.21.15.36:0/346838455 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:24.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f199b7fe700 1 --2- 172.21.15.36:0/346838455 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1988004500 0x7f199c062ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:24.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f199affd700 1 --2- 172.21.15.36:0/346838455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f199c159cd0 0x7f199c14abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:24.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f199bfff700 1 -- 172.21.15.36:0/346838455 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1988004500 msgr2=0x7f199c062ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:24.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f199bfff700 1 --2- 172.21.15.36:0/346838455 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1988004500 0x7f199c062ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:24.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f199bfff700 1 -- 172.21.15.36:0/346838455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f199c159cd0 msgr2=0x7f199c14abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:24.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f199bfff700 1 --2- 172.21.15.36:0/346838455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f199c159cd0 0x7f199c14abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:24.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f199bfff700 1 -- 172.21.15.36:0/346838455 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199c0a8990 con 0x7f199c156c80 2022-06-17T12:47:24.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f199affd700 1 --2- 172.21.15.36:0/346838455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f199c159cd0 0x7f199c14abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:24.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f199bfff700 1 --2- 172.21.15.36:0/346838455 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f199c156c80 0x7f199c157100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f1994000b60 tx=0x7f1994002190).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:24.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f1998ff9700 1 -- 172.21.15.36:0/346838455 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f199400f820 con 0x7f199c156c80 2022-06-17T12:47:24.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f1998ff9700 1 -- 172.21.15.36:0/346838455 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199400f980 con 0x7f199c156c80 2022-06-17T12:47:24.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f19a2262700 1 -- 172.21.15.36:0/346838455 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f199c1575f0 con 0x7f199c156c80 2022-06-17T12:47:24.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f1998ff9700 1 -- 172.21.15.36:0/346838455 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f199400fce0 con 0x7f199c156c80 2022-06-17T12:47:24.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.985+0000 7f19a2262700 1 -- 172.21.15.36:0/346838455 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f199c0a8f90 con 0x7f199c156c80 2022-06-17T12:47:24.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.989+0000 7f19a2262700 1 -- 172.21.15.36:0/346838455 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f199c0a87b0 con 0x7f199c156c80 2022-06-17T12:47:24.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.989+0000 7f1998ff9700 1 -- 172.21.15.36:0/346838455 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f199401b030 con 0x7f199c156c80 2022-06-17T12:47:24.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.989+0000 7f1998ff9700 1 --2- 172.21.15.36:0/346838455 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f197c053a60 0x7f197c055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:24.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.989+0000 7f1998ff9700 1 -- 172.21.15.36:0/346838455 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f199406bbc0 con 0x7f199c156c80 2022-06-17T12:47:24.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.989+0000 7f199b7fe700 1 --2- 172.21.15.36:0/346838455 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f197c053a60 0x7f197c055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:24.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.989+0000 7f199b7fe700 1 --2- 172.21.15.36:0/346838455 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f197c053a60 0x7f197c055f20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f1990003000 tx=0x7f1990007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:24.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:23.993+0000 7f1998ff9700 1 -- 172.21.15.36:0/346838455 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1994042960 con 0x7f199c156c80 2022-06-17T12:47:24.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.121+0000 7f19a2262700 1 -- 172.21.15.36:0/346838455 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f199c057410 con 0x7f199c156c80 2022-06-17T12:47:24.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.121+0000 7f1998ff9700 1 -- 172.21.15.36:0/346838455 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1994019070 con 0x7f199c156c80 2022-06-17T12:47:24.313 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:24.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.125+0000 7f19a2262700 1 -- 172.21.15.36:0/346838455 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f197c053a60 msgr2=0x7f197c055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:24.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.125+0000 7f19a2262700 1 --2- 172.21.15.36:0/346838455 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f197c053a60 0x7f197c055f20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f1990003000 tx=0x7f1990007a00).stop 2022-06-17T12:47:24.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.125+0000 7f19a2262700 1 -- 172.21.15.36:0/346838455 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f199c156c80 msgr2=0x7f199c157100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:24.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.125+0000 7f19a2262700 1 --2- 172.21.15.36:0/346838455 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f199c156c80 0x7f199c157100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f1994000b60 tx=0x7f1994002190).stop 2022-06-17T12:47:24.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.125+0000 7f19a2262700 1 -- 172.21.15.36:0/346838455 shutdown_connections 2022-06-17T12:47:24.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.125+0000 7f19a2262700 1 --2- 172.21.15.36:0/346838455 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f197c053a60 0x7f197c055f20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:24.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.125+0000 7f19a2262700 1 --2- 172.21.15.36:0/346838455 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f199c156c80 0x7f199c157100 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:24.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.125+0000 7f19a2262700 1 --2- 172.21.15.36:0/346838455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f199c159cd0 0x7f199c14abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:24.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.125+0000 7f19a2262700 1 --2- 172.21.15.36:0/346838455 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1988004500 0x7f199c062ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:24.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.125+0000 7f19a2262700 1 -- 172.21.15.36:0/346838455 >> 172.21.15.36:0/346838455 conn(0x7f199c05c550 msgr2=0x7f199c148950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:24.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.125+0000 7f19a2262700 1 -- 172.21.15.36:0/346838455 shutdown_connections 2022-06-17T12:47:24.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.125+0000 7f19a2262700 1 -- 172.21.15.36:0/346838455 wait complete. 2022-06-17T12:47:24.360 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:47:24.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.549+0000 7f012a080700 1 Processor -- start 2022-06-17T12:47:24.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.549+0000 7f012a080700 1 -- start start 2022-06-17T12:47:24.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.549+0000 7f012a080700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f01240b9860 0x7f0124143840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:24.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.549+0000 7f012a080700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01240b4160 con 0x7f01240b5dc0 2022-06-17T12:47:24.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.549+0000 7f012a080700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01240b9c40 con 0x7f01240b45c0 2022-06-17T12:47:24.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.549+0000 7f012a080700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01240b5120 con 0x7f01240b9860 2022-06-17T12:47:24.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.549+0000 7f012907e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f01240b9860 0x7f0124143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:24.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.549+0000 7f012987f700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f01240b5dc0 0x7f01240b6210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49678/0 (socket says 172.21.15.36:49678) 2022-06-17T12:47:24.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.549+0000 7f012987f700 1 -- 172.21.15.36:0/1438059619 learned_addr learned my addr 172.21.15.36:0/1438059619 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:24.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.549+0000 7f011bfff700 1 -- 172.21.15.36:0/1438059619 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165191771 0 0) 0x7f01240b4160 con 0x7f01240b5dc0 2022-06-17T12:47:24.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.549+0000 7f011bfff700 1 -- 172.21.15.36:0/1438059619 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0108003260 con 0x7f01240b5dc0 2022-06-17T12:47:24.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.549+0000 7f012907e700 1 -- 172.21.15.36:0/1438059619 >> v1:172.21.15.70:6789/0 conn(0x7f01240b45c0 legacy=0x7f01240b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:24.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.549+0000 7f012907e700 1 -- 172.21.15.36:0/1438059619 >> v1:172.21.15.36:6789/0 conn(0x7f01240b5dc0 legacy=0x7f01240b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:24.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.549+0000 7f012907e700 1 -- 172.21.15.36:0/1438059619 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01240627d0 con 0x7f01240b9860 2022-06-17T12:47:24.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.549+0000 7f012907e700 1 --2- 172.21.15.36:0/1438059619 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f01240b9860 0x7f0124143840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f0114002f70 tx=0x7f0114006df0).ready entity=mon.0 client_cookie=a7f0cf08458fa9c4 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:24.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.553+0000 7f011bfff700 1 -- 172.21.15.36:0/1438059619 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f011400d610 con 0x7f01240b9860 2022-06-17T12:47:24.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.553+0000 7f011bfff700 1 -- 172.21.15.36:0/1438059619 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0114000e90 con 0x7f01240b9860 2022-06-17T12:47:24.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.553+0000 7f011bfff700 1 -- 172.21.15.36:0/1438059619 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f011400d960 con 0x7f01240b9860 2022-06-17T12:47:24.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.553+0000 7f012a080700 1 -- 172.21.15.36:0/1438059619 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f01240b9860 msgr2=0x7f0124143840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:24.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.553+0000 7f012a080700 1 --2- 172.21.15.36:0/1438059619 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f01240b9860 0x7f0124143840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f0114002f70 tx=0x7f0114006df0).stop 2022-06-17T12:47:24.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.553+0000 7f012a080700 1 -- 172.21.15.36:0/1438059619 shutdown_connections 2022-06-17T12:47:24.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.553+0000 7f012a080700 1 --2- 172.21.15.36:0/1438059619 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f01240b9860 0x7f0124143840 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:24.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.553+0000 7f012a080700 1 -- 172.21.15.36:0/1438059619 >> 172.21.15.36:0/1438059619 conn(0x7f012405c550 msgr2=0x7f0124060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:24.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.553+0000 7f012a080700 1 -- 172.21.15.36:0/1438059619 shutdown_connections 2022-06-17T12:47:24.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.553+0000 7f012a080700 1 -- 172.21.15.36:0/1438059619 wait complete. 2022-06-17T12:47:24.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.557+0000 7f012a080700 1 Processor -- start 2022-06-17T12:47:24.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.557+0000 7f012a080700 1 -- start start 2022-06-17T12:47:24.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.557+0000 7f012a080700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f01240b45c0 0x7f0124145620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:24.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.557+0000 7f012a080700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01240b5dc0 0x7f0124144530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:24.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.557+0000 7f012a080700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01240b9860 0x7f0124152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:24.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.557+0000 7f012a080700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0124144f50 con 0x7f01240b45c0 2022-06-17T12:47:24.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.557+0000 7f012a080700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01240add80 con 0x7f01240b9860 2022-06-17T12:47:24.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.557+0000 7f012a080700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012413fb50 con 0x7f01240b5dc0 2022-06-17T12:47:24.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.557+0000 7f012907e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f01240b45c0 0x7f0124145620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:24.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.557+0000 7f012907e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f01240b45c0 0x7f0124145620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60382/0 (socket says 172.21.15.36:60382) 2022-06-17T12:47:24.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.557+0000 7f012907e700 1 -- 172.21.15.36:0/3041012712 learned_addr learned my addr 172.21.15.36:0/3041012712 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:24.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.557+0000 7f012987f700 1 --2- 172.21.15.36:0/3041012712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01240b9860 0x7f0124152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:24.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.557+0000 7f012887d700 1 --2- 172.21.15.36:0/3041012712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01240b5dc0 0x7f0124144530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:24.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.557+0000 7f012907e700 1 -- 172.21.15.36:0/3041012712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01240b5dc0 msgr2=0x7f0124144530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:24.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.557+0000 7f012907e700 1 --2- 172.21.15.36:0/3041012712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01240b5dc0 0x7f0124144530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:24.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.557+0000 7f012907e700 1 -- 172.21.15.36:0/3041012712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01240b9860 msgr2=0x7f0124152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:24.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.557+0000 7f012907e700 1 --2- 172.21.15.36:0/3041012712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01240b9860 0x7f0124152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:24.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.557+0000 7f012907e700 1 -- 172.21.15.36:0/3041012712 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011400b180 con 0x7f01240b45c0 2022-06-17T12:47:24.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.557+0000 7f012907e700 1 --2- 172.21.15.36:0/3041012712 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f01240b45c0 0x7f0124145620 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f0114002f70 tx=0x7f0114014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:24.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.557+0000 7f011a7fc700 1 -- 172.21.15.36:0/3041012712 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0114013de0 con 0x7f01240b45c0 2022-06-17T12:47:24.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.557+0000 7f011a7fc700 1 -- 172.21.15.36:0/3041012712 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f011400d290 con 0x7f01240b45c0 2022-06-17T12:47:24.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.557+0000 7f012a080700 1 -- 172.21.15.36:0/3041012712 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0124145180 con 0x7f01240b45c0 2022-06-17T12:47:24.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.557+0000 7f011a7fc700 1 -- 172.21.15.36:0/3041012712 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f011401e5f0 con 0x7f01240b45c0 2022-06-17T12:47:24.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.557+0000 7f012a080700 1 -- 172.21.15.36:0/3041012712 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012413f810 con 0x7f01240b45c0 2022-06-17T12:47:24.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.561+0000 7f012a080700 1 -- 172.21.15.36:0/3041012712 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0124062eb0 con 0x7f01240b45c0 2022-06-17T12:47:24.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.561+0000 7f011a7fc700 1 -- 172.21.15.36:0/3041012712 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f0114025030 con 0x7f01240b45c0 2022-06-17T12:47:24.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.561+0000 7f011a7fc700 1 --2- 172.21.15.36:0/3041012712 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f010c053a40 0x7f010c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:24.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.561+0000 7f011a7fc700 1 -- 172.21.15.36:0/3041012712 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f011406cf00 con 0x7f01240b45c0 2022-06-17T12:47:24.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.561+0000 7f012887d700 1 --2- 172.21.15.36:0/3041012712 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f010c053a40 0x7f010c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:24.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.561+0000 7f012887d700 1 --2- 172.21.15.36:0/3041012712 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f010c053a40 0x7f010c055f00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f012000d000 tx=0x7f0120009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:24.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.569+0000 7f011a7fc700 1 -- 172.21.15.36:0/3041012712 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0114043c30 con 0x7f01240b45c0 2022-06-17T12:47:24.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.693+0000 7f012a080700 1 -- 172.21.15.36:0/3041012712 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f012413fd10 con 0x7f01240b45c0 2022-06-17T12:47:24.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.693+0000 7f011a7fc700 1 -- 172.21.15.36:0/3041012712 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+10 (secure 0 0 0) 0x7f0114023020 con 0x7f01240b45c0 2022-06-17T12:47:24.683 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:24.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.697+0000 7f012a080700 1 -- 172.21.15.36:0/3041012712 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f010c053a40 msgr2=0x7f010c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:24.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.697+0000 7f012a080700 1 --2- 172.21.15.36:0/3041012712 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f010c053a40 0x7f010c055f00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f012000d000 tx=0x7f0120009040).stop 2022-06-17T12:47:24.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.697+0000 7f012a080700 1 -- 172.21.15.36:0/3041012712 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f01240b45c0 msgr2=0x7f0124145620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:24.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.697+0000 7f012a080700 1 --2- 172.21.15.36:0/3041012712 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f01240b45c0 0x7f0124145620 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f0114002f70 tx=0x7f0114014000).stop 2022-06-17T12:47:24.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.697+0000 7f012a080700 1 -- 172.21.15.36:0/3041012712 shutdown_connections 2022-06-17T12:47:24.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.697+0000 7f012a080700 1 --2- 172.21.15.36:0/3041012712 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f010c053a40 0x7f010c055f00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:24.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.697+0000 7f012a080700 1 --2- 172.21.15.36:0/3041012712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01240b9860 0x7f0124152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:24.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.697+0000 7f012a080700 1 --2- 172.21.15.36:0/3041012712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01240b5dc0 0x7f0124144530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:24.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.697+0000 7f012a080700 1 --2- 172.21.15.36:0/3041012712 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f01240b45c0 0x7f0124145620 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:24.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.697+0000 7f012a080700 1 -- 172.21.15.36:0/3041012712 >> 172.21.15.36:0/3041012712 conn(0x7f012405c550 msgr2=0x7f01240b88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:24.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.697+0000 7f012a080700 1 -- 172.21.15.36:0/3041012712 shutdown_connections 2022-06-17T12:47:24.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.697+0000 7f012a080700 1 -- 172.21.15.36:0/3041012712 wait complete. 2022-06-17T12:47:24.709 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:47:24.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.841+0000 7f6b96dd2700 1 Processor -- start 2022-06-17T12:47:24.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.841+0000 7f6b96dd2700 1 -- start start 2022-06-17T12:47:24.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b96dd2700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6b900afd80 0x7f6b90140bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:24.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b96dd2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b900af540 0x7f6b901410a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:24.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b96dd2700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b900b3140 con 0x7f6b90060ac0 2022-06-17T12:47:24.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b96dd2700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b901441e0 con 0x7f6b900afd80 2022-06-17T12:47:24.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b96dd2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b90060ea0 con 0x7f6b900af540 2022-06-17T12:47:24.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b955cf700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6b900afd80 0x7f6b90140bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:24.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b965d1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b900af540 0x7f6b901410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:24.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b95dd0700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6b90060ac0 0x7f6b90144b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:45422/0 (socket says 172.21.15.36:45422) 2022-06-17T12:47:24.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b95dd0700 1 -- 172.21.15.36:0/3123060416 learned_addr learned my addr 172.21.15.36:0/3123060416 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:24.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b955cf700 1 -- 172.21.15.36:0/3123060416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b900af540 msgr2=0x7f6b901410a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:24.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b955cf700 1 --2- 172.21.15.36:0/3123060416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b900af540 0x7f6b901410a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:24.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b955cf700 1 -- 172.21.15.36:0/3123060416 >> v1:172.21.15.70:6789/0 conn(0x7f6b90060ac0 legacy=0x7f6b90144b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:24.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b955cf700 1 -- 172.21.15.36:0/3123060416 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b901443d0 con 0x7f6b900afd80 2022-06-17T12:47:24.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b965d1700 1 --2- 172.21.15.36:0/3123060416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b900af540 0x7f6b901410a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:24.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b955cf700 1 --2- 172.21.15.36:0/3123060416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b900afd80 0x7f6b90140bb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f6b80002f70 tx=0x7f6b80006df0).ready entity=mon.1 client_cookie=5418e46b8c16666f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:24.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b94dce700 1 -- 172.21.15.36:0/3123060416 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b8000d570 con 0x7f6b900afd80 2022-06-17T12:47:24.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b94dce700 1 -- 172.21.15.36:0/3123060416 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b80000e90 con 0x7f6b900afd80 2022-06-17T12:47:24.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b94dce700 1 -- 172.21.15.36:0/3123060416 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b8000d8c0 con 0x7f6b900afd80 2022-06-17T12:47:24.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b96dd2700 1 -- 172.21.15.36:0/3123060416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b900afd80 msgr2=0x7f6b90140bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:24.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b96dd2700 1 --2- 172.21.15.36:0/3123060416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b900afd80 0x7f6b90140bb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f6b80002f70 tx=0x7f6b80006df0).stop 2022-06-17T12:47:24.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b96dd2700 1 -- 172.21.15.36:0/3123060416 shutdown_connections 2022-06-17T12:47:24.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b96dd2700 1 --2- 172.21.15.36:0/3123060416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b900af540 0x7f6b901410a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:24.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b96dd2700 1 --2- 172.21.15.36:0/3123060416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b900afd80 0x7f6b90140bb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:24.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b96dd2700 1 -- 172.21.15.36:0/3123060416 >> 172.21.15.36:0/3123060416 conn(0x7f6b900b65f0 msgr2=0x7f6b900b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:24.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b96dd2700 1 -- 172.21.15.36:0/3123060416 shutdown_connections 2022-06-17T12:47:24.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.845+0000 7f6b96dd2700 1 -- 172.21.15.36:0/3123060416 wait complete. 2022-06-17T12:47:24.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.849+0000 7f6b96dd2700 1 Processor -- start 2022-06-17T12:47:24.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.849+0000 7f6b96dd2700 1 -- start start 2022-06-17T12:47:24.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.849+0000 7f6b96dd2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b90060ac0 0x7f6b9013cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:24.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.849+0000 7f6b96dd2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b900af540 0x7f6b90140450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:24.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.849+0000 7f6b96dd2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6b900afd80 0x7f6b901387d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:24.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.849+0000 7f6b96dd2700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b90063da0 con 0x7f6b900afd80 2022-06-17T12:47:24.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.849+0000 7f6b96dd2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b90064040 con 0x7f6b90060ac0 2022-06-17T12:47:24.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.849+0000 7f6b96dd2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9013b7f0 con 0x7f6b900af540 2022-06-17T12:47:24.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.853+0000 7f6b965d1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6b900afd80 0x7f6b901387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:24.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.853+0000 7f6b965d1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6b900afd80 0x7f6b901387d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60400/0 (socket says 172.21.15.36:60400) 2022-06-17T12:47:24.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.853+0000 7f6b965d1700 1 -- 172.21.15.36:0/2861742243 learned_addr learned my addr 172.21.15.36:0/2861742243 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:24.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.853+0000 7f6b95dd0700 1 --2- 172.21.15.36:0/2861742243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b90060ac0 0x7f6b9013cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:24.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.853+0000 7f6b955cf700 1 --2- 172.21.15.36:0/2861742243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b900af540 0x7f6b90140450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:24.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.853+0000 7f6b965d1700 1 -- 172.21.15.36:0/2861742243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b900af540 msgr2=0x7f6b90140450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:24.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.853+0000 7f6b965d1700 1 --2- 172.21.15.36:0/2861742243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b900af540 0x7f6b90140450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:24.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.853+0000 7f6b965d1700 1 -- 172.21.15.36:0/2861742243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b90060ac0 msgr2=0x7f6b9013cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:24.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.853+0000 7f6b965d1700 1 --2- 172.21.15.36:0/2861742243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b90060ac0 0x7f6b9013cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:24.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.853+0000 7f6b965d1700 1 -- 172.21.15.36:0/2861742243 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b88009470 con 0x7f6b900afd80 2022-06-17T12:47:24.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.853+0000 7f6b965d1700 1 --2- 172.21.15.36:0/2861742243 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6b900afd80 0x7f6b901387d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f6b8800c000 tx=0x7f6b880091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:24.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.853+0000 7f6b7effd700 1 -- 172.21.15.36:0/2861742243 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b8800c850 con 0x7f6b900afd80 2022-06-17T12:47:24.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.853+0000 7f6b7effd700 1 -- 172.21.15.36:0/2861742243 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8800c9b0 con 0x7f6b900afd80 2022-06-17T12:47:24.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.853+0000 7f6b96dd2700 1 -- 172.21.15.36:0/2861742243 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b8000b180 con 0x7f6b900afd80 2022-06-17T12:47:24.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.853+0000 7f6b7effd700 1 -- 172.21.15.36:0/2861742243 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b8800cd10 con 0x7f6b900afd80 2022-06-17T12:47:24.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.853+0000 7f6b96dd2700 1 -- 172.21.15.36:0/2861742243 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b90067880 con 0x7f6b900afd80 2022-06-17T12:47:24.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.853+0000 7f6b96dd2700 1 -- 172.21.15.36:0/2861742243 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b9013b640 con 0x7f6b900afd80 2022-06-17T12:47:24.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.853+0000 7f6b7effd700 1 -- 172.21.15.36:0/2861742243 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6b88022070 con 0x7f6b900afd80 2022-06-17T12:47:24.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.857+0000 7f6b7effd700 1 --2- 172.21.15.36:0/2861742243 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6b70053ad0 0x7f6b70055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:24.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.857+0000 7f6b95dd0700 1 --2- 172.21.15.36:0/2861742243 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6b70053ad0 0x7f6b70055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:24.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.857+0000 7f6b7effd700 1 -- 172.21.15.36:0/2861742243 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f6b8806c770 con 0x7f6b900afd80 2022-06-17T12:47:24.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.857+0000 7f6b95dd0700 1 --2- 172.21.15.36:0/2861742243 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6b70053ad0 0x7f6b70055f90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f6b8c00d000 tx=0x7f6b8c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:24.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.861+0000 7f6b7effd700 1 -- 172.21.15.36:0/2861742243 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6b88043570 con 0x7f6b900afd80 2022-06-17T12:47:24.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.989+0000 7f6b96dd2700 1 -- 172.21.15.36:0/2861742243 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b9013ba10 con 0x7f6b900afd80 2022-06-17T12:47:24.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.989+0000 7f6b7effd700 1 -- 172.21.15.36:0/2861742243 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f6b88046b90 con 0x7f6b900afd80 2022-06-17T12:47:24.980 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:24.980 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":26,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:46:53.258075+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:46:47.465328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-17T12:46:49.893338+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:47:25.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.993+0000 7f6b96dd2700 1 -- 172.21.15.36:0/2861742243 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6b70053ad0 msgr2=0x7f6b70055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:25.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.993+0000 7f6b96dd2700 1 --2- 172.21.15.36:0/2861742243 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6b70053ad0 0x7f6b70055f90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f6b8c00d000 tx=0x7f6b8c009040).stop 2022-06-17T12:47:25.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.993+0000 7f6b96dd2700 1 -- 172.21.15.36:0/2861742243 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6b900afd80 msgr2=0x7f6b901387d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:25.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.993+0000 7f6b96dd2700 1 --2- 172.21.15.36:0/2861742243 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6b900afd80 0x7f6b901387d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f6b8800c000 tx=0x7f6b880091d0).stop 2022-06-17T12:47:25.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.993+0000 7f6b96dd2700 1 -- 172.21.15.36:0/2861742243 shutdown_connections 2022-06-17T12:47:25.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.993+0000 7f6b96dd2700 1 --2- 172.21.15.36:0/2861742243 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6b70053ad0 0x7f6b70055f90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:25.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.993+0000 7f6b96dd2700 1 --2- 172.21.15.36:0/2861742243 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6b900afd80 0x7f6b901387d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:25.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.993+0000 7f6b96dd2700 1 --2- 172.21.15.36:0/2861742243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b900af540 0x7f6b90140450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:25.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.993+0000 7f6b96dd2700 1 --2- 172.21.15.36:0/2861742243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b90060ac0 0x7f6b9013cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:25.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.993+0000 7f6b96dd2700 1 -- 172.21.15.36:0/2861742243 >> 172.21.15.36:0/2861742243 conn(0x7f6b900b65f0 msgr2=0x7f6b900b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:25.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.993+0000 7f6b96dd2700 1 -- 172.21.15.36:0/2861742243 shutdown_connections 2022-06-17T12:47:25.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:24.993+0000 7f6b96dd2700 1 -- 172.21.15.36:0/2861742243 wait complete. 2022-06-17T12:47:25.021 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:46:47.465328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:46:49.893338+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:47:25.022 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:47:25.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.153+0000 7f97f39aa700 1 Processor -- start 2022-06-17T12:47:25.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.153+0000 7f97f39aa700 1 -- start start 2022-06-17T12:47:25.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.153+0000 7f97f39aa700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f97ec0b2d80 0x7f97ec0b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:25.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.153+0000 7f97f39aa700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ec0aa170 con 0x7f97ec0b2350 2022-06-17T12:47:25.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.153+0000 7f97f39aa700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ec0a9c30 con 0x7f97ec143e60 2022-06-17T12:47:25.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.153+0000 7f97f39aa700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ec0a9ed0 con 0x7f97ec0b2d80 2022-06-17T12:47:25.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.153+0000 7f97f1746700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f97ec0b2d80 0x7f97ec0b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:25.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.153+0000 7f97f0f45700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f97ec0b2350 0x7f97ec143750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49704/0 (socket says 172.21.15.36:49704) 2022-06-17T12:47:25.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.153+0000 7f97f0f45700 1 -- 172.21.15.36:0/4120922691 learned_addr learned my addr 172.21.15.36:0/4120922691 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:25.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.153+0000 7f97f1746700 1 -- 172.21.15.36:0/4120922691 >> v1:172.21.15.70:6789/0 conn(0x7f97ec143e60 legacy=0x7f97ec140300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:25.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.153+0000 7f97f1746700 1 -- 172.21.15.36:0/4120922691 >> v1:172.21.15.36:6789/0 conn(0x7f97ec0b2350 legacy=0x7f97ec143750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:25.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.153+0000 7f97f1746700 1 -- 172.21.15.36:0/4120922691 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ec0b08c0 con 0x7f97ec0b2d80 2022-06-17T12:47:25.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.153+0000 7f97f1746700 1 --2- 172.21.15.36:0/4120922691 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97ec0b2d80 0x7f97ec0b3160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f97e8002f70 tx=0x7f97e8006df0).ready entity=mon.0 client_cookie=77fc487048004557 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:25.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.153+0000 7f97dbfff700 1 -- 172.21.15.36:0/4120922691 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97e800fa10 con 0x7f97ec0b2d80 2022-06-17T12:47:25.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.153+0000 7f97dbfff700 1 -- 172.21.15.36:0/4120922691 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e8000e90 con 0x7f97ec0b2d80 2022-06-17T12:47:25.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.153+0000 7f97dbfff700 1 -- 172.21.15.36:0/4120922691 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97e800fd60 con 0x7f97ec0b2d80 2022-06-17T12:47:25.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.153+0000 7f97f39aa700 1 -- 172.21.15.36:0/4120922691 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97ec0b2d80 msgr2=0x7f97ec0b3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:25.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.153+0000 7f97f39aa700 1 --2- 172.21.15.36:0/4120922691 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97ec0b2d80 0x7f97ec0b3160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f97e8002f70 tx=0x7f97e8006df0).stop 2022-06-17T12:47:25.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.153+0000 7f97f39aa700 1 -- 172.21.15.36:0/4120922691 shutdown_connections 2022-06-17T12:47:25.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.153+0000 7f97f39aa700 1 --2- 172.21.15.36:0/4120922691 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97ec0b2d80 0x7f97ec0b3160 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:25.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.153+0000 7f97f39aa700 1 -- 172.21.15.36:0/4120922691 >> 172.21.15.36:0/4120922691 conn(0x7f97ec05c550 msgr2=0x7f97ec05c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:25.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.153+0000 7f97f39aa700 1 -- 172.21.15.36:0/4120922691 shutdown_connections 2022-06-17T12:47:25.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.153+0000 7f97f39aa700 1 -- 172.21.15.36:0/4120922691 wait complete. 2022-06-17T12:47:25.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.157+0000 7f97f39aa700 1 Processor -- start 2022-06-17T12:47:25.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.157+0000 7f97f39aa700 1 -- start start 2022-06-17T12:47:25.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.157+0000 7f97f39aa700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97ec0b2350 0x7f97ec0657f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:25.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.157+0000 7f97f39aa700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97ec0b2d80 0x7f97ec0b4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:25.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.157+0000 7f97f39aa700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97ec143e60 0x7f97ec13f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:25.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.157+0000 7f97f39aa700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec0669d0 con 0x7f97ec0b2350 2022-06-17T12:47:25.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.157+0000 7f97f39aa700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec066c70 con 0x7f97ec143e60 2022-06-17T12:47:25.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.157+0000 7f97f39aa700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec066f10 con 0x7f97ec0b2d80 2022-06-17T12:47:25.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.157+0000 7f97f1746700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97ec0b2350 0x7f97ec0657f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:25.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.157+0000 7f97f1746700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97ec0b2350 0x7f97ec0657f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60410/0 (socket says 172.21.15.36:60410) 2022-06-17T12:47:25.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.157+0000 7f97f1746700 1 -- 172.21.15.36:0/601264108 learned_addr learned my addr 172.21.15.36:0/601264108 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:25.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.157+0000 7f97f1f47700 1 --2- 172.21.15.36:0/601264108 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97ec143e60 0x7f97ec13f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:25.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.157+0000 7f97f0f45700 1 --2- 172.21.15.36:0/601264108 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97ec0b2d80 0x7f97ec0b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:25.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.157+0000 7f97f1746700 1 -- 172.21.15.36:0/601264108 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97ec0b2d80 msgr2=0x7f97ec0b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:25.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.157+0000 7f97f1746700 1 --2- 172.21.15.36:0/601264108 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97ec0b2d80 0x7f97ec0b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:25.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.157+0000 7f97f1746700 1 -- 172.21.15.36:0/601264108 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97ec143e60 msgr2=0x7f97ec13f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:25.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.157+0000 7f97f1746700 1 --2- 172.21.15.36:0/601264108 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97ec143e60 0x7f97ec13f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:25.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.157+0000 7f97f1746700 1 -- 172.21.15.36:0/601264108 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e800b180 con 0x7f97ec0b2350 2022-06-17T12:47:25.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.157+0000 7f97f1746700 1 --2- 172.21.15.36:0/601264108 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97ec0b2350 0x7f97ec0657f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f97e8002f70 tx=0x7f97e8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:25.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.157+0000 7f97da7fc700 1 -- 172.21.15.36:0/601264108 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97e80185f0 con 0x7f97ec0b2350 2022-06-17T12:47:25.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.157+0000 7f97da7fc700 1 -- 172.21.15.36:0/601264108 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e800f4a0 con 0x7f97ec0b2350 2022-06-17T12:47:25.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.157+0000 7f97da7fc700 1 -- 172.21.15.36:0/601264108 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97e8018940 con 0x7f97ec0b2350 2022-06-17T12:47:25.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.157+0000 7f97f39aa700 1 -- 172.21.15.36:0/601264108 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ec065460 con 0x7f97ec0b2350 2022-06-17T12:47:25.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.157+0000 7f97f39aa700 1 -- 172.21.15.36:0/601264108 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ec152990 con 0x7f97ec0b2350 2022-06-17T12:47:25.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.161+0000 7f97f39aa700 1 -- 172.21.15.36:0/601264108 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ec0a4530 con 0x7f97ec0b2350 2022-06-17T12:47:25.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.161+0000 7f97da7fc700 1 -- 172.21.15.36:0/601264108 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f97e802b070 con 0x7f97ec0b2350 2022-06-17T12:47:25.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.161+0000 7f97da7fc700 1 --2- 172.21.15.36:0/601264108 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f97cc053ad0 0x7f97cc055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:25.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.161+0000 7f97da7fc700 1 -- 172.21.15.36:0/601264108 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f97e806d4d0 con 0x7f97ec0b2350 2022-06-17T12:47:25.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.161+0000 7f97f0f45700 1 --2- 172.21.15.36:0/601264108 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f97cc053ad0 0x7f97cc055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:25.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.161+0000 7f97f0f45700 1 --2- 172.21.15.36:0/601264108 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f97cc053ad0 0x7f97cc055f90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f97dc00d000 tx=0x7f97dc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:25.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.165+0000 7f97da7fc700 1 -- 172.21.15.36:0/601264108 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f97e80441c0 con 0x7f97ec0b2350 2022-06-17T12:47:25.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.293+0000 7f97f39aa700 1 -- 172.21.15.36:0/601264108 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f97ec1442d0 con 0x7f97ec0b2350 2022-06-17T12:47:25.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.293+0000 7f97da7fc700 1 -- 172.21.15.36:0/601264108 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7f97e80477e0 con 0x7f97ec0b2350 2022-06-17T12:47:25.363 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:47:25.366 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.293+0000 7f97f39aa700 1 -- 172.21.15.36:0/601264108 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f97cc053ad0 msgr2=0x7f97cc055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:25.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.293+0000 7f97f39aa700 1 --2- 172.21.15.36:0/601264108 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f97cc053ad0 0x7f97cc055f90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f97dc00d000 tx=0x7f97dc009040).stop 2022-06-17T12:47:25.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.293+0000 7f97f39aa700 1 -- 172.21.15.36:0/601264108 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97ec0b2350 msgr2=0x7f97ec0657f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:25.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.293+0000 7f97f39aa700 1 --2- 172.21.15.36:0/601264108 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97ec0b2350 0x7f97ec0657f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f97e8002f70 tx=0x7f97e8017000).stop 2022-06-17T12:47:25.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.293+0000 7f97f39aa700 1 -- 172.21.15.36:0/601264108 shutdown_connections 2022-06-17T12:47:25.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.293+0000 7f97f39aa700 1 --2- 172.21.15.36:0/601264108 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f97cc053ad0 0x7f97cc055f90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:25.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.293+0000 7f97f39aa700 1 --2- 172.21.15.36:0/601264108 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97ec143e60 0x7f97ec13f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:25.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.293+0000 7f97f39aa700 1 --2- 172.21.15.36:0/601264108 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97ec0b2d80 0x7f97ec0b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:25.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.293+0000 7f97f39aa700 1 --2- 172.21.15.36:0/601264108 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97ec0b2350 0x7f97ec0657f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:25.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.293+0000 7f97f39aa700 1 -- 172.21.15.36:0/601264108 >> 172.21.15.36:0/601264108 conn(0x7f97ec05c550 msgr2=0x7f97ec0afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:25.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.293+0000 7f97f39aa700 1 -- 172.21.15.36:0/601264108 shutdown_connections 2022-06-17T12:47:25.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.293+0000 7f97f39aa700 1 -- 172.21.15.36:0/601264108 wait complete. 2022-06-17T12:47:25.403 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:47:25.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.537+0000 7f819f595700 1 Processor -- start 2022-06-17T12:47:25.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.537+0000 7f819f595700 1 -- start start 2022-06-17T12:47:25.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.537+0000 7f819f595700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f81980b4120 0x7f8198146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:25.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.537+0000 7f819f595700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81981476e0 con 0x7f81980b64f0 2022-06-17T12:47:25.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.537+0000 7f819f595700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81980b68d0 con 0x7f8198147270 2022-06-17T12:47:25.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.537+0000 7f819f595700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81980b6e20 con 0x7f81980b4120 2022-06-17T12:47:25.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.537+0000 7f819d331700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f81980b64f0 0x7f8198143150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49716/0 (socket says 172.21.15.36:49716) 2022-06-17T12:47:25.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.537+0000 7f819d331700 1 -- 172.21.15.36:0/3271477125 learned_addr learned my addr 172.21.15.36:0/3271477125 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:25.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.537+0000 7f819cb30700 1 --2- 172.21.15.36:0/3271477125 >> v2:172.21.15.70:3300/0 conn(0x7f81980b4120 0x7f8198146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:25.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.537+0000 7f818ffff700 1 -- 172.21.15.36:0/3271477125 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804061271 0 0) 0x7f81981476e0 con 0x7f81980b64f0 2022-06-17T12:47:25.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.537+0000 7f818ffff700 1 -- 172.21.15.36:0/3271477125 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f817c003260 con 0x7f81980b64f0 2022-06-17T12:47:25.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.537+0000 7f818ffff700 1 -- 172.21.15.36:0/3271477125 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4000929492 0 0) 0x7f81980b68d0 con 0x7f8198147270 2022-06-17T12:47:25.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.537+0000 7f818ffff700 1 -- 172.21.15.36:0/3271477125 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81981476e0 con 0x7f8198147270 2022-06-17T12:47:25.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.537+0000 7f819cb30700 1 -- 172.21.15.36:0/3271477125 >> v1:172.21.15.70:6789/0 conn(0x7f8198147270 legacy=0x7f8198140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:25.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.537+0000 7f819cb30700 1 -- 172.21.15.36:0/3271477125 >> v1:172.21.15.36:6789/0 conn(0x7f81980b64f0 legacy=0x7f8198143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:25.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.537+0000 7f819cb30700 1 -- 172.21.15.36:0/3271477125 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81981468e0 con 0x7f81980b4120 2022-06-17T12:47:25.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.537+0000 7f819cb30700 1 --2- 172.21.15.36:0/3271477125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81980b4120 0x7f8198146d80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f8188002f70 tx=0x7f8188006df0).ready entity=mon.1 client_cookie=c080fe28a75b8728 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:25.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.537+0000 7f818ffff700 1 -- 172.21.15.36:0/3271477125 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f818800d540 con 0x7f81980b4120 2022-06-17T12:47:25.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.537+0000 7f818ffff700 1 -- 172.21.15.36:0/3271477125 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8188000e90 con 0x7f81980b4120 2022-06-17T12:47:25.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.537+0000 7f818ffff700 1 -- 172.21.15.36:0/3271477125 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f818800d890 con 0x7f81980b4120 2022-06-17T12:47:25.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819f595700 1 -- 172.21.15.36:0/3271477125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81980b4120 msgr2=0x7f8198146d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:25.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819f595700 1 --2- 172.21.15.36:0/3271477125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81980b4120 0x7f8198146d80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f8188002f70 tx=0x7f8188006df0).stop 2022-06-17T12:47:25.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819f595700 1 -- 172.21.15.36:0/3271477125 shutdown_connections 2022-06-17T12:47:25.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819f595700 1 --2- 172.21.15.36:0/3271477125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81980b4120 0x7f8198146d80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:25.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819f595700 1 -- 172.21.15.36:0/3271477125 >> 172.21.15.36:0/3271477125 conn(0x7f81980620f0 msgr2=0x7f8198062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:25.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819f595700 1 -- 172.21.15.36:0/3271477125 shutdown_connections 2022-06-17T12:47:25.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819f595700 1 -- 172.21.15.36:0/3271477125 wait complete. 2022-06-17T12:47:25.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819f595700 1 Processor -- start 2022-06-17T12:47:25.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819f595700 1 -- start start 2022-06-17T12:47:25.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819f595700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81980b4120 0x7f819813f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:25.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819f595700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f81980b64f0 0x7f8198155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:25.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819f595700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8198147270 0x7f81981561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:25.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819f595700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8198152800 con 0x7f81980b64f0 2022-06-17T12:47:25.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819f595700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81980b1620 con 0x7f8198147270 2022-06-17T12:47:25.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819f595700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81980b18c0 con 0x7f81980b4120 2022-06-17T12:47:25.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819cb30700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f81980b64f0 0x7f8198155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:25.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819cb30700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f81980b64f0 0x7f8198155cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60424/0 (socket says 172.21.15.36:60424) 2022-06-17T12:47:25.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819cb30700 1 -- 172.21.15.36:0/1621835832 learned_addr learned my addr 172.21.15.36:0/1621835832 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:25.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819db32700 1 --2- 172.21.15.36:0/1621835832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8198147270 0x7f81981561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:25.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819cb30700 1 -- 172.21.15.36:0/1621835832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81980b4120 msgr2=0x7f819813f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:25.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819d331700 1 --2- 172.21.15.36:0/1621835832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81980b4120 0x7f819813f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:25.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819cb30700 1 --2- 172.21.15.36:0/1621835832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81980b4120 0x7f819813f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:25.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819cb30700 1 -- 172.21.15.36:0/1621835832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8198147270 msgr2=0x7f81981561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:25.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819cb30700 1 --2- 172.21.15.36:0/1621835832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8198147270 0x7f81981561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:25.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819cb30700 1 -- 172.21.15.36:0/1621835832 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818800b180 con 0x7f81980b64f0 2022-06-17T12:47:25.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819db32700 1 --2- 172.21.15.36:0/1621835832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8198147270 0x7f81981561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:47:25.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819d331700 1 --2- 172.21.15.36:0/1621835832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81980b4120 0x7f819813f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:47:25.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819cb30700 1 --2- 172.21.15.36:0/1621835832 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f81980b64f0 0x7f8198155cd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f8188002f70 tx=0x7f8188014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:25.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f818e7fc700 1 -- 172.21.15.36:0/1621835832 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f818800d1c0 con 0x7f81980b64f0 2022-06-17T12:47:25.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f818e7fc700 1 -- 172.21.15.36:0/1621835832 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8188013d50 con 0x7f81980b64f0 2022-06-17T12:47:25.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819f595700 1 -- 172.21.15.36:0/1621835832 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8198152560 con 0x7f81980b64f0 2022-06-17T12:47:25.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f818e7fc700 1 -- 172.21.15.36:0/1621835832 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f818801e5f0 con 0x7f81980b64f0 2022-06-17T12:47:25.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.541+0000 7f819f595700 1 -- 172.21.15.36:0/1621835832 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81981566b0 con 0x7f81980b64f0 2022-06-17T12:47:25.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.545+0000 7f818e7fc700 1 -- 172.21.15.36:0/1621835832 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8188025030 con 0x7f81980b64f0 2022-06-17T12:47:25.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.545+0000 7f818e7fc700 1 --2- 172.21.15.36:0/1621835832 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8180053940 0x7f8180055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:25.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.545+0000 7f818e7fc700 1 -- 172.21.15.36:0/1621835832 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f818806cf50 con 0x7f81980b64f0 2022-06-17T12:47:25.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.545+0000 7f819d331700 1 --2- 172.21.15.36:0/1621835832 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8180053940 0x7f8180055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:25.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.545+0000 7f819f595700 1 -- 172.21.15.36:0/1621835832 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81980b3a60 con 0x7f81980b64f0 2022-06-17T12:47:25.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.545+0000 7f819d331700 1 --2- 172.21.15.36:0/1621835832 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8180053940 0x7f8180055e00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f8194008000 tx=0x7f8194009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:25.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.553+0000 7f818e7fc700 1 -- 172.21.15.36:0/1621835832 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8188023020 con 0x7f81980b64f0 2022-06-17T12:47:25.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.685+0000 7f819f595700 1 -- 172.21.15.36:0/1621835832 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8198057410 con 0x7f81980b64f0 2022-06-17T12:47:25.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.685+0000 7f818e7fc700 1 -- 172.21.15.36:0/1621835832 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8188043c40 con 0x7f81980b64f0 2022-06-17T12:47:25.688 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:25.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.689+0000 7f819f595700 1 -- 172.21.15.36:0/1621835832 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8180053940 msgr2=0x7f8180055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:25.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.689+0000 7f819f595700 1 --2- 172.21.15.36:0/1621835832 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8180053940 0x7f8180055e00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f8194008000 tx=0x7f8194009040).stop 2022-06-17T12:47:25.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.689+0000 7f819f595700 1 -- 172.21.15.36:0/1621835832 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f81980b64f0 msgr2=0x7f8198155cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:25.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.689+0000 7f819f595700 1 --2- 172.21.15.36:0/1621835832 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f81980b64f0 0x7f8198155cd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f8188002f70 tx=0x7f8188014000).stop 2022-06-17T12:47:25.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.689+0000 7f819f595700 1 -- 172.21.15.36:0/1621835832 shutdown_connections 2022-06-17T12:47:25.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.689+0000 7f819f595700 1 --2- 172.21.15.36:0/1621835832 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8180053940 0x7f8180055e00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:25.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.689+0000 7f819f595700 1 --2- 172.21.15.36:0/1621835832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8198147270 0x7f81981561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:25.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.689+0000 7f819f595700 1 --2- 172.21.15.36:0/1621835832 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f81980b64f0 0x7f8198155cd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:25.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.689+0000 7f819f595700 1 --2- 172.21.15.36:0/1621835832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81980b4120 0x7f819813f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:25.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.689+0000 7f819f595700 1 -- 172.21.15.36:0/1621835832 >> 172.21.15.36:0/1621835832 conn(0x7f81980620f0 msgr2=0x7f81980b7680 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:25.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.689+0000 7f819f595700 1 -- 172.21.15.36:0/1621835832 shutdown_connections 2022-06-17T12:47:25.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.689+0000 7f819f595700 1 -- 172.21.15.36:0/1621835832 wait complete. 2022-06-17T12:47:25.695 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:47:25.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.821+0000 7ff78d2e1700 1 Processor -- start 2022-06-17T12:47:25.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.821+0000 7ff78d2e1700 1 -- start start 2022-06-17T12:47:25.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff78d2e1700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff788060920 0x7ff788138ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:25.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff78d2e1700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff78813fe30 0x7ff7881394e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:25.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff78d2e1700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7880af610 con 0x7ff7880b11a0 2022-06-17T12:47:25.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff78d2e1700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7880ae190 con 0x7ff78813fe30 2022-06-17T12:47:25.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff78d2e1700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7880ae9d0 con 0x7ff788060920 2022-06-17T12:47:25.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff7877fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff78813fe30 0x7ff7881394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:25.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff7877fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff78813fe30 0x7ff7881394e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60434/0 (socket says 172.21.15.36:60434) 2022-06-17T12:47:25.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff7877fe700 1 -- 172.21.15.36:0/722589857 learned_addr learned my addr 172.21.15.36:0/722589857 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:25.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff787fff700 1 --2- 172.21.15.36:0/722589857 >> v2:172.21.15.70:3300/0 conn(0x7ff788060920 0x7ff788138ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:25.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff7877fe700 1 -- 172.21.15.36:0/722589857 >> v2:172.21.15.70:3300/0 conn(0x7ff788060920 msgr2=0x7ff788138ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:25.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff7877fe700 1 --2- 172.21.15.36:0/722589857 >> v2:172.21.15.70:3300/0 conn(0x7ff788060920 0x7ff788138ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:25.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff7877fe700 1 -- 172.21.15.36:0/722589857 >> v1:172.21.15.36:6789/0 conn(0x7ff7880b11a0 legacy=0x7ff7880b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:25.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff7877fe700 1 -- 172.21.15.36:0/722589857 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7880af8b0 con 0x7ff78813fe30 2022-06-17T12:47:25.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff7877fe700 1 --2- 172.21.15.36:0/722589857 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff78813fe30 0x7ff7881394e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7ff778002f70 tx=0x7ff778006eb0).ready entity=mon.0 client_cookie=1558f3fb309090c0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:25.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff786ffd700 1 -- 172.21.15.36:0/722589857 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff77800d610 con 0x7ff78813fe30 2022-06-17T12:47:25.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff786ffd700 1 -- 172.21.15.36:0/722589857 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff778000e90 con 0x7ff78813fe30 2022-06-17T12:47:25.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff786ffd700 1 -- 172.21.15.36:0/722589857 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff77800d9a0 con 0x7ff78813fe30 2022-06-17T12:47:25.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff78d2e1700 1 -- 172.21.15.36:0/722589857 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff78813fe30 msgr2=0x7ff7881394e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:25.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff78d2e1700 1 --2- 172.21.15.36:0/722589857 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff78813fe30 0x7ff7881394e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7ff778002f70 tx=0x7ff778006eb0).stop 2022-06-17T12:47:25.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff78d2e1700 1 -- 172.21.15.36:0/722589857 shutdown_connections 2022-06-17T12:47:25.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff78d2e1700 1 --2- 172.21.15.36:0/722589857 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff78813fe30 0x7ff7881394e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:25.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff78d2e1700 1 --2- 172.21.15.36:0/722589857 >> v2:172.21.15.70:3300/0 conn(0x7ff788060920 0x7ff788138ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:25.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff78d2e1700 1 -- 172.21.15.36:0/722589857 >> 172.21.15.36:0/722589857 conn(0x7ff7880b5d10 msgr2=0x7ff7880b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:25.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff78d2e1700 1 -- 172.21.15.36:0/722589857 shutdown_connections 2022-06-17T12:47:25.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff78d2e1700 1 -- 172.21.15.36:0/722589857 wait complete. 2022-06-17T12:47:25.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff78d2e1700 1 Processor -- start 2022-06-17T12:47:25.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff78d2e1700 1 -- start start 2022-06-17T12:47:25.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff78d2e1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff788060920 0x7ff7881431e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:25.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff78d2e1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff7880b11a0 0x7ff7880648e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:25.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff78d2e1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff78813fe30 0x7ff788064dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:25.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff78d2e1700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78813bda0 con 0x7ff78813fe30 2022-06-17T12:47:25.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff78d2e1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78813c6d0 con 0x7ff788060920 2022-06-17T12:47:25.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.825+0000 7ff78d2e1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78813bfd0 con 0x7ff7880b11a0 2022-06-17T12:47:25.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.829+0000 7ff78cae0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff78813fe30 0x7ff788064dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:25.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.829+0000 7ff78cae0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff78813fe30 0x7ff788064dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60442/0 (socket says 172.21.15.36:60442) 2022-06-17T12:47:25.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.829+0000 7ff78cae0700 1 -- 172.21.15.36:0/434086637 learned_addr learned my addr 172.21.15.36:0/434086637 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:25.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.829+0000 7ff787fff700 1 --2- 172.21.15.36:0/434086637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff788060920 0x7ff7881431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:25.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.829+0000 7ff7877fe700 1 --2- 172.21.15.36:0/434086637 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff7880b11a0 0x7ff7880648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:25.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.829+0000 7ff78cae0700 1 -- 172.21.15.36:0/434086637 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff7880b11a0 msgr2=0x7ff7880648e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:25.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.829+0000 7ff78cae0700 1 --2- 172.21.15.36:0/434086637 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff7880b11a0 0x7ff7880648e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:25.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.829+0000 7ff78cae0700 1 -- 172.21.15.36:0/434086637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff788060920 msgr2=0x7ff7881431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:25.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.829+0000 7ff78cae0700 1 --2- 172.21.15.36:0/434086637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff788060920 0x7ff7881431e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:25.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.829+0000 7ff78cae0700 1 -- 172.21.15.36:0/434086637 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77800b1e0 con 0x7ff78813fe30 2022-06-17T12:47:25.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.829+0000 7ff787fff700 1 --2- 172.21.15.36:0/434086637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff788060920 0x7ff7881431e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:25.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.829+0000 7ff78cae0700 1 --2- 172.21.15.36:0/434086637 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff78813fe30 0x7ff788064dd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7ff7880b0d50 tx=0x7ff780007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:25.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.829+0000 7ff7857fa700 1 -- 172.21.15.36:0/434086637 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff78000e4a0 con 0x7ff78813fe30 2022-06-17T12:47:25.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.829+0000 7ff7857fa700 1 -- 172.21.15.36:0/434086637 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78000e600 con 0x7ff78813fe30 2022-06-17T12:47:25.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.829+0000 7ff78d2e1700 1 -- 172.21.15.36:0/434086637 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7880627d0 con 0x7ff78813fe30 2022-06-17T12:47:25.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.829+0000 7ff7857fa700 1 -- 172.21.15.36:0/434086637 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff78000e960 con 0x7ff78813fe30 2022-06-17T12:47:25.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.829+0000 7ff78d2e1700 1 -- 172.21.15.36:0/434086637 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff788062b00 con 0x7ff78813fe30 2022-06-17T12:47:25.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.829+0000 7ff78d2e1700 1 -- 172.21.15.36:0/434086637 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff78813ccd0 con 0x7ff78813fe30 2022-06-17T12:47:25.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.829+0000 7ff7857fa700 1 -- 172.21.15.36:0/434086637 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff780019030 con 0x7ff78813fe30 2022-06-17T12:47:25.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.829+0000 7ff7857fa700 1 --2- 172.21.15.36:0/434086637 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff768053a40 0x7ff768055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:26.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.829+0000 7ff7857fa700 1 -- 172.21.15.36:0/434086637 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff78006b780 con 0x7ff78813fe30 2022-06-17T12:47:26.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.829+0000 7ff787fff700 1 --2- 172.21.15.36:0/434086637 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff768053a40 0x7ff768055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:26.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.833+0000 7ff787fff700 1 --2- 172.21.15.36:0/434086637 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff768053a40 0x7ff768055f00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7ff77c00c000 tx=0x7ff77c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:26.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.841+0000 7ff7857fa700 1 -- 172.21.15.36:0/434086637 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff780042580 con 0x7ff78813fe30 2022-06-17T12:47:26.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.965+0000 7ff78d2e1700 1 -- 172.21.15.36:0/434086637 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff788057410 con 0x7ff78813fe30 2022-06-17T12:47:26.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.969+0000 7ff7857fa700 1 -- 172.21.15.36:0/434086637 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+10 (secure 0 0 0) 0x7ff780045ba0 con 0x7ff78813fe30 2022-06-17T12:47:26.034 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:26.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.969+0000 7ff78d2e1700 1 -- 172.21.15.36:0/434086637 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff768053a40 msgr2=0x7ff768055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:26.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.969+0000 7ff78d2e1700 1 --2- 172.21.15.36:0/434086637 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff768053a40 0x7ff768055f00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7ff77c00c000 tx=0x7ff77c008040).stop 2022-06-17T12:47:26.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.969+0000 7ff78d2e1700 1 -- 172.21.15.36:0/434086637 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff78813fe30 msgr2=0x7ff788064dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:26.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.969+0000 7ff78d2e1700 1 --2- 172.21.15.36:0/434086637 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff78813fe30 0x7ff788064dd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7ff7880b0d50 tx=0x7ff780007f40).stop 2022-06-17T12:47:26.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.969+0000 7ff78d2e1700 1 -- 172.21.15.36:0/434086637 shutdown_connections 2022-06-17T12:47:26.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.969+0000 7ff78d2e1700 1 --2- 172.21.15.36:0/434086637 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff768053a40 0x7ff768055f00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.969+0000 7ff78d2e1700 1 --2- 172.21.15.36:0/434086637 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff78813fe30 0x7ff788064dd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.969+0000 7ff78d2e1700 1 --2- 172.21.15.36:0/434086637 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff7880b11a0 0x7ff7880648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.969+0000 7ff78d2e1700 1 --2- 172.21.15.36:0/434086637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff788060920 0x7ff7881431e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.973+0000 7ff78d2e1700 1 -- 172.21.15.36:0/434086637 >> 172.21.15.36:0/434086637 conn(0x7ff7880b5d10 msgr2=0x7ff7880ad180 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:26.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.973+0000 7ff78d2e1700 1 -- 172.21.15.36:0/434086637 shutdown_connections 2022-06-17T12:47:26.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:25.973+0000 7ff78d2e1700 1 -- 172.21.15.36:0/434086637 wait complete. 2022-06-17T12:47:26.046 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:47:26.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.181+0000 7fccddd89700 1 Processor -- start 2022-06-17T12:47:26.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.181+0000 7fccddd89700 1 -- start start 2022-06-17T12:47:26.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccddd89700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fccd80b64f0 0x7fccd8143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:26.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccddd89700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fccd80b4120 0x7fccd8146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:26.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccddd89700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd81476e0 con 0x7fccd8147270 2022-06-17T12:47:26.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccddd89700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd80b68d0 con 0x7fccd80b64f0 2022-06-17T12:47:26.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccddd89700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd80b6e20 con 0x7fccd80b4120 2022-06-17T12:47:26.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccd77fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fccd80b64f0 0x7fccd8143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:26.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccd77fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fccd80b64f0 0x7fccd8143150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60446/0 (socket says 172.21.15.36:60446) 2022-06-17T12:47:26.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccd77fe700 1 -- 172.21.15.36:0/1075930761 learned_addr learned my addr 172.21.15.36:0/1075930761 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:26.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccd6ffd700 1 --2- 172.21.15.36:0/1075930761 >> v2:172.21.15.70:3300/0 conn(0x7fccd80b4120 0x7fccd8146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:26.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccd77fe700 1 -- 172.21.15.36:0/1075930761 >> v2:172.21.15.70:3300/0 conn(0x7fccd80b4120 msgr2=0x7fccd8146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:26.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccd77fe700 1 --2- 172.21.15.36:0/1075930761 >> v2:172.21.15.70:3300/0 conn(0x7fccd80b4120 0x7fccd8146d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccd77fe700 1 -- 172.21.15.36:0/1075930761 >> v1:172.21.15.70:6789/0 conn(0x7fccd8147270 legacy=0x7fccd8140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:26.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccd77fe700 1 -- 172.21.15.36:0/1075930761 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd81468e0 con 0x7fccd80b64f0 2022-06-17T12:47:26.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccd77fe700 1 --2- 172.21.15.36:0/1075930761 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fccd80b64f0 0x7fccd8143150 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fccc8002f70 tx=0x7fccc8006eb0).ready entity=mon.0 client_cookie=e8c57973563eb524 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:26.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccd67fc700 1 -- 172.21.15.36:0/1075930761 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccc800d610 con 0x7fccd80b64f0 2022-06-17T12:47:26.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccd67fc700 1 -- 172.21.15.36:0/1075930761 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc8000e90 con 0x7fccd80b64f0 2022-06-17T12:47:26.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccd67fc700 1 -- 172.21.15.36:0/1075930761 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccc800d9a0 con 0x7fccd80b64f0 2022-06-17T12:47:26.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccddd89700 1 -- 172.21.15.36:0/1075930761 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fccd80b64f0 msgr2=0x7fccd8143150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:26.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccddd89700 1 --2- 172.21.15.36:0/1075930761 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fccd80b64f0 0x7fccd8143150 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fccc8002f70 tx=0x7fccc8006eb0).stop 2022-06-17T12:47:26.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccddd89700 1 -- 172.21.15.36:0/1075930761 shutdown_connections 2022-06-17T12:47:26.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccddd89700 1 --2- 172.21.15.36:0/1075930761 >> v2:172.21.15.70:3300/0 conn(0x7fccd80b4120 0x7fccd8146d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccddd89700 1 --2- 172.21.15.36:0/1075930761 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fccd80b64f0 0x7fccd8143150 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccddd89700 1 -- 172.21.15.36:0/1075930761 >> 172.21.15.36:0/1075930761 conn(0x7fccd80620f0 msgr2=0x7fccd8062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:26.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccddd89700 1 -- 172.21.15.36:0/1075930761 shutdown_connections 2022-06-17T12:47:26.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.185+0000 7fccddd89700 1 -- 172.21.15.36:0/1075930761 wait complete. 2022-06-17T12:47:26.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.189+0000 7fccddd89700 1 Processor -- start 2022-06-17T12:47:26.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.189+0000 7fccddd89700 1 -- start start 2022-06-17T12:47:26.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.189+0000 7fccddd89700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fccd80b4120 0x7fccd81530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:26.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.189+0000 7fccddd89700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fccd80b64f0 0x7fccd8145790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:26.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.189+0000 7fccddd89700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccd8147270 0x7fccd80b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:26.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.189+0000 7fccddd89700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd81535a0 con 0x7fccd80b4120 2022-06-17T12:47:26.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.189+0000 7fccddd89700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd80b1b90 con 0x7fccd8147270 2022-06-17T12:47:26.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.189+0000 7fccddd89700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd80b1e30 con 0x7fccd80b64f0 2022-06-17T12:47:26.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.189+0000 7fccd77fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fccd80b4120 0x7fccd81530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:26.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.189+0000 7fccd77fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fccd80b4120 0x7fccd81530b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60452/0 (socket says 172.21.15.36:60452) 2022-06-17T12:47:26.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.189+0000 7fccd77fe700 1 -- 172.21.15.36:0/3750532487 learned_addr learned my addr 172.21.15.36:0/3750532487 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:26.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.189+0000 7fccd7fff700 1 --2- 172.21.15.36:0/3750532487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccd8147270 0x7fccd80b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:26.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.189+0000 7fccd6ffd700 1 --2- 172.21.15.36:0/3750532487 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fccd80b64f0 0x7fccd8145790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:26.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.189+0000 7fccd77fe700 1 -- 172.21.15.36:0/3750532487 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fccd80b64f0 msgr2=0x7fccd8145790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:26.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.189+0000 7fccd77fe700 1 --2- 172.21.15.36:0/3750532487 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fccd80b64f0 0x7fccd8145790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.189+0000 7fccd77fe700 1 -- 172.21.15.36:0/3750532487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccd8147270 msgr2=0x7fccd80b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:26.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.189+0000 7fccd77fe700 1 --2- 172.21.15.36:0/3750532487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccd8147270 0x7fccd80b0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.189+0000 7fccd77fe700 1 -- 172.21.15.36:0/3750532487 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc800b1e0 con 0x7fccd80b4120 2022-06-17T12:47:26.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.189+0000 7fccd77fe700 1 --2- 172.21.15.36:0/3750532487 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fccd80b4120 0x7fccd81530b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fccc8002f70 tx=0x7fccc8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:26.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.189+0000 7fccd4ff9700 1 -- 172.21.15.36:0/3750532487 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccc8011860 con 0x7fccd80b4120 2022-06-17T12:47:26.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.189+0000 7fccd4ff9700 1 -- 172.21.15.36:0/3750532487 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc80119c0 con 0x7fccd80b4120 2022-06-17T12:47:26.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.189+0000 7fccddd89700 1 -- 172.21.15.36:0/3750532487 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd8145c80 con 0x7fccd80b4120 2022-06-17T12:47:26.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.189+0000 7fccd4ff9700 1 -- 172.21.15.36:0/3750532487 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccc8011d20 con 0x7fccd80b4120 2022-06-17T12:47:26.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.189+0000 7fccddd89700 1 -- 172.21.15.36:0/3750532487 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd80b04e0 con 0x7fccd80b4120 2022-06-17T12:47:26.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.193+0000 7fccddd89700 1 -- 172.21.15.36:0/3750532487 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd8140610 con 0x7fccd80b4120 2022-06-17T12:47:26.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.193+0000 7fccd4ff9700 1 -- 172.21.15.36:0/3750532487 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fccc801a040 con 0x7fccd80b4120 2022-06-17T12:47:26.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.193+0000 7fccd4ff9700 1 --2- 172.21.15.36:0/3750532487 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fccb8053ad0 0x7fccb8055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:26.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.193+0000 7fccd4ff9700 1 -- 172.21.15.36:0/3750532487 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fccc806cf50 con 0x7fccd80b4120 2022-06-17T12:47:26.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.193+0000 7fccd6ffd700 1 --2- 172.21.15.36:0/3750532487 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fccb8053ad0 0x7fccb8055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:26.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.193+0000 7fccd6ffd700 1 --2- 172.21.15.36:0/3750532487 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fccb8053ad0 0x7fccb8055f90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fcccc00c000 tx=0x7fcccc008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:26.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.197+0000 7fccd4ff9700 1 -- 172.21.15.36:0/3750532487 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fccd8140610 con 0x7fccd80b4120 2022-06-17T12:47:26.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.325+0000 7fccddd89700 1 -- 172.21.15.36:0/3750532487 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fccd8140610 con 0x7fccd80b4120 2022-06-17T12:47:26.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.329+0000 7fccd4ff9700 1 -- 172.21.15.36:0/3750532487 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+10 (secure 0 0 0) 0x7fccd8140610 con 0x7fccd80b4120 2022-06-17T12:47:26.316 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:26.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.329+0000 7fccddd89700 1 -- 172.21.15.36:0/3750532487 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fccb8053ad0 msgr2=0x7fccb8055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:26.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.329+0000 7fccddd89700 1 --2- 172.21.15.36:0/3750532487 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fccb8053ad0 0x7fccb8055f90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fcccc00c000 tx=0x7fcccc008040).stop 2022-06-17T12:47:26.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.329+0000 7fccddd89700 1 -- 172.21.15.36:0/3750532487 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fccd80b4120 msgr2=0x7fccd81530b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:26.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.329+0000 7fccddd89700 1 --2- 172.21.15.36:0/3750532487 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fccd80b4120 0x7fccd81530b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fccc8002f70 tx=0x7fccc8015000).stop 2022-06-17T12:47:26.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.329+0000 7fccddd89700 1 -- 172.21.15.36:0/3750532487 shutdown_connections 2022-06-17T12:47:26.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.329+0000 7fccddd89700 1 --2- 172.21.15.36:0/3750532487 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fccb8053ad0 0x7fccb8055f90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.329+0000 7fccddd89700 1 --2- 172.21.15.36:0/3750532487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccd8147270 0x7fccd80b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.329+0000 7fccddd89700 1 --2- 172.21.15.36:0/3750532487 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fccd80b64f0 0x7fccd8145790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.329+0000 7fccddd89700 1 --2- 172.21.15.36:0/3750532487 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fccd80b4120 0x7fccd81530b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.329+0000 7fccddd89700 1 -- 172.21.15.36:0/3750532487 >> 172.21.15.36:0/3750532487 conn(0x7fccd80620f0 msgr2=0x7fccd813f410 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:26.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.329+0000 7fccddd89700 1 -- 172.21.15.36:0/3750532487 shutdown_connections 2022-06-17T12:47:26.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.329+0000 7fccddd89700 1 -- 172.21.15.36:0/3750532487 wait complete. 2022-06-17T12:47:26.382 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:47:26.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.513+0000 7f4fe0d86700 1 Processor -- start 2022-06-17T12:47:26.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.513+0000 7f4fe0d86700 1 -- start start 2022-06-17T12:47:26.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.513+0000 7f4fe0d86700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fdc063730 0x7f4fdc0b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:26.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.513+0000 7f4fe0d86700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4fdc0644e0 0x7f4fdc064940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:26.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.513+0000 7f4fe0d86700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fdc060390 con 0x7f4fdc064e30 2022-06-17T12:47:26.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.513+0000 7f4fe0d86700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fdc065280 con 0x7f4fdc0644e0 2022-06-17T12:47:26.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.513+0000 7f4fe0d86700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc0b2230 con 0x7f4fdc063730 2022-06-17T12:47:26.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.513+0000 7f4fdaffd700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4fdc0644e0 0x7f4fdc064940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:26.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.513+0000 7f4fdaffd700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4fdc0644e0 0x7f4fdc064940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:52610/0 (socket says 172.21.15.36:52610) 2022-06-17T12:47:26.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.513+0000 7f4fdaffd700 1 -- 172.21.15.36:0/920591595 learned_addr learned my addr 172.21.15.36:0/920591595 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:26.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.513+0000 7f4fdbfff700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4fdc064e30 0x7f4fdc0b63a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:45496/0 (socket says 172.21.15.36:45496) 2022-06-17T12:47:26.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.513+0000 7f4fdb7fe700 1 --2- 172.21.15.36:0/920591595 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fdc063730 0x7f4fdc0b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:26.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.513+0000 7f4fdaffd700 1 -- 172.21.15.36:0/920591595 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fdc063730 msgr2=0x7f4fdc0b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:26.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.513+0000 7f4fdaffd700 1 --2- 172.21.15.36:0/920591595 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fdc063730 0x7f4fdc0b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.513+0000 7f4fdaffd700 1 -- 172.21.15.36:0/920591595 >> v1:172.21.15.70:6789/0 conn(0x7f4fdc064e30 legacy=0x7f4fdc0b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:26.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.513+0000 7f4fdaffd700 1 -- 172.21.15.36:0/920591595 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fdc060130 con 0x7f4fdc0644e0 2022-06-17T12:47:26.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.513+0000 7f4fdaffd700 1 --2- 172.21.15.36:0/920591595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fdc0644e0 0x7f4fdc064940 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f4fd40032f0 tx=0x7f4fd4009290).ready entity=mon.1 client_cookie=c54f51f61fe1edfd server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:26.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.517+0000 7f4fda7fc700 1 -- 172.21.15.36:0/920591595 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fd400b8a0 con 0x7f4fdc0644e0 2022-06-17T12:47:26.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.517+0000 7f4fda7fc700 1 -- 172.21.15.36:0/920591595 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd400ba00 con 0x7f4fdc0644e0 2022-06-17T12:47:26.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.517+0000 7f4fda7fc700 1 -- 172.21.15.36:0/920591595 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fd400bb70 con 0x7f4fdc0644e0 2022-06-17T12:47:26.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.517+0000 7f4fe0d86700 1 -- 172.21.15.36:0/920591595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fdc0644e0 msgr2=0x7f4fdc064940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:26.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.517+0000 7f4fe0d86700 1 --2- 172.21.15.36:0/920591595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fdc0644e0 0x7f4fdc064940 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f4fd40032f0 tx=0x7f4fd4009290).stop 2022-06-17T12:47:26.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.517+0000 7f4fe0d86700 1 -- 172.21.15.36:0/920591595 shutdown_connections 2022-06-17T12:47:26.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.517+0000 7f4fe0d86700 1 --2- 172.21.15.36:0/920591595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fdc0644e0 0x7f4fdc064940 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.517+0000 7f4fe0d86700 1 --2- 172.21.15.36:0/920591595 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fdc063730 0x7f4fdc0b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.517+0000 7f4fe0d86700 1 -- 172.21.15.36:0/920591595 >> 172.21.15.36:0/920591595 conn(0x7f4fdc0614b0 msgr2=0x7f4fdc0618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:26.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.517+0000 7f4fe0d86700 1 -- 172.21.15.36:0/920591595 shutdown_connections 2022-06-17T12:47:26.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.517+0000 7f4fe0d86700 1 -- 172.21.15.36:0/920591595 wait complete. 2022-06-17T12:47:26.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.517+0000 7f4fe0d86700 1 Processor -- start 2022-06-17T12:47:26.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.517+0000 7f4fe0d86700 1 -- start start 2022-06-17T12:47:26.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.521+0000 7f4fe0d86700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fdc063730 0x7f4fdc143740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:26.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.521+0000 7f4fe0d86700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fdc0644e0 0x7f4fdc0b6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:26.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.521+0000 7f4fe0d86700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fdc064e30 0x7f4fdc152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:26.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.521+0000 7f4fe0d86700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc153580 con 0x7f4fdc064e30 2022-06-17T12:47:26.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.521+0000 7f4fe0d86700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc0b5430 con 0x7f4fdc0644e0 2022-06-17T12:47:26.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.521+0000 7f4fe0d86700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc0b6740 con 0x7f4fdc063730 2022-06-17T12:47:26.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.521+0000 7f4fdbfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fdc064e30 0x7f4fdc152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:26.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.521+0000 7f4fdbfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fdc064e30 0x7f4fdc152310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60470/0 (socket says 172.21.15.36:60470) 2022-06-17T12:47:26.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.521+0000 7f4fdbfff700 1 -- 172.21.15.36:0/2994174641 learned_addr learned my addr 172.21.15.36:0/2994174641 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:26.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.521+0000 7f4fdaffd700 1 --2- 172.21.15.36:0/2994174641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fdc0644e0 0x7f4fdc0b6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:26.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.521+0000 7f4fdb7fe700 1 --2- 172.21.15.36:0/2994174641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fdc063730 0x7f4fdc143740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:26.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.521+0000 7f4fdbfff700 1 -- 172.21.15.36:0/2994174641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fdc063730 msgr2=0x7f4fdc143740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:26.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.521+0000 7f4fdbfff700 1 --2- 172.21.15.36:0/2994174641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fdc063730 0x7f4fdc143740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.521+0000 7f4fdbfff700 1 -- 172.21.15.36:0/2994174641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fdc0644e0 msgr2=0x7f4fdc0b6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:26.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.521+0000 7f4fdbfff700 1 --2- 172.21.15.36:0/2994174641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fdc0644e0 0x7f4fdc0b6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.521+0000 7f4fdbfff700 1 -- 172.21.15.36:0/2994174641 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd4009530 con 0x7f4fdc064e30 2022-06-17T12:47:26.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.521+0000 7f4fdbfff700 1 --2- 172.21.15.36:0/2994174641 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fdc064e30 0x7f4fdc152310 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f4fdc063e30 tx=0x7f4fc4007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:26.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.521+0000 7f4fd8ff9700 1 -- 172.21.15.36:0/2994174641 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fc40070d0 con 0x7f4fdc064e30 2022-06-17T12:47:26.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.521+0000 7f4fd8ff9700 1 -- 172.21.15.36:0/2994174641 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc4007230 con 0x7f4fdc064e30 2022-06-17T12:47:26.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.521+0000 7f4fe0d86700 1 -- 172.21.15.36:0/2994174641 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fdc0b4c50 con 0x7f4fdc064e30 2022-06-17T12:47:26.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.521+0000 7f4fd8ff9700 1 -- 172.21.15.36:0/2994174641 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fc4005550 con 0x7f4fdc064e30 2022-06-17T12:47:26.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.521+0000 7f4fe0d86700 1 -- 172.21.15.36:0/2994174641 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fdc0b7020 con 0x7f4fdc064e30 2022-06-17T12:47:26.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.521+0000 7f4fe0d86700 1 -- 172.21.15.36:0/2994174641 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fdc0b2e10 con 0x7f4fdc064e30 2022-06-17T12:47:26.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.525+0000 7f4fd8ff9700 1 -- 172.21.15.36:0/2994174641 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f4fc4010070 con 0x7f4fdc064e30 2022-06-17T12:47:26.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.525+0000 7f4fd8ff9700 1 --2- 172.21.15.36:0/2994174641 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4fc0053af0 0x7f4fc0055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:26.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.525+0000 7f4fd8ff9700 1 -- 172.21.15.36:0/2994174641 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f4fc406b640 con 0x7f4fdc064e30 2022-06-17T12:47:26.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.525+0000 7f4fdb7fe700 1 --2- 172.21.15.36:0/2994174641 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4fc0053af0 0x7f4fc0055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:26.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.525+0000 7f4fdb7fe700 1 --2- 172.21.15.36:0/2994174641 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4fc0053af0 0x7f4fc0055fb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f4fcc00b000 tx=0x7f4fcc006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:26.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.529+0000 7f4fd8ff9700 1 -- 172.21.15.36:0/2994174641 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4fc40423a0 con 0x7f4fdc064e30 2022-06-17T12:47:26.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.653+0000 7f4fe0d86700 1 -- 172.21.15.36:0/2994174641 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fdc0b4de0 con 0x7f4fdc064e30 2022-06-17T12:47:26.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.657+0000 7f4fd8ff9700 1 -- 172.21.15.36:0/2994174641 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f4fc4018020 con 0x7f4fdc064e30 2022-06-17T12:47:26.687 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:26.688 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":26,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:46:53.258075+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:46:47.465328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-17T12:46:49.893338+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:47:26.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.657+0000 7f4fe0d86700 1 -- 172.21.15.36:0/2994174641 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4fc0053af0 msgr2=0x7f4fc0055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:26.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.657+0000 7f4fe0d86700 1 --2- 172.21.15.36:0/2994174641 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4fc0053af0 0x7f4fc0055fb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f4fcc00b000 tx=0x7f4fcc006df0).stop 2022-06-17T12:47:26.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.657+0000 7f4fe0d86700 1 -- 172.21.15.36:0/2994174641 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fdc064e30 msgr2=0x7f4fdc152310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:26.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.657+0000 7f4fe0d86700 1 --2- 172.21.15.36:0/2994174641 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fdc064e30 0x7f4fdc152310 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f4fdc063e30 tx=0x7f4fc4007c20).stop 2022-06-17T12:47:26.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.657+0000 7f4fe0d86700 1 -- 172.21.15.36:0/2994174641 shutdown_connections 2022-06-17T12:47:26.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.657+0000 7f4fe0d86700 1 --2- 172.21.15.36:0/2994174641 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4fc0053af0 0x7f4fc0055fb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.657+0000 7f4fe0d86700 1 --2- 172.21.15.36:0/2994174641 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fdc064e30 0x7f4fdc152310 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.657+0000 7f4fe0d86700 1 --2- 172.21.15.36:0/2994174641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fdc0644e0 0x7f4fdc0b6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.657+0000 7f4fe0d86700 1 --2- 172.21.15.36:0/2994174641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fdc063730 0x7f4fdc143740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.657+0000 7f4fe0d86700 1 -- 172.21.15.36:0/2994174641 >> 172.21.15.36:0/2994174641 conn(0x7f4fdc0614b0 msgr2=0x7f4fdc05bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:26.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.657+0000 7f4fe0d86700 1 -- 172.21.15.36:0/2994174641 shutdown_connections 2022-06-17T12:47:26.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.657+0000 7f4fe0d86700 1 -- 172.21.15.36:0/2994174641 wait complete. 2022-06-17T12:47:26.700 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:46:47.465328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:46:49.893338+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:47:26.701 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:47:26.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.841+0000 7f204280f700 1 Processor -- start 2022-06-17T12:47:26.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.841+0000 7f204280f700 1 -- start start 2022-06-17T12:47:26.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.841+0000 7f204280f700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f203c0b65f0 0x7f203c0b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:26.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.841+0000 7f204280f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f203c0adf10 0x7f203c140530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:26.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.841+0000 7f204280f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f203c0b15b0 0x7f203c0af630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:26.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.841+0000 7f204280f700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f203c0ae8e0 con 0x7f203c0b15b0 2022-06-17T12:47:26.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.841+0000 7f204280f700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f203c13fdf0 con 0x7f203c0b65f0 2022-06-17T12:47:26.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.841+0000 7f204280f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203c140090 con 0x7f203c0adf10 2022-06-17T12:47:26.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.841+0000 7f204200e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f203c0b15b0 0x7f203c0af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:26.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.841+0000 7f204200e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f203c0b15b0 0x7f203c0af630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60478/0 (socket says 172.21.15.36:60478) 2022-06-17T12:47:26.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.841+0000 7f204200e700 1 -- 172.21.15.36:0/2065896080 learned_addr learned my addr 172.21.15.36:0/2065896080 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:26.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.841+0000 7f204180d700 1 --2- 172.21.15.36:0/2065896080 >> v2:172.21.15.70:3300/0 conn(0x7f203c0b65f0 0x7f203c0b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:26.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.841+0000 7f204100c700 1 --2- 172.21.15.36:0/2065896080 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f203c0adf10 0x7f203c140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:26.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.841+0000 7f204200e700 1 -- 172.21.15.36:0/2065896080 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f203c0adf10 msgr2=0x7f203c140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:26.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.841+0000 7f204200e700 1 --2- 172.21.15.36:0/2065896080 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f203c0adf10 0x7f203c140530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.841+0000 7f204200e700 1 -- 172.21.15.36:0/2065896080 >> v2:172.21.15.70:3300/0 conn(0x7f203c0b65f0 msgr2=0x7f203c0b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:26.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.841+0000 7f204200e700 1 --2- 172.21.15.36:0/2065896080 >> v2:172.21.15.70:3300/0 conn(0x7f203c0b65f0 0x7f203c0b69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.841+0000 7f204200e700 1 -- 172.21.15.36:0/2065896080 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c140a20 con 0x7f203c0b15b0 2022-06-17T12:47:26.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.841+0000 7f204200e700 1 --2- 172.21.15.36:0/2065896080 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f203c0b15b0 0x7f203c0af630 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f20340032f0 tx=0x7f2034009290).ready entity=mon.0 client_cookie=d88e5a7b7176c8d2 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:26.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.841+0000 7f204080b700 1 -- 172.21.15.36:0/2065896080 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f203400b870 con 0x7f203c0b15b0 2022-06-17T12:47:26.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.841+0000 7f204080b700 1 -- 172.21.15.36:0/2065896080 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203400b9d0 con 0x7f203c0b15b0 2022-06-17T12:47:26.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.841+0000 7f204080b700 1 -- 172.21.15.36:0/2065896080 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f203400bb40 con 0x7f203c0b15b0 2022-06-17T12:47:26.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.845+0000 7f204280f700 1 -- 172.21.15.36:0/2065896080 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f203c0b15b0 msgr2=0x7f203c0af630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:26.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.845+0000 7f204280f700 1 --2- 172.21.15.36:0/2065896080 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f203c0b15b0 0x7f203c0af630 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f20340032f0 tx=0x7f2034009290).stop 2022-06-17T12:47:26.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.845+0000 7f204280f700 1 -- 172.21.15.36:0/2065896080 shutdown_connections 2022-06-17T12:47:26.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.845+0000 7f204280f700 1 --2- 172.21.15.36:0/2065896080 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f203c0b15b0 0x7f203c0af630 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.845+0000 7f204280f700 1 --2- 172.21.15.36:0/2065896080 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f203c0adf10 0x7f203c140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.845+0000 7f204280f700 1 --2- 172.21.15.36:0/2065896080 >> v2:172.21.15.70:3300/0 conn(0x7f203c0b65f0 0x7f203c0b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.845+0000 7f204280f700 1 -- 172.21.15.36:0/2065896080 >> 172.21.15.36:0/2065896080 conn(0x7f203c0614b0 msgr2=0x7f203c061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:26.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.845+0000 7f204280f700 1 -- 172.21.15.36:0/2065896080 shutdown_connections 2022-06-17T12:47:26.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.845+0000 7f204280f700 1 -- 172.21.15.36:0/2065896080 wait complete. 2022-06-17T12:47:26.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.845+0000 7f204280f700 1 Processor -- start 2022-06-17T12:47:26.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.845+0000 7f204280f700 1 -- start start 2022-06-17T12:47:26.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.849+0000 7f204280f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f203c0adf10 0x7f203c146890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:26.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.849+0000 7f204280f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f203c0b15b0 0x7f203c13dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:26.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.849+0000 7f204280f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f203c0b65f0 0x7f203c13e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:26.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.849+0000 7f204280f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c13e740 con 0x7f203c0b65f0 2022-06-17T12:47:26.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.849+0000 7f204280f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c0651d0 con 0x7f203c0adf10 2022-06-17T12:47:26.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.849+0000 7f204280f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203c1523d0 con 0x7f203c0b15b0 2022-06-17T12:47:26.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.849+0000 7f204200e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f203c0b65f0 0x7f203c13e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:26.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.849+0000 7f204200e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f203c0b65f0 0x7f203c13e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60484/0 (socket says 172.21.15.36:60484) 2022-06-17T12:47:26.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.849+0000 7f204200e700 1 -- 172.21.15.36:0/3899349933 learned_addr learned my addr 172.21.15.36:0/3899349933 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:26.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.849+0000 7f204180d700 1 --2- 172.21.15.36:0/3899349933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f203c0adf10 0x7f203c146890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:26.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.849+0000 7f204100c700 1 --2- 172.21.15.36:0/3899349933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f203c0b15b0 0x7f203c13dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:26.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.849+0000 7f204200e700 1 -- 172.21.15.36:0/3899349933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f203c0b15b0 msgr2=0x7f203c13dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:26.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.849+0000 7f204200e700 1 --2- 172.21.15.36:0/3899349933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f203c0b15b0 0x7f203c13dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.849+0000 7f204200e700 1 -- 172.21.15.36:0/3899349933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f203c0adf10 msgr2=0x7f203c146890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:26.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.849+0000 7f204200e700 1 --2- 172.21.15.36:0/3899349933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f203c0adf10 0x7f203c146890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.849+0000 7f204200e700 1 -- 172.21.15.36:0/3899349933 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2034009530 con 0x7f203c0b65f0 2022-06-17T12:47:26.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.849+0000 7f204200e700 1 --2- 172.21.15.36:0/3899349933 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f203c0b65f0 0x7f203c13e250 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f20340032f0 tx=0x7f2034016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:26.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.849+0000 7f202affd700 1 -- 172.21.15.36:0/3899349933 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f203400b870 con 0x7f203c0b65f0 2022-06-17T12:47:26.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.849+0000 7f202affd700 1 -- 172.21.15.36:0/3899349933 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203401e400 con 0x7f203c0b65f0 2022-06-17T12:47:26.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.849+0000 7f204280f700 1 -- 172.21.15.36:0/3899349933 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203c147170 con 0x7f203c0b65f0 2022-06-17T12:47:26.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.849+0000 7f202affd700 1 -- 172.21.15.36:0/3899349933 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f203401e760 con 0x7f203c0b65f0 2022-06-17T12:47:26.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.849+0000 7f204280f700 1 -- 172.21.15.36:0/3899349933 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203c064ea0 con 0x7f203c0b65f0 2022-06-17T12:47:26.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.849+0000 7f204280f700 1 -- 172.21.15.36:0/3899349933 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f203c0653d0 con 0x7f203c0b65f0 2022-06-17T12:47:26.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.853+0000 7f202affd700 1 -- 172.21.15.36:0/3899349933 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f203400f070 con 0x7f203c0b65f0 2022-06-17T12:47:26.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.853+0000 7f202affd700 1 --2- 172.21.15.36:0/3899349933 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f201c053a40 0x7f201c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:26.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.853+0000 7f202affd700 1 -- 172.21.15.36:0/3899349933 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f203406f6d0 con 0x7f203c0b65f0 2022-06-17T12:47:26.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.853+0000 7f204180d700 1 --2- 172.21.15.36:0/3899349933 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f201c053a40 0x7f201c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:26.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.853+0000 7f204180d700 1 --2- 172.21.15.36:0/3899349933 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f201c053a40 0x7f201c055f00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f203800b000 tx=0x7f2038006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:26.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.857+0000 7f202affd700 1 -- 172.21.15.36:0/3899349933 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f20340464d0 con 0x7f203c0b65f0 2022-06-17T12:47:26.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.985+0000 7f204280f700 1 -- 172.21.15.36:0/3899349933 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f203c066e00 con 0x7f203c0b65f0 2022-06-17T12:47:26.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.985+0000 7f202affd700 1 -- 172.21.15.36:0/3899349933 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2034022020 con 0x7f203c0b65f0 2022-06-17T12:47:26.974 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:47:26.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.989+0000 7f204280f700 1 -- 172.21.15.36:0/3899349933 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f201c053a40 msgr2=0x7f201c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:26.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.989+0000 7f204280f700 1 --2- 172.21.15.36:0/3899349933 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f201c053a40 0x7f201c055f00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f203800b000 tx=0x7f2038006df0).stop 2022-06-17T12:47:26.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.989+0000 7f204280f700 1 -- 172.21.15.36:0/3899349933 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f203c0b65f0 msgr2=0x7f203c13e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:26.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.989+0000 7f204280f700 1 --2- 172.21.15.36:0/3899349933 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f203c0b65f0 0x7f203c13e250 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f20340032f0 tx=0x7f2034016000).stop 2022-06-17T12:47:26.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.989+0000 7f204280f700 1 -- 172.21.15.36:0/3899349933 shutdown_connections 2022-06-17T12:47:26.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.989+0000 7f204280f700 1 --2- 172.21.15.36:0/3899349933 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f201c053a40 0x7f201c055f00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.989+0000 7f204280f700 1 --2- 172.21.15.36:0/3899349933 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f203c0b65f0 0x7f203c13e250 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.989+0000 7f204280f700 1 --2- 172.21.15.36:0/3899349933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f203c0b15b0 0x7f203c13dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.989+0000 7f204280f700 1 --2- 172.21.15.36:0/3899349933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f203c0adf10 0x7f203c146890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:26.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.989+0000 7f204280f700 1 -- 172.21.15.36:0/3899349933 >> 172.21.15.36:0/3899349933 conn(0x7f203c0614b0 msgr2=0x7f203c0b52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:26.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.989+0000 7f204280f700 1 -- 172.21.15.36:0/3899349933 shutdown_connections 2022-06-17T12:47:26.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:26.989+0000 7f204280f700 1 -- 172.21.15.36:0/3899349933 wait complete. 2022-06-17T12:47:27.006 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:47:27.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.133+0000 7f2c6bf82700 1 Processor -- start 2022-06-17T12:47:27.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c6bf82700 1 -- start start 2022-06-17T12:47:27.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c6bf82700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2c640b2df0 0x7f2c640603b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:27.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c6bf82700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2c64141690 0x7f2c64145040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:27.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c6bf82700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c64145530 con 0x7f2c6405faa0 2022-06-17T12:47:27.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c6bf82700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c640608a0 con 0x7f2c64141690 2022-06-17T12:47:27.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c6bf82700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c641446a0 con 0x7f2c640b2df0 2022-06-17T12:47:27.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c6a51f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2c64141690 0x7f2c64145040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:27.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c6a51f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2c64141690 0x7f2c64145040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60492/0 (socket says 172.21.15.36:60492) 2022-06-17T12:47:27.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c6a51f700 1 -- 172.21.15.36:0/2684284547 learned_addr learned my addr 172.21.15.36:0/2684284547 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:27.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c69d1e700 1 --2- 172.21.15.36:0/2684284547 >> v2:172.21.15.70:3300/0 conn(0x7f2c640b2df0 0x7f2c640603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:27.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c6a51f700 1 -- 172.21.15.36:0/2684284547 >> v2:172.21.15.70:3300/0 conn(0x7f2c640b2df0 msgr2=0x7f2c640603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:27.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c6a51f700 1 --2- 172.21.15.36:0/2684284547 >> v2:172.21.15.70:3300/0 conn(0x7f2c640b2df0 0x7f2c640603b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:27.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c6a51f700 1 -- 172.21.15.36:0/2684284547 >> v1:172.21.15.70:6789/0 conn(0x7f2c6405faa0 legacy=0x7f2c64140f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:27.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c6a51f700 1 -- 172.21.15.36:0/2684284547 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c640b31d0 con 0x7f2c64141690 2022-06-17T12:47:27.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c6a51f700 1 --2- 172.21.15.36:0/2684284547 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2c64141690 0x7f2c64145040 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f2c5c0032f0 tx=0x7f2c5c009290).ready entity=mon.0 client_cookie=94d1dc09e8451d16 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:27.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c68d1c700 1 -- 172.21.15.36:0/2684284547 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c5c00b870 con 0x7f2c64141690 2022-06-17T12:47:27.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c68d1c700 1 -- 172.21.15.36:0/2684284547 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5c00b9d0 con 0x7f2c64141690 2022-06-17T12:47:27.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c68d1c700 1 -- 172.21.15.36:0/2684284547 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c5c00bb40 con 0x7f2c64141690 2022-06-17T12:47:27.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c6bf82700 1 -- 172.21.15.36:0/2684284547 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2c64141690 msgr2=0x7f2c64145040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:27.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c6bf82700 1 --2- 172.21.15.36:0/2684284547 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2c64141690 0x7f2c64145040 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f2c5c0032f0 tx=0x7f2c5c009290).stop 2022-06-17T12:47:27.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c6bf82700 1 -- 172.21.15.36:0/2684284547 shutdown_connections 2022-06-17T12:47:27.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c6bf82700 1 --2- 172.21.15.36:0/2684284547 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2c64141690 0x7f2c64145040 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:27.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c6bf82700 1 --2- 172.21.15.36:0/2684284547 >> v2:172.21.15.70:3300/0 conn(0x7f2c640b2df0 0x7f2c640603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:27.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c6bf82700 1 -- 172.21.15.36:0/2684284547 >> 172.21.15.36:0/2684284547 conn(0x7f2c64063020 msgr2=0x7f2c64063430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:27.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c6bf82700 1 -- 172.21.15.36:0/2684284547 shutdown_connections 2022-06-17T12:47:27.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.137+0000 7f2c6bf82700 1 -- 172.21.15.36:0/2684284547 wait complete. 2022-06-17T12:47:27.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c6bf82700 1 Processor -- start 2022-06-17T12:47:27.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c6bf82700 1 -- start start 2022-06-17T12:47:27.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c6bf82700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c6405faa0 0x7f2c641435c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:27.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c6bf82700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2c640b2df0 0x7f2c6413db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:27.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c6bf82700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c64141690 0x7f2c6413e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:27.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c6bf82700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c640a9f10 con 0x7f2c640b2df0 2022-06-17T12:47:27.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c6bf82700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c640ad8e0 con 0x7f2c6405faa0 2022-06-17T12:47:27.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c6bf82700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6413f0e0 con 0x7f2c64141690 2022-06-17T12:47:27.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c69d1e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c6405faa0 0x7f2c641435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:27.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c6a51f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c64141690 0x7f2c6413e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:27.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c6951d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2c640b2df0 0x7f2c6413db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:27.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c69d1e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c6405faa0 0x7f2c641435c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:52644/0 (socket says 172.21.15.36:52644) 2022-06-17T12:47:27.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c69d1e700 1 -- 172.21.15.36:0/1004660797 learned_addr learned my addr 172.21.15.36:0/1004660797 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:27.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c6a51f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c64141690 0x7f2c6413e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:47054/0 (socket says 172.21.15.36:47054) 2022-06-17T12:47:27.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c6951d700 1 -- 172.21.15.36:0/1004660797 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c64141690 msgr2=0x7f2c6413e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:27.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c6951d700 1 --2- 172.21.15.36:0/1004660797 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c64141690 0x7f2c6413e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:27.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c6951d700 1 -- 172.21.15.36:0/1004660797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c6405faa0 msgr2=0x7f2c641435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:27.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c6951d700 1 --2- 172.21.15.36:0/1004660797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c6405faa0 0x7f2c641435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:27.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c6951d700 1 -- 172.21.15.36:0/1004660797 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c009530 con 0x7f2c640b2df0 2022-06-17T12:47:27.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c6951d700 1 --2- 172.21.15.36:0/1004660797 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2c640b2df0 0x7f2c6413db40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f2c64148920 tx=0x7f2c60007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:27.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c56ffd700 1 -- 172.21.15.36:0/1004660797 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c60007030 con 0x7f2c640b2df0 2022-06-17T12:47:27.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c56ffd700 1 -- 172.21.15.36:0/1004660797 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c60007190 con 0x7f2c640b2df0 2022-06-17T12:47:27.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c6bf82700 1 -- 172.21.15.36:0/1004660797 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c6413f7b0 con 0x7f2c640b2df0 2022-06-17T12:47:27.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c56ffd700 1 -- 172.21.15.36:0/1004660797 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c60006510 con 0x7f2c640b2df0 2022-06-17T12:47:27.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c6bf82700 1 -- 172.21.15.36:0/1004660797 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c6413f310 con 0x7f2c640b2df0 2022-06-17T12:47:27.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.141+0000 7f2c6bf82700 1 -- 172.21.15.36:0/1004660797 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c6413f4d0 con 0x7f2c640b2df0 2022-06-17T12:47:27.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.145+0000 7f2c56ffd700 1 -- 172.21.15.36:0/1004660797 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2c60010030 con 0x7f2c640b2df0 2022-06-17T12:47:27.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.145+0000 7f2c56ffd700 1 --2- 172.21.15.36:0/1004660797 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2c44053a40 0x7f2c44055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:27.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.145+0000 7f2c56ffd700 1 -- 172.21.15.36:0/1004660797 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f2c6006b8b0 con 0x7f2c640b2df0 2022-06-17T12:47:27.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.145+0000 7f2c69d1e700 1 --2- 172.21.15.36:0/1004660797 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2c44053a40 0x7f2c44055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:27.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.145+0000 7f2c69d1e700 1 --2- 172.21.15.36:0/1004660797 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2c44053a40 0x7f2c44055f00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f2c58008000 tx=0x7f2c58008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:27.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.149+0000 7f2c56ffd700 1 -- 172.21.15.36:0/1004660797 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2c600426b0 con 0x7f2c640b2df0 2022-06-17T12:47:27.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.277+0000 7f2c6bf82700 1 -- 172.21.15.36:0/1004660797 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2c64145530 con 0x7f2c640b2df0 2022-06-17T12:47:27.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.277+0000 7f2c56ffd700 1 -- 172.21.15.36:0/1004660797 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2c64145530 con 0x7f2c640b2df0 2022-06-17T12:47:27.321 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:27.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.277+0000 7f2c6bf82700 1 -- 172.21.15.36:0/1004660797 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2c44053a40 msgr2=0x7f2c44055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:27.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.277+0000 7f2c6bf82700 1 --2- 172.21.15.36:0/1004660797 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2c44053a40 0x7f2c44055f00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f2c58008000 tx=0x7f2c58008040).stop 2022-06-17T12:47:27.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.277+0000 7f2c6bf82700 1 -- 172.21.15.36:0/1004660797 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2c640b2df0 msgr2=0x7f2c6413db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:27.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.277+0000 7f2c6bf82700 1 --2- 172.21.15.36:0/1004660797 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2c640b2df0 0x7f2c6413db40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f2c64148920 tx=0x7f2c60007b50).stop 2022-06-17T12:47:27.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.277+0000 7f2c6bf82700 1 -- 172.21.15.36:0/1004660797 shutdown_connections 2022-06-17T12:47:27.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.277+0000 7f2c6bf82700 1 --2- 172.21.15.36:0/1004660797 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2c44053a40 0x7f2c44055f00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:27.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.277+0000 7f2c6bf82700 1 --2- 172.21.15.36:0/1004660797 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c64141690 0x7f2c6413e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:27.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.277+0000 7f2c6bf82700 1 --2- 172.21.15.36:0/1004660797 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2c640b2df0 0x7f2c6413db40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:27.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.277+0000 7f2c6bf82700 1 --2- 172.21.15.36:0/1004660797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c6405faa0 0x7f2c641435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:27.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.277+0000 7f2c6bf82700 1 -- 172.21.15.36:0/1004660797 >> 172.21.15.36:0/1004660797 conn(0x7f2c64063020 msgr2=0x7f2c640b4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:27.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.277+0000 7f2c6bf82700 1 -- 172.21.15.36:0/1004660797 shutdown_connections 2022-06-17T12:47:27.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.277+0000 7f2c6bf82700 1 -- 172.21.15.36:0/1004660797 wait complete. 2022-06-17T12:47:27.360 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:47:27.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.505+0000 7fb5c8d9d700 1 Processor -- start 2022-06-17T12:47:27.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.505+0000 7fb5c8d9d700 1 -- start start 2022-06-17T12:47:27.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5c8d9d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb5c40b64f0 0x7fb5c4143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:27.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5c8d9d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb5c40b4120 0x7fb5c4146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:27.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5c8d9d700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5c40b68d0 con 0x7fb5c4147270 2022-06-17T12:47:27.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5c8d9d700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c41476d0 con 0x7fb5c40b4120 2022-06-17T12:47:27.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5c8d9d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c40b6e20 con 0x7fb5c40b64f0 2022-06-17T12:47:27.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5c259c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb5c40b64f0 0x7fb5c4143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:27.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5b9d9b700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb5c40b4120 0x7fb5c4146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:27.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5c2d9d700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb5c4147270 0x7fb5c4140100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:45536/0 (socket says 172.21.15.36:45536) 2022-06-17T12:47:27.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5c2d9d700 1 -- 172.21.15.36:0/60733798 learned_addr learned my addr 172.21.15.36:0/60733798 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:27.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5b9d9b700 1 -- 172.21.15.36:0/60733798 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb5c40b64f0 msgr2=0x7fb5c4143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:27.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5b9d9b700 1 --2- 172.21.15.36:0/60733798 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb5c40b64f0 0x7fb5c4143150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:27.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5b9d9b700 1 -- 172.21.15.36:0/60733798 >> v1:172.21.15.70:6789/0 conn(0x7fb5c4147270 legacy=0x7fb5c4140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:27.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5c1d9b700 1 -- 172.21.15.36:0/60733798 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888839960 0 0) 0x7fb5c40b68d0 con 0x7fb5c4147270 2022-06-17T12:47:27.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5b9d9b700 1 -- 172.21.15.36:0/60733798 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c41468e0 con 0x7fb5c40b4120 2022-06-17T12:47:27.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5c259c700 1 --2- 172.21.15.36:0/60733798 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb5c40b64f0 0x7fb5c4143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:27.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5b9d9b700 1 --2- 172.21.15.36:0/60733798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5c40b4120 0x7fb5c4146d80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fb5b4002f70 tx=0x7fb5b4006df0).ready entity=mon.1 client_cookie=8080fdb727231d7 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:27.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5c1d9b700 1 -- 172.21.15.36:0/60733798 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5b400d570 con 0x7fb5c40b4120 2022-06-17T12:47:27.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5c1d9b700 1 -- 172.21.15.36:0/60733798 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b4000e90 con 0x7fb5c40b4120 2022-06-17T12:47:27.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5c1d9b700 1 -- 172.21.15.36:0/60733798 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5b400d8c0 con 0x7fb5c40b4120 2022-06-17T12:47:27.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5c8d9d700 1 -- 172.21.15.36:0/60733798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5c40b4120 msgr2=0x7fb5c4146d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:27.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5c8d9d700 1 --2- 172.21.15.36:0/60733798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5c40b4120 0x7fb5c4146d80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fb5b4002f70 tx=0x7fb5b4006df0).stop 2022-06-17T12:47:27.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5c8d9d700 1 -- 172.21.15.36:0/60733798 shutdown_connections 2022-06-17T12:47:27.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5c8d9d700 1 --2- 172.21.15.36:0/60733798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5c40b4120 0x7fb5c4146d80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:27.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5c8d9d700 1 --2- 172.21.15.36:0/60733798 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb5c40b64f0 0x7fb5c4143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:27.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5c8d9d700 1 -- 172.21.15.36:0/60733798 >> 172.21.15.36:0/60733798 conn(0x7fb5c40620f0 msgr2=0x7fb5c4062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:27.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5c8d9d700 1 -- 172.21.15.36:0/60733798 shutdown_connections 2022-06-17T12:47:27.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.509+0000 7fb5c8d9d700 1 -- 172.21.15.36:0/60733798 wait complete. 2022-06-17T12:47:27.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.513+0000 7fb5c8d9d700 1 Processor -- start 2022-06-17T12:47:27.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.513+0000 7fb5c8d9d700 1 -- start start 2022-06-17T12:47:27.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.513+0000 7fb5c8d9d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb5c40b4120 0x7fb5c4152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:27.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.513+0000 7fb5c8d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb5c40b64f0 0x7fb5c414a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:27.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.513+0000 7fb5c8d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5c4147270 0x7fb5c4140500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:27.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.513+0000 7fb5c8d9d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c40b25f0 con 0x7fb5c40b64f0 2022-06-17T12:47:27.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.513+0000 7fb5c8d9d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c40b2820 con 0x7fb5c4147270 2022-06-17T12:47:27.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.513+0000 7fb5c8d9d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c40b2ac0 con 0x7fb5c40b4120 2022-06-17T12:47:27.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.513+0000 7fb5b9d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb5c40b64f0 0x7fb5c414a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:27.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.513+0000 7fb5b9d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb5c40b64f0 0x7fb5c414a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60508/0 (socket says 172.21.15.36:60508) 2022-06-17T12:47:27.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.513+0000 7fb5b9d9b700 1 -- 172.21.15.36:0/245183674 learned_addr learned my addr 172.21.15.36:0/245183674 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:27.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.513+0000 7fb5c259c700 1 --2- 172.21.15.36:0/245183674 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb5c40b4120 0x7fb5c4152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:27.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.513+0000 7fb5c2d9d700 1 --2- 172.21.15.36:0/245183674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5c4147270 0x7fb5c4140500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:27.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.513+0000 7fb5b9d9b700 1 -- 172.21.15.36:0/245183674 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb5c40b4120 msgr2=0x7fb5c4152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:27.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.513+0000 7fb5b9d9b700 1 --2- 172.21.15.36:0/245183674 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb5c40b4120 0x7fb5c4152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:27.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.513+0000 7fb5b9d9b700 1 -- 172.21.15.36:0/245183674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5c4147270 msgr2=0x7fb5c4140500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:27.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.513+0000 7fb5b9d9b700 1 --2- 172.21.15.36:0/245183674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5c4147270 0x7fb5c4140500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:27.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.513+0000 7fb5b9d9b700 1 -- 172.21.15.36:0/245183674 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ac00b180 con 0x7fb5c40b64f0 2022-06-17T12:47:27.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.513+0000 7fb5b9d9b700 1 --2- 172.21.15.36:0/245183674 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb5c40b64f0 0x7fb5c414a5e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fb5b4002f70 tx=0x7fb5b4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:27.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.513+0000 7fb5bbfff700 1 -- 172.21.15.36:0/245183674 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5b400dea0 con 0x7fb5c40b64f0 2022-06-17T12:47:27.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.513+0000 7fb5bbfff700 1 -- 172.21.15.36:0/245183674 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b4013de0 con 0x7fb5c40b64f0 2022-06-17T12:47:27.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.513+0000 7fb5c8d9d700 1 -- 172.21.15.36:0/245183674 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b400b180 con 0x7fb5c40b64f0 2022-06-17T12:47:27.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.513+0000 7fb5bbfff700 1 -- 172.21.15.36:0/245183674 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5b401e5f0 con 0x7fb5c40b64f0 2022-06-17T12:47:27.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.513+0000 7fb5c8d9d700 1 -- 172.21.15.36:0/245183674 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c41432e0 con 0x7fb5c40b64f0 2022-06-17T12:47:27.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.517+0000 7fb5c8d9d700 1 -- 172.21.15.36:0/245183674 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c40b35d0 con 0x7fb5c40b64f0 2022-06-17T12:47:27.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.517+0000 7fb5bbfff700 1 -- 172.21.15.36:0/245183674 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fb5b4025030 con 0x7fb5c40b64f0 2022-06-17T12:47:27.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.517+0000 7fb5bbfff700 1 --2- 172.21.15.36:0/245183674 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb59c053940 0x7fb59c055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:27.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.517+0000 7fb5bbfff700 1 -- 172.21.15.36:0/245183674 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb5b406cba0 con 0x7fb5c40b64f0 2022-06-17T12:47:27.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.517+0000 7fb5c259c700 1 --2- 172.21.15.36:0/245183674 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb59c053940 0x7fb59c055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:27.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.517+0000 7fb5c259c700 1 --2- 172.21.15.36:0/245183674 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb59c053940 0x7fb59c055e00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fb5ac00b000 tx=0x7fb5ac006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:27.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.521+0000 7fb5bbfff700 1 -- 172.21.15.36:0/245183674 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb5c40b35d0 con 0x7fb5c40b64f0 2022-06-17T12:47:27.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.653+0000 7fb5c8d9d700 1 -- 172.21.15.36:0/245183674 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb5c40b35d0 con 0x7fb5c40b64f0 2022-06-17T12:47:27.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.653+0000 7fb5bbfff700 1 -- 172.21.15.36:0/245183674 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+10 (secure 0 0 0) 0x7fb5c40b35d0 con 0x7fb5c40b64f0 2022-06-17T12:47:28.003 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:28.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.657+0000 7fb5c8d9d700 1 -- 172.21.15.36:0/245183674 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb59c053940 msgr2=0x7fb59c055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:28.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.657+0000 7fb5c8d9d700 1 --2- 172.21.15.36:0/245183674 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb59c053940 0x7fb59c055e00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fb5ac00b000 tx=0x7fb5ac006df0).stop 2022-06-17T12:47:28.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.657+0000 7fb5c8d9d700 1 -- 172.21.15.36:0/245183674 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb5c40b64f0 msgr2=0x7fb5c414a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:28.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.657+0000 7fb5c8d9d700 1 --2- 172.21.15.36:0/245183674 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb5c40b64f0 0x7fb5c414a5e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fb5b4002f70 tx=0x7fb5b4014000).stop 2022-06-17T12:47:28.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.657+0000 7fb5c8d9d700 1 -- 172.21.15.36:0/245183674 shutdown_connections 2022-06-17T12:47:28.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.657+0000 7fb5c8d9d700 1 --2- 172.21.15.36:0/245183674 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb59c053940 0x7fb59c055e00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:28.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.657+0000 7fb5c8d9d700 1 --2- 172.21.15.36:0/245183674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5c4147270 0x7fb5c4140500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:28.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.657+0000 7fb5c8d9d700 1 --2- 172.21.15.36:0/245183674 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb5c40b64f0 0x7fb5c414a5e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:28.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.657+0000 7fb5c8d9d700 1 --2- 172.21.15.36:0/245183674 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb5c40b4120 0x7fb5c4152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:28.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.657+0000 7fb5c8d9d700 1 -- 172.21.15.36:0/245183674 >> 172.21.15.36:0/245183674 conn(0x7fb5c40620f0 msgr2=0x7fb5c40624d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:28.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.657+0000 7fb5c8d9d700 1 -- 172.21.15.36:0/245183674 shutdown_connections 2022-06-17T12:47:28.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:27.657+0000 7fb5c8d9d700 1 -- 172.21.15.36:0/245183674 wait complete. 2022-06-17T12:47:28.103 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:47:28.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.253+0000 7f049ae56700 1 Processor -- start 2022-06-17T12:47:28.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.253+0000 7f049ae56700 1 -- start start 2022-06-17T12:47:28.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.253+0000 7f049ae56700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f04941473d0 0x7f049405fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:28.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.253+0000 7f049ae56700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0494060370 0x7f0494143200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:28.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.253+0000 7f049ae56700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04941467c0 con 0x7f04940b2df0 2022-06-17T12:47:28.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.253+0000 7f049ae56700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04940b31d0 con 0x7f0494060370 2022-06-17T12:47:28.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.253+0000 7f049ae56700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04941469f0 con 0x7f04941473d0 2022-06-17T12:47:28.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f04993f3700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0494060370 0x7f0494143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:28.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f04993f3700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0494060370 0x7f0494143200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60518/0 (socket says 172.21.15.36:60518) 2022-06-17T12:47:28.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f04993f3700 1 -- 172.21.15.36:0/2195967325 learned_addr learned my addr 172.21.15.36:0/2195967325 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:28.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f0493fff700 1 --2- 172.21.15.36:0/2195967325 >> v2:172.21.15.70:3300/0 conn(0x7f04941473d0 0x7f049405fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:28.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f04993f3700 1 -- 172.21.15.36:0/2195967325 >> v2:172.21.15.70:3300/0 conn(0x7f04941473d0 msgr2=0x7f049405fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:28.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f04993f3700 1 --2- 172.21.15.36:0/2195967325 >> v2:172.21.15.70:3300/0 conn(0x7f04941473d0 0x7f049405fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:28.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f04993f3700 1 -- 172.21.15.36:0/2195967325 >> v1:172.21.15.70:6789/0 conn(0x7f04940b2df0 legacy=0x7f049405f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:28.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f04993f3700 1 -- 172.21.15.36:0/2195967325 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0494146c90 con 0x7f0494060370 2022-06-17T12:47:28.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f04993f3700 1 --2- 172.21.15.36:0/2195967325 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0494060370 0x7f0494143200 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f048c002f70 tx=0x7f048c006eb0).ready entity=mon.0 client_cookie=147f9c8439fb627b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:28.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f04937fe700 1 -- 172.21.15.36:0/2195967325 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f048c00d5c0 con 0x7f0494060370 2022-06-17T12:47:28.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f04937fe700 1 -- 172.21.15.36:0/2195967325 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048c000e90 con 0x7f0494060370 2022-06-17T12:47:28.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f04937fe700 1 -- 172.21.15.36:0/2195967325 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f048c00d950 con 0x7f0494060370 2022-06-17T12:47:28.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f049ae56700 1 -- 172.21.15.36:0/2195967325 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0494060370 msgr2=0x7f0494143200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:28.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f049ae56700 1 --2- 172.21.15.36:0/2195967325 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0494060370 0x7f0494143200 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f048c002f70 tx=0x7f048c006eb0).stop 2022-06-17T12:47:28.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f049ae56700 1 -- 172.21.15.36:0/2195967325 shutdown_connections 2022-06-17T12:47:28.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f049ae56700 1 --2- 172.21.15.36:0/2195967325 >> v2:172.21.15.70:3300/0 conn(0x7f04941473d0 0x7f049405fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:28.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f049ae56700 1 --2- 172.21.15.36:0/2195967325 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0494060370 0x7f0494143200 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:28.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f049ae56700 1 -- 172.21.15.36:0/2195967325 >> 172.21.15.36:0/2195967325 conn(0x7f0494063020 msgr2=0x7f0494063430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:28.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f049ae56700 1 -- 172.21.15.36:0/2195967325 shutdown_connections 2022-06-17T12:47:28.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f049ae56700 1 -- 172.21.15.36:0/2195967325 wait complete. 2022-06-17T12:47:28.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f049ae56700 1 Processor -- start 2022-06-17T12:47:28.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f049ae56700 1 -- start start 2022-06-17T12:47:28.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f049ae56700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0494060370 0x7f04940ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:28.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f049ae56700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04940b2df0 0x7f04940aa710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:28.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f049ae56700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04941473d0 0x7f04940aac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:28.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f049ae56700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04940b5070 con 0x7f04941473d0 2022-06-17T12:47:28.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f049ae56700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049413eca0 con 0x7f0494060370 2022-06-17T12:47:28.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.257+0000 7f049ae56700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04940ab0f0 con 0x7f04940b2df0 2022-06-17T12:47:28.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.261+0000 7f04993f3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04941473d0 0x7f04940aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:28.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.261+0000 7f04993f3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04941473d0 0x7f04940aac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60526/0 (socket says 172.21.15.36:60526) 2022-06-17T12:47:28.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.261+0000 7f04993f3700 1 -- 172.21.15.36:0/1819247196 learned_addr learned my addr 172.21.15.36:0/1819247196 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:28.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.261+0000 7f0498bf2700 1 --2- 172.21.15.36:0/1819247196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0494060370 0x7f04940ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:28.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.261+0000 7f0493fff700 1 --2- 172.21.15.36:0/1819247196 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04940b2df0 0x7f04940aa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:28.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.261+0000 7f04993f3700 1 -- 172.21.15.36:0/1819247196 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04940b2df0 msgr2=0x7f04940aa710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:28.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.261+0000 7f04993f3700 1 --2- 172.21.15.36:0/1819247196 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04940b2df0 0x7f04940aa710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:28.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.261+0000 7f04993f3700 1 -- 172.21.15.36:0/1819247196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0494060370 msgr2=0x7f04940ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:28.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.261+0000 7f04993f3700 1 --2- 172.21.15.36:0/1819247196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0494060370 0x7f04940ac8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:28.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.261+0000 7f04993f3700 1 -- 172.21.15.36:0/1819247196 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048c00b1e0 con 0x7f04941473d0 2022-06-17T12:47:28.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.261+0000 7f04993f3700 1 --2- 172.21.15.36:0/1819247196 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04941473d0 0x7f04940aac00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f048c002f70 tx=0x7f048c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:28.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.261+0000 7f0491ffb700 1 -- 172.21.15.36:0/1819247196 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f048c0116e0 con 0x7f04941473d0 2022-06-17T12:47:28.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.261+0000 7f0491ffb700 1 -- 172.21.15.36:0/1819247196 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048c011840 con 0x7f04941473d0 2022-06-17T12:47:28.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.261+0000 7f049ae56700 1 -- 172.21.15.36:0/1819247196 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f049413ef40 con 0x7f04941473d0 2022-06-17T12:47:28.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.261+0000 7f0491ffb700 1 -- 172.21.15.36:0/1819247196 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f048c011ba0 con 0x7f04941473d0 2022-06-17T12:47:28.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.261+0000 7f049ae56700 1 -- 172.21.15.36:0/1819247196 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04941436a0 con 0x7f04941473d0 2022-06-17T12:47:28.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.261+0000 7f049ae56700 1 -- 172.21.15.36:0/1819247196 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04940aa340 con 0x7f04941473d0 2022-06-17T12:47:28.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.261+0000 7f0491ffb700 1 -- 172.21.15.36:0/1819247196 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f048c025030 con 0x7f04941473d0 2022-06-17T12:47:28.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.265+0000 7f0491ffb700 1 --2- 172.21.15.36:0/1819247196 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0478053a60 0x7f0478055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:28.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.265+0000 7f0491ffb700 1 -- 172.21.15.36:0/1819247196 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f048c06d080 con 0x7f04941473d0 2022-06-17T12:47:28.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.265+0000 7f0498bf2700 1 --2- 172.21.15.36:0/1819247196 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0478053a60 0x7f0478055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:28.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.265+0000 7f0498bf2700 1 --2- 172.21.15.36:0/1819247196 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0478053a60 0x7f0478055f20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f048400d000 tx=0x7f0484007f10).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:28.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.269+0000 7f0491ffb700 1 -- 172.21.15.36:0/1819247196 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f048c043e80 con 0x7f04941473d0 2022-06-17T12:47:28.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.397+0000 7f049ae56700 1 -- 172.21.15.36:0/1819247196 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0494146f30 con 0x7f04941473d0 2022-06-17T12:47:28.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.397+0000 7f0491ffb700 1 -- 172.21.15.36:0/1819247196 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+10 (secure 0 0 0) 0x7f048c023020 con 0x7f04941473d0 2022-06-17T12:47:28.447 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:28.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.397+0000 7f049ae56700 1 -- 172.21.15.36:0/1819247196 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0478053a60 msgr2=0x7f0478055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:28.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.397+0000 7f049ae56700 1 --2- 172.21.15.36:0/1819247196 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0478053a60 0x7f0478055f20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f048400d000 tx=0x7f0484007f10).stop 2022-06-17T12:47:28.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.397+0000 7f049ae56700 1 -- 172.21.15.36:0/1819247196 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04941473d0 msgr2=0x7f04940aac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:28.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.397+0000 7f049ae56700 1 --2- 172.21.15.36:0/1819247196 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04941473d0 0x7f04940aac00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f048c002f70 tx=0x7f048c015000).stop 2022-06-17T12:47:28.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.401+0000 7f049ae56700 1 -- 172.21.15.36:0/1819247196 shutdown_connections 2022-06-17T12:47:28.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.401+0000 7f049ae56700 1 --2- 172.21.15.36:0/1819247196 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0478053a60 0x7f0478055f20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:28.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.401+0000 7f049ae56700 1 --2- 172.21.15.36:0/1819247196 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04941473d0 0x7f04940aac00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:28.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.401+0000 7f049ae56700 1 --2- 172.21.15.36:0/1819247196 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04940b2df0 0x7f04940aa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:28.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.401+0000 7f049ae56700 1 --2- 172.21.15.36:0/1819247196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0494060370 0x7f04940ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:28.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.401+0000 7f049ae56700 1 -- 172.21.15.36:0/1819247196 >> 172.21.15.36:0/1819247196 conn(0x7f0494063020 msgr2=0x7f04940af510 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:28.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.401+0000 7f049ae56700 1 -- 172.21.15.36:0/1819247196 shutdown_connections 2022-06-17T12:47:28.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.401+0000 7f049ae56700 1 -- 172.21.15.36:0/1819247196 wait complete. 2022-06-17T12:47:28.823 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop) 2022-06-17T12:47:28.824 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command' 2022-06-17T12:47:29.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.949+0000 7fadc36e8700 1 Processor -- start 2022-06-17T12:47:29.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.949+0000 7fadc36e8700 1 -- start start 2022-06-17T12:47:29.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadc36e8700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadbc0627b0 0x7fadbc0af650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:29.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadc36e8700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fadbc0ad190 0x7fadbc142d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:29.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadc36e8700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadbc0b1170 con 0x7fadbc0ad930 2022-06-17T12:47:29.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadc36e8700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadbc13f670 con 0x7fadbc0ad190 2022-06-17T12:47:29.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadc36e8700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc062b90 con 0x7fadbc0627b0 2022-06-17T12:47:29.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadc0c83700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fadbc0ad190 0x7fadbc142d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:29.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadc1c85700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fadbc0ad930 0x7fadbc0aba70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49832/0 (socket says 172.21.15.36:49832) 2022-06-17T12:47:29.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadc1c85700 1 -- 172.21.15.36:0/2458650089 learned_addr learned my addr 172.21.15.36:0/2458650089 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:29.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadc1484700 1 --2- 172.21.15.36:0/2458650089 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadbc0627b0 0x7fadbc0af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:29.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadc0c83700 1 -- 172.21.15.36:0/2458650089 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadbc0627b0 msgr2=0x7fadbc0af650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:29.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadc0c83700 1 --2- 172.21.15.36:0/2458650089 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadbc0627b0 0x7fadbc0af650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:29.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadc0c83700 1 -- 172.21.15.36:0/2458650089 >> v1:172.21.15.36:6789/0 conn(0x7fadbc0ad930 legacy=0x7fadbc0aba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:29.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadc0c83700 1 -- 172.21.15.36:0/2458650089 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc143220 con 0x7fadbc0ad190 2022-06-17T12:47:29.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadc0c83700 1 --2- 172.21.15.36:0/2458650089 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadbc0ad190 0x7fadbc142d30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fadac002f70 tx=0x7fadac006df0).ready entity=mon.0 client_cookie=8f6077eb4f428b30 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:29.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadabfff700 1 -- 172.21.15.36:0/2458650089 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadac014070 con 0x7fadbc0ad190 2022-06-17T12:47:29.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadabfff700 1 -- 172.21.15.36:0/2458650089 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadac000e90 con 0x7fadbc0ad190 2022-06-17T12:47:29.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadabfff700 1 -- 172.21.15.36:0/2458650089 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadac00fb60 con 0x7fadbc0ad190 2022-06-17T12:47:29.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadc36e8700 1 -- 172.21.15.36:0/2458650089 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadbc0ad190 msgr2=0x7fadbc142d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:29.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadc36e8700 1 --2- 172.21.15.36:0/2458650089 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadbc0ad190 0x7fadbc142d30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fadac002f70 tx=0x7fadac006df0).stop 2022-06-17T12:47:29.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadc36e8700 1 -- 172.21.15.36:0/2458650089 shutdown_connections 2022-06-17T12:47:29.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadc36e8700 1 --2- 172.21.15.36:0/2458650089 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadbc0ad190 0x7fadbc142d30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:29.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadc36e8700 1 --2- 172.21.15.36:0/2458650089 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadbc0627b0 0x7fadbc0af650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:29.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadc36e8700 1 -- 172.21.15.36:0/2458650089 >> 172.21.15.36:0/2458650089 conn(0x7fadbc0b5430 msgr2=0x7fadbc0b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:29.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadc36e8700 1 -- 172.21.15.36:0/2458650089 shutdown_connections 2022-06-17T12:47:29.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.953+0000 7fadc36e8700 1 -- 172.21.15.36:0/2458650089 wait complete. 2022-06-17T12:47:29.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadc36e8700 1 Processor -- start 2022-06-17T12:47:29.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadc36e8700 1 -- start start 2022-06-17T12:47:29.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadc36e8700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadbc0627b0 0x7fadbc0a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:29.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadc36e8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadbc0ad190 0x7fadbc14e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:29.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadc36e8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadbc0ad930 0x7fadbc14eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:29.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadc36e8700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc0a20c0 con 0x7fadbc0ad930 2022-06-17T12:47:29.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadc36e8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc0a3690 con 0x7fadbc0ad190 2022-06-17T12:47:29.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadc36e8700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc0a3930 con 0x7fadbc0627b0 2022-06-17T12:47:29.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadc1c85700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadbc0ad930 0x7fadbc14eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:29.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadc1c85700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadbc0ad930 0x7fadbc14eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60538/0 (socket says 172.21.15.36:60538) 2022-06-17T12:47:29.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadc1c85700 1 -- 172.21.15.36:0/2426785781 learned_addr learned my addr 172.21.15.36:0/2426785781 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:29.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadc1484700 1 --2- 172.21.15.36:0/2426785781 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadbc0627b0 0x7fadbc0a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:29.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadc0c83700 1 --2- 172.21.15.36:0/2426785781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadbc0ad190 0x7fadbc14e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:29.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadc1c85700 1 -- 172.21.15.36:0/2426785781 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadbc0627b0 msgr2=0x7fadbc0a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:29.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadc1c85700 1 --2- 172.21.15.36:0/2426785781 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadbc0627b0 0x7fadbc0a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:29.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadc1c85700 1 -- 172.21.15.36:0/2426785781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadbc0ad190 msgr2=0x7fadbc14e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:29.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadc1c85700 1 --2- 172.21.15.36:0/2426785781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadbc0ad190 0x7fadbc14e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:29.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadc1c85700 1 -- 172.21.15.36:0/2426785781 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadac00b180 con 0x7fadbc0ad930 2022-06-17T12:47:29.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadc1c85700 1 --2- 172.21.15.36:0/2426785781 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadbc0ad930 0x7fadbc14eb90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fadbc0b63a0 tx=0x7fadb40013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:29.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadaa7fc700 1 -- 172.21.15.36:0/2426785781 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadb4005ba0 con 0x7fadbc0ad930 2022-06-17T12:47:29.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadaa7fc700 1 -- 172.21.15.36:0/2426785781 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb4005d00 con 0x7fadbc0ad930 2022-06-17T12:47:29.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadc36e8700 1 -- 172.21.15.36:0/2426785781 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadbc09f210 con 0x7fadbc0ad930 2022-06-17T12:47:29.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadaa7fc700 1 -- 172.21.15.36:0/2426785781 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadb40072c0 con 0x7fadbc0ad930 2022-06-17T12:47:29.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadc36e8700 1 -- 172.21.15.36:0/2426785781 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadbc09faa0 con 0x7fadbc0ad930 2022-06-17T12:47:29.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.957+0000 7fadc36e8700 1 -- 172.21.15.36:0/2426785781 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadbc0b13a0 con 0x7fadbc0ad930 2022-06-17T12:47:29.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.961+0000 7fadaa7fc700 1 -- 172.21.15.36:0/2426785781 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fadb401a060 con 0x7fadbc0ad930 2022-06-17T12:47:29.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.961+0000 7fadaa7fc700 1 --2- 172.21.15.36:0/2426785781 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fad9c053920 0x7fad9c055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:29.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.961+0000 7fadaa7fc700 1 -- 172.21.15.36:0/2426785781 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fadb406b8b0 con 0x7fadbc0ad930 2022-06-17T12:47:29.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.961+0000 7fadc1484700 1 --2- 172.21.15.36:0/2426785781 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fad9c053920 0x7fad9c055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:29.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.961+0000 7fadc1484700 1 --2- 172.21.15.36:0/2426785781 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fad9c053920 0x7fad9c055de0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fadb800b000 tx=0x7fadb8006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:29.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:28.965+0000 7fadaa7fc700 1 -- 172.21.15.36:0/2426785781 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fadb40426b0 con 0x7fadbc0ad930 2022-06-17T12:47:29.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.141+0000 7fadc36e8700 1 -- 172.21.15.36:0/2426785781 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]} v 0) v1 -- 0x7fadbc057410 con 0x7fadbc0ad930 2022-06-17T12:47:29.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.613+0000 7fadaa7fc700 1 -- 172.21.15.36:0/2426785781 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]}]=0 v55) v1 ==== 142+0+0 (secure 0 0 0) 0x7fadb4018020 con 0x7fadbc0ad930 2022-06-17T12:47:29.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.617+0000 7fadc36e8700 1 -- 172.21.15.36:0/2426785781 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fad9c053920 msgr2=0x7fad9c055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:29.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.617+0000 7fadc36e8700 1 --2- 172.21.15.36:0/2426785781 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fad9c053920 0x7fad9c055de0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fadb800b000 tx=0x7fadb8006df0).stop 2022-06-17T12:47:29.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.617+0000 7fadc36e8700 1 -- 172.21.15.36:0/2426785781 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadbc0ad930 msgr2=0x7fadbc14eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:29.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.617+0000 7fadc36e8700 1 --2- 172.21.15.36:0/2426785781 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadbc0ad930 0x7fadbc14eb90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fadbc0b63a0 tx=0x7fadb40013a0).stop 2022-06-17T12:47:29.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.617+0000 7fadc36e8700 1 -- 172.21.15.36:0/2426785781 shutdown_connections 2022-06-17T12:47:29.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.617+0000 7fadc36e8700 1 --2- 172.21.15.36:0/2426785781 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fad9c053920 0x7fad9c055de0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:29.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.617+0000 7fadc36e8700 1 --2- 172.21.15.36:0/2426785781 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadbc0ad930 0x7fadbc14eb90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:29.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.617+0000 7fadc36e8700 1 --2- 172.21.15.36:0/2426785781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadbc0ad190 0x7fadbc14e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:29.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.617+0000 7fadc36e8700 1 --2- 172.21.15.36:0/2426785781 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadbc0627b0 0x7fadbc0a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:29.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.617+0000 7fadc36e8700 1 -- 172.21.15.36:0/2426785781 >> 172.21.15.36:0/2426785781 conn(0x7fadbc0b5430 msgr2=0x7fadbc0b4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:29.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.617+0000 7fadc36e8700 1 -- 172.21.15.36:0/2426785781 shutdown_connections 2022-06-17T12:47:29.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.617+0000 7fadc36e8700 1 -- 172.21.15.36:0/2426785781 wait complete. 2022-06-17T12:47:29.675 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-06-17T12:47:29.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.821+0000 7f179de24700 1 Processor -- start 2022-06-17T12:47:29.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.821+0000 7f179de24700 1 -- start start 2022-06-17T12:47:29.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.821+0000 7f179de24700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f17980b64f0 0x7f1798143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:29.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.821+0000 7f179de24700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17980b68d0 con 0x7f17980b4120 2022-06-17T12:47:29.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.821+0000 7f179de24700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17980b6e20 con 0x7f1798147490 2022-06-17T12:47:29.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.821+0000 7f179de24700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17981468e0 con 0x7f17980b64f0 2022-06-17T12:47:29.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.821+0000 7f1796ffd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f17980b4120 0x7f1798146d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49844/0 (socket says 172.21.15.36:49844) 2022-06-17T12:47:29.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.821+0000 7f1796ffd700 1 -- 172.21.15.36:0/1453792834 learned_addr learned my addr 172.21.15.36:0/1453792834 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:29.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.821+0000 7f17977fe700 1 --2- 172.21.15.36:0/1453792834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f17980b64f0 0x7f1798143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:29.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.821+0000 7f17967fc700 1 -- 172.21.15.36:0/1453792834 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3317858339 0 0) 0x7f17980b68d0 con 0x7f17980b4120 2022-06-17T12:47:29.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.821+0000 7f17967fc700 1 -- 172.21.15.36:0/1453792834 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1778003260 con 0x7f17980b4120 2022-06-17T12:47:29.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.821+0000 7f17967fc700 1 -- 172.21.15.36:0/1453792834 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 835153113 0 0) 0x7f17980b6e20 con 0x7f1798147490 2022-06-17T12:47:29.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.821+0000 7f17967fc700 1 -- 172.21.15.36:0/1453792834 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17980b68d0 con 0x7f1798147490 2022-06-17T12:47:29.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.821+0000 7f17967fc700 1 -- 172.21.15.36:0/1453792834 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1562829752 0 0) 0x7f1778003260 con 0x7f17980b4120 2022-06-17T12:47:29.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f17967fc700 1 -- 172.21.15.36:0/1453792834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f17980b64f0 msgr2=0x7f1798143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:29.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f17967fc700 1 --2- 172.21.15.36:0/1453792834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f17980b64f0 0x7f1798143150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:29.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f17967fc700 1 -- 172.21.15.36:0/1453792834 >> v1:172.21.15.70:6789/0 conn(0x7f1798147490 legacy=0x7f1798140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:29.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f17967fc700 1 -- 172.21.15.36:0/1453792834 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1798143640 con 0x7f17980b4120 2022-06-17T12:47:29.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f17967fc700 1 -- 172.21.15.36:0/1453792834 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f1780001660 con 0x7f17980b4120 2022-06-17T12:47:29.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f17967fc700 1 -- 172.21.15.36:0/1453792834 >> v1:172.21.15.36:6789/0 conn(0x7f17980b4120 legacy=0x7f1798146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:29.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f17967fc700 1 --2- 172.21.15.36:0/1453792834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f17980b64f0 0x7f1778003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:29.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f17967fc700 1 --2- 172.21.15.36:0/1453792834 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1778003e20 0x7f1778004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:29.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f17967fc700 1 --2- 172.21.15.36:0/1453792834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17780049f0 0x7f1778004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:29.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f17967fc700 1 -- 172.21.15.36:0/1453792834 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17981468e0 con 0x7f1778003e20 2022-06-17T12:47:29.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f17967fc700 1 -- 172.21.15.36:0/1453792834 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177800b800 con 0x7f17780049f0 2022-06-17T12:47:29.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f17967fc700 1 -- 172.21.15.36:0/1453792834 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f177800b9c0 con 0x7f17980b64f0 2022-06-17T12:47:29.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f1796ffd700 1 --2- 172.21.15.36:0/1453792834 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1778003e20 0x7f1778004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:29.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f17977fe700 1 --2- 172.21.15.36:0/1453792834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f17980b64f0 0x7f1778003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:29.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f1797fff700 1 --2- 172.21.15.36:0/1453792834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17780049f0 0x7f1778004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:29.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f1796ffd700 1 -- 172.21.15.36:0/1453792834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f17980b64f0 msgr2=0x7f1778003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:29.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f1796ffd700 1 --2- 172.21.15.36:0/1453792834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f17980b64f0 0x7f1778003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:29.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f1796ffd700 1 -- 172.21.15.36:0/1453792834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17780049f0 msgr2=0x7f1778004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:29.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f1796ffd700 1 --2- 172.21.15.36:0/1453792834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17780049f0 0x7f1778004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:29.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f1796ffd700 1 -- 172.21.15.36:0/1453792834 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1778003260 con 0x7f1778003e20 2022-06-17T12:47:29.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f1796ffd700 1 --2- 172.21.15.36:0/1453792834 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1778003e20 0x7f1778004500 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f1780001430 tx=0x7f1780001dd0).ready entity=mon.0 client_cookie=f601bade7f23ae4b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:29.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f17967fc700 1 -- 172.21.15.36:0/1453792834 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17800070d0 con 0x7f1778003e20 2022-06-17T12:47:29.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f17967fc700 1 -- 172.21.15.36:0/1453792834 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1780007230 con 0x7f1778003e20 2022-06-17T12:47:29.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f17967fc700 1 -- 172.21.15.36:0/1453792834 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f178000c8c0 con 0x7f1778003e20 2022-06-17T12:47:29.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f179de24700 1 -- 172.21.15.36:0/1453792834 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1778003e20 msgr2=0x7f1778004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:29.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f179de24700 1 --2- 172.21.15.36:0/1453792834 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1778003e20 0x7f1778004500 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f1780001430 tx=0x7f1780001dd0).stop 2022-06-17T12:47:29.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f1797fff700 1 -- 172.21.15.36:0/1453792834 reap_dead start 2022-06-17T12:47:29.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f179de24700 1 -- 172.21.15.36:0/1453792834 shutdown_connections 2022-06-17T12:47:29.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f179de24700 1 -- 172.21.15.36:0/1453792834 >> 172.21.15.36:0/1453792834 conn(0x7f17980620f0 msgr2=0x7f1798062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:29.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f179de24700 1 -- 172.21.15.36:0/1453792834 shutdown_connections 2022-06-17T12:47:29.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.825+0000 7f179de24700 1 -- 172.21.15.36:0/1453792834 wait complete. 2022-06-17T12:47:29.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.829+0000 7f179de24700 1 Processor -- start 2022-06-17T12:47:29.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.829+0000 7f179de24700 1 -- start start 2022-06-17T12:47:29.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.829+0000 7f179de24700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1778003e20 0x7f179813e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:29.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.829+0000 7f179de24700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f17980b34a0 0x7f1798156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:29.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.829+0000 7f179de24700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1798156780 0x7f1798144660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:30.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.829+0000 7f179de24700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17980b1200 con 0x7f17980b34a0 2022-06-17T12:47:30.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.829+0000 7f179de24700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17981401e0 con 0x7f1778003e20 2022-06-17T12:47:30.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.829+0000 7f179de24700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17981405b0 con 0x7f1798156780 2022-06-17T12:47:30.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.829+0000 7f1796ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f17980b34a0 0x7f1798156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:30.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.829+0000 7f1796ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f17980b34a0 0x7f1798156290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60558/0 (socket says 172.21.15.36:60558) 2022-06-17T12:47:30.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.829+0000 7f1796ffd700 1 -- 172.21.15.36:0/490033892 learned_addr learned my addr 172.21.15.36:0/490033892 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:30.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.829+0000 7f1797fff700 1 --2- 172.21.15.36:0/490033892 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1798156780 0x7f1798144660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:30.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.829+0000 7f17977fe700 1 --2- 172.21.15.36:0/490033892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1778003e20 0x7f179813e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:30.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.833+0000 7f1796ffd700 1 -- 172.21.15.36:0/490033892 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1798156780 msgr2=0x7f1798144660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:30.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.833+0000 7f1796ffd700 1 --2- 172.21.15.36:0/490033892 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1798156780 0x7f1798144660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:30.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.833+0000 7f1796ffd700 1 -- 172.21.15.36:0/490033892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1778003e20 msgr2=0x7f179813e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:30.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.833+0000 7f1796ffd700 1 --2- 172.21.15.36:0/490033892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1778003e20 0x7f179813e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:30.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.833+0000 7f1796ffd700 1 -- 172.21.15.36:0/490033892 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17980b0f60 con 0x7f17980b34a0 2022-06-17T12:47:30.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.833+0000 7f17977fe700 1 --2- 172.21.15.36:0/490033892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1778003e20 0x7f179813e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:30.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.833+0000 7f1796ffd700 1 --2- 172.21.15.36:0/490033892 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f17980b34a0 0x7f1798156290 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f1780001430 tx=0x7f1780015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:30.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.833+0000 7f1794ff9700 1 -- 172.21.15.36:0/490033892 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17800070d0 con 0x7f17980b34a0 2022-06-17T12:47:30.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.833+0000 7f1794ff9700 1 -- 172.21.15.36:0/490033892 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1780000e90 con 0x7f17980b34a0 2022-06-17T12:47:30.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.833+0000 7f1794ff9700 1 -- 172.21.15.36:0/490033892 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1780004900 con 0x7f17980b34a0 2022-06-17T12:47:30.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.833+0000 7f179de24700 1 -- 172.21.15.36:0/490033892 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f179813fed0 con 0x7f17980b34a0 2022-06-17T12:47:30.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.833+0000 7f179de24700 1 -- 172.21.15.36:0/490033892 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1798156c10 con 0x7f17980b34a0 2022-06-17T12:47:30.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.833+0000 7f179de24700 1 -- 172.21.15.36:0/490033892 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17980b3a60 con 0x7f17980b34a0 2022-06-17T12:47:30.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.833+0000 7f1794ff9700 1 -- 172.21.15.36:0/490033892 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f1780010070 con 0x7f17980b34a0 2022-06-17T12:47:30.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.833+0000 7f1794ff9700 1 --2- 172.21.15.36:0/490033892 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f177c053ad0 0x7f177c055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:30.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.833+0000 7f1794ff9700 1 -- 172.21.15.36:0/490033892 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f178006cea0 con 0x7f17980b34a0 2022-06-17T12:47:30.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.833+0000 7f17977fe700 1 --2- 172.21.15.36:0/490033892 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f177c053ad0 0x7f177c055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:30.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.833+0000 7f17977fe700 1 --2- 172.21.15.36:0/490033892 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f177c053ad0 0x7f177c055f90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f178800b000 tx=0x7f1788008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:30.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.841+0000 7f1794ff9700 1 -- 172.21.15.36:0/490033892 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1780043ca0 con 0x7f17980b34a0 2022-06-17T12:47:30.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.965+0000 7f179de24700 1 -- 172.21.15.36:0/490033892 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1798057410 con 0x7f17980b34a0 2022-06-17T12:47:30.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.973+0000 7f1794ff9700 1 -- 172.21.15.36:0/490033892 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1780025070 con 0x7f17980b34a0 2022-06-17T12:47:30.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.973+0000 7f179de24700 1 -- 172.21.15.36:0/490033892 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f177c053ad0 msgr2=0x7f177c055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:30.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.973+0000 7f179de24700 1 --2- 172.21.15.36:0/490033892 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f177c053ad0 0x7f177c055f90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f178800b000 tx=0x7f1788008040).stop 2022-06-17T12:47:30.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.973+0000 7f179de24700 1 -- 172.21.15.36:0/490033892 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f17980b34a0 msgr2=0x7f1798156290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:30.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.973+0000 7f179de24700 1 --2- 172.21.15.36:0/490033892 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f17980b34a0 0x7f1798156290 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f1780001430 tx=0x7f1780015000).stop 2022-06-17T12:47:30.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.973+0000 7f179de24700 1 -- 172.21.15.36:0/490033892 shutdown_connections 2022-06-17T12:47:30.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.973+0000 7f179de24700 1 --2- 172.21.15.36:0/490033892 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f177c053ad0 0x7f177c055f90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:30.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.973+0000 7f179de24700 1 --2- 172.21.15.36:0/490033892 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1798156780 0x7f1798144660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:30.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.973+0000 7f179de24700 1 --2- 172.21.15.36:0/490033892 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f17980b34a0 0x7f1798156290 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:30.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.973+0000 7f179de24700 1 --2- 172.21.15.36:0/490033892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1778003e20 0x7f179813e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:30.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.973+0000 7f179de24700 1 -- 172.21.15.36:0/490033892 >> 172.21.15.36:0/490033892 conn(0x7f17980620f0 msgr2=0x7f17980afb20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:30.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.973+0000 7f179de24700 1 -- 172.21.15.36:0/490033892 shutdown_connections 2022-06-17T12:47:30.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:29.973+0000 7f179de24700 1 -- 172.21.15.36:0/490033892 wait complete. 2022-06-17T12:47:30.145 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-17T12:47:30.155 DEBUG:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:47:30.227 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:47:30.272 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:47:30.322 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:47:30.364 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:47:30.401 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-17T12:47:30.402 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:30.403 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:47:30.403 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:47:30.431 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:47:30.432 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:47:30.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:30.434 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:47:30.477 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:47:30.522 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:47:30.524 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:47:30.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:30.526 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:47:30.527 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:47:30.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:47:30.556 INFO:teuthology.orchestra.run.smithi107.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2022-06-17T12:47:30.556 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2022-06-17T12:47:30.557 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2022-06-17T12:47:30.557 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:47:30.558 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:47:30.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:30.652 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:47:30.653 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:47:30.736 INFO:teuthology.orchestra.run.smithi107.stdout:ext2/ext3 2022-06-17T12:47:30.737 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:47:30.740 INFO:teuthology.orchestra.run:waiting for 900 2022-06-17T12:47:30.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:47:30.794 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:47:30.856 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:30.857 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:47:30.857 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:47:30.936 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:47:30.995 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:47:30.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:31.024 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:47:31.024 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:47:31.121 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:47:31.122 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:47:31.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:31.124 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:47:31.137 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:47:31.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:47:31.196 INFO:teuthology.orchestra.run.smithi136.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': Transport endpoint is not connected 2022-06-17T12:47:31.197 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.1 2022-06-17T12:47:31.209 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi136.front.sepia.ceph.com... 2022-06-17T12:47:31.226 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:47:31.245 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:47:31.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:31.325 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:47:31.325 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:47:31.388 INFO:teuthology.orchestra.run.smithi136.stdout:ext2/ext3 2022-06-17T12:47:31.389 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:47:31.390 INFO:teuthology.orchestra.run:waiting for 900 2022-06-17T12:47:31.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:47:31.392 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:47:31.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:31.474 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:47:31.475 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:47:31.507 INFO:teuthology.orchestra.run.smithi143.stdout:fuseblk 2022-06-17T12:47:31.508 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2022-06-17T12:47:31.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:31.509 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:47:31.510 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:47:31.619 INFO:teuthology.orchestra.run.smithi143.stdout:fuseblk 2022-06-17T12:47:31.620 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2022-06-17T12:47:31.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:31.629 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:47:31.630 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:47:31.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:47:31.698 INFO:teuthology.orchestra.run.smithi143.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': Transport endpoint is not connected 2022-06-17T12:47:31.699 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.2 2022-06-17T12:47:31.700 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi143.front.sepia.ceph.com... 2022-06-17T12:47:31.701 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:47:31.736 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:47:31.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:31.854 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:47:31.855 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:47:31.888 INFO:teuthology.orchestra.run.smithi143.stdout:ext2/ext3 2022-06-17T12:47:31.889 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:47:31.911 INFO:teuthology.orchestra.run:waiting for 900 2022-06-17T12:47:31.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:47:31.912 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:47:31.942 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-17T12:47:32.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.065+0000 7f3901c3d700 1 Processor -- start 2022-06-17T12:47:32.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.065+0000 7f3901c3d700 1 -- start start 2022-06-17T12:47:32.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.065+0000 7f3901c3d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f38fc0adf10 0x7f38fc0b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:32.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.065+0000 7f3901c3d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f38fc0b15b0 0x7f38fc0af630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:32.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.065+0000 7f3901c3d700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38fc13fdf0 con 0x7f38fc0b65f0 2022-06-17T12:47:32.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.065+0000 7f3901c3d700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38fc0ae8e0 con 0x7f38fc0adf10 2022-06-17T12:47:32.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.065+0000 7f3901c3d700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38fc140090 con 0x7f38fc0b15b0 2022-06-17T12:47:32.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.065+0000 7f38fbfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f38fc0adf10 0x7f38fc0b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:32.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.065+0000 7f38fbfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f38fc0adf10 0x7f38fc0b69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60566/0 (socket says 172.21.15.36:60566) 2022-06-17T12:47:32.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.065+0000 7f38fbfff700 1 -- 172.21.15.36:0/1894370627 learned_addr learned my addr 172.21.15.36:0/1894370627 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:32.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.065+0000 7f390143c700 1 --2- 172.21.15.36:0/1894370627 >> v2:172.21.15.70:3300/0 conn(0x7f38fc0b15b0 0x7f38fc0af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:32.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.069+0000 7f38fbfff700 1 -- 172.21.15.36:0/1894370627 >> v2:172.21.15.70:3300/0 conn(0x7f38fc0b15b0 msgr2=0x7f38fc0af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:32.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.069+0000 7f38fbfff700 1 --2- 172.21.15.36:0/1894370627 >> v2:172.21.15.70:3300/0 conn(0x7f38fc0b15b0 0x7f38fc0af630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:32.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.069+0000 7f38fbfff700 1 -- 172.21.15.36:0/1894370627 >> v1:172.21.15.70:6789/0 conn(0x7f38fc0b65f0 legacy=0x7f38fc140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:32.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.069+0000 7f38fbfff700 1 -- 172.21.15.36:0/1894370627 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc0b1be0 con 0x7f38fc0adf10 2022-06-17T12:47:32.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.069+0000 7f38fbfff700 1 --2- 172.21.15.36:0/1894370627 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f38fc0adf10 0x7f38fc0b69d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f38ec002f70 tx=0x7f38ec006e50).ready entity=mon.0 client_cookie=f76ac5b0280a4672 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:32.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.069+0000 7f38fb7fe700 1 -- 172.21.15.36:0/1894370627 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38ec00d610 con 0x7f38fc0adf10 2022-06-17T12:47:32.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.069+0000 7f38fb7fe700 1 -- 172.21.15.36:0/1894370627 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ec000e90 con 0x7f38fc0adf10 2022-06-17T12:47:32.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.069+0000 7f38fb7fe700 1 -- 172.21.15.36:0/1894370627 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38ec00d9a0 con 0x7f38fc0adf10 2022-06-17T12:47:32.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.069+0000 7f3901c3d700 1 -- 172.21.15.36:0/1894370627 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f38fc0adf10 msgr2=0x7f38fc0b69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:32.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.069+0000 7f3901c3d700 1 --2- 172.21.15.36:0/1894370627 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f38fc0adf10 0x7f38fc0b69d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f38ec002f70 tx=0x7f38ec006e50).stop 2022-06-17T12:47:32.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.069+0000 7f3901c3d700 1 -- 172.21.15.36:0/1894370627 shutdown_connections 2022-06-17T12:47:32.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.069+0000 7f3901c3d700 1 --2- 172.21.15.36:0/1894370627 >> v2:172.21.15.70:3300/0 conn(0x7f38fc0b15b0 0x7f38fc0af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:32.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.069+0000 7f3901c3d700 1 --2- 172.21.15.36:0/1894370627 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f38fc0adf10 0x7f38fc0b69d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:32.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.069+0000 7f3901c3d700 1 -- 172.21.15.36:0/1894370627 >> 172.21.15.36:0/1894370627 conn(0x7f38fc0614b0 msgr2=0x7f38fc061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:32.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.069+0000 7f3901c3d700 1 -- 172.21.15.36:0/1894370627 shutdown_connections 2022-06-17T12:47:32.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.069+0000 7f3901c3d700 1 -- 172.21.15.36:0/1894370627 wait complete. 2022-06-17T12:47:32.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.073+0000 7f3901c3d700 1 Processor -- start 2022-06-17T12:47:32.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.073+0000 7f3901c3d700 1 -- start start 2022-06-17T12:47:32.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.073+0000 7f3901c3d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38fc0adf10 0x7f38fc05c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:32.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.073+0000 7f3901c3d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38fc0b15b0 0x7f38fc134780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:32.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.073+0000 7f3901c3d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f38fc0b65f0 0x7f38fc134c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:32.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.073+0000 7f3901c3d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc13f800 con 0x7f38fc0b65f0 2022-06-17T12:47:32.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.073+0000 7f3901c3d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc065830 con 0x7f38fc0b15b0 2022-06-17T12:47:32.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.073+0000 7f3901c3d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc13f560 con 0x7f38fc0adf10 2022-06-17T12:47:32.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.073+0000 7f390143c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f38fc0b65f0 0x7f38fc134c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:32.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.073+0000 7f390143c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f38fc0b65f0 0x7f38fc134c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60574/0 (socket says 172.21.15.36:60574) 2022-06-17T12:47:32.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.073+0000 7f38fbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38fc0b15b0 0x7f38fc134780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:32.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.073+0000 7f390143c700 1 -- 172.21.15.36:0/2411210834 learned_addr learned my addr 172.21.15.36:0/2411210834 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:32.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.073+0000 7f3900c3b700 1 --2- 172.21.15.36:0/2411210834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38fc0adf10 0x7f38fc05c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:32.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.073+0000 7f390143c700 1 -- 172.21.15.36:0/2411210834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38fc0adf10 msgr2=0x7f38fc05c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:32.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.073+0000 7f390143c700 1 --2- 172.21.15.36:0/2411210834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38fc0adf10 0x7f38fc05c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:32.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.073+0000 7f390143c700 1 -- 172.21.15.36:0/2411210834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38fc0b15b0 msgr2=0x7f38fc134780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:32.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.073+0000 7f390143c700 1 --2- 172.21.15.36:0/2411210834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38fc0b15b0 0x7f38fc134780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:32.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.073+0000 7f390143c700 1 -- 172.21.15.36:0/2411210834 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec00b1e0 con 0x7f38fc0b65f0 2022-06-17T12:47:32.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.073+0000 7f390143c700 1 --2- 172.21.15.36:0/2411210834 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f38fc0b65f0 0x7f38fc134c70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f38f4006e00 tx=0x7f38f40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:32.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.073+0000 7f38f9ffb700 1 -- 172.21.15.36:0/2411210834 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38f400c640 con 0x7f38fc0b65f0 2022-06-17T12:47:32.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.073+0000 7f38f9ffb700 1 -- 172.21.15.36:0/2411210834 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f400c7a0 con 0x7f38fc0b65f0 2022-06-17T12:47:32.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.073+0000 7f3901c3d700 1 -- 172.21.15.36:0/2411210834 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38fc12f940 con 0x7f38fc0b65f0 2022-06-17T12:47:32.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.073+0000 7f38f9ffb700 1 -- 172.21.15.36:0/2411210834 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38f400cb00 con 0x7f38fc0b65f0 2022-06-17T12:47:32.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.073+0000 7f3901c3d700 1 -- 172.21.15.36:0/2411210834 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38fc063c30 con 0x7f38fc0b65f0 2022-06-17T12:47:32.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.077+0000 7f3901c3d700 1 -- 172.21.15.36:0/2411210834 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38fc13fb20 con 0x7f38fc0b65f0 2022-06-17T12:47:32.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.077+0000 7f38f9ffb700 1 -- 172.21.15.36:0/2411210834 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f38f400cc60 con 0x7f38fc0b65f0 2022-06-17T12:47:32.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.077+0000 7f38f9ffb700 1 --2- 172.21.15.36:0/2411210834 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f38dc053af0 0x7f38dc055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:32.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.077+0000 7f38f9ffb700 1 -- 172.21.15.36:0/2411210834 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f38f406cb40 con 0x7f38fc0b65f0 2022-06-17T12:47:32.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.077+0000 7f3900c3b700 1 --2- 172.21.15.36:0/2411210834 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f38dc053af0 0x7f38dc055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:32.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.081+0000 7f3900c3b700 1 --2- 172.21.15.36:0/2411210834 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f38dc053af0 0x7f38dc055fb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f38f000d000 tx=0x7f38f0007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:32.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.085+0000 7f38f9ffb700 1 -- 172.21.15.36:0/2411210834 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f38f4043830 con 0x7f38fc0b65f0 2022-06-17T12:47:32.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.213+0000 7f3901c3d700 1 -- 172.21.15.36:0/2411210834 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f38fc057410 con 0x7f38fc0b65f0 2022-06-17T12:47:32.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.213+0000 7f38f9ffb700 1 -- 172.21.15.36:0/2411210834 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+34065 (secure 0 0 0) 0x7f38f4028090 con 0x7f38fc0b65f0 2022-06-17T12:47:32.224 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:32.224 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:47:32.225 INFO:teuthology.orchestra.run.smithi036.stdout: "epoch": 26, 2022-06-17T12:47:32.225 INFO:teuthology.orchestra.run.smithi036.stdout: "fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", 2022-06-17T12:47:32.225 INFO:teuthology.orchestra.run.smithi036.stdout: "created": "2022-06-17T12:46:30.671408+0000", 2022-06-17T12:47:32.225 INFO:teuthology.orchestra.run.smithi036.stdout: "modified": "2022-06-17T12:46:53.258075+0000", 2022-06-17T12:47:32.288 INFO:teuthology.orchestra.run.smithi036.stdout: "last_up_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:47:32.302 INFO:teuthology.orchestra.run.smithi036.stdout: "last_in_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:47:32.302 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-17T12:47:32.303 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_num": 5799936, 2022-06-17T12:47:32.303 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_set": [ 2022-06-17T12:47:32.304 INFO:teuthology.orchestra.run.smithi036.stdout: "pglog_hardlimit", 2022-06-17T12:47:32.304 INFO:teuthology.orchestra.run.smithi036.stdout: "purged_snapdirs", 2022-06-17T12:47:32.305 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_deletes", 2022-06-17T12:47:32.305 INFO:teuthology.orchestra.run.smithi036.stdout: "sortbitwise" 2022-06-17T12:47:32.305 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:47:32.312 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_version": 6, 2022-06-17T12:47:32.312 INFO:teuthology.orchestra.run.smithi036.stdout: "full_ratio": 0.94999998807907104, 2022-06-17T12:47:32.313 INFO:teuthology.orchestra.run.smithi036.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-17T12:47:32.313 INFO:teuthology.orchestra.run.smithi036.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-17T12:47:32.330 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_snapshot": "", 2022-06-17T12:47:32.330 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_max": 4, 2022-06-17T12:47:32.331 INFO:teuthology.orchestra.run.smithi036.stdout: "max_osd": 8, 2022-06-17T12:47:32.331 INFO:teuthology.orchestra.run.smithi036.stdout: "require_min_compat_client": "jewel", 2022-06-17T12:47:32.331 INFO:teuthology.orchestra.run.smithi036.stdout: "min_compat_client": "jewel", 2022-06-17T12:47:32.332 INFO:teuthology.orchestra.run.smithi036.stdout: "require_osd_release": "octopus", 2022-06-17T12:47:32.332 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:47:32.332 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:32.332 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 1, 2022-06-17T12:47:32.333 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "device_health_metrics", 2022-06-17T12:47:32.333 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:33.881212+0000", 2022-06-17T12:47:32.333 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:47:32.333 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:47:32.334 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:47:32.334 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:47:32.338 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:47:32.339 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:47:32.339 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:47:32.339 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:47:32.340 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 1, 2022-06-17T12:47:32.340 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 1, 2022-06-17T12:47:32.341 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 1, 2022-06-17T12:47:32.341 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 1, 2022-06-17T12:47:32.341 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 1, 2022-06-17T12:47:32.341 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:47:32.342 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:47:32.342 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:47:32.342 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:47:32.342 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:47:32.343 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:47:32.343 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:47:32.343 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.343 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "16", 2022-06-17T12:47:32.343 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:47:32.344 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:47:32.344 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:47:32.344 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:47:32.344 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:47:32.345 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:47:32.345 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:47:32.346 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:47:32.347 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:47:32.347 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:47:32.347 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:47:32.347 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:47:32.348 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:47:32.348 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:47:32.348 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:47:32.348 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:47:32.349 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:47:32.349 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:47:32.349 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:47:32.349 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:47:32.350 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:47:32.351 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:47:32.351 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:47:32.351 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:47:32.352 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:47:32.352 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:47:32.352 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.352 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:47:32.353 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:47:32.353 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:47:32.353 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:47:32.353 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:47:32.354 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:47:32.362 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:47:32.363 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:47:32.363 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:47:32.363 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:47:32.364 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:47:32.364 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:47:32.364 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 1 2022-06-17T12:47:32.364 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.365 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:47:32.365 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr_devicehealth": {} 2022-06-17T12:47:32.366 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:32.366 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.366 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:32.366 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 2, 2022-06-17T12:47:32.366 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "rbd", 2022-06-17T12:47:32.367 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:44.485460+0000", 2022-06-17T12:47:32.367 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:47:32.367 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:47:32.367 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:47:32.368 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:47:32.370 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:47:32.371 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:47:32.371 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:47:32.371 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:47:32.372 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:47:32.372 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:47:32.372 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:47:32.372 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:47:32.373 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:47:32.373 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:47:32.373 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:47:32.374 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:47:32.374 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:47:32.374 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:47:32.374 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:47:32.375 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:47:32.375 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.375 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "19", 2022-06-17T12:47:32.375 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:47:32.376 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:47:32.376 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:47:32.376 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:47:32.376 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:47:32.377 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:47:32.377 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:47:32.384 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:47:32.384 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:47:32.385 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:47:32.387 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:47:32.387 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:47:32.387 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:47:32.388 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:47:32.388 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:47:32.389 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:47:32.411 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:47:32.412 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:47:32.413 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:47:32.423 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:47:32.424 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:47:32.424 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:47:32.440 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:47:32.441 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:47:32.441 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:47:32.442 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:47:32.442 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.471 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:47:32.471 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:47:32.472 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:47:32.473 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:47:32.473 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:47:32.474 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:47:32.475 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:47:32.475 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:47:32.476 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:47:32.476 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:47:32.477 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:47:32.477 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:47:32.491 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:47:32.491 INFO:teuthology.orchestra.run.smithi036.stdout: "rbd": {} 2022-06-17T12:47:32.492 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:32.519 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.519 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:32.520 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 3, 2022-06-17T12:47:32.520 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_metadata", 2022-06-17T12:47:32.521 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:47.465328+0000", 2022-06-17T12:47:32.521 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:47:32.551 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:47:32.575 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:47:32.575 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:47:32.576 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:47:32.577 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:47:32.578 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:47:32.578 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:47:32.578 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:47:32.579 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:47:32.625 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:47:32.626 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:47:32.626 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:47:32.627 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:47:32.627 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:47:32.628 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:47:32.628 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:47:32.628 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:47:32.629 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:47:32.629 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:47:32.630 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.630 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "25", 2022-06-17T12:47:32.630 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:47:32.631 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:47:32.631 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:47:32.632 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:47:32.632 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:47:32.633 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:47:32.633 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:47:32.633 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:47:32.634 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:47:32.634 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:47:32.646 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:47:32.661 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:47:32.682 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:47:32.682 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:47:32.682 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:47:32.683 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:47:32.683 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:47:32.684 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:47:32.684 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:47:32.685 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:47:32.685 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:47:32.686 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:47:32.686 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:47:32.687 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:47:32.687 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:47:32.688 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:47:32.688 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.689 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:47:32.699 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:47:32.700 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:47:32.706 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:47:32.707 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:47:32.707 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:47:32.708 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:47:32.710 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:47:32.710 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:47:32.711 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:47:32.711 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:47:32.712 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:47:32.713 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_bias": 4, 2022-06-17T12:47:32.713 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 16, 2022-06-17T12:47:32.714 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_priority": 5 2022-06-17T12:47:32.714 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.715 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:47:32.715 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:47:32.716 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": "cephfs" 2022-06-17T12:47:32.716 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:32.716 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:32.717 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.717 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:32.718 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 4, 2022-06-17T12:47:32.718 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_data", 2022-06-17T12:47:32.720 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:49.893338+0000", 2022-06-17T12:47:32.757 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 5, 2022-06-17T12:47:32.758 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool,ec_overwrites", 2022-06-17T12:47:32.758 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 3, 2022-06-17T12:47:32.759 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 4, 2022-06-17T12:47:32.769 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 3, 2022-06-17T12:47:32.771 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 1, 2022-06-17T12:47:32.771 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:47:32.772 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:47:32.772 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:47:32.773 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:47:32.773 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:47:32.773 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:47:32.774 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:47:32.774 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:47:32.775 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:47:32.775 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:47:32.786 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:47:32.786 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:47:32.787 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:47:32.787 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:47:32.787 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.799 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "26", 2022-06-17T12:47:32.800 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:47:32.801 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:47:32.801 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:47:32.801 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:47:32.807 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:47:32.807 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:47:32.808 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:47:32.829 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:47:32.829 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:47:32.830 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:47:32.830 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:47:32.831 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:47:32.832 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:47:32.832 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:47:32.833 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:47:32.833 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:47:32.847 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:47:32.847 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:47:32.847 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:47:32.848 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:47:32.851 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:47:32.851 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:47:32.852 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:47:32.852 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "cephfs_data", 2022-06-17T12:47:32.853 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:47:32.853 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:47:32.853 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.854 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:47:32.854 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:47:32.855 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:47:32.855 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:47:32.855 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:47:32.856 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:47:32.856 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:47:32.857 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:47:32.857 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 8192, 2022-06-17T12:47:32.857 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:47:32.858 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:47:32.859 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:47:32.867 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:47:32.868 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:47:32.868 INFO:teuthology.orchestra.run.smithi036.stdout: "data": "cephfs" 2022-06-17T12:47:32.896 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:32.896 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:32.898 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:32.908 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:47:32.909 INFO:teuthology.orchestra.run.smithi036.stdout: "osds": [ 2022-06-17T12:47:32.910 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:32.911 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:47:32.911 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", 2022-06-17T12:47:32.912 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:47:32.912 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:47:32.913 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:47:32.914 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:47:32.914 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:47:32.915 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:47:32.915 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:47:32.916 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 23, 2022-06-17T12:47:32.916 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:47:32.916 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:47:32.917 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:47:32.917 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:32.918 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:32.918 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:32.919 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6802", 2022-06-17T12:47:32.919 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:47:32.920 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.921 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:32.922 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:32.922 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6803", 2022-06-17T12:47:32.923 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:47:32.924 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:32.924 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:32.925 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.925 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:47:32.927 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:32.930 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:32.930 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:32.931 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6804", 2022-06-17T12:47:32.931 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:47:32.931 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.932 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:32.932 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:32.933 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6805", 2022-06-17T12:47:32.933 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:47:32.934 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:32.934 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:32.934 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.935 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:47:32.935 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:32.936 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:32.936 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:32.936 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6808", 2022-06-17T12:47:32.937 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:47:32.937 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.938 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:32.938 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:32.938 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6809", 2022-06-17T12:47:32.939 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:47:32.939 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:32.940 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:32.940 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.940 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:47:32.941 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:32.941 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:32.941 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:32.942 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6806", 2022-06-17T12:47:32.942 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:47:32.943 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.943 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:32.943 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:32.944 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6807", 2022-06-17T12:47:32.946 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:47:32.946 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:32.947 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:32.947 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.948 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6803/22619", 2022-06-17T12:47:32.948 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6805/22619", 2022-06-17T12:47:32.949 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6809/22619", 2022-06-17T12:47:32.949 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6807/22619", 2022-06-17T12:47:32.949 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:47:32.950 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:47:32.950 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:47:32.951 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:32.951 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.951 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:32.952 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:47:32.952 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", 2022-06-17T12:47:32.953 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:47:32.953 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:47:32.953 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:47:32.954 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:47:32.954 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:47:32.955 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:47:32.971 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:47:32.971 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 23, 2022-06-17T12:47:32.972 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:47:32.972 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:47:32.973 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:47:32.973 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:32.975 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:32.975 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:32.976 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6810", 2022-06-17T12:47:32.976 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:47:32.977 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.977 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:32.978 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:32.978 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6811", 2022-06-17T12:47:32.979 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:47:32.979 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:32.979 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:32.980 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.980 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:47:32.981 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:32.981 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:32.982 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:32.982 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6812", 2022-06-17T12:47:32.982 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:47:32.983 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.983 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:32.984 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:32.984 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6813", 2022-06-17T12:47:32.985 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:47:32.985 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:32.986 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:32.986 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:32.987 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:47:32.987 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:32.987 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:32.988 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:32.988 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6816", 2022-06-17T12:47:32.989 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:47:32.989 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.033 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.034 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.034 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6817", 2022-06-17T12:47:33.035 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:47:33.035 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.036 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.036 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.037 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:47:33.037 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.038 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.038 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.040 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6814", 2022-06-17T12:47:33.040 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:47:33.041 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.041 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.042 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.042 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6815", 2022-06-17T12:47:33.042 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:47:33.043 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.043 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.082 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.083 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6811/22620", 2022-06-17T12:47:33.083 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6813/22620", 2022-06-17T12:47:33.083 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6817/22620", 2022-06-17T12:47:33.083 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6815/22620", 2022-06-17T12:47:33.084 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:47:33.084 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:47:33.084 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:47:33.084 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.109 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.109 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.109 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:47:33.110 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", 2022-06-17T12:47:33.112 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:47:33.113 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:47:33.113 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:47:33.113 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:47:33.113 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:47:33.114 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:47:33.114 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:47:33.114 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 17, 2022-06-17T12:47:33.115 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:47:33.115 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:47:33.115 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:47:33.115 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.116 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.116 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.116 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6826", 2022-06-17T12:47:33.116 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:47:33.117 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.117 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.128 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.129 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6827", 2022-06-17T12:47:33.129 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:47:33.129 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.142 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.166 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.166 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:47:33.167 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.167 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.167 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.167 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6828", 2022-06-17T12:47:33.168 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:47:33.168 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.168 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.168 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.169 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6829", 2022-06-17T12:47:33.169 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:47:33.169 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.169 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.170 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.170 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:47:33.170 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.170 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.172 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.173 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6832", 2022-06-17T12:47:33.173 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:47:33.173 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.174 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.174 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.175 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6833", 2022-06-17T12:47:33.178 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:47:33.178 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.190 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.191 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.191 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:47:33.191 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.191 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.192 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.192 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6830", 2022-06-17T12:47:33.192 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:47:33.193 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.193 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.193 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.193 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6831", 2022-06-17T12:47:33.239 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:47:33.239 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.239 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.240 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.240 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6827/22643", 2022-06-17T12:47:33.240 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6829/22643", 2022-06-17T12:47:33.240 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6833/22643", 2022-06-17T12:47:33.240 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6831/22643", 2022-06-17T12:47:33.241 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:47:33.241 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:47:33.241 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:47:33.241 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.242 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.242 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.242 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:47:33.242 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", 2022-06-17T12:47:33.243 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:47:33.243 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:47:33.243 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:47:33.244 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:47:33.244 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:47:33.244 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:47:33.244 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:47:33.245 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 23, 2022-06-17T12:47:33.245 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:47:33.245 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:47:33.245 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:47:33.246 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.246 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.246 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.247 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6818", 2022-06-17T12:47:33.247 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:47:33.247 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.247 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.248 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.248 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6819", 2022-06-17T12:47:33.248 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:47:33.248 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.248 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.249 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.249 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:47:33.249 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.249 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.250 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.250 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6820", 2022-06-17T12:47:33.250 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:47:33.250 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.251 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.251 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.251 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6821", 2022-06-17T12:47:33.251 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:47:33.252 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.252 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.252 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.262 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:47:33.263 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.263 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.263 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.264 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6824", 2022-06-17T12:47:33.264 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:47:33.264 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.265 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.265 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.265 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6825", 2022-06-17T12:47:33.265 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:47:33.266 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.266 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.266 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.266 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:47:33.267 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.267 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.268 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.268 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6822", 2022-06-17T12:47:33.270 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:47:33.270 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.286 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.287 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.287 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6823", 2022-06-17T12:47:33.308 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:47:33.308 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.308 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.309 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.309 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6819/22646", 2022-06-17T12:47:33.310 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6821/22646", 2022-06-17T12:47:33.310 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6825/22646", 2022-06-17T12:47:33.310 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6823/22646", 2022-06-17T12:47:33.311 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:47:33.311 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:47:33.312 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:47:33.312 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.312 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.313 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.313 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:47:33.338 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", 2022-06-17T12:47:33.338 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:47:33.339 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:47:33.339 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:47:33.339 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:47:33.339 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:47:33.340 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:47:33.340 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:47:33.340 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 23, 2022-06-17T12:47:33.340 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:47:33.340 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:47:33.341 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:47:33.341 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.341 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.341 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.341 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6808", 2022-06-17T12:47:33.342 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:47:33.342 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.342 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.342 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.343 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6809", 2022-06-17T12:47:33.344 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:47:33.344 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.345 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.345 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.346 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:47:33.347 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.347 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.347 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.347 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6810", 2022-06-17T12:47:33.349 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:47:33.349 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.350 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.350 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.350 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6811", 2022-06-17T12:47:33.350 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:47:33.350 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.351 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.351 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.351 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:47:33.351 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.352 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.352 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.352 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6814", 2022-06-17T12:47:33.352 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:47:33.352 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.353 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.353 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.353 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6815", 2022-06-17T12:47:33.355 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:47:33.356 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.357 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.357 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.358 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:47:33.358 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.359 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.360 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.360 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6812", 2022-06-17T12:47:33.360 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:47:33.360 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.361 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.361 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.361 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6813", 2022-06-17T12:47:33.361 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:47:33.362 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.362 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.362 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.362 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6809/22812", 2022-06-17T12:47:33.363 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6811/22812", 2022-06-17T12:47:33.363 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6815/22812", 2022-06-17T12:47:33.363 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6813/22812", 2022-06-17T12:47:33.363 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:47:33.364 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:47:33.364 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:47:33.365 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.365 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.366 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.367 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:47:33.368 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", 2022-06-17T12:47:33.368 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:47:33.368 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:47:33.368 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:47:33.369 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:47:33.369 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:47:33.369 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:47:33.369 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:47:33.370 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 20, 2022-06-17T12:47:33.370 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:47:33.370 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:47:33.370 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:47:33.371 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.371 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.371 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.371 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6800", 2022-06-17T12:47:33.371 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:47:33.372 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.372 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.372 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.372 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6801", 2022-06-17T12:47:33.373 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:47:33.373 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.373 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.373 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.384 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:47:33.384 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.384 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.407 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.408 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6802", 2022-06-17T12:47:33.408 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:47:33.408 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.409 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.409 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.409 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6803", 2022-06-17T12:47:33.409 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:47:33.410 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.413 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.413 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.418 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:47:33.420 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.420 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.420 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.420 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6806", 2022-06-17T12:47:33.421 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:47:33.421 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.421 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.421 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.422 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6807", 2022-06-17T12:47:33.422 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:47:33.422 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.422 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.422 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.423 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:47:33.423 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.423 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.423 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.424 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6804", 2022-06-17T12:47:33.424 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:47:33.424 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.424 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.424 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.425 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6805", 2022-06-17T12:47:33.425 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:47:33.425 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.444 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.444 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.444 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6801/22813", 2022-06-17T12:47:33.444 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6803/22813", 2022-06-17T12:47:33.445 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6807/22813", 2022-06-17T12:47:33.445 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6805/22813", 2022-06-17T12:47:33.445 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:47:33.445 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:47:33.445 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:47:33.446 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.446 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.446 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.446 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:47:33.447 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", 2022-06-17T12:47:33.447 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:47:33.447 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:47:33.447 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:47:33.447 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:47:33.448 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:47:33.448 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:47:33.448 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:47:33.448 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 23, 2022-06-17T12:47:33.449 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:47:33.449 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:47:33.449 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:47:33.449 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.450 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.450 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.450 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6816", 2022-06-17T12:47:33.450 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:47:33.451 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.452 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.463 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.471 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6817", 2022-06-17T12:47:33.492 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:47:33.499 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.499 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.500 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.500 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:47:33.501 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.501 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.502 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.502 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6818", 2022-06-17T12:47:33.503 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:47:33.529 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.529 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.539 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.540 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6819", 2022-06-17T12:47:33.540 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:47:33.541 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.541 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.579 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.579 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:47:33.580 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.580 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.580 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.581 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6822", 2022-06-17T12:47:33.581 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:47:33.594 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.594 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.595 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.595 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6823", 2022-06-17T12:47:33.596 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:47:33.596 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.597 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.597 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.597 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:47:33.609 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.614 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.614 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.615 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6820", 2022-06-17T12:47:33.615 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:47:33.615 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.616 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.616 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.617 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6821", 2022-06-17T12:47:33.617 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:47:33.619 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.620 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.620 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.621 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6817/22822", 2022-06-17T12:47:33.621 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6819/22822", 2022-06-17T12:47:33.622 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6823/22822", 2022-06-17T12:47:33.623 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6821/22822", 2022-06-17T12:47:33.624 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:47:33.624 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:47:33.625 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:47:33.625 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.626 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.627 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.627 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:47:33.628 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", 2022-06-17T12:47:33.628 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:47:33.629 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:47:33.629 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:47:33.638 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:47:33.638 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:47:33.639 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:47:33.639 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:47:33.640 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 17, 2022-06-17T12:47:33.640 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:47:33.646 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:47:33.647 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:47:33.647 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.647 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.648 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.648 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6824", 2022-06-17T12:47:33.649 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:47:33.649 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.659 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.661 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.662 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6825", 2022-06-17T12:47:33.662 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:47:33.663 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.663 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.664 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.664 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:47:33.665 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.666 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.667 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.667 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6826", 2022-06-17T12:47:33.668 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:47:33.668 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.669 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.669 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.669 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6827", 2022-06-17T12:47:33.670 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:47:33.671 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.671 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.672 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.672 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:47:33.672 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.673 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.673 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.696 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6830", 2022-06-17T12:47:33.697 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:47:33.698 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.699 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.699 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.700 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6831", 2022-06-17T12:47:33.701 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:47:33.701 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.702 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.702 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.702 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:47:33.703 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:47:33.703 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.704 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:47:33.704 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6828", 2022-06-17T12:47:33.704 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:47:33.705 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.705 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.705 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:47:33.706 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6829", 2022-06-17T12:47:33.708 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:47:33.709 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.710 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.710 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.711 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6825/22837", 2022-06-17T12:47:33.712 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6827/22837", 2022-06-17T12:47:33.713 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6831/22837", 2022-06-17T12:47:33.713 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6829/22837", 2022-06-17T12:47:33.713 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:47:33.714 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:47:33.714 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:47:33.715 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:33.716 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.716 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:47:33.717 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_xinfo": [ 2022-06-17T12:47:33.717 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.717 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:47:33.729 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:47:33.731 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:47:33.731 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:47:33.732 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:47:33.743 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:47:33.746 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.291461+0000", 2022-06-17T12:47:33.746 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:47:33.747 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.747 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.748 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:47:33.748 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:47:33.748 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:47:33.748 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:47:33.749 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:47:33.749 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:47:33.749 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.537912+0000", 2022-06-17T12:47:33.750 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:47:33.750 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.750 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.751 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:47:33.751 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:47:33.751 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:47:33.751 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:47:33.752 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:47:33.752 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:47:33.752 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.606947+0000", 2022-06-17T12:47:33.752 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:47:33.752 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.753 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.753 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:47:33.753 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:47:33.753 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:47:33.772 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:47:33.772 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:47:33.773 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:47:33.773 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.514412+0000", 2022-06-17T12:47:33.773 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:47:33.773 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.773 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.788 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:47:33.788 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:47:33.788 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:47:33.789 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:47:33.789 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:47:33.789 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:47:33.789 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.394645+0000", 2022-06-17T12:47:33.790 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:47:33.790 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.790 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.790 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:47:33.790 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:47:33.791 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:47:33.791 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:47:33.792 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:47:33.810 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:47:33.810 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.343806+0000", 2022-06-17T12:47:33.815 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:47:33.816 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.816 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.816 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:47:33.824 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:47:33.824 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:47:33.824 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:47:33.844 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:47:33.844 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:47:33.845 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.591864+0000", 2022-06-17T12:47:33.845 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:47:33.846 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.846 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:33.846 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:47:33.846 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:47:33.847 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:47:33.847 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:47:33.847 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:47:33.847 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:47:33.847 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.607045+0000", 2022-06-17T12:47:33.848 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:47:33.848 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.848 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:47:33.848 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap": [], 2022-06-17T12:47:33.849 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap_items": [], 2022-06-17T12:47:33.849 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_temp": [], 2022-06-17T12:47:33.865 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_temp": [], 2022-06-17T12:47:33.866 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklist": {}, 2022-06-17T12:47:33.867 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profiles": { 2022-06-17T12:47:33.867 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs_data": { 2022-06-17T12:47:33.867 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-device-class": "", 2022-06-17T12:47:33.868 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:47:33.868 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-root": "default", 2022-06-17T12:47:33.868 INFO:teuthology.orchestra.run.smithi036.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-17T12:47:33.869 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:47:33.869 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "2", 2022-06-17T12:47:33.870 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:47:33.916 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:47:33.916 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van", 2022-06-17T12:47:33.916 INFO:teuthology.orchestra.run.smithi036.stdout: "w": "8" 2022-06-17T12:47:33.917 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.917 INFO:teuthology.orchestra.run.smithi036.stdout: "default": { 2022-06-17T12:47:33.917 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:47:33.917 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:47:33.937 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "1", 2022-06-17T12:47:33.937 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:47:33.937 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:47:33.940 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van" 2022-06-17T12:47:33.941 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:33.941 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:33.941 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps_queue": [], 2022-06-17T12:47:33.941 INFO:teuthology.orchestra.run.smithi036.stdout: "new_removed_snaps": [], 2022-06-17T12:47:33.941 INFO:teuthology.orchestra.run.smithi036.stdout: "new_purged_snaps": [], 2022-06-17T12:47:33.942 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_node_flags": {}, 2022-06-17T12:47:33.942 INFO:teuthology.orchestra.run.smithi036.stdout: "device_class_flags": {} 2022-06-17T12:47:33.942 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:47:33.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.217+0000 7f3901c3d700 1 -- 172.21.15.36:0/2411210834 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f38dc053af0 msgr2=0x7f38dc055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:33.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.217+0000 7f3901c3d700 1 --2- 172.21.15.36:0/2411210834 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f38dc053af0 0x7f38dc055fb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f38f000d000 tx=0x7f38f0007b90).stop 2022-06-17T12:47:33.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.217+0000 7f3901c3d700 1 -- 172.21.15.36:0/2411210834 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f38fc0b65f0 msgr2=0x7f38fc134c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:33.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.217+0000 7f3901c3d700 1 --2- 172.21.15.36:0/2411210834 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f38fc0b65f0 0x7f38fc134c70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f38f4006e00 tx=0x7f38f40091d0).stop 2022-06-17T12:47:33.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.217+0000 7f3901c3d700 1 -- 172.21.15.36:0/2411210834 shutdown_connections 2022-06-17T12:47:33.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.217+0000 7f3901c3d700 1 --2- 172.21.15.36:0/2411210834 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f38dc053af0 0x7f38dc055fb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:33.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.217+0000 7f3901c3d700 1 --2- 172.21.15.36:0/2411210834 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f38fc0b65f0 0x7f38fc134c70 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:34.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.217+0000 7f3901c3d700 1 --2- 172.21.15.36:0/2411210834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38fc0b15b0 0x7f38fc134780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:34.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.217+0000 7f3901c3d700 1 --2- 172.21.15.36:0/2411210834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38fc0adf10 0x7f38fc05c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:34.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.217+0000 7f3901c3d700 1 -- 172.21.15.36:0/2411210834 >> 172.21.15.36:0/2411210834 conn(0x7f38fc0614b0 msgr2=0x7f38fc139740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:34.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.217+0000 7f3901c3d700 1 -- 172.21.15.36:0/2411210834 shutdown_connections 2022-06-17T12:47:34.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:32.217+0000 7f3901c3d700 1 -- 172.21.15.36:0/2411210834 wait complete. 2022-06-17T12:47:34.005 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:47:34.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.169+0000 7f35e75a5700 1 Processor -- start 2022-06-17T12:47:34.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.169+0000 7f35e75a5700 1 -- start start 2022-06-17T12:47:34.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35e75a5700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f35e0144c00 0x7f35e0062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:34.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35e75a5700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f35e00ad330 0x7f35e00ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:34.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35e75a5700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e00b0ba0 con 0x7f35e0062770 2022-06-17T12:47:34.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35e75a5700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e00ae820 con 0x7f35e00ad330 2022-06-17T12:47:34.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35e75a5700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e0061030 con 0x7f35e0144c00 2022-06-17T12:47:34.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35e5b42700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f35e00ad330 0x7f35e00ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:34.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35e5341700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f35e0062770 0x7f35e01444f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49880/0 (socket says 172.21.15.36:49880) 2022-06-17T12:47:34.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35e5341700 1 -- 172.21.15.36:0/860488408 learned_addr learned my addr 172.21.15.36:0/860488408 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:34.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35e4b40700 1 --2- 172.21.15.36:0/860488408 >> v2:172.21.15.70:3300/0 conn(0x7f35e0144c00 0x7f35e0062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:34.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35e5b42700 1 -- 172.21.15.36:0/860488408 >> v2:172.21.15.70:3300/0 conn(0x7f35e0144c00 msgr2=0x7f35e0062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:34.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35e5b42700 1 --2- 172.21.15.36:0/860488408 >> v2:172.21.15.70:3300/0 conn(0x7f35e0144c00 0x7f35e0062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:34.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35e5b42700 1 -- 172.21.15.36:0/860488408 >> v1:172.21.15.36:6789/0 conn(0x7f35e0062770 legacy=0x7f35e01444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:34.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35e5b42700 1 -- 172.21.15.36:0/860488408 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e0147e90 con 0x7f35e00ad330 2022-06-17T12:47:34.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35e4b40700 1 --2- 172.21.15.36:0/860488408 >> v2:172.21.15.70:3300/0 conn(0x7f35e0144c00 0x7f35e0062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:47:34.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35e5b42700 1 --2- 172.21.15.36:0/860488408 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f35e00ad330 0x7f35e00ad7c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f35e0061170 tx=0x7f35d0006df0).ready entity=mon.0 client_cookie=d0e10a458951ef56 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:34.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35dffff700 1 -- 172.21.15.36:0/860488408 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d000d5c0 con 0x7f35e00ad330 2022-06-17T12:47:34.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35dffff700 1 -- 172.21.15.36:0/860488408 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d0000e90 con 0x7f35e00ad330 2022-06-17T12:47:34.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35dffff700 1 -- 172.21.15.36:0/860488408 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d000d910 con 0x7f35e00ad330 2022-06-17T12:47:34.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35e75a5700 1 -- 172.21.15.36:0/860488408 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f35e00ad330 msgr2=0x7f35e00ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:34.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35e75a5700 1 --2- 172.21.15.36:0/860488408 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f35e00ad330 0x7f35e00ad7c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f35e0061170 tx=0x7f35d0006df0).stop 2022-06-17T12:47:34.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35e75a5700 1 -- 172.21.15.36:0/860488408 shutdown_connections 2022-06-17T12:47:34.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35e75a5700 1 --2- 172.21.15.36:0/860488408 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f35e00ad330 0x7f35e00ad7c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:34.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35e75a5700 1 --2- 172.21.15.36:0/860488408 >> v2:172.21.15.70:3300/0 conn(0x7f35e0144c00 0x7f35e0062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:34.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35e75a5700 1 -- 172.21.15.36:0/860488408 >> 172.21.15.36:0/860488408 conn(0x7f35e00b53e0 msgr2=0x7f35e00b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:34.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35e75a5700 1 -- 172.21.15.36:0/860488408 shutdown_connections 2022-06-17T12:47:34.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.173+0000 7f35e75a5700 1 -- 172.21.15.36:0/860488408 wait complete. 2022-06-17T12:47:34.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.177+0000 7f35e75a5700 1 Processor -- start 2022-06-17T12:47:34.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.177+0000 7f35e75a5700 1 -- start start 2022-06-17T12:47:34.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.177+0000 7f35e75a5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35e0062770 0x7f35e00639d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:34.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.177+0000 7f35e75a5700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35e00ad330 0x7f35e0156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:34.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.177+0000 7f35e75a5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f35e0144c00 0x7f35e0157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:34.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.177+0000 7f35e75a5700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e01568e0 con 0x7f35e0144c00 2022-06-17T12:47:34.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.177+0000 7f35e75a5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e0065af0 con 0x7f35e0062770 2022-06-17T12:47:34.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.177+0000 7f35e75a5700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e0065850 con 0x7f35e00ad330 2022-06-17T12:47:34.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.177+0000 7f35e5b42700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f35e0144c00 0x7f35e0157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:34.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.181+0000 7f35e5b42700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f35e0144c00 0x7f35e0157230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60592/0 (socket says 172.21.15.36:60592) 2022-06-17T12:47:34.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.181+0000 7f35e5b42700 1 -- 172.21.15.36:0/2086244522 learned_addr learned my addr 172.21.15.36:0/2086244522 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:34.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.181+0000 7f35e4b40700 1 --2- 172.21.15.36:0/2086244522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35e00ad330 0x7f35e0156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:34.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.181+0000 7f35e5341700 1 --2- 172.21.15.36:0/2086244522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35e0062770 0x7f35e00639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:34.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.181+0000 7f35e5b42700 1 -- 172.21.15.36:0/2086244522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35e00ad330 msgr2=0x7f35e0156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:34.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.181+0000 7f35e5b42700 1 --2- 172.21.15.36:0/2086244522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35e00ad330 0x7f35e0156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:34.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.181+0000 7f35e5b42700 1 -- 172.21.15.36:0/2086244522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35e0062770 msgr2=0x7f35e00639d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:34.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.181+0000 7f35e5b42700 1 --2- 172.21.15.36:0/2086244522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35e0062770 0x7f35e00639d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:34.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.181+0000 7f35e5b42700 1 -- 172.21.15.36:0/2086244522 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d000b180 con 0x7f35e0144c00 2022-06-17T12:47:34.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.181+0000 7f35e5b42700 1 --2- 172.21.15.36:0/2086244522 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f35e0144c00 0x7f35e0157230 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f35e0061170 tx=0x7f35d0012000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:34.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.181+0000 7f35de7fc700 1 -- 172.21.15.36:0/2086244522 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d0016070 con 0x7f35e0144c00 2022-06-17T12:47:34.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.181+0000 7f35de7fc700 1 -- 172.21.15.36:0/2086244522 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d000d940 con 0x7f35e0144c00 2022-06-17T12:47:34.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.181+0000 7f35e75a5700 1 -- 172.21.15.36:0/2086244522 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e014b5d0 con 0x7f35e0144c00 2022-06-17T12:47:34.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.181+0000 7f35de7fc700 1 -- 172.21.15.36:0/2086244522 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d0013de0 con 0x7f35e0144c00 2022-06-17T12:47:34.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.181+0000 7f35e75a5700 1 -- 172.21.15.36:0/2086244522 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e0157720 con 0x7f35e0144c00 2022-06-17T12:47:34.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.181+0000 7f35e75a5700 1 -- 172.21.15.36:0/2086244522 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e00656b0 con 0x7f35e0144c00 2022-06-17T12:47:34.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.181+0000 7f35de7fc700 1 -- 172.21.15.36:0/2086244522 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f35d0025030 con 0x7f35e0144c00 2022-06-17T12:47:34.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.185+0000 7f35de7fc700 1 --2- 172.21.15.36:0/2086244522 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f35c0053ad0 0x7f35c0055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:34.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.185+0000 7f35de7fc700 1 -- 172.21.15.36:0/2086244522 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f35d006d170 con 0x7f35e0144c00 2022-06-17T12:47:34.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.185+0000 7f35e5341700 1 --2- 172.21.15.36:0/2086244522 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f35c0053ad0 0x7f35c0055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:34.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.185+0000 7f35e5341700 1 --2- 172.21.15.36:0/2086244522 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f35c0053ad0 0x7f35c0055f90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f35d800d000 tx=0x7f35d8009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:34.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.189+0000 7f35de7fc700 1 -- 172.21.15.36:0/2086244522 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f35e00656b0 con 0x7f35e0144c00 2022-06-17T12:47:34.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.353+0000 7f35e75a5700 1 -- 172.21.15.36:0/2086244522 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35e00656b0 con 0x7f35e0144c00 2022-06-17T12:47:34.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.353+0000 7f35de7fc700 1 -- 172.21.15.36:0/2086244522 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2134 (secure 0 0 0) 0x7f35e00656b0 con 0x7f35e0144c00 2022-06-17T12:47:34.580 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:34.581 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/4243909763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":4243909763},{"type":"v1","addr":"172.21.15.36:6835","nonce":4243909763}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:46:52.253609+0000","modified":"2022-06-17T12:46:54.268223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/3425584732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3425584732},{"type":"v1","addr":"172.21.15.70:6833","nonce":3425584732}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-06-17T12:47:34.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.353+0000 7f35e75a5700 1 -- 172.21.15.36:0/2086244522 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f35c0053ad0 msgr2=0x7f35c0055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:34.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.353+0000 7f35e75a5700 1 --2- 172.21.15.36:0/2086244522 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f35c0053ad0 0x7f35c0055f90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f35d800d000 tx=0x7f35d8009040).stop 2022-06-17T12:47:34.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.353+0000 7f35e75a5700 1 -- 172.21.15.36:0/2086244522 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f35e0144c00 msgr2=0x7f35e0157230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:34.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.353+0000 7f35e75a5700 1 --2- 172.21.15.36:0/2086244522 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f35e0144c00 0x7f35e0157230 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f35e0061170 tx=0x7f35d0012000).stop 2022-06-17T12:47:34.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.353+0000 7f35e75a5700 1 -- 172.21.15.36:0/2086244522 shutdown_connections 2022-06-17T12:47:34.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.353+0000 7f35e75a5700 1 --2- 172.21.15.36:0/2086244522 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f35c0053ad0 0x7f35c0055f90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:34.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.353+0000 7f35e75a5700 1 --2- 172.21.15.36:0/2086244522 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f35e0144c00 0x7f35e0157230 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:34.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.353+0000 7f35e75a5700 1 --2- 172.21.15.36:0/2086244522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35e00ad330 0x7f35e0156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:34.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.353+0000 7f35e75a5700 1 --2- 172.21.15.36:0/2086244522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35e0062770 0x7f35e00639d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:34.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.353+0000 7f35e75a5700 1 -- 172.21.15.36:0/2086244522 >> 172.21.15.36:0/2086244522 conn(0x7f35e00b53e0 msgr2=0x7f35e00b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:34.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.353+0000 7f35e75a5700 1 -- 172.21.15.36:0/2086244522 shutdown_connections 2022-06-17T12:47:34.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.353+0000 7f35e75a5700 1 -- 172.21.15.36:0/2086244522 wait complete. 2022-06-17T12:47:34.679 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 7 2022-06-17T12:47:34.680 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-17T12:47:34.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.813+0000 7f8e5cf85700 1 Processor -- start 2022-06-17T12:47:34.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.813+0000 7f8e5cf85700 1 -- start start 2022-06-17T12:47:34.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.813+0000 7f8e5cf85700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8e58144c00 0x7f8e58062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:34.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.813+0000 7f8e5cf85700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e580b0ba0 con 0x7f8e580ad330 2022-06-17T12:47:34.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.813+0000 7f8e5cf85700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e58147e90 con 0x7f8e58062770 2022-06-17T12:47:34.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.813+0000 7f8e5cf85700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e580ae820 con 0x7f8e58144c00 2022-06-17T12:47:34.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.813+0000 7f8e55d9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8e58144c00 0x7f8e58062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:34.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.813+0000 7f8e55d9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8e58144c00 0x7f8e58062b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60596/0 (socket says 172.21.15.36:60596) 2022-06-17T12:47:34.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.813+0000 7f8e55d9b700 1 -- 172.21.15.36:0/3204848456 learned_addr learned my addr 172.21.15.36:0/3204848456 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:34.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.813+0000 7f8e56d9d700 1 --1- 172.21.15.36:0/3204848456 >> v1:172.21.15.36:6789/0 conn(0x7f8e580ad330 0x7f8e580ad7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49896/0 (socket says 172.21.15.36:49896) 2022-06-17T12:47:34.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.813+0000 7f8e55d9b700 1 -- 172.21.15.36:0/3204848456 >> v1:172.21.15.70:6789/0 conn(0x7f8e58062770 legacy=0x7f8e581444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:34.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.813+0000 7f8e55d9b700 1 -- 172.21.15.36:0/3204848456 >> v1:172.21.15.36:6789/0 conn(0x7f8e580ad330 legacy=0x7f8e580ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:34.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.813+0000 7f8e5559a700 1 -- 172.21.15.36:0/3204848456 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3514359544 0 0) 0x7f8e580b0ba0 con 0x7f8e580ad330 2022-06-17T12:47:34.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.813+0000 7f8e55d9b700 1 -- 172.21.15.36:0/3204848456 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e580b0dd0 con 0x7f8e58144c00 2022-06-17T12:47:34.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.817+0000 7f8e55d9b700 1 --2- 172.21.15.36:0/3204848456 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e58144c00 0x7f8e58062b50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f8e48002f70 tx=0x7f8e48006eb0).ready entity=mon.0 client_cookie=13e84d35a0a90809 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:34.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.817+0000 7f8e5559a700 1 -- 172.21.15.36:0/3204848456 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e4800d660 con 0x7f8e58144c00 2022-06-17T12:47:34.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.817+0000 7f8e5559a700 1 -- 172.21.15.36:0/3204848456 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e48000e90 con 0x7f8e58144c00 2022-06-17T12:47:34.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.817+0000 7f8e5559a700 1 -- 172.21.15.36:0/3204848456 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e4800d9b0 con 0x7f8e58144c00 2022-06-17T12:47:34.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.817+0000 7f8e5cf85700 1 -- 172.21.15.36:0/3204848456 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e58144c00 msgr2=0x7f8e58062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:34.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.817+0000 7f8e5cf85700 1 --2- 172.21.15.36:0/3204848456 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e58144c00 0x7f8e58062b50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f8e48002f70 tx=0x7f8e48006eb0).stop 2022-06-17T12:47:34.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.817+0000 7f8e5cf85700 1 -- 172.21.15.36:0/3204848456 shutdown_connections 2022-06-17T12:47:34.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.817+0000 7f8e5cf85700 1 --2- 172.21.15.36:0/3204848456 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e58144c00 0x7f8e58062b50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:34.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.817+0000 7f8e5cf85700 1 -- 172.21.15.36:0/3204848456 >> 172.21.15.36:0/3204848456 conn(0x7f8e580b53e0 msgr2=0x7f8e580b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:34.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.817+0000 7f8e5cf85700 1 -- 172.21.15.36:0/3204848456 shutdown_connections 2022-06-17T12:47:34.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.817+0000 7f8e5cf85700 1 -- 172.21.15.36:0/3204848456 wait complete. 2022-06-17T12:47:34.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.821+0000 7f8e5cf85700 1 Processor -- start 2022-06-17T12:47:34.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.821+0000 7f8e5cf85700 1 -- start start 2022-06-17T12:47:34.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.821+0000 7f8e5cf85700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e58062770 0x7f8e581565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:34.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.821+0000 7f8e5cf85700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e580ad330 0x7f8e58066b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:34.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.821+0000 7f8e5cf85700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e58144c00 0x7f8e5814e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:34.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.821+0000 7f8e5cf85700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e58064810 con 0x7f8e58144c00 2022-06-17T12:47:34.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.821+0000 7f8e5cf85700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e58064a40 con 0x7f8e580ad330 2022-06-17T12:47:34.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.821+0000 7f8e5cf85700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e58064ce0 con 0x7f8e58062770 2022-06-17T12:47:34.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.821+0000 7f8e56d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e58144c00 0x7f8e5814e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:34.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.821+0000 7f8e56d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e58144c00 0x7f8e5814e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60604/0 (socket says 172.21.15.36:60604) 2022-06-17T12:47:34.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.821+0000 7f8e56d9d700 1 -- 172.21.15.36:0/294832738 learned_addr learned my addr 172.21.15.36:0/294832738 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:34.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.821+0000 7f8e5659c700 1 --2- 172.21.15.36:0/294832738 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e58062770 0x7f8e581565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:34.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.821+0000 7f8e55d9b700 1 --2- 172.21.15.36:0/294832738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e580ad330 0x7f8e58066b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:34.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.821+0000 7f8e56d9d700 1 -- 172.21.15.36:0/294832738 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e58062770 msgr2=0x7f8e581565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:34.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.821+0000 7f8e56d9d700 1 --2- 172.21.15.36:0/294832738 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e58062770 0x7f8e581565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:34.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.821+0000 7f8e56d9d700 1 -- 172.21.15.36:0/294832738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e580ad330 msgr2=0x7f8e58066b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:34.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.821+0000 7f8e56d9d700 1 --2- 172.21.15.36:0/294832738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e580ad330 0x7f8e58066b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:34.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.821+0000 7f8e56d9d700 1 -- 172.21.15.36:0/294832738 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4800b1e0 con 0x7f8e58144c00 2022-06-17T12:47:34.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.821+0000 7f8e56d9d700 1 --2- 172.21.15.36:0/294832738 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e58144c00 0x7f8e5814e660 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f8e50000f40 tx=0x7f8e50007d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:34.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.821+0000 7f8e477fe700 1 -- 172.21.15.36:0/294832738 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e50005ae0 con 0x7f8e58144c00 2022-06-17T12:47:34.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.821+0000 7f8e477fe700 1 -- 172.21.15.36:0/294832738 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e50005c40 con 0x7f8e58144c00 2022-06-17T12:47:34.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.821+0000 7f8e5cf85700 1 -- 172.21.15.36:0/294832738 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e58064210 con 0x7f8e58144c00 2022-06-17T12:47:34.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.821+0000 7f8e477fe700 1 -- 172.21.15.36:0/294832738 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e500070d0 con 0x7f8e58144c00 2022-06-17T12:47:34.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.821+0000 7f8e5cf85700 1 -- 172.21.15.36:0/294832738 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e58064fb0 con 0x7f8e58144c00 2022-06-17T12:47:34.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.825+0000 7f8e5cf85700 1 -- 172.21.15.36:0/294832738 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e580b1c20 con 0x7f8e58144c00 2022-06-17T12:47:34.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.825+0000 7f8e477fe700 1 -- 172.21.15.36:0/294832738 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8e5001a060 con 0x7f8e58144c00 2022-06-17T12:47:34.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.825+0000 7f8e477fe700 1 --2- 172.21.15.36:0/294832738 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8e34053ad0 0x7f8e34055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:34.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.825+0000 7f8e477fe700 1 -- 172.21.15.36:0/294832738 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f8e5006bae0 con 0x7f8e58144c00 2022-06-17T12:47:34.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.825+0000 7f8e5659c700 1 --2- 172.21.15.36:0/294832738 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8e34053ad0 0x7f8e34055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:34.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.825+0000 7f8e5659c700 1 --2- 172.21.15.36:0/294832738 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8e34053ad0 0x7f8e34055f90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f8e4c00d000 tx=0x7f8e4c007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:35.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.833+0000 7f8e477fe700 1 -- 172.21.15.36:0/294832738 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8e500427d0 con 0x7f8e58144c00 2022-06-17T12:47:35.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:34.997+0000 7f8e5cf85700 1 -- 172.21.15.36:0/294832738 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8e58057410 con 0x7f8e58144c00 2022-06-17T12:47:35.637 INFO:tasks.ceph.mon.a.smithi036.stderr:2022-06-17T12:47:35.633+0000 7fb7a2649700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-17T12:47:35.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.637+0000 7f8e477fe700 1 -- 172.21.15.36:0/294832738 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8e50018030 con 0x7f8e58144c00 2022-06-17T12:47:35.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.641+0000 7f8e5cf85700 1 -- 172.21.15.36:0/294832738 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8e34053ad0 msgr2=0x7f8e34055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:35.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.641+0000 7f8e5cf85700 1 --2- 172.21.15.36:0/294832738 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8e34053ad0 0x7f8e34055f90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f8e4c00d000 tx=0x7f8e4c007b90).stop 2022-06-17T12:47:35.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.641+0000 7f8e5cf85700 1 -- 172.21.15.36:0/294832738 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e58144c00 msgr2=0x7f8e5814e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:35.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.641+0000 7f8e5cf85700 1 --2- 172.21.15.36:0/294832738 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e58144c00 0x7f8e5814e660 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f8e50000f40 tx=0x7f8e50007d60).stop 2022-06-17T12:47:35.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.641+0000 7f8e5cf85700 1 -- 172.21.15.36:0/294832738 shutdown_connections 2022-06-17T12:47:35.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.641+0000 7f8e5cf85700 1 --2- 172.21.15.36:0/294832738 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8e34053ad0 0x7f8e34055f90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:35.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.641+0000 7f8e5cf85700 1 --2- 172.21.15.36:0/294832738 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e58144c00 0x7f8e5814e660 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:35.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.641+0000 7f8e5cf85700 1 --2- 172.21.15.36:0/294832738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e580ad330 0x7f8e58066b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:35.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.641+0000 7f8e5cf85700 1 --2- 172.21.15.36:0/294832738 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e58062770 0x7f8e581565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:35.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.641+0000 7f8e5cf85700 1 -- 172.21.15.36:0/294832738 >> 172.21.15.36:0/294832738 conn(0x7f8e580b53e0 msgr2=0x7f8e580aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:35.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.641+0000 7f8e5cf85700 1 -- 172.21.15.36:0/294832738 shutdown_connections 2022-06-17T12:47:35.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.641+0000 7f8e5cf85700 1 -- 172.21.15.36:0/294832738 wait complete. 2022-06-17T12:47:35.650 INFO:teuthology.orchestra.run.smithi036.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-17T12:47:35.662 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2022-06-17T12:47:35.664 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:47:35.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa37116d700 1 Processor -- start 2022-06-17T12:47:35.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa37116d700 1 -- start start 2022-06-17T12:47:35.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa37116d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa36c062770 0x7fa36c062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:35.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa37116d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa36c0ad330 0x7fa36c0ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:35.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa37116d700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa36c0b0ba0 con 0x7fa36c1444f0 2022-06-17T12:47:35.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa37116d700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa36c0ae820 con 0x7fa36c062770 2022-06-17T12:47:35.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa37116d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c061030 con 0x7fa36c0ad330 2022-06-17T12:47:35.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa36ad9d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa36c062770 0x7fa36c062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:35.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa36259c700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fa36c1444f0 0x7fa36c144960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49908/0 (socket says 172.21.15.36:49908) 2022-06-17T12:47:35.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa36259c700 1 -- 172.21.15.36:0/2639549153 learned_addr learned my addr 172.21.15.36:0/2639549153 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:35.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa36b59e700 1 --2- 172.21.15.36:0/2639549153 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa36c0ad330 0x7fa36c0ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:35.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa36b59e700 1 -- 172.21.15.36:0/2639549153 >> v2:172.21.15.36:3300/0 conn(0x7fa36c062770 msgr2=0x7fa36c062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:35.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa36b59e700 1 --2- 172.21.15.36:0/2639549153 >> v2:172.21.15.36:3300/0 conn(0x7fa36c062770 0x7fa36c062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:35.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa36b59e700 1 -- 172.21.15.36:0/2639549153 >> v1:172.21.15.36:6789/0 conn(0x7fa36c1444f0 legacy=0x7fa36c144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:35.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa36a59c700 1 -- 172.21.15.36:0/2639549153 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2175386213 0 0) 0x7fa36c0b0ba0 con 0x7fa36c1444f0 2022-06-17T12:47:35.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa36b59e700 1 -- 172.21.15.36:0/2639549153 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36c147e90 con 0x7fa36c0ad330 2022-06-17T12:47:35.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa36ad9d700 1 --2- 172.21.15.36:0/2639549153 >> v2:172.21.15.36:3300/0 conn(0x7fa36c062770 0x7fa36c062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:35.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa36b59e700 1 --2- 172.21.15.36:0/2639549153 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa36c0ad330 0x7fa36c0ad7c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fa3640032f0 tx=0x7fa3640091d0).ready entity=mon.2 client_cookie=bea8282e292e39b3 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:35.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa36a59c700 1 -- 172.21.15.36:0/2639549153 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa36400b6c0 con 0x7fa36c0ad330 2022-06-17T12:47:35.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa36a59c700 1 -- 172.21.15.36:0/2639549153 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36400b820 con 0x7fa36c0ad330 2022-06-17T12:47:35.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa36a59c700 1 -- 172.21.15.36:0/2639549153 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa36400b990 con 0x7fa36c0ad330 2022-06-17T12:47:35.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa37116d700 1 -- 172.21.15.36:0/2639549153 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa36c0ad330 msgr2=0x7fa36c0ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:35.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa37116d700 1 --2- 172.21.15.36:0/2639549153 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa36c0ad330 0x7fa36c0ad7c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fa3640032f0 tx=0x7fa3640091d0).stop 2022-06-17T12:47:35.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa37116d700 1 -- 172.21.15.36:0/2639549153 shutdown_connections 2022-06-17T12:47:35.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa37116d700 1 --2- 172.21.15.36:0/2639549153 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa36c0ad330 0x7fa36c0ad7c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:35.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa37116d700 1 --2- 172.21.15.36:0/2639549153 >> v2:172.21.15.36:3300/0 conn(0x7fa36c062770 0x7fa36c062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:35.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa37116d700 1 -- 172.21.15.36:0/2639549153 >> 172.21.15.36:0/2639549153 conn(0x7fa36c0b53e0 msgr2=0x7fa36c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:35.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa37116d700 1 -- 172.21.15.36:0/2639549153 shutdown_connections 2022-06-17T12:47:35.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.805+0000 7fa37116d700 1 -- 172.21.15.36:0/2639549153 wait complete. 2022-06-17T12:47:35.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.809+0000 7fa37116d700 1 Processor -- start 2022-06-17T12:47:35.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.809+0000 7fa37116d700 1 -- start start 2022-06-17T12:47:35.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.809+0000 7fa37116d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa36c062770 0x7fa36c14c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:35.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.809+0000 7fa37116d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa36c0ad330 0x7fa36c14aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:35.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.809+0000 7fa37116d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa36c1444f0 0x7fa36c14af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:35.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.809+0000 7fa37116d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c131e80 con 0x7fa36c1444f0 2022-06-17T12:47:35.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.809+0000 7fa37116d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c1567d0 con 0x7fa36c0ad330 2022-06-17T12:47:35.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.809+0000 7fa37116d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c143f70 con 0x7fa36c062770 2022-06-17T12:47:35.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.809+0000 7fa36b59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa36c1444f0 0x7fa36c14af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:35.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.809+0000 7fa36b59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa36c1444f0 0x7fa36c14af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60620/0 (socket says 172.21.15.36:60620) 2022-06-17T12:47:35.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.809+0000 7fa36b59e700 1 -- 172.21.15.36:0/4059889812 learned_addr learned my addr 172.21.15.36:0/4059889812 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:35.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.809+0000 7fa36ad9d700 1 --2- 172.21.15.36:0/4059889812 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa36c062770 0x7fa36c14c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:35.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.809+0000 7fa36259c700 1 --2- 172.21.15.36:0/4059889812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa36c0ad330 0x7fa36c14aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:35.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.809+0000 7fa36b59e700 1 -- 172.21.15.36:0/4059889812 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa36c062770 msgr2=0x7fa36c14c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:35.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.809+0000 7fa36b59e700 1 --2- 172.21.15.36:0/4059889812 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa36c062770 0x7fa36c14c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:35.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.813+0000 7fa36b59e700 1 -- 172.21.15.36:0/4059889812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa36c0ad330 msgr2=0x7fa36c14aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:35.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.813+0000 7fa36b59e700 1 --2- 172.21.15.36:0/4059889812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa36c0ad330 0x7fa36c14aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:35.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.813+0000 7fa36b59e700 1 -- 172.21.15.36:0/4059889812 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c00b180 con 0x7fa36c1444f0 2022-06-17T12:47:35.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.813+0000 7fa36b59e700 1 --2- 172.21.15.36:0/4059889812 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa36c1444f0 0x7fa36c14af10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fa3640032f0 tx=0x7fa364016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:35.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.813+0000 7fa368d99700 1 -- 172.21.15.36:0/4059889812 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa36400b6c0 con 0x7fa36c1444f0 2022-06-17T12:47:35.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.813+0000 7fa368d99700 1 -- 172.21.15.36:0/4059889812 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa364021630 con 0x7fa36c1444f0 2022-06-17T12:47:35.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.813+0000 7fa37116d700 1 -- 172.21.15.36:0/4059889812 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa364009470 con 0x7fa36c1444f0 2022-06-17T12:47:35.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.813+0000 7fa368d99700 1 -- 172.21.15.36:0/4059889812 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa364021990 con 0x7fa36c1444f0 2022-06-17T12:47:35.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.813+0000 7fa37116d700 1 -- 172.21.15.36:0/4059889812 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36c0643f0 con 0x7fa36c1444f0 2022-06-17T12:47:35.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.813+0000 7fa37116d700 1 -- 172.21.15.36:0/4059889812 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36c0b1c20 con 0x7fa36c1444f0 2022-06-17T12:47:35.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.813+0000 7fa368d99700 1 -- 172.21.15.36:0/4059889812 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fa36400f070 con 0x7fa36c1444f0 2022-06-17T12:47:35.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.813+0000 7fa368d99700 1 --2- 172.21.15.36:0/4059889812 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa350053a60 0x7fa350055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:35.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.813+0000 7fa368d99700 1 -- 172.21.15.36:0/4059889812 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6517+0+0 (secure 0 0 0) 0x7fa36406d020 con 0x7fa36c1444f0 2022-06-17T12:47:35.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.813+0000 7fa36ad9d700 1 --2- 172.21.15.36:0/4059889812 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa350053a60 0x7fa350055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:35.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.813+0000 7fa36ad9d700 1 --2- 172.21.15.36:0/4059889812 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa350053a60 0x7fa350055f20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fa35c00b000 tx=0x7fa35c006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:35.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.817+0000 7fa368d99700 1 -- 172.21.15.36:0/4059889812 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa364043dc0 con 0x7fa36c1444f0 2022-06-17T12:47:35.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.985+0000 7fa37116d700 1 -- 172.21.15.36:0/4059889812 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa36c057410 con 0x7fa36c1444f0 2022-06-17T12:47:35.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.985+0000 7fa368d99700 1 -- 172.21.15.36:0/4059889812 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+1774 (secure 0 0 0) 0x7fa364015070 con 0x7fa36c1444f0 2022-06-17T12:47:35.993 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:35.993 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/4243909763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":4243909763},{"type":"v1","addr":"172.21.15.36:6835","nonce":4243909763}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:46:52.253609+0000","modified":"2022-06-17T12:47:35.634136+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-06-17T12:47:35.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.989+0000 7fa37116d700 1 -- 172.21.15.36:0/4059889812 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa350053a60 msgr2=0x7fa350055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:35.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.989+0000 7fa37116d700 1 --2- 172.21.15.36:0/4059889812 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa350053a60 0x7fa350055f20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fa35c00b000 tx=0x7fa35c006df0).stop 2022-06-17T12:47:35.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.989+0000 7fa37116d700 1 -- 172.21.15.36:0/4059889812 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa36c1444f0 msgr2=0x7fa36c14af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:36.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.989+0000 7fa37116d700 1 --2- 172.21.15.36:0/4059889812 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa36c1444f0 0x7fa36c14af10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fa3640032f0 tx=0x7fa364016000).stop 2022-06-17T12:47:36.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.989+0000 7fa37116d700 1 -- 172.21.15.36:0/4059889812 shutdown_connections 2022-06-17T12:47:36.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.989+0000 7fa37116d700 1 --2- 172.21.15.36:0/4059889812 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa350053a60 0x7fa350055f20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:36.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.989+0000 7fa37116d700 1 --2- 172.21.15.36:0/4059889812 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa36c1444f0 0x7fa36c14af10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:36.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.989+0000 7fa37116d700 1 --2- 172.21.15.36:0/4059889812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa36c0ad330 0x7fa36c14aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:36.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.989+0000 7fa37116d700 1 --2- 172.21.15.36:0/4059889812 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa36c062770 0x7fa36c14c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:36.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.989+0000 7fa37116d700 1 -- 172.21.15.36:0/4059889812 >> 172.21.15.36:0/4059889812 conn(0x7fa36c0b53e0 msgr2=0x7fa36c144960 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:36.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.989+0000 7fa37116d700 1 -- 172.21.15.36:0/4059889812 shutdown_connections 2022-06-17T12:47:36.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:35.989+0000 7fa37116d700 1 -- 172.21.15.36:0/4059889812 wait complete. 2022-06-17T12:47:36.025 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 8 2022-06-17T12:47:36.026 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-17T12:47:36.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.169+0000 7f8c9f522700 1 Processor -- start 2022-06-17T12:47:36.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.173+0000 7f8c9f522700 1 -- start start 2022-06-17T12:47:36.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.173+0000 7f8c9f522700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8c980b18e0 0x7f8c98143860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:36.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.173+0000 7f8c9f522700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c980b2b70 con 0x7f8c980b39a0 2022-06-17T12:47:36.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.173+0000 7f8c9f522700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c980b0d30 con 0x7f8c980b6860 2022-06-17T12:47:36.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.173+0000 7f8c9f522700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c980b6c40 con 0x7f8c980b18e0 2022-06-17T12:47:36.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.173+0000 7f8c9ed21700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f8c980b39a0 0x7f8c9813fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49926/0 (socket says 172.21.15.36:49926) 2022-06-17T12:47:36.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.173+0000 7f8c9ed21700 1 -- 172.21.15.36:0/18654662 learned_addr learned my addr 172.21.15.36:0/18654662 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:36.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.173+0000 7f8c9dd1f700 1 --2- 172.21.15.36:0/18654662 >> v2:172.21.15.70:3300/0 conn(0x7f8c980b18e0 0x7f8c98143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:36.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.173+0000 7f8c9d51e700 1 -- 172.21.15.36:0/18654662 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3759555568 0 0) 0x7f8c980b2b70 con 0x7f8c980b39a0 2022-06-17T12:47:36.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.173+0000 7f8c9d51e700 1 -- 172.21.15.36:0/18654662 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c80003260 con 0x7f8c980b39a0 2022-06-17T12:47:36.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.173+0000 7f8c9dd1f700 1 -- 172.21.15.36:0/18654662 >> v1:172.21.15.70:6789/0 conn(0x7f8c980b6860 legacy=0x7f8c98143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:36.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.173+0000 7f8c9dd1f700 1 -- 172.21.15.36:0/18654662 >> v1:172.21.15.36:6789/0 conn(0x7f8c980b39a0 legacy=0x7f8c9813fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:36.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.173+0000 7f8c9dd1f700 1 -- 172.21.15.36:0/18654662 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c980b0fd0 con 0x7f8c980b18e0 2022-06-17T12:47:36.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.173+0000 7f8c9dd1f700 1 --2- 172.21.15.36:0/18654662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c980b18e0 0x7f8c98143860 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f8c88002f70 tx=0x7f8c88006d90).ready entity=mon.1 client_cookie=59cc87bc1e9614b3 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:36.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.173+0000 7f8c9d51e700 1 -- 172.21.15.36:0/18654662 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c8800d4a0 con 0x7f8c980b18e0 2022-06-17T12:47:36.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.177+0000 7f8c9d51e700 1 -- 172.21.15.36:0/18654662 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c88000e90 con 0x7f8c980b18e0 2022-06-17T12:47:36.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.177+0000 7f8c9d51e700 1 -- 172.21.15.36:0/18654662 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c8800d7f0 con 0x7f8c980b18e0 2022-06-17T12:47:36.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.177+0000 7f8c9f522700 1 -- 172.21.15.36:0/18654662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c980b18e0 msgr2=0x7f8c98143860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:36.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.177+0000 7f8c9f522700 1 --2- 172.21.15.36:0/18654662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c980b18e0 0x7f8c98143860 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f8c88002f70 tx=0x7f8c88006d90).stop 2022-06-17T12:47:36.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.177+0000 7f8c9f522700 1 -- 172.21.15.36:0/18654662 shutdown_connections 2022-06-17T12:47:36.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.177+0000 7f8c9f522700 1 --2- 172.21.15.36:0/18654662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c980b18e0 0x7f8c98143860 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:36.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.177+0000 7f8c9f522700 1 -- 172.21.15.36:0/18654662 >> 172.21.15.36:0/18654662 conn(0x7f8c980614b0 msgr2=0x7f8c98061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:36.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.177+0000 7f8c9f522700 1 -- 172.21.15.36:0/18654662 shutdown_connections 2022-06-17T12:47:36.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.177+0000 7f8c9f522700 1 -- 172.21.15.36:0/18654662 wait complete. 2022-06-17T12:47:36.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.177+0000 7f8c9f522700 1 Processor -- start 2022-06-17T12:47:36.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.177+0000 7f8c9f522700 1 -- start start 2022-06-17T12:47:36.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.181+0000 7f8c9f522700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8c980b18e0 0x7f8c980b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:36.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.181+0000 7f8c9f522700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8c980b39a0 0x7f8c98155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:36.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.181+0000 7f8c9f522700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c980b6860 0x7f8c980afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:36.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.181+0000 7f8c9f522700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c980654f0 con 0x7f8c980b18e0 2022-06-17T12:47:36.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.181+0000 7f8c9f522700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c98065250 con 0x7f8c980b6860 2022-06-17T12:47:36.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.181+0000 7f8c9f522700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c981421b0 con 0x7f8c980b39a0 2022-06-17T12:47:36.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.181+0000 7f8c9e520700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8c980b18e0 0x7f8c980b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:36.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.181+0000 7f8c9e520700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8c980b18e0 0x7f8c980b54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60630/0 (socket says 172.21.15.36:60630) 2022-06-17T12:47:36.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.181+0000 7f8c9e520700 1 -- 172.21.15.36:0/2962966521 learned_addr learned my addr 172.21.15.36:0/2962966521 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:36.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.181+0000 7f8c9ed21700 1 --2- 172.21.15.36:0/2962966521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c980b6860 0x7f8c980afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:36.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.181+0000 7f8c9dd1f700 1 --2- 172.21.15.36:0/2962966521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8c980b39a0 0x7f8c98155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:36.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.181+0000 7f8c9e520700 1 -- 172.21.15.36:0/2962966521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8c980b39a0 msgr2=0x7f8c98155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:36.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.181+0000 7f8c9e520700 1 --2- 172.21.15.36:0/2962966521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8c980b39a0 0x7f8c98155780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:36.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.181+0000 7f8c9e520700 1 -- 172.21.15.36:0/2962966521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c980b6860 msgr2=0x7f8c980afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:36.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.181+0000 7f8c9e520700 1 --2- 172.21.15.36:0/2962966521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c980b6860 0x7f8c980afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:36.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.181+0000 7f8c9e520700 1 -- 172.21.15.36:0/2962966521 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8800b120 con 0x7f8c980b18e0 2022-06-17T12:47:36.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.181+0000 7f8c9e520700 1 --2- 172.21.15.36:0/2962966521 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8c980b18e0 0x7f8c980b54c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f8c980ae400 tx=0x7f8c94007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:36.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.181+0000 7f8c877fe700 1 -- 172.21.15.36:0/2962966521 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c94012400 con 0x7f8c980b18e0 2022-06-17T12:47:36.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.181+0000 7f8c877fe700 1 -- 172.21.15.36:0/2962966521 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c94012560 con 0x7f8c980b18e0 2022-06-17T12:47:36.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.181+0000 7f8c9f522700 1 -- 172.21.15.36:0/2962966521 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c980ae5d0 con 0x7f8c980b18e0 2022-06-17T12:47:36.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.181+0000 7f8c877fe700 1 -- 172.21.15.36:0/2962966521 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c940128c0 con 0x7f8c980b18e0 2022-06-17T12:47:36.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.181+0000 7f8c9f522700 1 -- 172.21.15.36:0/2962966521 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c980af810 con 0x7f8c980b18e0 2022-06-17T12:47:36.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.185+0000 7f8c9f522700 1 -- 172.21.15.36:0/2962966521 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c980afaa0 con 0x7f8c980b18e0 2022-06-17T12:47:36.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.185+0000 7f8c877fe700 1 -- 172.21.15.36:0/2962966521 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8c94019030 con 0x7f8c980b18e0 2022-06-17T12:47:36.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.185+0000 7f8c877fe700 1 --2- 172.21.15.36:0/2962966521 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8c78053af0 0x7f8c78055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:36.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.185+0000 7f8c877fe700 1 -- 172.21.15.36:0/2962966521 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6517+0+0 (secure 0 0 0) 0x7f8c9406b310 con 0x7f8c980b18e0 2022-06-17T12:47:36.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.185+0000 7f8c9dd1f700 1 --2- 172.21.15.36:0/2962966521 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8c78053af0 0x7f8c78055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:36.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.185+0000 7f8c9dd1f700 1 --2- 172.21.15.36:0/2962966521 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8c78053af0 0x7f8c78055fb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f8c8801a000 tx=0x7f8c88014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:36.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.193+0000 7f8c877fe700 1 -- 172.21.15.36:0/2962966521 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8c94042010 con 0x7f8c980b18e0 2022-06-17T12:47:36.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.357+0000 7f8c9f522700 1 -- 172.21.15.36:0/2962966521 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8c98057410 con 0x7f8c980b18e0 2022-06-17T12:47:36.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.657+0000 7f8c877fe700 1 -- 172.21.15.36:0/2962966521 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8c94045630 con 0x7f8c980b18e0 2022-06-17T12:47:36.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.657+0000 7f8c9f522700 1 -- 172.21.15.36:0/2962966521 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8c78053af0 msgr2=0x7f8c78055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:36.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.657+0000 7f8c9f522700 1 --2- 172.21.15.36:0/2962966521 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8c78053af0 0x7f8c78055fb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f8c8801a000 tx=0x7f8c88014000).stop 2022-06-17T12:47:36.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.657+0000 7f8c9f522700 1 -- 172.21.15.36:0/2962966521 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8c980b18e0 msgr2=0x7f8c980b54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:36.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.657+0000 7f8c9f522700 1 --2- 172.21.15.36:0/2962966521 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8c980b18e0 0x7f8c980b54c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f8c980ae400 tx=0x7f8c94007c50).stop 2022-06-17T12:47:36.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.661+0000 7f8c9f522700 1 -- 172.21.15.36:0/2962966521 shutdown_connections 2022-06-17T12:47:36.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.661+0000 7f8c9f522700 1 --2- 172.21.15.36:0/2962966521 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8c78053af0 0x7f8c78055fb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:36.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.661+0000 7f8c9f522700 1 --2- 172.21.15.36:0/2962966521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c980b6860 0x7f8c980afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:36.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.661+0000 7f8c9f522700 1 --2- 172.21.15.36:0/2962966521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8c980b39a0 0x7f8c98155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:36.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.661+0000 7f8c9f522700 1 --2- 172.21.15.36:0/2962966521 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8c980b18e0 0x7f8c980b54c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:36.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.661+0000 7f8c9f522700 1 -- 172.21.15.36:0/2962966521 >> 172.21.15.36:0/2962966521 conn(0x7f8c980614b0 msgr2=0x7f8c9805bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:36.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.661+0000 7f8c9f522700 1 -- 172.21.15.36:0/2962966521 shutdown_connections 2022-06-17T12:47:36.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.661+0000 7f8c9f522700 1 -- 172.21.15.36:0/2962966521 wait complete. 2022-06-17T12:47:36.771 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-17T12:47:37.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.949+0000 7f1840c6c700 1 Processor -- start 2022-06-17T12:47:37.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.949+0000 7f1840c6c700 1 -- start start 2022-06-17T12:47:37.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f1840c6c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f183c0b4080 0x7f183c142130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:37.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f1840c6c700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f183c0ba4f0 con 0x7f183c0b4e30 2022-06-17T12:47:37.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f1840c6c700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f183c143980 con 0x7f183c0ba110 2022-06-17T12:47:37.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f1840c6c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f183c146ab0 con 0x7f183c0b4080 2022-06-17T12:47:37.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f183affd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f183c0b4e30 0x7f183c0b5740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49938/0 (socket says 172.21.15.36:49938) 2022-06-17T12:47:37.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f183affd700 1 -- 172.21.15.36:0/2227204848 learned_addr learned my addr 172.21.15.36:0/2227204848 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:37.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f183bfff700 1 --2- 172.21.15.36:0/2227204848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f183c0b4080 0x7f183c142130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:37.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f183a7fc700 1 -- 172.21.15.36:0/2227204848 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3759522601 0 0) 0x7f183c0ba4f0 con 0x7f183c0b4e30 2022-06-17T12:47:37.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f183a7fc700 1 -- 172.21.15.36:0/2227204848 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f181c003260 con 0x7f183c0b4e30 2022-06-17T12:47:37.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f183a7fc700 1 -- 172.21.15.36:0/2227204848 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 428135266 0 0) 0x7f183c143980 con 0x7f183c0ba110 2022-06-17T12:47:37.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f183a7fc700 1 -- 172.21.15.36:0/2227204848 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f183c0ba4f0 con 0x7f183c0ba110 2022-06-17T12:47:37.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f183bfff700 1 -- 172.21.15.36:0/2227204848 >> v1:172.21.15.70:6789/0 conn(0x7f183c0ba110 legacy=0x7f183c1463a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:37.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f183bfff700 1 -- 172.21.15.36:0/2227204848 >> v1:172.21.15.36:6789/0 conn(0x7f183c0b4e30 legacy=0x7f183c0b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:37.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f183bfff700 1 -- 172.21.15.36:0/2227204848 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183c143c20 con 0x7f183c0b4080 2022-06-17T12:47:37.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f183bfff700 1 --2- 172.21.15.36:0/2227204848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f183c0b4080 0x7f183c142130 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f18340032f0 tx=0x7f18340091d0).ready entity=mon.2 client_cookie=d0def6d3f8c53c05 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:37.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f183a7fc700 1 -- 172.21.15.36:0/2227204848 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f183400b640 con 0x7f183c0b4080 2022-06-17T12:47:37.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f183a7fc700 1 -- 172.21.15.36:0/2227204848 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183400b7a0 con 0x7f183c0b4080 2022-06-17T12:47:37.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f183a7fc700 1 -- 172.21.15.36:0/2227204848 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f183400b910 con 0x7f183c0b4080 2022-06-17T12:47:37.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f1840c6c700 1 -- 172.21.15.36:0/2227204848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f183c0b4080 msgr2=0x7f183c142130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:37.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f1840c6c700 1 --2- 172.21.15.36:0/2227204848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f183c0b4080 0x7f183c142130 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f18340032f0 tx=0x7f18340091d0).stop 2022-06-17T12:47:37.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f1840c6c700 1 -- 172.21.15.36:0/2227204848 shutdown_connections 2022-06-17T12:47:37.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f1840c6c700 1 --2- 172.21.15.36:0/2227204848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f183c0b4080 0x7f183c142130 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:37.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f1840c6c700 1 -- 172.21.15.36:0/2227204848 >> 172.21.15.36:0/2227204848 conn(0x7f183c05c550 msgr2=0x7f183c0b96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:37.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f1840c6c700 1 -- 172.21.15.36:0/2227204848 shutdown_connections 2022-06-17T12:47:37.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.953+0000 7f1840c6c700 1 -- 172.21.15.36:0/2227204848 wait complete. 2022-06-17T12:47:37.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.957+0000 7f1840c6c700 1 Processor -- start 2022-06-17T12:47:37.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.957+0000 7f1840c6c700 1 -- start start 2022-06-17T12:47:37.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.957+0000 7f1840c6c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f183c0b4080 0x7f183c0b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:37.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.957+0000 7f1840c6c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f183c0b4e30 0x7f183c112930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:37.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.957+0000 7f1840c6c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f183c0ba110 0x7f183c13e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:37.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.957+0000 7f1840c6c700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183c0b2440 con 0x7f183c0b4080 2022-06-17T12:47:37.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.957+0000 7f1840c6c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183c0b1220 con 0x7f183c0b4e30 2022-06-17T12:47:37.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.957+0000 7f1840c6c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f183c152380 con 0x7f183c0ba110 2022-06-17T12:47:37.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.957+0000 7f183b7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f183c0b4080 0x7f183c0b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:37.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.957+0000 7f183affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f183c0b4e30 0x7f183c112930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:37.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.957+0000 7f183b7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f183c0b4080 0x7f183c0b1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60644/0 (socket says 172.21.15.36:60644) 2022-06-17T12:47:37.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.957+0000 7f183b7fe700 1 -- 172.21.15.36:0/453820817 learned_addr learned my addr 172.21.15.36:0/453820817 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:37.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.957+0000 7f183bfff700 1 --2- 172.21.15.36:0/453820817 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f183c0ba110 0x7f183c13e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:37.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.957+0000 7f183b7fe700 1 -- 172.21.15.36:0/453820817 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f183c0ba110 msgr2=0x7f183c13e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:37.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.957+0000 7f183b7fe700 1 --2- 172.21.15.36:0/453820817 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f183c0ba110 0x7f183c13e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:37.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.957+0000 7f183b7fe700 1 -- 172.21.15.36:0/453820817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f183c0b4e30 msgr2=0x7f183c112930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:37.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.957+0000 7f183b7fe700 1 --2- 172.21.15.36:0/453820817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f183c0b4e30 0x7f183c112930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:37.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.957+0000 7f183b7fe700 1 -- 172.21.15.36:0/453820817 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1834009470 con 0x7f183c0b4080 2022-06-17T12:47:37.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.957+0000 7f183b7fe700 1 --2- 172.21.15.36:0/453820817 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f183c0b4080 0x7f183c0b1a50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f1824001110 tx=0x7f1824007a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:37.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.957+0000 7f1838ff9700 1 -- 172.21.15.36:0/453820817 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f182400cde0 con 0x7f183c0b4080 2022-06-17T12:47:37.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.957+0000 7f1838ff9700 1 -- 172.21.15.36:0/453820817 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1824012400 con 0x7f183c0b4080 2022-06-17T12:47:37.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.957+0000 7f1840c6c700 1 -- 172.21.15.36:0/453820817 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f183c152680 con 0x7f183c0b4080 2022-06-17T12:47:37.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.957+0000 7f1838ff9700 1 -- 172.21.15.36:0/453820817 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1824012760 con 0x7f183c0b4080 2022-06-17T12:47:37.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.957+0000 7f1840c6c700 1 -- 172.21.15.36:0/453820817 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f183c0b1f40 con 0x7f183c0b4080 2022-06-17T12:47:37.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.961+0000 7f1840c6c700 1 -- 172.21.15.36:0/453820817 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f183c0b5280 con 0x7f183c0b4080 2022-06-17T12:47:37.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.961+0000 7f1838ff9700 1 -- 172.21.15.36:0/453820817 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f1824019030 con 0x7f183c0b4080 2022-06-17T12:47:37.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.961+0000 7f1838ff9700 1 --2- 172.21.15.36:0/453820817 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f18200539b0 0x7f1820055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:37.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.961+0000 7f1838ff9700 1 -- 172.21.15.36:0/453820817 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6517+0+0 (secure 0 0 0) 0x7f182406b180 con 0x7f183c0b4080 2022-06-17T12:47:37.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.961+0000 7f183affd700 1 --2- 172.21.15.36:0/453820817 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f18200539b0 0x7f1820055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:37.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.965+0000 7f183affd700 1 --2- 172.21.15.36:0/453820817 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f18200539b0 0x7f1820055e70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f182c00d000 tx=0x7f182c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:37.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:36.965+0000 7f1838ff9700 1 -- 172.21.15.36:0/453820817 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1824041f20 con 0x7f183c0b4080 2022-06-17T12:47:37.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.093+0000 7f1840c6c700 1 -- 172.21.15.36:0/453820817 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f183c057410 con 0x7f183c0b4080 2022-06-17T12:47:37.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.665+0000 7f1838ff9700 1 -- 172.21.15.36:0/453820817 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 185+0+0 (secure 0 0 0) 0x7f1824045540 con 0x7f183c0b4080 2022-06-17T12:47:37.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.673+0000 7f1840c6c700 1 -- 172.21.15.36:0/453820817 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f18200539b0 msgr2=0x7f1820055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:37.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.673+0000 7f1840c6c700 1 --2- 172.21.15.36:0/453820817 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f18200539b0 0x7f1820055e70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f182c00d000 tx=0x7f182c009040).stop 2022-06-17T12:47:37.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.673+0000 7f1840c6c700 1 -- 172.21.15.36:0/453820817 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f183c0b4080 msgr2=0x7f183c0b1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:37.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.673+0000 7f1840c6c700 1 --2- 172.21.15.36:0/453820817 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f183c0b4080 0x7f183c0b1a50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f1824001110 tx=0x7f1824007a90).stop 2022-06-17T12:47:37.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.673+0000 7f1840c6c700 1 -- 172.21.15.36:0/453820817 shutdown_connections 2022-06-17T12:47:37.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.673+0000 7f1840c6c700 1 --2- 172.21.15.36:0/453820817 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f18200539b0 0x7f1820055e70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:37.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.673+0000 7f1840c6c700 1 --2- 172.21.15.36:0/453820817 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f183c0ba110 0x7f183c13e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:37.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.673+0000 7f1840c6c700 1 --2- 172.21.15.36:0/453820817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f183c0b4e30 0x7f183c112930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:37.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.673+0000 7f1840c6c700 1 --2- 172.21.15.36:0/453820817 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f183c0b4080 0x7f183c0b1a50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:37.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.673+0000 7f1840c6c700 1 -- 172.21.15.36:0/453820817 >> 172.21.15.36:0/453820817 conn(0x7f183c05c550 msgr2=0x7f183c0b96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:37.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.673+0000 7f1840c6c700 1 -- 172.21.15.36:0/453820817 shutdown_connections 2022-06-17T12:47:37.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.673+0000 7f1840c6c700 1 -- 172.21.15.36:0/453820817 wait complete. 2022-06-17T12:47:37.688 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' removed 2022-06-17T12:47:37.702 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-17T12:47:38.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.877+0000 7fecd52ed700 1 Processor -- start 2022-06-17T12:47:38.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.877+0000 7fecd52ed700 1 -- start start 2022-06-17T12:47:38.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.877+0000 7fecd52ed700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fecd0060920 0x7fecd0138ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:38.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.877+0000 7fecd52ed700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd00af610 con 0x7fecd013fe30 2022-06-17T12:47:38.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.877+0000 7fecd52ed700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd00af8b0 con 0x7fecd00b11a0 2022-06-17T12:47:38.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.877+0000 7fecd52ed700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecd00ae190 con 0x7fecd0060920 2022-06-17T12:47:38.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.877+0000 7fecc77fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fecd013fe30 0x7fecd00ab9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49950/0 (socket says 172.21.15.36:49950) 2022-06-17T12:47:38.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.877+0000 7fecc77fe700 1 -- 172.21.15.36:0/3503525544 learned_addr learned my addr 172.21.15.36:0/3503525544 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:38.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.877+0000 7feccffff700 1 --2- 172.21.15.36:0/3503525544 >> v2:172.21.15.36:3300/0 conn(0x7fecd0060920 0x7fecd0138ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:38.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.881+0000 7feccf7fe700 1 -- 172.21.15.36:0/3503525544 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 838486891 0 0) 0x7fecd00af610 con 0x7fecd013fe30 2022-06-17T12:47:38.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.881+0000 7feccf7fe700 1 -- 172.21.15.36:0/3503525544 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecb0003260 con 0x7fecd013fe30 2022-06-17T12:47:38.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.881+0000 7feccffff700 1 -- 172.21.15.36:0/3503525544 >> v1:172.21.15.70:6789/0 conn(0x7fecd00b11a0 legacy=0x7fecd00afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:38.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.881+0000 7feccffff700 1 -- 172.21.15.36:0/3503525544 >> v1:172.21.15.36:6789/0 conn(0x7fecd013fe30 legacy=0x7fecd00ab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:38.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.881+0000 7feccffff700 1 -- 172.21.15.36:0/3503525544 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd00ac780 con 0x7fecd0060920 2022-06-17T12:47:38.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.881+0000 7feccffff700 1 --2- 172.21.15.36:0/3503525544 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fecd0060920 0x7fecd0138ff0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fecc0002f70 tx=0x7fecc0006df0).ready entity=mon.0 client_cookie=ea918d21c94eb2ed server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:38.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.881+0000 7feccf7fe700 1 -- 172.21.15.36:0/3503525544 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecc000d610 con 0x7fecd0060920 2022-06-17T12:47:38.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.881+0000 7feccf7fe700 1 -- 172.21.15.36:0/3503525544 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc0000e90 con 0x7fecd0060920 2022-06-17T12:47:38.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.881+0000 7feccf7fe700 1 -- 172.21.15.36:0/3503525544 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecc000d960 con 0x7fecd0060920 2022-06-17T12:47:38.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.881+0000 7fecd52ed700 1 -- 172.21.15.36:0/3503525544 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fecd0060920 msgr2=0x7fecd0138ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:38.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.881+0000 7fecd52ed700 1 --2- 172.21.15.36:0/3503525544 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fecd0060920 0x7fecd0138ff0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fecc0002f70 tx=0x7fecc0006df0).stop 2022-06-17T12:47:38.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.881+0000 7fecd52ed700 1 -- 172.21.15.36:0/3503525544 shutdown_connections 2022-06-17T12:47:38.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.881+0000 7fecd52ed700 1 --2- 172.21.15.36:0/3503525544 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fecd0060920 0x7fecd0138ff0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:38.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.881+0000 7fecd52ed700 1 -- 172.21.15.36:0/3503525544 >> 172.21.15.36:0/3503525544 conn(0x7fecd00b5d10 msgr2=0x7fecd00b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:38.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.881+0000 7fecd52ed700 1 -- 172.21.15.36:0/3503525544 shutdown_connections 2022-06-17T12:47:38.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.881+0000 7fecd52ed700 1 -- 172.21.15.36:0/3503525544 wait complete. 2022-06-17T12:47:38.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.881+0000 7fecd52ed700 1 Processor -- start 2022-06-17T12:47:38.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.881+0000 7fecd52ed700 1 -- start start 2022-06-17T12:47:38.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.881+0000 7fecd52ed700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecd0060920 0x7fecd0063df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:38.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.881+0000 7fecd52ed700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecd00b11a0 0x7fecd01480e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:38.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.881+0000 7fecd52ed700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fecd013fe30 0x7fecd0156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:38.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.881+0000 7fecd52ed700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd009ea70 con 0x7fecd013fe30 2022-06-17T12:47:38.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.881+0000 7fecd52ed700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd013c5b0 con 0x7fecd00b11a0 2022-06-17T12:47:38.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.881+0000 7fecd52ed700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd0064e30 con 0x7fecd0060920 2022-06-17T12:47:38.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.885+0000 7fecd4aec700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fecd013fe30 0x7fecd0156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:38.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.885+0000 7fecd4aec700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fecd013fe30 0x7fecd0156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60662/0 (socket says 172.21.15.36:60662) 2022-06-17T12:47:38.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.885+0000 7fecd4aec700 1 -- 172.21.15.36:0/2632456541 learned_addr learned my addr 172.21.15.36:0/2632456541 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:38.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.885+0000 7fecc77fe700 1 --2- 172.21.15.36:0/2632456541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecd00b11a0 0x7fecd01480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:38.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.885+0000 7feccffff700 1 --2- 172.21.15.36:0/2632456541 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecd0060920 0x7fecd0063df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:38.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.885+0000 7fecd4aec700 1 -- 172.21.15.36:0/2632456541 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecd0060920 msgr2=0x7fecd0063df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:38.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.885+0000 7fecd4aec700 1 --2- 172.21.15.36:0/2632456541 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecd0060920 0x7fecd0063df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:38.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.885+0000 7fecd4aec700 1 -- 172.21.15.36:0/2632456541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecd00b11a0 msgr2=0x7fecd01480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:38.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.885+0000 7fecd4aec700 1 --2- 172.21.15.36:0/2632456541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecd00b11a0 0x7fecd01480e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:38.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.885+0000 7fecd4aec700 1 -- 172.21.15.36:0/2632456541 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc000b180 con 0x7fecd013fe30 2022-06-17T12:47:38.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.885+0000 7fecc77fe700 1 --2- 172.21.15.36:0/2632456541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecd00b11a0 0x7fecd01480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:38.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.885+0000 7fecd4aec700 1 --2- 172.21.15.36:0/2632456541 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fecd013fe30 0x7fecd0156100 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7fecd013e7c0 tx=0x7fecc8007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:38.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.885+0000 7feccdffb700 1 -- 172.21.15.36:0/2632456541 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecc800f470 con 0x7fecd013fe30 2022-06-17T12:47:38.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.885+0000 7feccdffb700 1 -- 172.21.15.36:0/2632456541 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc800f5d0 con 0x7fecd013fe30 2022-06-17T12:47:38.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.885+0000 7fecd52ed700 1 -- 172.21.15.36:0/2632456541 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd00650d0 con 0x7fecd013fe30 2022-06-17T12:47:38.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.885+0000 7feccdffb700 1 -- 172.21.15.36:0/2632456541 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecc800f930 con 0x7fecd013fe30 2022-06-17T12:47:38.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.885+0000 7fecd52ed700 1 -- 172.21.15.36:0/2632456541 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd013e490 con 0x7fecd013fe30 2022-06-17T12:47:38.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.885+0000 7fecd52ed700 1 -- 172.21.15.36:0/2632456541 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd0057410 con 0x7fecd013fe30 2022-06-17T12:47:38.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.885+0000 7feccdffb700 1 -- 172.21.15.36:0/2632456541 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fecc8019030 con 0x7fecd013fe30 2022-06-17T12:47:38.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.889+0000 7feccdffb700 1 --2- 172.21.15.36:0/2632456541 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fecb4053a40 0x7fecb4055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:38.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.889+0000 7feccdffb700 1 -- 172.21.15.36:0/2632456541 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6079+0+0 (secure 0 0 0) 0x7fecc806b610 con 0x7fecd013fe30 2022-06-17T12:47:38.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.889+0000 7feccffff700 1 --2- 172.21.15.36:0/2632456541 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fecb4053a40 0x7fecb4055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:38.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.889+0000 7feccffff700 1 --2- 172.21.15.36:0/2632456541 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fecb4053a40 0x7fecb4055f00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fecc001a000 tx=0x7fecc0014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:38.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:37.897+0000 7feccdffb700 1 -- 172.21.15.36:0/2632456541 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fecc80423e0 con 0x7fecd013fe30 2022-06-17T12:47:38.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:38.029+0000 7fecd52ed700 1 -- 172.21.15.36:0/2632456541 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fecd00b1ad0 con 0x7fecd013fe30 2022-06-17T12:47:38.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:38.681+0000 7feccdffb700 1 -- 172.21.15.36:0/2632456541 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 173+0+0 (secure 0 0 0) 0x7fecc8045a00 con 0x7fecd013fe30 2022-06-17T12:47:38.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:38.685+0000 7fecd52ed700 1 -- 172.21.15.36:0/2632456541 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fecb4053a40 msgr2=0x7fecb4055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:38.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:38.685+0000 7fecd52ed700 1 --2- 172.21.15.36:0/2632456541 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fecb4053a40 0x7fecb4055f00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fecc001a000 tx=0x7fecc0014000).stop 2022-06-17T12:47:38.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:38.685+0000 7fecd52ed700 1 -- 172.21.15.36:0/2632456541 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fecd013fe30 msgr2=0x7fecd0156100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:38.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:38.685+0000 7fecd52ed700 1 --2- 172.21.15.36:0/2632456541 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fecd013fe30 0x7fecd0156100 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7fecd013e7c0 tx=0x7fecc8007f40).stop 2022-06-17T12:47:38.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:38.685+0000 7fecd52ed700 1 -- 172.21.15.36:0/2632456541 shutdown_connections 2022-06-17T12:47:38.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:38.685+0000 7fecd52ed700 1 --2- 172.21.15.36:0/2632456541 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fecb4053a40 0x7fecb4055f00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:38.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:38.685+0000 7fecd52ed700 1 --2- 172.21.15.36:0/2632456541 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fecd013fe30 0x7fecd0156100 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:38.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:38.685+0000 7fecd52ed700 1 --2- 172.21.15.36:0/2632456541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecd00b11a0 0x7fecd01480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:38.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:38.685+0000 7fecd52ed700 1 --2- 172.21.15.36:0/2632456541 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecd0060920 0x7fecd0063df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:38.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:38.685+0000 7fecd52ed700 1 -- 172.21.15.36:0/2632456541 >> 172.21.15.36:0/2632456541 conn(0x7fecd00b5d10 msgr2=0x7fecd00b1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:38.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:38.685+0000 7fecd52ed700 1 -- 172.21.15.36:0/2632456541 shutdown_connections 2022-06-17T12:47:38.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:38.685+0000 7fecd52ed700 1 -- 172.21.15.36:0/2632456541 wait complete. 2022-06-17T12:47:38.681 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' removed 2022-06-17T12:47:38.685 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-17T12:47:38.718 INFO:tasks.ceph.mds.a:Stopping old one... 2022-06-17T12:47:38.719 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-17T12:47:38.719 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T12:47:38.720 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-17T12:47:38.721 INFO:tasks.ceph.mds.b:Stopping old one... 2022-06-17T12:47:38.721 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-17T12:47:38.728 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T12:47:38.735 INFO:tasks.ceph.mds.a:Stopped 2022-06-17T12:47:38.736 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-17T12:47:38.739 INFO:tasks.ceph.mds.b:Stopped 2022-06-17T12:47:38.739 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-17T12:47:38.780 INFO:tasks.ceph.mds.b:Started 2022-06-17T12:47:38.781 INFO:tasks.ceph.mds.a:Started 2022-06-17T12:47:38.782 DEBUG:teuthology.parallel:result is None 2022-06-17T12:47:38.786 DEBUG:teuthology.parallel:result is None 2022-06-17T12:47:38.789 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-06-17T12:47:38.862 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2022-06-17T12:47:38.879 INFO:tasks.ceph.mds.a.smithi036.stdout:starting mds.a at 2022-06-17T12:47:38.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.005+0000 7f6c86e61700 1 Processor -- start 2022-06-17T12:47:38.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.005+0000 7f6c86e61700 1 -- start start 2022-06-17T12:47:39.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c86e61700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6c8013cb70 0x7f6c80143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:39.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c86e61700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c8013cf50 0x7f6c801442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:39.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c86e61700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c800af4b0 con 0x7f6c800aef60 2022-06-17T12:47:39.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c86e61700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c800ba500 con 0x7f6c8013cb70 2022-06-17T12:47:39.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c86e61700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c80142f50 con 0x7f6c8013cf50 2022-06-17T12:47:39.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c8565e700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f6c800aef60 0x7f6c800b9df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49978/0 (socket says 172.21.15.36:49978) 2022-06-17T12:47:39.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c8565e700 1 -- 172.21.15.36:0/4143024669 learned_addr learned my addr 172.21.15.36:0/4143024669 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:39.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c85e5f700 1 --2- 172.21.15.36:0/4143024669 >> v2:172.21.15.70:3300/0 conn(0x7f6c8013cb70 0x7f6c80143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:39.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c86660700 1 --2- 172.21.15.36:0/4143024669 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c8013cf50 0x7f6c801442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:39.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c84e5d700 1 -- 172.21.15.36:0/4143024669 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034108190 0 0) 0x7f6c800af4b0 con 0x7f6c800aef60 2022-06-17T12:47:39.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c84e5d700 1 -- 172.21.15.36:0/4143024669 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c68003260 con 0x7f6c800aef60 2022-06-17T12:47:39.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c86660700 1 -- 172.21.15.36:0/4143024669 >> v2:172.21.15.70:3300/0 conn(0x7f6c8013cb70 msgr2=0x7f6c80143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:39.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c86660700 1 --2- 172.21.15.36:0/4143024669 >> v2:172.21.15.70:3300/0 conn(0x7f6c8013cb70 0x7f6c80143690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:39.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c86660700 1 -- 172.21.15.36:0/4143024669 >> v1:172.21.15.36:6789/0 conn(0x7f6c800aef60 legacy=0x7f6c800b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:39.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c84e5d700 1 -- 172.21.15.36:0/4143024669 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2817190262 0 0) 0x7f6c68003260 con 0x7f6c800aef60 2022-06-17T12:47:39.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c86660700 1 -- 172.21.15.36:0/4143024669 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c801431f0 con 0x7f6c8013cf50 2022-06-17T12:47:39.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c85e5f700 1 --2- 172.21.15.36:0/4143024669 >> v2:172.21.15.70:3300/0 conn(0x7f6c8013cb70 0x7f6c80143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:39.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c86660700 1 --2- 172.21.15.36:0/4143024669 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c8013cf50 0x7f6c801442a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f6c70001800 tx=0x7f6c780091d0).ready entity=mon.2 client_cookie=f61ca22f4b48fed1 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:39.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c84e5d700 1 -- 172.21.15.36:0/4143024669 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c70001660 con 0x7f6c8013cf50 2022-06-17T12:47:39.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c84e5d700 1 -- 172.21.15.36:0/4143024669 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7800f510 con 0x7f6c8013cf50 2022-06-17T12:47:39.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c84e5d700 1 -- 172.21.15.36:0/4143024669 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c7800f680 con 0x7f6c8013cf50 2022-06-17T12:47:39.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c86e61700 1 -- 172.21.15.36:0/4143024669 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c8013cf50 msgr2=0x7f6c801442a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:39.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c86e61700 1 --2- 172.21.15.36:0/4143024669 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c8013cf50 0x7f6c801442a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f6c70001800 tx=0x7f6c780091d0).stop 2022-06-17T12:47:39.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c86e61700 1 -- 172.21.15.36:0/4143024669 shutdown_connections 2022-06-17T12:47:39.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c86e61700 1 --2- 172.21.15.36:0/4143024669 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c8013cf50 0x7f6c801442a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:39.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c86e61700 1 --2- 172.21.15.36:0/4143024669 >> v2:172.21.15.70:3300/0 conn(0x7f6c8013cb70 0x7f6c80143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:39.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c86e61700 1 -- 172.21.15.36:0/4143024669 >> 172.21.15.36:0/4143024669 conn(0x7f6c8005c550 msgr2=0x7f6c800b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:39.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c86e61700 1 -- 172.21.15.36:0/4143024669 shutdown_connections 2022-06-17T12:47:39.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.009+0000 7f6c86e61700 1 -- 172.21.15.36:0/4143024669 wait complete. 2022-06-17T12:47:39.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.013+0000 7f6c86e61700 1 Processor -- start 2022-06-17T12:47:39.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.013+0000 7f6c86e61700 1 -- start start 2022-06-17T12:47:39.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.013+0000 7f6c86e61700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c800aef60 0x7f6c8013b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:39.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.013+0000 7f6c86e61700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c8013cb70 0x7f6c8013d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:39.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.013+0000 7f6c86e61700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c8013cf50 0x7f6c8013d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:39.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.013+0000 7f6c86e61700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c801526c0 con 0x7f6c800aef60 2022-06-17T12:47:39.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.013+0000 7f6c86e61700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c80152960 con 0x7f6c8013cb70 2022-06-17T12:47:39.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.013+0000 7f6c86e61700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8013b770 con 0x7f6c8013cf50 2022-06-17T12:47:39.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.013+0000 7f6c85e5f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c800aef60 0x7f6c8013b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:39.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.013+0000 7f6c85e5f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c800aef60 0x7f6c8013b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60684/0 (socket says 172.21.15.36:60684) 2022-06-17T12:47:39.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.013+0000 7f6c85e5f700 1 -- 172.21.15.36:0/5932723 learned_addr learned my addr 172.21.15.36:0/5932723 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:39.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.013+0000 7f6c8565e700 1 --2- 172.21.15.36:0/5932723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c8013cb70 0x7f6c8013d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:39.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.013+0000 7f6c86660700 1 --2- 172.21.15.36:0/5932723 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c8013cf50 0x7f6c8013d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:39.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.013+0000 7f6c85e5f700 1 -- 172.21.15.36:0/5932723 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c8013cf50 msgr2=0x7f6c8013d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:39.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.013+0000 7f6c85e5f700 1 --2- 172.21.15.36:0/5932723 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c8013cf50 0x7f6c8013d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:39.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.013+0000 7f6c85e5f700 1 -- 172.21.15.36:0/5932723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c8013cb70 msgr2=0x7f6c8013d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:39.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.013+0000 7f6c85e5f700 1 --2- 172.21.15.36:0/5932723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c8013cb70 0x7f6c8013d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:39.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.013+0000 7f6c85e5f700 1 -- 172.21.15.36:0/5932723 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7c00b180 con 0x7f6c800aef60 2022-06-17T12:47:39.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.013+0000 7f6c8565e700 1 --2- 172.21.15.36:0/5932723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c8013cb70 0x7f6c8013d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:39.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.013+0000 7f6c85e5f700 1 --2- 172.21.15.36:0/5932723 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c800aef60 0x7f6c8013b280 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f6c7c00c000 tx=0x7f6c7c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:39.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.017+0000 7f6c6effd700 1 -- 172.21.15.36:0/5932723 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c7c00c890 con 0x7f6c800aef60 2022-06-17T12:47:39.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.017+0000 7f6c6effd700 1 -- 172.21.15.36:0/5932723 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7c00c9f0 con 0x7f6c800aef60 2022-06-17T12:47:39.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.017+0000 7f6c6effd700 1 -- 172.21.15.36:0/5932723 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c7c00cd50 con 0x7f6c800aef60 2022-06-17T12:47:39.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.017+0000 7f6c86e61700 1 -- 172.21.15.36:0/5932723 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c78009470 con 0x7f6c800aef60 2022-06-17T12:47:39.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.017+0000 7f6c86e61700 1 -- 172.21.15.36:0/5932723 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c800b7020 con 0x7f6c800aef60 2022-06-17T12:47:39.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.017+0000 7f6c86e61700 1 -- 172.21.15.36:0/5932723 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c80057410 con 0x7f6c800aef60 2022-06-17T12:47:39.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.017+0000 7f6c6effd700 1 -- 172.21.15.36:0/5932723 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6c7c01c460 con 0x7f6c800aef60 2022-06-17T12:47:39.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.017+0000 7f6c6effd700 1 --2- 172.21.15.36:0/5932723 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c60053ad0 0x7f6c60055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:39.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.017+0000 7f6c6effd700 1 -- 172.21.15.36:0/5932723 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f6c7c06c2e0 con 0x7f6c800aef60 2022-06-17T12:47:39.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.017+0000 7f6c8565e700 1 --2- 172.21.15.36:0/5932723 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c60053ad0 0x7f6c60055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:39.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.021+0000 7f6c8565e700 1 --2- 172.21.15.36:0/5932723 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c60053ad0 0x7f6c60055f90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f6c7000b000 tx=0x7f6c70001140).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:39.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.025+0000 7f6c6effd700 1 -- 172.21.15.36:0/5932723 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6c7c043360 con 0x7f6c800aef60 2022-06-17T12:47:39.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.165+0000 7f6c86e61700 1 -- 172.21.15.36:0/5932723 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f6c80062a80 con 0x7f6c800aef60 2022-06-17T12:47:39.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.905+0000 7f6c6effd700 1 -- 172.21.15.36:0/5932723 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v31) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6c7c046980 con 0x7f6c800aef60 2022-06-17T12:47:39.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.909+0000 7f6c86e61700 1 -- 172.21.15.36:0/5932723 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c60053ad0 msgr2=0x7f6c60055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:39.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.909+0000 7f6c86e61700 1 --2- 172.21.15.36:0/5932723 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c60053ad0 0x7f6c60055f90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f6c7000b000 tx=0x7f6c70001140).stop 2022-06-17T12:47:39.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.909+0000 7f6c86e61700 1 -- 172.21.15.36:0/5932723 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c800aef60 msgr2=0x7f6c8013b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:39.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.909+0000 7f6c86e61700 1 --2- 172.21.15.36:0/5932723 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c800aef60 0x7f6c8013b280 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f6c7c00c000 tx=0x7f6c7c006df0).stop 2022-06-17T12:47:39.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.909+0000 7f6c86e61700 1 -- 172.21.15.36:0/5932723 shutdown_connections 2022-06-17T12:47:39.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.909+0000 7f6c86e61700 1 --2- 172.21.15.36:0/5932723 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c60053ad0 0x7f6c60055f90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:39.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.909+0000 7f6c86e61700 1 --2- 172.21.15.36:0/5932723 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c8013cf50 0x7f6c8013d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:39.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.909+0000 7f6c86e61700 1 --2- 172.21.15.36:0/5932723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c8013cb70 0x7f6c8013d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:39.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.909+0000 7f6c86e61700 1 --2- 172.21.15.36:0/5932723 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c800aef60 0x7f6c8013b280 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:39.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.909+0000 7f6c86e61700 1 -- 172.21.15.36:0/5932723 >> 172.21.15.36:0/5932723 conn(0x7f6c8005c550 msgr2=0x7f6c80139df0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:39.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.909+0000 7f6c86e61700 1 -- 172.21.15.36:0/5932723 shutdown_connections 2022-06-17T12:47:39.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:39.909+0000 7f6c86e61700 1 -- 172.21.15.36:0/5932723 wait complete. 2022-06-17T12:47:39.904 INFO:teuthology.orchestra.run.smithi036.stderr: removed all blacklist entries 2022-06-17T12:47:39.934 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2022-06-17T12:47:39.935 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-06-17T12:47:40.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.065+0000 7feaa5357700 1 Processor -- start 2022-06-17T12:47:40.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.065+0000 7feaa5357700 1 -- start start 2022-06-17T12:47:40.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.065+0000 7feaa5357700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feaa00601c0 0x7feaa01443d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:40.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.065+0000 7feaa5357700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7feaa00ae360 0x7feaa01448c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:40.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.065+0000 7feaa5357700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaa00b2b60 con 0x7feaa00b1db0 2022-06-17T12:47:40.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.065+0000 7feaa5357700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaa00b3230 con 0x7feaa00ae360 2022-06-17T12:47:40.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.065+0000 7feaa5357700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaa00605a0 con 0x7feaa00601c0 2022-06-17T12:47:40.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.069+0000 7fea9f7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7feaa00ae360 0x7feaa01448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:40.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.069+0000 7feaa4b56700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7feaa00b1db0 0x7feaa01499f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:49996/0 (socket says 172.21.15.36:49996) 2022-06-17T12:47:40.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.069+0000 7feaa4b56700 1 -- 172.21.15.36:0/2848366588 learned_addr learned my addr 172.21.15.36:0/2848366588 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:40.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.069+0000 7fea9ffff700 1 --2- 172.21.15.36:0/2848366588 >> v2:172.21.15.70:3300/0 conn(0x7feaa00601c0 0x7feaa01443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:40.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.069+0000 7fea9f7fe700 1 -- 172.21.15.36:0/2848366588 >> v2:172.21.15.70:3300/0 conn(0x7feaa00601c0 msgr2=0x7feaa01443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:40.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.069+0000 7fea9f7fe700 1 --2- 172.21.15.36:0/2848366588 >> v2:172.21.15.70:3300/0 conn(0x7feaa00601c0 0x7feaa01443d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:40.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.069+0000 7fea9f7fe700 1 -- 172.21.15.36:0/2848366588 >> v1:172.21.15.36:6789/0 conn(0x7feaa00b1db0 legacy=0x7feaa01499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:40.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.069+0000 7fea9f7fe700 1 -- 172.21.15.36:0/2848366588 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa00662f0 con 0x7feaa00ae360 2022-06-17T12:47:40.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.069+0000 7fea9f7fe700 1 --2- 172.21.15.36:0/2848366588 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaa00ae360 0x7feaa01448c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fea880032f0 tx=0x7fea880091d0).ready entity=mon.0 client_cookie=706199a4edd79437 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:40.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.069+0000 7fea9effd700 1 -- 172.21.15.36:0/2848366588 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea88009ea0 con 0x7feaa00ae360 2022-06-17T12:47:40.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.069+0000 7fea9effd700 1 -- 172.21.15.36:0/2848366588 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8800b870 con 0x7feaa00ae360 2022-06-17T12:47:40.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.069+0000 7fea9effd700 1 -- 172.21.15.36:0/2848366588 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea8800b9e0 con 0x7feaa00ae360 2022-06-17T12:47:40.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.069+0000 7feaa5357700 1 -- 172.21.15.36:0/2848366588 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaa00ae360 msgr2=0x7feaa01448c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:40.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.069+0000 7feaa5357700 1 --2- 172.21.15.36:0/2848366588 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaa00ae360 0x7feaa01448c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fea880032f0 tx=0x7fea880091d0).stop 2022-06-17T12:47:40.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.069+0000 7feaa5357700 1 -- 172.21.15.36:0/2848366588 shutdown_connections 2022-06-17T12:47:40.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.069+0000 7feaa5357700 1 --2- 172.21.15.36:0/2848366588 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaa00ae360 0x7feaa01448c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:40.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.069+0000 7feaa5357700 1 --2- 172.21.15.36:0/2848366588 >> v2:172.21.15.70:3300/0 conn(0x7feaa00601c0 0x7feaa01443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:40.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.069+0000 7feaa5357700 1 -- 172.21.15.36:0/2848366588 >> 172.21.15.36:0/2848366588 conn(0x7feaa0060990 msgr2=0x7feaa005f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:40.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.069+0000 7feaa5357700 1 -- 172.21.15.36:0/2848366588 shutdown_connections 2022-06-17T12:47:40.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.069+0000 7feaa5357700 1 -- 172.21.15.36:0/2848366588 wait complete. 2022-06-17T12:47:40.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.069+0000 7feaa5357700 1 Processor -- start 2022-06-17T12:47:40.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.069+0000 7feaa5357700 1 -- start start 2022-06-17T12:47:40.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.073+0000 7feaa5357700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaa00601c0 0x7feaa0110600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:40.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.073+0000 7feaa5357700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaa00ae360 0x7feaa014dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:40.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.073+0000 7feaa5357700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaa00b1db0 0x7feaa0143450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:40.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.073+0000 7feaa5357700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa013e8e0 con 0x7feaa00ae360 2022-06-17T12:47:40.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.073+0000 7feaa5357700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa0111020 con 0x7feaa00b1db0 2022-06-17T12:47:40.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.073+0000 7feaa5357700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa0110af0 con 0x7feaa00601c0 2022-06-17T12:47:40.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.073+0000 7fea9f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaa00ae360 0x7feaa014dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:40.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.073+0000 7fea9f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaa00ae360 0x7feaa014dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60702/0 (socket says 172.21.15.36:60702) 2022-06-17T12:47:40.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.073+0000 7fea9f7fe700 1 -- 172.21.15.36:0/2504116715 learned_addr learned my addr 172.21.15.36:0/2504116715 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:40.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.073+0000 7fea9ffff700 1 --2- 172.21.15.36:0/2504116715 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaa00601c0 0x7feaa0110600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:40.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.073+0000 7feaa4b56700 1 --2- 172.21.15.36:0/2504116715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaa00b1db0 0x7feaa0143450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:40.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.073+0000 7fea9f7fe700 1 -- 172.21.15.36:0/2504116715 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaa00601c0 msgr2=0x7feaa0110600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:40.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.073+0000 7fea9f7fe700 1 --2- 172.21.15.36:0/2504116715 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaa00601c0 0x7feaa0110600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:40.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.073+0000 7fea9f7fe700 1 -- 172.21.15.36:0/2504116715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaa00b1db0 msgr2=0x7feaa0143450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:40.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.073+0000 7fea9f7fe700 1 --2- 172.21.15.36:0/2504116715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaa00b1db0 0x7feaa0143450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:40.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.073+0000 7fea9f7fe700 1 -- 172.21.15.36:0/2504116715 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea88009470 con 0x7feaa00ae360 2022-06-17T12:47:40.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.073+0000 7fea9f7fe700 1 --2- 172.21.15.36:0/2504116715 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaa00ae360 0x7feaa014dfc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fea880032f0 tx=0x7fea88016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:40.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.073+0000 7fea9d7fa700 1 -- 172.21.15.36:0/2504116715 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea88011b40 con 0x7feaa00ae360 2022-06-17T12:47:40.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.073+0000 7fea9d7fa700 1 -- 172.21.15.36:0/2504116715 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea88011ca0 con 0x7feaa00ae360 2022-06-17T12:47:40.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.073+0000 7feaa5357700 1 -- 172.21.15.36:0/2504116715 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa0112000 con 0x7feaa00ae360 2022-06-17T12:47:40.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.073+0000 7fea9d7fa700 1 -- 172.21.15.36:0/2504116715 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea8801f3f0 con 0x7feaa00ae360 2022-06-17T12:47:40.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.073+0000 7feaa5357700 1 -- 172.21.15.36:0/2504116715 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa014e4b0 con 0x7feaa00ae360 2022-06-17T12:47:40.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.073+0000 7feaa5357700 1 -- 172.21.15.36:0/2504116715 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaa0112190 con 0x7feaa00ae360 2022-06-17T12:47:40.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.073+0000 7fea9d7fa700 1 -- 172.21.15.36:0/2504116715 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fea88026030 con 0x7feaa00ae360 2022-06-17T12:47:40.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.077+0000 7fea9d7fa700 1 --2- 172.21.15.36:0/2504116715 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fea84053b20 0x7fea84055fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:40.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.077+0000 7fea9d7fa700 1 -- 172.21.15.36:0/2504116715 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fea8806c8c0 con 0x7feaa00ae360 2022-06-17T12:47:40.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.077+0000 7fea9ffff700 1 --2- 172.21.15.36:0/2504116715 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fea84053b20 0x7fea84055fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:40.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.077+0000 7fea9ffff700 1 --2- 172.21.15.36:0/2504116715 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fea84053b20 0x7fea84055fe0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fea9000c000 tx=0x7fea90008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:40.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.081+0000 7fea9d7fa700 1 -- 172.21.15.36:0/2504116715 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fea88043a00 con 0x7feaa00ae360 2022-06-17T12:47:40.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.245+0000 7feaa5357700 1 -- 172.21.15.36:0/2504116715 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7feaa0066590 con 0x7feaa00ae360 2022-06-17T12:47:40.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.245+0000 7fea9d7fa700 1 -- 172.21.15.36:0/2504116715 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+5627 (secure 0 0 0) 0x7fea88015020 con 0x7feaa00ae360 2022-06-17T12:47:40.393 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:40.394 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:47:40.394 INFO:teuthology.orchestra.run.smithi036.stdout: "auth_dump": [ 2022-06-17T12:47:40.395 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:40.395 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mds.a", 2022-06-17T12:47:40.396 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiyaUgARAAxp7bzOl4uVo2EsaoRBeMRw==", 2022-06-17T12:47:40.396 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:47:40.396 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:47:40.397 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:47:40.397 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:47:40.399 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:47:40.399 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:40.400 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:40.401 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:40.401 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mds.b", 2022-06-17T12:47:40.402 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xifyxLCRAABI389jmz1zboeDanzraT8Q==", 2022-06-17T12:47:40.402 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:47:40.403 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:47:40.403 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:47:40.404 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:47:40.404 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:47:40.405 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:40.405 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:40.406 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:40.417 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.0", 2022-06-17T12:47:40.417 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB+d6xibhXGNBAA6AuUT9ITWAZjDFpNsgA4Wg==", 2022-06-17T12:47:40.418 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:47:40.418 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:47:40.419 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:47:40.419 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:47:40.420 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:40.420 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:40.420 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:40.431 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.1", 2022-06-17T12:47:40.432 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCBd6xiOIPvGxAADkrBoJCy9x91zan6TWU0tQ==", 2022-06-17T12:47:40.432 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:47:40.433 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:47:40.433 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:47:40.433 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:47:40.434 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:40.434 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:40.435 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:40.435 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.2", 2022-06-17T12:47:40.436 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCEd6xiP4/9AhAAaP3cWXV6vbfGBPfqSLKqUg==", 2022-06-17T12:47:40.436 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:47:40.437 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:47:40.439 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:47:40.439 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:47:40.440 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:40.440 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:40.441 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:40.441 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.3", 2022-06-17T12:47:40.442 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCGd6xi+LZgJxAAJjGwP5qW6TFDZfR+2rz6ig==", 2022-06-17T12:47:40.442 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:47:40.443 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:47:40.443 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:47:40.444 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:47:40.444 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:40.444 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:40.445 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:40.445 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.4", 2022-06-17T12:47:40.445 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCKd6xikvrTChAAXIyWSETswDv+tb3zsGyYpA==", 2022-06-17T12:47:40.482 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:47:40.483 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:47:40.486 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:47:40.486 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:47:40.487 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:40.487 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:40.488 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:40.488 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.5", 2022-06-17T12:47:40.489 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCMd6xiqr2QLhAArhimHuNsVWrDGGreFYxj6A==", 2022-06-17T12:47:40.489 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:47:40.490 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:47:40.490 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:47:40.491 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:47:40.491 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:40.492 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:40.493 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:40.493 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.6", 2022-06-17T12:47:40.495 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCPd6xiUXcXGxAAdrYz8txdBWMIqF5DF4Aimw==", 2022-06-17T12:47:40.514 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:47:40.523 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:47:40.524 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:47:40.524 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:47:40.524 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:40.525 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:40.525 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:40.527 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.7", 2022-06-17T12:47:40.527 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCSd6xiBJ2IAhAAUkNfgmlxxWY7+VouPLAvyw==", 2022-06-17T12:47:40.527 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:47:40.529 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:47:40.530 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:47:40.530 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:47:40.531 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:40.531 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:40.532 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:40.532 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.0", 2022-06-17T12:47:40.533 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiXO3QFBAAsSLtQ0yKKZxhqUT/eYKdtg==", 2022-06-17T12:47:40.533 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:47:40.533 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:47:40.534 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow r", 2022-06-17T12:47:40.534 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow rw", 2022-06-17T12:47:40.535 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rwx" 2022-06-17T12:47:40.535 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:40.536 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:40.537 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:40.537 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.1", 2022-06-17T12:47:40.538 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xi51KdGBAAi7fGZ+Qepg6j3DAhlzoWxw==", 2022-06-17T12:47:40.538 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:47:40.538 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:47:40.540 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow r", 2022-06-17T12:47:40.540 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow rw", 2022-06-17T12:47:40.541 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rwx" 2022-06-17T12:47:40.541 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:40.542 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:40.542 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:40.542 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.2", 2022-06-17T12:47:40.543 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiug2hHBAAqHpLgwMdthSWtJ8Vwu+m1Q==", 2022-06-17T12:47:40.543 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:47:40.544 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:47:40.544 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow r", 2022-06-17T12:47:40.545 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow rw", 2022-06-17T12:47:40.545 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rwx" 2022-06-17T12:47:40.546 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:40.546 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:40.547 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:40.548 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.admin", 2022-06-17T12:47:40.548 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB5d6xiGDODKhAAPZMG/M2nVssdr4FJHcS6XQ==", 2022-06-17T12:47:40.548 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:47:40.549 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:47:40.550 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:47:40.551 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:47:40.551 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:47:40.552 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:40.552 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:40.553 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:40.553 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.bootstrap-mds", 2022-06-17T12:47:40.553 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCmd6xiKYcgKBAA1f/Nt6lOwx8QmDl8IB/Wgg==", 2022-06-17T12:47:40.561 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:47:40.583 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile bootstrap-mds" 2022-06-17T12:47:40.583 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:40.584 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:40.584 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:40.603 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.bootstrap-mgr", 2022-06-17T12:47:40.603 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCmd6xiQZ8gKBAA3dr9Pt0wRFHVAyndNyqPiw==", 2022-06-17T12:47:40.604 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:47:40.604 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile bootstrap-mgr" 2022-06-17T12:47:40.605 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:40.605 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:40.666 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:40.666 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.bootstrap-osd", 2022-06-17T12:47:40.667 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCmd6xiUbQgKBAAmcz9BAtl6VefV8UIFU0kdg==", 2022-06-17T12:47:40.667 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:47:40.668 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile bootstrap-osd" 2022-06-17T12:47:40.668 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:40.669 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:40.669 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:40.670 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.bootstrap-rbd", 2022-06-17T12:47:40.670 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCmd6xiWcogKBAAVexYdlL22Vio3Zw7G0LmMw==", 2022-06-17T12:47:40.671 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:47:40.671 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile bootstrap-rbd" 2022-06-17T12:47:40.671 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:40.672 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:40.672 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:40.673 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.bootstrap-rbd-mirror", 2022-06-17T12:47:40.673 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCmd6xiTuEgKBAABGzDw17OgHFrf0yZ4dpG1Q==", 2022-06-17T12:47:40.674 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:47:40.675 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2022-06-17T12:47:40.675 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:40.676 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:40.676 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:40.677 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.bootstrap-rgw", 2022-06-17T12:47:40.677 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCmd6xic/cgKBAA6r8Tw3byCP0Yn56cbX5nFQ==", 2022-06-17T12:47:40.677 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:47:40.678 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile bootstrap-rgw" 2022-06-17T12:47:40.678 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:40.679 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:40.680 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:40.680 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mgr.x", 2022-06-17T12:47:40.681 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB6d6xiyH9uORAATJb0chHE1CH0X02w7edVtw==", 2022-06-17T12:47:40.681 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:47:40.682 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:47:40.682 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile mgr", 2022-06-17T12:47:40.683 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:47:40.683 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:40.684 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:40.684 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:40.684 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mgr.y", 2022-06-17T12:47:40.685 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB6d6xiHXghMxAAzpsVRCbLWHWZPtrPk1jFlA==", 2022-06-17T12:47:40.685 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:47:40.685 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:47:40.687 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile mgr", 2022-06-17T12:47:40.687 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:47:40.687 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:40.688 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:40.688 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:47:40.689 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:47:40.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.249+0000 7feaa5357700 1 -- 172.21.15.36:0/2504116715 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fea84053b20 msgr2=0x7fea84055fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:40.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.249+0000 7feaa5357700 1 --2- 172.21.15.36:0/2504116715 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fea84053b20 0x7fea84055fe0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fea9000c000 tx=0x7fea90008040).stop 2022-06-17T12:47:40.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.249+0000 7feaa5357700 1 -- 172.21.15.36:0/2504116715 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaa00ae360 msgr2=0x7feaa014dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:40.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.249+0000 7feaa5357700 1 --2- 172.21.15.36:0/2504116715 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaa00ae360 0x7feaa014dfc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fea880032f0 tx=0x7fea88016000).stop 2022-06-17T12:47:40.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.249+0000 7feaa5357700 1 -- 172.21.15.36:0/2504116715 shutdown_connections 2022-06-17T12:47:40.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.249+0000 7feaa5357700 1 --2- 172.21.15.36:0/2504116715 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fea84053b20 0x7fea84055fe0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:40.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.249+0000 7feaa5357700 1 --2- 172.21.15.36:0/2504116715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaa00b1db0 0x7feaa0143450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:40.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.249+0000 7feaa5357700 1 --2- 172.21.15.36:0/2504116715 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaa00ae360 0x7feaa014dfc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:40.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.249+0000 7feaa5357700 1 --2- 172.21.15.36:0/2504116715 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaa00601c0 0x7feaa0110600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:40.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.249+0000 7feaa5357700 1 -- 172.21.15.36:0/2504116715 >> 172.21.15.36:0/2504116715 conn(0x7feaa0060990 msgr2=0x7feaa0067010 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:40.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.249+0000 7feaa5357700 1 -- 172.21.15.36:0/2504116715 shutdown_connections 2022-06-17T12:47:40.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.249+0000 7feaa5357700 1 -- 172.21.15.36:0/2504116715 wait complete. 2022-06-17T12:47:40.748 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2022-06-17T12:47:40.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.925+0000 7f6974e5c700 1 Processor -- start 2022-06-17T12:47:40.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.925+0000 7f6974e5c700 1 -- start start 2022-06-17T12:47:40.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.925+0000 7f6974e5c700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f69700ae360 0x7f6970144ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:40.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.925+0000 7f6974e5c700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69700b2b60 con 0x7f6970144fd0 2022-06-17T12:47:40.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.925+0000 7f6974e5c700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69700b3230 con 0x7f69700601c0 2022-06-17T12:47:40.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.925+0000 7f6974e5c700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69700605a0 con 0x7f69700ae360 2022-06-17T12:47:40.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.929+0000 7f696ffff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f6970144fd0 0x7f697014aa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50010/0 (socket says 172.21.15.36:50010) 2022-06-17T12:47:40.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.929+0000 7f696ffff700 1 -- 172.21.15.36:0/538613680 learned_addr learned my addr 172.21.15.36:0/538613680 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:40.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.929+0000 7f696effd700 1 --2- 172.21.15.36:0/538613680 >> v2:172.21.15.70:3300/0 conn(0x7f69700ae360 0x7f6970144ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:40.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.929+0000 7f696e7fc700 1 -- 172.21.15.36:0/538613680 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1625267567 0 0) 0x7f69700b2b60 con 0x7f6970144fd0 2022-06-17T12:47:40.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.929+0000 7f696e7fc700 1 -- 172.21.15.36:0/538613680 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6950003260 con 0x7f6970144fd0 2022-06-17T12:47:40.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.929+0000 7f696effd700 1 -- 172.21.15.36:0/538613680 >> v1:172.21.15.70:6789/0 conn(0x7f69700601c0 legacy=0x7f69701443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:40.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.929+0000 7f696effd700 1 -- 172.21.15.36:0/538613680 >> v1:172.21.15.36:6789/0 conn(0x7f6970144fd0 legacy=0x7f697014aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:40.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.929+0000 7f696e7fc700 1 -- 172.21.15.36:0/538613680 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3168218738 0 0) 0x7f6950003260 con 0x7f6970144fd0 2022-06-17T12:47:40.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.929+0000 7f696effd700 1 -- 172.21.15.36:0/538613680 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69700662f0 con 0x7f69700ae360 2022-06-17T12:47:40.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.929+0000 7f696effd700 1 --2- 172.21.15.36:0/538613680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69700ae360 0x7f6970144ae0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f6958002f70 tx=0x7f6958006df0).ready entity=mon.1 client_cookie=f5350404026502d0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:40.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.929+0000 7f696e7fc700 1 -- 172.21.15.36:0/538613680 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f695800d610 con 0x7f69700ae360 2022-06-17T12:47:40.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.929+0000 7f696e7fc700 1 -- 172.21.15.36:0/538613680 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6958000e90 con 0x7f69700ae360 2022-06-17T12:47:40.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.929+0000 7f696e7fc700 1 -- 172.21.15.36:0/538613680 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f695800d960 con 0x7f69700ae360 2022-06-17T12:47:40.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.929+0000 7f6974e5c700 1 -- 172.21.15.36:0/538613680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69700ae360 msgr2=0x7f6970144ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:40.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.929+0000 7f6974e5c700 1 --2- 172.21.15.36:0/538613680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69700ae360 0x7f6970144ae0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f6958002f70 tx=0x7f6958006df0).stop 2022-06-17T12:47:40.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.929+0000 7f6974e5c700 1 -- 172.21.15.36:0/538613680 shutdown_connections 2022-06-17T12:47:40.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.929+0000 7f6974e5c700 1 --2- 172.21.15.36:0/538613680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69700ae360 0x7f6970144ae0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:40.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.929+0000 7f6974e5c700 1 -- 172.21.15.36:0/538613680 >> 172.21.15.36:0/538613680 conn(0x7f6970060990 msgr2=0x7f697005f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:40.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.929+0000 7f6974e5c700 1 -- 172.21.15.36:0/538613680 shutdown_connections 2022-06-17T12:47:40.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.929+0000 7f6974e5c700 1 -- 172.21.15.36:0/538613680 wait complete. 2022-06-17T12:47:40.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.933+0000 7f6974e5c700 1 Processor -- start 2022-06-17T12:47:40.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.933+0000 7f6974e5c700 1 -- start start 2022-06-17T12:47:40.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.933+0000 7f6974e5c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f69700601c0 0x7f69700aea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:40.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.933+0000 7f6974e5c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69700ae360 0x7f69701429f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:40.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.933+0000 7f6974e5c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6970144fd0 0x7f6970151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:40.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.933+0000 7f6974e5c700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6970144740 con 0x7f69700601c0 2022-06-17T12:47:40.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.933+0000 7f6974e5c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69701449e0 con 0x7f6970144fd0 2022-06-17T12:47:40.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.933+0000 7f6974e5c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69700aef20 con 0x7f69700ae360 2022-06-17T12:47:41.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.933+0000 7f696f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f69700601c0 0x7f69700aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:41.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.933+0000 7f696f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f69700601c0 0x7f69700aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60716/0 (socket says 172.21.15.36:60716) 2022-06-17T12:47:41.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.933+0000 7f696f7fe700 1 -- 172.21.15.36:0/3134139014 learned_addr learned my addr 172.21.15.36:0/3134139014 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:41.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.933+0000 7f696effd700 1 --2- 172.21.15.36:0/3134139014 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69700ae360 0x7f69701429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:41.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.933+0000 7f696ffff700 1 --2- 172.21.15.36:0/3134139014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6970144fd0 0x7f6970151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:41.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.933+0000 7f696f7fe700 1 -- 172.21.15.36:0/3134139014 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69700ae360 msgr2=0x7f69701429f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:41.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.933+0000 7f696f7fe700 1 --2- 172.21.15.36:0/3134139014 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69700ae360 0x7f69701429f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:41.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.933+0000 7f696f7fe700 1 -- 172.21.15.36:0/3134139014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6970144fd0 msgr2=0x7f6970151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:41.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.933+0000 7f696f7fe700 1 --2- 172.21.15.36:0/3134139014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6970144fd0 0x7f6970151630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:41.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.933+0000 7f696f7fe700 1 -- 172.21.15.36:0/3134139014 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f695800b180 con 0x7f69700601c0 2022-06-17T12:47:41.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.933+0000 7f696f7fe700 1 --2- 172.21.15.36:0/3134139014 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f69700601c0 0x7f69700aea30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f6970061040 tx=0x7f6960007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:41.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.933+0000 7f696cff9700 1 -- 172.21.15.36:0/3134139014 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6960012400 con 0x7f69700601c0 2022-06-17T12:47:41.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.933+0000 7f696cff9700 1 -- 172.21.15.36:0/3134139014 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6960012560 con 0x7f69700601c0 2022-06-17T12:47:41.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.933+0000 7f6974e5c700 1 -- 172.21.15.36:0/3134139014 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69701423b0 con 0x7f69700601c0 2022-06-17T12:47:41.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.933+0000 7f696cff9700 1 -- 172.21.15.36:0/3134139014 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69600128c0 con 0x7f69700601c0 2022-06-17T12:47:41.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.933+0000 7f6974e5c700 1 -- 172.21.15.36:0/3134139014 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69700620f0 con 0x7f69700601c0 2022-06-17T12:47:41.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.937+0000 7f6974e5c700 1 -- 172.21.15.36:0/3134139014 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6970143220 con 0x7f69700601c0 2022-06-17T12:47:41.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.937+0000 7f696cff9700 1 -- 172.21.15.36:0/3134139014 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6960019030 con 0x7f69700601c0 2022-06-17T12:47:41.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.937+0000 7f696cff9700 1 --2- 172.21.15.36:0/3134139014 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6954053ad0 0x7f6954055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:41.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.937+0000 7f696cff9700 1 -- 172.21.15.36:0/3134139014 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f696006ae50 con 0x7f69700601c0 2022-06-17T12:47:41.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.937+0000 7f696effd700 1 --2- 172.21.15.36:0/3134139014 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6954053ad0 0x7f6954055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:41.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.937+0000 7f696effd700 1 --2- 172.21.15.36:0/3134139014 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6954053ad0 0x7f6954055f90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f6958018000 tx=0x7f6958014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:41.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:40.945+0000 7f696cff9700 1 -- 172.21.15.36:0/3134139014 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6970143220 con 0x7f69700601c0 2022-06-17T12:47:41.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.109+0000 7f6974e5c700 1 -- 172.21.15.36:0/3134139014 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f69700b3230 con 0x7f69700601c0 2022-06-17T12:47:41.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.113+0000 7f696cff9700 1 -- 172.21.15.36:0/3134139014 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 updated v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7f69700b3230 con 0x7f69700601c0 2022-06-17T12:47:41.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.113+0000 7f6974e5c700 1 -- 172.21.15.36:0/3134139014 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6954053ad0 msgr2=0x7f6954055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:41.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.113+0000 7f6974e5c700 1 --2- 172.21.15.36:0/3134139014 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6954053ad0 0x7f6954055f90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f6958018000 tx=0x7f6958014000).stop 2022-06-17T12:47:41.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.113+0000 7f6974e5c700 1 -- 172.21.15.36:0/3134139014 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f69700601c0 msgr2=0x7f69700aea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:41.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.113+0000 7f6974e5c700 1 --2- 172.21.15.36:0/3134139014 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f69700601c0 0x7f69700aea30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f6970061040 tx=0x7f6960007c50).stop 2022-06-17T12:47:41.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.117+0000 7f6974e5c700 1 -- 172.21.15.36:0/3134139014 shutdown_connections 2022-06-17T12:47:41.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.117+0000 7f6974e5c700 1 --2- 172.21.15.36:0/3134139014 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6954053ad0 0x7f6954055f90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:41.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.117+0000 7f6974e5c700 1 --2- 172.21.15.36:0/3134139014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6970144fd0 0x7f6970151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:41.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.117+0000 7f6974e5c700 1 --2- 172.21.15.36:0/3134139014 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69700ae360 0x7f69701429f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:41.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.117+0000 7f6974e5c700 1 --2- 172.21.15.36:0/3134139014 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f69700601c0 0x7f69700aea30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:41.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.117+0000 7f6974e5c700 1 -- 172.21.15.36:0/3134139014 >> 172.21.15.36:0/3134139014 conn(0x7f6970060990 msgr2=0x7f69700b6270 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:41.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.117+0000 7f6974e5c700 1 -- 172.21.15.36:0/3134139014 shutdown_connections 2022-06-17T12:47:41.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.117+0000 7f6974e5c700 1 -- 172.21.15.36:0/3134139014 wait complete. 2022-06-17T12:47:41.125 INFO:teuthology.orchestra.run.smithi036.stderr:updated 2022-06-17T12:47:41.127 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2022-06-17T12:47:41.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.249+0000 7f803e77f700 1 Processor -- start 2022-06-17T12:47:41.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.249+0000 7f803e77f700 1 -- start start 2022-06-17T12:47:41.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.249+0000 7f803e77f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f80380ad140 0x7f80380af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:41.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.249+0000 7f803e77f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80380ac340 0x7f80380ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:41.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.249+0000 7f803e77f700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f803814aca0 0x7f803814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:41.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.249+0000 7f803e77f700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8038148960 con 0x7f80380ad140 2022-06-17T12:47:41.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.249+0000 7f803e77f700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80380ac870 con 0x7f803814aca0 2022-06-17T12:47:41.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.249+0000 7f803e77f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80380acaa0 con 0x7f80380ac340 2022-06-17T12:47:41.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.249+0000 7f8037fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f80380ad140 0x7f80380af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:41.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.249+0000 7f8037fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f80380ad140 0x7f80380af7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60724/0 (socket says 172.21.15.36:60724) 2022-06-17T12:47:41.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.249+0000 7f8037fff700 1 -- 172.21.15.36:0/3041537331 learned_addr learned my addr 172.21.15.36:0/3041537331 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:41.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.253+0000 7f803cd1c700 1 --2- 172.21.15.36:0/3041537331 >> v2:172.21.15.70:3300/0 conn(0x7f803814aca0 0x7f803814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:41.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.253+0000 7f80377fe700 1 --2- 172.21.15.36:0/3041537331 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80380ac340 0x7f80380ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:41.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.253+0000 7f8037fff700 1 -- 172.21.15.36:0/3041537331 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80380ac340 msgr2=0x7f80380ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:41.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.253+0000 7f8037fff700 1 --2- 172.21.15.36:0/3041537331 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80380ac340 0x7f80380ad520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:41.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.253+0000 7f8037fff700 1 -- 172.21.15.36:0/3041537331 >> v2:172.21.15.70:3300/0 conn(0x7f803814aca0 msgr2=0x7f803814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:41.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.253+0000 7f8037fff700 1 --2- 172.21.15.36:0/3041537331 >> v2:172.21.15.70:3300/0 conn(0x7f803814aca0 0x7f803814b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:41.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.253+0000 7f8037fff700 1 -- 172.21.15.36:0/3041537331 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80380b0bf0 con 0x7f80380ad140 2022-06-17T12:47:41.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.253+0000 7f8037fff700 1 --2- 172.21.15.36:0/3041537331 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f80380ad140 0x7f80380af7f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f80380ac760 tx=0x7f8028006eb0).ready entity=mon.0 client_cookie=1f613685d100b693 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:41.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.253+0000 7f8036ffd700 1 -- 172.21.15.36:0/3041537331 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f802800e610 con 0x7f80380ad140 2022-06-17T12:47:41.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.253+0000 7f8036ffd700 1 -- 172.21.15.36:0/3041537331 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8028000e90 con 0x7f80380ad140 2022-06-17T12:47:41.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.253+0000 7f8036ffd700 1 -- 172.21.15.36:0/3041537331 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f802800e9a0 con 0x7f80380ad140 2022-06-17T12:47:41.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.253+0000 7f803e77f700 1 -- 172.21.15.36:0/3041537331 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f80380ad140 msgr2=0x7f80380af7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:41.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.253+0000 7f803e77f700 1 --2- 172.21.15.36:0/3041537331 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f80380ad140 0x7f80380af7f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f80380ac760 tx=0x7f8028006eb0).stop 2022-06-17T12:47:41.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.253+0000 7f803e77f700 1 -- 172.21.15.36:0/3041537331 shutdown_connections 2022-06-17T12:47:41.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.253+0000 7f803e77f700 1 --2- 172.21.15.36:0/3041537331 >> v2:172.21.15.70:3300/0 conn(0x7f803814aca0 0x7f803814b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:41.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.253+0000 7f803e77f700 1 --2- 172.21.15.36:0/3041537331 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80380ac340 0x7f80380ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:41.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.253+0000 7f803e77f700 1 --2- 172.21.15.36:0/3041537331 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f80380ad140 0x7f80380af7f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:41.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.253+0000 7f803e77f700 1 -- 172.21.15.36:0/3041537331 >> 172.21.15.36:0/3041537331 conn(0x7f80380b5430 msgr2=0x7f80380b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:41.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.253+0000 7f803e77f700 1 -- 172.21.15.36:0/3041537331 shutdown_connections 2022-06-17T12:47:41.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.253+0000 7f803e77f700 1 -- 172.21.15.36:0/3041537331 wait complete. 2022-06-17T12:47:41.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.253+0000 7f803e77f700 1 Processor -- start 2022-06-17T12:47:41.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.253+0000 7f803e77f700 1 -- start start 2022-06-17T12:47:41.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.257+0000 7f803e77f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80380ac340 0x7f80380a92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:41.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.257+0000 7f803e77f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80380ad140 0x7f803815a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:41.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.257+0000 7f803e77f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f803814aca0 0x7f8038156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:41.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.257+0000 7f803e77f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8038154570 con 0x7f803814aca0 2022-06-17T12:47:41.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.257+0000 7f803e77f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8038063f00 con 0x7f80380ad140 2022-06-17T12:47:41.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.257+0000 7f803e77f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803815a1e0 con 0x7f80380ac340 2022-06-17T12:47:41.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.257+0000 7f803cd1c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f803814aca0 0x7f8038156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:41.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.257+0000 7f8037fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80380ac340 0x7f80380a92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:41.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.257+0000 7f80377fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80380ad140 0x7f803815a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:41.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.257+0000 7f803cd1c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f803814aca0 0x7f8038156ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60732/0 (socket says 172.21.15.36:60732) 2022-06-17T12:47:41.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.257+0000 7f803cd1c700 1 -- 172.21.15.36:0/3011680902 learned_addr learned my addr 172.21.15.36:0/3011680902 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:41.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.257+0000 7f803cd1c700 1 -- 172.21.15.36:0/3011680902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80380ac340 msgr2=0x7f80380a92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:41.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.257+0000 7f803cd1c700 1 --2- 172.21.15.36:0/3011680902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80380ac340 0x7f80380a92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:41.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.257+0000 7f803cd1c700 1 -- 172.21.15.36:0/3011680902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80380ad140 msgr2=0x7f803815a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:41.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.257+0000 7f803cd1c700 1 --2- 172.21.15.36:0/3011680902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80380ad140 0x7f803815a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:41.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.257+0000 7f803cd1c700 1 -- 172.21.15.36:0/3011680902 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802800b1e0 con 0x7f803814aca0 2022-06-17T12:47:41.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.257+0000 7f803cd1c700 1 --2- 172.21.15.36:0/3011680902 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f803814aca0 0x7f8038156ac0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f8030006e00 tx=0x7f80300091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:41.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.257+0000 7f80357fa700 1 -- 172.21.15.36:0/3011680902 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f803000c670 con 0x7f803814aca0 2022-06-17T12:47:41.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.257+0000 7f80357fa700 1 -- 172.21.15.36:0/3011680902 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803000c7d0 con 0x7f803814aca0 2022-06-17T12:47:41.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.257+0000 7f803e77f700 1 -- 172.21.15.36:0/3011680902 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803815a480 con 0x7f803814aca0 2022-06-17T12:47:41.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.257+0000 7f80357fa700 1 -- 172.21.15.36:0/3011680902 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f803000cb30 con 0x7f803814aca0 2022-06-17T12:47:41.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.257+0000 7f803e77f700 1 -- 172.21.15.36:0/3011680902 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80380b4ff0 con 0x7f803814aca0 2022-06-17T12:47:41.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.257+0000 7f803e77f700 1 -- 172.21.15.36:0/3011680902 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f803815a610 con 0x7f803814aca0 2022-06-17T12:47:41.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.261+0000 7f80357fa700 1 -- 172.21.15.36:0/3011680902 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f803000cc90 con 0x7f803814aca0 2022-06-17T12:47:41.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.261+0000 7f80357fa700 1 --2- 172.21.15.36:0/3011680902 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f801c053a40 0x7f801c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:41.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.261+0000 7f80357fa700 1 -- 172.21.15.36:0/3011680902 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f803006bef0 con 0x7f803814aca0 2022-06-17T12:47:41.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.261+0000 7f8037fff700 1 --2- 172.21.15.36:0/3011680902 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f801c053a40 0x7f801c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:41.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.261+0000 7f8037fff700 1 --2- 172.21.15.36:0/3011680902 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f801c053a40 0x7f801c055f00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f802801c000 tx=0x7f8028015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:41.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.265+0000 7f80357fa700 1 -- 172.21.15.36:0/3011680902 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8030043030 con 0x7f803814aca0 2022-06-17T12:47:41.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.437+0000 7f803e77f700 1 -- 172.21.15.36:0/3011680902 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f8038057410 con 0x7f803814aca0 2022-06-17T12:47:41.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.445+0000 7f80357fa700 1 -- 172.21.15.36:0/3011680902 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 updated v8) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8030046650 con 0x7f803814aca0 2022-06-17T12:47:41.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.445+0000 7f803e77f700 1 -- 172.21.15.36:0/3011680902 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f801c053a40 msgr2=0x7f801c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:41.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.445+0000 7f803e77f700 1 --2- 172.21.15.36:0/3011680902 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f801c053a40 0x7f801c055f00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f802801c000 tx=0x7f8028015000).stop 2022-06-17T12:47:41.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.445+0000 7f803e77f700 1 -- 172.21.15.36:0/3011680902 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f803814aca0 msgr2=0x7f8038156ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:41.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.445+0000 7f803e77f700 1 --2- 172.21.15.36:0/3011680902 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f803814aca0 0x7f8038156ac0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f8030006e00 tx=0x7f80300091d0).stop 2022-06-17T12:47:41.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.445+0000 7f803e77f700 1 -- 172.21.15.36:0/3011680902 shutdown_connections 2022-06-17T12:47:41.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.445+0000 7f803e77f700 1 --2- 172.21.15.36:0/3011680902 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f801c053a40 0x7f801c055f00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:41.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.445+0000 7f803e77f700 1 --2- 172.21.15.36:0/3011680902 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f803814aca0 0x7f8038156ac0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:41.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.445+0000 7f803e77f700 1 --2- 172.21.15.36:0/3011680902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80380ad140 0x7f803815a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:41.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.445+0000 7f803e77f700 1 --2- 172.21.15.36:0/3011680902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80380ac340 0x7f80380a92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:41.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.445+0000 7f803e77f700 1 -- 172.21.15.36:0/3011680902 >> 172.21.15.36:0/3011680902 conn(0x7f80380b5430 msgr2=0x7f80380b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:41.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.445+0000 7f803e77f700 1 -- 172.21.15.36:0/3011680902 shutdown_connections 2022-06-17T12:47:41.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.445+0000 7f803e77f700 1 -- 172.21.15.36:0/3011680902 wait complete. 2022-06-17T12:47:41.503 INFO:teuthology.orchestra.run.smithi036.stderr:updated 2022-06-17T12:47:41.504 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2022-06-17T12:47:41.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b2588f700 1 Processor -- start 2022-06-17T12:47:41.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b2588f700 1 -- start start 2022-06-17T12:47:41.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b2588f700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4b20060ca0 0x7f4b201442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:41.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b2588f700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b2014b870 con 0x7f4b2014b2b0 2022-06-17T12:47:41.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b2588f700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b200ac500 con 0x7f4b200b27b0 2022-06-17T12:47:41.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b2588f700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b200ace90 con 0x7f4b20060ca0 2022-06-17T12:47:41.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b1ffff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f4b2014b2b0 0x7f4b20147bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50038/0 (socket says 172.21.15.36:50038) 2022-06-17T12:47:41.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b1ffff700 1 -- 172.21.15.36:0/706210268 learned_addr learned my addr 172.21.15.36:0/706210268 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:41.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b2508e700 1 --2- 172.21.15.36:0/706210268 >> v2:172.21.15.70:3300/0 conn(0x7f4b20060ca0 0x7f4b201442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:41.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b1f7fe700 1 -- 172.21.15.36:0/706210268 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3960953233 0 0) 0x7f4b2014b870 con 0x7f4b2014b2b0 2022-06-17T12:47:41.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b1f7fe700 1 -- 172.21.15.36:0/706210268 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b04003260 con 0x7f4b2014b2b0 2022-06-17T12:47:41.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b1f7fe700 1 -- 172.21.15.36:0/706210268 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3580945140 0 0) 0x7f4b200ac500 con 0x7f4b200b27b0 2022-06-17T12:47:41.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b1f7fe700 1 -- 172.21.15.36:0/706210268 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b2014b870 con 0x7f4b200b27b0 2022-06-17T12:47:41.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b2508e700 1 -- 172.21.15.36:0/706210268 >> v1:172.21.15.70:6789/0 conn(0x7f4b200b27b0 legacy=0x7f4b200ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:41.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b2508e700 1 -- 172.21.15.36:0/706210268 >> v1:172.21.15.36:6789/0 conn(0x7f4b2014b2b0 legacy=0x7f4b20147bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:41.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b2508e700 1 -- 172.21.15.36:0/706210268 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b201482e0 con 0x7f4b20060ca0 2022-06-17T12:47:41.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b2508e700 1 --2- 172.21.15.36:0/706210268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b20060ca0 0x7f4b201442d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f4b180032f0 tx=0x7f4b180091d0).ready entity=mon.1 client_cookie=4fab7956c6a9f078 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:41.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b1f7fe700 1 -- 172.21.15.36:0/706210268 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b1800b600 con 0x7f4b20060ca0 2022-06-17T12:47:41.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b1f7fe700 1 -- 172.21.15.36:0/706210268 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1800b760 con 0x7f4b20060ca0 2022-06-17T12:47:41.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b1f7fe700 1 -- 172.21.15.36:0/706210268 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b1800b8d0 con 0x7f4b20060ca0 2022-06-17T12:47:41.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b2588f700 1 -- 172.21.15.36:0/706210268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b20060ca0 msgr2=0x7f4b201442d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:41.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b2588f700 1 --2- 172.21.15.36:0/706210268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b20060ca0 0x7f4b201442d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f4b180032f0 tx=0x7f4b180091d0).stop 2022-06-17T12:47:41.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b2588f700 1 -- 172.21.15.36:0/706210268 shutdown_connections 2022-06-17T12:47:41.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b2588f700 1 --2- 172.21.15.36:0/706210268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b20060ca0 0x7f4b201442d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:41.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b2588f700 1 -- 172.21.15.36:0/706210268 >> 172.21.15.36:0/706210268 conn(0x7f4b200b5d10 msgr2=0x7f4b200b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:41.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b2588f700 1 -- 172.21.15.36:0/706210268 shutdown_connections 2022-06-17T12:47:41.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.649+0000 7f4b2588f700 1 -- 172.21.15.36:0/706210268 wait complete. 2022-06-17T12:47:41.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.653+0000 7f4b2588f700 1 Processor -- start 2022-06-17T12:47:41.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.653+0000 7f4b2588f700 1 -- start start 2022-06-17T12:47:41.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.653+0000 7f4b2588f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b20060ca0 0x7f4b2014a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:41.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.653+0000 7f4b2588f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4b200b27b0 0x7f4b201462e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:41.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.653+0000 7f4b2588f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4b2014b2b0 0x7f4b20063510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:41.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.653+0000 7f4b2588f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b20149c10 con 0x7f4b2014b2b0 2022-06-17T12:47:41.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.653+0000 7f4b2588f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b20149e40 con 0x7f4b20060ca0 2022-06-17T12:47:41.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.653+0000 7f4b2588f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2014a0e0 con 0x7f4b200b27b0 2022-06-17T12:47:41.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.653+0000 7f4b2508e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4b2014b2b0 0x7f4b20063510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:41.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.653+0000 7f4b2508e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4b2014b2b0 0x7f4b20063510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60744/0 (socket says 172.21.15.36:60744) 2022-06-17T12:47:41.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.653+0000 7f4b2508e700 1 -- 172.21.15.36:0/1193891025 learned_addr learned my addr 172.21.15.36:0/1193891025 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:41.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.653+0000 7f4b2488d700 1 --2- 172.21.15.36:0/1193891025 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b20060ca0 0x7f4b2014a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:41.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.653+0000 7f4b1ffff700 1 --2- 172.21.15.36:0/1193891025 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4b200b27b0 0x7f4b201462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:41.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.653+0000 7f4b2508e700 1 -- 172.21.15.36:0/1193891025 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4b200b27b0 msgr2=0x7f4b201462e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:41.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.653+0000 7f4b2508e700 1 --2- 172.21.15.36:0/1193891025 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4b200b27b0 0x7f4b201462e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:41.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.653+0000 7f4b2508e700 1 -- 172.21.15.36:0/1193891025 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b20060ca0 msgr2=0x7f4b2014a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:41.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.653+0000 7f4b2508e700 1 --2- 172.21.15.36:0/1193891025 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b20060ca0 0x7f4b2014a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:41.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.653+0000 7f4b2508e700 1 -- 172.21.15.36:0/1193891025 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b18009470 con 0x7f4b2014b2b0 2022-06-17T12:47:41.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.653+0000 7f4b2508e700 1 --2- 172.21.15.36:0/1193891025 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4b2014b2b0 0x7f4b20063510 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f4b180032f0 tx=0x7f4b18016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:41.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.653+0000 7f4b1dffb700 1 -- 172.21.15.36:0/1193891025 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b18011b60 con 0x7f4b2014b2b0 2022-06-17T12:47:41.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.653+0000 7f4b1dffb700 1 -- 172.21.15.36:0/1193891025 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b18011cc0 con 0x7f4b2014b2b0 2022-06-17T12:47:41.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.653+0000 7f4b2588f700 1 -- 172.21.15.36:0/1193891025 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b20066e00 con 0x7f4b2014b2b0 2022-06-17T12:47:41.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.653+0000 7f4b1dffb700 1 -- 172.21.15.36:0/1193891025 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b1801f5e0 con 0x7f4b2014b2b0 2022-06-17T12:47:41.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.653+0000 7f4b2588f700 1 -- 172.21.15.36:0/1193891025 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b20148e30 con 0x7f4b2014b2b0 2022-06-17T12:47:41.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.657+0000 7f4b2588f700 1 -- 172.21.15.36:0/1193891025 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b20148a50 con 0x7f4b2014b2b0 2022-06-17T12:47:41.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.657+0000 7f4b1dffb700 1 -- 172.21.15.36:0/1193891025 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f4b18026030 con 0x7f4b2014b2b0 2022-06-17T12:47:41.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.657+0000 7f4b1dffb700 1 --2- 172.21.15.36:0/1193891025 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4b00053a60 0x7f4b00055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:41.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.657+0000 7f4b1dffb700 1 -- 172.21.15.36:0/1193891025 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f4b1806c690 con 0x7f4b2014b2b0 2022-06-17T12:47:41.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.657+0000 7f4b2488d700 1 --2- 172.21.15.36:0/1193891025 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4b00053a60 0x7f4b00055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:41.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.657+0000 7f4b2488d700 1 --2- 172.21.15.36:0/1193891025 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4b00053a60 0x7f4b00055f20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f4b1400d000 tx=0x7f4b14007a20).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:41.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.665+0000 7f4b1dffb700 1 -- 172.21.15.36:0/1193891025 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4b20148a50 con 0x7f4b2014b2b0 2022-06-17T12:47:41.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.833+0000 7f4b2588f700 1 -- 172.21.15.36:0/1193891025 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f4b20148a50 con 0x7f4b2014b2b0 2022-06-17T12:47:41.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.845+0000 7f4b1dffb700 1 -- 172.21.15.36:0/1193891025 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 updated v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4b20148a50 con 0x7f4b2014b2b0 2022-06-17T12:47:41.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.845+0000 7f4b2588f700 1 -- 172.21.15.36:0/1193891025 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4b00053a60 msgr2=0x7f4b00055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:41.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.845+0000 7f4b2588f700 1 --2- 172.21.15.36:0/1193891025 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4b00053a60 0x7f4b00055f20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f4b1400d000 tx=0x7f4b14007a20).stop 2022-06-17T12:47:41.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.845+0000 7f4b2588f700 1 -- 172.21.15.36:0/1193891025 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4b2014b2b0 msgr2=0x7f4b20063510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:41.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.845+0000 7f4b2588f700 1 --2- 172.21.15.36:0/1193891025 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4b2014b2b0 0x7f4b20063510 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f4b180032f0 tx=0x7f4b18016000).stop 2022-06-17T12:47:41.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.845+0000 7f4b2588f700 1 -- 172.21.15.36:0/1193891025 shutdown_connections 2022-06-17T12:47:41.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.845+0000 7f4b2588f700 1 --2- 172.21.15.36:0/1193891025 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4b00053a60 0x7f4b00055f20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:41.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.845+0000 7f4b2588f700 1 --2- 172.21.15.36:0/1193891025 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4b2014b2b0 0x7f4b20063510 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:41.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.845+0000 7f4b2588f700 1 --2- 172.21.15.36:0/1193891025 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4b200b27b0 0x7f4b201462e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:41.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.845+0000 7f4b2588f700 1 --2- 172.21.15.36:0/1193891025 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b20060ca0 0x7f4b2014a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:41.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.845+0000 7f4b2588f700 1 -- 172.21.15.36:0/1193891025 >> 172.21.15.36:0/1193891025 conn(0x7f4b200b5d10 msgr2=0x7f4b200afb90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:41.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.845+0000 7f4b2588f700 1 -- 172.21.15.36:0/1193891025 shutdown_connections 2022-06-17T12:47:41.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:41.845+0000 7f4b2588f700 1 -- 172.21.15.36:0/1193891025 wait complete. 2022-06-17T12:47:41.940 INFO:teuthology.orchestra.run.smithi036.stderr:updated 2022-06-17T12:47:41.940 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2022-06-17T12:47:42.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.105+0000 7f8c15650700 1 Processor -- start 2022-06-17T12:47:42.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.105+0000 7f8c15650700 1 -- start start 2022-06-17T12:47:42.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c15650700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8c100b27b0 0x7f8c1014b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:42.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c15650700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1014b870 con 0x7f8c10060ca0 2022-06-17T12:47:42.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c15650700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c100ac500 con 0x7f8c10147bd0 2022-06-17T12:47:42.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c15650700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c100ace90 con 0x7f8c100b27b0 2022-06-17T12:47:42.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c0f7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f8c10060ca0 0x7f8c100ac780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50048/0 (socket says 172.21.15.36:50048) 2022-06-17T12:47:42.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c0f7fe700 1 -- 172.21.15.36:0/2736603010 learned_addr learned my addr 172.21.15.36:0/2736603010 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:42.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c0ffff700 1 --2- 172.21.15.36:0/2736603010 >> v2:172.21.15.70:3300/0 conn(0x7f8c100b27b0 0x7f8c1014b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:42.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c0effd700 1 -- 172.21.15.36:0/2736603010 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967286385 0 0) 0x7f8c1014b870 con 0x7f8c10060ca0 2022-06-17T12:47:42.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c0effd700 1 -- 172.21.15.36:0/2736603010 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bf0003260 con 0x7f8c10060ca0 2022-06-17T12:47:42.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c0effd700 1 -- 172.21.15.36:0/2736603010 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3584805497 0 0) 0x7f8bf0003260 con 0x7f8c10060ca0 2022-06-17T12:47:42.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c0effd700 1 -- 172.21.15.36:0/2736603010 >> v2:172.21.15.70:3300/0 conn(0x7f8c100b27b0 msgr2=0x7f8c1014b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:42.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c0effd700 1 --2- 172.21.15.36:0/2736603010 >> v2:172.21.15.70:3300/0 conn(0x7f8c100b27b0 0x7f8c1014b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:42.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c0effd700 1 -- 172.21.15.36:0/2736603010 >> v1:172.21.15.70:6789/0 conn(0x7f8c10147bd0 legacy=0x7f8c101442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:42.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c0effd700 1 -- 172.21.15.36:0/2736603010 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c100b2b90 con 0x7f8c10060ca0 2022-06-17T12:47:42.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c0effd700 1 -- 172.21.15.36:0/2736603010 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f8c000019e0 con 0x7f8c10060ca0 2022-06-17T12:47:42.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c0effd700 1 -- 172.21.15.36:0/2736603010 >> v1:172.21.15.36:6789/0 conn(0x7f8c10060ca0 legacy=0x7f8c100ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:42.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c0effd700 1 --2- 172.21.15.36:0/2736603010 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8bf0003750 0x7f8bf0003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:42.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c0effd700 1 --2- 172.21.15.36:0/2736603010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bf0004500 0x7f8bf0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:42.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c0effd700 1 --2- 172.21.15.36:0/2736603010 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8bf0004e70 0x7f8bf000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:42.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c0effd700 1 -- 172.21.15.36:0/2736603010 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c100ace90 con 0x7f8bf0004e70 2022-06-17T12:47:42.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c0effd700 1 -- 172.21.15.36:0/2736603010 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf000bcc0 con 0x7f8bf0004500 2022-06-17T12:47:42.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c0effd700 1 -- 172.21.15.36:0/2736603010 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf000be80 con 0x7f8bf0003750 2022-06-17T12:47:42.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c14e4f700 1 --2- 172.21.15.36:0/2736603010 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8bf0004e70 0x7f8bf000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:42.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c0f7fe700 1 --2- 172.21.15.36:0/2736603010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bf0004500 0x7f8bf0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:42.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c0ffff700 1 --2- 172.21.15.36:0/2736603010 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8bf0003750 0x7f8bf0003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:42.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c14e4f700 1 -- 172.21.15.36:0/2736603010 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8bf0003750 msgr2=0x7f8bf0003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:42.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c14e4f700 1 --2- 172.21.15.36:0/2736603010 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8bf0003750 0x7f8bf0003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:42.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c14e4f700 1 -- 172.21.15.36:0/2736603010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bf0004500 msgr2=0x7f8bf0004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:42.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c14e4f700 1 --2- 172.21.15.36:0/2736603010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bf0004500 0x7f8bf0004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:42.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c14e4f700 1 -- 172.21.15.36:0/2736603010 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf0003260 con 0x7f8bf0004e70 2022-06-17T12:47:42.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c14e4f700 1 -- 172.21.15.36:0/2736603010 reap_dead start 2022-06-17T12:47:42.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c14e4f700 1 --2- 172.21.15.36:0/2736603010 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8bf0004e70 0x7f8bf000b750 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f8c08001020 tx=0x7f8c080079e0).ready entity=mon.0 client_cookie=e44c70c71bd22ad6 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:42.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c0effd700 1 -- 172.21.15.36:0/2736603010 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c08007cf0 con 0x7f8bf0004e70 2022-06-17T12:47:42.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c0effd700 1 -- 172.21.15.36:0/2736603010 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c08007030 con 0x7f8bf0004e70 2022-06-17T12:47:42.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c0effd700 1 -- 172.21.15.36:0/2736603010 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c08007390 con 0x7f8bf0004e70 2022-06-17T12:47:42.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c15650700 1 -- 172.21.15.36:0/2736603010 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8bf0004e70 msgr2=0x7f8bf000b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:42.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c15650700 1 --2- 172.21.15.36:0/2736603010 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8bf0004e70 0x7f8bf000b750 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f8c08001020 tx=0x7f8c080079e0).stop 2022-06-17T12:47:42.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c15650700 1 -- 172.21.15.36:0/2736603010 shutdown_connections 2022-06-17T12:47:42.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c15650700 1 --2- 172.21.15.36:0/2736603010 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8bf0004e70 0x7f8bf000b750 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:42.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.109+0000 7f8c15650700 1 -- 172.21.15.36:0/2736603010 >> 172.21.15.36:0/2736603010 conn(0x7f8c100b5d10 msgr2=0x7f8c100b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:42.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c15650700 1 -- 172.21.15.36:0/2736603010 shutdown_connections 2022-06-17T12:47:42.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c15650700 1 -- 172.21.15.36:0/2736603010 wait complete. 2022-06-17T12:47:42.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c15650700 1 Processor -- start 2022-06-17T12:47:42.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c15650700 1 -- start start 2022-06-17T12:47:42.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c15650700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8bf0004e70 0x7f8c100a7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:42.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c15650700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8c100650f0 0x7f8c100afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:42.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c15650700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c10062390 0x7f8c10063d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:42.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c15650700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c10066a90 con 0x7f8c100650f0 2022-06-17T12:47:42.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c15650700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c10062810 con 0x7f8c10062390 2022-06-17T12:47:42.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c15650700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c10065f60 con 0x7f8bf0004e70 2022-06-17T12:47:42.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c0f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8c100650f0 0x7f8c100afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:42.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c0f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8c100650f0 0x7f8c100afed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60764/0 (socket says 172.21.15.36:60764) 2022-06-17T12:47:42.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c0f7fe700 1 -- 172.21.15.36:0/1148677229 learned_addr learned my addr 172.21.15.36:0/1148677229 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:42.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c0ffff700 1 --2- 172.21.15.36:0/1148677229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8bf0004e70 0x7f8c100a7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:42.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c14e4f700 1 --2- 172.21.15.36:0/1148677229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c10062390 0x7f8c10063d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:42.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c0f7fe700 1 -- 172.21.15.36:0/1148677229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8bf0004e70 msgr2=0x7f8c100a7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:42.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c0f7fe700 1 --2- 172.21.15.36:0/1148677229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8bf0004e70 0x7f8c100a7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:42.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c0f7fe700 1 -- 172.21.15.36:0/1148677229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c10062390 msgr2=0x7f8c10063d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:42.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c0f7fe700 1 --2- 172.21.15.36:0/1148677229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c10062390 0x7f8c10063d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:42.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c0f7fe700 1 -- 172.21.15.36:0/1148677229 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c10066200 con 0x7f8c100650f0 2022-06-17T12:47:42.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c0f7fe700 1 --2- 172.21.15.36:0/1148677229 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8c100650f0 0x7f8c100afed0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f8c10063af0 tx=0x7f8c100acfd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:42.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c0d7fa700 1 -- 172.21.15.36:0/1148677229 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c00011780 con 0x7f8c100650f0 2022-06-17T12:47:42.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c0d7fa700 1 -- 172.21.15.36:0/1148677229 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c000118e0 con 0x7f8c100650f0 2022-06-17T12:47:42.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c0d7fa700 1 -- 172.21.15.36:0/1148677229 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c00011c40 con 0x7f8c100650f0 2022-06-17T12:47:42.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c15650700 1 -- 172.21.15.36:0/1148677229 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c10065b30 con 0x7f8c100650f0 2022-06-17T12:47:42.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.113+0000 7f8c15650700 1 -- 172.21.15.36:0/1148677229 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c10159740 con 0x7f8c100650f0 2022-06-17T12:47:42.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.117+0000 7f8c15650700 1 -- 172.21.15.36:0/1148677229 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c10066390 con 0x7f8c100650f0 2022-06-17T12:47:42.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.117+0000 7f8c0d7fa700 1 -- 172.21.15.36:0/1148677229 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8c0001c360 con 0x7f8c100650f0 2022-06-17T12:47:42.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.117+0000 7f8c0d7fa700 1 --2- 172.21.15.36:0/1148677229 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8bf4053940 0x7f8bf4055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:42.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.117+0000 7f8c0ffff700 1 --2- 172.21.15.36:0/1148677229 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8bf4053940 0x7f8bf4055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:42.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.117+0000 7f8c0d7fa700 1 -- 172.21.15.36:0/1148677229 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f8c0006b870 con 0x7f8c100650f0 2022-06-17T12:47:42.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.117+0000 7f8c0ffff700 1 --2- 172.21.15.36:0/1148677229 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8bf4053940 0x7f8bf4055e00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f8bf800b000 tx=0x7f8bf8006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:42.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.121+0000 7f8c0d7fa700 1 -- 172.21.15.36:0/1148677229 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8c00042840 con 0x7f8c100650f0 2022-06-17T12:47:42.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.289+0000 7f8c15650700 1 -- 172.21.15.36:0/1148677229 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f8c10057410 con 0x7f8c100650f0 2022-06-17T12:47:42.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.297+0000 7f8c0d7fa700 1 -- 172.21.15.36:0/1148677229 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 updated v10) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8c00019030 con 0x7f8c100650f0 2022-06-17T12:47:42.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.301+0000 7f8c15650700 1 -- 172.21.15.36:0/1148677229 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8bf4053940 msgr2=0x7f8bf4055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:42.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.301+0000 7f8c15650700 1 --2- 172.21.15.36:0/1148677229 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8bf4053940 0x7f8bf4055e00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f8bf800b000 tx=0x7f8bf8006df0).stop 2022-06-17T12:47:42.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.301+0000 7f8c15650700 1 -- 172.21.15.36:0/1148677229 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8c100650f0 msgr2=0x7f8c100afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:42.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.301+0000 7f8c15650700 1 --2- 172.21.15.36:0/1148677229 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8c100650f0 0x7f8c100afed0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f8c10063af0 tx=0x7f8c100acfd0).stop 2022-06-17T12:47:42.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.301+0000 7f8c15650700 1 -- 172.21.15.36:0/1148677229 shutdown_connections 2022-06-17T12:47:42.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.301+0000 7f8c15650700 1 --2- 172.21.15.36:0/1148677229 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8bf4053940 0x7f8bf4055e00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:42.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.301+0000 7f8c15650700 1 --2- 172.21.15.36:0/1148677229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c10062390 0x7f8c10063d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:42.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.301+0000 7f8c15650700 1 --2- 172.21.15.36:0/1148677229 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8c100650f0 0x7f8c100afed0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:42.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.301+0000 7f8c15650700 1 --2- 172.21.15.36:0/1148677229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8bf0004e70 0x7f8c100a7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:42.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.301+0000 7f8c15650700 1 -- 172.21.15.36:0/1148677229 >> 172.21.15.36:0/1148677229 conn(0x7f8c100b5d10 msgr2=0x7f8c100b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:42.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.301+0000 7f8c15650700 1 -- 172.21.15.36:0/1148677229 shutdown_connections 2022-06-17T12:47:42.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.301+0000 7f8c15650700 1 -- 172.21.15.36:0/1148677229 wait complete. 2022-06-17T12:47:42.519 INFO:teuthology.orchestra.run.smithi036.stderr:updated 2022-06-17T12:47:42.520 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2022-06-17T12:47:42.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.653+0000 7fd99a8bb700 1 Processor -- start 2022-06-17T12:47:42.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.653+0000 7fd99a8bb700 1 -- start start 2022-06-17T12:47:42.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.653+0000 7fd99a8bb700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd9940ad140 0x7fd9940af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:42.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.653+0000 7fd99a8bb700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd99414b3b0 0x7fd9940ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:42.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.653+0000 7fd99a8bb700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9940ac870 con 0x7fd9940ac340 2022-06-17T12:47:42.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.653+0000 7fd99a8bb700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd994148960 con 0x7fd9940ad140 2022-06-17T12:47:42.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.653+0000 7fd99a8bb700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9940acaa0 con 0x7fd99414b3b0 2022-06-17T12:47:42.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.657+0000 7fd993fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd9940ad140 0x7fd9940af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:42.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.657+0000 7fd993fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd9940ad140 0x7fd9940af7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60770/0 (socket says 172.21.15.36:60770) 2022-06-17T12:47:42.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.657+0000 7fd993fff700 1 -- 172.21.15.36:0/138305949 learned_addr learned my addr 172.21.15.36:0/138305949 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:42.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.657+0000 7fd998e58700 1 --2- 172.21.15.36:0/138305949 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd99414b3b0 0x7fd9940ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:42.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.657+0000 7fd998e58700 1 -- 172.21.15.36:0/138305949 >> v2:172.21.15.36:3300/0 conn(0x7fd9940ad140 msgr2=0x7fd9940af7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:42.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.657+0000 7fd998e58700 1 --2- 172.21.15.36:0/138305949 >> v2:172.21.15.36:3300/0 conn(0x7fd9940ad140 0x7fd9940af7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:42.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.657+0000 7fd998e58700 1 -- 172.21.15.36:0/138305949 >> v1:172.21.15.70:6789/0 conn(0x7fd9940ac340 legacy=0x7fd99414aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:42.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.657+0000 7fd998e58700 1 -- 172.21.15.36:0/138305949 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9940b0bf0 con 0x7fd99414b3b0 2022-06-17T12:47:42.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.657+0000 7fd993fff700 1 --2- 172.21.15.36:0/138305949 >> v2:172.21.15.36:3300/0 conn(0x7fd9940ad140 0x7fd9940af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:42.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.657+0000 7fd998e58700 1 --2- 172.21.15.36:0/138305949 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd99414b3b0 0x7fd9940ad520 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fd98c002f70 tx=0x7fd98c006df0).ready entity=mon.2 client_cookie=efb50085699a3f1b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:42.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.657+0000 7fd992ffd700 1 -- 172.21.15.36:0/138305949 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd98c00d570 con 0x7fd99414b3b0 2022-06-17T12:47:42.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.657+0000 7fd992ffd700 1 -- 172.21.15.36:0/138305949 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98c000e90 con 0x7fd99414b3b0 2022-06-17T12:47:42.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.657+0000 7fd992ffd700 1 -- 172.21.15.36:0/138305949 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd98c00d8c0 con 0x7fd99414b3b0 2022-06-17T12:47:42.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.657+0000 7fd99a8bb700 1 -- 172.21.15.36:0/138305949 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd99414b3b0 msgr2=0x7fd9940ad520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:42.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.657+0000 7fd99a8bb700 1 --2- 172.21.15.36:0/138305949 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd99414b3b0 0x7fd9940ad520 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fd98c002f70 tx=0x7fd98c006df0).stop 2022-06-17T12:47:42.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.657+0000 7fd99a8bb700 1 -- 172.21.15.36:0/138305949 shutdown_connections 2022-06-17T12:47:42.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.657+0000 7fd99a8bb700 1 --2- 172.21.15.36:0/138305949 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd99414b3b0 0x7fd9940ad520 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:42.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.657+0000 7fd99a8bb700 1 --2- 172.21.15.36:0/138305949 >> v2:172.21.15.36:3300/0 conn(0x7fd9940ad140 0x7fd9940af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:42.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.657+0000 7fd99a8bb700 1 -- 172.21.15.36:0/138305949 >> 172.21.15.36:0/138305949 conn(0x7fd9940b5430 msgr2=0x7fd9940b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:42.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.657+0000 7fd99a8bb700 1 -- 172.21.15.36:0/138305949 shutdown_connections 2022-06-17T12:47:42.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.657+0000 7fd99a8bb700 1 -- 172.21.15.36:0/138305949 wait complete. 2022-06-17T12:47:42.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd99a8bb700 1 Processor -- start 2022-06-17T12:47:42.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd99a8bb700 1 -- start start 2022-06-17T12:47:42.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd99a8bb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd9940ac340 0x7fd99414ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:42.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd99a8bb700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd9940ad140 0x7fd994149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:42.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd99a8bb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd99414b3b0 0x7fd9940aab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:42.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd99a8bb700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9940a70e0 con 0x7fd9940ac340 2022-06-17T12:47:42.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd99a8bb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9940abb20 con 0x7fd99414b3b0 2022-06-17T12:47:42.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd99a8bb700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9940abdc0 con 0x7fd9940ad140 2022-06-17T12:47:42.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd993fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd9940ac340 0x7fd99414ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:42.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd993fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd9940ac340 0x7fd99414ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60776/0 (socket says 172.21.15.36:60776) 2022-06-17T12:47:42.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd993fff700 1 -- 172.21.15.36:0/810907485 learned_addr learned my addr 172.21.15.36:0/810907485 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:42.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd9937fe700 1 --2- 172.21.15.36:0/810907485 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd9940ad140 0x7fd994149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:42.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd998e58700 1 --2- 172.21.15.36:0/810907485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd99414b3b0 0x7fd9940aab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:42.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd993fff700 1 -- 172.21.15.36:0/810907485 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd9940ad140 msgr2=0x7fd994149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:42.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd993fff700 1 --2- 172.21.15.36:0/810907485 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd9940ad140 0x7fd994149e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:42.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd993fff700 1 -- 172.21.15.36:0/810907485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd99414b3b0 msgr2=0x7fd9940aab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:42.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd993fff700 1 --2- 172.21.15.36:0/810907485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd99414b3b0 0x7fd9940aab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:42.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd993fff700 1 -- 172.21.15.36:0/810907485 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd984009530 con 0x7fd9940ac340 2022-06-17T12:47:42.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd9937fe700 1 --2- 172.21.15.36:0/810907485 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd9940ad140 0x7fd994149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:42.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd993fff700 1 --2- 172.21.15.36:0/810907485 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd9940ac340 0x7fd99414ae70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fd984006b60 tx=0x7fd984009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:42.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd9917fa700 1 -- 172.21.15.36:0/810907485 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd98400c850 con 0x7fd9940ac340 2022-06-17T12:47:42.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd99a8bb700 1 -- 172.21.15.36:0/810907485 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd98c00b180 con 0x7fd9940ac340 2022-06-17T12:47:42.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd9917fa700 1 -- 172.21.15.36:0/810907485 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98400c9b0 con 0x7fd9940ac340 2022-06-17T12:47:42.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd9917fa700 1 -- 172.21.15.36:0/810907485 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd98400cd10 con 0x7fd9940ac340 2022-06-17T12:47:42.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.661+0000 7fd99a8bb700 1 -- 172.21.15.36:0/810907485 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9940ab790 con 0x7fd9940ac340 2022-06-17T12:47:42.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.665+0000 7fd99a8bb700 1 -- 172.21.15.36:0/810907485 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9940b1500 con 0x7fd9940ac340 2022-06-17T12:47:42.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.665+0000 7fd9917fa700 1 -- 172.21.15.36:0/810907485 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd984022070 con 0x7fd9940ac340 2022-06-17T12:47:42.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.665+0000 7fd9917fa700 1 --2- 172.21.15.36:0/810907485 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd96c053af0 0x7fd96c055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:42.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.665+0000 7fd9917fa700 1 -- 172.21.15.36:0/810907485 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fd98406c5a0 con 0x7fd9940ac340 2022-06-17T12:47:42.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.665+0000 7fd9937fe700 1 --2- 172.21.15.36:0/810907485 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd96c053af0 0x7fd96c055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:42.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.665+0000 7fd9937fe700 1 --2- 172.21.15.36:0/810907485 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd96c053af0 0x7fd96c055fb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fd97c00d000 tx=0x7fd97c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:42.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.673+0000 7fd9917fa700 1 -- 172.21.15.36:0/810907485 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd984043640 con 0x7fd9940ac340 2022-06-17T12:47:42.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.849+0000 7fd99a8bb700 1 -- 172.21.15.36:0/810907485 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fd9940687c0 con 0x7fd9940ac340 2022-06-17T12:47:42.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.857+0000 7fd9917fa700 1 -- 172.21.15.36:0/810907485 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 updated v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd984046c60 con 0x7fd9940ac340 2022-06-17T12:47:42.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.861+0000 7fd99a8bb700 1 -- 172.21.15.36:0/810907485 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd96c053af0 msgr2=0x7fd96c055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:42.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.861+0000 7fd99a8bb700 1 --2- 172.21.15.36:0/810907485 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd96c053af0 0x7fd96c055fb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fd97c00d000 tx=0x7fd97c009040).stop 2022-06-17T12:47:42.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.861+0000 7fd99a8bb700 1 -- 172.21.15.36:0/810907485 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd9940ac340 msgr2=0x7fd99414ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:42.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.861+0000 7fd99a8bb700 1 --2- 172.21.15.36:0/810907485 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd9940ac340 0x7fd99414ae70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fd984006b60 tx=0x7fd984009290).stop 2022-06-17T12:47:42.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.861+0000 7fd99a8bb700 1 -- 172.21.15.36:0/810907485 shutdown_connections 2022-06-17T12:47:42.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.861+0000 7fd99a8bb700 1 --2- 172.21.15.36:0/810907485 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd96c053af0 0x7fd96c055fb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:42.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.861+0000 7fd99a8bb700 1 --2- 172.21.15.36:0/810907485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd99414b3b0 0x7fd9940aab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:42.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.861+0000 7fd99a8bb700 1 --2- 172.21.15.36:0/810907485 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd9940ad140 0x7fd994149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:42.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.861+0000 7fd99a8bb700 1 --2- 172.21.15.36:0/810907485 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd9940ac340 0x7fd99414ae70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:42.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.861+0000 7fd99a8bb700 1 -- 172.21.15.36:0/810907485 >> 172.21.15.36:0/810907485 conn(0x7fd9940b5430 msgr2=0x7fd994147d70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:42.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.861+0000 7fd99a8bb700 1 -- 172.21.15.36:0/810907485 shutdown_connections 2022-06-17T12:47:42.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:42.861+0000 7fd99a8bb700 1 -- 172.21.15.36:0/810907485 wait complete. 2022-06-17T12:47:42.867 INFO:teuthology.orchestra.run.smithi036.stderr:updated 2022-06-17T12:47:42.870 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2022-06-17T12:47:43.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.037+0000 7f269b389700 1 Processor -- start 2022-06-17T12:47:43.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.037+0000 7f269b389700 1 -- start start 2022-06-17T12:47:43.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f269b389700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f26940ab470 0x7f26940b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:43.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f269b389700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26940b3210 0x7f269414ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:43.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f269b389700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26940b3d10 con 0x7f26940b64f0 2022-06-17T12:47:43.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f269b389700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f269414bb80 con 0x7f26940ab470 2022-06-17T12:47:43.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f269b389700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26940a9d70 con 0x7f26940b3210 2022-06-17T12:47:43.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f2699125700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f26940b64f0 0x7f2694146bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50082/0 (socket says 172.21.15.36:50082) 2022-06-17T12:47:43.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f2699125700 1 -- 172.21.15.36:0/1797193474 learned_addr learned my addr 172.21.15.36:0/1797193474 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:43.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f2698924700 1 --2- 172.21.15.36:0/1797193474 >> v2:172.21.15.70:3300/0 conn(0x7f26940ab470 0x7f26940b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:43.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f2699926700 1 --2- 172.21.15.36:0/1797193474 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26940b3210 0x7f269414ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:43.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f2693fff700 1 -- 172.21.15.36:0/1797193474 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3002065009 0 0) 0x7f26940b3d10 con 0x7f26940b64f0 2022-06-17T12:47:43.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f2693fff700 1 -- 172.21.15.36:0/1797193474 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2674003260 con 0x7f26940b64f0 2022-06-17T12:47:43.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f2698924700 1 -- 172.21.15.36:0/1797193474 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26940b3210 msgr2=0x7f269414ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:43.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f2698924700 1 --2- 172.21.15.36:0/1797193474 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26940b3210 0x7f269414ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:43.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f2698924700 1 -- 172.21.15.36:0/1797193474 >> v1:172.21.15.36:6789/0 conn(0x7f26940b64f0 legacy=0x7f2694146bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:43.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f2698924700 1 -- 172.21.15.36:0/1797193474 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26940aa010 con 0x7f26940ab470 2022-06-17T12:47:43.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f2699926700 1 --2- 172.21.15.36:0/1797193474 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26940b3210 0x7f269414ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:43.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f2698924700 1 --2- 172.21.15.36:0/1797193474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26940ab470 0x7f26940b68d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f267c0032f0 tx=0x7f267c0091d0).ready entity=mon.1 client_cookie=78e80add30a440cb server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:43.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f2693fff700 1 -- 172.21.15.36:0/1797193474 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f267c00b600 con 0x7f26940ab470 2022-06-17T12:47:43.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f2693fff700 1 -- 172.21.15.36:0/1797193474 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267c00b760 con 0x7f26940ab470 2022-06-17T12:47:43.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f2693fff700 1 -- 172.21.15.36:0/1797193474 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f267c00b8d0 con 0x7f26940ab470 2022-06-17T12:47:43.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f269b389700 1 -- 172.21.15.36:0/1797193474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26940ab470 msgr2=0x7f26940b68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:43.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f269b389700 1 --2- 172.21.15.36:0/1797193474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26940ab470 0x7f26940b68d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f267c0032f0 tx=0x7f267c0091d0).stop 2022-06-17T12:47:43.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f269b389700 1 -- 172.21.15.36:0/1797193474 shutdown_connections 2022-06-17T12:47:43.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f269b389700 1 --2- 172.21.15.36:0/1797193474 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26940b3210 0x7f269414ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:43.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f269b389700 1 --2- 172.21.15.36:0/1797193474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26940ab470 0x7f26940b68d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:43.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f269b389700 1 -- 172.21.15.36:0/1797193474 >> 172.21.15.36:0/1797193474 conn(0x7f26940636d0 msgr2=0x7f2694063ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:43.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f269b389700 1 -- 172.21.15.36:0/1797193474 shutdown_connections 2022-06-17T12:47:43.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.041+0000 7f269b389700 1 -- 172.21.15.36:0/1797193474 wait complete. 2022-06-17T12:47:43.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.045+0000 7f269b389700 1 Processor -- start 2022-06-17T12:47:43.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.045+0000 7f269b389700 1 -- start start 2022-06-17T12:47:43.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.045+0000 7f269b389700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26940ab470 0x7f26940ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:43.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.045+0000 7f269b389700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26940b3210 0x7f26940655d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:43.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.045+0000 7f269b389700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26940b64f0 0x7f2694156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:43.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.045+0000 7f269b389700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2694157ae0 con 0x7f26940b3210 2022-06-17T12:47:43.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.045+0000 7f269b389700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26940b6030 con 0x7f26940b64f0 2022-06-17T12:47:43.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.045+0000 7f269b389700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2694157840 con 0x7f26940ab470 2022-06-17T12:47:43.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.045+0000 7f2698924700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26940b3210 0x7f26940655d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:43.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.045+0000 7f2698924700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26940b3210 0x7f26940655d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60792/0 (socket says 172.21.15.36:60792) 2022-06-17T12:47:43.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.045+0000 7f2698924700 1 -- 172.21.15.36:0/3558214046 learned_addr learned my addr 172.21.15.36:0/3558214046 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:43.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.045+0000 7f2699125700 1 --2- 172.21.15.36:0/3558214046 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26940ab470 0x7f26940ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:43.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.045+0000 7f2699926700 1 --2- 172.21.15.36:0/3558214046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26940b64f0 0x7f2694156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:43.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.045+0000 7f2698924700 1 -- 172.21.15.36:0/3558214046 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26940ab470 msgr2=0x7f26940ae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:43.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.045+0000 7f2698924700 1 --2- 172.21.15.36:0/3558214046 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26940ab470 0x7f26940ae5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:43.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.045+0000 7f2698924700 1 -- 172.21.15.36:0/3558214046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26940b64f0 msgr2=0x7f2694156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:43.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.045+0000 7f2698924700 1 --2- 172.21.15.36:0/3558214046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26940b64f0 0x7f2694156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:43.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.045+0000 7f2698924700 1 -- 172.21.15.36:0/3558214046 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c00b180 con 0x7f26940b3210 2022-06-17T12:47:43.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.045+0000 7f2698924700 1 --2- 172.21.15.36:0/3558214046 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26940b3210 0x7f26940655d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f267c0032f0 tx=0x7f267c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:43.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.045+0000 7f26927fc700 1 -- 172.21.15.36:0/3558214046 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f267c00b8d0 con 0x7f26940b3210 2022-06-17T12:47:43.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.045+0000 7f26927fc700 1 -- 172.21.15.36:0/3558214046 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267c021670 con 0x7f26940b3210 2022-06-17T12:47:43.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.045+0000 7f269b389700 1 -- 172.21.15.36:0/3558214046 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267c009470 con 0x7f26940b3210 2022-06-17T12:47:43.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.049+0000 7f26927fc700 1 -- 172.21.15.36:0/3558214046 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f267c00b600 con 0x7f26940b3210 2022-06-17T12:47:43.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.049+0000 7f269b389700 1 -- 172.21.15.36:0/3558214046 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26940adfb0 con 0x7f26940b3210 2022-06-17T12:47:43.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.049+0000 7f269b389700 1 -- 172.21.15.36:0/3558214046 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2694065e00 con 0x7f26940b3210 2022-06-17T12:47:43.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.049+0000 7f26927fc700 1 -- 172.21.15.36:0/3558214046 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f267c00f070 con 0x7f26940b3210 2022-06-17T12:47:43.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.049+0000 7f26927fc700 1 --2- 172.21.15.36:0/3558214046 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2678053920 0x7f2678055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:43.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.049+0000 7f26927fc700 1 -- 172.21.15.36:0/3558214046 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f267c06d160 con 0x7f26940b3210 2022-06-17T12:47:43.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.049+0000 7f2699125700 1 --2- 172.21.15.36:0/3558214046 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2678053920 0x7f2678055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:43.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.049+0000 7f2699125700 1 --2- 172.21.15.36:0/3558214046 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2678053920 0x7f2678055de0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f268400d000 tx=0x7f26840079c0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:43.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.053+0000 7f26927fc700 1 -- 172.21.15.36:0/3558214046 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f267c044190 con 0x7f26940b3210 2022-06-17T12:47:43.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.221+0000 7f269b389700 1 -- 172.21.15.36:0/3558214046 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f2694057410 con 0x7f26940b3210 2022-06-17T12:47:43.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.229+0000 7f26927fc700 1 -- 172.21.15.36:0/3558214046 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 updated v12) v1 ==== 97+0+0 (secure 0 0 0) 0x7f267c0150a0 con 0x7f26940b3210 2022-06-17T12:47:43.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.233+0000 7f269b389700 1 -- 172.21.15.36:0/3558214046 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2678053920 msgr2=0x7f2678055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:43.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.233+0000 7f269b389700 1 --2- 172.21.15.36:0/3558214046 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2678053920 0x7f2678055de0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f268400d000 tx=0x7f26840079c0).stop 2022-06-17T12:47:43.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.233+0000 7f269b389700 1 -- 172.21.15.36:0/3558214046 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26940b3210 msgr2=0x7f26940655d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:43.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.233+0000 7f269b389700 1 --2- 172.21.15.36:0/3558214046 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26940b3210 0x7f26940655d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f267c0032f0 tx=0x7f267c016000).stop 2022-06-17T12:47:43.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.233+0000 7f269b389700 1 -- 172.21.15.36:0/3558214046 shutdown_connections 2022-06-17T12:47:43.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.233+0000 7f269b389700 1 --2- 172.21.15.36:0/3558214046 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2678053920 0x7f2678055de0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:43.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.233+0000 7f269b389700 1 --2- 172.21.15.36:0/3558214046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26940b64f0 0x7f2694156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:43.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.233+0000 7f269b389700 1 --2- 172.21.15.36:0/3558214046 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26940b3210 0x7f26940655d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:43.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.233+0000 7f269b389700 1 --2- 172.21.15.36:0/3558214046 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26940ab470 0x7f26940ae5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:43.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.233+0000 7f269b389700 1 -- 172.21.15.36:0/3558214046 >> 172.21.15.36:0/3558214046 conn(0x7f26940636d0 msgr2=0x7f2694144820 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:43.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.233+0000 7f269b389700 1 -- 172.21.15.36:0/3558214046 shutdown_connections 2022-06-17T12:47:43.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.233+0000 7f269b389700 1 -- 172.21.15.36:0/3558214046 wait complete. 2022-06-17T12:47:43.392 INFO:teuthology.orchestra.run.smithi036.stderr:updated 2022-06-17T12:47:43.394 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:47:43.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.529+0000 7fd2e6b41700 1 Processor -- start 2022-06-17T12:47:43.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.529+0000 7fd2e6b41700 1 -- start start 2022-06-17T12:47:43.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.529+0000 7fd2e6b41700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2e0144c00 0x7fd2e0062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:43.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.529+0000 7fd2e6b41700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd2e00ad330 0x7fd2e00ad780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:43.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.529+0000 7fd2e6b41700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e00b0ba0 con 0x7fd2e0062770 2022-06-17T12:47:43.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.529+0000 7fd2e6b41700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2e00ae820 con 0x7fd2e00ad330 2022-06-17T12:47:43.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.529+0000 7fd2e6b41700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e0061030 con 0x7fd2e0144c00 2022-06-17T12:47:43.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.529+0000 7fd2e48dd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fd2e0062770 0x7fd2e01444f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50096/0 (socket says 172.21.15.36:50096) 2022-06-17T12:47:43.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.529+0000 7fd2e48dd700 1 -- 172.21.15.36:0/3763723198 learned_addr learned my addr 172.21.15.36:0/3763723198 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:43.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.529+0000 7fd2dffff700 1 --2- 172.21.15.36:0/3763723198 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2e0144c00 0x7fd2e0062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:43.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.529+0000 7fd2e50de700 1 --2- 172.21.15.36:0/3763723198 >> v2:172.21.15.70:3300/0 conn(0x7fd2e00ad330 0x7fd2e00ad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:43.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.529+0000 7fd2df7fe700 1 -- 172.21.15.36:0/3763723198 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4084815475 0 0) 0x7fd2e00b0ba0 con 0x7fd2e0062770 2022-06-17T12:47:43.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.529+0000 7fd2df7fe700 1 -- 172.21.15.36:0/3763723198 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2c0003260 con 0x7fd2e0062770 2022-06-17T12:47:43.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.529+0000 7fd2dffff700 1 -- 172.21.15.36:0/3763723198 >> v2:172.21.15.70:3300/0 conn(0x7fd2e00ad330 msgr2=0x7fd2e00ad780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:43.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.529+0000 7fd2dffff700 1 --2- 172.21.15.36:0/3763723198 >> v2:172.21.15.70:3300/0 conn(0x7fd2e00ad330 0x7fd2e00ad780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:43.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.529+0000 7fd2dffff700 1 -- 172.21.15.36:0/3763723198 >> v1:172.21.15.36:6789/0 conn(0x7fd2e0062770 legacy=0x7fd2e01444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:43.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.529+0000 7fd2dffff700 1 -- 172.21.15.36:0/3763723198 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e0147e90 con 0x7fd2e0144c00 2022-06-17T12:47:43.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.529+0000 7fd2e50de700 1 --2- 172.21.15.36:0/3763723198 >> v2:172.21.15.70:3300/0 conn(0x7fd2e00ad330 0x7fd2e00ad780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:43.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.529+0000 7fd2dffff700 1 --2- 172.21.15.36:0/3763723198 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2e0144c00 0x7fd2e0062b50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fd2d00032f0 tx=0x7fd2d00091d0).ready entity=mon.2 client_cookie=a3886dab1590e95d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:43.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.529+0000 7fd2df7fe700 1 -- 172.21.15.36:0/3763723198 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2d000b6c0 con 0x7fd2e0144c00 2022-06-17T12:47:43.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.529+0000 7fd2df7fe700 1 -- 172.21.15.36:0/3763723198 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d000b820 con 0x7fd2e0144c00 2022-06-17T12:47:43.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.529+0000 7fd2df7fe700 1 -- 172.21.15.36:0/3763723198 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2d000b990 con 0x7fd2e0144c00 2022-06-17T12:47:43.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.533+0000 7fd2e6b41700 1 -- 172.21.15.36:0/3763723198 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2e0144c00 msgr2=0x7fd2e0062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:43.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.533+0000 7fd2e6b41700 1 --2- 172.21.15.36:0/3763723198 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2e0144c00 0x7fd2e0062b50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fd2d00032f0 tx=0x7fd2d00091d0).stop 2022-06-17T12:47:43.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.533+0000 7fd2e6b41700 1 -- 172.21.15.36:0/3763723198 shutdown_connections 2022-06-17T12:47:43.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.533+0000 7fd2e6b41700 1 --2- 172.21.15.36:0/3763723198 >> v2:172.21.15.70:3300/0 conn(0x7fd2e00ad330 0x7fd2e00ad780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:43.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.533+0000 7fd2e6b41700 1 --2- 172.21.15.36:0/3763723198 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2e0144c00 0x7fd2e0062b50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:43.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.533+0000 7fd2e6b41700 1 -- 172.21.15.36:0/3763723198 >> 172.21.15.36:0/3763723198 conn(0x7fd2e00b53e0 msgr2=0x7fd2e00b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:43.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.533+0000 7fd2e6b41700 1 -- 172.21.15.36:0/3763723198 shutdown_connections 2022-06-17T12:47:43.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.533+0000 7fd2e6b41700 1 -- 172.21.15.36:0/3763723198 wait complete. 2022-06-17T12:47:43.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.533+0000 7fd2e6b41700 1 Processor -- start 2022-06-17T12:47:43.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.533+0000 7fd2e6b41700 1 -- start start 2022-06-17T12:47:43.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.537+0000 7fd2e6b41700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2e0062770 0x7fd2e01314d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:43.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.537+0000 7fd2e6b41700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2e00ad330 0x7fd2e0143f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:43.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.537+0000 7fd2e6b41700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd2e0144c00 0x7fd2e0144460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:43.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.537+0000 7fd2e6b41700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e0065780 con 0x7fd2e0144c00 2022-06-17T12:47:43.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.537+0000 7fd2e6b41700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e0065c50 con 0x7fd2e00ad330 2022-06-17T12:47:43.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.537+0000 7fd2e6b41700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e00659b0 con 0x7fd2e0062770 2022-06-17T12:47:43.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.537+0000 7fd2e50de700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd2e0144c00 0x7fd2e0144460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:43.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.537+0000 7fd2e50de700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd2e0144c00 0x7fd2e0144460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60806/0 (socket says 172.21.15.36:60806) 2022-06-17T12:47:43.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.537+0000 7fd2e50de700 1 -- 172.21.15.36:0/3458408538 learned_addr learned my addr 172.21.15.36:0/3458408538 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:43.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.537+0000 7fd2dffff700 1 --2- 172.21.15.36:0/3458408538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2e00ad330 0x7fd2e0143f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:43.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.537+0000 7fd2e48dd700 1 --2- 172.21.15.36:0/3458408538 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2e0062770 0x7fd2e01314d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:43.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.537+0000 7fd2e50de700 1 -- 172.21.15.36:0/3458408538 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2e0062770 msgr2=0x7fd2e01314d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:43.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.537+0000 7fd2e50de700 1 --2- 172.21.15.36:0/3458408538 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2e0062770 0x7fd2e01314d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:43.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.537+0000 7fd2e50de700 1 -- 172.21.15.36:0/3458408538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2e00ad330 msgr2=0x7fd2e0143f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:43.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.537+0000 7fd2e50de700 1 --2- 172.21.15.36:0/3458408538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2e00ad330 0x7fd2e0143f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:43.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.537+0000 7fd2e50de700 1 -- 172.21.15.36:0/3458408538 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d0009470 con 0x7fd2e0144c00 2022-06-17T12:47:43.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.537+0000 7fd2e50de700 1 --2- 172.21.15.36:0/3458408538 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd2e0144c00 0x7fd2e0144460 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7fd2d8006940 tx=0x7fd2d8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:43.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.537+0000 7fd2ddffb700 1 -- 172.21.15.36:0/3458408538 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2d8011780 con 0x7fd2e0144c00 2022-06-17T12:47:43.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.537+0000 7fd2ddffb700 1 -- 172.21.15.36:0/3458408538 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d80118e0 con 0x7fd2e0144c00 2022-06-17T12:47:43.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.537+0000 7fd2e6b41700 1 -- 172.21.15.36:0/3458408538 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e0063700 con 0x7fd2e0144c00 2022-06-17T12:47:43.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.537+0000 7fd2ddffb700 1 -- 172.21.15.36:0/3458408538 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2d8011c40 con 0x7fd2e0144c00 2022-06-17T12:47:43.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.537+0000 7fd2e6b41700 1 -- 172.21.15.36:0/3458408538 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e0144950 con 0x7fd2e0144c00 2022-06-17T12:47:43.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.537+0000 7fd2e6b41700 1 -- 172.21.15.36:0/3458408538 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e0063d40 con 0x7fd2e0144c00 2022-06-17T12:47:43.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.541+0000 7fd2ddffb700 1 -- 172.21.15.36:0/3458408538 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd2d801c460 con 0x7fd2e0144c00 2022-06-17T12:47:43.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.541+0000 7fd2ddffb700 1 --2- 172.21.15.36:0/3458408538 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd2c40539b0 0x7fd2c4055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:43.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.541+0000 7fd2ddffb700 1 -- 172.21.15.36:0/3458408538 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fd2d806bf30 con 0x7fd2e0144c00 2022-06-17T12:47:43.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.541+0000 7fd2e48dd700 1 --2- 172.21.15.36:0/3458408538 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd2c40539b0 0x7fd2c4055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:43.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.541+0000 7fd2e48dd700 1 --2- 172.21.15.36:0/3458408538 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd2c40539b0 0x7fd2c4055e70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fd2c800d000 tx=0x7fd2c8009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:43.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.545+0000 7fd2ddffb700 1 -- 172.21.15.36:0/3458408538 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd2d8042f00 con 0x7fd2e0144c00 2022-06-17T12:47:43.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.673+0000 7fd2e6b41700 1 -- 172.21.15.36:0/3458408538 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2e0063890 con 0x7fd2e0144c00 2022-06-17T12:47:43.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.673+0000 7fd2ddffb700 1 -- 172.21.15.36:0/3458408538 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+13414 (secure 0 0 0) 0x7fd2d8046520 con 0x7fd2e0144c00 2022-06-17T12:47:43.748 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:43.749 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":31,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:47:39.895127+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:47:43.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.677+0000 7fd2e6b41700 1 -- 172.21.15.36:0/3458408538 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd2c40539b0 msgr2=0x7fd2c4055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:43.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.677+0000 7fd2e6b41700 1 --2- 172.21.15.36:0/3458408538 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd2c40539b0 0x7fd2c4055e70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fd2c800d000 tx=0x7fd2c8009040).stop 2022-06-17T12:47:43.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.677+0000 7fd2e6b41700 1 -- 172.21.15.36:0/3458408538 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd2e0144c00 msgr2=0x7fd2e0144460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:43.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.677+0000 7fd2e6b41700 1 --2- 172.21.15.36:0/3458408538 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd2e0144c00 0x7fd2e0144460 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7fd2d8006940 tx=0x7fd2d8008040).stop 2022-06-17T12:47:43.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.677+0000 7fd2e6b41700 1 -- 172.21.15.36:0/3458408538 shutdown_connections 2022-06-17T12:47:43.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.677+0000 7fd2e6b41700 1 --2- 172.21.15.36:0/3458408538 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd2c40539b0 0x7fd2c4055e70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:43.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.677+0000 7fd2e6b41700 1 --2- 172.21.15.36:0/3458408538 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd2e0144c00 0x7fd2e0144460 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:43.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.677+0000 7fd2e6b41700 1 --2- 172.21.15.36:0/3458408538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2e00ad330 0x7fd2e0143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:43.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.677+0000 7fd2e6b41700 1 --2- 172.21.15.36:0/3458408538 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2e0062770 0x7fd2e01314d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:43.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.677+0000 7fd2e6b41700 1 -- 172.21.15.36:0/3458408538 >> 172.21.15.36:0/3458408538 conn(0x7fd2e00b53e0 msgr2=0x7fd2e005f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:43.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.677+0000 7fd2e6b41700 1 -- 172.21.15.36:0/3458408538 shutdown_connections 2022-06-17T12:47:43.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.677+0000 7fd2e6b41700 1 -- 172.21.15.36:0/3458408538 wait complete. 2022-06-17T12:47:43.787 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-17T12:47:43.789 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:47:43.795 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:47:43.817 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:47:43.827 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:47:43.839 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:47:43.859 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:47:44.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.929+0000 7f6206d08700 1 Processor -- start 2022-06-17T12:47:44.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.929+0000 7f6206d08700 1 -- start start 2022-06-17T12:47:44.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f6206d08700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62000acb00 0x7f62000acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:44.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f6206d08700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62000b1900 con 0x7f6200144240 2022-06-17T12:47:44.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f6206d08700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62000af370 con 0x7f62000b05e0 2022-06-17T12:47:44.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f6206d08700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62000b0a60 con 0x7f62000acb00 2022-06-17T12:47:44.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f62052a5700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f6200144240 0x7f62001446d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50114/0 (socket says 172.21.15.36:50114) 2022-06-17T12:47:44.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f62052a5700 1 -- 172.21.15.36:0/2963815401 learned_addr learned my addr 172.21.15.36:0/2963815401 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:44.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f6204aa4700 1 --2- 172.21.15.36:0/2963815401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62000acb00 0x7f62000acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:44.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61ff7fe700 1 -- 172.21.15.36:0/2963815401 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3115162360 0 0) 0x7f62000b1900 con 0x7f6200144240 2022-06-17T12:47:44.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61ff7fe700 1 -- 172.21.15.36:0/2963815401 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61ec003260 con 0x7f6200144240 2022-06-17T12:47:44.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61ff7fe700 1 -- 172.21.15.36:0/2963815401 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4121541279 0 0) 0x7f62000af370 con 0x7f62000b05e0 2022-06-17T12:47:44.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61ff7fe700 1 -- 172.21.15.36:0/2963815401 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62000b1900 con 0x7f62000b05e0 2022-06-17T12:47:44.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61ff7fe700 1 -- 172.21.15.36:0/2963815401 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3969505566 0 0) 0x7f61ec003260 con 0x7f6200144240 2022-06-17T12:47:44.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61ff7fe700 1 -- 172.21.15.36:0/2963815401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62000acb00 msgr2=0x7f62000acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:44.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61ff7fe700 1 --2- 172.21.15.36:0/2963815401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62000acb00 0x7f62000acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61ff7fe700 1 -- 172.21.15.36:0/2963815401 >> v1:172.21.15.70:6789/0 conn(0x7f62000b05e0 legacy=0x7f6200147480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:44.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61ff7fe700 1 -- 172.21.15.36:0/2963815401 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62000b1b30 con 0x7f6200144240 2022-06-17T12:47:44.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61ff7fe700 1 -- 172.21.15.36:0/2963815401 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f61f8001660 con 0x7f6200144240 2022-06-17T12:47:44.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61ff7fe700 1 -- 172.21.15.36:0/2963815401 >> v1:172.21.15.36:6789/0 conn(0x7f6200144240 legacy=0x7f62001446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:44.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61ff7fe700 1 --2- 172.21.15.36:0/2963815401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62000acb00 0x7f61ec003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:44.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61ff7fe700 1 --2- 172.21.15.36:0/2963815401 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61ec003e90 0x7f61ec004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:44.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61ff7fe700 1 --2- 172.21.15.36:0/2963815401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61ec0049f0 0x7f61ec004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:44.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61ff7fe700 1 -- 172.21.15.36:0/2963815401 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62000b0a60 con 0x7f61ec003e90 2022-06-17T12:47:44.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61ff7fe700 1 -- 172.21.15.36:0/2963815401 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec00b830 con 0x7f61ec0049f0 2022-06-17T12:47:44.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61ff7fe700 1 -- 172.21.15.36:0/2963815401 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec00b9f0 con 0x7f62000acb00 2022-06-17T12:47:44.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61fffff700 1 --2- 172.21.15.36:0/2963815401 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61ec003e90 0x7f61ec004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:44.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f6204aa4700 1 --2- 172.21.15.36:0/2963815401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62000acb00 0x7f61ec003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:44.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f62052a5700 1 --2- 172.21.15.36:0/2963815401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61ec0049f0 0x7f61ec004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:44.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61fffff700 1 -- 172.21.15.36:0/2963815401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62000acb00 msgr2=0x7f61ec003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:44.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61fffff700 1 --2- 172.21.15.36:0/2963815401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62000acb00 0x7f61ec003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61fffff700 1 -- 172.21.15.36:0/2963815401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61ec0049f0 msgr2=0x7f61ec004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:44.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61fffff700 1 --2- 172.21.15.36:0/2963815401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61ec0049f0 0x7f61ec004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61fffff700 1 -- 172.21.15.36:0/2963815401 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ec003260 con 0x7f61ec003e90 2022-06-17T12:47:44.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61fffff700 1 --2- 172.21.15.36:0/2963815401 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61ec003e90 0x7f61ec004500 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f61f0001490 tx=0x7f61f0007d80).ready entity=mon.0 client_cookie=c6196c9b45f0117e server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:44.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.933+0000 7f61ff7fe700 1 -- 172.21.15.36:0/2963815401 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61f0001780 con 0x7f61ec003e90 2022-06-17T12:47:44.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.937+0000 7f61ff7fe700 1 -- 172.21.15.36:0/2963815401 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f0006890 con 0x7f61ec003e90 2022-06-17T12:47:44.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.937+0000 7f61ff7fe700 1 -- 172.21.15.36:0/2963815401 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61f0006bf0 con 0x7f61ec003e90 2022-06-17T12:47:44.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.937+0000 7f6206d08700 1 -- 172.21.15.36:0/2963815401 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61ec003e90 msgr2=0x7f61ec004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:44.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.937+0000 7f6206d08700 1 --2- 172.21.15.36:0/2963815401 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61ec003e90 0x7f61ec004500 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f61f0001490 tx=0x7f61f0007d80).stop 2022-06-17T12:47:44.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.937+0000 7f62052a5700 1 -- 172.21.15.36:0/2963815401 reap_dead start 2022-06-17T12:47:44.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.937+0000 7f6206d08700 1 -- 172.21.15.36:0/2963815401 shutdown_connections 2022-06-17T12:47:44.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.937+0000 7f6206d08700 1 -- 172.21.15.36:0/2963815401 >> 172.21.15.36:0/2963815401 conn(0x7f620005c550 msgr2=0x7f62000b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:44.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.937+0000 7f6206d08700 1 -- 172.21.15.36:0/2963815401 shutdown_connections 2022-06-17T12:47:44.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.937+0000 7f6206d08700 1 -- 172.21.15.36:0/2963815401 wait complete. 2022-06-17T12:47:44.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.937+0000 7f6206d08700 1 Processor -- start 2022-06-17T12:47:44.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.937+0000 7f6206d08700 1 -- start start 2022-06-17T12:47:44.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.937+0000 7f6206d08700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61ec003e90 0x7f6200147bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:44.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.937+0000 7f6206d08700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62000a9410 0x7f62000a8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:44.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.937+0000 7f6206d08700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6200156d40 0x7f62001571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:44.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.937+0000 7f6206d08700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6200156670 con 0x7f61ec003e90 2022-06-17T12:47:44.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.937+0000 7f6206d08700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62000ac170 con 0x7f62000a9410 2022-06-17T12:47:44.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.937+0000 7f6206d08700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62000a9920 con 0x7f6200156d40 2022-06-17T12:47:44.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.937+0000 7f6204aa4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61ec003e90 0x7f6200147bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:44.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.937+0000 7f61fffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62000a9410 0x7f62000a8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:44.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.937+0000 7f62052a5700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6200156d40 0x7f62001571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:44.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.937+0000 7f6204aa4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61ec003e90 0x7f6200147bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60824/0 (socket says 172.21.15.36:60824) 2022-06-17T12:47:44.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.937+0000 7f6204aa4700 1 -- 172.21.15.36:0/1572796924 learned_addr learned my addr 172.21.15.36:0/1572796924 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:44.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.941+0000 7f61fffff700 1 -- 172.21.15.36:0/1572796924 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6200156d40 msgr2=0x7f62001571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:44.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.941+0000 7f61fffff700 1 --2- 172.21.15.36:0/1572796924 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6200156d40 0x7f62001571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.941+0000 7f61fffff700 1 -- 172.21.15.36:0/1572796924 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61ec003e90 msgr2=0x7f6200147bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:44.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.941+0000 7f61fffff700 1 --2- 172.21.15.36:0/1572796924 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61ec003e90 0x7f6200147bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.941+0000 7f61fffff700 1 -- 172.21.15.36:0/1572796924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62001568a0 con 0x7f62000a9410 2022-06-17T12:47:44.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.941+0000 7f61fffff700 1 --2- 172.21.15.36:0/1572796924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62000a9410 0x7f62000a8070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f61f0001490 tx=0x7f61f0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:44.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.941+0000 7f61fdffb700 1 -- 172.21.15.36:0/1572796924 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61f0001780 con 0x7f62000a9410 2022-06-17T12:47:44.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.941+0000 7f61fdffb700 1 -- 172.21.15.36:0/1572796924 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f000ca10 con 0x7f62000a9410 2022-06-17T12:47:44.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.941+0000 7f6206d08700 1 -- 172.21.15.36:0/1572796924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62000a8560 con 0x7f62000a9410 2022-06-17T12:47:44.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.941+0000 7f61fdffb700 1 -- 172.21.15.36:0/1572796924 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61f000cd70 con 0x7f62000a9410 2022-06-17T12:47:44.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.941+0000 7f6206d08700 1 -- 172.21.15.36:0/1572796924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6200159ea0 con 0x7f62000a9410 2022-06-17T12:47:44.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.941+0000 7f6206d08700 1 -- 172.21.15.36:0/1572796924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6200156a30 con 0x7f62000a9410 2022-06-17T12:47:44.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.941+0000 7f61fdffb700 1 -- 172.21.15.36:0/1572796924 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f61f0010030 con 0x7f62000a9410 2022-06-17T12:47:44.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.945+0000 7f61fdffb700 1 --2- 172.21.15.36:0/1572796924 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f61e0053940 0x7f61e0055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:44.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.945+0000 7f61fdffb700 1 -- 172.21.15.36:0/1572796924 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f61f006cbd0 con 0x7f62000a9410 2022-06-17T12:47:44.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.945+0000 7f6204aa4700 1 --2- 172.21.15.36:0/1572796924 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f61e0053940 0x7f61e0055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:44.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.945+0000 7f6204aa4700 1 --2- 172.21.15.36:0/1572796924 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f61e0053940 0x7f61e0055e00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f61f400b000 tx=0x7f61f4006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:44.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:43.953+0000 7f61fdffb700 1 -- 172.21.15.36:0/1572796924 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f61f0043c70 con 0x7f62000a9410 2022-06-17T12:47:44.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.081+0000 7f6206d08700 1 -- 172.21.15.36:0/1572796924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6200057410 con 0x7f62000a9410 2022-06-17T12:47:44.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.081+0000 7f61fdffb700 1 -- 172.21.15.36:0/1572796924 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v31) v1 ==== 110+0+10 (secure 0 0 0) 0x7f61f0023070 con 0x7f62000a9410 2022-06-17T12:47:44.087 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:47:44.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.085+0000 7f6206d08700 1 -- 172.21.15.36:0/1572796924 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f61e0053940 msgr2=0x7f61e0055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:44.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.085+0000 7f6206d08700 1 --2- 172.21.15.36:0/1572796924 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f61e0053940 0x7f61e0055e00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f61f400b000 tx=0x7f61f4006df0).stop 2022-06-17T12:47:44.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.085+0000 7f6206d08700 1 -- 172.21.15.36:0/1572796924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62000a9410 msgr2=0x7f62000a8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:44.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.085+0000 7f6206d08700 1 --2- 172.21.15.36:0/1572796924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62000a9410 0x7f62000a8070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f61f0001490 tx=0x7f61f0016000).stop 2022-06-17T12:47:44.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.085+0000 7f6206d08700 1 -- 172.21.15.36:0/1572796924 shutdown_connections 2022-06-17T12:47:44.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.085+0000 7f6206d08700 1 --2- 172.21.15.36:0/1572796924 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f61e0053940 0x7f61e0055e00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.085+0000 7f6206d08700 1 --2- 172.21.15.36:0/1572796924 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6200156d40 0x7f62001571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.085+0000 7f6206d08700 1 --2- 172.21.15.36:0/1572796924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62000a9410 0x7f62000a8070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.085+0000 7f6206d08700 1 --2- 172.21.15.36:0/1572796924 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61ec003e90 0x7f6200147bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.085+0000 7f6206d08700 1 -- 172.21.15.36:0/1572796924 >> 172.21.15.36:0/1572796924 conn(0x7f620005c550 msgr2=0x7f6200141880 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:44.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.085+0000 7f6206d08700 1 -- 172.21.15.36:0/1572796924 shutdown_connections 2022-06-17T12:47:44.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.085+0000 7f6206d08700 1 -- 172.21.15.36:0/1572796924 wait complete. 2022-06-17T12:47:44.095 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:47:44.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.225+0000 7ff65bfff700 1 Processor -- start 2022-06-17T12:47:44.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.225+0000 7ff65bfff700 1 -- start start 2022-06-17T12:47:44.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.229+0000 7ff65bfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff65c0adf10 0x7ff65c0b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:44.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.229+0000 7ff65bfff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff65c0b15b0 0x7ff65c0af630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:44.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.229+0000 7ff65bfff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff65c13fdf0 con 0x7ff65c0b65f0 2022-06-17T12:47:44.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.229+0000 7ff65bfff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff65c0ae8e0 con 0x7ff65c0adf10 2022-06-17T12:47:44.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.229+0000 7ff65bfff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff65c140090 con 0x7ff65c0b15b0 2022-06-17T12:47:44.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.229+0000 7ff6527fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff65c0adf10 0x7ff65c0b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:44.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.229+0000 7ff65affd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7ff65c0b65f0 0x7ff65c140530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50130/0 (socket says 172.21.15.36:50130) 2022-06-17T12:47:44.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.229+0000 7ff65affd700 1 -- 172.21.15.36:0/454350348 learned_addr learned my addr 172.21.15.36:0/454350348 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:44.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.229+0000 7ff65b7fe700 1 --2- 172.21.15.36:0/454350348 >> v2:172.21.15.70:3300/0 conn(0x7ff65c0b15b0 0x7ff65c0af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:44.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.229+0000 7ff6527fc700 1 -- 172.21.15.36:0/454350348 >> v2:172.21.15.70:3300/0 conn(0x7ff65c0b15b0 msgr2=0x7ff65c0af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:44.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.229+0000 7ff6527fc700 1 --2- 172.21.15.36:0/454350348 >> v2:172.21.15.70:3300/0 conn(0x7ff65c0b15b0 0x7ff65c0af630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.229+0000 7ff6527fc700 1 -- 172.21.15.36:0/454350348 >> v1:172.21.15.36:6789/0 conn(0x7ff65c0b65f0 legacy=0x7ff65c140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:44.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.229+0000 7ff6527fc700 1 -- 172.21.15.36:0/454350348 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65c0b1be0 con 0x7ff65c0adf10 2022-06-17T12:47:44.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.229+0000 7ff6527fc700 1 --2- 172.21.15.36:0/454350348 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff65c0adf10 0x7ff65c0b69d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7ff644002f70 tx=0x7ff644006d90).ready entity=mon.0 client_cookie=8a2a1c737cbe308 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:44.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.229+0000 7ff65a7fc700 1 -- 172.21.15.36:0/454350348 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff64400d4a0 con 0x7ff65c0adf10 2022-06-17T12:47:44.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.229+0000 7ff65a7fc700 1 -- 172.21.15.36:0/454350348 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff644000e90 con 0x7ff65c0adf10 2022-06-17T12:47:44.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.229+0000 7ff65a7fc700 1 -- 172.21.15.36:0/454350348 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff64400d7f0 con 0x7ff65c0adf10 2022-06-17T12:47:44.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.229+0000 7ff65bfff700 1 -- 172.21.15.36:0/454350348 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff65c0adf10 msgr2=0x7ff65c0b69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:44.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.229+0000 7ff65bfff700 1 --2- 172.21.15.36:0/454350348 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff65c0adf10 0x7ff65c0b69d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7ff644002f70 tx=0x7ff644006d90).stop 2022-06-17T12:47:44.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.229+0000 7ff65bfff700 1 -- 172.21.15.36:0/454350348 shutdown_connections 2022-06-17T12:47:44.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.233+0000 7ff65bfff700 1 --2- 172.21.15.36:0/454350348 >> v2:172.21.15.70:3300/0 conn(0x7ff65c0b15b0 0x7ff65c0af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.233+0000 7ff65bfff700 1 --2- 172.21.15.36:0/454350348 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff65c0adf10 0x7ff65c0b69d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.233+0000 7ff65bfff700 1 -- 172.21.15.36:0/454350348 >> 172.21.15.36:0/454350348 conn(0x7ff65c0614b0 msgr2=0x7ff65c061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:44.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.233+0000 7ff65bfff700 1 -- 172.21.15.36:0/454350348 shutdown_connections 2022-06-17T12:47:44.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.233+0000 7ff65bfff700 1 -- 172.21.15.36:0/454350348 wait complete. 2022-06-17T12:47:44.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.233+0000 7ff65bfff700 1 Processor -- start 2022-06-17T12:47:44.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.233+0000 7ff65bfff700 1 -- start start 2022-06-17T12:47:44.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.233+0000 7ff65bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff65c0adf10 0x7ff65c063460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:44.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.233+0000 7ff65bfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff65c0b15b0 0x7ff65c134a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:44.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.233+0000 7ff65bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff65c0b65f0 0x7ff65c134f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:44.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.233+0000 7ff65bfff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c13f6b0 con 0x7ff65c0b65f0 2022-06-17T12:47:44.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.233+0000 7ff65bfff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c135430 con 0x7ff65c0adf10 2022-06-17T12:47:44.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.233+0000 7ff65bfff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c066230 con 0x7ff65c0b15b0 2022-06-17T12:47:44.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.237+0000 7ff65b7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff65c0b65f0 0x7ff65c134f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:44.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.237+0000 7ff65affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff65c0adf10 0x7ff65c063460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:44.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.237+0000 7ff65b7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff65c0b65f0 0x7ff65c134f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60842/0 (socket says 172.21.15.36:60842) 2022-06-17T12:47:44.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.237+0000 7ff6527fc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff65c0b15b0 0x7ff65c134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:44.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.237+0000 7ff65b7fe700 1 -- 172.21.15.36:0/3995990534 learned_addr learned my addr 172.21.15.36:0/3995990534 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:44.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.237+0000 7ff65b7fe700 1 -- 172.21.15.36:0/3995990534 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff65c0b15b0 msgr2=0x7ff65c134a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:44.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.237+0000 7ff65b7fe700 1 --2- 172.21.15.36:0/3995990534 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff65c0b15b0 0x7ff65c134a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.237+0000 7ff65b7fe700 1 -- 172.21.15.36:0/3995990534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff65c0adf10 msgr2=0x7ff65c063460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:44.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.237+0000 7ff65b7fe700 1 --2- 172.21.15.36:0/3995990534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff65c0adf10 0x7ff65c063460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.237+0000 7ff65b7fe700 1 -- 172.21.15.36:0/3995990534 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64400b120 con 0x7ff65c0b65f0 2022-06-17T12:47:44.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.237+0000 7ff65b7fe700 1 --2- 172.21.15.36:0/3995990534 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff65c0b65f0 0x7ff65c134f40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7ff654006e00 tx=0x7ff6540091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:44.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.237+0000 7ff658ff9700 1 -- 172.21.15.36:0/3995990534 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff65400c640 con 0x7ff65c0b65f0 2022-06-17T12:47:44.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.237+0000 7ff658ff9700 1 -- 172.21.15.36:0/3995990534 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65400c7a0 con 0x7ff65c0b65f0 2022-06-17T12:47:44.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.237+0000 7ff65bfff700 1 -- 172.21.15.36:0/3995990534 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff65c0664d0 con 0x7ff65c0b65f0 2022-06-17T12:47:44.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.237+0000 7ff658ff9700 1 -- 172.21.15.36:0/3995990534 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff65400cb00 con 0x7ff65c0b65f0 2022-06-17T12:47:44.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.237+0000 7ff65bfff700 1 -- 172.21.15.36:0/3995990534 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff65c065430 con 0x7ff65c0b65f0 2022-06-17T12:47:44.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.237+0000 7ff65bfff700 1 -- 172.21.15.36:0/3995990534 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff65c137750 con 0x7ff65c0b65f0 2022-06-17T12:47:44.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.237+0000 7ff658ff9700 1 -- 172.21.15.36:0/3995990534 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff65400cc60 con 0x7ff65c0b65f0 2022-06-17T12:47:44.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.241+0000 7ff658ff9700 1 --2- 172.21.15.36:0/3995990534 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff634053a60 0x7ff634055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:44.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.241+0000 7ff658ff9700 1 -- 172.21.15.36:0/3995990534 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7ff65406bfe0 con 0x7ff65c0b65f0 2022-06-17T12:47:44.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.241+0000 7ff65affd700 1 --2- 172.21.15.36:0/3995990534 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff634053a60 0x7ff634055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:44.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.241+0000 7ff65affd700 1 --2- 172.21.15.36:0/3995990534 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff634053a60 0x7ff634055f20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7ff64c00d000 tx=0x7ff64c000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:44.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.245+0000 7ff658ff9700 1 -- 172.21.15.36:0/3995990534 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff654042f90 con 0x7ff65c0b65f0 2022-06-17T12:47:44.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.369+0000 7ff65bfff700 1 -- 172.21.15.36:0/3995990534 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff65c137350 con 0x7ff65c0b65f0 2022-06-17T12:47:44.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.369+0000 7ff658ff9700 1 -- 172.21.15.36:0/3995990534 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v31) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff6540465b0 con 0x7ff65c0b65f0 2022-06-17T12:47:44.361 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:44.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.373+0000 7ff65bfff700 1 -- 172.21.15.36:0/3995990534 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff634053a60 msgr2=0x7ff634055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:44.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.373+0000 7ff65bfff700 1 --2- 172.21.15.36:0/3995990534 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff634053a60 0x7ff634055f20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7ff64c00d000 tx=0x7ff64c000bc0).stop 2022-06-17T12:47:44.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.373+0000 7ff65bfff700 1 -- 172.21.15.36:0/3995990534 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff65c0b65f0 msgr2=0x7ff65c134f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:44.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.373+0000 7ff65bfff700 1 --2- 172.21.15.36:0/3995990534 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff65c0b65f0 0x7ff65c134f40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7ff654006e00 tx=0x7ff6540091d0).stop 2022-06-17T12:47:44.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.373+0000 7ff65bfff700 1 -- 172.21.15.36:0/3995990534 shutdown_connections 2022-06-17T12:47:44.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.373+0000 7ff65bfff700 1 --2- 172.21.15.36:0/3995990534 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff634053a60 0x7ff634055f20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.373+0000 7ff65bfff700 1 --2- 172.21.15.36:0/3995990534 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff65c0b65f0 0x7ff65c134f40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.373+0000 7ff65bfff700 1 --2- 172.21.15.36:0/3995990534 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff65c0b15b0 0x7ff65c134a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.373+0000 7ff65bfff700 1 --2- 172.21.15.36:0/3995990534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff65c0adf10 0x7ff65c063460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.373+0000 7ff65bfff700 1 -- 172.21.15.36:0/3995990534 >> 172.21.15.36:0/3995990534 conn(0x7ff65c0614b0 msgr2=0x7ff65c133060 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:44.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.373+0000 7ff65bfff700 1 -- 172.21.15.36:0/3995990534 shutdown_connections 2022-06-17T12:47:44.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.373+0000 7ff65bfff700 1 -- 172.21.15.36:0/3995990534 wait complete. 2022-06-17T12:47:44.409 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-17T12:47:44.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.529+0000 7fd18c96f700 1 Processor -- start 2022-06-17T12:47:44.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.529+0000 7fd18c96f700 1 -- start start 2022-06-17T12:47:44.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.533+0000 7fd18c96f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd1880aee30 0x7fd1880afe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:44.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.533+0000 7fd18c96f700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd188060ea0 con 0x7fd188060ac0 2022-06-17T12:47:44.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.533+0000 7fd18c96f700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1880b3060 con 0x7fd1880b20f0 2022-06-17T12:47:44.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.533+0000 7fd18c96f700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd188067a20 con 0x7fd1880aee30 2022-06-17T12:47:44.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.533+0000 7fd1877fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fd188060ac0 0x7fd1880ae720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50144/0 (socket says 172.21.15.36:50144) 2022-06-17T12:47:44.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.533+0000 7fd1877fe700 1 -- 172.21.15.36:0/2328074598 learned_addr learned my addr 172.21.15.36:0/2328074598 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:44.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.533+0000 7fd187fff700 1 --2- 172.21.15.36:0/2328074598 >> v2:172.21.15.36:3300/0 conn(0x7fd1880aee30 0x7fd1880afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:44.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.533+0000 7fd1867fc700 1 -- 172.21.15.36:0/2328074598 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1165479098 0 0) 0x7fd188060ea0 con 0x7fd188060ac0 2022-06-17T12:47:44.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.533+0000 7fd1867fc700 1 -- 172.21.15.36:0/2328074598 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd168003260 con 0x7fd188060ac0 2022-06-17T12:47:44.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.533+0000 7fd187fff700 1 -- 172.21.15.36:0/2328074598 >> v1:172.21.15.70:6789/0 conn(0x7fd1880b20f0 legacy=0x7fd18813c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:44.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.533+0000 7fd187fff700 1 -- 172.21.15.36:0/2328074598 >> v1:172.21.15.36:6789/0 conn(0x7fd188060ac0 legacy=0x7fd1880ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:44.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.533+0000 7fd1867fc700 1 -- 172.21.15.36:0/2328074598 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1265698911 0 0) 0x7fd168003260 con 0x7fd188060ac0 2022-06-17T12:47:44.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.533+0000 7fd187fff700 1 -- 172.21.15.36:0/2328074598 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd188143fb0 con 0x7fd1880aee30 2022-06-17T12:47:44.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.533+0000 7fd187fff700 1 --2- 172.21.15.36:0/2328074598 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1880aee30 0x7fd1880afe50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7fd178001800 tx=0x7fd1800091d0).ready entity=mon.0 client_cookie=fe868db86d6e380c server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:44.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.533+0000 7fd1867fc700 1 -- 172.21.15.36:0/2328074598 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd178001660 con 0x7fd1880aee30 2022-06-17T12:47:44.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.533+0000 7fd1867fc700 1 -- 172.21.15.36:0/2328074598 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18000b580 con 0x7fd1880aee30 2022-06-17T12:47:44.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.533+0000 7fd1867fc700 1 -- 172.21.15.36:0/2328074598 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd18000b6f0 con 0x7fd1880aee30 2022-06-17T12:47:44.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd18c96f700 1 -- 172.21.15.36:0/2328074598 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1880aee30 msgr2=0x7fd1880afe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:44.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd18c96f700 1 --2- 172.21.15.36:0/2328074598 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1880aee30 0x7fd1880afe50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7fd178001800 tx=0x7fd1800091d0).stop 2022-06-17T12:47:44.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd18c96f700 1 -- 172.21.15.36:0/2328074598 shutdown_connections 2022-06-17T12:47:44.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd18c96f700 1 --2- 172.21.15.36:0/2328074598 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1880aee30 0x7fd1880afe50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd18c96f700 1 -- 172.21.15.36:0/2328074598 >> 172.21.15.36:0/2328074598 conn(0x7fd1880b65f0 msgr2=0x7fd1880b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:44.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd18c96f700 1 -- 172.21.15.36:0/2328074598 shutdown_connections 2022-06-17T12:47:44.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd18c96f700 1 -- 172.21.15.36:0/2328074598 wait complete. 2022-06-17T12:47:44.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd18c96f700 1 Processor -- start 2022-06-17T12:47:44.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd18c96f700 1 -- start start 2022-06-17T12:47:44.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd18c96f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd188060ac0 0x7fd188146700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:44.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd18c96f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1880aee30 0x7fd1881445f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:44.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd18c96f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1880b20f0 0x7fd188155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:44.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd18c96f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd188144eb0 con 0x7fd1880aee30 2022-06-17T12:47:44.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd18c96f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1881476a0 con 0x7fd1880b20f0 2022-06-17T12:47:44.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd18c96f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd188147940 con 0x7fd188060ac0 2022-06-17T12:47:44.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd186ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1880aee30 0x7fd1881445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:44.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd186ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1880aee30 0x7fd1881445f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60854/0 (socket says 172.21.15.36:60854) 2022-06-17T12:47:44.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd186ffd700 1 -- 172.21.15.36:0/2588720269 learned_addr learned my addr 172.21.15.36:0/2588720269 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:44.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd1877fe700 1 --2- 172.21.15.36:0/2588720269 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd188060ac0 0x7fd188146700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:44.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd187fff700 1 --2- 172.21.15.36:0/2588720269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1880b20f0 0x7fd188155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:44.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd186ffd700 1 -- 172.21.15.36:0/2588720269 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd188060ac0 msgr2=0x7fd188146700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:44.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd186ffd700 1 --2- 172.21.15.36:0/2588720269 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd188060ac0 0x7fd188146700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd186ffd700 1 -- 172.21.15.36:0/2588720269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1880b20f0 msgr2=0x7fd188155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:44.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd186ffd700 1 --2- 172.21.15.36:0/2588720269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1880b20f0 0x7fd188155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd186ffd700 1 -- 172.21.15.36:0/2588720269 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd180009470 con 0x7fd1880aee30 2022-06-17T12:47:44.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd187fff700 1 --2- 172.21.15.36:0/2588720269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1880b20f0 0x7fd188155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:44.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd186ffd700 1 --2- 172.21.15.36:0/2588720269 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1880aee30 0x7fd1881445f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fd1880622a0 tx=0x7fd170007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:44.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd184ff9700 1 -- 172.21.15.36:0/2588720269 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd17000cd80 con 0x7fd1880aee30 2022-06-17T12:47:44.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd184ff9700 1 -- 172.21.15.36:0/2588720269 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd170012400 con 0x7fd1880aee30 2022-06-17T12:47:44.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd18c96f700 1 -- 172.21.15.36:0/2588720269 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd188144ae0 con 0x7fd1880aee30 2022-06-17T12:47:44.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.537+0000 7fd184ff9700 1 -- 172.21.15.36:0/2588720269 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd170012760 con 0x7fd1880aee30 2022-06-17T12:47:44.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.541+0000 7fd18c96f700 1 -- 172.21.15.36:0/2588720269 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd188145c00 con 0x7fd1880aee30 2022-06-17T12:47:44.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.541+0000 7fd18c96f700 1 -- 172.21.15.36:0/2588720269 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd188147b00 con 0x7fd1880aee30 2022-06-17T12:47:44.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.541+0000 7fd184ff9700 1 -- 172.21.15.36:0/2588720269 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd170019030 con 0x7fd1880aee30 2022-06-17T12:47:44.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.541+0000 7fd184ff9700 1 --2- 172.21.15.36:0/2588720269 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd16c053a40 0x7fd16c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:44.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.541+0000 7fd1877fe700 1 --2- 172.21.15.36:0/2588720269 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd16c053a40 0x7fd16c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:44.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.541+0000 7fd184ff9700 1 -- 172.21.15.36:0/2588720269 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fd17006aff0 con 0x7fd1880aee30 2022-06-17T12:47:44.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.541+0000 7fd1877fe700 1 --2- 172.21.15.36:0/2588720269 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd16c053a40 0x7fd16c055f00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fd17800d000 tx=0x7fd178000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:44.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.549+0000 7fd184ff9700 1 -- 172.21.15.36:0/2588720269 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd1700420d0 con 0x7fd1880aee30 2022-06-17T12:47:44.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.681+0000 7fd18c96f700 1 -- 172.21.15.36:0/2588720269 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd188057410 con 0x7fd1880aee30 2022-06-17T12:47:44.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.681+0000 7fd184ff9700 1 -- 172.21.15.36:0/2588720269 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) v1 ==== 84+0+143 (secure 0 0 0) 0x7fd1700456f0 con 0x7fd1880aee30 2022-06-17T12:47:44.703 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:44.704 INFO:teuthology.orchestra.run.smithi036.stdout:[ 2022-06-17T12:47:44.704 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:44.704 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 1, 2022-06-17T12:47:44.705 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "device_health_metrics" 2022-06-17T12:47:44.705 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:47:44.706 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:47:44.706 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 2, 2022-06-17T12:47:44.714 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "rbd" 2022-06-17T12:47:44.714 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:47:44.714 INFO:teuthology.orchestra.run.smithi036.stdout:] 2022-06-17T12:47:44.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.685+0000 7fd18c96f700 1 -- 172.21.15.36:0/2588720269 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd16c053a40 msgr2=0x7fd16c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:44.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.685+0000 7fd18c96f700 1 --2- 172.21.15.36:0/2588720269 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd16c053a40 0x7fd16c055f00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fd17800d000 tx=0x7fd178000bc0).stop 2022-06-17T12:47:44.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.685+0000 7fd18c96f700 1 -- 172.21.15.36:0/2588720269 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1880aee30 msgr2=0x7fd1881445f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:44.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.685+0000 7fd18c96f700 1 --2- 172.21.15.36:0/2588720269 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1880aee30 0x7fd1881445f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fd1880622a0 tx=0x7fd170007c20).stop 2022-06-17T12:47:44.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.685+0000 7fd18c96f700 1 -- 172.21.15.36:0/2588720269 shutdown_connections 2022-06-17T12:47:44.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.685+0000 7fd18c96f700 1 --2- 172.21.15.36:0/2588720269 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd16c053a40 0x7fd16c055f00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.685+0000 7fd18c96f700 1 --2- 172.21.15.36:0/2588720269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1880b20f0 0x7fd188155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.685+0000 7fd18c96f700 1 --2- 172.21.15.36:0/2588720269 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1880aee30 0x7fd1881445f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.685+0000 7fd18c96f700 1 --2- 172.21.15.36:0/2588720269 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd188060ac0 0x7fd188146700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.685+0000 7fd18c96f700 1 -- 172.21.15.36:0/2588720269 >> 172.21.15.36:0/2588720269 conn(0x7fd1880b65f0 msgr2=0x7fd1880b04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:44.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.685+0000 7fd18c96f700 1 -- 172.21.15.36:0/2588720269 shutdown_connections 2022-06-17T12:47:44.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.685+0000 7fd18c96f700 1 -- 172.21.15.36:0/2588720269 wait complete. 2022-06-17T12:47:44.740 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-17T12:47:44.741 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-17T12:47:44.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.885+0000 7f38fffff700 1 Processor -- start 2022-06-17T12:47:44.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.885+0000 7f38fffff700 1 -- start start 2022-06-17T12:47:44.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.885+0000 7f38fffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f39000b27b0 0x7f390013f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:44.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.885+0000 7f38fffff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39000b2b90 con 0x7f39000b0d70 2022-06-17T12:47:44.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.885+0000 7f38fffff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39000610e0 con 0x7f39000b15f0 2022-06-17T12:47:44.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.885+0000 7f38fffff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39000ac190 con 0x7f39000b27b0 2022-06-17T12:47:44.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.885+0000 7f38f7fff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f39000b0d70 0x7f39000ac780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50160/0 (socket says 172.21.15.36:50160) 2022-06-17T12:47:44.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.885+0000 7f38feffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f39000b27b0 0x7f390013f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:44.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.885+0000 7f38f7fff700 1 -- 172.21.15.36:0/3426230449 learned_addr learned my addr 172.21.15.36:0/3426230449 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:44.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.889+0000 7f38fe7fc700 1 -- 172.21.15.36:0/3426230449 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1242040342 0 0) 0x7f39000b2b90 con 0x7f39000b0d70 2022-06-17T12:47:44.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.889+0000 7f38fe7fc700 1 -- 172.21.15.36:0/3426230449 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38e0003260 con 0x7f39000b0d70 2022-06-17T12:47:44.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.889+0000 7f38fe7fc700 1 -- 172.21.15.36:0/3426230449 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 264338637 0 0) 0x7f39000610e0 con 0x7f39000b15f0 2022-06-17T12:47:44.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.889+0000 7f38fe7fc700 1 -- 172.21.15.36:0/3426230449 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39000b2b90 con 0x7f39000b15f0 2022-06-17T12:47:44.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.889+0000 7f38fe7fc700 1 -- 172.21.15.36:0/3426230449 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3383531089 0 0) 0x7f38e0003260 con 0x7f39000b0d70 2022-06-17T12:47:44.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.889+0000 7f38feffd700 1 -- 172.21.15.36:0/3426230449 >> v1:172.21.15.70:6789/0 conn(0x7f39000b15f0 legacy=0x7f390014a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:44.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.889+0000 7f38feffd700 1 -- 172.21.15.36:0/3426230449 >> v1:172.21.15.36:6789/0 conn(0x7f39000b0d70 legacy=0x7f39000ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:44.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.889+0000 7f38feffd700 1 -- 172.21.15.36:0/3426230449 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39000ac430 con 0x7f39000b27b0 2022-06-17T12:47:44.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.889+0000 7f38feffd700 1 --2- 172.21.15.36:0/3426230449 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f39000b27b0 0x7f390013f110 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f38e8001110 tx=0x7f38f0006df0).ready entity=mon.0 client_cookie=6d109e5687d5c09f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:44.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.889+0000 7f38fe7fc700 1 -- 172.21.15.36:0/3426230449 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38e8001660 con 0x7f39000b27b0 2022-06-17T12:47:44.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.889+0000 7f38fe7fc700 1 -- 172.21.15.36:0/3426230449 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f0000e90 con 0x7f39000b27b0 2022-06-17T12:47:44.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.889+0000 7f38fe7fc700 1 -- 172.21.15.36:0/3426230449 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38f0012460 con 0x7f39000b27b0 2022-06-17T12:47:44.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.889+0000 7f38fffff700 1 -- 172.21.15.36:0/3426230449 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f39000b27b0 msgr2=0x7f390013f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:44.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.889+0000 7f38fffff700 1 --2- 172.21.15.36:0/3426230449 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f39000b27b0 0x7f390013f110 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f38e8001110 tx=0x7f38f0006df0).stop 2022-06-17T12:47:44.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.889+0000 7f38fffff700 1 -- 172.21.15.36:0/3426230449 shutdown_connections 2022-06-17T12:47:44.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.889+0000 7f38fffff700 1 --2- 172.21.15.36:0/3426230449 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f39000b27b0 0x7f390013f110 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:44.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.889+0000 7f38fffff700 1 -- 172.21.15.36:0/3426230449 >> 172.21.15.36:0/3426230449 conn(0x7f39000b5d10 msgr2=0x7f39000b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:44.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.889+0000 7f38fffff700 1 -- 172.21.15.36:0/3426230449 shutdown_connections 2022-06-17T12:47:44.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.889+0000 7f38fffff700 1 -- 172.21.15.36:0/3426230449 wait complete. 2022-06-17T12:47:44.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.889+0000 7f38fffff700 1 Processor -- start 2022-06-17T12:47:44.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.893+0000 7f38fffff700 1 -- start start 2022-06-17T12:47:44.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.893+0000 7f38fffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39000b0d70 0x7f390014fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:44.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.893+0000 7f38fffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39000b15f0 0x7f39000656f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:44.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.893+0000 7f38fffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f39000b27b0 0x7f3900062390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:44.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.893+0000 7f38fffff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390014f230 con 0x7f39000b27b0 2022-06-17T12:47:44.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.893+0000 7f38fffff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3900065d80 con 0x7f39000b0d70 2022-06-17T12:47:44.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.893+0000 7f38fffff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390014f420 con 0x7f39000b15f0 2022-06-17T12:47:44.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.893+0000 7f38ff7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f39000b27b0 0x7f3900062390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:44.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.893+0000 7f38ff7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f39000b27b0 0x7f3900062390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60870/0 (socket says 172.21.15.36:60870) 2022-06-17T12:47:44.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.893+0000 7f38ff7fe700 1 -- 172.21.15.36:0/2714020965 learned_addr learned my addr 172.21.15.36:0/2714020965 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:44.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.893+0000 7f38feffd700 1 --2- 172.21.15.36:0/2714020965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39000b0d70 0x7f390014fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:44.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.893+0000 7f38f7fff700 1 --2- 172.21.15.36:0/2714020965 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39000b15f0 0x7f39000656f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:45.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.893+0000 7f38ff7fe700 1 -- 172.21.15.36:0/2714020965 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39000b15f0 msgr2=0x7f39000656f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:45.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.893+0000 7f38ff7fe700 1 --2- 172.21.15.36:0/2714020965 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39000b15f0 0x7f39000656f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:45.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.893+0000 7f38ff7fe700 1 -- 172.21.15.36:0/2714020965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39000b0d70 msgr2=0x7f390014fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:45.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.893+0000 7f38ff7fe700 1 --2- 172.21.15.36:0/2714020965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39000b0d70 0x7f390014fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:45.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.893+0000 7f38ff7fe700 1 -- 172.21.15.36:0/2714020965 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f000b180 con 0x7f39000b27b0 2022-06-17T12:47:45.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.893+0000 7f38ff7fe700 1 --2- 172.21.15.36:0/2714020965 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f39000b27b0 0x7f3900062390 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f38f8001490 tx=0x7f38f8007e60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:45.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.893+0000 7f38fcff9700 1 -- 172.21.15.36:0/2714020965 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38f8002310 con 0x7f39000b27b0 2022-06-17T12:47:45.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.893+0000 7f38fcff9700 1 -- 172.21.15.36:0/2714020965 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f8002470 con 0x7f39000b27b0 2022-06-17T12:47:45.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.893+0000 7f38fcff9700 1 -- 172.21.15.36:0/2714020965 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38f8001840 con 0x7f39000b27b0 2022-06-17T12:47:45.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.893+0000 7f38fffff700 1 -- 172.21.15.36:0/2714020965 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390014f6c0 con 0x7f39000b27b0 2022-06-17T12:47:45.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.893+0000 7f38fffff700 1 -- 172.21.15.36:0/2714020965 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39000669d0 con 0x7f39000b27b0 2022-06-17T12:47:45.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.897+0000 7f38fffff700 1 -- 172.21.15.36:0/2714020965 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39001407b0 con 0x7f39000b27b0 2022-06-17T12:47:45.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.897+0000 7f38fcff9700 1 -- 172.21.15.36:0/2714020965 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f38f8021070 con 0x7f39000b27b0 2022-06-17T12:47:45.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.897+0000 7f38fcff9700 1 --2- 172.21.15.36:0/2714020965 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f38e4053ad0 0x7f38e4055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:45.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.897+0000 7f38fcff9700 1 -- 172.21.15.36:0/2714020965 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f38f806e350 con 0x7f39000b27b0 2022-06-17T12:47:45.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.897+0000 7f38feffd700 1 --2- 172.21.15.36:0/2714020965 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f38e4053ad0 0x7f38e4055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:45.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.897+0000 7f38feffd700 1 --2- 172.21.15.36:0/2714020965 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f38e4053ad0 0x7f38e4055f90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f38f001a000 tx=0x7f38f0014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:45.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:44.905+0000 7f38fcff9700 1 -- 172.21.15.36:0/2714020965 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f38f8045490 con 0x7f39000b27b0 2022-06-17T12:47:45.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.029+0000 7f38fffff700 1 -- 172.21.15.36:0/2714020965 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f3900057410 con 0x7f39000b27b0 2022-06-17T12:47:45.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.713+0000 7f38fcff9700 1 -- 172.21.15.36:0/2714020965 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v32) v1 ==== 133+0+0 (secure 0 0 0) 0x7f38f8015070 con 0x7f39000b27b0 2022-06-17T12:47:45.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.717+0000 7f38fffff700 1 -- 172.21.15.36:0/2714020965 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f38e4053ad0 msgr2=0x7f38e4055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:45.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.717+0000 7f38fffff700 1 --2- 172.21.15.36:0/2714020965 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f38e4053ad0 0x7f38e4055f90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f38f001a000 tx=0x7f38f0014000).stop 2022-06-17T12:47:45.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.717+0000 7f38fffff700 1 -- 172.21.15.36:0/2714020965 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f39000b27b0 msgr2=0x7f3900062390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:45.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.717+0000 7f38fffff700 1 --2- 172.21.15.36:0/2714020965 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f39000b27b0 0x7f3900062390 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f38f8001490 tx=0x7f38f8007e60).stop 2022-06-17T12:47:45.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.717+0000 7f38fffff700 1 -- 172.21.15.36:0/2714020965 shutdown_connections 2022-06-17T12:47:45.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.717+0000 7f38fffff700 1 --2- 172.21.15.36:0/2714020965 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f38e4053ad0 0x7f38e4055f90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:45.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.717+0000 7f38fffff700 1 --2- 172.21.15.36:0/2714020965 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f39000b27b0 0x7f3900062390 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:45.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.717+0000 7f38fffff700 1 --2- 172.21.15.36:0/2714020965 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39000b15f0 0x7f39000656f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:45.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.717+0000 7f38fffff700 1 --2- 172.21.15.36:0/2714020965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39000b0d70 0x7f390014fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:45.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.717+0000 7f38fffff700 1 -- 172.21.15.36:0/2714020965 >> 172.21.15.36:0/2714020965 conn(0x7f39000b5d10 msgr2=0x7f390013aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:45.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.717+0000 7f38fffff700 1 -- 172.21.15.36:0/2714020965 shutdown_connections 2022-06-17T12:47:45.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.717+0000 7f38fffff700 1 -- 172.21.15.36:0/2714020965 wait complete. 2022-06-17T12:47:45.730 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' created 2022-06-17T12:47:45.732 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-06-17T12:47:45.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.865+0000 7fcd5e306700 1 Processor -- start 2022-06-17T12:47:45.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.865+0000 7fcd5e306700 1 -- start start 2022-06-17T12:47:45.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.869+0000 7fcd5e306700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcd5805fe80 0x7fcd58060260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:45.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.869+0000 7fcd5e306700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd580b1db0 0x7fcd581437d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:45.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.869+0000 7fcd5e306700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd580633c0 con 0x7fcd580ae360 2022-06-17T12:47:45.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.869+0000 7fcd5e306700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd58143330 con 0x7fcd5805fe80 2022-06-17T12:47:45.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.869+0000 7fcd5e306700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd58063100 con 0x7fcd580b1db0 2022-06-17T12:47:45.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.869+0000 7fcd5d304700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcd5805fe80 0x7fcd58060260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:45.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.869+0000 7fcd5db05700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd580b1db0 0x7fcd581437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:45.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.869+0000 7fcd5cb03700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcd580ae360 0x7fcd581444f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:45908/0 (socket says 172.21.15.36:45908) 2022-06-17T12:47:45.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.869+0000 7fcd5cb03700 1 -- 172.21.15.36:0/2827869295 learned_addr learned my addr 172.21.15.36:0/2827869295 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:45.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.869+0000 7fcd5db05700 1 -- 172.21.15.36:0/2827869295 >> v2:172.21.15.70:3300/0 conn(0x7fcd5805fe80 msgr2=0x7fcd58060260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:45.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.869+0000 7fcd5db05700 1 --2- 172.21.15.36:0/2827869295 >> v2:172.21.15.70:3300/0 conn(0x7fcd5805fe80 0x7fcd58060260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:45.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.869+0000 7fcd5db05700 1 -- 172.21.15.36:0/2827869295 >> v1:172.21.15.70:6789/0 conn(0x7fcd580ae360 legacy=0x7fcd581444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:45.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.869+0000 7fcd5db05700 1 -- 172.21.15.36:0/2827869295 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd58143020 con 0x7fcd580b1db0 2022-06-17T12:47:45.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.869+0000 7fcd5d304700 1 --2- 172.21.15.36:0/2827869295 >> v2:172.21.15.70:3300/0 conn(0x7fcd5805fe80 0x7fcd58060260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:45.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.869+0000 7fcd5db05700 1 --2- 172.21.15.36:0/2827869295 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd580b1db0 0x7fcd581437d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fcd500032f0 tx=0x7fcd500091d0).ready entity=mon.2 client_cookie=f7abfd631a64b7cd server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:45.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.869+0000 7fcd4bfff700 1 -- 172.21.15.36:0/2827869295 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd5000b6c0 con 0x7fcd580b1db0 2022-06-17T12:47:45.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.869+0000 7fcd4bfff700 1 -- 172.21.15.36:0/2827869295 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5000b820 con 0x7fcd580b1db0 2022-06-17T12:47:45.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.869+0000 7fcd4bfff700 1 -- 172.21.15.36:0/2827869295 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd5000b990 con 0x7fcd580b1db0 2022-06-17T12:47:45.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.873+0000 7fcd5e306700 1 -- 172.21.15.36:0/2827869295 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd580b1db0 msgr2=0x7fcd581437d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:45.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.873+0000 7fcd5e306700 1 --2- 172.21.15.36:0/2827869295 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd580b1db0 0x7fcd581437d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fcd500032f0 tx=0x7fcd500091d0).stop 2022-06-17T12:47:45.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.873+0000 7fcd5e306700 1 -- 172.21.15.36:0/2827869295 shutdown_connections 2022-06-17T12:47:45.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.873+0000 7fcd5e306700 1 --2- 172.21.15.36:0/2827869295 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd580b1db0 0x7fcd581437d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:45.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.873+0000 7fcd5e306700 1 --2- 172.21.15.36:0/2827869295 >> v2:172.21.15.70:3300/0 conn(0x7fcd5805fe80 0x7fcd58060260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:45.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.873+0000 7fcd5e306700 1 -- 172.21.15.36:0/2827869295 >> 172.21.15.36:0/2827869295 conn(0x7fcd58060990 msgr2=0x7fcd5805f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:45.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.873+0000 7fcd5e306700 1 -- 172.21.15.36:0/2827869295 shutdown_connections 2022-06-17T12:47:45.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.873+0000 7fcd5e306700 1 -- 172.21.15.36:0/2827869295 wait complete. 2022-06-17T12:47:45.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.873+0000 7fcd5e306700 1 Processor -- start 2022-06-17T12:47:45.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.873+0000 7fcd5e306700 1 -- start start 2022-06-17T12:47:45.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.873+0000 7fcd5e306700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd5805fe80 0x7fcd580664e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:45.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.873+0000 7fcd5e306700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd580ae360 0x7fcd580b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:45.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.873+0000 7fcd5e306700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd580b1db0 0x7fcd581528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:45.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.873+0000 7fcd5e306700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd58067130 con 0x7fcd580b1db0 2022-06-17T12:47:45.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.873+0000 7fcd5e306700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd58064de0 con 0x7fcd5805fe80 2022-06-17T12:47:45.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.873+0000 7fcd5e306700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd58065130 con 0x7fcd580ae360 2022-06-17T12:47:45.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.877+0000 7fcd5db05700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd580b1db0 0x7fcd581528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:45.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.877+0000 7fcd5d304700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd5805fe80 0x7fcd580664e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:45.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.877+0000 7fcd5db05700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd580b1db0 0x7fcd581528e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60884/0 (socket says 172.21.15.36:60884) 2022-06-17T12:47:45.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.877+0000 7fcd5db05700 1 -- 172.21.15.36:0/944541980 learned_addr learned my addr 172.21.15.36:0/944541980 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:45.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.877+0000 7fcd5cb03700 1 --2- 172.21.15.36:0/944541980 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd580ae360 0x7fcd580b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:45.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.877+0000 7fcd5db05700 1 -- 172.21.15.36:0/944541980 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd580ae360 msgr2=0x7fcd580b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:45.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.877+0000 7fcd5db05700 1 --2- 172.21.15.36:0/944541980 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd580ae360 0x7fcd580b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:45.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.877+0000 7fcd5db05700 1 -- 172.21.15.36:0/944541980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd5805fe80 msgr2=0x7fcd580664e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:45.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.877+0000 7fcd5db05700 1 --2- 172.21.15.36:0/944541980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd5805fe80 0x7fcd580664e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:45.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.877+0000 7fcd5db05700 1 -- 172.21.15.36:0/944541980 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5400b180 con 0x7fcd580b1db0 2022-06-17T12:47:45.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.877+0000 7fcd5db05700 1 --2- 172.21.15.36:0/944541980 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd580b1db0 0x7fcd581528e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7fcd500032f0 tx=0x7fcd50016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:45.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.877+0000 7fcd4a7fc700 1 -- 172.21.15.36:0/944541980 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd5000b990 con 0x7fcd580b1db0 2022-06-17T12:47:45.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.877+0000 7fcd4a7fc700 1 -- 172.21.15.36:0/944541980 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5000b030 con 0x7fcd580b1db0 2022-06-17T12:47:45.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.877+0000 7fcd5e306700 1 -- 172.21.15.36:0/944541980 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd50009470 con 0x7fcd580b1db0 2022-06-17T12:47:45.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.877+0000 7fcd4a7fc700 1 -- 172.21.15.36:0/944541980 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd5000b6c0 con 0x7fcd580b1db0 2022-06-17T12:47:45.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.877+0000 7fcd5e306700 1 -- 172.21.15.36:0/944541980 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd58065670 con 0x7fcd580b1db0 2022-06-17T12:47:45.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.877+0000 7fcd5e306700 1 -- 172.21.15.36:0/944541980 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd58065870 con 0x7fcd580b1db0 2022-06-17T12:47:45.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.881+0000 7fcd4a7fc700 1 -- 172.21.15.36:0/944541980 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fcd5001e070 con 0x7fcd580b1db0 2022-06-17T12:47:45.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.881+0000 7fcd4a7fc700 1 --2- 172.21.15.36:0/944541980 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcd38053ad0 0x7fcd38055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:45.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.881+0000 7fcd4a7fc700 1 -- 172.21.15.36:0/944541980 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5954+0+0 (secure 0 0 0) 0x7fcd5006ca90 con 0x7fcd580b1db0 2022-06-17T12:47:45.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.881+0000 7fcd5d304700 1 --2- 172.21.15.36:0/944541980 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcd38053ad0 0x7fcd38055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:45.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.881+0000 7fcd5d304700 1 --2- 172.21.15.36:0/944541980 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcd38053ad0 0x7fcd38055f90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fcd5400b000 tx=0x7fcd54006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:45.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:45.885+0000 7fcd4a7fc700 1 -- 172.21.15.36:0/944541980 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcd58065870 con 0x7fcd580b1db0 2022-06-17T12:47:46.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.013+0000 7fcd5e306700 1 -- 172.21.15.36:0/944541980 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fcd58065870 con 0x7fcd580b1db0 2022-06-17T12:47:46.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.717+0000 7fcd4a7fc700 1 -- 172.21.15.36:0/944541980 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v33) v1 ==== 125+0+0 (secure 0 0 0) 0x7fcd58065870 con 0x7fcd580b1db0 2022-06-17T12:47:46.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.729+0000 7fcd5e306700 1 -- 172.21.15.36:0/944541980 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcd38053ad0 msgr2=0x7fcd38055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:46.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.729+0000 7fcd5e306700 1 --2- 172.21.15.36:0/944541980 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcd38053ad0 0x7fcd38055f90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fcd5400b000 tx=0x7fcd54006df0).stop 2022-06-17T12:47:46.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.729+0000 7fcd5e306700 1 -- 172.21.15.36:0/944541980 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd580b1db0 msgr2=0x7fcd581528e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:46.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.729+0000 7fcd5e306700 1 --2- 172.21.15.36:0/944541980 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd580b1db0 0x7fcd581528e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7fcd500032f0 tx=0x7fcd50016000).stop 2022-06-17T12:47:46.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.729+0000 7fcd5e306700 1 -- 172.21.15.36:0/944541980 shutdown_connections 2022-06-17T12:47:46.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.729+0000 7fcd5e306700 1 --2- 172.21.15.36:0/944541980 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcd38053ad0 0x7fcd38055f90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:46.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.729+0000 7fcd5e306700 1 --2- 172.21.15.36:0/944541980 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd580b1db0 0x7fcd581528e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:46.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.729+0000 7fcd5e306700 1 --2- 172.21.15.36:0/944541980 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd580ae360 0x7fcd580b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:46.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.729+0000 7fcd5e306700 1 --2- 172.21.15.36:0/944541980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd5805fe80 0x7fcd580664e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:46.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.729+0000 7fcd5e306700 1 -- 172.21.15.36:0/944541980 >> 172.21.15.36:0/944541980 conn(0x7fcd58060990 msgr2=0x7fcd580b5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:46.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.729+0000 7fcd5e306700 1 -- 172.21.15.36:0/944541980 shutdown_connections 2022-06-17T12:47:46.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.729+0000 7fcd5e306700 1 -- 172.21.15.36:0/944541980 wait complete. 2022-06-17T12:47:46.754 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' created 2022-06-17T12:47:46.756 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-17T12:47:46.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.889+0000 7fbb97a54700 1 Processor -- start 2022-06-17T12:47:46.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.889+0000 7fbb97a54700 1 -- start start 2022-06-17T12:47:46.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.889+0000 7fbb97a54700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbb900ad9c0 0x7fbb900b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:46.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.889+0000 7fbb97a54700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbb9013fde0 0x7fbb90140200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:46.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.889+0000 7fbb97a54700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb900afc30 con 0x7fbb900ae270 2022-06-17T12:47:46.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.889+0000 7fbb97a54700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb900b0cb0 con 0x7fbb9013fde0 2022-06-17T12:47:46.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.889+0000 7fbb97a54700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb900b3eb0 con 0x7fbb900ad9c0 2022-06-17T12:47:46.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.889+0000 7fbb95ff1700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbb9013fde0 0x7fbb90140200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:46.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.889+0000 7fbb957f0700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fbb900ae270 0x7fbb900b05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50186/0 (socket says 172.21.15.36:50186) 2022-06-17T12:47:46.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.889+0000 7fbb957f0700 1 -- 172.21.15.36:0/958052462 learned_addr learned my addr 172.21.15.36:0/958052462 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:46.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.889+0000 7fbb94fef700 1 --2- 172.21.15.36:0/958052462 >> v2:172.21.15.70:3300/0 conn(0x7fbb900ad9c0 0x7fbb900b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:46.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.889+0000 7fbb95ff1700 1 -- 172.21.15.36:0/958052462 >> v2:172.21.15.70:3300/0 conn(0x7fbb900ad9c0 msgr2=0x7fbb900b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:46.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.889+0000 7fbb95ff1700 1 --2- 172.21.15.36:0/958052462 >> v2:172.21.15.70:3300/0 conn(0x7fbb900ad9c0 0x7fbb900b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:46.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.889+0000 7fbb95ff1700 1 -- 172.21.15.36:0/958052462 >> v1:172.21.15.36:6789/0 conn(0x7fbb900ae270 legacy=0x7fbb900b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:46.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.889+0000 7fbb95ff1700 1 -- 172.21.15.36:0/958052462 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb900ae650 con 0x7fbb9013fde0 2022-06-17T12:47:46.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.889+0000 7fbb94fef700 1 --2- 172.21.15.36:0/958052462 >> v2:172.21.15.70:3300/0 conn(0x7fbb900ad9c0 0x7fbb900b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:47:46.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.889+0000 7fbb7ffff700 1 -- 172.21.15.36:0/958052462 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 194850098 0 0) 0x7fbb900afc30 con 0x7fbb900ae270 2022-06-17T12:47:46.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.889+0000 7fbb95ff1700 1 --2- 172.21.15.36:0/958052462 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb9013fde0 0x7fbb90140200 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fbb900b3ff0 tx=0x7fbb8800b470).ready entity=mon.0 client_cookie=b9aae1399ca89468 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:46.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.893+0000 7fbb7ffff700 1 -- 172.21.15.36:0/958052462 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb8800ddb0 con 0x7fbb9013fde0 2022-06-17T12:47:46.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.893+0000 7fbb7ffff700 1 -- 172.21.15.36:0/958052462 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb88010470 con 0x7fbb9013fde0 2022-06-17T12:47:46.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.893+0000 7fbb7ffff700 1 -- 172.21.15.36:0/958052462 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb880107d0 con 0x7fbb9013fde0 2022-06-17T12:47:46.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.893+0000 7fbb97a54700 1 -- 172.21.15.36:0/958052462 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb9013fde0 msgr2=0x7fbb90140200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:46.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.893+0000 7fbb97a54700 1 --2- 172.21.15.36:0/958052462 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb9013fde0 0x7fbb90140200 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fbb900b3ff0 tx=0x7fbb8800b470).stop 2022-06-17T12:47:46.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.893+0000 7fbb97a54700 1 -- 172.21.15.36:0/958052462 shutdown_connections 2022-06-17T12:47:46.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.893+0000 7fbb97a54700 1 --2- 172.21.15.36:0/958052462 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb9013fde0 0x7fbb90140200 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:46.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.893+0000 7fbb97a54700 1 --2- 172.21.15.36:0/958052462 >> v2:172.21.15.70:3300/0 conn(0x7fbb900ad9c0 0x7fbb900b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:46.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.893+0000 7fbb97a54700 1 -- 172.21.15.36:0/958052462 >> 172.21.15.36:0/958052462 conn(0x7fbb9005c550 msgr2=0x7fbb9005c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:46.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.893+0000 7fbb97a54700 1 -- 172.21.15.36:0/958052462 shutdown_connections 2022-06-17T12:47:46.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.893+0000 7fbb97a54700 1 -- 172.21.15.36:0/958052462 wait complete. 2022-06-17T12:47:46.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.893+0000 7fbb97a54700 1 Processor -- start 2022-06-17T12:47:46.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.893+0000 7fbb97a54700 1 -- start start 2022-06-17T12:47:46.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.893+0000 7fbb97a54700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb900ad9c0 0x7fbb900653e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:46.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.893+0000 7fbb97a54700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb900ae270 0x7fbb9014de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:46.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.893+0000 7fbb97a54700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb9013fde0 0x7fbb901447d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:46.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.893+0000 7fbb97a54700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb90151f00 con 0x7fbb900ae270 2022-06-17T12:47:46.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.893+0000 7fbb97a54700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb90066780 con 0x7fbb900ad9c0 2022-06-17T12:47:46.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.897+0000 7fbb97a54700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb901521a0 con 0x7fbb9013fde0 2022-06-17T12:47:46.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.897+0000 7fbb94fef700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb900ae270 0x7fbb9014de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:46.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.897+0000 7fbb94fef700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb900ae270 0x7fbb9014de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60896/0 (socket says 172.21.15.36:60896) 2022-06-17T12:47:46.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.897+0000 7fbb94fef700 1 -- 172.21.15.36:0/3468674901 learned_addr learned my addr 172.21.15.36:0/3468674901 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:46.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.897+0000 7fbb957f0700 1 --2- 172.21.15.36:0/3468674901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb900ad9c0 0x7fbb900653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:46.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.897+0000 7fbb95ff1700 1 --2- 172.21.15.36:0/3468674901 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb9013fde0 0x7fbb901447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:46.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.897+0000 7fbb94fef700 1 -- 172.21.15.36:0/3468674901 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb9013fde0 msgr2=0x7fbb901447d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:46.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.897+0000 7fbb94fef700 1 --2- 172.21.15.36:0/3468674901 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb9013fde0 0x7fbb901447d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:46.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.897+0000 7fbb94fef700 1 -- 172.21.15.36:0/3468674901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb900ad9c0 msgr2=0x7fbb900653e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:46.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.897+0000 7fbb94fef700 1 --2- 172.21.15.36:0/3468674901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb900ad9c0 0x7fbb900653e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:46.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.897+0000 7fbb94fef700 1 -- 172.21.15.36:0/3468674901 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8800b710 con 0x7fbb900ae270 2022-06-17T12:47:46.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.897+0000 7fbb957f0700 1 --2- 172.21.15.36:0/3468674901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb900ad9c0 0x7fbb900653e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:46.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.897+0000 7fbb94fef700 1 --2- 172.21.15.36:0/3468674901 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb900ae270 0x7fbb9014de00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fbb80007000 tx=0x7fbb80008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:46.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.897+0000 7fbb7e7fc700 1 -- 172.21.15.36:0/3468674901 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb80010580 con 0x7fbb900ae270 2022-06-17T12:47:46.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.897+0000 7fbb7e7fc700 1 -- 172.21.15.36:0/3468674901 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb800106e0 con 0x7fbb900ae270 2022-06-17T12:47:46.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.897+0000 7fbb97a54700 1 -- 172.21.15.36:0/3468674901 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb900660e0 con 0x7fbb900ae270 2022-06-17T12:47:46.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.897+0000 7fbb7e7fc700 1 -- 172.21.15.36:0/3468674901 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb80010a40 con 0x7fbb900ae270 2022-06-17T12:47:46.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.897+0000 7fbb97a54700 1 -- 172.21.15.36:0/3468674901 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb90066c20 con 0x7fbb900ae270 2022-06-17T12:47:46.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.897+0000 7fbb7e7fc700 1 -- 172.21.15.36:0/3468674901 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fbb80010ba0 con 0x7fbb900ae270 2022-06-17T12:47:46.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.901+0000 7fbb97a54700 1 -- 172.21.15.36:0/3468674901 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb90066940 con 0x7fbb900ae270 2022-06-17T12:47:46.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.901+0000 7fbb7e7fc700 1 --2- 172.21.15.36:0/3468674901 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb700539d0 0x7fbb70055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:46.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.901+0000 7fbb7e7fc700 1 -- 172.21.15.36:0/3468674901 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fbb8006bfe0 con 0x7fbb900ae270 2022-06-17T12:47:46.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.901+0000 7fbb957f0700 1 --2- 172.21.15.36:0/3468674901 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb700539d0 0x7fbb70055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:46.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.901+0000 7fbb957f0700 1 --2- 172.21.15.36:0/3468674901 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb700539d0 0x7fbb70055e90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fbb8c00d000 tx=0x7fbb8c007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:46.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:46.905+0000 7fbb7e7fc700 1 -- 172.21.15.36:0/3468674901 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbb80042e60 con 0x7fbb900ae270 2022-06-17T12:47:47.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.069+0000 7fbb97a54700 1 -- 172.21.15.36:0/3468674901 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fbb900afe60 con 0x7fbb900ae270 2022-06-17T12:47:47.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.077+0000 7fbb7e7fc700 1 -- 172.21.15.36:0/3468674901 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 5 and data pool 6 v11) v1 ==== 187+0+0 (secure 0 0 0) 0x7fbb80046480 con 0x7fbb900ae270 2022-06-17T12:47:47.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.081+0000 7fbb97a54700 1 -- 172.21.15.36:0/3468674901 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb700539d0 msgr2=0x7fbb70055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:47.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.081+0000 7fbb97a54700 1 --2- 172.21.15.36:0/3468674901 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb700539d0 0x7fbb70055e90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fbb8c00d000 tx=0x7fbb8c007a00).stop 2022-06-17T12:47:47.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.081+0000 7fbb97a54700 1 -- 172.21.15.36:0/3468674901 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb900ae270 msgr2=0x7fbb9014de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:47.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.081+0000 7fbb97a54700 1 --2- 172.21.15.36:0/3468674901 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb900ae270 0x7fbb9014de00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fbb80007000 tx=0x7fbb80008040).stop 2022-06-17T12:47:47.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.085+0000 7fbb97a54700 1 -- 172.21.15.36:0/3468674901 shutdown_connections 2022-06-17T12:47:47.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.085+0000 7fbb97a54700 1 --2- 172.21.15.36:0/3468674901 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb700539d0 0x7fbb70055e90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:47.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.085+0000 7fbb97a54700 1 --2- 172.21.15.36:0/3468674901 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb9013fde0 0x7fbb901447d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:47.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.085+0000 7fbb97a54700 1 --2- 172.21.15.36:0/3468674901 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb900ae270 0x7fbb9014de00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:47.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.085+0000 7fbb97a54700 1 --2- 172.21.15.36:0/3468674901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb900ad9c0 0x7fbb900653e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:47.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.085+0000 7fbb97a54700 1 -- 172.21.15.36:0/3468674901 >> 172.21.15.36:0/3468674901 conn(0x7fbb9005c550 msgr2=0x7fbb9005c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:47.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.089+0000 7fbb97a54700 1 -- 172.21.15.36:0/3468674901 shutdown_connections 2022-06-17T12:47:47.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.089+0000 7fbb97a54700 1 -- 172.21.15.36:0/3468674901 wait complete. 2022-06-17T12:47:47.281 INFO:teuthology.orchestra.run.smithi036.stderr:new fs with metadata pool 5 and data pool 6 2022-06-17T12:47:47.281 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:47:47.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.445+0000 7f22b760d700 1 Processor -- start 2022-06-17T12:47:47.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.445+0000 7f22b760d700 1 -- start start 2022-06-17T12:47:47.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.449+0000 7f22b760d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f22b0062bd0 0x7f22b00b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:47.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.449+0000 7f22b760d700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b0065090 con 0x7f22b0140e30 2022-06-17T12:47:47.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.449+0000 7f22b760d700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b00b1bc0 con 0x7f22b00b12f0 2022-06-17T12:47:47.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.449+0000 7f22b760d700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b0063510 con 0x7f22b0062bd0 2022-06-17T12:47:47.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.449+0000 7f22b5baa700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f22b0140e30 0x7f22b0141280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50216/0 (socket says 172.21.15.36:50216) 2022-06-17T12:47:47.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.449+0000 7f22b5baa700 1 -- 172.21.15.36:0/3641332659 learned_addr learned my addr 172.21.15.36:0/3641332659 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:47.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.449+0000 7f22b4ba8700 1 --2- 172.21.15.36:0/3641332659 >> v2:172.21.15.70:3300/0 conn(0x7f22b0062bd0 0x7f22b00b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:47.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.449+0000 7f229bfff700 1 -- 172.21.15.36:0/3641332659 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2696913179 0 0) 0x7f22b0065090 con 0x7f22b0140e30 2022-06-17T12:47:47.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.449+0000 7f229bfff700 1 -- 172.21.15.36:0/3641332659 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2294003260 con 0x7f22b0140e30 2022-06-17T12:47:47.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.449+0000 7f229bfff700 1 -- 172.21.15.36:0/3641332659 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1461442884 0 0) 0x7f2294003260 con 0x7f22b0140e30 2022-06-17T12:47:47.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.449+0000 7f22b4ba8700 1 -- 172.21.15.36:0/3641332659 >> v1:172.21.15.70:6789/0 conn(0x7f22b00b12f0 legacy=0x7f22b013aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:47.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.449+0000 7f22b4ba8700 1 -- 172.21.15.36:0/3641332659 >> v1:172.21.15.36:6789/0 conn(0x7f22b0140e30 legacy=0x7f22b0141280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:47.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.449+0000 7f22b4ba8700 1 -- 172.21.15.36:0/3641332659 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b00b0b40 con 0x7f22b0062bd0 2022-06-17T12:47:47.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.449+0000 7f22b4ba8700 1 --2- 172.21.15.36:0/3641332659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22b0062bd0 0x7f22b00b16d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f22b0063060 tx=0x7f22ac006df0).ready entity=mon.1 client_cookie=28e3b528578a19f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:47.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.449+0000 7f229bfff700 1 -- 172.21.15.36:0/3641332659 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22a80019e0 con 0x7f22b0062bd0 2022-06-17T12:47:47.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.449+0000 7f229bfff700 1 -- 172.21.15.36:0/3641332659 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ac000e90 con 0x7f22b0062bd0 2022-06-17T12:47:47.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.449+0000 7f229bfff700 1 -- 172.21.15.36:0/3641332659 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22ac00d4f0 con 0x7f22b0062bd0 2022-06-17T12:47:47.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.449+0000 7f22b760d700 1 -- 172.21.15.36:0/3641332659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22b0062bd0 msgr2=0x7f22b00b16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:47.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.449+0000 7f22b760d700 1 --2- 172.21.15.36:0/3641332659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22b0062bd0 0x7f22b00b16d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f22b0063060 tx=0x7f22ac006df0).stop 2022-06-17T12:47:47.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.449+0000 7f22b760d700 1 -- 172.21.15.36:0/3641332659 shutdown_connections 2022-06-17T12:47:47.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.449+0000 7f22b760d700 1 --2- 172.21.15.36:0/3641332659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22b0062bd0 0x7f22b00b16d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:47.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.449+0000 7f22b760d700 1 -- 172.21.15.36:0/3641332659 >> 172.21.15.36:0/3641332659 conn(0x7f22b00b53e0 msgr2=0x7f22b00b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:47.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.449+0000 7f22b760d700 1 -- 172.21.15.36:0/3641332659 shutdown_connections 2022-06-17T12:47:47.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.449+0000 7f22b760d700 1 -- 172.21.15.36:0/3641332659 wait complete. 2022-06-17T12:47:47.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22b760d700 1 Processor -- start 2022-06-17T12:47:47.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22b760d700 1 -- start start 2022-06-17T12:47:47.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22b760d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22b0062bd0 0x7f22b0111910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:47.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22b760d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22b00b12f0 0x7f22b013ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:47.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22b760d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22b0140e30 0x7f22b01507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:47.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22b760d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b013b710 con 0x7f22b0140e30 2022-06-17T12:47:47.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22b760d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b0144720 con 0x7f22b00b12f0 2022-06-17T12:47:47.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22b760d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b01449c0 con 0x7f22b0062bd0 2022-06-17T12:47:47.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22b5baa700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22b0140e30 0x7f22b01507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:47.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22b5baa700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22b0140e30 0x7f22b01507f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60924/0 (socket says 172.21.15.36:60924) 2022-06-17T12:47:47.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22b5baa700 1 -- 172.21.15.36:0/2052410672 learned_addr learned my addr 172.21.15.36:0/2052410672 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:47.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22b4ba8700 1 --2- 172.21.15.36:0/2052410672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22b00b12f0 0x7f22b013ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:47.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22b53a9700 1 --2- 172.21.15.36:0/2052410672 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22b0062bd0 0x7f22b0111910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:47.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22b5baa700 1 -- 172.21.15.36:0/2052410672 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22b0062bd0 msgr2=0x7f22b0111910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:47.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22b5baa700 1 --2- 172.21.15.36:0/2052410672 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22b0062bd0 0x7f22b0111910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:47.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22b5baa700 1 -- 172.21.15.36:0/2052410672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22b00b12f0 msgr2=0x7f22b013ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:47.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22b5baa700 1 --2- 172.21.15.36:0/2052410672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22b00b12f0 0x7f22b013ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:47.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22b5baa700 1 -- 172.21.15.36:0/2052410672 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ac00b180 con 0x7f22b0140e30 2022-06-17T12:47:47.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22b4ba8700 1 --2- 172.21.15.36:0/2052410672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22b00b12f0 0x7f22b013ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:47.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22b5baa700 1 --2- 172.21.15.36:0/2052410672 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22b0140e30 0x7f22b01507f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f22a8001c40 tx=0x7f22a8002160).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:47.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22a2ffd700 1 -- 172.21.15.36:0/2052410672 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22a8004ab0 con 0x7f22b0140e30 2022-06-17T12:47:47.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22a2ffd700 1 -- 172.21.15.36:0/2052410672 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a8004c10 con 0x7f22b0140e30 2022-06-17T12:47:47.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22b760d700 1 -- 172.21.15.36:0/2052410672 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b0143950 con 0x7f22b0140e30 2022-06-17T12:47:47.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22a2ffd700 1 -- 172.21.15.36:0/2052410672 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22a8007170 con 0x7f22b0140e30 2022-06-17T12:47:47.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.453+0000 7f22b760d700 1 -- 172.21.15.36:0/2052410672 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b0144360 con 0x7f22b0140e30 2022-06-17T12:47:47.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.457+0000 7f22b760d700 1 -- 172.21.15.36:0/2052410672 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b0153900 con 0x7f22b0140e30 2022-06-17T12:47:47.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.457+0000 7f22a2ffd700 1 -- 172.21.15.36:0/2052410672 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f22a8010070 con 0x7f22b0140e30 2022-06-17T12:47:47.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.457+0000 7f22a2ffd700 1 --2- 172.21.15.36:0/2052410672 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f228c053a60 0x7f228c055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:47.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.457+0000 7f22a2ffd700 1 -- 172.21.15.36:0/2052410672 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f22a806bc80 con 0x7f22b0140e30 2022-06-17T12:47:47.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.457+0000 7f22b53a9700 1 --2- 172.21.15.36:0/2052410672 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f228c053a60 0x7f228c055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:47.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.457+0000 7f22b53a9700 1 --2- 172.21.15.36:0/2052410672 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f228c053a60 0x7f228c055f20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f22a400b000 tx=0x7f22a4009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:47.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.465+0000 7f22a2ffd700 1 -- 172.21.15.36:0/2052410672 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f22a80429e0 con 0x7f22b0140e30 2022-06-17T12:47:47.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.589+0000 7f22b760d700 1 -- 172.21.15.36:0/2052410672 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22b00b03c0 con 0x7f22b0140e30 2022-06-17T12:47:47.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.593+0000 7f22a2ffd700 1 -- 172.21.15.36:0/2052410672 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16284 (secure 0 0 0) 0x7f22a8046000 con 0x7f22b0140e30 2022-06-17T12:47:47.593 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:47.619 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":34,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:47:47.071975+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:47:45.085233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2022-06-17T12:47:46.070152+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:47:47.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.593+0000 7f22b760d700 1 -- 172.21.15.36:0/2052410672 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f228c053a60 msgr2=0x7f228c055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:47.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.593+0000 7f22b760d700 1 --2- 172.21.15.36:0/2052410672 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f228c053a60 0x7f228c055f20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f22a400b000 tx=0x7f22a4009040).stop 2022-06-17T12:47:47.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.593+0000 7f22b760d700 1 -- 172.21.15.36:0/2052410672 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22b0140e30 msgr2=0x7f22b01507f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:47.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.593+0000 7f22b760d700 1 --2- 172.21.15.36:0/2052410672 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22b0140e30 0x7f22b01507f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f22a8001c40 tx=0x7f22a8002160).stop 2022-06-17T12:47:47.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.593+0000 7f22b760d700 1 -- 172.21.15.36:0/2052410672 shutdown_connections 2022-06-17T12:47:47.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.593+0000 7f22b760d700 1 --2- 172.21.15.36:0/2052410672 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f228c053a60 0x7f228c055f20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:47.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.593+0000 7f22b760d700 1 --2- 172.21.15.36:0/2052410672 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22b0140e30 0x7f22b01507f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:47.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.593+0000 7f22b760d700 1 --2- 172.21.15.36:0/2052410672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22b00b12f0 0x7f22b013ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:47.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.593+0000 7f22b760d700 1 --2- 172.21.15.36:0/2052410672 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22b0062bd0 0x7f22b0111910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:47.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.593+0000 7f22b760d700 1 -- 172.21.15.36:0/2052410672 >> 172.21.15.36:0/2052410672 conn(0x7f22b00b53e0 msgr2=0x7f22b013bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:47.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.593+0000 7f22b760d700 1 -- 172.21.15.36:0/2052410672 shutdown_connections 2022-06-17T12:47:47.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.593+0000 7f22b760d700 1 -- 172.21.15.36:0/2052410672 wait complete. 2022-06-17T12:47:47.703 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:47:48.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.845+0000 7f32b8bbe700 1 Processor -- start 2022-06-17T12:47:48.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.845+0000 7f32b8bbe700 1 -- start start 2022-06-17T12:47:48.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.845+0000 7f32b8bbe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f32b40ae270 0x7f32b40b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:48.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.845+0000 7f32b8bbe700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b40afc30 con 0x7f32b40ad9c0 2022-06-17T12:47:48.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.845+0000 7f32b8bbe700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b40ae650 con 0x7f32b413fde0 2022-06-17T12:47:48.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.845+0000 7f32b8bbe700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b40b0cb0 con 0x7f32b40ae270 2022-06-17T12:47:48.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.845+0000 7f32b259c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f32b40ae270 0x7f32b40b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:48.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.845+0000 7f32b1d9b700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f32b40ad9c0 0x7f32b40b05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50226/0 (socket says 172.21.15.36:50226) 2022-06-17T12:47:48.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.845+0000 7f32b1d9b700 1 -- 172.21.15.36:0/201289517 learned_addr learned my addr 172.21.15.36:0/201289517 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:48.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.845+0000 7f32b259c700 1 -- 172.21.15.36:0/201289517 >> v1:172.21.15.70:6789/0 conn(0x7f32b413fde0 legacy=0x7f32b413edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:48.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.845+0000 7f32b259c700 1 -- 172.21.15.36:0/201289517 >> v1:172.21.15.36:6789/0 conn(0x7f32b40ad9c0 legacy=0x7f32b40b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:48.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.845+0000 7f32b259c700 1 -- 172.21.15.36:0/201289517 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b40afe60 con 0x7f32b40ae270 2022-06-17T12:47:48.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.845+0000 7f32b259c700 1 --2- 172.21.15.36:0/201289517 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f32b40ae270 0x7f32b40b39c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f32a4002f70 tx=0x7f32a4006df0).ready entity=mon.0 client_cookie=f8bec99308504ec7 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:48.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.845+0000 7f32b159a700 1 -- 172.21.15.36:0/201289517 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32a400fa10 con 0x7f32b40ae270 2022-06-17T12:47:48.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.845+0000 7f32b159a700 1 -- 172.21.15.36:0/201289517 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a4000e90 con 0x7f32b40ae270 2022-06-17T12:47:48.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.845+0000 7f32b159a700 1 -- 172.21.15.36:0/201289517 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32a400fd60 con 0x7f32b40ae270 2022-06-17T12:47:48.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.845+0000 7f32b8bbe700 1 -- 172.21.15.36:0/201289517 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f32b40ae270 msgr2=0x7f32b40b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:48.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.845+0000 7f32b8bbe700 1 --2- 172.21.15.36:0/201289517 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f32b40ae270 0x7f32b40b39c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f32a4002f70 tx=0x7f32a4006df0).stop 2022-06-17T12:47:48.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.845+0000 7f32b8bbe700 1 -- 172.21.15.36:0/201289517 shutdown_connections 2022-06-17T12:47:48.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.845+0000 7f32b8bbe700 1 --2- 172.21.15.36:0/201289517 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f32b40ae270 0x7f32b40b39c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:48.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.845+0000 7f32b8bbe700 1 -- 172.21.15.36:0/201289517 >> 172.21.15.36:0/201289517 conn(0x7f32b405c550 msgr2=0x7f32b405c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:48.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.845+0000 7f32b8bbe700 1 -- 172.21.15.36:0/201289517 shutdown_connections 2022-06-17T12:47:48.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.845+0000 7f32b8bbe700 1 -- 172.21.15.36:0/201289517 wait complete. 2022-06-17T12:47:48.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.849+0000 7f32b8bbe700 1 Processor -- start 2022-06-17T12:47:48.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.849+0000 7f32b8bbe700 1 -- start start 2022-06-17T12:47:48.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.849+0000 7f32b8bbe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32b40ad9c0 0x7f32b4140310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:48.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.849+0000 7f32b8bbe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32b40ae270 0x7f32b413ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:48.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.849+0000 7f32b8bbe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f32b413fde0 0x7f32b413d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:48.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.849+0000 7f32b8bbe700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b4066ca0 con 0x7f32b413fde0 2022-06-17T12:47:48.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.849+0000 7f32b8bbe700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b414e270 con 0x7f32b40ad9c0 2022-06-17T12:47:48.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.849+0000 7f32b8bbe700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b4110540 con 0x7f32b40ae270 2022-06-17T12:47:48.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.849+0000 7f32b2d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f32b413fde0 0x7f32b413d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:48.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.849+0000 7f32b2d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f32b413fde0 0x7f32b413d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60938/0 (socket says 172.21.15.36:60938) 2022-06-17T12:47:48.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.849+0000 7f32b2d9d700 1 -- 172.21.15.36:0/208767062 learned_addr learned my addr 172.21.15.36:0/208767062 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:48.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.849+0000 7f32b259c700 1 --2- 172.21.15.36:0/208767062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32b40ad9c0 0x7f32b4140310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:48.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.849+0000 7f32b1d9b700 1 --2- 172.21.15.36:0/208767062 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32b40ae270 0x7f32b413ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:48.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.849+0000 7f32b2d9d700 1 -- 172.21.15.36:0/208767062 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32b40ae270 msgr2=0x7f32b413ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:48.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.849+0000 7f32b2d9d700 1 --2- 172.21.15.36:0/208767062 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32b40ae270 0x7f32b413ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:48.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.849+0000 7f32b2d9d700 1 -- 172.21.15.36:0/208767062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32b40ad9c0 msgr2=0x7f32b4140310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:48.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.849+0000 7f32b2d9d700 1 --2- 172.21.15.36:0/208767062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32b40ad9c0 0x7f32b4140310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:48.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.849+0000 7f32b2d9d700 1 -- 172.21.15.36:0/208767062 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a400b180 con 0x7f32b413fde0 2022-06-17T12:47:48.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.849+0000 7f32b259c700 1 --2- 172.21.15.36:0/208767062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32b40ad9c0 0x7f32b4140310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:48.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.849+0000 7f32b2d9d700 1 --2- 172.21.15.36:0/208767062 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f32b413fde0 0x7f32b413d2f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f32b4062470 tx=0x7f32ac0013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:48.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.853+0000 7f32a37fe700 1 -- 172.21.15.36:0/208767062 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32ac006aa0 con 0x7f32b413fde0 2022-06-17T12:47:48.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.853+0000 7f32a37fe700 1 -- 172.21.15.36:0/208767062 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ac006c00 con 0x7f32b413fde0 2022-06-17T12:47:48.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.853+0000 7f32b8bbe700 1 -- 172.21.15.36:0/208767062 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b4110770 con 0x7f32b413fde0 2022-06-17T12:47:48.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.853+0000 7f32a37fe700 1 -- 172.21.15.36:0/208767062 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32ac007030 con 0x7f32b413fde0 2022-06-17T12:47:48.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.853+0000 7f32b8bbe700 1 -- 172.21.15.36:0/208767062 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b4065540 con 0x7f32b413fde0 2022-06-17T12:47:48.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.853+0000 7f32b8bbe700 1 -- 172.21.15.36:0/208767062 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b4110900 con 0x7f32b413fde0 2022-06-17T12:47:48.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.853+0000 7f32a37fe700 1 -- 172.21.15.36:0/208767062 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f32ac010030 con 0x7f32b413fde0 2022-06-17T12:47:48.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.853+0000 7f32a37fe700 1 --2- 172.21.15.36:0/208767062 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3290053a40 0x7f3290055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:48.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.853+0000 7f32a37fe700 1 -- 172.21.15.36:0/208767062 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f32ac06c020 con 0x7f32b413fde0 2022-06-17T12:47:48.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.853+0000 7f32b259c700 1 --2- 172.21.15.36:0/208767062 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3290053a40 0x7f3290055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:48.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.853+0000 7f32b259c700 1 --2- 172.21.15.36:0/208767062 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3290053a40 0x7f3290055f00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f32a401c000 tx=0x7f32a4017000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:48.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.861+0000 7f32a37fe700 1 -- 172.21.15.36:0/208767062 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f32ac042d20 con 0x7f32b413fde0 2022-06-17T12:47:48.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.993+0000 7f32b8bbe700 1 -- 172.21.15.36:0/208767062 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32b40b0100 con 0x7f32b413fde0 2022-06-17T12:47:48.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.993+0000 7f32a37fe700 1 -- 172.21.15.36:0/208767062 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16284 (secure 0 0 0) 0x7f32ac046340 con 0x7f32b413fde0 2022-06-17T12:47:48.117 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:48.118 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":34,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:47:47.071975+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:47:45.085233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2022-06-17T12:47:46.070152+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:47:48.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.997+0000 7f32b8bbe700 1 -- 172.21.15.36:0/208767062 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3290053a40 msgr2=0x7f3290055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:48.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.997+0000 7f32b8bbe700 1 --2- 172.21.15.36:0/208767062 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3290053a40 0x7f3290055f00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f32a401c000 tx=0x7f32a4017000).stop 2022-06-17T12:47:48.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.997+0000 7f32b8bbe700 1 -- 172.21.15.36:0/208767062 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f32b413fde0 msgr2=0x7f32b413d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:48.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.997+0000 7f32b8bbe700 1 --2- 172.21.15.36:0/208767062 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f32b413fde0 0x7f32b413d2f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f32b4062470 tx=0x7f32ac0013a0).stop 2022-06-17T12:47:48.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.997+0000 7f32b8bbe700 1 -- 172.21.15.36:0/208767062 shutdown_connections 2022-06-17T12:47:48.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.997+0000 7f32b8bbe700 1 --2- 172.21.15.36:0/208767062 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3290053a40 0x7f3290055f00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:48.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.997+0000 7f32b8bbe700 1 --2- 172.21.15.36:0/208767062 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f32b413fde0 0x7f32b413d2f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:48.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.997+0000 7f32b8bbe700 1 --2- 172.21.15.36:0/208767062 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32b40ae270 0x7f32b413ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:48.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.997+0000 7f32b8bbe700 1 --2- 172.21.15.36:0/208767062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32b40ad9c0 0x7f32b4140310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:48.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.997+0000 7f32b8bbe700 1 -- 172.21.15.36:0/208767062 >> 172.21.15.36:0/208767062 conn(0x7f32b405c550 msgr2=0x7f32b4144ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:48.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.997+0000 7f32b8bbe700 1 -- 172.21.15.36:0/208767062 shutdown_connections 2022-06-17T12:47:48.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:47.997+0000 7f32b8bbe700 1 -- 172.21.15.36:0/208767062 wait complete. 2022-06-17T12:47:48.128 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-17T12:47:48.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.269+0000 7fd0491a4700 1 Processor -- start 2022-06-17T12:47:48.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.269+0000 7fd0491a4700 1 -- start start 2022-06-17T12:47:48.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.273+0000 7fd0491a4700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd0440b15b0 0x7fd0440b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:48.366 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.273+0000 7fd0491a4700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04413fdf0 con 0x7fd0440adf10 2022-06-17T12:47:48.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.273+0000 7fd0491a4700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd044140090 con 0x7fd0440b65f0 2022-06-17T12:47:48.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.273+0000 7fd0491a4700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0440ae8e0 con 0x7fd0440b15b0 2022-06-17T12:47:48.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.273+0000 7fd03b7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fd0440adf10 0x7fd0440af630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50242/0 (socket says 172.21.15.36:50242) 2022-06-17T12:47:48.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.273+0000 7fd03b7fe700 1 -- 172.21.15.36:0/37066561 learned_addr learned my addr 172.21.15.36:0/37066561 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:48.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.273+0000 7fd0489a3700 1 --2- 172.21.15.36:0/37066561 >> v2:172.21.15.70:3300/0 conn(0x7fd0440b15b0 0x7fd0440b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:48.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.273+0000 7fd0437fe700 1 -- 172.21.15.36:0/37066561 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3528668235 0 0) 0x7fd044140090 con 0x7fd0440b65f0 2022-06-17T12:47:48.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.273+0000 7fd0437fe700 1 -- 172.21.15.36:0/37066561 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd024003260 con 0x7fd0440b65f0 2022-06-17T12:47:48.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.273+0000 7fd0437fe700 1 -- 172.21.15.36:0/37066561 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4134974914 0 0) 0x7fd04413fdf0 con 0x7fd0440adf10 2022-06-17T12:47:48.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.273+0000 7fd0437fe700 1 -- 172.21.15.36:0/37066561 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd044140090 con 0x7fd0440adf10 2022-06-17T12:47:48.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.273+0000 7fd0489a3700 1 -- 172.21.15.36:0/37066561 >> v1:172.21.15.70:6789/0 conn(0x7fd0440b65f0 legacy=0x7fd044140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:48.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.273+0000 7fd0489a3700 1 -- 172.21.15.36:0/37066561 >> v1:172.21.15.36:6789/0 conn(0x7fd0440adf10 legacy=0x7fd0440af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:48.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.273+0000 7fd0489a3700 1 -- 172.21.15.36:0/37066561 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0440b1be0 con 0x7fd0440b15b0 2022-06-17T12:47:48.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.273+0000 7fd0489a3700 1 --2- 172.21.15.36:0/37066561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0440b15b0 0x7fd0440b69d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fd03c0032f0 tx=0x7fd03c0091d0).ready entity=mon.1 client_cookie=29002d4371374d5e server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:48.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.273+0000 7fd0437fe700 1 -- 172.21.15.36:0/37066561 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd03c00b600 con 0x7fd0440b15b0 2022-06-17T12:47:48.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.273+0000 7fd0437fe700 1 -- 172.21.15.36:0/37066561 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03c00b760 con 0x7fd0440b15b0 2022-06-17T12:47:48.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.273+0000 7fd0437fe700 1 -- 172.21.15.36:0/37066561 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd03c00b8d0 con 0x7fd0440b15b0 2022-06-17T12:47:48.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.273+0000 7fd0491a4700 1 -- 172.21.15.36:0/37066561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0440b15b0 msgr2=0x7fd0440b69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:48.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.273+0000 7fd0491a4700 1 --2- 172.21.15.36:0/37066561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0440b15b0 0x7fd0440b69d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fd03c0032f0 tx=0x7fd03c0091d0).stop 2022-06-17T12:47:48.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.273+0000 7fd0491a4700 1 -- 172.21.15.36:0/37066561 shutdown_connections 2022-06-17T12:47:48.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.273+0000 7fd0491a4700 1 --2- 172.21.15.36:0/37066561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0440b15b0 0x7fd0440b69d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:48.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.273+0000 7fd0491a4700 1 -- 172.21.15.36:0/37066561 >> 172.21.15.36:0/37066561 conn(0x7fd0440614b0 msgr2=0x7fd044061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:48.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.273+0000 7fd0491a4700 1 -- 172.21.15.36:0/37066561 shutdown_connections 2022-06-17T12:47:48.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.277+0000 7fd0491a4700 1 -- 172.21.15.36:0/37066561 wait complete. 2022-06-17T12:47:48.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.277+0000 7fd0491a4700 1 Processor -- start 2022-06-17T12:47:48.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.277+0000 7fd0491a4700 1 -- start start 2022-06-17T12:47:48.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.277+0000 7fd0491a4700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0440adf10 0x7fd044152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:48.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.277+0000 7fd0491a4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0440b15b0 0x7fd0440b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:48.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.277+0000 7fd0491a4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0440b65f0 0x7fd04413cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:48.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.277+0000 7fd0491a4700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd044153100 con 0x7fd0440b65f0 2022-06-17T12:47:48.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.277+0000 7fd0491a4700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd044065250 con 0x7fd0440b15b0 2022-06-17T12:47:48.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.277+0000 7fd0491a4700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd044064d10 con 0x7fd0440adf10 2022-06-17T12:47:48.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.277+0000 7fd0489a3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0440b65f0 0x7fd04413cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:48.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.277+0000 7fd0489a3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0440b65f0 0x7fd04413cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60952/0 (socket says 172.21.15.36:60952) 2022-06-17T12:47:48.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.277+0000 7fd0489a3700 1 -- 172.21.15.36:0/501297686 learned_addr learned my addr 172.21.15.36:0/501297686 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:48.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.277+0000 7fd03b7fe700 1 --2- 172.21.15.36:0/501297686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0440b15b0 0x7fd0440b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:48.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.277+0000 7fd043fff700 1 --2- 172.21.15.36:0/501297686 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0440adf10 0x7fd044152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:48.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.281+0000 7fd0489a3700 1 -- 172.21.15.36:0/501297686 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0440adf10 msgr2=0x7fd044152c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:48.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.281+0000 7fd0489a3700 1 --2- 172.21.15.36:0/501297686 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0440adf10 0x7fd044152c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:48.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.281+0000 7fd0489a3700 1 -- 172.21.15.36:0/501297686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0440b15b0 msgr2=0x7fd0440b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:48.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.281+0000 7fd0489a3700 1 --2- 172.21.15.36:0/501297686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0440b15b0 0x7fd0440b4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:48.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.281+0000 7fd0489a3700 1 -- 172.21.15.36:0/501297686 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03c009470 con 0x7fd0440b65f0 2022-06-17T12:47:48.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.281+0000 7fd0489a3700 1 --2- 172.21.15.36:0/501297686 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0440b65f0 0x7fd04413cbf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fd03c0032f0 tx=0x7fd03c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:48.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.281+0000 7fd041ffb700 1 -- 172.21.15.36:0/501297686 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd03c00b600 con 0x7fd0440b65f0 2022-06-17T12:47:48.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.281+0000 7fd041ffb700 1 -- 172.21.15.36:0/501297686 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03c011e60 con 0x7fd0440b65f0 2022-06-17T12:47:48.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.281+0000 7fd0491a4700 1 -- 172.21.15.36:0/501297686 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd044064fb0 con 0x7fd0440b65f0 2022-06-17T12:47:48.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.281+0000 7fd041ffb700 1 -- 172.21.15.36:0/501297686 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd03c01f5e0 con 0x7fd0440b65f0 2022-06-17T12:47:48.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.281+0000 7fd0491a4700 1 -- 172.21.15.36:0/501297686 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0440647d0 con 0x7fd0440b65f0 2022-06-17T12:47:48.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.281+0000 7fd0491a4700 1 -- 172.21.15.36:0/501297686 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd04413fae0 con 0x7fd0440b65f0 2022-06-17T12:47:48.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.281+0000 7fd041ffb700 1 -- 172.21.15.36:0/501297686 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd03c026030 con 0x7fd0440b65f0 2022-06-17T12:47:48.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.281+0000 7fd041ffb700 1 --2- 172.21.15.36:0/501297686 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd028053940 0x7fd028055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:48.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.285+0000 7fd041ffb700 1 -- 172.21.15.36:0/501297686 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd03c06ca60 con 0x7fd0440b65f0 2022-06-17T12:47:48.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.285+0000 7fd043fff700 1 --2- 172.21.15.36:0/501297686 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd028053940 0x7fd028055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:48.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.285+0000 7fd043fff700 1 --2- 172.21.15.36:0/501297686 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd028053940 0x7fd028055e00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fd03400d000 tx=0x7fd034007a40).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:48.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.285+0000 7fd041ffb700 1 -- 172.21.15.36:0/501297686 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd03c043860 con 0x7fd0440b65f0 2022-06-17T12:47:48.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:48.457+0000 7fd0491a4700 1 -- 172.21.15.36:0/501297686 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd044057410 con 0x7fd0440b65f0 2022-06-17T12:47:49.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.089+0000 7fd041ffb700 1 -- 172.21.15.36:0/501297686 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd03c0240b0 con 0x7fd0440b65f0 2022-06-17T12:47:49.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.093+0000 7fd0491a4700 1 -- 172.21.15.36:0/501297686 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd028053940 msgr2=0x7fd028055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:49.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.093+0000 7fd0491a4700 1 --2- 172.21.15.36:0/501297686 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd028053940 0x7fd028055e00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fd03400d000 tx=0x7fd034007a40).stop 2022-06-17T12:47:49.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.093+0000 7fd0491a4700 1 -- 172.21.15.36:0/501297686 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0440b65f0 msgr2=0x7fd04413cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:49.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.093+0000 7fd0491a4700 1 --2- 172.21.15.36:0/501297686 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0440b65f0 0x7fd04413cbf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fd03c0032f0 tx=0x7fd03c016000).stop 2022-06-17T12:47:49.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.093+0000 7fd0491a4700 1 -- 172.21.15.36:0/501297686 shutdown_connections 2022-06-17T12:47:49.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.093+0000 7fd0491a4700 1 --2- 172.21.15.36:0/501297686 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd028053940 0x7fd028055e00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:49.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.093+0000 7fd0491a4700 1 --2- 172.21.15.36:0/501297686 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0440b65f0 0x7fd04413cbf0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:49.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.093+0000 7fd0491a4700 1 --2- 172.21.15.36:0/501297686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0440b15b0 0x7fd0440b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:49.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.093+0000 7fd0491a4700 1 --2- 172.21.15.36:0/501297686 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0440adf10 0x7fd044152c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:49.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.093+0000 7fd0491a4700 1 -- 172.21.15.36:0/501297686 >> 172.21.15.36:0/501297686 conn(0x7fd0440614b0 msgr2=0x7fd04405cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:49.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.093+0000 7fd0491a4700 1 -- 172.21.15.36:0/501297686 shutdown_connections 2022-06-17T12:47:49.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.093+0000 7fd0491a4700 1 -- 172.21.15.36:0/501297686 wait complete. 2022-06-17T12:47:49.114 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:47:49.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.261+0000 7fc4da679700 1 Processor -- start 2022-06-17T12:47:49.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.261+0000 7fc4da679700 1 -- start start 2022-06-17T12:47:49.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.261+0000 7fc4da679700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc4d40ae270 0x7fc4d40b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:49.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.261+0000 7fc4da679700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4d413fde0 0x7fc4d4140200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:49.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.261+0000 7fc4da679700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4d40afc30 con 0x7fc4d40ad9c0 2022-06-17T12:47:49.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.261+0000 7fc4da679700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d40b0cb0 con 0x7fc4d40ae270 2022-06-17T12:47:49.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.261+0000 7fc4da679700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d40b3eb0 con 0x7fc4d413fde0 2022-06-17T12:47:49.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.261+0000 7fc4d37fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fc4d40ad9c0 0x7fc4d40b05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50254/0 (socket says 172.21.15.36:50254) 2022-06-17T12:47:49.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.261+0000 7fc4d37fe700 1 -- 172.21.15.36:0/1898864797 learned_addr learned my addr 172.21.15.36:0/1898864797 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:49.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.261+0000 7fc4d3fff700 1 --2- 172.21.15.36:0/1898864797 >> v2:172.21.15.36:3300/0 conn(0x7fc4d40ae270 0x7fc4d40b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:49.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.261+0000 7fc4d8c16700 1 --2- 172.21.15.36:0/1898864797 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4d413fde0 0x7fc4d4140200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:49.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.261+0000 7fc4d2ffd700 1 -- 172.21.15.36:0/1898864797 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 731406712 0 0) 0x7fc4d40afc30 con 0x7fc4d40ad9c0 2022-06-17T12:47:49.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.261+0000 7fc4d2ffd700 1 -- 172.21.15.36:0/1898864797 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4b8003260 con 0x7fc4d40ad9c0 2022-06-17T12:47:49.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.261+0000 7fc4d3fff700 1 -- 172.21.15.36:0/1898864797 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4d413fde0 msgr2=0x7fc4d4140200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:49.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.261+0000 7fc4d3fff700 1 --2- 172.21.15.36:0/1898864797 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4d413fde0 0x7fc4d4140200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:49.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.261+0000 7fc4d3fff700 1 -- 172.21.15.36:0/1898864797 >> v1:172.21.15.36:6789/0 conn(0x7fc4d40ad9c0 legacy=0x7fc4d40b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:49.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.261+0000 7fc4d3fff700 1 -- 172.21.15.36:0/1898864797 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d40ae650 con 0x7fc4d40ae270 2022-06-17T12:47:49.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.261+0000 7fc4d3fff700 1 --2- 172.21.15.36:0/1898864797 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc4d40ae270 0x7fc4d40b39c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fc4d40b3ff0 tx=0x7fc4bc006df0).ready entity=mon.0 client_cookie=55ed98f4ebf4f48f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:49.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.261+0000 7fc4d2ffd700 1 -- 172.21.15.36:0/1898864797 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4bc00d570 con 0x7fc4d40ae270 2022-06-17T12:47:49.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.261+0000 7fc4d2ffd700 1 -- 172.21.15.36:0/1898864797 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4bc000e90 con 0x7fc4d40ae270 2022-06-17T12:47:49.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.261+0000 7fc4d2ffd700 1 -- 172.21.15.36:0/1898864797 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4bc00d8c0 con 0x7fc4d40ae270 2022-06-17T12:47:49.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.265+0000 7fc4da679700 1 -- 172.21.15.36:0/1898864797 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc4d40ae270 msgr2=0x7fc4d40b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:49.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.265+0000 7fc4da679700 1 --2- 172.21.15.36:0/1898864797 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc4d40ae270 0x7fc4d40b39c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fc4d40b3ff0 tx=0x7fc4bc006df0).stop 2022-06-17T12:47:49.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.265+0000 7fc4da679700 1 -- 172.21.15.36:0/1898864797 shutdown_connections 2022-06-17T12:47:49.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.265+0000 7fc4da679700 1 --2- 172.21.15.36:0/1898864797 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4d413fde0 0x7fc4d4140200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:49.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.265+0000 7fc4da679700 1 --2- 172.21.15.36:0/1898864797 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc4d40ae270 0x7fc4d40b39c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:49.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.265+0000 7fc4da679700 1 -- 172.21.15.36:0/1898864797 >> 172.21.15.36:0/1898864797 conn(0x7fc4d405c550 msgr2=0x7fc4d405c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:49.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.265+0000 7fc4da679700 1 -- 172.21.15.36:0/1898864797 shutdown_connections 2022-06-17T12:47:49.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.265+0000 7fc4da679700 1 -- 172.21.15.36:0/1898864797 wait complete. 2022-06-17T12:47:49.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.265+0000 7fc4da679700 1 Processor -- start 2022-06-17T12:47:49.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.265+0000 7fc4da679700 1 -- start start 2022-06-17T12:47:49.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.269+0000 7fc4da679700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4d40ad9c0 0x7fc4d4066040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:49.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.269+0000 7fc4da679700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc4d40ae270 0x7fc4d4063630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:49.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.269+0000 7fc4da679700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4d413fde0 0x7fc4d4130980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:49.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.269+0000 7fc4da679700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d4064bc0 con 0x7fc4d40ae270 2022-06-17T12:47:49.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.269+0000 7fc4da679700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d4138360 con 0x7fc4d413fde0 2022-06-17T12:47:49.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.269+0000 7fc4da679700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d4064220 con 0x7fc4d40ad9c0 2022-06-17T12:47:49.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.269+0000 7fc4d37fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc4d40ae270 0x7fc4d4063630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:49.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.269+0000 7fc4d37fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc4d40ae270 0x7fc4d4063630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60964/0 (socket says 172.21.15.36:60964) 2022-06-17T12:47:49.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.269+0000 7fc4d37fe700 1 -- 172.21.15.36:0/3119765694 learned_addr learned my addr 172.21.15.36:0/3119765694 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:49.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.269+0000 7fc4d8c16700 1 --2- 172.21.15.36:0/3119765694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4d413fde0 0x7fc4d4130980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:49.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.269+0000 7fc4d3fff700 1 --2- 172.21.15.36:0/3119765694 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4d40ad9c0 0x7fc4d4066040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:49.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.269+0000 7fc4d37fe700 1 -- 172.21.15.36:0/3119765694 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4d40ad9c0 msgr2=0x7fc4d4066040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:49.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.269+0000 7fc4d37fe700 1 --2- 172.21.15.36:0/3119765694 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4d40ad9c0 0x7fc4d4066040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:49.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.269+0000 7fc4d37fe700 1 -- 172.21.15.36:0/3119765694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4d413fde0 msgr2=0x7fc4d4130980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:49.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.269+0000 7fc4d37fe700 1 --2- 172.21.15.36:0/3119765694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4d413fde0 0x7fc4d4130980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:49.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.269+0000 7fc4d37fe700 1 -- 172.21.15.36:0/3119765694 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4bc00b180 con 0x7fc4d40ae270 2022-06-17T12:47:49.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.269+0000 7fc4d37fe700 1 --2- 172.21.15.36:0/3119765694 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc4d40ae270 0x7fc4d4063630 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7fc4c4001110 tx=0x7fc4c4007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:49.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.269+0000 7fc4d17fa700 1 -- 172.21.15.36:0/3119765694 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4c40070d0 con 0x7fc4d40ae270 2022-06-17T12:47:49.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.269+0000 7fc4d17fa700 1 -- 172.21.15.36:0/3119765694 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c4007230 con 0x7fc4d40ae270 2022-06-17T12:47:49.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.269+0000 7fc4da679700 1 -- 172.21.15.36:0/3119765694 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d4138cb0 con 0x7fc4d40ae270 2022-06-17T12:47:49.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.269+0000 7fc4d17fa700 1 -- 172.21.15.36:0/3119765694 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4c4005610 con 0x7fc4d40ae270 2022-06-17T12:47:49.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.269+0000 7fc4da679700 1 -- 172.21.15.36:0/3119765694 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d4063b20 con 0x7fc4d40ae270 2022-06-17T12:47:49.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.269+0000 7fc4da679700 1 -- 172.21.15.36:0/3119765694 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d40afe60 con 0x7fc4d40ae270 2022-06-17T12:47:49.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.273+0000 7fc4d17fa700 1 -- 172.21.15.36:0/3119765694 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fc4c4010070 con 0x7fc4d40ae270 2022-06-17T12:47:49.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.273+0000 7fc4d17fa700 1 --2- 172.21.15.36:0/3119765694 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc4ac0539b0 0x7fc4ac055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:49.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.273+0000 7fc4d17fa700 1 -- 172.21.15.36:0/3119765694 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc4c406b690 con 0x7fc4d40ae270 2022-06-17T12:47:49.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.273+0000 7fc4d3fff700 1 --2- 172.21.15.36:0/3119765694 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc4ac0539b0 0x7fc4ac055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:49.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.273+0000 7fc4d3fff700 1 --2- 172.21.15.36:0/3119765694 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc4ac0539b0 0x7fc4ac055e70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fc4bc01a000 tx=0x7fc4bc014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:49.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.277+0000 7fc4d17fa700 1 -- 172.21.15.36:0/3119765694 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc4c40423f0 con 0x7fc4d40ae270 2022-06-17T12:47:49.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.441+0000 7fc4da679700 1 -- 172.21.15.36:0/3119765694 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4d4138fe0 con 0x7fc4d40ae270 2022-06-17T12:47:49.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.445+0000 7fc4d17fa700 1 -- 172.21.15.36:0/3119765694 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+2138 (secure 0 0 0) 0x7fc4c4018020 con 0x7fc4d40ae270 2022-06-17T12:47:49.598 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:49.598 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3597082917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3597082917},{"type":"v1","addr":"172.21.15.70:6833","nonce":3597082917}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:47:47.072139+0000","modified":"2022-06-17T12:47:49.085163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4947":{"gid":4947,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.36:6835/1648255567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":1648255567},{"type":"v1","addr":"172.21.15.36:6835","nonce":1648255567}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-06-17T12:47:49.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.445+0000 7fc4da679700 1 -- 172.21.15.36:0/3119765694 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc4ac0539b0 msgr2=0x7fc4ac055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:49.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.445+0000 7fc4da679700 1 --2- 172.21.15.36:0/3119765694 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc4ac0539b0 0x7fc4ac055e70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fc4bc01a000 tx=0x7fc4bc014000).stop 2022-06-17T12:47:49.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.445+0000 7fc4da679700 1 -- 172.21.15.36:0/3119765694 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc4d40ae270 msgr2=0x7fc4d4063630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:49.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.445+0000 7fc4da679700 1 --2- 172.21.15.36:0/3119765694 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc4d40ae270 0x7fc4d4063630 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7fc4c4001110 tx=0x7fc4c4007ad0).stop 2022-06-17T12:47:49.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.445+0000 7fc4da679700 1 -- 172.21.15.36:0/3119765694 shutdown_connections 2022-06-17T12:47:49.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.445+0000 7fc4da679700 1 --2- 172.21.15.36:0/3119765694 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc4ac0539b0 0x7fc4ac055e70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:49.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.445+0000 7fc4da679700 1 --2- 172.21.15.36:0/3119765694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4d413fde0 0x7fc4d4130980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:49.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.445+0000 7fc4da679700 1 --2- 172.21.15.36:0/3119765694 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc4d40ae270 0x7fc4d4063630 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:49.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.445+0000 7fc4da679700 1 --2- 172.21.15.36:0/3119765694 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4d40ad9c0 0x7fc4d4066040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:49.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.445+0000 7fc4da679700 1 -- 172.21.15.36:0/3119765694 >> 172.21.15.36:0/3119765694 conn(0x7fc4d405c550 msgr2=0x7fc4d405bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:49.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.445+0000 7fc4da679700 1 -- 172.21.15.36:0/3119765694 shutdown_connections 2022-06-17T12:47:49.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.445+0000 7fc4da679700 1 -- 172.21.15.36:0/3119765694 wait complete. 2022-06-17T12:47:49.723 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 14 2022-06-17T12:47:49.724 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:47:49.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.857+0000 7ffab1ab8700 1 Processor -- start 2022-06-17T12:47:49.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.857+0000 7ffab1ab8700 1 -- start start 2022-06-17T12:47:49.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffab1ab8700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffaac063730 0x7ffaac0b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:49.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffab1ab8700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ffaac064eb0 0x7ffaac13ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:49.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffab1ab8700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaac1403f0 0x7ffaac0b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:49.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffab1ab8700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaac147430 con 0x7ffaac064eb0 2022-06-17T12:47:49.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffab1ab8700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaac0b1ca0 con 0x7ffaac063730 2022-06-17T12:47:49.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffab1ab8700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac144180 con 0x7ffaac1403f0 2022-06-17T12:47:49.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffaabfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ffaac064eb0 0x7ffaac13ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:49.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffaabfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ffaac064eb0 0x7ffaac13ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60972/0 (socket says 172.21.15.36:60972) 2022-06-17T12:47:49.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffaabfff700 1 -- 172.21.15.36:0/534274368 learned_addr learned my addr 172.21.15.36:0/534274368 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:49.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffab0ab6700 1 --2- 172.21.15.36:0/534274368 >> v2:172.21.15.70:3300/0 conn(0x7ffaac063730 0x7ffaac0b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:49.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffab12b7700 1 --2- 172.21.15.36:0/534274368 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaac1403f0 0x7ffaac0b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:49.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffaabfff700 1 -- 172.21.15.36:0/534274368 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaac1403f0 msgr2=0x7ffaac0b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:49.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffaabfff700 1 --2- 172.21.15.36:0/534274368 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaac1403f0 0x7ffaac0b33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:49.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffaabfff700 1 -- 172.21.15.36:0/534274368 >> v2:172.21.15.70:3300/0 conn(0x7ffaac063730 msgr2=0x7ffaac0b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:49.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffaabfff700 1 --2- 172.21.15.36:0/534274368 >> v2:172.21.15.70:3300/0 conn(0x7ffaac063730 0x7ffaac0b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:49.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffaabfff700 1 -- 172.21.15.36:0/534274368 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac063b10 con 0x7ffaac064eb0 2022-06-17T12:47:49.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffaabfff700 1 --2- 172.21.15.36:0/534274368 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffaac064eb0 0x7ffaac13ff00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7ffaac142f30 tx=0x7ffa98006eb0).ready entity=mon.0 client_cookie=26106d99e8d626dc server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:49.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffaab7fe700 1 -- 172.21.15.36:0/534274368 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa9800d570 con 0x7ffaac064eb0 2022-06-17T12:47:49.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffaab7fe700 1 -- 172.21.15.36:0/534274368 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa98000e90 con 0x7ffaac064eb0 2022-06-17T12:47:49.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffaab7fe700 1 -- 172.21.15.36:0/534274368 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa9800d900 con 0x7ffaac064eb0 2022-06-17T12:47:49.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffab1ab8700 1 -- 172.21.15.36:0/534274368 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffaac064eb0 msgr2=0x7ffaac13ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:49.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffab1ab8700 1 --2- 172.21.15.36:0/534274368 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffaac064eb0 0x7ffaac13ff00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7ffaac142f30 tx=0x7ffa98006eb0).stop 2022-06-17T12:47:49.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffab1ab8700 1 -- 172.21.15.36:0/534274368 shutdown_connections 2022-06-17T12:47:49.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffab1ab8700 1 --2- 172.21.15.36:0/534274368 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaac1403f0 0x7ffaac0b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:49.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffab1ab8700 1 --2- 172.21.15.36:0/534274368 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffaac064eb0 0x7ffaac13ff00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:49.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffab1ab8700 1 --2- 172.21.15.36:0/534274368 >> v2:172.21.15.70:3300/0 conn(0x7ffaac063730 0x7ffaac0b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:49.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffab1ab8700 1 -- 172.21.15.36:0/534274368 >> 172.21.15.36:0/534274368 conn(0x7ffaac0614b0 msgr2=0x7ffaac0618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:49.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffab1ab8700 1 -- 172.21.15.36:0/534274368 shutdown_connections 2022-06-17T12:47:49.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.861+0000 7ffab1ab8700 1 -- 172.21.15.36:0/534274368 wait complete. 2022-06-17T12:47:49.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffab1ab8700 1 Processor -- start 2022-06-17T12:47:49.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffab1ab8700 1 -- start start 2022-06-17T12:47:49.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffab1ab8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffaac063730 0x7ffaac05c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:49.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffab1ab8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaac064eb0 0x7ffaac1410b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:49.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffab1ab8700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaac1403f0 0x7ffaac0662c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:49.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffab1ab8700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac065ed0 con 0x7ffaac063730 2022-06-17T12:47:49.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffab1ab8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac0657a0 con 0x7ffaac064eb0 2022-06-17T12:47:49.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffab1ab8700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac1458c0 con 0x7ffaac1403f0 2022-06-17T12:47:49.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffab0ab6700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffaac063730 0x7ffaac05c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:49.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffab0ab6700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffaac063730 0x7ffaac05c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60976/0 (socket says 172.21.15.36:60976) 2022-06-17T12:47:49.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffab0ab6700 1 -- 172.21.15.36:0/2566589013 learned_addr learned my addr 172.21.15.36:0/2566589013 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:49.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffaabfff700 1 --2- 172.21.15.36:0/2566589013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaac064eb0 0x7ffaac1410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:49.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffab12b7700 1 --2- 172.21.15.36:0/2566589013 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaac1403f0 0x7ffaac0662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:49.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffab0ab6700 1 -- 172.21.15.36:0/2566589013 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaac1403f0 msgr2=0x7ffaac0662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:49.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffab0ab6700 1 --2- 172.21.15.36:0/2566589013 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaac1403f0 0x7ffaac0662c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:49.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffab0ab6700 1 -- 172.21.15.36:0/2566589013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaac064eb0 msgr2=0x7ffaac1410b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:49.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffab0ab6700 1 --2- 172.21.15.36:0/2566589013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaac064eb0 0x7ffaac1410b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:49.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffab0ab6700 1 -- 172.21.15.36:0/2566589013 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9800b1e0 con 0x7ffaac063730 2022-06-17T12:47:49.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffaabfff700 1 --2- 172.21.15.36:0/2566589013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaac064eb0 0x7ffaac1410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:47:49.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffab0ab6700 1 --2- 172.21.15.36:0/2566589013 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffaac063730 0x7ffaac05c0a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7ffaac1522e0 tx=0x7ffaa0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:49.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffaa9ffb700 1 -- 172.21.15.36:0/2566589013 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffaa000c830 con 0x7ffaac063730 2022-06-17T12:47:49.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffaa9ffb700 1 -- 172.21.15.36:0/2566589013 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa000c990 con 0x7ffaac063730 2022-06-17T12:47:49.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffab1ab8700 1 -- 172.21.15.36:0/2566589013 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaac05c8c0 con 0x7ffaac063730 2022-06-17T12:47:49.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffaa9ffb700 1 -- 172.21.15.36:0/2566589013 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffaa000ccf0 con 0x7ffaac063730 2022-06-17T12:47:49.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.865+0000 7ffab1ab8700 1 -- 172.21.15.36:0/2566589013 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaac145420 con 0x7ffaac063730 2022-06-17T12:47:49.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.869+0000 7ffaa9ffb700 1 -- 172.21.15.36:0/2566589013 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ffaa001c460 con 0x7ffaac063730 2022-06-17T12:47:49.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.869+0000 7ffab1ab8700 1 -- 172.21.15.36:0/2566589013 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaac061e40 con 0x7ffaac063730 2022-06-17T12:47:49.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.869+0000 7ffaa9ffb700 1 --2- 172.21.15.36:0/2566589013 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffa8c0539b0 0x7ffa8c055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:49.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.869+0000 7ffaa9ffb700 1 -- 172.21.15.36:0/2566589013 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ffaa006c520 con 0x7ffaac063730 2022-06-17T12:47:49.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.869+0000 7ffaabfff700 1 --2- 172.21.15.36:0/2566589013 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffa8c0539b0 0x7ffa8c055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:49.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.869+0000 7ffaabfff700 1 --2- 172.21.15.36:0/2566589013 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffa8c0539b0 0x7ffa8c055e70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7ffa98016000 tx=0x7ffa98015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:49.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:49.873+0000 7ffaa9ffb700 1 -- 172.21.15.36:0/2566589013 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffaa00432c0 con 0x7ffaac063730 2022-06-17T12:47:50.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.005+0000 7ffab1ab8700 1 -- 172.21.15.36:0/2566589013 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffaac145ac0 con 0x7ffaac063730 2022-06-17T12:47:50.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.009+0000 7ffaa9ffb700 1 -- 172.21.15.36:0/2566589013 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16271 (secure 0 0 0) 0x7ffaa00468e0 con 0x7ffaac063730 2022-06-17T12:47:50.031 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:50.032 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":35,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:47:48.075718+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:47:45.085233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2022-06-17T12:47:46.070152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:47:50.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.013+0000 7ffab1ab8700 1 -- 172.21.15.36:0/2566589013 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffa8c0539b0 msgr2=0x7ffa8c055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:50.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.013+0000 7ffab1ab8700 1 --2- 172.21.15.36:0/2566589013 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffa8c0539b0 0x7ffa8c055e70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7ffa98016000 tx=0x7ffa98015000).stop 2022-06-17T12:47:50.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.013+0000 7ffab1ab8700 1 -- 172.21.15.36:0/2566589013 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffaac063730 msgr2=0x7ffaac05c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:50.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.013+0000 7ffab1ab8700 1 --2- 172.21.15.36:0/2566589013 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffaac063730 0x7ffaac05c0a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7ffaac1522e0 tx=0x7ffaa0006df0).stop 2022-06-17T12:47:50.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.013+0000 7ffab1ab8700 1 -- 172.21.15.36:0/2566589013 shutdown_connections 2022-06-17T12:47:50.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.013+0000 7ffab1ab8700 1 --2- 172.21.15.36:0/2566589013 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffa8c0539b0 0x7ffa8c055e70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:50.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.013+0000 7ffab1ab8700 1 --2- 172.21.15.36:0/2566589013 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaac1403f0 0x7ffaac0662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:50.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.013+0000 7ffab1ab8700 1 --2- 172.21.15.36:0/2566589013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaac064eb0 0x7ffaac1410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:50.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.013+0000 7ffab1ab8700 1 --2- 172.21.15.36:0/2566589013 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffaac063730 0x7ffaac05c0a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:50.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.013+0000 7ffab1ab8700 1 -- 172.21.15.36:0/2566589013 >> 172.21.15.36:0/2566589013 conn(0x7ffaac0614b0 msgr2=0x7ffaac0b4240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:50.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.013+0000 7ffab1ab8700 1 -- 172.21.15.36:0/2566589013 shutdown_connections 2022-06-17T12:47:50.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.013+0000 7ffab1ab8700 1 -- 172.21.15.36:0/2566589013 wait complete. 2022-06-17T12:47:50.107 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:47:50.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.245+0000 7fb64e61d700 1 Processor -- start 2022-06-17T12:47:50.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.245+0000 7fb64e61d700 1 -- start start 2022-06-17T12:47:50.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb64e61d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb64813cb70 0x7fb648143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:50.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb64e61d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb6480aef60 0x7fb6480b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:50.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb64e61d700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6480af4b0 con 0x7fb6480ba2e0 2022-06-17T12:47:50.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb64e61d700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb648142f50 con 0x7fb6480aef60 2022-06-17T12:47:50.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb64e61d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6481431f0 con 0x7fb64813cb70 2022-06-17T12:47:50.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb64ce1a700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb6480aef60 0x7fb6480b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:50.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb64de1c700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb6480ba2e0 0x7fb6481442a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:46016/0 (socket says 172.21.15.36:46016) 2022-06-17T12:47:50.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb64de1c700 1 -- 172.21.15.36:0/4202090691 learned_addr learned my addr 172.21.15.36:0/4202090691 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:50.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb64d61b700 1 --2- 172.21.15.36:0/4202090691 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb64813cb70 0x7fb648143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:50.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb64d61b700 1 -- 172.21.15.36:0/4202090691 >> v2:172.21.15.70:3300/0 conn(0x7fb6480aef60 msgr2=0x7fb6480b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:50.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb64d61b700 1 --2- 172.21.15.36:0/4202090691 >> v2:172.21.15.70:3300/0 conn(0x7fb6480aef60 0x7fb6480b9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:50.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb64d61b700 1 -- 172.21.15.36:0/4202090691 >> v1:172.21.15.70:6789/0 conn(0x7fb6480ba2e0 legacy=0x7fb6481442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:50.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb64d61b700 1 -- 172.21.15.36:0/4202090691 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6481449b0 con 0x7fb64813cb70 2022-06-17T12:47:50.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb64ce1a700 1 --2- 172.21.15.36:0/4202090691 >> v2:172.21.15.70:3300/0 conn(0x7fb6480aef60 0x7fb6480b9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:50.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb64d61b700 1 --2- 172.21.15.36:0/4202090691 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb64813cb70 0x7fb648143690 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fb638002f70 tx=0x7fb638006df0).ready entity=mon.2 client_cookie=aa6e51e2b7cd4efb server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:50.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb637fff700 1 -- 172.21.15.36:0/4202090691 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb63800d570 con 0x7fb64813cb70 2022-06-17T12:47:50.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb637fff700 1 -- 172.21.15.36:0/4202090691 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb638000e90 con 0x7fb64813cb70 2022-06-17T12:47:50.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb637fff700 1 -- 172.21.15.36:0/4202090691 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb63800d8c0 con 0x7fb64813cb70 2022-06-17T12:47:50.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb64e61d700 1 -- 172.21.15.36:0/4202090691 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb64813cb70 msgr2=0x7fb648143690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:50.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb64e61d700 1 --2- 172.21.15.36:0/4202090691 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb64813cb70 0x7fb648143690 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fb638002f70 tx=0x7fb638006df0).stop 2022-06-17T12:47:50.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb64e61d700 1 -- 172.21.15.36:0/4202090691 shutdown_connections 2022-06-17T12:47:50.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb64e61d700 1 --2- 172.21.15.36:0/4202090691 >> v2:172.21.15.70:3300/0 conn(0x7fb6480aef60 0x7fb6480b9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:50.366 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb64e61d700 1 --2- 172.21.15.36:0/4202090691 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb64813cb70 0x7fb648143690 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:50.366 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb64e61d700 1 -- 172.21.15.36:0/4202090691 >> 172.21.15.36:0/4202090691 conn(0x7fb64805c550 msgr2=0x7fb6480b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:50.366 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb64e61d700 1 -- 172.21.15.36:0/4202090691 shutdown_connections 2022-06-17T12:47:50.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.249+0000 7fb64e61d700 1 -- 172.21.15.36:0/4202090691 wait complete. 2022-06-17T12:47:50.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb64e61d700 1 Processor -- start 2022-06-17T12:47:50.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb64e61d700 1 -- start start 2022-06-17T12:47:50.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb64e61d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6480aef60 0x7fb6480b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:50.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb64e61d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb6480ba2e0 0x7fb64813f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:50.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb64e61d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb64813cb70 0x7fb648146490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:50.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb64e61d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb648156540 con 0x7fb6480ba2e0 2022-06-17T12:47:50.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb64e61d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6480afa00 con 0x7fb6480aef60 2022-06-17T12:47:50.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb64e61d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb648146980 con 0x7fb64813cb70 2022-06-17T12:47:50.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb64ce1a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb6480ba2e0 0x7fb64813f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:50.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb64ce1a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb6480ba2e0 0x7fb64813f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:60992/0 (socket says 172.21.15.36:60992) 2022-06-17T12:47:50.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb64ce1a700 1 -- 172.21.15.36:0/4149309660 learned_addr learned my addr 172.21.15.36:0/4149309660 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:50.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb64de1c700 1 --2- 172.21.15.36:0/4149309660 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb64813cb70 0x7fb648146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:50.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb64d61b700 1 --2- 172.21.15.36:0/4149309660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6480aef60 0x7fb6480b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:50.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb64ce1a700 1 -- 172.21.15.36:0/4149309660 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb64813cb70 msgr2=0x7fb648146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:50.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb64ce1a700 1 --2- 172.21.15.36:0/4149309660 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb64813cb70 0x7fb648146490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:50.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb64ce1a700 1 -- 172.21.15.36:0/4149309660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6480aef60 msgr2=0x7fb6480b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:50.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb64ce1a700 1 --2- 172.21.15.36:0/4149309660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6480aef60 0x7fb6480b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:50.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb64ce1a700 1 -- 172.21.15.36:0/4149309660 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64400b180 con 0x7fb6480ba2e0 2022-06-17T12:47:50.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb64de1c700 1 --2- 172.21.15.36:0/4149309660 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb64813cb70 0x7fb648146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:50.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb64ce1a700 1 --2- 172.21.15.36:0/4149309660 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb6480ba2e0 0x7fb64813f060 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7fb64400c000 tx=0x7fb644006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:50.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb6367fc700 1 -- 172.21.15.36:0/4149309660 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb64400c7e0 con 0x7fb6480ba2e0 2022-06-17T12:47:50.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb6367fc700 1 -- 172.21.15.36:0/4149309660 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64400c940 con 0x7fb6480ba2e0 2022-06-17T12:47:50.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb6367fc700 1 -- 172.21.15.36:0/4149309660 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb64400cca0 con 0x7fb6480ba2e0 2022-06-17T12:47:50.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb64e61d700 1 -- 172.21.15.36:0/4149309660 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb63800b180 con 0x7fb6480ba2e0 2022-06-17T12:47:50.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.253+0000 7fb64e61d700 1 -- 172.21.15.36:0/4149309660 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb648156190 con 0x7fb6480ba2e0 2022-06-17T12:47:50.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.257+0000 7fb6367fc700 1 -- 172.21.15.36:0/4149309660 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fb64401c460 con 0x7fb6480ba2e0 2022-06-17T12:47:50.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.257+0000 7fb64e61d700 1 -- 172.21.15.36:0/4149309660 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb648057410 con 0x7fb6480ba2e0 2022-06-17T12:47:50.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.257+0000 7fb6367fc700 1 --2- 172.21.15.36:0/4149309660 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb628053ad0 0x7fb628055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:50.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.257+0000 7fb6367fc700 1 -- 172.21.15.36:0/4149309660 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb64406c5f0 con 0x7fb6480ba2e0 2022-06-17T12:47:50.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.257+0000 7fb64d61b700 1 --2- 172.21.15.36:0/4149309660 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb628053ad0 0x7fb628055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:50.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.257+0000 7fb64d61b700 1 --2- 172.21.15.36:0/4149309660 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb628053ad0 0x7fb628055f90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fb63801a000 tx=0x7fb638014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:50.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.265+0000 7fb6367fc700 1 -- 172.21.15.36:0/4149309660 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb6440432f0 con 0x7fb6480ba2e0 2022-06-17T12:47:50.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.441+0000 7fb64e61d700 1 -- 172.21.15.36:0/4149309660 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fb648156710 con 0x7fb6480ba2e0 2022-06-17T12:47:50.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.449+0000 7fb6367fc700 1 -- 172.21.15.36:0/4149309660 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v13) v1 ==== 184+0+0 (secure 0 0 0) 0x7fb644046910 con 0x7fb6480ba2e0 2022-06-17T12:47:50.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.449+0000 7fb64e61d700 1 -- 172.21.15.36:0/4149309660 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb628053ad0 msgr2=0x7fb628055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:50.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.449+0000 7fb64e61d700 1 --2- 172.21.15.36:0/4149309660 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb628053ad0 0x7fb628055f90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fb63801a000 tx=0x7fb638014000).stop 2022-06-17T12:47:50.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.449+0000 7fb64e61d700 1 -- 172.21.15.36:0/4149309660 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb6480ba2e0 msgr2=0x7fb64813f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:50.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.449+0000 7fb64e61d700 1 --2- 172.21.15.36:0/4149309660 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb6480ba2e0 0x7fb64813f060 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7fb64400c000 tx=0x7fb644006df0).stop 2022-06-17T12:47:50.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.449+0000 7fb64e61d700 1 -- 172.21.15.36:0/4149309660 shutdown_connections 2022-06-17T12:47:50.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.449+0000 7fb64e61d700 1 --2- 172.21.15.36:0/4149309660 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb628053ad0 0x7fb628055f90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:50.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.449+0000 7fb64e61d700 1 --2- 172.21.15.36:0/4149309660 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb64813cb70 0x7fb648146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:50.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.449+0000 7fb64e61d700 1 --2- 172.21.15.36:0/4149309660 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb6480ba2e0 0x7fb64813f060 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:50.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.449+0000 7fb64e61d700 1 --2- 172.21.15.36:0/4149309660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6480aef60 0x7fb6480b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:50.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.449+0000 7fb64e61d700 1 -- 172.21.15.36:0/4149309660 >> 172.21.15.36:0/4149309660 conn(0x7fb64805c550 msgr2=0x7fb6480b8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:50.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.449+0000 7fb64e61d700 1 -- 172.21.15.36:0/4149309660 shutdown_connections 2022-06-17T12:47:50.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.449+0000 7fb64e61d700 1 -- 172.21.15.36:0/4149309660 wait complete. 2022-06-17T12:47:50.476 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.0 2022-06-17T12:47:50.476 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:47:50.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f5462b60700 1 Processor -- start 2022-06-17T12:47:50.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f5462b60700 1 -- start start 2022-06-17T12:47:50.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f5462b60700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f545c0b4820 0x7f545c0b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:50.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f5462b60700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545c0b83f0 con 0x7f545c143a90 2022-06-17T12:47:50.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f5462b60700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545c0b1ff0 con 0x7f545c062470 2022-06-17T12:47:50.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f5462b60700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545c142ec0 con 0x7f545c0b4820 2022-06-17T12:47:50.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f546235f700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f545c143a90 0x7f545c144390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50300/0 (socket says 172.21.15.36:50300) 2022-06-17T12:47:50.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f546235f700 1 -- 172.21.15.36:0/3926998756 learned_addr learned my addr 172.21.15.36:0/3926998756 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:50.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f5461b5e700 1 --2- 172.21.15.36:0/3926998756 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f545c0b4820 0x7f545c0b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:50.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f5460b5c700 1 -- 172.21.15.36:0/3926998756 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836602348 0 0) 0x7f545c0b83f0 con 0x7f545c143a90 2022-06-17T12:47:50.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f5460b5c700 1 -- 172.21.15.36:0/3926998756 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5444003260 con 0x7f545c143a90 2022-06-17T12:47:50.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f5460b5c700 1 -- 172.21.15.36:0/3926998756 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 189247030 0 0) 0x7f545c0b1ff0 con 0x7f545c062470 2022-06-17T12:47:50.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f5460b5c700 1 -- 172.21.15.36:0/3926998756 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f545c0b83f0 con 0x7f545c062470 2022-06-17T12:47:50.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f5460b5c700 1 -- 172.21.15.36:0/3926998756 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1358069667 0 0) 0x7f5444003260 con 0x7f545c143a90 2022-06-17T12:47:50.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f5460b5c700 1 -- 172.21.15.36:0/3926998756 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f545c0b4820 msgr2=0x7f545c0b4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:50.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f5460b5c700 1 --2- 172.21.15.36:0/3926998756 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f545c0b4820 0x7f545c0b4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:50.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f5460b5c700 1 -- 172.21.15.36:0/3926998756 >> v1:172.21.15.70:6789/0 conn(0x7f545c062470 legacy=0x7f545c0b7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:51.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f5460b5c700 1 -- 172.21.15.36:0/3926998756 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c1430b0 con 0x7f545c143a90 2022-06-17T12:47:51.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f5460b5c700 1 -- 172.21.15.36:0/3926998756 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f54540019e0 con 0x7f545c143a90 2022-06-17T12:47:51.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f5460b5c700 1 -- 172.21.15.36:0/3926998756 >> v1:172.21.15.36:6789/0 conn(0x7f545c143a90 legacy=0x7f545c144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:51.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f5460b5c700 1 --2- 172.21.15.36:0/3926998756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5444003910 0x7f5444003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:51.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f5460b5c700 1 --2- 172.21.15.36:0/3926998756 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5444004500 0x7f5444004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:51.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f5460b5c700 1 --2- 172.21.15.36:0/3926998756 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f545c0b4820 0x7f5444004e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:51.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f5460b5c700 1 -- 172.21.15.36:0/3926998756 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544400b800 con 0x7f5444004500 2022-06-17T12:47:51.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f5460b5c700 1 -- 172.21.15.36:0/3926998756 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544400b940 con 0x7f5444003910 2022-06-17T12:47:51.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f5460b5c700 1 -- 172.21.15.36:0/3926998756 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544400bad0 con 0x7f545c0b4820 2022-06-17T12:47:51.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f546135d700 1 --2- 172.21.15.36:0/3926998756 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5444004500 0x7f5444004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:51.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.709+0000 7f546235f700 1 --2- 172.21.15.36:0/3926998756 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f545c0b4820 0x7f5444004e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:51.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.713+0000 7f5461b5e700 1 --2- 172.21.15.36:0/3926998756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5444003910 0x7f5444003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:51.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.713+0000 7f546135d700 1 -- 172.21.15.36:0/3926998756 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f545c0b4820 msgr2=0x7f5444004e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:51.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.713+0000 7f546135d700 1 --2- 172.21.15.36:0/3926998756 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f545c0b4820 0x7f5444004e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:51.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.713+0000 7f546135d700 1 -- 172.21.15.36:0/3926998756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5444003910 msgr2=0x7f5444003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:51.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.713+0000 7f546135d700 1 --2- 172.21.15.36:0/3926998756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5444003910 0x7f5444003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:51.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.713+0000 7f546135d700 1 -- 172.21.15.36:0/3926998756 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5444003260 con 0x7f5444004500 2022-06-17T12:47:51.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.713+0000 7f546235f700 1 --2- 172.21.15.36:0/3926998756 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f545c0b4820 0x7f5444004e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:51.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.713+0000 7f5461b5e700 1 --2- 172.21.15.36:0/3926998756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5444003910 0x7f5444003d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:47:51.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.713+0000 7f546135d700 1 --2- 172.21.15.36:0/3926998756 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5444004500 0x7f5444004980 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f54440054d0 tx=0x7f544c007a00).ready entity=mon.0 client_cookie=a7ed2bd267fe1bca server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:51.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.713+0000 7f5460b5c700 1 -- 172.21.15.36:0/3926998756 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f544c007d10 con 0x7f5444004500 2022-06-17T12:47:51.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.713+0000 7f5460b5c700 1 -- 172.21.15.36:0/3926998756 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544c007030 con 0x7f5444004500 2022-06-17T12:47:51.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.713+0000 7f5460b5c700 1 -- 172.21.15.36:0/3926998756 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f544c007390 con 0x7f5444004500 2022-06-17T12:47:51.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.713+0000 7f5462b60700 1 -- 172.21.15.36:0/3926998756 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5444004500 msgr2=0x7f5444004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:51.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.713+0000 7f5462b60700 1 --2- 172.21.15.36:0/3926998756 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5444004500 0x7f5444004980 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f54440054d0 tx=0x7f544c007a00).stop 2022-06-17T12:47:51.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.713+0000 7f546235f700 1 -- 172.21.15.36:0/3926998756 reap_dead start 2022-06-17T12:47:51.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.713+0000 7f5462b60700 1 -- 172.21.15.36:0/3926998756 shutdown_connections 2022-06-17T12:47:51.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.713+0000 7f5462b60700 1 -- 172.21.15.36:0/3926998756 >> 172.21.15.36:0/3926998756 conn(0x7f545c05c550 msgr2=0x7f545c0b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:51.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.713+0000 7f5462b60700 1 -- 172.21.15.36:0/3926998756 shutdown_connections 2022-06-17T12:47:51.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.713+0000 7f5462b60700 1 -- 172.21.15.36:0/3926998756 wait complete. 2022-06-17T12:47:51.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.713+0000 7f5462b60700 1 Processor -- start 2022-06-17T12:47:51.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.717+0000 7f5462b60700 1 -- start start 2022-06-17T12:47:51.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.717+0000 7f5462b60700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f545c0669f0 0x7f545c155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:51.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.717+0000 7f5462b60700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f545c146760 0x7f545c155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:51.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.717+0000 7f5462b60700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f545c146c10 0x7f545c156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:51.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.717+0000 7f5462b60700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c066dd0 con 0x7f545c146760 2022-06-17T12:47:51.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.717+0000 7f5462b60700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c13fb40 con 0x7f545c146c10 2022-06-17T12:47:51.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.717+0000 7f5462b60700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545c13fde0 con 0x7f545c0669f0 2022-06-17T12:47:51.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.717+0000 7f546135d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f545c146760 0x7f545c155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:51.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.717+0000 7f546135d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f545c146760 0x7f545c155d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32780/0 (socket says 172.21.15.36:32780) 2022-06-17T12:47:51.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.717+0000 7f546135d700 1 -- 172.21.15.36:0/2495243585 learned_addr learned my addr 172.21.15.36:0/2495243585 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:51.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.717+0000 7f5461b5e700 1 --2- 172.21.15.36:0/2495243585 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f545c0669f0 0x7f545c155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:51.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.717+0000 7f546235f700 1 --2- 172.21.15.36:0/2495243585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f545c146c10 0x7f545c156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:51.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.717+0000 7f546135d700 1 -- 172.21.15.36:0/2495243585 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f545c0669f0 msgr2=0x7f545c155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:51.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.717+0000 7f546135d700 1 --2- 172.21.15.36:0/2495243585 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f545c0669f0 0x7f545c155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:51.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.717+0000 7f546135d700 1 -- 172.21.15.36:0/2495243585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f545c146c10 msgr2=0x7f545c156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:51.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.717+0000 7f546135d700 1 --2- 172.21.15.36:0/2495243585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f545c146c10 0x7f545c156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:51.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.717+0000 7f546135d700 1 -- 172.21.15.36:0/2495243585 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c140080 con 0x7f545c146760 2022-06-17T12:47:51.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.717+0000 7f5461b5e700 1 --2- 172.21.15.36:0/2495243585 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f545c0669f0 0x7f545c155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:51.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.717+0000 7f546135d700 1 --2- 172.21.15.36:0/2495243585 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f545c146760 0x7f545c155d20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f54440054d0 tx=0x7f544c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:51.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.717+0000 7f544affd700 1 -- 172.21.15.36:0/2495243585 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f544c01e430 con 0x7f545c146760 2022-06-17T12:47:51.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.717+0000 7f544affd700 1 -- 172.21.15.36:0/2495243585 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544c01e590 con 0x7f545c146760 2022-06-17T12:47:51.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.717+0000 7f5462b60700 1 -- 172.21.15.36:0/2495243585 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f545c13f830 con 0x7f545c146760 2022-06-17T12:47:51.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.717+0000 7f544affd700 1 -- 172.21.15.36:0/2495243585 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f544c01e8f0 con 0x7f545c146760 2022-06-17T12:47:51.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.717+0000 7f5462b60700 1 -- 172.21.15.36:0/2495243585 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f545c13d590 con 0x7f545c146760 2022-06-17T12:47:51.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.721+0000 7f5462b60700 1 -- 172.21.15.36:0/2495243585 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f545c140210 con 0x7f545c146760 2022-06-17T12:47:51.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.721+0000 7f544affd700 1 -- 172.21.15.36:0/2495243585 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f544c01b040 con 0x7f545c146760 2022-06-17T12:47:51.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.721+0000 7f544affd700 1 --2- 172.21.15.36:0/2495243585 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f543c053a60 0x7f543c055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:51.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.721+0000 7f5461b5e700 1 --2- 172.21.15.36:0/2495243585 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f543c053a60 0x7f543c055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:51.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.721+0000 7f544affd700 1 -- 172.21.15.36:0/2495243585 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f544c06d1f0 con 0x7f545c146760 2022-06-17T12:47:51.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.721+0000 7f5461b5e700 1 --2- 172.21.15.36:0/2495243585 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f543c053a60 0x7f543c055f20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f545800b000 tx=0x7f5458008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:51.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.725+0000 7f544affd700 1 -- 172.21.15.36:0/2495243585 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f544c043ff0 con 0x7f545c146760 2022-06-17T12:47:51.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.897+0000 7f5462b60700 1 -- 172.21.15.36:0/2495243585 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f545c140210 con 0x7f545c146760 2022-06-17T12:47:51.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.905+0000 7f544affd700 1 -- 172.21.15.36:0/2495243585 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v14) v1 ==== 184+0+0 (secure 0 0 0) 0x7f544c047610 con 0x7f545c146760 2022-06-17T12:47:51.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.905+0000 7f5462b60700 1 -- 172.21.15.36:0/2495243585 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f543c053a60 msgr2=0x7f543c055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:51.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.905+0000 7f5462b60700 1 --2- 172.21.15.36:0/2495243585 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f543c053a60 0x7f543c055f20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f545800b000 tx=0x7f5458008040).stop 2022-06-17T12:47:51.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.905+0000 7f5462b60700 1 -- 172.21.15.36:0/2495243585 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f545c146760 msgr2=0x7f545c155d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:51.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.905+0000 7f5462b60700 1 --2- 172.21.15.36:0/2495243585 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f545c146760 0x7f545c155d20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f54440054d0 tx=0x7f544c016000).stop 2022-06-17T12:47:51.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.905+0000 7f5462b60700 1 -- 172.21.15.36:0/2495243585 shutdown_connections 2022-06-17T12:47:51.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.905+0000 7f5462b60700 1 --2- 172.21.15.36:0/2495243585 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f543c053a60 0x7f543c055f20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:51.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.905+0000 7f5462b60700 1 --2- 172.21.15.36:0/2495243585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f545c146c10 0x7f545c156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:51.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.905+0000 7f5462b60700 1 --2- 172.21.15.36:0/2495243585 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f545c146760 0x7f545c155d20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:51.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.905+0000 7f5462b60700 1 --2- 172.21.15.36:0/2495243585 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f545c0669f0 0x7f545c155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:51.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.905+0000 7f5462b60700 1 -- 172.21.15.36:0/2495243585 >> 172.21.15.36:0/2495243585 conn(0x7f545c05c550 msgr2=0x7f545c0b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:51.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.905+0000 7f5462b60700 1 -- 172.21.15.36:0/2495243585 shutdown_connections 2022-06-17T12:47:51.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:50.905+0000 7f5462b60700 1 -- 172.21.15.36:0/2495243585 wait complete. 2022-06-17T12:47:51.236 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.1 2022-06-17T12:47:51.237 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:47:51.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.469+0000 7f49329bb700 1 Processor -- start 2022-06-17T12:47:51.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.469+0000 7f49329bb700 1 -- start start 2022-06-17T12:47:51.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.469+0000 7f49329bb700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f492c142a20 0x7f492c060780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:51.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.469+0000 7f49329bb700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f492c0b2340 0x7f492c140560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:51.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.469+0000 7f49329bb700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492c0b3520 con 0x7f492c060c70 2022-06-17T12:47:51.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.473+0000 7f49329bb700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f492c0b37c0 con 0x7f492c0b2340 2022-06-17T12:47:51.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.473+0000 7f49329bb700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c062f20 con 0x7f492c142a20 2022-06-17T12:47:51.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.473+0000 7f492bfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f492c142a20 0x7f492c060780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:51.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.473+0000 7f4930f58700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f492c0b2340 0x7f492c140560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:51.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.473+0000 7f492bfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f492c142a20 0x7f492c060780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:47576/0 (socket says 172.21.15.36:47576) 2022-06-17T12:47:51.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.473+0000 7f492bfff700 1 -- 172.21.15.36:0/225547743 learned_addr learned my addr 172.21.15.36:0/225547743 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:51.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.473+0000 7f4923fff700 1 --1- 172.21.15.36:0/225547743 >> v1:172.21.15.70:6789/0 conn(0x7f492c060c70 0x7f492c13fe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:46052/0 (socket says 172.21.15.36:46052) 2022-06-17T12:47:51.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.473+0000 7f492bfff700 1 -- 172.21.15.36:0/225547743 >> v2:172.21.15.70:3300/0 conn(0x7f492c0b2340 msgr2=0x7f492c140560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:51.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.473+0000 7f492bfff700 1 --2- 172.21.15.36:0/225547743 >> v2:172.21.15.70:3300/0 conn(0x7f492c0b2340 0x7f492c140560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:51.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.473+0000 7f492bfff700 1 -- 172.21.15.36:0/225547743 >> v1:172.21.15.70:6789/0 conn(0x7f492c060c70 legacy=0x7f492c13fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:51.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.473+0000 7f492bfff700 1 -- 172.21.15.36:0/225547743 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c0acf00 con 0x7f492c142a20 2022-06-17T12:47:51.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.473+0000 7f4930f58700 1 --2- 172.21.15.36:0/225547743 >> v2:172.21.15.70:3300/0 conn(0x7f492c0b2340 0x7f492c140560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:51.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.473+0000 7f492bfff700 1 --2- 172.21.15.36:0/225547743 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f492c142a20 0x7f492c060780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f491c002f70 tx=0x7f491c006f40).ready entity=mon.2 client_cookie=d2024271dcaaf2dc server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:51.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.473+0000 7f492b7fe700 1 -- 172.21.15.36:0/225547743 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f491c00da80 con 0x7f492c142a20 2022-06-17T12:47:51.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.473+0000 7f492b7fe700 1 -- 172.21.15.36:0/225547743 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491c000e90 con 0x7f492c142a20 2022-06-17T12:47:51.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.473+0000 7f492b7fe700 1 -- 172.21.15.36:0/225547743 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f491c00de10 con 0x7f492c142a20 2022-06-17T12:47:51.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.473+0000 7f49329bb700 1 -- 172.21.15.36:0/225547743 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f492c142a20 msgr2=0x7f492c060780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:51.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.473+0000 7f49329bb700 1 --2- 172.21.15.36:0/225547743 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f492c142a20 0x7f492c060780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f491c002f70 tx=0x7f491c006f40).stop 2022-06-17T12:47:51.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.473+0000 7f49329bb700 1 -- 172.21.15.36:0/225547743 shutdown_connections 2022-06-17T12:47:51.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.473+0000 7f49329bb700 1 --2- 172.21.15.36:0/225547743 >> v2:172.21.15.70:3300/0 conn(0x7f492c0b2340 0x7f492c140560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:51.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.473+0000 7f49329bb700 1 --2- 172.21.15.36:0/225547743 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f492c142a20 0x7f492c060780 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:51.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.473+0000 7f49329bb700 1 -- 172.21.15.36:0/225547743 >> 172.21.15.36:0/225547743 conn(0x7f492c0b6870 msgr2=0x7f492c0b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:51.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.473+0000 7f49329bb700 1 -- 172.21.15.36:0/225547743 shutdown_connections 2022-06-17T12:47:51.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.473+0000 7f49329bb700 1 -- 172.21.15.36:0/225547743 wait complete. 2022-06-17T12:47:51.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.477+0000 7f49329bb700 1 Processor -- start 2022-06-17T12:47:51.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.477+0000 7f49329bb700 1 -- start start 2022-06-17T12:47:51.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.477+0000 7f49329bb700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f492c060c70 0x7f492c065280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:51.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.477+0000 7f49329bb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c0b2340 0x7f492c0abd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:51.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.477+0000 7f49329bb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f492c142a20 0x7f492c0ac260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:51.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.477+0000 7f49329bb700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c065a90 con 0x7f492c142a20 2022-06-17T12:47:51.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.477+0000 7f49329bb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0646d0 con 0x7f492c0b2340 2022-06-17T12:47:51.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.477+0000 7f49329bb700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c064970 con 0x7f492c060c70 2022-06-17T12:47:51.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.477+0000 7f4930f58700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f492c142a20 0x7f492c0ac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:51.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.477+0000 7f4930f58700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f492c142a20 0x7f492c0ac260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32796/0 (socket says 172.21.15.36:32796) 2022-06-17T12:47:51.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.477+0000 7f4930f58700 1 -- 172.21.15.36:0/1092353868 learned_addr learned my addr 172.21.15.36:0/1092353868 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:51.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.477+0000 7f492bfff700 1 --2- 172.21.15.36:0/1092353868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f492c060c70 0x7f492c065280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:51.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.477+0000 7f4923fff700 1 --2- 172.21.15.36:0/1092353868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c0b2340 0x7f492c0abd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:51.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.477+0000 7f4930f58700 1 -- 172.21.15.36:0/1092353868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f492c060c70 msgr2=0x7f492c065280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:51.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.477+0000 7f4930f58700 1 --2- 172.21.15.36:0/1092353868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f492c060c70 0x7f492c065280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:51.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.477+0000 7f4930f58700 1 -- 172.21.15.36:0/1092353868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c0b2340 msgr2=0x7f492c0abd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:51.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.477+0000 7f4930f58700 1 --2- 172.21.15.36:0/1092353868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c0b2340 0x7f492c0abd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:51.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.477+0000 7f4930f58700 1 -- 172.21.15.36:0/1092353868 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4924009470 con 0x7f492c142a20 2022-06-17T12:47:51.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.481+0000 7f4930f58700 1 --2- 172.21.15.36:0/1092353868 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f492c142a20 0x7f492c0ac260 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f492400c000 tx=0x7f49240091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:51.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.481+0000 7f4929ffb700 1 -- 172.21.15.36:0/1092353868 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f492400c850 con 0x7f492c142a20 2022-06-17T12:47:51.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.481+0000 7f4929ffb700 1 -- 172.21.15.36:0/1092353868 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492400c9b0 con 0x7f492c142a20 2022-06-17T12:47:51.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.481+0000 7f49329bb700 1 -- 172.21.15.36:0/1092353868 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f491c00b460 con 0x7f492c142a20 2022-06-17T12:47:51.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.481+0000 7f4929ffb700 1 -- 172.21.15.36:0/1092353868 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f492400cd10 con 0x7f492c142a20 2022-06-17T12:47:51.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.481+0000 7f49329bb700 1 -- 172.21.15.36:0/1092353868 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492c0ac750 con 0x7f492c142a20 2022-06-17T12:47:51.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.481+0000 7f49329bb700 1 -- 172.21.15.36:0/1092353868 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492c057410 con 0x7f492c142a20 2022-06-17T12:47:51.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.481+0000 7f4929ffb700 1 -- 172.21.15.36:0/1092353868 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f4924022070 con 0x7f492c142a20 2022-06-17T12:47:51.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.481+0000 7f4929ffb700 1 --2- 172.21.15.36:0/1092353868 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4910053920 0x7f4910055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:51.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.481+0000 7f4929ffb700 1 -- 172.21.15.36:0/1092353868 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f492406c7e0 con 0x7f492c142a20 2022-06-17T12:47:51.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.481+0000 7f492bfff700 1 --2- 172.21.15.36:0/1092353868 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4910053920 0x7f4910055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:51.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.485+0000 7f492bfff700 1 --2- 172.21.15.36:0/1092353868 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4910053920 0x7f4910055de0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f491c01b000 tx=0x7f491c006f40).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:51.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.489+0000 7f4929ffb700 1 -- 172.21.15.36:0/1092353868 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f49240435e0 con 0x7f492c142a20 2022-06-17T12:47:51.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.661+0000 7f49329bb700 1 -- 172.21.15.36:0/1092353868 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f492c05f220 con 0x7f492c142a20 2022-06-17T12:47:51.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.669+0000 7f4929ffb700 1 -- 172.21.15.36:0/1092353868 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v15) v1 ==== 184+0+0 (secure 0 0 0) 0x7f49240160e0 con 0x7f492c142a20 2022-06-17T12:47:51.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.669+0000 7f49329bb700 1 -- 172.21.15.36:0/1092353868 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4910053920 msgr2=0x7f4910055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:51.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.669+0000 7f49329bb700 1 --2- 172.21.15.36:0/1092353868 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4910053920 0x7f4910055de0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f491c01b000 tx=0x7f491c006f40).stop 2022-06-17T12:47:51.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.669+0000 7f49329bb700 1 -- 172.21.15.36:0/1092353868 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f492c142a20 msgr2=0x7f492c0ac260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:51.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.669+0000 7f49329bb700 1 --2- 172.21.15.36:0/1092353868 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f492c142a20 0x7f492c0ac260 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f492400c000 tx=0x7f49240091d0).stop 2022-06-17T12:47:51.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.669+0000 7f49329bb700 1 -- 172.21.15.36:0/1092353868 shutdown_connections 2022-06-17T12:47:51.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.669+0000 7f49329bb700 1 --2- 172.21.15.36:0/1092353868 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4910053920 0x7f4910055de0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:51.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.669+0000 7f49329bb700 1 --2- 172.21.15.36:0/1092353868 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f492c142a20 0x7f492c0ac260 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:51.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.669+0000 7f49329bb700 1 --2- 172.21.15.36:0/1092353868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c0b2340 0x7f492c0abd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:51.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.669+0000 7f49329bb700 1 --2- 172.21.15.36:0/1092353868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f492c060c70 0x7f492c065280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:51.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.669+0000 7f49329bb700 1 -- 172.21.15.36:0/1092353868 >> 172.21.15.36:0/1092353868 conn(0x7f492c0b6870 msgr2=0x7f492c067270 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:51.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.669+0000 7f49329bb700 1 -- 172.21.15.36:0/1092353868 shutdown_connections 2022-06-17T12:47:51.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.669+0000 7f49329bb700 1 -- 172.21.15.36:0/1092353868 wait complete. 2022-06-17T12:47:51.683 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.2 2022-06-17T12:47:51.686 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:47:51.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.825+0000 7f5bc6518700 1 Processor -- start 2022-06-17T12:47:51.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.825+0000 7f5bc6518700 1 -- start start 2022-06-17T12:47:51.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.825+0000 7f5bc6518700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5bc00ad9c0 0x7f5bc00b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:51.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.825+0000 7f5bc6518700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5bc013fde0 0x7f5bc0140230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:51.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.825+0000 7f5bc6518700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc00afc30 con 0x7f5bc00ae270 2022-06-17T12:47:51.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.825+0000 7f5bc6518700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc00b0cb0 con 0x7f5bc013fde0 2022-06-17T12:47:51.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.825+0000 7f5bc6518700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc00b3eb0 con 0x7f5bc00ad9c0 2022-06-17T12:47:51.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.825+0000 7f5bbf7fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5bc00ad9c0 0x7f5bc00b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:51.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.825+0000 7f5bc4ab5700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5bc013fde0 0x7f5bc0140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:51.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.825+0000 7f5bbf7fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5bc00ad9c0 0x7f5bc00b39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:47590/0 (socket says 172.21.15.36:47590) 2022-06-17T12:47:51.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.825+0000 7f5bbf7fe700 1 -- 172.21.15.36:0/1521969670 learned_addr learned my addr 172.21.15.36:0/1521969670 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:51.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.825+0000 7f5bbffff700 1 --1- 172.21.15.36:0/1521969670 >> v1:172.21.15.70:6789/0 conn(0x7f5bc00ae270 0x7f5bc00b05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:46064/0 (socket says 172.21.15.36:46064) 2022-06-17T12:47:51.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.825+0000 7f5bbf7fe700 1 -- 172.21.15.36:0/1521969670 >> v2:172.21.15.36:3300/0 conn(0x7f5bc013fde0 msgr2=0x7f5bc0140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:51.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.825+0000 7f5bbf7fe700 1 --2- 172.21.15.36:0/1521969670 >> v2:172.21.15.36:3300/0 conn(0x7f5bc013fde0 0x7f5bc0140230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:51.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.825+0000 7f5bbf7fe700 1 -- 172.21.15.36:0/1521969670 >> v1:172.21.15.70:6789/0 conn(0x7f5bc00ae270 legacy=0x7f5bc00b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:51.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.825+0000 7f5bbf7fe700 1 -- 172.21.15.36:0/1521969670 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc00ae650 con 0x7f5bc00ad9c0 2022-06-17T12:47:51.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.825+0000 7f5bbf7fe700 1 --2- 172.21.15.36:0/1521969670 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5bc00ad9c0 0x7f5bc00b39c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f5bb80032f0 tx=0x7f5bb8009290).ready entity=mon.2 client_cookie=d58716e159c56d33 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:51.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.825+0000 7f5bbeffd700 1 -- 172.21.15.36:0/1521969670 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bb800b8a0 con 0x7f5bc00ad9c0 2022-06-17T12:47:51.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.825+0000 7f5bbeffd700 1 -- 172.21.15.36:0/1521969670 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb800ba00 con 0x7f5bc00ad9c0 2022-06-17T12:47:51.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.825+0000 7f5bbeffd700 1 -- 172.21.15.36:0/1521969670 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bb800bb70 con 0x7f5bc00ad9c0 2022-06-17T12:47:51.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.825+0000 7f5bc6518700 1 -- 172.21.15.36:0/1521969670 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5bc00ad9c0 msgr2=0x7f5bc00b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:51.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.825+0000 7f5bc6518700 1 --2- 172.21.15.36:0/1521969670 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5bc00ad9c0 0x7f5bc00b39c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f5bb80032f0 tx=0x7f5bb8009290).stop 2022-06-17T12:47:51.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.829+0000 7f5bc6518700 1 -- 172.21.15.36:0/1521969670 shutdown_connections 2022-06-17T12:47:51.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.829+0000 7f5bc6518700 1 --2- 172.21.15.36:0/1521969670 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5bc00ad9c0 0x7f5bc00b39c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:51.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.829+0000 7f5bc6518700 1 --2- 172.21.15.36:0/1521969670 >> v2:172.21.15.36:3300/0 conn(0x7f5bc013fde0 0x7f5bc0140230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:51.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.829+0000 7f5bc6518700 1 -- 172.21.15.36:0/1521969670 >> 172.21.15.36:0/1521969670 conn(0x7f5bc005c550 msgr2=0x7f5bc005c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:51.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.829+0000 7f5bc6518700 1 -- 172.21.15.36:0/1521969670 shutdown_connections 2022-06-17T12:47:51.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.829+0000 7f5bc6518700 1 -- 172.21.15.36:0/1521969670 wait complete. 2022-06-17T12:47:51.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.829+0000 7f5bc6518700 1 Processor -- start 2022-06-17T12:47:51.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.829+0000 7f5bc6518700 1 -- start start 2022-06-17T12:47:51.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.829+0000 7f5bc6518700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bc00ad9c0 0x7f5bc0063630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:51.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.829+0000 7f5bc6518700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5bc00ae270 0x7f5bc01412b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:51.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.829+0000 7f5bc6518700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5bc013fde0 0x7f5bc00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:51.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.829+0000 7f5bc6518700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc0063c60 con 0x7f5bc013fde0 2022-06-17T12:47:51.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.829+0000 7f5bc6518700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc0065340 con 0x7f5bc00ad9c0 2022-06-17T12:47:51.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.829+0000 7f5bc6518700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc01417a0 con 0x7f5bc00ae270 2022-06-17T12:47:51.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.829+0000 7f5bc4ab5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5bc013fde0 0x7f5bc00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:51.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.829+0000 7f5bbffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bc00ad9c0 0x7f5bc0063630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:51.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.829+0000 7f5bbf7fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5bc00ae270 0x7f5bc01412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:51.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.829+0000 7f5bc4ab5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5bc013fde0 0x7f5bc00b4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32810/0 (socket says 172.21.15.36:32810) 2022-06-17T12:47:51.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.829+0000 7f5bc4ab5700 1 -- 172.21.15.36:0/3544574731 learned_addr learned my addr 172.21.15.36:0/3544574731 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:52.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.833+0000 7f5bc4ab5700 1 -- 172.21.15.36:0/3544574731 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5bc00ae270 msgr2=0x7f5bc01412b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:52.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.833+0000 7f5bc4ab5700 1 --2- 172.21.15.36:0/3544574731 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5bc00ae270 0x7f5bc01412b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:52.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.833+0000 7f5bc4ab5700 1 -- 172.21.15.36:0/3544574731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bc00ad9c0 msgr2=0x7f5bc0063630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:52.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.833+0000 7f5bc4ab5700 1 --2- 172.21.15.36:0/3544574731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bc00ad9c0 0x7f5bc0063630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:52.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.833+0000 7f5bc4ab5700 1 -- 172.21.15.36:0/3544574731 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb8009530 con 0x7f5bc013fde0 2022-06-17T12:47:52.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.833+0000 7f5bbffff700 1 --2- 172.21.15.36:0/3544574731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bc00ad9c0 0x7f5bc0063630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:52.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.833+0000 7f5bc4ab5700 1 --2- 172.21.15.36:0/3544574731 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5bc013fde0 0x7f5bc00b4d40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f5bb000c000 tx=0x7f5bb0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:52.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.833+0000 7f5bbd7fa700 1 -- 172.21.15.36:0/3544574731 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bb0011750 con 0x7f5bc013fde0 2022-06-17T12:47:52.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.833+0000 7f5bbd7fa700 1 -- 172.21.15.36:0/3544574731 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb00118b0 con 0x7f5bc013fde0 2022-06-17T12:47:52.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.833+0000 7f5bc6518700 1 -- 172.21.15.36:0/3544574731 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc00b5230 con 0x7f5bc013fde0 2022-06-17T12:47:52.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.833+0000 7f5bbd7fa700 1 -- 172.21.15.36:0/3544574731 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bb0011c10 con 0x7f5bc013fde0 2022-06-17T12:47:52.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.833+0000 7f5bc6518700 1 -- 172.21.15.36:0/3544574731 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc0142b30 con 0x7f5bc013fde0 2022-06-17T12:47:52.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.833+0000 7f5bc6518700 1 -- 172.21.15.36:0/3544574731 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc0065500 con 0x7f5bc013fde0 2022-06-17T12:47:52.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.833+0000 7f5bbd7fa700 1 -- 172.21.15.36:0/3544574731 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f5bb0011d70 con 0x7f5bc013fde0 2022-06-17T12:47:52.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.833+0000 7f5bbd7fa700 1 --2- 172.21.15.36:0/3544574731 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5ba4053af0 0x7f5ba4055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:52.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.833+0000 7f5bbd7fa700 1 -- 172.21.15.36:0/3544574731 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5bb006c290 con 0x7f5bc013fde0 2022-06-17T12:47:52.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.837+0000 7f5bbffff700 1 --2- 172.21.15.36:0/3544574731 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5ba4053af0 0x7f5ba4055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:52.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.837+0000 7f5bbffff700 1 --2- 172.21.15.36:0/3544574731 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5ba4053af0 0x7f5ba4055fb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f5ba800d000 tx=0x7f5ba8007b40).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:52.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:51.845+0000 7f5bbd7fa700 1 -- 172.21.15.36:0/3544574731 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5bb0043090 con 0x7f5bc013fde0 2022-06-17T12:47:52.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.005+0000 7f5bc6518700 1 -- 172.21.15.36:0/3544574731 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5bc00afe60 con 0x7f5bc013fde0 2022-06-17T12:47:52.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.005+0000 7f5bbd7fa700 1 -- 172.21.15.36:0/3544574731 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+2138 (secure 0 0 0) 0x7f5bb00466b0 con 0x7f5bc013fde0 2022-06-17T12:47:52.131 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:52.132 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3597082917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3597082917},{"type":"v1","addr":"172.21.15.70:6833","nonce":3597082917}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:47:47.072139+0000","modified":"2022-06-17T12:47:49.085163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4947":{"gid":4947,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.36:6835/1648255567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":1648255567},{"type":"v1","addr":"172.21.15.36:6835","nonce":1648255567}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-06-17T12:47:52.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.009+0000 7f5bc6518700 1 -- 172.21.15.36:0/3544574731 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5ba4053af0 msgr2=0x7f5ba4055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:52.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.009+0000 7f5bc6518700 1 --2- 172.21.15.36:0/3544574731 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5ba4053af0 0x7f5ba4055fb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f5ba800d000 tx=0x7f5ba8007b40).stop 2022-06-17T12:47:52.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.009+0000 7f5bc6518700 1 -- 172.21.15.36:0/3544574731 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5bc013fde0 msgr2=0x7f5bc00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:52.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.009+0000 7f5bc6518700 1 --2- 172.21.15.36:0/3544574731 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5bc013fde0 0x7f5bc00b4d40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f5bb000c000 tx=0x7f5bb0006df0).stop 2022-06-17T12:47:52.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.009+0000 7f5bc6518700 1 -- 172.21.15.36:0/3544574731 shutdown_connections 2022-06-17T12:47:52.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.009+0000 7f5bc6518700 1 --2- 172.21.15.36:0/3544574731 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5ba4053af0 0x7f5ba4055fb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:52.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.009+0000 7f5bc6518700 1 --2- 172.21.15.36:0/3544574731 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5bc013fde0 0x7f5bc00b4d40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:52.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.009+0000 7f5bc6518700 1 --2- 172.21.15.36:0/3544574731 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5bc00ae270 0x7f5bc01412b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:52.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.009+0000 7f5bc6518700 1 --2- 172.21.15.36:0/3544574731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bc00ad9c0 0x7f5bc0063630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:52.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.009+0000 7f5bc6518700 1 -- 172.21.15.36:0/3544574731 >> 172.21.15.36:0/3544574731 conn(0x7f5bc005c550 msgr2=0x7f5bc0140740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:52.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.009+0000 7f5bc6518700 1 -- 172.21.15.36:0/3544574731 shutdown_connections 2022-06-17T12:47:52.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.009+0000 7f5bc6518700 1 -- 172.21.15.36:0/3544574731 wait complete. 2022-06-17T12:47:52.187 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 14 2022-06-17T12:47:52.408 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:47:47.072139+0000', 'modified': '2022-06-17T12:47:49.085163+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4947}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4947': {'gid': 4947, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.36:6835/1648255567', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 1648255567}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 1648255567}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:47:52.409 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:47:52.410 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:47:52.533 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":4947,"want_state":"up:active","state":"up:active","rank_uptime":5.4642253729999997,"mdsmap_epoch":14,"osdmap_epoch":35,"osdmap_epoch_barrier":35,"uptime":13.664691348} 2022-06-17T12:47:52.555 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 4947, "mdsmap_epoch": 14, "osdmap_epoch": 35, "osdmap_epoch_barrier": 35, "rank_uptime": 5.464225373, "state": "up:active", "uptime": 13.664691348, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:47:52.556 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:47:52.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.685+0000 7ffb59ab7700 1 Processor -- start 2022-06-17T12:47:52.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.685+0000 7ffb59ab7700 1 -- start start 2022-06-17T12:47:52.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb59ab7700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ffb5413cb70 0x7ffb54143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:52.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb59ab7700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffb540aef60 0x7ffb540b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:52.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb59ab7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb540ba2e0 0x7ffb541442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:52.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb59ab7700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb540af4b0 con 0x7ffb5413cb70 2022-06-17T12:47:52.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb59ab7700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb54142f50 con 0x7ffb540aef60 2022-06-17T12:47:52.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb59ab7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb541431f0 con 0x7ffb540ba2e0 2022-06-17T12:47:52.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb58ab5700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ffb5413cb70 0x7ffb54143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:52.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb58ab5700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ffb5413cb70 0x7ffb54143690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32816/0 (socket says 172.21.15.36:32816) 2022-06-17T12:47:52.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb58ab5700 1 -- 172.21.15.36:0/3287641116 learned_addr learned my addr 172.21.15.36:0/3287641116 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:52.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb53fff700 1 --2- 172.21.15.36:0/3287641116 >> v2:172.21.15.70:3300/0 conn(0x7ffb540aef60 0x7ffb540b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:52.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb592b6700 1 --2- 172.21.15.36:0/3287641116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb540ba2e0 0x7ffb541442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:52.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb58ab5700 1 -- 172.21.15.36:0/3287641116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb540ba2e0 msgr2=0x7ffb541442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:52.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb58ab5700 1 --2- 172.21.15.36:0/3287641116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb540ba2e0 0x7ffb541442a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:52.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb58ab5700 1 -- 172.21.15.36:0/3287641116 >> v2:172.21.15.70:3300/0 conn(0x7ffb540aef60 msgr2=0x7ffb540b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:52.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb58ab5700 1 --2- 172.21.15.36:0/3287641116 >> v2:172.21.15.70:3300/0 conn(0x7ffb540aef60 0x7ffb540b9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:52.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb58ab5700 1 -- 172.21.15.36:0/3287641116 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb54062e30 con 0x7ffb5413cb70 2022-06-17T12:47:52.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb58ab5700 1 --2- 172.21.15.36:0/3287641116 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb5413cb70 0x7ffb54143690 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7ffb44002f70 tx=0x7ffb44006eb0).ready entity=mon.0 client_cookie=160751571cbd6065 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:52.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb537fe700 1 -- 172.21.15.36:0/3287641116 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb4400d570 con 0x7ffb5413cb70 2022-06-17T12:47:52.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb537fe700 1 -- 172.21.15.36:0/3287641116 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb44000e90 con 0x7ffb5413cb70 2022-06-17T12:47:52.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb537fe700 1 -- 172.21.15.36:0/3287641116 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb4400d900 con 0x7ffb5413cb70 2022-06-17T12:47:52.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb59ab7700 1 -- 172.21.15.36:0/3287641116 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb5413cb70 msgr2=0x7ffb54143690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:52.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb59ab7700 1 --2- 172.21.15.36:0/3287641116 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb5413cb70 0x7ffb54143690 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7ffb44002f70 tx=0x7ffb44006eb0).stop 2022-06-17T12:47:52.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb59ab7700 1 -- 172.21.15.36:0/3287641116 shutdown_connections 2022-06-17T12:47:52.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb59ab7700 1 --2- 172.21.15.36:0/3287641116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb540ba2e0 0x7ffb541442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:52.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb59ab7700 1 --2- 172.21.15.36:0/3287641116 >> v2:172.21.15.70:3300/0 conn(0x7ffb540aef60 0x7ffb540b9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:52.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb59ab7700 1 --2- 172.21.15.36:0/3287641116 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb5413cb70 0x7ffb54143690 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:52.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb59ab7700 1 -- 172.21.15.36:0/3287641116 >> 172.21.15.36:0/3287641116 conn(0x7ffb5405c550 msgr2=0x7ffb540b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:52.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb59ab7700 1 -- 172.21.15.36:0/3287641116 shutdown_connections 2022-06-17T12:47:52.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.689+0000 7ffb59ab7700 1 -- 172.21.15.36:0/3287641116 wait complete. 2022-06-17T12:47:52.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb59ab7700 1 Processor -- start 2022-06-17T12:47:52.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb59ab7700 1 -- start start 2022-06-17T12:47:52.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb59ab7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb540aef60 0x7ffb54152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:52.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb59ab7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb540ba2e0 0x7ffb5413f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:52.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb59ab7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb5413cb70 0x7ffb54141230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:52.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb59ab7700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5413e1f0 con 0x7ffb540ba2e0 2022-06-17T12:47:52.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb59ab7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb54153350 con 0x7ffb5413cb70 2022-06-17T12:47:52.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb59ab7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5413fbe0 con 0x7ffb540aef60 2022-06-17T12:47:52.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb53fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb540ba2e0 0x7ffb5413f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:52.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb53fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb540ba2e0 0x7ffb5413f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32820/0 (socket says 172.21.15.36:32820) 2022-06-17T12:47:52.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb53fff700 1 -- 172.21.15.36:0/3638690551 learned_addr learned my addr 172.21.15.36:0/3638690551 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:52.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb592b6700 1 --2- 172.21.15.36:0/3638690551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb5413cb70 0x7ffb54141230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:52.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb58ab5700 1 --2- 172.21.15.36:0/3638690551 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb540aef60 0x7ffb54152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:52.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb53fff700 1 -- 172.21.15.36:0/3638690551 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb540aef60 msgr2=0x7ffb54152e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:52.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb53fff700 1 --2- 172.21.15.36:0/3638690551 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb540aef60 0x7ffb54152e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:52.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb53fff700 1 -- 172.21.15.36:0/3638690551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb5413cb70 msgr2=0x7ffb54141230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:52.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb53fff700 1 --2- 172.21.15.36:0/3638690551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb5413cb70 0x7ffb54141230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:52.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb53fff700 1 -- 172.21.15.36:0/3638690551 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4400b1e0 con 0x7ffb540ba2e0 2022-06-17T12:47:52.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb592b6700 1 --2- 172.21.15.36:0/3638690551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb5413cb70 0x7ffb54141230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:52.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb53fff700 1 --2- 172.21.15.36:0/3638690551 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb540ba2e0 0x7ffb5413f1f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7ffb54140490 tx=0x7ffb3c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:52.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb51ffb700 1 -- 172.21.15.36:0/3638690551 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb3c00c850 con 0x7ffb540ba2e0 2022-06-17T12:47:52.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb51ffb700 1 -- 172.21.15.36:0/3638690551 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3c00c9b0 con 0x7ffb540ba2e0 2022-06-17T12:47:52.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb59ab7700 1 -- 172.21.15.36:0/3638690551 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb5413eb80 con 0x7ffb540ba2e0 2022-06-17T12:47:52.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb51ffb700 1 -- 172.21.15.36:0/3638690551 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb3c00cd10 con 0x7ffb540ba2e0 2022-06-17T12:47:52.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb59ab7700 1 -- 172.21.15.36:0/3638690551 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5413d800 con 0x7ffb540ba2e0 2022-06-17T12:47:52.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.693+0000 7ffb59ab7700 1 -- 172.21.15.36:0/3638690551 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb54057410 con 0x7ffb540ba2e0 2022-06-17T12:47:52.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.697+0000 7ffb51ffb700 1 -- 172.21.15.36:0/3638690551 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ffb3c022070 con 0x7ffb540ba2e0 2022-06-17T12:47:52.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.697+0000 7ffb51ffb700 1 --2- 172.21.15.36:0/3638690551 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffb38053a40 0x7ffb38055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:52.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.697+0000 7ffb51ffb700 1 -- 172.21.15.36:0/3638690551 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ffb3c06c730 con 0x7ffb540ba2e0 2022-06-17T12:47:52.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.697+0000 7ffb58ab5700 1 --2- 172.21.15.36:0/3638690551 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffb38053a40 0x7ffb38055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:52.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.697+0000 7ffb58ab5700 1 --2- 172.21.15.36:0/3638690551 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffb38053a40 0x7ffb38055f00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7ffb4401c000 tx=0x7ffb44015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:52.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.701+0000 7ffb51ffb700 1 -- 172.21.15.36:0/3638690551 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffb3c043530 con 0x7ffb540ba2e0 2022-06-17T12:47:52.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.829+0000 7ffb59ab7700 1 -- 172.21.15.36:0/3638690551 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb54062bb0 con 0x7ffb540ba2e0 2022-06-17T12:47:52.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.829+0000 7ffb51ffb700 1 -- 172.21.15.36:0/3638690551 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16271 (secure 0 0 0) 0x7ffb3c046b50 con 0x7ffb540ba2e0 2022-06-17T12:47:52.867 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:52.868 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":35,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:47:48.075718+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:47:45.085233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2022-06-17T12:47:46.070152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:47:52.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.833+0000 7ffb59ab7700 1 -- 172.21.15.36:0/3638690551 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffb38053a40 msgr2=0x7ffb38055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:52.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.833+0000 7ffb59ab7700 1 --2- 172.21.15.36:0/3638690551 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffb38053a40 0x7ffb38055f00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7ffb4401c000 tx=0x7ffb44015000).stop 2022-06-17T12:47:52.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.833+0000 7ffb59ab7700 1 -- 172.21.15.36:0/3638690551 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb540ba2e0 msgr2=0x7ffb5413f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:52.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.833+0000 7ffb59ab7700 1 --2- 172.21.15.36:0/3638690551 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb540ba2e0 0x7ffb5413f1f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7ffb54140490 tx=0x7ffb3c0091d0).stop 2022-06-17T12:47:52.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.833+0000 7ffb59ab7700 1 -- 172.21.15.36:0/3638690551 shutdown_connections 2022-06-17T12:47:52.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.833+0000 7ffb59ab7700 1 --2- 172.21.15.36:0/3638690551 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffb38053a40 0x7ffb38055f00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:52.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.833+0000 7ffb59ab7700 1 --2- 172.21.15.36:0/3638690551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb5413cb70 0x7ffb54141230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:52.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.833+0000 7ffb59ab7700 1 --2- 172.21.15.36:0/3638690551 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb540ba2e0 0x7ffb5413f1f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:52.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.833+0000 7ffb59ab7700 1 --2- 172.21.15.36:0/3638690551 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb540aef60 0x7ffb54152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:52.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.833+0000 7ffb59ab7700 1 -- 172.21.15.36:0/3638690551 >> 172.21.15.36:0/3638690551 conn(0x7ffb5405c550 msgr2=0x7ffb5413c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:52.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.833+0000 7ffb59ab7700 1 -- 172.21.15.36:0/3638690551 shutdown_connections 2022-06-17T12:47:52.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:52.833+0000 7ffb59ab7700 1 -- 172.21.15.36:0/3638690551 wait complete. 2022-06-17T12:47:52.880 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:47:45.085233+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:47:46.070152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:47:52.882 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:47:53.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.013+0000 7f8f9e937700 1 Processor -- start 2022-06-17T12:47:53.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.013+0000 7f8f9e937700 1 -- start start 2022-06-17T12:47:53.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.013+0000 7f8f9e937700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f98060ac0 0x7f8f98140d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:53.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.013+0000 7f8f9e937700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8f980b2c10 0x7f8f980b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:53.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.013+0000 7f8f9e937700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f980b05e0 con 0x7f8f980b11a0 2022-06-17T12:47:53.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.013+0000 7f8f9e937700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f98060ea0 con 0x7f8f980b2c10 2022-06-17T12:47:53.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.013+0000 7f8f9e937700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f981450f0 con 0x7f8f98060ac0 2022-06-17T12:47:53.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.013+0000 7f8f9d134700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8f980b2c10 0x7f8f980b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:53.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.013+0000 7f8f9e136700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f8f980b11a0 0x7f8f981299a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50360/0 (socket says 172.21.15.36:50360) 2022-06-17T12:47:53.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.013+0000 7f8f9e136700 1 -- 172.21.15.36:0/703069246 learned_addr learned my addr 172.21.15.36:0/703069246 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:53.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.013+0000 7f8f9d935700 1 --2- 172.21.15.36:0/703069246 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f98060ac0 0x7f8f98140d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:53.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.013+0000 7f8f9c933700 1 -- 172.21.15.36:0/703069246 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 62039516 0 0) 0x7f8f980b05e0 con 0x7f8f980b11a0 2022-06-17T12:47:53.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.013+0000 7f8f9c933700 1 -- 172.21.15.36:0/703069246 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f80003260 con 0x7f8f980b11a0 2022-06-17T12:47:53.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.013+0000 7f8f9d134700 1 -- 172.21.15.36:0/703069246 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f98060ac0 msgr2=0x7f8f98140d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:53.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.013+0000 7f8f9d134700 1 --2- 172.21.15.36:0/703069246 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f98060ac0 0x7f8f98140d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:53.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.013+0000 7f8f9d134700 1 -- 172.21.15.36:0/703069246 >> v1:172.21.15.36:6789/0 conn(0x7f8f980b11a0 legacy=0x7f8f981299a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:53.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.013+0000 7f8f9c933700 1 -- 172.21.15.36:0/703069246 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3490539412 0 0) 0x7f8f80003260 con 0x7f8f980b11a0 2022-06-17T12:47:53.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.013+0000 7f8f9d134700 1 -- 172.21.15.36:0/703069246 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f980b0810 con 0x7f8f980b2c10 2022-06-17T12:47:53.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.017+0000 7f8f9d134700 1 --2- 172.21.15.36:0/703069246 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f980b2c10 0x7f8f980b0cb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f8f90001800 tx=0x7f8f8c006df0).ready entity=mon.0 client_cookie=98c4c281586ae62e server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:53.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.017+0000 7f8f9c933700 1 -- 172.21.15.36:0/703069246 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f90001660 con 0x7f8f980b2c10 2022-06-17T12:47:53.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.017+0000 7f8f9c933700 1 -- 172.21.15.36:0/703069246 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8c000e90 con 0x7f8f980b2c10 2022-06-17T12:47:53.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.017+0000 7f8f9c933700 1 -- 172.21.15.36:0/703069246 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f8c012460 con 0x7f8f980b2c10 2022-06-17T12:47:53.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.017+0000 7f8f9e937700 1 -- 172.21.15.36:0/703069246 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f980b2c10 msgr2=0x7f8f980b0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:53.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.017+0000 7f8f9e937700 1 --2- 172.21.15.36:0/703069246 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f980b2c10 0x7f8f980b0cb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f8f90001800 tx=0x7f8f8c006df0).stop 2022-06-17T12:47:53.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.017+0000 7f8f9e937700 1 -- 172.21.15.36:0/703069246 shutdown_connections 2022-06-17T12:47:53.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.017+0000 7f8f9e937700 1 --2- 172.21.15.36:0/703069246 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f980b2c10 0x7f8f980b0cb0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:53.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.017+0000 7f8f9e937700 1 --2- 172.21.15.36:0/703069246 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f98060ac0 0x7f8f98140d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:53.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.017+0000 7f8f9e937700 1 -- 172.21.15.36:0/703069246 >> 172.21.15.36:0/703069246 conn(0x7f8f980b64f0 msgr2=0x7f8f980b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:53.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.017+0000 7f8f9e937700 1 -- 172.21.15.36:0/703069246 shutdown_connections 2022-06-17T12:47:53.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.017+0000 7f8f9e937700 1 -- 172.21.15.36:0/703069246 wait complete. 2022-06-17T12:47:53.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.017+0000 7f8f9e937700 1 Processor -- start 2022-06-17T12:47:53.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.017+0000 7f8f9e937700 1 -- start start 2022-06-17T12:47:53.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.021+0000 7f8f9e937700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f98060ac0 0x7f8f98144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:53.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.021+0000 7f8f9e937700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f980b11a0 0x7f8f980aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:53.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.021+0000 7f8f9e937700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f980b2c10 0x7f8f98067110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:53.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.021+0000 7f8f9e937700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f98065650 con 0x7f8f980b11a0 2022-06-17T12:47:53.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.021+0000 7f8f9e937700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f98064cf0 con 0x7f8f980b2c10 2022-06-17T12:47:53.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.021+0000 7f8f9e937700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f980af380 con 0x7f8f98060ac0 2022-06-17T12:47:53.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.021+0000 7f8f9d134700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f980b11a0 0x7f8f980aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:53.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.021+0000 7f8f9d134700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f980b11a0 0x7f8f980aee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32836/0 (socket says 172.21.15.36:32836) 2022-06-17T12:47:53.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.021+0000 7f8f9d134700 1 -- 172.21.15.36:0/1895010674 learned_addr learned my addr 172.21.15.36:0/1895010674 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:53.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.021+0000 7f8f9e136700 1 --2- 172.21.15.36:0/1895010674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f980b2c10 0x7f8f98067110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:53.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.021+0000 7f8f9d935700 1 --2- 172.21.15.36:0/1895010674 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f98060ac0 0x7f8f98144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:53.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.021+0000 7f8f9e136700 1 -- 172.21.15.36:0/1895010674 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f98060ac0 msgr2=0x7f8f98144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:53.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.021+0000 7f8f9e136700 1 --2- 172.21.15.36:0/1895010674 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f98060ac0 0x7f8f98144600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:53.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.021+0000 7f8f9e136700 1 -- 172.21.15.36:0/1895010674 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f980b11a0 msgr2=0x7f8f980aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:53.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.021+0000 7f8f9e136700 1 --2- 172.21.15.36:0/1895010674 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f980b11a0 0x7f8f980aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:53.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.021+0000 7f8f9e136700 1 -- 172.21.15.36:0/1895010674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8c00b180 con 0x7f8f980b2c10 2022-06-17T12:47:53.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.021+0000 7f8f9e136700 1 --2- 172.21.15.36:0/1895010674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f980b2c10 0x7f8f98067110 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f8f900018c0 tx=0x7f8f90001dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:53.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.021+0000 7f8f8affd700 1 -- 172.21.15.36:0/1895010674 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f90007170 con 0x7f8f980b2c10 2022-06-17T12:47:53.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.021+0000 7f8f8affd700 1 -- 172.21.15.36:0/1895010674 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f900072d0 con 0x7f8f980b2c10 2022-06-17T12:47:53.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.021+0000 7f8f8affd700 1 -- 172.21.15.36:0/1895010674 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f90004710 con 0x7f8f980b2c10 2022-06-17T12:47:53.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.021+0000 7f8f9e937700 1 -- 172.21.15.36:0/1895010674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f980afd50 con 0x7f8f980b2c10 2022-06-17T12:47:53.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.021+0000 7f8f9e937700 1 -- 172.21.15.36:0/1895010674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f98067600 con 0x7f8f980b2c10 2022-06-17T12:47:53.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.021+0000 7f8f9e937700 1 -- 172.21.15.36:0/1895010674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f98063d00 con 0x7f8f980b2c10 2022-06-17T12:47:53.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.025+0000 7f8f8affd700 1 -- 172.21.15.36:0/1895010674 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8f90004870 con 0x7f8f980b2c10 2022-06-17T12:47:53.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.025+0000 7f8f8affd700 1 --2- 172.21.15.36:0/1895010674 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8f78053920 0x7f8f78055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:53.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.025+0000 7f8f8affd700 1 -- 172.21.15.36:0/1895010674 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8f9006b7a0 con 0x7f8f980b2c10 2022-06-17T12:47:53.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.025+0000 7f8f9d935700 1 --2- 172.21.15.36:0/1895010674 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8f78053920 0x7f8f78055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:53.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.025+0000 7f8f9d935700 1 --2- 172.21.15.36:0/1895010674 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8f78053920 0x7f8f78055de0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f8f9400b000 tx=0x7f8f940091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:53.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.033+0000 7f8f8affd700 1 -- 172.21.15.36:0/1895010674 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8f98063d00 con 0x7f8f980b2c10 2022-06-17T12:47:53.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.161+0000 7f8f9e937700 1 -- 172.21.15.36:0/1895010674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f8f98063d00 con 0x7f8f980b2c10 2022-06-17T12:47:53.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.161+0000 7f8f8affd700 1 -- 172.21.15.36:0/1895010674 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v35) v1 ==== 110+0+10 (secure 0 0 0) 0x7f8f98063d00 con 0x7f8f980b2c10 2022-06-17T12:47:53.304 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:47:53.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.165+0000 7f8f9e937700 1 -- 172.21.15.36:0/1895010674 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8f78053920 msgr2=0x7f8f78055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:53.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.165+0000 7f8f9e937700 1 --2- 172.21.15.36:0/1895010674 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8f78053920 0x7f8f78055de0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f8f9400b000 tx=0x7f8f940091d0).stop 2022-06-17T12:47:53.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.165+0000 7f8f9e937700 1 -- 172.21.15.36:0/1895010674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f980b2c10 msgr2=0x7f8f98067110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:53.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.165+0000 7f8f9e937700 1 --2- 172.21.15.36:0/1895010674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f980b2c10 0x7f8f98067110 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f8f900018c0 tx=0x7f8f90001dc0).stop 2022-06-17T12:47:53.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.165+0000 7f8f9e937700 1 -- 172.21.15.36:0/1895010674 shutdown_connections 2022-06-17T12:47:53.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.165+0000 7f8f9e937700 1 --2- 172.21.15.36:0/1895010674 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8f78053920 0x7f8f78055de0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:53.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.165+0000 7f8f9e937700 1 --2- 172.21.15.36:0/1895010674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f980b2c10 0x7f8f98067110 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:53.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.165+0000 7f8f9e937700 1 --2- 172.21.15.36:0/1895010674 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f980b11a0 0x7f8f980aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:53.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.165+0000 7f8f9e937700 1 --2- 172.21.15.36:0/1895010674 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f98060ac0 0x7f8f98144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:53.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.165+0000 7f8f9e937700 1 -- 172.21.15.36:0/1895010674 >> 172.21.15.36:0/1895010674 conn(0x7f8f980b64f0 msgr2=0x7f8f981485a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:53.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.165+0000 7f8f9e937700 1 -- 172.21.15.36:0/1895010674 shutdown_connections 2022-06-17T12:47:53.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.165+0000 7f8f9e937700 1 -- 172.21.15.36:0/1895010674 wait complete. 2022-06-17T12:47:53.310 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:47:53.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.505+0000 7f14cffff700 1 Processor -- start 2022-06-17T12:47:53.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.505+0000 7f14cffff700 1 -- start start 2022-06-17T12:47:53.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.505+0000 7f14cffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f14d00b45c0 0x7f14d00b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:53.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.505+0000 7f14cffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14d00b4f00 0x7f14d00b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:53.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.505+0000 7f14cffff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d00b4160 con 0x7f14d00b9860 2022-06-17T12:47:53.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.505+0000 7f14cffff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d00b9c40 con 0x7f14d00b45c0 2022-06-17T12:47:53.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.505+0000 7f14cffff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d00627d0 con 0x7f14d00b4f00 2022-06-17T12:47:53.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.505+0000 7f14ce7fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f14d00b45c0 0x7f14d00b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:53.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.505+0000 7f14ce7fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f14d00b45c0 0x7f14d00b4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32844/0 (socket says 172.21.15.36:32844) 2022-06-17T12:47:53.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.505+0000 7f14ce7fc700 1 -- 172.21.15.36:0/2316598802 learned_addr learned my addr 172.21.15.36:0/2316598802 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:53.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.505+0000 7f14cf7fe700 1 --2- 172.21.15.36:0/2316598802 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14d00b4f00 0x7f14d00b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:53.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.509+0000 7f14ce7fc700 1 -- 172.21.15.36:0/2316598802 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14d00b4f00 msgr2=0x7f14d00b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:53.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.509+0000 7f14ce7fc700 1 --2- 172.21.15.36:0/2316598802 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14d00b4f00 0x7f14d00b5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:53.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.509+0000 7f14ce7fc700 1 -- 172.21.15.36:0/2316598802 >> v1:172.21.15.70:6789/0 conn(0x7f14d00b9860 legacy=0x7f14d0143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:53.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.509+0000 7f14ce7fc700 1 -- 172.21.15.36:0/2316598802 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d0062a70 con 0x7f14d00b45c0 2022-06-17T12:47:53.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.509+0000 7f14ce7fc700 1 --2- 172.21.15.36:0/2316598802 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14d00b45c0 0x7f14d00b4a10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f14b8002f70 tx=0x7f14b8006eb0).ready entity=mon.0 client_cookie=20d637cf1c7bb3d1 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:53.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.509+0000 7f14cdffb700 1 -- 172.21.15.36:0/2316598802 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14b800d570 con 0x7f14d00b45c0 2022-06-17T12:47:53.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.509+0000 7f14cdffb700 1 -- 172.21.15.36:0/2316598802 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b8000e90 con 0x7f14d00b45c0 2022-06-17T12:47:53.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.509+0000 7f14cdffb700 1 -- 172.21.15.36:0/2316598802 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14b800d900 con 0x7f14d00b45c0 2022-06-17T12:47:53.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.509+0000 7f14cffff700 1 -- 172.21.15.36:0/2316598802 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14d00b45c0 msgr2=0x7f14d00b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:53.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.509+0000 7f14cffff700 1 --2- 172.21.15.36:0/2316598802 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14d00b45c0 0x7f14d00b4a10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f14b8002f70 tx=0x7f14b8006eb0).stop 2022-06-17T12:47:53.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.509+0000 7f14cffff700 1 -- 172.21.15.36:0/2316598802 shutdown_connections 2022-06-17T12:47:53.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.509+0000 7f14cffff700 1 --2- 172.21.15.36:0/2316598802 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14d00b4f00 0x7f14d00b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:53.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.509+0000 7f14cffff700 1 --2- 172.21.15.36:0/2316598802 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14d00b45c0 0x7f14d00b4a10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:53.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.509+0000 7f14cffff700 1 -- 172.21.15.36:0/2316598802 >> 172.21.15.36:0/2316598802 conn(0x7f14d005c550 msgr2=0x7f14d0060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:53.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.509+0000 7f14cffff700 1 -- 172.21.15.36:0/2316598802 shutdown_connections 2022-06-17T12:47:53.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.509+0000 7f14cffff700 1 -- 172.21.15.36:0/2316598802 wait complete. 2022-06-17T12:47:53.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.509+0000 7f14cffff700 1 Processor -- start 2022-06-17T12:47:53.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.509+0000 7f14cffff700 1 -- start start 2022-06-17T12:47:53.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.513+0000 7f14cffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14d00b45c0 0x7f14d00afe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:53.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.513+0000 7f14cffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14d00b4f00 0x7f14d0155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:53.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.513+0000 7f14cffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14d00b9860 0x7f14d013dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:53.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.513+0000 7f14cffff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d013d590 con 0x7f14d00b45c0 2022-06-17T12:47:53.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.513+0000 7f14cffff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d013d830 con 0x7f14d00b4f00 2022-06-17T12:47:53.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.513+0000 7f14cffff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d0156500 con 0x7f14d00b9860 2022-06-17T12:47:53.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.513+0000 7f14ceffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14d00b45c0 0x7f14d00afe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:53.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.513+0000 7f14ceffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14d00b45c0 0x7f14d00afe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32848/0 (socket says 172.21.15.36:32848) 2022-06-17T12:47:53.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.513+0000 7f14ceffd700 1 -- 172.21.15.36:0/4064329127 learned_addr learned my addr 172.21.15.36:0/4064329127 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:53.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.513+0000 7f14ce7fc700 1 --2- 172.21.15.36:0/4064329127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14d00b4f00 0x7f14d0155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:53.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.513+0000 7f14cf7fe700 1 --2- 172.21.15.36:0/4064329127 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14d00b9860 0x7f14d013dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:53.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.513+0000 7f14cf7fe700 1 -- 172.21.15.36:0/4064329127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14d00b4f00 msgr2=0x7f14d0155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:53.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.513+0000 7f14cf7fe700 1 --2- 172.21.15.36:0/4064329127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14d00b4f00 0x7f14d0155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:53.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.513+0000 7f14cf7fe700 1 -- 172.21.15.36:0/4064329127 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14d00b45c0 msgr2=0x7f14d00afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:53.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.513+0000 7f14cf7fe700 1 --2- 172.21.15.36:0/4064329127 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14d00b45c0 0x7f14d00afe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:53.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.513+0000 7f14cf7fe700 1 -- 172.21.15.36:0/4064329127 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b800b1e0 con 0x7f14d00b9860 2022-06-17T12:47:53.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.513+0000 7f14ceffd700 1 --2- 172.21.15.36:0/4064329127 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14d00b45c0 0x7f14d00afe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:53.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.513+0000 7f14cf7fe700 1 --2- 172.21.15.36:0/4064329127 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14d00b9860 0x7f14d013dcd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f14c800c000 tx=0x7f14c80091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:53.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.513+0000 7f14c7fff700 1 -- 172.21.15.36:0/4064329127 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14c8012600 con 0x7f14d00b9860 2022-06-17T12:47:53.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.513+0000 7f14c7fff700 1 -- 172.21.15.36:0/4064329127 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c8012760 con 0x7f14d00b9860 2022-06-17T12:47:53.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.513+0000 7f14cffff700 1 -- 172.21.15.36:0/4064329127 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d00aecc0 con 0x7f14d00b9860 2022-06-17T12:47:53.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.513+0000 7f14c7fff700 1 -- 172.21.15.36:0/4064329127 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14c8012ac0 con 0x7f14d00b9860 2022-06-17T12:47:53.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.513+0000 7f14cffff700 1 -- 172.21.15.36:0/4064329127 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d0152100 con 0x7f14d00b9860 2022-06-17T12:47:53.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.517+0000 7f14cffff700 1 -- 172.21.15.36:0/4064329127 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d013d9f0 con 0x7f14d00b9860 2022-06-17T12:47:53.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.517+0000 7f14c7fff700 1 -- 172.21.15.36:0/4064329127 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f14c8022070 con 0x7f14d00b9860 2022-06-17T12:47:53.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.517+0000 7f14c7fff700 1 --2- 172.21.15.36:0/4064329127 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f14b4053ad0 0x7f14b4055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:53.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.517+0000 7f14ceffd700 1 --2- 172.21.15.36:0/4064329127 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f14b4053ad0 0x7f14b4055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:53.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.517+0000 7f14c7fff700 1 -- 172.21.15.36:0/4064329127 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f14c806c6b0 con 0x7f14d00b9860 2022-06-17T12:47:53.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.517+0000 7f14ceffd700 1 --2- 172.21.15.36:0/4064329127 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f14b4053ad0 0x7f14b4055f90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f14c000d000 tx=0x7f14c0007c20).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:53.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.525+0000 7f14c7fff700 1 -- 172.21.15.36:0/4064329127 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f14c8043340 con 0x7f14d00b9860 2022-06-17T12:47:53.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.661+0000 7f14cffff700 1 -- 172.21.15.36:0/4064329127 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f14d0062eb0 con 0x7f14d00b9860 2022-06-17T12:47:53.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.661+0000 7f14c7fff700 1 -- 172.21.15.36:0/4064329127 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v35) v1 ==== 92+0+10 (secure 0 0 0) 0x7f14c8016020 con 0x7f14d00b9860 2022-06-17T12:47:53.889 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:53.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.661+0000 7f14cffff700 1 -- 172.21.15.36:0/4064329127 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f14b4053ad0 msgr2=0x7f14b4055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:53.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.661+0000 7f14cffff700 1 --2- 172.21.15.36:0/4064329127 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f14b4053ad0 0x7f14b4055f90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f14c000d000 tx=0x7f14c0007c20).stop 2022-06-17T12:47:53.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.661+0000 7f14cffff700 1 -- 172.21.15.36:0/4064329127 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14d00b9860 msgr2=0x7f14d013dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:53.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.661+0000 7f14cffff700 1 --2- 172.21.15.36:0/4064329127 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14d00b9860 0x7f14d013dcd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f14c800c000 tx=0x7f14c80091d0).stop 2022-06-17T12:47:53.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.661+0000 7f14cffff700 1 -- 172.21.15.36:0/4064329127 shutdown_connections 2022-06-17T12:47:53.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.661+0000 7f14cffff700 1 --2- 172.21.15.36:0/4064329127 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f14b4053ad0 0x7f14b4055f90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:53.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.661+0000 7f14cffff700 1 --2- 172.21.15.36:0/4064329127 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14d00b9860 0x7f14d013dcd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:53.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.661+0000 7f14cffff700 1 --2- 172.21.15.36:0/4064329127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14d00b4f00 0x7f14d0155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:53.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.661+0000 7f14cffff700 1 --2- 172.21.15.36:0/4064329127 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14d00b45c0 0x7f14d00afe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:53.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.661+0000 7f14cffff700 1 -- 172.21.15.36:0/4064329127 >> 172.21.15.36:0/4064329127 conn(0x7f14d005c550 msgr2=0x7f14d00b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:53.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.661+0000 7f14cffff700 1 -- 172.21.15.36:0/4064329127 shutdown_connections 2022-06-17T12:47:53.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:53.661+0000 7f14cffff700 1 -- 172.21.15.36:0/4064329127 wait complete. 2022-06-17T12:47:53.900 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:47:54.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.049+0000 7f1a593e1700 1 Processor -- start 2022-06-17T12:47:54.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.049+0000 7f1a593e1700 1 -- start start 2022-06-17T12:47:54.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.049+0000 7f1a593e1700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1a5413ca70 0x7f1a5413ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:54.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.049+0000 7f1a593e1700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a541432c0 con 0x7f1a54060020 2022-06-17T12:47:54.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.049+0000 7f1a593e1700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a54143df0 con 0x7f1a540b3f80 2022-06-17T12:47:54.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.049+0000 7f1a593e1700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a54062db0 con 0x7f1a5413ca70 2022-06-17T12:47:54.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.049+0000 7f1a53fff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f1a54060020 0x7f1a540626a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50386/0 (socket says 172.21.15.36:50386) 2022-06-17T12:47:54.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.049+0000 7f1a53fff700 1 -- 172.21.15.36:0/3428363038 learned_addr learned my addr 172.21.15.36:0/3428363038 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:47:54.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.049+0000 7f1a58be0700 1 --2- 172.21.15.36:0/3428363038 >> v2:172.21.15.70:3300/0 conn(0x7f1a5413ca70 0x7f1a5413ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:54.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.053+0000 7f1a52ffd700 1 -- 172.21.15.36:0/3428363038 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2716947974 0 0) 0x7f1a541432c0 con 0x7f1a54060020 2022-06-17T12:47:54.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.053+0000 7f1a52ffd700 1 -- 172.21.15.36:0/3428363038 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a40003260 con 0x7f1a54060020 2022-06-17T12:47:54.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.053+0000 7f1a52ffd700 1 -- 172.21.15.36:0/3428363038 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1907055016 0 0) 0x7f1a54143df0 con 0x7f1a540b3f80 2022-06-17T12:47:54.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.053+0000 7f1a52ffd700 1 -- 172.21.15.36:0/3428363038 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a541432c0 con 0x7f1a540b3f80 2022-06-17T12:47:54.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.053+0000 7f1a58be0700 1 -- 172.21.15.36:0/3428363038 >> v1:172.21.15.70:6789/0 conn(0x7f1a540b3f80 legacy=0x7f1a540b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:54.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.053+0000 7f1a58be0700 1 -- 172.21.15.36:0/3428363038 >> v1:172.21.15.36:6789/0 conn(0x7f1a54060020 legacy=0x7f1a540626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:54.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.053+0000 7f1a58be0700 1 -- 172.21.15.36:0/3428363038 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a54061f60 con 0x7f1a5413ca70 2022-06-17T12:47:54.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.053+0000 7f1a58be0700 1 --2- 172.21.15.36:0/3428363038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a5413ca70 0x7f1a5413ceb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f1a44002f70 tx=0x7f1a44006df0).ready entity=mon.1 client_cookie=75577132aa3550b4 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:54.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.053+0000 7f1a52ffd700 1 -- 172.21.15.36:0/3428363038 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a4400d540 con 0x7f1a5413ca70 2022-06-17T12:47:54.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.053+0000 7f1a52ffd700 1 -- 172.21.15.36:0/3428363038 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a44000e90 con 0x7f1a5413ca70 2022-06-17T12:47:54.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.053+0000 7f1a52ffd700 1 -- 172.21.15.36:0/3428363038 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a4400d890 con 0x7f1a5413ca70 2022-06-17T12:47:54.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.053+0000 7f1a593e1700 1 -- 172.21.15.36:0/3428363038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a5413ca70 msgr2=0x7f1a5413ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:54.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.053+0000 7f1a593e1700 1 --2- 172.21.15.36:0/3428363038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a5413ca70 0x7f1a5413ceb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f1a44002f70 tx=0x7f1a44006df0).stop 2022-06-17T12:47:54.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.053+0000 7f1a593e1700 1 -- 172.21.15.36:0/3428363038 shutdown_connections 2022-06-17T12:47:54.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.053+0000 7f1a593e1700 1 --2- 172.21.15.36:0/3428363038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a5413ca70 0x7f1a5413ceb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.053+0000 7f1a593e1700 1 -- 172.21.15.36:0/3428363038 >> 172.21.15.36:0/3428363038 conn(0x7f1a5405c550 msgr2=0x7f1a5405f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:54.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.053+0000 7f1a593e1700 1 -- 172.21.15.36:0/3428363038 shutdown_connections 2022-06-17T12:47:54.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.053+0000 7f1a593e1700 1 -- 172.21.15.36:0/3428363038 wait complete. 2022-06-17T12:47:54.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a593e1700 1 Processor -- start 2022-06-17T12:47:54.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a593e1700 1 -- start start 2022-06-17T12:47:54.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a593e1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a54060020 0x7f1a54152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:54.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a593e1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a540b3f80 0x7f1a540b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:54.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a593e1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a5413ca70 0x7f1a5413c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:54.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a593e1700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a54156460 con 0x7f1a540b3f80 2022-06-17T12:47:54.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a593e1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a54142890 con 0x7f1a5413ca70 2022-06-17T12:47:54.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a593e1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a54142d30 con 0x7f1a54060020 2022-06-17T12:47:54.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a537fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a540b3f80 0x7f1a540b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:54.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a537fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a540b3f80 0x7f1a540b83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32864/0 (socket says 172.21.15.36:32864) 2022-06-17T12:47:54.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a537fe700 1 -- 172.21.15.36:0/898340298 learned_addr learned my addr 172.21.15.36:0/898340298 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:54.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a53fff700 1 --2- 172.21.15.36:0/898340298 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a54060020 0x7f1a54152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:54.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a58be0700 1 --2- 172.21.15.36:0/898340298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a5413ca70 0x7f1a5413c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:54.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a537fe700 1 -- 172.21.15.36:0/898340298 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a54060020 msgr2=0x7f1a54152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:54.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a537fe700 1 --2- 172.21.15.36:0/898340298 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a54060020 0x7f1a54152100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a537fe700 1 -- 172.21.15.36:0/898340298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a5413ca70 msgr2=0x7f1a5413c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:54.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a537fe700 1 --2- 172.21.15.36:0/898340298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a5413ca70 0x7f1a5413c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a537fe700 1 -- 172.21.15.36:0/898340298 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4400b180 con 0x7f1a540b3f80 2022-06-17T12:47:54.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a58be0700 1 --2- 172.21.15.36:0/898340298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a5413ca70 0x7f1a5413c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:54.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a537fe700 1 --2- 172.21.15.36:0/898340298 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a540b3f80 0x7f1a540b83a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f1a48001110 tx=0x7f1a48007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:54.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a517fa700 1 -- 172.21.15.36:0/898340298 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a48012400 con 0x7f1a540b3f80 2022-06-17T12:47:54.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a517fa700 1 -- 172.21.15.36:0/898340298 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a48012560 con 0x7f1a540b3f80 2022-06-17T12:47:54.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a593e1700 1 -- 172.21.15.36:0/898340298 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a541560a0 con 0x7f1a540b3f80 2022-06-17T12:47:54.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a517fa700 1 -- 172.21.15.36:0/898340298 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a480128c0 con 0x7f1a540b3f80 2022-06-17T12:47:54.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.057+0000 7f1a593e1700 1 -- 172.21.15.36:0/898340298 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5413ee40 con 0x7f1a540b3f80 2022-06-17T12:47:54.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.061+0000 7f1a593e1700 1 -- 172.21.15.36:0/898340298 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a541426e0 con 0x7f1a540b3f80 2022-06-17T12:47:54.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.061+0000 7f1a517fa700 1 -- 172.21.15.36:0/898340298 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f1a48019030 con 0x7f1a540b3f80 2022-06-17T12:47:54.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.061+0000 7f1a517fa700 1 --2- 172.21.15.36:0/898340298 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1a34053a60 0x7f1a34055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:54.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.061+0000 7f1a517fa700 1 -- 172.21.15.36:0/898340298 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1a4806b360 con 0x7f1a540b3f80 2022-06-17T12:47:54.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.061+0000 7f1a53fff700 1 --2- 172.21.15.36:0/898340298 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1a34053a60 0x7f1a34055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:54.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.061+0000 7f1a53fff700 1 --2- 172.21.15.36:0/898340298 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1a34053a60 0x7f1a34055f20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f1a4c00d000 tx=0x7f1a4c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:54.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.069+0000 7f1a517fa700 1 -- 172.21.15.36:0/898340298 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1a48042160 con 0x7f1a540b3f80 2022-06-17T12:47:54.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.189+0000 7f1a593e1700 1 -- 172.21.15.36:0/898340298 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1a54062200 con 0x7f1a540b3f80 2022-06-17T12:47:54.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.189+0000 7f1a517fa700 1 -- 172.21.15.36:0/898340298 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1a48045780 con 0x7f1a540b3f80 2022-06-17T12:47:54.266 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:54.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.193+0000 7f1a593e1700 1 -- 172.21.15.36:0/898340298 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1a34053a60 msgr2=0x7f1a34055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:54.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.193+0000 7f1a593e1700 1 --2- 172.21.15.36:0/898340298 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1a34053a60 0x7f1a34055f20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f1a4c00d000 tx=0x7f1a4c009040).stop 2022-06-17T12:47:54.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.193+0000 7f1a593e1700 1 -- 172.21.15.36:0/898340298 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a540b3f80 msgr2=0x7f1a540b83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:54.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.193+0000 7f1a593e1700 1 --2- 172.21.15.36:0/898340298 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a540b3f80 0x7f1a540b83a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f1a48001110 tx=0x7f1a48007ae0).stop 2022-06-17T12:47:54.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.193+0000 7f1a593e1700 1 -- 172.21.15.36:0/898340298 shutdown_connections 2022-06-17T12:47:54.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.193+0000 7f1a593e1700 1 --2- 172.21.15.36:0/898340298 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1a34053a60 0x7f1a34055f20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.193+0000 7f1a593e1700 1 --2- 172.21.15.36:0/898340298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a5413ca70 0x7f1a5413c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.193+0000 7f1a593e1700 1 --2- 172.21.15.36:0/898340298 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a540b3f80 0x7f1a540b83a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.193+0000 7f1a593e1700 1 --2- 172.21.15.36:0/898340298 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a54060020 0x7f1a54152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.193+0000 7f1a593e1700 1 -- 172.21.15.36:0/898340298 >> 172.21.15.36:0/898340298 conn(0x7f1a5405c550 msgr2=0x7f1a54145d20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:54.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.193+0000 7f1a593e1700 1 -- 172.21.15.36:0/898340298 shutdown_connections 2022-06-17T12:47:54.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.193+0000 7f1a593e1700 1 -- 172.21.15.36:0/898340298 wait complete. 2022-06-17T12:47:54.295 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:47:54.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.461+0000 7ff1c49c1700 1 Processor -- start 2022-06-17T12:47:54.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.461+0000 7ff1c49c1700 1 -- start start 2022-06-17T12:47:54.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1c49c1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1c00b65f0 0x7ff1c00b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:54.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1c49c1700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff1c00b15b0 0x7ff1c00af630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:54.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1c49c1700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c013fdf0 con 0x7ff1c00adf10 2022-06-17T12:47:54.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1c49c1700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1c00ae8e0 con 0x7ff1c00b15b0 2022-06-17T12:47:54.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1c49c1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c0140090 con 0x7ff1c00b65f0 2022-06-17T12:47:54.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1bf59e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff1c00b15b0 0x7ff1c00af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:54.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1bf59e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff1c00b15b0 0x7ff1c00af630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32872/0 (socket says 172.21.15.36:32872) 2022-06-17T12:47:54.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1bf59e700 1 -- 172.21.15.36:0/2031625257 learned_addr learned my addr 172.21.15.36:0/2031625257 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:54.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1bed9d700 1 --2- 172.21.15.36:0/2031625257 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1c00b65f0 0x7ff1c00b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:54.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1bdd9b700 1 -- 172.21.15.36:0/2031625257 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2493318396 0 0) 0x7ff1c013fdf0 con 0x7ff1c00adf10 2022-06-17T12:47:54.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1bdd9b700 1 -- 172.21.15.36:0/2031625257 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1a4003260 con 0x7ff1c00adf10 2022-06-17T12:47:54.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1bf59e700 1 -- 172.21.15.36:0/2031625257 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1c00b65f0 msgr2=0x7ff1c00b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:54.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1bf59e700 1 --2- 172.21.15.36:0/2031625257 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1c00b65f0 0x7ff1c00b69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1bf59e700 1 -- 172.21.15.36:0/2031625257 >> v1:172.21.15.70:6789/0 conn(0x7ff1c00adf10 legacy=0x7ff1c0140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:54.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1bf59e700 1 -- 172.21.15.36:0/2031625257 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c00b1be0 con 0x7ff1c00b15b0 2022-06-17T12:47:54.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1bf59e700 1 --2- 172.21.15.36:0/2031625257 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1c00b15b0 0x7ff1c00af630 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7ff1b80032f0 tx=0x7ff1b8009290).ready entity=mon.0 client_cookie=8ce6c3ac50d83170 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:54.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1bdd9b700 1 -- 172.21.15.36:0/2031625257 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1b800b6c0 con 0x7ff1c00b15b0 2022-06-17T12:47:54.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1bdd9b700 1 -- 172.21.15.36:0/2031625257 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b800b820 con 0x7ff1c00b15b0 2022-06-17T12:47:54.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1bdd9b700 1 -- 172.21.15.36:0/2031625257 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1b800b990 con 0x7ff1c00b15b0 2022-06-17T12:47:54.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1c49c1700 1 -- 172.21.15.36:0/2031625257 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1c00b15b0 msgr2=0x7ff1c00af630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:54.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1c49c1700 1 --2- 172.21.15.36:0/2031625257 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1c00b15b0 0x7ff1c00af630 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7ff1b80032f0 tx=0x7ff1b8009290).stop 2022-06-17T12:47:54.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1c49c1700 1 -- 172.21.15.36:0/2031625257 shutdown_connections 2022-06-17T12:47:54.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1c49c1700 1 --2- 172.21.15.36:0/2031625257 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1c00b15b0 0x7ff1c00af630 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1c49c1700 1 --2- 172.21.15.36:0/2031625257 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1c00b65f0 0x7ff1c00b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1c49c1700 1 -- 172.21.15.36:0/2031625257 >> 172.21.15.36:0/2031625257 conn(0x7ff1c00614b0 msgr2=0x7ff1c0061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:54.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1c49c1700 1 -- 172.21.15.36:0/2031625257 shutdown_connections 2022-06-17T12:47:54.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.465+0000 7ff1c49c1700 1 -- 172.21.15.36:0/2031625257 wait complete. 2022-06-17T12:47:54.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1c49c1700 1 Processor -- start 2022-06-17T12:47:54.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1c49c1700 1 -- start start 2022-06-17T12:47:54.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1c49c1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1c00adf10 0x7ff1c01435a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:54.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1c49c1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1c00b15b0 0x7ff1c0064480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:54.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1c49c1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1c00b65f0 0x7ff1c0065f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:54.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1c49c1700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c01371b0 con 0x7ff1c00adf10 2022-06-17T12:47:54.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1c49c1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c0136f10 con 0x7ff1c00b15b0 2022-06-17T12:47:54.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1c49c1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c0063fe0 con 0x7ff1c00b65f0 2022-06-17T12:47:54.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1bed9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1c00adf10 0x7ff1c01435a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:54.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1bed9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1c00adf10 0x7ff1c01435a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32874/0 (socket says 172.21.15.36:32874) 2022-06-17T12:47:54.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1bed9d700 1 -- 172.21.15.36:0/3352162609 learned_addr learned my addr 172.21.15.36:0/3352162609 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:54.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1bf59e700 1 --2- 172.21.15.36:0/3352162609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1c00b65f0 0x7ff1c0065f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:54.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1be59c700 1 --2- 172.21.15.36:0/3352162609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1c00b15b0 0x7ff1c0064480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:54.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1bed9d700 1 -- 172.21.15.36:0/3352162609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1c00b65f0 msgr2=0x7ff1c0065f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:54.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1bed9d700 1 --2- 172.21.15.36:0/3352162609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1c00b65f0 0x7ff1c0065f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1bed9d700 1 -- 172.21.15.36:0/3352162609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1c00b15b0 msgr2=0x7ff1c0064480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:54.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1bed9d700 1 --2- 172.21.15.36:0/3352162609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1c00b15b0 0x7ff1c0064480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1bed9d700 1 -- 172.21.15.36:0/3352162609 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b8009530 con 0x7ff1c00adf10 2022-06-17T12:47:54.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1bf59e700 1 --2- 172.21.15.36:0/3352162609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1c00b65f0 0x7ff1c0065f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:47:54.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1bed9d700 1 --2- 172.21.15.36:0/3352162609 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1c00adf10 0x7ff1c01435a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7ff1b000c000 tx=0x7ff1b0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:54.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1affff700 1 -- 172.21.15.36:0/3352162609 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1b0010780 con 0x7ff1c00adf10 2022-06-17T12:47:54.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1affff700 1 -- 172.21.15.36:0/3352162609 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b00108e0 con 0x7ff1c00adf10 2022-06-17T12:47:54.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1affff700 1 -- 172.21.15.36:0/3352162609 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1b0010c40 con 0x7ff1c00adf10 2022-06-17T12:47:54.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1c49c1700 1 -- 172.21.15.36:0/3352162609 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c0064970 con 0x7ff1c00adf10 2022-06-17T12:47:54.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.469+0000 7ff1c49c1700 1 -- 172.21.15.36:0/3352162609 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c00665e0 con 0x7ff1c00adf10 2022-06-17T12:47:54.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.473+0000 7ff1c49c1700 1 -- 172.21.15.36:0/3352162609 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c0136d70 con 0x7ff1c00adf10 2022-06-17T12:47:54.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.473+0000 7ff1affff700 1 -- 172.21.15.36:0/3352162609 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff1b001c460 con 0x7ff1c00adf10 2022-06-17T12:47:54.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.473+0000 7ff1affff700 1 --2- 172.21.15.36:0/3352162609 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff19c0539b0 0x7ff19c055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:54.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.473+0000 7ff1affff700 1 -- 172.21.15.36:0/3352162609 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff1b006c480 con 0x7ff1c00adf10 2022-06-17T12:47:54.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.473+0000 7ff1be59c700 1 --2- 172.21.15.36:0/3352162609 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff19c0539b0 0x7ff19c055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:54.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.473+0000 7ff1be59c700 1 --2- 172.21.15.36:0/3352162609 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff19c0539b0 0x7ff19c055e70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7ff1b400b000 tx=0x7ff1b4009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:54.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.481+0000 7ff1affff700 1 -- 172.21.15.36:0/3352162609 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff1c0136d70 con 0x7ff1c00adf10 2022-06-17T12:47:54.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.605+0000 7ff1c49c1700 1 -- 172.21.15.36:0/3352162609 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff1c0136d70 con 0x7ff1c00adf10 2022-06-17T12:47:54.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.605+0000 7ff1affff700 1 -- 172.21.15.36:0/3352162609 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff1c0136d70 con 0x7ff1c00adf10 2022-06-17T12:47:54.608 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:47:54.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.605+0000 7ff1c49c1700 1 -- 172.21.15.36:0/3352162609 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff19c0539b0 msgr2=0x7ff19c055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:54.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.605+0000 7ff1c49c1700 1 --2- 172.21.15.36:0/3352162609 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff19c0539b0 0x7ff19c055e70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7ff1b400b000 tx=0x7ff1b4009040).stop 2022-06-17T12:47:54.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.605+0000 7ff1c49c1700 1 -- 172.21.15.36:0/3352162609 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1c00adf10 msgr2=0x7ff1c01435a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:54.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.605+0000 7ff1c49c1700 1 --2- 172.21.15.36:0/3352162609 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1c00adf10 0x7ff1c01435a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7ff1b000c000 tx=0x7ff1b0006df0).stop 2022-06-17T12:47:54.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.609+0000 7ff1c49c1700 1 -- 172.21.15.36:0/3352162609 shutdown_connections 2022-06-17T12:47:54.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.609+0000 7ff1c49c1700 1 --2- 172.21.15.36:0/3352162609 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff19c0539b0 0x7ff19c055e70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.609+0000 7ff1c49c1700 1 --2- 172.21.15.36:0/3352162609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1c00b65f0 0x7ff1c0065f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.609+0000 7ff1c49c1700 1 --2- 172.21.15.36:0/3352162609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1c00b15b0 0x7ff1c0064480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.609+0000 7ff1c49c1700 1 --2- 172.21.15.36:0/3352162609 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1c00adf10 0x7ff1c01435a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.609+0000 7ff1c49c1700 1 -- 172.21.15.36:0/3352162609 >> 172.21.15.36:0/3352162609 conn(0x7ff1c00614b0 msgr2=0x7ff1c00b69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:54.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.609+0000 7ff1c49c1700 1 -- 172.21.15.36:0/3352162609 shutdown_connections 2022-06-17T12:47:54.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.609+0000 7ff1c49c1700 1 -- 172.21.15.36:0/3352162609 wait complete. 2022-06-17T12:47:54.653 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:47:54.654 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:47:54.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.777+0000 7fe4f19e0700 1 Processor -- start 2022-06-17T12:47:54.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.777+0000 7fe4f19e0700 1 -- start start 2022-06-17T12:47:54.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4f19e0700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe4ec060ac0 0x7fe4ec144b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:54.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4f19e0700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe4ec0af540 0x7fe4ec13d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:54.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4f19e0700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4ec0b3140 con 0x7fe4ec0afd80 2022-06-17T12:47:54.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4f19e0700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4ec1441e0 con 0x7fe4ec060ac0 2022-06-17T12:47:54.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4f19e0700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4ec060ea0 con 0x7fe4ec0af540 2022-06-17T12:47:54.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4f09de700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe4ec060ac0 0x7fe4ec144b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:54.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4f11df700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe4ec0af540 0x7fe4ec13d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:54.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4f09de700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe4ec060ac0 0x7fe4ec144b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32884/0 (socket says 172.21.15.36:32884) 2022-06-17T12:47:54.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4f11df700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe4ec0af540 0x7fe4ec13d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:53268/0 (socket says 172.21.15.36:53268) 2022-06-17T12:47:54.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4f09de700 1 -- 172.21.15.36:0/2028290236 learned_addr learned my addr 172.21.15.36:0/2028290236 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:54.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4ebfff700 1 --1- 172.21.15.36:0/2028290236 >> v1:172.21.15.70:6789/0 conn(0x7fe4ec0afd80 0x7fe4ec140bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:46146/0 (socket says 172.21.15.36:46146) 2022-06-17T12:47:54.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4f11df700 1 -- 172.21.15.36:0/2028290236 >> v2:172.21.15.36:3300/0 conn(0x7fe4ec060ac0 msgr2=0x7fe4ec144b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:47:54.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4f11df700 1 --2- 172.21.15.36:0/2028290236 >> v2:172.21.15.36:3300/0 conn(0x7fe4ec060ac0 0x7fe4ec144b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4f11df700 1 -- 172.21.15.36:0/2028290236 >> v1:172.21.15.70:6789/0 conn(0x7fe4ec0afd80 legacy=0x7fe4ec140bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:54.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4f11df700 1 -- 172.21.15.36:0/2028290236 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ec1443d0 con 0x7fe4ec0af540 2022-06-17T12:47:54.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4f09de700 1 --2- 172.21.15.36:0/2028290236 >> v2:172.21.15.36:3300/0 conn(0x7fe4ec060ac0 0x7fe4ec144b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:54.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4f11df700 1 --2- 172.21.15.36:0/2028290236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4ec0af540 0x7fe4ec13d710 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fe4e40032f0 tx=0x7fe4e40091d0).ready entity=mon.1 client_cookie=a47c8459c3bac448 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:54.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4eb7fe700 1 -- 172.21.15.36:0/2028290236 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4e400b680 con 0x7fe4ec0af540 2022-06-17T12:47:54.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4eb7fe700 1 -- 172.21.15.36:0/2028290236 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e400b7e0 con 0x7fe4ec0af540 2022-06-17T12:47:54.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4eb7fe700 1 -- 172.21.15.36:0/2028290236 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4e400b990 con 0x7fe4ec0af540 2022-06-17T12:47:54.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4f19e0700 1 -- 172.21.15.36:0/2028290236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4ec0af540 msgr2=0x7fe4ec13d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:54.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4f19e0700 1 --2- 172.21.15.36:0/2028290236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4ec0af540 0x7fe4ec13d710 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fe4e40032f0 tx=0x7fe4e40091d0).stop 2022-06-17T12:47:54.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4f19e0700 1 -- 172.21.15.36:0/2028290236 shutdown_connections 2022-06-17T12:47:54.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4f19e0700 1 --2- 172.21.15.36:0/2028290236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4ec0af540 0x7fe4ec13d710 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4f19e0700 1 --2- 172.21.15.36:0/2028290236 >> v2:172.21.15.36:3300/0 conn(0x7fe4ec060ac0 0x7fe4ec144b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4f19e0700 1 -- 172.21.15.36:0/2028290236 >> 172.21.15.36:0/2028290236 conn(0x7fe4ec0b65f0 msgr2=0x7fe4ec0b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:54.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4f19e0700 1 -- 172.21.15.36:0/2028290236 shutdown_connections 2022-06-17T12:47:54.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.781+0000 7fe4f19e0700 1 -- 172.21.15.36:0/2028290236 wait complete. 2022-06-17T12:47:54.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.785+0000 7fe4f19e0700 1 Processor -- start 2022-06-17T12:47:54.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.785+0000 7fe4f19e0700 1 -- start start 2022-06-17T12:47:54.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.785+0000 7fe4f19e0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4ec060ac0 0x7fe4ec1509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:54.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.785+0000 7fe4f19e0700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4ec0af540 0x7fe4ec067660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:54.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.785+0000 7fe4f19e0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4ec0afd80 0x7fe4ec065a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:54.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.785+0000 7fe4f19e0700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec065320 con 0x7fe4ec060ac0 2022-06-17T12:47:54.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.785+0000 7fe4f19e0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec147e60 con 0x7fe4ec0afd80 2022-06-17T12:47:54.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.785+0000 7fe4f19e0700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec063f40 con 0x7fe4ec0af540 2022-06-17T12:47:54.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.785+0000 7fe4f09de700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4ec060ac0 0x7fe4ec1509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:54.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.785+0000 7fe4f09de700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4ec060ac0 0x7fe4ec1509c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32888/0 (socket says 172.21.15.36:32888) 2022-06-17T12:47:54.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.785+0000 7fe4f09de700 1 -- 172.21.15.36:0/2122965250 learned_addr learned my addr 172.21.15.36:0/2122965250 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:47:54.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.785+0000 7fe4ebfff700 1 --2- 172.21.15.36:0/2122965250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4ec0af540 0x7fe4ec067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:54.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.785+0000 7fe4f11df700 1 --2- 172.21.15.36:0/2122965250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4ec0afd80 0x7fe4ec065a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:54.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.789+0000 7fe4f11df700 1 -- 172.21.15.36:0/2122965250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4ec0af540 msgr2=0x7fe4ec067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:54.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.789+0000 7fe4f11df700 1 --2- 172.21.15.36:0/2122965250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4ec0af540 0x7fe4ec067660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.789+0000 7fe4f11df700 1 -- 172.21.15.36:0/2122965250 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4ec060ac0 msgr2=0x7fe4ec1509c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:54.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.789+0000 7fe4f11df700 1 --2- 172.21.15.36:0/2122965250 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4ec060ac0 0x7fe4ec1509c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.789+0000 7fe4f11df700 1 -- 172.21.15.36:0/2122965250 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e000b1e0 con 0x7fe4ec0afd80 2022-06-17T12:47:54.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.789+0000 7fe4ebfff700 1 --2- 172.21.15.36:0/2122965250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4ec0af540 0x7fe4ec067660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:54.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.789+0000 7fe4f09de700 1 --2- 172.21.15.36:0/2122965250 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4ec060ac0 0x7fe4ec1509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:47:54.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.789+0000 7fe4f11df700 1 --2- 172.21.15.36:0/2122965250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4ec0afd80 0x7fe4ec065a70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fe4e40032f0 tx=0x7fe4e4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:54.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.789+0000 7fe4e9ffb700 1 -- 172.21.15.36:0/2122965250 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4e4011d30 con 0x7fe4ec0afd80 2022-06-17T12:47:54.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.789+0000 7fe4e9ffb700 1 -- 172.21.15.36:0/2122965250 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e4011e90 con 0x7fe4ec0afd80 2022-06-17T12:47:54.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.789+0000 7fe4f19e0700 1 -- 172.21.15.36:0/2122965250 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e4009470 con 0x7fe4ec0afd80 2022-06-17T12:47:54.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.789+0000 7fe4e9ffb700 1 -- 172.21.15.36:0/2122965250 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4e4021630 con 0x7fe4ec0afd80 2022-06-17T12:47:54.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.789+0000 7fe4f19e0700 1 -- 172.21.15.36:0/2122965250 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4ec064510 con 0x7fe4ec0afd80 2022-06-17T12:47:54.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.789+0000 7fe4f19e0700 1 -- 172.21.15.36:0/2122965250 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4ec067cf0 con 0x7fe4ec0afd80 2022-06-17T12:47:54.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.789+0000 7fe4e9ffb700 1 -- 172.21.15.36:0/2122965250 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe4e4027020 con 0x7fe4ec0afd80 2022-06-17T12:47:54.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.793+0000 7fe4e9ffb700 1 --2- 172.21.15.36:0/2122965250 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe4cc053ad0 0x7fe4cc055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:47:54.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.793+0000 7fe4e9ffb700 1 -- 172.21.15.36:0/2122965250 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe4e406d310 con 0x7fe4ec0afd80 2022-06-17T12:47:54.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.793+0000 7fe4f09de700 1 --2- 172.21.15.36:0/2122965250 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe4cc053ad0 0x7fe4cc055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:47:54.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.793+0000 7fe4f09de700 1 --2- 172.21.15.36:0/2122965250 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe4cc053ad0 0x7fe4cc055f90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fe4e000b000 tx=0x7fe4e0006eb0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:47:54.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.801+0000 7fe4e9ffb700 1 -- 172.21.15.36:0/2122965250 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe4e40440b0 con 0x7fe4ec0afd80 2022-06-17T12:47:54.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.965+0000 7fe4f19e0700 1 -- 172.21.15.36:0/2122965250 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4ec063da0 con 0x7fe4ec0afd80 2022-06-17T12:47:54.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.965+0000 7fe4e9ffb700 1 -- 172.21.15.36:0/2122965250 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+2138 (secure 0 0 0) 0x7fe4e4015870 con 0x7fe4ec0afd80 2022-06-17T12:47:54.956 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:47:54.956 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3597082917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3597082917},{"type":"v1","addr":"172.21.15.70:6833","nonce":3597082917}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:47:47.072139+0000","modified":"2022-06-17T12:47:49.085163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4947":{"gid":4947,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.36:6835/1648255567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":1648255567},{"type":"v1","addr":"172.21.15.36:6835","nonce":1648255567}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-06-17T12:47:54.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.969+0000 7fe4f19e0700 1 -- 172.21.15.36:0/2122965250 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe4cc053ad0 msgr2=0x7fe4cc055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:54.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.969+0000 7fe4f19e0700 1 --2- 172.21.15.36:0/2122965250 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe4cc053ad0 0x7fe4cc055f90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fe4e000b000 tx=0x7fe4e0006eb0).stop 2022-06-17T12:47:54.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.969+0000 7fe4f19e0700 1 -- 172.21.15.36:0/2122965250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4ec0afd80 msgr2=0x7fe4ec065a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:47:54.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.969+0000 7fe4f19e0700 1 --2- 172.21.15.36:0/2122965250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4ec0afd80 0x7fe4ec065a70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fe4e40032f0 tx=0x7fe4e4016000).stop 2022-06-17T12:47:54.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.969+0000 7fe4f19e0700 1 -- 172.21.15.36:0/2122965250 shutdown_connections 2022-06-17T12:47:54.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.969+0000 7fe4f19e0700 1 --2- 172.21.15.36:0/2122965250 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe4cc053ad0 0x7fe4cc055f90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.969+0000 7fe4f19e0700 1 --2- 172.21.15.36:0/2122965250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4ec0afd80 0x7fe4ec065a70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.969+0000 7fe4f19e0700 1 --2- 172.21.15.36:0/2122965250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4ec0af540 0x7fe4ec067660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.969+0000 7fe4f19e0700 1 --2- 172.21.15.36:0/2122965250 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4ec060ac0 0x7fe4ec1509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:47:54.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.969+0000 7fe4f19e0700 1 -- 172.21.15.36:0/2122965250 >> 172.21.15.36:0/2122965250 conn(0x7fe4ec0b65f0 msgr2=0x7fe4ec0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:47:54.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.969+0000 7fe4f19e0700 1 -- 172.21.15.36:0/2122965250 shutdown_connections 2022-06-17T12:47:54.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:47:54.969+0000 7fe4f19e0700 1 -- 172.21.15.36:0/2122965250 wait complete. 2022-06-17T12:47:54.979 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 14 2022-06-17T12:47:55.212 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:47:47.072139+0000', 'modified': '2022-06-17T12:47:49.085163+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4947}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4947': {'gid': 4947, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.36:6835/1648255567', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 1648255567}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 1648255567}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:47:55.213 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:47:55.214 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:47:55.358 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":4947,"want_state":"up:active","state":"up:active","rank_uptime":8.2865203869999995,"mdsmap_epoch":14,"osdmap_epoch":35,"osdmap_epoch_barrier":35,"uptime":16.486977216} 2022-06-17T12:47:55.374 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 4947, "mdsmap_epoch": 14, "osdmap_epoch": 35, "osdmap_epoch_barrier": 35, "rank_uptime": 8.286520387, "state": "up:active", "uptime": 16.486977216, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:47:55.391 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:47:55.392 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-17T12:47:55.393 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi107.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-17T12:47:55.407 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:55.408 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:47:55.423 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:47:55.492 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:47:55.493 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:47:55.603 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:47:55.605 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:47:55.606 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:55.607 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:47:55.649 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:47:55.650 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:47:55.719 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:47:55.748 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:47:55.749 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:47:55.788 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:2022-06-17T12:47:55.799+0000 7f690baac080 -1 init, newargv = 0x55563893c3e0 newargc=9 2022-06-17T12:47:55.788 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[21817]: starting ceph client 2022-06-17T12:47:55.803 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[21817]: starting fuse 2022-06-17T12:47:55.858 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:47:55.866 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:47:55.867 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:55.868 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:47:55.944 INFO:teuthology.orchestra.run.smithi107.stdout:52 2022-06-17T12:47:55.945 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:47:55.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:55.946 DEBUG:teuthology.orchestra.run.smithi107:> sudo python3 -c ' 2022-06-17T12:47:55.954 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2022-06-17T12:47:55.954 DEBUG:teuthology.orchestra.run.smithi107:> import re 2022-06-17T12:47:55.955 DEBUG:teuthology.orchestra.run.smithi107:> import os 2022-06-17T12:47:55.955 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2022-06-17T12:47:55.956 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:47:55.956 DEBUG:teuthology.orchestra.run.smithi107:> def find_socket(client_name): 2022-06-17T12:47:55.970 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-17T12:47:55.971 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2022-06-17T12:47:55.972 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:47:55.973 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2022-06-17T12:47:55.973 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2022-06-17T12:47:55.973 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2022-06-17T12:47:55.991 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2022-06-17T12:47:55.992 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:47:55.993 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2022-06-17T12:47:55.994 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:47:55.994 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:47:55.995 DEBUG:teuthology.orchestra.run.smithi107:> return f 2022-06-17T12:47:55.996 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:47:55.997 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:47:55.997 DEBUG:teuthology.orchestra.run.smithi107:> print(find_socket("client.0")) 2022-06-17T12:47:55.999 DEBUG:teuthology.orchestra.run.smithi107:> ' 2022-06-17T12:47:56.056 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.0.21817.asok 2022-06-17T12:47:56.056 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.21817.asok 2022-06-17T12:47:56.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:56.058 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.21817.asok status 2022-06-17T12:47:56.167 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2022-06-17T12:47:56.168 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2022-06-17T12:47:56.169 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:47:56.169 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:47:56.169 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "0", 2022-06-17T12:47:56.170 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2022-06-17T12:47:56.170 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-17T12:47:56.171 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "21817", 2022-06-17T12:47:56.171 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/" 2022-06-17T12:47:56.171 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:47:56.172 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2022-06-17T12:47:56.187 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2022-06-17T12:47:56.188 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 4811, 2022-06-17T12:47:56.188 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2022-06-17T12:47:56.189 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2022-06-17T12:47:56.189 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2022-06-17T12:47:56.191 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 4811 2022-06-17T12:47:56.192 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:47:56.192 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:47:56.193 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:47:56.193 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:47:56.193 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1335437980 2022-06-17T12:47:56.194 INFO:teuthology.orchestra.run.smithi107.stdout: } 2022-06-17T12:47:56.194 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:47:56.195 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:47:56.195 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:47:56.196 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:47:56.196 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1335437980 2022-06-17T12:47:56.196 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:47:56.197 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.4811 172.21.15.107:0/1335437980", 2022-06-17T12:47:56.197 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "172.21.15.107:0/1335437980", 2022-06-17T12:47:56.206 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 1, 2022-06-17T12:47:56.206 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 14, 2022-06-17T12:47:56.206 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 35, 2022-06-17T12:47:56.207 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:47:56.207 INFO:teuthology.orchestra.run.smithi107.stdout: "blacklisted": false, 2022-06-17T12:47:56.207 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2022-06-17T12:47:56.208 INFO:teuthology.orchestra.run.smithi107.stdout:} 2022-06-17T12:47:56.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:56.217 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:47:56.223 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:47:56.240 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:47:56.250 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:47:56.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:56.251 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:47:56.320 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_min_caps_per_client 2022-06-17T12:47:56.524 INFO:teuthology.orchestra.run.smithi036.stdout:{"mds_min_caps_per_client":"100"} 2022-06-17T12:47:56.525 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_min_caps_per_client": "100" } 2022-06-17T12:47:56.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:47:56.526 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:47:56.528 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:47:56.595 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:47:56.595 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:47:56.597 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-06-17T12:47:56.600 DEBUG:teuthology.orchestra.run.smithi107:> import sys 2022-06-17T12:47:56.601 DEBUG:teuthology.orchestra.run.smithi107:> import time 2022-06-17T12:47:56.601 DEBUG:teuthology.orchestra.run.smithi107:> import os 2022-06-17T12:47:56.602 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:47:56.602 DEBUG:teuthology.orchestra.run.smithi107:> n = 1000 2022-06-17T12:47:56.602 DEBUG:teuthology.orchestra.run.smithi107:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2022-06-17T12:47:56.603 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:47:56.603 DEBUG:teuthology.orchestra.run.smithi107:> if not os.path.exists(os.path.dirname(abs_path)): 2022-06-17T12:47:56.603 DEBUG:teuthology.orchestra.run.smithi107:> os.makedirs(os.path.dirname(abs_path)) 2022-06-17T12:47:56.604 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:47:56.604 DEBUG:teuthology.orchestra.run.smithi107:> for i in range(0, n): 2022-06-17T12:47:56.615 DEBUG:teuthology.orchestra.run.smithi107:> fname = "{0}_{1}".format(abs_path, i) 2022-06-17T12:47:56.615 DEBUG:teuthology.orchestra.run.smithi107:> with open(fname, '"'"'w'"'"') as f: 2022-06-17T12:47:56.616 DEBUG:teuthology.orchestra.run.smithi107:> f.write('"'"'content'"'"') 2022-06-17T12:47:56.616 DEBUG:teuthology.orchestra.run.smithi107:> if True: 2022-06-17T12:47:56.617 DEBUG:teuthology.orchestra.run.smithi107:> f.flush() 2022-06-17T12:47:56.617 DEBUG:teuthology.orchestra.run.smithi107:> os.fsync(f.fileno()) 2022-06-17T12:47:56.617 DEBUG:teuthology.orchestra.run.smithi107:> ' 2022-06-17T12:48:07.719 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:48:07.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d428a9700 1 Processor -- start 2022-06-17T12:48:07.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d428a9700 1 -- start start 2022-06-17T12:48:07.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d428a9700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7d3c0b7c10 0x7f7d3c145070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:07.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d428a9700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7d3c066fd0 0x7f7d3c067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:07.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d428a9700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d3c1417a0 con 0x7f7d3c0ad330 2022-06-17T12:48:07.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d428a9700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d3c1422e0 con 0x7f7d3c0b7c10 2022-06-17T12:48:07.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d428a9700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d3c0b7ff0 con 0x7f7d3c066fd0 2022-06-17T12:48:07.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d3bfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7d3c0b7c10 0x7f7d3c145070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:07.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d3b7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f7d3c0ad330 0x7f7d3c141090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50430/0 (socket says 172.21.15.36:50430) 2022-06-17T12:48:07.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d3bfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7d3c0b7c10 0x7f7d3c145070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32898/0 (socket says 172.21.15.36:32898) 2022-06-17T12:48:07.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d3b7fe700 1 -- 172.21.15.36:0/3809933728 learned_addr learned my addr 172.21.15.36:0/3809933728 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:48:07.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d40e46700 1 --2- 172.21.15.36:0/3809933728 >> v2:172.21.15.70:3300/0 conn(0x7f7d3c066fd0 0x7f7d3c067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:07.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d3bfff700 1 -- 172.21.15.36:0/3809933728 >> v2:172.21.15.70:3300/0 conn(0x7f7d3c066fd0 msgr2=0x7f7d3c067400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:48:07.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d3bfff700 1 --2- 172.21.15.36:0/3809933728 >> v2:172.21.15.70:3300/0 conn(0x7f7d3c066fd0 0x7f7d3c067400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:07.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d3bfff700 1 -- 172.21.15.36:0/3809933728 >> v1:172.21.15.36:6789/0 conn(0x7f7d3c0ad330 legacy=0x7f7d3c141090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:07.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d3affd700 1 -- 172.21.15.36:0/3809933728 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 511439806 0 0) 0x7f7d3c1417a0 con 0x7f7d3c0ad330 2022-06-17T12:48:07.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d3bfff700 1 -- 172.21.15.36:0/3809933728 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3c0b30a0 con 0x7f7d3c0b7c10 2022-06-17T12:48:07.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d3bfff700 1 --2- 172.21.15.36:0/3809933728 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7d3c0b7c10 0x7f7d3c145070 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f7d240032f0 tx=0x7f7d240091d0).ready entity=mon.0 client_cookie=6ae3b52a6b88d3ac server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:07.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d3affd700 1 -- 172.21.15.36:0/3809933728 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d24009ea0 con 0x7f7d3c0b7c10 2022-06-17T12:48:07.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d3affd700 1 -- 172.21.15.36:0/3809933728 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2400b870 con 0x7f7d3c0b7c10 2022-06-17T12:48:07.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d3affd700 1 -- 172.21.15.36:0/3809933728 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d2400b9e0 con 0x7f7d3c0b7c10 2022-06-17T12:48:07.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d428a9700 1 -- 172.21.15.36:0/3809933728 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7d3c0b7c10 msgr2=0x7f7d3c145070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:07.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d428a9700 1 --2- 172.21.15.36:0/3809933728 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7d3c0b7c10 0x7f7d3c145070 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f7d240032f0 tx=0x7f7d240091d0).stop 2022-06-17T12:48:07.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d428a9700 1 -- 172.21.15.36:0/3809933728 shutdown_connections 2022-06-17T12:48:07.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d428a9700 1 --2- 172.21.15.36:0/3809933728 >> v2:172.21.15.70:3300/0 conn(0x7f7d3c066fd0 0x7f7d3c067400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:07.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d428a9700 1 --2- 172.21.15.36:0/3809933728 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7d3c0b7c10 0x7f7d3c145070 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:07.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d428a9700 1 -- 172.21.15.36:0/3809933728 >> 172.21.15.36:0/3809933728 conn(0x7f7d3c060c60 msgr2=0x7f7d3c05c550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:48:07.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d428a9700 1 -- 172.21.15.36:0/3809933728 shutdown_connections 2022-06-17T12:48:07.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.885+0000 7f7d428a9700 1 -- 172.21.15.36:0/3809933728 wait complete. 2022-06-17T12:48:08.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.889+0000 7f7d428a9700 1 Processor -- start 2022-06-17T12:48:08.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.889+0000 7f7d428a9700 1 -- start start 2022-06-17T12:48:08.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.889+0000 7f7d428a9700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d3c066fd0 0x7f7d3c143d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:08.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.889+0000 7f7d428a9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d3c0ad330 0x7f7d3c0afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:08.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.889+0000 7f7d428a9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7d3c0b7c10 0x7f7d3c0b1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:08.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.889+0000 7f7d428a9700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c0aebc0 con 0x7f7d3c0b7c10 2022-06-17T12:48:08.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.889+0000 7f7d428a9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c13fe60 con 0x7f7d3c0ad330 2022-06-17T12:48:08.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.889+0000 7f7d428a9700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c140230 con 0x7f7d3c066fd0 2022-06-17T12:48:08.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.889+0000 7f7d40e46700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7d3c0b7c10 0x7f7d3c0b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:08.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.889+0000 7f7d40e46700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7d3c0b7c10 0x7f7d3c0b1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32908/0 (socket says 172.21.15.36:32908) 2022-06-17T12:48:08.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.889+0000 7f7d40e46700 1 -- 172.21.15.36:0/2238368406 learned_addr learned my addr 172.21.15.36:0/2238368406 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:48:08.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.889+0000 7f7d3b7fe700 1 --2- 172.21.15.36:0/2238368406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d3c0ad330 0x7f7d3c0afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:08.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.889+0000 7f7d3bfff700 1 --2- 172.21.15.36:0/2238368406 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d3c066fd0 0x7f7d3c143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:08.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.889+0000 7f7d40e46700 1 -- 172.21.15.36:0/2238368406 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d3c066fd0 msgr2=0x7f7d3c143d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:08.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.889+0000 7f7d40e46700 1 --2- 172.21.15.36:0/2238368406 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d3c066fd0 0x7f7d3c143d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:08.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.889+0000 7f7d40e46700 1 -- 172.21.15.36:0/2238368406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d3c0ad330 msgr2=0x7f7d3c0afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:08.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.889+0000 7f7d40e46700 1 --2- 172.21.15.36:0/2238368406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d3c0ad330 0x7f7d3c0afed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:08.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.889+0000 7f7d40e46700 1 -- 172.21.15.36:0/2238368406 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d24009470 con 0x7f7d3c0b7c10 2022-06-17T12:48:08.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.889+0000 7f7d40e46700 1 --2- 172.21.15.36:0/2238368406 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7d3c0b7c10 0x7f7d3c0b1500 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f7d3400a000 tx=0x7f7d34008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:08.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.889+0000 7f7d397fa700 1 -- 172.21.15.36:0/2238368406 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d34011780 con 0x7f7d3c0b7c10 2022-06-17T12:48:08.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.889+0000 7f7d397fa700 1 -- 172.21.15.36:0/2238368406 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d340118e0 con 0x7f7d3c0b7c10 2022-06-17T12:48:08.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.889+0000 7f7d428a9700 1 -- 172.21.15.36:0/2238368406 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d3c0b1060 con 0x7f7d3c0b7c10 2022-06-17T12:48:08.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.889+0000 7f7d397fa700 1 -- 172.21.15.36:0/2238368406 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d34011c40 con 0x7f7d3c0b7c10 2022-06-17T12:48:08.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.889+0000 7f7d428a9700 1 -- 172.21.15.36:0/2238368406 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3c0b05e0 con 0x7f7d3c0b7c10 2022-06-17T12:48:08.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.893+0000 7f7d428a9700 1 -- 172.21.15.36:0/2238368406 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d3c0b3340 con 0x7f7d3c0b7c10 2022-06-17T12:48:08.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.893+0000 7f7d397fa700 1 -- 172.21.15.36:0/2238368406 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f7d34012420 con 0x7f7d3c0b7c10 2022-06-17T12:48:08.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.893+0000 7f7d397fa700 1 --2- 172.21.15.36:0/2238368406 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7d20053af0 0x7f7d20055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:08.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.893+0000 7f7d397fa700 1 -- 172.21.15.36:0/2238368406 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7d3406bc30 con 0x7f7d3c0b7c10 2022-06-17T12:48:08.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.893+0000 7f7d3bfff700 1 --2- 172.21.15.36:0/2238368406 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7d20053af0 0x7f7d20055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:08.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.893+0000 7f7d3bfff700 1 --2- 172.21.15.36:0/2238368406 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7d20053af0 0x7f7d20055fb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f7d2401b000 tx=0x7f7d24016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:08.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:07.901+0000 7f7d397fa700 1 -- 172.21.15.36:0/2238368406 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7d340429d0 con 0x7f7d3c0b7c10 2022-06-17T12:48:08.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.065+0000 7f7d428a9700 1 -- 172.21.15.36:0/2238368406 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d3c057410 con 0x7f7d3c0b7c10 2022-06-17T12:48:08.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.065+0000 7f7d397fa700 1 -- 172.21.15.36:0/2238368406 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+2138 (secure 0 0 0) 0x7f7d34045ff0 con 0x7f7d3c0b7c10 2022-06-17T12:48:08.062 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:48:08.066 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3597082917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3597082917},{"type":"v1","addr":"172.21.15.70:6833","nonce":3597082917}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:47:47.072139+0000","modified":"2022-06-17T12:47:49.085163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4947":{"gid":4947,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.36:6835/1648255567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":1648255567},{"type":"v1","addr":"172.21.15.36:6835","nonce":1648255567}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-06-17T12:48:08.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.065+0000 7f7d428a9700 1 -- 172.21.15.36:0/2238368406 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7d20053af0 msgr2=0x7f7d20055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:08.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.065+0000 7f7d428a9700 1 --2- 172.21.15.36:0/2238368406 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7d20053af0 0x7f7d20055fb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f7d2401b000 tx=0x7f7d24016000).stop 2022-06-17T12:48:08.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.065+0000 7f7d428a9700 1 -- 172.21.15.36:0/2238368406 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7d3c0b7c10 msgr2=0x7f7d3c0b1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:08.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.065+0000 7f7d428a9700 1 --2- 172.21.15.36:0/2238368406 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7d3c0b7c10 0x7f7d3c0b1500 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f7d3400a000 tx=0x7f7d34008040).stop 2022-06-17T12:48:08.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.065+0000 7f7d428a9700 1 -- 172.21.15.36:0/2238368406 shutdown_connections 2022-06-17T12:48:08.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.065+0000 7f7d428a9700 1 --2- 172.21.15.36:0/2238368406 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7d20053af0 0x7f7d20055fb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:08.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.065+0000 7f7d428a9700 1 --2- 172.21.15.36:0/2238368406 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7d3c0b7c10 0x7f7d3c0b1500 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:08.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.065+0000 7f7d428a9700 1 --2- 172.21.15.36:0/2238368406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d3c0ad330 0x7f7d3c0afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:08.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.065+0000 7f7d428a9700 1 --2- 172.21.15.36:0/2238368406 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d3c066fd0 0x7f7d3c143d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:08.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.065+0000 7f7d428a9700 1 -- 172.21.15.36:0/2238368406 >> 172.21.15.36:0/2238368406 conn(0x7f7d3c060c60 msgr2=0x7f7d3c064c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:48:08.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.065+0000 7f7d428a9700 1 -- 172.21.15.36:0/2238368406 shutdown_connections 2022-06-17T12:48:08.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.065+0000 7f7d428a9700 1 -- 172.21.15.36:0/2238368406 wait complete. 2022-06-17T12:48:08.089 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 14 2022-06-17T12:48:08.090 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_caps 20 2022-06-17T12:48:08.208 INFO:teuthology.orchestra.run.smithi036.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2022-06-17T12:48:08.209 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2022-06-17T12:48:08.210 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:48:08.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85ffef700 1 Processor -- start 2022-06-17T12:48:08.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85ffef700 1 -- start start 2022-06-17T12:48:08.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85ffef700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe8581469c0 0x7fe858146da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:08.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85ffef700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe8580b6600 0x7fe8580b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:08.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85ffef700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8580b54d0 0x7fe858062080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:08.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85ffef700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8580b48e0 con 0x7fe8581469c0 2022-06-17T12:48:08.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85ffef700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe858147290 con 0x7fe8580b6600 2022-06-17T12:48:08.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85ffef700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8580b4b40 con 0x7fe8580b54d0 2022-06-17T12:48:08.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85dd8b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe8581469c0 0x7fe858146da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:08.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85e58c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8580b54d0 0x7fe858062080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:08.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85d58a700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe8580b6600 0x7fe8580b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:08.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85d58a700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe8580b6600 0x7fe8580b4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:53294/0 (socket says 172.21.15.36:53294) 2022-06-17T12:48:08.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85d58a700 1 -- 172.21.15.36:0/2268274514 learned_addr learned my addr 172.21.15.36:0/2268274514 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:48:08.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85e58c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8580b54d0 0x7fe858062080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:47706/0 (socket says 172.21.15.36:47706) 2022-06-17T12:48:08.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85dd8b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe8581469c0 0x7fe858146da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32914/0 (socket says 172.21.15.36:32914) 2022-06-17T12:48:08.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85d58a700 1 -- 172.21.15.36:0/2268274514 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8580b54d0 msgr2=0x7fe858062080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:48:08.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85d58a700 1 --2- 172.21.15.36:0/2268274514 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8580b54d0 0x7fe858062080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:08.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85d58a700 1 -- 172.21.15.36:0/2268274514 >> v2:172.21.15.36:3300/0 conn(0x7fe8581469c0 msgr2=0x7fe858146da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:48:08.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85d58a700 1 --2- 172.21.15.36:0/2268274514 >> v2:172.21.15.36:3300/0 conn(0x7fe8581469c0 0x7fe858146da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:08.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85d58a700 1 -- 172.21.15.36:0/2268274514 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8580b6aa0 con 0x7fe8580b6600 2022-06-17T12:48:08.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85e58c700 1 --2- 172.21.15.36:0/2268274514 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8580b54d0 0x7fe858062080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:48:08.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85d58a700 1 --2- 172.21.15.36:0/2268274514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8580b6600 0x7fe8580b4fe0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fe8580b1060 tx=0x7fe848006eb0).ready entity=mon.1 client_cookie=32b39bb7109a5f34 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:08.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85cd89700 1 -- 172.21.15.36:0/2268274514 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe84800e610 con 0x7fe8580b6600 2022-06-17T12:48:08.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85cd89700 1 -- 172.21.15.36:0/2268274514 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe848000e90 con 0x7fe8580b6600 2022-06-17T12:48:08.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.329+0000 7fe85cd89700 1 -- 172.21.15.36:0/2268274514 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe84800e9a0 con 0x7fe8580b6600 2022-06-17T12:48:08.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.333+0000 7fe85ffef700 1 -- 172.21.15.36:0/2268274514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8580b6600 msgr2=0x7fe8580b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:08.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.333+0000 7fe85ffef700 1 --2- 172.21.15.36:0/2268274514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8580b6600 0x7fe8580b4fe0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fe8580b1060 tx=0x7fe848006eb0).stop 2022-06-17T12:48:08.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.333+0000 7fe85ffef700 1 -- 172.21.15.36:0/2268274514 shutdown_connections 2022-06-17T12:48:08.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.333+0000 7fe85ffef700 1 --2- 172.21.15.36:0/2268274514 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8580b54d0 0x7fe858062080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:08.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.333+0000 7fe85ffef700 1 --2- 172.21.15.36:0/2268274514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8580b6600 0x7fe8580b4fe0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:08.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.333+0000 7fe85ffef700 1 --2- 172.21.15.36:0/2268274514 >> v2:172.21.15.36:3300/0 conn(0x7fe8581469c0 0x7fe858146da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:08.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.333+0000 7fe85ffef700 1 -- 172.21.15.36:0/2268274514 >> 172.21.15.36:0/2268274514 conn(0x7fe85805f770 msgr2=0x7fe85805fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:48:08.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.333+0000 7fe85ffef700 1 -- 172.21.15.36:0/2268274514 shutdown_connections 2022-06-17T12:48:08.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.333+0000 7fe85ffef700 1 -- 172.21.15.36:0/2268274514 wait complete. 2022-06-17T12:48:08.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.333+0000 7fe85ffef700 1 Processor -- start 2022-06-17T12:48:08.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.333+0000 7fe85ffef700 1 -- start start 2022-06-17T12:48:08.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.337+0000 7fe85ffef700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8580b54d0 0x7fe8580b8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:08.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.337+0000 7fe85ffef700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8580b6600 0x7fe858148480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:08.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.337+0000 7fe85ffef700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8581469c0 0x7fe858148970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:08.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.337+0000 7fe85ffef700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe858143f40 con 0x7fe8580b6600 2022-06-17T12:48:08.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.337+0000 7fe85ffef700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85813efe0 con 0x7fe8580b54d0 2022-06-17T12:48:08.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.337+0000 7fe85ffef700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe858149410 con 0x7fe8581469c0 2022-06-17T12:48:08.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.337+0000 7fe85d58a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8580b6600 0x7fe858148480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:08.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.337+0000 7fe85d58a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8580b6600 0x7fe858148480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32920/0 (socket says 172.21.15.36:32920) 2022-06-17T12:48:08.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.337+0000 7fe85d58a700 1 -- 172.21.15.36:0/1408257816 learned_addr learned my addr 172.21.15.36:0/1408257816 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:48:08.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.337+0000 7fe85dd8b700 1 --2- 172.21.15.36:0/1408257816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8580b54d0 0x7fe8580b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:08.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.337+0000 7fe85e58c700 1 --2- 172.21.15.36:0/1408257816 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8581469c0 0x7fe858148970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:08.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.337+0000 7fe85d58a700 1 -- 172.21.15.36:0/1408257816 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8581469c0 msgr2=0x7fe858148970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:08.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.337+0000 7fe85d58a700 1 --2- 172.21.15.36:0/1408257816 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8581469c0 0x7fe858148970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:08.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.337+0000 7fe85d58a700 1 -- 172.21.15.36:0/1408257816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8580b54d0 msgr2=0x7fe8580b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:08.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.337+0000 7fe85d58a700 1 --2- 172.21.15.36:0/1408257816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8580b54d0 0x7fe8580b8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:08.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.337+0000 7fe85d58a700 1 -- 172.21.15.36:0/1408257816 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe850009470 con 0x7fe8580b6600 2022-06-17T12:48:08.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.337+0000 7fe85dd8b700 1 --2- 172.21.15.36:0/1408257816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8580b54d0 0x7fe8580b8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:48:08.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.337+0000 7fe85d58a700 1 --2- 172.21.15.36:0/1408257816 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8580b6600 0x7fe858148480 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fe8580b1060 tx=0x7fe848015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:08.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.337+0000 7fe846ffd700 1 -- 172.21.15.36:0/1408257816 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8480117e0 con 0x7fe8580b6600 2022-06-17T12:48:08.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.337+0000 7fe846ffd700 1 -- 172.21.15.36:0/1408257816 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe848011940 con 0x7fe8580b6600 2022-06-17T12:48:08.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.337+0000 7fe85ffef700 1 -- 172.21.15.36:0/1408257816 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84800b1e0 con 0x7fe8580b6600 2022-06-17T12:48:08.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.337+0000 7fe846ffd700 1 -- 172.21.15.36:0/1408257816 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe848011ca0 con 0x7fe8580b6600 2022-06-17T12:48:08.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.337+0000 7fe85ffef700 1 -- 172.21.15.36:0/1408257816 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8580b8a90 con 0x7fe8580b6600 2022-06-17T12:48:08.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.337+0000 7fe85ffef700 1 -- 172.21.15.36:0/1408257816 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe85813f1e0 con 0x7fe8580b6600 2022-06-17T12:48:08.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.341+0000 7fe846ffd700 1 -- 172.21.15.36:0/1408257816 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe848025030 con 0x7fe8580b6600 2022-06-17T12:48:08.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.341+0000 7fe846ffd700 1 --2- 172.21.15.36:0/1408257816 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe838053af0 0x7fe838055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:08.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.341+0000 7fe846ffd700 1 -- 172.21.15.36:0/1408257816 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe84806d2b0 con 0x7fe8580b6600 2022-06-17T12:48:08.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.341+0000 7fe85dd8b700 1 --2- 172.21.15.36:0/1408257816 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe838053af0 0x7fe838055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:08.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.341+0000 7fe85dd8b700 1 --2- 172.21.15.36:0/1408257816 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe838053af0 0x7fe838055fb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fe85400b000 tx=0x7fe854008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:08.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.349+0000 7fe846ffd700 1 -- 172.21.15.36:0/1408257816 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe848043fa0 con 0x7fe8580b6600 2022-06-17T12:48:08.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.509+0000 7fe85ffef700 1 -- 172.21.15.36:0/1408257816 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8580b5960 con 0x7fe8580b6600 2022-06-17T12:48:08.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.509+0000 7fe846ffd700 1 -- 172.21.15.36:0/1408257816 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+2138 (secure 0 0 0) 0x7fe848023090 con 0x7fe8580b6600 2022-06-17T12:48:08.513 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:48:08.514 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3597082917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3597082917},{"type":"v1","addr":"172.21.15.70:6833","nonce":3597082917}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:47:47.072139+0000","modified":"2022-06-17T12:47:49.085163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4947":{"gid":4947,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.36:6835/1648255567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":1648255567},{"type":"v1","addr":"172.21.15.36:6835","nonce":1648255567}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-06-17T12:48:08.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.513+0000 7fe85ffef700 1 -- 172.21.15.36:0/1408257816 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe838053af0 msgr2=0x7fe838055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:08.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.513+0000 7fe85ffef700 1 --2- 172.21.15.36:0/1408257816 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe838053af0 0x7fe838055fb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fe85400b000 tx=0x7fe854008040).stop 2022-06-17T12:48:08.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.513+0000 7fe85ffef700 1 -- 172.21.15.36:0/1408257816 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8580b6600 msgr2=0x7fe858148480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:08.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.513+0000 7fe85ffef700 1 --2- 172.21.15.36:0/1408257816 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8580b6600 0x7fe858148480 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fe8580b1060 tx=0x7fe848015000).stop 2022-06-17T12:48:08.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.513+0000 7fe85ffef700 1 -- 172.21.15.36:0/1408257816 shutdown_connections 2022-06-17T12:48:08.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.513+0000 7fe85ffef700 1 --2- 172.21.15.36:0/1408257816 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe838053af0 0x7fe838055fb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:08.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.513+0000 7fe85ffef700 1 --2- 172.21.15.36:0/1408257816 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8581469c0 0x7fe858148970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:08.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.513+0000 7fe85ffef700 1 --2- 172.21.15.36:0/1408257816 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8580b6600 0x7fe858148480 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:08.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.513+0000 7fe85ffef700 1 --2- 172.21.15.36:0/1408257816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8580b54d0 0x7fe8580b8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:08.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.513+0000 7fe85ffef700 1 -- 172.21.15.36:0/1408257816 >> 172.21.15.36:0/1408257816 conn(0x7fe85805f770 msgr2=0x7fe8581457b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:48:08.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.513+0000 7fe85ffef700 1 -- 172.21.15.36:0/1408257816 shutdown_connections 2022-06-17T12:48:08.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.513+0000 7fe85ffef700 1 -- 172.21.15.36:0/1408257816 wait complete. 2022-06-17T12:48:08.546 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 14 2022-06-17T12:48:08.547 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_decay_threshold 400 2022-06-17T12:48:08.670 INFO:teuthology.orchestra.run.smithi036.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2022-06-17T12:48:08.671 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2022-06-17T12:48:08.672 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:48:08.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.809+0000 7fc8ea45d700 1 Processor -- start 2022-06-17T12:48:08.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.809+0000 7fc8ea45d700 1 -- start start 2022-06-17T12:48:08.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8ea45d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc8e4062270 0x7fc8e4142f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:08.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8ea45d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8e4143e80 0x7fc8e4146970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:08.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8ea45d700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e40b44b0 con 0x7fc8e40b0cb0 2022-06-17T12:48:08.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8ea45d700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e40b6650 con 0x7fc8e4062270 2022-06-17T12:48:08.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8ea45d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e4062650 con 0x7fc8e4143e80 2022-06-17T12:48:08.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8e89fa700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8e4143e80 0x7fc8e4146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:08.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8e3fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc8e4062270 0x7fc8e4142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:08.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8e3fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc8e4062270 0x7fc8e4142f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:53308/0 (socket says 172.21.15.36:53308) 2022-06-17T12:48:08.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8e3fff700 1 -- 172.21.15.36:0/2380773676 learned_addr learned my addr 172.21.15.36:0/2380773676 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:48:08.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8e37fe700 1 --1- 172.21.15.36:0/2380773676 >> v1:172.21.15.70:6789/0 conn(0x7fc8e40b0cb0 0x7fc8e4143770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:46192/0 (socket says 172.21.15.36:46192) 2022-06-17T12:48:08.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8e3fff700 1 -- 172.21.15.36:0/2380773676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8e4143e80 msgr2=0x7fc8e4146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:48:08.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8e3fff700 1 --2- 172.21.15.36:0/2380773676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8e4143e80 0x7fc8e4146970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:08.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8e3fff700 1 -- 172.21.15.36:0/2380773676 >> v1:172.21.15.70:6789/0 conn(0x7fc8e40b0cb0 legacy=0x7fc8e4143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:08.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8e3fff700 1 -- 172.21.15.36:0/2380773676 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e40b4210 con 0x7fc8e4062270 2022-06-17T12:48:08.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8e89fa700 1 --2- 172.21.15.36:0/2380773676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8e4143e80 0x7fc8e4146970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:48:08.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8e3fff700 1 --2- 172.21.15.36:0/2380773676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e4062270 0x7fc8e4142f20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fc8d4002f70 tx=0x7fc8d4006eb0).ready entity=mon.1 client_cookie=5a5c097737744654 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:08.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8e2ffd700 1 -- 172.21.15.36:0/2380773676 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8d400d570 con 0x7fc8e4062270 2022-06-17T12:48:08.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8e2ffd700 1 -- 172.21.15.36:0/2380773676 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d4000e90 con 0x7fc8e4062270 2022-06-17T12:48:08.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8e2ffd700 1 -- 172.21.15.36:0/2380773676 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8d400d900 con 0x7fc8e4062270 2022-06-17T12:48:08.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8ea45d700 1 -- 172.21.15.36:0/2380773676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e4062270 msgr2=0x7fc8e4142f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:08.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8ea45d700 1 --2- 172.21.15.36:0/2380773676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e4062270 0x7fc8e4142f20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fc8d4002f70 tx=0x7fc8d4006eb0).stop 2022-06-17T12:48:08.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8ea45d700 1 -- 172.21.15.36:0/2380773676 shutdown_connections 2022-06-17T12:48:08.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8ea45d700 1 --2- 172.21.15.36:0/2380773676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8e4143e80 0x7fc8e4146970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:08.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8ea45d700 1 --2- 172.21.15.36:0/2380773676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e4062270 0x7fc8e4142f20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:08.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8ea45d700 1 -- 172.21.15.36:0/2380773676 >> 172.21.15.36:0/2380773676 conn(0x7fc8e40b9870 msgr2=0x7fc8e40b9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:48:08.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8ea45d700 1 -- 172.21.15.36:0/2380773676 shutdown_connections 2022-06-17T12:48:08.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.813+0000 7fc8ea45d700 1 -- 172.21.15.36:0/2380773676 wait complete. 2022-06-17T12:48:08.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.817+0000 7fc8ea45d700 1 Processor -- start 2022-06-17T12:48:08.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.817+0000 7fc8ea45d700 1 -- start start 2022-06-17T12:48:08.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.817+0000 7fc8ea45d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8e4062270 0x7fc8e4152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:08.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.817+0000 7fc8ea45d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e4143e80 0x7fc8e4148940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:08.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.817+0000 7fc8ea45d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8e41131b0 0x7fc8e40b2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:08.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.817+0000 7fc8ea45d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e4115500 con 0x7fc8e4062270 2022-06-17T12:48:08.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.817+0000 7fc8ea45d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e4115760 con 0x7fc8e4143e80 2022-06-17T12:48:08.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.817+0000 7fc8ea45d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e4148290 con 0x7fc8e41131b0 2022-06-17T12:48:08.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.817+0000 7fc8e3fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8e4062270 0x7fc8e4152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:08.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.817+0000 7fc8e3fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8e4062270 0x7fc8e4152620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32932/0 (socket says 172.21.15.36:32932) 2022-06-17T12:48:08.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.817+0000 7fc8e3fff700 1 -- 172.21.15.36:0/511522862 learned_addr learned my addr 172.21.15.36:0/511522862 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:48:08.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.817+0000 7fc8e89fa700 1 --2- 172.21.15.36:0/511522862 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8e41131b0 0x7fc8e40b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:08.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.817+0000 7fc8e37fe700 1 --2- 172.21.15.36:0/511522862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e4143e80 0x7fc8e4148940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:08.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.817+0000 7fc8e3fff700 1 -- 172.21.15.36:0/511522862 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8e41131b0 msgr2=0x7fc8e40b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:08.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.817+0000 7fc8e3fff700 1 --2- 172.21.15.36:0/511522862 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8e41131b0 0x7fc8e40b2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:08.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.817+0000 7fc8e3fff700 1 -- 172.21.15.36:0/511522862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e4143e80 msgr2=0x7fc8e4148940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:08.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.817+0000 7fc8e3fff700 1 --2- 172.21.15.36:0/511522862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e4143e80 0x7fc8e4148940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:08.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.817+0000 7fc8e3fff700 1 -- 172.21.15.36:0/511522862 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8dc009470 con 0x7fc8e4062270 2022-06-17T12:48:08.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.817+0000 7fc8e3fff700 1 --2- 172.21.15.36:0/511522862 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8e4062270 0x7fc8e4152620 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7fc8d4002f70 tx=0x7fc8d4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:08.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.817+0000 7fc8e17fa700 1 -- 172.21.15.36:0/511522862 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8d40117e0 con 0x7fc8e4062270 2022-06-17T12:48:08.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.817+0000 7fc8ea45d700 1 -- 172.21.15.36:0/511522862 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d400b1e0 con 0x7fc8e4062270 2022-06-17T12:48:08.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.817+0000 7fc8ea45d700 1 -- 172.21.15.36:0/511522862 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e4112f80 con 0x7fc8e4062270 2022-06-17T12:48:08.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.817+0000 7fc8e17fa700 1 -- 172.21.15.36:0/511522862 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d4011940 con 0x7fc8e4062270 2022-06-17T12:48:08.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.817+0000 7fc8e17fa700 1 -- 172.21.15.36:0/511522862 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8d4011ca0 con 0x7fc8e4062270 2022-06-17T12:48:08.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.821+0000 7fc8e17fa700 1 -- 172.21.15.36:0/511522862 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fc8d401a040 con 0x7fc8e4062270 2022-06-17T12:48:08.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.821+0000 7fc8e17fa700 1 --2- 172.21.15.36:0/511522862 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc8c4053ad0 0x7fc8c4055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:08.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.821+0000 7fc8e17fa700 1 -- 172.21.15.36:0/511522862 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc8d406cf20 con 0x7fc8e4062270 2022-06-17T12:48:08.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.821+0000 7fc8e37fe700 1 --2- 172.21.15.36:0/511522862 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc8c4053ad0 0x7fc8c4055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:08.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.821+0000 7fc8e37fe700 1 --2- 172.21.15.36:0/511522862 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc8c4053ad0 0x7fc8c4055f90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fc8d800d000 tx=0x7fc8d8007b80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:08.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.821+0000 7fc8ea45d700 1 -- 172.21.15.36:0/511522862 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e40b63b0 con 0x7fc8e4062270 2022-06-17T12:48:08.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.829+0000 7fc8e17fa700 1 -- 172.21.15.36:0/511522862 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc8d4023080 con 0x7fc8e4062270 2022-06-17T12:48:09.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.993+0000 7fc8ea45d700 1 -- 172.21.15.36:0/511522862 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8e4057410 con 0x7fc8e4062270 2022-06-17T12:48:09.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.993+0000 7fc8e17fa700 1 -- 172.21.15.36:0/511522862 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+2138 (secure 0 0 0) 0x7fc8d4043c80 con 0x7fc8e4062270 2022-06-17T12:48:09.016 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:48:09.017 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3597082917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3597082917},{"type":"v1","addr":"172.21.15.70:6833","nonce":3597082917}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:47:47.072139+0000","modified":"2022-06-17T12:47:49.085163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4947":{"gid":4947,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.36:6835/1648255567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":1648255567},{"type":"v1","addr":"172.21.15.36:6835","nonce":1648255567}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-06-17T12:48:09.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.997+0000 7fc8ea45d700 1 -- 172.21.15.36:0/511522862 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc8c4053ad0 msgr2=0x7fc8c4055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:09.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.997+0000 7fc8ea45d700 1 --2- 172.21.15.36:0/511522862 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc8c4053ad0 0x7fc8c4055f90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fc8d800d000 tx=0x7fc8d8007b80).stop 2022-06-17T12:48:09.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.997+0000 7fc8ea45d700 1 -- 172.21.15.36:0/511522862 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8e4062270 msgr2=0x7fc8e4152620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:09.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.997+0000 7fc8ea45d700 1 --2- 172.21.15.36:0/511522862 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8e4062270 0x7fc8e4152620 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7fc8d4002f70 tx=0x7fc8d4015000).stop 2022-06-17T12:48:09.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.997+0000 7fc8ea45d700 1 -- 172.21.15.36:0/511522862 shutdown_connections 2022-06-17T12:48:09.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.997+0000 7fc8ea45d700 1 --2- 172.21.15.36:0/511522862 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc8c4053ad0 0x7fc8c4055f90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.997+0000 7fc8ea45d700 1 --2- 172.21.15.36:0/511522862 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8e41131b0 0x7fc8e40b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.997+0000 7fc8ea45d700 1 --2- 172.21.15.36:0/511522862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e4143e80 0x7fc8e4148940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.997+0000 7fc8ea45d700 1 --2- 172.21.15.36:0/511522862 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8e4062270 0x7fc8e4152620 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.997+0000 7fc8ea45d700 1 -- 172.21.15.36:0/511522862 >> 172.21.15.36:0/511522862 conn(0x7fc8e40b9870 msgr2=0x7fc8e40b77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:48:09.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.997+0000 7fc8ea45d700 1 -- 172.21.15.36:0/511522862 shutdown_connections 2022-06-17T12:48:09.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:08.997+0000 7fc8ea45d700 1 -- 172.21.15.36:0/511522862 wait complete. 2022-06-17T12:48:09.039 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 14 2022-06-17T12:48:09.042 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a cache drop 2022-06-17T12:48:09.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.181+0000 7f37a320d700 1 Processor -- start 2022-06-17T12:48:09.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.181+0000 7f37a320d700 1 -- start start 2022-06-17T12:48:09.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.181+0000 7f37a320d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f379c0b7bc0 0x7f379c140e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:09.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.181+0000 7f37a320d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f379c0ae150 0x7f379c0b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:09.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.181+0000 7f37a320d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f379c0b2fc0 0x7f379c144450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:09.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.181+0000 7f37a320d700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f379c0b7fa0 con 0x7f379c0b2fc0 2022-06-17T12:48:09.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.181+0000 7f37a320d700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f379c0b20f0 con 0x7f379c0b7bc0 2022-06-17T12:48:09.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.181+0000 7f37a320d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379c0b2390 con 0x7f379c0ae150 2022-06-17T12:48:09.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.181+0000 7f37a17aa700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f379c0b2fc0 0x7f379c144450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:09.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.181+0000 7f37a17aa700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f379c0b2fc0 0x7f379c144450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32944/0 (socket says 172.21.15.36:32944) 2022-06-17T12:48:09.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.181+0000 7f37a17aa700 1 -- 172.21.15.36:0/1358007776 learned_addr learned my addr 172.21.15.36:0/1358007776 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:48:09.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.181+0000 7f379bfff700 1 --2- 172.21.15.36:0/1358007776 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f379c0ae150 0x7f379c0b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:09.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.181+0000 7f37a0fa9700 1 --2- 172.21.15.36:0/1358007776 >> v2:172.21.15.70:3300/0 conn(0x7f379c0b7bc0 0x7f379c140e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:09.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.181+0000 7f37a17aa700 1 -- 172.21.15.36:0/1358007776 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f379c0ae150 msgr2=0x7f379c0b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:48:09.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f37a17aa700 1 --2- 172.21.15.36:0/1358007776 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f379c0ae150 0x7f379c0b2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f37a17aa700 1 -- 172.21.15.36:0/1358007776 >> v2:172.21.15.70:3300/0 conn(0x7f379c0b7bc0 msgr2=0x7f379c140e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:48:09.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f37a17aa700 1 --2- 172.21.15.36:0/1358007776 >> v2:172.21.15.70:3300/0 conn(0x7f379c0b7bc0 0x7f379c140e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f37a17aa700 1 -- 172.21.15.36:0/1358007776 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c0b2630 con 0x7f379c0b2fc0 2022-06-17T12:48:09.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f37a0fa9700 1 --2- 172.21.15.36:0/1358007776 >> v2:172.21.15.70:3300/0 conn(0x7f379c0b7bc0 0x7f379c140e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:48:09.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f37a17aa700 1 --2- 172.21.15.36:0/1358007776 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f379c0b2fc0 0x7f379c144450 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f37940032f0 tx=0x7f3794009290).ready entity=mon.0 client_cookie=c709a6bb791f3fc server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:09.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f379b7fe700 1 -- 172.21.15.36:0/1358007776 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f379400b680 con 0x7f379c0b2fc0 2022-06-17T12:48:09.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f379b7fe700 1 -- 172.21.15.36:0/1358007776 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379400b7e0 con 0x7f379c0b2fc0 2022-06-17T12:48:09.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f379b7fe700 1 -- 172.21.15.36:0/1358007776 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f379400b950 con 0x7f379c0b2fc0 2022-06-17T12:48:09.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f37a320d700 1 -- 172.21.15.36:0/1358007776 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f379c0b2fc0 msgr2=0x7f379c144450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:09.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f37a320d700 1 --2- 172.21.15.36:0/1358007776 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f379c0b2fc0 0x7f379c144450 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f37940032f0 tx=0x7f3794009290).stop 2022-06-17T12:48:09.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f37a320d700 1 -- 172.21.15.36:0/1358007776 shutdown_connections 2022-06-17T12:48:09.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f37a320d700 1 --2- 172.21.15.36:0/1358007776 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f379c0b2fc0 0x7f379c144450 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f37a320d700 1 --2- 172.21.15.36:0/1358007776 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f379c0ae150 0x7f379c0b2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f37a320d700 1 --2- 172.21.15.36:0/1358007776 >> v2:172.21.15.70:3300/0 conn(0x7f379c0b7bc0 0x7f379c140e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f37a320d700 1 -- 172.21.15.36:0/1358007776 >> 172.21.15.36:0/1358007776 conn(0x7f379c060990 msgr2=0x7f379c05c550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:48:09.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f37a320d700 1 -- 172.21.15.36:0/1358007776 shutdown_connections 2022-06-17T12:48:09.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f37a320d700 1 -- 172.21.15.36:0/1358007776 wait complete. 2022-06-17T12:48:09.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f37a320d700 1 Processor -- start 2022-06-17T12:48:09.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f37a320d700 1 -- start start 2022-06-17T12:48:09.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f37a320d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f379c0ae150 0x7f379c0b6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:09.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f37a320d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f379c0b2fc0 0x7f379c0afe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:09.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f37a320d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f379c0b7bc0 0x7f379c12c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:09.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f37a320d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c0b1040 con 0x7f379c0b2fc0 2022-06-17T12:48:09.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f37a320d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c0b12e0 con 0x7f379c0ae150 2022-06-17T12:48:09.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f37a320d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379c0b0d80 con 0x7f379c0b7bc0 2022-06-17T12:48:09.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f379bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f379c0b2fc0 0x7f379c0afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:09.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f379bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f379c0b2fc0 0x7f379c0afe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32948/0 (socket says 172.21.15.36:32948) 2022-06-17T12:48:09.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f379bfff700 1 -- 172.21.15.36:0/1599923853 learned_addr learned my addr 172.21.15.36:0/1599923853 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:48:09.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.185+0000 7f37a17aa700 1 --2- 172.21.15.36:0/1599923853 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f379c0b7bc0 0x7f379c12c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:09.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.189+0000 7f37a0fa9700 1 --2- 172.21.15.36:0/1599923853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f379c0ae150 0x7f379c0b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:09.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.189+0000 7f379bfff700 1 -- 172.21.15.36:0/1599923853 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f379c0b7bc0 msgr2=0x7f379c12c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:09.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.189+0000 7f379bfff700 1 --2- 172.21.15.36:0/1599923853 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f379c0b7bc0 0x7f379c12c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.189+0000 7f379bfff700 1 -- 172.21.15.36:0/1599923853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f379c0ae150 msgr2=0x7f379c0b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:09.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.189+0000 7f379bfff700 1 --2- 172.21.15.36:0/1599923853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f379c0ae150 0x7f379c0b6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.189+0000 7f379bfff700 1 -- 172.21.15.36:0/1599923853 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379000b180 con 0x7f379c0b2fc0 2022-06-17T12:48:09.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.189+0000 7f379bfff700 1 --2- 172.21.15.36:0/1599923853 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f379c0b2fc0 0x7f379c0afe90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f378c00c000 tx=0x7f378c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:09.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.189+0000 7f3799ffb700 1 -- 172.21.15.36:0/1599923853 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f378c010780 con 0x7f379c0b2fc0 2022-06-17T12:48:09.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.189+0000 7f3799ffb700 1 -- 172.21.15.36:0/1599923853 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378c00b3b0 con 0x7f379c0b2fc0 2022-06-17T12:48:09.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.189+0000 7f37a320d700 1 -- 172.21.15.36:0/1599923853 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3794009530 con 0x7f379c0b2fc0 2022-06-17T12:48:09.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.189+0000 7f3799ffb700 1 -- 172.21.15.36:0/1599923853 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f378c010ad0 con 0x7f379c0b2fc0 2022-06-17T12:48:09.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.189+0000 7f37a320d700 1 -- 172.21.15.36:0/1599923853 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379c13dea0 con 0x7f379c0b2fc0 2022-06-17T12:48:09.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.189+0000 7f3799ffb700 1 -- 172.21.15.36:0/1599923853 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f378c010c30 con 0x7f379c0b2fc0 2022-06-17T12:48:09.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.189+0000 7f3799ffb700 1 --2- 172.21.15.36:0/1599923853 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f377c053a40 0x7f377c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:09.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.189+0000 7f3799ffb700 1 -- 172.21.15.36:0/1599923853 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f378c06c3f0 con 0x7f379c0b2fc0 2022-06-17T12:48:09.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.189+0000 7f37a0fa9700 1 --2- 172.21.15.36:0/1599923853 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f377c053a40 0x7f377c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:09.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.193+0000 7f37a0fa9700 1 --2- 172.21.15.36:0/1599923853 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f377c053a40 0x7f377c055f00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f379000b000 tx=0x7f3790006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:09.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.193+0000 7f37ad462740 1 Processor -- start 2022-06-17T12:48:09.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.193+0000 7f37ad462740 1 -- start start 2022-06-17T12:48:09.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.193+0000 7f37ad462740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x1fc7470 0x1f88100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:09.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.193+0000 7f37ad462740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x1f8b5d0 0x1f885f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:09.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.193+0000 7f37ad462740 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8d1c0 0x1f88ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:09.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.193+0000 7f37ad462740 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x1efdc60 con 0x1f8d1c0 2022-06-17T12:48:09.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.193+0000 7f37ad462740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x1eddec0 con 0x1f8b5d0 2022-06-17T12:48:09.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.193+0000 7f37ad462740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f88fd0 con 0x1fc7470 2022-06-17T12:48:09.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.193+0000 7f37a17aa700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8d1c0 0x1f88ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:09.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.193+0000 7f37a17aa700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8d1c0 0x1f88ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32958/0 (socket says 172.21.15.36:32958) 2022-06-17T12:48:09.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.193+0000 7f379bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x1f8b5d0 0x1f885f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:09.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.193+0000 7f37a0fa9700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x1fc7470 0x1f88100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:09.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.193+0000 7f37a17aa700 1 -- 172.21.15.36:0/3724653521 learned_addr learned my addr 172.21.15.36:0/3724653521 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:48:09.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.193+0000 7f37a17aa700 1 -- 172.21.15.36:0/3724653521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x1fc7470 msgr2=0x1f88100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:48:09.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.193+0000 7f37a17aa700 1 --2- 172.21.15.36:0/3724653521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x1fc7470 0x1f88100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.193+0000 7f37a17aa700 1 -- 172.21.15.36:0/3724653521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x1f8b5d0 msgr2=0x1f885f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:48:09.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.193+0000 7f37a17aa700 1 --2- 172.21.15.36:0/3724653521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x1f8b5d0 0x1f885f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.193+0000 7f37a17aa700 1 -- 172.21.15.36:0/3724653521 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1f35520 con 0x1f8d1c0 2022-06-17T12:48:09.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.193+0000 7f37a17aa700 1 --2- 172.21.15.36:0/3724653521 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8d1c0 0x1f88ae0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x1f8d610 tx=0x7f3794017000).ready entity=mon.0 client_cookie=12c988610f0b63f6 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:09.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37a320d700 1 -- 172.21.15.36:0/3724653521 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f379400b680 con 0x1f8d1c0 2022-06-17T12:48:09.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37a320d700 1 -- 172.21.15.36:0/3724653521 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3794006cc0 con 0x1f8d1c0 2022-06-17T12:48:09.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37a320d700 1 -- 172.21.15.36:0/3724653521 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f379400bb40 con 0x1f8d1c0 2022-06-17T12:48:09.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37ad462740 1 -- 172.21.15.36:0/3724653521 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8d1c0 msgr2=0x1f88ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:09.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37ad462740 1 --2- 172.21.15.36:0/3724653521 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8d1c0 0x1f88ae0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x1f8d610 tx=0x7f3794017000).stop 2022-06-17T12:48:09.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37ad462740 1 -- 172.21.15.36:0/3724653521 shutdown_connections 2022-06-17T12:48:09.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37ad462740 1 --2- 172.21.15.36:0/3724653521 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8d1c0 0x1f88ae0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37ad462740 1 --2- 172.21.15.36:0/3724653521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x1f8b5d0 0x1f885f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37ad462740 1 --2- 172.21.15.36:0/3724653521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x1fc7470 0x1f88100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37ad462740 1 -- 172.21.15.36:0/3724653521 >> 172.21.15.36:0/3724653521 conn(0x1f15340 msgr2=0x1fc6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:48:09.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37ad462740 1 -- 172.21.15.36:0/3724653521 shutdown_connections 2022-06-17T12:48:09.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37ad462740 1 -- 172.21.15.36:0/3724653521 wait complete. 2022-06-17T12:48:09.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37ad462740 1 -- start start 2022-06-17T12:48:09.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37ad462740 1 Processor -- start 2022-06-17T12:48:09.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37ad462740 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8b5d0 0x1f98350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:09.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37ad462740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x1f8d1c0 0x1f94d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:09.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37ad462740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x1fc7470 0x1fc0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:09.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37ad462740 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fd9a90 con 0x1f8b5d0 2022-06-17T12:48:09.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37ad462740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fd9cc0 con 0x1fc7470 2022-06-17T12:48:09.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37ad462740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f96c10 con 0x1f8d1c0 2022-06-17T12:48:09.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37ad462740 10 client.0 ll_register_callbacks cb 0x1f447d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-06-17T12:48:09.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37a0fa9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8b5d0 0x1f98350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:09.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37a0fa9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8b5d0 0x1f98350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32960/0 (socket says 172.21.15.36:32960) 2022-06-17T12:48:09.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37a0fa9700 1 -- 172.21.15.36:0/1621838456 learned_addr learned my addr 172.21.15.36:0/1621838456 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:48:09.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f379bfff700 1 --2- 172.21.15.36:0/1621838456 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x1f8d1c0 0x1f94d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:09.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37a17aa700 1 --2- 172.21.15.36:0/1621838456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x1fc7470 0x1fc0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:09.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37a0fa9700 1 -- 172.21.15.36:0/1621838456 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x1f8d1c0 msgr2=0x1f94d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:48:09.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37a0fa9700 1 --2- 172.21.15.36:0/1621838456 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x1f8d1c0 0x1f94d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37a0fa9700 1 -- 172.21.15.36:0/1621838456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x1fc7470 msgr2=0x1fc0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:48:09.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37a0fa9700 1 --2- 172.21.15.36:0/1621838456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x1fc7470 0x1fc0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.197+0000 7f37a0fa9700 1 -- 172.21.15.36:0/1621838456 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379c0b2630 con 0x1f8b5d0 2022-06-17T12:48:09.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f37a0fa9700 1 --2- 172.21.15.36:0/1621838456 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8b5d0 0x1f98350 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f379001a000 tx=0x7f3790016000).ready entity=mon.0 client_cookie=288e49ec88f7591 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:09.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f377b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.36:3300/0 2022-06-17T12:48:09.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f377b7fe700 1 -- 172.21.15.36:0/1621838456 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3790014e50 con 0x1f8b5d0 2022-06-17T12:48:09.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f377b7fe700 1 -- 172.21.15.36:0/1621838456 <== mon.0 v2:172.21.15.36:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3790013a10 con 0x1f8b5d0 2022-06-17T12:48:09.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f37ad462740 1 -- 172.21.15.36:0/1621838456 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1ef3b00 con 0x1f8b5d0 2022-06-17T12:48:09.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f377b7fe700 1 -- 172.21.15.36:0/1621838456 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3790013bd0 con 0x1f8b5d0 2022-06-17T12:48:09.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f37ad462740 10 client.5175 fetch_fsmap learned FSMap version 14 2022-06-17T12:48:09.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f37ad462740 1 -- 172.21.15.36:0/1621838456 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x1eed0f0 con 0x1f8b5d0 2022-06-17T12:48:09.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f377b7fe700 1 -- 172.21.15.36:0/1621838456 <== mon.0 v2:172.21.15.36:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1ef3b00 con 0x1f8b5d0 2022-06-17T12:48:09.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f377b7fe700 1 -- 172.21.15.36:0/1621838456 <== mon.0 v2:172.21.15.36:3300/0 5 ==== fsmap(e 14) v1 ==== 1247+0+0 (secure 0 0 0) 0x7f37900288f0 con 0x1f8b5d0 2022-06-17T12:48:09.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f37ad462740 10 client.5175 fetch_fsmap finished waiting for FSMap version 14 2022-06-17T12:48:09.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f37ad462740 10 client.5175 resolve_mds: resolved ID 'a' to GID 4947 2022-06-17T12:48:09.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f37ad462740 20 client.5175 populate_metadata read hostname 'smithi036' 2022-06-17T12:48:09.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f37ad462740 1 --2- 172.21.15.36:0/1621838456 >> [v2:172.21.15.36:6834/1648255567,v1:172.21.15.36:6835/1648255567] conn(0x1f95270 0x1fc13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:09.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f37ad462740 4 client.5175 mds_command: new command op to 4947 tid=0[{"prefix": "get_command_descriptions"}] 2022-06-17T12:48:09.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f37ad462740 1 -- 172.21.15.36:0/1621838456 --> [v2:172.21.15.36:6834/1648255567,v1:172.21.15.36:6835/1648255567] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1eebb80 con 0x1f95270 2022-06-17T12:48:09.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f379bfff700 1 --2- 172.21.15.36:0/1621838456 >> [v2:172.21.15.36:6834/1648255567,v1:172.21.15.36:6835/1648255567] conn(0x1f95270 0x1fc13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:09.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f377b7fe700 10 client.5175 ms_handle_connect on v2:172.21.15.36:6834/1648255567 2022-06-17T12:48:09.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f379bfff700 1 --2- 172.21.15.36:0/1621838456 >> [v2:172.21.15.36:6834/1648255567,v1:172.21.15.36:6835/1648255567] conn(0x1f95270 0x1fc13f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=494b41dbb275adba server_cookie=404808b9714a7902 in_seq=0 out_seq=0 2022-06-17T12:48:09.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f377b7fe700 1 -- 172.21.15.36:0/1621838456 <== mds.0 v2:172.21.15.36:6834/1648255567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1eebb80 con 0x1f95270 2022-06-17T12:48:09.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f377b7fe700 10 client.5175 handle_command_reply: tid=0 2022-06-17T12:48:09.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f37ad462740 1 client.5175 shutdown 2022-06-17T12:48:09.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f37ad462740 1 -- 172.21.15.36:0/1621838456 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8b5d0 msgr2=0x1f98350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:09.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f37ad462740 1 --2- 172.21.15.36:0/1621838456 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8b5d0 0x1f98350 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f379001a000 tx=0x7f3790016000).stop 2022-06-17T12:48:09.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f37ad462740 1 -- 172.21.15.36:0/1621838456 shutdown_connections 2022-06-17T12:48:09.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f37ad462740 1 --2- 172.21.15.36:0/1621838456 >> [v2:172.21.15.36:6834/1648255567,v1:172.21.15.36:6835/1648255567] conn(0x1f95270 0x1fc13f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f37ad462740 1 --2- 172.21.15.36:0/1621838456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x1fc7470 0x1fc0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f37ad462740 1 --2- 172.21.15.36:0/1621838456 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x1f8d1c0 0x1f94d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f37ad462740 1 --2- 172.21.15.36:0/1621838456 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8b5d0 0x1f98350 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f37ad462740 1 -- 172.21.15.36:0/1621838456 >> 172.21.15.36:0/1621838456 conn(0x1f15340 msgr2=0x1fc3430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:48:09.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f377b7fe700 0 client.5175 ms_handle_reset on v2:172.21.15.36:6834/1648255567 2022-06-17T12:48:09.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.201+0000 7f37ad462740 1 -- 172.21.15.36:0/1621838456 shutdown_connections 2022-06-17T12:48:09.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.205+0000 7f37ad462740 1 -- 172.21.15.36:0/1621838456 wait complete. 2022-06-17T12:48:09.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.205+0000 7f37ad462740 20 client.5175 trim_cache size 0 max 16384 2022-06-17T12:48:09.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.217+0000 7f37ad462740 1 Processor -- start 2022-06-17T12:48:09.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37ad462740 1 -- start start 2022-06-17T12:48:09.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37ad462740 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8b5d0 0x1fc0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:09.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37ad462740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x1f8d1c0 0x1f95d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:09.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37ad462740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x1fc7470 0x1f96280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:09.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37ad462740 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f99f80 con 0x1f8b5d0 2022-06-17T12:48:09.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37ad462740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f0f960 con 0x1fc7470 2022-06-17T12:48:09.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37ad462740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x1fc2b90 con 0x1f8d1c0 2022-06-17T12:48:09.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37a0fa9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8b5d0 0x1fc0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:09.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37a0fa9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8b5d0 0x1fc0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32968/0 (socket says 172.21.15.36:32968) 2022-06-17T12:48:09.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37a0fa9700 1 -- 172.21.15.36:0/3297105824 learned_addr learned my addr 172.21.15.36:0/3297105824 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:48:09.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f379bfff700 1 --2- 172.21.15.36:0/3297105824 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x1f8d1c0 0x1f95d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:09.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37a17aa700 1 --2- 172.21.15.36:0/3297105824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x1fc7470 0x1f96280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:09.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37a0fa9700 1 -- 172.21.15.36:0/3297105824 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x1f8d1c0 msgr2=0x1f95d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:48:09.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37a0fa9700 1 --2- 172.21.15.36:0/3297105824 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x1f8d1c0 0x1f95d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37a0fa9700 1 -- 172.21.15.36:0/3297105824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x1fc7470 msgr2=0x1f96280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:48:09.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37a0fa9700 1 --2- 172.21.15.36:0/3297105824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x1fc7470 0x1f96280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37a0fa9700 1 -- 172.21.15.36:0/3297105824 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1fc2700 con 0x1f8b5d0 2022-06-17T12:48:09.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37a17aa700 1 --2- 172.21.15.36:0/3297105824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x1fc7470 0x1f96280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:48:09.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37a0fa9700 1 --2- 172.21.15.36:0/3297105824 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8b5d0 0x1fc0ba0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x1fda0b0 tx=0x7f3790025000).ready entity=mon.0 client_cookie=1cfef786da5e432 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:09.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f3781ffb700 1 -- 172.21.15.36:0/3297105824 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3790037070 con 0x1f8b5d0 2022-06-17T12:48:09.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f3781ffb700 1 -- 172.21.15.36:0/3297105824 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3790000e90 con 0x1f8b5d0 2022-06-17T12:48:09.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f3781ffb700 1 -- 172.21.15.36:0/3297105824 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3790011e90 con 0x1f8b5d0 2022-06-17T12:48:09.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37ad462740 1 -- 172.21.15.36:0/3297105824 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8b5d0 msgr2=0x1fc0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:09.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37ad462740 1 --2- 172.21.15.36:0/3297105824 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8b5d0 0x1fc0ba0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x1fda0b0 tx=0x7f3790025000).stop 2022-06-17T12:48:09.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37ad462740 1 -- 172.21.15.36:0/3297105824 shutdown_connections 2022-06-17T12:48:09.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37ad462740 1 --2- 172.21.15.36:0/3297105824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x1fc7470 0x1f96280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37ad462740 1 --2- 172.21.15.36:0/3297105824 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x1f8d1c0 0x1f95d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37ad462740 1 --2- 172.21.15.36:0/3297105824 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8b5d0 0x1fc0ba0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37ad462740 1 -- 172.21.15.36:0/3297105824 >> 172.21.15.36:0/3297105824 conn(0x1f15340 msgr2=0x1f97870 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:48:09.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37ad462740 1 -- 172.21.15.36:0/3297105824 shutdown_connections 2022-06-17T12:48:09.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37ad462740 1 -- 172.21.15.36:0/3297105824 wait complete. 2022-06-17T12:48:09.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37ad462740 1 -- start start 2022-06-17T12:48:09.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.221+0000 7f37ad462740 1 Processor -- start 2022-06-17T12:48:09.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f37ad462740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x1f8b5d0 0x2026530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:09.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f37ad462740 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8d1c0 0x2026a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:09.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f37ad462740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x1fc7470 0x202cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:09.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f37ad462740 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fcb9b0 con 0x1f8d1c0 2022-06-17T12:48:09.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f37ad462740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x2026f10 con 0x1f8b5d0 2022-06-17T12:48:09.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f37ad462740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x1fd3360 con 0x1fc7470 2022-06-17T12:48:09.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f37ad462740 10 client.0 ll_register_callbacks cb 0x7f3790022030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-06-17T12:48:09.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f379bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8d1c0 0x2026a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:09.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f379bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8d1c0 0x2026a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32974/0 (socket says 172.21.15.36:32974) 2022-06-17T12:48:09.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f379bfff700 1 -- 172.21.15.36:0/3390196290 learned_addr learned my addr 172.21.15.36:0/3390196290 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:48:09.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f37a17aa700 1 --2- 172.21.15.36:0/3390196290 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x1fc7470 0x202cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:09.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f37a0fa9700 1 --2- 172.21.15.36:0/3390196290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x1f8b5d0 0x2026530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:09.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f379bfff700 1 -- 172.21.15.36:0/3390196290 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x1fc7470 msgr2=0x202cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:48:09.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f379bfff700 1 --2- 172.21.15.36:0/3390196290 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x1fc7470 0x202cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f379bfff700 1 -- 172.21.15.36:0/3390196290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x1f8b5d0 msgr2=0x2026530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:48:09.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f379bfff700 1 --2- 172.21.15.36:0/3390196290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x1f8b5d0 0x2026530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:09.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f379bfff700 1 -- 172.21.15.36:0/3390196290 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1eed0f0 con 0x1f8d1c0 2022-06-17T12:48:09.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f379bfff700 1 --2- 172.21.15.36:0/3390196290 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8d1c0 0x2026a20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f378c01d000 tx=0x7f378c014000).ready entity=mon.0 client_cookie=9e6f5f05c9d3f7cb server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:09.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f3780ff9700 10 client.0 ms_handle_connect on v2:172.21.15.36:3300/0 2022-06-17T12:48:09.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f3780ff9700 1 -- 172.21.15.36:0/3390196290 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f378c035030 con 0x1f8d1c0 2022-06-17T12:48:09.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f3780ff9700 1 -- 172.21.15.36:0/3390196290 <== mon.0 v2:172.21.15.36:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f378c036070 con 0x1f8d1c0 2022-06-17T12:48:09.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f37ad462740 1 -- 172.21.15.36:0/3390196290 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1fc0a10 con 0x1f8d1c0 2022-06-17T12:48:09.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f3780ff9700 1 -- 172.21.15.36:0/3390196290 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f378c02c040 con 0x1f8d1c0 2022-06-17T12:48:09.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f37ad462740 10 client.5181 fetch_fsmap learned FSMap version 14 2022-06-17T12:48:09.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f37ad462740 1 -- 172.21.15.36:0/3390196290 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x1ef26d0 con 0x1f8d1c0 2022-06-17T12:48:09.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f3780ff9700 1 -- 172.21.15.36:0/3390196290 <== mon.0 v2:172.21.15.36:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1fc0a10 con 0x1f8d1c0 2022-06-17T12:48:09.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f3780ff9700 1 -- 172.21.15.36:0/3390196290 <== mon.0 v2:172.21.15.36:3300/0 5 ==== fsmap(e 14) v1 ==== 1247+0+0 (secure 0 0 0) 0x7f378c042d00 con 0x1f8d1c0 2022-06-17T12:48:09.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f37ad462740 10 client.5181 fetch_fsmap finished waiting for FSMap version 14 2022-06-17T12:48:09.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f37ad462740 10 client.5181 resolve_mds: resolved ID 'a' to GID 4947 2022-06-17T12:48:09.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f37ad462740 20 client.5181 populate_metadata read hostname 'smithi036' 2022-06-17T12:48:09.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f37ad462740 1 --2- 172.21.15.36:0/3390196290 >> [v2:172.21.15.36:6834/1648255567,v1:172.21.15.36:6835/1648255567] conn(0x202d000 0x202d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:09.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f37ad462740 4 client.5181 mds_command: new command op to 4947 tid=0[{"prefix": "cache drop"}] 2022-06-17T12:48:09.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f37ad462740 1 -- 172.21.15.36:0/3390196290 --> [v2:172.21.15.36:6834/1648255567,v1:172.21.15.36:6835/1648255567] -- command(tid 0: {"prefix": "cache drop"}) v1 -- 0x7f3794006cc0 con 0x202d000 2022-06-17T12:48:09.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f37a0fa9700 1 --2- 172.21.15.36:0/3390196290 >> [v2:172.21.15.36:6834/1648255567,v1:172.21.15.36:6835/1648255567] conn(0x202d000 0x202d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:09.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.225+0000 7f3780ff9700 10 client.5181 ms_handle_connect on v2:172.21.15.36:6834/1648255567 2022-06-17T12:48:09.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:09.229+0000 7f37a0fa9700 1 --2- 172.21.15.36:0/3390196290 >> [v2:172.21.15.36:6834/1648255567,v1:172.21.15.36:6835/1648255567] conn(0x202d000 0x202d3e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=1ea90078d8186f14 server_cookie=b12c4ed4632849da in_seq=0 out_seq=0 2022-06-17T12:48:14.011 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:48:14.015 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:48:14.018 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:48:14.028 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:48:14.043 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:48:44.098 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:48:44.112 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:48:44.147 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:48:44.205 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:48:44.238 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:48:53.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f3780ff9700 1 -- 172.21.15.36:0/3390196290 <== mds.0 v2:172.21.15.36:6834/1648255567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+386 (crc 0 0 0) 0x7f3794006cc0 con 0x202d000 2022-06-17T12:48:53.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f3780ff9700 10 client.5181 handle_command_reply: tid=0 2022-06-17T12:48:53.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f37ad462740 1 client.5181 shutdown 2022-06-17T12:48:53.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f37ad462740 1 -- 172.21.15.36:0/3390196290 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8d1c0 msgr2=0x2026a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:53.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f37ad462740 1 --2- 172.21.15.36:0/3390196290 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8d1c0 0x2026a20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f378c01d000 tx=0x7f378c014000).stop 2022-06-17T12:48:53.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f37ad462740 1 -- 172.21.15.36:0/3390196290 shutdown_connections 2022-06-17T12:48:53.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f37ad462740 1 --2- 172.21.15.36:0/3390196290 >> [v2:172.21.15.36:6834/1648255567,v1:172.21.15.36:6835/1648255567] conn(0x202d000 0x202d3e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:53.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f37ad462740 1 --2- 172.21.15.36:0/3390196290 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x1fc7470 0x202cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:53.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f37ad462740 1 --2- 172.21.15.36:0/3390196290 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x1f8d1c0 0x2026a20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:53.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f37ad462740 1 --2- 172.21.15.36:0/3390196290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x1f8b5d0 0x2026530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:53.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f3780ff9700 0 client.5181 ms_handle_reset on v2:172.21.15.36:6834/1648255567 2022-06-17T12:48:53.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f37ad462740 1 -- 172.21.15.36:0/3390196290 >> 172.21.15.36:0/3390196290 conn(0x1f15340 msgr2=0x1f97870 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:48:53.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f37ad462740 1 -- 172.21.15.36:0/3390196290 shutdown_connections 2022-06-17T12:48:53.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f37ad462740 1 -- 172.21.15.36:0/3390196290 wait complete. 2022-06-17T12:48:53.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f37ad462740 20 client.5181 trim_cache size 0 max 16384 2022-06-17T12:48:53.653 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:48:53.653 INFO:teuthology.orchestra.run.smithi036.stdout: "client_recall": { 2022-06-17T12:48:53.653 INFO:teuthology.orchestra.run.smithi036.stdout: "return_code": 0, 2022-06-17T12:48:53.653 INFO:teuthology.orchestra.run.smithi036.stdout: "message": "(0) Success", 2022-06-17T12:48:53.654 INFO:teuthology.orchestra.run.smithi036.stdout: "recalled": 920 2022-06-17T12:48:53.654 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:53.654 INFO:teuthology.orchestra.run.smithi036.stdout: "flush_journal": { 2022-06-17T12:48:53.654 INFO:teuthology.orchestra.run.smithi036.stdout: "return_code": 0, 2022-06-17T12:48:53.655 INFO:teuthology.orchestra.run.smithi036.stdout: "message": "" 2022-06-17T12:48:53.655 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:53.655 INFO:teuthology.orchestra.run.smithi036.stdout: "trim_cache": { 2022-06-17T12:48:53.663 INFO:teuthology.orchestra.run.smithi036.stdout: "trimmed": 900 2022-06-17T12:48:53.664 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:53.664 INFO:teuthology.orchestra.run.smithi036.stdout: "cache": { 2022-06-17T12:48:53.664 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": { 2022-06-17T12:48:53.665 INFO:teuthology.orchestra.run.smithi036.stdout: "items": 2503, 2022-06-17T12:48:53.665 INFO:teuthology.orchestra.run.smithi036.stdout: "bytes": 431744 2022-06-17T12:48:53.665 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:53.665 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:53.666 INFO:teuthology.orchestra.run.smithi036.stdout: "duration": 44.373421745999998 2022-06-17T12:48:53.666 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:48:53.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f37a320d700 1 -- 172.21.15.36:0/1599923853 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f377c053a40 msgr2=0x7f377c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:53.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f37a320d700 1 --2- 172.21.15.36:0/1599923853 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f377c053a40 0x7f377c055f00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f379000b000 tx=0x7f3790006df0).stop 2022-06-17T12:48:53.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f37a320d700 1 -- 172.21.15.36:0/1599923853 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f379c0b2fc0 msgr2=0x7f379c0afe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:53.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f37a320d700 1 --2- 172.21.15.36:0/1599923853 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f379c0b2fc0 0x7f379c0afe90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f378c00c000 tx=0x7f378c006df0).stop 2022-06-17T12:48:53.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f37a320d700 1 -- 172.21.15.36:0/1599923853 shutdown_connections 2022-06-17T12:48:53.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f37a320d700 1 --2- 172.21.15.36:0/1599923853 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f377c053a40 0x7f377c055f00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:53.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f37a320d700 1 --2- 172.21.15.36:0/1599923853 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f379c0b7bc0 0x7f379c12c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:53.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f37a320d700 1 --2- 172.21.15.36:0/1599923853 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f379c0b2fc0 0x7f379c0afe90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:53.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f37a320d700 1 --2- 172.21.15.36:0/1599923853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f379c0ae150 0x7f379c0b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:53.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f37a320d700 1 -- 172.21.15.36:0/1599923853 >> 172.21.15.36:0/1599923853 conn(0x7f379c060990 msgr2=0x7f379c0b47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:48:53.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f37a320d700 1 -- 172.21.15.36:0/1599923853 shutdown_connections 2022-06-17T12:48:53.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:53.601+0000 7f37a320d700 1 -- 172.21.15.36:0/1599923853 wait complete. 2022-06-17T12:48:53.807 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-17T12:48:53.808 DEBUG:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:48:53.875 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:48:53.972 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:48:54.029 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:48:54.068 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:48:54.113 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-17T12:48:54.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:48:54.139 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:48:54.140 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:48:54.163 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:48:54.164 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:48:54.173 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2022-06-17T12:48:54.174 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:48:54.199 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:48:54.235 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[21817]: fuse finished with error 0 and tester_r 0 2022-06-17T12:48:54.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:48:54.277 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:48:54.282 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:48:54.345 INFO:teuthology.orchestra.run.smithi107.stdout:ext2/ext3 2022-06-17T12:48:54.348 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:48:54.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:48:54.364 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:48:54.395 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:48:54.396 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:48:54.397 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:48:54.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:48:54.427 INFO:teuthology.orchestra.run.smithi136.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-17T12:48:54.429 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-17T12:48:54.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:48:54.431 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:48:54.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:48:54.467 INFO:teuthology.orchestra.run.smithi136.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-17T12:48:54.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:48:54.469 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:48:54.469 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:48:54.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:48:54.548 INFO:teuthology.orchestra.run.smithi143.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:48:54.549 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-06-17T12:48:54.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:48:54.575 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:48:54.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:48:54.596 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:48:54.597 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-17T12:48:54.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb397c5700 1 Processor -- start 2022-06-17T12:48:54.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb397c5700 1 -- start start 2022-06-17T12:48:54.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb397c5700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb340ac340 0x7ffb3414b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:54.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb397c5700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb340ac870 con 0x7ffb340af7f0 2022-06-17T12:48:54.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb397c5700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb340acaa0 con 0x7ffb340ad140 2022-06-17T12:48:54.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb397c5700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb34148960 con 0x7ffb340ac340 2022-06-17T12:48:54.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb337fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7ffb340af7f0 0x7ffb34147b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50518/0 (socket says 172.21.15.36:50518) 2022-06-17T12:48:54.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb337fe700 1 -- 172.21.15.36:0/2382801906 learned_addr learned my addr 172.21.15.36:0/2382801906 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:48:54.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb327fc700 1 --2- 172.21.15.36:0/2382801906 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb340ac340 0x7ffb3414b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:54.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb31ffb700 1 -- 172.21.15.36:0/2382801906 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3207578587 0 0) 0x7ffb340ac870 con 0x7ffb340af7f0 2022-06-17T12:48:54.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb31ffb700 1 -- 172.21.15.36:0/2382801906 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb14003260 con 0x7ffb340af7f0 2022-06-17T12:48:54.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb31ffb700 1 -- 172.21.15.36:0/2382801906 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1744160297 0 0) 0x7ffb340acaa0 con 0x7ffb340ad140 2022-06-17T12:48:54.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb31ffb700 1 -- 172.21.15.36:0/2382801906 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb340ac870 con 0x7ffb340ad140 2022-06-17T12:48:54.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb327fc700 1 -- 172.21.15.36:0/2382801906 >> v1:172.21.15.70:6789/0 conn(0x7ffb340ad140 legacy=0x7ffb3414aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:54.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb327fc700 1 -- 172.21.15.36:0/2382801906 >> v1:172.21.15.36:6789/0 conn(0x7ffb340af7f0 legacy=0x7ffb34147b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:54.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb327fc700 1 -- 172.21.15.36:0/2382801906 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb340b0bf0 con 0x7ffb340ac340 2022-06-17T12:48:54.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb327fc700 1 --2- 172.21.15.36:0/2382801906 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb340ac340 0x7ffb3414b3b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7ffb1c001110 tx=0x7ffb2c0091d0).ready entity=mon.2 client_cookie=873611ebc0cf06cf server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:54.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb31ffb700 1 -- 172.21.15.36:0/2382801906 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb2c00b600 con 0x7ffb340ac340 2022-06-17T12:48:54.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb31ffb700 1 -- 172.21.15.36:0/2382801906 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2c00b760 con 0x7ffb340ac340 2022-06-17T12:48:54.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb31ffb700 1 -- 172.21.15.36:0/2382801906 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb2c00b8d0 con 0x7ffb340ac340 2022-06-17T12:48:54.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb397c5700 1 -- 172.21.15.36:0/2382801906 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb340ac340 msgr2=0x7ffb3414b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:54.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb397c5700 1 --2- 172.21.15.36:0/2382801906 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb340ac340 0x7ffb3414b3b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7ffb1c001110 tx=0x7ffb2c0091d0).stop 2022-06-17T12:48:54.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb397c5700 1 -- 172.21.15.36:0/2382801906 shutdown_connections 2022-06-17T12:48:54.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb397c5700 1 --2- 172.21.15.36:0/2382801906 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb340ac340 0x7ffb3414b3b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:54.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb397c5700 1 -- 172.21.15.36:0/2382801906 >> 172.21.15.36:0/2382801906 conn(0x7ffb340b5430 msgr2=0x7ffb340b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:48:54.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb397c5700 1 -- 172.21.15.36:0/2382801906 shutdown_connections 2022-06-17T12:48:54.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.761+0000 7ffb397c5700 1 -- 172.21.15.36:0/2382801906 wait complete. 2022-06-17T12:48:54.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.765+0000 7ffb397c5700 1 Processor -- start 2022-06-17T12:48:54.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.765+0000 7ffb397c5700 1 -- start start 2022-06-17T12:48:54.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.765+0000 7ffb397c5700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb340ac340 0x7ffb34147610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:54.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.765+0000 7ffb397c5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb340ad140 0x7ffb3415a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:54.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.765+0000 7ffb397c5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb340af7f0 0x7ffb3415aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:54.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.765+0000 7ffb397c5700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb340abec0 con 0x7ffb340ad140 2022-06-17T12:48:54.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.765+0000 7ffb397c5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3414a5d0 con 0x7ffb340af7f0 2022-06-17T12:48:54.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.765+0000 7ffb397c5700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3415a110 con 0x7ffb340ac340 2022-06-17T12:48:54.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.765+0000 7ffb327fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb340ad140 0x7ffb3415a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:54.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.765+0000 7ffb327fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb340ad140 0x7ffb3415a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:32992/0 (socket says 172.21.15.36:32992) 2022-06-17T12:48:54.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.765+0000 7ffb327fc700 1 -- 172.21.15.36:0/1381375977 learned_addr learned my addr 172.21.15.36:0/1381375977 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:48:54.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.765+0000 7ffb32ffd700 1 --2- 172.21.15.36:0/1381375977 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb340ac340 0x7ffb34147610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:54.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.765+0000 7ffb337fe700 1 --2- 172.21.15.36:0/1381375977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb340af7f0 0x7ffb3415aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:54.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.765+0000 7ffb327fc700 1 -- 172.21.15.36:0/1381375977 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb340ac340 msgr2=0x7ffb34147610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:54.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.765+0000 7ffb327fc700 1 --2- 172.21.15.36:0/1381375977 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb340ac340 0x7ffb34147610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:54.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.765+0000 7ffb327fc700 1 -- 172.21.15.36:0/1381375977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb340af7f0 msgr2=0x7ffb3415aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:54.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.765+0000 7ffb327fc700 1 --2- 172.21.15.36:0/1381375977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb340af7f0 0x7ffb3415aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:54.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.765+0000 7ffb327fc700 1 -- 172.21.15.36:0/1381375977 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2c009470 con 0x7ffb340ad140 2022-06-17T12:48:54.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.765+0000 7ffb327fc700 1 --2- 172.21.15.36:0/1381375977 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb340ad140 0x7ffb3415a540 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7ffb1c001110 tx=0x7ffb2c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:54.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.765+0000 7ffb2bfff700 1 -- 172.21.15.36:0/1381375977 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb2c00b8d0 con 0x7ffb340ad140 2022-06-17T12:48:54.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.765+0000 7ffb2bfff700 1 -- 172.21.15.36:0/1381375977 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2c021670 con 0x7ffb340ad140 2022-06-17T12:48:54.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.765+0000 7ffb2bfff700 1 -- 172.21.15.36:0/1381375977 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb2c00b600 con 0x7ffb340ad140 2022-06-17T12:48:54.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.765+0000 7ffb397c5700 1 -- 172.21.15.36:0/1381375977 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3414a9b0 con 0x7ffb340ad140 2022-06-17T12:48:54.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.765+0000 7ffb397c5700 1 -- 172.21.15.36:0/1381375977 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3415af20 con 0x7ffb340ad140 2022-06-17T12:48:54.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.769+0000 7ffb397c5700 1 -- 172.21.15.36:0/1381375977 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb340b1500 con 0x7ffb340ad140 2022-06-17T12:48:54.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.769+0000 7ffb2bfff700 1 -- 172.21.15.36:0/1381375977 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ffb2c00f070 con 0x7ffb340ad140 2022-06-17T12:48:54.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.769+0000 7ffb2bfff700 1 --2- 172.21.15.36:0/1381375977 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffb18053ad0 0x7ffb18055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:54.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.769+0000 7ffb2bfff700 1 -- 172.21.15.36:0/1381375977 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ffb2c06d4e0 con 0x7ffb340ad140 2022-06-17T12:48:54.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.769+0000 7ffb32ffd700 1 --2- 172.21.15.36:0/1381375977 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffb18053ad0 0x7ffb18055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:54.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.769+0000 7ffb32ffd700 1 --2- 172.21.15.36:0/1381375977 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffb18053ad0 0x7ffb18055f90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7ffb1c00d000 tx=0x7ffb1c0079e0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:54.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.777+0000 7ffb2bfff700 1 -- 172.21.15.36:0/1381375977 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffb2c044280 con 0x7ffb340ad140 2022-06-17T12:48:54.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.901+0000 7ffb397c5700 1 -- 172.21.15.36:0/1381375977 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7ffb34057410 con 0x7ffb340ad140 2022-06-17T12:48:54.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.905+0000 7ffb2bfff700 1 -- 172.21.15.36:0/1381375977 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v35) v1 ==== 81+0+34037 (secure 0 0 0) 0x7ffb2c015020 con 0x7ffb340ad140 2022-06-17T12:48:54.915 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:48:54.916 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:48:54.916 INFO:teuthology.orchestra.run.smithi036.stdout: "epoch": 35, 2022-06-17T12:48:54.917 INFO:teuthology.orchestra.run.smithi036.stdout: "fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", 2022-06-17T12:48:54.917 INFO:teuthology.orchestra.run.smithi036.stdout: "created": "2022-06-17T12:46:30.671408+0000", 2022-06-17T12:48:54.917 INFO:teuthology.orchestra.run.smithi036.stdout: "modified": "2022-06-17T12:47:48.075718+0000", 2022-06-17T12:48:54.917 INFO:teuthology.orchestra.run.smithi036.stdout: "last_up_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:48:54.917 INFO:teuthology.orchestra.run.smithi036.stdout: "last_in_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:48:54.918 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-17T12:48:54.918 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_num": 5799936, 2022-06-17T12:48:54.918 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_set": [ 2022-06-17T12:48:54.925 INFO:teuthology.orchestra.run.smithi036.stdout: "pglog_hardlimit", 2022-06-17T12:48:54.925 INFO:teuthology.orchestra.run.smithi036.stdout: "purged_snapdirs", 2022-06-17T12:48:54.926 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_deletes", 2022-06-17T12:48:54.926 INFO:teuthology.orchestra.run.smithi036.stdout: "sortbitwise" 2022-06-17T12:48:54.927 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:48:54.927 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_version": 6, 2022-06-17T12:48:54.927 INFO:teuthology.orchestra.run.smithi036.stdout: "full_ratio": 0.94999998807907104, 2022-06-17T12:48:54.927 INFO:teuthology.orchestra.run.smithi036.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-17T12:48:54.928 INFO:teuthology.orchestra.run.smithi036.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-17T12:48:54.928 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_snapshot": "", 2022-06-17T12:48:54.928 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_max": 6, 2022-06-17T12:48:54.928 INFO:teuthology.orchestra.run.smithi036.stdout: "max_osd": 8, 2022-06-17T12:48:54.929 INFO:teuthology.orchestra.run.smithi036.stdout: "require_min_compat_client": "jewel", 2022-06-17T12:48:54.929 INFO:teuthology.orchestra.run.smithi036.stdout: "min_compat_client": "jewel", 2022-06-17T12:48:54.929 INFO:teuthology.orchestra.run.smithi036.stdout: "require_osd_release": "octopus", 2022-06-17T12:48:54.929 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:48:54.929 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:54.930 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 1, 2022-06-17T12:48:54.930 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "device_health_metrics", 2022-06-17T12:48:54.954 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:33.881212+0000", 2022-06-17T12:48:54.955 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:48:54.955 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:48:54.955 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:48:54.955 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:48:54.956 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:48:54.956 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:48:54.956 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:48:54.956 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:48:54.956 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 1, 2022-06-17T12:48:54.990 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 1, 2022-06-17T12:48:54.990 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 1, 2022-06-17T12:48:54.991 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 1, 2022-06-17T12:48:54.991 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 1, 2022-06-17T12:48:54.992 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:48:54.992 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:48:55.018 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:48:55.018 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:48:55.018 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:48:55.019 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:48:55.019 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:48:55.019 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.019 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "16", 2022-06-17T12:48:55.020 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:48:55.020 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:48:55.020 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:48:55.020 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:48:55.021 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:48:55.021 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:48:55.021 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:48:55.021 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:48:55.021 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:48:55.039 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:48:55.040 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:48:55.040 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:48:55.040 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:48:55.040 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:48:55.040 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:48:55.041 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:48:55.061 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:48:55.061 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:48:55.064 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:48:55.064 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:48:55.065 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:48:55.065 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:48:55.074 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:48:55.075 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:48:55.075 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:48:55.076 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:48:55.076 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.077 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:48:55.077 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:48:55.081 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:48:55.081 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:48:55.082 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:48:55.082 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:48:55.083 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:48:55.083 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:48:55.084 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:48:55.084 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:48:55.092 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:48:55.092 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:48:55.092 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 1 2022-06-17T12:48:55.093 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.093 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:48:55.093 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr_devicehealth": {} 2022-06-17T12:48:55.094 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.094 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.095 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.129 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 2, 2022-06-17T12:48:55.130 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "rbd", 2022-06-17T12:48:55.130 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:44.485460+0000", 2022-06-17T12:48:55.131 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:48:55.131 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:48:55.132 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:48:55.132 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:48:55.133 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:48:55.133 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:48:55.134 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:48:55.147 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:48:55.148 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:48:55.148 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:48:55.160 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:48:55.160 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:48:55.161 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:48:55.161 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:48:55.162 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:48:55.162 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:48:55.162 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:48:55.170 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:48:55.170 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:48:55.171 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:48:55.171 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.171 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "19", 2022-06-17T12:48:55.171 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:48:55.172 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:48:55.172 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:48:55.172 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:48:55.172 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:48:55.181 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:48:55.182 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:48:55.183 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:48:55.183 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:48:55.184 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:48:55.184 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:48:55.185 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:48:55.185 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:48:55.186 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:48:55.186 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:48:55.186 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:48:55.187 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:48:55.187 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:48:55.187 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:48:55.188 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:48:55.188 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:48:55.190 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:48:55.193 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:48:55.219 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:48:55.219 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:48:55.219 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:48:55.220 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.220 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:48:55.220 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:48:55.220 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:48:55.221 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:48:55.221 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:48:55.221 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:48:55.221 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:48:55.221 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:48:55.222 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:48:55.222 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:48:55.222 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:48:55.222 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:48:55.223 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:48:55.225 INFO:teuthology.orchestra.run.smithi036.stdout: "rbd": {} 2022-06-17T12:48:55.226 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.226 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.226 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.226 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 5, 2022-06-17T12:48:55.227 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_metadata", 2022-06-17T12:48:55.227 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:47:45.085233+0000", 2022-06-17T12:48:55.227 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:48:55.227 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:48:55.227 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:48:55.228 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:48:55.228 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:48:55.228 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:48:55.228 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:48:55.228 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:48:55.229 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:48:55.229 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:48:55.229 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:48:55.229 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:48:55.230 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:48:55.230 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:48:55.230 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:48:55.230 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:48:55.230 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:48:55.231 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:48:55.231 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:48:55.231 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:48:55.231 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.232 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "34", 2022-06-17T12:48:55.232 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:48:55.232 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:48:55.232 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:48:55.232 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:48:55.233 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:48:55.233 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:48:55.233 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:48:55.233 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:48:55.233 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:48:55.234 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:48:55.234 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:48:55.234 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:48:55.234 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:48:55.235 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:48:55.236 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:48:55.236 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:48:55.236 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:48:55.236 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:48:55.237 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:48:55.237 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:48:55.237 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:48:55.237 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:48:55.237 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:48:55.238 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:48:55.238 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:48:55.238 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:48:55.238 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.239 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:48:55.239 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:48:55.239 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:48:55.239 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:48:55.240 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:48:55.240 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:48:55.240 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:48:55.240 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:48:55.240 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:48:55.241 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:48:55.241 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:48:55.241 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:48:55.241 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_bias": 4, 2022-06-17T12:48:55.241 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 16, 2022-06-17T12:48:55.242 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_priority": 5 2022-06-17T12:48:55.242 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.242 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:48:55.243 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:48:55.243 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": "cephfs" 2022-06-17T12:48:55.243 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.243 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.244 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.244 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.244 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 6, 2022-06-17T12:48:55.244 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_data", 2022-06-17T12:48:55.245 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:47:46.070152+0000", 2022-06-17T12:48:55.245 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:48:55.245 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:48:55.245 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:48:55.245 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:48:55.246 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:48:55.247 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:48:55.247 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:48:55.247 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:48:55.247 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:48:55.247 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:48:55.248 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:48:55.248 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:48:55.248 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:48:55.248 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:48:55.249 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:48:55.249 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:48:55.249 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:48:55.249 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:48:55.249 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:48:55.267 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:48:55.268 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.268 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "35", 2022-06-17T12:48:55.269 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:48:55.269 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:48:55.270 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:48:55.271 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:48:55.271 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:48:55.272 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:48:55.273 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:48:55.273 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:48:55.273 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:48:55.274 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:48:55.275 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:48:55.276 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:48:55.276 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:48:55.276 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:48:55.277 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:48:55.277 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:48:55.278 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:48:55.278 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:48:55.278 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:48:55.279 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:48:55.279 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:48:55.280 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:48:55.280 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:48:55.280 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:48:55.281 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:48:55.283 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:48:55.283 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.284 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:48:55.284 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:48:55.285 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:48:55.285 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:48:55.286 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:48:55.286 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:48:55.287 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:48:55.287 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:48:55.288 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:48:55.289 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:48:55.289 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:48:55.290 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:48:55.290 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:48:55.290 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:48:55.291 INFO:teuthology.orchestra.run.smithi036.stdout: "data": "cephfs" 2022-06-17T12:48:55.291 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.291 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.292 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.293 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:48:55.293 INFO:teuthology.orchestra.run.smithi036.stdout: "osds": [ 2022-06-17T12:48:55.294 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.294 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:48:55.294 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", 2022-06-17T12:48:55.295 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:48:55.295 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:48:55.296 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:48:55.296 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:48:55.298 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:48:55.298 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:48:55.299 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:48:55.299 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 33, 2022-06-17T12:48:55.300 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:48:55.300 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:48:55.301 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:48:55.301 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.313 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.314 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.314 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6802", 2022-06-17T12:48:55.315 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:48:55.315 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.315 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.316 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.316 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6803", 2022-06-17T12:48:55.317 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:48:55.342 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.354 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.354 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.355 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:48:55.355 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.356 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.365 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.366 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6804", 2022-06-17T12:48:55.366 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:48:55.367 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.367 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.368 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.368 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6805", 2022-06-17T12:48:55.369 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:48:55.369 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.369 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.370 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.370 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:48:55.371 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.371 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.371 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.372 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6808", 2022-06-17T12:48:55.372 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:48:55.373 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.373 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.373 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.374 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6809", 2022-06-17T12:48:55.374 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:48:55.375 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.375 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.377 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.377 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:48:55.378 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.378 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.379 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.379 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6806", 2022-06-17T12:48:55.380 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:48:55.389 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.389 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.390 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.390 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6807", 2022-06-17T12:48:55.390 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:48:55.391 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.391 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.392 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.392 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6803/22619", 2022-06-17T12:48:55.392 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6805/22619", 2022-06-17T12:48:55.393 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6809/22619", 2022-06-17T12:48:55.393 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6807/22619", 2022-06-17T12:48:55.393 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:48:55.420 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:48:55.420 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:48:55.421 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.421 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.422 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.422 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:48:55.423 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", 2022-06-17T12:48:55.423 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:48:55.423 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:48:55.424 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:48:55.424 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:48:55.425 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:48:55.425 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:48:55.429 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:48:55.429 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 33, 2022-06-17T12:48:55.444 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:48:55.461 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:48:55.465 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:48:55.465 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.466 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.466 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.474 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6810", 2022-06-17T12:48:55.474 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:48:55.475 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.475 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.475 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.476 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6811", 2022-06-17T12:48:55.476 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:48:55.477 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.477 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.477 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.487 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:48:55.488 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.488 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.488 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.489 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6812", 2022-06-17T12:48:55.489 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:48:55.490 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.490 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.491 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.491 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6813", 2022-06-17T12:48:55.492 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:48:55.492 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.492 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.493 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.494 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:48:55.494 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.495 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.495 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.506 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6816", 2022-06-17T12:48:55.517 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:48:55.517 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.518 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.518 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.519 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6817", 2022-06-17T12:48:55.519 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:48:55.519 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.520 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.526 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.526 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:48:55.527 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.527 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.530 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.531 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6814", 2022-06-17T12:48:55.531 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:48:55.531 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.532 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.532 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.533 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6815", 2022-06-17T12:48:55.533 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:48:55.554 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.554 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.555 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.555 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6811/22620", 2022-06-17T12:48:55.555 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6813/22620", 2022-06-17T12:48:55.557 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6817/22620", 2022-06-17T12:48:55.557 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6815/22620", 2022-06-17T12:48:55.558 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:48:55.558 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:48:55.559 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:48:55.559 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.560 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.560 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.561 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:48:55.562 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", 2022-06-17T12:48:55.562 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:48:55.562 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:48:55.563 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:48:55.563 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:48:55.564 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:48:55.565 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:48:55.565 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:48:55.565 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 32, 2022-06-17T12:48:55.566 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:48:55.566 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:48:55.567 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:48:55.567 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.568 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.568 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.568 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6826", 2022-06-17T12:48:55.569 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:48:55.569 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.570 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.571 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.573 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6827", 2022-06-17T12:48:55.574 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:48:55.574 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.575 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.575 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.575 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:48:55.576 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.577 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.577 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.577 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6828", 2022-06-17T12:48:55.578 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:48:55.579 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.580 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.580 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.580 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6829", 2022-06-17T12:48:55.581 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:48:55.581 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.591 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.592 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.592 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:48:55.593 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.593 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.596 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.597 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6832", 2022-06-17T12:48:55.597 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:48:55.597 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.598 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.598 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.599 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6833", 2022-06-17T12:48:55.600 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:48:55.600 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.601 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.602 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.602 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:48:55.603 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.603 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.604 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.604 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6830", 2022-06-17T12:48:55.605 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:48:55.605 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.605 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.606 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.606 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6831", 2022-06-17T12:48:55.607 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:48:55.607 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.608 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.608 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.609 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6827/22643", 2022-06-17T12:48:55.609 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6829/22643", 2022-06-17T12:48:55.610 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6833/22643", 2022-06-17T12:48:55.610 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6831/22643", 2022-06-17T12:48:55.611 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:48:55.611 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:48:55.612 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:48:55.613 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.613 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.614 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.614 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:48:55.615 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", 2022-06-17T12:48:55.615 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:48:55.615 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:48:55.616 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:48:55.616 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:48:55.617 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:48:55.617 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:48:55.618 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:48:55.618 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 33, 2022-06-17T12:48:55.619 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:48:55.619 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:48:55.620 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:48:55.620 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.621 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.621 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.621 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6818", 2022-06-17T12:48:55.622 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:48:55.622 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.623 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.624 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.624 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6819", 2022-06-17T12:48:55.627 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:48:55.628 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.628 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.628 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.629 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:48:55.629 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.666 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.667 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.667 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6820", 2022-06-17T12:48:55.676 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:48:55.677 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.677 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.687 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.688 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6821", 2022-06-17T12:48:55.688 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:48:55.689 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.689 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.689 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.690 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:48:55.691 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.691 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.692 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.692 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6824", 2022-06-17T12:48:55.693 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:48:55.694 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.694 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.694 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.695 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6825", 2022-06-17T12:48:55.695 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:48:55.696 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.696 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.697 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.697 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:48:55.698 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.698 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.699 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.699 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6822", 2022-06-17T12:48:55.699 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:48:55.709 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.719 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.719 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.720 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6823", 2022-06-17T12:48:55.720 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:48:55.720 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.721 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.721 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.722 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6819/22646", 2022-06-17T12:48:55.723 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6821/22646", 2022-06-17T12:48:55.724 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6825/22646", 2022-06-17T12:48:55.724 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6823/22646", 2022-06-17T12:48:55.725 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:48:55.725 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:48:55.738 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:48:55.739 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.739 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.740 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.740 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:48:55.741 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", 2022-06-17T12:48:55.741 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:48:55.741 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:48:55.755 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:48:55.763 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:48:55.764 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:48:55.764 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:48:55.765 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:48:55.765 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 33, 2022-06-17T12:48:55.765 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:48:55.766 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:48:55.767 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:48:55.773 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.787 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.788 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.788 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6808", 2022-06-17T12:48:55.789 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:48:55.789 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.789 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.798 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.799 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6809", 2022-06-17T12:48:55.799 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:48:55.800 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.811 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.812 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.812 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:48:55.812 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.813 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.813 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.817 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6810", 2022-06-17T12:48:55.817 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:48:55.818 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.818 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.819 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.819 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6811", 2022-06-17T12:48:55.819 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:48:55.820 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.820 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.823 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.824 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:48:55.824 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.825 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.825 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.826 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6814", 2022-06-17T12:48:55.826 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:48:55.827 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.827 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.827 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.828 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6815", 2022-06-17T12:48:55.828 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:48:55.829 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.829 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.829 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.830 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:48:55.831 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.831 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.831 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.832 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6812", 2022-06-17T12:48:55.832 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:48:55.833 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.833 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.834 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.835 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6813", 2022-06-17T12:48:55.835 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:48:55.835 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.836 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.836 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.836 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6809/22812", 2022-06-17T12:48:55.837 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6811/22812", 2022-06-17T12:48:55.837 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6815/22812", 2022-06-17T12:48:55.838 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6813/22812", 2022-06-17T12:48:55.838 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:48:55.839 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:48:55.839 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:48:55.839 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.840 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.840 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.841 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:48:55.841 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", 2022-06-17T12:48:55.842 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:48:55.842 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:48:55.843 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:48:55.843 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:48:55.843 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:48:55.844 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:48:55.853 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:48:55.853 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 33, 2022-06-17T12:48:55.854 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:48:55.855 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:48:55.858 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:48:55.859 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.859 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.860 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.860 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6800", 2022-06-17T12:48:55.860 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:48:55.861 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.861 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.876 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.877 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6801", 2022-06-17T12:48:55.877 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:48:55.882 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.883 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.884 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.884 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:48:55.884 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.885 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.885 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.885 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6802", 2022-06-17T12:48:55.886 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:48:55.887 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.887 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.888 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.888 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6803", 2022-06-17T12:48:55.888 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:48:55.889 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.889 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.889 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.890 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:48:55.890 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.891 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.891 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.891 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6806", 2022-06-17T12:48:55.892 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:48:55.892 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.893 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.893 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.918 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6807", 2022-06-17T12:48:55.919 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:48:55.919 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.919 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.920 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.920 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:48:55.921 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.921 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.922 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.922 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6804", 2022-06-17T12:48:55.922 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:48:55.923 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.923 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.924 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.924 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6805", 2022-06-17T12:48:55.925 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:48:55.925 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.925 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.926 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.926 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6801/22813", 2022-06-17T12:48:55.927 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6803/22813", 2022-06-17T12:48:55.927 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6807/22813", 2022-06-17T12:48:55.927 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6805/22813", 2022-06-17T12:48:55.928 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:48:55.928 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:48:55.929 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:48:55.929 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.929 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.930 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.930 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:48:55.931 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", 2022-06-17T12:48:55.931 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:48:55.931 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:48:55.932 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:48:55.932 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:48:55.933 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:48:55.933 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:48:55.958 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:48:55.959 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 32, 2022-06-17T12:48:55.959 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:48:55.960 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:48:55.960 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:48:55.960 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:55.961 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.961 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:55.962 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6816", 2022-06-17T12:48:55.962 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:48:55.963 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:55.963 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:55.995 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:55.996 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6817", 2022-06-17T12:48:55.996 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:48:55.996 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:55.997 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:55.997 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.007 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:48:56.007 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:56.008 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:56.008 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:56.009 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6818", 2022-06-17T12:48:56.009 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:48:56.009 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.010 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:56.010 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:56.011 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6819", 2022-06-17T12:48:56.051 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:48:56.051 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:56.052 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:56.052 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.053 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:48:56.053 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:56.070 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:56.071 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:56.071 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6822", 2022-06-17T12:48:56.072 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:48:56.072 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.072 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:56.073 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:56.073 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6823", 2022-06-17T12:48:56.073 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:48:56.256 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:56.276 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:56.276 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.277 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:48:56.277 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:56.290 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:56.290 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:56.291 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6820", 2022-06-17T12:48:56.291 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:48:56.292 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.313 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:56.313 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:56.313 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6821", 2022-06-17T12:48:56.314 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:48:56.329 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:56.330 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:56.332 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.333 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6817/22822", 2022-06-17T12:48:56.333 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6819/22822", 2022-06-17T12:48:56.338 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6823/22822", 2022-06-17T12:48:56.338 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6821/22822", 2022-06-17T12:48:56.339 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:48:56.339 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:48:56.339 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:48:56.340 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:56.340 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.341 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:56.341 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:48:56.342 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", 2022-06-17T12:48:56.387 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:48:56.388 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:48:56.388 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:48:56.389 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:48:56.389 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:48:56.390 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:48:56.390 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:48:56.391 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 33, 2022-06-17T12:48:56.391 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:48:56.391 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:48:56.392 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:48:56.392 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:56.393 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:56.393 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:56.393 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6824", 2022-06-17T12:48:56.429 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:48:56.430 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.438 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:56.438 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:56.438 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6825", 2022-06-17T12:48:56.439 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:48:56.439 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:56.440 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:56.452 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.452 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:48:56.453 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:56.453 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:56.454 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:56.454 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6826", 2022-06-17T12:48:56.478 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:48:56.479 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.503 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:56.505 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:56.505 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6827", 2022-06-17T12:48:56.518 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:48:56.542 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:56.543 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:56.543 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.544 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:48:56.544 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:56.545 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:56.551 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:56.551 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6830", 2022-06-17T12:48:56.565 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:48:56.569 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.570 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:56.570 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:56.571 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6831", 2022-06-17T12:48:56.571 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:48:56.572 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:56.572 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:56.572 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.573 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:48:56.573 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:48:56.574 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:56.574 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:48:56.575 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6828", 2022-06-17T12:48:56.575 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:48:56.576 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.576 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:56.576 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:48:56.577 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6829", 2022-06-17T12:48:56.577 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:48:56.577 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:56.642 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:56.642 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.643 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6825/22837", 2022-06-17T12:48:56.643 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6827/22837", 2022-06-17T12:48:56.643 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6831/22837", 2022-06-17T12:48:56.644 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6829/22837", 2022-06-17T12:48:56.644 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:48:56.644 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:48:56.645 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:48:56.645 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:48:56.645 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:56.660 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:48:56.661 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_xinfo": [ 2022-06-17T12:48:56.661 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:56.662 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:48:56.662 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:48:56.671 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:48:56.677 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:48:56.758 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:48:56.759 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:48:56.776 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.291461+0000", 2022-06-17T12:48:56.777 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:48:56.777 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.778 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:56.778 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:48:56.779 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:48:56.779 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:48:56.780 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:48:56.780 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:48:56.781 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:48:56.781 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.537912+0000", 2022-06-17T12:48:56.781 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:48:56.816 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.865 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:56.866 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:48:56.866 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:48:56.867 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:48:56.868 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:48:56.869 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:48:56.869 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:48:56.869 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.606947+0000", 2022-06-17T12:48:56.870 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:48:56.870 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.871 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:56.871 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:48:56.872 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:48:56.872 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:48:56.873 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:48:56.873 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:48:56.896 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:48:56.899 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.514412+0000", 2022-06-17T12:48:56.899 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:48:56.900 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.900 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:56.901 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:48:56.901 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:48:56.902 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:48:56.902 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:48:56.903 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:48:56.903 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:48:56.904 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.394645+0000", 2022-06-17T12:48:56.904 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:48:56.905 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.905 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:56.906 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:48:56.906 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:48:56.907 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:48:56.909 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:48:56.909 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:48:56.910 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:48:56.910 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.343806+0000", 2022-06-17T12:48:56.911 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:48:56.911 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.912 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:56.912 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:48:56.912 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:48:56.913 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:48:56.914 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:48:56.914 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:48:56.915 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:48:56.915 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.591864+0000", 2022-06-17T12:48:56.915 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:48:56.916 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.916 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:48:56.916 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:48:56.917 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:48:56.917 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:48:56.917 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:48:56.918 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:48:56.928 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:48:56.929 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.607045+0000", 2022-06-17T12:48:56.933 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:48:56.933 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:56.934 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:48:56.934 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap": [], 2022-06-17T12:48:56.934 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap_items": [], 2022-06-17T12:48:56.936 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_temp": [], 2022-06-17T12:48:56.938 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_temp": [], 2022-06-17T12:48:56.938 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklist": {}, 2022-06-17T12:48:56.939 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profiles": { 2022-06-17T12:48:56.939 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs_data": { 2022-06-17T12:48:56.942 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-device-class": "", 2022-06-17T12:48:56.943 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:48:56.943 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-root": "default", 2022-06-17T12:48:56.944 INFO:teuthology.orchestra.run.smithi036.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-17T12:48:56.944 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:48:56.945 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "2", 2022-06-17T12:48:56.945 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:48:56.946 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:48:56.946 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van", 2022-06-17T12:48:56.947 INFO:teuthology.orchestra.run.smithi036.stdout: "w": "8" 2022-06-17T12:48:56.947 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.950 INFO:teuthology.orchestra.run.smithi036.stdout: "default": { 2022-06-17T12:48:56.951 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:48:56.951 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:48:56.952 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "1", 2022-06-17T12:48:56.953 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:48:56.953 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:48:56.955 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van" 2022-06-17T12:48:56.955 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:48:56.956 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:48:56.956 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps_queue": [], 2022-06-17T12:48:56.956 INFO:teuthology.orchestra.run.smithi036.stdout: "new_removed_snaps": [], 2022-06-17T12:48:56.957 INFO:teuthology.orchestra.run.smithi036.stdout: "new_purged_snaps": [], 2022-06-17T12:48:56.957 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_node_flags": {}, 2022-06-17T12:48:56.957 INFO:teuthology.orchestra.run.smithi036.stdout: "device_class_flags": {} 2022-06-17T12:48:56.969 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:48:56.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.905+0000 7ffb397c5700 1 -- 172.21.15.36:0/1381375977 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffb18053ad0 msgr2=0x7ffb18055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:56.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.905+0000 7ffb397c5700 1 --2- 172.21.15.36:0/1381375977 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffb18053ad0 0x7ffb18055f90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7ffb1c00d000 tx=0x7ffb1c0079e0).stop 2022-06-17T12:48:56.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.905+0000 7ffb397c5700 1 -- 172.21.15.36:0/1381375977 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb340ad140 msgr2=0x7ffb3415a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:56.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.905+0000 7ffb397c5700 1 --2- 172.21.15.36:0/1381375977 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb340ad140 0x7ffb3415a540 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7ffb1c001110 tx=0x7ffb2c016000).stop 2022-06-17T12:48:56.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.905+0000 7ffb397c5700 1 -- 172.21.15.36:0/1381375977 shutdown_connections 2022-06-17T12:48:56.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.905+0000 7ffb397c5700 1 --2- 172.21.15.36:0/1381375977 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffb18053ad0 0x7ffb18055f90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:56.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.905+0000 7ffb397c5700 1 --2- 172.21.15.36:0/1381375977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb340af7f0 0x7ffb3415aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:56.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.905+0000 7ffb397c5700 1 --2- 172.21.15.36:0/1381375977 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb340ad140 0x7ffb3415a540 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:56.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.905+0000 7ffb397c5700 1 --2- 172.21.15.36:0/1381375977 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb340ac340 0x7ffb34147610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:56.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.905+0000 7ffb397c5700 1 -- 172.21.15.36:0/1381375977 >> 172.21.15.36:0/1381375977 conn(0x7ffb340b5430 msgr2=0x7ffb340b3260 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:48:56.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.905+0000 7ffb397c5700 1 -- 172.21.15.36:0/1381375977 shutdown_connections 2022-06-17T12:48:56.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:54.905+0000 7ffb397c5700 1 -- 172.21.15.36:0/1381375977 wait complete. 2022-06-17T12:48:56.998 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:48:57.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.129+0000 7fdfcb59e700 1 Processor -- start 2022-06-17T12:48:57.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.129+0000 7fdfcb59e700 1 -- start start 2022-06-17T12:48:57.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.129+0000 7fdfcb59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfcc0b80c0 0x7fdfcc140c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:57.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.129+0000 7fdfcb59e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fdfcc142d70 0x7fdfcc1431e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:57.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.129+0000 7fdfcb59e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfcc142160 con 0x7fdfcc0b7ce0 2022-06-17T12:48:57.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.129+0000 7fdfcb59e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfcc0b4f30 con 0x7fdfcc142d70 2022-06-17T12:48:57.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.129+0000 7fdfcb59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc142390 con 0x7fdfcc0b80c0 2022-06-17T12:48:57.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.129+0000 7fdfca59c700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fdfcc0b7ce0 0x7fdfcc13aa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50528/0 (socket says 172.21.15.36:50528) 2022-06-17T12:48:57.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.129+0000 7fdfca59c700 1 -- 172.21.15.36:0/3441758009 learned_addr learned my addr 172.21.15.36:0/3441758009 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:48:57.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.129+0000 7fdfcad9d700 1 --2- 172.21.15.36:0/3441758009 >> v2:172.21.15.36:3300/0 conn(0x7fdfcc142d70 0x7fdfcc1431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:57.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.129+0000 7fdfc9d9b700 1 --2- 172.21.15.36:0/3441758009 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfcc0b80c0 0x7fdfcc140c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:57.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.129+0000 7fdfc959a700 1 -- 172.21.15.36:0/3441758009 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335157726 0 0) 0x7fdfcc142160 con 0x7fdfcc0b7ce0 2022-06-17T12:48:57.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.129+0000 7fdfc959a700 1 -- 172.21.15.36:0/3441758009 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfb0003260 con 0x7fdfcc0b7ce0 2022-06-17T12:48:57.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.129+0000 7fdfcad9d700 1 -- 172.21.15.36:0/3441758009 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfcc0b80c0 msgr2=0x7fdfcc140c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:48:57.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.129+0000 7fdfcad9d700 1 --2- 172.21.15.36:0/3441758009 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfcc0b80c0 0x7fdfcc140c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:57.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.129+0000 7fdfc959a700 1 -- 172.21.15.36:0/3441758009 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4231226826 0 0) 0x7fdfb0003260 con 0x7fdfcc0b7ce0 2022-06-17T12:48:57.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.129+0000 7fdfcad9d700 1 -- 172.21.15.36:0/3441758009 >> v1:172.21.15.36:6789/0 conn(0x7fdfcc0b7ce0 legacy=0x7fdfcc13aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:57.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.129+0000 7fdfcad9d700 1 -- 172.21.15.36:0/3441758009 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfcc142630 con 0x7fdfcc142d70 2022-06-17T12:48:57.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.129+0000 7fdfcad9d700 1 --2- 172.21.15.36:0/3441758009 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfcc142d70 0x7fdfcc1431e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fdfbc001800 tx=0x7fdfc40091d0).ready entity=mon.0 client_cookie=310c819813eef38c server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:57.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.129+0000 7fdfc959a700 1 -- 172.21.15.36:0/3441758009 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdfbc001660 con 0x7fdfcc142d70 2022-06-17T12:48:57.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.129+0000 7fdfc959a700 1 -- 172.21.15.36:0/3441758009 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc400f510 con 0x7fdfcc142d70 2022-06-17T12:48:57.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.129+0000 7fdfc959a700 1 -- 172.21.15.36:0/3441758009 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdfc400f680 con 0x7fdfcc142d70 2022-06-17T12:48:57.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.129+0000 7fdfcb59e700 1 -- 172.21.15.36:0/3441758009 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfcc142d70 msgr2=0x7fdfcc1431e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:57.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.129+0000 7fdfcb59e700 1 --2- 172.21.15.36:0/3441758009 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfcc142d70 0x7fdfcc1431e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fdfbc001800 tx=0x7fdfc40091d0).stop 2022-06-17T12:48:57.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfcb59e700 1 -- 172.21.15.36:0/3441758009 shutdown_connections 2022-06-17T12:48:57.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfcb59e700 1 --2- 172.21.15.36:0/3441758009 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfcc142d70 0x7fdfcc1431e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:57.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfcb59e700 1 --2- 172.21.15.36:0/3441758009 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfcc0b80c0 0x7fdfcc140c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:57.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfcb59e700 1 -- 172.21.15.36:0/3441758009 >> 172.21.15.36:0/3441758009 conn(0x7fdfcc05c550 msgr2=0x7fdfcc0b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:48:57.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfcb59e700 1 -- 172.21.15.36:0/3441758009 shutdown_connections 2022-06-17T12:48:57.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfcb59e700 1 -- 172.21.15.36:0/3441758009 wait complete. 2022-06-17T12:48:57.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfcb59e700 1 Processor -- start 2022-06-17T12:48:57.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfcb59e700 1 -- start start 2022-06-17T12:48:57.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfcb59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfcc0b7ce0 0x7fdfcc0619c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:57.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfcb59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfcc0b80c0 0x7fdfcc144670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:57.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfcb59e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfcc142d70 0x7fdfcc062380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:57.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfcb59e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc064e00 con 0x7fdfcc0b7ce0 2022-06-17T12:48:57.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfcb59e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc156410 con 0x7fdfcc142d70 2022-06-17T12:48:57.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfcb59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc155ed0 con 0x7fdfcc0b80c0 2022-06-17T12:48:57.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfca59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfcc0b7ce0 0x7fdfcc0619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:57.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfca59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfcc0b7ce0 0x7fdfcc0619c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33004/0 (socket says 172.21.15.36:33004) 2022-06-17T12:48:57.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfca59c700 1 -- 172.21.15.36:0/2663990418 learned_addr learned my addr 172.21.15.36:0/2663990418 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:48:57.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfcad9d700 1 --2- 172.21.15.36:0/2663990418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfcc142d70 0x7fdfcc062380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:57.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfc9d9b700 1 --2- 172.21.15.36:0/2663990418 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfcc0b80c0 0x7fdfcc144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:57.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfca59c700 1 -- 172.21.15.36:0/2663990418 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfcc0b80c0 msgr2=0x7fdfcc144670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:57.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfca59c700 1 --2- 172.21.15.36:0/2663990418 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfcc0b80c0 0x7fdfcc144670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:57.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfca59c700 1 -- 172.21.15.36:0/2663990418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfcc142d70 msgr2=0x7fdfcc062380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:57.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfca59c700 1 --2- 172.21.15.36:0/2663990418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfcc142d70 0x7fdfcc062380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:57.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfca59c700 1 -- 172.21.15.36:0/2663990418 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc4009470 con 0x7fdfcc0b7ce0 2022-06-17T12:48:57.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfcad9d700 1 --2- 172.21.15.36:0/2663990418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfcc142d70 0x7fdfcc062380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:48:57.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfc9d9b700 1 --2- 172.21.15.36:0/2663990418 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfcc0b80c0 0x7fdfcc144670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:48:57.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfca59c700 1 --2- 172.21.15.36:0/2663990418 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfcc0b7ce0 0x7fdfcc0619c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fdfbc0018c0 tx=0x7fdfbc001b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:57.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfbb7fe700 1 -- 172.21.15.36:0/2663990418 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdfbc007170 con 0x7fdfcc0b7ce0 2022-06-17T12:48:57.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfbb7fe700 1 -- 172.21.15.36:0/2663990418 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfbc0072d0 con 0x7fdfcc0b7ce0 2022-06-17T12:48:57.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfcb59e700 1 -- 172.21.15.36:0/2663990418 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfcc156170 con 0x7fdfcc0b7ce0 2022-06-17T12:48:57.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfbb7fe700 1 -- 172.21.15.36:0/2663990418 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdfbc004920 con 0x7fdfcc0b7ce0 2022-06-17T12:48:57.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.133+0000 7fdfcb59e700 1 -- 172.21.15.36:0/2663990418 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfcc067250 con 0x7fdfcc0b7ce0 2022-06-17T12:48:57.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.137+0000 7fdfcb59e700 1 -- 172.21.15.36:0/2663990418 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfcc1428d0 con 0x7fdfcc0b7ce0 2022-06-17T12:48:57.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.137+0000 7fdfbb7fe700 1 -- 172.21.15.36:0/2663990418 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fdfbc004a80 con 0x7fdfcc0b7ce0 2022-06-17T12:48:57.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.137+0000 7fdfbb7fe700 1 --2- 172.21.15.36:0/2663990418 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdfa8053a40 0x7fdfa8055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:57.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.137+0000 7fdfbb7fe700 1 -- 172.21.15.36:0/2663990418 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdfbc06bef0 con 0x7fdfcc0b7ce0 2022-06-17T12:48:57.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.137+0000 7fdfc9d9b700 1 --2- 172.21.15.36:0/2663990418 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdfa8053a40 0x7fdfa8055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:57.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.137+0000 7fdfc9d9b700 1 --2- 172.21.15.36:0/2663990418 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdfa8053a40 0x7fdfa8055f00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fdfc000b000 tx=0x7fdfc0008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:57.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.141+0000 7fdfbb7fe700 1 -- 172.21.15.36:0/2663990418 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdfbc042b60 con 0x7fdfcc0b7ce0 2022-06-17T12:48:57.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.309+0000 7fdfcb59e700 1 -- 172.21.15.36:0/2663990418 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfcc057410 con 0x7fdfcc0b7ce0 2022-06-17T12:48:57.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.309+0000 7fdfbb7fe700 1 -- 172.21.15.36:0/2663990418 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+2138 (secure 0 0 0) 0x7fdfbc046180 con 0x7fdfcc0b7ce0 2022-06-17T12:48:57.300 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:48:57.301 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3597082917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3597082917},{"type":"v1","addr":"172.21.15.70:6833","nonce":3597082917}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:47:47.072139+0000","modified":"2022-06-17T12:47:49.085163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4947":{"gid":4947,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.36:6835/1648255567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":1648255567},{"type":"v1","addr":"172.21.15.36:6835","nonce":1648255567}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-06-17T12:48:57.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.309+0000 7fdfcb59e700 1 -- 172.21.15.36:0/2663990418 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdfa8053a40 msgr2=0x7fdfa8055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:57.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.309+0000 7fdfcb59e700 1 --2- 172.21.15.36:0/2663990418 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdfa8053a40 0x7fdfa8055f00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fdfc000b000 tx=0x7fdfc0008040).stop 2022-06-17T12:48:57.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.313+0000 7fdfcb59e700 1 -- 172.21.15.36:0/2663990418 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfcc0b7ce0 msgr2=0x7fdfcc0619c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:57.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.313+0000 7fdfcb59e700 1 --2- 172.21.15.36:0/2663990418 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfcc0b7ce0 0x7fdfcc0619c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fdfbc0018c0 tx=0x7fdfbc001b30).stop 2022-06-17T12:48:57.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.313+0000 7fdfcb59e700 1 -- 172.21.15.36:0/2663990418 shutdown_connections 2022-06-17T12:48:57.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.313+0000 7fdfcb59e700 1 --2- 172.21.15.36:0/2663990418 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdfa8053a40 0x7fdfa8055f00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:57.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.313+0000 7fdfcb59e700 1 --2- 172.21.15.36:0/2663990418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfcc142d70 0x7fdfcc062380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:57.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.313+0000 7fdfcb59e700 1 --2- 172.21.15.36:0/2663990418 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfcc0b80c0 0x7fdfcc144670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:57.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.313+0000 7fdfcb59e700 1 --2- 172.21.15.36:0/2663990418 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfcc0b7ce0 0x7fdfcc0619c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:57.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.313+0000 7fdfcb59e700 1 -- 172.21.15.36:0/2663990418 >> 172.21.15.36:0/2663990418 conn(0x7fdfcc05c550 msgr2=0x7fdfcc0b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:48:57.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.313+0000 7fdfcb59e700 1 -- 172.21.15.36:0/2663990418 shutdown_connections 2022-06-17T12:48:57.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.313+0000 7fdfcb59e700 1 -- 172.21.15.36:0/2663990418 wait complete. 2022-06-17T12:48:57.309 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 14 2022-06-17T12:48:57.312 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-17T12:48:57.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.445+0000 7fbb7359e700 1 Processor -- start 2022-06-17T12:48:57.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.445+0000 7fbb7359e700 1 -- start start 2022-06-17T12:48:57.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.449+0000 7fbb7359e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb740aef60 0x7fbb740b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:57.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.449+0000 7fbb7359e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb740af4b0 con 0x7fbb740ba2e0 2022-06-17T12:48:57.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.449+0000 7fbb7359e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb74142f50 con 0x7fbb7413cb70 2022-06-17T12:48:57.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.449+0000 7fbb7359e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb74143da0 con 0x7fbb740aef60 2022-06-17T12:48:57.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.449+0000 7fbb72d9d700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fbb740ba2e0 0x7fbb741442a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50546/0 (socket says 172.21.15.36:50546) 2022-06-17T12:48:57.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.449+0000 7fbb72d9d700 1 -- 172.21.15.36:0/2417115758 learned_addr learned my addr 172.21.15.36:0/2417115758 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:48:57.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.449+0000 7fbb71d9b700 1 --2- 172.21.15.36:0/2417115758 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb740aef60 0x7fbb740b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:57.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.449+0000 7fbb7159a700 1 -- 172.21.15.36:0/2417115758 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 652098122 0 0) 0x7fbb740af4b0 con 0x7fbb740ba2e0 2022-06-17T12:48:57.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.449+0000 7fbb7159a700 1 -- 172.21.15.36:0/2417115758 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb54003260 con 0x7fbb740ba2e0 2022-06-17T12:48:57.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.449+0000 7fbb7159a700 1 -- 172.21.15.36:0/2417115758 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1602040145 0 0) 0x7fbb74142f50 con 0x7fbb7413cb70 2022-06-17T12:48:57.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.449+0000 7fbb71d9b700 1 -- 172.21.15.36:0/2417115758 >> v1:172.21.15.70:6789/0 conn(0x7fbb7413cb70 legacy=0x7fbb74143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:57.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.449+0000 7fbb71d9b700 1 -- 172.21.15.36:0/2417115758 >> v1:172.21.15.36:6789/0 conn(0x7fbb740ba2e0 legacy=0x7fbb741442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:57.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.449+0000 7fbb71d9b700 1 -- 172.21.15.36:0/2417115758 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb741431f0 con 0x7fbb740aef60 2022-06-17T12:48:57.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.449+0000 7fbb71d9b700 1 --2- 172.21.15.36:0/2417115758 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb740aef60 0x7fbb740b9df0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fbb5c002f70 tx=0x7fbb5c006df0).ready entity=mon.2 client_cookie=9add9602a7d35044 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:57.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.449+0000 7fbb7159a700 1 -- 172.21.15.36:0/2417115758 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb5c00d570 con 0x7fbb740aef60 2022-06-17T12:48:57.602 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.449+0000 7fbb7159a700 1 -- 172.21.15.36:0/2417115758 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5c000e90 con 0x7fbb740aef60 2022-06-17T12:48:57.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.449+0000 7fbb7159a700 1 -- 172.21.15.36:0/2417115758 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb5c00d8c0 con 0x7fbb740aef60 2022-06-17T12:48:57.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.449+0000 7fbb7359e700 1 -- 172.21.15.36:0/2417115758 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb740aef60 msgr2=0x7fbb740b9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:57.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.449+0000 7fbb7359e700 1 --2- 172.21.15.36:0/2417115758 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb740aef60 0x7fbb740b9df0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fbb5c002f70 tx=0x7fbb5c006df0).stop 2022-06-17T12:48:57.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.449+0000 7fbb7359e700 1 -- 172.21.15.36:0/2417115758 shutdown_connections 2022-06-17T12:48:57.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.449+0000 7fbb7359e700 1 --2- 172.21.15.36:0/2417115758 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb740aef60 0x7fbb740b9df0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:57.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.449+0000 7fbb7359e700 1 -- 172.21.15.36:0/2417115758 >> 172.21.15.36:0/2417115758 conn(0x7fbb7405c550 msgr2=0x7fbb740b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:48:57.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.449+0000 7fbb7359e700 1 -- 172.21.15.36:0/2417115758 shutdown_connections 2022-06-17T12:48:57.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.449+0000 7fbb7359e700 1 -- 172.21.15.36:0/2417115758 wait complete. 2022-06-17T12:48:57.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb7359e700 1 Processor -- start 2022-06-17T12:48:57.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb7359e700 1 -- start start 2022-06-17T12:48:57.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb7359e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb740aef60 0x7fbb740b2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:57.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb7359e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb740ba2e0 0x7fbb741400f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:57.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb7359e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb7413cb70 0x7fbb741521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:57.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb7359e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb741526c0 con 0x7fbb740aef60 2022-06-17T12:48:57.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb7359e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb741528e0 con 0x7fbb740ba2e0 2022-06-17T12:48:57.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb7359e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb74152b80 con 0x7fbb7413cb70 2022-06-17T12:48:57.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb7259c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb740aef60 0x7fbb740b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:57.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb7259c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb740aef60 0x7fbb740b2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33018/0 (socket says 172.21.15.36:33018) 2022-06-17T12:48:57.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb7259c700 1 -- 172.21.15.36:0/1277905940 learned_addr learned my addr 172.21.15.36:0/1277905940 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:48:57.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb71d9b700 1 --2- 172.21.15.36:0/1277905940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb740ba2e0 0x7fbb741400f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:57.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb72d9d700 1 --2- 172.21.15.36:0/1277905940 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb7413cb70 0x7fbb741521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:57.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb7259c700 1 -- 172.21.15.36:0/1277905940 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb7413cb70 msgr2=0x7fbb741521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:57.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb7259c700 1 --2- 172.21.15.36:0/1277905940 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb7413cb70 0x7fbb741521d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:57.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb7259c700 1 -- 172.21.15.36:0/1277905940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb740ba2e0 msgr2=0x7fbb741400f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:57.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb7259c700 1 --2- 172.21.15.36:0/1277905940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb740ba2e0 0x7fbb741400f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:57.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb7259c700 1 -- 172.21.15.36:0/1277905940 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5c00b180 con 0x7fbb740aef60 2022-06-17T12:48:57.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb72d9d700 1 --2- 172.21.15.36:0/1277905940 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb7413cb70 0x7fbb741521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:48:57.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb7259c700 1 --2- 172.21.15.36:0/1277905940 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb740aef60 0x7fbb740b2540 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fbb64000ed0 tx=0x7fbb64007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:57.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb6b7fe700 1 -- 172.21.15.36:0/1277905940 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb64012400 con 0x7fbb740aef60 2022-06-17T12:48:57.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb7359e700 1 -- 172.21.15.36:0/1277905940 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb74152e20 con 0x7fbb740aef60 2022-06-17T12:48:57.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb6b7fe700 1 -- 172.21.15.36:0/1277905940 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb64012560 con 0x7fbb740aef60 2022-06-17T12:48:57.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb7359e700 1 -- 172.21.15.36:0/1277905940 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb740b0ae0 con 0x7fbb740aef60 2022-06-17T12:48:57.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.453+0000 7fbb6b7fe700 1 -- 172.21.15.36:0/1277905940 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb640128c0 con 0x7fbb740aef60 2022-06-17T12:48:57.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.457+0000 7fbb7359e700 1 -- 172.21.15.36:0/1277905940 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb74152fb0 con 0x7fbb740aef60 2022-06-17T12:48:57.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.457+0000 7fbb6b7fe700 1 -- 172.21.15.36:0/1277905940 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fbb64019030 con 0x7fbb740aef60 2022-06-17T12:48:57.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.457+0000 7fbb6b7fe700 1 --2- 172.21.15.36:0/1277905940 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb58053a40 0x7fbb58055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:57.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.457+0000 7fbb6b7fe700 1 -- 172.21.15.36:0/1277905940 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbb6406b2b0 con 0x7fbb740aef60 2022-06-17T12:48:57.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.457+0000 7fbb71d9b700 1 --2- 172.21.15.36:0/1277905940 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb58053a40 0x7fbb58055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:57.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.457+0000 7fbb71d9b700 1 --2- 172.21.15.36:0/1277905940 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb58053a40 0x7fbb58055f00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7fbb5c01a000 tx=0x7fbb5c014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:57.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.465+0000 7fbb6b7fe700 1 -- 172.21.15.36:0/1277905940 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbb640420c0 con 0x7fbb740aef60 2022-06-17T12:48:57.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:57.629+0000 7fbb7359e700 1 -- 172.21.15.36:0/1277905940 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbb741449b0 con 0x7fbb740aef60 2022-06-17T12:48:58.610 INFO:tasks.ceph.mon.a.smithi036.stderr:2022-06-17T12:48:58.617+0000 7fb7a2649700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-17T12:48:58.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.629+0000 7fbb6b7fe700 1 -- 172.21.15.36:0/1277905940 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v15) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbb640456e0 con 0x7fbb740aef60 2022-06-17T12:48:58.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.633+0000 7fbb7359e700 1 -- 172.21.15.36:0/1277905940 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb58053a40 msgr2=0x7fbb58055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:58.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.633+0000 7fbb7359e700 1 --2- 172.21.15.36:0/1277905940 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb58053a40 0x7fbb58055f00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7fbb5c01a000 tx=0x7fbb5c014000).stop 2022-06-17T12:48:58.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.633+0000 7fbb7359e700 1 -- 172.21.15.36:0/1277905940 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb740aef60 msgr2=0x7fbb740b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:58.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.633+0000 7fbb7359e700 1 --2- 172.21.15.36:0/1277905940 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb740aef60 0x7fbb740b2540 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fbb64000ed0 tx=0x7fbb64007c50).stop 2022-06-17T12:48:58.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.633+0000 7fbb7359e700 1 -- 172.21.15.36:0/1277905940 shutdown_connections 2022-06-17T12:48:58.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.633+0000 7fbb7359e700 1 --2- 172.21.15.36:0/1277905940 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb58053a40 0x7fbb58055f00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:58.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.633+0000 7fbb7359e700 1 --2- 172.21.15.36:0/1277905940 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb7413cb70 0x7fbb741521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:58.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.633+0000 7fbb7359e700 1 --2- 172.21.15.36:0/1277905940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb740ba2e0 0x7fbb741400f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:58.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.633+0000 7fbb7359e700 1 --2- 172.21.15.36:0/1277905940 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb740aef60 0x7fbb740b2540 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:58.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.633+0000 7fbb7359e700 1 -- 172.21.15.36:0/1277905940 >> 172.21.15.36:0/1277905940 conn(0x7fbb7405c550 msgr2=0x7fbb740b6630 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:48:58.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.633+0000 7fbb7359e700 1 -- 172.21.15.36:0/1277905940 shutdown_connections 2022-06-17T12:48:58.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.633+0000 7fbb7359e700 1 -- 172.21.15.36:0/1277905940 wait complete. 2022-06-17T12:48:58.646 INFO:teuthology.orchestra.run.smithi036.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-17T12:48:58.647 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:48:58.658 INFO:tasks.ceph.mds.a.smithi036.stdout:starting mds.a at 2022-06-17T12:48:58.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.769+0000 7fb84e3e2700 1 Processor -- start 2022-06-17T12:48:58.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.769+0000 7fb84e3e2700 1 -- start start 2022-06-17T12:48:58.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb84e3e2700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb8480aef60 0x7fb8480b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:58.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb84e3e2700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb8480ba2e0 0x7fb8481442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:58.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb84e3e2700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8480af4b0 con 0x7fb84813cb70 2022-06-17T12:48:58.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb84e3e2700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb848143da0 con 0x7fb8480ba2e0 2022-06-17T12:48:58.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb84e3e2700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb848142f50 con 0x7fb8480aef60 2022-06-17T12:48:58.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb84d3e0700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fb84813cb70 0x7fb848143690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50568/0 (socket says 172.21.15.36:50568) 2022-06-17T12:48:58.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb84dbe1700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb8480ba2e0 0x7fb8481442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:58.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb84d3e0700 1 -- 172.21.15.36:0/393699162 learned_addr learned my addr 172.21.15.36:0/393699162 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:48:58.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb84cbdf700 1 --2- 172.21.15.36:0/393699162 >> v2:172.21.15.70:3300/0 conn(0x7fb8480aef60 0x7fb8480b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:58.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb84dbe1700 1 -- 172.21.15.36:0/393699162 >> v2:172.21.15.70:3300/0 conn(0x7fb8480aef60 msgr2=0x7fb8480b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:48:58.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb84dbe1700 1 --2- 172.21.15.36:0/393699162 >> v2:172.21.15.70:3300/0 conn(0x7fb8480aef60 0x7fb8480b9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:58.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb84dbe1700 1 -- 172.21.15.36:0/393699162 >> v1:172.21.15.36:6789/0 conn(0x7fb84813cb70 legacy=0x7fb848143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:58.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb84dbe1700 1 -- 172.21.15.36:0/393699162 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8481431f0 con 0x7fb8480ba2e0 2022-06-17T12:48:58.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb84cbdf700 1 --2- 172.21.15.36:0/393699162 >> v2:172.21.15.70:3300/0 conn(0x7fb8480aef60 0x7fb8480b9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:48:58.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb84dbe1700 1 --2- 172.21.15.36:0/393699162 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb8480ba2e0 0x7fb8481442a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7fb8400032f0 tx=0x7fb8400091d0).ready entity=mon.0 client_cookie=72ff5022e3d0d152 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:58.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb847fff700 1 -- 172.21.15.36:0/393699162 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb840009ea0 con 0x7fb8480ba2e0 2022-06-17T12:48:58.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb847fff700 1 -- 172.21.15.36:0/393699162 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84000b870 con 0x7fb8480ba2e0 2022-06-17T12:48:58.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb847fff700 1 -- 172.21.15.36:0/393699162 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb84000b9e0 con 0x7fb8480ba2e0 2022-06-17T12:48:58.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb84e3e2700 1 -- 172.21.15.36:0/393699162 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb8480ba2e0 msgr2=0x7fb8481442a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:58.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb84e3e2700 1 --2- 172.21.15.36:0/393699162 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb8480ba2e0 0x7fb8481442a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7fb8400032f0 tx=0x7fb8400091d0).stop 2022-06-17T12:48:58.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb84e3e2700 1 -- 172.21.15.36:0/393699162 shutdown_connections 2022-06-17T12:48:58.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb84e3e2700 1 --2- 172.21.15.36:0/393699162 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb8480ba2e0 0x7fb8481442a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:58.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb84e3e2700 1 --2- 172.21.15.36:0/393699162 >> v2:172.21.15.70:3300/0 conn(0x7fb8480aef60 0x7fb8480b9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:58.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb84e3e2700 1 -- 172.21.15.36:0/393699162 >> 172.21.15.36:0/393699162 conn(0x7fb84805c550 msgr2=0x7fb8480b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:48:58.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb84e3e2700 1 -- 172.21.15.36:0/393699162 shutdown_connections 2022-06-17T12:48:58.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.773+0000 7fb84e3e2700 1 -- 172.21.15.36:0/393699162 wait complete. 2022-06-17T12:48:58.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.777+0000 7fb84e3e2700 1 Processor -- start 2022-06-17T12:48:58.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.777+0000 7fb84e3e2700 1 -- start start 2022-06-17T12:48:58.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.777+0000 7fb84e3e2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb8480aef60 0x7fb8480b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:58.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.777+0000 7fb84e3e2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8480ba2e0 0x7fb8480b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:58.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.777+0000 7fb84e3e2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb84813cb70 0x7fb848152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:58.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.777+0000 7fb84e3e2700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8480b0610 con 0x7fb8480aef60 2022-06-17T12:48:58.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.777+0000 7fb84e3e2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8480b16a0 con 0x7fb84813cb70 2022-06-17T12:48:58.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.777+0000 7fb84e3e2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8480b1400 con 0x7fb8480ba2e0 2022-06-17T12:48:58.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.777+0000 7fb84d3e0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb8480aef60 0x7fb8480b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:58.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.777+0000 7fb84d3e0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb8480aef60 0x7fb8480b3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33046/0 (socket says 172.21.15.36:33046) 2022-06-17T12:48:58.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.777+0000 7fb84d3e0700 1 -- 172.21.15.36:0/2900126598 learned_addr learned my addr 172.21.15.36:0/2900126598 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:48:58.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.777+0000 7fb84dbe1700 1 --2- 172.21.15.36:0/2900126598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb84813cb70 0x7fb848152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:58.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.777+0000 7fb84cbdf700 1 --2- 172.21.15.36:0/2900126598 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8480ba2e0 0x7fb8480b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:58.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.777+0000 7fb84d3e0700 1 -- 172.21.15.36:0/2900126598 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8480ba2e0 msgr2=0x7fb8480b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:58.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.777+0000 7fb84d3e0700 1 --2- 172.21.15.36:0/2900126598 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8480ba2e0 0x7fb8480b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:58.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.777+0000 7fb84d3e0700 1 -- 172.21.15.36:0/2900126598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb84813cb70 msgr2=0x7fb848152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:58.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.777+0000 7fb84d3e0700 1 --2- 172.21.15.36:0/2900126598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb84813cb70 0x7fb848152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:58.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.777+0000 7fb84d3e0700 1 -- 172.21.15.36:0/2900126598 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb840009470 con 0x7fb8480aef60 2022-06-17T12:48:58.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.777+0000 7fb84d3e0700 1 --2- 172.21.15.36:0/2900126598 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb8480aef60 0x7fb8480b3720 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fb848140960 tx=0x7fb838007c60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:58.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.777+0000 7fb8467fc700 1 -- 172.21.15.36:0/2900126598 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb838007110 con 0x7fb8480aef60 2022-06-17T12:48:58.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.777+0000 7fb84e3e2700 1 -- 172.21.15.36:0/2900126598 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb84813f9f0 con 0x7fb8480aef60 2022-06-17T12:48:58.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.777+0000 7fb8467fc700 1 -- 172.21.15.36:0/2900126598 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb838007270 con 0x7fb8480aef60 2022-06-17T12:48:58.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.777+0000 7fb8467fc700 1 -- 172.21.15.36:0/2900126598 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb838005030 con 0x7fb8480aef60 2022-06-17T12:48:58.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.777+0000 7fb84e3e2700 1 -- 172.21.15.36:0/2900126598 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb848140680 con 0x7fb8480aef60 2022-06-17T12:48:58.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.781+0000 7fb84e3e2700 1 -- 172.21.15.36:0/2900126598 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb848057410 con 0x7fb8480aef60 2022-06-17T12:48:58.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.781+0000 7fb8467fc700 1 -- 172.21.15.36:0/2900126598 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fb83801b070 con 0x7fb8480aef60 2022-06-17T12:48:58.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.781+0000 7fb8467fc700 1 --2- 172.21.15.36:0/2900126598 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb828053a40 0x7fb828055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:58.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.781+0000 7fb8467fc700 1 -- 172.21.15.36:0/2900126598 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fb83806b440 con 0x7fb8480aef60 2022-06-17T12:48:58.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.781+0000 7fb84cbdf700 1 --2- 172.21.15.36:0/2900126598 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb828053a40 0x7fb828055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:58.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.781+0000 7fb84cbdf700 1 --2- 172.21.15.36:0/2900126598 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb828053a40 0x7fb828055f00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fb83c00c000 tx=0x7fb83c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:58.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.789+0000 7fb8467fc700 1 -- 172.21.15.36:0/2900126598 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb838042150 con 0x7fb8480aef60 2022-06-17T12:48:58.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.945+0000 7fb84e3e2700 1 -- 172.21.15.36:0/2900126598 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb848062240 con 0x7fb8480aef60 2022-06-17T12:48:58.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.945+0000 7fb8467fc700 1 -- 172.21.15.36:0/2900126598 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+1777 (secure 0 0 0) 0x7fb838045770 con 0x7fb8480aef60 2022-06-17T12:48:58.958 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:48:58.959 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3597082917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3597082917},{"type":"v1","addr":"172.21.15.70:6833","nonce":3597082917}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:47:47.072139+0000","modified":"2022-06-17T12:48:58.619908+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-06-17T12:48:58.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.949+0000 7fb84e3e2700 1 -- 172.21.15.36:0/2900126598 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb828053a40 msgr2=0x7fb828055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:58.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.949+0000 7fb84e3e2700 1 --2- 172.21.15.36:0/2900126598 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb828053a40 0x7fb828055f00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fb83c00c000 tx=0x7fb83c008040).stop 2022-06-17T12:48:58.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.949+0000 7fb84e3e2700 1 -- 172.21.15.36:0/2900126598 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb8480aef60 msgr2=0x7fb8480b3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:58.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.949+0000 7fb84e3e2700 1 --2- 172.21.15.36:0/2900126598 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb8480aef60 0x7fb8480b3720 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fb848140960 tx=0x7fb838007c60).stop 2022-06-17T12:48:58.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.949+0000 7fb84e3e2700 1 -- 172.21.15.36:0/2900126598 shutdown_connections 2022-06-17T12:48:58.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.949+0000 7fb84e3e2700 1 --2- 172.21.15.36:0/2900126598 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb828053a40 0x7fb828055f00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:58.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.949+0000 7fb84e3e2700 1 --2- 172.21.15.36:0/2900126598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb84813cb70 0x7fb848152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:58.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.949+0000 7fb84e3e2700 1 --2- 172.21.15.36:0/2900126598 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8480ba2e0 0x7fb8480b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:58.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.949+0000 7fb84e3e2700 1 --2- 172.21.15.36:0/2900126598 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb8480aef60 0x7fb8480b3720 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:58.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.949+0000 7fb84e3e2700 1 -- 172.21.15.36:0/2900126598 >> 172.21.15.36:0/2900126598 conn(0x7fb84805c550 msgr2=0x7fb84813c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:48:58.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.949+0000 7fb84e3e2700 1 -- 172.21.15.36:0/2900126598 shutdown_connections 2022-06-17T12:48:58.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:58.949+0000 7fb84e3e2700 1 -- 172.21.15.36:0/2900126598 wait complete. 2022-06-17T12:48:58.983 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 15 2022-06-17T12:48:58.994 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-17T12:48:59.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.161+0000 7fd7d0d82700 1 Processor -- start 2022-06-17T12:48:59.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.161+0000 7fd7d0d82700 1 -- start start 2022-06-17T12:48:59.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7d0d82700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd7cc0b27b0 0x7fd7cc0b1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:59.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7d0d82700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7cc0ac0b0 con 0x7fd7cc140940 2022-06-17T12:48:59.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7d0d82700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7cc0b2b90 con 0x7fd7cc144d20 2022-06-17T12:48:59.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7d0d82700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7cc140d90 con 0x7fd7cc0b27b0 2022-06-17T12:48:59.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7cbfff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fd7cc140940 0x7fd7cc141180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50586/0 (socket says 172.21.15.36:50586) 2022-06-17T12:48:59.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7cbfff700 1 -- 172.21.15.36:0/3019603774 learned_addr learned my addr 172.21.15.36:0/3019603774 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:48:59.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7cb7fe700 1 --2- 172.21.15.36:0/3019603774 >> v2:172.21.15.70:3300/0 conn(0x7fd7cc0b27b0 0x7fd7cc0b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:59.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7ca7fc700 1 -- 172.21.15.36:0/3019603774 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2586418742 0 0) 0x7fd7cc0ac0b0 con 0x7fd7cc140940 2022-06-17T12:48:59.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7ca7fc700 1 -- 172.21.15.36:0/3019603774 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7ac003260 con 0x7fd7cc140940 2022-06-17T12:48:59.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7ca7fc700 1 -- 172.21.15.36:0/3019603774 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3026321817 0 0) 0x7fd7cc0b2b90 con 0x7fd7cc144d20 2022-06-17T12:48:59.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7ca7fc700 1 -- 172.21.15.36:0/3019603774 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7cc0ac0b0 con 0x7fd7cc144d20 2022-06-17T12:48:59.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7ca7fc700 1 -- 172.21.15.36:0/3019603774 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4185045123 0 0) 0x7fd7ac003260 con 0x7fd7cc140940 2022-06-17T12:48:59.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7ca7fc700 1 -- 172.21.15.36:0/3019603774 >> v2:172.21.15.70:3300/0 conn(0x7fd7cc0b27b0 msgr2=0x7fd7cc0b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:48:59.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7ca7fc700 1 --2- 172.21.15.36:0/3019603774 >> v2:172.21.15.70:3300/0 conn(0x7fd7cc0b27b0 0x7fd7cc0b1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:59.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7ca7fc700 1 -- 172.21.15.36:0/3019603774 >> v1:172.21.15.70:6789/0 conn(0x7fd7cc144d20 legacy=0x7fd7cc140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:59.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7ca7fc700 1 -- 172.21.15.36:0/3019603774 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc0afc60 con 0x7fd7cc140940 2022-06-17T12:48:59.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7ca7fc700 1 -- 172.21.15.36:0/3019603774 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7fd7c40019e0 con 0x7fd7cc140940 2022-06-17T12:48:59.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7ca7fc700 1 -- 172.21.15.36:0/3019603774 >> v1:172.21.15.36:6789/0 conn(0x7fd7cc140940 legacy=0x7fd7cc141180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:59.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7ca7fc700 1 --2- 172.21.15.36:0/3019603774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ac0038e0 0x7fd7ac003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:59.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7ca7fc700 1 --2- 172.21.15.36:0/3019603774 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7ac004500 0x7fd7ac004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:59.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7ca7fc700 1 --2- 172.21.15.36:0/3019603774 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7ac004e70 0x7fd7ac00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:59.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7ca7fc700 1 -- 172.21.15.36:0/3019603774 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac00bcf0 con 0x7fd7ac004500 2022-06-17T12:48:59.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7ca7fc700 1 -- 172.21.15.36:0/3019603774 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac00be30 con 0x7fd7ac0038e0 2022-06-17T12:48:59.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7ca7fc700 1 -- 172.21.15.36:0/3019603774 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac00bff0 con 0x7fd7ac004e70 2022-06-17T12:48:59.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7caffd700 1 --2- 172.21.15.36:0/3019603774 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7ac004500 0x7fd7ac004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:59.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7cb7fe700 1 --2- 172.21.15.36:0/3019603774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ac0038e0 0x7fd7ac003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:59.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7cbfff700 1 --2- 172.21.15.36:0/3019603774 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7ac004e70 0x7fd7ac00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:59.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7caffd700 1 -- 172.21.15.36:0/3019603774 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7ac004e70 msgr2=0x7fd7ac00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:48:59.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7caffd700 1 --2- 172.21.15.36:0/3019603774 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7ac004e70 0x7fd7ac00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:59.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7caffd700 1 -- 172.21.15.36:0/3019603774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ac0038e0 msgr2=0x7fd7ac003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:48:59.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7caffd700 1 --2- 172.21.15.36:0/3019603774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ac0038e0 0x7fd7ac003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:59.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7caffd700 1 -- 172.21.15.36:0/3019603774 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ac003260 con 0x7fd7ac004500 2022-06-17T12:48:59.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7cbfff700 1 -- 172.21.15.36:0/3019603774 reap_dead start 2022-06-17T12:48:59.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7caffd700 1 --2- 172.21.15.36:0/3019603774 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7ac004500 0x7fd7ac004980 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7fd7bc001110 tx=0x7fd7bc007a00).ready entity=mon.0 client_cookie=2d874134df77aa52 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:59.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7ca7fc700 1 -- 172.21.15.36:0/3019603774 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7bc007d10 con 0x7fd7ac004500 2022-06-17T12:48:59.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7ca7fc700 1 -- 172.21.15.36:0/3019603774 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7bc007030 con 0x7fd7ac004500 2022-06-17T12:48:59.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.165+0000 7fd7ca7fc700 1 -- 172.21.15.36:0/3019603774 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7bc007390 con 0x7fd7ac004500 2022-06-17T12:48:59.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7d0d82700 1 -- 172.21.15.36:0/3019603774 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7ac004500 msgr2=0x7fd7ac004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:59.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7d0d82700 1 --2- 172.21.15.36:0/3019603774 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7ac004500 0x7fd7ac004980 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7fd7bc001110 tx=0x7fd7bc007a00).stop 2022-06-17T12:48:59.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7d0d82700 1 -- 172.21.15.36:0/3019603774 shutdown_connections 2022-06-17T12:48:59.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7d0d82700 1 --2- 172.21.15.36:0/3019603774 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7ac004500 0x7fd7ac004980 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:59.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7d0d82700 1 -- 172.21.15.36:0/3019603774 >> 172.21.15.36:0/3019603774 conn(0x7fd7cc0b5d10 msgr2=0x7fd7cc0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:48:59.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7d0d82700 1 -- 172.21.15.36:0/3019603774 shutdown_connections 2022-06-17T12:48:59.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7d0d82700 1 -- 172.21.15.36:0/3019603774 wait complete. 2022-06-17T12:48:59.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7d0d82700 1 Processor -- start 2022-06-17T12:48:59.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7d0d82700 1 -- start start 2022-06-17T12:48:59.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7d0d82700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7ac004500 0x7fd7cc143810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:59.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7d0d82700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7cc13f310 0x7fd7cc13cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:59.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7d0d82700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7cc155630 0x7fd7cc155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:59.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7d0d82700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc0af840 con 0x7fd7ac004500 2022-06-17T12:48:59.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7d0d82700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc143dc0 con 0x7fd7cc13f310 2022-06-17T12:48:59.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7d0d82700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc142360 con 0x7fd7cc155630 2022-06-17T12:48:59.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7cb7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7ac004500 0x7fd7cc143810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:59.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7caffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7cc13f310 0x7fd7cc13cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:59.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7cb7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7ac004500 0x7fd7cc143810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33064/0 (socket says 172.21.15.36:33064) 2022-06-17T12:48:59.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7cb7fe700 1 -- 172.21.15.36:0/4291864352 learned_addr learned my addr 172.21.15.36:0/4291864352 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:48:59.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7cbfff700 1 --2- 172.21.15.36:0/4291864352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7cc155630 0x7fd7cc155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:59.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7caffd700 1 -- 172.21.15.36:0/4291864352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7cc155630 msgr2=0x7fd7cc155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:59.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7caffd700 1 --2- 172.21.15.36:0/4291864352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7cc155630 0x7fd7cc155ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:59.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7caffd700 1 -- 172.21.15.36:0/4291864352 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7ac004500 msgr2=0x7fd7cc143810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:59.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7caffd700 1 --2- 172.21.15.36:0/4291864352 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7ac004500 0x7fd7cc143810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:59.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7caffd700 1 -- 172.21.15.36:0/4291864352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc155fa0 con 0x7fd7cc13f310 2022-06-17T12:48:59.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7caffd700 1 --2- 172.21.15.36:0/4291864352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7cc13f310 0x7fd7cc13cc20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fd7cc0ae930 tx=0x7fd7bc015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:59.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7c8ff9700 1 -- 172.21.15.36:0/4291864352 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7bc006c70 con 0x7fd7cc13f310 2022-06-17T12:48:59.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7d0d82700 1 -- 172.21.15.36:0/4291864352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7cc1561f0 con 0x7fd7cc13f310 2022-06-17T12:48:59.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7c8ff9700 1 -- 172.21.15.36:0/4291864352 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7bc006dd0 con 0x7fd7cc13f310 2022-06-17T12:48:59.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.169+0000 7fd7c8ff9700 1 -- 172.21.15.36:0/4291864352 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7bc013640 con 0x7fd7cc13f310 2022-06-17T12:48:59.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.173+0000 7fd7d0d82700 1 -- 172.21.15.36:0/4291864352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7cc152350 con 0x7fd7cc13f310 2022-06-17T12:48:59.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.173+0000 7fd7d0d82700 1 -- 172.21.15.36:0/4291864352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7cc152010 con 0x7fd7cc13f310 2022-06-17T12:48:59.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.173+0000 7fd7c8ff9700 1 -- 172.21.15.36:0/4291864352 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd7bc010070 con 0x7fd7cc13f310 2022-06-17T12:48:59.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.173+0000 7fd7c8ff9700 1 --2- 172.21.15.36:0/4291864352 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7b0053a40 0x7fd7b0055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:48:59.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.173+0000 7fd7c8ff9700 1 -- 172.21.15.36:0/4291864352 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fd7bc06c7f0 con 0x7fd7cc13f310 2022-06-17T12:48:59.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.173+0000 7fd7cb7fe700 1 --2- 172.21.15.36:0/4291864352 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7b0053a40 0x7fd7b0055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:48:59.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.173+0000 7fd7cb7fe700 1 --2- 172.21.15.36:0/4291864352 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7b0053a40 0x7fd7b0055f00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fd7b400b000 tx=0x7fd7b4006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:48:59.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.181+0000 7fd7c8ff9700 1 -- 172.21.15.36:0/4291864352 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd7cc152010 con 0x7fd7cc13f310 2022-06-17T12:48:59.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.353+0000 7fd7d0d82700 1 -- 172.21.15.36:0/4291864352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd7cc152010 con 0x7fd7cc13f310 2022-06-17T12:48:59.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.633+0000 7fd7c8ff9700 1 -- 172.21.15.36:0/4291864352 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v16) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd7cc152010 con 0x7fd7cc13f310 2022-06-17T12:48:59.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.637+0000 7fd7d0d82700 1 -- 172.21.15.36:0/4291864352 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7b0053a40 msgr2=0x7fd7b0055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:59.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.637+0000 7fd7d0d82700 1 --2- 172.21.15.36:0/4291864352 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7b0053a40 0x7fd7b0055f00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fd7b400b000 tx=0x7fd7b4006df0).stop 2022-06-17T12:48:59.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.637+0000 7fd7d0d82700 1 -- 172.21.15.36:0/4291864352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7cc13f310 msgr2=0x7fd7cc13cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:48:59.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.637+0000 7fd7d0d82700 1 --2- 172.21.15.36:0/4291864352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7cc13f310 0x7fd7cc13cc20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fd7cc0ae930 tx=0x7fd7bc015000).stop 2022-06-17T12:48:59.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.637+0000 7fd7d0d82700 1 -- 172.21.15.36:0/4291864352 shutdown_connections 2022-06-17T12:48:59.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.637+0000 7fd7d0d82700 1 --2- 172.21.15.36:0/4291864352 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7b0053a40 0x7fd7b0055f00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:59.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.637+0000 7fd7d0d82700 1 --2- 172.21.15.36:0/4291864352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7cc155630 0x7fd7cc155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:59.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.637+0000 7fd7d0d82700 1 --2- 172.21.15.36:0/4291864352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7cc13f310 0x7fd7cc13cc20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:59.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.637+0000 7fd7d0d82700 1 --2- 172.21.15.36:0/4291864352 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7ac004500 0x7fd7cc143810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:48:59.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.637+0000 7fd7d0d82700 1 -- 172.21.15.36:0/4291864352 >> 172.21.15.36:0/4291864352 conn(0x7fd7cc0b5d10 msgr2=0x7fd7cc13ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:48:59.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.637+0000 7fd7d0d82700 1 -- 172.21.15.36:0/4291864352 shutdown_connections 2022-06-17T12:48:59.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.637+0000 7fd7d0d82700 1 -- 172.21.15.36:0/4291864352 wait complete. 2022-06-17T12:48:59.765 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-17T12:49:00.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.937+0000 7fa762b40700 1 Processor -- start 2022-06-17T12:49:00.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.937+0000 7fa762b40700 1 -- start start 2022-06-17T12:49:00.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.937+0000 7fa762b40700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa75c0b0f00 0x7fa75c140eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:00.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.937+0000 7fa762b40700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa75c0aa060 con 0x7fa75c0acf00 2022-06-17T12:49:00.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.937+0000 7fa762b40700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa75c0b66e0 con 0x7fa75c0abbb0 2022-06-17T12:49:00.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.937+0000 7fa762b40700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c0b64f0 con 0x7fa75c0b0f00 2022-06-17T12:49:00.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.937+0000 7fa7608dc700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fa75c0acf00 0x7fa75c0b1d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50604/0 (socket says 172.21.15.36:50604) 2022-06-17T12:49:00.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.937+0000 7fa7608dc700 1 -- 172.21.15.36:0/1498131751 learned_addr learned my addr 172.21.15.36:0/1498131751 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:00.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.937+0000 7fa7610dd700 1 --2- 172.21.15.36:0/1498131751 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa75c0b0f00 0x7fa75c140eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:00.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.937+0000 7fa75b7fe700 1 -- 172.21.15.36:0/1498131751 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1090881771 0 0) 0x7fa75c0aa060 con 0x7fa75c0acf00 2022-06-17T12:49:00.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.937+0000 7fa75b7fe700 1 -- 172.21.15.36:0/1498131751 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa740003260 con 0x7fa75c0acf00 2022-06-17T12:49:00.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.937+0000 7fa75b7fe700 1 -- 172.21.15.36:0/1498131751 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1332488562 0 0) 0x7fa75c0b66e0 con 0x7fa75c0abbb0 2022-06-17T12:49:00.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.937+0000 7fa75b7fe700 1 -- 172.21.15.36:0/1498131751 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa75c0aa060 con 0x7fa75c0abbb0 2022-06-17T12:49:00.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.937+0000 7fa7610dd700 1 -- 172.21.15.36:0/1498131751 >> v1:172.21.15.70:6789/0 conn(0x7fa75c0abbb0 legacy=0x7fa75c1407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:00.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.937+0000 7fa7610dd700 1 -- 172.21.15.36:0/1498131751 >> v1:172.21.15.36:6789/0 conn(0x7fa75c0acf00 legacy=0x7fa75c0b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:00.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.937+0000 7fa7610dd700 1 -- 172.21.15.36:0/1498131751 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75c0b2480 con 0x7fa75c0b0f00 2022-06-17T12:49:00.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.937+0000 7fa7610dd700 1 --2- 172.21.15.36:0/1498131751 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa75c0b0f00 0x7fa75c140eb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fa7540032f0 tx=0x7fa7540091d0).ready entity=mon.2 client_cookie=ad4164e68b8bdd09 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:00.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.941+0000 7fa75b7fe700 1 -- 172.21.15.36:0/1498131751 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa75400b640 con 0x7fa75c0b0f00 2022-06-17T12:49:00.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.941+0000 7fa75b7fe700 1 -- 172.21.15.36:0/1498131751 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75400b7a0 con 0x7fa75c0b0f00 2022-06-17T12:49:00.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.941+0000 7fa75b7fe700 1 -- 172.21.15.36:0/1498131751 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa75400b910 con 0x7fa75c0b0f00 2022-06-17T12:49:00.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.941+0000 7fa762b40700 1 -- 172.21.15.36:0/1498131751 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa75c0b0f00 msgr2=0x7fa75c140eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:00.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.941+0000 7fa762b40700 1 --2- 172.21.15.36:0/1498131751 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa75c0b0f00 0x7fa75c140eb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fa7540032f0 tx=0x7fa7540091d0).stop 2022-06-17T12:49:00.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.941+0000 7fa762b40700 1 -- 172.21.15.36:0/1498131751 shutdown_connections 2022-06-17T12:49:00.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.941+0000 7fa762b40700 1 --2- 172.21.15.36:0/1498131751 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa75c0b0f00 0x7fa75c140eb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:00.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.941+0000 7fa762b40700 1 -- 172.21.15.36:0/1498131751 >> 172.21.15.36:0/1498131751 conn(0x7fa75c0636d0 msgr2=0x7fa75c063ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:00.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.941+0000 7fa762b40700 1 -- 172.21.15.36:0/1498131751 shutdown_connections 2022-06-17T12:49:00.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.941+0000 7fa762b40700 1 -- 172.21.15.36:0/1498131751 wait complete. 2022-06-17T12:49:00.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.941+0000 7fa762b40700 1 Processor -- start 2022-06-17T12:49:00.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.945+0000 7fa762b40700 1 -- start start 2022-06-17T12:49:00.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.945+0000 7fa762b40700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa75c0abbb0 0x7fa75c0b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:00.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.945+0000 7fa762b40700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa75c0acf00 0x7fa75c065640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:00.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.945+0000 7fa762b40700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa75c0b0f00 0x7fa75c067030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:00.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.945+0000 7fa762b40700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c143f70 con 0x7fa75c0acf00 2022-06-17T12:49:00.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.945+0000 7fa762b40700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c0adf70 con 0x7fa75c0b0f00 2022-06-17T12:49:00.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.945+0000 7fa762b40700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c143a20 con 0x7fa75c0abbb0 2022-06-17T12:49:00.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.945+0000 7fa75bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa75c0acf00 0x7fa75c065640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:00.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.945+0000 7fa75bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa75c0acf00 0x7fa75c065640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33082/0 (socket says 172.21.15.36:33082) 2022-06-17T12:49:00.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.945+0000 7fa75bfff700 1 -- 172.21.15.36:0/3970506292 learned_addr learned my addr 172.21.15.36:0/3970506292 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:00.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.945+0000 7fa7608dc700 1 --2- 172.21.15.36:0/3970506292 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa75c0abbb0 0x7fa75c0b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:00.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.945+0000 7fa7610dd700 1 --2- 172.21.15.36:0/3970506292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa75c0b0f00 0x7fa75c067030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:00.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.945+0000 7fa75bfff700 1 -- 172.21.15.36:0/3970506292 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa75c0abbb0 msgr2=0x7fa75c0b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:00.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.945+0000 7fa75bfff700 1 --2- 172.21.15.36:0/3970506292 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa75c0abbb0 0x7fa75c0b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:00.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.945+0000 7fa75bfff700 1 -- 172.21.15.36:0/3970506292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa75c0b0f00 msgr2=0x7fa75c067030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:00.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.945+0000 7fa75bfff700 1 --2- 172.21.15.36:0/3970506292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa75c0b0f00 0x7fa75c067030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:00.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.945+0000 7fa75bfff700 1 -- 172.21.15.36:0/3970506292 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa754009470 con 0x7fa75c0acf00 2022-06-17T12:49:00.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.945+0000 7fa7610dd700 1 --2- 172.21.15.36:0/3970506292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa75c0b0f00 0x7fa75c067030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:49:00.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.945+0000 7fa75bfff700 1 --2- 172.21.15.36:0/3970506292 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa75c0acf00 0x7fa75c065640 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7fa75c143940 tx=0x7fa748007ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:00.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.945+0000 7fa759ffb700 1 -- 172.21.15.36:0/3970506292 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa748012400 con 0x7fa75c0acf00 2022-06-17T12:49:00.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.945+0000 7fa762b40700 1 -- 172.21.15.36:0/3970506292 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa75c065b30 con 0x7fa75c0acf00 2022-06-17T12:49:00.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.945+0000 7fa759ffb700 1 -- 172.21.15.36:0/3970506292 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa748012560 con 0x7fa75c0acf00 2022-06-17T12:49:00.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.945+0000 7fa759ffb700 1 -- 172.21.15.36:0/3970506292 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7480128c0 con 0x7fa75c0acf00 2022-06-17T12:49:00.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.945+0000 7fa762b40700 1 -- 172.21.15.36:0/3970506292 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa75c143510 con 0x7fa75c0acf00 2022-06-17T12:49:00.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.949+0000 7fa762b40700 1 -- 172.21.15.36:0/3970506292 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa75c143be0 con 0x7fa75c0acf00 2022-06-17T12:49:00.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.949+0000 7fa759ffb700 1 -- 172.21.15.36:0/3970506292 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fa748019030 con 0x7fa75c0acf00 2022-06-17T12:49:00.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.949+0000 7fa759ffb700 1 --2- 172.21.15.36:0/3970506292 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa73c053ad0 0x7fa73c055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:00.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.949+0000 7fa759ffb700 1 -- 172.21.15.36:0/3970506292 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fa74806b420 con 0x7fa75c0acf00 2022-06-17T12:49:00.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.949+0000 7fa7608dc700 1 --2- 172.21.15.36:0/3970506292 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa73c053ad0 0x7fa73c055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:00.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.949+0000 7fa7608dc700 1 --2- 172.21.15.36:0/3970506292 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa73c053ad0 0x7fa73c055f90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fa75000d000 tx=0x7fa750009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:00.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:48:59.953+0000 7fa759ffb700 1 -- 172.21.15.36:0/3970506292 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa7480421e0 con 0x7fa75c0acf00 2022-06-17T12:49:00.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.077+0000 7fa762b40700 1 -- 172.21.15.36:0/3970506292 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa75c0a9ce0 con 0x7fa75c0acf00 2022-06-17T12:49:00.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.709+0000 7fa759ffb700 1 -- 172.21.15.36:0/3970506292 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) v1 ==== 185+0+0 (secure 0 0 0) 0x7fa748045800 con 0x7fa75c0acf00 2022-06-17T12:49:00.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.713+0000 7fa762b40700 1 -- 172.21.15.36:0/3970506292 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa73c053ad0 msgr2=0x7fa73c055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:00.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.713+0000 7fa762b40700 1 --2- 172.21.15.36:0/3970506292 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa73c053ad0 0x7fa73c055f90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fa75000d000 tx=0x7fa750009040).stop 2022-06-17T12:49:00.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.713+0000 7fa762b40700 1 -- 172.21.15.36:0/3970506292 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa75c0acf00 msgr2=0x7fa75c065640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:00.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.713+0000 7fa762b40700 1 --2- 172.21.15.36:0/3970506292 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa75c0acf00 0x7fa75c065640 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7fa75c143940 tx=0x7fa748007ab0).stop 2022-06-17T12:49:00.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.717+0000 7fa762b40700 1 -- 172.21.15.36:0/3970506292 shutdown_connections 2022-06-17T12:49:00.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.717+0000 7fa762b40700 1 --2- 172.21.15.36:0/3970506292 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa73c053ad0 0x7fa73c055f90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:00.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.717+0000 7fa762b40700 1 --2- 172.21.15.36:0/3970506292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa75c0b0f00 0x7fa75c067030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:00.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.717+0000 7fa762b40700 1 --2- 172.21.15.36:0/3970506292 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa75c0acf00 0x7fa75c065640 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:00.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.717+0000 7fa762b40700 1 --2- 172.21.15.36:0/3970506292 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa75c0abbb0 0x7fa75c0b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:00.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.717+0000 7fa762b40700 1 -- 172.21.15.36:0/3970506292 >> 172.21.15.36:0/3970506292 conn(0x7fa75c0636d0 msgr2=0x7fa75c05bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:00.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.717+0000 7fa762b40700 1 -- 172.21.15.36:0/3970506292 shutdown_connections 2022-06-17T12:49:00.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.717+0000 7fa762b40700 1 -- 172.21.15.36:0/3970506292 wait complete. 2022-06-17T12:49:00.715 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' removed 2022-06-17T12:49:00.725 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-17T12:49:00.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.933+0000 7fca7e64b700 1 Processor -- start 2022-06-17T12:49:00.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.933+0000 7fca7e64b700 1 -- start start 2022-06-17T12:49:00.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7e64b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fca780acb00 0x7fca780a8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:00.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7e64b700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fca780acee0 0x7fca7813f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:00.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7e64b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fca7813fa20 0x7fca78143680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:00.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7e64b700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca780af6e0 con 0x7fca780acb00 2022-06-17T12:49:00.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7e64b700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca780af980 con 0x7fca780acee0 2022-06-17T12:49:00.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7e64b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca780ad770 con 0x7fca7813fa20 2022-06-17T12:49:00.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7d649700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fca780acb00 0x7fca780a8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:00.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7d649700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fca780acb00 0x7fca780a8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33088/0 (socket says 172.21.15.36:33088) 2022-06-17T12:49:00.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7d649700 1 -- 172.21.15.36:0/3755782788 learned_addr learned my addr 172.21.15.36:0/3755782788 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:00.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7ce48700 1 --2- 172.21.15.36:0/3755782788 >> v2:172.21.15.70:3300/0 conn(0x7fca780acee0 0x7fca7813f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:00.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7de4a700 1 --2- 172.21.15.36:0/3755782788 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fca7813fa20 0x7fca78143680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:00.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7d649700 1 -- 172.21.15.36:0/3755782788 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fca7813fa20 msgr2=0x7fca78143680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:00.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7d649700 1 --2- 172.21.15.36:0/3755782788 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fca7813fa20 0x7fca78143680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:00.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7d649700 1 -- 172.21.15.36:0/3755782788 >> v2:172.21.15.70:3300/0 conn(0x7fca780acee0 msgr2=0x7fca7813f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:00.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7d649700 1 --2- 172.21.15.36:0/3755782788 >> v2:172.21.15.70:3300/0 conn(0x7fca780acee0 0x7fca7813f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:00.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7d649700 1 -- 172.21.15.36:0/3755782788 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca78067b30 con 0x7fca780acb00 2022-06-17T12:49:00.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7d649700 1 --2- 172.21.15.36:0/3755782788 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fca780acb00 0x7fca780a8d60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7fca68002f70 tx=0x7fca68006eb0).ready entity=mon.0 client_cookie=1aa86d42e4993b5d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:00.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca67fff700 1 -- 172.21.15.36:0/3755782788 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca6800d570 con 0x7fca780acb00 2022-06-17T12:49:01.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca67fff700 1 -- 172.21.15.36:0/3755782788 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca68000e90 con 0x7fca780acb00 2022-06-17T12:49:01.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca67fff700 1 -- 172.21.15.36:0/3755782788 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca6800d900 con 0x7fca780acb00 2022-06-17T12:49:01.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7e64b700 1 -- 172.21.15.36:0/3755782788 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fca780acb00 msgr2=0x7fca780a8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:01.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7e64b700 1 --2- 172.21.15.36:0/3755782788 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fca780acb00 0x7fca780a8d60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7fca68002f70 tx=0x7fca68006eb0).stop 2022-06-17T12:49:01.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7e64b700 1 -- 172.21.15.36:0/3755782788 shutdown_connections 2022-06-17T12:49:01.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7e64b700 1 --2- 172.21.15.36:0/3755782788 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fca7813fa20 0x7fca78143680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:01.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7e64b700 1 --2- 172.21.15.36:0/3755782788 >> v2:172.21.15.70:3300/0 conn(0x7fca780acee0 0x7fca7813f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:01.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7e64b700 1 --2- 172.21.15.36:0/3755782788 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fca780acb00 0x7fca780a8d60 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:01.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7e64b700 1 -- 172.21.15.36:0/3755782788 >> 172.21.15.36:0/3755782788 conn(0x7fca780627f0 msgr2=0x7fca78062bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:01.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7e64b700 1 -- 172.21.15.36:0/3755782788 shutdown_connections 2022-06-17T12:49:01.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.937+0000 7fca7e64b700 1 -- 172.21.15.36:0/3755782788 wait complete. 2022-06-17T12:49:01.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.941+0000 7fca7e64b700 1 Processor -- start 2022-06-17T12:49:01.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.941+0000 7fca7e64b700 1 -- start start 2022-06-17T12:49:01.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.941+0000 7fca7e64b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fca780acb00 0x7fca780b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:01.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.941+0000 7fca7e64b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca780acee0 0x7fca78153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:01.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.941+0000 7fca7e64b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fca7813fa20 0x7fca781534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:01.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.941+0000 7fca7e64b700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca780b1110 con 0x7fca7813fa20 2022-06-17T12:49:01.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.941+0000 7fca7e64b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca780aa2d0 con 0x7fca780acee0 2022-06-17T12:49:01.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.941+0000 7fca7e64b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca780b1340 con 0x7fca780acb00 2022-06-17T12:49:01.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.941+0000 7fca7de4a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fca7813fa20 0x7fca781534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:01.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.941+0000 7fca7de4a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fca7813fa20 0x7fca781534d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33096/0 (socket says 172.21.15.36:33096) 2022-06-17T12:49:01.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.941+0000 7fca7de4a700 1 -- 172.21.15.36:0/2812433998 learned_addr learned my addr 172.21.15.36:0/2812433998 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:01.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.941+0000 7fca7d649700 1 --2- 172.21.15.36:0/2812433998 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fca780acb00 0x7fca780b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:01.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.941+0000 7fca7ce48700 1 --2- 172.21.15.36:0/2812433998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca780acee0 0x7fca78153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:01.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.941+0000 7fca7de4a700 1 -- 172.21.15.36:0/2812433998 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fca780acb00 msgr2=0x7fca780b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:01.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.941+0000 7fca7de4a700 1 --2- 172.21.15.36:0/2812433998 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fca780acb00 0x7fca780b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:01.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.941+0000 7fca7de4a700 1 -- 172.21.15.36:0/2812433998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca780acee0 msgr2=0x7fca78153ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:01.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.941+0000 7fca7de4a700 1 --2- 172.21.15.36:0/2812433998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca780acee0 0x7fca78153ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:01.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.941+0000 7fca7de4a700 1 -- 172.21.15.36:0/2812433998 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6800b1e0 con 0x7fca7813fa20 2022-06-17T12:49:01.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.941+0000 7fca7d649700 1 --2- 172.21.15.36:0/2812433998 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fca780acb00 0x7fca780b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:49:01.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.941+0000 7fca7de4a700 1 --2- 172.21.15.36:0/2812433998 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fca7813fa20 0x7fca781534d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fca70006e00 tx=0x7fca700091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:01.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.941+0000 7fca667fc700 1 -- 172.21.15.36:0/2812433998 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca7000c670 con 0x7fca7813fa20 2022-06-17T12:49:01.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.945+0000 7fca667fc700 1 -- 172.21.15.36:0/2812433998 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7000c7d0 con 0x7fca7813fa20 2022-06-17T12:49:01.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.945+0000 7fca7e64b700 1 -- 172.21.15.36:0/2812433998 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca78153030 con 0x7fca7813fa20 2022-06-17T12:49:01.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.945+0000 7fca667fc700 1 -- 172.21.15.36:0/2812433998 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca7000cb30 con 0x7fca7813fa20 2022-06-17T12:49:01.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.945+0000 7fca7e64b700 1 -- 172.21.15.36:0/2812433998 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca781539c0 con 0x7fca7813fa20 2022-06-17T12:49:01.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.945+0000 7fca7e64b700 1 -- 172.21.15.36:0/2812433998 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca781531c0 con 0x7fca7813fa20 2022-06-17T12:49:01.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.945+0000 7fca667fc700 1 -- 172.21.15.36:0/2812433998 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fca7000cc90 con 0x7fca7813fa20 2022-06-17T12:49:01.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.945+0000 7fca667fc700 1 --2- 172.21.15.36:0/2812433998 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fca580539d0 0x7fca58055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:01.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.945+0000 7fca667fc700 1 -- 172.21.15.36:0/2812433998 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6068+0+0 (secure 0 0 0) 0x7fca7006c100 con 0x7fca7813fa20 2022-06-17T12:49:01.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.945+0000 7fca7d649700 1 --2- 172.21.15.36:0/2812433998 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fca580539d0 0x7fca58055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:01.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.945+0000 7fca7d649700 1 --2- 172.21.15.36:0/2812433998 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fca580539d0 0x7fca58055e90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fca6801c000 tx=0x7fca68015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:01.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:00.949+0000 7fca667fc700 1 -- 172.21.15.36:0/2812433998 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fca70042f90 con 0x7fca7813fa20 2022-06-17T12:49:01.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.077+0000 7fca7e64b700 1 -- 172.21.15.36:0/2812433998 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fca780b6310 con 0x7fca7813fa20 2022-06-17T12:49:01.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.721+0000 7fca667fc700 1 -- 172.21.15.36:0/2812433998 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) v1 ==== 173+0+0 (secure 0 0 0) 0x7fca700465b0 con 0x7fca7813fa20 2022-06-17T12:49:01.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.729+0000 7fca7e64b700 1 -- 172.21.15.36:0/2812433998 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fca580539d0 msgr2=0x7fca58055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:01.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.729+0000 7fca7e64b700 1 --2- 172.21.15.36:0/2812433998 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fca580539d0 0x7fca58055e90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fca6801c000 tx=0x7fca68015000).stop 2022-06-17T12:49:01.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.729+0000 7fca7e64b700 1 -- 172.21.15.36:0/2812433998 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fca7813fa20 msgr2=0x7fca781534d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:01.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.729+0000 7fca7e64b700 1 --2- 172.21.15.36:0/2812433998 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fca7813fa20 0x7fca781534d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fca70006e00 tx=0x7fca700091d0).stop 2022-06-17T12:49:01.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.729+0000 7fca7e64b700 1 -- 172.21.15.36:0/2812433998 shutdown_connections 2022-06-17T12:49:01.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.729+0000 7fca7e64b700 1 --2- 172.21.15.36:0/2812433998 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fca580539d0 0x7fca58055e90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:01.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.729+0000 7fca7e64b700 1 --2- 172.21.15.36:0/2812433998 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fca7813fa20 0x7fca781534d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:01.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.729+0000 7fca7e64b700 1 --2- 172.21.15.36:0/2812433998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca780acee0 0x7fca78153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:01.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.729+0000 7fca7e64b700 1 --2- 172.21.15.36:0/2812433998 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fca780acb00 0x7fca780b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:01.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.729+0000 7fca7e64b700 1 -- 172.21.15.36:0/2812433998 >> 172.21.15.36:0/2812433998 conn(0x7fca780627f0 msgr2=0x7fca781426c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:01.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.729+0000 7fca7e64b700 1 -- 172.21.15.36:0/2812433998 shutdown_connections 2022-06-17T12:49:01.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.729+0000 7fca7e64b700 1 -- 172.21.15.36:0/2812433998 wait complete. 2022-06-17T12:49:01.740 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' removed 2022-06-17T12:49:01.740 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-06-17T12:49:01.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.873+0000 7efd827c0700 1 Processor -- start 2022-06-17T12:49:01.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.873+0000 7efd827c0700 1 -- start start 2022-06-17T12:49:01.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd827c0700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efd7c0b45c0 0x7efd7c0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:01.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd827c0700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7efd7c0b4f00 0x7efd7c0b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:01.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd827c0700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd7c0b4160 con 0x7efd7c0b9860 2022-06-17T12:49:01.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd827c0700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd7c0b9c40 con 0x7efd7c0b4f00 2022-06-17T12:49:01.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd827c0700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd7c0627d0 con 0x7efd7c0b45c0 2022-06-17T12:49:01.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd81fbf700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7efd7c0b4f00 0x7efd7c0b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:01.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd817be700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7efd7c0b9860 0x7efd7c143840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50630/0 (socket says 172.21.15.36:50630) 2022-06-17T12:49:01.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd817be700 1 -- 172.21.15.36:0/1533969373 learned_addr learned my addr 172.21.15.36:0/1533969373 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:01.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd80fbd700 1 --2- 172.21.15.36:0/1533969373 >> v2:172.21.15.70:3300/0 conn(0x7efd7c0b45c0 0x7efd7c0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:01.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd81fbf700 1 -- 172.21.15.36:0/1533969373 >> v2:172.21.15.70:3300/0 conn(0x7efd7c0b45c0 msgr2=0x7efd7c0b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:01.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd81fbf700 1 --2- 172.21.15.36:0/1533969373 >> v2:172.21.15.70:3300/0 conn(0x7efd7c0b45c0 0x7efd7c0b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:01.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd81fbf700 1 -- 172.21.15.36:0/1533969373 >> v1:172.21.15.36:6789/0 conn(0x7efd7c0b9860 legacy=0x7efd7c143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:01.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd81fbf700 1 -- 172.21.15.36:0/1533969373 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7c062a70 con 0x7efd7c0b4f00 2022-06-17T12:49:01.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd80fbd700 1 --2- 172.21.15.36:0/1533969373 >> v2:172.21.15.70:3300/0 conn(0x7efd7c0b45c0 0x7efd7c0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:49:01.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd81fbf700 1 --2- 172.21.15.36:0/1533969373 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd7c0b4f00 0x7efd7c0b5dc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7efd7c0ba5d0 tx=0x7efd74006df0).ready entity=mon.0 client_cookie=4897f1533c6a8c0d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:01.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd7bfff700 1 -- 172.21.15.36:0/1533969373 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd7400d5c0 con 0x7efd7c0b4f00 2022-06-17T12:49:01.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd7bfff700 1 -- 172.21.15.36:0/1533969373 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd74000e90 con 0x7efd7c0b4f00 2022-06-17T12:49:01.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd7bfff700 1 -- 172.21.15.36:0/1533969373 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd7400d910 con 0x7efd7c0b4f00 2022-06-17T12:49:01.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd827c0700 1 -- 172.21.15.36:0/1533969373 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd7c0b4f00 msgr2=0x7efd7c0b5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:01.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd827c0700 1 --2- 172.21.15.36:0/1533969373 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd7c0b4f00 0x7efd7c0b5dc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7efd7c0ba5d0 tx=0x7efd74006df0).stop 2022-06-17T12:49:01.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd827c0700 1 -- 172.21.15.36:0/1533969373 shutdown_connections 2022-06-17T12:49:01.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd827c0700 1 --2- 172.21.15.36:0/1533969373 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd7c0b4f00 0x7efd7c0b5dc0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:02.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd827c0700 1 --2- 172.21.15.36:0/1533969373 >> v2:172.21.15.70:3300/0 conn(0x7efd7c0b45c0 0x7efd7c0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:02.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd827c0700 1 -- 172.21.15.36:0/1533969373 >> 172.21.15.36:0/1533969373 conn(0x7efd7c05c550 msgr2=0x7efd7c060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:02.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd827c0700 1 -- 172.21.15.36:0/1533969373 shutdown_connections 2022-06-17T12:49:02.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd827c0700 1 -- 172.21.15.36:0/1533969373 wait complete. 2022-06-17T12:49:02.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd827c0700 1 Processor -- start 2022-06-17T12:49:02.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.877+0000 7efd827c0700 1 -- start start 2022-06-17T12:49:02.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd827c0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd7c0b45c0 0x7efd7c13d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:02.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd827c0700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efd7c0b4f00 0x7efd7c0ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:02.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd827c0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd7c0b9860 0x7efd7c0aea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:02.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd827c0700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c145b00 con 0x7efd7c0b45c0 2022-06-17T12:49:02.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd827c0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c13dd20 con 0x7efd7c0b9860 2022-06-17T12:49:02.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd827c0700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c13df60 con 0x7efd7c0b4f00 2022-06-17T12:49:02.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd817be700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd7c0b45c0 0x7efd7c13d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:02.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd817be700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd7c0b45c0 0x7efd7c13d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33106/0 (socket says 172.21.15.36:33106) 2022-06-17T12:49:02.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd817be700 1 -- 172.21.15.36:0/1695108003 learned_addr learned my addr 172.21.15.36:0/1695108003 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:02.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd81fbf700 1 --2- 172.21.15.36:0/1695108003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd7c0b9860 0x7efd7c0aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:02.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd80fbd700 1 --2- 172.21.15.36:0/1695108003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efd7c0b4f00 0x7efd7c0ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:02.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd817be700 1 -- 172.21.15.36:0/1695108003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efd7c0b4f00 msgr2=0x7efd7c0ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:02.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd817be700 1 --2- 172.21.15.36:0/1695108003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efd7c0b4f00 0x7efd7c0ae550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:02.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd817be700 1 -- 172.21.15.36:0/1695108003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd7c0b9860 msgr2=0x7efd7c0aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:02.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd817be700 1 --2- 172.21.15.36:0/1695108003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd7c0b9860 0x7efd7c0aea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:02.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd817be700 1 -- 172.21.15.36:0/1695108003 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7400b180 con 0x7efd7c0b45c0 2022-06-17T12:49:02.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd80fbd700 1 --2- 172.21.15.36:0/1695108003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efd7c0b4f00 0x7efd7c0ae550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:49:02.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd817be700 1 --2- 172.21.15.36:0/1695108003 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd7c0b45c0 0x7efd7c13d830 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7efd7c142ef0 tx=0x7efd6c0013c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:02.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd7a7fc700 1 -- 172.21.15.36:0/1695108003 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd6c005ba0 con 0x7efd7c0b45c0 2022-06-17T12:49:02.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd7a7fc700 1 -- 172.21.15.36:0/1695108003 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6c005d00 con 0x7efd7c0b45c0 2022-06-17T12:49:02.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd7a7fc700 1 -- 172.21.15.36:0/1695108003 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd6c0072c0 con 0x7efd7c0b45c0 2022-06-17T12:49:02.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd827c0700 1 -- 172.21.15.36:0/1695108003 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7c13e200 con 0x7efd7c0b45c0 2022-06-17T12:49:02.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd827c0700 1 -- 172.21.15.36:0/1695108003 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd7c156560 con 0x7efd7c0b45c0 2022-06-17T12:49:02.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd7a7fc700 1 -- 172.21.15.36:0/1695108003 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7efd6c010070 con 0x7efd7c0b45c0 2022-06-17T12:49:02.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd827c0700 1 -- 172.21.15.36:0/1695108003 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd7c062eb0 con 0x7efd7c0b45c0 2022-06-17T12:49:02.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd7a7fc700 1 --2- 172.21.15.36:0/1695108003 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efd5c0539b0 0x7efd5c055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:02.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd80fbd700 1 --2- 172.21.15.36:0/1695108003 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efd5c0539b0 0x7efd5c055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:02.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.881+0000 7efd7a7fc700 1 -- 172.21.15.36:0/1695108003 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5686+0+0 (secure 0 0 0) 0x7efd6c06b770 con 0x7efd7c0b45c0 2022-06-17T12:49:02.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.885+0000 7efd80fbd700 1 --2- 172.21.15.36:0/1695108003 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efd5c0539b0 0x7efd5c055e70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7efd70008000 tx=0x7efd70008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:02.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:01.885+0000 7efd7a7fc700 1 -- 172.21.15.36:0/1695108003 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efd6c042850 con 0x7efd7c0b45c0 2022-06-17T12:49:02.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.017+0000 7efd827c0700 1 -- 172.21.15.36:0/1695108003 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7efd7c156240 con 0x7efd7c0b45c0 2022-06-17T12:49:02.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.025+0000 7efd7a7fc700 1 -- 172.21.15.36:0/1695108003 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) v1 ==== 104+0+0 (secure 0 0 0) 0x7efd7c156240 con 0x7efd7c0b45c0 2022-06-17T12:49:02.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.025+0000 7efd827c0700 1 -- 172.21.15.36:0/1695108003 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efd5c0539b0 msgr2=0x7efd5c055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:02.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.025+0000 7efd827c0700 1 --2- 172.21.15.36:0/1695108003 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efd5c0539b0 0x7efd5c055e70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7efd70008000 tx=0x7efd70008040).stop 2022-06-17T12:49:02.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.025+0000 7efd827c0700 1 -- 172.21.15.36:0/1695108003 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd7c0b45c0 msgr2=0x7efd7c13d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:02.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.025+0000 7efd827c0700 1 --2- 172.21.15.36:0/1695108003 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd7c0b45c0 0x7efd7c13d830 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7efd7c142ef0 tx=0x7efd6c0013c0).stop 2022-06-17T12:49:02.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.025+0000 7efd827c0700 1 -- 172.21.15.36:0/1695108003 shutdown_connections 2022-06-17T12:49:02.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.025+0000 7efd827c0700 1 --2- 172.21.15.36:0/1695108003 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efd5c0539b0 0x7efd5c055e70 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:02.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.025+0000 7efd827c0700 1 --2- 172.21.15.36:0/1695108003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd7c0b9860 0x7efd7c0aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:02.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.025+0000 7efd827c0700 1 --2- 172.21.15.36:0/1695108003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efd7c0b4f00 0x7efd7c0ae550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:02.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.025+0000 7efd827c0700 1 --2- 172.21.15.36:0/1695108003 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd7c0b45c0 0x7efd7c13d830 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:02.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.025+0000 7efd827c0700 1 -- 172.21.15.36:0/1695108003 >> 172.21.15.36:0/1695108003 conn(0x7efd7c05c550 msgr2=0x7efd7c143490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:02.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.025+0000 7efd827c0700 1 -- 172.21.15.36:0/1695108003 shutdown_connections 2022-06-17T12:49:02.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.025+0000 7efd827c0700 1 -- 172.21.15.36:0/1695108003 wait complete. 2022-06-17T12:49:02.151 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command' 2022-06-17T12:49:02.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.297+0000 7f82c81ab700 1 Processor -- start 2022-06-17T12:49:02.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.297+0000 7f82c81ab700 1 -- start start 2022-06-17T12:49:02.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.297+0000 7f82c81ab700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f82c00ad140 0x7f82c00af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:02.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.297+0000 7f82c81ab700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f82c00ac340 0x7f82c00ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:02.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.297+0000 7f82c81ab700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c00ac870 con 0x7f82c014aca0 2022-06-17T12:49:02.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.297+0000 7f82c81ab700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c0148960 con 0x7f82c00ad140 2022-06-17T12:49:02.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.297+0000 7f82c81ab700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c00acaa0 con 0x7f82c00ac340 2022-06-17T12:49:02.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.297+0000 7f82c5f47700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f82c00ad140 0x7f82c00af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:02.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.297+0000 7f82c6748700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f82c014aca0 0x7f82c014b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50648/0 (socket says 172.21.15.36:50648) 2022-06-17T12:49:02.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.297+0000 7f82c6748700 1 -- 172.21.15.36:0/2855238820 learned_addr learned my addr 172.21.15.36:0/2855238820 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:02.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.297+0000 7f82c5746700 1 --2- 172.21.15.36:0/2855238820 >> v2:172.21.15.70:3300/0 conn(0x7f82c00ac340 0x7f82c00ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:02.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.297+0000 7f82c4f45700 1 -- 172.21.15.36:0/2855238820 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 933371334 0 0) 0x7f82c00ac870 con 0x7f82c014aca0 2022-06-17T12:49:02.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.297+0000 7f82c4f45700 1 -- 172.21.15.36:0/2855238820 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82a8003260 con 0x7f82c014aca0 2022-06-17T12:49:02.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.301+0000 7f82c5f47700 1 -- 172.21.15.36:0/2855238820 >> v2:172.21.15.70:3300/0 conn(0x7f82c00ac340 msgr2=0x7f82c00ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:02.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.301+0000 7f82c5f47700 1 --2- 172.21.15.36:0/2855238820 >> v2:172.21.15.70:3300/0 conn(0x7f82c00ac340 0x7f82c00ad520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:02.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.301+0000 7f82c4f45700 1 -- 172.21.15.36:0/2855238820 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1285153205 0 0) 0x7f82a8003260 con 0x7f82c014aca0 2022-06-17T12:49:02.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.301+0000 7f82c5f47700 1 -- 172.21.15.36:0/2855238820 >> v1:172.21.15.36:6789/0 conn(0x7f82c014aca0 legacy=0x7f82c014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:02.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.301+0000 7f82c5f47700 1 -- 172.21.15.36:0/2855238820 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c00b0bf0 con 0x7f82c00ad140 2022-06-17T12:49:02.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.301+0000 7f82c5f47700 1 --2- 172.21.15.36:0/2855238820 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f82c00ad140 0x7f82c00af7f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f82b8001b80 tx=0x7f82bc006df0).ready entity=mon.0 client_cookie=512d6f41b712074c server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:02.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.301+0000 7f82c4f45700 1 -- 172.21.15.36:0/2855238820 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82b80019e0 con 0x7f82c00ad140 2022-06-17T12:49:02.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.301+0000 7f82c4f45700 1 -- 172.21.15.36:0/2855238820 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82bc000e90 con 0x7f82c00ad140 2022-06-17T12:49:02.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.301+0000 7f82c4f45700 1 -- 172.21.15.36:0/2855238820 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82bc012460 con 0x7f82c00ad140 2022-06-17T12:49:02.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.301+0000 7f82c81ab700 1 -- 172.21.15.36:0/2855238820 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f82c00ad140 msgr2=0x7f82c00af7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:02.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.301+0000 7f82c81ab700 1 --2- 172.21.15.36:0/2855238820 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f82c00ad140 0x7f82c00af7f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f82b8001b80 tx=0x7f82bc006df0).stop 2022-06-17T12:49:02.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.301+0000 7f82c81ab700 1 -- 172.21.15.36:0/2855238820 shutdown_connections 2022-06-17T12:49:02.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.301+0000 7f82c81ab700 1 --2- 172.21.15.36:0/2855238820 >> v2:172.21.15.70:3300/0 conn(0x7f82c00ac340 0x7f82c00ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:02.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.301+0000 7f82c81ab700 1 --2- 172.21.15.36:0/2855238820 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f82c00ad140 0x7f82c00af7f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:02.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.301+0000 7f82c81ab700 1 -- 172.21.15.36:0/2855238820 >> 172.21.15.36:0/2855238820 conn(0x7f82c00b5430 msgr2=0x7f82c00b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:02.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.301+0000 7f82c81ab700 1 -- 172.21.15.36:0/2855238820 shutdown_connections 2022-06-17T12:49:02.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.301+0000 7f82c81ab700 1 -- 172.21.15.36:0/2855238820 wait complete. 2022-06-17T12:49:02.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.301+0000 7f82c81ab700 1 Processor -- start 2022-06-17T12:49:02.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.301+0000 7f82c81ab700 1 -- start start 2022-06-17T12:49:02.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.305+0000 7f82c81ab700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82c00ac340 0x7f82c00ab110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:02.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.305+0000 7f82c81ab700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f82c00ad140 0x7f82c00a8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:02.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.305+0000 7f82c81ab700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82c014aca0 0x7f82c015a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:02.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.305+0000 7f82c81ab700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c0064010 con 0x7f82c00ad140 2022-06-17T12:49:02.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.305+0000 7f82c81ab700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c0063630 con 0x7f82c014aca0 2022-06-17T12:49:02.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.305+0000 7f82c81ab700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c00638d0 con 0x7f82c00ac340 2022-06-17T12:49:02.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.305+0000 7f82c5746700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f82c00ad140 0x7f82c00a8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:02.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.305+0000 7f82c5746700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f82c00ad140 0x7f82c00a8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33122/0 (socket says 172.21.15.36:33122) 2022-06-17T12:49:02.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.305+0000 7f82c5f47700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82c00ac340 0x7f82c00ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:02.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.305+0000 7f82c5746700 1 -- 172.21.15.36:0/420448632 learned_addr learned my addr 172.21.15.36:0/420448632 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:02.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.305+0000 7f82c6748700 1 --2- 172.21.15.36:0/420448632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82c014aca0 0x7f82c015a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:02.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.305+0000 7f82c5746700 1 -- 172.21.15.36:0/420448632 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82c00ac340 msgr2=0x7f82c00ab110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:02.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.305+0000 7f82c5746700 1 --2- 172.21.15.36:0/420448632 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82c00ac340 0x7f82c00ab110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:02.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.305+0000 7f82c5746700 1 -- 172.21.15.36:0/420448632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82c014aca0 msgr2=0x7f82c015a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:02.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.305+0000 7f82c5746700 1 --2- 172.21.15.36:0/420448632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82c014aca0 0x7f82c015a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:02.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.305+0000 7f82c5746700 1 -- 172.21.15.36:0/420448632 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82bc00b180 con 0x7f82c00ad140 2022-06-17T12:49:02.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.305+0000 7f82c5746700 1 --2- 172.21.15.36:0/420448632 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f82c00ad140 0x7f82c00a8320 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f82b400c000 tx=0x7f82b4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:02.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.305+0000 7f82b2ffd700 1 -- 172.21.15.36:0/420448632 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82b4010780 con 0x7f82c00ad140 2022-06-17T12:49:02.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.305+0000 7f82c81ab700 1 -- 172.21.15.36:0/420448632 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c0063b70 con 0x7f82c00ad140 2022-06-17T12:49:02.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.305+0000 7f82b2ffd700 1 -- 172.21.15.36:0/420448632 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b40108e0 con 0x7f82c00ad140 2022-06-17T12:49:02.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.305+0000 7f82b2ffd700 1 -- 172.21.15.36:0/420448632 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82b4010c40 con 0x7f82c00ad140 2022-06-17T12:49:02.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.305+0000 7f82c81ab700 1 -- 172.21.15.36:0/420448632 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c00631b0 con 0x7f82c00ad140 2022-06-17T12:49:02.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.305+0000 7f82c81ab700 1 -- 172.21.15.36:0/420448632 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c0063d00 con 0x7f82c00ad140 2022-06-17T12:49:02.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.309+0000 7f82b2ffd700 1 -- 172.21.15.36:0/420448632 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f82b401c460 con 0x7f82c00ad140 2022-06-17T12:49:02.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.309+0000 7f82b2ffd700 1 --2- 172.21.15.36:0/420448632 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f82a0053ad0 0x7f82a0055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:02.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.309+0000 7f82b2ffd700 1 -- 172.21.15.36:0/420448632 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f82b406c150 con 0x7f82c00ad140 2022-06-17T12:49:02.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.309+0000 7f82c5f47700 1 --2- 172.21.15.36:0/420448632 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f82a0053ad0 0x7f82a0055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:02.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.309+0000 7f82c5f47700 1 --2- 172.21.15.36:0/420448632 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f82a0053ad0 0x7f82a0055f90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f82bc01a000 tx=0x7f82bc014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:02.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.317+0000 7f82b2ffd700 1 -- 172.21.15.36:0/420448632 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f82b4043130 con 0x7f82c00ad140 2022-06-17T12:49:02.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.489+0000 7f82c81ab700 1 -- 172.21.15.36:0/420448632 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]} v 0) v1 -- 0x7f82c00b1500 con 0x7f82c00ad140 2022-06-17T12:49:02.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.725+0000 7f82b2ffd700 1 -- 172.21.15.36:0/420448632 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]}]=0 v119) v1 ==== 139+0+0 (secure 0 0 0) 0x7f82b4046750 con 0x7f82c00ad140 2022-06-17T12:49:02.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.729+0000 7f82c81ab700 1 -- 172.21.15.36:0/420448632 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f82a0053ad0 msgr2=0x7f82a0055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:02.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.729+0000 7f82c81ab700 1 --2- 172.21.15.36:0/420448632 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f82a0053ad0 0x7f82a0055f90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f82bc01a000 tx=0x7f82bc014000).stop 2022-06-17T12:49:02.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.729+0000 7f82c81ab700 1 -- 172.21.15.36:0/420448632 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f82c00ad140 msgr2=0x7f82c00a8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:02.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.729+0000 7f82c81ab700 1 --2- 172.21.15.36:0/420448632 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f82c00ad140 0x7f82c00a8320 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f82b400c000 tx=0x7f82b4006df0).stop 2022-06-17T12:49:02.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.729+0000 7f82c81ab700 1 -- 172.21.15.36:0/420448632 shutdown_connections 2022-06-17T12:49:02.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.729+0000 7f82c81ab700 1 --2- 172.21.15.36:0/420448632 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f82a0053ad0 0x7f82a0055f90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:02.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.729+0000 7f82c81ab700 1 --2- 172.21.15.36:0/420448632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82c014aca0 0x7f82c015a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:02.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.729+0000 7f82c81ab700 1 --2- 172.21.15.36:0/420448632 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f82c00ad140 0x7f82c00a8320 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:02.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.729+0000 7f82c81ab700 1 --2- 172.21.15.36:0/420448632 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82c00ac340 0x7f82c00ab110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:02.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.729+0000 7f82c81ab700 1 -- 172.21.15.36:0/420448632 >> 172.21.15.36:0/420448632 conn(0x7f82c00b5430 msgr2=0x7f82c0145c70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:02.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.729+0000 7f82c81ab700 1 -- 172.21.15.36:0/420448632 shutdown_connections 2022-06-17T12:49:02.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:02.729+0000 7f82c81ab700 1 -- 172.21.15.36:0/420448632 wait complete. 2022-06-17T12:49:02.809 INFO:tasks.cephfs_test_runner:test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2022-06-17T12:49:02.842 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_dead (tasks.cephfs.test_misc.TestCacheDrop) 2022-06-17T12:49:02.859 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead' 2022-06-17T12:49:03.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.025+0000 7fe5527be700 1 Processor -- start 2022-06-17T12:49:03.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.025+0000 7fe5527be700 1 -- start start 2022-06-17T12:49:03.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe5527be700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe54c060920 0x7fe54c138ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:03.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe5527be700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe54c13fe30 0x7fe54c1394e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:03.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe5527be700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe54c0af610 con 0x7fe54c0b11a0 2022-06-17T12:49:03.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe5527be700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe54c0ae190 con 0x7fe54c060920 2022-06-17T12:49:03.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe5527be700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c0ae9d0 con 0x7fe54c13fe30 2022-06-17T12:49:03.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe551fbd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fe54c0b11a0 0x7fe54c0ab9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50662/0 (socket says 172.21.15.36:50662) 2022-06-17T12:49:03.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe551fbd700 1 -- 172.21.15.36:0/132466712 learned_addr learned my addr 172.21.15.36:0/132466712 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:03.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe5517bc700 1 --2- 172.21.15.36:0/132466712 >> v2:172.21.15.70:3300/0 conn(0x7fe54c060920 0x7fe54c138ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:03.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe550fbb700 1 --2- 172.21.15.36:0/132466712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe54c13fe30 0x7fe54c1394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:03.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe54bfff700 1 -- 172.21.15.36:0/132466712 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3914256720 0 0) 0x7fe54c0af610 con 0x7fe54c0b11a0 2022-06-17T12:49:03.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe54bfff700 1 -- 172.21.15.36:0/132466712 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe538003260 con 0x7fe54c0b11a0 2022-06-17T12:49:03.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe5517bc700 1 -- 172.21.15.36:0/132466712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe54c13fe30 msgr2=0x7fe54c1394e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:03.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe5517bc700 1 --2- 172.21.15.36:0/132466712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe54c13fe30 0x7fe54c1394e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:03.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe5517bc700 1 -- 172.21.15.36:0/132466712 >> v1:172.21.15.36:6789/0 conn(0x7fe54c0b11a0 legacy=0x7fe54c0ab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:03.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe5517bc700 1 -- 172.21.15.36:0/132466712 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54c0af8b0 con 0x7fe54c060920 2022-06-17T12:49:03.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe5517bc700 1 --2- 172.21.15.36:0/132466712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe54c060920 0x7fe54c138ff0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fe53c002f70 tx=0x7fe53c006df0).ready entity=mon.1 client_cookie=8023126bfc92a48d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:03.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe54bfff700 1 -- 172.21.15.36:0/132466712 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe53c00d4a0 con 0x7fe54c060920 2022-06-17T12:49:03.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe54bfff700 1 -- 172.21.15.36:0/132466712 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53c000e90 con 0x7fe54c060920 2022-06-17T12:49:03.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe54bfff700 1 -- 172.21.15.36:0/132466712 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe53c00d7f0 con 0x7fe54c060920 2022-06-17T12:49:03.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe5527be700 1 -- 172.21.15.36:0/132466712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe54c060920 msgr2=0x7fe54c138ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:03.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe5527be700 1 --2- 172.21.15.36:0/132466712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe54c060920 0x7fe54c138ff0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fe53c002f70 tx=0x7fe53c006df0).stop 2022-06-17T12:49:03.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe5527be700 1 -- 172.21.15.36:0/132466712 shutdown_connections 2022-06-17T12:49:03.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe5527be700 1 --2- 172.21.15.36:0/132466712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe54c13fe30 0x7fe54c1394e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:03.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe5527be700 1 --2- 172.21.15.36:0/132466712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe54c060920 0x7fe54c138ff0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:03.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe5527be700 1 -- 172.21.15.36:0/132466712 >> 172.21.15.36:0/132466712 conn(0x7fe54c0b5d10 msgr2=0x7fe54c0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:03.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe5527be700 1 -- 172.21.15.36:0/132466712 shutdown_connections 2022-06-17T12:49:03.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe5527be700 1 -- 172.21.15.36:0/132466712 wait complete. 2022-06-17T12:49:03.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe5527be700 1 Processor -- start 2022-06-17T12:49:03.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.029+0000 7fe5527be700 1 -- start start 2022-06-17T12:49:03.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.033+0000 7fe5527be700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe54c060920 0x7fe54c065000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:03.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.033+0000 7fe5527be700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe54c0b11a0 0x7fe54c1354e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:03.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.033+0000 7fe5527be700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe54c13fe30 0x7fe54c1359d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:03.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.033+0000 7fe5527be700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c132b10 con 0x7fe54c0b11a0 2022-06-17T12:49:03.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.033+0000 7fe5527be700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c132db0 con 0x7fe54c13fe30 2022-06-17T12:49:03.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.033+0000 7fe5527be700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c134090 con 0x7fe54c060920 2022-06-17T12:49:03.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.033+0000 7fe550fbb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe54c0b11a0 0x7fe54c1354e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:03.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.033+0000 7fe550fbb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe54c0b11a0 0x7fe54c1354e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33136/0 (socket says 172.21.15.36:33136) 2022-06-17T12:49:03.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.033+0000 7fe550fbb700 1 -- 172.21.15.36:0/2243741285 learned_addr learned my addr 172.21.15.36:0/2243741285 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:03.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.033+0000 7fe5517bc700 1 --2- 172.21.15.36:0/2243741285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe54c060920 0x7fe54c065000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:03.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.033+0000 7fe551fbd700 1 --2- 172.21.15.36:0/2243741285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe54c13fe30 0x7fe54c1359d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:03.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.033+0000 7fe5517bc700 1 -- 172.21.15.36:0/2243741285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe54c13fe30 msgr2=0x7fe54c1359d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:03.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.033+0000 7fe5517bc700 1 --2- 172.21.15.36:0/2243741285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe54c13fe30 0x7fe54c1359d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:03.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.033+0000 7fe5517bc700 1 -- 172.21.15.36:0/2243741285 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe54c0b11a0 msgr2=0x7fe54c1354e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:03.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.033+0000 7fe5517bc700 1 --2- 172.21.15.36:0/2243741285 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe54c0b11a0 0x7fe54c1354e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:03.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.033+0000 7fe5517bc700 1 -- 172.21.15.36:0/2243741285 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53c00b180 con 0x7fe54c060920 2022-06-17T12:49:03.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.033+0000 7fe550fbb700 1 --2- 172.21.15.36:0/2243741285 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe54c0b11a0 0x7fe54c1354e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:49:03.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.033+0000 7fe5517bc700 1 --2- 172.21.15.36:0/2243741285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe54c060920 0x7fe54c065000 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fe53c002f70 tx=0x7fe53c014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:03.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.033+0000 7fe54a7fc700 1 -- 172.21.15.36:0/2243741285 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe53c012930 con 0x7fe54c060920 2022-06-17T12:49:03.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.033+0000 7fe54a7fc700 1 -- 172.21.15.36:0/2243741285 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53c012a90 con 0x7fe54c060920 2022-06-17T12:49:03.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.033+0000 7fe5527be700 1 -- 172.21.15.36:0/2243741285 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe54c1322c0 con 0x7fe54c060920 2022-06-17T12:49:03.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.033+0000 7fe54a7fc700 1 -- 172.21.15.36:0/2243741285 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe53c012df0 con 0x7fe54c060920 2022-06-17T12:49:03.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.033+0000 7fe5527be700 1 -- 172.21.15.36:0/2243741285 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe54c065dd0 con 0x7fe54c060920 2022-06-17T12:49:03.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.033+0000 7fe5527be700 1 -- 172.21.15.36:0/2243741285 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe54c0ac780 con 0x7fe54c060920 2022-06-17T12:49:03.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.037+0000 7fe54a7fc700 1 -- 172.21.15.36:0/2243741285 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe53c01c070 con 0x7fe54c060920 2022-06-17T12:49:03.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.037+0000 7fe54a7fc700 1 --2- 172.21.15.36:0/2243741285 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe52c0539d0 0x7fe52c055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:03.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.037+0000 7fe54a7fc700 1 -- 172.21.15.36:0/2243741285 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fe53c06c7f0 con 0x7fe54c060920 2022-06-17T12:49:03.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.037+0000 7fe550fbb700 1 --2- 172.21.15.36:0/2243741285 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe52c0539d0 0x7fe52c055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:03.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.037+0000 7fe550fbb700 1 --2- 172.21.15.36:0/2243741285 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe52c0539d0 0x7fe52c055e90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fe54000b000 tx=0x7fe5400091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:03.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.045+0000 7fe54a7fc700 1 -- 172.21.15.36:0/2243741285 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe53c043740 con 0x7fe54c060920 2022-06-17T12:49:03.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.213+0000 7fe5527be700 1 -- 172.21.15.36:0/2243741285 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]} v 0) v1 -- 0x7fe54c057410 con 0x7fe54c060920 2022-06-17T12:49:03.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.733+0000 7fe54a7fc700 1 -- 172.21.15.36:0/2243741285 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]}]=0 v120) v1 ==== 147+0+0 (secure 0 0 0) 0x7fe53c019070 con 0x7fe54c060920 2022-06-17T12:49:03.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.737+0000 7fe5527be700 1 -- 172.21.15.36:0/2243741285 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe52c0539d0 msgr2=0x7fe52c055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:03.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.737+0000 7fe5527be700 1 --2- 172.21.15.36:0/2243741285 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe52c0539d0 0x7fe52c055e90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fe54000b000 tx=0x7fe5400091d0).stop 2022-06-17T12:49:03.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.737+0000 7fe5527be700 1 -- 172.21.15.36:0/2243741285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe54c060920 msgr2=0x7fe54c065000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:03.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.737+0000 7fe5527be700 1 --2- 172.21.15.36:0/2243741285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe54c060920 0x7fe54c065000 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fe53c002f70 tx=0x7fe53c014000).stop 2022-06-17T12:49:03.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.737+0000 7fe5527be700 1 -- 172.21.15.36:0/2243741285 shutdown_connections 2022-06-17T12:49:03.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.737+0000 7fe5527be700 1 --2- 172.21.15.36:0/2243741285 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe52c0539d0 0x7fe52c055e90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:03.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.737+0000 7fe5527be700 1 --2- 172.21.15.36:0/2243741285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe54c13fe30 0x7fe54c1359d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:03.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.737+0000 7fe5527be700 1 --2- 172.21.15.36:0/2243741285 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe54c0b11a0 0x7fe54c1354e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:03.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.737+0000 7fe5527be700 1 --2- 172.21.15.36:0/2243741285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe54c060920 0x7fe54c065000 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:03.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.737+0000 7fe5527be700 1 -- 172.21.15.36:0/2243741285 >> 172.21.15.36:0/2243741285 conn(0x7fe54c0b5d10 msgr2=0x7fe54c0b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:03.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.737+0000 7fe5527be700 1 -- 172.21.15.36:0/2243741285 shutdown_connections 2022-06-17T12:49:03.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.737+0000 7fe5527be700 1 -- 172.21.15.36:0/2243741285 wait complete. 2022-06-17T12:49:03.800 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-06-17T12:49:04.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.945+0000 7f5cb48dc700 1 Processor -- start 2022-06-17T12:49:04.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.945+0000 7f5cb48dc700 1 -- start start 2022-06-17T12:49:04.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.945+0000 7f5cb48dc700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5cb00b65f0 0x7f5cb00b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:04.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.945+0000 7f5cb48dc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5cb00adf10 0x7f5cb0140530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:04.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.945+0000 7f5cb48dc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5cb00b15b0 0x7f5cb00af630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:04.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.945+0000 7f5cb48dc700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cb00ae8e0 con 0x7f5cb00adf10 2022-06-17T12:49:04.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.945+0000 7f5cb48dc700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cb013fdf0 con 0x7f5cb00b65f0 2022-06-17T12:49:04.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5cb48dc700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb0140090 con 0x7f5cb00b15b0 2022-06-17T12:49:04.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5caeffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5cb00adf10 0x7f5cb0140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:04.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5caeffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5cb00adf10 0x7f5cb0140530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33142/0 (socket says 172.21.15.36:33142) 2022-06-17T12:49:04.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5caeffd700 1 -- 172.21.15.36:0/3330785679 learned_addr learned my addr 172.21.15.36:0/3330785679 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:04.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5caf7fe700 1 --2- 172.21.15.36:0/3330785679 >> v2:172.21.15.70:3300/0 conn(0x7f5cb00b65f0 0x7f5cb00b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:04.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5caffff700 1 --2- 172.21.15.36:0/3330785679 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5cb00b15b0 0x7f5cb00af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:04.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5caeffd700 1 -- 172.21.15.36:0/3330785679 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5cb00b15b0 msgr2=0x7f5cb00af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:04.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5caeffd700 1 --2- 172.21.15.36:0/3330785679 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5cb00b15b0 0x7f5cb00af630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:04.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5caeffd700 1 -- 172.21.15.36:0/3330785679 >> v2:172.21.15.70:3300/0 conn(0x7f5cb00b65f0 msgr2=0x7f5cb00b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:04.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5caeffd700 1 --2- 172.21.15.36:0/3330785679 >> v2:172.21.15.70:3300/0 conn(0x7f5cb00b65f0 0x7f5cb00b69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:04.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5caeffd700 1 -- 172.21.15.36:0/3330785679 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb0140a20 con 0x7f5cb00adf10 2022-06-17T12:49:04.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5caffff700 1 --2- 172.21.15.36:0/3330785679 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5cb00b15b0 0x7f5cb00af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:49:04.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5caeffd700 1 --2- 172.21.15.36:0/3330785679 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cb00adf10 0x7f5cb0140530 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f5c98002f70 tx=0x7f5c98006e50).ready entity=mon.0 client_cookie=cebfa0a7529d8b7b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:04.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5cae7fc700 1 -- 172.21.15.36:0/3330785679 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c9800d570 con 0x7f5cb00adf10 2022-06-17T12:49:04.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5cae7fc700 1 -- 172.21.15.36:0/3330785679 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c98000e90 con 0x7f5cb00adf10 2022-06-17T12:49:04.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5cae7fc700 1 -- 172.21.15.36:0/3330785679 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c9800d900 con 0x7f5cb00adf10 2022-06-17T12:49:04.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5cb48dc700 1 -- 172.21.15.36:0/3330785679 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cb00adf10 msgr2=0x7f5cb0140530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:04.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5cb48dc700 1 --2- 172.21.15.36:0/3330785679 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cb00adf10 0x7f5cb0140530 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f5c98002f70 tx=0x7f5c98006e50).stop 2022-06-17T12:49:04.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5cb48dc700 1 -- 172.21.15.36:0/3330785679 shutdown_connections 2022-06-17T12:49:04.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5cb48dc700 1 --2- 172.21.15.36:0/3330785679 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5cb00b15b0 0x7f5cb00af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:04.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5cb48dc700 1 --2- 172.21.15.36:0/3330785679 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cb00adf10 0x7f5cb0140530 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:04.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5cb48dc700 1 --2- 172.21.15.36:0/3330785679 >> v2:172.21.15.70:3300/0 conn(0x7f5cb00b65f0 0x7f5cb00b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:04.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5cb48dc700 1 -- 172.21.15.36:0/3330785679 >> 172.21.15.36:0/3330785679 conn(0x7f5cb00614b0 msgr2=0x7f5cb0061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:04.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5cb48dc700 1 -- 172.21.15.36:0/3330785679 shutdown_connections 2022-06-17T12:49:04.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5cb48dc700 1 -- 172.21.15.36:0/3330785679 wait complete. 2022-06-17T12:49:04.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5cb48dc700 1 Processor -- start 2022-06-17T12:49:04.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.949+0000 7f5cb48dc700 1 -- start start 2022-06-17T12:49:04.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5cb48dc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5cb00adf10 0x7f5cb0156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:04.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5cb48dc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cb00b15b0 0x7f5cb0157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:04.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5cb48dc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cb00b65f0 0x7f5cb00b3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:04.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5cb48dc700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb00645a0 con 0x7f5cb00b15b0 2022-06-17T12:49:04.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5cb48dc700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb00651e0 con 0x7f5cb00b65f0 2022-06-17T12:49:04.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5cb48dc700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb0159910 con 0x7f5cb00adf10 2022-06-17T12:49:04.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5caeffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cb00b15b0 0x7f5cb0157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:04.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5caeffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cb00b15b0 0x7f5cb0157070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33150/0 (socket says 172.21.15.36:33150) 2022-06-17T12:49:04.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5caeffd700 1 -- 172.21.15.36:0/1786145848 learned_addr learned my addr 172.21.15.36:0/1786145848 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:04.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5caffff700 1 --2- 172.21.15.36:0/1786145848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cb00b65f0 0x7f5cb00b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:04.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5caf7fe700 1 --2- 172.21.15.36:0/1786145848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5cb00adf10 0x7f5cb0156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:04.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5caeffd700 1 -- 172.21.15.36:0/1786145848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5cb00adf10 msgr2=0x7f5cb0156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:04.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5caeffd700 1 --2- 172.21.15.36:0/1786145848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5cb00adf10 0x7f5cb0156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:04.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5caeffd700 1 -- 172.21.15.36:0/1786145848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cb00b65f0 msgr2=0x7f5cb00b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:04.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5caeffd700 1 --2- 172.21.15.36:0/1786145848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cb00b65f0 0x7f5cb00b3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:04.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5caeffd700 1 -- 172.21.15.36:0/1786145848 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9800b1e0 con 0x7f5cb00b15b0 2022-06-17T12:49:04.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5caf7fe700 1 --2- 172.21.15.36:0/1786145848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5cb00adf10 0x7f5cb0156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:49:04.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5caffff700 1 --2- 172.21.15.36:0/1786145848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cb00b65f0 0x7f5cb00b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:49:04.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5caeffd700 1 --2- 172.21.15.36:0/1786145848 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cb00b15b0 0x7f5cb0157070 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f5cb0066af0 tx=0x7f5c98014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:04.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5cacff9700 1 -- 172.21.15.36:0/1786145848 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c9801e400 con 0x7f5cb00b15b0 2022-06-17T12:49:04.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5cacff9700 1 -- 172.21.15.36:0/1786145848 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9801e560 con 0x7f5cb00b15b0 2022-06-17T12:49:04.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5cb48dc700 1 -- 172.21.15.36:0/1786145848 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb00662d0 con 0x7f5cb00b15b0 2022-06-17T12:49:04.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5cacff9700 1 -- 172.21.15.36:0/1786145848 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c9801e8c0 con 0x7f5cb00b15b0 2022-06-17T12:49:04.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5cb48dc700 1 -- 172.21.15.36:0/1786145848 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb00653d0 con 0x7f5cb00b15b0 2022-06-17T12:49:04.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5cb48dc700 1 -- 172.21.15.36:0/1786145848 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cb00643c0 con 0x7f5cb00b15b0 2022-06-17T12:49:04.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.953+0000 7f5cacff9700 1 -- 172.21.15.36:0/1786145848 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f5c98025030 con 0x7f5cb00b15b0 2022-06-17T12:49:04.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.957+0000 7f5cacff9700 1 --2- 172.21.15.36:0/1786145848 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c940539b0 0x7f5c94055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:04.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.957+0000 7f5cacff9700 1 -- 172.21.15.36:0/1786145848 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f5c9806cb00 con 0x7f5cb00b15b0 2022-06-17T12:49:04.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.957+0000 7f5caf7fe700 1 --2- 172.21.15.36:0/1786145848 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c940539b0 0x7f5c94055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:04.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.957+0000 7f5caf7fe700 1 --2- 172.21.15.36:0/1786145848 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c940539b0 0x7f5c94055e70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f5ca000b000 tx=0x7f5ca0008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:04.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:03.965+0000 7f5cacff9700 1 -- 172.21.15.36:0/1786145848 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5c98043be0 con 0x7f5cb00b15b0 2022-06-17T12:49:04.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:04.089+0000 7f5cb48dc700 1 -- 172.21.15.36:0/1786145848 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5cb00b1be0 con 0x7f5cb00b15b0 2022-06-17T12:49:04.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:04.097+0000 7f5cacff9700 1 -- 172.21.15.36:0/1786145848 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v4)=0 v4) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5c98023020 con 0x7f5cb00b15b0 2022-06-17T12:49:04.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:04.101+0000 7f5cb48dc700 1 -- 172.21.15.36:0/1786145848 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c940539b0 msgr2=0x7f5c94055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:04.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:04.101+0000 7f5cb48dc700 1 --2- 172.21.15.36:0/1786145848 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c940539b0 0x7f5c94055e70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f5ca000b000 tx=0x7f5ca0008040).stop 2022-06-17T12:49:04.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:04.101+0000 7f5cb48dc700 1 -- 172.21.15.36:0/1786145848 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cb00b15b0 msgr2=0x7f5cb0157070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:04.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:04.101+0000 7f5cb48dc700 1 --2- 172.21.15.36:0/1786145848 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cb00b15b0 0x7f5cb0157070 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f5cb0066af0 tx=0x7f5c98014000).stop 2022-06-17T12:49:04.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:04.101+0000 7f5cb48dc700 1 -- 172.21.15.36:0/1786145848 shutdown_connections 2022-06-17T12:49:04.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:04.101+0000 7f5cb48dc700 1 --2- 172.21.15.36:0/1786145848 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c940539b0 0x7f5c94055e70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:04.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:04.101+0000 7f5cb48dc700 1 --2- 172.21.15.36:0/1786145848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cb00b65f0 0x7f5cb00b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:04.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:04.101+0000 7f5cb48dc700 1 --2- 172.21.15.36:0/1786145848 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cb00b15b0 0x7f5cb0157070 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:04.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:04.101+0000 7f5cb48dc700 1 --2- 172.21.15.36:0/1786145848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5cb00adf10 0x7f5cb0156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:04.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:04.101+0000 7f5cb48dc700 1 -- 172.21.15.36:0/1786145848 >> 172.21.15.36:0/1786145848 conn(0x7f5cb00614b0 msgr2=0x7f5cb013eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:04.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:04.101+0000 7f5cb48dc700 1 -- 172.21.15.36:0/1786145848 shutdown_connections 2022-06-17T12:49:04.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:04.101+0000 7f5cb48dc700 1 -- 172.21.15.36:0/1786145848 wait complete. 2022-06-17T12:49:04.245 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-17T12:49:04.268 DEBUG:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:49:04.331 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:49:04.450 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:49:04.519 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:49:04.566 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:49:04.661 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-17T12:49:04.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:49:04.663 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:49:04.664 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:49:04.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:49:04.806 INFO:teuthology.orchestra.run.smithi107.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-17T12:49:04.807 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-06-17T12:49:04.808 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:49:04.809 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:49:04.876 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:49:04.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:49:04.912 INFO:teuthology.orchestra.run.smithi136.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-17T12:49:04.913 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-17T12:49:04.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:49:04.976 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:49:04.977 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:49:05.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:49:05.155 INFO:teuthology.orchestra.run.smithi143.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:49:05.156 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-06-17T12:49:05.157 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-17T12:49:05.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.329+0000 7fcb13fff700 1 Processor -- start 2022-06-17T12:49:05.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.329+0000 7fcb13fff700 1 -- start start 2022-06-17T12:49:05.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb13fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb140b1680 0x7fcb140630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:05.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb13fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcb140ae6c0 0x7fcb1413fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:05.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb13fff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb14062c80 con 0x7fcb14060ac0 2022-06-17T12:49:05.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb13fff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb14060ea0 con 0x7fcb140ae6c0 2022-06-17T12:49:05.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb13fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb140b2f50 con 0x7fcb140b1680 2022-06-17T12:49:05.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb137fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcb140ae6c0 0x7fcb1413fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:05.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb137fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcb140ae6c0 0x7fcb1413fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33160/0 (socket says 172.21.15.36:33160) 2022-06-17T12:49:05.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb137fe700 1 -- 172.21.15.36:0/2734386832 learned_addr learned my addr 172.21.15.36:0/2734386832 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:05.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb0a7fc700 1 --2- 172.21.15.36:0/2734386832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb140b1680 0x7fcb140630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:05.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb137fe700 1 -- 172.21.15.36:0/2734386832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb140b1680 msgr2=0x7fcb140630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:05.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb137fe700 1 --2- 172.21.15.36:0/2734386832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb140b1680 0x7fcb140630e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:05.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb127fc700 1 -- 172.21.15.36:0/2734386832 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3013354212 0 0) 0x7fcb14062c80 con 0x7fcb14060ac0 2022-06-17T12:49:05.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb137fe700 1 -- 172.21.15.36:0/2734386832 >> v1:172.21.15.70:6789/0 conn(0x7fcb14060ac0 legacy=0x7fcb141431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:05.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb137fe700 1 -- 172.21.15.36:0/2734386832 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb140635d0 con 0x7fcb140ae6c0 2022-06-17T12:49:05.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb137fe700 1 --2- 172.21.15.36:0/2734386832 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb140ae6c0 0x7fcb1413fef0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7fcb0c0032f0 tx=0x7fcb0c009290).ready entity=mon.0 client_cookie=f2ab46b94ab7292d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:05.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb127fc700 1 -- 172.21.15.36:0/2734386832 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb0c00b8a0 con 0x7fcb140ae6c0 2022-06-17T12:49:05.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb127fc700 1 -- 172.21.15.36:0/2734386832 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0c00ba00 con 0x7fcb140ae6c0 2022-06-17T12:49:05.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb127fc700 1 -- 172.21.15.36:0/2734386832 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb0c00bb70 con 0x7fcb140ae6c0 2022-06-17T12:49:05.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb13fff700 1 -- 172.21.15.36:0/2734386832 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb140ae6c0 msgr2=0x7fcb1413fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:05.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb13fff700 1 --2- 172.21.15.36:0/2734386832 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb140ae6c0 0x7fcb1413fef0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7fcb0c0032f0 tx=0x7fcb0c009290).stop 2022-06-17T12:49:05.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb13fff700 1 -- 172.21.15.36:0/2734386832 shutdown_connections 2022-06-17T12:49:05.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb13fff700 1 --2- 172.21.15.36:0/2734386832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb140b1680 0x7fcb140630e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:05.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb13fff700 1 --2- 172.21.15.36:0/2734386832 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb140ae6c0 0x7fcb1413fef0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:05.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb13fff700 1 -- 172.21.15.36:0/2734386832 >> 172.21.15.36:0/2734386832 conn(0x7fcb140b64f0 msgr2=0x7fcb140b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:05.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb13fff700 1 -- 172.21.15.36:0/2734386832 shutdown_connections 2022-06-17T12:49:05.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.333+0000 7fcb13fff700 1 -- 172.21.15.36:0/2734386832 wait complete. 2022-06-17T12:49:05.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.337+0000 7fcb13fff700 1 Processor -- start 2022-06-17T12:49:05.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.337+0000 7fcb13fff700 1 -- start start 2022-06-17T12:49:05.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.337+0000 7fcb13fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb14060ac0 0x7fcb1409e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:05.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.337+0000 7fcb13fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb140ae6c0 0x7fcb14142370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:05.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.337+0000 7fcb13fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb140b1680 0x7fcb140abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:05.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.341+0000 7fcb13fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb141527a0 con 0x7fcb140ae6c0 2022-06-17T12:49:05.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.341+0000 7fcb13fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb14152a40 con 0x7fcb14060ac0 2022-06-17T12:49:05.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.341+0000 7fcb13fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb14152ce0 con 0x7fcb140b1680 2022-06-17T12:49:05.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.341+0000 7fcb0a7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb140ae6c0 0x7fcb14142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:05.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.341+0000 7fcb0a7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb140ae6c0 0x7fcb14142370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33164/0 (socket says 172.21.15.36:33164) 2022-06-17T12:49:05.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.341+0000 7fcb0a7fc700 1 -- 172.21.15.36:0/1429331326 learned_addr learned my addr 172.21.15.36:0/1429331326 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:05.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.341+0000 7fcb137fe700 1 --2- 172.21.15.36:0/1429331326 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb140b1680 0x7fcb140abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:05.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.341+0000 7fcb12ffd700 1 --2- 172.21.15.36:0/1429331326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb14060ac0 0x7fcb1409e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:05.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.341+0000 7fcb0a7fc700 1 -- 172.21.15.36:0/1429331326 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb140b1680 msgr2=0x7fcb140abde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:05.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.341+0000 7fcb0a7fc700 1 --2- 172.21.15.36:0/1429331326 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb140b1680 0x7fcb140abde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:05.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.341+0000 7fcb0a7fc700 1 -- 172.21.15.36:0/1429331326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb14060ac0 msgr2=0x7fcb1409e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:05.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.341+0000 7fcb0a7fc700 1 --2- 172.21.15.36:0/1429331326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb14060ac0 0x7fcb1409e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:05.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.341+0000 7fcb0a7fc700 1 -- 172.21.15.36:0/1429331326 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0c009530 con 0x7fcb140ae6c0 2022-06-17T12:49:05.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.341+0000 7fcb12ffd700 1 --2- 172.21.15.36:0/1429331326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb14060ac0 0x7fcb1409e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:49:05.602 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.341+0000 7fcb0a7fc700 1 --2- 172.21.15.36:0/1429331326 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb140ae6c0 0x7fcb14142370 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7fcafc00c000 tx=0x7fcafc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:05.602 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.341+0000 7fcb10ff9700 1 -- 172.21.15.36:0/1429331326 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcafc010780 con 0x7fcb140ae6c0 2022-06-17T12:49:05.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.341+0000 7fcb10ff9700 1 -- 172.21.15.36:0/1429331326 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcafc0108e0 con 0x7fcb140ae6c0 2022-06-17T12:49:05.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.341+0000 7fcb13fff700 1 -- 172.21.15.36:0/1429331326 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb14143650 con 0x7fcb140ae6c0 2022-06-17T12:49:05.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.341+0000 7fcb10ff9700 1 -- 172.21.15.36:0/1429331326 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcafc010c40 con 0x7fcb140ae6c0 2022-06-17T12:49:05.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.341+0000 7fcb13fff700 1 -- 172.21.15.36:0/1429331326 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb14152300 con 0x7fcb140ae6c0 2022-06-17T12:49:05.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.341+0000 7fcb13fff700 1 -- 172.21.15.36:0/1429331326 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb141525e0 con 0x7fcb140ae6c0 2022-06-17T12:49:05.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.345+0000 7fcb10ff9700 1 -- 172.21.15.36:0/1429331326 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fcafc01c460 con 0x7fcb140ae6c0 2022-06-17T12:49:05.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.345+0000 7fcb10ff9700 1 --2- 172.21.15.36:0/1429331326 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcaf80539b0 0x7fcaf8055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:05.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.345+0000 7fcb10ff9700 1 -- 172.21.15.36:0/1429331326 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fcafc06c290 con 0x7fcb140ae6c0 2022-06-17T12:49:05.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.345+0000 7fcb12ffd700 1 --2- 172.21.15.36:0/1429331326 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcaf80539b0 0x7fcaf8055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:05.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.345+0000 7fcb12ffd700 1 --2- 172.21.15.36:0/1429331326 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcaf80539b0 0x7fcaf8055e70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fcb04008000 tx=0x7fcb04009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:05.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.353+0000 7fcb10ff9700 1 -- 172.21.15.36:0/1429331326 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcafc043200 con 0x7fcb140ae6c0 2022-06-17T12:49:05.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.477+0000 7fcb13fff700 1 -- 172.21.15.36:0/1429331326 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fcb14152ea0 con 0x7fcb140ae6c0 2022-06-17T12:49:05.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.481+0000 7fcb10ff9700 1 -- 172.21.15.36:0/1429331326 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v38) v1 ==== 81+0+29219 (secure 0 0 0) 0x7fcafc046820 con 0x7fcb140ae6c0 2022-06-17T12:49:05.707 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:49:05.707 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:49:05.707 INFO:teuthology.orchestra.run.smithi036.stdout: "epoch": 38, 2022-06-17T12:49:05.708 INFO:teuthology.orchestra.run.smithi036.stdout: "fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", 2022-06-17T12:49:05.710 INFO:teuthology.orchestra.run.smithi036.stdout: "created": "2022-06-17T12:46:30.671408+0000", 2022-06-17T12:49:05.710 INFO:teuthology.orchestra.run.smithi036.stdout: "modified": "2022-06-17T12:49:01.708304+0000", 2022-06-17T12:49:05.711 INFO:teuthology.orchestra.run.smithi036.stdout: "last_up_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:49:05.711 INFO:teuthology.orchestra.run.smithi036.stdout: "last_in_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:49:05.712 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-17T12:49:05.712 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_num": 5799936, 2022-06-17T12:49:05.712 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_set": [ 2022-06-17T12:49:05.712 INFO:teuthology.orchestra.run.smithi036.stdout: "pglog_hardlimit", 2022-06-17T12:49:05.713 INFO:teuthology.orchestra.run.smithi036.stdout: "purged_snapdirs", 2022-06-17T12:49:05.713 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_deletes", 2022-06-17T12:49:05.713 INFO:teuthology.orchestra.run.smithi036.stdout: "sortbitwise" 2022-06-17T12:49:05.713 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:49:05.714 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_version": 6, 2022-06-17T12:49:05.714 INFO:teuthology.orchestra.run.smithi036.stdout: "full_ratio": 0.94999998807907104, 2022-06-17T12:49:05.714 INFO:teuthology.orchestra.run.smithi036.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-17T12:49:05.714 INFO:teuthology.orchestra.run.smithi036.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-17T12:49:05.716 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_snapshot": "", 2022-06-17T12:49:05.716 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_max": 6, 2022-06-17T12:49:05.716 INFO:teuthology.orchestra.run.smithi036.stdout: "max_osd": 8, 2022-06-17T12:49:05.717 INFO:teuthology.orchestra.run.smithi036.stdout: "require_min_compat_client": "jewel", 2022-06-17T12:49:05.717 INFO:teuthology.orchestra.run.smithi036.stdout: "min_compat_client": "jewel", 2022-06-17T12:49:05.740 INFO:teuthology.orchestra.run.smithi036.stdout: "require_osd_release": "octopus", 2022-06-17T12:49:05.741 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:49:05.741 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:05.742 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 1, 2022-06-17T12:49:05.742 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "device_health_metrics", 2022-06-17T12:49:05.743 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:33.881212+0000", 2022-06-17T12:49:05.743 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:49:05.743 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:49:05.743 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:49:05.744 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:49:05.744 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:49:05.744 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:49:05.745 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:49:05.745 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:49:05.745 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 1, 2022-06-17T12:49:05.745 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 1, 2022-06-17T12:49:05.746 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 1, 2022-06-17T12:49:05.747 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 1, 2022-06-17T12:49:05.747 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 1, 2022-06-17T12:49:05.747 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:49:05.748 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:49:05.748 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:49:05.748 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:49:05.748 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:49:05.749 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:49:05.749 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:49:05.749 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:05.749 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "16", 2022-06-17T12:49:05.750 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:49:05.750 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:49:05.750 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:49:05.750 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:49:05.751 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:49:05.751 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:49:05.751 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:49:05.751 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:49:05.751 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:49:05.752 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:49:05.752 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:49:05.752 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:49:05.752 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:49:05.753 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:49:05.753 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:49:05.753 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:49:05.753 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:49:05.754 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:49:05.754 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:49:05.754 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:49:05.754 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:49:05.755 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:49:05.755 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:49:05.755 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:49:05.755 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:49:05.756 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:49:05.756 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:05.757 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:49:05.757 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:49:05.758 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:49:05.758 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:49:05.758 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:49:05.758 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:49:05.759 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:49:05.759 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:49:05.759 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:49:05.764 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:49:05.764 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:49:05.764 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:49:05.765 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 1 2022-06-17T12:49:05.765 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:05.765 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:49:05.766 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr_devicehealth": {} 2022-06-17T12:49:05.772 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:05.774 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:05.780 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:05.781 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 2, 2022-06-17T12:49:05.782 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "rbd", 2022-06-17T12:49:05.782 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:44.485460+0000", 2022-06-17T12:49:05.782 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:49:05.782 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:49:05.783 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:49:05.783 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:49:05.783 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:49:05.783 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:49:05.784 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:49:05.784 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:49:05.784 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:49:05.784 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:49:05.785 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:49:05.785 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:49:05.785 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:49:05.785 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:49:05.786 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:49:05.786 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:49:05.786 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:49:05.786 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:49:05.787 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:49:05.837 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:49:05.837 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:05.838 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "19", 2022-06-17T12:49:05.838 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:49:05.838 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:49:05.838 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:49:05.839 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:49:05.839 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:49:05.839 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:49:05.839 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:49:05.840 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:49:05.840 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:49:05.840 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:49:05.840 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:49:05.840 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:49:05.841 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:49:05.841 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:49:05.841 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:49:05.841 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:49:05.853 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:49:05.853 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:49:05.854 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:49:05.854 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:49:05.854 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:49:05.855 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:49:05.855 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:49:05.875 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:49:05.890 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:49:05.891 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:49:05.891 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:05.892 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:49:05.892 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:49:05.893 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:49:05.893 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:49:05.918 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:49:05.918 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:49:05.919 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:49:05.919 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:49:05.919 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:49:05.920 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:49:05.920 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:49:05.921 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:49:05.921 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:49:05.922 INFO:teuthology.orchestra.run.smithi036.stdout: "rbd": {} 2022-06-17T12:49:05.922 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:05.922 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:05.923 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:49:05.923 INFO:teuthology.orchestra.run.smithi036.stdout: "osds": [ 2022-06-17T12:49:05.924 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:05.924 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:49:05.925 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", 2022-06-17T12:49:05.925 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:49:05.925 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:49:05.926 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:49:05.926 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:49:05.927 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:49:05.943 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:49:05.943 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:49:05.961 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 33, 2022-06-17T12:49:05.962 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:49:05.962 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:49:05.963 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:49:05.963 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:05.963 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:05.964 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:05.964 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6802", 2022-06-17T12:49:05.965 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:49:05.965 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:05.965 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:05.966 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:05.966 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6803", 2022-06-17T12:49:05.967 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:49:05.967 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:05.968 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:05.968 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:05.968 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:49:05.969 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:05.969 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.040 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:06.041 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6804", 2022-06-17T12:49:06.041 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:49:06.042 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.042 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.043 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:06.043 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6805", 2022-06-17T12:49:06.062 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:49:06.062 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:06.063 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:06.064 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.064 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:49:06.065 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:06.065 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.065 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:06.071 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6808", 2022-06-17T12:49:06.071 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:49:06.072 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.072 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.073 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:06.086 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6809", 2022-06-17T12:49:06.087 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:49:06.087 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:06.087 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:06.088 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.088 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:49:06.089 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:06.089 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.090 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:06.107 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6806", 2022-06-17T12:49:06.107 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:49:06.108 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.108 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.109 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:06.109 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6807", 2022-06-17T12:49:06.109 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:49:06.122 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:06.122 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:06.123 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.123 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6803/22619", 2022-06-17T12:49:06.124 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6805/22619", 2022-06-17T12:49:06.125 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6809/22619", 2022-06-17T12:49:06.126 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6807/22619", 2022-06-17T12:49:06.126 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:49:06.127 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:49:06.128 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:49:06.128 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:06.129 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.129 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.130 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:49:06.130 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", 2022-06-17T12:49:06.131 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:49:06.131 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:49:06.131 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:49:06.132 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:49:06.132 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:49:06.133 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:49:06.133 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:49:06.134 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 33, 2022-06-17T12:49:06.134 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:49:06.135 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:49:06.136 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:49:06.136 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:06.136 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.137 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:06.137 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6810", 2022-06-17T12:49:06.138 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:49:06.139 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.139 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.140 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:06.140 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6811", 2022-06-17T12:49:06.141 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:49:06.141 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:06.142 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:06.142 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.143 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:49:06.143 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:06.143 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.144 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:06.145 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6812", 2022-06-17T12:49:06.146 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:49:06.147 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.148 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.148 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:06.149 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6813", 2022-06-17T12:49:06.149 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:49:06.160 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:06.160 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:06.161 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.161 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:49:06.166 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:06.166 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.167 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:06.167 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6816", 2022-06-17T12:49:06.168 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:49:06.168 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.169 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.169 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:06.170 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6817", 2022-06-17T12:49:06.170 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:49:06.178 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:06.179 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:06.179 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.180 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:49:06.180 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:06.180 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.181 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:06.181 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6814", 2022-06-17T12:49:06.182 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:49:06.182 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.183 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.183 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:06.184 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6815", 2022-06-17T12:49:06.184 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:49:06.185 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:06.185 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:06.198 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.199 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6811/22620", 2022-06-17T12:49:06.199 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6813/22620", 2022-06-17T12:49:06.200 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6817/22620", 2022-06-17T12:49:06.200 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6815/22620", 2022-06-17T12:49:06.200 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:49:06.201 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:49:06.201 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:49:06.239 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:06.240 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.240 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.241 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:49:06.241 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", 2022-06-17T12:49:06.327 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:49:06.327 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:49:06.328 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:49:06.328 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:49:06.328 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:49:06.329 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:49:06.329 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:49:06.345 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 32, 2022-06-17T12:49:06.363 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:49:06.363 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:49:06.364 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:49:06.364 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:06.364 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.365 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:06.365 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6826", 2022-06-17T12:49:06.374 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:49:06.374 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.375 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.375 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:06.375 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6827", 2022-06-17T12:49:06.376 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:49:06.376 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:06.377 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:06.377 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.379 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:49:06.379 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:06.380 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.380 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:06.380 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6828", 2022-06-17T12:49:06.381 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:49:06.381 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.382 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.382 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:06.383 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6829", 2022-06-17T12:49:06.384 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:49:06.385 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:06.385 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:06.385 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.386 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:49:06.386 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:06.386 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.387 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:06.387 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6832", 2022-06-17T12:49:06.388 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:49:06.388 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.388 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.389 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:06.389 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6833", 2022-06-17T12:49:06.396 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:49:06.397 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:06.397 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:06.398 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.398 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:49:06.399 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:06.399 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.399 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:06.400 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6830", 2022-06-17T12:49:06.400 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:49:06.407 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.407 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.408 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:06.408 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6831", 2022-06-17T12:49:06.408 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:49:06.409 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:06.409 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:06.433 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.433 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6827/22643", 2022-06-17T12:49:06.434 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6829/22643", 2022-06-17T12:49:06.434 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6833/22643", 2022-06-17T12:49:06.434 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6831/22643", 2022-06-17T12:49:06.435 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:49:06.435 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:49:06.436 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:49:06.436 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:06.440 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.440 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.473 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:49:06.478 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", 2022-06-17T12:49:06.479 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:49:06.479 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:49:06.480 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:49:06.480 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:49:06.480 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:49:06.486 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:49:06.486 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:49:06.487 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 33, 2022-06-17T12:49:06.487 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:49:06.487 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:49:06.493 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:49:06.495 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:06.503 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.508 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:06.508 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6818", 2022-06-17T12:49:06.509 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:49:06.509 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.509 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.510 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:06.519 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6819", 2022-06-17T12:49:06.519 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:49:06.520 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:06.520 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:06.523 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.523 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:49:06.524 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:06.524 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.525 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:06.525 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6820", 2022-06-17T12:49:06.526 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:49:06.526 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.527 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.527 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:06.535 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6821", 2022-06-17T12:49:06.535 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:49:06.535 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:06.536 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:06.536 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.537 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:49:06.537 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:06.538 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.543 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:06.543 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6824", 2022-06-17T12:49:06.544 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:49:06.544 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.614 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.615 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:06.615 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6825", 2022-06-17T12:49:06.615 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:49:06.616 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:06.616 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:06.616 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.641 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:49:06.641 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:06.642 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.651 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:06.652 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6822", 2022-06-17T12:49:06.697 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:49:06.697 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.698 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.698 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:06.731 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6823", 2022-06-17T12:49:06.731 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:49:06.737 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:06.764 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:06.765 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.765 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6819/22646", 2022-06-17T12:49:06.774 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6821/22646", 2022-06-17T12:49:06.775 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6825/22646", 2022-06-17T12:49:06.775 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6823/22646", 2022-06-17T12:49:06.775 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:49:06.776 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:49:06.776 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:49:06.777 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:06.777 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.824 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.824 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:49:06.825 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", 2022-06-17T12:49:06.825 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:49:06.826 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:49:06.826 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:49:06.826 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:49:06.827 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:49:06.827 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:49:06.828 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:49:06.828 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 33, 2022-06-17T12:49:06.828 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:49:06.888 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:49:06.895 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:49:06.896 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:06.896 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.897 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:06.897 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6808", 2022-06-17T12:49:06.897 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:49:06.964 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.964 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.965 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:06.965 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6809", 2022-06-17T12:49:06.966 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:49:06.966 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:06.967 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:06.967 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.968 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:49:06.968 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:06.968 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.969 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:06.969 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6810", 2022-06-17T12:49:06.970 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:49:06.970 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.971 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.971 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:06.971 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6811", 2022-06-17T12:49:06.972 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:49:06.972 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:06.973 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:06.973 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.973 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:49:06.981 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:06.982 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.982 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:06.983 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6814", 2022-06-17T12:49:06.983 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:49:06.984 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.984 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.985 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:06.985 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6815", 2022-06-17T12:49:06.986 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:49:06.986 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:06.986 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:06.987 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:06.987 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:49:06.998 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:06.998 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:06.999 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:06.999 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6812", 2022-06-17T12:49:06.999 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:49:07.000 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.000 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.001 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:07.001 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6813", 2022-06-17T12:49:07.018 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:49:07.019 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:07.020 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:07.020 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.021 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6809/22812", 2022-06-17T12:49:07.022 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6811/22812", 2022-06-17T12:49:07.022 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6815/22812", 2022-06-17T12:49:07.023 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6813/22812", 2022-06-17T12:49:07.023 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:49:07.024 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:49:07.024 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:49:07.025 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:07.025 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.025 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.053 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:49:07.072 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", 2022-06-17T12:49:07.073 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:49:07.073 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:49:07.079 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:49:07.080 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:49:07.081 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:49:07.081 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:49:07.081 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:49:07.124 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 33, 2022-06-17T12:49:07.125 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:49:07.125 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:49:07.125 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:49:07.126 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:07.126 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.126 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:07.127 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6800", 2022-06-17T12:49:07.131 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:49:07.131 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.132 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.132 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:07.133 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6801", 2022-06-17T12:49:07.133 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:49:07.135 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:07.135 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:07.135 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.136 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:49:07.136 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:07.137 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.137 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:07.137 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6802", 2022-06-17T12:49:07.138 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:49:07.139 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.139 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.139 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:07.140 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6803", 2022-06-17T12:49:07.151 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:49:07.151 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:07.152 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:07.152 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.152 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:49:07.153 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:07.153 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.153 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:07.160 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6806", 2022-06-17T12:49:07.160 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:49:07.161 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.161 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.162 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:07.162 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6807", 2022-06-17T12:49:07.162 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:49:07.163 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:07.163 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:07.164 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.164 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:49:07.169 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:07.169 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.170 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:07.173 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6804", 2022-06-17T12:49:07.174 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:49:07.174 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.175 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.175 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:07.177 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6805", 2022-06-17T12:49:07.177 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:49:07.178 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:07.178 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:07.179 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.179 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6801/22813", 2022-06-17T12:49:07.180 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6803/22813", 2022-06-17T12:49:07.180 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6807/22813", 2022-06-17T12:49:07.180 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6805/22813", 2022-06-17T12:49:07.181 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:49:07.198 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:49:07.198 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:49:07.199 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:07.199 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.200 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.200 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:49:07.200 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", 2022-06-17T12:49:07.201 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:49:07.201 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:49:07.202 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:49:07.202 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:49:07.203 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:49:07.203 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:49:07.204 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:49:07.204 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 32, 2022-06-17T12:49:07.205 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:49:07.205 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:49:07.206 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:49:07.206 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:07.206 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.207 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:07.207 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6816", 2022-06-17T12:49:07.208 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:49:07.208 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.208 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.209 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:07.209 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6817", 2022-06-17T12:49:07.219 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:49:07.219 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:07.220 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:07.220 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.220 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:49:07.221 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:07.221 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.222 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:07.222 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6818", 2022-06-17T12:49:07.223 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:49:07.224 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.224 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.225 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:07.225 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6819", 2022-06-17T12:49:07.226 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:49:07.226 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:07.227 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:07.227 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.228 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:49:07.228 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:07.228 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.229 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:07.229 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6822", 2022-06-17T12:49:07.229 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:49:07.230 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.230 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.231 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:07.231 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6823", 2022-06-17T12:49:07.232 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:49:07.232 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:07.232 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:07.233 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.234 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:49:07.235 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:07.235 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.236 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:07.236 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6820", 2022-06-17T12:49:07.236 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:49:07.237 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.237 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.237 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:07.310 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6821", 2022-06-17T12:49:07.341 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:49:07.342 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:07.342 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:07.343 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.343 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6817/22822", 2022-06-17T12:49:07.344 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6819/22822", 2022-06-17T12:49:07.344 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6823/22822", 2022-06-17T12:49:07.345 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6821/22822", 2022-06-17T12:49:07.345 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:49:07.345 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:49:07.364 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:49:07.381 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:07.382 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.415 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.416 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:49:07.416 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", 2022-06-17T12:49:07.417 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:49:07.417 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:49:07.417 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:49:07.446 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:49:07.447 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:49:07.447 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:49:07.447 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:49:07.448 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 33, 2022-06-17T12:49:07.479 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:49:07.482 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:49:07.483 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:49:07.485 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:07.496 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.515 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:07.516 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6824", 2022-06-17T12:49:07.516 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:49:07.517 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.517 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.518 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:07.518 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6825", 2022-06-17T12:49:07.518 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:49:07.519 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:07.533 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:07.533 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.533 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:49:07.539 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:07.540 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.540 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:07.542 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6826", 2022-06-17T12:49:07.542 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:49:07.543 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.543 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.544 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:07.544 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6827", 2022-06-17T12:49:07.544 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:49:07.549 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:07.549 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:07.550 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.550 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:49:07.555 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:07.556 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.556 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:07.557 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6830", 2022-06-17T12:49:07.560 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:49:07.562 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.563 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.564 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:07.565 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6831", 2022-06-17T12:49:07.608 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:49:07.609 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:07.609 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:07.609 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.610 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:49:07.610 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:49:07.611 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.611 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:49:07.612 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6828", 2022-06-17T12:49:07.612 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:49:07.612 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.616 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.617 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:49:07.617 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6829", 2022-06-17T12:49:07.618 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:49:07.637 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:07.640 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:07.640 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.641 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6825/22837", 2022-06-17T12:49:07.642 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6827/22837", 2022-06-17T12:49:07.643 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6831/22837", 2022-06-17T12:49:07.643 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6829/22837", 2022-06-17T12:49:07.643 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:49:07.644 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:49:07.644 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:49:07.644 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:07.645 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:07.653 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:49:07.653 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_xinfo": [ 2022-06-17T12:49:07.654 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.822 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:49:07.823 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:49:07.823 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:49:07.824 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:49:07.824 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:49:07.825 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:49:07.825 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.291461+0000", 2022-06-17T12:49:07.877 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:49:07.878 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.879 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.879 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:49:07.879 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:49:07.880 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:49:07.880 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:49:07.880 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:49:07.881 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:49:07.891 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.537912+0000", 2022-06-17T12:49:07.891 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:49:07.891 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.892 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.892 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:49:07.892 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:49:07.893 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:49:07.893 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:49:07.929 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:49:07.929 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:49:07.932 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.606947+0000", 2022-06-17T12:49:07.933 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:49:07.933 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.933 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.933 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:49:07.943 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:49:07.943 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:49:07.943 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:49:07.947 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:49:07.947 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:49:07.947 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.514412+0000", 2022-06-17T12:49:07.947 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:49:07.948 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:07.948 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:07.949 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:49:07.949 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:49:07.974 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:49:07.975 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:49:07.975 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:49:07.976 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:49:07.998 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.394645+0000", 2022-06-17T12:49:07.999 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:49:08.000 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:08.000 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:08.000 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:49:08.001 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:49:08.001 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:49:08.001 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:49:08.002 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:49:08.002 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:49:08.003 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.343806+0000", 2022-06-17T12:49:08.003 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:49:08.003 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:08.004 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:08.004 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:49:08.005 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:49:08.007 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:49:08.007 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:49:08.008 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:49:08.008 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:49:08.008 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.591864+0000", 2022-06-17T12:49:08.009 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:49:08.032 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:08.033 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:08.033 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:49:08.034 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:49:08.034 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:49:08.035 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:49:08.035 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:49:08.036 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:49:08.036 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.607045+0000", 2022-06-17T12:49:08.036 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:49:08.037 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:08.037 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:49:08.098 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap": [], 2022-06-17T12:49:08.099 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap_items": [], 2022-06-17T12:49:08.100 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_temp": [], 2022-06-17T12:49:08.120 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_temp": [], 2022-06-17T12:49:08.121 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklist": { 2022-06-17T12:49:08.121 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.36:6835/1648255567": "2022-06-18T12:48:57.633566+0000", 2022-06-17T12:49:08.129 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.36:6834/1648255567": "2022-06-18T12:48:57.633566+0000" 2022-06-17T12:49:08.130 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:08.130 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profiles": { 2022-06-17T12:49:08.131 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs_data": { 2022-06-17T12:49:08.131 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-device-class": "", 2022-06-17T12:49:08.131 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:49:08.132 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-root": "default", 2022-06-17T12:49:08.132 INFO:teuthology.orchestra.run.smithi036.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-17T12:49:08.133 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:49:08.133 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "2", 2022-06-17T12:49:08.133 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:49:08.173 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:49:08.174 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van", 2022-06-17T12:49:08.174 INFO:teuthology.orchestra.run.smithi036.stdout: "w": "8" 2022-06-17T12:49:08.174 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:08.175 INFO:teuthology.orchestra.run.smithi036.stdout: "default": { 2022-06-17T12:49:08.175 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:49:08.176 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:49:08.176 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "1", 2022-06-17T12:49:08.176 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:49:08.177 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:49:08.177 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van" 2022-06-17T12:49:08.181 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:08.182 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:08.182 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps_queue": [], 2022-06-17T12:49:08.183 INFO:teuthology.orchestra.run.smithi036.stdout: "new_removed_snaps": [], 2022-06-17T12:49:08.183 INFO:teuthology.orchestra.run.smithi036.stdout: "new_purged_snaps": [], 2022-06-17T12:49:08.183 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_node_flags": {}, 2022-06-17T12:49:08.184 INFO:teuthology.orchestra.run.smithi036.stdout: "device_class_flags": {} 2022-06-17T12:49:08.184 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:49:08.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.481+0000 7fcb13fff700 1 -- 172.21.15.36:0/1429331326 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcaf80539b0 msgr2=0x7fcaf8055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:08.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.481+0000 7fcb13fff700 1 --2- 172.21.15.36:0/1429331326 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcaf80539b0 0x7fcaf8055e70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fcb04008000 tx=0x7fcb04009040).stop 2022-06-17T12:49:08.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.481+0000 7fcb13fff700 1 -- 172.21.15.36:0/1429331326 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb140ae6c0 msgr2=0x7fcb14142370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:08.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.481+0000 7fcb13fff700 1 --2- 172.21.15.36:0/1429331326 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb140ae6c0 0x7fcb14142370 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7fcafc00c000 tx=0x7fcafc006df0).stop 2022-06-17T12:49:08.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.481+0000 7fcb13fff700 1 -- 172.21.15.36:0/1429331326 shutdown_connections 2022-06-17T12:49:08.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.481+0000 7fcb13fff700 1 --2- 172.21.15.36:0/1429331326 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcaf80539b0 0x7fcaf8055e70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:08.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.481+0000 7fcb13fff700 1 --2- 172.21.15.36:0/1429331326 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb140b1680 0x7fcb140abde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:08.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.481+0000 7fcb13fff700 1 --2- 172.21.15.36:0/1429331326 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb140ae6c0 0x7fcb14142370 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:08.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.481+0000 7fcb13fff700 1 --2- 172.21.15.36:0/1429331326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb14060ac0 0x7fcb1409e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:08.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.481+0000 7fcb13fff700 1 -- 172.21.15.36:0/1429331326 >> 172.21.15.36:0/1429331326 conn(0x7fcb140b64f0 msgr2=0x7fcb1413e170 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:08.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.481+0000 7fcb13fff700 1 -- 172.21.15.36:0/1429331326 shutdown_connections 2022-06-17T12:49:08.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:05.481+0000 7fcb13fff700 1 -- 172.21.15.36:0/1429331326 wait complete. 2022-06-17T12:49:08.246 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:49:08.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fdd89f700 1 Processor -- start 2022-06-17T12:49:08.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fdd89f700 1 -- start start 2022-06-17T12:49:08.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fdd89f700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5fd80ad330 0x7f5fd8145070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:08.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fdd89f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5fd8066fd0 0x7f5fd813df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:08.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fdd89f700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd81417a0 con 0x7f5fd80b7c10 2022-06-17T12:49:08.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fdd89f700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd81422e0 con 0x7f5fd80ad330 2022-06-17T12:49:08.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fdd89f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd80b7ff0 con 0x7f5fd8066fd0 2022-06-17T12:49:08.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fd67fc700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5fd80ad330 0x7f5fd8145070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:08.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fd77fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5fd8066fd0 0x7f5fd813df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:08.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fd67fc700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5fd80ad330 0x7f5fd8145070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:53554/0 (socket says 172.21.15.36:53554) 2022-06-17T12:49:08.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fd6ffd700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5fd80b7c10 0x7f5fd8141090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:46434/0 (socket says 172.21.15.36:46434) 2022-06-17T12:49:08.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fd67fc700 1 -- 172.21.15.36:0/3555809731 learned_addr learned my addr 172.21.15.36:0/3555809731 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:08.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fd67fc700 1 -- 172.21.15.36:0/3555809731 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5fd8066fd0 msgr2=0x7f5fd813df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:08.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fd67fc700 1 --2- 172.21.15.36:0/3555809731 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5fd8066fd0 0x7f5fd813df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:08.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fd67fc700 1 -- 172.21.15.36:0/3555809731 >> v1:172.21.15.70:6789/0 conn(0x7f5fd80b7c10 legacy=0x7f5fd8141090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:08.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fd67fc700 1 -- 172.21.15.36:0/3555809731 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd80b30a0 con 0x7f5fd80ad330 2022-06-17T12:49:08.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fd67fc700 1 --2- 172.21.15.36:0/3555809731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd80ad330 0x7f5fd8145070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f5fc80032f0 tx=0x7f5fc8009290).ready entity=mon.1 client_cookie=b4abdaf22ab9e677 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:08.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fd5ffb700 1 -- 172.21.15.36:0/3555809731 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fc800b8a0 con 0x7f5fd80ad330 2022-06-17T12:49:08.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fd5ffb700 1 -- 172.21.15.36:0/3555809731 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc800ba00 con 0x7f5fd80ad330 2022-06-17T12:49:08.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fd5ffb700 1 -- 172.21.15.36:0/3555809731 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fc800bb70 con 0x7f5fd80ad330 2022-06-17T12:49:08.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fdd89f700 1 -- 172.21.15.36:0/3555809731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd80ad330 msgr2=0x7f5fd8145070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:08.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fdd89f700 1 --2- 172.21.15.36:0/3555809731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd80ad330 0x7f5fd8145070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f5fc80032f0 tx=0x7f5fc8009290).stop 2022-06-17T12:49:08.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fdd89f700 1 -- 172.21.15.36:0/3555809731 shutdown_connections 2022-06-17T12:49:08.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fdd89f700 1 --2- 172.21.15.36:0/3555809731 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5fd8066fd0 0x7f5fd813df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:08.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fdd89f700 1 --2- 172.21.15.36:0/3555809731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd80ad330 0x7f5fd8145070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:08.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fdd89f700 1 -- 172.21.15.36:0/3555809731 >> 172.21.15.36:0/3555809731 conn(0x7f5fd8060c60 msgr2=0x7f5fd805c550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:08.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fdd89f700 1 -- 172.21.15.36:0/3555809731 shutdown_connections 2022-06-17T12:49:08.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.373+0000 7f5fdd89f700 1 -- 172.21.15.36:0/3555809731 wait complete. 2022-06-17T12:49:08.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.377+0000 7f5fdd89f700 1 Processor -- start 2022-06-17T12:49:08.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.377+0000 7f5fdd89f700 1 -- start start 2022-06-17T12:49:08.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.377+0000 7f5fdd89f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5fd8066fd0 0x7f5fd8139ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:08.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.377+0000 7f5fdd89f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd80ad330 0x7f5fd80af300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:08.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.377+0000 7f5fdd89f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5fd80b7c10 0x7f5fd80af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:08.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.377+0000 7f5fdd89f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd80b0e30 con 0x7f5fd80b7c10 2022-06-17T12:49:08.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.377+0000 7f5fdd89f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd80b1060 con 0x7f5fd80ad330 2022-06-17T12:49:08.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.377+0000 7f5fdd89f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd813a2d0 con 0x7f5fd8066fd0 2022-06-17T12:49:08.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.377+0000 7f5fd67fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd80ad330 0x7f5fd80af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:08.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.377+0000 7f5fd77fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5fd80b7c10 0x7f5fd80af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:08.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.377+0000 7f5fd6ffd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5fd8066fd0 0x7f5fd8139ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:08.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.377+0000 7f5fd67fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd80ad330 0x7f5fd80af300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:53558/0 (socket says 172.21.15.36:53558) 2022-06-17T12:49:08.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.377+0000 7f5fd67fc700 1 -- 172.21.15.36:0/2410356263 learned_addr learned my addr 172.21.15.36:0/2410356263 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:08.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.377+0000 7f5fd67fc700 1 -- 172.21.15.36:0/2410356263 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5fd8066fd0 msgr2=0x7f5fd8139ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:08.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.377+0000 7f5fd67fc700 1 --2- 172.21.15.36:0/2410356263 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5fd8066fd0 0x7f5fd8139ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:08.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.377+0000 7f5fd67fc700 1 -- 172.21.15.36:0/2410356263 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5fd80b7c10 msgr2=0x7f5fd80af7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:08.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.377+0000 7f5fd67fc700 1 --2- 172.21.15.36:0/2410356263 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5fd80b7c10 0x7f5fd80af7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:08.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.377+0000 7f5fd67fc700 1 -- 172.21.15.36:0/2410356263 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc8009530 con 0x7f5fd80ad330 2022-06-17T12:49:08.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.381+0000 7f5fd67fc700 1 --2- 172.21.15.36:0/2410356263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd80ad330 0x7f5fd80af300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f5fd8138ef0 tx=0x7f5fc8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:08.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.381+0000 7f5fcffff700 1 -- 172.21.15.36:0/2410356263 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fc800b8a0 con 0x7f5fd80ad330 2022-06-17T12:49:08.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.381+0000 7f5fcffff700 1 -- 172.21.15.36:0/2410356263 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc8021440 con 0x7f5fd80ad330 2022-06-17T12:49:08.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.381+0000 7f5fdd89f700 1 -- 172.21.15.36:0/2410356263 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd81366b0 con 0x7f5fd80ad330 2022-06-17T12:49:08.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.381+0000 7f5fcffff700 1 -- 172.21.15.36:0/2410356263 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fc80217a0 con 0x7f5fd80ad330 2022-06-17T12:49:08.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.381+0000 7f5fdd89f700 1 -- 172.21.15.36:0/2410356263 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd80b0a30 con 0x7f5fd80ad330 2022-06-17T12:49:08.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.381+0000 7f5fdd89f700 1 -- 172.21.15.36:0/2410356263 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd80b0bf0 con 0x7f5fd80ad330 2022-06-17T12:49:08.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.381+0000 7f5fcffff700 1 -- 172.21.15.36:0/2410356263 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f5fc800f070 con 0x7f5fd80ad330 2022-06-17T12:49:08.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.381+0000 7f5fcffff700 1 --2- 172.21.15.36:0/2410356263 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5fb0053a40 0x7f5fb0055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:08.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.381+0000 7f5fd6ffd700 1 --2- 172.21.15.36:0/2410356263 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5fb0053a40 0x7f5fb0055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:08.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.381+0000 7f5fcffff700 1 -- 172.21.15.36:0/2410356263 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f5fc806cf40 con 0x7f5fd80ad330 2022-06-17T12:49:08.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.385+0000 7f5fd6ffd700 1 --2- 172.21.15.36:0/2410356263 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5fb0053a40 0x7f5fb0055f00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f5fc000d000 tx=0x7f5fc0007b80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:08.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.393+0000 7f5fcffff700 1 -- 172.21.15.36:0/2410356263 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5fc8043f50 con 0x7f5fd80ad330 2022-06-17T12:49:08.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.557+0000 7f5fdd89f700 1 -- 172.21.15.36:0/2410356263 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fd80b3340 con 0x7f5fd80ad330 2022-06-17T12:49:08.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.557+0000 7f5fcffff700 1 -- 172.21.15.36:0/2410356263 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+1209 (secure 0 0 0) 0x7f5fc8015070 con 0x7f5fd80ad330 2022-06-17T12:49:08.604 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:49:08.605 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":16,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3597082917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3597082917},{"type":"v1","addr":"172.21.15.70:6833","nonce":3597082917}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":5208,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/3806870890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3806870890},{"type":"v1","addr":"172.21.15.36:6835","nonce":3806870890}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":16}],"filesystems":[]} 2022-06-17T12:49:08.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.561+0000 7f5fdd89f700 1 -- 172.21.15.36:0/2410356263 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5fb0053a40 msgr2=0x7f5fb0055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:08.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.561+0000 7f5fdd89f700 1 --2- 172.21.15.36:0/2410356263 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5fb0053a40 0x7f5fb0055f00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f5fc000d000 tx=0x7f5fc0007b80).stop 2022-06-17T12:49:08.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.561+0000 7f5fdd89f700 1 -- 172.21.15.36:0/2410356263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd80ad330 msgr2=0x7f5fd80af300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:08.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.561+0000 7f5fdd89f700 1 --2- 172.21.15.36:0/2410356263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd80ad330 0x7f5fd80af300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f5fd8138ef0 tx=0x7f5fc8016000).stop 2022-06-17T12:49:08.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.561+0000 7f5fdd89f700 1 -- 172.21.15.36:0/2410356263 shutdown_connections 2022-06-17T12:49:08.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.561+0000 7f5fdd89f700 1 --2- 172.21.15.36:0/2410356263 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5fb0053a40 0x7f5fb0055f00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:08.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.561+0000 7f5fdd89f700 1 --2- 172.21.15.36:0/2410356263 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5fd80b7c10 0x7f5fd80af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:08.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.561+0000 7f5fdd89f700 1 --2- 172.21.15.36:0/2410356263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd80ad330 0x7f5fd80af300 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:08.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.561+0000 7f5fdd89f700 1 --2- 172.21.15.36:0/2410356263 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5fd8066fd0 0x7f5fd8139ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:08.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.561+0000 7f5fdd89f700 1 -- 172.21.15.36:0/2410356263 >> 172.21.15.36:0/2410356263 conn(0x7f5fd8060c60 msgr2=0x7f5fd813d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:08.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.561+0000 7f5fdd89f700 1 -- 172.21.15.36:0/2410356263 shutdown_connections 2022-06-17T12:49:08.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.561+0000 7f5fdd89f700 1 -- 172.21.15.36:0/2410356263 wait complete. 2022-06-17T12:49:08.654 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 16 2022-06-17T12:49:08.655 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:49:08.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.817+0000 7fa5b24c6700 1 Processor -- start 2022-06-17T12:49:08.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.817+0000 7fa5b24c6700 1 -- start start 2022-06-17T12:49:08.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.817+0000 7fa5b24c6700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa5ac062580 0x7fa5ac14b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:08.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.817+0000 7fa5b24c6700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5ac0b35d0 con 0x7fa5ac064e60 2022-06-17T12:49:08.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.817+0000 7fa5b24c6700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5ac067250 con 0x7fa5ac0b7b60 2022-06-17T12:49:08.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.817+0000 7fa5b24c6700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5ac0629f0 con 0x7fa5ac062580 2022-06-17T12:49:08.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.817+0000 7fa5b0a63700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fa5ac064e60 0x7fa5ac0652f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50718/0 (socket says 172.21.15.36:50718) 2022-06-17T12:49:08.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.817+0000 7fa5b0a63700 1 -- 172.21.15.36:0/2340451021 learned_addr learned my addr 172.21.15.36:0/2340451021 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:08.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.817+0000 7fa5ab7fe700 1 --2- 172.21.15.36:0/2340451021 >> v2:172.21.15.70:3300/0 conn(0x7fa5ac062580 0x7fa5ac14b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:08.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.817+0000 7fa5aaffd700 1 -- 172.21.15.36:0/2340451021 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4042322126 0 0) 0x7fa5ac0b35d0 con 0x7fa5ac064e60 2022-06-17T12:49:08.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.817+0000 7fa5aaffd700 1 -- 172.21.15.36:0/2340451021 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa58c003260 con 0x7fa5ac064e60 2022-06-17T12:49:08.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.817+0000 7fa5aaffd700 1 -- 172.21.15.36:0/2340451021 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200513067 0 0) 0x7fa5ac067250 con 0x7fa5ac0b7b60 2022-06-17T12:49:08.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.817+0000 7fa5aaffd700 1 -- 172.21.15.36:0/2340451021 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5ac0b35d0 con 0x7fa5ac0b7b60 2022-06-17T12:49:08.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.817+0000 7fa5ab7fe700 1 -- 172.21.15.36:0/2340451021 >> v1:172.21.15.70:6789/0 conn(0x7fa5ac0b7b60 legacy=0x7fa5ac14afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:08.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.817+0000 7fa5aaffd700 1 -- 172.21.15.36:0/2340451021 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1459660194 0 0) 0x7fa58c003260 con 0x7fa5ac064e60 2022-06-17T12:49:08.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.817+0000 7fa5ab7fe700 1 -- 172.21.15.36:0/2340451021 >> v1:172.21.15.36:6789/0 conn(0x7fa5ac064e60 legacy=0x7fa5ac0652f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:08.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.817+0000 7fa5ab7fe700 1 -- 172.21.15.36:0/2340451021 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ac0674f0 con 0x7fa5ac062580 2022-06-17T12:49:08.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.817+0000 7fa5ab7fe700 1 --2- 172.21.15.36:0/2340451021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5ac062580 0x7fa5ac14b6d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fa5a4001110 tx=0x7fa594006df0).ready entity=mon.1 client_cookie=3feff6ca46aa09d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:08.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.817+0000 7fa5aaffd700 1 -- 172.21.15.36:0/2340451021 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5a4001660 con 0x7fa5ac062580 2022-06-17T12:49:08.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.817+0000 7fa5aaffd700 1 -- 172.21.15.36:0/2340451021 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa594000e90 con 0x7fa5ac062580 2022-06-17T12:49:08.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.817+0000 7fa5aaffd700 1 -- 172.21.15.36:0/2340451021 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa594012460 con 0x7fa5ac062580 2022-06-17T12:49:08.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.821+0000 7fa5b24c6700 1 -- 172.21.15.36:0/2340451021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5ac062580 msgr2=0x7fa5ac14b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:08.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.821+0000 7fa5b24c6700 1 --2- 172.21.15.36:0/2340451021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5ac062580 0x7fa5ac14b6d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fa5a4001110 tx=0x7fa594006df0).stop 2022-06-17T12:49:08.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.821+0000 7fa5b24c6700 1 -- 172.21.15.36:0/2340451021 shutdown_connections 2022-06-17T12:49:08.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.821+0000 7fa5b24c6700 1 --2- 172.21.15.36:0/2340451021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5ac062580 0x7fa5ac14b6d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:08.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.821+0000 7fa5b24c6700 1 -- 172.21.15.36:0/2340451021 >> 172.21.15.36:0/2340451021 conn(0x7fa5ac060c60 msgr2=0x7fa5ac05c550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:08.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.821+0000 7fa5b24c6700 1 -- 172.21.15.36:0/2340451021 shutdown_connections 2022-06-17T12:49:08.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.821+0000 7fa5b24c6700 1 -- 172.21.15.36:0/2340451021 wait complete. 2022-06-17T12:49:08.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.821+0000 7fa5b24c6700 1 Processor -- start 2022-06-17T12:49:08.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.821+0000 7fa5b24c6700 1 -- start start 2022-06-17T12:49:08.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.821+0000 7fa5b24c6700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5ac062580 0x7fa5ac0b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:08.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.825+0000 7fa5b24c6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5ac064e60 0x7fa5ac0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:08.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.825+0000 7fa5b24c6700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa5ac0b7b60 0x7fa5ac0ad330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:08.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.825+0000 7fa5b24c6700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac0af870 con 0x7fa5ac0b7b60 2022-06-17T12:49:08.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.825+0000 7fa5b24c6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac0afb10 con 0x7fa5ac064e60 2022-06-17T12:49:08.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.825+0000 7fa5b24c6700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac0b04a0 con 0x7fa5ac062580 2022-06-17T12:49:08.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.825+0000 7fa5b0a63700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa5ac0b7b60 0x7fa5ac0ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:08.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.825+0000 7fa5b0a63700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa5ac0b7b60 0x7fa5ac0ad330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33194/0 (socket says 172.21.15.36:33194) 2022-06-17T12:49:08.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.825+0000 7fa5b0a63700 1 -- 172.21.15.36:0/1058153111 learned_addr learned my addr 172.21.15.36:0/1058153111 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:08.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.825+0000 7fa5abfff700 1 --2- 172.21.15.36:0/1058153111 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5ac062580 0x7fa5ac0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:08.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.825+0000 7fa5ab7fe700 1 --2- 172.21.15.36:0/1058153111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5ac064e60 0x7fa5ac0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:08.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.825+0000 7fa5b0a63700 1 -- 172.21.15.36:0/1058153111 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5ac062580 msgr2=0x7fa5ac0b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:08.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.825+0000 7fa5b0a63700 1 --2- 172.21.15.36:0/1058153111 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5ac062580 0x7fa5ac0b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:08.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.825+0000 7fa5b0a63700 1 -- 172.21.15.36:0/1058153111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5ac064e60 msgr2=0x7fa5ac0affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:08.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.825+0000 7fa5b0a63700 1 --2- 172.21.15.36:0/1058153111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5ac064e60 0x7fa5ac0affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:08.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.825+0000 7fa5b0a63700 1 -- 172.21.15.36:0/1058153111 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59400b180 con 0x7fa5ac0b7b60 2022-06-17T12:49:08.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.825+0000 7fa5b0a63700 1 --2- 172.21.15.36:0/1058153111 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa5ac0b7b60 0x7fa5ac0ad330 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fa5a4000fe0 tx=0x7fa5a4001720).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:08.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.825+0000 7fa5a97fa700 1 -- 172.21.15.36:0/1058153111 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5a4007170 con 0x7fa5ac0b7b60 2022-06-17T12:49:08.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.825+0000 7fa5b24c6700 1 -- 172.21.15.36:0/1058153111 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5ac0b1290 con 0x7fa5ac0b7b60 2022-06-17T12:49:08.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.825+0000 7fa5a97fa700 1 -- 172.21.15.36:0/1058153111 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a40072d0 con 0x7fa5ac0b7b60 2022-06-17T12:49:08.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.825+0000 7fa5a97fa700 1 -- 172.21.15.36:0/1058153111 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5a4004710 con 0x7fa5ac0b7b60 2022-06-17T12:49:08.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.825+0000 7fa5b24c6700 1 -- 172.21.15.36:0/1058153111 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5ac149fe0 con 0x7fa5ac0b7b60 2022-06-17T12:49:08.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.825+0000 7fa5b24c6700 1 -- 172.21.15.36:0/1058153111 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5ac0afd10 con 0x7fa5ac0b7b60 2022-06-17T12:49:08.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.829+0000 7fa5a97fa700 1 -- 172.21.15.36:0/1058153111 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fa5a401b070 con 0x7fa5ac0b7b60 2022-06-17T12:49:08.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.829+0000 7fa5a97fa700 1 --2- 172.21.15.36:0/1058153111 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa590053a40 0x7fa590055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:08.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.829+0000 7fa5a97fa700 1 -- 172.21.15.36:0/1058153111 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fa5a406b320 con 0x7fa5ac0b7b60 2022-06-17T12:49:08.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.829+0000 7fa5abfff700 1 --2- 172.21.15.36:0/1058153111 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa590053a40 0x7fa590055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:08.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.829+0000 7fa5abfff700 1 --2- 172.21.15.36:0/1058153111 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa590053a40 0x7fa590055f00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fa59c00d000 tx=0x7fa59c007da0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:08.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:08.837+0000 7fa5a97fa700 1 -- 172.21.15.36:0/1058153111 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa5ac0afd10 con 0x7fa5ac0b7b60 2022-06-17T12:49:09.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.005+0000 7fa5b24c6700 1 -- 172.21.15.36:0/1058153111 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5ac0afd10 con 0x7fa5ac0b7b60 2022-06-17T12:49:09.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.005+0000 7fa5a97fa700 1 -- 172.21.15.36:0/1058153111 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+1209 (secure 0 0 0) 0x7fa5ac0afd10 con 0x7fa5ac0b7b60 2022-06-17T12:49:09.015 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:49:09.017 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":16,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3597082917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3597082917},{"type":"v1","addr":"172.21.15.70:6833","nonce":3597082917}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":5208,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/3806870890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3806870890},{"type":"v1","addr":"172.21.15.36:6835","nonce":3806870890}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":16}],"filesystems":[]} 2022-06-17T12:49:09.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.005+0000 7fa5b24c6700 1 -- 172.21.15.36:0/1058153111 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa590053a40 msgr2=0x7fa590055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:09.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.005+0000 7fa5b24c6700 1 --2- 172.21.15.36:0/1058153111 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa590053a40 0x7fa590055f00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fa59c00d000 tx=0x7fa59c007da0).stop 2022-06-17T12:49:09.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.005+0000 7fa5b24c6700 1 -- 172.21.15.36:0/1058153111 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa5ac0b7b60 msgr2=0x7fa5ac0ad330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:09.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.005+0000 7fa5b24c6700 1 --2- 172.21.15.36:0/1058153111 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa5ac0b7b60 0x7fa5ac0ad330 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fa5a4000fe0 tx=0x7fa5a4001720).stop 2022-06-17T12:49:09.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.005+0000 7fa5b24c6700 1 -- 172.21.15.36:0/1058153111 shutdown_connections 2022-06-17T12:49:09.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.005+0000 7fa5b24c6700 1 --2- 172.21.15.36:0/1058153111 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa590053a40 0x7fa590055f00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:09.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.005+0000 7fa5b24c6700 1 --2- 172.21.15.36:0/1058153111 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa5ac0b7b60 0x7fa5ac0ad330 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:09.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.005+0000 7fa5b24c6700 1 --2- 172.21.15.36:0/1058153111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5ac064e60 0x7fa5ac0affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:09.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.005+0000 7fa5b24c6700 1 --2- 172.21.15.36:0/1058153111 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5ac062580 0x7fa5ac0b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:09.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.005+0000 7fa5b24c6700 1 -- 172.21.15.36:0/1058153111 >> 172.21.15.36:0/1058153111 conn(0x7fa5ac060c60 msgr2=0x7fa5ac0b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:09.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.009+0000 7fa5b24c6700 1 -- 172.21.15.36:0/1058153111 shutdown_connections 2022-06-17T12:49:09.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.009+0000 7fa5b24c6700 1 -- 172.21.15.36:0/1058153111 wait complete. 2022-06-17T12:49:09.061 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 16 2022-06-17T12:49:09.061 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-17T12:49:09.062 INFO:tasks.ceph.mds.a:Stopping old one... 2022-06-17T12:49:09.062 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-17T12:49:09.062 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T12:49:09.063 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-17T12:49:09.063 INFO:tasks.ceph.mds.b:Stopping old one... 2022-06-17T12:49:09.064 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-17T12:49:09.064 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T12:49:09.101 INFO:tasks.ceph.mds.b:Stopped 2022-06-17T12:49:09.102 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-17T12:49:09.103 INFO:tasks.ceph.mds.a:Stopped 2022-06-17T12:49:09.104 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-17T12:49:09.159 INFO:tasks.ceph.mds.a:Started 2022-06-17T12:49:09.160 INFO:tasks.ceph.mds.b:Started 2022-06-17T12:49:09.160 DEBUG:teuthology.parallel:result is None 2022-06-17T12:49:09.180 DEBUG:teuthology.parallel:result is None 2022-06-17T12:49:09.181 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-06-17T12:49:09.234 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2022-06-17T12:49:09.258 INFO:tasks.ceph.mds.a.smithi036.stdout:starting mds.a at 2022-06-17T12:49:09.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.309+0000 7f0ed2ef1700 1 Processor -- start 2022-06-17T12:49:09.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.309+0000 7f0ed2ef1700 1 -- start start 2022-06-17T12:49:09.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.309+0000 7f0ed2ef1700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0ecc064f70 0x7f0ecc0b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:09.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.309+0000 7f0ed2ef1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ecc14a8f0 0x7f0ecc146790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:09.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.309+0000 7f0ed2ef1700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ecc063b10 con 0x7f0ecc063730 2022-06-17T12:49:09.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.309+0000 7f0ed2ef1700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ecc14ad20 con 0x7f0ecc064f70 2022-06-17T12:49:09.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.309+0000 7f0ed2ef1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0b2e80 con 0x7f0ecc14a8f0 2022-06-17T12:49:09.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.313+0000 7f0ed16ee700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0ecc064f70 0x7f0ecc0b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:09.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.313+0000 7f0ed1eef700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f0ecc063730 0x7f0ecc064860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50740/0 (socket says 172.21.15.36:50740) 2022-06-17T12:49:09.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.313+0000 7f0ed1eef700 1 -- 172.21.15.36:0/327382642 learned_addr learned my addr 172.21.15.36:0/327382642 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:09.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.313+0000 7f0ed26f0700 1 --2- 172.21.15.36:0/327382642 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ecc14a8f0 0x7f0ecc146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:09.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.313+0000 7f0ed16ee700 1 -- 172.21.15.36:0/327382642 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ecc14a8f0 msgr2=0x7f0ecc146790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:09.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.313+0000 7f0ed16ee700 1 --2- 172.21.15.36:0/327382642 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ecc14a8f0 0x7f0ecc146790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:09.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.313+0000 7f0ed16ee700 1 -- 172.21.15.36:0/327382642 >> v1:172.21.15.36:6789/0 conn(0x7f0ecc063730 legacy=0x7f0ecc064860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:09.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.313+0000 7f0ed16ee700 1 -- 172.21.15.36:0/327382642 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc05ffb0 con 0x7f0ecc064f70 2022-06-17T12:49:09.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.313+0000 7f0ed0eed700 1 -- 172.21.15.36:0/327382642 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2943451711 0 0) 0x7f0ecc063b10 con 0x7f0ecc063730 2022-06-17T12:49:09.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.313+0000 7f0ed16ee700 1 --2- 172.21.15.36:0/327382642 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ecc064f70 0x7f0ecc0b3bf0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f0ec8002f70 tx=0x7f0ec8006df0).ready entity=mon.0 client_cookie=3cc9d07bd806797f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:09.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.313+0000 7f0ed0eed700 1 -- 172.21.15.36:0/327382642 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ec800d570 con 0x7f0ecc064f70 2022-06-17T12:49:09.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.313+0000 7f0ed0eed700 1 -- 172.21.15.36:0/327382642 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec8000e90 con 0x7f0ecc064f70 2022-06-17T12:49:09.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.313+0000 7f0ed0eed700 1 -- 172.21.15.36:0/327382642 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ec800d8c0 con 0x7f0ecc064f70 2022-06-17T12:49:09.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.313+0000 7f0ed2ef1700 1 -- 172.21.15.36:0/327382642 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ecc064f70 msgr2=0x7f0ecc0b3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:09.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.313+0000 7f0ed2ef1700 1 --2- 172.21.15.36:0/327382642 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ecc064f70 0x7f0ecc0b3bf0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f0ec8002f70 tx=0x7f0ec8006df0).stop 2022-06-17T12:49:09.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.313+0000 7f0ed2ef1700 1 -- 172.21.15.36:0/327382642 shutdown_connections 2022-06-17T12:49:09.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.313+0000 7f0ed2ef1700 1 --2- 172.21.15.36:0/327382642 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ecc14a8f0 0x7f0ecc146790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:09.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.313+0000 7f0ed2ef1700 1 --2- 172.21.15.36:0/327382642 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ecc064f70 0x7f0ecc0b3bf0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:09.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.313+0000 7f0ed2ef1700 1 -- 172.21.15.36:0/327382642 >> 172.21.15.36:0/327382642 conn(0x7f0ecc0614b0 msgr2=0x7f0ecc0618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:09.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.313+0000 7f0ed2ef1700 1 -- 172.21.15.36:0/327382642 shutdown_connections 2022-06-17T12:49:09.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.313+0000 7f0ed2ef1700 1 -- 172.21.15.36:0/327382642 wait complete. 2022-06-17T12:49:09.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.313+0000 7f0ed2ef1700 1 Processor -- start 2022-06-17T12:49:09.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.313+0000 7f0ed2ef1700 1 -- start start 2022-06-17T12:49:09.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ed2ef1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ecc063730 0x7f0ecc147be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:09.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ed2ef1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ecc064f70 0x7f0ecc15a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:09.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ed2ef1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ecc14a8f0 0x7f0ecc156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:09.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ed2ef1700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0af520 con 0x7f0ecc064f70 2022-06-17T12:49:09.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ed2ef1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0b09f0 con 0x7f0ecc14a8f0 2022-06-17T12:49:09.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ed2ef1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc159d80 con 0x7f0ecc063730 2022-06-17T12:49:09.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ed16ee700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ecc064f70 0x7f0ecc15a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:09.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ed1eef700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ecc063730 0x7f0ecc147be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:09.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ed16ee700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ecc064f70 0x7f0ecc15a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33218/0 (socket says 172.21.15.36:33218) 2022-06-17T12:49:09.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ed16ee700 1 -- 172.21.15.36:0/2530295314 learned_addr learned my addr 172.21.15.36:0/2530295314 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:09.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ed26f0700 1 --2- 172.21.15.36:0/2530295314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ecc14a8f0 0x7f0ecc156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:09.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ed1eef700 1 --2- 172.21.15.36:0/2530295314 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ecc063730 0x7f0ecc147be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:48006/0 (socket says 172.21.15.36:48006) 2022-06-17T12:49:09.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ed1eef700 1 -- 172.21.15.36:0/2530295314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ecc14a8f0 msgr2=0x7f0ecc156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:09.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ed1eef700 1 --2- 172.21.15.36:0/2530295314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ecc14a8f0 0x7f0ecc156630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:09.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ed1eef700 1 -- 172.21.15.36:0/2530295314 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ecc064f70 msgr2=0x7f0ecc15a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:09.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ed1eef700 1 --2- 172.21.15.36:0/2530295314 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ecc064f70 0x7f0ecc15a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:09.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ed1eef700 1 -- 172.21.15.36:0/2530295314 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec800b180 con 0x7f0ecc063730 2022-06-17T12:49:09.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ed16ee700 1 --2- 172.21.15.36:0/2530295314 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ecc064f70 0x7f0ecc15a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:49:09.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ed1eef700 1 --2- 172.21.15.36:0/2530295314 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ecc063730 0x7f0ecc147be0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f0ebc000ed0 tx=0x7f0ebc007a20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:09.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ebaffd700 1 -- 172.21.15.36:0/2530295314 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ebc00ce80 con 0x7f0ecc063730 2022-06-17T12:49:09.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ebaffd700 1 -- 172.21.15.36:0/2530295314 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc0070d0 con 0x7f0ecc063730 2022-06-17T12:49:09.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ed2ef1700 1 -- 172.21.15.36:0/2530295314 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ecc15a020 con 0x7f0ecc063730 2022-06-17T12:49:09.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ed2ef1700 1 -- 172.21.15.36:0/2530295314 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ecc15a9b0 con 0x7f0ecc063730 2022-06-17T12:49:09.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ebaffd700 1 -- 172.21.15.36:0/2530295314 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ebc007430 con 0x7f0ecc063730 2022-06-17T12:49:09.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.317+0000 7f0ed2ef1700 1 -- 172.21.15.36:0/2530295314 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ecc15a1b0 con 0x7f0ecc063730 2022-06-17T12:49:09.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.321+0000 7f0ebaffd700 1 -- 172.21.15.36:0/2530295314 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f0ebc01a060 con 0x7f0ecc063730 2022-06-17T12:49:09.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.321+0000 7f0ebaffd700 1 --2- 172.21.15.36:0/2530295314 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0eac053a60 0x7f0eac055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:09.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.321+0000 7f0ed16ee700 1 --2- 172.21.15.36:0/2530295314 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0eac053a60 0x7f0eac055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:09.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.321+0000 7f0ebaffd700 1 -- 172.21.15.36:0/2530295314 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f0ebc06b3b0 con 0x7f0ecc063730 2022-06-17T12:49:09.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.321+0000 7f0ed16ee700 1 --2- 172.21.15.36:0/2530295314 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0eac053a60 0x7f0eac055f20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f0ec801c000 tx=0x7f0ec8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:09.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.329+0000 7f0ebaffd700 1 -- 172.21.15.36:0/2530295314 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0ebc0423c0 con 0x7f0ecc063730 2022-06-17T12:49:09.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:09.469+0000 7f0ed2ef1700 1 -- 172.21.15.36:0/2530295314 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f0ecc057410 con 0x7f0ecc063730 2022-06-17T12:49:10.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.297+0000 7f0ebaffd700 1 -- 172.21.15.36:0/2530295314 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v41) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0ebc018070 con 0x7f0ecc063730 2022-06-17T12:49:10.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.301+0000 7f0ed2ef1700 1 -- 172.21.15.36:0/2530295314 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0eac053a60 msgr2=0x7f0eac055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:10.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.301+0000 7f0ed2ef1700 1 --2- 172.21.15.36:0/2530295314 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0eac053a60 0x7f0eac055f20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f0ec801c000 tx=0x7f0ec8015000).stop 2022-06-17T12:49:10.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.301+0000 7f0ed2ef1700 1 -- 172.21.15.36:0/2530295314 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ecc063730 msgr2=0x7f0ecc147be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:10.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.301+0000 7f0ed2ef1700 1 --2- 172.21.15.36:0/2530295314 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ecc063730 0x7f0ecc147be0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f0ebc000ed0 tx=0x7f0ebc007a20).stop 2022-06-17T12:49:10.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.301+0000 7f0ed2ef1700 1 -- 172.21.15.36:0/2530295314 shutdown_connections 2022-06-17T12:49:10.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.301+0000 7f0ed2ef1700 1 --2- 172.21.15.36:0/2530295314 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0eac053a60 0x7f0eac055f20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:10.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.301+0000 7f0ed2ef1700 1 --2- 172.21.15.36:0/2530295314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ecc14a8f0 0x7f0ecc156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:10.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.301+0000 7f0ed2ef1700 1 --2- 172.21.15.36:0/2530295314 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ecc064f70 0x7f0ecc15a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:10.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.301+0000 7f0ed2ef1700 1 --2- 172.21.15.36:0/2530295314 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ecc063730 0x7f0ecc147be0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:10.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.301+0000 7f0ed2ef1700 1 -- 172.21.15.36:0/2530295314 >> 172.21.15.36:0/2530295314 conn(0x7f0ecc0614b0 msgr2=0x7f0ecc061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:10.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.301+0000 7f0ed2ef1700 1 -- 172.21.15.36:0/2530295314 shutdown_connections 2022-06-17T12:49:10.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.301+0000 7f0ed2ef1700 1 -- 172.21.15.36:0/2530295314 wait complete. 2022-06-17T12:49:10.347 INFO:teuthology.orchestra.run.smithi036.stderr: removed all blacklist entries 2022-06-17T12:49:10.348 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2022-06-17T12:49:10.349 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-06-17T12:49:10.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.537+0000 7f57a1fd3700 1 Processor -- start 2022-06-17T12:49:10.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.537+0000 7f57a1fd3700 1 -- start start 2022-06-17T12:49:10.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.537+0000 7f57a1fd3700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f579c14b2b0 0x7f579c060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:10.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.537+0000 7f57a1fd3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f579c147bd0 0x7f579c1442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:10.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.537+0000 7f57a1fd3700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f579c14b870 con 0x7f579c0b27b0 2022-06-17T12:49:10.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.537+0000 7f57a1fd3700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f579c0ac500 con 0x7f579c14b2b0 2022-06-17T12:49:10.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.537+0000 7f57a1fd3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579c0ace90 con 0x7f579c147bd0 2022-06-17T12:49:10.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.537+0000 7f579bfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f579c14b2b0 0x7f579c060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:10.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.537+0000 7f57a0fd1700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f579c0b27b0 0x7f579c0ac780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50756/0 (socket says 172.21.15.36:50756) 2022-06-17T12:49:10.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.537+0000 7f57a0fd1700 1 -- 172.21.15.36:0/2080307208 learned_addr learned my addr 172.21.15.36:0/2080307208 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:10.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.537+0000 7f57a17d2700 1 --2- 172.21.15.36:0/2080307208 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f579c147bd0 0x7f579c1442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:10.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.541+0000 7f579bfff700 1 -- 172.21.15.36:0/2080307208 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f579c147bd0 msgr2=0x7f579c1442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:10.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.541+0000 7f579bfff700 1 --2- 172.21.15.36:0/2080307208 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f579c147bd0 0x7f579c1442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:10.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.541+0000 7f579bfff700 1 -- 172.21.15.36:0/2080307208 >> v1:172.21.15.36:6789/0 conn(0x7f579c0b27b0 legacy=0x7f579c0ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:10.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.541+0000 7f579b7fe700 1 -- 172.21.15.36:0/2080307208 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 91943892 0 0) 0x7f579c14b870 con 0x7f579c0b27b0 2022-06-17T12:49:10.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.541+0000 7f579bfff700 1 -- 172.21.15.36:0/2080307208 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c0b2b90 con 0x7f579c14b2b0 2022-06-17T12:49:10.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.541+0000 7f579bfff700 1 --2- 172.21.15.36:0/2080307208 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f579c14b2b0 0x7f579c060ca0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f578c0032f0 tx=0x7f578c0091d0).ready entity=mon.0 client_cookie=981ba7699eabc890 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:10.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.541+0000 7f579b7fe700 1 -- 172.21.15.36:0/2080307208 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f578c00b6c0 con 0x7f579c14b2b0 2022-06-17T12:49:10.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.541+0000 7f579b7fe700 1 -- 172.21.15.36:0/2080307208 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578c00b820 con 0x7f579c14b2b0 2022-06-17T12:49:10.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.541+0000 7f579b7fe700 1 -- 172.21.15.36:0/2080307208 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f578c00b990 con 0x7f579c14b2b0 2022-06-17T12:49:10.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.541+0000 7f57a1fd3700 1 -- 172.21.15.36:0/2080307208 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f579c14b2b0 msgr2=0x7f579c060ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:10.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.541+0000 7f57a1fd3700 1 --2- 172.21.15.36:0/2080307208 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f579c14b2b0 0x7f579c060ca0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f578c0032f0 tx=0x7f578c0091d0).stop 2022-06-17T12:49:10.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.541+0000 7f57a1fd3700 1 -- 172.21.15.36:0/2080307208 shutdown_connections 2022-06-17T12:49:10.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.541+0000 7f57a1fd3700 1 --2- 172.21.15.36:0/2080307208 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f579c147bd0 0x7f579c1442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:10.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.541+0000 7f57a1fd3700 1 --2- 172.21.15.36:0/2080307208 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f579c14b2b0 0x7f579c060ca0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:10.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.541+0000 7f57a1fd3700 1 -- 172.21.15.36:0/2080307208 >> 172.21.15.36:0/2080307208 conn(0x7f579c0b5d10 msgr2=0x7f579c0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:10.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.541+0000 7f57a1fd3700 1 -- 172.21.15.36:0/2080307208 shutdown_connections 2022-06-17T12:49:10.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.541+0000 7f57a1fd3700 1 -- 172.21.15.36:0/2080307208 wait complete. 2022-06-17T12:49:10.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.545+0000 7f57a1fd3700 1 Processor -- start 2022-06-17T12:49:10.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.545+0000 7f57a1fd3700 1 -- start start 2022-06-17T12:49:10.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.545+0000 7f57a1fd3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f579c0b27b0 0x7f579c064e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:10.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.545+0000 7f57a1fd3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f579c147bd0 0x7f579c063050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:10.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.545+0000 7f57a1fd3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f579c14b2b0 0x7f579c063540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:10.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.545+0000 7f57a1fd3700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579c064750 con 0x7f579c14b2b0 2022-06-17T12:49:10.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.545+0000 7f57a1fd3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579c14a110 con 0x7f579c147bd0 2022-06-17T12:49:10.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.545+0000 7f57a1fd3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579c065320 con 0x7f579c0b27b0 2022-06-17T12:49:10.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.545+0000 7f57a17d2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f579c14b2b0 0x7f579c063540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:10.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.545+0000 7f57a17d2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f579c14b2b0 0x7f579c063540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33236/0 (socket says 172.21.15.36:33236) 2022-06-17T12:49:10.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.545+0000 7f57a17d2700 1 -- 172.21.15.36:0/3981072749 learned_addr learned my addr 172.21.15.36:0/3981072749 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:10.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.545+0000 7f579bfff700 1 --2- 172.21.15.36:0/3981072749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f579c147bd0 0x7f579c063050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:10.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.545+0000 7f57a0fd1700 1 --2- 172.21.15.36:0/3981072749 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f579c0b27b0 0x7f579c064e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:10.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.545+0000 7f57a17d2700 1 -- 172.21.15.36:0/3981072749 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f579c0b27b0 msgr2=0x7f579c064e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:10.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.545+0000 7f57a17d2700 1 --2- 172.21.15.36:0/3981072749 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f579c0b27b0 0x7f579c064e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:10.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.545+0000 7f57a17d2700 1 -- 172.21.15.36:0/3981072749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f579c147bd0 msgr2=0x7f579c063050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:10.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.545+0000 7f57a17d2700 1 --2- 172.21.15.36:0/3981072749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f579c147bd0 0x7f579c063050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:10.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.545+0000 7f57a17d2700 1 -- 172.21.15.36:0/3981072749 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c009470 con 0x7f579c14b2b0 2022-06-17T12:49:10.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.545+0000 7f57a17d2700 1 --2- 172.21.15.36:0/3981072749 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f579c14b2b0 0x7f579c063540 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f579400c000 tx=0x7f5794006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:10.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.549+0000 7f5799ffb700 1 -- 172.21.15.36:0/3981072749 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5794011750 con 0x7f579c14b2b0 2022-06-17T12:49:10.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.549+0000 7f5799ffb700 1 -- 172.21.15.36:0/3981072749 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57940118b0 con 0x7f579c14b2b0 2022-06-17T12:49:10.602 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.549+0000 7f57a1fd3700 1 -- 172.21.15.36:0/3981072749 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f579c062500 con 0x7f579c14b2b0 2022-06-17T12:49:10.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.549+0000 7f5799ffb700 1 -- 172.21.15.36:0/3981072749 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5794011c10 con 0x7f579c14b2b0 2022-06-17T12:49:10.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.549+0000 7f57a1fd3700 1 -- 172.21.15.36:0/3981072749 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f579c063a30 con 0x7f579c14b2b0 2022-06-17T12:49:10.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.549+0000 7f57a1fd3700 1 -- 172.21.15.36:0/3981072749 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f579c14a2d0 con 0x7f579c14b2b0 2022-06-17T12:49:10.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.549+0000 7f5799ffb700 1 -- 172.21.15.36:0/3981072749 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f5794011d70 con 0x7f579c14b2b0 2022-06-17T12:49:10.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.549+0000 7f5799ffb700 1 --2- 172.21.15.36:0/3981072749 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f577c0539d0 0x7f577c055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:10.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.549+0000 7f5799ffb700 1 -- 172.21.15.36:0/3981072749 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f579406bfa0 con 0x7f579c14b2b0 2022-06-17T12:49:10.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.549+0000 7f57a0fd1700 1 --2- 172.21.15.36:0/3981072749 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f577c0539d0 0x7f577c055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:10.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.549+0000 7f57a0fd1700 1 --2- 172.21.15.36:0/3981072749 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f577c0539d0 0x7f577c055e90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f579000d000 tx=0x7f5790007a20).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:10.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.553+0000 7f5799ffb700 1 -- 172.21.15.36:0/3981072749 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f57940430e0 con 0x7f579c14b2b0 2022-06-17T12:49:10.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.721+0000 7f57a1fd3700 1 -- 172.21.15.36:0/3981072749 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f579c14a7d0 con 0x7f579c14b2b0 2022-06-17T12:49:10.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.721+0000 7f5799ffb700 1 -- 172.21.15.36:0/3981072749 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f5794046700 con 0x7f579c14b2b0 2022-06-17T12:49:10.749 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:49:10.750 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:49:10.751 INFO:teuthology.orchestra.run.smithi036.stdout: "auth_dump": [ 2022-06-17T12:49:10.752 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:10.752 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mds.a", 2022-06-17T12:49:10.753 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiyaUgARAAxp7bzOl4uVo2EsaoRBeMRw==", 2022-06-17T12:49:10.753 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:49:10.754 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:49:10.754 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:49:10.755 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:49:10.756 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:49:10.757 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:10.757 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:10.758 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:10.758 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mds.b", 2022-06-17T12:49:10.759 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xifyxLCRAABI389jmz1zboeDanzraT8Q==", 2022-06-17T12:49:10.759 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:49:10.760 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:49:10.760 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:49:10.761 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:49:10.761 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:49:10.762 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:10.763 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:10.763 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:10.764 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.0", 2022-06-17T12:49:10.764 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB+d6xibhXGNBAA6AuUT9ITWAZjDFpNsgA4Wg==", 2022-06-17T12:49:10.765 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:49:10.765 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:49:10.765 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:49:10.767 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:49:10.768 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:10.768 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:10.769 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:10.769 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.1", 2022-06-17T12:49:10.774 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCBd6xiOIPvGxAADkrBoJCy9x91zan6TWU0tQ==", 2022-06-17T12:49:10.774 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:49:10.775 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:49:10.775 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:49:10.776 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:49:10.776 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:10.842 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:10.843 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:10.843 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.2", 2022-06-17T12:49:10.843 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCEd6xiP4/9AhAAaP3cWXV6vbfGBPfqSLKqUg==", 2022-06-17T12:49:10.844 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:49:10.844 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:49:10.845 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:49:10.845 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:49:10.847 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:10.848 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:10.850 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:10.850 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.3", 2022-06-17T12:49:10.851 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCGd6xi+LZgJxAAJjGwP5qW6TFDZfR+2rz6ig==", 2022-06-17T12:49:10.851 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:49:10.860 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:49:10.861 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:49:10.861 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:49:10.862 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:10.862 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:10.887 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:10.888 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.4", 2022-06-17T12:49:10.888 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCKd6xikvrTChAAXIyWSETswDv+tb3zsGyYpA==", 2022-06-17T12:49:10.889 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:49:10.889 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:49:10.890 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:49:10.890 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:49:10.891 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:10.891 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:10.892 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:10.892 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.5", 2022-06-17T12:49:10.892 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCMd6xiqr2QLhAArhimHuNsVWrDGGreFYxj6A==", 2022-06-17T12:49:10.893 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:49:10.893 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:49:10.894 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:49:10.895 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:49:10.895 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:10.896 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:10.896 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:10.897 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.6", 2022-06-17T12:49:10.898 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCPd6xiUXcXGxAAdrYz8txdBWMIqF5DF4Aimw==", 2022-06-17T12:49:10.898 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:49:10.902 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:49:10.903 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:49:10.903 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:49:10.903 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:10.904 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:10.904 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:10.905 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.7", 2022-06-17T12:49:10.912 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCSd6xiBJ2IAhAAUkNfgmlxxWY7+VouPLAvyw==", 2022-06-17T12:49:10.913 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:49:10.914 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:49:10.914 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:49:10.915 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:49:10.934 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:10.935 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:10.935 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:10.936 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.0", 2022-06-17T12:49:10.936 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiXO3QFBAAsSLtQ0yKKZxhqUT/eYKdtg==", 2022-06-17T12:49:10.937 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:49:10.937 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:49:10.945 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:49:10.946 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:49:10.947 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:10.948 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:10.948 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:10.949 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.1", 2022-06-17T12:49:10.949 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xi51KdGBAAi7fGZ+Qepg6j3DAhlzoWxw==", 2022-06-17T12:49:10.949 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:49:10.950 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:49:10.950 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:49:10.951 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:49:10.951 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:10.952 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:10.952 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:10.953 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.2", 2022-06-17T12:49:10.953 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiug2hHBAAqHpLgwMdthSWtJ8Vwu+m1Q==", 2022-06-17T12:49:10.954 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:49:10.964 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:49:10.965 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:49:10.965 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:49:10.965 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:10.966 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:10.966 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:10.967 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.admin", 2022-06-17T12:49:10.968 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB5d6xiGDODKhAAPZMG/M2nVssdr4FJHcS6XQ==", 2022-06-17T12:49:10.968 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:49:10.969 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:49:10.969 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:49:10.970 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:49:10.971 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:49:10.971 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:10.971 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:10.972 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:10.972 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mgr.x", 2022-06-17T12:49:10.973 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB6d6xiyH9uORAATJb0chHE1CH0X02w7edVtw==", 2022-06-17T12:49:10.973 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:49:10.974 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:49:10.974 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile mgr", 2022-06-17T12:49:10.975 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:49:10.975 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:10.975 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:10.976 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:10.977 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mgr.y", 2022-06-17T12:49:10.977 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB6d6xiHXghMxAAzpsVRCbLWHWZPtrPk1jFlA==", 2022-06-17T12:49:10.978 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:49:10.978 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:49:10.979 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile mgr", 2022-06-17T12:49:10.979 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:49:10.979 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:10.980 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:10.980 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:49:10.981 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:49:10.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.725+0000 7f57a1fd3700 1 -- 172.21.15.36:0/3981072749 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f577c0539d0 msgr2=0x7f577c055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:10.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.725+0000 7f57a1fd3700 1 --2- 172.21.15.36:0/3981072749 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f577c0539d0 0x7f577c055e90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f579000d000 tx=0x7f5790007a20).stop 2022-06-17T12:49:10.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.725+0000 7f57a1fd3700 1 -- 172.21.15.36:0/3981072749 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f579c14b2b0 msgr2=0x7f579c063540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:10.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.725+0000 7f57a1fd3700 1 --2- 172.21.15.36:0/3981072749 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f579c14b2b0 0x7f579c063540 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f579400c000 tx=0x7f5794006df0).stop 2022-06-17T12:49:10.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.725+0000 7f57a1fd3700 1 -- 172.21.15.36:0/3981072749 shutdown_connections 2022-06-17T12:49:10.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.725+0000 7f57a1fd3700 1 --2- 172.21.15.36:0/3981072749 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f577c0539d0 0x7f577c055e90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:10.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.725+0000 7f57a1fd3700 1 --2- 172.21.15.36:0/3981072749 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f579c14b2b0 0x7f579c063540 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:10.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.725+0000 7f57a1fd3700 1 --2- 172.21.15.36:0/3981072749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f579c147bd0 0x7f579c063050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:10.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.725+0000 7f57a1fd3700 1 --2- 172.21.15.36:0/3981072749 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f579c0b27b0 0x7f579c064e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:10.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.725+0000 7f57a1fd3700 1 -- 172.21.15.36:0/3981072749 >> 172.21.15.36:0/3981072749 conn(0x7f579c0b5d10 msgr2=0x7f579c146790 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:10.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.725+0000 7f57a1fd3700 1 -- 172.21.15.36:0/3981072749 shutdown_connections 2022-06-17T12:49:10.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:10.725+0000 7f57a1fd3700 1 -- 172.21.15.36:0/3981072749 wait complete. 2022-06-17T12:49:10.990 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:49:11.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.117+0000 7f17c759e700 1 Processor -- start 2022-06-17T12:49:11.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.117+0000 7f17c759e700 1 -- start start 2022-06-17T12:49:11.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.117+0000 7f17c759e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f17c813cb70 0x7f17c8143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:11.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.117+0000 7f17c759e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f17c813cf50 0x7f17c81442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:11.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.117+0000 7f17c759e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c80af4b0 con 0x7f17c80aef60 2022-06-17T12:49:11.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.117+0000 7f17c759e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c80ba500 con 0x7f17c813cb70 2022-06-17T12:49:11.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.117+0000 7f17c759e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c8142f50 con 0x7f17c813cf50 2022-06-17T12:49:11.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.117+0000 7f17c5d9b700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f17c80aef60 0x7f17c80b9df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50770/0 (socket says 172.21.15.36:50770) 2022-06-17T12:49:11.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.117+0000 7f17c5d9b700 1 -- 172.21.15.36:0/3033802178 learned_addr learned my addr 172.21.15.36:0/3033802178 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:11.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.117+0000 7f17c659c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f17c813cb70 0x7f17c8143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:11.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.117+0000 7f17c6d9d700 1 --2- 172.21.15.36:0/3033802178 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f17c813cf50 0x7f17c81442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:11.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.117+0000 7f17c559a700 1 -- 172.21.15.36:0/3033802178 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4035432432 0 0) 0x7f17c80af4b0 con 0x7f17c80aef60 2022-06-17T12:49:11.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.117+0000 7f17c559a700 1 -- 172.21.15.36:0/3033802178 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17ac003260 con 0x7f17c80aef60 2022-06-17T12:49:11.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.117+0000 7f17c659c700 1 -- 172.21.15.36:0/3033802178 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f17c813cf50 msgr2=0x7f17c81442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:11.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.117+0000 7f17c659c700 1 --2- 172.21.15.36:0/3033802178 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f17c813cf50 0x7f17c81442a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.117+0000 7f17c659c700 1 -- 172.21.15.36:0/3033802178 >> v1:172.21.15.36:6789/0 conn(0x7f17c80aef60 legacy=0x7f17c80b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:11.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.117+0000 7f17c659c700 1 -- 172.21.15.36:0/3033802178 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c81431f0 con 0x7f17c813cb70 2022-06-17T12:49:11.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.117+0000 7f17c659c700 1 --2- 172.21.15.36:0/3033802178 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f17c813cb70 0x7f17c8143690 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f17b8002f70 tx=0x7f17b8006df0).ready entity=mon.0 client_cookie=3f54503b736e67f5 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:11.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.117+0000 7f17c559a700 1 -- 172.21.15.36:0/3033802178 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17b800d4a0 con 0x7f17c813cb70 2022-06-17T12:49:11.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.117+0000 7f17c559a700 1 -- 172.21.15.36:0/3033802178 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b8000e90 con 0x7f17c813cb70 2022-06-17T12:49:11.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.117+0000 7f17c559a700 1 -- 172.21.15.36:0/3033802178 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17b800d7f0 con 0x7f17c813cb70 2022-06-17T12:49:11.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c759e700 1 -- 172.21.15.36:0/3033802178 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f17c813cb70 msgr2=0x7f17c8143690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:11.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c759e700 1 --2- 172.21.15.36:0/3033802178 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f17c813cb70 0x7f17c8143690 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f17b8002f70 tx=0x7f17b8006df0).stop 2022-06-17T12:49:11.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c759e700 1 -- 172.21.15.36:0/3033802178 shutdown_connections 2022-06-17T12:49:11.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c759e700 1 --2- 172.21.15.36:0/3033802178 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f17c813cf50 0x7f17c81442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c759e700 1 --2- 172.21.15.36:0/3033802178 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f17c813cb70 0x7f17c8143690 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c759e700 1 -- 172.21.15.36:0/3033802178 >> 172.21.15.36:0/3033802178 conn(0x7f17c805c550 msgr2=0x7f17c80b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:11.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c759e700 1 -- 172.21.15.36:0/3033802178 shutdown_connections 2022-06-17T12:49:11.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c759e700 1 -- 172.21.15.36:0/3033802178 wait complete. 2022-06-17T12:49:11.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c759e700 1 Processor -- start 2022-06-17T12:49:11.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c759e700 1 -- start start 2022-06-17T12:49:11.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c759e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17c80aef60 0x7f17c813b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:11.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c759e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f17c813cb70 0x7f17c813fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:11.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c759e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f17c813cf50 0x7f17c813ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:11.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c759e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c80b1360 con 0x7f17c813cf50 2022-06-17T12:49:11.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c759e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c813de80 con 0x7f17c80aef60 2022-06-17T12:49:11.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c759e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c80b0fe0 con 0x7f17c813cb70 2022-06-17T12:49:11.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c6d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f17c813cf50 0x7f17c813ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:11.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c6d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f17c813cf50 0x7f17c813ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33250/0 (socket says 172.21.15.36:33250) 2022-06-17T12:49:11.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c6d9d700 1 -- 172.21.15.36:0/4116441786 learned_addr learned my addr 172.21.15.36:0/4116441786 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:11.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c659c700 1 --2- 172.21.15.36:0/4116441786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17c80aef60 0x7f17c813b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:11.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c5d9b700 1 --2- 172.21.15.36:0/4116441786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f17c813cb70 0x7f17c813fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:11.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c6d9d700 1 -- 172.21.15.36:0/4116441786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f17c813cb70 msgr2=0x7f17c813fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:11.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c6d9d700 1 --2- 172.21.15.36:0/4116441786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f17c813cb70 0x7f17c813fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c6d9d700 1 -- 172.21.15.36:0/4116441786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17c80aef60 msgr2=0x7f17c813b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:11.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c6d9d700 1 --2- 172.21.15.36:0/4116441786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17c80aef60 0x7f17c813b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.121+0000 7f17c6d9d700 1 -- 172.21.15.36:0/4116441786 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b800b180 con 0x7f17c813cf50 2022-06-17T12:49:11.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.125+0000 7f17c6d9d700 1 --2- 172.21.15.36:0/4116441786 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f17c813cf50 0x7f17c813ffa0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f17c0006e00 tx=0x7f17c00091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:11.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.125+0000 7f17b77fe700 1 -- 172.21.15.36:0/4116441786 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17c0012600 con 0x7f17c813cf50 2022-06-17T12:49:11.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.125+0000 7f17b77fe700 1 -- 172.21.15.36:0/4116441786 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c0012760 con 0x7f17c813cf50 2022-06-17T12:49:11.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.125+0000 7f17c759e700 1 -- 172.21.15.36:0/4116441786 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c813f4c0 con 0x7f17c813cf50 2022-06-17T12:49:11.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.125+0000 7f17b77fe700 1 -- 172.21.15.36:0/4116441786 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17c0012ac0 con 0x7f17c813cf50 2022-06-17T12:49:11.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.125+0000 7f17c759e700 1 -- 172.21.15.36:0/4116441786 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c8140630 con 0x7f17c813cf50 2022-06-17T12:49:11.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.125+0000 7f17b77fe700 1 -- 172.21.15.36:0/4116441786 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f17c0022070 con 0x7f17c813cf50 2022-06-17T12:49:11.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.125+0000 7f17c759e700 1 -- 172.21.15.36:0/4116441786 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c813e200 con 0x7f17c813cf50 2022-06-17T12:49:11.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.125+0000 7f17b77fe700 1 --2- 172.21.15.36:0/4116441786 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f17a4053a40 0x7f17a4055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:11.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.125+0000 7f17c659c700 1 --2- 172.21.15.36:0/4116441786 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f17a4053a40 0x7f17a4055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:11.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.125+0000 7f17b77fe700 1 -- 172.21.15.36:0/4116441786 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f17c006c380 con 0x7f17c813cf50 2022-06-17T12:49:11.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.125+0000 7f17c659c700 1 --2- 172.21.15.36:0/4116441786 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f17a4053a40 0x7f17a4055f00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f17b801a000 tx=0x7f17b8014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:11.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.133+0000 7f17b77fe700 1 -- 172.21.15.36:0/4116441786 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f17c0043330 con 0x7f17c813cf50 2022-06-17T12:49:11.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.261+0000 7f17c759e700 1 -- 172.21.15.36:0/4116441786 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17c8057410 con 0x7f17c813cf50 2022-06-17T12:49:11.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.265+0000 7f17b77fe700 1 -- 172.21.15.36:0/4116441786 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+13414 (secure 0 0 0) 0x7f17c0016090 con 0x7f17c813cf50 2022-06-17T12:49:11.298 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:49:11.299 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":41,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:49:10.284507+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:49:11.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.265+0000 7f17c759e700 1 -- 172.21.15.36:0/4116441786 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f17a4053a40 msgr2=0x7f17a4055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:11.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.265+0000 7f17c759e700 1 --2- 172.21.15.36:0/4116441786 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f17a4053a40 0x7f17a4055f00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f17b801a000 tx=0x7f17b8014000).stop 2022-06-17T12:49:11.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.265+0000 7f17c759e700 1 -- 172.21.15.36:0/4116441786 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f17c813cf50 msgr2=0x7f17c813ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:11.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.265+0000 7f17c759e700 1 --2- 172.21.15.36:0/4116441786 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f17c813cf50 0x7f17c813ffa0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f17c0006e00 tx=0x7f17c00091d0).stop 2022-06-17T12:49:11.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.265+0000 7f17c759e700 1 -- 172.21.15.36:0/4116441786 shutdown_connections 2022-06-17T12:49:11.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.265+0000 7f17c759e700 1 --2- 172.21.15.36:0/4116441786 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f17a4053a40 0x7f17a4055f00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.265+0000 7f17c759e700 1 --2- 172.21.15.36:0/4116441786 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f17c813cf50 0x7f17c813ffa0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.265+0000 7f17c759e700 1 --2- 172.21.15.36:0/4116441786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f17c813cb70 0x7f17c813fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.265+0000 7f17c759e700 1 --2- 172.21.15.36:0/4116441786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17c80aef60 0x7f17c813b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.265+0000 7f17c759e700 1 -- 172.21.15.36:0/4116441786 >> 172.21.15.36:0/4116441786 conn(0x7f17c805c550 msgr2=0x7f17c813c050 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:11.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.265+0000 7f17c759e700 1 -- 172.21.15.36:0/4116441786 shutdown_connections 2022-06-17T12:49:11.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.265+0000 7f17c759e700 1 -- 172.21.15.36:0/4116441786 wait complete. 2022-06-17T12:49:11.310 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-17T12:49:11.312 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:49:11.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.477+0000 7f6b9f9e1700 1 Processor -- start 2022-06-17T12:49:11.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.477+0000 7f6b9f9e1700 1 -- start start 2022-06-17T12:49:11.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.477+0000 7f6b9f9e1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b980b2350 0x7f6b980b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:11.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.477+0000 7f6b9f9e1700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6b98143e60 0x7f6b98140300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:11.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.477+0000 7f6b9f9e1700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b980aa170 con 0x7f6b980b2d80 2022-06-17T12:49:11.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.477+0000 7f6b9f9e1700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b980a9c30 con 0x7f6b98143e60 2022-06-17T12:49:11.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.477+0000 7f6b9f9e1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b980a9ed0 con 0x7f6b980b2350 2022-06-17T12:49:11.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.477+0000 7f6b9d77d700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f6b980b2d80 0x7f6b98143750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50784/0 (socket says 172.21.15.36:50784) 2022-06-17T12:49:11.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.477+0000 7f6b9d77d700 1 -- 172.21.15.36:0/3688149108 learned_addr learned my addr 172.21.15.36:0/3688149108 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:11.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.477+0000 7f6b9df7e700 1 --2- 172.21.15.36:0/3688149108 >> v2:172.21.15.70:3300/0 conn(0x7f6b98143e60 0x7f6b98140300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:11.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.477+0000 7f6b9cf7c700 1 --2- 172.21.15.36:0/3688149108 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b980b2350 0x7f6b980b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:11.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.477+0000 7f6b97fff700 1 -- 172.21.15.36:0/3688149108 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226059332 0 0) 0x7f6b980aa170 con 0x7f6b980b2d80 2022-06-17T12:49:11.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.477+0000 7f6b97fff700 1 -- 172.21.15.36:0/3688149108 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b80003260 con 0x7f6b980b2d80 2022-06-17T12:49:11.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.477+0000 7f6b9cf7c700 1 -- 172.21.15.36:0/3688149108 >> v2:172.21.15.70:3300/0 conn(0x7f6b98143e60 msgr2=0x7f6b98140300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:11.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.477+0000 7f6b97fff700 1 -- 172.21.15.36:0/3688149108 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3717634687 0 0) 0x7f6b80003260 con 0x7f6b980b2d80 2022-06-17T12:49:11.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.477+0000 7f6b9cf7c700 1 --2- 172.21.15.36:0/3688149108 >> v2:172.21.15.70:3300/0 conn(0x7f6b98143e60 0x7f6b98140300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.477+0000 7f6b9cf7c700 1 -- 172.21.15.36:0/3688149108 >> v1:172.21.15.36:6789/0 conn(0x7f6b980b2d80 legacy=0x7f6b98143750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:11.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.477+0000 7f6b9cf7c700 1 -- 172.21.15.36:0/3688149108 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b980b08c0 con 0x7f6b980b2350 2022-06-17T12:49:11.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.477+0000 7f6b9df7e700 1 --2- 172.21.15.36:0/3688149108 >> v2:172.21.15.70:3300/0 conn(0x7f6b98143e60 0x7f6b98140300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:49:11.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.477+0000 7f6b9cf7c700 1 --2- 172.21.15.36:0/3688149108 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b980b2350 0x7f6b980b3160 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f6b88001800 tx=0x7f6b840091d0).ready entity=mon.2 client_cookie=fa35c4c1d73e3a9 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:11.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.477+0000 7f6b97fff700 1 -- 172.21.15.36:0/3688149108 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b88001660 con 0x7f6b980b2350 2022-06-17T12:49:11.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.477+0000 7f6b97fff700 1 -- 172.21.15.36:0/3688149108 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8400f510 con 0x7f6b980b2350 2022-06-17T12:49:11.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.477+0000 7f6b97fff700 1 -- 172.21.15.36:0/3688149108 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b8400f680 con 0x7f6b980b2350 2022-06-17T12:49:11.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.481+0000 7f6b9f9e1700 1 -- 172.21.15.36:0/3688149108 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b980b2350 msgr2=0x7f6b980b3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:11.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.481+0000 7f6b9f9e1700 1 --2- 172.21.15.36:0/3688149108 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b980b2350 0x7f6b980b3160 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f6b88001800 tx=0x7f6b840091d0).stop 2022-06-17T12:49:11.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.481+0000 7f6b9f9e1700 1 -- 172.21.15.36:0/3688149108 shutdown_connections 2022-06-17T12:49:11.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.481+0000 7f6b9f9e1700 1 --2- 172.21.15.36:0/3688149108 >> v2:172.21.15.70:3300/0 conn(0x7f6b98143e60 0x7f6b98140300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.481+0000 7f6b9f9e1700 1 --2- 172.21.15.36:0/3688149108 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b980b2350 0x7f6b980b3160 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.481+0000 7f6b9f9e1700 1 -- 172.21.15.36:0/3688149108 >> 172.21.15.36:0/3688149108 conn(0x7f6b9805c550 msgr2=0x7f6b9805c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:11.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.481+0000 7f6b9f9e1700 1 -- 172.21.15.36:0/3688149108 shutdown_connections 2022-06-17T12:49:11.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.481+0000 7f6b9f9e1700 1 -- 172.21.15.36:0/3688149108 wait complete. 2022-06-17T12:49:11.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.481+0000 7f6b9f9e1700 1 Processor -- start 2022-06-17T12:49:11.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.481+0000 7f6b9f9e1700 1 -- start start 2022-06-17T12:49:11.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.481+0000 7f6b9f9e1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b980b2350 0x7f6b98156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:11.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.481+0000 7f6b9f9e1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6b980b2d80 0x7f6b98147e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:11.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.481+0000 7f6b9f9e1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b98143e60 0x7f6b98140300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:11.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.481+0000 7f6b9f9e1700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b980aadc0 con 0x7f6b980b2d80 2022-06-17T12:49:11.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.481+0000 7f6b9f9e1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b980aa5e0 con 0x7f6b980b2350 2022-06-17T12:49:11.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.481+0000 7f6b9f9e1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b980aa880 con 0x7f6b98143e60 2022-06-17T12:49:11.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.485+0000 7f6b9cf7c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6b980b2d80 0x7f6b98147e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:11.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.485+0000 7f6b9d77d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b980b2350 0x7f6b98156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:11.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.485+0000 7f6b9cf7c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6b980b2d80 0x7f6b98147e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33262/0 (socket says 172.21.15.36:33262) 2022-06-17T12:49:11.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.485+0000 7f6b9df7e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b98143e60 0x7f6b98140300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:11.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.485+0000 7f6b9cf7c700 1 -- 172.21.15.36:0/3096258208 learned_addr learned my addr 172.21.15.36:0/3096258208 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:11.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.485+0000 7f6b9cf7c700 1 -- 172.21.15.36:0/3096258208 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b98143e60 msgr2=0x7f6b98140300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:11.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.485+0000 7f6b9cf7c700 1 --2- 172.21.15.36:0/3096258208 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b98143e60 0x7f6b98140300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.485+0000 7f6b9cf7c700 1 -- 172.21.15.36:0/3096258208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b980b2350 msgr2=0x7f6b98156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:11.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.485+0000 7f6b9cf7c700 1 --2- 172.21.15.36:0/3096258208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b980b2350 0x7f6b98156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.485+0000 7f6b9cf7c700 1 -- 172.21.15.36:0/3096258208 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9000b180 con 0x7f6b980b2d80 2022-06-17T12:49:11.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.485+0000 7f6b9cf7c700 1 --2- 172.21.15.36:0/3096258208 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6b980b2d80 0x7f6b98147e30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f6b88001800 tx=0x7f6b84016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:11.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.485+0000 7f6b967fc700 1 -- 172.21.15.36:0/3096258208 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b8400f680 con 0x7f6b980b2d80 2022-06-17T12:49:11.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.485+0000 7f6b967fc700 1 -- 172.21.15.36:0/3096258208 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b840032f0 con 0x7f6b980b2d80 2022-06-17T12:49:11.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.485+0000 7f6b967fc700 1 -- 172.21.15.36:0/3096258208 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b84021670 con 0x7f6b980b2d80 2022-06-17T12:49:11.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.485+0000 7f6b9f9e1700 1 -- 172.21.15.36:0/3096258208 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b84009470 con 0x7f6b980b2d80 2022-06-17T12:49:11.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.485+0000 7f6b9f9e1700 1 -- 172.21.15.36:0/3096258208 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b980ac6d0 con 0x7f6b980b2d80 2022-06-17T12:49:11.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.485+0000 7f6b9f9e1700 1 -- 172.21.15.36:0/3096258208 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b981442f0 con 0x7f6b980b2d80 2022-06-17T12:49:11.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.485+0000 7f6b967fc700 1 -- 172.21.15.36:0/3096258208 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6b84028070 con 0x7f6b980b2d80 2022-06-17T12:49:11.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.489+0000 7f6b967fc700 1 --2- 172.21.15.36:0/3096258208 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6b74053ad0 0x7f6b74055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:11.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.489+0000 7f6b967fc700 1 -- 172.21.15.36:0/3096258208 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f6b8406cb90 con 0x7f6b980b2d80 2022-06-17T12:49:11.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.489+0000 7f6b9d77d700 1 --2- 172.21.15.36:0/3096258208 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6b74053ad0 0x7f6b74055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:11.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.489+0000 7f6b9d77d700 1 --2- 172.21.15.36:0/3096258208 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6b74053ad0 0x7f6b74055f90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f6b8800d000 tx=0x7f6b88001a80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:11.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.493+0000 7f6b967fc700 1 -- 172.21.15.36:0/3096258208 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6b84043cd0 con 0x7f6b980b2d80 2022-06-17T12:49:11.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.621+0000 7f6b9f9e1700 1 -- 172.21.15.36:0/3096258208 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6b98057410 con 0x7f6b980b2d80 2022-06-17T12:49:11.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.621+0000 7f6b967fc700 1 -- 172.21.15.36:0/3096258208 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v41) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6b84015070 con 0x7f6b980b2d80 2022-06-17T12:49:11.613 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:49:11.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.621+0000 7f6b9f9e1700 1 -- 172.21.15.36:0/3096258208 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6b74053ad0 msgr2=0x7f6b74055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:11.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.621+0000 7f6b9f9e1700 1 --2- 172.21.15.36:0/3096258208 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6b74053ad0 0x7f6b74055f90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f6b8800d000 tx=0x7f6b88001a80).stop 2022-06-17T12:49:11.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.621+0000 7f6b9f9e1700 1 -- 172.21.15.36:0/3096258208 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6b980b2d80 msgr2=0x7f6b98147e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:11.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.621+0000 7f6b9f9e1700 1 --2- 172.21.15.36:0/3096258208 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6b980b2d80 0x7f6b98147e30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f6b88001800 tx=0x7f6b84016000).stop 2022-06-17T12:49:11.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.621+0000 7f6b9f9e1700 1 -- 172.21.15.36:0/3096258208 shutdown_connections 2022-06-17T12:49:11.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.621+0000 7f6b9f9e1700 1 --2- 172.21.15.36:0/3096258208 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6b74053ad0 0x7f6b74055f90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.621+0000 7f6b9f9e1700 1 --2- 172.21.15.36:0/3096258208 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b98143e60 0x7f6b98140300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.621+0000 7f6b9f9e1700 1 --2- 172.21.15.36:0/3096258208 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6b980b2d80 0x7f6b98147e30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.621+0000 7f6b9f9e1700 1 --2- 172.21.15.36:0/3096258208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b980b2350 0x7f6b98156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.621+0000 7f6b9f9e1700 1 -- 172.21.15.36:0/3096258208 >> 172.21.15.36:0/3096258208 conn(0x7f6b9805c550 msgr2=0x7f6b98142690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:11.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.621+0000 7f6b9f9e1700 1 -- 172.21.15.36:0/3096258208 shutdown_connections 2022-06-17T12:49:11.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.625+0000 7f6b9f9e1700 1 -- 172.21.15.36:0/3096258208 wait complete. 2022-06-17T12:49:11.620 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:49:11.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.749+0000 7f7ff1b34700 1 Processor -- start 2022-06-17T12:49:11.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.749+0000 7f7ff1b34700 1 -- start start 2022-06-17T12:49:11.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.749+0000 7f7ff1b34700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7fec0b9860 0x7f7fec143840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:11.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.749+0000 7f7ff1b34700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7fec0b5dc0 0x7f7fec0b6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:11.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.749+0000 7f7ff1b34700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fec0b4160 con 0x7f7fec0b45c0 2022-06-17T12:49:11.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.749+0000 7f7ff1b34700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fec0b9c40 con 0x7f7fec0b5dc0 2022-06-17T12:49:11.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.749+0000 7f7ff1b34700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fec0b5120 con 0x7f7fec0b9860 2022-06-17T12:49:11.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.749+0000 7f7ff1333700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7fec0b5dc0 0x7f7fec0b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:11.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.749+0000 7f7ff1333700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7fec0b5dc0 0x7f7fec0b6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33272/0 (socket says 172.21.15.36:33272) 2022-06-17T12:49:11.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.749+0000 7f7ff1333700 1 -- 172.21.15.36:0/672691849 learned_addr learned my addr 172.21.15.36:0/672691849 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:11.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.749+0000 7f7ff0b32700 1 --2- 172.21.15.36:0/672691849 >> v2:172.21.15.70:3300/0 conn(0x7f7fec0b9860 0x7f7fec143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:11.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.749+0000 7f7ff1333700 1 -- 172.21.15.36:0/672691849 >> v2:172.21.15.70:3300/0 conn(0x7f7fec0b9860 msgr2=0x7f7fec143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:11.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.749+0000 7f7ff1333700 1 --2- 172.21.15.36:0/672691849 >> v2:172.21.15.70:3300/0 conn(0x7f7fec0b9860 0x7f7fec143840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.749+0000 7f7ff1333700 1 -- 172.21.15.36:0/672691849 >> v1:172.21.15.70:6789/0 conn(0x7f7fec0b45c0 legacy=0x7f7fec0b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:11.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.749+0000 7f7ff1333700 1 -- 172.21.15.36:0/672691849 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fec0627d0 con 0x7f7fec0b5dc0 2022-06-17T12:49:11.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.753+0000 7f7ff1333700 1 --2- 172.21.15.36:0/672691849 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fec0b5dc0 0x7f7fec0b6210 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f7fec0b0fd0 tx=0x7f7fe4009290).ready entity=mon.0 client_cookie=ebb6add11dc3f555 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:11.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.753+0000 7f7feb7fe700 1 -- 172.21.15.36:0/672691849 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fe400b830 con 0x7f7fec0b5dc0 2022-06-17T12:49:11.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.753+0000 7f7feb7fe700 1 -- 172.21.15.36:0/672691849 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe400b990 con 0x7f7fec0b5dc0 2022-06-17T12:49:11.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.753+0000 7f7feb7fe700 1 -- 172.21.15.36:0/672691849 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fe400bb40 con 0x7f7fec0b5dc0 2022-06-17T12:49:11.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.753+0000 7f7ff1b34700 1 -- 172.21.15.36:0/672691849 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fec0b5dc0 msgr2=0x7f7fec0b6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:11.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.753+0000 7f7ff1b34700 1 --2- 172.21.15.36:0/672691849 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fec0b5dc0 0x7f7fec0b6210 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f7fec0b0fd0 tx=0x7f7fe4009290).stop 2022-06-17T12:49:11.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.753+0000 7f7ff1b34700 1 -- 172.21.15.36:0/672691849 shutdown_connections 2022-06-17T12:49:11.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.753+0000 7f7ff1b34700 1 --2- 172.21.15.36:0/672691849 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fec0b5dc0 0x7f7fec0b6210 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.753+0000 7f7ff1b34700 1 --2- 172.21.15.36:0/672691849 >> v2:172.21.15.70:3300/0 conn(0x7f7fec0b9860 0x7f7fec143840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.753+0000 7f7ff1b34700 1 -- 172.21.15.36:0/672691849 >> 172.21.15.36:0/672691849 conn(0x7f7fec05c550 msgr2=0x7f7fec060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:11.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.753+0000 7f7ff1b34700 1 -- 172.21.15.36:0/672691849 shutdown_connections 2022-06-17T12:49:11.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.753+0000 7f7ff1b34700 1 -- 172.21.15.36:0/672691849 wait complete. 2022-06-17T12:49:11.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.753+0000 7f7ff1b34700 1 Processor -- start 2022-06-17T12:49:11.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.753+0000 7f7ff1b34700 1 -- start start 2022-06-17T12:49:11.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.757+0000 7f7ff1b34700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fec0b45c0 0x7f7fec0af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:11.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.757+0000 7f7ff1b34700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fec0b5dc0 0x7f7fec0b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:11.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.757+0000 7f7ff1b34700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fec0b9860 0x7f7fec155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:11.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.757+0000 7f7ff1b34700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec1557e0 con 0x7f7fec0b9860 2022-06-17T12:49:11.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.757+0000 7f7ff1b34700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec0afc00 con 0x7f7fec0b45c0 2022-06-17T12:49:11.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.757+0000 7f7ff1b34700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec0afea0 con 0x7f7fec0b5dc0 2022-06-17T12:49:11.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.757+0000 7f7ff1333700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fec0b9860 0x7f7fec155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:11.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.757+0000 7f7ff1333700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fec0b9860 0x7f7fec155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33278/0 (socket says 172.21.15.36:33278) 2022-06-17T12:49:11.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.757+0000 7f7ff1333700 1 -- 172.21.15.36:0/1172552534 learned_addr learned my addr 172.21.15.36:0/1172552534 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:11.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.757+0000 7f7ff0b32700 1 --2- 172.21.15.36:0/1172552534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fec0b45c0 0x7f7fec0af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:11.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.757+0000 7f7febfff700 1 --2- 172.21.15.36:0/1172552534 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fec0b5dc0 0x7f7fec0b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:11.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.757+0000 7f7ff1333700 1 -- 172.21.15.36:0/1172552534 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fec0b5dc0 msgr2=0x7f7fec0b7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:11.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.757+0000 7f7ff1333700 1 --2- 172.21.15.36:0/1172552534 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fec0b5dc0 0x7f7fec0b7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.757+0000 7f7ff1333700 1 -- 172.21.15.36:0/1172552534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fec0b45c0 msgr2=0x7f7fec0af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:11.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.757+0000 7f7ff1333700 1 --2- 172.21.15.36:0/1172552534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fec0b45c0 0x7f7fec0af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.757+0000 7f7ff1333700 1 -- 172.21.15.36:0/1172552534 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe4009530 con 0x7f7fec0b9860 2022-06-17T12:49:11.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.757+0000 7f7ff1333700 1 --2- 172.21.15.36:0/1172552534 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fec0b9860 0x7f7fec155c40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f7fec0b0fd0 tx=0x7f7fe4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:11.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.757+0000 7f7fe9ffb700 1 -- 172.21.15.36:0/1172552534 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fe4021630 con 0x7f7fec0b9860 2022-06-17T12:49:11.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.757+0000 7f7fe9ffb700 1 -- 172.21.15.36:0/1172552534 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe4021790 con 0x7f7fec0b9860 2022-06-17T12:49:11.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.757+0000 7f7ff1b34700 1 -- 172.21.15.36:0/1172552534 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fec0af670 con 0x7f7fec0b9860 2022-06-17T12:49:11.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.757+0000 7f7fe9ffb700 1 -- 172.21.15.36:0/1172552534 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fe4021af0 con 0x7f7fec0b9860 2022-06-17T12:49:11.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.757+0000 7f7ff1b34700 1 -- 172.21.15.36:0/1172552534 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fec140490 con 0x7f7fec0b9860 2022-06-17T12:49:11.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.761+0000 7f7ff1b34700 1 -- 172.21.15.36:0/1172552534 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fec0b0060 con 0x7f7fec0b9860 2022-06-17T12:49:11.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.761+0000 7f7fe9ffb700 1 -- 172.21.15.36:0/1172552534 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f7fe400f070 con 0x7f7fec0b9860 2022-06-17T12:49:11.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.761+0000 7f7fe9ffb700 1 --2- 172.21.15.36:0/1172552534 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7fd00539b0 0x7f7fd0055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:11.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.761+0000 7f7fe9ffb700 1 -- 172.21.15.36:0/1172552534 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f7fe406c5c0 con 0x7f7fec0b9860 2022-06-17T12:49:11.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.761+0000 7f7ff0b32700 1 --2- 172.21.15.36:0/1172552534 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7fd00539b0 0x7f7fd0055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:11.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.761+0000 7f7ff0b32700 1 --2- 172.21.15.36:0/1172552534 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7fd00539b0 0x7f7fd0055e70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f7fdc00c000 tx=0x7f7fdc008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:11.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.769+0000 7f7fe9ffb700 1 -- 172.21.15.36:0/1172552534 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7fe4043700 con 0x7f7fec0b9860 2022-06-17T12:49:11.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.897+0000 7f7ff1b34700 1 -- 172.21.15.36:0/1172552534 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7fec062a70 con 0x7f7fec0b9860 2022-06-17T12:49:11.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.897+0000 7f7fe9ffb700 1 -- 172.21.15.36:0/1172552534 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v41) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7fe4015070 con 0x7f7fec0b9860 2022-06-17T12:49:11.913 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:49:11.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.901+0000 7f7ff1b34700 1 -- 172.21.15.36:0/1172552534 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7fd00539b0 msgr2=0x7f7fd0055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:11.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.901+0000 7f7ff1b34700 1 --2- 172.21.15.36:0/1172552534 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7fd00539b0 0x7f7fd0055e70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f7fdc00c000 tx=0x7f7fdc008040).stop 2022-06-17T12:49:11.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.901+0000 7f7ff1b34700 1 -- 172.21.15.36:0/1172552534 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fec0b9860 msgr2=0x7f7fec155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:11.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.901+0000 7f7ff1b34700 1 --2- 172.21.15.36:0/1172552534 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fec0b9860 0x7f7fec155c40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f7fec0b0fd0 tx=0x7f7fe4016000).stop 2022-06-17T12:49:11.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.901+0000 7f7ff1b34700 1 -- 172.21.15.36:0/1172552534 shutdown_connections 2022-06-17T12:49:11.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.901+0000 7f7ff1b34700 1 --2- 172.21.15.36:0/1172552534 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7fd00539b0 0x7f7fd0055e70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.901+0000 7f7ff1b34700 1 --2- 172.21.15.36:0/1172552534 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fec0b9860 0x7f7fec155c40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.901+0000 7f7ff1b34700 1 --2- 172.21.15.36:0/1172552534 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fec0b5dc0 0x7f7fec0b7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.901+0000 7f7ff1b34700 1 --2- 172.21.15.36:0/1172552534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fec0b45c0 0x7f7fec0af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:11.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.901+0000 7f7ff1b34700 1 -- 172.21.15.36:0/1172552534 >> 172.21.15.36:0/1172552534 conn(0x7f7fec05c550 msgr2=0x7f7fec0b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:11.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.901+0000 7f7ff1b34700 1 -- 172.21.15.36:0/1172552534 shutdown_connections 2022-06-17T12:49:11.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:11.901+0000 7f7ff1b34700 1 -- 172.21.15.36:0/1172552534 wait complete. 2022-06-17T12:49:11.918 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-17T12:49:12.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.041+0000 7f9596f7a700 1 Processor -- start 2022-06-17T12:49:12.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f9596f7a700 1 -- start start 2022-06-17T12:49:12.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f9596f7a700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f95900ae270 0x7f95900b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:12.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f9596f7a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f959013fde0 0x7f9590140200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:12.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f9596f7a700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95900afc30 con 0x7f95900ad9c0 2022-06-17T12:49:12.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f9596f7a700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95900b0cb0 con 0x7f95900ae270 2022-06-17T12:49:12.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f9596f7a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95900b3eb0 con 0x7f959013fde0 2022-06-17T12:49:12.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f958ffff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f95900ad9c0 0x7f95900b05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50812/0 (socket says 172.21.15.36:50812) 2022-06-17T12:49:12.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f958ffff700 1 -- 172.21.15.36:0/1040446651 learned_addr learned my addr 172.21.15.36:0/1040446651 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:12.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f9595517700 1 --2- 172.21.15.36:0/1040446651 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f959013fde0 0x7f9590140200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:12.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f9594d16700 1 --2- 172.21.15.36:0/1040446651 >> v2:172.21.15.70:3300/0 conn(0x7f95900ae270 0x7f95900b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:12.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f958f7fe700 1 -- 172.21.15.36:0/1040446651 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3703262103 0 0) 0x7f95900afc30 con 0x7f95900ad9c0 2022-06-17T12:49:12.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f958f7fe700 1 -- 172.21.15.36:0/1040446651 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9570003260 con 0x7f95900ad9c0 2022-06-17T12:49:12.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f9595517700 1 -- 172.21.15.36:0/1040446651 >> v2:172.21.15.70:3300/0 conn(0x7f95900ae270 msgr2=0x7f95900b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:12.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f9595517700 1 --2- 172.21.15.36:0/1040446651 >> v2:172.21.15.70:3300/0 conn(0x7f95900ae270 0x7f95900b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:12.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f9595517700 1 -- 172.21.15.36:0/1040446651 >> v1:172.21.15.36:6789/0 conn(0x7f95900ad9c0 legacy=0x7f95900b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:12.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f9595517700 1 -- 172.21.15.36:0/1040446651 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95900ae650 con 0x7f959013fde0 2022-06-17T12:49:12.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f9595517700 1 --2- 172.21.15.36:0/1040446651 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f959013fde0 0x7f9590140200 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f95880032f0 tx=0x7f95880091d0).ready entity=mon.2 client_cookie=93e4aba8c3f37d19 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:12.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f958f7fe700 1 -- 172.21.15.36:0/1040446651 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f958800b600 con 0x7f959013fde0 2022-06-17T12:49:12.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f958f7fe700 1 -- 172.21.15.36:0/1040446651 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958800b760 con 0x7f959013fde0 2022-06-17T12:49:12.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f958f7fe700 1 -- 172.21.15.36:0/1040446651 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f958800b8d0 con 0x7f959013fde0 2022-06-17T12:49:12.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f9596f7a700 1 -- 172.21.15.36:0/1040446651 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f959013fde0 msgr2=0x7f9590140200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:12.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f9596f7a700 1 --2- 172.21.15.36:0/1040446651 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f959013fde0 0x7f9590140200 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f95880032f0 tx=0x7f95880091d0).stop 2022-06-17T12:49:12.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f9596f7a700 1 -- 172.21.15.36:0/1040446651 shutdown_connections 2022-06-17T12:49:12.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f9596f7a700 1 --2- 172.21.15.36:0/1040446651 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f959013fde0 0x7f9590140200 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:12.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f9596f7a700 1 --2- 172.21.15.36:0/1040446651 >> v2:172.21.15.70:3300/0 conn(0x7f95900ae270 0x7f95900b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:12.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f9596f7a700 1 -- 172.21.15.36:0/1040446651 >> 172.21.15.36:0/1040446651 conn(0x7f959005c550 msgr2=0x7f959005c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:12.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f9596f7a700 1 -- 172.21.15.36:0/1040446651 shutdown_connections 2022-06-17T12:49:12.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.045+0000 7f9596f7a700 1 -- 172.21.15.36:0/1040446651 wait complete. 2022-06-17T12:49:12.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f9596f7a700 1 Processor -- start 2022-06-17T12:49:12.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f9596f7a700 1 -- start start 2022-06-17T12:49:12.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f9596f7a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95900ad9c0 0x7f95900b4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:12.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f9596f7a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95900ae270 0x7f959014e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:12.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f9596f7a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f959013fde0 0x7f95901517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:12.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f9596f7a700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9590063e10 con 0x7f95900ad9c0 2022-06-17T12:49:12.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f9596f7a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959013b110 con 0x7f959013fde0 2022-06-17T12:49:12.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f9596f7a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9590063630 con 0x7f95900ae270 2022-06-17T12:49:12.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f9594d16700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95900ad9c0 0x7f95900b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:12.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f9594d16700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95900ad9c0 0x7f95900b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33288/0 (socket says 172.21.15.36:33288) 2022-06-17T12:49:12.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f9594d16700 1 -- 172.21.15.36:0/2528815932 learned_addr learned my addr 172.21.15.36:0/2528815932 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:12.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f9595517700 1 --2- 172.21.15.36:0/2528815932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f959013fde0 0x7f95901517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:12.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f958ffff700 1 --2- 172.21.15.36:0/2528815932 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95900ae270 0x7f959014e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:12.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f9594d16700 1 -- 172.21.15.36:0/2528815932 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95900ae270 msgr2=0x7f959014e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:12.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f9594d16700 1 --2- 172.21.15.36:0/2528815932 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95900ae270 0x7f959014e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:12.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f9594d16700 1 -- 172.21.15.36:0/2528815932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f959013fde0 msgr2=0x7f95901517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:12.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f9594d16700 1 --2- 172.21.15.36:0/2528815932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f959013fde0 0x7f95901517b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:12.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f9594d16700 1 -- 172.21.15.36:0/2528815932 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9588009470 con 0x7f95900ad9c0 2022-06-17T12:49:12.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f9594d16700 1 --2- 172.21.15.36:0/2528815932 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95900ad9c0 0x7f95900b4790 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f958000c000 tx=0x7f9580006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:12.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f958dffb700 1 -- 172.21.15.36:0/2528815932 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9580010780 con 0x7f95900ad9c0 2022-06-17T12:49:12.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f958dffb700 1 -- 172.21.15.36:0/2528815932 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95800108e0 con 0x7f95900ad9c0 2022-06-17T12:49:12.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f9596f7a700 1 -- 172.21.15.36:0/2528815932 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95900638d0 con 0x7f95900ad9c0 2022-06-17T12:49:12.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f958dffb700 1 -- 172.21.15.36:0/2528815932 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9580010c40 con 0x7f95900ad9c0 2022-06-17T12:49:12.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f9596f7a700 1 -- 172.21.15.36:0/2528815932 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95900b4ea0 con 0x7f95900ad9c0 2022-06-17T12:49:12.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.049+0000 7f9596f7a700 1 -- 172.21.15.36:0/2528815932 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9590063a60 con 0x7f95900ad9c0 2022-06-17T12:49:12.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.053+0000 7f958dffb700 1 -- 172.21.15.36:0/2528815932 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f958001c460 con 0x7f95900ad9c0 2022-06-17T12:49:12.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.053+0000 7f958dffb700 1 --2- 172.21.15.36:0/2528815932 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9574053a40 0x7f9574055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:12.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.053+0000 7f958dffb700 1 -- 172.21.15.36:0/2528815932 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f958006c220 con 0x7f95900ad9c0 2022-06-17T12:49:12.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.053+0000 7f958ffff700 1 --2- 172.21.15.36:0/2528815932 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9574053a40 0x7f9574055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:12.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.053+0000 7f958ffff700 1 --2- 172.21.15.36:0/2528815932 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9574053a40 0x7f9574055f00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f957800b000 tx=0x7f9578009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:12.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.061+0000 7f958dffb700 1 -- 172.21.15.36:0/2528815932 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9580043260 con 0x7f95900ad9c0 2022-06-17T12:49:12.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.189+0000 7f9596f7a700 1 -- 172.21.15.36:0/2528815932 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f95900afe60 con 0x7f95900ad9c0 2022-06-17T12:49:12.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.189+0000 7f958dffb700 1 -- 172.21.15.36:0/2528815932 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) v1 ==== 84+0+143 (secure 0 0 0) 0x7f9580046880 con 0x7f95900ad9c0 2022-06-17T12:49:12.181 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:49:12.196 INFO:teuthology.orchestra.run.smithi036.stdout:[ 2022-06-17T12:49:12.197 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:12.197 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 1, 2022-06-17T12:49:12.198 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "device_health_metrics" 2022-06-17T12:49:12.198 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:12.199 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:49:12.201 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 2, 2022-06-17T12:49:12.201 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "rbd" 2022-06-17T12:49:12.202 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:12.203 INFO:teuthology.orchestra.run.smithi036.stdout:] 2022-06-17T12:49:12.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.189+0000 7f9596f7a700 1 -- 172.21.15.36:0/2528815932 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9574053a40 msgr2=0x7f9574055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:12.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.189+0000 7f9596f7a700 1 --2- 172.21.15.36:0/2528815932 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9574053a40 0x7f9574055f00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f957800b000 tx=0x7f9578009040).stop 2022-06-17T12:49:12.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.189+0000 7f9596f7a700 1 -- 172.21.15.36:0/2528815932 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95900ad9c0 msgr2=0x7f95900b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:12.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.189+0000 7f9596f7a700 1 --2- 172.21.15.36:0/2528815932 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95900ad9c0 0x7f95900b4790 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f958000c000 tx=0x7f9580006df0).stop 2022-06-17T12:49:12.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.193+0000 7f9596f7a700 1 -- 172.21.15.36:0/2528815932 shutdown_connections 2022-06-17T12:49:12.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.193+0000 7f9596f7a700 1 --2- 172.21.15.36:0/2528815932 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9574053a40 0x7f9574055f00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:12.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.193+0000 7f9596f7a700 1 --2- 172.21.15.36:0/2528815932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f959013fde0 0x7f95901517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:12.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.193+0000 7f9596f7a700 1 --2- 172.21.15.36:0/2528815932 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95900ae270 0x7f959014e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:12.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.193+0000 7f9596f7a700 1 --2- 172.21.15.36:0/2528815932 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95900ad9c0 0x7f95900b4790 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:12.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.193+0000 7f9596f7a700 1 -- 172.21.15.36:0/2528815932 >> 172.21.15.36:0/2528815932 conn(0x7f959005c550 msgr2=0x7f959005bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:12.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.193+0000 7f9596f7a700 1 -- 172.21.15.36:0/2528815932 shutdown_connections 2022-06-17T12:49:12.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.193+0000 7f9596f7a700 1 -- 172.21.15.36:0/2528815932 wait complete. 2022-06-17T12:49:12.227 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-17T12:49:12.234 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-17T12:49:12.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.377+0000 7f10dd925700 1 Processor -- start 2022-06-17T12:49:12.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.377+0000 7f10dd925700 1 -- start start 2022-06-17T12:49:12.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10dd925700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f10d8141690 0x7f10d80603b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:12.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10dd925700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10d81446a0 con 0x7f10d805faa0 2022-06-17T12:49:12.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10dd925700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10d80b31d0 con 0x7f10d80b2df0 2022-06-17T12:49:12.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10dd925700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d80608a0 con 0x7f10d8141690 2022-06-17T12:49:12.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d77fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f10d805faa0 0x7f10d813fed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50830/0 (socket says 172.21.15.36:50830) 2022-06-17T12:49:12.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d77fe700 1 -- 172.21.15.36:0/546352662 learned_addr learned my addr 172.21.15.36:0/546352662 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:12.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d6ffd700 1 --1- 172.21.15.36:0/546352662 >> v1:172.21.15.70:6789/0 conn(0x7f10d80b2df0 0x7f10d8140f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:46560/0 (socket says 172.21.15.36:46560) 2022-06-17T12:49:12.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d67fc700 1 --2- 172.21.15.36:0/546352662 >> v2:172.21.15.70:3300/0 conn(0x7f10d8141690 0x7f10d80603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:12.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d5ffb700 1 -- 172.21.15.36:0/546352662 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3263268105 0 0) 0x7f10d81446a0 con 0x7f10d805faa0 2022-06-17T12:49:12.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d5ffb700 1 -- 172.21.15.36:0/546352662 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10bc003260 con 0x7f10d805faa0 2022-06-17T12:49:12.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d5ffb700 1 -- 172.21.15.36:0/546352662 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3781311062 0 0) 0x7f10bc003260 con 0x7f10d805faa0 2022-06-17T12:49:12.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d5ffb700 1 -- 172.21.15.36:0/546352662 >> v2:172.21.15.70:3300/0 conn(0x7f10d8141690 msgr2=0x7f10d80603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:12.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d5ffb700 1 --2- 172.21.15.36:0/546352662 >> v2:172.21.15.70:3300/0 conn(0x7f10d8141690 0x7f10d80603b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:12.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d5ffb700 1 -- 172.21.15.36:0/546352662 >> v1:172.21.15.70:6789/0 conn(0x7f10d80b2df0 legacy=0x7f10d8140f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:12.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d5ffb700 1 -- 172.21.15.36:0/546352662 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d8144900 con 0x7f10d805faa0 2022-06-17T12:49:12.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d5ffb700 1 -- 172.21.15.36:0/546352662 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f10d00019e0 con 0x7f10d805faa0 2022-06-17T12:49:12.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d5ffb700 1 -- 172.21.15.36:0/546352662 >> v1:172.21.15.36:6789/0 conn(0x7f10d805faa0 legacy=0x7f10d813fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:12.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d5ffb700 1 --2- 172.21.15.36:0/546352662 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f10bc003750 0x7f10bc003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:12.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d5ffb700 1 --2- 172.21.15.36:0/546352662 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f10bc004500 0x7f10bc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:12.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d5ffb700 1 --2- 172.21.15.36:0/546352662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10bc004e70 0x7f10bc00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:12.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d5ffb700 1 -- 172.21.15.36:0/546352662 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc00bcf0 con 0x7f10bc004500 2022-06-17T12:49:12.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d5ffb700 1 -- 172.21.15.36:0/546352662 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc00be60 con 0x7f10bc004e70 2022-06-17T12:49:12.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d5ffb700 1 -- 172.21.15.36:0/546352662 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc00c020 con 0x7f10bc003750 2022-06-17T12:49:12.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d67fc700 1 --2- 172.21.15.36:0/546352662 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f10bc004500 0x7f10bc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:12.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d6ffd700 1 --2- 172.21.15.36:0/546352662 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f10bc003750 0x7f10bc003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:12.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d77fe700 1 --2- 172.21.15.36:0/546352662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10bc004e70 0x7f10bc00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:12.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d67fc700 1 -- 172.21.15.36:0/546352662 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f10bc003750 msgr2=0x7f10bc003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:12.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d67fc700 1 --2- 172.21.15.36:0/546352662 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f10bc003750 0x7f10bc003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:12.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d67fc700 1 -- 172.21.15.36:0/546352662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10bc004e70 msgr2=0x7f10bc00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:12.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d67fc700 1 --2- 172.21.15.36:0/546352662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10bc004e70 0x7f10bc00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:12.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d67fc700 1 -- 172.21.15.36:0/546352662 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10bc003260 con 0x7f10bc004500 2022-06-17T12:49:12.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d77fe700 1 -- 172.21.15.36:0/546352662 reap_dead start 2022-06-17T12:49:12.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d67fc700 1 --2- 172.21.15.36:0/546352662 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f10bc004500 0x7f10bc004980 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f10cc00c000 tx=0x7f10cc006df0).ready entity=mon.0 client_cookie=13913e917b00e048 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:12.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d5ffb700 1 -- 172.21.15.36:0/546352662 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10cc00e830 con 0x7f10bc004500 2022-06-17T12:49:12.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d5ffb700 1 -- 172.21.15.36:0/546352662 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10cc00e990 con 0x7f10bc004500 2022-06-17T12:49:12.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.381+0000 7f10d5ffb700 1 -- 172.21.15.36:0/546352662 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10cc00ecf0 con 0x7f10bc004500 2022-06-17T12:49:12.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.385+0000 7f10dd925700 1 -- 172.21.15.36:0/546352662 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f10bc004500 msgr2=0x7f10bc004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:12.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.385+0000 7f10dd925700 1 --2- 172.21.15.36:0/546352662 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f10bc004500 0x7f10bc004980 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f10cc00c000 tx=0x7f10cc006df0).stop 2022-06-17T12:49:12.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.385+0000 7f10dd925700 1 -- 172.21.15.36:0/546352662 shutdown_connections 2022-06-17T12:49:12.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.385+0000 7f10dd925700 1 --2- 172.21.15.36:0/546352662 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f10bc004500 0x7f10bc004980 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:12.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.385+0000 7f10dd925700 1 -- 172.21.15.36:0/546352662 >> 172.21.15.36:0/546352662 conn(0x7f10d8063020 msgr2=0x7f10d8063430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:12.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.385+0000 7f10dd925700 1 -- 172.21.15.36:0/546352662 shutdown_connections 2022-06-17T12:49:12.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.385+0000 7f10dd925700 1 -- 172.21.15.36:0/546352662 wait complete. 2022-06-17T12:49:12.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.385+0000 7f10dd925700 1 Processor -- start 2022-06-17T12:49:12.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.385+0000 7f10dd925700 1 -- start start 2022-06-17T12:49:12.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.389+0000 7f10dd925700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f10bc004500 0x7f10d8067270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:12.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.389+0000 7f10dd925700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f10d80652d0 0x7f10d8066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:12.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.389+0000 7f10dd925700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10d813ca10 0x7f10d813ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:12.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.389+0000 7f10dd925700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d8066060 con 0x7f10bc004500 2022-06-17T12:49:12.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.389+0000 7f10dd925700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d809fe00 con 0x7f10d813ca10 2022-06-17T12:49:12.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.389+0000 7f10dd925700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d80a00a0 con 0x7f10d80652d0 2022-06-17T12:49:12.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.389+0000 7f10d6ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f10bc004500 0x7f10d8067270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:12.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.389+0000 7f10d6ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f10bc004500 0x7f10d8067270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33308/0 (socket says 172.21.15.36:33308) 2022-06-17T12:49:12.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.389+0000 7f10d6ffd700 1 -- 172.21.15.36:0/911816143 learned_addr learned my addr 172.21.15.36:0/911816143 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:12.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.389+0000 7f10d77fe700 1 --2- 172.21.15.36:0/911816143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10d813ca10 0x7f10d813ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:12.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.389+0000 7f10d67fc700 1 --2- 172.21.15.36:0/911816143 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f10d80652d0 0x7f10d8066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:12.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.389+0000 7f10d6ffd700 1 -- 172.21.15.36:0/911816143 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f10d80652d0 msgr2=0x7f10d8066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:12.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.389+0000 7f10d6ffd700 1 --2- 172.21.15.36:0/911816143 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f10d80652d0 0x7f10d8066ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:12.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.389+0000 7f10d6ffd700 1 -- 172.21.15.36:0/911816143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10d813ca10 msgr2=0x7f10d813ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:12.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.389+0000 7f10d6ffd700 1 --2- 172.21.15.36:0/911816143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10d813ca10 0x7f10d813ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:12.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.389+0000 7f10d6ffd700 1 -- 172.21.15.36:0/911816143 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d813d380 con 0x7f10bc004500 2022-06-17T12:49:12.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.389+0000 7f10d6ffd700 1 --2- 172.21.15.36:0/911816143 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f10bc004500 0x7f10d8067270 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f10c800d000 tx=0x7f10c80079f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:12.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.389+0000 7f10c7fff700 1 -- 172.21.15.36:0/911816143 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10c8012540 con 0x7f10bc004500 2022-06-17T12:49:12.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.389+0000 7f10c7fff700 1 -- 172.21.15.36:0/911816143 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c80126a0 con 0x7f10bc004500 2022-06-17T12:49:12.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.389+0000 7f10dd925700 1 -- 172.21.15.36:0/911816143 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d8065ce0 con 0x7f10bc004500 2022-06-17T12:49:12.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.389+0000 7f10c7fff700 1 -- 172.21.15.36:0/911816143 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10c8012a00 con 0x7f10bc004500 2022-06-17T12:49:12.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.389+0000 7f10dd925700 1 -- 172.21.15.36:0/911816143 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d8066520 con 0x7f10bc004500 2022-06-17T12:49:12.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.389+0000 7f10dd925700 1 -- 172.21.15.36:0/911816143 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d8066fb0 con 0x7f10bc004500 2022-06-17T12:49:12.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.393+0000 7f10c7fff700 1 -- 172.21.15.36:0/911816143 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f10c801b030 con 0x7f10bc004500 2022-06-17T12:49:12.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.393+0000 7f10c7fff700 1 --2- 172.21.15.36:0/911816143 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f10b4053920 0x7f10b4055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:12.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.393+0000 7f10c7fff700 1 -- 172.21.15.36:0/911816143 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f10c806b460 con 0x7f10bc004500 2022-06-17T12:49:12.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.393+0000 7f10d67fc700 1 --2- 172.21.15.36:0/911816143 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f10b4053920 0x7f10b4055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:12.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.393+0000 7f10d67fc700 1 --2- 172.21.15.36:0/911816143 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f10b4053920 0x7f10b4055de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f10cc01d000 tx=0x7f10cc019000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:12.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.397+0000 7f10c7fff700 1 -- 172.21.15.36:0/911816143 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f10d8066fb0 con 0x7f10bc004500 2022-06-17T12:49:12.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.525+0000 7f10dd925700 1 -- 172.21.15.36:0/911816143 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f10d8066fb0 con 0x7f10bc004500 2022-06-17T12:49:12.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.629+0000 7f10c7fff700 1 -- 172.21.15.36:0/911816143 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v42) v1 ==== 133+0+0 (secure 0 0 0) 0x7f10d8066fb0 con 0x7f10bc004500 2022-06-17T12:49:12.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.633+0000 7f10dd925700 1 -- 172.21.15.36:0/911816143 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f10b4053920 msgr2=0x7f10b4055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:12.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.633+0000 7f10dd925700 1 --2- 172.21.15.36:0/911816143 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f10b4053920 0x7f10b4055de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f10cc01d000 tx=0x7f10cc019000).stop 2022-06-17T12:49:12.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.633+0000 7f10dd925700 1 -- 172.21.15.36:0/911816143 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f10bc004500 msgr2=0x7f10d8067270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:12.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.633+0000 7f10dd925700 1 --2- 172.21.15.36:0/911816143 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f10bc004500 0x7f10d8067270 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f10c800d000 tx=0x7f10c80079f0).stop 2022-06-17T12:49:12.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.633+0000 7f10dd925700 1 -- 172.21.15.36:0/911816143 shutdown_connections 2022-06-17T12:49:12.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.633+0000 7f10dd925700 1 --2- 172.21.15.36:0/911816143 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f10b4053920 0x7f10b4055de0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:12.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.633+0000 7f10dd925700 1 --2- 172.21.15.36:0/911816143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10d813ca10 0x7f10d813ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:12.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.633+0000 7f10dd925700 1 --2- 172.21.15.36:0/911816143 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f10d80652d0 0x7f10d8066ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:12.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.633+0000 7f10dd925700 1 --2- 172.21.15.36:0/911816143 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f10bc004500 0x7f10d8067270 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:12.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.633+0000 7f10dd925700 1 -- 172.21.15.36:0/911816143 >> 172.21.15.36:0/911816143 conn(0x7f10d8063020 msgr2=0x7f10d80b39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:12.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.633+0000 7f10dd925700 1 -- 172.21.15.36:0/911816143 shutdown_connections 2022-06-17T12:49:12.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.633+0000 7f10dd925700 1 -- 172.21.15.36:0/911816143 wait complete. 2022-06-17T12:49:12.628 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' created 2022-06-17T12:49:12.644 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-06-17T12:49:12.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.785+0000 7f5e3f59e700 1 Processor -- start 2022-06-17T12:49:12.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.785+0000 7f5e3f59e700 1 -- start start 2022-06-17T12:49:12.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.785+0000 7f5e3f59e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5e400b13c0 0x7f5e40140d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:12.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.785+0000 7f5e3f59e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5e400b2c10 0x7f5e401299a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:12.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.785+0000 7f5e3f59e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e400b05e0 con 0x7f5e40060ac0 2022-06-17T12:49:12.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.785+0000 7f5e3f59e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e40060ea0 con 0x7f5e400b2c10 2022-06-17T12:49:12.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.785+0000 7f5e3f59e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e401450f0 con 0x7f5e400b13c0 2022-06-17T12:49:12.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.785+0000 7f5e3ed9d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5e400b2c10 0x7f5e401299a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:12.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.785+0000 7f5e3e59c700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f5e40060ac0 0x7f5e400b0cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50846/0 (socket says 172.21.15.36:50846) 2022-06-17T12:49:12.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3e59c700 1 -- 172.21.15.36:0/1934016676 learned_addr learned my addr 172.21.15.36:0/1934016676 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:12.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3dd9b700 1 --2- 172.21.15.36:0/1934016676 >> v2:172.21.15.70:3300/0 conn(0x7f5e400b13c0 0x7f5e40140d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:12.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3ed9d700 1 -- 172.21.15.36:0/1934016676 >> v2:172.21.15.70:3300/0 conn(0x7f5e400b13c0 msgr2=0x7f5e40140d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:12.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3ed9d700 1 --2- 172.21.15.36:0/1934016676 >> v2:172.21.15.70:3300/0 conn(0x7f5e400b13c0 0x7f5e40140d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:12.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3ed9d700 1 -- 172.21.15.36:0/1934016676 >> v1:172.21.15.36:6789/0 conn(0x7f5e40060ac0 legacy=0x7f5e400b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:12.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3d59a700 1 -- 172.21.15.36:0/1934016676 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2244647883 0 0) 0x7f5e400b05e0 con 0x7f5e40060ac0 2022-06-17T12:49:12.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3ed9d700 1 -- 172.21.15.36:0/1934016676 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e400b0810 con 0x7f5e400b2c10 2022-06-17T12:49:12.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3ed9d700 1 --2- 172.21.15.36:0/1934016676 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e400b2c10 0x7f5e401299a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f5e380032f0 tx=0x7f5e380091d0).ready entity=mon.0 client_cookie=694764b3772caa9f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:12.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3d59a700 1 -- 172.21.15.36:0/1934016676 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e3800b680 con 0x7f5e400b2c10 2022-06-17T12:49:12.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3d59a700 1 -- 172.21.15.36:0/1934016676 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3800b7e0 con 0x7f5e400b2c10 2022-06-17T12:49:12.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3d59a700 1 -- 172.21.15.36:0/1934016676 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e3800b950 con 0x7f5e400b2c10 2022-06-17T12:49:12.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3f59e700 1 -- 172.21.15.36:0/1934016676 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e400b2c10 msgr2=0x7f5e401299a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:12.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3f59e700 1 --2- 172.21.15.36:0/1934016676 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e400b2c10 0x7f5e401299a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f5e380032f0 tx=0x7f5e380091d0).stop 2022-06-17T12:49:12.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3f59e700 1 -- 172.21.15.36:0/1934016676 shutdown_connections 2022-06-17T12:49:12.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3f59e700 1 --2- 172.21.15.36:0/1934016676 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e400b2c10 0x7f5e401299a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:12.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3f59e700 1 --2- 172.21.15.36:0/1934016676 >> v2:172.21.15.70:3300/0 conn(0x7f5e400b13c0 0x7f5e40140d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:12.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3f59e700 1 -- 172.21.15.36:0/1934016676 >> 172.21.15.36:0/1934016676 conn(0x7f5e400b64f0 msgr2=0x7f5e400b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:12.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3f59e700 1 -- 172.21.15.36:0/1934016676 shutdown_connections 2022-06-17T12:49:12.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3f59e700 1 -- 172.21.15.36:0/1934016676 wait complete. 2022-06-17T12:49:12.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3f59e700 1 Processor -- start 2022-06-17T12:49:12.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3f59e700 1 -- start start 2022-06-17T12:49:12.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3f59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5e40060ac0 0x7f5e40121ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:12.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3f59e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e400b13c0 0x7f5e40067150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:12.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3f59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e400b2c10 0x7f5e40064db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:12.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3f59e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e400660c0 con 0x7f5e400b2c10 2022-06-17T12:49:12.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3f59e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e400af0f0 con 0x7f5e400b13c0 2022-06-17T12:49:12.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3f59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e40152b40 con 0x7f5e40060ac0 2022-06-17T12:49:12.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3ed9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e400b2c10 0x7f5e40064db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:12.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3ed9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e400b2c10 0x7f5e40064db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33326/0 (socket says 172.21.15.36:33326) 2022-06-17T12:49:12.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.789+0000 7f5e3ed9d700 1 -- 172.21.15.36:0/2192736874 learned_addr learned my addr 172.21.15.36:0/2192736874 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:12.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.793+0000 7f5e3dd9b700 1 --2- 172.21.15.36:0/2192736874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e400b13c0 0x7f5e40067150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:12.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.793+0000 7f5e3e59c700 1 --2- 172.21.15.36:0/2192736874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5e40060ac0 0x7f5e40121ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:12.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.793+0000 7f5e3ed9d700 1 -- 172.21.15.36:0/2192736874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5e40060ac0 msgr2=0x7f5e40121ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:12.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.793+0000 7f5e3ed9d700 1 --2- 172.21.15.36:0/2192736874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5e40060ac0 0x7f5e40121ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:12.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.793+0000 7f5e3ed9d700 1 -- 172.21.15.36:0/2192736874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e400b13c0 msgr2=0x7f5e40067150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:12.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.793+0000 7f5e3ed9d700 1 --2- 172.21.15.36:0/2192736874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e400b13c0 0x7f5e40067150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:12.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.793+0000 7f5e3ed9d700 1 -- 172.21.15.36:0/2192736874 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e38009470 con 0x7f5e400b2c10 2022-06-17T12:49:12.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.793+0000 7f5e3ed9d700 1 --2- 172.21.15.36:0/2192736874 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e400b2c10 0x7f5e40064db0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f5e380032f0 tx=0x7f5e38016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:12.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.793+0000 7f5e377fe700 1 -- 172.21.15.36:0/2192736874 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e3800b680 con 0x7f5e400b2c10 2022-06-17T12:49:12.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.793+0000 7f5e377fe700 1 -- 172.21.15.36:0/2192736874 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e38021630 con 0x7f5e400b2c10 2022-06-17T12:49:12.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.793+0000 7f5e3f59e700 1 -- 172.21.15.36:0/2192736874 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e40152de0 con 0x7f5e400b2c10 2022-06-17T12:49:12.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.793+0000 7f5e377fe700 1 -- 172.21.15.36:0/2192736874 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e38021990 con 0x7f5e400b2c10 2022-06-17T12:49:12.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.793+0000 7f5e3f59e700 1 -- 172.21.15.36:0/2192736874 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e40063ab0 con 0x7f5e400b2c10 2022-06-17T12:49:12.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.793+0000 7f5e3f59e700 1 -- 172.21.15.36:0/2192736874 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e40141220 con 0x7f5e400b2c10 2022-06-17T12:49:12.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.793+0000 7f5e377fe700 1 -- 172.21.15.36:0/2192736874 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f5e38027020 con 0x7f5e400b2c10 2022-06-17T12:49:12.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.797+0000 7f5e377fe700 1 --2- 172.21.15.36:0/2192736874 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5e24053ad0 0x7f5e24055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:12.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.797+0000 7f5e377fe700 1 -- 172.21.15.36:0/2192736874 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f5e3806d2f0 con 0x7f5e400b2c10 2022-06-17T12:49:12.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.797+0000 7f5e3e59c700 1 --2- 172.21.15.36:0/2192736874 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5e24053ad0 0x7f5e24055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:12.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.797+0000 7f5e3e59c700 1 --2- 172.21.15.36:0/2192736874 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5e24053ad0 0x7f5e24055f90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f5e3000d000 tx=0x7f5e30007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:12.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.805+0000 7f5e377fe700 1 -- 172.21.15.36:0/2192736874 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5e38044130 con 0x7f5e400b2c10 2022-06-17T12:49:12.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:12.929+0000 7f5e3f59e700 1 -- 172.21.15.36:0/2192736874 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f5e40057410 con 0x7f5e400b2c10 2022-06-17T12:49:13.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.645+0000 7f5e377fe700 1 -- 172.21.15.36:0/2192736874 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v43) v1 ==== 125+0+0 (secure 0 0 0) 0x7f5e38047750 con 0x7f5e400b2c10 2022-06-17T12:49:13.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.649+0000 7f5e3f59e700 1 -- 172.21.15.36:0/2192736874 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5e24053ad0 msgr2=0x7f5e24055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:13.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.649+0000 7f5e3f59e700 1 --2- 172.21.15.36:0/2192736874 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5e24053ad0 0x7f5e24055f90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f5e3000d000 tx=0x7f5e30007a00).stop 2022-06-17T12:49:13.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.649+0000 7f5e3f59e700 1 -- 172.21.15.36:0/2192736874 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e400b2c10 msgr2=0x7f5e40064db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:13.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.649+0000 7f5e3f59e700 1 --2- 172.21.15.36:0/2192736874 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e400b2c10 0x7f5e40064db0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f5e380032f0 tx=0x7f5e38016000).stop 2022-06-17T12:49:13.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.649+0000 7f5e3f59e700 1 -- 172.21.15.36:0/2192736874 shutdown_connections 2022-06-17T12:49:13.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.649+0000 7f5e3f59e700 1 --2- 172.21.15.36:0/2192736874 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5e24053ad0 0x7f5e24055f90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:13.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.649+0000 7f5e3f59e700 1 --2- 172.21.15.36:0/2192736874 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e400b2c10 0x7f5e40064db0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:13.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.649+0000 7f5e3f59e700 1 --2- 172.21.15.36:0/2192736874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e400b13c0 0x7f5e40067150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:13.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.649+0000 7f5e3f59e700 1 --2- 172.21.15.36:0/2192736874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5e40060ac0 0x7f5e40121ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:13.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.649+0000 7f5e3f59e700 1 -- 172.21.15.36:0/2192736874 >> 172.21.15.36:0/2192736874 conn(0x7f5e400b64f0 msgr2=0x7f5e4013e470 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:13.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.649+0000 7f5e3f59e700 1 -- 172.21.15.36:0/2192736874 shutdown_connections 2022-06-17T12:49:13.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.649+0000 7f5e3f59e700 1 -- 172.21.15.36:0/2192736874 wait complete. 2022-06-17T12:49:13.654 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' created 2022-06-17T12:49:13.696 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-17T12:49:13.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.833+0000 7febf89bf700 1 Processor -- start 2022-06-17T12:49:13.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.833+0000 7febf89bf700 1 -- start start 2022-06-17T12:49:13.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.837+0000 7febf89bf700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febf4063730 0x7febf40b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:13.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.837+0000 7febf89bf700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf4063b10 con 0x7febf4064eb0 2022-06-17T12:49:13.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.837+0000 7febf89bf700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf41464d0 con 0x7febf4140610 2022-06-17T12:49:13.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.837+0000 7febf89bf700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf4147430 con 0x7febf4063730 2022-06-17T12:49:13.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.837+0000 7febf259c700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7febf4064eb0 0x7febf413ff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50864/0 (socket says 172.21.15.36:50864) 2022-06-17T12:49:13.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.837+0000 7febf259c700 1 -- 172.21.15.36:0/3394804065 learned_addr learned my addr 172.21.15.36:0/3394804065 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:13.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.837+0000 7febf359e700 1 --1- 172.21.15.36:0/3394804065 >> v1:172.21.15.70:6789/0 conn(0x7febf4140610 0x7febf40b33c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:46596/0 (socket says 172.21.15.36:46596) 2022-06-17T12:49:13.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.837+0000 7febf2d9d700 1 --2- 172.21.15.36:0/3394804065 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febf4063730 0x7febf40b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:13.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.837+0000 7febf1d9b700 1 -- 172.21.15.36:0/3394804065 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3196077038 0 0) 0x7febf4063b10 con 0x7febf4064eb0 2022-06-17T12:49:13.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.837+0000 7febf1d9b700 1 -- 172.21.15.36:0/3394804065 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febd8003260 con 0x7febf4064eb0 2022-06-17T12:49:13.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.837+0000 7febf1d9b700 1 -- 172.21.15.36:0/3394804065 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1803867210 0 0) 0x7febf41464d0 con 0x7febf4140610 2022-06-17T12:49:13.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.837+0000 7febf1d9b700 1 -- 172.21.15.36:0/3394804065 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febf4063b10 con 0x7febf4140610 2022-06-17T12:49:13.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.837+0000 7febf2d9d700 1 -- 172.21.15.36:0/3394804065 >> v1:172.21.15.70:6789/0 conn(0x7febf4140610 legacy=0x7febf40b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:13.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.837+0000 7febf2d9d700 1 -- 172.21.15.36:0/3394804065 >> v1:172.21.15.36:6789/0 conn(0x7febf4064eb0 legacy=0x7febf413ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:13.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.837+0000 7febf2d9d700 1 -- 172.21.15.36:0/3394804065 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf40b15a0 con 0x7febf4063730 2022-06-17T12:49:13.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.837+0000 7febf2d9d700 1 --2- 172.21.15.36:0/3394804065 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febf4063730 0x7febf40b2e10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7febe40032f0 tx=0x7febe40091d0).ready entity=mon.2 client_cookie=a964db23134cfc10 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:13.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.837+0000 7febf1d9b700 1 -- 172.21.15.36:0/3394804065 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febe400b640 con 0x7febf4063730 2022-06-17T12:49:13.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.837+0000 7febf1d9b700 1 -- 172.21.15.36:0/3394804065 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe400b7a0 con 0x7febf4063730 2022-06-17T12:49:13.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.837+0000 7febf1d9b700 1 -- 172.21.15.36:0/3394804065 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febe400b910 con 0x7febf4063730 2022-06-17T12:49:13.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.837+0000 7febf89bf700 1 -- 172.21.15.36:0/3394804065 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febf4063730 msgr2=0x7febf40b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:13.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.837+0000 7febf89bf700 1 --2- 172.21.15.36:0/3394804065 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febf4063730 0x7febf40b2e10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7febe40032f0 tx=0x7febe40091d0).stop 2022-06-17T12:49:13.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.837+0000 7febf89bf700 1 -- 172.21.15.36:0/3394804065 shutdown_connections 2022-06-17T12:49:13.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.837+0000 7febf89bf700 1 --2- 172.21.15.36:0/3394804065 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febf4063730 0x7febf40b2e10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:13.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.837+0000 7febf89bf700 1 -- 172.21.15.36:0/3394804065 >> 172.21.15.36:0/3394804065 conn(0x7febf40614b0 msgr2=0x7febf40618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:13.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.841+0000 7febf89bf700 1 -- 172.21.15.36:0/3394804065 shutdown_connections 2022-06-17T12:49:13.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.841+0000 7febf89bf700 1 -- 172.21.15.36:0/3394804065 wait complete. 2022-06-17T12:49:13.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.841+0000 7febf89bf700 1 Processor -- start 2022-06-17T12:49:13.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.841+0000 7febf89bf700 1 -- start start 2022-06-17T12:49:13.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.841+0000 7febf89bf700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febf4063730 0x7febf4142140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:13.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.841+0000 7febf89bf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febf4064eb0 0x7febf4151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:13.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.841+0000 7febf89bf700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febf4140610 0x7febf4151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:13.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.841+0000 7febf89bf700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf40b1ca0 con 0x7febf4140610 2022-06-17T12:49:13.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.841+0000 7febf89bf700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf40b0900 con 0x7febf4064eb0 2022-06-17T12:49:13.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.841+0000 7febf89bf700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf40af3b0 con 0x7febf4063730 2022-06-17T12:49:13.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.845+0000 7febf359e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febf4140610 0x7febf4151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:13.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.845+0000 7febf359e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febf4140610 0x7febf4151d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33338/0 (socket says 172.21.15.36:33338) 2022-06-17T12:49:13.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.845+0000 7febf359e700 1 -- 172.21.15.36:0/2220704830 learned_addr learned my addr 172.21.15.36:0/2220704830 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:13.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.845+0000 7febf2d9d700 1 --2- 172.21.15.36:0/2220704830 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febf4063730 0x7febf4142140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:13.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.845+0000 7febf259c700 1 --2- 172.21.15.36:0/2220704830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febf4064eb0 0x7febf4151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:13.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.845+0000 7febf359e700 1 -- 172.21.15.36:0/2220704830 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febf4063730 msgr2=0x7febf4142140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:13.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.845+0000 7febf359e700 1 --2- 172.21.15.36:0/2220704830 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febf4063730 0x7febf4142140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:13.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.845+0000 7febf359e700 1 -- 172.21.15.36:0/2220704830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febf4064eb0 msgr2=0x7febf4151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:13.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.845+0000 7febf359e700 1 --2- 172.21.15.36:0/2220704830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febf4064eb0 0x7febf4151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:13.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.845+0000 7febf359e700 1 -- 172.21.15.36:0/2220704830 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe4009470 con 0x7febf4140610 2022-06-17T12:49:13.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.845+0000 7febf2d9d700 1 --2- 172.21.15.36:0/2220704830 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febf4063730 0x7febf4142140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:49:13.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.845+0000 7febf359e700 1 --2- 172.21.15.36:0/2220704830 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febf4140610 0x7febf4151d50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7febec001110 tx=0x7febec007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:13.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.845+0000 7febe3fff700 1 -- 172.21.15.36:0/2220704830 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febec0070d0 con 0x7febf4140610 2022-06-17T12:49:13.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.845+0000 7febe3fff700 1 -- 172.21.15.36:0/2220704830 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febec007230 con 0x7febf4140610 2022-06-17T12:49:13.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.845+0000 7febf89bf700 1 -- 172.21.15.36:0/2220704830 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf40af650 con 0x7febf4140610 2022-06-17T12:49:13.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.845+0000 7febe3fff700 1 -- 172.21.15.36:0/2220704830 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febec005740 con 0x7febf4140610 2022-06-17T12:49:13.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.845+0000 7febf89bf700 1 -- 172.21.15.36:0/2220704830 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf4152240 con 0x7febf4140610 2022-06-17T12:49:13.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.845+0000 7febf89bf700 1 -- 172.21.15.36:0/2220704830 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf41524f0 con 0x7febf4140610 2022-06-17T12:49:13.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.845+0000 7febe3fff700 1 -- 172.21.15.36:0/2220704830 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7febec01a060 con 0x7febf4140610 2022-06-17T12:49:13.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.849+0000 7febe3fff700 1 --2- 172.21.15.36:0/2220704830 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7febd0053a40 0x7febd0055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:13.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.849+0000 7febe3fff700 1 -- 172.21.15.36:0/2220704830 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6304+0+0 (secure 0 0 0) 0x7febec06b290 con 0x7febf4140610 2022-06-17T12:49:13.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.849+0000 7febf2d9d700 1 --2- 172.21.15.36:0/2220704830 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7febd0053a40 0x7febd0055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:13.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.849+0000 7febf2d9d700 1 --2- 172.21.15.36:0/2220704830 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7febd0053a40 0x7febd0055f00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7febe4019000 tx=0x7febe4016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:13.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:13.853+0000 7febe3fff700 1 -- 172.21.15.36:0/2220704830 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7febec042110 con 0x7febf4140610 2022-06-17T12:49:14.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.017+0000 7febf89bf700 1 -- 172.21.15.36:0/2220704830 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7febf4057410 con 0x7febf4140610 2022-06-17T12:49:14.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.029+0000 7febe3fff700 1 -- 172.21.15.36:0/2220704830 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 7 and data pool 8 v18) v1 ==== 187+0+0 (secure 0 0 0) 0x7febec0180d0 con 0x7febf4140610 2022-06-17T12:49:14.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.033+0000 7febf89bf700 1 -- 172.21.15.36:0/2220704830 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7febd0053a40 msgr2=0x7febd0055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:14.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.033+0000 7febf89bf700 1 --2- 172.21.15.36:0/2220704830 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7febd0053a40 0x7febd0055f00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7febe4019000 tx=0x7febe4016000).stop 2022-06-17T12:49:14.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.033+0000 7febf89bf700 1 -- 172.21.15.36:0/2220704830 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febf4140610 msgr2=0x7febf4151d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:14.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.033+0000 7febf89bf700 1 --2- 172.21.15.36:0/2220704830 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febf4140610 0x7febf4151d50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7febec001110 tx=0x7febec007a60).stop 2022-06-17T12:49:14.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.033+0000 7febf89bf700 1 -- 172.21.15.36:0/2220704830 shutdown_connections 2022-06-17T12:49:14.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.033+0000 7febf89bf700 1 --2- 172.21.15.36:0/2220704830 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7febd0053a40 0x7febd0055f00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.033+0000 7febf89bf700 1 --2- 172.21.15.36:0/2220704830 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febf4140610 0x7febf4151d50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.033+0000 7febf89bf700 1 --2- 172.21.15.36:0/2220704830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febf4064eb0 0x7febf4151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.033+0000 7febf89bf700 1 --2- 172.21.15.36:0/2220704830 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febf4063730 0x7febf4142140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.033+0000 7febf89bf700 1 -- 172.21.15.36:0/2220704830 >> 172.21.15.36:0/2220704830 conn(0x7febf40614b0 msgr2=0x7febf413fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:14.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.033+0000 7febf89bf700 1 -- 172.21.15.36:0/2220704830 shutdown_connections 2022-06-17T12:49:14.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.033+0000 7febf89bf700 1 -- 172.21.15.36:0/2220704830 wait complete. 2022-06-17T12:49:14.058 INFO:teuthology.orchestra.run.smithi036.stderr:new fs with metadata pool 7 and data pool 8 2022-06-17T12:49:14.059 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:49:14.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.213+0000 7fe24ffff700 1 Processor -- start 2022-06-17T12:49:14.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.213+0000 7fe24ffff700 1 -- start start 2022-06-17T12:49:14.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.213+0000 7fe24ffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe25013cb70 0x7fe250143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:14.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.213+0000 7fe24ffff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2500af4b0 con 0x7fe25013cf50 2022-06-17T12:49:14.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.213+0000 7fe24ffff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2500ba500 con 0x7fe2500aef60 2022-06-17T12:49:14.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.213+0000 7fe24ffff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe250142f50 con 0x7fe25013cb70 2022-06-17T12:49:14.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.213+0000 7fe24effd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe25013cb70 0x7fe250143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:14.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.213+0000 7fe24f7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fe25013cf50 0x7fe2501442a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50876/0 (socket says 172.21.15.36:50876) 2022-06-17T12:49:14.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.213+0000 7fe24effd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe25013cb70 0x7fe250143690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33344/0 (socket says 172.21.15.36:33344) 2022-06-17T12:49:14.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.213+0000 7fe24f7fe700 1 -- 172.21.15.36:0/1573618232 learned_addr learned my addr 172.21.15.36:0/1573618232 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:14.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.213+0000 7fe24effd700 1 -- 172.21.15.36:0/1573618232 >> v1:172.21.15.70:6789/0 conn(0x7fe2500aef60 legacy=0x7fe2500b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:14.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.213+0000 7fe24dffb700 1 -- 172.21.15.36:0/1573618232 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 172665917 0 0) 0x7fe2500af4b0 con 0x7fe25013cf50 2022-06-17T12:49:14.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.213+0000 7fe24effd700 1 -- 172.21.15.36:0/1573618232 >> v1:172.21.15.36:6789/0 conn(0x7fe25013cf50 legacy=0x7fe2501442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:14.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.213+0000 7fe24effd700 1 -- 172.21.15.36:0/1573618232 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2501431f0 con 0x7fe25013cb70 2022-06-17T12:49:14.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.213+0000 7fe24effd700 1 --2- 172.21.15.36:0/1573618232 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe25013cb70 0x7fe250143690 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7fe2400032f0 tx=0x7fe2400091d0).ready entity=mon.0 client_cookie=b2222f5d7c214d70 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:14.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.217+0000 7fe24dffb700 1 -- 172.21.15.36:0/1573618232 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe240009ea0 con 0x7fe25013cb70 2022-06-17T12:49:14.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.217+0000 7fe24dffb700 1 -- 172.21.15.36:0/1573618232 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24000b8a0 con 0x7fe25013cb70 2022-06-17T12:49:14.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.217+0000 7fe24dffb700 1 -- 172.21.15.36:0/1573618232 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe24000ba10 con 0x7fe25013cb70 2022-06-17T12:49:14.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.217+0000 7fe24ffff700 1 -- 172.21.15.36:0/1573618232 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe25013cb70 msgr2=0x7fe250143690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:14.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.217+0000 7fe24ffff700 1 --2- 172.21.15.36:0/1573618232 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe25013cb70 0x7fe250143690 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7fe2400032f0 tx=0x7fe2400091d0).stop 2022-06-17T12:49:14.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.217+0000 7fe24ffff700 1 -- 172.21.15.36:0/1573618232 shutdown_connections 2022-06-17T12:49:14.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.217+0000 7fe24ffff700 1 --2- 172.21.15.36:0/1573618232 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe25013cb70 0x7fe250143690 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.217+0000 7fe24ffff700 1 -- 172.21.15.36:0/1573618232 >> 172.21.15.36:0/1573618232 conn(0x7fe25005c550 msgr2=0x7fe2500b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:14.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.217+0000 7fe24ffff700 1 -- 172.21.15.36:0/1573618232 shutdown_connections 2022-06-17T12:49:14.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.217+0000 7fe24ffff700 1 -- 172.21.15.36:0/1573618232 wait complete. 2022-06-17T12:49:14.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.221+0000 7fe24ffff700 1 Processor -- start 2022-06-17T12:49:14.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.221+0000 7fe24ffff700 1 -- start start 2022-06-17T12:49:14.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.221+0000 7fe24ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2500aef60 0x7fe2500affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:14.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.221+0000 7fe24ffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe25013cb70 0x7fe25005fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:14.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.221+0000 7fe24ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe25013cf50 0x7fe2500b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:14.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.221+0000 7fe24ffff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe250140620 con 0x7fe2500aef60 2022-06-17T12:49:14.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.221+0000 7fe24ffff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2501401b0 con 0x7fe25013cf50 2022-06-17T12:49:14.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.221+0000 7fe24ffff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2500afb00 con 0x7fe25013cb70 2022-06-17T12:49:14.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.221+0000 7fe24effd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2500aef60 0x7fe2500affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:14.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.221+0000 7fe24effd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2500aef60 0x7fe2500affa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33350/0 (socket says 172.21.15.36:33350) 2022-06-17T12:49:14.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.221+0000 7fe24effd700 1 -- 172.21.15.36:0/1882032366 learned_addr learned my addr 172.21.15.36:0/1882032366 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:14.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.221+0000 7fe24e7fc700 1 --2- 172.21.15.36:0/1882032366 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe25013cb70 0x7fe25005fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:14.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.221+0000 7fe24f7fe700 1 --2- 172.21.15.36:0/1882032366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe25013cf50 0x7fe2500b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:14.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.221+0000 7fe24effd700 1 -- 172.21.15.36:0/1882032366 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe25013cb70 msgr2=0x7fe25005fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:14.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.221+0000 7fe24effd700 1 --2- 172.21.15.36:0/1882032366 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe25013cb70 0x7fe25005fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.221+0000 7fe24effd700 1 -- 172.21.15.36:0/1882032366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe25013cf50 msgr2=0x7fe2500b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:14.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.221+0000 7fe24effd700 1 --2- 172.21.15.36:0/1882032366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe25013cf50 0x7fe2500b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.221+0000 7fe24effd700 1 -- 172.21.15.36:0/1882032366 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe240009470 con 0x7fe2500aef60 2022-06-17T12:49:14.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.221+0000 7fe24effd700 1 --2- 172.21.15.36:0/1882032366 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2500aef60 0x7fe2500affa0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7fe2400032f0 tx=0x7fe240016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:14.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.221+0000 7fe247fff700 1 -- 172.21.15.36:0/1882032366 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe240009ea0 con 0x7fe2500aef60 2022-06-17T12:49:14.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.221+0000 7fe247fff700 1 -- 172.21.15.36:0/1882032366 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24001f3f0 con 0x7fe2500aef60 2022-06-17T12:49:14.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.221+0000 7fe247fff700 1 -- 172.21.15.36:0/1882032366 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe24001f750 con 0x7fe2500aef60 2022-06-17T12:49:14.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.221+0000 7fe24ffff700 1 -- 172.21.15.36:0/1882032366 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe250153300 con 0x7fe2500aef60 2022-06-17T12:49:14.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.225+0000 7fe24ffff700 1 -- 172.21.15.36:0/1882032366 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2500b0f50 con 0x7fe2500aef60 2022-06-17T12:49:14.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.225+0000 7fe24ffff700 1 -- 172.21.15.36:0/1882032366 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2500afcc0 con 0x7fe2500aef60 2022-06-17T12:49:14.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.225+0000 7fe247fff700 1 -- 172.21.15.36:0/1882032366 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe24000f070 con 0x7fe2500aef60 2022-06-17T12:49:14.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.225+0000 7fe247fff700 1 --2- 172.21.15.36:0/1882032366 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe234053ad0 0x7fe234055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:14.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.225+0000 7fe247fff700 1 -- 172.21.15.36:0/1882032366 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe24006ca00 con 0x7fe2500aef60 2022-06-17T12:49:14.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.225+0000 7fe24e7fc700 1 --2- 172.21.15.36:0/1882032366 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe234053ad0 0x7fe234055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:14.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.225+0000 7fe24e7fc700 1 --2- 172.21.15.36:0/1882032366 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe234053ad0 0x7fe234055f90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fe23800d000 tx=0x7fe238007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:14.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.233+0000 7fe247fff700 1 -- 172.21.15.36:0/1882032366 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe240043880 con 0x7fe2500aef60 2022-06-17T12:49:14.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.357+0000 7fe24ffff700 1 -- 172.21.15.36:0/1882032366 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe250140370 con 0x7fe2500aef60 2022-06-17T12:49:14.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.361+0000 7fe247fff700 1 -- 172.21.15.36:0/1882032366 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16284 (secure 0 0 0) 0x7fe240015090 con 0x7fe2500aef60 2022-06-17T12:49:14.347 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:49:14.347 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":44,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:49:14.023010+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:49:12.581297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-06-17T12:49:12.985806+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:49:14.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.361+0000 7fe24ffff700 1 -- 172.21.15.36:0/1882032366 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe234053ad0 msgr2=0x7fe234055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:14.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.361+0000 7fe24ffff700 1 --2- 172.21.15.36:0/1882032366 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe234053ad0 0x7fe234055f90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fe23800d000 tx=0x7fe238007b90).stop 2022-06-17T12:49:14.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.361+0000 7fe24ffff700 1 -- 172.21.15.36:0/1882032366 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2500aef60 msgr2=0x7fe2500affa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:14.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.361+0000 7fe24ffff700 1 --2- 172.21.15.36:0/1882032366 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2500aef60 0x7fe2500affa0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7fe2400032f0 tx=0x7fe240016000).stop 2022-06-17T12:49:14.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.361+0000 7fe24ffff700 1 -- 172.21.15.36:0/1882032366 shutdown_connections 2022-06-17T12:49:14.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.361+0000 7fe24ffff700 1 --2- 172.21.15.36:0/1882032366 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe234053ad0 0x7fe234055f90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.361+0000 7fe24ffff700 1 --2- 172.21.15.36:0/1882032366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe25013cf50 0x7fe2500b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.361+0000 7fe24ffff700 1 --2- 172.21.15.36:0/1882032366 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe25013cb70 0x7fe25005fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.361+0000 7fe24ffff700 1 --2- 172.21.15.36:0/1882032366 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2500aef60 0x7fe2500affa0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.361+0000 7fe24ffff700 1 -- 172.21.15.36:0/1882032366 >> 172.21.15.36:0/1882032366 conn(0x7fe25005c550 msgr2=0x7fe2500b87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:14.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.361+0000 7fe24ffff700 1 -- 172.21.15.36:0/1882032366 shutdown_connections 2022-06-17T12:49:14.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.361+0000 7fe24ffff700 1 -- 172.21.15.36:0/1882032366 wait complete. 2022-06-17T12:49:14.364 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:49:14.374 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:49:14.387 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:49:14.403 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:49:14.441 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:49:14.451 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:49:14.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.485+0000 7f96df8f1700 1 Processor -- start 2022-06-17T12:49:14.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.485+0000 7f96df8f1700 1 -- start start 2022-06-17T12:49:14.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.485+0000 7f96df8f1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96d8142f90 0x7f96d81465d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:14.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.485+0000 7f96df8f1700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f96d8146ac0 0x7f96d80b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:14.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.485+0000 7f96df8f1700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d8143430 con 0x7f96d80601c0 2022-06-17T12:49:14.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.485+0000 7f96df8f1700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d8063520 con 0x7f96d8146ac0 2022-06-17T12:49:14.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.485+0000 7f96df8f1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d80605a0 con 0x7f96d8142f90 2022-06-17T12:49:14.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.485+0000 7f96dd68d700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f96d80601c0 0x7f96d8062e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50890/0 (socket says 172.21.15.36:50890) 2022-06-17T12:49:14.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.485+0000 7f96dd68d700 1 -- 172.21.15.36:0/1316244267 learned_addr learned my addr 172.21.15.36:0/1316244267 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:14.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.485+0000 7f96dde8e700 1 --2- 172.21.15.36:0/1316244267 >> v2:172.21.15.36:3300/0 conn(0x7f96d8146ac0 0x7f96d80b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:14.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.485+0000 7f96dce8c700 1 --2- 172.21.15.36:0/1316244267 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96d8142f90 0x7f96d81465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:14.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.485+0000 7f96cffff700 1 -- 172.21.15.36:0/1316244267 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1601613473 0 0) 0x7f96d8143430 con 0x7f96d80601c0 2022-06-17T12:49:14.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.485+0000 7f96cffff700 1 -- 172.21.15.36:0/1316244267 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96bc003260 con 0x7f96d80601c0 2022-06-17T12:49:14.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.485+0000 7f96dde8e700 1 -- 172.21.15.36:0/1316244267 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96d8142f90 msgr2=0x7f96d81465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:14.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.485+0000 7f96dde8e700 1 --2- 172.21.15.36:0/1316244267 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96d8142f90 0x7f96d81465d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.485+0000 7f96dde8e700 1 -- 172.21.15.36:0/1316244267 >> v1:172.21.15.36:6789/0 conn(0x7f96d80601c0 legacy=0x7f96d8062e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:14.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.485+0000 7f96dde8e700 1 -- 172.21.15.36:0/1316244267 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d80626d0 con 0x7f96d8146ac0 2022-06-17T12:49:14.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.489+0000 7f96dde8e700 1 --2- 172.21.15.36:0/1316244267 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96d8146ac0 0x7f96d80b4010 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f96d00032f0 tx=0x7f96d00091d0).ready entity=mon.0 client_cookie=874f067d0d2c2d78 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:14.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.489+0000 7f96cffff700 1 -- 172.21.15.36:0/1316244267 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96d000b600 con 0x7f96d8146ac0 2022-06-17T12:49:14.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.489+0000 7f96cffff700 1 -- 172.21.15.36:0/1316244267 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d000b760 con 0x7f96d8146ac0 2022-06-17T12:49:14.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.489+0000 7f96cffff700 1 -- 172.21.15.36:0/1316244267 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96d000b8d0 con 0x7f96d8146ac0 2022-06-17T12:49:14.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.489+0000 7f96df8f1700 1 -- 172.21.15.36:0/1316244267 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96d8146ac0 msgr2=0x7f96d80b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:14.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.489+0000 7f96df8f1700 1 --2- 172.21.15.36:0/1316244267 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96d8146ac0 0x7f96d80b4010 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f96d00032f0 tx=0x7f96d00091d0).stop 2022-06-17T12:49:14.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.489+0000 7f96df8f1700 1 -- 172.21.15.36:0/1316244267 shutdown_connections 2022-06-17T12:49:14.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.489+0000 7f96df8f1700 1 --2- 172.21.15.36:0/1316244267 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96d8146ac0 0x7f96d80b4010 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.489+0000 7f96df8f1700 1 --2- 172.21.15.36:0/1316244267 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96d8142f90 0x7f96d81465d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.489+0000 7f96df8f1700 1 -- 172.21.15.36:0/1316244267 >> 172.21.15.36:0/1316244267 conn(0x7f96d8060990 msgr2=0x7f96d805f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:14.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.489+0000 7f96df8f1700 1 -- 172.21.15.36:0/1316244267 shutdown_connections 2022-06-17T12:49:14.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.489+0000 7f96df8f1700 1 -- 172.21.15.36:0/1316244267 wait complete. 2022-06-17T12:49:14.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.489+0000 7f96df8f1700 1 Processor -- start 2022-06-17T12:49:14.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.493+0000 7f96df8f1700 1 -- start start 2022-06-17T12:49:14.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.493+0000 7f96df8f1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96d80601c0 0x7f96d8145340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:14.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.493+0000 7f96df8f1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d8142f90 0x7f96d8137410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:14.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.493+0000 7f96df8f1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96d8146ac0 0x7f96d8137900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:14.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.493+0000 7f96df8f1700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d80b0a80 con 0x7f96d80601c0 2022-06-17T12:49:14.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.493+0000 7f96df8f1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d8135a60 con 0x7f96d8142f90 2022-06-17T12:49:14.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.493+0000 7f96df8f1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d80b06e0 con 0x7f96d8146ac0 2022-06-17T12:49:14.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.493+0000 7f96dd68d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96d80601c0 0x7f96d8145340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:14.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.493+0000 7f96dd68d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96d80601c0 0x7f96d8145340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33364/0 (socket says 172.21.15.36:33364) 2022-06-17T12:49:14.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.493+0000 7f96dd68d700 1 -- 172.21.15.36:0/2549244296 learned_addr learned my addr 172.21.15.36:0/2549244296 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:14.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.493+0000 7f96dce8c700 1 --2- 172.21.15.36:0/2549244296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d8142f90 0x7f96d8137410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:14.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.493+0000 7f96dde8e700 1 --2- 172.21.15.36:0/2549244296 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96d8146ac0 0x7f96d8137900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:14.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.493+0000 7f96dd68d700 1 -- 172.21.15.36:0/2549244296 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96d8146ac0 msgr2=0x7f96d8137900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:14.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.493+0000 7f96dd68d700 1 --2- 172.21.15.36:0/2549244296 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96d8146ac0 0x7f96d8137900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.493+0000 7f96dd68d700 1 -- 172.21.15.36:0/2549244296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d8142f90 msgr2=0x7f96d8137410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:14.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.493+0000 7f96dd68d700 1 --2- 172.21.15.36:0/2549244296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d8142f90 0x7f96d8137410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.493+0000 7f96dd68d700 1 -- 172.21.15.36:0/2549244296 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d0009470 con 0x7f96d80601c0 2022-06-17T12:49:14.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.493+0000 7f96dce8c700 1 --2- 172.21.15.36:0/2549244296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d8142f90 0x7f96d8137410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:49:14.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.493+0000 7f96dd68d700 1 --2- 172.21.15.36:0/2549244296 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96d80601c0 0x7f96d8145340 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f96d4001110 tx=0x7f96d4007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:14.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.493+0000 7f96ce7fc700 1 -- 172.21.15.36:0/2549244296 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96d400ce20 con 0x7f96d80601c0 2022-06-17T12:49:14.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.493+0000 7f96ce7fc700 1 -- 172.21.15.36:0/2549244296 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d40072e0 con 0x7f96d80601c0 2022-06-17T12:49:14.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.493+0000 7f96df8f1700 1 -- 172.21.15.36:0/2549244296 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d8137df0 con 0x7f96d80601c0 2022-06-17T12:49:14.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.493+0000 7f96ce7fc700 1 -- 172.21.15.36:0/2549244296 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96d4005800 con 0x7f96d80601c0 2022-06-17T12:49:14.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.493+0000 7f96df8f1700 1 -- 172.21.15.36:0/2549244296 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d80b04e0 con 0x7f96d80601c0 2022-06-17T12:49:14.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.497+0000 7f96df8f1700 1 -- 172.21.15.36:0/2549244296 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d8135c60 con 0x7f96d80601c0 2022-06-17T12:49:14.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.497+0000 7f96ce7fc700 1 -- 172.21.15.36:0/2549244296 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f96d4010070 con 0x7f96d80601c0 2022-06-17T12:49:14.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.497+0000 7f96ce7fc700 1 --2- 172.21.15.36:0/2549244296 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f96c0053a60 0x7f96c0055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:14.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.497+0000 7f96ce7fc700 1 -- 172.21.15.36:0/2549244296 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f96d406b670 con 0x7f96d80601c0 2022-06-17T12:49:14.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.497+0000 7f96dce8c700 1 --2- 172.21.15.36:0/2549244296 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f96c0053a60 0x7f96c0055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:14.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.497+0000 7f96dce8c700 1 --2- 172.21.15.36:0/2549244296 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f96c0053a60 0x7f96c0055f20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f96c800b000 tx=0x7f96c8008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:14.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.505+0000 7f96ce7fc700 1 -- 172.21.15.36:0/2549244296 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f96d4042470 con 0x7f96d80601c0 2022-06-17T12:49:14.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.629+0000 7f96df8f1700 1 -- 172.21.15.36:0/2549244296 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96d8143f80 con 0x7f96d80601c0 2022-06-17T12:49:14.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.633+0000 7f96ce7fc700 1 -- 172.21.15.36:0/2549244296 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16284 (secure 0 0 0) 0x7f96d4045a90 con 0x7f96d80601c0 2022-06-17T12:49:14.619 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:49:14.620 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":44,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:49:14.023010+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:49:12.581297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-06-17T12:49:12.985806+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:49:14.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.633+0000 7f96df8f1700 1 -- 172.21.15.36:0/2549244296 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f96c0053a60 msgr2=0x7f96c0055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:14.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.633+0000 7f96df8f1700 1 --2- 172.21.15.36:0/2549244296 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f96c0053a60 0x7f96c0055f20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f96c800b000 tx=0x7f96c8008040).stop 2022-06-17T12:49:14.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.633+0000 7f96df8f1700 1 -- 172.21.15.36:0/2549244296 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96d80601c0 msgr2=0x7f96d8145340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:14.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.633+0000 7f96df8f1700 1 --2- 172.21.15.36:0/2549244296 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96d80601c0 0x7f96d8145340 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f96d4001110 tx=0x7f96d4007ad0).stop 2022-06-17T12:49:14.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.633+0000 7f96df8f1700 1 -- 172.21.15.36:0/2549244296 shutdown_connections 2022-06-17T12:49:14.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.633+0000 7f96df8f1700 1 --2- 172.21.15.36:0/2549244296 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f96c0053a60 0x7f96c0055f20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.633+0000 7f96df8f1700 1 --2- 172.21.15.36:0/2549244296 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96d8146ac0 0x7f96d8137900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.633+0000 7f96df8f1700 1 --2- 172.21.15.36:0/2549244296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d8142f90 0x7f96d8137410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.633+0000 7f96df8f1700 1 --2- 172.21.15.36:0/2549244296 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96d80601c0 0x7f96d8145340 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.633+0000 7f96df8f1700 1 -- 172.21.15.36:0/2549244296 >> 172.21.15.36:0/2549244296 conn(0x7f96d8060990 msgr2=0x7f96d80b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:14.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.633+0000 7f96df8f1700 1 -- 172.21.15.36:0/2549244296 shutdown_connections 2022-06-17T12:49:14.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.633+0000 7f96df8f1700 1 -- 172.21.15.36:0/2549244296 wait complete. 2022-06-17T12:49:14.666 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-17T12:49:14.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.809+0000 7f9742460700 1 Processor -- start 2022-06-17T12:49:14.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.809+0000 7f9742460700 1 -- start start 2022-06-17T12:49:14.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f9742460700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f973c0b65f0 0x7f973c0b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:14.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f9742460700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f973c0adf10 0x7f973c140530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:14.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f9742460700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973c13fdf0 con 0x7f973c0b15b0 2022-06-17T12:49:14.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f9742460700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f973c0ae8e0 con 0x7f973c0adf10 2022-06-17T12:49:14.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f9742460700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c140090 con 0x7f973c0b65f0 2022-06-17T12:49:14.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f9740c5d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f973c0adf10 0x7f973c140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:14.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f9741c5f700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f973c0b15b0 0x7f973c0af630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50904/0 (socket says 172.21.15.36:50904) 2022-06-17T12:49:14.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f9741c5f700 1 -- 172.21.15.36:0/2257078944 learned_addr learned my addr 172.21.15.36:0/2257078944 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:14.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f974145e700 1 --2- 172.21.15.36:0/2257078944 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f973c0b65f0 0x7f973c0b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:14.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f972ffff700 1 -- 172.21.15.36:0/2257078944 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 717571448 0 0) 0x7f973c13fdf0 con 0x7f973c0b15b0 2022-06-17T12:49:14.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f972ffff700 1 -- 172.21.15.36:0/2257078944 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9724003260 con 0x7f973c0b15b0 2022-06-17T12:49:14.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f9740c5d700 1 -- 172.21.15.36:0/2257078944 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f973c0b65f0 msgr2=0x7f973c0b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:14.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f9740c5d700 1 --2- 172.21.15.36:0/2257078944 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f973c0b65f0 0x7f973c0b69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f9740c5d700 1 -- 172.21.15.36:0/2257078944 >> v1:172.21.15.36:6789/0 conn(0x7f973c0b15b0 legacy=0x7f973c0af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:14.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f9740c5d700 1 -- 172.21.15.36:0/2257078944 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c140a20 con 0x7f973c0adf10 2022-06-17T12:49:14.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f9740c5d700 1 --2- 172.21.15.36:0/2257078944 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f973c0adf10 0x7f973c140530 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f9738002f70 tx=0x7f9738006d90).ready entity=mon.0 client_cookie=3863c9c75d1a274f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:14.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f972ffff700 1 -- 172.21.15.36:0/2257078944 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f973800e590 con 0x7f973c0adf10 2022-06-17T12:49:14.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f972ffff700 1 -- 172.21.15.36:0/2257078944 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9738000e90 con 0x7f973c0adf10 2022-06-17T12:49:14.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f972ffff700 1 -- 172.21.15.36:0/2257078944 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f973800e8e0 con 0x7f973c0adf10 2022-06-17T12:49:14.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f9742460700 1 -- 172.21.15.36:0/2257078944 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f973c0adf10 msgr2=0x7f973c140530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:14.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f9742460700 1 --2- 172.21.15.36:0/2257078944 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f973c0adf10 0x7f973c140530 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f9738002f70 tx=0x7f9738006d90).stop 2022-06-17T12:49:14.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f9742460700 1 -- 172.21.15.36:0/2257078944 shutdown_connections 2022-06-17T12:49:14.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f9742460700 1 --2- 172.21.15.36:0/2257078944 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f973c0adf10 0x7f973c140530 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f9742460700 1 --2- 172.21.15.36:0/2257078944 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f973c0b65f0 0x7f973c0b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f9742460700 1 -- 172.21.15.36:0/2257078944 >> 172.21.15.36:0/2257078944 conn(0x7f973c0614b0 msgr2=0x7f973c061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:14.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f9742460700 1 -- 172.21.15.36:0/2257078944 shutdown_connections 2022-06-17T12:49:14.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.813+0000 7f9742460700 1 -- 172.21.15.36:0/2257078944 wait complete. 2022-06-17T12:49:14.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.817+0000 7f9742460700 1 Processor -- start 2022-06-17T12:49:14.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.817+0000 7f9742460700 1 -- start start 2022-06-17T12:49:14.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.817+0000 7f9742460700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f973c0adf10 0x7f973c13b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:14.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.817+0000 7f9742460700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f973c0b15b0 0x7f973c152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:14.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.817+0000 7f9742460700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f973c0b65f0 0x7f973c142fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:14.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.817+0000 7f9742460700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0668f0 con 0x7f973c0adf10 2022-06-17T12:49:14.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.817+0000 7f9742460700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c066b90 con 0x7f973c0b65f0 2022-06-17T12:49:14.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.817+0000 7f9742460700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0670d0 con 0x7f973c0b15b0 2022-06-17T12:49:14.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.817+0000 7f974145e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f973c0adf10 0x7f973c13b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:14.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.817+0000 7f974145e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f973c0adf10 0x7f973c13b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33378/0 (socket says 172.21.15.36:33378) 2022-06-17T12:49:14.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.817+0000 7f974145e700 1 -- 172.21.15.36:0/262574659 learned_addr learned my addr 172.21.15.36:0/262574659 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:14.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.817+0000 7f9740c5d700 1 --2- 172.21.15.36:0/262574659 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f973c0b15b0 0x7f973c152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:14.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.817+0000 7f9741c5f700 1 --2- 172.21.15.36:0/262574659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f973c0b65f0 0x7f973c142fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:14.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.817+0000 7f974145e700 1 -- 172.21.15.36:0/262574659 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f973c0b15b0 msgr2=0x7f973c152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:14.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.817+0000 7f974145e700 1 --2- 172.21.15.36:0/262574659 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f973c0b15b0 0x7f973c152aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.817+0000 7f974145e700 1 -- 172.21.15.36:0/262574659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f973c0b65f0 msgr2=0x7f973c142fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:14.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.817+0000 7f974145e700 1 --2- 172.21.15.36:0/262574659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f973c0b65f0 0x7f973c142fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:14.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.817+0000 7f974145e700 1 -- 172.21.15.36:0/262574659 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973800b120 con 0x7f973c0adf10 2022-06-17T12:49:14.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.817+0000 7f974145e700 1 --2- 172.21.15.36:0/262574659 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f973c0adf10 0x7f973c13b680 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f973000c000 tx=0x7f9730006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:14.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.817+0000 7f972e7fc700 1 -- 172.21.15.36:0/262574659 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9730011750 con 0x7f973c0adf10 2022-06-17T12:49:14.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.817+0000 7f972e7fc700 1 -- 172.21.15.36:0/262574659 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97300118b0 con 0x7f973c0adf10 2022-06-17T12:49:14.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.817+0000 7f9742460700 1 -- 172.21.15.36:0/262574659 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f973c13bb70 con 0x7f973c0adf10 2022-06-17T12:49:14.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.817+0000 7f972e7fc700 1 -- 172.21.15.36:0/262574659 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9730011c10 con 0x7f973c0adf10 2022-06-17T12:49:14.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.817+0000 7f9742460700 1 -- 172.21.15.36:0/262574659 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973c0665f0 con 0x7f973c0adf10 2022-06-17T12:49:14.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.821+0000 7f9742460700 1 -- 172.21.15.36:0/262574659 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f973c142d10 con 0x7f973c0adf10 2022-06-17T12:49:14.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.821+0000 7f972e7fc700 1 -- 172.21.15.36:0/262574659 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f9730011d70 con 0x7f973c0adf10 2022-06-17T12:49:14.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.821+0000 7f972e7fc700 1 --2- 172.21.15.36:0/262574659 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f971c053920 0x7f971c055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:14.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.821+0000 7f9740c5d700 1 --2- 172.21.15.36:0/262574659 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f971c053920 0x7f971c055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:14.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.821+0000 7f972e7fc700 1 -- 172.21.15.36:0/262574659 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f973006c4f0 con 0x7f973c0adf10 2022-06-17T12:49:14.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.821+0000 7f9740c5d700 1 --2- 172.21.15.36:0/262574659 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f971c053920 0x7f971c055de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f973801e000 tx=0x7f9738014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:14.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.825+0000 7f972e7fc700 1 -- 172.21.15.36:0/262574659 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f973c142d10 con 0x7f973c0adf10 2022-06-17T12:49:14.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:14.989+0000 7f9742460700 1 -- 172.21.15.36:0/262574659 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f973c142d10 con 0x7f973c0adf10 2022-06-17T12:49:15.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.029+0000 7f972e7fc700 1 -- 172.21.15.36:0/262574659 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v19) v1 ==== 118+0+0 (secure 0 0 0) 0x7f973c142d10 con 0x7f973c0adf10 2022-06-17T12:49:15.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.033+0000 7f9742460700 1 -- 172.21.15.36:0/262574659 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f971c053920 msgr2=0x7f971c055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:15.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.033+0000 7f9742460700 1 --2- 172.21.15.36:0/262574659 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f971c053920 0x7f971c055de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f973801e000 tx=0x7f9738014000).stop 2022-06-17T12:49:15.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.033+0000 7f9742460700 1 -- 172.21.15.36:0/262574659 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f973c0adf10 msgr2=0x7f973c13b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:15.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.033+0000 7f9742460700 1 --2- 172.21.15.36:0/262574659 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f973c0adf10 0x7f973c13b680 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f973000c000 tx=0x7f9730006df0).stop 2022-06-17T12:49:15.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.033+0000 7f9742460700 1 -- 172.21.15.36:0/262574659 shutdown_connections 2022-06-17T12:49:15.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.033+0000 7f9742460700 1 --2- 172.21.15.36:0/262574659 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f971c053920 0x7f971c055de0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:15.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.033+0000 7f9742460700 1 --2- 172.21.15.36:0/262574659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f973c0b65f0 0x7f973c142fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:15.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.033+0000 7f9742460700 1 --2- 172.21.15.36:0/262574659 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f973c0b15b0 0x7f973c152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:15.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.033+0000 7f9742460700 1 --2- 172.21.15.36:0/262574659 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f973c0adf10 0x7f973c13b680 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:15.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.033+0000 7f9742460700 1 -- 172.21.15.36:0/262574659 >> 172.21.15.36:0/262574659 conn(0x7f973c0614b0 msgr2=0x7f973c0b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:15.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.033+0000 7f9742460700 1 -- 172.21.15.36:0/262574659 shutdown_connections 2022-06-17T12:49:15.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.033+0000 7f9742460700 1 -- 172.21.15.36:0/262574659 wait complete. 2022-06-17T12:49:15.041 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:49:15.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7ceac52700 1 Processor -- start 2022-06-17T12:49:15.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7ceac52700 1 -- start start 2022-06-17T12:49:15.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7ceac52700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ce4144c00 0x7f7ce4062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:15.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7ceac52700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce40b0ba0 con 0x7f7ce40ad330 2022-06-17T12:49:15.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7ceac52700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce4147e90 con 0x7f7ce4062770 2022-06-17T12:49:15.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7ceac52700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce40ae820 con 0x7f7ce4144c00 2022-06-17T12:49:15.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7ce91ef700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f7ce40ad330 0x7f7ce40ad780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50920/0 (socket says 172.21.15.36:50920) 2022-06-17T12:49:15.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7ce91ef700 1 -- 172.21.15.36:0/89393677 learned_addr learned my addr 172.21.15.36:0/89393677 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:15.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7cdbfff700 1 --2- 172.21.15.36:0/89393677 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ce4144c00 0x7f7ce4062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:15.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7cdb7fe700 1 -- 172.21.15.36:0/89393677 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3001614677 0 0) 0x7f7ce40b0ba0 con 0x7f7ce40ad330 2022-06-17T12:49:15.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7cdb7fe700 1 -- 172.21.15.36:0/89393677 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cd0003260 con 0x7f7ce40ad330 2022-06-17T12:49:15.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7cdbfff700 1 -- 172.21.15.36:0/89393677 >> v1:172.21.15.70:6789/0 conn(0x7f7ce4062770 legacy=0x7f7ce41444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:15.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7cdbfff700 1 -- 172.21.15.36:0/89393677 >> v1:172.21.15.36:6789/0 conn(0x7f7ce40ad330 legacy=0x7f7ce40ad780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:15.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7cdb7fe700 1 -- 172.21.15.36:0/89393677 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2319679070 0 0) 0x7f7ce4147e90 con 0x7f7ce4062770 2022-06-17T12:49:15.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7cdbfff700 1 -- 172.21.15.36:0/89393677 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce40b0dd0 con 0x7f7ce4144c00 2022-06-17T12:49:15.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7cdbfff700 1 --2- 172.21.15.36:0/89393677 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ce4144c00 0x7f7ce4062b50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f7cd4002f70 tx=0x7f7cd4006df0).ready entity=mon.2 client_cookie=302338f7f059ab8 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:15.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7cdb7fe700 1 -- 172.21.15.36:0/89393677 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cd400d570 con 0x7f7ce4144c00 2022-06-17T12:49:15.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7cdb7fe700 1 -- 172.21.15.36:0/89393677 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd4000e90 con 0x7f7ce4144c00 2022-06-17T12:49:15.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7cdb7fe700 1 -- 172.21.15.36:0/89393677 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cd400d8c0 con 0x7f7ce4144c00 2022-06-17T12:49:15.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7ceac52700 1 -- 172.21.15.36:0/89393677 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ce4144c00 msgr2=0x7f7ce4062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:15.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7ceac52700 1 --2- 172.21.15.36:0/89393677 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ce4144c00 0x7f7ce4062b50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f7cd4002f70 tx=0x7f7cd4006df0).stop 2022-06-17T12:49:15.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7ceac52700 1 -- 172.21.15.36:0/89393677 shutdown_connections 2022-06-17T12:49:15.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7ceac52700 1 --2- 172.21.15.36:0/89393677 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ce4144c00 0x7f7ce4062b50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:15.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7ceac52700 1 -- 172.21.15.36:0/89393677 >> 172.21.15.36:0/89393677 conn(0x7f7ce40b53e0 msgr2=0x7f7ce40b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:15.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7ceac52700 1 -- 172.21.15.36:0/89393677 shutdown_connections 2022-06-17T12:49:15.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.193+0000 7f7ceac52700 1 -- 172.21.15.36:0/89393677 wait complete. 2022-06-17T12:49:15.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.197+0000 7f7ceac52700 1 Processor -- start 2022-06-17T12:49:15.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.197+0000 7f7ceac52700 1 -- start start 2022-06-17T12:49:15.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.197+0000 7f7ceac52700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ce4062770 0x7f7ce414e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:15.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.197+0000 7f7ceac52700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ce40ad330 0x7f7ce4143cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:15.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.197+0000 7f7ceac52700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ce4144c00 0x7f7ce4156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:15.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.197+0000 7f7ceac52700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce40673d0 con 0x7f7ce40ad330 2022-06-17T12:49:15.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.197+0000 7f7ceac52700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce414acd0 con 0x7f7ce4062770 2022-06-17T12:49:15.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.197+0000 7f7ceac52700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce4063090 con 0x7f7ce4144c00 2022-06-17T12:49:15.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.197+0000 7f7cdbfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ce40ad330 0x7f7ce4143cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:15.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.197+0000 7f7cdbfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ce40ad330 0x7f7ce4143cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33394/0 (socket says 172.21.15.36:33394) 2022-06-17T12:49:15.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.197+0000 7f7cdbfff700 1 -- 172.21.15.36:0/1145272836 learned_addr learned my addr 172.21.15.36:0/1145272836 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:15.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.197+0000 7f7ce91ef700 1 --2- 172.21.15.36:0/1145272836 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ce4144c00 0x7f7ce4156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:15.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.197+0000 7f7ce89ee700 1 --2- 172.21.15.36:0/1145272836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ce4062770 0x7f7ce414e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:15.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.197+0000 7f7cdbfff700 1 -- 172.21.15.36:0/1145272836 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ce4144c00 msgr2=0x7f7ce4156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:15.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.197+0000 7f7cdbfff700 1 --2- 172.21.15.36:0/1145272836 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ce4144c00 0x7f7ce4156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:15.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.197+0000 7f7cdbfff700 1 -- 172.21.15.36:0/1145272836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ce4062770 msgr2=0x7f7ce414e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:15.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.197+0000 7f7cdbfff700 1 --2- 172.21.15.36:0/1145272836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ce4062770 0x7f7ce414e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:15.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.197+0000 7f7cdbfff700 1 -- 172.21.15.36:0/1145272836 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd400b180 con 0x7f7ce40ad330 2022-06-17T12:49:15.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.197+0000 7f7cdbfff700 1 --2- 172.21.15.36:0/1145272836 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ce40ad330 0x7f7ce4143cd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f7cd4002f70 tx=0x7f7cd4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:15.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.197+0000 7f7cd9ffb700 1 -- 172.21.15.36:0/1145272836 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cd401e400 con 0x7f7ce40ad330 2022-06-17T12:49:15.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.197+0000 7f7cd9ffb700 1 -- 172.21.15.36:0/1145272836 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd401e560 con 0x7f7ce40ad330 2022-06-17T12:49:15.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.197+0000 7f7ceac52700 1 -- 172.21.15.36:0/1145272836 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce40636c0 con 0x7f7ce40ad330 2022-06-17T12:49:15.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.197+0000 7f7cd9ffb700 1 -- 172.21.15.36:0/1145272836 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cd401e8c0 con 0x7f7ce40ad330 2022-06-17T12:49:15.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.197+0000 7f7ceac52700 1 -- 172.21.15.36:0/1145272836 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce4156be0 con 0x7f7ce40ad330 2022-06-17T12:49:15.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.201+0000 7f7ceac52700 1 -- 172.21.15.36:0/1145272836 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce4067540 con 0x7f7ce40ad330 2022-06-17T12:49:15.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.201+0000 7f7cd9ffb700 1 -- 172.21.15.36:0/1145272836 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f7cd4025030 con 0x7f7ce40ad330 2022-06-17T12:49:15.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.201+0000 7f7cd9ffb700 1 --2- 172.21.15.36:0/1145272836 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7cc4053a40 0x7f7cc4055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:15.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.201+0000 7f7cd9ffb700 1 -- 172.21.15.36:0/1145272836 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7cd406cdd0 con 0x7f7ce40ad330 2022-06-17T12:49:15.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.201+0000 7f7ce89ee700 1 --2- 172.21.15.36:0/1145272836 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7cc4053a40 0x7f7cc4055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:15.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.201+0000 7f7ce89ee700 1 --2- 172.21.15.36:0/1145272836 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7cc4053a40 0x7f7cc4055f00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f7ce000b000 tx=0x7f7ce0009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:15.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.209+0000 7f7cd9ffb700 1 -- 172.21.15.36:0/1145272836 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7ce4067540 con 0x7f7ce40ad330 2022-06-17T12:49:15.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.373+0000 7f7ceac52700 1 -- 172.21.15.36:0/1145272836 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ce4067510 con 0x7f7ce40ad330 2022-06-17T12:49:15.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.373+0000 7f7cd9ffb700 1 -- 172.21.15.36:0/1145272836 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+2138 (secure 0 0 0) 0x7f7cd4023020 con 0x7f7ce40ad330 2022-06-17T12:49:15.439 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:49:15.440 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":19,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/3777038413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3777038413},{"type":"v1","addr":"172.21.15.36:6835","nonce":3777038413}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:49:14.023351+0000","modified":"2022-06-17T12:49:15.026177+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5292":{"gid":5292,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/4117532324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4117532324},{"type":"v1","addr":"172.21.15.70:6833","nonce":4117532324}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-06-17T12:49:15.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.377+0000 7f7ceac52700 1 -- 172.21.15.36:0/1145272836 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7cc4053a40 msgr2=0x7f7cc4055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:15.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.377+0000 7f7ceac52700 1 --2- 172.21.15.36:0/1145272836 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7cc4053a40 0x7f7cc4055f00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f7ce000b000 tx=0x7f7ce0009040).stop 2022-06-17T12:49:15.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.377+0000 7f7ceac52700 1 -- 172.21.15.36:0/1145272836 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ce40ad330 msgr2=0x7f7ce4143cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:15.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.377+0000 7f7ceac52700 1 --2- 172.21.15.36:0/1145272836 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ce40ad330 0x7f7ce4143cd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f7cd4002f70 tx=0x7f7cd4014000).stop 2022-06-17T12:49:15.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.377+0000 7f7ceac52700 1 -- 172.21.15.36:0/1145272836 shutdown_connections 2022-06-17T12:49:15.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.377+0000 7f7ceac52700 1 --2- 172.21.15.36:0/1145272836 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7cc4053a40 0x7f7cc4055f00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:15.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.377+0000 7f7ceac52700 1 --2- 172.21.15.36:0/1145272836 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ce4144c00 0x7f7ce4156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:15.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.377+0000 7f7ceac52700 1 --2- 172.21.15.36:0/1145272836 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ce40ad330 0x7f7ce4143cd0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:15.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.377+0000 7f7ceac52700 1 --2- 172.21.15.36:0/1145272836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ce4062770 0x7f7ce414e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:15.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.377+0000 7f7ceac52700 1 -- 172.21.15.36:0/1145272836 >> 172.21.15.36:0/1145272836 conn(0x7f7ce40b53e0 msgr2=0x7f7ce40aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:15.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.377+0000 7f7ceac52700 1 -- 172.21.15.36:0/1145272836 shutdown_connections 2022-06-17T12:49:15.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.377+0000 7f7ceac52700 1 -- 172.21.15.36:0/1145272836 wait complete. 2022-06-17T12:49:15.467 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 19 2022-06-17T12:49:15.468 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:49:15.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf2d1ec700 1 Processor -- start 2022-06-17T12:49:15.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf2d1ec700 1 -- start start 2022-06-17T12:49:15.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf2d1ec700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fdf28062770 0x7fdf28062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:15.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf2d1ec700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf280ad330 0x7fdf280ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:15.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf2d1ec700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf280b0ba0 con 0x7fdf281444f0 2022-06-17T12:49:15.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf2d1ec700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf280ae820 con 0x7fdf28062770 2022-06-17T12:49:15.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf2d1ec700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf28061030 con 0x7fdf280ad330 2022-06-17T12:49:15.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf26d9d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fdf28062770 0x7fdf28062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:15.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf26d9d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fdf28062770 0x7fdf28062b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33400/0 (socket says 172.21.15.36:33400) 2022-06-17T12:49:15.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf26d9d700 1 -- 172.21.15.36:0/1250691893 learned_addr learned my addr 172.21.15.36:0/1250691893 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:15.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf2659c700 1 --1- 172.21.15.36:0/1250691893 >> v1:172.21.15.36:6789/0 conn(0x7fdf281444f0 0x7fdf28144960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50932/0 (socket says 172.21.15.36:50932) 2022-06-17T12:49:15.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf2759e700 1 --2- 172.21.15.36:0/1250691893 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf280ad330 0x7fdf280ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:15.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf26d9d700 1 -- 172.21.15.36:0/1250691893 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf280ad330 msgr2=0x7fdf280ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:15.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf26d9d700 1 --2- 172.21.15.36:0/1250691893 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf280ad330 0x7fdf280ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:15.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf26d9d700 1 -- 172.21.15.36:0/1250691893 >> v1:172.21.15.36:6789/0 conn(0x7fdf281444f0 legacy=0x7fdf28144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:15.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf26d9d700 1 -- 172.21.15.36:0/1250691893 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf28147e90 con 0x7fdf28062770 2022-06-17T12:49:15.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf26d9d700 1 --2- 172.21.15.36:0/1250691893 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf28062770 0x7fdf28062b50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fdf28061170 tx=0x7fdf18006eb0).ready entity=mon.0 client_cookie=bf9f54646c40a44c server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:15.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf25d9b700 1 -- 172.21.15.36:0/1250691893 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf1800d5c0 con 0x7fdf28062770 2022-06-17T12:49:15.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf25d9b700 1 -- 172.21.15.36:0/1250691893 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf18000e90 con 0x7fdf28062770 2022-06-17T12:49:15.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf25d9b700 1 -- 172.21.15.36:0/1250691893 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf1800d950 con 0x7fdf28062770 2022-06-17T12:49:15.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf2d1ec700 1 -- 172.21.15.36:0/1250691893 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf28062770 msgr2=0x7fdf28062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:15.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf2d1ec700 1 --2- 172.21.15.36:0/1250691893 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf28062770 0x7fdf28062b50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fdf28061170 tx=0x7fdf18006eb0).stop 2022-06-17T12:49:15.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf2d1ec700 1 -- 172.21.15.36:0/1250691893 shutdown_connections 2022-06-17T12:49:15.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf2d1ec700 1 --2- 172.21.15.36:0/1250691893 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf280ad330 0x7fdf280ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:15.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf2d1ec700 1 --2- 172.21.15.36:0/1250691893 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf28062770 0x7fdf28062b50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:15.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf2d1ec700 1 -- 172.21.15.36:0/1250691893 >> 172.21.15.36:0/1250691893 conn(0x7fdf280b53e0 msgr2=0x7fdf280b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:15.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf2d1ec700 1 -- 172.21.15.36:0/1250691893 shutdown_connections 2022-06-17T12:49:15.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.609+0000 7fdf2d1ec700 1 -- 172.21.15.36:0/1250691893 wait complete. 2022-06-17T12:49:15.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.613+0000 7fdf2d1ec700 1 Processor -- start 2022-06-17T12:49:15.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.613+0000 7fdf2d1ec700 1 -- start start 2022-06-17T12:49:15.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.613+0000 7fdf2d1ec700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf28062770 0x7fdf28147350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:15.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.613+0000 7fdf2d1ec700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf280ad330 0x7fdf2812f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:15.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.613+0000 7fdf2d1ec700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf281444f0 0x7fdf2812f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:15.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.613+0000 7fdf2d1ec700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2814aa80 con 0x7fdf28062770 2022-06-17T12:49:15.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.613+0000 7fdf2d1ec700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf28063fb0 con 0x7fdf280ad330 2022-06-17T12:49:15.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.613+0000 7fdf2d1ec700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2814acb0 con 0x7fdf281444f0 2022-06-17T12:49:15.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.613+0000 7fdf26d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf28062770 0x7fdf28147350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:15.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.613+0000 7fdf26d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf28062770 0x7fdf28147350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33406/0 (socket says 172.21.15.36:33406) 2022-06-17T12:49:15.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.613+0000 7fdf26d9d700 1 -- 172.21.15.36:0/1095571115 learned_addr learned my addr 172.21.15.36:0/1095571115 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:15.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.613+0000 7fdf2659c700 1 --2- 172.21.15.36:0/1095571115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf280ad330 0x7fdf2812f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:15.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.613+0000 7fdf2759e700 1 --2- 172.21.15.36:0/1095571115 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf281444f0 0x7fdf2812f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:15.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.613+0000 7fdf26d9d700 1 -- 172.21.15.36:0/1095571115 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf281444f0 msgr2=0x7fdf2812f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:15.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.613+0000 7fdf26d9d700 1 --2- 172.21.15.36:0/1095571115 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf281444f0 0x7fdf2812f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:15.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.613+0000 7fdf26d9d700 1 -- 172.21.15.36:0/1095571115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf280ad330 msgr2=0x7fdf2812f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:15.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.613+0000 7fdf26d9d700 1 --2- 172.21.15.36:0/1095571115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf280ad330 0x7fdf2812f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:15.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.613+0000 7fdf26d9d700 1 -- 172.21.15.36:0/1095571115 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1800b1e0 con 0x7fdf28062770 2022-06-17T12:49:15.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.613+0000 7fdf26d9d700 1 --2- 172.21.15.36:0/1095571115 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf28062770 0x7fdf28147350 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fdf28061170 tx=0x7fdf18015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:15.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.617+0000 7fdf17fff700 1 -- 172.21.15.36:0/1095571115 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf18021070 con 0x7fdf28062770 2022-06-17T12:49:15.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.617+0000 7fdf2d1ec700 1 -- 172.21.15.36:0/1095571115 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2812fd30 con 0x7fdf28062770 2022-06-17T12:49:15.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.617+0000 7fdf17fff700 1 -- 172.21.15.36:0/1095571115 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf18002f70 con 0x7fdf28062770 2022-06-17T12:49:15.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.617+0000 7fdf17fff700 1 -- 172.21.15.36:0/1095571115 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf18011d70 con 0x7fdf28062770 2022-06-17T12:49:15.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.617+0000 7fdf2d1ec700 1 -- 172.21.15.36:0/1095571115 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2814a6c0 con 0x7fdf28062770 2022-06-17T12:49:15.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.617+0000 7fdf2d1ec700 1 -- 172.21.15.36:0/1095571115 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf280b1c20 con 0x7fdf28062770 2022-06-17T12:49:15.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.617+0000 7fdf17fff700 1 -- 172.21.15.36:0/1095571115 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fdf1800f070 con 0x7fdf28062770 2022-06-17T12:49:15.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.617+0000 7fdf17fff700 1 --2- 172.21.15.36:0/1095571115 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdf040539d0 0x7fdf04055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:15.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.617+0000 7fdf17fff700 1 -- 172.21.15.36:0/1095571115 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdf1806d340 con 0x7fdf28062770 2022-06-17T12:49:15.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.617+0000 7fdf2659c700 1 --2- 172.21.15.36:0/1095571115 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdf040539d0 0x7fdf04055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:15.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.617+0000 7fdf2659c700 1 --2- 172.21.15.36:0/1095571115 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdf040539d0 0x7fdf04055e90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fdf1c00d000 tx=0x7fdf1c000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:15.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.625+0000 7fdf17fff700 1 -- 172.21.15.36:0/1095571115 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdf18044060 con 0x7fdf28062770 2022-06-17T12:49:15.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.757+0000 7fdf2d1ec700 1 -- 172.21.15.36:0/1095571115 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf28057410 con 0x7fdf28062770 2022-06-17T12:49:15.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.757+0000 7fdf17fff700 1 -- 172.21.15.36:0/1095571115 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16271 (secure 0 0 0) 0x7fdf18047680 con 0x7fdf28062770 2022-06-17T12:49:15.748 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:49:15.748 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":45,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:49:15.026771+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:49:12.581297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-06-17T12:49:12.985806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:49:15.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.761+0000 7fdf2d1ec700 1 -- 172.21.15.36:0/1095571115 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdf040539d0 msgr2=0x7fdf04055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:15.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.761+0000 7fdf2d1ec700 1 --2- 172.21.15.36:0/1095571115 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdf040539d0 0x7fdf04055e90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fdf1c00d000 tx=0x7fdf1c000bc0).stop 2022-06-17T12:49:15.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.761+0000 7fdf2d1ec700 1 -- 172.21.15.36:0/1095571115 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf28062770 msgr2=0x7fdf28147350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:15.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.761+0000 7fdf2d1ec700 1 --2- 172.21.15.36:0/1095571115 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf28062770 0x7fdf28147350 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fdf28061170 tx=0x7fdf18015000).stop 2022-06-17T12:49:15.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.761+0000 7fdf2d1ec700 1 -- 172.21.15.36:0/1095571115 shutdown_connections 2022-06-17T12:49:15.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.761+0000 7fdf2d1ec700 1 --2- 172.21.15.36:0/1095571115 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdf040539d0 0x7fdf04055e90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:15.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.761+0000 7fdf2d1ec700 1 --2- 172.21.15.36:0/1095571115 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf281444f0 0x7fdf2812f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:15.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.761+0000 7fdf2d1ec700 1 --2- 172.21.15.36:0/1095571115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf280ad330 0x7fdf2812f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:15.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.761+0000 7fdf2d1ec700 1 --2- 172.21.15.36:0/1095571115 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf28062770 0x7fdf28147350 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:15.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.761+0000 7fdf2d1ec700 1 -- 172.21.15.36:0/1095571115 >> 172.21.15.36:0/1095571115 conn(0x7fdf280b53e0 msgr2=0x7fdf2814dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:15.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.761+0000 7fdf2d1ec700 1 -- 172.21.15.36:0/1095571115 shutdown_connections 2022-06-17T12:49:15.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.761+0000 7fdf2d1ec700 1 -- 172.21.15.36:0/1095571115 wait complete. 2022-06-17T12:49:15.770 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:49:15.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.913+0000 7fe843fff700 1 Processor -- start 2022-06-17T12:49:15.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.913+0000 7fe843fff700 1 -- start start 2022-06-17T12:49:15.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.913+0000 7fe843fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe8440aef60 0x7fe8440b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:15.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.913+0000 7fe843fff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8440af4b0 con 0x7fe84413cb70 2022-06-17T12:49:15.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.913+0000 7fe843fff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe844142f50 con 0x7fe8440ba2e0 2022-06-17T12:49:15.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.913+0000 7fe843fff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe844143da0 con 0x7fe8440aef60 2022-06-17T12:49:15.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.913+0000 7fe8427fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe8440aef60 0x7fe8440b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:15.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.913+0000 7fe842ffd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fe84413cb70 0x7fe844143690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50944/0 (socket says 172.21.15.36:50944) 2022-06-17T12:49:15.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.913+0000 7fe842ffd700 1 -- 172.21.15.36:0/902180034 learned_addr learned my addr 172.21.15.36:0/902180034 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:15.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.913+0000 7fe8427fc700 1 -- 172.21.15.36:0/902180034 >> v1:172.21.15.70:6789/0 conn(0x7fe8440ba2e0 legacy=0x7fe8441442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:15.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.913+0000 7fe8427fc700 1 -- 172.21.15.36:0/902180034 >> v1:172.21.15.36:6789/0 conn(0x7fe84413cb70 legacy=0x7fe844143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:15.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.913+0000 7fe8427fc700 1 -- 172.21.15.36:0/902180034 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8441431f0 con 0x7fe8440aef60 2022-06-17T12:49:15.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.917+0000 7fe8427fc700 1 --2- 172.21.15.36:0/902180034 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8440aef60 0x7fe8440b9df0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7fe834002f70 tx=0x7fe834006df0).ready entity=mon.0 client_cookie=ed9a0d250345818d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:15.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.917+0000 7fe841ffb700 1 -- 172.21.15.36:0/902180034 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe83400d610 con 0x7fe8440aef60 2022-06-17T12:49:15.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.917+0000 7fe841ffb700 1 -- 172.21.15.36:0/902180034 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe834000e90 con 0x7fe8440aef60 2022-06-17T12:49:15.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.917+0000 7fe841ffb700 1 -- 172.21.15.36:0/902180034 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe83400d960 con 0x7fe8440aef60 2022-06-17T12:49:15.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.917+0000 7fe843fff700 1 -- 172.21.15.36:0/902180034 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8440aef60 msgr2=0x7fe8440b9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:15.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.917+0000 7fe843fff700 1 --2- 172.21.15.36:0/902180034 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8440aef60 0x7fe8440b9df0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7fe834002f70 tx=0x7fe834006df0).stop 2022-06-17T12:49:15.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.917+0000 7fe843fff700 1 -- 172.21.15.36:0/902180034 shutdown_connections 2022-06-17T12:49:15.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.917+0000 7fe843fff700 1 --2- 172.21.15.36:0/902180034 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8440aef60 0x7fe8440b9df0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:15.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.917+0000 7fe843fff700 1 -- 172.21.15.36:0/902180034 >> 172.21.15.36:0/902180034 conn(0x7fe84405c550 msgr2=0x7fe8440b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:15.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.917+0000 7fe843fff700 1 -- 172.21.15.36:0/902180034 shutdown_connections 2022-06-17T12:49:15.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.917+0000 7fe843fff700 1 -- 172.21.15.36:0/902180034 wait complete. 2022-06-17T12:49:15.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.921+0000 7fe843fff700 1 Processor -- start 2022-06-17T12:49:15.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.921+0000 7fe843fff700 1 -- start start 2022-06-17T12:49:15.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.921+0000 7fe843fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8440aef60 0x7fe84413d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:15.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.921+0000 7fe843fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8440ba2e0 0x7fe844152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:15.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.921+0000 7fe843fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe84413cb70 0x7fe84413e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:15.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.921+0000 7fe843fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe844155ba0 con 0x7fe8440ba2e0 2022-06-17T12:49:15.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.921+0000 7fe843fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8441523d0 con 0x7fe84413cb70 2022-06-17T12:49:15.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.921+0000 7fe8427fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8440ba2e0 0x7fe844152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:15.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.921+0000 7fe843fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe844152670 con 0x7fe8440aef60 2022-06-17T12:49:15.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.921+0000 7fe8427fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8440ba2e0 0x7fe844152b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33422/0 (socket says 172.21.15.36:33422) 2022-06-17T12:49:15.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.921+0000 7fe8427fc700 1 -- 172.21.15.36:0/1626257991 learned_addr learned my addr 172.21.15.36:0/1626257991 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:15.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.921+0000 7fe842ffd700 1 --2- 172.21.15.36:0/1626257991 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8440aef60 0x7fe84413d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:15.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.921+0000 7fe8437fe700 1 --2- 172.21.15.36:0/1626257991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe84413cb70 0x7fe84413e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:15.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.921+0000 7fe8427fc700 1 -- 172.21.15.36:0/1626257991 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8440aef60 msgr2=0x7fe84413d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:15.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.921+0000 7fe8427fc700 1 --2- 172.21.15.36:0/1626257991 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8440aef60 0x7fe84413d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:15.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.921+0000 7fe8427fc700 1 -- 172.21.15.36:0/1626257991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe84413cb70 msgr2=0x7fe84413e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:15.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.921+0000 7fe8427fc700 1 --2- 172.21.15.36:0/1626257991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe84413cb70 0x7fe84413e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:15.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.921+0000 7fe8427fc700 1 -- 172.21.15.36:0/1626257991 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83400b180 con 0x7fe8440ba2e0 2022-06-17T12:49:15.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.921+0000 7fe8427fc700 1 --2- 172.21.15.36:0/1626257991 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8440ba2e0 0x7fe844152b10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7fe834002f70 tx=0x7fe834015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:15.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.921+0000 7fe833fff700 1 -- 172.21.15.36:0/1626257991 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe83400d990 con 0x7fe8440ba2e0 2022-06-17T12:49:15.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.921+0000 7fe833fff700 1 -- 172.21.15.36:0/1626257991 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe834011b40 con 0x7fe8440ba2e0 2022-06-17T12:49:15.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.921+0000 7fe843fff700 1 -- 172.21.15.36:0/1626257991 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8441566c0 con 0x7fe8440ba2e0 2022-06-17T12:49:15.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.921+0000 7fe833fff700 1 -- 172.21.15.36:0/1626257991 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe834011ea0 con 0x7fe8440ba2e0 2022-06-17T12:49:15.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.921+0000 7fe843fff700 1 -- 172.21.15.36:0/1626257991 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe844155820 con 0x7fe8440ba2e0 2022-06-17T12:49:15.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.925+0000 7fe843fff700 1 -- 172.21.15.36:0/1626257991 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe844152210 con 0x7fe8440ba2e0 2022-06-17T12:49:15.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.925+0000 7fe833fff700 1 -- 172.21.15.36:0/1626257991 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe834025030 con 0x7fe8440ba2e0 2022-06-17T12:49:15.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.925+0000 7fe833fff700 1 --2- 172.21.15.36:0/1626257991 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe820053a40 0x7fe820055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:15.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.925+0000 7fe833fff700 1 -- 172.21.15.36:0/1626257991 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe83406d270 con 0x7fe8440ba2e0 2022-06-17T12:49:15.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.925+0000 7fe842ffd700 1 --2- 172.21.15.36:0/1626257991 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe820053a40 0x7fe820055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:15.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.925+0000 7fe842ffd700 1 --2- 172.21.15.36:0/1626257991 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe820053a40 0x7fe820055f00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fe83800d000 tx=0x7fe838007be0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:15.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:15.933+0000 7fe833fff700 1 -- 172.21.15.36:0/1626257991 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe834044070 con 0x7fe8440ba2e0 2022-06-17T12:49:16.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.105+0000 7fe843fff700 1 -- 172.21.15.36:0/1626257991 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fe8441449b0 con 0x7fe8440ba2e0 2022-06-17T12:49:16.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.109+0000 7fe833fff700 1 -- 172.21.15.36:0/1626257991 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v16) v1 ==== 184+0+0 (secure 0 0 0) 0x7fe8340230e0 con 0x7fe8440ba2e0 2022-06-17T12:49:16.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.113+0000 7fe843fff700 1 -- 172.21.15.36:0/1626257991 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe820053a40 msgr2=0x7fe820055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:16.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.113+0000 7fe843fff700 1 --2- 172.21.15.36:0/1626257991 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe820053a40 0x7fe820055f00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fe83800d000 tx=0x7fe838007be0).stop 2022-06-17T12:49:16.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.113+0000 7fe843fff700 1 -- 172.21.15.36:0/1626257991 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8440ba2e0 msgr2=0x7fe844152b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:16.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.113+0000 7fe843fff700 1 --2- 172.21.15.36:0/1626257991 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8440ba2e0 0x7fe844152b10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7fe834002f70 tx=0x7fe834015000).stop 2022-06-17T12:49:16.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.113+0000 7fe843fff700 1 -- 172.21.15.36:0/1626257991 shutdown_connections 2022-06-17T12:49:16.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.113+0000 7fe843fff700 1 --2- 172.21.15.36:0/1626257991 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe820053a40 0x7fe820055f00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:16.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.113+0000 7fe843fff700 1 --2- 172.21.15.36:0/1626257991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe84413cb70 0x7fe84413e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:16.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.113+0000 7fe843fff700 1 --2- 172.21.15.36:0/1626257991 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8440ba2e0 0x7fe844152b10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:16.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.113+0000 7fe843fff700 1 --2- 172.21.15.36:0/1626257991 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8440aef60 0x7fe84413d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:16.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.113+0000 7fe843fff700 1 -- 172.21.15.36:0/1626257991 >> 172.21.15.36:0/1626257991 conn(0x7fe84405c550 msgr2=0x7fe8440b8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:16.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.113+0000 7fe843fff700 1 -- 172.21.15.36:0/1626257991 shutdown_connections 2022-06-17T12:49:16.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.113+0000 7fe843fff700 1 -- 172.21.15.36:0/1626257991 wait complete. 2022-06-17T12:49:16.110 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.0 2022-06-17T12:49:16.115 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:49:16.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.245+0000 7fe9dea7e700 1 Processor -- start 2022-06-17T12:49:16.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.245+0000 7fe9dea7e700 1 -- start start 2022-06-17T12:49:16.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.245+0000 7fe9dea7e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe9d80b4820 0x7fe9d80b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:16.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.249+0000 7fe9dea7e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d80b83f0 con 0x7fe9d8062470 2022-06-17T12:49:16.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.249+0000 7fe9dea7e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d80b1ff0 con 0x7fe9d8143a90 2022-06-17T12:49:16.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.249+0000 7fe9dea7e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9d8143f00 con 0x7fe9d80b4820 2022-06-17T12:49:16.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.249+0000 7fe9dda7c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe9d80b4820 0x7fe9d80b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:16.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.249+0000 7fe9dd27b700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fe9d8062470 0x7fe9d80b7ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50960/0 (socket says 172.21.15.36:50960) 2022-06-17T12:49:16.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.249+0000 7fe9dd27b700 1 -- 172.21.15.36:0/3618751395 learned_addr learned my addr 172.21.15.36:0/3618751395 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:16.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.249+0000 7fe9dda7c700 1 -- 172.21.15.36:0/3618751395 >> v1:172.21.15.70:6789/0 conn(0x7fe9d8143a90 legacy=0x7fe9d8144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:16.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.249+0000 7fe9dda7c700 1 -- 172.21.15.36:0/3618751395 >> v1:172.21.15.36:6789/0 conn(0x7fe9d8062470 legacy=0x7fe9d80b7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:16.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.249+0000 7fe9dca7a700 1 -- 172.21.15.36:0/3618751395 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 640767774 0 0) 0x7fe9d80b83f0 con 0x7fe9d8062470 2022-06-17T12:49:16.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.249+0000 7fe9dda7c700 1 -- 172.21.15.36:0/3618751395 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d81430b0 con 0x7fe9d80b4820 2022-06-17T12:49:16.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.249+0000 7fe9dda7c700 1 --2- 172.21.15.36:0/3618751395 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe9d80b4820 0x7fe9d80b4c00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7fe9d4002f70 tx=0x7fe9d4006df0).ready entity=mon.0 client_cookie=fbd17ebe7e6df301 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:16.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.249+0000 7fe9dca7a700 1 -- 172.21.15.36:0/3618751395 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9d400d610 con 0x7fe9d80b4820 2022-06-17T12:49:16.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.249+0000 7fe9dca7a700 1 -- 172.21.15.36:0/3618751395 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d4000e90 con 0x7fe9d80b4820 2022-06-17T12:49:16.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.249+0000 7fe9dca7a700 1 -- 172.21.15.36:0/3618751395 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9d400d960 con 0x7fe9d80b4820 2022-06-17T12:49:16.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.249+0000 7fe9dea7e700 1 -- 172.21.15.36:0/3618751395 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe9d80b4820 msgr2=0x7fe9d80b4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:16.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.249+0000 7fe9dea7e700 1 --2- 172.21.15.36:0/3618751395 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe9d80b4820 0x7fe9d80b4c00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7fe9d4002f70 tx=0x7fe9d4006df0).stop 2022-06-17T12:49:16.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.249+0000 7fe9dea7e700 1 -- 172.21.15.36:0/3618751395 shutdown_connections 2022-06-17T12:49:16.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.249+0000 7fe9dea7e700 1 --2- 172.21.15.36:0/3618751395 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe9d80b4820 0x7fe9d80b4c00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:16.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.249+0000 7fe9dea7e700 1 -- 172.21.15.36:0/3618751395 >> 172.21.15.36:0/3618751395 conn(0x7fe9d805c550 msgr2=0x7fe9d80b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:16.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.249+0000 7fe9dea7e700 1 -- 172.21.15.36:0/3618751395 shutdown_connections 2022-06-17T12:49:16.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.249+0000 7fe9dea7e700 1 -- 172.21.15.36:0/3618751395 wait complete. 2022-06-17T12:49:16.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.253+0000 7fe9dea7e700 1 Processor -- start 2022-06-17T12:49:16.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.253+0000 7fe9dea7e700 1 -- start start 2022-06-17T12:49:16.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.253+0000 7fe9dea7e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9d8062470 0x7fe9d81467e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:16.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.253+0000 7fe9dea7e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe9d80b4820 0x7fe9d81421d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:16.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.253+0000 7fe9dea7e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9d8143a90 0x7fe9d81426c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:16.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.253+0000 7fe9dea7e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d813f8b0 con 0x7fe9d80b4820 2022-06-17T12:49:16.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.253+0000 7fe9dea7e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d813e420 con 0x7fe9d8143a90 2022-06-17T12:49:16.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.253+0000 7fe9dea7e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d813fd30 con 0x7fe9d8062470 2022-06-17T12:49:16.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.253+0000 7fe9dd27b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe9d80b4820 0x7fe9d81421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:16.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.253+0000 7fe9dd27b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe9d80b4820 0x7fe9d81421d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33436/0 (socket says 172.21.15.36:33436) 2022-06-17T12:49:16.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.253+0000 7fe9dda7c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9d8062470 0x7fe9d81467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:16.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.253+0000 7fe9dd27b700 1 -- 172.21.15.36:0/1064904905 learned_addr learned my addr 172.21.15.36:0/1064904905 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:16.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.253+0000 7fe9de27d700 1 --2- 172.21.15.36:0/1064904905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9d8143a90 0x7fe9d81426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:16.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.253+0000 7fe9dd27b700 1 -- 172.21.15.36:0/1064904905 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9d8062470 msgr2=0x7fe9d81467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:16.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.253+0000 7fe9dd27b700 1 --2- 172.21.15.36:0/1064904905 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9d8062470 0x7fe9d81467e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:16.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.253+0000 7fe9dd27b700 1 -- 172.21.15.36:0/1064904905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9d8143a90 msgr2=0x7fe9d81426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:16.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.253+0000 7fe9dd27b700 1 --2- 172.21.15.36:0/1064904905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9d8143a90 0x7fe9d81426c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:16.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.253+0000 7fe9dd27b700 1 -- 172.21.15.36:0/1064904905 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d400b180 con 0x7fe9d80b4820 2022-06-17T12:49:16.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.253+0000 7fe9dd27b700 1 --2- 172.21.15.36:0/1064904905 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe9d80b4820 0x7fe9d81421d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7fe9c8000bc0 tx=0x7fe9c8007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:16.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.257+0000 7fe9c6ffd700 1 -- 172.21.15.36:0/1064904905 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9c80070d0 con 0x7fe9d80b4820 2022-06-17T12:49:16.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.257+0000 7fe9c6ffd700 1 -- 172.21.15.36:0/1064904905 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c8007230 con 0x7fe9d80b4820 2022-06-17T12:49:16.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.257+0000 7fe9dea7e700 1 -- 172.21.15.36:0/1064904905 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d80671b0 con 0x7fe9d80b4820 2022-06-17T12:49:16.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.257+0000 7fe9c6ffd700 1 -- 172.21.15.36:0/1064904905 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9c8005610 con 0x7fe9d80b4820 2022-06-17T12:49:16.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.257+0000 7fe9dea7e700 1 -- 172.21.15.36:0/1064904905 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d813e070 con 0x7fe9d80b4820 2022-06-17T12:49:16.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.257+0000 7fe9dea7e700 1 -- 172.21.15.36:0/1064904905 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d813dcf0 con 0x7fe9d80b4820 2022-06-17T12:49:16.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.257+0000 7fe9c6ffd700 1 -- 172.21.15.36:0/1064904905 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe9c801a060 con 0x7fe9d80b4820 2022-06-17T12:49:16.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.257+0000 7fe9c6ffd700 1 --2- 172.21.15.36:0/1064904905 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe9b8053af0 0x7fe9b8055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:16.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.257+0000 7fe9dda7c700 1 --2- 172.21.15.36:0/1064904905 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe9b8053af0 0x7fe9b8055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:16.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.257+0000 7fe9c6ffd700 1 -- 172.21.15.36:0/1064904905 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe9c806b610 con 0x7fe9d80b4820 2022-06-17T12:49:16.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.257+0000 7fe9dda7c700 1 --2- 172.21.15.36:0/1064904905 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe9b8053af0 0x7fe9b8055fb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fe9d401c000 tx=0x7fe9d4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:16.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.265+0000 7fe9c6ffd700 1 -- 172.21.15.36:0/1064904905 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe9c8042410 con 0x7fe9d80b4820 2022-06-17T12:49:16.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.433+0000 7fe9dea7e700 1 -- 172.21.15.36:0/1064904905 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fe9d8143350 con 0x7fe9d80b4820 2022-06-17T12:49:16.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.441+0000 7fe9c6ffd700 1 -- 172.21.15.36:0/1064904905 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v17) v1 ==== 184+0+0 (secure 0 0 0) 0x7fe9c8018070 con 0x7fe9d80b4820 2022-06-17T12:49:16.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.445+0000 7fe9dea7e700 1 -- 172.21.15.36:0/1064904905 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe9b8053af0 msgr2=0x7fe9b8055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:16.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.445+0000 7fe9dea7e700 1 --2- 172.21.15.36:0/1064904905 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe9b8053af0 0x7fe9b8055fb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fe9d401c000 tx=0x7fe9d4015000).stop 2022-06-17T12:49:16.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.445+0000 7fe9dea7e700 1 -- 172.21.15.36:0/1064904905 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe9d80b4820 msgr2=0x7fe9d81421d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:16.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.445+0000 7fe9dea7e700 1 --2- 172.21.15.36:0/1064904905 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe9d80b4820 0x7fe9d81421d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7fe9c8000bc0 tx=0x7fe9c8007a60).stop 2022-06-17T12:49:16.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.445+0000 7fe9dea7e700 1 -- 172.21.15.36:0/1064904905 shutdown_connections 2022-06-17T12:49:16.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.445+0000 7fe9dea7e700 1 --2- 172.21.15.36:0/1064904905 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe9b8053af0 0x7fe9b8055fb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:16.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.445+0000 7fe9dea7e700 1 --2- 172.21.15.36:0/1064904905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9d8143a90 0x7fe9d81426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:16.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.445+0000 7fe9dea7e700 1 --2- 172.21.15.36:0/1064904905 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe9d80b4820 0x7fe9d81421d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:16.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.445+0000 7fe9dea7e700 1 --2- 172.21.15.36:0/1064904905 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9d8062470 0x7fe9d81467e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:16.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.445+0000 7fe9dea7e700 1 -- 172.21.15.36:0/1064904905 >> 172.21.15.36:0/1064904905 conn(0x7fe9d805c550 msgr2=0x7fe9d80b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:16.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.445+0000 7fe9dea7e700 1 -- 172.21.15.36:0/1064904905 shutdown_connections 2022-06-17T12:49:16.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.445+0000 7fe9dea7e700 1 -- 172.21.15.36:0/1064904905 wait complete. 2022-06-17T12:49:16.479 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.1 2022-06-17T12:49:16.482 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:49:16.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.637+0000 7fac16617700 1 Processor -- start 2022-06-17T12:49:16.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.637+0000 7fac16617700 1 -- start start 2022-06-17T12:49:16.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.637+0000 7fac16617700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac10148050 0x7fac100b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:16.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.637+0000 7fac16617700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fac100adc80 0x7fac101460f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:16.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.637+0000 7fac16617700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac100ae0d0 con 0x7fac10060ac0 2022-06-17T12:49:16.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.637+0000 7fac16617700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac10060ea0 con 0x7fac100adc80 2022-06-17T12:49:16.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.637+0000 7fac16617700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac100ad920 con 0x7fac10148050 2022-06-17T12:49:16.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.637+0000 7fac15e16700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fac100adc80 0x7fac101460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:16.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.637+0000 7fac15e16700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fac100adc80 0x7fac101460f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33446/0 (socket says 172.21.15.36:33446) 2022-06-17T12:49:16.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.637+0000 7fac15e16700 1 -- 172.21.15.36:0/848742809 learned_addr learned my addr 172.21.15.36:0/848742809 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:16.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.637+0000 7fac14e14700 1 --2- 172.21.15.36:0/848742809 >> v2:172.21.15.70:3300/0 conn(0x7fac10148050 0x7fac100b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:16.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.637+0000 7fac15e16700 1 -- 172.21.15.36:0/848742809 >> v2:172.21.15.70:3300/0 conn(0x7fac10148050 msgr2=0x7fac100b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:16.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.637+0000 7fac15e16700 1 --2- 172.21.15.36:0/848742809 >> v2:172.21.15.70:3300/0 conn(0x7fac10148050 0x7fac100b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:16.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.637+0000 7fac15e16700 1 -- 172.21.15.36:0/848742809 >> v1:172.21.15.70:6789/0 conn(0x7fac10060ac0 legacy=0x7fac10147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:16.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.637+0000 7fac15e16700 1 -- 172.21.15.36:0/848742809 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac101474a0 con 0x7fac100adc80 2022-06-17T12:49:16.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.637+0000 7fac15e16700 1 --2- 172.21.15.36:0/848742809 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fac100adc80 0x7fac101460f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fac080032f0 tx=0x7fac08009290).ready entity=mon.0 client_cookie=eebbd2249ed73311 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:16.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.641+0000 7fac03fff700 1 -- 172.21.15.36:0/848742809 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac0800b870 con 0x7fac100adc80 2022-06-17T12:49:16.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.641+0000 7fac03fff700 1 -- 172.21.15.36:0/848742809 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0800b9d0 con 0x7fac100adc80 2022-06-17T12:49:16.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.641+0000 7fac03fff700 1 -- 172.21.15.36:0/848742809 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac0800bb40 con 0x7fac100adc80 2022-06-17T12:49:16.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.641+0000 7fac16617700 1 -- 172.21.15.36:0/848742809 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fac100adc80 msgr2=0x7fac101460f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:16.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.641+0000 7fac16617700 1 --2- 172.21.15.36:0/848742809 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fac100adc80 0x7fac101460f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fac080032f0 tx=0x7fac08009290).stop 2022-06-17T12:49:16.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.641+0000 7fac16617700 1 -- 172.21.15.36:0/848742809 shutdown_connections 2022-06-17T12:49:16.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.641+0000 7fac16617700 1 --2- 172.21.15.36:0/848742809 >> v2:172.21.15.70:3300/0 conn(0x7fac10148050 0x7fac100b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:16.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.641+0000 7fac16617700 1 --2- 172.21.15.36:0/848742809 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fac100adc80 0x7fac101460f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:16.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.641+0000 7fac16617700 1 -- 172.21.15.36:0/848742809 >> 172.21.15.36:0/848742809 conn(0x7fac100b65f0 msgr2=0x7fac100b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:16.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.641+0000 7fac16617700 1 -- 172.21.15.36:0/848742809 shutdown_connections 2022-06-17T12:49:16.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.641+0000 7fac16617700 1 -- 172.21.15.36:0/848742809 wait complete. 2022-06-17T12:49:16.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.641+0000 7fac16617700 1 Processor -- start 2022-06-17T12:49:16.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.641+0000 7fac16617700 1 -- start start 2022-06-17T12:49:16.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.641+0000 7fac16617700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac10060ac0 0x7fac1014ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:16.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.641+0000 7fac16617700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac100adc80 0x7fac101453a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:16.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.641+0000 7fac16617700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fac10148050 0x7fac10064380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:16.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.641+0000 7fac16617700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac10144ca0 con 0x7fac10148050 2022-06-17T12:49:16.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.641+0000 7fac16617700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac10144f00 con 0x7fac100adc80 2022-06-17T12:49:16.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.641+0000 7fac16617700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac10143ef0 con 0x7fac10060ac0 2022-06-17T12:49:16.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.641+0000 7fac15e16700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fac10148050 0x7fac10064380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:16.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.641+0000 7fac15e16700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fac10148050 0x7fac10064380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33452/0 (socket says 172.21.15.36:33452) 2022-06-17T12:49:16.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.641+0000 7fac15e16700 1 -- 172.21.15.36:0/2360319308 learned_addr learned my addr 172.21.15.36:0/2360319308 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:16.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.641+0000 7fac15615700 1 --2- 172.21.15.36:0/2360319308 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac10060ac0 0x7fac1014ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:16.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.641+0000 7fac14e14700 1 --2- 172.21.15.36:0/2360319308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac100adc80 0x7fac101453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:16.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.645+0000 7fac15e16700 1 -- 172.21.15.36:0/2360319308 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac10060ac0 msgr2=0x7fac1014ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:16.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.645+0000 7fac15e16700 1 --2- 172.21.15.36:0/2360319308 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac10060ac0 0x7fac1014ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:16.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.645+0000 7fac15e16700 1 -- 172.21.15.36:0/2360319308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac100adc80 msgr2=0x7fac101453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:16.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.645+0000 7fac15e16700 1 --2- 172.21.15.36:0/2360319308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac100adc80 0x7fac101453a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:16.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.645+0000 7fac15e16700 1 -- 172.21.15.36:0/2360319308 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac08009530 con 0x7fac10148050 2022-06-17T12:49:16.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.645+0000 7fac15e16700 1 --2- 172.21.15.36:0/2360319308 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fac10148050 0x7fac10064380 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fac080032f0 tx=0x7fac08016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:16.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.645+0000 7fac02ffd700 1 -- 172.21.15.36:0/2360319308 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac0800b870 con 0x7fac10148050 2022-06-17T12:49:16.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.645+0000 7fac02ffd700 1 -- 172.21.15.36:0/2360319308 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac08021480 con 0x7fac10148050 2022-06-17T12:49:16.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.645+0000 7fac16617700 1 -- 172.21.15.36:0/2360319308 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac10067c10 con 0x7fac10148050 2022-06-17T12:49:16.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.645+0000 7fac02ffd700 1 -- 172.21.15.36:0/2360319308 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac080217e0 con 0x7fac10148050 2022-06-17T12:49:16.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.645+0000 7fac16617700 1 -- 172.21.15.36:0/2360319308 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac10067770 con 0x7fac10148050 2022-06-17T12:49:16.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.645+0000 7fac16617700 1 -- 172.21.15.36:0/2360319308 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac10067a80 con 0x7fac10148050 2022-06-17T12:49:16.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.649+0000 7fac02ffd700 1 -- 172.21.15.36:0/2360319308 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fac08028070 con 0x7fac10148050 2022-06-17T12:49:16.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.653+0000 7fac02ffd700 1 --2- 172.21.15.36:0/2360319308 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fabec053ad0 0x7fabec055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:16.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.653+0000 7fac02ffd700 1 -- 172.21.15.36:0/2360319308 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fac0806d000 con 0x7fac10148050 2022-06-17T12:49:16.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.653+0000 7fac15615700 1 --2- 172.21.15.36:0/2360319308 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fabec053ad0 0x7fabec055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:16.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.653+0000 7fac15615700 1 --2- 172.21.15.36:0/2360319308 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fabec053ad0 0x7fabec055f90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fac0c00d000 tx=0x7fac0c000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:16.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.653+0000 7fac02ffd700 1 -- 172.21.15.36:0/2360319308 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fac08043eb0 con 0x7fac10148050 2022-06-17T12:49:16.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.821+0000 7fac16617700 1 -- 172.21.15.36:0/2360319308 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fac101450c0 con 0x7fac10148050 2022-06-17T12:49:16.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.829+0000 7fac02ffd700 1 -- 172.21.15.36:0/2360319308 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v18) v1 ==== 184+0+0 (secure 0 0 0) 0x7fac080260e0 con 0x7fac10148050 2022-06-17T12:49:16.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.833+0000 7fac16617700 1 -- 172.21.15.36:0/2360319308 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fabec053ad0 msgr2=0x7fabec055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:16.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.833+0000 7fac16617700 1 --2- 172.21.15.36:0/2360319308 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fabec053ad0 0x7fabec055f90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fac0c00d000 tx=0x7fac0c000bc0).stop 2022-06-17T12:49:16.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.833+0000 7fac16617700 1 -- 172.21.15.36:0/2360319308 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fac10148050 msgr2=0x7fac10064380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:16.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.833+0000 7fac16617700 1 --2- 172.21.15.36:0/2360319308 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fac10148050 0x7fac10064380 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fac080032f0 tx=0x7fac08016000).stop 2022-06-17T12:49:16.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.833+0000 7fac16617700 1 -- 172.21.15.36:0/2360319308 shutdown_connections 2022-06-17T12:49:16.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.833+0000 7fac16617700 1 --2- 172.21.15.36:0/2360319308 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fabec053ad0 0x7fabec055f90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:16.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.833+0000 7fac16617700 1 --2- 172.21.15.36:0/2360319308 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fac10148050 0x7fac10064380 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:16.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.833+0000 7fac16617700 1 --2- 172.21.15.36:0/2360319308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac100adc80 0x7fac101453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:16.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.833+0000 7fac16617700 1 --2- 172.21.15.36:0/2360319308 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac10060ac0 0x7fac1014ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:16.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.833+0000 7fac16617700 1 -- 172.21.15.36:0/2360319308 >> 172.21.15.36:0/2360319308 conn(0x7fac100b65f0 msgr2=0x7fac100b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:16.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.833+0000 7fac16617700 1 -- 172.21.15.36:0/2360319308 shutdown_connections 2022-06-17T12:49:16.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.833+0000 7fac16617700 1 -- 172.21.15.36:0/2360319308 wait complete. 2022-06-17T12:49:16.857 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.2 2022-06-17T12:49:16.857 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:49:16.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.989+0000 7f050eed8700 1 Processor -- start 2022-06-17T12:49:16.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.989+0000 7f050eed8700 1 -- start start 2022-06-17T12:49:16.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.989+0000 7f050eed8700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f05080b2200 0x7f05080af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:16.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.989+0000 7f050eed8700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05080b25e0 con 0x7f05080af8d0 2022-06-17T12:49:16.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.989+0000 7f050eed8700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f050814b6f0 con 0x7f05080b2cd0 2022-06-17T12:49:16.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.989+0000 7f050eed8700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05080ad790 con 0x7f05080b2200 2022-06-17T12:49:16.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.989+0000 7f050ded6700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f05080b2200 0x7f05080af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:17.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.989+0000 7f050d6d5700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f05080af8d0 0x7f050814a8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:50986/0 (socket says 172.21.15.36:50986) 2022-06-17T12:49:17.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.989+0000 7f050d6d5700 1 -- 172.21.15.36:0/2886517237 learned_addr learned my addr 172.21.15.36:0/2886517237 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:17.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.989+0000 7f050ced4700 1 -- 172.21.15.36:0/2886517237 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966800339 0 0) 0x7f05080b25e0 con 0x7f05080af8d0 2022-06-17T12:49:17.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.989+0000 7f050ced4700 1 -- 172.21.15.36:0/2886517237 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04f0003260 con 0x7f05080af8d0 2022-06-17T12:49:17.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.989+0000 7f050ced4700 1 -- 172.21.15.36:0/2886517237 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2649760960 0 0) 0x7f050814b6f0 con 0x7f05080b2cd0 2022-06-17T12:49:17.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.989+0000 7f050ced4700 1 -- 172.21.15.36:0/2886517237 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05080b25e0 con 0x7f05080b2cd0 2022-06-17T12:49:17.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.989+0000 7f050ded6700 1 -- 172.21.15.36:0/2886517237 >> v1:172.21.15.70:6789/0 conn(0x7f05080b2cd0 legacy=0x7f050814afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:17.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.989+0000 7f050ded6700 1 -- 172.21.15.36:0/2886517237 >> v1:172.21.15.36:6789/0 conn(0x7f05080af8d0 legacy=0x7f050814a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:17.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.989+0000 7f050ded6700 1 -- 172.21.15.36:0/2886517237 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05080aef40 con 0x7f05080b2200 2022-06-17T12:49:17.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.989+0000 7f050ded6700 1 --2- 172.21.15.36:0/2886517237 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05080b2200 0x7f05080af3e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f04f8002f70 tx=0x7f04f8006df0).ready entity=mon.0 client_cookie=f064acbc0181eb10 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:17.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.989+0000 7f050ced4700 1 -- 172.21.15.36:0/2886517237 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04f800d540 con 0x7f05080b2200 2022-06-17T12:49:17.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.989+0000 7f050ced4700 1 -- 172.21.15.36:0/2886517237 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f8000e90 con 0x7f05080b2200 2022-06-17T12:49:17.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.989+0000 7f050ced4700 1 -- 172.21.15.36:0/2886517237 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04f800d890 con 0x7f05080b2200 2022-06-17T12:49:17.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050eed8700 1 -- 172.21.15.36:0/2886517237 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05080b2200 msgr2=0x7f05080af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:17.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050eed8700 1 --2- 172.21.15.36:0/2886517237 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05080b2200 0x7f05080af3e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f04f8002f70 tx=0x7f04f8006df0).stop 2022-06-17T12:49:17.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050eed8700 1 -- 172.21.15.36:0/2886517237 shutdown_connections 2022-06-17T12:49:17.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050eed8700 1 --2- 172.21.15.36:0/2886517237 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05080b2200 0x7f05080af3e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:17.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050eed8700 1 -- 172.21.15.36:0/2886517237 >> 172.21.15.36:0/2886517237 conn(0x7f050805f770 msgr2=0x7f050805fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:17.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050eed8700 1 -- 172.21.15.36:0/2886517237 shutdown_connections 2022-06-17T12:49:17.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050eed8700 1 -- 172.21.15.36:0/2886517237 wait complete. 2022-06-17T12:49:17.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050eed8700 1 Processor -- start 2022-06-17T12:49:17.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050eed8700 1 -- start start 2022-06-17T12:49:17.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050eed8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05080af8d0 0x7f05081478a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:17.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050eed8700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f05080b2200 0x7f05080655e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:17.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050eed8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05080b2cd0 0x7f0508156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:17.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050eed8700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0508159c10 con 0x7f05080b2cd0 2022-06-17T12:49:17.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050eed8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0508159e40 con 0x7f05080af8d0 2022-06-17T12:49:17.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050eed8700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0508156910 con 0x7f05080b2200 2022-06-17T12:49:17.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050e6d7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05080b2cd0 0x7f0508156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:17.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050e6d7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05080b2cd0 0x7f0508156db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33466/0 (socket says 172.21.15.36:33466) 2022-06-17T12:49:17.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050e6d7700 1 -- 172.21.15.36:0/3522638565 learned_addr learned my addr 172.21.15.36:0/3522638565 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:17.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050d6d5700 1 --2- 172.21.15.36:0/3522638565 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f05080b2200 0x7f05080655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:17.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050ded6700 1 --2- 172.21.15.36:0/3522638565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05080af8d0 0x7f05081478a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:17.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050e6d7700 1 -- 172.21.15.36:0/3522638565 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f05080b2200 msgr2=0x7f05080655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:17.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050e6d7700 1 --2- 172.21.15.36:0/3522638565 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f05080b2200 0x7f05080655e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:17.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050e6d7700 1 -- 172.21.15.36:0/3522638565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05080af8d0 msgr2=0x7f05081478a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:17.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050e6d7700 1 --2- 172.21.15.36:0/3522638565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05080af8d0 0x7f05081478a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:17.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050e6d7700 1 -- 172.21.15.36:0/3522638565 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f800b180 con 0x7f05080b2cd0 2022-06-17T12:49:17.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050e6d7700 1 --2- 172.21.15.36:0/3522638565 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05080b2cd0 0x7f0508156db0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f0500001110 tx=0x7f0500007a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:17.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f04f6ffd700 1 -- 172.21.15.36:0/3522638565 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f050000ce30 con 0x7f05080b2cd0 2022-06-17T12:49:17.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f04f6ffd700 1 -- 172.21.15.36:0/3522638565 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0500012400 con 0x7f05080b2cd0 2022-06-17T12:49:17.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050eed8700 1 -- 172.21.15.36:0/3522638565 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0508144af0 con 0x7f05080b2cd0 2022-06-17T12:49:17.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f04f6ffd700 1 -- 172.21.15.36:0/3522638565 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0500012760 con 0x7f05080b2cd0 2022-06-17T12:49:17.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.993+0000 7f050eed8700 1 -- 172.21.15.36:0/3522638565 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0508156610 con 0x7f05080b2cd0 2022-06-17T12:49:17.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.997+0000 7f04f6ffd700 1 -- 172.21.15.36:0/3522638565 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f0500019030 con 0x7f05080b2cd0 2022-06-17T12:49:17.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.997+0000 7f050eed8700 1 -- 172.21.15.36:0/3522638565 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05080b1950 con 0x7f05080b2cd0 2022-06-17T12:49:17.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.997+0000 7f04f6ffd700 1 --2- 172.21.15.36:0/3522638565 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f04e8053a40 0x7f04e8055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:17.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.997+0000 7f04f6ffd700 1 -- 172.21.15.36:0/3522638565 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f050006b270 con 0x7f05080b2cd0 2022-06-17T12:49:17.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.997+0000 7f050ded6700 1 --2- 172.21.15.36:0/3522638565 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f04e8053a40 0x7f04e8055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:17.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:16.997+0000 7f050ded6700 1 --2- 172.21.15.36:0/3522638565 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f04e8053a40 0x7f04e8055f00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f04f8018000 tx=0x7f04f8014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:17.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.005+0000 7f04f6ffd700 1 -- 172.21.15.36:0/3522638565 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0500042070 con 0x7f05080b2cd0 2022-06-17T12:49:17.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.169+0000 7f050eed8700 1 -- 172.21.15.36:0/3522638565 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0508057410 con 0x7f05080b2cd0 2022-06-17T12:49:17.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.169+0000 7f04f6ffd700 1 -- 172.21.15.36:0/3522638565 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+2138 (secure 0 0 0) 0x7f0500045690 con 0x7f05080b2cd0 2022-06-17T12:49:17.185 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:49:17.185 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":19,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/3777038413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3777038413},{"type":"v1","addr":"172.21.15.36:6835","nonce":3777038413}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:49:14.023351+0000","modified":"2022-06-17T12:49:15.026177+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5292":{"gid":5292,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/4117532324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4117532324},{"type":"v1","addr":"172.21.15.70:6833","nonce":4117532324}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-06-17T12:49:17.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.173+0000 7f050eed8700 1 -- 172.21.15.36:0/3522638565 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f04e8053a40 msgr2=0x7f04e8055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:17.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.173+0000 7f050eed8700 1 --2- 172.21.15.36:0/3522638565 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f04e8053a40 0x7f04e8055f00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f04f8018000 tx=0x7f04f8014000).stop 2022-06-17T12:49:17.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.173+0000 7f050eed8700 1 -- 172.21.15.36:0/3522638565 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05080b2cd0 msgr2=0x7f0508156db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:17.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.173+0000 7f050eed8700 1 --2- 172.21.15.36:0/3522638565 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05080b2cd0 0x7f0508156db0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f0500001110 tx=0x7f0500007a50).stop 2022-06-17T12:49:17.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.173+0000 7f050eed8700 1 -- 172.21.15.36:0/3522638565 shutdown_connections 2022-06-17T12:49:17.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.173+0000 7f050eed8700 1 --2- 172.21.15.36:0/3522638565 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f04e8053a40 0x7f04e8055f00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:17.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.173+0000 7f050eed8700 1 --2- 172.21.15.36:0/3522638565 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05080b2cd0 0x7f0508156db0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:17.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.173+0000 7f050eed8700 1 --2- 172.21.15.36:0/3522638565 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f05080b2200 0x7f05080655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:17.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.173+0000 7f050eed8700 1 --2- 172.21.15.36:0/3522638565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05080af8d0 0x7f05081478a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:17.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.173+0000 7f050eed8700 1 -- 172.21.15.36:0/3522638565 >> 172.21.15.36:0/3522638565 conn(0x7f050805f770 msgr2=0x7f05081439a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:17.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.173+0000 7f050eed8700 1 -- 172.21.15.36:0/3522638565 shutdown_connections 2022-06-17T12:49:17.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.173+0000 7f050eed8700 1 -- 172.21.15.36:0/3522638565 wait complete. 2022-06-17T12:49:17.197 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 19 2022-06-17T12:49:17.407 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:49:14.023351+0000', 'modified': '2022-06-17T12:49:15.026177+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5292}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5292': {'gid': 5292, 'name': 'b', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6833/4117532324', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 4117532324}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 4117532324}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:49:17.408 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:49:17.409 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-17T12:49:17.560 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":5292,"want_state":"up:active","state":"up:active","rank_uptime":3.5353871809999999,"mdsmap_epoch":19,"osdmap_epoch":45,"osdmap_epoch_barrier":44,"uptime":8.3078170609999997} 2022-06-17T12:49:17.561 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 5292, "mdsmap_epoch": 19, "osdmap_epoch": 45, "osdmap_epoch_barrier": 44, "rank_uptime": 3.535387181, "state": "up:active", "uptime": 8.307817061, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:49:17.569 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:49:17.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.721+0000 7ff07c8ec700 1 Processor -- start 2022-06-17T12:49:17.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.721+0000 7ff07c8ec700 1 -- start start 2022-06-17T12:49:17.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff07c8ec700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff07813cb70 0x7ff078143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:17.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff07c8ec700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff07813cf50 0x7ff0781442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:17.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff07c8ec700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0780af4b0 con 0x7ff0780aef60 2022-06-17T12:49:17.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff07c8ec700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0780ba500 con 0x7ff07813cb70 2022-06-17T12:49:17.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff07c8ec700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff078142f50 con 0x7ff07813cf50 2022-06-17T12:49:17.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff076d9d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff07813cb70 0x7ff078143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:17.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff076d9d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff07813cb70 0x7ff078143690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33470/0 (socket says 172.21.15.36:33470) 2022-06-17T12:49:17.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff076d9d700 1 -- 172.21.15.36:0/2460032687 learned_addr learned my addr 172.21.15.36:0/2460032687 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:17.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff07759e700 1 --2- 172.21.15.36:0/2460032687 >> v2:172.21.15.70:3300/0 conn(0x7ff07813cf50 0x7ff0781442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:17.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff076d9d700 1 -- 172.21.15.36:0/2460032687 >> v2:172.21.15.70:3300/0 conn(0x7ff07813cf50 msgr2=0x7ff0781442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:17.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff076d9d700 1 --2- 172.21.15.36:0/2460032687 >> v2:172.21.15.70:3300/0 conn(0x7ff07813cf50 0x7ff0781442a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:17.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff076d9d700 1 -- 172.21.15.36:0/2460032687 >> v1:172.21.15.70:6789/0 conn(0x7ff0780aef60 legacy=0x7ff0780b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:17.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff076d9d700 1 -- 172.21.15.36:0/2460032687 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0781431f0 con 0x7ff07813cb70 2022-06-17T12:49:17.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff07759e700 1 --2- 172.21.15.36:0/2460032687 >> v2:172.21.15.70:3300/0 conn(0x7ff07813cf50 0x7ff0781442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:49:17.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff076d9d700 1 --2- 172.21.15.36:0/2460032687 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff07813cb70 0x7ff078143690 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7ff0700032f0 tx=0x7ff070009290).ready entity=mon.0 client_cookie=6218375a3f54bae1 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:17.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff075d9b700 1 -- 172.21.15.36:0/2460032687 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff07000b870 con 0x7ff07813cb70 2022-06-17T12:49:17.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff075d9b700 1 -- 172.21.15.36:0/2460032687 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07000b9d0 con 0x7ff07813cb70 2022-06-17T12:49:17.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff075d9b700 1 -- 172.21.15.36:0/2460032687 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff07000bb40 con 0x7ff07813cb70 2022-06-17T12:49:17.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff07c8ec700 1 -- 172.21.15.36:0/2460032687 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff07813cb70 msgr2=0x7ff078143690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:17.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff07c8ec700 1 --2- 172.21.15.36:0/2460032687 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff07813cb70 0x7ff078143690 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7ff0700032f0 tx=0x7ff070009290).stop 2022-06-17T12:49:17.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff07c8ec700 1 -- 172.21.15.36:0/2460032687 shutdown_connections 2022-06-17T12:49:17.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff07c8ec700 1 --2- 172.21.15.36:0/2460032687 >> v2:172.21.15.70:3300/0 conn(0x7ff07813cf50 0x7ff0781442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:17.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff07c8ec700 1 --2- 172.21.15.36:0/2460032687 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff07813cb70 0x7ff078143690 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:17.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff07c8ec700 1 -- 172.21.15.36:0/2460032687 >> 172.21.15.36:0/2460032687 conn(0x7ff07805c550 msgr2=0x7ff0780b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:17.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff07c8ec700 1 -- 172.21.15.36:0/2460032687 shutdown_connections 2022-06-17T12:49:17.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.725+0000 7ff07c8ec700 1 -- 172.21.15.36:0/2460032687 wait complete. 2022-06-17T12:49:17.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff07c8ec700 1 Processor -- start 2022-06-17T12:49:17.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff07c8ec700 1 -- start start 2022-06-17T12:49:17.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff07c8ec700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0780aef60 0x7ff078155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:17.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff07c8ec700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff07813cb70 0x7ff078145760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:17.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff07c8ec700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff07813cf50 0x7ff0780b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:17.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff07c8ec700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0780afc00 con 0x7ff07813cb70 2022-06-17T12:49:17.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff07c8ec700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0780b0990 con 0x7ff0780aef60 2022-06-17T12:49:17.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff07c8ec700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0780b0c30 con 0x7ff07813cf50 2022-06-17T12:49:17.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff07659c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff07813cb70 0x7ff078145760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:17.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff07759e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff07813cf50 0x7ff0780b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:17.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff07659c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff07813cb70 0x7ff078145760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33478/0 (socket says 172.21.15.36:33478) 2022-06-17T12:49:17.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff076d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0780aef60 0x7ff078155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:17.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff07659c700 1 -- 172.21.15.36:0/2143884179 learned_addr learned my addr 172.21.15.36:0/2143884179 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:17.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff07659c700 1 -- 172.21.15.36:0/2143884179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff07813cf50 msgr2=0x7ff0780b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:17.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff07659c700 1 --2- 172.21.15.36:0/2143884179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff07813cf50 0x7ff0780b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:17.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff07659c700 1 -- 172.21.15.36:0/2143884179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0780aef60 msgr2=0x7ff078155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:17.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff07659c700 1 --2- 172.21.15.36:0/2143884179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0780aef60 0x7ff078155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:17.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff07659c700 1 -- 172.21.15.36:0/2143884179 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff070009530 con 0x7ff07813cb70 2022-06-17T12:49:17.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff076d9d700 1 --2- 172.21.15.36:0/2143884179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0780aef60 0x7ff078155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:49:17.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff07759e700 1 --2- 172.21.15.36:0/2143884179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff07813cf50 0x7ff0780b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:49:17.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff07659c700 1 --2- 172.21.15.36:0/2143884179 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff07813cb70 0x7ff078145760 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7ff078140960 tx=0x7ff06c007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:17.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff067fff700 1 -- 172.21.15.36:0/2143884179 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff06c012400 con 0x7ff07813cb70 2022-06-17T12:49:17.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff067fff700 1 -- 172.21.15.36:0/2143884179 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06c012560 con 0x7ff07813cb70 2022-06-17T12:49:17.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff067fff700 1 -- 172.21.15.36:0/2143884179 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff06c0128c0 con 0x7ff07813cb70 2022-06-17T12:49:17.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff07c8ec700 1 -- 172.21.15.36:0/2143884179 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff07813f800 con 0x7ff07813cb70 2022-06-17T12:49:17.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.729+0000 7ff07c8ec700 1 -- 172.21.15.36:0/2143884179 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff078112a70 con 0x7ff07813cb70 2022-06-17T12:49:17.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.733+0000 7ff07c8ec700 1 -- 172.21.15.36:0/2143884179 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff078057410 con 0x7ff07813cb70 2022-06-17T12:49:17.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.733+0000 7ff067fff700 1 -- 172.21.15.36:0/2143884179 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff06c019030 con 0x7ff07813cb70 2022-06-17T12:49:17.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.733+0000 7ff067fff700 1 --2- 172.21.15.36:0/2143884179 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff054053ad0 0x7ff054055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:17.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.733+0000 7ff067fff700 1 -- 172.21.15.36:0/2143884179 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff06c06b390 con 0x7ff07813cb70 2022-06-17T12:49:17.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.733+0000 7ff076d9d700 1 --2- 172.21.15.36:0/2143884179 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff054053ad0 0x7ff054055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:17.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.733+0000 7ff076d9d700 1 --2- 172.21.15.36:0/2143884179 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff054053ad0 0x7ff054055f90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7ff07001a000 tx=0x7ff070016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:17.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.741+0000 7ff067fff700 1 -- 172.21.15.36:0/2143884179 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff06c0420f0 con 0x7ff07813cb70 2022-06-17T12:49:17.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.861+0000 7ff07c8ec700 1 -- 172.21.15.36:0/2143884179 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0780b3c90 con 0x7ff07813cb70 2022-06-17T12:49:17.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.865+0000 7ff067fff700 1 -- 172.21.15.36:0/2143884179 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16271 (secure 0 0 0) 0x7ff06c045710 con 0x7ff07813cb70 2022-06-17T12:49:17.853 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:49:17.854 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":45,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:49:15.026771+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:49:12.581297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-06-17T12:49:12.985806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:49:17.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.865+0000 7ff07c8ec700 1 -- 172.21.15.36:0/2143884179 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff054053ad0 msgr2=0x7ff054055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:17.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.865+0000 7ff07c8ec700 1 --2- 172.21.15.36:0/2143884179 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff054053ad0 0x7ff054055f90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7ff07001a000 tx=0x7ff070016000).stop 2022-06-17T12:49:17.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.865+0000 7ff07c8ec700 1 -- 172.21.15.36:0/2143884179 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff07813cb70 msgr2=0x7ff078145760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:17.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.865+0000 7ff07c8ec700 1 --2- 172.21.15.36:0/2143884179 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff07813cb70 0x7ff078145760 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7ff078140960 tx=0x7ff06c007c50).stop 2022-06-17T12:49:17.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.865+0000 7ff07c8ec700 1 -- 172.21.15.36:0/2143884179 shutdown_connections 2022-06-17T12:49:17.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.865+0000 7ff07c8ec700 1 --2- 172.21.15.36:0/2143884179 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff054053ad0 0x7ff054055f90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:17.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.865+0000 7ff07c8ec700 1 --2- 172.21.15.36:0/2143884179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff07813cf50 0x7ff0780b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:17.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.865+0000 7ff07c8ec700 1 --2- 172.21.15.36:0/2143884179 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff07813cb70 0x7ff078145760 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:17.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.865+0000 7ff07c8ec700 1 --2- 172.21.15.36:0/2143884179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0780aef60 0x7ff078155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:17.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.865+0000 7ff07c8ec700 1 -- 172.21.15.36:0/2143884179 >> 172.21.15.36:0/2143884179 conn(0x7ff07805c550 msgr2=0x7ff07805f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:17.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.865+0000 7ff07c8ec700 1 -- 172.21.15.36:0/2143884179 shutdown_connections 2022-06-17T12:49:17.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:17.865+0000 7ff07c8ec700 1 -- 172.21.15.36:0/2143884179 wait complete. 2022-06-17T12:49:17.881 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:49:12.581297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:49:12.985806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:49:17.903 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:49:18.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.045+0000 7faa8dbf3700 1 Processor -- start 2022-06-17T12:49:18.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.045+0000 7faa8dbf3700 1 -- start start 2022-06-17T12:49:18.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.045+0000 7faa8dbf3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa880b65f0 0x7faa880b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:18.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.045+0000 7faa8dbf3700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7faa880b15b0 0x7faa880af630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:18.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.045+0000 7faa8dbf3700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa8813fdf0 con 0x7faa880adf10 2022-06-17T12:49:18.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.045+0000 7faa8dbf3700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa880ae8e0 con 0x7faa880b15b0 2022-06-17T12:49:18.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.045+0000 7faa8dbf3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa88140090 con 0x7faa880b65f0 2022-06-17T12:49:18.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.045+0000 7faa8d3f2700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7faa880b15b0 0x7faa880af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:18.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.045+0000 7faa8d3f2700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7faa880b15b0 0x7faa880af630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33488/0 (socket says 172.21.15.36:33488) 2022-06-17T12:49:18.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.045+0000 7faa8d3f2700 1 -- 172.21.15.36:0/1187459401 learned_addr learned my addr 172.21.15.36:0/1187459401 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:18.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.045+0000 7faa8cbf1700 1 --2- 172.21.15.36:0/1187459401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa880b65f0 0x7faa880b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:18.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.045+0000 7faa877fe700 1 -- 172.21.15.36:0/1187459401 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2547131466 0 0) 0x7faa8813fdf0 con 0x7faa880adf10 2022-06-17T12:49:18.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.045+0000 7faa877fe700 1 -- 172.21.15.36:0/1187459401 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa68003260 con 0x7faa880adf10 2022-06-17T12:49:18.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.045+0000 7faa8cbf1700 1 -- 172.21.15.36:0/1187459401 >> v2:172.21.15.36:3300/0 conn(0x7faa880b15b0 msgr2=0x7faa880af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:18.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.045+0000 7faa8cbf1700 1 --2- 172.21.15.36:0/1187459401 >> v2:172.21.15.36:3300/0 conn(0x7faa880b15b0 0x7faa880af630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.045+0000 7faa8cbf1700 1 -- 172.21.15.36:0/1187459401 >> v1:172.21.15.70:6789/0 conn(0x7faa880adf10 legacy=0x7faa88140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:18.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.045+0000 7faa8cbf1700 1 -- 172.21.15.36:0/1187459401 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa880b1be0 con 0x7faa880b65f0 2022-06-17T12:49:18.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.045+0000 7faa8d3f2700 1 --2- 172.21.15.36:0/1187459401 >> v2:172.21.15.36:3300/0 conn(0x7faa880b15b0 0x7faa880af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:49:18.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.045+0000 7faa8cbf1700 1 --2- 172.21.15.36:0/1187459401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa880b65f0 0x7faa880b69d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7faa78002f70 tx=0x7faa78006df0).ready entity=mon.2 client_cookie=2992bcd31b0b3bef server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:18.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.049+0000 7faa877fe700 1 -- 172.21.15.36:0/1187459401 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa7800d4a0 con 0x7faa880b65f0 2022-06-17T12:49:18.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.049+0000 7faa877fe700 1 -- 172.21.15.36:0/1187459401 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa78000e90 con 0x7faa880b65f0 2022-06-17T12:49:18.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.049+0000 7faa877fe700 1 -- 172.21.15.36:0/1187459401 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa7800d7f0 con 0x7faa880b65f0 2022-06-17T12:49:18.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.049+0000 7faa8dbf3700 1 -- 172.21.15.36:0/1187459401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa880b65f0 msgr2=0x7faa880b69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:18.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.049+0000 7faa8dbf3700 1 --2- 172.21.15.36:0/1187459401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa880b65f0 0x7faa880b69d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7faa78002f70 tx=0x7faa78006df0).stop 2022-06-17T12:49:18.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.049+0000 7faa8dbf3700 1 -- 172.21.15.36:0/1187459401 shutdown_connections 2022-06-17T12:49:18.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.049+0000 7faa8dbf3700 1 --2- 172.21.15.36:0/1187459401 >> v2:172.21.15.36:3300/0 conn(0x7faa880b15b0 0x7faa880af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.049+0000 7faa8dbf3700 1 --2- 172.21.15.36:0/1187459401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa880b65f0 0x7faa880b69d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.049+0000 7faa8dbf3700 1 -- 172.21.15.36:0/1187459401 >> 172.21.15.36:0/1187459401 conn(0x7faa880614b0 msgr2=0x7faa88061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:18.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.049+0000 7faa8dbf3700 1 -- 172.21.15.36:0/1187459401 shutdown_connections 2022-06-17T12:49:18.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.049+0000 7faa8dbf3700 1 -- 172.21.15.36:0/1187459401 wait complete. 2022-06-17T12:49:18.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.049+0000 7faa8dbf3700 1 Processor -- start 2022-06-17T12:49:18.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.049+0000 7faa8dbf3700 1 -- start start 2022-06-17T12:49:18.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.053+0000 7faa8dbf3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa880adf10 0x7faa8813a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:18.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.053+0000 7faa8dbf3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa880b15b0 0x7faa8813aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:18.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.053+0000 7faa8dbf3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa880b65f0 0x7faa8814ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:18.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.053+0000 7faa8dbf3700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa88152fc0 con 0x7faa880b15b0 2022-06-17T12:49:18.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.053+0000 7faa8dbf3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa88063d40 con 0x7faa880adf10 2022-06-17T12:49:18.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.053+0000 7faa8dbf3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa88063fe0 con 0x7faa880b65f0 2022-06-17T12:49:18.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.053+0000 7faa87fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa880b15b0 0x7faa8813aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:18.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.053+0000 7faa87fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa880b15b0 0x7faa8813aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33492/0 (socket says 172.21.15.36:33492) 2022-06-17T12:49:18.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.053+0000 7faa87fff700 1 -- 172.21.15.36:0/1615969052 learned_addr learned my addr 172.21.15.36:0/1615969052 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:18.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.053+0000 7faa8cbf1700 1 --2- 172.21.15.36:0/1615969052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa880adf10 0x7faa8813a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:18.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.053+0000 7faa8d3f2700 1 --2- 172.21.15.36:0/1615969052 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa880b65f0 0x7faa8814ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:18.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.053+0000 7faa87fff700 1 -- 172.21.15.36:0/1615969052 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa880b65f0 msgr2=0x7faa8814ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:18.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.053+0000 7faa87fff700 1 --2- 172.21.15.36:0/1615969052 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa880b65f0 0x7faa8814ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.053+0000 7faa87fff700 1 -- 172.21.15.36:0/1615969052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa880adf10 msgr2=0x7faa8813a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:18.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.053+0000 7faa87fff700 1 --2- 172.21.15.36:0/1615969052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa880adf10 0x7faa8813a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.053+0000 7faa87fff700 1 -- 172.21.15.36:0/1615969052 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa80009530 con 0x7faa880b15b0 2022-06-17T12:49:18.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.053+0000 7faa87fff700 1 --2- 172.21.15.36:0/1615969052 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa880b15b0 0x7faa8813aa20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7faa70001160 tx=0x7faa70007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:18.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.053+0000 7faa85ffb700 1 -- 172.21.15.36:0/1615969052 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa7000f470 con 0x7faa880b15b0 2022-06-17T12:49:18.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.053+0000 7faa85ffb700 1 -- 172.21.15.36:0/1615969052 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7000f5d0 con 0x7faa880b15b0 2022-06-17T12:49:18.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.053+0000 7faa8dbf3700 1 -- 172.21.15.36:0/1615969052 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa7800b180 con 0x7faa880b15b0 2022-06-17T12:49:18.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.053+0000 7faa85ffb700 1 -- 172.21.15.36:0/1615969052 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa7000f930 con 0x7faa880b15b0 2022-06-17T12:49:18.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.053+0000 7faa8dbf3700 1 -- 172.21.15.36:0/1615969052 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa88152b40 con 0x7faa880b15b0 2022-06-17T12:49:18.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.053+0000 7faa8dbf3700 1 -- 172.21.15.36:0/1615969052 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa880641a0 con 0x7faa880b15b0 2022-06-17T12:49:18.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.053+0000 7faa85ffb700 1 -- 172.21.15.36:0/1615969052 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7faa70019030 con 0x7faa880b15b0 2022-06-17T12:49:18.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.057+0000 7faa85ffb700 1 --2- 172.21.15.36:0/1615969052 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa6c053ad0 0x7faa6c055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:18.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.057+0000 7faa85ffb700 1 -- 172.21.15.36:0/1615969052 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7faa7006b4e0 con 0x7faa880b15b0 2022-06-17T12:49:18.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.057+0000 7faa8cbf1700 1 --2- 172.21.15.36:0/1615969052 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa6c053ad0 0x7faa6c055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:18.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.057+0000 7faa8cbf1700 1 --2- 172.21.15.36:0/1615969052 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa6c053ad0 0x7faa6c055f90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7faa78018000 tx=0x7faa78014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:18.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.061+0000 7faa85ffb700 1 -- 172.21.15.36:0/1615969052 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faa700422e0 con 0x7faa880b15b0 2022-06-17T12:49:18.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.189+0000 7faa8dbf3700 1 -- 172.21.15.36:0/1615969052 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7faa88152df0 con 0x7faa880b15b0 2022-06-17T12:49:18.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.193+0000 7faa85ffb700 1 -- 172.21.15.36:0/1615969052 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v45) v1 ==== 110+0+10 (secure 0 0 0) 0x7faa70045900 con 0x7faa880b15b0 2022-06-17T12:49:18.180 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:49:18.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.193+0000 7faa8dbf3700 1 -- 172.21.15.36:0/1615969052 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa6c053ad0 msgr2=0x7faa6c055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:18.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.193+0000 7faa8dbf3700 1 --2- 172.21.15.36:0/1615969052 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa6c053ad0 0x7faa6c055f90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7faa78018000 tx=0x7faa78014000).stop 2022-06-17T12:49:18.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.193+0000 7faa8dbf3700 1 -- 172.21.15.36:0/1615969052 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa880b15b0 msgr2=0x7faa8813aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:18.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.193+0000 7faa8dbf3700 1 --2- 172.21.15.36:0/1615969052 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa880b15b0 0x7faa8813aa20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7faa70001160 tx=0x7faa70007ad0).stop 2022-06-17T12:49:18.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.193+0000 7faa8dbf3700 1 -- 172.21.15.36:0/1615969052 shutdown_connections 2022-06-17T12:49:18.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.193+0000 7faa8dbf3700 1 --2- 172.21.15.36:0/1615969052 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa6c053ad0 0x7faa6c055f90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.193+0000 7faa8dbf3700 1 --2- 172.21.15.36:0/1615969052 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa880b65f0 0x7faa8814ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.193+0000 7faa8dbf3700 1 --2- 172.21.15.36:0/1615969052 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa880b15b0 0x7faa8813aa20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.193+0000 7faa8dbf3700 1 --2- 172.21.15.36:0/1615969052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa880adf10 0x7faa8813a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.193+0000 7faa8dbf3700 1 -- 172.21.15.36:0/1615969052 >> 172.21.15.36:0/1615969052 conn(0x7faa880614b0 msgr2=0x7faa880ad330 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:18.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.193+0000 7faa8dbf3700 1 -- 172.21.15.36:0/1615969052 shutdown_connections 2022-06-17T12:49:18.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.193+0000 7faa8dbf3700 1 -- 172.21.15.36:0/1615969052 wait complete. 2022-06-17T12:49:18.206 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:49:18.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.337+0000 7fe0c3fff700 1 Processor -- start 2022-06-17T12:49:18.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.337+0000 7fe0c3fff700 1 -- start start 2022-06-17T12:49:18.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.337+0000 7fe0c3fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0c414b2b0 0x7fe0c4060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:18.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.337+0000 7fe0c3fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe0c4147bd0 0x7fe0c41442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:18.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.337+0000 7fe0c3fff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c414b870 con 0x7fe0c40b27b0 2022-06-17T12:49:18.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.337+0000 7fe0c3fff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c40ac500 con 0x7fe0c4147bd0 2022-06-17T12:49:18.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.337+0000 7fe0c3fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40ace90 con 0x7fe0c414b2b0 2022-06-17T12:49:18.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.337+0000 7fe0c27fc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0c414b2b0 0x7fe0c4060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:18.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.337+0000 7fe0c27fc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0c414b2b0 0x7fe0c4060ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:48290/0 (socket says 172.21.15.36:48290) 2022-06-17T12:49:18.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.337+0000 7fe0c27fc700 1 -- 172.21.15.36:0/1240662687 learned_addr learned my addr 172.21.15.36:0/1240662687 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:18.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.337+0000 7fe0c37fe700 1 --2- 172.21.15.36:0/1240662687 >> v2:172.21.15.70:3300/0 conn(0x7fe0c4147bd0 0x7fe0c41442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:18.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.341+0000 7fe0c27fc700 1 -- 172.21.15.36:0/1240662687 >> v2:172.21.15.70:3300/0 conn(0x7fe0c4147bd0 msgr2=0x7fe0c41442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:18.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.341+0000 7fe0c27fc700 1 --2- 172.21.15.36:0/1240662687 >> v2:172.21.15.70:3300/0 conn(0x7fe0c4147bd0 0x7fe0c41442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.341+0000 7fe0c27fc700 1 -- 172.21.15.36:0/1240662687 >> v1:172.21.15.70:6789/0 conn(0x7fe0c40b27b0 legacy=0x7fe0c40ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:18.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.341+0000 7fe0c27fc700 1 -- 172.21.15.36:0/1240662687 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c40b2b90 con 0x7fe0c414b2b0 2022-06-17T12:49:18.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.341+0000 7fe0c27fc700 1 --2- 172.21.15.36:0/1240662687 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0c414b2b0 0x7fe0c4060ca0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fe0b4002f70 tx=0x7fe0b4006eb0).ready entity=mon.2 client_cookie=b0e5f769501743a4 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:18.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.341+0000 7fe0c1ffb700 1 -- 172.21.15.36:0/1240662687 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0b400d570 con 0x7fe0c414b2b0 2022-06-17T12:49:18.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.341+0000 7fe0c1ffb700 1 -- 172.21.15.36:0/1240662687 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b4000e90 con 0x7fe0c414b2b0 2022-06-17T12:49:18.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.341+0000 7fe0c1ffb700 1 -- 172.21.15.36:0/1240662687 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0b400d900 con 0x7fe0c414b2b0 2022-06-17T12:49:18.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.341+0000 7fe0c3fff700 1 -- 172.21.15.36:0/1240662687 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0c414b2b0 msgr2=0x7fe0c4060ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:18.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.341+0000 7fe0c3fff700 1 --2- 172.21.15.36:0/1240662687 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0c414b2b0 0x7fe0c4060ca0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fe0b4002f70 tx=0x7fe0b4006eb0).stop 2022-06-17T12:49:18.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.341+0000 7fe0c3fff700 1 -- 172.21.15.36:0/1240662687 shutdown_connections 2022-06-17T12:49:18.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.341+0000 7fe0c3fff700 1 --2- 172.21.15.36:0/1240662687 >> v2:172.21.15.70:3300/0 conn(0x7fe0c4147bd0 0x7fe0c41442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.341+0000 7fe0c3fff700 1 --2- 172.21.15.36:0/1240662687 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0c414b2b0 0x7fe0c4060ca0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.341+0000 7fe0c3fff700 1 -- 172.21.15.36:0/1240662687 >> 172.21.15.36:0/1240662687 conn(0x7fe0c40b5d10 msgr2=0x7fe0c40b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:18.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.341+0000 7fe0c3fff700 1 -- 172.21.15.36:0/1240662687 shutdown_connections 2022-06-17T12:49:18.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.341+0000 7fe0c3fff700 1 -- 172.21.15.36:0/1240662687 wait complete. 2022-06-17T12:49:18.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.341+0000 7fe0c3fff700 1 Processor -- start 2022-06-17T12:49:18.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.341+0000 7fe0c3fff700 1 -- start start 2022-06-17T12:49:18.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.345+0000 7fe0c3fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0c40b27b0 0x7fe0c414a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:18.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.345+0000 7fe0c3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0c4147bd0 0x7fe0c4156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:18.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.345+0000 7fe0c3fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0c414b2b0 0x7fe0c40a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:18.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.345+0000 7fe0c3fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40645f0 con 0x7fe0c414b2b0 2022-06-17T12:49:18.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.345+0000 7fe0c3fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c4064890 con 0x7fe0c4147bd0 2022-06-17T12:49:18.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.345+0000 7fe0c3fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40a75e0 con 0x7fe0c40b27b0 2022-06-17T12:49:18.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.345+0000 7fe0c37fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0c414b2b0 0x7fe0c40a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:18.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.345+0000 7fe0c37fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0c414b2b0 0x7fe0c40a7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33508/0 (socket says 172.21.15.36:33508) 2022-06-17T12:49:18.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.345+0000 7fe0c37fe700 1 -- 172.21.15.36:0/2221211323 learned_addr learned my addr 172.21.15.36:0/2221211323 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:18.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.345+0000 7fe0c27fc700 1 --2- 172.21.15.36:0/2221211323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0c4147bd0 0x7fe0c4156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:18.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.345+0000 7fe0c2ffd700 1 --2- 172.21.15.36:0/2221211323 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0c40b27b0 0x7fe0c414a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:18.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.345+0000 7fe0c37fe700 1 -- 172.21.15.36:0/2221211323 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0c40b27b0 msgr2=0x7fe0c414a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:18.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.345+0000 7fe0c37fe700 1 --2- 172.21.15.36:0/2221211323 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0c40b27b0 0x7fe0c414a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.345+0000 7fe0c37fe700 1 -- 172.21.15.36:0/2221211323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0c4147bd0 msgr2=0x7fe0c4156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:18.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.345+0000 7fe0c37fe700 1 --2- 172.21.15.36:0/2221211323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0c4147bd0 0x7fe0c4156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.345+0000 7fe0c37fe700 1 -- 172.21.15.36:0/2221211323 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b400b1e0 con 0x7fe0c414b2b0 2022-06-17T12:49:18.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.345+0000 7fe0c37fe700 1 --2- 172.21.15.36:0/2221211323 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0c414b2b0 0x7fe0c40a7a80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7fe0bc006e00 tx=0x7fe0bc0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:18.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.345+0000 7fe0b3fff700 1 -- 172.21.15.36:0/2221211323 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0bc00c640 con 0x7fe0c414b2b0 2022-06-17T12:49:18.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.345+0000 7fe0b3fff700 1 -- 172.21.15.36:0/2221211323 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0bc00c7a0 con 0x7fe0c414b2b0 2022-06-17T12:49:18.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.345+0000 7fe0c3fff700 1 -- 172.21.15.36:0/2221211323 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c41568f0 con 0x7fe0c414b2b0 2022-06-17T12:49:18.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.345+0000 7fe0b3fff700 1 -- 172.21.15.36:0/2221211323 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0bc00cb00 con 0x7fe0c414b2b0 2022-06-17T12:49:18.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.345+0000 7fe0c3fff700 1 -- 172.21.15.36:0/2221211323 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c4066320 con 0x7fe0c414b2b0 2022-06-17T12:49:18.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.345+0000 7fe0c3fff700 1 -- 172.21.15.36:0/2221211323 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c40a77a0 con 0x7fe0c414b2b0 2022-06-17T12:49:18.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.349+0000 7fe0b3fff700 1 -- 172.21.15.36:0/2221211323 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe0bc00cc60 con 0x7fe0c414b2b0 2022-06-17T12:49:18.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.349+0000 7fe0b3fff700 1 --2- 172.21.15.36:0/2221211323 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe0a0053a40 0x7fe0a0055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:18.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.349+0000 7fe0b3fff700 1 -- 172.21.15.36:0/2221211323 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe0bc06c310 con 0x7fe0c414b2b0 2022-06-17T12:49:18.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.349+0000 7fe0c2ffd700 1 --2- 172.21.15.36:0/2221211323 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe0a0053a40 0x7fe0a0055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:18.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.349+0000 7fe0c2ffd700 1 --2- 172.21.15.36:0/2221211323 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe0a0053a40 0x7fe0a0055f00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fe0b800d000 tx=0x7fe0b8007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:18.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.353+0000 7fe0b3fff700 1 -- 172.21.15.36:0/2221211323 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe0bc043000 con 0x7fe0c414b2b0 2022-06-17T12:49:18.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.477+0000 7fe0c3fff700 1 -- 172.21.15.36:0/2221211323 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe0c4057410 con 0x7fe0c414b2b0 2022-06-17T12:49:18.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.477+0000 7fe0b3fff700 1 -- 172.21.15.36:0/2221211323 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe0bc046620 con 0x7fe0c414b2b0 2022-06-17T12:49:18.467 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:49:18.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.481+0000 7fe0c3fff700 1 -- 172.21.15.36:0/2221211323 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe0a0053a40 msgr2=0x7fe0a0055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:18.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.481+0000 7fe0c3fff700 1 --2- 172.21.15.36:0/2221211323 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe0a0053a40 0x7fe0a0055f00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fe0b800d000 tx=0x7fe0b8007b90).stop 2022-06-17T12:49:18.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.481+0000 7fe0c3fff700 1 -- 172.21.15.36:0/2221211323 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0c414b2b0 msgr2=0x7fe0c40a7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:18.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.481+0000 7fe0c3fff700 1 --2- 172.21.15.36:0/2221211323 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0c414b2b0 0x7fe0c40a7a80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7fe0bc006e00 tx=0x7fe0bc0091d0).stop 2022-06-17T12:49:18.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.481+0000 7fe0c3fff700 1 -- 172.21.15.36:0/2221211323 shutdown_connections 2022-06-17T12:49:18.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.481+0000 7fe0c3fff700 1 --2- 172.21.15.36:0/2221211323 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe0a0053a40 0x7fe0a0055f00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.481+0000 7fe0c3fff700 1 --2- 172.21.15.36:0/2221211323 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0c414b2b0 0x7fe0c40a7a80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.481+0000 7fe0c3fff700 1 --2- 172.21.15.36:0/2221211323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0c4147bd0 0x7fe0c4156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.481+0000 7fe0c3fff700 1 --2- 172.21.15.36:0/2221211323 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0c40b27b0 0x7fe0c414a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.481+0000 7fe0c3fff700 1 -- 172.21.15.36:0/2221211323 >> 172.21.15.36:0/2221211323 conn(0x7fe0c40b5d10 msgr2=0x7fe0c40b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:18.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.481+0000 7fe0c3fff700 1 -- 172.21.15.36:0/2221211323 shutdown_connections 2022-06-17T12:49:18.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.481+0000 7fe0c3fff700 1 -- 172.21.15.36:0/2221211323 wait complete. 2022-06-17T12:49:18.493 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:49:18.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.633+0000 7f7ef0d09700 1 Processor -- start 2022-06-17T12:49:18.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.633+0000 7f7ef0d09700 1 -- start start 2022-06-17T12:49:18.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7ef0d09700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7eec13f220 0x7f7eec143420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:18.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7ef0d09700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eec0ad4c0 con 0x7f7eec0b2ee0 2022-06-17T12:49:18.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7ef0d09700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eec0b32c0 con 0x7f7eec0b1120 2022-06-17T12:49:18.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7ef0d09700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eec0ad720 con 0x7f7eec13f220 2022-06-17T12:49:18.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7eea59c700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f7eec0b2ee0 0x7f7eec13eb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51042/0 (socket says 172.21.15.36:51042) 2022-06-17T12:49:18.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7eea59c700 1 -- 172.21.15.36:0/3742915912 learned_addr learned my addr 172.21.15.36:0/3742915912 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:18.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7eead9d700 1 --2- 172.21.15.36:0/3742915912 >> v2:172.21.15.70:3300/0 conn(0x7f7eec13f220 0x7f7eec143420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:18.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7ee959a700 1 -- 172.21.15.36:0/3742915912 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3842555992 0 0) 0x7f7eec0ad4c0 con 0x7f7eec0b2ee0 2022-06-17T12:49:18.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7ee959a700 1 -- 172.21.15.36:0/3742915912 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ecc004370 con 0x7f7eec0b2ee0 2022-06-17T12:49:18.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7ee959a700 1 -- 172.21.15.36:0/3742915912 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4145199973 0 0) 0x7f7ecc004370 con 0x7f7eec0b2ee0 2022-06-17T12:49:18.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7ee959a700 1 -- 172.21.15.36:0/3742915912 >> v2:172.21.15.70:3300/0 conn(0x7f7eec13f220 msgr2=0x7f7eec143420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:18.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7ee959a700 1 --2- 172.21.15.36:0/3742915912 >> v2:172.21.15.70:3300/0 conn(0x7f7eec13f220 0x7f7eec143420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7ee959a700 1 -- 172.21.15.36:0/3742915912 >> v1:172.21.15.70:6789/0 conn(0x7f7eec0b1120 legacy=0x7f7eec142d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:18.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7ee959a700 1 -- 172.21.15.36:0/3742915912 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eec0ad9c0 con 0x7f7eec0b2ee0 2022-06-17T12:49:18.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7ee959a700 1 -- 172.21.15.36:0/3742915912 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f7edc001a00 con 0x7f7eec0b2ee0 2022-06-17T12:49:18.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7ee959a700 1 -- 172.21.15.36:0/3742915912 >> v1:172.21.15.36:6789/0 conn(0x7f7eec0b2ee0 legacy=0x7f7eec13eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:18.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7ee959a700 1 --2- 172.21.15.36:0/3742915912 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ecc004860 0x7f7ecc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:18.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7ee959a700 1 --2- 172.21.15.36:0/3742915912 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ecc005610 0x7f7ecc005a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:18.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7ee959a700 1 --2- 172.21.15.36:0/3742915912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ecc005f80 0x7f7ecc00c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:18.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7ee959a700 1 -- 172.21.15.36:0/3742915912 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec0ad720 con 0x7f7ecc004860 2022-06-17T12:49:18.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7ee959a700 1 -- 172.21.15.36:0/3742915912 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc00cd80 con 0x7f7ecc005f80 2022-06-17T12:49:18.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7ee959a700 1 -- 172.21.15.36:0/3742915912 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc00cf40 con 0x7f7ecc005610 2022-06-17T12:49:18.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7eea59c700 1 --2- 172.21.15.36:0/3742915912 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ecc004860 0x7f7ecc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:18.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7ee9d9b700 1 --2- 172.21.15.36:0/3742915912 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ecc005610 0x7f7ecc005a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:18.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7eead9d700 1 --2- 172.21.15.36:0/3742915912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ecc005f80 0x7f7ecc00c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:18.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7eea59c700 1 -- 172.21.15.36:0/3742915912 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ecc005610 msgr2=0x7f7ecc005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:18.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7eea59c700 1 --2- 172.21.15.36:0/3742915912 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ecc005610 0x7f7ecc005a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7eea59c700 1 -- 172.21.15.36:0/3742915912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ecc005f80 msgr2=0x7f7ecc00c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:18.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7eea59c700 1 --2- 172.21.15.36:0/3742915912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ecc005f80 0x7f7ecc00c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7eea59c700 1 -- 172.21.15.36:0/3742915912 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ecc004370 con 0x7f7ecc004860 2022-06-17T12:49:18.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7eead9d700 1 -- 172.21.15.36:0/3742915912 reap_dead start 2022-06-17T12:49:18.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7eea59c700 1 --2- 172.21.15.36:0/3742915912 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ecc004860 0x7f7ecc004c40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f7edc001710 tx=0x7f7eec0adb50).ready entity=mon.0 client_cookie=346f0dad7c1a75bc server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:18.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7ee959a700 1 -- 172.21.15.36:0/3742915912 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7edc00cbe0 con 0x7f7ecc004860 2022-06-17T12:49:18.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7ee959a700 1 -- 172.21.15.36:0/3742915912 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7edc00cd40 con 0x7f7ecc004860 2022-06-17T12:49:18.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.637+0000 7f7ee959a700 1 -- 172.21.15.36:0/3742915912 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7edc004b50 con 0x7f7ecc004860 2022-06-17T12:49:18.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7ef0d09700 1 -- 172.21.15.36:0/3742915912 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ecc004860 msgr2=0x7f7ecc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:18.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7ef0d09700 1 --2- 172.21.15.36:0/3742915912 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ecc004860 0x7f7ecc004c40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f7edc001710 tx=0x7f7eec0adb50).stop 2022-06-17T12:49:18.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7ef0d09700 1 -- 172.21.15.36:0/3742915912 shutdown_connections 2022-06-17T12:49:18.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7ef0d09700 1 --2- 172.21.15.36:0/3742915912 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ecc004860 0x7f7ecc004c40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7ef0d09700 1 -- 172.21.15.36:0/3742915912 >> 172.21.15.36:0/3742915912 conn(0x7f7eec063020 msgr2=0x7f7eec063430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:18.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7ef0d09700 1 -- 172.21.15.36:0/3742915912 shutdown_connections 2022-06-17T12:49:18.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7ef0d09700 1 -- 172.21.15.36:0/3742915912 wait complete. 2022-06-17T12:49:18.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7ef0d09700 1 Processor -- start 2022-06-17T12:49:18.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7ef0d09700 1 -- start start 2022-06-17T12:49:18.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7ef0d09700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ecc004860 0x7f7eec0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:18.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7ef0d09700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7eec0abdb0 0x7f7eec13b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:18.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7ef0d09700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7eec1559a0 0x7f7eec155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:18.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7ef0d09700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec13daa0 con 0x7f7eec1559a0 2022-06-17T12:49:18.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7ef0d09700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec13dd40 con 0x7f7eec0abdb0 2022-06-17T12:49:18.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7ef0d09700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec156320 con 0x7f7ecc004860 2022-06-17T12:49:18.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7eead9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7eec1559a0 0x7f7eec155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:18.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7eea59c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ecc004860 0x7f7eec0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:18.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7ee9d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7eec0abdb0 0x7f7eec13b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:18.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7eead9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7eec1559a0 0x7f7eec155e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33528/0 (socket says 172.21.15.36:33528) 2022-06-17T12:49:18.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7eead9d700 1 -- 172.21.15.36:0/3416290136 learned_addr learned my addr 172.21.15.36:0/3416290136 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:18.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7eead9d700 1 -- 172.21.15.36:0/3416290136 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ecc004860 msgr2=0x7f7eec0b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:18.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7eead9d700 1 --2- 172.21.15.36:0/3416290136 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ecc004860 0x7f7eec0b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7eead9d700 1 -- 172.21.15.36:0/3416290136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7eec0abdb0 msgr2=0x7f7eec13b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:18.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7eead9d700 1 --2- 172.21.15.36:0/3416290136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7eec0abdb0 0x7f7eec13b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7eead9d700 1 -- 172.21.15.36:0/3416290136 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eec13dfe0 con 0x7f7eec1559a0 2022-06-17T12:49:18.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7eead9d700 1 --2- 172.21.15.36:0/3416290136 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7eec1559a0 0x7f7eec155e30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f7ee400b000 tx=0x7f7ee4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:18.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7ee37fe700 1 -- 172.21.15.36:0/3416290136 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ee4018070 con 0x7f7eec1559a0 2022-06-17T12:49:18.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7ee37fe700 1 -- 172.21.15.36:0/3416290136 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee4010ec0 con 0x7f7eec1559a0 2022-06-17T12:49:18.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7ef0d09700 1 -- 172.21.15.36:0/3416290136 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eec0aaed0 con 0x7f7eec1559a0 2022-06-17T12:49:18.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7ee37fe700 1 -- 172.21.15.36:0/3416290136 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ee40137d0 con 0x7f7eec1559a0 2022-06-17T12:49:18.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.641+0000 7f7ef0d09700 1 -- 172.21.15.36:0/3416290136 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eec0acad0 con 0x7f7eec1559a0 2022-06-17T12:49:18.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.645+0000 7f7ef0d09700 1 -- 172.21.15.36:0/3416290136 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eec0b2120 con 0x7f7eec1559a0 2022-06-17T12:49:18.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.645+0000 7f7ee37fe700 1 -- 172.21.15.36:0/3416290136 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f7ee4013930 con 0x7f7eec1559a0 2022-06-17T12:49:18.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.645+0000 7f7ee37fe700 1 --2- 172.21.15.36:0/3416290136 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7ed0053a40 0x7f7ed0055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:18.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.645+0000 7f7ee37fe700 1 -- 172.21.15.36:0/3416290136 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7ee406bfc0 con 0x7f7eec1559a0 2022-06-17T12:49:18.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.645+0000 7f7eea59c700 1 --2- 172.21.15.36:0/3416290136 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7ed0053a40 0x7f7ed0055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:18.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.649+0000 7f7eea59c700 1 --2- 172.21.15.36:0/3416290136 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7ed0053a40 0x7f7ed0055f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f7edc01b000 tx=0x7f7edc015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:18.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.649+0000 7f7ee37fe700 1 -- 172.21.15.36:0/3416290136 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7ee4042dc0 con 0x7f7eec1559a0 2022-06-17T12:49:18.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.781+0000 7f7ef0d09700 1 -- 172.21.15.36:0/3416290136 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7eec057410 con 0x7f7eec1559a0 2022-06-17T12:49:18.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.781+0000 7f7ee37fe700 1 -- 172.21.15.36:0/3416290136 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+10 (secure 0 0 0) 0x7f7ee4006e60 con 0x7f7eec1559a0 2022-06-17T12:49:18.769 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:49:18.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.785+0000 7f7ef0d09700 1 -- 172.21.15.36:0/3416290136 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7ed0053a40 msgr2=0x7f7ed0055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:18.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.785+0000 7f7ef0d09700 1 --2- 172.21.15.36:0/3416290136 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7ed0053a40 0x7f7ed0055f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f7edc01b000 tx=0x7f7edc015000).stop 2022-06-17T12:49:18.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.785+0000 7f7ef0d09700 1 -- 172.21.15.36:0/3416290136 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7eec1559a0 msgr2=0x7f7eec155e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:18.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.785+0000 7f7ef0d09700 1 --2- 172.21.15.36:0/3416290136 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7eec1559a0 0x7f7eec155e30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f7ee400b000 tx=0x7f7ee4008040).stop 2022-06-17T12:49:18.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.785+0000 7f7ef0d09700 1 -- 172.21.15.36:0/3416290136 shutdown_connections 2022-06-17T12:49:18.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.785+0000 7f7ef0d09700 1 --2- 172.21.15.36:0/3416290136 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7ed0053a40 0x7f7ed0055f00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.785+0000 7f7ef0d09700 1 --2- 172.21.15.36:0/3416290136 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7eec1559a0 0x7f7eec155e30 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.785+0000 7f7ef0d09700 1 --2- 172.21.15.36:0/3416290136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7eec0abdb0 0x7f7eec13b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.785+0000 7f7ef0d09700 1 --2- 172.21.15.36:0/3416290136 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ecc004860 0x7f7eec0b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:18.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.785+0000 7f7ef0d09700 1 -- 172.21.15.36:0/3416290136 >> 172.21.15.36:0/3416290136 conn(0x7f7eec063020 msgr2=0x7f7eec0aec20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:18.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.785+0000 7f7ef0d09700 1 -- 172.21.15.36:0/3416290136 shutdown_connections 2022-06-17T12:49:18.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.785+0000 7f7ef0d09700 1 -- 172.21.15.36:0/3416290136 wait complete. 2022-06-17T12:49:18.800 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:49:19.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.925+0000 7f7407fff700 1 Processor -- start 2022-06-17T12:49:19.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.925+0000 7f7407fff700 1 -- start start 2022-06-17T12:49:19.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f7407fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f74080b27b0 0x7f74080b1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:19.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f7407fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7408140940 0x7f7408141180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:19.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f7407fff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74080ac0b0 con 0x7f7408144d20 2022-06-17T12:49:19.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f7407fff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7408140d90 con 0x7f74080b27b0 2022-06-17T12:49:19.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f7407fff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74080b2b90 con 0x7f7408140940 2022-06-17T12:49:19.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f7406ffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f74080b27b0 0x7f74080b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:19.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f7406ffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f74080b27b0 0x7f74080b1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33532/0 (socket says 172.21.15.36:33532) 2022-06-17T12:49:19.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f7406ffd700 1 -- 172.21.15.36:0/1629665240 learned_addr learned my addr 172.21.15.36:0/1629665240 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:19.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f74077fe700 1 --2- 172.21.15.36:0/1629665240 >> v2:172.21.15.70:3300/0 conn(0x7f7408140940 0x7f7408141180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:19.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f7406ffd700 1 -- 172.21.15.36:0/1629665240 >> v2:172.21.15.70:3300/0 conn(0x7f7408140940 msgr2=0x7f7408141180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:19.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f7406ffd700 1 --2- 172.21.15.36:0/1629665240 >> v2:172.21.15.70:3300/0 conn(0x7f7408140940 0x7f7408141180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:19.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f7406ffd700 1 -- 172.21.15.36:0/1629665240 >> v1:172.21.15.70:6789/0 conn(0x7f7408144d20 legacy=0x7f7408140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:19.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f7406ffd700 1 -- 172.21.15.36:0/1629665240 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74080afc60 con 0x7f74080b27b0 2022-06-17T12:49:19.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f7406ffd700 1 --2- 172.21.15.36:0/1629665240 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74080b27b0 0x7f74080b1830 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f74080b1d20 tx=0x7f73fc006eb0).ready entity=mon.0 client_cookie=d48b3a632f2335c5 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:19.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f7405ffb700 1 -- 172.21.15.36:0/1629665240 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73fc00d610 con 0x7f74080b27b0 2022-06-17T12:49:19.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f7405ffb700 1 -- 172.21.15.36:0/1629665240 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73fc000e90 con 0x7f74080b27b0 2022-06-17T12:49:19.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f7405ffb700 1 -- 172.21.15.36:0/1629665240 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73fc00d9a0 con 0x7f74080b27b0 2022-06-17T12:49:19.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f7407fff700 1 -- 172.21.15.36:0/1629665240 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74080b27b0 msgr2=0x7f74080b1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:19.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f7407fff700 1 --2- 172.21.15.36:0/1629665240 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74080b27b0 0x7f74080b1830 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f74080b1d20 tx=0x7f73fc006eb0).stop 2022-06-17T12:49:19.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f7407fff700 1 -- 172.21.15.36:0/1629665240 shutdown_connections 2022-06-17T12:49:19.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f7407fff700 1 --2- 172.21.15.36:0/1629665240 >> v2:172.21.15.70:3300/0 conn(0x7f7408140940 0x7f7408141180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:19.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f7407fff700 1 --2- 172.21.15.36:0/1629665240 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74080b27b0 0x7f74080b1830 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:19.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f7407fff700 1 -- 172.21.15.36:0/1629665240 >> 172.21.15.36:0/1629665240 conn(0x7f74080b5d10 msgr2=0x7f74080b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:19.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f7407fff700 1 -- 172.21.15.36:0/1629665240 shutdown_connections 2022-06-17T12:49:19.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.929+0000 7f7407fff700 1 -- 172.21.15.36:0/1629665240 wait complete. 2022-06-17T12:49:19.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.933+0000 7f7407fff700 1 Processor -- start 2022-06-17T12:49:19.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.933+0000 7f7407fff700 1 -- start start 2022-06-17T12:49:19.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.933+0000 7f7407fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74080b27b0 0x7f74080b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:19.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.933+0000 7f7407fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7408140940 0x7f740813ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:19.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.933+0000 7f7407fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7408144d20 0x7f740813f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:19.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.933+0000 7f7407fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74080add30 con 0x7f74080b27b0 2022-06-17T12:49:19.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.933+0000 7f7407fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74080ae250 con 0x7f7408144d20 2022-06-17T12:49:19.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.933+0000 7f7407fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740813cc50 con 0x7f7408140940 2022-06-17T12:49:19.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.933+0000 7f7406ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74080b27b0 0x7f74080b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:19.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.933+0000 7f7406ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74080b27b0 0x7f74080b0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33538/0 (socket says 172.21.15.36:33538) 2022-06-17T12:49:19.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.933+0000 7f7406ffd700 1 -- 172.21.15.36:0/443700663 learned_addr learned my addr 172.21.15.36:0/443700663 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:19.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.933+0000 7f74077fe700 1 --2- 172.21.15.36:0/443700663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7408144d20 0x7f740813f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:19.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.933+0000 7f74067fc700 1 --2- 172.21.15.36:0/443700663 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7408140940 0x7f740813ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:19.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.933+0000 7f7406ffd700 1 -- 172.21.15.36:0/443700663 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7408140940 msgr2=0x7f740813ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:19.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.933+0000 7f7406ffd700 1 --2- 172.21.15.36:0/443700663 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7408140940 0x7f740813ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:19.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.933+0000 7f7406ffd700 1 -- 172.21.15.36:0/443700663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7408144d20 msgr2=0x7f740813f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:19.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.933+0000 7f7406ffd700 1 --2- 172.21.15.36:0/443700663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7408144d20 0x7f740813f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:19.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.933+0000 7f7406ffd700 1 -- 172.21.15.36:0/443700663 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc00b1e0 con 0x7f74080b27b0 2022-06-17T12:49:19.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.933+0000 7f7406ffd700 1 --2- 172.21.15.36:0/443700663 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74080b27b0 0x7f74080b0a60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f74080b1d20 tx=0x7f73fc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:19.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.933+0000 7f73f7fff700 1 -- 172.21.15.36:0/443700663 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73fc0117e0 con 0x7f74080b27b0 2022-06-17T12:49:19.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.937+0000 7f73f7fff700 1 -- 172.21.15.36:0/443700663 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73fc011940 con 0x7f74080b27b0 2022-06-17T12:49:19.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.937+0000 7f7407fff700 1 -- 172.21.15.36:0/443700663 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74080aca20 con 0x7f74080b27b0 2022-06-17T12:49:19.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.937+0000 7f73f7fff700 1 -- 172.21.15.36:0/443700663 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73fc011ca0 con 0x7f74080b27b0 2022-06-17T12:49:19.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.937+0000 7f7407fff700 1 -- 172.21.15.36:0/443700663 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74080ad710 con 0x7f74080b27b0 2022-06-17T12:49:19.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.937+0000 7f7407fff700 1 -- 172.21.15.36:0/443700663 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74080ae090 con 0x7f74080b27b0 2022-06-17T12:49:19.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.937+0000 7f73f7fff700 1 -- 172.21.15.36:0/443700663 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f73fc025030 con 0x7f74080b27b0 2022-06-17T12:49:19.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.937+0000 7f73f7fff700 1 --2- 172.21.15.36:0/443700663 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f73e4053ad0 0x7f73e4055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:19.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.937+0000 7f73f7fff700 1 -- 172.21.15.36:0/443700663 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f73fc06d0f0 con 0x7f74080b27b0 2022-06-17T12:49:19.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.937+0000 7f74067fc700 1 --2- 172.21.15.36:0/443700663 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f73e4053ad0 0x7f73e4055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:19.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.941+0000 7f74067fc700 1 --2- 172.21.15.36:0/443700663 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f73e4053ad0 0x7f73e4055f90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f73f800d000 tx=0x7f73f8007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:19.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:18.945+0000 7f73f7fff700 1 -- 172.21.15.36:0/443700663 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f73fc043ef0 con 0x7f74080b27b0 2022-06-17T12:49:19.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.073+0000 7f7407fff700 1 -- 172.21.15.36:0/443700663 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f74080ac310 con 0x7f74080b27b0 2022-06-17T12:49:19.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.073+0000 7f73f7fff700 1 -- 172.21.15.36:0/443700663 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+10 (secure 0 0 0) 0x7f73fc0230a0 con 0x7f74080b27b0 2022-06-17T12:49:19.094 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:49:19.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.073+0000 7f7407fff700 1 -- 172.21.15.36:0/443700663 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f73e4053ad0 msgr2=0x7f73e4055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:19.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.073+0000 7f7407fff700 1 --2- 172.21.15.36:0/443700663 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f73e4053ad0 0x7f73e4055f90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f73f800d000 tx=0x7f73f8007b90).stop 2022-06-17T12:49:19.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.073+0000 7f7407fff700 1 -- 172.21.15.36:0/443700663 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74080b27b0 msgr2=0x7f74080b0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:19.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.073+0000 7f7407fff700 1 --2- 172.21.15.36:0/443700663 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74080b27b0 0x7f74080b0a60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f74080b1d20 tx=0x7f73fc015000).stop 2022-06-17T12:49:19.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.073+0000 7f7407fff700 1 -- 172.21.15.36:0/443700663 shutdown_connections 2022-06-17T12:49:19.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.073+0000 7f7407fff700 1 --2- 172.21.15.36:0/443700663 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f73e4053ad0 0x7f73e4055f90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:19.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.073+0000 7f7407fff700 1 --2- 172.21.15.36:0/443700663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7408144d20 0x7f740813f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:19.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.073+0000 7f7407fff700 1 --2- 172.21.15.36:0/443700663 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7408140940 0x7f740813ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:19.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.073+0000 7f7407fff700 1 --2- 172.21.15.36:0/443700663 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74080b27b0 0x7f74080b0a60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:19.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.073+0000 7f7407fff700 1 -- 172.21.15.36:0/443700663 >> 172.21.15.36:0/443700663 conn(0x7f74080b5d10 msgr2=0x7f740805f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:19.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.073+0000 7f7407fff700 1 -- 172.21.15.36:0/443700663 shutdown_connections 2022-06-17T12:49:19.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.073+0000 7f7407fff700 1 -- 172.21.15.36:0/443700663 wait complete. 2022-06-17T12:49:19.104 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:49:19.107 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:49:19.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.265+0000 7f2bbb59e700 1 Processor -- start 2022-06-17T12:49:19.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.265+0000 7f2bbb59e700 1 -- start start 2022-06-17T12:49:19.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.265+0000 7f2bbb59e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2bbc146d00 0x7f2bbc147150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:19.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.265+0000 7f2bbb59e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bbc0b25e0 con 0x7f2bbc0b2cd0 2022-06-17T12:49:19.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.265+0000 7f2bbb59e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bbc0ad790 con 0x7f2bbc0b2200 2022-06-17T12:49:19.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.265+0000 7f2bbb59e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bbc0aef40 con 0x7f2bbc146d00 2022-06-17T12:49:19.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.265+0000 7f2bb9d9b700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f2bbc0b2cd0 0x7f2bbc1465f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51076/0 (socket says 172.21.15.36:51076) 2022-06-17T12:49:19.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.265+0000 7f2bb9d9b700 1 -- 172.21.15.36:0/3164811741 learned_addr learned my addr 172.21.15.36:0/3164811741 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:19.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.265+0000 7f2bbad9d700 1 --2- 172.21.15.36:0/3164811741 >> v2:172.21.15.70:3300/0 conn(0x7f2bbc146d00 0x7f2bbc147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:19.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.265+0000 7f2bb959a700 1 -- 172.21.15.36:0/3164811741 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2834645429 0 0) 0x7f2bbc0b25e0 con 0x7f2bbc0b2cd0 2022-06-17T12:49:19.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.265+0000 7f2bb959a700 1 -- 172.21.15.36:0/3164811741 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ba0003260 con 0x7f2bbc0b2cd0 2022-06-17T12:49:19.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.265+0000 7f2bb959a700 1 -- 172.21.15.36:0/3164811741 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2458837987 0 0) 0x7f2bbc0ad790 con 0x7f2bbc0b2200 2022-06-17T12:49:19.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.265+0000 7f2bb959a700 1 -- 172.21.15.36:0/3164811741 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bbc0b25e0 con 0x7f2bbc0b2200 2022-06-17T12:49:19.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.265+0000 7f2bbad9d700 1 -- 172.21.15.36:0/3164811741 >> v1:172.21.15.70:6789/0 conn(0x7f2bbc0b2200 legacy=0x7f2bbc0af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:19.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.265+0000 7f2bbad9d700 1 -- 172.21.15.36:0/3164811741 >> v1:172.21.15.36:6789/0 conn(0x7f2bbc0b2cd0 legacy=0x7f2bbc1465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:19.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.265+0000 7f2bbad9d700 1 -- 172.21.15.36:0/3164811741 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc0afaf0 con 0x7f2bbc146d00 2022-06-17T12:49:19.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.265+0000 7f2bbad9d700 1 --2- 172.21.15.36:0/3164811741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bbc146d00 0x7f2bbc147150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f2bb4002f70 tx=0x7f2bb4006df0).ready entity=mon.1 client_cookie=7e307ec5cae5c0eb server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:19.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.269+0000 7f2bb959a700 1 -- 172.21.15.36:0/3164811741 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bb400d540 con 0x7f2bbc146d00 2022-06-17T12:49:19.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.269+0000 7f2bb959a700 1 -- 172.21.15.36:0/3164811741 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb4000e90 con 0x7f2bbc146d00 2022-06-17T12:49:19.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.269+0000 7f2bb959a700 1 -- 172.21.15.36:0/3164811741 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bb400d890 con 0x7f2bbc146d00 2022-06-17T12:49:19.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.269+0000 7f2bbb59e700 1 -- 172.21.15.36:0/3164811741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bbc146d00 msgr2=0x7f2bbc147150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:19.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.269+0000 7f2bbb59e700 1 --2- 172.21.15.36:0/3164811741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bbc146d00 0x7f2bbc147150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f2bb4002f70 tx=0x7f2bb4006df0).stop 2022-06-17T12:49:19.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.269+0000 7f2bbb59e700 1 -- 172.21.15.36:0/3164811741 shutdown_connections 2022-06-17T12:49:19.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.269+0000 7f2bbb59e700 1 --2- 172.21.15.36:0/3164811741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bbc146d00 0x7f2bbc147150 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:19.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.269+0000 7f2bbb59e700 1 -- 172.21.15.36:0/3164811741 >> 172.21.15.36:0/3164811741 conn(0x7f2bbc05f770 msgr2=0x7f2bbc05fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:19.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.269+0000 7f2bbb59e700 1 -- 172.21.15.36:0/3164811741 shutdown_connections 2022-06-17T12:49:19.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.269+0000 7f2bbb59e700 1 -- 172.21.15.36:0/3164811741 wait complete. 2022-06-17T12:49:19.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.269+0000 7f2bbb59e700 1 Processor -- start 2022-06-17T12:49:19.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.269+0000 7f2bbb59e700 1 -- start start 2022-06-17T12:49:19.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.269+0000 7f2bbb59e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bbc0b2200 0x7f2bbc0a7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:19.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.269+0000 7f2bbb59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2bbc0b2cd0 0x7f2bbc145ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:19.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.269+0000 7f2bbb59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2bbc146d00 0x7f2bbc1446c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:19.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.269+0000 7f2bbb59e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc0a9a40 con 0x7f2bbc0b2cd0 2022-06-17T12:49:19.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.269+0000 7f2bbb59e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc145fd0 con 0x7f2bbc0b2200 2022-06-17T12:49:19.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.269+0000 7f2bbb59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc0ab260 con 0x7f2bbc146d00 2022-06-17T12:49:19.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.269+0000 7f2bb9d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2bbc0b2cd0 0x7f2bbc145ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:19.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.269+0000 7f2bb9d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2bbc0b2cd0 0x7f2bbc145ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33552/0 (socket says 172.21.15.36:33552) 2022-06-17T12:49:19.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.269+0000 7f2bb9d9b700 1 -- 172.21.15.36:0/3626833229 learned_addr learned my addr 172.21.15.36:0/3626833229 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:19.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.269+0000 7f2bbad9d700 1 --2- 172.21.15.36:0/3626833229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2bbc146d00 0x7f2bbc1446c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:19.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.269+0000 7f2bba59c700 1 --2- 172.21.15.36:0/3626833229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bbc0b2200 0x7f2bbc0a7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:19.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.273+0000 7f2bb9d9b700 1 -- 172.21.15.36:0/3626833229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2bbc146d00 msgr2=0x7f2bbc1446c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:19.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.273+0000 7f2bb9d9b700 1 --2- 172.21.15.36:0/3626833229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2bbc146d00 0x7f2bbc1446c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:19.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.273+0000 7f2bb9d9b700 1 -- 172.21.15.36:0/3626833229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bbc0b2200 msgr2=0x7f2bbc0a7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:19.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.273+0000 7f2bb9d9b700 1 --2- 172.21.15.36:0/3626833229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bbc0b2200 0x7f2bbc0a7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:19.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.273+0000 7f2bb9d9b700 1 -- 172.21.15.36:0/3626833229 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb400b180 con 0x7f2bbc0b2cd0 2022-06-17T12:49:19.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.273+0000 7f2bb9d9b700 1 --2- 172.21.15.36:0/3626833229 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2bbc0b2cd0 0x7f2bbc145ae0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f2bac001550 tx=0x7f2bac007ed0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:19.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.273+0000 7f2bab7fe700 1 -- 172.21.15.36:0/3626833229 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bac005ba0 con 0x7f2bbc0b2cd0 2022-06-17T12:49:19.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.273+0000 7f2bab7fe700 1 -- 172.21.15.36:0/3626833229 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac005d00 con 0x7f2bbc0b2cd0 2022-06-17T12:49:19.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.273+0000 7f2bbb59e700 1 -- 172.21.15.36:0/3626833229 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bbc0a9e50 con 0x7f2bbc0b2cd0 2022-06-17T12:49:19.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.273+0000 7f2bab7fe700 1 -- 172.21.15.36:0/3626833229 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bac0072c0 con 0x7f2bbc0b2cd0 2022-06-17T12:49:19.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.273+0000 7f2bbb59e700 1 -- 172.21.15.36:0/3626833229 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bbc14c610 con 0x7f2bbc0b2cd0 2022-06-17T12:49:19.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.273+0000 7f2bab7fe700 1 -- 172.21.15.36:0/3626833229 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2bac010070 con 0x7f2bbc0b2cd0 2022-06-17T12:49:19.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.273+0000 7f2bbb59e700 1 -- 172.21.15.36:0/3626833229 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bbc0ab800 con 0x7f2bbc0b2cd0 2022-06-17T12:49:19.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.273+0000 7f2bab7fe700 1 --2- 172.21.15.36:0/3626833229 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b98053ad0 0x7f2b98055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:19.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.273+0000 7f2bab7fe700 1 -- 172.21.15.36:0/3626833229 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2bac06b9a0 con 0x7f2bbc0b2cd0 2022-06-17T12:49:19.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.273+0000 7f2bba59c700 1 --2- 172.21.15.36:0/3626833229 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b98053ad0 0x7f2b98055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:19.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.277+0000 7f2bba59c700 1 --2- 172.21.15.36:0/3626833229 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b98053ad0 0x7f2b98055f90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f2bb000d000 tx=0x7f2bb0007a20).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:19.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.281+0000 7f2bab7fe700 1 -- 172.21.15.36:0/3626833229 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2bac0427a0 con 0x7f2bbc0b2cd0 2022-06-17T12:49:19.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.445+0000 7f2bbb59e700 1 -- 172.21.15.36:0/3626833229 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bbc0a9b80 con 0x7f2bbc0b2cd0 2022-06-17T12:49:19.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.445+0000 7f2bab7fe700 1 -- 172.21.15.36:0/3626833229 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+2138 (secure 0 0 0) 0x7f2bac045dc0 con 0x7f2bbc0b2cd0 2022-06-17T12:49:19.484 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:49:19.485 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":19,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/3777038413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3777038413},{"type":"v1","addr":"172.21.15.36:6835","nonce":3777038413}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:49:14.023351+0000","modified":"2022-06-17T12:49:15.026177+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5292":{"gid":5292,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/4117532324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4117532324},{"type":"v1","addr":"172.21.15.70:6833","nonce":4117532324}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-06-17T12:49:19.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.445+0000 7f2bbb59e700 1 -- 172.21.15.36:0/3626833229 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b98053ad0 msgr2=0x7f2b98055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:19.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.445+0000 7f2bbb59e700 1 --2- 172.21.15.36:0/3626833229 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b98053ad0 0x7f2b98055f90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f2bb000d000 tx=0x7f2bb0007a20).stop 2022-06-17T12:49:19.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.445+0000 7f2bbb59e700 1 -- 172.21.15.36:0/3626833229 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2bbc0b2cd0 msgr2=0x7f2bbc145ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:19.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.445+0000 7f2bbb59e700 1 --2- 172.21.15.36:0/3626833229 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2bbc0b2cd0 0x7f2bbc145ae0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f2bac001550 tx=0x7f2bac007ed0).stop 2022-06-17T12:49:19.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.449+0000 7f2bbb59e700 1 -- 172.21.15.36:0/3626833229 shutdown_connections 2022-06-17T12:49:19.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.449+0000 7f2bbb59e700 1 --2- 172.21.15.36:0/3626833229 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b98053ad0 0x7f2b98055f90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:19.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.449+0000 7f2bbb59e700 1 --2- 172.21.15.36:0/3626833229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2bbc146d00 0x7f2bbc1446c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:19.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.449+0000 7f2bbb59e700 1 --2- 172.21.15.36:0/3626833229 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2bbc0b2cd0 0x7f2bbc145ae0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:19.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.449+0000 7f2bbb59e700 1 --2- 172.21.15.36:0/3626833229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bbc0b2200 0x7f2bbc0a7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:19.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.449+0000 7f2bbb59e700 1 -- 172.21.15.36:0/3626833229 >> 172.21.15.36:0/3626833229 conn(0x7f2bbc05f770 msgr2=0x7f2bbc0afe30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:19.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.449+0000 7f2bbb59e700 1 -- 172.21.15.36:0/3626833229 shutdown_connections 2022-06-17T12:49:19.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:19.449+0000 7f2bbb59e700 1 -- 172.21.15.36:0/3626833229 wait complete. 2022-06-17T12:49:19.494 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 19 2022-06-17T12:49:19.722 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:49:14.023351+0000', 'modified': '2022-06-17T12:49:15.026177+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5292}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5292': {'gid': 5292, 'name': 'b', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6833/4117532324', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 4117532324}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 4117532324}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:49:19.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:49:19.724 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-17T12:49:19.840 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":5292,"want_state":"up:active","state":"up:active","rank_uptime":5.8216637689999997,"mdsmap_epoch":19,"osdmap_epoch":45,"osdmap_epoch_barrier":44,"uptime":10.594098427} 2022-06-17T12:49:19.841 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 5292, "mdsmap_epoch": 19, "osdmap_epoch": 45, "osdmap_epoch_barrier": 44, "rank_uptime": 5.821663769, "state": "up:active", "uptime": 10.594098427, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:49:19.846 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:49:19.847 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-17T12:49:19.848 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi107.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-17T12:49:19.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:49:19.849 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:49:19.878 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:49:19.947 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:49:19.948 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:49:20.045 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:49:20.061 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:49:20.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:49:20.063 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:49:20.076 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:49:20.077 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:49:20.152 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:49:20.192 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:49:20.193 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:49:20.225 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:2022-06-17T12:49:20.231+0000 7f3e340bb080 -1 init, newargv = 0x55b39c111270 newargc=9 2022-06-17T12:49:20.228 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[21928]: starting ceph client 2022-06-17T12:49:20.237 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[21928]: starting fuse 2022-06-17T12:49:20.273 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:49:20.275 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:49:20.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:49:20.363 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:49:20.395 INFO:teuthology.orchestra.run.smithi107.stdout:52 2022-06-17T12:49:20.397 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:49:20.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:49:20.399 DEBUG:teuthology.orchestra.run.smithi107:> sudo python3 -c ' 2022-06-17T12:49:20.399 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2022-06-17T12:49:20.400 DEBUG:teuthology.orchestra.run.smithi107:> import re 2022-06-17T12:49:20.401 DEBUG:teuthology.orchestra.run.smithi107:> import os 2022-06-17T12:49:20.402 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2022-06-17T12:49:20.485 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:49:20.485 DEBUG:teuthology.orchestra.run.smithi107:> def find_socket(client_name): 2022-06-17T12:49:20.486 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-17T12:49:20.487 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2022-06-17T12:49:20.487 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:49:20.488 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2022-06-17T12:49:20.507 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2022-06-17T12:49:20.507 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2022-06-17T12:49:20.508 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2022-06-17T12:49:20.509 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:49:20.510 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2022-06-17T12:49:20.510 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:49:20.511 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:49:20.512 DEBUG:teuthology.orchestra.run.smithi107:> return f 2022-06-17T12:49:20.512 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:49:20.513 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:49:20.538 DEBUG:teuthology.orchestra.run.smithi107:> print(find_socket("client.0")) 2022-06-17T12:49:20.539 DEBUG:teuthology.orchestra.run.smithi107:> ' 2022-06-17T12:49:20.630 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.0.21928.asok 2022-06-17T12:49:20.631 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.21928.asok 2022-06-17T12:49:20.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:49:20.633 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.21928.asok status 2022-06-17T12:49:20.788 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2022-06-17T12:49:20.789 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2022-06-17T12:49:20.789 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:49:20.790 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:49:20.790 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "0", 2022-06-17T12:49:20.791 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2022-06-17T12:49:20.791 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-17T12:49:20.792 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "21928", 2022-06-17T12:49:20.792 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/" 2022-06-17T12:49:20.793 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:49:20.793 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2022-06-17T12:49:20.793 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2022-06-17T12:49:20.794 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 5380, 2022-06-17T12:49:20.795 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2022-06-17T12:49:20.796 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2022-06-17T12:49:20.796 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2022-06-17T12:49:20.796 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 5380 2022-06-17T12:49:20.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:49:20.797 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:49:20.798 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:49:20.798 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:49:20.798 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 343763098 2022-06-17T12:49:20.799 INFO:teuthology.orchestra.run.smithi107.stdout: } 2022-06-17T12:49:20.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:49:20.799 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:49:20.800 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:49:20.800 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:49:20.801 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 343763098 2022-06-17T12:49:20.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:49:20.809 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.5380 172.21.15.107:0/343763098", 2022-06-17T12:49:20.809 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "172.21.15.107:0/343763098", 2022-06-17T12:49:20.809 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 1, 2022-06-17T12:49:20.810 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 19, 2022-06-17T12:49:20.810 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 45, 2022-06-17T12:49:20.811 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:49:20.811 INFO:teuthology.orchestra.run.smithi107.stdout: "blacklisted": false, 2022-06-17T12:49:20.814 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2022-06-17T12:49:20.836 INFO:teuthology.orchestra.run.smithi107.stdout:} 2022-06-17T12:49:20.862 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:49:20.863 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:49:20.864 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:49:20.882 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:49:20.883 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:49:20.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:49:20.884 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:49:20.984 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:49:20.985 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:49:20.985 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:49:21.055 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:49:21.056 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:49:21.057 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-06-17T12:49:21.058 DEBUG:teuthology.orchestra.run.smithi107:> import sys 2022-06-17T12:49:21.059 DEBUG:teuthology.orchestra.run.smithi107:> import time 2022-06-17T12:49:21.061 DEBUG:teuthology.orchestra.run.smithi107:> import os 2022-06-17T12:49:21.061 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:49:21.062 DEBUG:teuthology.orchestra.run.smithi107:> n = 1000 2022-06-17T12:49:21.063 DEBUG:teuthology.orchestra.run.smithi107:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2022-06-17T12:49:21.064 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:49:21.065 DEBUG:teuthology.orchestra.run.smithi107:> if not os.path.exists(os.path.dirname(abs_path)): 2022-06-17T12:49:21.066 DEBUG:teuthology.orchestra.run.smithi107:> os.makedirs(os.path.dirname(abs_path)) 2022-06-17T12:49:21.068 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:49:21.069 DEBUG:teuthology.orchestra.run.smithi107:> for i in range(0, n): 2022-06-17T12:49:21.069 DEBUG:teuthology.orchestra.run.smithi107:> fname = "{0}_{1}".format(abs_path, i) 2022-06-17T12:49:21.070 DEBUG:teuthology.orchestra.run.smithi107:> with open(fname, '"'"'w'"'"') as f: 2022-06-17T12:49:21.071 DEBUG:teuthology.orchestra.run.smithi107:> f.write('"'"'content'"'"') 2022-06-17T12:49:21.072 DEBUG:teuthology.orchestra.run.smithi107:> if True: 2022-06-17T12:49:21.072 DEBUG:teuthology.orchestra.run.smithi107:> f.flush() 2022-06-17T12:49:21.073 DEBUG:teuthology.orchestra.run.smithi107:> os.fsync(f.fileno()) 2022-06-17T12:49:21.074 DEBUG:teuthology.orchestra.run.smithi107:> ' 2022-06-17T12:49:32.224 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:49:32.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.353+0000 7fcfc359e700 1 Processor -- start 2022-06-17T12:49:32.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.353+0000 7fcfc359e700 1 -- start start 2022-06-17T12:49:32.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc359e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcfc40b2200 0x7fcfc40af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:32.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc359e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcfc40b2cd0 0x7fcfc4147600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:32.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc359e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfc40b25e0 con 0x7fcfc40af8d0 2022-06-17T12:49:32.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc359e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfc40ad790 con 0x7fcfc40b2cd0 2022-06-17T12:49:32.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc359e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc40aef40 con 0x7fcfc40b2200 2022-06-17T12:49:32.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc2d9d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcfc40b2cd0 0x7fcfc4147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:32.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc259c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcfc40b2200 0x7fcfc40af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:32.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc2d9d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcfc40b2cd0 0x7fcfc4147600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:53946/0 (socket says 172.21.15.36:53946) 2022-06-17T12:49:32.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc2d9d700 1 -- 172.21.15.36:0/942370083 learned_addr learned my addr 172.21.15.36:0/942370083 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:32.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc2d9d700 1 -- 172.21.15.36:0/942370083 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcfc40b2200 msgr2=0x7fcfc40af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:32.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc2d9d700 1 --2- 172.21.15.36:0/942370083 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcfc40b2200 0x7fcfc40af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:32.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc2d9d700 1 -- 172.21.15.36:0/942370083 >> v1:172.21.15.70:6789/0 conn(0x7fcfc40af8d0 legacy=0x7fcfc414a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:32.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc2d9d700 1 -- 172.21.15.36:0/942370083 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc40b1950 con 0x7fcfc40b2cd0 2022-06-17T12:49:32.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc2d9d700 1 --2- 172.21.15.36:0/942370083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfc40b2cd0 0x7fcfc4147600 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fcfbc0032f0 tx=0x7fcfbc009290).ready entity=mon.1 client_cookie=1a52980bdfae741f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:32.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc159a700 1 -- 172.21.15.36:0/942370083 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfbc00b8a0 con 0x7fcfc40b2cd0 2022-06-17T12:49:32.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc159a700 1 -- 172.21.15.36:0/942370083 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfbc00ba00 con 0x7fcfc40b2cd0 2022-06-17T12:49:32.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc159a700 1 -- 172.21.15.36:0/942370083 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfbc00bb70 con 0x7fcfc40b2cd0 2022-06-17T12:49:32.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc359e700 1 -- 172.21.15.36:0/942370083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfc40b2cd0 msgr2=0x7fcfc4147600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:32.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc359e700 1 --2- 172.21.15.36:0/942370083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfc40b2cd0 0x7fcfc4147600 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fcfbc0032f0 tx=0x7fcfbc009290).stop 2022-06-17T12:49:32.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc359e700 1 -- 172.21.15.36:0/942370083 shutdown_connections 2022-06-17T12:49:32.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc359e700 1 --2- 172.21.15.36:0/942370083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfc40b2cd0 0x7fcfc4147600 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:32.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc359e700 1 --2- 172.21.15.36:0/942370083 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcfc40b2200 0x7fcfc40af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:32.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc359e700 1 -- 172.21.15.36:0/942370083 >> 172.21.15.36:0/942370083 conn(0x7fcfc405f770 msgr2=0x7fcfc405fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:32.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc359e700 1 -- 172.21.15.36:0/942370083 shutdown_connections 2022-06-17T12:49:32.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.357+0000 7fcfc359e700 1 -- 172.21.15.36:0/942370083 wait complete. 2022-06-17T12:49:32.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.361+0000 7fcfc359e700 1 Processor -- start 2022-06-17T12:49:32.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.361+0000 7fcfc359e700 1 -- start start 2022-06-17T12:49:32.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.361+0000 7fcfc359e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfc40af8d0 0x7fcfc4156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:32.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.361+0000 7fcfc359e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcfc40b2200 0x7fcfc414e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:32.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.361+0000 7fcfc359e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcfc40b2cd0 0x7fcfc40a84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:32.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.361+0000 7fcfc359e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc40a9b80 con 0x7fcfc40b2cd0 2022-06-17T12:49:32.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.361+0000 7fcfc359e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc40a9e20 con 0x7fcfc40af8d0 2022-06-17T12:49:32.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.361+0000 7fcfc359e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc40a7180 con 0x7fcfc40b2200 2022-06-17T12:49:32.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.361+0000 7fcfc2d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcfc40b2cd0 0x7fcfc40a84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:32.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.361+0000 7fcfc1d9b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcfc40b2200 0x7fcfc414e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:32.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.361+0000 7fcfc259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfc40af8d0 0x7fcfc4156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:32.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.361+0000 7fcfc2d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcfc40b2cd0 0x7fcfc40a84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33570/0 (socket says 172.21.15.36:33570) 2022-06-17T12:49:32.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.361+0000 7fcfc1d9b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcfc40b2200 0x7fcfc414e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:48358/0 (socket says 172.21.15.36:48358) 2022-06-17T12:49:32.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.361+0000 7fcfc2d9d700 1 -- 172.21.15.36:0/4038378780 learned_addr learned my addr 172.21.15.36:0/4038378780 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:32.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.361+0000 7fcfc1d9b700 1 -- 172.21.15.36:0/4038378780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfc40af8d0 msgr2=0x7fcfc4156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:32.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.361+0000 7fcfc1d9b700 1 --2- 172.21.15.36:0/4038378780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfc40af8d0 0x7fcfc4156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:32.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.361+0000 7fcfc1d9b700 1 -- 172.21.15.36:0/4038378780 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcfc40b2cd0 msgr2=0x7fcfc40a84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:32.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.361+0000 7fcfc1d9b700 1 --2- 172.21.15.36:0/4038378780 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcfc40b2cd0 0x7fcfc40a84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:32.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.361+0000 7fcfc1d9b700 1 -- 172.21.15.36:0/4038378780 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfbc009530 con 0x7fcfc40b2200 2022-06-17T12:49:32.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.365+0000 7fcfc1d9b700 1 --2- 172.21.15.36:0/4038378780 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcfc40b2200 0x7fcfc414e3f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fcfc40afcd0 tx=0x7fcfb4007b50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:32.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.365+0000 7fcfb37fe700 1 -- 172.21.15.36:0/4038378780 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfb400ce60 con 0x7fcfc40b2200 2022-06-17T12:49:32.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.365+0000 7fcfb37fe700 1 -- 172.21.15.36:0/4038378780 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb4007030 con 0x7fcfc40b2200 2022-06-17T12:49:32.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.365+0000 7fcfc359e700 1 -- 172.21.15.36:0/4038378780 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc40a9800 con 0x7fcfc40b2200 2022-06-17T12:49:32.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.365+0000 7fcfb37fe700 1 -- 172.21.15.36:0/4038378780 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfb4007390 con 0x7fcfc40b2200 2022-06-17T12:49:32.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.365+0000 7fcfc359e700 1 -- 172.21.15.36:0/4038378780 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc40a7620 con 0x7fcfc40b2200 2022-06-17T12:49:32.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.365+0000 7fcfc359e700 1 -- 172.21.15.36:0/4038378780 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc40a7340 con 0x7fcfc40b2200 2022-06-17T12:49:32.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.365+0000 7fcfb37fe700 1 -- 172.21.15.36:0/4038378780 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fcfb4006510 con 0x7fcfc40b2200 2022-06-17T12:49:32.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.365+0000 7fcfb37fe700 1 --2- 172.21.15.36:0/4038378780 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcfa0053a60 0x7fcfa0055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:32.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.365+0000 7fcfb37fe700 1 -- 172.21.15.36:0/4038378780 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcfb406b9a0 con 0x7fcfc40b2200 2022-06-17T12:49:32.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.365+0000 7fcfc259c700 1 --2- 172.21.15.36:0/4038378780 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcfa0053a60 0x7fcfa0055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:32.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.369+0000 7fcfc259c700 1 --2- 172.21.15.36:0/4038378780 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcfa0053a60 0x7fcfa0055f20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fcfb800b000 tx=0x7fcfb8006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:32.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.373+0000 7fcfb37fe700 1 -- 172.21.15.36:0/4038378780 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcfb4042630 con 0x7fcfc40b2200 2022-06-17T12:49:32.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.537+0000 7fcfc359e700 1 -- 172.21.15.36:0/4038378780 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfc40ad330 con 0x7fcfc40b2200 2022-06-17T12:49:32.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.537+0000 7fcfb37fe700 1 -- 172.21.15.36:0/4038378780 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+2138 (secure 0 0 0) 0x7fcfb4045c50 con 0x7fcfc40b2200 2022-06-17T12:49:32.535 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:49:32.535 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":19,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/3777038413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3777038413},{"type":"v1","addr":"172.21.15.36:6835","nonce":3777038413}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:49:14.023351+0000","modified":"2022-06-17T12:49:15.026177+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5292":{"gid":5292,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/4117532324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4117532324},{"type":"v1","addr":"172.21.15.70:6833","nonce":4117532324}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-06-17T12:49:32.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.541+0000 7fcfc359e700 1 -- 172.21.15.36:0/4038378780 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcfa0053a60 msgr2=0x7fcfa0055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:32.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.541+0000 7fcfc359e700 1 --2- 172.21.15.36:0/4038378780 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcfa0053a60 0x7fcfa0055f20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fcfb800b000 tx=0x7fcfb8006df0).stop 2022-06-17T12:49:32.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.541+0000 7fcfc359e700 1 -- 172.21.15.36:0/4038378780 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcfc40b2200 msgr2=0x7fcfc414e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:32.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.541+0000 7fcfc359e700 1 --2- 172.21.15.36:0/4038378780 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcfc40b2200 0x7fcfc414e3f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fcfc40afcd0 tx=0x7fcfb4007b50).stop 2022-06-17T12:49:32.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.541+0000 7fcfc359e700 1 -- 172.21.15.36:0/4038378780 shutdown_connections 2022-06-17T12:49:32.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.541+0000 7fcfc359e700 1 --2- 172.21.15.36:0/4038378780 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcfa0053a60 0x7fcfa0055f20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:32.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.541+0000 7fcfc359e700 1 --2- 172.21.15.36:0/4038378780 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcfc40b2cd0 0x7fcfc40a84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:32.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.541+0000 7fcfc359e700 1 --2- 172.21.15.36:0/4038378780 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcfc40b2200 0x7fcfc414e3f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:32.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.541+0000 7fcfc359e700 1 --2- 172.21.15.36:0/4038378780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfc40af8d0 0x7fcfc4156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:32.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.541+0000 7fcfc359e700 1 -- 172.21.15.36:0/4038378780 >> 172.21.15.36:0/4038378780 conn(0x7fcfc405f770 msgr2=0x7fcfc41463c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:32.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.541+0000 7fcfc359e700 1 -- 172.21.15.36:0/4038378780 shutdown_connections 2022-06-17T12:49:32.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.541+0000 7fcfc359e700 1 -- 172.21.15.36:0/4038378780 wait complete. 2022-06-17T12:49:32.546 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 19 2022-06-17T12:49:32.548 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_recall_max_caps 20 2022-06-17T12:49:32.695 INFO:teuthology.orchestra.run.smithi070.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2022-06-17T12:49:32.696 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2022-06-17T12:49:32.697 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:49:32.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f4155d3f700 1 Processor -- start 2022-06-17T12:49:32.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f4155d3f700 1 -- start start 2022-06-17T12:49:32.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f4155d3f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f41500b0cb0 0x7f4150142f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:32.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f4155d3f700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4150143e80 0x7f4150146970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:32.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f4155d3f700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41500b44b0 con 0x7f4150062270 2022-06-17T12:49:32.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f4155d3f700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41500b6650 con 0x7f41500b0cb0 2022-06-17T12:49:32.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f4155d3f700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4150062650 con 0x7f4150143e80 2022-06-17T12:49:32.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f4146ffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f41500b0cb0 0x7f4150142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:32.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f4146ffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f41500b0cb0 0x7f4150142f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33574/0 (socket says 172.21.15.36:33574) 2022-06-17T12:49:32.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f414f7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f4150062270 0x7f4150143770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51104/0 (socket says 172.21.15.36:51104) 2022-06-17T12:49:32.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f4146ffd700 1 -- 172.21.15.36:0/1275337984 learned_addr learned my addr 172.21.15.36:0/1275337984 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:32.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f414ffff700 1 --2- 172.21.15.36:0/1275337984 >> v2:172.21.15.70:3300/0 conn(0x7f4150143e80 0x7f4150146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:32.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f4146ffd700 1 -- 172.21.15.36:0/1275337984 >> v2:172.21.15.70:3300/0 conn(0x7f4150143e80 msgr2=0x7f4150146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:32.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f414effd700 1 -- 172.21.15.36:0/1275337984 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1489792913 0 0) 0x7f41500b44b0 con 0x7f4150062270 2022-06-17T12:49:32.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f4146ffd700 1 --2- 172.21.15.36:0/1275337984 >> v2:172.21.15.70:3300/0 conn(0x7f4150143e80 0x7f4150146970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:32.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f4146ffd700 1 -- 172.21.15.36:0/1275337984 >> v1:172.21.15.36:6789/0 conn(0x7f4150062270 legacy=0x7f4150143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:32.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f4146ffd700 1 -- 172.21.15.36:0/1275337984 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41500b4210 con 0x7f41500b0cb0 2022-06-17T12:49:32.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f4146ffd700 1 --2- 172.21.15.36:0/1275337984 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f41500b0cb0 0x7f4150142f20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f4138002f70 tx=0x7f4138006eb0).ready entity=mon.0 client_cookie=dca221e83e3b3983 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:32.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f414effd700 1 -- 172.21.15.36:0/1275337984 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f413800d610 con 0x7f41500b0cb0 2022-06-17T12:49:32.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f414effd700 1 -- 172.21.15.36:0/1275337984 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4138000e90 con 0x7f41500b0cb0 2022-06-17T12:49:32.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f414effd700 1 -- 172.21.15.36:0/1275337984 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f413800d960 con 0x7f41500b0cb0 2022-06-17T12:49:32.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f4155d3f700 1 -- 172.21.15.36:0/1275337984 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f41500b0cb0 msgr2=0x7f4150142f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:32.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f4155d3f700 1 --2- 172.21.15.36:0/1275337984 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f41500b0cb0 0x7f4150142f20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f4138002f70 tx=0x7f4138006eb0).stop 2022-06-17T12:49:32.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f4155d3f700 1 -- 172.21.15.36:0/1275337984 shutdown_connections 2022-06-17T12:49:32.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f4155d3f700 1 --2- 172.21.15.36:0/1275337984 >> v2:172.21.15.70:3300/0 conn(0x7f4150143e80 0x7f4150146970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:32.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f4155d3f700 1 --2- 172.21.15.36:0/1275337984 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f41500b0cb0 0x7f4150142f20 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:32.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f4155d3f700 1 -- 172.21.15.36:0/1275337984 >> 172.21.15.36:0/1275337984 conn(0x7f41500b9870 msgr2=0x7f41500b9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:32.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.825+0000 7f4155d3f700 1 -- 172.21.15.36:0/1275337984 shutdown_connections 2022-06-17T12:49:32.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.829+0000 7f4155d3f700 1 -- 172.21.15.36:0/1275337984 wait complete. 2022-06-17T12:49:32.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.829+0000 7f4155d3f700 1 Processor -- start 2022-06-17T12:49:32.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.829+0000 7f4155d3f700 1 -- start start 2022-06-17T12:49:32.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.829+0000 7f4155d3f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4150062270 0x7f41500af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:32.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.829+0000 7f4155d3f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4150143e80 0x7f41500b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:32.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.829+0000 7f4155d3f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f41500aea90 0x7f4150148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:32.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.829+0000 7f4155d3f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4150146710 con 0x7f41500aea90 2022-06-17T12:49:32.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.829+0000 7f4155d3f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41500ae770 con 0x7f4150143e80 2022-06-17T12:49:32.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.829+0000 7f4155d3f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4150146940 con 0x7f4150062270 2022-06-17T12:49:32.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.829+0000 7f414ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f41500aea90 0x7f4150148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:32.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.829+0000 7f414ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f41500aea90 0x7f4150148150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33584/0 (socket says 172.21.15.36:33584) 2022-06-17T12:49:32.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.829+0000 7f4146ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4150143e80 0x7f41500b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:32.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.829+0000 7f414ffff700 1 -- 172.21.15.36:0/1710843291 learned_addr learned my addr 172.21.15.36:0/1710843291 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:32.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.829+0000 7f414f7fe700 1 --2- 172.21.15.36:0/1710843291 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4150062270 0x7f41500af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:32.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.829+0000 7f414ffff700 1 -- 172.21.15.36:0/1710843291 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4150062270 msgr2=0x7f41500af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:32.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.829+0000 7f414ffff700 1 --2- 172.21.15.36:0/1710843291 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4150062270 0x7f41500af1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:32.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.829+0000 7f414ffff700 1 -- 172.21.15.36:0/1710843291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4150143e80 msgr2=0x7f41500b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:32.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.829+0000 7f414ffff700 1 --2- 172.21.15.36:0/1710843291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4150143e80 0x7f41500b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:32.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.833+0000 7f414ffff700 1 -- 172.21.15.36:0/1710843291 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413800b1e0 con 0x7f41500aea90 2022-06-17T12:49:32.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.833+0000 7f414ffff700 1 --2- 172.21.15.36:0/1710843291 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f41500aea90 0x7f4150148150 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f414800a000 tx=0x7f4148008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:32.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.833+0000 7f414d7fa700 1 -- 172.21.15.36:0/1710843291 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41480127e0 con 0x7f41500aea90 2022-06-17T12:49:32.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.833+0000 7f4155d3f700 1 -- 172.21.15.36:0/1710843291 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41500ae460 con 0x7f41500aea90 2022-06-17T12:49:32.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.833+0000 7f414d7fa700 1 -- 172.21.15.36:0/1710843291 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4148012940 con 0x7f41500aea90 2022-06-17T12:49:32.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.833+0000 7f4155d3f700 1 -- 172.21.15.36:0/1710843291 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4150148640 con 0x7f41500aea90 2022-06-17T12:49:32.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.833+0000 7f414d7fa700 1 -- 172.21.15.36:0/1710843291 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4148012ca0 con 0x7f41500aea90 2022-06-17T12:49:32.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.833+0000 7f4155d3f700 1 -- 172.21.15.36:0/1710843291 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4150146b00 con 0x7f41500aea90 2022-06-17T12:49:32.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.833+0000 7f414d7fa700 1 -- 172.21.15.36:0/1710843291 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f4148013480 con 0x7f41500aea90 2022-06-17T12:49:32.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.833+0000 7f414d7fa700 1 --2- 172.21.15.36:0/1710843291 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4134053a40 0x7f4134055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:32.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.833+0000 7f414d7fa700 1 -- 172.21.15.36:0/1710843291 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f414806c4c0 con 0x7f41500aea90 2022-06-17T12:49:32.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.833+0000 7f414f7fe700 1 --2- 172.21.15.36:0/1710843291 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4134053a40 0x7f4134055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:32.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.837+0000 7f414f7fe700 1 --2- 172.21.15.36:0/1710843291 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4134053a40 0x7f4134055f00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f414000d000 tx=0x7f4140007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:32.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:32.841+0000 7f414d7fa700 1 -- 172.21.15.36:0/1710843291 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f41480430d0 con 0x7f41500aea90 2022-06-17T12:49:32.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.005+0000 7f4155d3f700 1 -- 172.21.15.36:0/1710843291 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41500b63b0 con 0x7f41500aea90 2022-06-17T12:49:32.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.005+0000 7f414d7fa700 1 -- 172.21.15.36:0/1710843291 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+2138 (secure 0 0 0) 0x7f4148012e00 con 0x7f41500aea90 2022-06-17T12:49:32.998 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:49:32.998 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":19,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/3777038413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3777038413},{"type":"v1","addr":"172.21.15.36:6835","nonce":3777038413}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:49:14.023351+0000","modified":"2022-06-17T12:49:15.026177+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5292":{"gid":5292,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/4117532324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4117532324},{"type":"v1","addr":"172.21.15.70:6833","nonce":4117532324}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-06-17T12:49:33.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.005+0000 7f4155d3f700 1 -- 172.21.15.36:0/1710843291 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4134053a40 msgr2=0x7f4134055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:33.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.005+0000 7f4155d3f700 1 --2- 172.21.15.36:0/1710843291 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4134053a40 0x7f4134055f00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f414000d000 tx=0x7f4140007a00).stop 2022-06-17T12:49:33.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.005+0000 7f4155d3f700 1 -- 172.21.15.36:0/1710843291 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f41500aea90 msgr2=0x7f4150148150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:33.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.005+0000 7f4155d3f700 1 --2- 172.21.15.36:0/1710843291 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f41500aea90 0x7f4150148150 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f414800a000 tx=0x7f4148008040).stop 2022-06-17T12:49:33.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.009+0000 7f4155d3f700 1 -- 172.21.15.36:0/1710843291 shutdown_connections 2022-06-17T12:49:33.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.009+0000 7f4155d3f700 1 --2- 172.21.15.36:0/1710843291 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4134053a40 0x7f4134055f00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.009+0000 7f4155d3f700 1 --2- 172.21.15.36:0/1710843291 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f41500aea90 0x7f4150148150 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.009+0000 7f4155d3f700 1 --2- 172.21.15.36:0/1710843291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4150143e80 0x7f41500b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.009+0000 7f4155d3f700 1 --2- 172.21.15.36:0/1710843291 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4150062270 0x7f41500af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.009+0000 7f4155d3f700 1 -- 172.21.15.36:0/1710843291 >> 172.21.15.36:0/1710843291 conn(0x7f41500b9870 msgr2=0x7f41500b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:33.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.009+0000 7f4155d3f700 1 -- 172.21.15.36:0/1710843291 shutdown_connections 2022-06-17T12:49:33.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.009+0000 7f4155d3f700 1 -- 172.21.15.36:0/1710843291 wait complete. 2022-06-17T12:49:33.015 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 19 2022-06-17T12:49:33.023 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_recall_max_decay_threshold 400 2022-06-17T12:49:33.155 INFO:teuthology.orchestra.run.smithi070.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2022-06-17T12:49:33.156 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2022-06-17T12:49:33.157 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@smithi107.front.sepia.ceph.com... 2022-06-17T12:49:33.187 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:49:33.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.321+0000 7f48e906c700 1 Processor -- start 2022-06-17T12:49:33.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.321+0000 7f48e906c700 1 -- start start 2022-06-17T12:49:33.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.321+0000 7f48e906c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f48e40b0cb0 0x7f48e4142f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.321+0000 7f48e906c700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48e40b44b0 con 0x7f48e4143e80 2022-06-17T12:49:33.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.321+0000 7f48e906c700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48e4062650 con 0x7f48e4062270 2022-06-17T12:49:33.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.321+0000 7f48e906c700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e40b6650 con 0x7f48e40b0cb0 2022-06-17T12:49:33.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.321+0000 7f48e259c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f48e40b0cb0 0x7f48e4142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.321+0000 7f48e359e700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f48e4143e80 0x7f48e4146970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51122/0 (socket says 172.21.15.36:51122) 2022-06-17T12:49:33.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.321+0000 7f48e359e700 1 -- 172.21.15.36:0/646126185 learned_addr learned my addr 172.21.15.36:0/646126185 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:33.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.321+0000 7f48e259c700 1 -- 172.21.15.36:0/646126185 >> v1:172.21.15.70:6789/0 conn(0x7f48e4062270 legacy=0x7f48e4143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:33.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.321+0000 7f48e259c700 1 -- 172.21.15.36:0/646126185 >> v1:172.21.15.36:6789/0 conn(0x7f48e4143e80 legacy=0x7f48e4146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:33.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.321+0000 7f48e1d9b700 1 -- 172.21.15.36:0/646126185 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2639414447 0 0) 0x7f48e40b44b0 con 0x7f48e4143e80 2022-06-17T12:49:33.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.321+0000 7f48e259c700 1 -- 172.21.15.36:0/646126185 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e40b4210 con 0x7f48e40b0cb0 2022-06-17T12:49:33.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.321+0000 7f48e259c700 1 --2- 172.21.15.36:0/646126185 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f48e40b0cb0 0x7f48e4142f20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f48cc002f70 tx=0x7f48cc006df0).ready entity=mon.0 client_cookie=d041bec557a39bb3 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:33.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.321+0000 7f48e1d9b700 1 -- 172.21.15.36:0/646126185 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48cc00d610 con 0x7f48e40b0cb0 2022-06-17T12:49:33.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.321+0000 7f48e1d9b700 1 -- 172.21.15.36:0/646126185 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48cc000e90 con 0x7f48e40b0cb0 2022-06-17T12:49:33.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.321+0000 7f48e1d9b700 1 -- 172.21.15.36:0/646126185 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48cc00d960 con 0x7f48e40b0cb0 2022-06-17T12:49:33.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.325+0000 7f48e906c700 1 -- 172.21.15.36:0/646126185 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f48e40b0cb0 msgr2=0x7f48e4142f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:33.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.325+0000 7f48e906c700 1 --2- 172.21.15.36:0/646126185 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f48e40b0cb0 0x7f48e4142f20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f48cc002f70 tx=0x7f48cc006df0).stop 2022-06-17T12:49:33.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.325+0000 7f48e906c700 1 -- 172.21.15.36:0/646126185 shutdown_connections 2022-06-17T12:49:33.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.325+0000 7f48e906c700 1 --2- 172.21.15.36:0/646126185 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f48e40b0cb0 0x7f48e4142f20 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.325+0000 7f48e906c700 1 -- 172.21.15.36:0/646126185 >> 172.21.15.36:0/646126185 conn(0x7f48e40b9870 msgr2=0x7f48e40b9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:33.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.325+0000 7f48e906c700 1 -- 172.21.15.36:0/646126185 shutdown_connections 2022-06-17T12:49:33.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.325+0000 7f48e906c700 1 -- 172.21.15.36:0/646126185 wait complete. 2022-06-17T12:49:33.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.325+0000 7f48e906c700 1 Processor -- start 2022-06-17T12:49:33.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.325+0000 7f48e906c700 1 -- start start 2022-06-17T12:49:33.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.329+0000 7f48e906c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f48e4062270 0x7f48e4144b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.329+0000 7f48e906c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48e4143e80 0x7f48e413e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.329+0000 7f48e906c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f48e4152d00 0x7f48e413ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.329+0000 7f48e906c700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e41534b0 con 0x7f48e4062270 2022-06-17T12:49:33.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.329+0000 7f48e906c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e413f0e0 con 0x7f48e4143e80 2022-06-17T12:49:33.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.329+0000 7f48e906c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e4146890 con 0x7f48e4152d00 2022-06-17T12:49:33.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.329+0000 7f48e2d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f48e4062270 0x7f48e4144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.329+0000 7f48e2d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f48e4062270 0x7f48e4144b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33594/0 (socket says 172.21.15.36:33594) 2022-06-17T12:49:33.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.329+0000 7f48e2d9d700 1 -- 172.21.15.36:0/2455374495 learned_addr learned my addr 172.21.15.36:0/2455374495 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:33.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.329+0000 7f48e359e700 1 --2- 172.21.15.36:0/2455374495 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f48e4152d00 0x7f48e413ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.329+0000 7f48e259c700 1 --2- 172.21.15.36:0/2455374495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48e4143e80 0x7f48e413e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.329+0000 7f48e2d9d700 1 -- 172.21.15.36:0/2455374495 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f48e4152d00 msgr2=0x7f48e413ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:33.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.329+0000 7f48e2d9d700 1 --2- 172.21.15.36:0/2455374495 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f48e4152d00 0x7f48e413ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.329+0000 7f48e2d9d700 1 -- 172.21.15.36:0/2455374495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48e4143e80 msgr2=0x7f48e413e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:33.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.329+0000 7f48e2d9d700 1 --2- 172.21.15.36:0/2455374495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48e4143e80 0x7f48e413e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.329+0000 7f48e2d9d700 1 -- 172.21.15.36:0/2455374495 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48cc00b180 con 0x7f48e4062270 2022-06-17T12:49:33.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.329+0000 7f48e2d9d700 1 --2- 172.21.15.36:0/2455374495 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f48e4062270 0x7f48e4144b00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f48e4063520 tx=0x7f48d4007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:33.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.329+0000 7f48dbfff700 1 -- 172.21.15.36:0/2455374495 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48d400ce80 con 0x7f48e4062270 2022-06-17T12:49:33.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.329+0000 7f48dbfff700 1 -- 172.21.15.36:0/2455374495 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d4012400 con 0x7f48e4062270 2022-06-17T12:49:33.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.329+0000 7f48e906c700 1 -- 172.21.15.36:0/2455374495 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e4146b30 con 0x7f48e4062270 2022-06-17T12:49:33.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.329+0000 7f48dbfff700 1 -- 172.21.15.36:0/2455374495 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48d4012760 con 0x7f48e4062270 2022-06-17T12:49:33.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.329+0000 7f48e906c700 1 -- 172.21.15.36:0/2455374495 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e4149530 con 0x7f48e4062270 2022-06-17T12:49:33.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.333+0000 7f48dbfff700 1 -- 172.21.15.36:0/2455374495 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f48d4019030 con 0x7f48e4062270 2022-06-17T12:49:33.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.333+0000 7f48e906c700 1 -- 172.21.15.36:0/2455374495 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e4146cc0 con 0x7f48e4062270 2022-06-17T12:49:33.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.333+0000 7f48dbfff700 1 --2- 172.21.15.36:0/2455374495 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f48c8053af0 0x7f48c8055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.333+0000 7f48dbfff700 1 -- 172.21.15.36:0/2455374495 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f48d406b290 con 0x7f48e4062270 2022-06-17T12:49:33.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.333+0000 7f48e259c700 1 --2- 172.21.15.36:0/2455374495 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f48c8053af0 0x7f48c8055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.333+0000 7f48e259c700 1 --2- 172.21.15.36:0/2455374495 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f48c8053af0 0x7f48c8055fb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f48cc01c000 tx=0x7f48cc015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:33.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.341+0000 7f48dbfff700 1 -- 172.21.15.36:0/2455374495 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f48d4042090 con 0x7f48e4062270 2022-06-17T12:49:33.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.501+0000 7f48e906c700 1 -- 172.21.15.36:0/2455374495 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48e40b63b0 con 0x7f48e4062270 2022-06-17T12:49:33.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.501+0000 7f48dbfff700 1 -- 172.21.15.36:0/2455374495 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+2138 (secure 0 0 0) 0x7f48d40456b0 con 0x7f48e4062270 2022-06-17T12:49:33.500 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:49:33.500 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":19,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/3777038413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3777038413},{"type":"v1","addr":"172.21.15.36:6835","nonce":3777038413}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:49:14.023351+0000","modified":"2022-06-17T12:49:15.026177+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5292":{"gid":5292,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/4117532324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4117532324},{"type":"v1","addr":"172.21.15.70:6833","nonce":4117532324}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-06-17T12:49:33.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.505+0000 7f48e906c700 1 -- 172.21.15.36:0/2455374495 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f48c8053af0 msgr2=0x7f48c8055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:33.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.505+0000 7f48e906c700 1 --2- 172.21.15.36:0/2455374495 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f48c8053af0 0x7f48c8055fb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f48cc01c000 tx=0x7f48cc015000).stop 2022-06-17T12:49:33.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.505+0000 7f48e906c700 1 -- 172.21.15.36:0/2455374495 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f48e4062270 msgr2=0x7f48e4144b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:33.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.505+0000 7f48e906c700 1 --2- 172.21.15.36:0/2455374495 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f48e4062270 0x7f48e4144b00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f48e4063520 tx=0x7f48d4007c50).stop 2022-06-17T12:49:33.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.505+0000 7f48e906c700 1 -- 172.21.15.36:0/2455374495 shutdown_connections 2022-06-17T12:49:33.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.505+0000 7f48e906c700 1 --2- 172.21.15.36:0/2455374495 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f48c8053af0 0x7f48c8055fb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.505+0000 7f48e906c700 1 --2- 172.21.15.36:0/2455374495 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f48e4152d00 0x7f48e413ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.505+0000 7f48e906c700 1 --2- 172.21.15.36:0/2455374495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48e4143e80 0x7f48e413e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.505+0000 7f48e906c700 1 --2- 172.21.15.36:0/2455374495 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f48e4062270 0x7f48e4144b00 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.505+0000 7f48e906c700 1 -- 172.21.15.36:0/2455374495 >> 172.21.15.36:0/2455374495 conn(0x7f48e40b9870 msgr2=0x7f48e40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:33.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.505+0000 7f48e906c700 1 -- 172.21.15.36:0/2455374495 shutdown_connections 2022-06-17T12:49:33.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.505+0000 7f48e906c700 1 -- 172.21.15.36:0/2455374495 wait complete. 2022-06-17T12:49:33.515 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 19 2022-06-17T12:49:33.516 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b cache drop 2022-06-17T12:49:33.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.649+0000 7f8b6b59e700 1 Processor -- start 2022-06-17T12:49:33.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.649+0000 7f8b6b59e700 1 -- start start 2022-06-17T12:49:33.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.649+0000 7f8b6b59e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8b6c0b0d70 0x7f8b6c0ac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.649+0000 7f8b6b59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b6c0b15f0 0x7f8b6c0acc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.649+0000 7f8b6b59e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b6c0b2b90 con 0x7f8b6c0b27b0 2022-06-17T12:49:33.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.649+0000 7f8b6b59e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b6c0610e0 con 0x7f8b6c0b0d70 2022-06-17T12:49:33.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.649+0000 7f8b6b59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c0ac190 con 0x7f8b6c0b15f0 2022-06-17T12:49:33.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.649+0000 7f8b6a59c700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f8b6c0b27b0 0x7f8b6c13f110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51132/0 (socket says 172.21.15.36:51132) 2022-06-17T12:49:33.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6a59c700 1 -- 172.21.15.36:0/3247472140 learned_addr learned my addr 172.21.15.36:0/3247472140 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:33.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b69d9b700 1 --2- 172.21.15.36:0/3247472140 >> v2:172.21.15.70:3300/0 conn(0x7f8b6c0b0d70 0x7f8b6c0ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6ad9d700 1 --2- 172.21.15.36:0/3247472140 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b6c0b15f0 0x7f8b6c0acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6959a700 1 -- 172.21.15.36:0/3247472140 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 851779133 0 0) 0x7f8b6c0b2b90 con 0x7f8b6c0b27b0 2022-06-17T12:49:33.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6959a700 1 -- 172.21.15.36:0/3247472140 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b4c003260 con 0x7f8b6c0b27b0 2022-06-17T12:49:33.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6959a700 1 -- 172.21.15.36:0/3247472140 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2031218066 0 0) 0x7f8b4c003260 con 0x7f8b6c0b27b0 2022-06-17T12:49:33.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6959a700 1 -- 172.21.15.36:0/3247472140 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b6c0b15f0 msgr2=0x7f8b6c0acc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:33.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6959a700 1 --2- 172.21.15.36:0/3247472140 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b6c0b15f0 0x7f8b6c0acc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6959a700 1 -- 172.21.15.36:0/3247472140 >> v2:172.21.15.70:3300/0 conn(0x7f8b6c0b0d70 msgr2=0x7f8b6c0ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:33.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6959a700 1 --2- 172.21.15.36:0/3247472140 >> v2:172.21.15.70:3300/0 conn(0x7f8b6c0b0d70 0x7f8b6c0ac780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6959a700 1 -- 172.21.15.36:0/3247472140 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b6c0ac430 con 0x7f8b6c0b27b0 2022-06-17T12:49:33.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6959a700 1 -- 172.21.15.36:0/3247472140 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f8b64001660 con 0x7f8b6c0b27b0 2022-06-17T12:49:33.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6959a700 1 -- 172.21.15.36:0/3247472140 >> v1:172.21.15.36:6789/0 conn(0x7f8b6c0b27b0 legacy=0x7f8b6c13f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:33.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6959a700 1 --2- 172.21.15.36:0/3247472140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b4c003750 0x7f8b4c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6959a700 1 --2- 172.21.15.36:0/3247472140 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b4c004500 0x7f8b4c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6959a700 1 --2- 172.21.15.36:0/3247472140 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b6c0b15f0 0x7f8b4c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6959a700 1 -- 172.21.15.36:0/3247472140 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c0610e0 con 0x7f8b4c004500 2022-06-17T12:49:33.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6959a700 1 -- 172.21.15.36:0/3247472140 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c0ac190 con 0x7f8b4c003750 2022-06-17T12:49:33.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6959a700 1 -- 172.21.15.36:0/3247472140 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c00b750 con 0x7f8b6c0b15f0 2022-06-17T12:49:33.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b69d9b700 1 --2- 172.21.15.36:0/3247472140 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b4c004500 0x7f8b4c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6a59c700 1 --2- 172.21.15.36:0/3247472140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b4c003750 0x7f8b4c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6ad9d700 1 --2- 172.21.15.36:0/3247472140 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b6c0b15f0 0x7f8b4c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b69d9b700 1 -- 172.21.15.36:0/3247472140 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b6c0b15f0 msgr2=0x7f8b4c004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:33.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b69d9b700 1 --2- 172.21.15.36:0/3247472140 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b6c0b15f0 0x7f8b4c004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b69d9b700 1 -- 172.21.15.36:0/3247472140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b4c003750 msgr2=0x7f8b4c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:33.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b69d9b700 1 --2- 172.21.15.36:0/3247472140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b4c003750 0x7f8b4c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b69d9b700 1 -- 172.21.15.36:0/3247472140 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4c003260 con 0x7f8b4c004500 2022-06-17T12:49:33.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b69d9b700 1 --2- 172.21.15.36:0/3247472140 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b4c004500 0x7f8b4c004980 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f8b5400c000 tx=0x7f8b54006df0).ready entity=mon.0 client_cookie=c180115777e7df1e server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:33.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6959a700 1 -- 172.21.15.36:0/3247472140 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b5400b880 con 0x7f8b4c004500 2022-06-17T12:49:33.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6959a700 1 -- 172.21.15.36:0/3247472140 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b54009e90 con 0x7f8b4c004500 2022-06-17T12:49:33.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6959a700 1 -- 172.21.15.36:0/3247472140 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b5400bbd0 con 0x7f8b4c004500 2022-06-17T12:49:33.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6b59e700 1 -- 172.21.15.36:0/3247472140 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b4c004500 msgr2=0x7f8b4c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:33.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6b59e700 1 --2- 172.21.15.36:0/3247472140 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b4c004500 0x7f8b4c004980 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f8b5400c000 tx=0x7f8b54006df0).stop 2022-06-17T12:49:33.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6ad9d700 1 -- 172.21.15.36:0/3247472140 reap_dead start 2022-06-17T12:49:33.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6b59e700 1 -- 172.21.15.36:0/3247472140 shutdown_connections 2022-06-17T12:49:33.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6b59e700 1 -- 172.21.15.36:0/3247472140 >> 172.21.15.36:0/3247472140 conn(0x7f8b6c0b5d10 msgr2=0x7f8b6c0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:33.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6b59e700 1 -- 172.21.15.36:0/3247472140 shutdown_connections 2022-06-17T12:49:33.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.653+0000 7f8b6b59e700 1 -- 172.21.15.36:0/3247472140 wait complete. 2022-06-17T12:49:33.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.657+0000 7f8b6b59e700 1 Processor -- start 2022-06-17T12:49:33.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.657+0000 7f8b6b59e700 1 -- start start 2022-06-17T12:49:33.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.657+0000 7f8b6b59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b4c004500 0x7f8b6c066ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.657+0000 7f8b6b59e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b6c144b50 0x7f8b6c141c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.657+0000 7f8b6b59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b6c143ba0 0x7f8b6c144030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.657+0000 7f8b6b59e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c1423b0 con 0x7f8b4c004500 2022-06-17T12:49:33.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.657+0000 7f8b6b59e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c142160 con 0x7f8b6c144b50 2022-06-17T12:49:33.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.657+0000 7f8b6b59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c13e1c0 con 0x7f8b6c143ba0 2022-06-17T12:49:33.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.657+0000 7f8b6a59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b4c004500 0x7f8b6c066ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.657+0000 7f8b6a59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b4c004500 0x7f8b6c066ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33614/0 (socket says 172.21.15.36:33614) 2022-06-17T12:49:33.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.657+0000 7f8b6a59c700 1 -- 172.21.15.36:0/1308411708 learned_addr learned my addr 172.21.15.36:0/1308411708 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:33.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.657+0000 7f8b69d9b700 1 --2- 172.21.15.36:0/1308411708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b6c144b50 0x7f8b6c141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.657+0000 7f8b6ad9d700 1 --2- 172.21.15.36:0/1308411708 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b6c143ba0 0x7f8b6c144030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.657+0000 7f8b6a59c700 1 -- 172.21.15.36:0/1308411708 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b6c143ba0 msgr2=0x7f8b6c144030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:33.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.657+0000 7f8b6a59c700 1 --2- 172.21.15.36:0/1308411708 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b6c143ba0 0x7f8b6c144030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.657+0000 7f8b6a59c700 1 -- 172.21.15.36:0/1308411708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b6c144b50 msgr2=0x7f8b6c141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:33.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.657+0000 7f8b6a59c700 1 --2- 172.21.15.36:0/1308411708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b6c144b50 0x7f8b6c141c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.657+0000 7f8b6a59c700 1 -- 172.21.15.36:0/1308411708 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b6c1425a0 con 0x7f8b4c004500 2022-06-17T12:49:33.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.657+0000 7f8b6a59c700 1 --2- 172.21.15.36:0/1308411708 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b4c004500 0x7f8b6c066ae0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f8b6c1422f0 tx=0x7f8b64001e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:33.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.657+0000 7f8b637fe700 1 -- 172.21.15.36:0/1308411708 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b64011780 con 0x7f8b4c004500 2022-06-17T12:49:33.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.657+0000 7f8b637fe700 1 -- 172.21.15.36:0/1308411708 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b640118e0 con 0x7f8b4c004500 2022-06-17T12:49:33.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.657+0000 7f8b6b59e700 1 -- 172.21.15.36:0/1308411708 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6c13dd90 con 0x7f8b4c004500 2022-06-17T12:49:33.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.657+0000 7f8b637fe700 1 -- 172.21.15.36:0/1308411708 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b64011c40 con 0x7f8b4c004500 2022-06-17T12:49:33.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.657+0000 7f8b6b59e700 1 -- 172.21.15.36:0/1308411708 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6c13ea70 con 0x7f8b4c004500 2022-06-17T12:49:33.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.661+0000 7f8b637fe700 1 -- 172.21.15.36:0/1308411708 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8b6401b030 con 0x7f8b4c004500 2022-06-17T12:49:33.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.661+0000 7f8b637fe700 1 --2- 172.21.15.36:0/1308411708 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b500539b0 0x7f8b50055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.661+0000 7f8b637fe700 1 -- 172.21.15.36:0/1308411708 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8b6406bb30 con 0x7f8b4c004500 2022-06-17T12:49:33.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.661+0000 7f8b69d9b700 1 --2- 172.21.15.36:0/1308411708 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b500539b0 0x7f8b50055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.661+0000 7f8b69d9b700 1 --2- 172.21.15.36:0/1308411708 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b500539b0 0x7f8b50055e70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f8b5401b000 tx=0x7f8b54006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:33.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.661+0000 7f8b7ab8f740 1 Processor -- start 2022-06-17T12:49:33.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.661+0000 7f8b7ab8f740 1 -- start start 2022-06-17T12:49:33.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b7ab8f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x119b6a0 0x115fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b7ab8f740 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x11609b0 0x1160e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b7ab8f740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x115c330 0x115c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b7ab8f740 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x10d1c60 con 0x11609b0 2022-06-17T12:49:33.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b7ab8f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x10b1ec0 con 0x119b6a0 2022-06-17T12:49:33.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b7ab8f740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x1196bd0 con 0x115c330 2022-06-17T12:49:33.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b69d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x11609b0 0x1160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b69d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x11609b0 0x1160e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33626/0 (socket says 172.21.15.36:33626) 2022-06-17T12:49:33.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b69d9b700 1 -- 172.21.15.36:0/168236571 learned_addr learned my addr 172.21.15.36:0/168236571 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:33.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b6a59c700 1 --2- 172.21.15.36:0/168236571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x119b6a0 0x115fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b6ad9d700 1 --2- 172.21.15.36:0/168236571 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x115c330 0x115c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b69d9b700 1 -- 172.21.15.36:0/168236571 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x115c330 msgr2=0x115c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:33.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b69d9b700 1 --2- 172.21.15.36:0/168236571 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x115c330 0x115c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b69d9b700 1 -- 172.21.15.36:0/168236571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x119b6a0 msgr2=0x115fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:33.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b69d9b700 1 --2- 172.21.15.36:0/168236571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x119b6a0 0x115fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b69d9b700 1 -- 172.21.15.36:0/168236571 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1109520 con 0x11609b0 2022-06-17T12:49:33.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b69d9b700 1 --2- 172.21.15.36:0/168236571 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x11609b0 0x1160e10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f8b54021000 tx=0x7f8b54022070).ready entity=mon.0 client_cookie=12761df5c779dfb3 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:33.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b6b59e700 1 -- 172.21.15.36:0/168236571 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b54015470 con 0x11609b0 2022-06-17T12:49:33.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b6b59e700 1 -- 172.21.15.36:0/168236571 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b54000e90 con 0x11609b0 2022-06-17T12:49:33.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b6b59e700 1 -- 172.21.15.36:0/168236571 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b540157c0 con 0x11609b0 2022-06-17T12:49:33.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/168236571 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x11609b0 msgr2=0x1160e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:33.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b7ab8f740 1 --2- 172.21.15.36:0/168236571 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x11609b0 0x1160e10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f8b54021000 tx=0x7f8b54022070).stop 2022-06-17T12:49:33.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/168236571 shutdown_connections 2022-06-17T12:49:33.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b7ab8f740 1 --2- 172.21.15.36:0/168236571 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x115c330 0x115c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b7ab8f740 1 --2- 172.21.15.36:0/168236571 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x11609b0 0x1160e10 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b7ab8f740 1 --2- 172.21.15.36:0/168236571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x119b6a0 0x115fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/168236571 >> 172.21.15.36:0/168236571 conn(0x10e9340 msgr2=0x119ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:33.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/168236571 shutdown_connections 2022-06-17T12:49:33.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/168236571 wait complete. 2022-06-17T12:49:33.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b7ab8f740 1 -- start start 2022-06-17T12:49:33.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.665+0000 7f8b7ab8f740 1 Processor -- start 2022-06-17T12:49:33.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b7ab8f740 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x115c330 0x1192b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b7ab8f740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x11609b0 0x11a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b7ab8f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x119b6a0 0x1190e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b7ab8f740 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x118df60 con 0x115c330 2022-06-17T12:49:33.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b7ab8f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x11a5bd0 con 0x119b6a0 2022-06-17T12:49:33.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b7ab8f740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x11a5e60 con 0x11609b0 2022-06-17T12:49:33.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b7ab8f740 10 client.0 ll_register_callbacks cb 0x11187d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-06-17T12:49:33.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b6a59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x115c330 0x1192b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b6a59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x115c330 0x1192b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33628/0 (socket says 172.21.15.36:33628) 2022-06-17T12:49:33.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b6a59c700 1 -- 172.21.15.36:0/1459939403 learned_addr learned my addr 172.21.15.36:0/1459939403 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:33.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b6ad9d700 1 --2- 172.21.15.36:0/1459939403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x119b6a0 0x1190e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b69d9b700 1 --2- 172.21.15.36:0/1459939403 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x11609b0 0x11a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b6a59c700 1 -- 172.21.15.36:0/1459939403 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x11609b0 msgr2=0x11a56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:33.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b6a59c700 1 --2- 172.21.15.36:0/1459939403 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x11609b0 0x11a56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b6a59c700 1 -- 172.21.15.36:0/1459939403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x119b6a0 msgr2=0x1190e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:33.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b6a59c700 1 --2- 172.21.15.36:0/1459939403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x119b6a0 0x1190e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b6a59c700 1 -- 172.21.15.36:0/1459939403 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4c003260 con 0x115c330 2022-06-17T12:49:33.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b69d9b700 1 --2- 172.21.15.36:0/1459939403 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x11609b0 0x11a56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:49:33.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b6a59c700 1 --2- 172.21.15.36:0/1459939403 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x115c330 0x1192b10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f8b64024000 tx=0x7f8b64001640).ready entity=mon.0 client_cookie=7cffab4d7691f55d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:33.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b497fa700 10 client.0 ms_handle_connect on v2:172.21.15.36:3300/0 2022-06-17T12:49:33.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b497fa700 1 -- 172.21.15.36:0/1459939403 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b640423e0 con 0x115c330 2022-06-17T12:49:33.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b497fa700 1 -- 172.21.15.36:0/1459939403 <== mon.0 v2:172.21.15.36:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8b64042540 con 0x115c330 2022-06-17T12:49:33.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/1459939403 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x10c7b00 con 0x115c330 2022-06-17T12:49:33.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b497fa700 1 -- 172.21.15.36:0/1459939403 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b64042700 con 0x115c330 2022-06-17T12:49:33.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b7ab8f740 10 client.5508 fetch_fsmap learned FSMap version 19 2022-06-17T12:49:33.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/1459939403 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0x10c10f0 con 0x115c330 2022-06-17T12:49:33.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b497fa700 1 -- 172.21.15.36:0/1459939403 <== mon.0 v2:172.21.15.36:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x10c7b00 con 0x115c330 2022-06-17T12:49:33.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b497fa700 1 -- 172.21.15.36:0/1459939403 <== mon.0 v2:172.21.15.36:3300/0 5 ==== fsmap(e 19) v1 ==== 1247+0+0 (secure 0 0 0) 0x7f8b64044c50 con 0x115c330 2022-06-17T12:49:33.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b7ab8f740 10 client.5508 fetch_fsmap finished waiting for FSMap version 19 2022-06-17T12:49:33.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b7ab8f740 10 client.5508 resolve_mds: resolved ID 'b' to GID 5292 2022-06-17T12:49:33.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b7ab8f740 20 client.5508 populate_metadata read hostname 'smithi036' 2022-06-17T12:49:33.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b7ab8f740 1 --2- 172.21.15.36:0/1459939403 >> [v2:172.21.15.70:6832/4117532324,v1:172.21.15.70:6833/4117532324] conn(0x11a8f00 0x11a92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b7ab8f740 4 client.5508 mds_command: new command op to 5292 tid=0[{"prefix": "get_command_descriptions"}] 2022-06-17T12:49:33.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/1459939403 --> [v2:172.21.15.70:6832/4117532324,v1:172.21.15.70:6833/4117532324] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x10bfb80 con 0x11a8f00 2022-06-17T12:49:33.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.669+0000 7f8b69d9b700 1 --2- 172.21.15.36:0/1459939403 >> [v2:172.21.15.70:6832/4117532324,v1:172.21.15.70:6833/4117532324] conn(0x11a8f00 0x11a92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.673+0000 7f8b69d9b700 1 --2- 172.21.15.36:0/1459939403 >> [v2:172.21.15.70:6832/4117532324,v1:172.21.15.70:6833/4117532324] conn(0x11a8f00 0x11a92e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=ab699833156ad4c2 server_cookie=46878d14406152b9 in_seq=0 out_seq=0 2022-06-17T12:49:33.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.673+0000 7f8b497fa700 10 client.5508 ms_handle_connect on v2:172.21.15.70:6832/4117532324 2022-06-17T12:49:33.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.673+0000 7f8b497fa700 1 -- 172.21.15.36:0/1459939403 <== mds.0 v2:172.21.15.70:6832/4117532324 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x10bfb80 con 0x11a8f00 2022-06-17T12:49:33.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.673+0000 7f8b497fa700 10 client.5508 handle_command_reply: tid=0 2022-06-17T12:49:33.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.673+0000 7f8b7ab8f740 1 client.5508 shutdown 2022-06-17T12:49:33.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.673+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/1459939403 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x115c330 msgr2=0x1192b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:33.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.673+0000 7f8b7ab8f740 1 --2- 172.21.15.36:0/1459939403 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x115c330 0x1192b10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f8b64024000 tx=0x7f8b64001640).stop 2022-06-17T12:49:33.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.673+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/1459939403 shutdown_connections 2022-06-17T12:49:33.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.673+0000 7f8b7ab8f740 1 --2- 172.21.15.36:0/1459939403 >> [v2:172.21.15.70:6832/4117532324,v1:172.21.15.70:6833/4117532324] conn(0x11a8f00 0x11a92e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.673+0000 7f8b7ab8f740 1 --2- 172.21.15.36:0/1459939403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x119b6a0 0x1190e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.673+0000 7f8b7ab8f740 1 --2- 172.21.15.36:0/1459939403 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x11609b0 0x11a56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.673+0000 7f8b7ab8f740 1 --2- 172.21.15.36:0/1459939403 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x115c330 0x1192b10 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.673+0000 7f8b497fa700 0 client.5508 ms_handle_reset on v2:172.21.15.70:6832/4117532324 2022-06-17T12:49:33.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.673+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/1459939403 >> 172.21.15.36:0/1459939403 conn(0x10e9340 msgr2=0x115c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:33.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.673+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/1459939403 shutdown_connections 2022-06-17T12:49:33.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.673+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/1459939403 wait complete. 2022-06-17T12:49:33.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.673+0000 7f8b7ab8f740 20 client.5508 trim_cache size 0 max 16384 2022-06-17T12:49:33.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.689+0000 7f8b7ab8f740 1 Processor -- start 2022-06-17T12:49:33.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.689+0000 7f8b7ab8f740 1 -- start start 2022-06-17T12:49:33.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.689+0000 7f8b7ab8f740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x115c330 0x11a5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.689+0000 7f8b7ab8f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x11609b0 0x115e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.689+0000 7f8b7ab8f740 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x119b6a0 0x118cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.689+0000 7f8b7ab8f740 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x10e3960 con 0x119b6a0 2022-06-17T12:49:33.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.689+0000 7f8b7ab8f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x10c7b00 con 0x11609b0 2022-06-17T12:49:33.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.689+0000 7f8b7ab8f740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x1131580 con 0x115c330 2022-06-17T12:49:33.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.689+0000 7f8b6ad9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x119b6a0 0x118cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.689+0000 7f8b6ad9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x119b6a0 0x118cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33638/0 (socket says 172.21.15.36:33638) 2022-06-17T12:49:33.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.689+0000 7f8b6ad9d700 1 -- 172.21.15.36:0/1026129568 learned_addr learned my addr 172.21.15.36:0/1026129568 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:33.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.689+0000 7f8b69d9b700 1 --2- 172.21.15.36:0/1026129568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x11609b0 0x115e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.689+0000 7f8b6a59c700 1 --2- 172.21.15.36:0/1026129568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x115c330 0x11a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.689+0000 7f8b6ad9d700 1 -- 172.21.15.36:0/1026129568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x115c330 msgr2=0x11a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:33.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.689+0000 7f8b6ad9d700 1 --2- 172.21.15.36:0/1026129568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x115c330 0x11a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.689+0000 7f8b6ad9d700 1 -- 172.21.15.36:0/1026129568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x11609b0 msgr2=0x115e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:33.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.689+0000 7f8b6ad9d700 1 --2- 172.21.15.36:0/1026129568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x11609b0 0x115e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.689+0000 7f8b6ad9d700 1 -- 172.21.15.36:0/1026129568 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x10c66d0 con 0x119b6a0 2022-06-17T12:49:33.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.689+0000 7f8b6ad9d700 1 --2- 172.21.15.36:0/1026129568 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x119b6a0 0x118cce0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x10bc630 tx=0x7f8b5c017040).ready entity=mon.0 client_cookie=fba59015769a7afa server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:33.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.693+0000 7f8b4b7fe700 1 -- 172.21.15.36:0/1026129568 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b5c01a070 con 0x119b6a0 2022-06-17T12:49:33.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.693+0000 7f8b4b7fe700 1 -- 172.21.15.36:0/1026129568 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5c01b040 con 0x119b6a0 2022-06-17T12:49:33.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.693+0000 7f8b4b7fe700 1 -- 172.21.15.36:0/1026129568 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b5c015c60 con 0x119b6a0 2022-06-17T12:49:33.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.693+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/1026129568 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x119b6a0 msgr2=0x118cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:33.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.693+0000 7f8b7ab8f740 1 --2- 172.21.15.36:0/1026129568 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x119b6a0 0x118cce0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x10bc630 tx=0x7f8b5c017040).stop 2022-06-17T12:49:33.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.693+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/1026129568 shutdown_connections 2022-06-17T12:49:33.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.693+0000 7f8b7ab8f740 1 --2- 172.21.15.36:0/1026129568 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x119b6a0 0x118cce0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.693+0000 7f8b7ab8f740 1 --2- 172.21.15.36:0/1026129568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x11609b0 0x115e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.693+0000 7f8b7ab8f740 1 --2- 172.21.15.36:0/1026129568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x115c330 0x11a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.693+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/1026129568 >> 172.21.15.36:0/1026129568 conn(0x10e9340 msgr2=0x115fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:33.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.693+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/1026129568 shutdown_connections 2022-06-17T12:49:33.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.693+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/1026129568 wait complete. 2022-06-17T12:49:33.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.693+0000 7f8b7ab8f740 1 -- start start 2022-06-17T12:49:33.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.693+0000 7f8b7ab8f740 1 Processor -- start 2022-06-17T12:49:33.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.693+0000 7f8b7ab8f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x115c330 0x11fa290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.693+0000 7f8b7ab8f740 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x11609b0 0x11f9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.693+0000 7f8b7ab8f740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x119b6a0 0x11f9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.693+0000 7f8b7ab8f740 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x11f9e30 con 0x11609b0 2022-06-17T12:49:33.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.693+0000 7f8b7ab8f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x11f8d80 con 0x115c330 2022-06-17T12:49:33.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.693+0000 7f8b7ab8f740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x115d5e0 con 0x119b6a0 2022-06-17T12:49:33.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.693+0000 7f8b7ab8f740 10 client.0 ll_register_callbacks cb 0x11928c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-06-17T12:49:33.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b69d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x11609b0 0x11f9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b69d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x11609b0 0x11f9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33646/0 (socket says 172.21.15.36:33646) 2022-06-17T12:49:33.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b69d9b700 1 -- 172.21.15.36:0/2858901204 learned_addr learned my addr 172.21.15.36:0/2858901204 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:33.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b6a59c700 1 --2- 172.21.15.36:0/2858901204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x115c330 0x11fa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b6ad9d700 1 --2- 172.21.15.36:0/2858901204 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x119b6a0 0x11f9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b69d9b700 1 -- 172.21.15.36:0/2858901204 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x119b6a0 msgr2=0x11f9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:33.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b69d9b700 1 --2- 172.21.15.36:0/2858901204 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x119b6a0 0x11f9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b69d9b700 1 -- 172.21.15.36:0/2858901204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x115c330 msgr2=0x11fa290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:33.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b69d9b700 1 --2- 172.21.15.36:0/2858901204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x115c330 0x11fa290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:33.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b69d9b700 1 -- 172.21.15.36:0/2858901204 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5c009a10 con 0x11609b0 2022-06-17T12:49:33.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b6ad9d700 1 --2- 172.21.15.36:0/2858901204 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x119b6a0 0x11f9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:49:33.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b4a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.36:3300/0 2022-06-17T12:49:33.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b69d9b700 1 --2- 172.21.15.36:0/2858901204 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x11609b0 0x11f9450 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f8b54033000 tx=0x7f8b54022040).ready entity=mon.0 client_cookie=90f0082eb9948927 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:33.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b4a7fc700 1 -- 172.21.15.36:0/2858901204 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b54038040 con 0x11609b0 2022-06-17T12:49:33.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b4a7fc700 1 -- 172.21.15.36:0/2858901204 <== mon.0 v2:172.21.15.36:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8b5403ede0 con 0x11609b0 2022-06-17T12:49:33.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/2858901204 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b54000e90 con 0x11609b0 2022-06-17T12:49:33.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b4a7fc700 1 -- 172.21.15.36:0/2858901204 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b5403c280 con 0x11609b0 2022-06-17T12:49:33.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b7ab8f740 10 client.5514 fetch_fsmap learned FSMap version 19 2022-06-17T12:49:33.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/2858901204 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0x109b480 con 0x11609b0 2022-06-17T12:49:33.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b4a7fc700 1 -- 172.21.15.36:0/2858901204 <== mon.0 v2:172.21.15.36:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b54000e90 con 0x11609b0 2022-06-17T12:49:33.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b4a7fc700 1 -- 172.21.15.36:0/2858901204 <== mon.0 v2:172.21.15.36:3300/0 5 ==== fsmap(e 19) v1 ==== 1247+0+0 (secure 0 0 0) 0x7f8b54013070 con 0x11609b0 2022-06-17T12:49:33.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b7ab8f740 10 client.5514 fetch_fsmap finished waiting for FSMap version 19 2022-06-17T12:49:33.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b7ab8f740 10 client.5514 resolve_mds: resolved ID 'b' to GID 5292 2022-06-17T12:49:33.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b7ab8f740 20 client.5514 populate_metadata read hostname 'smithi036' 2022-06-17T12:49:33.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b7ab8f740 1 --2- 172.21.15.36:0/2858901204 >> [v2:172.21.15.70:6832/4117532324,v1:172.21.15.70:6833/4117532324] conn(0x11f28b0 0x11f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:33.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b7ab8f740 4 client.5514 mds_command: new command op to 5292 tid=0[{"prefix": "cache drop"}] 2022-06-17T12:49:33.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/2858901204 --> [v2:172.21.15.70:6832/4117532324,v1:172.21.15.70:6833/4117532324] -- command(tid 0: {"prefix": "cache drop"}) v1 -- 0xfb5590 con 0x11f28b0 2022-06-17T12:49:33.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b6a59c700 1 --2- 172.21.15.36:0/2858901204 >> [v2:172.21.15.70:6832/4117532324,v1:172.21.15.70:6833/4117532324] conn(0x11f28b0 0x11f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:33.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b4a7fc700 10 client.5514 ms_handle_connect on v2:172.21.15.70:6832/4117532324 2022-06-17T12:49:33.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:33.697+0000 7f8b6a59c700 1 --2- 172.21.15.36:0/2858901204 >> [v2:172.21.15.70:6832/4117532324,v1:172.21.15.70:6833/4117532324] conn(0x11f28b0 0x11f2c90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=3ede959a92e65398 server_cookie=3579c71fc619a967 in_seq=0 out_seq=0 2022-06-17T12:49:44.620 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:49:44.625 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:49:44.629 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:49:44.651 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:49:44.682 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:49:44.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b4a7fc700 1 -- 172.21.15.36:0/2858901204 <== mds.0 v2:172.21.15.70:6832/4117532324 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+380 (crc 0 0 0) 0xfb5590 con 0x11f28b0 2022-06-17T12:49:44.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b4a7fc700 10 client.5514 handle_command_reply: tid=0 2022-06-17T12:49:44.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b7ab8f740 1 client.5514 shutdown 2022-06-17T12:49:44.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/2858901204 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x11609b0 msgr2=0x11f9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:44.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b7ab8f740 1 --2- 172.21.15.36:0/2858901204 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x11609b0 0x11f9450 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f8b54033000 tx=0x7f8b54022040).stop 2022-06-17T12:49:44.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/2858901204 shutdown_connections 2022-06-17T12:49:44.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b7ab8f740 1 --2- 172.21.15.36:0/2858901204 >> [v2:172.21.15.70:6832/4117532324,v1:172.21.15.70:6833/4117532324] conn(0x11f28b0 0x11f2c90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:44.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b7ab8f740 1 --2- 172.21.15.36:0/2858901204 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x119b6a0 0x11f9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:44.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b7ab8f740 1 --2- 172.21.15.36:0/2858901204 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x11609b0 0x11f9450 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:44.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b7ab8f740 1 --2- 172.21.15.36:0/2858901204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x115c330 0x11fa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:44.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/2858901204 >> 172.21.15.36:0/2858901204 conn(0x10e9340 msgr2=0x1190bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:44.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b4a7fc700 0 client.5514 ms_handle_reset on v2:172.21.15.70:6832/4117532324 2022-06-17T12:49:44.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/2858901204 shutdown_connections 2022-06-17T12:49:44.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b7ab8f740 1 -- 172.21.15.36:0/2858901204 wait complete. 2022-06-17T12:49:44.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b7ab8f740 20 client.5514 trim_cache size 0 max 16384 2022-06-17T12:49:44.875 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:49:44.876 INFO:teuthology.orchestra.run.smithi036.stdout: "client_recall": { 2022-06-17T12:49:44.876 INFO:teuthology.orchestra.run.smithi036.stdout: "return_code": 0, 2022-06-17T12:49:44.878 INFO:teuthology.orchestra.run.smithi036.stdout: "message": "(0) Success", 2022-06-17T12:49:44.878 INFO:teuthology.orchestra.run.smithi036.stdout: "recalled": 220 2022-06-17T12:49:44.879 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:44.879 INFO:teuthology.orchestra.run.smithi036.stdout: "flush_journal": { 2022-06-17T12:49:44.880 INFO:teuthology.orchestra.run.smithi036.stdout: "return_code": 0, 2022-06-17T12:49:44.891 INFO:teuthology.orchestra.run.smithi036.stdout: "message": "" 2022-06-17T12:49:44.918 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:44.918 INFO:teuthology.orchestra.run.smithi036.stdout: "trim_cache": { 2022-06-17T12:49:44.918 INFO:teuthology.orchestra.run.smithi036.stdout: "trimmed": 0 2022-06-17T12:49:44.918 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:44.919 INFO:teuthology.orchestra.run.smithi036.stdout: "cache": { 2022-06-17T12:49:44.920 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": { 2022-06-17T12:49:44.920 INFO:teuthology.orchestra.run.smithi036.stdout: "items": 20025, 2022-06-17T12:49:44.920 INFO:teuthology.orchestra.run.smithi036.stdout: "bytes": 3101920 2022-06-17T12:49:44.921 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:49:44.921 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:49:44.921 INFO:teuthology.orchestra.run.smithi036.stdout: "duration": 11.170044286 2022-06-17T12:49:44.921 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:49:44.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b6b59e700 1 -- 172.21.15.36:0/1308411708 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b500539b0 msgr2=0x7f8b50055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:44.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b6b59e700 1 --2- 172.21.15.36:0/1308411708 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b500539b0 0x7f8b50055e70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f8b5401b000 tx=0x7f8b54006df0).stop 2022-06-17T12:49:44.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b6b59e700 1 -- 172.21.15.36:0/1308411708 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b4c004500 msgr2=0x7f8b6c066ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:44.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b6b59e700 1 --2- 172.21.15.36:0/1308411708 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b4c004500 0x7f8b6c066ae0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f8b6c1422f0 tx=0x7f8b64001e50).stop 2022-06-17T12:49:44.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b6b59e700 1 -- 172.21.15.36:0/1308411708 shutdown_connections 2022-06-17T12:49:44.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b6b59e700 1 --2- 172.21.15.36:0/1308411708 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b500539b0 0x7f8b50055e70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:44.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b6b59e700 1 --2- 172.21.15.36:0/1308411708 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b6c143ba0 0x7f8b6c144030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:44.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b6b59e700 1 --2- 172.21.15.36:0/1308411708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b6c144b50 0x7f8b6c141c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:44.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b6b59e700 1 --2- 172.21.15.36:0/1308411708 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b4c004500 0x7f8b6c066ae0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:44.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b6b59e700 1 -- 172.21.15.36:0/1308411708 >> 172.21.15.36:0/1308411708 conn(0x7f8b6c0b5d10 msgr2=0x7f8b6c0ac780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:44.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b6b59e700 1 -- 172.21.15.36:0/1308411708 shutdown_connections 2022-06-17T12:49:44.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:44.869+0000 7f8b6b59e700 1 -- 172.21.15.36:0/1308411708 wait complete. 2022-06-17T12:49:44.940 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2022-06-17T12:49:44.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:49:44.945 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:49:44.946 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:49:44.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:49:44.978 INFO:teuthology.orchestra.run.smithi107.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2022-06-17T12:49:44.979 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2022-06-17T12:49:44.980 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2022-06-17T12:49:44.981 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:49:44.982 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:49:45.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:49:45.027 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:49:45.028 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:49:45.072 INFO:teuthology.orchestra.run.smithi107.stdout:ext2/ext3 2022-06-17T12:49:45.074 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:49:45.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:49:45.078 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:49:45.086 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:49:45.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.217+0000 7fbda37a8700 1 Processor -- start 2022-06-17T12:49:45.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.217+0000 7fbda37a8700 1 -- start start 2022-06-17T12:49:45.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbda37a8700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbd9c062bd0 0x7fbd9c0b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:45.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbda37a8700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbd9c140e30 0x7fbd9c141280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:45.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbda37a8700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd9c065090 con 0x7fbd9c0b12f0 2022-06-17T12:49:45.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbda37a8700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd9c063510 con 0x7fbd9c062bd0 2022-06-17T12:49:45.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbda37a8700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd9c0b0b40 con 0x7fbd9c140e30 2022-06-17T12:49:45.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbda0d43700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbd9c062bd0 0x7fbd9c0b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:45.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbda0d43700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbd9c062bd0 0x7fbd9c0b16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33650/0 (socket says 172.21.15.36:33650) 2022-06-17T12:49:45.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbda0d43700 1 -- 172.21.15.36:0/2716757276 learned_addr learned my addr 172.21.15.36:0/2716757276 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:45.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbda1d45700 1 --2- 172.21.15.36:0/2716757276 >> v2:172.21.15.70:3300/0 conn(0x7fbd9c140e30 0x7fbd9c141280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:45.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbda0d43700 1 -- 172.21.15.36:0/2716757276 >> v2:172.21.15.70:3300/0 conn(0x7fbd9c140e30 msgr2=0x7fbd9c141280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:45.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbda0d43700 1 --2- 172.21.15.36:0/2716757276 >> v2:172.21.15.70:3300/0 conn(0x7fbd9c140e30 0x7fbd9c141280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:45.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbda0d43700 1 -- 172.21.15.36:0/2716757276 >> v1:172.21.15.70:6789/0 conn(0x7fbd9c0b12f0 legacy=0x7fbd9c13aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:45.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbda0d43700 1 -- 172.21.15.36:0/2716757276 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c0b1bc0 con 0x7fbd9c062bd0 2022-06-17T12:49:45.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbda0d43700 1 --2- 172.21.15.36:0/2716757276 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbd9c062bd0 0x7fbd9c0b16d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7fbd9c063060 tx=0x7fbd8c006eb0).ready entity=mon.0 client_cookie=e3f5b86ea26b9768 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:45.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbd93fff700 1 -- 172.21.15.36:0/2716757276 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd8c00d610 con 0x7fbd9c062bd0 2022-06-17T12:49:45.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbd93fff700 1 -- 172.21.15.36:0/2716757276 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8c000e90 con 0x7fbd9c062bd0 2022-06-17T12:49:45.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbd93fff700 1 -- 172.21.15.36:0/2716757276 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd8c00d9a0 con 0x7fbd9c062bd0 2022-06-17T12:49:45.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbda37a8700 1 -- 172.21.15.36:0/2716757276 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbd9c062bd0 msgr2=0x7fbd9c0b16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:45.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbda37a8700 1 --2- 172.21.15.36:0/2716757276 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbd9c062bd0 0x7fbd9c0b16d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7fbd9c063060 tx=0x7fbd8c006eb0).stop 2022-06-17T12:49:45.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbda37a8700 1 -- 172.21.15.36:0/2716757276 shutdown_connections 2022-06-17T12:49:45.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbda37a8700 1 --2- 172.21.15.36:0/2716757276 >> v2:172.21.15.70:3300/0 conn(0x7fbd9c140e30 0x7fbd9c141280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:45.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbda37a8700 1 --2- 172.21.15.36:0/2716757276 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbd9c062bd0 0x7fbd9c0b16d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:45.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbda37a8700 1 -- 172.21.15.36:0/2716757276 >> 172.21.15.36:0/2716757276 conn(0x7fbd9c0b53e0 msgr2=0x7fbd9c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:45.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbda37a8700 1 -- 172.21.15.36:0/2716757276 shutdown_connections 2022-06-17T12:49:45.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.221+0000 7fbda37a8700 1 -- 172.21.15.36:0/2716757276 wait complete. 2022-06-17T12:49:45.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.225+0000 7fbda37a8700 1 Processor -- start 2022-06-17T12:49:45.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.225+0000 7fbda37a8700 1 -- start start 2022-06-17T12:49:45.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.225+0000 7fbda37a8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd9c062bd0 0x7fbd9c0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:45.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.225+0000 7fbda37a8700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd9c0b12f0 0x7fbd9c12db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:45.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.225+0000 7fbda37a8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbd9c140e30 0x7fbd9c12cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:45.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.229+0000 7fbda37a8700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c0ade30 con 0x7fbd9c140e30 2022-06-17T12:49:45.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.229+0000 7fbda37a8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c0aecd0 con 0x7fbd9c062bd0 2022-06-17T12:49:45.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.229+0000 7fbda37a8700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c0aef70 con 0x7fbd9c0b12f0 2022-06-17T12:49:45.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.229+0000 7fbda1544700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd9c062bd0 0x7fbd9c0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:45.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.229+0000 7fbda1d45700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbd9c140e30 0x7fbd9c12cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:45.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.229+0000 7fbda1544700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd9c062bd0 0x7fbd9c0b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:54038/0 (socket says 172.21.15.36:54038) 2022-06-17T12:49:45.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.229+0000 7fbda0d43700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd9c0b12f0 0x7fbd9c12db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:45.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.229+0000 7fbda1544700 1 -- 172.21.15.36:0/2891522678 learned_addr learned my addr 172.21.15.36:0/2891522678 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:45.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.229+0000 7fbda1d45700 1 -- 172.21.15.36:0/2891522678 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd9c0b12f0 msgr2=0x7fbd9c12db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:45.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.229+0000 7fbda1d45700 1 --2- 172.21.15.36:0/2891522678 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd9c0b12f0 0x7fbd9c12db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:45.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.229+0000 7fbda1d45700 1 -- 172.21.15.36:0/2891522678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd9c062bd0 msgr2=0x7fbd9c0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:45.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.229+0000 7fbda1d45700 1 --2- 172.21.15.36:0/2891522678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd9c062bd0 0x7fbd9c0b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:45.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.229+0000 7fbda1d45700 1 -- 172.21.15.36:0/2891522678 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c00b1e0 con 0x7fbd9c140e30 2022-06-17T12:49:45.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.229+0000 7fbda1544700 1 --2- 172.21.15.36:0/2891522678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd9c062bd0 0x7fbd9c0b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:49:45.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.229+0000 7fbda1d45700 1 --2- 172.21.15.36:0/2891522678 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbd9c140e30 0x7fbd9c12cf10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fbd9400c000 tx=0x7fbd940091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:45.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.229+0000 7fbd927fc700 1 -- 172.21.15.36:0/2891522678 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd94009520 con 0x7fbd9c140e30 2022-06-17T12:49:45.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.229+0000 7fbd927fc700 1 -- 172.21.15.36:0/2891522678 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd94012600 con 0x7fbd9c140e30 2022-06-17T12:49:45.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.229+0000 7fbda37a8700 1 -- 172.21.15.36:0/2891522678 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd9c0af210 con 0x7fbd9c140e30 2022-06-17T12:49:45.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.229+0000 7fbd927fc700 1 -- 172.21.15.36:0/2891522678 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd94012960 con 0x7fbd9c140e30 2022-06-17T12:49:45.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.229+0000 7fbda37a8700 1 -- 172.21.15.36:0/2891522678 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd9c0af6b0 con 0x7fbd9c140e30 2022-06-17T12:49:45.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.229+0000 7fbda37a8700 1 -- 172.21.15.36:0/2891522678 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9c0af3a0 con 0x7fbd9c140e30 2022-06-17T12:49:45.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.229+0000 7fbd927fc700 1 -- 172.21.15.36:0/2891522678 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fbd94022070 con 0x7fbd9c140e30 2022-06-17T12:49:45.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.229+0000 7fbd927fc700 1 --2- 172.21.15.36:0/2891522678 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbd84053ad0 0x7fbd84055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:45.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.233+0000 7fbd927fc700 1 -- 172.21.15.36:0/2891522678 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbd9406c4b0 con 0x7fbd9c140e30 2022-06-17T12:49:45.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.233+0000 7fbda1544700 1 --2- 172.21.15.36:0/2891522678 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbd84053ad0 0x7fbd84055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:45.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.233+0000 7fbda1544700 1 --2- 172.21.15.36:0/2891522678 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbd84053ad0 0x7fbd84055f90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fbd9800d000 tx=0x7fbd98009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:45.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.237+0000 7fbd927fc700 1 -- 172.21.15.36:0/2891522678 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbd94043260 con 0x7fbd9c140e30 2022-06-17T12:49:45.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.365+0000 7fbda37a8700 1 -- 172.21.15.36:0/2891522678 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd9c0b03c0 con 0x7fbd9c140e30 2022-06-17T12:49:45.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.369+0000 7fbd927fc700 1 -- 172.21.15.36:0/2891522678 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16271 (secure 0 0 0) 0x7fbd94016020 con 0x7fbd9c140e30 2022-06-17T12:49:45.356 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:49:45.356 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":45,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:49:15.026771+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:49:12.581297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-06-17T12:49:12.985806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:49:45.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.369+0000 7fbda37a8700 1 -- 172.21.15.36:0/2891522678 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbd84053ad0 msgr2=0x7fbd84055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:45.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.369+0000 7fbda37a8700 1 --2- 172.21.15.36:0/2891522678 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbd84053ad0 0x7fbd84055f90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fbd9800d000 tx=0x7fbd98009040).stop 2022-06-17T12:49:45.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.369+0000 7fbda37a8700 1 -- 172.21.15.36:0/2891522678 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbd9c140e30 msgr2=0x7fbd9c12cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:45.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.369+0000 7fbda37a8700 1 --2- 172.21.15.36:0/2891522678 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbd9c140e30 0x7fbd9c12cf10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fbd9400c000 tx=0x7fbd940091d0).stop 2022-06-17T12:49:45.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.369+0000 7fbda37a8700 1 -- 172.21.15.36:0/2891522678 shutdown_connections 2022-06-17T12:49:45.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.369+0000 7fbda37a8700 1 --2- 172.21.15.36:0/2891522678 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbd84053ad0 0x7fbd84055f90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:45.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.369+0000 7fbda37a8700 1 --2- 172.21.15.36:0/2891522678 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbd9c140e30 0x7fbd9c12cf10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:45.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.369+0000 7fbda37a8700 1 --2- 172.21.15.36:0/2891522678 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd9c0b12f0 0x7fbd9c12db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:45.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.369+0000 7fbda37a8700 1 --2- 172.21.15.36:0/2891522678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd9c062bd0 0x7fbd9c0b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:45.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.369+0000 7fbda37a8700 1 -- 172.21.15.36:0/2891522678 >> 172.21.15.36:0/2891522678 conn(0x7fbd9c0b53e0 msgr2=0x7fbd9c13add0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:45.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.369+0000 7fbda37a8700 1 -- 172.21.15.36:0/2891522678 shutdown_connections 2022-06-17T12:49:45.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.369+0000 7fbda37a8700 1 -- 172.21.15.36:0/2891522678 wait complete. 2022-06-17T12:49:45.394 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:49:12.581297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:49:12.985806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:49:45.396 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:49:45.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.533+0000 7f66493b1700 1 Processor -- start 2022-06-17T12:49:45.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.533+0000 7f66493b1700 1 -- start start 2022-06-17T12:49:45.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.533+0000 7f66493b1700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f66440b2770 0x7f664414b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:45.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.533+0000 7f66493b1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6644147820 0x7f6644147cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:45.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.533+0000 7f66493b1700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66440605a0 con 0x7f66440601c0 2022-06-17T12:49:45.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.533+0000 7f66493b1700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6644148900 con 0x7f66440b2770 2022-06-17T12:49:45.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.533+0000 7f66493b1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66440b1ec0 con 0x7f6644147820 2022-06-17T12:49:45.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.533+0000 7f6643fff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f66440601c0 0x7f664414ab90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51194/0 (socket says 172.21.15.36:51194) 2022-06-17T12:49:45.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.533+0000 7f6643fff700 1 -- 172.21.15.36:0/4033416003 learned_addr learned my addr 172.21.15.36:0/4033416003 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:45.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.533+0000 7f6648bb0700 1 --2- 172.21.15.36:0/4033416003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6644147820 0x7f6644147cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:45.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.533+0000 7f66437fe700 1 --2- 172.21.15.36:0/4033416003 >> v2:172.21.15.70:3300/0 conn(0x7f66440b2770 0x7f664414b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:45.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.533+0000 7f6642ffd700 1 -- 172.21.15.36:0/4033416003 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2740546774 0 0) 0x7f66440605a0 con 0x7f66440601c0 2022-06-17T12:49:45.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.533+0000 7f6642ffd700 1 -- 172.21.15.36:0/4033416003 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6628003260 con 0x7f66440601c0 2022-06-17T12:49:45.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.533+0000 7f6648bb0700 1 -- 172.21.15.36:0/4033416003 >> v2:172.21.15.70:3300/0 conn(0x7f66440b2770 msgr2=0x7f664414b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:45.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.533+0000 7f6648bb0700 1 --2- 172.21.15.36:0/4033416003 >> v2:172.21.15.70:3300/0 conn(0x7f66440b2770 0x7f664414b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:45.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.533+0000 7f6648bb0700 1 -- 172.21.15.36:0/4033416003 >> v1:172.21.15.36:6789/0 conn(0x7f66440601c0 legacy=0x7f664414ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:45.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.533+0000 7f6648bb0700 1 -- 172.21.15.36:0/4033416003 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66440b2160 con 0x7f6644147820 2022-06-17T12:49:45.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.533+0000 7f6648bb0700 1 --2- 172.21.15.36:0/4033416003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6644147820 0x7f6644147cd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f663c002f70 tx=0x7f663c006df0).ready entity=mon.2 client_cookie=f8a03372d8a68968 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:45.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.533+0000 7f6642ffd700 1 -- 172.21.15.36:0/4033416003 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f663c00d4a0 con 0x7f6644147820 2022-06-17T12:49:45.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.533+0000 7f6642ffd700 1 -- 172.21.15.36:0/4033416003 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663c000e90 con 0x7f6644147820 2022-06-17T12:49:45.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.533+0000 7f6642ffd700 1 -- 172.21.15.36:0/4033416003 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f663c00d7f0 con 0x7f6644147820 2022-06-17T12:49:45.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f66493b1700 1 -- 172.21.15.36:0/4033416003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6644147820 msgr2=0x7f6644147cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:45.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f66493b1700 1 --2- 172.21.15.36:0/4033416003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6644147820 0x7f6644147cd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f663c002f70 tx=0x7f663c006df0).stop 2022-06-17T12:49:45.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f66493b1700 1 -- 172.21.15.36:0/4033416003 shutdown_connections 2022-06-17T12:49:45.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f66493b1700 1 --2- 172.21.15.36:0/4033416003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6644147820 0x7f6644147cd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:45.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f66493b1700 1 --2- 172.21.15.36:0/4033416003 >> v2:172.21.15.70:3300/0 conn(0x7f66440b2770 0x7f664414b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:45.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f66493b1700 1 -- 172.21.15.36:0/4033416003 >> 172.21.15.36:0/4033416003 conn(0x7f6644060990 msgr2=0x7f664405f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:45.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f66493b1700 1 -- 172.21.15.36:0/4033416003 shutdown_connections 2022-06-17T12:49:45.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f66493b1700 1 -- 172.21.15.36:0/4033416003 wait complete. 2022-06-17T12:49:45.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f66493b1700 1 Processor -- start 2022-06-17T12:49:45.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f66493b1700 1 -- start start 2022-06-17T12:49:45.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f66493b1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66440601c0 0x7f6644143dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:45.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f66493b1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66440b2770 0x7f6644156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:45.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f66493b1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6644147820 0x7f6644065280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:45.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f66493b1700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664414d7d0 con 0x7f6644147820 2022-06-17T12:49:45.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f66493b1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66440b3f20 con 0x7f66440601c0 2022-06-17T12:49:45.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f66493b1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6644064e20 con 0x7f66440b2770 2022-06-17T12:49:45.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f6648bb0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6644147820 0x7f6644065280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:45.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f6648bb0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6644147820 0x7f6644065280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33672/0 (socket says 172.21.15.36:33672) 2022-06-17T12:49:45.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f6648bb0700 1 -- 172.21.15.36:0/1704248009 learned_addr learned my addr 172.21.15.36:0/1704248009 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:45.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f6643fff700 1 --2- 172.21.15.36:0/1704248009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66440601c0 0x7f6644143dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:45.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f66437fe700 1 --2- 172.21.15.36:0/1704248009 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66440b2770 0x7f6644156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:45.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f6648bb0700 1 -- 172.21.15.36:0/1704248009 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66440b2770 msgr2=0x7f6644156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:45.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f6648bb0700 1 --2- 172.21.15.36:0/1704248009 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66440b2770 0x7f6644156740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:45.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f6648bb0700 1 -- 172.21.15.36:0/1704248009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66440601c0 msgr2=0x7f6644143dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:45.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f6648bb0700 1 --2- 172.21.15.36:0/1704248009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66440601c0 0x7f6644143dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:45.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f6648bb0700 1 -- 172.21.15.36:0/1704248009 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c00b180 con 0x7f6644147820 2022-06-17T12:49:45.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f6648bb0700 1 --2- 172.21.15.36:0/1704248009 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6644147820 0x7f6644065280 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f663c002f70 tx=0x7f663c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:45.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.537+0000 7f66417fa700 1 -- 172.21.15.36:0/1704248009 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f663c016070 con 0x7f6644147820 2022-06-17T12:49:45.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.541+0000 7f66417fa700 1 -- 172.21.15.36:0/1704248009 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663c0137d0 con 0x7f6644147820 2022-06-17T12:49:45.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.541+0000 7f66493b1700 1 -- 172.21.15.36:0/1704248009 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f664414d530 con 0x7f6644147820 2022-06-17T12:49:45.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.541+0000 7f66417fa700 1 -- 172.21.15.36:0/1704248009 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f663c013b30 con 0x7f6644147820 2022-06-17T12:49:45.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.541+0000 7f66493b1700 1 -- 172.21.15.36:0/1704248009 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6644144930 con 0x7f6644147820 2022-06-17T12:49:45.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.541+0000 7f66493b1700 1 -- 172.21.15.36:0/1704248009 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66440634a0 con 0x7f6644147820 2022-06-17T12:49:45.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.541+0000 7f66417fa700 1 -- 172.21.15.36:0/1704248009 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f663c025030 con 0x7f6644147820 2022-06-17T12:49:45.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.541+0000 7f66417fa700 1 --2- 172.21.15.36:0/1704248009 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f661c0539b0 0x7f661c055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:45.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.541+0000 7f66417fa700 1 -- 172.21.15.36:0/1704248009 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f663c06cf50 con 0x7f6644147820 2022-06-17T12:49:45.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.541+0000 7f6643fff700 1 --2- 172.21.15.36:0/1704248009 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f661c0539b0 0x7f661c055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:45.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.545+0000 7f6643fff700 1 --2- 172.21.15.36:0/1704248009 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f661c0539b0 0x7f661c055e70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f662c00d000 tx=0x7f662c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:45.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.549+0000 7f66417fa700 1 -- 172.21.15.36:0/1704248009 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f663c043cb0 con 0x7f6644147820 2022-06-17T12:49:45.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.677+0000 7f66493b1700 1 -- 172.21.15.36:0/1704248009 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6644064fb0 con 0x7f6644147820 2022-06-17T12:49:45.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.677+0000 7f66417fa700 1 -- 172.21.15.36:0/1704248009 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v45) v1 ==== 110+0+10 (secure 0 0 0) 0x7f663c023020 con 0x7f6644147820 2022-06-17T12:49:45.680 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:49:45.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.677+0000 7f66493b1700 1 -- 172.21.15.36:0/1704248009 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f661c0539b0 msgr2=0x7f661c055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:45.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.677+0000 7f66493b1700 1 --2- 172.21.15.36:0/1704248009 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f661c0539b0 0x7f661c055e70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f662c00d000 tx=0x7f662c009040).stop 2022-06-17T12:49:45.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.677+0000 7f66493b1700 1 -- 172.21.15.36:0/1704248009 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6644147820 msgr2=0x7f6644065280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:45.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.677+0000 7f66493b1700 1 --2- 172.21.15.36:0/1704248009 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6644147820 0x7f6644065280 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f663c002f70 tx=0x7f663c014000).stop 2022-06-17T12:49:45.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.677+0000 7f66493b1700 1 -- 172.21.15.36:0/1704248009 shutdown_connections 2022-06-17T12:49:45.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.677+0000 7f66493b1700 1 --2- 172.21.15.36:0/1704248009 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f661c0539b0 0x7f661c055e70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:45.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.677+0000 7f66493b1700 1 --2- 172.21.15.36:0/1704248009 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6644147820 0x7f6644065280 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:45.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.677+0000 7f66493b1700 1 --2- 172.21.15.36:0/1704248009 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66440b2770 0x7f6644156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:45.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.677+0000 7f66493b1700 1 --2- 172.21.15.36:0/1704248009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66440601c0 0x7f6644143dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:45.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.677+0000 7f66493b1700 1 -- 172.21.15.36:0/1704248009 >> 172.21.15.36:0/1704248009 conn(0x7f6644060990 msgr2=0x7f66440b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:45.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.681+0000 7f66493b1700 1 -- 172.21.15.36:0/1704248009 shutdown_connections 2022-06-17T12:49:45.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.681+0000 7f66493b1700 1 -- 172.21.15.36:0/1704248009 wait complete. 2022-06-17T12:49:45.712 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:49:45.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.865+0000 7f0a252d4700 1 Processor -- start 2022-06-17T12:49:45.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.865+0000 7f0a252d4700 1 -- start start 2022-06-17T12:49:45.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a252d4700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0a200b64f0 0x7f0a20143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:45.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a252d4700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0a200b4120 0x7f0a20146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:45.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a252d4700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a201476e0 con 0x7f0a20147270 2022-06-17T12:49:45.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a252d4700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a200b68d0 con 0x7f0a200b4120 2022-06-17T12:49:45.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a252d4700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a200b6e20 con 0x7f0a200b64f0 2022-06-17T12:49:45.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a1e7fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0a200b4120 0x7f0a20146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:45.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a1e7fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0a200b4120 0x7f0a20146d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33678/0 (socket says 172.21.15.36:33678) 2022-06-17T12:49:45.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a1e7fc700 1 -- 172.21.15.36:0/568162101 learned_addr learned my addr 172.21.15.36:0/568162101 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:45.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a1effd700 1 --2- 172.21.15.36:0/568162101 >> v2:172.21.15.70:3300/0 conn(0x7f0a200b64f0 0x7f0a20143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:45.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a1dffb700 1 -- 172.21.15.36:0/568162101 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 296390160 0 0) 0x7f0a201476e0 con 0x7f0a20147270 2022-06-17T12:49:45.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a1dffb700 1 -- 172.21.15.36:0/568162101 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a00003260 con 0x7f0a20147270 2022-06-17T12:49:45.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a1e7fc700 1 -- 172.21.15.36:0/568162101 >> v2:172.21.15.70:3300/0 conn(0x7f0a200b64f0 msgr2=0x7f0a20143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:45.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a1e7fc700 1 --2- 172.21.15.36:0/568162101 >> v2:172.21.15.70:3300/0 conn(0x7f0a200b64f0 0x7f0a20143150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:45.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a1e7fc700 1 -- 172.21.15.36:0/568162101 >> v1:172.21.15.70:6789/0 conn(0x7f0a20147270 legacy=0x7f0a20140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:45.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a1e7fc700 1 -- 172.21.15.36:0/568162101 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a201468e0 con 0x7f0a200b4120 2022-06-17T12:49:45.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a1e7fc700 1 --2- 172.21.15.36:0/568162101 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0a200b4120 0x7f0a20146d80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f0a10002f70 tx=0x7f0a10006eb0).ready entity=mon.0 client_cookie=c3301699077a2bee server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:45.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a1dffb700 1 -- 172.21.15.36:0/568162101 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a1000d610 con 0x7f0a200b4120 2022-06-17T12:49:45.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a1dffb700 1 -- 172.21.15.36:0/568162101 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a10000e90 con 0x7f0a200b4120 2022-06-17T12:49:45.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a1dffb700 1 -- 172.21.15.36:0/568162101 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a1000d9a0 con 0x7f0a200b4120 2022-06-17T12:49:45.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a252d4700 1 -- 172.21.15.36:0/568162101 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0a200b4120 msgr2=0x7f0a20146d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:45.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a252d4700 1 --2- 172.21.15.36:0/568162101 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0a200b4120 0x7f0a20146d80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f0a10002f70 tx=0x7f0a10006eb0).stop 2022-06-17T12:49:45.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a252d4700 1 -- 172.21.15.36:0/568162101 shutdown_connections 2022-06-17T12:49:45.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a252d4700 1 --2- 172.21.15.36:0/568162101 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0a200b4120 0x7f0a20146d80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:45.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a252d4700 1 --2- 172.21.15.36:0/568162101 >> v2:172.21.15.70:3300/0 conn(0x7f0a200b64f0 0x7f0a20143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:45.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a252d4700 1 -- 172.21.15.36:0/568162101 >> 172.21.15.36:0/568162101 conn(0x7f0a200620f0 msgr2=0x7f0a20062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:45.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.869+0000 7f0a252d4700 1 -- 172.21.15.36:0/568162101 shutdown_connections 2022-06-17T12:49:45.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a252d4700 1 -- 172.21.15.36:0/568162101 wait complete. 2022-06-17T12:49:45.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a252d4700 1 Processor -- start 2022-06-17T12:49:45.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a252d4700 1 -- start start 2022-06-17T12:49:45.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a252d4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0a200b4120 0x7f0a20148270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:45.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a252d4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a200b64f0 0x7f0a201418d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:45.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a252d4700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0a20147270 0x7f0a200b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:45.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a252d4700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a200b3330 con 0x7f0a200b4120 2022-06-17T12:49:45.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a252d4700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a200b1680 con 0x7f0a200b64f0 2022-06-17T12:49:45.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a252d4700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a200b1840 con 0x7f0a20147270 2022-06-17T12:49:45.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a1effd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0a200b4120 0x7f0a20148270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:45.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a1effd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0a200b4120 0x7f0a20148270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33684/0 (socket says 172.21.15.36:33684) 2022-06-17T12:49:45.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a1effd700 1 -- 172.21.15.36:0/3403040273 learned_addr learned my addr 172.21.15.36:0/3403040273 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:45.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a1e7fc700 1 --2- 172.21.15.36:0/3403040273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a200b64f0 0x7f0a201418d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:45.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a1f7fe700 1 --2- 172.21.15.36:0/3403040273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0a20147270 0x7f0a200b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:45.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a1effd700 1 -- 172.21.15.36:0/3403040273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0a20147270 msgr2=0x7f0a200b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:45.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a1effd700 1 --2- 172.21.15.36:0/3403040273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0a20147270 0x7f0a200b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:45.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a1effd700 1 -- 172.21.15.36:0/3403040273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a200b64f0 msgr2=0x7f0a201418d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:45.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a1effd700 1 --2- 172.21.15.36:0/3403040273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a200b64f0 0x7f0a201418d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:45.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a1effd700 1 -- 172.21.15.36:0/3403040273 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1000b1e0 con 0x7f0a200b4120 2022-06-17T12:49:45.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a1e7fc700 1 --2- 172.21.15.36:0/3403040273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a200b64f0 0x7f0a201418d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:49:45.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a1f7fe700 1 --2- 172.21.15.36:0/3403040273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0a20147270 0x7f0a200b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:49:45.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a1effd700 1 --2- 172.21.15.36:0/3403040273 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0a200b4120 0x7f0a20148270 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f0a20148ca0 tx=0x7f0a08006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:45.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a17fff700 1 -- 172.21.15.36:0/3403040273 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a0800c830 con 0x7f0a200b4120 2022-06-17T12:49:45.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a17fff700 1 -- 172.21.15.36:0/3403040273 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0800c990 con 0x7f0a200b4120 2022-06-17T12:49:45.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a252d4700 1 -- 172.21.15.36:0/3403040273 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a20141dc0 con 0x7f0a200b4120 2022-06-17T12:49:45.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a17fff700 1 -- 172.21.15.36:0/3403040273 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a0800ccf0 con 0x7f0a200b4120 2022-06-17T12:49:45.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.873+0000 7f0a252d4700 1 -- 172.21.15.36:0/3403040273 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a200b2e30 con 0x7f0a200b4120 2022-06-17T12:49:45.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.877+0000 7f0a252d4700 1 -- 172.21.15.36:0/3403040273 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a20141fb0 con 0x7f0a200b4120 2022-06-17T12:49:45.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.877+0000 7f0a17fff700 1 -- 172.21.15.36:0/3403040273 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f0a0801c460 con 0x7f0a200b4120 2022-06-17T12:49:45.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.877+0000 7f0a17fff700 1 --2- 172.21.15.36:0/3403040273 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0a040539b0 0x7f0a04055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:45.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.877+0000 7f0a1e7fc700 1 --2- 172.21.15.36:0/3403040273 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0a040539b0 0x7f0a04055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:45.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.877+0000 7f0a17fff700 1 -- 172.21.15.36:0/3403040273 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0a0806c520 con 0x7f0a200b4120 2022-06-17T12:49:45.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.877+0000 7f0a1e7fc700 1 --2- 172.21.15.36:0/3403040273 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0a040539b0 0x7f0a04055e70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f0a10017000 tx=0x7f0a10015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:45.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:45.885+0000 7f0a17fff700 1 -- 172.21.15.36:0/3403040273 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0a080432c0 con 0x7f0a200b4120 2022-06-17T12:49:46.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.013+0000 7f0a252d4700 1 -- 172.21.15.36:0/3403040273 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0a20140310 con 0x7f0a200b4120 2022-06-17T12:49:46.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.013+0000 7f0a17fff700 1 -- 172.21.15.36:0/3403040273 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0a080468e0 con 0x7f0a200b4120 2022-06-17T12:49:46.105 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:49:46.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.013+0000 7f0a252d4700 1 -- 172.21.15.36:0/3403040273 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0a040539b0 msgr2=0x7f0a04055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:46.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.013+0000 7f0a252d4700 1 --2- 172.21.15.36:0/3403040273 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0a040539b0 0x7f0a04055e70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f0a10017000 tx=0x7f0a10015000).stop 2022-06-17T12:49:46.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.013+0000 7f0a252d4700 1 -- 172.21.15.36:0/3403040273 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0a200b4120 msgr2=0x7f0a20148270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:46.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.013+0000 7f0a252d4700 1 --2- 172.21.15.36:0/3403040273 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0a200b4120 0x7f0a20148270 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f0a20148ca0 tx=0x7f0a08006df0).stop 2022-06-17T12:49:46.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.013+0000 7f0a252d4700 1 -- 172.21.15.36:0/3403040273 shutdown_connections 2022-06-17T12:49:46.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.013+0000 7f0a252d4700 1 --2- 172.21.15.36:0/3403040273 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0a040539b0 0x7f0a04055e70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:46.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.013+0000 7f0a252d4700 1 --2- 172.21.15.36:0/3403040273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0a20147270 0x7f0a200b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:46.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.013+0000 7f0a252d4700 1 --2- 172.21.15.36:0/3403040273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a200b64f0 0x7f0a201418d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:46.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.013+0000 7f0a252d4700 1 --2- 172.21.15.36:0/3403040273 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0a200b4120 0x7f0a20148270 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:46.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.013+0000 7f0a252d4700 1 -- 172.21.15.36:0/3403040273 >> 172.21.15.36:0/3403040273 conn(0x7f0a200620f0 msgr2=0x7f0a200b53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:46.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.013+0000 7f0a252d4700 1 -- 172.21.15.36:0/3403040273 shutdown_connections 2022-06-17T12:49:46.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.013+0000 7f0a252d4700 1 -- 172.21.15.36:0/3403040273 wait complete. 2022-06-17T12:49:46.117 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:49:46.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.273+0000 7f3e23fff700 1 Processor -- start 2022-06-17T12:49:46.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.273+0000 7f3e23fff700 1 -- start start 2022-06-17T12:49:46.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e23fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3e240b27b0 0x7f3e2413f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:46.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e23fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f3e240b0d70 0x7f3e2413f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:46.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e23fff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e240b2b90 con 0x7f3e240b15f0 2022-06-17T12:49:46.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e23fff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e240610e0 con 0x7f3e240b0d70 2022-06-17T12:49:46.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e23fff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e240ac190 con 0x7f3e240b27b0 2022-06-17T12:49:46.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e1a7fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f3e240b0d70 0x7f3e2413f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:46.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e1a7fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f3e240b0d70 0x7f3e2413f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33696/0 (socket says 172.21.15.36:33696) 2022-06-17T12:49:46.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e1a7fc700 1 -- 172.21.15.36:0/1954511744 learned_addr learned my addr 172.21.15.36:0/1954511744 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:46.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e22ffd700 1 --2- 172.21.15.36:0/1954511744 >> v2:172.21.15.70:3300/0 conn(0x7f3e240b27b0 0x7f3e2413f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:46.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e1a7fc700 1 -- 172.21.15.36:0/1954511744 >> v2:172.21.15.70:3300/0 conn(0x7f3e240b27b0 msgr2=0x7f3e2413f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:46.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e1a7fc700 1 --2- 172.21.15.36:0/1954511744 >> v2:172.21.15.70:3300/0 conn(0x7f3e240b27b0 0x7f3e2413f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:46.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e1a7fc700 1 -- 172.21.15.36:0/1954511744 >> v1:172.21.15.70:6789/0 conn(0x7f3e240b15f0 legacy=0x7f3e240ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:46.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e1a7fc700 1 -- 172.21.15.36:0/1954511744 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e240ac430 con 0x7f3e240b0d70 2022-06-17T12:49:46.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e1a7fc700 1 --2- 172.21.15.36:0/1954511744 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3e240b0d70 0x7f3e2413f600 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f3e14002f70 tx=0x7f3e14006eb0).ready entity=mon.0 client_cookie=718c7ecb37804465 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:46.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e227fc700 1 -- 172.21.15.36:0/1954511744 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e1400d610 con 0x7f3e240b0d70 2022-06-17T12:49:46.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e227fc700 1 -- 172.21.15.36:0/1954511744 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e14000e90 con 0x7f3e240b0d70 2022-06-17T12:49:46.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e227fc700 1 -- 172.21.15.36:0/1954511744 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e1400d9a0 con 0x7f3e240b0d70 2022-06-17T12:49:46.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e23fff700 1 -- 172.21.15.36:0/1954511744 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3e240b0d70 msgr2=0x7f3e2413f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:46.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e23fff700 1 --2- 172.21.15.36:0/1954511744 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3e240b0d70 0x7f3e2413f600 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f3e14002f70 tx=0x7f3e14006eb0).stop 2022-06-17T12:49:46.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e23fff700 1 -- 172.21.15.36:0/1954511744 shutdown_connections 2022-06-17T12:49:46.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e23fff700 1 --2- 172.21.15.36:0/1954511744 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3e240b0d70 0x7f3e2413f600 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:46.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e23fff700 1 --2- 172.21.15.36:0/1954511744 >> v2:172.21.15.70:3300/0 conn(0x7f3e240b27b0 0x7f3e2413f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:46.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e23fff700 1 -- 172.21.15.36:0/1954511744 >> 172.21.15.36:0/1954511744 conn(0x7f3e240b5d10 msgr2=0x7f3e240b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:46.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e23fff700 1 -- 172.21.15.36:0/1954511744 shutdown_connections 2022-06-17T12:49:46.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.277+0000 7f3e23fff700 1 -- 172.21.15.36:0/1954511744 wait complete. 2022-06-17T12:49:46.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e23fff700 1 Processor -- start 2022-06-17T12:49:46.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e23fff700 1 -- start start 2022-06-17T12:49:46.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e23fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e240b0d70 0x7f3e24131320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:46.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e23fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3e240b15f0 0x7f3e2413cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:46.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e23fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e240b27b0 0x7f3e240b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:46.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e23fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e24060e90 con 0x7f3e240b15f0 2022-06-17T12:49:46.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e23fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2412d3f0 con 0x7f3e240b27b0 2022-06-17T12:49:46.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e23fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2412d690 con 0x7f3e240b0d70 2022-06-17T12:49:46.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e1a7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3e240b15f0 0x7f3e2413cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:46.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e1a7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3e240b15f0 0x7f3e2413cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33700/0 (socket says 172.21.15.36:33700) 2022-06-17T12:49:46.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e1a7fc700 1 -- 172.21.15.36:0/1701409067 learned_addr learned my addr 172.21.15.36:0/1701409067 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:46.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e22ffd700 1 --2- 172.21.15.36:0/1701409067 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e240b0d70 0x7f3e24131320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:46.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e237fe700 1 --2- 172.21.15.36:0/1701409067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e240b27b0 0x7f3e240b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:46.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e1a7fc700 1 -- 172.21.15.36:0/1701409067 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e240b0d70 msgr2=0x7f3e24131320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:46.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e1a7fc700 1 --2- 172.21.15.36:0/1701409067 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e240b0d70 0x7f3e24131320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:46.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e1a7fc700 1 -- 172.21.15.36:0/1701409067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e240b27b0 msgr2=0x7f3e240b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:46.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e1a7fc700 1 --2- 172.21.15.36:0/1701409067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e240b27b0 0x7f3e240b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:46.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e1a7fc700 1 -- 172.21.15.36:0/1701409067 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e1400b1e0 con 0x7f3e240b15f0 2022-06-17T12:49:46.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e237fe700 1 --2- 172.21.15.36:0/1701409067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e240b27b0 0x7f3e240b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:49:46.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e1a7fc700 1 --2- 172.21.15.36:0/1701409067 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3e240b15f0 0x7f3e2413cbf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f3e14002f70 tx=0x7f3e14015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:46.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e20ff9700 1 -- 172.21.15.36:0/1701409067 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e14011860 con 0x7f3e240b15f0 2022-06-17T12:49:46.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e20ff9700 1 -- 172.21.15.36:0/1701409067 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e140119c0 con 0x7f3e240b15f0 2022-06-17T12:49:46.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e23fff700 1 -- 172.21.15.36:0/1701409067 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e2412d930 con 0x7f3e240b15f0 2022-06-17T12:49:46.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e20ff9700 1 -- 172.21.15.36:0/1701409067 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e14011d20 con 0x7f3e240b15f0 2022-06-17T12:49:46.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.281+0000 7f3e23fff700 1 -- 172.21.15.36:0/1701409067 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e2412ddc0 con 0x7f3e240b15f0 2022-06-17T12:49:46.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.285+0000 7f3e20ff9700 1 -- 172.21.15.36:0/1701409067 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f3e14025030 con 0x7f3e240b15f0 2022-06-17T12:49:46.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.285+0000 7f3e23fff700 1 -- 172.21.15.36:0/1701409067 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e2412e5e0 con 0x7f3e240b15f0 2022-06-17T12:49:46.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.285+0000 7f3e20ff9700 1 --2- 172.21.15.36:0/1701409067 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3e08053ad0 0x7f3e08055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:46.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.285+0000 7f3e20ff9700 1 -- 172.21.15.36:0/1701409067 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3e1406d230 con 0x7f3e240b15f0 2022-06-17T12:49:46.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.285+0000 7f3e22ffd700 1 --2- 172.21.15.36:0/1701409067 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3e08053ad0 0x7f3e08055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:46.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.285+0000 7f3e22ffd700 1 --2- 172.21.15.36:0/1701409067 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3e08053ad0 0x7f3e08055f90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f3e0c00c000 tx=0x7f3e0c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:46.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.293+0000 7f3e20ff9700 1 -- 172.21.15.36:0/1701409067 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3e2412e5e0 con 0x7f3e240b15f0 2022-06-17T12:49:46.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.421+0000 7f3e23fff700 1 -- 172.21.15.36:0/1701409067 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3e2412e5e0 con 0x7f3e240b15f0 2022-06-17T12:49:46.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.421+0000 7f3e20ff9700 1 -- 172.21.15.36:0/1701409067 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3e14023020 con 0x7f3e240b15f0 2022-06-17T12:49:46.428 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:49:46.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.421+0000 7f3e23fff700 1 -- 172.21.15.36:0/1701409067 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3e08053ad0 msgr2=0x7f3e08055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:46.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.421+0000 7f3e23fff700 1 --2- 172.21.15.36:0/1701409067 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3e08053ad0 0x7f3e08055f90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f3e0c00c000 tx=0x7f3e0c008040).stop 2022-06-17T12:49:46.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.421+0000 7f3e23fff700 1 -- 172.21.15.36:0/1701409067 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3e240b15f0 msgr2=0x7f3e2413cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:46.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.421+0000 7f3e23fff700 1 --2- 172.21.15.36:0/1701409067 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3e240b15f0 0x7f3e2413cbf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f3e14002f70 tx=0x7f3e14015000).stop 2022-06-17T12:49:46.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.421+0000 7f3e23fff700 1 -- 172.21.15.36:0/1701409067 shutdown_connections 2022-06-17T12:49:46.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.421+0000 7f3e23fff700 1 --2- 172.21.15.36:0/1701409067 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3e08053ad0 0x7f3e08055f90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:46.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.421+0000 7f3e23fff700 1 --2- 172.21.15.36:0/1701409067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e240b27b0 0x7f3e240b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:46.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.421+0000 7f3e23fff700 1 --2- 172.21.15.36:0/1701409067 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3e240b15f0 0x7f3e2413cbf0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:46.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.421+0000 7f3e23fff700 1 --2- 172.21.15.36:0/1701409067 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e240b0d70 0x7f3e24131320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:46.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.421+0000 7f3e23fff700 1 -- 172.21.15.36:0/1701409067 >> 172.21.15.36:0/1701409067 conn(0x7f3e240b5d10 msgr2=0x7f3e24130860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:46.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.421+0000 7f3e23fff700 1 -- 172.21.15.36:0/1701409067 shutdown_connections 2022-06-17T12:49:46.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.421+0000 7f3e23fff700 1 -- 172.21.15.36:0/1701409067 wait complete. 2022-06-17T12:49:46.553 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:49:46.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.693+0000 7f4a30b34700 1 Processor -- start 2022-06-17T12:49:46.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.693+0000 7f4a30b34700 1 -- start start 2022-06-17T12:49:46.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.693+0000 7f4a30b34700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4a2c14b820 0x7f4a2c14dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:46.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.693+0000 7f4a30b34700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a2c0b4160 con 0x7f4a2c0b45c0 2022-06-17T12:49:46.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.693+0000 7f4a30b34700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a2c0b9c40 con 0x7f4a2c0b9860 2022-06-17T12:49:46.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.693+0000 7f4a30b34700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a2c0b5120 con 0x7f4a2c14b820 2022-06-17T12:49:46.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.693+0000 7f4a2affd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f4a2c0b45c0 0x7f4a2c0b4a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51238/0 (socket says 172.21.15.36:51238) 2022-06-17T12:49:46.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.693+0000 7f4a2affd700 1 -- 172.21.15.36:0/4152580982 learned_addr learned my addr 172.21.15.36:0/4152580982 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:46.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.693+0000 7f4a2bfff700 1 --2- 172.21.15.36:0/4152580982 >> v2:172.21.15.70:3300/0 conn(0x7f4a2c14b820 0x7f4a2c14dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:46.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.693+0000 7f4a2a7fc700 1 -- 172.21.15.36:0/4152580982 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1655038926 0 0) 0x7f4a2c0b4160 con 0x7f4a2c0b45c0 2022-06-17T12:49:46.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.697+0000 7f4a2a7fc700 1 -- 172.21.15.36:0/4152580982 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a0c003260 con 0x7f4a2c0b45c0 2022-06-17T12:49:46.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.697+0000 7f4a2bfff700 1 -- 172.21.15.36:0/4152580982 >> v1:172.21.15.70:6789/0 conn(0x7f4a2c0b9860 legacy=0x7f4a2c143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:46.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.697+0000 7f4a2bfff700 1 -- 172.21.15.36:0/4152580982 >> v1:172.21.15.36:6789/0 conn(0x7f4a2c0b45c0 legacy=0x7f4a2c0b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:46.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.697+0000 7f4a2bfff700 1 -- 172.21.15.36:0/4152580982 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2c0627d0 con 0x7f4a2c14b820 2022-06-17T12:49:46.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.697+0000 7f4a2bfff700 1 --2- 172.21.15.36:0/4152580982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a2c14b820 0x7f4a2c14dc10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f4a240032f0 tx=0x7f4a240091d0).ready entity=mon.1 client_cookie=ac09c0a7b86a2493 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:46.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.697+0000 7f4a2a7fc700 1 -- 172.21.15.36:0/4152580982 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a2400b6c0 con 0x7f4a2c14b820 2022-06-17T12:49:46.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.697+0000 7f4a2a7fc700 1 -- 172.21.15.36:0/4152580982 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2400b820 con 0x7f4a2c14b820 2022-06-17T12:49:46.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.697+0000 7f4a2a7fc700 1 -- 172.21.15.36:0/4152580982 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a2400b990 con 0x7f4a2c14b820 2022-06-17T12:49:46.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.697+0000 7f4a30b34700 1 -- 172.21.15.36:0/4152580982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a2c14b820 msgr2=0x7f4a2c14dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:46.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.697+0000 7f4a30b34700 1 --2- 172.21.15.36:0/4152580982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a2c14b820 0x7f4a2c14dc10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f4a240032f0 tx=0x7f4a240091d0).stop 2022-06-17T12:49:46.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.697+0000 7f4a30b34700 1 -- 172.21.15.36:0/4152580982 shutdown_connections 2022-06-17T12:49:46.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.697+0000 7f4a30b34700 1 --2- 172.21.15.36:0/4152580982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a2c14b820 0x7f4a2c14dc10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:46.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.697+0000 7f4a30b34700 1 -- 172.21.15.36:0/4152580982 >> 172.21.15.36:0/4152580982 conn(0x7f4a2c05c550 msgr2=0x7f4a2c060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:46.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.697+0000 7f4a30b34700 1 -- 172.21.15.36:0/4152580982 shutdown_connections 2022-06-17T12:49:46.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.697+0000 7f4a30b34700 1 -- 172.21.15.36:0/4152580982 wait complete. 2022-06-17T12:49:46.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.701+0000 7f4a30b34700 1 Processor -- start 2022-06-17T12:49:46.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.701+0000 7f4a30b34700 1 -- start start 2022-06-17T12:49:46.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.701+0000 7f4a30b34700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4a2c0b45c0 0x7f4a2c12c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:46.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.701+0000 7f4a30b34700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4a2c0b9860 0x7f4a2c13ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:46.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.701+0000 7f4a30b34700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a2c14b820 0x7f4a2c140020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:46.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.701+0000 7f4a30b34700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2c13fc40 con 0x7f4a2c0b9860 2022-06-17T12:49:46.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.701+0000 7f4a30b34700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2c1469a0 con 0x7f4a2c14b820 2022-06-17T12:49:46.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.701+0000 7f4a30b34700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2c13d700 con 0x7f4a2c0b45c0 2022-06-17T12:49:46.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.701+0000 7f4a2affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4a2c0b9860 0x7f4a2c13ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:46.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.701+0000 7f4a2bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a2c14b820 0x7f4a2c140020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:46.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.701+0000 7f4a2b7fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4a2c0b45c0 0x7f4a2c12c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:46.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.701+0000 7f4a2affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4a2c0b9860 0x7f4a2c13ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33714/0 (socket says 172.21.15.36:33714) 2022-06-17T12:49:46.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.701+0000 7f4a2affd700 1 -- 172.21.15.36:0/1718491369 learned_addr learned my addr 172.21.15.36:0/1718491369 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:46.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.701+0000 7f4a2affd700 1 -- 172.21.15.36:0/1718491369 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4a2c0b45c0 msgr2=0x7f4a2c12c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:46.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.701+0000 7f4a2affd700 1 --2- 172.21.15.36:0/1718491369 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4a2c0b45c0 0x7f4a2c12c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:46.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.701+0000 7f4a2affd700 1 -- 172.21.15.36:0/1718491369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a2c14b820 msgr2=0x7f4a2c140020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:46.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.701+0000 7f4a2affd700 1 --2- 172.21.15.36:0/1718491369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a2c14b820 0x7f4a2c140020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:46.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.701+0000 7f4a2affd700 1 -- 172.21.15.36:0/1718491369 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a24009470 con 0x7f4a2c0b9860 2022-06-17T12:49:46.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.701+0000 7f4a2bfff700 1 --2- 172.21.15.36:0/1718491369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a2c14b820 0x7f4a2c140020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:49:46.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.701+0000 7f4a2b7fe700 1 --2- 172.21.15.36:0/1718491369 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4a2c0b45c0 0x7f4a2c12c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:49:46.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.705+0000 7f4a2affd700 1 --2- 172.21.15.36:0/1718491369 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4a2c0b9860 0x7f4a2c13ec40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f4a1c001110 tx=0x7f4a1c007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:46.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.705+0000 7f4a28ff9700 1 -- 172.21.15.36:0/1718491369 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a1c0070d0 con 0x7f4a2c0b9860 2022-06-17T12:49:46.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.705+0000 7f4a30b34700 1 -- 172.21.15.36:0/1718491369 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a2c13d930 con 0x7f4a2c0b9860 2022-06-17T12:49:46.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.705+0000 7f4a28ff9700 1 -- 172.21.15.36:0/1718491369 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1c007230 con 0x7f4a2c0b9860 2022-06-17T12:49:46.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.705+0000 7f4a28ff9700 1 -- 172.21.15.36:0/1718491369 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a1c005610 con 0x7f4a2c0b9860 2022-06-17T12:49:46.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.705+0000 7f4a30b34700 1 -- 172.21.15.36:0/1718491369 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a2c13f130 con 0x7f4a2c0b9860 2022-06-17T12:49:46.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.705+0000 7f4a30b34700 1 -- 172.21.15.36:0/1718491369 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a2c062eb0 con 0x7f4a2c0b9860 2022-06-17T12:49:46.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.705+0000 7f4a28ff9700 1 -- 172.21.15.36:0/1718491369 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f4a1c010070 con 0x7f4a2c0b9860 2022-06-17T12:49:46.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.709+0000 7f4a28ff9700 1 --2- 172.21.15.36:0/1718491369 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4a10053a40 0x7f4a10055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:46.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.709+0000 7f4a28ff9700 1 -- 172.21.15.36:0/1718491369 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4a1c06b660 con 0x7f4a2c0b9860 2022-06-17T12:49:46.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.709+0000 7f4a2b7fe700 1 --2- 172.21.15.36:0/1718491369 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4a10053a40 0x7f4a10055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:46.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.709+0000 7f4a2b7fe700 1 --2- 172.21.15.36:0/1718491369 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4a10053a40 0x7f4a10055f00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f4a1400d000 tx=0x7f4a14009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:46.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.713+0000 7f4a28ff9700 1 -- 172.21.15.36:0/1718491369 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4a1c0423c0 con 0x7f4a2c0b9860 2022-06-17T12:49:46.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.841+0000 7f4a30b34700 1 -- 172.21.15.36:0/1718491369 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4a2c057410 con 0x7f4a2c0b9860 2022-06-17T12:49:46.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.841+0000 7f4a28ff9700 1 -- 172.21.15.36:0/1718491369 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+10 (secure 0 0 0) 0x7f4a1c0459e0 con 0x7f4a2c0b9860 2022-06-17T12:49:46.829 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:49:46.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.841+0000 7f4a30b34700 1 -- 172.21.15.36:0/1718491369 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4a10053a40 msgr2=0x7f4a10055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:46.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.841+0000 7f4a30b34700 1 --2- 172.21.15.36:0/1718491369 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4a10053a40 0x7f4a10055f00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f4a1400d000 tx=0x7f4a14009040).stop 2022-06-17T12:49:46.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.841+0000 7f4a30b34700 1 -- 172.21.15.36:0/1718491369 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4a2c0b9860 msgr2=0x7f4a2c13ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:46.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.841+0000 7f4a30b34700 1 --2- 172.21.15.36:0/1718491369 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4a2c0b9860 0x7f4a2c13ec40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f4a1c001110 tx=0x7f4a1c007ad0).stop 2022-06-17T12:49:46.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.841+0000 7f4a30b34700 1 -- 172.21.15.36:0/1718491369 shutdown_connections 2022-06-17T12:49:46.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.841+0000 7f4a30b34700 1 --2- 172.21.15.36:0/1718491369 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4a10053a40 0x7f4a10055f00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:46.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.841+0000 7f4a30b34700 1 --2- 172.21.15.36:0/1718491369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a2c14b820 0x7f4a2c140020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:46.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.841+0000 7f4a30b34700 1 --2- 172.21.15.36:0/1718491369 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4a2c0b9860 0x7f4a2c13ec40 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:46.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.841+0000 7f4a30b34700 1 --2- 172.21.15.36:0/1718491369 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4a2c0b45c0 0x7f4a2c12c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:46.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.841+0000 7f4a30b34700 1 -- 172.21.15.36:0/1718491369 >> 172.21.15.36:0/1718491369 conn(0x7f4a2c05c550 msgr2=0x7f4a2c0b1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:46.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.841+0000 7f4a30b34700 1 -- 172.21.15.36:0/1718491369 shutdown_connections 2022-06-17T12:49:46.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.841+0000 7f4a30b34700 1 -- 172.21.15.36:0/1718491369 wait complete. 2022-06-17T12:49:46.848 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:49:46.849 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:49:46.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.977+0000 7f3476c97700 1 Processor -- start 2022-06-17T12:49:46.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.977+0000 7f3476c97700 1 -- start start 2022-06-17T12:49:46.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.977+0000 7f3476c97700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f34700b2cd0 0x7f34701465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:46.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.977+0000 7f3476c97700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3470146ae0 0x7f3470146f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:46.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.977+0000 7f3476c97700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34700b25e0 con 0x7f34700b2200 2022-06-17T12:49:46.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.977+0000 7f3476c97700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34700ad790 con 0x7f34700b2cd0 2022-06-17T12:49:46.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.977+0000 7f3476c97700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34700aef40 con 0x7f3470146ae0 2022-06-17T12:49:46.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.977+0000 7f3475c95700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f34700b2200 0x7f34700af3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51250/0 (socket says 172.21.15.36:51250) 2022-06-17T12:49:46.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.977+0000 7f3475c95700 1 -- 172.21.15.36:0/1471846863 learned_addr learned my addr 172.21.15.36:0/1471846863 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:49:46.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.977+0000 7f3475494700 1 --2- 172.21.15.36:0/1471846863 >> v2:172.21.15.36:3300/0 conn(0x7f34700b2cd0 0x7f34701465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:46.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.977+0000 7f3476496700 1 --2- 172.21.15.36:0/1471846863 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3470146ae0 0x7f3470146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:46.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.977+0000 7f3474c93700 1 -- 172.21.15.36:0/1471846863 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2107527201 0 0) 0x7f34700b25e0 con 0x7f34700b2200 2022-06-17T12:49:46.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3474c93700 1 -- 172.21.15.36:0/1471846863 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f345c003260 con 0x7f34700b2200 2022-06-17T12:49:46.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3476496700 1 -- 172.21.15.36:0/1471846863 >> v2:172.21.15.36:3300/0 conn(0x7f34700b2cd0 msgr2=0x7f34701465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:49:46.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3476496700 1 --2- 172.21.15.36:0/1471846863 >> v2:172.21.15.36:3300/0 conn(0x7f34700b2cd0 0x7f34701465f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:47.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3476496700 1 -- 172.21.15.36:0/1471846863 >> v1:172.21.15.36:6789/0 conn(0x7f34700b2200 legacy=0x7f34700af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:47.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3476496700 1 -- 172.21.15.36:0/1471846863 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34700afaf0 con 0x7f3470146ae0 2022-06-17T12:49:47.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3476496700 1 --2- 172.21.15.36:0/1471846863 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3470146ae0 0x7f3470146f30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f3460002f70 tx=0x7f3460006df0).ready entity=mon.2 client_cookie=a6372a0946777949 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:47.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3474c93700 1 -- 172.21.15.36:0/1471846863 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f346000d4a0 con 0x7f3470146ae0 2022-06-17T12:49:47.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3474c93700 1 -- 172.21.15.36:0/1471846863 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3460000e90 con 0x7f3470146ae0 2022-06-17T12:49:47.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3474c93700 1 -- 172.21.15.36:0/1471846863 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f346000d7f0 con 0x7f3470146ae0 2022-06-17T12:49:47.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3476c97700 1 -- 172.21.15.36:0/1471846863 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3470146ae0 msgr2=0x7f3470146f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:47.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3476c97700 1 --2- 172.21.15.36:0/1471846863 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3470146ae0 0x7f3470146f30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f3460002f70 tx=0x7f3460006df0).stop 2022-06-17T12:49:47.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3476c97700 1 -- 172.21.15.36:0/1471846863 shutdown_connections 2022-06-17T12:49:47.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3476c97700 1 --2- 172.21.15.36:0/1471846863 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3470146ae0 0x7f3470146f30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:47.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3476c97700 1 --2- 172.21.15.36:0/1471846863 >> v2:172.21.15.36:3300/0 conn(0x7f34700b2cd0 0x7f34701465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:47.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3476c97700 1 -- 172.21.15.36:0/1471846863 >> 172.21.15.36:0/1471846863 conn(0x7f347005f770 msgr2=0x7f347005fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:47.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3476c97700 1 -- 172.21.15.36:0/1471846863 shutdown_connections 2022-06-17T12:49:47.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3476c97700 1 -- 172.21.15.36:0/1471846863 wait complete. 2022-06-17T12:49:47.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3476c97700 1 Processor -- start 2022-06-17T12:49:47.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3476c97700 1 -- start start 2022-06-17T12:49:47.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3476c97700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34700b2200 0x7f3470159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:47.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3476c97700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f34700b2cd0 0x7f34700a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:47.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3476c97700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3470146ae0 0x7f3470149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:47.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3476c97700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34700a83a0 con 0x7f34700b2cd0 2022-06-17T12:49:47.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3476c97700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34700a85d0 con 0x7f34700b2200 2022-06-17T12:49:47.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3476c97700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34700a8870 con 0x7f3470146ae0 2022-06-17T12:49:47.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3475494700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f34700b2cd0 0x7f34700a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:47.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3475494700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f34700b2cd0 0x7f34700a97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33730/0 (socket says 172.21.15.36:33730) 2022-06-17T12:49:47.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.981+0000 7f3475c95700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34700b2200 0x7f3470159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:47.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.985+0000 7f3475494700 1 -- 172.21.15.36:0/1776410048 learned_addr learned my addr 172.21.15.36:0/1776410048 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:49:47.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.985+0000 7f3476496700 1 --2- 172.21.15.36:0/1776410048 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3470146ae0 0x7f3470149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:47.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.985+0000 7f3475494700 1 -- 172.21.15.36:0/1776410048 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3470146ae0 msgr2=0x7f3470149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:47.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.985+0000 7f3475494700 1 --2- 172.21.15.36:0/1776410048 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3470146ae0 0x7f3470149c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:47.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.985+0000 7f3475494700 1 -- 172.21.15.36:0/1776410048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34700b2200 msgr2=0x7f3470159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:47.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.985+0000 7f3475494700 1 --2- 172.21.15.36:0/1776410048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34700b2200 0x7f3470159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:47.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.985+0000 7f3475494700 1 -- 172.21.15.36:0/1776410048 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346000b180 con 0x7f34700b2cd0 2022-06-17T12:49:47.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.985+0000 7f3475494700 1 --2- 172.21.15.36:0/1776410048 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f34700b2cd0 0x7f34700a97f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f346400c000 tx=0x7f3464006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:47.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.985+0000 7f346effd700 1 -- 172.21.15.36:0/1776410048 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3464010780 con 0x7f34700b2cd0 2022-06-17T12:49:47.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.985+0000 7f346effd700 1 -- 172.21.15.36:0/1776410048 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34640108e0 con 0x7f34700b2cd0 2022-06-17T12:49:47.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.985+0000 7f3476c97700 1 -- 172.21.15.36:0/1776410048 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34700a8cb0 con 0x7f34700b2cd0 2022-06-17T12:49:47.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.985+0000 7f346effd700 1 -- 172.21.15.36:0/1776410048 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3464010c40 con 0x7f34700b2cd0 2022-06-17T12:49:47.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.985+0000 7f3476c97700 1 -- 172.21.15.36:0/1776410048 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3470062d50 con 0x7f34700b2cd0 2022-06-17T12:49:47.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.985+0000 7f3476c97700 1 -- 172.21.15.36:0/1776410048 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34700a8a30 con 0x7f34700b2cd0 2022-06-17T12:49:47.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.985+0000 7f346effd700 1 -- 172.21.15.36:0/1776410048 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f346401c460 con 0x7f34700b2cd0 2022-06-17T12:49:47.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.989+0000 7f346effd700 1 --2- 172.21.15.36:0/1776410048 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3450053ad0 0x7f3450055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:49:47.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.989+0000 7f346effd700 1 -- 172.21.15.36:0/1776410048 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f346406c3e0 con 0x7f34700b2cd0 2022-06-17T12:49:47.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.989+0000 7f3475c95700 1 --2- 172.21.15.36:0/1776410048 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3450053ad0 0x7f3450055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:49:47.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.989+0000 7f3475c95700 1 --2- 172.21.15.36:0/1776410048 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3450053ad0 0x7f3450055f90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f346800d000 tx=0x7f3468009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:49:47.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:46.993+0000 7f346effd700 1 -- 172.21.15.36:0/1776410048 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3464043180 con 0x7f34700b2cd0 2022-06-17T12:49:47.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:47.153+0000 7f3476c97700 1 -- 172.21.15.36:0/1776410048 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34700b1950 con 0x7f34700b2cd0 2022-06-17T12:49:47.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:47.153+0000 7f346effd700 1 -- 172.21.15.36:0/1776410048 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+2138 (secure 0 0 0) 0x7f34640467a0 con 0x7f34700b2cd0 2022-06-17T12:49:47.142 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:49:47.142 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":19,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/3777038413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3777038413},{"type":"v1","addr":"172.21.15.36:6835","nonce":3777038413}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:49:14.023351+0000","modified":"2022-06-17T12:49:15.026177+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5292":{"gid":5292,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/4117532324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4117532324},{"type":"v1","addr":"172.21.15.70:6833","nonce":4117532324}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-06-17T12:49:47.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:47.157+0000 7f3476c97700 1 -- 172.21.15.36:0/1776410048 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3450053ad0 msgr2=0x7f3450055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:47.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:47.157+0000 7f3476c97700 1 --2- 172.21.15.36:0/1776410048 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3450053ad0 0x7f3450055f90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f346800d000 tx=0x7f3468009040).stop 2022-06-17T12:49:47.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:47.157+0000 7f3476c97700 1 -- 172.21.15.36:0/1776410048 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f34700b2cd0 msgr2=0x7f34700a97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:49:47.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:47.157+0000 7f3476c97700 1 --2- 172.21.15.36:0/1776410048 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f34700b2cd0 0x7f34700a97f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f346400c000 tx=0x7f3464006df0).stop 2022-06-17T12:49:47.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:47.157+0000 7f3476c97700 1 -- 172.21.15.36:0/1776410048 shutdown_connections 2022-06-17T12:49:47.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:47.157+0000 7f3476c97700 1 --2- 172.21.15.36:0/1776410048 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3450053ad0 0x7f3450055f90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:47.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:47.157+0000 7f3476c97700 1 --2- 172.21.15.36:0/1776410048 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3470146ae0 0x7f3470149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:47.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:47.157+0000 7f3476c97700 1 --2- 172.21.15.36:0/1776410048 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f34700b2cd0 0x7f34700a97f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:47.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:47.157+0000 7f3476c97700 1 --2- 172.21.15.36:0/1776410048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34700b2200 0x7f3470159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:49:47.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:47.157+0000 7f3476c97700 1 -- 172.21.15.36:0/1776410048 >> 172.21.15.36:0/1776410048 conn(0x7f347005f770 msgr2=0x7f34700ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:49:47.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:47.157+0000 7f3476c97700 1 -- 172.21.15.36:0/1776410048 shutdown_connections 2022-06-17T12:49:47.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:49:47.157+0000 7f3476c97700 1 -- 172.21.15.36:0/1776410048 wait complete. 2022-06-17T12:49:47.162 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 19 2022-06-17T12:49:47.364 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:49:14.023351+0000', 'modified': '2022-06-17T12:49:15.026177+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5292}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5292': {'gid': 5292, 'name': 'b', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6833/4117532324', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 4117532324}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 4117532324}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:49:47.365 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:49:47.366 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-17T12:49:47.492 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":5292,"want_state":"up:active","state":"up:active","rank_uptime":33.463980521000003,"mdsmap_epoch":19,"osdmap_epoch":45,"osdmap_epoch_barrier":44,"uptime":38.236417314999997} 2022-06-17T12:49:47.493 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 5292, "mdsmap_epoch": 19, "osdmap_epoch": 45, "osdmap_epoch_barrier": 44, "rank_uptime": 33.463980521, "state": "up:active", "uptime": 38.236417315, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:49:47.494 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:49:47.495 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-17T12:49:47.495 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi107.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-17T12:49:47.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:49:47.497 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:49:47.513 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:49:47.536 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:49:47.537 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:49:47.600 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:49:47.602 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:49:47.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:49:47.604 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:49:47.636 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:49:47.638 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:49:47.696 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:49:47.724 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:49:47.725 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:49:47.771 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:2022-06-17T12:49:47.783+0000 7ff9b6b48080 -1 init, newargv = 0x55eb1b181940 newargc=9ceph-fuse[22004]: starting ceph client 2022-06-17T12:49:47.772 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr: 2022-06-17T12:49:47.828 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22004]: starting fuse 2022-06-17T12:49:47.830 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:49:47.831 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:49:47.832 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:49:47.833 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:49:47.865 INFO:teuthology.orchestra.run.smithi107.stdout:52 2022-06-17T12:49:47.866 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:49:47.866 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:49:47.867 DEBUG:teuthology.orchestra.run.smithi107:> sudo python3 -c ' 2022-06-17T12:49:47.867 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2022-06-17T12:49:47.868 DEBUG:teuthology.orchestra.run.smithi107:> import re 2022-06-17T12:49:47.868 DEBUG:teuthology.orchestra.run.smithi107:> import os 2022-06-17T12:49:47.873 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2022-06-17T12:49:47.873 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:49:47.874 DEBUG:teuthology.orchestra.run.smithi107:> def find_socket(client_name): 2022-06-17T12:49:47.874 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-17T12:49:47.875 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2022-06-17T12:49:47.875 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:49:47.875 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2022-06-17T12:49:47.876 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2022-06-17T12:49:47.876 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2022-06-17T12:49:47.877 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2022-06-17T12:49:47.877 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:49:47.877 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2022-06-17T12:49:47.878 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:49:47.878 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:49:47.878 DEBUG:teuthology.orchestra.run.smithi107:> return f 2022-06-17T12:49:47.879 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:49:47.879 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:49:47.884 DEBUG:teuthology.orchestra.run.smithi107:> print(find_socket("client.0")) 2022-06-17T12:49:47.885 DEBUG:teuthology.orchestra.run.smithi107:> ' 2022-06-17T12:49:47.953 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.0.22004.asok 2022-06-17T12:49:47.959 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22004.asok 2022-06-17T12:49:47.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:49:47.961 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22004.asok status 2022-06-17T12:49:48.098 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2022-06-17T12:49:48.099 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2022-06-17T12:49:48.099 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:49:48.100 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:49:48.100 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "0", 2022-06-17T12:49:48.101 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2022-06-17T12:49:48.101 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-17T12:49:48.115 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "22004", 2022-06-17T12:49:48.116 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/" 2022-06-17T12:49:48.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:49:48.116 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2022-06-17T12:49:48.117 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2022-06-17T12:49:48.117 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 5562, 2022-06-17T12:49:48.118 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2022-06-17T12:49:48.119 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2022-06-17T12:49:48.119 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2022-06-17T12:49:48.120 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 5562 2022-06-17T12:49:48.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:49:48.122 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:49:48.123 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:49:48.123 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:49:48.124 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3120469715 2022-06-17T12:49:48.124 INFO:teuthology.orchestra.run.smithi107.stdout: } 2022-06-17T12:49:48.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:49:48.125 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:49:48.126 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:49:48.126 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:49:48.131 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3120469715 2022-06-17T12:49:48.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:49:48.133 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.5562 172.21.15.107:0/3120469715", 2022-06-17T12:49:48.133 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "172.21.15.107:0/3120469715", 2022-06-17T12:49:48.134 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 1, 2022-06-17T12:49:48.134 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 19, 2022-06-17T12:49:48.135 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 45, 2022-06-17T12:49:48.136 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:49:48.136 INFO:teuthology.orchestra.run.smithi107.stdout: "blacklisted": false, 2022-06-17T12:49:48.136 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2022-06-17T12:49:48.137 INFO:teuthology.orchestra.run.smithi107.stdout:} 2022-06-17T12:49:48.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:49:48.141 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:49:48.142 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:49:48.180 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:49:48.181 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:49:48.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:49:48.195 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:50:14.776 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:50:14.781 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:50:14.795 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:50:14.798 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:50:14.809 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:50:34.250 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-17T12:50:34.252 DEBUG:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:50:34.280 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:50:34.312 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:50:34.340 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:50:34.364 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:50:34.391 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-17T12:50:34.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:50:34.393 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:50:34.395 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:50:34.406 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:50:34.408 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:50:34.409 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2022-06-17T12:50:34.410 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:50:34.411 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:50:34.471 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22004]: fuse finished with error 0 and tester_r 0 2022-06-17T12:50:34.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:50:34.473 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:50:34.474 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:50:34.482 INFO:teuthology.orchestra.run.smithi107.stdout:ext2/ext3 2022-06-17T12:50:34.484 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:50:34.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:50:34.623 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:50:34.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:50:34.634 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:50:34.635 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:50:34.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:50:34.655 INFO:teuthology.orchestra.run.smithi136.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-17T12:50:34.656 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-17T12:50:34.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:50:34.659 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:50:34.669 INFO:teuthology.orchestra.run.smithi136.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-17T12:50:34.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:50:34.673 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:50:34.674 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:50:34.674 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:50:34.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:50:34.695 INFO:teuthology.orchestra.run.smithi143.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:50:34.703 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-06-17T12:50:34.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:50:34.705 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:50:34.715 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:50:34.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:50:34.728 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-17T12:50:34.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.861+0000 7f3807634700 1 Processor -- start 2022-06-17T12:50:34.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.861+0000 7f3807634700 1 -- start start 2022-06-17T12:50:34.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.861+0000 7f3807634700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f38000627b0 0x7f38000af650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:34.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.861+0000 7f3807634700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3800142d30 0x7f38000aba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:34.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.861+0000 7f3807634700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38000b1170 con 0x7f38000ad190 2022-06-17T12:50:34.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.861+0000 7f3807634700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f380013f670 con 0x7f38000627b0 2022-06-17T12:50:34.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.861+0000 7f3807634700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3800062b90 con 0x7f3800142d30 2022-06-17T12:50:34.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.861+0000 7f3804bcf700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f38000ad190 0x7f38000ad930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51266/0 (socket says 172.21.15.36:51266) 2022-06-17T12:50:34.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.861+0000 7f3804bcf700 1 -- 172.21.15.36:0/838858243 learned_addr learned my addr 172.21.15.36:0/838858243 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:50:34.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.861+0000 7f3805bd1700 1 --2- 172.21.15.36:0/838858243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3800142d30 0x7f38000aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:34.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.861+0000 7f38053d0700 1 --2- 172.21.15.36:0/838858243 >> v2:172.21.15.70:3300/0 conn(0x7f38000627b0 0x7f38000af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:34.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.861+0000 7f37effff700 1 -- 172.21.15.36:0/838858243 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2731175781 0 0) 0x7f38000b1170 con 0x7f38000ad190 2022-06-17T12:50:34.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.861+0000 7f37effff700 1 -- 172.21.15.36:0/838858243 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37e8003260 con 0x7f38000ad190 2022-06-17T12:50:34.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.861+0000 7f38053d0700 1 -- 172.21.15.36:0/838858243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3800142d30 msgr2=0x7f38000aba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:34.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.861+0000 7f38053d0700 1 --2- 172.21.15.36:0/838858243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3800142d30 0x7f38000aba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:34.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.861+0000 7f38053d0700 1 -- 172.21.15.36:0/838858243 >> v1:172.21.15.36:6789/0 conn(0x7f38000ad190 legacy=0x7f38000ad930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:34.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.861+0000 7f38053d0700 1 -- 172.21.15.36:0/838858243 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38000b13a0 con 0x7f38000627b0 2022-06-17T12:50:34.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.861+0000 7f3805bd1700 1 --2- 172.21.15.36:0/838858243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3800142d30 0x7f38000aba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:50:34.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.861+0000 7f38053d0700 1 --2- 172.21.15.36:0/838858243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38000627b0 0x7f38000af650 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f37fc002f70 tx=0x7f37fc006df0).ready entity=mon.1 client_cookie=b124c2c0844b900a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:34.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.861+0000 7f37effff700 1 -- 172.21.15.36:0/838858243 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37fc014070 con 0x7f38000627b0 2022-06-17T12:50:34.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.861+0000 7f37effff700 1 -- 172.21.15.36:0/838858243 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37fc000e90 con 0x7f38000627b0 2022-06-17T12:50:34.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.861+0000 7f37effff700 1 -- 172.21.15.36:0/838858243 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37fc00fa90 con 0x7f38000627b0 2022-06-17T12:50:34.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.861+0000 7f3807634700 1 -- 172.21.15.36:0/838858243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38000627b0 msgr2=0x7f38000af650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:34.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.861+0000 7f3807634700 1 --2- 172.21.15.36:0/838858243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38000627b0 0x7f38000af650 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f37fc002f70 tx=0x7f37fc006df0).stop 2022-06-17T12:50:34.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.865+0000 7f3807634700 1 -- 172.21.15.36:0/838858243 shutdown_connections 2022-06-17T12:50:34.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.865+0000 7f3807634700 1 --2- 172.21.15.36:0/838858243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3800142d30 0x7f38000aba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:34.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.865+0000 7f3807634700 1 --2- 172.21.15.36:0/838858243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38000627b0 0x7f38000af650 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:34.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.865+0000 7f3807634700 1 -- 172.21.15.36:0/838858243 >> 172.21.15.36:0/838858243 conn(0x7f38000b5430 msgr2=0x7f38000b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:34.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.865+0000 7f3807634700 1 -- 172.21.15.36:0/838858243 shutdown_connections 2022-06-17T12:50:34.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.865+0000 7f3807634700 1 -- 172.21.15.36:0/838858243 wait complete. 2022-06-17T12:50:34.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.865+0000 7f3807634700 1 Processor -- start 2022-06-17T12:50:34.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.865+0000 7f3807634700 1 -- start start 2022-06-17T12:50:34.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.865+0000 7f3807634700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38000627b0 0x7f38000ab3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:34.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.865+0000 7f3807634700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38000ad190 0x7f3800151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:34.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.865+0000 7f3807634700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3800142d30 0x7f3800152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:34.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.865+0000 7f3807634700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38000a3290 con 0x7f3800142d30 2022-06-17T12:50:34.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.865+0000 7f3807634700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38000a34c0 con 0x7f38000627b0 2022-06-17T12:50:34.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.865+0000 7f3807634700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f380009f3e0 con 0x7f38000ad190 2022-06-17T12:50:34.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.865+0000 7f3805bd1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3800142d30 0x7f3800152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:34.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.869+0000 7f3805bd1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3800142d30 0x7f3800152390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33744/0 (socket says 172.21.15.36:33744) 2022-06-17T12:50:34.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.869+0000 7f3805bd1700 1 -- 172.21.15.36:0/3005403297 learned_addr learned my addr 172.21.15.36:0/3005403297 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:34.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.869+0000 7f38053d0700 1 --2- 172.21.15.36:0/3005403297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38000627b0 0x7f38000ab3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:34.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.869+0000 7f3804bcf700 1 --2- 172.21.15.36:0/3005403297 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38000ad190 0x7f3800151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:34.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.869+0000 7f3805bd1700 1 -- 172.21.15.36:0/3005403297 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38000ad190 msgr2=0x7f3800151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:34.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.869+0000 7f3805bd1700 1 --2- 172.21.15.36:0/3005403297 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38000ad190 0x7f3800151ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:34.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.869+0000 7f3805bd1700 1 -- 172.21.15.36:0/3005403297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38000627b0 msgr2=0x7f38000ab3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:34.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.869+0000 7f3805bd1700 1 --2- 172.21.15.36:0/3005403297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38000627b0 0x7f38000ab3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:34.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.869+0000 7f3805bd1700 1 -- 172.21.15.36:0/3005403297 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f800a610 con 0x7f3800142d30 2022-06-17T12:50:34.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.869+0000 7f3805bd1700 1 --2- 172.21.15.36:0/3005403297 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3800142d30 0x7f3800152390 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f37f800d000 tx=0x7f37f800a370).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:34.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.869+0000 7f37ee7fc700 1 -- 172.21.15.36:0/3005403297 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37f800c6f0 con 0x7f3800142d30 2022-06-17T12:50:34.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.869+0000 7f37ee7fc700 1 -- 172.21.15.36:0/3005403297 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f800c850 con 0x7f3800142d30 2022-06-17T12:50:34.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.869+0000 7f3807634700 1 -- 172.21.15.36:0/3005403297 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37fc00b180 con 0x7f3800142d30 2022-06-17T12:50:34.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.869+0000 7f37ee7fc700 1 -- 172.21.15.36:0/3005403297 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37f800cbb0 con 0x7f3800142d30 2022-06-17T12:50:34.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.869+0000 7f3807634700 1 -- 172.21.15.36:0/3005403297 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f380009ee60 con 0x7f3800142d30 2022-06-17T12:50:34.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.869+0000 7f3807634700 1 -- 172.21.15.36:0/3005403297 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f380009f170 con 0x7f3800142d30 2022-06-17T12:50:34.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.869+0000 7f37ee7fc700 1 -- 172.21.15.36:0/3005403297 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f37f8021070 con 0x7f3800142d30 2022-06-17T12:50:34.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.873+0000 7f37ee7fc700 1 --2- 172.21.15.36:0/3005403297 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f37e0053a40 0x7f37e0055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:34.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.873+0000 7f37ee7fc700 1 -- 172.21.15.36:0/3005403297 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f37f806c350 con 0x7f3800142d30 2022-06-17T12:50:34.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.873+0000 7f38053d0700 1 --2- 172.21.15.36:0/3005403297 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f37e0053a40 0x7f37e0055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:34.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.873+0000 7f38053d0700 1 --2- 172.21.15.36:0/3005403297 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f37e0053a40 0x7f37e0055f00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f37fc01d000 tx=0x7f37fc016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:34.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:34.877+0000 7f37ee7fc700 1 -- 172.21.15.36:0/3005403297 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f37f8043150 con 0x7f3800142d30 2022-06-17T12:50:34.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.005+0000 7f3807634700 1 -- 172.21.15.36:0/3005403297 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f38000a3680 con 0x7f3800142d30 2022-06-17T12:50:34.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.009+0000 7f37ee7fc700 1 -- 172.21.15.36:0/3005403297 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v45) v1 ==== 81+0+34037 (secure 0 0 0) 0x7f37f8015090 con 0x7f3800142d30 2022-06-17T12:50:34.995 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:50:34.996 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:50:34.996 INFO:teuthology.orchestra.run.smithi036.stdout: "epoch": 45, 2022-06-17T12:50:34.996 INFO:teuthology.orchestra.run.smithi036.stdout: "fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", 2022-06-17T12:50:34.997 INFO:teuthology.orchestra.run.smithi036.stdout: "created": "2022-06-17T12:46:30.671408+0000", 2022-06-17T12:50:34.997 INFO:teuthology.orchestra.run.smithi036.stdout: "modified": "2022-06-17T12:49:15.026771+0000", 2022-06-17T12:50:34.998 INFO:teuthology.orchestra.run.smithi036.stdout: "last_up_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:50:34.998 INFO:teuthology.orchestra.run.smithi036.stdout: "last_in_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:50:34.998 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-17T12:50:34.999 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_num": 5799936, 2022-06-17T12:50:35.000 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_set": [ 2022-06-17T12:50:35.000 INFO:teuthology.orchestra.run.smithi036.stdout: "pglog_hardlimit", 2022-06-17T12:50:35.001 INFO:teuthology.orchestra.run.smithi036.stdout: "purged_snapdirs", 2022-06-17T12:50:35.001 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_deletes", 2022-06-17T12:50:35.001 INFO:teuthology.orchestra.run.smithi036.stdout: "sortbitwise" 2022-06-17T12:50:35.003 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:50:35.003 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_version": 6, 2022-06-17T12:50:35.004 INFO:teuthology.orchestra.run.smithi036.stdout: "full_ratio": 0.94999998807907104, 2022-06-17T12:50:35.004 INFO:teuthology.orchestra.run.smithi036.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-17T12:50:35.005 INFO:teuthology.orchestra.run.smithi036.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-17T12:50:35.005 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_snapshot": "", 2022-06-17T12:50:35.005 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_max": 8, 2022-06-17T12:50:35.006 INFO:teuthology.orchestra.run.smithi036.stdout: "max_osd": 8, 2022-06-17T12:50:35.006 INFO:teuthology.orchestra.run.smithi036.stdout: "require_min_compat_client": "jewel", 2022-06-17T12:50:35.023 INFO:teuthology.orchestra.run.smithi036.stdout: "min_compat_client": "jewel", 2022-06-17T12:50:35.024 INFO:teuthology.orchestra.run.smithi036.stdout: "require_osd_release": "octopus", 2022-06-17T12:50:35.025 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:50:35.028 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.028 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 1, 2022-06-17T12:50:35.028 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "device_health_metrics", 2022-06-17T12:50:35.029 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:33.881212+0000", 2022-06-17T12:50:35.030 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:50:35.030 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:50:35.030 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:50:35.031 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:50:35.031 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:50:35.031 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:50:35.031 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:50:35.031 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:50:35.032 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 1, 2022-06-17T12:50:35.032 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 1, 2022-06-17T12:50:35.032 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 1, 2022-06-17T12:50:35.032 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 1, 2022-06-17T12:50:35.033 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 1, 2022-06-17T12:50:35.033 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:50:35.033 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:50:35.033 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:50:35.034 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:50:35.034 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:50:35.034 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:50:35.034 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:50:35.035 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.035 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "16", 2022-06-17T12:50:35.036 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:50:35.040 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:50:35.041 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:50:35.041 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:50:35.042 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:50:35.042 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:50:35.043 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:50:35.043 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:50:35.044 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:50:35.044 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:50:35.045 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:50:35.045 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:50:35.046 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:50:35.046 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:50:35.047 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:50:35.047 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:50:35.048 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:50:35.048 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:50:35.049 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:50:35.049 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:50:35.049 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:50:35.050 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:50:35.051 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:50:35.051 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:50:35.052 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:50:35.052 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:50:35.074 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.074 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:50:35.074 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:50:35.075 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:50:35.075 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:50:35.076 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:50:35.076 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:50:35.077 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:50:35.077 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:50:35.077 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:50:35.078 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:50:35.078 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:50:35.079 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:50:35.079 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 1 2022-06-17T12:50:35.079 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.080 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:50:35.080 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr_devicehealth": {} 2022-06-17T12:50:35.080 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.081 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.081 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.082 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 2, 2022-06-17T12:50:35.082 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "rbd", 2022-06-17T12:50:35.082 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:44.485460+0000", 2022-06-17T12:50:35.083 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:50:35.083 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:50:35.084 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:50:35.084 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:50:35.084 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:50:35.084 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:50:35.085 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:50:35.085 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:50:35.085 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:50:35.086 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:50:35.086 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:50:35.086 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:50:35.087 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:50:35.087 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:50:35.087 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:50:35.088 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:50:35.089 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:50:35.089 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:50:35.090 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:50:35.090 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:50:35.091 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.091 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "19", 2022-06-17T12:50:35.092 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:50:35.092 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:50:35.092 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:50:35.093 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:50:35.093 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:50:35.094 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:50:35.094 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:50:35.095 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:50:35.096 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:50:35.097 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:50:35.097 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:50:35.097 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:50:35.098 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:50:35.098 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:50:35.099 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:50:35.099 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:50:35.100 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:50:35.100 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:50:35.101 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:50:35.103 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:50:35.103 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:50:35.104 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:50:35.104 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:50:35.104 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:50:35.105 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:50:35.105 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:50:35.106 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.106 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:50:35.107 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:50:35.108 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:50:35.109 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:50:35.109 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:50:35.111 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:50:35.111 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:50:35.112 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:50:35.112 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:50:35.114 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:50:35.114 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:50:35.115 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:50:35.115 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:50:35.115 INFO:teuthology.orchestra.run.smithi036.stdout: "rbd": {} 2022-06-17T12:50:35.115 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.116 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.116 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.116 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 7, 2022-06-17T12:50:35.117 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_metadata", 2022-06-17T12:50:35.117 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:49:12.581297+0000", 2022-06-17T12:50:35.117 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:50:35.117 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:50:35.118 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:50:35.118 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:50:35.118 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:50:35.118 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:50:35.119 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:50:35.119 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:50:35.119 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:50:35.119 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:50:35.120 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:50:35.120 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:50:35.120 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:50:35.120 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:50:35.120 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:50:35.121 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:50:35.121 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:50:35.121 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:50:35.121 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:50:35.122 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:50:35.122 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.122 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "44", 2022-06-17T12:50:35.122 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:50:35.123 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:50:35.123 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:50:35.123 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:50:35.123 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:50:35.124 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:50:35.124 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:50:35.124 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:50:35.125 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:50:35.125 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:50:35.125 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:50:35.126 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:50:35.127 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:50:35.127 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:50:35.127 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:50:35.127 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:50:35.127 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:50:35.128 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:50:35.128 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:50:35.128 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:50:35.128 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:50:35.129 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:50:35.129 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:50:35.129 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:50:35.129 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:50:35.130 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:50:35.130 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.130 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:50:35.130 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:50:35.131 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:50:35.131 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:50:35.131 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:50:35.131 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:50:35.132 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:50:35.132 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:50:35.132 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:50:35.132 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:50:35.132 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:50:35.133 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:50:35.133 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_bias": 4, 2022-06-17T12:50:35.133 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 16, 2022-06-17T12:50:35.134 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_priority": 5 2022-06-17T12:50:35.134 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.134 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:50:35.134 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:50:35.134 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": "cephfs" 2022-06-17T12:50:35.135 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.135 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.135 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.135 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.136 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 8, 2022-06-17T12:50:35.136 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_data", 2022-06-17T12:50:35.136 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:49:12.985806+0000", 2022-06-17T12:50:35.136 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:50:35.137 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:50:35.137 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:50:35.137 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:50:35.137 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:50:35.138 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:50:35.138 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:50:35.139 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:50:35.141 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:50:35.141 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:50:35.141 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:50:35.142 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:50:35.142 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:50:35.143 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:50:35.143 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:50:35.144 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:50:35.144 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:50:35.144 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:50:35.145 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:50:35.145 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:50:35.146 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.146 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "45", 2022-06-17T12:50:35.146 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:50:35.147 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:50:35.148 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:50:35.148 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:50:35.149 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:50:35.149 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:50:35.151 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:50:35.151 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:50:35.152 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:50:35.152 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:50:35.153 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:50:35.153 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:50:35.154 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:50:35.154 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:50:35.155 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:50:35.155 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:50:35.156 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:50:35.156 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:50:35.157 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:50:35.157 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:50:35.158 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:50:35.158 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:50:35.159 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:50:35.159 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:50:35.160 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:50:35.160 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:50:35.162 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.162 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:50:35.163 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:50:35.163 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:50:35.164 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:50:35.164 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:50:35.165 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:50:35.165 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:50:35.165 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:50:35.166 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:50:35.166 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:50:35.167 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:50:35.167 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:50:35.167 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:50:35.167 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:50:35.168 INFO:teuthology.orchestra.run.smithi036.stdout: "data": "cephfs" 2022-06-17T12:50:35.168 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.168 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.168 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.169 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:50:35.169 INFO:teuthology.orchestra.run.smithi036.stdout: "osds": [ 2022-06-17T12:50:35.169 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.169 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:50:35.170 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", 2022-06-17T12:50:35.170 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:50:35.170 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:50:35.170 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:50:35.171 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:50:35.171 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:50:35.171 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:50:35.172 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:50:35.172 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 43, 2022-06-17T12:50:35.172 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:50:35.172 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:50:35.172 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:50:35.173 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.173 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.173 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.173 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6802", 2022-06-17T12:50:35.174 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:50:35.175 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.175 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.175 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.175 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6803", 2022-06-17T12:50:35.176 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:50:35.176 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.176 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.177 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.177 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:50:35.177 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.177 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.180 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.184 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6804", 2022-06-17T12:50:35.184 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:50:35.185 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.185 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.185 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.186 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6805", 2022-06-17T12:50:35.186 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:50:35.187 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.187 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.187 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.188 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:50:35.188 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.189 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.189 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.190 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6808", 2022-06-17T12:50:35.190 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:50:35.190 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.191 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.191 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.192 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6809", 2022-06-17T12:50:35.192 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:50:35.193 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.193 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.193 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.195 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:50:35.195 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.195 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.196 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.196 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6806", 2022-06-17T12:50:35.197 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:50:35.197 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.198 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.198 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.198 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6807", 2022-06-17T12:50:35.199 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:50:35.199 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.200 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.200 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.201 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6803/22619", 2022-06-17T12:50:35.201 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6805/22619", 2022-06-17T12:50:35.201 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6809/22619", 2022-06-17T12:50:35.202 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6807/22619", 2022-06-17T12:50:35.203 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:50:35.204 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:50:35.204 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:50:35.204 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.205 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.206 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.206 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:50:35.207 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", 2022-06-17T12:50:35.207 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:50:35.208 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:50:35.208 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:50:35.208 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:50:35.209 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:50:35.209 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:50:35.210 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:50:35.210 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 43, 2022-06-17T12:50:35.211 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:50:35.211 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:50:35.212 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:50:35.212 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.213 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.213 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.214 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6810", 2022-06-17T12:50:35.214 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:50:35.215 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.215 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.216 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.216 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6811", 2022-06-17T12:50:35.216 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:50:35.217 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.217 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.218 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.218 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:50:35.219 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.219 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.220 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.220 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6812", 2022-06-17T12:50:35.221 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:50:35.221 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.226 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.226 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.227 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6813", 2022-06-17T12:50:35.227 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:50:35.227 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.228 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.228 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.230 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:50:35.231 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.231 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.231 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.232 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6816", 2022-06-17T12:50:35.232 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:50:35.232 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.232 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.233 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.233 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6817", 2022-06-17T12:50:35.233 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:50:35.233 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.234 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.235 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.235 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:50:35.235 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.236 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.236 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.236 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6814", 2022-06-17T12:50:35.236 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:50:35.237 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.237 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.237 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.237 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6815", 2022-06-17T12:50:35.238 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:50:35.238 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.238 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.239 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.239 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6811/22620", 2022-06-17T12:50:35.241 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6813/22620", 2022-06-17T12:50:35.241 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6817/22620", 2022-06-17T12:50:35.241 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6815/22620", 2022-06-17T12:50:35.242 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:50:35.242 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:50:35.243 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:50:35.243 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.244 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.244 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.246 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:50:35.250 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", 2022-06-17T12:50:35.250 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:50:35.250 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:50:35.251 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:50:35.251 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:50:35.251 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:50:35.252 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:50:35.252 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:50:35.252 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 32, 2022-06-17T12:50:35.252 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:50:35.253 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:50:35.253 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:50:35.253 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.253 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.254 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.254 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6826", 2022-06-17T12:50:35.254 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:50:35.255 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.255 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.256 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.256 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6827", 2022-06-17T12:50:35.256 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:50:35.257 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.257 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.257 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.258 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:50:35.258 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.259 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.259 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.260 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6828", 2022-06-17T12:50:35.260 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:50:35.261 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.261 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.262 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.262 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6829", 2022-06-17T12:50:35.263 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:50:35.263 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.264 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.265 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.265 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:50:35.266 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.267 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.267 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.268 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6832", 2022-06-17T12:50:35.268 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:50:35.268 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.269 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.269 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.270 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6833", 2022-06-17T12:50:35.270 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:50:35.271 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.271 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.272 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.272 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:50:35.273 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.273 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.274 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.274 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6830", 2022-06-17T12:50:35.274 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:50:35.275 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.275 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.276 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.276 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6831", 2022-06-17T12:50:35.276 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:50:35.277 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.277 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.277 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.278 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6827/22643", 2022-06-17T12:50:35.278 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6829/22643", 2022-06-17T12:50:35.279 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6833/22643", 2022-06-17T12:50:35.280 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6831/22643", 2022-06-17T12:50:35.281 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:50:35.281 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:50:35.281 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:50:35.282 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.282 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.283 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.283 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:50:35.284 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", 2022-06-17T12:50:35.284 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:50:35.285 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:50:35.285 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:50:35.286 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:50:35.286 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:50:35.287 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:50:35.287 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:50:35.288 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 43, 2022-06-17T12:50:35.288 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:50:35.288 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:50:35.289 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:50:35.289 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.290 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.290 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.291 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6818", 2022-06-17T12:50:35.291 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:50:35.292 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.292 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.293 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.293 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6819", 2022-06-17T12:50:35.294 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:50:35.294 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.295 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.295 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.295 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:50:35.296 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.297 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.298 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.298 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6820", 2022-06-17T12:50:35.298 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:50:35.299 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.299 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.299 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.300 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6821", 2022-06-17T12:50:35.300 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:50:35.301 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.301 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.301 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.302 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:50:35.302 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.303 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.303 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.304 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6824", 2022-06-17T12:50:35.304 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:50:35.304 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.305 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.305 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.306 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6825", 2022-06-17T12:50:35.306 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:50:35.308 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.309 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.309 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.310 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:50:35.310 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.311 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.311 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.312 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6822", 2022-06-17T12:50:35.313 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:50:35.313 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.314 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.314 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.315 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6823", 2022-06-17T12:50:35.315 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:50:35.315 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.316 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.316 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.317 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6819/22646", 2022-06-17T12:50:35.317 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6821/22646", 2022-06-17T12:50:35.317 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6825/22646", 2022-06-17T12:50:35.318 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6823/22646", 2022-06-17T12:50:35.318 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:50:35.319 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:50:35.319 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:50:35.319 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.320 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.321 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.321 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:50:35.322 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", 2022-06-17T12:50:35.322 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:50:35.322 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:50:35.323 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:50:35.323 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:50:35.324 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:50:35.325 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:50:35.325 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:50:35.326 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 43, 2022-06-17T12:50:35.326 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:50:35.326 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:50:35.327 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:50:35.327 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.328 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.328 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.329 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6808", 2022-06-17T12:50:35.329 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:50:35.329 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.330 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.330 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.331 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6809", 2022-06-17T12:50:35.331 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:50:35.331 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.332 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.332 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.333 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:50:35.333 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.334 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.334 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.334 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6810", 2022-06-17T12:50:35.335 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:50:35.336 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.336 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.336 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.337 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6811", 2022-06-17T12:50:35.337 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:50:35.338 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.339 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.339 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.339 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:50:35.340 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.340 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.341 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.341 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6814", 2022-06-17T12:50:35.342 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:50:35.342 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.342 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.343 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.343 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6815", 2022-06-17T12:50:35.344 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:50:35.344 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.345 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.345 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.346 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:50:35.346 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.347 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.347 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.347 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6812", 2022-06-17T12:50:35.348 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:50:35.349 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.349 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.349 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.350 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6813", 2022-06-17T12:50:35.350 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:50:35.350 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.351 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.351 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.351 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6809/22812", 2022-06-17T12:50:35.352 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6811/22812", 2022-06-17T12:50:35.352 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6815/22812", 2022-06-17T12:50:35.352 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6813/22812", 2022-06-17T12:50:35.353 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:50:35.353 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:50:35.354 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:50:35.354 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.355 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.355 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.356 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:50:35.356 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", 2022-06-17T12:50:35.357 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:50:35.357 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:50:35.357 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:50:35.358 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:50:35.358 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:50:35.359 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:50:35.359 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:50:35.359 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 43, 2022-06-17T12:50:35.360 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:50:35.360 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:50:35.361 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:50:35.361 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.362 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.362 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.363 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6800", 2022-06-17T12:50:35.363 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:50:35.364 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.364 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.364 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.365 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6801", 2022-06-17T12:50:35.365 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:50:35.366 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.366 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.367 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.367 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:50:35.368 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.368 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.369 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.369 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6802", 2022-06-17T12:50:35.369 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:50:35.370 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.370 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.371 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.371 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6803", 2022-06-17T12:50:35.372 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:50:35.372 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.373 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.373 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.374 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:50:35.375 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.375 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.376 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.377 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6806", 2022-06-17T12:50:35.377 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:50:35.378 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.379 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.379 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.380 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6807", 2022-06-17T12:50:35.380 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:50:35.381 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.381 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.381 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.384 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:50:35.384 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.385 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.385 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.386 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6804", 2022-06-17T12:50:35.386 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:50:35.386 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.387 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.387 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.387 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6805", 2022-06-17T12:50:35.388 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:50:35.388 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.389 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.389 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.390 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6801/22813", 2022-06-17T12:50:35.390 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6803/22813", 2022-06-17T12:50:35.391 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6807/22813", 2022-06-17T12:50:35.392 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6805/22813", 2022-06-17T12:50:35.392 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:50:35.392 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:50:35.392 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:50:35.393 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.393 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.393 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.394 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:50:35.394 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", 2022-06-17T12:50:35.399 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:50:35.400 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:50:35.400 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:50:35.400 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:50:35.401 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:50:35.401 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:50:35.401 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:50:35.401 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 43, 2022-06-17T12:50:35.402 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:50:35.402 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:50:35.402 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:50:35.402 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.403 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.403 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.403 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6816", 2022-06-17T12:50:35.404 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:50:35.404 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.404 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.405 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.405 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6817", 2022-06-17T12:50:35.405 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:50:35.406 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.406 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.406 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.407 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:50:35.407 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.408 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.408 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.408 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6818", 2022-06-17T12:50:35.409 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:50:35.409 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.409 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.409 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.410 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6819", 2022-06-17T12:50:35.410 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:50:35.410 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.411 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.411 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.411 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:50:35.412 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.412 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.412 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.413 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6822", 2022-06-17T12:50:35.413 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:50:35.413 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.414 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.414 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.414 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6823", 2022-06-17T12:50:35.414 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:50:35.415 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.415 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.416 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.416 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:50:35.416 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.417 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.417 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.418 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6820", 2022-06-17T12:50:35.418 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:50:35.418 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.419 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.419 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.420 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6821", 2022-06-17T12:50:35.420 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:50:35.420 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.421 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.421 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.421 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6817/22822", 2022-06-17T12:50:35.421 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6819/22822", 2022-06-17T12:50:35.421 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6823/22822", 2022-06-17T12:50:35.422 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6821/22822", 2022-06-17T12:50:35.422 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:50:35.422 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:50:35.422 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:50:35.423 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.423 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.423 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.423 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:50:35.424 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", 2022-06-17T12:50:35.424 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:50:35.424 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:50:35.424 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:50:35.425 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:50:35.425 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:50:35.425 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:50:35.425 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:50:35.426 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 42, 2022-06-17T12:50:35.426 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:50:35.426 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:50:35.426 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:50:35.427 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.427 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.427 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.427 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6824", 2022-06-17T12:50:35.428 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:50:35.428 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.428 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.429 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.429 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6825", 2022-06-17T12:50:35.430 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:50:35.430 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.430 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.430 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.431 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:50:35.431 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.431 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.431 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.432 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6826", 2022-06-17T12:50:35.432 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:50:35.432 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.432 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.433 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.433 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6827", 2022-06-17T12:50:35.433 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:50:35.433 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.434 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.434 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.434 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:50:35.434 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.435 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.435 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.435 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6830", 2022-06-17T12:50:35.436 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:50:35.436 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.436 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.437 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.437 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6831", 2022-06-17T12:50:35.437 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:50:35.438 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.438 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.439 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.440 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:50:35.440 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:35.440 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.441 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:35.442 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6828", 2022-06-17T12:50:35.442 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:50:35.443 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.443 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.444 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:35.445 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6829", 2022-06-17T12:50:35.445 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:50:35.446 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.447 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.447 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.448 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6825/22837", 2022-06-17T12:50:35.448 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6827/22837", 2022-06-17T12:50:35.449 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6831/22837", 2022-06-17T12:50:35.450 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6829/22837", 2022-06-17T12:50:35.451 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:50:35.451 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:50:35.451 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:50:35.451 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:35.452 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.452 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:50:35.452 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_xinfo": [ 2022-06-17T12:50:35.453 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.454 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:50:35.455 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:50:35.456 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:50:35.456 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:50:35.456 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:50:35.457 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:50:35.457 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.291461+0000", 2022-06-17T12:50:35.457 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:50:35.457 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.458 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.458 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:50:35.458 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:50:35.458 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:50:35.459 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:50:35.459 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:50:35.460 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:50:35.460 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.537912+0000", 2022-06-17T12:50:35.460 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:50:35.461 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.461 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.462 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:50:35.462 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:50:35.462 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:50:35.463 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:50:35.463 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:50:35.465 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:50:35.465 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.606947+0000", 2022-06-17T12:50:35.466 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:50:35.467 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.467 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.467 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:50:35.468 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:50:35.468 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:50:35.469 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:50:35.469 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:50:35.470 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:50:35.470 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.514412+0000", 2022-06-17T12:50:35.471 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:50:35.472 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.472 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.473 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:50:35.476 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:50:35.477 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:50:35.477 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:50:35.477 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:50:35.478 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:50:35.478 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.394645+0000", 2022-06-17T12:50:35.479 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:50:35.479 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.480 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.480 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:50:35.481 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:50:35.481 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:50:35.482 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:50:35.486 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:50:35.486 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:50:35.487 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.343806+0000", 2022-06-17T12:50:35.488 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:50:35.488 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.489 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.489 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:50:35.490 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:50:35.490 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:50:35.491 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:50:35.491 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:50:35.492 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:50:35.492 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.591864+0000", 2022-06-17T12:50:35.492 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:50:35.493 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.493 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:35.494 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:50:35.494 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:50:35.494 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:50:35.495 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:50:35.496 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:50:35.496 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:50:35.497 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.607045+0000", 2022-06-17T12:50:35.497 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:50:35.497 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.498 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:50:35.498 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap": [], 2022-06-17T12:50:35.499 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap_items": [], 2022-06-17T12:50:35.499 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_temp": [], 2022-06-17T12:50:35.500 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_temp": [], 2022-06-17T12:50:35.500 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklist": {}, 2022-06-17T12:50:35.501 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profiles": { 2022-06-17T12:50:35.501 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs_data": { 2022-06-17T12:50:35.501 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-device-class": "", 2022-06-17T12:50:35.502 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:50:35.502 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-root": "default", 2022-06-17T12:50:35.503 INFO:teuthology.orchestra.run.smithi036.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-17T12:50:35.503 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:50:35.504 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "2", 2022-06-17T12:50:35.504 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:50:35.505 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:50:35.505 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van", 2022-06-17T12:50:35.506 INFO:teuthology.orchestra.run.smithi036.stdout: "w": "8" 2022-06-17T12:50:35.506 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.507 INFO:teuthology.orchestra.run.smithi036.stdout: "default": { 2022-06-17T12:50:35.507 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:50:35.508 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:50:35.509 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "1", 2022-06-17T12:50:35.509 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:50:35.510 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:50:35.511 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van" 2022-06-17T12:50:35.511 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:35.512 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:35.512 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps_queue": [], 2022-06-17T12:50:35.513 INFO:teuthology.orchestra.run.smithi036.stdout: "new_removed_snaps": [], 2022-06-17T12:50:35.513 INFO:teuthology.orchestra.run.smithi036.stdout: "new_purged_snaps": [], 2022-06-17T12:50:35.513 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_node_flags": {}, 2022-06-17T12:50:35.514 INFO:teuthology.orchestra.run.smithi036.stdout: "device_class_flags": {} 2022-06-17T12:50:35.514 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:50:35.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.009+0000 7f3807634700 1 -- 172.21.15.36:0/3005403297 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f37e0053a40 msgr2=0x7f37e0055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:35.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.009+0000 7f3807634700 1 --2- 172.21.15.36:0/3005403297 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f37e0053a40 0x7f37e0055f00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f37fc01d000 tx=0x7f37fc016000).stop 2022-06-17T12:50:35.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.009+0000 7f3807634700 1 -- 172.21.15.36:0/3005403297 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3800142d30 msgr2=0x7f3800152390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:35.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.009+0000 7f3807634700 1 --2- 172.21.15.36:0/3005403297 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3800142d30 0x7f3800152390 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f37f800d000 tx=0x7f37f800a370).stop 2022-06-17T12:50:35.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.009+0000 7f3807634700 1 -- 172.21.15.36:0/3005403297 shutdown_connections 2022-06-17T12:50:35.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.009+0000 7f3807634700 1 --2- 172.21.15.36:0/3005403297 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f37e0053a40 0x7f37e0055f00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:35.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.009+0000 7f3807634700 1 --2- 172.21.15.36:0/3005403297 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3800142d30 0x7f3800152390 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:35.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.009+0000 7f3807634700 1 --2- 172.21.15.36:0/3005403297 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38000ad190 0x7f3800151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:35.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.009+0000 7f3807634700 1 --2- 172.21.15.36:0/3005403297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38000627b0 0x7f38000ab3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:35.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.009+0000 7f3807634700 1 -- 172.21.15.36:0/3005403297 >> 172.21.15.36:0/3005403297 conn(0x7f38000b5430 msgr2=0x7f38000b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:35.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.009+0000 7f3807634700 1 -- 172.21.15.36:0/3005403297 shutdown_connections 2022-06-17T12:50:35.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.009+0000 7f3807634700 1 -- 172.21.15.36:0/3005403297 wait complete. 2022-06-17T12:50:35.528 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:50:35.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.657+0000 7f96bdc2b700 1 Processor -- start 2022-06-17T12:50:35.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.657+0000 7f96bdc2b700 1 -- start start 2022-06-17T12:50:35.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96bdc2b700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f96b80b16d0 0x7f96b8141540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:35.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96bdc2b700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b8065090 con 0x7f96b80b12f0 2022-06-17T12:50:35.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96bdc2b700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b80b0b40 con 0x7f96b8062bd0 2022-06-17T12:50:35.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96bdc2b700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b8063510 con 0x7f96b80b16d0 2022-06-17T12:50:35.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96b77fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f96b80b12f0 0x7f96b813aac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51280/0 (socket says 172.21.15.36:51280) 2022-06-17T12:50:35.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96b77fe700 1 -- 172.21.15.36:0/87557905 learned_addr learned my addr 172.21.15.36:0/87557905 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:50:35.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96b7fff700 1 --2- 172.21.15.36:0/87557905 >> v2:172.21.15.70:3300/0 conn(0x7f96b80b16d0 0x7f96b8141540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:35.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96b67fc700 1 -- 172.21.15.36:0/87557905 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 423608435 0 0) 0x7f96b8065090 con 0x7f96b80b12f0 2022-06-17T12:50:35.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96b67fc700 1 -- 172.21.15.36:0/87557905 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9698003260 con 0x7f96b80b12f0 2022-06-17T12:50:35.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96b67fc700 1 -- 172.21.15.36:0/87557905 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3452592226 0 0) 0x7f96b80b0b40 con 0x7f96b8062bd0 2022-06-17T12:50:35.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96b67fc700 1 -- 172.21.15.36:0/87557905 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96b8065090 con 0x7f96b8062bd0 2022-06-17T12:50:35.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96b7fff700 1 -- 172.21.15.36:0/87557905 >> v1:172.21.15.70:6789/0 conn(0x7f96b8062bd0 legacy=0x7f96b8140e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:35.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96b7fff700 1 -- 172.21.15.36:0/87557905 >> v1:172.21.15.36:6789/0 conn(0x7f96b80b12f0 legacy=0x7f96b813aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:35.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96b7fff700 1 -- 172.21.15.36:0/87557905 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b8064df0 con 0x7f96b80b16d0 2022-06-17T12:50:35.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96b7fff700 1 --2- 172.21.15.36:0/87557905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b80b16d0 0x7f96b8141540 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f96b8063060 tx=0x7f96b00091d0).ready entity=mon.1 client_cookie=2f6956231b87830c server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:35.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96b67fc700 1 -- 172.21.15.36:0/87557905 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96b000b600 con 0x7f96b80b16d0 2022-06-17T12:50:35.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96b67fc700 1 -- 172.21.15.36:0/87557905 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b000b760 con 0x7f96b80b16d0 2022-06-17T12:50:35.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96b67fc700 1 -- 172.21.15.36:0/87557905 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96b000b8d0 con 0x7f96b80b16d0 2022-06-17T12:50:35.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96bdc2b700 1 -- 172.21.15.36:0/87557905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b80b16d0 msgr2=0x7f96b8141540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:35.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96bdc2b700 1 --2- 172.21.15.36:0/87557905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b80b16d0 0x7f96b8141540 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f96b8063060 tx=0x7f96b00091d0).stop 2022-06-17T12:50:35.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96bdc2b700 1 -- 172.21.15.36:0/87557905 shutdown_connections 2022-06-17T12:50:35.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96bdc2b700 1 --2- 172.21.15.36:0/87557905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b80b16d0 0x7f96b8141540 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:35.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96bdc2b700 1 -- 172.21.15.36:0/87557905 >> 172.21.15.36:0/87557905 conn(0x7f96b80b53e0 msgr2=0x7f96b80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:35.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96bdc2b700 1 -- 172.21.15.36:0/87557905 shutdown_connections 2022-06-17T12:50:35.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.661+0000 7f96bdc2b700 1 -- 172.21.15.36:0/87557905 wait complete. 2022-06-17T12:50:35.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.665+0000 7f96bdc2b700 1 Processor -- start 2022-06-17T12:50:35.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.665+0000 7f96bdc2b700 1 -- start start 2022-06-17T12:50:35.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.665+0000 7f96bdc2b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96b8062bd0 0x7f96b8150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:35.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.665+0000 7f96bdc2b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96b80b12f0 0x7f96b8153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:35.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.665+0000 7f96bdc2b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b80b16d0 0x7f96b8153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:35.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.665+0000 7f96bdc2b700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b8146190 con 0x7f96b8062bd0 2022-06-17T12:50:35.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.665+0000 7f96bdc2b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b8141290 con 0x7f96b80b16d0 2022-06-17T12:50:35.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.665+0000 7f96bdc2b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b8140d50 con 0x7f96b80b12f0 2022-06-17T12:50:35.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.665+0000 7f96b77fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96b8062bd0 0x7f96b8150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:35.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.665+0000 7f96b77fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96b8062bd0 0x7f96b8150f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33754/0 (socket says 172.21.15.36:33754) 2022-06-17T12:50:35.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.665+0000 7f96b77fe700 1 -- 172.21.15.36:0/1506901807 learned_addr learned my addr 172.21.15.36:0/1506901807 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:35.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.665+0000 7f96b6ffd700 1 --2- 172.21.15.36:0/1506901807 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96b80b12f0 0x7f96b8153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:35.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.665+0000 7f96b7fff700 1 --2- 172.21.15.36:0/1506901807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b80b16d0 0x7f96b8153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:35.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.665+0000 7f96b77fe700 1 -- 172.21.15.36:0/1506901807 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96b80b12f0 msgr2=0x7f96b8153a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:35.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.665+0000 7f96b77fe700 1 --2- 172.21.15.36:0/1506901807 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96b80b12f0 0x7f96b8153a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:35.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.665+0000 7f96b77fe700 1 -- 172.21.15.36:0/1506901807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b80b16d0 msgr2=0x7f96b8153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:35.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.665+0000 7f96b77fe700 1 --2- 172.21.15.36:0/1506901807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b80b16d0 0x7f96b8153f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:35.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.665+0000 7f96b77fe700 1 -- 172.21.15.36:0/1506901807 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b0009470 con 0x7f96b8062bd0 2022-06-17T12:50:35.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.665+0000 7f96b6ffd700 1 --2- 172.21.15.36:0/1506901807 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96b80b12f0 0x7f96b8153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:50:35.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.665+0000 7f96b77fe700 1 --2- 172.21.15.36:0/1506901807 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96b8062bd0 0x7f96b8150f50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f96a80011d0 tx=0x7f96a8007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:35.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.665+0000 7f96b4ff9700 1 -- 172.21.15.36:0/1506901807 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96a80070d0 con 0x7f96b8062bd0 2022-06-17T12:50:35.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.665+0000 7f96b4ff9700 1 -- 172.21.15.36:0/1506901807 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a8007230 con 0x7f96b8062bd0 2022-06-17T12:50:35.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.669+0000 7f96bdc2b700 1 -- 172.21.15.36:0/1506901807 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b8140ff0 con 0x7f96b8062bd0 2022-06-17T12:50:35.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.669+0000 7f96b4ff9700 1 -- 172.21.15.36:0/1506901807 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96a8005610 con 0x7f96b8062bd0 2022-06-17T12:50:35.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.669+0000 7f96bdc2b700 1 -- 172.21.15.36:0/1506901807 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b8141e90 con 0x7f96b8062bd0 2022-06-17T12:50:35.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.669+0000 7f96bdc2b700 1 -- 172.21.15.36:0/1506901807 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b8150cf0 con 0x7f96b8062bd0 2022-06-17T12:50:35.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.669+0000 7f96b4ff9700 1 -- 172.21.15.36:0/1506901807 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f96a801a060 con 0x7f96b8062bd0 2022-06-17T12:50:35.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.669+0000 7f96b4ff9700 1 --2- 172.21.15.36:0/1506901807 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f969c053a40 0x7f969c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:35.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.669+0000 7f96b4ff9700 1 -- 172.21.15.36:0/1506901807 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f96a806b6d0 con 0x7f96b8062bd0 2022-06-17T12:50:35.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.669+0000 7f96b6ffd700 1 --2- 172.21.15.36:0/1506901807 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f969c053a40 0x7f969c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:35.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.669+0000 7f96b6ffd700 1 --2- 172.21.15.36:0/1506901807 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f969c053a40 0x7f969c055f00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f96a000d000 tx=0x7f96a0007ab0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:35.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.677+0000 7f96b4ff9700 1 -- 172.21.15.36:0/1506901807 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f96a80424d0 con 0x7f96b8062bd0 2022-06-17T12:50:35.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.841+0000 7f96bdc2b700 1 -- 172.21.15.36:0/1506901807 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96b80b03c0 con 0x7f96b8062bd0 2022-06-17T12:50:35.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.841+0000 7f96b4ff9700 1 -- 172.21.15.36:0/1506901807 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+2138 (secure 0 0 0) 0x7f96a8018020 con 0x7f96b8062bd0 2022-06-17T12:50:35.829 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:50:35.830 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":19,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/3777038413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3777038413},{"type":"v1","addr":"172.21.15.36:6835","nonce":3777038413}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:49:14.023351+0000","modified":"2022-06-17T12:49:15.026177+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5292":{"gid":5292,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/4117532324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4117532324},{"type":"v1","addr":"172.21.15.70:6833","nonce":4117532324}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-06-17T12:50:35.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.845+0000 7f96bdc2b700 1 -- 172.21.15.36:0/1506901807 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f969c053a40 msgr2=0x7f969c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:35.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.845+0000 7f96bdc2b700 1 --2- 172.21.15.36:0/1506901807 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f969c053a40 0x7f969c055f00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f96a000d000 tx=0x7f96a0007ab0).stop 2022-06-17T12:50:35.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.845+0000 7f96bdc2b700 1 -- 172.21.15.36:0/1506901807 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96b8062bd0 msgr2=0x7f96b8150f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:35.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.845+0000 7f96bdc2b700 1 --2- 172.21.15.36:0/1506901807 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96b8062bd0 0x7f96b8150f50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f96a80011d0 tx=0x7f96a8007b50).stop 2022-06-17T12:50:35.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.845+0000 7f96bdc2b700 1 -- 172.21.15.36:0/1506901807 shutdown_connections 2022-06-17T12:50:35.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.845+0000 7f96bdc2b700 1 --2- 172.21.15.36:0/1506901807 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f969c053a40 0x7f969c055f00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:35.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.845+0000 7f96bdc2b700 1 --2- 172.21.15.36:0/1506901807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b80b16d0 0x7f96b8153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:35.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.845+0000 7f96bdc2b700 1 --2- 172.21.15.36:0/1506901807 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96b80b12f0 0x7f96b8153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:35.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.845+0000 7f96bdc2b700 1 --2- 172.21.15.36:0/1506901807 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96b8062bd0 0x7f96b8150f50 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:35.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.845+0000 7f96bdc2b700 1 -- 172.21.15.36:0/1506901807 >> 172.21.15.36:0/1506901807 conn(0x7f96b80b53e0 msgr2=0x7f96b80b4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:35.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.845+0000 7f96bdc2b700 1 -- 172.21.15.36:0/1506901807 shutdown_connections 2022-06-17T12:50:35.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.845+0000 7f96bdc2b700 1 -- 172.21.15.36:0/1506901807 wait complete. 2022-06-17T12:50:35.836 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 19 2022-06-17T12:50:35.850 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-17T12:50:35.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9902ec3700 1 Processor -- start 2022-06-17T12:50:35.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9902ec3700 1 -- start start 2022-06-17T12:50:35.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9902ec3700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f98fc0b2200 0x7f98fc0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:35.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9902ec3700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f98fc0af8d0 0x7f98fc14a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:35.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9902ec3700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98fc0b25e0 con 0x7f98fc0b2cd0 2022-06-17T12:50:35.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9902ec3700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98fc0ad790 con 0x7f98fc0af8d0 2022-06-17T12:50:35.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9902ec3700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98fc0aef40 con 0x7f98fc0b2200 2022-06-17T12:50:35.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f99016c0700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f98fc0af8d0 0x7f98fc14a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:35.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f99026c2700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f98fc0b2cd0 0x7f98fc14adc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51296/0 (socket says 172.21.15.36:51296) 2022-06-17T12:50:35.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f99026c2700 1 -- 172.21.15.36:0/2134925982 learned_addr learned my addr 172.21.15.36:0/2134925982 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:50:35.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9901ec1700 1 --2- 172.21.15.36:0/2134925982 >> v2:172.21.15.70:3300/0 conn(0x7f98fc0b2200 0x7f98fc0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:35.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9900ebf700 1 -- 172.21.15.36:0/2134925982 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 260305410 0 0) 0x7f98fc0b25e0 con 0x7f98fc0b2cd0 2022-06-17T12:50:35.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9900ebf700 1 -- 172.21.15.36:0/2134925982 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98e4003260 con 0x7f98fc0b2cd0 2022-06-17T12:50:35.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9900ebf700 1 -- 172.21.15.36:0/2134925982 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4008683346 0 0) 0x7f98e4003260 con 0x7f98fc0b2cd0 2022-06-17T12:50:35.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9900ebf700 1 -- 172.21.15.36:0/2134925982 >> v2:172.21.15.70:3300/0 conn(0x7f98fc0b2200 msgr2=0x7f98fc0af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:35.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9900ebf700 1 --2- 172.21.15.36:0/2134925982 >> v2:172.21.15.70:3300/0 conn(0x7f98fc0b2200 0x7f98fc0af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:35.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9900ebf700 1 -- 172.21.15.36:0/2134925982 >> v2:172.21.15.36:3300/0 conn(0x7f98fc0af8d0 msgr2=0x7f98fc14a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:35.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9900ebf700 1 --2- 172.21.15.36:0/2134925982 >> v2:172.21.15.36:3300/0 conn(0x7f98fc0af8d0 0x7f98fc14a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:35.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9900ebf700 1 -- 172.21.15.36:0/2134925982 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98fc0b1950 con 0x7f98fc0b2cd0 2022-06-17T12:50:35.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9900ebf700 1 -- 172.21.15.36:0/2134925982 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f98f4001660 con 0x7f98fc0b2cd0 2022-06-17T12:50:35.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9900ebf700 1 -- 172.21.15.36:0/2134925982 >> v1:172.21.15.36:6789/0 conn(0x7f98fc0b2cd0 legacy=0x7f98fc14adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:35.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9900ebf700 1 --2- 172.21.15.36:0/2134925982 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98e4003750 0x7f98e4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:35.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9900ebf700 1 --2- 172.21.15.36:0/2134925982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98e4004500 0x7f98e4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:35.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9900ebf700 1 --2- 172.21.15.36:0/2134925982 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98e4004e70 0x7f98e400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:35.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9900ebf700 1 -- 172.21.15.36:0/2134925982 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc0ad790 con 0x7f98e4004e70 2022-06-17T12:50:35.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9900ebf700 1 -- 172.21.15.36:0/2134925982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc0aef40 con 0x7f98e4004500 2022-06-17T12:50:35.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9900ebf700 1 -- 172.21.15.36:0/2134925982 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e400bc40 con 0x7f98e4003750 2022-06-17T12:50:35.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f99026c2700 1 --2- 172.21.15.36:0/2134925982 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98e4004e70 0x7f98e400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:35.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f99016c0700 1 --2- 172.21.15.36:0/2134925982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98e4004500 0x7f98e4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:35.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9901ec1700 1 --2- 172.21.15.36:0/2134925982 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98e4003750 0x7f98e4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:35.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f99016c0700 1 -- 172.21.15.36:0/2134925982 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98e4003750 msgr2=0x7f98e4003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:35.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f99016c0700 1 --2- 172.21.15.36:0/2134925982 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98e4003750 0x7f98e4003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:35.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f99016c0700 1 -- 172.21.15.36:0/2134925982 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98e4004e70 msgr2=0x7f98e400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:35.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f99016c0700 1 --2- 172.21.15.36:0/2134925982 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98e4004e70 0x7f98e400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:35.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f99016c0700 1 -- 172.21.15.36:0/2134925982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e4003260 con 0x7f98e4004500 2022-06-17T12:50:35.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f99026c2700 1 --2- 172.21.15.36:0/2134925982 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98e4004e70 0x7f98e400b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:50:35.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f99026c2700 1 -- 172.21.15.36:0/2134925982 reap_dead start 2022-06-17T12:50:35.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f99016c0700 1 --2- 172.21.15.36:0/2134925982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98e4004500 0x7f98e4004980 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f98f800c000 tx=0x7f98f80091d0).ready entity=mon.1 client_cookie=985cae6cf80e9a56 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:35.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9900ebf700 1 -- 172.21.15.36:0/2134925982 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98f800e780 con 0x7f98e4004500 2022-06-17T12:50:35.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9900ebf700 1 -- 172.21.15.36:0/2134925982 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f800e8e0 con 0x7f98e4004500 2022-06-17T12:50:35.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.973+0000 7f9900ebf700 1 -- 172.21.15.36:0/2134925982 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98f800ec40 con 0x7f98e4004500 2022-06-17T12:50:35.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.977+0000 7f9902ec3700 1 -- 172.21.15.36:0/2134925982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98e4004500 msgr2=0x7f98e4004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:35.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.977+0000 7f9902ec3700 1 --2- 172.21.15.36:0/2134925982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98e4004500 0x7f98e4004980 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f98f800c000 tx=0x7f98f80091d0).stop 2022-06-17T12:50:35.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.977+0000 7f9902ec3700 1 -- 172.21.15.36:0/2134925982 shutdown_connections 2022-06-17T12:50:35.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.977+0000 7f9902ec3700 1 --2- 172.21.15.36:0/2134925982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98e4004500 0x7f98e4004980 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:35.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.977+0000 7f9902ec3700 1 -- 172.21.15.36:0/2134925982 >> 172.21.15.36:0/2134925982 conn(0x7f98fc05f770 msgr2=0x7f98fc05fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:35.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.977+0000 7f9902ec3700 1 -- 172.21.15.36:0/2134925982 shutdown_connections 2022-06-17T12:50:35.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.977+0000 7f9902ec3700 1 -- 172.21.15.36:0/2134925982 wait complete. 2022-06-17T12:50:35.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.977+0000 7f9902ec3700 1 Processor -- start 2022-06-17T12:50:35.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.977+0000 7f9902ec3700 1 -- start start 2022-06-17T12:50:35.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.977+0000 7f9902ec3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98e4004500 0x7f98fc159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:35.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.977+0000 7f9902ec3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98fc1446b0 0x7f98fc142c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:35.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.977+0000 7f9902ec3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98fc143150 0x7f98fc0a70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:35.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.977+0000 7f9902ec3700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc1456e0 con 0x7f98fc1446b0 2022-06-17T12:50:35.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.977+0000 7f9902ec3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc145c90 con 0x7f98e4004500 2022-06-17T12:50:35.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.977+0000 7f9902ec3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc15a2c0 con 0x7f98fc143150 2022-06-17T12:50:35.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.977+0000 7f99016c0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98fc1446b0 0x7f98fc142c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:35.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.977+0000 7f99016c0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98fc1446b0 0x7f98fc142c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33776/0 (socket says 172.21.15.36:33776) 2022-06-17T12:50:35.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.977+0000 7f99016c0700 1 -- 172.21.15.36:0/1961045989 learned_addr learned my addr 172.21.15.36:0/1961045989 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:35.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.981+0000 7f99026c2700 1 --2- 172.21.15.36:0/1961045989 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98fc143150 0x7f98fc0a70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:35.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.981+0000 7f9901ec1700 1 --2- 172.21.15.36:0/1961045989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98e4004500 0x7f98fc159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:35.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.981+0000 7f99016c0700 1 -- 172.21.15.36:0/1961045989 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98fc143150 msgr2=0x7f98fc0a70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:35.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.981+0000 7f99016c0700 1 --2- 172.21.15.36:0/1961045989 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98fc143150 0x7f98fc0a70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:35.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.981+0000 7f99016c0700 1 -- 172.21.15.36:0/1961045989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98e4004500 msgr2=0x7f98fc159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:35.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.981+0000 7f99016c0700 1 --2- 172.21.15.36:0/1961045989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98e4004500 0x7f98fc159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:35.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.981+0000 7f99016c0700 1 -- 172.21.15.36:0/1961045989 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98fc1459f0 con 0x7f98fc1446b0 2022-06-17T12:50:35.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.981+0000 7f9901ec1700 1 --2- 172.21.15.36:0/1961045989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98e4004500 0x7f98fc159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:50:35.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.981+0000 7f99016c0700 1 --2- 172.21.15.36:0/1961045989 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98fc1446b0 0x7f98fc142c60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f98fc1568b0 tx=0x7f98f801a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:35.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.981+0000 7f98eaffd700 1 -- 172.21.15.36:0/1961045989 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98f8009600 con 0x7f98fc1446b0 2022-06-17T12:50:35.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.981+0000 7f98eaffd700 1 -- 172.21.15.36:0/1961045989 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f801de20 con 0x7f98fc1446b0 2022-06-17T12:50:35.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.981+0000 7f98eaffd700 1 -- 172.21.15.36:0/1961045989 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98f801b640 con 0x7f98fc1446b0 2022-06-17T12:50:35.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.981+0000 7f9902ec3700 1 -- 172.21.15.36:0/1961045989 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98fc145240 con 0x7f98fc1446b0 2022-06-17T12:50:35.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.981+0000 7f9902ec3700 1 -- 172.21.15.36:0/1961045989 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98fc0a7770 con 0x7f98fc1446b0 2022-06-17T12:50:35.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.981+0000 7f9902ec3700 1 -- 172.21.15.36:0/1961045989 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98fc0a9ca0 con 0x7f98fc1446b0 2022-06-17T12:50:35.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.981+0000 7f98eaffd700 1 -- 172.21.15.36:0/1961045989 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f98f801b7a0 con 0x7f98fc1446b0 2022-06-17T12:50:35.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.985+0000 7f98eaffd700 1 --2- 172.21.15.36:0/1961045989 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f98dc053a60 0x7f98dc055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:35.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.985+0000 7f98eaffd700 1 -- 172.21.15.36:0/1961045989 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f98f806dec0 con 0x7f98fc1446b0 2022-06-17T12:50:35.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.985+0000 7f9901ec1700 1 --2- 172.21.15.36:0/1961045989 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f98dc053a60 0x7f98dc055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:35.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.985+0000 7f9901ec1700 1 --2- 172.21.15.36:0/1961045989 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f98dc053a60 0x7f98dc055f20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f98ec00b000 tx=0x7f98ec008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:35.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:35.989+0000 7f98eaffd700 1 -- 172.21.15.36:0/1961045989 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f98fc0a9ca0 con 0x7f98fc1446b0 2022-06-17T12:50:36.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.149+0000 7f9902ec3700 1 -- 172.21.15.36:0/1961045989 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f98fc0a9ca0 con 0x7f98fc1446b0 2022-06-17T12:50:36.626 INFO:tasks.ceph.mon.a.smithi036.stderr:2022-06-17T12:50:36.641+0000 7fb7a2649700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-17T12:50:36.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.653+0000 7f98eaffd700 1 -- 172.21.15.36:0/1961045989 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v20) v1 ==== 161+0+0 (secure 0 0 0) 0x7f98fc0a9ca0 con 0x7f98fc1446b0 2022-06-17T12:50:36.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.657+0000 7f9902ec3700 1 -- 172.21.15.36:0/1961045989 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f98dc053a60 msgr2=0x7f98dc055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:36.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.657+0000 7f9902ec3700 1 --2- 172.21.15.36:0/1961045989 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f98dc053a60 0x7f98dc055f20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f98ec00b000 tx=0x7f98ec008040).stop 2022-06-17T12:50:36.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.657+0000 7f9902ec3700 1 -- 172.21.15.36:0/1961045989 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98fc1446b0 msgr2=0x7f98fc142c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:36.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.657+0000 7f9902ec3700 1 --2- 172.21.15.36:0/1961045989 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98fc1446b0 0x7f98fc142c60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f98fc1568b0 tx=0x7f98f801a000).stop 2022-06-17T12:50:36.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.657+0000 7f9902ec3700 1 -- 172.21.15.36:0/1961045989 shutdown_connections 2022-06-17T12:50:36.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.657+0000 7f9902ec3700 1 --2- 172.21.15.36:0/1961045989 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f98dc053a60 0x7f98dc055f20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:36.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.657+0000 7f9902ec3700 1 --2- 172.21.15.36:0/1961045989 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98fc143150 0x7f98fc0a70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:36.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.657+0000 7f9902ec3700 1 --2- 172.21.15.36:0/1961045989 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98fc1446b0 0x7f98fc142c60 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:36.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.657+0000 7f9902ec3700 1 --2- 172.21.15.36:0/1961045989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98e4004500 0x7f98fc159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:36.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.657+0000 7f9902ec3700 1 -- 172.21.15.36:0/1961045989 >> 172.21.15.36:0/1961045989 conn(0x7f98fc05f770 msgr2=0x7f98fc0b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:36.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.657+0000 7f9902ec3700 1 -- 172.21.15.36:0/1961045989 shutdown_connections 2022-06-17T12:50:36.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.657+0000 7f9902ec3700 1 -- 172.21.15.36:0/1961045989 wait complete. 2022-06-17T12:50:36.648 INFO:teuthology.orchestra.run.smithi036.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-17T12:50:36.659 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:50:36.674 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2022-06-17T12:50:36.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.793+0000 7f9b008e9700 1 Processor -- start 2022-06-17T12:50:36.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.793+0000 7f9b008e9700 1 -- start start 2022-06-17T12:50:36.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9b008e9700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9afc0b7b60 0x7f9afc0b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:36.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9b008e9700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9afc0616c0 0x7f9afc13cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:36.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9b008e9700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9afc0b7f40 con 0x7f9afc061eb0 2022-06-17T12:50:36.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9b008e9700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9afc0b3670 con 0x7f9afc0b7b60 2022-06-17T12:50:36.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9b008e9700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc1433b0 con 0x7f9afc0616c0 2022-06-17T12:50:36.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9afa59c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9afc0b7b60 0x7f9afc0b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:36.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9afa59c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9afc0b7b60 0x7f9afc0b3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33782/0 (socket says 172.21.15.36:33782) 2022-06-17T12:50:36.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9afa59c700 1 -- 172.21.15.36:0/40641322 learned_addr learned my addr 172.21.15.36:0/40641322 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:36.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9afad9d700 1 --2- 172.21.15.36:0/40641322 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9afc0616c0 0x7f9afc13cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:36.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9afa59c700 1 -- 172.21.15.36:0/40641322 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9afc0616c0 msgr2=0x7f9afc13cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:36.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9afa59c700 1 --2- 172.21.15.36:0/40641322 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9afc0616c0 0x7f9afc13cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:36.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9afa59c700 1 -- 172.21.15.36:0/40641322 >> v1:172.21.15.70:6789/0 conn(0x7f9afc061eb0 legacy=0x7f9afc143af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:36.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9afa59c700 1 -- 172.21.15.36:0/40641322 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc143650 con 0x7f9afc0b7b60 2022-06-17T12:50:36.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9afa59c700 1 --2- 172.21.15.36:0/40641322 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9afc0b7b60 0x7f9afc0b3e50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f9aec002f70 tx=0x7f9aec006eb0).ready entity=mon.0 client_cookie=e4f48d9e532c86a8 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:36.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9af9d9b700 1 -- 172.21.15.36:0/40641322 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9aec00d5c0 con 0x7f9afc0b7b60 2022-06-17T12:50:36.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9af9d9b700 1 -- 172.21.15.36:0/40641322 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aec000e90 con 0x7f9afc0b7b60 2022-06-17T12:50:36.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9af9d9b700 1 -- 172.21.15.36:0/40641322 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9aec00d950 con 0x7f9afc0b7b60 2022-06-17T12:50:36.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9b008e9700 1 -- 172.21.15.36:0/40641322 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9afc0b7b60 msgr2=0x7f9afc0b3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:36.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9b008e9700 1 --2- 172.21.15.36:0/40641322 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9afc0b7b60 0x7f9afc0b3e50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f9aec002f70 tx=0x7f9aec006eb0).stop 2022-06-17T12:50:36.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9b008e9700 1 -- 172.21.15.36:0/40641322 shutdown_connections 2022-06-17T12:50:36.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9b008e9700 1 --2- 172.21.15.36:0/40641322 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9afc0616c0 0x7f9afc13cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:36.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9b008e9700 1 --2- 172.21.15.36:0/40641322 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9afc0b7b60 0x7f9afc0b3e50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:36.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9b008e9700 1 -- 172.21.15.36:0/40641322 >> 172.21.15.36:0/40641322 conn(0x7f9afc060c60 msgr2=0x7f9afc05c550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:36.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9b008e9700 1 -- 172.21.15.36:0/40641322 shutdown_connections 2022-06-17T12:50:36.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.797+0000 7f9b008e9700 1 -- 172.21.15.36:0/40641322 wait complete. 2022-06-17T12:50:36.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.801+0000 7f9b008e9700 1 Processor -- start 2022-06-17T12:50:36.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.801+0000 7f9b008e9700 1 -- start start 2022-06-17T12:50:36.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.801+0000 7f9b008e9700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9afc0616c0 0x7f9afc146580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:36.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.805+0000 7f9b008e9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9afc061eb0 0x7f9afc145a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:36.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.805+0000 7f9b008e9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9afc0b7b60 0x7f9afc13be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:36.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.805+0000 7f9b008e9700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc13fe40 con 0x7f9afc061eb0 2022-06-17T12:50:36.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.805+0000 7f9b008e9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc1400e0 con 0x7f9afc0b7b60 2022-06-17T12:50:36.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.805+0000 7f9b008e9700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc140380 con 0x7f9afc0616c0 2022-06-17T12:50:36.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.805+0000 7f9af3fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9afc061eb0 0x7f9afc145a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:36.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.805+0000 7f9af3fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9afc061eb0 0x7f9afc145a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33790/0 (socket says 172.21.15.36:33790) 2022-06-17T12:50:36.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.805+0000 7f9af3fff700 1 -- 172.21.15.36:0/2258751820 learned_addr learned my addr 172.21.15.36:0/2258751820 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:36.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.805+0000 7f9afa59c700 1 --2- 172.21.15.36:0/2258751820 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9afc0616c0 0x7f9afc146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:36.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.805+0000 7f9afad9d700 1 --2- 172.21.15.36:0/2258751820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9afc0b7b60 0x7f9afc13be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:36.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.805+0000 7f9af3fff700 1 -- 172.21.15.36:0/2258751820 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9afc0616c0 msgr2=0x7f9afc146580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:36.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.805+0000 7f9af3fff700 1 --2- 172.21.15.36:0/2258751820 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9afc0616c0 0x7f9afc146580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:36.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.805+0000 7f9af3fff700 1 -- 172.21.15.36:0/2258751820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9afc0b7b60 msgr2=0x7f9afc13be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:36.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.805+0000 7f9af3fff700 1 --2- 172.21.15.36:0/2258751820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9afc0b7b60 0x7f9afc13be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:36.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.805+0000 7f9af3fff700 1 -- 172.21.15.36:0/2258751820 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aec00b1e0 con 0x7f9afc061eb0 2022-06-17T12:50:36.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.805+0000 7f9afad9d700 1 --2- 172.21.15.36:0/2258751820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9afc0b7b60 0x7f9afc13be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:50:36.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.805+0000 7f9af3fff700 1 --2- 172.21.15.36:0/2258751820 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9afc061eb0 0x7f9afc145a40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f9afc064480 tx=0x7f9ae4007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:36.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.805+0000 7f9af37fe700 1 -- 172.21.15.36:0/2258751820 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ae4012400 con 0x7f9afc061eb0 2022-06-17T12:50:36.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.805+0000 7f9af37fe700 1 -- 172.21.15.36:0/2258751820 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae4012560 con 0x7f9afc061eb0 2022-06-17T12:50:36.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.805+0000 7f9b008e9700 1 -- 172.21.15.36:0/2258751820 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9afc140820 con 0x7f9afc061eb0 2022-06-17T12:50:36.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.805+0000 7f9af37fe700 1 -- 172.21.15.36:0/2258751820 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ae40128c0 con 0x7f9afc061eb0 2022-06-17T12:50:36.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.805+0000 7f9b008e9700 1 -- 172.21.15.36:0/2258751820 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9afc0671c0 con 0x7f9afc061eb0 2022-06-17T12:50:36.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.805+0000 7f9b008e9700 1 -- 172.21.15.36:0/2258751820 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9afc13fca0 con 0x7f9afc061eb0 2022-06-17T12:50:36.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.805+0000 7f9af37fe700 1 -- 172.21.15.36:0/2258751820 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f9ae4019030 con 0x7f9afc061eb0 2022-06-17T12:50:36.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.809+0000 7f9af37fe700 1 --2- 172.21.15.36:0/2258751820 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9ae0053a40 0x7f9ae0055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:36.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.809+0000 7f9af37fe700 1 -- 172.21.15.36:0/2258751820 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f9ae406b410 con 0x7f9afc061eb0 2022-06-17T12:50:36.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.809+0000 7f9afa59c700 1 --2- 172.21.15.36:0/2258751820 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9ae0053a40 0x7f9ae0055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:36.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.809+0000 7f9afa59c700 1 --2- 172.21.15.36:0/2258751820 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9ae0053a40 0x7f9ae0055f00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f9aec01c000 tx=0x7f9aec015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:36.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.813+0000 7f9af37fe700 1 -- 172.21.15.36:0/2258751820 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9afc13fca0 con 0x7f9afc061eb0 2022-06-17T12:50:36.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.973+0000 7f9b008e9700 1 -- 172.21.15.36:0/2258751820 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9afc13fca0 con 0x7f9afc061eb0 2022-06-17T12:50:36.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.973+0000 7f9af37fe700 1 -- 172.21.15.36:0/2258751820 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+1777 (secure 0 0 0) 0x7f9afc13fca0 con 0x7f9afc061eb0 2022-06-17T12:50:36.961 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:50:36.961 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/3777038413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3777038413},{"type":"v1","addr":"172.21.15.36:6835","nonce":3777038413}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:49:14.023351+0000","modified":"2022-06-17T12:50:36.645063+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-06-17T12:50:36.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.977+0000 7f9b008e9700 1 -- 172.21.15.36:0/2258751820 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9ae0053a40 msgr2=0x7f9ae0055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:36.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.977+0000 7f9b008e9700 1 --2- 172.21.15.36:0/2258751820 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9ae0053a40 0x7f9ae0055f00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f9aec01c000 tx=0x7f9aec015000).stop 2022-06-17T12:50:36.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.977+0000 7f9b008e9700 1 -- 172.21.15.36:0/2258751820 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9afc061eb0 msgr2=0x7f9afc145a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:36.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.977+0000 7f9b008e9700 1 --2- 172.21.15.36:0/2258751820 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9afc061eb0 0x7f9afc145a40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f9afc064480 tx=0x7f9ae4007c50).stop 2022-06-17T12:50:36.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.977+0000 7f9b008e9700 1 -- 172.21.15.36:0/2258751820 shutdown_connections 2022-06-17T12:50:36.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.977+0000 7f9b008e9700 1 --2- 172.21.15.36:0/2258751820 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9ae0053a40 0x7f9ae0055f00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:36.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.977+0000 7f9b008e9700 1 --2- 172.21.15.36:0/2258751820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9afc0b7b60 0x7f9afc13be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:36.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.977+0000 7f9b008e9700 1 --2- 172.21.15.36:0/2258751820 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9afc061eb0 0x7f9afc145a40 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:36.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.977+0000 7f9b008e9700 1 --2- 172.21.15.36:0/2258751820 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9afc0616c0 0x7f9afc146580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:36.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.977+0000 7f9b008e9700 1 -- 172.21.15.36:0/2258751820 >> 172.21.15.36:0/2258751820 conn(0x7f9afc060c60 msgr2=0x7f9afc140f90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:36.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.977+0000 7f9b008e9700 1 -- 172.21.15.36:0/2258751820 shutdown_connections 2022-06-17T12:50:36.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:36.977+0000 7f9b008e9700 1 -- 172.21.15.36:0/2258751820 wait complete. 2022-06-17T12:50:36.967 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 20 2022-06-17T12:50:36.973 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-17T12:50:37.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.097+0000 7f61d759e700 1 Processor -- start 2022-06-17T12:50:37.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.097+0000 7f61d759e700 1 -- start start 2022-06-17T12:50:37.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.097+0000 7f61d759e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f61d80b9860 0x7f61d8143840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:37.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.097+0000 7f61d759e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f61d80b45c0 0x7f61d80b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:37.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.097+0000 7f61d759e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61d80b4f00 0x7f61d8146490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:37.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.097+0000 7f61d759e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d80b4160 con 0x7f61d80b45c0 2022-06-17T12:50:37.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.097+0000 7f61d759e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d80b9c40 con 0x7f61d80b9860 2022-06-17T12:50:37.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.097+0000 7f61d759e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d80627d0 con 0x7f61d80b4f00 2022-06-17T12:50:37.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d5d9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f61d80b45c0 0x7f61d80b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:37.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d5d9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f61d80b45c0 0x7f61d80b4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33798/0 (socket says 172.21.15.36:33798) 2022-06-17T12:50:37.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d5d9b700 1 -- 172.21.15.36:0/2212104318 learned_addr learned my addr 172.21.15.36:0/2212104318 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:37.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d659c700 1 --2- 172.21.15.36:0/2212104318 >> v2:172.21.15.70:3300/0 conn(0x7f61d80b9860 0x7f61d8143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:37.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d6d9d700 1 --2- 172.21.15.36:0/2212104318 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61d80b4f00 0x7f61d8146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:37.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d5d9b700 1 -- 172.21.15.36:0/2212104318 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61d80b4f00 msgr2=0x7f61d8146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:37.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d5d9b700 1 --2- 172.21.15.36:0/2212104318 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61d80b4f00 0x7f61d8146490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:37.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d5d9b700 1 -- 172.21.15.36:0/2212104318 >> v2:172.21.15.70:3300/0 conn(0x7f61d80b9860 msgr2=0x7f61d8143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:37.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d5d9b700 1 --2- 172.21.15.36:0/2212104318 >> v2:172.21.15.70:3300/0 conn(0x7f61d80b9860 0x7f61d8143840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:37.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d5d9b700 1 -- 172.21.15.36:0/2212104318 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d8062a70 con 0x7f61d80b45c0 2022-06-17T12:50:37.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d659c700 1 --2- 172.21.15.36:0/2212104318 >> v2:172.21.15.70:3300/0 conn(0x7f61d80b9860 0x7f61d8143840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:50:37.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d5d9b700 1 --2- 172.21.15.36:0/2212104318 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61d80b45c0 0x7f61d80b4a10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f61c0002f70 tx=0x7f61c0006eb0).ready entity=mon.0 client_cookie=7d9bcaaa07bf5c3a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:37.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d559a700 1 -- 172.21.15.36:0/2212104318 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61c000d570 con 0x7f61d80b45c0 2022-06-17T12:50:37.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d559a700 1 -- 172.21.15.36:0/2212104318 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c0000e90 con 0x7f61d80b45c0 2022-06-17T12:50:37.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d559a700 1 -- 172.21.15.36:0/2212104318 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61c000d900 con 0x7f61d80b45c0 2022-06-17T12:50:37.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d759e700 1 -- 172.21.15.36:0/2212104318 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61d80b45c0 msgr2=0x7f61d80b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:37.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d759e700 1 --2- 172.21.15.36:0/2212104318 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61d80b45c0 0x7f61d80b4a10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f61c0002f70 tx=0x7f61c0006eb0).stop 2022-06-17T12:50:37.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d759e700 1 -- 172.21.15.36:0/2212104318 shutdown_connections 2022-06-17T12:50:37.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d759e700 1 --2- 172.21.15.36:0/2212104318 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61d80b4f00 0x7f61d8146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:37.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d759e700 1 --2- 172.21.15.36:0/2212104318 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61d80b45c0 0x7f61d80b4a10 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:37.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d759e700 1 --2- 172.21.15.36:0/2212104318 >> v2:172.21.15.70:3300/0 conn(0x7f61d80b9860 0x7f61d8143840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:37.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d759e700 1 -- 172.21.15.36:0/2212104318 >> 172.21.15.36:0/2212104318 conn(0x7f61d805c550 msgr2=0x7f61d8060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:37.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d759e700 1 -- 172.21.15.36:0/2212104318 shutdown_connections 2022-06-17T12:50:37.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d759e700 1 -- 172.21.15.36:0/2212104318 wait complete. 2022-06-17T12:50:37.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d759e700 1 Processor -- start 2022-06-17T12:50:37.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.101+0000 7f61d759e700 1 -- start start 2022-06-17T12:50:37.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61d759e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61d80b45c0 0x7f61d813dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:37.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61d759e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61d80b4f00 0x7f61d81524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:37.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61d759e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61d80b9860 0x7f61d81529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:37.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61d759e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d813f220 con 0x7f61d80b4f00 2022-06-17T12:50:37.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61d759e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d813efb0 con 0x7f61d80b9860 2022-06-17T12:50:37.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61d759e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d813d0e0 con 0x7f61d80b45c0 2022-06-17T12:50:37.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61d5d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61d80b4f00 0x7f61d81524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:37.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61d5d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61d80b4f00 0x7f61d81524e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33802/0 (socket says 172.21.15.36:33802) 2022-06-17T12:50:37.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61d5d9b700 1 -- 172.21.15.36:0/1330915223 learned_addr learned my addr 172.21.15.36:0/1330915223 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:37.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61d6d9d700 1 --2- 172.21.15.36:0/1330915223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61d80b9860 0x7f61d81529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:37.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61d659c700 1 --2- 172.21.15.36:0/1330915223 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61d80b45c0 0x7f61d813dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:37.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61d5d9b700 1 -- 172.21.15.36:0/1330915223 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61d80b45c0 msgr2=0x7f61d813dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:37.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61d5d9b700 1 --2- 172.21.15.36:0/1330915223 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61d80b45c0 0x7f61d813dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:37.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61d5d9b700 1 -- 172.21.15.36:0/1330915223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61d80b9860 msgr2=0x7f61d81529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:37.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61d5d9b700 1 --2- 172.21.15.36:0/1330915223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61d80b9860 0x7f61d81529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:37.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61d5d9b700 1 -- 172.21.15.36:0/1330915223 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c000b1e0 con 0x7f61d80b4f00 2022-06-17T12:50:37.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61d6d9d700 1 --2- 172.21.15.36:0/1330915223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61d80b9860 0x7f61d81529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:50:37.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61d5d9b700 1 --2- 172.21.15.36:0/1330915223 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61d80b4f00 0x7f61d81524e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f61d813f140 tx=0x7f61c0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:37.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61cf7fe700 1 -- 172.21.15.36:0/1330915223 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61c0006b40 con 0x7f61d80b4f00 2022-06-17T12:50:37.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61cf7fe700 1 -- 172.21.15.36:0/1330915223 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c0011ca0 con 0x7f61d80b4f00 2022-06-17T12:50:37.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61d759e700 1 -- 172.21.15.36:0/1330915223 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d8152040 con 0x7f61d80b4f00 2022-06-17T12:50:37.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61cf7fe700 1 -- 172.21.15.36:0/1330915223 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61c001e460 con 0x7f61d80b4f00 2022-06-17T12:50:37.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61d759e700 1 -- 172.21.15.36:0/1330915223 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d80afc40 con 0x7f61d80b4f00 2022-06-17T12:50:37.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61d759e700 1 -- 172.21.15.36:0/1330915223 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d8153060 con 0x7f61d80b4f00 2022-06-17T12:50:37.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.105+0000 7f61cf7fe700 1 -- 172.21.15.36:0/1330915223 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f61c0025030 con 0x7f61d80b4f00 2022-06-17T12:50:37.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.109+0000 7f61cf7fe700 1 --2- 172.21.15.36:0/1330915223 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f61bc0539b0 0x7f61bc055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:37.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.109+0000 7f61cf7fe700 1 -- 172.21.15.36:0/1330915223 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f61c006d260 con 0x7f61d80b4f00 2022-06-17T12:50:37.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.109+0000 7f61d659c700 1 --2- 172.21.15.36:0/1330915223 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f61bc0539b0 0x7f61bc055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:37.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.109+0000 7f61d659c700 1 --2- 172.21.15.36:0/1330915223 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f61bc0539b0 0x7f61bc055e70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f61c800c000 tx=0x7f61c8008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:37.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.113+0000 7f61cf7fe700 1 -- 172.21.15.36:0/1330915223 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f61c0044010 con 0x7f61d80b4f00 2022-06-17T12:50:37.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.281+0000 7f61d759e700 1 -- 172.21.15.36:0/1330915223 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f61d8062eb0 con 0x7f61d80b4f00 2022-06-17T12:50:37.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.657+0000 7f61cf7fe700 1 -- 172.21.15.36:0/1330915223 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f61c0047630 con 0x7f61d80b4f00 2022-06-17T12:50:37.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.661+0000 7f61d759e700 1 -- 172.21.15.36:0/1330915223 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f61bc0539b0 msgr2=0x7f61bc055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:37.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.661+0000 7f61d759e700 1 --2- 172.21.15.36:0/1330915223 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f61bc0539b0 0x7f61bc055e70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f61c800c000 tx=0x7f61c8008040).stop 2022-06-17T12:50:37.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.661+0000 7f61d759e700 1 -- 172.21.15.36:0/1330915223 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61d80b4f00 msgr2=0x7f61d81524e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:37.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.661+0000 7f61d759e700 1 --2- 172.21.15.36:0/1330915223 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61d80b4f00 0x7f61d81524e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f61d813f140 tx=0x7f61c0015000).stop 2022-06-17T12:50:37.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.661+0000 7f61d759e700 1 -- 172.21.15.36:0/1330915223 shutdown_connections 2022-06-17T12:50:37.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.661+0000 7f61d759e700 1 --2- 172.21.15.36:0/1330915223 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f61bc0539b0 0x7f61bc055e70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:37.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.661+0000 7f61d759e700 1 --2- 172.21.15.36:0/1330915223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61d80b9860 0x7f61d81529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:37.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.661+0000 7f61d759e700 1 --2- 172.21.15.36:0/1330915223 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61d80b4f00 0x7f61d81524e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:37.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.661+0000 7f61d759e700 1 --2- 172.21.15.36:0/1330915223 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61d80b45c0 0x7f61d813dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:37.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.661+0000 7f61d759e700 1 -- 172.21.15.36:0/1330915223 >> 172.21.15.36:0/1330915223 conn(0x7f61d805c550 msgr2=0x7f61d80b8510 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:37.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.661+0000 7f61d759e700 1 -- 172.21.15.36:0/1330915223 shutdown_connections 2022-06-17T12:50:37.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.661+0000 7f61d759e700 1 -- 172.21.15.36:0/1330915223 wait complete. 2022-06-17T12:50:37.664 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-17T12:50:37.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.817+0000 7f6d5b59e700 1 Processor -- start 2022-06-17T12:50:37.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.817+0000 7f6d5b59e700 1 -- start start 2022-06-17T12:50:37.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.817+0000 7f6d5b59e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f6d5c063730 0x7f6d5c0af380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:37.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.817+0000 7f6d5b59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d5c143e70 0x7f6d5c128b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:37.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.817+0000 7f6d5b59e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d5c063b10 con 0x7f6d5c0b2430 2022-06-17T12:50:37.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.817+0000 7f6d5b59e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d5c0ada90 con 0x7f6d5c063730 2022-06-17T12:50:37.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.817+0000 7f6d5b59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c0b3d50 con 0x7f6d5c143e70 2022-06-17T12:50:37.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.817+0000 7f6d5a59c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f6d5c063730 0x7f6d5c0af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:37.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.817+0000 7f6d5a59c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f6d5c063730 0x7f6d5c0af380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33810/0 (socket says 172.21.15.36:33810) 2022-06-17T12:50:37.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.817+0000 7f6d5a59c700 1 -- 172.21.15.36:0/3879502296 learned_addr learned my addr 172.21.15.36:0/3879502296 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:37.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.817+0000 7f6d59d9b700 1 --1- 172.21.15.36:0/3879502296 >> v1:172.21.15.36:6789/0 conn(0x7f6d5c0b2430 0x7f6d5c143760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51342/0 (socket says 172.21.15.36:51342) 2022-06-17T12:50:37.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.817+0000 7f6d5ad9d700 1 --2- 172.21.15.36:0/3879502296 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d5c143e70 0x7f6d5c128b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:37.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.817+0000 7f6d5959a700 1 -- 172.21.15.36:0/3879502296 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4102224738 0 0) 0x7f6d5c063b10 con 0x7f6d5c0b2430 2022-06-17T12:50:37.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.817+0000 7f6d5a59c700 1 -- 172.21.15.36:0/3879502296 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d5c143e70 msgr2=0x7f6d5c128b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:37.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.817+0000 7f6d5a59c700 1 --2- 172.21.15.36:0/3879502296 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d5c143e70 0x7f6d5c128b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:37.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.817+0000 7f6d5a59c700 1 -- 172.21.15.36:0/3879502296 >> v1:172.21.15.36:6789/0 conn(0x7f6d5c0b2430 legacy=0x7f6d5c143760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:37.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.817+0000 7f6d5a59c700 1 -- 172.21.15.36:0/3879502296 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5c0af0e0 con 0x7f6d5c063730 2022-06-17T12:50:37.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.817+0000 7f6d5ad9d700 1 --2- 172.21.15.36:0/3879502296 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d5c143e70 0x7f6d5c128b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:50:37.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.817+0000 7f6d5a59c700 1 --2- 172.21.15.36:0/3879502296 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d5c063730 0x7f6d5c0af380 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f6d4c0032f0 tx=0x7f6d4c00b530).ready entity=mon.0 client_cookie=33aa5c7d9c93ad12 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:37.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.817+0000 7f6d5959a700 1 -- 172.21.15.36:0/3879502296 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d4c010660 con 0x7f6d5c063730 2022-06-17T12:50:37.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.817+0000 7f6d5959a700 1 -- 172.21.15.36:0/3879502296 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4c0107c0 con 0x7f6d5c063730 2022-06-17T12:50:37.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.817+0000 7f6d5959a700 1 -- 172.21.15.36:0/3879502296 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d4c010b20 con 0x7f6d5c063730 2022-06-17T12:50:37.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.821+0000 7f6d5b59e700 1 -- 172.21.15.36:0/3879502296 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d5c063730 msgr2=0x7f6d5c0af380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:37.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.821+0000 7f6d5b59e700 1 --2- 172.21.15.36:0/3879502296 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d5c063730 0x7f6d5c0af380 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f6d4c0032f0 tx=0x7f6d4c00b530).stop 2022-06-17T12:50:37.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.821+0000 7f6d5b59e700 1 -- 172.21.15.36:0/3879502296 shutdown_connections 2022-06-17T12:50:37.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.821+0000 7f6d5b59e700 1 --2- 172.21.15.36:0/3879502296 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d5c143e70 0x7f6d5c128b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:37.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.821+0000 7f6d5b59e700 1 --2- 172.21.15.36:0/3879502296 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d5c063730 0x7f6d5c0af380 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:37.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.821+0000 7f6d5b59e700 1 -- 172.21.15.36:0/3879502296 >> 172.21.15.36:0/3879502296 conn(0x7f6d5c060880 msgr2=0x7f6d5c0614b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:37.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.821+0000 7f6d5b59e700 1 -- 172.21.15.36:0/3879502296 shutdown_connections 2022-06-17T12:50:37.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.821+0000 7f6d5b59e700 1 -- 172.21.15.36:0/3879502296 wait complete. 2022-06-17T12:50:37.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.821+0000 7f6d5b59e700 1 Processor -- start 2022-06-17T12:50:37.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.821+0000 7f6d5b59e700 1 -- start start 2022-06-17T12:50:37.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.821+0000 7f6d5b59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d5c063730 0x7f6d5c128c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:37.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.821+0000 7f6d5b59e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d5c0b2430 0x7f6d5c1554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:37.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.821+0000 7f6d5b59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d5c143e70 0x7f6d5c1559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:37.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.821+0000 7f6d5b59e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c152e40 con 0x7f6d5c143e70 2022-06-17T12:50:37.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.821+0000 7f6d5b59e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c152920 con 0x7f6d5c0b2430 2022-06-17T12:50:37.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.821+0000 7f6d5b59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c152ba0 con 0x7f6d5c063730 2022-06-17T12:50:37.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.821+0000 7f6d5ad9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d5c143e70 0x7f6d5c1559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:37.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.821+0000 7f6d5ad9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d5c143e70 0x7f6d5c1559b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33820/0 (socket says 172.21.15.36:33820) 2022-06-17T12:50:37.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.821+0000 7f6d5ad9d700 1 -- 172.21.15.36:0/2177103088 learned_addr learned my addr 172.21.15.36:0/2177103088 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:37.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.821+0000 7f6d59d9b700 1 --2- 172.21.15.36:0/2177103088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d5c0b2430 0x7f6d5c1554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:37.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.821+0000 7f6d5a59c700 1 --2- 172.21.15.36:0/2177103088 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d5c063730 0x7f6d5c128c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:37.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.825+0000 7f6d5ad9d700 1 -- 172.21.15.36:0/2177103088 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d5c063730 msgr2=0x7f6d5c128c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:37.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.825+0000 7f6d5ad9d700 1 --2- 172.21.15.36:0/2177103088 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d5c063730 0x7f6d5c128c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:37.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.825+0000 7f6d5ad9d700 1 -- 172.21.15.36:0/2177103088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d5c0b2430 msgr2=0x7f6d5c1554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:37.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.825+0000 7f6d5ad9d700 1 --2- 172.21.15.36:0/2177103088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d5c0b2430 0x7f6d5c1554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:37.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.825+0000 7f6d5ad9d700 1 -- 172.21.15.36:0/2177103088 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4c00b7d0 con 0x7f6d5c143e70 2022-06-17T12:50:37.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.825+0000 7f6d59d9b700 1 --2- 172.21.15.36:0/2177103088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d5c0b2430 0x7f6d5c1554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:50:37.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.825+0000 7f6d5a59c700 1 --2- 172.21.15.36:0/2177103088 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d5c063730 0x7f6d5c128c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:50:37.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.825+0000 7f6d5ad9d700 1 --2- 172.21.15.36:0/2177103088 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d5c143e70 0x7f6d5c1559b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f6d54006940 tx=0x7f6d54008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:37.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.825+0000 7f6d4b7fe700 1 -- 172.21.15.36:0/2177103088 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d54011780 con 0x7f6d5c143e70 2022-06-17T12:50:37.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.825+0000 7f6d4b7fe700 1 -- 172.21.15.36:0/2177103088 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d540118e0 con 0x7f6d5c143e70 2022-06-17T12:50:37.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.825+0000 7f6d5b59e700 1 -- 172.21.15.36:0/2177103088 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d5c153020 con 0x7f6d5c143e70 2022-06-17T12:50:37.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.825+0000 7f6d4b7fe700 1 -- 172.21.15.36:0/2177103088 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d54011c40 con 0x7f6d5c143e70 2022-06-17T12:50:37.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.825+0000 7f6d5b59e700 1 -- 172.21.15.36:0/2177103088 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d5c13ec90 con 0x7f6d5c143e70 2022-06-17T12:50:37.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.825+0000 7f6d5b59e700 1 -- 172.21.15.36:0/2177103088 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d5c1432c0 con 0x7f6d5c143e70 2022-06-17T12:50:37.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.825+0000 7f6d4b7fe700 1 -- 172.21.15.36:0/2177103088 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6d5401c460 con 0x7f6d5c143e70 2022-06-17T12:50:37.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.829+0000 7f6d4b7fe700 1 --2- 172.21.15.36:0/2177103088 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6d38053a60 0x7f6d38055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:37.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.829+0000 7f6d4b7fe700 1 -- 172.21.15.36:0/2177103088 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f6d5406cf30 con 0x7f6d5c143e70 2022-06-17T12:50:37.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.829+0000 7f6d5a59c700 1 --2- 172.21.15.36:0/2177103088 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6d38053a60 0x7f6d38055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:37.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.829+0000 7f6d5a59c700 1 --2- 172.21.15.36:0/2177103088 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6d38053a60 0x7f6d38055f20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f6d4c019000 tx=0x7f6d5c0ae3c0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:37.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.833+0000 7f6d4b7fe700 1 -- 172.21.15.36:0/2177103088 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6d54043c80 con 0x7f6d5c143e70 2022-06-17T12:50:37.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:37.965+0000 7f6d5b59e700 1 -- 172.21.15.36:0/2177103088 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6d5c057410 con 0x7f6d5c143e70 2022-06-17T12:50:38.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.713+0000 7f6d4b7fe700 1 -- 172.21.15.36:0/2177103088 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v47) v1 ==== 185+0+0 (secure 0 0 0) 0x7f6d540472a0 con 0x7f6d5c143e70 2022-06-17T12:50:38.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.717+0000 7f6d5b59e700 1 -- 172.21.15.36:0/2177103088 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6d38053a60 msgr2=0x7f6d38055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:38.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.717+0000 7f6d5b59e700 1 --2- 172.21.15.36:0/2177103088 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6d38053a60 0x7f6d38055f20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f6d4c019000 tx=0x7f6d5c0ae3c0).stop 2022-06-17T12:50:38.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.717+0000 7f6d5b59e700 1 -- 172.21.15.36:0/2177103088 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d5c143e70 msgr2=0x7f6d5c1559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:38.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.717+0000 7f6d5b59e700 1 --2- 172.21.15.36:0/2177103088 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d5c143e70 0x7f6d5c1559b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f6d54006940 tx=0x7f6d54008040).stop 2022-06-17T12:50:38.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.717+0000 7f6d5b59e700 1 -- 172.21.15.36:0/2177103088 shutdown_connections 2022-06-17T12:50:38.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.717+0000 7f6d5b59e700 1 --2- 172.21.15.36:0/2177103088 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6d38053a60 0x7f6d38055f20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:38.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.717+0000 7f6d5b59e700 1 --2- 172.21.15.36:0/2177103088 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d5c143e70 0x7f6d5c1559b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:38.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.717+0000 7f6d5b59e700 1 --2- 172.21.15.36:0/2177103088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d5c0b2430 0x7f6d5c1554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:38.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.717+0000 7f6d5b59e700 1 --2- 172.21.15.36:0/2177103088 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d5c063730 0x7f6d5c128c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:38.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.717+0000 7f6d5b59e700 1 -- 172.21.15.36:0/2177103088 >> 172.21.15.36:0/2177103088 conn(0x7f6d5c060880 msgr2=0x7f6d5c0614b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:38.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.717+0000 7f6d5b59e700 1 -- 172.21.15.36:0/2177103088 shutdown_connections 2022-06-17T12:50:38.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.717+0000 7f6d5b59e700 1 -- 172.21.15.36:0/2177103088 wait complete. 2022-06-17T12:50:38.707 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' removed 2022-06-17T12:50:38.718 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-17T12:50:38.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.861+0000 7ff0ab1f2700 1 Processor -- start 2022-06-17T12:50:38.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.861+0000 7ff0ab1f2700 1 -- start start 2022-06-17T12:50:38.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.861+0000 7ff0ab1f2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff0a40b8020 0x7ff0a4062830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:38.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.861+0000 7ff0ab1f2700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff0a40b3a40 0x7ff0a4062d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:38.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.861+0000 7ff0ab1f2700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0a40b8400 con 0x7ff0a4063210 2022-06-17T12:50:38.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.861+0000 7ff0ab1f2700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a40b21b0 con 0x7ff0a40b3a40 2022-06-17T12:50:38.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.861+0000 7ff0ab1f2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a4067a90 con 0x7ff0a40b8020 2022-06-17T12:50:38.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.861+0000 7ff0a99ef700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff0a40b3a40 0x7ff0a4062d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:38.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.865+0000 7ff0a99ef700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff0a40b3a40 0x7ff0a4062d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33828/0 (socket says 172.21.15.36:33828) 2022-06-17T12:50:38.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.865+0000 7ff0a99ef700 1 -- 172.21.15.36:0/71691638 learned_addr learned my addr 172.21.15.36:0/71691638 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:38.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.865+0000 7ff0aa1f0700 1 --2- 172.21.15.36:0/71691638 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff0a40b8020 0x7ff0a4062830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:38.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.865+0000 7ff0a99ef700 1 -- 172.21.15.36:0/71691638 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff0a40b8020 msgr2=0x7ff0a4062830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:38.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.865+0000 7ff0a99ef700 1 --2- 172.21.15.36:0/71691638 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff0a40b8020 0x7ff0a4062830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:38.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.865+0000 7ff0a99ef700 1 -- 172.21.15.36:0/71691638 >> v1:172.21.15.70:6789/0 conn(0x7ff0a4063210 legacy=0x7ff0a40b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:38.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.865+0000 7ff0a99ef700 1 -- 172.21.15.36:0/71691638 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a4146420 con 0x7ff0a40b3a40 2022-06-17T12:50:38.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.865+0000 7ff0a91ee700 1 -- 172.21.15.36:0/71691638 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1027154398 0 0) 0x7ff0a40b8400 con 0x7ff0a4063210 2022-06-17T12:50:38.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.865+0000 7ff0aa1f0700 1 --2- 172.21.15.36:0/71691638 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff0a40b8020 0x7ff0a4062830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:50:38.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.865+0000 7ff0a99ef700 1 --2- 172.21.15.36:0/71691638 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff0a40b3a40 0x7ff0a4062d20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7ff094002f70 tx=0x7ff094006eb0).ready entity=mon.0 client_cookie=4e6fa89ecf917204 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:38.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.865+0000 7ff0a91ee700 1 -- 172.21.15.36:0/71691638 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff09400d5c0 con 0x7ff0a40b3a40 2022-06-17T12:50:38.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.865+0000 7ff0a91ee700 1 -- 172.21.15.36:0/71691638 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff094000e90 con 0x7ff0a40b3a40 2022-06-17T12:50:38.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.865+0000 7ff0a91ee700 1 -- 172.21.15.36:0/71691638 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff09400d910 con 0x7ff0a40b3a40 2022-06-17T12:50:38.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.865+0000 7ff0ab1f2700 1 -- 172.21.15.36:0/71691638 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff0a40b3a40 msgr2=0x7ff0a4062d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:38.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.865+0000 7ff0ab1f2700 1 --2- 172.21.15.36:0/71691638 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff0a40b3a40 0x7ff0a4062d20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7ff094002f70 tx=0x7ff094006eb0).stop 2022-06-17T12:50:38.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.865+0000 7ff0ab1f2700 1 -- 172.21.15.36:0/71691638 shutdown_connections 2022-06-17T12:50:38.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.865+0000 7ff0ab1f2700 1 --2- 172.21.15.36:0/71691638 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff0a40b3a40 0x7ff0a4062d20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:38.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.865+0000 7ff0ab1f2700 1 --2- 172.21.15.36:0/71691638 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff0a40b8020 0x7ff0a4062830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:38.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.865+0000 7ff0ab1f2700 1 -- 172.21.15.36:0/71691638 >> 172.21.15.36:0/71691638 conn(0x7ff0a405c550 msgr2=0x7ff0a40b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:38.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.865+0000 7ff0ab1f2700 1 -- 172.21.15.36:0/71691638 shutdown_connections 2022-06-17T12:50:38.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.865+0000 7ff0ab1f2700 1 -- 172.21.15.36:0/71691638 wait complete. 2022-06-17T12:50:38.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.869+0000 7ff0ab1f2700 1 Processor -- start 2022-06-17T12:50:38.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.869+0000 7ff0ab1f2700 1 -- start start 2022-06-17T12:50:38.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.869+0000 7ff0ab1f2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff0a4063210 0x7ff0a4066f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:38.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.869+0000 7ff0ab1f2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0a40b3a40 0x7ff0a40645a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:38.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.869+0000 7ff0ab1f2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff0a40b8020 0x7ff0a41424e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:38.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.869+0000 7ff0ab1f2700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a4152f80 con 0x7ff0a40b8020 2022-06-17T12:50:38.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.869+0000 7ff0ab1f2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a4152b20 con 0x7ff0a40b3a40 2022-06-17T12:50:38.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.869+0000 7ff0ab1f2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a4066aa0 con 0x7ff0a4063210 2022-06-17T12:50:38.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.869+0000 7ff0aa9f1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff0a40b8020 0x7ff0a41424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:38.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.869+0000 7ff0aa9f1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff0a40b8020 0x7ff0a41424e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33834/0 (socket says 172.21.15.36:33834) 2022-06-17T12:50:38.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.869+0000 7ff0aa9f1700 1 -- 172.21.15.36:0/3165550609 learned_addr learned my addr 172.21.15.36:0/3165550609 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:38.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.869+0000 7ff0aa1f0700 1 --2- 172.21.15.36:0/3165550609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff0a4063210 0x7ff0a4066f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:38.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.869+0000 7ff0a99ef700 1 --2- 172.21.15.36:0/3165550609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0a40b3a40 0x7ff0a40645a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:38.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.869+0000 7ff0aa9f1700 1 -- 172.21.15.36:0/3165550609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff0a4063210 msgr2=0x7ff0a4066f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:38.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.869+0000 7ff0aa9f1700 1 --2- 172.21.15.36:0/3165550609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff0a4063210 0x7ff0a4066f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:38.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.869+0000 7ff0aa9f1700 1 -- 172.21.15.36:0/3165550609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0a40b3a40 msgr2=0x7ff0a40645a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:38.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.869+0000 7ff0aa9f1700 1 --2- 172.21.15.36:0/3165550609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0a40b3a40 0x7ff0a40645a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:38.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.869+0000 7ff0aa9f1700 1 -- 172.21.15.36:0/3165550609 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09400b1e0 con 0x7ff0a40b8020 2022-06-17T12:50:38.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.869+0000 7ff0aa9f1700 1 --2- 172.21.15.36:0/3165550609 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff0a40b8020 0x7ff0a41424e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7ff09c000f40 tx=0x7ff09c007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:38.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.869+0000 7ff0937fe700 1 -- 172.21.15.36:0/3165550609 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff09c006aa0 con 0x7ff0a40b8020 2022-06-17T12:50:38.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.869+0000 7ff0937fe700 1 -- 172.21.15.36:0/3165550609 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09c006c00 con 0x7ff0a40b8020 2022-06-17T12:50:38.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.869+0000 7ff0ab1f2700 1 -- 172.21.15.36:0/3165550609 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a4066800 con 0x7ff0a40b8020 2022-06-17T12:50:38.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.869+0000 7ff0937fe700 1 -- 172.21.15.36:0/3165550609 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff09c007030 con 0x7ff0a40b8020 2022-06-17T12:50:38.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.869+0000 7ff0ab1f2700 1 -- 172.21.15.36:0/3165550609 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a40b7230 con 0x7ff0a40b8020 2022-06-17T12:50:38.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.873+0000 7ff0ab1f2700 1 -- 172.21.15.36:0/3165550609 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a4152c90 con 0x7ff0a40b8020 2022-06-17T12:50:38.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.873+0000 7ff0937fe700 1 -- 172.21.15.36:0/3165550609 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff09c010030 con 0x7ff0a40b8020 2022-06-17T12:50:38.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.873+0000 7ff0937fe700 1 --2- 172.21.15.36:0/3165550609 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff084053a40 0x7ff084055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:38.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.873+0000 7ff0937fe700 1 -- 172.21.15.36:0/3165550609 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6068+0+0 (secure 0 0 0) 0x7ff09c06bdb0 con 0x7ff0a40b8020 2022-06-17T12:50:38.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.873+0000 7ff0aa1f0700 1 --2- 172.21.15.36:0/3165550609 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff084053a40 0x7ff084055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:38.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.873+0000 7ff0aa1f0700 1 --2- 172.21.15.36:0/3165550609 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff084053a40 0x7ff084055f00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7ff0a000c000 tx=0x7ff0a0008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:38.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:38.881+0000 7ff0937fe700 1 -- 172.21.15.36:0/3165550609 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff09c042d10 con 0x7ff0a40b8020 2022-06-17T12:50:38.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.009+0000 7ff0ab1f2700 1 -- 172.21.15.36:0/3165550609 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff0a4057410 con 0x7ff0a40b8020 2022-06-17T12:50:39.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.721+0000 7ff0937fe700 1 -- 172.21.15.36:0/3165550609 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v48) v1 ==== 173+0+0 (secure 0 0 0) 0x7ff09c046330 con 0x7ff0a40b8020 2022-06-17T12:50:39.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.729+0000 7ff0ab1f2700 1 -- 172.21.15.36:0/3165550609 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff084053a40 msgr2=0x7ff084055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:39.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.729+0000 7ff0ab1f2700 1 --2- 172.21.15.36:0/3165550609 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff084053a40 0x7ff084055f00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7ff0a000c000 tx=0x7ff0a0008040).stop 2022-06-17T12:50:39.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.729+0000 7ff0ab1f2700 1 -- 172.21.15.36:0/3165550609 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff0a40b8020 msgr2=0x7ff0a41424e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:39.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.729+0000 7ff0ab1f2700 1 --2- 172.21.15.36:0/3165550609 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff0a40b8020 0x7ff0a41424e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7ff09c000f40 tx=0x7ff09c007f90).stop 2022-06-17T12:50:39.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.729+0000 7ff0ab1f2700 1 -- 172.21.15.36:0/3165550609 shutdown_connections 2022-06-17T12:50:39.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.729+0000 7ff0ab1f2700 1 --2- 172.21.15.36:0/3165550609 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff084053a40 0x7ff084055f00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:39.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.729+0000 7ff0ab1f2700 1 --2- 172.21.15.36:0/3165550609 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff0a40b8020 0x7ff0a41424e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:39.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.729+0000 7ff0ab1f2700 1 --2- 172.21.15.36:0/3165550609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0a40b3a40 0x7ff0a40645a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:39.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.729+0000 7ff0ab1f2700 1 --2- 172.21.15.36:0/3165550609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff0a4063210 0x7ff0a4066f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:39.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.729+0000 7ff0ab1f2700 1 -- 172.21.15.36:0/3165550609 >> 172.21.15.36:0/3165550609 conn(0x7ff0a405c550 msgr2=0x7ff0a41436d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:39.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.729+0000 7ff0ab1f2700 1 -- 172.21.15.36:0/3165550609 shutdown_connections 2022-06-17T12:50:39.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.729+0000 7ff0ab1f2700 1 -- 172.21.15.36:0/3165550609 wait complete. 2022-06-17T12:50:39.721 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' removed 2022-06-17T12:50:39.735 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-06-17T12:50:39.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.861+0000 7ff476f09700 1 Processor -- start 2022-06-17T12:50:39.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.861+0000 7ff476f09700 1 -- start start 2022-06-17T12:50:39.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff476f09700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff4700b4190 0x7ff4700b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:39.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff476f09700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff470143d40 0x7ff4701471d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:39.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff476f09700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff470143130 con 0x7ff470062270 2022-06-17T12:50:39.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff476f09700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff470144140 con 0x7ff4700b4190 2022-06-17T12:50:39.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff476f09700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4700b6080 con 0x7ff470143d40 2022-06-17T12:50:39.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff46ffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff4700b4190 0x7ff4700b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:39.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff46ffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff4700b4190 0x7ff4700b4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33838/0 (socket says 172.21.15.36:33838) 2022-06-17T12:50:39.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff46ffff700 1 -- 172.21.15.36:0/3421369225 learned_addr learned my addr 172.21.15.36:0/3421369225 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:39.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff474ca5700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7ff470062270 0x7ff4700b48b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51368/0 (socket says 172.21.15.36:51368) 2022-06-17T12:50:39.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff4754a6700 1 --2- 172.21.15.36:0/3421369225 >> v2:172.21.15.70:3300/0 conn(0x7ff470143d40 0x7ff4701471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:39.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff46ffff700 1 -- 172.21.15.36:0/3421369225 >> v2:172.21.15.70:3300/0 conn(0x7ff470143d40 msgr2=0x7ff4701471d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:39.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff46ffff700 1 --2- 172.21.15.36:0/3421369225 >> v2:172.21.15.70:3300/0 conn(0x7ff470143d40 0x7ff4701471d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:39.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff46ffff700 1 -- 172.21.15.36:0/3421369225 >> v1:172.21.15.36:6789/0 conn(0x7ff470062270 legacy=0x7ff4700b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:39.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff46ffff700 1 -- 172.21.15.36:0/3421369225 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff470062650 con 0x7ff4700b4190 2022-06-17T12:50:39.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff46ffff700 1 --2- 172.21.15.36:0/3421369225 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff4700b4190 0x7ff4700b4fc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7ff45c002f70 tx=0x7ff45c006eb0).ready entity=mon.0 client_cookie=5cf6080b453d9bc7 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:39.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff46f7fe700 1 -- 172.21.15.36:0/3421369225 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff45c00d5c0 con 0x7ff4700b4190 2022-06-17T12:50:39.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff46f7fe700 1 -- 172.21.15.36:0/3421369225 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45c000e90 con 0x7ff4700b4190 2022-06-17T12:50:39.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff46f7fe700 1 -- 172.21.15.36:0/3421369225 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff45c00d950 con 0x7ff4700b4190 2022-06-17T12:50:39.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff476f09700 1 -- 172.21.15.36:0/3421369225 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff4700b4190 msgr2=0x7ff4700b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:39.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff476f09700 1 --2- 172.21.15.36:0/3421369225 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff4700b4190 0x7ff4700b4fc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7ff45c002f70 tx=0x7ff45c006eb0).stop 2022-06-17T12:50:39.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff476f09700 1 -- 172.21.15.36:0/3421369225 shutdown_connections 2022-06-17T12:50:39.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff476f09700 1 --2- 172.21.15.36:0/3421369225 >> v2:172.21.15.70:3300/0 conn(0x7ff470143d40 0x7ff4701471d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:39.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff476f09700 1 --2- 172.21.15.36:0/3421369225 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff4700b4190 0x7ff4700b4fc0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:39.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff476f09700 1 -- 172.21.15.36:0/3421369225 >> 172.21.15.36:0/3421369225 conn(0x7ff4700b97f0 msgr2=0x7ff4700b9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:39.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff476f09700 1 -- 172.21.15.36:0/3421369225 shutdown_connections 2022-06-17T12:50:39.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.865+0000 7ff476f09700 1 -- 172.21.15.36:0/3421369225 wait complete. 2022-06-17T12:50:39.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff476f09700 1 Processor -- start 2022-06-17T12:50:39.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff476f09700 1 -- start start 2022-06-17T12:50:39.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff476f09700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff470062270 0x7ff4700b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:39.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff476f09700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4700b4190 0x7ff470152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:39.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff476f09700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff470143d40 0x7ff470152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:39.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff476f09700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff470153370 con 0x7ff470143d40 2022-06-17T12:50:39.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff476f09700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4700b3150 con 0x7ff4700b4190 2022-06-17T12:50:39.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff476f09700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47012df20 con 0x7ff470062270 2022-06-17T12:50:39.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff4754a6700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff470143d40 0x7ff470152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:39.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff4754a6700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff470143d40 0x7ff470152e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33848/0 (socket says 172.21.15.36:33848) 2022-06-17T12:50:39.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff4754a6700 1 -- 172.21.15.36:0/1105118670 learned_addr learned my addr 172.21.15.36:0/1105118670 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:39.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff46ffff700 1 --2- 172.21.15.36:0/1105118670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4700b4190 0x7ff470152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:39.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff474ca5700 1 --2- 172.21.15.36:0/1105118670 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff470062270 0x7ff4700b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:39.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff4754a6700 1 -- 172.21.15.36:0/1105118670 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff470062270 msgr2=0x7ff4700b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:39.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff4754a6700 1 --2- 172.21.15.36:0/1105118670 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff470062270 0x7ff4700b80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:39.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff4754a6700 1 -- 172.21.15.36:0/1105118670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4700b4190 msgr2=0x7ff470152990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:39.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff4754a6700 1 --2- 172.21.15.36:0/1105118670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4700b4190 0x7ff470152990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:39.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff4754a6700 1 -- 172.21.15.36:0/1105118670 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c00b1e0 con 0x7ff470143d40 2022-06-17T12:50:39.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff46ffff700 1 --2- 172.21.15.36:0/1105118670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4700b4190 0x7ff470152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:50:39.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff4754a6700 1 --2- 172.21.15.36:0/1105118670 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff470143d40 0x7ff470152e80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7ff468006e00 tx=0x7ff4680091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:39.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff46dffb700 1 -- 172.21.15.36:0/1105118670 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff468012600 con 0x7ff470143d40 2022-06-17T12:50:39.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff46dffb700 1 -- 172.21.15.36:0/1105118670 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff468012760 con 0x7ff470143d40 2022-06-17T12:50:39.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff476f09700 1 -- 172.21.15.36:0/1105118670 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4700b37d0 con 0x7ff470143d40 2022-06-17T12:50:39.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff46dffb700 1 -- 172.21.15.36:0/1105118670 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff468012ac0 con 0x7ff470143d40 2022-06-17T12:50:39.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.869+0000 7ff476f09700 1 -- 172.21.15.36:0/1105118670 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4701525b0 con 0x7ff470143d40 2022-06-17T12:50:39.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.873+0000 7ff476f09700 1 -- 172.21.15.36:0/1105118670 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff47012dd70 con 0x7ff470143d40 2022-06-17T12:50:39.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.873+0000 7ff46dffb700 1 -- 172.21.15.36:0/1105118670 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff468022070 con 0x7ff470143d40 2022-06-17T12:50:39.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.873+0000 7ff46dffb700 1 --2- 172.21.15.36:0/1105118670 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff450053a40 0x7ff450055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:39.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.873+0000 7ff46dffb700 1 -- 172.21.15.36:0/1105118670 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5686+0+0 (secure 0 0 0) 0x7ff46806c360 con 0x7ff470143d40 2022-06-17T12:50:39.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.873+0000 7ff474ca5700 1 --2- 172.21.15.36:0/1105118670 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff450053a40 0x7ff450055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:39.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.873+0000 7ff474ca5700 1 --2- 172.21.15.36:0/1105118670 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff450053a40 0x7ff450055f00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7ff46400d000 tx=0x7ff464007b80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:39.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:39.881+0000 7ff46dffb700 1 -- 172.21.15.36:0/1105118670 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff468043360 con 0x7ff470143d40 2022-06-17T12:50:40.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.001+0000 7ff476f09700 1 -- 172.21.15.36:0/1105118670 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff4700b3310 con 0x7ff470143d40 2022-06-17T12:50:40.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.009+0000 7ff46dffb700 1 -- 172.21.15.36:0/1105118670 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff4680160a0 con 0x7ff470143d40 2022-06-17T12:50:40.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.009+0000 7ff476f09700 1 -- 172.21.15.36:0/1105118670 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff450053a40 msgr2=0x7ff450055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:40.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.009+0000 7ff476f09700 1 --2- 172.21.15.36:0/1105118670 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff450053a40 0x7ff450055f00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7ff46400d000 tx=0x7ff464007b80).stop 2022-06-17T12:50:40.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.009+0000 7ff476f09700 1 -- 172.21.15.36:0/1105118670 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff470143d40 msgr2=0x7ff470152e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:40.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.009+0000 7ff476f09700 1 --2- 172.21.15.36:0/1105118670 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff470143d40 0x7ff470152e80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7ff468006e00 tx=0x7ff4680091d0).stop 2022-06-17T12:50:40.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.009+0000 7ff476f09700 1 -- 172.21.15.36:0/1105118670 shutdown_connections 2022-06-17T12:50:40.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.009+0000 7ff476f09700 1 --2- 172.21.15.36:0/1105118670 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff450053a40 0x7ff450055f00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:40.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.009+0000 7ff476f09700 1 --2- 172.21.15.36:0/1105118670 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff470143d40 0x7ff470152e80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:40.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.009+0000 7ff476f09700 1 --2- 172.21.15.36:0/1105118670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4700b4190 0x7ff470152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:40.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.009+0000 7ff476f09700 1 --2- 172.21.15.36:0/1105118670 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff470062270 0x7ff4700b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:40.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.009+0000 7ff476f09700 1 -- 172.21.15.36:0/1105118670 >> 172.21.15.36:0/1105118670 conn(0x7ff4700b97f0 msgr2=0x7ff4700b52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:40.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.009+0000 7ff476f09700 1 -- 172.21.15.36:0/1105118670 shutdown_connections 2022-06-17T12:50:40.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.009+0000 7ff476f09700 1 -- 172.21.15.36:0/1105118670 wait complete. 2022-06-17T12:50:40.013 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead' 2022-06-17T12:50:40.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.133+0000 7fabfbfff700 1 Processor -- start 2022-06-17T12:50:40.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.133+0000 7fabfbfff700 1 -- start start 2022-06-17T12:50:40.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.137+0000 7fabfbfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fabfc060ca0 0x7fabfc1442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:40.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.137+0000 7fabfbfff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabfc14b870 con 0x7fabfc14b2b0 2022-06-17T12:50:40.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.137+0000 7fabfbfff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabfc0ac500 con 0x7fabfc0b27b0 2022-06-17T12:50:40.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.137+0000 7fabfbfff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc0ace90 con 0x7fabfc060ca0 2022-06-17T12:50:40.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.137+0000 7fabfa7fc700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fabfc14b2b0 0x7fabfc147bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51382/0 (socket says 172.21.15.36:51382) 2022-06-17T12:50:40.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.137+0000 7fabfa7fc700 1 -- 172.21.15.36:0/3491605344 learned_addr learned my addr 172.21.15.36:0/3491605344 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:50:40.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.137+0000 7fabfb7fe700 1 --2- 172.21.15.36:0/3491605344 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fabfc060ca0 0x7fabfc1442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:40.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.137+0000 7fabf9ffb700 1 -- 172.21.15.36:0/3491605344 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3876463382 0 0) 0x7fabfc14b870 con 0x7fabfc14b2b0 2022-06-17T12:50:40.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.137+0000 7fabf9ffb700 1 -- 172.21.15.36:0/3491605344 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabe0003260 con 0x7fabfc14b2b0 2022-06-17T12:50:40.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.137+0000 7fabfb7fe700 1 -- 172.21.15.36:0/3491605344 >> v1:172.21.15.70:6789/0 conn(0x7fabfc0b27b0 legacy=0x7fabfc0ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:40.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.137+0000 7fabf9ffb700 1 -- 172.21.15.36:0/3491605344 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3434262677 0 0) 0x7fabfc0ac500 con 0x7fabfc0b27b0 2022-06-17T12:50:40.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.137+0000 7fabfb7fe700 1 -- 172.21.15.36:0/3491605344 >> v1:172.21.15.36:6789/0 conn(0x7fabfc14b2b0 legacy=0x7fabfc147bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:40.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.137+0000 7fabfb7fe700 1 -- 172.21.15.36:0/3491605344 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabfc1482e0 con 0x7fabfc060ca0 2022-06-17T12:50:40.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.137+0000 7fabfb7fe700 1 --2- 172.21.15.36:0/3491605344 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fabfc060ca0 0x7fabfc1442d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fabf40032f0 tx=0x7fabf40091d0).ready entity=mon.2 client_cookie=f2f1670141815645 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:40.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.137+0000 7fabf9ffb700 1 -- 172.21.15.36:0/3491605344 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabf400b6c0 con 0x7fabfc060ca0 2022-06-17T12:50:40.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.137+0000 7fabf9ffb700 1 -- 172.21.15.36:0/3491605344 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf400b820 con 0x7fabfc060ca0 2022-06-17T12:50:40.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.137+0000 7fabf9ffb700 1 -- 172.21.15.36:0/3491605344 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabf400b990 con 0x7fabfc060ca0 2022-06-17T12:50:40.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.137+0000 7fabfbfff700 1 -- 172.21.15.36:0/3491605344 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fabfc060ca0 msgr2=0x7fabfc1442d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:40.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.137+0000 7fabfbfff700 1 --2- 172.21.15.36:0/3491605344 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fabfc060ca0 0x7fabfc1442d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fabf40032f0 tx=0x7fabf40091d0).stop 2022-06-17T12:50:40.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.137+0000 7fabfbfff700 1 -- 172.21.15.36:0/3491605344 shutdown_connections 2022-06-17T12:50:40.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.137+0000 7fabfbfff700 1 --2- 172.21.15.36:0/3491605344 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fabfc060ca0 0x7fabfc1442d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:40.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.137+0000 7fabfbfff700 1 -- 172.21.15.36:0/3491605344 >> 172.21.15.36:0/3491605344 conn(0x7fabfc0b5d10 msgr2=0x7fabfc0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:40.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.137+0000 7fabfbfff700 1 -- 172.21.15.36:0/3491605344 shutdown_connections 2022-06-17T12:50:40.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.137+0000 7fabfbfff700 1 -- 172.21.15.36:0/3491605344 wait complete. 2022-06-17T12:50:40.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.141+0000 7fabfbfff700 1 Processor -- start 2022-06-17T12:50:40.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.141+0000 7fabfbfff700 1 -- start start 2022-06-17T12:50:40.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.141+0000 7fabfbfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fabfc060ca0 0x7fabfc14a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:40.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.141+0000 7fabfbfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fabfc0b27b0 0x7fabfc148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:40.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.141+0000 7fabfbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabfc14b2b0 0x7fabfc156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:40.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.141+0000 7fabfbfff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc1427b0 con 0x7fabfc060ca0 2022-06-17T12:50:40.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.141+0000 7fabfbfff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc149ae0 con 0x7fabfc14b2b0 2022-06-17T12:50:40.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.141+0000 7fabfbfff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc149d80 con 0x7fabfc0b27b0 2022-06-17T12:50:40.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.141+0000 7fabfaffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fabfc060ca0 0x7fabfc14a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:40.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.141+0000 7fabfaffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fabfc060ca0 0x7fabfc14a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33858/0 (socket says 172.21.15.36:33858) 2022-06-17T12:50:40.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.141+0000 7fabfaffd700 1 -- 172.21.15.36:0/2904568958 learned_addr learned my addr 172.21.15.36:0/2904568958 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:40.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.141+0000 7fabfb7fe700 1 --2- 172.21.15.36:0/2904568958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabfc14b2b0 0x7fabfc156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:40.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.141+0000 7fabfa7fc700 1 --2- 172.21.15.36:0/2904568958 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fabfc0b27b0 0x7fabfc148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:40.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.141+0000 7fabfaffd700 1 -- 172.21.15.36:0/2904568958 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fabfc0b27b0 msgr2=0x7fabfc148e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:40.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.141+0000 7fabfaffd700 1 --2- 172.21.15.36:0/2904568958 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fabfc0b27b0 0x7fabfc148e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:40.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.141+0000 7fabfaffd700 1 -- 172.21.15.36:0/2904568958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabfc14b2b0 msgr2=0x7fabfc156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:40.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.141+0000 7fabfaffd700 1 --2- 172.21.15.36:0/2904568958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabfc14b2b0 0x7fabfc156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:40.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.141+0000 7fabfaffd700 1 -- 172.21.15.36:0/2904568958 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf4009470 con 0x7fabfc060ca0 2022-06-17T12:50:40.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.141+0000 7fabfaffd700 1 --2- 172.21.15.36:0/2904568958 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fabfc060ca0 0x7fabfc14a4c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fabf0000ed0 tx=0x7fabf0007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:40.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.141+0000 7fabebfff700 1 -- 172.21.15.36:0/2904568958 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabf000ce20 con 0x7fabfc060ca0 2022-06-17T12:50:40.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.141+0000 7fabebfff700 1 -- 172.21.15.36:0/2904568958 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf000f470 con 0x7fabfc060ca0 2022-06-17T12:50:40.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.141+0000 7fabebfff700 1 -- 172.21.15.36:0/2904568958 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabf000f7d0 con 0x7fabfc060ca0 2022-06-17T12:50:40.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.141+0000 7fabfbfff700 1 -- 172.21.15.36:0/2904568958 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabfc14a020 con 0x7fabfc060ca0 2022-06-17T12:50:40.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.141+0000 7fabfbfff700 1 -- 172.21.15.36:0/2904568958 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabfc149530 con 0x7fabfc060ca0 2022-06-17T12:50:40.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.145+0000 7fabfbfff700 1 -- 172.21.15.36:0/2904568958 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabfc0626d0 con 0x7fabfc060ca0 2022-06-17T12:50:40.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.145+0000 7fabebfff700 1 -- 172.21.15.36:0/2904568958 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fabf0019030 con 0x7fabfc060ca0 2022-06-17T12:50:40.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.145+0000 7fabebfff700 1 --2- 172.21.15.36:0/2904568958 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fabd8053920 0x7fabd8055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:40.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.145+0000 7fabebfff700 1 -- 172.21.15.36:0/2904568958 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fabf006b190 con 0x7fabfc060ca0 2022-06-17T12:50:40.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.145+0000 7fabfa7fc700 1 --2- 172.21.15.36:0/2904568958 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fabd8053920 0x7fabd8055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:40.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.145+0000 7fabfa7fc700 1 --2- 172.21.15.36:0/2904568958 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fabd8053920 0x7fabd8055de0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fabec008000 tx=0x7fabec009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:40.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.149+0000 7fabebfff700 1 -- 172.21.15.36:0/2904568958 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fabfc0626d0 con 0x7fabfc060ca0 2022-06-17T12:50:40.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.317+0000 7fabfbfff700 1 -- 172.21.15.36:0/2904568958 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]} v 0) v1 -- 0x7fabfc0626d0 con 0x7fabfc060ca0 2022-06-17T12:50:40.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.721+0000 7fabebfff700 1 -- 172.21.15.36:0/2904568958 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]}]=0 v182) v1 ==== 144+0+0 (secure 0 0 0) 0x7fabfc0626d0 con 0x7fabfc060ca0 2022-06-17T12:50:40.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.725+0000 7fabfbfff700 1 -- 172.21.15.36:0/2904568958 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fabd8053920 msgr2=0x7fabd8055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:40.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.725+0000 7fabfbfff700 1 --2- 172.21.15.36:0/2904568958 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fabd8053920 0x7fabd8055de0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fabec008000 tx=0x7fabec009040).stop 2022-06-17T12:50:40.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.725+0000 7fabfbfff700 1 -- 172.21.15.36:0/2904568958 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fabfc060ca0 msgr2=0x7fabfc14a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:40.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.725+0000 7fabfbfff700 1 --2- 172.21.15.36:0/2904568958 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fabfc060ca0 0x7fabfc14a4c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fabf0000ed0 tx=0x7fabf0007ac0).stop 2022-06-17T12:50:40.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.725+0000 7fabfbfff700 1 -- 172.21.15.36:0/2904568958 shutdown_connections 2022-06-17T12:50:40.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.725+0000 7fabfbfff700 1 --2- 172.21.15.36:0/2904568958 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fabd8053920 0x7fabd8055de0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:40.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.725+0000 7fabfbfff700 1 --2- 172.21.15.36:0/2904568958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabfc14b2b0 0x7fabfc156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:40.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.725+0000 7fabfbfff700 1 --2- 172.21.15.36:0/2904568958 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fabfc0b27b0 0x7fabfc148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:40.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.725+0000 7fabfbfff700 1 --2- 172.21.15.36:0/2904568958 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fabfc060ca0 0x7fabfc14a4c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:40.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.725+0000 7fabfbfff700 1 -- 172.21.15.36:0/2904568958 >> 172.21.15.36:0/2904568958 conn(0x7fabfc0b5d10 msgr2=0x7fabfc0afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:40.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.725+0000 7fabfbfff700 1 -- 172.21.15.36:0/2904568958 shutdown_connections 2022-06-17T12:50:40.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.725+0000 7fabfbfff700 1 -- 172.21.15.36:0/2904568958 wait complete. 2022-06-17T12:50:40.729 INFO:tasks.cephfs_test_runner:test_drop_cache_command_dead (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2022-06-17T12:50:40.730 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_dead_timeout (tasks.cephfs.test_misc.TestCacheDrop) 2022-06-17T12:50:40.731 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout' 2022-06-17T12:50:40.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.853+0000 7f4fb7fff700 1 Processor -- start 2022-06-17T12:50:40.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.853+0000 7f4fb7fff700 1 -- start start 2022-06-17T12:50:40.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.857+0000 7f4fb7fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4fb80b39a0 0x7f4fb8143860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:40.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.857+0000 7f4fb7fff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb80b2b70 con 0x7f4fb80b6860 2022-06-17T12:50:40.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.857+0000 7f4fb7fff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb80b0d30 con 0x7f4fb80b18e0 2022-06-17T12:50:40.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.857+0000 7f4fb7fff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fb80b6c40 con 0x7f4fb80b39a0 2022-06-17T12:50:40.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.857+0000 7f4fb6ffd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f4fb80b6860 0x7f4fb8143150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51396/0 (socket says 172.21.15.36:51396) 2022-06-17T12:50:40.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.857+0000 7f4fb77fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4fb80b39a0 0x7f4fb8143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:40.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.857+0000 7f4fb6ffd700 1 -- 172.21.15.36:0/2833218689 learned_addr learned my addr 172.21.15.36:0/2833218689 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:50:40.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.857+0000 7f4fb77fe700 1 -- 172.21.15.36:0/2833218689 >> v1:172.21.15.70:6789/0 conn(0x7f4fb80b18e0 legacy=0x7f4fb813fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:40.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.857+0000 7f4fb77fe700 1 -- 172.21.15.36:0/2833218689 >> v1:172.21.15.36:6789/0 conn(0x7f4fb80b6860 legacy=0x7f4fb8143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:40.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.857+0000 7f4fb77fe700 1 -- 172.21.15.36:0/2833218689 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb80b0fd0 con 0x7f4fb80b39a0 2022-06-17T12:50:40.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.857+0000 7f4fb77fe700 1 --2- 172.21.15.36:0/2833218689 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fb80b39a0 0x7f4fb8143860 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f4fb00032f0 tx=0x7f4fb00091d0).ready entity=mon.0 client_cookie=3b8c5a4e08c8c4ed server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:40.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.857+0000 7f4fb67fc700 1 -- 172.21.15.36:0/2833218689 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fb000b8a0 con 0x7f4fb80b39a0 2022-06-17T12:50:40.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.857+0000 7f4fb67fc700 1 -- 172.21.15.36:0/2833218689 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb000ba00 con 0x7f4fb80b39a0 2022-06-17T12:50:40.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.857+0000 7f4fb67fc700 1 -- 172.21.15.36:0/2833218689 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fb000bb70 con 0x7f4fb80b39a0 2022-06-17T12:50:40.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.857+0000 7f4fb7fff700 1 -- 172.21.15.36:0/2833218689 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fb80b39a0 msgr2=0x7f4fb8143860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:40.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.857+0000 7f4fb7fff700 1 --2- 172.21.15.36:0/2833218689 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fb80b39a0 0x7f4fb8143860 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f4fb00032f0 tx=0x7f4fb00091d0).stop 2022-06-17T12:50:40.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.857+0000 7f4fb7fff700 1 -- 172.21.15.36:0/2833218689 shutdown_connections 2022-06-17T12:50:40.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.857+0000 7f4fb7fff700 1 --2- 172.21.15.36:0/2833218689 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fb80b39a0 0x7f4fb8143860 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:40.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.857+0000 7f4fb7fff700 1 -- 172.21.15.36:0/2833218689 >> 172.21.15.36:0/2833218689 conn(0x7f4fb80614b0 msgr2=0x7f4fb8061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:40.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.857+0000 7f4fb7fff700 1 -- 172.21.15.36:0/2833218689 shutdown_connections 2022-06-17T12:50:40.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.857+0000 7f4fb7fff700 1 -- 172.21.15.36:0/2833218689 wait complete. 2022-06-17T12:50:40.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.861+0000 7f4fb7fff700 1 Processor -- start 2022-06-17T12:50:40.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.861+0000 7f4fb7fff700 1 -- start start 2022-06-17T12:50:40.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.861+0000 7f4fb7fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fb80b18e0 0x7f4fb8141c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:40.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.861+0000 7f4fb7fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fb80b39a0 0x7f4fb8155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:40.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.861+0000 7f4fb7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb80b6860 0x7f4fb8114e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:40.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.861+0000 7f4fb7fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb8065ed0 con 0x7f4fb80b18e0 2022-06-17T12:50:40.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.861+0000 7f4fb7fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb8066100 con 0x7f4fb80b6860 2022-06-17T12:50:40.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.861+0000 7f4fb7fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb8114980 con 0x7f4fb80b39a0 2022-06-17T12:50:40.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.861+0000 7f4fb6ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fb80b18e0 0x7f4fb8141c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:40.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.861+0000 7f4fb6ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fb80b18e0 0x7f4fb8141c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33872/0 (socket says 172.21.15.36:33872) 2022-06-17T12:50:40.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.861+0000 7f4fb6ffd700 1 -- 172.21.15.36:0/415940032 learned_addr learned my addr 172.21.15.36:0/415940032 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:40.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.861+0000 7f4faffff700 1 --2- 172.21.15.36:0/415940032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fb80b39a0 0x7f4fb8155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:40.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.861+0000 7f4fb77fe700 1 --2- 172.21.15.36:0/415940032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb80b6860 0x7f4fb8114e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:40.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.861+0000 7f4fb6ffd700 1 -- 172.21.15.36:0/415940032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fb80b39a0 msgr2=0x7f4fb8155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:40.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.861+0000 7f4fb6ffd700 1 --2- 172.21.15.36:0/415940032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fb80b39a0 0x7f4fb8155ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:40.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.861+0000 7f4fb6ffd700 1 -- 172.21.15.36:0/415940032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb80b6860 msgr2=0x7f4fb8114e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:40.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.861+0000 7f4fb6ffd700 1 --2- 172.21.15.36:0/415940032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb80b6860 0x7f4fb8114e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:40.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.861+0000 7f4fb6ffd700 1 -- 172.21.15.36:0/415940032 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb0009470 con 0x7f4fb80b18e0 2022-06-17T12:50:40.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.861+0000 7f4fb6ffd700 1 --2- 172.21.15.36:0/415940032 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fb80b18e0 0x7f4fb8141c90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f4fb80b3760 tx=0x7f4fa8007ca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:40.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.861+0000 7f4fb4ff9700 1 -- 172.21.15.36:0/415940032 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fa80070d0 con 0x7f4fb80b18e0 2022-06-17T12:50:40.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.861+0000 7f4fb4ff9700 1 -- 172.21.15.36:0/415940032 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa8007230 con 0x7f4fb80b18e0 2022-06-17T12:50:40.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.861+0000 7f4fb4ff9700 1 -- 172.21.15.36:0/415940032 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fa8005610 con 0x7f4fb80b18e0 2022-06-17T12:50:40.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.861+0000 7f4fb7fff700 1 -- 172.21.15.36:0/415940032 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb80654c0 con 0x7f4fb80b18e0 2022-06-17T12:50:40.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.861+0000 7f4fb7fff700 1 -- 172.21.15.36:0/415940032 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb8064eb0 con 0x7f4fb80b18e0 2022-06-17T12:50:40.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.865+0000 7f4fb7fff700 1 -- 172.21.15.36:0/415940032 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb8142c70 con 0x7f4fb80b18e0 2022-06-17T12:50:40.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.865+0000 7f4fb4ff9700 1 -- 172.21.15.36:0/415940032 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f4fa8010070 con 0x7f4fb80b18e0 2022-06-17T12:50:40.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.865+0000 7f4fb4ff9700 1 --2- 172.21.15.36:0/415940032 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4f9c053af0 0x7f4f9c055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:40.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.865+0000 7f4faffff700 1 --2- 172.21.15.36:0/415940032 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4f9c053af0 0x7f4f9c055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:40.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.865+0000 7f4fb4ff9700 1 -- 172.21.15.36:0/415940032 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f4fa806b200 con 0x7f4fb80b18e0 2022-06-17T12:50:40.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.865+0000 7f4faffff700 1 --2- 172.21.15.36:0/415940032 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4f9c053af0 0x7f4f9c055fb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f4fa000d000 tx=0x7f4fa0007b80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:40.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:40.869+0000 7f4fb4ff9700 1 -- 172.21.15.36:0/415940032 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4fa80422e0 con 0x7f4fb80b18e0 2022-06-17T12:50:41.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.037+0000 7f4fb7fff700 1 -- 172.21.15.36:0/415940032 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]} v 0) v1 -- 0x7f4fb8057410 con 0x7f4fb80b18e0 2022-06-17T12:50:41.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.729+0000 7f4fb4ff9700 1 -- 172.21.15.36:0/415940032 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]}]=0 v183) v1 ==== 155+0+0 (secure 0 0 0) 0x7f4fa8018020 con 0x7f4fb80b18e0 2022-06-17T12:50:41.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.733+0000 7f4fb7fff700 1 -- 172.21.15.36:0/415940032 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4f9c053af0 msgr2=0x7f4f9c055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:41.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.733+0000 7f4fb7fff700 1 --2- 172.21.15.36:0/415940032 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4f9c053af0 0x7f4f9c055fb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f4fa000d000 tx=0x7f4fa0007b80).stop 2022-06-17T12:50:41.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.733+0000 7f4fb7fff700 1 -- 172.21.15.36:0/415940032 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fb80b18e0 msgr2=0x7f4fb8141c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:41.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.733+0000 7f4fb7fff700 1 --2- 172.21.15.36:0/415940032 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fb80b18e0 0x7f4fb8141c90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f4fb80b3760 tx=0x7f4fa8007ca0).stop 2022-06-17T12:50:41.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.733+0000 7f4fb7fff700 1 -- 172.21.15.36:0/415940032 shutdown_connections 2022-06-17T12:50:41.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.733+0000 7f4fb7fff700 1 --2- 172.21.15.36:0/415940032 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4f9c053af0 0x7f4f9c055fb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:41.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.733+0000 7f4fb7fff700 1 --2- 172.21.15.36:0/415940032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb80b6860 0x7f4fb8114e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:41.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.733+0000 7f4fb7fff700 1 --2- 172.21.15.36:0/415940032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fb80b39a0 0x7f4fb8155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:41.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.733+0000 7f4fb7fff700 1 --2- 172.21.15.36:0/415940032 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fb80b18e0 0x7f4fb8141c90 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:41.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.733+0000 7f4fb7fff700 1 -- 172.21.15.36:0/415940032 >> 172.21.15.36:0/415940032 conn(0x7f4fb80614b0 msgr2=0x7f4fb8144770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:41.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.733+0000 7f4fb7fff700 1 -- 172.21.15.36:0/415940032 shutdown_connections 2022-06-17T12:50:41.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.733+0000 7f4fb7fff700 1 -- 172.21.15.36:0/415940032 wait complete. 2022-06-17T12:50:41.771 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-06-17T12:50:41.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.905+0000 7fd850452700 1 Processor -- start 2022-06-17T12:50:41.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.905+0000 7fd850452700 1 -- start start 2022-06-17T12:50:41.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd850452700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8480afe40 0x7fd8480b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:41.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd850452700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd8480b3440 0x7fd84813aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:41.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd850452700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8480a9b90 con 0x7fd84813af10 2022-06-17T12:50:41.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd850452700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd848141f70 con 0x7fd8480b3440 2022-06-17T12:50:41.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd850452700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8480b0220 con 0x7fd8480afe40 2022-06-17T12:50:41.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd84e9ef700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fd84813af10 0x7fd84805f770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51414/0 (socket says 172.21.15.36:51414) 2022-06-17T12:50:41.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd84e9ef700 1 -- 172.21.15.36:0/1271702212 learned_addr learned my addr 172.21.15.36:0/1271702212 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:50:41.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd84d9ed700 1 --2- 172.21.15.36:0/1271702212 >> v2:172.21.15.70:3300/0 conn(0x7fd8480b3440 0x7fd84813aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:41.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd84e1ee700 1 --2- 172.21.15.36:0/1271702212 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8480afe40 0x7fd8480b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:41.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd84d1ec700 1 -- 172.21.15.36:0/1271702212 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2863905004 0 0) 0x7fd8480a9b90 con 0x7fd84813af10 2022-06-17T12:50:41.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd84d1ec700 1 -- 172.21.15.36:0/1271702212 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd830003260 con 0x7fd84813af10 2022-06-17T12:50:41.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd84d9ed700 1 -- 172.21.15.36:0/1271702212 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8480afe40 msgr2=0x7fd8480b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:41.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd84d9ed700 1 --2- 172.21.15.36:0/1271702212 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8480afe40 0x7fd8480b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:41.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd84d9ed700 1 -- 172.21.15.36:0/1271702212 >> v1:172.21.15.36:6789/0 conn(0x7fd84813af10 legacy=0x7fd84805f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:41.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd84d9ed700 1 -- 172.21.15.36:0/1271702212 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8480a9dc0 con 0x7fd8480b3440 2022-06-17T12:50:41.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd84d9ed700 1 --2- 172.21.15.36:0/1271702212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8480b3440 0x7fd84813aa20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fd838002f70 tx=0x7fd838006df0).ready entity=mon.1 client_cookie=d519ac743caee241 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:41.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd84d1ec700 1 -- 172.21.15.36:0/1271702212 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd83800d4a0 con 0x7fd8480b3440 2022-06-17T12:50:41.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd84d1ec700 1 -- 172.21.15.36:0/1271702212 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd838000e90 con 0x7fd8480b3440 2022-06-17T12:50:41.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd84d1ec700 1 -- 172.21.15.36:0/1271702212 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd83800d7f0 con 0x7fd8480b3440 2022-06-17T12:50:41.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd850452700 1 -- 172.21.15.36:0/1271702212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8480b3440 msgr2=0x7fd84813aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:41.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd850452700 1 --2- 172.21.15.36:0/1271702212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8480b3440 0x7fd84813aa20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fd838002f70 tx=0x7fd838006df0).stop 2022-06-17T12:50:41.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd850452700 1 -- 172.21.15.36:0/1271702212 shutdown_connections 2022-06-17T12:50:41.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd850452700 1 --2- 172.21.15.36:0/1271702212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8480b3440 0x7fd84813aa20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:41.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd850452700 1 --2- 172.21.15.36:0/1271702212 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8480afe40 0x7fd8480b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:41.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd850452700 1 -- 172.21.15.36:0/1271702212 >> 172.21.15.36:0/1271702212 conn(0x7fd84805c550 msgr2=0x7fd84805c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:41.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd850452700 1 -- 172.21.15.36:0/1271702212 shutdown_connections 2022-06-17T12:50:41.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.909+0000 7fd850452700 1 -- 172.21.15.36:0/1271702212 wait complete. 2022-06-17T12:50:41.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.913+0000 7fd850452700 1 Processor -- start 2022-06-17T12:50:41.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.913+0000 7fd850452700 1 -- start start 2022-06-17T12:50:41.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.913+0000 7fd850452700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8480afe40 0x7fd848153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:41.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.917+0000 7fd850452700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8480b3440 0x7fd8480acb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:41.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.917+0000 7fd850452700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd84813af10 0x7fd84813d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:41.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.917+0000 7fd850452700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd848067080 con 0x7fd8480b3440 2022-06-17T12:50:41.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.917+0000 7fd850452700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84813ccd0 con 0x7fd84813af10 2022-06-17T12:50:41.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.917+0000 7fd850452700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd848065620 con 0x7fd8480afe40 2022-06-17T12:50:41.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.917+0000 7fd84d9ed700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8480b3440 0x7fd8480acb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:41.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.917+0000 7fd84d9ed700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8480b3440 0x7fd8480acb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33888/0 (socket says 172.21.15.36:33888) 2022-06-17T12:50:41.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.917+0000 7fd84d9ed700 1 -- 172.21.15.36:0/2806065763 learned_addr learned my addr 172.21.15.36:0/2806065763 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:41.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.917+0000 7fd84e9ef700 1 --2- 172.21.15.36:0/2806065763 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd84813af10 0x7fd84813d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:41.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.917+0000 7fd84e1ee700 1 --2- 172.21.15.36:0/2806065763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8480afe40 0x7fd848153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:41.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.917+0000 7fd84d9ed700 1 -- 172.21.15.36:0/2806065763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8480afe40 msgr2=0x7fd848153e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:41.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.917+0000 7fd84d9ed700 1 --2- 172.21.15.36:0/2806065763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8480afe40 0x7fd848153e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:41.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.917+0000 7fd84d9ed700 1 -- 172.21.15.36:0/2806065763 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd84813af10 msgr2=0x7fd84813d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:41.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.917+0000 7fd84d9ed700 1 --2- 172.21.15.36:0/2806065763 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd84813af10 0x7fd84813d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:41.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.917+0000 7fd84d9ed700 1 -- 172.21.15.36:0/2806065763 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83800b180 con 0x7fd8480b3440 2022-06-17T12:50:41.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.917+0000 7fd84e1ee700 1 --2- 172.21.15.36:0/2806065763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8480afe40 0x7fd848153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:50:41.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.917+0000 7fd84d9ed700 1 --2- 172.21.15.36:0/2806065763 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8480b3440 0x7fd8480acb30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7fd838002f70 tx=0x7fd838014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:41.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.917+0000 7fd8377fe700 1 -- 172.21.15.36:0/2806065763 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd838016070 con 0x7fd8480b3440 2022-06-17T12:50:41.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.917+0000 7fd8377fe700 1 -- 172.21.15.36:0/2806065763 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8380138d0 con 0x7fd8480b3440 2022-06-17T12:50:41.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.917+0000 7fd8377fe700 1 -- 172.21.15.36:0/2806065763 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd838013c30 con 0x7fd8480b3440 2022-06-17T12:50:41.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.917+0000 7fd850452700 1 -- 172.21.15.36:0/2806065763 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8480658c0 con 0x7fd8480b3440 2022-06-17T12:50:41.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.917+0000 7fd850452700 1 -- 172.21.15.36:0/2806065763 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8480ad020 con 0x7fd8480b3440 2022-06-17T12:50:41.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.917+0000 7fd850452700 1 -- 172.21.15.36:0/2806065763 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd848144f40 con 0x7fd8480b3440 2022-06-17T12:50:41.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.921+0000 7fd8377fe700 1 -- 172.21.15.36:0/2806065763 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd83800f030 con 0x7fd8480b3440 2022-06-17T12:50:41.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.921+0000 7fd8377fe700 1 --2- 172.21.15.36:0/2806065763 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd8280539b0 0x7fd828055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:41.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.921+0000 7fd8377fe700 1 -- 172.21.15.36:0/2806065763 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fd83806ca50 con 0x7fd8480b3440 2022-06-17T12:50:41.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.921+0000 7fd84e1ee700 1 --2- 172.21.15.36:0/2806065763 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd8280539b0 0x7fd828055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:41.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.921+0000 7fd84e1ee700 1 --2- 172.21.15.36:0/2806065763 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd8280539b0 0x7fd828055e70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fd84400b000 tx=0x7fd844008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:41.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:41.929+0000 7fd8377fe700 1 -- 172.21.15.36:0/2806065763 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd838043b30 con 0x7fd8480b3440 2022-06-17T12:50:42.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.057+0000 7fd850452700 1 -- 172.21.15.36:0/2806065763 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd848057410 con 0x7fd8480b3440 2022-06-17T12:50:42.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.065+0000 7fd8377fe700 1 -- 172.21.15.36:0/2806065763 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd838023070 con 0x7fd8480b3440 2022-06-17T12:50:42.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.065+0000 7fd850452700 1 -- 172.21.15.36:0/2806065763 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd8280539b0 msgr2=0x7fd828055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:42.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.069+0000 7fd850452700 1 --2- 172.21.15.36:0/2806065763 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd8280539b0 0x7fd828055e70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fd84400b000 tx=0x7fd844008040).stop 2022-06-17T12:50:42.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.069+0000 7fd850452700 1 -- 172.21.15.36:0/2806065763 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8480b3440 msgr2=0x7fd8480acb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:42.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.069+0000 7fd850452700 1 --2- 172.21.15.36:0/2806065763 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8480b3440 0x7fd8480acb30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7fd838002f70 tx=0x7fd838014000).stop 2022-06-17T12:50:42.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.069+0000 7fd850452700 1 -- 172.21.15.36:0/2806065763 shutdown_connections 2022-06-17T12:50:42.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.069+0000 7fd850452700 1 --2- 172.21.15.36:0/2806065763 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd8280539b0 0x7fd828055e70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:42.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.069+0000 7fd850452700 1 --2- 172.21.15.36:0/2806065763 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd84813af10 0x7fd84813d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:42.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.069+0000 7fd850452700 1 --2- 172.21.15.36:0/2806065763 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8480b3440 0x7fd8480acb30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:42.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.069+0000 7fd850452700 1 --2- 172.21.15.36:0/2806065763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8480afe40 0x7fd848153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:42.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.069+0000 7fd850452700 1 -- 172.21.15.36:0/2806065763 >> 172.21.15.36:0/2806065763 conn(0x7fd84805c550 msgr2=0x7fd848139b90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:42.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.069+0000 7fd850452700 1 -- 172.21.15.36:0/2806065763 shutdown_connections 2022-06-17T12:50:42.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.069+0000 7fd850452700 1 -- 172.21.15.36:0/2806065763 wait complete. 2022-06-17T12:50:42.080 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-17T12:50:42.081 DEBUG:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:50:42.112 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:50:42.149 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:50:42.171 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:50:42.193 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:50:42.215 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-17T12:50:42.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:50:42.217 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:50:42.219 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:50:42.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:50:42.241 INFO:teuthology.orchestra.run.smithi107.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-17T12:50:42.242 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-06-17T12:50:42.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:50:42.243 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:50:42.244 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:50:42.251 INFO:teuthology.orchestra.run.smithi136.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-17T12:50:42.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:50:42.253 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-17T12:50:42.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:50:42.262 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:50:42.262 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:50:42.270 INFO:teuthology.orchestra.run.smithi143.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:50:42.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:50:42.272 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-06-17T12:50:42.272 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-17T12:50:42.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.401+0000 7efe2e7f2700 1 Processor -- start 2022-06-17T12:50:42.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.401+0000 7efe2e7f2700 1 -- start start 2022-06-17T12:50:42.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.405+0000 7efe2e7f2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe280b6860 0x7efe28146d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:42.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.405+0000 7efe2e7f2700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efe280b39a0 0x7efe28143860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:42.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.405+0000 7efe2e7f2700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe280b2b70 con 0x7efe280b18e0 2022-06-17T12:50:42.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.405+0000 7efe2e7f2700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe280b6c40 con 0x7efe280b39a0 2022-06-17T12:50:42.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.405+0000 7efe2e7f2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe28142d30 con 0x7efe280b6860 2022-06-17T12:50:42.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.405+0000 7efe2dff1700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efe280b39a0 0x7efe28143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:42.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.405+0000 7efe2cfef700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7efe280b18e0 0x7efe28143150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51426/0 (socket says 172.21.15.36:51426) 2022-06-17T12:50:42.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.405+0000 7efe2cfef700 1 -- 172.21.15.36:0/133538763 learned_addr learned my addr 172.21.15.36:0/133538763 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:50:42.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.405+0000 7efe2d7f0700 1 --2- 172.21.15.36:0/133538763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe280b6860 0x7efe28146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:42.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.405+0000 7efe17fff700 1 -- 172.21.15.36:0/133538763 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 363188855 0 0) 0x7efe280b2b70 con 0x7efe280b18e0 2022-06-17T12:50:42.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.405+0000 7efe17fff700 1 -- 172.21.15.36:0/133538763 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe10003260 con 0x7efe280b18e0 2022-06-17T12:50:42.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.405+0000 7efe2d7f0700 1 -- 172.21.15.36:0/133538763 >> v2:172.21.15.70:3300/0 conn(0x7efe280b39a0 msgr2=0x7efe28143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:42.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.405+0000 7efe2d7f0700 1 --2- 172.21.15.36:0/133538763 >> v2:172.21.15.70:3300/0 conn(0x7efe280b39a0 0x7efe28143860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:42.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.405+0000 7efe2d7f0700 1 -- 172.21.15.36:0/133538763 >> v1:172.21.15.36:6789/0 conn(0x7efe280b18e0 legacy=0x7efe28143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:42.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.405+0000 7efe2d7f0700 1 -- 172.21.15.36:0/133538763 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe280b0d30 con 0x7efe280b6860 2022-06-17T12:50:42.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.405+0000 7efe2d7f0700 1 --2- 172.21.15.36:0/133538763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe280b6860 0x7efe28146d40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7efe24002f70 tx=0x7efe24006df0).ready entity=mon.2 client_cookie=dd41950c843dee8e server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:42.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.405+0000 7efe17fff700 1 -- 172.21.15.36:0/133538763 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe2400d4a0 con 0x7efe280b6860 2022-06-17T12:50:42.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.405+0000 7efe17fff700 1 -- 172.21.15.36:0/133538763 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe24000e90 con 0x7efe280b6860 2022-06-17T12:50:42.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.405+0000 7efe17fff700 1 -- 172.21.15.36:0/133538763 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe2400d7f0 con 0x7efe280b6860 2022-06-17T12:50:42.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2e7f2700 1 -- 172.21.15.36:0/133538763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe280b6860 msgr2=0x7efe28146d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:42.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2e7f2700 1 --2- 172.21.15.36:0/133538763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe280b6860 0x7efe28146d40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7efe24002f70 tx=0x7efe24006df0).stop 2022-06-17T12:50:42.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2e7f2700 1 -- 172.21.15.36:0/133538763 shutdown_connections 2022-06-17T12:50:42.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2e7f2700 1 --2- 172.21.15.36:0/133538763 >> v2:172.21.15.70:3300/0 conn(0x7efe280b39a0 0x7efe28143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:42.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2e7f2700 1 --2- 172.21.15.36:0/133538763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe280b6860 0x7efe28146d40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:42.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2e7f2700 1 -- 172.21.15.36:0/133538763 >> 172.21.15.36:0/133538763 conn(0x7efe280614b0 msgr2=0x7efe28061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:42.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2e7f2700 1 -- 172.21.15.36:0/133538763 shutdown_connections 2022-06-17T12:50:42.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2e7f2700 1 -- 172.21.15.36:0/133538763 wait complete. 2022-06-17T12:50:42.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2e7f2700 1 Processor -- start 2022-06-17T12:50:42.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2e7f2700 1 -- start start 2022-06-17T12:50:42.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2e7f2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe280b18e0 0x7efe28152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:42.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2e7f2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe280b39a0 0x7efe28063460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:42.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2e7f2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe280b6860 0x7efe280afa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:42.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2e7f2700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe281530c0 con 0x7efe280b6860 2022-06-17T12:50:42.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2e7f2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe28066340 con 0x7efe280b39a0 2022-06-17T12:50:42.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2e7f2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe28063950 con 0x7efe280b18e0 2022-06-17T12:50:42.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2dff1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe280b6860 0x7efe280afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:42.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2cfef700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe280b39a0 0x7efe28063460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:42.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2d7f0700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe280b18e0 0x7efe28152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:42.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2dff1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe280b6860 0x7efe280afa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33904/0 (socket says 172.21.15.36:33904) 2022-06-17T12:50:42.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2cfef700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe280b39a0 0x7efe28063460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:54284/0 (socket says 172.21.15.36:54284) 2022-06-17T12:50:42.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2dff1700 1 -- 172.21.15.36:0/3143200255 learned_addr learned my addr 172.21.15.36:0/3143200255 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:42.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2dff1700 1 -- 172.21.15.36:0/3143200255 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe280b18e0 msgr2=0x7efe28152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:42.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2dff1700 1 --2- 172.21.15.36:0/3143200255 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe280b18e0 0x7efe28152010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:42.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2dff1700 1 -- 172.21.15.36:0/3143200255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe280b39a0 msgr2=0x7efe28063460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:42.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2dff1700 1 --2- 172.21.15.36:0/3143200255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe280b39a0 0x7efe28063460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:42.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.409+0000 7efe2dff1700 1 -- 172.21.15.36:0/3143200255 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2400b180 con 0x7efe280b6860 2022-06-17T12:50:42.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.413+0000 7efe2cfef700 1 --2- 172.21.15.36:0/3143200255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe280b39a0 0x7efe28063460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:50:42.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.413+0000 7efe2dff1700 1 --2- 172.21.15.36:0/3143200255 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe280b6860 0x7efe280afa10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7efe20006e00 tx=0x7efe200091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:42.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.413+0000 7efe167fc700 1 -- 172.21.15.36:0/3143200255 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe2000c640 con 0x7efe280b6860 2022-06-17T12:50:42.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.413+0000 7efe167fc700 1 -- 172.21.15.36:0/3143200255 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2000c7a0 con 0x7efe280b6860 2022-06-17T12:50:42.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.413+0000 7efe167fc700 1 -- 172.21.15.36:0/3143200255 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe2000cb00 con 0x7efe280b6860 2022-06-17T12:50:42.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.413+0000 7efe2e7f2700 1 -- 172.21.15.36:0/3143200255 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe28063b70 con 0x7efe280b6860 2022-06-17T12:50:42.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.413+0000 7efe2e7f2700 1 -- 172.21.15.36:0/3143200255 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe280aff00 con 0x7efe280b6860 2022-06-17T12:50:42.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.413+0000 7efe2e7f2700 1 -- 172.21.15.36:0/3143200255 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe28067160 con 0x7efe280b6860 2022-06-17T12:50:42.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.417+0000 7efe167fc700 1 -- 172.21.15.36:0/3143200255 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7efe20022070 con 0x7efe280b6860 2022-06-17T12:50:42.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.417+0000 7efe167fc700 1 --2- 172.21.15.36:0/3143200255 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efe08053920 0x7efe08055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:42.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.417+0000 7efe167fc700 1 -- 172.21.15.36:0/3143200255 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5686+0+0 (secure 0 0 0) 0x7efe2006c850 con 0x7efe280b6860 2022-06-17T12:50:42.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.417+0000 7efe2d7f0700 1 --2- 172.21.15.36:0/3143200255 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efe08053920 0x7efe08055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:42.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.417+0000 7efe2d7f0700 1 --2- 172.21.15.36:0/3143200255 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efe08053920 0x7efe08055de0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7efe2401a000 tx=0x7efe24014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:42.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.421+0000 7efe167fc700 1 -- 172.21.15.36:0/3143200255 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efe20043820 con 0x7efe280b6860 2022-06-17T12:50:42.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.545+0000 7efe2e7f2700 1 -- 172.21.15.36:0/3143200255 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7efe28067160 con 0x7efe280b6860 2022-06-17T12:50:42.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.549+0000 7efe167fc700 1 -- 172.21.15.36:0/3143200255 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v48) v1 ==== 81+0+29219 (secure 0 0 0) 0x7efe28067160 con 0x7efe280b6860 2022-06-17T12:50:42.540 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:50:42.541 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:50:42.541 INFO:teuthology.orchestra.run.smithi036.stdout: "epoch": 48, 2022-06-17T12:50:42.541 INFO:teuthology.orchestra.run.smithi036.stdout: "fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", 2022-06-17T12:50:42.541 INFO:teuthology.orchestra.run.smithi036.stdout: "created": "2022-06-17T12:46:30.671408+0000", 2022-06-17T12:50:42.542 INFO:teuthology.orchestra.run.smithi036.stdout: "modified": "2022-06-17T12:50:39.709572+0000", 2022-06-17T12:50:42.542 INFO:teuthology.orchestra.run.smithi036.stdout: "last_up_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:50:42.542 INFO:teuthology.orchestra.run.smithi036.stdout: "last_in_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:50:42.543 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-17T12:50:42.543 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_num": 5799936, 2022-06-17T12:50:42.544 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_set": [ 2022-06-17T12:50:42.544 INFO:teuthology.orchestra.run.smithi036.stdout: "pglog_hardlimit", 2022-06-17T12:50:42.545 INFO:teuthology.orchestra.run.smithi036.stdout: "purged_snapdirs", 2022-06-17T12:50:42.545 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_deletes", 2022-06-17T12:50:42.545 INFO:teuthology.orchestra.run.smithi036.stdout: "sortbitwise" 2022-06-17T12:50:42.545 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:50:42.546 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_version": 6, 2022-06-17T12:50:42.546 INFO:teuthology.orchestra.run.smithi036.stdout: "full_ratio": 0.94999998807907104, 2022-06-17T12:50:42.546 INFO:teuthology.orchestra.run.smithi036.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-17T12:50:42.546 INFO:teuthology.orchestra.run.smithi036.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-17T12:50:42.547 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_snapshot": "", 2022-06-17T12:50:42.547 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_max": 8, 2022-06-17T12:50:42.547 INFO:teuthology.orchestra.run.smithi036.stdout: "max_osd": 8, 2022-06-17T12:50:42.547 INFO:teuthology.orchestra.run.smithi036.stdout: "require_min_compat_client": "jewel", 2022-06-17T12:50:42.547 INFO:teuthology.orchestra.run.smithi036.stdout: "min_compat_client": "jewel", 2022-06-17T12:50:42.548 INFO:teuthology.orchestra.run.smithi036.stdout: "require_osd_release": "octopus", 2022-06-17T12:50:42.548 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:50:42.549 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.549 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 1, 2022-06-17T12:50:42.549 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "device_health_metrics", 2022-06-17T12:50:42.550 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:33.881212+0000", 2022-06-17T12:50:42.550 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:50:42.550 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:50:42.550 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:50:42.551 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:50:42.551 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:50:42.551 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:50:42.552 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:50:42.552 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:50:42.552 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 1, 2022-06-17T12:50:42.552 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 1, 2022-06-17T12:50:42.552 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 1, 2022-06-17T12:50:42.553 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 1, 2022-06-17T12:50:42.553 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 1, 2022-06-17T12:50:42.553 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:50:42.554 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:50:42.555 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:50:42.555 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:50:42.556 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:50:42.556 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:50:42.556 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:50:42.556 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.557 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "16", 2022-06-17T12:50:42.557 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:50:42.557 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:50:42.557 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:50:42.558 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:50:42.558 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:50:42.558 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:50:42.559 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:50:42.559 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:50:42.559 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:50:42.560 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:50:42.560 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:50:42.560 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:50:42.560 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:50:42.561 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:50:42.561 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:50:42.586 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:50:42.586 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:50:42.587 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:50:42.587 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:50:42.588 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:50:42.589 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:50:42.590 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:50:42.590 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:50:42.590 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:50:42.590 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:50:42.591 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:50:42.591 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.591 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:50:42.591 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:50:42.591 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:50:42.592 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:50:42.592 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:50:42.592 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:50:42.592 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:50:42.593 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:50:42.593 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:50:42.593 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:50:42.593 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:50:42.594 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:50:42.594 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 1 2022-06-17T12:50:42.594 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.594 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:50:42.594 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr_devicehealth": {} 2022-06-17T12:50:42.595 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.595 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.595 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.596 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 2, 2022-06-17T12:50:42.596 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "rbd", 2022-06-17T12:50:42.596 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:44.485460+0000", 2022-06-17T12:50:42.596 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:50:42.597 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:50:42.597 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:50:42.597 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:50:42.597 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:50:42.597 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:50:42.598 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:50:42.598 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:50:42.598 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:50:42.598 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:50:42.599 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:50:42.599 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:50:42.600 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:50:42.600 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:50:42.600 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:50:42.601 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:50:42.601 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:50:42.601 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:50:42.601 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:50:42.604 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:50:42.605 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.605 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "19", 2022-06-17T12:50:42.607 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:50:42.607 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:50:42.608 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:50:42.608 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:50:42.609 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:50:42.618 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:50:42.618 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:50:42.619 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:50:42.619 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:50:42.620 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:50:42.620 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:50:42.621 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:50:42.621 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:50:42.630 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:50:42.631 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:50:42.632 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:50:42.632 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:50:42.632 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:50:42.633 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:50:42.634 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:50:42.634 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:50:42.635 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:50:42.635 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:50:42.636 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:50:42.636 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:50:42.637 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:50:42.638 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.638 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:50:42.639 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:50:42.639 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:50:42.639 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:50:42.640 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:50:42.640 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:50:42.641 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:50:42.641 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:50:42.642 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:50:42.642 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:50:42.643 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:50:42.643 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:50:42.643 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:50:42.644 INFO:teuthology.orchestra.run.smithi036.stdout: "rbd": {} 2022-06-17T12:50:42.645 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.646 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.647 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:50:42.647 INFO:teuthology.orchestra.run.smithi036.stdout: "osds": [ 2022-06-17T12:50:42.648 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.648 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:50:42.648 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", 2022-06-17T12:50:42.649 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:50:42.649 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:50:42.650 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:50:42.650 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:50:42.650 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:50:42.651 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:50:42.651 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:50:42.652 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 43, 2022-06-17T12:50:42.652 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:50:42.653 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:50:42.653 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:50:42.654 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.655 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.656 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.656 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6802", 2022-06-17T12:50:42.657 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:50:42.657 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.657 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.658 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.658 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6803", 2022-06-17T12:50:42.659 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:50:42.659 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.660 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.660 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.660 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:50:42.661 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.661 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.670 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.671 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6804", 2022-06-17T12:50:42.672 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:50:42.672 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.675 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.676 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.676 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6805", 2022-06-17T12:50:42.676 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:50:42.676 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.683 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.683 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.684 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:50:42.684 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.685 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.685 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.686 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6808", 2022-06-17T12:50:42.686 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:50:42.687 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.687 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.687 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.689 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6809", 2022-06-17T12:50:42.689 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:50:42.689 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.690 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.690 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.690 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:50:42.691 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.691 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.691 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.692 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6806", 2022-06-17T12:50:42.692 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:50:42.693 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.693 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.693 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.694 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6807", 2022-06-17T12:50:42.694 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:50:42.694 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.695 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.695 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.695 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6803/22619", 2022-06-17T12:50:42.696 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6805/22619", 2022-06-17T12:50:42.696 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6809/22619", 2022-06-17T12:50:42.696 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6807/22619", 2022-06-17T12:50:42.696 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:50:42.697 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:50:42.697 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:50:42.697 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.697 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.698 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.698 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:50:42.698 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", 2022-06-17T12:50:42.698 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:50:42.699 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:50:42.699 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:50:42.699 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:50:42.699 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:50:42.699 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:50:42.700 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:50:42.700 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 43, 2022-06-17T12:50:42.700 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:50:42.700 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:50:42.701 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:50:42.701 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.701 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.701 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.702 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6810", 2022-06-17T12:50:42.702 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:50:42.702 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.702 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.703 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.703 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6811", 2022-06-17T12:50:42.703 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:50:42.703 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.704 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.704 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.704 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:50:42.704 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.705 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.705 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.705 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6812", 2022-06-17T12:50:42.705 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:50:42.706 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.706 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.706 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.707 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6813", 2022-06-17T12:50:42.707 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:50:42.708 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.708 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.708 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.709 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:50:42.709 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.710 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.710 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.710 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6816", 2022-06-17T12:50:42.711 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:50:42.711 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.711 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.712 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.712 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6817", 2022-06-17T12:50:42.713 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:50:42.713 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.713 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.714 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.714 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:50:42.714 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.715 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.715 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.716 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6814", 2022-06-17T12:50:42.716 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:50:42.716 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.717 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.717 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.717 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6815", 2022-06-17T12:50:42.718 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:50:42.718 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.718 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.719 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.719 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6811/22620", 2022-06-17T12:50:42.719 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6813/22620", 2022-06-17T12:50:42.720 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6817/22620", 2022-06-17T12:50:42.720 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6815/22620", 2022-06-17T12:50:42.721 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:50:42.721 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:50:42.722 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:50:42.722 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.723 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.724 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.724 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:50:42.725 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", 2022-06-17T12:50:42.725 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:50:42.725 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:50:42.726 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:50:42.727 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:50:42.727 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:50:42.728 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:50:42.728 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:50:42.729 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 32, 2022-06-17T12:50:42.729 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:50:42.730 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:50:42.730 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:50:42.731 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.731 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.732 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.732 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6826", 2022-06-17T12:50:42.733 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:50:42.733 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.733 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.734 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.734 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6827", 2022-06-17T12:50:42.734 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:50:42.734 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.735 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.735 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.736 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:50:42.736 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.736 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.736 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.736 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6828", 2022-06-17T12:50:42.737 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:50:42.737 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.737 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.738 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.738 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6829", 2022-06-17T12:50:42.738 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:50:42.739 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.739 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.739 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.739 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:50:42.739 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.740 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.740 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.740 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6832", 2022-06-17T12:50:42.740 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:50:42.741 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.741 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.741 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.742 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6833", 2022-06-17T12:50:42.742 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:50:42.742 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.742 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.742 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.743 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:50:42.743 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.743 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.743 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.744 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6830", 2022-06-17T12:50:42.744 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:50:42.744 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.744 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.744 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.745 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6831", 2022-06-17T12:50:42.745 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:50:42.745 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.745 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.746 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.746 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6827/22643", 2022-06-17T12:50:42.746 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6829/22643", 2022-06-17T12:50:42.747 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6833/22643", 2022-06-17T12:50:42.747 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6831/22643", 2022-06-17T12:50:42.747 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:50:42.747 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:50:42.747 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:50:42.748 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.748 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.748 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.748 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:50:42.749 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", 2022-06-17T12:50:42.749 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:50:42.749 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:50:42.749 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:50:42.750 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:50:42.750 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:50:42.750 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:50:42.750 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:50:42.751 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 43, 2022-06-17T12:50:42.751 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:50:42.751 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:50:42.751 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:50:42.751 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.752 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.752 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.752 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6818", 2022-06-17T12:50:42.752 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:50:42.753 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.753 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.753 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.753 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6819", 2022-06-17T12:50:42.753 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:50:42.754 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.754 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.754 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.754 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:50:42.755 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.755 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.755 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.755 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6820", 2022-06-17T12:50:42.756 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:50:42.756 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.757 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.757 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.758 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6821", 2022-06-17T12:50:42.759 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:50:42.759 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.759 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.760 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.760 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:50:42.761 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.762 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.762 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.762 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6824", 2022-06-17T12:50:42.762 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:50:42.763 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.763 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.763 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.763 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6825", 2022-06-17T12:50:42.764 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:50:42.764 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.764 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.764 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.765 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:50:42.765 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.765 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.765 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.765 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6822", 2022-06-17T12:50:42.766 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:50:42.766 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.766 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.767 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.767 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6823", 2022-06-17T12:50:42.767 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:50:42.767 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.768 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.768 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.768 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6819/22646", 2022-06-17T12:50:42.768 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6821/22646", 2022-06-17T12:50:42.769 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6825/22646", 2022-06-17T12:50:42.769 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6823/22646", 2022-06-17T12:50:42.769 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:50:42.769 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:50:42.769 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:50:42.770 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.770 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.770 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.770 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:50:42.771 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", 2022-06-17T12:50:42.771 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:50:42.771 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:50:42.771 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:50:42.771 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:50:42.772 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:50:42.772 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:50:42.772 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:50:42.772 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 43, 2022-06-17T12:50:42.773 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:50:42.773 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:50:42.773 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:50:42.773 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.773 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.774 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.774 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6808", 2022-06-17T12:50:42.774 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:50:42.775 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.775 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.775 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.775 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6809", 2022-06-17T12:50:42.776 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:50:42.776 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.776 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.776 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.777 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:50:42.777 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.777 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.777 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.777 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6810", 2022-06-17T12:50:42.778 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:50:42.783 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.783 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.784 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.784 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6811", 2022-06-17T12:50:42.784 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:50:42.785 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.785 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.785 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.785 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:50:42.786 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.786 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.786 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.787 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6814", 2022-06-17T12:50:42.787 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:50:42.788 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.788 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.790 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.790 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6815", 2022-06-17T12:50:42.791 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:50:42.791 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.792 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.792 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.793 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:50:42.793 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.794 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.794 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.794 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6812", 2022-06-17T12:50:42.795 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:50:42.795 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.795 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.795 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.795 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6813", 2022-06-17T12:50:42.796 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:50:42.796 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.796 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.796 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.797 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6809/22812", 2022-06-17T12:50:42.797 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6811/22812", 2022-06-17T12:50:42.797 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6815/22812", 2022-06-17T12:50:42.797 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6813/22812", 2022-06-17T12:50:42.797 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:50:42.798 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:50:42.798 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:50:42.798 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.798 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.799 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.799 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:50:42.799 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", 2022-06-17T12:50:42.799 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:50:42.800 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:50:42.801 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:50:42.801 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:50:42.801 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:50:42.801 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:50:42.802 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:50:42.802 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 43, 2022-06-17T12:50:42.802 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:50:42.802 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:50:42.803 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:50:42.803 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.803 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.803 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.804 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6800", 2022-06-17T12:50:42.804 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:50:42.804 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.804 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.805 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.805 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6801", 2022-06-17T12:50:42.805 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:50:42.805 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.806 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.806 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.806 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:50:42.806 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.807 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.807 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.807 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6802", 2022-06-17T12:50:42.807 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:50:42.807 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.808 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.808 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.809 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6803", 2022-06-17T12:50:42.809 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:50:42.810 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.810 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.820 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.821 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:50:42.821 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.822 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.822 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.823 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6806", 2022-06-17T12:50:42.823 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:50:42.824 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.824 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.825 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.825 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6807", 2022-06-17T12:50:42.826 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:50:42.826 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.826 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.827 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.827 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:50:42.828 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.828 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.830 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.830 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6804", 2022-06-17T12:50:42.830 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:50:42.831 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.831 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.832 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.832 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6805", 2022-06-17T12:50:42.833 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:50:42.833 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.833 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.834 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.834 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6801/22813", 2022-06-17T12:50:42.835 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6803/22813", 2022-06-17T12:50:42.835 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6807/22813", 2022-06-17T12:50:42.836 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6805/22813", 2022-06-17T12:50:42.836 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:50:42.836 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:50:42.837 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:50:42.837 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.838 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.838 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.839 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:50:42.839 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", 2022-06-17T12:50:42.840 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:50:42.840 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:50:42.840 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:50:42.841 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:50:42.841 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:50:42.842 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:50:42.842 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:50:42.843 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 43, 2022-06-17T12:50:42.843 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:50:42.844 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:50:42.844 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:50:42.844 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.845 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.845 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.846 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6816", 2022-06-17T12:50:42.847 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:50:42.848 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.848 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.849 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.849 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6817", 2022-06-17T12:50:42.850 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:50:42.850 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.850 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.851 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.851 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:50:42.852 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.852 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.853 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.853 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6818", 2022-06-17T12:50:42.854 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:50:42.854 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.855 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.855 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.855 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6819", 2022-06-17T12:50:42.856 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:50:42.856 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.856 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.857 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.857 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:50:42.866 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.866 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.867 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.867 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6822", 2022-06-17T12:50:42.868 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:50:42.868 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.869 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.869 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.869 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6823", 2022-06-17T12:50:42.870 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:50:42.870 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.871 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.871 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.872 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:50:42.872 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.872 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.873 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.873 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6820", 2022-06-17T12:50:42.874 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:50:42.874 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.875 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.875 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.876 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6821", 2022-06-17T12:50:42.876 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:50:42.876 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.877 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.877 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.878 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6817/22822", 2022-06-17T12:50:42.879 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6819/22822", 2022-06-17T12:50:42.879 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6823/22822", 2022-06-17T12:50:42.879 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6821/22822", 2022-06-17T12:50:42.880 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:50:42.880 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:50:42.881 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:50:42.881 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.882 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.882 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.883 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:50:42.883 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", 2022-06-17T12:50:42.883 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:50:42.884 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:50:42.884 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:50:42.885 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:50:42.885 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:50:42.886 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:50:42.886 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:50:42.887 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 42, 2022-06-17T12:50:42.887 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:50:42.888 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:50:42.888 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:50:42.889 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.889 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.889 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.890 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6824", 2022-06-17T12:50:42.890 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:50:42.891 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.891 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.892 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.892 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6825", 2022-06-17T12:50:42.892 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:50:42.893 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.893 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.893 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.894 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:50:42.894 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.895 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.895 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.895 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6826", 2022-06-17T12:50:42.896 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:50:42.896 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.896 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.896 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.897 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6827", 2022-06-17T12:50:42.897 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:50:42.897 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.897 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.897 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.898 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:50:42.898 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.898 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.899 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.899 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6830", 2022-06-17T12:50:42.899 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:50:42.899 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.899 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.900 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.900 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6831", 2022-06-17T12:50:42.900 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:50:42.900 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.901 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.901 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.901 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:50:42.901 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:50:42.901 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.902 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:50:42.902 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6828", 2022-06-17T12:50:42.902 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:50:42.902 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.903 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.903 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:50:42.903 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6829", 2022-06-17T12:50:42.903 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:50:42.903 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.904 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.904 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.904 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6825/22837", 2022-06-17T12:50:42.904 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6827/22837", 2022-06-17T12:50:42.905 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6831/22837", 2022-06-17T12:50:42.905 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6829/22837", 2022-06-17T12:50:42.905 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:50:42.905 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:50:42.906 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:50:42.906 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:42.907 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.907 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:50:42.908 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_xinfo": [ 2022-06-17T12:50:42.908 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.908 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:50:42.909 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:50:42.909 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:50:42.910 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:50:42.910 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:50:42.910 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:50:42.911 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.291461+0000", 2022-06-17T12:50:42.911 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:50:42.911 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.912 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.912 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:50:42.912 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:50:42.913 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:50:42.913 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:50:42.913 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:50:42.914 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:50:42.914 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.537912+0000", 2022-06-17T12:50:42.915 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:50:42.915 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.915 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.916 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:50:42.916 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:50:42.917 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:50:42.917 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:50:42.917 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:50:42.918 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:50:42.918 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.606947+0000", 2022-06-17T12:50:42.918 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:50:42.919 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.919 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.919 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:50:42.920 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:50:42.920 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:50:42.920 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:50:42.921 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:50:42.921 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:50:42.921 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.514412+0000", 2022-06-17T12:50:42.922 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:50:42.922 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.923 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.923 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:50:42.923 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:50:42.924 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:50:42.924 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:50:42.924 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:50:42.925 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:50:42.925 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.394645+0000", 2022-06-17T12:50:42.926 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:50:42.926 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.926 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.927 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:50:42.927 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:50:42.927 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:50:42.928 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:50:42.928 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:50:42.929 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:50:42.929 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.343806+0000", 2022-06-17T12:50:42.929 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:50:42.930 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.930 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.931 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:50:42.931 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:50:42.931 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:50:42.932 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:50:42.932 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:50:42.933 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:50:42.933 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.591864+0000", 2022-06-17T12:50:42.934 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:50:42.934 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.936 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:42.937 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:50:42.937 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:50:42.938 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:50:42.938 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:50:42.939 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:50:42.939 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:50:42.940 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.607045+0000", 2022-06-17T12:50:42.940 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:50:42.941 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.941 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:50:42.941 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap": [], 2022-06-17T12:50:42.942 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap_items": [], 2022-06-17T12:50:42.942 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_temp": [], 2022-06-17T12:50:42.943 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_temp": [], 2022-06-17T12:50:42.943 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklist": { 2022-06-17T12:50:42.943 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.70:6833/4117532324": "2022-06-18T12:50:36.154766+0000", 2022-06-17T12:50:42.944 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.70:6832/4117532324": "2022-06-18T12:50:36.154766+0000" 2022-06-17T12:50:42.944 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.944 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profiles": { 2022-06-17T12:50:42.945 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs_data": { 2022-06-17T12:50:42.945 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-device-class": "", 2022-06-17T12:50:42.946 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:50:42.947 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-root": "default", 2022-06-17T12:50:42.948 INFO:teuthology.orchestra.run.smithi036.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-17T12:50:42.948 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:50:42.949 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "2", 2022-06-17T12:50:42.949 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:50:42.950 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:50:42.950 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van", 2022-06-17T12:50:42.951 INFO:teuthology.orchestra.run.smithi036.stdout: "w": "8" 2022-06-17T12:50:42.951 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.952 INFO:teuthology.orchestra.run.smithi036.stdout: "default": { 2022-06-17T12:50:42.952 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:50:42.953 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:50:42.953 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "1", 2022-06-17T12:50:42.953 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:50:42.954 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:50:42.954 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van" 2022-06-17T12:50:42.955 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:42.955 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:42.956 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps_queue": [], 2022-06-17T12:50:42.956 INFO:teuthology.orchestra.run.smithi036.stdout: "new_removed_snaps": [], 2022-06-17T12:50:42.957 INFO:teuthology.orchestra.run.smithi036.stdout: "new_purged_snaps": [], 2022-06-17T12:50:42.958 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_node_flags": {}, 2022-06-17T12:50:42.959 INFO:teuthology.orchestra.run.smithi036.stdout: "device_class_flags": {} 2022-06-17T12:50:42.959 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:50:42.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.549+0000 7efe2e7f2700 1 -- 172.21.15.36:0/3143200255 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efe08053920 msgr2=0x7efe08055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:42.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.549+0000 7efe2e7f2700 1 --2- 172.21.15.36:0/3143200255 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efe08053920 0x7efe08055de0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7efe2401a000 tx=0x7efe24014000).stop 2022-06-17T12:50:42.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.549+0000 7efe2e7f2700 1 -- 172.21.15.36:0/3143200255 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe280b6860 msgr2=0x7efe280afa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:42.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.549+0000 7efe2e7f2700 1 --2- 172.21.15.36:0/3143200255 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe280b6860 0x7efe280afa10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7efe20006e00 tx=0x7efe200091d0).stop 2022-06-17T12:50:42.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.549+0000 7efe2e7f2700 1 -- 172.21.15.36:0/3143200255 shutdown_connections 2022-06-17T12:50:42.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.549+0000 7efe2e7f2700 1 --2- 172.21.15.36:0/3143200255 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efe08053920 0x7efe08055de0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:42.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.549+0000 7efe2e7f2700 1 --2- 172.21.15.36:0/3143200255 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe280b6860 0x7efe280afa10 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:42.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.549+0000 7efe2e7f2700 1 --2- 172.21.15.36:0/3143200255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe280b39a0 0x7efe28063460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:42.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.549+0000 7efe2e7f2700 1 --2- 172.21.15.36:0/3143200255 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe280b18e0 0x7efe28152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:42.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.549+0000 7efe2e7f2700 1 -- 172.21.15.36:0/3143200255 >> 172.21.15.36:0/3143200255 conn(0x7efe280614b0 msgr2=0x7efe280b5330 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:42.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.549+0000 7efe2e7f2700 1 -- 172.21.15.36:0/3143200255 shutdown_connections 2022-06-17T12:50:42.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:42.549+0000 7efe2e7f2700 1 -- 172.21.15.36:0/3143200255 wait complete. 2022-06-17T12:50:42.971 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:50:43.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.105+0000 7f5a1aee3700 1 Processor -- start 2022-06-17T12:50:43.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.105+0000 7f5a1aee3700 1 -- start start 2022-06-17T12:50:43.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a1aee3700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5a140b7b60 0x7f5a140b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:43.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a1aee3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5a14061eb0 0x7f5a14143af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:43.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a1aee3700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5a14143fe0 0x7f5a1413cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:43.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a1aee3700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a140b7f40 con 0x7f5a140b7b60 2022-06-17T12:50:43.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a1aee3700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a140b3670 con 0x7f5a14143fe0 2022-06-17T12:50:43.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a1aee3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a141433b0 con 0x7f5a14061eb0 2022-06-17T12:50:43.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a18c7f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5a140b7b60 0x7f5a140b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:43.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a18c7f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5a140b7b60 0x7f5a140b3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33908/0 (socket says 172.21.15.36:33908) 2022-06-17T12:50:43.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a18c7f700 1 -- 172.21.15.36:0/2790158158 learned_addr learned my addr 172.21.15.36:0/2790158158 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:43.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a19480700 1 --2- 172.21.15.36:0/2790158158 >> v2:172.21.15.70:3300/0 conn(0x7f5a14143fe0 0x7f5a1413cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:43.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a0bfff700 1 --2- 172.21.15.36:0/2790158158 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5a14061eb0 0x7f5a14143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:43.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a18c7f700 1 -- 172.21.15.36:0/2790158158 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5a14061eb0 msgr2=0x7f5a14143af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:43.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a18c7f700 1 --2- 172.21.15.36:0/2790158158 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5a14061eb0 0x7f5a14143af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a18c7f700 1 -- 172.21.15.36:0/2790158158 >> v2:172.21.15.70:3300/0 conn(0x7f5a14143fe0 msgr2=0x7f5a1413cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:43.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a18c7f700 1 --2- 172.21.15.36:0/2790158158 >> v2:172.21.15.70:3300/0 conn(0x7f5a14143fe0 0x7f5a1413cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a18c7f700 1 -- 172.21.15.36:0/2790158158 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a14143650 con 0x7f5a140b7b60 2022-06-17T12:50:43.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a0bfff700 1 --2- 172.21.15.36:0/2790158158 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5a14061eb0 0x7f5a14143af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:50:43.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a18c7f700 1 --2- 172.21.15.36:0/2790158158 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a140b7b60 0x7f5a140b3e50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f5a14063340 tx=0x7f5a10006eb0).ready entity=mon.0 client_cookie=a475d83a57b8e4a6 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:43.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a0b7fe700 1 -- 172.21.15.36:0/2790158158 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a1000d570 con 0x7f5a140b7b60 2022-06-17T12:50:43.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a0b7fe700 1 -- 172.21.15.36:0/2790158158 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a10000e90 con 0x7f5a140b7b60 2022-06-17T12:50:43.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a0b7fe700 1 -- 172.21.15.36:0/2790158158 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a1000d900 con 0x7f5a140b7b60 2022-06-17T12:50:43.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a1aee3700 1 -- 172.21.15.36:0/2790158158 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a140b7b60 msgr2=0x7f5a140b3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:43.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a1aee3700 1 --2- 172.21.15.36:0/2790158158 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a140b7b60 0x7f5a140b3e50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f5a14063340 tx=0x7f5a10006eb0).stop 2022-06-17T12:50:43.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a1aee3700 1 -- 172.21.15.36:0/2790158158 shutdown_connections 2022-06-17T12:50:43.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a1aee3700 1 --2- 172.21.15.36:0/2790158158 >> v2:172.21.15.70:3300/0 conn(0x7f5a14143fe0 0x7f5a1413cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a1aee3700 1 --2- 172.21.15.36:0/2790158158 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5a14061eb0 0x7f5a14143af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a1aee3700 1 --2- 172.21.15.36:0/2790158158 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a140b7b60 0x7f5a140b3e50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a1aee3700 1 -- 172.21.15.36:0/2790158158 >> 172.21.15.36:0/2790158158 conn(0x7f5a14060c60 msgr2=0x7f5a1405c550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:43.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a1aee3700 1 -- 172.21.15.36:0/2790158158 shutdown_connections 2022-06-17T12:50:43.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.109+0000 7f5a1aee3700 1 -- 172.21.15.36:0/2790158158 wait complete. 2022-06-17T12:50:43.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a1aee3700 1 Processor -- start 2022-06-17T12:50:43.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a1aee3700 1 -- start start 2022-06-17T12:50:43.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a1aee3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a14061eb0 0x7f5a14145a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:43.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a1aee3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5a140b7b60 0x7f5a14066800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:43.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a1aee3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a14143fe0 0x7f5a14064680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:43.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a1aee3700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a14152a70 con 0x7f5a14061eb0 2022-06-17T12:50:43.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a1aee3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a14152d10 con 0x7f5a14143fe0 2022-06-17T12:50:43.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a1aee3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a14152730 con 0x7f5a140b7b60 2022-06-17T12:50:43.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a18c7f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a14061eb0 0x7f5a14145a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:43.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a18c7f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a14061eb0 0x7f5a14145a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33916/0 (socket says 172.21.15.36:33916) 2022-06-17T12:50:43.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a18c7f700 1 -- 172.21.15.36:0/1420459868 learned_addr learned my addr 172.21.15.36:0/1420459868 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:43.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a19480700 1 --2- 172.21.15.36:0/1420459868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a14143fe0 0x7f5a14064680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:43.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a0bfff700 1 --2- 172.21.15.36:0/1420459868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5a140b7b60 0x7f5a14066800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:43.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a18c7f700 1 -- 172.21.15.36:0/1420459868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5a140b7b60 msgr2=0x7f5a14066800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:43.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a18c7f700 1 --2- 172.21.15.36:0/1420459868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5a140b7b60 0x7f5a14066800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a18c7f700 1 -- 172.21.15.36:0/1420459868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a14143fe0 msgr2=0x7f5a14064680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:43.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a18c7f700 1 --2- 172.21.15.36:0/1420459868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a14143fe0 0x7f5a14064680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a18c7f700 1 -- 172.21.15.36:0/1420459868 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1000b1e0 con 0x7f5a14061eb0 2022-06-17T12:50:43.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a0bfff700 1 --2- 172.21.15.36:0/1420459868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5a140b7b60 0x7f5a14066800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:50:43.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a18c7f700 1 --2- 172.21.15.36:0/1420459868 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a14061eb0 0x7f5a14145a40 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f5a14063340 tx=0x7f5a10015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:43.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a09ffb700 1 -- 172.21.15.36:0/1420459868 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a10011690 con 0x7f5a14061eb0 2022-06-17T12:50:43.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a09ffb700 1 -- 172.21.15.36:0/1420459868 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a100117f0 con 0x7f5a14061eb0 2022-06-17T12:50:43.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a1aee3700 1 -- 172.21.15.36:0/1420459868 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a14064b70 con 0x7f5a14061eb0 2022-06-17T12:50:43.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a09ffb700 1 -- 172.21.15.36:0/1420459868 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a10011b50 con 0x7f5a14061eb0 2022-06-17T12:50:43.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.113+0000 7f5a1aee3700 1 -- 172.21.15.36:0/1420459868 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a140670e0 con 0x7f5a14061eb0 2022-06-17T12:50:43.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.117+0000 7f5a1aee3700 1 -- 172.21.15.36:0/1420459868 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a14066cf0 con 0x7f5a14061eb0 2022-06-17T12:50:43.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.117+0000 7f5a09ffb700 1 -- 172.21.15.36:0/1420459868 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f5a10025030 con 0x7f5a14061eb0 2022-06-17T12:50:43.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.117+0000 7f5a09ffb700 1 --2- 172.21.15.36:0/1420459868 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f59f40539b0 0x7f59f4055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:43.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.117+0000 7f5a09ffb700 1 -- 172.21.15.36:0/1420459868 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f5a1006cc80 con 0x7f5a14061eb0 2022-06-17T12:50:43.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.117+0000 7f5a0bfff700 1 --2- 172.21.15.36:0/1420459868 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f59f40539b0 0x7f59f4055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:43.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.117+0000 7f5a0bfff700 1 --2- 172.21.15.36:0/1420459868 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f59f40539b0 0x7f59f4055e70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f5a0400c000 tx=0x7f5a04008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:43.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.121+0000 7f5a09ffb700 1 -- 172.21.15.36:0/1420459868 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5a10043d60 con 0x7f5a14061eb0 2022-06-17T12:50:43.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.289+0000 7f5a1aee3700 1 -- 172.21.15.36:0/1420459868 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a14152ed0 con 0x7f5a14061eb0 2022-06-17T12:50:43.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.289+0000 7f5a09ffb700 1 -- 172.21.15.36:0/1420459868 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+1209 (secure 0 0 0) 0x7f5a10023070 con 0x7f5a14061eb0 2022-06-17T12:50:43.275 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:50:43.275 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":21,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1295481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1295481638},{"type":"v1","addr":"172.21.15.70:6833","nonce":1295481638}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/3777038413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3777038413},{"type":"v1","addr":"172.21.15.36:6835","nonce":3777038413}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":17}],"filesystems":[]} 2022-06-17T12:50:43.278 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.289+0000 7f5a1aee3700 1 -- 172.21.15.36:0/1420459868 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f59f40539b0 msgr2=0x7f59f4055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:43.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.289+0000 7f5a1aee3700 1 --2- 172.21.15.36:0/1420459868 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f59f40539b0 0x7f59f4055e70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f5a0400c000 tx=0x7f5a04008040).stop 2022-06-17T12:50:43.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.289+0000 7f5a1aee3700 1 -- 172.21.15.36:0/1420459868 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a14061eb0 msgr2=0x7f5a14145a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:43.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.289+0000 7f5a1aee3700 1 --2- 172.21.15.36:0/1420459868 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a14061eb0 0x7f5a14145a40 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f5a14063340 tx=0x7f5a10015000).stop 2022-06-17T12:50:43.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.289+0000 7f5a1aee3700 1 -- 172.21.15.36:0/1420459868 shutdown_connections 2022-06-17T12:50:43.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.289+0000 7f5a1aee3700 1 --2- 172.21.15.36:0/1420459868 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f59f40539b0 0x7f59f4055e70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.289+0000 7f5a1aee3700 1 --2- 172.21.15.36:0/1420459868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a14143fe0 0x7f5a14064680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.289+0000 7f5a1aee3700 1 --2- 172.21.15.36:0/1420459868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5a140b7b60 0x7f5a14066800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.289+0000 7f5a1aee3700 1 --2- 172.21.15.36:0/1420459868 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a14061eb0 0x7f5a14145a40 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.289+0000 7f5a1aee3700 1 -- 172.21.15.36:0/1420459868 >> 172.21.15.36:0/1420459868 conn(0x7f5a14060c60 msgr2=0x7f5a1413ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:43.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.289+0000 7f5a1aee3700 1 -- 172.21.15.36:0/1420459868 shutdown_connections 2022-06-17T12:50:43.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.289+0000 7f5a1aee3700 1 -- 172.21.15.36:0/1420459868 wait complete. 2022-06-17T12:50:43.284 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 21 2022-06-17T12:50:43.287 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:50:43.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.413+0000 7f64c3289700 1 Processor -- start 2022-06-17T12:50:43.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.413+0000 7f64c3289700 1 -- start start 2022-06-17T12:50:43.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c3289700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64bc0b2200 0x7f64bc0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:43.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c3289700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f64bc0af8d0 0x7f64bc14a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:43.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c3289700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64bc0b25e0 con 0x7f64bc0b2cd0 2022-06-17T12:50:43.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c3289700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64bc0ad790 con 0x7f64bc0af8d0 2022-06-17T12:50:43.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c3289700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc0aef40 con 0x7f64bc0b2200 2022-06-17T12:50:43.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c1a86700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f64bc0af8d0 0x7f64bc14a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:43.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c1a86700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f64bc0af8d0 0x7f64bc14a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33922/0 (socket says 172.21.15.36:33922) 2022-06-17T12:50:43.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c1a86700 1 -- 172.21.15.36:0/3063063072 learned_addr learned my addr 172.21.15.36:0/3063063072 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:43.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c2287700 1 --2- 172.21.15.36:0/3063063072 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64bc0b2200 0x7f64bc0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:43.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c1a86700 1 -- 172.21.15.36:0/3063063072 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64bc0b2200 msgr2=0x7f64bc0af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:43.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c1a86700 1 --2- 172.21.15.36:0/3063063072 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64bc0b2200 0x7f64bc0af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c1a86700 1 -- 172.21.15.36:0/3063063072 >> v1:172.21.15.70:6789/0 conn(0x7f64bc0b2cd0 legacy=0x7f64bc14adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:43.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c1a86700 1 -- 172.21.15.36:0/3063063072 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64bc0b1950 con 0x7f64bc0af8d0 2022-06-17T12:50:43.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c2287700 1 --2- 172.21.15.36:0/3063063072 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64bc0b2200 0x7f64bc0af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:50:43.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c1a86700 1 --2- 172.21.15.36:0/3063063072 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64bc0af8d0 0x7f64bc14a8d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f64b80032f0 tx=0x7f64b8009290).ready entity=mon.0 client_cookie=1143f946c635b6e3 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:43.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c1285700 1 -- 172.21.15.36:0/3063063072 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64b800b8a0 con 0x7f64bc0af8d0 2022-06-17T12:50:43.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c1285700 1 -- 172.21.15.36:0/3063063072 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b800ba00 con 0x7f64bc0af8d0 2022-06-17T12:50:43.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c1285700 1 -- 172.21.15.36:0/3063063072 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64b800bb70 con 0x7f64bc0af8d0 2022-06-17T12:50:43.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c3289700 1 -- 172.21.15.36:0/3063063072 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64bc0af8d0 msgr2=0x7f64bc14a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:43.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c3289700 1 --2- 172.21.15.36:0/3063063072 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64bc0af8d0 0x7f64bc14a8d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f64b80032f0 tx=0x7f64b8009290).stop 2022-06-17T12:50:43.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c3289700 1 -- 172.21.15.36:0/3063063072 shutdown_connections 2022-06-17T12:50:43.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c3289700 1 --2- 172.21.15.36:0/3063063072 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64bc0af8d0 0x7f64bc14a8d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c3289700 1 --2- 172.21.15.36:0/3063063072 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64bc0b2200 0x7f64bc0af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c3289700 1 -- 172.21.15.36:0/3063063072 >> 172.21.15.36:0/3063063072 conn(0x7f64bc05f770 msgr2=0x7f64bc05fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:43.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c3289700 1 -- 172.21.15.36:0/3063063072 shutdown_connections 2022-06-17T12:50:43.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c3289700 1 -- 172.21.15.36:0/3063063072 wait complete. 2022-06-17T12:50:43.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c3289700 1 Processor -- start 2022-06-17T12:50:43.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.417+0000 7f64c3289700 1 -- start start 2022-06-17T12:50:43.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.421+0000 7f64c3289700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64bc0af8d0 0x7f64bc146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:43.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.421+0000 7f64c3289700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64bc0b2200 0x7f64bc062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:43.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.421+0000 7f64c3289700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64bc0b2cd0 0x7f64bc0630a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:43.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.421+0000 7f64c3289700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc0a8c40 con 0x7f64bc0b2cd0 2022-06-17T12:50:43.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.421+0000 7f64c3289700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc0a90f0 con 0x7f64bc0af8d0 2022-06-17T12:50:43.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.421+0000 7f64c3289700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc147540 con 0x7f64bc0b2200 2022-06-17T12:50:43.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.421+0000 7f64c2a88700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64bc0b2cd0 0x7f64bc0630a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:43.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.421+0000 7f64c2a88700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64bc0b2cd0 0x7f64bc0630a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33932/0 (socket says 172.21.15.36:33932) 2022-06-17T12:50:43.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.421+0000 7f64c2a88700 1 -- 172.21.15.36:0/3633690421 learned_addr learned my addr 172.21.15.36:0/3633690421 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:43.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.421+0000 7f64c1a86700 1 --2- 172.21.15.36:0/3633690421 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64bc0b2200 0x7f64bc062bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:43.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.421+0000 7f64c2287700 1 --2- 172.21.15.36:0/3633690421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64bc0af8d0 0x7f64bc146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:43.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.421+0000 7f64c2a88700 1 -- 172.21.15.36:0/3633690421 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64bc0b2200 msgr2=0x7f64bc062bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:43.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.421+0000 7f64c2a88700 1 --2- 172.21.15.36:0/3633690421 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64bc0b2200 0x7f64bc062bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.421+0000 7f64c2a88700 1 -- 172.21.15.36:0/3633690421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64bc0af8d0 msgr2=0x7f64bc146bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:43.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.421+0000 7f64c2a88700 1 --2- 172.21.15.36:0/3633690421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64bc0af8d0 0x7f64bc146bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.421+0000 7f64c2a88700 1 -- 172.21.15.36:0/3633690421 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b8009530 con 0x7f64bc0b2cd0 2022-06-17T12:50:43.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.421+0000 7f64c2a88700 1 --2- 172.21.15.36:0/3633690421 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64bc0b2cd0 0x7f64bc0630a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f64bc0afcd0 tx=0x7f64b4007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:43.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.421+0000 7f64ab7fe700 1 -- 172.21.15.36:0/3633690421 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64b4007e40 con 0x7f64bc0b2cd0 2022-06-17T12:50:43.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.421+0000 7f64ab7fe700 1 -- 172.21.15.36:0/3633690421 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b4007030 con 0x7f64bc0b2cd0 2022-06-17T12:50:43.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.421+0000 7f64ab7fe700 1 -- 172.21.15.36:0/3633690421 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64b4007390 con 0x7f64bc0b2cd0 2022-06-17T12:50:43.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.421+0000 7f64c3289700 1 -- 172.21.15.36:0/3633690421 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64bc0aac40 con 0x7f64bc0b2cd0 2022-06-17T12:50:43.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.421+0000 7f64c3289700 1 -- 172.21.15.36:0/3633690421 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64bc063590 con 0x7f64bc0b2cd0 2022-06-17T12:50:43.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.421+0000 7f64c3289700 1 -- 172.21.15.36:0/3633690421 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64bc0a9ae0 con 0x7f64bc0b2cd0 2022-06-17T12:50:43.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.425+0000 7f64ab7fe700 1 -- 172.21.15.36:0/3633690421 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f64b4006510 con 0x7f64bc0b2cd0 2022-06-17T12:50:43.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.425+0000 7f64ab7fe700 1 --2- 172.21.15.36:0/3633690421 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f649c053a40 0x7f649c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:43.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.425+0000 7f64ab7fe700 1 -- 172.21.15.36:0/3633690421 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f64b406b540 con 0x7f64bc0b2cd0 2022-06-17T12:50:43.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.425+0000 7f64c2287700 1 --2- 172.21.15.36:0/3633690421 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f649c053a40 0x7f649c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:43.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.425+0000 7f64c2287700 1 --2- 172.21.15.36:0/3633690421 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f649c053a40 0x7f649c055f00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f64ac00d000 tx=0x7f64ac008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:43.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.429+0000 7f64ab7fe700 1 -- 172.21.15.36:0/3633690421 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f64b4042520 con 0x7f64bc0b2cd0 2022-06-17T12:50:43.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.593+0000 7f64c3289700 1 -- 172.21.15.36:0/3633690421 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64bc0ad330 con 0x7f64bc0b2cd0 2022-06-17T12:50:43.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.593+0000 7f64ab7fe700 1 -- 172.21.15.36:0/3633690421 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+1209 (secure 0 0 0) 0x7f64b4045b40 con 0x7f64bc0b2cd0 2022-06-17T12:50:43.594 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:50:43.594 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":21,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1295481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1295481638},{"type":"v1","addr":"172.21.15.70:6833","nonce":1295481638}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/3777038413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3777038413},{"type":"v1","addr":"172.21.15.36:6835","nonce":3777038413}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":17}],"filesystems":[]} 2022-06-17T12:50:43.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.597+0000 7f64c3289700 1 -- 172.21.15.36:0/3633690421 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f649c053a40 msgr2=0x7f649c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:43.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.597+0000 7f64c3289700 1 --2- 172.21.15.36:0/3633690421 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f649c053a40 0x7f649c055f00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f64ac00d000 tx=0x7f64ac008040).stop 2022-06-17T12:50:43.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.597+0000 7f64c3289700 1 -- 172.21.15.36:0/3633690421 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64bc0b2cd0 msgr2=0x7f64bc0630a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:43.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.597+0000 7f64c3289700 1 --2- 172.21.15.36:0/3633690421 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64bc0b2cd0 0x7f64bc0630a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f64bc0afcd0 tx=0x7f64b4007bf0).stop 2022-06-17T12:50:43.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.597+0000 7f64c3289700 1 -- 172.21.15.36:0/3633690421 shutdown_connections 2022-06-17T12:50:43.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.597+0000 7f64c3289700 1 --2- 172.21.15.36:0/3633690421 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f649c053a40 0x7f649c055f00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.602 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.597+0000 7f64c3289700 1 --2- 172.21.15.36:0/3633690421 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64bc0b2cd0 0x7f64bc0630a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.602 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.597+0000 7f64c3289700 1 --2- 172.21.15.36:0/3633690421 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64bc0b2200 0x7f64bc062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.602 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.597+0000 7f64c3289700 1 --2- 172.21.15.36:0/3633690421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64bc0af8d0 0x7f64bc146bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.602 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.597+0000 7f64c3289700 1 -- 172.21.15.36:0/3633690421 >> 172.21.15.36:0/3633690421 conn(0x7f64bc05f770 msgr2=0x7f64bc0b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:43.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.597+0000 7f64c3289700 1 -- 172.21.15.36:0/3633690421 shutdown_connections 2022-06-17T12:50:43.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.597+0000 7f64c3289700 1 -- 172.21.15.36:0/3633690421 wait complete. 2022-06-17T12:50:43.603 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 21 2022-06-17T12:50:43.604 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-17T12:50:43.604 INFO:tasks.ceph.mds.a:Stopping old one... 2022-06-17T12:50:43.605 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-17T12:50:43.605 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T12:50:43.611 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-17T12:50:43.612 INFO:tasks.ceph.mds.b:Stopping old one... 2022-06-17T12:50:43.613 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-17T12:50:43.613 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T12:50:43.618 INFO:tasks.ceph.mds.a:Stopped 2022-06-17T12:50:43.619 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-17T12:50:43.625 INFO:tasks.ceph.mds.a:Started 2022-06-17T12:50:43.626 DEBUG:teuthology.parallel:result is None 2022-06-17T12:50:43.639 INFO:tasks.ceph.mds.b:Stopped 2022-06-17T12:50:43.640 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-17T12:50:43.701 INFO:tasks.ceph.mds.a.smithi036.stdout:starting mds.a at 2022-06-17T12:50:43.704 INFO:tasks.ceph.mds.b:Started 2022-06-17T12:50:43.715 DEBUG:teuthology.parallel:result is None 2022-06-17T12:50:43.716 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-06-17T12:50:43.783 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2022-06-17T12:50:43.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd90b25c700 1 Processor -- start 2022-06-17T12:50:43.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd90b25c700 1 -- start start 2022-06-17T12:50:43.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd90b25c700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd9040b22b0 0x7fd904064840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:43.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd90b25c700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd904060ea0 con 0x7fd904140e70 2022-06-17T12:50:43.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd90b25c700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9041445f0 con 0x7fd904060ac0 2022-06-17T12:50:43.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd90b25c700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd904144890 con 0x7fd9040b22b0 2022-06-17T12:50:43.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9097f9700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fd904140e70 0x7fd90413ddc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51480/0 (socket says 172.21.15.36:51480) 2022-06-17T12:50:43.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9097f9700 1 -- 172.21.15.36:0/3812310201 learned_addr learned my addr 172.21.15.36:0/3812310201 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:50:43.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd903fff700 1 --2- 172.21.15.36:0/3812310201 >> v2:172.21.15.70:3300/0 conn(0x7fd9040b22b0 0x7fd904064840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:43.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9037fe700 1 -- 172.21.15.36:0/3812310201 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3402366452 0 0) 0x7fd904060ea0 con 0x7fd904140e70 2022-06-17T12:50:43.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9037fe700 1 -- 172.21.15.36:0/3812310201 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8f0003260 con 0x7fd904140e70 2022-06-17T12:50:43.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9037fe700 1 -- 172.21.15.36:0/3812310201 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4122674951 0 0) 0x7fd9041445f0 con 0x7fd904060ac0 2022-06-17T12:50:43.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9037fe700 1 -- 172.21.15.36:0/3812310201 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd904060ea0 con 0x7fd904060ac0 2022-06-17T12:50:43.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9037fe700 1 -- 172.21.15.36:0/3812310201 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 39087341 0 0) 0x7fd8f0003260 con 0x7fd904140e70 2022-06-17T12:50:43.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9037fe700 1 -- 172.21.15.36:0/3812310201 >> v2:172.21.15.70:3300/0 conn(0x7fd9040b22b0 msgr2=0x7fd904064840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:43.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9037fe700 1 --2- 172.21.15.36:0/3812310201 >> v2:172.21.15.70:3300/0 conn(0x7fd9040b22b0 0x7fd904064840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9037fe700 1 -- 172.21.15.36:0/3812310201 >> v1:172.21.15.70:6789/0 conn(0x7fd904060ac0 legacy=0x7fd904144d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:43.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9037fe700 1 -- 172.21.15.36:0/3812310201 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9040b0910 con 0x7fd904140e70 2022-06-17T12:50:43.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9037fe700 1 -- 172.21.15.36:0/3812310201 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7fd8fc0019e0 con 0x7fd904140e70 2022-06-17T12:50:43.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9037fe700 1 -- 172.21.15.36:0/3812310201 >> v1:172.21.15.36:6789/0 conn(0x7fd904140e70 legacy=0x7fd90413ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:43.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9037fe700 1 --2- 172.21.15.36:0/3812310201 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8f00038e0 0x7fd8f0003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:43.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9037fe700 1 --2- 172.21.15.36:0/3812310201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8f0004500 0x7fd8f0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:43.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9037fe700 1 --2- 172.21.15.36:0/3812310201 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8f0004e70 0x7fd8f000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:43.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9037fe700 1 -- 172.21.15.36:0/3812310201 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd904144890 con 0x7fd8f0004e70 2022-06-17T12:50:43.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9037fe700 1 -- 172.21.15.36:0/3812310201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f000bcf0 con 0x7fd8f0004500 2022-06-17T12:50:43.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9037fe700 1 -- 172.21.15.36:0/3812310201 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f000beb0 con 0x7fd8f00038e0 2022-06-17T12:50:43.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9097f9700 1 --2- 172.21.15.36:0/3812310201 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8f0004e70 0x7fd8f000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:43.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd908ff8700 1 --2- 172.21.15.36:0/3812310201 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8f00038e0 0x7fd8f0003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:43.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd903fff700 1 --2- 172.21.15.36:0/3812310201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8f0004500 0x7fd8f0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:43.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9097f9700 1 -- 172.21.15.36:0/3812310201 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8f00038e0 msgr2=0x7fd8f0003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:43.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9097f9700 1 --2- 172.21.15.36:0/3812310201 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8f00038e0 0x7fd8f0003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9097f9700 1 -- 172.21.15.36:0/3812310201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8f0004500 msgr2=0x7fd8f0004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:43.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9097f9700 1 --2- 172.21.15.36:0/3812310201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8f0004500 0x7fd8f0004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9097f9700 1 -- 172.21.15.36:0/3812310201 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f0003260 con 0x7fd8f0004e70 2022-06-17T12:50:43.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd903fff700 1 --2- 172.21.15.36:0/3812310201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8f0004500 0x7fd8f0004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:50:43.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9097f9700 1 -- 172.21.15.36:0/3812310201 reap_dead start 2022-06-17T12:50:43.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9097f9700 1 --2- 172.21.15.36:0/3812310201 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8f0004e70 0x7fd8f000b750 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7fd8fc0017b0 tx=0x7fd8fc002150).ready entity=mon.0 client_cookie=a083b09e49749982 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:43.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9037fe700 1 -- 172.21.15.36:0/3812310201 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8fc00cb50 con 0x7fd8f0004e70 2022-06-17T12:50:43.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9037fe700 1 -- 172.21.15.36:0/3812310201 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8fc00ccb0 con 0x7fd8f0004e70 2022-06-17T12:50:43.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.841+0000 7fd9037fe700 1 -- 172.21.15.36:0/3812310201 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8fc005b80 con 0x7fd8f0004e70 2022-06-17T12:50:43.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd90b25c700 1 -- 172.21.15.36:0/3812310201 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8f0004e70 msgr2=0x7fd8f000b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:43.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd90b25c700 1 --2- 172.21.15.36:0/3812310201 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8f0004e70 0x7fd8f000b750 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7fd8fc0017b0 tx=0x7fd8fc002150).stop 2022-06-17T12:50:43.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd90b25c700 1 -- 172.21.15.36:0/3812310201 shutdown_connections 2022-06-17T12:50:43.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd90b25c700 1 --2- 172.21.15.36:0/3812310201 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8f0004e70 0x7fd8f000b750 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd90b25c700 1 -- 172.21.15.36:0/3812310201 >> 172.21.15.36:0/3812310201 conn(0x7fd9040b6870 msgr2=0x7fd9040b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:43.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd90b25c700 1 -- 172.21.15.36:0/3812310201 shutdown_connections 2022-06-17T12:50:43.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd90b25c700 1 -- 172.21.15.36:0/3812310201 wait complete. 2022-06-17T12:50:43.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd90b25c700 1 Processor -- start 2022-06-17T12:50:43.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd90b25c700 1 -- start start 2022-06-17T12:50:43.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd90b25c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8f0004e70 0x7fd90409ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:43.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd90b25c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd904067a80 0x7fd904067240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:43.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd90b25c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9040ac900 0x7fd9040a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:43.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd90b25c700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd904066000 con 0x7fd8f0004e70 2022-06-17T12:50:43.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd90b25c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd904066230 con 0x7fd9040ac900 2022-06-17T12:50:43.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd90b25c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9040664d0 con 0x7fd904067a80 2022-06-17T12:50:43.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd908ff8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8f0004e70 0x7fd90409ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:43.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd908ff8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8f0004e70 0x7fd90409ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33960/0 (socket says 172.21.15.36:33960) 2022-06-17T12:50:43.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd908ff8700 1 -- 172.21.15.36:0/3675623558 learned_addr learned my addr 172.21.15.36:0/3675623558 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:43.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd903fff700 1 --2- 172.21.15.36:0/3675623558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd904067a80 0x7fd904067240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:43.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd9097f9700 1 --2- 172.21.15.36:0/3675623558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9040ac900 0x7fd9040a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:43.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd908ff8700 1 -- 172.21.15.36:0/3675623558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd904067a80 msgr2=0x7fd904067240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:43.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd908ff8700 1 --2- 172.21.15.36:0/3675623558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd904067a80 0x7fd904067240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd908ff8700 1 -- 172.21.15.36:0/3675623558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9040ac900 msgr2=0x7fd9040a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:43.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd908ff8700 1 --2- 172.21.15.36:0/3675623558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9040ac900 0x7fd9040a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:43.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd908ff8700 1 -- 172.21.15.36:0/3675623558 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd904066da0 con 0x7fd8f0004e70 2022-06-17T12:50:43.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd9097f9700 1 --2- 172.21.15.36:0/3675623558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9040ac900 0x7fd9040a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:50:43.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd903fff700 1 --2- 172.21.15.36:0/3675623558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd904067a80 0x7fd904067240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:50:43.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd908ff8700 1 --2- 172.21.15.36:0/3675623558 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8f0004e70 0x7fd90409ec10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fd8f400d000 tx=0x7fd8f4007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:43.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd901ffb700 1 -- 172.21.15.36:0/3675623558 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8f4011780 con 0x7fd8f0004e70 2022-06-17T12:50:43.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd901ffb700 1 -- 172.21.15.36:0/3675623558 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f40118e0 con 0x7fd8f0004e70 2022-06-17T12:50:43.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd90b25c700 1 -- 172.21.15.36:0/3675623558 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd904067730 con 0x7fd8f0004e70 2022-06-17T12:50:43.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.845+0000 7fd901ffb700 1 -- 172.21.15.36:0/3675623558 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8f4011c40 con 0x7fd8f0004e70 2022-06-17T12:50:43.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.849+0000 7fd90b25c700 1 -- 172.21.15.36:0/3675623558 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9040a0a80 con 0x7fd8f0004e70 2022-06-17T12:50:43.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.849+0000 7fd90b25c700 1 -- 172.21.15.36:0/3675623558 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd904057410 con 0x7fd8f0004e70 2022-06-17T12:50:43.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.849+0000 7fd901ffb700 1 -- 172.21.15.36:0/3675623558 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd8f401b030 con 0x7fd8f0004e70 2022-06-17T12:50:43.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.849+0000 7fd901ffb700 1 --2- 172.21.15.36:0/3675623558 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd8e40539d0 0x7fd8e4055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:43.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.849+0000 7fd901ffb700 1 -- 172.21.15.36:0/3675623558 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fd8f406bf50 con 0x7fd8f0004e70 2022-06-17T12:50:43.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.849+0000 7fd903fff700 1 --2- 172.21.15.36:0/3675623558 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd8e40539d0 0x7fd8e4055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:43.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.849+0000 7fd903fff700 1 --2- 172.21.15.36:0/3675623558 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd8e40539d0 0x7fd8e4055e90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fd8f800b000 tx=0x7fd8f8008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:43.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.857+0000 7fd901ffb700 1 -- 172.21.15.36:0/3675623558 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd8f4042f90 con 0x7fd8f0004e70 2022-06-17T12:50:43.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:43.997+0000 7fd90b25c700 1 -- 172.21.15.36:0/3675623558 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fd9040b0b60 con 0x7fd8f0004e70 2022-06-17T12:50:44.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.841+0000 7fd901ffb700 1 -- 172.21.15.36:0/3675623558 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v50) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd8f40465b0 con 0x7fd8f0004e70 2022-06-17T12:50:44.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.845+0000 7fd90b25c700 1 -- 172.21.15.36:0/3675623558 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd8e40539d0 msgr2=0x7fd8e4055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:44.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.845+0000 7fd90b25c700 1 --2- 172.21.15.36:0/3675623558 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd8e40539d0 0x7fd8e4055e90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fd8f800b000 tx=0x7fd8f8008040).stop 2022-06-17T12:50:44.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.845+0000 7fd90b25c700 1 -- 172.21.15.36:0/3675623558 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8f0004e70 msgr2=0x7fd90409ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:44.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.845+0000 7fd90b25c700 1 --2- 172.21.15.36:0/3675623558 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8f0004e70 0x7fd90409ec10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fd8f400d000 tx=0x7fd8f4007a00).stop 2022-06-17T12:50:44.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.845+0000 7fd90b25c700 1 -- 172.21.15.36:0/3675623558 shutdown_connections 2022-06-17T12:50:44.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.845+0000 7fd90b25c700 1 --2- 172.21.15.36:0/3675623558 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd8e40539d0 0x7fd8e4055e90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:44.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.845+0000 7fd90b25c700 1 --2- 172.21.15.36:0/3675623558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9040ac900 0x7fd9040a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:44.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.845+0000 7fd90b25c700 1 --2- 172.21.15.36:0/3675623558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd904067a80 0x7fd904067240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:44.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.845+0000 7fd90b25c700 1 --2- 172.21.15.36:0/3675623558 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8f0004e70 0x7fd90409ec10 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:44.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.845+0000 7fd90b25c700 1 -- 172.21.15.36:0/3675623558 >> 172.21.15.36:0/3675623558 conn(0x7fd9040b6870 msgr2=0x7fd9040b1590 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:44.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.845+0000 7fd90b25c700 1 -- 172.21.15.36:0/3675623558 shutdown_connections 2022-06-17T12:50:44.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.845+0000 7fd90b25c700 1 -- 172.21.15.36:0/3675623558 wait complete. 2022-06-17T12:50:44.834 INFO:teuthology.orchestra.run.smithi036.stderr: removed all blacklist entries 2022-06-17T12:50:44.844 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2022-06-17T12:50:44.845 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-06-17T12:50:44.854 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:50:44.869 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:50:44.873 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:50:44.876 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:50:44.889 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:50:44.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.969+0000 7f50b4560700 1 Processor -- start 2022-06-17T12:50:44.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.969+0000 7f50b4560700 1 -- start start 2022-06-17T12:50:44.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.969+0000 7f50b4560700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f50ac0ad140 0x7f50ac0af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:44.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.969+0000 7f50b4560700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f50ac0ac340 0x7f50ac0ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:44.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.969+0000 7f50b4560700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50ac0ac870 con 0x7f50ac14aca0 2022-06-17T12:50:44.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.969+0000 7f50b4560700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50ac148960 con 0x7f50ac0ad140 2022-06-17T12:50:44.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.969+0000 7f50b4560700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50ac0acaa0 con 0x7f50ac0ac340 2022-06-17T12:50:44.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.969+0000 7f50b2afd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f50ac14aca0 0x7f50ac14b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51498/0 (socket says 172.21.15.36:51498) 2022-06-17T12:50:44.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.969+0000 7f50b2afd700 1 -- 172.21.15.36:0/3619710744 learned_addr learned my addr 172.21.15.36:0/3619710744 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:50:44.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.969+0000 7f50b22fc700 1 --2- 172.21.15.36:0/3619710744 >> v2:172.21.15.36:3300/0 conn(0x7f50ac0ad140 0x7f50ac0af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:44.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.969+0000 7f50b1afb700 1 --2- 172.21.15.36:0/3619710744 >> v2:172.21.15.70:3300/0 conn(0x7f50ac0ac340 0x7f50ac0ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:44.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.969+0000 7f50b12fa700 1 -- 172.21.15.36:0/3619710744 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3344055569 0 0) 0x7f50ac0ac870 con 0x7f50ac14aca0 2022-06-17T12:50:44.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.969+0000 7f50b12fa700 1 -- 172.21.15.36:0/3619710744 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5094003260 con 0x7f50ac14aca0 2022-06-17T12:50:44.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.969+0000 7f50b12fa700 1 -- 172.21.15.36:0/3619710744 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1531704885 0 0) 0x7f5094003260 con 0x7f50ac14aca0 2022-06-17T12:50:44.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.969+0000 7f50b22fc700 1 -- 172.21.15.36:0/3619710744 >> v2:172.21.15.70:3300/0 conn(0x7f50ac0ac340 msgr2=0x7f50ac0ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:44.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.969+0000 7f50b22fc700 1 --2- 172.21.15.36:0/3619710744 >> v2:172.21.15.70:3300/0 conn(0x7f50ac0ac340 0x7f50ac0ad520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:44.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.969+0000 7f50b22fc700 1 -- 172.21.15.36:0/3619710744 >> v1:172.21.15.36:6789/0 conn(0x7f50ac14aca0 legacy=0x7f50ac14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:44.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.969+0000 7f50b22fc700 1 -- 172.21.15.36:0/3619710744 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ac0b0bf0 con 0x7f50ac0ad140 2022-06-17T12:50:44.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.969+0000 7f50b22fc700 1 --2- 172.21.15.36:0/3619710744 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f50ac0ad140 0x7f50ac0af7f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f50a4001b80 tx=0x7f509c006df0).ready entity=mon.0 client_cookie=2b161c66c4ed589f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:44.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.969+0000 7f50b12fa700 1 -- 172.21.15.36:0/3619710744 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50a40019e0 con 0x7f50ac0ad140 2022-06-17T12:50:44.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.969+0000 7f50b12fa700 1 -- 172.21.15.36:0/3619710744 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509c000e90 con 0x7f50ac0ad140 2022-06-17T12:50:44.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.969+0000 7f50b12fa700 1 -- 172.21.15.36:0/3619710744 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f509c012460 con 0x7f50ac0ad140 2022-06-17T12:50:44.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.969+0000 7f50b4560700 1 -- 172.21.15.36:0/3619710744 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f50ac0ad140 msgr2=0x7f50ac0af7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:44.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.969+0000 7f50b4560700 1 --2- 172.21.15.36:0/3619710744 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f50ac0ad140 0x7f50ac0af7f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f50a4001b80 tx=0x7f509c006df0).stop 2022-06-17T12:50:44.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b4560700 1 -- 172.21.15.36:0/3619710744 shutdown_connections 2022-06-17T12:50:44.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b4560700 1 --2- 172.21.15.36:0/3619710744 >> v2:172.21.15.70:3300/0 conn(0x7f50ac0ac340 0x7f50ac0ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:44.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b4560700 1 --2- 172.21.15.36:0/3619710744 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f50ac0ad140 0x7f50ac0af7f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:44.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b4560700 1 -- 172.21.15.36:0/3619710744 >> 172.21.15.36:0/3619710744 conn(0x7f50ac0b5430 msgr2=0x7f50ac0b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:44.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b4560700 1 -- 172.21.15.36:0/3619710744 shutdown_connections 2022-06-17T12:50:44.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b4560700 1 -- 172.21.15.36:0/3619710744 wait complete. 2022-06-17T12:50:44.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b4560700 1 Processor -- start 2022-06-17T12:50:44.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b4560700 1 -- start start 2022-06-17T12:50:44.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b4560700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f50ac0ac340 0x7f50ac0ab110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:44.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b4560700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50ac0ad140 0x7f50ac0a8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:44.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b4560700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f50ac14aca0 0x7f50ac15a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:44.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b4560700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac064010 con 0x7f50ac0ac340 2022-06-17T12:50:44.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b4560700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac063630 con 0x7f50ac0ad140 2022-06-17T12:50:45.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b4560700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac0638d0 con 0x7f50ac14aca0 2022-06-17T12:50:45.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b22fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f50ac0ac340 0x7f50ac0ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:45.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b22fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f50ac0ac340 0x7f50ac0ab110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33974/0 (socket says 172.21.15.36:33974) 2022-06-17T12:50:45.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b22fc700 1 -- 172.21.15.36:0/1251246428 learned_addr learned my addr 172.21.15.36:0/1251246428 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:45.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b1afb700 1 --2- 172.21.15.36:0/1251246428 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50ac0ad140 0x7f50ac0a8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:45.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b2afd700 1 --2- 172.21.15.36:0/1251246428 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f50ac14aca0 0x7f50ac15a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:45.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b22fc700 1 -- 172.21.15.36:0/1251246428 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f50ac14aca0 msgr2=0x7f50ac15a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:45.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b22fc700 1 --2- 172.21.15.36:0/1251246428 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f50ac14aca0 0x7f50ac15a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b22fc700 1 -- 172.21.15.36:0/1251246428 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50ac0ad140 msgr2=0x7f50ac0a8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:45.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b22fc700 1 --2- 172.21.15.36:0/1251246428 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50ac0ad140 0x7f50ac0a8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b22fc700 1 -- 172.21.15.36:0/1251246428 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c00b180 con 0x7f50ac0ac340 2022-06-17T12:50:45.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b1afb700 1 --2- 172.21.15.36:0/1251246428 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50ac0ad140 0x7f50ac0a8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:50:45.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b22fc700 1 --2- 172.21.15.36:0/1251246428 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f50ac0ac340 0x7f50ac0ab110 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f50a4001b80 tx=0x7f509c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:45.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f509b7fe700 1 -- 172.21.15.36:0/1251246428 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f509c00da20 con 0x7f50ac0ac340 2022-06-17T12:50:45.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f509b7fe700 1 -- 172.21.15.36:0/1251246428 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509c00db80 con 0x7f50ac0ac340 2022-06-17T12:50:45.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b4560700 1 -- 172.21.15.36:0/1251246428 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ac063b70 con 0x7f50ac0ac340 2022-06-17T12:50:45.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f509b7fe700 1 -- 172.21.15.36:0/1251246428 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f509c01e400 con 0x7f50ac0ac340 2022-06-17T12:50:45.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.973+0000 7f50b4560700 1 -- 172.21.15.36:0/1251246428 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ac0631b0 con 0x7f50ac0ac340 2022-06-17T12:50:45.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.977+0000 7f50b4560700 1 -- 172.21.15.36:0/1251246428 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50ac063490 con 0x7f50ac0ac340 2022-06-17T12:50:45.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.977+0000 7f509b7fe700 1 -- 172.21.15.36:0/1251246428 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f509c025030 con 0x7f50ac0ac340 2022-06-17T12:50:45.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.977+0000 7f509b7fe700 1 --2- 172.21.15.36:0/1251246428 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f508c053ad0 0x7f508c055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:45.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.977+0000 7f509b7fe700 1 -- 172.21.15.36:0/1251246428 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f509c06c9a0 con 0x7f50ac0ac340 2022-06-17T12:50:45.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.977+0000 7f50b1afb700 1 --2- 172.21.15.36:0/1251246428 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f508c053ad0 0x7f508c055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:45.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.977+0000 7f50b1afb700 1 --2- 172.21.15.36:0/1251246428 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f508c053ad0 0x7f508c055f90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f50a800b000 tx=0x7f50a8008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:45.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:44.985+0000 7f509b7fe700 1 -- 172.21.15.36:0/1251246428 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f50ac063490 con 0x7f50ac0ac340 2022-06-17T12:50:45.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.149+0000 7f50b4560700 1 -- 172.21.15.36:0/1251246428 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f50ac063490 con 0x7f50ac0ac340 2022-06-17T12:50:45.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.149+0000 7f509b7fe700 1 -- 172.21.15.36:0/1251246428 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f50ac063490 con 0x7f50ac0ac340 2022-06-17T12:50:45.139 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:50:45.139 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:50:45.140 INFO:teuthology.orchestra.run.smithi036.stdout: "auth_dump": [ 2022-06-17T12:50:45.140 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:45.140 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mds.a", 2022-06-17T12:50:45.141 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiyaUgARAAxp7bzOl4uVo2EsaoRBeMRw==", 2022-06-17T12:50:45.141 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:50:45.141 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:50:45.142 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:50:45.142 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:50:45.142 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:50:45.142 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:45.142 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:45.143 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:45.144 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mds.b", 2022-06-17T12:50:45.144 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xifyxLCRAABI389jmz1zboeDanzraT8Q==", 2022-06-17T12:50:45.144 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:50:45.144 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:50:45.145 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:50:45.145 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:50:45.145 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:50:45.145 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:45.146 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:45.146 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:45.146 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.0", 2022-06-17T12:50:45.147 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB+d6xibhXGNBAA6AuUT9ITWAZjDFpNsgA4Wg==", 2022-06-17T12:50:45.147 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:50:45.147 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:50:45.147 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:50:45.148 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:50:45.148 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:45.148 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:45.148 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:45.149 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.1", 2022-06-17T12:50:45.149 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCBd6xiOIPvGxAADkrBoJCy9x91zan6TWU0tQ==", 2022-06-17T12:50:45.149 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:50:45.149 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:50:45.149 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:50:45.150 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:50:45.150 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:45.150 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:45.150 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:45.151 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.2", 2022-06-17T12:50:45.151 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCEd6xiP4/9AhAAaP3cWXV6vbfGBPfqSLKqUg==", 2022-06-17T12:50:45.151 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:50:45.152 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:50:45.152 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:50:45.152 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:50:45.153 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:45.153 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:45.154 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:45.155 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.3", 2022-06-17T12:50:45.155 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCGd6xi+LZgJxAAJjGwP5qW6TFDZfR+2rz6ig==", 2022-06-17T12:50:45.155 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:50:45.155 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:50:45.156 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:50:45.156 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:50:45.156 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:45.157 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:45.157 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:45.158 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.4", 2022-06-17T12:50:45.158 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCKd6xikvrTChAAXIyWSETswDv+tb3zsGyYpA==", 2022-06-17T12:50:45.159 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:50:45.159 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:50:45.160 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:50:45.160 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:50:45.160 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:45.160 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:45.161 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:45.161 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.5", 2022-06-17T12:50:45.161 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCMd6xiqr2QLhAArhimHuNsVWrDGGreFYxj6A==", 2022-06-17T12:50:45.161 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:50:45.162 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:50:45.162 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:50:45.162 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:50:45.163 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:45.163 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:45.163 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:45.163 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.6", 2022-06-17T12:50:45.164 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCPd6xiUXcXGxAAdrYz8txdBWMIqF5DF4Aimw==", 2022-06-17T12:50:45.165 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:50:45.165 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:50:45.166 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:50:45.166 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:50:45.166 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:45.167 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:45.167 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:45.168 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.7", 2022-06-17T12:50:45.168 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCSd6xiBJ2IAhAAUkNfgmlxxWY7+VouPLAvyw==", 2022-06-17T12:50:45.168 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:50:45.168 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:50:45.169 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:50:45.169 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:50:45.169 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:45.170 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:45.170 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:45.170 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.0", 2022-06-17T12:50:45.170 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiXO3QFBAAsSLtQ0yKKZxhqUT/eYKdtg==", 2022-06-17T12:50:45.171 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:50:45.171 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:50:45.171 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:50:45.172 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:50:45.172 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:45.172 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:45.173 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:45.176 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.1", 2022-06-17T12:50:45.177 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xi51KdGBAAi7fGZ+Qepg6j3DAhlzoWxw==", 2022-06-17T12:50:45.177 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:50:45.177 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:50:45.178 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:50:45.178 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:50:45.179 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:45.179 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:45.180 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:45.180 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.2", 2022-06-17T12:50:45.181 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiug2hHBAAqHpLgwMdthSWtJ8Vwu+m1Q==", 2022-06-17T12:50:45.181 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:50:45.181 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:50:45.182 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:50:45.182 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:50:45.183 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:45.183 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:45.184 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:45.184 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.admin", 2022-06-17T12:50:45.185 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB5d6xiGDODKhAAPZMG/M2nVssdr4FJHcS6XQ==", 2022-06-17T12:50:45.185 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:50:45.185 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:50:45.186 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:50:45.186 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:50:45.187 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:50:45.187 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:45.188 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:45.188 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:45.189 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mgr.x", 2022-06-17T12:50:45.189 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB6d6xiyH9uORAATJb0chHE1CH0X02w7edVtw==", 2022-06-17T12:50:45.190 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:50:45.190 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:50:45.191 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile mgr", 2022-06-17T12:50:45.191 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:50:45.191 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:45.192 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:45.192 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:45.193 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mgr.y", 2022-06-17T12:50:45.193 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB6d6xiHXghMxAAzpsVRCbLWHWZPtrPk1jFlA==", 2022-06-17T12:50:45.194 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:50:45.194 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:50:45.195 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile mgr", 2022-06-17T12:50:45.195 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:50:45.195 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:45.196 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:45.196 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:50:45.197 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:50:45.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.149+0000 7f50b4560700 1 -- 172.21.15.36:0/1251246428 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f508c053ad0 msgr2=0x7f508c055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:45.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.149+0000 7f50b4560700 1 --2- 172.21.15.36:0/1251246428 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f508c053ad0 0x7f508c055f90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f50a800b000 tx=0x7f50a8008040).stop 2022-06-17T12:50:45.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.149+0000 7f50b4560700 1 -- 172.21.15.36:0/1251246428 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f50ac0ac340 msgr2=0x7f50ac0ab110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:45.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.149+0000 7f50b4560700 1 --2- 172.21.15.36:0/1251246428 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f50ac0ac340 0x7f50ac0ab110 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f50a4001b80 tx=0x7f509c014000).stop 2022-06-17T12:50:45.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.149+0000 7f50b4560700 1 -- 172.21.15.36:0/1251246428 shutdown_connections 2022-06-17T12:50:45.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.149+0000 7f50b4560700 1 --2- 172.21.15.36:0/1251246428 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f508c053ad0 0x7f508c055f90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.149+0000 7f50b4560700 1 --2- 172.21.15.36:0/1251246428 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f50ac14aca0 0x7f50ac15a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.149+0000 7f50b4560700 1 --2- 172.21.15.36:0/1251246428 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50ac0ad140 0x7f50ac0a8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.149+0000 7f50b4560700 1 --2- 172.21.15.36:0/1251246428 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f50ac0ac340 0x7f50ac0ab110 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.149+0000 7f50b4560700 1 -- 172.21.15.36:0/1251246428 >> 172.21.15.36:0/1251246428 conn(0x7f50ac0b5430 msgr2=0x7f50ac145c70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:45.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.149+0000 7f50b4560700 1 -- 172.21.15.36:0/1251246428 shutdown_connections 2022-06-17T12:50:45.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.149+0000 7f50b4560700 1 -- 172.21.15.36:0/1251246428 wait complete. 2022-06-17T12:50:45.208 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:50:45.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.333+0000 7fd7dd152700 1 Processor -- start 2022-06-17T12:50:45.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.333+0000 7fd7dd152700 1 -- start start 2022-06-17T12:50:45.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.333+0000 7fd7dd152700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd7d80b2200 0x7fd7d80af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:45.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.333+0000 7fd7dd152700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7d80af8d0 0x7fd7d814a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:45.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.333+0000 7fd7dd152700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d80b25e0 con 0x7fd7d80b2cd0 2022-06-17T12:50:45.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.333+0000 7fd7dd152700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d80ad790 con 0x7fd7d80b2200 2022-06-17T12:50:45.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.333+0000 7fd7dd152700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d80aef40 con 0x7fd7d80af8d0 2022-06-17T12:50:45.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.333+0000 7fd7d7fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd7d80b2200 0x7fd7d80af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:45.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.333+0000 7fd7dc951700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fd7d80b2cd0 0x7fd7d814adc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51516/0 (socket says 172.21.15.36:51516) 2022-06-17T12:50:45.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.333+0000 7fd7dc951700 1 -- 172.21.15.36:0/1720940324 learned_addr learned my addr 172.21.15.36:0/1720940324 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:50:45.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.333+0000 7fd7d77fe700 1 --2- 172.21.15.36:0/1720940324 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7d80af8d0 0x7fd7d814a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:45.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.333+0000 7fd7d7fff700 1 -- 172.21.15.36:0/1720940324 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7d80af8d0 msgr2=0x7fd7d814a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:45.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.333+0000 7fd7d7fff700 1 --2- 172.21.15.36:0/1720940324 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7d80af8d0 0x7fd7d814a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.333+0000 7fd7d7fff700 1 -- 172.21.15.36:0/1720940324 >> v1:172.21.15.36:6789/0 conn(0x7fd7d80b2cd0 legacy=0x7fd7d814adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:45.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.333+0000 7fd7d6ffd700 1 -- 172.21.15.36:0/1720940324 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2030707081 0 0) 0x7fd7d80b25e0 con 0x7fd7d80b2cd0 2022-06-17T12:50:45.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.333+0000 7fd7d7fff700 1 -- 172.21.15.36:0/1720940324 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d80b1950 con 0x7fd7d80b2200 2022-06-17T12:50:45.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.333+0000 7fd7d7fff700 1 --2- 172.21.15.36:0/1720940324 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7d80b2200 0x7fd7d80af3e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7fd7d00032f0 tx=0x7fd7d00091d0).ready entity=mon.0 client_cookie=1ad57f3c81d31a2a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:45.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.337+0000 7fd7d6ffd700 1 -- 172.21.15.36:0/1720940324 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7d000b6c0 con 0x7fd7d80b2200 2022-06-17T12:50:45.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.337+0000 7fd7d6ffd700 1 -- 172.21.15.36:0/1720940324 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d000b820 con 0x7fd7d80b2200 2022-06-17T12:50:45.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.337+0000 7fd7d6ffd700 1 -- 172.21.15.36:0/1720940324 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7d000b990 con 0x7fd7d80b2200 2022-06-17T12:50:45.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.337+0000 7fd7dd152700 1 -- 172.21.15.36:0/1720940324 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7d80b2200 msgr2=0x7fd7d80af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:45.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.337+0000 7fd7dd152700 1 --2- 172.21.15.36:0/1720940324 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7d80b2200 0x7fd7d80af3e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7fd7d00032f0 tx=0x7fd7d00091d0).stop 2022-06-17T12:50:45.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.337+0000 7fd7dd152700 1 -- 172.21.15.36:0/1720940324 shutdown_connections 2022-06-17T12:50:45.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.337+0000 7fd7dd152700 1 --2- 172.21.15.36:0/1720940324 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7d80af8d0 0x7fd7d814a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.337+0000 7fd7dd152700 1 --2- 172.21.15.36:0/1720940324 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7d80b2200 0x7fd7d80af3e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.337+0000 7fd7dd152700 1 -- 172.21.15.36:0/1720940324 >> 172.21.15.36:0/1720940324 conn(0x7fd7d805f770 msgr2=0x7fd7d805fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:45.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.337+0000 7fd7dd152700 1 -- 172.21.15.36:0/1720940324 shutdown_connections 2022-06-17T12:50:45.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.337+0000 7fd7dd152700 1 -- 172.21.15.36:0/1720940324 wait complete. 2022-06-17T12:50:45.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.337+0000 7fd7dd152700 1 Processor -- start 2022-06-17T12:50:45.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.337+0000 7fd7dd152700 1 -- start start 2022-06-17T12:50:45.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.337+0000 7fd7dd152700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7d80af8d0 0x7fd7d8145230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:45.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.337+0000 7fd7dd152700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7d80b2200 0x7fd7d814aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:45.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.337+0000 7fd7dd152700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7d80b2cd0 0x7fd7d8157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:45.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.337+0000 7fd7dd152700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d8145e40 con 0x7fd7d80b2cd0 2022-06-17T12:50:45.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.337+0000 7fd7dd152700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d81443b0 con 0x7fd7d80b2200 2022-06-17T12:50:45.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.337+0000 7fd7dd152700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d814afc0 con 0x7fd7d80af8d0 2022-06-17T12:50:45.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.337+0000 7fd7dc951700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7d80b2cd0 0x7fd7d8157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:45.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.337+0000 7fd7d7fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7d80af8d0 0x7fd7d8145230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:45.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.337+0000 7fd7d7fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7d80af8d0 0x7fd7d8145230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:48778/0 (socket says 172.21.15.36:48778) 2022-06-17T12:50:45.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.337+0000 7fd7d7fff700 1 -- 172.21.15.36:0/3137071907 learned_addr learned my addr 172.21.15.36:0/3137071907 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:45.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.337+0000 7fd7d77fe700 1 --2- 172.21.15.36:0/3137071907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7d80b2200 0x7fd7d814aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:45.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.341+0000 7fd7d7fff700 1 -- 172.21.15.36:0/3137071907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7d80b2200 msgr2=0x7fd7d814aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:45.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.341+0000 7fd7d7fff700 1 --2- 172.21.15.36:0/3137071907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7d80b2200 0x7fd7d814aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.341+0000 7fd7d7fff700 1 -- 172.21.15.36:0/3137071907 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7d80b2cd0 msgr2=0x7fd7d8157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:45.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.341+0000 7fd7d7fff700 1 --2- 172.21.15.36:0/3137071907 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7d80b2cd0 0x7fd7d8157180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.341+0000 7fd7d7fff700 1 -- 172.21.15.36:0/3137071907 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d0009470 con 0x7fd7d80af8d0 2022-06-17T12:50:45.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.341+0000 7fd7d7fff700 1 --2- 172.21.15.36:0/3137071907 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7d80af8d0 0x7fd7d8145230 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fd7d00032f0 tx=0x7fd7d0016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:45.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.341+0000 7fd7d57fa700 1 -- 172.21.15.36:0/3137071907 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7d000b990 con 0x7fd7d80af8d0 2022-06-17T12:50:45.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.341+0000 7fd7d57fa700 1 -- 172.21.15.36:0/3137071907 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d0021630 con 0x7fd7d80af8d0 2022-06-17T12:50:45.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.341+0000 7fd7dd152700 1 -- 172.21.15.36:0/3137071907 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d8145ac0 con 0x7fd7d80af8d0 2022-06-17T12:50:45.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.341+0000 7fd7d57fa700 1 -- 172.21.15.36:0/3137071907 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7d000b6c0 con 0x7fd7d80af8d0 2022-06-17T12:50:45.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.341+0000 7fd7dd152700 1 -- 172.21.15.36:0/3137071907 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d8062b90 con 0x7fd7d80af8d0 2022-06-17T12:50:45.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.341+0000 7fd7dd152700 1 -- 172.21.15.36:0/3137071907 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d80ad330 con 0x7fd7d80af8d0 2022-06-17T12:50:45.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.341+0000 7fd7d57fa700 1 -- 172.21.15.36:0/3137071907 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd7d000f070 con 0x7fd7d80af8d0 2022-06-17T12:50:45.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.341+0000 7fd7d57fa700 1 --2- 172.21.15.36:0/3137071907 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7b80539b0 0x7fd7b8055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:45.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.341+0000 7fd7d57fa700 1 -- 172.21.15.36:0/3137071907 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fd7d006d0f0 con 0x7fd7d80af8d0 2022-06-17T12:50:45.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.341+0000 7fd7d77fe700 1 --2- 172.21.15.36:0/3137071907 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7b80539b0 0x7fd7b8055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:45.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.345+0000 7fd7d77fe700 1 --2- 172.21.15.36:0/3137071907 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7b80539b0 0x7fd7b8055e70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fd7cc00b000 tx=0x7fd7cc006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:45.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.349+0000 7fd7d57fa700 1 -- 172.21.15.36:0/3137071907 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd7d00441d0 con 0x7fd7d80af8d0 2022-06-17T12:50:45.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.477+0000 7fd7dd152700 1 -- 172.21.15.36:0/3137071907 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7d8145fb0 con 0x7fd7d80af8d0 2022-06-17T12:50:45.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.481+0000 7fd7d57fa700 1 -- 172.21.15.36:0/3137071907 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+13414 (secure 0 0 0) 0x7fd7d0015020 con 0x7fd7d80af8d0 2022-06-17T12:50:45.467 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:50:45.468 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":50,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:50:44.836842+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:50:45.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.481+0000 7fd7dd152700 1 -- 172.21.15.36:0/3137071907 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7b80539b0 msgr2=0x7fd7b8055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:45.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.481+0000 7fd7dd152700 1 --2- 172.21.15.36:0/3137071907 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7b80539b0 0x7fd7b8055e70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fd7cc00b000 tx=0x7fd7cc006df0).stop 2022-06-17T12:50:45.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.481+0000 7fd7dd152700 1 -- 172.21.15.36:0/3137071907 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7d80af8d0 msgr2=0x7fd7d8145230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:45.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.481+0000 7fd7dd152700 1 --2- 172.21.15.36:0/3137071907 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7d80af8d0 0x7fd7d8145230 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fd7d00032f0 tx=0x7fd7d0016000).stop 2022-06-17T12:50:45.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.481+0000 7fd7dd152700 1 -- 172.21.15.36:0/3137071907 shutdown_connections 2022-06-17T12:50:45.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.481+0000 7fd7dd152700 1 --2- 172.21.15.36:0/3137071907 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7b80539b0 0x7fd7b8055e70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.481+0000 7fd7dd152700 1 --2- 172.21.15.36:0/3137071907 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7d80b2cd0 0x7fd7d8157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.481+0000 7fd7dd152700 1 --2- 172.21.15.36:0/3137071907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7d80b2200 0x7fd7d814aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.481+0000 7fd7dd152700 1 --2- 172.21.15.36:0/3137071907 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7d80af8d0 0x7fd7d8145230 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.481+0000 7fd7dd152700 1 -- 172.21.15.36:0/3137071907 >> 172.21.15.36:0/3137071907 conn(0x7fd7d805f770 msgr2=0x7fd7d8146f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:45.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.481+0000 7fd7dd152700 1 -- 172.21.15.36:0/3137071907 shutdown_connections 2022-06-17T12:50:45.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.481+0000 7fd7dd152700 1 -- 172.21.15.36:0/3137071907 wait complete. 2022-06-17T12:50:45.508 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-17T12:50:45.509 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:50:45.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.653+0000 7f062e39a700 1 Processor -- start 2022-06-17T12:50:45.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.653+0000 7f062e39a700 1 -- start start 2022-06-17T12:50:45.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.653+0000 7f062e39a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06280b45c0 0x7f06280b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:45.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.653+0000 7f062e39a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f06280b4f00 0x7f06280b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:45.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.653+0000 7f062e39a700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06280b4160 con 0x7f06280b9860 2022-06-17T12:50:45.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.653+0000 7f062e39a700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06280b9c40 con 0x7f06280b4f00 2022-06-17T12:50:45.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.653+0000 7f062e39a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06280627d0 con 0x7f06280b45c0 2022-06-17T12:50:45.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.653+0000 7f062db99700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f06280b4f00 0x7f06280b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:45.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.653+0000 7f062db99700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f06280b4f00 0x7f06280b5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:33998/0 (socket says 172.21.15.36:33998) 2022-06-17T12:50:45.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.653+0000 7f062db99700 1 -- 172.21.15.36:0/525488575 learned_addr learned my addr 172.21.15.36:0/525488575 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:45.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.653+0000 7f062d398700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f06280b9860 0x7f0628143840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51526/0 (socket says 172.21.15.36:51526) 2022-06-17T12:50:45.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.653+0000 7f062cb97700 1 --2- 172.21.15.36:0/525488575 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06280b45c0 0x7f06280b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:45.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062db99700 1 -- 172.21.15.36:0/525488575 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06280b45c0 msgr2=0x7f06280b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:45.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062db99700 1 --2- 172.21.15.36:0/525488575 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06280b45c0 0x7f06280b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062db99700 1 -- 172.21.15.36:0/525488575 >> v1:172.21.15.36:6789/0 conn(0x7f06280b9860 legacy=0x7f0628143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:45.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062db99700 1 -- 172.21.15.36:0/525488575 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0628062a70 con 0x7f06280b4f00 2022-06-17T12:50:45.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062db99700 1 --2- 172.21.15.36:0/525488575 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f06280b4f00 0x7f06280b5dc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f06200032f0 tx=0x7f0620009290).ready entity=mon.0 client_cookie=1e20308a2e254817 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:45.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f0613fff700 1 -- 172.21.15.36:0/525488575 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f062000b8a0 con 0x7f06280b4f00 2022-06-17T12:50:45.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f0613fff700 1 -- 172.21.15.36:0/525488575 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062000ba00 con 0x7f06280b4f00 2022-06-17T12:50:45.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f0613fff700 1 -- 172.21.15.36:0/525488575 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f062000bb70 con 0x7f06280b4f00 2022-06-17T12:50:45.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062e39a700 1 -- 172.21.15.36:0/525488575 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f06280b4f00 msgr2=0x7f06280b5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:45.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062e39a700 1 --2- 172.21.15.36:0/525488575 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f06280b4f00 0x7f06280b5dc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f06200032f0 tx=0x7f0620009290).stop 2022-06-17T12:50:45.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062e39a700 1 -- 172.21.15.36:0/525488575 shutdown_connections 2022-06-17T12:50:45.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062e39a700 1 --2- 172.21.15.36:0/525488575 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f06280b4f00 0x7f06280b5dc0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062e39a700 1 --2- 172.21.15.36:0/525488575 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06280b45c0 0x7f06280b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062e39a700 1 -- 172.21.15.36:0/525488575 >> 172.21.15.36:0/525488575 conn(0x7f062805c550 msgr2=0x7f0628060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:45.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062e39a700 1 -- 172.21.15.36:0/525488575 shutdown_connections 2022-06-17T12:50:45.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062e39a700 1 -- 172.21.15.36:0/525488575 wait complete. 2022-06-17T12:50:45.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062e39a700 1 Processor -- start 2022-06-17T12:50:45.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062e39a700 1 -- start start 2022-06-17T12:50:45.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062e39a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06280b45c0 0x7f06280ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:45.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062e39a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f06280b4f00 0x7f06280aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:45.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062e39a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06280b9860 0x7f0628152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:45.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062e39a700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062813d280 con 0x7f06280b4f00 2022-06-17T12:50:45.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062e39a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062813e2d0 con 0x7f06280b9860 2022-06-17T12:50:45.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062e39a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f062813e030 con 0x7f06280b45c0 2022-06-17T12:50:45.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062cb97700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f06280b4f00 0x7f06280aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:45.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062cb97700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f06280b4f00 0x7f06280aed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34002/0 (socket says 172.21.15.36:34002) 2022-06-17T12:50:45.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062cb97700 1 -- 172.21.15.36:0/953932364 learned_addr learned my addr 172.21.15.36:0/953932364 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:45.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062d398700 1 --2- 172.21.15.36:0/953932364 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06280b45c0 0x7f06280ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:45.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.657+0000 7f062db99700 1 --2- 172.21.15.36:0/953932364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06280b9860 0x7f0628152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:45.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.661+0000 7f062cb97700 1 -- 172.21.15.36:0/953932364 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06280b45c0 msgr2=0x7f06280ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:45.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.661+0000 7f062cb97700 1 --2- 172.21.15.36:0/953932364 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06280b45c0 0x7f06280ae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.661+0000 7f062cb97700 1 -- 172.21.15.36:0/953932364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06280b9860 msgr2=0x7f0628152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:45.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.661+0000 7f062cb97700 1 --2- 172.21.15.36:0/953932364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06280b9860 0x7f0628152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.661+0000 7f062cb97700 1 -- 172.21.15.36:0/953932364 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0620009530 con 0x7f06280b4f00 2022-06-17T12:50:45.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.661+0000 7f062cb97700 1 --2- 172.21.15.36:0/953932364 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f06280b4f00 0x7f06280aed20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f061c00a000 tx=0x7f061c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:45.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.661+0000 7f061affd700 1 -- 172.21.15.36:0/953932364 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f061c011780 con 0x7f06280b4f00 2022-06-17T12:50:45.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.661+0000 7f061affd700 1 -- 172.21.15.36:0/953932364 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061c0118e0 con 0x7f06280b4f00 2022-06-17T12:50:45.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.661+0000 7f062e39a700 1 -- 172.21.15.36:0/953932364 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062813cf00 con 0x7f06280b4f00 2022-06-17T12:50:45.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.661+0000 7f061affd700 1 -- 172.21.15.36:0/953932364 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f061c011c40 con 0x7f06280b4f00 2022-06-17T12:50:45.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.661+0000 7f062e39a700 1 -- 172.21.15.36:0/953932364 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0628146db0 con 0x7f06280b4f00 2022-06-17T12:50:45.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.661+0000 7f061affd700 1 -- 172.21.15.36:0/953932364 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f061c012420 con 0x7f06280b4f00 2022-06-17T12:50:45.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.661+0000 7f062e39a700 1 -- 172.21.15.36:0/953932364 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0628062eb0 con 0x7f06280b4f00 2022-06-17T12:50:45.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.661+0000 7f061affd700 1 --2- 172.21.15.36:0/953932364 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0604053a40 0x7f0604055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:45.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.661+0000 7f061affd700 1 -- 172.21.15.36:0/953932364 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f061c06bac0 con 0x7f06280b4f00 2022-06-17T12:50:45.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.661+0000 7f062d398700 1 --2- 172.21.15.36:0/953932364 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0604053a40 0x7f0604055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:45.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.661+0000 7f062d398700 1 --2- 172.21.15.36:0/953932364 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0604053a40 0x7f0604055f00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f062400d000 tx=0x7f0624007b80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:45.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.669+0000 7f061affd700 1 -- 172.21.15.36:0/953932364 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f061c042b00 con 0x7f06280b4f00 2022-06-17T12:50:45.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.797+0000 7f062e39a700 1 -- 172.21.15.36:0/953932364 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f062805f220 con 0x7f06280b4f00 2022-06-17T12:50:45.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.797+0000 7f061affd700 1 -- 172.21.15.36:0/953932364 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v50) v1 ==== 110+0+10 (secure 0 0 0) 0x7f061c046120 con 0x7f06280b4f00 2022-06-17T12:50:45.784 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:50:45.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.801+0000 7f062e39a700 1 -- 172.21.15.36:0/953932364 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0604053a40 msgr2=0x7f0604055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:45.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.801+0000 7f062e39a700 1 --2- 172.21.15.36:0/953932364 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0604053a40 0x7f0604055f00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f062400d000 tx=0x7f0624007b80).stop 2022-06-17T12:50:45.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.801+0000 7f062e39a700 1 -- 172.21.15.36:0/953932364 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f06280b4f00 msgr2=0x7f06280aed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:45.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.801+0000 7f062e39a700 1 --2- 172.21.15.36:0/953932364 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f06280b4f00 0x7f06280aed20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f061c00a000 tx=0x7f061c008040).stop 2022-06-17T12:50:45.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.801+0000 7f062e39a700 1 -- 172.21.15.36:0/953932364 shutdown_connections 2022-06-17T12:50:45.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.801+0000 7f062e39a700 1 --2- 172.21.15.36:0/953932364 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0604053a40 0x7f0604055f00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.801+0000 7f062e39a700 1 --2- 172.21.15.36:0/953932364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06280b9860 0x7f0628152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.801+0000 7f062e39a700 1 --2- 172.21.15.36:0/953932364 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f06280b4f00 0x7f06280aed20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.801+0000 7f062e39a700 1 --2- 172.21.15.36:0/953932364 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06280b45c0 0x7f06280ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.801+0000 7f062e39a700 1 -- 172.21.15.36:0/953932364 >> 172.21.15.36:0/953932364 conn(0x7f062805c550 msgr2=0x7f06280b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:45.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.801+0000 7f062e39a700 1 -- 172.21.15.36:0/953932364 shutdown_connections 2022-06-17T12:50:45.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.801+0000 7f062e39a700 1 -- 172.21.15.36:0/953932364 wait complete. 2022-06-17T12:50:45.805 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:50:45.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.941+0000 7fd0b4090700 1 Processor -- start 2022-06-17T12:50:45.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.941+0000 7fd0b4090700 1 -- start start 2022-06-17T12:50:45.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b4090700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0ac0b64f0 0x7fd0ac143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:45.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b4090700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd0ac0b4120 0x7fd0ac146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:45.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b4090700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0ac0b68d0 con 0x7fd0ac147270 2022-06-17T12:50:45.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b4090700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0ac1476d0 con 0x7fd0ac0b4120 2022-06-17T12:50:45.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b4090700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac0b6e20 con 0x7fd0ac0b64f0 2022-06-17T12:50:45.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b162b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd0ac0b4120 0x7fd0ac146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:45.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b162b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd0ac0b4120 0x7fd0ac146d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34012/0 (socket says 172.21.15.36:34012) 2022-06-17T12:50:45.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b162b700 1 -- 172.21.15.36:0/3207520157 learned_addr learned my addr 172.21.15.36:0/3207520157 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:45.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b1e2c700 1 --2- 172.21.15.36:0/3207520157 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0ac0b64f0 0x7fd0ac143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:45.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b162b700 1 -- 172.21.15.36:0/3207520157 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0ac0b64f0 msgr2=0x7fd0ac143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:45.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b162b700 1 --2- 172.21.15.36:0/3207520157 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0ac0b64f0 0x7fd0ac143150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b162b700 1 -- 172.21.15.36:0/3207520157 >> v1:172.21.15.70:6789/0 conn(0x7fd0ac147270 legacy=0x7fd0ac140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:45.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b162b700 1 -- 172.21.15.36:0/3207520157 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ac1468e0 con 0x7fd0ac0b4120 2022-06-17T12:50:45.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b1e2c700 1 --2- 172.21.15.36:0/3207520157 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0ac0b64f0 0x7fd0ac143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:50:45.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b162b700 1 --2- 172.21.15.36:0/3207520157 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0ac0b4120 0x7fd0ac146d80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7fd0ac061d20 tx=0x7fd0a8006eb0).ready entity=mon.0 client_cookie=f1fce1f05abb2b60 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:45.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b0e2a700 1 -- 172.21.15.36:0/3207520157 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0a800d570 con 0x7fd0ac0b4120 2022-06-17T12:50:45.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b0e2a700 1 -- 172.21.15.36:0/3207520157 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a8000e90 con 0x7fd0ac0b4120 2022-06-17T12:50:45.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b0e2a700 1 -- 172.21.15.36:0/3207520157 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0a800d8c0 con 0x7fd0ac0b4120 2022-06-17T12:50:45.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b4090700 1 -- 172.21.15.36:0/3207520157 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0ac0b4120 msgr2=0x7fd0ac146d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:45.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b4090700 1 --2- 172.21.15.36:0/3207520157 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0ac0b4120 0x7fd0ac146d80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7fd0ac061d20 tx=0x7fd0a8006eb0).stop 2022-06-17T12:50:45.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b4090700 1 -- 172.21.15.36:0/3207520157 shutdown_connections 2022-06-17T12:50:45.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b4090700 1 --2- 172.21.15.36:0/3207520157 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0ac0b4120 0x7fd0ac146d80 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b4090700 1 --2- 172.21.15.36:0/3207520157 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0ac0b64f0 0x7fd0ac143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b4090700 1 -- 172.21.15.36:0/3207520157 >> 172.21.15.36:0/3207520157 conn(0x7fd0ac0620f0 msgr2=0x7fd0ac062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:45.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b4090700 1 -- 172.21.15.36:0/3207520157 shutdown_connections 2022-06-17T12:50:45.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.945+0000 7fd0b4090700 1 -- 172.21.15.36:0/3207520157 wait complete. 2022-06-17T12:50:45.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.949+0000 7fd0b4090700 1 Processor -- start 2022-06-17T12:50:45.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.949+0000 7fd0b4090700 1 -- start start 2022-06-17T12:50:45.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.949+0000 7fd0b4090700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0ac0b4120 0x7fd0ac14a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:45.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.949+0000 7fd0b4090700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0ac0b64f0 0x7fd0ac141740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:45.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.949+0000 7fd0b4090700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0ac147270 0x7fd0ac142bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:45.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.949+0000 7fd0b4090700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac1427a0 con 0x7fd0ac147270 2022-06-17T12:50:45.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.949+0000 7fd0b4090700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac140860 con 0x7fd0ac0b4120 2022-06-17T12:50:45.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.949+0000 7fd0b4090700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac0b2bb0 con 0x7fd0ac0b64f0 2022-06-17T12:50:45.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.949+0000 7fd0b262d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0ac147270 0x7fd0ac142bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:45.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.949+0000 7fd0b262d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0ac147270 0x7fd0ac142bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34020/0 (socket says 172.21.15.36:34020) 2022-06-17T12:50:45.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.949+0000 7fd0b262d700 1 -- 172.21.15.36:0/832818451 learned_addr learned my addr 172.21.15.36:0/832818451 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:45.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.949+0000 7fd0b162b700 1 --2- 172.21.15.36:0/832818451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0ac0b64f0 0x7fd0ac141740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:45.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.949+0000 7fd0b1e2c700 1 --2- 172.21.15.36:0/832818451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0ac0b4120 0x7fd0ac14a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:45.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.953+0000 7fd0b262d700 1 -- 172.21.15.36:0/832818451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0ac0b64f0 msgr2=0x7fd0ac141740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:45.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.953+0000 7fd0b262d700 1 --2- 172.21.15.36:0/832818451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0ac0b64f0 0x7fd0ac141740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.953+0000 7fd0b262d700 1 -- 172.21.15.36:0/832818451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0ac0b4120 msgr2=0x7fd0ac14a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:45.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.953+0000 7fd0b262d700 1 --2- 172.21.15.36:0/832818451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0ac0b4120 0x7fd0ac14a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:45.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.953+0000 7fd0b262d700 1 -- 172.21.15.36:0/832818451 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a800b1e0 con 0x7fd0ac147270 2022-06-17T12:50:45.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.953+0000 7fd0b262d700 1 --2- 172.21.15.36:0/832818451 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0ac147270 0x7fd0ac142bd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7fd0a40032b0 tx=0x7fd0a40090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:45.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.953+0000 7fd09affd700 1 -- 172.21.15.36:0/832818451 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0a400e470 con 0x7fd0ac147270 2022-06-17T12:50:45.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.953+0000 7fd09affd700 1 -- 172.21.15.36:0/832818451 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a400e5d0 con 0x7fd0ac147270 2022-06-17T12:50:45.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.953+0000 7fd0b4090700 1 -- 172.21.15.36:0/832818451 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0ac0b2e50 con 0x7fd0ac147270 2022-06-17T12:50:45.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.953+0000 7fd09affd700 1 -- 172.21.15.36:0/832818451 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0a400e930 con 0x7fd0ac147270 2022-06-17T12:50:45.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.953+0000 7fd0b4090700 1 -- 172.21.15.36:0/832818451 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0ac140270 con 0x7fd0ac147270 2022-06-17T12:50:45.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.953+0000 7fd09affd700 1 -- 172.21.15.36:0/832818451 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd0a4018030 con 0x7fd0ac147270 2022-06-17T12:50:45.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.953+0000 7fd0b4090700 1 -- 172.21.15.36:0/832818451 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0ac142910 con 0x7fd0ac147270 2022-06-17T12:50:45.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.953+0000 7fd09affd700 1 --2- 172.21.15.36:0/832818451 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd08c053a40 0x7fd08c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:45.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.953+0000 7fd09affd700 1 -- 172.21.15.36:0/832818451 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fd0a406b3a0 con 0x7fd0ac147270 2022-06-17T12:50:45.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.953+0000 7fd0b1e2c700 1 --2- 172.21.15.36:0/832818451 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd08c053a40 0x7fd08c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:45.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.953+0000 7fd0b1e2c700 1 --2- 172.21.15.36:0/832818451 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd08c053a40 0x7fd08c055f00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fd09c00c000 tx=0x7fd09c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:45.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:45.961+0000 7fd09affd700 1 -- 172.21.15.36:0/832818451 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd0ac142910 con 0x7fd0ac147270 2022-06-17T12:50:46.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.085+0000 7fd0b4090700 1 -- 172.21.15.36:0/832818451 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd0ac1428e0 con 0x7fd0ac147270 2022-06-17T12:50:46.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.089+0000 7fd09affd700 1 -- 172.21.15.36:0/832818451 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v50) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd0a4042590 con 0x7fd0ac147270 2022-06-17T12:50:46.079 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:50:46.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.089+0000 7fd0b4090700 1 -- 172.21.15.36:0/832818451 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd08c053a40 msgr2=0x7fd08c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:46.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.089+0000 7fd0b4090700 1 --2- 172.21.15.36:0/832818451 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd08c053a40 0x7fd08c055f00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fd09c00c000 tx=0x7fd09c008040).stop 2022-06-17T12:50:46.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.089+0000 7fd0b4090700 1 -- 172.21.15.36:0/832818451 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0ac147270 msgr2=0x7fd0ac142bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:46.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.089+0000 7fd0b4090700 1 --2- 172.21.15.36:0/832818451 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0ac147270 0x7fd0ac142bd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7fd0a40032b0 tx=0x7fd0a40090f0).stop 2022-06-17T12:50:46.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.089+0000 7fd0b4090700 1 -- 172.21.15.36:0/832818451 shutdown_connections 2022-06-17T12:50:46.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.089+0000 7fd0b4090700 1 --2- 172.21.15.36:0/832818451 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd08c053a40 0x7fd08c055f00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:46.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.089+0000 7fd0b4090700 1 --2- 172.21.15.36:0/832818451 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0ac147270 0x7fd0ac142bd0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:46.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.089+0000 7fd0b4090700 1 --2- 172.21.15.36:0/832818451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0ac0b64f0 0x7fd0ac141740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:46.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.089+0000 7fd0b4090700 1 --2- 172.21.15.36:0/832818451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0ac0b4120 0x7fd0ac14a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:46.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.089+0000 7fd0b4090700 1 -- 172.21.15.36:0/832818451 >> 172.21.15.36:0/832818451 conn(0x7fd0ac0620f0 msgr2=0x7fd0ac143840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:46.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.089+0000 7fd0b4090700 1 -- 172.21.15.36:0/832818451 shutdown_connections 2022-06-17T12:50:46.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.089+0000 7fd0b4090700 1 -- 172.21.15.36:0/832818451 wait complete. 2022-06-17T12:50:46.092 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-17T12:50:46.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b2934700 1 Processor -- start 2022-06-17T12:50:46.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b2934700 1 -- start start 2022-06-17T12:50:46.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b2934700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f14ac13cf50 0x7f14ac1442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:46.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b2934700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14ac0af4b0 con 0x7f14ac0aef60 2022-06-17T12:50:46.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b2934700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14ac0ba500 con 0x7f14ac13cb70 2022-06-17T12:50:46.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b2934700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ac143da0 con 0x7f14ac13cf50 2022-06-17T12:50:46.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b1131700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f14ac0aef60 0x7f14ac0b9df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51558/0 (socket says 172.21.15.36:51558) 2022-06-17T12:50:46.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b1131700 1 -- 172.21.15.36:0/3690183494 learned_addr learned my addr 172.21.15.36:0/3690183494 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:50:46.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b2133700 1 --2- 172.21.15.36:0/3690183494 >> v2:172.21.15.70:3300/0 conn(0x7f14ac13cf50 0x7f14ac1442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:46.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b0930700 1 -- 172.21.15.36:0/3690183494 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3658788297 0 0) 0x7f14ac0af4b0 con 0x7f14ac0aef60 2022-06-17T12:50:46.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b0930700 1 -- 172.21.15.36:0/3690183494 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1494003260 con 0x7f14ac0aef60 2022-06-17T12:50:46.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b2133700 1 -- 172.21.15.36:0/3690183494 >> v1:172.21.15.70:6789/0 conn(0x7f14ac13cb70 legacy=0x7f14ac143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:46.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b0930700 1 -- 172.21.15.36:0/3690183494 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3491280628 0 0) 0x7f14ac0ba500 con 0x7f14ac13cb70 2022-06-17T12:50:46.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b2133700 1 -- 172.21.15.36:0/3690183494 >> v1:172.21.15.36:6789/0 conn(0x7f14ac0aef60 legacy=0x7f14ac0b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:46.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b2133700 1 -- 172.21.15.36:0/3690183494 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac142f50 con 0x7f14ac13cf50 2022-06-17T12:50:46.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b2133700 1 --2- 172.21.15.36:0/3690183494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14ac13cf50 0x7f14ac1442a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f14a40032f0 tx=0x7f14a40091d0).ready entity=mon.1 client_cookie=f47e4dbdce4e7fd9 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:46.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b0930700 1 -- 172.21.15.36:0/3690183494 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14a400b6c0 con 0x7f14ac13cf50 2022-06-17T12:50:46.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b0930700 1 -- 172.21.15.36:0/3690183494 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a400b820 con 0x7f14ac13cf50 2022-06-17T12:50:46.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b0930700 1 -- 172.21.15.36:0/3690183494 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14a400b990 con 0x7f14ac13cf50 2022-06-17T12:50:46.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b2934700 1 -- 172.21.15.36:0/3690183494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14ac13cf50 msgr2=0x7f14ac1442a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:46.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b2934700 1 --2- 172.21.15.36:0/3690183494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14ac13cf50 0x7f14ac1442a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f14a40032f0 tx=0x7f14a40091d0).stop 2022-06-17T12:50:46.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b2934700 1 -- 172.21.15.36:0/3690183494 shutdown_connections 2022-06-17T12:50:46.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b2934700 1 --2- 172.21.15.36:0/3690183494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14ac13cf50 0x7f14ac1442a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:46.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b2934700 1 -- 172.21.15.36:0/3690183494 >> 172.21.15.36:0/3690183494 conn(0x7f14ac05c550 msgr2=0x7f14ac0b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:46.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b2934700 1 -- 172.21.15.36:0/3690183494 shutdown_connections 2022-06-17T12:50:46.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.225+0000 7f14b2934700 1 -- 172.21.15.36:0/3690183494 wait complete. 2022-06-17T12:50:46.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.229+0000 7f14b2934700 1 Processor -- start 2022-06-17T12:50:46.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.229+0000 7f14b2934700 1 -- start start 2022-06-17T12:50:46.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.229+0000 7f14b2934700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14ac0aef60 0x7f14ac147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:46.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.229+0000 7f14b2934700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14ac13cb70 0x7f14ac13dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:46.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.229+0000 7f14b2934700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14ac13cf50 0x7f14ac1520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:46.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.229+0000 7f14b2934700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac13d6a0 con 0x7f14ac13cb70 2022-06-17T12:50:46.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.229+0000 7f14b2934700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac0b28f0 con 0x7f14ac13cf50 2022-06-17T12:50:46.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.229+0000 7f14b2934700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac0b2b60 con 0x7f14ac0aef60 2022-06-17T12:50:46.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.229+0000 7f14b1131700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14ac13cb70 0x7f14ac13dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:46.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.229+0000 7f14b1131700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14ac13cb70 0x7f14ac13dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34032/0 (socket says 172.21.15.36:34032) 2022-06-17T12:50:46.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.229+0000 7f14b1131700 1 -- 172.21.15.36:0/3074021529 learned_addr learned my addr 172.21.15.36:0/3074021529 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:46.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.229+0000 7f14b2133700 1 --2- 172.21.15.36:0/3074021529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14ac13cf50 0x7f14ac1520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:46.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.229+0000 7f14b1932700 1 --2- 172.21.15.36:0/3074021529 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14ac0aef60 0x7f14ac147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:46.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.229+0000 7f14b1131700 1 -- 172.21.15.36:0/3074021529 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14ac0aef60 msgr2=0x7f14ac147b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:46.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.229+0000 7f14b1131700 1 --2- 172.21.15.36:0/3074021529 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14ac0aef60 0x7f14ac147b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:46.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.229+0000 7f14b1131700 1 -- 172.21.15.36:0/3074021529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14ac13cf50 msgr2=0x7f14ac1520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:46.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.229+0000 7f14b1131700 1 --2- 172.21.15.36:0/3074021529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14ac13cf50 0x7f14ac1520a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:46.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.229+0000 7f14b1131700 1 -- 172.21.15.36:0/3074021529 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a4009470 con 0x7f14ac13cb70 2022-06-17T12:50:46.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.229+0000 7f14b1131700 1 --2- 172.21.15.36:0/3074021529 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14ac13cb70 0x7f14ac13dd70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f14ac0b2a80 tx=0x7f149c007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:46.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.229+0000 7f149affd700 1 -- 172.21.15.36:0/3074021529 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f149c00ce70 con 0x7f14ac13cb70 2022-06-17T12:50:46.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.229+0000 7f149affd700 1 -- 172.21.15.36:0/3074021529 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149c0070d0 con 0x7f14ac13cb70 2022-06-17T12:50:46.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.229+0000 7f14b2934700 1 -- 172.21.15.36:0/3074021529 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ac13d8d0 con 0x7f14ac13cb70 2022-06-17T12:50:46.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.229+0000 7f149affd700 1 -- 172.21.15.36:0/3074021529 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f149c007430 con 0x7f14ac13cb70 2022-06-17T12:50:46.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.229+0000 7f14b2934700 1 -- 172.21.15.36:0/3074021529 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ac13e260 con 0x7f14ac13cb70 2022-06-17T12:50:46.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.233+0000 7f149affd700 1 -- 172.21.15.36:0/3074021529 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f149c010070 con 0x7f14ac13cb70 2022-06-17T12:50:46.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.233+0000 7f14b2934700 1 -- 172.21.15.36:0/3074021529 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ac1431f0 con 0x7f14ac13cb70 2022-06-17T12:50:46.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.233+0000 7f149affd700 1 --2- 172.21.15.36:0/3074021529 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f148c053a60 0x7f148c055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:46.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.233+0000 7f149affd700 1 -- 172.21.15.36:0/3074021529 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f149c06b150 con 0x7f14ac13cb70 2022-06-17T12:50:46.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.233+0000 7f14b1932700 1 --2- 172.21.15.36:0/3074021529 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f148c053a60 0x7f148c055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:46.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.233+0000 7f14b1932700 1 --2- 172.21.15.36:0/3074021529 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f148c053a60 0x7f148c055f20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f14a800d000 tx=0x7f14a8007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:46.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.241+0000 7f149affd700 1 -- 172.21.15.36:0/3074021529 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f149c042290 con 0x7f14ac13cb70 2022-06-17T12:50:46.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.365+0000 7f14b2934700 1 -- 172.21.15.36:0/3074021529 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f14ac057410 con 0x7f14ac13cb70 2022-06-17T12:50:46.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.365+0000 7f149affd700 1 -- 172.21.15.36:0/3074021529 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v50) v1 ==== 84+0+143 (secure 0 0 0) 0x7f149c0458b0 con 0x7f14ac13cb70 2022-06-17T12:50:46.352 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:50:46.352 INFO:teuthology.orchestra.run.smithi036.stdout:[ 2022-06-17T12:50:46.353 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:46.353 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 1, 2022-06-17T12:50:46.354 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "device_health_metrics" 2022-06-17T12:50:46.354 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:50:46.355 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:50:46.355 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 2, 2022-06-17T12:50:46.355 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "rbd" 2022-06-17T12:50:46.355 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:50:46.357 INFO:teuthology.orchestra.run.smithi036.stdout:] 2022-06-17T12:50:46.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.369+0000 7f14b2934700 1 -- 172.21.15.36:0/3074021529 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f148c053a60 msgr2=0x7f148c055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:46.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.369+0000 7f14b2934700 1 --2- 172.21.15.36:0/3074021529 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f148c053a60 0x7f148c055f20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f14a800d000 tx=0x7f14a8007a00).stop 2022-06-17T12:50:46.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.369+0000 7f14b2934700 1 -- 172.21.15.36:0/3074021529 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14ac13cb70 msgr2=0x7f14ac13dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:46.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.369+0000 7f14b2934700 1 --2- 172.21.15.36:0/3074021529 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14ac13cb70 0x7f14ac13dd70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f14ac0b2a80 tx=0x7f149c007aa0).stop 2022-06-17T12:50:46.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.369+0000 7f14b2934700 1 -- 172.21.15.36:0/3074021529 shutdown_connections 2022-06-17T12:50:46.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.369+0000 7f14b2934700 1 --2- 172.21.15.36:0/3074021529 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f148c053a60 0x7f148c055f20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:46.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.369+0000 7f14b2934700 1 --2- 172.21.15.36:0/3074021529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14ac13cf50 0x7f14ac1520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:46.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.369+0000 7f14b2934700 1 --2- 172.21.15.36:0/3074021529 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14ac13cb70 0x7f14ac13dd70 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:46.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.369+0000 7f14b2934700 1 --2- 172.21.15.36:0/3074021529 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14ac0aef60 0x7f14ac147b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:46.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.369+0000 7f14b2934700 1 -- 172.21.15.36:0/3074021529 >> 172.21.15.36:0/3074021529 conn(0x7f14ac05c550 msgr2=0x7f14ac13bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:46.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.369+0000 7f14b2934700 1 -- 172.21.15.36:0/3074021529 shutdown_connections 2022-06-17T12:50:46.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.369+0000 7f14b2934700 1 -- 172.21.15.36:0/3074021529 wait complete. 2022-06-17T12:50:46.372 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-17T12:50:46.373 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-17T12:50:46.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.501+0000 7f54e1404700 1 Processor -- start 2022-06-17T12:50:46.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.501+0000 7f54e1404700 1 -- start start 2022-06-17T12:50:46.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54e1404700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54dc060920 0x7f54dc138ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:46.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54e1404700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f54dc13fe30 0x7f54dc1394e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:46.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54e1404700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0af610 con 0x7f54dc0b11a0 2022-06-17T12:50:46.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54e1404700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc0ae190 con 0x7f54dc13fe30 2022-06-17T12:50:46.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54e1404700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54dc0ae9d0 con 0x7f54dc060920 2022-06-17T12:50:46.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54d3fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f54dc13fe30 0x7f54dc1394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:46.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54dbfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54dc060920 0x7f54dc138ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:46.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54d3fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f54dc13fe30 0x7f54dc1394e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:54420/0 (socket says 172.21.15.36:54420) 2022-06-17T12:50:46.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54d3fff700 1 -- 172.21.15.36:0/1940083570 learned_addr learned my addr 172.21.15.36:0/1940083570 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:46.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54d3fff700 1 -- 172.21.15.36:0/1940083570 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54dc060920 msgr2=0x7f54dc138ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:46.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54d3fff700 1 --2- 172.21.15.36:0/1940083570 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54dc060920 0x7f54dc138ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:46.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54d3fff700 1 -- 172.21.15.36:0/1940083570 >> v1:172.21.15.70:6789/0 conn(0x7f54dc0b11a0 legacy=0x7f54dc0ab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:46.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54d3fff700 1 -- 172.21.15.36:0/1940083570 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54dc0af8b0 con 0x7f54dc13fe30 2022-06-17T12:50:46.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54d3fff700 1 --2- 172.21.15.36:0/1940083570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54dc13fe30 0x7f54dc1394e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f54dc0af1b0 tx=0x7f54c4006eb0).ready entity=mon.1 client_cookie=f726830aa63451d5 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:46.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54db7fe700 1 -- 172.21.15.36:0/1940083570 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54c400d5c0 con 0x7f54dc13fe30 2022-06-17T12:50:46.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54db7fe700 1 -- 172.21.15.36:0/1940083570 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c4000e90 con 0x7f54dc13fe30 2022-06-17T12:50:46.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54db7fe700 1 -- 172.21.15.36:0/1940083570 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54c400d950 con 0x7f54dc13fe30 2022-06-17T12:50:46.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54e1404700 1 -- 172.21.15.36:0/1940083570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54dc13fe30 msgr2=0x7f54dc1394e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:46.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54e1404700 1 --2- 172.21.15.36:0/1940083570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54dc13fe30 0x7f54dc1394e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f54dc0af1b0 tx=0x7f54c4006eb0).stop 2022-06-17T12:50:46.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54e1404700 1 -- 172.21.15.36:0/1940083570 shutdown_connections 2022-06-17T12:50:46.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54e1404700 1 --2- 172.21.15.36:0/1940083570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54dc13fe30 0x7f54dc1394e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:46.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54e1404700 1 --2- 172.21.15.36:0/1940083570 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54dc060920 0x7f54dc138ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:46.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54e1404700 1 -- 172.21.15.36:0/1940083570 >> 172.21.15.36:0/1940083570 conn(0x7f54dc0b5d10 msgr2=0x7f54dc0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:46.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54e1404700 1 -- 172.21.15.36:0/1940083570 shutdown_connections 2022-06-17T12:50:46.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.505+0000 7f54e1404700 1 -- 172.21.15.36:0/1940083570 wait complete. 2022-06-17T12:50:46.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.509+0000 7f54e1404700 1 Processor -- start 2022-06-17T12:50:46.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.509+0000 7f54e1404700 1 -- start start 2022-06-17T12:50:46.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.509+0000 7f54e1404700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54dc060920 0x7f54dc062d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:46.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.509+0000 7f54e1404700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54dc0b11a0 0x7f54dc13ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:46.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.509+0000 7f54e1404700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f54dc13fe30 0x7f54dc13f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:46.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.509+0000 7f54e1404700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54dc0643c0 con 0x7f54dc13fe30 2022-06-17T12:50:46.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.509+0000 7f54e1404700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54dc143a30 con 0x7f54dc060920 2022-06-17T12:50:46.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.509+0000 7f54e1404700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54dc143cd0 con 0x7f54dc0b11a0 2022-06-17T12:50:46.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.509+0000 7f54e0c03700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f54dc13fe30 0x7f54dc13f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:46.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.509+0000 7f54d3fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54dc0b11a0 0x7f54dc13ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:46.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.509+0000 7f54e0c03700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f54dc13fe30 0x7f54dc13f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34046/0 (socket says 172.21.15.36:34046) 2022-06-17T12:50:46.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.509+0000 7f54dbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54dc060920 0x7f54dc062d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:46.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.509+0000 7f54e0c03700 1 -- 172.21.15.36:0/3634428035 learned_addr learned my addr 172.21.15.36:0/3634428035 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:46.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.513+0000 7f54d3fff700 1 -- 172.21.15.36:0/3634428035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54dc060920 msgr2=0x7f54dc062d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:46.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.513+0000 7f54d3fff700 1 --2- 172.21.15.36:0/3634428035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54dc060920 0x7f54dc062d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:46.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.513+0000 7f54d3fff700 1 -- 172.21.15.36:0/3634428035 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f54dc13fe30 msgr2=0x7f54dc13f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:46.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.513+0000 7f54d3fff700 1 --2- 172.21.15.36:0/3634428035 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f54dc13fe30 0x7f54dc13f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:46.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.513+0000 7f54d3fff700 1 -- 172.21.15.36:0/3634428035 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c400b1e0 con 0x7f54dc0b11a0 2022-06-17T12:50:46.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.513+0000 7f54e0c03700 1 --2- 172.21.15.36:0/3634428035 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f54dc13fe30 0x7f54dc13f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:50:46.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.513+0000 7f54d3fff700 1 --2- 172.21.15.36:0/3634428035 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54dc0b11a0 0x7f54dc13ee20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f54dc0af1b0 tx=0x7f54c4015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:46.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.513+0000 7f54d9ffb700 1 -- 172.21.15.36:0/3634428035 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54c4021070 con 0x7f54dc0b11a0 2022-06-17T12:50:46.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.513+0000 7f54d9ffb700 1 -- 172.21.15.36:0/3634428035 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c4011760 con 0x7f54dc0b11a0 2022-06-17T12:50:46.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.513+0000 7f54e1404700 1 -- 172.21.15.36:0/3634428035 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54dc143f70 con 0x7f54dc0b11a0 2022-06-17T12:50:46.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.513+0000 7f54d9ffb700 1 -- 172.21.15.36:0/3634428035 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54c4011ac0 con 0x7f54dc0b11a0 2022-06-17T12:50:46.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.513+0000 7f54e1404700 1 -- 172.21.15.36:0/3634428035 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54dc13f880 con 0x7f54dc0b11a0 2022-06-17T12:50:46.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.513+0000 7f54e1404700 1 -- 172.21.15.36:0/3634428035 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54dc0ac780 con 0x7f54dc0b11a0 2022-06-17T12:50:46.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.513+0000 7f54d9ffb700 1 -- 172.21.15.36:0/3634428035 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f54c400f070 con 0x7f54dc0b11a0 2022-06-17T12:50:46.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.513+0000 7f54d9ffb700 1 --2- 172.21.15.36:0/3634428035 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f54c0053920 0x7f54c0055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:46.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.513+0000 7f54d9ffb700 1 -- 172.21.15.36:0/3634428035 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f54c406c9e0 con 0x7f54dc0b11a0 2022-06-17T12:50:46.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.513+0000 7f54dbfff700 1 --2- 172.21.15.36:0/3634428035 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f54c0053920 0x7f54c0055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:46.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.513+0000 7f54dbfff700 1 --2- 172.21.15.36:0/3634428035 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f54c0053920 0x7f54c0055de0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f54cc008000 tx=0x7f54cc008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:46.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.521+0000 7f54d9ffb700 1 -- 172.21.15.36:0/3634428035 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f54c4043a80 con 0x7f54dc0b11a0 2022-06-17T12:50:46.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:46.649+0000 7f54e1404700 1 -- 172.21.15.36:0/3634428035 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f54dc057410 con 0x7f54dc0b11a0 2022-06-17T12:50:47.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.465+0000 7f54d9ffb700 1 -- 172.21.15.36:0/3634428035 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v51) v1 ==== 133+0+0 (secure 0 0 0) 0x7f54c40470a0 con 0x7f54dc0b11a0 2022-06-17T12:50:47.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.473+0000 7f54e1404700 1 -- 172.21.15.36:0/3634428035 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f54c0053920 msgr2=0x7f54c0055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:47.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.473+0000 7f54e1404700 1 --2- 172.21.15.36:0/3634428035 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f54c0053920 0x7f54c0055de0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f54cc008000 tx=0x7f54cc008040).stop 2022-06-17T12:50:47.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.473+0000 7f54e1404700 1 -- 172.21.15.36:0/3634428035 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54dc0b11a0 msgr2=0x7f54dc13ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:47.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.473+0000 7f54e1404700 1 --2- 172.21.15.36:0/3634428035 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54dc0b11a0 0x7f54dc13ee20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f54dc0af1b0 tx=0x7f54c4015000).stop 2022-06-17T12:50:47.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.473+0000 7f54e1404700 1 -- 172.21.15.36:0/3634428035 shutdown_connections 2022-06-17T12:50:47.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.473+0000 7f54e1404700 1 --2- 172.21.15.36:0/3634428035 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f54c0053920 0x7f54c0055de0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:47.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.473+0000 7f54e1404700 1 --2- 172.21.15.36:0/3634428035 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f54dc13fe30 0x7f54dc13f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:47.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.473+0000 7f54e1404700 1 --2- 172.21.15.36:0/3634428035 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54dc0b11a0 0x7f54dc13ee20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:47.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.473+0000 7f54e1404700 1 --2- 172.21.15.36:0/3634428035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54dc060920 0x7f54dc062d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:47.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.473+0000 7f54e1404700 1 -- 172.21.15.36:0/3634428035 >> 172.21.15.36:0/3634428035 conn(0x7f54dc0b5d10 msgr2=0x7f54dc0ac9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:47.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.473+0000 7f54e1404700 1 -- 172.21.15.36:0/3634428035 shutdown_connections 2022-06-17T12:50:47.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.473+0000 7f54e1404700 1 -- 172.21.15.36:0/3634428035 wait complete. 2022-06-17T12:50:47.477 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' created 2022-06-17T12:50:47.480 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-06-17T12:50:47.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.609+0000 7fd7e6c20700 1 Processor -- start 2022-06-17T12:50:47.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.609+0000 7fd7e6c20700 1 -- start start 2022-06-17T12:50:47.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.609+0000 7fd7e6c20700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7e00b64f0 0x7fd7e0143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:47.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.609+0000 7fd7e6c20700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd7e00b4120 0x7fd7e0146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:47.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.609+0000 7fd7e6c20700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e00b68d0 con 0x7fd7e0147270 2022-06-17T12:50:47.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.609+0000 7fd7e6c20700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e01476d0 con 0x7fd7e00b4120 2022-06-17T12:50:47.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.609+0000 7fd7e6c20700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e00b6e20 con 0x7fd7e00b64f0 2022-06-17T12:50:47.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.609+0000 7fd7dffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd7e00b4120 0x7fd7e0146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:47.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.613+0000 7fd7dffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd7e00b4120 0x7fd7e0146d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34050/0 (socket says 172.21.15.36:34050) 2022-06-17T12:50:47.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.613+0000 7fd7dffff700 1 -- 172.21.15.36:0/3746601425 learned_addr learned my addr 172.21.15.36:0/3746601425 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:47.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.613+0000 7fd7e49bc700 1 --2- 172.21.15.36:0/3746601425 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7e00b64f0 0x7fd7e0143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:47.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.613+0000 7fd7dffff700 1 -- 172.21.15.36:0/3746601425 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7e00b64f0 msgr2=0x7fd7e0143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:47.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.613+0000 7fd7dffff700 1 --2- 172.21.15.36:0/3746601425 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7e00b64f0 0x7fd7e0143150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:47.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.613+0000 7fd7dffff700 1 -- 172.21.15.36:0/3746601425 >> v1:172.21.15.70:6789/0 conn(0x7fd7e0147270 legacy=0x7fd7e0140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:47.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.613+0000 7fd7dffff700 1 -- 172.21.15.36:0/3746601425 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e01468e0 con 0x7fd7e00b4120 2022-06-17T12:50:47.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.613+0000 7fd7dffff700 1 --2- 172.21.15.36:0/3746601425 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7e00b4120 0x7fd7e0146d80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7fd7d4002f70 tx=0x7fd7d4006eb0).ready entity=mon.0 client_cookie=bde2144713bd4322 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:47.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.613+0000 7fd7df7fe700 1 -- 172.21.15.36:0/3746601425 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7d400d5c0 con 0x7fd7e00b4120 2022-06-17T12:50:47.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.613+0000 7fd7df7fe700 1 -- 172.21.15.36:0/3746601425 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d4000e90 con 0x7fd7e00b4120 2022-06-17T12:50:47.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.613+0000 7fd7df7fe700 1 -- 172.21.15.36:0/3746601425 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7d400d950 con 0x7fd7e00b4120 2022-06-17T12:50:47.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.613+0000 7fd7e6c20700 1 -- 172.21.15.36:0/3746601425 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7e00b4120 msgr2=0x7fd7e0146d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:47.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.613+0000 7fd7e6c20700 1 --2- 172.21.15.36:0/3746601425 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7e00b4120 0x7fd7e0146d80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7fd7d4002f70 tx=0x7fd7d4006eb0).stop 2022-06-17T12:50:47.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.613+0000 7fd7e6c20700 1 -- 172.21.15.36:0/3746601425 shutdown_connections 2022-06-17T12:50:47.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.613+0000 7fd7e6c20700 1 --2- 172.21.15.36:0/3746601425 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7e00b4120 0x7fd7e0146d80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:47.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.613+0000 7fd7e6c20700 1 --2- 172.21.15.36:0/3746601425 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7e00b64f0 0x7fd7e0143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:47.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.613+0000 7fd7e6c20700 1 -- 172.21.15.36:0/3746601425 >> 172.21.15.36:0/3746601425 conn(0x7fd7e00620f0 msgr2=0x7fd7e0062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:47.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.613+0000 7fd7e6c20700 1 -- 172.21.15.36:0/3746601425 shutdown_connections 2022-06-17T12:50:47.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.613+0000 7fd7e6c20700 1 -- 172.21.15.36:0/3746601425 wait complete. 2022-06-17T12:50:47.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7e6c20700 1 Processor -- start 2022-06-17T12:50:47.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7e6c20700 1 -- start start 2022-06-17T12:50:47.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7e6c20700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7e00b4120 0x7fd7e01480f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:47.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7e6c20700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7e00b64f0 0x7fd7e00b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:47.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7e6c20700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7e0147270 0x7fd7e0155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:47.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7e6c20700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e00b2280 con 0x7fd7e00b4120 2022-06-17T12:50:47.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7e6c20700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e00b3040 con 0x7fd7e0147270 2022-06-17T12:50:47.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7e6c20700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e00b0fa0 con 0x7fd7e00b64f0 2022-06-17T12:50:47.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7e49bc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7e00b4120 0x7fd7e01480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:47.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7e49bc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7e00b4120 0x7fd7e01480f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34056/0 (socket says 172.21.15.36:34056) 2022-06-17T12:50:47.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7e49bc700 1 -- 172.21.15.36:0/3927002502 learned_addr learned my addr 172.21.15.36:0/3927002502 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:47.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7dffff700 1 --2- 172.21.15.36:0/3927002502 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7e00b64f0 0x7fd7e00b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:47.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7e51bd700 1 --2- 172.21.15.36:0/3927002502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7e0147270 0x7fd7e0155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:47.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7e49bc700 1 -- 172.21.15.36:0/3927002502 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7e00b64f0 msgr2=0x7fd7e00b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:47.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7e49bc700 1 --2- 172.21.15.36:0/3927002502 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7e00b64f0 0x7fd7e00b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:47.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7e49bc700 1 -- 172.21.15.36:0/3927002502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7e0147270 msgr2=0x7fd7e0155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:47.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7e49bc700 1 --2- 172.21.15.36:0/3927002502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7e0147270 0x7fd7e0155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:47.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7e49bc700 1 -- 172.21.15.36:0/3927002502 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d400b1e0 con 0x7fd7e00b4120 2022-06-17T12:50:47.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7e51bd700 1 --2- 172.21.15.36:0/3927002502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7e0147270 0x7fd7e0155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:50:47.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7dffff700 1 --2- 172.21.15.36:0/3927002502 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7e00b64f0 0x7fd7e00b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:50:47.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7e49bc700 1 --2- 172.21.15.36:0/3927002502 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7e00b4120 0x7fd7e01480f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7fd7d00030b0 tx=0x7fd7d0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:47.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7ddffb700 1 -- 172.21.15.36:0/3927002502 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7d0011780 con 0x7fd7e00b4120 2022-06-17T12:50:47.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7ddffb700 1 -- 172.21.15.36:0/3927002502 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d00118e0 con 0x7fd7e00b4120 2022-06-17T12:50:47.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7e6c20700 1 -- 172.21.15.36:0/3927002502 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e00b0bc0 con 0x7fd7e00b4120 2022-06-17T12:50:47.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7ddffb700 1 -- 172.21.15.36:0/3927002502 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7d0011c40 con 0x7fd7e00b4120 2022-06-17T12:50:47.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.617+0000 7fd7e6c20700 1 -- 172.21.15.36:0/3927002502 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e0143390 con 0x7fd7e00b4120 2022-06-17T12:50:47.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.621+0000 7fd7e6c20700 1 -- 172.21.15.36:0/3927002502 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e00b2e80 con 0x7fd7e00b4120 2022-06-17T12:50:47.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.621+0000 7fd7ddffb700 1 -- 172.21.15.36:0/3927002502 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd7d0012420 con 0x7fd7e00b4120 2022-06-17T12:50:47.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.621+0000 7fd7ddffb700 1 --2- 172.21.15.36:0/3927002502 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7c0053a90 0x7fd7c0055f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:47.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.621+0000 7fd7ddffb700 1 -- 172.21.15.36:0/3927002502 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5954+0+0 (secure 0 0 0) 0x7fd7d006bcd0 con 0x7fd7e00b4120 2022-06-17T12:50:47.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.621+0000 7fd7dffff700 1 --2- 172.21.15.36:0/3927002502 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7c0053a90 0x7fd7c0055f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:47.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.621+0000 7fd7dffff700 1 --2- 172.21.15.36:0/3927002502 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7c0053a90 0x7fd7c0055f50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7fd7d401a000 tx=0x7fd7d4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:47.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.629+0000 7fd7ddffb700 1 -- 172.21.15.36:0/3927002502 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd7d0042c40 con 0x7fd7e00b4120 2022-06-17T12:50:47.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:47.757+0000 7fd7e6c20700 1 -- 172.21.15.36:0/3927002502 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fd7e0143640 con 0x7fd7e00b4120 2022-06-17T12:50:48.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.765+0000 7fd7ddffb700 1 -- 172.21.15.36:0/3927002502 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v53) v1 ==== 125+0+0 (secure 0 0 0) 0x7fd7d0046260 con 0x7fd7e00b4120 2022-06-17T12:50:48.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.773+0000 7fd7e6c20700 1 -- 172.21.15.36:0/3927002502 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7c0053a90 msgr2=0x7fd7c0055f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:48.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.773+0000 7fd7e6c20700 1 --2- 172.21.15.36:0/3927002502 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7c0053a90 0x7fd7c0055f50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7fd7d401a000 tx=0x7fd7d4015000).stop 2022-06-17T12:50:48.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.773+0000 7fd7e6c20700 1 -- 172.21.15.36:0/3927002502 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7e00b4120 msgr2=0x7fd7e01480f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:48.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.773+0000 7fd7e6c20700 1 --2- 172.21.15.36:0/3927002502 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7e00b4120 0x7fd7e01480f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7fd7d00030b0 tx=0x7fd7d0008040).stop 2022-06-17T12:50:48.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.773+0000 7fd7e6c20700 1 -- 172.21.15.36:0/3927002502 shutdown_connections 2022-06-17T12:50:48.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.773+0000 7fd7e6c20700 1 --2- 172.21.15.36:0/3927002502 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7c0053a90 0x7fd7c0055f50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:48.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.773+0000 7fd7e6c20700 1 --2- 172.21.15.36:0/3927002502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7e0147270 0x7fd7e0155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:48.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.773+0000 7fd7e6c20700 1 --2- 172.21.15.36:0/3927002502 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7e00b64f0 0x7fd7e00b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:48.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.773+0000 7fd7e6c20700 1 --2- 172.21.15.36:0/3927002502 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7e00b4120 0x7fd7e01480f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:48.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.773+0000 7fd7e6c20700 1 -- 172.21.15.36:0/3927002502 >> 172.21.15.36:0/3927002502 conn(0x7fd7e00620f0 msgr2=0x7fd7e00b4840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:48.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.773+0000 7fd7e6c20700 1 -- 172.21.15.36:0/3927002502 shutdown_connections 2022-06-17T12:50:48.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.773+0000 7fd7e6c20700 1 -- 172.21.15.36:0/3927002502 wait complete. 2022-06-17T12:50:48.771 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' created 2022-06-17T12:50:48.774 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-17T12:50:48.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.909+0000 7fafd36ec700 1 Processor -- start 2022-06-17T12:50:48.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.909+0000 7fafd36ec700 1 -- start start 2022-06-17T12:50:48.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.909+0000 7fafd36ec700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafcc0b0cb0 0x7fafcc142f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:48.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.909+0000 7fafd36ec700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafcc0b44b0 con 0x7fafcc143e80 2022-06-17T12:50:48.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.909+0000 7fafd36ec700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafcc062650 con 0x7fafcc062270 2022-06-17T12:50:48.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.909+0000 7fafd36ec700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc0b6650 con 0x7fafcc0b0cb0 2022-06-17T12:50:48.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.909+0000 7fafd1c89700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fafcc143e80 0x7fafcc146970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51598/0 (socket says 172.21.15.36:51598) 2022-06-17T12:50:48.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.909+0000 7fafd1c89700 1 -- 172.21.15.36:0/4049266805 learned_addr learned my addr 172.21.15.36:0/4049266805 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:50:48.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.909+0000 7fafd0c87700 1 --2- 172.21.15.36:0/4049266805 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafcc0b0cb0 0x7fafcc142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:48.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.909+0000 7fafcbfff700 1 -- 172.21.15.36:0/4049266805 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2825395995 0 0) 0x7fafcc0b44b0 con 0x7fafcc143e80 2022-06-17T12:50:48.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.909+0000 7fafcbfff700 1 -- 172.21.15.36:0/4049266805 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafb8003260 con 0x7fafcc143e80 2022-06-17T12:50:48.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.909+0000 7fafcbfff700 1 -- 172.21.15.36:0/4049266805 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828941245 0 0) 0x7fafcc062650 con 0x7fafcc062270 2022-06-17T12:50:48.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.909+0000 7fafcbfff700 1 -- 172.21.15.36:0/4049266805 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafcc0b44b0 con 0x7fafcc062270 2022-06-17T12:50:48.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.913+0000 7fafd0c87700 1 -- 172.21.15.36:0/4049266805 >> v1:172.21.15.70:6789/0 conn(0x7fafcc062270 legacy=0x7fafcc143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:48.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.913+0000 7fafd0c87700 1 -- 172.21.15.36:0/4049266805 >> v1:172.21.15.36:6789/0 conn(0x7fafcc143e80 legacy=0x7fafcc146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:48.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.913+0000 7fafd0c87700 1 -- 172.21.15.36:0/4049266805 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafcc0b4210 con 0x7fafcc0b0cb0 2022-06-17T12:50:48.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.913+0000 7fafd0c87700 1 --2- 172.21.15.36:0/4049266805 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafcc0b0cb0 0x7fafcc142f20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fafc0002f70 tx=0x7fafc0006df0).ready entity=mon.2 client_cookie=92a6a900acaa7d9d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:48.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.913+0000 7fafcbfff700 1 -- 172.21.15.36:0/4049266805 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafc000d5c0 con 0x7fafcc0b0cb0 2022-06-17T12:50:48.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.913+0000 7fafcbfff700 1 -- 172.21.15.36:0/4049266805 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc0000e90 con 0x7fafcc0b0cb0 2022-06-17T12:50:48.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.913+0000 7fafcbfff700 1 -- 172.21.15.36:0/4049266805 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafc000d910 con 0x7fafcc0b0cb0 2022-06-17T12:50:48.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.913+0000 7fafd36ec700 1 -- 172.21.15.36:0/4049266805 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafcc0b0cb0 msgr2=0x7fafcc142f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:48.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.913+0000 7fafd36ec700 1 --2- 172.21.15.36:0/4049266805 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafcc0b0cb0 0x7fafcc142f20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fafc0002f70 tx=0x7fafc0006df0).stop 2022-06-17T12:50:48.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.913+0000 7fafd36ec700 1 -- 172.21.15.36:0/4049266805 shutdown_connections 2022-06-17T12:50:48.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.913+0000 7fafd36ec700 1 --2- 172.21.15.36:0/4049266805 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafcc0b0cb0 0x7fafcc142f20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:48.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.913+0000 7fafd36ec700 1 -- 172.21.15.36:0/4049266805 >> 172.21.15.36:0/4049266805 conn(0x7fafcc0b9870 msgr2=0x7fafcc0b9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:48.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.913+0000 7fafd36ec700 1 -- 172.21.15.36:0/4049266805 shutdown_connections 2022-06-17T12:50:48.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.913+0000 7fafd36ec700 1 -- 172.21.15.36:0/4049266805 wait complete. 2022-06-17T12:50:48.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.913+0000 7fafd36ec700 1 Processor -- start 2022-06-17T12:50:48.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.913+0000 7fafd36ec700 1 -- start start 2022-06-17T12:50:48.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.917+0000 7fafd36ec700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafcc062270 0x7fafcc155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:48.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.917+0000 7fafd36ec700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafcc143e80 0x7fafcc0ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:48.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.917+0000 7fafd36ec700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fafcc146710 0x7fafcc0b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:48.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.917+0000 7fafd36ec700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc1412f0 con 0x7fafcc146710 2022-06-17T12:50:48.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.917+0000 7fafd36ec700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc146bb0 con 0x7fafcc143e80 2022-06-17T12:50:48.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.917+0000 7fafd36ec700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc146e50 con 0x7fafcc062270 2022-06-17T12:50:48.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.917+0000 7fafd1c89700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fafcc146710 0x7fafcc0b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:48.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.917+0000 7fafd1c89700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fafcc146710 0x7fafcc0b30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34076/0 (socket says 172.21.15.36:34076) 2022-06-17T12:50:48.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.917+0000 7fafd1488700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafcc062270 0x7fafcc155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:48.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.917+0000 7fafd1c89700 1 -- 172.21.15.36:0/3535508025 learned_addr learned my addr 172.21.15.36:0/3535508025 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:48.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.917+0000 7fafd0c87700 1 --2- 172.21.15.36:0/3535508025 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafcc143e80 0x7fafcc0ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:48.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.917+0000 7fafd1c89700 1 -- 172.21.15.36:0/3535508025 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafcc062270 msgr2=0x7fafcc155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:48.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.917+0000 7fafd1c89700 1 --2- 172.21.15.36:0/3535508025 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafcc062270 0x7fafcc155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:48.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.917+0000 7fafd1c89700 1 -- 172.21.15.36:0/3535508025 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafcc143e80 msgr2=0x7fafcc0ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:48.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.917+0000 7fafd1c89700 1 --2- 172.21.15.36:0/3535508025 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafcc143e80 0x7fafcc0ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:48.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.917+0000 7fafd1c89700 1 -- 172.21.15.36:0/3535508025 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc000b180 con 0x7fafcc146710 2022-06-17T12:50:48.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.917+0000 7fafd1488700 1 --2- 172.21.15.36:0/3535508025 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafcc062270 0x7fafcc155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:50:48.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.917+0000 7fafd1c89700 1 --2- 172.21.15.36:0/3535508025 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fafcc146710 0x7fafcc0b30c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7fafc4001490 tx=0x7fafc4007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:48.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.917+0000 7fafca7fc700 1 -- 172.21.15.36:0/3535508025 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafc4005ba0 con 0x7fafcc146710 2022-06-17T12:50:48.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.917+0000 7fafca7fc700 1 -- 172.21.15.36:0/3535508025 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc4005d00 con 0x7fafcc146710 2022-06-17T12:50:48.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.917+0000 7fafd36ec700 1 -- 172.21.15.36:0/3535508025 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafcc152290 con 0x7fafcc146710 2022-06-17T12:50:48.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.917+0000 7fafca7fc700 1 -- 172.21.15.36:0/3535508025 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafc40072c0 con 0x7fafcc146710 2022-06-17T12:50:48.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.917+0000 7fafd36ec700 1 -- 172.21.15.36:0/3535508025 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafcc1528d0 con 0x7fafcc146710 2022-06-17T12:50:48.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.921+0000 7fafd36ec700 1 -- 172.21.15.36:0/3535508025 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafcc0b63b0 con 0x7fafcc146710 2022-06-17T12:50:48.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.921+0000 7fafca7fc700 1 -- 172.21.15.36:0/3535508025 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fafc401a060 con 0x7fafcc146710 2022-06-17T12:50:48.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.921+0000 7fafca7fc700 1 --2- 172.21.15.36:0/3535508025 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fafac0539b0 0x7fafac055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:48.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.921+0000 7fafca7fc700 1 -- 172.21.15.36:0/3535508025 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fafc406b6c0 con 0x7fafcc146710 2022-06-17T12:50:48.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.921+0000 7fafd1488700 1 --2- 172.21.15.36:0/3535508025 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fafac0539b0 0x7fafac055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:48.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.921+0000 7fafd1488700 1 --2- 172.21.15.36:0/3535508025 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fafac0539b0 0x7fafac055e70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fafbc00d000 tx=0x7fafbc007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:48.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:48.925+0000 7fafca7fc700 1 -- 172.21.15.36:0/3535508025 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fafc4042430 con 0x7fafcc146710 2022-06-17T12:50:49.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.093+0000 7fafd36ec700 1 -- 172.21.15.36:0/3535508025 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fafcc057410 con 0x7fafcc146710 2022-06-17T12:50:49.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.101+0000 7fafca7fc700 1 -- 172.21.15.36:0/3535508025 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 9 and data pool 10 v24) v1 ==== 188+0+0 (secure 0 0 0) 0x7fafc4018020 con 0x7fafcc146710 2022-06-17T12:50:49.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.105+0000 7fafd36ec700 1 -- 172.21.15.36:0/3535508025 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fafac0539b0 msgr2=0x7fafac055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:49.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.105+0000 7fafd36ec700 1 --2- 172.21.15.36:0/3535508025 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fafac0539b0 0x7fafac055e70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fafbc00d000 tx=0x7fafbc007a00).stop 2022-06-17T12:50:49.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.105+0000 7fafd36ec700 1 -- 172.21.15.36:0/3535508025 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fafcc146710 msgr2=0x7fafcc0b30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:49.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.105+0000 7fafd36ec700 1 --2- 172.21.15.36:0/3535508025 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fafcc146710 0x7fafcc0b30c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7fafc4001490 tx=0x7fafc4007e00).stop 2022-06-17T12:50:49.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.105+0000 7fafd36ec700 1 -- 172.21.15.36:0/3535508025 shutdown_connections 2022-06-17T12:50:49.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.105+0000 7fafd36ec700 1 --2- 172.21.15.36:0/3535508025 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fafac0539b0 0x7fafac055e70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.105+0000 7fafd36ec700 1 --2- 172.21.15.36:0/3535508025 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fafcc146710 0x7fafcc0b30c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.105+0000 7fafd36ec700 1 --2- 172.21.15.36:0/3535508025 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafcc143e80 0x7fafcc0ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.105+0000 7fafd36ec700 1 --2- 172.21.15.36:0/3535508025 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafcc062270 0x7fafcc155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.105+0000 7fafd36ec700 1 -- 172.21.15.36:0/3535508025 >> 172.21.15.36:0/3535508025 conn(0x7fafcc0b9870 msgr2=0x7fafcc141700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:49.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.105+0000 7fafd36ec700 1 -- 172.21.15.36:0/3535508025 shutdown_connections 2022-06-17T12:50:49.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.105+0000 7fafd36ec700 1 -- 172.21.15.36:0/3535508025 wait complete. 2022-06-17T12:50:49.097 INFO:teuthology.orchestra.run.smithi036.stderr:new fs with metadata pool 9 and data pool 10 2022-06-17T12:50:49.111 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:50:49.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.221+0000 7fef02953700 1 Processor -- start 2022-06-17T12:50:49.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.221+0000 7fef02953700 1 -- start start 2022-06-17T12:50:49.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.221+0000 7fef02953700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7feefc0b2cd0 0x7feefc1465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:49.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.221+0000 7fef02953700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feefc146ae0 0x7feefc146f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:49.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.221+0000 7fef02953700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feefc0b25e0 con 0x7feefc0b2200 2022-06-17T12:50:49.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.221+0000 7fef02953700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feefc0ad790 con 0x7feefc0b2cd0 2022-06-17T12:50:49.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.221+0000 7fef02953700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0aef40 con 0x7feefc146ae0 2022-06-17T12:50:49.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.221+0000 7fef01150700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7feefc0b2cd0 0x7feefc1465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:49.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.221+0000 7fef01150700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7feefc0b2cd0 0x7feefc1465f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34080/0 (socket says 172.21.15.36:34080) 2022-06-17T12:50:49.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.221+0000 7fef01150700 1 -- 172.21.15.36:0/2638985117 learned_addr learned my addr 172.21.15.36:0/2638985117 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:49.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.221+0000 7fef02152700 1 --2- 172.21.15.36:0/2638985117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feefc146ae0 0x7feefc146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:49.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.221+0000 7fef01150700 1 -- 172.21.15.36:0/2638985117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feefc146ae0 msgr2=0x7feefc146f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:49.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.221+0000 7fef01150700 1 --2- 172.21.15.36:0/2638985117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feefc146ae0 0x7feefc146f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.221+0000 7fef01150700 1 -- 172.21.15.36:0/2638985117 >> v1:172.21.15.70:6789/0 conn(0x7feefc0b2200 legacy=0x7feefc0af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:49.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.221+0000 7fef01150700 1 -- 172.21.15.36:0/2638985117 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc0afaf0 con 0x7feefc0b2cd0 2022-06-17T12:50:49.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.225+0000 7fef01150700 1 --2- 172.21.15.36:0/2638985117 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feefc0b2cd0 0x7feefc1465f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7feefc06ad20 tx=0x7feeec009290).ready entity=mon.0 client_cookie=888793a8d7c23fcb server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:49.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.225+0000 7fef0094f700 1 -- 172.21.15.36:0/2638985117 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feeec00b830 con 0x7feefc0b2cd0 2022-06-17T12:50:49.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.225+0000 7fef0094f700 1 -- 172.21.15.36:0/2638985117 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec00b990 con 0x7feefc0b2cd0 2022-06-17T12:50:49.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.225+0000 7fef0094f700 1 -- 172.21.15.36:0/2638985117 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feeec00bb00 con 0x7feefc0b2cd0 2022-06-17T12:50:49.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.225+0000 7fef02953700 1 -- 172.21.15.36:0/2638985117 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feefc0b2cd0 msgr2=0x7feefc1465f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:49.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.225+0000 7fef02953700 1 --2- 172.21.15.36:0/2638985117 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feefc0b2cd0 0x7feefc1465f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7feefc06ad20 tx=0x7feeec009290).stop 2022-06-17T12:50:49.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.225+0000 7fef02953700 1 -- 172.21.15.36:0/2638985117 shutdown_connections 2022-06-17T12:50:49.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.225+0000 7fef02953700 1 --2- 172.21.15.36:0/2638985117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feefc146ae0 0x7feefc146f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.225+0000 7fef02953700 1 --2- 172.21.15.36:0/2638985117 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feefc0b2cd0 0x7feefc1465f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.225+0000 7fef02953700 1 -- 172.21.15.36:0/2638985117 >> 172.21.15.36:0/2638985117 conn(0x7feefc05f770 msgr2=0x7feefc05fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:49.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.225+0000 7fef02953700 1 -- 172.21.15.36:0/2638985117 shutdown_connections 2022-06-17T12:50:49.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.225+0000 7fef02953700 1 -- 172.21.15.36:0/2638985117 wait complete. 2022-06-17T12:50:49.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.229+0000 7fef02953700 1 Processor -- start 2022-06-17T12:50:49.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.229+0000 7fef02953700 1 -- start start 2022-06-17T12:50:49.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.229+0000 7fef02953700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feefc0b2200 0x7feefc14b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:49.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.229+0000 7fef02953700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feefc0b2cd0 0x7feefc156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:49.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.229+0000 7fef02953700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feefc146ae0 0x7feefc15a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:49.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.229+0000 7fef02953700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0ab220 con 0x7feefc0b2cd0 2022-06-17T12:50:49.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.229+0000 7fef02953700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc159f80 con 0x7feefc0b2200 2022-06-17T12:50:49.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.229+0000 7fef02953700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0a70e0 con 0x7feefc146ae0 2022-06-17T12:50:49.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.229+0000 7fef01150700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feefc0b2cd0 0x7feefc156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:49.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.229+0000 7fef01150700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feefc0b2cd0 0x7feefc156960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34088/0 (socket says 172.21.15.36:34088) 2022-06-17T12:50:49.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.229+0000 7fef01150700 1 -- 172.21.15.36:0/1926676899 learned_addr learned my addr 172.21.15.36:0/1926676899 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:49.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.229+0000 7fef02152700 1 --2- 172.21.15.36:0/1926676899 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feefc146ae0 0x7feefc15a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:49.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.229+0000 7fef01951700 1 --2- 172.21.15.36:0/1926676899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feefc0b2200 0x7feefc14b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:49.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.229+0000 7fef01150700 1 -- 172.21.15.36:0/1926676899 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feefc146ae0 msgr2=0x7feefc15a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:49.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.229+0000 7fef01150700 1 --2- 172.21.15.36:0/1926676899 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feefc146ae0 0x7feefc15a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.229+0000 7fef01150700 1 -- 172.21.15.36:0/1926676899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feefc0b2200 msgr2=0x7feefc14b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:49.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.229+0000 7fef01150700 1 --2- 172.21.15.36:0/1926676899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feefc0b2200 0x7feefc14b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.229+0000 7fef01150700 1 -- 172.21.15.36:0/1926676899 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec009530 con 0x7feefc0b2cd0 2022-06-17T12:50:49.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.229+0000 7fef01150700 1 --2- 172.21.15.36:0/1926676899 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feefc0b2cd0 0x7feefc156960 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7feefc06ad20 tx=0x7feeec016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:49.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.229+0000 7feefaffd700 1 -- 172.21.15.36:0/1926676899 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feeec00b830 con 0x7feefc0b2cd0 2022-06-17T12:50:49.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.229+0000 7feefaffd700 1 -- 172.21.15.36:0/1926676899 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec021630 con 0x7feefc0b2cd0 2022-06-17T12:50:49.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.229+0000 7fef02953700 1 -- 172.21.15.36:0/1926676899 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc15a1b0 con 0x7feefc0b2cd0 2022-06-17T12:50:49.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.229+0000 7feefaffd700 1 -- 172.21.15.36:0/1926676899 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feeec021990 con 0x7feefc0b2cd0 2022-06-17T12:50:49.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.229+0000 7fef02953700 1 -- 172.21.15.36:0/1926676899 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc062390 con 0x7feefc0b2cd0 2022-06-17T12:50:49.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.233+0000 7fef02953700 1 -- 172.21.15.36:0/1926676899 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feefc15a340 con 0x7feefc0b2cd0 2022-06-17T12:50:49.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.233+0000 7feefaffd700 1 -- 172.21.15.36:0/1926676899 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7feeec00f070 con 0x7feefc0b2cd0 2022-06-17T12:50:49.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.233+0000 7feefaffd700 1 --2- 172.21.15.36:0/1926676899 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7feedc053ad0 0x7feedc055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:49.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.233+0000 7fef01951700 1 --2- 172.21.15.36:0/1926676899 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7feedc053ad0 0x7feedc055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:49.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.233+0000 7feefaffd700 1 -- 172.21.15.36:0/1926676899 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7feeec06c470 con 0x7feefc0b2cd0 2022-06-17T12:50:49.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.233+0000 7fef01951700 1 --2- 172.21.15.36:0/1926676899 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7feedc053ad0 0x7feedc055f90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7feef000d000 tx=0x7feef0007b70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:49.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.241+0000 7feefaffd700 1 -- 172.21.15.36:0/1926676899 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7feeec043160 con 0x7feefc0b2cd0 2022-06-17T12:50:49.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.365+0000 7fef02953700 1 -- 172.21.15.36:0/1926676899 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feefc0b1950 con 0x7feefc0b2cd0 2022-06-17T12:50:49.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.369+0000 7feefaffd700 1 -- 172.21.15.36:0/1926676899 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16286 (secure 0 0 0) 0x7feeec015070 con 0x7feefc0b2cd0 2022-06-17T12:50:49.356 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:50:49.357 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":54,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:50:49.096081+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:50:46.705901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-06-17T12:50:47.812736+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:50:49.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.369+0000 7fef02953700 1 -- 172.21.15.36:0/1926676899 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7feedc053ad0 msgr2=0x7feedc055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:49.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.369+0000 7fef02953700 1 --2- 172.21.15.36:0/1926676899 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7feedc053ad0 0x7feedc055f90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7feef000d000 tx=0x7feef0007b70).stop 2022-06-17T12:50:49.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.369+0000 7fef02953700 1 -- 172.21.15.36:0/1926676899 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feefc0b2cd0 msgr2=0x7feefc156960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:49.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.369+0000 7fef02953700 1 --2- 172.21.15.36:0/1926676899 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feefc0b2cd0 0x7feefc156960 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7feefc06ad20 tx=0x7feeec016000).stop 2022-06-17T12:50:49.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.369+0000 7fef02953700 1 -- 172.21.15.36:0/1926676899 shutdown_connections 2022-06-17T12:50:49.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.369+0000 7fef02953700 1 --2- 172.21.15.36:0/1926676899 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7feedc053ad0 0x7feedc055f90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.369+0000 7fef02953700 1 --2- 172.21.15.36:0/1926676899 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feefc146ae0 0x7feefc15a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.369+0000 7fef02953700 1 --2- 172.21.15.36:0/1926676899 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feefc0b2cd0 0x7feefc156960 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.369+0000 7fef02953700 1 --2- 172.21.15.36:0/1926676899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feefc0b2200 0x7feefc14b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.369+0000 7fef02953700 1 -- 172.21.15.36:0/1926676899 >> 172.21.15.36:0/1926676899 conn(0x7feefc05f770 msgr2=0x7feefc0ab3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:49.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.373+0000 7fef02953700 1 -- 172.21.15.36:0/1926676899 shutdown_connections 2022-06-17T12:50:49.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.373+0000 7fef02953700 1 -- 172.21.15.36:0/1926676899 wait complete. 2022-06-17T12:50:49.368 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:50:49.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.493+0000 7fc824aba700 1 Processor -- start 2022-06-17T12:50:49.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.493+0000 7fc824aba700 1 -- start start 2022-06-17T12:50:49.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.493+0000 7fc824aba700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc8200b1890 0x7fc82013c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:49.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.493+0000 7fc824aba700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8200b0230 con 0x7fc820060ac0 2022-06-17T12:50:49.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.493+0000 7fc824aba700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc820060ea0 con 0x7fc8200b1ff0 2022-06-17T12:50:49.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.493+0000 7fc824aba700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc820143800 con 0x7fc8200b1890 2022-06-17T12:50:49.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.493+0000 7fc81effd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc8200b1890 0x7fc82013c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:49.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.493+0000 7fc81f7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fc820060ac0 0x7fc82013c2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51624/0 (socket says 172.21.15.36:51624) 2022-06-17T12:50:49.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.493+0000 7fc81f7fe700 1 -- 172.21.15.36:0/1262725867 learned_addr learned my addr 172.21.15.36:0/1262725867 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:50:49.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.493+0000 7fc81effd700 1 -- 172.21.15.36:0/1262725867 >> v1:172.21.15.70:6789/0 conn(0x7fc8200b1ff0 legacy=0x7fc82013f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:49.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.493+0000 7fc81effd700 1 -- 172.21.15.36:0/1262725867 >> v1:172.21.15.36:6789/0 conn(0x7fc820060ac0 legacy=0x7fc82013c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:49.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.493+0000 7fc81effd700 1 -- 172.21.15.36:0/1262725867 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8200afcf0 con 0x7fc8200b1890 2022-06-17T12:50:49.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.493+0000 7fc81effd700 1 --2- 172.21.15.36:0/1262725867 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8200b1890 0x7fc82013c9c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7fc8200626d0 tx=0x7fc8100091d0).ready entity=mon.0 client_cookie=c02cf6130eee301b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:49.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.493+0000 7fc81e7fc700 1 -- 172.21.15.36:0/1262725867 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc810009ea0 con 0x7fc8200b1890 2022-06-17T12:50:49.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.493+0000 7fc81e7fc700 1 -- 172.21.15.36:0/1262725867 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81000ba50 con 0x7fc8200b1890 2022-06-17T12:50:49.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.493+0000 7fc81e7fc700 1 -- 172.21.15.36:0/1262725867 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc81000bdb0 con 0x7fc8200b1890 2022-06-17T12:50:49.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.493+0000 7fc824aba700 1 -- 172.21.15.36:0/1262725867 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8200b1890 msgr2=0x7fc82013c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:49.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.493+0000 7fc824aba700 1 --2- 172.21.15.36:0/1262725867 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8200b1890 0x7fc82013c9c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7fc8200626d0 tx=0x7fc8100091d0).stop 2022-06-17T12:50:49.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.493+0000 7fc824aba700 1 -- 172.21.15.36:0/1262725867 shutdown_connections 2022-06-17T12:50:49.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.493+0000 7fc824aba700 1 --2- 172.21.15.36:0/1262725867 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8200b1890 0x7fc82013c9c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.493+0000 7fc824aba700 1 -- 172.21.15.36:0/1262725867 >> 172.21.15.36:0/1262725867 conn(0x7fc8200b65f0 msgr2=0x7fc8200b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:49.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.493+0000 7fc824aba700 1 -- 172.21.15.36:0/1262725867 shutdown_connections 2022-06-17T12:50:49.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.493+0000 7fc824aba700 1 -- 172.21.15.36:0/1262725867 wait complete. 2022-06-17T12:50:49.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.497+0000 7fc824aba700 1 Processor -- start 2022-06-17T12:50:49.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.497+0000 7fc824aba700 1 -- start start 2022-06-17T12:50:49.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.497+0000 7fc824aba700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc820060ac0 0x7fc82013c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:49.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.497+0000 7fc824aba700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8200b1890 0x7fc820132630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:49.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.497+0000 7fc824aba700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8200b1ff0 0x7fc820132b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:49.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.497+0000 7fc824aba700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8201341b0 con 0x7fc8200b1890 2022-06-17T12:50:49.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.497+0000 7fc824aba700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc820066fd0 con 0x7fc8200b1ff0 2022-06-17T12:50:49.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.497+0000 7fc824aba700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82013e810 con 0x7fc820060ac0 2022-06-17T12:50:49.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.497+0000 7fc81effd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8200b1890 0x7fc820132630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:49.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.497+0000 7fc81effd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8200b1890 0x7fc820132630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34104/0 (socket says 172.21.15.36:34104) 2022-06-17T12:50:49.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.497+0000 7fc81effd700 1 -- 172.21.15.36:0/1767233780 learned_addr learned my addr 172.21.15.36:0/1767233780 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:49.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.497+0000 7fc81f7fe700 1 --2- 172.21.15.36:0/1767233780 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc820060ac0 0x7fc82013c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:49.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.497+0000 7fc81ffff700 1 --2- 172.21.15.36:0/1767233780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8200b1ff0 0x7fc820132b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:49.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.497+0000 7fc81effd700 1 -- 172.21.15.36:0/1767233780 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc820060ac0 msgr2=0x7fc82013c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:49.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.497+0000 7fc81effd700 1 --2- 172.21.15.36:0/1767233780 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc820060ac0 0x7fc82013c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.497+0000 7fc81effd700 1 -- 172.21.15.36:0/1767233780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8200b1ff0 msgr2=0x7fc820132b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:49.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.497+0000 7fc81effd700 1 --2- 172.21.15.36:0/1767233780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8200b1ff0 0x7fc820132b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.497+0000 7fc81effd700 1 -- 172.21.15.36:0/1767233780 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc810009470 con 0x7fc8200b1890 2022-06-17T12:50:49.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.497+0000 7fc81effd700 1 --2- 172.21.15.36:0/1767233780 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8200b1890 0x7fc820132630 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7fc820067110 tx=0x7fc810018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:49.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.497+0000 7fc81cff9700 1 -- 172.21.15.36:0/1767233780 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8100032d0 con 0x7fc8200b1890 2022-06-17T12:50:49.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.497+0000 7fc81cff9700 1 -- 172.21.15.36:0/1767233780 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81001e040 con 0x7fc8200b1890 2022-06-17T12:50:49.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.497+0000 7fc824aba700 1 -- 172.21.15.36:0/1767233780 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc82013e570 con 0x7fc8200b1890 2022-06-17T12:50:49.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.497+0000 7fc81cff9700 1 -- 172.21.15.36:0/1767233780 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc810023630 con 0x7fc8200b1890 2022-06-17T12:50:49.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.501+0000 7fc824aba700 1 -- 172.21.15.36:0/1767233780 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc820133010 con 0x7fc8200b1890 2022-06-17T12:50:49.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.501+0000 7fc81cff9700 1 -- 172.21.15.36:0/1767233780 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fc810029020 con 0x7fc8200b1890 2022-06-17T12:50:49.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.501+0000 7fc824aba700 1 -- 172.21.15.36:0/1767233780 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc820133b60 con 0x7fc8200b1890 2022-06-17T12:50:49.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.501+0000 7fc81cff9700 1 --2- 172.21.15.36:0/1767233780 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc804053ad0 0x7fc804055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:49.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.501+0000 7fc81cff9700 1 -- 172.21.15.36:0/1767233780 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc81006f9a0 con 0x7fc8200b1890 2022-06-17T12:50:49.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.501+0000 7fc81f7fe700 1 --2- 172.21.15.36:0/1767233780 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc804053ad0 0x7fc804055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:49.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.501+0000 7fc81f7fe700 1 --2- 172.21.15.36:0/1767233780 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc804053ad0 0x7fc804055f90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fc80800d000 tx=0x7fc808009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:49.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.509+0000 7fc81cff9700 1 -- 172.21.15.36:0/1767233780 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc8100466d0 con 0x7fc8200b1890 2022-06-17T12:50:49.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.633+0000 7fc824aba700 1 -- 172.21.15.36:0/1767233780 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8200aff90 con 0x7fc8200b1890 2022-06-17T12:50:49.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.637+0000 7fc81cff9700 1 -- 172.21.15.36:0/1767233780 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16286 (secure 0 0 0) 0x7fc810049cb0 con 0x7fc8200b1890 2022-06-17T12:50:49.621 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:50:49.621 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":54,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:50:49.096081+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:50:46.705901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-06-17T12:50:47.812736+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:50:49.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.637+0000 7fc824aba700 1 -- 172.21.15.36:0/1767233780 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc804053ad0 msgr2=0x7fc804055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:49.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.637+0000 7fc824aba700 1 --2- 172.21.15.36:0/1767233780 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc804053ad0 0x7fc804055f90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fc80800d000 tx=0x7fc808009040).stop 2022-06-17T12:50:49.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.637+0000 7fc824aba700 1 -- 172.21.15.36:0/1767233780 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8200b1890 msgr2=0x7fc820132630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:49.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.637+0000 7fc824aba700 1 --2- 172.21.15.36:0/1767233780 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8200b1890 0x7fc820132630 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7fc820067110 tx=0x7fc810018000).stop 2022-06-17T12:50:49.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.637+0000 7fc824aba700 1 -- 172.21.15.36:0/1767233780 shutdown_connections 2022-06-17T12:50:49.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.637+0000 7fc824aba700 1 --2- 172.21.15.36:0/1767233780 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc804053ad0 0x7fc804055f90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.637+0000 7fc824aba700 1 --2- 172.21.15.36:0/1767233780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8200b1ff0 0x7fc820132b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.637+0000 7fc824aba700 1 --2- 172.21.15.36:0/1767233780 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8200b1890 0x7fc820132630 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.637+0000 7fc824aba700 1 --2- 172.21.15.36:0/1767233780 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc820060ac0 0x7fc82013c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.637+0000 7fc824aba700 1 -- 172.21.15.36:0/1767233780 >> 172.21.15.36:0/1767233780 conn(0x7fc8200b65f0 msgr2=0x7fc82013a040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:49.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.637+0000 7fc824aba700 1 -- 172.21.15.36:0/1767233780 shutdown_connections 2022-06-17T12:50:49.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.637+0000 7fc824aba700 1 -- 172.21.15.36:0/1767233780 wait complete. 2022-06-17T12:50:49.636 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-17T12:50:49.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.765+0000 7f40ebfff700 1 Processor -- start 2022-06-17T12:50:49.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.765+0000 7f40ebfff700 1 -- start start 2022-06-17T12:50:49.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.765+0000 7f40ebfff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f40ec0b9860 0x7f40ec143840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:49.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.765+0000 7f40ebfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f40ec0b45c0 0x7f40ec0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:49.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.765+0000 7f40ebfff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40ec0b4160 con 0x7f40ec0b4f00 2022-06-17T12:50:49.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.765+0000 7f40ebfff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40ec0b9c40 con 0x7f40ec0b45c0 2022-06-17T12:50:49.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.765+0000 7f40ebfff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40ec0627d0 con 0x7f40ec0b9860 2022-06-17T12:50:49.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.765+0000 7f40e27fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f40ec0b45c0 0x7f40ec0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:49.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.765+0000 7f40eb7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f40ec0b4f00 0x7f40ec146490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51642/0 (socket says 172.21.15.36:51642) 2022-06-17T12:50:49.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.765+0000 7f40eb7fe700 1 -- 172.21.15.36:0/4272754267 learned_addr learned my addr 172.21.15.36:0/4272754267 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:50:49.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.769+0000 7f40e27fc700 1 -- 172.21.15.36:0/4272754267 >> v2:172.21.15.70:3300/0 conn(0x7f40ec0b9860 msgr2=0x7f40ec143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:49.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.765+0000 7f40eaffd700 1 --2- 172.21.15.36:0/4272754267 >> v2:172.21.15.70:3300/0 conn(0x7f40ec0b9860 0x7f40ec143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:49.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.769+0000 7f40ea7fc700 1 -- 172.21.15.36:0/4272754267 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1338790683 0 0) 0x7f40ec0b4160 con 0x7f40ec0b4f00 2022-06-17T12:50:49.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.769+0000 7f40e27fc700 1 --2- 172.21.15.36:0/4272754267 >> v2:172.21.15.70:3300/0 conn(0x7f40ec0b9860 0x7f40ec143840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.769+0000 7f40e27fc700 1 -- 172.21.15.36:0/4272754267 >> v1:172.21.15.36:6789/0 conn(0x7f40ec0b4f00 legacy=0x7f40ec146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:49.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.769+0000 7f40e27fc700 1 -- 172.21.15.36:0/4272754267 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec062a70 con 0x7f40ec0b45c0 2022-06-17T12:50:49.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.769+0000 7f40eaffd700 1 --2- 172.21.15.36:0/4272754267 >> v2:172.21.15.70:3300/0 conn(0x7f40ec0b9860 0x7f40ec143840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:50:49.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.769+0000 7f40e27fc700 1 --2- 172.21.15.36:0/4272754267 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f40ec0b45c0 0x7f40ec0b4a10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f40d4002f70 tx=0x7f40d4006df0).ready entity=mon.0 client_cookie=8ebb49f718ff2459 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:49.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.769+0000 7f40ea7fc700 1 -- 172.21.15.36:0/4272754267 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40d400d5c0 con 0x7f40ec0b45c0 2022-06-17T12:50:49.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.769+0000 7f40ea7fc700 1 -- 172.21.15.36:0/4272754267 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d4000e90 con 0x7f40ec0b45c0 2022-06-17T12:50:49.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.769+0000 7f40ea7fc700 1 -- 172.21.15.36:0/4272754267 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40d400d910 con 0x7f40ec0b45c0 2022-06-17T12:50:49.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.769+0000 7f40ebfff700 1 -- 172.21.15.36:0/4272754267 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f40ec0b45c0 msgr2=0x7f40ec0b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:49.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.769+0000 7f40ebfff700 1 --2- 172.21.15.36:0/4272754267 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f40ec0b45c0 0x7f40ec0b4a10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f40d4002f70 tx=0x7f40d4006df0).stop 2022-06-17T12:50:49.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.769+0000 7f40ebfff700 1 -- 172.21.15.36:0/4272754267 shutdown_connections 2022-06-17T12:50:49.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.769+0000 7f40ebfff700 1 --2- 172.21.15.36:0/4272754267 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f40ec0b45c0 0x7f40ec0b4a10 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.769+0000 7f40ebfff700 1 --2- 172.21.15.36:0/4272754267 >> v2:172.21.15.70:3300/0 conn(0x7f40ec0b9860 0x7f40ec143840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.769+0000 7f40ebfff700 1 -- 172.21.15.36:0/4272754267 >> 172.21.15.36:0/4272754267 conn(0x7f40ec05c550 msgr2=0x7f40ec060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:49.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.769+0000 7f40ebfff700 1 -- 172.21.15.36:0/4272754267 shutdown_connections 2022-06-17T12:50:49.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.769+0000 7f40ebfff700 1 -- 172.21.15.36:0/4272754267 wait complete. 2022-06-17T12:50:49.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.769+0000 7f40ebfff700 1 Processor -- start 2022-06-17T12:50:49.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.769+0000 7f40ebfff700 1 -- start start 2022-06-17T12:50:49.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.773+0000 7f40ebfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40ec0b45c0 0x7f40ec0ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:49.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.773+0000 7f40ebfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f40ec0b4f00 0x7f40ec13da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:49.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.773+0000 7f40ebfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40ec0b9860 0x7f40ec152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:49.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.773+0000 7f40ebfff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec13fec0 con 0x7f40ec0b4f00 2022-06-17T12:50:49.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.773+0000 7f40ebfff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec152140 con 0x7f40ec0b45c0 2022-06-17T12:50:49.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.773+0000 7f40ebfff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec155ac0 con 0x7f40ec0b9860 2022-06-17T12:50:49.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.773+0000 7f40e27fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f40ec0b4f00 0x7f40ec13da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:49.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.773+0000 7f40e27fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f40ec0b4f00 0x7f40ec13da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34116/0 (socket says 172.21.15.36:34116) 2022-06-17T12:50:49.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.773+0000 7f40e27fc700 1 -- 172.21.15.36:0/2668588168 learned_addr learned my addr 172.21.15.36:0/2668588168 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:49.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.773+0000 7f40eb7fe700 1 --2- 172.21.15.36:0/2668588168 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40ec0b9860 0x7f40ec152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:49.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.773+0000 7f40eaffd700 1 --2- 172.21.15.36:0/2668588168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40ec0b45c0 0x7f40ec0ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:49.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.773+0000 7f40eb7fe700 1 -- 172.21.15.36:0/2668588168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40ec0b45c0 msgr2=0x7f40ec0ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:49.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.773+0000 7f40eb7fe700 1 --2- 172.21.15.36:0/2668588168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40ec0b45c0 0x7f40ec0ad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.773+0000 7f40eb7fe700 1 -- 172.21.15.36:0/2668588168 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f40ec0b4f00 msgr2=0x7f40ec13da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:49.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.773+0000 7f40eb7fe700 1 --2- 172.21.15.36:0/2668588168 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f40ec0b4f00 0x7f40ec13da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:49.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.773+0000 7f40eb7fe700 1 -- 172.21.15.36:0/2668588168 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d400b180 con 0x7f40ec0b9860 2022-06-17T12:50:49.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.773+0000 7f40e27fc700 1 --2- 172.21.15.36:0/2668588168 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f40ec0b4f00 0x7f40ec13da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:50:49.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.773+0000 7f40eb7fe700 1 --2- 172.21.15.36:0/2668588168 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40ec0b9860 0x7f40ec152790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f40ec13fcb0 tx=0x7f40e4007d20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:49.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.773+0000 7f40e8ff9700 1 -- 172.21.15.36:0/2668588168 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40e400cea0 con 0x7f40ec0b9860 2022-06-17T12:50:49.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.773+0000 7f40e8ff9700 1 -- 172.21.15.36:0/2668588168 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e4005ba0 con 0x7f40ec0b9860 2022-06-17T12:50:49.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.773+0000 7f40ebfff700 1 -- 172.21.15.36:0/2668588168 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ec152330 con 0x7f40ec0b9860 2022-06-17T12:50:49.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.773+0000 7f40e8ff9700 1 -- 172.21.15.36:0/2668588168 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40e40048a0 con 0x7f40ec0b9860 2022-06-17T12:50:49.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.773+0000 7f40ebfff700 1 -- 172.21.15.36:0/2668588168 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ec13d210 con 0x7f40ec0b9860 2022-06-17T12:50:49.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.777+0000 7f40ebfff700 1 -- 172.21.15.36:0/2668588168 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40ec062ee0 con 0x7f40ec0b9860 2022-06-17T12:50:49.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.777+0000 7f40e8ff9700 1 -- 172.21.15.36:0/2668588168 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f40e401b070 con 0x7f40ec0b9860 2022-06-17T12:50:49.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.777+0000 7f40e8ff9700 1 --2- 172.21.15.36:0/2668588168 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f40d0053920 0x7f40d0055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:49.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.777+0000 7f40e8ff9700 1 -- 172.21.15.36:0/2668588168 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f40e406b730 con 0x7f40ec0b9860 2022-06-17T12:50:49.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.777+0000 7f40eaffd700 1 --2- 172.21.15.36:0/2668588168 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f40d0053920 0x7f40d0055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:49.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.777+0000 7f40eaffd700 1 --2- 172.21.15.36:0/2668588168 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f40d0053920 0x7f40d0055de0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f40dc00d000 tx=0x7f40dc008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:49.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.785+0000 7f40e8ff9700 1 -- 172.21.15.36:0/2668588168 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f40e40424b0 con 0x7f40ec0b9860 2022-06-17T12:50:49.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:49.965+0000 7f40ebfff700 1 -- 172.21.15.36:0/2668588168 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f40ec057410 con 0x7f40ec0b9860 2022-06-17T12:50:50.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.105+0000 7f40e8ff9700 1 -- 172.21.15.36:0/2668588168 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) v1 ==== 118+0+0 (secure 0 0 0) 0x7f40e4045ad0 con 0x7f40ec0b9860 2022-06-17T12:50:50.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.105+0000 7f40ebfff700 1 -- 172.21.15.36:0/2668588168 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f40d0053920 msgr2=0x7f40d0055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:50.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.105+0000 7f40ebfff700 1 --2- 172.21.15.36:0/2668588168 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f40d0053920 0x7f40d0055de0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f40dc00d000 tx=0x7f40dc008040).stop 2022-06-17T12:50:50.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.105+0000 7f40ebfff700 1 -- 172.21.15.36:0/2668588168 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40ec0b9860 msgr2=0x7f40ec152790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:50.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.105+0000 7f40ebfff700 1 --2- 172.21.15.36:0/2668588168 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40ec0b9860 0x7f40ec152790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f40ec13fcb0 tx=0x7f40e4007d20).stop 2022-06-17T12:50:50.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.105+0000 7f40ebfff700 1 -- 172.21.15.36:0/2668588168 shutdown_connections 2022-06-17T12:50:50.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.105+0000 7f40ebfff700 1 --2- 172.21.15.36:0/2668588168 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f40d0053920 0x7f40d0055de0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.105+0000 7f40ebfff700 1 --2- 172.21.15.36:0/2668588168 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40ec0b9860 0x7f40ec152790 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.105+0000 7f40ebfff700 1 --2- 172.21.15.36:0/2668588168 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f40ec0b4f00 0x7f40ec13da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.105+0000 7f40ebfff700 1 --2- 172.21.15.36:0/2668588168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40ec0b45c0 0x7f40ec0ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.105+0000 7f40ebfff700 1 -- 172.21.15.36:0/2668588168 >> 172.21.15.36:0/2668588168 conn(0x7f40ec05c550 msgr2=0x7f40ec146b10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:50.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.109+0000 7f40ebfff700 1 -- 172.21.15.36:0/2668588168 shutdown_connections 2022-06-17T12:50:50.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.109+0000 7f40ebfff700 1 -- 172.21.15.36:0/2668588168 wait complete. 2022-06-17T12:50:50.108 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:50:50.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.225+0000 7f098c88a700 1 Processor -- start 2022-06-17T12:50:50.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.225+0000 7f098c88a700 1 -- start start 2022-06-17T12:50:50.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.225+0000 7f098c88a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0988064f70 0x7f09880b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:50.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.225+0000 7f098c88a700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0988063b10 con 0x7f0988063730 2022-06-17T12:50:50.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.225+0000 7f098c88a700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f098814ad20 con 0x7f098814a8f0 2022-06-17T12:50:50.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.225+0000 7f098c88a700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09880b2e80 con 0x7f0988064f70 2022-06-17T12:50:50.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.225+0000 7f0986d9d700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f0988063730 0x7f0988064860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51652/0 (socket says 172.21.15.36:51652) 2022-06-17T12:50:50.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.225+0000 7f0986d9d700 1 -- 172.21.15.36:0/3277293961 learned_addr learned my addr 172.21.15.36:0/3277293961 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:50:50.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.225+0000 7f098659c700 1 --2- 172.21.15.36:0/3277293961 >> v2:172.21.15.36:3300/0 conn(0x7f0988064f70 0x7f09880b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:50.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.225+0000 7f0985d9b700 1 -- 172.21.15.36:0/3277293961 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4264666553 0 0) 0x7f0988063b10 con 0x7f0988063730 2022-06-17T12:50:50.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.225+0000 7f0985d9b700 1 -- 172.21.15.36:0/3277293961 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f096c003260 con 0x7f0988063730 2022-06-17T12:50:50.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.229+0000 7f098659c700 1 -- 172.21.15.36:0/3277293961 >> v1:172.21.15.70:6789/0 conn(0x7f098814a8f0 legacy=0x7f0988146790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:50.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.229+0000 7f0985d9b700 1 -- 172.21.15.36:0/3277293961 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1536374543 0 0) 0x7f096c003260 con 0x7f0988063730 2022-06-17T12:50:50.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.229+0000 7f098659c700 1 -- 172.21.15.36:0/3277293961 >> v1:172.21.15.36:6789/0 conn(0x7f0988063730 legacy=0x7f0988064860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:50.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.229+0000 7f098659c700 1 -- 172.21.15.36:0/3277293961 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098805ffb0 con 0x7f0988064f70 2022-06-17T12:50:50.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.229+0000 7f098659c700 1 --2- 172.21.15.36:0/3277293961 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0988064f70 0x7f09880b3bf0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f0980001b80 tx=0x7f097c006df0).ready entity=mon.0 client_cookie=87a2d508881744a1 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:50.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.229+0000 7f0985d9b700 1 -- 172.21.15.36:0/3277293961 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09800019e0 con 0x7f0988064f70 2022-06-17T12:50:50.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.229+0000 7f0985d9b700 1 -- 172.21.15.36:0/3277293961 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f097c000e90 con 0x7f0988064f70 2022-06-17T12:50:50.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.229+0000 7f0985d9b700 1 -- 172.21.15.36:0/3277293961 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f097c00d4f0 con 0x7f0988064f70 2022-06-17T12:50:50.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.229+0000 7f098c88a700 1 -- 172.21.15.36:0/3277293961 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0988064f70 msgr2=0x7f09880b3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:50.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.229+0000 7f098c88a700 1 --2- 172.21.15.36:0/3277293961 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0988064f70 0x7f09880b3bf0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f0980001b80 tx=0x7f097c006df0).stop 2022-06-17T12:50:50.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.229+0000 7f098c88a700 1 -- 172.21.15.36:0/3277293961 shutdown_connections 2022-06-17T12:50:50.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.229+0000 7f098c88a700 1 --2- 172.21.15.36:0/3277293961 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0988064f70 0x7f09880b3bf0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.229+0000 7f098c88a700 1 -- 172.21.15.36:0/3277293961 >> 172.21.15.36:0/3277293961 conn(0x7f09880614b0 msgr2=0x7f09880618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:50.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.229+0000 7f098c88a700 1 -- 172.21.15.36:0/3277293961 shutdown_connections 2022-06-17T12:50:50.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.229+0000 7f098c88a700 1 -- 172.21.15.36:0/3277293961 wait complete. 2022-06-17T12:50:50.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.229+0000 7f098c88a700 1 Processor -- start 2022-06-17T12:50:50.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.229+0000 7f098c88a700 1 -- start start 2022-06-17T12:50:50.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.233+0000 7f098c88a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0988063730 0x7f098814c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:50.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.233+0000 7f098c88a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0988064f70 0x7f09880b6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:50.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.233+0000 7f098c88a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f098814a8f0 0x7f09881571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:50.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.233+0000 7f098c88a700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098814cff0 con 0x7f0988064f70 2022-06-17T12:50:50.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.233+0000 7f098c88a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09880ada70 con 0x7f098814a8f0 2022-06-17T12:50:50.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.233+0000 7f098c88a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09880add10 con 0x7f0988063730 2022-06-17T12:50:50.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.233+0000 7f098659c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0988064f70 0x7f09880b6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:50.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.233+0000 7f098659c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0988064f70 0x7f09880b6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34130/0 (socket says 172.21.15.36:34130) 2022-06-17T12:50:50.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.233+0000 7f098659c700 1 -- 172.21.15.36:0/2233392457 learned_addr learned my addr 172.21.15.36:0/2233392457 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:50.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.233+0000 7f0986d9d700 1 --2- 172.21.15.36:0/2233392457 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0988063730 0x7f098814c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:50.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.233+0000 7f098759e700 1 --2- 172.21.15.36:0/2233392457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f098814a8f0 0x7f09881571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:50.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.233+0000 7f098659c700 1 -- 172.21.15.36:0/2233392457 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0988063730 msgr2=0x7f098814c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:50.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.233+0000 7f098659c700 1 --2- 172.21.15.36:0/2233392457 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0988063730 0x7f098814c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.233+0000 7f098659c700 1 -- 172.21.15.36:0/2233392457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f098814a8f0 msgr2=0x7f09881571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:50.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.233+0000 7f098659c700 1 --2- 172.21.15.36:0/2233392457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f098814a8f0 0x7f09881571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.233+0000 7f098659c700 1 -- 172.21.15.36:0/2233392457 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097c00b180 con 0x7f0988064f70 2022-06-17T12:50:50.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.233+0000 7f098759e700 1 --2- 172.21.15.36:0/2233392457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f098814a8f0 0x7f09881571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:50:50.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.233+0000 7f0986d9d700 1 --2- 172.21.15.36:0/2233392457 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0988063730 0x7f098814c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:50:50.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.233+0000 7f098659c700 1 --2- 172.21.15.36:0/2233392457 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0988064f70 0x7f09880b6550 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f0980001b80 tx=0x7f097c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:50.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.233+0000 7f0977fff700 1 -- 172.21.15.36:0/2233392457 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f097c002f50 con 0x7f0988064f70 2022-06-17T12:50:50.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.233+0000 7f0977fff700 1 -- 172.21.15.36:0/2233392457 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f097c00de40 con 0x7f0988064f70 2022-06-17T12:50:50.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.233+0000 7f098c88a700 1 -- 172.21.15.36:0/2233392457 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09880adfb0 con 0x7f0988064f70 2022-06-17T12:50:50.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.233+0000 7f0977fff700 1 -- 172.21.15.36:0/2233392457 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f097c01e5f0 con 0x7f0988064f70 2022-06-17T12:50:50.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.233+0000 7f098c88a700 1 -- 172.21.15.36:0/2233392457 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09880b6a40 con 0x7f0988064f70 2022-06-17T12:50:50.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.237+0000 7f098c88a700 1 -- 172.21.15.36:0/2233392457 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09880ad8d0 con 0x7f0988064f70 2022-06-17T12:50:50.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.237+0000 7f0977fff700 1 -- 172.21.15.36:0/2233392457 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f097c025030 con 0x7f0988064f70 2022-06-17T12:50:50.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.237+0000 7f0977fff700 1 --2- 172.21.15.36:0/2233392457 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0964053a40 0x7f0964055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:50.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.237+0000 7f0977fff700 1 -- 172.21.15.36:0/2233392457 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f097c06cd20 con 0x7f0988064f70 2022-06-17T12:50:50.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.237+0000 7f0986d9d700 1 --2- 172.21.15.36:0/2233392457 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0964053a40 0x7f0964055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:50.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.237+0000 7f0986d9d700 1 --2- 172.21.15.36:0/2233392457 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0964053a40 0x7f0964055f00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f098000b000 tx=0x7f0980000f40).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:50.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.245+0000 7f0977fff700 1 -- 172.21.15.36:0/2233392457 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f09880ad8d0 con 0x7f0988064f70 2022-06-17T12:50:50.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.397+0000 7f098c88a700 1 -- 172.21.15.36:0/2233392457 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09880ad8d0 con 0x7f0988064f70 2022-06-17T12:50:50.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.401+0000 7f0977fff700 1 -- 172.21.15.36:0/2233392457 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f09880ad8d0 con 0x7f0988064f70 2022-06-17T12:50:50.389 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:50:50.389 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":25,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5670,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.36:6835/2598001220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2598001220},{"type":"v1","addr":"172.21.15.36:6835","nonce":2598001220}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":23}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:50:49.096455+0000","modified":"2022-06-17T12:50:50.099539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"b","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/2723288138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2723288138},{"type":"v1","addr":"172.21.15.70:6833","nonce":2723288138}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-06-17T12:50:50.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.401+0000 7f098c88a700 1 -- 172.21.15.36:0/2233392457 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0964053a40 msgr2=0x7f0964055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:50.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.401+0000 7f098c88a700 1 --2- 172.21.15.36:0/2233392457 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0964053a40 0x7f0964055f00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f098000b000 tx=0x7f0980000f40).stop 2022-06-17T12:50:50.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.401+0000 7f098c88a700 1 -- 172.21.15.36:0/2233392457 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0988064f70 msgr2=0x7f09880b6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:50.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.401+0000 7f098c88a700 1 --2- 172.21.15.36:0/2233392457 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0988064f70 0x7f09880b6550 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f0980001b80 tx=0x7f097c014000).stop 2022-06-17T12:50:50.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.401+0000 7f098c88a700 1 -- 172.21.15.36:0/2233392457 shutdown_connections 2022-06-17T12:50:50.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.401+0000 7f098c88a700 1 --2- 172.21.15.36:0/2233392457 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0964053a40 0x7f0964055f00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.401+0000 7f098c88a700 1 --2- 172.21.15.36:0/2233392457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f098814a8f0 0x7f09881571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.401+0000 7f098c88a700 1 --2- 172.21.15.36:0/2233392457 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0988064f70 0x7f09880b6550 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.401+0000 7f098c88a700 1 --2- 172.21.15.36:0/2233392457 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0988063730 0x7f098814c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.401+0000 7f098c88a700 1 -- 172.21.15.36:0/2233392457 >> 172.21.15.36:0/2233392457 conn(0x7f09880614b0 msgr2=0x7f0988066c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:50.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.401+0000 7f098c88a700 1 -- 172.21.15.36:0/2233392457 shutdown_connections 2022-06-17T12:50:50.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.401+0000 7f098c88a700 1 -- 172.21.15.36:0/2233392457 wait complete. 2022-06-17T12:50:50.402 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 25 2022-06-17T12:50:50.405 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:50:50.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3eedad700 1 Processor -- start 2022-06-17T12:50:50.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3eedad700 1 -- start start 2022-06-17T12:50:50.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3eedad700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb3e80aef60 0x7fb3e80b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:50.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3eedad700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb3e80ba2e0 0x7fb3e81442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:50.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3eedad700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e80af4b0 con 0x7fb3e813cb70 2022-06-17T12:50:50.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3eedad700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e8143da0 con 0x7fb3e80aef60 2022-06-17T12:50:50.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3eedad700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e8142f50 con 0x7fb3e80ba2e0 2022-06-17T12:50:50.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3ed5aa700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb3e80aef60 0x7fb3e80b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:50.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3ed5aa700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb3e80aef60 0x7fb3e80b9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34138/0 (socket says 172.21.15.36:34138) 2022-06-17T12:50:50.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3ed5aa700 1 -- 172.21.15.36:0/3284235519 learned_addr learned my addr 172.21.15.36:0/3284235519 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:50.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3eddab700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fb3e813cb70 0x7fb3e8143690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51666/0 (socket says 172.21.15.36:51666) 2022-06-17T12:50:50.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3ee5ac700 1 --2- 172.21.15.36:0/3284235519 >> v2:172.21.15.70:3300/0 conn(0x7fb3e80ba2e0 0x7fb3e81442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:50.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3ed5aa700 1 -- 172.21.15.36:0/3284235519 >> v2:172.21.15.70:3300/0 conn(0x7fb3e80ba2e0 msgr2=0x7fb3e81442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:50.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3ed5aa700 1 --2- 172.21.15.36:0/3284235519 >> v2:172.21.15.70:3300/0 conn(0x7fb3e80ba2e0 0x7fb3e81442a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3ed5aa700 1 -- 172.21.15.36:0/3284235519 >> v1:172.21.15.36:6789/0 conn(0x7fb3e813cb70 legacy=0x7fb3e8143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:50.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3ed5aa700 1 -- 172.21.15.36:0/3284235519 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e81431f0 con 0x7fb3e80aef60 2022-06-17T12:50:50.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3ed5aa700 1 --2- 172.21.15.36:0/3284235519 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3e80aef60 0x7fb3e80b9df0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7fb3d8002f70 tx=0x7fb3d8006eb0).ready entity=mon.0 client_cookie=7ad11a6c80f8ef16 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:50.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3ecda9700 1 -- 172.21.15.36:0/3284235519 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3d800d610 con 0x7fb3e80aef60 2022-06-17T12:50:50.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3ecda9700 1 -- 172.21.15.36:0/3284235519 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d8000e90 con 0x7fb3e80aef60 2022-06-17T12:50:50.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3ecda9700 1 -- 172.21.15.36:0/3284235519 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3d800d9a0 con 0x7fb3e80aef60 2022-06-17T12:50:50.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3eedad700 1 -- 172.21.15.36:0/3284235519 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3e80aef60 msgr2=0x7fb3e80b9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:50.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3eedad700 1 --2- 172.21.15.36:0/3284235519 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3e80aef60 0x7fb3e80b9df0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7fb3d8002f70 tx=0x7fb3d8006eb0).stop 2022-06-17T12:50:50.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3eedad700 1 -- 172.21.15.36:0/3284235519 shutdown_connections 2022-06-17T12:50:50.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3eedad700 1 --2- 172.21.15.36:0/3284235519 >> v2:172.21.15.70:3300/0 conn(0x7fb3e80ba2e0 0x7fb3e81442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3eedad700 1 --2- 172.21.15.36:0/3284235519 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3e80aef60 0x7fb3e80b9df0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3eedad700 1 -- 172.21.15.36:0/3284235519 >> 172.21.15.36:0/3284235519 conn(0x7fb3e805c550 msgr2=0x7fb3e80b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:50.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3eedad700 1 -- 172.21.15.36:0/3284235519 shutdown_connections 2022-06-17T12:50:50.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.541+0000 7fb3eedad700 1 -- 172.21.15.36:0/3284235519 wait complete. 2022-06-17T12:50:50.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3eedad700 1 Processor -- start 2022-06-17T12:50:50.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3eedad700 1 -- start start 2022-06-17T12:50:50.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3eedad700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3e80aef60 0x7fb3e8142100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:50.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3eedad700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3e80ba2e0 0x7fb3e80b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:50.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3eedad700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e813cb70 0x7fb3e80b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:50.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3eedad700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e80b02e0 con 0x7fb3e80ba2e0 2022-06-17T12:50:50.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3eedad700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e813ffa0 con 0x7fb3e813cb70 2022-06-17T12:50:50.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3eedad700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e80b0040 con 0x7fb3e80aef60 2022-06-17T12:50:50.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3ed5aa700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3e80ba2e0 0x7fb3e80b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:50.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3eddab700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3e80aef60 0x7fb3e8142100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:50.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3ee5ac700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e813cb70 0x7fb3e80b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:50.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3ee5ac700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e813cb70 0x7fb3e80b2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:54528/0 (socket says 172.21.15.36:54528) 2022-06-17T12:50:50.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3eddab700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3e80aef60 0x7fb3e8142100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:48932/0 (socket says 172.21.15.36:48932) 2022-06-17T12:50:50.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3ed5aa700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3e80ba2e0 0x7fb3e80b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34144/0 (socket says 172.21.15.36:34144) 2022-06-17T12:50:50.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3ee5ac700 1 -- 172.21.15.36:0/2500288019 learned_addr learned my addr 172.21.15.36:0/2500288019 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:50.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3ed5aa700 1 -- 172.21.15.36:0/2500288019 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3e80aef60 msgr2=0x7fb3e8142100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:50.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3ed5aa700 1 --2- 172.21.15.36:0/2500288019 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3e80aef60 0x7fb3e8142100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3ed5aa700 1 -- 172.21.15.36:0/2500288019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e813cb70 msgr2=0x7fb3e80b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:50.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3ed5aa700 1 --2- 172.21.15.36:0/2500288019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e813cb70 0x7fb3e80b2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3ed5aa700 1 -- 172.21.15.36:0/2500288019 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d800b1e0 con 0x7fb3e80ba2e0 2022-06-17T12:50:50.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3ee5ac700 1 --2- 172.21.15.36:0/2500288019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e813cb70 0x7fb3e80b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:50:50.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3eddab700 1 --2- 172.21.15.36:0/2500288019 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3e80aef60 0x7fb3e8142100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:50:50.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3ed5aa700 1 --2- 172.21.15.36:0/2500288019 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3e80ba2e0 0x7fb3e80b2620 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fb3d8002f70 tx=0x7fb3d8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:50.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3d6ffd700 1 -- 172.21.15.36:0/2500288019 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3d8011760 con 0x7fb3e80ba2e0 2022-06-17T12:50:50.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3d6ffd700 1 -- 172.21.15.36:0/2500288019 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d80118c0 con 0x7fb3e80ba2e0 2022-06-17T12:50:50.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3eedad700 1 -- 172.21.15.36:0/2500288019 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e8140400 con 0x7fb3e80ba2e0 2022-06-17T12:50:50.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3d6ffd700 1 -- 172.21.15.36:0/2500288019 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3d8011c20 con 0x7fb3e80ba2e0 2022-06-17T12:50:50.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.545+0000 7fb3eedad700 1 -- 172.21.15.36:0/2500288019 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e80b3630 con 0x7fb3e80ba2e0 2022-06-17T12:50:50.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.549+0000 7fb3eedad700 1 -- 172.21.15.36:0/2500288019 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e8057410 con 0x7fb3e80ba2e0 2022-06-17T12:50:50.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.549+0000 7fb3d6ffd700 1 -- 172.21.15.36:0/2500288019 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fb3d801a040 con 0x7fb3e80ba2e0 2022-06-17T12:50:50.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.549+0000 7fb3d6ffd700 1 --2- 172.21.15.36:0/2500288019 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb3c8053a40 0x7fb3c8055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:50.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.549+0000 7fb3d6ffd700 1 -- 172.21.15.36:0/2500288019 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb3d806d040 con 0x7fb3e80ba2e0 2022-06-17T12:50:50.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.549+0000 7fb3eddab700 1 --2- 172.21.15.36:0/2500288019 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb3c8053a40 0x7fb3c8055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:50.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.549+0000 7fb3eddab700 1 --2- 172.21.15.36:0/2500288019 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb3c8053a40 0x7fb3c8055f00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fb3e400d000 tx=0x7fb3e4007b80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:50.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.557+0000 7fb3d6ffd700 1 -- 172.21.15.36:0/2500288019 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb3d8043e50 con 0x7fb3e80ba2e0 2022-06-17T12:50:50.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.681+0000 7fb3eedad700 1 -- 172.21.15.36:0/2500288019 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3e8140160 con 0x7fb3e80ba2e0 2022-06-17T12:50:50.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.681+0000 7fb3d6ffd700 1 -- 172.21.15.36:0/2500288019 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16286 (secure 0 0 0) 0x7fb3d8047470 con 0x7fb3e80ba2e0 2022-06-17T12:50:50.667 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:50:50.668 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":55,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:50:50.100178+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:50:46.705901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-06-17T12:50:47.812736+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:50:50.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.681+0000 7fb3eedad700 1 -- 172.21.15.36:0/2500288019 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb3c8053a40 msgr2=0x7fb3c8055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:50.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.681+0000 7fb3eedad700 1 --2- 172.21.15.36:0/2500288019 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb3c8053a40 0x7fb3c8055f00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fb3e400d000 tx=0x7fb3e4007b80).stop 2022-06-17T12:50:50.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.681+0000 7fb3eedad700 1 -- 172.21.15.36:0/2500288019 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3e80ba2e0 msgr2=0x7fb3e80b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:50.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.681+0000 7fb3eedad700 1 --2- 172.21.15.36:0/2500288019 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3e80ba2e0 0x7fb3e80b2620 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fb3d8002f70 tx=0x7fb3d8015000).stop 2022-06-17T12:50:50.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.681+0000 7fb3eedad700 1 -- 172.21.15.36:0/2500288019 shutdown_connections 2022-06-17T12:50:50.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.681+0000 7fb3eedad700 1 --2- 172.21.15.36:0/2500288019 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb3c8053a40 0x7fb3c8055f00 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.681+0000 7fb3eedad700 1 --2- 172.21.15.36:0/2500288019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e813cb70 0x7fb3e80b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.681+0000 7fb3eedad700 1 --2- 172.21.15.36:0/2500288019 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3e80ba2e0 0x7fb3e80b2620 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.681+0000 7fb3eedad700 1 --2- 172.21.15.36:0/2500288019 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3e80aef60 0x7fb3e8142100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.681+0000 7fb3eedad700 1 -- 172.21.15.36:0/2500288019 >> 172.21.15.36:0/2500288019 conn(0x7fb3e805c550 msgr2=0x7fb3e8144900 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:50.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.681+0000 7fb3eedad700 1 -- 172.21.15.36:0/2500288019 shutdown_connections 2022-06-17T12:50:50.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.681+0000 7fb3eedad700 1 -- 172.21.15.36:0/2500288019 wait complete. 2022-06-17T12:50:50.682 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:50:50.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.817+0000 7f238f59e700 1 Processor -- start 2022-06-17T12:50:50.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.817+0000 7f238f59e700 1 -- start start 2022-06-17T12:50:50.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.817+0000 7f238f59e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f239013cb70 0x7f2390143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:50.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.817+0000 7f238f59e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f239013cf50 0x7f23901442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:50.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.817+0000 7f238f59e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23900af4b0 con 0x7f23900aef60 2022-06-17T12:50:50.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.817+0000 7f238f59e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23900ba500 con 0x7f239013cb70 2022-06-17T12:50:50.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.817+0000 7f238f59e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2390142f50 con 0x7f239013cf50 2022-06-17T12:50:50.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.817+0000 7f238e59c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f239013cb70 0x7f2390143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:50.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.817+0000 7f238dd9b700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f23900aef60 0x7f23900b9df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51682/0 (socket says 172.21.15.36:51682) 2022-06-17T12:50:50.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.817+0000 7f238dd9b700 1 -- 172.21.15.36:0/1346103385 learned_addr learned my addr 172.21.15.36:0/1346103385 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:50:50.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.817+0000 7f238ed9d700 1 --2- 172.21.15.36:0/1346103385 >> v2:172.21.15.70:3300/0 conn(0x7f239013cf50 0x7f23901442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:50.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.817+0000 7f238d59a700 1 -- 172.21.15.36:0/1346103385 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636175763 0 0) 0x7f23900af4b0 con 0x7f23900aef60 2022-06-17T12:50:50.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.817+0000 7f238d59a700 1 -- 172.21.15.36:0/1346103385 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2374003260 con 0x7f23900aef60 2022-06-17T12:50:50.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.817+0000 7f238e59c700 1 -- 172.21.15.36:0/1346103385 >> v2:172.21.15.70:3300/0 conn(0x7f239013cf50 msgr2=0x7f23901442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:50.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.817+0000 7f238e59c700 1 --2- 172.21.15.36:0/1346103385 >> v2:172.21.15.70:3300/0 conn(0x7f239013cf50 0x7f23901442a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.817+0000 7f238e59c700 1 -- 172.21.15.36:0/1346103385 >> v1:172.21.15.36:6789/0 conn(0x7f23900aef60 legacy=0x7f23900b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:50.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.817+0000 7f238d59a700 1 -- 172.21.15.36:0/1346103385 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1662278786 0 0) 0x7f2374003260 con 0x7f23900aef60 2022-06-17T12:50:50.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.817+0000 7f238e59c700 1 -- 172.21.15.36:0/1346103385 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23901431f0 con 0x7f239013cb70 2022-06-17T12:50:50.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.817+0000 7f238ed9d700 1 --2- 172.21.15.36:0/1346103385 >> v2:172.21.15.70:3300/0 conn(0x7f239013cf50 0x7f23901442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:50:50.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.817+0000 7f238e59c700 1 --2- 172.21.15.36:0/1346103385 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f239013cb70 0x7f2390143690 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f2384001800 tx=0x7f2380006df0).ready entity=mon.0 client_cookie=515040e7926a936c server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:50.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.817+0000 7f238d59a700 1 -- 172.21.15.36:0/1346103385 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2384001660 con 0x7f239013cb70 2022-06-17T12:50:50.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.817+0000 7f238d59a700 1 -- 172.21.15.36:0/1346103385 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2380000e90 con 0x7f239013cb70 2022-06-17T12:50:50.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.817+0000 7f238d59a700 1 -- 172.21.15.36:0/1346103385 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2380012460 con 0x7f239013cb70 2022-06-17T12:50:50.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.821+0000 7f238f59e700 1 -- 172.21.15.36:0/1346103385 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f239013cb70 msgr2=0x7f2390143690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:50.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.821+0000 7f238f59e700 1 --2- 172.21.15.36:0/1346103385 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f239013cb70 0x7f2390143690 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f2384001800 tx=0x7f2380006df0).stop 2022-06-17T12:50:50.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.821+0000 7f238f59e700 1 -- 172.21.15.36:0/1346103385 shutdown_connections 2022-06-17T12:50:50.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.821+0000 7f238f59e700 1 --2- 172.21.15.36:0/1346103385 >> v2:172.21.15.70:3300/0 conn(0x7f239013cf50 0x7f23901442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.821+0000 7f238f59e700 1 --2- 172.21.15.36:0/1346103385 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f239013cb70 0x7f2390143690 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.821+0000 7f238f59e700 1 -- 172.21.15.36:0/1346103385 >> 172.21.15.36:0/1346103385 conn(0x7f239005c550 msgr2=0x7f23900b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:50.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.821+0000 7f238f59e700 1 -- 172.21.15.36:0/1346103385 shutdown_connections 2022-06-17T12:50:50.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.821+0000 7f238f59e700 1 -- 172.21.15.36:0/1346103385 wait complete. 2022-06-17T12:50:50.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.821+0000 7f238f59e700 1 Processor -- start 2022-06-17T12:50:50.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.821+0000 7f238f59e700 1 -- start start 2022-06-17T12:50:50.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.825+0000 7f238f59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f23900aef60 0x7f2390140c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:50.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.825+0000 7f238f59e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f239013cb70 0x7f23900b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:50.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.825+0000 7f238f59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f239013cf50 0x7f23900b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:50.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.825+0000 7f238f59e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2390152d10 con 0x7f239013cf50 2022-06-17T12:50:50.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.825+0000 7f238f59e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2390152f30 con 0x7f239013cb70 2022-06-17T12:50:50.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.825+0000 7f238f59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2390153190 con 0x7f23900aef60 2022-06-17T12:50:50.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.825+0000 7f238ed9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f239013cf50 0x7f23900b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:50.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.825+0000 7f238e59c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f23900aef60 0x7f2390140c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:50.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.825+0000 7f238ed9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f239013cf50 0x7f23900b2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34160/0 (socket says 172.21.15.36:34160) 2022-06-17T12:50:50.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.825+0000 7f238ed9d700 1 -- 172.21.15.36:0/3573270826 learned_addr learned my addr 172.21.15.36:0/3573270826 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:50.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.825+0000 7f238dd9b700 1 --2- 172.21.15.36:0/3573270826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f239013cb70 0x7f23900b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:50.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.825+0000 7f238ed9d700 1 -- 172.21.15.36:0/3573270826 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f23900aef60 msgr2=0x7f2390140c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:50.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.825+0000 7f238ed9d700 1 --2- 172.21.15.36:0/3573270826 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f23900aef60 0x7f2390140c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.825+0000 7f238ed9d700 1 -- 172.21.15.36:0/3573270826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f239013cb70 msgr2=0x7f23900b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:50.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.825+0000 7f238ed9d700 1 --2- 172.21.15.36:0/3573270826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f239013cb70 0x7f23900b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:50.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.825+0000 7f238ed9d700 1 -- 172.21.15.36:0/3573270826 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2388009470 con 0x7f239013cf50 2022-06-17T12:50:50.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.825+0000 7f238ed9d700 1 --2- 172.21.15.36:0/3573270826 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f239013cf50 0x7f23900b2c90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f238800c000 tx=0x7f23880091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:50.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.825+0000 7f237f7fe700 1 -- 172.21.15.36:0/3573270826 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f238800ee60 con 0x7f239013cf50 2022-06-17T12:50:50.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.825+0000 7f237f7fe700 1 -- 172.21.15.36:0/3573270826 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238800c850 con 0x7f239013cf50 2022-06-17T12:50:50.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.825+0000 7f238f59e700 1 -- 172.21.15.36:0/3573270826 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f238000b180 con 0x7f239013cf50 2022-06-17T12:50:50.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.825+0000 7f237f7fe700 1 -- 172.21.15.36:0/3573270826 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f238800cbb0 con 0x7f239013cf50 2022-06-17T12:50:50.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.825+0000 7f238f59e700 1 -- 172.21.15.36:0/3573270826 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2390141120 con 0x7f239013cf50 2022-06-17T12:50:50.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.825+0000 7f238f59e700 1 -- 172.21.15.36:0/3573270826 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2390057410 con 0x7f239013cf50 2022-06-17T12:50:50.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.829+0000 7f237f7fe700 1 -- 172.21.15.36:0/3573270826 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2388022070 con 0x7f239013cf50 2022-06-17T12:50:50.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.829+0000 7f237f7fe700 1 --2- 172.21.15.36:0/3573270826 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f236c053920 0x7f236c055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:50.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.829+0000 7f237f7fe700 1 -- 172.21.15.36:0/3573270826 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f238806c830 con 0x7f239013cf50 2022-06-17T12:50:50.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.829+0000 7f238e59c700 1 --2- 172.21.15.36:0/3573270826 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f236c053920 0x7f236c055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:50.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.829+0000 7f238e59c700 1 --2- 172.21.15.36:0/3573270826 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f236c053920 0x7f236c055de0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f238001a000 tx=0x7f2380014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:50.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:50.833+0000 7f237f7fe700 1 -- 172.21.15.36:0/3573270826 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2388043520 con 0x7f239013cf50 2022-06-17T12:50:50.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.001+0000 7f238f59e700 1 -- 172.21.15.36:0/3573270826 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f2390153360 con 0x7f239013cf50 2022-06-17T12:50:51.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.009+0000 7f237f7fe700 1 -- 172.21.15.36:0/3573270826 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v19) v1 ==== 184+0+0 (secure 0 0 0) 0x7f2388016020 con 0x7f239013cf50 2022-06-17T12:50:51.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.013+0000 7f238f59e700 1 -- 172.21.15.36:0/3573270826 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f236c053920 msgr2=0x7f236c055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:51.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.013+0000 7f238f59e700 1 --2- 172.21.15.36:0/3573270826 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f236c053920 0x7f236c055de0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f238001a000 tx=0x7f2380014000).stop 2022-06-17T12:50:51.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.013+0000 7f238f59e700 1 -- 172.21.15.36:0/3573270826 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f239013cf50 msgr2=0x7f23900b2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:51.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.013+0000 7f238f59e700 1 --2- 172.21.15.36:0/3573270826 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f239013cf50 0x7f23900b2c90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f238800c000 tx=0x7f23880091d0).stop 2022-06-17T12:50:51.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.013+0000 7f238f59e700 1 -- 172.21.15.36:0/3573270826 shutdown_connections 2022-06-17T12:50:51.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.013+0000 7f238f59e700 1 --2- 172.21.15.36:0/3573270826 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f236c053920 0x7f236c055de0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:51.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.013+0000 7f238f59e700 1 --2- 172.21.15.36:0/3573270826 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f239013cf50 0x7f23900b2c90 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:51.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.013+0000 7f238f59e700 1 --2- 172.21.15.36:0/3573270826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f239013cb70 0x7f23900b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:51.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.013+0000 7f238f59e700 1 --2- 172.21.15.36:0/3573270826 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f23900aef60 0x7f2390140c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:51.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.013+0000 7f238f59e700 1 -- 172.21.15.36:0/3573270826 >> 172.21.15.36:0/3573270826 conn(0x7f239005c550 msgr2=0x7f23901442a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:51.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.013+0000 7f238f59e700 1 -- 172.21.15.36:0/3573270826 shutdown_connections 2022-06-17T12:50:51.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.013+0000 7f238f59e700 1 -- 172.21.15.36:0/3573270826 wait complete. 2022-06-17T12:50:51.020 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.0 2022-06-17T12:50:51.021 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:50:51.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.149+0000 7fd8a720b700 1 Processor -- start 2022-06-17T12:50:51.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.149+0000 7fd8a720b700 1 -- start start 2022-06-17T12:50:51.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a720b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8a00b1890 0x7fd8a013c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:51.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a720b700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a00b0230 con 0x7fd8a00b1ff0 2022-06-17T12:50:51.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a720b700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a0060ea0 con 0x7fd8a0060ac0 2022-06-17T12:50:51.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a720b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a0143800 con 0x7fd8a00b1890 2022-06-17T12:50:51.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a6a0a700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fd8a00b1ff0 0x7fd8a013f270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51696/0 (socket says 172.21.15.36:51696) 2022-06-17T12:50:51.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a6a0a700 1 -- 172.21.15.36:0/2565998807 learned_addr learned my addr 172.21.15.36:0/2565998807 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:50:51.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a5a08700 1 --2- 172.21.15.36:0/2565998807 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8a00b1890 0x7fd8a013c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:51.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a5207700 1 -- 172.21.15.36:0/2565998807 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2788338910 0 0) 0x7fd8a00b0230 con 0x7fd8a00b1ff0 2022-06-17T12:50:51.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a5207700 1 -- 172.21.15.36:0/2565998807 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd888003260 con 0x7fd8a00b1ff0 2022-06-17T12:50:51.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a5207700 1 -- 172.21.15.36:0/2565998807 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122832477 0 0) 0x7fd8a0060ea0 con 0x7fd8a0060ac0 2022-06-17T12:50:51.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a5207700 1 -- 172.21.15.36:0/2565998807 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8a00b0230 con 0x7fd8a0060ac0 2022-06-17T12:50:51.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a5207700 1 -- 172.21.15.36:0/2565998807 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2746779008 0 0) 0x7fd888003260 con 0x7fd8a00b1ff0 2022-06-17T12:50:51.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a5207700 1 -- 172.21.15.36:0/2565998807 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8a00b1890 msgr2=0x7fd8a013c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:51.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a5207700 1 --2- 172.21.15.36:0/2565998807 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8a00b1890 0x7fd8a013c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:51.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a5207700 1 -- 172.21.15.36:0/2565998807 >> v1:172.21.15.70:6789/0 conn(0x7fd8a0060ac0 legacy=0x7fd8a013c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:51.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a5207700 1 -- 172.21.15.36:0/2565998807 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a00afcf0 con 0x7fd8a00b1ff0 2022-06-17T12:50:51.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a5207700 1 -- 172.21.15.36:0/2565998807 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7fd898001a00 con 0x7fd8a00b1ff0 2022-06-17T12:50:51.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a5207700 1 -- 172.21.15.36:0/2565998807 >> v1:172.21.15.36:6789/0 conn(0x7fd8a00b1ff0 legacy=0x7fd8a013f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:51.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a5207700 1 --2- 172.21.15.36:0/2565998807 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd888003910 0x7fd888003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:51.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a5207700 1 --2- 172.21.15.36:0/2565998807 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8a00b1890 0x7fd888004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:51.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a5207700 1 --2- 172.21.15.36:0/2565998807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8880049f0 0x7fd888004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:51.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a6209700 1 --2- 172.21.15.36:0/2565998807 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd888003910 0x7fd888003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:51.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a5207700 1 -- 172.21.15.36:0/2565998807 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a0143800 con 0x7fd888003910 2022-06-17T12:50:51.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a5207700 1 -- 172.21.15.36:0/2565998807 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88800c860 con 0x7fd8880049f0 2022-06-17T12:50:51.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a5207700 1 -- 172.21.15.36:0/2565998807 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88800c9d0 con 0x7fd8a00b1890 2022-06-17T12:50:51.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a5a08700 1 --2- 172.21.15.36:0/2565998807 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8a00b1890 0x7fd888004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:51.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a6a0a700 1 --2- 172.21.15.36:0/2565998807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8880049f0 0x7fd888004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:51.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a6209700 1 -- 172.21.15.36:0/2565998807 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8a00b1890 msgr2=0x7fd888004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:51.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a6209700 1 --2- 172.21.15.36:0/2565998807 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8a00b1890 0x7fd888004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:51.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a6209700 1 -- 172.21.15.36:0/2565998807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8880049f0 msgr2=0x7fd888004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:51.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a6209700 1 --2- 172.21.15.36:0/2565998807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8880049f0 0x7fd888004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:51.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a6209700 1 -- 172.21.15.36:0/2565998807 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd888003260 con 0x7fd888003910 2022-06-17T12:50:51.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a6209700 1 --2- 172.21.15.36:0/2565998807 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd888003910 0x7fd888003cf0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7fd89c001110 tx=0x7fd89c007a00).ready entity=mon.0 client_cookie=3bc9c47f87e0cbfb server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:51.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a5207700 1 -- 172.21.15.36:0/2565998807 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd89c007d10 con 0x7fd888003910 2022-06-17T12:50:51.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a5207700 1 -- 172.21.15.36:0/2565998807 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89c007030 con 0x7fd888003910 2022-06-17T12:50:51.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.153+0000 7fd8a5207700 1 -- 172.21.15.36:0/2565998807 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd89c007390 con 0x7fd888003910 2022-06-17T12:50:51.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.157+0000 7fd8a720b700 1 -- 172.21.15.36:0/2565998807 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd888003910 msgr2=0x7fd888003cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:51.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.157+0000 7fd8a720b700 1 --2- 172.21.15.36:0/2565998807 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd888003910 0x7fd888003cf0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7fd89c001110 tx=0x7fd89c007a00).stop 2022-06-17T12:50:51.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.157+0000 7fd8a6a0a700 1 -- 172.21.15.36:0/2565998807 reap_dead start 2022-06-17T12:50:51.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.157+0000 7fd8a720b700 1 -- 172.21.15.36:0/2565998807 shutdown_connections 2022-06-17T12:50:51.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.157+0000 7fd8a720b700 1 -- 172.21.15.36:0/2565998807 >> 172.21.15.36:0/2565998807 conn(0x7fd8a00b65f0 msgr2=0x7fd8a00b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:51.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.157+0000 7fd8a720b700 1 -- 172.21.15.36:0/2565998807 shutdown_connections 2022-06-17T12:50:51.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.157+0000 7fd8a720b700 1 -- 172.21.15.36:0/2565998807 wait complete. 2022-06-17T12:50:51.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.157+0000 7fd8a720b700 1 Processor -- start 2022-06-17T12:50:51.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.157+0000 7fd8a720b700 1 -- start start 2022-06-17T12:50:51.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.161+0000 7fd8a720b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd888003910 0x7fd8a01540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:51.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.161+0000 7fd8a720b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8a0143980 0x7fd8a0153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:51.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.161+0000 7fd8a720b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8a0150cf0 0x7fd8a0064010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:51.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.161+0000 7fd8a720b700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a00659a0 con 0x7fd888003910 2022-06-17T12:50:51.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.161+0000 7fd8a720b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a00669d0 con 0x7fd8a0150cf0 2022-06-17T12:50:51.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.161+0000 7fd8a720b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a0151170 con 0x7fd8a0143980 2022-06-17T12:50:51.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.161+0000 7fd8a6209700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd888003910 0x7fd8a01540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:51.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.161+0000 7fd8a6209700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd888003910 0x7fd8a01540e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34176/0 (socket says 172.21.15.36:34176) 2022-06-17T12:50:51.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.161+0000 7fd8a6209700 1 -- 172.21.15.36:0/55981952 learned_addr learned my addr 172.21.15.36:0/55981952 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:51.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.161+0000 7fd8a5a08700 1 --2- 172.21.15.36:0/55981952 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8a0143980 0x7fd8a0153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:51.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.161+0000 7fd8a6a0a700 1 --2- 172.21.15.36:0/55981952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8a0150cf0 0x7fd8a0064010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:51.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.161+0000 7fd8a6209700 1 -- 172.21.15.36:0/55981952 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8a0143980 msgr2=0x7fd8a0153850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:51.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.161+0000 7fd8a6209700 1 --2- 172.21.15.36:0/55981952 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8a0143980 0x7fd8a0153850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:51.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.161+0000 7fd8a6209700 1 -- 172.21.15.36:0/55981952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8a0150cf0 msgr2=0x7fd8a0064010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:51.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.161+0000 7fd8a6209700 1 --2- 172.21.15.36:0/55981952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8a0150cf0 0x7fd8a0064010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:51.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.161+0000 7fd8a6209700 1 -- 172.21.15.36:0/55981952 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a0066490 con 0x7fd888003910 2022-06-17T12:50:51.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.161+0000 7fd8a6209700 1 --2- 172.21.15.36:0/55981952 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd888003910 0x7fd8a01540e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7fd8a00b0480 tx=0x7fd89c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:51.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.161+0000 7fd88f7fe700 1 -- 172.21.15.36:0/55981952 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd89c013450 con 0x7fd888003910 2022-06-17T12:50:51.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.161+0000 7fd88f7fe700 1 -- 172.21.15.36:0/55981952 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89c0135b0 con 0x7fd888003910 2022-06-17T12:50:51.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.161+0000 7fd8a720b700 1 -- 172.21.15.36:0/55981952 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a0066730 con 0x7fd888003910 2022-06-17T12:50:51.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.161+0000 7fd88f7fe700 1 -- 172.21.15.36:0/55981952 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd89c013910 con 0x7fd888003910 2022-06-17T12:50:51.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.161+0000 7fd8a720b700 1 -- 172.21.15.36:0/55981952 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a00646a0 con 0x7fd888003910 2022-06-17T12:50:51.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.165+0000 7fd88f7fe700 1 -- 172.21.15.36:0/55981952 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd89c027070 con 0x7fd888003910 2022-06-17T12:50:51.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.165+0000 7fd8a720b700 1 -- 172.21.15.36:0/55981952 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a00aff90 con 0x7fd888003910 2022-06-17T12:50:51.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.165+0000 7fd88f7fe700 1 --2- 172.21.15.36:0/55981952 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd880053ad0 0x7fd880055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:51.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.165+0000 7fd88f7fe700 1 -- 172.21.15.36:0/55981952 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd89c06ca30 con 0x7fd888003910 2022-06-17T12:50:51.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.165+0000 7fd8a5a08700 1 --2- 172.21.15.36:0/55981952 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd880053ad0 0x7fd880055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:51.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.165+0000 7fd8a5a08700 1 --2- 172.21.15.36:0/55981952 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd880053ad0 0x7fd880055f90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fd89000b000 tx=0x7fd890006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:51.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.169+0000 7fd88f7fe700 1 -- 172.21.15.36:0/55981952 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd89c043970 con 0x7fd888003910 2022-06-17T12:50:51.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.337+0000 7fd8a720b700 1 -- 172.21.15.36:0/55981952 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fd8a0057410 con 0x7fd888003910 2022-06-17T12:50:51.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.345+0000 7fd88f7fe700 1 -- 172.21.15.36:0/55981952 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v20) v1 ==== 184+0+0 (secure 0 0 0) 0x7fd89c046e50 con 0x7fd888003910 2022-06-17T12:50:51.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.345+0000 7fd8a720b700 1 -- 172.21.15.36:0/55981952 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd880053ad0 msgr2=0x7fd880055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:51.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.345+0000 7fd8a720b700 1 --2- 172.21.15.36:0/55981952 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd880053ad0 0x7fd880055f90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fd89000b000 tx=0x7fd890006df0).stop 2022-06-17T12:50:51.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.345+0000 7fd8a720b700 1 -- 172.21.15.36:0/55981952 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd888003910 msgr2=0x7fd8a01540e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:51.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.345+0000 7fd8a720b700 1 --2- 172.21.15.36:0/55981952 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd888003910 0x7fd8a01540e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7fd8a00b0480 tx=0x7fd89c015000).stop 2022-06-17T12:50:51.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.345+0000 7fd8a720b700 1 -- 172.21.15.36:0/55981952 shutdown_connections 2022-06-17T12:50:51.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.345+0000 7fd8a720b700 1 --2- 172.21.15.36:0/55981952 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd880053ad0 0x7fd880055f90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:51.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.345+0000 7fd8a720b700 1 --2- 172.21.15.36:0/55981952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8a0150cf0 0x7fd8a0064010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:51.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.345+0000 7fd8a720b700 1 --2- 172.21.15.36:0/55981952 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8a0143980 0x7fd8a0153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:51.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.345+0000 7fd8a720b700 1 --2- 172.21.15.36:0/55981952 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd888003910 0x7fd8a01540e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:51.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.345+0000 7fd8a720b700 1 -- 172.21.15.36:0/55981952 >> 172.21.15.36:0/55981952 conn(0x7fd8a00b65f0 msgr2=0x7fd8a00adb60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:51.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.345+0000 7fd8a720b700 1 -- 172.21.15.36:0/55981952 shutdown_connections 2022-06-17T12:50:51.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.345+0000 7fd8a720b700 1 -- 172.21.15.36:0/55981952 wait complete. 2022-06-17T12:50:51.339 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.1 2022-06-17T12:50:51.344 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:50:51.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.489+0000 7f4da36be700 1 Processor -- start 2022-06-17T12:50:51.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.489+0000 7f4da36be700 1 -- start start 2022-06-17T12:50:51.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.489+0000 7f4da36be700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d9c0b16d0 0x7f4d9c141540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:51.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.489+0000 7f4da36be700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d9c065090 con 0x7f4d9c0b12f0 2022-06-17T12:50:51.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.489+0000 7f4da36be700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d9c0b0b40 con 0x7f4d9c062bd0 2022-06-17T12:50:51.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.489+0000 7f4da36be700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c063510 con 0x7f4d9c0b16d0 2022-06-17T12:50:51.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.489+0000 7f4da145a700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f4d9c0b12f0 0x7f4d9c13aac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51714/0 (socket says 172.21.15.36:51714) 2022-06-17T12:50:51.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.489+0000 7f4da145a700 1 -- 172.21.15.36:0/1695600405 learned_addr learned my addr 172.21.15.36:0/1695600405 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:50:51.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.489+0000 7f4da1c5b700 1 --2- 172.21.15.36:0/1695600405 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d9c0b16d0 0x7f4d9c141540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:51.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.489+0000 7f4d93fff700 1 -- 172.21.15.36:0/1695600405 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2043795846 0 0) 0x7f4d9c065090 con 0x7f4d9c0b12f0 2022-06-17T12:50:51.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4d93fff700 1 -- 172.21.15.36:0/1695600405 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d80003260 con 0x7f4d9c0b12f0 2022-06-17T12:50:51.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4d93fff700 1 -- 172.21.15.36:0/1695600405 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3026702530 0 0) 0x7f4d9c0b0b40 con 0x7f4d9c062bd0 2022-06-17T12:50:51.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4d93fff700 1 -- 172.21.15.36:0/1695600405 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d9c065090 con 0x7f4d9c062bd0 2022-06-17T12:50:51.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4da1c5b700 1 -- 172.21.15.36:0/1695600405 >> v1:172.21.15.70:6789/0 conn(0x7f4d9c062bd0 legacy=0x7f4d9c140e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:51.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4da1c5b700 1 -- 172.21.15.36:0/1695600405 >> v1:172.21.15.36:6789/0 conn(0x7f4d9c0b12f0 legacy=0x7f4d9c13aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:51.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4da1c5b700 1 -- 172.21.15.36:0/1695600405 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9c064df0 con 0x7f4d9c0b16d0 2022-06-17T12:50:51.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4da1c5b700 1 --2- 172.21.15.36:0/1695600405 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d9c0b16d0 0x7f4d9c141540 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f4d940032f0 tx=0x7f4d940091d0).ready entity=mon.2 client_cookie=e63b08a2c27f64c4 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:51.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4d93fff700 1 -- 172.21.15.36:0/1695600405 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d9400b640 con 0x7f4d9c0b16d0 2022-06-17T12:50:51.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4d93fff700 1 -- 172.21.15.36:0/1695600405 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9400b7a0 con 0x7f4d9c0b16d0 2022-06-17T12:50:51.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4d93fff700 1 -- 172.21.15.36:0/1695600405 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d9400b910 con 0x7f4d9c0b16d0 2022-06-17T12:50:51.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4da36be700 1 -- 172.21.15.36:0/1695600405 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d9c0b16d0 msgr2=0x7f4d9c141540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:51.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4da36be700 1 --2- 172.21.15.36:0/1695600405 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d9c0b16d0 0x7f4d9c141540 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f4d940032f0 tx=0x7f4d940091d0).stop 2022-06-17T12:50:51.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4da36be700 1 -- 172.21.15.36:0/1695600405 shutdown_connections 2022-06-17T12:50:51.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4da36be700 1 --2- 172.21.15.36:0/1695600405 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d9c0b16d0 0x7f4d9c141540 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:51.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4da36be700 1 -- 172.21.15.36:0/1695600405 >> 172.21.15.36:0/1695600405 conn(0x7f4d9c0b53e0 msgr2=0x7f4d9c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:51.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4da36be700 1 -- 172.21.15.36:0/1695600405 shutdown_connections 2022-06-17T12:50:51.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4da36be700 1 -- 172.21.15.36:0/1695600405 wait complete. 2022-06-17T12:50:51.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4da36be700 1 Processor -- start 2022-06-17T12:50:51.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4da36be700 1 -- start start 2022-06-17T12:50:51.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4da36be700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d9c062bd0 0x7f4d9c13f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:51.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4da36be700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d9c0b12f0 0x7f4d9c145080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:51.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4da36be700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d9c0b16d0 0x7f4d9c05fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:51.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4da36be700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c1404a0 con 0x7f4d9c0b16d0 2022-06-17T12:50:51.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4da36be700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c145570 con 0x7f4d9c062bd0 2022-06-17T12:50:51.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4da36be700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c1463d0 con 0x7f4d9c0b12f0 2022-06-17T12:50:51.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4da1c5b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d9c0b16d0 0x7f4d9c05fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:51.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4da1c5b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d9c0b16d0 0x7f4d9c05fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34192/0 (socket says 172.21.15.36:34192) 2022-06-17T12:50:51.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4da1c5b700 1 -- 172.21.15.36:0/2856586741 learned_addr learned my addr 172.21.15.36:0/2856586741 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:51.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4da0c59700 1 --2- 172.21.15.36:0/2856586741 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d9c0b12f0 0x7f4d9c145080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:51.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.493+0000 7f4da145a700 1 --2- 172.21.15.36:0/2856586741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d9c062bd0 0x7f4d9c13f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:51.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.497+0000 7f4da1c5b700 1 -- 172.21.15.36:0/2856586741 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d9c0b12f0 msgr2=0x7f4d9c145080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:51.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.497+0000 7f4da1c5b700 1 --2- 172.21.15.36:0/2856586741 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d9c0b12f0 0x7f4d9c145080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:51.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.497+0000 7f4da1c5b700 1 -- 172.21.15.36:0/2856586741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d9c062bd0 msgr2=0x7f4d9c13f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:51.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.497+0000 7f4da1c5b700 1 --2- 172.21.15.36:0/2856586741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d9c062bd0 0x7f4d9c13f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:51.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.497+0000 7f4da1c5b700 1 -- 172.21.15.36:0/2856586741 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d94009470 con 0x7f4d9c0b16d0 2022-06-17T12:50:51.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.497+0000 7f4da1c5b700 1 --2- 172.21.15.36:0/2856586741 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d9c0b16d0 0x7f4d9c05fa70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f4d9c145700 tx=0x7f4d94016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:51.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.497+0000 7f4d927fc700 1 -- 172.21.15.36:0/2856586741 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d9400b910 con 0x7f4d9c0b16d0 2022-06-17T12:50:51.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.497+0000 7f4d927fc700 1 -- 172.21.15.36:0/2856586741 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d94011dd0 con 0x7f4d9c0b16d0 2022-06-17T12:50:51.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.497+0000 7f4d927fc700 1 -- 172.21.15.36:0/2856586741 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d9400b640 con 0x7f4d9c0b16d0 2022-06-17T12:50:51.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.497+0000 7f4da36be700 1 -- 172.21.15.36:0/2856586741 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d9c1446a0 con 0x7f4d9c0b16d0 2022-06-17T12:50:51.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.497+0000 7f4da36be700 1 -- 172.21.15.36:0/2856586741 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d9c05ff60 con 0x7f4d9c0b16d0 2022-06-17T12:50:51.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.497+0000 7f4da36be700 1 -- 172.21.15.36:0/2856586741 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d9c0b03c0 con 0x7f4d9c0b16d0 2022-06-17T12:50:51.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.497+0000 7f4d927fc700 1 -- 172.21.15.36:0/2856586741 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f4d9400f070 con 0x7f4d9c0b16d0 2022-06-17T12:50:51.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.501+0000 7f4d927fc700 1 --2- 172.21.15.36:0/2856586741 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d840539b0 0x7f4d84055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:51.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.501+0000 7f4d927fc700 1 -- 172.21.15.36:0/2856586741 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4d9406cc10 con 0x7f4d9c0b16d0 2022-06-17T12:50:51.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.501+0000 7f4da145a700 1 --2- 172.21.15.36:0/2856586741 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d840539b0 0x7f4d84055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:51.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.501+0000 7f4da145a700 1 --2- 172.21.15.36:0/2856586741 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d840539b0 0x7f4d84055e70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f4d9800d000 tx=0x7f4d98009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:51.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.505+0000 7f4d927fc700 1 -- 172.21.15.36:0/2856586741 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4d940438a0 con 0x7f4d9c0b16d0 2022-06-17T12:50:51.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.673+0000 7f4da36be700 1 -- 172.21.15.36:0/2856586741 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f4d9c057410 con 0x7f4d9c0b16d0 2022-06-17T12:50:51.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.685+0000 7f4d927fc700 1 -- 172.21.15.36:0/2856586741 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v21) v1 ==== 184+0+0 (secure 0 0 0) 0x7f4d94015070 con 0x7f4d9c0b16d0 2022-06-17T12:50:51.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.685+0000 7f4da36be700 1 -- 172.21.15.36:0/2856586741 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d840539b0 msgr2=0x7f4d84055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:51.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.685+0000 7f4da36be700 1 --2- 172.21.15.36:0/2856586741 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d840539b0 0x7f4d84055e70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f4d9800d000 tx=0x7f4d98009040).stop 2022-06-17T12:50:51.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.685+0000 7f4da36be700 1 -- 172.21.15.36:0/2856586741 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d9c0b16d0 msgr2=0x7f4d9c05fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:51.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.685+0000 7f4da36be700 1 --2- 172.21.15.36:0/2856586741 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d9c0b16d0 0x7f4d9c05fa70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f4d9c145700 tx=0x7f4d94016000).stop 2022-06-17T12:50:51.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.685+0000 7f4da36be700 1 -- 172.21.15.36:0/2856586741 shutdown_connections 2022-06-17T12:50:51.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.685+0000 7f4da36be700 1 --2- 172.21.15.36:0/2856586741 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d840539b0 0x7f4d84055e70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:51.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.685+0000 7f4da36be700 1 --2- 172.21.15.36:0/2856586741 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d9c0b16d0 0x7f4d9c05fa70 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:51.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.685+0000 7f4da36be700 1 --2- 172.21.15.36:0/2856586741 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d9c0b12f0 0x7f4d9c145080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:51.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.685+0000 7f4da36be700 1 --2- 172.21.15.36:0/2856586741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d9c062bd0 0x7f4d9c13f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:51.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.685+0000 7f4da36be700 1 -- 172.21.15.36:0/2856586741 >> 172.21.15.36:0/2856586741 conn(0x7f4d9c0b53e0 msgr2=0x7f4d9c0b49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:51.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.685+0000 7f4da36be700 1 -- 172.21.15.36:0/2856586741 shutdown_connections 2022-06-17T12:50:51.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.685+0000 7f4da36be700 1 -- 172.21.15.36:0/2856586741 wait complete. 2022-06-17T12:50:51.673 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.2 2022-06-17T12:50:51.699 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:50:51.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.833+0000 7fa7fba54700 1 Processor -- start 2022-06-17T12:50:51.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.833+0000 7fa7fba54700 1 -- start start 2022-06-17T12:50:51.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.833+0000 7fa7fba54700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa7f4062770 0x7fa7f4062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:51.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.833+0000 7fa7fba54700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7f40b0ba0 con 0x7fa7f40ad330 2022-06-17T12:50:51.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.833+0000 7fa7fba54700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7f4147e90 con 0x7fa7f41444f0 2022-06-17T12:50:51.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.833+0000 7fa7fba54700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f40ae820 con 0x7fa7f4062770 2022-06-17T12:50:51.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.833+0000 7fa7f97f0700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa7f4062770 0x7fa7f4062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:51.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.833+0000 7fa7f9ff1700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fa7f40ad330 0x7fa7f40ad7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51732/0 (socket says 172.21.15.36:51732) 2022-06-17T12:50:51.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.833+0000 7fa7f9ff1700 1 -- 172.21.15.36:0/3117927997 learned_addr learned my addr 172.21.15.36:0/3117927997 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:50:51.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.833+0000 7fa7f97f0700 1 -- 172.21.15.36:0/3117927997 >> v1:172.21.15.70:6789/0 conn(0x7fa7f41444f0 legacy=0x7fa7f4144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:51.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.833+0000 7fa7f97f0700 1 -- 172.21.15.36:0/3117927997 >> v1:172.21.15.36:6789/0 conn(0x7fa7f40ad330 legacy=0x7fa7f40ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:51.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.833+0000 7fa7e7fff700 1 -- 172.21.15.36:0/3117927997 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1924574428 0 0) 0x7fa7f40b0ba0 con 0x7fa7f40ad330 2022-06-17T12:50:51.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.833+0000 7fa7f97f0700 1 -- 172.21.15.36:0/3117927997 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f40b0dd0 con 0x7fa7f4062770 2022-06-17T12:50:51.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.833+0000 7fa7f97f0700 1 --2- 172.21.15.36:0/3117927997 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa7f4062770 0x7fa7f4062b50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7fa7f0002f70 tx=0x7fa7f0006df0).ready entity=mon.0 client_cookie=e69efb1384734cd5 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:51.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.833+0000 7fa7e7fff700 1 -- 172.21.15.36:0/3117927997 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7f000d610 con 0x7fa7f4062770 2022-06-17T12:50:51.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.833+0000 7fa7e7fff700 1 -- 172.21.15.36:0/3117927997 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f0000e90 con 0x7fa7f4062770 2022-06-17T12:50:51.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.833+0000 7fa7e7fff700 1 -- 172.21.15.36:0/3117927997 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7f000d960 con 0x7fa7f4062770 2022-06-17T12:50:51.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.837+0000 7fa7fba54700 1 -- 172.21.15.36:0/3117927997 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa7f4062770 msgr2=0x7fa7f4062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:51.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.837+0000 7fa7fba54700 1 --2- 172.21.15.36:0/3117927997 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa7f4062770 0x7fa7f4062b50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7fa7f0002f70 tx=0x7fa7f0006df0).stop 2022-06-17T12:50:51.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.837+0000 7fa7fba54700 1 -- 172.21.15.36:0/3117927997 shutdown_connections 2022-06-17T12:50:51.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.837+0000 7fa7fba54700 1 --2- 172.21.15.36:0/3117927997 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa7f4062770 0x7fa7f4062b50 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:51.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.837+0000 7fa7fba54700 1 -- 172.21.15.36:0/3117927997 >> 172.21.15.36:0/3117927997 conn(0x7fa7f40b53e0 msgr2=0x7fa7f40b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:51.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.837+0000 7fa7fba54700 1 -- 172.21.15.36:0/3117927997 shutdown_connections 2022-06-17T12:50:51.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.837+0000 7fa7fba54700 1 -- 172.21.15.36:0/3117927997 wait complete. 2022-06-17T12:50:51.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.837+0000 7fa7fba54700 1 Processor -- start 2022-06-17T12:50:51.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.837+0000 7fa7fba54700 1 -- start start 2022-06-17T12:50:51.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.837+0000 7fa7fba54700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7f4062770 0x7fa7f414d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:51.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.837+0000 7fa7fba54700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa7f40ad330 0x7fa7f4066fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:51.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.837+0000 7fa7fba54700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7f41444f0 0x7fa7f4159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:51.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.841+0000 7fa7fba54700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f40668f0 con 0x7fa7f40ad330 2022-06-17T12:50:51.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.841+0000 7fa7fba54700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f4066310 con 0x7fa7f4062770 2022-06-17T12:50:51.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.841+0000 7fa7fba54700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f40665b0 con 0x7fa7f41444f0 2022-06-17T12:50:51.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.841+0000 7fa7f8fef700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa7f40ad330 0x7fa7f4066fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:51.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.841+0000 7fa7f8fef700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa7f40ad330 0x7fa7f4066fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34206/0 (socket says 172.21.15.36:34206) 2022-06-17T12:50:51.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.841+0000 7fa7f8fef700 1 -- 172.21.15.36:0/1571267208 learned_addr learned my addr 172.21.15.36:0/1571267208 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:51.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.841+0000 7fa7f97f0700 1 --2- 172.21.15.36:0/1571267208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7f4062770 0x7fa7f414d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:51.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.841+0000 7fa7f9ff1700 1 --2- 172.21.15.36:0/1571267208 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7f41444f0 0x7fa7f4159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:51.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.841+0000 7fa7f8fef700 1 -- 172.21.15.36:0/1571267208 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7f41444f0 msgr2=0x7fa7f4159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:51.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.841+0000 7fa7f8fef700 1 --2- 172.21.15.36:0/1571267208 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7f41444f0 0x7fa7f4159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:51.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.841+0000 7fa7f8fef700 1 -- 172.21.15.36:0/1571267208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7f4062770 msgr2=0x7fa7f414d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:51.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.841+0000 7fa7f8fef700 1 --2- 172.21.15.36:0/1571267208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7f4062770 0x7fa7f414d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:51.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.841+0000 7fa7f8fef700 1 -- 172.21.15.36:0/1571267208 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f000b180 con 0x7fa7f40ad330 2022-06-17T12:50:51.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.841+0000 7fa7f8fef700 1 --2- 172.21.15.36:0/1571267208 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa7f40ad330 0x7fa7f4066fe0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7fa7f405d2b0 tx=0x7fa7e8007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:51.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.841+0000 7fa7e67fc700 1 -- 172.21.15.36:0/1571267208 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7e8012400 con 0x7fa7f40ad330 2022-06-17T12:50:51.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.841+0000 7fa7e67fc700 1 -- 172.21.15.36:0/1571267208 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e8012560 con 0x7fa7f40ad330 2022-06-17T12:50:51.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.841+0000 7fa7fba54700 1 -- 172.21.15.36:0/1571267208 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f4066b40 con 0x7fa7f40ad330 2022-06-17T12:50:51.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.841+0000 7fa7e67fc700 1 -- 172.21.15.36:0/1571267208 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7e80128c0 con 0x7fa7f40ad330 2022-06-17T12:50:51.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.841+0000 7fa7fba54700 1 -- 172.21.15.36:0/1571267208 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f414ac90 con 0x7fa7f40ad330 2022-06-17T12:50:51.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.841+0000 7fa7fba54700 1 -- 172.21.15.36:0/1571267208 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f414b980 con 0x7fa7f40ad330 2022-06-17T12:50:51.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.841+0000 7fa7e67fc700 1 -- 172.21.15.36:0/1571267208 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fa7e8010070 con 0x7fa7f40ad330 2022-06-17T12:50:51.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.845+0000 7fa7e67fc700 1 --2- 172.21.15.36:0/1571267208 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa7dc053a40 0x7fa7dc055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:51.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.845+0000 7fa7f97f0700 1 --2- 172.21.15.36:0/1571267208 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa7dc053a40 0x7fa7dc055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:51.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.845+0000 7fa7e67fc700 1 -- 172.21.15.36:0/1571267208 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa7e806af20 con 0x7fa7f40ad330 2022-06-17T12:50:51.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.845+0000 7fa7f97f0700 1 --2- 172.21.15.36:0/1571267208 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa7dc053a40 0x7fa7dc055f00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fa7f001c000 tx=0x7fa7f0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:51.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:51.853+0000 7fa7e67fc700 1 -- 172.21.15.36:0/1571267208 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa7f414b980 con 0x7fa7f40ad330 2022-06-17T12:50:52.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.013+0000 7fa7fba54700 1 -- 172.21.15.36:0/1571267208 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7f414b980 con 0x7fa7f40ad330 2022-06-17T12:50:52.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.017+0000 7fa7e67fc700 1 -- 172.21.15.36:0/1571267208 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+2139 (secure 0 0 0) 0x7fa7f414b980 con 0x7fa7f40ad330 2022-06-17T12:50:52.007 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:50:52.007 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":25,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5670,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.36:6835/2598001220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2598001220},{"type":"v1","addr":"172.21.15.36:6835","nonce":2598001220}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":23}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:50:49.096455+0000","modified":"2022-06-17T12:50:50.099539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"b","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/2723288138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2723288138},{"type":"v1","addr":"172.21.15.70:6833","nonce":2723288138}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-06-17T12:50:52.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.017+0000 7fa7fba54700 1 -- 172.21.15.36:0/1571267208 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa7dc053a40 msgr2=0x7fa7dc055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:52.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.017+0000 7fa7fba54700 1 --2- 172.21.15.36:0/1571267208 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa7dc053a40 0x7fa7dc055f00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fa7f001c000 tx=0x7fa7f0015000).stop 2022-06-17T12:50:52.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.017+0000 7fa7fba54700 1 -- 172.21.15.36:0/1571267208 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa7f40ad330 msgr2=0x7fa7f4066fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:52.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.017+0000 7fa7fba54700 1 --2- 172.21.15.36:0/1571267208 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa7f40ad330 0x7fa7f4066fe0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7fa7f405d2b0 tx=0x7fa7e8007c50).stop 2022-06-17T12:50:52.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.017+0000 7fa7fba54700 1 -- 172.21.15.36:0/1571267208 shutdown_connections 2022-06-17T12:50:52.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.017+0000 7fa7fba54700 1 --2- 172.21.15.36:0/1571267208 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa7dc053a40 0x7fa7dc055f00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:52.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.017+0000 7fa7fba54700 1 --2- 172.21.15.36:0/1571267208 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7f41444f0 0x7fa7f4159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:52.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.017+0000 7fa7fba54700 1 --2- 172.21.15.36:0/1571267208 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa7f40ad330 0x7fa7f4066fe0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:52.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.017+0000 7fa7fba54700 1 --2- 172.21.15.36:0/1571267208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7f4062770 0x7fa7f414d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:52.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.017+0000 7fa7fba54700 1 -- 172.21.15.36:0/1571267208 >> 172.21.15.36:0/1571267208 conn(0x7fa7f40b53e0 msgr2=0x7fa7f4144960 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:52.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.017+0000 7fa7fba54700 1 -- 172.21.15.36:0/1571267208 shutdown_connections 2022-06-17T12:50:52.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.017+0000 7fa7fba54700 1 -- 172.21.15.36:0/1571267208 wait complete. 2022-06-17T12:50:52.041 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 25 2022-06-17T12:50:52.242 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:50:49.096455+0000', 'modified': '2022-06-17T12:50:50.099539+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5679}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5679': {'gid': 5679, 'name': 'b', 'rank': 0, 'incarnation': 24, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6833/2723288138', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 2723288138}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 2723288138}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:50:52.244 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:50:52.245 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-17T12:50:52.369 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":5679,"want_state":"up:active","state":"up:active","rank_uptime":3.2719230669999999,"mdsmap_epoch":25,"osdmap_epoch":56,"osdmap_epoch_barrier":54,"uptime":8.5728437119999992} 2022-06-17T12:50:52.374 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 5679, "mdsmap_epoch": 25, "osdmap_epoch": 56, "osdmap_epoch_barrier": 54, "rank_uptime": 3.271923067, "state": "up:active", "uptime": 8.572843712, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:50:52.375 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:50:52.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.501+0000 7fb43200a700 1 Processor -- start 2022-06-17T12:50:52.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.501+0000 7fb43200a700 1 -- start start 2022-06-17T12:50:52.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb43200a700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb42c0b27c0 0x7fb42c0acf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:52.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb43200a700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb42c0af6c0 con 0x7fb42c0b2ba0 2022-06-17T12:50:52.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb43200a700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb42c0af920 con 0x7fb42c0b14d0 2022-06-17T12:50:52.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb43200a700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb42c0b18b0 con 0x7fb42c0b27c0 2022-06-17T12:50:52.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb430807700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fb42c0b2ba0 0x7fb42c0afe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51746/0 (socket says 172.21.15.36:51746) 2022-06-17T12:50:52.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb430807700 1 -- 172.21.15.36:0/1759504695 learned_addr learned my addr 172.21.15.36:0/1759504695 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:50:52.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb431008700 1 --2- 172.21.15.36:0/1759504695 >> v2:172.21.15.70:3300/0 conn(0x7fb42c0b27c0 0x7fb42c0acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:52.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb423fff700 1 -- 172.21.15.36:0/1759504695 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378072010 0 0) 0x7fb42c0af6c0 con 0x7fb42c0b2ba0 2022-06-17T12:50:52.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb423fff700 1 -- 172.21.15.36:0/1759504695 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb410004370 con 0x7fb42c0b2ba0 2022-06-17T12:50:52.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb431008700 1 -- 172.21.15.36:0/1759504695 >> v1:172.21.15.70:6789/0 conn(0x7fb42c0b14d0 legacy=0x7fb42c149d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:52.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb431008700 1 -- 172.21.15.36:0/1759504695 >> v1:172.21.15.36:6789/0 conn(0x7fb42c0b2ba0 legacy=0x7fb42c0afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:52.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb423fff700 1 -- 172.21.15.36:0/1759504695 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3634040923 0 0) 0x7fb410004370 con 0x7fb42c0b2ba0 2022-06-17T12:50:52.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb431008700 1 -- 172.21.15.36:0/1759504695 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42c0ad890 con 0x7fb42c0b27c0 2022-06-17T12:50:52.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb431008700 1 --2- 172.21.15.36:0/1759504695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb42c0b27c0 0x7fb42c0acf90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fb41c001820 tx=0x7fb428006df0).ready entity=mon.1 client_cookie=d2f47ba815f61f70 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:52.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb423fff700 1 -- 172.21.15.36:0/1759504695 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb41c001680 con 0x7fb42c0b27c0 2022-06-17T12:50:52.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb423fff700 1 -- 172.21.15.36:0/1759504695 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb428000e90 con 0x7fb42c0b27c0 2022-06-17T12:50:52.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb423fff700 1 -- 172.21.15.36:0/1759504695 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb42800d4f0 con 0x7fb42c0b27c0 2022-06-17T12:50:52.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb43200a700 1 -- 172.21.15.36:0/1759504695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb42c0b27c0 msgr2=0x7fb42c0acf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:52.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb43200a700 1 --2- 172.21.15.36:0/1759504695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb42c0b27c0 0x7fb42c0acf90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fb41c001820 tx=0x7fb428006df0).stop 2022-06-17T12:50:52.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb43200a700 1 -- 172.21.15.36:0/1759504695 shutdown_connections 2022-06-17T12:50:52.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb43200a700 1 --2- 172.21.15.36:0/1759504695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb42c0b27c0 0x7fb42c0acf90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:52.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb43200a700 1 -- 172.21.15.36:0/1759504695 >> 172.21.15.36:0/1759504695 conn(0x7fb42c05f770 msgr2=0x7fb42c05fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:52.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb43200a700 1 -- 172.21.15.36:0/1759504695 shutdown_connections 2022-06-17T12:50:52.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb43200a700 1 -- 172.21.15.36:0/1759504695 wait complete. 2022-06-17T12:50:52.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb43200a700 1 Processor -- start 2022-06-17T12:50:52.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.505+0000 7fb43200a700 1 -- start start 2022-06-17T12:50:52.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb43200a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb42c0b14d0 0x7fb42c0a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:52.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb43200a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb42c0b27c0 0x7fb42c1536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:52.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb43200a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb42c0b2ba0 0x7fb42c0a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:52.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb43200a700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c145690 con 0x7fb42c0b27c0 2022-06-17T12:50:52.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb43200a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c152ad0 con 0x7fb42c0b14d0 2022-06-17T12:50:52.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb43200a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c152d40 con 0x7fb42c0b2ba0 2022-06-17T12:50:52.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb430807700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb42c0b27c0 0x7fb42c1536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:52.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb430807700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb42c0b27c0 0x7fb42c1536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34220/0 (socket says 172.21.15.36:34220) 2022-06-17T12:50:52.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb430807700 1 -- 172.21.15.36:0/1802857053 learned_addr learned my addr 172.21.15.36:0/1802857053 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:52.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb431809700 1 --2- 172.21.15.36:0/1802857053 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb42c0b2ba0 0x7fb42c0a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:52.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb431008700 1 --2- 172.21.15.36:0/1802857053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb42c0b14d0 0x7fb42c0a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:52.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb430807700 1 -- 172.21.15.36:0/1802857053 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb42c0b2ba0 msgr2=0x7fb42c0a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:52.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb430807700 1 --2- 172.21.15.36:0/1802857053 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb42c0b2ba0 0x7fb42c0a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:52.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb430807700 1 -- 172.21.15.36:0/1802857053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb42c0b14d0 msgr2=0x7fb42c0a41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:52.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb430807700 1 --2- 172.21.15.36:0/1802857053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb42c0b14d0 0x7fb42c0a41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:52.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb430807700 1 -- 172.21.15.36:0/1802857053 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42800b180 con 0x7fb42c0b27c0 2022-06-17T12:50:52.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb431008700 1 --2- 172.21.15.36:0/1802857053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb42c0b14d0 0x7fb42c0a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:50:52.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb430807700 1 --2- 172.21.15.36:0/1802857053 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb42c0b27c0 0x7fb42c1536f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fb42c152ed0 tx=0x7fb41c001da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:52.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb4227fc700 1 -- 172.21.15.36:0/1802857053 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb41c007170 con 0x7fb42c0b27c0 2022-06-17T12:50:52.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb4227fc700 1 -- 172.21.15.36:0/1802857053 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41c0072d0 con 0x7fb42c0b27c0 2022-06-17T12:50:52.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb4227fc700 1 -- 172.21.15.36:0/1802857053 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb41c004710 con 0x7fb42c0b27c0 2022-06-17T12:50:52.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb43200a700 1 -- 172.21.15.36:0/1802857053 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42c153250 con 0x7fb42c0b27c0 2022-06-17T12:50:52.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb43200a700 1 -- 172.21.15.36:0/1802857053 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb42c0a1540 con 0x7fb42c0b27c0 2022-06-17T12:50:52.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb43200a700 1 -- 172.21.15.36:0/1802857053 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb42c0ac4b0 con 0x7fb42c0b27c0 2022-06-17T12:50:52.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.509+0000 7fb4227fc700 1 -- 172.21.15.36:0/1802857053 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fb41c01b070 con 0x7fb42c0b27c0 2022-06-17T12:50:52.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.513+0000 7fb4227fc700 1 --2- 172.21.15.36:0/1802857053 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb414053940 0x7fb414055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:52.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.513+0000 7fb4227fc700 1 -- 172.21.15.36:0/1802857053 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb41c06b4a0 con 0x7fb42c0b27c0 2022-06-17T12:50:52.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.513+0000 7fb431008700 1 --2- 172.21.15.36:0/1802857053 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb414053940 0x7fb414055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:52.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.513+0000 7fb431008700 1 --2- 172.21.15.36:0/1802857053 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb414053940 0x7fb414055e00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7fb428018000 tx=0x7fb428014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:52.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.517+0000 7fb4227fc700 1 -- 172.21.15.36:0/1802857053 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb41c0422a0 con 0x7fb42c0b27c0 2022-06-17T12:50:52.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.645+0000 7fb43200a700 1 -- 172.21.15.36:0/1802857053 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb42c057410 con 0x7fb42c0b27c0 2022-06-17T12:50:52.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.649+0000 7fb4227fc700 1 -- 172.21.15.36:0/1802857053 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16273 (secure 0 0 0) 0x7fb41c0458c0 con 0x7fb42c0b27c0 2022-06-17T12:50:52.655 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:50:52.670 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":56,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:50:51.108569+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:50:46.705901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-06-17T12:50:47.812736+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:50:52.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.649+0000 7fb43200a700 1 -- 172.21.15.36:0/1802857053 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb414053940 msgr2=0x7fb414055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:52.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.649+0000 7fb43200a700 1 --2- 172.21.15.36:0/1802857053 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb414053940 0x7fb414055e00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7fb428018000 tx=0x7fb428014000).stop 2022-06-17T12:50:52.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.649+0000 7fb43200a700 1 -- 172.21.15.36:0/1802857053 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb42c0b27c0 msgr2=0x7fb42c1536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:52.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.649+0000 7fb43200a700 1 --2- 172.21.15.36:0/1802857053 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb42c0b27c0 0x7fb42c1536f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fb42c152ed0 tx=0x7fb41c001da0).stop 2022-06-17T12:50:52.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.649+0000 7fb43200a700 1 -- 172.21.15.36:0/1802857053 shutdown_connections 2022-06-17T12:50:52.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.649+0000 7fb43200a700 1 --2- 172.21.15.36:0/1802857053 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb414053940 0x7fb414055e00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:52.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.649+0000 7fb43200a700 1 --2- 172.21.15.36:0/1802857053 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb42c0b2ba0 0x7fb42c0a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:52.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.653+0000 7fb43200a700 1 --2- 172.21.15.36:0/1802857053 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb42c0b27c0 0x7fb42c1536f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:52.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.653+0000 7fb43200a700 1 --2- 172.21.15.36:0/1802857053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb42c0b14d0 0x7fb42c0a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:52.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.653+0000 7fb43200a700 1 -- 172.21.15.36:0/1802857053 >> 172.21.15.36:0/1802857053 conn(0x7fb42c05f770 msgr2=0x7fb42c0b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:52.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.653+0000 7fb43200a700 1 -- 172.21.15.36:0/1802857053 shutdown_connections 2022-06-17T12:50:52.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.653+0000 7fb43200a700 1 -- 172.21.15.36:0/1802857053 wait complete. 2022-06-17T12:50:52.688 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:50:46.705901+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:50:47.812736+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:50:52.689 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:50:52.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.817+0000 7fce0ed0f700 1 Processor -- start 2022-06-17T12:50:52.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.817+0000 7fce0ed0f700 1 -- start start 2022-06-17T12:50:52.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.817+0000 7fce0ed0f700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fce080b2df0 0x7fce080603b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:52.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.817+0000 7fce0ed0f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce08141690 0x7fce08145040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:52.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.817+0000 7fce0ed0f700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce08145530 con 0x7fce0805faa0 2022-06-17T12:50:52.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.817+0000 7fce0ed0f700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce080608a0 con 0x7fce080b2df0 2022-06-17T12:50:52.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.817+0000 7fce0ed0f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce081446a0 con 0x7fce08141690 2022-06-17T12:50:52.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.817+0000 7fce07fff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fce0805faa0 0x7fce08140f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51758/0 (socket says 172.21.15.36:51758) 2022-06-17T12:50:52.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.817+0000 7fce07fff700 1 -- 172.21.15.36:0/1159202052 learned_addr learned my addr 172.21.15.36:0/1159202052 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:50:52.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.817+0000 7fce0caab700 1 --2- 172.21.15.36:0/1159202052 >> v2:172.21.15.70:3300/0 conn(0x7fce080b2df0 0x7fce080603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:52.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.817+0000 7fce0d2ac700 1 --2- 172.21.15.36:0/1159202052 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce08141690 0x7fce08145040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:52.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.817+0000 7fce077fe700 1 -- 172.21.15.36:0/1159202052 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3335037289 0 0) 0x7fcdf4001110 con 0x7fce0805faa0 2022-06-17T12:50:52.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.817+0000 7fce077fe700 1 -- 172.21.15.36:0/1159202052 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdec003260 con 0x7fce0805faa0 2022-06-17T12:50:52.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.817+0000 7fce0caab700 1 -- 172.21.15.36:0/1159202052 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce08141690 msgr2=0x7fce08145040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:52.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.817+0000 7fce0caab700 1 --2- 172.21.15.36:0/1159202052 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce08141690 0x7fce08145040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:52.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.817+0000 7fce0caab700 1 -- 172.21.15.36:0/1159202052 >> v1:172.21.15.36:6789/0 conn(0x7fce0805faa0 legacy=0x7fce08140f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:52.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.817+0000 7fce0caab700 1 -- 172.21.15.36:0/1159202052 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce080b31d0 con 0x7fce080b2df0 2022-06-17T12:50:52.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.817+0000 7fce0caab700 1 --2- 172.21.15.36:0/1159202052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce080b2df0 0x7fce080603b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fcdfc002f70 tx=0x7fcdfc006df0).ready entity=mon.1 client_cookie=cf81e7b3752139ba server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:52.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.821+0000 7fce077fe700 1 -- 172.21.15.36:0/1159202052 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcdfc00d4a0 con 0x7fce080b2df0 2022-06-17T12:50:52.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.821+0000 7fce077fe700 1 -- 172.21.15.36:0/1159202052 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc000e90 con 0x7fce080b2df0 2022-06-17T12:50:52.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.821+0000 7fce077fe700 1 -- 172.21.15.36:0/1159202052 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcdfc00d7f0 con 0x7fce080b2df0 2022-06-17T12:50:52.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.821+0000 7fce0ed0f700 1 -- 172.21.15.36:0/1159202052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce080b2df0 msgr2=0x7fce080603b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:52.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.821+0000 7fce0ed0f700 1 --2- 172.21.15.36:0/1159202052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce080b2df0 0x7fce080603b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fcdfc002f70 tx=0x7fcdfc006df0).stop 2022-06-17T12:50:52.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.821+0000 7fce0ed0f700 1 -- 172.21.15.36:0/1159202052 shutdown_connections 2022-06-17T12:50:52.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.821+0000 7fce0ed0f700 1 --2- 172.21.15.36:0/1159202052 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce08141690 0x7fce08145040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:52.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.821+0000 7fce0ed0f700 1 --2- 172.21.15.36:0/1159202052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce080b2df0 0x7fce080603b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:52.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.821+0000 7fce0ed0f700 1 -- 172.21.15.36:0/1159202052 >> 172.21.15.36:0/1159202052 conn(0x7fce08063020 msgr2=0x7fce08063430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:52.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.821+0000 7fce0ed0f700 1 -- 172.21.15.36:0/1159202052 shutdown_connections 2022-06-17T12:50:52.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.821+0000 7fce0ed0f700 1 -- 172.21.15.36:0/1159202052 wait complete. 2022-06-17T12:50:52.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.821+0000 7fce0ed0f700 1 Processor -- start 2022-06-17T12:50:52.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.821+0000 7fce0ed0f700 1 -- start start 2022-06-17T12:50:52.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce0ed0f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce0805faa0 0x7fce08145f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:52.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce0ed0f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce080b2df0 0x7fce081501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:52.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce0ed0f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce08141690 0x7fce08066880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:52.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce0ed0f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0813f9e0 con 0x7fce080b2df0 2022-06-17T12:50:52.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce0ed0f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce080663e0 con 0x7fce08141690 2022-06-17T12:50:52.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce0ed0f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce08067ca0 con 0x7fce0805faa0 2022-06-17T12:50:52.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce07fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce080b2df0 0x7fce081501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:52.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce07fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce080b2df0 0x7fce081501c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34234/0 (socket says 172.21.15.36:34234) 2022-06-17T12:50:52.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce07fff700 1 -- 172.21.15.36:0/3032524260 learned_addr learned my addr 172.21.15.36:0/3032524260 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:52.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce0caab700 1 --2- 172.21.15.36:0/3032524260 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce0805faa0 0x7fce08145f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:52.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce0d2ac700 1 --2- 172.21.15.36:0/3032524260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce08141690 0x7fce08066880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:52.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce07fff700 1 -- 172.21.15.36:0/3032524260 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce0805faa0 msgr2=0x7fce08145f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:52.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce07fff700 1 --2- 172.21.15.36:0/3032524260 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce0805faa0 0x7fce08145f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:52.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce07fff700 1 -- 172.21.15.36:0/3032524260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce08141690 msgr2=0x7fce08066880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:52.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce07fff700 1 --2- 172.21.15.36:0/3032524260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce08141690 0x7fce08066880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:52.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce07fff700 1 -- 172.21.15.36:0/3032524260 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdfc00b180 con 0x7fce080b2df0 2022-06-17T12:50:52.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce0d2ac700 1 --2- 172.21.15.36:0/3032524260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce08141690 0x7fce08066880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:50:52.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce07fff700 1 --2- 172.21.15.36:0/3032524260 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce080b2df0 0x7fce081501c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7fcdf40012a0 tx=0x7fcdf4007900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:52.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce05ffb700 1 -- 172.21.15.36:0/3032524260 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcdf400ce30 con 0x7fce080b2df0 2022-06-17T12:50:52.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce05ffb700 1 -- 172.21.15.36:0/3032524260 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf40072e0 con 0x7fce080b2df0 2022-06-17T12:50:52.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce0ed0f700 1 -- 172.21.15.36:0/3032524260 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce081506b0 con 0x7fce080b2df0 2022-06-17T12:50:52.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce05ffb700 1 -- 172.21.15.36:0/3032524260 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcdf4005610 con 0x7fce080b2df0 2022-06-17T12:50:52.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce0ed0f700 1 -- 172.21.15.36:0/3032524260 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce0813f6e0 con 0x7fce080b2df0 2022-06-17T12:50:52.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce05ffb700 1 -- 172.21.15.36:0/3032524260 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fcdf401a070 con 0x7fce080b2df0 2022-06-17T12:50:52.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce0ed0f700 1 -- 172.21.15.36:0/3032524260 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce0813fbe0 con 0x7fce080b2df0 2022-06-17T12:50:52.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce05ffb700 1 --2- 172.21.15.36:0/3032524260 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcde8053a40 0x7fcde8055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:52.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce05ffb700 1 -- 172.21.15.36:0/3032524260 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcdf406af80 con 0x7fce080b2df0 2022-06-17T12:50:52.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.825+0000 7fce0caab700 1 --2- 172.21.15.36:0/3032524260 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcde8053a40 0x7fcde8055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:52.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.829+0000 7fce0caab700 1 --2- 172.21.15.36:0/3032524260 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcde8053a40 0x7fcde8055f00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fcdfc018000 tx=0x7fcdfc014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:52.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.829+0000 7fce05ffb700 1 -- 172.21.15.36:0/3032524260 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcdf4041d00 con 0x7fce080b2df0 2022-06-17T12:50:52.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.957+0000 7fce0ed0f700 1 -- 172.21.15.36:0/3032524260 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fce08145530 con 0x7fce080b2df0 2022-06-17T12:50:52.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.957+0000 7fce05ffb700 1 -- 172.21.15.36:0/3032524260 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v56) v1 ==== 110+0+10 (secure 0 0 0) 0x7fce08145530 con 0x7fce080b2df0 2022-06-17T12:50:52.944 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:50:52.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.957+0000 7fce0ed0f700 1 -- 172.21.15.36:0/3032524260 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcde8053a40 msgr2=0x7fcde8055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:52.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.957+0000 7fce0ed0f700 1 --2- 172.21.15.36:0/3032524260 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcde8053a40 0x7fcde8055f00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fcdfc018000 tx=0x7fcdfc014000).stop 2022-06-17T12:50:52.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.957+0000 7fce0ed0f700 1 -- 172.21.15.36:0/3032524260 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce080b2df0 msgr2=0x7fce081501c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:52.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.957+0000 7fce0ed0f700 1 --2- 172.21.15.36:0/3032524260 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce080b2df0 0x7fce081501c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7fcdf40012a0 tx=0x7fcdf4007900).stop 2022-06-17T12:50:52.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.957+0000 7fce0ed0f700 1 -- 172.21.15.36:0/3032524260 shutdown_connections 2022-06-17T12:50:52.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.957+0000 7fce0ed0f700 1 --2- 172.21.15.36:0/3032524260 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcde8053a40 0x7fcde8055f00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:52.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.957+0000 7fce0ed0f700 1 --2- 172.21.15.36:0/3032524260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce08141690 0x7fce08066880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:52.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.957+0000 7fce0ed0f700 1 --2- 172.21.15.36:0/3032524260 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce080b2df0 0x7fce081501c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:52.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.957+0000 7fce0ed0f700 1 --2- 172.21.15.36:0/3032524260 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce0805faa0 0x7fce08145f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:52.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.957+0000 7fce0ed0f700 1 -- 172.21.15.36:0/3032524260 >> 172.21.15.36:0/3032524260 conn(0x7fce08063020 msgr2=0x7fce080b5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:52.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.957+0000 7fce0ed0f700 1 -- 172.21.15.36:0/3032524260 shutdown_connections 2022-06-17T12:50:52.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:52.957+0000 7fce0ed0f700 1 -- 172.21.15.36:0/3032524260 wait complete. 2022-06-17T12:50:52.956 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:50:53.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.085+0000 7f5494eae700 1 Processor -- start 2022-06-17T12:50:53.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.085+0000 7f5494eae700 1 -- start start 2022-06-17T12:50:53.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.089+0000 7f5494eae700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f54900b2c80 0x7f54900af540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:53.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.089+0000 7f5494eae700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5490060ea0 con 0x7f5490060ac0 2022-06-17T12:50:53.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.089+0000 7f5494eae700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54900aebb0 con 0x7f5490140970 2022-06-17T12:50:53.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.089+0000 7f5494eae700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54900b2210 con 0x7f54900b2c80 2022-06-17T12:50:53.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.089+0000 7f548f7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f5490060ac0 0x7f5490140260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51770/0 (socket says 172.21.15.36:51770) 2022-06-17T12:50:53.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.089+0000 7f548f7fe700 1 -- 172.21.15.36:0/4249464134 learned_addr learned my addr 172.21.15.36:0/4249464134 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:50:53.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.089+0000 7f548ffff700 1 --2- 172.21.15.36:0/4249464134 >> v2:172.21.15.70:3300/0 conn(0x7f54900b2c80 0x7f54900af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:53.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.089+0000 7f548effd700 1 -- 172.21.15.36:0/4249464134 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4115839485 0 0) 0x7f5490060ea0 con 0x7f5490060ac0 2022-06-17T12:50:53.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.089+0000 7f548effd700 1 -- 172.21.15.36:0/4249464134 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5474003260 con 0x7f5490060ac0 2022-06-17T12:50:53.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.089+0000 7f548ffff700 1 -- 172.21.15.36:0/4249464134 >> v1:172.21.15.70:6789/0 conn(0x7f5490140970 legacy=0x7f54900aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:53.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.089+0000 7f548ffff700 1 -- 172.21.15.36:0/4249464134 >> v1:172.21.15.36:6789/0 conn(0x7f5490060ac0 legacy=0x7f5490140260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:53.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.089+0000 7f548ffff700 1 -- 172.21.15.36:0/4249464134 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54900b1ad0 con 0x7f54900b2c80 2022-06-17T12:50:53.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.089+0000 7f548ffff700 1 --2- 172.21.15.36:0/4249464134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54900b2c80 0x7f54900af540 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f54880032f0 tx=0x7f54880091d0).ready entity=mon.1 client_cookie=b99a8bd8bc3f434d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:53.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.089+0000 7f548effd700 1 -- 172.21.15.36:0/4249464134 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f548800b6c0 con 0x7f54900b2c80 2022-06-17T12:50:53.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.089+0000 7f548effd700 1 -- 172.21.15.36:0/4249464134 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548800b820 con 0x7f54900b2c80 2022-06-17T12:50:53.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.089+0000 7f548effd700 1 -- 172.21.15.36:0/4249464134 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f548800b990 con 0x7f54900b2c80 2022-06-17T12:50:53.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.093+0000 7f5494eae700 1 -- 172.21.15.36:0/4249464134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54900b2c80 msgr2=0x7f54900af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:53.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.093+0000 7f5494eae700 1 --2- 172.21.15.36:0/4249464134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54900b2c80 0x7f54900af540 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f54880032f0 tx=0x7f54880091d0).stop 2022-06-17T12:50:53.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.093+0000 7f5494eae700 1 -- 172.21.15.36:0/4249464134 shutdown_connections 2022-06-17T12:50:53.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.093+0000 7f5494eae700 1 --2- 172.21.15.36:0/4249464134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54900b2c80 0x7f54900af540 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.093+0000 7f5494eae700 1 -- 172.21.15.36:0/4249464134 >> 172.21.15.36:0/4249464134 conn(0x7f54900b64f0 msgr2=0x7f54900b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:53.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.093+0000 7f5494eae700 1 -- 172.21.15.36:0/4249464134 shutdown_connections 2022-06-17T12:50:53.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.093+0000 7f5494eae700 1 -- 172.21.15.36:0/4249464134 wait complete. 2022-06-17T12:50:53.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.093+0000 7f5494eae700 1 Processor -- start 2022-06-17T12:50:53.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.093+0000 7f5494eae700 1 -- start start 2022-06-17T12:50:53.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.097+0000 7f5494eae700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5490060ac0 0x7f5490144c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:53.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.097+0000 7f5494eae700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54900b2c80 0x7f549013ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:53.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.097+0000 7f5494eae700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5490140970 0x7f549013f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:53.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.097+0000 7f5494eae700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549014a700 con 0x7f5490060ac0 2022-06-17T12:50:53.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.097+0000 7f5494eae700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549013cbd0 con 0x7f54900b2c80 2022-06-17T12:50:53.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.097+0000 7f5494eae700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549013ce50 con 0x7f5490140970 2022-06-17T12:50:53.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.097+0000 7f548f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5490060ac0 0x7f5490144c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:53.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.097+0000 7f548f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5490060ac0 0x7f5490144c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34246/0 (socket says 172.21.15.36:34246) 2022-06-17T12:50:53.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.097+0000 7f548f7fe700 1 -- 172.21.15.36:0/177092838 learned_addr learned my addr 172.21.15.36:0/177092838 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:53.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.097+0000 7f5487fff700 1 --2- 172.21.15.36:0/177092838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54900b2c80 0x7f549013ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:53.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.097+0000 7f548ffff700 1 --2- 172.21.15.36:0/177092838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5490140970 0x7f549013f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:53.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.097+0000 7f5487fff700 1 -- 172.21.15.36:0/177092838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5490140970 msgr2=0x7f549013f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:53.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.097+0000 7f5487fff700 1 --2- 172.21.15.36:0/177092838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5490140970 0x7f549013f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.097+0000 7f5487fff700 1 -- 172.21.15.36:0/177092838 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5490060ac0 msgr2=0x7f5490144c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:53.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.097+0000 7f5487fff700 1 --2- 172.21.15.36:0/177092838 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5490060ac0 0x7f5490144c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.097+0000 7f5487fff700 1 -- 172.21.15.36:0/177092838 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5488009470 con 0x7f54900b2c80 2022-06-17T12:50:53.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.097+0000 7f548f7fe700 1 --2- 172.21.15.36:0/177092838 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5490060ac0 0x7f5490144c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:50:53.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.097+0000 7f548ffff700 1 --2- 172.21.15.36:0/177092838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5490140970 0x7f549013f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:50:53.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.097+0000 7f5487fff700 1 --2- 172.21.15.36:0/177092838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54900b2c80 0x7f549013ef10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f549005c8e0 tx=0x7f5478007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:53.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.097+0000 7f548d7fa700 1 -- 172.21.15.36:0/177092838 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f547800ccd0 con 0x7f54900b2c80 2022-06-17T12:50:53.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.097+0000 7f548d7fa700 1 -- 172.21.15.36:0/177092838 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547800ce30 con 0x7f54900b2c80 2022-06-17T12:50:53.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.097+0000 7f5494eae700 1 -- 172.21.15.36:0/177092838 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549013d0f0 con 0x7f54900b2c80 2022-06-17T12:50:53.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.097+0000 7f548d7fa700 1 -- 172.21.15.36:0/177092838 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54780125f0 con 0x7f54900b2c80 2022-06-17T12:50:53.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.097+0000 7f5494eae700 1 -- 172.21.15.36:0/177092838 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54901465e0 con 0x7f54900b2c80 2022-06-17T12:50:53.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.101+0000 7f5494eae700 1 -- 172.21.15.36:0/177092838 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54900b1d70 con 0x7f54900b2c80 2022-06-17T12:50:53.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.101+0000 7f548d7fa700 1 -- 172.21.15.36:0/177092838 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f5478019030 con 0x7f54900b2c80 2022-06-17T12:50:53.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.101+0000 7f548d7fa700 1 --2- 172.21.15.36:0/177092838 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5468053af0 0x7f5468055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:53.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.101+0000 7f548d7fa700 1 -- 172.21.15.36:0/177092838 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f547806b260 con 0x7f54900b2c80 2022-06-17T12:50:53.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.101+0000 7f548f7fe700 1 --2- 172.21.15.36:0/177092838 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5468053af0 0x7f5468055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:53.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.101+0000 7f548f7fe700 1 --2- 172.21.15.36:0/177092838 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5468053af0 0x7f5468055fb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f548000d000 tx=0x7f5480009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:53.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.109+0000 7f548d7fa700 1 -- 172.21.15.36:0/177092838 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5478041f50 con 0x7f54900b2c80 2022-06-17T12:50:53.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.237+0000 7f5494eae700 1 -- 172.21.15.36:0/177092838 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5490057410 con 0x7f54900b2c80 2022-06-17T12:50:53.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.241+0000 7f548d7fa700 1 -- 172.21.15.36:0/177092838 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v56) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5478045570 con 0x7f54900b2c80 2022-06-17T12:50:53.226 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:50:53.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.241+0000 7f5494eae700 1 -- 172.21.15.36:0/177092838 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5468053af0 msgr2=0x7f5468055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:53.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.241+0000 7f5494eae700 1 --2- 172.21.15.36:0/177092838 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5468053af0 0x7f5468055fb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f548000d000 tx=0x7f5480009040).stop 2022-06-17T12:50:53.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.241+0000 7f5494eae700 1 -- 172.21.15.36:0/177092838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54900b2c80 msgr2=0x7f549013ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:53.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.241+0000 7f5494eae700 1 --2- 172.21.15.36:0/177092838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54900b2c80 0x7f549013ef10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f549005c8e0 tx=0x7f5478007b90).stop 2022-06-17T12:50:53.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.241+0000 7f5494eae700 1 -- 172.21.15.36:0/177092838 shutdown_connections 2022-06-17T12:50:53.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.241+0000 7f5494eae700 1 --2- 172.21.15.36:0/177092838 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5468053af0 0x7f5468055fb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.241+0000 7f5494eae700 1 --2- 172.21.15.36:0/177092838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5490140970 0x7f549013f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.241+0000 7f5494eae700 1 --2- 172.21.15.36:0/177092838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54900b2c80 0x7f549013ef10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.241+0000 7f5494eae700 1 --2- 172.21.15.36:0/177092838 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5490060ac0 0x7f5490144c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.241+0000 7f5494eae700 1 -- 172.21.15.36:0/177092838 >> 172.21.15.36:0/177092838 conn(0x7f54900b64f0 msgr2=0x7f54900b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:53.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.241+0000 7f5494eae700 1 -- 172.21.15.36:0/177092838 shutdown_connections 2022-06-17T12:50:53.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.241+0000 7f5494eae700 1 -- 172.21.15.36:0/177092838 wait complete. 2022-06-17T12:50:53.246 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:50:53.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.381+0000 7f9b96ed2700 1 Processor -- start 2022-06-17T12:50:53.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.381+0000 7f9b96ed2700 1 -- start start 2022-06-17T12:50:53.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.385+0000 7f9b96ed2700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9b900b1680 0x7f9b900630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:53.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.385+0000 7f9b96ed2700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b90062c80 con 0x7f9b900ae6c0 2022-06-17T12:50:53.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.385+0000 7f9b96ed2700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b90060ea0 con 0x7f9b90060ac0 2022-06-17T12:50:53.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.385+0000 7f9b96ed2700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b900b2f50 con 0x7f9b900b1680 2022-06-17T12:50:53.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.385+0000 7f9b966d1700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f9b900ae6c0 0x7f9b9013fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51788/0 (socket says 172.21.15.36:51788) 2022-06-17T12:50:53.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.385+0000 7f9b966d1700 1 -- 172.21.15.36:0/776757526 learned_addr learned my addr 172.21.15.36:0/776757526 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:50:53.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.385+0000 7f9b956cf700 1 --2- 172.21.15.36:0/776757526 >> v2:172.21.15.70:3300/0 conn(0x7f9b900b1680 0x7f9b900630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:53.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.385+0000 7f9b94ece700 1 -- 172.21.15.36:0/776757526 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2580260526 0 0) 0x7f9b90062c80 con 0x7f9b900ae6c0 2022-06-17T12:50:53.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.385+0000 7f9b94ece700 1 -- 172.21.15.36:0/776757526 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b78003260 con 0x7f9b900ae6c0 2022-06-17T12:50:53.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.385+0000 7f9b956cf700 1 -- 172.21.15.36:0/776757526 >> v1:172.21.15.70:6789/0 conn(0x7f9b90060ac0 legacy=0x7f9b901431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:53.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.385+0000 7f9b956cf700 1 -- 172.21.15.36:0/776757526 >> v1:172.21.15.36:6789/0 conn(0x7f9b900ae6c0 legacy=0x7f9b9013fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:53.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.385+0000 7f9b94ece700 1 -- 172.21.15.36:0/776757526 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 818169957 0 0) 0x7f9b78003260 con 0x7f9b900ae6c0 2022-06-17T12:50:53.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.385+0000 7f9b956cf700 1 -- 172.21.15.36:0/776757526 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b900635d0 con 0x7f9b900b1680 2022-06-17T12:50:53.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.385+0000 7f9b956cf700 1 --2- 172.21.15.36:0/776757526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b900b1680 0x7f9b900630e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f9b88001ba0 tx=0x7f9b84006df0).ready entity=mon.1 client_cookie=16e7a184269bd6f3 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:53.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.385+0000 7f9b94ece700 1 -- 172.21.15.36:0/776757526 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b88001a00 con 0x7f9b900b1680 2022-06-17T12:50:53.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.385+0000 7f9b94ece700 1 -- 172.21.15.36:0/776757526 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b84000e90 con 0x7f9b900b1680 2022-06-17T12:50:53.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.385+0000 7f9b94ece700 1 -- 172.21.15.36:0/776757526 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b8400d4f0 con 0x7f9b900b1680 2022-06-17T12:50:53.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.389+0000 7f9b96ed2700 1 -- 172.21.15.36:0/776757526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b900b1680 msgr2=0x7f9b900630e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:53.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.389+0000 7f9b96ed2700 1 --2- 172.21.15.36:0/776757526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b900b1680 0x7f9b900630e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f9b88001ba0 tx=0x7f9b84006df0).stop 2022-06-17T12:50:53.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.389+0000 7f9b96ed2700 1 -- 172.21.15.36:0/776757526 shutdown_connections 2022-06-17T12:50:53.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.389+0000 7f9b96ed2700 1 --2- 172.21.15.36:0/776757526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b900b1680 0x7f9b900630e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.389+0000 7f9b96ed2700 1 -- 172.21.15.36:0/776757526 >> 172.21.15.36:0/776757526 conn(0x7f9b900b64f0 msgr2=0x7f9b900b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:53.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.389+0000 7f9b96ed2700 1 -- 172.21.15.36:0/776757526 shutdown_connections 2022-06-17T12:50:53.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.389+0000 7f9b96ed2700 1 -- 172.21.15.36:0/776757526 wait complete. 2022-06-17T12:50:53.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.389+0000 7f9b96ed2700 1 Processor -- start 2022-06-17T12:50:53.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.389+0000 7f9b96ed2700 1 -- start start 2022-06-17T12:50:53.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.393+0000 7f9b96ed2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b90060ac0 0x7f9b90148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:53.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.393+0000 7f9b96ed2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9b900ae6c0 0x7f9b90146060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:53.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.393+0000 7f9b96ed2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9b900b1680 0x7f9b901441e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:53.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.393+0000 7f9b96ed2700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9009fa40 con 0x7f9b900b1680 2022-06-17T12:50:53.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.393+0000 7f9b96ed2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b90146550 con 0x7f9b90060ac0 2022-06-17T12:50:53.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.393+0000 7f9b96ed2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b90145b20 con 0x7f9b900ae6c0 2022-06-17T12:50:53.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.393+0000 7f9b966d1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9b900b1680 0x7f9b901441e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:53.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.393+0000 7f9b966d1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9b900b1680 0x7f9b901441e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34264/0 (socket says 172.21.15.36:34264) 2022-06-17T12:50:53.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.393+0000 7f9b966d1700 1 -- 172.21.15.36:0/2578261858 learned_addr learned my addr 172.21.15.36:0/2578261858 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:53.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.393+0000 7f9b956cf700 1 --2- 172.21.15.36:0/2578261858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9b900ae6c0 0x7f9b90146060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:53.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.393+0000 7f9b95ed0700 1 --2- 172.21.15.36:0/2578261858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b90060ac0 0x7f9b90148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:53.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.393+0000 7f9b966d1700 1 -- 172.21.15.36:0/2578261858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9b900ae6c0 msgr2=0x7f9b90146060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:53.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.393+0000 7f9b966d1700 1 --2- 172.21.15.36:0/2578261858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9b900ae6c0 0x7f9b90146060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.393+0000 7f9b966d1700 1 -- 172.21.15.36:0/2578261858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b90060ac0 msgr2=0x7f9b90148950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:53.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.393+0000 7f9b966d1700 1 --2- 172.21.15.36:0/2578261858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b90060ac0 0x7f9b90148950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.393+0000 7f9b966d1700 1 -- 172.21.15.36:0/2578261858 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8400b180 con 0x7f9b900b1680 2022-06-17T12:50:53.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.393+0000 7f9b95ed0700 1 --2- 172.21.15.36:0/2578261858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b90060ac0 0x7f9b90148950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:50:53.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.393+0000 7f9b966d1700 1 --2- 172.21.15.36:0/2578261858 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9b900b1680 0x7f9b901441e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f9b88001c60 tx=0x7f9b880021c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:53.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.393+0000 7f9b82ffd700 1 -- 172.21.15.36:0/2578261858 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b88004ab0 con 0x7f9b900b1680 2022-06-17T12:50:53.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.393+0000 7f9b82ffd700 1 -- 172.21.15.36:0/2578261858 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b88004c10 con 0x7f9b900b1680 2022-06-17T12:50:53.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.393+0000 7f9b96ed2700 1 -- 172.21.15.36:0/2578261858 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b90145dc0 con 0x7f9b900b1680 2022-06-17T12:50:53.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.393+0000 7f9b82ffd700 1 -- 172.21.15.36:0/2578261858 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b88007170 con 0x7f9b900b1680 2022-06-17T12:50:53.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.393+0000 7f9b96ed2700 1 -- 172.21.15.36:0/2578261858 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b90148130 con 0x7f9b900b1680 2022-06-17T12:50:53.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.397+0000 7f9b82ffd700 1 -- 172.21.15.36:0/2578261858 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f9b8801b060 con 0x7f9b900b1680 2022-06-17T12:50:53.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.397+0000 7f9b96ed2700 1 -- 172.21.15.36:0/2578261858 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b9009f870 con 0x7f9b900b1680 2022-06-17T12:50:53.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.397+0000 7f9b82ffd700 1 --2- 172.21.15.36:0/2578261858 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9b70053ad0 0x7f9b70055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:53.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.397+0000 7f9b82ffd700 1 -- 172.21.15.36:0/2578261858 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9b8806bda0 con 0x7f9b900b1680 2022-06-17T12:50:53.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.397+0000 7f9b95ed0700 1 --2- 172.21.15.36:0/2578261858 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9b70053ad0 0x7f9b70055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:53.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.397+0000 7f9b95ed0700 1 --2- 172.21.15.36:0/2578261858 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9b70053ad0 0x7f9b70055f90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f9b8c00b000 tx=0x7f9b8c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:53.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.401+0000 7f9b82ffd700 1 -- 172.21.15.36:0/2578261858 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9b88042a90 con 0x7f9b900b1680 2022-06-17T12:50:53.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.529+0000 7f9b96ed2700 1 -- 172.21.15.36:0/2578261858 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9b901438b0 con 0x7f9b900b1680 2022-06-17T12:50:53.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.529+0000 7f9b82ffd700 1 -- 172.21.15.36:0/2578261858 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+10 (secure 0 0 0) 0x7f9b88019070 con 0x7f9b900b1680 2022-06-17T12:50:53.516 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:50:53.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.533+0000 7f9b96ed2700 1 -- 172.21.15.36:0/2578261858 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9b70053ad0 msgr2=0x7f9b70055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:53.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.533+0000 7f9b96ed2700 1 --2- 172.21.15.36:0/2578261858 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9b70053ad0 0x7f9b70055f90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f9b8c00b000 tx=0x7f9b8c009040).stop 2022-06-17T12:50:53.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.533+0000 7f9b96ed2700 1 -- 172.21.15.36:0/2578261858 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9b900b1680 msgr2=0x7f9b901441e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:53.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.533+0000 7f9b96ed2700 1 --2- 172.21.15.36:0/2578261858 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9b900b1680 0x7f9b901441e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f9b88001c60 tx=0x7f9b880021c0).stop 2022-06-17T12:50:53.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.533+0000 7f9b96ed2700 1 -- 172.21.15.36:0/2578261858 shutdown_connections 2022-06-17T12:50:53.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.533+0000 7f9b96ed2700 1 --2- 172.21.15.36:0/2578261858 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9b70053ad0 0x7f9b70055f90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.533+0000 7f9b96ed2700 1 --2- 172.21.15.36:0/2578261858 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9b900b1680 0x7f9b901441e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.533+0000 7f9b96ed2700 1 --2- 172.21.15.36:0/2578261858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9b900ae6c0 0x7f9b90146060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.533+0000 7f9b96ed2700 1 --2- 172.21.15.36:0/2578261858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b90060ac0 0x7f9b90148950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.533+0000 7f9b96ed2700 1 -- 172.21.15.36:0/2578261858 >> 172.21.15.36:0/2578261858 conn(0x7f9b900b64f0 msgr2=0x7f9b9013f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:53.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.533+0000 7f9b96ed2700 1 -- 172.21.15.36:0/2578261858 shutdown_connections 2022-06-17T12:50:53.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.533+0000 7f9b96ed2700 1 -- 172.21.15.36:0/2578261858 wait complete. 2022-06-17T12:50:53.535 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:50:53.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.681+0000 7fdf5d418700 1 Processor -- start 2022-06-17T12:50:53.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.681+0000 7fdf5d418700 1 -- start start 2022-06-17T12:50:53.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.681+0000 7fdf5d418700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf580b9860 0x7fdf58143840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:53.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.681+0000 7fdf5d418700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fdf580b5dc0 0x7fdf580b6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:53.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.681+0000 7fdf5d418700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf580b4160 con 0x7fdf580b45c0 2022-06-17T12:50:53.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.681+0000 7fdf5d418700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf580b9c40 con 0x7fdf580b5dc0 2022-06-17T12:50:53.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.681+0000 7fdf5d418700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf580627d0 con 0x7fdf580b9860 2022-06-17T12:50:53.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.681+0000 7fdf5cc17700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fdf580b5dc0 0x7fdf580b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:53.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.681+0000 7fdf5cc17700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fdf580b5dc0 0x7fdf580b6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34272/0 (socket says 172.21.15.36:34272) 2022-06-17T12:50:53.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.685+0000 7fdf5cc17700 1 -- 172.21.15.36:0/230626173 learned_addr learned my addr 172.21.15.36:0/230626173 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:53.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.685+0000 7fdf57fff700 1 --2- 172.21.15.36:0/230626173 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf580b9860 0x7fdf58143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:53.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.685+0000 7fdf5cc17700 1 -- 172.21.15.36:0/230626173 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf580b9860 msgr2=0x7fdf58143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:53.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.685+0000 7fdf5cc17700 1 --2- 172.21.15.36:0/230626173 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf580b9860 0x7fdf58143840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.685+0000 7fdf5cc17700 1 -- 172.21.15.36:0/230626173 >> v1:172.21.15.70:6789/0 conn(0x7fdf580b45c0 legacy=0x7fdf580b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:53.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.685+0000 7fdf5cc17700 1 -- 172.21.15.36:0/230626173 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf58062a70 con 0x7fdf580b5dc0 2022-06-17T12:50:53.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.685+0000 7fdf5cc17700 1 --2- 172.21.15.36:0/230626173 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf580b5dc0 0x7fdf580b6210 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7fdf500032f0 tx=0x7fdf50009290).ready entity=mon.0 client_cookie=a83442135cee0f9 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:53.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.685+0000 7fdf56ffd700 1 -- 172.21.15.36:0/230626173 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf5000b8a0 con 0x7fdf580b5dc0 2022-06-17T12:50:53.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.685+0000 7fdf56ffd700 1 -- 172.21.15.36:0/230626173 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5000ba00 con 0x7fdf580b5dc0 2022-06-17T12:50:53.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.685+0000 7fdf56ffd700 1 -- 172.21.15.36:0/230626173 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf5000bb70 con 0x7fdf580b5dc0 2022-06-17T12:50:53.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.685+0000 7fdf5d418700 1 -- 172.21.15.36:0/230626173 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf580b5dc0 msgr2=0x7fdf580b6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:53.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.685+0000 7fdf5d418700 1 --2- 172.21.15.36:0/230626173 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf580b5dc0 0x7fdf580b6210 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7fdf500032f0 tx=0x7fdf50009290).stop 2022-06-17T12:50:53.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.685+0000 7fdf5d418700 1 -- 172.21.15.36:0/230626173 shutdown_connections 2022-06-17T12:50:53.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.685+0000 7fdf5d418700 1 --2- 172.21.15.36:0/230626173 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf580b5dc0 0x7fdf580b6210 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.685+0000 7fdf5d418700 1 --2- 172.21.15.36:0/230626173 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf580b9860 0x7fdf58143840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.685+0000 7fdf5d418700 1 -- 172.21.15.36:0/230626173 >> 172.21.15.36:0/230626173 conn(0x7fdf5805c550 msgr2=0x7fdf58060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:53.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.685+0000 7fdf5d418700 1 -- 172.21.15.36:0/230626173 shutdown_connections 2022-06-17T12:50:53.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.685+0000 7fdf5d418700 1 -- 172.21.15.36:0/230626173 wait complete. 2022-06-17T12:50:53.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.689+0000 7fdf5d418700 1 Processor -- start 2022-06-17T12:50:53.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.689+0000 7fdf5d418700 1 -- start start 2022-06-17T12:50:53.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.689+0000 7fdf5d418700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf580b45c0 0x7fdf5812fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:53.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.689+0000 7fdf5d418700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf580b5dc0 0x7fdf581434f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:53.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.689+0000 7fdf5d418700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf580b9860 0x7fdf58155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:53.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.689+0000 7fdf5d418700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf580afa20 con 0x7fdf580b45c0 2022-06-17T12:50:53.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.689+0000 7fdf5d418700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf581559e0 con 0x7fdf580b9860 2022-06-17T12:50:53.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.689+0000 7fdf5d418700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf580af4e0 con 0x7fdf580b5dc0 2022-06-17T12:50:53.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.689+0000 7fdf57fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf580b45c0 0x7fdf5812fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:53.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.689+0000 7fdf57fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf580b45c0 0x7fdf5812fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34274/0 (socket says 172.21.15.36:34274) 2022-06-17T12:50:53.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.689+0000 7fdf57fff700 1 -- 172.21.15.36:0/2577451229 learned_addr learned my addr 172.21.15.36:0/2577451229 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:53.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.689+0000 7fdf577fe700 1 --2- 172.21.15.36:0/2577451229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf580b5dc0 0x7fdf581434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:53.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.689+0000 7fdf5cc17700 1 --2- 172.21.15.36:0/2577451229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf580b9860 0x7fdf58155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:53.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.689+0000 7fdf57fff700 1 -- 172.21.15.36:0/2577451229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf580b5dc0 msgr2=0x7fdf581434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:53.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.689+0000 7fdf57fff700 1 --2- 172.21.15.36:0/2577451229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf580b5dc0 0x7fdf581434f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.689+0000 7fdf57fff700 1 -- 172.21.15.36:0/2577451229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf580b9860 msgr2=0x7fdf58155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:53.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.689+0000 7fdf57fff700 1 --2- 172.21.15.36:0/2577451229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf580b9860 0x7fdf58155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.689+0000 7fdf57fff700 1 -- 172.21.15.36:0/2577451229 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf50009530 con 0x7fdf580b45c0 2022-06-17T12:50:53.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.689+0000 7fdf57fff700 1 --2- 172.21.15.36:0/2577451229 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf580b45c0 0x7fdf5812fe20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7fdf480030b0 tx=0x7fdf48008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:53.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.693+0000 7fdf557fa700 1 -- 172.21.15.36:0/2577451229 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf48011780 con 0x7fdf580b45c0 2022-06-17T12:50:53.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.693+0000 7fdf557fa700 1 -- 172.21.15.36:0/2577451229 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf480118e0 con 0x7fdf580b45c0 2022-06-17T12:50:53.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.693+0000 7fdf5d418700 1 -- 172.21.15.36:0/2577451229 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf580af780 con 0x7fdf580b45c0 2022-06-17T12:50:53.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.693+0000 7fdf557fa700 1 -- 172.21.15.36:0/2577451229 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf48011c40 con 0x7fdf580b45c0 2022-06-17T12:50:53.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.693+0000 7fdf5d418700 1 -- 172.21.15.36:0/2577451229 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf580b0120 con 0x7fdf580b45c0 2022-06-17T12:50:53.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.693+0000 7fdf5d418700 1 -- 172.21.15.36:0/2577451229 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf58155ba0 con 0x7fdf580b45c0 2022-06-17T12:50:53.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.693+0000 7fdf557fa700 1 -- 172.21.15.36:0/2577451229 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fdf48012420 con 0x7fdf580b45c0 2022-06-17T12:50:53.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.693+0000 7fdf557fa700 1 --2- 172.21.15.36:0/2577451229 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdf3c053a40 0x7fdf3c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:53.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.693+0000 7fdf557fa700 1 -- 172.21.15.36:0/2577451229 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdf4806be30 con 0x7fdf580b45c0 2022-06-17T12:50:53.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.693+0000 7fdf577fe700 1 --2- 172.21.15.36:0/2577451229 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdf3c053a40 0x7fdf3c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:53.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.697+0000 7fdf577fe700 1 --2- 172.21.15.36:0/2577451229 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdf3c053a40 0x7fdf3c055f00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7fdf4000d000 tx=0x7fdf40009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:53.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.701+0000 7fdf557fa700 1 -- 172.21.15.36:0/2577451229 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdf58155ba0 con 0x7fdf580b45c0 2022-06-17T12:50:53.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.829+0000 7fdf5d418700 1 -- 172.21.15.36:0/2577451229 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdf58155ba0 con 0x7fdf580b45c0 2022-06-17T12:50:53.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.829+0000 7fdf557fa700 1 -- 172.21.15.36:0/2577451229 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+10 (secure 0 0 0) 0x7fdf58155ba0 con 0x7fdf580b45c0 2022-06-17T12:50:53.825 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:50:53.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.833+0000 7fdf5d418700 1 -- 172.21.15.36:0/2577451229 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdf3c053a40 msgr2=0x7fdf3c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:53.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.833+0000 7fdf5d418700 1 --2- 172.21.15.36:0/2577451229 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdf3c053a40 0x7fdf3c055f00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7fdf4000d000 tx=0x7fdf40009040).stop 2022-06-17T12:50:53.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.833+0000 7fdf5d418700 1 -- 172.21.15.36:0/2577451229 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf580b45c0 msgr2=0x7fdf5812fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:53.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.833+0000 7fdf5d418700 1 --2- 172.21.15.36:0/2577451229 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf580b45c0 0x7fdf5812fe20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7fdf480030b0 tx=0x7fdf48008040).stop 2022-06-17T12:50:53.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.833+0000 7fdf5d418700 1 -- 172.21.15.36:0/2577451229 shutdown_connections 2022-06-17T12:50:53.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.833+0000 7fdf5d418700 1 --2- 172.21.15.36:0/2577451229 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdf3c053a40 0x7fdf3c055f00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.833+0000 7fdf5d418700 1 --2- 172.21.15.36:0/2577451229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf580b9860 0x7fdf58155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.833+0000 7fdf5d418700 1 --2- 172.21.15.36:0/2577451229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf580b5dc0 0x7fdf581434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.833+0000 7fdf5d418700 1 --2- 172.21.15.36:0/2577451229 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf580b45c0 0x7fdf5812fe20 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.833+0000 7fdf5d418700 1 -- 172.21.15.36:0/2577451229 >> 172.21.15.36:0/2577451229 conn(0x7fdf5805c550 msgr2=0x7fdf580b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:53.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.833+0000 7fdf5d418700 1 -- 172.21.15.36:0/2577451229 shutdown_connections 2022-06-17T12:50:53.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.833+0000 7fdf5d418700 1 -- 172.21.15.36:0/2577451229 wait complete. 2022-06-17T12:50:53.831 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:50:53.831 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:50:53.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.953+0000 7f3033fff700 1 Processor -- start 2022-06-17T12:50:53.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.953+0000 7f3033fff700 1 -- start start 2022-06-17T12:50:53.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.953+0000 7f3033fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3034062470 0x7f30340b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:53.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.953+0000 7f3033fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3034143a90 0x7f3034144390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:53.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.953+0000 7f3033fff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30340b83f0 con 0x7f30340b4820 2022-06-17T12:50:53.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.953+0000 7f3033fff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3034142ec0 con 0x7f3034062470 2022-06-17T12:50:53.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.953+0000 7f3033fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3034143ed0 con 0x7f3034143a90 2022-06-17T12:50:53.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.957+0000 7f3032ffd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f30340b4820 0x7f30340b7ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51814/0 (socket says 172.21.15.36:51814) 2022-06-17T12:50:53.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.957+0000 7f3032ffd700 1 -- 172.21.15.36:0/661623764 learned_addr learned my addr 172.21.15.36:0/661623764 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:50:53.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.957+0000 7f30337fe700 1 --2- 172.21.15.36:0/661623764 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3034143a90 0x7f3034144390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:53.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.957+0000 7f30327fc700 1 --2- 172.21.15.36:0/661623764 >> v2:172.21.15.70:3300/0 conn(0x7f3034062470 0x7f30340b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:53.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.957+0000 7f3031ffb700 1 -- 172.21.15.36:0/661623764 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3522653460 0 0) 0x7f30340b83f0 con 0x7f30340b4820 2022-06-17T12:50:53.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.957+0000 7f3031ffb700 1 -- 172.21.15.36:0/661623764 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3014003260 con 0x7f30340b4820 2022-06-17T12:50:53.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.957+0000 7f30337fe700 1 -- 172.21.15.36:0/661623764 >> v2:172.21.15.70:3300/0 conn(0x7f3034062470 msgr2=0x7f30340b4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:50:53.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.957+0000 7f30337fe700 1 --2- 172.21.15.36:0/661623764 >> v2:172.21.15.70:3300/0 conn(0x7f3034062470 0x7f30340b4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.957+0000 7f30337fe700 1 -- 172.21.15.36:0/661623764 >> v1:172.21.15.36:6789/0 conn(0x7f30340b4820 legacy=0x7f30340b7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:53.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.957+0000 7f30337fe700 1 -- 172.21.15.36:0/661623764 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30340b1ff0 con 0x7f3034143a90 2022-06-17T12:50:53.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.957+0000 7f30337fe700 1 --2- 172.21.15.36:0/661623764 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3034143a90 0x7f3034144390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f302c0032f0 tx=0x7f302c0091d0).ready entity=mon.2 client_cookie=a6a5877d6030d942 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:53.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.957+0000 7f3031ffb700 1 -- 172.21.15.36:0/661623764 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f302c00b600 con 0x7f3034143a90 2022-06-17T12:50:53.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.957+0000 7f3031ffb700 1 -- 172.21.15.36:0/661623764 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302c00b760 con 0x7f3034143a90 2022-06-17T12:50:53.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.957+0000 7f3031ffb700 1 -- 172.21.15.36:0/661623764 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f302c00b8d0 con 0x7f3034143a90 2022-06-17T12:50:53.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.957+0000 7f3033fff700 1 -- 172.21.15.36:0/661623764 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3034143a90 msgr2=0x7f3034144390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:53.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.957+0000 7f3033fff700 1 --2- 172.21.15.36:0/661623764 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3034143a90 0x7f3034144390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f302c0032f0 tx=0x7f302c0091d0).stop 2022-06-17T12:50:53.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.957+0000 7f3033fff700 1 -- 172.21.15.36:0/661623764 shutdown_connections 2022-06-17T12:50:53.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.957+0000 7f3033fff700 1 --2- 172.21.15.36:0/661623764 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3034143a90 0x7f3034144390 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.957+0000 7f3033fff700 1 --2- 172.21.15.36:0/661623764 >> v2:172.21.15.70:3300/0 conn(0x7f3034062470 0x7f30340b4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.957+0000 7f3033fff700 1 -- 172.21.15.36:0/661623764 >> 172.21.15.36:0/661623764 conn(0x7f303405c550 msgr2=0x7f30340b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:53.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.957+0000 7f3033fff700 1 -- 172.21.15.36:0/661623764 shutdown_connections 2022-06-17T12:50:53.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.957+0000 7f3033fff700 1 -- 172.21.15.36:0/661623764 wait complete. 2022-06-17T12:50:53.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.957+0000 7f3033fff700 1 Processor -- start 2022-06-17T12:50:53.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.957+0000 7f3033fff700 1 -- start start 2022-06-17T12:50:53.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.961+0000 7f3033fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3034062470 0x7f3034144390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:53.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.961+0000 7f3033fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f30340b4820 0x7f3034145a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:53.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.961+0000 7f3033fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3034143a90 0x7f30340b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:53.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.961+0000 7f3033fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30340b7320 con 0x7f30340b4820 2022-06-17T12:50:53.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.961+0000 7f3033fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30341460a0 con 0x7f3034143a90 2022-06-17T12:50:53.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.961+0000 7f3033fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30341523d0 con 0x7f3034062470 2022-06-17T12:50:53.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.961+0000 7f30327fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f30340b4820 0x7f3034145a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:53.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.961+0000 7f30327fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f30340b4820 0x7f3034145a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34290/0 (socket says 172.21.15.36:34290) 2022-06-17T12:50:53.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.961+0000 7f30327fc700 1 -- 172.21.15.36:0/1173044557 learned_addr learned my addr 172.21.15.36:0/1173044557 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:50:53.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.961+0000 7f3032ffd700 1 --2- 172.21.15.36:0/1173044557 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3034062470 0x7f3034144390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:53.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.961+0000 7f30337fe700 1 --2- 172.21.15.36:0/1173044557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3034143a90 0x7f30340b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:53.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.961+0000 7f30327fc700 1 -- 172.21.15.36:0/1173044557 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3034062470 msgr2=0x7f3034144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:53.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.961+0000 7f30327fc700 1 --2- 172.21.15.36:0/1173044557 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3034062470 0x7f3034144390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.961+0000 7f30327fc700 1 -- 172.21.15.36:0/1173044557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3034143a90 msgr2=0x7f30340b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:53.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.961+0000 7f30327fc700 1 --2- 172.21.15.36:0/1173044557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3034143a90 0x7f30340b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:53.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.961+0000 7f30327fc700 1 -- 172.21.15.36:0/1173044557 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302c009470 con 0x7f30340b4820 2022-06-17T12:50:53.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.961+0000 7f30327fc700 1 --2- 172.21.15.36:0/1173044557 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f30340b4820 0x7f3034145a70 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f301c00c000 tx=0x7f301c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:53.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.961+0000 7f302bfff700 1 -- 172.21.15.36:0/1173044557 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f301c010780 con 0x7f30340b4820 2022-06-17T12:50:53.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.961+0000 7f302bfff700 1 -- 172.21.15.36:0/1173044557 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f301c0108e0 con 0x7f30340b4820 2022-06-17T12:50:53.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.961+0000 7f3033fff700 1 -- 172.21.15.36:0/1173044557 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3034146340 con 0x7f30340b4820 2022-06-17T12:50:53.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.961+0000 7f302bfff700 1 -- 172.21.15.36:0/1173044557 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f301c010c40 con 0x7f30340b4820 2022-06-17T12:50:53.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.961+0000 7f3033fff700 1 -- 172.21.15.36:0/1173044557 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30341471c0 con 0x7f30340b4820 2022-06-17T12:50:53.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.961+0000 7f3033fff700 1 -- 172.21.15.36:0/1173044557 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3034152590 con 0x7f30340b4820 2022-06-17T12:50:53.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.965+0000 7f302bfff700 1 -- 172.21.15.36:0/1173044557 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f301c01c460 con 0x7f30340b4820 2022-06-17T12:50:54.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.965+0000 7f302bfff700 1 --2- 172.21.15.36:0/1173044557 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3018053ad0 0x7f3018055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:50:54.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.965+0000 7f302bfff700 1 -- 172.21.15.36:0/1173044557 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f301c06c4a0 con 0x7f30340b4820 2022-06-17T12:50:54.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.965+0000 7f3032ffd700 1 --2- 172.21.15.36:0/1173044557 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3018053ad0 0x7f3018055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:50:54.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.965+0000 7f3032ffd700 1 --2- 172.21.15.36:0/1173044557 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3018053ad0 0x7f3018055f90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f302400d000 tx=0x7f3024009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:50:54.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:53.969+0000 7f302bfff700 1 -- 172.21.15.36:0/1173044557 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f301c043130 con 0x7f30340b4820 2022-06-17T12:50:54.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:54.133+0000 7f3033fff700 1 -- 172.21.15.36:0/1173044557 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30341430b0 con 0x7f30340b4820 2022-06-17T12:50:54.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:54.133+0000 7f302bfff700 1 -- 172.21.15.36:0/1173044557 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f301c046750 con 0x7f30340b4820 2022-06-17T12:50:54.119 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:50:54.120 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":25,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5670,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.36:6835/2598001220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2598001220},{"type":"v1","addr":"172.21.15.36:6835","nonce":2598001220}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":23}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:50:49.096455+0000","modified":"2022-06-17T12:50:50.099539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"b","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/2723288138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2723288138},{"type":"v1","addr":"172.21.15.70:6833","nonce":2723288138}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-06-17T12:50:54.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:54.137+0000 7f3033fff700 1 -- 172.21.15.36:0/1173044557 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3018053ad0 msgr2=0x7f3018055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:54.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:54.137+0000 7f3033fff700 1 --2- 172.21.15.36:0/1173044557 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3018053ad0 0x7f3018055f90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f302400d000 tx=0x7f3024009040).stop 2022-06-17T12:50:54.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:54.137+0000 7f3033fff700 1 -- 172.21.15.36:0/1173044557 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f30340b4820 msgr2=0x7f3034145a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:50:54.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:54.137+0000 7f3033fff700 1 --2- 172.21.15.36:0/1173044557 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f30340b4820 0x7f3034145a70 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f301c00c000 tx=0x7f301c006df0).stop 2022-06-17T12:50:54.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:54.137+0000 7f3033fff700 1 -- 172.21.15.36:0/1173044557 shutdown_connections 2022-06-17T12:50:54.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:54.137+0000 7f3033fff700 1 --2- 172.21.15.36:0/1173044557 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3018053ad0 0x7f3018055f90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:54.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:54.137+0000 7f3033fff700 1 --2- 172.21.15.36:0/1173044557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3034143a90 0x7f30340b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:54.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:54.137+0000 7f3033fff700 1 --2- 172.21.15.36:0/1173044557 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f30340b4820 0x7f3034145a70 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:54.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:54.137+0000 7f3033fff700 1 --2- 172.21.15.36:0/1173044557 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3034062470 0x7f3034144390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:50:54.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:54.137+0000 7f3033fff700 1 -- 172.21.15.36:0/1173044557 >> 172.21.15.36:0/1173044557 conn(0x7f303405c550 msgr2=0x7f30340b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:50:54.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:54.137+0000 7f3033fff700 1 -- 172.21.15.36:0/1173044557 shutdown_connections 2022-06-17T12:50:54.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:50:54.137+0000 7f3033fff700 1 -- 172.21.15.36:0/1173044557 wait complete. 2022-06-17T12:50:54.153 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 25 2022-06-17T12:50:54.354 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:50:49.096455+0000', 'modified': '2022-06-17T12:50:50.099539+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5679}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5679': {'gid': 5679, 'name': 'b', 'rank': 0, 'incarnation': 24, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6833/2723288138', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 2723288138}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 2723288138}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:50:54.355 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:50:54.355 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-17T12:50:54.475 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":5679,"want_state":"up:active","state":"up:active","rank_uptime":5.3829781739999998,"mdsmap_epoch":25,"osdmap_epoch":56,"osdmap_epoch_barrier":54,"uptime":10.683896528} 2022-06-17T12:50:54.476 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 5679, "mdsmap_epoch": 25, "osdmap_epoch": 56, "osdmap_epoch_barrier": 54, "rank_uptime": 5.382978174, "state": "up:active", "uptime": 10.683896528, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:50:54.477 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:50:54.478 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-17T12:50:54.479 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi107.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-17T12:50:54.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:50:54.481 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:50:54.534 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:50:54.603 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:50:54.604 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:50:54.648 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:50:54.650 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:50:54.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:50:54.651 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:50:54.712 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:50:54.713 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:50:54.739 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:50:54.773 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:50:54.774 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:50:54.793 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:2022-06-17T12:50:54.807+0000 7fee94410080 -1 init, newargv = 0x5625ad953500 newargc=9ceph-fuse[22096]: starting ceph client 2022-06-17T12:50:54.794 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr: 2022-06-17T12:50:54.806 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22096]: starting fuse 2022-06-17T12:50:54.811 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:50:54.812 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:50:54.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:50:54.813 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:50:54.878 INFO:teuthology.orchestra.run.smithi107.stdout:52 2022-06-17T12:50:54.878 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:50:54.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:50:54.880 DEBUG:teuthology.orchestra.run.smithi107:> sudo python3 -c ' 2022-06-17T12:50:54.881 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2022-06-17T12:50:54.881 DEBUG:teuthology.orchestra.run.smithi107:> import re 2022-06-17T12:50:54.882 DEBUG:teuthology.orchestra.run.smithi107:> import os 2022-06-17T12:50:54.882 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2022-06-17T12:50:54.883 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:50:54.883 DEBUG:teuthology.orchestra.run.smithi107:> def find_socket(client_name): 2022-06-17T12:50:54.884 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-17T12:50:54.884 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2022-06-17T12:50:54.885 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:50:54.885 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2022-06-17T12:50:54.895 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2022-06-17T12:50:54.901 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2022-06-17T12:50:54.902 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2022-06-17T12:50:54.902 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:50:54.903 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2022-06-17T12:50:54.903 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:50:54.904 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:50:54.904 DEBUG:teuthology.orchestra.run.smithi107:> return f 2022-06-17T12:50:54.905 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:50:54.905 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:50:54.905 DEBUG:teuthology.orchestra.run.smithi107:> print(find_socket("client.0")) 2022-06-17T12:50:54.908 DEBUG:teuthology.orchestra.run.smithi107:> ' 2022-06-17T12:50:54.958 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.0.22096.asok 2022-06-17T12:50:54.970 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22096.asok 2022-06-17T12:50:54.971 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:50:54.971 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22096.asok status 2022-06-17T12:50:55.084 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2022-06-17T12:50:55.088 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2022-06-17T12:50:55.088 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:50:55.089 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:50:55.089 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "0", 2022-06-17T12:50:55.089 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2022-06-17T12:50:55.092 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-17T12:50:55.092 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "22096", 2022-06-17T12:50:55.093 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/" 2022-06-17T12:50:55.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:50:55.093 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2022-06-17T12:50:55.094 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2022-06-17T12:50:55.094 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 5282, 2022-06-17T12:50:55.095 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2022-06-17T12:50:55.095 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2022-06-17T12:50:55.097 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2022-06-17T12:50:55.097 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 5282 2022-06-17T12:50:55.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:50:55.098 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:50:55.098 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:50:55.099 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:50:55.099 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1472453020 2022-06-17T12:50:55.100 INFO:teuthology.orchestra.run.smithi107.stdout: } 2022-06-17T12:50:55.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:50:55.112 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:50:55.113 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:50:55.113 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:50:55.113 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1472453020 2022-06-17T12:50:55.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:50:55.114 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.5282 172.21.15.107:0/1472453020", 2022-06-17T12:50:55.115 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "172.21.15.107:0/1472453020", 2022-06-17T12:50:55.115 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 1, 2022-06-17T12:50:55.115 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 25, 2022-06-17T12:50:55.116 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 56, 2022-06-17T12:50:55.166 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:50:55.167 INFO:teuthology.orchestra.run.smithi107.stdout: "blacklisted": false, 2022-06-17T12:50:55.169 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2022-06-17T12:50:55.170 INFO:teuthology.orchestra.run.smithi107.stdout:} 2022-06-17T12:50:55.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:50:55.173 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:50:55.174 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:50:55.213 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:50:55.214 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:50:55.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:50:55.216 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:50:55.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:50:55.291 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:50:55.292 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:50:55.376 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:50:55.376 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:50:55.381 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-06-17T12:50:55.382 DEBUG:teuthology.orchestra.run.smithi107:> import sys 2022-06-17T12:50:55.382 DEBUG:teuthology.orchestra.run.smithi107:> import time 2022-06-17T12:50:55.383 DEBUG:teuthology.orchestra.run.smithi107:> import os 2022-06-17T12:50:55.384 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:50:55.385 DEBUG:teuthology.orchestra.run.smithi107:> n = 1000 2022-06-17T12:50:55.386 DEBUG:teuthology.orchestra.run.smithi107:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2022-06-17T12:50:55.386 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:50:55.387 DEBUG:teuthology.orchestra.run.smithi107:> if not os.path.exists(os.path.dirname(abs_path)): 2022-06-17T12:50:55.389 DEBUG:teuthology.orchestra.run.smithi107:> os.makedirs(os.path.dirname(abs_path)) 2022-06-17T12:50:55.389 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:50:55.390 DEBUG:teuthology.orchestra.run.smithi107:> for i in range(0, n): 2022-06-17T12:50:55.390 DEBUG:teuthology.orchestra.run.smithi107:> fname = "{0}_{1}".format(abs_path, i) 2022-06-17T12:50:55.391 DEBUG:teuthology.orchestra.run.smithi107:> with open(fname, '"'"'w'"'"') as f: 2022-06-17T12:50:55.391 DEBUG:teuthology.orchestra.run.smithi107:> f.write('"'"'content'"'"') 2022-06-17T12:50:55.392 DEBUG:teuthology.orchestra.run.smithi107:> if True: 2022-06-17T12:50:55.392 DEBUG:teuthology.orchestra.run.smithi107:> f.flush() 2022-06-17T12:50:55.392 DEBUG:teuthology.orchestra.run.smithi107:> os.fsync(f.fileno()) 2022-06-17T12:50:55.393 DEBUG:teuthology.orchestra.run.smithi107:> ' 2022-06-17T12:51:06.288 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:51:06.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b2cf46700 1 Processor -- start 2022-06-17T12:51:06.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b2cf46700 1 -- start start 2022-06-17T12:51:06.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b2cf46700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2b28146d00 0x7f2b28147150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:06.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b2cf46700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b280b25e0 con 0x7f2b280b2cd0 2022-06-17T12:51:06.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b2cf46700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b280ad790 con 0x7f2b280b2200 2022-06-17T12:51:06.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b2cf46700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b280aef40 con 0x7f2b28146d00 2022-06-17T12:51:06.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b1effd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f2b280b2cd0 0x7f2b281465f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51828/0 (socket says 172.21.15.36:51828) 2022-06-17T12:51:06.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b1effd700 1 -- 172.21.15.36:0/2404825142 learned_addr learned my addr 172.21.15.36:0/2404825142 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:51:06.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b27fff700 1 --2- 172.21.15.36:0/2404825142 >> v2:172.21.15.70:3300/0 conn(0x7f2b28146d00 0x7f2b28147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:06.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b26ffd700 1 -- 172.21.15.36:0/2404825142 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3444424006 0 0) 0x7f2b280b25e0 con 0x7f2b280b2cd0 2022-06-17T12:51:06.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b26ffd700 1 -- 172.21.15.36:0/2404825142 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b08003260 con 0x7f2b280b2cd0 2022-06-17T12:51:06.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b26ffd700 1 -- 172.21.15.36:0/2404825142 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020680851 0 0) 0x7f2b280ad790 con 0x7f2b280b2200 2022-06-17T12:51:06.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b26ffd700 1 -- 172.21.15.36:0/2404825142 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b280b25e0 con 0x7f2b280b2200 2022-06-17T12:51:06.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b26ffd700 1 -- 172.21.15.36:0/2404825142 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 134917885 0 0) 0x7f2b08003260 con 0x7f2b280b2cd0 2022-06-17T12:51:06.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b26ffd700 1 -- 172.21.15.36:0/2404825142 >> v2:172.21.15.70:3300/0 conn(0x7f2b28146d00 msgr2=0x7f2b28147150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:06.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b26ffd700 1 --2- 172.21.15.36:0/2404825142 >> v2:172.21.15.70:3300/0 conn(0x7f2b28146d00 0x7f2b28147150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:06.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b26ffd700 1 -- 172.21.15.36:0/2404825142 >> v1:172.21.15.70:6789/0 conn(0x7f2b280b2200 legacy=0x7f2b280af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:06.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b26ffd700 1 -- 172.21.15.36:0/2404825142 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b280afaf0 con 0x7f2b280b2cd0 2022-06-17T12:51:06.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b26ffd700 1 -- 172.21.15.36:0/2404825142 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f2b10001660 con 0x7f2b280b2cd0 2022-06-17T12:51:06.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b27fff700 1 --2- 172.21.15.36:0/2404825142 >> v2:172.21.15.70:3300/0 conn(0x7f2b28146d00 0x7f2b28147150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:51:06.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b26ffd700 1 -- 172.21.15.36:0/2404825142 >> v1:172.21.15.36:6789/0 conn(0x7f2b280b2cd0 legacy=0x7f2b281465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:06.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b26ffd700 1 --2- 172.21.15.36:0/2404825142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b080038e0 0x7f2b08003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:06.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b26ffd700 1 --2- 172.21.15.36:0/2404825142 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b08004500 0x7f2b08004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:06.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b26ffd700 1 --2- 172.21.15.36:0/2404825142 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b08004e70 0x7f2b0800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:06.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b26ffd700 1 -- 172.21.15.36:0/2404825142 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b280aef40 con 0x7f2b08004e70 2022-06-17T12:51:06.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b26ffd700 1 -- 172.21.15.36:0/2404825142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0800bcf0 con 0x7f2b080038e0 2022-06-17T12:51:06.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b26ffd700 1 -- 172.21.15.36:0/2404825142 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0800beb0 con 0x7f2b08004500 2022-06-17T12:51:06.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b27fff700 1 --2- 172.21.15.36:0/2404825142 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b08004e70 0x7f2b0800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:06.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b1effd700 1 --2- 172.21.15.36:0/2404825142 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b08004500 0x7f2b08004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:06.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b277fe700 1 --2- 172.21.15.36:0/2404825142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b080038e0 0x7f2b08003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:06.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b27fff700 1 -- 172.21.15.36:0/2404825142 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b08004500 msgr2=0x7f2b08004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:06.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b27fff700 1 --2- 172.21.15.36:0/2404825142 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b08004500 0x7f2b08004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:06.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b27fff700 1 -- 172.21.15.36:0/2404825142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b080038e0 msgr2=0x7f2b08003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:06.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b27fff700 1 --2- 172.21.15.36:0/2404825142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b080038e0 0x7f2b08003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:06.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b27fff700 1 -- 172.21.15.36:0/2404825142 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b08003260 con 0x7f2b08004e70 2022-06-17T12:51:06.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.429+0000 7f2b277fe700 1 --2- 172.21.15.36:0/2404825142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b080038e0 0x7f2b08003ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:51:06.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.433+0000 7f2b27fff700 1 -- 172.21.15.36:0/2404825142 reap_dead start 2022-06-17T12:51:06.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.433+0000 7f2b27fff700 1 --2- 172.21.15.36:0/2404825142 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b08004e70 0x7f2b0800b750 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f2b2000c000 tx=0x7f2b200091d0).ready entity=mon.0 client_cookie=9fdbfa0dc68de191 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:06.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.433+0000 7f2b26ffd700 1 -- 172.21.15.36:0/2404825142 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b2000e8f0 con 0x7f2b08004e70 2022-06-17T12:51:06.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.433+0000 7f2b26ffd700 1 -- 172.21.15.36:0/2404825142 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2000ea50 con 0x7f2b08004e70 2022-06-17T12:51:06.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.433+0000 7f2b26ffd700 1 -- 172.21.15.36:0/2404825142 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b2000edb0 con 0x7f2b08004e70 2022-06-17T12:51:06.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.433+0000 7f2b2cf46700 1 -- 172.21.15.36:0/2404825142 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b08004e70 msgr2=0x7f2b0800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:06.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.433+0000 7f2b2cf46700 1 --2- 172.21.15.36:0/2404825142 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b08004e70 0x7f2b0800b750 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f2b2000c000 tx=0x7f2b200091d0).stop 2022-06-17T12:51:06.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.433+0000 7f2b2cf46700 1 -- 172.21.15.36:0/2404825142 shutdown_connections 2022-06-17T12:51:06.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.433+0000 7f2b2cf46700 1 --2- 172.21.15.36:0/2404825142 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b08004e70 0x7f2b0800b750 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:06.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.433+0000 7f2b2cf46700 1 -- 172.21.15.36:0/2404825142 >> 172.21.15.36:0/2404825142 conn(0x7f2b2805f770 msgr2=0x7f2b2805fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:51:06.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.433+0000 7f2b2cf46700 1 -- 172.21.15.36:0/2404825142 shutdown_connections 2022-06-17T12:51:06.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.433+0000 7f2b2cf46700 1 -- 172.21.15.36:0/2404825142 wait complete. 2022-06-17T12:51:06.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b2cf46700 1 Processor -- start 2022-06-17T12:51:06.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b2cf46700 1 -- start start 2022-06-17T12:51:06.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b2cf46700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b08004e70 0x7f2b2814e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:06.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b2cf46700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b280ab0c0 0x7f2b28066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:06.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b2cf46700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b280aa180 0x7f2b280671c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:06.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b2cf46700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b280ab560 con 0x7f2b08004e70 2022-06-17T12:51:06.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b2cf46700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b280a91e0 con 0x7f2b280aa180 2022-06-17T12:51:06.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b2cf46700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b280a93d0 con 0x7f2b280ab0c0 2022-06-17T12:51:06.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b277fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b08004e70 0x7f2b2814e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:06.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b277fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b08004e70 0x7f2b2814e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34308/0 (socket says 172.21.15.36:34308) 2022-06-17T12:51:06.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b277fe700 1 -- 172.21.15.36:0/1078526306 learned_addr learned my addr 172.21.15.36:0/1078526306 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:51:06.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b1effd700 1 --2- 172.21.15.36:0/1078526306 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b280ab0c0 0x7f2b28066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:06.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b27fff700 1 --2- 172.21.15.36:0/1078526306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b280aa180 0x7f2b280671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:06.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b277fe700 1 -- 172.21.15.36:0/1078526306 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b280ab0c0 msgr2=0x7f2b28066cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:06.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b277fe700 1 --2- 172.21.15.36:0/1078526306 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b280ab0c0 0x7f2b28066cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:06.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b277fe700 1 -- 172.21.15.36:0/1078526306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b280aa180 msgr2=0x7f2b280671c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:06.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b277fe700 1 --2- 172.21.15.36:0/1078526306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b280aa180 0x7f2b280671c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:06.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b277fe700 1 -- 172.21.15.36:0/1078526306 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b280a9670 con 0x7f2b08004e70 2022-06-17T12:51:06.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b27fff700 1 --2- 172.21.15.36:0/1078526306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b280aa180 0x7f2b280671c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:51:06.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b277fe700 1 --2- 172.21.15.36:0/1078526306 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b08004e70 0x7f2b2814e5b0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f2b18007580 tx=0x7f2b18009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:06.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b257fa700 1 -- 172.21.15.36:0/1078526306 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b180155f0 con 0x7f2b08004e70 2022-06-17T12:51:06.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b257fa700 1 -- 172.21.15.36:0/1078526306 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b18010040 con 0x7f2b08004e70 2022-06-17T12:51:06.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b2cf46700 1 -- 172.21.15.36:0/1078526306 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b280a8ec0 con 0x7f2b08004e70 2022-06-17T12:51:06.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b257fa700 1 -- 172.21.15.36:0/1078526306 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b18015940 con 0x7f2b08004e70 2022-06-17T12:51:06.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b2cf46700 1 -- 172.21.15.36:0/1078526306 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b28159da0 con 0x7f2b08004e70 2022-06-17T12:51:06.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.437+0000 7f2b2cf46700 1 -- 172.21.15.36:0/1078526306 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b28057410 con 0x7f2b08004e70 2022-06-17T12:51:06.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.441+0000 7f2b257fa700 1 -- 172.21.15.36:0/1078526306 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2b18015aa0 con 0x7f2b08004e70 2022-06-17T12:51:06.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.441+0000 7f2b257fa700 1 --2- 172.21.15.36:0/1078526306 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b0c053a40 0x7f2b0c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:06.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.441+0000 7f2b257fa700 1 -- 172.21.15.36:0/1078526306 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2b1806c200 con 0x7f2b08004e70 2022-06-17T12:51:06.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.441+0000 7f2b1effd700 1 --2- 172.21.15.36:0/1078526306 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b0c053a40 0x7f2b0c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:06.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.441+0000 7f2b1effd700 1 --2- 172.21.15.36:0/1078526306 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b0c053a40 0x7f2b0c055f00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f2b10005000 tx=0x7f2b28060500).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:06.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.445+0000 7f2b257fa700 1 -- 172.21.15.36:0/1078526306 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2b18043000 con 0x7f2b08004e70 2022-06-17T12:51:06.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.613+0000 7f2b2cf46700 1 -- 172.21.15.36:0/1078526306 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b28069cb0 con 0x7f2b08004e70 2022-06-17T12:51:06.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.613+0000 7f2b257fa700 1 -- 172.21.15.36:0/1078526306 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f2b18046620 con 0x7f2b08004e70 2022-06-17T12:51:06.608 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:51:06.609 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":25,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5670,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.36:6835/2598001220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2598001220},{"type":"v1","addr":"172.21.15.36:6835","nonce":2598001220}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":23}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:50:49.096455+0000","modified":"2022-06-17T12:50:50.099539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"b","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/2723288138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2723288138},{"type":"v1","addr":"172.21.15.70:6833","nonce":2723288138}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-06-17T12:51:06.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.617+0000 7f2b2cf46700 1 -- 172.21.15.36:0/1078526306 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b0c053a40 msgr2=0x7f2b0c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:06.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.617+0000 7f2b2cf46700 1 --2- 172.21.15.36:0/1078526306 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b0c053a40 0x7f2b0c055f00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f2b10005000 tx=0x7f2b28060500).stop 2022-06-17T12:51:06.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.617+0000 7f2b2cf46700 1 -- 172.21.15.36:0/1078526306 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b08004e70 msgr2=0x7f2b2814e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:06.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.617+0000 7f2b2cf46700 1 --2- 172.21.15.36:0/1078526306 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b08004e70 0x7f2b2814e5b0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f2b18007580 tx=0x7f2b18009040).stop 2022-06-17T12:51:06.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.617+0000 7f2b2cf46700 1 -- 172.21.15.36:0/1078526306 shutdown_connections 2022-06-17T12:51:06.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.617+0000 7f2b2cf46700 1 --2- 172.21.15.36:0/1078526306 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b0c053a40 0x7f2b0c055f00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:06.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.617+0000 7f2b2cf46700 1 --2- 172.21.15.36:0/1078526306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b280aa180 0x7f2b280671c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:06.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.617+0000 7f2b2cf46700 1 --2- 172.21.15.36:0/1078526306 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b280ab0c0 0x7f2b28066cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:06.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.617+0000 7f2b2cf46700 1 --2- 172.21.15.36:0/1078526306 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b08004e70 0x7f2b2814e5b0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:06.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.617+0000 7f2b2cf46700 1 -- 172.21.15.36:0/1078526306 >> 172.21.15.36:0/1078526306 conn(0x7f2b2805f770 msgr2=0x7f2b280b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:51:06.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.617+0000 7f2b2cf46700 1 -- 172.21.15.36:0/1078526306 shutdown_connections 2022-06-17T12:51:06.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.617+0000 7f2b2cf46700 1 -- 172.21.15.36:0/1078526306 wait complete. 2022-06-17T12:51:06.618 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 25 2022-06-17T12:51:06.621 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_recall_max_caps 20 2022-06-17T12:51:06.772 INFO:teuthology.orchestra.run.smithi070.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2022-06-17T12:51:06.773 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2022-06-17T12:51:06.774 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:51:06.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98768c0700 1 Processor -- start 2022-06-17T12:51:06.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98768c0700 1 -- start start 2022-06-17T12:51:06.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98768c0700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f98700b2200 0x7f98700af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:06.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98768c0700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98700b2cd0 0x7f9870147600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:06.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98768c0700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98700b25e0 con 0x7f98700af8d0 2022-06-17T12:51:06.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98768c0700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98700ad790 con 0x7f98700b2200 2022-06-17T12:51:06.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98768c0700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98700aef40 con 0x7f98700b2cd0 2022-06-17T12:51:06.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98758be700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f98700b2200 0x7f98700af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:06.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98758be700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f98700b2200 0x7f98700af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34318/0 (socket says 172.21.15.36:34318) 2022-06-17T12:51:06.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98758be700 1 -- 172.21.15.36:0/2196698659 learned_addr learned my addr 172.21.15.36:0/2196698659 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:51:06.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98760bf700 1 --2- 172.21.15.36:0/2196698659 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98700b2cd0 0x7f9870147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:06.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98758be700 1 -- 172.21.15.36:0/2196698659 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98700b2cd0 msgr2=0x7f9870147600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:06.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98758be700 1 --2- 172.21.15.36:0/2196698659 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98700b2cd0 0x7f9870147600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:06.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98758be700 1 -- 172.21.15.36:0/2196698659 >> v1:172.21.15.70:6789/0 conn(0x7f98700af8d0 legacy=0x7f987014a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:06.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98758be700 1 -- 172.21.15.36:0/2196698659 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98700b1950 con 0x7f98700b2200 2022-06-17T12:51:06.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98760bf700 1 --2- 172.21.15.36:0/2196698659 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98700b2cd0 0x7f9870147600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:51:06.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98758be700 1 --2- 172.21.15.36:0/2196698659 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98700b2200 0x7f98700af3e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f9864002f70 tx=0x7f9864006eb0).ready entity=mon.0 client_cookie=6ca4dab9c395ddce server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:06.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98748bc700 1 -- 172.21.15.36:0/2196698659 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f986400d5c0 con 0x7f98700b2200 2022-06-17T12:51:06.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98748bc700 1 -- 172.21.15.36:0/2196698659 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9864000e90 con 0x7f98700b2200 2022-06-17T12:51:06.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98748bc700 1 -- 172.21.15.36:0/2196698659 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f986400d950 con 0x7f98700b2200 2022-06-17T12:51:06.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98768c0700 1 -- 172.21.15.36:0/2196698659 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98700b2200 msgr2=0x7f98700af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:06.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98768c0700 1 --2- 172.21.15.36:0/2196698659 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98700b2200 0x7f98700af3e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f9864002f70 tx=0x7f9864006eb0).stop 2022-06-17T12:51:06.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98768c0700 1 -- 172.21.15.36:0/2196698659 shutdown_connections 2022-06-17T12:51:06.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98768c0700 1 --2- 172.21.15.36:0/2196698659 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98700b2cd0 0x7f9870147600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:06.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98768c0700 1 --2- 172.21.15.36:0/2196698659 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98700b2200 0x7f98700af3e0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:06.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98768c0700 1 -- 172.21.15.36:0/2196698659 >> 172.21.15.36:0/2196698659 conn(0x7f987005f770 msgr2=0x7f987005fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:51:06.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98768c0700 1 -- 172.21.15.36:0/2196698659 shutdown_connections 2022-06-17T12:51:06.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.905+0000 7f98768c0700 1 -- 172.21.15.36:0/2196698659 wait complete. 2022-06-17T12:51:06.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f98768c0700 1 Processor -- start 2022-06-17T12:51:06.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f98768c0700 1 -- start start 2022-06-17T12:51:06.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f98768c0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98700af8d0 0x7f9870063cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:06.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f98768c0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98700b2200 0x7f98700641a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:06.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f98768c0700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98700b2cd0 0x7f9870066c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:06.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f98768c0700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9870157700 con 0x7f98700af8d0 2022-06-17T12:51:06.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f98768c0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9870157460 con 0x7f98700b2200 2022-06-17T12:51:06.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f98768c0700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98700aa1d0 con 0x7f98700b2cd0 2022-06-17T12:51:06.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f98758be700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98700af8d0 0x7f9870063cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:06.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f98758be700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98700af8d0 0x7f9870063cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34322/0 (socket says 172.21.15.36:34322) 2022-06-17T12:51:06.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f98758be700 1 -- 172.21.15.36:0/3575587324 learned_addr learned my addr 172.21.15.36:0/3575587324 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:51:06.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f98760bf700 1 --2- 172.21.15.36:0/3575587324 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98700b2cd0 0x7f9870066c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:06.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f98750bd700 1 --2- 172.21.15.36:0/3575587324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98700b2200 0x7f98700641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:06.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f98758be700 1 -- 172.21.15.36:0/3575587324 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98700b2cd0 msgr2=0x7f9870066c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:06.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f98758be700 1 --2- 172.21.15.36:0/3575587324 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98700b2cd0 0x7f9870066c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:06.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f98758be700 1 -- 172.21.15.36:0/3575587324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98700b2200 msgr2=0x7f98700641a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:06.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f98758be700 1 --2- 172.21.15.36:0/3575587324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98700b2200 0x7f98700641a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:06.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f98758be700 1 -- 172.21.15.36:0/3575587324 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986400b1e0 con 0x7f98700af8d0 2022-06-17T12:51:06.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f98760bf700 1 --2- 172.21.15.36:0/3575587324 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98700b2cd0 0x7f9870066c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:51:06.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f98758be700 1 --2- 172.21.15.36:0/3575587324 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98700af8d0 0x7f9870063cb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f9864002f70 tx=0x7f9864015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:06.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f986effd700 1 -- 172.21.15.36:0/3575587324 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9864011990 con 0x7f98700af8d0 2022-06-17T12:51:06.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f986effd700 1 -- 172.21.15.36:0/3575587324 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9864011af0 con 0x7f98700af8d0 2022-06-17T12:51:06.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f98768c0700 1 -- 172.21.15.36:0/3575587324 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9870157180 con 0x7f98700af8d0 2022-06-17T12:51:06.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f986effd700 1 -- 172.21.15.36:0/3575587324 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9864011e50 con 0x7f98700af8d0 2022-06-17T12:51:06.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f98768c0700 1 -- 172.21.15.36:0/3575587324 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9870067160 con 0x7f98700af8d0 2022-06-17T12:51:06.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.909+0000 7f98768c0700 1 -- 172.21.15.36:0/3575587324 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98700ad330 con 0x7f98700af8d0 2022-06-17T12:51:06.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.913+0000 7f986effd700 1 -- 172.21.15.36:0/3575587324 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f9864025030 con 0x7f98700af8d0 2022-06-17T12:51:06.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.913+0000 7f986effd700 1 --2- 172.21.15.36:0/3575587324 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f98500539b0 0x7f9850055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:06.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.913+0000 7f986effd700 1 -- 172.21.15.36:0/3575587324 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f986406d370 con 0x7f98700af8d0 2022-06-17T12:51:06.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.913+0000 7f98750bd700 1 --2- 172.21.15.36:0/3575587324 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f98500539b0 0x7f9850055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:06.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.913+0000 7f98750bd700 1 --2- 172.21.15.36:0/3575587324 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f98500539b0 0x7f9850055e70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f986000b000 tx=0x7f9860009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:06.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:06.917+0000 7f986effd700 1 -- 172.21.15.36:0/3575587324 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9864044170 con 0x7f98700af8d0 2022-06-17T12:51:07.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.081+0000 7f98768c0700 1 -- 172.21.15.36:0/3575587324 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9870057410 con 0x7f98700af8d0 2022-06-17T12:51:07.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.081+0000 7f986effd700 1 -- 172.21.15.36:0/3575587324 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f9864023090 con 0x7f98700af8d0 2022-06-17T12:51:07.071 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:51:07.071 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":25,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5670,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.36:6835/2598001220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2598001220},{"type":"v1","addr":"172.21.15.36:6835","nonce":2598001220}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":23}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:50:49.096455+0000","modified":"2022-06-17T12:50:50.099539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"b","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/2723288138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2723288138},{"type":"v1","addr":"172.21.15.70:6833","nonce":2723288138}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-06-17T12:51:07.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.085+0000 7f98768c0700 1 -- 172.21.15.36:0/3575587324 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f98500539b0 msgr2=0x7f9850055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:07.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.085+0000 7f98768c0700 1 --2- 172.21.15.36:0/3575587324 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f98500539b0 0x7f9850055e70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f986000b000 tx=0x7f9860009040).stop 2022-06-17T12:51:07.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.085+0000 7f98768c0700 1 -- 172.21.15.36:0/3575587324 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98700af8d0 msgr2=0x7f9870063cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:07.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.085+0000 7f98768c0700 1 --2- 172.21.15.36:0/3575587324 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98700af8d0 0x7f9870063cb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f9864002f70 tx=0x7f9864015000).stop 2022-06-17T12:51:07.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.085+0000 7f98768c0700 1 -- 172.21.15.36:0/3575587324 shutdown_connections 2022-06-17T12:51:07.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.085+0000 7f98768c0700 1 --2- 172.21.15.36:0/3575587324 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f98500539b0 0x7f9850055e70 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.085+0000 7f98768c0700 1 --2- 172.21.15.36:0/3575587324 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98700b2cd0 0x7f9870066c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.085+0000 7f98768c0700 1 --2- 172.21.15.36:0/3575587324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98700b2200 0x7f98700641a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.085+0000 7f98768c0700 1 --2- 172.21.15.36:0/3575587324 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98700af8d0 0x7f9870063cb0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.085+0000 7f98768c0700 1 -- 172.21.15.36:0/3575587324 >> 172.21.15.36:0/3575587324 conn(0x7f987005f770 msgr2=0x7f98700b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:51:07.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.085+0000 7f98768c0700 1 -- 172.21.15.36:0/3575587324 shutdown_connections 2022-06-17T12:51:07.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.085+0000 7f98768c0700 1 -- 172.21.15.36:0/3575587324 wait complete. 2022-06-17T12:51:07.077 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 25 2022-06-17T12:51:07.082 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_recall_max_decay_threshold 400 2022-06-17T12:51:07.198 INFO:teuthology.orchestra.run.smithi070.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2022-06-17T12:51:07.198 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2022-06-17T12:51:07.199 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@smithi107.front.sepia.ceph.com... 2022-06-17T12:51:07.227 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:51:07.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.353+0000 7faa54020700 1 Processor -- start 2022-06-17T12:51:07.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.353+0000 7faa54020700 1 -- start start 2022-06-17T12:51:07.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.353+0000 7faa54020700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7faa4c0b7c10 0x7faa4c145070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.353+0000 7faa54020700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa4c0ad330 0x7faa4c141090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.353+0000 7faa54020700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faa4c066fd0 0x7faa4c13df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.353+0000 7faa54020700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa4c1422e0 con 0x7faa4c0b7c10 2022-06-17T12:51:07.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.353+0000 7faa54020700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa4c0b7ff0 con 0x7faa4c066fd0 2022-06-17T12:51:07.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.353+0000 7faa54020700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c0b30a0 con 0x7faa4c0ad330 2022-06-17T12:51:07.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.353+0000 7faa51dbc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7faa4c0b7c10 0x7faa4c145070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.353+0000 7faa51dbc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7faa4c0b7c10 0x7faa4c145070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34330/0 (socket says 172.21.15.36:34330) 2022-06-17T12:51:07.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.353+0000 7faa51dbc700 1 -- 172.21.15.36:0/3480078139 learned_addr learned my addr 172.21.15.36:0/3480078139 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:51:07.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.353+0000 7faa515bb700 1 --2- 172.21.15.36:0/3480078139 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa4c0ad330 0x7faa4c141090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.353+0000 7faa525bd700 1 --2- 172.21.15.36:0/3480078139 >> v2:172.21.15.70:3300/0 conn(0x7faa4c066fd0 0x7faa4c13df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.353+0000 7faa51dbc700 1 -- 172.21.15.36:0/3480078139 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa4c0ad330 msgr2=0x7faa4c141090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:07.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.353+0000 7faa51dbc700 1 --2- 172.21.15.36:0/3480078139 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa4c0ad330 0x7faa4c141090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.353+0000 7faa51dbc700 1 -- 172.21.15.36:0/3480078139 >> v2:172.21.15.70:3300/0 conn(0x7faa4c066fd0 msgr2=0x7faa4c13df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:07.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.353+0000 7faa51dbc700 1 --2- 172.21.15.36:0/3480078139 >> v2:172.21.15.70:3300/0 conn(0x7faa4c066fd0 0x7faa4c13df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.353+0000 7faa51dbc700 1 -- 172.21.15.36:0/3480078139 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4c0b3340 con 0x7faa4c0b7c10 2022-06-17T12:51:07.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.353+0000 7faa51dbc700 1 --2- 172.21.15.36:0/3480078139 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa4c0b7c10 0x7faa4c145070 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7faa44002f70 tx=0x7faa44006eb0).ready entity=mon.0 client_cookie=c4c4c8fd93bf2c1a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:07.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.353+0000 7faa50dba700 1 -- 172.21.15.36:0/3480078139 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa4400d570 con 0x7faa4c0b7c10 2022-06-17T12:51:07.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.353+0000 7faa50dba700 1 -- 172.21.15.36:0/3480078139 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa44000e90 con 0x7faa4c0b7c10 2022-06-17T12:51:07.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.353+0000 7faa50dba700 1 -- 172.21.15.36:0/3480078139 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa4400d900 con 0x7faa4c0b7c10 2022-06-17T12:51:07.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.357+0000 7faa54020700 1 -- 172.21.15.36:0/3480078139 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa4c0b7c10 msgr2=0x7faa4c145070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:07.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.357+0000 7faa54020700 1 --2- 172.21.15.36:0/3480078139 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa4c0b7c10 0x7faa4c145070 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7faa44002f70 tx=0x7faa44006eb0).stop 2022-06-17T12:51:07.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.357+0000 7faa54020700 1 -- 172.21.15.36:0/3480078139 shutdown_connections 2022-06-17T12:51:07.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.357+0000 7faa54020700 1 --2- 172.21.15.36:0/3480078139 >> v2:172.21.15.70:3300/0 conn(0x7faa4c066fd0 0x7faa4c13df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.357+0000 7faa54020700 1 --2- 172.21.15.36:0/3480078139 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa4c0ad330 0x7faa4c141090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.357+0000 7faa54020700 1 --2- 172.21.15.36:0/3480078139 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa4c0b7c10 0x7faa4c145070 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.357+0000 7faa54020700 1 -- 172.21.15.36:0/3480078139 >> 172.21.15.36:0/3480078139 conn(0x7faa4c060c60 msgr2=0x7faa4c05c550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:51:07.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.357+0000 7faa54020700 1 -- 172.21.15.36:0/3480078139 shutdown_connections 2022-06-17T12:51:07.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.357+0000 7faa54020700 1 -- 172.21.15.36:0/3480078139 wait complete. 2022-06-17T12:51:07.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.357+0000 7faa54020700 1 Processor -- start 2022-06-17T12:51:07.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.357+0000 7faa54020700 1 -- start start 2022-06-17T12:51:07.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.361+0000 7faa54020700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa4c066fd0 0x7faa4c13a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.361+0000 7faa54020700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa4c0ad330 0x7faa4c0b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.361+0000 7faa54020700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa4c0b7c10 0x7faa4c0ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.361+0000 7faa54020700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c140860 con 0x7faa4c0b7c10 2022-06-17T12:51:07.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.361+0000 7faa54020700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c150160 con 0x7faa4c066fd0 2022-06-17T12:51:07.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.361+0000 7faa54020700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c150ce0 con 0x7faa4c0ad330 2022-06-17T12:51:07.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.361+0000 7faa525bd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa4c0b7c10 0x7faa4c0ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.361+0000 7faa515bb700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa4c0ad330 0x7faa4c0b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.361+0000 7faa525bd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa4c0b7c10 0x7faa4c0ae960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34340/0 (socket says 172.21.15.36:34340) 2022-06-17T12:51:07.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.361+0000 7faa51dbc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa4c066fd0 0x7faa4c13a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.361+0000 7faa525bd700 1 -- 172.21.15.36:0/3136540198 learned_addr learned my addr 172.21.15.36:0/3136540198 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:51:07.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.361+0000 7faa525bd700 1 -- 172.21.15.36:0/3136540198 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa4c0ad330 msgr2=0x7faa4c0b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:07.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.361+0000 7faa525bd700 1 --2- 172.21.15.36:0/3136540198 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa4c0ad330 0x7faa4c0b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.361+0000 7faa525bd700 1 -- 172.21.15.36:0/3136540198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa4c066fd0 msgr2=0x7faa4c13a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:07.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.361+0000 7faa525bd700 1 --2- 172.21.15.36:0/3136540198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa4c066fd0 0x7faa4c13a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.361+0000 7faa525bd700 1 -- 172.21.15.36:0/3136540198 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4400b1e0 con 0x7faa4c0b7c10 2022-06-17T12:51:07.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.361+0000 7faa525bd700 1 --2- 172.21.15.36:0/3136540198 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa4c0b7c10 0x7faa4c0ae960 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7faa3c00c000 tx=0x7faa3c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:07.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.361+0000 7faa42ffd700 1 -- 172.21.15.36:0/3136540198 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa3c011750 con 0x7faa4c0b7c10 2022-06-17T12:51:07.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.361+0000 7faa42ffd700 1 -- 172.21.15.36:0/3136540198 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3c0118b0 con 0x7faa4c0b7c10 2022-06-17T12:51:07.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.361+0000 7faa42ffd700 1 -- 172.21.15.36:0/3136540198 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa3c011c10 con 0x7faa4c0b7c10 2022-06-17T12:51:07.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.361+0000 7faa54020700 1 -- 172.21.15.36:0/3136540198 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa4c150f50 con 0x7faa4c0b7c10 2022-06-17T12:51:07.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.361+0000 7faa54020700 1 -- 172.21.15.36:0/3136540198 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa4c0adee0 con 0x7faa4c0b7c10 2022-06-17T12:51:07.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.365+0000 7faa54020700 1 -- 172.21.15.36:0/3136540198 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa4c057410 con 0x7faa4c0b7c10 2022-06-17T12:51:07.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.365+0000 7faa42ffd700 1 -- 172.21.15.36:0/3136540198 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7faa3c011d70 con 0x7faa4c0b7c10 2022-06-17T12:51:07.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.365+0000 7faa42ffd700 1 --2- 172.21.15.36:0/3136540198 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa2c053ad0 0x7faa2c055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.365+0000 7faa42ffd700 1 -- 172.21.15.36:0/3136540198 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7faa3c06c2c0 con 0x7faa4c0b7c10 2022-06-17T12:51:07.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.365+0000 7faa51dbc700 1 --2- 172.21.15.36:0/3136540198 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa2c053ad0 0x7faa2c055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.365+0000 7faa51dbc700 1 --2- 172.21.15.36:0/3136540198 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa2c053ad0 0x7faa2c055f90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7faa4401c000 tx=0x7faa44015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:07.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.373+0000 7faa42ffd700 1 -- 172.21.15.36:0/3136540198 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faa3c0430c0 con 0x7faa4c0b7c10 2022-06-17T12:51:07.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.537+0000 7faa54020700 1 -- 172.21.15.36:0/3136540198 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa4c0b2890 con 0x7faa4c0b7c10 2022-06-17T12:51:07.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.541+0000 7faa42ffd700 1 -- 172.21.15.36:0/3136540198 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+2139 (secure 0 0 0) 0x7faa3c0466e0 con 0x7faa4c0b7c10 2022-06-17T12:51:07.525 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:51:07.525 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":25,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5670,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.36:6835/2598001220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2598001220},{"type":"v1","addr":"172.21.15.36:6835","nonce":2598001220}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":23}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:50:49.096455+0000","modified":"2022-06-17T12:50:50.099539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"b","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/2723288138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2723288138},{"type":"v1","addr":"172.21.15.70:6833","nonce":2723288138}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-06-17T12:51:07.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.541+0000 7faa54020700 1 -- 172.21.15.36:0/3136540198 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa2c053ad0 msgr2=0x7faa2c055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:07.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.541+0000 7faa54020700 1 --2- 172.21.15.36:0/3136540198 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa2c053ad0 0x7faa2c055f90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7faa4401c000 tx=0x7faa44015000).stop 2022-06-17T12:51:07.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.541+0000 7faa54020700 1 -- 172.21.15.36:0/3136540198 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa4c0b7c10 msgr2=0x7faa4c0ae960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:07.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.541+0000 7faa54020700 1 --2- 172.21.15.36:0/3136540198 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa4c0b7c10 0x7faa4c0ae960 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7faa3c00c000 tx=0x7faa3c006df0).stop 2022-06-17T12:51:07.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.541+0000 7faa54020700 1 -- 172.21.15.36:0/3136540198 shutdown_connections 2022-06-17T12:51:07.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.541+0000 7faa54020700 1 --2- 172.21.15.36:0/3136540198 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa2c053ad0 0x7faa2c055f90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.541+0000 7faa54020700 1 --2- 172.21.15.36:0/3136540198 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa4c0b7c10 0x7faa4c0ae960 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.541+0000 7faa54020700 1 --2- 172.21.15.36:0/3136540198 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa4c0ad330 0x7faa4c0b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.541+0000 7faa54020700 1 --2- 172.21.15.36:0/3136540198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa4c066fd0 0x7faa4c13a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.541+0000 7faa54020700 1 -- 172.21.15.36:0/3136540198 >> 172.21.15.36:0/3136540198 conn(0x7faa4c060c60 msgr2=0x7faa4c13aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:51:07.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.541+0000 7faa54020700 1 -- 172.21.15.36:0/3136540198 shutdown_connections 2022-06-17T12:51:07.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.541+0000 7faa54020700 1 -- 172.21.15.36:0/3136540198 wait complete. 2022-06-17T12:51:07.542 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 25 2022-06-17T12:51:07.544 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b cache drop 5 2022-06-17T12:51:07.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc41ec67700 1 Processor -- start 2022-06-17T12:51:07.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc41ec67700 1 -- start start 2022-06-17T12:51:07.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc41ec67700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc4180b7d40 0x7fc418067230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc41ec67700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc418141050 0x7fc418141470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc41ec67700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4180b8120 con 0x7fc4180649c0 2022-06-17T12:51:07.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc41ec67700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc418067e40 con 0x7fc418141050 2022-06-17T12:51:07.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc41ec67700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc418067720 con 0x7fc4180b7d40 2022-06-17T12:51:07.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc41d204700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc418141050 0x7fc418141470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc41d204700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc418141050 0x7fc418141470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34346/0 (socket says 172.21.15.36:34346) 2022-06-17T12:51:07.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc41d204700 1 -- 172.21.15.36:0/2400411617 learned_addr learned my addr 172.21.15.36:0/2400411617 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:51:07.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc41ca03700 1 --2- 172.21.15.36:0/2400411617 >> v2:172.21.15.70:3300/0 conn(0x7fc4180b7d40 0x7fc418067230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc41d204700 1 -- 172.21.15.36:0/2400411617 >> v2:172.21.15.70:3300/0 conn(0x7fc4180b7d40 msgr2=0x7fc418067230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:07.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc41d204700 1 --2- 172.21.15.36:0/2400411617 >> v2:172.21.15.70:3300/0 conn(0x7fc4180b7d40 0x7fc418067230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc41d204700 1 -- 172.21.15.36:0/2400411617 >> v1:172.21.15.70:6789/0 conn(0x7fc4180649c0 legacy=0x7fc418064e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:07.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc41d204700 1 -- 172.21.15.36:0/2400411617 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc418067900 con 0x7fc418141050 2022-06-17T12:51:07.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc41d204700 1 --2- 172.21.15.36:0/2400411617 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc418141050 0x7fc418141470 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7fc4100032f0 tx=0x7fc410009290).ready entity=mon.0 client_cookie=6e47799cf3e637bf server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:07.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc4177fe700 1 -- 172.21.15.36:0/2400411617 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc41000b870 con 0x7fc418141050 2022-06-17T12:51:07.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc4177fe700 1 -- 172.21.15.36:0/2400411617 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41000b9d0 con 0x7fc418141050 2022-06-17T12:51:07.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc4177fe700 1 -- 172.21.15.36:0/2400411617 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc41000bb40 con 0x7fc418141050 2022-06-17T12:51:07.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc41ec67700 1 -- 172.21.15.36:0/2400411617 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc418141050 msgr2=0x7fc418141470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:07.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc41ec67700 1 --2- 172.21.15.36:0/2400411617 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc418141050 0x7fc418141470 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7fc4100032f0 tx=0x7fc410009290).stop 2022-06-17T12:51:07.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc41ec67700 1 -- 172.21.15.36:0/2400411617 shutdown_connections 2022-06-17T12:51:07.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc41ec67700 1 --2- 172.21.15.36:0/2400411617 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc418141050 0x7fc418141470 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc41ec67700 1 --2- 172.21.15.36:0/2400411617 >> v2:172.21.15.70:3300/0 conn(0x7fc4180b7d40 0x7fc418067230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc41ec67700 1 -- 172.21.15.36:0/2400411617 >> 172.21.15.36:0/2400411617 conn(0x7fc41805c920 msgr2=0x7fc418060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:51:07.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc41ec67700 1 -- 172.21.15.36:0/2400411617 shutdown_connections 2022-06-17T12:51:07.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.673+0000 7fc41ec67700 1 -- 172.21.15.36:0/2400411617 wait complete. 2022-06-17T12:51:07.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.677+0000 7fc41ec67700 1 Processor -- start 2022-06-17T12:51:07.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.677+0000 7fc41ec67700 1 -- start start 2022-06-17T12:51:07.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.677+0000 7fc41ec67700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4180649c0 0x7fc4180b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.677+0000 7fc41ec67700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4180b7d40 0x7fc418126d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.677+0000 7fc41ec67700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc418141050 0x7fc418127210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.677+0000 7fc41ec67700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41812b420 con 0x7fc418141050 2022-06-17T12:51:07.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.677+0000 7fc41ec67700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41812b6c0 con 0x7fc4180649c0 2022-06-17T12:51:07.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.677+0000 7fc41ec67700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41812b960 con 0x7fc4180b7d40 2022-06-17T12:51:07.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.677+0000 7fc41d204700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc418141050 0x7fc418127210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.677+0000 7fc41d204700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc418141050 0x7fc418127210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34354/0 (socket says 172.21.15.36:34354) 2022-06-17T12:51:07.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.677+0000 7fc41d204700 1 -- 172.21.15.36:0/2669035062 learned_addr learned my addr 172.21.15.36:0/2669035062 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:51:07.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.677+0000 7fc41ca03700 1 --2- 172.21.15.36:0/2669035062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4180649c0 0x7fc4180b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.677+0000 7fc417fff700 1 --2- 172.21.15.36:0/2669035062 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4180b7d40 0x7fc418126d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.677+0000 7fc41d204700 1 -- 172.21.15.36:0/2669035062 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4180b7d40 msgr2=0x7fc418126d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:07.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.677+0000 7fc41d204700 1 --2- 172.21.15.36:0/2669035062 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4180b7d40 0x7fc418126d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.677+0000 7fc41d204700 1 -- 172.21.15.36:0/2669035062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4180649c0 msgr2=0x7fc4180b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:07.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.677+0000 7fc41d204700 1 --2- 172.21.15.36:0/2669035062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4180649c0 0x7fc4180b56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.677+0000 7fc41d204700 1 -- 172.21.15.36:0/2669035062 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc410009530 con 0x7fc418141050 2022-06-17T12:51:07.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.677+0000 7fc41d204700 1 --2- 172.21.15.36:0/2669035062 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc418141050 0x7fc418127210 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7fc4100032f0 tx=0x7fc410016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:07.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.677+0000 7fc415ffb700 1 -- 172.21.15.36:0/2669035062 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc41000b870 con 0x7fc418141050 2022-06-17T12:51:07.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.677+0000 7fc415ffb700 1 -- 172.21.15.36:0/2669035062 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc410021670 con 0x7fc418141050 2022-06-17T12:51:07.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.677+0000 7fc41ec67700 1 -- 172.21.15.36:0/2669035062 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc418140b50 con 0x7fc418141050 2022-06-17T12:51:07.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.677+0000 7fc415ffb700 1 -- 172.21.15.36:0/2669035062 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4100219d0 con 0x7fc418141050 2022-06-17T12:51:07.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.677+0000 7fc41ec67700 1 -- 172.21.15.36:0/2669035062 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41812be00 con 0x7fc418141050 2022-06-17T12:51:07.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.681+0000 7fc415ffb700 1 -- 172.21.15.36:0/2669035062 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fc41000f070 con 0x7fc418141050 2022-06-17T12:51:07.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.681+0000 7fc415ffb700 1 --2- 172.21.15.36:0/2669035062 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc3f8053a40 0x7fc3f8055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.681+0000 7fc415ffb700 1 -- 172.21.15.36:0/2669035062 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc41006d1b0 con 0x7fc418141050 2022-06-17T12:51:07.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.681+0000 7fc41ca03700 1 --2- 172.21.15.36:0/2669035062 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc3f8053a40 0x7fc3f8055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.681+0000 7fc41ca03700 1 --2- 172.21.15.36:0/2669035062 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc3f8053a40 0x7fc3f8055f00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fc40c00b000 tx=0x7fc40c006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:07.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.685+0000 7fc428ebc740 1 Processor -- start 2022-06-17T12:51:07.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.685+0000 7fc428ebc740 1 -- start start 2022-06-17T12:51:07.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.685+0000 7fc428ebc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x225c630 0x225ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.685+0000 7fc428ebc740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2297b70 0x2297fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.685+0000 7fc428ebc740 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225d610 0x225da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.685+0000 7fc428ebc740 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x21d1e60 con 0x225d610 2022-06-17T12:51:07.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.685+0000 7fc428ebc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x21b20c0 con 0x225c630 2022-06-17T12:51:07.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.685+0000 7fc428ebc740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x225df80 con 0x2297b70 2022-06-17T12:51:07.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.685+0000 7fc41d204700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225d610 0x225da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.685+0000 7fc41d204700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225d610 0x225da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34362/0 (socket says 172.21.15.36:34362) 2022-06-17T12:51:07.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.685+0000 7fc41d204700 1 -- 172.21.15.36:0/453696445 learned_addr learned my addr 172.21.15.36:0/453696445 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:51:07.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.685+0000 7fc41ca03700 1 --2- 172.21.15.36:0/453696445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x225c630 0x225ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.685+0000 7fc417fff700 1 --2- 172.21.15.36:0/453696445 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2297b70 0x2297fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.685+0000 7fc41d204700 1 -- 172.21.15.36:0/453696445 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2297b70 msgr2=0x2297fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:07.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.685+0000 7fc41d204700 1 --2- 172.21.15.36:0/453696445 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2297b70 0x2297fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.685+0000 7fc41d204700 1 -- 172.21.15.36:0/453696445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x225c630 msgr2=0x225ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:07.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.685+0000 7fc41d204700 1 --2- 172.21.15.36:0/453696445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x225c630 0x225ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.685+0000 7fc41d204700 1 -- 172.21.15.36:0/453696445 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2209720 con 0x225d610 2022-06-17T12:51:07.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.685+0000 7fc41d204700 1 --2- 172.21.15.36:0/453696445 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225d610 0x225da90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x2297a50 tx=0x7fc410012000).ready entity=mon.0 client_cookie=ab389ad6759b4253 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:07.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.685+0000 7fc41ec67700 1 -- 172.21.15.36:0/453696445 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc41002f070 con 0x225d610 2022-06-17T12:51:07.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.685+0000 7fc41ec67700 1 -- 172.21.15.36:0/453696445 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4100460c0 con 0x225d610 2022-06-17T12:51:07.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.685+0000 7fc41ec67700 1 -- 172.21.15.36:0/453696445 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc410021d20 con 0x225d610 2022-06-17T12:51:07.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.685+0000 7fc428ebc740 1 -- 172.21.15.36:0/453696445 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225d610 msgr2=0x225da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:07.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.685+0000 7fc428ebc740 1 --2- 172.21.15.36:0/453696445 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225d610 0x225da90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x2297a50 tx=0x7fc410012000).stop 2022-06-17T12:51:07.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc428ebc740 1 -- 172.21.15.36:0/453696445 shutdown_connections 2022-06-17T12:51:07.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc428ebc740 1 --2- 172.21.15.36:0/453696445 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225d610 0x225da90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc428ebc740 1 --2- 172.21.15.36:0/453696445 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2297b70 0x2297fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc428ebc740 1 --2- 172.21.15.36:0/453696445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x225c630 0x225ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc428ebc740 1 -- 172.21.15.36:0/453696445 >> 172.21.15.36:0/453696445 conn(0x21e9540 msgr2=0x2261480 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:51:07.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc428ebc740 1 -- 172.21.15.36:0/453696445 shutdown_connections 2022-06-17T12:51:07.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc428ebc740 1 -- 172.21.15.36:0/453696445 wait complete. 2022-06-17T12:51:07.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc428ebc740 1 -- start start 2022-06-17T12:51:07.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc428ebc740 1 Processor -- start 2022-06-17T12:51:07.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc428ebc740 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225c630 0x22a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc428ebc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x225d610 0x2292c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc428ebc740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2297b70 0x2293170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc428ebc740 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x22a5a90 con 0x225c630 2022-06-17T12:51:07.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc428ebc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x2262490 con 0x225d610 2022-06-17T12:51:07.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc428ebc740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x22a5cb0 con 0x2297b70 2022-06-17T12:51:07.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc428ebc740 10 client.0 ll_register_callbacks cb 0x2218d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-06-17T12:51:07.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc41ca03700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225c630 0x22a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc41ca03700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225c630 0x22a55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34364/0 (socket says 172.21.15.36:34364) 2022-06-17T12:51:07.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc41ca03700 1 -- 172.21.15.36:0/3070405027 learned_addr learned my addr 172.21.15.36:0/3070405027 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:51:07.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc41d204700 1 --2- 172.21.15.36:0/3070405027 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2297b70 0x2293170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc417fff700 1 --2- 172.21.15.36:0/3070405027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x225d610 0x2292c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc41ca03700 1 -- 172.21.15.36:0/3070405027 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2297b70 msgr2=0x2293170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:07.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc41ca03700 1 --2- 172.21.15.36:0/3070405027 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2297b70 0x2293170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc41ca03700 1 -- 172.21.15.36:0/3070405027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x225d610 msgr2=0x2292c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:07.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc41ca03700 1 --2- 172.21.15.36:0/3070405027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x225d610 0x2292c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc41ca03700 1 -- 172.21.15.36:0/3070405027 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41812be00 con 0x225c630 2022-06-17T12:51:07.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc41ca03700 1 --2- 172.21.15.36:0/3070405027 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225c630 0x22a55a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fc40c01a000 tx=0x7fc40c016000).ready entity=mon.0 client_cookie=68fb2829de28ce12 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:07.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc3f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.36:3300/0 2022-06-17T12:51:07.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc3f77fe700 1 -- 172.21.15.36:0/3070405027 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc40c0135b0 con 0x225c630 2022-06-17T12:51:07.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc428ebc740 1 -- 172.21.15.36:0/3070405027 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x21c7d00 con 0x225c630 2022-06-17T12:51:07.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc3f77fe700 1 -- 172.21.15.36:0/3070405027 <== mon.0 v2:172.21.15.36:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc40c013710 con 0x225c630 2022-06-17T12:51:07.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc3f77fe700 1 -- 172.21.15.36:0/3070405027 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc40c0138d0 con 0x225c630 2022-06-17T12:51:07.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc428ebc740 10 client.5877 fetch_fsmap learned FSMap version 25 2022-06-17T12:51:07.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc428ebc740 1 -- 172.21.15.36:0/3070405027 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x21c12f0 con 0x225c630 2022-06-17T12:51:07.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.689+0000 7fc3f77fe700 1 -- 172.21.15.36:0/3070405027 <== mon.0 v2:172.21.15.36:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x21c7d00 con 0x225c630 2022-06-17T12:51:07.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc3f77fe700 1 -- 172.21.15.36:0/3070405027 <== mon.0 v2:172.21.15.36:3300/0 5 ==== fsmap(e 25) v1 ==== 1247+0+0 (secure 0 0 0) 0x7fc40c0272f0 con 0x225c630 2022-06-17T12:51:07.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc428ebc740 10 client.5877 fetch_fsmap finished waiting for FSMap version 25 2022-06-17T12:51:07.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc428ebc740 10 client.5877 resolve_mds: resolved ID 'b' to GID 5679 2022-06-17T12:51:07.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc428ebc740 20 client.5877 populate_metadata read hostname 'smithi036' 2022-06-17T12:51:07.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc428ebc740 1 --2- 172.21.15.36:0/3070405027 >> [v2:172.21.15.70:6832/2723288138,v1:172.21.15.70:6833/2723288138] conn(0x2265150 0x2265530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc428ebc740 4 client.5877 mds_command: new command op to 5679 tid=0[{"prefix": "get_command_descriptions"}] 2022-06-17T12:51:07.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc428ebc740 1 -- 172.21.15.36:0/3070405027 --> [v2:172.21.15.70:6832/2723288138,v1:172.21.15.70:6833/2723288138] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x21bfd80 con 0x2265150 2022-06-17T12:51:07.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc417fff700 1 --2- 172.21.15.36:0/3070405027 >> [v2:172.21.15.70:6832/2723288138,v1:172.21.15.70:6833/2723288138] conn(0x2265150 0x2265530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc417fff700 1 --2- 172.21.15.36:0/3070405027 >> [v2:172.21.15.70:6832/2723288138,v1:172.21.15.70:6833/2723288138] conn(0x2265150 0x2265530 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=55f210b9368920e1 server_cookie=70bb1a9854b37576 in_seq=0 out_seq=0 2022-06-17T12:51:07.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc3f77fe700 10 client.5877 ms_handle_connect on v2:172.21.15.70:6832/2723288138 2022-06-17T12:51:07.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc3f77fe700 1 -- 172.21.15.36:0/3070405027 <== mds.0 v2:172.21.15.70:6832/2723288138 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x21bfd80 con 0x2265150 2022-06-17T12:51:07.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc3f77fe700 10 client.5877 handle_command_reply: tid=0 2022-06-17T12:51:07.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc428ebc740 1 client.5877 shutdown 2022-06-17T12:51:07.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc428ebc740 1 -- 172.21.15.36:0/3070405027 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225c630 msgr2=0x22a55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:07.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc428ebc740 1 --2- 172.21.15.36:0/3070405027 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225c630 0x22a55a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fc40c01a000 tx=0x7fc40c016000).stop 2022-06-17T12:51:07.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc428ebc740 1 -- 172.21.15.36:0/3070405027 shutdown_connections 2022-06-17T12:51:07.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc428ebc740 1 --2- 172.21.15.36:0/3070405027 >> [v2:172.21.15.70:6832/2723288138,v1:172.21.15.70:6833/2723288138] conn(0x2265150 0x2265530 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc428ebc740 1 --2- 172.21.15.36:0/3070405027 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2297b70 0x2293170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc428ebc740 1 --2- 172.21.15.36:0/3070405027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x225d610 0x2292c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc428ebc740 1 --2- 172.21.15.36:0/3070405027 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225c630 0x22a55a0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc428ebc740 1 -- 172.21.15.36:0/3070405027 >> 172.21.15.36:0/3070405027 conn(0x21e9540 msgr2=0x2295030 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:51:07.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc3f77fe700 0 client.5877 ms_handle_reset on v2:172.21.15.70:6832/2723288138 2022-06-17T12:51:07.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc428ebc740 1 -- 172.21.15.36:0/3070405027 shutdown_connections 2022-06-17T12:51:07.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc428ebc740 1 -- 172.21.15.36:0/3070405027 wait complete. 2022-06-17T12:51:07.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.693+0000 7fc428ebc740 20 client.5877 trim_cache size 0 max 16384 2022-06-17T12:51:07.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.709+0000 7fc428ebc740 1 Processor -- start 2022-06-17T12:51:07.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.709+0000 7fc428ebc740 1 -- start start 2022-06-17T12:51:07.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc428ebc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x225c630 0x2296ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc428ebc740 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225d610 0x22628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc428ebc740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2297b70 0x2262d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc428ebc740 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x21fe480 con 0x225d610 2022-06-17T12:51:07.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc428ebc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x21bc4b0 con 0x225c630 2022-06-17T12:51:07.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc428ebc740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x2231da0 con 0x2297b70 2022-06-17T12:51:07.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc417fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225d610 0x22628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc417fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225d610 0x22628a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34376/0 (socket says 172.21.15.36:34376) 2022-06-17T12:51:07.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc417fff700 1 -- 172.21.15.36:0/1301159368 learned_addr learned my addr 172.21.15.36:0/1301159368 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:51:07.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc41d204700 1 --2- 172.21.15.36:0/1301159368 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2297b70 0x2262d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc41ca03700 1 --2- 172.21.15.36:0/1301159368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x225c630 0x2296ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc417fff700 1 -- 172.21.15.36:0/1301159368 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2297b70 msgr2=0x2262d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:07.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc417fff700 1 --2- 172.21.15.36:0/1301159368 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2297b70 0x2262d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc417fff700 1 -- 172.21.15.36:0/1301159368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x225c630 msgr2=0x2296ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:07.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc417fff700 1 --2- 172.21.15.36:0/1301159368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x225c630 0x2296ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc417fff700 1 -- 172.21.15.36:0/1301159368 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x21c9ee0 con 0x225d610 2022-06-17T12:51:07.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc41d204700 1 --2- 172.21.15.36:0/1301159368 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2297b70 0x2262d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:51:07.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc417fff700 1 --2- 172.21.15.36:0/1301159368 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225d610 0x22628a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x21d8ba0 tx=0x7fc404009040).ready entity=mon.0 client_cookie=80934b600a28392e server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:07.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc401ffb700 1 -- 172.21.15.36:0/1301159368 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc40401d070 con 0x225d610 2022-06-17T12:51:07.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc401ffb700 1 -- 172.21.15.36:0/1301159368 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4040032c0 con 0x225d610 2022-06-17T12:51:07.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc401ffb700 1 -- 172.21.15.36:0/1301159368 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc404003040 con 0x225d610 2022-06-17T12:51:07.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc428ebc740 1 -- 172.21.15.36:0/1301159368 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225d610 msgr2=0x22628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:07.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc428ebc740 1 --2- 172.21.15.36:0/1301159368 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225d610 0x22628a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x21d8ba0 tx=0x7fc404009040).stop 2022-06-17T12:51:07.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc428ebc740 1 -- 172.21.15.36:0/1301159368 shutdown_connections 2022-06-17T12:51:07.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc428ebc740 1 --2- 172.21.15.36:0/1301159368 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2297b70 0x2262d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc428ebc740 1 --2- 172.21.15.36:0/1301159368 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225d610 0x22628a0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc428ebc740 1 --2- 172.21.15.36:0/1301159368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x225c630 0x2296ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc428ebc740 1 -- 172.21.15.36:0/1301159368 >> 172.21.15.36:0/1301159368 conn(0x21e9540 msgr2=0x22650e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:51:07.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc428ebc740 1 -- 172.21.15.36:0/1301159368 shutdown_connections 2022-06-17T12:51:07.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc428ebc740 1 -- 172.21.15.36:0/1301159368 wait complete. 2022-06-17T12:51:07.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc428ebc740 1 -- start start 2022-06-17T12:51:07.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.713+0000 7fc428ebc740 1 Processor -- start 2022-06-17T12:51:07.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc428ebc740 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225c630 0x2293ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc428ebc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x225d610 0x22b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc428ebc740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2297b70 0x2301200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc428ebc740 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x2300b70 con 0x225c630 2022-06-17T12:51:07.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc428ebc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x2294390 con 0x225d610 2022-06-17T12:51:07.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc428ebc740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x22ffc40 con 0x2297b70 2022-06-17T12:51:07.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc41ca03700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225c630 0x2293ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc428ebc740 10 client.0 ll_register_callbacks cb 0x22a5eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-06-17T12:51:07.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc41ca03700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225c630 0x2293ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34378/0 (socket says 172.21.15.36:34378) 2022-06-17T12:51:07.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc41ca03700 1 -- 172.21.15.36:0/3143019992 learned_addr learned my addr 172.21.15.36:0/3143019992 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:51:07.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc417fff700 1 --2- 172.21.15.36:0/3143019992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x225d610 0x22b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc41d204700 1 --2- 172.21.15.36:0/3143019992 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2297b70 0x2301200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc41ca03700 1 -- 172.21.15.36:0/3143019992 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2297b70 msgr2=0x2301200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:07.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc41ca03700 1 --2- 172.21.15.36:0/3143019992 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2297b70 0x2301200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc41ca03700 1 -- 172.21.15.36:0/3143019992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x225d610 msgr2=0x22b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:07.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc41ca03700 1 --2- 172.21.15.36:0/3143019992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x225d610 0x22b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:07.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc41ca03700 1 -- 172.21.15.36:0/3143019992 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2209720 con 0x225c630 2022-06-17T12:51:07.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc41ca03700 1 --2- 172.21.15.36:0/3143019992 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225c630 0x2293ea0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x2294520 tx=0x7fc40c024000).ready entity=mon.0 client_cookie=f2a961add1f61158 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:07.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc400ff9700 10 client.0 ms_handle_connect on v2:172.21.15.36:3300/0 2022-06-17T12:51:07.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc400ff9700 1 -- 172.21.15.36:0/3143019992 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc40c0277d0 con 0x225c630 2022-06-17T12:51:07.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc400ff9700 1 -- 172.21.15.36:0/3143019992 <== mon.0 v2:172.21.15.36:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc40c0272f0 con 0x225c630 2022-06-17T12:51:07.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc428ebc740 1 -- 172.21.15.36:0/3143019992 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4100460c0 con 0x225c630 2022-06-17T12:51:07.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc400ff9700 1 -- 172.21.15.36:0/3143019992 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc40c027930 con 0x225c630 2022-06-17T12:51:07.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc428ebc740 10 client.5883 fetch_fsmap learned FSMap version 25 2022-06-17T12:51:07.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc428ebc740 1 -- 172.21.15.36:0/3143019992 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x7fc4180b8120 con 0x225c630 2022-06-17T12:51:07.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc400ff9700 1 -- 172.21.15.36:0/3143019992 <== mon.0 v2:172.21.15.36:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4100460c0 con 0x225c630 2022-06-17T12:51:07.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc400ff9700 1 -- 172.21.15.36:0/3143019992 <== mon.0 v2:172.21.15.36:3300/0 5 ==== fsmap(e 25) v1 ==== 1247+0+0 (secure 0 0 0) 0x7fc40c014440 con 0x225c630 2022-06-17T12:51:07.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc428ebc740 10 client.5883 fetch_fsmap finished waiting for FSMap version 25 2022-06-17T12:51:07.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc428ebc740 10 client.5883 resolve_mds: resolved ID 'b' to GID 5679 2022-06-17T12:51:07.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc428ebc740 20 client.5883 populate_metadata read hostname 'smithi036' 2022-06-17T12:51:07.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc428ebc740 1 --2- 172.21.15.36:0/3143019992 >> [v2:172.21.15.70:6832/2723288138,v1:172.21.15.70:6833/2723288138] conn(0x23016f0 0x22f8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:07.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc428ebc740 4 client.5883 mds_command: new command op to 5679 tid=0[{"prefix": "cache drop", "timeout": 5}] 2022-06-17T12:51:07.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.717+0000 7fc428ebc740 1 -- 172.21.15.36:0/3143019992 --> [v2:172.21.15.70:6832/2723288138,v1:172.21.15.70:6833/2723288138] -- command(tid 0: {"prefix": "cache drop", "timeout": 5}) v1 -- 0x20b55e0 con 0x23016f0 2022-06-17T12:51:07.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.721+0000 7fc417fff700 1 --2- 172.21.15.36:0/3143019992 >> [v2:172.21.15.70:6832/2723288138,v1:172.21.15.70:6833/2723288138] conn(0x23016f0 0x22f8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:07.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.721+0000 7fc417fff700 1 --2- 172.21.15.36:0/3143019992 >> [v2:172.21.15.70:6832/2723288138,v1:172.21.15.70:6833/2723288138] conn(0x23016f0 0x22f8670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=763c19c8ee9310a0 server_cookie=3e74c910d0a69547 in_seq=0 out_seq=0 2022-06-17T12:51:07.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:07.721+0000 7fc400ff9700 10 client.5883 ms_handle_connect on v2:172.21.15.70:6832/2723288138 2022-06-17T12:51:12.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc400ff9700 1 -- 172.21.15.36:0/3143019992 <== mds.0 v2:172.21.15.70:6832/2723288138 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+404 (crc 0 0 0) 0x20b55e0 con 0x23016f0 2022-06-17T12:51:12.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc400ff9700 10 client.5883 handle_command_reply: tid=0 2022-06-17T12:51:12.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc428ebc740 1 client.5883 shutdown 2022-06-17T12:51:12.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc428ebc740 1 -- 172.21.15.36:0/3143019992 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225c630 msgr2=0x2293ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:12.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc428ebc740 1 --2- 172.21.15.36:0/3143019992 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225c630 0x2293ea0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x2294520 tx=0x7fc40c024000).stop 2022-06-17T12:51:12.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc428ebc740 1 -- 172.21.15.36:0/3143019992 shutdown_connections 2022-06-17T12:51:12.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc428ebc740 1 --2- 172.21.15.36:0/3143019992 >> [v2:172.21.15.70:6832/2723288138,v1:172.21.15.70:6833/2723288138] conn(0x23016f0 0x22f8670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:12.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc428ebc740 1 --2- 172.21.15.36:0/3143019992 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2297b70 0x2301200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:12.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc428ebc740 1 --2- 172.21.15.36:0/3143019992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x225d610 0x22b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:12.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc428ebc740 1 --2- 172.21.15.36:0/3143019992 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x225c630 0x2293ea0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:12.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc428ebc740 1 -- 172.21.15.36:0/3143019992 >> 172.21.15.36:0/3143019992 conn(0x21e9540 msgr2=0x22650e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:51:12.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc400ff9700 0 client.5883 ms_handle_reset on v2:172.21.15.70:6832/2723288138 2022-06-17T12:51:12.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc428ebc740 1 -- 172.21.15.36:0/3143019992 shutdown_connections 2022-06-17T12:51:12.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc428ebc740 1 -- 172.21.15.36:0/3143019992 wait complete. 2022-06-17T12:51:12.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc428ebc740 20 client.5883 trim_cache size 0 max 16384 2022-06-17T12:51:12.899 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:51:12.900 INFO:teuthology.orchestra.run.smithi036.stdout: "client_recall": { 2022-06-17T12:51:12.900 INFO:teuthology.orchestra.run.smithi036.stdout: "return_code": -110, 2022-06-17T12:51:12.901 INFO:teuthology.orchestra.run.smithi036.stdout: "message": "(110) Connection timed out", 2022-06-17T12:51:12.901 INFO:teuthology.orchestra.run.smithi036.stdout: "recalled": 120 2022-06-17T12:51:12.902 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:51:12.902 INFO:teuthology.orchestra.run.smithi036.stdout: "flush_journal": { 2022-06-17T12:51:12.903 INFO:teuthology.orchestra.run.smithi036.stdout: "return_code": 0, 2022-06-17T12:51:12.903 INFO:teuthology.orchestra.run.smithi036.stdout: "message": "" 2022-06-17T12:51:12.904 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:51:12.904 INFO:teuthology.orchestra.run.smithi036.stdout: "trim_cache": { 2022-06-17T12:51:12.904 INFO:teuthology.orchestra.run.smithi036.stdout: "trimmed": 0 2022-06-17T12:51:12.905 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:51:12.905 INFO:teuthology.orchestra.run.smithi036.stdout: "cache": { 2022-06-17T12:51:12.906 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": { 2022-06-17T12:51:12.907 INFO:teuthology.orchestra.run.smithi036.stdout: "items": 19972, 2022-06-17T12:51:12.907 INFO:teuthology.orchestra.run.smithi036.stdout: "bytes": 3098528 2022-06-17T12:51:12.908 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:51:12.908 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:51:12.908 INFO:teuthology.orchestra.run.smithi036.stdout: "duration": 5.1850052450000002 2022-06-17T12:51:12.909 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:51:12.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc41ec67700 1 -- 172.21.15.36:0/2669035062 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc3f8053a40 msgr2=0x7fc3f8055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:12.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc41ec67700 1 --2- 172.21.15.36:0/2669035062 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc3f8053a40 0x7fc3f8055f00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fc40c00b000 tx=0x7fc40c006df0).stop 2022-06-17T12:51:12.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc41ec67700 1 -- 172.21.15.36:0/2669035062 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc418141050 msgr2=0x7fc418127210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:12.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc41ec67700 1 --2- 172.21.15.36:0/2669035062 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc418141050 0x7fc418127210 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7fc4100032f0 tx=0x7fc410016000).stop 2022-06-17T12:51:12.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc41ec67700 1 -- 172.21.15.36:0/2669035062 shutdown_connections 2022-06-17T12:51:12.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc41ec67700 1 --2- 172.21.15.36:0/2669035062 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc3f8053a40 0x7fc3f8055f00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:12.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc41ec67700 1 --2- 172.21.15.36:0/2669035062 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc418141050 0x7fc418127210 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:12.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc41ec67700 1 --2- 172.21.15.36:0/2669035062 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4180b7d40 0x7fc418126d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:12.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc41ec67700 1 --2- 172.21.15.36:0/2669035062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4180649c0 0x7fc4180b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:12.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc41ec67700 1 -- 172.21.15.36:0/2669035062 >> 172.21.15.36:0/2669035062 conn(0x7fc41805c920 msgr2=0x7fc4180b6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:51:12.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc41ec67700 1 -- 172.21.15.36:0/2669035062 shutdown_connections 2022-06-17T12:51:12.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:12.905+0000 7fc41ec67700 1 -- 172.21.15.36:0/2669035062 wait complete. 2022-06-17T12:51:12.918 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2022-06-17T12:51:12.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:51:12.931 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:51:12.931 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:51:12.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:51:12.960 INFO:teuthology.orchestra.run.smithi107.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2022-06-17T12:51:12.961 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2022-06-17T12:51:12.962 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2022-06-17T12:51:12.962 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:51:12.963 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:51:12.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:51:12.998 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:51:13.007 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:51:13.043 INFO:teuthology.orchestra.run.smithi107.stdout:ext2/ext3 2022-06-17T12:51:13.045 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:51:13.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:51:13.047 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:51:13.058 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:51:13.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.185+0000 7f8b59afc700 1 Processor -- start 2022-06-17T12:51:13.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.185+0000 7f8b59afc700 1 -- start start 2022-06-17T12:51:13.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.185+0000 7f8b59afc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8b540b2200 0x7f8b540af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:13.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.185+0000 7f8b59afc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b540b2cd0 0x7f8b54147600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:13.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.185+0000 7f8b59afc700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b540b25e0 con 0x7f8b540af8d0 2022-06-17T12:51:13.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.185+0000 7f8b59afc700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b540ad790 con 0x7f8b540b2200 2022-06-17T12:51:13.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.185+0000 7f8b59afc700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b540aef40 con 0x7f8b540b2cd0 2022-06-17T12:51:13.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.185+0000 7f8b58afa700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8b540b2200 0x7f8b540af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:13.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.189+0000 7f8b58afa700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8b540b2200 0x7f8b540af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34388/0 (socket says 172.21.15.36:34388) 2022-06-17T12:51:13.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.189+0000 7f8b58afa700 1 -- 172.21.15.36:0/2142123652 learned_addr learned my addr 172.21.15.36:0/2142123652 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:51:13.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.189+0000 7f8b592fb700 1 --2- 172.21.15.36:0/2142123652 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b540b2cd0 0x7f8b54147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:13.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.189+0000 7f8b58afa700 1 -- 172.21.15.36:0/2142123652 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b540b2cd0 msgr2=0x7f8b54147600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:13.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.189+0000 7f8b58afa700 1 --2- 172.21.15.36:0/2142123652 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b540b2cd0 0x7f8b54147600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.189+0000 7f8b537fe700 1 -- 172.21.15.36:0/2142123652 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3900335704 0 0) 0x7f8b540b25e0 con 0x7f8b540af8d0 2022-06-17T12:51:13.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.189+0000 7f8b58afa700 1 -- 172.21.15.36:0/2142123652 >> v1:172.21.15.70:6789/0 conn(0x7f8b540af8d0 legacy=0x7f8b5414a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:13.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.189+0000 7f8b58afa700 1 -- 172.21.15.36:0/2142123652 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b540b1950 con 0x7f8b540b2200 2022-06-17T12:51:13.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.189+0000 7f8b58afa700 1 --2- 172.21.15.36:0/2142123652 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b540b2200 0x7f8b540af3e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f8b48002f70 tx=0x7f8b48006eb0).ready entity=mon.0 client_cookie=bffe85ff3e274c7c server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:13.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.189+0000 7f8b537fe700 1 -- 172.21.15.36:0/2142123652 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b4800d570 con 0x7f8b540b2200 2022-06-17T12:51:13.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.189+0000 7f8b537fe700 1 -- 172.21.15.36:0/2142123652 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b48000e90 con 0x7f8b540b2200 2022-06-17T12:51:13.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.189+0000 7f8b537fe700 1 -- 172.21.15.36:0/2142123652 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b4800d8c0 con 0x7f8b540b2200 2022-06-17T12:51:13.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.189+0000 7f8b59afc700 1 -- 172.21.15.36:0/2142123652 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b540b2200 msgr2=0x7f8b540af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:13.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.189+0000 7f8b59afc700 1 --2- 172.21.15.36:0/2142123652 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b540b2200 0x7f8b540af3e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f8b48002f70 tx=0x7f8b48006eb0).stop 2022-06-17T12:51:13.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.189+0000 7f8b59afc700 1 -- 172.21.15.36:0/2142123652 shutdown_connections 2022-06-17T12:51:13.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.189+0000 7f8b59afc700 1 --2- 172.21.15.36:0/2142123652 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b540b2cd0 0x7f8b54147600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.189+0000 7f8b59afc700 1 --2- 172.21.15.36:0/2142123652 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b540b2200 0x7f8b540af3e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.189+0000 7f8b59afc700 1 -- 172.21.15.36:0/2142123652 >> 172.21.15.36:0/2142123652 conn(0x7f8b5405f770 msgr2=0x7f8b5405fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:51:13.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.189+0000 7f8b59afc700 1 -- 172.21.15.36:0/2142123652 shutdown_connections 2022-06-17T12:51:13.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.189+0000 7f8b59afc700 1 -- 172.21.15.36:0/2142123652 wait complete. 2022-06-17T12:51:13.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b59afc700 1 Processor -- start 2022-06-17T12:51:13.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b59afc700 1 -- start start 2022-06-17T12:51:13.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b59afc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b540af8d0 0x7f8b54144c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:13.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b59afc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b540b2200 0x7f8b54145170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:13.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b59afc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b540b2cd0 0x7f8b540a8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:13.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b59afc700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b540a8070 con 0x7f8b540b2cd0 2022-06-17T12:51:13.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b59afc700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b540a72f0 con 0x7f8b540b2200 2022-06-17T12:51:13.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b59afc700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b541428f0 con 0x7f8b540af8d0 2022-06-17T12:51:13.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b592fb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b540b2cd0 0x7f8b540a8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:13.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b592fb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b540b2cd0 0x7f8b540a8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34396/0 (socket says 172.21.15.36:34396) 2022-06-17T12:51:13.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b592fb700 1 -- 172.21.15.36:0/3640002000 learned_addr learned my addr 172.21.15.36:0/3640002000 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:51:13.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b53fff700 1 --2- 172.21.15.36:0/3640002000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b540b2200 0x7f8b54145170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:13.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b58afa700 1 --2- 172.21.15.36:0/3640002000 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b540af8d0 0x7f8b54144c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:13.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b592fb700 1 -- 172.21.15.36:0/3640002000 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b540af8d0 msgr2=0x7f8b54144c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:13.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b592fb700 1 --2- 172.21.15.36:0/3640002000 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b540af8d0 0x7f8b54144c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b592fb700 1 -- 172.21.15.36:0/3640002000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b540b2200 msgr2=0x7f8b54145170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:13.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b592fb700 1 --2- 172.21.15.36:0/3640002000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b540b2200 0x7f8b54145170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b592fb700 1 -- 172.21.15.36:0/3640002000 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4800b1e0 con 0x7f8b540b2cd0 2022-06-17T12:51:13.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b53fff700 1 --2- 172.21.15.36:0/3640002000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b540b2200 0x7f8b54145170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:51:13.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b592fb700 1 --2- 172.21.15.36:0/3640002000 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b540b2cd0 0x7f8b540a8b50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f8b54143ff0 tx=0x7f8b4c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:13.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b51ffb700 1 -- 172.21.15.36:0/3640002000 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b4c00c7f0 con 0x7f8b540b2cd0 2022-06-17T12:51:13.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b51ffb700 1 -- 172.21.15.36:0/3640002000 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4c00c950 con 0x7f8b540b2cd0 2022-06-17T12:51:13.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b59afc700 1 -- 172.21.15.36:0/3640002000 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b541440d0 con 0x7f8b540b2cd0 2022-06-17T12:51:13.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b51ffb700 1 -- 172.21.15.36:0/3640002000 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b4c00ccb0 con 0x7f8b540b2cd0 2022-06-17T12:51:13.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.193+0000 7f8b59afc700 1 -- 172.21.15.36:0/3640002000 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b540a7720 con 0x7f8b540b2cd0 2022-06-17T12:51:13.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.197+0000 7f8b59afc700 1 -- 172.21.15.36:0/3640002000 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b54144260 con 0x7f8b540b2cd0 2022-06-17T12:51:13.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.197+0000 7f8b51ffb700 1 -- 172.21.15.36:0/3640002000 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8b4c022070 con 0x7f8b540b2cd0 2022-06-17T12:51:13.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.197+0000 7f8b51ffb700 1 --2- 172.21.15.36:0/3640002000 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b34053a40 0x7f8b34055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:13.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.197+0000 7f8b51ffb700 1 -- 172.21.15.36:0/3640002000 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8b4c06c720 con 0x7f8b540b2cd0 2022-06-17T12:51:13.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.197+0000 7f8b58afa700 1 --2- 172.21.15.36:0/3640002000 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b34053a40 0x7f8b34055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:13.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.197+0000 7f8b58afa700 1 --2- 172.21.15.36:0/3640002000 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b34053a40 0x7f8b34055f00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f8b4801a000 tx=0x7f8b48015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:13.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.201+0000 7f8b51ffb700 1 -- 172.21.15.36:0/3640002000 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8b4c043410 con 0x7f8b540b2cd0 2022-06-17T12:51:13.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.329+0000 7f8b59afc700 1 -- 172.21.15.36:0/3640002000 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b540ad330 con 0x7f8b540b2cd0 2022-06-17T12:51:13.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.333+0000 7f8b51ffb700 1 -- 172.21.15.36:0/3640002000 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16273 (secure 0 0 0) 0x7f8b4c016020 con 0x7f8b540b2cd0 2022-06-17T12:51:13.322 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:51:13.322 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":56,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:50:51.108569+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:50:46.705901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-06-17T12:50:47.812736+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:51:13.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.333+0000 7f8b59afc700 1 -- 172.21.15.36:0/3640002000 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b34053a40 msgr2=0x7f8b34055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:13.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.333+0000 7f8b59afc700 1 --2- 172.21.15.36:0/3640002000 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b34053a40 0x7f8b34055f00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f8b4801a000 tx=0x7f8b48015000).stop 2022-06-17T12:51:13.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.333+0000 7f8b59afc700 1 -- 172.21.15.36:0/3640002000 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b540b2cd0 msgr2=0x7f8b540a8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:13.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.333+0000 7f8b59afc700 1 --2- 172.21.15.36:0/3640002000 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b540b2cd0 0x7f8b540a8b50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f8b54143ff0 tx=0x7f8b4c0091d0).stop 2022-06-17T12:51:13.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.333+0000 7f8b59afc700 1 -- 172.21.15.36:0/3640002000 shutdown_connections 2022-06-17T12:51:13.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.333+0000 7f8b59afc700 1 --2- 172.21.15.36:0/3640002000 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b34053a40 0x7f8b34055f00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.333+0000 7f8b59afc700 1 --2- 172.21.15.36:0/3640002000 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b540b2cd0 0x7f8b540a8b50 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.333+0000 7f8b59afc700 1 --2- 172.21.15.36:0/3640002000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b540b2200 0x7f8b54145170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.333+0000 7f8b59afc700 1 --2- 172.21.15.36:0/3640002000 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b540af8d0 0x7f8b54144c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.333+0000 7f8b59afc700 1 -- 172.21.15.36:0/3640002000 >> 172.21.15.36:0/3640002000 conn(0x7f8b5405f770 msgr2=0x7f8b541479b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:51:13.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.333+0000 7f8b59afc700 1 -- 172.21.15.36:0/3640002000 shutdown_connections 2022-06-17T12:51:13.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.333+0000 7f8b59afc700 1 -- 172.21.15.36:0/3640002000 wait complete. 2022-06-17T12:51:13.329 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:50:46.705901+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:50:47.812736+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:51:13.332 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:51:13.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.465+0000 7f83a7b13700 1 Processor -- start 2022-06-17T12:51:13.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.465+0000 7f83a7b13700 1 -- start start 2022-06-17T12:51:13.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a7b13700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f83a00605f0 0x7f83a01436e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:13.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a7b13700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a00af780 con 0x7f83a0066df0 2022-06-17T12:51:13.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a7b13700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a00af240 con 0x7f83a00b17e0 2022-06-17T12:51:13.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a7b13700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83a0067260 con 0x7f83a00605f0 2022-06-17T12:51:13.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a60b0700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f83a0066df0 0x7f83a00ace80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51930/0 (socket says 172.21.15.36:51930) 2022-06-17T12:51:13.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a60b0700 1 -- 172.21.15.36:0/3153355618 learned_addr learned my addr 172.21.15.36:0/3153355618 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:51:13.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a58af700 1 --2- 172.21.15.36:0/3153355618 >> v2:172.21.15.70:3300/0 conn(0x7f83a00605f0 0x7f83a01436e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:13.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a48ad700 1 -- 172.21.15.36:0/3153355618 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3696701365 0 0) 0x7f83a00af780 con 0x7f83a0066df0 2022-06-17T12:51:13.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a48ad700 1 -- 172.21.15.36:0/3153355618 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8388003260 con 0x7f83a0066df0 2022-06-17T12:51:13.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a48ad700 1 -- 172.21.15.36:0/3153355618 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4143109124 0 0) 0x7f8388003260 con 0x7f83a0066df0 2022-06-17T12:51:13.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a48ad700 1 -- 172.21.15.36:0/3153355618 >> v2:172.21.15.70:3300/0 conn(0x7f83a00605f0 msgr2=0x7f83a01436e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:13.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a48ad700 1 --2- 172.21.15.36:0/3153355618 >> v2:172.21.15.70:3300/0 conn(0x7f83a00605f0 0x7f83a01436e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a48ad700 1 -- 172.21.15.36:0/3153355618 >> v1:172.21.15.70:6789/0 conn(0x7f83a00b17e0 legacy=0x7f83a00666e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:13.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a48ad700 1 -- 172.21.15.36:0/3153355618 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a00af4e0 con 0x7f83a0066df0 2022-06-17T12:51:13.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a48ad700 1 -- 172.21.15.36:0/3153355618 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f8398001a00 con 0x7f83a0066df0 2022-06-17T12:51:13.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a48ad700 1 -- 172.21.15.36:0/3153355618 >> v1:172.21.15.36:6789/0 conn(0x7f83a0066df0 legacy=0x7f83a00ace80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:13.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a48ad700 1 --2- 172.21.15.36:0/3153355618 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8388003750 0x7f8388003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:13.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a48ad700 1 --2- 172.21.15.36:0/3153355618 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8388004500 0x7f8388004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:13.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a48ad700 1 --2- 172.21.15.36:0/3153355618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8388004ec0 0x7f838800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:13.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a48ad700 1 -- 172.21.15.36:0/3153355618 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a0067260 con 0x7f8388003750 2022-06-17T12:51:13.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a48ad700 1 -- 172.21.15.36:0/3153355618 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838800bda0 con 0x7f8388004ec0 2022-06-17T12:51:13.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a48ad700 1 -- 172.21.15.36:0/3153355618 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838800e170 con 0x7f8388004500 2022-06-17T12:51:13.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a58af700 1 --2- 172.21.15.36:0/3153355618 >> v2:172.21.15.70:3300/0 conn(0x7f83a00605f0 0x7f83a01436e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:51:13.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a58af700 1 --2- 172.21.15.36:0/3153355618 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8388003750 0x7f8388003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:13.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a60b0700 1 --2- 172.21.15.36:0/3153355618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8388004ec0 0x7f838800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:13.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a50ae700 1 --2- 172.21.15.36:0/3153355618 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8388004500 0x7f8388004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:13.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a58af700 1 -- 172.21.15.36:0/3153355618 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8388004500 msgr2=0x7f8388004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:13.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a58af700 1 --2- 172.21.15.36:0/3153355618 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8388004500 0x7f8388004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a58af700 1 -- 172.21.15.36:0/3153355618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8388004ec0 msgr2=0x7f838800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:13.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a58af700 1 --2- 172.21.15.36:0/3153355618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8388004ec0 0x7f838800b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a58af700 1 -- 172.21.15.36:0/3153355618 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8388003260 con 0x7f8388003750 2022-06-17T12:51:13.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a60b0700 1 --2- 172.21.15.36:0/3153355618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8388004ec0 0x7f838800b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:51:13.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a60b0700 1 -- 172.21.15.36:0/3153355618 reap_dead start 2022-06-17T12:51:13.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a58af700 1 --2- 172.21.15.36:0/3153355618 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8388003750 0x7f8388003b70 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f839c00a000 tx=0x7f839c008040).ready entity=mon.0 client_cookie=773648b7bce8efb7 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:13.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a48ad700 1 -- 172.21.15.36:0/3153355618 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f839c012610 con 0x7f8388003750 2022-06-17T12:51:13.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a48ad700 1 -- 172.21.15.36:0/3153355618 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839c012770 con 0x7f8388003750 2022-06-17T12:51:13.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a48ad700 1 -- 172.21.15.36:0/3153355618 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f839c012ad0 con 0x7f8388003750 2022-06-17T12:51:13.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a7b13700 1 -- 172.21.15.36:0/3153355618 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8388003750 msgr2=0x7f8388003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:13.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a7b13700 1 --2- 172.21.15.36:0/3153355618 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8388003750 0x7f8388003b70 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f839c00a000 tx=0x7f839c008040).stop 2022-06-17T12:51:13.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a7b13700 1 -- 172.21.15.36:0/3153355618 shutdown_connections 2022-06-17T12:51:13.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a7b13700 1 --2- 172.21.15.36:0/3153355618 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8388003750 0x7f8388003b70 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a7b13700 1 -- 172.21.15.36:0/3153355618 >> 172.21.15.36:0/3153355618 conn(0x7f83a005c550 msgr2=0x7f83a00b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:51:13.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a7b13700 1 -- 172.21.15.36:0/3153355618 shutdown_connections 2022-06-17T12:51:13.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.469+0000 7f83a7b13700 1 -- 172.21.15.36:0/3153355618 wait complete. 2022-06-17T12:51:13.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f83a7b13700 1 Processor -- start 2022-06-17T12:51:13.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f83a7b13700 1 -- start start 2022-06-17T12:51:13.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f83a7b13700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8388003750 0x7f83a00641b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:13.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f83a7b13700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f83a00ab590 0x7f83a00b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:13.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f83a7b13700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83a013acd0 0x7f83a013b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:13.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f83a7b13700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a013b640 con 0x7f8388003750 2022-06-17T12:51:13.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f83a7b13700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a0065c80 con 0x7f83a013acd0 2022-06-17T12:51:13.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f83a7b13700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a00654e0 con 0x7f83a00ab590 2022-06-17T12:51:13.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f83a58af700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8388003750 0x7f83a00641b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:13.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f83a50ae700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f83a00ab590 0x7f83a00b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:13.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f83a58af700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8388003750 0x7f83a00641b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34416/0 (socket says 172.21.15.36:34416) 2022-06-17T12:51:13.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f83a58af700 1 -- 172.21.15.36:0/2626796580 learned_addr learned my addr 172.21.15.36:0/2626796580 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:51:13.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f83a60b0700 1 --2- 172.21.15.36:0/2626796580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83a013acd0 0x7f83a013b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:13.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f83a58af700 1 -- 172.21.15.36:0/2626796580 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f83a00ab590 msgr2=0x7f83a00b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:13.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f83a58af700 1 --2- 172.21.15.36:0/2626796580 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f83a00ab590 0x7f83a00b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f83a58af700 1 -- 172.21.15.36:0/2626796580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83a013acd0 msgr2=0x7f83a013b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:13.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f83a58af700 1 --2- 172.21.15.36:0/2626796580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83a013acd0 0x7f83a013b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f83a58af700 1 -- 172.21.15.36:0/2626796580 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a0065780 con 0x7f8388003750 2022-06-17T12:51:13.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f83a60b0700 1 --2- 172.21.15.36:0/2626796580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83a013acd0 0x7f83a013b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:51:13.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f83a58af700 1 --2- 172.21.15.36:0/2626796580 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8388003750 0x7f83a00641b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f839c00a000 tx=0x7f839c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:13.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f8392ffd700 1 -- 172.21.15.36:0/2626796580 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f839c006440 con 0x7f8388003750 2022-06-17T12:51:13.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f8392ffd700 1 -- 172.21.15.36:0/2626796580 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839c000e90 con 0x7f8388003750 2022-06-17T12:51:13.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f83a7b13700 1 -- 172.21.15.36:0/2626796580 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a00635c0 con 0x7f8388003750 2022-06-17T12:51:13.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f8392ffd700 1 -- 172.21.15.36:0/2626796580 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f839c0056f0 con 0x7f8388003750 2022-06-17T12:51:13.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.473+0000 7f83a7b13700 1 -- 172.21.15.36:0/2626796580 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a0063a60 con 0x7f8388003750 2022-06-17T12:51:13.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.477+0000 7f83a7b13700 1 -- 172.21.15.36:0/2626796580 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a0063750 con 0x7f8388003750 2022-06-17T12:51:13.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.477+0000 7f8392ffd700 1 -- 172.21.15.36:0/2626796580 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f839c005850 con 0x7f8388003750 2022-06-17T12:51:13.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.477+0000 7f8392ffd700 1 --2- 172.21.15.36:0/2626796580 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8380053af0 0x7f8380055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:13.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.477+0000 7f8392ffd700 1 -- 172.21.15.36:0/2626796580 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f839c06d6a0 con 0x7f8388003750 2022-06-17T12:51:13.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.477+0000 7f83a50ae700 1 --2- 172.21.15.36:0/2626796580 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8380053af0 0x7f8380055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:13.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.477+0000 7f83a50ae700 1 --2- 172.21.15.36:0/2626796580 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8380053af0 0x7f8380055fb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f839400d000 tx=0x7f8394009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:13.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.485+0000 7f8392ffd700 1 -- 172.21.15.36:0/2626796580 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f839c0443a0 con 0x7f8388003750 2022-06-17T12:51:13.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.609+0000 7f83a7b13700 1 -- 172.21.15.36:0/2626796580 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f83a0057410 con 0x7f8388003750 2022-06-17T12:51:13.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.613+0000 7f8392ffd700 1 -- 172.21.15.36:0/2626796580 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v56) v1 ==== 110+0+10 (secure 0 0 0) 0x7f839c0479c0 con 0x7f8388003750 2022-06-17T12:51:13.596 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:51:13.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.613+0000 7f83a7b13700 1 -- 172.21.15.36:0/2626796580 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8380053af0 msgr2=0x7f8380055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:13.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.613+0000 7f83a7b13700 1 --2- 172.21.15.36:0/2626796580 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8380053af0 0x7f8380055fb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f839400d000 tx=0x7f8394009040).stop 2022-06-17T12:51:13.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.613+0000 7f83a7b13700 1 -- 172.21.15.36:0/2626796580 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8388003750 msgr2=0x7f83a00641b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:13.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.613+0000 7f83a7b13700 1 --2- 172.21.15.36:0/2626796580 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8388003750 0x7f83a00641b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f839c00a000 tx=0x7f839c008040).stop 2022-06-17T12:51:13.602 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.613+0000 7f83a7b13700 1 -- 172.21.15.36:0/2626796580 shutdown_connections 2022-06-17T12:51:13.602 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.613+0000 7f83a7b13700 1 --2- 172.21.15.36:0/2626796580 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8380053af0 0x7f8380055fb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.602 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.613+0000 7f83a7b13700 1 --2- 172.21.15.36:0/2626796580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83a013acd0 0x7f83a013b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.602 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.613+0000 7f83a7b13700 1 --2- 172.21.15.36:0/2626796580 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f83a00ab590 0x7f83a00b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.613+0000 7f83a7b13700 1 --2- 172.21.15.36:0/2626796580 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8388003750 0x7f83a00641b0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.613+0000 7f83a7b13700 1 -- 172.21.15.36:0/2626796580 >> 172.21.15.36:0/2626796580 conn(0x7f83a005c550 msgr2=0x7f83a00666b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:51:13.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.613+0000 7f83a7b13700 1 -- 172.21.15.36:0/2626796580 shutdown_connections 2022-06-17T12:51:13.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.613+0000 7f83a7b13700 1 -- 172.21.15.36:0/2626796580 wait complete. 2022-06-17T12:51:13.612 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:51:13.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.745+0000 7f43c261e700 1 Processor -- start 2022-06-17T12:51:13.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.745+0000 7f43c261e700 1 -- start start 2022-06-17T12:51:13.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.745+0000 7f43c261e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f43bc0b2350 0x7f43bc0b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:13.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.745+0000 7f43c261e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43bc0aa170 con 0x7f43bc0b2d80 2022-06-17T12:51:13.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.745+0000 7f43c261e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43bc0a9c30 con 0x7f43bc143e60 2022-06-17T12:51:13.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.745+0000 7f43c261e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc0a9ed0 con 0x7f43bc0b2350 2022-06-17T12:51:13.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.745+0000 7f43bbfff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f43bc0b2d80 0x7f43bc143750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51950/0 (socket says 172.21.15.36:51950) 2022-06-17T12:51:13.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.745+0000 7f43bbfff700 1 -- 172.21.15.36:0/2423274383 learned_addr learned my addr 172.21.15.36:0/2423274383 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:51:13.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.745+0000 7f43bb7fe700 1 --2- 172.21.15.36:0/2423274383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f43bc0b2350 0x7f43bc0b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:13.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.745+0000 7f43baffd700 1 -- 172.21.15.36:0/2423274383 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 89469587 0 0) 0x7f43bc0aa170 con 0x7f43bc0b2d80 2022-06-17T12:51:13.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.749+0000 7f43baffd700 1 -- 172.21.15.36:0/2423274383 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43a8003260 con 0x7f43bc0b2d80 2022-06-17T12:51:13.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.749+0000 7f43bb7fe700 1 -- 172.21.15.36:0/2423274383 >> v1:172.21.15.70:6789/0 conn(0x7f43bc143e60 legacy=0x7f43bc140300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:13.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.749+0000 7f43bb7fe700 1 -- 172.21.15.36:0/2423274383 >> v1:172.21.15.36:6789/0 conn(0x7f43bc0b2d80 legacy=0x7f43bc143750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:13.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.749+0000 7f43bb7fe700 1 -- 172.21.15.36:0/2423274383 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43bc0b08c0 con 0x7f43bc0b2350 2022-06-17T12:51:13.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.749+0000 7f43bb7fe700 1 --2- 172.21.15.36:0/2423274383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f43bc0b2350 0x7f43bc0b3160 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f43b0002f70 tx=0x7f43b0006df0).ready entity=mon.2 client_cookie=494f720cd854aee8 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:13.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.749+0000 7f43baffd700 1 -- 172.21.15.36:0/2423274383 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43b000d570 con 0x7f43bc0b2350 2022-06-17T12:51:13.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.749+0000 7f43baffd700 1 -- 172.21.15.36:0/2423274383 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b0000e90 con 0x7f43bc0b2350 2022-06-17T12:51:13.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.749+0000 7f43baffd700 1 -- 172.21.15.36:0/2423274383 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43b000d8c0 con 0x7f43bc0b2350 2022-06-17T12:51:13.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.749+0000 7f43c261e700 1 -- 172.21.15.36:0/2423274383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f43bc0b2350 msgr2=0x7f43bc0b3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:13.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.749+0000 7f43c261e700 1 --2- 172.21.15.36:0/2423274383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f43bc0b2350 0x7f43bc0b3160 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f43b0002f70 tx=0x7f43b0006df0).stop 2022-06-17T12:51:13.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.749+0000 7f43c261e700 1 -- 172.21.15.36:0/2423274383 shutdown_connections 2022-06-17T12:51:13.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.749+0000 7f43c261e700 1 --2- 172.21.15.36:0/2423274383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f43bc0b2350 0x7f43bc0b3160 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.749+0000 7f43c261e700 1 -- 172.21.15.36:0/2423274383 >> 172.21.15.36:0/2423274383 conn(0x7f43bc05c550 msgr2=0x7f43bc05c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:51:13.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.749+0000 7f43c261e700 1 -- 172.21.15.36:0/2423274383 shutdown_connections 2022-06-17T12:51:13.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.749+0000 7f43c261e700 1 -- 172.21.15.36:0/2423274383 wait complete. 2022-06-17T12:51:13.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.753+0000 7f43c261e700 1 Processor -- start 2022-06-17T12:51:13.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.753+0000 7f43c261e700 1 -- start start 2022-06-17T12:51:13.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.753+0000 7f43c261e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43bc0b2350 0x7f43bc1399d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:13.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.753+0000 7f43c261e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f43bc0b2d80 0x7f43bc0648c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:13.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.753+0000 7f43c261e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f43bc143e60 0x7f43bc13d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:13.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.753+0000 7f43c261e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc064fc0 con 0x7f43bc143e60 2022-06-17T12:51:13.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.753+0000 7f43c261e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc0661e0 con 0x7f43bc0b2350 2022-06-17T12:51:13.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.753+0000 7f43c261e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc066480 con 0x7f43bc0b2d80 2022-06-17T12:51:13.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.753+0000 7f43c0bbb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f43bc143e60 0x7f43bc13d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:13.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.753+0000 7f43bb7fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f43bc0b2d80 0x7f43bc0648c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:13.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.753+0000 7f43bb7fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f43bc0b2d80 0x7f43bc0648c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:49218/0 (socket says 172.21.15.36:49218) 2022-06-17T12:51:13.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.753+0000 7f43bb7fe700 1 -- 172.21.15.36:0/1335792403 learned_addr learned my addr 172.21.15.36:0/1335792403 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:51:13.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.753+0000 7f43bbfff700 1 --2- 172.21.15.36:0/1335792403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43bc0b2350 0x7f43bc1399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:13.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.753+0000 7f43c0bbb700 1 -- 172.21.15.36:0/1335792403 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f43bc0b2d80 msgr2=0x7f43bc0648c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:13.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.753+0000 7f43c0bbb700 1 --2- 172.21.15.36:0/1335792403 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f43bc0b2d80 0x7f43bc0648c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.753+0000 7f43c0bbb700 1 -- 172.21.15.36:0/1335792403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43bc0b2350 msgr2=0x7f43bc1399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:13.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.753+0000 7f43c0bbb700 1 --2- 172.21.15.36:0/1335792403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43bc0b2350 0x7f43bc1399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.753+0000 7f43c0bbb700 1 -- 172.21.15.36:0/1335792403 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b000b180 con 0x7f43bc143e60 2022-06-17T12:51:13.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.753+0000 7f43c0bbb700 1 --2- 172.21.15.36:0/1335792403 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f43bc143e60 0x7f43bc13d230 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f43bc0633f0 tx=0x7f43b4007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:13.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.753+0000 7f43b97fa700 1 -- 172.21.15.36:0/1335792403 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43b400ce10 con 0x7f43bc143e60 2022-06-17T12:51:13.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.753+0000 7f43b97fa700 1 -- 172.21.15.36:0/1335792403 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b4006aa0 con 0x7f43bc143e60 2022-06-17T12:51:13.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.753+0000 7f43c261e700 1 -- 172.21.15.36:0/1335792403 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43bc066720 con 0x7f43bc143e60 2022-06-17T12:51:13.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.757+0000 7f43b97fa700 1 -- 172.21.15.36:0/1335792403 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43b4006e00 con 0x7f43bc143e60 2022-06-17T12:51:13.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.757+0000 7f43c261e700 1 -- 172.21.15.36:0/1335792403 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43bc0659a0 con 0x7f43bc143e60 2022-06-17T12:51:13.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.757+0000 7f43b97fa700 1 -- 172.21.15.36:0/1335792403 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f43b4010030 con 0x7f43bc143e60 2022-06-17T12:51:13.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.757+0000 7f43b97fa700 1 --2- 172.21.15.36:0/1335792403 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f439c053a40 0x7f439c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:13.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.757+0000 7f43b97fa700 1 -- 172.21.15.36:0/1335792403 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f43b406bce0 con 0x7f43bc143e60 2022-06-17T12:51:13.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.757+0000 7f43bbfff700 1 --2- 172.21.15.36:0/1335792403 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f439c053a40 0x7f439c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:13.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.757+0000 7f43c261e700 1 -- 172.21.15.36:0/1335792403 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43bc0668b0 con 0x7f43bc143e60 2022-06-17T12:51:13.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.757+0000 7f43bbfff700 1 --2- 172.21.15.36:0/1335792403 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f439c053a40 0x7f439c055f00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f43ac00d000 tx=0x7f43ac009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:13.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.765+0000 7f43b97fa700 1 -- 172.21.15.36:0/1335792403 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f43b4042ae0 con 0x7f43bc143e60 2022-06-17T12:51:13.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.897+0000 7f43c261e700 1 -- 172.21.15.36:0/1335792403 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f43bc1442f0 con 0x7f43bc143e60 2022-06-17T12:51:13.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.897+0000 7f43b97fa700 1 -- 172.21.15.36:0/1335792403 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v56) v1 ==== 92+0+10 (secure 0 0 0) 0x7f43b4046100 con 0x7f43bc143e60 2022-06-17T12:51:13.882 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:51:13.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.897+0000 7f43c261e700 1 -- 172.21.15.36:0/1335792403 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f439c053a40 msgr2=0x7f439c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:13.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.897+0000 7f43c261e700 1 --2- 172.21.15.36:0/1335792403 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f439c053a40 0x7f439c055f00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f43ac00d000 tx=0x7f43ac009040).stop 2022-06-17T12:51:13.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.897+0000 7f43c261e700 1 -- 172.21.15.36:0/1335792403 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f43bc143e60 msgr2=0x7f43bc13d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:13.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.897+0000 7f43c261e700 1 --2- 172.21.15.36:0/1335792403 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f43bc143e60 0x7f43bc13d230 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f43bc0633f0 tx=0x7f43b4007f10).stop 2022-06-17T12:51:13.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.897+0000 7f43c261e700 1 -- 172.21.15.36:0/1335792403 shutdown_connections 2022-06-17T12:51:13.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.897+0000 7f43c261e700 1 --2- 172.21.15.36:0/1335792403 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f439c053a40 0x7f439c055f00 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.897+0000 7f43c261e700 1 --2- 172.21.15.36:0/1335792403 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f43bc143e60 0x7f43bc13d230 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.897+0000 7f43c261e700 1 --2- 172.21.15.36:0/1335792403 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f43bc0b2d80 0x7f43bc0648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.897+0000 7f43c261e700 1 --2- 172.21.15.36:0/1335792403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43bc0b2350 0x7f43bc1399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:13.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.897+0000 7f43c261e700 1 -- 172.21.15.36:0/1335792403 >> 172.21.15.36:0/1335792403 conn(0x7f43bc05c550 msgr2=0x7f43bc140650 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:51:13.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.897+0000 7f43c261e700 1 -- 172.21.15.36:0/1335792403 shutdown_connections 2022-06-17T12:51:13.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:13.897+0000 7f43c261e700 1 -- 172.21.15.36:0/1335792403 wait complete. 2022-06-17T12:51:13.902 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:51:14.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.037+0000 7f8f22b0b700 1 Processor -- start 2022-06-17T12:51:14.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.037+0000 7f8f22b0b700 1 -- start start 2022-06-17T12:51:14.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.037+0000 7f8f22b0b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8f1c0b27b0 0x7f8f1c14b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:14.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.037+0000 7f8f22b0b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f1c060ca0 0x7f8f1c0ac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:14.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.037+0000 7f8f22b0b700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f1c14b870 con 0x7f8f1c147bd0 2022-06-17T12:51:14.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.037+0000 7f8f22b0b700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f1c0ac500 con 0x7f8f1c0b27b0 2022-06-17T12:51:14.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.037+0000 7f8f22b0b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c0b2b90 con 0x7f8f1c060ca0 2022-06-17T12:51:14.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.037+0000 7f8f2230a700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f8f1c147bd0 0x7f8f1c1442d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51968/0 (socket says 172.21.15.36:51968) 2022-06-17T12:51:14.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.037+0000 7f8f21b09700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8f1c0b27b0 0x7f8f1c14b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:14.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.037+0000 7f8f2230a700 1 -- 172.21.15.36:0/1266032371 learned_addr learned my addr 172.21.15.36:0/1266032371 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:51:14.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.037+0000 7f8f21308700 1 --2- 172.21.15.36:0/1266032371 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f1c060ca0 0x7f8f1c0ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:14.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.037+0000 7f8f21b09700 1 -- 172.21.15.36:0/1266032371 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f1c060ca0 msgr2=0x7f8f1c0ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:14.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.037+0000 7f8f21b09700 1 --2- 172.21.15.36:0/1266032371 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f1c060ca0 0x7f8f1c0ac780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.037+0000 7f8f21b09700 1 -- 172.21.15.36:0/1266032371 >> v1:172.21.15.36:6789/0 conn(0x7f8f1c147bd0 legacy=0x7f8f1c1442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:14.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.037+0000 7f8f21b09700 1 -- 172.21.15.36:0/1266032371 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1c147490 con 0x7f8f1c0b27b0 2022-06-17T12:51:14.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.037+0000 7f8f20b07700 1 -- 172.21.15.36:0/1266032371 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2597487914 0 0) 0x7f8f1c14b870 con 0x7f8f1c147bd0 2022-06-17T12:51:14.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.037+0000 7f8f21b09700 1 --2- 172.21.15.36:0/1266032371 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f1c0b27b0 0x7f8f1c14b2b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f8f18002f70 tx=0x7f8f18006df0).ready entity=mon.0 client_cookie=8f513fac81140bea server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:14.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.037+0000 7f8f20b07700 1 -- 172.21.15.36:0/1266032371 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f1800d570 con 0x7f8f1c0b27b0 2022-06-17T12:51:14.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.037+0000 7f8f20b07700 1 -- 172.21.15.36:0/1266032371 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f18000e90 con 0x7f8f1c0b27b0 2022-06-17T12:51:14.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.037+0000 7f8f20b07700 1 -- 172.21.15.36:0/1266032371 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f1800d8c0 con 0x7f8f1c0b27b0 2022-06-17T12:51:14.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.041+0000 7f8f22b0b700 1 -- 172.21.15.36:0/1266032371 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f1c0b27b0 msgr2=0x7f8f1c14b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:14.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.041+0000 7f8f22b0b700 1 --2- 172.21.15.36:0/1266032371 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f1c0b27b0 0x7f8f1c14b2b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f8f18002f70 tx=0x7f8f18006df0).stop 2022-06-17T12:51:14.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.041+0000 7f8f22b0b700 1 -- 172.21.15.36:0/1266032371 shutdown_connections 2022-06-17T12:51:14.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.041+0000 7f8f22b0b700 1 --2- 172.21.15.36:0/1266032371 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f1c060ca0 0x7f8f1c0ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.041+0000 7f8f22b0b700 1 --2- 172.21.15.36:0/1266032371 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f1c0b27b0 0x7f8f1c14b2b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.041+0000 7f8f22b0b700 1 -- 172.21.15.36:0/1266032371 >> 172.21.15.36:0/1266032371 conn(0x7f8f1c0b5d10 msgr2=0x7f8f1c0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:51:14.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.041+0000 7f8f22b0b700 1 -- 172.21.15.36:0/1266032371 shutdown_connections 2022-06-17T12:51:14.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.041+0000 7f8f22b0b700 1 -- 172.21.15.36:0/1266032371 wait complete. 2022-06-17T12:51:14.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.041+0000 7f8f22b0b700 1 Processor -- start 2022-06-17T12:51:14.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.041+0000 7f8f22b0b700 1 -- start start 2022-06-17T12:51:14.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.041+0000 7f8f22b0b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f1c060ca0 0x7f8f1c14e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:14.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.041+0000 7f8f22b0b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f1c0b27b0 0x7f8f1c156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:14.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.041+0000 7f8f22b0b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f1c147bd0 0x7f8f1c156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:14.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.041+0000 7f8f22b0b700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c0a87d0 con 0x7f8f1c060ca0 2022-06-17T12:51:14.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.041+0000 7f8f22b0b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c157340 con 0x7f8f1c147bd0 2022-06-17T12:51:14.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.041+0000 7f8f22b0b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c0a8530 con 0x7f8f1c0b27b0 2022-06-17T12:51:14.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.045+0000 7f8f21b09700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f1c060ca0 0x7f8f1c14e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:14.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.045+0000 7f8f21308700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f1c0b27b0 0x7f8f1c156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:14.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.045+0000 7f8f21b09700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f1c060ca0 0x7f8f1c14e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34440/0 (socket says 172.21.15.36:34440) 2022-06-17T12:51:14.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.045+0000 7f8f21b09700 1 -- 172.21.15.36:0/1517227322 learned_addr learned my addr 172.21.15.36:0/1517227322 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:51:14.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.045+0000 7f8f2230a700 1 --2- 172.21.15.36:0/1517227322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f1c147bd0 0x7f8f1c156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:14.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.045+0000 7f8f21b09700 1 -- 172.21.15.36:0/1517227322 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f1c0b27b0 msgr2=0x7f8f1c156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:14.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.045+0000 7f8f21b09700 1 --2- 172.21.15.36:0/1517227322 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f1c0b27b0 0x7f8f1c156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.045+0000 7f8f21b09700 1 -- 172.21.15.36:0/1517227322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f1c147bd0 msgr2=0x7f8f1c156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:14.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.045+0000 7f8f21b09700 1 --2- 172.21.15.36:0/1517227322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f1c147bd0 0x7f8f1c156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.045+0000 7f8f21b09700 1 -- 172.21.15.36:0/1517227322 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1800b180 con 0x7f8f1c060ca0 2022-06-17T12:51:14.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.045+0000 7f8f21308700 1 --2- 172.21.15.36:0/1517227322 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f1c0b27b0 0x7f8f1c156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:51:14.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.045+0000 7f8f21b09700 1 --2- 172.21.15.36:0/1517227322 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f1c060ca0 0x7f8f1c14e3c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f8f18002f70 tx=0x7f8f18014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:14.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.045+0000 7f8f0affd700 1 -- 172.21.15.36:0/1517227322 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f18016070 con 0x7f8f1c060ca0 2022-06-17T12:51:14.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.045+0000 7f8f0affd700 1 -- 172.21.15.36:0/1517227322 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f180138d0 con 0x7f8f1c060ca0 2022-06-17T12:51:14.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.045+0000 7f8f22b0b700 1 -- 172.21.15.36:0/1517227322 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f1c0644b0 con 0x7f8f1c060ca0 2022-06-17T12:51:14.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.045+0000 7f8f0affd700 1 -- 172.21.15.36:0/1517227322 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f18013c30 con 0x7f8f1c060ca0 2022-06-17T12:51:14.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.045+0000 7f8f22b0b700 1 -- 172.21.15.36:0/1517227322 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f1c156340 con 0x7f8f1c060ca0 2022-06-17T12:51:14.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.045+0000 7f8f22b0b700 1 -- 172.21.15.36:0/1517227322 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f1c159910 con 0x7f8f1c060ca0 2022-06-17T12:51:14.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.045+0000 7f8f0affd700 1 -- 172.21.15.36:0/1517227322 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8f18025030 con 0x7f8f1c060ca0 2022-06-17T12:51:14.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.049+0000 7f8f0affd700 1 --2- 172.21.15.36:0/1517227322 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8efc053a40 0x7f8efc055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:14.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.049+0000 7f8f0affd700 1 -- 172.21.15.36:0/1517227322 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8f1806ce00 con 0x7f8f1c060ca0 2022-06-17T12:51:14.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.049+0000 7f8f21308700 1 --2- 172.21.15.36:0/1517227322 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8efc053a40 0x7f8efc055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:14.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.049+0000 7f8f21308700 1 --2- 172.21.15.36:0/1517227322 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8efc053a40 0x7f8efc055f00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f8f0c00b000 tx=0x7f8f0c006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:14.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.053+0000 7f8f0affd700 1 -- 172.21.15.36:0/1517227322 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8f18043c00 con 0x7f8f1c060ca0 2022-06-17T12:51:14.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.177+0000 7f8f22b0b700 1 -- 172.21.15.36:0/1517227322 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8f1c057410 con 0x7f8f1c060ca0 2022-06-17T12:51:14.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.177+0000 7f8f0affd700 1 -- 172.21.15.36:0/1517227322 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+10 (secure 0 0 0) 0x7f8f18023020 con 0x7f8f1c060ca0 2022-06-17T12:51:14.161 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:51:14.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.177+0000 7f8f22b0b700 1 -- 172.21.15.36:0/1517227322 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8efc053a40 msgr2=0x7f8efc055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:14.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.177+0000 7f8f22b0b700 1 --2- 172.21.15.36:0/1517227322 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8efc053a40 0x7f8efc055f00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f8f0c00b000 tx=0x7f8f0c006df0).stop 2022-06-17T12:51:14.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.177+0000 7f8f22b0b700 1 -- 172.21.15.36:0/1517227322 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f1c060ca0 msgr2=0x7f8f1c14e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:14.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.177+0000 7f8f22b0b700 1 --2- 172.21.15.36:0/1517227322 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f1c060ca0 0x7f8f1c14e3c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f8f18002f70 tx=0x7f8f18014000).stop 2022-06-17T12:51:14.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.177+0000 7f8f22b0b700 1 -- 172.21.15.36:0/1517227322 shutdown_connections 2022-06-17T12:51:14.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.177+0000 7f8f22b0b700 1 --2- 172.21.15.36:0/1517227322 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8efc053a40 0x7f8efc055f00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.177+0000 7f8f22b0b700 1 --2- 172.21.15.36:0/1517227322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f1c147bd0 0x7f8f1c156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.177+0000 7f8f22b0b700 1 --2- 172.21.15.36:0/1517227322 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f1c0b27b0 0x7f8f1c156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.177+0000 7f8f22b0b700 1 --2- 172.21.15.36:0/1517227322 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f1c060ca0 0x7f8f1c14e3c0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.177+0000 7f8f22b0b700 1 -- 172.21.15.36:0/1517227322 >> 172.21.15.36:0/1517227322 conn(0x7f8f1c0b5d10 msgr2=0x7f8f1c0accf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:51:14.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.177+0000 7f8f22b0b700 1 -- 172.21.15.36:0/1517227322 shutdown_connections 2022-06-17T12:51:14.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.177+0000 7f8f22b0b700 1 -- 172.21.15.36:0/1517227322 wait complete. 2022-06-17T12:51:14.182 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:51:14.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa724a84700 1 Processor -- start 2022-06-17T12:51:14.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa724a84700 1 -- start start 2022-06-17T12:51:14.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa724a84700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa7200b45c0 0x7fa7200b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:14.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa724a84700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa7200b4f00 0x7fa7200b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:14.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa724a84700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7200b4160 con 0x7fa7200b9860 2022-06-17T12:51:14.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa724a84700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7200b9c40 con 0x7fa7200b4f00 2022-06-17T12:51:14.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa724a84700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7200627d0 con 0x7fa7200b45c0 2022-06-17T12:51:14.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa71f7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fa7200b9860 0x7fa720143840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51978/0 (socket says 172.21.15.36:51978) 2022-06-17T12:51:14.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa71f7fe700 1 -- 172.21.15.36:0/3478273079 learned_addr learned my addr 172.21.15.36:0/3478273079 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:51:14.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa71ffff700 1 --2- 172.21.15.36:0/3478273079 >> v2:172.21.15.36:3300/0 conn(0x7fa7200b4f00 0x7fa7200b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:14.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa716ffd700 1 --2- 172.21.15.36:0/3478273079 >> v2:172.21.15.70:3300/0 conn(0x7fa7200b45c0 0x7fa7200b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:14.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa71effd700 1 -- 172.21.15.36:0/3478273079 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2679379341 0 0) 0x7fa7200b4160 con 0x7fa7200b9860 2022-06-17T12:51:14.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa71effd700 1 -- 172.21.15.36:0/3478273079 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa700003260 con 0x7fa7200b9860 2022-06-17T12:51:14.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa71effd700 1 -- 172.21.15.36:0/3478273079 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2781970306 0 0) 0x7fa700003260 con 0x7fa7200b9860 2022-06-17T12:51:14.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa71effd700 1 -- 172.21.15.36:0/3478273079 >> v2:172.21.15.70:3300/0 conn(0x7fa7200b45c0 msgr2=0x7fa7200b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:14.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa71effd700 1 --2- 172.21.15.36:0/3478273079 >> v2:172.21.15.70:3300/0 conn(0x7fa7200b45c0 0x7fa7200b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa71effd700 1 -- 172.21.15.36:0/3478273079 >> v2:172.21.15.36:3300/0 conn(0x7fa7200b4f00 msgr2=0x7fa7200b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:14.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa71effd700 1 --2- 172.21.15.36:0/3478273079 >> v2:172.21.15.36:3300/0 conn(0x7fa7200b4f00 0x7fa7200b5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa71effd700 1 -- 172.21.15.36:0/3478273079 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa720062a70 con 0x7fa7200b9860 2022-06-17T12:51:14.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa71effd700 1 -- 172.21.15.36:0/3478273079 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7fa710001680 con 0x7fa7200b9860 2022-06-17T12:51:14.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa71effd700 1 -- 172.21.15.36:0/3478273079 >> v1:172.21.15.36:6789/0 conn(0x7fa7200b9860 legacy=0x7fa720143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:14.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa71effd700 1 --2- 172.21.15.36:0/3478273079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa700003750 0x7fa700003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:14.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa71effd700 1 --2- 172.21.15.36:0/3478273079 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa700004500 0x7fa700004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:14.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa71effd700 1 --2- 172.21.15.36:0/3478273079 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa700004e70 0x7fa70000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:14.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa71effd700 1 -- 172.21.15.36:0/3478273079 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7200b9c40 con 0x7fa700004e70 2022-06-17T12:51:14.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa71effd700 1 -- 172.21.15.36:0/3478273079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7200627d0 con 0x7fa700003750 2022-06-17T12:51:14.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa71effd700 1 -- 172.21.15.36:0/3478273079 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70000bc40 con 0x7fa700004500 2022-06-17T12:51:14.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa71ffff700 1 --2- 172.21.15.36:0/3478273079 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa700004e70 0x7fa70000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:14.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa716ffd700 1 --2- 172.21.15.36:0/3478273079 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa700004500 0x7fa700004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:14.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.313+0000 7fa71f7fe700 1 --2- 172.21.15.36:0/3478273079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa700003750 0x7fa700003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:14.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa71ffff700 1 -- 172.21.15.36:0/3478273079 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa700004500 msgr2=0x7fa700004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:14.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa71ffff700 1 --2- 172.21.15.36:0/3478273079 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa700004500 0x7fa700004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa71ffff700 1 -- 172.21.15.36:0/3478273079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa700003750 msgr2=0x7fa700003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:14.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa71ffff700 1 --2- 172.21.15.36:0/3478273079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa700003750 0x7fa700003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa71ffff700 1 -- 172.21.15.36:0/3478273079 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa700003260 con 0x7fa700004e70 2022-06-17T12:51:14.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa716ffd700 1 --2- 172.21.15.36:0/3478273079 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa700004500 0x7fa700004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:51:14.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa71ffff700 1 -- 172.21.15.36:0/3478273079 reap_dead start 2022-06-17T12:51:14.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa71ffff700 1 --2- 172.21.15.36:0/3478273079 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa700004e70 0x7fa70000b750 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fa71800c000 tx=0x7fa7180091d0).ready entity=mon.0 client_cookie=2f8faf99466502f5 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:14.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa71effd700 1 -- 172.21.15.36:0/3478273079 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa71800e680 con 0x7fa700004e70 2022-06-17T12:51:14.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa71effd700 1 -- 172.21.15.36:0/3478273079 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71800e7e0 con 0x7fa700004e70 2022-06-17T12:51:14.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa71effd700 1 -- 172.21.15.36:0/3478273079 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa71800eb40 con 0x7fa700004e70 2022-06-17T12:51:14.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa724a84700 1 -- 172.21.15.36:0/3478273079 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa700004e70 msgr2=0x7fa70000b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:14.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa724a84700 1 --2- 172.21.15.36:0/3478273079 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa700004e70 0x7fa70000b750 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fa71800c000 tx=0x7fa7180091d0).stop 2022-06-17T12:51:14.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa724a84700 1 -- 172.21.15.36:0/3478273079 shutdown_connections 2022-06-17T12:51:14.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa724a84700 1 --2- 172.21.15.36:0/3478273079 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa700004e70 0x7fa70000b750 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa724a84700 1 -- 172.21.15.36:0/3478273079 >> 172.21.15.36:0/3478273079 conn(0x7fa72005c550 msgr2=0x7fa720060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:51:14.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa724a84700 1 -- 172.21.15.36:0/3478273079 shutdown_connections 2022-06-17T12:51:14.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa724a84700 1 -- 172.21.15.36:0/3478273079 wait complete. 2022-06-17T12:51:14.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa724a84700 1 Processor -- start 2022-06-17T12:51:14.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa724a84700 1 -- start start 2022-06-17T12:51:14.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa724a84700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa700004e70 0x7fa7200ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:14.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa724a84700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa72013d8c0 0x7fa720152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:14.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa724a84700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7201447c0 0x7fa720144c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:14.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa724a84700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa720144390 con 0x7fa72013d8c0 2022-06-17T12:51:14.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa724a84700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa720146e80 con 0x7fa7201447c0 2022-06-17T12:51:14.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa724a84700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa720147120 con 0x7fa700004e70 2022-06-17T12:51:14.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa716ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa72013d8c0 0x7fa720152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:14.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa716ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa72013d8c0 0x7fa720152300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34462/0 (socket says 172.21.15.36:34462) 2022-06-17T12:51:14.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa716ffd700 1 -- 172.21.15.36:0/586884293 learned_addr learned my addr 172.21.15.36:0/586884293 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:51:14.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa71f7fe700 1 --2- 172.21.15.36:0/586884293 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa700004e70 0x7fa7200ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:14.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.317+0000 7fa71ffff700 1 --2- 172.21.15.36:0/586884293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7201447c0 0x7fa720144c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:14.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.321+0000 7fa716ffd700 1 -- 172.21.15.36:0/586884293 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa700004e70 msgr2=0x7fa7200ae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:14.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.321+0000 7fa716ffd700 1 --2- 172.21.15.36:0/586884293 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa700004e70 0x7fa7200ae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.321+0000 7fa716ffd700 1 -- 172.21.15.36:0/586884293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7201447c0 msgr2=0x7fa720144c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:14.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.321+0000 7fa716ffd700 1 --2- 172.21.15.36:0/586884293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7201447c0 0x7fa720144c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.321+0000 7fa716ffd700 1 -- 172.21.15.36:0/586884293 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa720145160 con 0x7fa72013d8c0 2022-06-17T12:51:14.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.321+0000 7fa716ffd700 1 --2- 172.21.15.36:0/586884293 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa72013d8c0 0x7fa720152300 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fa70800b000 tx=0x7fa708006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:14.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.321+0000 7fa71d7fa700 1 -- 172.21.15.36:0/586884293 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa708011e80 con 0x7fa72013d8c0 2022-06-17T12:51:14.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.321+0000 7fa71d7fa700 1 -- 172.21.15.36:0/586884293 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa70800f940 con 0x7fa72013d8c0 2022-06-17T12:51:14.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.321+0000 7fa724a84700 1 -- 172.21.15.36:0/586884293 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7201454c0 con 0x7fa72013d8c0 2022-06-17T12:51:14.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.321+0000 7fa71d7fa700 1 -- 172.21.15.36:0/586884293 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa70800fab0 con 0x7fa72013d8c0 2022-06-17T12:51:14.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.321+0000 7fa724a84700 1 -- 172.21.15.36:0/586884293 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa720145a60 con 0x7fa72013d8c0 2022-06-17T12:51:14.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.321+0000 7fa71d7fa700 1 -- 172.21.15.36:0/586884293 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fa708024070 con 0x7fa72013d8c0 2022-06-17T12:51:14.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.321+0000 7fa724a84700 1 -- 172.21.15.36:0/586884293 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa720062eb0 con 0x7fa72013d8c0 2022-06-17T12:51:14.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.321+0000 7fa71d7fa700 1 --2- 172.21.15.36:0/586884293 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa704053a60 0x7fa704055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:14.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.321+0000 7fa71d7fa700 1 -- 172.21.15.36:0/586884293 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa70806c860 con 0x7fa72013d8c0 2022-06-17T12:51:14.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.321+0000 7fa71f7fe700 1 --2- 172.21.15.36:0/586884293 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa704053a60 0x7fa704055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:14.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.321+0000 7fa71f7fe700 1 --2- 172.21.15.36:0/586884293 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa704053a60 0x7fa704055f20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fa710005000 tx=0x7fa720062c00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:14.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.329+0000 7fa71d7fa700 1 -- 172.21.15.36:0/586884293 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa708043660 con 0x7fa72013d8c0 2022-06-17T12:51:14.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.453+0000 7fa724a84700 1 -- 172.21.15.36:0/586884293 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa720057410 con 0x7fa72013d8c0 2022-06-17T12:51:14.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.453+0000 7fa71d7fa700 1 -- 172.21.15.36:0/586884293 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+10 (secure 0 0 0) 0x7fa708046c80 con 0x7fa72013d8c0 2022-06-17T12:51:14.438 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:51:14.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.457+0000 7fa724a84700 1 -- 172.21.15.36:0/586884293 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa704053a60 msgr2=0x7fa704055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:14.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.457+0000 7fa724a84700 1 --2- 172.21.15.36:0/586884293 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa704053a60 0x7fa704055f20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fa710005000 tx=0x7fa720062c00).stop 2022-06-17T12:51:14.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.457+0000 7fa724a84700 1 -- 172.21.15.36:0/586884293 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa72013d8c0 msgr2=0x7fa720152300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:14.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.457+0000 7fa724a84700 1 --2- 172.21.15.36:0/586884293 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa72013d8c0 0x7fa720152300 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fa70800b000 tx=0x7fa708006df0).stop 2022-06-17T12:51:14.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.457+0000 7fa724a84700 1 -- 172.21.15.36:0/586884293 shutdown_connections 2022-06-17T12:51:14.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.457+0000 7fa724a84700 1 --2- 172.21.15.36:0/586884293 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa704053a60 0x7fa704055f20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.457+0000 7fa724a84700 1 --2- 172.21.15.36:0/586884293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7201447c0 0x7fa720144c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.457+0000 7fa724a84700 1 --2- 172.21.15.36:0/586884293 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa72013d8c0 0x7fa720152300 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.457+0000 7fa724a84700 1 --2- 172.21.15.36:0/586884293 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa700004e70 0x7fa7200ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.457+0000 7fa724a84700 1 -- 172.21.15.36:0/586884293 >> 172.21.15.36:0/586884293 conn(0x7fa72005c550 msgr2=0x7fa7200b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:51:14.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.457+0000 7fa724a84700 1 -- 172.21.15.36:0/586884293 shutdown_connections 2022-06-17T12:51:14.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.457+0000 7fa724a84700 1 -- 172.21.15.36:0/586884293 wait complete. 2022-06-17T12:51:14.453 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:51:14.454 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:51:14.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.577+0000 7f61f2b96700 1 Processor -- start 2022-06-17T12:51:14.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.577+0000 7f61f2b96700 1 -- start start 2022-06-17T12:51:14.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f2b96700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f61ec0afe30 0x7f61ec143f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:14.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f2b96700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61ec067510 0x7f61ec0addb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:14.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f2b96700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61ec0b1570 con 0x7f61ec066bc0 2022-06-17T12:51:14.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f2b96700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61ec0acb10 con 0x7f61ec0afe30 2022-06-17T12:51:14.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f2b96700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec066fa0 con 0x7f61ec067510 2022-06-17T12:51:14.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f1393700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f61ec0afe30 0x7f61ec143f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:14.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f1b94700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f61ec066bc0 0x7f61ec143880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:51998/0 (socket says 172.21.15.36:51998) 2022-06-17T12:51:14.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f1393700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f61ec0afe30 0x7f61ec143f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34468/0 (socket says 172.21.15.36:34468) 2022-06-17T12:51:14.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f1b94700 1 -- 172.21.15.36:0/1096659477 learned_addr learned my addr 172.21.15.36:0/1096659477 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:51:14.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f2395700 1 --2- 172.21.15.36:0/1096659477 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61ec067510 0x7f61ec0addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:14.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f1393700 1 -- 172.21.15.36:0/1096659477 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61ec067510 msgr2=0x7f61ec0addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:51:14.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f1393700 1 --2- 172.21.15.36:0/1096659477 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61ec067510 0x7f61ec0addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f1393700 1 -- 172.21.15.36:0/1096659477 >> v1:172.21.15.36:6789/0 conn(0x7f61ec066bc0 legacy=0x7f61ec143880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:14.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f1393700 1 -- 172.21.15.36:0/1096659477 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ec0b17a0 con 0x7f61ec0afe30 2022-06-17T12:51:14.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f2395700 1 --2- 172.21.15.36:0/1096659477 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61ec067510 0x7f61ec0addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:51:14.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f1393700 1 --2- 172.21.15.36:0/1096659477 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61ec0afe30 0x7f61ec143f90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f61e8002f70 tx=0x7f61e8006df0).ready entity=mon.0 client_cookie=a62531790f13ef97 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:14.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f0b92700 1 -- 172.21.15.36:0/1096659477 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61e800d570 con 0x7f61ec0afe30 2022-06-17T12:51:14.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f0b92700 1 -- 172.21.15.36:0/1096659477 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e8000e90 con 0x7f61ec0afe30 2022-06-17T12:51:14.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f0b92700 1 -- 172.21.15.36:0/1096659477 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61e800d8c0 con 0x7f61ec0afe30 2022-06-17T12:51:14.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f2b96700 1 -- 172.21.15.36:0/1096659477 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61ec0afe30 msgr2=0x7f61ec143f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:14.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f2b96700 1 --2- 172.21.15.36:0/1096659477 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61ec0afe30 0x7f61ec143f90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f61e8002f70 tx=0x7f61e8006df0).stop 2022-06-17T12:51:14.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f2b96700 1 -- 172.21.15.36:0/1096659477 shutdown_connections 2022-06-17T12:51:14.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f2b96700 1 --2- 172.21.15.36:0/1096659477 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61ec067510 0x7f61ec0addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f2b96700 1 --2- 172.21.15.36:0/1096659477 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61ec0afe30 0x7f61ec143f90 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f2b96700 1 -- 172.21.15.36:0/1096659477 >> 172.21.15.36:0/1096659477 conn(0x7f61ec05f770 msgr2=0x7f61ec05fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:51:14.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f2b96700 1 -- 172.21.15.36:0/1096659477 shutdown_connections 2022-06-17T12:51:14.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f2b96700 1 -- 172.21.15.36:0/1096659477 wait complete. 2022-06-17T12:51:14.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f2b96700 1 Processor -- start 2022-06-17T12:51:14.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.581+0000 7f61f2b96700 1 -- start start 2022-06-17T12:51:14.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.585+0000 7f61f2b96700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61ec066bc0 0x7f61ec09f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:14.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.585+0000 7f61f2b96700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61ec067510 0x7f61ec0a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:14.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.585+0000 7f61f2b96700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61ec0afe30 0x7f61ec0a32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:14.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.585+0000 7f61f2b96700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec0a39a0 con 0x7f61ec067510 2022-06-17T12:51:14.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.585+0000 7f61f2b96700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec0a3c40 con 0x7f61ec066bc0 2022-06-17T12:51:14.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.585+0000 7f61f2b96700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec0a3ee0 con 0x7f61ec0afe30 2022-06-17T12:51:14.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.585+0000 7f61f1393700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61ec067510 0x7f61ec0a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:14.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.585+0000 7f61f1393700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61ec067510 0x7f61ec0a2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34476/0 (socket says 172.21.15.36:34476) 2022-06-17T12:51:14.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.585+0000 7f61f1393700 1 -- 172.21.15.36:0/2030684141 learned_addr learned my addr 172.21.15.36:0/2030684141 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:51:14.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.585+0000 7f61f1b94700 1 --2- 172.21.15.36:0/2030684141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61ec066bc0 0x7f61ec09f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:14.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.585+0000 7f61f2395700 1 --2- 172.21.15.36:0/2030684141 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61ec0afe30 0x7f61ec0a32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:14.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.585+0000 7f61f1393700 1 -- 172.21.15.36:0/2030684141 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61ec0afe30 msgr2=0x7f61ec0a32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:14.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.585+0000 7f61f1393700 1 --2- 172.21.15.36:0/2030684141 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61ec0afe30 0x7f61ec0a32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.585+0000 7f61f1393700 1 -- 172.21.15.36:0/2030684141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61ec066bc0 msgr2=0x7f61ec09f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:14.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.585+0000 7f61f1393700 1 --2- 172.21.15.36:0/2030684141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61ec066bc0 0x7f61ec09f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.585+0000 7f61f1393700 1 -- 172.21.15.36:0/2030684141 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e800b180 con 0x7f61ec067510 2022-06-17T12:51:14.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.585+0000 7f61f1393700 1 --2- 172.21.15.36:0/2030684141 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61ec067510 0x7f61ec0a2dc0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f61e8002f70 tx=0x7f61e8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:14.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.585+0000 7f61daffd700 1 -- 172.21.15.36:0/2030684141 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61e800d8f0 con 0x7f61ec067510 2022-06-17T12:51:14.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.585+0000 7f61daffd700 1 -- 172.21.15.36:0/2030684141 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e8011ce0 con 0x7f61ec067510 2022-06-17T12:51:14.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.585+0000 7f61f2b96700 1 -- 172.21.15.36:0/2030684141 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61ec0a4380 con 0x7f61ec067510 2022-06-17T12:51:14.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.585+0000 7f61daffd700 1 -- 172.21.15.36:0/2030684141 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61e801e650 con 0x7f61ec067510 2022-06-17T12:51:14.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.585+0000 7f61f2b96700 1 -- 172.21.15.36:0/2030684141 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ec0a5ac0 con 0x7f61ec067510 2022-06-17T12:51:14.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.585+0000 7f61f2b96700 1 -- 172.21.15.36:0/2030684141 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61ec0a40a0 con 0x7f61ec067510 2022-06-17T12:51:14.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.589+0000 7f61daffd700 1 -- 172.21.15.36:0/2030684141 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f61e8025030 con 0x7f61ec067510 2022-06-17T12:51:14.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.589+0000 7f61daffd700 1 --2- 172.21.15.36:0/2030684141 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f61cc053a40 0x7f61cc055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:51:14.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.589+0000 7f61daffd700 1 -- 172.21.15.36:0/2030684141 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f61e806d160 con 0x7f61ec067510 2022-06-17T12:51:14.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.589+0000 7f61f1b94700 1 --2- 172.21.15.36:0/2030684141 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f61cc053a40 0x7f61cc055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:51:14.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.589+0000 7f61f1b94700 1 --2- 172.21.15.36:0/2030684141 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f61cc053a40 0x7f61cc055f00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f61dc00b000 tx=0x7f61dc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:51:14.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.593+0000 7f61daffd700 1 -- 172.21.15.36:0/2030684141 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f61e8043f60 con 0x7f61ec067510 2022-06-17T12:51:14.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.761+0000 7f61f2b96700 1 -- 172.21.15.36:0/2030684141 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61ec0b0550 con 0x7f61ec067510 2022-06-17T12:51:14.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.761+0000 7f61daffd700 1 -- 172.21.15.36:0/2030684141 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f61e8023090 con 0x7f61ec067510 2022-06-17T12:51:14.747 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:51:14.747 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":25,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5670,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.36:6835/2598001220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2598001220},{"type":"v1","addr":"172.21.15.36:6835","nonce":2598001220}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":23}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:50:49.096455+0000","modified":"2022-06-17T12:50:50.099539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"b","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/2723288138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2723288138},{"type":"v1","addr":"172.21.15.70:6833","nonce":2723288138}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-06-17T12:51:14.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.765+0000 7f61f2b96700 1 -- 172.21.15.36:0/2030684141 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f61cc053a40 msgr2=0x7f61cc055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:14.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.765+0000 7f61f2b96700 1 --2- 172.21.15.36:0/2030684141 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f61cc053a40 0x7f61cc055f00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f61dc00b000 tx=0x7f61dc009040).stop 2022-06-17T12:51:14.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.765+0000 7f61f2b96700 1 -- 172.21.15.36:0/2030684141 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61ec067510 msgr2=0x7f61ec0a2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:51:14.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.765+0000 7f61f2b96700 1 --2- 172.21.15.36:0/2030684141 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61ec067510 0x7f61ec0a2dc0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f61e8002f70 tx=0x7f61e8015000).stop 2022-06-17T12:51:14.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.765+0000 7f61f2b96700 1 -- 172.21.15.36:0/2030684141 shutdown_connections 2022-06-17T12:51:14.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.765+0000 7f61f2b96700 1 --2- 172.21.15.36:0/2030684141 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f61cc053a40 0x7f61cc055f00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.765+0000 7f61f2b96700 1 --2- 172.21.15.36:0/2030684141 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61ec0afe30 0x7f61ec0a32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.765+0000 7f61f2b96700 1 --2- 172.21.15.36:0/2030684141 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61ec067510 0x7f61ec0a2dc0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.765+0000 7f61f2b96700 1 --2- 172.21.15.36:0/2030684141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61ec066bc0 0x7f61ec09f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:51:14.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.765+0000 7f61f2b96700 1 -- 172.21.15.36:0/2030684141 >> 172.21.15.36:0/2030684141 conn(0x7f61ec05f770 msgr2=0x7f61ec146d70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:51:14.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.765+0000 7f61f2b96700 1 -- 172.21.15.36:0/2030684141 shutdown_connections 2022-06-17T12:51:14.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:51:14.765+0000 7f61f2b96700 1 -- 172.21.15.36:0/2030684141 wait complete. 2022-06-17T12:51:14.753 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 25 2022-06-17T12:51:14.974 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:50:49.096455+0000', 'modified': '2022-06-17T12:50:50.099539+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5679}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5679': {'gid': 5679, 'name': 'b', 'rank': 0, 'incarnation': 24, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6833/2723288138', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 2723288138}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 2723288138}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:51:14.975 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:51:14.976 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-17T12:51:14.978 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:51:14.983 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:51:14.991 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:51:15.002 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:51:15.007 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:51:15.103 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":5679,"want_state":"up:active","state":"up:active","rank_uptime":26.003032137999998,"mdsmap_epoch":25,"osdmap_epoch":56,"osdmap_epoch_barrier":54,"uptime":31.303954439999998} 2022-06-17T12:51:15.104 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 5679, "mdsmap_epoch": 25, "osdmap_epoch": 56, "osdmap_epoch_barrier": 54, "rank_uptime": 26.003032138, "state": "up:active", "uptime": 31.30395444, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:51:15.120 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:51:15.121 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-17T12:51:15.122 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi107.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-17T12:51:15.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:51:15.124 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:51:15.146 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:51:15.206 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:51:15.209 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:51:15.230 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:51:15.232 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:51:15.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:51:15.233 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:51:15.311 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:51:15.314 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:51:15.347 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:51:15.396 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:51:15.396 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:51:15.406 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:2022-06-17T12:51:15.419+0000 7f4b2313e080 -1 init, newargv = 0x55e634eb6410 newargc=9ceph-fuse[22171 2022-06-17T12:51:15.407 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:]: starting ceph client 2022-06-17T12:51:15.414 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22171]: starting fuse 2022-06-17T12:51:15.427 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:51:15.428 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:51:15.429 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:51:15.429 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:51:15.488 INFO:teuthology.orchestra.run.smithi107.stdout:52 2022-06-17T12:51:15.489 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:51:15.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:51:15.493 DEBUG:teuthology.orchestra.run.smithi107:> sudo python3 -c ' 2022-06-17T12:51:15.494 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2022-06-17T12:51:15.498 DEBUG:teuthology.orchestra.run.smithi107:> import re 2022-06-17T12:51:15.499 DEBUG:teuthology.orchestra.run.smithi107:> import os 2022-06-17T12:51:15.501 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2022-06-17T12:51:15.503 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:51:15.505 DEBUG:teuthology.orchestra.run.smithi107:> def find_socket(client_name): 2022-06-17T12:51:15.508 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-17T12:51:15.510 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2022-06-17T12:51:15.511 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:51:15.513 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2022-06-17T12:51:15.514 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2022-06-17T12:51:15.516 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2022-06-17T12:51:15.517 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2022-06-17T12:51:15.520 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:51:15.521 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2022-06-17T12:51:15.522 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:51:15.522 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:51:15.523 DEBUG:teuthology.orchestra.run.smithi107:> return f 2022-06-17T12:51:15.524 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:51:15.525 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:51:15.526 DEBUG:teuthology.orchestra.run.smithi107:> print(find_socket("client.0")) 2022-06-17T12:51:15.526 DEBUG:teuthology.orchestra.run.smithi107:> ' 2022-06-17T12:51:15.593 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.0.22171.asok 2022-06-17T12:51:15.593 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22171.asok 2022-06-17T12:51:15.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:51:15.595 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22171.asok status 2022-06-17T12:51:15.749 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2022-06-17T12:51:15.751 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2022-06-17T12:51:15.751 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:51:15.752 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:51:15.754 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "0", 2022-06-17T12:51:15.755 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2022-06-17T12:51:15.755 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-17T12:51:15.756 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "22171", 2022-06-17T12:51:15.756 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/" 2022-06-17T12:51:15.757 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:51:15.757 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2022-06-17T12:51:15.757 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2022-06-17T12:51:15.758 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 5345, 2022-06-17T12:51:15.759 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2022-06-17T12:51:15.759 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2022-06-17T12:51:15.760 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2022-06-17T12:51:15.760 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 5345 2022-06-17T12:51:15.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:51:15.809 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:51:15.809 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:51:15.809 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:51:15.810 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1541630863 2022-06-17T12:51:15.810 INFO:teuthology.orchestra.run.smithi107.stdout: } 2022-06-17T12:51:15.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:51:15.811 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:51:15.812 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:51:15.812 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:51:15.813 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1541630863 2022-06-17T12:51:15.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:51:15.815 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.5345 172.21.15.107:0/1541630863", 2022-06-17T12:51:15.815 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "172.21.15.107:0/1541630863", 2022-06-17T12:51:15.815 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 1, 2022-06-17T12:51:15.816 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 25, 2022-06-17T12:51:15.816 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 56, 2022-06-17T12:51:15.817 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:51:15.819 INFO:teuthology.orchestra.run.smithi107.stdout: "blacklisted": false, 2022-06-17T12:51:15.820 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2022-06-17T12:51:15.820 INFO:teuthology.orchestra.run.smithi107.stdout:} 2022-06-17T12:51:15.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:51:15.824 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:51:15.829 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:51:15.850 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:51:15.851 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:51:15.852 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:51:15.853 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:51:45.352 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:51:45.366 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:51:45.370 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:51:45.374 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:51:45.390 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:52:08.787 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-17T12:52:08.789 DEBUG:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:52:08.817 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:52:08.840 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:52:08.863 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:52:08.886 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:52:08.916 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-17T12:52:08.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:52:08.918 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:52:08.919 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:52:08.933 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:52:08.933 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:52:08.934 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2022-06-17T12:52:08.939 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:52:08.939 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:52:09.004 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22171]: fuse finished with error 0 and tester_r 0 2022-06-17T12:52:09.006 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:52:09.007 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:52:09.007 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:52:09.058 INFO:teuthology.orchestra.run.smithi107.stdout:ext2/ext3 2022-06-17T12:52:09.059 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:52:09.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:52:09.078 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:52:09.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:52:09.133 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:52:09.134 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:52:09.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:52:09.145 INFO:teuthology.orchestra.run.smithi136.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-17T12:52:09.145 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-17T12:52:09.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:52:09.147 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:52:09.155 INFO:teuthology.orchestra.run.smithi136.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-17T12:52:09.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:52:09.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:52:09.159 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:52:09.160 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:52:09.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:52:09.180 INFO:teuthology.orchestra.run.smithi143.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:52:09.180 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-06-17T12:52:09.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:52:09.181 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:52:09.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:52:09.232 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:52:09.233 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-17T12:52:09.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb952e700 1 Processor -- start 2022-06-17T12:52:09.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb952e700 1 -- start start 2022-06-17T12:52:09.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb952e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7effb40ad140 0x7effb40af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:09.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb952e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7effb40ac340 0x7effb40ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:09.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb952e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb40ac870 con 0x7effb414aca0 2022-06-17T12:52:09.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb952e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb4148960 con 0x7effb40ad140 2022-06-17T12:52:09.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb952e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb40acaa0 con 0x7effb40ac340 2022-06-17T12:52:09.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb2ffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7effb40ad140 0x7effb40af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:09.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb37fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7effb414aca0 0x7effb414b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52016/0 (socket says 172.21.15.36:52016) 2022-06-17T12:52:09.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb37fe700 1 -- 172.21.15.36:0/582789778 learned_addr learned my addr 172.21.15.36:0/582789778 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:09.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb27fc700 1 --2- 172.21.15.36:0/582789778 >> v2:172.21.15.70:3300/0 conn(0x7effb40ac340 0x7effb40ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:09.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb1ffb700 1 -- 172.21.15.36:0/582789778 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167195368 0 0) 0x7effb40ac870 con 0x7effb414aca0 2022-06-17T12:52:09.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb1ffb700 1 -- 172.21.15.36:0/582789778 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff98003260 con 0x7effb414aca0 2022-06-17T12:52:09.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb2ffd700 1 -- 172.21.15.36:0/582789778 >> v2:172.21.15.70:3300/0 conn(0x7effb40ac340 msgr2=0x7effb40ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:09.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb2ffd700 1 --2- 172.21.15.36:0/582789778 >> v2:172.21.15.70:3300/0 conn(0x7effb40ac340 0x7effb40ad520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:09.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb2ffd700 1 -- 172.21.15.36:0/582789778 >> v1:172.21.15.36:6789/0 conn(0x7effb414aca0 legacy=0x7effb414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:09.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb2ffd700 1 -- 172.21.15.36:0/582789778 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb40b0bf0 con 0x7effb40ad140 2022-06-17T12:52:09.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb2ffd700 1 --2- 172.21.15.36:0/582789778 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7effb40ad140 0x7effb40af7f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7eff9c002f70 tx=0x7eff9c006df0).ready entity=mon.0 client_cookie=ad5a02772b4a5a67 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:09.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb1ffb700 1 -- 172.21.15.36:0/582789778 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff9c00d4f0 con 0x7effb40ad140 2022-06-17T12:52:09.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb1ffb700 1 -- 172.21.15.36:0/582789778 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9c000e90 con 0x7effb40ad140 2022-06-17T12:52:09.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb1ffb700 1 -- 172.21.15.36:0/582789778 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff9c00d840 con 0x7effb40ad140 2022-06-17T12:52:09.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb952e700 1 -- 172.21.15.36:0/582789778 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7effb40ad140 msgr2=0x7effb40af7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:09.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb952e700 1 --2- 172.21.15.36:0/582789778 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7effb40ad140 0x7effb40af7f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7eff9c002f70 tx=0x7eff9c006df0).stop 2022-06-17T12:52:09.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb952e700 1 -- 172.21.15.36:0/582789778 shutdown_connections 2022-06-17T12:52:09.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb952e700 1 --2- 172.21.15.36:0/582789778 >> v2:172.21.15.70:3300/0 conn(0x7effb40ac340 0x7effb40ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:09.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb952e700 1 --2- 172.21.15.36:0/582789778 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7effb40ad140 0x7effb40af7f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:09.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb952e700 1 -- 172.21.15.36:0/582789778 >> 172.21.15.36:0/582789778 conn(0x7effb40b5430 msgr2=0x7effb40b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:09.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.373+0000 7effb952e700 1 -- 172.21.15.36:0/582789778 shutdown_connections 2022-06-17T12:52:09.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.377+0000 7effb952e700 1 -- 172.21.15.36:0/582789778 wait complete. 2022-06-17T12:52:09.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.377+0000 7effb952e700 1 Processor -- start 2022-06-17T12:52:09.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.377+0000 7effb952e700 1 -- start start 2022-06-17T12:52:09.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.377+0000 7effb952e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7effb40ac340 0x7effb4063460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:09.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.377+0000 7effb952e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7effb40ad140 0x7effb4063950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:09.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.377+0000 7effb952e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effb414aca0 0x7effb4155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:09.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.377+0000 7effb952e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb4157c20 con 0x7effb40ac340 2022-06-17T12:52:09.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.377+0000 7effb952e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb40b5170 con 0x7effb414aca0 2022-06-17T12:52:09.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.377+0000 7effb952e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb40a9280 con 0x7effb40ad140 2022-06-17T12:52:09.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.377+0000 7effb2ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7effb40ac340 0x7effb4063460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:09.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.377+0000 7effb2ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7effb40ac340 0x7effb4063460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34488/0 (socket says 172.21.15.36:34488) 2022-06-17T12:52:09.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.377+0000 7effb2ffd700 1 -- 172.21.15.36:0/3004732021 learned_addr learned my addr 172.21.15.36:0/3004732021 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:09.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.377+0000 7effb27fc700 1 --2- 172.21.15.36:0/3004732021 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7effb40ad140 0x7effb4063950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:09.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.377+0000 7effb37fe700 1 --2- 172.21.15.36:0/3004732021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effb414aca0 0x7effb4155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:09.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.377+0000 7effb2ffd700 1 -- 172.21.15.36:0/3004732021 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7effb40ad140 msgr2=0x7effb4063950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:09.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.377+0000 7effb2ffd700 1 --2- 172.21.15.36:0/3004732021 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7effb40ad140 0x7effb4063950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:09.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.377+0000 7effb2ffd700 1 -- 172.21.15.36:0/3004732021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effb414aca0 msgr2=0x7effb4155e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:09.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.377+0000 7effb2ffd700 1 --2- 172.21.15.36:0/3004732021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effb414aca0 0x7effb4155e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:09.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.377+0000 7effb2ffd700 1 -- 172.21.15.36:0/3004732021 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9c00b180 con 0x7effb40ac340 2022-06-17T12:52:09.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.377+0000 7effb37fe700 1 --2- 172.21.15.36:0/3004732021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effb414aca0 0x7effb4155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:52:09.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.377+0000 7effb2ffd700 1 --2- 172.21.15.36:0/3004732021 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7effb40ac340 0x7effb4063460 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7eff9c002f70 tx=0x7eff9c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:09.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.377+0000 7effabfff700 1 -- 172.21.15.36:0/3004732021 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff9c013e90 con 0x7effb40ac340 2022-06-17T12:52:09.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.377+0000 7effabfff700 1 -- 172.21.15.36:0/3004732021 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9c00de60 con 0x7effb40ac340 2022-06-17T12:52:09.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.377+0000 7effb952e700 1 -- 172.21.15.36:0/3004732021 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb40a9520 con 0x7effb40ac340 2022-06-17T12:52:09.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.381+0000 7effabfff700 1 -- 172.21.15.36:0/3004732021 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff9c01e5f0 con 0x7effb40ac340 2022-06-17T12:52:09.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.381+0000 7effb952e700 1 -- 172.21.15.36:0/3004732021 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb41578c0 con 0x7effb40ac340 2022-06-17T12:52:09.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.381+0000 7effb952e700 1 -- 172.21.15.36:0/3004732021 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb40a96b0 con 0x7effb40ac340 2022-06-17T12:52:09.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.381+0000 7effabfff700 1 -- 172.21.15.36:0/3004732021 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7eff9c025030 con 0x7effb40ac340 2022-06-17T12:52:09.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.381+0000 7effabfff700 1 --2- 172.21.15.36:0/3004732021 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7eff8c053a40 0x7eff8c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:09.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.381+0000 7effabfff700 1 -- 172.21.15.36:0/3004732021 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7eff9c06cd40 con 0x7effb40ac340 2022-06-17T12:52:09.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.381+0000 7effb27fc700 1 --2- 172.21.15.36:0/3004732021 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7eff8c053a40 0x7eff8c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:09.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.381+0000 7effb27fc700 1 --2- 172.21.15.36:0/3004732021 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7eff8c053a40 0x7eff8c055f00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7effac00b000 tx=0x7effac0091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:09.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.389+0000 7effabfff700 1 -- 172.21.15.36:0/3004732021 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7eff9c043a30 con 0x7effb40ac340 2022-06-17T12:52:09.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.517+0000 7effb952e700 1 -- 172.21.15.36:0/3004732021 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7effb4057410 con 0x7effb40ac340 2022-06-17T12:52:09.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.521+0000 7effabfff700 1 -- 172.21.15.36:0/3004732021 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v56) v1 ==== 81+0+34039 (secure 0 0 0) 0x7eff9c023070 con 0x7effb40ac340 2022-06-17T12:52:09.504 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:52:09.504 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:52:09.504 INFO:teuthology.orchestra.run.smithi036.stdout: "epoch": 56, 2022-06-17T12:52:09.505 INFO:teuthology.orchestra.run.smithi036.stdout: "fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", 2022-06-17T12:52:09.505 INFO:teuthology.orchestra.run.smithi036.stdout: "created": "2022-06-17T12:46:30.671408+0000", 2022-06-17T12:52:09.505 INFO:teuthology.orchestra.run.smithi036.stdout: "modified": "2022-06-17T12:50:51.108569+0000", 2022-06-17T12:52:09.505 INFO:teuthology.orchestra.run.smithi036.stdout: "last_up_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:52:09.505 INFO:teuthology.orchestra.run.smithi036.stdout: "last_in_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:52:09.506 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-17T12:52:09.506 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_num": 5799936, 2022-06-17T12:52:09.506 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_set": [ 2022-06-17T12:52:09.506 INFO:teuthology.orchestra.run.smithi036.stdout: "pglog_hardlimit", 2022-06-17T12:52:09.507 INFO:teuthology.orchestra.run.smithi036.stdout: "purged_snapdirs", 2022-06-17T12:52:09.507 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_deletes", 2022-06-17T12:52:09.507 INFO:teuthology.orchestra.run.smithi036.stdout: "sortbitwise" 2022-06-17T12:52:09.507 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:52:09.508 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_version": 6, 2022-06-17T12:52:09.508 INFO:teuthology.orchestra.run.smithi036.stdout: "full_ratio": 0.94999998807907104, 2022-06-17T12:52:09.508 INFO:teuthology.orchestra.run.smithi036.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-17T12:52:09.508 INFO:teuthology.orchestra.run.smithi036.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-17T12:52:09.508 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_snapshot": "", 2022-06-17T12:52:09.509 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_max": 10, 2022-06-17T12:52:09.509 INFO:teuthology.orchestra.run.smithi036.stdout: "max_osd": 8, 2022-06-17T12:52:09.509 INFO:teuthology.orchestra.run.smithi036.stdout: "require_min_compat_client": "jewel", 2022-06-17T12:52:09.509 INFO:teuthology.orchestra.run.smithi036.stdout: "min_compat_client": "jewel", 2022-06-17T12:52:09.510 INFO:teuthology.orchestra.run.smithi036.stdout: "require_osd_release": "octopus", 2022-06-17T12:52:09.510 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:52:09.510 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.510 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 1, 2022-06-17T12:52:09.511 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "device_health_metrics", 2022-06-17T12:52:09.511 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:33.881212+0000", 2022-06-17T12:52:09.511 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:52:09.511 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:52:09.512 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:52:09.512 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:52:09.513 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:52:09.513 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:52:09.513 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:52:09.514 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:52:09.514 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 1, 2022-06-17T12:52:09.516 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 1, 2022-06-17T12:52:09.516 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 1, 2022-06-17T12:52:09.516 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 1, 2022-06-17T12:52:09.517 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 1, 2022-06-17T12:52:09.517 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:52:09.517 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:52:09.518 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:52:09.518 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:52:09.519 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:52:09.519 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:52:09.519 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:52:09.519 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.520 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "16", 2022-06-17T12:52:09.520 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:52:09.520 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:52:09.520 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:52:09.521 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:52:09.521 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:52:09.522 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:52:09.522 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:52:09.522 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:52:09.522 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:52:09.523 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:52:09.523 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:52:09.523 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:52:09.523 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:52:09.523 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:52:09.524 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:52:09.524 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:52:09.524 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:52:09.524 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:52:09.525 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:52:09.525 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:52:09.525 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:52:09.525 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:52:09.526 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:52:09.526 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:52:09.526 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:52:09.526 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:52:09.526 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.527 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:52:09.527 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:52:09.527 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:52:09.527 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:52:09.528 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:52:09.528 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:52:09.528 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:52:09.528 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:52:09.529 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:52:09.529 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:52:09.529 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:52:09.529 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:52:09.530 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 1 2022-06-17T12:52:09.530 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.530 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:52:09.530 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr_devicehealth": {} 2022-06-17T12:52:09.531 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.531 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.531 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.531 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 2, 2022-06-17T12:52:09.532 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "rbd", 2022-06-17T12:52:09.532 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:44.485460+0000", 2022-06-17T12:52:09.532 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:52:09.532 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:52:09.532 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:52:09.533 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:52:09.533 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:52:09.533 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:52:09.533 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:52:09.533 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:52:09.534 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:52:09.534 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:52:09.534 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:52:09.534 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:52:09.535 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:52:09.535 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:52:09.535 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:52:09.535 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:52:09.536 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:52:09.536 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:52:09.537 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:52:09.537 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:52:09.537 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.537 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "19", 2022-06-17T12:52:09.538 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:52:09.538 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:52:09.538 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:52:09.538 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:52:09.539 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:52:09.539 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:52:09.539 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:52:09.539 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:52:09.539 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:52:09.540 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:52:09.540 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:52:09.540 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:52:09.541 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:52:09.541 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:52:09.541 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:52:09.541 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:52:09.541 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:52:09.542 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:52:09.542 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:52:09.542 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:52:09.543 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:52:09.543 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:52:09.544 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:52:09.544 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:52:09.545 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:52:09.545 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:52:09.545 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.546 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:52:09.546 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:52:09.547 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:52:09.547 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:52:09.547 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:52:09.548 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:52:09.548 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:52:09.548 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:52:09.548 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:52:09.549 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:52:09.549 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:52:09.550 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:52:09.550 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:52:09.550 INFO:teuthology.orchestra.run.smithi036.stdout: "rbd": {} 2022-06-17T12:52:09.550 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.551 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.551 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.551 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 9, 2022-06-17T12:52:09.551 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_metadata", 2022-06-17T12:52:09.551 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:50:46.705901+0000", 2022-06-17T12:52:09.552 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:52:09.552 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:52:09.552 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:52:09.552 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:52:09.553 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:52:09.553 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:52:09.553 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:52:09.553 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:52:09.553 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:52:09.554 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:52:09.554 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:52:09.554 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:52:09.555 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:52:09.555 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:52:09.555 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:52:09.555 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:52:09.555 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:52:09.556 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:52:09.556 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:52:09.556 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:52:09.556 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.557 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "54", 2022-06-17T12:52:09.557 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:52:09.558 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:52:09.558 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:52:09.558 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:52:09.559 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:52:09.559 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:52:09.560 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:52:09.560 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:52:09.561 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:52:09.561 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:52:09.562 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:52:09.562 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:52:09.565 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:52:09.565 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:52:09.566 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:52:09.567 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:52:09.567 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:52:09.567 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:52:09.567 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:52:09.568 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:52:09.568 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:52:09.568 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:52:09.568 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:52:09.569 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:52:09.569 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:52:09.569 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:52:09.569 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.569 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:52:09.570 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:52:09.570 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:52:09.570 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:52:09.570 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:52:09.570 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:52:09.571 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:52:09.571 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:52:09.571 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:52:09.571 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:52:09.572 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:52:09.572 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:52:09.572 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_bias": 4, 2022-06-17T12:52:09.572 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 16, 2022-06-17T12:52:09.572 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_priority": 5 2022-06-17T12:52:09.573 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.573 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:52:09.573 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:52:09.573 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": "cephfs" 2022-06-17T12:52:09.573 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.574 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.574 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.574 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.574 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 10, 2022-06-17T12:52:09.575 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_data", 2022-06-17T12:52:09.575 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:50:47.812736+0000", 2022-06-17T12:52:09.575 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:52:09.575 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:52:09.575 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:52:09.576 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:52:09.576 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:52:09.577 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:52:09.577 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:52:09.577 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:52:09.577 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:52:09.578 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:52:09.578 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:52:09.578 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:52:09.578 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:52:09.578 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:52:09.579 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:52:09.579 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:52:09.579 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:52:09.579 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:52:09.580 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:52:09.580 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:52:09.580 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.580 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "56", 2022-06-17T12:52:09.580 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:52:09.581 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:52:09.581 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:52:09.581 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:52:09.582 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:52:09.582 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:52:09.582 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:52:09.582 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:52:09.582 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:52:09.583 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:52:09.583 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:52:09.583 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:52:09.583 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:52:09.583 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:52:09.584 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:52:09.584 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:52:09.584 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:52:09.584 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:52:09.585 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:52:09.585 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:52:09.585 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:52:09.585 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:52:09.586 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:52:09.586 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:52:09.586 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:52:09.586 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:52:09.586 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.587 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:52:09.587 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:52:09.587 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:52:09.587 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:52:09.588 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:52:09.589 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:52:09.589 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:52:09.590 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:52:09.590 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:52:09.591 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:52:09.591 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:52:09.592 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:52:09.592 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:52:09.593 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:52:09.594 INFO:teuthology.orchestra.run.smithi036.stdout: "data": "cephfs" 2022-06-17T12:52:09.594 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.594 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.594 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.595 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:52:09.595 INFO:teuthology.orchestra.run.smithi036.stdout: "osds": [ 2022-06-17T12:52:09.595 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.596 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:52:09.596 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", 2022-06-17T12:52:09.597 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:09.598 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:09.598 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:09.599 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:09.600 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:09.600 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:09.603 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:52:09.604 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 53, 2022-06-17T12:52:09.604 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:09.605 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:09.605 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:09.606 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.606 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.607 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.607 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6802", 2022-06-17T12:52:09.608 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:09.608 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.609 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.609 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.610 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6803", 2022-06-17T12:52:09.610 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:09.611 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.611 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.612 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.612 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:09.612 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.613 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.613 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.614 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6804", 2022-06-17T12:52:09.614 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:09.614 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.615 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.615 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.616 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6805", 2022-06-17T12:52:09.616 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:09.617 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.617 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.618 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.618 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:09.618 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.618 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.619 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.619 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6808", 2022-06-17T12:52:09.619 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:09.619 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.620 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.620 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.620 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6809", 2022-06-17T12:52:09.620 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:09.620 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.621 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.621 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.621 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:09.621 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.622 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.622 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.622 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6806", 2022-06-17T12:52:09.622 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:09.623 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.623 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.623 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.623 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6807", 2022-06-17T12:52:09.623 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:09.624 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.624 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.624 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.624 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6803/22619", 2022-06-17T12:52:09.625 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6805/22619", 2022-06-17T12:52:09.625 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6809/22619", 2022-06-17T12:52:09.625 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6807/22619", 2022-06-17T12:52:09.625 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:09.625 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:09.626 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:09.626 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.626 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.626 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.627 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:52:09.627 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", 2022-06-17T12:52:09.628 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:09.629 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:09.629 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:09.629 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:09.630 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:09.630 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:09.630 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:52:09.630 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 54, 2022-06-17T12:52:09.630 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:09.631 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:09.631 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:09.631 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.631 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.632 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.632 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6810", 2022-06-17T12:52:09.632 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:09.632 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.633 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.633 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.633 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6811", 2022-06-17T12:52:09.633 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:09.634 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.634 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.634 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.634 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:09.635 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.635 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.636 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.636 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6812", 2022-06-17T12:52:09.636 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:09.637 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.637 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.637 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.638 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6813", 2022-06-17T12:52:09.638 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:09.638 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.638 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.639 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.639 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:09.639 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.639 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.640 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.640 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6816", 2022-06-17T12:52:09.640 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:09.640 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.640 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.641 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.641 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6817", 2022-06-17T12:52:09.641 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:09.641 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.642 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.642 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.642 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:09.642 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.643 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.643 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.643 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6814", 2022-06-17T12:52:09.643 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:09.644 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.644 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.644 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.645 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6815", 2022-06-17T12:52:09.646 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:09.646 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.646 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.646 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.647 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6811/22620", 2022-06-17T12:52:09.647 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6813/22620", 2022-06-17T12:52:09.647 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6817/22620", 2022-06-17T12:52:09.647 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6815/22620", 2022-06-17T12:52:09.647 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:09.648 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:09.648 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:09.649 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.649 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.649 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.649 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:52:09.649 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", 2022-06-17T12:52:09.650 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:09.650 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:09.650 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:09.650 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:09.651 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:09.651 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:09.651 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:52:09.651 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 53, 2022-06-17T12:52:09.652 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:09.652 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:09.652 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:09.652 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.652 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.653 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.653 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6826", 2022-06-17T12:52:09.653 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:09.653 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.654 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.654 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.654 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6827", 2022-06-17T12:52:09.654 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:09.655 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.655 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.655 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.656 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:09.656 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.656 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.657 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.657 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6828", 2022-06-17T12:52:09.657 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:09.657 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.658 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.658 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.658 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6829", 2022-06-17T12:52:09.658 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:09.659 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.659 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.659 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.659 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:09.659 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.660 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.660 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.660 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6832", 2022-06-17T12:52:09.660 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:09.661 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.661 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.661 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.661 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6833", 2022-06-17T12:52:09.662 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:09.662 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.662 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.662 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.663 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:09.663 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.663 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.663 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.664 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6830", 2022-06-17T12:52:09.664 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:09.664 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.664 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.665 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.665 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6831", 2022-06-17T12:52:09.665 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:09.665 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.666 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.667 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.668 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6827/22643", 2022-06-17T12:52:09.668 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6829/22643", 2022-06-17T12:52:09.669 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6833/22643", 2022-06-17T12:52:09.669 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6831/22643", 2022-06-17T12:52:09.669 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:09.670 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:09.670 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:09.670 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.671 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.671 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.671 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:52:09.671 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", 2022-06-17T12:52:09.671 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:09.672 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:09.672 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:09.672 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:09.672 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:09.673 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:09.673 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:52:09.673 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 51, 2022-06-17T12:52:09.673 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:09.673 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:09.674 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:09.674 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.674 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.675 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.675 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6818", 2022-06-17T12:52:09.675 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:09.675 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.676 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.676 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.676 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6819", 2022-06-17T12:52:09.677 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:09.677 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.677 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.677 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.677 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:09.678 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.678 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.679 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.679 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6820", 2022-06-17T12:52:09.679 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:09.680 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.680 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.680 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.680 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6821", 2022-06-17T12:52:09.680 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:09.681 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.681 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.681 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.681 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:09.682 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.682 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.682 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.682 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6824", 2022-06-17T12:52:09.683 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:09.683 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.683 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.684 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.684 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6825", 2022-06-17T12:52:09.684 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:09.685 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.685 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.685 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.685 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:09.686 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.686 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.686 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.687 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6822", 2022-06-17T12:52:09.687 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:09.687 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.687 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.688 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.688 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6823", 2022-06-17T12:52:09.689 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:09.689 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.689 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.690 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.690 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6819/22646", 2022-06-17T12:52:09.690 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6821/22646", 2022-06-17T12:52:09.690 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6825/22646", 2022-06-17T12:52:09.691 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6823/22646", 2022-06-17T12:52:09.691 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:09.691 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:09.692 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:09.692 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.692 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.692 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.692 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:52:09.693 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", 2022-06-17T12:52:09.693 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:09.693 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:09.693 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:09.694 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:09.694 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:09.694 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:09.694 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:52:09.695 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 53, 2022-06-17T12:52:09.695 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:09.695 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:09.695 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:09.695 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.696 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.696 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.696 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6808", 2022-06-17T12:52:09.696 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:09.697 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.697 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.697 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.698 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6809", 2022-06-17T12:52:09.698 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:09.698 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.698 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.698 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.699 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:09.699 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.699 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.700 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.700 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6810", 2022-06-17T12:52:09.700 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:09.700 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.700 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.701 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.701 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6811", 2022-06-17T12:52:09.701 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:09.701 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.702 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.702 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.702 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:09.702 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.703 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.703 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.703 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6814", 2022-06-17T12:52:09.703 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:09.703 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.704 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.704 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.704 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6815", 2022-06-17T12:52:09.705 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:09.705 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.705 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.706 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.706 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:09.706 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.706 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.707 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.707 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6812", 2022-06-17T12:52:09.707 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:09.707 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.708 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.708 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.708 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6813", 2022-06-17T12:52:09.708 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:09.708 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.709 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.709 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.709 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6809/22812", 2022-06-17T12:52:09.710 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6811/22812", 2022-06-17T12:52:09.710 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6815/22812", 2022-06-17T12:52:09.710 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6813/22812", 2022-06-17T12:52:09.710 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:09.711 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:09.711 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:09.711 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.711 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.712 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.712 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:52:09.712 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", 2022-06-17T12:52:09.712 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:09.713 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:09.713 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:09.713 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:09.714 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:09.714 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:09.714 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:52:09.714 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 53, 2022-06-17T12:52:09.714 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:09.715 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:09.715 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:09.715 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.715 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.716 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.716 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6800", 2022-06-17T12:52:09.716 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:09.716 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.717 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.717 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.717 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6801", 2022-06-17T12:52:09.718 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:09.718 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.719 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.719 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.719 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:09.720 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.720 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.720 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.720 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6802", 2022-06-17T12:52:09.721 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:09.721 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.721 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.721 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.722 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6803", 2022-06-17T12:52:09.722 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:09.722 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.722 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.722 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.723 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:09.723 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.723 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.723 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.724 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6806", 2022-06-17T12:52:09.725 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:09.725 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.725 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.725 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.726 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6807", 2022-06-17T12:52:09.726 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:09.726 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.727 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.727 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.727 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:09.727 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.728 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.728 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.728 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6804", 2022-06-17T12:52:09.728 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:09.728 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.729 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.729 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.729 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6805", 2022-06-17T12:52:09.729 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:09.730 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.730 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.730 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.730 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6801/22813", 2022-06-17T12:52:09.731 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6803/22813", 2022-06-17T12:52:09.731 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6807/22813", 2022-06-17T12:52:09.731 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6805/22813", 2022-06-17T12:52:09.731 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:09.732 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:09.732 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:09.732 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.732 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.732 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.733 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:52:09.733 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", 2022-06-17T12:52:09.733 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:09.733 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:09.734 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:09.734 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:09.734 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:09.734 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:09.735 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:52:09.735 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 54, 2022-06-17T12:52:09.736 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:09.737 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:09.737 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:09.737 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.737 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.737 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.738 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6816", 2022-06-17T12:52:09.738 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:09.738 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.738 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.739 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.739 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6817", 2022-06-17T12:52:09.739 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:09.739 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.740 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.740 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.740 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:09.740 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.740 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.741 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.741 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6818", 2022-06-17T12:52:09.741 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:09.741 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.741 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.742 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.742 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6819", 2022-06-17T12:52:09.742 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:09.742 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.743 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.743 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.743 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:09.743 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.743 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.744 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.744 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6822", 2022-06-17T12:52:09.745 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:09.745 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.745 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.745 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.746 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6823", 2022-06-17T12:52:09.746 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:09.746 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.746 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.747 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.747 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:09.747 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.747 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.747 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.748 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6820", 2022-06-17T12:52:09.748 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:09.748 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.748 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.749 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.749 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6821", 2022-06-17T12:52:09.750 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:09.750 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.750 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.750 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.751 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6817/22822", 2022-06-17T12:52:09.751 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6819/22822", 2022-06-17T12:52:09.751 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6823/22822", 2022-06-17T12:52:09.751 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6821/22822", 2022-06-17T12:52:09.751 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:09.752 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:09.752 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:09.752 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.752 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.752 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.753 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:52:09.753 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", 2022-06-17T12:52:09.753 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:09.753 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:09.754 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:09.754 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:09.754 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:09.754 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:09.755 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:52:09.755 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 51, 2022-06-17T12:52:09.755 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:09.755 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:09.755 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:09.756 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.756 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.756 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.757 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6824", 2022-06-17T12:52:09.757 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:09.758 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.758 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.759 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.759 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6825", 2022-06-17T12:52:09.759 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:09.760 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.760 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.760 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.760 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:09.761 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.761 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.761 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.761 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6826", 2022-06-17T12:52:09.762 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:09.762 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.763 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.763 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.764 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6827", 2022-06-17T12:52:09.764 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:09.764 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.765 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.766 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.766 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:09.766 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.767 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.767 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.768 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6830", 2022-06-17T12:52:09.768 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:09.768 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.769 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.769 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.770 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6831", 2022-06-17T12:52:09.772 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:09.772 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.773 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.773 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.774 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:09.774 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:09.775 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.775 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:09.775 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6828", 2022-06-17T12:52:09.776 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:09.776 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.776 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.777 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:09.777 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6829", 2022-06-17T12:52:09.778 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:09.778 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.778 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.779 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.779 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6825/22837", 2022-06-17T12:52:09.779 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6827/22837", 2022-06-17T12:52:09.780 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6831/22837", 2022-06-17T12:52:09.780 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6829/22837", 2022-06-17T12:52:09.781 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:09.781 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:09.781 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:09.782 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:09.783 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.783 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:52:09.784 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_xinfo": [ 2022-06-17T12:52:09.784 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.785 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:52:09.785 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:09.786 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:09.786 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:09.786 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:09.787 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:09.787 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.291461+0000", 2022-06-17T12:52:09.788 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:09.794 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.794 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.794 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:52:09.795 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:09.795 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:09.795 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:09.795 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:09.796 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:09.796 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.537912+0000", 2022-06-17T12:52:09.796 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:09.796 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.797 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.797 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:52:09.797 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:09.797 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:09.797 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:09.803 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:09.803 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:09.804 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.606947+0000", 2022-06-17T12:52:09.805 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:09.805 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.805 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.805 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:52:09.806 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:09.806 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:09.806 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:09.806 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:09.807 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:09.807 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.514412+0000", 2022-06-17T12:52:09.807 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:09.808 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.808 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.808 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:52:09.808 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:09.809 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:09.809 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:09.809 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:09.815 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:09.815 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.394645+0000", 2022-06-17T12:52:09.815 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:09.815 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.816 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.816 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:52:09.816 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:09.816 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:09.822 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:09.823 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:09.823 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:09.824 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.343806+0000", 2022-06-17T12:52:09.824 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:09.824 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.825 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.825 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:52:09.826 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:09.826 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:09.827 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:09.827 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:09.828 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:09.828 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.591864+0000", 2022-06-17T12:52:09.829 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:09.829 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.829 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:09.830 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:52:09.830 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:09.832 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:09.833 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:09.833 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:09.834 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:09.836 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.607045+0000", 2022-06-17T12:52:09.836 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:09.837 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.837 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:52:09.838 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap": [], 2022-06-17T12:52:09.839 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap_items": [], 2022-06-17T12:52:09.839 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_temp": [], 2022-06-17T12:52:09.839 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_temp": [], 2022-06-17T12:52:09.839 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklist": {}, 2022-06-17T12:52:09.840 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profiles": { 2022-06-17T12:52:09.840 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs_data": { 2022-06-17T12:52:09.840 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-device-class": "", 2022-06-17T12:52:09.841 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:52:09.841 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-root": "default", 2022-06-17T12:52:09.841 INFO:teuthology.orchestra.run.smithi036.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-17T12:52:09.842 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:52:09.842 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "2", 2022-06-17T12:52:09.842 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:52:09.842 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:52:09.843 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van", 2022-06-17T12:52:09.843 INFO:teuthology.orchestra.run.smithi036.stdout: "w": "8" 2022-06-17T12:52:09.843 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.844 INFO:teuthology.orchestra.run.smithi036.stdout: "default": { 2022-06-17T12:52:09.844 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:52:09.844 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:52:09.844 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "1", 2022-06-17T12:52:09.845 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:52:09.845 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:52:09.845 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van" 2022-06-17T12:52:09.846 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:09.846 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:09.847 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps_queue": [], 2022-06-17T12:52:09.847 INFO:teuthology.orchestra.run.smithi036.stdout: "new_removed_snaps": [], 2022-06-17T12:52:09.847 INFO:teuthology.orchestra.run.smithi036.stdout: "new_purged_snaps": [], 2022-06-17T12:52:09.847 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_node_flags": {}, 2022-06-17T12:52:09.848 INFO:teuthology.orchestra.run.smithi036.stdout: "device_class_flags": {} 2022-06-17T12:52:09.848 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:52:09.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.521+0000 7effb952e700 1 -- 172.21.15.36:0/3004732021 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7eff8c053a40 msgr2=0x7eff8c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:09.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.521+0000 7effb952e700 1 --2- 172.21.15.36:0/3004732021 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7eff8c053a40 0x7eff8c055f00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7effac00b000 tx=0x7effac0091d0).stop 2022-06-17T12:52:09.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.521+0000 7effb952e700 1 -- 172.21.15.36:0/3004732021 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7effb40ac340 msgr2=0x7effb4063460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:09.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.521+0000 7effb952e700 1 --2- 172.21.15.36:0/3004732021 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7effb40ac340 0x7effb4063460 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7eff9c002f70 tx=0x7eff9c014000).stop 2022-06-17T12:52:09.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.521+0000 7effb952e700 1 -- 172.21.15.36:0/3004732021 shutdown_connections 2022-06-17T12:52:09.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.521+0000 7effb952e700 1 --2- 172.21.15.36:0/3004732021 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7eff8c053a40 0x7eff8c055f00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:09.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.521+0000 7effb952e700 1 --2- 172.21.15.36:0/3004732021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effb414aca0 0x7effb4155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:09.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.521+0000 7effb952e700 1 --2- 172.21.15.36:0/3004732021 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7effb40ad140 0x7effb4063950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:09.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.521+0000 7effb952e700 1 --2- 172.21.15.36:0/3004732021 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7effb40ac340 0x7effb4063460 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:09.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.521+0000 7effb952e700 1 -- 172.21.15.36:0/3004732021 >> 172.21.15.36:0/3004732021 conn(0x7effb40b5430 msgr2=0x7effb414b150 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:09.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.521+0000 7effb952e700 1 -- 172.21.15.36:0/3004732021 shutdown_connections 2022-06-17T12:52:09.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.521+0000 7effb952e700 1 -- 172.21.15.36:0/3004732021 wait complete. 2022-06-17T12:52:09.855 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:52:09.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.985+0000 7f0d28f3a700 1 Processor -- start 2022-06-17T12:52:09.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.985+0000 7f0d28f3a700 1 -- start start 2022-06-17T12:52:09.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d28f3a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d240b0cb0 0x7f0d24142f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:09.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d28f3a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0d24143e80 0x7f0d24146970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:09.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d28f3a700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d240b44b0 con 0x7f0d24062270 2022-06-17T12:52:09.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d28f3a700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d240b6650 con 0x7f0d24143e80 2022-06-17T12:52:09.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d28f3a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d24144300 con 0x7f0d240b0cb0 2022-06-17T12:52:09.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d22d9d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0d24143e80 0x7f0d24146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:09.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d2259c700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f0d24062270 0x7f0d24143770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52028/0 (socket says 172.21.15.36:52028) 2022-06-17T12:52:09.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d21d9b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d240b0cb0 0x7f0d24142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:09.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d22d9d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0d24143e80 0x7f0d24146970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34500/0 (socket says 172.21.15.36:34500) 2022-06-17T12:52:09.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d2259c700 1 -- 172.21.15.36:0/593727329 learned_addr learned my addr 172.21.15.36:0/593727329 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:09.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d22d9d700 1 -- 172.21.15.36:0/593727329 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d240b0cb0 msgr2=0x7f0d24142f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:09.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d22d9d700 1 --2- 172.21.15.36:0/593727329 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d240b0cb0 0x7f0d24142f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:09.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d22d9d700 1 -- 172.21.15.36:0/593727329 >> v1:172.21.15.36:6789/0 conn(0x7f0d24062270 legacy=0x7f0d24143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:09.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d2159a700 1 -- 172.21.15.36:0/593727329 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725256887 0 0) 0x7f0d240b44b0 con 0x7f0d24062270 2022-06-17T12:52:09.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d22d9d700 1 -- 172.21.15.36:0/593727329 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d24062650 con 0x7f0d24143e80 2022-06-17T12:52:09.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d22d9d700 1 --2- 172.21.15.36:0/593727329 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0d24143e80 0x7f0d24146970 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f0d1c0032f0 tx=0x7f0d1c0091d0).ready entity=mon.0 client_cookie=c94b5e45c10a2f8e server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:09.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d2159a700 1 -- 172.21.15.36:0/593727329 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d1c00b6c0 con 0x7f0d24143e80 2022-06-17T12:52:09.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d2159a700 1 -- 172.21.15.36:0/593727329 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1c00b820 con 0x7f0d24143e80 2022-06-17T12:52:09.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d2159a700 1 -- 172.21.15.36:0/593727329 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d1c00b990 con 0x7f0d24143e80 2022-06-17T12:52:09.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d28f3a700 1 -- 172.21.15.36:0/593727329 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0d24143e80 msgr2=0x7f0d24146970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:09.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d28f3a700 1 --2- 172.21.15.36:0/593727329 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0d24143e80 0x7f0d24146970 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f0d1c0032f0 tx=0x7f0d1c0091d0).stop 2022-06-17T12:52:09.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d28f3a700 1 -- 172.21.15.36:0/593727329 shutdown_connections 2022-06-17T12:52:09.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d28f3a700 1 --2- 172.21.15.36:0/593727329 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0d24143e80 0x7f0d24146970 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:09.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d28f3a700 1 --2- 172.21.15.36:0/593727329 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d240b0cb0 0x7f0d24142f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:09.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d28f3a700 1 -- 172.21.15.36:0/593727329 >> 172.21.15.36:0/593727329 conn(0x7f0d240b9870 msgr2=0x7f0d240b9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:09.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d28f3a700 1 -- 172.21.15.36:0/593727329 shutdown_connections 2022-06-17T12:52:09.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.989+0000 7f0d28f3a700 1 -- 172.21.15.36:0/593727329 wait complete. 2022-06-17T12:52:09.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.993+0000 7f0d28f3a700 1 Processor -- start 2022-06-17T12:52:09.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.993+0000 7f0d28f3a700 1 -- start start 2022-06-17T12:52:09.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.993+0000 7f0d28f3a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d24062270 0x7f0d240ae160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:09.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.993+0000 7f0d28f3a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d24143e80 0x7f0d240b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:09.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.993+0000 7f0d28f3a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0d24144530 0x7f0d24146f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:09.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.993+0000 7f0d28f3a700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2413de40 con 0x7f0d24144530 2022-06-17T12:52:09.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.993+0000 7f0d28f3a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d24145ed0 con 0x7f0d24143e80 2022-06-17T12:52:09.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.993+0000 7f0d28f3a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d24146100 con 0x7f0d24062270 2022-06-17T12:52:09.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.993+0000 7f0d22d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0d24144530 0x7f0d24146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:09.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.993+0000 7f0d22d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0d24144530 0x7f0d24146f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34506/0 (socket says 172.21.15.36:34506) 2022-06-17T12:52:09.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.993+0000 7f0d22d9d700 1 -- 172.21.15.36:0/314750213 learned_addr learned my addr 172.21.15.36:0/314750213 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:09.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.993+0000 7f0d2259c700 1 --2- 172.21.15.36:0/314750213 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d24062270 0x7f0d240ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:09.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.993+0000 7f0d21d9b700 1 --2- 172.21.15.36:0/314750213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d24143e80 0x7f0d240b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:09.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.993+0000 7f0d22d9d700 1 -- 172.21.15.36:0/314750213 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d24062270 msgr2=0x7f0d240ae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:09.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.993+0000 7f0d22d9d700 1 --2- 172.21.15.36:0/314750213 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d24062270 0x7f0d240ae160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:09.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.993+0000 7f0d22d9d700 1 -- 172.21.15.36:0/314750213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d24143e80 msgr2=0x7f0d240b8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:09.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.993+0000 7f0d22d9d700 1 --2- 172.21.15.36:0/314750213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d24143e80 0x7f0d240b8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:10.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.993+0000 7f0d22d9d700 1 -- 172.21.15.36:0/314750213 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c009470 con 0x7f0d24144530 2022-06-17T12:52:10.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.993+0000 7f0d22d9d700 1 --2- 172.21.15.36:0/314750213 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0d24144530 0x7f0d24146f30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f0d1c0032f0 tx=0x7f0d1c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:10.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.993+0000 7f0d137fe700 1 -- 172.21.15.36:0/314750213 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d1c00b6c0 con 0x7f0d24144530 2022-06-17T12:52:10.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.993+0000 7f0d137fe700 1 -- 172.21.15.36:0/314750213 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1c0205f0 con 0x7f0d24144530 2022-06-17T12:52:10.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.993+0000 7f0d28f3a700 1 -- 172.21.15.36:0/314750213 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d24146a90 con 0x7f0d24144530 2022-06-17T12:52:10.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.993+0000 7f0d137fe700 1 -- 172.21.15.36:0/314750213 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d1c020950 con 0x7f0d24144530 2022-06-17T12:52:10.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.993+0000 7f0d28f3a700 1 -- 172.21.15.36:0/314750213 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d240ad3e0 con 0x7f0d24144530 2022-06-17T12:52:10.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.997+0000 7f0d28f3a700 1 -- 172.21.15.36:0/314750213 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d24147760 con 0x7f0d24144530 2022-06-17T12:52:10.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.997+0000 7f0d137fe700 1 -- 172.21.15.36:0/314750213 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f0d1c01e070 con 0x7f0d24144530 2022-06-17T12:52:10.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.997+0000 7f0d137fe700 1 --2- 172.21.15.36:0/314750213 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0d00053a40 0x7f0d00055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:10.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.997+0000 7f0d137fe700 1 -- 172.21.15.36:0/314750213 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0d1c06cfc0 con 0x7f0d24144530 2022-06-17T12:52:10.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.997+0000 7f0d2259c700 1 --2- 172.21.15.36:0/314750213 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0d00053a40 0x7f0d00055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:10.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:09.997+0000 7f0d2259c700 1 --2- 172.21.15.36:0/314750213 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0d00053a40 0x7f0d00055f00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f0d1800d000 tx=0x7f0d18007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:10.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.001+0000 7f0d137fe700 1 -- 172.21.15.36:0/314750213 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0d1c043cf0 con 0x7f0d24144530 2022-06-17T12:52:10.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.169+0000 7f0d28f3a700 1 -- 172.21.15.36:0/314750213 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d24146c20 con 0x7f0d24144530 2022-06-17T12:52:10.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.169+0000 7f0d137fe700 1 -- 172.21.15.36:0/314750213 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f0d1c015020 con 0x7f0d24144530 2022-06-17T12:52:10.154 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:52:10.155 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":25,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5670,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.36:6835/2598001220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2598001220},{"type":"v1","addr":"172.21.15.36:6835","nonce":2598001220}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":23}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:50:49.096455+0000","modified":"2022-06-17T12:50:50.099539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"b","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/2723288138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2723288138},{"type":"v1","addr":"172.21.15.70:6833","nonce":2723288138}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-06-17T12:52:10.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.173+0000 7f0d28f3a700 1 -- 172.21.15.36:0/314750213 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0d00053a40 msgr2=0x7f0d00055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:10.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.173+0000 7f0d28f3a700 1 --2- 172.21.15.36:0/314750213 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0d00053a40 0x7f0d00055f00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f0d1800d000 tx=0x7f0d18007a00).stop 2022-06-17T12:52:10.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.173+0000 7f0d28f3a700 1 -- 172.21.15.36:0/314750213 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0d24144530 msgr2=0x7f0d24146f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:10.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.173+0000 7f0d28f3a700 1 --2- 172.21.15.36:0/314750213 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0d24144530 0x7f0d24146f30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f0d1c0032f0 tx=0x7f0d1c016000).stop 2022-06-17T12:52:10.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.173+0000 7f0d28f3a700 1 -- 172.21.15.36:0/314750213 shutdown_connections 2022-06-17T12:52:10.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.173+0000 7f0d28f3a700 1 --2- 172.21.15.36:0/314750213 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0d00053a40 0x7f0d00055f00 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:10.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.173+0000 7f0d28f3a700 1 --2- 172.21.15.36:0/314750213 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0d24144530 0x7f0d24146f30 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:10.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.173+0000 7f0d28f3a700 1 --2- 172.21.15.36:0/314750213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d24143e80 0x7f0d240b8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:10.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.173+0000 7f0d28f3a700 1 --2- 172.21.15.36:0/314750213 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d24062270 0x7f0d240ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:10.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.173+0000 7f0d28f3a700 1 -- 172.21.15.36:0/314750213 >> 172.21.15.36:0/314750213 conn(0x7f0d240b9870 msgr2=0x7f0d24141670 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:10.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.173+0000 7f0d28f3a700 1 -- 172.21.15.36:0/314750213 shutdown_connections 2022-06-17T12:52:10.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.173+0000 7f0d28f3a700 1 -- 172.21.15.36:0/314750213 wait complete. 2022-06-17T12:52:10.164 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 25 2022-06-17T12:52:10.168 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-17T12:52:10.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.293+0000 7f8b59c2f700 1 Processor -- start 2022-06-17T12:52:10.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.293+0000 7f8b59c2f700 1 -- start start 2022-06-17T12:52:10.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.293+0000 7f8b59c2f700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8b540ad330 0x7f8b54129150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:10.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.293+0000 7f8b59c2f700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b54140930 con 0x7f8b540ad840 2022-06-17T12:52:10.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.293+0000 7f8b59c2f700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b5405fd80 con 0x7f8b54063570 2022-06-17T12:52:10.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.293+0000 7f8b59c2f700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b54140d90 con 0x7f8b540ad330 2022-06-17T12:52:10.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.293+0000 7f8b53fff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f8b540ad840 0x7f8b540adc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52044/0 (socket says 172.21.15.36:52044) 2022-06-17T12:52:10.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.293+0000 7f8b53fff700 1 -- 172.21.15.36:0/1821098605 learned_addr learned my addr 172.21.15.36:0/1821098605 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:10.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.293+0000 7f8b52ffd700 1 --2- 172.21.15.36:0/1821098605 >> v2:172.21.15.70:3300/0 conn(0x7f8b540ad330 0x7f8b54129150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:10.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.293+0000 7f8b527fc700 1 -- 172.21.15.36:0/1821098605 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2782398681 0 0) 0x7f8b54140930 con 0x7f8b540ad840 2022-06-17T12:52:10.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.293+0000 7f8b527fc700 1 -- 172.21.15.36:0/1821098605 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b40003260 con 0x7f8b540ad840 2022-06-17T12:52:10.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.293+0000 7f8b52ffd700 1 -- 172.21.15.36:0/1821098605 >> v1:172.21.15.70:6789/0 conn(0x7f8b54063570 legacy=0x7f8b54128a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:10.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.293+0000 7f8b52ffd700 1 -- 172.21.15.36:0/1821098605 >> v1:172.21.15.36:6789/0 conn(0x7f8b540ad840 legacy=0x7f8b540adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:10.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.293+0000 7f8b527fc700 1 -- 172.21.15.36:0/1821098605 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 953605237 0 0) 0x7f8b40003260 con 0x7f8b540ad840 2022-06-17T12:52:10.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.293+0000 7f8b52ffd700 1 -- 172.21.15.36:0/1821098605 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b54063c30 con 0x7f8b540ad330 2022-06-17T12:52:10.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.293+0000 7f8b52ffd700 1 --2- 172.21.15.36:0/1821098605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b540ad330 0x7f8b54129150 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f8b4c001ba0 tx=0x7f8b48006df0).ready entity=mon.1 client_cookie=a5965bd14e753474 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:10.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.293+0000 7f8b527fc700 1 -- 172.21.15.36:0/1821098605 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b4c001a00 con 0x7f8b540ad330 2022-06-17T12:52:10.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.293+0000 7f8b527fc700 1 -- 172.21.15.36:0/1821098605 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b48000e90 con 0x7f8b540ad330 2022-06-17T12:52:10.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.293+0000 7f8b527fc700 1 -- 172.21.15.36:0/1821098605 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b4800d4f0 con 0x7f8b540ad330 2022-06-17T12:52:10.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b59c2f700 1 -- 172.21.15.36:0/1821098605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b540ad330 msgr2=0x7f8b54129150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:10.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b59c2f700 1 --2- 172.21.15.36:0/1821098605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b540ad330 0x7f8b54129150 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f8b4c001ba0 tx=0x7f8b48006df0).stop 2022-06-17T12:52:10.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b59c2f700 1 -- 172.21.15.36:0/1821098605 shutdown_connections 2022-06-17T12:52:10.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b59c2f700 1 --2- 172.21.15.36:0/1821098605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b540ad330 0x7f8b54129150 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:10.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b59c2f700 1 -- 172.21.15.36:0/1821098605 >> 172.21.15.36:0/1821098605 conn(0x7f8b540b53e0 msgr2=0x7f8b540b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:10.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b59c2f700 1 -- 172.21.15.36:0/1821098605 shutdown_connections 2022-06-17T12:52:10.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b59c2f700 1 -- 172.21.15.36:0/1821098605 wait complete. 2022-06-17T12:52:10.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b59c2f700 1 Processor -- start 2022-06-17T12:52:10.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b59c2f700 1 -- start start 2022-06-17T12:52:10.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b59c2f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b54063570 0x7f8b54143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:10.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b59c2f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b540ad330 0x7f8b540651d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:10.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b59c2f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b540ad840 0x7f8b541415f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:10.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b59c2f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b54144260 con 0x7f8b54063570 2022-06-17T12:52:10.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b59c2f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5413e0b0 con 0x7f8b540ad330 2022-06-17T12:52:10.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b59c2f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5413e350 con 0x7f8b540ad840 2022-06-17T12:52:10.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b537fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b54063570 0x7f8b54143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:10.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b537fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b54063570 0x7f8b54143510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34518/0 (socket says 172.21.15.36:34518) 2022-06-17T12:52:10.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b537fe700 1 -- 172.21.15.36:0/1436108017 learned_addr learned my addr 172.21.15.36:0/1436108017 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:10.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b52ffd700 1 --2- 172.21.15.36:0/1436108017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b540ad330 0x7f8b540651d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:10.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b53fff700 1 --2- 172.21.15.36:0/1436108017 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b540ad840 0x7f8b541415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:10.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b537fe700 1 -- 172.21.15.36:0/1436108017 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b540ad840 msgr2=0x7f8b541415f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:10.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b537fe700 1 --2- 172.21.15.36:0/1436108017 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b540ad840 0x7f8b541415f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:10.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b537fe700 1 -- 172.21.15.36:0/1436108017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b540ad330 msgr2=0x7f8b540651d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:10.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b537fe700 1 --2- 172.21.15.36:0/1436108017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b540ad330 0x7f8b540651d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:10.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.297+0000 7f8b537fe700 1 -- 172.21.15.36:0/1436108017 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4800b180 con 0x7f8b54063570 2022-06-17T12:52:10.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.301+0000 7f8b537fe700 1 --2- 172.21.15.36:0/1436108017 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b54063570 0x7f8b54143510 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f8b5405d350 tx=0x7f8b44007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:10.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.301+0000 7f8b50ff9700 1 -- 172.21.15.36:0/1436108017 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b44012400 con 0x7f8b54063570 2022-06-17T12:52:10.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.301+0000 7f8b50ff9700 1 -- 172.21.15.36:0/1436108017 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b44012560 con 0x7f8b54063570 2022-06-17T12:52:10.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.301+0000 7f8b59c2f700 1 -- 172.21.15.36:0/1436108017 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b5413e950 con 0x7f8b54063570 2022-06-17T12:52:10.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.301+0000 7f8b50ff9700 1 -- 172.21.15.36:0/1436108017 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b440128c0 con 0x7f8b54063570 2022-06-17T12:52:10.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.301+0000 7f8b59c2f700 1 -- 172.21.15.36:0/1436108017 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5413dba0 con 0x7f8b54063570 2022-06-17T12:52:10.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.301+0000 7f8b50ff9700 1 -- 172.21.15.36:0/1436108017 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8b44019030 con 0x7f8b54063570 2022-06-17T12:52:10.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.301+0000 7f8b59c2f700 1 -- 172.21.15.36:0/1436108017 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b5413eae0 con 0x7f8b54063570 2022-06-17T12:52:10.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.301+0000 7f8b50ff9700 1 --2- 172.21.15.36:0/1436108017 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b34053920 0x7f8b34055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:10.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.301+0000 7f8b50ff9700 1 -- 172.21.15.36:0/1436108017 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8b4406b2d0 con 0x7f8b54063570 2022-06-17T12:52:10.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.301+0000 7f8b52ffd700 1 --2- 172.21.15.36:0/1436108017 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b34053920 0x7f8b34055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:10.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.301+0000 7f8b52ffd700 1 --2- 172.21.15.36:0/1436108017 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b34053920 0x7f8b34055de0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f8b4801a000 tx=0x7f8b48014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:10.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.305+0000 7f8b50ff9700 1 -- 172.21.15.36:0/1436108017 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8b440420d0 con 0x7f8b54063570 2022-06-17T12:52:10.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.469+0000 7f8b59c2f700 1 -- 172.21.15.36:0/1436108017 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8b5413deb0 con 0x7f8b54063570 2022-06-17T12:52:10.639 INFO:tasks.ceph.mon.a.smithi036.stderr:2022-06-17T12:52:10.657+0000 7fb7a2649700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-17T12:52:10.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.661+0000 7f8b50ff9700 1 -- 172.21.15.36:0/1436108017 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8b440456f0 con 0x7f8b54063570 2022-06-17T12:52:10.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.665+0000 7f8b59c2f700 1 -- 172.21.15.36:0/1436108017 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b34053920 msgr2=0x7f8b34055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:10.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.665+0000 7f8b59c2f700 1 --2- 172.21.15.36:0/1436108017 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b34053920 0x7f8b34055de0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f8b4801a000 tx=0x7f8b48014000).stop 2022-06-17T12:52:10.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.665+0000 7f8b59c2f700 1 -- 172.21.15.36:0/1436108017 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b54063570 msgr2=0x7f8b54143510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:10.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.665+0000 7f8b59c2f700 1 --2- 172.21.15.36:0/1436108017 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b54063570 0x7f8b54143510 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f8b5405d350 tx=0x7f8b44007c50).stop 2022-06-17T12:52:10.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.665+0000 7f8b59c2f700 1 -- 172.21.15.36:0/1436108017 shutdown_connections 2022-06-17T12:52:10.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.665+0000 7f8b59c2f700 1 --2- 172.21.15.36:0/1436108017 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b34053920 0x7f8b34055de0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:10.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.665+0000 7f8b59c2f700 1 --2- 172.21.15.36:0/1436108017 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b540ad840 0x7f8b541415f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:10.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.665+0000 7f8b59c2f700 1 --2- 172.21.15.36:0/1436108017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b540ad330 0x7f8b540651d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:10.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.665+0000 7f8b59c2f700 1 --2- 172.21.15.36:0/1436108017 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8b54063570 0x7f8b54143510 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:10.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.665+0000 7f8b59c2f700 1 -- 172.21.15.36:0/1436108017 >> 172.21.15.36:0/1436108017 conn(0x7f8b540b53e0 msgr2=0x7f8b540b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:10.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.665+0000 7f8b59c2f700 1 -- 172.21.15.36:0/1436108017 shutdown_connections 2022-06-17T12:52:10.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.665+0000 7f8b59c2f700 1 -- 172.21.15.36:0/1436108017 wait complete. 2022-06-17T12:52:10.658 INFO:teuthology.orchestra.run.smithi036.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-17T12:52:10.671 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:52:10.685 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2022-06-17T12:52:10.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.809+0000 7fbdb56ce700 1 Processor -- start 2022-06-17T12:52:10.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.809+0000 7fbdb56ce700 1 -- start start 2022-06-17T12:52:10.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.809+0000 7fbdb56ce700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbdb00b3730 0x7fbdb014b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:10.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.809+0000 7fbdb56ce700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb0060ea0 con 0x7fbdb0060ac0 2022-06-17T12:52:10.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.809+0000 7fbdb56ce700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb00af7b0 con 0x7fbdb014b670 2022-06-17T12:52:10.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.809+0000 7fbdb56ce700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb00afa50 con 0x7fbdb00b3730 2022-06-17T12:52:10.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.809+0000 7fbdae7fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbdb00b3730 0x7fbdb014b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:10.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.809+0000 7fbdae7fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbdb00b3730 0x7fbdb014b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34524/0 (socket says 172.21.15.36:34524) 2022-06-17T12:52:10.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.809+0000 7fbdaeffd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fbdb0060ac0 0x7fbdb014aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52054/0 (socket says 172.21.15.36:52054) 2022-06-17T12:52:10.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.809+0000 7fbdae7fc700 1 -- 172.21.15.36:0/2533609455 learned_addr learned my addr 172.21.15.36:0/2533609455 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:10.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.809+0000 7fbdae7fc700 1 -- 172.21.15.36:0/2533609455 >> v1:172.21.15.70:6789/0 conn(0x7fbdb014b670 legacy=0x7fbdb01477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:10.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.809+0000 7fbdae7fc700 1 -- 172.21.15.36:0/2533609455 >> v1:172.21.15.36:6789/0 conn(0x7fbdb0060ac0 legacy=0x7fbdb014aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:10.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.809+0000 7fbdae7fc700 1 -- 172.21.15.36:0/2533609455 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb00afcf0 con 0x7fbdb00b3730 2022-06-17T12:52:10.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.809+0000 7fbdae7fc700 1 --2- 172.21.15.36:0/2533609455 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbdb00b3730 0x7fbdb014b180 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7fbda0002f70 tx=0x7fbda0006eb0).ready entity=mon.0 client_cookie=b4cbb353ffa898ec server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:10.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.809+0000 7fbdadffb700 1 -- 172.21.15.36:0/2533609455 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbda000d660 con 0x7fbdb00b3730 2022-06-17T12:52:10.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.809+0000 7fbdadffb700 1 -- 172.21.15.36:0/2533609455 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda0000e90 con 0x7fbdb00b3730 2022-06-17T12:52:10.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.809+0000 7fbdadffb700 1 -- 172.21.15.36:0/2533609455 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbda000d9f0 con 0x7fbdb00b3730 2022-06-17T12:52:10.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.809+0000 7fbdb56ce700 1 -- 172.21.15.36:0/2533609455 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbdb00b3730 msgr2=0x7fbdb014b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:10.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.809+0000 7fbdb56ce700 1 --2- 172.21.15.36:0/2533609455 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbdb00b3730 0x7fbdb014b180 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7fbda0002f70 tx=0x7fbda0006eb0).stop 2022-06-17T12:52:10.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdb56ce700 1 -- 172.21.15.36:0/2533609455 shutdown_connections 2022-06-17T12:52:10.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdb56ce700 1 --2- 172.21.15.36:0/2533609455 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbdb00b3730 0x7fbdb014b180 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:10.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdb56ce700 1 -- 172.21.15.36:0/2533609455 >> 172.21.15.36:0/2533609455 conn(0x7fbdb00b6870 msgr2=0x7fbdb00b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:10.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdb56ce700 1 -- 172.21.15.36:0/2533609455 shutdown_connections 2022-06-17T12:52:10.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdb56ce700 1 -- 172.21.15.36:0/2533609455 wait complete. 2022-06-17T12:52:10.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdb56ce700 1 Processor -- start 2022-06-17T12:52:10.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdb56ce700 1 -- start start 2022-06-17T12:52:10.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdb56ce700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbdb0060ac0 0x7fbdb0062e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:10.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdb56ce700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdb00b3730 0x7fbdb01446f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:10.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdb56ce700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdb014b670 0x7fbdb0144be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:10.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdb56ce700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb0067bb0 con 0x7fbdb0060ac0 2022-06-17T12:52:10.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdb56ce700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb0063370 con 0x7fbdb00b3730 2022-06-17T12:52:10.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdb56ce700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb0067470 con 0x7fbdb014b670 2022-06-17T12:52:10.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdaeffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbdb0060ac0 0x7fbdb0062e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:10.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdaeffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbdb0060ac0 0x7fbdb0062e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34530/0 (socket says 172.21.15.36:34530) 2022-06-17T12:52:10.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdaeffd700 1 -- 172.21.15.36:0/108333882 learned_addr learned my addr 172.21.15.36:0/108333882 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:10.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdae7fc700 1 --2- 172.21.15.36:0/108333882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdb00b3730 0x7fbdb01446f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:10.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdaf7fe700 1 --2- 172.21.15.36:0/108333882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdb014b670 0x7fbdb0144be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:10.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdaeffd700 1 -- 172.21.15.36:0/108333882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdb014b670 msgr2=0x7fbdb0144be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:10.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdaeffd700 1 --2- 172.21.15.36:0/108333882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdb014b670 0x7fbdb0144be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:10.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdaeffd700 1 -- 172.21.15.36:0/108333882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdb00b3730 msgr2=0x7fbdb01446f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:10.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdaeffd700 1 --2- 172.21.15.36:0/108333882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdb00b3730 0x7fbdb01446f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:10.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdaeffd700 1 -- 172.21.15.36:0/108333882 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda000b1e0 con 0x7fbdb0060ac0 2022-06-17T12:52:10.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdaf7fe700 1 --2- 172.21.15.36:0/108333882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdb014b670 0x7fbdb0144be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:52:10.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.813+0000 7fbdae7fc700 1 --2- 172.21.15.36:0/108333882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdb00b3730 0x7fbdb01446f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:52:10.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.817+0000 7fbdaeffd700 1 --2- 172.21.15.36:0/108333882 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbdb0060ac0 0x7fbdb0062e80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7fbdb00634b0 tx=0x7fbd98007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:10.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.817+0000 7fbda7fff700 1 -- 172.21.15.36:0/108333882 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd9800ce20 con 0x7fbdb0060ac0 2022-06-17T12:52:10.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.817+0000 7fbda7fff700 1 -- 172.21.15.36:0/108333882 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd980072e0 con 0x7fbdb0060ac0 2022-06-17T12:52:10.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.817+0000 7fbdb56ce700 1 -- 172.21.15.36:0/108333882 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb0067710 con 0x7fbdb0060ac0 2022-06-17T12:52:10.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.817+0000 7fbda7fff700 1 -- 172.21.15.36:0/108333882 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd98007450 con 0x7fbdb0060ac0 2022-06-17T12:52:10.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.817+0000 7fbdb56ce700 1 -- 172.21.15.36:0/108333882 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb015a1c0 con 0x7fbdb0060ac0 2022-06-17T12:52:10.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.817+0000 7fbda7fff700 1 -- 172.21.15.36:0/108333882 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fbd9801a070 con 0x7fbdb0060ac0 2022-06-17T12:52:10.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.817+0000 7fbdb56ce700 1 -- 172.21.15.36:0/108333882 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb00634f0 con 0x7fbdb0060ac0 2022-06-17T12:52:10.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.817+0000 7fbda7fff700 1 --2- 172.21.15.36:0/108333882 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbd94053a60 0x7fbd94055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:10.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.817+0000 7fbda7fff700 1 -- 172.21.15.36:0/108333882 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fbd9806b020 con 0x7fbdb0060ac0 2022-06-17T12:52:10.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.817+0000 7fbdae7fc700 1 --2- 172.21.15.36:0/108333882 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbd94053a60 0x7fbd94055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:10.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.817+0000 7fbdae7fc700 1 --2- 172.21.15.36:0/108333882 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbd94053a60 0x7fbd94055f20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7fbda001a000 tx=0x7fbda0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:10.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.825+0000 7fbda7fff700 1 -- 172.21.15.36:0/108333882 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbd98041dd0 con 0x7fbdb0060ac0 2022-06-17T12:52:10.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.989+0000 7fbdb56ce700 1 -- 172.21.15.36:0/108333882 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbdb00678a0 con 0x7fbdb0060ac0 2022-06-17T12:52:10.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.993+0000 7fbda7fff700 1 -- 172.21.15.36:0/108333882 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+1778 (secure 0 0 0) 0x7fbd980453f0 con 0x7fbdb0060ac0 2022-06-17T12:52:10.976 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:52:10.977 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":26,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5670,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.36:6835/2598001220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2598001220},{"type":"v1","addr":"172.21.15.36:6835","nonce":2598001220}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":23}],"filesystems":[{"mdsmap":{"epoch":26,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:50:49.096455+0000","modified":"2022-06-17T12:52:10.658586+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-06-17T12:52:10.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.993+0000 7fbdb56ce700 1 -- 172.21.15.36:0/108333882 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbd94053a60 msgr2=0x7fbd94055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:10.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.993+0000 7fbdb56ce700 1 --2- 172.21.15.36:0/108333882 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbd94053a60 0x7fbd94055f20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7fbda001a000 tx=0x7fbda0015000).stop 2022-06-17T12:52:10.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.993+0000 7fbdb56ce700 1 -- 172.21.15.36:0/108333882 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbdb0060ac0 msgr2=0x7fbdb0062e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:10.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.993+0000 7fbdb56ce700 1 --2- 172.21.15.36:0/108333882 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbdb0060ac0 0x7fbdb0062e80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7fbdb00634b0 tx=0x7fbd98007c40).stop 2022-06-17T12:52:10.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.993+0000 7fbdb56ce700 1 -- 172.21.15.36:0/108333882 shutdown_connections 2022-06-17T12:52:10.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.993+0000 7fbdb56ce700 1 --2- 172.21.15.36:0/108333882 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbd94053a60 0x7fbd94055f20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:10.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.993+0000 7fbdb56ce700 1 --2- 172.21.15.36:0/108333882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdb014b670 0x7fbdb0144be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:10.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.993+0000 7fbdb56ce700 1 --2- 172.21.15.36:0/108333882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdb00b3730 0x7fbdb01446f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:10.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.993+0000 7fbdb56ce700 1 --2- 172.21.15.36:0/108333882 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbdb0060ac0 0x7fbdb0062e80 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:10.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.993+0000 7fbdb56ce700 1 -- 172.21.15.36:0/108333882 >> 172.21.15.36:0/108333882 conn(0x7fbdb00b6870 msgr2=0x7fbdb00b5090 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:10.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.993+0000 7fbdb56ce700 1 -- 172.21.15.36:0/108333882 shutdown_connections 2022-06-17T12:52:10.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:10.993+0000 7fbdb56ce700 1 -- 172.21.15.36:0/108333882 wait complete. 2022-06-17T12:52:10.986 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 26 2022-06-17T12:52:10.989 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-17T12:52:11.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.117+0000 7f8c98966700 1 Processor -- start 2022-06-17T12:52:11.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.117+0000 7f8c98966700 1 -- start start 2022-06-17T12:52:11.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.117+0000 7f8c98966700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8c940b27b0 0x7f8c9414b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:11.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.117+0000 7f8c98966700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8c94060ca0 0x7f8c940ac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:11.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.117+0000 7f8c98966700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c9414b870 con 0x7f8c94147bd0 2022-06-17T12:52:11.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.117+0000 7f8c98966700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c940ac500 con 0x7f8c94060ca0 2022-06-17T12:52:11.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.117+0000 7f8c98966700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c940b2b90 con 0x7f8c940b27b0 2022-06-17T12:52:11.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.117+0000 7f8c92d9d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8c940b27b0 0x7f8c9414b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:11.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.117+0000 7f8c9259c700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8c94060ca0 0x7f8c940ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:11.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.117+0000 7f8c9259c700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8c94060ca0 0x7f8c940ac780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:54920/0 (socket says 172.21.15.36:54920) 2022-06-17T12:52:11.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.117+0000 7f8c9259c700 1 -- 172.21.15.36:0/1499882962 learned_addr learned my addr 172.21.15.36:0/1499882962 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:11.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.117+0000 7f8c9359e700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8c94147bd0 0x7f8c941442d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:47804/0 (socket says 172.21.15.36:47804) 2022-06-17T12:52:11.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.117+0000 7f8c92d9d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8c940b27b0 0x7f8c9414b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:49332/0 (socket says 172.21.15.36:49332) 2022-06-17T12:52:11.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.117+0000 7f8c9259c700 1 -- 172.21.15.36:0/1499882962 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8c940b27b0 msgr2=0x7f8c9414b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:11.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.117+0000 7f8c9259c700 1 --2- 172.21.15.36:0/1499882962 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8c940b27b0 0x7f8c9414b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:11.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.117+0000 7f8c9259c700 1 -- 172.21.15.36:0/1499882962 >> v1:172.21.15.70:6789/0 conn(0x7f8c94147bd0 legacy=0x7f8c941442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:11.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.117+0000 7f8c9259c700 1 -- 172.21.15.36:0/1499882962 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c94147490 con 0x7f8c94060ca0 2022-06-17T12:52:11.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.117+0000 7f8c9259c700 1 --2- 172.21.15.36:0/1499882962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c94060ca0 0x7f8c940ac780 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f8c84002f70 tx=0x7f8c84006eb0).ready entity=mon.1 client_cookie=ac0df383dfa7add5 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:11.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.117+0000 7f8c91d9b700 1 -- 172.21.15.36:0/1499882962 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c8400d570 con 0x7f8c94060ca0 2022-06-17T12:52:11.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.117+0000 7f8c91d9b700 1 -- 172.21.15.36:0/1499882962 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c84000e90 con 0x7f8c94060ca0 2022-06-17T12:52:11.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.117+0000 7f8c91d9b700 1 -- 172.21.15.36:0/1499882962 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c8400d900 con 0x7f8c94060ca0 2022-06-17T12:52:11.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c98966700 1 -- 172.21.15.36:0/1499882962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c94060ca0 msgr2=0x7f8c940ac780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:11.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c98966700 1 --2- 172.21.15.36:0/1499882962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c94060ca0 0x7f8c940ac780 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f8c84002f70 tx=0x7f8c84006eb0).stop 2022-06-17T12:52:11.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c98966700 1 -- 172.21.15.36:0/1499882962 shutdown_connections 2022-06-17T12:52:11.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c98966700 1 --2- 172.21.15.36:0/1499882962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c94060ca0 0x7f8c940ac780 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:11.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c98966700 1 --2- 172.21.15.36:0/1499882962 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8c940b27b0 0x7f8c9414b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:11.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c98966700 1 -- 172.21.15.36:0/1499882962 >> 172.21.15.36:0/1499882962 conn(0x7f8c940b5d10 msgr2=0x7f8c940b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:11.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c98966700 1 -- 172.21.15.36:0/1499882962 shutdown_connections 2022-06-17T12:52:11.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c98966700 1 -- 172.21.15.36:0/1499882962 wait complete. 2022-06-17T12:52:11.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c98966700 1 Processor -- start 2022-06-17T12:52:11.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c98966700 1 -- start start 2022-06-17T12:52:11.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c98966700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c94060ca0 0x7f8c941455f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:11.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c98966700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8c940b27b0 0x7f8c94066b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:11.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c98966700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8c94147bd0 0x7f8c94062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:11.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c98966700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c940641c0 con 0x7f8c940b27b0 2022-06-17T12:52:11.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c98966700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c940666b0 con 0x7f8c94060ca0 2022-06-17T12:52:11.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c98966700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c940643f0 con 0x7f8c94147bd0 2022-06-17T12:52:11.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c9259c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8c940b27b0 0x7f8c94066b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:11.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c9259c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8c940b27b0 0x7f8c94066b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34544/0 (socket says 172.21.15.36:34544) 2022-06-17T12:52:11.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c9259c700 1 -- 172.21.15.36:0/2053859549 learned_addr learned my addr 172.21.15.36:0/2053859549 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:11.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c9359e700 1 --2- 172.21.15.36:0/2053859549 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8c94147bd0 0x7f8c94062c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:11.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c92d9d700 1 --2- 172.21.15.36:0/2053859549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c94060ca0 0x7f8c941455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:11.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c9259c700 1 -- 172.21.15.36:0/2053859549 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8c94147bd0 msgr2=0x7f8c94062c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:11.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c9259c700 1 --2- 172.21.15.36:0/2053859549 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8c94147bd0 0x7f8c94062c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:11.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c9259c700 1 -- 172.21.15.36:0/2053859549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c94060ca0 msgr2=0x7f8c941455f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:11.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c9259c700 1 --2- 172.21.15.36:0/2053859549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c94060ca0 0x7f8c941455f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:11.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c9259c700 1 -- 172.21.15.36:0/2053859549 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8400b1e0 con 0x7f8c940b27b0 2022-06-17T12:52:11.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.121+0000 7f8c9259c700 1 --2- 172.21.15.36:0/2053859549 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8c940b27b0 0x7f8c94066b10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f8c84002f70 tx=0x7f8c84015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:11.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.125+0000 7f8c83fff700 1 -- 172.21.15.36:0/2053859549 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c840068b0 con 0x7f8c940b27b0 2022-06-17T12:52:11.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.125+0000 7f8c83fff700 1 -- 172.21.15.36:0/2053859549 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8400d8f0 con 0x7f8c940b27b0 2022-06-17T12:52:11.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.125+0000 7f8c98966700 1 -- 172.21.15.36:0/2053859549 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c94145e90 con 0x7f8c940b27b0 2022-06-17T12:52:11.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.125+0000 7f8c83fff700 1 -- 172.21.15.36:0/2053859549 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c84011e20 con 0x7f8c940b27b0 2022-06-17T12:52:11.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.125+0000 7f8c98966700 1 -- 172.21.15.36:0/2053859549 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c941441d0 con 0x7f8c940b27b0 2022-06-17T12:52:11.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.125+0000 7f8c83fff700 1 -- 172.21.15.36:0/2053859549 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8c84025030 con 0x7f8c940b27b0 2022-06-17T12:52:11.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.125+0000 7f8c98966700 1 -- 172.21.15.36:0/2053859549 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c94057410 con 0x7f8c940b27b0 2022-06-17T12:52:11.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.125+0000 7f8c83fff700 1 --2- 172.21.15.36:0/2053859549 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8c70053ad0 0x7f8c70055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:11.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.125+0000 7f8c83fff700 1 -- 172.21.15.36:0/2053859549 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f8c8406d240 con 0x7f8c940b27b0 2022-06-17T12:52:11.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.125+0000 7f8c92d9d700 1 --2- 172.21.15.36:0/2053859549 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8c70053ad0 0x7f8c70055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:11.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.125+0000 7f8c92d9d700 1 --2- 172.21.15.36:0/2053859549 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8c70053ad0 0x7f8c70055f90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f8c8800b000 tx=0x7f8c88006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:11.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.133+0000 7f8c83fff700 1 -- 172.21.15.36:0/2053859549 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8c84043ff0 con 0x7f8c940b27b0 2022-06-17T12:52:11.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.293+0000 7f8c98966700 1 -- 172.21.15.36:0/2053859549 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8c9405d9b0 con 0x7f8c940b27b0 2022-06-17T12:52:11.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.673+0000 7f8c83fff700 1 -- 172.21.15.36:0/2053859549 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8c84047610 con 0x7f8c940b27b0 2022-06-17T12:52:11.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.677+0000 7f8c98966700 1 -- 172.21.15.36:0/2053859549 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8c70053ad0 msgr2=0x7f8c70055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:11.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.677+0000 7f8c98966700 1 --2- 172.21.15.36:0/2053859549 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8c70053ad0 0x7f8c70055f90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f8c8800b000 tx=0x7f8c88006df0).stop 2022-06-17T12:52:11.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.677+0000 7f8c98966700 1 -- 172.21.15.36:0/2053859549 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8c940b27b0 msgr2=0x7f8c94066b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:11.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.677+0000 7f8c98966700 1 --2- 172.21.15.36:0/2053859549 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8c940b27b0 0x7f8c94066b10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f8c84002f70 tx=0x7f8c84015000).stop 2022-06-17T12:52:11.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.677+0000 7f8c98966700 1 -- 172.21.15.36:0/2053859549 shutdown_connections 2022-06-17T12:52:11.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.677+0000 7f8c98966700 1 --2- 172.21.15.36:0/2053859549 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8c70053ad0 0x7f8c70055f90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:11.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.677+0000 7f8c98966700 1 --2- 172.21.15.36:0/2053859549 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8c94147bd0 0x7f8c94062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:11.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.677+0000 7f8c98966700 1 --2- 172.21.15.36:0/2053859549 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8c940b27b0 0x7f8c94066b10 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:11.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.677+0000 7f8c98966700 1 --2- 172.21.15.36:0/2053859549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c94060ca0 0x7f8c941455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:11.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.677+0000 7f8c98966700 1 -- 172.21.15.36:0/2053859549 >> 172.21.15.36:0/2053859549 conn(0x7f8c940b5d10 msgr2=0x7f8c940b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:11.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.677+0000 7f8c98966700 1 -- 172.21.15.36:0/2053859549 shutdown_connections 2022-06-17T12:52:11.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.677+0000 7f8c98966700 1 -- 172.21.15.36:0/2053859549 wait complete. 2022-06-17T12:52:11.683 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-17T12:52:11.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.821+0000 7ff8b8b46700 1 Processor -- start 2022-06-17T12:52:11.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.821+0000 7ff8b8b46700 1 -- start start 2022-06-17T12:52:11.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.821+0000 7ff8b8b46700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff8b40b11a0 0x7ff8b4146540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:11.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.821+0000 7ff8b8b46700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8b4063b10 con 0x7ff8b4063730 2022-06-17T12:52:11.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.821+0000 7ff8b8b46700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8b4146c50 con 0x7ff8b41405f0 2022-06-17T12:52:11.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.821+0000 7ff8b8b46700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8b4146a30 con 0x7ff8b40b11a0 2022-06-17T12:52:11.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.821+0000 7ff8b37fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7ff8b4063730 0x7ff8b413fee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52082/0 (socket says 172.21.15.36:52082) 2022-06-17T12:52:11.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.821+0000 7ff8b37fe700 1 -- 172.21.15.36:0/2954253904 learned_addr learned my addr 172.21.15.36:0/2954253904 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:11.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.821+0000 7ff8b3fff700 1 --2- 172.21.15.36:0/2954253904 >> v2:172.21.15.36:3300/0 conn(0x7ff8b40b11a0 0x7ff8b4146540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:11.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.821+0000 7ff8b27fc700 1 -- 172.21.15.36:0/2954253904 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4237611140 0 0) 0x7ff8b4063b10 con 0x7ff8b4063730 2022-06-17T12:52:11.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.821+0000 7ff8b27fc700 1 -- 172.21.15.36:0/2954253904 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff894003260 con 0x7ff8b4063730 2022-06-17T12:52:11.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.821+0000 7ff8b27fc700 1 -- 172.21.15.36:0/2954253904 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 264555630 0 0) 0x7ff894003260 con 0x7ff8b4063730 2022-06-17T12:52:11.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.821+0000 7ff8b3fff700 1 -- 172.21.15.36:0/2954253904 >> v1:172.21.15.70:6789/0 conn(0x7ff8b41405f0 legacy=0x7ff8b4143090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:11.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.821+0000 7ff8b3fff700 1 -- 172.21.15.36:0/2954253904 >> v1:172.21.15.36:6789/0 conn(0x7ff8b4063730 legacy=0x7ff8b413fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:11.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.821+0000 7ff8b3fff700 1 -- 172.21.15.36:0/2954253904 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b4146ef0 con 0x7ff8b40b11a0 2022-06-17T12:52:11.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.821+0000 7ff8b3fff700 1 --2- 172.21.15.36:0/2954253904 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8b40b11a0 0x7ff8b4146540 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7ff89c001800 tx=0x7ff8ac006df0).ready entity=mon.0 client_cookie=a5a4165dd8e3069f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:11.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.821+0000 7ff8b27fc700 1 -- 172.21.15.36:0/2954253904 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff89c001660 con 0x7ff8b40b11a0 2022-06-17T12:52:11.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.821+0000 7ff8b27fc700 1 -- 172.21.15.36:0/2954253904 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ac000e90 con 0x7ff8b40b11a0 2022-06-17T12:52:11.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.821+0000 7ff8b27fc700 1 -- 172.21.15.36:0/2954253904 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8ac011400 con 0x7ff8b40b11a0 2022-06-17T12:52:11.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.825+0000 7ff8b8b46700 1 -- 172.21.15.36:0/2954253904 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8b40b11a0 msgr2=0x7ff8b4146540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:11.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.825+0000 7ff8b8b46700 1 --2- 172.21.15.36:0/2954253904 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8b40b11a0 0x7ff8b4146540 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7ff89c001800 tx=0x7ff8ac006df0).stop 2022-06-17T12:52:11.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.825+0000 7ff8b8b46700 1 -- 172.21.15.36:0/2954253904 shutdown_connections 2022-06-17T12:52:11.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.825+0000 7ff8b8b46700 1 --2- 172.21.15.36:0/2954253904 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8b40b11a0 0x7ff8b4146540 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:11.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.825+0000 7ff8b8b46700 1 -- 172.21.15.36:0/2954253904 >> 172.21.15.36:0/2954253904 conn(0x7ff8b4060880 msgr2=0x7ff8b40614b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:11.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.825+0000 7ff8b8b46700 1 -- 172.21.15.36:0/2954253904 shutdown_connections 2022-06-17T12:52:11.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.825+0000 7ff8b8b46700 1 -- 172.21.15.36:0/2954253904 wait complete. 2022-06-17T12:52:11.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.825+0000 7ff8b8b46700 1 Processor -- start 2022-06-17T12:52:11.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.825+0000 7ff8b8b46700 1 -- start start 2022-06-17T12:52:11.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.829+0000 7ff8b8b46700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8b4063730 0x7ff8b4064800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:11.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.829+0000 7ff8b8b46700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8b40b11a0 0x7ff8b4152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:11.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.829+0000 7ff8b8b46700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8b41405f0 0x7ff8b4152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:11.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.829+0000 7ff8b8b46700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b413d070 con 0x7ff8b4063730 2022-06-17T12:52:11.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.829+0000 7ff8b8b46700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b40ae030 con 0x7ff8b41405f0 2022-06-17T12:52:11.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.829+0000 7ff8b8b46700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b40667c0 con 0x7ff8b40b11a0 2022-06-17T12:52:11.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.829+0000 7ff8b37fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8b4063730 0x7ff8b4064800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:11.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.829+0000 7ff8b37fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8b4063730 0x7ff8b4064800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34558/0 (socket says 172.21.15.36:34558) 2022-06-17T12:52:11.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.829+0000 7ff8b37fe700 1 -- 172.21.15.36:0/1478794834 learned_addr learned my addr 172.21.15.36:0/1478794834 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:11.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.829+0000 7ff8b2ffd700 1 --2- 172.21.15.36:0/1478794834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8b40b11a0 0x7ff8b4152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:11.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.829+0000 7ff8b3fff700 1 --2- 172.21.15.36:0/1478794834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8b41405f0 0x7ff8b4152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:11.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.829+0000 7ff8b37fe700 1 -- 172.21.15.36:0/1478794834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8b40b11a0 msgr2=0x7ff8b4152830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:11.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.829+0000 7ff8b37fe700 1 --2- 172.21.15.36:0/1478794834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8b40b11a0 0x7ff8b4152830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:11.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.829+0000 7ff8b37fe700 1 -- 172.21.15.36:0/1478794834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8b41405f0 msgr2=0x7ff8b4152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:11.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.829+0000 7ff8b37fe700 1 --2- 172.21.15.36:0/1478794834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8b41405f0 0x7ff8b4152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:11.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.829+0000 7ff8b37fe700 1 -- 172.21.15.36:0/1478794834 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ac00b180 con 0x7ff8b4063730 2022-06-17T12:52:11.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.829+0000 7ff8b3fff700 1 --2- 172.21.15.36:0/1478794834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8b41405f0 0x7ff8b4152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:52:11.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.829+0000 7ff8b2ffd700 1 --2- 172.21.15.36:0/1478794834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8b40b11a0 0x7ff8b4152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:52:11.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.829+0000 7ff8b37fe700 1 --2- 172.21.15.36:0/1478794834 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8b4063730 0x7ff8b4064800 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7ff89c001840 tx=0x7ff89c001de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:11.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.829+0000 7ff8b0ff9700 1 -- 172.21.15.36:0/1478794834 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff89c00ce80 con 0x7ff8b4063730 2022-06-17T12:52:11.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.829+0000 7ff8b0ff9700 1 -- 172.21.15.36:0/1478794834 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89c007170 con 0x7ff8b4063730 2022-06-17T12:52:11.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.829+0000 7ff8b8b46700 1 -- 172.21.15.36:0/1478794834 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b4066a60 con 0x7ff8b4063730 2022-06-17T12:52:11.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.829+0000 7ff8b0ff9700 1 -- 172.21.15.36:0/1478794834 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff89c004730 con 0x7ff8b4063730 2022-06-17T12:52:11.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.829+0000 7ff8b8b46700 1 -- 172.21.15.36:0/1478794834 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b4152320 con 0x7ff8b4063730 2022-06-17T12:52:11.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.833+0000 7ff8b8b46700 1 -- 172.21.15.36:0/1478794834 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8b4152570 con 0x7ff8b4063730 2022-06-17T12:52:11.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.833+0000 7ff8b0ff9700 1 -- 172.21.15.36:0/1478794834 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff89c01b070 con 0x7ff8b4063730 2022-06-17T12:52:11.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.833+0000 7ff8b0ff9700 1 --2- 172.21.15.36:0/1478794834 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff898053a40 0x7ff898055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:11.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.833+0000 7ff8b0ff9700 1 -- 172.21.15.36:0/1478794834 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6506+0+0 (secure 0 0 0) 0x7ff89c06b810 con 0x7ff8b4063730 2022-06-17T12:52:11.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.833+0000 7ff8b2ffd700 1 --2- 172.21.15.36:0/1478794834 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff898053a40 0x7ff898055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:11.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.833+0000 7ff8b2ffd700 1 --2- 172.21.15.36:0/1478794834 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff898053a40 0x7ff898055f00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7ff8a400b000 tx=0x7ff8a4009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:11.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.841+0000 7ff8b0ff9700 1 -- 172.21.15.36:0/1478794834 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff8b4152570 con 0x7ff8b4063730 2022-06-17T12:52:11.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:11.965+0000 7ff8b8b46700 1 -- 172.21.15.36:0/1478794834 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff8b4152570 con 0x7ff8b4063730 2022-06-17T12:52:12.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.737+0000 7ff8b0ff9700 1 -- 172.21.15.36:0/1478794834 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v58) v1 ==== 185+0+0 (secure 0 0 0) 0x7ff8b4152570 con 0x7ff8b4063730 2022-06-17T12:52:12.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.741+0000 7ff8b8b46700 1 -- 172.21.15.36:0/1478794834 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff898053a40 msgr2=0x7ff898055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:12.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.741+0000 7ff8b8b46700 1 --2- 172.21.15.36:0/1478794834 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff898053a40 0x7ff898055f00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7ff8a400b000 tx=0x7ff8a4009040).stop 2022-06-17T12:52:12.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.741+0000 7ff8b8b46700 1 -- 172.21.15.36:0/1478794834 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8b4063730 msgr2=0x7ff8b4064800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:12.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.741+0000 7ff8b8b46700 1 --2- 172.21.15.36:0/1478794834 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8b4063730 0x7ff8b4064800 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7ff89c001840 tx=0x7ff89c001de0).stop 2022-06-17T12:52:12.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.741+0000 7ff8b8b46700 1 -- 172.21.15.36:0/1478794834 shutdown_connections 2022-06-17T12:52:12.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.741+0000 7ff8b8b46700 1 --2- 172.21.15.36:0/1478794834 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff898053a40 0x7ff898055f00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:12.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.741+0000 7ff8b8b46700 1 --2- 172.21.15.36:0/1478794834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8b41405f0 0x7ff8b4152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:12.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.741+0000 7ff8b8b46700 1 --2- 172.21.15.36:0/1478794834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8b40b11a0 0x7ff8b4152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:12.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.741+0000 7ff8b8b46700 1 --2- 172.21.15.36:0/1478794834 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8b4063730 0x7ff8b4064800 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:12.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.741+0000 7ff8b8b46700 1 -- 172.21.15.36:0/1478794834 >> 172.21.15.36:0/1478794834 conn(0x7ff8b4060880 msgr2=0x7ff8b4142480 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:12.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.741+0000 7ff8b8b46700 1 -- 172.21.15.36:0/1478794834 shutdown_connections 2022-06-17T12:52:12.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.741+0000 7ff8b8b46700 1 -- 172.21.15.36:0/1478794834 wait complete. 2022-06-17T12:52:12.736 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' removed 2022-06-17T12:52:12.743 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-17T12:52:12.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a2bfa700 1 Processor -- start 2022-06-17T12:52:12.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a2bfa700 1 -- start start 2022-06-17T12:52:12.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a2bfa700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f489c0ba110 0x7f489c1463a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:12.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a2bfa700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f489c0b4080 0x7f489c142130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:12.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a2bfa700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f489c0ba4f0 con 0x7f489c0b4e30 2022-06-17T12:52:12.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a2bfa700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f489c143980 con 0x7f489c0ba110 2022-06-17T12:52:12.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a2bfa700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489c143c20 con 0x7f489c0b4080 2022-06-17T12:52:12.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a1bf8700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f489c0ba110 0x7f489c1463a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:12.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a13f7700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f489c0b4e30 0x7f489c0b5740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52096/0 (socket says 172.21.15.36:52096) 2022-06-17T12:52:12.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a13f7700 1 -- 172.21.15.36:0/3339970535 learned_addr learned my addr 172.21.15.36:0/3339970535 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:12.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a23f9700 1 --2- 172.21.15.36:0/3339970535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f489c0b4080 0x7f489c142130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:12.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a1bf8700 1 -- 172.21.15.36:0/3339970535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f489c0b4080 msgr2=0x7f489c142130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:12.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a1bf8700 1 --2- 172.21.15.36:0/3339970535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f489c0b4080 0x7f489c142130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:12.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a1bf8700 1 -- 172.21.15.36:0/3339970535 >> v1:172.21.15.36:6789/0 conn(0x7f489c0b4e30 legacy=0x7f489c0b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:12.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a1bf8700 1 -- 172.21.15.36:0/3339970535 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489c0b5280 con 0x7f489c0ba110 2022-06-17T12:52:12.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a1bf8700 1 --2- 172.21.15.36:0/3339970535 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f489c0ba110 0x7f489c1463a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f4898002f70 tx=0x7f4898006df0).ready entity=mon.0 client_cookie=572ebc019db87cf5 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:12.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a0bf6700 1 -- 172.21.15.36:0/3339970535 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f489800d570 con 0x7f489c0ba110 2022-06-17T12:52:12.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a0bf6700 1 -- 172.21.15.36:0/3339970535 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4898000e90 con 0x7f489c0ba110 2022-06-17T12:52:12.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a0bf6700 1 -- 172.21.15.36:0/3339970535 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f489800d8c0 con 0x7f489c0ba110 2022-06-17T12:52:12.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a2bfa700 1 -- 172.21.15.36:0/3339970535 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f489c0ba110 msgr2=0x7f489c1463a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:12.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a2bfa700 1 --2- 172.21.15.36:0/3339970535 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f489c0ba110 0x7f489c1463a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f4898002f70 tx=0x7f4898006df0).stop 2022-06-17T12:52:12.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a2bfa700 1 -- 172.21.15.36:0/3339970535 shutdown_connections 2022-06-17T12:52:12.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a2bfa700 1 --2- 172.21.15.36:0/3339970535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f489c0b4080 0x7f489c142130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:12.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a2bfa700 1 --2- 172.21.15.36:0/3339970535 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f489c0ba110 0x7f489c1463a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:12.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.861+0000 7f48a2bfa700 1 -- 172.21.15.36:0/3339970535 >> 172.21.15.36:0/3339970535 conn(0x7f489c05c550 msgr2=0x7f489c0b96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:12.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.865+0000 7f48a2bfa700 1 -- 172.21.15.36:0/3339970535 shutdown_connections 2022-06-17T12:52:12.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.865+0000 7f48a2bfa700 1 -- 172.21.15.36:0/3339970535 wait complete. 2022-06-17T12:52:12.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.865+0000 7f48a2bfa700 1 Processor -- start 2022-06-17T12:52:12.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.865+0000 7f48a2bfa700 1 -- start start 2022-06-17T12:52:12.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.865+0000 7f48a2bfa700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f489c0b4080 0x7f489c114db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:12.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.865+0000 7f48a2bfa700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f489c0b4e30 0x7f489c112a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:12.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.865+0000 7f48a2bfa700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f489c0ba110 0x7f489c0b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:12.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.865+0000 7f48a2bfa700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489c114440 con 0x7f489c0ba110 2022-06-17T12:52:12.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.865+0000 7f48a2bfa700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489c114670 con 0x7f489c0b4e30 2022-06-17T12:52:12.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.865+0000 7f48a2bfa700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489c114910 con 0x7f489c0b4080 2022-06-17T12:52:12.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.865+0000 7f48a23f9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f489c0ba110 0x7f489c0b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:12.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.865+0000 7f48a23f9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f489c0ba110 0x7f489c0b0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34576/0 (socket says 172.21.15.36:34576) 2022-06-17T12:52:12.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.865+0000 7f48a23f9700 1 -- 172.21.15.36:0/3670869592 learned_addr learned my addr 172.21.15.36:0/3670869592 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:12.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.865+0000 7f48a1bf8700 1 --2- 172.21.15.36:0/3670869592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f489c0b4080 0x7f489c114db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:12.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.865+0000 7f48a13f7700 1 --2- 172.21.15.36:0/3670869592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f489c0b4e30 0x7f489c112a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:12.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.865+0000 7f48a23f9700 1 -- 172.21.15.36:0/3670869592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f489c0b4080 msgr2=0x7f489c114db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:12.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.865+0000 7f48a23f9700 1 --2- 172.21.15.36:0/3670869592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f489c0b4080 0x7f489c114db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:12.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.865+0000 7f48a23f9700 1 -- 172.21.15.36:0/3670869592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f489c0b4e30 msgr2=0x7f489c112a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:12.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.865+0000 7f48a23f9700 1 --2- 172.21.15.36:0/3670869592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f489c0b4e30 0x7f489c112a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:12.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.865+0000 7f48a23f9700 1 -- 172.21.15.36:0/3670869592 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489800b180 con 0x7f489c0ba110 2022-06-17T12:52:12.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.865+0000 7f48a1bf8700 1 --2- 172.21.15.36:0/3670869592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f489c0b4080 0x7f489c114db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:52:12.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.865+0000 7f48a23f9700 1 --2- 172.21.15.36:0/3670869592 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f489c0ba110 0x7f489c0b0640 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f489400a000 tx=0x7f4894008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:12.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.869+0000 7f488effd700 1 -- 172.21.15.36:0/3670869592 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4894013620 con 0x7f489c0ba110 2022-06-17T12:52:12.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.869+0000 7f488effd700 1 -- 172.21.15.36:0/3670869592 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4894013780 con 0x7f489c0ba110 2022-06-17T12:52:12.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.869+0000 7f48a2bfa700 1 -- 172.21.15.36:0/3670869592 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f489c112f40 con 0x7f489c0ba110 2022-06-17T12:52:12.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.869+0000 7f488effd700 1 -- 172.21.15.36:0/3670869592 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4894013ae0 con 0x7f489c0ba110 2022-06-17T12:52:12.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.869+0000 7f48a2bfa700 1 -- 172.21.15.36:0/3670869592 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f489c0b28d0 con 0x7f489c0ba110 2022-06-17T12:52:12.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.869+0000 7f48a2bfa700 1 -- 172.21.15.36:0/3670869592 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f489c114ad0 con 0x7f489c0ba110 2022-06-17T12:52:12.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.869+0000 7f488effd700 1 -- 172.21.15.36:0/3670869592 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f4894013c40 con 0x7f489c0ba110 2022-06-17T12:52:12.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.869+0000 7f488effd700 1 --2- 172.21.15.36:0/3670869592 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f487c053a40 0x7f487c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:12.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.869+0000 7f488effd700 1 -- 172.21.15.36:0/3670869592 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f489406bf90 con 0x7f489c0ba110 2022-06-17T12:52:12.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.869+0000 7f48a1bf8700 1 --2- 172.21.15.36:0/3670869592 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f487c053a40 0x7f487c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:12.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.869+0000 7f48a1bf8700 1 --2- 172.21.15.36:0/3670869592 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f487c053a40 0x7f487c055f00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f489801c000 tx=0x7f4898015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:12.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:12.877+0000 7f488effd700 1 -- 172.21.15.36:0/3670869592 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4894042df0 con 0x7f489c0ba110 2022-06-17T12:52:12.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.001+0000 7f48a2bfa700 1 -- 172.21.15.36:0/3670869592 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f489c0b2b80 con 0x7f489c0ba110 2022-06-17T12:52:13.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.749+0000 7f488effd700 1 -- 172.21.15.36:0/3670869592 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v59) v1 ==== 173+0+0 (secure 0 0 0) 0x7f4894046410 con 0x7f489c0ba110 2022-06-17T12:52:13.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.753+0000 7f48a2bfa700 1 -- 172.21.15.36:0/3670869592 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f487c053a40 msgr2=0x7f487c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:13.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.753+0000 7f48a2bfa700 1 --2- 172.21.15.36:0/3670869592 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f487c053a40 0x7f487c055f00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f489801c000 tx=0x7f4898015000).stop 2022-06-17T12:52:13.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.753+0000 7f48a2bfa700 1 -- 172.21.15.36:0/3670869592 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f489c0ba110 msgr2=0x7f489c0b0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:13.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.753+0000 7f48a2bfa700 1 --2- 172.21.15.36:0/3670869592 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f489c0ba110 0x7f489c0b0640 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f489400a000 tx=0x7f4894008040).stop 2022-06-17T12:52:13.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.753+0000 7f48a2bfa700 1 -- 172.21.15.36:0/3670869592 shutdown_connections 2022-06-17T12:52:13.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.753+0000 7f48a2bfa700 1 --2- 172.21.15.36:0/3670869592 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f487c053a40 0x7f487c055f00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:13.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.753+0000 7f48a2bfa700 1 --2- 172.21.15.36:0/3670869592 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f489c0ba110 0x7f489c0b0640 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:13.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.753+0000 7f48a2bfa700 1 --2- 172.21.15.36:0/3670869592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f489c0b4e30 0x7f489c112a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:13.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.753+0000 7f48a2bfa700 1 --2- 172.21.15.36:0/3670869592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f489c0b4080 0x7f489c114db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:13.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.753+0000 7f48a2bfa700 1 -- 172.21.15.36:0/3670869592 >> 172.21.15.36:0/3670869592 conn(0x7f489c05c550 msgr2=0x7f489c05fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:13.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.753+0000 7f48a2bfa700 1 -- 172.21.15.36:0/3670869592 shutdown_connections 2022-06-17T12:52:13.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.753+0000 7f48a2bfa700 1 -- 172.21.15.36:0/3670869592 wait complete. 2022-06-17T12:52:13.743 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' removed 2022-06-17T12:52:13.750 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-06-17T12:52:13.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.873+0000 7f0924915700 1 Processor -- start 2022-06-17T12:52:13.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.873+0000 7f0924915700 1 -- start start 2022-06-17T12:52:13.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.873+0000 7f0924915700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f09200b9860 0x7f0920143840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:13.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.873+0000 7f0924915700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f09200b45c0 0x7f09200b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:13.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f0924915700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f09200b4ed0 0x7f0920146490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:13.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f0924915700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09200b4160 con 0x7f09200b45c0 2022-06-17T12:52:13.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f0924915700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09200b9c40 con 0x7f09200b4ed0 2022-06-17T12:52:13.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f0924915700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09200627d0 con 0x7f09200b9860 2022-06-17T12:52:13.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f091659c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f09200b45c0 0x7f09200b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:13.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f091659c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f09200b45c0 0x7f09200b49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34582/0 (socket says 172.21.15.36:34582) 2022-06-17T12:52:13.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f091659c700 1 -- 172.21.15.36:0/4171988706 learned_addr learned my addr 172.21.15.36:0/4171988706 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:13.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f091f59e700 1 --2- 172.21.15.36:0/4171988706 >> v2:172.21.15.70:3300/0 conn(0x7f09200b4ed0 0x7f0920146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:13.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f091ed9d700 1 --2- 172.21.15.36:0/4171988706 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f09200b9860 0x7f0920143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:13.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f091659c700 1 -- 172.21.15.36:0/4171988706 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f09200b9860 msgr2=0x7f0920143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:13.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f091659c700 1 --2- 172.21.15.36:0/4171988706 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f09200b9860 0x7f0920143840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:13.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f091659c700 1 -- 172.21.15.36:0/4171988706 >> v2:172.21.15.70:3300/0 conn(0x7f09200b4ed0 msgr2=0x7f0920146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:13.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f091659c700 1 --2- 172.21.15.36:0/4171988706 >> v2:172.21.15.70:3300/0 conn(0x7f09200b4ed0 0x7f0920146490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:13.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f091659c700 1 -- 172.21.15.36:0/4171988706 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0920062a70 con 0x7f09200b45c0 2022-06-17T12:52:13.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f091659c700 1 --2- 172.21.15.36:0/4171988706 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f09200b45c0 0x7f09200b49e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f0908002f70 tx=0x7f0908006eb0).ready entity=mon.0 client_cookie=5f469c72ac797a92 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:13.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f091e59c700 1 -- 172.21.15.36:0/4171988706 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f090800d570 con 0x7f09200b45c0 2022-06-17T12:52:13.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f091e59c700 1 -- 172.21.15.36:0/4171988706 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0908000e90 con 0x7f09200b45c0 2022-06-17T12:52:13.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f091e59c700 1 -- 172.21.15.36:0/4171988706 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f090800d900 con 0x7f09200b45c0 2022-06-17T12:52:13.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f0924915700 1 -- 172.21.15.36:0/4171988706 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f09200b45c0 msgr2=0x7f09200b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:13.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f0924915700 1 --2- 172.21.15.36:0/4171988706 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f09200b45c0 0x7f09200b49e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f0908002f70 tx=0x7f0908006eb0).stop 2022-06-17T12:52:13.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f0924915700 1 -- 172.21.15.36:0/4171988706 shutdown_connections 2022-06-17T12:52:13.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f0924915700 1 --2- 172.21.15.36:0/4171988706 >> v2:172.21.15.70:3300/0 conn(0x7f09200b4ed0 0x7f0920146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:13.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f0924915700 1 --2- 172.21.15.36:0/4171988706 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f09200b45c0 0x7f09200b49e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:13.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f0924915700 1 --2- 172.21.15.36:0/4171988706 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f09200b9860 0x7f0920143840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:13.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f0924915700 1 -- 172.21.15.36:0/4171988706 >> 172.21.15.36:0/4171988706 conn(0x7f092005c550 msgr2=0x7f0920060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:13.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f0924915700 1 -- 172.21.15.36:0/4171988706 shutdown_connections 2022-06-17T12:52:13.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.877+0000 7f0924915700 1 -- 172.21.15.36:0/4171988706 wait complete. 2022-06-17T12:52:13.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.881+0000 7f0924915700 1 Processor -- start 2022-06-17T12:52:13.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.881+0000 7f0924915700 1 -- start start 2022-06-17T12:52:13.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.881+0000 7f0924915700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f09200b45c0 0x7f0920141f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:13.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.881+0000 7f0924915700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f09200b4ed0 0x7f0920152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:13.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.881+0000 7f0924915700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09200b9860 0x7f0920152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:13.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.881+0000 7f0924915700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092013fc70 con 0x7f09200b45c0 2022-06-17T12:52:13.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.881+0000 7f0924915700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092012d560 con 0x7f09200b9860 2022-06-17T12:52:13.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.881+0000 7f0924915700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092012d790 con 0x7f09200b4ed0 2022-06-17T12:52:13.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.881+0000 7f091ed9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f09200b45c0 0x7f0920141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:13.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.881+0000 7f091ed9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f09200b45c0 0x7f0920141f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34586/0 (socket says 172.21.15.36:34586) 2022-06-17T12:52:13.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.881+0000 7f091ed9d700 1 -- 172.21.15.36:0/122599683 learned_addr learned my addr 172.21.15.36:0/122599683 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:13.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.881+0000 7f091659c700 1 --2- 172.21.15.36:0/122599683 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f09200b4ed0 0x7f0920152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:13.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.881+0000 7f091f59e700 1 --2- 172.21.15.36:0/122599683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09200b9860 0x7f0920152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:13.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.881+0000 7f091ed9d700 1 -- 172.21.15.36:0/122599683 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f09200b4ed0 msgr2=0x7f0920152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:13.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.881+0000 7f091ed9d700 1 --2- 172.21.15.36:0/122599683 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f09200b4ed0 0x7f0920152570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:13.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.881+0000 7f091ed9d700 1 -- 172.21.15.36:0/122599683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09200b9860 msgr2=0x7f0920152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:13.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.881+0000 7f091ed9d700 1 --2- 172.21.15.36:0/122599683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09200b9860 0x7f0920152a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:13.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.881+0000 7f091ed9d700 1 -- 172.21.15.36:0/122599683 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090800b1e0 con 0x7f09200b45c0 2022-06-17T12:52:13.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.881+0000 7f091ed9d700 1 --2- 172.21.15.36:0/122599683 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f09200b45c0 0x7f0920141f50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f091000c000 tx=0x7f0910006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:13.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.881+0000 7f091cd99700 1 -- 172.21.15.36:0/122599683 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0910011750 con 0x7f09200b45c0 2022-06-17T12:52:13.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.881+0000 7f091cd99700 1 -- 172.21.15.36:0/122599683 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09100118b0 con 0x7f09200b45c0 2022-06-17T12:52:13.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.881+0000 7f0924915700 1 -- 172.21.15.36:0/122599683 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09201520d0 con 0x7f09200b45c0 2022-06-17T12:52:13.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.881+0000 7f091cd99700 1 -- 172.21.15.36:0/122599683 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0910011c10 con 0x7f09200b45c0 2022-06-17T12:52:13.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.881+0000 7f0924915700 1 -- 172.21.15.36:0/122599683 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0920152f50 con 0x7f09200b45c0 2022-06-17T12:52:13.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.885+0000 7f0924915700 1 -- 172.21.15.36:0/122599683 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f092013d830 con 0x7f09200b45c0 2022-06-17T12:52:13.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.885+0000 7f091cd99700 1 -- 172.21.15.36:0/122599683 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f0910011d70 con 0x7f09200b45c0 2022-06-17T12:52:13.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.885+0000 7f091cd99700 1 --2- 172.21.15.36:0/122599683 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0904053ad0 0x7f0904055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:13.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.885+0000 7f091cd99700 1 -- 172.21.15.36:0/122599683 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f091006c200 con 0x7f09200b45c0 2022-06-17T12:52:13.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.885+0000 7f091659c700 1 --2- 172.21.15.36:0/122599683 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0904053ad0 0x7f0904055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:13.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.885+0000 7f091659c700 1 --2- 172.21.15.36:0/122599683 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0904053ad0 0x7f0904055f90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f090801c000 tx=0x7f0908015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:13.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:13.893+0000 7f091cd99700 1 -- 172.21.15.36:0/122599683 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f092013d830 con 0x7f09200b45c0 2022-06-17T12:52:14.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.021+0000 7f0924915700 1 -- 172.21.15.36:0/122599683 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f092013d830 con 0x7f09200b45c0 2022-06-17T12:52:14.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.029+0000 7f091cd99700 1 -- 172.21.15.36:0/122599683 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7f092013d830 con 0x7f09200b45c0 2022-06-17T12:52:14.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.029+0000 7f0924915700 1 -- 172.21.15.36:0/122599683 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0904053ad0 msgr2=0x7f0904055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:14.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.029+0000 7f0924915700 1 --2- 172.21.15.36:0/122599683 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0904053ad0 0x7f0904055f90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f090801c000 tx=0x7f0908015000).stop 2022-06-17T12:52:14.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.029+0000 7f0924915700 1 -- 172.21.15.36:0/122599683 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f09200b45c0 msgr2=0x7f0920141f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:14.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.029+0000 7f0924915700 1 --2- 172.21.15.36:0/122599683 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f09200b45c0 0x7f0920141f50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f091000c000 tx=0x7f0910006df0).stop 2022-06-17T12:52:14.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.029+0000 7f0924915700 1 -- 172.21.15.36:0/122599683 shutdown_connections 2022-06-17T12:52:14.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.029+0000 7f0924915700 1 --2- 172.21.15.36:0/122599683 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0904053ad0 0x7f0904055f90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:14.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.029+0000 7f0924915700 1 --2- 172.21.15.36:0/122599683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09200b9860 0x7f0920152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:14.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.029+0000 7f0924915700 1 --2- 172.21.15.36:0/122599683 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f09200b4ed0 0x7f0920152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:14.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.029+0000 7f0924915700 1 --2- 172.21.15.36:0/122599683 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f09200b45c0 0x7f0920141f50 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:14.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.029+0000 7f0924915700 1 -- 172.21.15.36:0/122599683 >> 172.21.15.36:0/122599683 conn(0x7f092005c550 msgr2=0x7f09201465c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:14.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.029+0000 7f0924915700 1 -- 172.21.15.36:0/122599683 shutdown_connections 2022-06-17T12:52:14.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.029+0000 7f0924915700 1 -- 172.21.15.36:0/122599683 wait complete. 2022-06-17T12:52:14.028 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout' 2022-06-17T12:52:14.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.141+0000 7f549f4bd700 1 Processor -- start 2022-06-17T12:52:14.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.141+0000 7f549f4bd700 1 -- start start 2022-06-17T12:52:14.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f549f4bd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5498060340 0x7f549813ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:14.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f549f4bd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f54980ad330 0x7f54981431b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:14.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f549f4bd700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5498060720 con 0x7f549813f100 2022-06-17T12:52:14.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f549f4bd700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5498062d30 con 0x7f54980ad330 2022-06-17T12:52:14.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f549f4bd700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5498062fd0 con 0x7f5498060340 2022-06-17T12:52:14.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f549da5a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f54980ad330 0x7f54981431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:14.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f549ca58700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f549813f100 0x7f5498142aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52126/0 (socket says 172.21.15.36:52126) 2022-06-17T12:52:14.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f549da5a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f54980ad330 0x7f54981431b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34598/0 (socket says 172.21.15.36:34598) 2022-06-17T12:52:14.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f549ca58700 1 -- 172.21.15.36:0/2043380443 learned_addr learned my addr 172.21.15.36:0/2043380443 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:14.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f549d259700 1 --2- 172.21.15.36:0/2043380443 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5498060340 0x7f549813ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:14.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f5487fff700 1 -- 172.21.15.36:0/2043380443 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1477857234 0 0) 0x7f5498060720 con 0x7f549813f100 2022-06-17T12:52:14.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f5487fff700 1 -- 172.21.15.36:0/2043380443 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5480003260 con 0x7f549813f100 2022-06-17T12:52:14.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f549d259700 1 -- 172.21.15.36:0/2043380443 >> v2:172.21.15.36:3300/0 conn(0x7f54980ad330 msgr2=0x7f54981431b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:14.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f549d259700 1 --2- 172.21.15.36:0/2043380443 >> v2:172.21.15.36:3300/0 conn(0x7f54980ad330 0x7f54981431b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:14.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f549d259700 1 -- 172.21.15.36:0/2043380443 >> v1:172.21.15.36:6789/0 conn(0x7f549813f100 legacy=0x7f5498142aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:14.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f549d259700 1 -- 172.21.15.36:0/2043380443 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5498063710 con 0x7f5498060340 2022-06-17T12:52:14.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f549da5a700 1 --2- 172.21.15.36:0/2043380443 >> v2:172.21.15.36:3300/0 conn(0x7f54980ad330 0x7f54981431b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:52:14.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f549d259700 1 --2- 172.21.15.36:0/2043380443 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5498060340 0x7f549813ec10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f5494002f70 tx=0x7f5494006df0).ready entity=mon.2 client_cookie=4fa874123a1a28c server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:14.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f5487fff700 1 -- 172.21.15.36:0/2043380443 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f549400d4a0 con 0x7f5498060340 2022-06-17T12:52:14.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f5487fff700 1 -- 172.21.15.36:0/2043380443 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5494000e90 con 0x7f5498060340 2022-06-17T12:52:14.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f5487fff700 1 -- 172.21.15.36:0/2043380443 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f549400d830 con 0x7f5498060340 2022-06-17T12:52:14.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f549f4bd700 1 -- 172.21.15.36:0/2043380443 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5498060340 msgr2=0x7f549813ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:14.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f549f4bd700 1 --2- 172.21.15.36:0/2043380443 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5498060340 0x7f549813ec10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f5494002f70 tx=0x7f5494006df0).stop 2022-06-17T12:52:14.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f549f4bd700 1 -- 172.21.15.36:0/2043380443 shutdown_connections 2022-06-17T12:52:14.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f549f4bd700 1 --2- 172.21.15.36:0/2043380443 >> v2:172.21.15.36:3300/0 conn(0x7f54980ad330 0x7f54981431b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:14.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f549f4bd700 1 --2- 172.21.15.36:0/2043380443 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5498060340 0x7f549813ec10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:14.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f549f4bd700 1 -- 172.21.15.36:0/2043380443 >> 172.21.15.36:0/2043380443 conn(0x7f54980b53e0 msgr2=0x7f54980b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:14.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f549f4bd700 1 -- 172.21.15.36:0/2043380443 shutdown_connections 2022-06-17T12:52:14.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.145+0000 7f549f4bd700 1 -- 172.21.15.36:0/2043380443 wait complete. 2022-06-17T12:52:14.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.149+0000 7f549f4bd700 1 Processor -- start 2022-06-17T12:52:14.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.149+0000 7f549f4bd700 1 -- start start 2022-06-17T12:52:14.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.149+0000 7f549f4bd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5498060340 0x7f5498066200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:14.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.149+0000 7f549f4bd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54980ad330 0x7f5498150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:14.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.149+0000 7f549f4bd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f549813f100 0x7f5498153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:14.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.149+0000 7f549f4bd700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549813dc10 con 0x7f5498060340 2022-06-17T12:52:14.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.149+0000 7f549f4bd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54981513a0 con 0x7f54980ad330 2022-06-17T12:52:14.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.149+0000 7f549f4bd700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54981536e0 con 0x7f549813f100 2022-06-17T12:52:14.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.149+0000 7f549d259700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5498060340 0x7f5498066200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:14.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.149+0000 7f549d259700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5498060340 0x7f5498066200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34600/0 (socket says 172.21.15.36:34600) 2022-06-17T12:52:14.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.149+0000 7f549d259700 1 -- 172.21.15.36:0/1430148725 learned_addr learned my addr 172.21.15.36:0/1430148725 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:14.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.149+0000 7f549ca58700 1 --2- 172.21.15.36:0/1430148725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54980ad330 0x7f5498150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:14.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.149+0000 7f549da5a700 1 --2- 172.21.15.36:0/1430148725 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f549813f100 0x7f5498153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:14.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.149+0000 7f549d259700 1 -- 172.21.15.36:0/1430148725 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f549813f100 msgr2=0x7f5498153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:14.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.149+0000 7f549d259700 1 --2- 172.21.15.36:0/1430148725 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f549813f100 0x7f5498153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:14.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.149+0000 7f549d259700 1 -- 172.21.15.36:0/1430148725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54980ad330 msgr2=0x7f5498150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:14.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.149+0000 7f549d259700 1 --2- 172.21.15.36:0/1430148725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54980ad330 0x7f5498150190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:14.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.149+0000 7f549d259700 1 -- 172.21.15.36:0/1430148725 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5490009470 con 0x7f5498060340 2022-06-17T12:52:14.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.149+0000 7f549d259700 1 --2- 172.21.15.36:0/1430148725 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5498060340 0x7f5498066200 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f5494002f70 tx=0x7f5494014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:14.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.149+0000 7f54867fc700 1 -- 172.21.15.36:0/1430148725 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5494013a40 con 0x7f5498060340 2022-06-17T12:52:14.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.149+0000 7f54867fc700 1 -- 172.21.15.36:0/1430148725 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5494013ba0 con 0x7f5498060340 2022-06-17T12:52:14.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.149+0000 7f549f4bd700 1 -- 172.21.15.36:0/1430148725 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549400b180 con 0x7f5498060340 2022-06-17T12:52:14.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.149+0000 7f54867fc700 1 -- 172.21.15.36:0/1430148725 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f549401e3f0 con 0x7f5498060340 2022-06-17T12:52:14.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.149+0000 7f549f4bd700 1 -- 172.21.15.36:0/1430148725 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5498153910 con 0x7f5498060340 2022-06-17T12:52:14.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.153+0000 7f54867fc700 1 -- 172.21.15.36:0/1430148725 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f549401c070 con 0x7f5498060340 2022-06-17T12:52:14.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.153+0000 7f549f4bd700 1 -- 172.21.15.36:0/1430148725 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5498153ad0 con 0x7f5498060340 2022-06-17T12:52:14.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.153+0000 7f54867fc700 1 --2- 172.21.15.36:0/1430148725 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5478053ad0 0x7f5478055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:14.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.153+0000 7f549ca58700 1 --2- 172.21.15.36:0/1430148725 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5478053ad0 0x7f5478055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:14.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.153+0000 7f54867fc700 1 -- 172.21.15.36:0/1430148725 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f549406ca30 con 0x7f5498060340 2022-06-17T12:52:14.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.153+0000 7f549ca58700 1 --2- 172.21.15.36:0/1430148725 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5478053ad0 0x7f5478055f90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f548800d000 tx=0x7f5488009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:14.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.157+0000 7f54867fc700 1 -- 172.21.15.36:0/1430148725 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5494043ab0 con 0x7f5498060340 2022-06-17T12:52:14.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.325+0000 7f549f4bd700 1 -- 172.21.15.36:0/1430148725 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]} v 0) v1 -- 0x7f5498057410 con 0x7f5498060340 2022-06-17T12:52:14.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.752+0000 7f54867fc700 1 -- 172.21.15.36:0/1430148725 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]}]=0 v242) v1 ==== 152+0+0 (secure 0 0 0) 0x7f5494019020 con 0x7f5498060340 2022-06-17T12:52:14.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.756+0000 7f549f4bd700 1 -- 172.21.15.36:0/1430148725 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5478053ad0 msgr2=0x7f5478055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:14.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.756+0000 7f549f4bd700 1 --2- 172.21.15.36:0/1430148725 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5478053ad0 0x7f5478055f90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f548800d000 tx=0x7f5488009040).stop 2022-06-17T12:52:14.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.756+0000 7f549f4bd700 1 -- 172.21.15.36:0/1430148725 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5498060340 msgr2=0x7f5498066200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:14.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.756+0000 7f549f4bd700 1 --2- 172.21.15.36:0/1430148725 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5498060340 0x7f5498066200 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f5494002f70 tx=0x7f5494014000).stop 2022-06-17T12:52:14.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.756+0000 7f549f4bd700 1 -- 172.21.15.36:0/1430148725 shutdown_connections 2022-06-17T12:52:14.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.756+0000 7f549f4bd700 1 --2- 172.21.15.36:0/1430148725 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5478053ad0 0x7f5478055f90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:14.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.756+0000 7f549f4bd700 1 --2- 172.21.15.36:0/1430148725 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f549813f100 0x7f5498153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:14.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.756+0000 7f549f4bd700 1 --2- 172.21.15.36:0/1430148725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54980ad330 0x7f5498150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:14.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.756+0000 7f549f4bd700 1 --2- 172.21.15.36:0/1430148725 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5498060340 0x7f5498066200 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:14.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.756+0000 7f549f4bd700 1 -- 172.21.15.36:0/1430148725 >> 172.21.15.36:0/1430148725 conn(0x7f54980b53e0 msgr2=0x7f54980b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:14.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.756+0000 7f549f4bd700 1 -- 172.21.15.36:0/1430148725 shutdown_connections 2022-06-17T12:52:14.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.756+0000 7f549f4bd700 1 -- 172.21.15.36:0/1430148725 wait complete. 2022-06-17T12:52:14.765 INFO:tasks.cephfs_test_runner:test_drop_cache_command_dead_timeout (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2022-06-17T12:52:14.766 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_timeout (tasks.cephfs.test_misc.TestCacheDrop) 2022-06-17T12:52:14.767 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout' 2022-06-17T12:52:14.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d47a9700 1 Processor -- start 2022-06-17T12:52:14.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d47a9700 1 -- start start 2022-06-17T12:52:14.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d47a9700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01cc05f770 0x7f01cc14b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:14.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d47a9700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f01cc147f50 0x7f01cc141210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:14.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d47a9700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01cc14b930 con 0x7f01cc141700 2022-06-17T12:52:14.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d47a9700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01cc1475b0 con 0x7f01cc147f50 2022-06-17T12:52:14.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d47a9700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc147810 con 0x7f01cc05f770 2022-06-17T12:52:14.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d1d44700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f01cc147f50 0x7f01cc141210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:14.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d2545700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01cc05f770 0x7f01cc14b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:14.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d1d44700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f01cc147f50 0x7f01cc141210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:54990/0 (socket says 172.21.15.36:54990) 2022-06-17T12:52:14.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d1d44700 1 -- 172.21.15.36:0/3552823208 learned_addr learned my addr 172.21.15.36:0/3552823208 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:14.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d2545700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01cc05f770 0x7f01cc14b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:49398/0 (socket says 172.21.15.36:49398) 2022-06-17T12:52:14.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d2d46700 1 --1- 172.21.15.36:0/3552823208 >> v1:172.21.15.70:6789/0 conn(0x7f01cc141700 0x7f01cc05fb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:47874/0 (socket says 172.21.15.36:47874) 2022-06-17T12:52:14.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d1d44700 1 -- 172.21.15.36:0/3552823208 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01cc05f770 msgr2=0x7f01cc14b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:14.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d1d44700 1 --2- 172.21.15.36:0/3552823208 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01cc05f770 0x7f01cc14b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:14.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d1d44700 1 -- 172.21.15.36:0/3552823208 >> v1:172.21.15.70:6789/0 conn(0x7f01cc141700 legacy=0x7f01cc05fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:14.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d1d44700 1 -- 172.21.15.36:0/3552823208 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01cc147ab0 con 0x7f01cc147f50 2022-06-17T12:52:14.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d2545700 1 --2- 172.21.15.36:0/3552823208 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01cc05f770 0x7f01cc14b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:52:14.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d1d44700 1 --2- 172.21.15.36:0/3552823208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01cc147f50 0x7f01cc141210 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f01c4001490 tx=0x7f01bc006eb0).ready entity=mon.1 client_cookie=729ba7e847ec4083 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:14.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d1543700 1 -- 172.21.15.36:0/3552823208 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01bc00e540 con 0x7f01cc147f50 2022-06-17T12:52:14.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d1543700 1 -- 172.21.15.36:0/3552823208 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01bc000e90 con 0x7f01cc147f50 2022-06-17T12:52:14.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d1543700 1 -- 172.21.15.36:0/3552823208 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01bc00e8d0 con 0x7f01cc147f50 2022-06-17T12:52:14.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d47a9700 1 -- 172.21.15.36:0/3552823208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01cc147f50 msgr2=0x7f01cc141210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:14.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d47a9700 1 --2- 172.21.15.36:0/3552823208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01cc147f50 0x7f01cc141210 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f01c4001490 tx=0x7f01bc006eb0).stop 2022-06-17T12:52:14.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d47a9700 1 -- 172.21.15.36:0/3552823208 shutdown_connections 2022-06-17T12:52:14.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d47a9700 1 --2- 172.21.15.36:0/3552823208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01cc147f50 0x7f01cc141210 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:14.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d47a9700 1 --2- 172.21.15.36:0/3552823208 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01cc05f770 0x7f01cc14b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:14.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d47a9700 1 -- 172.21.15.36:0/3552823208 >> 172.21.15.36:0/3552823208 conn(0x7f01cc05c550 msgr2=0x7f01cc05c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:14.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d47a9700 1 -- 172.21.15.36:0/3552823208 shutdown_connections 2022-06-17T12:52:14.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.892+0000 7f01d47a9700 1 -- 172.21.15.36:0/3552823208 wait complete. 2022-06-17T12:52:14.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01d47a9700 1 Processor -- start 2022-06-17T12:52:14.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01d47a9700 1 -- start start 2022-06-17T12:52:14.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01d47a9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f01cc05f770 0x7f01cc1433a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:14.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01d47a9700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01cc141700 0x7f01cc13e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:14.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01d47a9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01cc147f50 0x7f01cc13eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:14.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01d47a9700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc142660 con 0x7f01cc05f770 2022-06-17T12:52:14.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01d47a9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc157280 con 0x7f01cc147f50 2022-06-17T12:52:14.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01d47a9700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc157520 con 0x7f01cc141700 2022-06-17T12:52:14.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01d2545700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f01cc05f770 0x7f01cc1433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:14.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01d1d44700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01cc141700 0x7f01cc13e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:14.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01d1d44700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01cc141700 0x7f01cc13e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:49402/0 (socket says 172.21.15.36:49402) 2022-06-17T12:52:14.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01d1d44700 1 -- 172.21.15.36:0/3111170641 learned_addr learned my addr 172.21.15.36:0/3111170641 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:14.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01d2d46700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01cc147f50 0x7f01cc13eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:14.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01d2545700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f01cc05f770 0x7f01cc1433a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34616/0 (socket says 172.21.15.36:34616) 2022-06-17T12:52:14.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01d1d44700 1 -- 172.21.15.36:0/3111170641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01cc147f50 msgr2=0x7f01cc13eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:14.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01d1d44700 1 --2- 172.21.15.36:0/3111170641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01cc147f50 0x7f01cc13eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:14.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01d1d44700 1 -- 172.21.15.36:0/3111170641 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f01cc05f770 msgr2=0x7f01cc1433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:14.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01d1d44700 1 --2- 172.21.15.36:0/3111170641 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f01cc05f770 0x7f01cc1433a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:14.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01d1d44700 1 -- 172.21.15.36:0/3111170641 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c800b180 con 0x7f01cc141700 2022-06-17T12:52:14.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01d1d44700 1 --2- 172.21.15.36:0/3111170641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01cc141700 0x7f01cc13e640 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f01cc063c40 tx=0x7f01bc015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:14.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01bb7fe700 1 -- 172.21.15.36:0/3111170641 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01bc0116e0 con 0x7f01cc141700 2022-06-17T12:52:14.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01bb7fe700 1 -- 172.21.15.36:0/3111170641 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01bc011840 con 0x7f01cc141700 2022-06-17T12:52:14.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01d47a9700 1 -- 172.21.15.36:0/3111170641 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01bc00b1e0 con 0x7f01cc141700 2022-06-17T12:52:14.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01bb7fe700 1 -- 172.21.15.36:0/3111170641 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01bc011ba0 con 0x7f01cc141700 2022-06-17T12:52:14.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.896+0000 7f01d47a9700 1 -- 172.21.15.36:0/3111170641 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01cc142890 con 0x7f01cc141700 2022-06-17T12:52:14.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.900+0000 7f01d47a9700 1 -- 172.21.15.36:0/3111170641 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01cc157720 con 0x7f01cc141700 2022-06-17T12:52:14.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.900+0000 7f01bb7fe700 1 -- 172.21.15.36:0/3111170641 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f01bc025030 con 0x7f01cc141700 2022-06-17T12:52:14.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.900+0000 7f01bb7fe700 1 --2- 172.21.15.36:0/3111170641 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f01ac0539b0 0x7f01ac055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:14.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.900+0000 7f01bb7fe700 1 -- 172.21.15.36:0/3111170641 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f01bc06cef0 con 0x7f01cc141700 2022-06-17T12:52:14.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.900+0000 7f01d2545700 1 --2- 172.21.15.36:0/3111170641 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f01ac0539b0 0x7f01ac055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:14.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.900+0000 7f01d2545700 1 --2- 172.21.15.36:0/3111170641 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f01ac0539b0 0x7f01ac055e70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f01c800b000 tx=0x7f01c8006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:14.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:14.908+0000 7f01bb7fe700 1 -- 172.21.15.36:0/3111170641 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f01bc043f30 con 0x7f01cc141700 2022-06-17T12:52:15.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.080+0000 7f01d47a9700 1 -- 172.21.15.36:0/3111170641 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]} v 0) v1 -- 0x7f01cc148390 con 0x7f01cc141700 2022-06-17T12:52:15.627 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:52:15.631 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:52:15.636 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:52:15.645 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:52:15.667 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:52:15.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.756+0000 7f01bb7fe700 1 -- 172.21.15.36:0/3111170641 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]}]=0 v243) v1 ==== 150+0+0 (secure 0 0 0) 0x7f01bc023020 con 0x7f01cc141700 2022-06-17T12:52:15.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.756+0000 7f01d47a9700 1 -- 172.21.15.36:0/3111170641 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f01ac0539b0 msgr2=0x7f01ac055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:15.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.756+0000 7f01d47a9700 1 --2- 172.21.15.36:0/3111170641 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f01ac0539b0 0x7f01ac055e70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f01c800b000 tx=0x7f01c8006df0).stop 2022-06-17T12:52:15.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.760+0000 7f01d47a9700 1 -- 172.21.15.36:0/3111170641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01cc141700 msgr2=0x7f01cc13e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:15.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.760+0000 7f01d47a9700 1 --2- 172.21.15.36:0/3111170641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01cc141700 0x7f01cc13e640 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f01cc063c40 tx=0x7f01bc015000).stop 2022-06-17T12:52:15.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.760+0000 7f01d47a9700 1 -- 172.21.15.36:0/3111170641 shutdown_connections 2022-06-17T12:52:15.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.760+0000 7f01d47a9700 1 --2- 172.21.15.36:0/3111170641 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f01ac0539b0 0x7f01ac055e70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:15.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.760+0000 7f01d47a9700 1 --2- 172.21.15.36:0/3111170641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01cc147f50 0x7f01cc13eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:15.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.760+0000 7f01d47a9700 1 --2- 172.21.15.36:0/3111170641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01cc141700 0x7f01cc13e640 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:15.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.760+0000 7f01d47a9700 1 --2- 172.21.15.36:0/3111170641 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f01cc05f770 0x7f01cc1433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:15.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.760+0000 7f01d47a9700 1 -- 172.21.15.36:0/3111170641 >> 172.21.15.36:0/3111170641 conn(0x7f01cc05c550 msgr2=0x7f01cc05fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:15.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.760+0000 7f01d47a9700 1 -- 172.21.15.36:0/3111170641 shutdown_connections 2022-06-17T12:52:15.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.760+0000 7f01d47a9700 1 -- 172.21.15.36:0/3111170641 wait complete. 2022-06-17T12:52:15.767 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-06-17T12:52:15.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.896+0000 7febfa0d6700 1 Processor -- start 2022-06-17T12:52:15.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.896+0000 7febfa0d6700 1 -- start start 2022-06-17T12:52:15.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.896+0000 7febfa0d6700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7febf40b05e0 0x7febf4144950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:15.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.896+0000 7febfa0d6700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf40b1900 con 0x7febf40acee0 2022-06-17T12:52:15.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.896+0000 7febfa0d6700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf40af370 con 0x7febf40acb00 2022-06-17T12:52:15.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.896+0000 7febfa0d6700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf40b0a50 con 0x7febf40b05e0 2022-06-17T12:52:15.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.896+0000 7febf2ffd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7febf40acee0 0x7febf4144240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52152/0 (socket says 172.21.15.36:52152) 2022-06-17T12:52:15.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.896+0000 7febf2ffd700 1 -- 172.21.15.36:0/10208902 learned_addr learned my addr 172.21.15.36:0/10208902 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:15.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.896+0000 7febf3fff700 1 --2- 172.21.15.36:0/10208902 >> v2:172.21.15.36:3300/0 conn(0x7febf40b05e0 0x7febf4144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:15.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.896+0000 7febf27fc700 1 -- 172.21.15.36:0/10208902 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1742865140 0 0) 0x7febf40b1900 con 0x7febf40acee0 2022-06-17T12:52:15.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.896+0000 7febf27fc700 1 -- 172.21.15.36:0/10208902 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febe0003260 con 0x7febf40acee0 2022-06-17T12:52:15.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.896+0000 7febf27fc700 1 -- 172.21.15.36:0/10208902 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1846924026 0 0) 0x7febf40af370 con 0x7febf40acb00 2022-06-17T12:52:15.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.896+0000 7febf3fff700 1 -- 172.21.15.36:0/10208902 >> v1:172.21.15.70:6789/0 conn(0x7febf40acb00 legacy=0x7febf4147480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:15.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.896+0000 7febf3fff700 1 -- 172.21.15.36:0/10208902 >> v1:172.21.15.36:6789/0 conn(0x7febf40acee0 legacy=0x7febf4144240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:15.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.896+0000 7febf3fff700 1 -- 172.21.15.36:0/10208902 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf4144e40 con 0x7febf40b05e0 2022-06-17T12:52:15.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.896+0000 7febf3fff700 1 --2- 172.21.15.36:0/10208902 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febf40b05e0 0x7febf4144950 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7febe8001800 tx=0x7febec0091d0).ready entity=mon.0 client_cookie=e72aecf449ad19c9 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:15.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.896+0000 7febf27fc700 1 -- 172.21.15.36:0/10208902 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febe8001660 con 0x7febf40b05e0 2022-06-17T12:52:15.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.900+0000 7febf27fc700 1 -- 172.21.15.36:0/10208902 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febec00b580 con 0x7febf40b05e0 2022-06-17T12:52:15.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.900+0000 7febf27fc700 1 -- 172.21.15.36:0/10208902 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febec00b6f0 con 0x7febf40b05e0 2022-06-17T12:52:15.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.900+0000 7febfa0d6700 1 -- 172.21.15.36:0/10208902 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febf40b05e0 msgr2=0x7febf4144950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:15.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.900+0000 7febfa0d6700 1 --2- 172.21.15.36:0/10208902 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febf40b05e0 0x7febf4144950 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7febe8001800 tx=0x7febec0091d0).stop 2022-06-17T12:52:15.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.900+0000 7febfa0d6700 1 -- 172.21.15.36:0/10208902 shutdown_connections 2022-06-17T12:52:15.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.900+0000 7febfa0d6700 1 --2- 172.21.15.36:0/10208902 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febf40b05e0 0x7febf4144950 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:15.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.900+0000 7febfa0d6700 1 -- 172.21.15.36:0/10208902 >> 172.21.15.36:0/10208902 conn(0x7febf405c550 msgr2=0x7febf40b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:15.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.900+0000 7febfa0d6700 1 -- 172.21.15.36:0/10208902 shutdown_connections 2022-06-17T12:52:15.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.900+0000 7febfa0d6700 1 -- 172.21.15.36:0/10208902 wait complete. 2022-06-17T12:52:15.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.900+0000 7febfa0d6700 1 Processor -- start 2022-06-17T12:52:15.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.900+0000 7febfa0d6700 1 -- start start 2022-06-17T12:52:15.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.900+0000 7febfa0d6700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febf40acb00 0x7febf40a70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:15.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.900+0000 7febfa0d6700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febf40acee0 0x7febf40a7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:15.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.900+0000 7febfa0d6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febf40b05e0 0x7febf40a8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:15.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.900+0000 7febfa0d6700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf40a9e60 con 0x7febf40acee0 2022-06-17T12:52:15.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.900+0000 7febfa0d6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf40aa100 con 0x7febf40b05e0 2022-06-17T12:52:15.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.900+0000 7febfa0d6700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf40aa3a0 con 0x7febf40acb00 2022-06-17T12:52:15.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.900+0000 7febf2ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febf40acee0 0x7febf40a7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:15.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.900+0000 7febf2ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febf40acee0 0x7febf40a7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34628/0 (socket says 172.21.15.36:34628) 2022-06-17T12:52:15.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.900+0000 7febf2ffd700 1 -- 172.21.15.36:0/2376175652 learned_addr learned my addr 172.21.15.36:0/2376175652 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:15.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.900+0000 7febf37fe700 1 --2- 172.21.15.36:0/2376175652 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febf40acb00 0x7febf40a70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:15.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.900+0000 7febf3fff700 1 --2- 172.21.15.36:0/2376175652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febf40b05e0 0x7febf40a8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:15.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.904+0000 7febf2ffd700 1 -- 172.21.15.36:0/2376175652 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febf40acb00 msgr2=0x7febf40a70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:15.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.904+0000 7febf2ffd700 1 --2- 172.21.15.36:0/2376175652 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febf40acb00 0x7febf40a70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:15.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.904+0000 7febf2ffd700 1 -- 172.21.15.36:0/2376175652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febf40b05e0 msgr2=0x7febf40a8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:15.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.904+0000 7febf2ffd700 1 --2- 172.21.15.36:0/2376175652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febf40b05e0 0x7febf40a8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:15.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.904+0000 7febf2ffd700 1 -- 172.21.15.36:0/2376175652 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febec009470 con 0x7febf40acee0 2022-06-17T12:52:15.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.904+0000 7febf2ffd700 1 --2- 172.21.15.36:0/2376175652 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febf40acee0 0x7febf40a7f20 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7febe80011d0 tx=0x7febe8001ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:15.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.904+0000 7febf0ff9700 1 -- 172.21.15.36:0/2376175652 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febe8007170 con 0x7febf40acee0 2022-06-17T12:52:15.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.904+0000 7febf0ff9700 1 -- 172.21.15.36:0/2376175652 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe80072d0 con 0x7febf40acee0 2022-06-17T12:52:15.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.904+0000 7febfa0d6700 1 -- 172.21.15.36:0/2376175652 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf40aa640 con 0x7febf40acee0 2022-06-17T12:52:15.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.904+0000 7febf0ff9700 1 -- 172.21.15.36:0/2376175652 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febe8004920 con 0x7febf40acee0 2022-06-17T12:52:15.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.904+0000 7febfa0d6700 1 -- 172.21.15.36:0/2376175652 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf40aaae0 con 0x7febf40acee0 2022-06-17T12:52:15.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.904+0000 7febfa0d6700 1 -- 172.21.15.36:0/2376175652 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf4057410 con 0x7febf40acee0 2022-06-17T12:52:15.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.904+0000 7febf0ff9700 1 -- 172.21.15.36:0/2376175652 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7febe801b070 con 0x7febf40acee0 2022-06-17T12:52:15.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.908+0000 7febf0ff9700 1 --2- 172.21.15.36:0/2376175652 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7febd4053ad0 0x7febd4055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:15.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.908+0000 7febf37fe700 1 --2- 172.21.15.36:0/2376175652 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7febd4053ad0 0x7febd4055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:15.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.908+0000 7febf0ff9700 1 -- 172.21.15.36:0/2376175652 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5686+0+0 (secure 0 0 0) 0x7febe806b580 con 0x7febf40acee0 2022-06-17T12:52:15.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.908+0000 7febf37fe700 1 --2- 172.21.15.36:0/2376175652 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7febd4053ad0 0x7febd4055f90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7febe400d000 tx=0x7febe4007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:15.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:15.912+0000 7febf0ff9700 1 -- 172.21.15.36:0/2376175652 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7febe80424f0 con 0x7febf40acee0 2022-06-17T12:52:16.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.044+0000 7febfa0d6700 1 -- 172.21.15.36:0/2376175652 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7febf405dad0 con 0x7febf40acee0 2022-06-17T12:52:16.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.052+0000 7febf0ff9700 1 -- 172.21.15.36:0/2376175652 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v8)=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7febe8019020 con 0x7febf40acee0 2022-06-17T12:52:16.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.052+0000 7febfa0d6700 1 -- 172.21.15.36:0/2376175652 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7febd4053ad0 msgr2=0x7febd4055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:16.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.052+0000 7febfa0d6700 1 --2- 172.21.15.36:0/2376175652 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7febd4053ad0 0x7febd4055f90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7febe400d000 tx=0x7febe4007a00).stop 2022-06-17T12:52:16.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.052+0000 7febfa0d6700 1 -- 172.21.15.36:0/2376175652 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febf40acee0 msgr2=0x7febf40a7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:16.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.052+0000 7febfa0d6700 1 --2- 172.21.15.36:0/2376175652 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febf40acee0 0x7febf40a7f20 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7febe80011d0 tx=0x7febe8001ae0).stop 2022-06-17T12:52:16.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.056+0000 7febfa0d6700 1 -- 172.21.15.36:0/2376175652 shutdown_connections 2022-06-17T12:52:16.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.056+0000 7febfa0d6700 1 --2- 172.21.15.36:0/2376175652 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7febd4053ad0 0x7febd4055f90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:16.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.056+0000 7febfa0d6700 1 --2- 172.21.15.36:0/2376175652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febf40b05e0 0x7febf40a8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:16.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.056+0000 7febfa0d6700 1 --2- 172.21.15.36:0/2376175652 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febf40acee0 0x7febf40a7f20 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:16.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.056+0000 7febfa0d6700 1 --2- 172.21.15.36:0/2376175652 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febf40acb00 0x7febf40a70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:16.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.056+0000 7febfa0d6700 1 -- 172.21.15.36:0/2376175652 >> 172.21.15.36:0/2376175652 conn(0x7febf405c550 msgr2=0x7febf4148950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:16.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.056+0000 7febfa0d6700 1 -- 172.21.15.36:0/2376175652 shutdown_connections 2022-06-17T12:52:16.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.056+0000 7febfa0d6700 1 -- 172.21.15.36:0/2376175652 wait complete. 2022-06-17T12:52:16.058 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-17T12:52:16.059 DEBUG:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:52:16.086 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:52:16.111 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:52:16.146 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:52:16.169 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:52:16.191 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-17T12:52:16.192 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:52:16.193 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:52:16.194 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:52:16.201 INFO:teuthology.orchestra.run.smithi107.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-17T12:52:16.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:52:16.203 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-06-17T12:52:16.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:52:16.204 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:52:16.205 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:52:16.215 INFO:teuthology.orchestra.run.smithi136.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-17T12:52:16.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:52:16.217 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-17T12:52:16.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:52:16.218 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:52:16.219 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:52:16.228 INFO:teuthology.orchestra.run.smithi143.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:52:16.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:52:16.230 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-06-17T12:52:16.231 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-17T12:52:16.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.356+0000 7eff0457e700 1 Processor -- start 2022-06-17T12:52:16.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.356+0000 7eff0457e700 1 -- start start 2022-06-17T12:52:16.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.356+0000 7eff0457e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efefc0b64f0 0x7efefc0b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:16.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.356+0000 7eff0457e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efefc146770 0x7efefc0aaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:16.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.356+0000 7eff0457e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efefc0ab7f0 con 0x7efefc0b3210 2022-06-17T12:52:16.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.356+0000 7eff0457e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efefc05f810 con 0x7efefc0b64f0 2022-06-17T12:52:16.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.356+0000 7eff0457e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efefc05fab0 con 0x7efefc146770 2022-06-17T12:52:16.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.356+0000 7eff02b1b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efefc146770 0x7efefc0aaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:16.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.356+0000 7eff02b1b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efefc146770 0x7efefc0aaa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:49428/0 (socket says 172.21.15.36:49428) 2022-06-17T12:52:16.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.356+0000 7eff02b1b700 1 -- 172.21.15.36:0/923821199 learned_addr learned my addr 172.21.15.36:0/923821199 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:16.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.356+0000 7eff01b19700 1 --1- 172.21.15.36:0/923821199 >> v1:172.21.15.70:6789/0 conn(0x7efefc0b3210 0x7efefc05ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:47898/0 (socket says 172.21.15.36:47898) 2022-06-17T12:52:16.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.356+0000 7eff0231a700 1 --2- 172.21.15.36:0/923821199 >> v2:172.21.15.70:3300/0 conn(0x7efefc0b64f0 0x7efefc0b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:16.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.360+0000 7eff01318700 1 -- 172.21.15.36:0/923821199 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2067342396 0 0) 0x7efefc0ab7f0 con 0x7efefc0b3210 2022-06-17T12:52:16.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.360+0000 7eff01318700 1 -- 172.21.15.36:0/923821199 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efee4003260 con 0x7efefc0b3210 2022-06-17T12:52:16.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.360+0000 7eff02b1b700 1 -- 172.21.15.36:0/923821199 >> v2:172.21.15.70:3300/0 conn(0x7efefc0b64f0 msgr2=0x7efefc0b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:16.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.360+0000 7eff02b1b700 1 --2- 172.21.15.36:0/923821199 >> v2:172.21.15.70:3300/0 conn(0x7efefc0b64f0 0x7efefc0b68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:16.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.360+0000 7eff02b1b700 1 -- 172.21.15.36:0/923821199 >> v1:172.21.15.70:6789/0 conn(0x7efefc0b3210 legacy=0x7efefc05ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:16.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.360+0000 7eff02b1b700 1 -- 172.21.15.36:0/923821199 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efefc140b70 con 0x7efefc146770 2022-06-17T12:52:16.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.360+0000 7eff02b1b700 1 --2- 172.21.15.36:0/923821199 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efefc146770 0x7efefc0aaa50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7efef40032f0 tx=0x7efef4009290).ready entity=mon.2 client_cookie=dc0ba0d992579 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:16.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.360+0000 7eff01318700 1 -- 172.21.15.36:0/923821199 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efef400b6c0 con 0x7efefc146770 2022-06-17T12:52:16.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.360+0000 7eff01318700 1 -- 172.21.15.36:0/923821199 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef400b820 con 0x7efefc146770 2022-06-17T12:52:16.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.360+0000 7eff01318700 1 -- 172.21.15.36:0/923821199 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efef400b990 con 0x7efefc146770 2022-06-17T12:52:16.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.360+0000 7eff0457e700 1 -- 172.21.15.36:0/923821199 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efefc146770 msgr2=0x7efefc0aaa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:16.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.360+0000 7eff0457e700 1 --2- 172.21.15.36:0/923821199 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efefc146770 0x7efefc0aaa50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7efef40032f0 tx=0x7efef4009290).stop 2022-06-17T12:52:16.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.360+0000 7eff0457e700 1 -- 172.21.15.36:0/923821199 shutdown_connections 2022-06-17T12:52:16.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.360+0000 7eff0457e700 1 --2- 172.21.15.36:0/923821199 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efefc146770 0x7efefc0aaa50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:16.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.360+0000 7eff0457e700 1 --2- 172.21.15.36:0/923821199 >> v2:172.21.15.70:3300/0 conn(0x7efefc0b64f0 0x7efefc0b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:16.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.360+0000 7eff0457e700 1 -- 172.21.15.36:0/923821199 >> 172.21.15.36:0/923821199 conn(0x7efefc0636d0 msgr2=0x7efefc063ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:16.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.360+0000 7eff0457e700 1 -- 172.21.15.36:0/923821199 shutdown_connections 2022-06-17T12:52:16.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.360+0000 7eff0457e700 1 -- 172.21.15.36:0/923821199 wait complete. 2022-06-17T12:52:16.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7eff0457e700 1 Processor -- start 2022-06-17T12:52:16.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7eff0457e700 1 -- start start 2022-06-17T12:52:16.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7eff0457e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efefc0b3210 0x7efefc0b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:16.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7eff0457e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efefc0b64f0 0x7efefc0ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:16.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7eff0457e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efefc146770 0x7efefc064de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:16.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7eff0457e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc0aeaa0 con 0x7efefc0b64f0 2022-06-17T12:52:16.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7eff0457e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc064710 con 0x7efefc146770 2022-06-17T12:52:16.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7eff0457e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efefc0addc0 con 0x7efefc0b3210 2022-06-17T12:52:16.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7eff01b19700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efefc0b64f0 0x7efefc0ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:16.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7eff01b19700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efefc0b64f0 0x7efefc0ad8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34642/0 (socket says 172.21.15.36:34642) 2022-06-17T12:52:16.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7eff01b19700 1 -- 172.21.15.36:0/2501697526 learned_addr learned my addr 172.21.15.36:0/2501697526 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:16.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7eff0231a700 1 --2- 172.21.15.36:0/2501697526 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efefc0b3210 0x7efefc0b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:16.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7eff02b1b700 1 --2- 172.21.15.36:0/2501697526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efefc146770 0x7efefc064de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:16.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7eff02b1b700 1 -- 172.21.15.36:0/2501697526 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efefc0b3210 msgr2=0x7efefc0b5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:16.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7eff02b1b700 1 --2- 172.21.15.36:0/2501697526 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efefc0b3210 0x7efefc0b5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:16.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7eff02b1b700 1 -- 172.21.15.36:0/2501697526 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efefc0b64f0 msgr2=0x7efefc0ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:16.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7eff02b1b700 1 --2- 172.21.15.36:0/2501697526 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efefc0b64f0 0x7efefc0ad8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:16.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7eff02b1b700 1 -- 172.21.15.36:0/2501697526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef4009530 con 0x7efefc146770 2022-06-17T12:52:16.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7eff01b19700 1 --2- 172.21.15.36:0/2501697526 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efefc0b64f0 0x7efefc0ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:52:16.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7eff02b1b700 1 --2- 172.21.15.36:0/2501697526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efefc146770 0x7efefc064de0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7efefc065a00 tx=0x7efef4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:16.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7efeeb7fe700 1 -- 172.21.15.36:0/2501697526 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efef4011dc0 con 0x7efefc146770 2022-06-17T12:52:16.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7efeeb7fe700 1 -- 172.21.15.36:0/2501697526 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef4021430 con 0x7efefc146770 2022-06-17T12:52:16.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7efeeb7fe700 1 -- 172.21.15.36:0/2501697526 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efef4021790 con 0x7efefc146770 2022-06-17T12:52:16.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7eff0457e700 1 -- 172.21.15.36:0/2501697526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efefc155b70 con 0x7efefc146770 2022-06-17T12:52:16.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.364+0000 7eff0457e700 1 -- 172.21.15.36:0/2501697526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efefc155e10 con 0x7efefc146770 2022-06-17T12:52:16.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.368+0000 7eff0457e700 1 -- 172.21.15.36:0/2501697526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efefc0b3b80 con 0x7efefc146770 2022-06-17T12:52:16.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.368+0000 7efeeb7fe700 1 -- 172.21.15.36:0/2501697526 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7efef4028070 con 0x7efefc146770 2022-06-17T12:52:16.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.368+0000 7efeeb7fe700 1 --2- 172.21.15.36:0/2501697526 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efedc053a40 0x7efedc055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:16.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.368+0000 7efeeb7fe700 1 -- 172.21.15.36:0/2501697526 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5686+0+0 (secure 0 0 0) 0x7efef406cbd0 con 0x7efefc146770 2022-06-17T12:52:16.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.368+0000 7eff0231a700 1 --2- 172.21.15.36:0/2501697526 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efedc053a40 0x7efedc055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:16.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.368+0000 7eff0231a700 1 --2- 172.21.15.36:0/2501697526 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efedc053a40 0x7efedc055f00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7efef800b000 tx=0x7efef8006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:16.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.376+0000 7efeeb7fe700 1 -- 172.21.15.36:0/2501697526 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efef4043c10 con 0x7efefc146770 2022-06-17T12:52:16.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.508+0000 7eff0457e700 1 -- 172.21.15.36:0/2501697526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7efefc057410 con 0x7efefc146770 2022-06-17T12:52:16.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.512+0000 7efeeb7fe700 1 -- 172.21.15.36:0/2501697526 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v59) v1 ==== 81+0+29220 (secure 0 0 0) 0x7efef4015030 con 0x7efefc146770 2022-06-17T12:52:16.497 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:52:16.498 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:52:16.498 INFO:teuthology.orchestra.run.smithi036.stdout: "epoch": 59, 2022-06-17T12:52:16.499 INFO:teuthology.orchestra.run.smithi036.stdout: "fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", 2022-06-17T12:52:16.499 INFO:teuthology.orchestra.run.smithi036.stdout: "created": "2022-06-17T12:46:30.671408+0000", 2022-06-17T12:52:16.500 INFO:teuthology.orchestra.run.smithi036.stdout: "modified": "2022-06-17T12:52:13.737480+0000", 2022-06-17T12:52:16.503 INFO:teuthology.orchestra.run.smithi036.stdout: "last_up_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:52:16.504 INFO:teuthology.orchestra.run.smithi036.stdout: "last_in_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:52:16.504 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-17T12:52:16.504 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_num": 5799936, 2022-06-17T12:52:16.505 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_set": [ 2022-06-17T12:52:16.505 INFO:teuthology.orchestra.run.smithi036.stdout: "pglog_hardlimit", 2022-06-17T12:52:16.506 INFO:teuthology.orchestra.run.smithi036.stdout: "purged_snapdirs", 2022-06-17T12:52:16.507 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_deletes", 2022-06-17T12:52:16.515 INFO:teuthology.orchestra.run.smithi036.stdout: "sortbitwise" 2022-06-17T12:52:16.516 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:52:16.518 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_version": 6, 2022-06-17T12:52:16.518 INFO:teuthology.orchestra.run.smithi036.stdout: "full_ratio": 0.94999998807907104, 2022-06-17T12:52:16.523 INFO:teuthology.orchestra.run.smithi036.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-17T12:52:16.524 INFO:teuthology.orchestra.run.smithi036.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-17T12:52:16.524 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_snapshot": "", 2022-06-17T12:52:16.524 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_max": 10, 2022-06-17T12:52:16.525 INFO:teuthology.orchestra.run.smithi036.stdout: "max_osd": 8, 2022-06-17T12:52:16.525 INFO:teuthology.orchestra.run.smithi036.stdout: "require_min_compat_client": "jewel", 2022-06-17T12:52:16.526 INFO:teuthology.orchestra.run.smithi036.stdout: "min_compat_client": "jewel", 2022-06-17T12:52:16.526 INFO:teuthology.orchestra.run.smithi036.stdout: "require_osd_release": "octopus", 2022-06-17T12:52:16.526 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:52:16.527 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.527 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 1, 2022-06-17T12:52:16.527 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "device_health_metrics", 2022-06-17T12:52:16.528 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:33.881212+0000", 2022-06-17T12:52:16.528 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:52:16.528 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:52:16.528 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:52:16.529 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:52:16.529 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:52:16.529 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:52:16.529 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:52:16.530 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:52:16.530 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 1, 2022-06-17T12:52:16.530 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 1, 2022-06-17T12:52:16.530 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 1, 2022-06-17T12:52:16.530 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 1, 2022-06-17T12:52:16.531 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 1, 2022-06-17T12:52:16.531 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:52:16.531 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:52:16.531 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:52:16.532 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:52:16.532 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:52:16.532 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:52:16.533 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:52:16.533 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.533 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "16", 2022-06-17T12:52:16.534 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:52:16.534 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:52:16.534 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:52:16.535 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:52:16.535 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:52:16.536 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:52:16.536 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:52:16.536 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:52:16.537 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:52:16.537 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:52:16.537 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:52:16.538 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:52:16.538 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:52:16.538 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:52:16.539 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:52:16.539 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:52:16.540 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:52:16.540 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:52:16.540 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:52:16.541 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:52:16.541 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:52:16.541 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:52:16.542 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:52:16.542 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:52:16.553 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:52:16.554 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:52:16.554 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.554 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:52:16.555 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:52:16.555 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:52:16.555 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:52:16.556 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:52:16.556 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:52:16.557 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:52:16.557 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:52:16.557 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:52:16.563 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:52:16.564 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:52:16.564 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:52:16.565 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 1 2022-06-17T12:52:16.565 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.566 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:52:16.566 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr_devicehealth": {} 2022-06-17T12:52:16.567 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.567 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.568 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.568 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 2, 2022-06-17T12:52:16.569 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "rbd", 2022-06-17T12:52:16.569 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:44.485460+0000", 2022-06-17T12:52:16.569 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:52:16.570 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:52:16.570 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:52:16.571 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:52:16.571 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:52:16.572 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:52:16.572 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:52:16.573 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:52:16.573 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:52:16.574 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:52:16.574 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:52:16.575 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:52:16.575 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:52:16.575 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:52:16.576 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:52:16.576 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:52:16.577 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:52:16.577 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:52:16.577 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:52:16.578 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:52:16.578 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.579 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "19", 2022-06-17T12:52:16.579 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:52:16.580 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:52:16.580 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:52:16.581 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:52:16.582 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:52:16.582 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:52:16.583 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:52:16.583 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:52:16.584 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:52:16.584 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:52:16.584 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:52:16.585 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:52:16.585 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:52:16.586 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:52:16.587 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:52:16.587 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:52:16.588 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:52:16.588 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:52:16.588 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:52:16.589 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:52:16.590 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:52:16.591 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:52:16.591 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:52:16.592 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:52:16.592 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:52:16.593 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:52:16.593 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.594 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:52:16.594 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:52:16.595 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:52:16.595 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:52:16.596 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:52:16.596 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:52:16.597 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:52:16.597 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:52:16.598 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:52:16.598 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:52:16.599 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:52:16.599 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:52:16.599 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:52:16.600 INFO:teuthology.orchestra.run.smithi036.stdout: "rbd": {} 2022-06-17T12:52:16.600 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.600 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.601 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:52:16.601 INFO:teuthology.orchestra.run.smithi036.stdout: "osds": [ 2022-06-17T12:52:16.603 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.604 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:52:16.604 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", 2022-06-17T12:52:16.605 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:16.605 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:16.606 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:16.606 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:16.606 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:16.607 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:16.607 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:52:16.607 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 53, 2022-06-17T12:52:16.608 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:16.608 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:16.608 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:16.608 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.609 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.609 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.609 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6802", 2022-06-17T12:52:16.609 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:16.610 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.610 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.611 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.611 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6803", 2022-06-17T12:52:16.612 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:16.612 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.613 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.613 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.614 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:16.614 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.615 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.615 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.617 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6804", 2022-06-17T12:52:16.617 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:16.617 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.617 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.618 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.618 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6805", 2022-06-17T12:52:16.618 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:16.618 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.618 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.619 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.619 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:16.619 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.619 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.620 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.620 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6808", 2022-06-17T12:52:16.620 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:16.620 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.621 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.622 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.622 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6809", 2022-06-17T12:52:16.623 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:16.628 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.629 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.629 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.629 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:16.629 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.630 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.630 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.630 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6806", 2022-06-17T12:52:16.631 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:16.631 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.631 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.631 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.632 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6807", 2022-06-17T12:52:16.632 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:16.632 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.632 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.633 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.633 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6803/22619", 2022-06-17T12:52:16.633 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6805/22619", 2022-06-17T12:52:16.633 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6809/22619", 2022-06-17T12:52:16.633 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6807/22619", 2022-06-17T12:52:16.634 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:16.634 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:16.634 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:16.635 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.635 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.635 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.635 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:52:16.636 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", 2022-06-17T12:52:16.636 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:16.636 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:16.636 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:16.636 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:16.639 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:16.639 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:16.640 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:52:16.640 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 54, 2022-06-17T12:52:16.640 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:16.640 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:16.641 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:16.641 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.641 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.641 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.642 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6810", 2022-06-17T12:52:16.642 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:16.642 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.642 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.642 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.643 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6811", 2022-06-17T12:52:16.643 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:16.643 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.643 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.644 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.644 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:16.644 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.644 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.644 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.645 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6812", 2022-06-17T12:52:16.645 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:16.645 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.645 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.646 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.647 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6813", 2022-06-17T12:52:16.647 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:16.648 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.648 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.648 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.649 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:16.649 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.650 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.650 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.651 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6816", 2022-06-17T12:52:16.653 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:16.653 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.653 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.654 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.654 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6817", 2022-06-17T12:52:16.654 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:16.654 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.655 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.655 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.655 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:16.655 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.656 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.656 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.656 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6814", 2022-06-17T12:52:16.656 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:16.656 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.657 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.657 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.657 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6815", 2022-06-17T12:52:16.657 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:16.658 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.658 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.658 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.658 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6811/22620", 2022-06-17T12:52:16.659 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6813/22620", 2022-06-17T12:52:16.659 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6817/22620", 2022-06-17T12:52:16.659 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6815/22620", 2022-06-17T12:52:16.659 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:16.660 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:16.660 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:16.660 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.660 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.660 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.661 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:52:16.661 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", 2022-06-17T12:52:16.661 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:16.661 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:16.662 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:16.663 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:16.663 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:16.663 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:16.663 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:52:16.664 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 53, 2022-06-17T12:52:16.664 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:16.664 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:16.664 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:16.665 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.665 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.665 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.665 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6826", 2022-06-17T12:52:16.666 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:16.666 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.666 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.666 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.667 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6827", 2022-06-17T12:52:16.667 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:16.667 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.667 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.668 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.668 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:16.668 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.668 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.669 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.669 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6828", 2022-06-17T12:52:16.669 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:16.669 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.669 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.670 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.670 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6829", 2022-06-17T12:52:16.670 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:16.670 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.671 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.671 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.672 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:16.672 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.672 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.672 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.673 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6832", 2022-06-17T12:52:16.673 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:16.673 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.673 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.673 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.674 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6833", 2022-06-17T12:52:16.674 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:16.675 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.675 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.675 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.675 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:16.675 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.676 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.676 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.676 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6830", 2022-06-17T12:52:16.676 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:16.677 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.677 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.677 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.677 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6831", 2022-06-17T12:52:16.678 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:16.678 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.679 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.679 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.679 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6827/22643", 2022-06-17T12:52:16.679 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6829/22643", 2022-06-17T12:52:16.680 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6833/22643", 2022-06-17T12:52:16.681 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6831/22643", 2022-06-17T12:52:16.681 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:16.681 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:16.681 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:16.681 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.682 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.682 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.682 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:52:16.683 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", 2022-06-17T12:52:16.683 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:16.683 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:16.683 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:16.685 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:16.685 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:16.686 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:16.686 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:52:16.686 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 51, 2022-06-17T12:52:16.686 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:16.687 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:16.687 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:16.687 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.687 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.688 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.688 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6818", 2022-06-17T12:52:16.688 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:16.689 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.689 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.689 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.689 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6819", 2022-06-17T12:52:16.689 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:16.690 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.690 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.690 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.690 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:16.691 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.691 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.691 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.691 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6820", 2022-06-17T12:52:16.692 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:16.692 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.692 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.692 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.692 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6821", 2022-06-17T12:52:16.693 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:16.693 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.693 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.693 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.694 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:16.694 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.694 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.694 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.695 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6824", 2022-06-17T12:52:16.695 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:16.695 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.695 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.695 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.696 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6825", 2022-06-17T12:52:16.696 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:16.696 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.696 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.697 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.697 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:16.697 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.698 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.698 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.698 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6822", 2022-06-17T12:52:16.698 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:16.699 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.699 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.699 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.699 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6823", 2022-06-17T12:52:16.699 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:16.700 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.700 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.700 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.700 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6819/22646", 2022-06-17T12:52:16.701 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6821/22646", 2022-06-17T12:52:16.701 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6825/22646", 2022-06-17T12:52:16.701 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6823/22646", 2022-06-17T12:52:16.701 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:16.701 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:16.702 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:16.702 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.702 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.702 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.703 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:52:16.703 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", 2022-06-17T12:52:16.703 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:16.703 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:16.704 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:16.704 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:16.704 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:16.704 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:16.704 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:52:16.705 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 53, 2022-06-17T12:52:16.705 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:16.705 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:16.705 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:16.706 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.706 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.706 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.706 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6808", 2022-06-17T12:52:16.707 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:16.707 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.707 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.707 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.707 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6809", 2022-06-17T12:52:16.708 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:16.708 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.708 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.708 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.709 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:16.709 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.709 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.709 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.709 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6810", 2022-06-17T12:52:16.710 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:16.710 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.710 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.710 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.711 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6811", 2022-06-17T12:52:16.711 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:16.711 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.711 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.712 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.712 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:16.712 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.712 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.713 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.713 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6814", 2022-06-17T12:52:16.713 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:16.713 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.714 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.714 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.714 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6815", 2022-06-17T12:52:16.714 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:16.715 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.715 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.715 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.715 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:16.715 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.716 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.716 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.716 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6812", 2022-06-17T12:52:16.717 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:16.717 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.717 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.717 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.718 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6813", 2022-06-17T12:52:16.718 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:16.718 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.719 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.719 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.719 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6809/22812", 2022-06-17T12:52:16.719 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6811/22812", 2022-06-17T12:52:16.720 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6815/22812", 2022-06-17T12:52:16.720 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6813/22812", 2022-06-17T12:52:16.720 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:16.720 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:16.721 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:16.721 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.721 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.722 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.722 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:52:16.722 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", 2022-06-17T12:52:16.722 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:16.723 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:16.723 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:16.723 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:16.724 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:16.724 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:16.724 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:52:16.724 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 53, 2022-06-17T12:52:16.725 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:16.725 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:16.725 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:16.725 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.726 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.726 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.726 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6800", 2022-06-17T12:52:16.726 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:16.727 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.727 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.727 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.728 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6801", 2022-06-17T12:52:16.728 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:16.728 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.728 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.729 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.729 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:16.729 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.730 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.730 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.730 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6802", 2022-06-17T12:52:16.730 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:16.731 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.731 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.731 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.732 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6803", 2022-06-17T12:52:16.732 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:16.733 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.733 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.733 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.733 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:16.734 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.734 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.735 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.735 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6806", 2022-06-17T12:52:16.735 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:16.736 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.737 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.737 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.741 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6807", 2022-06-17T12:52:16.741 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:16.742 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.743 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.743 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.744 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:16.744 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.745 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.745 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.746 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6804", 2022-06-17T12:52:16.746 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:16.747 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.747 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.748 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.748 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6805", 2022-06-17T12:52:16.748 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:16.749 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.749 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.750 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.751 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6801/22813", 2022-06-17T12:52:16.752 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6803/22813", 2022-06-17T12:52:16.752 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6807/22813", 2022-06-17T12:52:16.753 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6805/22813", 2022-06-17T12:52:16.754 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:16.755 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:16.755 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:16.755 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.756 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.756 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.757 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:52:16.757 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", 2022-06-17T12:52:16.758 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:16.758 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:16.759 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:16.759 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:16.760 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:16.760 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:16.760 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:52:16.761 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 54, 2022-06-17T12:52:16.761 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:16.761 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:16.762 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:16.762 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.762 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.763 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.763 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6816", 2022-06-17T12:52:16.763 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:16.764 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.764 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.764 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.765 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6817", 2022-06-17T12:52:16.765 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:16.765 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.765 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.766 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.766 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:16.766 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.766 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.767 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.767 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6818", 2022-06-17T12:52:16.767 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:16.767 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.767 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.768 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.768 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6819", 2022-06-17T12:52:16.768 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:16.768 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.769 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.769 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.769 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:16.769 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.770 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.770 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.770 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6822", 2022-06-17T12:52:16.770 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:16.771 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.771 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.771 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.772 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6823", 2022-06-17T12:52:16.772 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:16.773 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.773 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.776 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.776 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:16.777 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.777 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.777 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.777 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6820", 2022-06-17T12:52:16.778 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:16.778 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.778 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.778 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.779 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6821", 2022-06-17T12:52:16.779 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:16.779 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.779 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.780 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.780 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6817/22822", 2022-06-17T12:52:16.780 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6819/22822", 2022-06-17T12:52:16.780 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6823/22822", 2022-06-17T12:52:16.781 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6821/22822", 2022-06-17T12:52:16.781 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:16.781 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:16.782 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:16.782 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.782 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.782 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.783 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:52:16.783 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", 2022-06-17T12:52:16.783 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:16.783 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:16.783 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:16.784 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:16.786 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:16.793 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:16.793 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:52:16.793 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 51, 2022-06-17T12:52:16.794 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:16.794 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:16.794 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:16.795 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.795 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.795 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.796 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6824", 2022-06-17T12:52:16.796 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:16.796 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.797 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.797 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.797 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6825", 2022-06-17T12:52:16.798 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:16.798 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.798 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.798 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.799 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:16.799 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.799 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.800 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.800 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6826", 2022-06-17T12:52:16.800 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:16.801 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.801 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.801 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.802 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6827", 2022-06-17T12:52:16.802 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:16.802 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.802 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.803 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.803 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:16.803 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.804 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.804 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.804 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6830", 2022-06-17T12:52:16.804 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:16.805 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.805 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.805 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.806 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6831", 2022-06-17T12:52:16.806 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:16.806 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.807 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.808 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.808 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:16.808 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:16.808 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.809 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:16.809 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6828", 2022-06-17T12:52:16.810 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:16.810 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.811 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.811 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:16.811 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6829", 2022-06-17T12:52:16.811 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:16.811 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.812 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.812 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.812 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6825/22837", 2022-06-17T12:52:16.812 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6827/22837", 2022-06-17T12:52:16.813 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6831/22837", 2022-06-17T12:52:16.813 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6829/22837", 2022-06-17T12:52:16.813 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:16.813 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:16.814 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:16.814 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:16.814 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.815 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:52:16.815 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_xinfo": [ 2022-06-17T12:52:16.815 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.815 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:52:16.815 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:16.816 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:16.816 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:16.816 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:16.817 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:16.817 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.291461+0000", 2022-06-17T12:52:16.817 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:16.817 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.818 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.818 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:52:16.819 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:16.819 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:16.819 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:16.819 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:16.819 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:16.820 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.537912+0000", 2022-06-17T12:52:16.820 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:16.821 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.821 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.821 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:52:16.822 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:16.822 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:16.822 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:16.822 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:16.823 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:16.823 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.606947+0000", 2022-06-17T12:52:16.823 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:16.823 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.823 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.824 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:52:16.824 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:16.824 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:16.824 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:16.825 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:16.825 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:16.825 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.514412+0000", 2022-06-17T12:52:16.825 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:16.826 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.826 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.826 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:52:16.827 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:16.827 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:16.828 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:16.828 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:16.829 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:16.829 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.394645+0000", 2022-06-17T12:52:16.830 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:16.831 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.831 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.832 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:52:16.832 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:16.833 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:16.833 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:16.834 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:16.834 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:16.834 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.343806+0000", 2022-06-17T12:52:16.835 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:16.835 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.836 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.837 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:52:16.837 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:16.838 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:16.838 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:16.839 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:16.839 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:16.840 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.591864+0000", 2022-06-17T12:52:16.840 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:16.841 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.851 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:16.852 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:52:16.852 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:16.853 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:16.853 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:16.858 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:16.858 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:16.859 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.607045+0000", 2022-06-17T12:52:16.859 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:16.860 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.860 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:52:16.860 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap": [], 2022-06-17T12:52:16.861 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap_items": [], 2022-06-17T12:52:16.861 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_temp": [], 2022-06-17T12:52:16.862 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_temp": [], 2022-06-17T12:52:16.862 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklist": { 2022-06-17T12:52:16.863 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.70:6833/2723288138": "2022-06-18T12:52:10.474790+0000", 2022-06-17T12:52:16.863 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.70:6832/2723288138": "2022-06-18T12:52:10.474790+0000" 2022-06-17T12:52:16.864 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.864 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profiles": { 2022-06-17T12:52:16.865 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs_data": { 2022-06-17T12:52:16.865 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-device-class": "", 2022-06-17T12:52:16.866 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:52:16.866 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-root": "default", 2022-06-17T12:52:16.867 INFO:teuthology.orchestra.run.smithi036.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-17T12:52:16.867 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:52:16.867 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "2", 2022-06-17T12:52:16.868 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:52:16.868 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:52:16.869 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van", 2022-06-17T12:52:16.869 INFO:teuthology.orchestra.run.smithi036.stdout: "w": "8" 2022-06-17T12:52:16.871 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.871 INFO:teuthology.orchestra.run.smithi036.stdout: "default": { 2022-06-17T12:52:16.872 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:52:16.872 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:52:16.874 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "1", 2022-06-17T12:52:16.875 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:52:16.875 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:52:16.876 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van" 2022-06-17T12:52:16.876 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:16.877 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:16.877 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps_queue": [], 2022-06-17T12:52:16.877 INFO:teuthology.orchestra.run.smithi036.stdout: "new_removed_snaps": [], 2022-06-17T12:52:16.878 INFO:teuthology.orchestra.run.smithi036.stdout: "new_purged_snaps": [], 2022-06-17T12:52:16.878 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_node_flags": {}, 2022-06-17T12:52:16.879 INFO:teuthology.orchestra.run.smithi036.stdout: "device_class_flags": {} 2022-06-17T12:52:16.879 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:52:16.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.512+0000 7eff0457e700 1 -- 172.21.15.36:0/2501697526 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efedc053a40 msgr2=0x7efedc055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:16.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.512+0000 7eff0457e700 1 --2- 172.21.15.36:0/2501697526 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efedc053a40 0x7efedc055f00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7efef800b000 tx=0x7efef8006df0).stop 2022-06-17T12:52:16.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.512+0000 7eff0457e700 1 -- 172.21.15.36:0/2501697526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efefc146770 msgr2=0x7efefc064de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:16.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.512+0000 7eff0457e700 1 --2- 172.21.15.36:0/2501697526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efefc146770 0x7efefc064de0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7efefc065a00 tx=0x7efef4016000).stop 2022-06-17T12:52:16.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.512+0000 7eff0457e700 1 -- 172.21.15.36:0/2501697526 shutdown_connections 2022-06-17T12:52:16.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.512+0000 7eff0457e700 1 --2- 172.21.15.36:0/2501697526 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efedc053a40 0x7efedc055f00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:16.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.512+0000 7eff0457e700 1 --2- 172.21.15.36:0/2501697526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efefc146770 0x7efefc064de0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:16.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.512+0000 7eff0457e700 1 --2- 172.21.15.36:0/2501697526 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efefc0b64f0 0x7efefc0ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:16.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.512+0000 7eff0457e700 1 --2- 172.21.15.36:0/2501697526 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efefc0b3210 0x7efefc0b5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:16.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.512+0000 7eff0457e700 1 -- 172.21.15.36:0/2501697526 >> 172.21.15.36:0/2501697526 conn(0x7efefc0636d0 msgr2=0x7efefc0ac290 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:16.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.512+0000 7eff0457e700 1 -- 172.21.15.36:0/2501697526 shutdown_connections 2022-06-17T12:52:16.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:16.512+0000 7eff0457e700 1 -- 172.21.15.36:0/2501697526 wait complete. 2022-06-17T12:52:16.891 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:52:17.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880c3b4700 1 Processor -- start 2022-06-17T12:52:17.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880c3b4700 1 -- start start 2022-06-17T12:52:17.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880c3b4700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8804062770 0x7f8804062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:17.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880c3b4700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f88041444f0 0x7f8804144960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:17.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880c3b4700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88040b0ba0 con 0x7f88040ad330 2022-06-17T12:52:17.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880c3b4700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88040ae820 con 0x7f88041444f0 2022-06-17T12:52:17.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880c3b4700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8804061030 con 0x7f8804062770 2022-06-17T12:52:17.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880994f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f88041444f0 0x7f8804144960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:17.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880994f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f88041444f0 0x7f8804144960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34648/0 (socket says 172.21.15.36:34648) 2022-06-17T12:52:17.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880994f700 1 -- 172.21.15.36:0/2245878506 learned_addr learned my addr 172.21.15.36:0/2245878506 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:17.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880a150700 1 --2- 172.21.15.36:0/2245878506 >> v2:172.21.15.70:3300/0 conn(0x7f8804062770 0x7f8804062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:17.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880994f700 1 -- 172.21.15.36:0/2245878506 >> v2:172.21.15.70:3300/0 conn(0x7f8804062770 msgr2=0x7f8804062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:17.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880994f700 1 --2- 172.21.15.36:0/2245878506 >> v2:172.21.15.70:3300/0 conn(0x7f8804062770 0x7f8804062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:17.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880994f700 1 -- 172.21.15.36:0/2245878506 >> v1:172.21.15.36:6789/0 conn(0x7f88040ad330 legacy=0x7f88040ad7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:17.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880994f700 1 -- 172.21.15.36:0/2245878506 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8804147e90 con 0x7f88041444f0 2022-06-17T12:52:17.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880a150700 1 --2- 172.21.15.36:0/2245878506 >> v2:172.21.15.70:3300/0 conn(0x7f8804062770 0x7f8804062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:52:17.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880994f700 1 --2- 172.21.15.36:0/2245878506 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88041444f0 0x7f8804144960 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f8804061170 tx=0x7f87f4006eb0).ready entity=mon.0 client_cookie=2ed9038e178d6466 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:17.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880914e700 1 -- 172.21.15.36:0/2245878506 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87f400d610 con 0x7f88041444f0 2022-06-17T12:52:17.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880914e700 1 -- 172.21.15.36:0/2245878506 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f4000e90 con 0x7f88041444f0 2022-06-17T12:52:17.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880914e700 1 -- 172.21.15.36:0/2245878506 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87f400d960 con 0x7f88041444f0 2022-06-17T12:52:17.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880c3b4700 1 -- 172.21.15.36:0/2245878506 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88041444f0 msgr2=0x7f8804144960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:17.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880c3b4700 1 --2- 172.21.15.36:0/2245878506 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88041444f0 0x7f8804144960 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f8804061170 tx=0x7f87f4006eb0).stop 2022-06-17T12:52:17.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880c3b4700 1 -- 172.21.15.36:0/2245878506 shutdown_connections 2022-06-17T12:52:17.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880c3b4700 1 --2- 172.21.15.36:0/2245878506 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88041444f0 0x7f8804144960 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:17.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880c3b4700 1 --2- 172.21.15.36:0/2245878506 >> v2:172.21.15.70:3300/0 conn(0x7f8804062770 0x7f8804062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:17.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880c3b4700 1 -- 172.21.15.36:0/2245878506 >> 172.21.15.36:0/2245878506 conn(0x7f88040b53e0 msgr2=0x7f88040b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:17.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880c3b4700 1 -- 172.21.15.36:0/2245878506 shutdown_connections 2022-06-17T12:52:17.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.020+0000 7f880c3b4700 1 -- 172.21.15.36:0/2245878506 wait complete. 2022-06-17T12:52:17.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f880c3b4700 1 Processor -- start 2022-06-17T12:52:17.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f880c3b4700 1 -- start start 2022-06-17T12:52:17.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f880c3b4700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8804062770 0x7f8804156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:17.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f880c3b4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88040ad330 0x7f880414b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:17.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f880c3b4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88041444f0 0x7f8804147020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:17.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f880c3b4700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8804156cf0 con 0x7f88041444f0 2022-06-17T12:52:17.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f880c3b4700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8804063930 con 0x7f88040ad330 2022-06-17T12:52:17.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f880c3b4700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88041494f0 con 0x7f8804062770 2022-06-17T12:52:17.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f880a951700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88041444f0 0x7f8804147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:17.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f880a951700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88041444f0 0x7f8804147020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34658/0 (socket says 172.21.15.36:34658) 2022-06-17T12:52:17.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f880a951700 1 -- 172.21.15.36:0/2446961927 learned_addr learned my addr 172.21.15.36:0/2446961927 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:17.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f880994f700 1 --2- 172.21.15.36:0/2446961927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88040ad330 0x7f880414b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:17.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f880a150700 1 --2- 172.21.15.36:0/2446961927 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8804062770 0x7f8804156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:17.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f880a951700 1 -- 172.21.15.36:0/2446961927 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8804062770 msgr2=0x7f8804156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:17.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f880a951700 1 --2- 172.21.15.36:0/2446961927 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8804062770 0x7f8804156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:17.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f880a951700 1 -- 172.21.15.36:0/2446961927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88040ad330 msgr2=0x7f880414b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:17.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f880a951700 1 --2- 172.21.15.36:0/2446961927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88040ad330 0x7f880414b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:17.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f880a951700 1 -- 172.21.15.36:0/2446961927 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f400b1e0 con 0x7f88041444f0 2022-06-17T12:52:17.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f880994f700 1 --2- 172.21.15.36:0/2446961927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88040ad330 0x7f880414b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:52:17.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f880a951700 1 --2- 172.21.15.36:0/2446961927 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88041444f0 0x7f8804147020 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f880405d270 tx=0x7f87fc007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:17.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f87f37fe700 1 -- 172.21.15.36:0/2446961927 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87fc00e4a0 con 0x7f88041444f0 2022-06-17T12:52:17.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f87f37fe700 1 -- 172.21.15.36:0/2446961927 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87fc00e600 con 0x7f88041444f0 2022-06-17T12:52:17.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f880c3b4700 1 -- 172.21.15.36:0/2446961927 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8804149790 con 0x7f88041444f0 2022-06-17T12:52:17.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f87f37fe700 1 -- 172.21.15.36:0/2446961927 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87fc00e960 con 0x7f88041444f0 2022-06-17T12:52:17.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.024+0000 7f880c3b4700 1 -- 172.21.15.36:0/2446961927 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8804149290 con 0x7f88041444f0 2022-06-17T12:52:17.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.028+0000 7f880c3b4700 1 -- 172.21.15.36:0/2446961927 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8804144e50 con 0x7f88041444f0 2022-06-17T12:52:17.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.028+0000 7f87f37fe700 1 -- 172.21.15.36:0/2446961927 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f87fc019030 con 0x7f88041444f0 2022-06-17T12:52:17.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.028+0000 7f87f37fe700 1 --2- 172.21.15.36:0/2446961927 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f87e40539b0 0x7f87e4055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:17.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.028+0000 7f880a150700 1 --2- 172.21.15.36:0/2446961927 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f87e40539b0 0x7f87e4055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:17.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.028+0000 7f87f37fe700 1 -- 172.21.15.36:0/2446961927 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f87fc06b470 con 0x7f88041444f0 2022-06-17T12:52:17.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.028+0000 7f880a150700 1 --2- 172.21.15.36:0/2446961927 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f87e40539b0 0x7f87e4055e70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f880000b000 tx=0x7f8800008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:17.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.036+0000 7f87f37fe700 1 -- 172.21.15.36:0/2446961927 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f87fc042550 con 0x7f88041444f0 2022-06-17T12:52:17.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.200+0000 7f880c3b4700 1 -- 172.21.15.36:0/2446961927 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8804057410 con 0x7f88041444f0 2022-06-17T12:52:17.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.204+0000 7f87f37fe700 1 -- 172.21.15.36:0/2446961927 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+1209 (secure 0 0 0) 0x7f87fc045b70 con 0x7f88041444f0 2022-06-17T12:52:17.188 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:52:17.188 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5360,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3785730699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3785730699},{"type":"v1","addr":"172.21.15.70:6833","nonce":3785730699}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":5670,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.36:6835/2598001220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2598001220},{"type":"v1","addr":"172.21.15.36:6835","nonce":2598001220}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":23}],"filesystems":[]} 2022-06-17T12:52:17.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.204+0000 7f880c3b4700 1 -- 172.21.15.36:0/2446961927 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f87e40539b0 msgr2=0x7f87e4055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:17.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.204+0000 7f880c3b4700 1 --2- 172.21.15.36:0/2446961927 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f87e40539b0 0x7f87e4055e70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f880000b000 tx=0x7f8800008040).stop 2022-06-17T12:52:17.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.204+0000 7f880c3b4700 1 -- 172.21.15.36:0/2446961927 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88041444f0 msgr2=0x7f8804147020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:17.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.204+0000 7f880c3b4700 1 --2- 172.21.15.36:0/2446961927 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88041444f0 0x7f8804147020 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f880405d270 tx=0x7f87fc007f40).stop 2022-06-17T12:52:17.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.204+0000 7f880c3b4700 1 -- 172.21.15.36:0/2446961927 shutdown_connections 2022-06-17T12:52:17.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.204+0000 7f880c3b4700 1 --2- 172.21.15.36:0/2446961927 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f87e40539b0 0x7f87e4055e70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:17.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.204+0000 7f880c3b4700 1 --2- 172.21.15.36:0/2446961927 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88041444f0 0x7f8804147020 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:17.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.204+0000 7f880c3b4700 1 --2- 172.21.15.36:0/2446961927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88040ad330 0x7f880414b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:17.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.204+0000 7f880c3b4700 1 --2- 172.21.15.36:0/2446961927 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8804062770 0x7f8804156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:17.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.204+0000 7f880c3b4700 1 -- 172.21.15.36:0/2446961927 >> 172.21.15.36:0/2446961927 conn(0x7f88040b53e0 msgr2=0x7f880414dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:17.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.204+0000 7f880c3b4700 1 -- 172.21.15.36:0/2446961927 shutdown_connections 2022-06-17T12:52:17.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.204+0000 7f880c3b4700 1 -- 172.21.15.36:0/2446961927 wait complete. 2022-06-17T12:52:17.199 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 27 2022-06-17T12:52:17.206 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:52:17.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.340+0000 7fe59d841700 1 Processor -- start 2022-06-17T12:52:17.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.340+0000 7fe59d841700 1 -- start start 2022-06-17T12:52:17.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe59d841700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe59813cb70 0x7fe598143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:17.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe59d841700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe59813cf50 0x7fe5981442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:17.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe59d841700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5980af4b0 con 0x7fe5980aef60 2022-06-17T12:52:17.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe59d841700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5980ba500 con 0x7fe59813cb70 2022-06-17T12:52:17.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe59d841700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe598142f50 con 0x7fe59813cf50 2022-06-17T12:52:17.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe59c83f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe59813cb70 0x7fe598143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:17.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe597fff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fe5980aef60 0x7fe5980b9df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52192/0 (socket says 172.21.15.36:52192) 2022-06-17T12:52:17.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe597fff700 1 -- 172.21.15.36:0/3320323645 learned_addr learned my addr 172.21.15.36:0/3320323645 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:17.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe59d040700 1 --2- 172.21.15.36:0/3320323645 >> v2:172.21.15.70:3300/0 conn(0x7fe59813cf50 0x7fe5981442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:17.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe5977fe700 1 -- 172.21.15.36:0/3320323645 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007140623 0 0) 0x7fe5980af4b0 con 0x7fe5980aef60 2022-06-17T12:52:17.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe5977fe700 1 -- 172.21.15.36:0/3320323645 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe57c003260 con 0x7fe5980aef60 2022-06-17T12:52:17.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe59c83f700 1 -- 172.21.15.36:0/3320323645 >> v2:172.21.15.70:3300/0 conn(0x7fe59813cf50 msgr2=0x7fe5981442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:17.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe59c83f700 1 --2- 172.21.15.36:0/3320323645 >> v2:172.21.15.70:3300/0 conn(0x7fe59813cf50 0x7fe5981442a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:17.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe59c83f700 1 -- 172.21.15.36:0/3320323645 >> v1:172.21.15.36:6789/0 conn(0x7fe5980aef60 legacy=0x7fe5980b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:17.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe59c83f700 1 -- 172.21.15.36:0/3320323645 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5981431f0 con 0x7fe59813cb70 2022-06-17T12:52:17.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe59c83f700 1 --2- 172.21.15.36:0/3320323645 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe59813cb70 0x7fe598143690 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fe58c002f70 tx=0x7fe58c006df0).ready entity=mon.0 client_cookie=40aba49a86d9ed18 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:17.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe5977fe700 1 -- 172.21.15.36:0/3320323645 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe58c00d4f0 con 0x7fe59813cb70 2022-06-17T12:52:17.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe5977fe700 1 -- 172.21.15.36:0/3320323645 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58c000e90 con 0x7fe59813cb70 2022-06-17T12:52:17.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe5977fe700 1 -- 172.21.15.36:0/3320323645 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe58c00d840 con 0x7fe59813cb70 2022-06-17T12:52:17.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe59d841700 1 -- 172.21.15.36:0/3320323645 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe59813cb70 msgr2=0x7fe598143690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:17.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe59d841700 1 --2- 172.21.15.36:0/3320323645 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe59813cb70 0x7fe598143690 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fe58c002f70 tx=0x7fe58c006df0).stop 2022-06-17T12:52:17.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe59d841700 1 -- 172.21.15.36:0/3320323645 shutdown_connections 2022-06-17T12:52:17.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe59d841700 1 --2- 172.21.15.36:0/3320323645 >> v2:172.21.15.70:3300/0 conn(0x7fe59813cf50 0x7fe5981442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:17.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe59d841700 1 --2- 172.21.15.36:0/3320323645 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe59813cb70 0x7fe598143690 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:17.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe59d841700 1 -- 172.21.15.36:0/3320323645 >> 172.21.15.36:0/3320323645 conn(0x7fe59805c550 msgr2=0x7fe5980b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:17.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe59d841700 1 -- 172.21.15.36:0/3320323645 shutdown_connections 2022-06-17T12:52:17.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.344+0000 7fe59d841700 1 -- 172.21.15.36:0/3320323645 wait complete. 2022-06-17T12:52:17.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.348+0000 7fe59d841700 1 Processor -- start 2022-06-17T12:52:17.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.348+0000 7fe59d841700 1 -- start start 2022-06-17T12:52:17.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.348+0000 7fe59d841700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5980aef60 0x7fe5981451f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:17.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.348+0000 7fe59d841700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe59813cb70 0x7fe5980b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:17.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.348+0000 7fe59d841700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe59813cf50 0x7fe5981556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:17.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.348+0000 7fe59d841700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59813fc70 con 0x7fe59813cb70 2022-06-17T12:52:17.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.348+0000 7fe59d841700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5980b1790 con 0x7fe5980aef60 2022-06-17T12:52:17.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.348+0000 7fe59d841700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5981413d0 con 0x7fe59813cf50 2022-06-17T12:52:17.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.348+0000 7fe597fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe59813cb70 0x7fe5980b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:17.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.348+0000 7fe597fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe59813cb70 0x7fe5980b1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34670/0 (socket says 172.21.15.36:34670) 2022-06-17T12:52:17.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.348+0000 7fe597fff700 1 -- 172.21.15.36:0/2754738067 learned_addr learned my addr 172.21.15.36:0/2754738067 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:17.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.348+0000 7fe59c83f700 1 --2- 172.21.15.36:0/2754738067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5980aef60 0x7fe5981451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:17.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.348+0000 7fe59d040700 1 --2- 172.21.15.36:0/2754738067 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe59813cf50 0x7fe5981556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:17.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.348+0000 7fe597fff700 1 -- 172.21.15.36:0/2754738067 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe59813cf50 msgr2=0x7fe5981556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:17.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.348+0000 7fe597fff700 1 --2- 172.21.15.36:0/2754738067 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe59813cf50 0x7fe5981556b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:17.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.348+0000 7fe597fff700 1 -- 172.21.15.36:0/2754738067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5980aef60 msgr2=0x7fe5981451f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:17.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.348+0000 7fe597fff700 1 --2- 172.21.15.36:0/2754738067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5980aef60 0x7fe5981451f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:17.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.348+0000 7fe597fff700 1 -- 172.21.15.36:0/2754738067 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58c00b180 con 0x7fe59813cb70 2022-06-17T12:52:17.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.348+0000 7fe597fff700 1 --2- 172.21.15.36:0/2754738067 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe59813cb70 0x7fe5980b1e60 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7fe584001110 tx=0x7fe584007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:17.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.348+0000 7fe595ffb700 1 -- 172.21.15.36:0/2754738067 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5840070d0 con 0x7fe59813cb70 2022-06-17T12:52:17.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.348+0000 7fe595ffb700 1 -- 172.21.15.36:0/2754738067 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe584007230 con 0x7fe59813cb70 2022-06-17T12:52:17.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.348+0000 7fe59d841700 1 -- 172.21.15.36:0/2754738067 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5980b19c0 con 0x7fe59813cb70 2022-06-17T12:52:17.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.348+0000 7fe595ffb700 1 -- 172.21.15.36:0/2754738067 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe584005610 con 0x7fe59813cb70 2022-06-17T12:52:17.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.348+0000 7fe59d841700 1 -- 172.21.15.36:0/2754738067 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe59813d3e0 con 0x7fe59813cb70 2022-06-17T12:52:17.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.352+0000 7fe59d841700 1 -- 172.21.15.36:0/2754738067 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5980b1b50 con 0x7fe59813cb70 2022-06-17T12:52:17.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.352+0000 7fe595ffb700 1 -- 172.21.15.36:0/2754738067 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe58401a060 con 0x7fe59813cb70 2022-06-17T12:52:17.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.352+0000 7fe595ffb700 1 --2- 172.21.15.36:0/2754738067 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe5780539b0 0x7fe578055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:17.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.352+0000 7fe595ffb700 1 -- 172.21.15.36:0/2754738067 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fe58406b410 con 0x7fe59813cb70 2022-06-17T12:52:17.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.352+0000 7fe59c83f700 1 --2- 172.21.15.36:0/2754738067 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe5780539b0 0x7fe578055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:17.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.352+0000 7fe59c83f700 1 --2- 172.21.15.36:0/2754738067 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe5780539b0 0x7fe578055e70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7fe58c01a000 tx=0x7fe58c014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:17.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.360+0000 7fe595ffb700 1 -- 172.21.15.36:0/2754738067 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe5840424f0 con 0x7fe59813cb70 2022-06-17T12:52:17.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.528+0000 7fe59d841700 1 -- 172.21.15.36:0/2754738067 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe598057410 con 0x7fe59813cb70 2022-06-17T12:52:17.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.528+0000 7fe595ffb700 1 -- 172.21.15.36:0/2754738067 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+1209 (secure 0 0 0) 0x7fe584018020 con 0x7fe59813cb70 2022-06-17T12:52:17.514 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:52:17.515 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5360,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3785730699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3785730699},{"type":"v1","addr":"172.21.15.70:6833","nonce":3785730699}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":5670,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.36:6835/2598001220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2598001220},{"type":"v1","addr":"172.21.15.36:6835","nonce":2598001220}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":23}],"filesystems":[]} 2022-06-17T12:52:17.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.532+0000 7fe59d841700 1 -- 172.21.15.36:0/2754738067 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe5780539b0 msgr2=0x7fe578055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:17.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.532+0000 7fe59d841700 1 --2- 172.21.15.36:0/2754738067 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe5780539b0 0x7fe578055e70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7fe58c01a000 tx=0x7fe58c014000).stop 2022-06-17T12:52:17.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.532+0000 7fe59d841700 1 -- 172.21.15.36:0/2754738067 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe59813cb70 msgr2=0x7fe5980b1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:17.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.532+0000 7fe59d841700 1 --2- 172.21.15.36:0/2754738067 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe59813cb70 0x7fe5980b1e60 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7fe584001110 tx=0x7fe584007ad0).stop 2022-06-17T12:52:17.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.532+0000 7fe59d841700 1 -- 172.21.15.36:0/2754738067 shutdown_connections 2022-06-17T12:52:17.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.532+0000 7fe59d841700 1 --2- 172.21.15.36:0/2754738067 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe5780539b0 0x7fe578055e70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:17.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.532+0000 7fe59d841700 1 --2- 172.21.15.36:0/2754738067 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe59813cf50 0x7fe5981556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:17.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.532+0000 7fe59d841700 1 --2- 172.21.15.36:0/2754738067 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe59813cb70 0x7fe5980b1e60 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:17.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.532+0000 7fe59d841700 1 --2- 172.21.15.36:0/2754738067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5980aef60 0x7fe5981451f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:17.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.532+0000 7fe59d841700 1 -- 172.21.15.36:0/2754738067 >> 172.21.15.36:0/2754738067 conn(0x7fe59805c550 msgr2=0x7fe59813b660 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:17.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.532+0000 7fe59d841700 1 -- 172.21.15.36:0/2754738067 shutdown_connections 2022-06-17T12:52:17.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.532+0000 7fe59d841700 1 -- 172.21.15.36:0/2754738067 wait complete. 2022-06-17T12:52:17.529 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 27 2022-06-17T12:52:17.534 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-17T12:52:17.714 INFO:tasks.ceph.mds.a:Stopping old one... 2022-06-17T12:52:17.715 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-17T12:52:17.716 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T12:52:17.716 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-17T12:52:17.717 INFO:tasks.ceph.mds.b:Stopping old one... 2022-06-17T12:52:17.718 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-17T12:52:17.719 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T12:52:17.745 INFO:tasks.ceph.mds.b:Stopped 2022-06-17T12:52:17.751 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-17T12:52:17.752 INFO:tasks.ceph.mds.a:Stopped 2022-06-17T12:52:17.753 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-17T12:52:17.769 INFO:tasks.ceph.mds.b:Started 2022-06-17T12:52:17.769 DEBUG:teuthology.parallel:result is None 2022-06-17T12:52:17.770 INFO:tasks.ceph.mds.a:Started 2022-06-17T12:52:17.771 DEBUG:teuthology.parallel:result is None 2022-06-17T12:52:17.771 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-06-17T12:52:17.845 INFO:tasks.ceph.mds.a.smithi036.stdout:starting mds.a at 2022-06-17T12:52:17.847 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2022-06-17T12:52:17.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e7030700 1 Processor -- start 2022-06-17T12:52:17.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e7030700 1 -- start start 2022-06-17T12:52:17.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e7030700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26e00af540 0x7f26e014b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:17.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e7030700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e00b3140 con 0x7f26e0060ac0 2022-06-17T12:52:17.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e7030700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e0060ea0 con 0x7f26e00afd80 2022-06-17T12:52:17.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e7030700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e01441e0 con 0x7f26e00af540 2022-06-17T12:52:17.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e602e700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f26e0060ac0 0x7f26e0144b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52218/0 (socket says 172.21.15.36:52218) 2022-06-17T12:52:17.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e602e700 1 -- 172.21.15.36:0/3348930511 learned_addr learned my addr 172.21.15.36:0/3348930511 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:17.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e682f700 1 --2- 172.21.15.36:0/3348930511 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26e00af540 0x7f26e014b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:17.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e502c700 1 -- 172.21.15.36:0/3348930511 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395718011 0 0) 0x7f26e00b3140 con 0x7f26e0060ac0 2022-06-17T12:52:17.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e502c700 1 -- 172.21.15.36:0/3348930511 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26c8003260 con 0x7f26e0060ac0 2022-06-17T12:52:17.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e502c700 1 -- 172.21.15.36:0/3348930511 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3004503912 0 0) 0x7f26e0060ea0 con 0x7f26e00afd80 2022-06-17T12:52:17.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e502c700 1 -- 172.21.15.36:0/3348930511 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26e00b3140 con 0x7f26e00afd80 2022-06-17T12:52:17.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e682f700 1 -- 172.21.15.36:0/3348930511 >> v1:172.21.15.70:6789/0 conn(0x7f26e00afd80 legacy=0x7f26e0140bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:17.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e682f700 1 -- 172.21.15.36:0/3348930511 >> v1:172.21.15.36:6789/0 conn(0x7f26e0060ac0 legacy=0x7f26e0144b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:17.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e682f700 1 -- 172.21.15.36:0/3348930511 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e01443d0 con 0x7f26e00af540 2022-06-17T12:52:17.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e682f700 1 --2- 172.21.15.36:0/3348930511 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26e00af540 0x7f26e014b790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f26d4001110 tx=0x7f26d80091d0).ready entity=mon.2 client_cookie=93735bb327477d88 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:17.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e502c700 1 -- 172.21.15.36:0/3348930511 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26d800b600 con 0x7f26e00af540 2022-06-17T12:52:17.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e502c700 1 -- 172.21.15.36:0/3348930511 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d800b760 con 0x7f26e00af540 2022-06-17T12:52:17.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e502c700 1 -- 172.21.15.36:0/3348930511 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26d800b8d0 con 0x7f26e00af540 2022-06-17T12:52:17.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e7030700 1 -- 172.21.15.36:0/3348930511 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26e00af540 msgr2=0x7f26e014b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:17.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e7030700 1 --2- 172.21.15.36:0/3348930511 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26e00af540 0x7f26e014b790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f26d4001110 tx=0x7f26d80091d0).stop 2022-06-17T12:52:17.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e7030700 1 -- 172.21.15.36:0/3348930511 shutdown_connections 2022-06-17T12:52:17.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e7030700 1 --2- 172.21.15.36:0/3348930511 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26e00af540 0x7f26e014b790 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:17.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e7030700 1 -- 172.21.15.36:0/3348930511 >> 172.21.15.36:0/3348930511 conn(0x7f26e00b65f0 msgr2=0x7f26e00b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:17.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e7030700 1 -- 172.21.15.36:0/3348930511 shutdown_connections 2022-06-17T12:52:17.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.900+0000 7f26e7030700 1 -- 172.21.15.36:0/3348930511 wait complete. 2022-06-17T12:52:17.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.904+0000 7f26e7030700 1 Processor -- start 2022-06-17T12:52:17.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.904+0000 7f26e7030700 1 -- start start 2022-06-17T12:52:17.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.904+0000 7f26e7030700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26e0060ac0 0x7f26e013a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:17.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.904+0000 7f26e7030700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26e00af540 0x7f26e013fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:17.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.904+0000 7f26e7030700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26e00afd80 0x7f26e014fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:17.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.904+0000 7f26e7030700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e01399f0 con 0x7f26e00afd80 2022-06-17T12:52:17.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.904+0000 7f26e7030700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e0139c90 con 0x7f26e00af540 2022-06-17T12:52:17.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.904+0000 7f26e7030700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e0139720 con 0x7f26e0060ac0 2022-06-17T12:52:17.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.904+0000 7f26e682f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26e00afd80 0x7f26e014fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:17.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.904+0000 7f26e682f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26e00afd80 0x7f26e014fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34696/0 (socket says 172.21.15.36:34696) 2022-06-17T12:52:17.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.904+0000 7f26e682f700 1 -- 172.21.15.36:0/3186210778 learned_addr learned my addr 172.21.15.36:0/3186210778 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:17.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.904+0000 7f26e582d700 1 --2- 172.21.15.36:0/3186210778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26e00af540 0x7f26e013fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:17.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.904+0000 7f26e602e700 1 --2- 172.21.15.36:0/3186210778 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26e0060ac0 0x7f26e013a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:17.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.904+0000 7f26e682f700 1 -- 172.21.15.36:0/3186210778 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26e0060ac0 msgr2=0x7f26e013a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:17.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.904+0000 7f26e682f700 1 --2- 172.21.15.36:0/3186210778 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26e0060ac0 0x7f26e013a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:17.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.904+0000 7f26e682f700 1 -- 172.21.15.36:0/3186210778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26e00af540 msgr2=0x7f26e013fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:17.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.904+0000 7f26e682f700 1 --2- 172.21.15.36:0/3186210778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26e00af540 0x7f26e013fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:17.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.904+0000 7f26e682f700 1 -- 172.21.15.36:0/3186210778 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d8009470 con 0x7f26e00afd80 2022-06-17T12:52:17.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.904+0000 7f26e682f700 1 --2- 172.21.15.36:0/3186210778 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26e00afd80 0x7f26e014fec0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f26d4001110 tx=0x7f26d8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:17.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.904+0000 7f26d37fe700 1 -- 172.21.15.36:0/3186210778 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26d800b8d0 con 0x7f26e00afd80 2022-06-17T12:52:17.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.904+0000 7f26d37fe700 1 -- 172.21.15.36:0/3186210778 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d80032f0 con 0x7f26e00afd80 2022-06-17T12:52:17.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.904+0000 7f26e7030700 1 -- 172.21.15.36:0/3186210778 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e0140f40 con 0x7f26e00afd80 2022-06-17T12:52:17.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.904+0000 7f26d37fe700 1 -- 172.21.15.36:0/3186210778 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26d800b600 con 0x7f26e00afd80 2022-06-17T12:52:17.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.904+0000 7f26e7030700 1 -- 172.21.15.36:0/3186210778 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e0066890 con 0x7f26e00afd80 2022-06-17T12:52:17.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.908+0000 7f26e7030700 1 -- 172.21.15.36:0/3186210778 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e0066ee0 con 0x7f26e00afd80 2022-06-17T12:52:17.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.908+0000 7f26d37fe700 1 -- 172.21.15.36:0/3186210778 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f26d8026030 con 0x7f26e00afd80 2022-06-17T12:52:17.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.908+0000 7f26d37fe700 1 --2- 172.21.15.36:0/3186210778 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26c00539d0 0x7f26c0055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:17.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.908+0000 7f26e602e700 1 --2- 172.21.15.36:0/3186210778 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26c00539d0 0x7f26c0055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:17.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.908+0000 7f26d37fe700 1 -- 172.21.15.36:0/3186210778 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f26d806c7f0 con 0x7f26e00afd80 2022-06-17T12:52:17.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.908+0000 7f26e602e700 1 --2- 172.21.15.36:0/3186210778 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26c00539d0 0x7f26c0055e90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f26dc00d000 tx=0x7f26dc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:17.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:17.912+0000 7f26d37fe700 1 -- 172.21.15.36:0/3186210778 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f26d8047230 con 0x7f26e00afd80 2022-06-17T12:52:18.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:18.044+0000 7f26e7030700 1 -- 172.21.15.36:0/3186210778 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f26e0057410 con 0x7f26e00afd80 2022-06-17T12:52:18.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:18.904+0000 7f26d37fe700 1 -- 172.21.15.36:0/3186210778 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v61) v1 ==== 97+0+0 (secure 0 0 0) 0x7f26d8015080 con 0x7f26e00afd80 2022-06-17T12:52:18.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:18.908+0000 7f26e7030700 1 -- 172.21.15.36:0/3186210778 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26c00539d0 msgr2=0x7f26c0055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:18.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:18.908+0000 7f26e7030700 1 --2- 172.21.15.36:0/3186210778 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26c00539d0 0x7f26c0055e90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f26dc00d000 tx=0x7f26dc009040).stop 2022-06-17T12:52:18.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:18.908+0000 7f26e7030700 1 -- 172.21.15.36:0/3186210778 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26e00afd80 msgr2=0x7f26e014fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:18.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:18.908+0000 7f26e7030700 1 --2- 172.21.15.36:0/3186210778 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26e00afd80 0x7f26e014fec0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f26d4001110 tx=0x7f26d8016000).stop 2022-06-17T12:52:18.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:18.908+0000 7f26e7030700 1 -- 172.21.15.36:0/3186210778 shutdown_connections 2022-06-17T12:52:18.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:18.908+0000 7f26e7030700 1 --2- 172.21.15.36:0/3186210778 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26c00539d0 0x7f26c0055e90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:18.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:18.908+0000 7f26e7030700 1 --2- 172.21.15.36:0/3186210778 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26e00afd80 0x7f26e014fec0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:18.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:18.908+0000 7f26e7030700 1 --2- 172.21.15.36:0/3186210778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26e00af540 0x7f26e013fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:18.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:18.908+0000 7f26e7030700 1 --2- 172.21.15.36:0/3186210778 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26e0060ac0 0x7f26e013a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:18.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:18.908+0000 7f26e7030700 1 -- 172.21.15.36:0/3186210778 >> 172.21.15.36:0/3186210778 conn(0x7f26e00b65f0 msgr2=0x7f26e01463a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:18.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:18.908+0000 7f26e7030700 1 -- 172.21.15.36:0/3186210778 shutdown_connections 2022-06-17T12:52:18.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:18.908+0000 7f26e7030700 1 -- 172.21.15.36:0/3186210778 wait complete. 2022-06-17T12:52:18.901 INFO:teuthology.orchestra.run.smithi036.stderr: removed all blacklist entries 2022-06-17T12:52:18.908 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2022-06-17T12:52:18.909 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-06-17T12:52:19.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfdb2a4700 1 Processor -- start 2022-06-17T12:52:19.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfdb2a4700 1 -- start start 2022-06-17T12:52:19.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfdb2a4700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fdfd40ac340 0x7fdfd414b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:19.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfdb2a4700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdfd40af7f0 0x7fdfd40ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:19.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfdb2a4700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd40ac870 con 0x7fdfd40ad140 2022-06-17T12:52:19.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfdb2a4700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfd4148960 con 0x7fdfd40ac340 2022-06-17T12:52:19.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfdb2a4700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfd40acaa0 con 0x7fdfd40af7f0 2022-06-17T12:52:19.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfd883f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fdfd40ac340 0x7fdfd414b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:19.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfd883f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fdfd40ac340 0x7fdfd414b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34704/0 (socket says 172.21.15.36:34704) 2022-06-17T12:52:19.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfd883f700 1 -- 172.21.15.36:0/3613328927 learned_addr learned my addr 172.21.15.36:0/3613328927 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:19.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfd9841700 1 --2- 172.21.15.36:0/3613328927 >> v2:172.21.15.70:3300/0 conn(0x7fdfd40af7f0 0x7fdfd40ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:19.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfd883f700 1 -- 172.21.15.36:0/3613328927 >> v2:172.21.15.70:3300/0 conn(0x7fdfd40af7f0 msgr2=0x7fdfd40ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:19.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfd883f700 1 --2- 172.21.15.36:0/3613328927 >> v2:172.21.15.70:3300/0 conn(0x7fdfd40af7f0 0x7fdfd40ad520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfd883f700 1 -- 172.21.15.36:0/3613328927 >> v1:172.21.15.70:6789/0 conn(0x7fdfd40ad140 legacy=0x7fdfd414aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:19.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfd883f700 1 -- 172.21.15.36:0/3613328927 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd40b0bf0 con 0x7fdfd40ac340 2022-06-17T12:52:19.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfd9841700 1 --2- 172.21.15.36:0/3613328927 >> v2:172.21.15.70:3300/0 conn(0x7fdfd40af7f0 0x7fdfd40ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:52:19.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfd883f700 1 --2- 172.21.15.36:0/3613328927 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfd40ac340 0x7fdfd414b3b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7fdfd40b10c0 tx=0x7fdfcc009290).ready entity=mon.0 client_cookie=363fb0194586aed1 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:19.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfd3fff700 1 -- 172.21.15.36:0/3613328927 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdfcc00b860 con 0x7fdfd40ac340 2022-06-17T12:52:19.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfd3fff700 1 -- 172.21.15.36:0/3613328927 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfcc00b9c0 con 0x7fdfd40ac340 2022-06-17T12:52:19.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfd3fff700 1 -- 172.21.15.36:0/3613328927 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdfcc00bb30 con 0x7fdfd40ac340 2022-06-17T12:52:19.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfdb2a4700 1 -- 172.21.15.36:0/3613328927 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfd40ac340 msgr2=0x7fdfd414b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:19.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfdb2a4700 1 --2- 172.21.15.36:0/3613328927 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfd40ac340 0x7fdfd414b3b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7fdfd40b10c0 tx=0x7fdfcc009290).stop 2022-06-17T12:52:19.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfdb2a4700 1 -- 172.21.15.36:0/3613328927 shutdown_connections 2022-06-17T12:52:19.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfdb2a4700 1 --2- 172.21.15.36:0/3613328927 >> v2:172.21.15.70:3300/0 conn(0x7fdfd40af7f0 0x7fdfd40ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfdb2a4700 1 --2- 172.21.15.36:0/3613328927 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfd40ac340 0x7fdfd414b3b0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfdb2a4700 1 -- 172.21.15.36:0/3613328927 >> 172.21.15.36:0/3613328927 conn(0x7fdfd40b5430 msgr2=0x7fdfd40b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:19.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfdb2a4700 1 -- 172.21.15.36:0/3613328927 shutdown_connections 2022-06-17T12:52:19.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.032+0000 7fdfdb2a4700 1 -- 172.21.15.36:0/3613328927 wait complete. 2022-06-17T12:52:19.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.036+0000 7fdfdb2a4700 1 Processor -- start 2022-06-17T12:52:19.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.036+0000 7fdfdb2a4700 1 -- start start 2022-06-17T12:52:19.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.036+0000 7fdfdb2a4700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfd40ac340 0x7fdfd40a86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:19.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.036+0000 7fdfdb2a4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfd40ad140 0x7fdfd4156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:19.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.036+0000 7fdfdb2a4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfd40af7f0 0x7fdfd4149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:19.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.036+0000 7fdfdb2a4700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd41464d0 con 0x7fdfd40ad140 2022-06-17T12:52:19.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.036+0000 7fdfdb2a4700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd4145dd0 con 0x7fdfd40af7f0 2022-06-17T12:52:19.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.036+0000 7fdfdb2a4700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd4146070 con 0x7fdfd40ac340 2022-06-17T12:52:19.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.036+0000 7fdfd883f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfd40ad140 0x7fdfd4156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:19.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.036+0000 7fdfd9841700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfd40af7f0 0x7fdfd4149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:19.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.036+0000 7fdfd883f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfd40ad140 0x7fdfd4156eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34712/0 (socket says 172.21.15.36:34712) 2022-06-17T12:52:19.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.036+0000 7fdfd883f700 1 -- 172.21.15.36:0/2131986424 learned_addr learned my addr 172.21.15.36:0/2131986424 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:19.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.036+0000 7fdfd9040700 1 --2- 172.21.15.36:0/2131986424 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfd40ac340 0x7fdfd40a86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:19.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.036+0000 7fdfd883f700 1 -- 172.21.15.36:0/2131986424 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfd40ac340 msgr2=0x7fdfd40a86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:19.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.036+0000 7fdfd883f700 1 --2- 172.21.15.36:0/2131986424 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfd40ac340 0x7fdfd40a86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.036+0000 7fdfd883f700 1 -- 172.21.15.36:0/2131986424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfd40af7f0 msgr2=0x7fdfd4149f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:19.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.036+0000 7fdfd883f700 1 --2- 172.21.15.36:0/2131986424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfd40af7f0 0x7fdfd4149f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.036+0000 7fdfd883f700 1 -- 172.21.15.36:0/2131986424 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfcc009530 con 0x7fdfd40ad140 2022-06-17T12:52:19.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.036+0000 7fdfd883f700 1 --2- 172.21.15.36:0/2131986424 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfd40ad140 0x7fdfd4156eb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7fdfd40b10c0 tx=0x7fdfcc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:19.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.036+0000 7fdfd27fc700 1 -- 172.21.15.36:0/2131986424 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdfcc0032f0 con 0x7fdfd40ad140 2022-06-17T12:52:19.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.036+0000 7fdfd27fc700 1 -- 172.21.15.36:0/2131986424 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfcc00bb30 con 0x7fdfd40ad140 2022-06-17T12:52:19.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.036+0000 7fdfdb2a4700 1 -- 172.21.15.36:0/2131986424 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd4144690 con 0x7fdfd40ad140 2022-06-17T12:52:19.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.036+0000 7fdfd27fc700 1 -- 172.21.15.36:0/2131986424 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdfcc020610 con 0x7fdfd40ad140 2022-06-17T12:52:19.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.040+0000 7fdfdb2a4700 1 -- 172.21.15.36:0/2131986424 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd40a7a10 con 0x7fdfd40ad140 2022-06-17T12:52:19.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.040+0000 7fdfdb2a4700 1 -- 172.21.15.36:0/2131986424 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfd4144820 con 0x7fdfd40ad140 2022-06-17T12:52:19.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.040+0000 7fdfd27fc700 1 -- 172.21.15.36:0/2131986424 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fdfcc027070 con 0x7fdfd40ad140 2022-06-17T12:52:19.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.040+0000 7fdfd27fc700 1 --2- 172.21.15.36:0/2131986424 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdfb40539b0 0x7fdfb4055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:19.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.044+0000 7fdfd9040700 1 --2- 172.21.15.36:0/2131986424 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdfb40539b0 0x7fdfb4055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:19.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.044+0000 7fdfd27fc700 1 -- 172.21.15.36:0/2131986424 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fdfcc06c980 con 0x7fdfd40ad140 2022-06-17T12:52:19.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.044+0000 7fdfd9040700 1 --2- 172.21.15.36:0/2131986424 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdfb40539b0 0x7fdfb4055e70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fdfc800d000 tx=0x7fdfc8000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:19.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.044+0000 7fdfd27fc700 1 -- 172.21.15.36:0/2131986424 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdfcc043ac0 con 0x7fdfd40ad140 2022-06-17T12:52:19.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.212+0000 7fdfdb2a4700 1 -- 172.21.15.36:0/2131986424 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fdfd4057410 con 0x7fdfd40ad140 2022-06-17T12:52:19.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.212+0000 7fdfd27fc700 1 -- 172.21.15.36:0/2131986424 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+4242 (secure 0 0 0) 0x7fdfcc025020 con 0x7fdfd40ad140 2022-06-17T12:52:19.201 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:52:19.202 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:52:19.202 INFO:teuthology.orchestra.run.smithi036.stdout: "auth_dump": [ 2022-06-17T12:52:19.203 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:19.203 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mds.a", 2022-06-17T12:52:19.203 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiyaUgARAAxp7bzOl4uVo2EsaoRBeMRw==", 2022-06-17T12:52:19.204 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:52:19.205 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:52:19.206 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:52:19.207 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:52:19.207 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:52:19.208 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:19.208 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:19.208 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:19.209 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mds.b", 2022-06-17T12:52:19.209 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xifyxLCRAABI389jmz1zboeDanzraT8Q==", 2022-06-17T12:52:19.210 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:52:19.210 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:52:19.211 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:52:19.212 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:52:19.212 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:52:19.212 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:19.213 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:19.213 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:19.214 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.0", 2022-06-17T12:52:19.215 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB+d6xibhXGNBAA6AuUT9ITWAZjDFpNsgA4Wg==", 2022-06-17T12:52:19.215 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:52:19.218 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:52:19.219 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:52:19.219 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:52:19.219 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:19.220 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:19.220 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:19.221 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.1", 2022-06-17T12:52:19.221 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCBd6xiOIPvGxAADkrBoJCy9x91zan6TWU0tQ==", 2022-06-17T12:52:19.222 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:52:19.222 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:52:19.223 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:52:19.223 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:52:19.224 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:19.224 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:19.225 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:19.226 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.2", 2022-06-17T12:52:19.226 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCEd6xiP4/9AhAAaP3cWXV6vbfGBPfqSLKqUg==", 2022-06-17T12:52:19.226 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:52:19.227 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:52:19.227 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:52:19.228 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:52:19.228 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:19.229 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:19.229 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:19.229 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.3", 2022-06-17T12:52:19.230 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCGd6xi+LZgJxAAJjGwP5qW6TFDZfR+2rz6ig==", 2022-06-17T12:52:19.230 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:52:19.231 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:52:19.231 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:52:19.232 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:52:19.232 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:19.233 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:19.233 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:19.234 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.4", 2022-06-17T12:52:19.234 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCKd6xikvrTChAAXIyWSETswDv+tb3zsGyYpA==", 2022-06-17T12:52:19.234 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:52:19.236 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:52:19.236 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:52:19.236 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:52:19.236 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:19.237 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:19.237 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:19.237 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.5", 2022-06-17T12:52:19.237 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCMd6xiqr2QLhAArhimHuNsVWrDGGreFYxj6A==", 2022-06-17T12:52:19.238 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:52:19.238 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:52:19.238 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:52:19.239 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:52:19.239 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:19.240 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:19.240 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:19.241 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.6", 2022-06-17T12:52:19.241 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCPd6xiUXcXGxAAdrYz8txdBWMIqF5DF4Aimw==", 2022-06-17T12:52:19.242 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:52:19.242 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:52:19.243 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:52:19.243 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:52:19.243 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:19.244 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:19.244 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:19.245 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.7", 2022-06-17T12:52:19.245 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCSd6xiBJ2IAhAAUkNfgmlxxWY7+VouPLAvyw==", 2022-06-17T12:52:19.245 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:52:19.246 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:52:19.246 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:52:19.247 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:52:19.247 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:19.248 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:19.248 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:19.248 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.0", 2022-06-17T12:52:19.249 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiXO3QFBAAsSLtQ0yKKZxhqUT/eYKdtg==", 2022-06-17T12:52:19.250 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:52:19.250 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:52:19.251 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:52:19.251 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:52:19.252 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:19.252 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:19.252 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:19.253 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.1", 2022-06-17T12:52:19.253 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xi51KdGBAAi7fGZ+Qepg6j3DAhlzoWxw==", 2022-06-17T12:52:19.253 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:52:19.253 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:52:19.254 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:52:19.254 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:52:19.254 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:19.254 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:19.255 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:19.255 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.2", 2022-06-17T12:52:19.255 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiug2hHBAAqHpLgwMdthSWtJ8Vwu+m1Q==", 2022-06-17T12:52:19.255 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:52:19.255 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:52:19.256 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:52:19.256 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:52:19.256 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:19.256 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:19.257 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:19.257 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.admin", 2022-06-17T12:52:19.257 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB5d6xiGDODKhAAPZMG/M2nVssdr4FJHcS6XQ==", 2022-06-17T12:52:19.257 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:52:19.257 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:52:19.258 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:52:19.258 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:52:19.258 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:52:19.259 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:19.259 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:19.259 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:19.259 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mgr.x", 2022-06-17T12:52:19.260 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB6d6xiyH9uORAATJb0chHE1CH0X02w7edVtw==", 2022-06-17T12:52:19.260 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:52:19.260 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:52:19.261 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile mgr", 2022-06-17T12:52:19.261 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:52:19.261 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:19.262 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:19.262 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:19.263 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mgr.y", 2022-06-17T12:52:19.263 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB6d6xiHXghMxAAzpsVRCbLWHWZPtrPk1jFlA==", 2022-06-17T12:52:19.264 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:52:19.264 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:52:19.265 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile mgr", 2022-06-17T12:52:19.265 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:52:19.266 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:19.266 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:19.267 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:19.267 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:52:19.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.216+0000 7fdfdb2a4700 1 -- 172.21.15.36:0/2131986424 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdfb40539b0 msgr2=0x7fdfb4055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:19.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.216+0000 7fdfdb2a4700 1 --2- 172.21.15.36:0/2131986424 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdfb40539b0 0x7fdfb4055e70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fdfc800d000 tx=0x7fdfc8000bc0).stop 2022-06-17T12:52:19.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.216+0000 7fdfdb2a4700 1 -- 172.21.15.36:0/2131986424 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfd40ad140 msgr2=0x7fdfd4156eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:19.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.216+0000 7fdfdb2a4700 1 --2- 172.21.15.36:0/2131986424 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfd40ad140 0x7fdfd4156eb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7fdfd40b10c0 tx=0x7fdfcc016000).stop 2022-06-17T12:52:19.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.216+0000 7fdfdb2a4700 1 -- 172.21.15.36:0/2131986424 shutdown_connections 2022-06-17T12:52:19.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.216+0000 7fdfdb2a4700 1 --2- 172.21.15.36:0/2131986424 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdfb40539b0 0x7fdfb4055e70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.216+0000 7fdfdb2a4700 1 --2- 172.21.15.36:0/2131986424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfd40af7f0 0x7fdfd4149f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.216+0000 7fdfdb2a4700 1 --2- 172.21.15.36:0/2131986424 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfd40ad140 0x7fdfd4156eb0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.216+0000 7fdfdb2a4700 1 --2- 172.21.15.36:0/2131986424 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfd40ac340 0x7fdfd40a86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.216+0000 7fdfdb2a4700 1 -- 172.21.15.36:0/2131986424 >> 172.21.15.36:0/2131986424 conn(0x7fdfd40b5430 msgr2=0x7fdfd40b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:19.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.216+0000 7fdfdb2a4700 1 -- 172.21.15.36:0/2131986424 shutdown_connections 2022-06-17T12:52:19.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.216+0000 7fdfdb2a4700 1 -- 172.21.15.36:0/2131986424 wait complete. 2022-06-17T12:52:19.277 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:52:19.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d9d32e700 1 Processor -- start 2022-06-17T12:52:19.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d9d32e700 1 -- start start 2022-06-17T12:52:19.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d9d32e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d98062770 0x7f4d98062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:19.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d9d32e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4d981444f0 0x7f4d98144930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:19.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d9d32e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d980b0ba0 con 0x7f4d980ad330 2022-06-17T12:52:19.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d9d32e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d980ae820 con 0x7f4d981444f0 2022-06-17T12:52:19.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d9d32e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d98061030 con 0x7f4d98062770 2022-06-17T12:52:19.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d967fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4d981444f0 0x7f4d98144930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:19.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d967fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4d981444f0 0x7f4d98144930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34718/0 (socket says 172.21.15.36:34718) 2022-06-17T12:52:19.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d967fc700 1 -- 172.21.15.36:0/2454098311 learned_addr learned my addr 172.21.15.36:0/2454098311 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:19.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d96ffd700 1 --2- 172.21.15.36:0/2454098311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d98062770 0x7f4d98062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:19.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d967fc700 1 -- 172.21.15.36:0/2454098311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d98062770 msgr2=0x7f4d98062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:19.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d967fc700 1 --2- 172.21.15.36:0/2454098311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d98062770 0x7f4d98062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d967fc700 1 -- 172.21.15.36:0/2454098311 >> v1:172.21.15.70:6789/0 conn(0x7f4d980ad330 legacy=0x7f4d980ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:19.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d967fc700 1 -- 172.21.15.36:0/2454098311 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d98147e90 con 0x7f4d981444f0 2022-06-17T12:52:19.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d967fc700 1 --2- 172.21.15.36:0/2454098311 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d981444f0 0x7f4d98144930 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f4d9805d270 tx=0x7f4d90009290).ready entity=mon.0 client_cookie=c250764857cc609e server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:19.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d95ffb700 1 -- 172.21.15.36:0/2454098311 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d9000b820 con 0x7f4d981444f0 2022-06-17T12:52:19.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d95ffb700 1 -- 172.21.15.36:0/2454098311 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9000b980 con 0x7f4d981444f0 2022-06-17T12:52:19.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d95ffb700 1 -- 172.21.15.36:0/2454098311 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d9000bb30 con 0x7f4d981444f0 2022-06-17T12:52:19.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d9d32e700 1 -- 172.21.15.36:0/2454098311 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d981444f0 msgr2=0x7f4d98144930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:19.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d9d32e700 1 --2- 172.21.15.36:0/2454098311 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d981444f0 0x7f4d98144930 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f4d9805d270 tx=0x7f4d90009290).stop 2022-06-17T12:52:19.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d9d32e700 1 -- 172.21.15.36:0/2454098311 shutdown_connections 2022-06-17T12:52:19.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d9d32e700 1 --2- 172.21.15.36:0/2454098311 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d981444f0 0x7f4d98144930 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d9d32e700 1 --2- 172.21.15.36:0/2454098311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d98062770 0x7f4d98062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d9d32e700 1 -- 172.21.15.36:0/2454098311 >> 172.21.15.36:0/2454098311 conn(0x7f4d980b53e0 msgr2=0x7f4d980b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:19.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d9d32e700 1 -- 172.21.15.36:0/2454098311 shutdown_connections 2022-06-17T12:52:19.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.404+0000 7f4d9d32e700 1 -- 172.21.15.36:0/2454098311 wait complete. 2022-06-17T12:52:19.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.408+0000 7f4d9d32e700 1 Processor -- start 2022-06-17T12:52:19.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.408+0000 7f4d9d32e700 1 -- start start 2022-06-17T12:52:19.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.408+0000 7f4d9d32e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d98062770 0x7f4d9814e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:19.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.408+0000 7f4d9d32e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d980ad330 0x7f4d9812f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:19.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.408+0000 7f4d9d32e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d981444f0 0x7f4d9812f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:19.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.408+0000 7f4d9d32e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d98149780 con 0x7f4d980ad330 2022-06-17T12:52:19.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.408+0000 7f4d9d32e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9812fb70 con 0x7f4d981444f0 2022-06-17T12:52:19.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.408+0000 7f4d9d32e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d980631e0 con 0x7f4d98062770 2022-06-17T12:52:19.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.408+0000 7f4d967fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d980ad330 0x7f4d9812f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:19.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.408+0000 7f4d96ffd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d98062770 0x7f4d9814e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:19.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.408+0000 7f4d967fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d980ad330 0x7f4d9812f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34726/0 (socket says 172.21.15.36:34726) 2022-06-17T12:52:19.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.408+0000 7f4d967fc700 1 -- 172.21.15.36:0/3012849440 learned_addr learned my addr 172.21.15.36:0/3012849440 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:19.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.408+0000 7f4d977fe700 1 --2- 172.21.15.36:0/3012849440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d981444f0 0x7f4d9812f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:19.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.412+0000 7f4d967fc700 1 -- 172.21.15.36:0/3012849440 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d98062770 msgr2=0x7f4d9814e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:19.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.412+0000 7f4d967fc700 1 --2- 172.21.15.36:0/3012849440 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d98062770 0x7f4d9814e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.412+0000 7f4d967fc700 1 -- 172.21.15.36:0/3012849440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d981444f0 msgr2=0x7f4d9812f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:19.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.412+0000 7f4d967fc700 1 --2- 172.21.15.36:0/3012849440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d981444f0 0x7f4d9812f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.412+0000 7f4d967fc700 1 -- 172.21.15.36:0/3012849440 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d90009530 con 0x7f4d980ad330 2022-06-17T12:52:19.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.412+0000 7f4d967fc700 1 --2- 172.21.15.36:0/3012849440 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d980ad330 0x7f4d9812f190 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f4d9805d270 tx=0x7f4d90016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:19.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.412+0000 7f4d8ffff700 1 -- 172.21.15.36:0/3012849440 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d90021440 con 0x7f4d980ad330 2022-06-17T12:52:19.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.412+0000 7f4d8ffff700 1 -- 172.21.15.36:0/3012849440 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d900215a0 con 0x7f4d980ad330 2022-06-17T12:52:19.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.412+0000 7f4d9d32e700 1 -- 172.21.15.36:0/3012849440 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d9814b620 con 0x7f4d980ad330 2022-06-17T12:52:19.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.412+0000 7f4d8ffff700 1 -- 172.21.15.36:0/3012849440 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d90021900 con 0x7f4d980ad330 2022-06-17T12:52:19.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.412+0000 7f4d9d32e700 1 -- 172.21.15.36:0/3012849440 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d98157870 con 0x7f4d980ad330 2022-06-17T12:52:19.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.412+0000 7f4d9d32e700 1 -- 172.21.15.36:0/3012849440 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d980b1c20 con 0x7f4d980ad330 2022-06-17T12:52:19.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.412+0000 7f4d8ffff700 1 -- 172.21.15.36:0/3012849440 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f4d90027020 con 0x7f4d980ad330 2022-06-17T12:52:19.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.412+0000 7f4d8ffff700 1 --2- 172.21.15.36:0/3012849440 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d7c0539b0 0x7f4d7c055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:19.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.416+0000 7f4d96ffd700 1 --2- 172.21.15.36:0/3012849440 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d7c0539b0 0x7f4d7c055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:19.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.416+0000 7f4d8ffff700 1 -- 172.21.15.36:0/3012849440 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f4d9006d0b0 con 0x7f4d980ad330 2022-06-17T12:52:19.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.416+0000 7f4d96ffd700 1 --2- 172.21.15.36:0/3012849440 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d7c0539b0 0x7f4d7c055e70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f4d8000c000 tx=0x7f4d80008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:19.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.420+0000 7f4d8ffff700 1 -- 172.21.15.36:0/3012849440 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4d900441d0 con 0x7f4d980ad330 2022-06-17T12:52:19.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.548+0000 7f4d9d32e700 1 -- 172.21.15.36:0/3012849440 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d98057410 con 0x7f4d980ad330 2022-06-17T12:52:19.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.552+0000 7f4d8ffff700 1 -- 172.21.15.36:0/3012849440 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+13415 (secure 0 0 0) 0x7f4d90015070 con 0x7f4d980ad330 2022-06-17T12:52:19.538 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:52:19.539 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":61,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:52:18.900498+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:52:19.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.552+0000 7f4d9d32e700 1 -- 172.21.15.36:0/3012849440 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d7c0539b0 msgr2=0x7f4d7c055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:19.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.552+0000 7f4d9d32e700 1 --2- 172.21.15.36:0/3012849440 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d7c0539b0 0x7f4d7c055e70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f4d8000c000 tx=0x7f4d80008040).stop 2022-06-17T12:52:19.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.552+0000 7f4d9d32e700 1 -- 172.21.15.36:0/3012849440 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d980ad330 msgr2=0x7f4d9812f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:19.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.552+0000 7f4d9d32e700 1 --2- 172.21.15.36:0/3012849440 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d980ad330 0x7f4d9812f190 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f4d9805d270 tx=0x7f4d90016000).stop 2022-06-17T12:52:19.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.552+0000 7f4d9d32e700 1 -- 172.21.15.36:0/3012849440 shutdown_connections 2022-06-17T12:52:19.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.552+0000 7f4d9d32e700 1 --2- 172.21.15.36:0/3012849440 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d7c0539b0 0x7f4d7c055e70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.552+0000 7f4d9d32e700 1 --2- 172.21.15.36:0/3012849440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d981444f0 0x7f4d9812f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.552+0000 7f4d9d32e700 1 --2- 172.21.15.36:0/3012849440 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d980ad330 0x7f4d9812f190 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.552+0000 7f4d9d32e700 1 --2- 172.21.15.36:0/3012849440 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d98062770 0x7f4d9814e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.552+0000 7f4d9d32e700 1 -- 172.21.15.36:0/3012849440 >> 172.21.15.36:0/3012849440 conn(0x7f4d980b53e0 msgr2=0x7f4d980b4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:19.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.552+0000 7f4d9d32e700 1 -- 172.21.15.36:0/3012849440 shutdown_connections 2022-06-17T12:52:19.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.552+0000 7f4d9d32e700 1 -- 172.21.15.36:0/3012849440 wait complete. 2022-06-17T12:52:19.557 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-17T12:52:19.559 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:52:19.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.684+0000 7f3a1112e700 1 Processor -- start 2022-06-17T12:52:19.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.684+0000 7f3a1112e700 1 -- start start 2022-06-17T12:52:19.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.684+0000 7f3a1112e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3a0c0b27b0 0x7f3a0c14b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:19.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.684+0000 7f3a1112e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f3a0c060ca0 0x7f3a0c0ac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:19.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.684+0000 7f3a1112e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a0c147bd0 0x7f3a0c1442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:19.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.684+0000 7f3a1112e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a0c14b870 con 0x7f3a0c060ca0 2022-06-17T12:52:19.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.684+0000 7f3a1112e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a0c0ac500 con 0x7f3a0c0b27b0 2022-06-17T12:52:19.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.684+0000 7f3a1112e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c0b2b90 con 0x7f3a0c147bd0 2022-06-17T12:52:19.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.684+0000 7f3a0b7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f3a0c060ca0 0x7f3a0c0ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:19.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.684+0000 7f3a0b7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f3a0c060ca0 0x7f3a0c0ac780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34736/0 (socket says 172.21.15.36:34736) 2022-06-17T12:52:19.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.684+0000 7f3a0b7fe700 1 -- 172.21.15.36:0/1827274642 learned_addr learned my addr 172.21.15.36:0/1827274642 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:19.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.684+0000 7f3a1092d700 1 --2- 172.21.15.36:0/1827274642 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a0c147bd0 0x7f3a0c1442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:19.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.684+0000 7f3a0bfff700 1 --2- 172.21.15.36:0/1827274642 >> v2:172.21.15.70:3300/0 conn(0x7f3a0c0b27b0 0x7f3a0c14b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:19.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.684+0000 7f3a0b7fe700 1 -- 172.21.15.36:0/1827274642 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a0c147bd0 msgr2=0x7f3a0c1442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:19.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.684+0000 7f3a0b7fe700 1 --2- 172.21.15.36:0/1827274642 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a0c147bd0 0x7f3a0c1442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.684+0000 7f3a0b7fe700 1 -- 172.21.15.36:0/1827274642 >> v2:172.21.15.70:3300/0 conn(0x7f3a0c0b27b0 msgr2=0x7f3a0c14b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:19.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.684+0000 7f3a0b7fe700 1 --2- 172.21.15.36:0/1827274642 >> v2:172.21.15.70:3300/0 conn(0x7f3a0c0b27b0 0x7f3a0c14b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.684+0000 7f3a0b7fe700 1 -- 172.21.15.36:0/1827274642 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0c147490 con 0x7f3a0c060ca0 2022-06-17T12:52:19.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.684+0000 7f3a0bfff700 1 --2- 172.21.15.36:0/1827274642 >> v2:172.21.15.70:3300/0 conn(0x7f3a0c0b27b0 0x7f3a0c14b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:52:19.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.684+0000 7f3a0b7fe700 1 --2- 172.21.15.36:0/1827274642 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3a0c060ca0 0x7f3a0c0ac780 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f39fc002f70 tx=0x7f39fc006eb0).ready entity=mon.0 client_cookie=81606615ba1cd2f7 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:19.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.684+0000 7f3a0affd700 1 -- 172.21.15.36:0/1827274642 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39fc00d570 con 0x7f3a0c060ca0 2022-06-17T12:52:19.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a0affd700 1 -- 172.21.15.36:0/1827274642 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39fc000e90 con 0x7f3a0c060ca0 2022-06-17T12:52:19.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a0affd700 1 -- 172.21.15.36:0/1827274642 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39fc00d900 con 0x7f3a0c060ca0 2022-06-17T12:52:19.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a1112e700 1 -- 172.21.15.36:0/1827274642 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3a0c060ca0 msgr2=0x7f3a0c0ac780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:19.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a1112e700 1 --2- 172.21.15.36:0/1827274642 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3a0c060ca0 0x7f3a0c0ac780 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f39fc002f70 tx=0x7f39fc006eb0).stop 2022-06-17T12:52:19.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a1112e700 1 -- 172.21.15.36:0/1827274642 shutdown_connections 2022-06-17T12:52:19.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a1112e700 1 --2- 172.21.15.36:0/1827274642 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a0c147bd0 0x7f3a0c1442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a1112e700 1 --2- 172.21.15.36:0/1827274642 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3a0c060ca0 0x7f3a0c0ac780 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a1112e700 1 --2- 172.21.15.36:0/1827274642 >> v2:172.21.15.70:3300/0 conn(0x7f3a0c0b27b0 0x7f3a0c14b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a1112e700 1 -- 172.21.15.36:0/1827274642 >> 172.21.15.36:0/1827274642 conn(0x7f3a0c0b5d10 msgr2=0x7f3a0c0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:19.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a1112e700 1 -- 172.21.15.36:0/1827274642 shutdown_connections 2022-06-17T12:52:19.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a1112e700 1 -- 172.21.15.36:0/1827274642 wait complete. 2022-06-17T12:52:19.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a1112e700 1 Processor -- start 2022-06-17T12:52:19.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a1112e700 1 -- start start 2022-06-17T12:52:19.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a1112e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3a0c0b27b0 0x7f3a0c14a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:19.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a1112e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a0c147bd0 0x7f3a0c149ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:19.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a1112e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a0c144500 0x7f3a0c1449b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:19.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a1112e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c0ab7f0 con 0x7f3a0c0b27b0 2022-06-17T12:52:19.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a1112e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c0a9360 con 0x7f3a0c144500 2022-06-17T12:52:19.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a1112e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c0aba50 con 0x7f3a0c147bd0 2022-06-17T12:52:19.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a0bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3a0c0b27b0 0x7f3a0c14a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:19.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a0bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3a0c0b27b0 0x7f3a0c14a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34738/0 (socket says 172.21.15.36:34738) 2022-06-17T12:52:19.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a0bfff700 1 -- 172.21.15.36:0/586572626 learned_addr learned my addr 172.21.15.36:0/586572626 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:19.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a0b7fe700 1 --2- 172.21.15.36:0/586572626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a0c147bd0 0x7f3a0c149ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:19.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a1092d700 1 --2- 172.21.15.36:0/586572626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a0c144500 0x7f3a0c1449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:19.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a0bfff700 1 -- 172.21.15.36:0/586572626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a0c147bd0 msgr2=0x7f3a0c149ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:19.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a0bfff700 1 --2- 172.21.15.36:0/586572626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a0c147bd0 0x7f3a0c149ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a0bfff700 1 -- 172.21.15.36:0/586572626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a0c144500 msgr2=0x7f3a0c1449b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:19.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a0bfff700 1 --2- 172.21.15.36:0/586572626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a0c144500 0x7f3a0c1449b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a0bfff700 1 -- 172.21.15.36:0/586572626 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39fc00b1e0 con 0x7f3a0c0b27b0 2022-06-17T12:52:19.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a0b7fe700 1 --2- 172.21.15.36:0/586572626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a0c147bd0 0x7f3a0c149ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:52:19.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a0bfff700 1 --2- 172.21.15.36:0/586572626 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3a0c0b27b0 0x7f3a0c14a7b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f3a00006940 tx=0x7f3a00008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:19.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a097fa700 1 -- 172.21.15.36:0/586572626 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a00011780 con 0x7f3a0c0b27b0 2022-06-17T12:52:19.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a097fa700 1 -- 172.21.15.36:0/586572626 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a000118e0 con 0x7f3a0c0b27b0 2022-06-17T12:52:19.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a1112e700 1 -- 172.21.15.36:0/586572626 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a0c0a9020 con 0x7f3a0c0b27b0 2022-06-17T12:52:19.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a097fa700 1 -- 172.21.15.36:0/586572626 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a00011c40 con 0x7f3a0c0b27b0 2022-06-17T12:52:19.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.688+0000 7f3a1112e700 1 -- 172.21.15.36:0/586572626 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a0c144ea0 con 0x7f3a0c0b27b0 2022-06-17T12:52:19.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.692+0000 7f3a1112e700 1 -- 172.21.15.36:0/586572626 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a0c0abc10 con 0x7f3a0c0b27b0 2022-06-17T12:52:19.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.692+0000 7f3a097fa700 1 -- 172.21.15.36:0/586572626 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f3a0001c460 con 0x7f3a0c0b27b0 2022-06-17T12:52:19.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.692+0000 7f3a097fa700 1 --2- 172.21.15.36:0/586572626 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f39ec0539b0 0x7f39ec055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:19.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.692+0000 7f3a097fa700 1 -- 172.21.15.36:0/586572626 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f3a0006bf80 con 0x7f3a0c0b27b0 2022-06-17T12:52:19.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.692+0000 7f3a0b7fe700 1 --2- 172.21.15.36:0/586572626 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f39ec0539b0 0x7f39ec055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:19.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.692+0000 7f3a0b7fe700 1 --2- 172.21.15.36:0/586572626 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f39ec0539b0 0x7f39ec055e70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f39fc01a000 tx=0x7f39fc015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:19.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.700+0000 7f3a097fa700 1 -- 172.21.15.36:0/586572626 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3a00043060 con 0x7f3a0c0b27b0 2022-06-17T12:52:19.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.824+0000 7f3a1112e700 1 -- 172.21.15.36:0/586572626 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f3a0c057410 con 0x7f3a0c0b27b0 2022-06-17T12:52:19.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.824+0000 7f3a097fa700 1 -- 172.21.15.36:0/586572626 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v61) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3a00046680 con 0x7f3a0c0b27b0 2022-06-17T12:52:19.812 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:52:19.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.828+0000 7f3a1112e700 1 -- 172.21.15.36:0/586572626 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f39ec0539b0 msgr2=0x7f39ec055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:19.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.828+0000 7f3a1112e700 1 --2- 172.21.15.36:0/586572626 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f39ec0539b0 0x7f39ec055e70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f39fc01a000 tx=0x7f39fc015000).stop 2022-06-17T12:52:19.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.828+0000 7f3a1112e700 1 -- 172.21.15.36:0/586572626 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3a0c0b27b0 msgr2=0x7f3a0c14a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:19.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.828+0000 7f3a1112e700 1 --2- 172.21.15.36:0/586572626 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3a0c0b27b0 0x7f3a0c14a7b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f3a00006940 tx=0x7f3a00008040).stop 2022-06-17T12:52:19.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.828+0000 7f3a1112e700 1 -- 172.21.15.36:0/586572626 shutdown_connections 2022-06-17T12:52:19.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.828+0000 7f3a1112e700 1 --2- 172.21.15.36:0/586572626 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f39ec0539b0 0x7f39ec055e70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.828+0000 7f3a1112e700 1 --2- 172.21.15.36:0/586572626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a0c144500 0x7f3a0c1449b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.828+0000 7f3a1112e700 1 --2- 172.21.15.36:0/586572626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a0c147bd0 0x7f3a0c149ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.828+0000 7f3a1112e700 1 --2- 172.21.15.36:0/586572626 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3a0c0b27b0 0x7f3a0c14a7b0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.828+0000 7f3a1112e700 1 -- 172.21.15.36:0/586572626 >> 172.21.15.36:0/586572626 conn(0x7f3a0c0b5d10 msgr2=0x7f3a0c0b0210 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:19.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.828+0000 7f3a1112e700 1 -- 172.21.15.36:0/586572626 shutdown_connections 2022-06-17T12:52:19.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.828+0000 7f3a1112e700 1 -- 172.21.15.36:0/586572626 wait complete. 2022-06-17T12:52:19.827 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:52:19.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.952+0000 7f836bfff700 1 Processor -- start 2022-06-17T12:52:19.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.952+0000 7f836bfff700 1 -- start start 2022-06-17T12:52:19.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.952+0000 7f836bfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f836c13fe30 0x7f836c0ab9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:19.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.952+0000 7f836bfff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f836c0af610 con 0x7f836c060920 2022-06-17T12:52:19.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.952+0000 7f836bfff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f836c0af8b0 con 0x7f836c0b11a0 2022-06-17T12:52:19.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.952+0000 7f836bfff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f836c0ae190 con 0x7f836c13fe30 2022-06-17T12:52:19.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.952+0000 7f836affd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f836c060920 0x7f836c138ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52276/0 (socket says 172.21.15.36:52276) 2022-06-17T12:52:19.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.952+0000 7f836affd700 1 -- 172.21.15.36:0/621513985 learned_addr learned my addr 172.21.15.36:0/621513985 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:19.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.952+0000 7f836a7fc700 1 --2- 172.21.15.36:0/621513985 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f836c13fe30 0x7f836c0ab9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:19.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.952+0000 7f8369ffb700 1 -- 172.21.15.36:0/621513985 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 702135827 0 0) 0x7f836c0af610 con 0x7f836c060920 2022-06-17T12:52:19.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.952+0000 7f8369ffb700 1 -- 172.21.15.36:0/621513985 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8350004370 con 0x7f836c060920 2022-06-17T12:52:19.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.952+0000 7f8369ffb700 1 -- 172.21.15.36:0/621513985 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 922242033 0 0) 0x7f836c0af8b0 con 0x7f836c0b11a0 2022-06-17T12:52:19.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.952+0000 7f8369ffb700 1 -- 172.21.15.36:0/621513985 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f836c0af610 con 0x7f836c0b11a0 2022-06-17T12:52:19.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.952+0000 7f8369ffb700 1 -- 172.21.15.36:0/621513985 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2525417751 0 0) 0x7f8350004370 con 0x7f836c060920 2022-06-17T12:52:19.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.952+0000 7f836a7fc700 1 -- 172.21.15.36:0/621513985 >> v1:172.21.15.70:6789/0 conn(0x7f836c0b11a0 legacy=0x7f836c0abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:19.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.952+0000 7f836a7fc700 1 -- 172.21.15.36:0/621513985 >> v1:172.21.15.36:6789/0 conn(0x7f836c060920 legacy=0x7f836c138ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:19.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.952+0000 7f836a7fc700 1 -- 172.21.15.36:0/621513985 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836c0ac780 con 0x7f836c13fe30 2022-06-17T12:52:19.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.956+0000 7f836a7fc700 1 --2- 172.21.15.36:0/621513985 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f836c13fe30 0x7f836c0ab9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f835c001820 tx=0x7f8360006df0).ready entity=mon.2 client_cookie=9df29f11cf8a9ec3 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:19.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.956+0000 7f8369ffb700 1 -- 172.21.15.36:0/621513985 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f835c001680 con 0x7f836c13fe30 2022-06-17T12:52:19.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.956+0000 7f8369ffb700 1 -- 172.21.15.36:0/621513985 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8360000e90 con 0x7f836c13fe30 2022-06-17T12:52:19.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.956+0000 7f8369ffb700 1 -- 172.21.15.36:0/621513985 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8360012460 con 0x7f836c13fe30 2022-06-17T12:52:19.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.956+0000 7f836bfff700 1 -- 172.21.15.36:0/621513985 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f836c13fe30 msgr2=0x7f836c0ab9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:19.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.956+0000 7f836bfff700 1 --2- 172.21.15.36:0/621513985 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f836c13fe30 0x7f836c0ab9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f835c001820 tx=0x7f8360006df0).stop 2022-06-17T12:52:19.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.956+0000 7f836bfff700 1 -- 172.21.15.36:0/621513985 shutdown_connections 2022-06-17T12:52:19.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.956+0000 7f836bfff700 1 --2- 172.21.15.36:0/621513985 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f836c13fe30 0x7f836c0ab9a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.956+0000 7f836bfff700 1 -- 172.21.15.36:0/621513985 >> 172.21.15.36:0/621513985 conn(0x7f836c0b5d10 msgr2=0x7f836c0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:19.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.956+0000 7f836bfff700 1 -- 172.21.15.36:0/621513985 shutdown_connections 2022-06-17T12:52:19.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.956+0000 7f836bfff700 1 -- 172.21.15.36:0/621513985 wait complete. 2022-06-17T12:52:19.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.956+0000 7f836bfff700 1 Processor -- start 2022-06-17T12:52:19.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.956+0000 7f836bfff700 1 -- start start 2022-06-17T12:52:19.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.960+0000 7f836bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f836c060920 0x7f836c065140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:19.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.960+0000 7f836bfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f836c0b11a0 0x7f836c1518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:19.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.960+0000 7f836bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f836c13fe30 0x7f836c09f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:19.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.960+0000 7f836bfff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836c0a0390 con 0x7f836c13fe30 2022-06-17T12:52:19.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.960+0000 7f836bfff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836c065630 con 0x7f836c060920 2022-06-17T12:52:19.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.960+0000 7f836bfff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f836c09ee30 con 0x7f836c0b11a0 2022-06-17T12:52:19.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.960+0000 7f836b7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f836c13fe30 0x7f836c09f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:19.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.960+0000 7f836b7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f836c13fe30 0x7f836c09f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34756/0 (socket says 172.21.15.36:34756) 2022-06-17T12:52:19.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.960+0000 7f836b7fe700 1 -- 172.21.15.36:0/2804977737 learned_addr learned my addr 172.21.15.36:0/2804977737 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:19.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.960+0000 7f836a7fc700 1 --2- 172.21.15.36:0/2804977737 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f836c0b11a0 0x7f836c1518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:19.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.960+0000 7f836affd700 1 --2- 172.21.15.36:0/2804977737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f836c060920 0x7f836c065140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:19.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.960+0000 7f836b7fe700 1 -- 172.21.15.36:0/2804977737 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f836c0b11a0 msgr2=0x7f836c1518f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:19.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.960+0000 7f836b7fe700 1 --2- 172.21.15.36:0/2804977737 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f836c0b11a0 0x7f836c1518f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.960+0000 7f836b7fe700 1 -- 172.21.15.36:0/2804977737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f836c060920 msgr2=0x7f836c065140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:19.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.960+0000 7f836b7fe700 1 --2- 172.21.15.36:0/2804977737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f836c060920 0x7f836c065140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:19.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.960+0000 7f836b7fe700 1 -- 172.21.15.36:0/2804977737 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836000b180 con 0x7f836c13fe30 2022-06-17T12:52:19.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.960+0000 7f836b7fe700 1 --2- 172.21.15.36:0/2804977737 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f836c13fe30 0x7f836c09f150 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f8364001490 tx=0x7f8364007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:19.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.960+0000 7f835bfff700 1 -- 172.21.15.36:0/2804977737 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8364002310 con 0x7f836c13fe30 2022-06-17T12:52:19.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.960+0000 7f835bfff700 1 -- 172.21.15.36:0/2804977737 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8364002470 con 0x7f836c13fe30 2022-06-17T12:52:19.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.960+0000 7f836bfff700 1 -- 172.21.15.36:0/2804977737 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f836c062de0 con 0x7f836c13fe30 2022-06-17T12:52:19.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.960+0000 7f835bfff700 1 -- 172.21.15.36:0/2804977737 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8364001840 con 0x7f836c13fe30 2022-06-17T12:52:19.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.960+0000 7f836bfff700 1 -- 172.21.15.36:0/2804977737 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f836c0643c0 con 0x7f836c13fe30 2022-06-17T12:52:19.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.960+0000 7f836bfff700 1 -- 172.21.15.36:0/2804977737 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f836c057410 con 0x7f836c13fe30 2022-06-17T12:52:19.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.964+0000 7f835bfff700 1 -- 172.21.15.36:0/2804977737 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8364010030 con 0x7f836c13fe30 2022-06-17T12:52:19.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.964+0000 7f835bfff700 1 --2- 172.21.15.36:0/2804977737 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8348053920 0x7f8348055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:19.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.964+0000 7f835bfff700 1 -- 172.21.15.36:0/2804977737 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f836406bd90 con 0x7f836c13fe30 2022-06-17T12:52:19.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.964+0000 7f836affd700 1 --2- 172.21.15.36:0/2804977737 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8348053920 0x7f8348055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:19.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.964+0000 7f836affd700 1 --2- 172.21.15.36:0/2804977737 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8348053920 0x7f8348055de0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f835c00d000 tx=0x7f835c001700).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:19.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:19.968+0000 7f835bfff700 1 -- 172.21.15.36:0/2804977737 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8364042d40 con 0x7f836c13fe30 2022-06-17T12:52:20.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.100+0000 7f836bfff700 1 -- 172.21.15.36:0/2804977737 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f836c1521a0 con 0x7f836c13fe30 2022-06-17T12:52:20.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.100+0000 7f835bfff700 1 -- 172.21.15.36:0/2804977737 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v61) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8364046360 con 0x7f836c13fe30 2022-06-17T12:52:20.087 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:52:20.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.104+0000 7f836bfff700 1 -- 172.21.15.36:0/2804977737 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8348053920 msgr2=0x7f8348055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:20.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.104+0000 7f836bfff700 1 --2- 172.21.15.36:0/2804977737 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8348053920 0x7f8348055de0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f835c00d000 tx=0x7f835c001700).stop 2022-06-17T12:52:20.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.104+0000 7f836bfff700 1 -- 172.21.15.36:0/2804977737 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f836c13fe30 msgr2=0x7f836c09f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:20.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.104+0000 7f836bfff700 1 --2- 172.21.15.36:0/2804977737 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f836c13fe30 0x7f836c09f150 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f8364001490 tx=0x7f8364007dd0).stop 2022-06-17T12:52:20.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.104+0000 7f836bfff700 1 -- 172.21.15.36:0/2804977737 shutdown_connections 2022-06-17T12:52:20.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.104+0000 7f836bfff700 1 --2- 172.21.15.36:0/2804977737 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8348053920 0x7f8348055de0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:20.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.104+0000 7f836bfff700 1 --2- 172.21.15.36:0/2804977737 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f836c13fe30 0x7f836c09f150 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:20.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.104+0000 7f836bfff700 1 --2- 172.21.15.36:0/2804977737 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f836c0b11a0 0x7f836c1518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:20.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.104+0000 7f836bfff700 1 --2- 172.21.15.36:0/2804977737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f836c060920 0x7f836c065140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:20.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.104+0000 7f836bfff700 1 -- 172.21.15.36:0/2804977737 >> 172.21.15.36:0/2804977737 conn(0x7f836c0b5d10 msgr2=0x7f836c0b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:20.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.104+0000 7f836bfff700 1 -- 172.21.15.36:0/2804977737 shutdown_connections 2022-06-17T12:52:20.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.104+0000 7f836bfff700 1 -- 172.21.15.36:0/2804977737 wait complete. 2022-06-17T12:52:20.101 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-17T12:52:20.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.216+0000 7f4912c36700 1 Processor -- start 2022-06-17T12:52:20.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.216+0000 7f4912c36700 1 -- start start 2022-06-17T12:52:20.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f4912c36700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f490c062770 0x7f490c062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:20.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f4912c36700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f490c1444f0 0x7f490c144930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:20.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f4912c36700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490c0b0ba0 con 0x7f490c0ad330 2022-06-17T12:52:20.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f4912c36700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f490c0ae820 con 0x7f490c062770 2022-06-17T12:52:20.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f4912c36700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c061030 con 0x7f490c1444f0 2022-06-17T12:52:20.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f49111d3700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f490c0ad330 0x7f490c0ad770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:48028/0 (socket says 172.21.15.36:48028) 2022-06-17T12:52:20.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f49111d3700 1 -- 172.21.15.36:0/634089209 learned_addr learned my addr 172.21.15.36:0/634089209 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:20.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f490bfff700 1 --2- 172.21.15.36:0/634089209 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f490c1444f0 0x7f490c144930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:20.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f49109d2700 1 --2- 172.21.15.36:0/634089209 >> v2:172.21.15.70:3300/0 conn(0x7f490c062770 0x7f490c062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:20.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f490b7fe700 1 -- 172.21.15.36:0/634089209 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3336312686 0 0) 0x7f490c0b0ba0 con 0x7f490c0ad330 2022-06-17T12:52:20.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f490b7fe700 1 -- 172.21.15.36:0/634089209 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48f8003260 con 0x7f490c0ad330 2022-06-17T12:52:20.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f490bfff700 1 -- 172.21.15.36:0/634089209 >> v2:172.21.15.70:3300/0 conn(0x7f490c062770 msgr2=0x7f490c062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:20.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f490bfff700 1 --2- 172.21.15.36:0/634089209 >> v2:172.21.15.70:3300/0 conn(0x7f490c062770 0x7f490c062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:20.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f490bfff700 1 -- 172.21.15.36:0/634089209 >> v1:172.21.15.70:6789/0 conn(0x7f490c0ad330 legacy=0x7f490c0ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:20.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f490bfff700 1 -- 172.21.15.36:0/634089209 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c147e90 con 0x7f490c1444f0 2022-06-17T12:52:20.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f490bfff700 1 --2- 172.21.15.36:0/634089209 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f490c1444f0 0x7f490c144930 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f4900002f70 tx=0x7f4900006df0).ready entity=mon.2 client_cookie=ebb413e1c83de797 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:20.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f490b7fe700 1 -- 172.21.15.36:0/634089209 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f490000d4a0 con 0x7f490c1444f0 2022-06-17T12:52:20.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f490b7fe700 1 -- 172.21.15.36:0/634089209 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4900000e90 con 0x7f490c1444f0 2022-06-17T12:52:20.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f490b7fe700 1 -- 172.21.15.36:0/634089209 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f490000d7f0 con 0x7f490c1444f0 2022-06-17T12:52:20.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f4912c36700 1 -- 172.21.15.36:0/634089209 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f490c1444f0 msgr2=0x7f490c144930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:20.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f4912c36700 1 --2- 172.21.15.36:0/634089209 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f490c1444f0 0x7f490c144930 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f4900002f70 tx=0x7f4900006df0).stop 2022-06-17T12:52:20.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f4912c36700 1 -- 172.21.15.36:0/634089209 shutdown_connections 2022-06-17T12:52:20.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f4912c36700 1 --2- 172.21.15.36:0/634089209 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f490c1444f0 0x7f490c144930 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:20.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f4912c36700 1 --2- 172.21.15.36:0/634089209 >> v2:172.21.15.70:3300/0 conn(0x7f490c062770 0x7f490c062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:20.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f4912c36700 1 -- 172.21.15.36:0/634089209 >> 172.21.15.36:0/634089209 conn(0x7f490c0b53e0 msgr2=0x7f490c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:20.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f4912c36700 1 -- 172.21.15.36:0/634089209 shutdown_connections 2022-06-17T12:52:20.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.220+0000 7f4912c36700 1 -- 172.21.15.36:0/634089209 wait complete. 2022-06-17T12:52:20.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.224+0000 7f4912c36700 1 Processor -- start 2022-06-17T12:52:20.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.224+0000 7f4912c36700 1 -- start start 2022-06-17T12:52:20.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.224+0000 7f4912c36700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f490c062770 0x7f490c063090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:20.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.224+0000 7f4912c36700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f490c0ad330 0x7f490c0adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:20.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.224+0000 7f4912c36700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f490c1444f0 0x7f490c159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:20.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.224+0000 7f4912c36700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c065550 con 0x7f490c0ad330 2022-06-17T12:52:20.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.224+0000 7f4912c36700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c064520 con 0x7f490c1444f0 2022-06-17T12:52:20.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.224+0000 7f4912c36700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c0647c0 con 0x7f490c062770 2022-06-17T12:52:20.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.224+0000 7f490bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f490c0ad330 0x7f490c0adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:20.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.224+0000 7f490bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f490c0ad330 0x7f490c0adab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34768/0 (socket says 172.21.15.36:34768) 2022-06-17T12:52:20.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.224+0000 7f490bfff700 1 -- 172.21.15.36:0/3721071481 learned_addr learned my addr 172.21.15.36:0/3721071481 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:20.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.224+0000 7f49111d3700 1 --2- 172.21.15.36:0/3721071481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f490c1444f0 0x7f490c159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:20.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.224+0000 7f49109d2700 1 --2- 172.21.15.36:0/3721071481 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f490c062770 0x7f490c063090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:20.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.224+0000 7f490bfff700 1 -- 172.21.15.36:0/3721071481 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f490c062770 msgr2=0x7f490c063090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:20.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.224+0000 7f490bfff700 1 --2- 172.21.15.36:0/3721071481 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f490c062770 0x7f490c063090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:20.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.224+0000 7f490bfff700 1 -- 172.21.15.36:0/3721071481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f490c1444f0 msgr2=0x7f490c159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:20.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.224+0000 7f490bfff700 1 --2- 172.21.15.36:0/3721071481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f490c1444f0 0x7f490c159be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:20.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.224+0000 7f490bfff700 1 -- 172.21.15.36:0/3721071481 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490000b180 con 0x7f490c0ad330 2022-06-17T12:52:20.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.224+0000 7f490bfff700 1 --2- 172.21.15.36:0/3721071481 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f490c0ad330 0x7f490c0adab0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f4900002f70 tx=0x7f4900014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:20.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.224+0000 7f4909ffb700 1 -- 172.21.15.36:0/3721071481 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49000139c0 con 0x7f490c0ad330 2022-06-17T12:52:20.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.224+0000 7f4909ffb700 1 -- 172.21.15.36:0/3721071481 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4900013b20 con 0x7f490c0ad330 2022-06-17T12:52:20.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.224+0000 7f4912c36700 1 -- 172.21.15.36:0/3721071481 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490c066900 con 0x7f490c0ad330 2022-06-17T12:52:20.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.224+0000 7f4909ffb700 1 -- 172.21.15.36:0/3721071481 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4900013e80 con 0x7f490c0ad330 2022-06-17T12:52:20.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.224+0000 7f4912c36700 1 -- 172.21.15.36:0/3721071481 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490c063730 con 0x7f490c0ad330 2022-06-17T12:52:20.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.228+0000 7f4909ffb700 1 -- 172.21.15.36:0/3721071481 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f4900025030 con 0x7f490c0ad330 2022-06-17T12:52:20.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.228+0000 7f4912c36700 1 -- 172.21.15.36:0/3721071481 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f490c0b1c20 con 0x7f490c0ad330 2022-06-17T12:52:20.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.228+0000 7f4909ffb700 1 --2- 172.21.15.36:0/3721071481 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f48ec053a40 0x7f48ec055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:20.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.228+0000 7f4909ffb700 1 -- 172.21.15.36:0/3721071481 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f490006cc80 con 0x7f490c0ad330 2022-06-17T12:52:20.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.228+0000 7f49109d2700 1 --2- 172.21.15.36:0/3721071481 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f48ec053a40 0x7f48ec055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:20.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.232+0000 7f49109d2700 1 --2- 172.21.15.36:0/3721071481 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f48ec053a40 0x7f48ec055f00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f48fc00b000 tx=0x7f48fc006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:20.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.236+0000 7f4909ffb700 1 -- 172.21.15.36:0/3721071481 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4900043cb0 con 0x7f490c0ad330 2022-06-17T12:52:20.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.364+0000 7f4912c36700 1 -- 172.21.15.36:0/3721071481 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f490c057410 con 0x7f490c0ad330 2022-06-17T12:52:20.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.364+0000 7f4909ffb700 1 -- 172.21.15.36:0/3721071481 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v61) v1 ==== 84+0+143 (secure 0 0 0) 0x7f4900023070 con 0x7f490c0ad330 2022-06-17T12:52:20.351 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:52:20.351 INFO:teuthology.orchestra.run.smithi036.stdout:[ 2022-06-17T12:52:20.351 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:20.352 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 1, 2022-06-17T12:52:20.352 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "device_health_metrics" 2022-06-17T12:52:20.352 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:20.353 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:20.353 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 2, 2022-06-17T12:52:20.353 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "rbd" 2022-06-17T12:52:20.353 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:20.354 INFO:teuthology.orchestra.run.smithi036.stdout:] 2022-06-17T12:52:20.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.368+0000 7f4912c36700 1 -- 172.21.15.36:0/3721071481 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f48ec053a40 msgr2=0x7f48ec055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:20.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.368+0000 7f4912c36700 1 --2- 172.21.15.36:0/3721071481 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f48ec053a40 0x7f48ec055f00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f48fc00b000 tx=0x7f48fc006df0).stop 2022-06-17T12:52:20.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.368+0000 7f4912c36700 1 -- 172.21.15.36:0/3721071481 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f490c0ad330 msgr2=0x7f490c0adab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:20.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.368+0000 7f4912c36700 1 --2- 172.21.15.36:0/3721071481 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f490c0ad330 0x7f490c0adab0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f4900002f70 tx=0x7f4900014000).stop 2022-06-17T12:52:20.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.368+0000 7f4912c36700 1 -- 172.21.15.36:0/3721071481 shutdown_connections 2022-06-17T12:52:20.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.368+0000 7f4912c36700 1 --2- 172.21.15.36:0/3721071481 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f48ec053a40 0x7f48ec055f00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:20.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.368+0000 7f4912c36700 1 --2- 172.21.15.36:0/3721071481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f490c1444f0 0x7f490c159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:20.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.368+0000 7f4912c36700 1 --2- 172.21.15.36:0/3721071481 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f490c0ad330 0x7f490c0adab0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:20.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.368+0000 7f4912c36700 1 --2- 172.21.15.36:0/3721071481 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f490c062770 0x7f490c063090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:20.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.368+0000 7f4912c36700 1 -- 172.21.15.36:0/3721071481 >> 172.21.15.36:0/3721071481 conn(0x7f490c0b53e0 msgr2=0x7f490c05fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:20.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.368+0000 7f4912c36700 1 -- 172.21.15.36:0/3721071481 shutdown_connections 2022-06-17T12:52:20.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.368+0000 7f4912c36700 1 -- 172.21.15.36:0/3721071481 wait complete. 2022-06-17T12:52:20.365 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-17T12:52:20.366 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-17T12:52:20.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.492+0000 7f6ca983d700 1 Processor -- start 2022-06-17T12:52:20.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.492+0000 7f6ca983d700 1 -- start start 2022-06-17T12:52:20.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.492+0000 7f6ca983d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f6ca40b64f0 0x7f6ca4143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:20.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.492+0000 7f6ca983d700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca40b68d0 con 0x7f6ca40b4120 2022-06-17T12:52:20.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.492+0000 7f6ca983d700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca40b6e20 con 0x7f6ca4147490 2022-06-17T12:52:20.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.492+0000 7f6ca983d700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca41468e0 con 0x7f6ca40b64f0 2022-06-17T12:52:20.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.492+0000 7f6ca2ffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f6ca40b64f0 0x7f6ca4143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:20.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.492+0000 7f6c9a7fc700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f6ca40b4120 0x7f6ca4146d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52306/0 (socket says 172.21.15.36:52306) 2022-06-17T12:52:20.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.492+0000 7f6c9a7fc700 1 -- 172.21.15.36:0/3813484113 learned_addr learned my addr 172.21.15.36:0/3813484113 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:20.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.492+0000 7f6ca2ffd700 1 -- 172.21.15.36:0/3813484113 >> v1:172.21.15.70:6789/0 conn(0x7f6ca4147490 legacy=0x7f6ca4140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:20.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.492+0000 7f6ca2ffd700 1 -- 172.21.15.36:0/3813484113 >> v1:172.21.15.36:6789/0 conn(0x7f6ca40b4120 legacy=0x7f6ca4146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:20.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.492+0000 7f6ca27fc700 1 -- 172.21.15.36:0/3813484113 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764624824 0 0) 0x7f6ca40b6e20 con 0x7f6ca4147490 2022-06-17T12:52:20.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.492+0000 7f6ca2ffd700 1 -- 172.21.15.36:0/3813484113 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca4143640 con 0x7f6ca40b64f0 2022-06-17T12:52:20.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.492+0000 7f6ca2ffd700 1 --2- 172.21.15.36:0/3813484113 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca40b64f0 0x7f6ca4143150 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f6c8c001110 tx=0x7f6c94006df0).ready entity=mon.0 client_cookie=d2250c79b41b2faa server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:20.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.492+0000 7f6ca27fc700 1 -- 172.21.15.36:0/3813484113 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c9400d5c0 con 0x7f6ca40b64f0 2022-06-17T12:52:20.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.492+0000 7f6ca27fc700 1 -- 172.21.15.36:0/3813484113 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c94000e90 con 0x7f6ca40b64f0 2022-06-17T12:52:20.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.492+0000 7f6ca27fc700 1 -- 172.21.15.36:0/3813484113 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c9400d910 con 0x7f6ca40b64f0 2022-06-17T12:52:20.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca983d700 1 -- 172.21.15.36:0/3813484113 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca40b64f0 msgr2=0x7f6ca4143150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:20.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca983d700 1 --2- 172.21.15.36:0/3813484113 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca40b64f0 0x7f6ca4143150 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f6c8c001110 tx=0x7f6c94006df0).stop 2022-06-17T12:52:20.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca983d700 1 -- 172.21.15.36:0/3813484113 shutdown_connections 2022-06-17T12:52:20.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca983d700 1 --2- 172.21.15.36:0/3813484113 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca40b64f0 0x7f6ca4143150 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:20.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca983d700 1 -- 172.21.15.36:0/3813484113 >> 172.21.15.36:0/3813484113 conn(0x7f6ca40620f0 msgr2=0x7f6ca4062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:20.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca983d700 1 -- 172.21.15.36:0/3813484113 shutdown_connections 2022-06-17T12:52:20.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca983d700 1 -- 172.21.15.36:0/3813484113 wait complete. 2022-06-17T12:52:20.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca983d700 1 Processor -- start 2022-06-17T12:52:20.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca983d700 1 -- start start 2022-06-17T12:52:20.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca983d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ca40b4120 0x7f6ca413e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:20.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca983d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ca40b64f0 0x7f6ca40b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:20.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca983d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca4147490 0x7f6ca4141800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:20.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca983d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca4140280 con 0x7f6ca4147490 2022-06-17T12:52:20.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca983d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca40b3710 con 0x7f6ca40b64f0 2022-06-17T12:52:20.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca983d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca413ffe0 con 0x7f6ca40b4120 2022-06-17T12:52:20.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca37fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca4147490 0x7f6ca4141800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:20.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca37fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca4147490 0x7f6ca4141800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34782/0 (socket says 172.21.15.36:34782) 2022-06-17T12:52:20.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca37fe700 1 -- 172.21.15.36:0/1239843151 learned_addr learned my addr 172.21.15.36:0/1239843151 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:20.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca2ffd700 1 --2- 172.21.15.36:0/1239843151 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ca40b4120 0x7f6ca413e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:20.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6c9a7fc700 1 --2- 172.21.15.36:0/1239843151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ca40b64f0 0x7f6ca40b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:20.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca37fe700 1 -- 172.21.15.36:0/1239843151 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ca40b4120 msgr2=0x7f6ca413e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:20.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca37fe700 1 --2- 172.21.15.36:0/1239843151 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ca40b4120 0x7f6ca413e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:20.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca37fe700 1 -- 172.21.15.36:0/1239843151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ca40b64f0 msgr2=0x7f6ca40b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:20.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca37fe700 1 --2- 172.21.15.36:0/1239843151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ca40b64f0 0x7f6ca40b0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:20.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca37fe700 1 -- 172.21.15.36:0/1239843151 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9400b180 con 0x7f6ca4147490 2022-06-17T12:52:20.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6c9a7fc700 1 --2- 172.21.15.36:0/1239843151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ca40b64f0 0x7f6ca40b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:52:20.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca37fe700 1 --2- 172.21.15.36:0/1239843151 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca4147490 0x7f6ca4141800 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f6c9c001250 tx=0x7f6c9c007f50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:20.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca0ff9700 1 -- 172.21.15.36:0/1239843151 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c9c006aa0 con 0x7f6ca4147490 2022-06-17T12:52:20.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca0ff9700 1 -- 172.21.15.36:0/1239843151 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9c006c00 con 0x7f6ca4147490 2022-06-17T12:52:20.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca983d700 1 -- 172.21.15.36:0/1239843151 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca40b3400 con 0x7f6ca4147490 2022-06-17T12:52:20.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca0ff9700 1 -- 172.21.15.36:0/1239843151 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c9c007030 con 0x7f6ca4147490 2022-06-17T12:52:20.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.496+0000 7f6ca983d700 1 -- 172.21.15.36:0/1239843151 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca4142d90 con 0x7f6ca4147490 2022-06-17T12:52:20.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.500+0000 7f6ca983d700 1 -- 172.21.15.36:0/1239843151 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca41412f0 con 0x7f6ca4147490 2022-06-17T12:52:20.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.500+0000 7f6ca0ff9700 1 -- 172.21.15.36:0/1239843151 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6c9c010030 con 0x7f6ca4147490 2022-06-17T12:52:20.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.500+0000 7f6ca0ff9700 1 --2- 172.21.15.36:0/1239843151 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c880539b0 0x7f6c88055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:20.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.500+0000 7f6ca0ff9700 1 -- 172.21.15.36:0/1239843151 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f6c9c06bce0 con 0x7f6ca4147490 2022-06-17T12:52:20.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.500+0000 7f6ca2ffd700 1 --2- 172.21.15.36:0/1239843151 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c880539b0 0x7f6c88055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:20.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.500+0000 7f6ca2ffd700 1 --2- 172.21.15.36:0/1239843151 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c880539b0 0x7f6c88055e70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f6c94015000 tx=0x7f6c94014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:20.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.504+0000 7f6ca0ff9700 1 -- 172.21.15.36:0/1239843151 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6c9c042d80 con 0x7f6ca4147490 2022-06-17T12:52:20.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:20.636+0000 7f6ca983d700 1 -- 172.21.15.36:0/1239843151 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f6ca40b3a60 con 0x7f6ca4147490 2022-06-17T12:52:21.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.492+0000 7f6ca0ff9700 1 -- 172.21.15.36:0/1239843151 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v62) v1 ==== 133+0+0 (secure 0 0 0) 0x7f6c9c0463a0 con 0x7f6ca4147490 2022-06-17T12:52:21.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.496+0000 7f6ca983d700 1 -- 172.21.15.36:0/1239843151 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c880539b0 msgr2=0x7f6c88055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:21.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.496+0000 7f6ca983d700 1 --2- 172.21.15.36:0/1239843151 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c880539b0 0x7f6c88055e70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f6c94015000 tx=0x7f6c94014000).stop 2022-06-17T12:52:21.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.496+0000 7f6ca983d700 1 -- 172.21.15.36:0/1239843151 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca4147490 msgr2=0x7f6ca4141800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:21.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.496+0000 7f6ca983d700 1 --2- 172.21.15.36:0/1239843151 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca4147490 0x7f6ca4141800 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f6c9c001250 tx=0x7f6c9c007f50).stop 2022-06-17T12:52:21.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.496+0000 7f6ca983d700 1 -- 172.21.15.36:0/1239843151 shutdown_connections 2022-06-17T12:52:21.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.496+0000 7f6ca983d700 1 --2- 172.21.15.36:0/1239843151 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c880539b0 0x7f6c88055e70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:21.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.496+0000 7f6ca983d700 1 --2- 172.21.15.36:0/1239843151 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca4147490 0x7f6ca4141800 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:21.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.496+0000 7f6ca983d700 1 --2- 172.21.15.36:0/1239843151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ca40b64f0 0x7f6ca40b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:21.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.496+0000 7f6ca983d700 1 --2- 172.21.15.36:0/1239843151 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ca40b4120 0x7f6ca413e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:21.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.496+0000 7f6ca983d700 1 -- 172.21.15.36:0/1239843151 >> 172.21.15.36:0/1239843151 conn(0x7f6ca40620f0 msgr2=0x7f6ca413f300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:21.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.496+0000 7f6ca983d700 1 -- 172.21.15.36:0/1239843151 shutdown_connections 2022-06-17T12:52:21.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.496+0000 7f6ca983d700 1 -- 172.21.15.36:0/1239843151 wait complete. 2022-06-17T12:52:21.485 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' created 2022-06-17T12:52:21.501 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-06-17T12:52:21.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.628+0000 7f038efcf700 1 Processor -- start 2022-06-17T12:52:21.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.628+0000 7f038efcf700 1 -- start start 2022-06-17T12:52:21.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.632+0000 7f038efcf700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f03880b27b0 0x7f038814b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:21.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.632+0000 7f038efcf700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038814b870 con 0x7f0388060ca0 2022-06-17T12:52:21.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.632+0000 7f038efcf700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03880ac500 con 0x7f0388147bd0 2022-06-17T12:52:21.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.632+0000 7f038efcf700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03880ace90 con 0x7f03880b27b0 2022-06-17T12:52:21.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.632+0000 7f038dfcd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f03880b27b0 0x7f038814b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:21.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.632+0000 7f038d7cc700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f0388060ca0 0x7f03880ac780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52320/0 (socket says 172.21.15.36:52320) 2022-06-17T12:52:21.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.632+0000 7f038d7cc700 1 -- 172.21.15.36:0/3941158149 learned_addr learned my addr 172.21.15.36:0/3941158149 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:21.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.632+0000 7f038cfcb700 1 -- 172.21.15.36:0/3941158149 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1465339776 0 0) 0x7f038814b870 con 0x7f0388060ca0 2022-06-17T12:52:21.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.632+0000 7f038cfcb700 1 -- 172.21.15.36:0/3941158149 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0370003260 con 0x7f0388060ca0 2022-06-17T12:52:21.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.632+0000 7f038dfcd700 1 -- 172.21.15.36:0/3941158149 >> v1:172.21.15.70:6789/0 conn(0x7f0388147bd0 legacy=0x7f03881442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:21.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.632+0000 7f038dfcd700 1 -- 172.21.15.36:0/3941158149 >> v1:172.21.15.36:6789/0 conn(0x7f0388060ca0 legacy=0x7f03880ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:21.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.632+0000 7f038dfcd700 1 -- 172.21.15.36:0/3941158149 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03880b2b90 con 0x7f03880b27b0 2022-06-17T12:52:21.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.632+0000 7f038dfcd700 1 --2- 172.21.15.36:0/3941158149 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f03880b27b0 0x7f038814b2b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f0378002f70 tx=0x7f0378006df0).ready entity=mon.0 client_cookie=2db93747fed48503 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:21.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.632+0000 7f038cfcb700 1 -- 172.21.15.36:0/3941158149 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f037800d610 con 0x7f03880b27b0 2022-06-17T12:52:21.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.632+0000 7f038cfcb700 1 -- 172.21.15.36:0/3941158149 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0378000e90 con 0x7f03880b27b0 2022-06-17T12:52:21.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.632+0000 7f038cfcb700 1 -- 172.21.15.36:0/3941158149 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f037800d960 con 0x7f03880b27b0 2022-06-17T12:52:21.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.632+0000 7f038efcf700 1 -- 172.21.15.36:0/3941158149 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f03880b27b0 msgr2=0x7f038814b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:21.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.632+0000 7f038efcf700 1 --2- 172.21.15.36:0/3941158149 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f03880b27b0 0x7f038814b2b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f0378002f70 tx=0x7f0378006df0).stop 2022-06-17T12:52:21.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.632+0000 7f038efcf700 1 -- 172.21.15.36:0/3941158149 shutdown_connections 2022-06-17T12:52:21.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.632+0000 7f038efcf700 1 --2- 172.21.15.36:0/3941158149 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f03880b27b0 0x7f038814b2b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:21.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.632+0000 7f038efcf700 1 -- 172.21.15.36:0/3941158149 >> 172.21.15.36:0/3941158149 conn(0x7f03880b5d10 msgr2=0x7f03880b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:21.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f038efcf700 1 -- 172.21.15.36:0/3941158149 shutdown_connections 2022-06-17T12:52:21.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f038efcf700 1 -- 172.21.15.36:0/3941158149 wait complete. 2022-06-17T12:52:21.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f038efcf700 1 Processor -- start 2022-06-17T12:52:21.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f038efcf700 1 -- start start 2022-06-17T12:52:21.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f038efcf700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0388060ca0 0x7f0388062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:21.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f038efcf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03880b27b0 0x7f038814cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:21.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f038efcf700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0388147bd0 0x7f0388064150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:21.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f038efcf700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038814a750 con 0x7f0388147bd0 2022-06-17T12:52:21.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f038efcf700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0388064ea0 con 0x7f03880b27b0 2022-06-17T12:52:21.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f038efcf700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038814d690 con 0x7f0388060ca0 2022-06-17T12:52:21.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f038e7ce700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0388147bd0 0x7f0388064150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:21.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f038e7ce700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0388147bd0 0x7f0388064150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34796/0 (socket says 172.21.15.36:34796) 2022-06-17T12:52:21.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f038e7ce700 1 -- 172.21.15.36:0/3714953031 learned_addr learned my addr 172.21.15.36:0/3714953031 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:21.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f038d7cc700 1 --2- 172.21.15.36:0/3714953031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03880b27b0 0x7f038814cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:21.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f038dfcd700 1 --2- 172.21.15.36:0/3714953031 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0388060ca0 0x7f0388062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:21.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f038e7ce700 1 -- 172.21.15.36:0/3714953031 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0388060ca0 msgr2=0x7f0388062b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:21.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f038e7ce700 1 --2- 172.21.15.36:0/3714953031 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0388060ca0 0x7f0388062b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:21.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f038e7ce700 1 -- 172.21.15.36:0/3714953031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03880b27b0 msgr2=0x7f038814cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:21.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f038e7ce700 1 --2- 172.21.15.36:0/3714953031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03880b27b0 0x7f038814cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:21.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f038e7ce700 1 -- 172.21.15.36:0/3714953031 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f037800b180 con 0x7f0388147bd0 2022-06-17T12:52:21.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f038e7ce700 1 --2- 172.21.15.36:0/3714953031 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0388147bd0 0x7f0388064150 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f03880620b0 tx=0x7f03800013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:21.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f0376ffd700 1 -- 172.21.15.36:0/3714953031 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0380006aa0 con 0x7f0388147bd0 2022-06-17T12:52:21.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f0376ffd700 1 -- 172.21.15.36:0/3714953031 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0380006c00 con 0x7f0388147bd0 2022-06-17T12:52:21.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f038efcf700 1 -- 172.21.15.36:0/3714953031 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038814d270 con 0x7f0388147bd0 2022-06-17T12:52:21.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f0376ffd700 1 -- 172.21.15.36:0/3714953031 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0380007030 con 0x7f0388147bd0 2022-06-17T12:52:21.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.636+0000 7f038efcf700 1 -- 172.21.15.36:0/3714953031 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0388062930 con 0x7f0388147bd0 2022-06-17T12:52:21.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.640+0000 7f038efcf700 1 -- 172.21.15.36:0/3714953031 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f038814c3a0 con 0x7f0388147bd0 2022-06-17T12:52:21.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.640+0000 7f0376ffd700 1 -- 172.21.15.36:0/3714953031 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f0380010030 con 0x7f0388147bd0 2022-06-17T12:52:21.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.640+0000 7f0376ffd700 1 --2- 172.21.15.36:0/3714953031 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f03680539b0 0x7f0368055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:21.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.640+0000 7f0376ffd700 1 -- 172.21.15.36:0/3714953031 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f038006bd20 con 0x7f0388147bd0 2022-06-17T12:52:21.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.640+0000 7f038dfcd700 1 --2- 172.21.15.36:0/3714953031 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f03680539b0 0x7f0368055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:21.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.640+0000 7f038dfcd700 1 --2- 172.21.15.36:0/3714953031 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f03680539b0 0x7f0368055e70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f0378018000 tx=0x7f0378014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:21.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.648+0000 7f0376ffd700 1 -- 172.21.15.36:0/3714953031 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0380042c90 con 0x7f0388147bd0 2022-06-17T12:52:21.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.776+0000 7f038efcf700 1 -- 172.21.15.36:0/3714953031 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f0388062450 con 0x7f0388147bd0 2022-06-17T12:52:21.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.920+0000 7f0376ffd700 1 -- 172.21.15.36:0/3714953031 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v63) v1 ==== 125+0+0 (secure 0 0 0) 0x7f0388062450 con 0x7f0388147bd0 2022-06-17T12:52:21.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.924+0000 7f038efcf700 1 -- 172.21.15.36:0/3714953031 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f03680539b0 msgr2=0x7f0368055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:21.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.924+0000 7f038efcf700 1 --2- 172.21.15.36:0/3714953031 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f03680539b0 0x7f0368055e70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f0378018000 tx=0x7f0378014000).stop 2022-06-17T12:52:21.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.924+0000 7f038efcf700 1 -- 172.21.15.36:0/3714953031 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0388147bd0 msgr2=0x7f0388064150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:21.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.924+0000 7f038efcf700 1 --2- 172.21.15.36:0/3714953031 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0388147bd0 0x7f0388064150 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f03880620b0 tx=0x7f03800013a0).stop 2022-06-17T12:52:21.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.928+0000 7f038efcf700 1 -- 172.21.15.36:0/3714953031 shutdown_connections 2022-06-17T12:52:21.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.928+0000 7f038efcf700 1 --2- 172.21.15.36:0/3714953031 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f03680539b0 0x7f0368055e70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:21.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.928+0000 7f038efcf700 1 --2- 172.21.15.36:0/3714953031 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0388147bd0 0x7f0388064150 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:21.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.928+0000 7f038efcf700 1 --2- 172.21.15.36:0/3714953031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03880b27b0 0x7f038814cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:21.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.928+0000 7f038efcf700 1 --2- 172.21.15.36:0/3714953031 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0388060ca0 0x7f0388062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:21.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.928+0000 7f038efcf700 1 -- 172.21.15.36:0/3714953031 >> 172.21.15.36:0/3714953031 conn(0x7f03880b5d10 msgr2=0x7f03880b0450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:21.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.928+0000 7f038efcf700 1 -- 172.21.15.36:0/3714953031 shutdown_connections 2022-06-17T12:52:21.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:21.928+0000 7f038efcf700 1 -- 172.21.15.36:0/3714953031 wait complete. 2022-06-17T12:52:21.920 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' created 2022-06-17T12:52:21.925 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-17T12:52:22.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.060+0000 7f6f28cc3700 1 Processor -- start 2022-06-17T12:52:22.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.060+0000 7f6f28cc3700 1 -- start start 2022-06-17T12:52:22.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.064+0000 7f6f28cc3700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f6f24060780 0x7f6f24060b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.064+0000 7f6f28cc3700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6f240afec0 0x7f6f240b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.064+0000 7f6f28cc3700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f24143990 con 0x7f6f241434f0 2022-06-17T12:52:22.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.064+0000 7f6f28cc3700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f240b1a80 con 0x7f6f24060780 2022-06-17T12:52:22.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.064+0000 7f6f28cc3700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f240b1d20 con 0x7f6f240afec0 2022-06-17T12:52:22.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.064+0000 7f6f2259c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f6f24060780 0x7f6f24060b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.064+0000 7f6f1bfff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f6f241434f0 0x7f6f240b07c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52334/0 (socket says 172.21.15.36:52334) 2022-06-17T12:52:22.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.064+0000 7f6f1bfff700 1 -- 172.21.15.36:0/386327758 learned_addr learned my addr 172.21.15.36:0/386327758 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:22.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.064+0000 7f6f22d9d700 1 --2- 172.21.15.36:0/386327758 >> v2:172.21.15.70:3300/0 conn(0x7f6f240afec0 0x7f6f240b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.064+0000 7f6f21d9b700 1 -- 172.21.15.36:0/386327758 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2005749825 0 0) 0x7f6f24143990 con 0x7f6f241434f0 2022-06-17T12:52:22.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.064+0000 7f6f21d9b700 1 -- 172.21.15.36:0/386327758 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f04003260 con 0x7f6f241434f0 2022-06-17T12:52:22.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.064+0000 7f6f2259c700 1 -- 172.21.15.36:0/386327758 >> v2:172.21.15.70:3300/0 conn(0x7f6f240afec0 msgr2=0x7f6f240b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:22.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.064+0000 7f6f2259c700 1 --2- 172.21.15.36:0/386327758 >> v2:172.21.15.70:3300/0 conn(0x7f6f240afec0 0x7f6f240b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.064+0000 7f6f2259c700 1 -- 172.21.15.36:0/386327758 >> v1:172.21.15.36:6789/0 conn(0x7f6f241434f0 legacy=0x7f6f240b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:22.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.064+0000 7f6f2259c700 1 -- 172.21.15.36:0/386327758 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f240b2640 con 0x7f6f24060780 2022-06-17T12:52:22.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.064+0000 7f6f2259c700 1 --2- 172.21.15.36:0/386327758 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6f24060780 0x7f6f24060b60 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f6f14002f70 tx=0x7f6f14006df0).ready entity=mon.0 client_cookie=7101887f734c6f84 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:22.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.064+0000 7f6f21d9b700 1 -- 172.21.15.36:0/386327758 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f1400d4f0 con 0x7f6f24060780 2022-06-17T12:52:22.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.064+0000 7f6f21d9b700 1 -- 172.21.15.36:0/386327758 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f14000e90 con 0x7f6f24060780 2022-06-17T12:52:22.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.064+0000 7f6f21d9b700 1 -- 172.21.15.36:0/386327758 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f1400d840 con 0x7f6f24060780 2022-06-17T12:52:22.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.068+0000 7f6f28cc3700 1 -- 172.21.15.36:0/386327758 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6f24060780 msgr2=0x7f6f24060b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:22.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.068+0000 7f6f28cc3700 1 --2- 172.21.15.36:0/386327758 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6f24060780 0x7f6f24060b60 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f6f14002f70 tx=0x7f6f14006df0).stop 2022-06-17T12:52:22.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.068+0000 7f6f28cc3700 1 -- 172.21.15.36:0/386327758 shutdown_connections 2022-06-17T12:52:22.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.068+0000 7f6f28cc3700 1 --2- 172.21.15.36:0/386327758 >> v2:172.21.15.70:3300/0 conn(0x7f6f240afec0 0x7f6f240b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.068+0000 7f6f28cc3700 1 --2- 172.21.15.36:0/386327758 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6f24060780 0x7f6f24060b60 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.068+0000 7f6f28cc3700 1 -- 172.21.15.36:0/386327758 >> 172.21.15.36:0/386327758 conn(0x7f6f240b6870 msgr2=0x7f6f240b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:22.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.068+0000 7f6f28cc3700 1 -- 172.21.15.36:0/386327758 shutdown_connections 2022-06-17T12:52:22.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.068+0000 7f6f28cc3700 1 -- 172.21.15.36:0/386327758 wait complete. 2022-06-17T12:52:22.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.068+0000 7f6f28cc3700 1 Processor -- start 2022-06-17T12:52:22.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.068+0000 7f6f28cc3700 1 -- start start 2022-06-17T12:52:22.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.068+0000 7f6f28cc3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f24060780 0x7f6f2413f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.068+0000 7f6f28cc3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6f240afec0 0x7f6f240ae120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.068+0000 7f6f28cc3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f241434f0 0x7f6f2413b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.068+0000 7f6f28cc3700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2413fe00 con 0x7f6f240afec0 2022-06-17T12:52:22.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.068+0000 7f6f28cc3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f240ae610 con 0x7f6f24060780 2022-06-17T12:52:22.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.068+0000 7f6f28cc3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f240661f0 con 0x7f6f241434f0 2022-06-17T12:52:22.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.068+0000 7f6f1bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6f240afec0 0x7f6f240ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.068+0000 7f6f1bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6f240afec0 0x7f6f240ae120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34810/0 (socket says 172.21.15.36:34810) 2022-06-17T12:52:22.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.068+0000 7f6f1bfff700 1 -- 172.21.15.36:0/2745446644 learned_addr learned my addr 172.21.15.36:0/2745446644 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:22.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.072+0000 7f6f2259c700 1 --2- 172.21.15.36:0/2745446644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f24060780 0x7f6f2413f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.072+0000 7f6f22d9d700 1 --2- 172.21.15.36:0/2745446644 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f241434f0 0x7f6f2413b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.072+0000 7f6f1bfff700 1 -- 172.21.15.36:0/2745446644 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f241434f0 msgr2=0x7f6f2413b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:22.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.072+0000 7f6f1bfff700 1 --2- 172.21.15.36:0/2745446644 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f241434f0 0x7f6f2413b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.072+0000 7f6f1bfff700 1 -- 172.21.15.36:0/2745446644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f24060780 msgr2=0x7f6f2413f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:22.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.072+0000 7f6f1bfff700 1 --2- 172.21.15.36:0/2745446644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f24060780 0x7f6f2413f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.072+0000 7f6f1bfff700 1 -- 172.21.15.36:0/2745446644 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1400b180 con 0x7f6f240afec0 2022-06-17T12:52:22.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.072+0000 7f6f1bfff700 1 --2- 172.21.15.36:0/2745446644 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6f240afec0 0x7f6f240ae120 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f6f0c001110 tx=0x7f6f0c007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:22.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.072+0000 7f6f1b7fe700 1 -- 172.21.15.36:0/2745446644 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f0c0070d0 con 0x7f6f240afec0 2022-06-17T12:52:22.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.072+0000 7f6f1b7fe700 1 -- 172.21.15.36:0/2745446644 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0c007230 con 0x7f6f240afec0 2022-06-17T12:52:22.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.072+0000 7f6f28cc3700 1 -- 172.21.15.36:0/2745446644 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f2413b7b0 con 0x7f6f240afec0 2022-06-17T12:52:22.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.072+0000 7f6f1b7fe700 1 -- 172.21.15.36:0/2745446644 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f0c005610 con 0x7f6f240afec0 2022-06-17T12:52:22.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.072+0000 7f6f28cc3700 1 -- 172.21.15.36:0/2745446644 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f241361e0 con 0x7f6f240afec0 2022-06-17T12:52:22.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.072+0000 7f6f28cc3700 1 -- 172.21.15.36:0/2745446644 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f2413fc50 con 0x7f6f240afec0 2022-06-17T12:52:22.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.072+0000 7f6f1b7fe700 1 -- 172.21.15.36:0/2745446644 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6f0c010070 con 0x7f6f240afec0 2022-06-17T12:52:22.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.076+0000 7f6f1b7fe700 1 --2- 172.21.15.36:0/2745446644 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6f08053a40 0x7f6f08055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.076+0000 7f6f1b7fe700 1 -- 172.21.15.36:0/2745446644 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f6f0c06b4f0 con 0x7f6f240afec0 2022-06-17T12:52:22.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.076+0000 7f6f2259c700 1 --2- 172.21.15.36:0/2745446644 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6f08053a40 0x7f6f08055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.076+0000 7f6f2259c700 1 --2- 172.21.15.36:0/2745446644 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6f08053a40 0x7f6f08055f00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f6f1401a000 tx=0x7f6f14014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:22.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.080+0000 7f6f1b7fe700 1 -- 172.21.15.36:0/2745446644 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6f0c042370 con 0x7f6f240afec0 2022-06-17T12:52:22.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.248+0000 7f6f28cc3700 1 -- 172.21.15.36:0/2745446644 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f6f24057410 con 0x7f6f240afec0 2022-06-17T12:52:22.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.256+0000 7f6f1b7fe700 1 -- 172.21.15.36:0/2745446644 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 11 and data pool 12 v29) v1 ==== 189+0+0 (secure 0 0 0) 0x7f6f0c045990 con 0x7f6f240afec0 2022-06-17T12:52:22.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.264+0000 7f6f28cc3700 1 -- 172.21.15.36:0/2745446644 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6f08053a40 msgr2=0x7f6f08055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:22.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.264+0000 7f6f28cc3700 1 --2- 172.21.15.36:0/2745446644 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6f08053a40 0x7f6f08055f00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f6f1401a000 tx=0x7f6f14014000).stop 2022-06-17T12:52:22.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.264+0000 7f6f28cc3700 1 -- 172.21.15.36:0/2745446644 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6f240afec0 msgr2=0x7f6f240ae120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:22.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.264+0000 7f6f28cc3700 1 --2- 172.21.15.36:0/2745446644 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6f240afec0 0x7f6f240ae120 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f6f0c001110 tx=0x7f6f0c007ad0).stop 2022-06-17T12:52:22.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.264+0000 7f6f28cc3700 1 -- 172.21.15.36:0/2745446644 shutdown_connections 2022-06-17T12:52:22.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.264+0000 7f6f28cc3700 1 --2- 172.21.15.36:0/2745446644 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6f08053a40 0x7f6f08055f00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.264+0000 7f6f28cc3700 1 --2- 172.21.15.36:0/2745446644 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f241434f0 0x7f6f2413b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.264+0000 7f6f28cc3700 1 --2- 172.21.15.36:0/2745446644 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6f240afec0 0x7f6f240ae120 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.264+0000 7f6f28cc3700 1 --2- 172.21.15.36:0/2745446644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f24060780 0x7f6f2413f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.264+0000 7f6f28cc3700 1 -- 172.21.15.36:0/2745446644 >> 172.21.15.36:0/2745446644 conn(0x7f6f240b6870 msgr2=0x7f6f240ad2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:22.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.264+0000 7f6f28cc3700 1 -- 172.21.15.36:0/2745446644 shutdown_connections 2022-06-17T12:52:22.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.264+0000 7f6f28cc3700 1 -- 172.21.15.36:0/2745446644 wait complete. 2022-06-17T12:52:22.257 INFO:teuthology.orchestra.run.smithi036.stderr:new fs with metadata pool 11 and data pool 12 2022-06-17T12:52:22.265 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:52:22.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.368+0000 7f37a48e5700 1 Processor -- start 2022-06-17T12:52:22.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.368+0000 7f37a48e5700 1 -- start start 2022-06-17T12:52:22.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.368+0000 7f37a48e5700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37a0062770 0x7f37a0062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.368+0000 7f37a48e5700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a00b0ba0 con 0x7f37a00ad330 2022-06-17T12:52:22.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.368+0000 7f37a48e5700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a0147e90 con 0x7f37a01444f0 2022-06-17T12:52:22.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.368+0000 7f37a48e5700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a00ae820 con 0x7f37a0062770 2022-06-17T12:52:22.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.368+0000 7f379ed9d700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f37a00ad330 0x7f37a00ad780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52360/0 (socket says 172.21.15.36:52360) 2022-06-17T12:52:22.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.368+0000 7f379ed9d700 1 -- 172.21.15.36:0/2482367547 learned_addr learned my addr 172.21.15.36:0/2482367547 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:22.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.368+0000 7f379e59c700 1 --2- 172.21.15.36:0/2482367547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37a0062770 0x7f37a0062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.368+0000 7f379d59a700 1 -- 172.21.15.36:0/2482367547 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1642750904 0 0) 0x7f37a00b0ba0 con 0x7f37a00ad330 2022-06-17T12:52:22.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.368+0000 7f379d59a700 1 -- 172.21.15.36:0/2482367547 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3780003260 con 0x7f37a00ad330 2022-06-17T12:52:22.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.368+0000 7f379e59c700 1 -- 172.21.15.36:0/2482367547 >> v1:172.21.15.70:6789/0 conn(0x7f37a01444f0 legacy=0x7f37a0144930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:22.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.368+0000 7f379e59c700 1 -- 172.21.15.36:0/2482367547 >> v1:172.21.15.36:6789/0 conn(0x7f37a00ad330 legacy=0x7f37a00ad780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:22.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.368+0000 7f379e59c700 1 -- 172.21.15.36:0/2482367547 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a00b0dd0 con 0x7f37a0062770 2022-06-17T12:52:22.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.372+0000 7f379e59c700 1 --2- 172.21.15.36:0/2482367547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37a0062770 0x7f37a0062b50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f37980032f0 tx=0x7f37980091d0).ready entity=mon.2 client_cookie=95ccff3c33e02231 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:22.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.372+0000 7f379d59a700 1 -- 172.21.15.36:0/2482367547 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f379800b6c0 con 0x7f37a0062770 2022-06-17T12:52:22.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.372+0000 7f379d59a700 1 -- 172.21.15.36:0/2482367547 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379800b820 con 0x7f37a0062770 2022-06-17T12:52:22.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.372+0000 7f379d59a700 1 -- 172.21.15.36:0/2482367547 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f379800b990 con 0x7f37a0062770 2022-06-17T12:52:22.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.372+0000 7f37a48e5700 1 -- 172.21.15.36:0/2482367547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37a0062770 msgr2=0x7f37a0062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:22.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.372+0000 7f37a48e5700 1 --2- 172.21.15.36:0/2482367547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37a0062770 0x7f37a0062b50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f37980032f0 tx=0x7f37980091d0).stop 2022-06-17T12:52:22.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.372+0000 7f37a48e5700 1 -- 172.21.15.36:0/2482367547 shutdown_connections 2022-06-17T12:52:22.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.372+0000 7f37a48e5700 1 --2- 172.21.15.36:0/2482367547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37a0062770 0x7f37a0062b50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.372+0000 7f37a48e5700 1 -- 172.21.15.36:0/2482367547 >> 172.21.15.36:0/2482367547 conn(0x7f37a00b53e0 msgr2=0x7f37a00b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:22.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.372+0000 7f37a48e5700 1 -- 172.21.15.36:0/2482367547 shutdown_connections 2022-06-17T12:52:22.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.372+0000 7f37a48e5700 1 -- 172.21.15.36:0/2482367547 wait complete. 2022-06-17T12:52:22.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.372+0000 7f37a48e5700 1 Processor -- start 2022-06-17T12:52:22.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.372+0000 7f37a48e5700 1 -- start start 2022-06-17T12:52:22.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.376+0000 7f37a48e5700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37a0062770 0x7f37a00adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.376+0000 7f37a48e5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37a00ad330 0x7f37a0156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.376+0000 7f37a48e5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37a01444f0 0x7f37a015a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.376+0000 7f37a48e5700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a0065f30 con 0x7f37a00ad330 2022-06-17T12:52:22.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.376+0000 7f37a48e5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a0156e50 con 0x7f37a01444f0 2022-06-17T12:52:22.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.376+0000 7f37a48e5700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a0063210 con 0x7f37a0062770 2022-06-17T12:52:22.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.376+0000 7f379dd9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37a00ad330 0x7f37a0156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.376+0000 7f379dd9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37a00ad330 0x7f37a0156960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34836/0 (socket says 172.21.15.36:34836) 2022-06-17T12:52:22.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.376+0000 7f379dd9b700 1 -- 172.21.15.36:0/1816278848 learned_addr learned my addr 172.21.15.36:0/1816278848 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:22.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.376+0000 7f379ed9d700 1 --2- 172.21.15.36:0/1816278848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37a01444f0 0x7f37a015a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.376+0000 7f379e59c700 1 --2- 172.21.15.36:0/1816278848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37a0062770 0x7f37a00adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.376+0000 7f379dd9b700 1 -- 172.21.15.36:0/1816278848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37a0062770 msgr2=0x7f37a00adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:22.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.376+0000 7f379dd9b700 1 --2- 172.21.15.36:0/1816278848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37a0062770 0x7f37a00adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.376+0000 7f379dd9b700 1 -- 172.21.15.36:0/1816278848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37a01444f0 msgr2=0x7f37a015a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:22.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.376+0000 7f379dd9b700 1 --2- 172.21.15.36:0/1816278848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37a01444f0 0x7f37a015a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.376+0000 7f379dd9b700 1 -- 172.21.15.36:0/1816278848 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3798009470 con 0x7f37a00ad330 2022-06-17T12:52:22.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.376+0000 7f379dd9b700 1 --2- 172.21.15.36:0/1816278848 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37a00ad330 0x7f37a0156960 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f37a005d2b0 tx=0x7f3788007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:22.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.376+0000 7f37977fe700 1 -- 172.21.15.36:0/1816278848 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f378800ce80 con 0x7f37a00ad330 2022-06-17T12:52:22.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.376+0000 7f37977fe700 1 -- 172.21.15.36:0/1816278848 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3788012400 con 0x7f37a00ad330 2022-06-17T12:52:22.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.376+0000 7f37a48e5700 1 -- 172.21.15.36:0/1816278848 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a015a0b0 con 0x7f37a00ad330 2022-06-17T12:52:22.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.376+0000 7f37977fe700 1 -- 172.21.15.36:0/1816278848 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3788012760 con 0x7f37a00ad330 2022-06-17T12:52:22.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.376+0000 7f37a48e5700 1 -- 172.21.15.36:0/1816278848 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a0067420 con 0x7f37a00ad330 2022-06-17T12:52:22.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.376+0000 7f37a48e5700 1 -- 172.21.15.36:0/1816278848 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a00b1c20 con 0x7f37a00ad330 2022-06-17T12:52:22.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.380+0000 7f37977fe700 1 -- 172.21.15.36:0/1816278848 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f3788019030 con 0x7f37a00ad330 2022-06-17T12:52:22.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.380+0000 7f37977fe700 1 --2- 172.21.15.36:0/1816278848 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3784053a60 0x7f3784055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.380+0000 7f37977fe700 1 -- 172.21.15.36:0/1816278848 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f378806b2f0 con 0x7f37a00ad330 2022-06-17T12:52:22.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.380+0000 7f379e59c700 1 --2- 172.21.15.36:0/1816278848 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3784053a60 0x7f3784055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.380+0000 7f379e59c700 1 --2- 172.21.15.36:0/1816278848 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3784053a60 0x7f3784055f20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f379801b000 tx=0x7f3798016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:22.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.384+0000 7f37977fe700 1 -- 172.21.15.36:0/1816278848 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3788041fe0 con 0x7f37a00ad330 2022-06-17T12:52:22.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.512+0000 7f37a48e5700 1 -- 172.21.15.36:0/1816278848 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37a0057410 con 0x7f37a00ad330 2022-06-17T12:52:22.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.516+0000 7f37977fe700 1 -- 172.21.15.36:0/1816278848 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16300 (secure 0 0 0) 0x7f3788045600 con 0x7f37a00ad330 2022-06-17T12:52:22.503 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:52:22.504 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":64,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:52:22.252837+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:52:20.693971+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2022-06-17T12:52:21.830992+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:52:22.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.516+0000 7f37a48e5700 1 -- 172.21.15.36:0/1816278848 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3784053a60 msgr2=0x7f3784055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:22.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.516+0000 7f37a48e5700 1 --2- 172.21.15.36:0/1816278848 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3784053a60 0x7f3784055f20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f379801b000 tx=0x7f3798016000).stop 2022-06-17T12:52:22.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.516+0000 7f37a48e5700 1 -- 172.21.15.36:0/1816278848 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37a00ad330 msgr2=0x7f37a0156960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:22.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.516+0000 7f37a48e5700 1 --2- 172.21.15.36:0/1816278848 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37a00ad330 0x7f37a0156960 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f37a005d2b0 tx=0x7f3788007c50).stop 2022-06-17T12:52:22.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.516+0000 7f37a48e5700 1 -- 172.21.15.36:0/1816278848 shutdown_connections 2022-06-17T12:52:22.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.516+0000 7f37a48e5700 1 --2- 172.21.15.36:0/1816278848 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3784053a60 0x7f3784055f20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.516+0000 7f37a48e5700 1 --2- 172.21.15.36:0/1816278848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37a01444f0 0x7f37a015a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.516+0000 7f37a48e5700 1 --2- 172.21.15.36:0/1816278848 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37a00ad330 0x7f37a0156960 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.516+0000 7f37a48e5700 1 --2- 172.21.15.36:0/1816278848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37a0062770 0x7f37a00adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.516+0000 7f37a48e5700 1 -- 172.21.15.36:0/1816278848 >> 172.21.15.36:0/1816278848 conn(0x7f37a00b53e0 msgr2=0x7f37a00aff70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:22.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.516+0000 7f37a48e5700 1 -- 172.21.15.36:0/1816278848 shutdown_connections 2022-06-17T12:52:22.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.516+0000 7f37a48e5700 1 -- 172.21.15.36:0/1816278848 wait complete. 2022-06-17T12:52:22.515 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:52:22.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.632+0000 7f498359e700 1 Processor -- start 2022-06-17T12:52:22.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.632+0000 7f498359e700 1 -- start start 2022-06-17T12:52:22.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.632+0000 7f498359e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4984063730 0x7f4984065e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498359e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4984063b10 con 0x7f49840b1930 2022-06-17T12:52:22.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498359e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4984065430 con 0x7f49840b2900 2022-06-17T12:52:22.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498359e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49840656d0 con 0x7f4984063730 2022-06-17T12:52:22.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f4981d9b700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f49840b1930 0x7f49841431f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52372/0 (socket says 172.21.15.36:52372) 2022-06-17T12:52:22.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f4981d9b700 1 -- 172.21.15.36:0/436626943 learned_addr learned my addr 172.21.15.36:0/436626943 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:22.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498259c700 1 --2- 172.21.15.36:0/436626943 >> v2:172.21.15.70:3300/0 conn(0x7f4984063730 0x7f4984065e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498159a700 1 -- 172.21.15.36:0/436626943 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4027504146 0 0) 0x7f4984063b10 con 0x7f49840b1930 2022-06-17T12:52:22.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498159a700 1 -- 172.21.15.36:0/436626943 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4968003260 con 0x7f49840b1930 2022-06-17T12:52:22.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498159a700 1 -- 172.21.15.36:0/436626943 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 640580684 0 0) 0x7f4984065430 con 0x7f49840b2900 2022-06-17T12:52:22.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498159a700 1 -- 172.21.15.36:0/436626943 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4984063b10 con 0x7f49840b2900 2022-06-17T12:52:22.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498159a700 1 -- 172.21.15.36:0/436626943 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 613193044 0 0) 0x7f4968003260 con 0x7f49840b1930 2022-06-17T12:52:22.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498159a700 1 -- 172.21.15.36:0/436626943 >> v2:172.21.15.70:3300/0 conn(0x7f4984063730 msgr2=0x7f4984065e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:22.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498159a700 1 --2- 172.21.15.36:0/436626943 >> v2:172.21.15.70:3300/0 conn(0x7f4984063730 0x7f4984065e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498159a700 1 -- 172.21.15.36:0/436626943 >> v1:172.21.15.70:6789/0 conn(0x7f49840b2900 legacy=0x7f498413eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:22.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498159a700 1 -- 172.21.15.36:0/436626943 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4984065970 con 0x7f49840b1930 2022-06-17T12:52:22.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498159a700 1 -- 172.21.15.36:0/436626943 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f49780019e0 con 0x7f49840b1930 2022-06-17T12:52:22.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498159a700 1 -- 172.21.15.36:0/436626943 >> v1:172.21.15.36:6789/0 conn(0x7f49840b1930 legacy=0x7f49841431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:22.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498159a700 1 --2- 172.21.15.36:0/436626943 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f49680038e0 0x7f4968003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498159a700 1 --2- 172.21.15.36:0/436626943 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4968004500 0x7f4968004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498159a700 1 --2- 172.21.15.36:0/436626943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4968004e70 0x7f496800fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498159a700 1 -- 172.21.15.36:0/436626943 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49840656d0 con 0x7f49680038e0 2022-06-17T12:52:22.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498159a700 1 -- 172.21.15.36:0/436626943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4968010080 con 0x7f4968004e70 2022-06-17T12:52:22.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498159a700 1 -- 172.21.15.36:0/436626943 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4968010240 con 0x7f4968004500 2022-06-17T12:52:22.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498259c700 1 --2- 172.21.15.36:0/436626943 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f49680038e0 0x7f4968003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f4982d9d700 1 --2- 172.21.15.36:0/436626943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4968004e70 0x7f496800fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f4981d9b700 1 --2- 172.21.15.36:0/436626943 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4968004500 0x7f4968004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498259c700 1 -- 172.21.15.36:0/436626943 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4968004500 msgr2=0x7f4968004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:22.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498259c700 1 --2- 172.21.15.36:0/436626943 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4968004500 0x7f4968004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498259c700 1 -- 172.21.15.36:0/436626943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4968004e70 msgr2=0x7f496800fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:22.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498259c700 1 --2- 172.21.15.36:0/436626943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4968004e70 0x7f496800fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f4982d9d700 1 -- 172.21.15.36:0/436626943 reap_dead start 2022-06-17T12:52:22.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498259c700 1 -- 172.21.15.36:0/436626943 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4968003260 con 0x7f49680038e0 2022-06-17T12:52:22.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498259c700 1 --2- 172.21.15.36:0/436626943 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f49680038e0 0x7f4968003cc0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f497400c000 tx=0x7f4974006df0).ready entity=mon.0 client_cookie=6dd6dcbcb312804 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:22.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498159a700 1 -- 172.21.15.36:0/436626943 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f497400e830 con 0x7f49680038e0 2022-06-17T12:52:22.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498159a700 1 -- 172.21.15.36:0/436626943 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497400e990 con 0x7f49680038e0 2022-06-17T12:52:22.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498159a700 1 -- 172.21.15.36:0/436626943 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f497400ecf0 con 0x7f49680038e0 2022-06-17T12:52:22.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498359e700 1 -- 172.21.15.36:0/436626943 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f49680038e0 msgr2=0x7f4968003cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:22.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498359e700 1 --2- 172.21.15.36:0/436626943 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f49680038e0 0x7f4968003cc0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f497400c000 tx=0x7f4974006df0).stop 2022-06-17T12:52:22.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498359e700 1 -- 172.21.15.36:0/436626943 shutdown_connections 2022-06-17T12:52:22.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498359e700 1 --2- 172.21.15.36:0/436626943 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f49680038e0 0x7f4968003cc0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498359e700 1 -- 172.21.15.36:0/436626943 >> 172.21.15.36:0/436626943 conn(0x7f49840614b0 msgr2=0x7f49840618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:22.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498359e700 1 -- 172.21.15.36:0/436626943 shutdown_connections 2022-06-17T12:52:22.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.636+0000 7f498359e700 1 -- 172.21.15.36:0/436626943 wait complete. 2022-06-17T12:52:22.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.640+0000 7f498359e700 1 Processor -- start 2022-06-17T12:52:22.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.640+0000 7f498359e700 1 -- start start 2022-06-17T12:52:22.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.640+0000 7f498359e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f49680038e0 0x7f498405c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.640+0000 7f498359e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f498413db30 0x7f498413df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.640+0000 7f498359e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f498413e480 0x7f49841109f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.640+0000 7f498359e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49840b02e0 con 0x7f498413e480 2022-06-17T12:52:22.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.640+0000 7f498359e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49840aeb40 con 0x7f498413db30 2022-06-17T12:52:22.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.640+0000 7f498359e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49840b05d0 con 0x7f49680038e0 2022-06-17T12:52:22.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.640+0000 7f4982d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f498413e480 0x7f49841109f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.640+0000 7f4982d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f498413e480 0x7f49841109f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34858/0 (socket says 172.21.15.36:34858) 2022-06-17T12:52:22.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.640+0000 7f4982d9d700 1 -- 172.21.15.36:0/2553233688 learned_addr learned my addr 172.21.15.36:0/2553233688 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:22.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.640+0000 7f498259c700 1 --2- 172.21.15.36:0/2553233688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f49680038e0 0x7f498405c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.640+0000 7f4981d9b700 1 --2- 172.21.15.36:0/2553233688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f498413db30 0x7f498413df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.640+0000 7f4982d9d700 1 -- 172.21.15.36:0/2553233688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f49680038e0 msgr2=0x7f498405c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:22.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.640+0000 7f4982d9d700 1 --2- 172.21.15.36:0/2553233688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f49680038e0 0x7f498405c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.640+0000 7f4982d9d700 1 -- 172.21.15.36:0/2553233688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f498413db30 msgr2=0x7f498413df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:22.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.640+0000 7f4982d9d700 1 --2- 172.21.15.36:0/2553233688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f498413db30 0x7f498413df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.640+0000 7f4982d9d700 1 -- 172.21.15.36:0/2553233688 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49840b0870 con 0x7f498413e480 2022-06-17T12:52:22.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.640+0000 7f4982d9d700 1 --2- 172.21.15.36:0/2553233688 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f498413e480 0x7f49841109f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f497c0011d0 tx=0x7f497c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:22.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.640+0000 7f49737fe700 1 -- 172.21.15.36:0/2553233688 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f497c007430 con 0x7f498413e480 2022-06-17T12:52:22.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.640+0000 7f49737fe700 1 -- 172.21.15.36:0/2553233688 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497c00e9d0 con 0x7f498413e480 2022-06-17T12:52:22.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.640+0000 7f498359e700 1 -- 172.21.15.36:0/2553233688 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49840ae7c0 con 0x7f498413e480 2022-06-17T12:52:22.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.640+0000 7f49737fe700 1 -- 172.21.15.36:0/2553233688 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f497c00ed30 con 0x7f498413e480 2022-06-17T12:52:22.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.640+0000 7f498359e700 1 -- 172.21.15.36:0/2553233688 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4984151420 con 0x7f498413e480 2022-06-17T12:52:22.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.644+0000 7f498359e700 1 -- 172.21.15.36:0/2553233688 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49840b3b30 con 0x7f498413e480 2022-06-17T12:52:22.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.644+0000 7f49737fe700 1 -- 172.21.15.36:0/2553233688 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f497c01d060 con 0x7f498413e480 2022-06-17T12:52:22.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.644+0000 7f49737fe700 1 --2- 172.21.15.36:0/2553233688 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f49600539d0 0x7f4960055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.644+0000 7f498259c700 1 --2- 172.21.15.36:0/2553233688 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f49600539d0 0x7f4960055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.644+0000 7f49737fe700 1 -- 172.21.15.36:0/2553233688 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f497c06bb50 con 0x7f498413e480 2022-06-17T12:52:22.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.644+0000 7f498259c700 1 --2- 172.21.15.36:0/2553233688 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f49600539d0 0x7f4960055e90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f497401d000 tx=0x7f4974019000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:22.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.652+0000 7f49737fe700 1 -- 172.21.15.36:0/2553233688 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f497c042880 con 0x7f498413e480 2022-06-17T12:52:22.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.780+0000 7f498359e700 1 -- 172.21.15.36:0/2553233688 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49840ae9b0 con 0x7f498413e480 2022-06-17T12:52:22.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.784+0000 7f49737fe700 1 -- 172.21.15.36:0/2553233688 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16300 (secure 0 0 0) 0x7f497c01b070 con 0x7f498413e480 2022-06-17T12:52:22.769 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:52:22.769 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":64,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:52:22.252837+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:52:20.693971+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2022-06-17T12:52:21.830992+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:52:22.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.784+0000 7f498359e700 1 -- 172.21.15.36:0/2553233688 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f49600539d0 msgr2=0x7f4960055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:22.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.784+0000 7f498359e700 1 --2- 172.21.15.36:0/2553233688 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f49600539d0 0x7f4960055e90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f497401d000 tx=0x7f4974019000).stop 2022-06-17T12:52:22.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.784+0000 7f498359e700 1 -- 172.21.15.36:0/2553233688 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f498413e480 msgr2=0x7f49841109f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:22.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.784+0000 7f498359e700 1 --2- 172.21.15.36:0/2553233688 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f498413e480 0x7f49841109f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f497c0011d0 tx=0x7f497c009040).stop 2022-06-17T12:52:22.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.784+0000 7f498359e700 1 -- 172.21.15.36:0/2553233688 shutdown_connections 2022-06-17T12:52:22.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.784+0000 7f498359e700 1 --2- 172.21.15.36:0/2553233688 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f49600539d0 0x7f4960055e90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.784+0000 7f498359e700 1 --2- 172.21.15.36:0/2553233688 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f498413e480 0x7f49841109f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.784+0000 7f498359e700 1 --2- 172.21.15.36:0/2553233688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f498413db30 0x7f498413df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.784+0000 7f498359e700 1 --2- 172.21.15.36:0/2553233688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f49680038e0 0x7f498405c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.784+0000 7f498359e700 1 -- 172.21.15.36:0/2553233688 >> 172.21.15.36:0/2553233688 conn(0x7f49840614b0 msgr2=0x7f498413e980 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:22.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.784+0000 7f498359e700 1 -- 172.21.15.36:0/2553233688 shutdown_connections 2022-06-17T12:52:22.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.784+0000 7f498359e700 1 -- 172.21.15.36:0/2553233688 wait complete. 2022-06-17T12:52:22.785 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-17T12:52:22.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.912+0000 7efe0afe8700 1 Processor -- start 2022-06-17T12:52:22.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.912+0000 7efe0afe8700 1 -- start start 2022-06-17T12:52:22.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe0afe8700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efe040b45c0 0x7efe040b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe0afe8700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7efe040b4f00 0x7efe040b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe0afe8700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe040b4160 con 0x7efe040b9860 2022-06-17T12:52:22.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe0afe8700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe040b9c40 con 0x7efe040b4f00 2022-06-17T12:52:22.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe0afe8700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe040627d0 con 0x7efe040b45c0 2022-06-17T12:52:22.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe0a7e7700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7efe040b4f00 0x7efe040b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe0a7e7700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7efe040b4f00 0x7efe040b5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34866/0 (socket says 172.21.15.36:34866) 2022-06-17T12:52:22.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe0a7e7700 1 -- 172.21.15.36:0/1263084606 learned_addr learned my addr 172.21.15.36:0/1263084606 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:22.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe097e5700 1 --2- 172.21.15.36:0/1263084606 >> v2:172.21.15.70:3300/0 conn(0x7efe040b45c0 0x7efe040b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe0a7e7700 1 -- 172.21.15.36:0/1263084606 >> v2:172.21.15.70:3300/0 conn(0x7efe040b45c0 msgr2=0x7efe040b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:22.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe0a7e7700 1 --2- 172.21.15.36:0/1263084606 >> v2:172.21.15.70:3300/0 conn(0x7efe040b45c0 0x7efe040b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe0a7e7700 1 -- 172.21.15.36:0/1263084606 >> v1:172.21.15.70:6789/0 conn(0x7efe040b9860 legacy=0x7efe04143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:22.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe0a7e7700 1 -- 172.21.15.36:0/1263084606 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe04062a70 con 0x7efe040b4f00 2022-06-17T12:52:22.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe0a7e7700 1 --2- 172.21.15.36:0/1263084606 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe040b4f00 0x7efe040b5dc0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7efe04142ef0 tx=0x7efdfc009290).ready entity=mon.0 client_cookie=1de66f290f7ebed2 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:22.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe08fe4700 1 -- 172.21.15.36:0/1263084606 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efdfc00b7f0 con 0x7efe040b4f00 2022-06-17T12:52:22.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe08fe4700 1 -- 172.21.15.36:0/1263084606 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdfc00b950 con 0x7efe040b4f00 2022-06-17T12:52:22.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe08fe4700 1 -- 172.21.15.36:0/1263084606 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efdfc00bac0 con 0x7efe040b4f00 2022-06-17T12:52:22.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe0afe8700 1 -- 172.21.15.36:0/1263084606 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe040b4f00 msgr2=0x7efe040b5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:22.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe0afe8700 1 --2- 172.21.15.36:0/1263084606 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe040b4f00 0x7efe040b5dc0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7efe04142ef0 tx=0x7efdfc009290).stop 2022-06-17T12:52:22.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe0afe8700 1 -- 172.21.15.36:0/1263084606 shutdown_connections 2022-06-17T12:52:22.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe0afe8700 1 --2- 172.21.15.36:0/1263084606 >> v2:172.21.15.70:3300/0 conn(0x7efe040b45c0 0x7efe040b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe0afe8700 1 --2- 172.21.15.36:0/1263084606 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe040b4f00 0x7efe040b5dc0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe0afe8700 1 -- 172.21.15.36:0/1263084606 >> 172.21.15.36:0/1263084606 conn(0x7efe0405c550 msgr2=0x7efe04060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:22.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe0afe8700 1 -- 172.21.15.36:0/1263084606 shutdown_connections 2022-06-17T12:52:22.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe0afe8700 1 -- 172.21.15.36:0/1263084606 wait complete. 2022-06-17T12:52:22.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe0afe8700 1 Processor -- start 2022-06-17T12:52:22.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.916+0000 7efe0afe8700 1 -- start start 2022-06-17T12:52:22.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.920+0000 7efe0afe8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe040b45c0 0x7efe04144390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.920+0000 7efe0afe8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe040b4f00 0x7efe040af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.920+0000 7efe0afe8700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe040b9860 0x7efe0413d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.920+0000 7efe0afe8700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe041472a0 con 0x7efe040b4f00 2022-06-17T12:52:22.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.920+0000 7efe0afe8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0412baf0 con 0x7efe040b45c0 2022-06-17T12:52:22.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.920+0000 7efe0afe8700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe04145ba0 con 0x7efe040b9860 2022-06-17T12:52:22.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.920+0000 7efe097e5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe040b4f00 0x7efe040af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.920+0000 7efe097e5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe040b4f00 0x7efe040af780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34870/0 (socket says 172.21.15.36:34870) 2022-06-17T12:52:22.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.920+0000 7efe097e5700 1 -- 172.21.15.36:0/3026664822 learned_addr learned my addr 172.21.15.36:0/3026664822 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:22.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.920+0000 7efe0a7e7700 1 --2- 172.21.15.36:0/3026664822 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe040b9860 0x7efe0413d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.920+0000 7efe09fe6700 1 --2- 172.21.15.36:0/3026664822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe040b45c0 0x7efe04144390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.920+0000 7efe097e5700 1 -- 172.21.15.36:0/3026664822 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe040b9860 msgr2=0x7efe0413d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:22.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.920+0000 7efe097e5700 1 --2- 172.21.15.36:0/3026664822 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe040b9860 0x7efe0413d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.920+0000 7efe097e5700 1 -- 172.21.15.36:0/3026664822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe040b45c0 msgr2=0x7efe04144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:22.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.920+0000 7efe097e5700 1 --2- 172.21.15.36:0/3026664822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe040b45c0 0x7efe04144390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:22.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.920+0000 7efe097e5700 1 -- 172.21.15.36:0/3026664822 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc009530 con 0x7efe040b4f00 2022-06-17T12:52:22.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.920+0000 7efe097e5700 1 --2- 172.21.15.36:0/3026664822 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe040b4f00 0x7efe040af780 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7efe04140610 tx=0x7efdf4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:22.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.920+0000 7efdf2ffd700 1 -- 172.21.15.36:0/3026664822 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efdf400dea0 con 0x7efe040b4f00 2022-06-17T12:52:22.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.920+0000 7efdf2ffd700 1 -- 172.21.15.36:0/3026664822 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf4011780 con 0x7efe040b4f00 2022-06-17T12:52:22.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.920+0000 7efe0afe8700 1 -- 172.21.15.36:0/3026664822 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe04146270 con 0x7efe040b4f00 2022-06-17T12:52:22.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.920+0000 7efdf2ffd700 1 -- 172.21.15.36:0/3026664822 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efdf4011ae0 con 0x7efe040b4f00 2022-06-17T12:52:22.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.920+0000 7efe0afe8700 1 -- 172.21.15.36:0/3026664822 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe04140340 con 0x7efe040b4f00 2022-06-17T12:52:22.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.920+0000 7efe0afe8700 1 -- 172.21.15.36:0/3026664822 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe04145f70 con 0x7efe040b4f00 2022-06-17T12:52:22.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.920+0000 7efdf2ffd700 1 -- 172.21.15.36:0/3026664822 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7efdf4011c40 con 0x7efe040b4f00 2022-06-17T12:52:22.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.924+0000 7efdf2ffd700 1 --2- 172.21.15.36:0/3026664822 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efde4053ad0 0x7efde4055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:22.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.924+0000 7efdf2ffd700 1 -- 172.21.15.36:0/3026664822 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6420+0+0 (secure 0 0 0) 0x7efdf406bd60 con 0x7efe040b4f00 2022-06-17T12:52:22.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.924+0000 7efe09fe6700 1 --2- 172.21.15.36:0/3026664822 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efde4053ad0 0x7efde4055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:22.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.924+0000 7efe09fe6700 1 --2- 172.21.15.36:0/3026664822 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efde4053ad0 0x7efde4055f90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7efe0000c000 tx=0x7efe00008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:22.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:22.928+0000 7efdf2ffd700 1 -- 172.21.15.36:0/3026664822 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efdf4042b60 con 0x7efe040b4f00 2022-06-17T12:52:23.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.092+0000 7efe0afe8700 1 -- 172.21.15.36:0/3026664822 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7efe04057410 con 0x7efe040b4f00 2022-06-17T12:52:23.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.268+0000 7efdf2ffd700 1 -- 172.21.15.36:0/3026664822 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v30) v1 ==== 118+0+0 (secure 0 0 0) 0x7efdf4046180 con 0x7efe040b4f00 2022-06-17T12:52:23.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.268+0000 7efe0afe8700 1 -- 172.21.15.36:0/3026664822 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efde4053ad0 msgr2=0x7efde4055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:23.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.268+0000 7efe0afe8700 1 --2- 172.21.15.36:0/3026664822 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efde4053ad0 0x7efde4055f90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7efe0000c000 tx=0x7efe00008040).stop 2022-06-17T12:52:23.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.268+0000 7efe0afe8700 1 -- 172.21.15.36:0/3026664822 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe040b4f00 msgr2=0x7efe040af780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:23.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.268+0000 7efe0afe8700 1 --2- 172.21.15.36:0/3026664822 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe040b4f00 0x7efe040af780 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7efe04140610 tx=0x7efdf4008040).stop 2022-06-17T12:52:23.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.268+0000 7efe0afe8700 1 -- 172.21.15.36:0/3026664822 shutdown_connections 2022-06-17T12:52:23.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.268+0000 7efe0afe8700 1 --2- 172.21.15.36:0/3026664822 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efde4053ad0 0x7efde4055f90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.268+0000 7efe0afe8700 1 --2- 172.21.15.36:0/3026664822 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe040b9860 0x7efe0413d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.268+0000 7efe0afe8700 1 --2- 172.21.15.36:0/3026664822 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe040b4f00 0x7efe040af780 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.268+0000 7efe0afe8700 1 --2- 172.21.15.36:0/3026664822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe040b45c0 0x7efe04144390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.268+0000 7efe0afe8700 1 -- 172.21.15.36:0/3026664822 >> 172.21.15.36:0/3026664822 conn(0x7efe0405c550 msgr2=0x7efe040b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:23.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.268+0000 7efe0afe8700 1 -- 172.21.15.36:0/3026664822 shutdown_connections 2022-06-17T12:52:23.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.268+0000 7efe0afe8700 1 -- 172.21.15.36:0/3026664822 wait complete. 2022-06-17T12:52:23.269 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:52:23.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.388+0000 7fb48634d700 1 Processor -- start 2022-06-17T12:52:23.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.388+0000 7fb48634d700 1 -- start start 2022-06-17T12:52:23.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.388+0000 7fb48634d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb4800aef60 0x7fb4800b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:23.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.388+0000 7fb48634d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb4800ba2e0 0x7fb4801442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:23.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.388+0000 7fb48634d700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4800af4b0 con 0x7fb48013cb70 2022-06-17T12:52:23.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.388+0000 7fb48634d700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb480143da0 con 0x7fb4800aef60 2022-06-17T12:52:23.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.388+0000 7fb48634d700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb480142f50 con 0x7fb4800ba2e0 2022-06-17T12:52:23.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.388+0000 7fb484b4a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb4800aef60 0x7fb4800b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:23.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.388+0000 7fb48534b700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fb48013cb70 0x7fb480143690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52406/0 (socket says 172.21.15.36:52406) 2022-06-17T12:52:23.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.388+0000 7fb48534b700 1 -- 172.21.15.36:0/703132117 learned_addr learned my addr 172.21.15.36:0/703132117 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:23.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.388+0000 7fb485b4c700 1 --2- 172.21.15.36:0/703132117 >> v2:172.21.15.70:3300/0 conn(0x7fb4800ba2e0 0x7fb4801442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:23.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.388+0000 7fb47ffff700 1 -- 172.21.15.36:0/703132117 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4093848839 0 0) 0x7fb4800af4b0 con 0x7fb48013cb70 2022-06-17T12:52:23.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.388+0000 7fb47ffff700 1 -- 172.21.15.36:0/703132117 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb46c003260 con 0x7fb48013cb70 2022-06-17T12:52:23.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.388+0000 7fb484b4a700 1 -- 172.21.15.36:0/703132117 >> v2:172.21.15.70:3300/0 conn(0x7fb4800ba2e0 msgr2=0x7fb4801442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:23.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.388+0000 7fb484b4a700 1 --2- 172.21.15.36:0/703132117 >> v2:172.21.15.70:3300/0 conn(0x7fb4800ba2e0 0x7fb4801442a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.388+0000 7fb484b4a700 1 -- 172.21.15.36:0/703132117 >> v1:172.21.15.36:6789/0 conn(0x7fb48013cb70 legacy=0x7fb480143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:23.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.388+0000 7fb484b4a700 1 -- 172.21.15.36:0/703132117 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4801431f0 con 0x7fb4800aef60 2022-06-17T12:52:23.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.392+0000 7fb484b4a700 1 --2- 172.21.15.36:0/703132117 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb4800aef60 0x7fb4800b9df0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7fb474002f70 tx=0x7fb474006df0).ready entity=mon.0 client_cookie=a2217455fc396a3 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:23.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.392+0000 7fb47ffff700 1 -- 172.21.15.36:0/703132117 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb47400d4f0 con 0x7fb4800aef60 2022-06-17T12:52:23.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.392+0000 7fb47ffff700 1 -- 172.21.15.36:0/703132117 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb474000e90 con 0x7fb4800aef60 2022-06-17T12:52:23.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.392+0000 7fb47ffff700 1 -- 172.21.15.36:0/703132117 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb47400d840 con 0x7fb4800aef60 2022-06-17T12:52:23.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.392+0000 7fb48634d700 1 -- 172.21.15.36:0/703132117 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb4800aef60 msgr2=0x7fb4800b9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:23.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.392+0000 7fb48634d700 1 --2- 172.21.15.36:0/703132117 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb4800aef60 0x7fb4800b9df0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7fb474002f70 tx=0x7fb474006df0).stop 2022-06-17T12:52:23.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.392+0000 7fb48634d700 1 -- 172.21.15.36:0/703132117 shutdown_connections 2022-06-17T12:52:23.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.392+0000 7fb48634d700 1 --2- 172.21.15.36:0/703132117 >> v2:172.21.15.70:3300/0 conn(0x7fb4800ba2e0 0x7fb4801442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.392+0000 7fb48634d700 1 --2- 172.21.15.36:0/703132117 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb4800aef60 0x7fb4800b9df0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.392+0000 7fb48634d700 1 -- 172.21.15.36:0/703132117 >> 172.21.15.36:0/703132117 conn(0x7fb48005c550 msgr2=0x7fb4800b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:23.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.392+0000 7fb48634d700 1 -- 172.21.15.36:0/703132117 shutdown_connections 2022-06-17T12:52:23.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.392+0000 7fb48634d700 1 -- 172.21.15.36:0/703132117 wait complete. 2022-06-17T12:52:23.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.392+0000 7fb48634d700 1 Processor -- start 2022-06-17T12:52:23.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.392+0000 7fb48634d700 1 -- start start 2022-06-17T12:52:23.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.396+0000 7fb48634d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb4800aef60 0x7fb4800b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:23.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.396+0000 7fb48634d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4800ba2e0 0x7fb4800b3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:23.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.396+0000 7fb48634d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb48013cb70 0x7fb4801128d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:23.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.396+0000 7fb48634d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4800b00b0 con 0x7fb4800aef60 2022-06-17T12:52:23.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.396+0000 7fb48634d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb480155770 con 0x7fb4800ba2e0 2022-06-17T12:52:23.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.396+0000 7fb48634d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4801402a0 con 0x7fb48013cb70 2022-06-17T12:52:23.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.396+0000 7fb48534b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb4800aef60 0x7fb4800b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:23.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.396+0000 7fb48534b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb4800aef60 0x7fb4800b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34882/0 (socket says 172.21.15.36:34882) 2022-06-17T12:52:23.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.396+0000 7fb48534b700 1 -- 172.21.15.36:0/2219283032 learned_addr learned my addr 172.21.15.36:0/2219283032 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:23.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.396+0000 7fb485b4c700 1 --2- 172.21.15.36:0/2219283032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb48013cb70 0x7fb4801128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:23.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.396+0000 7fb484b4a700 1 --2- 172.21.15.36:0/2219283032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4800ba2e0 0x7fb4800b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:23.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.396+0000 7fb48534b700 1 -- 172.21.15.36:0/2219283032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb48013cb70 msgr2=0x7fb4801128d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:23.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.396+0000 7fb48534b700 1 --2- 172.21.15.36:0/2219283032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb48013cb70 0x7fb4801128d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.396+0000 7fb48534b700 1 -- 172.21.15.36:0/2219283032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4800ba2e0 msgr2=0x7fb4800b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:23.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.396+0000 7fb48534b700 1 --2- 172.21.15.36:0/2219283032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4800ba2e0 0x7fb4800b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.396+0000 7fb48534b700 1 -- 172.21.15.36:0/2219283032 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47400b180 con 0x7fb4800aef60 2022-06-17T12:52:23.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.396+0000 7fb48534b700 1 --2- 172.21.15.36:0/2219283032 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb4800aef60 0x7fb4800b1fa0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7fb470001110 tx=0x7fb470007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:23.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.396+0000 7fb47e7fc700 1 -- 172.21.15.36:0/2219283032 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4700070d0 con 0x7fb4800aef60 2022-06-17T12:52:23.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.396+0000 7fb47e7fc700 1 -- 172.21.15.36:0/2219283032 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb470007230 con 0x7fb4800aef60 2022-06-17T12:52:23.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.396+0000 7fb47e7fc700 1 -- 172.21.15.36:0/2219283032 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb470005610 con 0x7fb4800aef60 2022-06-17T12:52:23.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.396+0000 7fb48634d700 1 -- 172.21.15.36:0/2219283032 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4800afba0 con 0x7fb4800aef60 2022-06-17T12:52:23.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.396+0000 7fb48634d700 1 -- 172.21.15.36:0/2219283032 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb480112f60 con 0x7fb4800aef60 2022-06-17T12:52:23.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.396+0000 7fb48634d700 1 -- 172.21.15.36:0/2219283032 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4800b0f30 con 0x7fb4800aef60 2022-06-17T12:52:23.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.400+0000 7fb47e7fc700 1 -- 172.21.15.36:0/2219283032 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fb470010070 con 0x7fb4800aef60 2022-06-17T12:52:23.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.400+0000 7fb47e7fc700 1 --2- 172.21.15.36:0/2219283032 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb460053a40 0x7fb460055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:23.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.400+0000 7fb47e7fc700 1 -- 172.21.15.36:0/2219283032 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb47006b5a0 con 0x7fb4800aef60 2022-06-17T12:52:23.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.400+0000 7fb484b4a700 1 --2- 172.21.15.36:0/2219283032 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb460053a40 0x7fb460055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:23.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.400+0000 7fb484b4a700 1 --2- 172.21.15.36:0/2219283032 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb460053a40 0x7fb460055f00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fb47401a000 tx=0x7fb474014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:23.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.404+0000 7fb47e7fc700 1 -- 172.21.15.36:0/2219283032 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb4700423a0 con 0x7fb4800aef60 2022-06-17T12:52:23.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.572+0000 7fb48634d700 1 -- 172.21.15.36:0/2219283032 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb480057410 con 0x7fb4800aef60 2022-06-17T12:52:23.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.576+0000 7fb47e7fc700 1 -- 172.21.15.36:0/2219283032 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2140 (secure 0 0 0) 0x7fb4700459c0 con 0x7fb4800aef60 2022-06-17T12:52:23.561 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:52:23.561 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":30,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5420,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/3105680199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3105680199},{"type":"v1","addr":"172.21.15.70:6833","nonce":3105680199}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":29}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:52:22.253232+0000","modified":"2022-06-17T12:52:23.257219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6054},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6054":{"gid":6054,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/4091208534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":4091208534},{"type":"v1","addr":"172.21.15.36:6835","nonce":4091208534}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-06-17T12:52:23.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.576+0000 7fb48634d700 1 -- 172.21.15.36:0/2219283032 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb460053a40 msgr2=0x7fb460055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:23.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.576+0000 7fb48634d700 1 --2- 172.21.15.36:0/2219283032 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb460053a40 0x7fb460055f00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fb47401a000 tx=0x7fb474014000).stop 2022-06-17T12:52:23.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.576+0000 7fb48634d700 1 -- 172.21.15.36:0/2219283032 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb4800aef60 msgr2=0x7fb4800b1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:23.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.576+0000 7fb48634d700 1 --2- 172.21.15.36:0/2219283032 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb4800aef60 0x7fb4800b1fa0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7fb470001110 tx=0x7fb470007ad0).stop 2022-06-17T12:52:23.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.576+0000 7fb48634d700 1 -- 172.21.15.36:0/2219283032 shutdown_connections 2022-06-17T12:52:23.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.576+0000 7fb48634d700 1 --2- 172.21.15.36:0/2219283032 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb460053a40 0x7fb460055f00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.576+0000 7fb48634d700 1 --2- 172.21.15.36:0/2219283032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb48013cb70 0x7fb4801128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.576+0000 7fb48634d700 1 --2- 172.21.15.36:0/2219283032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4800ba2e0 0x7fb4800b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.576+0000 7fb48634d700 1 --2- 172.21.15.36:0/2219283032 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb4800aef60 0x7fb4800b1fa0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.576+0000 7fb48634d700 1 -- 172.21.15.36:0/2219283032 >> 172.21.15.36:0/2219283032 conn(0x7fb48005c550 msgr2=0x7fb4800b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:23.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.576+0000 7fb48634d700 1 -- 172.21.15.36:0/2219283032 shutdown_connections 2022-06-17T12:52:23.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.576+0000 7fb48634d700 1 -- 172.21.15.36:0/2219283032 wait complete. 2022-06-17T12:52:23.571 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 30 2022-06-17T12:52:23.577 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:52:23.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.688+0000 7f1bb2dfa700 1 Processor -- start 2022-06-17T12:52:23.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.688+0000 7f1bb2dfa700 1 -- start start 2022-06-17T12:52:23.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb2dfa700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f1bac13cb70 0x7f1bac143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:23.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb2dfa700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1bac0aef60 0x7f1bac0b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:23.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb2dfa700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bac0af4b0 con 0x7f1bac0ba2e0 2022-06-17T12:52:23.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb2dfa700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bac142f50 con 0x7f1bac13cb70 2022-06-17T12:52:23.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb2dfa700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bac1431f0 con 0x7f1bac0aef60 2022-06-17T12:52:23.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb1df8700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f1bac13cb70 0x7f1bac143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:23.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb25f9700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f1bac0ba2e0 0x7f1bac1442a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52424/0 (socket says 172.21.15.36:52424) 2022-06-17T12:52:23.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb25f9700 1 -- 172.21.15.36:0/1587664822 learned_addr learned my addr 172.21.15.36:0/1587664822 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:23.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb15f7700 1 --2- 172.21.15.36:0/1587664822 >> v2:172.21.15.70:3300/0 conn(0x7f1bac0aef60 0x7f1bac0b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:23.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb1df8700 1 -- 172.21.15.36:0/1587664822 >> v2:172.21.15.70:3300/0 conn(0x7f1bac0aef60 msgr2=0x7f1bac0b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:23.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb1df8700 1 --2- 172.21.15.36:0/1587664822 >> v2:172.21.15.70:3300/0 conn(0x7f1bac0aef60 0x7f1bac0b9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb1df8700 1 -- 172.21.15.36:0/1587664822 >> v1:172.21.15.36:6789/0 conn(0x7f1bac0ba2e0 legacy=0x7f1bac1442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:23.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb0df6700 1 -- 172.21.15.36:0/1587664822 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4066859727 0 0) 0x7f1bac0af4b0 con 0x7f1bac0ba2e0 2022-06-17T12:52:23.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb1df8700 1 -- 172.21.15.36:0/1587664822 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bac1449b0 con 0x7f1bac13cb70 2022-06-17T12:52:23.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb1df8700 1 --2- 172.21.15.36:0/1587664822 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1bac13cb70 0x7f1bac143690 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f1ba8002f70 tx=0x7f1ba8006df0).ready entity=mon.0 client_cookie=668dc1c82f27df7 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:23.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb0df6700 1 -- 172.21.15.36:0/1587664822 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ba800d570 con 0x7f1bac13cb70 2022-06-17T12:52:23.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb0df6700 1 -- 172.21.15.36:0/1587664822 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba8000e90 con 0x7f1bac13cb70 2022-06-17T12:52:23.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb0df6700 1 -- 172.21.15.36:0/1587664822 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ba800d8c0 con 0x7f1bac13cb70 2022-06-17T12:52:23.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb2dfa700 1 -- 172.21.15.36:0/1587664822 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1bac13cb70 msgr2=0x7f1bac143690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:23.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb2dfa700 1 --2- 172.21.15.36:0/1587664822 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1bac13cb70 0x7f1bac143690 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f1ba8002f70 tx=0x7f1ba8006df0).stop 2022-06-17T12:52:23.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb2dfa700 1 -- 172.21.15.36:0/1587664822 shutdown_connections 2022-06-17T12:52:23.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb2dfa700 1 --2- 172.21.15.36:0/1587664822 >> v2:172.21.15.70:3300/0 conn(0x7f1bac0aef60 0x7f1bac0b9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb2dfa700 1 --2- 172.21.15.36:0/1587664822 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1bac13cb70 0x7f1bac143690 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb2dfa700 1 -- 172.21.15.36:0/1587664822 >> 172.21.15.36:0/1587664822 conn(0x7f1bac05c550 msgr2=0x7f1bac0b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:23.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb2dfa700 1 -- 172.21.15.36:0/1587664822 shutdown_connections 2022-06-17T12:52:23.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.692+0000 7f1bb2dfa700 1 -- 172.21.15.36:0/1587664822 wait complete. 2022-06-17T12:52:23.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.696+0000 7f1bb2dfa700 1 Processor -- start 2022-06-17T12:52:23.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.696+0000 7f1bb2dfa700 1 -- start start 2022-06-17T12:52:23.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.696+0000 7f1bb2dfa700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1bac0aef60 0x7f1bac152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:23.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.696+0000 7f1bb2dfa700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1bac0ba2e0 0x7f1bac13f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:23.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.696+0000 7f1bb2dfa700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1bac13cb70 0x7f1bac140960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:23.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.696+0000 7f1bb2dfa700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac13fa70 con 0x7f1bac13cb70 2022-06-17T12:52:23.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.696+0000 7f1bb2dfa700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac13e3f0 con 0x7f1bac0ba2e0 2022-06-17T12:52:23.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.696+0000 7f1bb2dfa700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac13e690 con 0x7f1bac0aef60 2022-06-17T12:52:23.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.696+0000 7f1bb25f9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1bac13cb70 0x7f1bac140960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:23.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.696+0000 7f1bb1df8700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1bac0aef60 0x7f1bac152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:23.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.696+0000 7f1bb25f9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1bac13cb70 0x7f1bac140960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34900/0 (socket says 172.21.15.36:34900) 2022-06-17T12:52:23.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.696+0000 7f1bb25f9700 1 -- 172.21.15.36:0/2435549680 learned_addr learned my addr 172.21.15.36:0/2435549680 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:23.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.696+0000 7f1bb15f7700 1 --2- 172.21.15.36:0/2435549680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1bac0ba2e0 0x7f1bac13f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:23.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.696+0000 7f1bb25f9700 1 -- 172.21.15.36:0/2435549680 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1bac0aef60 msgr2=0x7f1bac152fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:23.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.696+0000 7f1bb25f9700 1 --2- 172.21.15.36:0/2435549680 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1bac0aef60 0x7f1bac152fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.696+0000 7f1bb25f9700 1 -- 172.21.15.36:0/2435549680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1bac0ba2e0 msgr2=0x7f1bac13f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:23.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.696+0000 7f1bb25f9700 1 --2- 172.21.15.36:0/2435549680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1bac0ba2e0 0x7f1bac13f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.696+0000 7f1bb25f9700 1 -- 172.21.15.36:0/2435549680 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba800b180 con 0x7f1bac13cb70 2022-06-17T12:52:23.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.696+0000 7f1bb25f9700 1 --2- 172.21.15.36:0/2435549680 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1bac13cb70 0x7f1bac140960 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f1ba4000f40 tx=0x7f1ba4007d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:23.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.696+0000 7f1b9affd700 1 -- 172.21.15.36:0/2435549680 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ba4005ba0 con 0x7f1bac13cb70 2022-06-17T12:52:23.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.696+0000 7f1b9affd700 1 -- 172.21.15.36:0/2435549680 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba4005d00 con 0x7f1bac13cb70 2022-06-17T12:52:23.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.696+0000 7f1b9affd700 1 -- 172.21.15.36:0/2435549680 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ba40072c0 con 0x7f1bac13cb70 2022-06-17T12:52:23.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.696+0000 7f1bb2dfa700 1 -- 172.21.15.36:0/2435549680 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bac13e930 con 0x7f1bac13cb70 2022-06-17T12:52:23.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.700+0000 7f1bb2dfa700 1 -- 172.21.15.36:0/2435549680 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bac13d070 con 0x7f1bac13cb70 2022-06-17T12:52:23.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.700+0000 7f1bb2dfa700 1 -- 172.21.15.36:0/2435549680 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bac057410 con 0x7f1bac13cb70 2022-06-17T12:52:23.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.700+0000 7f1b9affd700 1 -- 172.21.15.36:0/2435549680 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f1ba4010070 con 0x7f1bac13cb70 2022-06-17T12:52:23.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.700+0000 7f1b9affd700 1 --2- 172.21.15.36:0/2435549680 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1b8c0539b0 0x7f1b8c055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:23.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.700+0000 7f1b9affd700 1 -- 172.21.15.36:0/2435549680 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1ba406ba30 con 0x7f1bac13cb70 2022-06-17T12:52:23.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.700+0000 7f1bb1df8700 1 --2- 172.21.15.36:0/2435549680 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1b8c0539b0 0x7f1b8c055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:23.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.700+0000 7f1bb1df8700 1 --2- 172.21.15.36:0/2435549680 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1b8c0539b0 0x7f1b8c055e70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f1ba801a000 tx=0x7f1ba8014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:23.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.708+0000 7f1b9affd700 1 -- 172.21.15.36:0/2435549680 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1ba4042720 con 0x7f1bac13cb70 2022-06-17T12:52:23.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.836+0000 7f1bb2dfa700 1 -- 172.21.15.36:0/2435549680 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1bac13eac0 con 0x7f1bac13cb70 2022-06-17T12:52:23.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.836+0000 7f1b9affd700 1 -- 172.21.15.36:0/2435549680 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f1ba4045d40 con 0x7f1bac13cb70 2022-06-17T12:52:23.825 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:52:23.825 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":65,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:52:23.256104+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:52:20.693971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2022-06-17T12:52:21.830992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:52:23.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.840+0000 7f1bb2dfa700 1 -- 172.21.15.36:0/2435549680 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1b8c0539b0 msgr2=0x7f1b8c055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:23.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.840+0000 7f1bb2dfa700 1 --2- 172.21.15.36:0/2435549680 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1b8c0539b0 0x7f1b8c055e70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f1ba801a000 tx=0x7f1ba8014000).stop 2022-06-17T12:52:23.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.840+0000 7f1bb2dfa700 1 -- 172.21.15.36:0/2435549680 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1bac13cb70 msgr2=0x7f1bac140960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:23.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.840+0000 7f1bb2dfa700 1 --2- 172.21.15.36:0/2435549680 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1bac13cb70 0x7f1bac140960 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f1ba4000f40 tx=0x7f1ba4007d40).stop 2022-06-17T12:52:23.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.840+0000 7f1bb2dfa700 1 -- 172.21.15.36:0/2435549680 shutdown_connections 2022-06-17T12:52:23.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.840+0000 7f1bb2dfa700 1 --2- 172.21.15.36:0/2435549680 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1b8c0539b0 0x7f1b8c055e70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.840+0000 7f1bb2dfa700 1 --2- 172.21.15.36:0/2435549680 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1bac13cb70 0x7f1bac140960 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.840+0000 7f1bb2dfa700 1 --2- 172.21.15.36:0/2435549680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1bac0ba2e0 0x7f1bac13f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.840+0000 7f1bb2dfa700 1 --2- 172.21.15.36:0/2435549680 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1bac0aef60 0x7f1bac152fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.840+0000 7f1bb2dfa700 1 -- 172.21.15.36:0/2435549680 >> 172.21.15.36:0/2435549680 conn(0x7f1bac05c550 msgr2=0x7f1bac05f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:23.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.840+0000 7f1bb2dfa700 1 -- 172.21.15.36:0/2435549680 shutdown_connections 2022-06-17T12:52:23.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.840+0000 7f1bb2dfa700 1 -- 172.21.15.36:0/2435549680 wait complete. 2022-06-17T12:52:23.840 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:52:23.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.984+0000 7fd7cf59e700 1 Processor -- start 2022-06-17T12:52:23.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.984+0000 7fd7cf59e700 1 -- start start 2022-06-17T12:52:23.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.984+0000 7fd7cf59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7d00aef60 0x7fd7d00b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:23.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.984+0000 7fd7cf59e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd7d00ba2e0 0x7fd7d01442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:23.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.984+0000 7fd7cf59e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d00af4b0 con 0x7fd7d013cb70 2022-06-17T12:52:23.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.984+0000 7fd7cf59e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d0143da0 con 0x7fd7d00ba2e0 2022-06-17T12:52:23.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.984+0000 7fd7cf59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d0142f50 con 0x7fd7d00aef60 2022-06-17T12:52:23.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.984+0000 7fd7ce59c700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fd7d013cb70 0x7fd7d0143690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52434/0 (socket says 172.21.15.36:52434) 2022-06-17T12:52:23.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.984+0000 7fd7ce59c700 1 -- 172.21.15.36:0/2898011378 learned_addr learned my addr 172.21.15.36:0/2898011378 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:23.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.984+0000 7fd7ced9d700 1 --2- 172.21.15.36:0/2898011378 >> v2:172.21.15.36:3300/0 conn(0x7fd7d00ba2e0 0x7fd7d01442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:23.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.984+0000 7fd7cdd9b700 1 --2- 172.21.15.36:0/2898011378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7d00aef60 0x7fd7d00b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:23.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.984+0000 7fd7cd59a700 1 -- 172.21.15.36:0/2898011378 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1549447267 0 0) 0x7fd7d00af4b0 con 0x7fd7d013cb70 2022-06-17T12:52:23.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.984+0000 7fd7cd59a700 1 -- 172.21.15.36:0/2898011378 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7b4003260 con 0x7fd7d013cb70 2022-06-17T12:52:23.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.984+0000 7fd7ced9d700 1 -- 172.21.15.36:0/2898011378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7d00aef60 msgr2=0x7fd7d00b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:23.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.984+0000 7fd7ced9d700 1 --2- 172.21.15.36:0/2898011378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7d00aef60 0x7fd7d00b9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.984+0000 7fd7ced9d700 1 -- 172.21.15.36:0/2898011378 >> v1:172.21.15.36:6789/0 conn(0x7fd7d013cb70 legacy=0x7fd7d0143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:23.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.984+0000 7fd7ced9d700 1 -- 172.21.15.36:0/2898011378 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d01431f0 con 0x7fd7d00ba2e0 2022-06-17T12:52:23.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.984+0000 7fd7ced9d700 1 --2- 172.21.15.36:0/2898011378 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7d00ba2e0 0x7fd7d01442a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fd7c8002f70 tx=0x7fd7c8006df0).ready entity=mon.0 client_cookie=3443443d875eb03 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:23.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.984+0000 7fd7cd59a700 1 -- 172.21.15.36:0/2898011378 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7c800d4a0 con 0x7fd7d00ba2e0 2022-06-17T12:52:23.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.984+0000 7fd7cd59a700 1 -- 172.21.15.36:0/2898011378 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c8000e90 con 0x7fd7d00ba2e0 2022-06-17T12:52:23.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.984+0000 7fd7cd59a700 1 -- 172.21.15.36:0/2898011378 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7c800d7f0 con 0x7fd7d00ba2e0 2022-06-17T12:52:23.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.988+0000 7fd7cf59e700 1 -- 172.21.15.36:0/2898011378 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7d00ba2e0 msgr2=0x7fd7d01442a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:23.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.988+0000 7fd7cf59e700 1 --2- 172.21.15.36:0/2898011378 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7d00ba2e0 0x7fd7d01442a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fd7c8002f70 tx=0x7fd7c8006df0).stop 2022-06-17T12:52:23.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.988+0000 7fd7cf59e700 1 -- 172.21.15.36:0/2898011378 shutdown_connections 2022-06-17T12:52:23.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.988+0000 7fd7cf59e700 1 --2- 172.21.15.36:0/2898011378 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7d00ba2e0 0x7fd7d01442a0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.988+0000 7fd7cf59e700 1 --2- 172.21.15.36:0/2898011378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7d00aef60 0x7fd7d00b9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:23.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.988+0000 7fd7cf59e700 1 -- 172.21.15.36:0/2898011378 >> 172.21.15.36:0/2898011378 conn(0x7fd7d005c550 msgr2=0x7fd7d00b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:23.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.988+0000 7fd7cf59e700 1 -- 172.21.15.36:0/2898011378 shutdown_connections 2022-06-17T12:52:23.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.988+0000 7fd7cf59e700 1 -- 172.21.15.36:0/2898011378 wait complete. 2022-06-17T12:52:23.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.988+0000 7fd7cf59e700 1 Processor -- start 2022-06-17T12:52:23.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.988+0000 7fd7cf59e700 1 -- start start 2022-06-17T12:52:23.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.988+0000 7fd7cf59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7d00aef60 0x7fd7d013d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:23.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.988+0000 7fd7cf59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7d00ba2e0 0x7fd7d0155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:23.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.988+0000 7fd7cf59e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7d013cb70 0x7fd7d0156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:23.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.988+0000 7fd7cf59e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d0145740 con 0x7fd7d00aef60 2022-06-17T12:52:23.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.988+0000 7fd7cf59e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d01459e0 con 0x7fd7d013cb70 2022-06-17T12:52:23.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.988+0000 7fd7cf59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d0145c80 con 0x7fd7d00ba2e0 2022-06-17T12:52:24.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.988+0000 7fd7ce59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7d00aef60 0x7fd7d013d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:24.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.988+0000 7fd7ce59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7d00aef60 0x7fd7d013d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34910/0 (socket says 172.21.15.36:34910) 2022-06-17T12:52:24.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.988+0000 7fd7ce59c700 1 -- 172.21.15.36:0/2096246712 learned_addr learned my addr 172.21.15.36:0/2096246712 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:24.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.992+0000 7fd7cdd9b700 1 --2- 172.21.15.36:0/2096246712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7d00ba2e0 0x7fd7d0155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:24.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.992+0000 7fd7ced9d700 1 --2- 172.21.15.36:0/2096246712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7d013cb70 0x7fd7d0156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:24.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.992+0000 7fd7ce59c700 1 -- 172.21.15.36:0/2096246712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7d00ba2e0 msgr2=0x7fd7d0155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:24.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.992+0000 7fd7ce59c700 1 --2- 172.21.15.36:0/2096246712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7d00ba2e0 0x7fd7d0155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.992+0000 7fd7ce59c700 1 -- 172.21.15.36:0/2096246712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7d013cb70 msgr2=0x7fd7d0156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:24.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.992+0000 7fd7ce59c700 1 --2- 172.21.15.36:0/2096246712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7d013cb70 0x7fd7d0156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.992+0000 7fd7ce59c700 1 -- 172.21.15.36:0/2096246712 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c800b180 con 0x7fd7d00aef60 2022-06-17T12:52:24.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.992+0000 7fd7ce59c700 1 --2- 172.21.15.36:0/2096246712 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7d00aef60 0x7fd7d013d290 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7fd7c0001490 tx=0x7fd7c0007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:24.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.992+0000 7fd7bf7fe700 1 -- 172.21.15.36:0/2096246712 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7c0005ba0 con 0x7fd7d00aef60 2022-06-17T12:52:24.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.992+0000 7fd7bf7fe700 1 -- 172.21.15.36:0/2096246712 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c0005d00 con 0x7fd7d00aef60 2022-06-17T12:52:24.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.992+0000 7fd7bf7fe700 1 -- 172.21.15.36:0/2096246712 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7c00072c0 con 0x7fd7d00aef60 2022-06-17T12:52:24.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.992+0000 7fd7cf59e700 1 -- 172.21.15.36:0/2096246712 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d0148910 con 0x7fd7d00aef60 2022-06-17T12:52:24.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.992+0000 7fd7cf59e700 1 -- 172.21.15.36:0/2096246712 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d013ea30 con 0x7fd7d00aef60 2022-06-17T12:52:24.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.992+0000 7fd7cf59e700 1 -- 172.21.15.36:0/2096246712 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d00af950 con 0x7fd7d00aef60 2022-06-17T12:52:24.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.992+0000 7fd7bf7fe700 1 -- 172.21.15.36:0/2096246712 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd7c001a070 con 0x7fd7d00aef60 2022-06-17T12:52:24.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.992+0000 7fd7bf7fe700 1 --2- 172.21.15.36:0/2096246712 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7ac053ad0 0x7fd7ac055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:24.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.996+0000 7fd7cdd9b700 1 --2- 172.21.15.36:0/2096246712 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7ac053ad0 0x7fd7ac055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:24.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.996+0000 7fd7bf7fe700 1 -- 172.21.15.36:0/2096246712 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd7c006b620 con 0x7fd7d00aef60 2022-06-17T12:52:24.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:23.996+0000 7fd7cdd9b700 1 --2- 172.21.15.36:0/2096246712 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7ac053ad0 0x7fd7ac055f90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fd7c400b000 tx=0x7fd7c4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:24.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.000+0000 7fd7bf7fe700 1 -- 172.21.15.36:0/2096246712 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd7c0042420 con 0x7fd7d00aef60 2022-06-17T12:52:24.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.168+0000 7fd7cf59e700 1 -- 172.21.15.36:0/2096246712 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fd7d0057410 con 0x7fd7d00aef60 2022-06-17T12:52:24.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.176+0000 7fd7bf7fe700 1 -- 172.21.15.36:0/2096246712 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v22) v1 ==== 184+0+0 (secure 0 0 0) 0x7fd7c0045a40 con 0x7fd7d00aef60 2022-06-17T12:52:24.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.180+0000 7fd7cf59e700 1 -- 172.21.15.36:0/2096246712 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7ac053ad0 msgr2=0x7fd7ac055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:24.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.180+0000 7fd7cf59e700 1 --2- 172.21.15.36:0/2096246712 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7ac053ad0 0x7fd7ac055f90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fd7c400b000 tx=0x7fd7c4008040).stop 2022-06-17T12:52:24.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.180+0000 7fd7cf59e700 1 -- 172.21.15.36:0/2096246712 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7d00aef60 msgr2=0x7fd7d013d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:24.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.180+0000 7fd7cf59e700 1 --2- 172.21.15.36:0/2096246712 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7d00aef60 0x7fd7d013d290 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7fd7c0001490 tx=0x7fd7c0007e20).stop 2022-06-17T12:52:24.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.180+0000 7fd7cf59e700 1 -- 172.21.15.36:0/2096246712 shutdown_connections 2022-06-17T12:52:24.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.180+0000 7fd7cf59e700 1 --2- 172.21.15.36:0/2096246712 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7ac053ad0 0x7fd7ac055f90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.180+0000 7fd7cf59e700 1 --2- 172.21.15.36:0/2096246712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7d013cb70 0x7fd7d0156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.180+0000 7fd7cf59e700 1 --2- 172.21.15.36:0/2096246712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7d00ba2e0 0x7fd7d0155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.180+0000 7fd7cf59e700 1 --2- 172.21.15.36:0/2096246712 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7d00aef60 0x7fd7d013d290 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.180+0000 7fd7cf59e700 1 -- 172.21.15.36:0/2096246712 >> 172.21.15.36:0/2096246712 conn(0x7fd7d005c550 msgr2=0x7fd7d00b7510 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:24.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.180+0000 7fd7cf59e700 1 -- 172.21.15.36:0/2096246712 shutdown_connections 2022-06-17T12:52:24.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.180+0000 7fd7cf59e700 1 -- 172.21.15.36:0/2096246712 wait complete. 2022-06-17T12:52:24.169 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.0 2022-06-17T12:52:24.179 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:52:24.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.312+0000 7f3ddaaf6700 1 Processor -- start 2022-06-17T12:52:24.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.312+0000 7f3ddaaf6700 1 -- start start 2022-06-17T12:52:24.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.312+0000 7f3ddaaf6700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3dd40b1890 0x7f3dd413c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:24.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.312+0000 7f3ddaaf6700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd40b0230 con 0x7f3dd40b1ff0 2022-06-17T12:52:24.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.312+0000 7f3ddaaf6700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd4060ea0 con 0x7f3dd4060ac0 2022-06-17T12:52:24.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.312+0000 7f3ddaaf6700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd4143800 con 0x7f3dd40b1890 2022-06-17T12:52:24.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.316+0000 7f3dda2f5700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f3dd40b1ff0 0x7f3dd413f270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52452/0 (socket says 172.21.15.36:52452) 2022-06-17T12:52:24.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.316+0000 7f3dda2f5700 1 -- 172.21.15.36:0/4288647165 learned_addr learned my addr 172.21.15.36:0/4288647165 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:24.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.316+0000 7f3dd92f3700 1 --2- 172.21.15.36:0/4288647165 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3dd40b1890 0x7f3dd413c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:24.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.316+0000 7f3dd8af2700 1 -- 172.21.15.36:0/4288647165 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4278194653 0 0) 0x7f3dd40b0230 con 0x7f3dd40b1ff0 2022-06-17T12:52:24.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.316+0000 7f3dd8af2700 1 -- 172.21.15.36:0/4288647165 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dbc003260 con 0x7f3dd40b1ff0 2022-06-17T12:52:24.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.316+0000 7f3dd8af2700 1 -- 172.21.15.36:0/4288647165 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1180308921 0 0) 0x7f3dd4060ea0 con 0x7f3dd4060ac0 2022-06-17T12:52:24.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.316+0000 7f3dd8af2700 1 -- 172.21.15.36:0/4288647165 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dd40b0230 con 0x7f3dd4060ac0 2022-06-17T12:52:24.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.316+0000 7f3dd8af2700 1 -- 172.21.15.36:0/4288647165 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4132606068 0 0) 0x7f3dbc003260 con 0x7f3dd40b1ff0 2022-06-17T12:52:24.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.316+0000 7f3dd92f3700 1 -- 172.21.15.36:0/4288647165 >> v1:172.21.15.70:6789/0 conn(0x7f3dd4060ac0 legacy=0x7f3dd413c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:24.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.316+0000 7f3dd92f3700 1 -- 172.21.15.36:0/4288647165 >> v1:172.21.15.36:6789/0 conn(0x7f3dd40b1ff0 legacy=0x7f3dd413f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:24.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.316+0000 7f3dd92f3700 1 -- 172.21.15.36:0/4288647165 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd40afcf0 con 0x7f3dd40b1890 2022-06-17T12:52:24.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.316+0000 7f3dd92f3700 1 --2- 172.21.15.36:0/4288647165 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3dd40b1890 0x7f3dd413c9c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f3dcc001490 tx=0x7f3dc4006df0).ready entity=mon.2 client_cookie=dbe6158ccd8cd3cf server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:24.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.316+0000 7f3dd8af2700 1 -- 172.21.15.36:0/4288647165 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dcc001a00 con 0x7f3dd40b1890 2022-06-17T12:52:24.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.316+0000 7f3dd8af2700 1 -- 172.21.15.36:0/4288647165 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc4000e90 con 0x7f3dd40b1890 2022-06-17T12:52:24.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.316+0000 7f3dd8af2700 1 -- 172.21.15.36:0/4288647165 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dc4012460 con 0x7f3dd40b1890 2022-06-17T12:52:24.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.316+0000 7f3ddaaf6700 1 -- 172.21.15.36:0/4288647165 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3dd40b1890 msgr2=0x7f3dd413c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:24.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.316+0000 7f3ddaaf6700 1 --2- 172.21.15.36:0/4288647165 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3dd40b1890 0x7f3dd413c9c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f3dcc001490 tx=0x7f3dc4006df0).stop 2022-06-17T12:52:24.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.316+0000 7f3ddaaf6700 1 -- 172.21.15.36:0/4288647165 shutdown_connections 2022-06-17T12:52:24.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.316+0000 7f3ddaaf6700 1 --2- 172.21.15.36:0/4288647165 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3dd40b1890 0x7f3dd413c9c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.316+0000 7f3ddaaf6700 1 -- 172.21.15.36:0/4288647165 >> 172.21.15.36:0/4288647165 conn(0x7f3dd40b65f0 msgr2=0x7f3dd40b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:24.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.316+0000 7f3ddaaf6700 1 -- 172.21.15.36:0/4288647165 shutdown_connections 2022-06-17T12:52:24.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.316+0000 7f3ddaaf6700 1 -- 172.21.15.36:0/4288647165 wait complete. 2022-06-17T12:52:24.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.320+0000 7f3ddaaf6700 1 Processor -- start 2022-06-17T12:52:24.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.320+0000 7f3ddaaf6700 1 -- start start 2022-06-17T12:52:24.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.320+0000 7f3ddaaf6700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3dd4060ac0 0x7f3dd412eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:24.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.320+0000 7f3ddaaf6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dd40b1890 0x7f3dd413e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:24.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.320+0000 7f3ddaaf6700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3dd40b1ff0 0x7f3dd413e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:24.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.320+0000 7f3ddaaf6700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd4063b00 con 0x7f3dd4060ac0 2022-06-17T12:52:24.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.320+0000 7f3ddaaf6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd413e5c0 con 0x7f3dd40b1890 2022-06-17T12:52:24.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.320+0000 7f3ddaaf6700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd412fad0 con 0x7f3dd40b1ff0 2022-06-17T12:52:24.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.324+0000 7f3dd9af4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3dd4060ac0 0x7f3dd412eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:24.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.324+0000 7f3dd9af4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3dd4060ac0 0x7f3dd412eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34924/0 (socket says 172.21.15.36:34924) 2022-06-17T12:52:24.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.324+0000 7f3dd9af4700 1 -- 172.21.15.36:0/1291504345 learned_addr learned my addr 172.21.15.36:0/1291504345 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:24.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.324+0000 7f3dd92f3700 1 --2- 172.21.15.36:0/1291504345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dd40b1890 0x7f3dd413e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:24.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.324+0000 7f3dda2f5700 1 --2- 172.21.15.36:0/1291504345 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3dd40b1ff0 0x7f3dd413e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:24.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.324+0000 7f3dd9af4700 1 -- 172.21.15.36:0/1291504345 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3dd40b1ff0 msgr2=0x7f3dd413e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:24.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.324+0000 7f3dd9af4700 1 --2- 172.21.15.36:0/1291504345 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3dd40b1ff0 0x7f3dd413e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.324+0000 7f3dd9af4700 1 -- 172.21.15.36:0/1291504345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dd40b1890 msgr2=0x7f3dd413e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:24.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.324+0000 7f3dd9af4700 1 --2- 172.21.15.36:0/1291504345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dd40b1890 0x7f3dd413e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.324+0000 7f3dd9af4700 1 -- 172.21.15.36:0/1291504345 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc400b180 con 0x7f3dd4060ac0 2022-06-17T12:52:24.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.324+0000 7f3dd92f3700 1 --2- 172.21.15.36:0/1291504345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dd40b1890 0x7f3dd413e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:52:24.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.324+0000 7f3dd9af4700 1 --2- 172.21.15.36:0/1291504345 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3dd4060ac0 0x7f3dd412eef0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f3dd0001110 tx=0x7f3dd0007b70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:24.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.324+0000 7f3dc2ffd700 1 -- 172.21.15.36:0/1291504345 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dd00106b0 con 0x7f3dd4060ac0 2022-06-17T12:52:24.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.324+0000 7f3dc2ffd700 1 -- 172.21.15.36:0/1291504345 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd0010810 con 0x7f3dd4060ac0 2022-06-17T12:52:24.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.324+0000 7f3ddaaf6700 1 -- 172.21.15.36:0/1291504345 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd4063190 con 0x7f3dd4060ac0 2022-06-17T12:52:24.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.324+0000 7f3dc2ffd700 1 -- 172.21.15.36:0/1291504345 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dd0010b70 con 0x7f3dd4060ac0 2022-06-17T12:52:24.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.324+0000 7f3ddaaf6700 1 -- 172.21.15.36:0/1291504345 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd40645e0 con 0x7f3dd4060ac0 2022-06-17T12:52:24.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.324+0000 7f3ddaaf6700 1 -- 172.21.15.36:0/1291504345 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dd40aff90 con 0x7f3dd4060ac0 2022-06-17T12:52:24.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.328+0000 7f3dc2ffd700 1 -- 172.21.15.36:0/1291504345 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f3dd0021070 con 0x7f3dd4060ac0 2022-06-17T12:52:24.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.328+0000 7f3dc2ffd700 1 --2- 172.21.15.36:0/1291504345 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3db4053af0 0x7f3db4055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:24.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.328+0000 7f3dc2ffd700 1 -- 172.21.15.36:0/1291504345 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3dd006ee20 con 0x7f3dd4060ac0 2022-06-17T12:52:24.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.328+0000 7f3dd92f3700 1 --2- 172.21.15.36:0/1291504345 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3db4053af0 0x7f3db4055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:24.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.328+0000 7f3dd92f3700 1 --2- 172.21.15.36:0/1291504345 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3db4053af0 0x7f3db4055fb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f3dc4018000 tx=0x7f3dc4014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:24.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.332+0000 7f3dc2ffd700 1 -- 172.21.15.36:0/1291504345 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3dd0045c20 con 0x7f3dd4060ac0 2022-06-17T12:52:24.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.500+0000 7f3ddaaf6700 1 -- 172.21.15.36:0/1291504345 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f3dd4057410 con 0x7f3dd4060ac0 2022-06-17T12:52:24.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.516+0000 7f3dc2ffd700 1 -- 172.21.15.36:0/1291504345 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v23) v1 ==== 184+0+0 (secure 0 0 0) 0x7f3dd0015020 con 0x7f3dd4060ac0 2022-06-17T12:52:24.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.520+0000 7f3ddaaf6700 1 -- 172.21.15.36:0/1291504345 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3db4053af0 msgr2=0x7f3db4055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:24.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.520+0000 7f3ddaaf6700 1 --2- 172.21.15.36:0/1291504345 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3db4053af0 0x7f3db4055fb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f3dc4018000 tx=0x7f3dc4014000).stop 2022-06-17T12:52:24.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.520+0000 7f3ddaaf6700 1 -- 172.21.15.36:0/1291504345 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3dd4060ac0 msgr2=0x7f3dd412eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:24.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.520+0000 7f3ddaaf6700 1 --2- 172.21.15.36:0/1291504345 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3dd4060ac0 0x7f3dd412eef0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f3dd0001110 tx=0x7f3dd0007b70).stop 2022-06-17T12:52:24.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.520+0000 7f3ddaaf6700 1 -- 172.21.15.36:0/1291504345 shutdown_connections 2022-06-17T12:52:24.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.520+0000 7f3ddaaf6700 1 --2- 172.21.15.36:0/1291504345 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3db4053af0 0x7f3db4055fb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.520+0000 7f3ddaaf6700 1 --2- 172.21.15.36:0/1291504345 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3dd40b1ff0 0x7f3dd413e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.520+0000 7f3ddaaf6700 1 --2- 172.21.15.36:0/1291504345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dd40b1890 0x7f3dd413e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.520+0000 7f3ddaaf6700 1 --2- 172.21.15.36:0/1291504345 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3dd4060ac0 0x7f3dd412eef0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.520+0000 7f3ddaaf6700 1 -- 172.21.15.36:0/1291504345 >> 172.21.15.36:0/1291504345 conn(0x7f3dd40b65f0 msgr2=0x7f3dd40b04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:24.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.520+0000 7f3ddaaf6700 1 -- 172.21.15.36:0/1291504345 shutdown_connections 2022-06-17T12:52:24.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.520+0000 7f3ddaaf6700 1 -- 172.21.15.36:0/1291504345 wait complete. 2022-06-17T12:52:24.516 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.1 2022-06-17T12:52:24.520 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:52:24.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9937968700 1 Processor -- start 2022-06-17T12:52:24.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9937968700 1 -- start start 2022-06-17T12:52:24.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9937968700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9930062270 0x7f9930142f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:24.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9937968700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99300b44b0 con 0x7f99300b0cb0 2022-06-17T12:52:24.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9937968700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9930062650 con 0x7f9930143e80 2022-06-17T12:52:24.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9937968700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99300b6650 con 0x7f9930062270 2022-06-17T12:52:24.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9934f03700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f99300b0cb0 0x7f9930143770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52462/0 (socket says 172.21.15.36:52462) 2022-06-17T12:52:24.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9934f03700 1 -- 172.21.15.36:0/862840594 learned_addr learned my addr 172.21.15.36:0/862840594 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:24.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9935704700 1 --2- 172.21.15.36:0/862840594 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9930062270 0x7f9930142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:24.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9927fff700 1 -- 172.21.15.36:0/862840594 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 795424367 0 0) 0x7f99300b44b0 con 0x7f99300b0cb0 2022-06-17T12:52:24.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9927fff700 1 -- 172.21.15.36:0/862840594 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9914003260 con 0x7f99300b0cb0 2022-06-17T12:52:24.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9927fff700 1 -- 172.21.15.36:0/862840594 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3244627701 0 0) 0x7f9914003260 con 0x7f99300b0cb0 2022-06-17T12:52:24.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9927fff700 1 -- 172.21.15.36:0/862840594 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9930062270 msgr2=0x7f9930142f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:24.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9927fff700 1 --2- 172.21.15.36:0/862840594 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9930062270 0x7f9930142f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9927fff700 1 -- 172.21.15.36:0/862840594 >> v1:172.21.15.70:6789/0 conn(0x7f9930143e80 legacy=0x7f9930146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:24.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9927fff700 1 -- 172.21.15.36:0/862840594 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99300b4210 con 0x7f99300b0cb0 2022-06-17T12:52:24.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9927fff700 1 -- 172.21.15.36:0/862840594 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f9920001660 con 0x7f99300b0cb0 2022-06-17T12:52:24.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9927fff700 1 -- 172.21.15.36:0/862840594 >> v1:172.21.15.36:6789/0 conn(0x7f99300b0cb0 legacy=0x7f9930143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:24.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9927fff700 1 --2- 172.21.15.36:0/862840594 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9914003780 0x7f9914003b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:24.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9927fff700 1 --2- 172.21.15.36:0/862840594 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9930062270 0x7f9914004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:24.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9927fff700 1 --2- 172.21.15.36:0/862840594 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f99140049f0 0x7f9914004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:24.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9927fff700 1 -- 172.21.15.36:0/862840594 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99300b6650 con 0x7f99140049f0 2022-06-17T12:52:24.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9927fff700 1 -- 172.21.15.36:0/862840594 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991400b750 con 0x7f9914003780 2022-06-17T12:52:24.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9927fff700 1 -- 172.21.15.36:0/862840594 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991400b890 con 0x7f9930062270 2022-06-17T12:52:24.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9935f05700 1 --2- 172.21.15.36:0/862840594 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f99140049f0 0x7f9914004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:24.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9934f03700 1 --2- 172.21.15.36:0/862840594 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9930062270 0x7f9914004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:24.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9935704700 1 --2- 172.21.15.36:0/862840594 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9914003780 0x7f9914003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:24.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9935f05700 1 -- 172.21.15.36:0/862840594 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9930062270 msgr2=0x7f9914004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:24.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9935f05700 1 --2- 172.21.15.36:0/862840594 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9930062270 0x7f9914004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9935f05700 1 -- 172.21.15.36:0/862840594 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9914003780 msgr2=0x7f9914003b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:24.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9935f05700 1 --2- 172.21.15.36:0/862840594 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9914003780 0x7f9914003b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.648+0000 7f9935f05700 1 -- 172.21.15.36:0/862840594 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9914003260 con 0x7f99140049f0 2022-06-17T12:52:24.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.652+0000 7f9935704700 1 --2- 172.21.15.36:0/862840594 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9914003780 0x7f9914003b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:52:24.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.652+0000 7f9935f05700 1 --2- 172.21.15.36:0/862840594 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f99140049f0 0x7f9914004e70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f9914004490 tx=0x7f9928007d60).ready entity=mon.0 client_cookie=1422ec0630dfe3b0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:24.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.652+0000 7f9927fff700 1 -- 172.21.15.36:0/862840594 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99280016c0 con 0x7f99140049f0 2022-06-17T12:52:24.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.652+0000 7f9927fff700 1 -- 172.21.15.36:0/862840594 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9928006890 con 0x7f99140049f0 2022-06-17T12:52:24.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.652+0000 7f9927fff700 1 -- 172.21.15.36:0/862840594 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9928006bf0 con 0x7f99140049f0 2022-06-17T12:52:24.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.652+0000 7f9937968700 1 -- 172.21.15.36:0/862840594 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f99140049f0 msgr2=0x7f9914004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:24.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.652+0000 7f9937968700 1 --2- 172.21.15.36:0/862840594 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f99140049f0 0x7f9914004e70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f9914004490 tx=0x7f9928007d60).stop 2022-06-17T12:52:24.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.652+0000 7f9935f05700 1 -- 172.21.15.36:0/862840594 reap_dead start 2022-06-17T12:52:24.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.652+0000 7f9937968700 1 -- 172.21.15.36:0/862840594 shutdown_connections 2022-06-17T12:52:24.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.652+0000 7f9937968700 1 -- 172.21.15.36:0/862840594 >> 172.21.15.36:0/862840594 conn(0x7f99300b9870 msgr2=0x7f99300b9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:24.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.652+0000 7f9937968700 1 -- 172.21.15.36:0/862840594 shutdown_connections 2022-06-17T12:52:24.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.652+0000 7f9937968700 1 -- 172.21.15.36:0/862840594 wait complete. 2022-06-17T12:52:24.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.652+0000 7f9937968700 1 Processor -- start 2022-06-17T12:52:24.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.652+0000 7f9937968700 1 -- start start 2022-06-17T12:52:24.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.656+0000 7f9937968700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99140049f0 0x7f99300af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:24.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.656+0000 7f9937968700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9930149400 0x7f99300b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:24.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.656+0000 7f9937968700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f99300ad840 0x7f993013d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:24.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.656+0000 7f9937968700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99300ae5f0 con 0x7f99300ad840 2022-06-17T12:52:24.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.656+0000 7f9937968700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99300ae0b0 con 0x7f99140049f0 2022-06-17T12:52:24.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.656+0000 7f9937968700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99300ae350 con 0x7f9930149400 2022-06-17T12:52:24.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.656+0000 7f9935f05700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f99300ad840 0x7f993013d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:24.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.656+0000 7f9935f05700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f99300ad840 0x7f993013d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34948/0 (socket says 172.21.15.36:34948) 2022-06-17T12:52:24.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.656+0000 7f9935f05700 1 -- 172.21.15.36:0/3170267042 learned_addr learned my addr 172.21.15.36:0/3170267042 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:24.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.656+0000 7f9934f03700 1 --2- 172.21.15.36:0/3170267042 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9930149400 0x7f99300b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:24.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.656+0000 7f9935704700 1 --2- 172.21.15.36:0/3170267042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99140049f0 0x7f99300af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:24.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.656+0000 7f9935f05700 1 -- 172.21.15.36:0/3170267042 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9930149400 msgr2=0x7f99300b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:24.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.656+0000 7f9935f05700 1 --2- 172.21.15.36:0/3170267042 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9930149400 0x7f99300b2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.656+0000 7f9935f05700 1 -- 172.21.15.36:0/3170267042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99140049f0 msgr2=0x7f99300af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:24.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.656+0000 7f9935f05700 1 --2- 172.21.15.36:0/3170267042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99140049f0 0x7f99300af3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.656+0000 7f9935f05700 1 -- 172.21.15.36:0/3170267042 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993013d860 con 0x7f99300ad840 2022-06-17T12:52:24.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.656+0000 7f9934f03700 1 --2- 172.21.15.36:0/3170267042 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9930149400 0x7f99300b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:52:24.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.656+0000 7f9935f05700 1 --2- 172.21.15.36:0/3170267042 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f99300ad840 0x7f993013d370 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f99300617d0 tx=0x7f9928015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:24.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.656+0000 7f99267fc700 1 -- 172.21.15.36:0/3170267042 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99280016c0 con 0x7f99300ad840 2022-06-17T12:52:24.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.656+0000 7f99267fc700 1 -- 172.21.15.36:0/3170267042 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992800c9d0 con 0x7f99300ad840 2022-06-17T12:52:24.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.656+0000 7f9937968700 1 -- 172.21.15.36:0/3170267042 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993013dab0 con 0x7f99300ad840 2022-06-17T12:52:24.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.656+0000 7f99267fc700 1 -- 172.21.15.36:0/3170267042 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9928006ea0 con 0x7f99300ad840 2022-06-17T12:52:24.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.656+0000 7f9937968700 1 -- 172.21.15.36:0/3170267042 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9930147000 con 0x7f99300ad840 2022-06-17T12:52:24.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.656+0000 7f9937968700 1 -- 172.21.15.36:0/3170267042 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99300b63b0 con 0x7f99300ad840 2022-06-17T12:52:24.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.660+0000 7f99267fc700 1 -- 172.21.15.36:0/3170267042 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f9928010070 con 0x7f99300ad840 2022-06-17T12:52:24.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.660+0000 7f99267fc700 1 --2- 172.21.15.36:0/3170267042 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9918053af0 0x7f9918055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:24.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.660+0000 7f99267fc700 1 -- 172.21.15.36:0/3170267042 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9928003070 con 0x7f99300ad840 2022-06-17T12:52:24.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.660+0000 7f9935704700 1 --2- 172.21.15.36:0/3170267042 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9918053af0 0x7f9918055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:24.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.660+0000 7f9935704700 1 --2- 172.21.15.36:0/3170267042 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9918053af0 0x7f9918055fb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f992c00b000 tx=0x7f992c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:24.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.664+0000 7f99267fc700 1 -- 172.21.15.36:0/3170267042 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9928043b20 con 0x7f99300ad840 2022-06-17T12:52:24.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.836+0000 7f9937968700 1 -- 172.21.15.36:0/3170267042 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f9930057410 con 0x7f99300ad840 2022-06-17T12:52:24.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.844+0000 7f99267fc700 1 -- 172.21.15.36:0/3170267042 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v24) v1 ==== 184+0+0 (secure 0 0 0) 0x7f9928047140 con 0x7f99300ad840 2022-06-17T12:52:24.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.844+0000 7f9937968700 1 -- 172.21.15.36:0/3170267042 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9918053af0 msgr2=0x7f9918055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:24.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.844+0000 7f9937968700 1 --2- 172.21.15.36:0/3170267042 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9918053af0 0x7f9918055fb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f992c00b000 tx=0x7f992c008040).stop 2022-06-17T12:52:24.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.844+0000 7f9937968700 1 -- 172.21.15.36:0/3170267042 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f99300ad840 msgr2=0x7f993013d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:24.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.844+0000 7f9937968700 1 --2- 172.21.15.36:0/3170267042 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f99300ad840 0x7f993013d370 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f99300617d0 tx=0x7f9928015000).stop 2022-06-17T12:52:24.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.844+0000 7f9937968700 1 -- 172.21.15.36:0/3170267042 shutdown_connections 2022-06-17T12:52:24.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.844+0000 7f9937968700 1 --2- 172.21.15.36:0/3170267042 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9918053af0 0x7f9918055fb0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.844+0000 7f9937968700 1 --2- 172.21.15.36:0/3170267042 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f99300ad840 0x7f993013d370 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.844+0000 7f9937968700 1 --2- 172.21.15.36:0/3170267042 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9930149400 0x7f99300b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.844+0000 7f9937968700 1 --2- 172.21.15.36:0/3170267042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99140049f0 0x7f99300af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.844+0000 7f9937968700 1 -- 172.21.15.36:0/3170267042 >> 172.21.15.36:0/3170267042 conn(0x7f99300b9870 msgr2=0x7f99300b9140 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:24.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.844+0000 7f9937968700 1 -- 172.21.15.36:0/3170267042 shutdown_connections 2022-06-17T12:52:24.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.844+0000 7f9937968700 1 -- 172.21.15.36:0/3170267042 wait complete. 2022-06-17T12:52:24.835 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.2 2022-06-17T12:52:24.848 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:52:24.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febd724d700 1 Processor -- start 2022-06-17T12:52:24.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febd724d700 1 -- start start 2022-06-17T12:52:24.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febd724d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7febd0060ac0 0x7febd014aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:24.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febd724d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febd014b670 0x7febd01477c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:24.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febd724d700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd0060ea0 con 0x7febd00b3730 2022-06-17T12:52:24.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febd724d700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd00af7b0 con 0x7febd0060ac0 2022-06-17T12:52:24.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febd724d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd00afa50 con 0x7febd014b670 2022-06-17T12:52:24.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febcffff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7febd00b3730 0x7febd014af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52484/0 (socket says 172.21.15.36:52484) 2022-06-17T12:52:24.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febcffff700 1 -- 172.21.15.36:0/103388923 learned_addr learned my addr 172.21.15.36:0/103388923 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:24.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febd4fe9700 1 --2- 172.21.15.36:0/103388923 >> v2:172.21.15.36:3300/0 conn(0x7febd0060ac0 0x7febd014aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:24.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febd57ea700 1 --2- 172.21.15.36:0/103388923 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febd014b670 0x7febd01477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:24.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febcf7fe700 1 -- 172.21.15.36:0/103388923 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1965092129 0 0) 0x7febd0060ea0 con 0x7febd00b3730 2022-06-17T12:52:24.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febcf7fe700 1 -- 172.21.15.36:0/103388923 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febbc003260 con 0x7febd00b3730 2022-06-17T12:52:24.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febd4fe9700 1 -- 172.21.15.36:0/103388923 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febd014b670 msgr2=0x7febd01477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:24.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febd4fe9700 1 --2- 172.21.15.36:0/103388923 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febd014b670 0x7febd01477c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febd4fe9700 1 -- 172.21.15.36:0/103388923 >> v1:172.21.15.36:6789/0 conn(0x7febd00b3730 legacy=0x7febd014af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:24.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febd4fe9700 1 -- 172.21.15.36:0/103388923 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd00afcf0 con 0x7febd0060ac0 2022-06-17T12:52:24.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febd4fe9700 1 --2- 172.21.15.36:0/103388923 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febd0060ac0 0x7febd014aa70 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7febc0001800 tx=0x7febc4006df0).ready entity=mon.0 client_cookie=19722ab446db116e server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:24.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febcf7fe700 1 -- 172.21.15.36:0/103388923 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febc0001660 con 0x7febd0060ac0 2022-06-17T12:52:24.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febcf7fe700 1 -- 172.21.15.36:0/103388923 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc4000e90 con 0x7febd0060ac0 2022-06-17T12:52:24.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febcf7fe700 1 -- 172.21.15.36:0/103388923 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febc4011400 con 0x7febd0060ac0 2022-06-17T12:52:24.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febd724d700 1 -- 172.21.15.36:0/103388923 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febd0060ac0 msgr2=0x7febd014aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:24.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febd724d700 1 --2- 172.21.15.36:0/103388923 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febd0060ac0 0x7febd014aa70 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7febc0001800 tx=0x7febc4006df0).stop 2022-06-17T12:52:24.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febd724d700 1 -- 172.21.15.36:0/103388923 shutdown_connections 2022-06-17T12:52:24.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febd724d700 1 --2- 172.21.15.36:0/103388923 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febd014b670 0x7febd01477c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febd724d700 1 --2- 172.21.15.36:0/103388923 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febd0060ac0 0x7febd014aa70 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febd724d700 1 -- 172.21.15.36:0/103388923 >> 172.21.15.36:0/103388923 conn(0x7febd00b6870 msgr2=0x7febd00b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:24.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febd724d700 1 -- 172.21.15.36:0/103388923 shutdown_connections 2022-06-17T12:52:24.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.968+0000 7febd724d700 1 -- 172.21.15.36:0/103388923 wait complete. 2022-06-17T12:52:24.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febd724d700 1 Processor -- start 2022-06-17T12:52:24.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febd724d700 1 -- start start 2022-06-17T12:52:24.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febd724d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febd0060ac0 0x7febd00b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:24.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febd724d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febd00b3730 0x7febd0063fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:24.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febd724d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febd014b670 0x7febd0064490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:24.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febd724d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd0143eb0 con 0x7febd014b670 2022-06-17T12:52:24.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febd724d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd0145d80 con 0x7febd0060ac0 2022-06-17T12:52:24.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febd724d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd01461e0 con 0x7febd00b3730 2022-06-17T12:52:24.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febd57ea700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febd014b670 0x7febd0064490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:24.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febd57ea700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febd014b670 0x7febd0064490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34962/0 (socket says 172.21.15.36:34962) 2022-06-17T12:52:24.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febd57ea700 1 -- 172.21.15.36:0/3691969275 learned_addr learned my addr 172.21.15.36:0/3691969275 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:24.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febd4fe9700 1 --2- 172.21.15.36:0/3691969275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febd0060ac0 0x7febd00b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:24.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febcffff700 1 --2- 172.21.15.36:0/3691969275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febd00b3730 0x7febd0063fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:24.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febd57ea700 1 -- 172.21.15.36:0/3691969275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febd00b3730 msgr2=0x7febd0063fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:24.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febd57ea700 1 --2- 172.21.15.36:0/3691969275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febd00b3730 0x7febd0063fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febd57ea700 1 -- 172.21.15.36:0/3691969275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febd0060ac0 msgr2=0x7febd00b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:24.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febd57ea700 1 --2- 172.21.15.36:0/3691969275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febd0060ac0 0x7febd00b1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:24.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febd57ea700 1 -- 172.21.15.36:0/3691969275 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc400b180 con 0x7febd014b670 2022-06-17T12:52:24.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febd4fe9700 1 --2- 172.21.15.36:0/3691969275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febd0060ac0 0x7febd00b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:52:24.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febd57ea700 1 --2- 172.21.15.36:0/3691969275 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febd014b670 0x7febd0064490 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7febc8006e00 tx=0x7febc80091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:24.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febcdffb700 1 -- 172.21.15.36:0/3691969275 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febc8012600 con 0x7febd014b670 2022-06-17T12:52:24.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febcdffb700 1 -- 172.21.15.36:0/3691969275 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc8012760 con 0x7febd014b670 2022-06-17T12:52:24.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febd724d700 1 -- 172.21.15.36:0/3691969275 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd0143c10 con 0x7febd014b670 2022-06-17T12:52:24.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febcdffb700 1 -- 172.21.15.36:0/3691969275 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febc8012ac0 con 0x7febd014b670 2022-06-17T12:52:24.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.972+0000 7febd724d700 1 -- 172.21.15.36:0/3691969275 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd01400e0 con 0x7febd014b670 2022-06-17T12:52:24.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.976+0000 7febd724d700 1 -- 172.21.15.36:0/3691969275 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd0145be0 con 0x7febd014b670 2022-06-17T12:52:24.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.976+0000 7febcdffb700 1 -- 172.21.15.36:0/3691969275 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7febc8022070 con 0x7febd014b670 2022-06-17T12:52:24.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.976+0000 7febcdffb700 1 --2- 172.21.15.36:0/3691969275 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7febb0053ad0 0x7febb0055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:24.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.976+0000 7febcdffb700 1 -- 172.21.15.36:0/3691969275 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7febc806c530 con 0x7febd014b670 2022-06-17T12:52:24.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.976+0000 7febd4fe9700 1 --2- 172.21.15.36:0/3691969275 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7febb0053ad0 0x7febb0055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:24.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.976+0000 7febd4fe9700 1 --2- 172.21.15.36:0/3691969275 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7febb0053ad0 0x7febb0055f90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7febc4019000 tx=0x7febd00afe80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:24.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:24.980+0000 7febcdffb700 1 -- 172.21.15.36:0/3691969275 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7febd0145be0 con 0x7febd014b670 2022-06-17T12:52:25.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.144+0000 7febd724d700 1 -- 172.21.15.36:0/3691969275 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febd0145be0 con 0x7febd014b670 2022-06-17T12:52:25.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.144+0000 7febcdffb700 1 -- 172.21.15.36:0/3691969275 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2140 (secure 0 0 0) 0x7febd0145be0 con 0x7febd014b670 2022-06-17T12:52:25.133 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:52:25.133 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":30,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5420,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/3105680199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3105680199},{"type":"v1","addr":"172.21.15.70:6833","nonce":3105680199}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":29}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:52:22.253232+0000","modified":"2022-06-17T12:52:23.257219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6054},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6054":{"gid":6054,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/4091208534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":4091208534},{"type":"v1","addr":"172.21.15.36:6835","nonce":4091208534}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-06-17T12:52:25.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.148+0000 7febd724d700 1 -- 172.21.15.36:0/3691969275 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7febb0053ad0 msgr2=0x7febb0055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:25.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.148+0000 7febd724d700 1 --2- 172.21.15.36:0/3691969275 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7febb0053ad0 0x7febb0055f90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7febc4019000 tx=0x7febd00afe80).stop 2022-06-17T12:52:25.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.148+0000 7febd724d700 1 -- 172.21.15.36:0/3691969275 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febd014b670 msgr2=0x7febd0064490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:25.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.148+0000 7febd724d700 1 --2- 172.21.15.36:0/3691969275 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febd014b670 0x7febd0064490 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7febc8006e00 tx=0x7febc80091d0).stop 2022-06-17T12:52:25.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.148+0000 7febd724d700 1 -- 172.21.15.36:0/3691969275 shutdown_connections 2022-06-17T12:52:25.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.148+0000 7febd724d700 1 --2- 172.21.15.36:0/3691969275 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7febb0053ad0 0x7febb0055f90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:25.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.148+0000 7febd724d700 1 --2- 172.21.15.36:0/3691969275 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febd014b670 0x7febd0064490 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:25.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.148+0000 7febd724d700 1 --2- 172.21.15.36:0/3691969275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febd00b3730 0x7febd0063fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:25.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.148+0000 7febd724d700 1 --2- 172.21.15.36:0/3691969275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febd0060ac0 0x7febd00b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:25.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.148+0000 7febd724d700 1 -- 172.21.15.36:0/3691969275 >> 172.21.15.36:0/3691969275 conn(0x7febd00b6870 msgr2=0x7febd014c030 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:25.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.148+0000 7febd724d700 1 -- 172.21.15.36:0/3691969275 shutdown_connections 2022-06-17T12:52:25.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.148+0000 7febd724d700 1 -- 172.21.15.36:0/3691969275 wait complete. 2022-06-17T12:52:25.144 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 30 2022-06-17T12:52:25.348 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:52:22.253232+0000', 'modified': '2022-06-17T12:52:23.257219+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6054}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6054': {'gid': 6054, 'name': 'a', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.36:6835/4091208534', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 4091208534}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 4091208534}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:52:25.349 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:52:25.350 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:52:25.474 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":6054,"want_state":"up:active","state":"up:active","rank_uptime":3.222337064,"mdsmap_epoch":30,"osdmap_epoch":65,"osdmap_epoch_barrier":65,"uptime":7.6145364210000004} 2022-06-17T12:52:25.476 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 6054, "mdsmap_epoch": 30, "osdmap_epoch": 65, "osdmap_epoch_barrier": 65, "rank_uptime": 3.222337064, "state": "up:active", "uptime": 7.614536421, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:52:25.477 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:52:25.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.596+0000 7ff489e7c700 1 Processor -- start 2022-06-17T12:52:25.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.596+0000 7ff489e7c700 1 -- start start 2022-06-17T12:52:25.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.596+0000 7ff489e7c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4840aeea0 0x7ff484064230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:25.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.596+0000 7ff489e7c700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4840af280 con 0x7ff4840af930 2022-06-17T12:52:25.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.596+0000 7ff489e7c700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4840b14f0 con 0x7ff4840afe30 2022-06-17T12:52:25.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.596+0000 7ff489e7c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff484067560 con 0x7ff4840aeea0 2022-06-17T12:52:25.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.596+0000 7ff483fff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7ff4840af930 0x7ff4840b19e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52496/0 (socket says 172.21.15.36:52496) 2022-06-17T12:52:25.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.596+0000 7ff483fff700 1 -- 172.21.15.36:0/209022855 learned_addr learned my addr 172.21.15.36:0/209022855 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:25.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.596+0000 7ff488e7a700 1 --2- 172.21.15.36:0/209022855 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4840aeea0 0x7ff484064230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:25.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.596+0000 7ff4837fe700 1 -- 172.21.15.36:0/209022855 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 221983621 0 0) 0x7ff4840af280 con 0x7ff4840af930 2022-06-17T12:52:25.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.596+0000 7ff4837fe700 1 -- 172.21.15.36:0/209022855 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff470003260 con 0x7ff4840af930 2022-06-17T12:52:25.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.596+0000 7ff4837fe700 1 -- 172.21.15.36:0/209022855 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3101382907 0 0) 0x7ff4840b14f0 con 0x7ff4840afe30 2022-06-17T12:52:25.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.596+0000 7ff4837fe700 1 -- 172.21.15.36:0/209022855 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4840af280 con 0x7ff4840afe30 2022-06-17T12:52:25.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.596+0000 7ff4837fe700 1 -- 172.21.15.36:0/209022855 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2621107241 0 0) 0x7ff470003260 con 0x7ff4840af930 2022-06-17T12:52:25.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.596+0000 7ff4837fe700 1 -- 172.21.15.36:0/209022855 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4840aeea0 msgr2=0x7ff484064230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:25.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.596+0000 7ff4837fe700 1 --2- 172.21.15.36:0/209022855 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4840aeea0 0x7ff484064230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:25.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.596+0000 7ff4837fe700 1 -- 172.21.15.36:0/209022855 >> v1:172.21.15.70:6789/0 conn(0x7ff4840afe30 legacy=0x7ff4840ab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:25.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.596+0000 7ff4837fe700 1 -- 172.21.15.36:0/209022855 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4840b20f0 con 0x7ff4840af930 2022-06-17T12:52:25.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.596+0000 7ff4837fe700 1 -- 172.21.15.36:0/209022855 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7ff474001660 con 0x7ff4840af930 2022-06-17T12:52:25.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff4837fe700 1 -- 172.21.15.36:0/209022855 >> v1:172.21.15.36:6789/0 conn(0x7ff4840af930 legacy=0x7ff4840b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:25.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff4837fe700 1 --2- 172.21.15.36:0/209022855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff470003910 0x7ff470003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:25.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff4837fe700 1 --2- 172.21.15.36:0/209022855 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff470004500 0x7ff470004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:25.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff4837fe700 1 --2- 172.21.15.36:0/209022855 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4840aeea0 0x7ff470004e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:25.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff4837fe700 1 -- 172.21.15.36:0/209022855 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff484067560 con 0x7ff470004500 2022-06-17T12:52:25.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff4837fe700 1 -- 172.21.15.36:0/209022855 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47000b800 con 0x7ff470003910 2022-06-17T12:52:25.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff4837fe700 1 -- 172.21.15.36:0/209022855 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47000b990 con 0x7ff4840aeea0 2022-06-17T12:52:25.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff483fff700 1 --2- 172.21.15.36:0/209022855 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff470004500 0x7ff470004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:25.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff488e7a700 1 --2- 172.21.15.36:0/209022855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff470003910 0x7ff470003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:25.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff48967b700 1 --2- 172.21.15.36:0/209022855 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4840aeea0 0x7ff470004e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:25.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff483fff700 1 -- 172.21.15.36:0/209022855 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4840aeea0 msgr2=0x7ff470004e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:25.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff483fff700 1 --2- 172.21.15.36:0/209022855 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4840aeea0 0x7ff470004e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:25.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff483fff700 1 -- 172.21.15.36:0/209022855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff470003910 msgr2=0x7ff470003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:25.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff483fff700 1 --2- 172.21.15.36:0/209022855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff470003910 0x7ff470003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:25.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff483fff700 1 -- 172.21.15.36:0/209022855 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff470003260 con 0x7ff470004500 2022-06-17T12:52:25.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff483fff700 1 --2- 172.21.15.36:0/209022855 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff470004500 0x7ff470004980 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7ff4700054d0 tx=0x7ff474001d00).ready entity=mon.0 client_cookie=2e2811b2d8a46963 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:25.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff4837fe700 1 -- 172.21.15.36:0/209022855 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff474001d70 con 0x7ff470004500 2022-06-17T12:52:25.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff4837fe700 1 -- 172.21.15.36:0/209022855 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4740070d0 con 0x7ff470004500 2022-06-17T12:52:25.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff4837fe700 1 -- 172.21.15.36:0/209022855 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff474007430 con 0x7ff470004500 2022-06-17T12:52:25.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff489e7c700 1 -- 172.21.15.36:0/209022855 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff470004500 msgr2=0x7ff470004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:25.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff489e7c700 1 --2- 172.21.15.36:0/209022855 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff470004500 0x7ff470004980 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7ff4700054d0 tx=0x7ff474001d00).stop 2022-06-17T12:52:25.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff48967b700 1 -- 172.21.15.36:0/209022855 reap_dead start 2022-06-17T12:52:25.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff489e7c700 1 -- 172.21.15.36:0/209022855 shutdown_connections 2022-06-17T12:52:25.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff489e7c700 1 -- 172.21.15.36:0/209022855 >> 172.21.15.36:0/209022855 conn(0x7ff48405f770 msgr2=0x7ff48405fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:25.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff489e7c700 1 -- 172.21.15.36:0/209022855 shutdown_connections 2022-06-17T12:52:25.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.600+0000 7ff489e7c700 1 -- 172.21.15.36:0/209022855 wait complete. 2022-06-17T12:52:25.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff489e7c700 1 Processor -- start 2022-06-17T12:52:25.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff489e7c700 1 -- start start 2022-06-17T12:52:25.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff489e7c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff470004500 0x7ff4840a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:25.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff489e7c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4840a2850 0x7ff4840a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:25.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff489e7c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4841535f0 0x7ff484153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:25.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff489e7c700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48414ff30 con 0x7ff470004500 2022-06-17T12:52:25.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff489e7c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4840a2440 con 0x7ff4840a2850 2022-06-17T12:52:25.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff489e7c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff484150b40 con 0x7ff4841535f0 2022-06-17T12:52:25.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff488e7a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff470004500 0x7ff4840a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:25.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff488e7a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff470004500 0x7ff4840a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34980/0 (socket says 172.21.15.36:34980) 2022-06-17T12:52:25.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff488e7a700 1 -- 172.21.15.36:0/4027673284 learned_addr learned my addr 172.21.15.36:0/4027673284 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:25.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff483fff700 1 --2- 172.21.15.36:0/4027673284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4840a2850 0x7ff4840a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:25.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff48967b700 1 --2- 172.21.15.36:0/4027673284 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4841535f0 0x7ff484153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:25.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff488e7a700 1 -- 172.21.15.36:0/4027673284 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4841535f0 msgr2=0x7ff484153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:25.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff488e7a700 1 --2- 172.21.15.36:0/4027673284 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4841535f0 0x7ff484153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:25.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff488e7a700 1 -- 172.21.15.36:0/4027673284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4840a2850 msgr2=0x7ff4840a6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:25.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff488e7a700 1 --2- 172.21.15.36:0/4027673284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4840a2850 0x7ff4840a6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:25.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff488e7a700 1 -- 172.21.15.36:0/4027673284 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff484150de0 con 0x7ff470004500 2022-06-17T12:52:25.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff488e7a700 1 --2- 172.21.15.36:0/4027673284 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff470004500 0x7ff4840a5c90 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7ff47800b000 tx=0x7ff478006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:25.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff481ffb700 1 -- 172.21.15.36:0/4027673284 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff478011e70 con 0x7ff470004500 2022-06-17T12:52:25.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff481ffb700 1 -- 172.21.15.36:0/4027673284 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47800f940 con 0x7ff470004500 2022-06-17T12:52:25.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff489e7c700 1 -- 172.21.15.36:0/4027673284 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4840a6670 con 0x7ff470004500 2022-06-17T12:52:25.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff481ffb700 1 -- 172.21.15.36:0/4027673284 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff47800fca0 con 0x7ff470004500 2022-06-17T12:52:25.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff489e7c700 1 -- 172.21.15.36:0/4027673284 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4840a2060 con 0x7ff470004500 2022-06-17T12:52:25.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.604+0000 7ff489e7c700 1 -- 172.21.15.36:0/4027673284 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff484150f70 con 0x7ff470004500 2022-06-17T12:52:25.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.608+0000 7ff481ffb700 1 -- 172.21.15.36:0/4027673284 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff478014440 con 0x7ff470004500 2022-06-17T12:52:25.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.608+0000 7ff481ffb700 1 --2- 172.21.15.36:0/4027673284 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff464053920 0x7ff464055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:25.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.608+0000 7ff483fff700 1 --2- 172.21.15.36:0/4027673284 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff464053920 0x7ff464055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:25.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.608+0000 7ff481ffb700 1 -- 172.21.15.36:0/4027673284 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff47806d000 con 0x7ff470004500 2022-06-17T12:52:25.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.608+0000 7ff483fff700 1 --2- 172.21.15.36:0/4027673284 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff464053920 0x7ff464055de0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7ff474017000 tx=0x7ff474015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:25.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.616+0000 7ff481ffb700 1 -- 172.21.15.36:0/4027673284 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff478043da0 con 0x7ff470004500 2022-06-17T12:52:25.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.744+0000 7ff489e7c700 1 -- 172.21.15.36:0/4027673284 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff484057410 con 0x7ff470004500 2022-06-17T12:52:25.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.748+0000 7ff481ffb700 1 -- 172.21.15.36:0/4027673284 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16274 (secure 0 0 0) 0x7ff478018070 con 0x7ff470004500 2022-06-17T12:52:25.734 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:52:25.734 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":65,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:52:23.256104+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:52:20.693971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2022-06-17T12:52:21.830992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:52:25.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.748+0000 7ff489e7c700 1 -- 172.21.15.36:0/4027673284 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff464053920 msgr2=0x7ff464055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:25.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.748+0000 7ff489e7c700 1 --2- 172.21.15.36:0/4027673284 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff464053920 0x7ff464055de0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7ff474017000 tx=0x7ff474015000).stop 2022-06-17T12:52:25.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.748+0000 7ff489e7c700 1 -- 172.21.15.36:0/4027673284 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff470004500 msgr2=0x7ff4840a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:25.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.748+0000 7ff489e7c700 1 --2- 172.21.15.36:0/4027673284 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff470004500 0x7ff4840a5c90 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7ff47800b000 tx=0x7ff478006df0).stop 2022-06-17T12:52:25.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.748+0000 7ff489e7c700 1 -- 172.21.15.36:0/4027673284 shutdown_connections 2022-06-17T12:52:25.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.748+0000 7ff489e7c700 1 --2- 172.21.15.36:0/4027673284 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff464053920 0x7ff464055de0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:25.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.748+0000 7ff489e7c700 1 --2- 172.21.15.36:0/4027673284 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4841535f0 0x7ff484153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:25.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.748+0000 7ff489e7c700 1 --2- 172.21.15.36:0/4027673284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4840a2850 0x7ff4840a6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:25.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.748+0000 7ff489e7c700 1 --2- 172.21.15.36:0/4027673284 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff470004500 0x7ff4840a5c90 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:25.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.748+0000 7ff489e7c700 1 -- 172.21.15.36:0/4027673284 >> 172.21.15.36:0/4027673284 conn(0x7ff48405f770 msgr2=0x7ff4841416c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:25.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.748+0000 7ff489e7c700 1 -- 172.21.15.36:0/4027673284 shutdown_connections 2022-06-17T12:52:25.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.748+0000 7ff489e7c700 1 -- 172.21.15.36:0/4027673284 wait complete. 2022-06-17T12:52:25.748 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:52:20.693971+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:52:21.830992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:52:25.749 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:52:25.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.872+0000 7f2255f70700 1 Processor -- start 2022-06-17T12:52:25.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.872+0000 7f2255f70700 1 -- start start 2022-06-17T12:52:25.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.876+0000 7f2255f70700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22500b0420 0x7f225005f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:25.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.876+0000 7f2255f70700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22500b10a0 con 0x7f225005fc60 2022-06-17T12:52:25.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.876+0000 7f2255f70700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2250144140 con 0x7f2250060260 2022-06-17T12:52:25.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.876+0000 7f2255f70700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2250147570 con 0x7f22500b0420 2022-06-17T12:52:25.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.876+0000 7f224effd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f225005fc60 0x7f2250146750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52516/0 (socket says 172.21.15.36:52516) 2022-06-17T12:52:25.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.876+0000 7f224effd700 1 -- 172.21.15.36:0/3011920895 learned_addr learned my addr 172.21.15.36:0/3011920895 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:25.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.876+0000 7f224f7fe700 1 --2- 172.21.15.36:0/3011920895 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22500b0420 0x7f225005f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:25.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.876+0000 7f224e7fc700 1 -- 172.21.15.36:0/3011920895 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1832880236 0 0) 0x7f22500b10a0 con 0x7f225005fc60 2022-06-17T12:52:25.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.876+0000 7f224e7fc700 1 -- 172.21.15.36:0/3011920895 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2230003260 con 0x7f225005fc60 2022-06-17T12:52:25.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.876+0000 7f224e7fc700 1 -- 172.21.15.36:0/3011920895 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3486665373 0 0) 0x7f2230003260 con 0x7f225005fc60 2022-06-17T12:52:25.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.876+0000 7f224f7fe700 1 -- 172.21.15.36:0/3011920895 >> v1:172.21.15.70:6789/0 conn(0x7f2250060260 legacy=0x7f2250146e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:25.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.876+0000 7f224f7fe700 1 -- 172.21.15.36:0/3011920895 >> v1:172.21.15.36:6789/0 conn(0x7f225005fc60 legacy=0x7f2250146750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:25.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.876+0000 7f224f7fe700 1 -- 172.21.15.36:0/3011920895 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2250143ea0 con 0x7f22500b0420 2022-06-17T12:52:25.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.876+0000 7f224f7fe700 1 --2- 172.21.15.36:0/3011920895 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22500b0420 0x7f225005f770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f2240001800 tx=0x7f2238006df0).ready entity=mon.2 client_cookie=3cdc57e7d8ade2ca server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:25.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.876+0000 7f224e7fc700 1 -- 172.21.15.36:0/3011920895 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2240001660 con 0x7f22500b0420 2022-06-17T12:52:25.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.876+0000 7f224e7fc700 1 -- 172.21.15.36:0/3011920895 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2238000e90 con 0x7f22500b0420 2022-06-17T12:52:25.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.876+0000 7f224e7fc700 1 -- 172.21.15.36:0/3011920895 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f223800d4a0 con 0x7f22500b0420 2022-06-17T12:52:25.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.876+0000 7f2255f70700 1 -- 172.21.15.36:0/3011920895 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22500b0420 msgr2=0x7f225005f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:25.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.876+0000 7f2255f70700 1 --2- 172.21.15.36:0/3011920895 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22500b0420 0x7f225005f770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f2240001800 tx=0x7f2238006df0).stop 2022-06-17T12:52:25.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.876+0000 7f2255f70700 1 -- 172.21.15.36:0/3011920895 shutdown_connections 2022-06-17T12:52:25.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.876+0000 7f2255f70700 1 --2- 172.21.15.36:0/3011920895 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22500b0420 0x7f225005f770 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:25.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.876+0000 7f2255f70700 1 -- 172.21.15.36:0/3011920895 >> 172.21.15.36:0/3011920895 conn(0x7f225005c550 msgr2=0x7f225005c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:25.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.876+0000 7f2255f70700 1 -- 172.21.15.36:0/3011920895 shutdown_connections 2022-06-17T12:52:25.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.876+0000 7f2255f70700 1 -- 172.21.15.36:0/3011920895 wait complete. 2022-06-17T12:52:25.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f2255f70700 1 Processor -- start 2022-06-17T12:52:25.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f2255f70700 1 -- start start 2022-06-17T12:52:25.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f2255f70700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f225005fc60 0x7f22500b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:25.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f2255f70700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2250060260 0x7f2250148440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:25.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f2255f70700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f225014b260 0x7f225013c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:25.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f2255f70700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22500aa4c0 con 0x7f225005fc60 2022-06-17T12:52:25.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f2255f70700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22500aa760 con 0x7f225014b260 2022-06-17T12:52:25.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f2255f70700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2250153140 con 0x7f2250060260 2022-06-17T12:52:25.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f224f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f225005fc60 0x7f22500b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:25.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f224f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f225005fc60 0x7f22500b1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:34992/0 (socket says 172.21.15.36:34992) 2022-06-17T12:52:25.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f224f7fe700 1 -- 172.21.15.36:0/4155029727 learned_addr learned my addr 172.21.15.36:0/4155029727 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:25.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f224effd700 1 --2- 172.21.15.36:0/4155029727 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2250060260 0x7f2250148440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:25.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f224ffff700 1 --2- 172.21.15.36:0/4155029727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f225014b260 0x7f225013c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:25.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f224f7fe700 1 -- 172.21.15.36:0/4155029727 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2250060260 msgr2=0x7f2250148440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:25.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f224f7fe700 1 --2- 172.21.15.36:0/4155029727 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2250060260 0x7f2250148440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:25.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f224f7fe700 1 -- 172.21.15.36:0/4155029727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f225014b260 msgr2=0x7f225013c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:25.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f224f7fe700 1 --2- 172.21.15.36:0/4155029727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f225014b260 0x7f225013c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:25.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f224f7fe700 1 -- 172.21.15.36:0/4155029727 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223800b180 con 0x7f225005fc60 2022-06-17T12:52:25.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f224ffff700 1 --2- 172.21.15.36:0/4155029727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f225014b260 0x7f225013c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:52:25.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f224f7fe700 1 --2- 172.21.15.36:0/4155029727 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f225005fc60 0x7f22500b1720 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f2240001800 tx=0x7f2238006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:25.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f224cff9700 1 -- 172.21.15.36:0/4155029727 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2238012b00 con 0x7f225005fc60 2022-06-17T12:52:25.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f224cff9700 1 -- 172.21.15.36:0/4155029727 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2238012c60 con 0x7f225005fc60 2022-06-17T12:52:25.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f2255f70700 1 -- 172.21.15.36:0/4155029727 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f225013cb10 con 0x7f225005fc60 2022-06-17T12:52:25.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f224cff9700 1 -- 172.21.15.36:0/4155029727 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f223801d400 con 0x7f225005fc60 2022-06-17T12:52:25.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.880+0000 7f2255f70700 1 -- 172.21.15.36:0/4155029727 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f225013cf60 con 0x7f225005fc60 2022-06-17T12:52:25.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.884+0000 7f224cff9700 1 -- 172.21.15.36:0/4155029727 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2238024030 con 0x7f225005fc60 2022-06-17T12:52:25.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.884+0000 7f2255f70700 1 -- 172.21.15.36:0/4155029727 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2250143bc0 con 0x7f225005fc60 2022-06-17T12:52:25.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.884+0000 7f224cff9700 1 --2- 172.21.15.36:0/4155029727 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2234053ad0 0x7f2234055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:25.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.884+0000 7f224effd700 1 --2- 172.21.15.36:0/4155029727 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2234053ad0 0x7f2234055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:25.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.884+0000 7f224cff9700 1 -- 172.21.15.36:0/4155029727 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f223806c7b0 con 0x7f225005fc60 2022-06-17T12:52:25.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.884+0000 7f224effd700 1 --2- 172.21.15.36:0/4155029727 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2234053ad0 0x7f2234055f90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f224000d000 tx=0x7f2240001e90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:25.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:25.888+0000 7f224cff9700 1 -- 172.21.15.36:0/4155029727 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f22380435b0 con 0x7f225005fc60 2022-06-17T12:52:26.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.016+0000 7f2255f70700 1 -- 172.21.15.36:0/4155029727 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f2250057410 con 0x7f225005fc60 2022-06-17T12:52:26.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.020+0000 7f224cff9700 1 -- 172.21.15.36:0/4155029727 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v65) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2238022070 con 0x7f225005fc60 2022-06-17T12:52:26.005 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:52:26.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.020+0000 7f2255f70700 1 -- 172.21.15.36:0/4155029727 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2234053ad0 msgr2=0x7f2234055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.020+0000 7f2255f70700 1 --2- 172.21.15.36:0/4155029727 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2234053ad0 0x7f2234055f90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f224000d000 tx=0x7f2240001e90).stop 2022-06-17T12:52:26.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.020+0000 7f2255f70700 1 -- 172.21.15.36:0/4155029727 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f225005fc60 msgr2=0x7f22500b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.020+0000 7f2255f70700 1 --2- 172.21.15.36:0/4155029727 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f225005fc60 0x7f22500b1720 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f2240001800 tx=0x7f2238006df0).stop 2022-06-17T12:52:26.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.020+0000 7f2255f70700 1 -- 172.21.15.36:0/4155029727 shutdown_connections 2022-06-17T12:52:26.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.020+0000 7f2255f70700 1 --2- 172.21.15.36:0/4155029727 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2234053ad0 0x7f2234055f90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.020+0000 7f2255f70700 1 --2- 172.21.15.36:0/4155029727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f225014b260 0x7f225013c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.020+0000 7f2255f70700 1 --2- 172.21.15.36:0/4155029727 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2250060260 0x7f2250148440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.020+0000 7f2255f70700 1 --2- 172.21.15.36:0/4155029727 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f225005fc60 0x7f22500b1720 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.020+0000 7f2255f70700 1 -- 172.21.15.36:0/4155029727 >> 172.21.15.36:0/4155029727 conn(0x7f225005c550 msgr2=0x7f2250146660 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:26.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.020+0000 7f2255f70700 1 -- 172.21.15.36:0/4155029727 shutdown_connections 2022-06-17T12:52:26.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.020+0000 7f2255f70700 1 -- 172.21.15.36:0/4155029727 wait complete. 2022-06-17T12:52:26.018 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:52:26.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.144+0000 7f2e81889700 1 Processor -- start 2022-06-17T12:52:26.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.144+0000 7f2e81889700 1 -- start start 2022-06-17T12:52:26.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.144+0000 7f2e81889700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2e7c060920 0x7f2e7c067610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:26.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.144+0000 7f2e81889700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e7c067b00 con 0x7f2e7c0b1e00 2022-06-17T12:52:26.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.144+0000 7f2e81889700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e7c0ad020 con 0x7f2e7c13e8c0 2022-06-17T12:52:26.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.144+0000 7f2e81889700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7c0aff80 con 0x7f2e7c060920 2022-06-17T12:52:26.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.148+0000 7f2e7bfff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f2e7c0b1e00 0x7f2e7c0ad760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52532/0 (socket says 172.21.15.36:52532) 2022-06-17T12:52:26.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.148+0000 7f2e7bfff700 1 -- 172.21.15.36:0/944112454 learned_addr learned my addr 172.21.15.36:0/944112454 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:26.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.148+0000 7f2e80887700 1 --2- 172.21.15.36:0/944112454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2e7c060920 0x7f2e7c067610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:26.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.148+0000 7f2e7b7fe700 1 -- 172.21.15.36:0/944112454 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3497759924 0 0) 0x7f2e7c067b00 con 0x7f2e7c0b1e00 2022-06-17T12:52:26.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.148+0000 7f2e7b7fe700 1 -- 172.21.15.36:0/944112454 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e60003260 con 0x7f2e7c0b1e00 2022-06-17T12:52:26.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.148+0000 7f2e7b7fe700 1 -- 172.21.15.36:0/944112454 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 510633213 0 0) 0x7f2e7c0ad020 con 0x7f2e7c13e8c0 2022-06-17T12:52:26.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.148+0000 7f2e7b7fe700 1 -- 172.21.15.36:0/944112454 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e7c067b00 con 0x7f2e7c13e8c0 2022-06-17T12:52:26.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.148+0000 7f2e7b7fe700 1 -- 172.21.15.36:0/944112454 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1868774398 0 0) 0x7f2e60003260 con 0x7f2e7c0b1e00 2022-06-17T12:52:26.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.148+0000 7f2e80887700 1 -- 172.21.15.36:0/944112454 >> v1:172.21.15.70:6789/0 conn(0x7f2e7c13e8c0 legacy=0x7f2e7c13ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.148+0000 7f2e80887700 1 -- 172.21.15.36:0/944112454 >> v1:172.21.15.36:6789/0 conn(0x7f2e7c0b1e00 legacy=0x7f2e7c0ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.148+0000 7f2e80887700 1 -- 172.21.15.36:0/944112454 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7c0ad2c0 con 0x7f2e7c060920 2022-06-17T12:52:26.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.148+0000 7f2e80887700 1 --2- 172.21.15.36:0/944112454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2e7c060920 0x7f2e7c067610 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f2e68001110 tx=0x7f2e70006df0).ready entity=mon.2 client_cookie=c94be27784d5fc03 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:26.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.148+0000 7f2e7b7fe700 1 -- 172.21.15.36:0/944112454 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e68001660 con 0x7f2e7c060920 2022-06-17T12:52:26.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.148+0000 7f2e7b7fe700 1 -- 172.21.15.36:0/944112454 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e70000e90 con 0x7f2e7c060920 2022-06-17T12:52:26.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.148+0000 7f2e7b7fe700 1 -- 172.21.15.36:0/944112454 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e70012460 con 0x7f2e7c060920 2022-06-17T12:52:26.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.148+0000 7f2e81889700 1 -- 172.21.15.36:0/944112454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2e7c060920 msgr2=0x7f2e7c067610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.148+0000 7f2e81889700 1 --2- 172.21.15.36:0/944112454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2e7c060920 0x7f2e7c067610 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f2e68001110 tx=0x7f2e70006df0).stop 2022-06-17T12:52:26.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.148+0000 7f2e81889700 1 -- 172.21.15.36:0/944112454 shutdown_connections 2022-06-17T12:52:26.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.148+0000 7f2e81889700 1 --2- 172.21.15.36:0/944112454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2e7c060920 0x7f2e7c067610 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.148+0000 7f2e81889700 1 -- 172.21.15.36:0/944112454 >> 172.21.15.36:0/944112454 conn(0x7f2e7c0b64f0 msgr2=0x7f2e7c0b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:26.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.148+0000 7f2e81889700 1 -- 172.21.15.36:0/944112454 shutdown_connections 2022-06-17T12:52:26.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.148+0000 7f2e81889700 1 -- 172.21.15.36:0/944112454 wait complete. 2022-06-17T12:52:26.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.152+0000 7f2e81889700 1 Processor -- start 2022-06-17T12:52:26.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.152+0000 7f2e81889700 1 -- start start 2022-06-17T12:52:26.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.152+0000 7f2e81889700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e7c060920 0x7f2e7c13e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:26.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.152+0000 7f2e81889700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2e7c0b1e00 0x7f2e7c139570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:26.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.152+0000 7f2e81889700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2e7c13e8c0 0x7f2e7c139a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:26.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.152+0000 7f2e81889700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7c066020 con 0x7f2e7c0b1e00 2022-06-17T12:52:26.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.152+0000 7f2e81889700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7c0662c0 con 0x7f2e7c060920 2022-06-17T12:52:26.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.152+0000 7f2e81889700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7c066560 con 0x7f2e7c13e8c0 2022-06-17T12:52:26.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.152+0000 7f2e7bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2e7c0b1e00 0x7f2e7c139570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:26.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.152+0000 7f2e7bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2e7c0b1e00 0x7f2e7c139570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35008/0 (socket says 172.21.15.36:35008) 2022-06-17T12:52:26.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.152+0000 7f2e7bfff700 1 -- 172.21.15.36:0/3706980954 learned_addr learned my addr 172.21.15.36:0/3706980954 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:26.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.152+0000 7f2e80887700 1 --2- 172.21.15.36:0/3706980954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e7c060920 0x7f2e7c13e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:26.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.152+0000 7f2e81088700 1 --2- 172.21.15.36:0/3706980954 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2e7c13e8c0 0x7f2e7c139a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:26.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.152+0000 7f2e7bfff700 1 -- 172.21.15.36:0/3706980954 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2e7c13e8c0 msgr2=0x7f2e7c139a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.152+0000 7f2e7bfff700 1 --2- 172.21.15.36:0/3706980954 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2e7c13e8c0 0x7f2e7c139a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.152+0000 7f2e7bfff700 1 -- 172.21.15.36:0/3706980954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e7c060920 msgr2=0x7f2e7c13e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.152+0000 7f2e7bfff700 1 --2- 172.21.15.36:0/3706980954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e7c060920 0x7f2e7c13e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.152+0000 7f2e7bfff700 1 -- 172.21.15.36:0/3706980954 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7000b180 con 0x7f2e7c0b1e00 2022-06-17T12:52:26.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.152+0000 7f2e7bfff700 1 --2- 172.21.15.36:0/3706980954 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2e7c0b1e00 0x7f2e7c139570 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f2e68001840 tx=0x7f2e68001720).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:26.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.152+0000 7f2e79ffb700 1 -- 172.21.15.36:0/3706980954 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e68007170 con 0x7f2e7c0b1e00 2022-06-17T12:52:26.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.152+0000 7f2e79ffb700 1 -- 172.21.15.36:0/3706980954 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e680072d0 con 0x7f2e7c0b1e00 2022-06-17T12:52:26.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.152+0000 7f2e79ffb700 1 -- 172.21.15.36:0/3706980954 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e68004710 con 0x7f2e7c0b1e00 2022-06-17T12:52:26.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.152+0000 7f2e81889700 1 -- 172.21.15.36:0/3706980954 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e7c065d10 con 0x7f2e7c0b1e00 2022-06-17T12:52:26.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.152+0000 7f2e81889700 1 -- 172.21.15.36:0/3706980954 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e7c1321b0 con 0x7f2e7c0b1e00 2022-06-17T12:52:26.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.156+0000 7f2e81889700 1 -- 172.21.15.36:0/3706980954 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e7c0ade70 con 0x7f2e7c0b1e00 2022-06-17T12:52:26.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.156+0000 7f2e79ffb700 1 -- 172.21.15.36:0/3706980954 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2e6801b070 con 0x7f2e7c0b1e00 2022-06-17T12:52:26.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.156+0000 7f2e79ffb700 1 --2- 172.21.15.36:0/3706980954 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2e5c0539b0 0x7f2e5c055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:26.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.156+0000 7f2e79ffb700 1 -- 172.21.15.36:0/3706980954 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2e6806b6c0 con 0x7f2e7c0b1e00 2022-06-17T12:52:26.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.156+0000 7f2e80887700 1 --2- 172.21.15.36:0/3706980954 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2e5c0539b0 0x7f2e5c055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:26.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.156+0000 7f2e80887700 1 --2- 172.21.15.36:0/3706980954 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2e5c0539b0 0x7f2e5c055e70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f2e7001a000 tx=0x7f2e70014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:26.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.164+0000 7f2e79ffb700 1 -- 172.21.15.36:0/3706980954 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2e68042330 con 0x7f2e7c0b1e00 2022-06-17T12:52:26.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.288+0000 7f2e81889700 1 -- 172.21.15.36:0/3706980954 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2e7c066720 con 0x7f2e7c0b1e00 2022-06-17T12:52:26.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.292+0000 7f2e79ffb700 1 -- 172.21.15.36:0/3706980954 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2e68045950 con 0x7f2e7c0b1e00 2022-06-17T12:52:26.277 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:52:26.279 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.292+0000 7f2e81889700 1 -- 172.21.15.36:0/3706980954 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2e5c0539b0 msgr2=0x7f2e5c055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.292+0000 7f2e81889700 1 --2- 172.21.15.36:0/3706980954 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2e5c0539b0 0x7f2e5c055e70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f2e7001a000 tx=0x7f2e70014000).stop 2022-06-17T12:52:26.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.292+0000 7f2e81889700 1 -- 172.21.15.36:0/3706980954 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2e7c0b1e00 msgr2=0x7f2e7c139570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.292+0000 7f2e81889700 1 --2- 172.21.15.36:0/3706980954 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2e7c0b1e00 0x7f2e7c139570 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f2e68001840 tx=0x7f2e68001720).stop 2022-06-17T12:52:26.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.292+0000 7f2e81889700 1 -- 172.21.15.36:0/3706980954 shutdown_connections 2022-06-17T12:52:26.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.292+0000 7f2e81889700 1 --2- 172.21.15.36:0/3706980954 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2e5c0539b0 0x7f2e5c055e70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.292+0000 7f2e81889700 1 --2- 172.21.15.36:0/3706980954 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2e7c13e8c0 0x7f2e7c139a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.292+0000 7f2e81889700 1 --2- 172.21.15.36:0/3706980954 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2e7c0b1e00 0x7f2e7c139570 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.292+0000 7f2e81889700 1 --2- 172.21.15.36:0/3706980954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e7c060920 0x7f2e7c13e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.292+0000 7f2e81889700 1 -- 172.21.15.36:0/3706980954 >> 172.21.15.36:0/3706980954 conn(0x7f2e7c0b64f0 msgr2=0x7f2e7c136720 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:26.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.292+0000 7f2e81889700 1 -- 172.21.15.36:0/3706980954 shutdown_connections 2022-06-17T12:52:26.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.292+0000 7f2e81889700 1 -- 172.21.15.36:0/3706980954 wait complete. 2022-06-17T12:52:26.290 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:52:26.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.416+0000 7fcaaa32b700 1 Processor -- start 2022-06-17T12:52:26.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.416+0000 7fcaaa32b700 1 -- start start 2022-06-17T12:52:26.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.416+0000 7fcaaa32b700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcaa40b9860 0x7fcaa4143840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:26.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.416+0000 7fcaaa32b700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa40b4160 con 0x7fcaa40b5dc0 2022-06-17T12:52:26.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.416+0000 7fcaaa32b700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa40b9c40 con 0x7fcaa40b45c0 2022-06-17T12:52:26.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.416+0000 7fcaaa32b700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa40b5120 con 0x7fcaa40b9860 2022-06-17T12:52:26.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.416+0000 7fcaa9b2a700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fcaa40b5dc0 0x7fcaa40b6210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52546/0 (socket says 172.21.15.36:52546) 2022-06-17T12:52:26.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.416+0000 7fcaa9b2a700 1 -- 172.21.15.36:0/877546800 learned_addr learned my addr 172.21.15.36:0/877546800 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:26.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.416+0000 7fcaa9329700 1 --2- 172.21.15.36:0/877546800 >> v2:172.21.15.70:3300/0 conn(0x7fcaa40b9860 0x7fcaa4143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:26.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.416+0000 7fcaa3fff700 1 -- 172.21.15.36:0/877546800 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1771516125 0 0) 0x7fcaa40b4160 con 0x7fcaa40b5dc0 2022-06-17T12:52:26.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.416+0000 7fcaa3fff700 1 -- 172.21.15.36:0/877546800 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca90003260 con 0x7fcaa40b5dc0 2022-06-17T12:52:26.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.416+0000 7fcaa9329700 1 -- 172.21.15.36:0/877546800 >> v1:172.21.15.70:6789/0 conn(0x7fcaa40b45c0 legacy=0x7fcaa40b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.416+0000 7fcaa9329700 1 -- 172.21.15.36:0/877546800 >> v1:172.21.15.36:6789/0 conn(0x7fcaa40b5dc0 legacy=0x7fcaa40b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.416+0000 7fcaa9329700 1 -- 172.21.15.36:0/877546800 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa40627d0 con 0x7fcaa40b9860 2022-06-17T12:52:26.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.416+0000 7fcaa9329700 1 --2- 172.21.15.36:0/877546800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa40b9860 0x7fcaa4143840 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7fca94002f70 tx=0x7fca94006df0).ready entity=mon.1 client_cookie=5afd3f446495bb3b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:26.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.420+0000 7fcaa3fff700 1 -- 172.21.15.36:0/877546800 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca9400d610 con 0x7fcaa40b9860 2022-06-17T12:52:26.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.420+0000 7fcaa3fff700 1 -- 172.21.15.36:0/877546800 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca94000e90 con 0x7fcaa40b9860 2022-06-17T12:52:26.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.420+0000 7fcaa3fff700 1 -- 172.21.15.36:0/877546800 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca9400d960 con 0x7fcaa40b9860 2022-06-17T12:52:26.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.420+0000 7fcaaa32b700 1 -- 172.21.15.36:0/877546800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa40b9860 msgr2=0x7fcaa4143840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.420+0000 7fcaaa32b700 1 --2- 172.21.15.36:0/877546800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa40b9860 0x7fcaa4143840 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7fca94002f70 tx=0x7fca94006df0).stop 2022-06-17T12:52:26.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.420+0000 7fcaaa32b700 1 -- 172.21.15.36:0/877546800 shutdown_connections 2022-06-17T12:52:26.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.420+0000 7fcaaa32b700 1 --2- 172.21.15.36:0/877546800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa40b9860 0x7fcaa4143840 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.420+0000 7fcaaa32b700 1 -- 172.21.15.36:0/877546800 >> 172.21.15.36:0/877546800 conn(0x7fcaa405c550 msgr2=0x7fcaa4060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:26.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.420+0000 7fcaaa32b700 1 -- 172.21.15.36:0/877546800 shutdown_connections 2022-06-17T12:52:26.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.420+0000 7fcaaa32b700 1 -- 172.21.15.36:0/877546800 wait complete. 2022-06-17T12:52:26.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.420+0000 7fcaaa32b700 1 Processor -- start 2022-06-17T12:52:26.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.424+0000 7fcaaa32b700 1 -- start start 2022-06-17T12:52:26.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.424+0000 7fcaaa32b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcaa40b45c0 0x7fcaa4145620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:26.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.424+0000 7fcaaa32b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa40b5dc0 0x7fcaa4144530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:26.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.424+0000 7fcaaa32b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcaa40b9860 0x7fcaa4152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:26.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.424+0000 7fcaaa32b700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa4144f50 con 0x7fcaa40b45c0 2022-06-17T12:52:26.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.424+0000 7fcaaa32b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa40add80 con 0x7fcaa40b5dc0 2022-06-17T12:52:26.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.424+0000 7fcaaa32b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa413fb50 con 0x7fcaa40b9860 2022-06-17T12:52:26.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.424+0000 7fcaa9329700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcaa40b45c0 0x7fcaa4145620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:26.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.424+0000 7fcaa9329700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcaa40b45c0 0x7fcaa4145620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35020/0 (socket says 172.21.15.36:35020) 2022-06-17T12:52:26.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.424+0000 7fcaa9329700 1 -- 172.21.15.36:0/1161754820 learned_addr learned my addr 172.21.15.36:0/1161754820 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:26.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.424+0000 7fcaa9b2a700 1 --2- 172.21.15.36:0/1161754820 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcaa40b9860 0x7fcaa4152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:26.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.424+0000 7fcaa8b28700 1 --2- 172.21.15.36:0/1161754820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa40b5dc0 0x7fcaa4144530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:26.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.424+0000 7fcaa9329700 1 -- 172.21.15.36:0/1161754820 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcaa40b9860 msgr2=0x7fcaa4152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.424+0000 7fcaa9329700 1 --2- 172.21.15.36:0/1161754820 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcaa40b9860 0x7fcaa4152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.424+0000 7fcaa9329700 1 -- 172.21.15.36:0/1161754820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa40b5dc0 msgr2=0x7fcaa4144530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.424+0000 7fcaa9329700 1 --2- 172.21.15.36:0/1161754820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa40b5dc0 0x7fcaa4144530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.424+0000 7fcaa9329700 1 -- 172.21.15.36:0/1161754820 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9400b180 con 0x7fcaa40b45c0 2022-06-17T12:52:26.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.424+0000 7fcaa9329700 1 --2- 172.21.15.36:0/1161754820 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcaa40b45c0 0x7fcaa4145620 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7fca94002f70 tx=0x7fca94014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:26.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.424+0000 7fcaa27fc700 1 -- 172.21.15.36:0/1161754820 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca94013de0 con 0x7fcaa40b45c0 2022-06-17T12:52:26.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.424+0000 7fcaa27fc700 1 -- 172.21.15.36:0/1161754820 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9400d290 con 0x7fcaa40b45c0 2022-06-17T12:52:26.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.424+0000 7fcaaa32b700 1 -- 172.21.15.36:0/1161754820 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa4145180 con 0x7fcaa40b45c0 2022-06-17T12:52:26.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.424+0000 7fcaa27fc700 1 -- 172.21.15.36:0/1161754820 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca9401e5f0 con 0x7fcaa40b45c0 2022-06-17T12:52:26.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.424+0000 7fcaaa32b700 1 -- 172.21.15.36:0/1161754820 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa413f810 con 0x7fcaa40b45c0 2022-06-17T12:52:26.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.424+0000 7fcaaa32b700 1 -- 172.21.15.36:0/1161754820 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa4062eb0 con 0x7fcaa40b45c0 2022-06-17T12:52:26.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.428+0000 7fcaa27fc700 1 -- 172.21.15.36:0/1161754820 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fca94025030 con 0x7fcaa40b45c0 2022-06-17T12:52:26.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.428+0000 7fcaa27fc700 1 --2- 172.21.15.36:0/1161754820 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fca84053a40 0x7fca84055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:26.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.428+0000 7fcaa27fc700 1 -- 172.21.15.36:0/1161754820 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fca9406cdf0 con 0x7fcaa40b45c0 2022-06-17T12:52:26.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.428+0000 7fcaa8b28700 1 --2- 172.21.15.36:0/1161754820 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fca84053a40 0x7fca84055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:26.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.428+0000 7fcaa8b28700 1 --2- 172.21.15.36:0/1161754820 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fca84053a40 0x7fca84055f00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fca9800d000 tx=0x7fca98009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:26.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.436+0000 7fcaa27fc700 1 -- 172.21.15.36:0/1161754820 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fca94043c30 con 0x7fcaa40b45c0 2022-06-17T12:52:26.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.564+0000 7fcaaa32b700 1 -- 172.21.15.36:0/1161754820 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcaa413fd10 con 0x7fcaa40b45c0 2022-06-17T12:52:26.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.564+0000 7fcaa27fc700 1 -- 172.21.15.36:0/1161754820 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+10 (secure 0 0 0) 0x7fca94023020 con 0x7fcaa40b45c0 2022-06-17T12:52:26.550 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:52:26.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.568+0000 7fcaaa32b700 1 -- 172.21.15.36:0/1161754820 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fca84053a40 msgr2=0x7fca84055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.568+0000 7fcaaa32b700 1 --2- 172.21.15.36:0/1161754820 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fca84053a40 0x7fca84055f00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fca9800d000 tx=0x7fca98009040).stop 2022-06-17T12:52:26.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.568+0000 7fcaaa32b700 1 -- 172.21.15.36:0/1161754820 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcaa40b45c0 msgr2=0x7fcaa4145620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.568+0000 7fcaaa32b700 1 --2- 172.21.15.36:0/1161754820 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcaa40b45c0 0x7fcaa4145620 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7fca94002f70 tx=0x7fca94014000).stop 2022-06-17T12:52:26.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.568+0000 7fcaaa32b700 1 -- 172.21.15.36:0/1161754820 shutdown_connections 2022-06-17T12:52:26.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.568+0000 7fcaaa32b700 1 --2- 172.21.15.36:0/1161754820 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fca84053a40 0x7fca84055f00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.568+0000 7fcaaa32b700 1 --2- 172.21.15.36:0/1161754820 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcaa40b9860 0x7fcaa4152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.568+0000 7fcaaa32b700 1 --2- 172.21.15.36:0/1161754820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa40b5dc0 0x7fcaa4144530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.568+0000 7fcaaa32b700 1 --2- 172.21.15.36:0/1161754820 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcaa40b45c0 0x7fcaa4145620 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.568+0000 7fcaaa32b700 1 -- 172.21.15.36:0/1161754820 >> 172.21.15.36:0/1161754820 conn(0x7fcaa405c550 msgr2=0x7fcaa40b88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:26.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.568+0000 7fcaaa32b700 1 -- 172.21.15.36:0/1161754820 shutdown_connections 2022-06-17T12:52:26.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.568+0000 7fcaaa32b700 1 -- 172.21.15.36:0/1161754820 wait complete. 2022-06-17T12:52:26.567 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:52:26.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.692+0000 7fe3821fe700 1 Processor -- start 2022-06-17T12:52:26.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.692+0000 7fe3821fe700 1 -- start start 2022-06-17T12:52:26.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.692+0000 7fe3821fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe37c0b65f0 0x7fe37c0b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:26.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.692+0000 7fe3821fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe37c0b15b0 0x7fe37c0af630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:26.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.692+0000 7fe3821fe700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe37c13fdf0 con 0x7fe37c0adf10 2022-06-17T12:52:26.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.692+0000 7fe3821fe700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe37c0ae8e0 con 0x7fe37c0b15b0 2022-06-17T12:52:26.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.692+0000 7fe3821fe700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c140090 con 0x7fe37c0b65f0 2022-06-17T12:52:26.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.692+0000 7fe3819fd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe37c0b15b0 0x7fe37c0af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:26.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.692+0000 7fe3809fb700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fe37c0adf10 0x7fe37c140530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52558/0 (socket says 172.21.15.36:52558) 2022-06-17T12:52:26.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.692+0000 7fe3809fb700 1 -- 172.21.15.36:0/3972850968 learned_addr learned my addr 172.21.15.36:0/3972850968 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:26.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.692+0000 7fe3811fc700 1 --2- 172.21.15.36:0/3972850968 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe37c0b65f0 0x7fe37c0b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:26.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.692+0000 7fe373fff700 1 -- 172.21.15.36:0/3972850968 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940185993 0 0) 0x7fe37c13fdf0 con 0x7fe37c0adf10 2022-06-17T12:52:26.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.692+0000 7fe373fff700 1 -- 172.21.15.36:0/3972850968 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe360003260 con 0x7fe37c0adf10 2022-06-17T12:52:26.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.692+0000 7fe3819fd700 1 -- 172.21.15.36:0/3972850968 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe37c0b65f0 msgr2=0x7fe37c0b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:26.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.692+0000 7fe3819fd700 1 --2- 172.21.15.36:0/3972850968 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe37c0b65f0 0x7fe37c0b69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.692+0000 7fe3819fd700 1 -- 172.21.15.36:0/3972850968 >> v1:172.21.15.36:6789/0 conn(0x7fe37c0adf10 legacy=0x7fe37c140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.692+0000 7fe373fff700 1 -- 172.21.15.36:0/3972850968 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2347037883 0 0) 0x7fe360003260 con 0x7fe37c0adf10 2022-06-17T12:52:26.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.692+0000 7fe3819fd700 1 -- 172.21.15.36:0/3972850968 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c0b1be0 con 0x7fe37c0b15b0 2022-06-17T12:52:26.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.692+0000 7fe3819fd700 1 --2- 172.21.15.36:0/3972850968 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe37c0b15b0 0x7fe37c0af630 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7fe36c001980 tx=0x7fe378006df0).ready entity=mon.0 client_cookie=9b9fb34d34ece814 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:26.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.692+0000 7fe373fff700 1 -- 172.21.15.36:0/3972850968 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe36c0017e0 con 0x7fe37c0b15b0 2022-06-17T12:52:26.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.692+0000 7fe373fff700 1 -- 172.21.15.36:0/3972850968 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe378000e90 con 0x7fe37c0b15b0 2022-06-17T12:52:26.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.692+0000 7fe373fff700 1 -- 172.21.15.36:0/3972850968 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe378012460 con 0x7fe37c0b15b0 2022-06-17T12:52:26.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3821fe700 1 -- 172.21.15.36:0/3972850968 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe37c0b15b0 msgr2=0x7fe37c0af630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3821fe700 1 --2- 172.21.15.36:0/3972850968 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe37c0b15b0 0x7fe37c0af630 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7fe36c001980 tx=0x7fe378006df0).stop 2022-06-17T12:52:26.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3821fe700 1 -- 172.21.15.36:0/3972850968 shutdown_connections 2022-06-17T12:52:26.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3821fe700 1 --2- 172.21.15.36:0/3972850968 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe37c0b15b0 0x7fe37c0af630 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3821fe700 1 --2- 172.21.15.36:0/3972850968 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe37c0b65f0 0x7fe37c0b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3821fe700 1 -- 172.21.15.36:0/3972850968 >> 172.21.15.36:0/3972850968 conn(0x7fe37c0614b0 msgr2=0x7fe37c061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:26.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3821fe700 1 -- 172.21.15.36:0/3972850968 shutdown_connections 2022-06-17T12:52:26.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3821fe700 1 -- 172.21.15.36:0/3972850968 wait complete. 2022-06-17T12:52:26.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3821fe700 1 Processor -- start 2022-06-17T12:52:26.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3821fe700 1 -- start start 2022-06-17T12:52:26.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3821fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe37c0adf10 0x7fe37c145ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:26.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3821fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe37c0b15b0 0x7fe37c138d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:26.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3821fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe37c0b65f0 0x7fe37c1391f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:26.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3821fe700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c155580 con 0x7fe37c0adf10 2022-06-17T12:52:26.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3821fe700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c10f580 con 0x7fe37c0b15b0 2022-06-17T12:52:26.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3821fe700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c10f820 con 0x7fe37c0b65f0 2022-06-17T12:52:26.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3811fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe37c0adf10 0x7fe37c145ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:26.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3811fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe37c0adf10 0x7fe37c145ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35034/0 (socket says 172.21.15.36:35034) 2022-06-17T12:52:26.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3811fc700 1 -- 172.21.15.36:0/1377021876 learned_addr learned my addr 172.21.15.36:0/1377021876 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:26.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3819fd700 1 --2- 172.21.15.36:0/1377021876 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe37c0b65f0 0x7fe37c1391f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:26.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3811fc700 1 -- 172.21.15.36:0/1377021876 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe37c0b65f0 msgr2=0x7fe37c1391f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3809fb700 1 --2- 172.21.15.36:0/1377021876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe37c0b15b0 0x7fe37c138d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:26.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3811fc700 1 --2- 172.21.15.36:0/1377021876 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe37c0b65f0 0x7fe37c1391f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3811fc700 1 -- 172.21.15.36:0/1377021876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe37c0b15b0 msgr2=0x7fe37c138d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3811fc700 1 --2- 172.21.15.36:0/1377021876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe37c0b15b0 0x7fe37c138d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3811fc700 1 -- 172.21.15.36:0/1377021876 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37800b180 con 0x7fe37c0adf10 2022-06-17T12:52:26.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.696+0000 7fe3809fb700 1 --2- 172.21.15.36:0/1377021876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe37c0b15b0 0x7fe37c138d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:52:26.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.700+0000 7fe3811fc700 1 --2- 172.21.15.36:0/1377021876 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe37c0adf10 0x7fe37c145ac0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fe37400c000 tx=0x7fe3740091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:26.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.700+0000 7fe3727fc700 1 -- 172.21.15.36:0/1377021876 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe37400c850 con 0x7fe37c0adf10 2022-06-17T12:52:26.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.700+0000 7fe3727fc700 1 -- 172.21.15.36:0/1377021876 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37400c9b0 con 0x7fe37c0adf10 2022-06-17T12:52:26.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.700+0000 7fe3821fe700 1 -- 172.21.15.36:0/1377021876 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe37c064160 con 0x7fe37c0adf10 2022-06-17T12:52:26.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.700+0000 7fe3727fc700 1 -- 172.21.15.36:0/1377021876 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe37400cd10 con 0x7fe37c0adf10 2022-06-17T12:52:26.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.700+0000 7fe3821fe700 1 -- 172.21.15.36:0/1377021876 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37c1396e0 con 0x7fe37c0adf10 2022-06-17T12:52:26.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.700+0000 7fe3821fe700 1 -- 172.21.15.36:0/1377021876 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe37c0642f0 con 0x7fe37c0adf10 2022-06-17T12:52:26.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.700+0000 7fe3727fc700 1 -- 172.21.15.36:0/1377021876 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe374022070 con 0x7fe37c0adf10 2022-06-17T12:52:26.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.700+0000 7fe3727fc700 1 --2- 172.21.15.36:0/1377021876 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe364053a40 0x7fe364055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:26.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.700+0000 7fe3727fc700 1 -- 172.21.15.36:0/1377021876 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe37406c6e0 con 0x7fe37c0adf10 2022-06-17T12:52:26.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.700+0000 7fe3809fb700 1 --2- 172.21.15.36:0/1377021876 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe364053a40 0x7fe364055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:26.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.704+0000 7fe3809fb700 1 --2- 172.21.15.36:0/1377021876 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe364053a40 0x7fe364055f00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fe36c008000 tx=0x7fe36c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:26.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.704+0000 7fe3727fc700 1 -- 172.21.15.36:0/1377021876 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe3740434e0 con 0x7fe37c0adf10 2022-06-17T12:52:26.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.832+0000 7fe3821fe700 1 -- 172.21.15.36:0/1377021876 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe37c13fae0 con 0x7fe37c0adf10 2022-06-17T12:52:26.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.832+0000 7fe3727fc700 1 -- 172.21.15.36:0/1377021876 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+10 (secure 0 0 0) 0x7fe374046b00 con 0x7fe37c0adf10 2022-06-17T12:52:26.820 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:52:26.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.836+0000 7fe3821fe700 1 -- 172.21.15.36:0/1377021876 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe364053a40 msgr2=0x7fe364055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.836+0000 7fe3821fe700 1 --2- 172.21.15.36:0/1377021876 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe364053a40 0x7fe364055f00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fe36c008000 tx=0x7fe36c009040).stop 2022-06-17T12:52:26.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.836+0000 7fe3821fe700 1 -- 172.21.15.36:0/1377021876 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe37c0adf10 msgr2=0x7fe37c145ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.836+0000 7fe3821fe700 1 --2- 172.21.15.36:0/1377021876 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe37c0adf10 0x7fe37c145ac0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fe37400c000 tx=0x7fe3740091d0).stop 2022-06-17T12:52:26.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.836+0000 7fe3821fe700 1 -- 172.21.15.36:0/1377021876 shutdown_connections 2022-06-17T12:52:26.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.836+0000 7fe3821fe700 1 --2- 172.21.15.36:0/1377021876 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe364053a40 0x7fe364055f00 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.836+0000 7fe3821fe700 1 --2- 172.21.15.36:0/1377021876 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe37c0b65f0 0x7fe37c1391f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.836+0000 7fe3821fe700 1 --2- 172.21.15.36:0/1377021876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe37c0b15b0 0x7fe37c138d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.836+0000 7fe3821fe700 1 --2- 172.21.15.36:0/1377021876 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe37c0adf10 0x7fe37c145ac0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.836+0000 7fe3821fe700 1 -- 172.21.15.36:0/1377021876 >> 172.21.15.36:0/1377021876 conn(0x7fe37c0614b0 msgr2=0x7fe37c146520 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:26.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.836+0000 7fe3821fe700 1 -- 172.21.15.36:0/1377021876 shutdown_connections 2022-06-17T12:52:26.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.836+0000 7fe3821fe700 1 -- 172.21.15.36:0/1377021876 wait complete. 2022-06-17T12:52:26.832 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:52:26.833 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:52:26.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.952+0000 7f74cc943700 1 Processor -- start 2022-06-17T12:52:26.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.952+0000 7f74cc943700 1 -- start start 2022-06-17T12:52:26.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.952+0000 7f74cc943700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f74c80b2200 0x7f74c80af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:26.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.952+0000 7f74cc943700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74c80af8d0 0x7f74c814a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:26.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.952+0000 7f74cc943700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c80b25e0 con 0x7f74c80b2cd0 2022-06-17T12:52:26.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.952+0000 7f74cc943700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c80ad790 con 0x7f74c80b2200 2022-06-17T12:52:26.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.952+0000 7f74cc943700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80aef40 con 0x7f74c80af8d0 2022-06-17T12:52:26.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.952+0000 7f74c6d9d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f74c80b2200 0x7f74c80af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:26.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.952+0000 7f74c6d9d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f74c80b2200 0x7f74c80af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35044/0 (socket says 172.21.15.36:35044) 2022-06-17T12:52:26.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.952+0000 7f74c6d9d700 1 -- 172.21.15.36:0/1932130297 learned_addr learned my addr 172.21.15.36:0/1932130297 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:26.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.952+0000 7f74c659c700 1 --2- 172.21.15.36:0/1932130297 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74c80af8d0 0x7f74c814a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:26.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.956+0000 7f74c6d9d700 1 -- 172.21.15.36:0/1932130297 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74c80af8d0 msgr2=0x7f74c814a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:26.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.956+0000 7f74c6d9d700 1 --2- 172.21.15.36:0/1932130297 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74c80af8d0 0x7f74c814a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.956+0000 7f74c6d9d700 1 -- 172.21.15.36:0/1932130297 >> v1:172.21.15.70:6789/0 conn(0x7f74c80b2cd0 legacy=0x7f74c814adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.956+0000 7f74c6d9d700 1 -- 172.21.15.36:0/1932130297 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c80b1950 con 0x7f74c80b2200 2022-06-17T12:52:26.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.956+0000 7f74c6d9d700 1 --2- 172.21.15.36:0/1932130297 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74c80b2200 0x7f74c80af3e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f74bc002f70 tx=0x7f74bc006eb0).ready entity=mon.0 client_cookie=40334cfb4f39fa01 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:26.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.956+0000 7f74c5d9b700 1 -- 172.21.15.36:0/1932130297 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74bc00d570 con 0x7f74c80b2200 2022-06-17T12:52:26.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.956+0000 7f74c5d9b700 1 -- 172.21.15.36:0/1932130297 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74bc000e90 con 0x7f74c80b2200 2022-06-17T12:52:26.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.956+0000 7f74c5d9b700 1 -- 172.21.15.36:0/1932130297 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74bc00d900 con 0x7f74c80b2200 2022-06-17T12:52:26.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.956+0000 7f74cc943700 1 -- 172.21.15.36:0/1932130297 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74c80b2200 msgr2=0x7f74c80af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.956+0000 7f74cc943700 1 --2- 172.21.15.36:0/1932130297 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74c80b2200 0x7f74c80af3e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f74bc002f70 tx=0x7f74bc006eb0).stop 2022-06-17T12:52:26.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.956+0000 7f74cc943700 1 -- 172.21.15.36:0/1932130297 shutdown_connections 2022-06-17T12:52:26.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.956+0000 7f74cc943700 1 --2- 172.21.15.36:0/1932130297 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74c80af8d0 0x7f74c814a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.956+0000 7f74cc943700 1 --2- 172.21.15.36:0/1932130297 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74c80b2200 0x7f74c80af3e0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.956+0000 7f74cc943700 1 -- 172.21.15.36:0/1932130297 >> 172.21.15.36:0/1932130297 conn(0x7f74c805f770 msgr2=0x7f74c805fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:26.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.956+0000 7f74cc943700 1 -- 172.21.15.36:0/1932130297 shutdown_connections 2022-06-17T12:52:26.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.956+0000 7f74cc943700 1 -- 172.21.15.36:0/1932130297 wait complete. 2022-06-17T12:52:26.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.956+0000 7f74cc943700 1 Processor -- start 2022-06-17T12:52:26.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.956+0000 7f74cc943700 1 -- start start 2022-06-17T12:52:26.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.960+0000 7f74cc943700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74c80af8d0 0x7f74c814d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:26.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.960+0000 7f74cc943700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74c80b2200 0x7f74c80aa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:26.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.960+0000 7f74cc943700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74c80b2cd0 0x7f74c80a70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:26.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.960+0000 7f74cc943700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80a95d0 con 0x7f74c80b2cd0 2022-06-17T12:52:26.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.960+0000 7f74cc943700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80ab880 con 0x7f74c80b2200 2022-06-17T12:52:26.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.960+0000 7f74cc943700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80aad90 con 0x7f74c80af8d0 2022-06-17T12:52:26.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.960+0000 7f74c759e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74c80b2cd0 0x7f74c80a70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:26.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.960+0000 7f74c759e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74c80b2cd0 0x7f74c80a70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35052/0 (socket says 172.21.15.36:35052) 2022-06-17T12:52:26.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.960+0000 7f74c759e700 1 -- 172.21.15.36:0/115616896 learned_addr learned my addr 172.21.15.36:0/115616896 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:26.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.960+0000 7f74c659c700 1 --2- 172.21.15.36:0/115616896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74c80b2200 0x7f74c80aa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:26.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.960+0000 7f74c6d9d700 1 --2- 172.21.15.36:0/115616896 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74c80af8d0 0x7f74c814d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:26.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.960+0000 7f74c759e700 1 -- 172.21.15.36:0/115616896 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74c80af8d0 msgr2=0x7f74c814d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.960+0000 7f74c759e700 1 --2- 172.21.15.36:0/115616896 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74c80af8d0 0x7f74c814d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.960+0000 7f74c759e700 1 -- 172.21.15.36:0/115616896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74c80b2200 msgr2=0x7f74c80aa8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:26.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.960+0000 7f74c759e700 1 --2- 172.21.15.36:0/115616896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74c80b2200 0x7f74c80aa8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:26.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.960+0000 7f74c759e700 1 -- 172.21.15.36:0/115616896 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc00b1e0 con 0x7f74c80b2cd0 2022-06-17T12:52:26.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.960+0000 7f74c759e700 1 --2- 172.21.15.36:0/115616896 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74c80b2cd0 0x7f74c80a70e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f74c80aaed0 tx=0x7f74c0007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:26.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.960+0000 7f74b7fff700 1 -- 172.21.15.36:0/115616896 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74c0006b70 con 0x7f74c80b2cd0 2022-06-17T12:52:26.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.960+0000 7f74b7fff700 1 -- 172.21.15.36:0/115616896 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c0006cd0 con 0x7f74c80b2cd0 2022-06-17T12:52:26.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.960+0000 7f74cc943700 1 -- 172.21.15.36:0/115616896 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c80aafb0 con 0x7f74c80b2cd0 2022-06-17T12:52:26.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.960+0000 7f74b7fff700 1 -- 172.21.15.36:0/115616896 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74c0007220 con 0x7f74c80b2cd0 2022-06-17T12:52:26.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.960+0000 7f74cc943700 1 -- 172.21.15.36:0/115616896 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c80a9f10 con 0x7f74c80b2cd0 2022-06-17T12:52:26.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.960+0000 7f74cc943700 1 -- 172.21.15.36:0/115616896 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c80ab140 con 0x7f74c80b2cd0 2022-06-17T12:52:26.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.964+0000 7f74b7fff700 1 -- 172.21.15.36:0/115616896 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f74c0010070 con 0x7f74c80b2cd0 2022-06-17T12:52:26.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.964+0000 7f74b7fff700 1 --2- 172.21.15.36:0/115616896 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f74a40539b0 0x7f74a4055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:26.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.964+0000 7f74b7fff700 1 -- 172.21.15.36:0/115616896 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f74c006bc30 con 0x7f74c80b2cd0 2022-06-17T12:52:26.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.964+0000 7f74c6d9d700 1 --2- 172.21.15.36:0/115616896 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f74a40539b0 0x7f74a4055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:26.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.964+0000 7f74c6d9d700 1 --2- 172.21.15.36:0/115616896 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f74a40539b0 0x7f74a4055e70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f74bc01a000 tx=0x7f74bc015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:26.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:26.968+0000 7f74b7fff700 1 -- 172.21.15.36:0/115616896 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f74c0042a30 con 0x7f74c80b2cd0 2022-06-17T12:52:27.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:27.132+0000 7f74cc943700 1 -- 172.21.15.36:0/115616896 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74c80ad330 con 0x7f74c80b2cd0 2022-06-17T12:52:27.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:27.132+0000 7f74b7fff700 1 -- 172.21.15.36:0/115616896 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f74c0046050 con 0x7f74c80b2cd0 2022-06-17T12:52:27.120 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:52:27.120 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":30,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5420,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/3105680199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3105680199},{"type":"v1","addr":"172.21.15.70:6833","nonce":3105680199}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":29}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:52:22.253232+0000","modified":"2022-06-17T12:52:23.257219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6054},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6054":{"gid":6054,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/4091208534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":4091208534},{"type":"v1","addr":"172.21.15.36:6835","nonce":4091208534}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-06-17T12:52:27.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:27.136+0000 7f74cc943700 1 -- 172.21.15.36:0/115616896 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f74a40539b0 msgr2=0x7f74a4055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:27.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:27.136+0000 7f74cc943700 1 --2- 172.21.15.36:0/115616896 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f74a40539b0 0x7f74a4055e70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f74bc01a000 tx=0x7f74bc015000).stop 2022-06-17T12:52:27.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:27.136+0000 7f74cc943700 1 -- 172.21.15.36:0/115616896 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74c80b2cd0 msgr2=0x7f74c80a70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:27.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:27.136+0000 7f74cc943700 1 --2- 172.21.15.36:0/115616896 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74c80b2cd0 0x7f74c80a70e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f74c80aaed0 tx=0x7f74c0007f70).stop 2022-06-17T12:52:27.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:27.136+0000 7f74cc943700 1 -- 172.21.15.36:0/115616896 shutdown_connections 2022-06-17T12:52:27.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:27.136+0000 7f74cc943700 1 --2- 172.21.15.36:0/115616896 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f74a40539b0 0x7f74a4055e70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:27.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:27.136+0000 7f74cc943700 1 --2- 172.21.15.36:0/115616896 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74c80b2cd0 0x7f74c80a70e0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:27.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:27.136+0000 7f74cc943700 1 --2- 172.21.15.36:0/115616896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74c80b2200 0x7f74c80aa8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:27.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:27.136+0000 7f74cc943700 1 --2- 172.21.15.36:0/115616896 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74c80af8d0 0x7f74c814d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:27.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:27.136+0000 7f74cc943700 1 -- 172.21.15.36:0/115616896 >> 172.21.15.36:0/115616896 conn(0x7f74c805f770 msgr2=0x7f74c81463c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:27.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:27.136+0000 7f74cc943700 1 -- 172.21.15.36:0/115616896 shutdown_connections 2022-06-17T12:52:27.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:27.136+0000 7f74cc943700 1 -- 172.21.15.36:0/115616896 wait complete. 2022-06-17T12:52:27.126 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 30 2022-06-17T12:52:27.334 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:52:22.253232+0000', 'modified': '2022-06-17T12:52:23.257219+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6054}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6054': {'gid': 6054, 'name': 'a', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.36:6835/4091208534', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 4091208534}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 4091208534}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:52:27.335 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:52:27.336 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:52:27.454 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":6054,"want_state":"up:active","state":"up:active","rank_uptime":5.206294035,"mdsmap_epoch":30,"osdmap_epoch":65,"osdmap_epoch_barrier":65,"uptime":9.5984942259999997} 2022-06-17T12:52:27.455 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 6054, "mdsmap_epoch": 30, "osdmap_epoch": 65, "osdmap_epoch_barrier": 65, "rank_uptime": 5.206294035, "state": "up:active", "uptime": 9.598494226, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:52:27.455 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:52:27.456 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-17T12:52:27.456 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi107.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-17T12:52:27.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:52:27.457 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:52:27.466 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:52:27.483 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:52:27.484 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:52:27.546 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:52:27.548 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:52:27.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:52:27.551 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:52:27.604 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:52:27.604 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:52:27.608 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:52:27.623 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:52:27.624 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:52:27.683 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:52:27.684 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:52:27.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:52:27.685 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:52:27.687 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:2022-06-17T12:52:27.701+0000 7fef034d0080 -1 init, newargv = 0x558732316520 newargc=9ceph-fuse[22254]: starting ceph client 2022-06-17T12:52:27.687 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr: 2022-06-17T12:52:27.694 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22254]: starting fuse 2022-06-17T12:52:28.794 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:52:28.817 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:52:28.818 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:52:28.876 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:52:28.878 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:52:28.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:52:28.881 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:52:28.890 INFO:teuthology.orchestra.run.smithi107.stdout:52 2022-06-17T12:52:28.891 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:52:28.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:52:28.894 DEBUG:teuthology.orchestra.run.smithi107:> sudo python3 -c ' 2022-06-17T12:52:28.895 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2022-06-17T12:52:28.895 DEBUG:teuthology.orchestra.run.smithi107:> import re 2022-06-17T12:52:28.896 DEBUG:teuthology.orchestra.run.smithi107:> import os 2022-06-17T12:52:28.896 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2022-06-17T12:52:28.897 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:52:28.897 DEBUG:teuthology.orchestra.run.smithi107:> def find_socket(client_name): 2022-06-17T12:52:28.898 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-17T12:52:28.899 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2022-06-17T12:52:28.899 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:52:28.900 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2022-06-17T12:52:28.901 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2022-06-17T12:52:28.901 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2022-06-17T12:52:28.902 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2022-06-17T12:52:28.903 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:52:28.903 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2022-06-17T12:52:28.904 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:52:28.905 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:52:28.906 DEBUG:teuthology.orchestra.run.smithi107:> return f 2022-06-17T12:52:28.907 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:52:28.907 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:52:28.908 DEBUG:teuthology.orchestra.run.smithi107:> print(find_socket("client.0")) 2022-06-17T12:52:28.909 DEBUG:teuthology.orchestra.run.smithi107:> ' 2022-06-17T12:52:28.977 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.0.22254.asok 2022-06-17T12:52:28.981 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22254.asok 2022-06-17T12:52:28.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:52:28.982 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22254.asok status 2022-06-17T12:52:29.073 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2022-06-17T12:52:29.073 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2022-06-17T12:52:29.073 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:52:29.074 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:52:29.074 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "0", 2022-06-17T12:52:29.074 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2022-06-17T12:52:29.075 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-17T12:52:29.075 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "22254", 2022-06-17T12:52:29.075 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/" 2022-06-17T12:52:29.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:52:29.076 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2022-06-17T12:52:29.076 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2022-06-17T12:52:29.076 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 6040, 2022-06-17T12:52:29.077 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2022-06-17T12:52:29.077 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2022-06-17T12:52:29.077 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2022-06-17T12:52:29.077 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 6040 2022-06-17T12:52:29.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:52:29.078 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:52:29.079 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:52:29.079 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:52:29.079 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3312831868 2022-06-17T12:52:29.079 INFO:teuthology.orchestra.run.smithi107.stdout: } 2022-06-17T12:52:29.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:52:29.080 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:52:29.080 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:52:29.080 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:52:29.081 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3312831868 2022-06-17T12:52:29.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:52:29.081 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.6040 172.21.15.107:0/3312831868", 2022-06-17T12:52:29.081 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "172.21.15.107:0/3312831868", 2022-06-17T12:52:29.082 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 1, 2022-06-17T12:52:29.082 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 30, 2022-06-17T12:52:29.082 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 65, 2022-06-17T12:52:29.082 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:52:29.082 INFO:teuthology.orchestra.run.smithi107.stdout: "blacklisted": false, 2022-06-17T12:52:29.083 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2022-06-17T12:52:29.083 INFO:teuthology.orchestra.run.smithi107.stdout:} 2022-06-17T12:52:29.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:52:29.086 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:52:29.087 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:52:29.100 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:52:29.101 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:52:29.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:52:29.103 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:52:29.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:52:29.168 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:52:29.169 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:52:29.179 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:52:29.179 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:52:29.180 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-06-17T12:52:29.180 DEBUG:teuthology.orchestra.run.smithi107:> import sys 2022-06-17T12:52:29.181 DEBUG:teuthology.orchestra.run.smithi107:> import time 2022-06-17T12:52:29.181 DEBUG:teuthology.orchestra.run.smithi107:> import os 2022-06-17T12:52:29.182 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:52:29.182 DEBUG:teuthology.orchestra.run.smithi107:> n = 1000 2022-06-17T12:52:29.183 DEBUG:teuthology.orchestra.run.smithi107:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2022-06-17T12:52:29.183 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:52:29.184 DEBUG:teuthology.orchestra.run.smithi107:> if not os.path.exists(os.path.dirname(abs_path)): 2022-06-17T12:52:29.184 DEBUG:teuthology.orchestra.run.smithi107:> os.makedirs(os.path.dirname(abs_path)) 2022-06-17T12:52:29.185 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:52:29.185 DEBUG:teuthology.orchestra.run.smithi107:> for i in range(0, n): 2022-06-17T12:52:29.186 DEBUG:teuthology.orchestra.run.smithi107:> fname = "{0}_{1}".format(abs_path, i) 2022-06-17T12:52:29.186 DEBUG:teuthology.orchestra.run.smithi107:> with open(fname, '"'"'w'"'"') as f: 2022-06-17T12:52:29.187 DEBUG:teuthology.orchestra.run.smithi107:> f.write('"'"'content'"'"') 2022-06-17T12:52:29.187 DEBUG:teuthology.orchestra.run.smithi107:> if True: 2022-06-17T12:52:29.188 DEBUG:teuthology.orchestra.run.smithi107:> f.flush() 2022-06-17T12:52:29.188 DEBUG:teuthology.orchestra.run.smithi107:> os.fsync(f.fileno()) 2022-06-17T12:52:29.189 DEBUG:teuthology.orchestra.run.smithi107:> ' 2022-06-17T12:52:40.506 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:52:40.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.627+0000 7fb169d6a700 1 Processor -- start 2022-06-17T12:52:40.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.627+0000 7fb169d6a700 1 -- start start 2022-06-17T12:52:40.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.627+0000 7fb169d6a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb1640b0cb0 0x7fb164142f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:40.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.627+0000 7fb169d6a700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb164143e80 0x7fb164146970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:40.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.627+0000 7fb169d6a700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1640b44b0 con 0x7fb164062270 2022-06-17T12:52:40.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.627+0000 7fb169d6a700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1640b6650 con 0x7fb1640b0cb0 2022-06-17T12:52:40.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.627+0000 7fb169d6a700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb164062650 con 0x7fb164143e80 2022-06-17T12:52:40.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.627+0000 7fb15bfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb1640b0cb0 0x7fb164142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:40.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.627+0000 7fb15bfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb1640b0cb0 0x7fb164142f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35056/0 (socket says 172.21.15.36:35056) 2022-06-17T12:52:40.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.627+0000 7fb15bfff700 1 -- 172.21.15.36:0/4178935070 learned_addr learned my addr 172.21.15.36:0/4178935070 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:40.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.627+0000 7fb163fff700 1 --2- 172.21.15.36:0/4178935070 >> v2:172.21.15.70:3300/0 conn(0x7fb164143e80 0x7fb164146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:40.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.631+0000 7fb15bfff700 1 -- 172.21.15.36:0/4178935070 >> v2:172.21.15.70:3300/0 conn(0x7fb164143e80 msgr2=0x7fb164146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:40.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.631+0000 7fb15bfff700 1 --2- 172.21.15.36:0/4178935070 >> v2:172.21.15.70:3300/0 conn(0x7fb164143e80 0x7fb164146970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:40.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.631+0000 7fb15bfff700 1 -- 172.21.15.36:0/4178935070 >> v1:172.21.15.70:6789/0 conn(0x7fb164062270 legacy=0x7fb164143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:40.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.631+0000 7fb15bfff700 1 -- 172.21.15.36:0/4178935070 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1640b4210 con 0x7fb1640b0cb0 2022-06-17T12:52:40.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.631+0000 7fb15bfff700 1 --2- 172.21.15.36:0/4178935070 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb1640b0cb0 0x7fb164142f20 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7fb14c002f70 tx=0x7fb14c006eb0).ready entity=mon.0 client_cookie=a530a1d422e0f9ea server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:40.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.631+0000 7fb162ffd700 1 -- 172.21.15.36:0/4178935070 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb14c00d610 con 0x7fb1640b0cb0 2022-06-17T12:52:40.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.631+0000 7fb162ffd700 1 -- 172.21.15.36:0/4178935070 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14c000e90 con 0x7fb1640b0cb0 2022-06-17T12:52:40.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.631+0000 7fb162ffd700 1 -- 172.21.15.36:0/4178935070 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb14c00d9a0 con 0x7fb1640b0cb0 2022-06-17T12:52:40.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.631+0000 7fb169d6a700 1 -- 172.21.15.36:0/4178935070 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb1640b0cb0 msgr2=0x7fb164142f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:40.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.631+0000 7fb169d6a700 1 --2- 172.21.15.36:0/4178935070 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb1640b0cb0 0x7fb164142f20 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7fb14c002f70 tx=0x7fb14c006eb0).stop 2022-06-17T12:52:40.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.631+0000 7fb169d6a700 1 -- 172.21.15.36:0/4178935070 shutdown_connections 2022-06-17T12:52:40.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.631+0000 7fb169d6a700 1 --2- 172.21.15.36:0/4178935070 >> v2:172.21.15.70:3300/0 conn(0x7fb164143e80 0x7fb164146970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:40.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.631+0000 7fb169d6a700 1 --2- 172.21.15.36:0/4178935070 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb1640b0cb0 0x7fb164142f20 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:40.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.631+0000 7fb169d6a700 1 -- 172.21.15.36:0/4178935070 >> 172.21.15.36:0/4178935070 conn(0x7fb1640b9870 msgr2=0x7fb1640b9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:40.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.631+0000 7fb169d6a700 1 -- 172.21.15.36:0/4178935070 shutdown_connections 2022-06-17T12:52:40.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.631+0000 7fb169d6a700 1 -- 172.21.15.36:0/4178935070 wait complete. 2022-06-17T12:52:40.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.631+0000 7fb169d6a700 1 Processor -- start 2022-06-17T12:52:40.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.631+0000 7fb169d6a700 1 -- start start 2022-06-17T12:52:40.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.635+0000 7fb169d6a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb164062270 0x7fb164142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:40.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.635+0000 7fb169d6a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb164143e80 0x7fb164155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:40.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.635+0000 7fb169d6a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1640ae2d0 0x7fb1641561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:40.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.635+0000 7fb169d6a700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1640ad330 con 0x7fb164062270 2022-06-17T12:52:40.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.635+0000 7fb169d6a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1640af160 con 0x7fb1640ae2d0 2022-06-17T12:52:40.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.635+0000 7fb169d6a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1640ad560 con 0x7fb164143e80 2022-06-17T12:52:40.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.635+0000 7fb1637fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb164062270 0x7fb164142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:40.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.635+0000 7fb1637fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb164062270 0x7fb164142050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35062/0 (socket says 172.21.15.36:35062) 2022-06-17T12:52:40.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.635+0000 7fb163fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1640ae2d0 0x7fb1641561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:40.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.635+0000 7fb1637fe700 1 -- 172.21.15.36:0/4101239752 learned_addr learned my addr 172.21.15.36:0/4101239752 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:40.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.635+0000 7fb15bfff700 1 --2- 172.21.15.36:0/4101239752 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb164143e80 0x7fb164155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:40.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.635+0000 7fb1637fe700 1 -- 172.21.15.36:0/4101239752 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb164143e80 msgr2=0x7fb164155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:40.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.635+0000 7fb1637fe700 1 --2- 172.21.15.36:0/4101239752 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb164143e80 0x7fb164155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:40.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.635+0000 7fb1637fe700 1 -- 172.21.15.36:0/4101239752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1640ae2d0 msgr2=0x7fb1641561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:40.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.635+0000 7fb1637fe700 1 --2- 172.21.15.36:0/4101239752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1640ae2d0 0x7fb1641561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:40.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.635+0000 7fb1637fe700 1 -- 172.21.15.36:0/4101239752 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14c00b1e0 con 0x7fb164062270 2022-06-17T12:52:40.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.635+0000 7fb1637fe700 1 --2- 172.21.15.36:0/4101239752 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb164062270 0x7fb164142050 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7fb1640608b0 tx=0x7fb154007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:40.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.635+0000 7fb1617fa700 1 -- 172.21.15.36:0/4101239752 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb154007030 con 0x7fb164062270 2022-06-17T12:52:40.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.635+0000 7fb1617fa700 1 -- 172.21.15.36:0/4101239752 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb154007190 con 0x7fb164062270 2022-06-17T12:52:40.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.635+0000 7fb169d6a700 1 -- 172.21.15.36:0/4101239752 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1640af3c0 con 0x7fb164062270 2022-06-17T12:52:40.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.635+0000 7fb1617fa700 1 -- 172.21.15.36:0/4101239752 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb154006510 con 0x7fb164062270 2022-06-17T12:52:40.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.635+0000 7fb169d6a700 1 -- 172.21.15.36:0/4101239752 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb164156690 con 0x7fb164062270 2022-06-17T12:52:40.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.635+0000 7fb169d6a700 1 -- 172.21.15.36:0/4101239752 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1640b63b0 con 0x7fb164062270 2022-06-17T12:52:40.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.639+0000 7fb1617fa700 1 -- 172.21.15.36:0/4101239752 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fb154010030 con 0x7fb164062270 2022-06-17T12:52:40.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.639+0000 7fb1617fa700 1 --2- 172.21.15.36:0/4101239752 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb148053af0 0x7fb148055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:40.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.639+0000 7fb1617fa700 1 -- 172.21.15.36:0/4101239752 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb15406bc30 con 0x7fb164062270 2022-06-17T12:52:40.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.639+0000 7fb15bfff700 1 --2- 172.21.15.36:0/4101239752 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb148053af0 0x7fb148055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:40.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.639+0000 7fb15bfff700 1 --2- 172.21.15.36:0/4101239752 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb148053af0 0x7fb148055fb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7fb14c01c000 tx=0x7fb14c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:40.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.643+0000 7fb1617fa700 1 -- 172.21.15.36:0/4101239752 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb1540428a0 con 0x7fb164062270 2022-06-17T12:52:40.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.835+0000 7fb169d6a700 1 -- 172.21.15.36:0/4101239752 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb164057410 con 0x7fb164062270 2022-06-17T12:52:40.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.835+0000 7fb1617fa700 1 -- 172.21.15.36:0/4101239752 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2140 (secure 0 0 0) 0x7fb154045ec0 con 0x7fb164062270 2022-06-17T12:52:40.824 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:52:40.824 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":30,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5420,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/3105680199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3105680199},{"type":"v1","addr":"172.21.15.70:6833","nonce":3105680199}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":29}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:52:22.253232+0000","modified":"2022-06-17T12:52:23.257219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6054},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6054":{"gid":6054,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/4091208534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":4091208534},{"type":"v1","addr":"172.21.15.36:6835","nonce":4091208534}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-06-17T12:52:40.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.839+0000 7fb169d6a700 1 -- 172.21.15.36:0/4101239752 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb148053af0 msgr2=0x7fb148055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:40.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.839+0000 7fb169d6a700 1 --2- 172.21.15.36:0/4101239752 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb148053af0 0x7fb148055fb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7fb14c01c000 tx=0x7fb14c015000).stop 2022-06-17T12:52:40.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.839+0000 7fb169d6a700 1 -- 172.21.15.36:0/4101239752 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb164062270 msgr2=0x7fb164142050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:40.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.839+0000 7fb169d6a700 1 --2- 172.21.15.36:0/4101239752 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb164062270 0x7fb164142050 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7fb1640608b0 tx=0x7fb154007c10).stop 2022-06-17T12:52:40.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.839+0000 7fb169d6a700 1 -- 172.21.15.36:0/4101239752 shutdown_connections 2022-06-17T12:52:40.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.839+0000 7fb169d6a700 1 --2- 172.21.15.36:0/4101239752 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb148053af0 0x7fb148055fb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:40.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.839+0000 7fb169d6a700 1 --2- 172.21.15.36:0/4101239752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1640ae2d0 0x7fb1641561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:40.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.839+0000 7fb169d6a700 1 --2- 172.21.15.36:0/4101239752 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb164143e80 0x7fb164155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:40.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.839+0000 7fb169d6a700 1 --2- 172.21.15.36:0/4101239752 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb164062270 0x7fb164142050 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:40.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.839+0000 7fb169d6a700 1 -- 172.21.15.36:0/4101239752 >> 172.21.15.36:0/4101239752 conn(0x7fb1640b9870 msgr2=0x7fb1640b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:40.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.839+0000 7fb169d6a700 1 -- 172.21.15.36:0/4101239752 shutdown_connections 2022-06-17T12:52:40.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:40.839+0000 7fb169d6a700 1 -- 172.21.15.36:0/4101239752 wait complete. 2022-06-17T12:52:40.830 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 30 2022-06-17T12:52:40.837 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_caps 20 2022-06-17T12:52:40.940 INFO:teuthology.orchestra.run.smithi036.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2022-06-17T12:52:40.941 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2022-06-17T12:52:40.942 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:52:41.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.051+0000 7f6696939700 1 Processor -- start 2022-06-17T12:52:41.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.051+0000 7f6696939700 1 -- start start 2022-06-17T12:52:41.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6696939700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66900b2200 0x7f66900af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6696939700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f66900af8d0 0x7f669014a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6696939700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66900b25e0 con 0x7f66900b2cd0 2022-06-17T12:52:41.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6696939700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66900ad790 con 0x7f66900af8d0 2022-06-17T12:52:41.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6696939700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66900aef40 con 0x7f66900b2200 2022-06-17T12:52:41.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6696138700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f66900b2cd0 0x7f669014adc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52604/0 (socket says 172.21.15.36:52604) 2022-06-17T12:52:41.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6696138700 1 -- 172.21.15.36:0/4102312688 learned_addr learned my addr 172.21.15.36:0/4102312688 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:41.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6695136700 1 --2- 172.21.15.36:0/4102312688 >> v2:172.21.15.70:3300/0 conn(0x7f66900af8d0 0x7f669014a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6695937700 1 --2- 172.21.15.36:0/4102312688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66900b2200 0x7f66900af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6694935700 1 -- 172.21.15.36:0/4102312688 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3662575006 0 0) 0x7f66900b25e0 con 0x7f66900b2cd0 2022-06-17T12:52:41.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6694935700 1 -- 172.21.15.36:0/4102312688 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6678003260 con 0x7f66900b2cd0 2022-06-17T12:52:41.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6695937700 1 -- 172.21.15.36:0/4102312688 >> v2:172.21.15.70:3300/0 conn(0x7f66900af8d0 msgr2=0x7f669014a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:41.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6695937700 1 --2- 172.21.15.36:0/4102312688 >> v2:172.21.15.70:3300/0 conn(0x7f66900af8d0 0x7f669014a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6695937700 1 -- 172.21.15.36:0/4102312688 >> v1:172.21.15.36:6789/0 conn(0x7f66900b2cd0 legacy=0x7f669014adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:41.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6695937700 1 -- 172.21.15.36:0/4102312688 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66900b1950 con 0x7f66900b2200 2022-06-17T12:52:41.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6695937700 1 --2- 172.21.15.36:0/4102312688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66900b2200 0x7f66900af3e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f6680002f70 tx=0x7f6680006df0).ready entity=mon.2 client_cookie=5a1f4cf0916cb20f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:41.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6694935700 1 -- 172.21.15.36:0/4102312688 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f668000d4a0 con 0x7f66900b2200 2022-06-17T12:52:41.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6694935700 1 -- 172.21.15.36:0/4102312688 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6680000e90 con 0x7f66900b2200 2022-06-17T12:52:41.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6694935700 1 -- 172.21.15.36:0/4102312688 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f668000d7f0 con 0x7f66900b2200 2022-06-17T12:52:41.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6696939700 1 -- 172.21.15.36:0/4102312688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66900b2200 msgr2=0x7f66900af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:41.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6696939700 1 --2- 172.21.15.36:0/4102312688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66900b2200 0x7f66900af3e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f6680002f70 tx=0x7f6680006df0).stop 2022-06-17T12:52:41.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6696939700 1 -- 172.21.15.36:0/4102312688 shutdown_connections 2022-06-17T12:52:41.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6696939700 1 --2- 172.21.15.36:0/4102312688 >> v2:172.21.15.70:3300/0 conn(0x7f66900af8d0 0x7f669014a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6696939700 1 --2- 172.21.15.36:0/4102312688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66900b2200 0x7f66900af3e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6696939700 1 -- 172.21.15.36:0/4102312688 >> 172.21.15.36:0/4102312688 conn(0x7f669005f770 msgr2=0x7f669005fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:41.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6696939700 1 -- 172.21.15.36:0/4102312688 shutdown_connections 2022-06-17T12:52:41.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.055+0000 7f6696939700 1 -- 172.21.15.36:0/4102312688 wait complete. 2022-06-17T12:52:41.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f6696939700 1 Processor -- start 2022-06-17T12:52:41.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f6696939700 1 -- start start 2022-06-17T12:52:41.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f6696939700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f66900af8d0 0x7f6690149ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f6696939700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66900b2200 0x7f6690142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f6696939700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66900b2cd0 0x7f6690159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f6696939700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6690145b90 con 0x7f66900af8d0 2022-06-17T12:52:41.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f6696939700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66900a9720 con 0x7f66900b2cd0 2022-06-17T12:52:41.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f6696939700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66900a99c0 con 0x7f66900b2200 2022-06-17T12:52:41.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f6695937700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f66900af8d0 0x7f6690149ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f6695937700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f66900af8d0 0x7f6690149ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35076/0 (socket says 172.21.15.36:35076) 2022-06-17T12:52:41.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f6695937700 1 -- 172.21.15.36:0/2730155377 learned_addr learned my addr 172.21.15.36:0/2730155377 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:41.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f6696138700 1 --2- 172.21.15.36:0/2730155377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66900b2cd0 0x7f6690159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f6695136700 1 --2- 172.21.15.36:0/2730155377 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66900b2200 0x7f6690142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f6695937700 1 -- 172.21.15.36:0/2730155377 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66900b2200 msgr2=0x7f6690142250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:41.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f6695937700 1 --2- 172.21.15.36:0/2730155377 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66900b2200 0x7f6690142250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f6695937700 1 -- 172.21.15.36:0/2730155377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66900b2cd0 msgr2=0x7f6690159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:41.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f6695937700 1 --2- 172.21.15.36:0/2730155377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66900b2cd0 0x7f6690159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f6695937700 1 -- 172.21.15.36:0/2730155377 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668000b180 con 0x7f66900af8d0 2022-06-17T12:52:41.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f6696138700 1 --2- 172.21.15.36:0/2730155377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66900b2cd0 0x7f6690159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:52:41.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f6695937700 1 --2- 172.21.15.36:0/2730155377 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f66900af8d0 0x7f6690149ea0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f6680002f70 tx=0x7f6680014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:41.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f667effd700 1 -- 172.21.15.36:0/2730155377 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6680016070 con 0x7f66900af8d0 2022-06-17T12:52:41.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f667effd700 1 -- 172.21.15.36:0/2730155377 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6680013a40 con 0x7f66900af8d0 2022-06-17T12:52:41.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f6696939700 1 -- 172.21.15.36:0/2730155377 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66900a70e0 con 0x7f66900af8d0 2022-06-17T12:52:41.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f667effd700 1 -- 172.21.15.36:0/2730155377 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6680013da0 con 0x7f66900af8d0 2022-06-17T12:52:41.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.059+0000 7f6696939700 1 -- 172.21.15.36:0/2730155377 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66901428e0 con 0x7f66900af8d0 2022-06-17T12:52:41.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.063+0000 7f667effd700 1 -- 172.21.15.36:0/2730155377 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6680025030 con 0x7f66900af8d0 2022-06-17T12:52:41.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.063+0000 7f6696939700 1 -- 172.21.15.36:0/2730155377 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66900a7a20 con 0x7f66900af8d0 2022-06-17T12:52:41.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.063+0000 7f667effd700 1 --2- 172.21.15.36:0/2730155377 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6670053a40 0x7f6670055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.063+0000 7f667effd700 1 -- 172.21.15.36:0/2730155377 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f668006cf10 con 0x7f66900af8d0 2022-06-17T12:52:41.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.063+0000 7f6695136700 1 --2- 172.21.15.36:0/2730155377 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6670053a40 0x7f6670055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.063+0000 7f6695136700 1 --2- 172.21.15.36:0/2730155377 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6670053a40 0x7f6670055f00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f668c00b000 tx=0x7f668c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:41.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.067+0000 7f667effd700 1 -- 172.21.15.36:0/2730155377 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f66900a7a20 con 0x7f66900af8d0 2022-06-17T12:52:41.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.231+0000 7f6696939700 1 -- 172.21.15.36:0/2730155377 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66900a7a20 con 0x7f66900af8d0 2022-06-17T12:52:41.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.231+0000 7f667effd700 1 -- 172.21.15.36:0/2730155377 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f6680023870 con 0x7f66900af8d0 2022-06-17T12:52:41.218 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:52:41.218 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":30,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5420,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/3105680199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3105680199},{"type":"v1","addr":"172.21.15.70:6833","nonce":3105680199}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":29}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:52:22.253232+0000","modified":"2022-06-17T12:52:23.257219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6054},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6054":{"gid":6054,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/4091208534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":4091208534},{"type":"v1","addr":"172.21.15.36:6835","nonce":4091208534}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-06-17T12:52:41.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.231+0000 7f6696939700 1 -- 172.21.15.36:0/2730155377 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6670053a40 msgr2=0x7f6670055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:41.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.231+0000 7f6696939700 1 --2- 172.21.15.36:0/2730155377 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6670053a40 0x7f6670055f00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f668c00b000 tx=0x7f668c008040).stop 2022-06-17T12:52:41.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.231+0000 7f6696939700 1 -- 172.21.15.36:0/2730155377 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f66900af8d0 msgr2=0x7f6690149ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:41.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.231+0000 7f6696939700 1 --2- 172.21.15.36:0/2730155377 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f66900af8d0 0x7f6690149ea0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f6680002f70 tx=0x7f6680014000).stop 2022-06-17T12:52:41.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.231+0000 7f6696939700 1 -- 172.21.15.36:0/2730155377 shutdown_connections 2022-06-17T12:52:41.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.231+0000 7f6696939700 1 --2- 172.21.15.36:0/2730155377 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6670053a40 0x7f6670055f00 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.231+0000 7f6696939700 1 --2- 172.21.15.36:0/2730155377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66900b2cd0 0x7f6690159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.231+0000 7f6696939700 1 --2- 172.21.15.36:0/2730155377 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66900b2200 0x7f6690142250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.231+0000 7f6696939700 1 --2- 172.21.15.36:0/2730155377 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f66900af8d0 0x7f6690149ea0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.231+0000 7f6696939700 1 -- 172.21.15.36:0/2730155377 >> 172.21.15.36:0/2730155377 conn(0x7f669005f770 msgr2=0x7f6690147e80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:41.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.231+0000 7f6696939700 1 -- 172.21.15.36:0/2730155377 shutdown_connections 2022-06-17T12:52:41.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.231+0000 7f6696939700 1 -- 172.21.15.36:0/2730155377 wait complete. 2022-06-17T12:52:41.223 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 30 2022-06-17T12:52:41.231 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_decay_threshold 400 2022-06-17T12:52:41.335 INFO:teuthology.orchestra.run.smithi036.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2022-06-17T12:52:41.336 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2022-06-17T12:52:41.336 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:52:41.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.451+0000 7f49559f8700 1 Processor -- start 2022-06-17T12:52:41.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.451+0000 7f49559f8700 1 -- start start 2022-06-17T12:52:41.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.455+0000 7f49559f8700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f495013cf50 0x7f49501442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.455+0000 7f49559f8700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49500af4b0 con 0x7f49500aef60 2022-06-17T12:52:41.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.455+0000 7f49559f8700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49500ba500 con 0x7f495013cb70 2022-06-17T12:52:41.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.455+0000 7f49559f8700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4950143da0 con 0x7f495013cf50 2022-06-17T12:52:41.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.455+0000 7f494ffff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f49500aef60 0x7f49500b9df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52616/0 (socket says 172.21.15.36:52616) 2022-06-17T12:52:41.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.455+0000 7f494ffff700 1 -- 172.21.15.36:0/2723932372 learned_addr learned my addr 172.21.15.36:0/2723932372 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:41.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.455+0000 7f49551f7700 1 --2- 172.21.15.36:0/2723932372 >> v2:172.21.15.36:3300/0 conn(0x7f495013cf50 0x7f49501442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.455+0000 7f494f7fe700 1 -- 172.21.15.36:0/2723932372 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 744846990 0 0) 0x7f49500af4b0 con 0x7f49500aef60 2022-06-17T12:52:41.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.455+0000 7f494f7fe700 1 -- 172.21.15.36:0/2723932372 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f493c003260 con 0x7f49500aef60 2022-06-17T12:52:41.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.455+0000 7f494f7fe700 1 -- 172.21.15.36:0/2723932372 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1067269466 0 0) 0x7f49500ba500 con 0x7f495013cb70 2022-06-17T12:52:41.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.455+0000 7f494f7fe700 1 -- 172.21.15.36:0/2723932372 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49500af4b0 con 0x7f495013cb70 2022-06-17T12:52:41.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.455+0000 7f49551f7700 1 -- 172.21.15.36:0/2723932372 >> v1:172.21.15.70:6789/0 conn(0x7f495013cb70 legacy=0x7f4950143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:41.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.455+0000 7f49551f7700 1 -- 172.21.15.36:0/2723932372 >> v1:172.21.15.36:6789/0 conn(0x7f49500aef60 legacy=0x7f49500b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:41.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.455+0000 7f49551f7700 1 -- 172.21.15.36:0/2723932372 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4950142f50 con 0x7f495013cf50 2022-06-17T12:52:41.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.455+0000 7f49551f7700 1 --2- 172.21.15.36:0/2723932372 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f495013cf50 0x7f49501442a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f4940002f70 tx=0x7f4940006df0).ready entity=mon.0 client_cookie=6e92e2afa0770ee5 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:41.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.455+0000 7f494f7fe700 1 -- 172.21.15.36:0/2723932372 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f494000d540 con 0x7f495013cf50 2022-06-17T12:52:41.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.455+0000 7f494f7fe700 1 -- 172.21.15.36:0/2723932372 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4940000e90 con 0x7f495013cf50 2022-06-17T12:52:41.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.455+0000 7f494f7fe700 1 -- 172.21.15.36:0/2723932372 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f494000d890 con 0x7f495013cf50 2022-06-17T12:52:41.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.455+0000 7f49559f8700 1 -- 172.21.15.36:0/2723932372 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f495013cf50 msgr2=0x7f49501442a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:41.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.455+0000 7f49559f8700 1 --2- 172.21.15.36:0/2723932372 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f495013cf50 0x7f49501442a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f4940002f70 tx=0x7f4940006df0).stop 2022-06-17T12:52:41.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.455+0000 7f49559f8700 1 -- 172.21.15.36:0/2723932372 shutdown_connections 2022-06-17T12:52:41.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.455+0000 7f49559f8700 1 --2- 172.21.15.36:0/2723932372 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f495013cf50 0x7f49501442a0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.455+0000 7f49559f8700 1 -- 172.21.15.36:0/2723932372 >> 172.21.15.36:0/2723932372 conn(0x7f495005c550 msgr2=0x7f49500b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:41.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.459+0000 7f49559f8700 1 -- 172.21.15.36:0/2723932372 shutdown_connections 2022-06-17T12:52:41.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.459+0000 7f49559f8700 1 -- 172.21.15.36:0/2723932372 wait complete. 2022-06-17T12:52:41.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.459+0000 7f49559f8700 1 Processor -- start 2022-06-17T12:52:41.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.459+0000 7f49559f8700 1 -- start start 2022-06-17T12:52:41.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.459+0000 7f49559f8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49500aef60 0x7f495013c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.459+0000 7f49559f8700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f495013cb70 0x7f4950141a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.459+0000 7f49559f8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f495013cf50 0x7f4950141f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.459+0000 7f49559f8700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495013fc10 con 0x7f495013cf50 2022-06-17T12:52:41.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.459+0000 7f49559f8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495013feb0 con 0x7f49500aef60 2022-06-17T12:52:41.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.459+0000 7f49559f8700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4950140150 con 0x7f495013cb70 2022-06-17T12:52:41.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.459+0000 7f49551f7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f495013cf50 0x7f4950141f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.459+0000 7f49551f7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f495013cf50 0x7f4950141f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35094/0 (socket says 172.21.15.36:35094) 2022-06-17T12:52:41.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.459+0000 7f49551f7700 1 -- 172.21.15.36:0/2156216595 learned_addr learned my addr 172.21.15.36:0/2156216595 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:41.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.459+0000 7f49549f6700 1 --2- 172.21.15.36:0/2156216595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49500aef60 0x7f495013c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.459+0000 7f494ffff700 1 --2- 172.21.15.36:0/2156216595 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f495013cb70 0x7f4950141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.459+0000 7f49551f7700 1 -- 172.21.15.36:0/2156216595 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f495013cb70 msgr2=0x7f4950141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:41.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.459+0000 7f49551f7700 1 --2- 172.21.15.36:0/2156216595 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f495013cb70 0x7f4950141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.459+0000 7f49551f7700 1 -- 172.21.15.36:0/2156216595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49500aef60 msgr2=0x7f495013c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:41.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.459+0000 7f49551f7700 1 --2- 172.21.15.36:0/2156216595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49500aef60 0x7f495013c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.459+0000 7f49551f7700 1 -- 172.21.15.36:0/2156216595 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494000b180 con 0x7f495013cf50 2022-06-17T12:52:41.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.463+0000 7f49551f7700 1 --2- 172.21.15.36:0/2156216595 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f495013cf50 0x7f4950141f00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f4940002f70 tx=0x7f4940014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:41.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.463+0000 7f494dffb700 1 -- 172.21.15.36:0/2156216595 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4940016070 con 0x7f495013cf50 2022-06-17T12:52:41.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.463+0000 7f494dffb700 1 -- 172.21.15.36:0/2156216595 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49400137d0 con 0x7f495013cf50 2022-06-17T12:52:41.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.463+0000 7f49559f8700 1 -- 172.21.15.36:0/2156216595 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49501405f0 con 0x7f495013cf50 2022-06-17T12:52:41.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.463+0000 7f494dffb700 1 -- 172.21.15.36:0/2156216595 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4940013b30 con 0x7f495013cf50 2022-06-17T12:52:41.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.463+0000 7f49559f8700 1 -- 172.21.15.36:0/2156216595 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f495013f850 con 0x7f495013cf50 2022-06-17T12:52:41.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.463+0000 7f49559f8700 1 -- 172.21.15.36:0/2156216595 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f495013f490 con 0x7f495013cf50 2022-06-17T12:52:41.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.463+0000 7f494dffb700 1 -- 172.21.15.36:0/2156216595 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f4940025030 con 0x7f495013cf50 2022-06-17T12:52:41.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.463+0000 7f494dffb700 1 --2- 172.21.15.36:0/2156216595 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4930053920 0x7f4930055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.463+0000 7f494dffb700 1 -- 172.21.15.36:0/2156216595 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f494006cdc0 con 0x7f495013cf50 2022-06-17T12:52:41.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.463+0000 7f49549f6700 1 --2- 172.21.15.36:0/2156216595 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4930053920 0x7f4930055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.463+0000 7f49549f6700 1 --2- 172.21.15.36:0/2156216595 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4930053920 0x7f4930055de0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f494400d000 tx=0x7f4944007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:41.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.471+0000 7f494dffb700 1 -- 172.21.15.36:0/2156216595 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f495013f490 con 0x7f495013cf50 2022-06-17T12:52:41.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.635+0000 7f49559f8700 1 -- 172.21.15.36:0/2156216595 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f495013fa70 con 0x7f495013cf50 2022-06-17T12:52:41.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.635+0000 7f494dffb700 1 -- 172.21.15.36:0/2156216595 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f495013fa70 con 0x7f495013cf50 2022-06-17T12:52:41.623 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:52:41.624 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":30,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5420,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/3105680199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3105680199},{"type":"v1","addr":"172.21.15.70:6833","nonce":3105680199}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":29}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:52:22.253232+0000","modified":"2022-06-17T12:52:23.257219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6054},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6054":{"gid":6054,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/4091208534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":4091208534},{"type":"v1","addr":"172.21.15.36:6835","nonce":4091208534}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-06-17T12:52:41.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.639+0000 7f49559f8700 1 -- 172.21.15.36:0/2156216595 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4930053920 msgr2=0x7f4930055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:41.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.639+0000 7f49559f8700 1 --2- 172.21.15.36:0/2156216595 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4930053920 0x7f4930055de0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f494400d000 tx=0x7f4944007a00).stop 2022-06-17T12:52:41.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.639+0000 7f49559f8700 1 -- 172.21.15.36:0/2156216595 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f495013cf50 msgr2=0x7f4950141f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:41.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.639+0000 7f49559f8700 1 --2- 172.21.15.36:0/2156216595 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f495013cf50 0x7f4950141f00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f4940002f70 tx=0x7f4940014000).stop 2022-06-17T12:52:41.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.639+0000 7f49559f8700 1 -- 172.21.15.36:0/2156216595 shutdown_connections 2022-06-17T12:52:41.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.639+0000 7f49559f8700 1 --2- 172.21.15.36:0/2156216595 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4930053920 0x7f4930055de0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.639+0000 7f49559f8700 1 --2- 172.21.15.36:0/2156216595 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f495013cf50 0x7f4950141f00 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.639+0000 7f49559f8700 1 --2- 172.21.15.36:0/2156216595 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f495013cb70 0x7f4950141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.639+0000 7f49559f8700 1 --2- 172.21.15.36:0/2156216595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49500aef60 0x7f495013c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.639+0000 7f49559f8700 1 -- 172.21.15.36:0/2156216595 >> 172.21.15.36:0/2156216595 conn(0x7f495005c550 msgr2=0x7f495005ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:41.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.639+0000 7f49559f8700 1 -- 172.21.15.36:0/2156216595 shutdown_connections 2022-06-17T12:52:41.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.639+0000 7f49559f8700 1 -- 172.21.15.36:0/2156216595 wait complete. 2022-06-17T12:52:41.629 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 30 2022-06-17T12:52:41.637 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a cache drop 10 2022-06-17T12:52:41.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.771+0000 7fafce6dd700 1 Processor -- start 2022-06-17T12:52:41.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.771+0000 7fafce6dd700 1 -- start start 2022-06-17T12:52:41.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.771+0000 7fafce6dd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafc80b18e0 0x7fafc8143860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.771+0000 7fafce6dd700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc80b2b70 con 0x7fafc80b6860 2022-06-17T12:52:41.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.771+0000 7fafce6dd700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc80b0d30 con 0x7fafc80b39a0 2022-06-17T12:52:41.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.771+0000 7fafce6dd700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc80b6c40 con 0x7fafc80b18e0 2022-06-17T12:52:41.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.771+0000 7fafcd6db700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fafc80b6860 0x7fafc8143150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52628/0 (socket says 172.21.15.36:52628) 2022-06-17T12:52:41.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.771+0000 7fafcd6db700 1 -- 172.21.15.36:0/1927660581 learned_addr learned my addr 172.21.15.36:0/1927660581 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:41.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.771+0000 7fafcceda700 1 --2- 172.21.15.36:0/1927660581 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafc80b18e0 0x7fafc8143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.771+0000 7fafbffff700 1 -- 172.21.15.36:0/1927660581 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1995534793 0 0) 0x7fafc80b2b70 con 0x7fafc80b6860 2022-06-17T12:52:41.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.771+0000 7fafbffff700 1 -- 172.21.15.36:0/1927660581 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafac003260 con 0x7fafc80b6860 2022-06-17T12:52:41.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.771+0000 7fafcceda700 1 -- 172.21.15.36:0/1927660581 >> v1:172.21.15.70:6789/0 conn(0x7fafc80b39a0 legacy=0x7fafc813fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:41.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.771+0000 7fafcceda700 1 -- 172.21.15.36:0/1927660581 >> v1:172.21.15.36:6789/0 conn(0x7fafc80b6860 legacy=0x7fafc8143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:41.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.771+0000 7fafbffff700 1 -- 172.21.15.36:0/1927660581 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1152570448 0 0) 0x7fafc80b0d30 con 0x7fafc80b39a0 2022-06-17T12:52:41.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.771+0000 7fafcceda700 1 -- 172.21.15.36:0/1927660581 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc80b0fd0 con 0x7fafc80b18e0 2022-06-17T12:52:41.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.771+0000 7fafcceda700 1 --2- 172.21.15.36:0/1927660581 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafc80b18e0 0x7fafc8143860 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fafc4001370 tx=0x7fafb8006d90).ready entity=mon.2 client_cookie=dc0d64e7768513a0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:41.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.771+0000 7fafbffff700 1 -- 172.21.15.36:0/1927660581 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafb800e470 con 0x7fafc80b18e0 2022-06-17T12:52:41.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.771+0000 7fafbffff700 1 -- 172.21.15.36:0/1927660581 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb8000e90 con 0x7fafc80b18e0 2022-06-17T12:52:41.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.771+0000 7fafbffff700 1 -- 172.21.15.36:0/1927660581 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafb800e7c0 con 0x7fafc80b18e0 2022-06-17T12:52:41.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.775+0000 7fafce6dd700 1 -- 172.21.15.36:0/1927660581 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafc80b18e0 msgr2=0x7fafc8143860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:41.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.775+0000 7fafce6dd700 1 --2- 172.21.15.36:0/1927660581 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafc80b18e0 0x7fafc8143860 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fafc4001370 tx=0x7fafb8006d90).stop 2022-06-17T12:52:41.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.775+0000 7fafce6dd700 1 -- 172.21.15.36:0/1927660581 shutdown_connections 2022-06-17T12:52:41.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.775+0000 7fafce6dd700 1 --2- 172.21.15.36:0/1927660581 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafc80b18e0 0x7fafc8143860 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.775+0000 7fafce6dd700 1 -- 172.21.15.36:0/1927660581 >> 172.21.15.36:0/1927660581 conn(0x7fafc80614b0 msgr2=0x7fafc8061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:41.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.775+0000 7fafce6dd700 1 -- 172.21.15.36:0/1927660581 shutdown_connections 2022-06-17T12:52:41.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.775+0000 7fafce6dd700 1 -- 172.21.15.36:0/1927660581 wait complete. 2022-06-17T12:52:41.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.775+0000 7fafce6dd700 1 Processor -- start 2022-06-17T12:52:41.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.775+0000 7fafce6dd700 1 -- start start 2022-06-17T12:52:41.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.775+0000 7fafce6dd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fafc80b18e0 0x7fafc813f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.775+0000 7fafce6dd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafc80b39a0 0x7fafc80650d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.775+0000 7fafce6dd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafc80b6860 0x7fafc8152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.775+0000 7fafce6dd700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc8155790 con 0x7fafc80b18e0 2022-06-17T12:52:41.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.775+0000 7fafce6dd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc80ae7c0 con 0x7fafc80b39a0 2022-06-17T12:52:41.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.775+0000 7fafce6dd700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc8141980 con 0x7fafc80b6860 2022-06-17T12:52:41.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.775+0000 7fafcd6db700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fafc80b18e0 0x7fafc813f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.775+0000 7fafcd6db700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fafc80b18e0 0x7fafc813f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35104/0 (socket says 172.21.15.36:35104) 2022-06-17T12:52:41.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.775+0000 7fafcd6db700 1 -- 172.21.15.36:0/1789104552 learned_addr learned my addr 172.21.15.36:0/1789104552 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:41.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.775+0000 7fafcdedc700 1 --2- 172.21.15.36:0/1789104552 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafc80b6860 0x7fafc8152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.775+0000 7fafcceda700 1 --2- 172.21.15.36:0/1789104552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafc80b39a0 0x7fafc80650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.779+0000 7fafcd6db700 1 -- 172.21.15.36:0/1789104552 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafc80b6860 msgr2=0x7fafc8152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:41.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.779+0000 7fafcd6db700 1 --2- 172.21.15.36:0/1789104552 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafc80b6860 0x7fafc8152120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.779+0000 7fafcd6db700 1 -- 172.21.15.36:0/1789104552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafc80b39a0 msgr2=0x7fafc80650d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:41.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.779+0000 7fafcd6db700 1 --2- 172.21.15.36:0/1789104552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafc80b39a0 0x7fafc80650d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.779+0000 7fafcd6db700 1 -- 172.21.15.36:0/1789104552 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb800b120 con 0x7fafc80b18e0 2022-06-17T12:52:41.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.779+0000 7fafcceda700 1 --2- 172.21.15.36:0/1789104552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafc80b39a0 0x7fafc80650d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:52:41.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.779+0000 7fafcdedc700 1 --2- 172.21.15.36:0/1789104552 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafc80b6860 0x7fafc8152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:52:41.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.779+0000 7fafcd6db700 1 --2- 172.21.15.36:0/1789104552 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fafc80b18e0 0x7fafc813f600 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7fafc4000bc0 tx=0x7fafc4007a70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:41.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.779+0000 7fafbe7fc700 1 -- 172.21.15.36:0/1789104552 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafc4007170 con 0x7fafc80b18e0 2022-06-17T12:52:41.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.779+0000 7fafbe7fc700 1 -- 172.21.15.36:0/1789104552 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc40072d0 con 0x7fafc80b18e0 2022-06-17T12:52:41.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.779+0000 7fafce6dd700 1 -- 172.21.15.36:0/1789104552 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc80aea60 con 0x7fafc80b18e0 2022-06-17T12:52:41.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.779+0000 7fafbe7fc700 1 -- 172.21.15.36:0/1789104552 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafc4004920 con 0x7fafc80b18e0 2022-06-17T12:52:41.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.779+0000 7fafce6dd700 1 -- 172.21.15.36:0/1789104552 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc8155bc0 con 0x7fafc80b18e0 2022-06-17T12:52:41.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.779+0000 7fafbe7fc700 1 -- 172.21.15.36:0/1789104552 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fafc4004a80 con 0x7fafc80b18e0 2022-06-17T12:52:41.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.779+0000 7fafbe7fc700 1 --2- 172.21.15.36:0/1789104552 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fafb0053a60 0x7fafb0055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.779+0000 7fafcceda700 1 --2- 172.21.15.36:0/1789104552 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fafb0053a60 0x7fafb0055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.779+0000 7fafbe7fc700 1 -- 172.21.15.36:0/1789104552 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fafc406bb80 con 0x7fafc80b18e0 2022-06-17T12:52:41.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafd9b94740 1 Processor -- start 2022-06-17T12:52:41.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafd9b94740 1 -- start start 2022-06-17T12:52:41.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafcceda700 1 --2- 172.21.15.36:0/1789104552 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fafb0053a60 0x7fafb0055f20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fafb8018000 tx=0x7fafb8014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:41.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafd9b94740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x21779d0 0x217ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafd9b94740 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x2177db0 0x21791a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafd9b94740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x2179690 0x21b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafd9b94740 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x20ebe60 con 0x2177db0 2022-06-17T12:52:41.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafd9b94740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x20cc0c0 con 0x2179690 2022-06-17T12:52:41.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafd9b94740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x21789a0 con 0x21779d0 2022-06-17T12:52:41.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafcceda700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x2177db0 0x21791a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafcceda700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x2177db0 0x21791a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35114/0 (socket says 172.21.15.36:35114) 2022-06-17T12:52:41.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafcceda700 1 -- 172.21.15.36:0/3328347411 learned_addr learned my addr 172.21.15.36:0/3328347411 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:41.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafcd6db700 1 --2- 172.21.15.36:0/3328347411 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x21779d0 0x217ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafcdedc700 1 --2- 172.21.15.36:0/3328347411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x2179690 0x21b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafcceda700 1 -- 172.21.15.36:0/3328347411 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x21779d0 msgr2=0x217ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:41.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafcceda700 1 --2- 172.21.15.36:0/3328347411 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x21779d0 0x217ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafcceda700 1 -- 172.21.15.36:0/3328347411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x2179690 msgr2=0x21b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:41.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafcceda700 1 --2- 172.21.15.36:0/3328347411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x2179690 0x21b14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafcceda700 1 -- 172.21.15.36:0/3328347411 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2123720 con 0x2177db0 2022-06-17T12:52:41.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafcceda700 1 --2- 172.21.15.36:0/3328347411 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x2177db0 0x21791a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x217c2c0 tx=0x7fafb801e000).ready entity=mon.0 client_cookie=ddc7b4144ce3ff68 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:41.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafce6dd700 1 -- 172.21.15.36:0/3328347411 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafb801e7f0 con 0x2177db0 2022-06-17T12:52:41.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafce6dd700 1 -- 172.21.15.36:0/3328347411 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb801e950 con 0x2177db0 2022-06-17T12:52:41.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafce6dd700 1 -- 172.21.15.36:0/3328347411 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafb801ecb0 con 0x2177db0 2022-06-17T12:52:41.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafd9b94740 1 -- 172.21.15.36:0/3328347411 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x2177db0 msgr2=0x21791a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:41.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafd9b94740 1 --2- 172.21.15.36:0/3328347411 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x2177db0 0x21791a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x217c2c0 tx=0x7fafb801e000).stop 2022-06-17T12:52:41.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafd9b94740 1 -- 172.21.15.36:0/3328347411 shutdown_connections 2022-06-17T12:52:41.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafd9b94740 1 --2- 172.21.15.36:0/3328347411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x2179690 0x21b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafd9b94740 1 --2- 172.21.15.36:0/3328347411 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x2177db0 0x21791a0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafd9b94740 1 --2- 172.21.15.36:0/3328347411 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x21779d0 0x217ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafd9b94740 1 -- 172.21.15.36:0/3328347411 >> 172.21.15.36:0/3328347411 conn(0x2103540 msgr2=0x217b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:41.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafd9b94740 1 -- 172.21.15.36:0/3328347411 shutdown_connections 2022-06-17T12:52:41.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafd9b94740 1 -- 172.21.15.36:0/3328347411 wait complete. 2022-06-17T12:52:41.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafd9b94740 1 -- start start 2022-06-17T12:52:41.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.783+0000 7fafd9b94740 1 Processor -- start 2022-06-17T12:52:41.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafd9b94740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x21779d0 0x21abe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafd9b94740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2177db0 0x21b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafd9b94740 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x2179690 0x21ad610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafd9b94740 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x21ab4b0 con 0x2179690 2022-06-17T12:52:41.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafd9b94740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x21ab750 con 0x21779d0 2022-06-17T12:52:41.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafd9b94740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x21ab9f0 con 0x2177db0 2022-06-17T12:52:41.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafd9b94740 10 client.0 ll_register_callbacks cb 0x2132d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-06-17T12:52:41.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafcdedc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x2179690 0x21ad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafcdedc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x2179690 0x21ad610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35122/0 (socket says 172.21.15.36:35122) 2022-06-17T12:52:41.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafcceda700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2177db0 0x21b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafcdedc700 1 -- 172.21.15.36:0/3433490741 learned_addr learned my addr 172.21.15.36:0/3433490741 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:41.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafcd6db700 1 --2- 172.21.15.36:0/3433490741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x21779d0 0x21abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafcdedc700 1 -- 172.21.15.36:0/3433490741 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2177db0 msgr2=0x21b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:41.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafcdedc700 1 --2- 172.21.15.36:0/3433490741 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2177db0 0x21b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafcdedc700 1 -- 172.21.15.36:0/3433490741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x21779d0 msgr2=0x21abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:41.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafcdedc700 1 --2- 172.21.15.36:0/3433490741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x21779d0 0x21abe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafcdedc700 1 -- 172.21.15.36:0/3433490741 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc80b0fd0 con 0x2179690 2022-06-17T12:52:41.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafcd6db700 1 --2- 172.21.15.36:0/3433490741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x21779d0 0x21abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:52:41.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafcdedc700 1 --2- 172.21.15.36:0/3433490741 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x2179690 0x21ad610 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7fafc000d000 tx=0x7fafc0009040).ready entity=mon.0 client_cookie=448a82abb0e8d4fa server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:41.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafa3fff700 10 client.0 ms_handle_connect on v2:172.21.15.36:3300/0 2022-06-17T12:52:41.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafa3fff700 1 -- 172.21.15.36:0/3433490741 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafc0014640 con 0x2179690 2022-06-17T12:52:41.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafa3fff700 1 -- 172.21.15.36:0/3433490741 <== mon.0 v2:172.21.15.36:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fafc00133b0 con 0x2179690 2022-06-17T12:52:41.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafd9b94740 1 -- 172.21.15.36:0/3433490741 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x20e1d00 con 0x2179690 2022-06-17T12:52:41.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafa3fff700 1 -- 172.21.15.36:0/3433490741 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafc00147a0 con 0x2179690 2022-06-17T12:52:41.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafd9b94740 10 client.6261 fetch_fsmap learned FSMap version 30 2022-06-17T12:52:41.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafd9b94740 1 -- 172.21.15.36:0/3433490741 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x20db2f0 con 0x2179690 2022-06-17T12:52:41.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafa3fff700 1 -- 172.21.15.36:0/3433490741 <== mon.0 v2:172.21.15.36:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x20e1d00 con 0x2179690 2022-06-17T12:52:41.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafa3fff700 1 -- 172.21.15.36:0/3433490741 <== mon.0 v2:172.21.15.36:3300/0 5 ==== fsmap(e 30) v1 ==== 1247+0+0 (secure 0 0 0) 0x7fafc0013b00 con 0x2179690 2022-06-17T12:52:41.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafd9b94740 10 client.6261 fetch_fsmap finished waiting for FSMap version 30 2022-06-17T12:52:41.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafd9b94740 10 client.6261 resolve_mds: resolved ID 'a' to GID 6054 2022-06-17T12:52:41.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafd9b94740 20 client.6261 populate_metadata read hostname 'smithi036' 2022-06-17T12:52:41.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafd9b94740 1 --2- 172.21.15.36:0/3433490741 >> [v2:172.21.15.36:6834/4091208534,v1:172.21.15.36:6835/4091208534] conn(0x21c2f60 0x21c3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafd9b94740 4 client.6261 mds_command: new command op to 6054 tid=0[{"prefix": "get_command_descriptions"}] 2022-06-17T12:52:41.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafd9b94740 1 -- 172.21.15.36:0/3433490741 --> [v2:172.21.15.36:6834/4091208534,v1:172.21.15.36:6835/4091208534] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x20d9d80 con 0x21c2f60 2022-06-17T12:52:41.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafcd6db700 1 --2- 172.21.15.36:0/3433490741 >> [v2:172.21.15.36:6834/4091208534,v1:172.21.15.36:6835/4091208534] conn(0x21c2f60 0x21c3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafcd6db700 1 --2- 172.21.15.36:0/3433490741 >> [v2:172.21.15.36:6834/4091208534,v1:172.21.15.36:6835/4091208534] conn(0x21c2f60 0x21c3340 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=e58a199305055b14 server_cookie=70f03dad15031719 in_seq=0 out_seq=0 2022-06-17T12:52:41.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafa3fff700 10 client.6261 ms_handle_connect on v2:172.21.15.36:6834/4091208534 2022-06-17T12:52:41.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafa3fff700 1 -- 172.21.15.36:0/3433490741 <== mds.0 v2:172.21.15.36:6834/4091208534 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x20d9d80 con 0x21c2f60 2022-06-17T12:52:41.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafa3fff700 10 client.6261 handle_command_reply: tid=0 2022-06-17T12:52:41.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.787+0000 7fafd9b94740 1 client.6261 shutdown 2022-06-17T12:52:41.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.791+0000 7fafd9b94740 1 -- 172.21.15.36:0/3433490741 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x2179690 msgr2=0x21ad610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:41.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.791+0000 7fafd9b94740 1 --2- 172.21.15.36:0/3433490741 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x2179690 0x21ad610 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7fafc000d000 tx=0x7fafc0009040).stop 2022-06-17T12:52:41.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.791+0000 7fafd9b94740 1 -- 172.21.15.36:0/3433490741 shutdown_connections 2022-06-17T12:52:41.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.791+0000 7fafd9b94740 1 --2- 172.21.15.36:0/3433490741 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x2179690 0x21ad610 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.791+0000 7fafd9b94740 1 --2- 172.21.15.36:0/3433490741 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2177db0 0x21b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.791+0000 7fafd9b94740 1 --2- 172.21.15.36:0/3433490741 >> [v2:172.21.15.36:6834/4091208534,v1:172.21.15.36:6835/4091208534] conn(0x21c2f60 0x21c3340 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.791+0000 7fafd9b94740 1 --2- 172.21.15.36:0/3433490741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x21779d0 0x21abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.791+0000 7fafa3fff700 0 client.6261 ms_handle_reset on v2:172.21.15.36:6834/4091208534 2022-06-17T12:52:41.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.791+0000 7fafd9b94740 1 -- 172.21.15.36:0/3433490741 >> 172.21.15.36:0/3433490741 conn(0x2103540 msgr2=0x217b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:41.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.791+0000 7fafd9b94740 1 -- 172.21.15.36:0/3433490741 shutdown_connections 2022-06-17T12:52:41.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.791+0000 7fafd9b94740 1 -- 172.21.15.36:0/3433490741 wait complete. 2022-06-17T12:52:41.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.791+0000 7fafd9b94740 20 client.6261 trim_cache size 0 max 16384 2022-06-17T12:52:41.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.803+0000 7fafd9b94740 1 Processor -- start 2022-06-17T12:52:41.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.803+0000 7fafd9b94740 1 -- start start 2022-06-17T12:52:41.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafd9b94740 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x21779d0 0x21b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafd9b94740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x2177db0 0x21b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafd9b94740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2179690 0x21b5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafd9b94740 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x2118480 con 0x21779d0 2022-06-17T12:52:41.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafd9b94740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x20fdb60 con 0x2177db0 2022-06-17T12:52:41.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafd9b94740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x20d9d80 con 0x2179690 2022-06-17T12:52:41.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafcd6db700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x21779d0 0x21b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafcd6db700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x21779d0 0x21b14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35126/0 (socket says 172.21.15.36:35126) 2022-06-17T12:52:41.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafcd6db700 1 -- 172.21.15.36:0/586023699 learned_addr learned my addr 172.21.15.36:0/586023699 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:41.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafcdedc700 1 --2- 172.21.15.36:0/586023699 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2179690 0x21b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafcd6db700 1 -- 172.21.15.36:0/586023699 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2179690 msgr2=0x21b5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:41.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafcceda700 1 --2- 172.21.15.36:0/586023699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x2177db0 0x21b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafcd6db700 1 --2- 172.21.15.36:0/586023699 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2179690 0x21b5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafcd6db700 1 -- 172.21.15.36:0/586023699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x2177db0 msgr2=0x21b52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:41.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafcd6db700 1 --2- 172.21.15.36:0/586023699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x2177db0 0x21b52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafcd6db700 1 -- 172.21.15.36:0/586023699 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x20e3ee0 con 0x21779d0 2022-06-17T12:52:41.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafcceda700 1 --2- 172.21.15.36:0/586023699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x2177db0 0x21b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:52:41.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafcdedc700 1 --2- 172.21.15.36:0/586023699 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2179690 0x21b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:52:41.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafcd6db700 1 --2- 172.21.15.36:0/586023699 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x21779d0 0x21b14e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7fafc4056000 tx=0x7fafc4048040).ready entity=mon.0 client_cookie=e7855c85aae8ceb5 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:41.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafaa7fc700 1 -- 172.21.15.36:0/586023699 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafc4044850 con 0x21779d0 2022-06-17T12:52:41.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafaa7fc700 1 -- 172.21.15.36:0/586023699 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc404c040 con 0x21779d0 2022-06-17T12:52:41.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafaa7fc700 1 -- 172.21.15.36:0/586023699 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafc40449e0 con 0x21779d0 2022-06-17T12:52:41.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafd9b94740 1 -- 172.21.15.36:0/586023699 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x21779d0 msgr2=0x21b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:41.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafd9b94740 1 --2- 172.21.15.36:0/586023699 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x21779d0 0x21b14e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7fafc4056000 tx=0x7fafc4048040).stop 2022-06-17T12:52:41.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafd9b94740 1 -- 172.21.15.36:0/586023699 shutdown_connections 2022-06-17T12:52:41.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafd9b94740 1 --2- 172.21.15.36:0/586023699 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2179690 0x21b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafd9b94740 1 --2- 172.21.15.36:0/586023699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x2177db0 0x21b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafd9b94740 1 --2- 172.21.15.36:0/586023699 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x21779d0 0x21b14e0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafd9b94740 1 -- 172.21.15.36:0/586023699 >> 172.21.15.36:0/586023699 conn(0x2103540 msgr2=0x217ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:41.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafd9b94740 1 -- 172.21.15.36:0/586023699 shutdown_connections 2022-06-17T12:52:41.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafd9b94740 1 -- 172.21.15.36:0/586023699 wait complete. 2022-06-17T12:52:41.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafd9b94740 1 -- start start 2022-06-17T12:52:41.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafd9b94740 1 Processor -- start 2022-06-17T12:52:41.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafd9b94740 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x21779d0 0x21bff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafd9b94740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2177db0 0x2179a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafd9b94740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x2179690 0x2179f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafd9b94740 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x21a9250 con 0x21779d0 2022-06-17T12:52:41.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafd9b94740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x21d04a0 con 0x2179690 2022-06-17T12:52:41.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafd9b94740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x21bf890 con 0x2177db0 2022-06-17T12:52:41.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafd9b94740 10 client.0 ll_register_callbacks cb 0x21ad370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-06-17T12:52:41.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafcd6db700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x21779d0 0x21bff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafcd6db700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x21779d0 0x21bff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35132/0 (socket says 172.21.15.36:35132) 2022-06-17T12:52:41.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafcdedc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x2179690 0x2179f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.807+0000 7fafcd6db700 1 -- 172.21.15.36:0/1352923742 learned_addr learned my addr 172.21.15.36:0/1352923742 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:41.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafcceda700 1 --2- 172.21.15.36:0/1352923742 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2177db0 0x2179a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafcd6db700 1 -- 172.21.15.36:0/1352923742 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2177db0 msgr2=0x2179a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:41.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafcd6db700 1 --2- 172.21.15.36:0/1352923742 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2177db0 0x2179a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafcd6db700 1 -- 172.21.15.36:0/1352923742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x2179690 msgr2=0x2179f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:41.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafcd6db700 1 --2- 172.21.15.36:0/1352923742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x2179690 0x2179f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:41.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafcd6db700 1 -- 172.21.15.36:0/1352923742 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc8155bc0 con 0x21779d0 2022-06-17T12:52:41.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafcd6db700 1 --2- 172.21.15.36:0/1352923742 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x21779d0 0x21bff90 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x21d05e0 tx=0x7fafc4058000).ready entity=mon.0 client_cookie=2f23ccfcf240727 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:41.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafa97fa700 10 client.0 ms_handle_connect on v2:172.21.15.36:3300/0 2022-06-17T12:52:41.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafa97fa700 1 -- 172.21.15.36:0/1352923742 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafc4063070 con 0x21779d0 2022-06-17T12:52:41.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafa97fa700 1 -- 172.21.15.36:0/1352923742 <== mon.0 v2:172.21.15.36:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fafc4045b30 con 0x21779d0 2022-06-17T12:52:41.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafd9b94740 1 -- 172.21.15.36:0/1352923742 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fafb801e950 con 0x21779d0 2022-06-17T12:52:41.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafa97fa700 1 -- 172.21.15.36:0/1352923742 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafc4045cf0 con 0x21779d0 2022-06-17T12:52:41.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafd9b94740 10 client.6267 fetch_fsmap learned FSMap version 30 2022-06-17T12:52:41.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafd9b94740 1 -- 172.21.15.36:0/1352923742 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x20e08d0 con 0x21779d0 2022-06-17T12:52:41.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafa97fa700 1 -- 172.21.15.36:0/1352923742 <== mon.0 v2:172.21.15.36:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb801e950 con 0x21779d0 2022-06-17T12:52:41.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafa97fa700 1 -- 172.21.15.36:0/1352923742 <== mon.0 v2:172.21.15.36:3300/0 5 ==== fsmap(e 30) v1 ==== 1247+0+0 (secure 0 0 0) 0x7fafc4042430 con 0x21779d0 2022-06-17T12:52:41.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafd9b94740 10 client.6267 fetch_fsmap finished waiting for FSMap version 30 2022-06-17T12:52:41.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafd9b94740 10 client.6267 resolve_mds: resolved ID 'a' to GID 6054 2022-06-17T12:52:41.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafd9b94740 20 client.6267 populate_metadata read hostname 'smithi036' 2022-06-17T12:52:41.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafd9b94740 1 --2- 172.21.15.36:0/1352923742 >> [v2:172.21.15.36:6834/4091208534,v1:172.21.15.36:6835/4091208534] conn(0x217b5d0 0x2212640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:41.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafd9b94740 4 client.6267 mds_command: new command op to 6054 tid=0[{"prefix": "cache drop", "timeout": 10}] 2022-06-17T12:52:41.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafd9b94740 1 -- 172.21.15.36:0/1352923742 --> [v2:172.21.15.36:6834/4091208534,v1:172.21.15.36:6835/4091208534] -- command(tid 0: {"prefix": "cache drop", "timeout": 10}) v1 -- 0x1fcf5e0 con 0x217b5d0 2022-06-17T12:52:41.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafcceda700 1 --2- 172.21.15.36:0/1352923742 >> [v2:172.21.15.36:6834/4091208534,v1:172.21.15.36:6835/4091208534] conn(0x217b5d0 0x2212640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:41.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafcceda700 1 --2- 172.21.15.36:0/1352923742 >> [v2:172.21.15.36:6834/4091208534,v1:172.21.15.36:6835/4091208534] conn(0x217b5d0 0x2212640 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=5130d92f7743701c server_cookie=62e0f1ff30dd1e61 in_seq=0 out_seq=0 2022-06-17T12:52:41.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:41.811+0000 7fafa97fa700 10 client.6267 ms_handle_connect on v2:172.21.15.36:6834/4091208534 2022-06-17T12:52:45.732 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:52:45.735 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:52:45.738 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:52:45.741 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:52:45.744 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:52:52.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafa97fa700 1 -- 172.21.15.36:0/1352923742 <== mds.0 v2:172.21.15.36:6834/4091208534 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+400 (crc 0 0 0) 0x1fcf5e0 con 0x217b5d0 2022-06-17T12:52:52.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafa97fa700 10 client.6267 handle_command_reply: tid=0 2022-06-17T12:52:52.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafd9b94740 1 client.6267 shutdown 2022-06-17T12:52:52.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafd9b94740 1 -- 172.21.15.36:0/1352923742 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x21779d0 msgr2=0x21bff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:52.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafd9b94740 1 --2- 172.21.15.36:0/1352923742 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x21779d0 0x21bff90 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x21d05e0 tx=0x7fafc4058000).stop 2022-06-17T12:52:52.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafd9b94740 1 -- 172.21.15.36:0/1352923742 shutdown_connections 2022-06-17T12:52:52.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafd9b94740 1 --2- 172.21.15.36:0/1352923742 >> [v2:172.21.15.36:6834/4091208534,v1:172.21.15.36:6835/4091208534] conn(0x217b5d0 0x2212640 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:52.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafd9b94740 1 --2- 172.21.15.36:0/1352923742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x2179690 0x2179f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:52.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafa97fa700 0 client.6267 ms_handle_reset on v2:172.21.15.36:6834/4091208534 2022-06-17T12:52:52.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafd9b94740 1 --2- 172.21.15.36:0/1352923742 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x2177db0 0x2179a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:52.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafd9b94740 1 --2- 172.21.15.36:0/1352923742 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x21779d0 0x21bff90 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:52.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafd9b94740 1 -- 172.21.15.36:0/1352923742 >> 172.21.15.36:0/1352923742 conn(0x2103540 msgr2=0x21b5090 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:52.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafd9b94740 1 -- 172.21.15.36:0/1352923742 shutdown_connections 2022-06-17T12:52:52.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafd9b94740 1 -- 172.21.15.36:0/1352923742 wait complete. 2022-06-17T12:52:52.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafd9b94740 20 client.6267 trim_cache size 0 max 16384 2022-06-17T12:52:52.047 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:52:52.047 INFO:teuthology.orchestra.run.smithi036.stdout: "client_recall": { 2022-06-17T12:52:52.048 INFO:teuthology.orchestra.run.smithi036.stdout: "return_code": -110, 2022-06-17T12:52:52.048 INFO:teuthology.orchestra.run.smithi036.stdout: "message": "(110) Connection timed out", 2022-06-17T12:52:52.048 INFO:teuthology.orchestra.run.smithi036.stdout: "recalled": 240 2022-06-17T12:52:52.049 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.049 INFO:teuthology.orchestra.run.smithi036.stdout: "flush_journal": { 2022-06-17T12:52:52.049 INFO:teuthology.orchestra.run.smithi036.stdout: "return_code": 0, 2022-06-17T12:52:52.049 INFO:teuthology.orchestra.run.smithi036.stdout: "message": "" 2022-06-17T12:52:52.050 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.050 INFO:teuthology.orchestra.run.smithi036.stdout: "trim_cache": { 2022-06-17T12:52:52.050 INFO:teuthology.orchestra.run.smithi036.stdout: "trimmed": 220 2022-06-17T12:52:52.050 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.051 INFO:teuthology.orchestra.run.smithi036.stdout: "cache": { 2022-06-17T12:52:52.051 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": { 2022-06-17T12:52:52.051 INFO:teuthology.orchestra.run.smithi036.stdout: "items": 15410, 2022-06-17T12:52:52.051 INFO:teuthology.orchestra.run.smithi036.stdout: "bytes": 2428160 2022-06-17T12:52:52.051 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.052 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.052 INFO:teuthology.orchestra.run.smithi036.stdout: "duration": 10.235027701 2022-06-17T12:52:52.053 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:52:52.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafce6dd700 1 -- 172.21.15.36:0/1789104552 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fafb0053a60 msgr2=0x7fafb0055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:52.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafce6dd700 1 --2- 172.21.15.36:0/1789104552 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fafb0053a60 0x7fafb0055f20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fafb8018000 tx=0x7fafb8014000).stop 2022-06-17T12:52:52.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafce6dd700 1 -- 172.21.15.36:0/1789104552 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fafc80b18e0 msgr2=0x7fafc813f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:52.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafce6dd700 1 --2- 172.21.15.36:0/1789104552 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fafc80b18e0 0x7fafc813f600 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7fafc4000bc0 tx=0x7fafc4007a70).stop 2022-06-17T12:52:52.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafce6dd700 1 -- 172.21.15.36:0/1789104552 shutdown_connections 2022-06-17T12:52:52.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafce6dd700 1 --2- 172.21.15.36:0/1789104552 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fafb0053a60 0x7fafb0055f20 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:52.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafce6dd700 1 --2- 172.21.15.36:0/1789104552 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafc80b6860 0x7fafc8152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:52.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafce6dd700 1 --2- 172.21.15.36:0/1789104552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafc80b39a0 0x7fafc80650d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:52.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafce6dd700 1 --2- 172.21.15.36:0/1789104552 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fafc80b18e0 0x7fafc813f600 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:52.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafce6dd700 1 -- 172.21.15.36:0/1789104552 >> 172.21.15.36:0/1789104552 conn(0x7fafc80614b0 msgr2=0x7fafc805cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:52.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafce6dd700 1 -- 172.21.15.36:0/1789104552 shutdown_connections 2022-06-17T12:52:52.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.050+0000 7fafce6dd700 1 -- 172.21.15.36:0/1789104552 wait complete. 2022-06-17T12:52:52.058 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-17T12:52:52.059 DEBUG:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:52:52.080 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:52:52.102 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:52:52.125 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:52:52.147 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:52:52.171 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-17T12:52:52.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:52:52.172 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:52:52.173 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:52:52.184 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:52:52.184 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:52:52.185 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2022-06-17T12:52:52.185 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:52:52.186 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:52:52.202 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22254]: fuse finished with error 0 and tester_r 0 2022-06-17T12:52:52.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:52:52.204 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:52:52.205 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:52:52.211 INFO:teuthology.orchestra.run.smithi107.stdout:ext2/ext3 2022-06-17T12:52:52.212 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:52:52.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:52:52.312 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:52:52.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:52:52.320 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:52:52.321 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:52:52.329 INFO:teuthology.orchestra.run.smithi136.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-17T12:52:52.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:52:52.330 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-17T12:52:52.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:52:52.332 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:52:52.339 INFO:teuthology.orchestra.run.smithi136.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-17T12:52:52.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:52:52.341 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:52:52.342 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:52:52.342 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:52:52.349 INFO:teuthology.orchestra.run.smithi143.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:52:52.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:52:52.351 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-06-17T12:52:52.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:52:52.352 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:52:52.359 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:52:52.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:52:52.361 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-17T12:52:52.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a4cd91700 1 Processor -- start 2022-06-17T12:52:52.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a4cd91700 1 -- start start 2022-06-17T12:52:52.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a4cd91700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a480b6800 0x7f8a48147ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:52.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a4cd91700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8a480aeea0 0x7f8a4814a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:52.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a4cd91700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8a4814acc0 0x7f8a481463a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:52.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a4cd91700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a480af360 con 0x7f8a4814acc0 2022-06-17T12:52:52.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a4cd91700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a480b6be0 con 0x7f8a480aeea0 2022-06-17T12:52:52.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a4cd91700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a481473a0 con 0x7f8a480b6800 2022-06-17T12:52:52.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a47fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8a4814acc0 0x7f8a481463a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:52.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a47fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8a4814acc0 0x7f8a481463a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35146/0 (socket says 172.21.15.36:35146) 2022-06-17T12:52:52.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a47fff700 1 -- 172.21.15.36:0/3786998566 learned_addr learned my addr 172.21.15.36:0/3786998566 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:52.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a46ffd700 1 --2- 172.21.15.36:0/3786998566 >> v2:172.21.15.70:3300/0 conn(0x7f8a480aeea0 0x7f8a4814a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:52.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a477fe700 1 --2- 172.21.15.36:0/3786998566 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a480b6800 0x7f8a48147ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:52.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a47fff700 1 -- 172.21.15.36:0/3786998566 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a480b6800 msgr2=0x7f8a48147ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:52.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a47fff700 1 --2- 172.21.15.36:0/3786998566 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a480b6800 0x7f8a48147ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:52.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a47fff700 1 -- 172.21.15.36:0/3786998566 >> v2:172.21.15.70:3300/0 conn(0x7f8a480aeea0 msgr2=0x7f8a4814a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:52.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a47fff700 1 --2- 172.21.15.36:0/3786998566 >> v2:172.21.15.70:3300/0 conn(0x7f8a480aeea0 0x7f8a4814a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:52.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a47fff700 1 -- 172.21.15.36:0/3786998566 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a48147640 con 0x7f8a4814acc0 2022-06-17T12:52:52.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a46ffd700 1 --2- 172.21.15.36:0/3786998566 >> v2:172.21.15.70:3300/0 conn(0x7f8a480aeea0 0x7f8a4814a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:52:52.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a47fff700 1 --2- 172.21.15.36:0/3786998566 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8a4814acc0 0x7f8a481463a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f8a400032f0 tx=0x7f8a40009290).ready entity=mon.0 client_cookie=b87caca6a06627b4 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:52.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a467fc700 1 -- 172.21.15.36:0/3786998566 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a4000b870 con 0x7f8a4814acc0 2022-06-17T12:52:52.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a467fc700 1 -- 172.21.15.36:0/3786998566 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4000b9d0 con 0x7f8a4814acc0 2022-06-17T12:52:52.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a467fc700 1 -- 172.21.15.36:0/3786998566 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a4000bb40 con 0x7f8a4814acc0 2022-06-17T12:52:52.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a4cd91700 1 -- 172.21.15.36:0/3786998566 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8a4814acc0 msgr2=0x7f8a481463a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:52.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a4cd91700 1 --2- 172.21.15.36:0/3786998566 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8a4814acc0 0x7f8a481463a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f8a400032f0 tx=0x7f8a40009290).stop 2022-06-17T12:52:52.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a4cd91700 1 -- 172.21.15.36:0/3786998566 shutdown_connections 2022-06-17T12:52:52.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a4cd91700 1 --2- 172.21.15.36:0/3786998566 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8a4814acc0 0x7f8a481463a0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:52.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a4cd91700 1 --2- 172.21.15.36:0/3786998566 >> v2:172.21.15.70:3300/0 conn(0x7f8a480aeea0 0x7f8a4814a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:52.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a4cd91700 1 --2- 172.21.15.36:0/3786998566 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a480b6800 0x7f8a48147ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:52.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a4cd91700 1 -- 172.21.15.36:0/3786998566 >> 172.21.15.36:0/3786998566 conn(0x7f8a480614b0 msgr2=0x7f8a48061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:52.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a4cd91700 1 -- 172.21.15.36:0/3786998566 shutdown_connections 2022-06-17T12:52:52.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.490+0000 7f8a4cd91700 1 -- 172.21.15.36:0/3786998566 wait complete. 2022-06-17T12:52:52.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a4cd91700 1 Processor -- start 2022-06-17T12:52:52.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a4cd91700 1 -- start start 2022-06-17T12:52:52.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a4cd91700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a480aeea0 0x7f8a4810f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:52.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a4cd91700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8a480b6800 0x7f8a4810fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:52.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a4cd91700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a4814acc0 0x7f8a481451e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:52.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a4cd91700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4815a700 con 0x7f8a480b6800 2022-06-17T12:52:52.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a4cd91700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4815a9a0 con 0x7f8a480aeea0 2022-06-17T12:52:52.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a4cd91700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4811c1a0 con 0x7f8a4814acc0 2022-06-17T12:52:52.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a46ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8a480b6800 0x7f8a4810fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:52.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a46ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8a480b6800 0x7f8a4810fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35150/0 (socket says 172.21.15.36:35150) 2022-06-17T12:52:52.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a46ffd700 1 -- 172.21.15.36:0/1271605635 learned_addr learned my addr 172.21.15.36:0/1271605635 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:52.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a47fff700 1 --2- 172.21.15.36:0/1271605635 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a4814acc0 0x7f8a481451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:52.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a477fe700 1 --2- 172.21.15.36:0/1271605635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a480aeea0 0x7f8a4810f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:52.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a46ffd700 1 -- 172.21.15.36:0/1271605635 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a4814acc0 msgr2=0x7f8a481451e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:52.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a46ffd700 1 --2- 172.21.15.36:0/1271605635 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a4814acc0 0x7f8a481451e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:52.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a46ffd700 1 -- 172.21.15.36:0/1271605635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a480aeea0 msgr2=0x7f8a4810f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:52.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a46ffd700 1 --2- 172.21.15.36:0/1271605635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a480aeea0 0x7f8a4810f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:52.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a46ffd700 1 -- 172.21.15.36:0/1271605635 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a40009530 con 0x7f8a480b6800 2022-06-17T12:52:52.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a47fff700 1 --2- 172.21.15.36:0/1271605635 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a4814acc0 0x7f8a481451e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:52:52.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a46ffd700 1 --2- 172.21.15.36:0/1271605635 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8a480b6800 0x7f8a4810fb70 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f8a380068e0 tx=0x7f8a38008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:52.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a44ff9700 1 -- 172.21.15.36:0/1271605635 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a380127e0 con 0x7f8a480b6800 2022-06-17T12:52:52.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a44ff9700 1 -- 172.21.15.36:0/1271605635 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a38012940 con 0x7f8a480b6800 2022-06-17T12:52:52.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a4cd91700 1 -- 172.21.15.36:0/1271605635 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a48066830 con 0x7f8a480b6800 2022-06-17T12:52:52.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a44ff9700 1 -- 172.21.15.36:0/1271605635 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a38012ca0 con 0x7f8a480b6800 2022-06-17T12:52:52.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.494+0000 7f8a4cd91700 1 -- 172.21.15.36:0/1271605635 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a48112da0 con 0x7f8a480b6800 2022-06-17T12:52:52.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.498+0000 7f8a4cd91700 1 -- 172.21.15.36:0/1271605635 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a4815a540 con 0x7f8a480b6800 2022-06-17T12:52:52.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.498+0000 7f8a44ff9700 1 -- 172.21.15.36:0/1271605635 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8a38013480 con 0x7f8a480b6800 2022-06-17T12:52:52.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.498+0000 7f8a44ff9700 1 --2- 172.21.15.36:0/1271605635 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8a2c053940 0x7f8a2c055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:52.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.498+0000 7f8a44ff9700 1 -- 172.21.15.36:0/1271605635 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8a3806c2e0 con 0x7f8a480b6800 2022-06-17T12:52:52.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.498+0000 7f8a477fe700 1 --2- 172.21.15.36:0/1271605635 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8a2c053940 0x7f8a2c055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:52.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.498+0000 7f8a477fe700 1 --2- 172.21.15.36:0/1271605635 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8a2c053940 0x7f8a2c055e00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f8a3000c000 tx=0x7f8a30008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:52.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.502+0000 7f8a44ff9700 1 -- 172.21.15.36:0/1271605635 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8a38043080 con 0x7f8a480b6800 2022-06-17T12:52:52.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.630+0000 7f8a4cd91700 1 -- 172.21.15.36:0/1271605635 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f8a4811c360 con 0x7f8a480b6800 2022-06-17T12:52:52.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.634+0000 7f8a44ff9700 1 -- 172.21.15.36:0/1271605635 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v65) v1 ==== 81+0+34040 (secure 0 0 0) 0x7f8a38012e00 con 0x7f8a480b6800 2022-06-17T12:52:52.619 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:52:52.619 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:52:52.620 INFO:teuthology.orchestra.run.smithi036.stdout: "epoch": 65, 2022-06-17T12:52:52.620 INFO:teuthology.orchestra.run.smithi036.stdout: "fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", 2022-06-17T12:52:52.620 INFO:teuthology.orchestra.run.smithi036.stdout: "created": "2022-06-17T12:46:30.671408+0000", 2022-06-17T12:52:52.620 INFO:teuthology.orchestra.run.smithi036.stdout: "modified": "2022-06-17T12:52:23.256104+0000", 2022-06-17T12:52:52.621 INFO:teuthology.orchestra.run.smithi036.stdout: "last_up_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:52:52.621 INFO:teuthology.orchestra.run.smithi036.stdout: "last_in_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:52:52.621 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-17T12:52:52.621 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_num": 5799936, 2022-06-17T12:52:52.622 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_set": [ 2022-06-17T12:52:52.622 INFO:teuthology.orchestra.run.smithi036.stdout: "pglog_hardlimit", 2022-06-17T12:52:52.622 INFO:teuthology.orchestra.run.smithi036.stdout: "purged_snapdirs", 2022-06-17T12:52:52.622 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_deletes", 2022-06-17T12:52:52.623 INFO:teuthology.orchestra.run.smithi036.stdout: "sortbitwise" 2022-06-17T12:52:52.623 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:52:52.623 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_version": 6, 2022-06-17T12:52:52.623 INFO:teuthology.orchestra.run.smithi036.stdout: "full_ratio": 0.94999998807907104, 2022-06-17T12:52:52.624 INFO:teuthology.orchestra.run.smithi036.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-17T12:52:52.624 INFO:teuthology.orchestra.run.smithi036.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-17T12:52:52.624 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_snapshot": "", 2022-06-17T12:52:52.624 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_max": 12, 2022-06-17T12:52:52.625 INFO:teuthology.orchestra.run.smithi036.stdout: "max_osd": 8, 2022-06-17T12:52:52.625 INFO:teuthology.orchestra.run.smithi036.stdout: "require_min_compat_client": "jewel", 2022-06-17T12:52:52.625 INFO:teuthology.orchestra.run.smithi036.stdout: "min_compat_client": "jewel", 2022-06-17T12:52:52.625 INFO:teuthology.orchestra.run.smithi036.stdout: "require_osd_release": "octopus", 2022-06-17T12:52:52.626 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:52:52.626 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.626 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 1, 2022-06-17T12:52:52.626 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "device_health_metrics", 2022-06-17T12:52:52.627 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:33.881212+0000", 2022-06-17T12:52:52.627 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:52:52.627 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:52:52.627 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:52:52.628 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:52:52.628 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:52:52.628 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:52:52.628 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:52:52.628 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:52:52.629 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 1, 2022-06-17T12:52:52.629 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 1, 2022-06-17T12:52:52.629 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 1, 2022-06-17T12:52:52.629 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 1, 2022-06-17T12:52:52.630 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 1, 2022-06-17T12:52:52.630 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:52:52.630 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:52:52.630 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:52:52.631 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:52:52.631 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:52:52.631 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:52:52.632 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:52:52.632 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.632 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "16", 2022-06-17T12:52:52.632 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:52:52.633 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:52:52.633 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:52:52.633 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:52:52.633 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:52:52.633 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:52:52.634 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:52:52.634 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:52:52.634 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:52:52.635 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:52:52.636 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:52:52.637 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:52:52.638 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:52:52.638 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:52:52.638 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:52:52.639 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:52:52.639 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:52:52.639 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:52:52.640 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:52:52.640 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:52:52.640 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:52:52.640 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:52:52.641 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:52:52.641 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:52:52.641 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:52:52.641 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:52:52.642 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.642 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:52:52.642 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:52:52.643 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:52:52.643 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:52:52.643 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:52:52.646 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:52:52.646 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:52:52.646 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:52:52.647 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:52:52.647 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:52:52.647 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:52:52.647 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:52:52.648 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 1 2022-06-17T12:52:52.648 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.648 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:52:52.648 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr_devicehealth": {} 2022-06-17T12:52:52.649 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.649 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.649 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.649 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 2, 2022-06-17T12:52:52.650 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "rbd", 2022-06-17T12:52:52.650 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:44.485460+0000", 2022-06-17T12:52:52.650 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:52:52.650 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:52:52.651 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:52:52.651 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:52:52.652 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:52:52.652 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:52:52.653 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:52:52.653 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:52:52.653 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:52:52.654 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:52:52.654 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:52:52.655 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:52:52.655 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:52:52.656 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:52:52.656 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:52:52.657 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:52:52.657 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:52:52.657 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:52:52.658 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:52:52.658 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:52:52.658 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.658 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "19", 2022-06-17T12:52:52.659 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:52:52.659 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:52:52.659 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:52:52.659 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:52:52.660 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:52:52.660 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:52:52.660 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:52:52.660 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:52:52.660 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:52:52.661 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:52:52.661 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:52:52.661 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:52:52.661 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:52:52.662 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:52:52.662 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:52:52.662 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:52:52.663 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:52:52.663 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:52:52.664 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:52:52.664 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:52:52.664 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:52:52.665 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:52:52.665 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:52:52.666 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:52:52.666 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:52:52.666 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:52:52.667 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.667 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:52:52.668 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:52:52.668 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:52:52.668 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:52:52.669 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:52:52.669 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:52:52.670 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:52:52.670 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:52:52.671 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:52:52.671 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:52:52.672 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:52:52.672 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:52:52.672 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:52:52.673 INFO:teuthology.orchestra.run.smithi036.stdout: "rbd": {} 2022-06-17T12:52:52.673 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.673 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.673 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.674 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 11, 2022-06-17T12:52:52.674 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_metadata", 2022-06-17T12:52:52.674 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:52:20.693971+0000", 2022-06-17T12:52:52.674 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:52:52.675 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:52:52.675 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:52:52.675 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:52:52.675 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:52:52.675 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:52:52.676 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:52:52.676 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:52:52.676 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:52:52.676 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:52:52.676 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:52:52.677 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:52:52.677 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:52:52.677 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:52:52.677 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:52:52.678 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:52:52.678 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:52:52.678 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:52:52.678 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:52:52.679 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:52:52.679 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.679 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "65", 2022-06-17T12:52:52.679 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:52:52.679 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:52:52.680 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:52:52.680 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:52:52.680 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:52:52.680 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:52:52.680 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:52:52.681 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:52:52.681 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:52:52.681 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:52:52.681 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:52:52.682 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:52:52.682 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:52:52.682 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:52:52.682 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:52:52.682 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:52:52.683 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:52:52.683 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:52:52.683 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:52:52.683 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:52:52.684 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:52:52.684 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:52:52.684 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:52:52.684 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:52:52.684 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:52:52.685 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:52:52.685 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.685 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:52:52.685 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:52:52.686 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:52:52.686 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:52:52.686 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:52:52.686 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:52:52.686 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:52:52.687 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:52:52.687 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:52:52.687 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:52:52.687 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:52:52.688 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:52:52.688 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_bias": 4, 2022-06-17T12:52:52.688 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 16, 2022-06-17T12:52:52.688 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_priority": 5 2022-06-17T12:52:52.689 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.689 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:52:52.689 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:52:52.689 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": "cephfs" 2022-06-17T12:52:52.689 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.690 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.690 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.690 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.690 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 12, 2022-06-17T12:52:52.691 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_data", 2022-06-17T12:52:52.691 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:52:21.830992+0000", 2022-06-17T12:52:52.691 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:52:52.691 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:52:52.692 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:52:52.692 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:52:52.692 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:52:52.692 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:52:52.692 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:52:52.693 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:52:52.693 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:52:52.693 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:52:52.693 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:52:52.693 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:52:52.694 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:52:52.694 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:52:52.694 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:52:52.694 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:52:52.695 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:52:52.695 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:52:52.695 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:52:52.695 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:52:52.695 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.696 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "65", 2022-06-17T12:52:52.696 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:52:52.696 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:52:52.696 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:52:52.697 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:52:52.697 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:52:52.697 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:52:52.698 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:52:52.698 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:52:52.698 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:52:52.698 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:52:52.698 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:52:52.699 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:52:52.699 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:52:52.699 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:52:52.699 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:52:52.700 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:52:52.700 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:52:52.700 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:52:52.700 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:52:52.700 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:52:52.701 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:52:52.701 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:52:52.701 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:52:52.701 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:52:52.701 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:52:52.702 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:52:52.702 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.702 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:52:52.702 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:52:52.703 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:52:52.703 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:52:52.703 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:52:52.703 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:52:52.703 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:52:52.704 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:52:52.704 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:52:52.704 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:52:52.704 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:52:52.705 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:52:52.705 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:52:52.705 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:52:52.705 INFO:teuthology.orchestra.run.smithi036.stdout: "data": "cephfs" 2022-06-17T12:52:52.705 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.706 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.706 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.706 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:52:52.706 INFO:teuthology.orchestra.run.smithi036.stdout: "osds": [ 2022-06-17T12:52:52.707 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.707 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:52:52.707 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", 2022-06-17T12:52:52.707 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:52.707 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:52.708 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:52.708 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:52.708 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:52.708 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:52.709 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:52:52.709 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 63, 2022-06-17T12:52:52.709 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:52.709 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:52.710 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:52.710 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.710 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.710 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.711 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6802", 2022-06-17T12:52:52.711 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:52.711 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.711 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.711 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.712 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6803", 2022-06-17T12:52:52.712 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:52.712 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.712 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.713 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.713 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:52.713 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.713 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.713 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.714 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6804", 2022-06-17T12:52:52.714 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:52.714 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.714 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.715 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.715 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6805", 2022-06-17T12:52:52.715 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:52.715 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.715 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.716 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.716 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:52.716 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.716 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.717 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.717 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6808", 2022-06-17T12:52:52.717 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:52.717 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.717 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.718 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.718 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6809", 2022-06-17T12:52:52.718 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:52.718 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.719 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.719 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.719 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:52.719 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.719 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.720 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.720 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6806", 2022-06-17T12:52:52.720 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:52.720 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.720 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.721 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.721 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6807", 2022-06-17T12:52:52.721 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:52.721 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.722 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.722 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.722 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6803/22619", 2022-06-17T12:52:52.722 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6805/22619", 2022-06-17T12:52:52.722 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6809/22619", 2022-06-17T12:52:52.723 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6807/22619", 2022-06-17T12:52:52.723 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:52.723 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:52.723 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:52.724 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.724 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.724 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.724 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:52:52.724 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", 2022-06-17T12:52:52.725 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:52.725 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:52.725 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:52.725 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:52.726 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:52.726 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:52.726 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:52:52.726 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 63, 2022-06-17T12:52:52.726 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:52.727 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:52.727 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:52.727 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.727 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.728 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.728 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6810", 2022-06-17T12:52:52.728 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:52.728 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.728 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.729 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.729 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6811", 2022-06-17T12:52:52.729 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:52.729 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.729 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.730 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.730 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:52.730 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.730 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.731 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.731 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6812", 2022-06-17T12:52:52.731 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:52.731 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.731 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.732 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.732 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6813", 2022-06-17T12:52:52.732 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:52.732 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.733 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.733 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.733 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:52.734 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.734 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.734 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.734 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6816", 2022-06-17T12:52:52.734 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:52.735 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.735 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.735 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.736 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6817", 2022-06-17T12:52:52.736 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:52.736 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.736 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.736 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.737 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:52.737 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.737 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.737 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.738 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6814", 2022-06-17T12:52:52.738 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:52.738 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.738 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.738 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.739 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6815", 2022-06-17T12:52:52.739 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:52.739 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.739 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.740 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.740 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6811/22620", 2022-06-17T12:52:52.740 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6813/22620", 2022-06-17T12:52:52.740 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6817/22620", 2022-06-17T12:52:52.741 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6815/22620", 2022-06-17T12:52:52.741 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:52.741 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:52.741 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:52.741 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.742 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.742 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.742 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:52:52.742 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", 2022-06-17T12:52:52.743 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:52.743 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:52.743 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:52.743 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:52.743 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:52.744 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:52.744 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:52:52.744 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 63, 2022-06-17T12:52:52.744 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:52.744 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:52.745 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:52.745 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.745 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.745 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.746 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6826", 2022-06-17T12:52:52.746 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:52.746 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.746 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.747 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.747 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6827", 2022-06-17T12:52:52.747 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:52.747 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.747 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.748 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.748 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:52.748 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.748 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.749 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.749 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6828", 2022-06-17T12:52:52.749 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:52.749 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.749 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.750 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.750 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6829", 2022-06-17T12:52:52.750 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:52.750 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.751 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.751 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.751 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:52.751 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.751 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.752 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.752 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6832", 2022-06-17T12:52:52.752 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:52.752 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.752 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.753 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.753 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6833", 2022-06-17T12:52:52.753 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:52.753 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.754 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.754 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.754 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:52.754 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.754 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.755 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.755 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6830", 2022-06-17T12:52:52.755 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:52.755 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.756 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.756 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.756 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6831", 2022-06-17T12:52:52.756 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:52.756 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.757 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.757 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.757 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6827/22643", 2022-06-17T12:52:52.757 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6829/22643", 2022-06-17T12:52:52.758 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6833/22643", 2022-06-17T12:52:52.758 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6831/22643", 2022-06-17T12:52:52.758 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:52.758 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:52.758 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:52.759 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.759 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.759 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.759 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:52:52.760 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", 2022-06-17T12:52:52.760 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:52.760 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:52.760 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:52.760 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:52.761 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:52.761 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:52.761 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:52:52.761 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 51, 2022-06-17T12:52:52.761 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:52.762 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:52.762 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:52.762 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.762 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.763 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.763 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6818", 2022-06-17T12:52:52.763 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:52.763 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.763 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.764 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.764 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6819", 2022-06-17T12:52:52.764 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:52.764 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.765 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.765 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.765 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:52.765 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.765 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.766 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.766 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6820", 2022-06-17T12:52:52.766 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:52.766 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.767 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.767 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.767 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6821", 2022-06-17T12:52:52.767 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:52.767 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.768 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.768 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.768 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:52.768 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.769 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.769 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.769 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6824", 2022-06-17T12:52:52.769 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:52.769 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.770 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.770 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.770 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6825", 2022-06-17T12:52:52.770 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:52.771 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.771 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.771 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.771 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:52.771 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.772 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.772 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.772 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6822", 2022-06-17T12:52:52.772 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:52.773 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.773 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.773 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.773 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6823", 2022-06-17T12:52:52.773 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:52.774 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.774 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.774 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.774 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6819/22646", 2022-06-17T12:52:52.775 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6821/22646", 2022-06-17T12:52:52.775 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6825/22646", 2022-06-17T12:52:52.775 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6823/22646", 2022-06-17T12:52:52.775 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:52.775 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:52.776 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:52.776 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.776 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.776 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.777 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:52:52.777 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", 2022-06-17T12:52:52.777 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:52.777 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:52.777 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:52.778 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:52.778 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:52.778 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:52.778 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:52:52.778 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 63, 2022-06-17T12:52:52.779 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:52.779 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:52.779 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:52.779 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.780 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.780 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.780 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6808", 2022-06-17T12:52:52.780 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:52.780 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.781 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.781 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.781 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6809", 2022-06-17T12:52:52.781 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:52.782 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.782 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.782 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.782 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:52.782 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.783 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.783 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.783 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6810", 2022-06-17T12:52:52.783 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:52.784 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.784 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.784 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.784 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6811", 2022-06-17T12:52:52.784 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:52.785 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.785 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.785 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.785 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:52.786 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.786 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.786 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.786 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6814", 2022-06-17T12:52:52.786 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:52.787 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.787 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.787 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.787 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6815", 2022-06-17T12:52:52.788 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:52.788 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.788 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.788 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.788 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:52.789 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.789 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.789 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.789 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6812", 2022-06-17T12:52:52.789 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:52.790 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.790 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.790 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.790 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6813", 2022-06-17T12:52:52.791 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:52.791 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.791 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.791 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.791 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6809/22812", 2022-06-17T12:52:52.792 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6811/22812", 2022-06-17T12:52:52.792 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6815/22812", 2022-06-17T12:52:52.792 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6813/22812", 2022-06-17T12:52:52.792 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:52.793 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:52.793 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:52.793 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.793 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.793 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.794 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:52:52.794 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", 2022-06-17T12:52:52.794 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:52.794 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:52.795 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:52.795 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:52.795 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:52.795 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:52.795 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:52:52.796 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 62, 2022-06-17T12:52:52.796 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:52.796 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:52.796 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:52.796 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.797 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.797 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.797 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6800", 2022-06-17T12:52:52.797 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:52.798 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.798 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.798 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.798 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6801", 2022-06-17T12:52:52.798 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:52.799 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.799 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.799 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.799 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:52.800 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.800 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.800 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.800 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6802", 2022-06-17T12:52:52.800 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:52.801 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.801 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.801 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.801 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6803", 2022-06-17T12:52:52.802 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:52.802 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.802 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.802 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.802 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:52.803 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.803 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.803 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.803 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6806", 2022-06-17T12:52:52.804 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:52.804 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.804 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.804 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.804 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6807", 2022-06-17T12:52:52.805 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:52.805 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.805 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.805 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.806 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:52.806 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.806 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.806 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.806 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6804", 2022-06-17T12:52:52.807 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:52.807 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.807 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.807 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.808 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6805", 2022-06-17T12:52:52.808 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:52.808 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.808 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.808 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.809 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6801/22813", 2022-06-17T12:52:52.809 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6803/22813", 2022-06-17T12:52:52.809 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6807/22813", 2022-06-17T12:52:52.809 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6805/22813", 2022-06-17T12:52:52.810 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:52.810 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:52.810 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:52.810 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.810 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.811 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.811 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:52:52.811 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", 2022-06-17T12:52:52.811 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:52.812 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:52.812 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:52.812 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:52.812 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:52.812 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:52.813 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:52:52.813 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 63, 2022-06-17T12:52:52.813 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:52.813 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:52.814 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:52.814 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.814 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.814 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.814 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6816", 2022-06-17T12:52:52.815 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:52.815 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.815 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.815 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.816 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6817", 2022-06-17T12:52:52.816 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:52.816 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.816 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.816 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.817 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:52.817 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.817 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.817 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.818 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6818", 2022-06-17T12:52:52.818 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:52.818 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.818 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.818 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.819 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6819", 2022-06-17T12:52:52.819 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:52.819 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.819 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.820 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.820 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:52.820 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.820 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.820 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.821 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6822", 2022-06-17T12:52:52.821 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:52.821 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.821 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.822 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.822 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6823", 2022-06-17T12:52:52.822 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:52.822 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.823 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.823 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.823 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:52.823 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.824 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.824 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.824 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6820", 2022-06-17T12:52:52.824 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:52.824 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.825 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.825 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.825 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6821", 2022-06-17T12:52:52.825 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:52.826 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.826 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.826 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.826 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6817/22822", 2022-06-17T12:52:52.826 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6819/22822", 2022-06-17T12:52:52.837 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6823/22822", 2022-06-17T12:52:52.837 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6821/22822", 2022-06-17T12:52:52.838 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:52.838 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:52.838 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:52.838 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.839 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.839 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.839 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:52:52.839 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", 2022-06-17T12:52:52.840 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:52.840 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:52.840 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:52.840 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:52.840 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:52.841 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:52.841 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:52:52.841 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 62, 2022-06-17T12:52:52.841 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:52.841 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:52.842 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:52.842 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.842 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.842 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.843 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6824", 2022-06-17T12:52:52.843 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:52.843 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.843 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.844 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.844 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6825", 2022-06-17T12:52:52.844 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:52.844 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.844 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.845 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.845 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:52.845 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.845 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.846 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.846 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6826", 2022-06-17T12:52:52.846 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:52.846 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.846 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.847 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.847 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6827", 2022-06-17T12:52:52.847 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:52.847 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.848 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.848 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.848 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:52.848 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.848 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.849 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.849 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6830", 2022-06-17T12:52:52.849 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:52.849 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.849 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.850 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.850 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6831", 2022-06-17T12:52:52.850 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:52.850 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.851 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.851 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.851 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:52.851 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:52.851 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.852 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:52.852 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6828", 2022-06-17T12:52:52.852 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:52.852 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.853 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.853 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:52.853 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6829", 2022-06-17T12:52:52.853 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:52.853 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.854 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.854 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.854 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6825/22837", 2022-06-17T12:52:52.854 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6827/22837", 2022-06-17T12:52:52.855 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6831/22837", 2022-06-17T12:52:52.855 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6829/22837", 2022-06-17T12:52:52.855 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:52.855 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:52.855 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:52.856 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:52.856 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.856 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:52:52.856 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_xinfo": [ 2022-06-17T12:52:52.856 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.857 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:52:52.857 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:52.857 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:52.857 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:52.858 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:52.858 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:52.858 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.291461+0000", 2022-06-17T12:52:52.858 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:52.859 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.859 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.859 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:52:52.859 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:52.860 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:52.860 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:52.860 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:52.860 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:52.861 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.537912+0000", 2022-06-17T12:52:52.861 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:52.861 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.861 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.862 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:52:52.862 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:52.862 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:52.862 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:52.862 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:52.863 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:52.863 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.606947+0000", 2022-06-17T12:52:52.863 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:52.863 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.864 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.864 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:52:52.864 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:52.864 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:52.864 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:52.865 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:52.865 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:52.865 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.514412+0000", 2022-06-17T12:52:52.865 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:52.866 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.866 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.866 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:52:52.866 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:52.866 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:52.867 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:52.867 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:52.867 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:52.867 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.394645+0000", 2022-06-17T12:52:52.868 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:52.868 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.868 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.868 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:52:52.868 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:52.869 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:52.869 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:52.869 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:52.869 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:52.869 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.343806+0000", 2022-06-17T12:52:52.870 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:52.870 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.870 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.870 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:52:52.871 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:52.871 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:52.871 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:52.871 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:52.871 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:52.872 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.591864+0000", 2022-06-17T12:52:52.872 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:52.872 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.872 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:52.873 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:52:52.873 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:52.873 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:52.873 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:52.873 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:52.874 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:52.874 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.607045+0000", 2022-06-17T12:52:52.874 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:52.874 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.875 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:52:52.875 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap": [], 2022-06-17T12:52:52.875 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap_items": [], 2022-06-17T12:52:52.875 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_temp": [], 2022-06-17T12:52:52.875 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_temp": [], 2022-06-17T12:52:52.876 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklist": {}, 2022-06-17T12:52:52.876 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profiles": { 2022-06-17T12:52:52.876 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs_data": { 2022-06-17T12:52:52.876 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-device-class": "", 2022-06-17T12:52:52.877 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:52:52.877 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-root": "default", 2022-06-17T12:52:52.877 INFO:teuthology.orchestra.run.smithi036.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-17T12:52:52.877 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:52:52.877 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "2", 2022-06-17T12:52:52.878 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:52:52.878 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:52:52.878 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van", 2022-06-17T12:52:52.878 INFO:teuthology.orchestra.run.smithi036.stdout: "w": "8" 2022-06-17T12:52:52.879 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.879 INFO:teuthology.orchestra.run.smithi036.stdout: "default": { 2022-06-17T12:52:52.879 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:52:52.879 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:52:52.879 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "1", 2022-06-17T12:52:52.880 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:52:52.880 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:52:52.880 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van" 2022-06-17T12:52:52.880 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:52.880 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:52.881 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps_queue": [], 2022-06-17T12:52:52.881 INFO:teuthology.orchestra.run.smithi036.stdout: "new_removed_snaps": [], 2022-06-17T12:52:52.881 INFO:teuthology.orchestra.run.smithi036.stdout: "new_purged_snaps": [], 2022-06-17T12:52:52.881 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_node_flags": {}, 2022-06-17T12:52:52.882 INFO:teuthology.orchestra.run.smithi036.stdout: "device_class_flags": {} 2022-06-17T12:52:52.882 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:52:52.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.634+0000 7f8a4cd91700 1 -- 172.21.15.36:0/1271605635 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8a2c053940 msgr2=0x7f8a2c055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:52.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.634+0000 7f8a4cd91700 1 --2- 172.21.15.36:0/1271605635 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8a2c053940 0x7f8a2c055e00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f8a3000c000 tx=0x7f8a30008040).stop 2022-06-17T12:52:52.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.634+0000 7f8a4cd91700 1 -- 172.21.15.36:0/1271605635 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8a480b6800 msgr2=0x7f8a4810fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:52.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.634+0000 7f8a4cd91700 1 --2- 172.21.15.36:0/1271605635 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8a480b6800 0x7f8a4810fb70 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f8a380068e0 tx=0x7f8a38008040).stop 2022-06-17T12:52:52.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.634+0000 7f8a4cd91700 1 -- 172.21.15.36:0/1271605635 shutdown_connections 2022-06-17T12:52:52.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.634+0000 7f8a4cd91700 1 --2- 172.21.15.36:0/1271605635 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8a2c053940 0x7f8a2c055e00 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:52.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.634+0000 7f8a4cd91700 1 --2- 172.21.15.36:0/1271605635 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a4814acc0 0x7f8a481451e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:52.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.634+0000 7f8a4cd91700 1 --2- 172.21.15.36:0/1271605635 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8a480b6800 0x7f8a4810fb70 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:52.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.634+0000 7f8a4cd91700 1 --2- 172.21.15.36:0/1271605635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a480aeea0 0x7f8a4810f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:52.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.634+0000 7f8a4cd91700 1 -- 172.21.15.36:0/1271605635 >> 172.21.15.36:0/1271605635 conn(0x7f8a480614b0 msgr2=0x7f8a480b5580 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:52.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.634+0000 7f8a4cd91700 1 -- 172.21.15.36:0/1271605635 shutdown_connections 2022-06-17T12:52:52.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:52.634+0000 7f8a4cd91700 1 -- 172.21.15.36:0/1271605635 wait complete. 2022-06-17T12:52:52.887 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:52:53.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.014+0000 7fe74f3b3700 1 Processor -- start 2022-06-17T12:52:53.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.014+0000 7fe74f3b3700 1 -- start start 2022-06-17T12:52:53.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74f3b3700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe7480aef60 0x7fe7480b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:53.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74f3b3700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe7480ba2e0 0x7fe7481442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:53.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74f3b3700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7480af4b0 con 0x7fe74813cb70 2022-06-17T12:52:53.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74f3b3700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe748143da0 con 0x7fe7480ba2e0 2022-06-17T12:52:53.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74f3b3700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe748142f50 con 0x7fe7480aef60 2022-06-17T12:52:53.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74ebb2700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe7480ba2e0 0x7fe7481442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:53.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74ebb2700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe7480ba2e0 0x7fe7481442a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35160/0 (socket says 172.21.15.36:35160) 2022-06-17T12:52:53.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74ebb2700 1 -- 172.21.15.36:0/272519457 learned_addr learned my addr 172.21.15.36:0/272519457 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:53.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74dbb0700 1 --2- 172.21.15.36:0/272519457 >> v2:172.21.15.70:3300/0 conn(0x7fe7480aef60 0x7fe7480b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:53.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74ebb2700 1 -- 172.21.15.36:0/272519457 >> v2:172.21.15.70:3300/0 conn(0x7fe7480aef60 msgr2=0x7fe7480b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:53.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74ebb2700 1 --2- 172.21.15.36:0/272519457 >> v2:172.21.15.70:3300/0 conn(0x7fe7480aef60 0x7fe7480b9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74ebb2700 1 -- 172.21.15.36:0/272519457 >> v1:172.21.15.70:6789/0 conn(0x7fe74813cb70 legacy=0x7fe748143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:53.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74ebb2700 1 -- 172.21.15.36:0/272519457 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7481431f0 con 0x7fe7480ba2e0 2022-06-17T12:52:53.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74ebb2700 1 --2- 172.21.15.36:0/272519457 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe7480ba2e0 0x7fe7481442a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7fe7400032f0 tx=0x7fe740009290).ready entity=mon.0 client_cookie=ac983abeca1334cb server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:53.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74d3af700 1 -- 172.21.15.36:0/272519457 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe74000b870 con 0x7fe7480ba2e0 2022-06-17T12:52:53.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74d3af700 1 -- 172.21.15.36:0/272519457 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74000b9d0 con 0x7fe7480ba2e0 2022-06-17T12:52:53.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74d3af700 1 -- 172.21.15.36:0/272519457 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe74000bb40 con 0x7fe7480ba2e0 2022-06-17T12:52:53.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74f3b3700 1 -- 172.21.15.36:0/272519457 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe7480ba2e0 msgr2=0x7fe7481442a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:53.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74f3b3700 1 --2- 172.21.15.36:0/272519457 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe7480ba2e0 0x7fe7481442a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7fe7400032f0 tx=0x7fe740009290).stop 2022-06-17T12:52:53.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74f3b3700 1 -- 172.21.15.36:0/272519457 shutdown_connections 2022-06-17T12:52:53.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74f3b3700 1 --2- 172.21.15.36:0/272519457 >> v2:172.21.15.70:3300/0 conn(0x7fe7480aef60 0x7fe7480b9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74f3b3700 1 --2- 172.21.15.36:0/272519457 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe7480ba2e0 0x7fe7481442a0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74f3b3700 1 -- 172.21.15.36:0/272519457 >> 172.21.15.36:0/272519457 conn(0x7fe74805c550 msgr2=0x7fe7480b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:53.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74f3b3700 1 -- 172.21.15.36:0/272519457 shutdown_connections 2022-06-17T12:52:53.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.018+0000 7fe74f3b3700 1 -- 172.21.15.36:0/272519457 wait complete. 2022-06-17T12:52:53.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.022+0000 7fe74f3b3700 1 Processor -- start 2022-06-17T12:52:53.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.022+0000 7fe74f3b3700 1 -- start start 2022-06-17T12:52:53.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.022+0000 7fe74f3b3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7480aef60 0x7fe74813ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:53.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.022+0000 7fe74f3b3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7480ba2e0 0x7fe748112d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:53.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.022+0000 7fe74f3b3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe74813cb70 0x7fe748152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:53.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.022+0000 7fe74f3b3700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7480b0b60 con 0x7fe74813cb70 2022-06-17T12:52:53.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.022+0000 7fe74f3b3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74813f8d0 con 0x7fe7480aef60 2022-06-17T12:52:53.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.022+0000 7fe74f3b3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7480b0dc0 con 0x7fe7480ba2e0 2022-06-17T12:52:53.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.022+0000 7fe74ebb2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe74813cb70 0x7fe748152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:53.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.022+0000 7fe74ebb2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe74813cb70 0x7fe748152740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35166/0 (socket says 172.21.15.36:35166) 2022-06-17T12:52:53.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.022+0000 7fe74ebb2700 1 -- 172.21.15.36:0/548565860 learned_addr learned my addr 172.21.15.36:0/548565860 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:53.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.022+0000 7fe74dbb0700 1 --2- 172.21.15.36:0/548565860 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7480ba2e0 0x7fe748112d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:53.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.022+0000 7fe74e3b1700 1 --2- 172.21.15.36:0/548565860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7480aef60 0x7fe74813ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:53.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.022+0000 7fe74ebb2700 1 -- 172.21.15.36:0/548565860 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7480ba2e0 msgr2=0x7fe748112d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:53.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.022+0000 7fe74ebb2700 1 --2- 172.21.15.36:0/548565860 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7480ba2e0 0x7fe748112d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.022+0000 7fe74ebb2700 1 -- 172.21.15.36:0/548565860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7480aef60 msgr2=0x7fe74813ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:53.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.022+0000 7fe74ebb2700 1 --2- 172.21.15.36:0/548565860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7480aef60 0x7fe74813ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.022+0000 7fe74ebb2700 1 -- 172.21.15.36:0/548565860 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe740009530 con 0x7fe74813cb70 2022-06-17T12:52:53.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.026+0000 7fe74ebb2700 1 --2- 172.21.15.36:0/548565860 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe74813cb70 0x7fe748152740 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7fe7400032f0 tx=0x7fe740016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:53.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.026+0000 7fe73b7fe700 1 -- 172.21.15.36:0/548565860 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe74000bb40 con 0x7fe74813cb70 2022-06-17T12:52:53.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.026+0000 7fe73b7fe700 1 -- 172.21.15.36:0/548565860 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe740021630 con 0x7fe74813cb70 2022-06-17T12:52:53.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.026+0000 7fe74f3b3700 1 -- 172.21.15.36:0/548565860 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7481128d0 con 0x7fe74813cb70 2022-06-17T12:52:53.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.026+0000 7fe73b7fe700 1 -- 172.21.15.36:0/548565860 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe74000b870 con 0x7fe74813cb70 2022-06-17T12:52:53.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.026+0000 7fe74f3b3700 1 -- 172.21.15.36:0/548565860 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7480b3b80 con 0x7fe74813cb70 2022-06-17T12:52:53.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.026+0000 7fe74f3b3700 1 -- 172.21.15.36:0/548565860 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe748112a90 con 0x7fe74813cb70 2022-06-17T12:52:53.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.026+0000 7fe73b7fe700 1 -- 172.21.15.36:0/548565860 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe740027020 con 0x7fe74813cb70 2022-06-17T12:52:53.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.026+0000 7fe73b7fe700 1 --2- 172.21.15.36:0/548565860 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe728053ad0 0x7fe728055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:53.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.030+0000 7fe73b7fe700 1 -- 172.21.15.36:0/548565860 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe74006d440 con 0x7fe74813cb70 2022-06-17T12:52:53.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.030+0000 7fe74e3b1700 1 --2- 172.21.15.36:0/548565860 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe728053ad0 0x7fe728055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:53.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.030+0000 7fe74e3b1700 1 --2- 172.21.15.36:0/548565860 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe728053ad0 0x7fe728055f90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fe73c00d000 tx=0x7fe73c007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:53.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.034+0000 7fe73b7fe700 1 -- 172.21.15.36:0/548565860 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe740044130 con 0x7fe74813cb70 2022-06-17T12:52:53.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.202+0000 7fe74f3b3700 1 -- 172.21.15.36:0/548565860 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe748057410 con 0x7fe74813cb70 2022-06-17T12:52:53.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.202+0000 7fe73b7fe700 1 -- 172.21.15.36:0/548565860 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2140 (secure 0 0 0) 0x7fe740015020 con 0x7fe74813cb70 2022-06-17T12:52:53.189 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:52:53.190 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":30,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5420,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/3105680199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3105680199},{"type":"v1","addr":"172.21.15.70:6833","nonce":3105680199}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":29}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:52:22.253232+0000","modified":"2022-06-17T12:52:23.257219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6054},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6054":{"gid":6054,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/4091208534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":4091208534},{"type":"v1","addr":"172.21.15.36:6835","nonce":4091208534}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-06-17T12:52:53.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.202+0000 7fe74f3b3700 1 -- 172.21.15.36:0/548565860 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe728053ad0 msgr2=0x7fe728055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:53.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.206+0000 7fe74f3b3700 1 --2- 172.21.15.36:0/548565860 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe728053ad0 0x7fe728055f90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fe73c00d000 tx=0x7fe73c007b90).stop 2022-06-17T12:52:53.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.206+0000 7fe74f3b3700 1 -- 172.21.15.36:0/548565860 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe74813cb70 msgr2=0x7fe748152740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:53.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.206+0000 7fe74f3b3700 1 --2- 172.21.15.36:0/548565860 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe74813cb70 0x7fe748152740 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7fe7400032f0 tx=0x7fe740016000).stop 2022-06-17T12:52:53.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.206+0000 7fe74f3b3700 1 -- 172.21.15.36:0/548565860 shutdown_connections 2022-06-17T12:52:53.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.206+0000 7fe74f3b3700 1 --2- 172.21.15.36:0/548565860 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe728053ad0 0x7fe728055f90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.206+0000 7fe74f3b3700 1 --2- 172.21.15.36:0/548565860 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe74813cb70 0x7fe748152740 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.206+0000 7fe74f3b3700 1 --2- 172.21.15.36:0/548565860 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7480ba2e0 0x7fe748112d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.206+0000 7fe74f3b3700 1 --2- 172.21.15.36:0/548565860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7480aef60 0x7fe74813ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.206+0000 7fe74f3b3700 1 -- 172.21.15.36:0/548565860 >> 172.21.15.36:0/548565860 conn(0x7fe74805c550 msgr2=0x7fe7481452c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:53.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.206+0000 7fe74f3b3700 1 -- 172.21.15.36:0/548565860 shutdown_connections 2022-06-17T12:52:53.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.206+0000 7fe74f3b3700 1 -- 172.21.15.36:0/548565860 wait complete. 2022-06-17T12:52:53.197 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 30 2022-06-17T12:52:53.201 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-17T12:52:53.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.318+0000 7efe44062700 1 Processor -- start 2022-06-17T12:52:53.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.318+0000 7efe44062700 1 -- start start 2022-06-17T12:52:53.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.318+0000 7efe44062700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7efe3c0b6600 0x7efe3c146da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:53.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.318+0000 7efe44062700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efe3c0b56f0 0x7efe3c1420a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:53.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.318+0000 7efe44062700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe3c0b48e0 con 0x7efe3c1469c0 2022-06-17T12:52:53.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.318+0000 7efe44062700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe3c147290 con 0x7efe3c0b6600 2022-06-17T12:52:53.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.318+0000 7efe44062700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe3c0b4b40 con 0x7efe3c0b56f0 2022-06-17T12:52:53.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.322+0000 7efe415fd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7efe3c0b6600 0x7efe3c146da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:53.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.322+0000 7efe415fd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7efe3c0b6600 0x7efe3c146da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35174/0 (socket says 172.21.15.36:35174) 2022-06-17T12:52:53.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.322+0000 7efe415fd700 1 -- 172.21.15.36:0/1692742276 learned_addr learned my addr 172.21.15.36:0/1692742276 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:53.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.322+0000 7efe425ff700 1 --2- 172.21.15.36:0/1692742276 >> v2:172.21.15.70:3300/0 conn(0x7efe3c0b56f0 0x7efe3c1420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:53.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.322+0000 7efe415fd700 1 -- 172.21.15.36:0/1692742276 >> v2:172.21.15.70:3300/0 conn(0x7efe3c0b56f0 msgr2=0x7efe3c1420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:53.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.322+0000 7efe415fd700 1 --2- 172.21.15.36:0/1692742276 >> v2:172.21.15.70:3300/0 conn(0x7efe3c0b56f0 0x7efe3c1420a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.322+0000 7efe415fd700 1 -- 172.21.15.36:0/1692742276 >> v1:172.21.15.70:6789/0 conn(0x7efe3c1469c0 legacy=0x7efe3c0b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:53.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.322+0000 7efe415fd700 1 -- 172.21.15.36:0/1692742276 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3c0b6aa0 con 0x7efe3c0b6600 2022-06-17T12:52:53.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.322+0000 7efe415fd700 1 --2- 172.21.15.36:0/1692742276 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe3c0b6600 0x7efe3c146da0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7efe3c0b1060 tx=0x7efe2c006eb0).ready entity=mon.0 client_cookie=d0d659688db9b869 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:53.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.322+0000 7efe40dfc700 1 -- 172.21.15.36:0/1692742276 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe2c00d610 con 0x7efe3c0b6600 2022-06-17T12:52:53.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.322+0000 7efe40dfc700 1 -- 172.21.15.36:0/1692742276 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c000e90 con 0x7efe3c0b6600 2022-06-17T12:52:53.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.322+0000 7efe40dfc700 1 -- 172.21.15.36:0/1692742276 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe2c00d9a0 con 0x7efe3c0b6600 2022-06-17T12:52:53.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.322+0000 7efe44062700 1 -- 172.21.15.36:0/1692742276 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe3c0b6600 msgr2=0x7efe3c146da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:53.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.322+0000 7efe44062700 1 --2- 172.21.15.36:0/1692742276 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe3c0b6600 0x7efe3c146da0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7efe3c0b1060 tx=0x7efe2c006eb0).stop 2022-06-17T12:52:53.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.322+0000 7efe44062700 1 -- 172.21.15.36:0/1692742276 shutdown_connections 2022-06-17T12:52:53.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.322+0000 7efe44062700 1 --2- 172.21.15.36:0/1692742276 >> v2:172.21.15.70:3300/0 conn(0x7efe3c0b56f0 0x7efe3c1420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.322+0000 7efe44062700 1 --2- 172.21.15.36:0/1692742276 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe3c0b6600 0x7efe3c146da0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.322+0000 7efe44062700 1 -- 172.21.15.36:0/1692742276 >> 172.21.15.36:0/1692742276 conn(0x7efe3c05f770 msgr2=0x7efe3c05fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:53.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.322+0000 7efe44062700 1 -- 172.21.15.36:0/1692742276 shutdown_connections 2022-06-17T12:52:53.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.322+0000 7efe44062700 1 -- 172.21.15.36:0/1692742276 wait complete. 2022-06-17T12:52:53.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.322+0000 7efe44062700 1 Processor -- start 2022-06-17T12:52:53.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.326+0000 7efe44062700 1 -- start start 2022-06-17T12:52:53.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.326+0000 7efe44062700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe3c0b56f0 0x7efe3c0ae310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:53.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.326+0000 7efe44062700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe3c0b6600 0x7efe3c1404c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:53.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.326+0000 7efe44062700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe3c1469c0 0x7efe3c151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:53.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.326+0000 7efe44062700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c0af2a0 con 0x7efe3c1469c0 2022-06-17T12:52:53.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.326+0000 7efe44062700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c0af4d0 con 0x7efe3c0b6600 2022-06-17T12:52:53.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.326+0000 7efe44062700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c0af770 con 0x7efe3c0b56f0 2022-06-17T12:52:53.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.326+0000 7efe425ff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe3c1469c0 0x7efe3c151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:53.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.326+0000 7efe425ff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe3c1469c0 0x7efe3c151c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35180/0 (socket says 172.21.15.36:35180) 2022-06-17T12:52:53.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.326+0000 7efe425ff700 1 -- 172.21.15.36:0/631533373 learned_addr learned my addr 172.21.15.36:0/631533373 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:53.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.326+0000 7efe415fd700 1 --2- 172.21.15.36:0/631533373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe3c0b6600 0x7efe3c1404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:53.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.326+0000 7efe41dfe700 1 --2- 172.21.15.36:0/631533373 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe3c0b56f0 0x7efe3c0ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:53.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.326+0000 7efe425ff700 1 -- 172.21.15.36:0/631533373 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe3c0b56f0 msgr2=0x7efe3c0ae310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:53.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.326+0000 7efe425ff700 1 --2- 172.21.15.36:0/631533373 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe3c0b56f0 0x7efe3c0ae310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.326+0000 7efe425ff700 1 -- 172.21.15.36:0/631533373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe3c0b6600 msgr2=0x7efe3c1404c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:53.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.326+0000 7efe425ff700 1 --2- 172.21.15.36:0/631533373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe3c0b6600 0x7efe3c1404c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.326+0000 7efe425ff700 1 -- 172.21.15.36:0/631533373 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2c00b1e0 con 0x7efe3c1469c0 2022-06-17T12:52:53.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.326+0000 7efe425ff700 1 --2- 172.21.15.36:0/631533373 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe3c1469c0 0x7efe3c151c10 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7efe3400c000 tx=0x7efe340091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:53.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.326+0000 7efe2affd700 1 -- 172.21.15.36:0/631533373 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe34012600 con 0x7efe3c1469c0 2022-06-17T12:52:53.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.326+0000 7efe2affd700 1 -- 172.21.15.36:0/631533373 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe34012760 con 0x7efe3c1469c0 2022-06-17T12:52:53.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.326+0000 7efe2affd700 1 -- 172.21.15.36:0/631533373 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe34012ac0 con 0x7efe3c1469c0 2022-06-17T12:52:53.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.326+0000 7efe44062700 1 -- 172.21.15.36:0/631533373 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3c1409b0 con 0x7efe3c1469c0 2022-06-17T12:52:53.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.326+0000 7efe44062700 1 -- 172.21.15.36:0/631533373 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3c155ce0 con 0x7efe3c1469c0 2022-06-17T12:52:53.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.330+0000 7efe44062700 1 -- 172.21.15.36:0/631533373 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe3c057410 con 0x7efe3c1469c0 2022-06-17T12:52:53.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.330+0000 7efe2affd700 1 -- 172.21.15.36:0/631533373 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7efe34022070 con 0x7efe3c1469c0 2022-06-17T12:52:53.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.330+0000 7efe2affd700 1 --2- 172.21.15.36:0/631533373 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efe1c053ad0 0x7efe1c055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:53.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.330+0000 7efe41dfe700 1 --2- 172.21.15.36:0/631533373 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efe1c053ad0 0x7efe1c055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:53.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.330+0000 7efe2affd700 1 -- 172.21.15.36:0/631533373 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7efe3406c6a0 con 0x7efe3c1469c0 2022-06-17T12:52:53.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.330+0000 7efe41dfe700 1 --2- 172.21.15.36:0/631533373 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efe1c053ad0 0x7efe1c055f90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7efe3800d000 tx=0x7efe38007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:53.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.334+0000 7efe2affd700 1 -- 172.21.15.36:0/631533373 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efe340433d0 con 0x7efe3c1469c0 2022-06-17T12:52:53.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.502+0000 7efe44062700 1 -- 172.21.15.36:0/631533373 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7efe3c05d820 con 0x7efe3c1469c0 2022-06-17T12:52:53.656 INFO:tasks.ceph.mon.a.smithi036.stderr:2022-06-17T12:52:53.670+0000 7fb7a2649700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-17T12:52:53.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.678+0000 7efe2affd700 1 -- 172.21.15.36:0/631533373 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) v1 ==== 161+0+0 (secure 0 0 0) 0x7efe34016020 con 0x7efe3c1469c0 2022-06-17T12:52:53.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.682+0000 7efe44062700 1 -- 172.21.15.36:0/631533373 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efe1c053ad0 msgr2=0x7efe1c055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:53.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.682+0000 7efe44062700 1 --2- 172.21.15.36:0/631533373 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efe1c053ad0 0x7efe1c055f90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7efe3800d000 tx=0x7efe38007b90).stop 2022-06-17T12:52:53.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.682+0000 7efe44062700 1 -- 172.21.15.36:0/631533373 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe3c1469c0 msgr2=0x7efe3c151c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:53.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.682+0000 7efe44062700 1 --2- 172.21.15.36:0/631533373 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe3c1469c0 0x7efe3c151c10 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7efe3400c000 tx=0x7efe340091d0).stop 2022-06-17T12:52:53.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.682+0000 7efe44062700 1 -- 172.21.15.36:0/631533373 shutdown_connections 2022-06-17T12:52:53.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.682+0000 7efe44062700 1 --2- 172.21.15.36:0/631533373 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efe1c053ad0 0x7efe1c055f90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.682+0000 7efe44062700 1 --2- 172.21.15.36:0/631533373 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efe3c1469c0 0x7efe3c151c10 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.682+0000 7efe44062700 1 --2- 172.21.15.36:0/631533373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe3c0b6600 0x7efe3c1404c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.682+0000 7efe44062700 1 --2- 172.21.15.36:0/631533373 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe3c0b56f0 0x7efe3c0ae310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.682+0000 7efe44062700 1 -- 172.21.15.36:0/631533373 >> 172.21.15.36:0/631533373 conn(0x7efe3c05f770 msgr2=0x7efe3c13f310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:53.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.682+0000 7efe44062700 1 -- 172.21.15.36:0/631533373 shutdown_connections 2022-06-17T12:52:53.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.682+0000 7efe44062700 1 -- 172.21.15.36:0/631533373 wait complete. 2022-06-17T12:52:53.671 INFO:teuthology.orchestra.run.smithi036.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-17T12:52:53.680 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:52:53.701 INFO:tasks.ceph.mds.a.smithi036.stdout:starting mds.a at 2022-06-17T12:52:53.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649f59e700 1 Processor -- start 2022-06-17T12:52:53.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649f59e700 1 -- start start 2022-06-17T12:52:53.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649f59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64a00b2200 0x7f64a00af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:53.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649f59e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64a00b25e0 con 0x7f64a00af8d0 2022-06-17T12:52:53.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649f59e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64a014b6f0 con 0x7f64a00b2cd0 2022-06-17T12:52:53.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649f59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a00ad790 con 0x7f64a00b2200 2022-06-17T12:52:53.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649dd9b700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f64a00af8d0 0x7f64a014a8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52728/0 (socket says 172.21.15.36:52728) 2022-06-17T12:52:53.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649dd9b700 1 -- 172.21.15.36:0/2509864705 learned_addr learned my addr 172.21.15.36:0/2509864705 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:53.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649e59c700 1 --2- 172.21.15.36:0/2509864705 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64a00b2200 0x7f64a00af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:53.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649d59a700 1 -- 172.21.15.36:0/2509864705 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2711057673 0 0) 0x7f64a00b25e0 con 0x7f64a00af8d0 2022-06-17T12:52:53.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649d59a700 1 -- 172.21.15.36:0/2509864705 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6480003260 con 0x7f64a00af8d0 2022-06-17T12:52:53.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649d59a700 1 -- 172.21.15.36:0/2509864705 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1461936281 0 0) 0x7f64a014b6f0 con 0x7f64a00b2cd0 2022-06-17T12:52:53.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649d59a700 1 -- 172.21.15.36:0/2509864705 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64a00b25e0 con 0x7f64a00b2cd0 2022-06-17T12:52:53.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649d59a700 1 -- 172.21.15.36:0/2509864705 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 622594299 0 0) 0x7f6480003260 con 0x7f64a00af8d0 2022-06-17T12:52:53.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649d59a700 1 -- 172.21.15.36:0/2509864705 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64a00b2200 msgr2=0x7f64a00af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:53.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649d59a700 1 --2- 172.21.15.36:0/2509864705 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64a00b2200 0x7f64a00af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649d59a700 1 -- 172.21.15.36:0/2509864705 >> v1:172.21.15.70:6789/0 conn(0x7f64a00b2cd0 legacy=0x7f64a014afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:53.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649d59a700 1 -- 172.21.15.36:0/2509864705 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a00aef40 con 0x7f64a00af8d0 2022-06-17T12:52:53.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649d59a700 1 -- 172.21.15.36:0/2509864705 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f6498001a00 con 0x7f64a00af8d0 2022-06-17T12:52:53.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649d59a700 1 -- 172.21.15.36:0/2509864705 >> v1:172.21.15.36:6789/0 conn(0x7f64a00af8d0 legacy=0x7f64a014a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:53.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649d59a700 1 --2- 172.21.15.36:0/2509864705 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6480003910 0x7f6480003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:53.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649d59a700 1 --2- 172.21.15.36:0/2509864705 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64a00b2200 0x7f6480004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:53.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649d59a700 1 --2- 172.21.15.36:0/2509864705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64800049f0 0x7f6480004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:53.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649d59a700 1 -- 172.21.15.36:0/2509864705 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a00ad790 con 0x7f6480003910 2022-06-17T12:52:53.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649d59a700 1 -- 172.21.15.36:0/2509864705 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648000b780 con 0x7f64800049f0 2022-06-17T12:52:53.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649d59a700 1 -- 172.21.15.36:0/2509864705 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f648000b940 con 0x7f64a00b2200 2022-06-17T12:52:53.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649e59c700 1 --2- 172.21.15.36:0/2509864705 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6480003910 0x7f6480003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:53.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649ed9d700 1 --2- 172.21.15.36:0/2509864705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64800049f0 0x7f6480004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:53.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649dd9b700 1 --2- 172.21.15.36:0/2509864705 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64a00b2200 0x7f6480004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:53.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649e59c700 1 -- 172.21.15.36:0/2509864705 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64a00b2200 msgr2=0x7f6480004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:53.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649e59c700 1 --2- 172.21.15.36:0/2509864705 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64a00b2200 0x7f6480004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649e59c700 1 -- 172.21.15.36:0/2509864705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64800049f0 msgr2=0x7f6480004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:53.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649e59c700 1 --2- 172.21.15.36:0/2509864705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64800049f0 0x7f6480004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649e59c700 1 -- 172.21.15.36:0/2509864705 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6480003260 con 0x7f6480003910 2022-06-17T12:52:53.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649e59c700 1 --2- 172.21.15.36:0/2509864705 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6480003910 0x7f6480003d10 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f649000c000 tx=0x7f6490006df0).ready entity=mon.0 client_cookie=4e9aae32dfdbcaf8 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:53.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649d59a700 1 -- 172.21.15.36:0/2509864705 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f649000e830 con 0x7f6480003910 2022-06-17T12:52:53.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649d59a700 1 -- 172.21.15.36:0/2509864705 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649000e990 con 0x7f6480003910 2022-06-17T12:52:53.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.798+0000 7f649d59a700 1 -- 172.21.15.36:0/2509864705 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f649000ecf0 con 0x7f6480003910 2022-06-17T12:52:53.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649f59e700 1 -- 172.21.15.36:0/2509864705 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6480003910 msgr2=0x7f6480003d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:53.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649f59e700 1 --2- 172.21.15.36:0/2509864705 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6480003910 0x7f6480003d10 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f649000c000 tx=0x7f6490006df0).stop 2022-06-17T12:52:53.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649ed9d700 1 -- 172.21.15.36:0/2509864705 reap_dead start 2022-06-17T12:52:53.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649f59e700 1 -- 172.21.15.36:0/2509864705 shutdown_connections 2022-06-17T12:52:53.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649f59e700 1 -- 172.21.15.36:0/2509864705 >> 172.21.15.36:0/2509864705 conn(0x7f64a005f770 msgr2=0x7f64a005fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:53.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649f59e700 1 -- 172.21.15.36:0/2509864705 shutdown_connections 2022-06-17T12:52:53.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649f59e700 1 -- 172.21.15.36:0/2509864705 wait complete. 2022-06-17T12:52:53.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649f59e700 1 Processor -- start 2022-06-17T12:52:53.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649f59e700 1 -- start start 2022-06-17T12:52:53.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649f59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6480003910 0x7f64a0064a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:53.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649f59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64a0156750 0x7f64a0159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:53.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649f59e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64a0065900 0x7f64a015a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:53.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649f59e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a0156c30 con 0x7f64a0156750 2022-06-17T12:52:53.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649f59e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a0156ed0 con 0x7f64a0065900 2022-06-17T12:52:53.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649f59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a0157170 con 0x7f6480003910 2022-06-17T12:52:53.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649dd9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64a0156750 0x7f64a0159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:53.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649dd9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64a0156750 0x7f64a0159d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35208/0 (socket says 172.21.15.36:35208) 2022-06-17T12:52:53.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649dd9b700 1 -- 172.21.15.36:0/3861669027 learned_addr learned my addr 172.21.15.36:0/3861669027 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:53.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649ed9d700 1 --2- 172.21.15.36:0/3861669027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64a0065900 0x7f64a015a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:53.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649e59c700 1 --2- 172.21.15.36:0/3861669027 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6480003910 0x7f64a0064a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:53.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649dd9b700 1 -- 172.21.15.36:0/3861669027 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6480003910 msgr2=0x7f64a0064a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:53.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649dd9b700 1 --2- 172.21.15.36:0/3861669027 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6480003910 0x7f64a0064a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649dd9b700 1 -- 172.21.15.36:0/3861669027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64a0065900 msgr2=0x7f64a015a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:53.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649dd9b700 1 --2- 172.21.15.36:0/3861669027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64a0065900 0x7f64a015a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649dd9b700 1 -- 172.21.15.36:0/3861669027 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a0157610 con 0x7f64a0156750 2022-06-17T12:52:53.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649ed9d700 1 --2- 172.21.15.36:0/3861669027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64a0065900 0x7f64a015a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:52:53.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649dd9b700 1 --2- 172.21.15.36:0/3861669027 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64a0156750 0x7f64a0159d60 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f649800b000 tx=0x7f64a00ae260).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:53.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f64977fe700 1 -- 172.21.15.36:0/3861669027 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f649800c800 con 0x7f64a0156750 2022-06-17T12:52:53.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f64977fe700 1 -- 172.21.15.36:0/3861669027 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649800c960 con 0x7f64a0156750 2022-06-17T12:52:53.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649f59e700 1 -- 172.21.15.36:0/3861669027 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a0066bf0 con 0x7f64a0156750 2022-06-17T12:52:53.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f64977fe700 1 -- 172.21.15.36:0/3861669027 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f649800ccc0 con 0x7f64a0156750 2022-06-17T12:52:53.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.802+0000 7f649f59e700 1 -- 172.21.15.36:0/3861669027 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a015a8e0 con 0x7f64a0156750 2022-06-17T12:52:53.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.806+0000 7f649f59e700 1 -- 172.21.15.36:0/3861669027 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64a0157330 con 0x7f64a0156750 2022-06-17T12:52:53.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.806+0000 7f64977fe700 1 -- 172.21.15.36:0/3861669027 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6498014430 con 0x7f64a0156750 2022-06-17T12:52:53.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.810+0000 7f64977fe700 1 --2- 172.21.15.36:0/3861669027 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6484053a60 0x7f6484055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:53.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.810+0000 7f649e59c700 1 --2- 172.21.15.36:0/3861669027 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6484053a60 0x7f6484055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:53.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.810+0000 7f64977fe700 1 -- 172.21.15.36:0/3861669027 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f649806cc40 con 0x7f64a0156750 2022-06-17T12:52:53.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.810+0000 7f649e59c700 1 --2- 172.21.15.36:0/3861669027 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6484053a60 0x7f6484055f20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f649001e000 tx=0x7f6490019000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:53.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.814+0000 7f64977fe700 1 -- 172.21.15.36:0/3861669027 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6498043880 con 0x7f64a0156750 2022-06-17T12:52:53.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.974+0000 7f649f59e700 1 -- 172.21.15.36:0/3861669027 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64a00b1950 con 0x7f64a0156750 2022-06-17T12:52:53.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.974+0000 7f64977fe700 1 -- 172.21.15.36:0/3861669027 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+1779 (secure 0 0 0) 0x7f6498046ea0 con 0x7f64a0156750 2022-06-17T12:52:53.961 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:52:53.961 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":31,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5420,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/3105680199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3105680199},{"type":"v1","addr":"172.21.15.70:6833","nonce":3105680199}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":29}],"filesystems":[{"mdsmap":{"epoch":31,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:52:22.253232+0000","modified":"2022-06-17T12:52:53.672351+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-06-17T12:52:53.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.978+0000 7f649f59e700 1 -- 172.21.15.36:0/3861669027 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6484053a60 msgr2=0x7f6484055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:53.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.978+0000 7f649f59e700 1 --2- 172.21.15.36:0/3861669027 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6484053a60 0x7f6484055f20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f649001e000 tx=0x7f6490019000).stop 2022-06-17T12:52:53.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.978+0000 7f649f59e700 1 -- 172.21.15.36:0/3861669027 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64a0156750 msgr2=0x7f64a0159d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:53.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.978+0000 7f649f59e700 1 --2- 172.21.15.36:0/3861669027 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64a0156750 0x7f64a0159d60 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f649800b000 tx=0x7f64a00ae260).stop 2022-06-17T12:52:53.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.978+0000 7f649f59e700 1 -- 172.21.15.36:0/3861669027 shutdown_connections 2022-06-17T12:52:53.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.978+0000 7f649f59e700 1 --2- 172.21.15.36:0/3861669027 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6484053a60 0x7f6484055f20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.978+0000 7f649f59e700 1 --2- 172.21.15.36:0/3861669027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64a0065900 0x7f64a015a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.978+0000 7f649f59e700 1 --2- 172.21.15.36:0/3861669027 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64a0156750 0x7f64a0159d60 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.978+0000 7f649f59e700 1 --2- 172.21.15.36:0/3861669027 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6480003910 0x7f64a0064a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:53.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.978+0000 7f649f59e700 1 -- 172.21.15.36:0/3861669027 >> 172.21.15.36:0/3861669027 conn(0x7f64a005f770 msgr2=0x7f64a00b4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:53.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.978+0000 7f649f59e700 1 -- 172.21.15.36:0/3861669027 shutdown_connections 2022-06-17T12:52:53.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:53.978+0000 7f649f59e700 1 -- 172.21.15.36:0/3861669027 wait complete. 2022-06-17T12:52:53.967 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 31 2022-06-17T12:52:53.973 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-17T12:52:54.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.094+0000 7fe101daa700 1 Processor -- start 2022-06-17T12:52:54.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.094+0000 7fe101daa700 1 -- start start 2022-06-17T12:52:54.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe101daa700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0fc14b2b0 0x7fe0fc060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:54.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe101daa700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe0fc147bd0 0x7fe0fc1442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:54.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe101daa700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0fc14b870 con 0x7fe0fc0b27b0 2022-06-17T12:52:54.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe101daa700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0fc0ac500 con 0x7fe0fc147bd0 2022-06-17T12:52:54.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe101daa700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc0ace90 con 0x7fe0fc14b2b0 2022-06-17T12:52:54.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe1015a9700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe0fc147bd0 0x7fe0fc1442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:54.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe100da8700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fe0fc0b27b0 0x7fe0fc0ac780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52746/0 (socket says 172.21.15.36:52746) 2022-06-17T12:52:54.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe100da8700 1 -- 172.21.15.36:0/3935404867 learned_addr learned my addr 172.21.15.36:0/3935404867 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:54.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe0fbfff700 1 --2- 172.21.15.36:0/3935404867 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0fc14b2b0 0x7fe0fc060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:54.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe1015a9700 1 -- 172.21.15.36:0/3935404867 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0fc14b2b0 msgr2=0x7fe0fc060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:54.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe1015a9700 1 --2- 172.21.15.36:0/3935404867 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0fc14b2b0 0x7fe0fc060ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:54.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe1015a9700 1 -- 172.21.15.36:0/3935404867 >> v1:172.21.15.36:6789/0 conn(0x7fe0fc0b27b0 legacy=0x7fe0fc0ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:54.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe0fb7fe700 1 -- 172.21.15.36:0/3935404867 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1443714999 0 0) 0x7fe0fc14b870 con 0x7fe0fc0b27b0 2022-06-17T12:52:54.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe1015a9700 1 -- 172.21.15.36:0/3935404867 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0fc0b2b90 con 0x7fe0fc147bd0 2022-06-17T12:52:54.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe1015a9700 1 --2- 172.21.15.36:0/3935404867 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0fc147bd0 0x7fe0fc1442d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7fe0f40032f0 tx=0x7fe0f40091d0).ready entity=mon.0 client_cookie=97bd5710c04b451b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:54.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe0fb7fe700 1 -- 172.21.15.36:0/3935404867 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0f400b6c0 con 0x7fe0fc147bd0 2022-06-17T12:52:54.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe0fb7fe700 1 -- 172.21.15.36:0/3935404867 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f400b820 con 0x7fe0fc147bd0 2022-06-17T12:52:54.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe0fb7fe700 1 -- 172.21.15.36:0/3935404867 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0f400b990 con 0x7fe0fc147bd0 2022-06-17T12:52:54.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe101daa700 1 -- 172.21.15.36:0/3935404867 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0fc147bd0 msgr2=0x7fe0fc1442d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:54.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe101daa700 1 --2- 172.21.15.36:0/3935404867 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0fc147bd0 0x7fe0fc1442d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7fe0f40032f0 tx=0x7fe0f40091d0).stop 2022-06-17T12:52:54.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe101daa700 1 -- 172.21.15.36:0/3935404867 shutdown_connections 2022-06-17T12:52:54.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe101daa700 1 --2- 172.21.15.36:0/3935404867 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0fc147bd0 0x7fe0fc1442d0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:54.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe101daa700 1 --2- 172.21.15.36:0/3935404867 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0fc14b2b0 0x7fe0fc060ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:54.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe101daa700 1 -- 172.21.15.36:0/3935404867 >> 172.21.15.36:0/3935404867 conn(0x7fe0fc0b5d10 msgr2=0x7fe0fc0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:54.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe101daa700 1 -- 172.21.15.36:0/3935404867 shutdown_connections 2022-06-17T12:52:54.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.098+0000 7fe101daa700 1 -- 172.21.15.36:0/3935404867 wait complete. 2022-06-17T12:52:54.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.102+0000 7fe101daa700 1 Processor -- start 2022-06-17T12:52:54.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.102+0000 7fe101daa700 1 -- start start 2022-06-17T12:52:54.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.102+0000 7fe101daa700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0fc0b27b0 0x7fe0fc155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:54.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.102+0000 7fe101daa700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0fc147bd0 0x7fe0fc0a9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:54.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.102+0000 7fe101daa700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0fc14b2b0 0x7fe0fc0aa260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:54.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.102+0000 7fe101daa700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc156dd0 con 0x7fe0fc14b2b0 2022-06-17T12:52:54.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.102+0000 7fe101daa700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc157040 con 0x7fe0fc147bd0 2022-06-17T12:52:54.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.102+0000 7fe101daa700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc1572e0 con 0x7fe0fc0b27b0 2022-06-17T12:52:54.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.102+0000 7fe1015a9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0fc14b2b0 0x7fe0fc0aa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:54.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.102+0000 7fe1015a9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0fc14b2b0 0x7fe0fc0aa260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35224/0 (socket says 172.21.15.36:35224) 2022-06-17T12:52:54.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.102+0000 7fe1015a9700 1 -- 172.21.15.36:0/2642738305 learned_addr learned my addr 172.21.15.36:0/2642738305 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:54.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.102+0000 7fe0fbfff700 1 --2- 172.21.15.36:0/2642738305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0fc147bd0 0x7fe0fc0a9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:54.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.102+0000 7fe100da8700 1 --2- 172.21.15.36:0/2642738305 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0fc0b27b0 0x7fe0fc155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:54.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.102+0000 7fe1015a9700 1 -- 172.21.15.36:0/2642738305 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0fc0b27b0 msgr2=0x7fe0fc155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:54.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.102+0000 7fe1015a9700 1 --2- 172.21.15.36:0/2642738305 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0fc0b27b0 0x7fe0fc155de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:54.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.102+0000 7fe1015a9700 1 -- 172.21.15.36:0/2642738305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0fc147bd0 msgr2=0x7fe0fc0a9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:54.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.102+0000 7fe1015a9700 1 --2- 172.21.15.36:0/2642738305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0fc147bd0 0x7fe0fc0a9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:54.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.102+0000 7fe1015a9700 1 -- 172.21.15.36:0/2642738305 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f4009470 con 0x7fe0fc14b2b0 2022-06-17T12:52:54.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.102+0000 7fe1015a9700 1 --2- 172.21.15.36:0/2642738305 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0fc14b2b0 0x7fe0fc0aa260 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7fe0f40032f0 tx=0x7fe0f4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:54.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.102+0000 7fe0f9ffb700 1 -- 172.21.15.36:0/2642738305 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0f400b990 con 0x7fe0fc14b2b0 2022-06-17T12:52:54.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.102+0000 7fe0f9ffb700 1 -- 172.21.15.36:0/2642738305 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f4021630 con 0x7fe0fc14b2b0 2022-06-17T12:52:54.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.102+0000 7fe0f9ffb700 1 -- 172.21.15.36:0/2642738305 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0f400b6c0 con 0x7fe0fc14b2b0 2022-06-17T12:52:54.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.102+0000 7fe101daa700 1 -- 172.21.15.36:0/2642738305 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0fc0644a0 con 0x7fe0fc14b2b0 2022-06-17T12:52:54.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.102+0000 7fe101daa700 1 -- 172.21.15.36:0/2642738305 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0fc0aa750 con 0x7fe0fc14b2b0 2022-06-17T12:52:54.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.106+0000 7fe101daa700 1 -- 172.21.15.36:0/2642738305 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0fc0aaa30 con 0x7fe0fc14b2b0 2022-06-17T12:52:54.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.106+0000 7fe0f9ffb700 1 -- 172.21.15.36:0/2642738305 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe0f4027020 con 0x7fe0fc14b2b0 2022-06-17T12:52:54.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.106+0000 7fe0f9ffb700 1 --2- 172.21.15.36:0/2642738305 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe0dc0539b0 0x7fe0dc055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:54.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.106+0000 7fe0f9ffb700 1 -- 172.21.15.36:0/2642738305 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fe0f406d790 con 0x7fe0fc14b2b0 2022-06-17T12:52:54.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.106+0000 7fe100da8700 1 --2- 172.21.15.36:0/2642738305 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe0dc0539b0 0x7fe0dc055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:54.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.106+0000 7fe100da8700 1 --2- 172.21.15.36:0/2642738305 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe0dc0539b0 0x7fe0dc055e70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7fe0f000d000 tx=0x7fe0f0009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:54.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.110+0000 7fe0f9ffb700 1 -- 172.21.15.36:0/2642738305 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe0f4044540 con 0x7fe0fc14b2b0 2022-06-17T12:52:54.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.270+0000 7fe101daa700 1 -- 172.21.15.36:0/2642738305 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe0fc057410 con 0x7fe0fc14b2b0 2022-06-17T12:52:54.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.690+0000 7fe0f9ffb700 1 -- 172.21.15.36:0/2642738305 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe0f40150a0 con 0x7fe0fc14b2b0 2022-06-17T12:52:54.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.694+0000 7fe101daa700 1 -- 172.21.15.36:0/2642738305 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe0dc0539b0 msgr2=0x7fe0dc055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:54.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.694+0000 7fe101daa700 1 --2- 172.21.15.36:0/2642738305 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe0dc0539b0 0x7fe0dc055e70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7fe0f000d000 tx=0x7fe0f0009040).stop 2022-06-17T12:52:54.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.694+0000 7fe101daa700 1 -- 172.21.15.36:0/2642738305 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0fc14b2b0 msgr2=0x7fe0fc0aa260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:54.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.694+0000 7fe101daa700 1 --2- 172.21.15.36:0/2642738305 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0fc14b2b0 0x7fe0fc0aa260 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7fe0f40032f0 tx=0x7fe0f4016000).stop 2022-06-17T12:52:54.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.698+0000 7fe101daa700 1 -- 172.21.15.36:0/2642738305 shutdown_connections 2022-06-17T12:52:54.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.698+0000 7fe101daa700 1 --2- 172.21.15.36:0/2642738305 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe0dc0539b0 0x7fe0dc055e70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:54.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.698+0000 7fe101daa700 1 --2- 172.21.15.36:0/2642738305 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0fc14b2b0 0x7fe0fc0aa260 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:54.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.698+0000 7fe101daa700 1 --2- 172.21.15.36:0/2642738305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0fc147bd0 0x7fe0fc0a9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:54.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.698+0000 7fe101daa700 1 --2- 172.21.15.36:0/2642738305 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0fc0b27b0 0x7fe0fc155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:54.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.698+0000 7fe101daa700 1 -- 172.21.15.36:0/2642738305 >> 172.21.15.36:0/2642738305 conn(0x7fe0fc0b5d10 msgr2=0x7fe0fc145be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:54.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.698+0000 7fe101daa700 1 -- 172.21.15.36:0/2642738305 shutdown_connections 2022-06-17T12:52:54.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.698+0000 7fe101daa700 1 -- 172.21.15.36:0/2642738305 wait complete. 2022-06-17T12:52:54.697 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-17T12:52:54.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.826+0000 7f272e184700 1 Processor -- start 2022-06-17T12:52:54.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.826+0000 7f272e184700 1 -- start start 2022-06-17T12:52:54.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f272e184700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f27280b45c0 0x7f27280b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:54.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f272e184700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f27280b4f00 0x7f27280b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:54.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f272e184700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27280b4160 con 0x7f27280b9860 2022-06-17T12:52:54.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f272e184700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27280b9c40 con 0x7f27280b45c0 2022-06-17T12:52:54.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f272e184700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27280627d0 con 0x7f27280b4f00 2022-06-17T12:52:54.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f272c981700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f27280b45c0 0x7f27280b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:54.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f272d182700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f27280b9860 0x7f2728143840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52762/0 (socket says 172.21.15.36:52762) 2022-06-17T12:52:54.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f272d182700 1 -- 172.21.15.36:0/754663755 learned_addr learned my addr 172.21.15.36:0/754663755 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:54.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f272d983700 1 --2- 172.21.15.36:0/754663755 >> v2:172.21.15.70:3300/0 conn(0x7f27280b4f00 0x7f27280b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:54.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f272c981700 1 -- 172.21.15.36:0/754663755 >> v2:172.21.15.70:3300/0 conn(0x7f27280b4f00 msgr2=0x7f27280b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:54.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f272c981700 1 --2- 172.21.15.36:0/754663755 >> v2:172.21.15.70:3300/0 conn(0x7f27280b4f00 0x7f27280b5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:54.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f272c981700 1 -- 172.21.15.36:0/754663755 >> v1:172.21.15.36:6789/0 conn(0x7f27280b9860 legacy=0x7f2728143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:54.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f271ffff700 1 -- 172.21.15.36:0/754663755 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 490754314 0 0) 0x7f27280b4160 con 0x7f27280b9860 2022-06-17T12:52:54.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f272c981700 1 -- 172.21.15.36:0/754663755 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2728062a70 con 0x7f27280b45c0 2022-06-17T12:52:54.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f272c981700 1 --2- 172.21.15.36:0/754663755 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f27280b45c0 0x7f27280b4a10 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f27280ba5d0 tx=0x7f2724006df0).ready entity=mon.0 client_cookie=54012d28e274659e server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:54.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f271ffff700 1 -- 172.21.15.36:0/754663755 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f272400d5c0 con 0x7f27280b45c0 2022-06-17T12:52:54.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f271ffff700 1 -- 172.21.15.36:0/754663755 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2724000e90 con 0x7f27280b45c0 2022-06-17T12:52:54.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f271ffff700 1 -- 172.21.15.36:0/754663755 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f272400d910 con 0x7f27280b45c0 2022-06-17T12:52:54.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f272e184700 1 -- 172.21.15.36:0/754663755 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f27280b45c0 msgr2=0x7f27280b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:54.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f272e184700 1 --2- 172.21.15.36:0/754663755 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f27280b45c0 0x7f27280b4a10 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f27280ba5d0 tx=0x7f2724006df0).stop 2022-06-17T12:52:54.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f272e184700 1 -- 172.21.15.36:0/754663755 shutdown_connections 2022-06-17T12:52:54.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f272e184700 1 --2- 172.21.15.36:0/754663755 >> v2:172.21.15.70:3300/0 conn(0x7f27280b4f00 0x7f27280b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:54.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f272e184700 1 --2- 172.21.15.36:0/754663755 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f27280b45c0 0x7f27280b4a10 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:54.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f272e184700 1 -- 172.21.15.36:0/754663755 >> 172.21.15.36:0/754663755 conn(0x7f272805c550 msgr2=0x7f2728060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:54.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f272e184700 1 -- 172.21.15.36:0/754663755 shutdown_connections 2022-06-17T12:52:54.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.830+0000 7f272e184700 1 -- 172.21.15.36:0/754663755 wait complete. 2022-06-17T12:52:54.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.834+0000 7f272e184700 1 Processor -- start 2022-06-17T12:52:54.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.834+0000 7f272e184700 1 -- start start 2022-06-17T12:52:54.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.834+0000 7f272e184700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f27280b45c0 0x7f272813d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:54.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.834+0000 7f272e184700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f27280b4f00 0x7f2728152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:54.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.834+0000 7f272e184700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27280b9860 0x7f2728152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:54.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.834+0000 7f272e184700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2728145b20 con 0x7f27280b4f00 2022-06-17T12:52:54.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.834+0000 7f272e184700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2728153150 con 0x7f27280b9860 2022-06-17T12:52:54.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.834+0000 7f272e184700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272813df10 con 0x7f27280b45c0 2022-06-17T12:52:54.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.834+0000 7f272c981700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f27280b4f00 0x7f2728152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:54.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.834+0000 7f272c981700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f27280b4f00 0x7f2728152260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35240/0 (socket says 172.21.15.36:35240) 2022-06-17T12:52:54.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.834+0000 7f272c981700 1 -- 172.21.15.36:0/4273671349 learned_addr learned my addr 172.21.15.36:0/4273671349 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:54.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.834+0000 7f272d983700 1 --2- 172.21.15.36:0/4273671349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27280b9860 0x7f2728152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:54.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.834+0000 7f272d182700 1 --2- 172.21.15.36:0/4273671349 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f27280b45c0 0x7f272813d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:54.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.834+0000 7f272c981700 1 -- 172.21.15.36:0/4273671349 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f27280b45c0 msgr2=0x7f272813d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:54.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.834+0000 7f272c981700 1 --2- 172.21.15.36:0/4273671349 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f27280b45c0 0x7f272813d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:54.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.834+0000 7f272c981700 1 -- 172.21.15.36:0/4273671349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27280b9860 msgr2=0x7f2728152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:54.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.834+0000 7f272c981700 1 --2- 172.21.15.36:0/4273671349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27280b9860 0x7f2728152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:54.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.834+0000 7f272c981700 1 -- 172.21.15.36:0/4273671349 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272400b180 con 0x7f27280b4f00 2022-06-17T12:52:54.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.834+0000 7f272c981700 1 --2- 172.21.15.36:0/4273671349 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f27280b4f00 0x7f2728152260 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f27280ba5d0 tx=0x7f2724012000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:54.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.834+0000 7f271e7fc700 1 -- 172.21.15.36:0/4273671349 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2724002f70 con 0x7f27280b4f00 2022-06-17T12:52:54.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.834+0000 7f271e7fc700 1 -- 172.21.15.36:0/4273671349 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f272400de70 con 0x7f27280b4f00 2022-06-17T12:52:54.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.834+0000 7f272e184700 1 -- 172.21.15.36:0/4273671349 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f272813e1b0 con 0x7f27280b4f00 2022-06-17T12:52:54.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.834+0000 7f271e7fc700 1 -- 172.21.15.36:0/4273671349 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2724013e90 con 0x7f27280b4f00 2022-06-17T12:52:54.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.834+0000 7f272e184700 1 -- 172.21.15.36:0/4273671349 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2728152c40 con 0x7f27280b4f00 2022-06-17T12:52:54.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.838+0000 7f272e184700 1 -- 172.21.15.36:0/4273671349 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2728152fa0 con 0x7f27280b4f00 2022-06-17T12:52:54.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.838+0000 7f271e7fc700 1 -- 172.21.15.36:0/4273671349 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f272401c070 con 0x7f27280b4f00 2022-06-17T12:52:54.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.838+0000 7f271e7fc700 1 --2- 172.21.15.36:0/4273671349 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2710053ab0 0x7f2710055f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:54.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.838+0000 7f271e7fc700 1 -- 172.21.15.36:0/4273671349 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f272406cc30 con 0x7f27280b4f00 2022-06-17T12:52:54.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.838+0000 7f272d182700 1 --2- 172.21.15.36:0/4273671349 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2710053ab0 0x7f2710055f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:54.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.838+0000 7f272d182700 1 --2- 172.21.15.36:0/4273671349 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2710053ab0 0x7f2710055f70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f271800b000 tx=0x7f2718009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:54.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.846+0000 7f271e7fc700 1 -- 172.21.15.36:0/4273671349 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2724073080 con 0x7f27280b4f00 2022-06-17T12:52:54.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:54.974+0000 7f272e184700 1 -- 172.21.15.36:0/4273671349 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2728057410 con 0x7f27280b4f00 2022-06-17T12:52:55.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.710+0000 7f271e7fc700 1 -- 172.21.15.36:0/4273671349 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v67) v1 ==== 185+0+0 (secure 0 0 0) 0x7f2724047410 con 0x7f27280b4f00 2022-06-17T12:52:55.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.714+0000 7f272e184700 1 -- 172.21.15.36:0/4273671349 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2710053ab0 msgr2=0x7f2710055f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:55.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.714+0000 7f272e184700 1 --2- 172.21.15.36:0/4273671349 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2710053ab0 0x7f2710055f70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f271800b000 tx=0x7f2718009040).stop 2022-06-17T12:52:55.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.714+0000 7f272e184700 1 -- 172.21.15.36:0/4273671349 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f27280b4f00 msgr2=0x7f2728152260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:55.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.714+0000 7f272e184700 1 --2- 172.21.15.36:0/4273671349 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f27280b4f00 0x7f2728152260 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f27280ba5d0 tx=0x7f2724012000).stop 2022-06-17T12:52:55.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.714+0000 7f272e184700 1 -- 172.21.15.36:0/4273671349 shutdown_connections 2022-06-17T12:52:55.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.714+0000 7f272e184700 1 --2- 172.21.15.36:0/4273671349 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2710053ab0 0x7f2710055f70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:55.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.714+0000 7f272e184700 1 --2- 172.21.15.36:0/4273671349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27280b9860 0x7f2728152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:55.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.714+0000 7f272e184700 1 --2- 172.21.15.36:0/4273671349 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f27280b4f00 0x7f2728152260 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:55.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.714+0000 7f272e184700 1 --2- 172.21.15.36:0/4273671349 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f27280b45c0 0x7f272813d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:55.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.714+0000 7f272e184700 1 -- 172.21.15.36:0/4273671349 >> 172.21.15.36:0/4273671349 conn(0x7f272805c550 msgr2=0x7f27280b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:55.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.718+0000 7f272e184700 1 -- 172.21.15.36:0/4273671349 shutdown_connections 2022-06-17T12:52:55.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.718+0000 7f272e184700 1 -- 172.21.15.36:0/4273671349 wait complete. 2022-06-17T12:52:55.704 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' removed 2022-06-17T12:52:55.715 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-17T12:52:55.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.854+0000 7f4d62827700 1 Processor -- start 2022-06-17T12:52:55.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.854+0000 7f4d62827700 1 -- start start 2022-06-17T12:52:55.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d62827700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d5c0b45c0 0x7f4d5c0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:55.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d62827700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4d5c0b4f00 0x7f4d5c0b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:55.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d62827700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d5c0b4160 con 0x7f4d5c0b9860 2022-06-17T12:52:55.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d62827700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d5c0b9c40 con 0x7f4d5c0b4f00 2022-06-17T12:52:55.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d62827700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c0627d0 con 0x7f4d5c0b45c0 2022-06-17T12:52:55.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d62026700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4d5c0b4f00 0x7f4d5c0b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:55.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d62026700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4d5c0b4f00 0x7f4d5c0b5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35248/0 (socket says 172.21.15.36:35248) 2022-06-17T12:52:55.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d62026700 1 -- 172.21.15.36:0/1224090472 learned_addr learned my addr 172.21.15.36:0/1224090472 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:55.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d61024700 1 --2- 172.21.15.36:0/1224090472 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d5c0b45c0 0x7f4d5c0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:55.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d62026700 1 -- 172.21.15.36:0/1224090472 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d5c0b45c0 msgr2=0x7f4d5c0b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:55.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d62026700 1 --2- 172.21.15.36:0/1224090472 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d5c0b45c0 0x7f4d5c0b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:55.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d62026700 1 -- 172.21.15.36:0/1224090472 >> v1:172.21.15.70:6789/0 conn(0x7f4d5c0b9860 legacy=0x7f4d5c143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:55.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d62026700 1 -- 172.21.15.36:0/1224090472 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5c062a70 con 0x7f4d5c0b4f00 2022-06-17T12:52:55.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d62026700 1 --2- 172.21.15.36:0/1224090472 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d5c0b4f00 0x7f4d5c0b5dc0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7f4d540032f0 tx=0x7f4d54009290).ready entity=mon.0 client_cookie=181c1c86dab9ae68 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:55.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d60823700 1 -- 172.21.15.36:0/1224090472 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d5400b8a0 con 0x7f4d5c0b4f00 2022-06-17T12:52:55.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d60823700 1 -- 172.21.15.36:0/1224090472 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5400ba00 con 0x7f4d5c0b4f00 2022-06-17T12:52:55.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d60823700 1 -- 172.21.15.36:0/1224090472 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d5400bb70 con 0x7f4d5c0b4f00 2022-06-17T12:52:55.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d62827700 1 -- 172.21.15.36:0/1224090472 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d5c0b4f00 msgr2=0x7f4d5c0b5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:55.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d62827700 1 --2- 172.21.15.36:0/1224090472 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d5c0b4f00 0x7f4d5c0b5dc0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7f4d540032f0 tx=0x7f4d54009290).stop 2022-06-17T12:52:55.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d62827700 1 -- 172.21.15.36:0/1224090472 shutdown_connections 2022-06-17T12:52:55.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d62827700 1 --2- 172.21.15.36:0/1224090472 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d5c0b45c0 0x7f4d5c0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:55.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d62827700 1 --2- 172.21.15.36:0/1224090472 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d5c0b4f00 0x7f4d5c0b5dc0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:55.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d62827700 1 -- 172.21.15.36:0/1224090472 >> 172.21.15.36:0/1224090472 conn(0x7f4d5c05c550 msgr2=0x7f4d5c060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:55.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d62827700 1 -- 172.21.15.36:0/1224090472 shutdown_connections 2022-06-17T12:52:55.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.858+0000 7f4d62827700 1 -- 172.21.15.36:0/1224090472 wait complete. 2022-06-17T12:52:55.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.862+0000 7f4d62827700 1 Processor -- start 2022-06-17T12:52:55.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.862+0000 7f4d62827700 1 -- start start 2022-06-17T12:52:55.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.862+0000 7f4d62827700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d5c0b45c0 0x7f4d5c0aff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:55.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.862+0000 7f4d62827700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d5c0b4f00 0x7f4d5c152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:55.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.862+0000 7f4d62827700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d5c0b9860 0x7f4d5c152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:55.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.862+0000 7f4d62827700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c1464e0 con 0x7f4d5c0b4f00 2022-06-17T12:52:55.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.862+0000 7f4d62827700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c1422e0 con 0x7f4d5c0b45c0 2022-06-17T12:52:55.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.862+0000 7f4d62827700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c1563f0 con 0x7f4d5c0b9860 2022-06-17T12:52:55.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.862+0000 7f4d61024700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d5c0b4f00 0x7f4d5c152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:55.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.862+0000 7f4d61024700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d5c0b4f00 0x7f4d5c152290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35254/0 (socket says 172.21.15.36:35254) 2022-06-17T12:52:55.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.862+0000 7f4d61825700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d5c0b45c0 0x7f4d5c0aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:55.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.862+0000 7f4d61024700 1 -- 172.21.15.36:0/1960557633 learned_addr learned my addr 172.21.15.36:0/1960557633 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:55.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.862+0000 7f4d62026700 1 --2- 172.21.15.36:0/1960557633 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d5c0b9860 0x7f4d5c152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:55.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.862+0000 7f4d61024700 1 -- 172.21.15.36:0/1960557633 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d5c0b9860 msgr2=0x7f4d5c152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:55.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.866+0000 7f4d61024700 1 --2- 172.21.15.36:0/1960557633 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d5c0b9860 0x7f4d5c152780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:55.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.866+0000 7f4d61024700 1 -- 172.21.15.36:0/1960557633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d5c0b45c0 msgr2=0x7f4d5c0aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:55.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.866+0000 7f4d61024700 1 --2- 172.21.15.36:0/1960557633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d5c0b45c0 0x7f4d5c0aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:55.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.866+0000 7f4d61024700 1 -- 172.21.15.36:0/1960557633 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d54009530 con 0x7f4d5c0b4f00 2022-06-17T12:52:55.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.866+0000 7f4d61825700 1 --2- 172.21.15.36:0/1960557633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d5c0b45c0 0x7f4d5c0aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:52:55.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.866+0000 7f4d61024700 1 --2- 172.21.15.36:0/1960557633 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d5c0b4f00 0x7f4d5c152290 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f4d5c0afae0 tx=0x7f4d4c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:55.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.866+0000 7f4d4affd700 1 -- 172.21.15.36:0/1960557633 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d4c00dea0 con 0x7f4d5c0b4f00 2022-06-17T12:52:55.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.866+0000 7f4d4affd700 1 -- 172.21.15.36:0/1960557633 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4c011780 con 0x7f4d5c0b4f00 2022-06-17T12:52:55.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.866+0000 7f4d62827700 1 -- 172.21.15.36:0/1960557633 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d5c156690 con 0x7f4d5c0b4f00 2022-06-17T12:52:55.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.866+0000 7f4d4affd700 1 -- 172.21.15.36:0/1960557633 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d4c011ae0 con 0x7f4d5c0b4f00 2022-06-17T12:52:55.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.866+0000 7f4d62827700 1 -- 172.21.15.36:0/1960557633 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d5c1470d0 con 0x7f4d5c0b4f00 2022-06-17T12:52:55.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.866+0000 7f4d62827700 1 -- 172.21.15.36:0/1960557633 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d5c156230 con 0x7f4d5c0b4f00 2022-06-17T12:52:55.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.866+0000 7f4d4affd700 1 -- 172.21.15.36:0/1960557633 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f4d4c011c40 con 0x7f4d5c0b4f00 2022-06-17T12:52:55.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.870+0000 7f4d4affd700 1 --2- 172.21.15.36:0/1960557633 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d3c053ad0 0x7f4d3c055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:55.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.870+0000 7f4d4affd700 1 -- 172.21.15.36:0/1960557633 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f4d4c06bf80 con 0x7f4d5c0b4f00 2022-06-17T12:52:55.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.870+0000 7f4d61825700 1 --2- 172.21.15.36:0/1960557633 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d3c053ad0 0x7f4d3c055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:55.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.870+0000 7f4d61825700 1 --2- 172.21.15.36:0/1960557633 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d3c053ad0 0x7f4d3c055f90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f4d5800d000 tx=0x7f4d58009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:55.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:55.878+0000 7f4d4affd700 1 -- 172.21.15.36:0/1960557633 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4d4c042ee0 con 0x7f4d5c0b4f00 2022-06-17T12:52:55.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.006+0000 7f4d62827700 1 -- 172.21.15.36:0/1960557633 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4d5c142420 con 0x7f4d5c0b4f00 2022-06-17T12:52:56.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.718+0000 7f4d4affd700 1 -- 172.21.15.36:0/1960557633 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v68) v1 ==== 173+0+0 (secure 0 0 0) 0x7f4d4c046500 con 0x7f4d5c0b4f00 2022-06-17T12:52:56.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.722+0000 7f4d62827700 1 -- 172.21.15.36:0/1960557633 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d3c053ad0 msgr2=0x7f4d3c055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:56.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.722+0000 7f4d62827700 1 --2- 172.21.15.36:0/1960557633 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d3c053ad0 0x7f4d3c055f90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f4d5800d000 tx=0x7f4d58009040).stop 2022-06-17T12:52:56.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.722+0000 7f4d62827700 1 -- 172.21.15.36:0/1960557633 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d5c0b4f00 msgr2=0x7f4d5c152290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:56.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.722+0000 7f4d62827700 1 --2- 172.21.15.36:0/1960557633 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d5c0b4f00 0x7f4d5c152290 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f4d5c0afae0 tx=0x7f4d4c008040).stop 2022-06-17T12:52:56.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.722+0000 7f4d62827700 1 -- 172.21.15.36:0/1960557633 shutdown_connections 2022-06-17T12:52:56.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.722+0000 7f4d62827700 1 --2- 172.21.15.36:0/1960557633 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d3c053ad0 0x7f4d3c055f90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:56.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.722+0000 7f4d62827700 1 --2- 172.21.15.36:0/1960557633 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d5c0b9860 0x7f4d5c152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:56.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.722+0000 7f4d62827700 1 --2- 172.21.15.36:0/1960557633 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d5c0b4f00 0x7f4d5c152290 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:56.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.722+0000 7f4d62827700 1 --2- 172.21.15.36:0/1960557633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d5c0b45c0 0x7f4d5c0aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:56.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.722+0000 7f4d62827700 1 -- 172.21.15.36:0/1960557633 >> 172.21.15.36:0/1960557633 conn(0x7f4d5c05c550 msgr2=0x7f4d5c0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:56.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.722+0000 7f4d62827700 1 -- 172.21.15.36:0/1960557633 shutdown_connections 2022-06-17T12:52:56.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.722+0000 7f4d62827700 1 -- 172.21.15.36:0/1960557633 wait complete. 2022-06-17T12:52:56.715 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' removed 2022-06-17T12:52:56.728 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-06-17T12:52:56.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.846+0000 7f34d6140700 1 Processor -- start 2022-06-17T12:52:56.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.846+0000 7f34d6140700 1 -- start start 2022-06-17T12:52:56.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d6140700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34d00b6800 0x7f34d0147ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:56.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d6140700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f34d00aeea0 0x7f34d014a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:56.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d6140700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f34d014acc0 0x7f34d01463a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:56.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d6140700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d00af360 con 0x7f34d014acc0 2022-06-17T12:52:56.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d6140700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d00b6be0 con 0x7f34d00aeea0 2022-06-17T12:52:56.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d6140700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d01473a0 con 0x7f34d00b6800 2022-06-17T12:52:56.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d593f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f34d014acc0 0x7f34d01463a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:56.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d593f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f34d014acc0 0x7f34d01463a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35262/0 (socket says 172.21.15.36:35262) 2022-06-17T12:52:56.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d593f700 1 -- 172.21.15.36:0/1033808763 learned_addr learned my addr 172.21.15.36:0/1033808763 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:56.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d513e700 1 --2- 172.21.15.36:0/1033808763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34d00b6800 0x7f34d0147ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:56.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d493d700 1 --2- 172.21.15.36:0/1033808763 >> v2:172.21.15.70:3300/0 conn(0x7f34d00aeea0 0x7f34d014a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:56.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d593f700 1 -- 172.21.15.36:0/1033808763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34d00b6800 msgr2=0x7f34d0147ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:56.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d593f700 1 --2- 172.21.15.36:0/1033808763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34d00b6800 0x7f34d0147ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:56.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d593f700 1 -- 172.21.15.36:0/1033808763 >> v2:172.21.15.70:3300/0 conn(0x7f34d00aeea0 msgr2=0x7f34d014a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:56.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d593f700 1 --2- 172.21.15.36:0/1033808763 >> v2:172.21.15.70:3300/0 conn(0x7f34d00aeea0 0x7f34d014a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:56.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d593f700 1 -- 172.21.15.36:0/1033808763 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d0147640 con 0x7f34d014acc0 2022-06-17T12:52:56.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d593f700 1 --2- 172.21.15.36:0/1033808763 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f34d014acc0 0x7f34d01463a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f34c80032f0 tx=0x7f34c8009290).ready entity=mon.0 client_cookie=e8a94d24636dbad7 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:56.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34cffff700 1 -- 172.21.15.36:0/1033808763 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34c800b680 con 0x7f34d014acc0 2022-06-17T12:52:56.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34cffff700 1 -- 172.21.15.36:0/1033808763 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c800b7e0 con 0x7f34d014acc0 2022-06-17T12:52:56.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34cffff700 1 -- 172.21.15.36:0/1033808763 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34c800b950 con 0x7f34d014acc0 2022-06-17T12:52:56.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d6140700 1 -- 172.21.15.36:0/1033808763 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f34d014acc0 msgr2=0x7f34d01463a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:56.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d6140700 1 --2- 172.21.15.36:0/1033808763 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f34d014acc0 0x7f34d01463a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f34c80032f0 tx=0x7f34c8009290).stop 2022-06-17T12:52:56.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d6140700 1 -- 172.21.15.36:0/1033808763 shutdown_connections 2022-06-17T12:52:56.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d6140700 1 --2- 172.21.15.36:0/1033808763 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f34d014acc0 0x7f34d01463a0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:56.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d6140700 1 --2- 172.21.15.36:0/1033808763 >> v2:172.21.15.70:3300/0 conn(0x7f34d00aeea0 0x7f34d014a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:56.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d6140700 1 --2- 172.21.15.36:0/1033808763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34d00b6800 0x7f34d0147ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:56.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d6140700 1 -- 172.21.15.36:0/1033808763 >> 172.21.15.36:0/1033808763 conn(0x7f34d00614b0 msgr2=0x7f34d0061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:56.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d6140700 1 -- 172.21.15.36:0/1033808763 shutdown_connections 2022-06-17T12:52:56.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.850+0000 7f34d6140700 1 -- 172.21.15.36:0/1033808763 wait complete. 2022-06-17T12:52:56.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.854+0000 7f34d6140700 1 Processor -- start 2022-06-17T12:52:56.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.854+0000 7f34d6140700 1 -- start start 2022-06-17T12:52:56.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.854+0000 7f34d6140700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34d00aeea0 0x7f34d0146160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:56.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.854+0000 7f34d6140700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f34d00b6800 0x7f34d0143ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:56.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.854+0000 7f34d6140700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34d014acc0 0x7f34d01443b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:56.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.854+0000 7f34d6140700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d0066370 con 0x7f34d00b6800 2022-06-17T12:52:56.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.854+0000 7f34d6140700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d01130c0 con 0x7f34d00aeea0 2022-06-17T12:52:56.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.854+0000 7f34d6140700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d0112e20 con 0x7f34d014acc0 2022-06-17T12:52:56.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.854+0000 7f34d493d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f34d00b6800 0x7f34d0143ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:56.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.854+0000 7f34d493d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f34d00b6800 0x7f34d0143ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35270/0 (socket says 172.21.15.36:35270) 2022-06-17T12:52:56.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.854+0000 7f34d513e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34d00aeea0 0x7f34d0146160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:56.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.854+0000 7f34d593f700 1 --2- 172.21.15.36:0/3648700191 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34d014acc0 0x7f34d01443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:56.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.854+0000 7f34d493d700 1 -- 172.21.15.36:0/3648700191 learned_addr learned my addr 172.21.15.36:0/3648700191 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:56.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.854+0000 7f34d493d700 1 -- 172.21.15.36:0/3648700191 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34d014acc0 msgr2=0x7f34d01443b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:56.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.854+0000 7f34d493d700 1 --2- 172.21.15.36:0/3648700191 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34d014acc0 0x7f34d01443b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:56.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.854+0000 7f34d493d700 1 -- 172.21.15.36:0/3648700191 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34d00aeea0 msgr2=0x7f34d0146160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:56.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.854+0000 7f34d493d700 1 --2- 172.21.15.36:0/3648700191 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34d00aeea0 0x7f34d0146160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:56.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.854+0000 7f34d493d700 1 -- 172.21.15.36:0/3648700191 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c8009530 con 0x7f34d00b6800 2022-06-17T12:52:56.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.854+0000 7f34d493d700 1 --2- 172.21.15.36:0/3648700191 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f34d00b6800 0x7f34d0143ec0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f34c400c000 tx=0x7f34c4006d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:56.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.854+0000 7f34ce7fc700 1 -- 172.21.15.36:0/3648700191 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34c40104c0 con 0x7f34d00b6800 2022-06-17T12:52:56.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.854+0000 7f34ce7fc700 1 -- 172.21.15.36:0/3648700191 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c4010620 con 0x7f34d00b6800 2022-06-17T12:52:56.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.854+0000 7f34d6140700 1 -- 172.21.15.36:0/3648700191 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d00665d0 con 0x7f34d00b6800 2022-06-17T12:52:56.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.854+0000 7f34ce7fc700 1 -- 172.21.15.36:0/3648700191 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34c4010980 con 0x7f34d00b6800 2022-06-17T12:52:56.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.854+0000 7f34d6140700 1 -- 172.21.15.36:0/3648700191 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d01448a0 con 0x7f34d00b6800 2022-06-17T12:52:56.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.858+0000 7f34ce7fc700 1 -- 172.21.15.36:0/3648700191 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f34c401a030 con 0x7f34d00b6800 2022-06-17T12:52:56.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.858+0000 7f34d6140700 1 -- 172.21.15.36:0/3648700191 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d01129d0 con 0x7f34d00b6800 2022-06-17T12:52:56.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.858+0000 7f34ce7fc700 1 --2- 172.21.15.36:0/3648700191 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f34b00539d0 0x7f34b0055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:56.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.858+0000 7f34ce7fc700 1 -- 172.21.15.36:0/3648700191 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f34c406b680 con 0x7f34d00b6800 2022-06-17T12:52:56.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.858+0000 7f34d513e700 1 --2- 172.21.15.36:0/3648700191 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f34b00539d0 0x7f34b0055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:56.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.858+0000 7f34d513e700 1 --2- 172.21.15.36:0/3648700191 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f34b00539d0 0x7f34b0055e90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f34c000b000 tx=0x7f34c0006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:56.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.862+0000 7f34ce7fc700 1 -- 172.21.15.36:0/3648700191 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f34c40426a0 con 0x7f34d00b6800 2022-06-17T12:52:56.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.990+0000 7f34d6140700 1 -- 172.21.15.36:0/3648700191 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f34d0066110 con 0x7f34d00b6800 2022-06-17T12:52:56.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.998+0000 7f34ce7fc700 1 -- 172.21.15.36:0/3648700191 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f34c4045cc0 con 0x7f34d00b6800 2022-06-17T12:52:56.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.998+0000 7f34d6140700 1 -- 172.21.15.36:0/3648700191 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f34b00539d0 msgr2=0x7f34b0055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:56.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.998+0000 7f34d6140700 1 --2- 172.21.15.36:0/3648700191 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f34b00539d0 0x7f34b0055e90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f34c000b000 tx=0x7f34c0006df0).stop 2022-06-17T12:52:56.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.998+0000 7f34d6140700 1 -- 172.21.15.36:0/3648700191 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f34d00b6800 msgr2=0x7f34d0143ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:56.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.998+0000 7f34d6140700 1 --2- 172.21.15.36:0/3648700191 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f34d00b6800 0x7f34d0143ec0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f34c400c000 tx=0x7f34c4006d90).stop 2022-06-17T12:52:56.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.998+0000 7f34d6140700 1 -- 172.21.15.36:0/3648700191 shutdown_connections 2022-06-17T12:52:56.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.998+0000 7f34d6140700 1 --2- 172.21.15.36:0/3648700191 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f34b00539d0 0x7f34b0055e90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:56.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.998+0000 7f34d6140700 1 --2- 172.21.15.36:0/3648700191 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34d014acc0 0x7f34d01443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:56.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.998+0000 7f34d6140700 1 --2- 172.21.15.36:0/3648700191 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f34d00b6800 0x7f34d0143ec0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:56.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.998+0000 7f34d6140700 1 --2- 172.21.15.36:0/3648700191 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34d00aeea0 0x7f34d0146160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:56.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:56.998+0000 7f34d6140700 1 -- 172.21.15.36:0/3648700191 >> 172.21.15.36:0/3648700191 conn(0x7f34d00614b0 msgr2=0x7f34d005bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:56.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.002+0000 7f34d6140700 1 -- 172.21.15.36:0/3648700191 shutdown_connections 2022-06-17T12:52:56.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.002+0000 7f34d6140700 1 -- 172.21.15.36:0/3648700191 wait complete. 2022-06-17T12:52:56.998 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout' 2022-06-17T12:52:57.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.126+0000 7fe8c9940700 1 Processor -- start 2022-06-17T12:52:57.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.126+0000 7fe8c9940700 1 -- start start 2022-06-17T12:52:57.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.126+0000 7fe8c9940700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe8c40b9860 0x7fe8c4143840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:57.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.126+0000 7fe8c9940700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe8c40b45c0 0x7fe8c40b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:57.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.126+0000 7fe8c9940700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c40b4160 con 0x7fe8c40b4f00 2022-06-17T12:52:57.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.126+0000 7fe8c9940700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c40b9c40 con 0x7fe8c40b45c0 2022-06-17T12:52:57.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.126+0000 7fe8c9940700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c40627d0 con 0x7fe8c40b9860 2022-06-17T12:52:57.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.126+0000 7fe8c3fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe8c40b45c0 0x7fe8c40b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:57.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.126+0000 7fe8c3fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe8c40b45c0 0x7fe8c40b4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35274/0 (socket says 172.21.15.36:35274) 2022-06-17T12:52:57.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.126+0000 7fe8c3fff700 1 -- 172.21.15.36:0/369586852 learned_addr learned my addr 172.21.15.36:0/369586852 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:57.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.126+0000 7fe8c893e700 1 --2- 172.21.15.36:0/369586852 >> v2:172.21.15.70:3300/0 conn(0x7fe8c40b9860 0x7fe8c4143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:57.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.126+0000 7fe8c3fff700 1 -- 172.21.15.36:0/369586852 >> v2:172.21.15.70:3300/0 conn(0x7fe8c40b9860 msgr2=0x7fe8c4143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:57.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.126+0000 7fe8c3fff700 1 --2- 172.21.15.36:0/369586852 >> v2:172.21.15.70:3300/0 conn(0x7fe8c40b9860 0x7fe8c4143840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:57.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.126+0000 7fe8c3fff700 1 -- 172.21.15.36:0/369586852 >> v1:172.21.15.70:6789/0 conn(0x7fe8c40b4f00 legacy=0x7fe8c4146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:57.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.126+0000 7fe8c3fff700 1 -- 172.21.15.36:0/369586852 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c4062a70 con 0x7fe8c40b45c0 2022-06-17T12:52:57.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.126+0000 7fe8c3fff700 1 --2- 172.21.15.36:0/369586852 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8c40b45c0 0x7fe8c40b4a10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7fe8b4002f70 tx=0x7fe8b4006eb0).ready entity=mon.0 client_cookie=57ac454c7f438e24 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:57.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.126+0000 7fe8c37fe700 1 -- 172.21.15.36:0/369586852 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8b400d5c0 con 0x7fe8c40b45c0 2022-06-17T12:52:57.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.126+0000 7fe8c37fe700 1 -- 172.21.15.36:0/369586852 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b4000e90 con 0x7fe8c40b45c0 2022-06-17T12:52:57.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.126+0000 7fe8c37fe700 1 -- 172.21.15.36:0/369586852 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8b400d950 con 0x7fe8c40b45c0 2022-06-17T12:52:57.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.126+0000 7fe8c9940700 1 -- 172.21.15.36:0/369586852 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8c40b45c0 msgr2=0x7fe8c40b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:57.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.126+0000 7fe8c9940700 1 --2- 172.21.15.36:0/369586852 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8c40b45c0 0x7fe8c40b4a10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7fe8b4002f70 tx=0x7fe8b4006eb0).stop 2022-06-17T12:52:57.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.130+0000 7fe8c9940700 1 -- 172.21.15.36:0/369586852 shutdown_connections 2022-06-17T12:52:57.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.130+0000 7fe8c9940700 1 --2- 172.21.15.36:0/369586852 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8c40b45c0 0x7fe8c40b4a10 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:57.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.130+0000 7fe8c9940700 1 --2- 172.21.15.36:0/369586852 >> v2:172.21.15.70:3300/0 conn(0x7fe8c40b9860 0x7fe8c4143840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:57.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.130+0000 7fe8c9940700 1 -- 172.21.15.36:0/369586852 >> 172.21.15.36:0/369586852 conn(0x7fe8c405c550 msgr2=0x7fe8c4060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:57.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.130+0000 7fe8c9940700 1 -- 172.21.15.36:0/369586852 shutdown_connections 2022-06-17T12:52:57.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.130+0000 7fe8c9940700 1 -- 172.21.15.36:0/369586852 wait complete. 2022-06-17T12:52:57.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.130+0000 7fe8c9940700 1 Processor -- start 2022-06-17T12:52:57.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.130+0000 7fe8c9940700 1 -- start start 2022-06-17T12:52:57.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.134+0000 7fe8c9940700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8c40b45c0 0x7fe8c4152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:57.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.134+0000 7fe8c9940700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8c40b4f00 0x7fe8c413e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:57.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.134+0000 7fe8c9940700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8c40b9860 0x7fe8c40af990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:57.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.134+0000 7fe8c9940700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c413edc0 con 0x7fe8c40b9860 2022-06-17T12:52:57.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.134+0000 7fe8c9940700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c40adc50 con 0x7fe8c40b45c0 2022-06-17T12:52:57.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.134+0000 7fe8c9940700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c413f000 con 0x7fe8c40b4f00 2022-06-17T12:52:57.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.134+0000 7fe8c913f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8c40b9860 0x7fe8c40af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:57.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.134+0000 7fe8c913f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8c40b9860 0x7fe8c40af990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35284/0 (socket says 172.21.15.36:35284) 2022-06-17T12:52:57.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.134+0000 7fe8c913f700 1 -- 172.21.15.36:0/1779093221 learned_addr learned my addr 172.21.15.36:0/1779093221 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:57.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.134+0000 7fe8c893e700 1 --2- 172.21.15.36:0/1779093221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8c40b45c0 0x7fe8c4152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:57.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.134+0000 7fe8c3fff700 1 --2- 172.21.15.36:0/1779093221 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8c40b4f00 0x7fe8c413e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:57.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.134+0000 7fe8c913f700 1 -- 172.21.15.36:0/1779093221 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8c40b4f00 msgr2=0x7fe8c413e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:57.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.134+0000 7fe8c913f700 1 --2- 172.21.15.36:0/1779093221 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8c40b4f00 0x7fe8c413e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:57.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.134+0000 7fe8c913f700 1 -- 172.21.15.36:0/1779093221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8c40b45c0 msgr2=0x7fe8c4152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:57.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.134+0000 7fe8c913f700 1 --2- 172.21.15.36:0/1779093221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8c40b45c0 0x7fe8c4152bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:57.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.134+0000 7fe8c913f700 1 -- 172.21.15.36:0/1779093221 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b400b1e0 con 0x7fe8c40b9860 2022-06-17T12:52:57.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.134+0000 7fe8c913f700 1 --2- 172.21.15.36:0/1779093221 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8c40b9860 0x7fe8c40af990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7fe8c40b0fd0 tx=0x7fe8bc007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:57.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.134+0000 7fe8c1ffb700 1 -- 172.21.15.36:0/1779093221 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8bc002490 con 0x7fe8c40b9860 2022-06-17T12:52:57.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.134+0000 7fe8c1ffb700 1 -- 172.21.15.36:0/1779093221 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8bc0025f0 con 0x7fe8c40b9860 2022-06-17T12:52:57.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.134+0000 7fe8c9940700 1 -- 172.21.15.36:0/1779093221 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c413f2a0 con 0x7fe8c40b9860 2022-06-17T12:52:57.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.134+0000 7fe8c1ffb700 1 -- 172.21.15.36:0/1779093221 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8bc002950 con 0x7fe8c40b9860 2022-06-17T12:52:57.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.134+0000 7fe8c9940700 1 -- 172.21.15.36:0/1779093221 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c413cee0 con 0x7fe8c40b9860 2022-06-17T12:52:57.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.134+0000 7fe8c1ffb700 1 -- 172.21.15.36:0/1779093221 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe8bc010030 con 0x7fe8c40b9860 2022-06-17T12:52:57.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.134+0000 7fe8c9940700 1 -- 172.21.15.36:0/1779093221 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c4062ee0 con 0x7fe8c40b9860 2022-06-17T12:52:57.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.138+0000 7fe8c1ffb700 1 --2- 172.21.15.36:0/1779093221 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe89c053ad0 0x7fe89c055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:57.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.138+0000 7fe8c1ffb700 1 -- 172.21.15.36:0/1779093221 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fe8bc06be90 con 0x7fe8c40b9860 2022-06-17T12:52:57.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.138+0000 7fe8c893e700 1 --2- 172.21.15.36:0/1779093221 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe89c053ad0 0x7fe89c055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:57.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.138+0000 7fe8c893e700 1 --2- 172.21.15.36:0/1779093221 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe89c053ad0 0x7fe89c055f90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fe8ac00b000 tx=0x7fe8ac006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:57.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.142+0000 7fe8c1ffb700 1 -- 172.21.15.36:0/1779093221 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe8bc042ed0 con 0x7fe8c40b9860 2022-06-17T12:52:57.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.310+0000 7fe8c9940700 1 -- 172.21.15.36:0/1779093221 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]} v 0) v1 -- 0x7fe8c4057410 con 0x7fe8c40b9860 2022-06-17T12:52:57.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.726+0000 7fe8c1ffb700 1 -- 172.21.15.36:0/1779093221 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]}]=0 v274) v1 ==== 147+0+0 (secure 0 0 0) 0x7fe8bc0464f0 con 0x7fe8c40b9860 2022-06-17T12:52:57.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.726+0000 7fe8c9940700 1 -- 172.21.15.36:0/1779093221 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe89c053ad0 msgr2=0x7fe89c055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:57.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.726+0000 7fe8c9940700 1 --2- 172.21.15.36:0/1779093221 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe89c053ad0 0x7fe89c055f90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fe8ac00b000 tx=0x7fe8ac006df0).stop 2022-06-17T12:52:57.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.726+0000 7fe8c9940700 1 -- 172.21.15.36:0/1779093221 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8c40b9860 msgr2=0x7fe8c40af990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:57.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.726+0000 7fe8c9940700 1 --2- 172.21.15.36:0/1779093221 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8c40b9860 0x7fe8c40af990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7fe8c40b0fd0 tx=0x7fe8bc007f10).stop 2022-06-17T12:52:57.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.726+0000 7fe8c9940700 1 -- 172.21.15.36:0/1779093221 shutdown_connections 2022-06-17T12:52:57.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.726+0000 7fe8c9940700 1 --2- 172.21.15.36:0/1779093221 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe89c053ad0 0x7fe89c055f90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:57.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.726+0000 7fe8c9940700 1 --2- 172.21.15.36:0/1779093221 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8c40b9860 0x7fe8c40af990 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:57.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.726+0000 7fe8c9940700 1 --2- 172.21.15.36:0/1779093221 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8c40b4f00 0x7fe8c413e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:57.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.726+0000 7fe8c9940700 1 --2- 172.21.15.36:0/1779093221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8c40b45c0 0x7fe8c4152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:57.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.726+0000 7fe8c9940700 1 -- 172.21.15.36:0/1779093221 >> 172.21.15.36:0/1779093221 conn(0x7fe8c405c550 msgr2=0x7fe8c4146660 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:57.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.730+0000 7fe8c9940700 1 -- 172.21.15.36:0/1779093221 shutdown_connections 2022-06-17T12:52:57.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.730+0000 7fe8c9940700 1 -- 172.21.15.36:0/1779093221 wait complete. 2022-06-17T12:52:57.729 INFO:tasks.cephfs_test_runner:test_drop_cache_command_timeout (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2022-06-17T12:52:57.730 INFO:tasks.cephfs_test_runner:Starting test: test_cap_revoke_nonresponder (tasks.cephfs.test_misc.TestMisc) 2022-06-17T12:52:57.731 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder' 2022-06-17T12:52:57.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f9f72700 1 Processor -- start 2022-06-17T12:52:57.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f9f72700 1 -- start start 2022-06-17T12:52:57.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f9f72700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f82f4060ca0 0x7f82f41442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:57.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f9f72700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f414b870 con 0x7f82f414b2b0 2022-06-17T12:52:57.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f9f72700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f40ac500 con 0x7f82f40b27b0 2022-06-17T12:52:57.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f9f72700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f40ace90 con 0x7f82f4060ca0 2022-06-17T12:52:57.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f9771700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f82f4060ca0 0x7f82f41442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:57.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f3fff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f82f414b2b0 0x7f82f4147bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52818/0 (socket says 172.21.15.36:52818) 2022-06-17T12:52:57.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f3fff700 1 -- 172.21.15.36:0/785223132 learned_addr learned my addr 172.21.15.36:0/785223132 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:57.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f9771700 1 -- 172.21.15.36:0/785223132 >> v1:172.21.15.70:6789/0 conn(0x7f82f40b27b0 legacy=0x7f82f40ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:57.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f9771700 1 -- 172.21.15.36:0/785223132 >> v1:172.21.15.36:6789/0 conn(0x7f82f414b2b0 legacy=0x7f82f4147bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:57.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f9771700 1 -- 172.21.15.36:0/785223132 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f41482e0 con 0x7f82f4060ca0 2022-06-17T12:52:57.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f9771700 1 --2- 172.21.15.36:0/785223132 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f82f4060ca0 0x7f82f41442d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f82ec0032f0 tx=0x7f82ec0091d0).ready entity=mon.0 client_cookie=f907c5c6f677d21a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:57.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f37fe700 1 -- 172.21.15.36:0/785223132 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82ec009ea0 con 0x7f82f4060ca0 2022-06-17T12:52:57.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f37fe700 1 -- 172.21.15.36:0/785223132 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ec00b8a0 con 0x7f82f4060ca0 2022-06-17T12:52:57.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f37fe700 1 -- 172.21.15.36:0/785223132 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82ec00ba10 con 0x7f82f4060ca0 2022-06-17T12:52:57.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f9f72700 1 -- 172.21.15.36:0/785223132 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f82f4060ca0 msgr2=0x7f82f41442d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:57.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f9f72700 1 --2- 172.21.15.36:0/785223132 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f82f4060ca0 0x7f82f41442d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f82ec0032f0 tx=0x7f82ec0091d0).stop 2022-06-17T12:52:57.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f9f72700 1 -- 172.21.15.36:0/785223132 shutdown_connections 2022-06-17T12:52:57.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f9f72700 1 --2- 172.21.15.36:0/785223132 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f82f4060ca0 0x7f82f41442d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:57.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f9f72700 1 -- 172.21.15.36:0/785223132 >> 172.21.15.36:0/785223132 conn(0x7f82f40b5d10 msgr2=0x7f82f40b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:57.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f9f72700 1 -- 172.21.15.36:0/785223132 shutdown_connections 2022-06-17T12:52:57.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f9f72700 1 -- 172.21.15.36:0/785223132 wait complete. 2022-06-17T12:52:57.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f9f72700 1 Processor -- start 2022-06-17T12:52:57.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.854+0000 7f82f9f72700 1 -- start start 2022-06-17T12:52:57.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.858+0000 7f82f9f72700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f4060ca0 0x7f82f4145480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:57.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.858+0000 7f82f9f72700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f82f40b27b0 0x7f82f40a78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:57.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.858+0000 7f82f9f72700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82f414b2b0 0x7f82f4159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:57.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.858+0000 7f82f9f72700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f40abc30 con 0x7f82f40b27b0 2022-06-17T12:52:57.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.858+0000 7f82f9f72700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f40a7da0 con 0x7f82f4060ca0 2022-06-17T12:52:57.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.858+0000 7f82f9f72700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f40a8190 con 0x7f82f414b2b0 2022-06-17T12:52:57.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.858+0000 7f82f3fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f82f40b27b0 0x7f82f40a78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:57.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.858+0000 7f82f9771700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82f414b2b0 0x7f82f4159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:57.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.858+0000 7f82f3fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f82f40b27b0 0x7f82f40a78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35298/0 (socket says 172.21.15.36:35298) 2022-06-17T12:52:57.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.858+0000 7f82f8f70700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f4060ca0 0x7f82f4145480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:57.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.858+0000 7f82f3fff700 1 -- 172.21.15.36:0/4138912848 learned_addr learned my addr 172.21.15.36:0/4138912848 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:57.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.858+0000 7f82f3fff700 1 -- 172.21.15.36:0/4138912848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82f414b2b0 msgr2=0x7f82f4159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:57.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.858+0000 7f82f3fff700 1 --2- 172.21.15.36:0/4138912848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82f414b2b0 0x7f82f4159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:57.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.858+0000 7f82f3fff700 1 -- 172.21.15.36:0/4138912848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f4060ca0 msgr2=0x7f82f4145480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:57.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.858+0000 7f82f3fff700 1 --2- 172.21.15.36:0/4138912848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f4060ca0 0x7f82f4145480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:57.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.858+0000 7f82f3fff700 1 -- 172.21.15.36:0/4138912848 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ec009470 con 0x7f82f40b27b0 2022-06-17T12:52:57.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.858+0000 7f82f3fff700 1 --2- 172.21.15.36:0/4138912848 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f82f40b27b0 0x7f82f40a78b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f82f40b5190 tx=0x7f82e4007ca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:57.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.858+0000 7f82f1ffb700 1 -- 172.21.15.36:0/4138912848 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82e400ce80 con 0x7f82f40b27b0 2022-06-17T12:52:57.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.858+0000 7f82f1ffb700 1 -- 172.21.15.36:0/4138912848 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e40070d0 con 0x7f82f40b27b0 2022-06-17T12:52:57.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.858+0000 7f82f1ffb700 1 -- 172.21.15.36:0/4138912848 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82e4007430 con 0x7f82f40b27b0 2022-06-17T12:52:57.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.858+0000 7f82f9f72700 1 -- 172.21.15.36:0/4138912848 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f40a8ac0 con 0x7f82f40b27b0 2022-06-17T12:52:57.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.858+0000 7f82f9f72700 1 -- 172.21.15.36:0/4138912848 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f40a8d60 con 0x7f82f40b27b0 2022-06-17T12:52:57.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.862+0000 7f82f9f72700 1 -- 172.21.15.36:0/4138912848 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f4057410 con 0x7f82f40b27b0 2022-06-17T12:52:57.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.862+0000 7f82f1ffb700 1 -- 172.21.15.36:0/4138912848 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f82e4010070 con 0x7f82f40b27b0 2022-06-17T12:52:57.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.862+0000 7f82f1ffb700 1 --2- 172.21.15.36:0/4138912848 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f82d4053ad0 0x7f82d4055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:57.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.862+0000 7f82f1ffb700 1 -- 172.21.15.36:0/4138912848 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f82e406b210 con 0x7f82f40b27b0 2022-06-17T12:52:57.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.862+0000 7f82f8f70700 1 --2- 172.21.15.36:0/4138912848 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f82d4053ad0 0x7f82d4055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:57.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.862+0000 7f82f8f70700 1 --2- 172.21.15.36:0/4138912848 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f82d4053ad0 0x7f82d4055f90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f82e800d000 tx=0x7f82e8007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:57.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:57.866+0000 7f82f1ffb700 1 -- 172.21.15.36:0/4138912848 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f82e40422f0 con 0x7f82f40b27b0 2022-06-17T12:52:58.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.038+0000 7f82f9f72700 1 -- 172.21.15.36:0/4138912848 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]} v 0) v1 -- 0x7f82f40a83a0 con 0x7f82f40b27b0 2022-06-17T12:52:58.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.726+0000 7f82f1ffb700 1 -- 172.21.15.36:0/4138912848 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]}]=0 v275) v1 ==== 142+0+0 (secure 0 0 0) 0x7f82e4018020 con 0x7f82f40b27b0 2022-06-17T12:52:58.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.730+0000 7f82f9f72700 1 -- 172.21.15.36:0/4138912848 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f82d4053ad0 msgr2=0x7f82d4055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:58.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.730+0000 7f82f9f72700 1 --2- 172.21.15.36:0/4138912848 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f82d4053ad0 0x7f82d4055f90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f82e800d000 tx=0x7f82e8007b90).stop 2022-06-17T12:52:58.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.730+0000 7f82f9f72700 1 -- 172.21.15.36:0/4138912848 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f82f40b27b0 msgr2=0x7f82f40a78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:58.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.730+0000 7f82f9f72700 1 --2- 172.21.15.36:0/4138912848 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f82f40b27b0 0x7f82f40a78b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f82f40b5190 tx=0x7f82e4007ca0).stop 2022-06-17T12:52:58.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.730+0000 7f82f9f72700 1 -- 172.21.15.36:0/4138912848 shutdown_connections 2022-06-17T12:52:58.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.730+0000 7f82f9f72700 1 --2- 172.21.15.36:0/4138912848 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f82d4053ad0 0x7f82d4055f90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:58.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.730+0000 7f82f9f72700 1 --2- 172.21.15.36:0/4138912848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82f414b2b0 0x7f82f4159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:58.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.730+0000 7f82f9f72700 1 --2- 172.21.15.36:0/4138912848 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f82f40b27b0 0x7f82f40a78b0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:58.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.730+0000 7f82f9f72700 1 --2- 172.21.15.36:0/4138912848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f4060ca0 0x7f82f4145480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:58.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.730+0000 7f82f9f72700 1 -- 172.21.15.36:0/4138912848 >> 172.21.15.36:0/4138912848 conn(0x7f82f40b5d10 msgr2=0x7f82f4146820 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:58.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.730+0000 7f82f9f72700 1 -- 172.21.15.36:0/4138912848 shutdown_connections 2022-06-17T12:52:58.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.730+0000 7f82f9f72700 1 -- 172.21.15.36:0/4138912848 wait complete. 2022-06-17T12:52:58.734 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-06-17T12:52:58.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.866+0000 7f33a392e700 1 Processor -- start 2022-06-17T12:52:58.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.866+0000 7f33a392e700 1 -- start start 2022-06-17T12:52:58.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f33a392e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f339c0b7bc0 0x7f339c140e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:58.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f33a392e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f339c0ae150 0x7f339c0b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:58.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f33a392e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f339c0b7fa0 con 0x7f339c0b2fc0 2022-06-17T12:52:58.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f33a392e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f339c0b20f0 con 0x7f339c0ae150 2022-06-17T12:52:58.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f33a392e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339c0b2390 con 0x7f339c0b7bc0 2022-06-17T12:52:58.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f33a1ecb700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f339c0b2fc0 0x7f339c144450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52836/0 (socket says 172.21.15.36:52836) 2022-06-17T12:52:58.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f33a1ecb700 1 -- 172.21.15.36:0/2764277919 learned_addr learned my addr 172.21.15.36:0/2764277919 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:58.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f33a0ec9700 1 --2- 172.21.15.36:0/2764277919 >> v2:172.21.15.70:3300/0 conn(0x7f339c0ae150 0x7f339c0b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:58.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f33a16ca700 1 --2- 172.21.15.36:0/2764277919 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f339c0b7bc0 0x7f339c140e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:58.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f338bfff700 1 -- 172.21.15.36:0/2764277919 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 506783953 0 0) 0x7f339c0b7fa0 con 0x7f339c0b2fc0 2022-06-17T12:52:58.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f338bfff700 1 -- 172.21.15.36:0/2764277919 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3384003260 con 0x7f339c0b2fc0 2022-06-17T12:52:58.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f33a0ec9700 1 -- 172.21.15.36:0/2764277919 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f339c0b7bc0 msgr2=0x7f339c140e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:58.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f33a0ec9700 1 --2- 172.21.15.36:0/2764277919 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f339c0b7bc0 0x7f339c140e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:58.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f33a0ec9700 1 -- 172.21.15.36:0/2764277919 >> v1:172.21.15.36:6789/0 conn(0x7f339c0b2fc0 legacy=0x7f339c144450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:58.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f33a0ec9700 1 -- 172.21.15.36:0/2764277919 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339c0b2630 con 0x7f339c0ae150 2022-06-17T12:52:58.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f33a0ec9700 1 --2- 172.21.15.36:0/2764277919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f339c0ae150 0x7f339c0b2ad0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f338c002f70 tx=0x7f338c006df0).ready entity=mon.1 client_cookie=c23b386472fcd0a0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:58.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f338bfff700 1 -- 172.21.15.36:0/2764277919 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f338c00d4a0 con 0x7f339c0ae150 2022-06-17T12:52:58.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f338bfff700 1 -- 172.21.15.36:0/2764277919 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338c000e90 con 0x7f339c0ae150 2022-06-17T12:52:58.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f338bfff700 1 -- 172.21.15.36:0/2764277919 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f338c00d7f0 con 0x7f339c0ae150 2022-06-17T12:52:58.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f33a392e700 1 -- 172.21.15.36:0/2764277919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f339c0ae150 msgr2=0x7f339c0b2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:58.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f33a392e700 1 --2- 172.21.15.36:0/2764277919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f339c0ae150 0x7f339c0b2ad0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f338c002f70 tx=0x7f338c006df0).stop 2022-06-17T12:52:58.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f33a392e700 1 -- 172.21.15.36:0/2764277919 shutdown_connections 2022-06-17T12:52:58.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f33a392e700 1 --2- 172.21.15.36:0/2764277919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f339c0ae150 0x7f339c0b2ad0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:58.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f33a392e700 1 --2- 172.21.15.36:0/2764277919 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f339c0b7bc0 0x7f339c140e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:58.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f33a392e700 1 -- 172.21.15.36:0/2764277919 >> 172.21.15.36:0/2764277919 conn(0x7f339c060990 msgr2=0x7f339c05c550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:58.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f33a392e700 1 -- 172.21.15.36:0/2764277919 shutdown_connections 2022-06-17T12:52:58.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.870+0000 7f33a392e700 1 -- 172.21.15.36:0/2764277919 wait complete. 2022-06-17T12:52:58.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.874+0000 7f33a392e700 1 Processor -- start 2022-06-17T12:52:58.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.874+0000 7f33a392e700 1 -- start start 2022-06-17T12:52:58.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.874+0000 7f33a392e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f339c0ae150 0x7f339c062a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:58.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.874+0000 7f33a392e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f339c0b2fc0 0x7f339c152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:58.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.874+0000 7f33a392e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f339c0b7bc0 0x7f339c0b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:58.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.874+0000 7f33a392e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339c140270 con 0x7f339c0b2fc0 2022-06-17T12:52:58.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.874+0000 7f33a392e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339c13ee30 con 0x7f339c0ae150 2022-06-17T12:52:58.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.874+0000 7f33a392e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339c152820 con 0x7f339c0b7bc0 2022-06-17T12:52:58.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.874+0000 7f33a0ec9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f339c0b2fc0 0x7f339c152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:58.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.874+0000 7f33a16ca700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f339c0ae150 0x7f339c062a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:58.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.874+0000 7f33a16ca700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f339c0ae150 0x7f339c062a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:55690/0 (socket says 172.21.15.36:55690) 2022-06-17T12:52:58.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.874+0000 7f33a16ca700 1 -- 172.21.15.36:0/764063934 learned_addr learned my addr 172.21.15.36:0/764063934 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:58.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.874+0000 7f33a1ecb700 1 --2- 172.21.15.36:0/764063934 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f339c0b7bc0 0x7f339c0b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:58.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.878+0000 7f33a16ca700 1 -- 172.21.15.36:0/764063934 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f339c0b7bc0 msgr2=0x7f339c0b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:58.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.878+0000 7f33a16ca700 1 --2- 172.21.15.36:0/764063934 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f339c0b7bc0 0x7f339c0b5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:58.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.878+0000 7f33a16ca700 1 -- 172.21.15.36:0/764063934 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f339c0b2fc0 msgr2=0x7f339c152330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:58.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.878+0000 7f33a16ca700 1 --2- 172.21.15.36:0/764063934 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f339c0b2fc0 0x7f339c152330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:58.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.878+0000 7f33a16ca700 1 -- 172.21.15.36:0/764063934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338c00b180 con 0x7f339c0ae150 2022-06-17T12:52:58.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.878+0000 7f33a16ca700 1 --2- 172.21.15.36:0/764063934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f339c0ae150 0x7f339c062a80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f339c0b6a80 tx=0x7f3398006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:58.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.878+0000 7f338a7fc700 1 -- 172.21.15.36:0/764063934 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3398010780 con 0x7f339c0ae150 2022-06-17T12:52:58.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.878+0000 7f338a7fc700 1 -- 172.21.15.36:0/764063934 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33980108e0 con 0x7f339c0ae150 2022-06-17T12:52:58.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.878+0000 7f33a392e700 1 -- 172.21.15.36:0/764063934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f339c063350 con 0x7f339c0ae150 2022-06-17T12:52:58.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.878+0000 7f338a7fc700 1 -- 172.21.15.36:0/764063934 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3398010c40 con 0x7f339c0ae150 2022-06-17T12:52:58.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.878+0000 7f33a392e700 1 -- 172.21.15.36:0/764063934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f339c141080 con 0x7f339c0ae150 2022-06-17T12:52:58.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.878+0000 7f33a392e700 1 -- 172.21.15.36:0/764063934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f339c057410 con 0x7f339c0ae150 2022-06-17T12:52:58.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.878+0000 7f338a7fc700 1 -- 172.21.15.36:0/764063934 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f339801c460 con 0x7f339c0ae150 2022-06-17T12:52:58.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.878+0000 7f338a7fc700 1 --2- 172.21.15.36:0/764063934 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f337c053ad0 0x7f337c055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:58.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.878+0000 7f338a7fc700 1 -- 172.21.15.36:0/764063934 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f339806c060 con 0x7f339c0ae150 2022-06-17T12:52:58.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.878+0000 7f33a0ec9700 1 --2- 172.21.15.36:0/764063934 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f337c053ad0 0x7f337c055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:58.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.878+0000 7f33a0ec9700 1 --2- 172.21.15.36:0/764063934 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f337c053ad0 0x7f337c055f90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f338c01a000 tx=0x7f338c014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:58.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:58.890+0000 7f338a7fc700 1 -- 172.21.15.36:0/764063934 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f33980430e0 con 0x7f339c0ae150 2022-06-17T12:52:59.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.018+0000 7f33a392e700 1 -- 172.21.15.36:0/764063934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f339c0b72b0 con 0x7f339c0ae150 2022-06-17T12:52:59.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.026+0000 7f338a7fc700 1 -- 172.21.15.36:0/764063934 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3398046700 con 0x7f339c0ae150 2022-06-17T12:52:59.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.030+0000 7f33a392e700 1 -- 172.21.15.36:0/764063934 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f337c053ad0 msgr2=0x7f337c055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:59.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.030+0000 7f33a392e700 1 --2- 172.21.15.36:0/764063934 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f337c053ad0 0x7f337c055f90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f338c01a000 tx=0x7f338c014000).stop 2022-06-17T12:52:59.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.030+0000 7f33a392e700 1 -- 172.21.15.36:0/764063934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f339c0ae150 msgr2=0x7f339c062a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:59.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.030+0000 7f33a392e700 1 --2- 172.21.15.36:0/764063934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f339c0ae150 0x7f339c062a80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f339c0b6a80 tx=0x7f3398006df0).stop 2022-06-17T12:52:59.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.030+0000 7f33a392e700 1 -- 172.21.15.36:0/764063934 shutdown_connections 2022-06-17T12:52:59.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.030+0000 7f33a392e700 1 --2- 172.21.15.36:0/764063934 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f337c053ad0 0x7f337c055f90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:59.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.030+0000 7f33a392e700 1 --2- 172.21.15.36:0/764063934 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f339c0b7bc0 0x7f339c0b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:59.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.030+0000 7f33a392e700 1 --2- 172.21.15.36:0/764063934 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f339c0b2fc0 0x7f339c152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:59.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.030+0000 7f33a392e700 1 --2- 172.21.15.36:0/764063934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f339c0ae150 0x7f339c062a80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:59.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.030+0000 7f33a392e700 1 -- 172.21.15.36:0/764063934 >> 172.21.15.36:0/764063934 conn(0x7f339c060990 msgr2=0x7f339c05c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:59.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.030+0000 7f33a392e700 1 -- 172.21.15.36:0/764063934 shutdown_connections 2022-06-17T12:52:59.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.030+0000 7f33a392e700 1 -- 172.21.15.36:0/764063934 wait complete. 2022-06-17T12:52:59.025 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-17T12:52:59.026 DEBUG:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:52:59.040 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:52:59.065 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:52:59.087 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:52:59.107 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:52:59.131 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-17T12:52:59.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:52:59.133 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:52:59.134 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:52:59.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:52:59.143 INFO:teuthology.orchestra.run.smithi107.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-17T12:52:59.144 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-06-17T12:52:59.144 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:52:59.145 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:52:59.145 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:52:59.154 INFO:teuthology.orchestra.run.smithi136.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-17T12:52:59.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:52:59.156 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-17T12:52:59.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:52:59.158 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:52:59.159 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:52:59.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:52:59.183 INFO:teuthology.orchestra.run.smithi143.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:52:59.184 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-06-17T12:52:59.185 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-17T12:52:59.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.314+0000 7f30dc9b1700 1 Processor -- start 2022-06-17T12:52:59.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.314+0000 7f30dc9b1700 1 -- start start 2022-06-17T12:52:59.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.314+0000 7f30dc9b1700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f30d8064340 0x7f30d80b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:59.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.314+0000 7f30dc9b1700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d80af1a0 con 0x7f30d80aedc0 2022-06-17T12:52:59.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.314+0000 7f30dc9b1700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d8063960 con 0x7f30d8067290 2022-06-17T12:52:59.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.314+0000 7f30dc9b1700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d8063c00 con 0x7f30d8064340 2022-06-17T12:52:59.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.314+0000 7f30d659c700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f30d80aedc0 0x7f30d8066b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52846/0 (socket says 172.21.15.36:52846) 2022-06-17T12:52:59.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.314+0000 7f30d659c700 1 -- 172.21.15.36:0/52121838 learned_addr learned my addr 172.21.15.36:0/52121838 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:52:59.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.314+0000 7f30d6d9d700 1 --2- 172.21.15.36:0/52121838 >> v2:172.21.15.70:3300/0 conn(0x7f30d8064340 0x7f30d80b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:59.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.314+0000 7f30d559a700 1 -- 172.21.15.36:0/52121838 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2747779405 0 0) 0x7f30d80af1a0 con 0x7f30d80aedc0 2022-06-17T12:52:59.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.314+0000 7f30d559a700 1 -- 172.21.15.36:0/52121838 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30bc003260 con 0x7f30d80aedc0 2022-06-17T12:52:59.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.314+0000 7f30d559a700 1 -- 172.21.15.36:0/52121838 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3481289767 0 0) 0x7f30d8063960 con 0x7f30d8067290 2022-06-17T12:52:59.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.314+0000 7f30d559a700 1 -- 172.21.15.36:0/52121838 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30d80af1a0 con 0x7f30d8067290 2022-06-17T12:52:59.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.314+0000 7f30d6d9d700 1 -- 172.21.15.36:0/52121838 >> v1:172.21.15.70:6789/0 conn(0x7f30d8067290 legacy=0x7f30d805f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:59.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.314+0000 7f30d6d9d700 1 -- 172.21.15.36:0/52121838 >> v1:172.21.15.36:6789/0 conn(0x7f30d80aedc0 legacy=0x7f30d8066b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:59.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.314+0000 7f30d6d9d700 1 -- 172.21.15.36:0/52121838 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d8063ea0 con 0x7f30d8064340 2022-06-17T12:52:59.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.314+0000 7f30d6d9d700 1 --2- 172.21.15.36:0/52121838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d8064340 0x7f30d80b0560 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f30d00032f0 tx=0x7f30d00091d0).ready entity=mon.1 client_cookie=30c67c96ab21bc55 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:59.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.314+0000 7f30d559a700 1 -- 172.21.15.36:0/52121838 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30d000b600 con 0x7f30d8064340 2022-06-17T12:52:59.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.314+0000 7f30d559a700 1 -- 172.21.15.36:0/52121838 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d000b760 con 0x7f30d8064340 2022-06-17T12:52:59.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.314+0000 7f30d559a700 1 -- 172.21.15.36:0/52121838 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30d000b8d0 con 0x7f30d8064340 2022-06-17T12:52:59.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.318+0000 7f30dc9b1700 1 -- 172.21.15.36:0/52121838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d8064340 msgr2=0x7f30d80b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:59.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.318+0000 7f30dc9b1700 1 --2- 172.21.15.36:0/52121838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d8064340 0x7f30d80b0560 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f30d00032f0 tx=0x7f30d00091d0).stop 2022-06-17T12:52:59.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.318+0000 7f30dc9b1700 1 -- 172.21.15.36:0/52121838 shutdown_connections 2022-06-17T12:52:59.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.318+0000 7f30dc9b1700 1 --2- 172.21.15.36:0/52121838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d8064340 0x7f30d80b0560 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:59.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.318+0000 7f30dc9b1700 1 -- 172.21.15.36:0/52121838 >> 172.21.15.36:0/52121838 conn(0x7f30d80b5430 msgr2=0x7f30d80b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:59.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.318+0000 7f30dc9b1700 1 -- 172.21.15.36:0/52121838 shutdown_connections 2022-06-17T12:52:59.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.318+0000 7f30dc9b1700 1 -- 172.21.15.36:0/52121838 wait complete. 2022-06-17T12:52:59.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.318+0000 7f30dc9b1700 1 Processor -- start 2022-06-17T12:52:59.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.318+0000 7f30dc9b1700 1 -- start start 2022-06-17T12:52:59.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.322+0000 7f30dc9b1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f30d8064340 0x7f30d8141810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:59.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.322+0000 7f30dc9b1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d8067290 0x7f30d8140610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:59.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.322+0000 7f30dc9b1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f30d80aedc0 0x7f30d8140b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:59.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.322+0000 7f30dc9b1700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d80a1780 con 0x7f30d8064340 2022-06-17T12:52:59.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.322+0000 7f30dc9b1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d8146440 con 0x7f30d8067290 2022-06-17T12:52:59.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.322+0000 7f30dc9b1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d80ab810 con 0x7f30d80aedc0 2022-06-17T12:52:59.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.322+0000 7f30d659c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f30d8064340 0x7f30d8141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:59.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.322+0000 7f30d659c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f30d8064340 0x7f30d8141810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35322/0 (socket says 172.21.15.36:35322) 2022-06-17T12:52:59.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.322+0000 7f30d659c700 1 -- 172.21.15.36:0/1324810190 learned_addr learned my addr 172.21.15.36:0/1324810190 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:59.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.322+0000 7f30d6d9d700 1 --2- 172.21.15.36:0/1324810190 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f30d80aedc0 0x7f30d8140b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:59.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.322+0000 7f30d5d9b700 1 --2- 172.21.15.36:0/1324810190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d8067290 0x7f30d8140610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:59.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.322+0000 7f30d659c700 1 -- 172.21.15.36:0/1324810190 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f30d80aedc0 msgr2=0x7f30d8140b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:59.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.322+0000 7f30d659c700 1 --2- 172.21.15.36:0/1324810190 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f30d80aedc0 0x7f30d8140b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:59.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.322+0000 7f30d659c700 1 -- 172.21.15.36:0/1324810190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d8067290 msgr2=0x7f30d8140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:59.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.322+0000 7f30d659c700 1 --2- 172.21.15.36:0/1324810190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d8067290 0x7f30d8140610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:59.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.322+0000 7f30d659c700 1 -- 172.21.15.36:0/1324810190 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d0009470 con 0x7f30d8064340 2022-06-17T12:52:59.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.322+0000 7f30d659c700 1 --2- 172.21.15.36:0/1324810190 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f30d8064340 0x7f30d8141810 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f30c8001040 tx=0x7f30c8007b70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:59.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.322+0000 7f30c77fe700 1 -- 172.21.15.36:0/1324810190 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30c80070d0 con 0x7f30d8064340 2022-06-17T12:52:59.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.322+0000 7f30c77fe700 1 -- 172.21.15.36:0/1324810190 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c8007230 con 0x7f30d8064340 2022-06-17T12:52:59.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.322+0000 7f30dc9b1700 1 -- 172.21.15.36:0/1324810190 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d8146670 con 0x7f30d8064340 2022-06-17T12:52:59.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.322+0000 7f30c77fe700 1 -- 172.21.15.36:0/1324810190 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30c8005610 con 0x7f30d8064340 2022-06-17T12:52:59.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.322+0000 7f30dc9b1700 1 -- 172.21.15.36:0/1324810190 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d80ab080 con 0x7f30d8064340 2022-06-17T12:52:59.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.326+0000 7f30dc9b1700 1 -- 172.21.15.36:0/1324810190 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d8057410 con 0x7f30d8064340 2022-06-17T12:52:59.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.326+0000 7f30c77fe700 1 -- 172.21.15.36:0/1324810190 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f30c8010070 con 0x7f30d8064340 2022-06-17T12:52:59.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.326+0000 7f30c77fe700 1 --2- 172.21.15.36:0/1324810190 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f30b4053920 0x7f30b4055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:59.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.326+0000 7f30c77fe700 1 -- 172.21.15.36:0/1324810190 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f30c806b360 con 0x7f30d8064340 2022-06-17T12:52:59.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.326+0000 7f30d5d9b700 1 --2- 172.21.15.36:0/1324810190 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f30b4053920 0x7f30b4055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:59.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.326+0000 7f30d5d9b700 1 --2- 172.21.15.36:0/1324810190 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f30b4053920 0x7f30b4055de0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f30cc00d000 tx=0x7f30cc007a20).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:59.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.334+0000 7f30c77fe700 1 -- 172.21.15.36:0/1324810190 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f30c8042330 con 0x7f30d8064340 2022-06-17T12:52:59.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.466+0000 7f30dc9b1700 1 -- 172.21.15.36:0/1324810190 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f30d80687c0 con 0x7f30d8064340 2022-06-17T12:52:59.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.470+0000 7f30c77fe700 1 -- 172.21.15.36:0/1324810190 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v68) v1 ==== 81+0+29220 (secure 0 0 0) 0x7f30c8045950 con 0x7f30d8064340 2022-06-17T12:52:59.454 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:52:59.454 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:52:59.455 INFO:teuthology.orchestra.run.smithi036.stdout: "epoch": 68, 2022-06-17T12:52:59.455 INFO:teuthology.orchestra.run.smithi036.stdout: "fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", 2022-06-17T12:52:59.456 INFO:teuthology.orchestra.run.smithi036.stdout: "created": "2022-06-17T12:46:30.671408+0000", 2022-06-17T12:52:59.456 INFO:teuthology.orchestra.run.smithi036.stdout: "modified": "2022-06-17T12:52:56.708486+0000", 2022-06-17T12:52:59.456 INFO:teuthology.orchestra.run.smithi036.stdout: "last_up_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:52:59.457 INFO:teuthology.orchestra.run.smithi036.stdout: "last_in_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:52:59.457 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-17T12:52:59.457 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_num": 5799936, 2022-06-17T12:52:59.457 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_set": [ 2022-06-17T12:52:59.458 INFO:teuthology.orchestra.run.smithi036.stdout: "pglog_hardlimit", 2022-06-17T12:52:59.458 INFO:teuthology.orchestra.run.smithi036.stdout: "purged_snapdirs", 2022-06-17T12:52:59.459 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_deletes", 2022-06-17T12:52:59.459 INFO:teuthology.orchestra.run.smithi036.stdout: "sortbitwise" 2022-06-17T12:52:59.459 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:52:59.460 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_version": 6, 2022-06-17T12:52:59.460 INFO:teuthology.orchestra.run.smithi036.stdout: "full_ratio": 0.94999998807907104, 2022-06-17T12:52:59.462 INFO:teuthology.orchestra.run.smithi036.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-17T12:52:59.462 INFO:teuthology.orchestra.run.smithi036.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-17T12:52:59.463 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_snapshot": "", 2022-06-17T12:52:59.463 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_max": 12, 2022-06-17T12:52:59.464 INFO:teuthology.orchestra.run.smithi036.stdout: "max_osd": 8, 2022-06-17T12:52:59.464 INFO:teuthology.orchestra.run.smithi036.stdout: "require_min_compat_client": "jewel", 2022-06-17T12:52:59.465 INFO:teuthology.orchestra.run.smithi036.stdout: "min_compat_client": "jewel", 2022-06-17T12:52:59.465 INFO:teuthology.orchestra.run.smithi036.stdout: "require_osd_release": "octopus", 2022-06-17T12:52:59.466 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:52:59.466 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.466 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 1, 2022-06-17T12:52:59.467 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "device_health_metrics", 2022-06-17T12:52:59.467 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:33.881212+0000", 2022-06-17T12:52:59.468 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:52:59.468 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:52:59.469 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:52:59.469 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:52:59.470 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:52:59.470 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:52:59.471 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:52:59.471 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:52:59.471 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 1, 2022-06-17T12:52:59.472 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 1, 2022-06-17T12:52:59.472 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 1, 2022-06-17T12:52:59.473 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 1, 2022-06-17T12:52:59.473 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 1, 2022-06-17T12:52:59.474 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:52:59.474 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:52:59.475 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:52:59.475 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:52:59.475 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:52:59.476 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:52:59.476 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:52:59.477 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.477 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "16", 2022-06-17T12:52:59.478 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:52:59.478 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:52:59.479 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:52:59.479 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:52:59.480 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:52:59.480 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:52:59.481 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:52:59.481 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:52:59.482 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:52:59.482 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:52:59.483 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:52:59.483 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:52:59.483 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:52:59.484 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:52:59.484 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:52:59.485 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:52:59.485 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:52:59.486 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:52:59.486 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:52:59.487 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:52:59.487 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:52:59.487 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:52:59.488 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:52:59.488 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:52:59.488 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:52:59.489 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:52:59.489 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.490 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:52:59.490 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:52:59.490 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:52:59.491 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:52:59.491 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:52:59.492 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:52:59.492 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:52:59.493 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:52:59.493 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:52:59.493 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:52:59.494 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:52:59.494 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:52:59.494 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 1 2022-06-17T12:52:59.495 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.495 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:52:59.495 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr_devicehealth": {} 2022-06-17T12:52:59.495 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.496 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.496 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.496 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 2, 2022-06-17T12:52:59.497 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "rbd", 2022-06-17T12:52:59.497 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:44.485460+0000", 2022-06-17T12:52:59.497 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:52:59.498 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:52:59.498 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:52:59.498 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:52:59.499 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:52:59.499 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:52:59.499 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:52:59.499 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:52:59.500 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:52:59.500 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:52:59.500 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:52:59.500 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:52:59.501 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:52:59.501 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:52:59.501 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:52:59.502 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:52:59.502 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:52:59.502 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:52:59.503 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:52:59.503 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:52:59.503 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.504 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "19", 2022-06-17T12:52:59.504 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:52:59.504 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:52:59.505 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:52:59.505 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:52:59.505 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:52:59.505 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:52:59.506 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:52:59.507 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:52:59.507 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:52:59.507 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:52:59.508 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:52:59.508 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:52:59.509 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:52:59.509 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:52:59.509 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:52:59.509 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:52:59.510 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:52:59.510 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:52:59.511 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:52:59.511 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:52:59.511 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:52:59.512 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:52:59.512 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:52:59.513 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:52:59.513 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:52:59.513 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:52:59.513 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.513 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:52:59.514 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:52:59.514 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:52:59.515 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:52:59.515 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:52:59.515 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:52:59.515 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:52:59.516 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:52:59.516 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:52:59.516 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:52:59.516 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:52:59.517 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:52:59.517 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:52:59.517 INFO:teuthology.orchestra.run.smithi036.stdout: "rbd": {} 2022-06-17T12:52:59.518 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.518 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.518 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:52:59.519 INFO:teuthology.orchestra.run.smithi036.stdout: "osds": [ 2022-06-17T12:52:59.519 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.519 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:52:59.519 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", 2022-06-17T12:52:59.520 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:59.520 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:59.520 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:59.520 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:59.521 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:59.521 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:59.521 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:52:59.521 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 63, 2022-06-17T12:52:59.522 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:59.522 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:59.522 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:59.522 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.523 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.523 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.523 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6802", 2022-06-17T12:52:59.523 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:59.523 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.524 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.524 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.524 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6803", 2022-06-17T12:52:59.524 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:59.525 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.525 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.525 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.525 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:59.525 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.526 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.526 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.527 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6804", 2022-06-17T12:52:59.527 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:59.527 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.528 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.528 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.528 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6805", 2022-06-17T12:52:59.529 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:59.529 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.529 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.530 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.530 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:59.531 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.531 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.531 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.532 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6808", 2022-06-17T12:52:59.532 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:59.533 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.533 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.534 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.534 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6809", 2022-06-17T12:52:59.534 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:59.535 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.535 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.536 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.536 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:59.536 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.537 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.537 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.537 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6806", 2022-06-17T12:52:59.538 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:59.538 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.539 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.539 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.539 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6807", 2022-06-17T12:52:59.540 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:52:59.540 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.540 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.540 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.541 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6803/22619", 2022-06-17T12:52:59.541 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6805/22619", 2022-06-17T12:52:59.541 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6809/22619", 2022-06-17T12:52:59.541 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6807/22619", 2022-06-17T12:52:59.542 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:59.542 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:59.542 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:59.542 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.542 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.543 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.543 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:52:59.543 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", 2022-06-17T12:52:59.544 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:59.544 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:59.544 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:59.545 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:59.545 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:59.545 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:59.545 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:52:59.545 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 63, 2022-06-17T12:52:59.546 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:59.546 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:59.546 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:59.547 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.547 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.547 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.547 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6810", 2022-06-17T12:52:59.547 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:59.548 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.548 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.548 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.548 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6811", 2022-06-17T12:52:59.549 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:59.549 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.549 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.549 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.549 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:59.550 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.550 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.550 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.550 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6812", 2022-06-17T12:52:59.551 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:59.551 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.551 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.551 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.551 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6813", 2022-06-17T12:52:59.552 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:59.552 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.552 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.552 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.553 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:59.553 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.553 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.554 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.554 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6816", 2022-06-17T12:52:59.554 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:59.554 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.555 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.555 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.555 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6817", 2022-06-17T12:52:59.555 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:59.556 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.556 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.556 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.557 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:59.557 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.557 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.558 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.558 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6814", 2022-06-17T12:52:59.558 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:59.559 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.559 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.559 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.559 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6815", 2022-06-17T12:52:59.559 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:52:59.560 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.560 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.560 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.561 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6811/22620", 2022-06-17T12:52:59.561 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6813/22620", 2022-06-17T12:52:59.561 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6817/22620", 2022-06-17T12:52:59.562 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6815/22620", 2022-06-17T12:52:59.562 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:59.562 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:59.563 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:59.563 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.563 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.564 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.564 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:52:59.564 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", 2022-06-17T12:52:59.565 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:59.565 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:59.565 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:59.565 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:59.566 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:59.566 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:59.566 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:52:59.566 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 63, 2022-06-17T12:52:59.567 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:59.567 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:59.567 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:59.567 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.567 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.568 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.568 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6826", 2022-06-17T12:52:59.568 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:59.568 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.569 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.569 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.569 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6827", 2022-06-17T12:52:59.569 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:59.570 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.570 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.571 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.571 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:59.571 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.571 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.572 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.572 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6828", 2022-06-17T12:52:59.572 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:59.572 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.573 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.573 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.573 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6829", 2022-06-17T12:52:59.573 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:59.573 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.574 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.574 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.574 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:59.574 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.575 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.575 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.575 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6832", 2022-06-17T12:52:59.575 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:59.576 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.576 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.576 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.576 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6833", 2022-06-17T12:52:59.576 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:59.577 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.577 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.577 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.577 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:59.578 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.578 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.578 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.578 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6830", 2022-06-17T12:52:59.579 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:59.579 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.579 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.580 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.580 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6831", 2022-06-17T12:52:59.580 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:52:59.580 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.580 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.581 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.581 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6827/22643", 2022-06-17T12:52:59.581 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6829/22643", 2022-06-17T12:52:59.581 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6833/22643", 2022-06-17T12:52:59.582 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6831/22643", 2022-06-17T12:52:59.582 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:59.583 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:59.583 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:59.584 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.584 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.584 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.584 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:52:59.585 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", 2022-06-17T12:52:59.585 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:59.585 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:59.586 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:59.586 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:59.586 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:59.586 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:59.587 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:52:59.587 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 51, 2022-06-17T12:52:59.588 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:59.588 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:59.588 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:59.588 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.588 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.589 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.589 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6818", 2022-06-17T12:52:59.589 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:59.590 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.590 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.590 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.591 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6819", 2022-06-17T12:52:59.591 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:59.591 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.592 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.592 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.592 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:59.593 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.593 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.593 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.594 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6820", 2022-06-17T12:52:59.594 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:59.595 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.595 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.595 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.596 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6821", 2022-06-17T12:52:59.596 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:59.597 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.597 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.597 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.598 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:59.598 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.599 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.599 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.599 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6824", 2022-06-17T12:52:59.599 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:59.600 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.600 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.601 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.601 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6825", 2022-06-17T12:52:59.601 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:59.601 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.602 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.602 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.603 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:59.603 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.604 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.604 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.604 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6822", 2022-06-17T12:52:59.605 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:59.605 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.605 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.605 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.606 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6823", 2022-06-17T12:52:59.607 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:52:59.607 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.607 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.608 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.608 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6819/22646", 2022-06-17T12:52:59.609 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6821/22646", 2022-06-17T12:52:59.609 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6825/22646", 2022-06-17T12:52:59.609 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6823/22646", 2022-06-17T12:52:59.609 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:59.610 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:59.610 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:59.611 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.611 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.611 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.612 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:52:59.612 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", 2022-06-17T12:52:59.612 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:59.613 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:59.613 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:59.613 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:59.613 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:59.614 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:59.614 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:52:59.614 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 63, 2022-06-17T12:52:59.615 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:59.615 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:59.615 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:59.616 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.616 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.616 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.617 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6808", 2022-06-17T12:52:59.617 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:59.617 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.618 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.618 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.618 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6809", 2022-06-17T12:52:59.619 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:59.619 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.619 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.620 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.620 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:59.620 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.621 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.621 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.621 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6810", 2022-06-17T12:52:59.621 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:59.622 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.622 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.623 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.623 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6811", 2022-06-17T12:52:59.624 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:59.624 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.624 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.625 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.625 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:59.625 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.626 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.626 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.626 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6814", 2022-06-17T12:52:59.627 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:59.627 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.627 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.628 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.628 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6815", 2022-06-17T12:52:59.628 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:59.629 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.629 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.630 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.630 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:59.630 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.631 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.631 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.631 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6812", 2022-06-17T12:52:59.632 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:59.632 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.632 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.633 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.633 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6813", 2022-06-17T12:52:59.633 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:52:59.633 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.634 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.634 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.635 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6809/22812", 2022-06-17T12:52:59.635 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6811/22812", 2022-06-17T12:52:59.635 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6815/22812", 2022-06-17T12:52:59.636 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6813/22812", 2022-06-17T12:52:59.636 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:59.636 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:59.636 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:59.637 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.637 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.637 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.638 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:52:59.638 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", 2022-06-17T12:52:59.639 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:59.639 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:59.640 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:59.640 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:59.641 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:59.641 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:59.642 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:52:59.642 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 62, 2022-06-17T12:52:59.642 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:59.643 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:59.643 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:59.643 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.644 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.644 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.645 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6800", 2022-06-17T12:52:59.645 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:59.646 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.646 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.646 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.647 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6801", 2022-06-17T12:52:59.647 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:59.648 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.649 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.649 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.649 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:59.650 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.650 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.651 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.651 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6802", 2022-06-17T12:52:59.651 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:59.651 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.652 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.652 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.652 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6803", 2022-06-17T12:52:59.652 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:59.653 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.653 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.653 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.653 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:59.654 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.654 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.654 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.655 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6806", 2022-06-17T12:52:59.655 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:59.655 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.656 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.656 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.656 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6807", 2022-06-17T12:52:59.656 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:59.657 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.657 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.657 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.657 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:59.658 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.658 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.658 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.658 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6804", 2022-06-17T12:52:59.659 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:59.659 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.659 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.659 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.660 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6805", 2022-06-17T12:52:59.660 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:52:59.660 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.660 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.661 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.661 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6801/22813", 2022-06-17T12:52:59.661 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6803/22813", 2022-06-17T12:52:59.661 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6807/22813", 2022-06-17T12:52:59.661 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6805/22813", 2022-06-17T12:52:59.662 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:59.662 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:59.663 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:59.663 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.663 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.663 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.663 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:52:59.664 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", 2022-06-17T12:52:59.664 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:59.664 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:59.664 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:59.664 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:59.665 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:59.665 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:59.665 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:52:59.665 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 63, 2022-06-17T12:52:59.666 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:59.666 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:59.666 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:59.666 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.667 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.667 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.667 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6816", 2022-06-17T12:52:59.667 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:59.668 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.668 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.668 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.668 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6817", 2022-06-17T12:52:59.668 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:59.669 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.669 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.669 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.669 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:59.670 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.670 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.670 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.670 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6818", 2022-06-17T12:52:59.670 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:59.671 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.671 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.671 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.671 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6819", 2022-06-17T12:52:59.672 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:59.672 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.672 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.672 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.673 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:59.673 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.673 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.673 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.674 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6822", 2022-06-17T12:52:59.674 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:59.674 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.674 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.675 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.675 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6823", 2022-06-17T12:52:59.675 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:59.676 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.676 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.677 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.677 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:59.677 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.678 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.678 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.678 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6820", 2022-06-17T12:52:59.679 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:59.679 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.679 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.679 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.680 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6821", 2022-06-17T12:52:59.680 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:52:59.680 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.680 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.680 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.681 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6817/22822", 2022-06-17T12:52:59.681 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6819/22822", 2022-06-17T12:52:59.681 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6823/22822", 2022-06-17T12:52:59.681 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6821/22822", 2022-06-17T12:52:59.682 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:59.682 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:59.682 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:59.682 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.683 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.683 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.683 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:52:59.683 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", 2022-06-17T12:52:59.683 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:52:59.684 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:52:59.684 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:52:59.684 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:52:59.684 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:52:59.684 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:52:59.685 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:52:59.685 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 62, 2022-06-17T12:52:59.685 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:52:59.685 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:52:59.686 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:52:59.686 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.686 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.686 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.687 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6824", 2022-06-17T12:52:59.687 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:59.687 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.687 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.688 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.688 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6825", 2022-06-17T12:52:59.688 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:59.688 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.689 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.689 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.689 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:52:59.689 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.690 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.690 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.690 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6826", 2022-06-17T12:52:59.690 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:59.691 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.691 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.691 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.691 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6827", 2022-06-17T12:52:59.691 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:59.692 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.692 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.692 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.693 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:52:59.693 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.694 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.694 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.694 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6830", 2022-06-17T12:52:59.695 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:59.695 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.696 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.696 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.697 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6831", 2022-06-17T12:52:59.697 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:59.697 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.698 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.698 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.699 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:52:59.699 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:52:59.700 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.700 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:52:59.700 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6828", 2022-06-17T12:52:59.701 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:59.701 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.702 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.702 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:52:59.703 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6829", 2022-06-17T12:52:59.704 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:52:59.704 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.704 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.704 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.705 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6825/22837", 2022-06-17T12:52:59.705 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6827/22837", 2022-06-17T12:52:59.705 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6831/22837", 2022-06-17T12:52:59.705 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6829/22837", 2022-06-17T12:52:59.706 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:52:59.706 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:52:59.706 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:52:59.706 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:52:59.707 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.707 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:52:59.707 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_xinfo": [ 2022-06-17T12:52:59.707 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.707 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:52:59.708 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:59.708 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:59.708 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:59.708 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:59.709 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:59.709 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.291461+0000", 2022-06-17T12:52:59.709 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:59.709 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.710 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.710 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:52:59.710 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:59.711 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:59.711 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:59.711 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:59.712 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:59.712 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.537912+0000", 2022-06-17T12:52:59.712 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:59.712 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.713 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.713 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:52:59.713 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:59.714 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:59.714 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:59.714 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:59.715 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:59.715 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.606947+0000", 2022-06-17T12:52:59.716 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:59.716 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.717 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.717 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:52:59.717 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:59.718 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:59.718 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:59.719 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:59.719 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:59.719 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.514412+0000", 2022-06-17T12:52:59.720 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:59.720 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.721 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.721 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:52:59.722 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:59.722 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:59.722 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:59.723 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:59.723 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:59.724 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.394645+0000", 2022-06-17T12:52:59.724 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:59.724 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.725 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.725 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:52:59.726 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:59.726 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:59.727 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:59.727 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:59.728 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:59.728 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.343806+0000", 2022-06-17T12:52:59.729 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:59.729 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.730 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.730 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:52:59.730 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:59.731 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:59.731 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:59.732 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:59.732 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:59.733 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.591864+0000", 2022-06-17T12:52:59.733 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:59.734 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.734 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:52:59.734 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:52:59.735 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:52:59.735 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:52:59.736 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:52:59.736 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:52:59.736 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:52:59.737 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.607045+0000", 2022-06-17T12:52:59.737 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:52:59.738 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.738 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:52:59.738 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap": [], 2022-06-17T12:52:59.739 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap_items": [], 2022-06-17T12:52:59.739 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_temp": [], 2022-06-17T12:52:59.739 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_temp": [], 2022-06-17T12:52:59.740 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklist": { 2022-06-17T12:52:59.740 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.36:6835/4091208534": "2022-06-18T12:52:53.504814+0000", 2022-06-17T12:52:59.740 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.36:6834/4091208534": "2022-06-18T12:52:53.504814+0000" 2022-06-17T12:52:59.741 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.741 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profiles": { 2022-06-17T12:52:59.741 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs_data": { 2022-06-17T12:52:59.742 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-device-class": "", 2022-06-17T12:52:59.742 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:52:59.742 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-root": "default", 2022-06-17T12:52:59.742 INFO:teuthology.orchestra.run.smithi036.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-17T12:52:59.742 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:52:59.743 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "2", 2022-06-17T12:52:59.743 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:52:59.743 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:52:59.743 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van", 2022-06-17T12:52:59.744 INFO:teuthology.orchestra.run.smithi036.stdout: "w": "8" 2022-06-17T12:52:59.744 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.744 INFO:teuthology.orchestra.run.smithi036.stdout: "default": { 2022-06-17T12:52:59.744 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:52:59.745 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:52:59.746 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "1", 2022-06-17T12:52:59.747 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:52:59.747 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:52:59.747 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van" 2022-06-17T12:52:59.748 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:52:59.748 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:52:59.749 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps_queue": [], 2022-06-17T12:52:59.749 INFO:teuthology.orchestra.run.smithi036.stdout: "new_removed_snaps": [], 2022-06-17T12:52:59.750 INFO:teuthology.orchestra.run.smithi036.stdout: "new_purged_snaps": [], 2022-06-17T12:52:59.750 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_node_flags": {}, 2022-06-17T12:52:59.750 INFO:teuthology.orchestra.run.smithi036.stdout: "device_class_flags": {} 2022-06-17T12:52:59.751 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:52:59.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.470+0000 7f30dc9b1700 1 -- 172.21.15.36:0/1324810190 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f30b4053920 msgr2=0x7f30b4055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:59.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.470+0000 7f30dc9b1700 1 --2- 172.21.15.36:0/1324810190 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f30b4053920 0x7f30b4055de0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f30cc00d000 tx=0x7f30cc007a20).stop 2022-06-17T12:52:59.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.470+0000 7f30dc9b1700 1 -- 172.21.15.36:0/1324810190 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f30d8064340 msgr2=0x7f30d8141810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:59.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.470+0000 7f30dc9b1700 1 --2- 172.21.15.36:0/1324810190 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f30d8064340 0x7f30d8141810 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f30c8001040 tx=0x7f30c8007b70).stop 2022-06-17T12:52:59.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.470+0000 7f30dc9b1700 1 -- 172.21.15.36:0/1324810190 shutdown_connections 2022-06-17T12:52:59.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.470+0000 7f30dc9b1700 1 --2- 172.21.15.36:0/1324810190 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f30b4053920 0x7f30b4055de0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:59.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.470+0000 7f30dc9b1700 1 --2- 172.21.15.36:0/1324810190 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f30d80aedc0 0x7f30d8140b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:59.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.470+0000 7f30dc9b1700 1 --2- 172.21.15.36:0/1324810190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d8067290 0x7f30d8140610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:59.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.470+0000 7f30dc9b1700 1 --2- 172.21.15.36:0/1324810190 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f30d8064340 0x7f30d8141810 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:59.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.470+0000 7f30dc9b1700 1 -- 172.21.15.36:0/1324810190 >> 172.21.15.36:0/1324810190 conn(0x7f30d80b5430 msgr2=0x7f30d80aba40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:59.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.470+0000 7f30dc9b1700 1 -- 172.21.15.36:0/1324810190 shutdown_connections 2022-06-17T12:52:59.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.470+0000 7f30dc9b1700 1 -- 172.21.15.36:0/1324810190 wait complete. 2022-06-17T12:52:59.757 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:52:59.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.878+0000 7fc74cb7e700 1 Processor -- start 2022-06-17T12:52:59.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.878+0000 7fc74cb7e700 1 -- start start 2022-06-17T12:52:59.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.878+0000 7fc74cb7e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc748060780 0x7fc748060b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:59.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.878+0000 7fc74cb7e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7481434f0 0x7fc7480b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:59.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.878+0000 7fc74cb7e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc748143990 con 0x7fc7480afec0 2022-06-17T12:52:59.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.878+0000 7fc74cb7e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7480b1a80 con 0x7fc748060780 2022-06-17T12:52:59.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.882+0000 7fc74cb7e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7480b1d20 con 0x7fc7481434f0 2022-06-17T12:52:59.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.882+0000 7fc74659c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc748060780 0x7fc748060b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:59.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.882+0000 7fc74659c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc748060780 0x7fc748060b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35330/0 (socket says 172.21.15.36:35330) 2022-06-17T12:52:59.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.882+0000 7fc74659c700 1 -- 172.21.15.36:0/2331555954 learned_addr learned my addr 172.21.15.36:0/2331555954 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:59.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.882+0000 7fc73ffff700 1 --2- 172.21.15.36:0/2331555954 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7481434f0 0x7fc7480b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:59.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.882+0000 7fc74659c700 1 -- 172.21.15.36:0/2331555954 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7481434f0 msgr2=0x7fc7480b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:52:59.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.882+0000 7fc74659c700 1 --2- 172.21.15.36:0/2331555954 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7481434f0 0x7fc7480b07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:59.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.882+0000 7fc74659c700 1 -- 172.21.15.36:0/2331555954 >> v1:172.21.15.70:6789/0 conn(0x7fc7480afec0 legacy=0x7fc7480b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:59.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.882+0000 7fc74659c700 1 -- 172.21.15.36:0/2331555954 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7480b2640 con 0x7fc748060780 2022-06-17T12:52:59.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.882+0000 7fc74659c700 1 --2- 172.21.15.36:0/2331555954 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc748060780 0x7fc748060b60 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7fc738002f70 tx=0x7fc738006eb0).ready entity=mon.0 client_cookie=fccb49059925032d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:59.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.882+0000 7fc745d9b700 1 -- 172.21.15.36:0/2331555954 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc73800d5c0 con 0x7fc748060780 2022-06-17T12:52:59.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.882+0000 7fc745d9b700 1 -- 172.21.15.36:0/2331555954 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc738000e90 con 0x7fc748060780 2022-06-17T12:52:59.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.882+0000 7fc745d9b700 1 -- 172.21.15.36:0/2331555954 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc73800d950 con 0x7fc748060780 2022-06-17T12:52:59.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.882+0000 7fc74cb7e700 1 -- 172.21.15.36:0/2331555954 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc748060780 msgr2=0x7fc748060b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:59.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.882+0000 7fc74cb7e700 1 --2- 172.21.15.36:0/2331555954 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc748060780 0x7fc748060b60 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7fc738002f70 tx=0x7fc738006eb0).stop 2022-06-17T12:52:59.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.882+0000 7fc74cb7e700 1 -- 172.21.15.36:0/2331555954 shutdown_connections 2022-06-17T12:52:59.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.882+0000 7fc74cb7e700 1 --2- 172.21.15.36:0/2331555954 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7481434f0 0x7fc7480b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:59.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.882+0000 7fc74cb7e700 1 --2- 172.21.15.36:0/2331555954 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc748060780 0x7fc748060b60 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:59.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.882+0000 7fc74cb7e700 1 -- 172.21.15.36:0/2331555954 >> 172.21.15.36:0/2331555954 conn(0x7fc7480b6870 msgr2=0x7fc7480b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:52:59.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.882+0000 7fc74cb7e700 1 -- 172.21.15.36:0/2331555954 shutdown_connections 2022-06-17T12:52:59.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.882+0000 7fc74cb7e700 1 -- 172.21.15.36:0/2331555954 wait complete. 2022-06-17T12:52:59.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.886+0000 7fc74cb7e700 1 Processor -- start 2022-06-17T12:52:59.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.886+0000 7fc74cb7e700 1 -- start start 2022-06-17T12:52:59.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.886+0000 7fc74cb7e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc748060780 0x7fc7480a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:59.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.886+0000 7fc74cb7e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7480afec0 0x7fc7480a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:59.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.886+0000 7fc74cb7e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7481434f0 0x7fc748150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:59.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.886+0000 7fc74cb7e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7480a4f00 con 0x7fc748060780 2022-06-17T12:52:59.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.886+0000 7fc74cb7e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7480a51a0 con 0x7fc7480afec0 2022-06-17T12:52:59.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.886+0000 7fc74cb7e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7480a0b80 con 0x7fc7481434f0 2022-06-17T12:52:59.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.886+0000 7fc74659c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc748060780 0x7fc7480a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:59.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.886+0000 7fc74659c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc748060780 0x7fc7480a2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35336/0 (socket says 172.21.15.36:35336) 2022-06-17T12:52:59.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.886+0000 7fc74659c700 1 -- 172.21.15.36:0/618198867 learned_addr learned my addr 172.21.15.36:0/618198867 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:52:59.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.886+0000 7fc73ffff700 1 --2- 172.21.15.36:0/618198867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7480afec0 0x7fc7480a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:59.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.886+0000 7fc746d9d700 1 --2- 172.21.15.36:0/618198867 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7481434f0 0x7fc748150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:59.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.886+0000 7fc74659c700 1 -- 172.21.15.36:0/618198867 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7481434f0 msgr2=0x7fc748150100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:59.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.886+0000 7fc74659c700 1 --2- 172.21.15.36:0/618198867 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7481434f0 0x7fc748150100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:59.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.886+0000 7fc74659c700 1 -- 172.21.15.36:0/618198867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7480afec0 msgr2=0x7fc7480a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:52:59.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.886+0000 7fc74659c700 1 --2- 172.21.15.36:0/618198867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7480afec0 0x7fc7480a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:52:59.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.886+0000 7fc74659c700 1 -- 172.21.15.36:0/618198867 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73800b1e0 con 0x7fc748060780 2022-06-17T12:52:59.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.890+0000 7fc74659c700 1 --2- 172.21.15.36:0/618198867 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc748060780 0x7fc7480a2ac0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7fc738002f70 tx=0x7fc738015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:59.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.890+0000 7fc73f7fe700 1 -- 172.21.15.36:0/618198867 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7380117e0 con 0x7fc748060780 2022-06-17T12:52:59.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.890+0000 7fc73f7fe700 1 -- 172.21.15.36:0/618198867 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc738011940 con 0x7fc748060780 2022-06-17T12:52:59.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.890+0000 7fc73f7fe700 1 -- 172.21.15.36:0/618198867 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc738011ca0 con 0x7fc748060780 2022-06-17T12:52:59.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.890+0000 7fc74cb7e700 1 -- 172.21.15.36:0/618198867 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7480a1020 con 0x7fc748060780 2022-06-17T12:52:59.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.890+0000 7fc74cb7e700 1 -- 172.21.15.36:0/618198867 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7481505f0 con 0x7fc748060780 2022-06-17T12:52:59.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.890+0000 7fc74cb7e700 1 -- 172.21.15.36:0/618198867 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc748057410 con 0x7fc748060780 2022-06-17T12:52:59.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.890+0000 7fc73f7fe700 1 -- 172.21.15.36:0/618198867 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fc738025030 con 0x7fc748060780 2022-06-17T12:52:59.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.890+0000 7fc73f7fe700 1 --2- 172.21.15.36:0/618198867 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc72c0539b0 0x7fc72c055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:52:59.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.890+0000 7fc73f7fe700 1 -- 172.21.15.36:0/618198867 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fc73806cf10 con 0x7fc748060780 2022-06-17T12:52:59.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.890+0000 7fc73ffff700 1 --2- 172.21.15.36:0/618198867 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc72c0539b0 0x7fc72c055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:52:59.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.894+0000 7fc73ffff700 1 --2- 172.21.15.36:0/618198867 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc72c0539b0 0x7fc72c055e70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fc73000c000 tx=0x7fc730008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:52:59.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:52:59.898+0000 7fc73f7fe700 1 -- 172.21.15.36:0/618198867 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc738043ff0 con 0x7fc748060780 2022-06-17T12:53:00.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.066+0000 7fc74cb7e700 1 -- 172.21.15.36:0/618198867 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7480a0d40 con 0x7fc748060780 2022-06-17T12:53:00.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.066+0000 7fc73f7fe700 1 -- 172.21.15.36:0/618198867 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+1206 (secure 0 0 0) 0x7fc738023090 con 0x7fc748060780 2022-06-17T12:53:00.052 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:53:00.053 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":32,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5420,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/3105680199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3105680199},{"type":"v1","addr":"172.21.15.70:6833","nonce":3105680199}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":29},{"gid":6291,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/381420820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":381420820},{"type":"v1","addr":"172.21.15.36:6835","nonce":381420820}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32}],"filesystems":[]} 2022-06-17T12:53:00.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.070+0000 7fc74cb7e700 1 -- 172.21.15.36:0/618198867 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc72c0539b0 msgr2=0x7fc72c055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:00.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.070+0000 7fc74cb7e700 1 --2- 172.21.15.36:0/618198867 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc72c0539b0 0x7fc72c055e70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fc73000c000 tx=0x7fc730008040).stop 2022-06-17T12:53:00.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.070+0000 7fc74cb7e700 1 -- 172.21.15.36:0/618198867 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc748060780 msgr2=0x7fc7480a2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:00.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.070+0000 7fc74cb7e700 1 --2- 172.21.15.36:0/618198867 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc748060780 0x7fc7480a2ac0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7fc738002f70 tx=0x7fc738015000).stop 2022-06-17T12:53:00.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.070+0000 7fc74cb7e700 1 -- 172.21.15.36:0/618198867 shutdown_connections 2022-06-17T12:53:00.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.070+0000 7fc74cb7e700 1 --2- 172.21.15.36:0/618198867 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc72c0539b0 0x7fc72c055e70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:00.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.070+0000 7fc74cb7e700 1 --2- 172.21.15.36:0/618198867 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7481434f0 0x7fc748150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:00.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.070+0000 7fc74cb7e700 1 --2- 172.21.15.36:0/618198867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7480afec0 0x7fc7480a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:00.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.070+0000 7fc74cb7e700 1 --2- 172.21.15.36:0/618198867 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc748060780 0x7fc7480a2ac0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:00.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.070+0000 7fc74cb7e700 1 -- 172.21.15.36:0/618198867 >> 172.21.15.36:0/618198867 conn(0x7fc7480b6870 msgr2=0x7fc7480b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:00.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.070+0000 7fc74cb7e700 1 -- 172.21.15.36:0/618198867 shutdown_connections 2022-06-17T12:53:00.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.070+0000 7fc74cb7e700 1 -- 172.21.15.36:0/618198867 wait complete. 2022-06-17T12:53:00.058 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 32 2022-06-17T12:53:00.065 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:53:00.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.182+0000 7f77dbfff700 1 Processor -- start 2022-06-17T12:53:00.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.182+0000 7f77dbfff700 1 -- start start 2022-06-17T12:53:00.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.182+0000 7f77dbfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f77dc060ac0 0x7f77dc144b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:00.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.182+0000 7f77dbfff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f77dc0afd80 0x7f77dc140bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:00.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.182+0000 7f77dbfff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77dc0b3140 con 0x7f77dc0af540 2022-06-17T12:53:00.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.182+0000 7f77dbfff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77dc1441e0 con 0x7f77dc060ac0 2022-06-17T12:53:00.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.182+0000 7f77dbfff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77dc060ea0 con 0x7f77dc0afd80 2022-06-17T12:53:00.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.182+0000 7f77daffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f77dc060ac0 0x7f77dc144b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:00.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.182+0000 7f77db7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f77dc0af540 0x7f77dc0b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52878/0 (socket says 172.21.15.36:52878) 2022-06-17T12:53:00.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.182+0000 7f77db7fe700 1 -- 172.21.15.36:0/1459315812 learned_addr learned my addr 172.21.15.36:0/1459315812 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:53:00.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.182+0000 7f77da7fc700 1 --2- 172.21.15.36:0/1459315812 >> v2:172.21.15.70:3300/0 conn(0x7f77dc0afd80 0x7f77dc140bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:00.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.182+0000 7f77daffd700 1 -- 172.21.15.36:0/1459315812 >> v2:172.21.15.70:3300/0 conn(0x7f77dc0afd80 msgr2=0x7f77dc140bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:00.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.182+0000 7f77daffd700 1 --2- 172.21.15.36:0/1459315812 >> v2:172.21.15.70:3300/0 conn(0x7f77dc0afd80 0x7f77dc140bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:00.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.182+0000 7f77daffd700 1 -- 172.21.15.36:0/1459315812 >> v1:172.21.15.36:6789/0 conn(0x7f77dc0af540 legacy=0x7f77dc0b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:00.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.182+0000 7f77d9ffb700 1 -- 172.21.15.36:0/1459315812 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 823850605 0 0) 0x7f77dc0b3140 con 0x7f77dc0af540 2022-06-17T12:53:00.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.182+0000 7f77daffd700 1 -- 172.21.15.36:0/1459315812 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77dc1443d0 con 0x7f77dc060ac0 2022-06-17T12:53:00.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.182+0000 7f77daffd700 1 --2- 172.21.15.36:0/1459315812 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77dc060ac0 0x7f77dc144b10 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f77d0002f70 tx=0x7f77d0006df0).ready entity=mon.0 client_cookie=fc31b9a65865bfb server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:00.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.182+0000 7f77d9ffb700 1 -- 172.21.15.36:0/1459315812 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77d000d570 con 0x7f77dc060ac0 2022-06-17T12:53:00.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.182+0000 7f77d9ffb700 1 -- 172.21.15.36:0/1459315812 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d0000e90 con 0x7f77dc060ac0 2022-06-17T12:53:00.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.186+0000 7f77d9ffb700 1 -- 172.21.15.36:0/1459315812 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77d000d8c0 con 0x7f77dc060ac0 2022-06-17T12:53:00.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.186+0000 7f77dbfff700 1 -- 172.21.15.36:0/1459315812 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77dc060ac0 msgr2=0x7f77dc144b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:00.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.186+0000 7f77dbfff700 1 --2- 172.21.15.36:0/1459315812 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77dc060ac0 0x7f77dc144b10 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f77d0002f70 tx=0x7f77d0006df0).stop 2022-06-17T12:53:00.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.186+0000 7f77dbfff700 1 -- 172.21.15.36:0/1459315812 shutdown_connections 2022-06-17T12:53:00.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.186+0000 7f77dbfff700 1 --2- 172.21.15.36:0/1459315812 >> v2:172.21.15.70:3300/0 conn(0x7f77dc0afd80 0x7f77dc140bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:00.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.186+0000 7f77dbfff700 1 --2- 172.21.15.36:0/1459315812 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77dc060ac0 0x7f77dc144b10 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:00.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.186+0000 7f77dbfff700 1 -- 172.21.15.36:0/1459315812 >> 172.21.15.36:0/1459315812 conn(0x7f77dc0b65f0 msgr2=0x7f77dc0b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:00.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.186+0000 7f77dbfff700 1 -- 172.21.15.36:0/1459315812 shutdown_connections 2022-06-17T12:53:00.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.186+0000 7f77dbfff700 1 -- 172.21.15.36:0/1459315812 wait complete. 2022-06-17T12:53:00.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.186+0000 7f77dbfff700 1 Processor -- start 2022-06-17T12:53:00.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.186+0000 7f77dbfff700 1 -- start start 2022-06-17T12:53:00.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.186+0000 7f77dbfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77dc060ac0 0x7f77dc13c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:00.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.186+0000 7f77dbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77dc0af540 0x7f77dc13fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:00.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.186+0000 7f77dbfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77dc0afd80 0x7f77dc140df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:00.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.186+0000 7f77dbfff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc0ad4b0 con 0x7f77dc0afd80 2022-06-17T12:53:00.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.186+0000 7f77dbfff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc0ae320 con 0x7f77dc0af540 2022-06-17T12:53:00.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.186+0000 7f77dbfff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc143b70 con 0x7f77dc060ac0 2022-06-17T12:53:00.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.186+0000 7f77db7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77dc0afd80 0x7f77dc140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:00.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.186+0000 7f77db7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77dc0afd80 0x7f77dc140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35354/0 (socket says 172.21.15.36:35354) 2022-06-17T12:53:00.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.186+0000 7f77db7fe700 1 -- 172.21.15.36:0/3864087368 learned_addr learned my addr 172.21.15.36:0/3864087368 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:00.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.186+0000 7f77da7fc700 1 --2- 172.21.15.36:0/3864087368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77dc0af540 0x7f77dc13fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:00.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.186+0000 7f77daffd700 1 --2- 172.21.15.36:0/3864087368 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77dc060ac0 0x7f77dc13c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:00.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.190+0000 7f77db7fe700 1 -- 172.21.15.36:0/3864087368 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77dc060ac0 msgr2=0x7f77dc13c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:00.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.190+0000 7f77db7fe700 1 --2- 172.21.15.36:0/3864087368 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77dc060ac0 0x7f77dc13c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:00.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.190+0000 7f77db7fe700 1 -- 172.21.15.36:0/3864087368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77dc0af540 msgr2=0x7f77dc13fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:00.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.190+0000 7f77db7fe700 1 --2- 172.21.15.36:0/3864087368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77dc0af540 0x7f77dc13fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:00.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.190+0000 7f77db7fe700 1 -- 172.21.15.36:0/3864087368 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d000b180 con 0x7f77dc0afd80 2022-06-17T12:53:00.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.190+0000 7f77da7fc700 1 --2- 172.21.15.36:0/3864087368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77dc0af540 0x7f77dc13fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:53:00.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.190+0000 7f77db7fe700 1 --2- 172.21.15.36:0/3864087368 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77dc0afd80 0x7f77dc140df0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f77d4001250 tx=0x7f77d4007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:00.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.190+0000 7f77cbfff700 1 -- 172.21.15.36:0/3864087368 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77d4005c70 con 0x7f77dc0afd80 2022-06-17T12:53:00.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.190+0000 7f77cbfff700 1 -- 172.21.15.36:0/3864087368 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d4005dd0 con 0x7f77dc0afd80 2022-06-17T12:53:00.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.190+0000 7f77dbfff700 1 -- 172.21.15.36:0/3864087368 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77dc0b4ea0 con 0x7f77dc0afd80 2022-06-17T12:53:00.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.190+0000 7f77cbfff700 1 -- 172.21.15.36:0/3864087368 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77d4004a90 con 0x7f77dc0afd80 2022-06-17T12:53:00.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.190+0000 7f77dbfff700 1 -- 172.21.15.36:0/3864087368 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77dc141b10 con 0x7f77dc0afd80 2022-06-17T12:53:00.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.190+0000 7f77dbfff700 1 -- 172.21.15.36:0/3864087368 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77dc144670 con 0x7f77dc0afd80 2022-06-17T12:53:00.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.190+0000 7f77cbfff700 1 -- 172.21.15.36:0/3864087368 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f77d4004bf0 con 0x7f77dc0afd80 2022-06-17T12:53:00.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.194+0000 7f77cbfff700 1 --2- 172.21.15.36:0/3864087368 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f77b8053a60 0x7f77b8055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:00.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.194+0000 7f77cbfff700 1 -- 172.21.15.36:0/3864087368 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f77d406b9a0 con 0x7f77dc0afd80 2022-06-17T12:53:00.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.194+0000 7f77daffd700 1 --2- 172.21.15.36:0/3864087368 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f77b8053a60 0x7f77b8055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:00.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.194+0000 7f77daffd700 1 --2- 172.21.15.36:0/3864087368 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f77b8053a60 0x7f77b8055f20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f77d001a000 tx=0x7f77d0014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:00.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.198+0000 7f77cbfff700 1 -- 172.21.15.36:0/3864087368 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f77d4042a00 con 0x7f77dc0afd80 2022-06-17T12:53:00.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.362+0000 7f77dbfff700 1 -- 172.21.15.36:0/3864087368 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77dc057410 con 0x7f77dc0afd80 2022-06-17T12:53:00.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.362+0000 7f77cbfff700 1 -- 172.21.15.36:0/3864087368 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+1206 (secure 0 0 0) 0x7f77d406b4d0 con 0x7f77dc0afd80 2022-06-17T12:53:00.347 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:53:00.347 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":32,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5420,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/3105680199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3105680199},{"type":"v1","addr":"172.21.15.70:6833","nonce":3105680199}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":29},{"gid":6291,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/381420820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":381420820},{"type":"v1","addr":"172.21.15.36:6835","nonce":381420820}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32}],"filesystems":[]} 2022-06-17T12:53:00.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.362+0000 7f77dbfff700 1 -- 172.21.15.36:0/3864087368 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f77b8053a60 msgr2=0x7f77b8055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:00.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.362+0000 7f77dbfff700 1 --2- 172.21.15.36:0/3864087368 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f77b8053a60 0x7f77b8055f20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f77d001a000 tx=0x7f77d0014000).stop 2022-06-17T12:53:00.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.362+0000 7f77dbfff700 1 -- 172.21.15.36:0/3864087368 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77dc0afd80 msgr2=0x7f77dc140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:00.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.362+0000 7f77dbfff700 1 --2- 172.21.15.36:0/3864087368 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77dc0afd80 0x7f77dc140df0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f77d4001250 tx=0x7f77d4007dc0).stop 2022-06-17T12:53:00.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.362+0000 7f77dbfff700 1 -- 172.21.15.36:0/3864087368 shutdown_connections 2022-06-17T12:53:00.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.362+0000 7f77dbfff700 1 --2- 172.21.15.36:0/3864087368 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f77b8053a60 0x7f77b8055f20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:00.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.362+0000 7f77dbfff700 1 --2- 172.21.15.36:0/3864087368 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77dc0afd80 0x7f77dc140df0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:00.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.362+0000 7f77dbfff700 1 --2- 172.21.15.36:0/3864087368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77dc0af540 0x7f77dc13fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:00.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.362+0000 7f77dbfff700 1 --2- 172.21.15.36:0/3864087368 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77dc060ac0 0x7f77dc13c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:00.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.362+0000 7f77dbfff700 1 -- 172.21.15.36:0/3864087368 >> 172.21.15.36:0/3864087368 conn(0x7f77dc0b65f0 msgr2=0x7f77dc13dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:00.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.362+0000 7f77dbfff700 1 -- 172.21.15.36:0/3864087368 shutdown_connections 2022-06-17T12:53:00.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.362+0000 7f77dbfff700 1 -- 172.21.15.36:0/3864087368 wait complete. 2022-06-17T12:53:00.353 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 32 2022-06-17T12:53:00.360 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-17T12:53:00.360 INFO:tasks.ceph.mds.a:Stopping old one... 2022-06-17T12:53:00.361 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-17T12:53:00.361 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T12:53:00.362 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-17T12:53:00.362 INFO:tasks.ceph.mds.b:Stopping old one... 2022-06-17T12:53:00.362 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-17T12:53:00.363 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T12:53:00.376 INFO:tasks.ceph.mds.a:Stopped 2022-06-17T12:53:00.377 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-17T12:53:00.379 INFO:tasks.ceph.mds.b:Stopped 2022-06-17T12:53:00.379 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-17T12:53:00.382 INFO:tasks.ceph.mds.b:Started 2022-06-17T12:53:00.383 DEBUG:teuthology.parallel:result is None 2022-06-17T12:53:00.418 INFO:tasks.ceph.mds.a:Started 2022-06-17T12:53:00.419 DEBUG:teuthology.parallel:result is None 2022-06-17T12:53:00.419 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-06-17T12:53:00.456 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2022-06-17T12:53:00.496 INFO:tasks.ceph.mds.a.smithi036.stdout:starting mds.a at 2022-06-17T12:53:00.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f648974e700 1 Processor -- start 2022-06-17T12:53:00.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f648974e700 1 -- start start 2022-06-17T12:53:00.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f648974e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6484063730 0x7f64840b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:00.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f648974e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f6484140610 0x7f64840b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:00.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f648974e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6484063b10 con 0x7f6484064eb0 2022-06-17T12:53:00.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f648974e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6484147430 con 0x7f6484140610 2022-06-17T12:53:00.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f648974e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64840b1ca0 con 0x7f6484063730 2022-06-17T12:53:00.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f6488f4d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f6484140610 0x7f64840b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:00.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f6488f4d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f6484140610 0x7f64840b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35374/0 (socket says 172.21.15.36:35374) 2022-06-17T12:53:00.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f6488f4d700 1 -- 172.21.15.36:0/2831832393 learned_addr learned my addr 172.21.15.36:0/2831832393 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:00.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f6483fff700 1 --2- 172.21.15.36:0/2831832393 >> v2:172.21.15.70:3300/0 conn(0x7f6484063730 0x7f64840b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:00.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f6488f4d700 1 -- 172.21.15.36:0/2831832393 >> v2:172.21.15.70:3300/0 conn(0x7f6484063730 msgr2=0x7f64840b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:00.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f6488f4d700 1 --2- 172.21.15.36:0/2831832393 >> v2:172.21.15.70:3300/0 conn(0x7f6484063730 0x7f64840b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:00.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f6488f4d700 1 -- 172.21.15.36:0/2831832393 >> v1:172.21.15.70:6789/0 conn(0x7f6484064eb0 legacy=0x7f648413ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:00.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f6488f4d700 1 -- 172.21.15.36:0/2831832393 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64841464d0 con 0x7f6484140610 2022-06-17T12:53:00.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f6488f4d700 1 --2- 172.21.15.36:0/2831832393 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6484140610 0x7f64840b33c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f647c0032f0 tx=0x7f647c009290).ready entity=mon.0 client_cookie=2915883ce4f14613 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:00.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f64837fe700 1 -- 172.21.15.36:0/2831832393 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f647c00b870 con 0x7f6484140610 2022-06-17T12:53:00.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f64837fe700 1 -- 172.21.15.36:0/2831832393 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647c00b9d0 con 0x7f6484140610 2022-06-17T12:53:00.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f64837fe700 1 -- 172.21.15.36:0/2831832393 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f647c00bb40 con 0x7f6484140610 2022-06-17T12:53:00.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f648974e700 1 -- 172.21.15.36:0/2831832393 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6484140610 msgr2=0x7f64840b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:00.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f648974e700 1 --2- 172.21.15.36:0/2831832393 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6484140610 0x7f64840b33c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f647c0032f0 tx=0x7f647c009290).stop 2022-06-17T12:53:00.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f648974e700 1 -- 172.21.15.36:0/2831832393 shutdown_connections 2022-06-17T12:53:00.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f648974e700 1 --2- 172.21.15.36:0/2831832393 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6484140610 0x7f64840b33c0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:00.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f648974e700 1 --2- 172.21.15.36:0/2831832393 >> v2:172.21.15.70:3300/0 conn(0x7f6484063730 0x7f64840b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:00.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f648974e700 1 -- 172.21.15.36:0/2831832393 >> 172.21.15.36:0/2831832393 conn(0x7f64840614b0 msgr2=0x7f64840618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:00.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f648974e700 1 -- 172.21.15.36:0/2831832393 shutdown_connections 2022-06-17T12:53:00.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.542+0000 7f648974e700 1 -- 172.21.15.36:0/2831832393 wait complete. 2022-06-17T12:53:00.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.546+0000 7f648974e700 1 Processor -- start 2022-06-17T12:53:00.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.546+0000 7f648974e700 1 -- start start 2022-06-17T12:53:00.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.546+0000 7f648974e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6484063730 0x7f6484152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:00.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.546+0000 7f648974e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6484064eb0 0x7f6484132b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:00.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.546+0000 7f648974e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6484140610 0x7f6484133050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:00.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.546+0000 7f648974e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64841525e0 con 0x7f6484140610 2022-06-17T12:53:00.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.546+0000 7f648974e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6484152880 con 0x7f6484063730 2022-06-17T12:53:00.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.546+0000 7f648974e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6484152b20 con 0x7f6484064eb0 2022-06-17T12:53:00.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.546+0000 7f6488f4d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6484140610 0x7f6484133050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:00.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.546+0000 7f6483fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6484063730 0x7f6484152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:00.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.546+0000 7f647b7fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6484064eb0 0x7f6484132b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:00.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.546+0000 7f6488f4d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6484140610 0x7f6484133050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35380/0 (socket says 172.21.15.36:35380) 2022-06-17T12:53:00.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.546+0000 7f6488f4d700 1 -- 172.21.15.36:0/2585326183 learned_addr learned my addr 172.21.15.36:0/2585326183 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:00.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.550+0000 7f6488f4d700 1 -- 172.21.15.36:0/2585326183 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6484064eb0 msgr2=0x7f6484132b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:00.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.550+0000 7f6488f4d700 1 --2- 172.21.15.36:0/2585326183 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6484064eb0 0x7f6484132b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:00.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.550+0000 7f6488f4d700 1 -- 172.21.15.36:0/2585326183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6484063730 msgr2=0x7f6484152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:00.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.550+0000 7f6488f4d700 1 --2- 172.21.15.36:0/2585326183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6484063730 0x7f6484152fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:00.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.550+0000 7f6488f4d700 1 -- 172.21.15.36:0/2585326183 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f647c009530 con 0x7f6484140610 2022-06-17T12:53:00.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.550+0000 7f6488f4d700 1 --2- 172.21.15.36:0/2585326183 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6484140610 0x7f6484133050 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f647c0032f0 tx=0x7f647c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:00.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.550+0000 7f6481ffb700 1 -- 172.21.15.36:0/2585326183 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f647c00b870 con 0x7f6484140610 2022-06-17T12:53:00.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.550+0000 7f6481ffb700 1 -- 172.21.15.36:0/2585326183 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647c021440 con 0x7f6484140610 2022-06-17T12:53:00.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.550+0000 7f648974e700 1 -- 172.21.15.36:0/2585326183 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6484114260 con 0x7f6484140610 2022-06-17T12:53:00.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.550+0000 7f6481ffb700 1 -- 172.21.15.36:0/2585326183 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f647c0217a0 con 0x7f6484140610 2022-06-17T12:53:00.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.550+0000 7f648974e700 1 -- 172.21.15.36:0/2585326183 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6484152070 con 0x7f6484140610 2022-06-17T12:53:00.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.550+0000 7f648974e700 1 -- 172.21.15.36:0/2585326183 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6484152ce0 con 0x7f6484140610 2022-06-17T12:53:00.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.550+0000 7f6481ffb700 1 -- 172.21.15.36:0/2585326183 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f647c027020 con 0x7f6484140610 2022-06-17T12:53:00.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.550+0000 7f6481ffb700 1 --2- 172.21.15.36:0/2585326183 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f64680539b0 0x7f6468055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:00.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.550+0000 7f6481ffb700 1 -- 172.21.15.36:0/2585326183 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f647c06d190 con 0x7f6484140610 2022-06-17T12:53:00.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.550+0000 7f6483fff700 1 --2- 172.21.15.36:0/2585326183 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f64680539b0 0x7f6468055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:00.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.554+0000 7f6483fff700 1 --2- 172.21.15.36:0/2585326183 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f64680539b0 0x7f6468055e70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f647400c000 tx=0x7f6474008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:00.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.558+0000 7f6481ffb700 1 -- 172.21.15.36:0/2585326183 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f647c044320 con 0x7f6484140610 2022-06-17T12:53:00.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:00.694+0000 7f648974e700 1 -- 172.21.15.36:0/2585326183 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f6484057410 con 0x7f6484140610 2022-06-17T12:53:01.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.514+0000 7f6481ffb700 1 -- 172.21.15.36:0/2585326183 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v70) v1 ==== 97+0+0 (secure 0 0 0) 0x7f647c015020 con 0x7f6484140610 2022-06-17T12:53:01.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.518+0000 7f648974e700 1 -- 172.21.15.36:0/2585326183 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f64680539b0 msgr2=0x7f6468055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:01.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.518+0000 7f648974e700 1 --2- 172.21.15.36:0/2585326183 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f64680539b0 0x7f6468055e70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f647400c000 tx=0x7f6474008040).stop 2022-06-17T12:53:01.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.518+0000 7f648974e700 1 -- 172.21.15.36:0/2585326183 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6484140610 msgr2=0x7f6484133050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:01.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.518+0000 7f648974e700 1 --2- 172.21.15.36:0/2585326183 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6484140610 0x7f6484133050 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f647c0032f0 tx=0x7f647c016000).stop 2022-06-17T12:53:01.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.518+0000 7f648974e700 1 -- 172.21.15.36:0/2585326183 shutdown_connections 2022-06-17T12:53:01.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.518+0000 7f648974e700 1 --2- 172.21.15.36:0/2585326183 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f64680539b0 0x7f6468055e70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:01.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.518+0000 7f648974e700 1 --2- 172.21.15.36:0/2585326183 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6484140610 0x7f6484133050 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:01.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.518+0000 7f648974e700 1 --2- 172.21.15.36:0/2585326183 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6484064eb0 0x7f6484132b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:01.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.518+0000 7f648974e700 1 --2- 172.21.15.36:0/2585326183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6484063730 0x7f6484152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:01.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.518+0000 7f648974e700 1 -- 172.21.15.36:0/2585326183 >> 172.21.15.36:0/2585326183 conn(0x7f64840614b0 msgr2=0x7f648413fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:01.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.518+0000 7f648974e700 1 -- 172.21.15.36:0/2585326183 shutdown_connections 2022-06-17T12:53:01.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.518+0000 7f648974e700 1 -- 172.21.15.36:0/2585326183 wait complete. 2022-06-17T12:53:01.506 INFO:teuthology.orchestra.run.smithi036.stderr: removed all blacklist entries 2022-06-17T12:53:01.518 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2022-06-17T12:53:01.518 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-06-17T12:53:01.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.642+0000 7f2b7c9dc700 1 Processor -- start 2022-06-17T12:53:01.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.642+0000 7f2b7c9dc700 1 -- start start 2022-06-17T12:53:01.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.646+0000 7f2b7c9dc700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2b78060ac0 0x7f2b780630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:01.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.646+0000 7f2b7c9dc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b780b1680 0x7f2b781431a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:01.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.646+0000 7f2b7c9dc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2b780ae6c0 0x7f2b78143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:01.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.646+0000 7f2b7c9dc700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b78062c80 con 0x7f2b780ae6c0 2022-06-17T12:53:01.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.646+0000 7f2b7c9dc700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b78060ea0 con 0x7f2b78060ac0 2022-06-17T12:53:01.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.646+0000 7f2b7c9dc700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b780b2f50 con 0x7f2b780b1680 2022-06-17T12:53:01.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.646+0000 7f2b7759e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2b780ae6c0 0x7f2b78143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:01.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.646+0000 7f2b7759e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2b780ae6c0 0x7f2b78143690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35388/0 (socket says 172.21.15.36:35388) 2022-06-17T12:53:01.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.646+0000 7f2b7759e700 1 -- 172.21.15.36:0/2084507459 learned_addr learned my addr 172.21.15.36:0/2084507459 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:01.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.646+0000 7f2b76d9d700 1 --2- 172.21.15.36:0/2084507459 >> v2:172.21.15.70:3300/0 conn(0x7f2b78060ac0 0x7f2b780630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:01.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.646+0000 7f2b7659c700 1 --2- 172.21.15.36:0/2084507459 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b780b1680 0x7f2b781431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:01.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.646+0000 7f2b7759e700 1 -- 172.21.15.36:0/2084507459 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b780b1680 msgr2=0x7f2b781431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:01.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.646+0000 7f2b7759e700 1 --2- 172.21.15.36:0/2084507459 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b780b1680 0x7f2b781431a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:01.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.646+0000 7f2b7759e700 1 -- 172.21.15.36:0/2084507459 >> v2:172.21.15.70:3300/0 conn(0x7f2b78060ac0 msgr2=0x7f2b780630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:01.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.646+0000 7f2b7759e700 1 --2- 172.21.15.36:0/2084507459 >> v2:172.21.15.70:3300/0 conn(0x7f2b78060ac0 0x7f2b780630e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:01.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.646+0000 7f2b7759e700 1 -- 172.21.15.36:0/2084507459 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b780635d0 con 0x7f2b780ae6c0 2022-06-17T12:53:01.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.646+0000 7f2b76d9d700 1 --2- 172.21.15.36:0/2084507459 >> v2:172.21.15.70:3300/0 conn(0x7f2b78060ac0 0x7f2b780630e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:53:01.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.646+0000 7f2b7759e700 1 --2- 172.21.15.36:0/2084507459 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b780ae6c0 0x7f2b78143690 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f2b700032f0 tx=0x7f2b70009290).ready entity=mon.0 client_cookie=50af3e80ffa56da6 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:01.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.646+0000 7f2b75d9b700 1 -- 172.21.15.36:0/2084507459 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b7000b680 con 0x7f2b780ae6c0 2022-06-17T12:53:01.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.646+0000 7f2b75d9b700 1 -- 172.21.15.36:0/2084507459 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b7000b7e0 con 0x7f2b780ae6c0 2022-06-17T12:53:01.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.646+0000 7f2b75d9b700 1 -- 172.21.15.36:0/2084507459 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b7000b950 con 0x7f2b780ae6c0 2022-06-17T12:53:01.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.650+0000 7f2b7c9dc700 1 -- 172.21.15.36:0/2084507459 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b780ae6c0 msgr2=0x7f2b78143690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:01.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.650+0000 7f2b7c9dc700 1 --2- 172.21.15.36:0/2084507459 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b780ae6c0 0x7f2b78143690 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f2b700032f0 tx=0x7f2b70009290).stop 2022-06-17T12:53:01.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.650+0000 7f2b7c9dc700 1 -- 172.21.15.36:0/2084507459 shutdown_connections 2022-06-17T12:53:01.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.650+0000 7f2b7c9dc700 1 --2- 172.21.15.36:0/2084507459 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b780ae6c0 0x7f2b78143690 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:01.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.650+0000 7f2b7c9dc700 1 --2- 172.21.15.36:0/2084507459 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b780b1680 0x7f2b781431a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:01.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.650+0000 7f2b7c9dc700 1 --2- 172.21.15.36:0/2084507459 >> v2:172.21.15.70:3300/0 conn(0x7f2b78060ac0 0x7f2b780630e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:01.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.650+0000 7f2b7c9dc700 1 -- 172.21.15.36:0/2084507459 >> 172.21.15.36:0/2084507459 conn(0x7f2b780b64f0 msgr2=0x7f2b780b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:01.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.650+0000 7f2b7c9dc700 1 -- 172.21.15.36:0/2084507459 shutdown_connections 2022-06-17T12:53:01.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.650+0000 7f2b7c9dc700 1 -- 172.21.15.36:0/2084507459 wait complete. 2022-06-17T12:53:01.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.650+0000 7f2b7c9dc700 1 Processor -- start 2022-06-17T12:53:01.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.650+0000 7f2b7c9dc700 1 -- start start 2022-06-17T12:53:01.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.650+0000 7f2b7c9dc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b78060ac0 0x7f2b781521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:01.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.650+0000 7f2b7c9dc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b780ae6c0 0x7f2b7813e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:01.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.650+0000 7f2b7c9dc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b780b1680 0x7f2b7813bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:01.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.650+0000 7f2b7c9dc700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b781533c0 con 0x7f2b780ae6c0 2022-06-17T12:53:01.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.650+0000 7f2b7c9dc700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7809e8d0 con 0x7f2b78060ac0 2022-06-17T12:53:01.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.650+0000 7f2b7c9dc700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7809eb70 con 0x7f2b780b1680 2022-06-17T12:53:01.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.654+0000 7f2b7659c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b780ae6c0 0x7f2b7813e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:01.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.654+0000 7f2b7659c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b780ae6c0 0x7f2b7813e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35392/0 (socket says 172.21.15.36:35392) 2022-06-17T12:53:01.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.654+0000 7f2b7659c700 1 -- 172.21.15.36:0/3478046905 learned_addr learned my addr 172.21.15.36:0/3478046905 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:01.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.654+0000 7f2b7759e700 1 --2- 172.21.15.36:0/3478046905 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b780b1680 0x7f2b7813bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:01.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.654+0000 7f2b76d9d700 1 --2- 172.21.15.36:0/3478046905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b78060ac0 0x7f2b781521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:01.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.654+0000 7f2b7659c700 1 -- 172.21.15.36:0/3478046905 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b780b1680 msgr2=0x7f2b7813bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:01.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.654+0000 7f2b7659c700 1 --2- 172.21.15.36:0/3478046905 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b780b1680 0x7f2b7813bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:01.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.654+0000 7f2b7659c700 1 -- 172.21.15.36:0/3478046905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b78060ac0 msgr2=0x7f2b781521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:01.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.654+0000 7f2b7659c700 1 --2- 172.21.15.36:0/3478046905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b78060ac0 0x7f2b781521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:01.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.654+0000 7f2b7659c700 1 -- 172.21.15.36:0/3478046905 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6800b180 con 0x7f2b780ae6c0 2022-06-17T12:53:01.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.654+0000 7f2b76d9d700 1 --2- 172.21.15.36:0/3478046905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b78060ac0 0x7f2b781521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:53:01.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.654+0000 7f2b7659c700 1 --2- 172.21.15.36:0/3478046905 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b780ae6c0 0x7f2b7813e2e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f2b6c00c000 tx=0x7f2b6c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:01.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.654+0000 7f2b67fff700 1 -- 172.21.15.36:0/3478046905 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b6c010780 con 0x7f2b780ae6c0 2022-06-17T12:53:01.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.654+0000 7f2b67fff700 1 -- 172.21.15.36:0/3478046905 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6c0108e0 con 0x7f2b780ae6c0 2022-06-17T12:53:01.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.654+0000 7f2b7c9dc700 1 -- 172.21.15.36:0/3478046905 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b70009530 con 0x7f2b780ae6c0 2022-06-17T12:53:01.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.654+0000 7f2b67fff700 1 -- 172.21.15.36:0/3478046905 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b6c010c40 con 0x7f2b780ae6c0 2022-06-17T12:53:01.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.654+0000 7f2b7c9dc700 1 -- 172.21.15.36:0/3478046905 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7809f0b0 con 0x7f2b780ae6c0 2022-06-17T12:53:01.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.654+0000 7f2b7c9dc700 1 -- 172.21.15.36:0/3478046905 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b7809f270 con 0x7f2b780ae6c0 2022-06-17T12:53:01.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.654+0000 7f2b67fff700 1 -- 172.21.15.36:0/3478046905 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2b6c01c460 con 0x7f2b780ae6c0 2022-06-17T12:53:01.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.658+0000 7f2b67fff700 1 --2- 172.21.15.36:0/3478046905 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b54053920 0x7f2b54055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:01.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.658+0000 7f2b67fff700 1 -- 172.21.15.36:0/3478046905 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f2b6c06c140 con 0x7f2b780ae6c0 2022-06-17T12:53:01.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.658+0000 7f2b76d9d700 1 --2- 172.21.15.36:0/3478046905 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b54053920 0x7f2b54055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:01.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.658+0000 7f2b76d9d700 1 --2- 172.21.15.36:0/3478046905 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b54053920 0x7f2b54055de0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f2b6800b000 tx=0x7f2b68008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:01.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.662+0000 7f2b67fff700 1 -- 172.21.15.36:0/3478046905 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2b6c043180 con 0x7f2b780ae6c0 2022-06-17T12:53:01.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.826+0000 7f2b7c9dc700 1 -- 172.21.15.36:0/3478046905 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2b780b2cb0 con 0x7f2b780ae6c0 2022-06-17T12:53:01.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.830+0000 7f2b67fff700 1 -- 172.21.15.36:0/3478046905 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f2b6c0467a0 con 0x7f2b780ae6c0 2022-06-17T12:53:01.813 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:53:01.814 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:53:01.814 INFO:teuthology.orchestra.run.smithi036.stdout: "auth_dump": [ 2022-06-17T12:53:01.814 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:53:01.815 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mds.a", 2022-06-17T12:53:01.815 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiyaUgARAAxp7bzOl4uVo2EsaoRBeMRw==", 2022-06-17T12:53:01.815 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:53:01.815 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:53:01.815 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:53:01.816 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:53:01.816 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:53:01.816 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:53:01.816 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:53:01.817 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:53:01.817 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mds.b", 2022-06-17T12:53:01.817 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xifyxLCRAABI389jmz1zboeDanzraT8Q==", 2022-06-17T12:53:01.818 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:53:01.818 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:53:01.818 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:53:01.818 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:53:01.819 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:53:01.819 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:53:01.819 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:53:01.819 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:53:01.820 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.0", 2022-06-17T12:53:01.820 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB+d6xibhXGNBAA6AuUT9ITWAZjDFpNsgA4Wg==", 2022-06-17T12:53:01.820 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:53:01.820 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:53:01.820 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:53:01.821 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:53:01.821 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:53:01.821 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:53:01.821 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:53:01.822 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.1", 2022-06-17T12:53:01.822 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCBd6xiOIPvGxAADkrBoJCy9x91zan6TWU0tQ==", 2022-06-17T12:53:01.822 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:53:01.822 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:53:01.823 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:53:01.823 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:53:01.823 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:53:01.823 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:53:01.824 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:53:01.824 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.2", 2022-06-17T12:53:01.824 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCEd6xiP4/9AhAAaP3cWXV6vbfGBPfqSLKqUg==", 2022-06-17T12:53:01.824 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:53:01.825 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:53:01.825 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:53:01.825 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:53:01.825 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:53:01.825 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:53:01.826 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:53:01.826 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.3", 2022-06-17T12:53:01.826 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCGd6xi+LZgJxAAJjGwP5qW6TFDZfR+2rz6ig==", 2022-06-17T12:53:01.827 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:53:01.827 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:53:01.827 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:53:01.828 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:53:01.828 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:53:01.828 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:53:01.828 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:53:01.829 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.4", 2022-06-17T12:53:01.829 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCKd6xikvrTChAAXIyWSETswDv+tb3zsGyYpA==", 2022-06-17T12:53:01.829 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:53:01.829 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:53:01.830 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:53:01.830 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:53:01.830 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:53:01.830 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:53:01.831 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:53:01.831 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.5", 2022-06-17T12:53:01.831 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCMd6xiqr2QLhAArhimHuNsVWrDGGreFYxj6A==", 2022-06-17T12:53:01.832 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:53:01.832 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:53:01.833 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:53:01.833 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:53:01.833 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:53:01.834 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:53:01.834 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:53:01.834 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.6", 2022-06-17T12:53:01.834 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCPd6xiUXcXGxAAdrYz8txdBWMIqF5DF4Aimw==", 2022-06-17T12:53:01.835 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:53:01.836 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:53:01.836 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:53:01.836 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:53:01.837 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:53:01.837 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:53:01.837 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:53:01.837 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.7", 2022-06-17T12:53:01.838 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCSd6xiBJ2IAhAAUkNfgmlxxWY7+VouPLAvyw==", 2022-06-17T12:53:01.839 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:53:01.839 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:53:01.840 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:53:01.840 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:53:01.841 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:53:01.841 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:53:01.842 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:53:01.842 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.0", 2022-06-17T12:53:01.843 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiXO3QFBAAsSLtQ0yKKZxhqUT/eYKdtg==", 2022-06-17T12:53:01.843 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:53:01.843 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:53:01.844 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:53:01.844 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:53:01.845 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:53:01.845 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:53:01.846 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:53:01.846 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.1", 2022-06-17T12:53:01.847 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xi51KdGBAAi7fGZ+Qepg6j3DAhlzoWxw==", 2022-06-17T12:53:01.847 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:53:01.847 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:53:01.848 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:53:01.848 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:53:01.849 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:53:01.849 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:53:01.849 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:53:01.849 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.2", 2022-06-17T12:53:01.850 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiug2hHBAAqHpLgwMdthSWtJ8Vwu+m1Q==", 2022-06-17T12:53:01.850 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:53:01.850 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:53:01.851 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:53:01.851 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:53:01.851 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:53:01.851 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:53:01.852 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:53:01.852 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.admin", 2022-06-17T12:53:01.852 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB5d6xiGDODKhAAPZMG/M2nVssdr4FJHcS6XQ==", 2022-06-17T12:53:01.852 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:53:01.853 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:53:01.853 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:53:01.853 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:53:01.854 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:53:01.854 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:53:01.854 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:53:01.855 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:53:01.855 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mgr.x", 2022-06-17T12:53:01.855 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB6d6xiyH9uORAATJb0chHE1CH0X02w7edVtw==", 2022-06-17T12:53:01.855 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:53:01.856 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:53:01.856 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile mgr", 2022-06-17T12:53:01.856 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:53:01.857 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:53:01.857 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:53:01.857 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:53:01.857 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mgr.y", 2022-06-17T12:53:01.858 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB6d6xiHXghMxAAzpsVRCbLWHWZPtrPk1jFlA==", 2022-06-17T12:53:01.858 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:53:01.858 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:53:01.858 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile mgr", 2022-06-17T12:53:01.859 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:53:01.859 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:53:01.859 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:53:01.859 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:53:01.860 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:53:01.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.830+0000 7f2b7c9dc700 1 -- 172.21.15.36:0/3478046905 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b54053920 msgr2=0x7f2b54055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:01.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.830+0000 7f2b7c9dc700 1 --2- 172.21.15.36:0/3478046905 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b54053920 0x7f2b54055de0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f2b6800b000 tx=0x7f2b68008040).stop 2022-06-17T12:53:01.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.830+0000 7f2b7c9dc700 1 -- 172.21.15.36:0/3478046905 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b780ae6c0 msgr2=0x7f2b7813e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:01.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.830+0000 7f2b7c9dc700 1 --2- 172.21.15.36:0/3478046905 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b780ae6c0 0x7f2b7813e2e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f2b6c00c000 tx=0x7f2b6c006df0).stop 2022-06-17T12:53:01.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.830+0000 7f2b7c9dc700 1 -- 172.21.15.36:0/3478046905 shutdown_connections 2022-06-17T12:53:01.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.830+0000 7f2b7c9dc700 1 --2- 172.21.15.36:0/3478046905 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b54053920 0x7f2b54055de0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:01.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.830+0000 7f2b7c9dc700 1 --2- 172.21.15.36:0/3478046905 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b780b1680 0x7f2b7813bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:01.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.830+0000 7f2b7c9dc700 1 --2- 172.21.15.36:0/3478046905 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b780ae6c0 0x7f2b7813e2e0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:01.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.830+0000 7f2b7c9dc700 1 --2- 172.21.15.36:0/3478046905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b78060ac0 0x7f2b781521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:01.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.830+0000 7f2b7c9dc700 1 -- 172.21.15.36:0/3478046905 >> 172.21.15.36:0/3478046905 conn(0x7f2b780b64f0 msgr2=0x7f2b780b1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:01.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.830+0000 7f2b7c9dc700 1 -- 172.21.15.36:0/3478046905 shutdown_connections 2022-06-17T12:53:01.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.830+0000 7f2b7c9dc700 1 -- 172.21.15.36:0/3478046905 wait complete. 2022-06-17T12:53:01.866 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:53:01.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.990+0000 7f8a560ad700 1 Processor -- start 2022-06-17T12:53:01.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.990+0000 7f8a560ad700 1 -- start start 2022-06-17T12:53:01.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.990+0000 7f8a560ad700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8a500b20f0 0x7f8a500aee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:01.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.990+0000 7f8a560ad700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8a500afe50 0x7f8a5013c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:01.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.990+0000 7f8a560ad700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a50060ea0 con 0x7f8a50060ac0 2022-06-17T12:53:01.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.990+0000 7f8a560ad700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a500b3060 con 0x7f8a500afe50 2022-06-17T12:53:01.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.990+0000 7f8a560ad700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a50067a20 con 0x7f8a500b20f0 2022-06-17T12:53:01.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.990+0000 7f8a558ac700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8a500afe50 0x7f8a5013c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:01.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.990+0000 7f8a558ac700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8a500afe50 0x7f8a5013c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35400/0 (socket says 172.21.15.36:35400) 2022-06-17T12:53:01.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.990+0000 7f8a558ac700 1 -- 172.21.15.36:0/1769549708 learned_addr learned my addr 172.21.15.36:0/1769549708 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:01.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.990+0000 7f8a548aa700 1 --2- 172.21.15.36:0/1769549708 >> v2:172.21.15.70:3300/0 conn(0x7f8a500b20f0 0x7f8a500aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:01.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.990+0000 7f8a558ac700 1 -- 172.21.15.36:0/1769549708 >> v2:172.21.15.70:3300/0 conn(0x7f8a500b20f0 msgr2=0x7f8a500aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:01.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.990+0000 7f8a558ac700 1 --2- 172.21.15.36:0/1769549708 >> v2:172.21.15.70:3300/0 conn(0x7f8a500b20f0 0x7f8a500aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:01.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.994+0000 7f8a558ac700 1 -- 172.21.15.36:0/1769549708 >> v1:172.21.15.70:6789/0 conn(0x7f8a50060ac0 legacy=0x7f8a500ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:01.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.994+0000 7f8a558ac700 1 -- 172.21.15.36:0/1769549708 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a50143fb0 con 0x7f8a500afe50 2022-06-17T12:53:01.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.994+0000 7f8a558ac700 1 --2- 172.21.15.36:0/1769549708 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8a500afe50 0x7f8a5013c7e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f8a480032f0 tx=0x7f8a48009290).ready entity=mon.0 client_cookie=48b400a9205ffd06 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:01.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.994+0000 7f8a43fff700 1 -- 172.21.15.36:0/1769549708 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a4800b870 con 0x7f8a500afe50 2022-06-17T12:53:01.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.994+0000 7f8a43fff700 1 -- 172.21.15.36:0/1769549708 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4800b9d0 con 0x7f8a500afe50 2022-06-17T12:53:01.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.994+0000 7f8a43fff700 1 -- 172.21.15.36:0/1769549708 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a4800bb40 con 0x7f8a500afe50 2022-06-17T12:53:01.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.994+0000 7f8a560ad700 1 -- 172.21.15.36:0/1769549708 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8a500afe50 msgr2=0x7f8a5013c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:01.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.994+0000 7f8a560ad700 1 --2- 172.21.15.36:0/1769549708 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8a500afe50 0x7f8a5013c7e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f8a480032f0 tx=0x7f8a48009290).stop 2022-06-17T12:53:01.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.994+0000 7f8a560ad700 1 -- 172.21.15.36:0/1769549708 shutdown_connections 2022-06-17T12:53:01.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.994+0000 7f8a560ad700 1 --2- 172.21.15.36:0/1769549708 >> v2:172.21.15.70:3300/0 conn(0x7f8a500b20f0 0x7f8a500aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:01.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.994+0000 7f8a560ad700 1 --2- 172.21.15.36:0/1769549708 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8a500afe50 0x7f8a5013c7e0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:01.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.994+0000 7f8a560ad700 1 -- 172.21.15.36:0/1769549708 >> 172.21.15.36:0/1769549708 conn(0x7f8a500b65f0 msgr2=0x7f8a500b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:01.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.994+0000 7f8a560ad700 1 -- 172.21.15.36:0/1769549708 shutdown_connections 2022-06-17T12:53:01.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.994+0000 7f8a560ad700 1 -- 172.21.15.36:0/1769549708 wait complete. 2022-06-17T12:53:01.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.994+0000 7f8a560ad700 1 Processor -- start 2022-06-17T12:53:01.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.994+0000 7f8a560ad700 1 -- start start 2022-06-17T12:53:01.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.998+0000 7f8a560ad700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a50060ac0 0x7f8a5013ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:01.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.998+0000 7f8a560ad700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a500afe50 0x7f8a5013fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:01.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.998+0000 7f8a560ad700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8a500b20f0 0x7f8a501521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:01.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.998+0000 7f8a560ad700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a50142ad0 con 0x7f8a500b20f0 2022-06-17T12:53:01.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.998+0000 7f8a560ad700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a50140680 con 0x7f8a50060ac0 2022-06-17T12:53:01.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.998+0000 7f8a560ad700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5013e580 con 0x7f8a500afe50 2022-06-17T12:53:01.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.998+0000 7f8a558ac700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8a500b20f0 0x7f8a501521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:01.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.998+0000 7f8a558ac700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8a500b20f0 0x7f8a501521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35408/0 (socket says 172.21.15.36:35408) 2022-06-17T12:53:01.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.998+0000 7f8a558ac700 1 -- 172.21.15.36:0/1974368840 learned_addr learned my addr 172.21.15.36:0/1974368840 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:01.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.998+0000 7f8a548aa700 1 --2- 172.21.15.36:0/1974368840 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a500afe50 0x7f8a5013fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:01.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.998+0000 7f8a550ab700 1 --2- 172.21.15.36:0/1974368840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a50060ac0 0x7f8a5013ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:01.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.998+0000 7f8a558ac700 1 -- 172.21.15.36:0/1974368840 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a500afe50 msgr2=0x7f8a5013fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:01.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.998+0000 7f8a558ac700 1 --2- 172.21.15.36:0/1974368840 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a500afe50 0x7f8a5013fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:01.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.998+0000 7f8a558ac700 1 -- 172.21.15.36:0/1974368840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a50060ac0 msgr2=0x7f8a5013ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:01.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.998+0000 7f8a558ac700 1 --2- 172.21.15.36:0/1974368840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a50060ac0 0x7f8a5013ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:01.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.998+0000 7f8a558ac700 1 -- 172.21.15.36:0/1974368840 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a48009530 con 0x7f8a500b20f0 2022-06-17T12:53:01.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.998+0000 7f8a550ab700 1 --2- 172.21.15.36:0/1974368840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a50060ac0 0x7f8a5013ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:53:01.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.998+0000 7f8a558ac700 1 --2- 172.21.15.36:0/1974368840 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8a500b20f0 0x7f8a501521d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7f8a480032f0 tx=0x7f8a48016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:01.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.998+0000 7f8a427fc700 1 -- 172.21.15.36:0/1974368840 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a4800b870 con 0x7f8a500b20f0 2022-06-17T12:53:01.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.998+0000 7f8a427fc700 1 -- 172.21.15.36:0/1974368840 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a48021440 con 0x7f8a500b20f0 2022-06-17T12:53:01.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.998+0000 7f8a560ad700 1 -- 172.21.15.36:0/1974368840 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a501408b0 con 0x7f8a500b20f0 2022-06-17T12:53:01.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.998+0000 7f8a427fc700 1 -- 172.21.15.36:0/1974368840 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a480217a0 con 0x7f8a500b20f0 2022-06-17T12:53:01.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:01.998+0000 7f8a560ad700 1 -- 172.21.15.36:0/1974368840 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a5013dab0 con 0x7f8a500b20f0 2022-06-17T12:53:01.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.002+0000 7f8a560ad700 1 -- 172.21.15.36:0/1974368840 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a50057410 con 0x7f8a500b20f0 2022-06-17T12:53:01.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.002+0000 7f8a427fc700 1 -- 172.21.15.36:0/1974368840 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8a48027020 con 0x7f8a500b20f0 2022-06-17T12:53:01.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.002+0000 7f8a427fc700 1 --2- 172.21.15.36:0/1974368840 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8a30053a40 0x7f8a30055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:02.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.002+0000 7f8a427fc700 1 -- 172.21.15.36:0/1974368840 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f8a4806d180 con 0x7f8a500b20f0 2022-06-17T12:53:02.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.002+0000 7f8a550ab700 1 --2- 172.21.15.36:0/1974368840 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8a30053a40 0x7f8a30055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:02.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.002+0000 7f8a550ab700 1 --2- 172.21.15.36:0/1974368840 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8a30053a40 0x7f8a30055f00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f8a4c00b000 tx=0x7f8a4c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:02.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.010+0000 7f8a427fc700 1 -- 172.21.15.36:0/1974368840 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8a480442c0 con 0x7f8a500b20f0 2022-06-17T12:53:02.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.138+0000 7f8a560ad700 1 -- 172.21.15.36:0/1974368840 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a50140460 con 0x7f8a500b20f0 2022-06-17T12:53:02.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.138+0000 7f8a427fc700 1 -- 172.21.15.36:0/1974368840 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+13415 (secure 0 0 0) 0x7f8a48015020 con 0x7f8a500b20f0 2022-06-17T12:53:02.124 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:53:02.125 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":70,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:53:01.507288+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:53:02.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.142+0000 7f8a560ad700 1 -- 172.21.15.36:0/1974368840 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8a30053a40 msgr2=0x7f8a30055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:02.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.142+0000 7f8a560ad700 1 --2- 172.21.15.36:0/1974368840 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8a30053a40 0x7f8a30055f00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f8a4c00b000 tx=0x7f8a4c009040).stop 2022-06-17T12:53:02.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.142+0000 7f8a560ad700 1 -- 172.21.15.36:0/1974368840 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8a500b20f0 msgr2=0x7f8a501521d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:02.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.142+0000 7f8a560ad700 1 --2- 172.21.15.36:0/1974368840 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8a500b20f0 0x7f8a501521d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7f8a480032f0 tx=0x7f8a48016000).stop 2022-06-17T12:53:02.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.142+0000 7f8a560ad700 1 -- 172.21.15.36:0/1974368840 shutdown_connections 2022-06-17T12:53:02.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.142+0000 7f8a560ad700 1 --2- 172.21.15.36:0/1974368840 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8a30053a40 0x7f8a30055f00 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.142+0000 7f8a560ad700 1 --2- 172.21.15.36:0/1974368840 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8a500b20f0 0x7f8a501521d0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.142+0000 7f8a560ad700 1 --2- 172.21.15.36:0/1974368840 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a500afe50 0x7f8a5013fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.142+0000 7f8a560ad700 1 --2- 172.21.15.36:0/1974368840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a50060ac0 0x7f8a5013ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.142+0000 7f8a560ad700 1 -- 172.21.15.36:0/1974368840 >> 172.21.15.36:0/1974368840 conn(0x7f8a500b65f0 msgr2=0x7f8a500ae6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:02.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.142+0000 7f8a560ad700 1 -- 172.21.15.36:0/1974368840 shutdown_connections 2022-06-17T12:53:02.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.142+0000 7f8a560ad700 1 -- 172.21.15.36:0/1974368840 wait complete. 2022-06-17T12:53:02.139 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-17T12:53:02.140 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:53:02.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.266+0000 7f7f8ffff700 1 Processor -- start 2022-06-17T12:53:02.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.266+0000 7f7f8ffff700 1 -- start start 2022-06-17T12:53:02.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.266+0000 7f7f8ffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7f900b9860 0x7f7f90143840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:02.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.266+0000 7f7f8ffff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f900b4160 con 0x7f7f900b5dc0 2022-06-17T12:53:02.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.266+0000 7f7f8ffff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f900b9c40 con 0x7f7f900b45c0 2022-06-17T12:53:02.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.266+0000 7f7f8ffff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f900b5120 con 0x7f7f900b9860 2022-06-17T12:53:02.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.266+0000 7f7f8f7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f7f900b5dc0 0x7f7f900b6210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52946/0 (socket says 172.21.15.36:52946) 2022-06-17T12:53:02.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.266+0000 7f7f8f7fe700 1 -- 172.21.15.36:0/1398661096 learned_addr learned my addr 172.21.15.36:0/1398661096 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:53:02.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.266+0000 7f7f8effd700 1 --2- 172.21.15.36:0/1398661096 >> v2:172.21.15.36:3300/0 conn(0x7f7f900b9860 0x7f7f90143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:02.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.270+0000 7f7f8dffb700 1 -- 172.21.15.36:0/1398661096 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1284435387 0 0) 0x7f7f900b4160 con 0x7f7f900b5dc0 2022-06-17T12:53:02.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.270+0000 7f7f8dffb700 1 -- 172.21.15.36:0/1398661096 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f70003260 con 0x7f7f900b5dc0 2022-06-17T12:53:02.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.270+0000 7f7f8dffb700 1 -- 172.21.15.36:0/1398661096 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924028744 0 0) 0x7f7f900b9c40 con 0x7f7f900b45c0 2022-06-17T12:53:02.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.270+0000 7f7f8dffb700 1 -- 172.21.15.36:0/1398661096 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f900b4160 con 0x7f7f900b45c0 2022-06-17T12:53:02.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.270+0000 7f7f8effd700 1 -- 172.21.15.36:0/1398661096 >> v1:172.21.15.70:6789/0 conn(0x7f7f900b45c0 legacy=0x7f7f900b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:02.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.270+0000 7f7f8effd700 1 -- 172.21.15.36:0/1398661096 >> v1:172.21.15.36:6789/0 conn(0x7f7f900b5dc0 legacy=0x7f7f900b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:02.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.270+0000 7f7f8effd700 1 -- 172.21.15.36:0/1398661096 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f900627d0 con 0x7f7f900b9860 2022-06-17T12:53:02.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.270+0000 7f7f8effd700 1 --2- 172.21.15.36:0/1398661096 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7f900b9860 0x7f7f90143840 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f7f80002f70 tx=0x7f7f80006df0).ready entity=mon.0 client_cookie=d0a559f0ecd6bfd server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:02.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.270+0000 7f7f8dffb700 1 -- 172.21.15.36:0/1398661096 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f8000d540 con 0x7f7f900b9860 2022-06-17T12:53:02.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.270+0000 7f7f8dffb700 1 -- 172.21.15.36:0/1398661096 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f80000e90 con 0x7f7f900b9860 2022-06-17T12:53:02.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.270+0000 7f7f8dffb700 1 -- 172.21.15.36:0/1398661096 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f8000d890 con 0x7f7f900b9860 2022-06-17T12:53:02.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.270+0000 7f7f8ffff700 1 -- 172.21.15.36:0/1398661096 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7f900b9860 msgr2=0x7f7f90143840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:02.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.270+0000 7f7f8ffff700 1 --2- 172.21.15.36:0/1398661096 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7f900b9860 0x7f7f90143840 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f7f80002f70 tx=0x7f7f80006df0).stop 2022-06-17T12:53:02.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.270+0000 7f7f8ffff700 1 -- 172.21.15.36:0/1398661096 shutdown_connections 2022-06-17T12:53:02.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.270+0000 7f7f8ffff700 1 --2- 172.21.15.36:0/1398661096 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7f900b9860 0x7f7f90143840 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.270+0000 7f7f8ffff700 1 -- 172.21.15.36:0/1398661096 >> 172.21.15.36:0/1398661096 conn(0x7f7f9005c550 msgr2=0x7f7f90060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:02.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.270+0000 7f7f8ffff700 1 -- 172.21.15.36:0/1398661096 shutdown_connections 2022-06-17T12:53:02.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.270+0000 7f7f8ffff700 1 -- 172.21.15.36:0/1398661096 wait complete. 2022-06-17T12:53:02.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.270+0000 7f7f8ffff700 1 Processor -- start 2022-06-17T12:53:02.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.270+0000 7f7f8ffff700 1 -- start start 2022-06-17T12:53:02.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.274+0000 7f7f8ffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f900b45c0 0x7f7f9013e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:02.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.274+0000 7f7f8ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7f900b5dc0 0x7f7f9013d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:02.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.274+0000 7f7f8ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f900b9860 0x7f7f900ad330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:02.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.274+0000 7f7f8ffff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9013cef0 con 0x7f7f900b5dc0 2022-06-17T12:53:02.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.274+0000 7f7f8ffff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9013e560 con 0x7f7f900b9860 2022-06-17T12:53:02.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.274+0000 7f7f8ffff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9013d120 con 0x7f7f900b45c0 2022-06-17T12:53:02.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.274+0000 7f7f8e7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7f900b5dc0 0x7f7f9013d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:02.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.274+0000 7f7f8e7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7f900b5dc0 0x7f7f9013d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35418/0 (socket says 172.21.15.36:35418) 2022-06-17T12:53:02.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.274+0000 7f7f8e7fc700 1 -- 172.21.15.36:0/2418629941 learned_addr learned my addr 172.21.15.36:0/2418629941 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:02.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.274+0000 7f7f8effd700 1 --2- 172.21.15.36:0/2418629941 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f900b45c0 0x7f7f9013e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:02.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.274+0000 7f7f8f7fe700 1 --2- 172.21.15.36:0/2418629941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f900b9860 0x7f7f900ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:02.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.274+0000 7f7f8e7fc700 1 -- 172.21.15.36:0/2418629941 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f900b45c0 msgr2=0x7f7f9013e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:02.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.274+0000 7f7f8e7fc700 1 --2- 172.21.15.36:0/2418629941 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f900b45c0 0x7f7f9013e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.274+0000 7f7f8e7fc700 1 -- 172.21.15.36:0/2418629941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f900b9860 msgr2=0x7f7f900ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:02.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.274+0000 7f7f8e7fc700 1 --2- 172.21.15.36:0/2418629941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f900b9860 0x7f7f900ad330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.274+0000 7f7f8e7fc700 1 -- 172.21.15.36:0/2418629941 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8000b180 con 0x7f7f900b5dc0 2022-06-17T12:53:02.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.274+0000 7f7f8f7fe700 1 --2- 172.21.15.36:0/2418629941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f900b9860 0x7f7f900ad330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:53:02.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.274+0000 7f7f8e7fc700 1 --2- 172.21.15.36:0/2418629941 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7f900b5dc0 0x7f7f9013d8b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f7f78001110 tx=0x7f7f78007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:02.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.274+0000 7f7f87fff700 1 -- 172.21.15.36:0/2418629941 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f7800ce50 con 0x7f7f900b5dc0 2022-06-17T12:53:02.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.274+0000 7f7f8ffff700 1 -- 172.21.15.36:0/2418629941 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f90152ce0 con 0x7f7f900b5dc0 2022-06-17T12:53:02.278 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.274+0000 7f7f87fff700 1 -- 172.21.15.36:0/2418629941 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7800f470 con 0x7f7f900b5dc0 2022-06-17T12:53:02.278 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.274+0000 7f7f8ffff700 1 -- 172.21.15.36:0/2418629941 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f9013de20 con 0x7f7f900b5dc0 2022-06-17T12:53:02.278 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.274+0000 7f7f87fff700 1 -- 172.21.15.36:0/2418629941 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f7800f7d0 con 0x7f7f900b5dc0 2022-06-17T12:53:02.278 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.278+0000 7f7f8ffff700 1 -- 172.21.15.36:0/2418629941 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f90062a70 con 0x7f7f900b5dc0 2022-06-17T12:53:02.279 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.278+0000 7f7f87fff700 1 -- 172.21.15.36:0/2418629941 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f7f78019030 con 0x7f7f900b5dc0 2022-06-17T12:53:02.279 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.278+0000 7f7f87fff700 1 --2- 172.21.15.36:0/2418629941 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7f74053a40 0x7f7f74055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:02.279 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.278+0000 7f7f87fff700 1 -- 172.21.15.36:0/2418629941 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f7f7806b060 con 0x7f7f900b5dc0 2022-06-17T12:53:02.279 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.278+0000 7f7f8effd700 1 --2- 172.21.15.36:0/2418629941 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7f74053a40 0x7f7f74055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:02.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.278+0000 7f7f8effd700 1 --2- 172.21.15.36:0/2418629941 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7f74053a40 0x7f7f74055f00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f7f8001a000 tx=0x7f7f80014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:02.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.286+0000 7f7f87fff700 1 -- 172.21.15.36:0/2418629941 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7f780421b0 con 0x7f7f900b5dc0 2022-06-17T12:53:02.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.414+0000 7f7f8ffff700 1 -- 172.21.15.36:0/2418629941 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7f90057410 con 0x7f7f900b5dc0 2022-06-17T12:53:02.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.414+0000 7f7f87fff700 1 -- 172.21.15.36:0/2418629941 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v70) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7f780457d0 con 0x7f7f900b5dc0 2022-06-17T12:53:02.399 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:53:02.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.414+0000 7f7f8ffff700 1 -- 172.21.15.36:0/2418629941 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7f74053a40 msgr2=0x7f7f74055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:02.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.414+0000 7f7f8ffff700 1 --2- 172.21.15.36:0/2418629941 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7f74053a40 0x7f7f74055f00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f7f8001a000 tx=0x7f7f80014000).stop 2022-06-17T12:53:02.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.414+0000 7f7f8ffff700 1 -- 172.21.15.36:0/2418629941 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7f900b5dc0 msgr2=0x7f7f9013d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:02.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.414+0000 7f7f8ffff700 1 --2- 172.21.15.36:0/2418629941 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7f900b5dc0 0x7f7f9013d8b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f7f78001110 tx=0x7f7f78007ae0).stop 2022-06-17T12:53:02.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.414+0000 7f7f8ffff700 1 -- 172.21.15.36:0/2418629941 shutdown_connections 2022-06-17T12:53:02.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.414+0000 7f7f8ffff700 1 --2- 172.21.15.36:0/2418629941 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7f74053a40 0x7f7f74055f00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.414+0000 7f7f8ffff700 1 --2- 172.21.15.36:0/2418629941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f900b9860 0x7f7f900ad330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.414+0000 7f7f8ffff700 1 --2- 172.21.15.36:0/2418629941 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7f900b5dc0 0x7f7f9013d8b0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.414+0000 7f7f8ffff700 1 --2- 172.21.15.36:0/2418629941 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f900b45c0 0x7f7f9013e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.414+0000 7f7f8ffff700 1 -- 172.21.15.36:0/2418629941 >> 172.21.15.36:0/2418629941 conn(0x7f7f9005c550 msgr2=0x7f7f900b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:02.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.414+0000 7f7f8ffff700 1 -- 172.21.15.36:0/2418629941 shutdown_connections 2022-06-17T12:53:02.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.414+0000 7f7f8ffff700 1 -- 172.21.15.36:0/2418629941 wait complete. 2022-06-17T12:53:02.412 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:53:02.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.526+0000 7f0ed7d8a700 1 Processor -- start 2022-06-17T12:53:02.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.526+0000 7f0ed7d8a700 1 -- start start 2022-06-17T12:53:02.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.526+0000 7f0ed7d8a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed00acee0 0x7f0ed00b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:02.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.526+0000 7f0ed7d8a700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed00b1900 con 0x7f0ed0144240 2022-06-17T12:53:02.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.526+0000 7f0ed7d8a700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed00af370 con 0x7f0ed00acb00 2022-06-17T12:53:02.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.526+0000 7f0ed7d8a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed00ad360 con 0x7f0ed00acee0 2022-06-17T12:53:02.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.526+0000 7f0ed6327700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f0ed0144240 0x7f0ed01446d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52958/0 (socket says 172.21.15.36:52958) 2022-06-17T12:53:02.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.526+0000 7f0ed6327700 1 -- 172.21.15.36:0/492896019 learned_addr learned my addr 172.21.15.36:0/492896019 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:53:02.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.526+0000 7f0ed5325700 1 --2- 172.21.15.36:0/492896019 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed00acee0 0x7f0ed00b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:02.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.526+0000 7f0ed4b24700 1 -- 172.21.15.36:0/492896019 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 591436726 0 0) 0x7f0ed00b1900 con 0x7f0ed0144240 2022-06-17T12:53:02.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.526+0000 7f0ed4b24700 1 -- 172.21.15.36:0/492896019 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ebc003260 con 0x7f0ed0144240 2022-06-17T12:53:02.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.526+0000 7f0ed4b24700 1 -- 172.21.15.36:0/492896019 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993334824 0 0) 0x7f0ed00af370 con 0x7f0ed00acb00 2022-06-17T12:53:02.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.526+0000 7f0ed4b24700 1 -- 172.21.15.36:0/492896019 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ed00b1900 con 0x7f0ed00acb00 2022-06-17T12:53:02.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.526+0000 7f0ed5325700 1 -- 172.21.15.36:0/492896019 >> v1:172.21.15.70:6789/0 conn(0x7f0ed00acb00 legacy=0x7f0ed0147480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:02.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.526+0000 7f0ed4b24700 1 -- 172.21.15.36:0/492896019 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 558472552 0 0) 0x7f0ebc003260 con 0x7f0ed0144240 2022-06-17T12:53:02.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.526+0000 7f0ed5325700 1 -- 172.21.15.36:0/492896019 >> v1:172.21.15.36:6789/0 conn(0x7f0ed0144240 legacy=0x7f0ed01446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:02.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.526+0000 7f0ed5325700 1 -- 172.21.15.36:0/492896019 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed00b1b30 con 0x7f0ed00acee0 2022-06-17T12:53:02.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.526+0000 7f0ed5325700 1 --2- 172.21.15.36:0/492896019 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed00acee0 0x7f0ed00b05e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f0ec0001800 tx=0x7f0ec4006df0).ready entity=mon.2 client_cookie=2a5a6b0682e7f21f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:02.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.526+0000 7f0ed4b24700 1 -- 172.21.15.36:0/492896019 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ec0001660 con 0x7f0ed00acee0 2022-06-17T12:53:02.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.526+0000 7f0ed4b24700 1 -- 172.21.15.36:0/492896019 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec4000e90 con 0x7f0ed00acee0 2022-06-17T12:53:02.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.526+0000 7f0ed4b24700 1 -- 172.21.15.36:0/492896019 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ec4012460 con 0x7f0ed00acee0 2022-06-17T12:53:02.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.526+0000 7f0ed7d8a700 1 -- 172.21.15.36:0/492896019 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed00acee0 msgr2=0x7f0ed00b05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:02.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.526+0000 7f0ed7d8a700 1 --2- 172.21.15.36:0/492896019 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed00acee0 0x7f0ed00b05e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f0ec0001800 tx=0x7f0ec4006df0).stop 2022-06-17T12:53:02.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.530+0000 7f0ed7d8a700 1 -- 172.21.15.36:0/492896019 shutdown_connections 2022-06-17T12:53:02.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.530+0000 7f0ed7d8a700 1 --2- 172.21.15.36:0/492896019 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed00acee0 0x7f0ed00b05e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.530+0000 7f0ed7d8a700 1 -- 172.21.15.36:0/492896019 >> 172.21.15.36:0/492896019 conn(0x7f0ed005c550 msgr2=0x7f0ed00b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:02.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.530+0000 7f0ed7d8a700 1 -- 172.21.15.36:0/492896019 shutdown_connections 2022-06-17T12:53:02.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.530+0000 7f0ed7d8a700 1 -- 172.21.15.36:0/492896019 wait complete. 2022-06-17T12:53:02.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.530+0000 7f0ed7d8a700 1 Processor -- start 2022-06-17T12:53:02.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.530+0000 7f0ed7d8a700 1 -- start start 2022-06-17T12:53:02.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.530+0000 7f0ed7d8a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ed00acb00 0x7f0ed0062860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:02.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.530+0000 7f0ed7d8a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed00acee0 0x7f0ed0156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:02.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.530+0000 7f0ed7d8a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ed0144240 0x7f0ed01572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:02.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.530+0000 7f0ed7d8a700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed00ac540 con 0x7f0ed00acb00 2022-06-17T12:53:02.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.530+0000 7f0ed7d8a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed0156740 con 0x7f0ed0144240 2022-06-17T12:53:02.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.530+0000 7f0ed7d8a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed0066840 con 0x7f0ed00acee0 2022-06-17T12:53:02.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.530+0000 7f0ed5b26700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ed00acb00 0x7f0ed0062860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:02.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.530+0000 7f0ed5325700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed00acee0 0x7f0ed0156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:02.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.530+0000 7f0ed5b26700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ed00acb00 0x7f0ed0062860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35432/0 (socket says 172.21.15.36:35432) 2022-06-17T12:53:02.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.530+0000 7f0ed5b26700 1 -- 172.21.15.36:0/3150611075 learned_addr learned my addr 172.21.15.36:0/3150611075 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:02.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.534+0000 7f0ed6327700 1 --2- 172.21.15.36:0/3150611075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ed0144240 0x7f0ed01572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:02.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.534+0000 7f0ed5325700 1 -- 172.21.15.36:0/3150611075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ed0144240 msgr2=0x7f0ed01572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:02.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.534+0000 7f0ed5325700 1 --2- 172.21.15.36:0/3150611075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ed0144240 0x7f0ed01572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.534+0000 7f0ed5325700 1 -- 172.21.15.36:0/3150611075 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ed00acb00 msgr2=0x7f0ed0062860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:02.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.534+0000 7f0ed5325700 1 --2- 172.21.15.36:0/3150611075 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ed00acb00 0x7f0ed0062860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.534+0000 7f0ed5325700 1 -- 172.21.15.36:0/3150611075 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec400b180 con 0x7f0ed00acee0 2022-06-17T12:53:02.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.534+0000 7f0ed5325700 1 --2- 172.21.15.36:0/3150611075 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed00acee0 0x7f0ed0156dd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f0ec0001800 tx=0x7f0ec4014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:02.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.534+0000 7f0eceffd700 1 -- 172.21.15.36:0/3150611075 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ec400d8a0 con 0x7f0ed00acee0 2022-06-17T12:53:02.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.534+0000 7f0eceffd700 1 -- 172.21.15.36:0/3150611075 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec400da00 con 0x7f0ed00acee0 2022-06-17T12:53:02.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.534+0000 7f0ed7d8a700 1 -- 172.21.15.36:0/3150611075 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed0156930 con 0x7f0ed00acee0 2022-06-17T12:53:02.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.534+0000 7f0eceffd700 1 -- 172.21.15.36:0/3150611075 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ec400dd60 con 0x7f0ed00acee0 2022-06-17T12:53:02.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.534+0000 7f0ed7d8a700 1 -- 172.21.15.36:0/3150611075 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed0066510 con 0x7f0ed00acee0 2022-06-17T12:53:02.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.534+0000 7f0ed7d8a700 1 -- 172.21.15.36:0/3150611075 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed0156ac0 con 0x7f0ed00acee0 2022-06-17T12:53:02.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.534+0000 7f0eceffd700 1 -- 172.21.15.36:0/3150611075 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f0ec4025030 con 0x7f0ed00acee0 2022-06-17T12:53:02.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.534+0000 7f0eceffd700 1 --2- 172.21.15.36:0/3150611075 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0eb00539b0 0x7f0eb0055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:02.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.534+0000 7f0eceffd700 1 -- 172.21.15.36:0/3150611075 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f0ec406c6f0 con 0x7f0ed00acee0 2022-06-17T12:53:02.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.534+0000 7f0ed5b26700 1 --2- 172.21.15.36:0/3150611075 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0eb00539b0 0x7f0eb0055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:02.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.538+0000 7f0ed5b26700 1 --2- 172.21.15.36:0/3150611075 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0eb00539b0 0x7f0eb0055e70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f0ec800d000 tx=0x7f0ec8007e60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:02.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.542+0000 7f0eceffd700 1 -- 172.21.15.36:0/3150611075 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0ec4043830 con 0x7f0ed00acee0 2022-06-17T12:53:02.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.670+0000 7f0ed7d8a700 1 -- 172.21.15.36:0/3150611075 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0ed0057410 con 0x7f0ed00acee0 2022-06-17T12:53:02.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.670+0000 7f0eceffd700 1 -- 172.21.15.36:0/3150611075 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v70) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0ec40230a0 con 0x7f0ed00acee0 2022-06-17T12:53:02.657 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:53:02.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.674+0000 7f0ed7d8a700 1 -- 172.21.15.36:0/3150611075 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0eb00539b0 msgr2=0x7f0eb0055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:02.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.674+0000 7f0ed7d8a700 1 --2- 172.21.15.36:0/3150611075 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0eb00539b0 0x7f0eb0055e70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f0ec800d000 tx=0x7f0ec8007e60).stop 2022-06-17T12:53:02.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.674+0000 7f0ed7d8a700 1 -- 172.21.15.36:0/3150611075 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed00acee0 msgr2=0x7f0ed0156dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:02.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.674+0000 7f0ed7d8a700 1 --2- 172.21.15.36:0/3150611075 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed00acee0 0x7f0ed0156dd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f0ec0001800 tx=0x7f0ec4014000).stop 2022-06-17T12:53:02.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.674+0000 7f0ed7d8a700 1 -- 172.21.15.36:0/3150611075 shutdown_connections 2022-06-17T12:53:02.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.674+0000 7f0ed7d8a700 1 --2- 172.21.15.36:0/3150611075 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0eb00539b0 0x7f0eb0055e70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.674+0000 7f0ed7d8a700 1 --2- 172.21.15.36:0/3150611075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ed0144240 0x7f0ed01572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.674+0000 7f0ed7d8a700 1 --2- 172.21.15.36:0/3150611075 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed00acee0 0x7f0ed0156dd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.674+0000 7f0ed7d8a700 1 --2- 172.21.15.36:0/3150611075 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ed00acb00 0x7f0ed0062860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.674+0000 7f0ed7d8a700 1 -- 172.21.15.36:0/3150611075 >> 172.21.15.36:0/3150611075 conn(0x7f0ed005c550 msgr2=0x7f0ed00b51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:02.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.674+0000 7f0ed7d8a700 1 -- 172.21.15.36:0/3150611075 shutdown_connections 2022-06-17T12:53:02.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.674+0000 7f0ed7d8a700 1 -- 172.21.15.36:0/3150611075 wait complete. 2022-06-17T12:53:02.671 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-17T12:53:02.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a9662700 1 Processor -- start 2022-06-17T12:53:02.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a9662700 1 -- start start 2022-06-17T12:53:02.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a9662700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff7a4064f70 0x7ff7a40b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:02.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a9662700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff7a414a8f0 0x7ff7a4146790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:02.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a9662700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a4063b10 con 0x7ff7a4063730 2022-06-17T12:53:02.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a9662700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7a414ad20 con 0x7ff7a4064f70 2022-06-17T12:53:02.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a9662700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a40b2e80 con 0x7ff7a414a8f0 2022-06-17T12:53:02.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a3fff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7ff7a4063730 0x7ff7a4064860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:52972/0 (socket says 172.21.15.36:52972) 2022-06-17T12:53:02.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a3fff700 1 -- 172.21.15.36:0/3638449904 learned_addr learned my addr 172.21.15.36:0/3638449904 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:53:02.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a37fe700 1 --2- 172.21.15.36:0/3638449904 >> v2:172.21.15.70:3300/0 conn(0x7ff7a4064f70 0x7ff7a40b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:02.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a8e61700 1 --2- 172.21.15.36:0/3638449904 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff7a414a8f0 0x7ff7a4146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:02.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a2ffd700 1 -- 172.21.15.36:0/3638449904 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1284483571 0 0) 0x7ff7a4063b10 con 0x7ff7a4063730 2022-06-17T12:53:02.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a2ffd700 1 -- 172.21.15.36:0/3638449904 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff790003260 con 0x7ff7a4063730 2022-06-17T12:53:02.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a37fe700 1 -- 172.21.15.36:0/3638449904 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff7a414a8f0 msgr2=0x7ff7a4146790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:02.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a37fe700 1 --2- 172.21.15.36:0/3638449904 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff7a414a8f0 0x7ff7a4146790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a37fe700 1 -- 172.21.15.36:0/3638449904 >> v1:172.21.15.36:6789/0 conn(0x7ff7a4063730 legacy=0x7ff7a4064860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:02.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a37fe700 1 -- 172.21.15.36:0/3638449904 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a405ffb0 con 0x7ff7a4064f70 2022-06-17T12:53:02.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a37fe700 1 --2- 172.21.15.36:0/3638449904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7a4064f70 0x7ff7a40b3bf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7ff798002f70 tx=0x7ff798006df0).ready entity=mon.1 client_cookie=e9068c69b5c2d78 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:02.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a2ffd700 1 -- 172.21.15.36:0/3638449904 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff79800d4a0 con 0x7ff7a4064f70 2022-06-17T12:53:02.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a2ffd700 1 -- 172.21.15.36:0/3638449904 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff798000e90 con 0x7ff7a4064f70 2022-06-17T12:53:02.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a2ffd700 1 -- 172.21.15.36:0/3638449904 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff79800d7f0 con 0x7ff7a4064f70 2022-06-17T12:53:02.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a9662700 1 -- 172.21.15.36:0/3638449904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7a4064f70 msgr2=0x7ff7a40b3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:02.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a9662700 1 --2- 172.21.15.36:0/3638449904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7a4064f70 0x7ff7a40b3bf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7ff798002f70 tx=0x7ff798006df0).stop 2022-06-17T12:53:02.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a9662700 1 -- 172.21.15.36:0/3638449904 shutdown_connections 2022-06-17T12:53:02.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a9662700 1 --2- 172.21.15.36:0/3638449904 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff7a414a8f0 0x7ff7a4146790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a9662700 1 --2- 172.21.15.36:0/3638449904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7a4064f70 0x7ff7a40b3bf0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a9662700 1 -- 172.21.15.36:0/3638449904 >> 172.21.15.36:0/3638449904 conn(0x7ff7a40614b0 msgr2=0x7ff7a40618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:02.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a9662700 1 -- 172.21.15.36:0/3638449904 shutdown_connections 2022-06-17T12:53:02.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.794+0000 7ff7a9662700 1 -- 172.21.15.36:0/3638449904 wait complete. 2022-06-17T12:53:02.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a9662700 1 Processor -- start 2022-06-17T12:53:02.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a9662700 1 -- start start 2022-06-17T12:53:02.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a9662700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff7a4063730 0x7ff7a414e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:02.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a9662700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7a4064f70 0x7ff7a4156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:02.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a9662700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff7a414a8f0 0x7ff7a415a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:02.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a9662700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a410f8a0 con 0x7ff7a414a8f0 2022-06-17T12:53:02.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a9662700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a40b0b90 con 0x7ff7a4064f70 2022-06-17T12:53:02.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a9662700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a4159c50 con 0x7ff7a4063730 2022-06-17T12:53:02.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a8e61700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff7a414a8f0 0x7ff7a415a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:02.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7a4064f70 0x7ff7a4156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:02.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a8e61700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff7a414a8f0 0x7ff7a415a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35450/0 (socket says 172.21.15.36:35450) 2022-06-17T12:53:02.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7a4064f70 0x7ff7a4156700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:55828/0 (socket says 172.21.15.36:55828) 2022-06-17T12:53:02.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a37fe700 1 -- 172.21.15.36:0/1435223200 learned_addr learned my addr 172.21.15.36:0/1435223200 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:02.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a3fff700 1 --2- 172.21.15.36:0/1435223200 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff7a4063730 0x7ff7a414e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:02.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a37fe700 1 -- 172.21.15.36:0/1435223200 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff7a4063730 msgr2=0x7ff7a414e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:02.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a37fe700 1 --2- 172.21.15.36:0/1435223200 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff7a4063730 0x7ff7a414e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a37fe700 1 -- 172.21.15.36:0/1435223200 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff7a414a8f0 msgr2=0x7ff7a415a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:02.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a37fe700 1 --2- 172.21.15.36:0/1435223200 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff7a414a8f0 0x7ff7a415a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a37fe700 1 -- 172.21.15.36:0/1435223200 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79800b180 con 0x7ff7a4064f70 2022-06-17T12:53:02.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a37fe700 1 --2- 172.21.15.36:0/1435223200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7a4064f70 0x7ff7a4156700 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7ff798002f70 tx=0x7ff798014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:02.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a17fa700 1 -- 172.21.15.36:0/1435223200 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff798016070 con 0x7ff7a4064f70 2022-06-17T12:53:02.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a17fa700 1 -- 172.21.15.36:0/1435223200 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7980137d0 con 0x7ff7a4064f70 2022-06-17T12:53:02.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a9662700 1 -- 172.21.15.36:0/1435223200 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a410fad0 con 0x7ff7a4064f70 2022-06-17T12:53:02.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a17fa700 1 -- 172.21.15.36:0/1435223200 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff798013b30 con 0x7ff7a4064f70 2022-06-17T12:53:02.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.798+0000 7ff7a9662700 1 -- 172.21.15.36:0/1435223200 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a414e730 con 0x7ff7a4064f70 2022-06-17T12:53:02.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.802+0000 7ff7a9662700 1 -- 172.21.15.36:0/1435223200 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a4156d90 con 0x7ff7a4064f70 2022-06-17T12:53:02.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.802+0000 7ff7a17fa700 1 -- 172.21.15.36:0/1435223200 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff798025030 con 0x7ff7a4064f70 2022-06-17T12:53:02.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.802+0000 7ff7a17fa700 1 --2- 172.21.15.36:0/1435223200 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff784053a40 0x7ff784055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:02.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.802+0000 7ff7a17fa700 1 -- 172.21.15.36:0/1435223200 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5600+0+0 (secure 0 0 0) 0x7ff79806cbe0 con 0x7ff7a4064f70 2022-06-17T12:53:02.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.802+0000 7ff7a3fff700 1 --2- 172.21.15.36:0/1435223200 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff784053a40 0x7ff784055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:02.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.802+0000 7ff7a3fff700 1 --2- 172.21.15.36:0/1435223200 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff784053a40 0x7ff784055f00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7ff79400d000 tx=0x7ff794009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:02.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.810+0000 7ff7a17fa700 1 -- 172.21.15.36:0/1435223200 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff798043c10 con 0x7ff7a4064f70 2022-06-17T12:53:02.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.938+0000 7ff7a9662700 1 -- 172.21.15.36:0/1435223200 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff7a4057410 con 0x7ff7a4064f70 2022-06-17T12:53:02.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.938+0000 7ff7a17fa700 1 -- 172.21.15.36:0/1435223200 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) v1 ==== 84+0+143 (secure 0 0 0) 0x7ff798023070 con 0x7ff7a4064f70 2022-06-17T12:53:02.925 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:53:02.926 INFO:teuthology.orchestra.run.smithi036.stdout:[ 2022-06-17T12:53:02.926 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:53:02.927 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 1, 2022-06-17T12:53:02.927 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "device_health_metrics" 2022-06-17T12:53:02.928 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:53:02.928 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:53:02.928 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 2, 2022-06-17T12:53:02.928 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "rbd" 2022-06-17T12:53:02.929 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:53:02.929 INFO:teuthology.orchestra.run.smithi036.stdout:] 2022-06-17T12:53:02.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.942+0000 7ff7a9662700 1 -- 172.21.15.36:0/1435223200 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff784053a40 msgr2=0x7ff784055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:02.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.942+0000 7ff7a9662700 1 --2- 172.21.15.36:0/1435223200 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff784053a40 0x7ff784055f00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7ff79400d000 tx=0x7ff794009040).stop 2022-06-17T12:53:02.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.942+0000 7ff7a9662700 1 -- 172.21.15.36:0/1435223200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7a4064f70 msgr2=0x7ff7a4156700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:02.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.942+0000 7ff7a9662700 1 --2- 172.21.15.36:0/1435223200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7a4064f70 0x7ff7a4156700 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7ff798002f70 tx=0x7ff798014000).stop 2022-06-17T12:53:02.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.942+0000 7ff7a9662700 1 -- 172.21.15.36:0/1435223200 shutdown_connections 2022-06-17T12:53:02.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.942+0000 7ff7a9662700 1 --2- 172.21.15.36:0/1435223200 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff784053a40 0x7ff784055f00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.942+0000 7ff7a9662700 1 --2- 172.21.15.36:0/1435223200 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff7a414a8f0 0x7ff7a415a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.942+0000 7ff7a9662700 1 --2- 172.21.15.36:0/1435223200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7a4064f70 0x7ff7a4156700 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.942+0000 7ff7a9662700 1 --2- 172.21.15.36:0/1435223200 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff7a4063730 0x7ff7a414e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:02.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.942+0000 7ff7a9662700 1 -- 172.21.15.36:0/1435223200 >> 172.21.15.36:0/1435223200 conn(0x7ff7a40614b0 msgr2=0x7ff7a405cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:02.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.942+0000 7ff7a9662700 1 -- 172.21.15.36:0/1435223200 shutdown_connections 2022-06-17T12:53:02.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:02.942+0000 7ff7a9662700 1 -- 172.21.15.36:0/1435223200 wait complete. 2022-06-17T12:53:02.937 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-17T12:53:02.939 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-17T12:53:03.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.066+0000 7ff925fd9700 1 Processor -- start 2022-06-17T12:53:03.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.066+0000 7ff925fd9700 1 -- start start 2022-06-17T12:53:03.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.066+0000 7ff925fd9700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff920060020 0x7ff9200626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:03.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.066+0000 7ff925fd9700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff92013ca70 0x7ff92013ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:03.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.066+0000 7ff925fd9700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9201432c0 con 0x7ff9200b3f80 2022-06-17T12:53:03.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.066+0000 7ff925fd9700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff92013d3a0 con 0x7ff920060020 2022-06-17T12:53:03.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.066+0000 7ff925fd9700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff920143df0 con 0x7ff92013ca70 2022-06-17T12:53:03.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.066+0000 7ff924fd7700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff920060020 0x7ff9200626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:03.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.066+0000 7ff924fd7700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff920060020 0x7ff9200626a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35456/0 (socket says 172.21.15.36:35456) 2022-06-17T12:53:03.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.066+0000 7ff924fd7700 1 -- 172.21.15.36:0/1812712317 learned_addr learned my addr 172.21.15.36:0/1812712317 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:03.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.066+0000 7ff9257d8700 1 --2- 172.21.15.36:0/1812712317 >> v2:172.21.15.70:3300/0 conn(0x7ff92013ca70 0x7ff92013ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:03.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.066+0000 7ff924fd7700 1 -- 172.21.15.36:0/1812712317 >> v2:172.21.15.70:3300/0 conn(0x7ff92013ca70 msgr2=0x7ff92013ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:03.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.066+0000 7ff924fd7700 1 --2- 172.21.15.36:0/1812712317 >> v2:172.21.15.70:3300/0 conn(0x7ff92013ca70 0x7ff92013ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:03.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.066+0000 7ff924fd7700 1 -- 172.21.15.36:0/1812712317 >> v1:172.21.15.70:6789/0 conn(0x7ff9200b3f80 legacy=0x7ff9200b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:03.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.066+0000 7ff924fd7700 1 -- 172.21.15.36:0/1812712317 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff920061f60 con 0x7ff920060020 2022-06-17T12:53:03.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.066+0000 7ff924fd7700 1 --2- 172.21.15.36:0/1812712317 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff920060020 0x7ff9200626a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7ff914002f70 tx=0x7ff914006eb0).ready entity=mon.0 client_cookie=3cb08466dec7bc53 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:03.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.070+0000 7ff91f7fe700 1 -- 172.21.15.36:0/1812712317 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff91400d5c0 con 0x7ff920060020 2022-06-17T12:53:03.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.070+0000 7ff91f7fe700 1 -- 172.21.15.36:0/1812712317 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff914000e90 con 0x7ff920060020 2022-06-17T12:53:03.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.070+0000 7ff91f7fe700 1 -- 172.21.15.36:0/1812712317 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff91400d950 con 0x7ff920060020 2022-06-17T12:53:03.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.070+0000 7ff925fd9700 1 -- 172.21.15.36:0/1812712317 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff920060020 msgr2=0x7ff9200626a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:03.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.070+0000 7ff925fd9700 1 --2- 172.21.15.36:0/1812712317 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff920060020 0x7ff9200626a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7ff914002f70 tx=0x7ff914006eb0).stop 2022-06-17T12:53:03.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.070+0000 7ff925fd9700 1 -- 172.21.15.36:0/1812712317 shutdown_connections 2022-06-17T12:53:03.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.070+0000 7ff925fd9700 1 --2- 172.21.15.36:0/1812712317 >> v2:172.21.15.70:3300/0 conn(0x7ff92013ca70 0x7ff92013ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:03.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.070+0000 7ff925fd9700 1 --2- 172.21.15.36:0/1812712317 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff920060020 0x7ff9200626a0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:03.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.070+0000 7ff925fd9700 1 -- 172.21.15.36:0/1812712317 >> 172.21.15.36:0/1812712317 conn(0x7ff92005c550 msgr2=0x7ff92005f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:03.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.070+0000 7ff925fd9700 1 -- 172.21.15.36:0/1812712317 shutdown_connections 2022-06-17T12:53:03.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.070+0000 7ff925fd9700 1 -- 172.21.15.36:0/1812712317 wait complete. 2022-06-17T12:53:03.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.070+0000 7ff925fd9700 1 Processor -- start 2022-06-17T12:53:03.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.070+0000 7ff925fd9700 1 -- start start 2022-06-17T12:53:03.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff925fd9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff920060020 0x7ff920152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:03.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff925fd9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9200b3f80 0x7ff9200aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:03.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff925fd9700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff92013ca70 0x7ff92013f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:03.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff925fd9700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9201411e0 con 0x7ff920060020 2022-06-17T12:53:03.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff925fd9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9200b07b0 con 0x7ff9200b3f80 2022-06-17T12:53:03.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff925fd9700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff920152740 con 0x7ff92013ca70 2022-06-17T12:53:03.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff924fd7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff920060020 0x7ff920152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:03.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff924fd7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff920060020 0x7ff920152e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35460/0 (socket says 172.21.15.36:35460) 2022-06-17T12:53:03.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff924fd7700 1 -- 172.21.15.36:0/4294130582 learned_addr learned my addr 172.21.15.36:0/4294130582 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:03.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff9257d8700 1 --2- 172.21.15.36:0/4294130582 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff92013ca70 0x7ff92013f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:03.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff91ffff700 1 --2- 172.21.15.36:0/4294130582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9200b3f80 0x7ff9200aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:03.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff924fd7700 1 -- 172.21.15.36:0/4294130582 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff92013ca70 msgr2=0x7ff92013f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:03.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff924fd7700 1 --2- 172.21.15.36:0/4294130582 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff92013ca70 0x7ff92013f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:03.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff924fd7700 1 -- 172.21.15.36:0/4294130582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9200b3f80 msgr2=0x7ff9200aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:03.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff924fd7700 1 --2- 172.21.15.36:0/4294130582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9200b3f80 0x7ff9200aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:03.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff924fd7700 1 -- 172.21.15.36:0/4294130582 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91400b1e0 con 0x7ff920060020 2022-06-17T12:53:03.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff91ffff700 1 --2- 172.21.15.36:0/4294130582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9200b3f80 0x7ff9200aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:53:03.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff9257d8700 1 --2- 172.21.15.36:0/4294130582 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff92013ca70 0x7ff92013f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:53:03.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff924fd7700 1 --2- 172.21.15.36:0/4294130582 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff920060020 0x7ff920152e80 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7ff914002f70 tx=0x7ff914015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:03.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff91dffb700 1 -- 172.21.15.36:0/4294130582 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff91400de90 con 0x7ff920060020 2022-06-17T12:53:03.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff91dffb700 1 -- 172.21.15.36:0/4294130582 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9140068b0 con 0x7ff920060020 2022-06-17T12:53:03.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff925fd9700 1 -- 172.21.15.36:0/4294130582 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff920153370 con 0x7ff920060020 2022-06-17T12:53:03.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff91dffb700 1 -- 172.21.15.36:0/4294130582 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff91401e460 con 0x7ff920060020 2022-06-17T12:53:03.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff925fd9700 1 -- 172.21.15.36:0/4294130582 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9201535d0 con 0x7ff920060020 2022-06-17T12:53:03.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff925fd9700 1 -- 172.21.15.36:0/4294130582 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff920062200 con 0x7ff920060020 2022-06-17T12:53:03.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff91dffb700 1 -- 172.21.15.36:0/4294130582 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff914025030 con 0x7ff920060020 2022-06-17T12:53:03.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.074+0000 7ff91dffb700 1 --2- 172.21.15.36:0/4294130582 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff9000539d0 0x7ff900055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:03.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.078+0000 7ff91ffff700 1 --2- 172.21.15.36:0/4294130582 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff9000539d0 0x7ff900055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:03.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.078+0000 7ff91dffb700 1 -- 172.21.15.36:0/4294130582 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5600+0+0 (secure 0 0 0) 0x7ff91406cef0 con 0x7ff920060020 2022-06-17T12:53:03.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.078+0000 7ff91ffff700 1 --2- 172.21.15.36:0/4294130582 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff9000539d0 0x7ff900055e90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7ff910008000 tx=0x7ff910009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:03.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.082+0000 7ff91dffb700 1 -- 172.21.15.36:0/4294130582 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff914043f20 con 0x7ff920060020 2022-06-17T12:53:03.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:03.210+0000 7ff925fd9700 1 -- 172.21.15.36:0/4294130582 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7ff920057410 con 0x7ff920060020 2022-06-17T12:53:04.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.050+0000 7ff91dffb700 1 -- 172.21.15.36:0/4294130582 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v71) v1 ==== 133+0+0 (secure 0 0 0) 0x7ff914023070 con 0x7ff920060020 2022-06-17T12:53:04.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.054+0000 7ff925fd9700 1 -- 172.21.15.36:0/4294130582 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff9000539d0 msgr2=0x7ff900055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:04.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.054+0000 7ff925fd9700 1 --2- 172.21.15.36:0/4294130582 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff9000539d0 0x7ff900055e90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7ff910008000 tx=0x7ff910009040).stop 2022-06-17T12:53:04.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.054+0000 7ff925fd9700 1 -- 172.21.15.36:0/4294130582 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff920060020 msgr2=0x7ff920152e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:04.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.054+0000 7ff925fd9700 1 --2- 172.21.15.36:0/4294130582 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff920060020 0x7ff920152e80 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7ff914002f70 tx=0x7ff914015000).stop 2022-06-17T12:53:04.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.054+0000 7ff925fd9700 1 -- 172.21.15.36:0/4294130582 shutdown_connections 2022-06-17T12:53:04.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.054+0000 7ff925fd9700 1 --2- 172.21.15.36:0/4294130582 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff9000539d0 0x7ff900055e90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:04.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.054+0000 7ff925fd9700 1 --2- 172.21.15.36:0/4294130582 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff92013ca70 0x7ff92013f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:04.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.054+0000 7ff925fd9700 1 --2- 172.21.15.36:0/4294130582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9200b3f80 0x7ff9200aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:04.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.054+0000 7ff925fd9700 1 --2- 172.21.15.36:0/4294130582 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff920060020 0x7ff920152e80 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:04.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.054+0000 7ff925fd9700 1 -- 172.21.15.36:0/4294130582 >> 172.21.15.36:0/4294130582 conn(0x7ff92005c550 msgr2=0x7ff92013c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:04.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.054+0000 7ff925fd9700 1 -- 172.21.15.36:0/4294130582 shutdown_connections 2022-06-17T12:53:04.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.054+0000 7ff925fd9700 1 -- 172.21.15.36:0/4294130582 wait complete. 2022-06-17T12:53:04.042 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' created 2022-06-17T12:53:04.053 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-06-17T12:53:04.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.174+0000 7f543d7c5700 1 Processor -- start 2022-06-17T12:53:04.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.174+0000 7f543d7c5700 1 -- start start 2022-06-17T12:53:04.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.178+0000 7f543d7c5700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5438060ac0 0x7f54380630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:04.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.178+0000 7f543d7c5700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f54380b1680 0x7f54381431a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:04.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.178+0000 7f543d7c5700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f54380ae6c0 0x7f5438143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:04.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.178+0000 7f543d7c5700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5438062c80 con 0x7f54380b1680 2022-06-17T12:53:04.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.178+0000 7f543d7c5700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5438060ea0 con 0x7f54380ae6c0 2022-06-17T12:53:04.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.178+0000 7f543d7c5700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54380b2f50 con 0x7f5438060ac0 2022-06-17T12:53:04.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.178+0000 7f542ffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f54380b1680 0x7f54381431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:04.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.178+0000 7f542ffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f54380b1680 0x7f54381431a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35470/0 (socket says 172.21.15.36:35470) 2022-06-17T12:53:04.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.178+0000 7f542ffff700 1 -- 172.21.15.36:0/2359170907 learned_addr learned my addr 172.21.15.36:0/2359170907 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:04.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.178+0000 7f5437fff700 1 --2- 172.21.15.36:0/2359170907 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5438060ac0 0x7f54380630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:04.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.178+0000 7f543cfc4700 1 --2- 172.21.15.36:0/2359170907 >> v2:172.21.15.70:3300/0 conn(0x7f54380ae6c0 0x7f5438143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:04.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.178+0000 7f542ffff700 1 -- 172.21.15.36:0/2359170907 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5438060ac0 msgr2=0x7f54380630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:04.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.178+0000 7f542ffff700 1 --2- 172.21.15.36:0/2359170907 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5438060ac0 0x7f54380630e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:04.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.178+0000 7f542ffff700 1 -- 172.21.15.36:0/2359170907 >> v2:172.21.15.70:3300/0 conn(0x7f54380ae6c0 msgr2=0x7f5438143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:04.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.178+0000 7f542ffff700 1 --2- 172.21.15.36:0/2359170907 >> v2:172.21.15.70:3300/0 conn(0x7f54380ae6c0 0x7f5438143690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:04.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.178+0000 7f542ffff700 1 -- 172.21.15.36:0/2359170907 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54380635d0 con 0x7f54380b1680 2022-06-17T12:53:04.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.178+0000 7f542ffff700 1 --2- 172.21.15.36:0/2359170907 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f54380b1680 0x7f54381431a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7f5420002f70 tx=0x7f5420006eb0).ready entity=mon.0 client_cookie=a1b75ebd147eccb4 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:04.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.178+0000 7f54377fe700 1 -- 172.21.15.36:0/2359170907 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f542000d570 con 0x7f54380b1680 2022-06-17T12:53:04.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.178+0000 7f54377fe700 1 -- 172.21.15.36:0/2359170907 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5420000e90 con 0x7f54380b1680 2022-06-17T12:53:04.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.178+0000 7f54377fe700 1 -- 172.21.15.36:0/2359170907 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f542000d900 con 0x7f54380b1680 2022-06-17T12:53:04.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f543d7c5700 1 -- 172.21.15.36:0/2359170907 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f54380b1680 msgr2=0x7f54381431a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:04.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f543d7c5700 1 --2- 172.21.15.36:0/2359170907 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f54380b1680 0x7f54381431a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7f5420002f70 tx=0x7f5420006eb0).stop 2022-06-17T12:53:04.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f543d7c5700 1 -- 172.21.15.36:0/2359170907 shutdown_connections 2022-06-17T12:53:04.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f543d7c5700 1 --2- 172.21.15.36:0/2359170907 >> v2:172.21.15.70:3300/0 conn(0x7f54380ae6c0 0x7f5438143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:04.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f543d7c5700 1 --2- 172.21.15.36:0/2359170907 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f54380b1680 0x7f54381431a0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:04.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f543d7c5700 1 --2- 172.21.15.36:0/2359170907 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5438060ac0 0x7f54380630e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:04.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f543d7c5700 1 -- 172.21.15.36:0/2359170907 >> 172.21.15.36:0/2359170907 conn(0x7f54380b64f0 msgr2=0x7f54380b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:04.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f543d7c5700 1 -- 172.21.15.36:0/2359170907 shutdown_connections 2022-06-17T12:53:04.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f543d7c5700 1 -- 172.21.15.36:0/2359170907 wait complete. 2022-06-17T12:53:04.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f543d7c5700 1 Processor -- start 2022-06-17T12:53:04.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f543d7c5700 1 -- start start 2022-06-17T12:53:04.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f543d7c5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5438060ac0 0x7f543813e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:04.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f543d7c5700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54380ae6c0 0x7f54381410a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:04.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f543d7c5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f54380b1680 0x7f5438144900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:04.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f543d7c5700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5438142c00 con 0x7f54380b1680 2022-06-17T12:53:04.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f543d7c5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5438147190 con 0x7f5438060ac0 2022-06-17T12:53:04.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f543d7c5700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5438144460 con 0x7f54380ae6c0 2022-06-17T12:53:04.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f543cfc4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f54380b1680 0x7f5438144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:04.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f5437fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5438060ac0 0x7f543813e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:04.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f5437fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5438060ac0 0x7f543813e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:55856/0 (socket says 172.21.15.36:55856) 2022-06-17T12:53:04.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f5437fff700 1 -- 172.21.15.36:0/2674844466 learned_addr learned my addr 172.21.15.36:0/2674844466 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:04.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f542ffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54380ae6c0 0x7f54381410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:04.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f5437fff700 1 -- 172.21.15.36:0/2674844466 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54380ae6c0 msgr2=0x7f54381410a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:04.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f5437fff700 1 --2- 172.21.15.36:0/2674844466 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54380ae6c0 0x7f54381410a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:04.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f5437fff700 1 -- 172.21.15.36:0/2674844466 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f54380b1680 msgr2=0x7f5438144900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:04.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f5437fff700 1 --2- 172.21.15.36:0/2674844466 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f54380b1680 0x7f5438144900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:04.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f5437fff700 1 -- 172.21.15.36:0/2674844466 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542000b1e0 con 0x7f5438060ac0 2022-06-17T12:53:04.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f5437fff700 1 --2- 172.21.15.36:0/2674844466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5438060ac0 0x7f543813e0a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f542800c000 tx=0x7f5428006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:04.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f5435ffb700 1 -- 172.21.15.36:0/2674844466 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5428010780 con 0x7f5438060ac0 2022-06-17T12:53:04.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f5435ffb700 1 -- 172.21.15.36:0/2674844466 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54280108e0 con 0x7f5438060ac0 2022-06-17T12:53:04.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f543d7c5700 1 -- 172.21.15.36:0/2674844466 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5438141590 con 0x7f5438060ac0 2022-06-17T12:53:04.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f5435ffb700 1 -- 172.21.15.36:0/2674844466 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5428010c40 con 0x7f5438060ac0 2022-06-17T12:53:04.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.182+0000 7f543d7c5700 1 -- 172.21.15.36:0/2674844466 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54381430d0 con 0x7f5438060ac0 2022-06-17T12:53:04.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.186+0000 7f543d7c5700 1 -- 172.21.15.36:0/2674844466 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5438147350 con 0x7f5438060ac0 2022-06-17T12:53:04.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.186+0000 7f5435ffb700 1 -- 172.21.15.36:0/2674844466 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f542801c460 con 0x7f5438060ac0 2022-06-17T12:53:04.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.186+0000 7f5435ffb700 1 --2- 172.21.15.36:0/2674844466 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f541c053920 0x7f541c055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:04.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.186+0000 7f5435ffb700 1 -- 172.21.15.36:0/2674844466 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f542806c290 con 0x7f5438060ac0 2022-06-17T12:53:04.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.186+0000 7f542ffff700 1 --2- 172.21.15.36:0/2674844466 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f541c053920 0x7f541c055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:04.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.186+0000 7f542ffff700 1 --2- 172.21.15.36:0/2674844466 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f541c053920 0x7f541c055de0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f542001c000 tx=0x7f5420015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:04.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.194+0000 7f5435ffb700 1 -- 172.21.15.36:0/2674844466 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5428043160 con 0x7f5438060ac0 2022-06-17T12:53:04.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.326+0000 7f543d7c5700 1 -- 172.21.15.36:0/2674844466 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f5438057410 con 0x7f5438060ac0 2022-06-17T12:53:04.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.574+0000 7f5435ffb700 1 -- 172.21.15.36:0/2674844466 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v72) v1 ==== 125+0+0 (secure 0 0 0) 0x7f5428046780 con 0x7f5438060ac0 2022-06-17T12:53:04.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.582+0000 7f543d7c5700 1 -- 172.21.15.36:0/2674844466 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f541c053920 msgr2=0x7f541c055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:04.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.582+0000 7f543d7c5700 1 --2- 172.21.15.36:0/2674844466 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f541c053920 0x7f541c055de0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f542001c000 tx=0x7f5420015000).stop 2022-06-17T12:53:04.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.582+0000 7f543d7c5700 1 -- 172.21.15.36:0/2674844466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5438060ac0 msgr2=0x7f543813e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:04.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.582+0000 7f543d7c5700 1 --2- 172.21.15.36:0/2674844466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5438060ac0 0x7f543813e0a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f542800c000 tx=0x7f5428006df0).stop 2022-06-17T12:53:04.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.582+0000 7f543d7c5700 1 -- 172.21.15.36:0/2674844466 shutdown_connections 2022-06-17T12:53:04.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.582+0000 7f543d7c5700 1 --2- 172.21.15.36:0/2674844466 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f541c053920 0x7f541c055de0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:04.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.582+0000 7f543d7c5700 1 --2- 172.21.15.36:0/2674844466 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f54380b1680 0x7f5438144900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:04.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.582+0000 7f543d7c5700 1 --2- 172.21.15.36:0/2674844466 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54380ae6c0 0x7f54381410a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:04.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.582+0000 7f543d7c5700 1 --2- 172.21.15.36:0/2674844466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5438060ac0 0x7f543813e0a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:04.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.582+0000 7f543d7c5700 1 -- 172.21.15.36:0/2674844466 >> 172.21.15.36:0/2674844466 conn(0x7f54380b64f0 msgr2=0x7f54380b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:04.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.582+0000 7f543d7c5700 1 -- 172.21.15.36:0/2674844466 shutdown_connections 2022-06-17T12:53:04.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.582+0000 7f543d7c5700 1 -- 172.21.15.36:0/2674844466 wait complete. 2022-06-17T12:53:04.578 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' created 2022-06-17T12:53:04.590 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-17T12:53:04.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6caeb8700 1 Processor -- start 2022-06-17T12:53:04.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6caeb8700 1 -- start start 2022-06-17T12:53:04.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6caeb8700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc6c40b2200 0x7fc6c40af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:04.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6caeb8700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6c40b2cd0 0x7fc6c4147600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:04.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6caeb8700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c40b25e0 con 0x7fc6c40af8d0 2022-06-17T12:53:04.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6caeb8700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c40ad790 con 0x7fc6c40b2200 2022-06-17T12:53:04.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6caeb8700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40aef40 con 0x7fc6c40b2cd0 2022-06-17T12:53:04.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6c9eb6700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc6c40b2200 0x7fc6c40af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:04.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6c9eb6700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc6c40b2200 0x7fc6c40af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35482/0 (socket says 172.21.15.36:35482) 2022-06-17T12:53:04.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6c9eb6700 1 -- 172.21.15.36:0/1185822473 learned_addr learned my addr 172.21.15.36:0/1185822473 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:04.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6c96b5700 1 --1- 172.21.15.36:0/1185822473 >> v1:172.21.15.70:6789/0 conn(0x7fc6c40af8d0 0x7fc6c414a8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:48748/0 (socket says 172.21.15.36:48748) 2022-06-17T12:53:04.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6ca6b7700 1 --2- 172.21.15.36:0/1185822473 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6c40b2cd0 0x7fc6c4147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:04.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6c9eb6700 1 -- 172.21.15.36:0/1185822473 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6c40b2cd0 msgr2=0x7fc6c4147600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:04.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6c9eb6700 1 --2- 172.21.15.36:0/1185822473 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6c40b2cd0 0x7fc6c4147600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:04.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6c9eb6700 1 -- 172.21.15.36:0/1185822473 >> v1:172.21.15.70:6789/0 conn(0x7fc6c40af8d0 legacy=0x7fc6c414a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:04.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6c9eb6700 1 -- 172.21.15.36:0/1185822473 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c40b1950 con 0x7fc6c40b2200 2022-06-17T12:53:04.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6c9eb6700 1 --2- 172.21.15.36:0/1185822473 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc6c40b2200 0x7fc6c40af3e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7fc6c0002f70 tx=0x7fc6c0006eb0).ready entity=mon.0 client_cookie=ddbb8e2a11fa4bb8 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:04.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6c8eb4700 1 -- 172.21.15.36:0/1185822473 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6c000d570 con 0x7fc6c40b2200 2022-06-17T12:53:04.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6c8eb4700 1 -- 172.21.15.36:0/1185822473 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c0000e90 con 0x7fc6c40b2200 2022-06-17T12:53:04.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6c8eb4700 1 -- 172.21.15.36:0/1185822473 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6c000d900 con 0x7fc6c40b2200 2022-06-17T12:53:04.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6caeb8700 1 -- 172.21.15.36:0/1185822473 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc6c40b2200 msgr2=0x7fc6c40af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:04.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6caeb8700 1 --2- 172.21.15.36:0/1185822473 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc6c40b2200 0x7fc6c40af3e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7fc6c0002f70 tx=0x7fc6c0006eb0).stop 2022-06-17T12:53:04.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6caeb8700 1 -- 172.21.15.36:0/1185822473 shutdown_connections 2022-06-17T12:53:04.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6caeb8700 1 --2- 172.21.15.36:0/1185822473 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6c40b2cd0 0x7fc6c4147600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:04.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6caeb8700 1 --2- 172.21.15.36:0/1185822473 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc6c40b2200 0x7fc6c40af3e0 secure :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 rx=0x7fc6c0002f70 tx=0x7fc6c0006eb0).stop 2022-06-17T12:53:04.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6caeb8700 1 -- 172.21.15.36:0/1185822473 >> 172.21.15.36:0/1185822473 conn(0x7fc6c405f770 msgr2=0x7fc6c405fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:04.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.718+0000 7fc6caeb8700 1 -- 172.21.15.36:0/1185822473 shutdown_connections 2022-06-17T12:53:04.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.722+0000 7fc6caeb8700 1 -- 172.21.15.36:0/1185822473 wait complete. 2022-06-17T12:53:04.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.722+0000 7fc6caeb8700 1 Processor -- start 2022-06-17T12:53:04.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.722+0000 7fc6caeb8700 1 -- start start 2022-06-17T12:53:04.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.722+0000 7fc6caeb8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc6c40af8d0 0x7fc6c4062390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:04.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.722+0000 7fc6caeb8700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6c40b2cd0 0x7fc6c40b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:04.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.722+0000 7fc6caeb8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6c40a8a00 0x7fc6c415a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:04.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.722+0000 7fc6caeb8700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c4145440 con 0x7fc6c40af8d0 2022-06-17T12:53:04.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.722+0000 7fc6caeb8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40a7830 con 0x7fc6c40a8a00 2022-06-17T12:53:04.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.722+0000 7fc6caeb8700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40a7ad0 con 0x7fc6c40b2cd0 2022-06-17T12:53:04.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.722+0000 7fc6c9eb6700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc6c40af8d0 0x7fc6c4062390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:04.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.722+0000 7fc6c9eb6700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc6c40af8d0 0x7fc6c4062390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35490/0 (socket says 172.21.15.36:35490) 2022-06-17T12:53:04.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.722+0000 7fc6ca6b7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6c40a8a00 0x7fc6c415a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:04.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.722+0000 7fc6c9eb6700 1 -- 172.21.15.36:0/2622556152 learned_addr learned my addr 172.21.15.36:0/2622556152 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:04.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.722+0000 7fc6c96b5700 1 --2- 172.21.15.36:0/2622556152 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6c40b2cd0 0x7fc6c40b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:04.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.722+0000 7fc6c9eb6700 1 -- 172.21.15.36:0/2622556152 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6c40b2cd0 msgr2=0x7fc6c40b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:04.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.722+0000 7fc6c9eb6700 1 --2- 172.21.15.36:0/2622556152 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6c40b2cd0 0x7fc6c40b00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:04.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.722+0000 7fc6c9eb6700 1 -- 172.21.15.36:0/2622556152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6c40a8a00 msgr2=0x7fc6c415a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:04.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.722+0000 7fc6c9eb6700 1 --2- 172.21.15.36:0/2622556152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6c40a8a00 0x7fc6c415a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:04.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.722+0000 7fc6c9eb6700 1 -- 172.21.15.36:0/2622556152 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c000b1e0 con 0x7fc6c40af8d0 2022-06-17T12:53:04.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.722+0000 7fc6c9eb6700 1 --2- 172.21.15.36:0/2622556152 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc6c40af8d0 0x7fc6c4062390 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7fc6c0002f70 tx=0x7fc6c0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:04.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.726+0000 7fc6b2ffd700 1 -- 172.21.15.36:0/2622556152 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6c0011690 con 0x7fc6c40af8d0 2022-06-17T12:53:04.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.726+0000 7fc6b2ffd700 1 -- 172.21.15.36:0/2622556152 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c00117f0 con 0x7fc6c40af8d0 2022-06-17T12:53:04.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.726+0000 7fc6b2ffd700 1 -- 172.21.15.36:0/2622556152 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6c0011b50 con 0x7fc6c40af8d0 2022-06-17T12:53:04.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.726+0000 7fc6caeb8700 1 -- 172.21.15.36:0/2622556152 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c40a7f70 con 0x7fc6c40af8d0 2022-06-17T12:53:04.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.726+0000 7fc6caeb8700 1 -- 172.21.15.36:0/2622556152 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c415a810 con 0x7fc6c40af8d0 2022-06-17T12:53:04.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.726+0000 7fc6caeb8700 1 -- 172.21.15.36:0/2622556152 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c40a8e10 con 0x7fc6c40af8d0 2022-06-17T12:53:04.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.726+0000 7fc6b2ffd700 1 -- 172.21.15.36:0/2622556152 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fc6c001a040 con 0x7fc6c40af8d0 2022-06-17T12:53:04.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.726+0000 7fc6b2ffd700 1 --2- 172.21.15.36:0/2622556152 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc6a4053ad0 0x7fc6a4055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:04.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.726+0000 7fc6b2ffd700 1 -- 172.21.15.36:0/2622556152 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fc6c006ccf0 con 0x7fc6c40af8d0 2022-06-17T12:53:04.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.726+0000 7fc6c96b5700 1 --2- 172.21.15.36:0/2622556152 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc6a4053ad0 0x7fc6a4055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:04.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.730+0000 7fc6c96b5700 1 --2- 172.21.15.36:0/2622556152 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc6a4053ad0 0x7fc6a4055f90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fc6b400d000 tx=0x7fc6b4009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:04.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.738+0000 7fc6b2ffd700 1 -- 172.21.15.36:0/2622556152 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc6c0043ae0 con 0x7fc6c40af8d0 2022-06-17T12:53:04.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.902+0000 7fc6caeb8700 1 -- 172.21.15.36:0/2622556152 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fc6c40ad330 con 0x7fc6c40af8d0 2022-06-17T12:53:04.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.910+0000 7fc6b2ffd700 1 -- 172.21.15.36:0/2622556152 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 13 and data pool 14 v34) v1 ==== 189+0+0 (secure 0 0 0) 0x7fc6c0023020 con 0x7fc6c40af8d0 2022-06-17T12:53:04.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.922+0000 7fc6caeb8700 1 -- 172.21.15.36:0/2622556152 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc6a4053ad0 msgr2=0x7fc6a4055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:04.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.922+0000 7fc6caeb8700 1 --2- 172.21.15.36:0/2622556152 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc6a4053ad0 0x7fc6a4055f90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fc6b400d000 tx=0x7fc6b4009040).stop 2022-06-17T12:53:04.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.922+0000 7fc6caeb8700 1 -- 172.21.15.36:0/2622556152 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc6c40af8d0 msgr2=0x7fc6c4062390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:04.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.922+0000 7fc6caeb8700 1 --2- 172.21.15.36:0/2622556152 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc6c40af8d0 0x7fc6c4062390 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7fc6c0002f70 tx=0x7fc6c0015000).stop 2022-06-17T12:53:04.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.922+0000 7fc6caeb8700 1 -- 172.21.15.36:0/2622556152 shutdown_connections 2022-06-17T12:53:04.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.922+0000 7fc6caeb8700 1 --2- 172.21.15.36:0/2622556152 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc6a4053ad0 0x7fc6a4055f90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:04.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.922+0000 7fc6caeb8700 1 --2- 172.21.15.36:0/2622556152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6c40a8a00 0x7fc6c415a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:04.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.922+0000 7fc6caeb8700 1 --2- 172.21.15.36:0/2622556152 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6c40b2cd0 0x7fc6c40b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:04.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.922+0000 7fc6caeb8700 1 --2- 172.21.15.36:0/2622556152 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc6c40af8d0 0x7fc6c4062390 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:04.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.922+0000 7fc6caeb8700 1 -- 172.21.15.36:0/2622556152 >> 172.21.15.36:0/2622556152 conn(0x7fc6c405f770 msgr2=0x7fc6c40ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:04.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.922+0000 7fc6caeb8700 1 -- 172.21.15.36:0/2622556152 shutdown_connections 2022-06-17T12:53:04.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:04.922+0000 7fc6caeb8700 1 -- 172.21.15.36:0/2622556152 wait complete. 2022-06-17T12:53:04.913 INFO:teuthology.orchestra.run.smithi036.stderr:new fs with metadata pool 13 and data pool 14 2022-06-17T12:53:04.920 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:53:05.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.030+0000 7f7e4e0a7700 1 Processor -- start 2022-06-17T12:53:05.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.030+0000 7f7e4e0a7700 1 -- start start 2022-06-17T12:53:05.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4e0a7700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7e480afe30 0x7f7e48064230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:05.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4e0a7700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e480af280 con 0x7f7e480af930 2022-06-17T12:53:05.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4e0a7700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e480b14f0 con 0x7f7e480aeea0 2022-06-17T12:53:05.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4e0a7700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e48067560 con 0x7f7e480afe30 2022-06-17T12:53:05.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4c8a4700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f7e480af930 0x7f7e480ab9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53038/0 (socket says 172.21.15.36:53038) 2022-06-17T12:53:05.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4c8a4700 1 -- 172.21.15.36:0/2990040496 learned_addr learned my addr 172.21.15.36:0/2990040496 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:53:05.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4d8a6700 1 --2- 172.21.15.36:0/2990040496 >> v2:172.21.15.70:3300/0 conn(0x7f7e480afe30 0x7f7e48064230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:05.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e37fff700 1 -- 172.21.15.36:0/2990040496 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609168139 0 0) 0x7f7e480af280 con 0x7f7e480af930 2022-06-17T12:53:05.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e37fff700 1 -- 172.21.15.36:0/2990040496 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e30003260 con 0x7f7e480af930 2022-06-17T12:53:05.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e37fff700 1 -- 172.21.15.36:0/2990040496 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3012442997 0 0) 0x7f7e30003260 con 0x7f7e480af930 2022-06-17T12:53:05.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e37fff700 1 -- 172.21.15.36:0/2990040496 >> v2:172.21.15.70:3300/0 conn(0x7f7e480afe30 msgr2=0x7f7e48064230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:05.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e37fff700 1 --2- 172.21.15.36:0/2990040496 >> v2:172.21.15.70:3300/0 conn(0x7f7e480afe30 0x7f7e48064230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:05.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e37fff700 1 -- 172.21.15.36:0/2990040496 >> v1:172.21.15.70:6789/0 conn(0x7f7e480aeea0 legacy=0x7f7e480b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:05.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e37fff700 1 -- 172.21.15.36:0/2990040496 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e480b20f0 con 0x7f7e480af930 2022-06-17T12:53:05.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e37fff700 1 -- 172.21.15.36:0/2990040496 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f7e38001660 con 0x7f7e480af930 2022-06-17T12:53:05.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4d8a6700 1 --2- 172.21.15.36:0/2990040496 >> v2:172.21.15.70:3300/0 conn(0x7f7e480afe30 0x7f7e48064230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:53:05.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e37fff700 1 -- 172.21.15.36:0/2990040496 >> v1:172.21.15.36:6789/0 conn(0x7f7e480af930 legacy=0x7f7e480ab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:05.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e37fff700 1 --2- 172.21.15.36:0/2990040496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e30003750 0x7f7e30003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:05.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e37fff700 1 --2- 172.21.15.36:0/2990040496 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e30004500 0x7f7e30004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:05.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e37fff700 1 --2- 172.21.15.36:0/2990040496 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7e30004e70 0x7f7e3000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:05.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e37fff700 1 -- 172.21.15.36:0/2990040496 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e48067560 con 0x7f7e30004e70 2022-06-17T12:53:05.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e37fff700 1 -- 172.21.15.36:0/2990040496 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3000bd20 con 0x7f7e30003750 2022-06-17T12:53:05.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e37fff700 1 -- 172.21.15.36:0/2990040496 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3000bee0 con 0x7f7e30004500 2022-06-17T12:53:05.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4d0a5700 1 --2- 172.21.15.36:0/2990040496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e30003750 0x7f7e30003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:05.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4d8a6700 1 --2- 172.21.15.36:0/2990040496 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7e30004e70 0x7f7e3000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:05.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4c8a4700 1 --2- 172.21.15.36:0/2990040496 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e30004500 0x7f7e30004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:05.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4d0a5700 1 -- 172.21.15.36:0/2990040496 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e30004500 msgr2=0x7f7e30004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:05.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4d0a5700 1 --2- 172.21.15.36:0/2990040496 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e30004500 0x7f7e30004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:05.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4d0a5700 1 -- 172.21.15.36:0/2990040496 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7e30004e70 msgr2=0x7f7e3000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:05.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4d0a5700 1 --2- 172.21.15.36:0/2990040496 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7e30004e70 0x7f7e3000b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:05.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4d0a5700 1 -- 172.21.15.36:0/2990040496 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e30003260 con 0x7f7e30003750 2022-06-17T12:53:05.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4d8a6700 1 --2- 172.21.15.36:0/2990040496 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7e30004e70 0x7f7e3000b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:53:05.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4d8a6700 1 -- 172.21.15.36:0/2990040496 reap_dead start 2022-06-17T12:53:05.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4d0a5700 1 --2- 172.21.15.36:0/2990040496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e30003750 0x7f7e30003b70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f7e480b6450 tx=0x7f7e44007a00).ready entity=mon.1 client_cookie=43e1895508797dcc server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:05.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e37fff700 1 -- 172.21.15.36:0/2990040496 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e44007d10 con 0x7f7e30003750 2022-06-17T12:53:05.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e37fff700 1 -- 172.21.15.36:0/2990040496 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e44007030 con 0x7f7e30003750 2022-06-17T12:53:05.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e37fff700 1 -- 172.21.15.36:0/2990040496 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e44007390 con 0x7f7e30003750 2022-06-17T12:53:05.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4e0a7700 1 -- 172.21.15.36:0/2990040496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e30003750 msgr2=0x7f7e30003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:05.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4e0a7700 1 --2- 172.21.15.36:0/2990040496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e30003750 0x7f7e30003b70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f7e480b6450 tx=0x7f7e44007a00).stop 2022-06-17T12:53:05.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4e0a7700 1 -- 172.21.15.36:0/2990040496 shutdown_connections 2022-06-17T12:53:05.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4e0a7700 1 --2- 172.21.15.36:0/2990040496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e30003750 0x7f7e30003b70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:05.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4e0a7700 1 -- 172.21.15.36:0/2990040496 >> 172.21.15.36:0/2990040496 conn(0x7f7e4805f770 msgr2=0x7f7e4805fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:05.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4e0a7700 1 -- 172.21.15.36:0/2990040496 shutdown_connections 2022-06-17T12:53:05.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.034+0000 7f7e4e0a7700 1 -- 172.21.15.36:0/2990040496 wait complete. 2022-06-17T12:53:05.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e4e0a7700 1 Processor -- start 2022-06-17T12:53:05.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e4e0a7700 1 -- start start 2022-06-17T12:53:05.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e4e0a7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7e30003750 0x7f7e480a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:05.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e4e0a7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e480a4c00 0x7f7e48154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:05.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e4e0a7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e48150390 0x7f7e48150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:05.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e4e0a7700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e48153ae0 con 0x7f7e30003750 2022-06-17T12:53:05.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e4e0a7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e48153d80 con 0x7f7e48150390 2022-06-17T12:53:05.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e4e0a7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e480a4870 con 0x7f7e480a4c00 2022-06-17T12:53:05.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e4d0a5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7e30003750 0x7f7e480a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:05.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e4d0a5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7e30003750 0x7f7e480a1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35520/0 (socket says 172.21.15.36:35520) 2022-06-17T12:53:05.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e4d0a5700 1 -- 172.21.15.36:0/1814766103 learned_addr learned my addr 172.21.15.36:0/1814766103 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:05.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e4c8a4700 1 --2- 172.21.15.36:0/1814766103 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e480a4c00 0x7f7e48154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:05.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e4d8a6700 1 --2- 172.21.15.36:0/1814766103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e48150390 0x7f7e48150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:05.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e4d0a5700 1 -- 172.21.15.36:0/1814766103 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e480a4c00 msgr2=0x7f7e48154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:05.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e4d0a5700 1 --2- 172.21.15.36:0/1814766103 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e480a4c00 0x7f7e48154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:05.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e4d0a5700 1 -- 172.21.15.36:0/1814766103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e48150390 msgr2=0x7f7e48150810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:05.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e4d0a5700 1 --2- 172.21.15.36:0/1814766103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e48150390 0x7f7e48150810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:05.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e4d0a5700 1 -- 172.21.15.36:0/1814766103 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e480a4a30 con 0x7f7e30003750 2022-06-17T12:53:05.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e4d8a6700 1 --2- 172.21.15.36:0/1814766103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e48150390 0x7f7e48150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:53:05.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e4d0a5700 1 --2- 172.21.15.36:0/1814766103 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7e30003750 0x7f7e480a1f80 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7f7e480b6450 tx=0x7f7e44015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:05.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e367fc700 1 -- 172.21.15.36:0/1814766103 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e44020450 con 0x7f7e30003750 2022-06-17T12:53:05.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e367fc700 1 -- 172.21.15.36:0/1814766103 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e440205b0 con 0x7f7e30003750 2022-06-17T12:53:05.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e367fc700 1 -- 172.21.15.36:0/1814766103 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e44020910 con 0x7f7e30003750 2022-06-17T12:53:05.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e4e0a7700 1 -- 172.21.15.36:0/1814766103 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e48154710 con 0x7f7e30003750 2022-06-17T12:53:05.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.038+0000 7f7e4e0a7700 1 -- 172.21.15.36:0/1814766103 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e4814d450 con 0x7f7e30003750 2022-06-17T12:53:05.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.042+0000 7f7e4e0a7700 1 -- 172.21.15.36:0/1814766103 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e48057410 con 0x7f7e30003750 2022-06-17T12:53:05.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.042+0000 7f7e367fc700 1 -- 172.21.15.36:0/1814766103 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f7e4401d070 con 0x7f7e30003750 2022-06-17T12:53:05.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.042+0000 7f7e367fc700 1 --2- 172.21.15.36:0/1814766103 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7e28053af0 0x7f7e28055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:05.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.042+0000 7f7e367fc700 1 -- 172.21.15.36:0/1814766103 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7e4406ce30 con 0x7f7e30003750 2022-06-17T12:53:05.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.042+0000 7f7e4c8a4700 1 --2- 172.21.15.36:0/1814766103 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7e28053af0 0x7f7e28055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:05.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.042+0000 7f7e4c8a4700 1 --2- 172.21.15.36:0/1814766103 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7e28053af0 0x7f7e28055fb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f7e3800d000 tx=0x7f7e38007040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:05.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.050+0000 7f7e367fc700 1 -- 172.21.15.36:0/1814766103 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7e44043ac0 con 0x7f7e30003750 2022-06-17T12:53:05.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.178+0000 7f7e4e0a7700 1 -- 172.21.15.36:0/1814766103 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e48060460 con 0x7f7e30003750 2022-06-17T12:53:05.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.182+0000 7f7e367fc700 1 -- 172.21.15.36:0/1814766103 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f7e44025070 con 0x7f7e30003750 2022-06-17T12:53:05.167 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:53:05.168 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":73,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:53:04.907315+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:53:03.265804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-06-17T12:53:04.385158+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:53:05.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.182+0000 7f7e4e0a7700 1 -- 172.21.15.36:0/1814766103 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7e28053af0 msgr2=0x7f7e28055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:05.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.182+0000 7f7e4e0a7700 1 --2- 172.21.15.36:0/1814766103 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7e28053af0 0x7f7e28055fb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f7e3800d000 tx=0x7f7e38007040).stop 2022-06-17T12:53:05.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.182+0000 7f7e4e0a7700 1 -- 172.21.15.36:0/1814766103 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7e30003750 msgr2=0x7f7e480a1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:05.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.182+0000 7f7e4e0a7700 1 --2- 172.21.15.36:0/1814766103 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7e30003750 0x7f7e480a1f80 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7f7e480b6450 tx=0x7f7e44015000).stop 2022-06-17T12:53:05.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.182+0000 7f7e4e0a7700 1 -- 172.21.15.36:0/1814766103 shutdown_connections 2022-06-17T12:53:05.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.182+0000 7f7e4e0a7700 1 --2- 172.21.15.36:0/1814766103 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7e28053af0 0x7f7e28055fb0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:05.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.182+0000 7f7e4e0a7700 1 --2- 172.21.15.36:0/1814766103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e48150390 0x7f7e48150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:05.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.182+0000 7f7e4e0a7700 1 --2- 172.21.15.36:0/1814766103 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e480a4c00 0x7f7e48154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:05.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.182+0000 7f7e4e0a7700 1 --2- 172.21.15.36:0/1814766103 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7e30003750 0x7f7e480a1f80 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:05.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.182+0000 7f7e4e0a7700 1 -- 172.21.15.36:0/1814766103 >> 172.21.15.36:0/1814766103 conn(0x7f7e4805f770 msgr2=0x7f7e480b5490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:05.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.182+0000 7f7e4e0a7700 1 -- 172.21.15.36:0/1814766103 shutdown_connections 2022-06-17T12:53:05.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.182+0000 7f7e4e0a7700 1 -- 172.21.15.36:0/1814766103 wait complete. 2022-06-17T12:53:05.180 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:53:05.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.310+0000 7f7fbc975700 1 Processor -- start 2022-06-17T12:53:05.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.310+0000 7f7fbc975700 1 -- start start 2022-06-17T12:53:05.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.310+0000 7f7fbc975700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fb80b2200 0x7f7fb80af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:05.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.310+0000 7f7fbc975700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7fb80af8d0 0x7f7fb814a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:05.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.310+0000 7f7fbc975700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb80b25e0 con 0x7f7fb80b2cd0 2022-06-17T12:53:05.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.310+0000 7f7fbc975700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb80ad790 con 0x7f7fb80af8d0 2022-06-17T12:53:05.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.310+0000 7f7fbc975700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb80aef40 con 0x7f7fb80b2200 2022-06-17T12:53:05.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.310+0000 7f7fb6ffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7fb80af8d0 0x7f7fb814a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:05.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.310+0000 7f7fb6ffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7fb80af8d0 0x7f7fb814a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35528/0 (socket says 172.21.15.36:35528) 2022-06-17T12:53:05.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.310+0000 7f7fb6ffd700 1 -- 172.21.15.36:0/1375213201 learned_addr learned my addr 172.21.15.36:0/1375213201 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:05.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.310+0000 7f7fb77fe700 1 --2- 172.21.15.36:0/1375213201 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fb80b2200 0x7f7fb80af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:05.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.310+0000 7f7fb6ffd700 1 -- 172.21.15.36:0/1375213201 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fb80b2200 msgr2=0x7f7fb80af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:05.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.310+0000 7f7fb6ffd700 1 --2- 172.21.15.36:0/1375213201 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fb80b2200 0x7f7fb80af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:05.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.310+0000 7f7fb6ffd700 1 -- 172.21.15.36:0/1375213201 >> v1:172.21.15.70:6789/0 conn(0x7f7fb80b2cd0 legacy=0x7f7fb814adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:05.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.310+0000 7f7fb6ffd700 1 -- 172.21.15.36:0/1375213201 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb80b1950 con 0x7f7fb80af8d0 2022-06-17T12:53:05.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.314+0000 7f7fb6ffd700 1 --2- 172.21.15.36:0/1375213201 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fb80af8d0 0x7f7fb814a8d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7f7fa80032f0 tx=0x7f7fa8009290).ready entity=mon.0 client_cookie=7f5e4c65125bd0bc server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:05.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.314+0000 7f7fb67fc700 1 -- 172.21.15.36:0/1375213201 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fa800b8a0 con 0x7f7fb80af8d0 2022-06-17T12:53:05.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.314+0000 7f7fb67fc700 1 -- 172.21.15.36:0/1375213201 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa800ba00 con 0x7f7fb80af8d0 2022-06-17T12:53:05.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.314+0000 7f7fb67fc700 1 -- 172.21.15.36:0/1375213201 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fa800bb70 con 0x7f7fb80af8d0 2022-06-17T12:53:05.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.314+0000 7f7fbc975700 1 -- 172.21.15.36:0/1375213201 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fb80af8d0 msgr2=0x7f7fb814a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:05.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.314+0000 7f7fbc975700 1 --2- 172.21.15.36:0/1375213201 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fb80af8d0 0x7f7fb814a8d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7f7fa80032f0 tx=0x7f7fa8009290).stop 2022-06-17T12:53:05.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.314+0000 7f7fbc975700 1 -- 172.21.15.36:0/1375213201 shutdown_connections 2022-06-17T12:53:05.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.314+0000 7f7fbc975700 1 --2- 172.21.15.36:0/1375213201 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fb80af8d0 0x7f7fb814a8d0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:05.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.314+0000 7f7fbc975700 1 --2- 172.21.15.36:0/1375213201 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fb80b2200 0x7f7fb80af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:05.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.314+0000 7f7fbc975700 1 -- 172.21.15.36:0/1375213201 >> 172.21.15.36:0/1375213201 conn(0x7f7fb805f770 msgr2=0x7f7fb805fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:05.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.314+0000 7f7fbc975700 1 -- 172.21.15.36:0/1375213201 shutdown_connections 2022-06-17T12:53:05.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.314+0000 7f7fbc975700 1 -- 172.21.15.36:0/1375213201 wait complete. 2022-06-17T12:53:05.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.314+0000 7f7fbc975700 1 Processor -- start 2022-06-17T12:53:05.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.314+0000 7f7fbc975700 1 -- start start 2022-06-17T12:53:05.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.314+0000 7f7fbc975700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fb80af8d0 0x7f7fb80644d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:05.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.314+0000 7f7fbc975700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fb80b2200 0x7f7fb8062530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:05.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.314+0000 7f7fbc975700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fb80b2cd0 0x7f7fb8066cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:05.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.314+0000 7f7fbc975700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb8063af0 con 0x7f7fb80b2cd0 2022-06-17T12:53:05.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.314+0000 7f7fbc975700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb8063d90 con 0x7f7fb80b2200 2022-06-17T12:53:05.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.314+0000 7f7fbc975700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb8064030 con 0x7f7fb80af8d0 2022-06-17T12:53:05.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.314+0000 7f7fb77fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fb80af8d0 0x7f7fb80644d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:05.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.314+0000 7f7fb7fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fb80b2cd0 0x7f7fb8066cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:05.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.314+0000 7f7fb77fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fb80af8d0 0x7f7fb80644d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:50324/0 (socket says 172.21.15.36:50324) 2022-06-17T12:53:05.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.314+0000 7f7fb77fe700 1 -- 172.21.15.36:0/536856919 learned_addr learned my addr 172.21.15.36:0/536856919 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:05.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.318+0000 7f7fb6ffd700 1 --2- 172.21.15.36:0/536856919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fb80b2200 0x7f7fb8062530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:05.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.318+0000 7f7fb7fff700 1 -- 172.21.15.36:0/536856919 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fb80af8d0 msgr2=0x7f7fb80644d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:05.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.318+0000 7f7fb7fff700 1 --2- 172.21.15.36:0/536856919 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fb80af8d0 0x7f7fb80644d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:05.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.318+0000 7f7fb7fff700 1 -- 172.21.15.36:0/536856919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fb80b2200 msgr2=0x7f7fb8062530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:05.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.318+0000 7f7fb7fff700 1 --2- 172.21.15.36:0/536856919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fb80b2200 0x7f7fb8062530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:05.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.318+0000 7f7fb7fff700 1 -- 172.21.15.36:0/536856919 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa8009530 con 0x7f7fb80b2cd0 2022-06-17T12:53:05.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.318+0000 7f7fb7fff700 1 --2- 172.21.15.36:0/536856919 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fb80b2cd0 0x7f7fb8066cb0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f7fb80afcd0 tx=0x7f7fb0007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:05.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.318+0000 7f7fb4ff9700 1 -- 172.21.15.36:0/536856919 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fb000cc70 con 0x7f7fb80b2cd0 2022-06-17T12:53:05.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.318+0000 7f7fb4ff9700 1 -- 172.21.15.36:0/536856919 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb000cdd0 con 0x7f7fb80b2cd0 2022-06-17T12:53:05.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.318+0000 7f7fbc975700 1 -- 172.21.15.36:0/536856919 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb8062a20 con 0x7f7fb80b2cd0 2022-06-17T12:53:05.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.318+0000 7f7fb4ff9700 1 -- 172.21.15.36:0/536856919 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fb0007220 con 0x7f7fb80b2cd0 2022-06-17T12:53:05.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.318+0000 7f7fbc975700 1 -- 172.21.15.36:0/536856919 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb80a9690 con 0x7f7fb80b2cd0 2022-06-17T12:53:05.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.318+0000 7f7fbc975700 1 -- 172.21.15.36:0/536856919 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb80a9dd0 con 0x7f7fb80b2cd0 2022-06-17T12:53:05.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.318+0000 7f7fb4ff9700 1 -- 172.21.15.36:0/536856919 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f7fb0006510 con 0x7f7fb80b2cd0 2022-06-17T12:53:05.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.318+0000 7f7fb4ff9700 1 --2- 172.21.15.36:0/536856919 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7f98053ad0 0x7f7f98055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:05.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.318+0000 7f7fb4ff9700 1 -- 172.21.15.36:0/536856919 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7fb006b740 con 0x7f7fb80b2cd0 2022-06-17T12:53:05.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.318+0000 7f7fb77fe700 1 --2- 172.21.15.36:0/536856919 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7f98053ad0 0x7f7f98055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:05.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.322+0000 7f7fb77fe700 1 --2- 172.21.15.36:0/536856919 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7f98053ad0 0x7f7f98055f90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f7fac008000 tx=0x7f7fac008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:05.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.326+0000 7f7fb4ff9700 1 -- 172.21.15.36:0/536856919 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7fb80a9dd0 con 0x7f7fb80b2cd0 2022-06-17T12:53:05.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.450+0000 7f7fbc975700 1 -- 172.21.15.36:0/536856919 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7fb80a9dd0 con 0x7f7fb80b2cd0 2022-06-17T12:53:05.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.454+0000 7f7fb4ff9700 1 -- 172.21.15.36:0/536856919 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f7fb0025fb0 con 0x7f7fb80b2cd0 2022-06-17T12:53:05.439 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:53:05.440 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":73,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:53:04.907315+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:53:03.265804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-06-17T12:53:04.385158+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:53:05.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.454+0000 7f7fbc975700 1 -- 172.21.15.36:0/536856919 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7f98053ad0 msgr2=0x7f7f98055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:05.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.454+0000 7f7fbc975700 1 --2- 172.21.15.36:0/536856919 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7f98053ad0 0x7f7f98055f90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f7fac008000 tx=0x7f7fac008040).stop 2022-06-17T12:53:05.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.454+0000 7f7fbc975700 1 -- 172.21.15.36:0/536856919 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fb80b2cd0 msgr2=0x7f7fb8066cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:05.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.454+0000 7f7fbc975700 1 --2- 172.21.15.36:0/536856919 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fb80b2cd0 0x7f7fb8066cb0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f7fb80afcd0 tx=0x7f7fb0007b90).stop 2022-06-17T12:53:05.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.454+0000 7f7fbc975700 1 -- 172.21.15.36:0/536856919 shutdown_connections 2022-06-17T12:53:05.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.454+0000 7f7fbc975700 1 --2- 172.21.15.36:0/536856919 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7f98053ad0 0x7f7f98055f90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:05.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.454+0000 7f7fbc975700 1 --2- 172.21.15.36:0/536856919 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fb80b2cd0 0x7f7fb8066cb0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:05.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.454+0000 7f7fbc975700 1 --2- 172.21.15.36:0/536856919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fb80b2200 0x7f7fb8062530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:05.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.454+0000 7f7fbc975700 1 --2- 172.21.15.36:0/536856919 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fb80af8d0 0x7f7fb80644d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:05.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.454+0000 7f7fbc975700 1 -- 172.21.15.36:0/536856919 >> 172.21.15.36:0/536856919 conn(0x7f7fb805f770 msgr2=0x7f7fb80b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:05.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.454+0000 7f7fbc975700 1 -- 172.21.15.36:0/536856919 shutdown_connections 2022-06-17T12:53:05.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.454+0000 7f7fbc975700 1 -- 172.21.15.36:0/536856919 wait complete. 2022-06-17T12:53:05.455 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-17T12:53:05.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.594+0000 7fde3908f700 1 Processor -- start 2022-06-17T12:53:05.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.594+0000 7fde3908f700 1 -- start start 2022-06-17T12:53:05.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde3908f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde34060ac0 0x7fde34140260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:05.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde3908f700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde34060ea0 con 0x7fde34067260 2022-06-17T12:53:05.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde3908f700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde340aebb0 con 0x7fde340b2c80 2022-06-17T12:53:05.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde3908f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde340b2210 con 0x7fde34060ac0 2022-06-17T12:53:05.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde3888e700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fde34067260 0x7fde340af540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53072/0 (socket says 172.21.15.36:53072) 2022-06-17T12:53:05.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde3888e700 1 -- 172.21.15.36:0/1584620821 learned_addr learned my addr 172.21.15.36:0/1584620821 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:53:05.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde33fff700 1 --2- 172.21.15.36:0/1584620821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde34060ac0 0x7fde34140260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:05.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde337fe700 1 -- 172.21.15.36:0/1584620821 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 875093782 0 0) 0x7fde34060ea0 con 0x7fde34067260 2022-06-17T12:53:05.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde337fe700 1 -- 172.21.15.36:0/1584620821 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde14003260 con 0x7fde34067260 2022-06-17T12:53:05.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde337fe700 1 -- 172.21.15.36:0/1584620821 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2499707694 0 0) 0x7fde340aebb0 con 0x7fde340b2c80 2022-06-17T12:53:05.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde337fe700 1 -- 172.21.15.36:0/1584620821 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde34060ea0 con 0x7fde340b2c80 2022-06-17T12:53:05.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde337fe700 1 -- 172.21.15.36:0/1584620821 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 89291511 0 0) 0x7fde14003260 con 0x7fde34067260 2022-06-17T12:53:05.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde337fe700 1 -- 172.21.15.36:0/1584620821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde34060ac0 msgr2=0x7fde34140260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:05.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde337fe700 1 --2- 172.21.15.36:0/1584620821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde34060ac0 0x7fde34140260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:05.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde337fe700 1 -- 172.21.15.36:0/1584620821 >> v1:172.21.15.70:6789/0 conn(0x7fde340b2c80 legacy=0x7fde340aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:05.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde337fe700 1 -- 172.21.15.36:0/1584620821 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde340b1ad0 con 0x7fde34067260 2022-06-17T12:53:05.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde337fe700 1 -- 172.21.15.36:0/1584620821 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7fde2c0019e0 con 0x7fde34067260 2022-06-17T12:53:05.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde337fe700 1 -- 172.21.15.36:0/1584620821 >> v1:172.21.15.36:6789/0 conn(0x7fde34067260 legacy=0x7fde340af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:05.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde337fe700 1 --2- 172.21.15.36:0/1584620821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde34060ac0 0x7fde14003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:05.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde337fe700 1 --2- 172.21.15.36:0/1584620821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde14003e20 0x7fde14004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:05.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde337fe700 1 --2- 172.21.15.36:0/1584620821 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde140049f0 0x7fde14004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:05.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde337fe700 1 -- 172.21.15.36:0/1584620821 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde340b2210 con 0x7fde140049f0 2022-06-17T12:53:05.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde337fe700 1 -- 172.21.15.36:0/1584620821 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1400b800 con 0x7fde14003e20 2022-06-17T12:53:05.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde337fe700 1 -- 172.21.15.36:0/1584620821 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1400b9c0 con 0x7fde34060ac0 2022-06-17T12:53:05.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde3888e700 1 --2- 172.21.15.36:0/1584620821 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde140049f0 0x7fde14004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:05.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde33fff700 1 --2- 172.21.15.36:0/1584620821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde34060ac0 0x7fde14003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:05.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde2bfff700 1 --2- 172.21.15.36:0/1584620821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde14003e20 0x7fde14004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:05.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde3888e700 1 -- 172.21.15.36:0/1584620821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde34060ac0 msgr2=0x7fde14003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:05.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde3888e700 1 --2- 172.21.15.36:0/1584620821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde34060ac0 0x7fde14003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:05.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde3888e700 1 -- 172.21.15.36:0/1584620821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde14003e20 msgr2=0x7fde14004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:05.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde3888e700 1 --2- 172.21.15.36:0/1584620821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde14003e20 0x7fde14004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:05.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde3888e700 1 -- 172.21.15.36:0/1584620821 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde14003260 con 0x7fde140049f0 2022-06-17T12:53:05.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde33fff700 1 --2- 172.21.15.36:0/1584620821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde34060ac0 0x7fde14003910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:53:05.602 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde3888e700 1 --2- 172.21.15.36:0/1584620821 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde140049f0 0x7fde14004e70 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7fde2c0017b0 tx=0x7fde2c002150).ready entity=mon.0 client_cookie=68f7549ae7cb79b8 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:05.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde337fe700 1 -- 172.21.15.36:0/1584620821 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde2c00cc40 con 0x7fde140049f0 2022-06-17T12:53:05.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde337fe700 1 -- 172.21.15.36:0/1584620821 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2c00cda0 con 0x7fde140049f0 2022-06-17T12:53:05.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde337fe700 1 -- 172.21.15.36:0/1584620821 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde2c005b80 con 0x7fde140049f0 2022-06-17T12:53:05.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde3908f700 1 -- 172.21.15.36:0/1584620821 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde140049f0 msgr2=0x7fde14004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:05.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde3908f700 1 --2- 172.21.15.36:0/1584620821 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde140049f0 0x7fde14004e70 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7fde2c0017b0 tx=0x7fde2c002150).stop 2022-06-17T12:53:05.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde3888e700 1 -- 172.21.15.36:0/1584620821 reap_dead start 2022-06-17T12:53:05.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde3908f700 1 -- 172.21.15.36:0/1584620821 shutdown_connections 2022-06-17T12:53:05.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.598+0000 7fde3908f700 1 -- 172.21.15.36:0/1584620821 >> 172.21.15.36:0/1584620821 conn(0x7fde340b64f0 msgr2=0x7fde340b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:05.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.602+0000 7fde3908f700 1 -- 172.21.15.36:0/1584620821 shutdown_connections 2022-06-17T12:53:05.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.602+0000 7fde3908f700 1 -- 172.21.15.36:0/1584620821 wait complete. 2022-06-17T12:53:05.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.602+0000 7fde3908f700 1 Processor -- start 2022-06-17T12:53:05.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.602+0000 7fde3908f700 1 -- start start 2022-06-17T12:53:05.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.602+0000 7fde3908f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde140049f0 0x7fde34143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:05.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.602+0000 7fde3908f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde341520f0 0x7fde3413b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:05.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.602+0000 7fde3908f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde34152570 0x7fde3413bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:05.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.602+0000 7fde3908f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde34141720 con 0x7fde341520f0 2022-06-17T12:53:05.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.602+0000 7fde3908f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde34143610 con 0x7fde140049f0 2022-06-17T12:53:05.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.602+0000 7fde3908f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3413e420 con 0x7fde34152570 2022-06-17T12:53:05.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.602+0000 7fde2bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde341520f0 0x7fde3413b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:05.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.602+0000 7fde2bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde341520f0 0x7fde3413b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35554/0 (socket says 172.21.15.36:35554) 2022-06-17T12:53:05.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.602+0000 7fde2bfff700 1 -- 172.21.15.36:0/1448656483 learned_addr learned my addr 172.21.15.36:0/1448656483 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:05.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.602+0000 7fde3888e700 1 --2- 172.21.15.36:0/1448656483 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde34152570 0x7fde3413bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:05.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.602+0000 7fde33fff700 1 --2- 172.21.15.36:0/1448656483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde140049f0 0x7fde34143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:05.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.606+0000 7fde2bfff700 1 -- 172.21.15.36:0/1448656483 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde34152570 msgr2=0x7fde3413bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:05.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.606+0000 7fde2bfff700 1 --2- 172.21.15.36:0/1448656483 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde34152570 0x7fde3413bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:05.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.606+0000 7fde2bfff700 1 -- 172.21.15.36:0/1448656483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde140049f0 msgr2=0x7fde34143a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:05.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.606+0000 7fde2bfff700 1 --2- 172.21.15.36:0/1448656483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde140049f0 0x7fde34143a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:05.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.606+0000 7fde2bfff700 1 -- 172.21.15.36:0/1448656483 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3413e180 con 0x7fde341520f0 2022-06-17T12:53:05.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.606+0000 7fde3888e700 1 --2- 172.21.15.36:0/1448656483 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde34152570 0x7fde3413bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:53:05.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.606+0000 7fde2bfff700 1 --2- 172.21.15.36:0/1448656483 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde341520f0 0x7fde3413b890 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7fde1c00d000 tx=0x7fde1c007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:05.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.606+0000 7fde31ffb700 1 -- 172.21.15.36:0/1448656483 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde1c011780 con 0x7fde341520f0 2022-06-17T12:53:05.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.606+0000 7fde31ffb700 1 -- 172.21.15.36:0/1448656483 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1c0118e0 con 0x7fde341520f0 2022-06-17T12:53:05.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.606+0000 7fde3908f700 1 -- 172.21.15.36:0/1448656483 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde34143f20 con 0x7fde341520f0 2022-06-17T12:53:05.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.606+0000 7fde31ffb700 1 -- 172.21.15.36:0/1448656483 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde1c011c40 con 0x7fde341520f0 2022-06-17T12:53:05.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.606+0000 7fde3908f700 1 -- 172.21.15.36:0/1448656483 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde34141050 con 0x7fde341520f0 2022-06-17T12:53:05.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.606+0000 7fde31ffb700 1 -- 172.21.15.36:0/1448656483 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fde1c01b030 con 0x7fde341520f0 2022-06-17T12:53:05.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.606+0000 7fde3908f700 1 -- 172.21.15.36:0/1448656483 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde341440b0 con 0x7fde341520f0 2022-06-17T12:53:05.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.606+0000 7fde31ffb700 1 --2- 172.21.15.36:0/1448656483 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde18053a40 0x7fde18055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:05.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.606+0000 7fde31ffb700 1 -- 172.21.15.36:0/1448656483 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fde1c06c150 con 0x7fde341520f0 2022-06-17T12:53:05.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.606+0000 7fde33fff700 1 --2- 172.21.15.36:0/1448656483 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde18053a40 0x7fde18055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:05.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.610+0000 7fde33fff700 1 --2- 172.21.15.36:0/1448656483 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde18053a40 0x7fde18055f00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7fde2400b000 tx=0x7fde24008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:05.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.614+0000 7fde31ffb700 1 -- 172.21.15.36:0/1448656483 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fde341440b0 con 0x7fde341520f0 2022-06-17T12:53:05.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:05.774+0000 7fde3908f700 1 -- 172.21.15.36:0/1448656483 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fde341440b0 con 0x7fde341520f0 2022-06-17T12:53:06.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.522+0000 7fde31ffb700 1 -- 172.21.15.36:0/1448656483 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v36) v1 ==== 118+0+0 (secure 0 0 0) 0x7fde341440b0 con 0x7fde341520f0 2022-06-17T12:53:06.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.526+0000 7fde3908f700 1 -- 172.21.15.36:0/1448656483 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde18053a40 msgr2=0x7fde18055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:06.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.526+0000 7fde3908f700 1 --2- 172.21.15.36:0/1448656483 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde18053a40 0x7fde18055f00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7fde2400b000 tx=0x7fde24008040).stop 2022-06-17T12:53:06.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.526+0000 7fde3908f700 1 -- 172.21.15.36:0/1448656483 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde341520f0 msgr2=0x7fde3413b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:06.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.526+0000 7fde3908f700 1 --2- 172.21.15.36:0/1448656483 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde341520f0 0x7fde3413b890 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7fde1c00d000 tx=0x7fde1c007a00).stop 2022-06-17T12:53:06.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.526+0000 7fde3908f700 1 -- 172.21.15.36:0/1448656483 shutdown_connections 2022-06-17T12:53:06.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.526+0000 7fde3908f700 1 --2- 172.21.15.36:0/1448656483 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde18053a40 0x7fde18055f00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:06.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.526+0000 7fde3908f700 1 --2- 172.21.15.36:0/1448656483 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde34152570 0x7fde3413bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:06.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.526+0000 7fde3908f700 1 --2- 172.21.15.36:0/1448656483 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde341520f0 0x7fde3413b890 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:06.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.526+0000 7fde3908f700 1 --2- 172.21.15.36:0/1448656483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde140049f0 0x7fde34143a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:06.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.526+0000 7fde3908f700 1 -- 172.21.15.36:0/1448656483 >> 172.21.15.36:0/1448656483 conn(0x7fde340b64f0 msgr2=0x7fde340aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:06.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.526+0000 7fde3908f700 1 -- 172.21.15.36:0/1448656483 shutdown_connections 2022-06-17T12:53:06.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.526+0000 7fde3908f700 1 -- 172.21.15.36:0/1448656483 wait complete. 2022-06-17T12:53:06.527 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:53:06.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.654+0000 7fe13759e700 1 Processor -- start 2022-06-17T12:53:06.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.654+0000 7fe13759e700 1 -- start start 2022-06-17T12:53:06.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.654+0000 7fe13759e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1380b1930 0x7fe138065e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:06.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.654+0000 7fe13759e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe1380b2900 0x7fe13813eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:06.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.654+0000 7fe13759e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe138063b10 con 0x7fe138063730 2022-06-17T12:53:06.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.654+0000 7fe13759e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe138065430 con 0x7fe1380b2900 2022-06-17T12:53:06.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.654+0000 7fe13759e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1380656d0 con 0x7fe1380b1930 2022-06-17T12:53:06.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.654+0000 7fe13659c700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe138063730 0x7fe1381431f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:48824/0 (socket says 172.21.15.36:48824) 2022-06-17T12:53:06.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.654+0000 7fe13659c700 1 -- 172.21.15.36:0/3632326249 learned_addr learned my addr 172.21.15.36:0/3632326249 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:53:06.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.654+0000 7fe135d9b700 1 --2- 172.21.15.36:0/3632326249 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1380b1930 0x7fe138065e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:06.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.654+0000 7fe136d9d700 1 --2- 172.21.15.36:0/3632326249 >> v2:172.21.15.70:3300/0 conn(0x7fe1380b2900 0x7fe13813eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:06.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.654+0000 7fe13559a700 1 -- 172.21.15.36:0/3632326249 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2119802974 0 0) 0x7fe138063b10 con 0x7fe138063730 2022-06-17T12:53:06.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.654+0000 7fe13559a700 1 -- 172.21.15.36:0/3632326249 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe11c003260 con 0x7fe138063730 2022-06-17T12:53:06.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.654+0000 7fe135d9b700 1 -- 172.21.15.36:0/3632326249 >> v2:172.21.15.70:3300/0 conn(0x7fe1380b2900 msgr2=0x7fe13813eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:06.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.654+0000 7fe135d9b700 1 --2- 172.21.15.36:0/3632326249 >> v2:172.21.15.70:3300/0 conn(0x7fe1380b2900 0x7fe13813eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:06.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.654+0000 7fe135d9b700 1 -- 172.21.15.36:0/3632326249 >> v1:172.21.15.70:6789/0 conn(0x7fe138063730 legacy=0x7fe1381431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:06.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.654+0000 7fe135d9b700 1 -- 172.21.15.36:0/3632326249 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe138065970 con 0x7fe1380b1930 2022-06-17T12:53:06.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.654+0000 7fe135d9b700 1 --2- 172.21.15.36:0/3632326249 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1380b1930 0x7fe138065e10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fe1280032f0 tx=0x7fe1280091d0).ready entity=mon.2 client_cookie=7e488db5a17ee264 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:06.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.654+0000 7fe13559a700 1 -- 172.21.15.36:0/3632326249 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe12800b600 con 0x7fe1380b1930 2022-06-17T12:53:06.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.654+0000 7fe13559a700 1 -- 172.21.15.36:0/3632326249 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12800b760 con 0x7fe1380b1930 2022-06-17T12:53:06.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.654+0000 7fe13559a700 1 -- 172.21.15.36:0/3632326249 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe12800b8d0 con 0x7fe1380b1930 2022-06-17T12:53:06.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.658+0000 7fe13759e700 1 -- 172.21.15.36:0/3632326249 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1380b1930 msgr2=0x7fe138065e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:06.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.658+0000 7fe13759e700 1 --2- 172.21.15.36:0/3632326249 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1380b1930 0x7fe138065e10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fe1280032f0 tx=0x7fe1280091d0).stop 2022-06-17T12:53:06.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.658+0000 7fe13759e700 1 -- 172.21.15.36:0/3632326249 shutdown_connections 2022-06-17T12:53:06.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.658+0000 7fe13759e700 1 --2- 172.21.15.36:0/3632326249 >> v2:172.21.15.70:3300/0 conn(0x7fe1380b2900 0x7fe13813eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:06.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.658+0000 7fe13759e700 1 --2- 172.21.15.36:0/3632326249 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1380b1930 0x7fe138065e10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:06.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.658+0000 7fe13759e700 1 -- 172.21.15.36:0/3632326249 >> 172.21.15.36:0/3632326249 conn(0x7fe1380614b0 msgr2=0x7fe1380618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:06.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.658+0000 7fe13759e700 1 -- 172.21.15.36:0/3632326249 shutdown_connections 2022-06-17T12:53:06.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.658+0000 7fe13759e700 1 -- 172.21.15.36:0/3632326249 wait complete. 2022-06-17T12:53:06.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.658+0000 7fe13759e700 1 Processor -- start 2022-06-17T12:53:06.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.658+0000 7fe13759e700 1 -- start start 2022-06-17T12:53:06.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.662+0000 7fe13759e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe138063730 0x7fe1380b5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:06.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.662+0000 7fe13759e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1380b1930 0x7fe1380adc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:06.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.662+0000 7fe13759e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe1380b2900 0x7fe1380ae110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:06.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.662+0000 7fe13759e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1380ad8c0 con 0x7fe1380b2900 2022-06-17T12:53:06.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.662+0000 7fe13759e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13813a6c0 con 0x7fe138063730 2022-06-17T12:53:06.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.662+0000 7fe13759e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1380b07a0 con 0x7fe1380b1930 2022-06-17T12:53:06.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.662+0000 7fe136d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe1380b2900 0x7fe1380ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:06.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.662+0000 7fe136d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe1380b2900 0x7fe1380ae110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35570/0 (socket says 172.21.15.36:35570) 2022-06-17T12:53:06.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.662+0000 7fe136d9d700 1 -- 172.21.15.36:0/3578202201 learned_addr learned my addr 172.21.15.36:0/3578202201 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:06.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.662+0000 7fe13659c700 1 --2- 172.21.15.36:0/3578202201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe138063730 0x7fe1380b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:06.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.662+0000 7fe135d9b700 1 --2- 172.21.15.36:0/3578202201 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1380b1930 0x7fe1380adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:06.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.662+0000 7fe136d9d700 1 -- 172.21.15.36:0/3578202201 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1380b1930 msgr2=0x7fe1380adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:06.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.662+0000 7fe136d9d700 1 --2- 172.21.15.36:0/3578202201 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1380b1930 0x7fe1380adc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:06.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.662+0000 7fe136d9d700 1 -- 172.21.15.36:0/3578202201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe138063730 msgr2=0x7fe1380b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:06.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.662+0000 7fe136d9d700 1 --2- 172.21.15.36:0/3578202201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe138063730 0x7fe1380b5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:06.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.662+0000 7fe136d9d700 1 -- 172.21.15.36:0/3578202201 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe128009470 con 0x7fe1380b2900 2022-06-17T12:53:06.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.662+0000 7fe136d9d700 1 --2- 172.21.15.36:0/3578202201 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe1380b2900 0x7fe1380ae110 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7fe13000c000 tx=0x7fe130006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:06.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.662+0000 7fe12f7fe700 1 -- 172.21.15.36:0/3578202201 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe130011750 con 0x7fe1380b2900 2022-06-17T12:53:06.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.662+0000 7fe13759e700 1 -- 172.21.15.36:0/3578202201 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1380b0500 con 0x7fe1380b2900 2022-06-17T12:53:06.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.662+0000 7fe12f7fe700 1 -- 172.21.15.36:0/3578202201 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1300118b0 con 0x7fe1380b2900 2022-06-17T12:53:06.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.662+0000 7fe12f7fe700 1 -- 172.21.15.36:0/3578202201 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe130011c10 con 0x7fe1380b2900 2022-06-17T12:53:06.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.662+0000 7fe13759e700 1 -- 172.21.15.36:0/3578202201 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe138143460 con 0x7fe1380b2900 2022-06-17T12:53:06.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.666+0000 7fe13759e700 1 -- 172.21.15.36:0/3578202201 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe114003410 con 0x7fe1380b2900 2022-06-17T12:53:06.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.666+0000 7fe12f7fe700 1 -- 172.21.15.36:0/3578202201 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe130011d70 con 0x7fe1380b2900 2022-06-17T12:53:06.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.666+0000 7fe12f7fe700 1 --2- 172.21.15.36:0/3578202201 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe110053a40 0x7fe110055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:06.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.666+0000 7fe12f7fe700 1 -- 172.21.15.36:0/3578202201 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe13006c490 con 0x7fe1380b2900 2022-06-17T12:53:06.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.666+0000 7fe13659c700 1 --2- 172.21.15.36:0/3578202201 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe110053a40 0x7fe110055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:06.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.666+0000 7fe13659c700 1 --2- 172.21.15.36:0/3578202201 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe110053a40 0x7fe110055f00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fe12000d000 tx=0x7fe120009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:06.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.674+0000 7fe12f7fe700 1 -- 172.21.15.36:0/3578202201 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe130043100 con 0x7fe1380b2900 2022-06-17T12:53:06.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.838+0000 7fe13759e700 1 -- 172.21.15.36:0/3578202201 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe114003c40 con 0x7fe1380b2900 2022-06-17T12:53:06.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.838+0000 7fe12f7fe700 1 -- 172.21.15.36:0/3578202201 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+2140 (secure 0 0 0) 0x7fe130046720 con 0x7fe1380b2900 2022-06-17T12:53:06.823 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:53:06.823 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":36,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/2541998896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2541998896},{"type":"v1","addr":"172.21.15.70:6833","nonce":2541998896}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:53:04.907795+0000","modified":"2022-06-17T12:53:06.511529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6381":{"gid":6381,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/2514005995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2514005995},{"type":"v1","addr":"172.21.15.36:6835","nonce":2514005995}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-06-17T12:53:06.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.838+0000 7fe13759e700 1 -- 172.21.15.36:0/3578202201 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe110053a40 msgr2=0x7fe110055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:06.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.838+0000 7fe13759e700 1 --2- 172.21.15.36:0/3578202201 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe110053a40 0x7fe110055f00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fe12000d000 tx=0x7fe120009040).stop 2022-06-17T12:53:06.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.838+0000 7fe13759e700 1 -- 172.21.15.36:0/3578202201 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe1380b2900 msgr2=0x7fe1380ae110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:06.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.838+0000 7fe13759e700 1 --2- 172.21.15.36:0/3578202201 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe1380b2900 0x7fe1380ae110 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7fe13000c000 tx=0x7fe130006df0).stop 2022-06-17T12:53:06.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.838+0000 7fe13759e700 1 -- 172.21.15.36:0/3578202201 shutdown_connections 2022-06-17T12:53:06.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.838+0000 7fe13759e700 1 --2- 172.21.15.36:0/3578202201 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe110053a40 0x7fe110055f00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:06.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.838+0000 7fe13759e700 1 --2- 172.21.15.36:0/3578202201 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe1380b2900 0x7fe1380ae110 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:06.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.838+0000 7fe13759e700 1 --2- 172.21.15.36:0/3578202201 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1380b1930 0x7fe1380adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:06.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.838+0000 7fe13759e700 1 --2- 172.21.15.36:0/3578202201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe138063730 0x7fe1380b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:06.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.838+0000 7fe13759e700 1 -- 172.21.15.36:0/3578202201 >> 172.21.15.36:0/3578202201 conn(0x7fe1380614b0 msgr2=0x7fe13813fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:06.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.842+0000 7fe13759e700 1 -- 172.21.15.36:0/3578202201 shutdown_connections 2022-06-17T12:53:06.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.842+0000 7fe13759e700 1 -- 172.21.15.36:0/3578202201 wait complete. 2022-06-17T12:53:06.831 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 36 2022-06-17T12:53:06.837 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:53:06.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.962+0000 7f2f89bc7700 1 Processor -- start 2022-06-17T12:53:06.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.962+0000 7f2f89bc7700 1 -- start start 2022-06-17T12:53:06.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.962+0000 7f2f89bc7700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2f840b2cd0 0x7f2f841465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:06.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.962+0000 7f2f89bc7700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2f84146ae0 0x7f2f84146f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:06.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.962+0000 7f2f89bc7700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f840b25e0 con 0x7f2f840b2200 2022-06-17T12:53:06.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.962+0000 7f2f89bc7700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f840ad790 con 0x7f2f84146ae0 2022-06-17T12:53:06.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.962+0000 7f2f89bc7700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f840aef40 con 0x7f2f840b2cd0 2022-06-17T12:53:06.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.962+0000 7f2f893c6700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2f84146ae0 0x7f2f84146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:06.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.962+0000 7f2f893c6700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2f84146ae0 0x7f2f84146f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35576/0 (socket says 172.21.15.36:35576) 2022-06-17T12:53:06.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.962+0000 7f2f893c6700 1 -- 172.21.15.36:0/3464397981 learned_addr learned my addr 172.21.15.36:0/3464397981 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:06.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.962+0000 7f2f83fff700 1 --2- 172.21.15.36:0/3464397981 >> v2:172.21.15.70:3300/0 conn(0x7f2f840b2cd0 0x7f2f841465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:06.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.966+0000 7f2f83fff700 1 -- 172.21.15.36:0/3464397981 >> v2:172.21.15.36:3300/0 conn(0x7f2f84146ae0 msgr2=0x7f2f84146f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:06.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.966+0000 7f2f83fff700 1 --2- 172.21.15.36:0/3464397981 >> v2:172.21.15.36:3300/0 conn(0x7f2f84146ae0 0x7f2f84146f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:06.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.966+0000 7f2f83fff700 1 -- 172.21.15.36:0/3464397981 >> v1:172.21.15.70:6789/0 conn(0x7f2f840b2200 legacy=0x7f2f840af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:06.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.966+0000 7f2f83fff700 1 -- 172.21.15.36:0/3464397981 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f840afaf0 con 0x7f2f840b2cd0 2022-06-17T12:53:06.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.966+0000 7f2f83fff700 1 --2- 172.21.15.36:0/3464397981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f840b2cd0 0x7f2f841465f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f2f74002f70 tx=0x7f2f74006df0).ready entity=mon.1 client_cookie=63748d2963f0c60 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:06.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.966+0000 7f2f837fe700 1 -- 172.21.15.36:0/3464397981 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f7400d570 con 0x7f2f840b2cd0 2022-06-17T12:53:06.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.966+0000 7f2f837fe700 1 -- 172.21.15.36:0/3464397981 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f74000e90 con 0x7f2f840b2cd0 2022-06-17T12:53:06.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.966+0000 7f2f837fe700 1 -- 172.21.15.36:0/3464397981 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f7400d8c0 con 0x7f2f840b2cd0 2022-06-17T12:53:06.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.966+0000 7f2f89bc7700 1 -- 172.21.15.36:0/3464397981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f840b2cd0 msgr2=0x7f2f841465f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:06.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.966+0000 7f2f89bc7700 1 --2- 172.21.15.36:0/3464397981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f840b2cd0 0x7f2f841465f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f2f74002f70 tx=0x7f2f74006df0).stop 2022-06-17T12:53:06.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.966+0000 7f2f89bc7700 1 -- 172.21.15.36:0/3464397981 shutdown_connections 2022-06-17T12:53:06.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.966+0000 7f2f89bc7700 1 --2- 172.21.15.36:0/3464397981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f840b2cd0 0x7f2f841465f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:06.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.966+0000 7f2f89bc7700 1 --2- 172.21.15.36:0/3464397981 >> v2:172.21.15.36:3300/0 conn(0x7f2f84146ae0 0x7f2f84146f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:06.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.966+0000 7f2f89bc7700 1 -- 172.21.15.36:0/3464397981 >> 172.21.15.36:0/3464397981 conn(0x7f2f8405f770 msgr2=0x7f2f8405fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:06.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.966+0000 7f2f89bc7700 1 -- 172.21.15.36:0/3464397981 shutdown_connections 2022-06-17T12:53:06.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.966+0000 7f2f89bc7700 1 -- 172.21.15.36:0/3464397981 wait complete. 2022-06-17T12:53:06.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.966+0000 7f2f89bc7700 1 Processor -- start 2022-06-17T12:53:06.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.966+0000 7f2f89bc7700 1 -- start start 2022-06-17T12:53:06.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.970+0000 7f2f89bc7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f840b2200 0x7f2f8414ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:06.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.970+0000 7f2f89bc7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f840b2cd0 0x7f2f84062390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:06.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.970+0000 7f2f89bc7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f84146ae0 0x7f2f841571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:06.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.970+0000 7f2f89bc7700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f84156b00 con 0x7f2f840b2200 2022-06-17T12:53:06.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.970+0000 7f2f89bc7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f840a70e0 con 0x7f2f84146ae0 2022-06-17T12:53:06.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.970+0000 7f2f89bc7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f84156d30 con 0x7f2f840b2cd0 2022-06-17T12:53:06.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.970+0000 7f2f88bc5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f840b2200 0x7f2f8414ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:06.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.970+0000 7f2f88bc5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f840b2200 0x7f2f8414ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35578/0 (socket says 172.21.15.36:35578) 2022-06-17T12:53:06.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.970+0000 7f2f88bc5700 1 -- 172.21.15.36:0/812847558 learned_addr learned my addr 172.21.15.36:0/812847558 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:06.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.970+0000 7f2f893c6700 1 --2- 172.21.15.36:0/812847558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f84146ae0 0x7f2f841571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:06.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.970+0000 7f2f83fff700 1 --2- 172.21.15.36:0/812847558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f840b2cd0 0x7f2f84062390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:06.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.970+0000 7f2f88bc5700 1 -- 172.21.15.36:0/812847558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f840b2cd0 msgr2=0x7f2f84062390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:06.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.970+0000 7f2f88bc5700 1 --2- 172.21.15.36:0/812847558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f840b2cd0 0x7f2f84062390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:06.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.970+0000 7f2f88bc5700 1 -- 172.21.15.36:0/812847558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f84146ae0 msgr2=0x7f2f841571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:06.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.970+0000 7f2f88bc5700 1 --2- 172.21.15.36:0/812847558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f84146ae0 0x7f2f841571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:06.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.970+0000 7f2f88bc5700 1 -- 172.21.15.36:0/812847558 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f7400b180 con 0x7f2f840b2200 2022-06-17T12:53:06.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.970+0000 7f2f88bc5700 1 --2- 172.21.15.36:0/812847558 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f840b2200 0x7f2f8414ac20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f2f8406ad20 tx=0x7f2f78007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:06.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.970+0000 7f2f81ffb700 1 -- 172.21.15.36:0/812847558 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f78012400 con 0x7f2f840b2200 2022-06-17T12:53:06.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.970+0000 7f2f81ffb700 1 -- 172.21.15.36:0/812847558 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f78012560 con 0x7f2f840b2200 2022-06-17T12:53:06.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.970+0000 7f2f81ffb700 1 -- 172.21.15.36:0/812847558 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f780128c0 con 0x7f2f840b2200 2022-06-17T12:53:06.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.970+0000 7f2f89bc7700 1 -- 172.21.15.36:0/812847558 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f840a7380 con 0x7f2f840b2200 2022-06-17T12:53:06.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.970+0000 7f2f89bc7700 1 -- 172.21.15.36:0/812847558 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f840ac1f0 con 0x7f2f840b2200 2022-06-17T12:53:06.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.970+0000 7f2f89bc7700 1 -- 172.21.15.36:0/812847558 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f84063d10 con 0x7f2f840b2200 2022-06-17T12:53:06.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.974+0000 7f2f81ffb700 1 -- 172.21.15.36:0/812847558 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2f78019030 con 0x7f2f840b2200 2022-06-17T12:53:06.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.974+0000 7f2f81ffb700 1 --2- 172.21.15.36:0/812847558 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2f64053af0 0x7f2f64055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:06.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.974+0000 7f2f81ffb700 1 -- 172.21.15.36:0/812847558 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2f7806b3f0 con 0x7f2f840b2200 2022-06-17T12:53:06.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.974+0000 7f2f83fff700 1 --2- 172.21.15.36:0/812847558 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2f64053af0 0x7f2f64055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:06.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.974+0000 7f2f83fff700 1 --2- 172.21.15.36:0/812847558 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2f64053af0 0x7f2f64055fb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f2f7401a000 tx=0x7f2f74014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:06.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:06.978+0000 7f2f81ffb700 1 -- 172.21.15.36:0/812847558 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2f84063d10 con 0x7f2f840b2200 2022-06-17T12:53:07.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.110+0000 7f2f89bc7700 1 -- 172.21.15.36:0/812847558 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f84063d10 con 0x7f2f840b2200 2022-06-17T12:53:07.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.110+0000 7f2f81ffb700 1 -- 172.21.15.36:0/812847558 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f2f84063d10 con 0x7f2f840b2200 2022-06-17T12:53:07.095 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:53:07.096 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":74,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:53:05.910998+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:53:03.265804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-06-17T12:53:04.385158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:53:07.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.110+0000 7f2f89bc7700 1 -- 172.21.15.36:0/812847558 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2f64053af0 msgr2=0x7f2f64055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:07.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.110+0000 7f2f89bc7700 1 --2- 172.21.15.36:0/812847558 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2f64053af0 0x7f2f64055fb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f2f7401a000 tx=0x7f2f74014000).stop 2022-06-17T12:53:07.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.110+0000 7f2f89bc7700 1 -- 172.21.15.36:0/812847558 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f840b2200 msgr2=0x7f2f8414ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:07.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.110+0000 7f2f89bc7700 1 --2- 172.21.15.36:0/812847558 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f840b2200 0x7f2f8414ac20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f2f8406ad20 tx=0x7f2f78007c50).stop 2022-06-17T12:53:07.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.110+0000 7f2f89bc7700 1 -- 172.21.15.36:0/812847558 shutdown_connections 2022-06-17T12:53:07.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.110+0000 7f2f89bc7700 1 --2- 172.21.15.36:0/812847558 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2f64053af0 0x7f2f64055fb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.110+0000 7f2f89bc7700 1 --2- 172.21.15.36:0/812847558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f84146ae0 0x7f2f841571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.110+0000 7f2f89bc7700 1 --2- 172.21.15.36:0/812847558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f840b2cd0 0x7f2f84062390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.110+0000 7f2f89bc7700 1 --2- 172.21.15.36:0/812847558 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f840b2200 0x7f2f8414ac20 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.110+0000 7f2f89bc7700 1 -- 172.21.15.36:0/812847558 >> 172.21.15.36:0/812847558 conn(0x7f2f8405f770 msgr2=0x7f2f840acbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:07.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.114+0000 7f2f89bc7700 1 -- 172.21.15.36:0/812847558 shutdown_connections 2022-06-17T12:53:07.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.114+0000 7f2f89bc7700 1 -- 172.21.15.36:0/812847558 wait complete. 2022-06-17T12:53:07.110 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:53:07.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.242+0000 7ff5cc986700 1 Processor -- start 2022-06-17T12:53:07.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.242+0000 7ff5cc986700 1 -- start start 2022-06-17T12:53:07.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.246+0000 7ff5cc986700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff5c80aee30 0x7ff5c80ada40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:07.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.246+0000 7ff5cc986700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c8060ea0 con 0x7ff5c8063940 2022-06-17T12:53:07.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.246+0000 7ff5cc986700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c8062f60 con 0x7ff5c8060ac0 2022-06-17T12:53:07.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.246+0000 7ff5cc986700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c813fa00 con 0x7ff5c80aee30 2022-06-17T12:53:07.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.246+0000 7ff5bffff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7ff5c8063940 0x7ff5c813e950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53118/0 (socket says 172.21.15.36:53118) 2022-06-17T12:53:07.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.246+0000 7ff5bffff700 1 -- 172.21.15.36:0/853516491 learned_addr learned my addr 172.21.15.36:0/853516491 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:53:07.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.246+0000 7ff5c7fff700 1 --2- 172.21.15.36:0/853516491 >> v2:172.21.15.70:3300/0 conn(0x7ff5c80aee30 0x7ff5c80ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:07.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.246+0000 7ff5c6ffd700 1 -- 172.21.15.36:0/853516491 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800462124 0 0) 0x7ff5c8060ea0 con 0x7ff5c8063940 2022-06-17T12:53:07.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.246+0000 7ff5c6ffd700 1 -- 172.21.15.36:0/853516491 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5a8003260 con 0x7ff5c8063940 2022-06-17T12:53:07.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.246+0000 7ff5c7fff700 1 -- 172.21.15.36:0/853516491 >> v1:172.21.15.70:6789/0 conn(0x7ff5c8060ac0 legacy=0x7ff5c80ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:07.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.246+0000 7ff5c6ffd700 1 -- 172.21.15.36:0/853516491 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3964348928 0 0) 0x7ff5a8003260 con 0x7ff5c8063940 2022-06-17T12:53:07.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.246+0000 7ff5c7fff700 1 -- 172.21.15.36:0/853516491 >> v1:172.21.15.36:6789/0 conn(0x7ff5c8063940 legacy=0x7ff5c813e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:07.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.246+0000 7ff5c7fff700 1 -- 172.21.15.36:0/853516491 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c8063200 con 0x7ff5c80aee30 2022-06-17T12:53:07.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.246+0000 7ff5c7fff700 1 --2- 172.21.15.36:0/853516491 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c80aee30 0x7ff5c80ada40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7ff5b0001800 tx=0x7ff5c00091d0).ready entity=mon.1 client_cookie=81a3b9f941807d7f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:07.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.246+0000 7ff5c6ffd700 1 -- 172.21.15.36:0/853516491 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5b0001660 con 0x7ff5c80aee30 2022-06-17T12:53:07.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.246+0000 7ff5c6ffd700 1 -- 172.21.15.36:0/853516491 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c000b580 con 0x7ff5c80aee30 2022-06-17T12:53:07.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.246+0000 7ff5c6ffd700 1 -- 172.21.15.36:0/853516491 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5c000b6f0 con 0x7ff5c80aee30 2022-06-17T12:53:07.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.246+0000 7ff5cc986700 1 -- 172.21.15.36:0/853516491 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c80aee30 msgr2=0x7ff5c80ada40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:07.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.246+0000 7ff5cc986700 1 --2- 172.21.15.36:0/853516491 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c80aee30 0x7ff5c80ada40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7ff5b0001800 tx=0x7ff5c00091d0).stop 2022-06-17T12:53:07.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.246+0000 7ff5cc986700 1 -- 172.21.15.36:0/853516491 shutdown_connections 2022-06-17T12:53:07.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.246+0000 7ff5cc986700 1 --2- 172.21.15.36:0/853516491 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c80aee30 0x7ff5c80ada40 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.246+0000 7ff5cc986700 1 -- 172.21.15.36:0/853516491 >> 172.21.15.36:0/853516491 conn(0x7ff5c80b65f0 msgr2=0x7ff5c80b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:07.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.246+0000 7ff5cc986700 1 -- 172.21.15.36:0/853516491 shutdown_connections 2022-06-17T12:53:07.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.246+0000 7ff5cc986700 1 -- 172.21.15.36:0/853516491 wait complete. 2022-06-17T12:53:07.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5cc986700 1 Processor -- start 2022-06-17T12:53:07.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5cc986700 1 -- start start 2022-06-17T12:53:07.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5cc986700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c8060ac0 0x7ff5c81406a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:07.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5cc986700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff5c8063940 0x7ff5c8065170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:07.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5cc986700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff5c80aee30 0x7ff5c8065660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:07.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5cc986700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c8064420 con 0x7ff5c80aee30 2022-06-17T12:53:07.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5cc986700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c813c0b0 con 0x7ff5c8060ac0 2022-06-17T12:53:07.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5cc986700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c813d440 con 0x7ff5c8063940 2022-06-17T12:53:07.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5c7fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff5c80aee30 0x7ff5c8065660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:07.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5c7fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff5c80aee30 0x7ff5c8065660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35596/0 (socket says 172.21.15.36:35596) 2022-06-17T12:53:07.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5c7fff700 1 -- 172.21.15.36:0/2860367494 learned_addr learned my addr 172.21.15.36:0/2860367494 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:07.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5bffff700 1 --2- 172.21.15.36:0/2860367494 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff5c8063940 0x7ff5c8065170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:07.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5c77fe700 1 --2- 172.21.15.36:0/2860367494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c8060ac0 0x7ff5c81406a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:07.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5c7fff700 1 -- 172.21.15.36:0/2860367494 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff5c8063940 msgr2=0x7ff5c8065170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:07.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5c7fff700 1 --2- 172.21.15.36:0/2860367494 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff5c8063940 0x7ff5c8065170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5c7fff700 1 -- 172.21.15.36:0/2860367494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c8060ac0 msgr2=0x7ff5c81406a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:07.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5c7fff700 1 --2- 172.21.15.36:0/2860367494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c8060ac0 0x7ff5c81406a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5c7fff700 1 -- 172.21.15.36:0/2860367494 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c0009470 con 0x7ff5c80aee30 2022-06-17T12:53:07.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5c77fe700 1 --2- 172.21.15.36:0/2860367494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c8060ac0 0x7ff5c81406a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:53:07.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5c7fff700 1 --2- 172.21.15.36:0/2860367494 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff5c80aee30 0x7ff5c8065660 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7ff5b0001800 tx=0x7ff5c0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:07.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5c57fa700 1 -- 172.21.15.36:0/2860367494 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5c000b6f0 con 0x7ff5c80aee30 2022-06-17T12:53:07.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5c57fa700 1 -- 172.21.15.36:0/2860367494 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c00032d0 con 0x7ff5c80aee30 2022-06-17T12:53:07.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5cc986700 1 -- 172.21.15.36:0/2860367494 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c813c630 con 0x7ff5c80aee30 2022-06-17T12:53:07.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5c57fa700 1 -- 172.21.15.36:0/2860367494 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5c0021630 con 0x7ff5c80aee30 2022-06-17T12:53:07.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.250+0000 7ff5cc986700 1 -- 172.21.15.36:0/2860367494 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c813b560 con 0x7ff5c80aee30 2022-06-17T12:53:07.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.254+0000 7ff5cc986700 1 -- 172.21.15.36:0/2860367494 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c813c240 con 0x7ff5c80aee30 2022-06-17T12:53:07.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.254+0000 7ff5c57fa700 1 -- 172.21.15.36:0/2860367494 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff5c000f070 con 0x7ff5c80aee30 2022-06-17T12:53:07.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.254+0000 7ff5c57fa700 1 --2- 172.21.15.36:0/2860367494 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff5ac053940 0x7ff5ac055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:07.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.254+0000 7ff5c57fa700 1 -- 172.21.15.36:0/2860367494 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff5c006d040 con 0x7ff5c80aee30 2022-06-17T12:53:07.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.254+0000 7ff5c77fe700 1 --2- 172.21.15.36:0/2860367494 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff5ac053940 0x7ff5ac055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:07.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.254+0000 7ff5c77fe700 1 --2- 172.21.15.36:0/2860367494 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff5ac053940 0x7ff5ac055e00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7ff5b800b000 tx=0x7ff5b8009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:07.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.258+0000 7ff5c57fa700 1 -- 172.21.15.36:0/2860367494 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff5c0043e40 con 0x7ff5c80aee30 2022-06-17T12:53:07.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.426+0000 7ff5cc986700 1 -- 172.21.15.36:0/2860367494 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7ff5c813d580 con 0x7ff5c80aee30 2022-06-17T12:53:07.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.434+0000 7ff5c57fa700 1 -- 172.21.15.36:0/2860367494 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v25) v1 ==== 184+0+0 (secure 0 0 0) 0x7ff5c00150e0 con 0x7ff5c80aee30 2022-06-17T12:53:07.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.434+0000 7ff5cc986700 1 -- 172.21.15.36:0/2860367494 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff5ac053940 msgr2=0x7ff5ac055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:07.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.434+0000 7ff5cc986700 1 --2- 172.21.15.36:0/2860367494 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff5ac053940 0x7ff5ac055e00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7ff5b800b000 tx=0x7ff5b8009040).stop 2022-06-17T12:53:07.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.434+0000 7ff5cc986700 1 -- 172.21.15.36:0/2860367494 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff5c80aee30 msgr2=0x7ff5c8065660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:07.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.434+0000 7ff5cc986700 1 --2- 172.21.15.36:0/2860367494 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff5c80aee30 0x7ff5c8065660 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7ff5b0001800 tx=0x7ff5c0016000).stop 2022-06-17T12:53:07.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.434+0000 7ff5cc986700 1 -- 172.21.15.36:0/2860367494 shutdown_connections 2022-06-17T12:53:07.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.434+0000 7ff5cc986700 1 --2- 172.21.15.36:0/2860367494 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff5ac053940 0x7ff5ac055e00 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.434+0000 7ff5cc986700 1 --2- 172.21.15.36:0/2860367494 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff5c80aee30 0x7ff5c8065660 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.434+0000 7ff5cc986700 1 --2- 172.21.15.36:0/2860367494 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff5c8063940 0x7ff5c8065170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.434+0000 7ff5cc986700 1 --2- 172.21.15.36:0/2860367494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c8060ac0 0x7ff5c81406a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.434+0000 7ff5cc986700 1 -- 172.21.15.36:0/2860367494 >> 172.21.15.36:0/2860367494 conn(0x7ff5c80b65f0 msgr2=0x7ff5c81453c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:07.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.434+0000 7ff5cc986700 1 -- 172.21.15.36:0/2860367494 shutdown_connections 2022-06-17T12:53:07.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.434+0000 7ff5cc986700 1 -- 172.21.15.36:0/2860367494 wait complete. 2022-06-17T12:53:07.427 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.0 2022-06-17T12:53:07.435 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:53:07.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.570+0000 7f8805180700 1 Processor -- start 2022-06-17T12:53:07.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.570+0000 7f8805180700 1 -- start start 2022-06-17T12:53:07.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.570+0000 7f8805180700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f88000b1890 0x7f880013c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:07.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.570+0000 7f8805180700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f88000b1ff0 0x7f880013f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:07.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.570+0000 7f8805180700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88000b0230 con 0x7f8800060ac0 2022-06-17T12:53:07.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.570+0000 7f8805180700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8800143800 con 0x7f88000b1ff0 2022-06-17T12:53:07.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.570+0000 7f8805180700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8800060ea0 con 0x7f88000b1890 2022-06-17T12:53:07.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.570+0000 7f880497f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f88000b1ff0 0x7f880013f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:07.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.570+0000 7f880497f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f88000b1ff0 0x7f880013f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35604/0 (socket says 172.21.15.36:35604) 2022-06-17T12:53:07.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.570+0000 7f880497f700 1 -- 172.21.15.36:0/872332404 learned_addr learned my addr 172.21.15.36:0/872332404 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:07.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.570+0000 7f87ff7fe700 1 --2- 172.21.15.36:0/872332404 >> v2:172.21.15.70:3300/0 conn(0x7f88000b1890 0x7f880013c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:07.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.570+0000 7f880497f700 1 -- 172.21.15.36:0/872332404 >> v2:172.21.15.70:3300/0 conn(0x7f88000b1890 msgr2=0x7f880013c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:07.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.570+0000 7f880497f700 1 --2- 172.21.15.36:0/872332404 >> v2:172.21.15.70:3300/0 conn(0x7f88000b1890 0x7f880013c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.570+0000 7f880497f700 1 -- 172.21.15.36:0/872332404 >> v1:172.21.15.70:6789/0 conn(0x7f8800060ac0 legacy=0x7f880013c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:07.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.570+0000 7f880497f700 1 -- 172.21.15.36:0/872332404 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88000afcf0 con 0x7f88000b1ff0 2022-06-17T12:53:07.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.574+0000 7f880497f700 1 --2- 172.21.15.36:0/872332404 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88000b1ff0 0x7f880013f270 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7f87f8002f70 tx=0x7f87f8006eb0).ready entity=mon.0 client_cookie=657f94f2201a944f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:07.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.574+0000 7f87feffd700 1 -- 172.21.15.36:0/872332404 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87f800d5c0 con 0x7f88000b1ff0 2022-06-17T12:53:07.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.574+0000 7f87feffd700 1 -- 172.21.15.36:0/872332404 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f8000e90 con 0x7f88000b1ff0 2022-06-17T12:53:07.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.574+0000 7f87feffd700 1 -- 172.21.15.36:0/872332404 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87f800d950 con 0x7f88000b1ff0 2022-06-17T12:53:07.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.574+0000 7f8805180700 1 -- 172.21.15.36:0/872332404 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88000b1ff0 msgr2=0x7f880013f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:07.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.574+0000 7f8805180700 1 --2- 172.21.15.36:0/872332404 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88000b1ff0 0x7f880013f270 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7f87f8002f70 tx=0x7f87f8006eb0).stop 2022-06-17T12:53:07.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.574+0000 7f8805180700 1 -- 172.21.15.36:0/872332404 shutdown_connections 2022-06-17T12:53:07.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.574+0000 7f8805180700 1 --2- 172.21.15.36:0/872332404 >> v2:172.21.15.70:3300/0 conn(0x7f88000b1890 0x7f880013c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.574+0000 7f8805180700 1 --2- 172.21.15.36:0/872332404 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88000b1ff0 0x7f880013f270 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.574+0000 7f8805180700 1 -- 172.21.15.36:0/872332404 >> 172.21.15.36:0/872332404 conn(0x7f88000b65f0 msgr2=0x7f88000b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:07.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.574+0000 7f8805180700 1 -- 172.21.15.36:0/872332404 shutdown_connections 2022-06-17T12:53:07.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.574+0000 7f8805180700 1 -- 172.21.15.36:0/872332404 wait complete. 2022-06-17T12:53:07.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.574+0000 7f8805180700 1 Processor -- start 2022-06-17T12:53:07.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.574+0000 7f8805180700 1 -- start start 2022-06-17T12:53:07.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.578+0000 7f8805180700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8800060ac0 0x7f88000650d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:07.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.578+0000 7f8805180700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88000b1890 0x7f8800062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:07.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.578+0000 7f8805180700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88000b1ff0 0x7f8800066720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:07.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.578+0000 7f8805180700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8800140090 con 0x7f8800060ac0 2022-06-17T12:53:07.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.578+0000 7f8805180700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880013fdf0 con 0x7f88000b1ff0 2022-06-17T12:53:07.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.578+0000 7f8805180700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f880013e420 con 0x7f88000b1890 2022-06-17T12:53:07.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.578+0000 7f87fffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8800060ac0 0x7f88000650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:07.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.578+0000 7f87fffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8800060ac0 0x7f88000650d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35606/0 (socket says 172.21.15.36:35606) 2022-06-17T12:53:07.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.578+0000 7f87fffff700 1 -- 172.21.15.36:0/3595697762 learned_addr learned my addr 172.21.15.36:0/3595697762 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:07.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.578+0000 7f880497f700 1 --2- 172.21.15.36:0/3595697762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88000b1ff0 0x7f8800066720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:07.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.578+0000 7f87ff7fe700 1 --2- 172.21.15.36:0/3595697762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88000b1890 0x7f8800062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:07.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.578+0000 7f87fffff700 1 -- 172.21.15.36:0/3595697762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88000b1890 msgr2=0x7f8800062d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:07.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.578+0000 7f87fffff700 1 --2- 172.21.15.36:0/3595697762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88000b1890 0x7f8800062d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.578+0000 7f87fffff700 1 -- 172.21.15.36:0/3595697762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88000b1ff0 msgr2=0x7f8800066720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:07.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.578+0000 7f87fffff700 1 --2- 172.21.15.36:0/3595697762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88000b1ff0 0x7f8800066720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.578+0000 7f87fffff700 1 -- 172.21.15.36:0/3595697762 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f800b1e0 con 0x7f8800060ac0 2022-06-17T12:53:07.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.578+0000 7f87fffff700 1 --2- 172.21.15.36:0/3595697762 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8800060ac0 0x7f88000650d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f88000622a0 tx=0x7f87f0007f60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:07.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.578+0000 7f87fd7fa700 1 -- 172.21.15.36:0/3595697762 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87f0006aa0 con 0x7f8800060ac0 2022-06-17T12:53:07.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.578+0000 7f87fd7fa700 1 -- 172.21.15.36:0/3595697762 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f0006c00 con 0x7f8800060ac0 2022-06-17T12:53:07.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.578+0000 7f8805180700 1 -- 172.21.15.36:0/3595697762 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f880013e8c0 con 0x7f8800060ac0 2022-06-17T12:53:07.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.578+0000 7f87fd7fa700 1 -- 172.21.15.36:0/3595697762 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87f0007030 con 0x7f8800060ac0 2022-06-17T12:53:07.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.578+0000 7f8805180700 1 -- 172.21.15.36:0/3595697762 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880013f5a0 con 0x7f8800060ac0 2022-06-17T12:53:07.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.582+0000 7f8805180700 1 -- 172.21.15.36:0/3595697762 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f880013e5e0 con 0x7f8800060ac0 2022-06-17T12:53:07.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.582+0000 7f87fd7fa700 1 -- 172.21.15.36:0/3595697762 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f87f001b060 con 0x7f8800060ac0 2022-06-17T12:53:07.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.582+0000 7f87fd7fa700 1 --2- 172.21.15.36:0/3595697762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f87e0053ad0 0x7f87e0055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:07.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.582+0000 7f87fd7fa700 1 -- 172.21.15.36:0/3595697762 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f87f006bd80 con 0x7f8800060ac0 2022-06-17T12:53:07.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.582+0000 7f87ff7fe700 1 --2- 172.21.15.36:0/3595697762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f87e0053ad0 0x7f87e0055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:07.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.582+0000 7f87ff7fe700 1 --2- 172.21.15.36:0/3595697762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f87e0053ad0 0x7f87e0055f90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f87f400b000 tx=0x7f87f4006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:07.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.586+0000 7f87fd7fa700 1 -- 172.21.15.36:0/3595697762 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f87f0042ae0 con 0x7f8800060ac0 2022-06-17T12:53:07.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.750+0000 7f8805180700 1 -- 172.21.15.36:0/3595697762 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f88000aff90 con 0x7f8800060ac0 2022-06-17T12:53:07.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.758+0000 7f87fd7fa700 1 -- 172.21.15.36:0/3595697762 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v26) v1 ==== 184+0+0 (secure 0 0 0) 0x7f87f006d020 con 0x7f8800060ac0 2022-06-17T12:53:07.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.762+0000 7f8805180700 1 -- 172.21.15.36:0/3595697762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f87e0053ad0 msgr2=0x7f87e0055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:07.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.762+0000 7f8805180700 1 --2- 172.21.15.36:0/3595697762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f87e0053ad0 0x7f87e0055f90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f87f400b000 tx=0x7f87f4006df0).stop 2022-06-17T12:53:07.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.762+0000 7f8805180700 1 -- 172.21.15.36:0/3595697762 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8800060ac0 msgr2=0x7f88000650d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:07.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.762+0000 7f8805180700 1 --2- 172.21.15.36:0/3595697762 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8800060ac0 0x7f88000650d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f88000622a0 tx=0x7f87f0007f60).stop 2022-06-17T12:53:07.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.762+0000 7f8805180700 1 -- 172.21.15.36:0/3595697762 shutdown_connections 2022-06-17T12:53:07.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.762+0000 7f8805180700 1 --2- 172.21.15.36:0/3595697762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f87e0053ad0 0x7f87e0055f90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.762+0000 7f8805180700 1 --2- 172.21.15.36:0/3595697762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88000b1ff0 0x7f8800066720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.762+0000 7f8805180700 1 --2- 172.21.15.36:0/3595697762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88000b1890 0x7f8800062d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.762+0000 7f8805180700 1 --2- 172.21.15.36:0/3595697762 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8800060ac0 0x7f88000650d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.762+0000 7f8805180700 1 -- 172.21.15.36:0/3595697762 >> 172.21.15.36:0/3595697762 conn(0x7f88000b65f0 msgr2=0x7f88000ad960 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:07.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.762+0000 7f8805180700 1 -- 172.21.15.36:0/3595697762 shutdown_connections 2022-06-17T12:53:07.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.762+0000 7f8805180700 1 -- 172.21.15.36:0/3595697762 wait complete. 2022-06-17T12:53:07.756 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.1 2022-06-17T12:53:07.767 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:53:07.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.906+0000 7f8184842700 1 Processor -- start 2022-06-17T12:53:07.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.906+0000 7f8184842700 1 -- start start 2022-06-17T12:53:07.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.906+0000 7f8184842700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8180062270 0x7f8180142f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:07.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.906+0000 7f8184842700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81800b0cb0 0x7f8180143770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:07.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.906+0000 7f8184842700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81800b44b0 con 0x7f8180143c60 2022-06-17T12:53:07.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.906+0000 7f8184842700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81800b6650 con 0x7f8180062270 2022-06-17T12:53:07.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.906+0000 7f8184842700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8180062650 con 0x7f81800b0cb0 2022-06-17T12:53:07.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.906+0000 7f817e59c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8180062270 0x7f8180142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:07.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.906+0000 7f817ed9d700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f8180143c60 0x7f8180146970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53148/0 (socket says 172.21.15.36:53148) 2022-06-17T12:53:07.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.906+0000 7f817ed9d700 1 -- 172.21.15.36:0/974564820 learned_addr learned my addr 172.21.15.36:0/974564820 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:53:07.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.906+0000 7f8175d9b700 1 --2- 172.21.15.36:0/974564820 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81800b0cb0 0x7f8180143770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:07.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.906+0000 7f817e59c700 1 -- 172.21.15.36:0/974564820 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81800b0cb0 msgr2=0x7f8180143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:07.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.906+0000 7f817e59c700 1 --2- 172.21.15.36:0/974564820 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81800b0cb0 0x7f8180143770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.906+0000 7f817e59c700 1 -- 172.21.15.36:0/974564820 >> v1:172.21.15.36:6789/0 conn(0x7f8180143c60 legacy=0x7f8180146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:07.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.906+0000 7f817e59c700 1 -- 172.21.15.36:0/974564820 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81800b4210 con 0x7f8180062270 2022-06-17T12:53:07.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.906+0000 7f8175d9b700 1 --2- 172.21.15.36:0/974564820 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81800b0cb0 0x7f8180143770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:53:07.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.906+0000 7f817e59c700 1 --2- 172.21.15.36:0/974564820 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8180062270 0x7f8180142f20 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f8170002f70 tx=0x7f8170006df0).ready entity=mon.0 client_cookie=1cd750d9049cc8f6 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:07.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.906+0000 7f817dd9b700 1 -- 172.21.15.36:0/974564820 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f817000d570 con 0x7f8180062270 2022-06-17T12:53:07.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.906+0000 7f817dd9b700 1 -- 172.21.15.36:0/974564820 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8170000e90 con 0x7f8180062270 2022-06-17T12:53:07.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.906+0000 7f817dd9b700 1 -- 172.21.15.36:0/974564820 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f817000d8c0 con 0x7f8180062270 2022-06-17T12:53:07.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.910+0000 7f8184842700 1 -- 172.21.15.36:0/974564820 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8180062270 msgr2=0x7f8180142f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:07.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.910+0000 7f8184842700 1 --2- 172.21.15.36:0/974564820 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8180062270 0x7f8180142f20 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f8170002f70 tx=0x7f8170006df0).stop 2022-06-17T12:53:07.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.910+0000 7f8184842700 1 -- 172.21.15.36:0/974564820 shutdown_connections 2022-06-17T12:53:07.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.910+0000 7f8184842700 1 --2- 172.21.15.36:0/974564820 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81800b0cb0 0x7f8180143770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.910+0000 7f8184842700 1 --2- 172.21.15.36:0/974564820 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8180062270 0x7f8180142f20 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.910+0000 7f8184842700 1 -- 172.21.15.36:0/974564820 >> 172.21.15.36:0/974564820 conn(0x7f81800b9870 msgr2=0x7f81800b9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:07.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.910+0000 7f8184842700 1 -- 172.21.15.36:0/974564820 shutdown_connections 2022-06-17T12:53:07.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.910+0000 7f8184842700 1 -- 172.21.15.36:0/974564820 wait complete. 2022-06-17T12:53:07.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.910+0000 7f8184842700 1 Processor -- start 2022-06-17T12:53:07.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.910+0000 7f8184842700 1 -- start start 2022-06-17T12:53:07.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.910+0000 7f8184842700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8180062270 0x7f81801565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:07.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.910+0000 7f8184842700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8180143c60 0x7f8180142160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:07.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.910+0000 7f8184842700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81800ad330 0x7f81800aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:07.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.910+0000 7f8184842700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8180155bf0 con 0x7f8180062270 2022-06-17T12:53:07.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.910+0000 7f8184842700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8180155e90 con 0x7f81800ad330 2022-06-17T12:53:07.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.910+0000 7f8184842700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8180156130 con 0x7f8180143c60 2022-06-17T12:53:07.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.914+0000 7f817e59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8180062270 0x7f81801565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:07.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.914+0000 7f817e59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8180062270 0x7f81801565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35622/0 (socket says 172.21.15.36:35622) 2022-06-17T12:53:07.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.914+0000 7f817e59c700 1 -- 172.21.15.36:0/2736785649 learned_addr learned my addr 172.21.15.36:0/2736785649 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:07.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.914+0000 7f8175d9b700 1 --2- 172.21.15.36:0/2736785649 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8180143c60 0x7f8180142160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:07.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.914+0000 7f817ed9d700 1 --2- 172.21.15.36:0/2736785649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81800ad330 0x7f81800aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:07.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.914+0000 7f817e59c700 1 -- 172.21.15.36:0/2736785649 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8180143c60 msgr2=0x7f8180142160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:07.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.914+0000 7f817e59c700 1 --2- 172.21.15.36:0/2736785649 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8180143c60 0x7f8180142160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.914+0000 7f817e59c700 1 -- 172.21.15.36:0/2736785649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81800ad330 msgr2=0x7f81800aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:07.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.914+0000 7f817e59c700 1 --2- 172.21.15.36:0/2736785649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81800ad330 0x7f81800aec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:07.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.914+0000 7f817e59c700 1 -- 172.21.15.36:0/2736785649 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817000b180 con 0x7f8180062270 2022-06-17T12:53:07.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.914+0000 7f817ed9d700 1 --2- 172.21.15.36:0/2736785649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81800ad330 0x7f81800aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:53:07.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.914+0000 7f817e59c700 1 --2- 172.21.15.36:0/2736785649 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8180062270 0x7f81801565d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7f8170002f70 tx=0x7f8170015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:07.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.914+0000 7f8177fff700 1 -- 172.21.15.36:0/2736785649 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f817000bed0 con 0x7f8180062270 2022-06-17T12:53:07.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.914+0000 7f8177fff700 1 -- 172.21.15.36:0/2736785649 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817000de30 con 0x7f8180062270 2022-06-17T12:53:07.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.914+0000 7f8184842700 1 -- 172.21.15.36:0/2736785649 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8180146e50 con 0x7f8180062270 2022-06-17T12:53:07.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.914+0000 7f8177fff700 1 -- 172.21.15.36:0/2736785649 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8170011e20 con 0x7f8180062270 2022-06-17T12:53:07.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.914+0000 7f8184842700 1 -- 172.21.15.36:0/2736785649 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81800af180 con 0x7f8180062270 2022-06-17T12:53:07.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.914+0000 7f8184842700 1 -- 172.21.15.36:0/2736785649 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81800b63b0 con 0x7f8180062270 2022-06-17T12:53:07.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.914+0000 7f8177fff700 1 -- 172.21.15.36:0/2736785649 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f817001a040 con 0x7f8180062270 2022-06-17T12:53:07.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.914+0000 7f8177fff700 1 --2- 172.21.15.36:0/2736785649 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f81640539b0 0x7f8164055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:07.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.914+0000 7f8177fff700 1 -- 172.21.15.36:0/2736785649 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f817006cf80 con 0x7f8180062270 2022-06-17T12:53:07.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.918+0000 7f8175d9b700 1 --2- 172.21.15.36:0/2736785649 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f81640539b0 0x7f8164055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:07.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.918+0000 7f8175d9b700 1 --2- 172.21.15.36:0/2736785649 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f81640539b0 0x7f8164055e70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f816800c000 tx=0x7f8168008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:07.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:07.926+0000 7f8177fff700 1 -- 172.21.15.36:0/2736785649 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8170043ce0 con 0x7f8180062270 2022-06-17T12:53:08.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.086+0000 7f8184842700 1 -- 172.21.15.36:0/2736785649 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f8180057410 con 0x7f8180062270 2022-06-17T12:53:08.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.094+0000 7f8177fff700 1 -- 172.21.15.36:0/2736785649 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v27) v1 ==== 184+0+0 (secure 0 0 0) 0x7f8170023070 con 0x7f8180062270 2022-06-17T12:53:08.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.098+0000 7f8184842700 1 -- 172.21.15.36:0/2736785649 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f81640539b0 msgr2=0x7f8164055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:08.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.098+0000 7f8184842700 1 --2- 172.21.15.36:0/2736785649 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f81640539b0 0x7f8164055e70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f816800c000 tx=0x7f8168008040).stop 2022-06-17T12:53:08.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.098+0000 7f8184842700 1 -- 172.21.15.36:0/2736785649 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8180062270 msgr2=0x7f81801565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:08.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.098+0000 7f8184842700 1 --2- 172.21.15.36:0/2736785649 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8180062270 0x7f81801565d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7f8170002f70 tx=0x7f8170015000).stop 2022-06-17T12:53:08.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.098+0000 7f8184842700 1 -- 172.21.15.36:0/2736785649 shutdown_connections 2022-06-17T12:53:08.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.098+0000 7f8184842700 1 --2- 172.21.15.36:0/2736785649 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f81640539b0 0x7f8164055e70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:08.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.098+0000 7f8184842700 1 --2- 172.21.15.36:0/2736785649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81800ad330 0x7f81800aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:08.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.098+0000 7f8184842700 1 --2- 172.21.15.36:0/2736785649 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8180143c60 0x7f8180142160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:08.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.098+0000 7f8184842700 1 --2- 172.21.15.36:0/2736785649 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8180062270 0x7f81801565d0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:08.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.098+0000 7f8184842700 1 -- 172.21.15.36:0/2736785649 >> 172.21.15.36:0/2736785649 conn(0x7f81800b9870 msgr2=0x7f81800b8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:08.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.098+0000 7f8184842700 1 -- 172.21.15.36:0/2736785649 shutdown_connections 2022-06-17T12:53:08.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.098+0000 7f8184842700 1 -- 172.21.15.36:0/2736785649 wait complete. 2022-06-17T12:53:08.095 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.2 2022-06-17T12:53:08.096 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:53:08.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.218+0000 7ffb4f2f5700 1 Processor -- start 2022-06-17T12:53:08.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.218+0000 7ffb4f2f5700 1 -- start start 2022-06-17T12:53:08.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.218+0000 7ffb4f2f5700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb480aeea0 0x7ffb48064230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:08.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.218+0000 7ffb4f2f5700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ffb480af930 0x7ffb480b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:08.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.218+0000 7ffb4f2f5700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffb480afe30 0x7ffb480b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:08.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.218+0000 7ffb4f2f5700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb480af280 con 0x7ffb480af930 2022-06-17T12:53:08.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.218+0000 7ffb4f2f5700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb480b14f0 con 0x7ffb480afe30 2022-06-17T12:53:08.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.218+0000 7ffb4f2f5700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb48067560 con 0x7ffb480aeea0 2022-06-17T12:53:08.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4daf2700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ffb480af930 0x7ffb480b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:08.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4daf2700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ffb480af930 0x7ffb480b19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35628/0 (socket says 172.21.15.36:35628) 2022-06-17T12:53:08.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4daf2700 1 -- 172.21.15.36:0/3283541841 learned_addr learned my addr 172.21.15.36:0/3283541841 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:08.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4eaf4700 1 --2- 172.21.15.36:0/3283541841 >> v2:172.21.15.70:3300/0 conn(0x7ffb480afe30 0x7ffb480b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:08.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4e2f3700 1 --2- 172.21.15.36:0/3283541841 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb480aeea0 0x7ffb48064230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:08.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4daf2700 1 -- 172.21.15.36:0/3283541841 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb480aeea0 msgr2=0x7ffb48064230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:08.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4daf2700 1 --2- 172.21.15.36:0/3283541841 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb480aeea0 0x7ffb48064230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:08.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4daf2700 1 -- 172.21.15.36:0/3283541841 >> v2:172.21.15.70:3300/0 conn(0x7ffb480afe30 msgr2=0x7ffb480b1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:08.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4daf2700 1 --2- 172.21.15.36:0/3283541841 >> v2:172.21.15.70:3300/0 conn(0x7ffb480afe30 0x7ffb480b1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:08.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4daf2700 1 -- 172.21.15.36:0/3283541841 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb480b23c0 con 0x7ffb480af930 2022-06-17T12:53:08.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4daf2700 1 --2- 172.21.15.36:0/3283541841 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb480af930 0x7ffb480b19e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7ffb44002f70 tx=0x7ffb44006eb0).ready entity=mon.0 client_cookie=4c4f75d9b847b23e server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:08.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4d2f1700 1 -- 172.21.15.36:0/3283541841 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb4400d570 con 0x7ffb480af930 2022-06-17T12:53:08.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4d2f1700 1 -- 172.21.15.36:0/3283541841 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb44000e90 con 0x7ffb480af930 2022-06-17T12:53:08.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4d2f1700 1 -- 172.21.15.36:0/3283541841 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb4400d900 con 0x7ffb480af930 2022-06-17T12:53:08.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4f2f5700 1 -- 172.21.15.36:0/3283541841 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb480af930 msgr2=0x7ffb480b19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:08.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4f2f5700 1 --2- 172.21.15.36:0/3283541841 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb480af930 0x7ffb480b19e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7ffb44002f70 tx=0x7ffb44006eb0).stop 2022-06-17T12:53:08.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4f2f5700 1 -- 172.21.15.36:0/3283541841 shutdown_connections 2022-06-17T12:53:08.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4f2f5700 1 --2- 172.21.15.36:0/3283541841 >> v2:172.21.15.70:3300/0 conn(0x7ffb480afe30 0x7ffb480b1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:08.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4f2f5700 1 --2- 172.21.15.36:0/3283541841 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb480af930 0x7ffb480b19e0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:08.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4f2f5700 1 --2- 172.21.15.36:0/3283541841 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb480aeea0 0x7ffb48064230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:08.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4f2f5700 1 -- 172.21.15.36:0/3283541841 >> 172.21.15.36:0/3283541841 conn(0x7ffb4805f770 msgr2=0x7ffb4805fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:08.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4f2f5700 1 -- 172.21.15.36:0/3283541841 shutdown_connections 2022-06-17T12:53:08.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4f2f5700 1 -- 172.21.15.36:0/3283541841 wait complete. 2022-06-17T12:53:08.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4f2f5700 1 Processor -- start 2022-06-17T12:53:08.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4f2f5700 1 -- start start 2022-06-17T12:53:08.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.222+0000 7ffb4f2f5700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb480aeea0 0x7ffb480a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:08.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.226+0000 7ffb4f2f5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb480af930 0x7ffb480a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:08.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.226+0000 7ffb4f2f5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb480afe30 0x7ffb48153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:08.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.226+0000 7ffb4f2f5700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb48150ba0 con 0x7ffb480af930 2022-06-17T12:53:08.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.226+0000 7ffb4f2f5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb48150e40 con 0x7ffb480afe30 2022-06-17T12:53:08.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.226+0000 7ffb4f2f5700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb480a1520 con 0x7ffb480aeea0 2022-06-17T12:53:08.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.226+0000 7ffb4daf2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb480af930 0x7ffb480a60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:08.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.226+0000 7ffb4daf2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb480af930 0x7ffb480a60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35634/0 (socket says 172.21.15.36:35634) 2022-06-17T12:53:08.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.226+0000 7ffb4daf2700 1 -- 172.21.15.36:0/3098521689 learned_addr learned my addr 172.21.15.36:0/3098521689 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:08.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.226+0000 7ffb4eaf4700 1 --2- 172.21.15.36:0/3098521689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb480afe30 0x7ffb48153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:08.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.226+0000 7ffb4e2f3700 1 --2- 172.21.15.36:0/3098521689 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb480aeea0 0x7ffb480a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:08.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.226+0000 7ffb4daf2700 1 -- 172.21.15.36:0/3098521689 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb480aeea0 msgr2=0x7ffb480a5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:08.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.226+0000 7ffb4daf2700 1 --2- 172.21.15.36:0/3098521689 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb480aeea0 0x7ffb480a5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:08.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.226+0000 7ffb4daf2700 1 -- 172.21.15.36:0/3098521689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb480afe30 msgr2=0x7ffb48153650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:08.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.226+0000 7ffb4daf2700 1 --2- 172.21.15.36:0/3098521689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb480afe30 0x7ffb48153650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:08.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.226+0000 7ffb4daf2700 1 -- 172.21.15.36:0/3098521689 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4400b1e0 con 0x7ffb480af930 2022-06-17T12:53:08.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.226+0000 7ffb4daf2700 1 --2- 172.21.15.36:0/3098521689 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb480af930 0x7ffb480a60b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7ffb44002f70 tx=0x7ffb44015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:08.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.226+0000 7ffb377fe700 1 -- 172.21.15.36:0/3098521689 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb44011860 con 0x7ffb480af930 2022-06-17T12:53:08.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.226+0000 7ffb377fe700 1 -- 172.21.15.36:0/3098521689 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb440119c0 con 0x7ffb480af930 2022-06-17T12:53:08.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.226+0000 7ffb4f2f5700 1 -- 172.21.15.36:0/3098521689 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb480a65a0 con 0x7ffb480af930 2022-06-17T12:53:08.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.226+0000 7ffb377fe700 1 -- 172.21.15.36:0/3098521689 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb44011d20 con 0x7ffb480af930 2022-06-17T12:53:08.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.226+0000 7ffb4f2f5700 1 -- 172.21.15.36:0/3098521689 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb480a2130 con 0x7ffb480af930 2022-06-17T12:53:08.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.226+0000 7ffb4f2f5700 1 -- 172.21.15.36:0/3098521689 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb48150a00 con 0x7ffb480af930 2022-06-17T12:53:08.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.226+0000 7ffb377fe700 1 -- 172.21.15.36:0/3098521689 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ffb44025030 con 0x7ffb480af930 2022-06-17T12:53:08.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.226+0000 7ffb377fe700 1 --2- 172.21.15.36:0/3098521689 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffb280539b0 0x7ffb28055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:08.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.230+0000 7ffb377fe700 1 -- 172.21.15.36:0/3098521689 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ffb4406d370 con 0x7ffb480af930 2022-06-17T12:53:08.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.230+0000 7ffb4e2f3700 1 --2- 172.21.15.36:0/3098521689 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffb280539b0 0x7ffb28055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:08.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.230+0000 7ffb4e2f3700 1 --2- 172.21.15.36:0/3098521689 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffb280539b0 0x7ffb28055e70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7ffb3800c000 tx=0x7ffb38008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:08.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.234+0000 7ffb377fe700 1 -- 172.21.15.36:0/3098521689 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffb48150a00 con 0x7ffb480af930 2022-06-17T12:53:08.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.394+0000 7ffb4f2f5700 1 -- 172.21.15.36:0/3098521689 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb48150a00 con 0x7ffb480af930 2022-06-17T12:53:08.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.394+0000 7ffb377fe700 1 -- 172.21.15.36:0/3098521689 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+2140 (secure 0 0 0) 0x7ffb48150a00 con 0x7ffb480af930 2022-06-17T12:53:08.381 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:53:08.382 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":36,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/2541998896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2541998896},{"type":"v1","addr":"172.21.15.70:6833","nonce":2541998896}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:53:04.907795+0000","modified":"2022-06-17T12:53:06.511529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6381":{"gid":6381,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/2514005995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2514005995},{"type":"v1","addr":"172.21.15.36:6835","nonce":2514005995}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-06-17T12:53:08.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.398+0000 7ffb4f2f5700 1 -- 172.21.15.36:0/3098521689 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffb280539b0 msgr2=0x7ffb28055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:08.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.398+0000 7ffb4f2f5700 1 --2- 172.21.15.36:0/3098521689 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffb280539b0 0x7ffb28055e70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7ffb3800c000 tx=0x7ffb38008040).stop 2022-06-17T12:53:08.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.398+0000 7ffb4f2f5700 1 -- 172.21.15.36:0/3098521689 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb480af930 msgr2=0x7ffb480a60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:08.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.398+0000 7ffb4f2f5700 1 --2- 172.21.15.36:0/3098521689 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb480af930 0x7ffb480a60b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7ffb44002f70 tx=0x7ffb44015000).stop 2022-06-17T12:53:08.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.398+0000 7ffb4f2f5700 1 -- 172.21.15.36:0/3098521689 shutdown_connections 2022-06-17T12:53:08.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.398+0000 7ffb4f2f5700 1 --2- 172.21.15.36:0/3098521689 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffb280539b0 0x7ffb28055e70 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:08.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.398+0000 7ffb4f2f5700 1 --2- 172.21.15.36:0/3098521689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb480afe30 0x7ffb48153650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:08.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.398+0000 7ffb4f2f5700 1 --2- 172.21.15.36:0/3098521689 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffb480af930 0x7ffb480a60b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:08.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.398+0000 7ffb4f2f5700 1 --2- 172.21.15.36:0/3098521689 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb480aeea0 0x7ffb480a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:08.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.398+0000 7ffb4f2f5700 1 -- 172.21.15.36:0/3098521689 >> 172.21.15.36:0/3098521689 conn(0x7ffb4805f770 msgr2=0x7ffb480b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:08.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.398+0000 7ffb4f2f5700 1 -- 172.21.15.36:0/3098521689 shutdown_connections 2022-06-17T12:53:08.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.398+0000 7ffb4f2f5700 1 -- 172.21.15.36:0/3098521689 wait complete. 2022-06-17T12:53:08.390 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 36 2022-06-17T12:53:08.595 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:53:04.907795+0000', 'modified': '2022-06-17T12:53:06.511529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6381}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6381': {'gid': 6381, 'name': 'a', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.36:6835/2514005995', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 2514005995}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 2514005995}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:53:08.596 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:53:08.597 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:53:08.725 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":6381,"want_state":"up:active","state":"up:active","rank_uptime":3.8127749249999998,"mdsmap_epoch":36,"osdmap_epoch":74,"osdmap_epoch_barrier":74,"uptime":8.2076502940000005} 2022-06-17T12:53:08.726 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 6381, "mdsmap_epoch": 36, "osdmap_epoch": 74, "osdmap_epoch_barrier": 74, "rank_uptime": 3.812774925, "state": "up:active", "uptime": 8.207650294, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:53:08.727 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:53:08.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.850+0000 7fddf72dd700 1 Processor -- start 2022-06-17T12:53:08.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.850+0000 7fddf72dd700 1 -- start start 2022-06-17T12:53:08.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf72dd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddf0060ac0 0x7fddf00ad330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:08.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf72dd700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fddf00aee30 0x7fddf013e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:08.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf72dd700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf0060ea0 con 0x7fddf0063940 2022-06-17T12:53:08.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf72dd700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddf013fa00 con 0x7fddf00aee30 2022-06-17T12:53:08.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf72dd700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf0062f60 con 0x7fddf0060ac0 2022-06-17T12:53:08.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf62db700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddf0060ac0 0x7fddf00ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:08.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf6adc700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fddf00aee30 0x7fddf013e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:08.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf62db700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddf0060ac0 0x7fddf00ad330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:50432/0 (socket says 172.21.15.36:50432) 2022-06-17T12:53:08.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf62db700 1 -- 172.21.15.36:0/383654617 learned_addr learned my addr 172.21.15.36:0/383654617 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:08.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf62db700 1 -- 172.21.15.36:0/383654617 >> v2:172.21.15.70:3300/0 conn(0x7fddf00aee30 msgr2=0x7fddf013e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:08.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf62db700 1 --2- 172.21.15.36:0/383654617 >> v2:172.21.15.70:3300/0 conn(0x7fddf00aee30 0x7fddf013e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:08.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf62db700 1 -- 172.21.15.36:0/383654617 >> v1:172.21.15.70:6789/0 conn(0x7fddf0063940 legacy=0x7fddf00ad820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:08.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf62db700 1 -- 172.21.15.36:0/383654617 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf0063200 con 0x7fddf0060ac0 2022-06-17T12:53:08.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf6adc700 1 --2- 172.21.15.36:0/383654617 >> v2:172.21.15.70:3300/0 conn(0x7fddf00aee30 0x7fddf013e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:53:08.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf62db700 1 --2- 172.21.15.36:0/383654617 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddf0060ac0 0x7fddf00ad330 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fddf0062c80 tx=0x7fddec006eb0).ready entity=mon.2 client_cookie=7e3e7ae7f6f608cd server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:08.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf52d9700 1 -- 172.21.15.36:0/383654617 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddec00d5c0 con 0x7fddf0060ac0 2022-06-17T12:53:08.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf52d9700 1 -- 172.21.15.36:0/383654617 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddec000e90 con 0x7fddf0060ac0 2022-06-17T12:53:08.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf52d9700 1 -- 172.21.15.36:0/383654617 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddec00d950 con 0x7fddf0060ac0 2022-06-17T12:53:08.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf72dd700 1 -- 172.21.15.36:0/383654617 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddf0060ac0 msgr2=0x7fddf00ad330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:08.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf72dd700 1 --2- 172.21.15.36:0/383654617 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddf0060ac0 0x7fddf00ad330 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fddf0062c80 tx=0x7fddec006eb0).stop 2022-06-17T12:53:08.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf72dd700 1 -- 172.21.15.36:0/383654617 shutdown_connections 2022-06-17T12:53:08.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf72dd700 1 --2- 172.21.15.36:0/383654617 >> v2:172.21.15.70:3300/0 conn(0x7fddf00aee30 0x7fddf013e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:08.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf72dd700 1 --2- 172.21.15.36:0/383654617 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddf0060ac0 0x7fddf00ad330 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:08.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf72dd700 1 -- 172.21.15.36:0/383654617 >> 172.21.15.36:0/383654617 conn(0x7fddf00b65f0 msgr2=0x7fddf00b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:08.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf72dd700 1 -- 172.21.15.36:0/383654617 shutdown_connections 2022-06-17T12:53:08.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.854+0000 7fddf72dd700 1 -- 172.21.15.36:0/383654617 wait complete. 2022-06-17T12:53:08.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.858+0000 7fddf72dd700 1 Processor -- start 2022-06-17T12:53:08.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.858+0000 7fddf72dd700 1 -- start start 2022-06-17T12:53:08.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.858+0000 7fddf72dd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fddf0060ac0 0x7fddf00b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:08.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.858+0000 7fddf72dd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddf0063940 0x7fddf0064c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:08.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.858+0000 7fddf72dd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddf00aee30 0x7fddf0065170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:08.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.858+0000 7fddf72dd700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf00647e0 con 0x7fddf0060ac0 2022-06-17T12:53:08.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.858+0000 7fddf72dd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf0067570 con 0x7fddf0063940 2022-06-17T12:53:08.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.858+0000 7fddf72dd700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf0067810 con 0x7fddf00aee30 2022-06-17T12:53:08.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.858+0000 7fddf62db700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fddf0060ac0 0x7fddf00b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:08.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.858+0000 7fddf62db700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fddf0060ac0 0x7fddf00b0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35648/0 (socket says 172.21.15.36:35648) 2022-06-17T12:53:08.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.858+0000 7fddf62db700 1 -- 172.21.15.36:0/2626180163 learned_addr learned my addr 172.21.15.36:0/2626180163 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:08.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.858+0000 7fddf6adc700 1 --2- 172.21.15.36:0/2626180163 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddf00aee30 0x7fddf0065170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:08.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.858+0000 7fddf5ada700 1 --2- 172.21.15.36:0/2626180163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddf0063940 0x7fddf0064c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:08.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.858+0000 7fddf62db700 1 -- 172.21.15.36:0/2626180163 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddf00aee30 msgr2=0x7fddf0065170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:08.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.858+0000 7fddf62db700 1 --2- 172.21.15.36:0/2626180163 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddf00aee30 0x7fddf0065170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:08.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.858+0000 7fddf62db700 1 -- 172.21.15.36:0/2626180163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddf0063940 msgr2=0x7fddf0064c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:08.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.858+0000 7fddf62db700 1 --2- 172.21.15.36:0/2626180163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddf0063940 0x7fddf0064c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:08.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.858+0000 7fddf62db700 1 -- 172.21.15.36:0/2626180163 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde8009470 con 0x7fddf0060ac0 2022-06-17T12:53:08.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.858+0000 7fddf62db700 1 --2- 172.21.15.36:0/2626180163 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fddf0060ac0 0x7fddf00b0590 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fddf0062c80 tx=0x7fddec015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:08.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.858+0000 7fdddf7fe700 1 -- 172.21.15.36:0/2626180163 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddec011860 con 0x7fddf0060ac0 2022-06-17T12:53:08.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.858+0000 7fdddf7fe700 1 -- 172.21.15.36:0/2626180163 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddec0119c0 con 0x7fddf0060ac0 2022-06-17T12:53:08.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.862+0000 7fddf72dd700 1 -- 172.21.15.36:0/2626180163 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddec00b1e0 con 0x7fddf0060ac0 2022-06-17T12:53:08.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.862+0000 7fdddf7fe700 1 -- 172.21.15.36:0/2626180163 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddec011d20 con 0x7fddf0060ac0 2022-06-17T12:53:08.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.862+0000 7fddf72dd700 1 -- 172.21.15.36:0/2626180163 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf00b0a80 con 0x7fddf0060ac0 2022-06-17T12:53:08.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.862+0000 7fddf72dd700 1 -- 172.21.15.36:0/2626180163 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf0067390 con 0x7fddf0060ac0 2022-06-17T12:53:08.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.862+0000 7fdddf7fe700 1 -- 172.21.15.36:0/2626180163 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fddec00f070 con 0x7fddf0060ac0 2022-06-17T12:53:08.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.862+0000 7fdddf7fe700 1 --2- 172.21.15.36:0/2626180163 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fddd0053a40 0x7fddd0055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:08.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.862+0000 7fdddf7fe700 1 -- 172.21.15.36:0/2626180163 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fddec06d930 con 0x7fddf0060ac0 2022-06-17T12:53:08.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.862+0000 7fddf5ada700 1 --2- 172.21.15.36:0/2626180163 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fddd0053a40 0x7fddd0055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:08.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.862+0000 7fddf5ada700 1 --2- 172.21.15.36:0/2626180163 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fddd0053a40 0x7fddd0055f00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fdde000d000 tx=0x7fdde0007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:08.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.870+0000 7fdddf7fe700 1 -- 172.21.15.36:0/2626180163 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fddec044730 con 0x7fddf0060ac0 2022-06-17T12:53:08.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:08.998+0000 7fddf72dd700 1 -- 172.21.15.36:0/2626180163 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fddf00634a0 con 0x7fddf0060ac0 2022-06-17T12:53:08.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.002+0000 7fdddf7fe700 1 -- 172.21.15.36:0/2626180163 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16274 (secure 0 0 0) 0x7fddec006a40 con 0x7fddf0060ac0 2022-06-17T12:53:08.986 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:53:08.986 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":74,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:53:05.910998+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:53:03.265804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-06-17T12:53:04.385158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:53:08.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.002+0000 7fddf72dd700 1 -- 172.21.15.36:0/2626180163 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fddd0053a40 msgr2=0x7fddd0055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:08.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.002+0000 7fddf72dd700 1 --2- 172.21.15.36:0/2626180163 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fddd0053a40 0x7fddd0055f00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fdde000d000 tx=0x7fdde0007b90).stop 2022-06-17T12:53:08.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.002+0000 7fddf72dd700 1 -- 172.21.15.36:0/2626180163 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fddf0060ac0 msgr2=0x7fddf00b0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:08.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.002+0000 7fddf72dd700 1 --2- 172.21.15.36:0/2626180163 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fddf0060ac0 0x7fddf00b0590 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fddf0062c80 tx=0x7fddec015000).stop 2022-06-17T12:53:08.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.002+0000 7fddf72dd700 1 -- 172.21.15.36:0/2626180163 shutdown_connections 2022-06-17T12:53:08.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.002+0000 7fddf72dd700 1 --2- 172.21.15.36:0/2626180163 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fddd0053a40 0x7fddd0055f00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:08.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.002+0000 7fddf72dd700 1 --2- 172.21.15.36:0/2626180163 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddf00aee30 0x7fddf0065170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:08.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.002+0000 7fddf72dd700 1 --2- 172.21.15.36:0/2626180163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddf0063940 0x7fddf0064c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:08.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.002+0000 7fddf72dd700 1 --2- 172.21.15.36:0/2626180163 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fddf0060ac0 0x7fddf00b0590 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:08.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.002+0000 7fddf72dd700 1 -- 172.21.15.36:0/2626180163 >> 172.21.15.36:0/2626180163 conn(0x7fddf00b65f0 msgr2=0x7fddf005c380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:08.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.002+0000 7fddf72dd700 1 -- 172.21.15.36:0/2626180163 shutdown_connections 2022-06-17T12:53:08.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.002+0000 7fddf72dd700 1 -- 172.21.15.36:0/2626180163 wait complete. 2022-06-17T12:53:08.999 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:53:03.265804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:53:04.385158+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:53:09.000 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:53:09.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.126+0000 7f9b78e1c700 1 Processor -- start 2022-06-17T12:53:09.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.126+0000 7f9b78e1c700 1 -- start start 2022-06-17T12:53:09.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.126+0000 7f9b78e1c700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9b740aeac0 0x7f9b74143360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:09.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.126+0000 7f9b78e1c700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b740b2b90 con 0x7f9b74060ca0 2022-06-17T12:53:09.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.126+0000 7f9b78e1c700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b74147340 con 0x7f9b740b27b0 2022-06-17T12:53:09.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.126+0000 7f9b78e1c700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b74066b90 con 0x7f9b740aeac0 2022-06-17T12:53:09.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.126+0000 7f9b72ffd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f9b74060ca0 0x7f9b74146c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53188/0 (socket says 172.21.15.36:53188) 2022-06-17T12:53:09.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.130+0000 7f9b72ffd700 1 -- 172.21.15.36:0/709526271 learned_addr learned my addr 172.21.15.36:0/709526271 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:53:09.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.130+0000 7f9b73fff700 1 --2- 172.21.15.36:0/709526271 >> v2:172.21.15.70:3300/0 conn(0x7f9b740aeac0 0x7f9b74143360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:09.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.130+0000 7f9b727fc700 1 -- 172.21.15.36:0/709526271 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1223249047 0 0) 0x7f9b740b2b90 con 0x7f9b74060ca0 2022-06-17T12:53:09.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.130+0000 7f9b727fc700 1 -- 172.21.15.36:0/709526271 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b60003260 con 0x7f9b74060ca0 2022-06-17T12:53:09.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.130+0000 7f9b73fff700 1 -- 172.21.15.36:0/709526271 >> v1:172.21.15.70:6789/0 conn(0x7f9b740b27b0 legacy=0x7f9b74146520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:09.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.130+0000 7f9b73fff700 1 -- 172.21.15.36:0/709526271 >> v1:172.21.15.36:6789/0 conn(0x7f9b74060ca0 legacy=0x7f9b74146c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:09.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.130+0000 7f9b73fff700 1 -- 172.21.15.36:0/709526271 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b740af610 con 0x7f9b740aeac0 2022-06-17T12:53:09.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.130+0000 7f9b73fff700 1 --2- 172.21.15.36:0/709526271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b740aeac0 0x7f9b74143360 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f9b6c0032f0 tx=0x7f9b6c0091d0).ready entity=mon.1 client_cookie=99685f02c0cc68c5 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:09.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.130+0000 7f9b727fc700 1 -- 172.21.15.36:0/709526271 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b6c00b6c0 con 0x7f9b740aeac0 2022-06-17T12:53:09.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.130+0000 7f9b727fc700 1 -- 172.21.15.36:0/709526271 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6c00b820 con 0x7f9b740aeac0 2022-06-17T12:53:09.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.130+0000 7f9b727fc700 1 -- 172.21.15.36:0/709526271 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b6c00b990 con 0x7f9b740aeac0 2022-06-17T12:53:09.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.130+0000 7f9b78e1c700 1 -- 172.21.15.36:0/709526271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b740aeac0 msgr2=0x7f9b74143360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:09.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.130+0000 7f9b78e1c700 1 --2- 172.21.15.36:0/709526271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b740aeac0 0x7f9b74143360 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f9b6c0032f0 tx=0x7f9b6c0091d0).stop 2022-06-17T12:53:09.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.130+0000 7f9b78e1c700 1 -- 172.21.15.36:0/709526271 shutdown_connections 2022-06-17T12:53:09.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.130+0000 7f9b78e1c700 1 --2- 172.21.15.36:0/709526271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b740aeac0 0x7f9b74143360 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.130+0000 7f9b78e1c700 1 -- 172.21.15.36:0/709526271 >> 172.21.15.36:0/709526271 conn(0x7f9b740b5d10 msgr2=0x7f9b740b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:09.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.130+0000 7f9b78e1c700 1 -- 172.21.15.36:0/709526271 shutdown_connections 2022-06-17T12:53:09.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.130+0000 7f9b78e1c700 1 -- 172.21.15.36:0/709526271 wait complete. 2022-06-17T12:53:09.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b78e1c700 1 Processor -- start 2022-06-17T12:53:09.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b78e1c700 1 -- start start 2022-06-17T12:53:09.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b78e1c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b74060ca0 0x7f9b7409f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:09.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b78e1c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9b740b27b0 0x7f9b74152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:09.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b78e1c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9b740a0ba0 0x7f9b74152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:09.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b78e1c700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b740a2000 con 0x7f9b740a0ba0 2022-06-17T12:53:09.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b78e1c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b740a30e0 con 0x7f9b74060ca0 2022-06-17T12:53:09.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b78e1c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b740a3380 con 0x7f9b740b27b0 2022-06-17T12:53:09.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b73fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9b740a0ba0 0x7f9b74152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:09.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b73fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9b740a0ba0 0x7f9b74152710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35664/0 (socket says 172.21.15.36:35664) 2022-06-17T12:53:09.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b73fff700 1 -- 172.21.15.36:0/2252156813 learned_addr learned my addr 172.21.15.36:0/2252156813 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:09.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b737fe700 1 --2- 172.21.15.36:0/2252156813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b74060ca0 0x7f9b7409f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:09.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b72ffd700 1 --2- 172.21.15.36:0/2252156813 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9b740b27b0 0x7f9b74152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:09.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b73fff700 1 -- 172.21.15.36:0/2252156813 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9b740b27b0 msgr2=0x7f9b74152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:09.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b73fff700 1 --2- 172.21.15.36:0/2252156813 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9b740b27b0 0x7f9b74152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b73fff700 1 -- 172.21.15.36:0/2252156813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b74060ca0 msgr2=0x7f9b7409f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:09.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b73fff700 1 --2- 172.21.15.36:0/2252156813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b74060ca0 0x7f9b7409f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b73fff700 1 -- 172.21.15.36:0/2252156813 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6c009470 con 0x7f9b740a0ba0 2022-06-17T12:53:09.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b737fe700 1 --2- 172.21.15.36:0/2252156813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b74060ca0 0x7f9b7409f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:53:09.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b73fff700 1 --2- 172.21.15.36:0/2252156813 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9b740a0ba0 0x7f9b74152710 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f9b6c0032f0 tx=0x7f9b6c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:09.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b70ff9700 1 -- 172.21.15.36:0/2252156813 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b6c00b6c0 con 0x7f9b740a0ba0 2022-06-17T12:53:09.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b70ff9700 1 -- 172.21.15.36:0/2252156813 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6c0107a0 con 0x7f9b740a0ba0 2022-06-17T12:53:09.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b78e1c700 1 -- 172.21.15.36:0/2252156813 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b740a3620 con 0x7f9b740a0ba0 2022-06-17T12:53:09.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b70ff9700 1 -- 172.21.15.36:0/2252156813 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b6c010b00 con 0x7f9b740a0ba0 2022-06-17T12:53:09.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.134+0000 7f9b78e1c700 1 -- 172.21.15.36:0/2252156813 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b740a2d60 con 0x7f9b740a0ba0 2022-06-17T12:53:09.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.138+0000 7f9b78e1c700 1 -- 172.21.15.36:0/2252156813 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b740a37b0 con 0x7f9b740a0ba0 2022-06-17T12:53:09.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.138+0000 7f9b70ff9700 1 -- 172.21.15.36:0/2252156813 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f9b6c00f070 con 0x7f9b740a0ba0 2022-06-17T12:53:09.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.138+0000 7f9b70ff9700 1 --2- 172.21.15.36:0/2252156813 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9b54053a60 0x7f9b54055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:09.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.138+0000 7f9b70ff9700 1 -- 172.21.15.36:0/2252156813 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9b6c06d520 con 0x7f9b740a0ba0 2022-06-17T12:53:09.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.138+0000 7f9b737fe700 1 --2- 172.21.15.36:0/2252156813 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9b54053a60 0x7f9b54055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:09.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.138+0000 7f9b737fe700 1 --2- 172.21.15.36:0/2252156813 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9b54053a60 0x7f9b54055f20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f9b6800b000 tx=0x7f9b68009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:09.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.142+0000 7f9b70ff9700 1 -- 172.21.15.36:0/2252156813 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9b6c044320 con 0x7f9b740a0ba0 2022-06-17T12:53:09.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.270+0000 7f9b78e1c700 1 -- 172.21.15.36:0/2252156813 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9b740620f0 con 0x7f9b740a0ba0 2022-06-17T12:53:09.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.270+0000 7f9b70ff9700 1 -- 172.21.15.36:0/2252156813 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v74) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9b6c015020 con 0x7f9b740a0ba0 2022-06-17T12:53:09.255 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:53:09.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.274+0000 7f9b78e1c700 1 -- 172.21.15.36:0/2252156813 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9b54053a60 msgr2=0x7f9b54055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:09.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.274+0000 7f9b78e1c700 1 --2- 172.21.15.36:0/2252156813 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9b54053a60 0x7f9b54055f20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f9b6800b000 tx=0x7f9b68009040).stop 2022-06-17T12:53:09.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.274+0000 7f9b78e1c700 1 -- 172.21.15.36:0/2252156813 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9b740a0ba0 msgr2=0x7f9b74152710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:09.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.274+0000 7f9b78e1c700 1 --2- 172.21.15.36:0/2252156813 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9b740a0ba0 0x7f9b74152710 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f9b6c0032f0 tx=0x7f9b6c016000).stop 2022-06-17T12:53:09.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.274+0000 7f9b78e1c700 1 -- 172.21.15.36:0/2252156813 shutdown_connections 2022-06-17T12:53:09.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.274+0000 7f9b78e1c700 1 --2- 172.21.15.36:0/2252156813 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9b54053a60 0x7f9b54055f20 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.274+0000 7f9b78e1c700 1 --2- 172.21.15.36:0/2252156813 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9b740a0ba0 0x7f9b74152710 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.274+0000 7f9b78e1c700 1 --2- 172.21.15.36:0/2252156813 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9b740b27b0 0x7f9b74152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.274+0000 7f9b78e1c700 1 --2- 172.21.15.36:0/2252156813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b74060ca0 0x7f9b7409f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.274+0000 7f9b78e1c700 1 -- 172.21.15.36:0/2252156813 >> 172.21.15.36:0/2252156813 conn(0x7f9b740b5d10 msgr2=0x7f9b74142250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:09.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.274+0000 7f9b78e1c700 1 -- 172.21.15.36:0/2252156813 shutdown_connections 2022-06-17T12:53:09.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.274+0000 7f9b78e1c700 1 -- 172.21.15.36:0/2252156813 wait complete. 2022-06-17T12:53:09.272 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:53:09.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.402+0000 7f74f9a89700 1 Processor -- start 2022-06-17T12:53:09.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.402+0000 7f74f9a89700 1 -- start start 2022-06-17T12:53:09.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.402+0000 7f74f9a89700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f74f40b2d80 0x7f74f40b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:09.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.402+0000 7f74f9a89700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74f40b2350 0x7f74f4143750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:09.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.402+0000 7f74f9a89700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f40aa170 con 0x7f74f4143c40 2022-06-17T12:53:09.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.402+0000 7f74f9a89700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74f40a9c30 con 0x7f74f40b2d80 2022-06-17T12:53:09.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.402+0000 7f74f9a89700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f40a9ed0 con 0x7f74f40b2350 2022-06-17T12:53:09.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.402+0000 7f74eaffd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74f40b2350 0x7f74f4143750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:09.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.402+0000 7f74f37fe700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f74f40b2d80 0x7f74f40b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:09.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.402+0000 7f74eaffd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74f40b2350 0x7f74f4143750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:50462/0 (socket says 172.21.15.36:50462) 2022-06-17T12:53:09.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.402+0000 7f74eaffd700 1 -- 172.21.15.36:0/2065058311 learned_addr learned my addr 172.21.15.36:0/2065058311 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:09.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.402+0000 7f74eaffd700 1 -- 172.21.15.36:0/2065058311 >> v2:172.21.15.70:3300/0 conn(0x7f74f40b2d80 msgr2=0x7f74f40b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:09.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.402+0000 7f74eaffd700 1 --2- 172.21.15.36:0/2065058311 >> v2:172.21.15.70:3300/0 conn(0x7f74f40b2d80 0x7f74f40b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.402+0000 7f74eaffd700 1 -- 172.21.15.36:0/2065058311 >> v1:172.21.15.70:6789/0 conn(0x7f74f4143c40 legacy=0x7f74f413f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:09.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.402+0000 7f74eaffd700 1 -- 172.21.15.36:0/2065058311 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f40b08c0 con 0x7f74f40b2350 2022-06-17T12:53:09.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.402+0000 7f74eaffd700 1 --2- 172.21.15.36:0/2065058311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74f40b2350 0x7f74f4143750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f74dc002f70 tx=0x7f74dc006eb0).ready entity=mon.2 client_cookie=b923338b078fd246 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:09.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.406+0000 7f74f2ffd700 1 -- 172.21.15.36:0/2065058311 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74dc00d570 con 0x7f74f40b2350 2022-06-17T12:53:09.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.406+0000 7f74f2ffd700 1 -- 172.21.15.36:0/2065058311 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74dc000e90 con 0x7f74f40b2350 2022-06-17T12:53:09.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.406+0000 7f74f2ffd700 1 -- 172.21.15.36:0/2065058311 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74dc00d900 con 0x7f74f40b2350 2022-06-17T12:53:09.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.406+0000 7f74f9a89700 1 -- 172.21.15.36:0/2065058311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74f40b2350 msgr2=0x7f74f4143750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:09.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.406+0000 7f74f9a89700 1 --2- 172.21.15.36:0/2065058311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74f40b2350 0x7f74f4143750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f74dc002f70 tx=0x7f74dc006eb0).stop 2022-06-17T12:53:09.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.406+0000 7f74f9a89700 1 -- 172.21.15.36:0/2065058311 shutdown_connections 2022-06-17T12:53:09.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.406+0000 7f74f9a89700 1 --2- 172.21.15.36:0/2065058311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74f40b2350 0x7f74f4143750 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.406+0000 7f74f9a89700 1 --2- 172.21.15.36:0/2065058311 >> v2:172.21.15.70:3300/0 conn(0x7f74f40b2d80 0x7f74f40b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.406+0000 7f74f9a89700 1 -- 172.21.15.36:0/2065058311 >> 172.21.15.36:0/2065058311 conn(0x7f74f405c550 msgr2=0x7f74f405c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:09.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.406+0000 7f74f9a89700 1 -- 172.21.15.36:0/2065058311 shutdown_connections 2022-06-17T12:53:09.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.406+0000 7f74f9a89700 1 -- 172.21.15.36:0/2065058311 wait complete. 2022-06-17T12:53:09.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.406+0000 7f74f9a89700 1 Processor -- start 2022-06-17T12:53:09.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.406+0000 7f74f9a89700 1 -- start start 2022-06-17T12:53:09.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.406+0000 7f74f9a89700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74f40b2350 0x7f74f40acc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:09.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.406+0000 7f74f9a89700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74f40b2d80 0x7f74f4154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:09.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.406+0000 7f74f9a89700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74f4143c40 0x7f74f4154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:09.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.406+0000 7f74f9a89700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f40a17c0 con 0x7f74f40b2350 2022-06-17T12:53:09.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.406+0000 7f74f9a89700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f40662a0 con 0x7f74f40b2d80 2022-06-17T12:53:09.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.406+0000 7f74f9a89700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f40a1c50 con 0x7f74f4143c40 2022-06-17T12:53:09.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.410+0000 7f74f37fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74f40b2350 0x7f74f40acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:09.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.410+0000 7f74f37fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74f40b2350 0x7f74f40acc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35676/0 (socket says 172.21.15.36:35676) 2022-06-17T12:53:09.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.410+0000 7f74f37fe700 1 -- 172.21.15.36:0/1838700451 learned_addr learned my addr 172.21.15.36:0/1838700451 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:09.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.410+0000 7f74f3fff700 1 --2- 172.21.15.36:0/1838700451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74f4143c40 0x7f74f4154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:09.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.410+0000 7f74eaffd700 1 --2- 172.21.15.36:0/1838700451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74f40b2d80 0x7f74f4154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:09.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.410+0000 7f74f3fff700 1 -- 172.21.15.36:0/1838700451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74f40b2d80 msgr2=0x7f74f4154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:09.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.410+0000 7f74f3fff700 1 --2- 172.21.15.36:0/1838700451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74f40b2d80 0x7f74f4154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.410+0000 7f74f3fff700 1 -- 172.21.15.36:0/1838700451 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74f40b2350 msgr2=0x7f74f40acc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:09.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.410+0000 7f74f3fff700 1 --2- 172.21.15.36:0/1838700451 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74f40b2350 0x7f74f40acc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.410+0000 7f74f3fff700 1 -- 172.21.15.36:0/1838700451 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74dc00b1e0 con 0x7f74f4143c40 2022-06-17T12:53:09.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.410+0000 7f74f37fe700 1 --2- 172.21.15.36:0/1838700451 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74f40b2350 0x7f74f40acc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:53:09.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.410+0000 7f74f3fff700 1 --2- 172.21.15.36:0/1838700451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74f4143c40 0x7f74f4154810 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f74f4153da0 tx=0x7f74ec007f10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:09.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.410+0000 7f74f17fa700 1 -- 172.21.15.36:0/1838700451 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74ec00ce00 con 0x7f74f4143c40 2022-06-17T12:53:09.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.410+0000 7f74f17fa700 1 -- 172.21.15.36:0/1838700451 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ec006aa0 con 0x7f74f4143c40 2022-06-17T12:53:09.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.410+0000 7f74f9a89700 1 -- 172.21.15.36:0/1838700451 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f40a19b0 con 0x7f74f4143c40 2022-06-17T12:53:09.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.410+0000 7f74f17fa700 1 -- 172.21.15.36:0/1838700451 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74ec006e00 con 0x7f74f4143c40 2022-06-17T12:53:09.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.410+0000 7f74f9a89700 1 -- 172.21.15.36:0/1838700451 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f4067200 con 0x7f74f4143c40 2022-06-17T12:53:09.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.410+0000 7f74f9a89700 1 -- 172.21.15.36:0/1838700451 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f40ad8f0 con 0x7f74f4143c40 2022-06-17T12:53:09.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.410+0000 7f74f17fa700 1 -- 172.21.15.36:0/1838700451 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f74ec010030 con 0x7f74f4143c40 2022-06-17T12:53:09.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.414+0000 7f74f17fa700 1 --2- 172.21.15.36:0/1838700451 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f74d8053a40 0x7f74d8055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:09.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.414+0000 7f74f17fa700 1 -- 172.21.15.36:0/1838700451 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f74ec06bcd0 con 0x7f74f4143c40 2022-06-17T12:53:09.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.414+0000 7f74f37fe700 1 --2- 172.21.15.36:0/1838700451 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f74d8053a40 0x7f74d8055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:09.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.414+0000 7f74f37fe700 1 --2- 172.21.15.36:0/1838700451 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f74d8053a40 0x7f74d8055f00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f74e400b000 tx=0x7f74e4006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:09.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.422+0000 7f74f17fa700 1 -- 172.21.15.36:0/1838700451 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f74ec042a30 con 0x7f74f4143c40 2022-06-17T12:53:09.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.550+0000 7f74f9a89700 1 -- 172.21.15.36:0/1838700451 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f74f4057410 con 0x7f74f4143c40 2022-06-17T12:53:09.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.554+0000 7f74f17fa700 1 -- 172.21.15.36:0/1838700451 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v74) v1 ==== 92+0+10 (secure 0 0 0) 0x7f74ec046050 con 0x7f74f4143c40 2022-06-17T12:53:09.539 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:53:09.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.554+0000 7f74f9a89700 1 -- 172.21.15.36:0/1838700451 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f74d8053a40 msgr2=0x7f74d8055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:09.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.554+0000 7f74f9a89700 1 --2- 172.21.15.36:0/1838700451 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f74d8053a40 0x7f74d8055f00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f74e400b000 tx=0x7f74e4006df0).stop 2022-06-17T12:53:09.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.554+0000 7f74f9a89700 1 -- 172.21.15.36:0/1838700451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74f4143c40 msgr2=0x7f74f4154810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:09.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.554+0000 7f74f9a89700 1 --2- 172.21.15.36:0/1838700451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74f4143c40 0x7f74f4154810 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f74f4153da0 tx=0x7f74ec007f10).stop 2022-06-17T12:53:09.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.554+0000 7f74f9a89700 1 -- 172.21.15.36:0/1838700451 shutdown_connections 2022-06-17T12:53:09.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.554+0000 7f74f9a89700 1 --2- 172.21.15.36:0/1838700451 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f74d8053a40 0x7f74d8055f00 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.554+0000 7f74f9a89700 1 --2- 172.21.15.36:0/1838700451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74f4143c40 0x7f74f4154810 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.554+0000 7f74f9a89700 1 --2- 172.21.15.36:0/1838700451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74f40b2d80 0x7f74f4154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.554+0000 7f74f9a89700 1 --2- 172.21.15.36:0/1838700451 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f74f40b2350 0x7f74f40acc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.554+0000 7f74f9a89700 1 -- 172.21.15.36:0/1838700451 >> 172.21.15.36:0/1838700451 conn(0x7f74f405c550 msgr2=0x7f74f4141a00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:09.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.554+0000 7f74f9a89700 1 -- 172.21.15.36:0/1838700451 shutdown_connections 2022-06-17T12:53:09.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.554+0000 7f74f9a89700 1 -- 172.21.15.36:0/1838700451 wait complete. 2022-06-17T12:53:09.552 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:53:09.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.674+0000 7f9065421700 1 Processor -- start 2022-06-17T12:53:09.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.674+0000 7f9065421700 1 -- start start 2022-06-17T12:53:09.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f9065421700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90600b4120 0x7f9060146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:09.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f9065421700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90600b68d0 con 0x7f9060147270 2022-06-17T12:53:09.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f9065421700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90601476b0 con 0x7f90600b64f0 2022-06-17T12:53:09.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f9065421700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90600b6e20 con 0x7f90600b4120 2022-06-17T12:53:09.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f905f7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f9060147270 0x7f9060140100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53218/0 (socket says 172.21.15.36:53218) 2022-06-17T12:53:09.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f905f7fe700 1 -- 172.21.15.36:0/3250909576 learned_addr learned my addr 172.21.15.36:0/3250909576 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:53:09.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f905e7fc700 1 --2- 172.21.15.36:0/3250909576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90600b4120 0x7f9060146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:09.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f905dffb700 1 -- 172.21.15.36:0/3250909576 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3970922345 0 0) 0x7f90600b68d0 con 0x7f9060147270 2022-06-17T12:53:09.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f905dffb700 1 -- 172.21.15.36:0/3250909576 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9040003260 con 0x7f9060147270 2022-06-17T12:53:09.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f905dffb700 1 -- 172.21.15.36:0/3250909576 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1642389401 0 0) 0x7f90601476b0 con 0x7f90600b64f0 2022-06-17T12:53:09.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f905dffb700 1 -- 172.21.15.36:0/3250909576 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90600b68d0 con 0x7f90600b64f0 2022-06-17T12:53:09.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f905dffb700 1 -- 172.21.15.36:0/3250909576 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3996540591 0 0) 0x7f9040003260 con 0x7f9060147270 2022-06-17T12:53:09.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f905e7fc700 1 -- 172.21.15.36:0/3250909576 >> v1:172.21.15.70:6789/0 conn(0x7f90600b64f0 legacy=0x7f9060143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:09.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f905e7fc700 1 -- 172.21.15.36:0/3250909576 >> v1:172.21.15.36:6789/0 conn(0x7f9060147270 legacy=0x7f9060140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:09.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f905e7fc700 1 -- 172.21.15.36:0/3250909576 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90601468e0 con 0x7f90600b4120 2022-06-17T12:53:09.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f905e7fc700 1 --2- 172.21.15.36:0/3250909576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90600b4120 0x7f9060146d80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f9058001490 tx=0x7f9050006df0).ready entity=mon.2 client_cookie=1f890fc1c0af3614 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:09.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f905dffb700 1 -- 172.21.15.36:0/3250909576 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90580019e0 con 0x7f90600b4120 2022-06-17T12:53:09.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f905dffb700 1 -- 172.21.15.36:0/3250909576 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9050000e90 con 0x7f90600b4120 2022-06-17T12:53:09.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f905dffb700 1 -- 172.21.15.36:0/3250909576 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9050012460 con 0x7f90600b4120 2022-06-17T12:53:09.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f9065421700 1 -- 172.21.15.36:0/3250909576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90600b4120 msgr2=0x7f9060146d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:09.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f9065421700 1 --2- 172.21.15.36:0/3250909576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90600b4120 0x7f9060146d80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f9058001490 tx=0x7f9050006df0).stop 2022-06-17T12:53:09.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f9065421700 1 -- 172.21.15.36:0/3250909576 shutdown_connections 2022-06-17T12:53:09.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f9065421700 1 --2- 172.21.15.36:0/3250909576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90600b4120 0x7f9060146d80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f9065421700 1 -- 172.21.15.36:0/3250909576 >> 172.21.15.36:0/3250909576 conn(0x7f90600620f0 msgr2=0x7f9060062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:09.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f9065421700 1 -- 172.21.15.36:0/3250909576 shutdown_connections 2022-06-17T12:53:09.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.678+0000 7f9065421700 1 -- 172.21.15.36:0/3250909576 wait complete. 2022-06-17T12:53:09.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.682+0000 7f9065421700 1 Processor -- start 2022-06-17T12:53:09.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.682+0000 7f9065421700 1 -- start start 2022-06-17T12:53:09.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.682+0000 7f9065421700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90600b4120 0x7f90600b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:09.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.682+0000 7f9065421700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f90600b64f0 0x7f90600b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:09.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.682+0000 7f9065421700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9060147270 0x7f9060155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:09.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.682+0000 7f9065421700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90600b16a0 con 0x7f90600b64f0 2022-06-17T12:53:09.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.682+0000 7f9065421700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90601566a0 con 0x7f9060147270 2022-06-17T12:53:09.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.682+0000 7f9065421700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90600b1930 con 0x7f90600b4120 2022-06-17T12:53:09.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.682+0000 7f905e7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f90600b64f0 0x7f90600b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:09.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.682+0000 7f905e7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f90600b64f0 0x7f90600b0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35690/0 (socket says 172.21.15.36:35690) 2022-06-17T12:53:09.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.682+0000 7f905e7fc700 1 -- 172.21.15.36:0/620065107 learned_addr learned my addr 172.21.15.36:0/620065107 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:09.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.682+0000 7f905effd700 1 --2- 172.21.15.36:0/620065107 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90600b4120 0x7f90600b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:09.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.682+0000 7f905f7fe700 1 --2- 172.21.15.36:0/620065107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9060147270 0x7f9060155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:09.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.682+0000 7f905e7fc700 1 -- 172.21.15.36:0/620065107 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90600b4120 msgr2=0x7f90600b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:09.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.682+0000 7f905e7fc700 1 --2- 172.21.15.36:0/620065107 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90600b4120 0x7f90600b1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.682+0000 7f905e7fc700 1 -- 172.21.15.36:0/620065107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9060147270 msgr2=0x7f9060155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:09.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.682+0000 7f905e7fc700 1 --2- 172.21.15.36:0/620065107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9060147270 0x7f9060155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.682+0000 7f905e7fc700 1 -- 172.21.15.36:0/620065107 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905000b180 con 0x7f90600b64f0 2022-06-17T12:53:09.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.682+0000 7f905effd700 1 --2- 172.21.15.36:0/620065107 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90600b4120 0x7f90600b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:53:09.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.682+0000 7f905f7fe700 1 --2- 172.21.15.36:0/620065107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9060147270 0x7f9060155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:53:09.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.682+0000 7f905e7fc700 1 --2- 172.21.15.36:0/620065107 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f90600b64f0 0x7f90600b0730 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f9058001490 tx=0x7f9050014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:09.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.686+0000 7f9057fff700 1 -- 172.21.15.36:0/620065107 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9050016070 con 0x7f90600b64f0 2022-06-17T12:53:09.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.686+0000 7f9057fff700 1 -- 172.21.15.36:0/620065107 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9050002f50 con 0x7f90600b64f0 2022-06-17T12:53:09.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.686+0000 7f9065421700 1 -- 172.21.15.36:0/620065107 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f906014a800 con 0x7f90600b64f0 2022-06-17T12:53:09.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.686+0000 7f9057fff700 1 -- 172.21.15.36:0/620065107 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9050012460 con 0x7f90600b64f0 2022-06-17T12:53:09.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.686+0000 7f9065421700 1 -- 172.21.15.36:0/620065107 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90601563d0 con 0x7f90600b64f0 2022-06-17T12:53:09.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.686+0000 7f9065421700 1 -- 172.21.15.36:0/620065107 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90600b1af0 con 0x7f90600b64f0 2022-06-17T12:53:09.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.686+0000 7f9057fff700 1 -- 172.21.15.36:0/620065107 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f9050025030 con 0x7f90600b64f0 2022-06-17T12:53:09.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.686+0000 7f9057fff700 1 --2- 172.21.15.36:0/620065107 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9044053920 0x7f9044055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:09.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.686+0000 7f9057fff700 1 -- 172.21.15.36:0/620065107 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f905006cf00 con 0x7f90600b64f0 2022-06-17T12:53:09.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.686+0000 7f905effd700 1 --2- 172.21.15.36:0/620065107 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9044053920 0x7f9044055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:09.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.690+0000 7f905effd700 1 --2- 172.21.15.36:0/620065107 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9044053920 0x7f9044055de0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f904800d000 tx=0x7f9048007a20).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:09.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.694+0000 7f9057fff700 1 -- 172.21.15.36:0/620065107 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9050043d00 con 0x7f90600b64f0 2022-06-17T12:53:09.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.822+0000 7f9065421700 1 -- 172.21.15.36:0/620065107 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f90600b3a60 con 0x7f90600b64f0 2022-06-17T12:53:09.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.822+0000 7f9057fff700 1 -- 172.21.15.36:0/620065107 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+10 (secure 0 0 0) 0x7f9050023020 con 0x7f90600b64f0 2022-06-17T12:53:09.808 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:53:09.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.822+0000 7f9065421700 1 -- 172.21.15.36:0/620065107 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9044053920 msgr2=0x7f9044055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:09.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.822+0000 7f9065421700 1 --2- 172.21.15.36:0/620065107 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9044053920 0x7f9044055de0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f904800d000 tx=0x7f9048007a20).stop 2022-06-17T12:53:09.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.822+0000 7f9065421700 1 -- 172.21.15.36:0/620065107 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f90600b64f0 msgr2=0x7f90600b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:09.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.822+0000 7f9065421700 1 --2- 172.21.15.36:0/620065107 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f90600b64f0 0x7f90600b0730 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f9058001490 tx=0x7f9050014000).stop 2022-06-17T12:53:09.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.822+0000 7f9065421700 1 -- 172.21.15.36:0/620065107 shutdown_connections 2022-06-17T12:53:09.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.822+0000 7f9065421700 1 --2- 172.21.15.36:0/620065107 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9044053920 0x7f9044055de0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.822+0000 7f9065421700 1 --2- 172.21.15.36:0/620065107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9060147270 0x7f9060155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.822+0000 7f9065421700 1 --2- 172.21.15.36:0/620065107 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f90600b64f0 0x7f90600b0730 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.822+0000 7f9065421700 1 --2- 172.21.15.36:0/620065107 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90600b4120 0x7f90600b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.822+0000 7f9065421700 1 -- 172.21.15.36:0/620065107 >> 172.21.15.36:0/620065107 conn(0x7f90600620f0 msgr2=0x7f90600b8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:09.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.822+0000 7f9065421700 1 -- 172.21.15.36:0/620065107 shutdown_connections 2022-06-17T12:53:09.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.822+0000 7f9065421700 1 -- 172.21.15.36:0/620065107 wait complete. 2022-06-17T12:53:09.821 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:53:09.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.950+0000 7f087bfff700 1 Processor -- start 2022-06-17T12:53:09.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.950+0000 7f087bfff700 1 -- start start 2022-06-17T12:53:09.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.950+0000 7f087bfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f087c0b27b0 0x7f087c13f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:09.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.950+0000 7f087bfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f087c0b15f0 0x7f087c0ace90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:09.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.950+0000 7f087bfff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f087c0b2b90 con 0x7f087c0b0d70 2022-06-17T12:53:09.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.950+0000 7f087bfff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f087c0ad380 con 0x7f087c0b27b0 2022-06-17T12:53:09.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.950+0000 7f087bfff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087c0610e0 con 0x7f087c0b15f0 2022-06-17T12:53:09.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.950+0000 7f087affd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f087c0b27b0 0x7f087c13f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:09.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.950+0000 7f087affd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f087c0b27b0 0x7f087c13f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35698/0 (socket says 172.21.15.36:35698) 2022-06-17T12:53:09.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.950+0000 7f087affd700 1 -- 172.21.15.36:0/2590693622 learned_addr learned my addr 172.21.15.36:0/2590693622 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:09.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.950+0000 7f087b7fe700 1 --2- 172.21.15.36:0/2590693622 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f087c0b15f0 0x7f087c0ace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:09.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.954+0000 7f087affd700 1 -- 172.21.15.36:0/2590693622 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f087c0b15f0 msgr2=0x7f087c0ace90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:09.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.954+0000 7f087affd700 1 --2- 172.21.15.36:0/2590693622 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f087c0b15f0 0x7f087c0ace90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.954+0000 7f087affd700 1 -- 172.21.15.36:0/2590693622 >> v1:172.21.15.70:6789/0 conn(0x7f087c0b0d70 legacy=0x7f087c0ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:09.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.954+0000 7f0879ffb700 1 -- 172.21.15.36:0/2590693622 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3688912495 0 0) 0x7f087c0b2b90 con 0x7f087c0b0d70 2022-06-17T12:53:09.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.954+0000 7f087affd700 1 -- 172.21.15.36:0/2590693622 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087c0ac190 con 0x7f087c0b27b0 2022-06-17T12:53:09.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.954+0000 7f087b7fe700 1 --2- 172.21.15.36:0/2590693622 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f087c0b15f0 0x7f087c0ace90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:53:09.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.954+0000 7f087affd700 1 --2- 172.21.15.36:0/2590693622 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f087c0b27b0 0x7f087c13f110 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7f087c0b2d20 tx=0x7f0874009290).ready entity=mon.0 client_cookie=de5967ea06083c34 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:09.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.954+0000 7f0879ffb700 1 -- 172.21.15.36:0/2590693622 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f087400b860 con 0x7f087c0b27b0 2022-06-17T12:53:09.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.954+0000 7f0879ffb700 1 -- 172.21.15.36:0/2590693622 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087400b9c0 con 0x7f087c0b27b0 2022-06-17T12:53:09.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.954+0000 7f0879ffb700 1 -- 172.21.15.36:0/2590693622 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f087400bb30 con 0x7f087c0b27b0 2022-06-17T12:53:09.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.954+0000 7f087bfff700 1 -- 172.21.15.36:0/2590693622 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f087c0b27b0 msgr2=0x7f087c13f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:09.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.954+0000 7f087bfff700 1 --2- 172.21.15.36:0/2590693622 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f087c0b27b0 0x7f087c13f110 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7f087c0b2d20 tx=0x7f0874009290).stop 2022-06-17T12:53:09.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.954+0000 7f087bfff700 1 -- 172.21.15.36:0/2590693622 shutdown_connections 2022-06-17T12:53:09.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.954+0000 7f087bfff700 1 --2- 172.21.15.36:0/2590693622 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f087c0b15f0 0x7f087c0ace90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.954+0000 7f087bfff700 1 --2- 172.21.15.36:0/2590693622 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f087c0b27b0 0x7f087c13f110 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.954+0000 7f087bfff700 1 -- 172.21.15.36:0/2590693622 >> 172.21.15.36:0/2590693622 conn(0x7f087c0b5d10 msgr2=0x7f087c0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:09.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.954+0000 7f087bfff700 1 -- 172.21.15.36:0/2590693622 shutdown_connections 2022-06-17T12:53:09.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.954+0000 7f087bfff700 1 -- 172.21.15.36:0/2590693622 wait complete. 2022-06-17T12:53:09.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.958+0000 7f087bfff700 1 Processor -- start 2022-06-17T12:53:09.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.958+0000 7f087bfff700 1 -- start start 2022-06-17T12:53:09.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.958+0000 7f087bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f087c0b0d70 0x7f087c130590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:09.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.958+0000 7f087bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f087c0b15f0 0x7f087c062860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:09.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.958+0000 7f087bfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f087c0b27b0 0x7f087c12a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:09.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.958+0000 7f087bfff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c12b740 con 0x7f087c0b0d70 2022-06-17T12:53:09.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.958+0000 7f087bfff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c064d90 con 0x7f087c0b15f0 2022-06-17T12:53:09.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.958+0000 7f087bfff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087c12f6b0 con 0x7f087c0b27b0 2022-06-17T12:53:09.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.958+0000 7f087affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f087c0b0d70 0x7f087c130590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:09.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.958+0000 7f087affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f087c0b0d70 0x7f087c130590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35704/0 (socket says 172.21.15.36:35704) 2022-06-17T12:53:09.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.958+0000 7f087affd700 1 -- 172.21.15.36:0/1392147245 learned_addr learned my addr 172.21.15.36:0/1392147245 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:09.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.958+0000 7f087b7fe700 1 --2- 172.21.15.36:0/1392147245 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f087c0b27b0 0x7f087c12a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:09.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.958+0000 7f087a7fc700 1 --2- 172.21.15.36:0/1392147245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f087c0b15f0 0x7f087c062860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:09.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.958+0000 7f087affd700 1 -- 172.21.15.36:0/1392147245 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f087c0b27b0 msgr2=0x7f087c12a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:09.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.958+0000 7f087affd700 1 --2- 172.21.15.36:0/1392147245 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f087c0b27b0 0x7f087c12a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.958+0000 7f087affd700 1 -- 172.21.15.36:0/1392147245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f087c0b15f0 msgr2=0x7f087c062860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:09.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.958+0000 7f087affd700 1 --2- 172.21.15.36:0/1392147245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f087c0b15f0 0x7f087c062860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:09.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.958+0000 7f087affd700 1 -- 172.21.15.36:0/1392147245 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086c00b180 con 0x7f087c0b0d70 2022-06-17T12:53:09.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.958+0000 7f087affd700 1 --2- 172.21.15.36:0/1392147245 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f087c0b0d70 0x7f087c130590 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7f087c0b2d20 tx=0x7f0874016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:09.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.958+0000 7f0873fff700 1 -- 172.21.15.36:0/1392147245 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0874021670 con 0x7f087c0b0d70 2022-06-17T12:53:09.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.958+0000 7f0873fff700 1 -- 172.21.15.36:0/1392147245 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08740217d0 con 0x7f087c0b0d70 2022-06-17T12:53:09.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.958+0000 7f087bfff700 1 -- 172.21.15.36:0/1392147245 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0874009530 con 0x7f087c0b0d70 2022-06-17T12:53:09.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.958+0000 7f0873fff700 1 -- 172.21.15.36:0/1392147245 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0874021b30 con 0x7f087c0b0d70 2022-06-17T12:53:09.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.958+0000 7f087bfff700 1 -- 172.21.15.36:0/1392147245 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087c130bb0 con 0x7f087c0b0d70 2022-06-17T12:53:09.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.962+0000 7f0873fff700 1 -- 172.21.15.36:0/1392147245 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f087400f070 con 0x7f087c0b0d70 2022-06-17T12:53:09.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.962+0000 7f087bfff700 1 -- 172.21.15.36:0/1392147245 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f087c064f00 con 0x7f087c0b0d70 2022-06-17T12:53:09.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.962+0000 7f0873fff700 1 --2- 172.21.15.36:0/1392147245 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0860053ad0 0x7f0860055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:09.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.962+0000 7f0873fff700 1 -- 172.21.15.36:0/1392147245 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f087406d460 con 0x7f087c0b0d70 2022-06-17T12:53:09.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.962+0000 7f087a7fc700 1 --2- 172.21.15.36:0/1392147245 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0860053ad0 0x7f0860055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:09.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.962+0000 7f087a7fc700 1 --2- 172.21.15.36:0/1392147245 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0860053ad0 0x7f0860055f90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f086400d000 tx=0x7f0864007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:09.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:09.966+0000 7f0873fff700 1 -- 172.21.15.36:0/1392147245 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0874047e60 con 0x7f087c0b0d70 2022-06-17T12:53:10.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.086+0000 7f087bfff700 1 -- 172.21.15.36:0/1392147245 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f087c0ac430 con 0x7f087c0b0d70 2022-06-17T12:53:10.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.086+0000 7f0873fff700 1 -- 172.21.15.36:0/1392147245 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+10 (secure 0 0 0) 0x7f0874015070 con 0x7f087c0b0d70 2022-06-17T12:53:10.075 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:53:10.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.090+0000 7f087bfff700 1 -- 172.21.15.36:0/1392147245 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0860053ad0 msgr2=0x7f0860055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:10.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.090+0000 7f087bfff700 1 --2- 172.21.15.36:0/1392147245 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0860053ad0 0x7f0860055f90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f086400d000 tx=0x7f0864007a00).stop 2022-06-17T12:53:10.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.090+0000 7f087bfff700 1 -- 172.21.15.36:0/1392147245 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f087c0b0d70 msgr2=0x7f087c130590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:10.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.090+0000 7f087bfff700 1 --2- 172.21.15.36:0/1392147245 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f087c0b0d70 0x7f087c130590 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7f087c0b2d20 tx=0x7f0874016000).stop 2022-06-17T12:53:10.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.090+0000 7f087bfff700 1 -- 172.21.15.36:0/1392147245 shutdown_connections 2022-06-17T12:53:10.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.090+0000 7f087bfff700 1 --2- 172.21.15.36:0/1392147245 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0860053ad0 0x7f0860055f90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:10.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.090+0000 7f087bfff700 1 --2- 172.21.15.36:0/1392147245 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f087c0b27b0 0x7f087c12a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:10.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.090+0000 7f087bfff700 1 --2- 172.21.15.36:0/1392147245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f087c0b15f0 0x7f087c062860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:10.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.090+0000 7f087bfff700 1 --2- 172.21.15.36:0/1392147245 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f087c0b0d70 0x7f087c130590 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:10.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.090+0000 7f087bfff700 1 -- 172.21.15.36:0/1392147245 >> 172.21.15.36:0/1392147245 conn(0x7f087c0b5d10 msgr2=0x7f087c0afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:10.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.090+0000 7f087bfff700 1 -- 172.21.15.36:0/1392147245 shutdown_connections 2022-06-17T12:53:10.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.090+0000 7f087bfff700 1 -- 172.21.15.36:0/1392147245 wait complete. 2022-06-17T12:53:10.085 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:53:10.085 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:53:10.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.214+0000 7febb228c700 1 Processor -- start 2022-06-17T12:53:10.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.214+0000 7febb228c700 1 -- start start 2022-06-17T12:53:10.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.214+0000 7febb228c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7febac0af930 0x7febac064230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:10.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.214+0000 7febb228c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febac0afe30 0x7febac0b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:10.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.214+0000 7febb228c700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febac0af280 con 0x7febac0aeea0 2022-06-17T12:53:10.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.214+0000 7febb228c700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febac0b14f0 con 0x7febac0af930 2022-06-17T12:53:10.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.214+0000 7febb228c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febac067560 con 0x7febac0afe30 2022-06-17T12:53:10.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.214+0000 7febb0a89700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7febac0af930 0x7febac064230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:10.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.214+0000 7febb128a700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7febac0aeea0 0x7febac0b19e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53242/0 (socket says 172.21.15.36:53242) 2022-06-17T12:53:10.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.214+0000 7febb0a89700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7febac0af930 0x7febac064230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35712/0 (socket says 172.21.15.36:35712) 2022-06-17T12:53:10.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.214+0000 7febb128a700 1 -- 172.21.15.36:0/1753530380 learned_addr learned my addr 172.21.15.36:0/1753530380 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:53:10.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.214+0000 7febb1a8b700 1 --2- 172.21.15.36:0/1753530380 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febac0afe30 0x7febac0b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:10.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.214+0000 7febb0a89700 1 -- 172.21.15.36:0/1753530380 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febac0afe30 msgr2=0x7febac0b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:10.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.214+0000 7febb0a89700 1 --2- 172.21.15.36:0/1753530380 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febac0afe30 0x7febac0b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:10.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.214+0000 7febb0a89700 1 -- 172.21.15.36:0/1753530380 >> v1:172.21.15.36:6789/0 conn(0x7febac0aeea0 legacy=0x7febac0b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:10.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.214+0000 7febb0a89700 1 -- 172.21.15.36:0/1753530380 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febac0ab9a0 con 0x7febac0af930 2022-06-17T12:53:10.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.214+0000 7febb0a89700 1 --2- 172.21.15.36:0/1753530380 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febac0af930 0x7febac064230 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7febac067440 tx=0x7feb9c006df0).ready entity=mon.0 client_cookie=5752b678114986db server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:10.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.214+0000 7feb9bfff700 1 -- 172.21.15.36:0/1753530380 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb9c00d570 con 0x7febac0af930 2022-06-17T12:53:10.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.218+0000 7feb9bfff700 1 -- 172.21.15.36:0/1753530380 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9c000e90 con 0x7febac0af930 2022-06-17T12:53:10.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.218+0000 7feb9bfff700 1 -- 172.21.15.36:0/1753530380 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb9c00d8c0 con 0x7febac0af930 2022-06-17T12:53:10.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.218+0000 7febb228c700 1 -- 172.21.15.36:0/1753530380 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febac0af930 msgr2=0x7febac064230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:10.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.218+0000 7febb228c700 1 --2- 172.21.15.36:0/1753530380 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febac0af930 0x7febac064230 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7febac067440 tx=0x7feb9c006df0).stop 2022-06-17T12:53:10.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.218+0000 7febb228c700 1 -- 172.21.15.36:0/1753530380 shutdown_connections 2022-06-17T12:53:10.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.218+0000 7febb228c700 1 --2- 172.21.15.36:0/1753530380 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febac0afe30 0x7febac0b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:10.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.218+0000 7febb228c700 1 --2- 172.21.15.36:0/1753530380 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febac0af930 0x7febac064230 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:10.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.218+0000 7febb228c700 1 -- 172.21.15.36:0/1753530380 >> 172.21.15.36:0/1753530380 conn(0x7febac05f770 msgr2=0x7febac05fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:10.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.218+0000 7febb228c700 1 -- 172.21.15.36:0/1753530380 shutdown_connections 2022-06-17T12:53:10.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.218+0000 7febb228c700 1 -- 172.21.15.36:0/1753530380 wait complete. 2022-06-17T12:53:10.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.218+0000 7febb228c700 1 Processor -- start 2022-06-17T12:53:10.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.218+0000 7febb228c700 1 -- start start 2022-06-17T12:53:10.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.222+0000 7febb228c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febac0aeea0 0x7febac0a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:10.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.222+0000 7febb228c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febac0af930 0x7febac0a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:10.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.222+0000 7febb228c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febac0afe30 0x7febac0a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:10.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.222+0000 7febb228c700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac0a55b0 con 0x7febac0afe30 2022-06-17T12:53:10.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.222+0000 7febb228c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac0b5f70 con 0x7febac0aeea0 2022-06-17T12:53:10.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.222+0000 7febb228c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febac0a57e0 con 0x7febac0af930 2022-06-17T12:53:10.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.222+0000 7febb1a8b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febac0afe30 0x7febac0a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:10.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.222+0000 7febb1a8b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febac0afe30 0x7febac0a3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35722/0 (socket says 172.21.15.36:35722) 2022-06-17T12:53:10.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.222+0000 7febb0a89700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febac0af930 0x7febac0a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:10.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.222+0000 7febb1a8b700 1 -- 172.21.15.36:0/959969238 learned_addr learned my addr 172.21.15.36:0/959969238 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:10.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.222+0000 7febb128a700 1 --2- 172.21.15.36:0/959969238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febac0aeea0 0x7febac0a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:10.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.222+0000 7febb1a8b700 1 -- 172.21.15.36:0/959969238 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febac0af930 msgr2=0x7febac0a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:10.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.222+0000 7febb1a8b700 1 --2- 172.21.15.36:0/959969238 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febac0af930 0x7febac0a1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:10.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.222+0000 7febb1a8b700 1 -- 172.21.15.36:0/959969238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febac0aeea0 msgr2=0x7febac0a5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:10.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.222+0000 7febb1a8b700 1 --2- 172.21.15.36:0/959969238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febac0aeea0 0x7febac0a5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:10.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.222+0000 7febb1a8b700 1 -- 172.21.15.36:0/959969238 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb9c00b180 con 0x7febac0afe30 2022-06-17T12:53:10.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.222+0000 7febb128a700 1 --2- 172.21.15.36:0/959969238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febac0aeea0 0x7febac0a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:53:10.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.222+0000 7febb1a8b700 1 --2- 172.21.15.36:0/959969238 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febac0afe30 0x7febac0a3d30 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7feba400a000 tx=0x7feba4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:10.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.222+0000 7feb9a7fc700 1 -- 172.21.15.36:0/959969238 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feba400f710 con 0x7febac0afe30 2022-06-17T12:53:10.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.222+0000 7feb9a7fc700 1 -- 172.21.15.36:0/959969238 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba400f870 con 0x7febac0afe30 2022-06-17T12:53:10.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.222+0000 7febb228c700 1 -- 172.21.15.36:0/959969238 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febac0a5a80 con 0x7febac0afe30 2022-06-17T12:53:10.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.222+0000 7feb9a7fc700 1 -- 172.21.15.36:0/959969238 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feba400fbd0 con 0x7febac0afe30 2022-06-17T12:53:10.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.222+0000 7febb228c700 1 -- 172.21.15.36:0/959969238 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febac0a2d40 con 0x7febac0afe30 2022-06-17T12:53:10.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.226+0000 7feb9a7fc700 1 -- 172.21.15.36:0/959969238 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7feba401a030 con 0x7febac0afe30 2022-06-17T12:53:10.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.226+0000 7febb228c700 1 -- 172.21.15.36:0/959969238 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febac0abd80 con 0x7febac0afe30 2022-06-17T12:53:10.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.226+0000 7feb9a7fc700 1 --2- 172.21.15.36:0/959969238 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7feb8c0539b0 0x7feb8c055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:10.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.226+0000 7feb9a7fc700 1 -- 172.21.15.36:0/959969238 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6420+0+0 (secure 0 0 0) 0x7feba406be10 con 0x7febac0afe30 2022-06-17T12:53:10.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.226+0000 7febb128a700 1 --2- 172.21.15.36:0/959969238 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7feb8c0539b0 0x7feb8c055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:10.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.226+0000 7febb128a700 1 --2- 172.21.15.36:0/959969238 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7feb8c0539b0 0x7feb8c055e70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7feba800d000 tx=0x7feba8009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:10.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.230+0000 7feb9a7fc700 1 -- 172.21.15.36:0/959969238 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7feba4042bb0 con 0x7febac0afe30 2022-06-17T12:53:10.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.394+0000 7febb228c700 1 -- 172.21.15.36:0/959969238 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febac057410 con 0x7febac0afe30 2022-06-17T12:53:10.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.394+0000 7feb9a7fc700 1 -- 172.21.15.36:0/959969238 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+2140 (secure 0 0 0) 0x7feba40461d0 con 0x7febac0afe30 2022-06-17T12:53:10.381 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:53:10.381 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":36,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/2541998896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2541998896},{"type":"v1","addr":"172.21.15.70:6833","nonce":2541998896}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:53:04.907795+0000","modified":"2022-06-17T12:53:06.511529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6381":{"gid":6381,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/2514005995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2514005995},{"type":"v1","addr":"172.21.15.36:6835","nonce":2514005995}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-06-17T12:53:10.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.398+0000 7febb228c700 1 -- 172.21.15.36:0/959969238 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7feb8c0539b0 msgr2=0x7feb8c055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:10.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.398+0000 7febb228c700 1 --2- 172.21.15.36:0/959969238 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7feb8c0539b0 0x7feb8c055e70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7feba800d000 tx=0x7feba8009040).stop 2022-06-17T12:53:10.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.398+0000 7febb228c700 1 -- 172.21.15.36:0/959969238 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febac0afe30 msgr2=0x7febac0a3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:10.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.398+0000 7febb228c700 1 --2- 172.21.15.36:0/959969238 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febac0afe30 0x7febac0a3d30 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7feba400a000 tx=0x7feba4008040).stop 2022-06-17T12:53:10.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.398+0000 7febb228c700 1 -- 172.21.15.36:0/959969238 shutdown_connections 2022-06-17T12:53:10.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.398+0000 7febb228c700 1 --2- 172.21.15.36:0/959969238 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7feb8c0539b0 0x7feb8c055e70 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:10.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.398+0000 7febb228c700 1 --2- 172.21.15.36:0/959969238 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7febac0afe30 0x7febac0a3d30 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:10.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.398+0000 7febb228c700 1 --2- 172.21.15.36:0/959969238 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febac0af930 0x7febac0a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:10.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.398+0000 7febb228c700 1 --2- 172.21.15.36:0/959969238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febac0aeea0 0x7febac0a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:10.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.398+0000 7febb228c700 1 -- 172.21.15.36:0/959969238 >> 172.21.15.36:0/959969238 conn(0x7febac05f770 msgr2=0x7febac1418b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:10.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.398+0000 7febb228c700 1 -- 172.21.15.36:0/959969238 shutdown_connections 2022-06-17T12:53:10.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:10.398+0000 7febb228c700 1 -- 172.21.15.36:0/959969238 wait complete. 2022-06-17T12:53:10.389 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 36 2022-06-17T12:53:10.595 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:53:04.907795+0000', 'modified': '2022-06-17T12:53:06.511529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6381}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6381': {'gid': 6381, 'name': 'a', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.36:6835/2514005995', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 2514005995}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 2514005995}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:53:10.596 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:53:10.596 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:53:10.715 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":6381,"want_state":"up:active","state":"up:active","rank_uptime":5.8121505190000002,"mdsmap_epoch":36,"osdmap_epoch":74,"osdmap_epoch_barrier":74,"uptime":10.207025781} 2022-06-17T12:53:10.716 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 6381, "mdsmap_epoch": 36, "osdmap_epoch": 74, "osdmap_epoch_barrier": 74, "rank_uptime": 5.812150519, "state": "up:active", "uptime": 10.207025781, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:53:10.716 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:53:10.717 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-17T12:53:10.718 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi107.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-17T12:53:10.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:10.719 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:53:10.728 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:53:10.747 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:53:10.747 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:53:10.810 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:53:10.811 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:53:10.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:10.812 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:53:10.822 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:53:10.823 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:53:10.875 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:53:10.893 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:53:10.893 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:53:10.943 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:2022-06-17T12:53:10.953+0000 7fafcbcbd080 -1 init, newargv = 0x563352802610 newargc=9 2022-06-17T12:53:10.944 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22351]: starting ceph client 2022-06-17T12:53:10.947 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:53:10.948 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:53:10.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:10.950 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:53:10.954 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22351]: starting fuse 2022-06-17T12:53:10.956 INFO:teuthology.orchestra.run.smithi107.stdout:52 2022-06-17T12:53:10.957 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:53:10.958 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:10.960 DEBUG:teuthology.orchestra.run.smithi107:> sudo python3 -c ' 2022-06-17T12:53:10.960 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2022-06-17T12:53:10.961 DEBUG:teuthology.orchestra.run.smithi107:> import re 2022-06-17T12:53:10.962 DEBUG:teuthology.orchestra.run.smithi107:> import os 2022-06-17T12:53:10.964 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2022-06-17T12:53:10.964 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:53:10.965 DEBUG:teuthology.orchestra.run.smithi107:> def find_socket(client_name): 2022-06-17T12:53:10.966 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-17T12:53:10.967 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2022-06-17T12:53:10.967 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:53:10.968 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2022-06-17T12:53:10.969 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2022-06-17T12:53:10.970 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2022-06-17T12:53:10.970 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2022-06-17T12:53:10.971 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:53:10.972 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2022-06-17T12:53:10.973 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:53:10.974 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:53:10.975 DEBUG:teuthology.orchestra.run.smithi107:> return f 2022-06-17T12:53:10.976 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:53:10.978 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:53:10.978 DEBUG:teuthology.orchestra.run.smithi107:> print(find_socket("client.0")) 2022-06-17T12:53:10.979 DEBUG:teuthology.orchestra.run.smithi107:> ' 2022-06-17T12:53:11.023 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.0.22351.asok 2022-06-17T12:53:11.029 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22351.asok 2022-06-17T12:53:11.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:11.031 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22351.asok status 2022-06-17T12:53:11.122 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2022-06-17T12:53:11.123 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2022-06-17T12:53:11.123 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:53:11.124 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:53:11.124 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "0", 2022-06-17T12:53:11.125 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2022-06-17T12:53:11.125 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-17T12:53:11.125 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "22351", 2022-06-17T12:53:11.126 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/" 2022-06-17T12:53:11.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:53:11.127 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2022-06-17T12:53:11.127 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2022-06-17T12:53:11.127 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 6352, 2022-06-17T12:53:11.128 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2022-06-17T12:53:11.128 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2022-06-17T12:53:11.129 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2022-06-17T12:53:11.129 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 6352 2022-06-17T12:53:11.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:53:11.130 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:53:11.130 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:53:11.131 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:53:11.131 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 973797138 2022-06-17T12:53:11.132 INFO:teuthology.orchestra.run.smithi107.stdout: } 2022-06-17T12:53:11.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:53:11.132 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:53:11.133 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:53:11.133 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:53:11.133 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 973797138 2022-06-17T12:53:11.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:53:11.134 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.6352 172.21.15.107:0/973797138", 2022-06-17T12:53:11.135 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "172.21.15.107:0/973797138", 2022-06-17T12:53:11.135 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 1, 2022-06-17T12:53:11.135 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 36, 2022-06-17T12:53:11.136 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 74, 2022-06-17T12:53:11.136 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:53:11.136 INFO:teuthology.orchestra.run.smithi107.stdout: "blacklisted": false, 2022-06-17T12:53:11.137 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2022-06-17T12:53:11.137 INFO:teuthology.orchestra.run.smithi107.stdout:} 2022-06-17T12:53:11.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:11.141 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:53:11.142 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:53:11.151 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:53:11.153 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:53:11.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:11.155 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:53:11.175 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:53:11.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbfae32700 1 Processor -- start 2022-06-17T12:53:11.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbfae32700 1 -- start start 2022-06-17T12:53:11.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbfae32700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbbf4144c00 0x7fbbf4062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:11.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbfae32700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbbf40ad330 0x7fbbf40ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:11.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbfae32700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf40b0ba0 con 0x7fbbf4062770 2022-06-17T12:53:11.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbfae32700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbf40ae820 con 0x7fbbf4144c00 2022-06-17T12:53:11.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbfae32700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbf4061030 con 0x7fbbf40ad330 2022-06-17T12:53:11.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbf3fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbbf4144c00 0x7fbbf4062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:11.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbf3fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbbf4144c00 0x7fbbf4062b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35726/0 (socket says 172.21.15.36:35726) 2022-06-17T12:53:11.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbf3fff700 1 -- 172.21.15.36:0/1940406109 learned_addr learned my addr 172.21.15.36:0/1940406109 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:11.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbf8bce700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fbbf4062770 0x7fbbf41444f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53258/0 (socket says 172.21.15.36:53258) 2022-06-17T12:53:11.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbf93cf700 1 --2- 172.21.15.36:0/1940406109 >> v2:172.21.15.70:3300/0 conn(0x7fbbf40ad330 0x7fbbf40ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:11.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbf3fff700 1 -- 172.21.15.36:0/1940406109 >> v2:172.21.15.70:3300/0 conn(0x7fbbf40ad330 msgr2=0x7fbbf40ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:11.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbf3fff700 1 --2- 172.21.15.36:0/1940406109 >> v2:172.21.15.70:3300/0 conn(0x7fbbf40ad330 0x7fbbf40ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbf3fff700 1 -- 172.21.15.36:0/1940406109 >> v1:172.21.15.36:6789/0 conn(0x7fbbf4062770 legacy=0x7fbbf41444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:11.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbf37fe700 1 -- 172.21.15.36:0/1940406109 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 70524461 0 0) 0x7fbbf40b0ba0 con 0x7fbbf4062770 2022-06-17T12:53:11.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbf3fff700 1 -- 172.21.15.36:0/1940406109 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf4147e90 con 0x7fbbf4144c00 2022-06-17T12:53:11.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbf3fff700 1 --2- 172.21.15.36:0/1940406109 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbbf4144c00 0x7fbbf4062b50 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7fbbf4061170 tx=0x7fbbe4006eb0).ready entity=mon.0 client_cookie=a599175ab536c6b6 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:11.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbf37fe700 1 -- 172.21.15.36:0/1940406109 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbe400d610 con 0x7fbbf4144c00 2022-06-17T12:53:11.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbf37fe700 1 -- 172.21.15.36:0/1940406109 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe4000e90 con 0x7fbbf4144c00 2022-06-17T12:53:11.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbf37fe700 1 -- 172.21.15.36:0/1940406109 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbe400d960 con 0x7fbbf4144c00 2022-06-17T12:53:11.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbfae32700 1 -- 172.21.15.36:0/1940406109 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbbf4144c00 msgr2=0x7fbbf4062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:11.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbfae32700 1 --2- 172.21.15.36:0/1940406109 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbbf4144c00 0x7fbbf4062b50 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7fbbf4061170 tx=0x7fbbe4006eb0).stop 2022-06-17T12:53:11.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbfae32700 1 -- 172.21.15.36:0/1940406109 shutdown_connections 2022-06-17T12:53:11.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbfae32700 1 --2- 172.21.15.36:0/1940406109 >> v2:172.21.15.70:3300/0 conn(0x7fbbf40ad330 0x7fbbf40ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbfae32700 1 --2- 172.21.15.36:0/1940406109 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbbf4144c00 0x7fbbf4062b50 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbfae32700 1 -- 172.21.15.36:0/1940406109 >> 172.21.15.36:0/1940406109 conn(0x7fbbf40b53e0 msgr2=0x7fbbf40b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:11.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbfae32700 1 -- 172.21.15.36:0/1940406109 shutdown_connections 2022-06-17T12:53:11.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.306+0000 7fbbfae32700 1 -- 172.21.15.36:0/1940406109 wait complete. 2022-06-17T12:53:11.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.310+0000 7fbbfae32700 1 Processor -- start 2022-06-17T12:53:11.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.310+0000 7fbbfae32700 1 -- start start 2022-06-17T12:53:11.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.310+0000 7fbbfae32700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbf4062770 0x7fbbf41314d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:11.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.310+0000 7fbbfae32700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbf40ad330 0x7fbbf40670a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:11.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.310+0000 7fbbfae32700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbbf4144c00 0x7fbbf4144090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:11.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.310+0000 7fbbfae32700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf4067590 con 0x7fbbf4144c00 2022-06-17T12:53:11.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.310+0000 7fbbfae32700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf4065a10 con 0x7fbbf4062770 2022-06-17T12:53:11.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.310+0000 7fbbfae32700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf41576c0 con 0x7fbbf40ad330 2022-06-17T12:53:11.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.310+0000 7fbbf93cf700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbbf4144c00 0x7fbbf4144090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:11.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.310+0000 7fbbf93cf700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbbf4144c00 0x7fbbf4144090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35736/0 (socket says 172.21.15.36:35736) 2022-06-17T12:53:11.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.310+0000 7fbbf93cf700 1 -- 172.21.15.36:0/3439584312 learned_addr learned my addr 172.21.15.36:0/3439584312 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:11.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.310+0000 7fbbf8bce700 1 --2- 172.21.15.36:0/3439584312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbf4062770 0x7fbbf41314d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:11.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.310+0000 7fbbf3fff700 1 --2- 172.21.15.36:0/3439584312 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbf40ad330 0x7fbbf40670a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:11.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.310+0000 7fbbf93cf700 1 -- 172.21.15.36:0/3439584312 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbf40ad330 msgr2=0x7fbbf40670a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:11.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.310+0000 7fbbf93cf700 1 --2- 172.21.15.36:0/3439584312 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbf40ad330 0x7fbbf40670a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.310+0000 7fbbf93cf700 1 -- 172.21.15.36:0/3439584312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbf4062770 msgr2=0x7fbbf41314d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:11.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.310+0000 7fbbf93cf700 1 --2- 172.21.15.36:0/3439584312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbf4062770 0x7fbbf41314d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.310+0000 7fbbf93cf700 1 -- 172.21.15.36:0/3439584312 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe400b1e0 con 0x7fbbf4144c00 2022-06-17T12:53:11.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.310+0000 7fbbf93cf700 1 --2- 172.21.15.36:0/3439584312 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbbf4144c00 0x7fbbf4144090 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7fbbf40676d0 tx=0x7fbbec0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:11.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.314+0000 7fbbf1ffb700 1 -- 172.21.15.36:0/3439584312 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbec00c640 con 0x7fbbf4144c00 2022-06-17T12:53:11.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.314+0000 7fbbf1ffb700 1 -- 172.21.15.36:0/3439584312 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbec00c7a0 con 0x7fbbf4144c00 2022-06-17T12:53:11.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.314+0000 7fbbfae32700 1 -- 172.21.15.36:0/3439584312 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf40649b0 con 0x7fbbf4144c00 2022-06-17T12:53:11.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.314+0000 7fbbf1ffb700 1 -- 172.21.15.36:0/3439584312 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbec00cb00 con 0x7fbbf4144c00 2022-06-17T12:53:11.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.314+0000 7fbbfae32700 1 -- 172.21.15.36:0/3439584312 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf414b530 con 0x7fbbf4144c00 2022-06-17T12:53:11.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.314+0000 7fbbfae32700 1 -- 172.21.15.36:0/3439584312 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf4065b50 con 0x7fbbf4144c00 2022-06-17T12:53:11.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.314+0000 7fbbf1ffb700 1 -- 172.21.15.36:0/3439584312 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fbbec022070 con 0x7fbbf4144c00 2022-06-17T12:53:11.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.318+0000 7fbbf1ffb700 1 --2- 172.21.15.36:0/3439584312 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbbd4053ad0 0x7fbbd4055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:11.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.318+0000 7fbbf8bce700 1 --2- 172.21.15.36:0/3439584312 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbbd4053ad0 0x7fbbd4055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:11.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.318+0000 7fbbf1ffb700 1 -- 172.21.15.36:0/3439584312 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbbec06c5a0 con 0x7fbbf4144c00 2022-06-17T12:53:11.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.318+0000 7fbbf8bce700 1 --2- 172.21.15.36:0/3439584312 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbbd4053ad0 0x7fbbd4055f90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fbbe800d000 tx=0x7fbbe8007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:11.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.322+0000 7fbbf1ffb700 1 -- 172.21.15.36:0/3439584312 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbbec0433a0 con 0x7fbbf4144c00 2022-06-17T12:53:11.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.454+0000 7fbbfae32700 1 -- 172.21.15.36:0/3439584312 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbf40b1c20 con 0x7fbbf4144c00 2022-06-17T12:53:11.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.454+0000 7fbbf1ffb700 1 -- 172.21.15.36:0/3439584312 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16274 (secure 0 0 0) 0x7fbbec0469c0 con 0x7fbbf4144c00 2022-06-17T12:53:11.440 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:53:11.441 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":74,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:53:05.910998+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:53:03.265804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-06-17T12:53:04.385158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:53:11.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.458+0000 7fbbfae32700 1 -- 172.21.15.36:0/3439584312 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbbd4053ad0 msgr2=0x7fbbd4055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:11.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.458+0000 7fbbfae32700 1 --2- 172.21.15.36:0/3439584312 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbbd4053ad0 0x7fbbd4055f90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fbbe800d000 tx=0x7fbbe8007a00).stop 2022-06-17T12:53:11.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.458+0000 7fbbfae32700 1 -- 172.21.15.36:0/3439584312 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbbf4144c00 msgr2=0x7fbbf4144090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:11.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.458+0000 7fbbfae32700 1 --2- 172.21.15.36:0/3439584312 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbbf4144c00 0x7fbbf4144090 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7fbbf40676d0 tx=0x7fbbec0091d0).stop 2022-06-17T12:53:11.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.458+0000 7fbbfae32700 1 -- 172.21.15.36:0/3439584312 shutdown_connections 2022-06-17T12:53:11.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.458+0000 7fbbfae32700 1 --2- 172.21.15.36:0/3439584312 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbbd4053ad0 0x7fbbd4055f90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.458+0000 7fbbfae32700 1 --2- 172.21.15.36:0/3439584312 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbbf4144c00 0x7fbbf4144090 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.458+0000 7fbbfae32700 1 --2- 172.21.15.36:0/3439584312 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbf40ad330 0x7fbbf40670a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.458+0000 7fbbfae32700 1 --2- 172.21.15.36:0/3439584312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbf4062770 0x7fbbf41314d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.458+0000 7fbbfae32700 1 -- 172.21.15.36:0/3439584312 >> 172.21.15.36:0/3439584312 conn(0x7fbbf40b53e0 msgr2=0x7fbbf40ad7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:11.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.458+0000 7fbbfae32700 1 -- 172.21.15.36:0/3439584312 shutdown_connections 2022-06-17T12:53:11.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.458+0000 7fbbfae32700 1 -- 172.21.15.36:0/3439584312 wait complete. 2022-06-17T12:53:11.459 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:53:03.265804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:53:04.385158+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:53:11.461 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:53:11.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.586+0000 7f46a49e3700 1 Processor -- start 2022-06-17T12:53:11.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.586+0000 7f46a49e3700 1 -- start start 2022-06-17T12:53:11.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.586+0000 7f46a49e3700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f46a00ae360 0x7f46a0060260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:11.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.586+0000 7f46a49e3700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a00633c0 con 0x7f46a00b1db0 2022-06-17T12:53:11.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.586+0000 7f46a49e3700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a0143330 con 0x7f46a005fe80 2022-06-17T12:53:11.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.586+0000 7f46a49e3700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a0063100 con 0x7f46a00ae360 2022-06-17T12:53:11.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.586+0000 7f469ffff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f46a00b1db0 0x7f46a01437d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53272/0 (socket says 172.21.15.36:53272) 2022-06-17T12:53:11.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.586+0000 7f469ffff700 1 -- 172.21.15.36:0/3433605914 learned_addr learned my addr 172.21.15.36:0/3433605914 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:53:11.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.586+0000 7f4697fff700 1 --2- 172.21.15.36:0/3433605914 >> v2:172.21.15.70:3300/0 conn(0x7f46a00ae360 0x7f46a0060260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:11.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.586+0000 7f469effd700 1 -- 172.21.15.36:0/3433605914 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2335251545 0 0) 0x7f46a00633c0 con 0x7f46a00b1db0 2022-06-17T12:53:11.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.586+0000 7f469effd700 1 -- 172.21.15.36:0/3433605914 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4680003260 con 0x7f46a00b1db0 2022-06-17T12:53:11.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.586+0000 7f4697fff700 1 -- 172.21.15.36:0/3433605914 >> v1:172.21.15.70:6789/0 conn(0x7f46a005fe80 legacy=0x7f46a01444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:11.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.586+0000 7f4697fff700 1 -- 172.21.15.36:0/3433605914 >> v1:172.21.15.36:6789/0 conn(0x7f46a00b1db0 legacy=0x7f46a01437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:11.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.586+0000 7f4697fff700 1 -- 172.21.15.36:0/3433605914 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a0143020 con 0x7f46a00ae360 2022-06-17T12:53:11.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f4697fff700 1 --2- 172.21.15.36:0/3433605914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46a00ae360 0x7f46a0060260 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f4688002f70 tx=0x7f4688006df0).ready entity=mon.1 client_cookie=aa83685c88663c55 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:11.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f469effd700 1 -- 172.21.15.36:0/3433605914 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f468800d610 con 0x7f46a00ae360 2022-06-17T12:53:11.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f469effd700 1 -- 172.21.15.36:0/3433605914 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4688000e90 con 0x7f46a00ae360 2022-06-17T12:53:11.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f469effd700 1 -- 172.21.15.36:0/3433605914 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f468800d960 con 0x7f46a00ae360 2022-06-17T12:53:11.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f46a49e3700 1 -- 172.21.15.36:0/3433605914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46a00ae360 msgr2=0x7f46a0060260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:11.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f46a49e3700 1 --2- 172.21.15.36:0/3433605914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46a00ae360 0x7f46a0060260 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f4688002f70 tx=0x7f4688006df0).stop 2022-06-17T12:53:11.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f46a49e3700 1 -- 172.21.15.36:0/3433605914 shutdown_connections 2022-06-17T12:53:11.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f46a49e3700 1 --2- 172.21.15.36:0/3433605914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46a00ae360 0x7f46a0060260 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f46a49e3700 1 -- 172.21.15.36:0/3433605914 >> 172.21.15.36:0/3433605914 conn(0x7f46a0060990 msgr2=0x7f46a005f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:11.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f46a49e3700 1 -- 172.21.15.36:0/3433605914 shutdown_connections 2022-06-17T12:53:11.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f46a49e3700 1 -- 172.21.15.36:0/3433605914 wait complete. 2022-06-17T12:53:11.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f46a49e3700 1 Processor -- start 2022-06-17T12:53:11.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f46a49e3700 1 -- start start 2022-06-17T12:53:11.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f46a49e3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f46a005fe80 0x7f46a0155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:11.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f46a49e3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46a00ae360 0x7f46a013ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:11.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f46a49e3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f46a00b1db0 0x7f46a00b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:11.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f46a49e3700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a013e8c0 con 0x7f46a00b1db0 2022-06-17T12:53:11.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f46a49e3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a0066a20 con 0x7f46a00ae360 2022-06-17T12:53:11.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f46a49e3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a0066b90 con 0x7f46a005fe80 2022-06-17T12:53:11.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f469ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f46a00b1db0 0x7f46a00b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:11.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f469ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f46a00b1db0 0x7f46a00b0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35750/0 (socket says 172.21.15.36:35750) 2022-06-17T12:53:11.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f469ffff700 1 -- 172.21.15.36:0/1823334731 learned_addr learned my addr 172.21.15.36:0/1823334731 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:11.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f4697fff700 1 --2- 172.21.15.36:0/1823334731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46a00ae360 0x7f46a013ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:11.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f469f7fe700 1 --2- 172.21.15.36:0/1823334731 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f46a005fe80 0x7f46a0155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:11.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f469ffff700 1 -- 172.21.15.36:0/1823334731 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f46a005fe80 msgr2=0x7f46a0155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:11.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f469ffff700 1 --2- 172.21.15.36:0/1823334731 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f46a005fe80 0x7f46a0155a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f469ffff700 1 -- 172.21.15.36:0/1823334731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46a00ae360 msgr2=0x7f46a013ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:11.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f469ffff700 1 --2- 172.21.15.36:0/1823334731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46a00ae360 0x7f46a013ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f469ffff700 1 -- 172.21.15.36:0/1823334731 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468800b180 con 0x7f46a00b1db0 2022-06-17T12:53:11.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.590+0000 7f469ffff700 1 --2- 172.21.15.36:0/1823334731 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f46a00b1db0 0x7f46a00b0f30 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7f4698001490 tx=0x7f4698007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:11.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.594+0000 7f469d7fa700 1 -- 172.21.15.36:0/1823334731 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4698005ba0 con 0x7f46a00b1db0 2022-06-17T12:53:11.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.594+0000 7f469d7fa700 1 -- 172.21.15.36:0/1823334731 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4698005d00 con 0x7f46a00b1db0 2022-06-17T12:53:11.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.594+0000 7f46a49e3700 1 -- 172.21.15.36:0/1823334731 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a0066460 con 0x7f46a00b1db0 2022-06-17T12:53:11.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.594+0000 7f469d7fa700 1 -- 172.21.15.36:0/1823334731 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46980072c0 con 0x7f46a00b1db0 2022-06-17T12:53:11.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.594+0000 7f46a49e3700 1 -- 172.21.15.36:0/1823334731 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a00b1420 con 0x7f46a00b1db0 2022-06-17T12:53:11.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.594+0000 7f46a49e3700 1 -- 172.21.15.36:0/1823334731 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a0067550 con 0x7f46a00b1db0 2022-06-17T12:53:11.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.594+0000 7f469d7fa700 1 -- 172.21.15.36:0/1823334731 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f4698010070 con 0x7f46a00b1db0 2022-06-17T12:53:11.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.594+0000 7f469d7fa700 1 --2- 172.21.15.36:0/1823334731 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4684053920 0x7f4684055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:11.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.594+0000 7f469d7fa700 1 -- 172.21.15.36:0/1823334731 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f469806ba70 con 0x7f46a00b1db0 2022-06-17T12:53:11.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.594+0000 7f469f7fe700 1 --2- 172.21.15.36:0/1823334731 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4684053920 0x7f4684055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:11.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.594+0000 7f469f7fe700 1 --2- 172.21.15.36:0/1823334731 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4684053920 0x7f4684055de0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f469000d000 tx=0x7f4690007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:11.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.602+0000 7f469d7fa700 1 -- 172.21.15.36:0/1823334731 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f46a0067550 con 0x7f46a00b1db0 2022-06-17T12:53:11.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.730+0000 7f46a49e3700 1 -- 172.21.15.36:0/1823334731 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f46a0067550 con 0x7f46a00b1db0 2022-06-17T12:53:11.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.730+0000 7f469d7fa700 1 -- 172.21.15.36:0/1823334731 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v74) v1 ==== 110+0+10 (secure 0 0 0) 0x7f46a0067550 con 0x7f46a00b1db0 2022-06-17T12:53:11.716 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:53:11.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.730+0000 7f46a49e3700 1 -- 172.21.15.36:0/1823334731 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4684053920 msgr2=0x7f4684055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:11.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.730+0000 7f46a49e3700 1 --2- 172.21.15.36:0/1823334731 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4684053920 0x7f4684055de0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f469000d000 tx=0x7f4690007b90).stop 2022-06-17T12:53:11.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.730+0000 7f46a49e3700 1 -- 172.21.15.36:0/1823334731 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f46a00b1db0 msgr2=0x7f46a00b0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:11.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.730+0000 7f46a49e3700 1 --2- 172.21.15.36:0/1823334731 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f46a00b1db0 0x7f46a00b0f30 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7f4698001490 tx=0x7f4698007dc0).stop 2022-06-17T12:53:11.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.730+0000 7f46a49e3700 1 -- 172.21.15.36:0/1823334731 shutdown_connections 2022-06-17T12:53:11.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.730+0000 7f46a49e3700 1 --2- 172.21.15.36:0/1823334731 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4684053920 0x7f4684055de0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.730+0000 7f46a49e3700 1 --2- 172.21.15.36:0/1823334731 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f46a00b1db0 0x7f46a00b0f30 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.730+0000 7f46a49e3700 1 --2- 172.21.15.36:0/1823334731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46a00ae360 0x7f46a013ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.730+0000 7f46a49e3700 1 --2- 172.21.15.36:0/1823334731 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f46a005fe80 0x7f46a0155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.730+0000 7f46a49e3700 1 -- 172.21.15.36:0/1823334731 >> 172.21.15.36:0/1823334731 conn(0x7f46a0060990 msgr2=0x7f46a00b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:11.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.734+0000 7f46a49e3700 1 -- 172.21.15.36:0/1823334731 shutdown_connections 2022-06-17T12:53:11.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.734+0000 7f46a49e3700 1 -- 172.21.15.36:0/1823334731 wait complete. 2022-06-17T12:53:11.728 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:53:11.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.854+0000 7f2a5e436700 1 Processor -- start 2022-06-17T12:53:11.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.854+0000 7f2a5e436700 1 -- start start 2022-06-17T12:53:11.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.854+0000 7f2a5e436700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2a580b45c0 0x7f2a580b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:11.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.854+0000 7f2a5e436700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2a580b4f00 0x7f2a580b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:11.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.854+0000 7f2a5e436700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a580b4160 con 0x7f2a580b9860 2022-06-17T12:53:11.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.854+0000 7f2a5e436700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a580b9c40 con 0x7f2a580b4f00 2022-06-17T12:53:11.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.854+0000 7f2a5e436700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a580627d0 con 0x7f2a580b45c0 2022-06-17T12:53:11.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.854+0000 7f2a5d434700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f2a580b9860 0x7f2a58143840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53284/0 (socket says 172.21.15.36:53284) 2022-06-17T12:53:11.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.854+0000 7f2a5d434700 1 -- 172.21.15.36:0/1129485146 learned_addr learned my addr 172.21.15.36:0/1129485146 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:53:11.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.854+0000 7f2a5cc33700 1 --2- 172.21.15.36:0/1129485146 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2a580b45c0 0x7f2a580b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:11.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.854+0000 7f2a5dc35700 1 --2- 172.21.15.36:0/1129485146 >> v2:172.21.15.70:3300/0 conn(0x7f2a580b4f00 0x7f2a580b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:11.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.854+0000 7f2a4ffff700 1 -- 172.21.15.36:0/1129485146 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1761788618 0 0) 0x7f2a580b4160 con 0x7f2a580b9860 2022-06-17T12:53:11.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.854+0000 7f2a4ffff700 1 -- 172.21.15.36:0/1129485146 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a3c003260 con 0x7f2a580b9860 2022-06-17T12:53:11.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.854+0000 7f2a5cc33700 1 -- 172.21.15.36:0/1129485146 >> v2:172.21.15.70:3300/0 conn(0x7f2a580b4f00 msgr2=0x7f2a580b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:11.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.854+0000 7f2a5cc33700 1 --2- 172.21.15.36:0/1129485146 >> v2:172.21.15.70:3300/0 conn(0x7f2a580b4f00 0x7f2a580b5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.854+0000 7f2a5cc33700 1 -- 172.21.15.36:0/1129485146 >> v1:172.21.15.36:6789/0 conn(0x7f2a580b9860 legacy=0x7f2a58143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:11.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.854+0000 7f2a5cc33700 1 -- 172.21.15.36:0/1129485146 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a58062a70 con 0x7f2a580b45c0 2022-06-17T12:53:11.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.854+0000 7f2a5cc33700 1 --2- 172.21.15.36:0/1129485146 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2a580b45c0 0x7f2a580b4a10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f2a48002f70 tx=0x7f2a48006df0).ready entity=mon.2 client_cookie=d22ec09b2e1665e3 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:11.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.854+0000 7f2a4ffff700 1 -- 172.21.15.36:0/1129485146 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a4800d4a0 con 0x7f2a580b45c0 2022-06-17T12:53:11.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.854+0000 7f2a4ffff700 1 -- 172.21.15.36:0/1129485146 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a48000e90 con 0x7f2a580b45c0 2022-06-17T12:53:11.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.854+0000 7f2a4ffff700 1 -- 172.21.15.36:0/1129485146 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a4800d7f0 con 0x7f2a580b45c0 2022-06-17T12:53:11.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.858+0000 7f2a5e436700 1 -- 172.21.15.36:0/1129485146 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2a580b45c0 msgr2=0x7f2a580b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:11.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.858+0000 7f2a5e436700 1 --2- 172.21.15.36:0/1129485146 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2a580b45c0 0x7f2a580b4a10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f2a48002f70 tx=0x7f2a48006df0).stop 2022-06-17T12:53:11.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.858+0000 7f2a5e436700 1 -- 172.21.15.36:0/1129485146 shutdown_connections 2022-06-17T12:53:11.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.858+0000 7f2a5e436700 1 --2- 172.21.15.36:0/1129485146 >> v2:172.21.15.70:3300/0 conn(0x7f2a580b4f00 0x7f2a580b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.858+0000 7f2a5e436700 1 --2- 172.21.15.36:0/1129485146 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2a580b45c0 0x7f2a580b4a10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.858+0000 7f2a5e436700 1 -- 172.21.15.36:0/1129485146 >> 172.21.15.36:0/1129485146 conn(0x7f2a5805c550 msgr2=0x7f2a58060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:11.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.858+0000 7f2a5e436700 1 -- 172.21.15.36:0/1129485146 shutdown_connections 2022-06-17T12:53:11.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.858+0000 7f2a5e436700 1 -- 172.21.15.36:0/1129485146 wait complete. 2022-06-17T12:53:11.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.858+0000 7f2a5e436700 1 Processor -- start 2022-06-17T12:53:11.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.858+0000 7f2a5e436700 1 -- start start 2022-06-17T12:53:11.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.858+0000 7f2a5e436700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2a580b45c0 0x7f2a5813cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:11.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.862+0000 7f2a5e436700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a580b4f00 0x7f2a5813d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:11.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.862+0000 7f2a5e436700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2a580b9860 0x7f2a5813dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:11.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.862+0000 7f2a5e436700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a580af030 con 0x7f2a580b45c0 2022-06-17T12:53:11.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.862+0000 7f2a5d434700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2a580b45c0 0x7f2a5813cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:11.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.862+0000 7f2a5e436700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a58147290 con 0x7f2a580b4f00 2022-06-17T12:53:11.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.862+0000 7f2a5e436700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a58146ff0 con 0x7f2a580b9860 2022-06-17T12:53:11.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.862+0000 7f2a5d434700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2a580b45c0 0x7f2a5813cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35758/0 (socket says 172.21.15.36:35758) 2022-06-17T12:53:11.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.862+0000 7f2a5d434700 1 -- 172.21.15.36:0/2366056529 learned_addr learned my addr 172.21.15.36:0/2366056529 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:11.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.862+0000 7f2a5cc33700 1 --2- 172.21.15.36:0/2366056529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a580b4f00 0x7f2a5813d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:11.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.862+0000 7f2a5dc35700 1 --2- 172.21.15.36:0/2366056529 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2a580b9860 0x7f2a5813dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:11.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.862+0000 7f2a5d434700 1 -- 172.21.15.36:0/2366056529 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2a580b9860 msgr2=0x7f2a5813dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:11.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.862+0000 7f2a5d434700 1 --2- 172.21.15.36:0/2366056529 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2a580b9860 0x7f2a5813dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.862+0000 7f2a5d434700 1 -- 172.21.15.36:0/2366056529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a580b4f00 msgr2=0x7f2a5813d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:11.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.862+0000 7f2a5d434700 1 --2- 172.21.15.36:0/2366056529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a580b4f00 0x7f2a5813d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.862+0000 7f2a5d434700 1 -- 172.21.15.36:0/2366056529 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4800b180 con 0x7f2a580b45c0 2022-06-17T12:53:11.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.862+0000 7f2a5d434700 1 --2- 172.21.15.36:0/2366056529 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2a580b45c0 0x7f2a5813cf00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7f2a54001490 tx=0x7f2a54007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:11.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.862+0000 7f2a4e7fc700 1 -- 172.21.15.36:0/2366056529 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a54005ba0 con 0x7f2a580b45c0 2022-06-17T12:53:11.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.862+0000 7f2a4e7fc700 1 -- 172.21.15.36:0/2366056529 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a54005d00 con 0x7f2a580b45c0 2022-06-17T12:53:11.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.862+0000 7f2a4e7fc700 1 -- 172.21.15.36:0/2366056529 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a540072c0 con 0x7f2a580b45c0 2022-06-17T12:53:11.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.862+0000 7f2a5e436700 1 -- 172.21.15.36:0/2366056529 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a58152d90 con 0x7f2a580b45c0 2022-06-17T12:53:11.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.862+0000 7f2a5e436700 1 -- 172.21.15.36:0/2366056529 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a581557c0 con 0x7f2a580b45c0 2022-06-17T12:53:11.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.862+0000 7f2a5e436700 1 -- 172.21.15.36:0/2366056529 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a58152f20 con 0x7f2a580b45c0 2022-06-17T12:53:11.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.862+0000 7f2a4e7fc700 1 -- 172.21.15.36:0/2366056529 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2a5401a070 con 0x7f2a580b45c0 2022-06-17T12:53:11.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.866+0000 7f2a4e7fc700 1 --2- 172.21.15.36:0/2366056529 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2a40053ad0 0x7f2a40055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:11.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.866+0000 7f2a4e7fc700 1 -- 172.21.15.36:0/2366056529 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2a5406b640 con 0x7f2a580b45c0 2022-06-17T12:53:11.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.866+0000 7f2a5cc33700 1 --2- 172.21.15.36:0/2366056529 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2a40053ad0 0x7f2a40055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:11.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.866+0000 7f2a5cc33700 1 --2- 172.21.15.36:0/2366056529 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2a40053ad0 0x7f2a40055f90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f2a48018000 tx=0x7f2a48014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:11.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.870+0000 7f2a4e7fc700 1 -- 172.21.15.36:0/2366056529 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2a54042440 con 0x7f2a580b45c0 2022-06-17T12:53:11.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.998+0000 7f2a5e436700 1 -- 172.21.15.36:0/2366056529 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2a58062eb0 con 0x7f2a580b45c0 2022-06-17T12:53:11.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:11.998+0000 7f2a4e7fc700 1 -- 172.21.15.36:0/2366056529 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v74) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2a54045a60 con 0x7f2a580b45c0 2022-06-17T12:53:11.984 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:53:11.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.002+0000 7f2a5e436700 1 -- 172.21.15.36:0/2366056529 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2a40053ad0 msgr2=0x7f2a40055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:11.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.002+0000 7f2a5e436700 1 --2- 172.21.15.36:0/2366056529 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2a40053ad0 0x7f2a40055f90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f2a48018000 tx=0x7f2a48014000).stop 2022-06-17T12:53:11.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.002+0000 7f2a5e436700 1 -- 172.21.15.36:0/2366056529 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2a580b45c0 msgr2=0x7f2a5813cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:11.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.002+0000 7f2a5e436700 1 --2- 172.21.15.36:0/2366056529 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2a580b45c0 0x7f2a5813cf00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7f2a54001490 tx=0x7f2a54007e20).stop 2022-06-17T12:53:11.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.002+0000 7f2a5e436700 1 -- 172.21.15.36:0/2366056529 shutdown_connections 2022-06-17T12:53:11.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.002+0000 7f2a5e436700 1 --2- 172.21.15.36:0/2366056529 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2a40053ad0 0x7f2a40055f90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.002+0000 7f2a5e436700 1 --2- 172.21.15.36:0/2366056529 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2a580b9860 0x7f2a5813dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.002+0000 7f2a5e436700 1 --2- 172.21.15.36:0/2366056529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a580b4f00 0x7f2a5813d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.002+0000 7f2a5e436700 1 --2- 172.21.15.36:0/2366056529 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2a580b45c0 0x7f2a5813cf00 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:11.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.002+0000 7f2a5e436700 1 -- 172.21.15.36:0/2366056529 >> 172.21.15.36:0/2366056529 conn(0x7f2a5805c550 msgr2=0x7f2a580b5be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:11.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.002+0000 7f2a5e436700 1 -- 172.21.15.36:0/2366056529 shutdown_connections 2022-06-17T12:53:11.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.002+0000 7f2a5e436700 1 -- 172.21.15.36:0/2366056529 wait complete. 2022-06-17T12:53:11.997 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:53:12.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.122+0000 7fee65e2a700 1 Processor -- start 2022-06-17T12:53:12.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.122+0000 7fee65e2a700 1 -- start start 2022-06-17T12:53:12.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.126+0000 7fee65e2a700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fee600b2ab0 0x7fee600b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:12.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.126+0000 7fee65e2a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fee600af5a0 0x7fee60140840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:12.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.126+0000 7fee65e2a700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee6013fe60 con 0x7fee600b1d40 2022-06-17T12:53:12.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.126+0000 7fee65e2a700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee60140100 con 0x7fee600b2ab0 2022-06-17T12:53:12.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.126+0000 7fee65e2a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee601403a0 con 0x7fee600af5a0 2022-06-17T12:53:12.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.126+0000 7fee5f7fe700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fee600b2ab0 0x7fee600b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:12.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.126+0000 7fee5effd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fee600af5a0 0x7fee60140840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:12.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.126+0000 7fee5ffff700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fee600b1d40 0x7fee60139bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:49034/0 (socket says 172.21.15.36:49034) 2022-06-17T12:53:12.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.126+0000 7fee5ffff700 1 -- 172.21.15.36:0/928335150 learned_addr learned my addr 172.21.15.36:0/928335150 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:53:12.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.126+0000 7fee5e7fc700 1 -- 172.21.15.36:0/928335150 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1123533545 0 0) 0x7fee6013fe60 con 0x7fee600b1d40 2022-06-17T12:53:12.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.126+0000 7fee5e7fc700 1 -- 172.21.15.36:0/928335150 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee4c003260 con 0x7fee600b1d40 2022-06-17T12:53:12.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.126+0000 7fee5f7fe700 1 -- 172.21.15.36:0/928335150 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fee600af5a0 msgr2=0x7fee60140840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:12.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.126+0000 7fee5f7fe700 1 --2- 172.21.15.36:0/928335150 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fee600af5a0 0x7fee60140840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.126+0000 7fee5f7fe700 1 -- 172.21.15.36:0/928335150 >> v1:172.21.15.70:6789/0 conn(0x7fee600b1d40 legacy=0x7fee60139bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:12.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.126+0000 7fee5f7fe700 1 -- 172.21.15.36:0/928335150 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6013a300 con 0x7fee600b2ab0 2022-06-17T12:53:12.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.126+0000 7fee5f7fe700 1 --2- 172.21.15.36:0/928335150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee600b2ab0 0x7fee600b2e90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7fee50002f70 tx=0x7fee50006df0).ready entity=mon.1 client_cookie=c615b593de3452cf server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:12.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.126+0000 7fee5e7fc700 1 -- 172.21.15.36:0/928335150 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee5000d4a0 con 0x7fee600b2ab0 2022-06-17T12:53:12.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.126+0000 7fee5e7fc700 1 -- 172.21.15.36:0/928335150 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee50000e90 con 0x7fee600b2ab0 2022-06-17T12:53:12.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.126+0000 7fee5e7fc700 1 -- 172.21.15.36:0/928335150 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee5000d7f0 con 0x7fee600b2ab0 2022-06-17T12:53:12.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.130+0000 7fee65e2a700 1 -- 172.21.15.36:0/928335150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee600b2ab0 msgr2=0x7fee600b2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:12.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.130+0000 7fee65e2a700 1 --2- 172.21.15.36:0/928335150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee600b2ab0 0x7fee600b2e90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7fee50002f70 tx=0x7fee50006df0).stop 2022-06-17T12:53:12.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.130+0000 7fee65e2a700 1 -- 172.21.15.36:0/928335150 shutdown_connections 2022-06-17T12:53:12.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.130+0000 7fee65e2a700 1 --2- 172.21.15.36:0/928335150 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fee600af5a0 0x7fee60140840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.130+0000 7fee65e2a700 1 --2- 172.21.15.36:0/928335150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee600b2ab0 0x7fee600b2e90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.130+0000 7fee65e2a700 1 -- 172.21.15.36:0/928335150 >> 172.21.15.36:0/928335150 conn(0x7fee60063020 msgr2=0x7fee60063430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:12.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.130+0000 7fee65e2a700 1 -- 172.21.15.36:0/928335150 shutdown_connections 2022-06-17T12:53:12.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.130+0000 7fee65e2a700 1 -- 172.21.15.36:0/928335150 wait complete. 2022-06-17T12:53:12.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.130+0000 7fee65e2a700 1 Processor -- start 2022-06-17T12:53:12.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.130+0000 7fee65e2a700 1 -- start start 2022-06-17T12:53:12.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.134+0000 7fee65e2a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee600af5a0 0x7fee6014fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:12.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.134+0000 7fee65e2a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fee600b1d40 0x7fee600643d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:12.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.134+0000 7fee65e2a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fee600b2ab0 0x7fee6013f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:12.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.134+0000 7fee65e2a700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee60142ef0 con 0x7fee600b2ab0 2022-06-17T12:53:12.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.134+0000 7fee65e2a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6014f3d0 con 0x7fee600af5a0 2022-06-17T12:53:12.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.134+0000 7fee65e2a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6014f670 con 0x7fee600b1d40 2022-06-17T12:53:12.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.134+0000 7fee5ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fee600b2ab0 0x7fee6013f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:12.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.134+0000 7fee5effd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fee600b1d40 0x7fee600643d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:12.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.134+0000 7fee5f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee600af5a0 0x7fee6014fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:12.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.134+0000 7fee5ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fee600b2ab0 0x7fee6013f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35776/0 (socket says 172.21.15.36:35776) 2022-06-17T12:53:12.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.134+0000 7fee5ffff700 1 -- 172.21.15.36:0/735938115 learned_addr learned my addr 172.21.15.36:0/735938115 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:12.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.134+0000 7fee5ffff700 1 -- 172.21.15.36:0/735938115 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fee600b1d40 msgr2=0x7fee600643d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:12.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.134+0000 7fee5ffff700 1 --2- 172.21.15.36:0/735938115 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fee600b1d40 0x7fee600643d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.134+0000 7fee5ffff700 1 -- 172.21.15.36:0/735938115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee600af5a0 msgr2=0x7fee6014fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:12.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.134+0000 7fee5ffff700 1 --2- 172.21.15.36:0/735938115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee600af5a0 0x7fee6014fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.134+0000 7fee5ffff700 1 -- 172.21.15.36:0/735938115 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5000b180 con 0x7fee600b2ab0 2022-06-17T12:53:12.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.134+0000 7fee5ffff700 1 --2- 172.21.15.36:0/735938115 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fee600b2ab0 0x7fee6013f5c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7fee58001490 tx=0x7fee58007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:12.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.134+0000 7fee5cff9700 1 -- 172.21.15.36:0/735938115 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee58005c70 con 0x7fee600b2ab0 2022-06-17T12:53:12.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.134+0000 7fee5cff9700 1 -- 172.21.15.36:0/735938115 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee58005dd0 con 0x7fee600b2ab0 2022-06-17T12:53:12.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.134+0000 7fee65e2a700 1 -- 172.21.15.36:0/735938115 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6014f910 con 0x7fee600b2ab0 2022-06-17T12:53:12.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.134+0000 7fee5cff9700 1 -- 172.21.15.36:0/735938115 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee580072c0 con 0x7fee600b2ab0 2022-06-17T12:53:12.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.134+0000 7fee65e2a700 1 -- 172.21.15.36:0/735938115 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee6014f0e0 con 0x7fee600b2ab0 2022-06-17T12:53:12.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.138+0000 7fee65e2a700 1 -- 172.21.15.36:0/735938115 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee600656d0 con 0x7fee600b2ab0 2022-06-17T12:53:12.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.138+0000 7fee5cff9700 1 -- 172.21.15.36:0/735938115 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fee58010070 con 0x7fee600b2ab0 2022-06-17T12:53:12.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.138+0000 7fee5cff9700 1 --2- 172.21.15.36:0/735938115 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fee40053920 0x7fee40055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:12.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.138+0000 7fee5cff9700 1 -- 172.21.15.36:0/735938115 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fee5806bb90 con 0x7fee600b2ab0 2022-06-17T12:53:12.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.138+0000 7fee5f7fe700 1 --2- 172.21.15.36:0/735938115 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fee40053920 0x7fee40055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:12.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.138+0000 7fee5f7fe700 1 --2- 172.21.15.36:0/735938115 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fee40053920 0x7fee40055de0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7fee5001a000 tx=0x7fee50014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:12.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.146+0000 7fee5cff9700 1 -- 172.21.15.36:0/735938115 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fee58042990 con 0x7fee600b2ab0 2022-06-17T12:53:12.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.274+0000 7fee65e2a700 1 -- 172.21.15.36:0/735938115 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fee6014faa0 con 0x7fee600b2ab0 2022-06-17T12:53:12.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.274+0000 7fee5cff9700 1 -- 172.21.15.36:0/735938115 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+10 (secure 0 0 0) 0x7fee58045fb0 con 0x7fee600b2ab0 2022-06-17T12:53:12.260 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:53:12.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.278+0000 7fee65e2a700 1 -- 172.21.15.36:0/735938115 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fee40053920 msgr2=0x7fee40055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:12.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.278+0000 7fee65e2a700 1 --2- 172.21.15.36:0/735938115 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fee40053920 0x7fee40055de0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7fee5001a000 tx=0x7fee50014000).stop 2022-06-17T12:53:12.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.278+0000 7fee65e2a700 1 -- 172.21.15.36:0/735938115 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fee600b2ab0 msgr2=0x7fee6013f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:12.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.278+0000 7fee65e2a700 1 --2- 172.21.15.36:0/735938115 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fee600b2ab0 0x7fee6013f5c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7fee58001490 tx=0x7fee58007dc0).stop 2022-06-17T12:53:12.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.278+0000 7fee65e2a700 1 -- 172.21.15.36:0/735938115 shutdown_connections 2022-06-17T12:53:12.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.278+0000 7fee65e2a700 1 --2- 172.21.15.36:0/735938115 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fee40053920 0x7fee40055de0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.278+0000 7fee65e2a700 1 --2- 172.21.15.36:0/735938115 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fee600b2ab0 0x7fee6013f5c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.278+0000 7fee65e2a700 1 --2- 172.21.15.36:0/735938115 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fee600b1d40 0x7fee600643d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.278+0000 7fee65e2a700 1 --2- 172.21.15.36:0/735938115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee600af5a0 0x7fee6014fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.278+0000 7fee65e2a700 1 -- 172.21.15.36:0/735938115 >> 172.21.15.36:0/735938115 conn(0x7fee60063020 msgr2=0x7fee600ac630 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:12.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.278+0000 7fee65e2a700 1 -- 172.21.15.36:0/735938115 shutdown_connections 2022-06-17T12:53:12.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.278+0000 7fee65e2a700 1 -- 172.21.15.36:0/735938115 wait complete. 2022-06-17T12:53:12.279 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:53:12.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.402+0000 7fadc7036700 1 Processor -- start 2022-06-17T12:53:12.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.402+0000 7fadc7036700 1 -- start start 2022-06-17T12:53:12.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc7036700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadc00b5be0 0x7fadc0144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:12.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc7036700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fadc0147270 0x7fadc014ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:12.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc7036700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc00b2c80 con 0x7fadc00601c0 2022-06-17T12:53:12.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc7036700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc00b72d0 con 0x7fadc0147270 2022-06-17T12:53:12.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc7036700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc00b0e50 con 0x7fadc00b5be0 2022-06-17T12:53:12.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc6034700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fadc00601c0 0x7fadc0144270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53310/0 (socket says 172.21.15.36:53310) 2022-06-17T12:53:12.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc6835700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fadc0147270 0x7fadc014ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:12.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc6034700 1 -- 172.21.15.36:0/3063759670 learned_addr learned my addr 172.21.15.36:0/3063759670 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:53:12.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc5833700 1 --2- 172.21.15.36:0/3063759670 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadc00b5be0 0x7fadc0144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:12.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc6835700 1 -- 172.21.15.36:0/3063759670 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadc00b5be0 msgr2=0x7fadc0144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:12.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc6835700 1 --2- 172.21.15.36:0/3063759670 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadc00b5be0 0x7fadc0144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc6835700 1 -- 172.21.15.36:0/3063759670 >> v1:172.21.15.36:6789/0 conn(0x7fadc00601c0 legacy=0x7fadc0144270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:12.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc5032700 1 -- 172.21.15.36:0/3063759670 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 696166413 0 0) 0x7fadc00b2c80 con 0x7fadc00601c0 2022-06-17T12:53:12.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc6835700 1 -- 172.21.15.36:0/3063759670 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc00605a0 con 0x7fadc0147270 2022-06-17T12:53:12.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc6835700 1 --2- 172.21.15.36:0/3063759670 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadc0147270 0x7fadc014ab80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7fadb80032f0 tx=0x7fadb80091d0).ready entity=mon.0 client_cookie=e90f865c5041abca server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:12.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc5032700 1 -- 172.21.15.36:0/3063759670 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadb800b6c0 con 0x7fadc0147270 2022-06-17T12:53:12.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc5032700 1 -- 172.21.15.36:0/3063759670 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb800b820 con 0x7fadc0147270 2022-06-17T12:53:12.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc5032700 1 -- 172.21.15.36:0/3063759670 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadb800b990 con 0x7fadc0147270 2022-06-17T12:53:12.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc7036700 1 -- 172.21.15.36:0/3063759670 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadc0147270 msgr2=0x7fadc014ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:12.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc7036700 1 --2- 172.21.15.36:0/3063759670 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadc0147270 0x7fadc014ab80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7fadb80032f0 tx=0x7fadb80091d0).stop 2022-06-17T12:53:12.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc7036700 1 -- 172.21.15.36:0/3063759670 shutdown_connections 2022-06-17T12:53:12.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc7036700 1 --2- 172.21.15.36:0/3063759670 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadc0147270 0x7fadc014ab80 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc7036700 1 --2- 172.21.15.36:0/3063759670 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadc00b5be0 0x7fadc0144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc7036700 1 -- 172.21.15.36:0/3063759670 >> 172.21.15.36:0/3063759670 conn(0x7fadc005c550 msgr2=0x7fadc005f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:12.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc7036700 1 -- 172.21.15.36:0/3063759670 shutdown_connections 2022-06-17T12:53:12.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.406+0000 7fadc7036700 1 -- 172.21.15.36:0/3063759670 wait complete. 2022-06-17T12:53:12.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.410+0000 7fadc7036700 1 Processor -- start 2022-06-17T12:53:12.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.410+0000 7fadc7036700 1 -- start start 2022-06-17T12:53:12.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.410+0000 7fadc7036700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadc00601c0 0x7fadc0159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:12.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.410+0000 7fadc7036700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadc00b5be0 0x7fadc00afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:12.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.410+0000 7fadc7036700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadc0147270 0x7fadc014b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:12.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.410+0000 7fadc7036700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc015a830 con 0x7fadc00b5be0 2022-06-17T12:53:12.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.410+0000 7fadc7036700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc015a2f0 con 0x7fadc00601c0 2022-06-17T12:53:12.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.410+0000 7fadc7036700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc015a590 con 0x7fadc0147270 2022-06-17T12:53:12.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.414+0000 7fadc5833700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadc00b5be0 0x7fadc00afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:12.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.414+0000 7fadc5833700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadc00b5be0 0x7fadc00afb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35788/0 (socket says 172.21.15.36:35788) 2022-06-17T12:53:12.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.414+0000 7fadc5833700 1 -- 172.21.15.36:0/3197202700 learned_addr learned my addr 172.21.15.36:0/3197202700 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:12.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.414+0000 7fadc6835700 1 --2- 172.21.15.36:0/3197202700 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadc0147270 0x7fadc014b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:12.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.414+0000 7fadc6034700 1 --2- 172.21.15.36:0/3197202700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadc00601c0 0x7fadc0159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:12.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.414+0000 7fadc5833700 1 -- 172.21.15.36:0/3197202700 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadc0147270 msgr2=0x7fadc014b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:12.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.414+0000 7fadc5833700 1 --2- 172.21.15.36:0/3197202700 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadc0147270 0x7fadc014b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.414+0000 7fadc5833700 1 -- 172.21.15.36:0/3197202700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadc00601c0 msgr2=0x7fadc0159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:12.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.414+0000 7fadc5833700 1 --2- 172.21.15.36:0/3197202700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadc00601c0 0x7fadc0159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.414+0000 7fadc5833700 1 -- 172.21.15.36:0/3197202700 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb8009470 con 0x7fadc00b5be0 2022-06-17T12:53:12.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.414+0000 7fadc5833700 1 --2- 172.21.15.36:0/3197202700 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadc00b5be0 0x7fadc00afb40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7fadb000c000 tx=0x7fadb0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:12.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.414+0000 7fadaf7fe700 1 -- 172.21.15.36:0/3197202700 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadb0010780 con 0x7fadc00b5be0 2022-06-17T12:53:12.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.414+0000 7fadaf7fe700 1 -- 172.21.15.36:0/3197202700 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb00108e0 con 0x7fadc00b5be0 2022-06-17T12:53:12.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.414+0000 7fadc7036700 1 -- 172.21.15.36:0/3197202700 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc00b0030 con 0x7fadc00b5be0 2022-06-17T12:53:12.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.414+0000 7fadaf7fe700 1 -- 172.21.15.36:0/3197202700 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadb0010c40 con 0x7fadc00b5be0 2022-06-17T12:53:12.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.414+0000 7fadc7036700 1 -- 172.21.15.36:0/3197202700 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc00ae310 con 0x7fadc00b5be0 2022-06-17T12:53:12.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.414+0000 7fadc7036700 1 -- 172.21.15.36:0/3197202700 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc00b29e0 con 0x7fadc00b5be0 2022-06-17T12:53:12.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.414+0000 7fadaf7fe700 1 -- 172.21.15.36:0/3197202700 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fadb001c460 con 0x7fadc00b5be0 2022-06-17T12:53:12.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.418+0000 7fadaf7fe700 1 --2- 172.21.15.36:0/3197202700 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fada0053920 0x7fada0055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:12.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.418+0000 7fadaf7fe700 1 -- 172.21.15.36:0/3197202700 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fadb006c520 con 0x7fadc00b5be0 2022-06-17T12:53:12.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.418+0000 7fadc6034700 1 --2- 172.21.15.36:0/3197202700 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fada0053920 0x7fada0055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:12.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.418+0000 7fadc6034700 1 --2- 172.21.15.36:0/3197202700 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fada0053920 0x7fada0055de0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7fadbc00d000 tx=0x7fadbc007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:12.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.422+0000 7fadaf7fe700 1 -- 172.21.15.36:0/3197202700 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fadb0043130 con 0x7fadc00b5be0 2022-06-17T12:53:12.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.550+0000 7fadc7036700 1 -- 172.21.15.36:0/3197202700 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fadc0057410 con 0x7fadc00b5be0 2022-06-17T12:53:12.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.554+0000 7fadaf7fe700 1 -- 172.21.15.36:0/3197202700 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+10 (secure 0 0 0) 0x7fadb0046750 con 0x7fadc00b5be0 2022-06-17T12:53:12.545 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:53:12.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.558+0000 7fadc7036700 1 -- 172.21.15.36:0/3197202700 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fada0053920 msgr2=0x7fada0055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:12.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.558+0000 7fadc7036700 1 --2- 172.21.15.36:0/3197202700 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fada0053920 0x7fada0055de0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7fadbc00d000 tx=0x7fadbc007a00).stop 2022-06-17T12:53:12.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.558+0000 7fadc7036700 1 -- 172.21.15.36:0/3197202700 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadc00b5be0 msgr2=0x7fadc00afb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:12.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.558+0000 7fadc7036700 1 --2- 172.21.15.36:0/3197202700 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadc00b5be0 0x7fadc00afb40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7fadb000c000 tx=0x7fadb0006df0).stop 2022-06-17T12:53:12.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.558+0000 7fadc7036700 1 -- 172.21.15.36:0/3197202700 shutdown_connections 2022-06-17T12:53:12.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.558+0000 7fadc7036700 1 --2- 172.21.15.36:0/3197202700 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fada0053920 0x7fada0055de0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.558+0000 7fadc7036700 1 --2- 172.21.15.36:0/3197202700 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadc0147270 0x7fadc014b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.558+0000 7fadc7036700 1 --2- 172.21.15.36:0/3197202700 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadc00b5be0 0x7fadc00afb40 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.558+0000 7fadc7036700 1 --2- 172.21.15.36:0/3197202700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadc00601c0 0x7fadc0159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.558+0000 7fadc7036700 1 -- 172.21.15.36:0/3197202700 >> 172.21.15.36:0/3197202700 conn(0x7fadc005c550 msgr2=0x7fadc0146930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:12.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.558+0000 7fadc7036700 1 -- 172.21.15.36:0/3197202700 shutdown_connections 2022-06-17T12:53:12.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.558+0000 7fadc7036700 1 -- 172.21.15.36:0/3197202700 wait complete. 2022-06-17T12:53:12.556 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:53:12.557 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:53:12.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.674+0000 7fa464832700 1 Processor -- start 2022-06-17T12:53:12.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.674+0000 7fa464832700 1 -- start start 2022-06-17T12:53:12.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.674+0000 7fa464832700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa4600b2cd0 0x7fa4601465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:12.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.674+0000 7fa464832700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4600b25e0 con 0x7fa460146ae0 2022-06-17T12:53:12.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.674+0000 7fa464832700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4600ad790 con 0x7fa4600b2200 2022-06-17T12:53:12.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.674+0000 7fa464832700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4600aef40 con 0x7fa4600b2cd0 2022-06-17T12:53:12.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.678+0000 7fa45effd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa4600b2cd0 0x7fa4601465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:12.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.678+0000 7fa45ffff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fa460146ae0 0x7fa460146f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53326/0 (socket says 172.21.15.36:53326) 2022-06-17T12:53:12.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.678+0000 7fa45ffff700 1 -- 172.21.15.36:0/3937030035 learned_addr learned my addr 172.21.15.36:0/3937030035 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:53:12.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.678+0000 7fa45effd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa4600b2cd0 0x7fa4601465f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35794/0 (socket says 172.21.15.36:35794) 2022-06-17T12:53:12.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.678+0000 7fa45e7fc700 1 -- 172.21.15.36:0/3937030035 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2643511265 0 0) 0x7fa4600b25e0 con 0x7fa460146ae0 2022-06-17T12:53:12.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.678+0000 7fa45effd700 1 -- 172.21.15.36:0/3937030035 >> v1:172.21.15.70:6789/0 conn(0x7fa4600b2200 legacy=0x7fa4600af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:12.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.678+0000 7fa45effd700 1 -- 172.21.15.36:0/3937030035 >> v1:172.21.15.36:6789/0 conn(0x7fa460146ae0 legacy=0x7fa460146f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:12.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.678+0000 7fa45effd700 1 -- 172.21.15.36:0/3937030035 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4600afaf0 con 0x7fa4600b2cd0 2022-06-17T12:53:12.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.678+0000 7fa45effd700 1 --2- 172.21.15.36:0/3937030035 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4600b2cd0 0x7fa4601465f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7fa450002f70 tx=0x7fa450006df0).ready entity=mon.0 client_cookie=8e22d204e78ba1c9 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:12.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.678+0000 7fa45e7fc700 1 -- 172.21.15.36:0/3937030035 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa45000d610 con 0x7fa4600b2cd0 2022-06-17T12:53:12.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.678+0000 7fa45e7fc700 1 -- 172.21.15.36:0/3937030035 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa450000e90 con 0x7fa4600b2cd0 2022-06-17T12:53:12.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.678+0000 7fa45e7fc700 1 -- 172.21.15.36:0/3937030035 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa45000d960 con 0x7fa4600b2cd0 2022-06-17T12:53:12.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.678+0000 7fa464832700 1 -- 172.21.15.36:0/3937030035 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4600b2cd0 msgr2=0x7fa4601465f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:12.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.678+0000 7fa464832700 1 --2- 172.21.15.36:0/3937030035 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4600b2cd0 0x7fa4601465f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7fa450002f70 tx=0x7fa450006df0).stop 2022-06-17T12:53:12.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.678+0000 7fa464832700 1 -- 172.21.15.36:0/3937030035 shutdown_connections 2022-06-17T12:53:12.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.678+0000 7fa464832700 1 --2- 172.21.15.36:0/3937030035 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4600b2cd0 0x7fa4601465f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.678+0000 7fa464832700 1 -- 172.21.15.36:0/3937030035 >> 172.21.15.36:0/3937030035 conn(0x7fa46005f770 msgr2=0x7fa46005fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:12.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.678+0000 7fa464832700 1 -- 172.21.15.36:0/3937030035 shutdown_connections 2022-06-17T12:53:12.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.678+0000 7fa464832700 1 -- 172.21.15.36:0/3937030035 wait complete. 2022-06-17T12:53:12.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.678+0000 7fa464832700 1 Processor -- start 2022-06-17T12:53:12.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.678+0000 7fa464832700 1 -- start start 2022-06-17T12:53:12.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.682+0000 7fa464832700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4600b2200 0x7fa4600b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:12.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.682+0000 7fa464832700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4600b2cd0 0x7fa4600633c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:12.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.682+0000 7fa464832700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa460146ae0 0x7fa460066c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:12.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.682+0000 7fa464832700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4600a9950 con 0x7fa460146ae0 2022-06-17T12:53:12.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.682+0000 7fa464832700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4600a9410 con 0x7fa4600b2200 2022-06-17T12:53:12.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.682+0000 7fa464832700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4600a96b0 con 0x7fa4600b2cd0 2022-06-17T12:53:12.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.682+0000 7fa45ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa460146ae0 0x7fa460066c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:12.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.682+0000 7fa45ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa460146ae0 0x7fa460066c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35800/0 (socket says 172.21.15.36:35800) 2022-06-17T12:53:12.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.682+0000 7fa45ffff700 1 -- 172.21.15.36:0/3765866219 learned_addr learned my addr 172.21.15.36:0/3765866219 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:12.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.682+0000 7fa45f7fe700 1 --2- 172.21.15.36:0/3765866219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4600b2200 0x7fa4600b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:12.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.682+0000 7fa45effd700 1 --2- 172.21.15.36:0/3765866219 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4600b2cd0 0x7fa4600633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:12.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.682+0000 7fa45ffff700 1 -- 172.21.15.36:0/3765866219 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4600b2cd0 msgr2=0x7fa4600633c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:12.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.682+0000 7fa45ffff700 1 --2- 172.21.15.36:0/3765866219 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4600b2cd0 0x7fa4600633c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.682+0000 7fa45ffff700 1 -- 172.21.15.36:0/3765866219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4600b2200 msgr2=0x7fa4600b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:12.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.682+0000 7fa45ffff700 1 --2- 172.21.15.36:0/3765866219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4600b2200 0x7fa4600b06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.682+0000 7fa45ffff700 1 -- 172.21.15.36:0/3765866219 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa45000b180 con 0x7fa460146ae0 2022-06-17T12:53:12.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.682+0000 7fa45ffff700 1 --2- 172.21.15.36:0/3765866219 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa460146ae0 0x7fa460066c20 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7fa458000f40 tx=0x7fa458001360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:12.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.682+0000 7fa45cff9700 1 -- 172.21.15.36:0/3765866219 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa458005ba0 con 0x7fa460146ae0 2022-06-17T12:53:12.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.682+0000 7fa45cff9700 1 -- 172.21.15.36:0/3765866219 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa458005d00 con 0x7fa460146ae0 2022-06-17T12:53:12.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.682+0000 7fa464832700 1 -- 172.21.15.36:0/3765866219 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4600a8ed0 con 0x7fa460146ae0 2022-06-17T12:53:12.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.682+0000 7fa45cff9700 1 -- 172.21.15.36:0/3765866219 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4580072c0 con 0x7fa460146ae0 2022-06-17T12:53:12.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.682+0000 7fa464832700 1 -- 172.21.15.36:0/3765866219 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa460159f40 con 0x7fa460146ae0 2022-06-17T12:53:12.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.682+0000 7fa464832700 1 -- 172.21.15.36:0/3765866219 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4600b1950 con 0x7fa460146ae0 2022-06-17T12:53:12.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.686+0000 7fa45cff9700 1 -- 172.21.15.36:0/3765866219 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fa45801a060 con 0x7fa460146ae0 2022-06-17T12:53:12.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.686+0000 7fa45cff9700 1 --2- 172.21.15.36:0/3765866219 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa440053a40 0x7fa440055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:12.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.686+0000 7fa45cff9700 1 -- 172.21.15.36:0/3765866219 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa45806b520 con 0x7fa460146ae0 2022-06-17T12:53:12.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.686+0000 7fa45f7fe700 1 --2- 172.21.15.36:0/3765866219 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa440053a40 0x7fa440055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:12.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.686+0000 7fa45f7fe700 1 --2- 172.21.15.36:0/3765866219 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa440053a40 0x7fa440055f00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7fa45400d000 tx=0x7fa454007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:12.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.690+0000 7fa45cff9700 1 -- 172.21.15.36:0/3765866219 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa458042320 con 0x7fa460146ae0 2022-06-17T12:53:12.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.862+0000 7fa464832700 1 -- 172.21.15.36:0/3765866219 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa460057410 con 0x7fa460146ae0 2022-06-17T12:53:12.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.862+0000 7fa45cff9700 1 -- 172.21.15.36:0/3765866219 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+2140 (secure 0 0 0) 0x7fa458018020 con 0x7fa460146ae0 2022-06-17T12:53:12.853 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:53:12.854 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":36,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/2541998896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2541998896},{"type":"v1","addr":"172.21.15.70:6833","nonce":2541998896}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:53:04.907795+0000","modified":"2022-06-17T12:53:06.511529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6381":{"gid":6381,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/2514005995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2514005995},{"type":"v1","addr":"172.21.15.36:6835","nonce":2514005995}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-06-17T12:53:12.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.866+0000 7fa464832700 1 -- 172.21.15.36:0/3765866219 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa440053a40 msgr2=0x7fa440055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:12.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.866+0000 7fa464832700 1 --2- 172.21.15.36:0/3765866219 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa440053a40 0x7fa440055f00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7fa45400d000 tx=0x7fa454007b90).stop 2022-06-17T12:53:12.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.866+0000 7fa464832700 1 -- 172.21.15.36:0/3765866219 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa460146ae0 msgr2=0x7fa460066c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:12.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.866+0000 7fa464832700 1 --2- 172.21.15.36:0/3765866219 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa460146ae0 0x7fa460066c20 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7fa458000f40 tx=0x7fa458001360).stop 2022-06-17T12:53:12.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.866+0000 7fa464832700 1 -- 172.21.15.36:0/3765866219 shutdown_connections 2022-06-17T12:53:12.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.866+0000 7fa464832700 1 --2- 172.21.15.36:0/3765866219 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa440053a40 0x7fa440055f00 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.866+0000 7fa464832700 1 --2- 172.21.15.36:0/3765866219 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa460146ae0 0x7fa460066c20 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.866+0000 7fa464832700 1 --2- 172.21.15.36:0/3765866219 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4600b2cd0 0x7fa4600633c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.866+0000 7fa464832700 1 --2- 172.21.15.36:0/3765866219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4600b2200 0x7fa4600b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:12.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.866+0000 7fa464832700 1 -- 172.21.15.36:0/3765866219 >> 172.21.15.36:0/3765866219 conn(0x7fa46005f770 msgr2=0x7fa46014cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:12.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.866+0000 7fa464832700 1 -- 172.21.15.36:0/3765866219 shutdown_connections 2022-06-17T12:53:12.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:12.866+0000 7fa464832700 1 -- 172.21.15.36:0/3765866219 wait complete. 2022-06-17T12:53:12.859 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 36 2022-06-17T12:53:13.068 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:53:04.907795+0000', 'modified': '2022-06-17T12:53:06.511529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6381}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6381': {'gid': 6381, 'name': 'a', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.36:6835/2514005995', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 2514005995}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 2514005995}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:53:13.069 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:53:13.070 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:53:13.191 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":6381,"want_state":"up:active","state":"up:active","rank_uptime":8.2869593669999997,"mdsmap_epoch":36,"osdmap_epoch":74,"osdmap_epoch_barrier":74,"uptime":12.681834079} 2022-06-17T12:53:13.192 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 6381, "mdsmap_epoch": 36, "osdmap_epoch": 74, "osdmap_epoch_barrier": 74, "rank_uptime": 8.286959367, "state": "up:active", "uptime": 12.681834079, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:53:13.193 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:53:13.193 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-06-17T12:53:13.194 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi136.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-06-17T12:53:13.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:13.195 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:53:13.205 DEBUG:teuthology.orchestra.run.smithi136:> sudo modprobe fuse 2022-06-17T12:53:13.222 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:53:13.223 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:53:13.277 INFO:teuthology.orchestra.run.smithi136.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:53:13.278 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:53:13.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:13.280 DEBUG:teuthology.orchestra.run.smithi136:> ls /sys/fs/fuse/connections 2022-06-17T12:53:13.287 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:53:13.287 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:53:13.333 DEBUG:teuthology.orchestra.run.smithi136:> sudo modprobe fuse 2022-06-17T12:53:13.349 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:53:13.349 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:53:13.364 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:53:13.365 INFO:teuthology.orchestra.run.smithi136.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:53:13.366 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:13.366 DEBUG:teuthology.orchestra.run.smithi136:> ls /sys/fs/fuse/connections 2022-06-17T12:53:13.407 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:2022-06-17T12:53:13.424+0000 7fb3db725080 -1 init, newargv = 0x5566be6d4af0 newargc=9ceph-fuse[21883]: starting ceph client 2022-06-17T12:53:13.408 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr: 2022-06-17T12:53:13.418 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:ceph-fuse[21883]: starting fuse 2022-06-17T12:53:14.485 DEBUG:teuthology.orchestra.run.smithi136:> sudo modprobe fuse 2022-06-17T12:53:14.505 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:53:14.506 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:53:14.563 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:53:14.565 INFO:teuthology.orchestra.run.smithi136.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:53:14.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:14.566 DEBUG:teuthology.orchestra.run.smithi136:> ls /sys/fs/fuse/connections 2022-06-17T12:53:14.573 INFO:teuthology.orchestra.run.smithi136.stdout:52 2022-06-17T12:53:14.573 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:53:14.574 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:14.575 DEBUG:teuthology.orchestra.run.smithi136:> sudo python3 -c ' 2022-06-17T12:53:14.575 DEBUG:teuthology.orchestra.run.smithi136:> import glob 2022-06-17T12:53:14.576 DEBUG:teuthology.orchestra.run.smithi136:> import re 2022-06-17T12:53:14.576 DEBUG:teuthology.orchestra.run.smithi136:> import os 2022-06-17T12:53:14.577 DEBUG:teuthology.orchestra.run.smithi136:> import subprocess 2022-06-17T12:53:14.577 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:53:14.578 DEBUG:teuthology.orchestra.run.smithi136:> def find_socket(client_name): 2022-06-17T12:53:14.578 DEBUG:teuthology.orchestra.run.smithi136:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-06-17T12:53:14.579 DEBUG:teuthology.orchestra.run.smithi136:> files = glob.glob(asok_path) 2022-06-17T12:53:14.579 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:53:14.580 DEBUG:teuthology.orchestra.run.smithi136:> # Given a non-glob path, it better be there 2022-06-17T12:53:14.580 DEBUG:teuthology.orchestra.run.smithi136:> if "*" not in asok_path: 2022-06-17T12:53:14.580 DEBUG:teuthology.orchestra.run.smithi136:> assert(len(files) == 1) 2022-06-17T12:53:14.581 DEBUG:teuthology.orchestra.run.smithi136:> return files[0] 2022-06-17T12:53:14.582 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:53:14.582 DEBUG:teuthology.orchestra.run.smithi136:> for f in files: 2022-06-17T12:53:14.583 DEBUG:teuthology.orchestra.run.smithi136:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:53:14.583 DEBUG:teuthology.orchestra.run.smithi136:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:53:14.584 DEBUG:teuthology.orchestra.run.smithi136:> return f 2022-06-17T12:53:14.584 DEBUG:teuthology.orchestra.run.smithi136:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:53:14.588 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:53:14.589 DEBUG:teuthology.orchestra.run.smithi136:> print(find_socket("client.1")) 2022-06-17T12:53:14.589 DEBUG:teuthology.orchestra.run.smithi136:> ' 2022-06-17T12:53:14.642 INFO:teuthology.orchestra.run.smithi136.stdout:/var/run/ceph/ceph-client.1.21883.asok 2022-06-17T12:53:14.647 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.21883.asok 2022-06-17T12:53:14.649 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:14.650 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.21883.asok status 2022-06-17T12:53:14.786 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2022-06-17T12:53:14.787 INFO:teuthology.orchestra.run.smithi136.stdout: "metadata": { 2022-06-17T12:53:14.787 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:53:14.788 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:53:14.788 INFO:teuthology.orchestra.run.smithi136.stdout: "entity_id": "1", 2022-06-17T12:53:14.789 INFO:teuthology.orchestra.run.smithi136.stdout: "hostname": "smithi136", 2022-06-17T12:53:14.789 INFO:teuthology.orchestra.run.smithi136.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-06-17T12:53:14.791 INFO:teuthology.orchestra.run.smithi136.stdout: "pid": "21883", 2022-06-17T12:53:14.791 INFO:teuthology.orchestra.run.smithi136.stdout: "root": "/" 2022-06-17T12:53:14.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:53:14.792 INFO:teuthology.orchestra.run.smithi136.stdout: "dentry_count": 0, 2022-06-17T12:53:14.793 INFO:teuthology.orchestra.run.smithi136.stdout: "dentry_pinned_count": 0, 2022-06-17T12:53:14.793 INFO:teuthology.orchestra.run.smithi136.stdout: "id": 6579, 2022-06-17T12:53:14.793 INFO:teuthology.orchestra.run.smithi136.stdout: "inst": { 2022-06-17T12:53:14.794 INFO:teuthology.orchestra.run.smithi136.stdout: "name": { 2022-06-17T12:53:14.794 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "client", 2022-06-17T12:53:14.795 INFO:teuthology.orchestra.run.smithi136.stdout: "num": 6579 2022-06-17T12:53:14.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:53:14.796 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": { 2022-06-17T12:53:14.796 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "v1", 2022-06-17T12:53:14.796 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": "172.21.15.136:0", 2022-06-17T12:53:14.796 INFO:teuthology.orchestra.run.smithi136.stdout: "nonce": 2846092803 2022-06-17T12:53:14.797 INFO:teuthology.orchestra.run.smithi136.stdout: } 2022-06-17T12:53:14.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:53:14.797 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": { 2022-06-17T12:53:14.798 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "v1", 2022-06-17T12:53:14.798 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": "172.21.15.136:0", 2022-06-17T12:53:14.799 INFO:teuthology.orchestra.run.smithi136.stdout: "nonce": 2846092803 2022-06-17T12:53:14.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:53:14.800 INFO:teuthology.orchestra.run.smithi136.stdout: "inst_str": "client.6579 172.21.15.136:0/2846092803", 2022-06-17T12:53:14.800 INFO:teuthology.orchestra.run.smithi136.stdout: "addr_str": "172.21.15.136:0/2846092803", 2022-06-17T12:53:14.801 INFO:teuthology.orchestra.run.smithi136.stdout: "inode_count": 1, 2022-06-17T12:53:14.801 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_epoch": 36, 2022-06-17T12:53:14.802 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_epoch": 74, 2022-06-17T12:53:14.802 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:53:14.803 INFO:teuthology.orchestra.run.smithi136.stdout: "blacklisted": false, 2022-06-17T12:53:14.803 INFO:teuthology.orchestra.run.smithi136.stdout: "fs_name": "cephfs" 2022-06-17T12:53:14.804 INFO:teuthology.orchestra.run.smithi136.stdout:} 2022-06-17T12:53:14.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:14.807 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:53:14.808 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:53:14.820 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:53:14.820 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:53:14.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:14.821 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:53:14.889 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:53:15.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.018+0000 7f0aac2bf700 1 Processor -- start 2022-06-17T12:53:15.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.022+0000 7f0aac2bf700 1 -- start start 2022-06-17T12:53:15.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.022+0000 7f0aac2bf700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0aa4062b50 0x7f0aa40ada40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:15.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.022+0000 7f0aac2bf700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa40b0ba0 con 0x7f0aa4144c00 2022-06-17T12:53:15.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.022+0000 7f0aac2bf700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa4147e90 con 0x7f0aa4062770 2022-06-17T12:53:15.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.022+0000 7f0aac2bf700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aa40ae820 con 0x7f0aa4062b50 2022-06-17T12:53:15.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.022+0000 7f0aa985a700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f0aa4144c00 0x7f0aa40ad330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53336/0 (socket says 172.21.15.36:53336) 2022-06-17T12:53:15.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.022+0000 7f0aa985a700 1 -- 172.21.15.36:0/2496744929 learned_addr learned my addr 172.21.15.36:0/2496744929 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:53:15.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.022+0000 7f0aaa85c700 1 --2- 172.21.15.36:0/2496744929 >> v2:172.21.15.70:3300/0 conn(0x7f0aa4062b50 0x7f0aa40ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:15.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.022+0000 7f0aa9059700 1 -- 172.21.15.36:0/2496744929 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3051761958 0 0) 0x7f0aa40b0ba0 con 0x7f0aa4144c00 2022-06-17T12:53:15.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.022+0000 7f0aa9059700 1 -- 172.21.15.36:0/2496744929 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a8c003260 con 0x7f0aa4144c00 2022-06-17T12:53:15.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.022+0000 7f0aaa85c700 1 -- 172.21.15.36:0/2496744929 >> v1:172.21.15.70:6789/0 conn(0x7f0aa4062770 legacy=0x7f0aa41444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:15.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.022+0000 7f0aaa85c700 1 -- 172.21.15.36:0/2496744929 >> v1:172.21.15.36:6789/0 conn(0x7f0aa4144c00 legacy=0x7f0aa40ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:15.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.022+0000 7f0aaa85c700 1 -- 172.21.15.36:0/2496744929 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa40b0dd0 con 0x7f0aa4062b50 2022-06-17T12:53:15.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.022+0000 7f0aaa85c700 1 --2- 172.21.15.36:0/2496744929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aa4062b50 0x7f0aa40ada40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f0a9c0032f0 tx=0x7f0a9c0091d0).ready entity=mon.1 client_cookie=e731f11762d234ee server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:15.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.022+0000 7f0aa9059700 1 -- 172.21.15.36:0/2496744929 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a9c00b6c0 con 0x7f0aa4062b50 2022-06-17T12:53:15.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.022+0000 7f0aa9059700 1 -- 172.21.15.36:0/2496744929 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9c00b820 con 0x7f0aa4062b50 2022-06-17T12:53:15.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.022+0000 7f0aa9059700 1 -- 172.21.15.36:0/2496744929 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a9c00b990 con 0x7f0aa4062b50 2022-06-17T12:53:15.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.022+0000 7f0aac2bf700 1 -- 172.21.15.36:0/2496744929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aa4062b50 msgr2=0x7f0aa40ada40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:15.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.022+0000 7f0aac2bf700 1 --2- 172.21.15.36:0/2496744929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aa4062b50 0x7f0aa40ada40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f0a9c0032f0 tx=0x7f0a9c0091d0).stop 2022-06-17T12:53:15.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.022+0000 7f0aac2bf700 1 -- 172.21.15.36:0/2496744929 shutdown_connections 2022-06-17T12:53:15.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.022+0000 7f0aac2bf700 1 --2- 172.21.15.36:0/2496744929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aa4062b50 0x7f0aa40ada40 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:15.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.022+0000 7f0aac2bf700 1 -- 172.21.15.36:0/2496744929 >> 172.21.15.36:0/2496744929 conn(0x7f0aa40b53e0 msgr2=0x7f0aa40b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:15.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.022+0000 7f0aac2bf700 1 -- 172.21.15.36:0/2496744929 shutdown_connections 2022-06-17T12:53:15.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.022+0000 7f0aac2bf700 1 -- 172.21.15.36:0/2496744929 wait complete. 2022-06-17T12:53:15.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.026+0000 7f0aac2bf700 1 Processor -- start 2022-06-17T12:53:15.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.026+0000 7f0aac2bf700 1 -- start start 2022-06-17T12:53:15.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.026+0000 7f0aac2bf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aa4062770 0x7f0aa41574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:15.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.026+0000 7f0aac2bf700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0aa4062b50 0x7f0aa4066580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:15.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.026+0000 7f0aac2bf700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0aa4144c00 0x7f0aa4148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:15.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.026+0000 7f0aac2bf700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa414b940 con 0x7f0aa4062b50 2022-06-17T12:53:15.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.026+0000 7f0aac2bf700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa4066bd0 con 0x7f0aa4062770 2022-06-17T12:53:15.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.026+0000 7f0aac2bf700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa4066e70 con 0x7f0aa4144c00 2022-06-17T12:53:15.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.026+0000 7f0aa985a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0aa4062b50 0x7f0aa4066580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:15.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.026+0000 7f0aaa05b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aa4062770 0x7f0aa41574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:15.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.026+0000 7f0aaa05b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aa4062770 0x7f0aa41574f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:56194/0 (socket says 172.21.15.36:56194) 2022-06-17T12:53:15.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.026+0000 7f0aaa05b700 1 -- 172.21.15.36:0/2634153067 learned_addr learned my addr 172.21.15.36:0/2634153067 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:15.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.026+0000 7f0aaa85c700 1 --2- 172.21.15.36:0/2634153067 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0aa4144c00 0x7f0aa4148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:15.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.026+0000 7f0aaa05b700 1 -- 172.21.15.36:0/2634153067 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0aa4144c00 msgr2=0x7f0aa4148910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:15.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.026+0000 7f0aaa05b700 1 --2- 172.21.15.36:0/2634153067 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0aa4144c00 0x7f0aa4148910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:15.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.026+0000 7f0aaa05b700 1 -- 172.21.15.36:0/2634153067 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0aa4062b50 msgr2=0x7f0aa4066580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:15.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.026+0000 7f0aaa05b700 1 --2- 172.21.15.36:0/2634153067 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0aa4062b50 0x7f0aa4066580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:15.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.026+0000 7f0aaa05b700 1 -- 172.21.15.36:0/2634153067 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c009470 con 0x7f0aa4062770 2022-06-17T12:53:15.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.026+0000 7f0aaa05b700 1 --2- 172.21.15.36:0/2634153067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aa4062770 0x7f0aa41574f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f0aa405d2b0 tx=0x7f0aa0007c50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:15.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.026+0000 7f0a937fe700 1 -- 172.21.15.36:0/2634153067 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0aa0012400 con 0x7f0aa4062770 2022-06-17T12:53:15.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.026+0000 7f0a937fe700 1 -- 172.21.15.36:0/2634153067 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa0012560 con 0x7f0aa4062770 2022-06-17T12:53:15.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.026+0000 7f0a937fe700 1 -- 172.21.15.36:0/2634153067 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0aa00128c0 con 0x7f0aa4062770 2022-06-17T12:53:15.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.026+0000 7f0aac2bf700 1 -- 172.21.15.36:0/2634153067 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa4067250 con 0x7f0aa4062770 2022-06-17T12:53:15.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.026+0000 7f0aac2bf700 1 -- 172.21.15.36:0/2634153067 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa41444d0 con 0x7f0aa4062770 2022-06-17T12:53:15.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.030+0000 7f0aac2bf700 1 -- 172.21.15.36:0/2634153067 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa4067070 con 0x7f0aa4062770 2022-06-17T12:53:15.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.030+0000 7f0a937fe700 1 -- 172.21.15.36:0/2634153067 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f0aa0019030 con 0x7f0aa4062770 2022-06-17T12:53:15.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.030+0000 7f0a937fe700 1 --2- 172.21.15.36:0/2634153067 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0a84053a40 0x7f0a84055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:15.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.030+0000 7f0a937fe700 1 -- 172.21.15.36:0/2634153067 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0aa006b760 con 0x7f0aa4062770 2022-06-17T12:53:15.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.030+0000 7f0aa985a700 1 --2- 172.21.15.36:0/2634153067 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0a84053a40 0x7f0a84055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:15.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.030+0000 7f0aa985a700 1 --2- 172.21.15.36:0/2634153067 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0a84053a40 0x7f0a84055f00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f0a9400d000 tx=0x7f0a94009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:15.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.038+0000 7f0a937fe700 1 -- 172.21.15.36:0/2634153067 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0aa0042500 con 0x7f0aa4062770 2022-06-17T12:53:15.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.206+0000 7f0aac2bf700 1 -- 172.21.15.36:0/2634153067 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0aa40b1c20 con 0x7f0aa4062770 2022-06-17T12:53:15.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.206+0000 7f0a937fe700 1 -- 172.21.15.36:0/2634153067 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f0aa0045b20 con 0x7f0aa4062770 2022-06-17T12:53:15.192 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:53:15.192 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":36,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/2541998896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2541998896},{"type":"v1","addr":"172.21.15.70:6833","nonce":2541998896}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:53:04.907795+0000","modified":"2022-06-17T12:53:06.511529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6381":{"gid":6381,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/2514005995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2514005995},{"type":"v1","addr":"172.21.15.36:6835","nonce":2514005995}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-06-17T12:53:15.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.206+0000 7f0aac2bf700 1 -- 172.21.15.36:0/2634153067 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0a84053a40 msgr2=0x7f0a84055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:15.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.206+0000 7f0aac2bf700 1 --2- 172.21.15.36:0/2634153067 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0a84053a40 0x7f0a84055f00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f0a9400d000 tx=0x7f0a94009040).stop 2022-06-17T12:53:15.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.210+0000 7f0aac2bf700 1 -- 172.21.15.36:0/2634153067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aa4062770 msgr2=0x7f0aa41574f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:15.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.210+0000 7f0aac2bf700 1 --2- 172.21.15.36:0/2634153067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aa4062770 0x7f0aa41574f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f0aa405d2b0 tx=0x7f0aa0007c50).stop 2022-06-17T12:53:15.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.210+0000 7f0aac2bf700 1 -- 172.21.15.36:0/2634153067 shutdown_connections 2022-06-17T12:53:15.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.210+0000 7f0aac2bf700 1 --2- 172.21.15.36:0/2634153067 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0a84053a40 0x7f0a84055f00 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:15.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.210+0000 7f0aac2bf700 1 --2- 172.21.15.36:0/2634153067 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0aa4144c00 0x7f0aa4148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:15.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.210+0000 7f0aac2bf700 1 --2- 172.21.15.36:0/2634153067 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0aa4062b50 0x7f0aa4066580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:15.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.210+0000 7f0aac2bf700 1 --2- 172.21.15.36:0/2634153067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aa4062770 0x7f0aa41574f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:15.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.210+0000 7f0aac2bf700 1 -- 172.21.15.36:0/2634153067 >> 172.21.15.36:0/2634153067 conn(0x7f0aa40b53e0 msgr2=0x7f0aa40b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:15.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.210+0000 7f0aac2bf700 1 -- 172.21.15.36:0/2634153067 shutdown_connections 2022-06-17T12:53:15.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.210+0000 7f0aac2bf700 1 -- 172.21.15.36:0/2634153067 wait complete. 2022-06-17T12:53:15.203 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 36 2022-06-17T12:53:15.205 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:53:15.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.314+0000 7ff490269700 1 Processor -- start 2022-06-17T12:53:15.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.314+0000 7ff490269700 1 -- start start 2022-06-17T12:53:15.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.314+0000 7ff490269700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff488062b50 0x7ff4880ada40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:15.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.314+0000 7ff490269700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4880b0ba0 con 0x7ff488062770 2022-06-17T12:53:15.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.314+0000 7ff490269700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff488147e90 con 0x7ff488144c00 2022-06-17T12:53:15.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.314+0000 7ff490269700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4880ae820 con 0x7ff488062b50 2022-06-17T12:53:15.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.318+0000 7ff48e005700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7ff488062770 0x7ff4881444f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53350/0 (socket says 172.21.15.36:53350) 2022-06-17T12:53:15.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.318+0000 7ff48e005700 1 -- 172.21.15.36:0/137667167 learned_addr learned my addr 172.21.15.36:0/137667167 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:53:15.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.318+0000 7ff48e806700 1 --2- 172.21.15.36:0/137667167 >> v2:172.21.15.70:3300/0 conn(0x7ff488062b50 0x7ff4880ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:15.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.318+0000 7ff48d003700 1 -- 172.21.15.36:0/137667167 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871366830 0 0) 0x7ff488147e90 con 0x7ff488144c00 2022-06-17T12:53:15.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.318+0000 7ff48d003700 1 -- 172.21.15.36:0/137667167 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff470003260 con 0x7ff488144c00 2022-06-17T12:53:15.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.318+0000 7ff48d003700 1 -- 172.21.15.36:0/137667167 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345145336 0 0) 0x7ff4880b0ba0 con 0x7ff488062770 2022-06-17T12:53:15.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.318+0000 7ff48d003700 1 -- 172.21.15.36:0/137667167 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff488147e90 con 0x7ff488062770 2022-06-17T12:53:15.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.318+0000 7ff48e806700 1 -- 172.21.15.36:0/137667167 >> v1:172.21.15.70:6789/0 conn(0x7ff488144c00 legacy=0x7ff4880ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:15.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.318+0000 7ff48e806700 1 -- 172.21.15.36:0/137667167 >> v1:172.21.15.36:6789/0 conn(0x7ff488062770 legacy=0x7ff4881444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:15.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.318+0000 7ff48e806700 1 -- 172.21.15.36:0/137667167 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4880b0dd0 con 0x7ff488062b50 2022-06-17T12:53:15.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.318+0000 7ff48e806700 1 --2- 172.21.15.36:0/137667167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff488062b50 0x7ff4880ada40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7ff478002f70 tx=0x7ff478006df0).ready entity=mon.1 client_cookie=a758e2045b5c1a00 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:15.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.318+0000 7ff48d003700 1 -- 172.21.15.36:0/137667167 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff47800d540 con 0x7ff488062b50 2022-06-17T12:53:15.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.318+0000 7ff48d003700 1 -- 172.21.15.36:0/137667167 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff478000e90 con 0x7ff488062b50 2022-06-17T12:53:15.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.318+0000 7ff48d003700 1 -- 172.21.15.36:0/137667167 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff47800d890 con 0x7ff488062b50 2022-06-17T12:53:15.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.318+0000 7ff490269700 1 -- 172.21.15.36:0/137667167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff488062b50 msgr2=0x7ff4880ada40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:15.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.318+0000 7ff490269700 1 --2- 172.21.15.36:0/137667167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff488062b50 0x7ff4880ada40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7ff478002f70 tx=0x7ff478006df0).stop 2022-06-17T12:53:15.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.318+0000 7ff490269700 1 -- 172.21.15.36:0/137667167 shutdown_connections 2022-06-17T12:53:15.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.318+0000 7ff490269700 1 --2- 172.21.15.36:0/137667167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff488062b50 0x7ff4880ada40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:15.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.318+0000 7ff490269700 1 -- 172.21.15.36:0/137667167 >> 172.21.15.36:0/137667167 conn(0x7ff4880b53e0 msgr2=0x7ff4880b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:15.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.318+0000 7ff490269700 1 -- 172.21.15.36:0/137667167 shutdown_connections 2022-06-17T12:53:15.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.318+0000 7ff490269700 1 -- 172.21.15.36:0/137667167 wait complete. 2022-06-17T12:53:15.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.322+0000 7ff490269700 1 Processor -- start 2022-06-17T12:53:15.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.322+0000 7ff490269700 1 -- start start 2022-06-17T12:53:15.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.322+0000 7ff490269700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff488062770 0x7ff48814cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:15.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.322+0000 7ff490269700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff488062b50 0x7ff488159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:15.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.322+0000 7ff490269700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff488144c00 0x7ff48815a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:15.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.322+0000 7ff490269700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff488066e90 con 0x7ff488062770 2022-06-17T12:53:15.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.322+0000 7ff490269700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48814ad40 con 0x7ff488062b50 2022-06-17T12:53:15.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.322+0000 7ff490269700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48814afe0 con 0x7ff488144c00 2022-06-17T12:53:15.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.322+0000 7ff48e005700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff488062770 0x7ff48814cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:15.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.322+0000 7ff48e005700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff488062770 0x7ff48814cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35828/0 (socket says 172.21.15.36:35828) 2022-06-17T12:53:15.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.322+0000 7ff48d804700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff488062b50 0x7ff488159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:15.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.322+0000 7ff48e005700 1 -- 172.21.15.36:0/2692607902 learned_addr learned my addr 172.21.15.36:0/2692607902 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:15.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.322+0000 7ff48e806700 1 --2- 172.21.15.36:0/2692607902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff488144c00 0x7ff48815a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:15.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.322+0000 7ff48e005700 1 -- 172.21.15.36:0/2692607902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff488144c00 msgr2=0x7ff48815a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:15.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.322+0000 7ff48e005700 1 --2- 172.21.15.36:0/2692607902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff488144c00 0x7ff48815a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:15.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.322+0000 7ff48e005700 1 -- 172.21.15.36:0/2692607902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff488062b50 msgr2=0x7ff488159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:15.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.322+0000 7ff48e005700 1 --2- 172.21.15.36:0/2692607902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff488062b50 0x7ff488159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:15.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.322+0000 7ff48e005700 1 -- 172.21.15.36:0/2692607902 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47800b180 con 0x7ff488062770 2022-06-17T12:53:15.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.322+0000 7ff48e005700 1 --2- 172.21.15.36:0/2692607902 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff488062770 0x7ff48814cf10 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7ff484001110 tx=0x7ff484007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:15.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.322+0000 7ff4777fe700 1 -- 172.21.15.36:0/2692607902 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4840070d0 con 0x7ff488062770 2022-06-17T12:53:15.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.322+0000 7ff4777fe700 1 -- 172.21.15.36:0/2692607902 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff484007230 con 0x7ff488062770 2022-06-17T12:53:15.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.322+0000 7ff490269700 1 -- 172.21.15.36:0/2692607902 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff488066740 con 0x7ff488062770 2022-06-17T12:53:15.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.322+0000 7ff4777fe700 1 -- 172.21.15.36:0/2692607902 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff484005610 con 0x7ff488062770 2022-06-17T12:53:15.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.322+0000 7ff490269700 1 -- 172.21.15.36:0/2692607902 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff488066a20 con 0x7ff488062770 2022-06-17T12:53:15.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.326+0000 7ff490269700 1 -- 172.21.15.36:0/2692607902 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff488067680 con 0x7ff488062770 2022-06-17T12:53:15.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.326+0000 7ff4777fe700 1 -- 172.21.15.36:0/2692607902 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff484010070 con 0x7ff488062770 2022-06-17T12:53:15.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.326+0000 7ff4777fe700 1 --2- 172.21.15.36:0/2692607902 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff468053940 0x7ff468055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:15.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.326+0000 7ff4777fe700 1 -- 172.21.15.36:0/2692607902 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff48406b5b0 con 0x7ff488062770 2022-06-17T12:53:15.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.326+0000 7ff48d804700 1 --2- 172.21.15.36:0/2692607902 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff468053940 0x7ff468055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:15.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.326+0000 7ff48d804700 1 --2- 172.21.15.36:0/2692607902 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff468053940 0x7ff468055e00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7ff48000d000 tx=0x7ff480007dc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:15.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.334+0000 7ff4777fe700 1 -- 172.21.15.36:0/2692607902 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff4840423b0 con 0x7ff488062770 2022-06-17T12:53:15.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.498+0000 7ff490269700 1 -- 172.21.15.36:0/2692607902 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff48814b1a0 con 0x7ff488062770 2022-06-17T12:53:15.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.498+0000 7ff4777fe700 1 -- 172.21.15.36:0/2692607902 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+2140 (secure 0 0 0) 0x7ff4840459d0 con 0x7ff488062770 2022-06-17T12:53:15.484 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:53:15.485 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":36,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/2541998896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2541998896},{"type":"v1","addr":"172.21.15.70:6833","nonce":2541998896}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:53:04.907795+0000","modified":"2022-06-17T12:53:06.511529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6381":{"gid":6381,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/2514005995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2514005995},{"type":"v1","addr":"172.21.15.36:6835","nonce":2514005995}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-06-17T12:53:15.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.498+0000 7ff490269700 1 -- 172.21.15.36:0/2692607902 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff468053940 msgr2=0x7ff468055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:15.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.498+0000 7ff490269700 1 --2- 172.21.15.36:0/2692607902 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff468053940 0x7ff468055e00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7ff48000d000 tx=0x7ff480007dc0).stop 2022-06-17T12:53:15.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.498+0000 7ff490269700 1 -- 172.21.15.36:0/2692607902 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff488062770 msgr2=0x7ff48814cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:15.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.498+0000 7ff490269700 1 --2- 172.21.15.36:0/2692607902 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff488062770 0x7ff48814cf10 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7ff484001110 tx=0x7ff484007ad0).stop 2022-06-17T12:53:15.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.502+0000 7ff490269700 1 -- 172.21.15.36:0/2692607902 shutdown_connections 2022-06-17T12:53:15.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.502+0000 7ff490269700 1 --2- 172.21.15.36:0/2692607902 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff468053940 0x7ff468055e00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:15.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.502+0000 7ff490269700 1 --2- 172.21.15.36:0/2692607902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff488144c00 0x7ff48815a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:15.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.502+0000 7ff490269700 1 --2- 172.21.15.36:0/2692607902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff488062b50 0x7ff488159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:15.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.502+0000 7ff490269700 1 --2- 172.21.15.36:0/2692607902 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff488062770 0x7ff48814cf10 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:15.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.502+0000 7ff490269700 1 -- 172.21.15.36:0/2692607902 >> 172.21.15.36:0/2692607902 conn(0x7ff4880b53e0 msgr2=0x7ff4880b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:15.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.502+0000 7ff490269700 1 -- 172.21.15.36:0/2692607902 shutdown_connections 2022-06-17T12:53:15.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:15.502+0000 7ff490269700 1 -- 172.21.15.36:0/2692607902 wait complete. 2022-06-17T12:53:15.494 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 36 2022-06-17T12:53:15.499 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_cap_revoke_eviction_timeout 30.0 2022-06-17T12:53:15.611 INFO:teuthology.orchestra.run.smithi036.stdout:{"success":"mds_cap_revoke_eviction_timeout = '30.000000' "} 2022-06-17T12:53:15.612 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_cap_revoke_eviction_timeout = '30.000000' " } 2022-06-17T12:53:15.614 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:15.614 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:53:15.615 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:53:15.625 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:53:15.626 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:53:15.627 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-06-17T12:53:15.628 DEBUG:teuthology.orchestra.run.smithi107:> import time 2022-06-17T12:53:15.629 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:53:15.629 DEBUG:teuthology.orchestra.run.smithi107:> with open("/home/ubuntu/cephtest/mnt.0/background_file", '"'"'w'"'"') as f: 2022-06-17T12:53:15.630 DEBUG:teuthology.orchestra.run.smithi107:> f.write('"'"'content'"'"') 2022-06-17T12:53:15.630 DEBUG:teuthology.orchestra.run.smithi107:> f.flush() 2022-06-17T12:53:15.631 DEBUG:teuthology.orchestra.run.smithi107:> f.write('"'"'content2'"'"') 2022-06-17T12:53:15.631 DEBUG:teuthology.orchestra.run.smithi107:> while True: 2022-06-17T12:53:15.631 DEBUG:teuthology.orchestra.run.smithi107:> time.sleep(1) 2022-06-17T12:53:15.632 DEBUG:teuthology.orchestra.run.smithi107:> ' 2022-06-17T12:53:15.635 DEBUG:teuthology.orchestra.run.smithi107:> sudo ls /home/ubuntu/cephtest/mnt.0/background_file 2022-06-17T12:53:15.655 INFO:teuthology.orchestra.run.smithi107.stderr:ls: cannot access '/home/ubuntu/cephtest/mnt.0/background_file': No such file or directory 2022-06-17T12:53:15.657 DEBUG:teuthology.orchestra.run:got remote process result: 2 2022-06-17T12:53:15.802 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:53:15.807 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:53:15.809 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:53:15.813 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:53:15.818 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:53:16.660 DEBUG:teuthology.orchestra.run.smithi107:> sudo ls /home/ubuntu/cephtest/mnt.0/background_file 2022-06-17T12:53:16.680 INFO:teuthology.orchestra.run.smithi107.stdout:/home/ubuntu/cephtest/mnt.0/background_file 2022-06-17T12:53:16.681 DEBUG:tasks.cephfs.mount:File background_file became visible from 0 after 1s 2022-06-17T12:53:16.682 DEBUG:teuthology.orchestra.run.smithi136:> sudo ls /home/ubuntu/cephtest/mnt.1/background_file 2022-06-17T12:53:16.721 INFO:teuthology.orchestra.run.smithi136.stdout:/home/ubuntu/cephtest/mnt.1/background_file 2022-06-17T12:53:16.730 DEBUG:tasks.cephfs.mount:File background_file became visible from 1 after 0s 2022-06-17T12:53:16.731 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@smithi107.front.sepia.ceph.com... 2022-06-17T12:53:16.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:16.749 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:53:16.750 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:53:16.779 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:53:16.781 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:53:16.784 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-06-17T12:53:16.785 DEBUG:teuthology.orchestra.run.smithi136:> import os 2022-06-17T12:53:16.786 DEBUG:teuthology.orchestra.run.smithi136:> import time 2022-06-17T12:53:16.787 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:53:16.789 DEBUG:teuthology.orchestra.run.smithi136:> fd = os.open("/home/ubuntu/cephtest/mnt.1/background_file", os.O_RDWR | os.O_CREAT, 0o644) 2022-06-17T12:53:16.790 DEBUG:teuthology.orchestra.run.smithi136:> try: 2022-06-17T12:53:16.791 DEBUG:teuthology.orchestra.run.smithi136:> while True: 2022-06-17T12:53:16.792 DEBUG:teuthology.orchestra.run.smithi136:> os.write(fd, b'"'"'content'"'"') 2022-06-17T12:53:16.792 DEBUG:teuthology.orchestra.run.smithi136:> time.sleep(1) 2022-06-17T12:53:16.793 DEBUG:teuthology.orchestra.run.smithi136:> if not False: 2022-06-17T12:53:16.794 DEBUG:teuthology.orchestra.run.smithi136:> break 2022-06-17T12:53:16.795 DEBUG:teuthology.orchestra.run.smithi136:> except IOError as e: 2022-06-17T12:53:16.796 DEBUG:teuthology.orchestra.run.smithi136:> pass 2022-06-17T12:53:16.796 DEBUG:teuthology.orchestra.run.smithi136:> os.close(fd) 2022-06-17T12:53:16.797 DEBUG:teuthology.orchestra.run.smithi136:> ' 2022-06-17T12:53:45.883 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:53:45.896 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:53:45.908 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:53:45.915 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:53:45.919 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:53:56.730 INFO:tasks.cephfs.test_misc:cap_waiter waited 39.92809081077576s 2022-06-17T12:53:56.731 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist ls --format=json 2022-06-17T12:53:56.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.856+0000 7f1d95378700 1 Processor -- start 2022-06-17T12:53:56.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.856+0000 7f1d95378700 1 -- start start 2022-06-17T12:53:56.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d95378700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d900ad330 0x7f1d900ad880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:56.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d95378700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d900add70 con 0x7f1d90066c60 2022-06-17T12:53:56.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d95378700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d90146d50 con 0x7f1d900afd80 2022-06-17T12:53:56.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d95378700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d90067040 con 0x7f1d900ad330 2022-06-17T12:53:56.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8effd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f1d90066c60 0x7f1d90146640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53368/0 (socket says 172.21.15.36:53368) 2022-06-17T12:53:56.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8effd700 1 -- 172.21.15.36:0/4223812188 learned_addr learned my addr 172.21.15.36:0/4223812188 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:53:56.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8f7fe700 1 --2- 172.21.15.36:0/4223812188 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d900ad330 0x7f1d900ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:56.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8dffb700 1 -- 172.21.15.36:0/4223812188 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 987898105 0 0) 0x7f1d80001110 con 0x7f1d90066c60 2022-06-17T12:53:56.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8dffb700 1 -- 172.21.15.36:0/4223812188 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d70003260 con 0x7f1d90066c60 2022-06-17T12:53:56.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8dffb700 1 -- 172.21.15.36:0/4223812188 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518667944 0 0) 0x7f1d90146d50 con 0x7f1d900afd80 2022-06-17T12:53:56.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8dffb700 1 -- 172.21.15.36:0/4223812188 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d80001110 con 0x7f1d900afd80 2022-06-17T12:53:56.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8dffb700 1 -- 172.21.15.36:0/4223812188 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2900455923 0 0) 0x7f1d70003260 con 0x7f1d90066c60 2022-06-17T12:53:56.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8dffb700 1 -- 172.21.15.36:0/4223812188 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d900ad330 msgr2=0x7f1d900ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:56.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8dffb700 1 --2- 172.21.15.36:0/4223812188 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d900ad330 0x7f1d900ad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:56.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8dffb700 1 -- 172.21.15.36:0/4223812188 >> v1:172.21.15.70:6789/0 conn(0x7f1d900afd80 legacy=0x7f1d9005f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:56.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8dffb700 1 -- 172.21.15.36:0/4223812188 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d900b0940 con 0x7f1d90066c60 2022-06-17T12:53:56.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8dffb700 1 -- 172.21.15.36:0/4223812188 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f1d80001810 con 0x7f1d90066c60 2022-06-17T12:53:56.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8dffb700 1 -- 172.21.15.36:0/4223812188 >> v1:172.21.15.36:6789/0 conn(0x7f1d90066c60 legacy=0x7f1d90146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:56.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8dffb700 1 --2- 172.21.15.36:0/4223812188 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d70003910 0x7f1d70003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:56.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8dffb700 1 --2- 172.21.15.36:0/4223812188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d70004500 0x7f1d70004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:56.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8dffb700 1 --2- 172.21.15.36:0/4223812188 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d900ad330 0x7f1d70004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:56.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8dffb700 1 -- 172.21.15.36:0/4223812188 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d90067040 con 0x7f1d70003910 2022-06-17T12:53:56.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8dffb700 1 -- 172.21.15.36:0/4223812188 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7000b7b0 con 0x7f1d70004500 2022-06-17T12:53:56.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8dffb700 1 -- 172.21.15.36:0/4223812188 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7000b970 con 0x7f1d900ad330 2022-06-17T12:53:56.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8effd700 1 --2- 172.21.15.36:0/4223812188 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d70003910 0x7f1d70003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:56.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8e7fc700 1 --2- 172.21.15.36:0/4223812188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d70004500 0x7f1d70004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:56.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8f7fe700 1 --2- 172.21.15.36:0/4223812188 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d900ad330 0x7f1d70004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:56.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8effd700 1 -- 172.21.15.36:0/4223812188 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d900ad330 msgr2=0x7f1d70004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:56.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8effd700 1 --2- 172.21.15.36:0/4223812188 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d900ad330 0x7f1d70004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:56.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8effd700 1 -- 172.21.15.36:0/4223812188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d70004500 msgr2=0x7f1d70004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:56.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8effd700 1 --2- 172.21.15.36:0/4223812188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d70004500 0x7f1d70004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:56.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8effd700 1 -- 172.21.15.36:0/4223812188 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d70003260 con 0x7f1d70003910 2022-06-17T12:53:56.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8e7fc700 1 --2- 172.21.15.36:0/4223812188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d70004500 0x7f1d70004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:53:56.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8effd700 1 --2- 172.21.15.36:0/4223812188 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d70003910 0x7f1d70003d10 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f1d800015e0 tx=0x7f1d900ae830).ready entity=mon.0 client_cookie=fbb2105cdfdc2632 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:56.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8dffb700 1 -- 172.21.15.36:0/4223812188 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d80007170 con 0x7f1d70003910 2022-06-17T12:53:56.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8dffb700 1 -- 172.21.15.36:0/4223812188 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d800072d0 con 0x7f1d70003910 2022-06-17T12:53:56.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8dffb700 1 -- 172.21.15.36:0/4223812188 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d8000ca20 con 0x7f1d70003910 2022-06-17T12:53:56.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d95378700 1 -- 172.21.15.36:0/4223812188 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d70003910 msgr2=0x7f1d70003d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:56.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d95378700 1 --2- 172.21.15.36:0/4223812188 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d70003910 0x7f1d70003d10 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f1d800015e0 tx=0x7f1d900ae830).stop 2022-06-17T12:53:56.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d8f7fe700 1 -- 172.21.15.36:0/4223812188 reap_dead start 2022-06-17T12:53:56.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d95378700 1 -- 172.21.15.36:0/4223812188 shutdown_connections 2022-06-17T12:53:56.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d95378700 1 -- 172.21.15.36:0/4223812188 >> 172.21.15.36:0/4223812188 conn(0x7f1d900b53e0 msgr2=0x7f1d900b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:56.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d95378700 1 -- 172.21.15.36:0/4223812188 shutdown_connections 2022-06-17T12:53:56.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.860+0000 7f1d95378700 1 -- 172.21.15.36:0/4223812188 wait complete. 2022-06-17T12:53:56.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.864+0000 7f1d95378700 1 Processor -- start 2022-06-17T12:53:56.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.864+0000 7f1d95378700 1 -- start start 2022-06-17T12:53:56.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.864+0000 7f1d95378700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d90141560 0x7f1d9010e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:56.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.864+0000 7f1d95378700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d9014a1c0 0x7f1d90156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:56.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.864+0000 7f1d95378700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d90156840 0x7f1d90147e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:56.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.864+0000 7f1d95378700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d90155c80 con 0x7f1d9014a1c0 2022-06-17T12:53:56.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.864+0000 7f1d95378700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d90141940 con 0x7f1d90156840 2022-06-17T12:53:56.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.864+0000 7f1d95378700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d90111730 con 0x7f1d90141560 2022-06-17T12:53:56.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.864+0000 7f1d8e7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d9014a1c0 0x7f1d90156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:56.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.864+0000 7f1d8e7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d9014a1c0 0x7f1d90156350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35850/0 (socket says 172.21.15.36:35850) 2022-06-17T12:53:56.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.864+0000 7f1d8e7fc700 1 -- 172.21.15.36:0/4114172802 learned_addr learned my addr 172.21.15.36:0/4114172802 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:56.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.864+0000 7f1d8f7fe700 1 --2- 172.21.15.36:0/4114172802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d90156840 0x7f1d90147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:56.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.864+0000 7f1d8effd700 1 --2- 172.21.15.36:0/4114172802 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d90141560 0x7f1d9010e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:56.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.864+0000 7f1d8e7fc700 1 -- 172.21.15.36:0/4114172802 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d90141560 msgr2=0x7f1d9010e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:56.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.864+0000 7f1d8e7fc700 1 --2- 172.21.15.36:0/4114172802 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d90141560 0x7f1d9010e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:56.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.864+0000 7f1d8e7fc700 1 -- 172.21.15.36:0/4114172802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d90156840 msgr2=0x7f1d90147e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:56.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.864+0000 7f1d8e7fc700 1 --2- 172.21.15.36:0/4114172802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d90156840 0x7f1d90147e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:56.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.864+0000 7f1d8e7fc700 1 -- 172.21.15.36:0/4114172802 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d901119d0 con 0x7f1d9014a1c0 2022-06-17T12:53:56.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.864+0000 7f1d8e7fc700 1 --2- 172.21.15.36:0/4114172802 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d9014a1c0 0x7f1d90156350 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f1d78007580 tx=0x7f1d78007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:56.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.864+0000 7f1d87fff700 1 -- 172.21.15.36:0/4114172802 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d780116b0 con 0x7f1d9014a1c0 2022-06-17T12:53:56.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.864+0000 7f1d87fff700 1 -- 172.21.15.36:0/4114172802 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d78011810 con 0x7f1d9014a1c0 2022-06-17T12:53:56.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.864+0000 7f1d87fff700 1 -- 172.21.15.36:0/4114172802 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d78011b70 con 0x7f1d9014a1c0 2022-06-17T12:53:56.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.864+0000 7f1d95378700 1 -- 172.21.15.36:0/4114172802 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d90155eb0 con 0x7f1d9014a1c0 2022-06-17T12:53:56.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.864+0000 7f1d95378700 1 -- 172.21.15.36:0/4114172802 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9014aa30 con 0x7f1d9014a1c0 2022-06-17T12:53:56.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.868+0000 7f1d95378700 1 -- 172.21.15.36:0/4114172802 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d90156040 con 0x7f1d9014a1c0 2022-06-17T12:53:56.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.868+0000 7f1d87fff700 1 -- 172.21.15.36:0/4114172802 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f1d7801b030 con 0x7f1d9014a1c0 2022-06-17T12:53:56.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.868+0000 7f1d87fff700 1 --2- 172.21.15.36:0/4114172802 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1d74053a40 0x7f1d74055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:56.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.868+0000 7f1d87fff700 1 -- 172.21.15.36:0/4114172802 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f1d7806c060 con 0x7f1d9014a1c0 2022-06-17T12:53:56.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.868+0000 7f1d8effd700 1 --2- 172.21.15.36:0/4114172802 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1d74053a40 0x7f1d74055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:56.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.868+0000 7f1d8effd700 1 --2- 172.21.15.36:0/4114172802 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1d74053a40 0x7f1d74055f00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f1d80000e90 tx=0x7f1d80015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:56.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:56.872+0000 7f1d87fff700 1 -- 172.21.15.36:0/4114172802 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1d78042cb0 con 0x7f1d9014a1c0 2022-06-17T12:53:56.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.008+0000 7f1d95378700 1 -- 172.21.15.36:0/4114172802 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd blacklist ls", "format": "json"} v 0) v1 -- 0x7f1d900b1300 con 0x7f1d9014a1c0 2022-06-17T12:53:56.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.012+0000 7f1d87fff700 1 -- 172.21.15.36:0/4114172802 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist ls", "format": "json"}]=0 listed 1 entries v75) v1 ==== 98+0+80 (secure 0 0 0) 0x7f1d780462d0 con 0x7f1d9014a1c0 2022-06-17T12:53:56.999 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:53:57.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.012+0000 7f1d95378700 1 -- 172.21.15.36:0/4114172802 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1d74053a40 msgr2=0x7f1d74055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:57.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.012+0000 7f1d95378700 1 --2- 172.21.15.36:0/4114172802 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1d74053a40 0x7f1d74055f00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f1d80000e90 tx=0x7f1d80015000).stop 2022-06-17T12:53:57.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.012+0000 7f1d95378700 1 -- 172.21.15.36:0/4114172802 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d9014a1c0 msgr2=0x7f1d90156350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:57.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.012+0000 7f1d95378700 1 --2- 172.21.15.36:0/4114172802 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d9014a1c0 0x7f1d90156350 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f1d78007580 tx=0x7f1d78007a00).stop 2022-06-17T12:53:57.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.012+0000 7f1d95378700 1 -- 172.21.15.36:0/4114172802 shutdown_connections 2022-06-17T12:53:57.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.012+0000 7f1d95378700 1 --2- 172.21.15.36:0/4114172802 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1d74053a40 0x7f1d74055f00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.012+0000 7f1d95378700 1 --2- 172.21.15.36:0/4114172802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d90156840 0x7f1d90147e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.012+0000 7f1d95378700 1 --2- 172.21.15.36:0/4114172802 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d9014a1c0 0x7f1d90156350 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.012+0000 7f1d95378700 1 --2- 172.21.15.36:0/4114172802 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d90141560 0x7f1d9010e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.012+0000 7f1d95378700 1 -- 172.21.15.36:0/4114172802 >> 172.21.15.36:0/4114172802 conn(0x7f1d900b53e0 msgr2=0x7f1d9013f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:57.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.012+0000 7f1d95378700 1 -- 172.21.15.36:0/4114172802 shutdown_connections 2022-06-17T12:53:57.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.012+0000 7f1d95378700 1 -- 172.21.15.36:0/4114172802 wait complete. 2022-06-17T12:53:57.007 INFO:teuthology.orchestra.run.smithi036.stderr:listed 1 entries 2022-06-17T12:53:57.012 INFO:teuthology.orchestra.run.smithi036.stdout:[{"addr":"172.21.15.107:0/973797138","until":"2022-06-17T13:53:50.522507+0000"}] 2022-06-17T12:53:57.030 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2022-06-17T12:53:57.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:57.031 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:53:57.031 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:53:57.079 INFO:teuthology.orchestra.run.smithi107.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2022-06-17T12:53:57.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:53:57.081 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2022-06-17T12:53:57.081 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2022-06-17T12:53:57.082 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:53:57.082 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:53:57.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:57.102 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:53:57.102 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:53:57.110 INFO:teuthology.orchestra.run.smithi107.stdout:ext2/ext3 2022-06-17T12:53:57.111 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:53:57.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:53:57.112 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:53:57.160 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:53:57.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.280+0000 7ff1df8e9700 1 Processor -- start 2022-06-17T12:53:57.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.280+0000 7ff1df8e9700 1 -- start start 2022-06-17T12:53:57.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1df8e9700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1d80afd80 0x7ff1d80ad880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:57.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1df8e9700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff1d80ad330 0x7ff1d805f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:57.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1df8e9700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d80add70 con 0x7ff1d8066c60 2022-06-17T12:53:57.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1df8e9700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d8146d50 con 0x7ff1d80ad330 2022-06-17T12:53:57.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1df8e9700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d8067040 con 0x7ff1d80afd80 2022-06-17T12:53:57.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1dde86700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff1d80ad330 0x7ff1d805f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:57.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1dce84700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1d80afd80 0x7ff1d80ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:57.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1dd685700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff1d8066c60 0x7ff1d8146640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:49120/0 (socket says 172.21.15.36:49120) 2022-06-17T12:53:57.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1dd685700 1 -- 172.21.15.36:0/3422273233 learned_addr learned my addr 172.21.15.36:0/3422273233 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:53:57.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1dce84700 1 -- 172.21.15.36:0/3422273233 >> v2:172.21.15.36:3300/0 conn(0x7ff1d80ad330 msgr2=0x7ff1d805f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:57.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1dce84700 1 --2- 172.21.15.36:0/3422273233 >> v2:172.21.15.36:3300/0 conn(0x7ff1d80ad330 0x7ff1d805f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.278 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1dce84700 1 -- 172.21.15.36:0/3422273233 >> v1:172.21.15.70:6789/0 conn(0x7ff1d8066c60 legacy=0x7ff1d8146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:57.278 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1cffff700 1 -- 172.21.15.36:0/3422273233 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323604079 0 0) 0x7ff1c8001110 con 0x7ff1d8066c60 2022-06-17T12:53:57.279 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1dce84700 1 -- 172.21.15.36:0/3422273233 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d80b0940 con 0x7ff1d80afd80 2022-06-17T12:53:57.279 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1dce84700 1 --2- 172.21.15.36:0/3422273233 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1d80afd80 0x7ff1d80ad880 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7ff1d4002f70 tx=0x7ff1d4006df0).ready entity=mon.2 client_cookie=d8baf868532cb682 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:57.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1cffff700 1 -- 172.21.15.36:0/3422273233 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1d400d570 con 0x7ff1d80afd80 2022-06-17T12:53:57.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1cffff700 1 -- 172.21.15.36:0/3422273233 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d4000e90 con 0x7ff1d80afd80 2022-06-17T12:53:57.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1cffff700 1 -- 172.21.15.36:0/3422273233 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1d400d8c0 con 0x7ff1d80afd80 2022-06-17T12:53:57.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1df8e9700 1 -- 172.21.15.36:0/3422273233 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1d80afd80 msgr2=0x7ff1d80ad880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:57.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1df8e9700 1 --2- 172.21.15.36:0/3422273233 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1d80afd80 0x7ff1d80ad880 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7ff1d4002f70 tx=0x7ff1d4006df0).stop 2022-06-17T12:53:57.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1df8e9700 1 -- 172.21.15.36:0/3422273233 shutdown_connections 2022-06-17T12:53:57.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1df8e9700 1 --2- 172.21.15.36:0/3422273233 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1d80afd80 0x7ff1d80ad880 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1df8e9700 1 --2- 172.21.15.36:0/3422273233 >> v2:172.21.15.36:3300/0 conn(0x7ff1d80ad330 0x7ff1d805f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1df8e9700 1 -- 172.21.15.36:0/3422273233 >> 172.21.15.36:0/3422273233 conn(0x7ff1d80b53e0 msgr2=0x7ff1d80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:57.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1df8e9700 1 -- 172.21.15.36:0/3422273233 shutdown_connections 2022-06-17T12:53:57.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.284+0000 7ff1df8e9700 1 -- 172.21.15.36:0/3422273233 wait complete. 2022-06-17T12:53:57.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.288+0000 7ff1df8e9700 1 Processor -- start 2022-06-17T12:53:57.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.288+0000 7ff1df8e9700 1 -- start start 2022-06-17T12:53:57.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.288+0000 7ff1df8e9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1d8066c60 0x7ff1d8145a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:57.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.288+0000 7ff1df8e9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1d80ad330 0x7ff1d8152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:57.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.288+0000 7ff1df8e9700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1d80afd80 0x7ff1d813ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:57.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.288+0000 7ff1df8e9700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d8140a30 con 0x7ff1d8066c60 2022-06-17T12:53:57.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.288+0000 7ff1df8e9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d8140790 con 0x7ff1d80ad330 2022-06-17T12:53:57.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.288+0000 7ff1df8e9700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d810f8d0 con 0x7ff1d80afd80 2022-06-17T12:53:57.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.288+0000 7ff1dd685700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1d8066c60 0x7ff1d8145a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:57.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.288+0000 7ff1dd685700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1d8066c60 0x7ff1d8145a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35862/0 (socket says 172.21.15.36:35862) 2022-06-17T12:53:57.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.288+0000 7ff1dd685700 1 -- 172.21.15.36:0/3325903448 learned_addr learned my addr 172.21.15.36:0/3325903448 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:57.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.288+0000 7ff1dde86700 1 --2- 172.21.15.36:0/3325903448 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1d80afd80 0x7ff1d813ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:57.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.288+0000 7ff1dce84700 1 --2- 172.21.15.36:0/3325903448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1d80ad330 0x7ff1d8152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:57.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.292+0000 7ff1dd685700 1 -- 172.21.15.36:0/3325903448 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1d80afd80 msgr2=0x7ff1d813ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:57.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.292+0000 7ff1dd685700 1 --2- 172.21.15.36:0/3325903448 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1d80afd80 0x7ff1d813ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.292+0000 7ff1dd685700 1 -- 172.21.15.36:0/3325903448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1d80ad330 msgr2=0x7ff1d8152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:57.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.292+0000 7ff1dd685700 1 --2- 172.21.15.36:0/3325903448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1d80ad330 0x7ff1d8152490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.292+0000 7ff1dd685700 1 -- 172.21.15.36:0/3325903448 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d400b180 con 0x7ff1d8066c60 2022-06-17T12:53:57.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.292+0000 7ff1dd685700 1 --2- 172.21.15.36:0/3325903448 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1d8066c60 0x7ff1d8145a00 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7ff1c8000ed0 tx=0x7ff1c8007920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:57.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.292+0000 7ff1ce7fc700 1 -- 172.21.15.36:0/3325903448 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1c80070d0 con 0x7ff1d8066c60 2022-06-17T12:53:57.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.292+0000 7ff1ce7fc700 1 -- 172.21.15.36:0/3325903448 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c8007230 con 0x7ff1d8066c60 2022-06-17T12:53:57.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.292+0000 7ff1df8e9700 1 -- 172.21.15.36:0/3325903448 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d81463c0 con 0x7ff1d8066c60 2022-06-17T12:53:57.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.292+0000 7ff1ce7fc700 1 -- 172.21.15.36:0/3325903448 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1c8005610 con 0x7ff1d8066c60 2022-06-17T12:53:57.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.292+0000 7ff1df8e9700 1 -- 172.21.15.36:0/3325903448 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d8140360 con 0x7ff1d8066c60 2022-06-17T12:53:57.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.292+0000 7ff1df8e9700 1 -- 172.21.15.36:0/3325903448 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d813f6c0 con 0x7ff1d8066c60 2022-06-17T12:53:57.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.292+0000 7ff1ce7fc700 1 -- 172.21.15.36:0/3325903448 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff1c8010070 con 0x7ff1d8066c60 2022-06-17T12:53:57.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.296+0000 7ff1ce7fc700 1 --2- 172.21.15.36:0/3325903448 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff1c00539d0 0x7ff1c0055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:57.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.296+0000 7ff1dce84700 1 --2- 172.21.15.36:0/3325903448 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff1c00539d0 0x7ff1c0055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:57.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.296+0000 7ff1ce7fc700 1 -- 172.21.15.36:0/3325903448 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6463+0+0 (secure 0 0 0) 0x7ff1c806b4c0 con 0x7ff1d8066c60 2022-06-17T12:53:57.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.296+0000 7ff1dce84700 1 --2- 172.21.15.36:0/3325903448 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff1c00539d0 0x7ff1c0055e90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7ff1d401a000 tx=0x7ff1d4014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:57.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.300+0000 7ff1ce7fc700 1 -- 172.21.15.36:0/3325903448 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff1d813f6c0 con 0x7ff1d8066c60 2022-06-17T12:53:57.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.428+0000 7ff1df8e9700 1 -- 172.21.15.36:0/3325903448 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1d80add70 con 0x7ff1d8066c60 2022-06-17T12:53:57.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.428+0000 7ff1ce7fc700 1 -- 172.21.15.36:0/3325903448 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16335 (secure 0 0 0) 0x7ff1d80add70 con 0x7ff1d8066c60 2022-06-17T12:53:57.419 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:53:57.420 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":75,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:53:51.518039+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:53:03.265804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-06-17T12:53:04.385158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.107:0/973797138":"2022-06-17T13:53:50.522507+0000"},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:53:57.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.432+0000 7ff1df8e9700 1 -- 172.21.15.36:0/3325903448 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff1c00539d0 msgr2=0x7ff1c0055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:57.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.432+0000 7ff1df8e9700 1 --2- 172.21.15.36:0/3325903448 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff1c00539d0 0x7ff1c0055e90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7ff1d401a000 tx=0x7ff1d4014000).stop 2022-06-17T12:53:57.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.432+0000 7ff1df8e9700 1 -- 172.21.15.36:0/3325903448 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1d8066c60 msgr2=0x7ff1d8145a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:57.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.432+0000 7ff1df8e9700 1 --2- 172.21.15.36:0/3325903448 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1d8066c60 0x7ff1d8145a00 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7ff1c8000ed0 tx=0x7ff1c8007920).stop 2022-06-17T12:53:57.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.432+0000 7ff1df8e9700 1 -- 172.21.15.36:0/3325903448 shutdown_connections 2022-06-17T12:53:57.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.432+0000 7ff1df8e9700 1 --2- 172.21.15.36:0/3325903448 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff1c00539d0 0x7ff1c0055e90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.432+0000 7ff1df8e9700 1 --2- 172.21.15.36:0/3325903448 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1d80afd80 0x7ff1d813ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.432+0000 7ff1df8e9700 1 --2- 172.21.15.36:0/3325903448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1d80ad330 0x7ff1d8152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.432+0000 7ff1df8e9700 1 --2- 172.21.15.36:0/3325903448 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1d8066c60 0x7ff1d8145a00 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.432+0000 7ff1df8e9700 1 -- 172.21.15.36:0/3325903448 >> 172.21.15.36:0/3325903448 conn(0x7ff1d80b53e0 msgr2=0x7ff1d8143bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:57.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.432+0000 7ff1df8e9700 1 -- 172.21.15.36:0/3325903448 shutdown_connections 2022-06-17T12:53:57.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.432+0000 7ff1df8e9700 1 -- 172.21.15.36:0/3325903448 wait complete. 2022-06-17T12:53:57.431 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:53:03.265804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:53:04.385158+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:53:57.432 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:53:57.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.552+0000 7f95eb93a700 1 Processor -- start 2022-06-17T12:53:57.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.552+0000 7f95eb93a700 1 -- start start 2022-06-17T12:53:57.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95eb93a700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f95e40b4120 0x7f95e4146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:57.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95eb93a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f95e4147270 0x7f95e4140100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:57.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95eb93a700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e41476e0 con 0x7f95e40b64f0 2022-06-17T12:53:57.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95eb93a700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e40b68d0 con 0x7f95e4147270 2022-06-17T12:53:57.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95eb93a700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e40b6e20 con 0x7f95e40b4120 2022-06-17T12:53:57.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95e9ed7700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f95e4147270 0x7f95e4140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:57.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95e9ed7700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f95e4147270 0x7f95e4140100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35872/0 (socket says 172.21.15.36:35872) 2022-06-17T12:53:57.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95e9ed7700 1 -- 172.21.15.36:0/1837925108 learned_addr learned my addr 172.21.15.36:0/1837925108 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:57.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95e8ed5700 1 --2- 172.21.15.36:0/1837925108 >> v2:172.21.15.70:3300/0 conn(0x7f95e40b4120 0x7f95e4146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:57.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95e9ed7700 1 -- 172.21.15.36:0/1837925108 >> v2:172.21.15.70:3300/0 conn(0x7f95e40b4120 msgr2=0x7f95e4146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:57.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95e9ed7700 1 --2- 172.21.15.36:0/1837925108 >> v2:172.21.15.70:3300/0 conn(0x7f95e40b4120 0x7f95e4146d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95e9ed7700 1 -- 172.21.15.36:0/1837925108 >> v1:172.21.15.70:6789/0 conn(0x7f95e40b64f0 legacy=0x7f95e4143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:57.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95e9ed7700 1 -- 172.21.15.36:0/1837925108 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e41468e0 con 0x7f95e4147270 2022-06-17T12:53:57.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95e9ed7700 1 --2- 172.21.15.36:0/1837925108 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95e4147270 0x7f95e4140100 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f95dc0032f0 tx=0x7f95dc009290).ready entity=mon.0 client_cookie=ede3a00cbeb66f43 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:57.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95dbfff700 1 -- 172.21.15.36:0/1837925108 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95dc00b870 con 0x7f95e4147270 2022-06-17T12:53:57.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95dbfff700 1 -- 172.21.15.36:0/1837925108 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95dc00b9d0 con 0x7f95e4147270 2022-06-17T12:53:57.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95dbfff700 1 -- 172.21.15.36:0/1837925108 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95dc00bb40 con 0x7f95e4147270 2022-06-17T12:53:57.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95eb93a700 1 -- 172.21.15.36:0/1837925108 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95e4147270 msgr2=0x7f95e4140100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:57.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95eb93a700 1 --2- 172.21.15.36:0/1837925108 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95e4147270 0x7f95e4140100 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f95dc0032f0 tx=0x7f95dc009290).stop 2022-06-17T12:53:57.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95eb93a700 1 -- 172.21.15.36:0/1837925108 shutdown_connections 2022-06-17T12:53:57.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95eb93a700 1 --2- 172.21.15.36:0/1837925108 >> v2:172.21.15.70:3300/0 conn(0x7f95e40b4120 0x7f95e4146d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95eb93a700 1 --2- 172.21.15.36:0/1837925108 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95e4147270 0x7f95e4140100 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95eb93a700 1 -- 172.21.15.36:0/1837925108 >> 172.21.15.36:0/1837925108 conn(0x7f95e40620f0 msgr2=0x7f95e4062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:57.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95eb93a700 1 -- 172.21.15.36:0/1837925108 shutdown_connections 2022-06-17T12:53:57.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95eb93a700 1 -- 172.21.15.36:0/1837925108 wait complete. 2022-06-17T12:53:57.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95eb93a700 1 Processor -- start 2022-06-17T12:53:57.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.556+0000 7f95eb93a700 1 -- start start 2022-06-17T12:53:57.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.560+0000 7f95eb93a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95e40b4120 0x7f95e413e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:57.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.560+0000 7f95eb93a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95e40b64f0 0x7f95e40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:57.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.560+0000 7f95eb93a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95e4147270 0x7f95e40b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:57.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.560+0000 7f95eb93a700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e4141e50 con 0x7f95e40b4120 2022-06-17T12:53:57.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.560+0000 7f95eb93a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e4143100 con 0x7f95e4147270 2022-06-17T12:53:57.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.560+0000 7f95eb93a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e4142e40 con 0x7f95e40b64f0 2022-06-17T12:53:57.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.560+0000 7f95e96d6700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95e40b4120 0x7f95e413e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:57.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.560+0000 7f95e96d6700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95e40b4120 0x7f95e413e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35876/0 (socket says 172.21.15.36:35876) 2022-06-17T12:53:57.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.560+0000 7f95e96d6700 1 -- 172.21.15.36:0/1456430583 learned_addr learned my addr 172.21.15.36:0/1456430583 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:57.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.560+0000 7f95e8ed5700 1 --2- 172.21.15.36:0/1456430583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95e40b64f0 0x7f95e40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:57.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.560+0000 7f95e9ed7700 1 --2- 172.21.15.36:0/1456430583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95e4147270 0x7f95e40b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:57.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.560+0000 7f95e96d6700 1 -- 172.21.15.36:0/1456430583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95e40b64f0 msgr2=0x7f95e40b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:57.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.560+0000 7f95e96d6700 1 --2- 172.21.15.36:0/1456430583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95e40b64f0 0x7f95e40b4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.560+0000 7f95e96d6700 1 -- 172.21.15.36:0/1456430583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95e4147270 msgr2=0x7f95e40b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:57.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.560+0000 7f95e96d6700 1 --2- 172.21.15.36:0/1456430583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95e4147270 0x7f95e40b2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.560+0000 7f95e96d6700 1 -- 172.21.15.36:0/1456430583 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95dc009530 con 0x7f95e40b4120 2022-06-17T12:53:57.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.560+0000 7f95e96d6700 1 --2- 172.21.15.36:0/1456430583 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95e40b4120 0x7f95e413e860 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f95e40ae4b0 tx=0x7f95e0007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:57.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.560+0000 7f95da7fc700 1 -- 172.21.15.36:0/1456430583 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95e0012400 con 0x7f95e40b4120 2022-06-17T12:53:57.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.560+0000 7f95da7fc700 1 -- 172.21.15.36:0/1456430583 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e0012560 con 0x7f95e40b4120 2022-06-17T12:53:57.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.560+0000 7f95da7fc700 1 -- 172.21.15.36:0/1456430583 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95e00128c0 con 0x7f95e40b4120 2022-06-17T12:53:57.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.560+0000 7f95eb93a700 1 -- 172.21.15.36:0/1456430583 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e41408a0 con 0x7f95e40b4120 2022-06-17T12:53:57.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.560+0000 7f95eb93a700 1 -- 172.21.15.36:0/1456430583 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e40b1e00 con 0x7f95e40b4120 2022-06-17T12:53:57.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.560+0000 7f95eb93a700 1 -- 172.21.15.36:0/1456430583 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e40b35b0 con 0x7f95e40b4120 2022-06-17T12:53:57.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.564+0000 7f95da7fc700 1 -- 172.21.15.36:0/1456430583 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f95e0019030 con 0x7f95e40b4120 2022-06-17T12:53:57.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.564+0000 7f95da7fc700 1 --2- 172.21.15.36:0/1456430583 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f95cc0539b0 0x7f95cc055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:57.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.564+0000 7f95da7fc700 1 -- 172.21.15.36:0/1456430583 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f95e006b3e0 con 0x7f95e40b4120 2022-06-17T12:53:57.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.564+0000 7f95e8ed5700 1 --2- 172.21.15.36:0/1456430583 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f95cc0539b0 0x7f95cc055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:57.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.564+0000 7f95e8ed5700 1 --2- 172.21.15.36:0/1456430583 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f95cc0539b0 0x7f95cc055e70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f95d400b000 tx=0x7f95d4006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:57.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.568+0000 7f95da7fc700 1 -- 172.21.15.36:0/1456430583 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f95e0042160 con 0x7f95e40b4120 2022-06-17T12:53:57.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.696+0000 7f95eb93a700 1 -- 172.21.15.36:0/1456430583 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f95e40b3a60 con 0x7f95e40b4120 2022-06-17T12:53:57.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.696+0000 7f95da7fc700 1 -- 172.21.15.36:0/1456430583 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v75) v1 ==== 110+0+10 (secure 0 0 0) 0x7f95e0045780 con 0x7f95e40b4120 2022-06-17T12:53:57.685 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:53:57.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.700+0000 7f95eb93a700 1 -- 172.21.15.36:0/1456430583 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f95cc0539b0 msgr2=0x7f95cc055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:57.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.700+0000 7f95eb93a700 1 --2- 172.21.15.36:0/1456430583 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f95cc0539b0 0x7f95cc055e70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f95d400b000 tx=0x7f95d4006df0).stop 2022-06-17T12:53:57.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.700+0000 7f95eb93a700 1 -- 172.21.15.36:0/1456430583 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95e40b4120 msgr2=0x7f95e413e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:57.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.700+0000 7f95eb93a700 1 --2- 172.21.15.36:0/1456430583 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95e40b4120 0x7f95e413e860 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f95e40ae4b0 tx=0x7f95e0007c50).stop 2022-06-17T12:53:57.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.700+0000 7f95eb93a700 1 -- 172.21.15.36:0/1456430583 shutdown_connections 2022-06-17T12:53:57.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.700+0000 7f95eb93a700 1 --2- 172.21.15.36:0/1456430583 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f95cc0539b0 0x7f95cc055e70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.700+0000 7f95eb93a700 1 --2- 172.21.15.36:0/1456430583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95e4147270 0x7f95e40b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.700+0000 7f95eb93a700 1 --2- 172.21.15.36:0/1456430583 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95e40b64f0 0x7f95e40b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.700+0000 7f95eb93a700 1 --2- 172.21.15.36:0/1456430583 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95e40b4120 0x7f95e413e860 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.700+0000 7f95eb93a700 1 -- 172.21.15.36:0/1456430583 >> 172.21.15.36:0/1456430583 conn(0x7f95e40620f0 msgr2=0x7f95e40b7680 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:57.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.700+0000 7f95eb93a700 1 -- 172.21.15.36:0/1456430583 shutdown_connections 2022-06-17T12:53:57.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.700+0000 7f95eb93a700 1 -- 172.21.15.36:0/1456430583 wait complete. 2022-06-17T12:53:57.703 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:53:57.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.820+0000 7fc965331700 1 Processor -- start 2022-06-17T12:53:57.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.820+0000 7fc965331700 1 -- start start 2022-06-17T12:53:57.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.820+0000 7fc965331700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc960060020 0x7fc9600626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:57.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.820+0000 7fc965331700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9600b3f80 0x7fc9600b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:57.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.820+0000 7fc965331700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9601432c0 con 0x7fc9600b4b90 2022-06-17T12:53:57.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.820+0000 7fc965331700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc960143df0 con 0x7fc960060020 2022-06-17T12:53:57.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.820+0000 7fc965331700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc960061f60 con 0x7fc9600b3f80 2022-06-17T12:53:57.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.820+0000 7fc964b30700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fc9600b4b90 0x7fc96013ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53416/0 (socket says 172.21.15.36:53416) 2022-06-17T12:53:57.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.820+0000 7fc964b30700 1 -- 172.21.15.36:0/558843117 learned_addr learned my addr 172.21.15.36:0/558843117 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:53:57.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.820+0000 7fc95f7fe700 1 --2- 172.21.15.36:0/558843117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9600b3f80 0x7fc9600b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:57.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.820+0000 7fc95ffff700 1 --2- 172.21.15.36:0/558843117 >> v2:172.21.15.70:3300/0 conn(0x7fc960060020 0x7fc9600626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:57.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.820+0000 7fc95effd700 1 -- 172.21.15.36:0/558843117 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3391401651 0 0) 0x7fc9601432c0 con 0x7fc9600b4b90 2022-06-17T12:53:57.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.820+0000 7fc95effd700 1 -- 172.21.15.36:0/558843117 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc94c003260 con 0x7fc9600b4b90 2022-06-17T12:53:57.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.820+0000 7fc95f7fe700 1 -- 172.21.15.36:0/558843117 >> v2:172.21.15.70:3300/0 conn(0x7fc960060020 msgr2=0x7fc9600626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:57.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.820+0000 7fc95f7fe700 1 --2- 172.21.15.36:0/558843117 >> v2:172.21.15.70:3300/0 conn(0x7fc960060020 0x7fc9600626a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.820+0000 7fc95f7fe700 1 -- 172.21.15.36:0/558843117 >> v1:172.21.15.36:6789/0 conn(0x7fc9600b4b90 legacy=0x7fc96013ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:57.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.820+0000 7fc95f7fe700 1 -- 172.21.15.36:0/558843117 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc960062200 con 0x7fc9600b3f80 2022-06-17T12:53:57.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.820+0000 7fc95f7fe700 1 --2- 172.21.15.36:0/558843117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9600b3f80 0x7fc9600b46a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fc954002f70 tx=0x7fc954006df0).ready entity=mon.2 client_cookie=24974022f98b1bf3 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:57.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.820+0000 7fc95effd700 1 -- 172.21.15.36:0/558843117 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc95400d4a0 con 0x7fc9600b3f80 2022-06-17T12:53:57.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.820+0000 7fc95effd700 1 -- 172.21.15.36:0/558843117 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc954000e90 con 0x7fc9600b3f80 2022-06-17T12:53:57.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.824+0000 7fc95effd700 1 -- 172.21.15.36:0/558843117 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc95400d7f0 con 0x7fc9600b3f80 2022-06-17T12:53:57.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.824+0000 7fc965331700 1 -- 172.21.15.36:0/558843117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9600b3f80 msgr2=0x7fc9600b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:57.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.824+0000 7fc965331700 1 --2- 172.21.15.36:0/558843117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9600b3f80 0x7fc9600b46a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fc954002f70 tx=0x7fc954006df0).stop 2022-06-17T12:53:57.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.824+0000 7fc965331700 1 -- 172.21.15.36:0/558843117 shutdown_connections 2022-06-17T12:53:57.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.824+0000 7fc965331700 1 --2- 172.21.15.36:0/558843117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9600b3f80 0x7fc9600b46a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.824+0000 7fc965331700 1 --2- 172.21.15.36:0/558843117 >> v2:172.21.15.70:3300/0 conn(0x7fc960060020 0x7fc9600626a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.824+0000 7fc965331700 1 -- 172.21.15.36:0/558843117 >> 172.21.15.36:0/558843117 conn(0x7fc96005c550 msgr2=0x7fc96005f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:57.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.824+0000 7fc965331700 1 -- 172.21.15.36:0/558843117 shutdown_connections 2022-06-17T12:53:57.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.824+0000 7fc965331700 1 -- 172.21.15.36:0/558843117 wait complete. 2022-06-17T12:53:57.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.824+0000 7fc965331700 1 Processor -- start 2022-06-17T12:53:57.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.824+0000 7fc965331700 1 -- start start 2022-06-17T12:53:57.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.824+0000 7fc965331700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc960060020 0x7fc9601496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:57.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.824+0000 7fc965331700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc9600b3f80 0x7fc960146b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:57.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.824+0000 7fc965331700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9600b4b90 0x7fc96013eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:57.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.824+0000 7fc965331700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9600ad6c0 con 0x7fc9600b3f80 2022-06-17T12:53:57.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.824+0000 7fc965331700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9600ad960 con 0x7fc9600b4b90 2022-06-17T12:53:57.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.824+0000 7fc965331700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc960149b90 con 0x7fc960060020 2022-06-17T12:53:57.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.828+0000 7fc95f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc9600b3f80 0x7fc960146b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:57.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.828+0000 7fc95f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc9600b3f80 0x7fc960146b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35892/0 (socket says 172.21.15.36:35892) 2022-06-17T12:53:57.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.828+0000 7fc95f7fe700 1 -- 172.21.15.36:0/4279062932 learned_addr learned my addr 172.21.15.36:0/4279062932 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:57.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.828+0000 7fc964b30700 1 --2- 172.21.15.36:0/4279062932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9600b4b90 0x7fc96013eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:57.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.828+0000 7fc95ffff700 1 --2- 172.21.15.36:0/4279062932 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc960060020 0x7fc9601496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:57.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.828+0000 7fc95f7fe700 1 -- 172.21.15.36:0/4279062932 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc960060020 msgr2=0x7fc9601496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:57.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.828+0000 7fc95f7fe700 1 --2- 172.21.15.36:0/4279062932 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc960060020 0x7fc9601496a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.828+0000 7fc95f7fe700 1 -- 172.21.15.36:0/4279062932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9600b4b90 msgr2=0x7fc96013eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:57.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.828+0000 7fc95f7fe700 1 --2- 172.21.15.36:0/4279062932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9600b4b90 0x7fc96013eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.828+0000 7fc95f7fe700 1 -- 172.21.15.36:0/4279062932 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95400b180 con 0x7fc9600b3f80 2022-06-17T12:53:57.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.828+0000 7fc95f7fe700 1 --2- 172.21.15.36:0/4279062932 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc9600b3f80 0x7fc960146b60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7fc954002f70 tx=0x7fc954014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:57.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.828+0000 7fc95d7fa700 1 -- 172.21.15.36:0/4279062932 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc954016070 con 0x7fc9600b3f80 2022-06-17T12:53:57.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.828+0000 7fc95d7fa700 1 -- 172.21.15.36:0/4279062932 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9540138d0 con 0x7fc9600b3f80 2022-06-17T12:53:57.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.828+0000 7fc95d7fa700 1 -- 172.21.15.36:0/4279062932 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc954013c30 con 0x7fc9600b3f80 2022-06-17T12:53:57.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.828+0000 7fc965331700 1 -- 172.21.15.36:0/4279062932 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc960147050 con 0x7fc9600b3f80 2022-06-17T12:53:57.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.828+0000 7fc965331700 1 -- 172.21.15.36:0/4279062932 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96014a600 con 0x7fc9600b3f80 2022-06-17T12:53:57.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.828+0000 7fc965331700 1 -- 172.21.15.36:0/4279062932 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc960149d50 con 0x7fc9600b3f80 2022-06-17T12:53:57.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.828+0000 7fc95d7fa700 1 -- 172.21.15.36:0/4279062932 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fc95400f030 con 0x7fc9600b3f80 2022-06-17T12:53:57.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.832+0000 7fc95d7fa700 1 --2- 172.21.15.36:0/4279062932 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc940053ad0 0x7fc940055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:57.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.832+0000 7fc95d7fa700 1 -- 172.21.15.36:0/4279062932 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6463+0+0 (secure 0 0 0) 0x7fc95406ccb0 con 0x7fc9600b3f80 2022-06-17T12:53:57.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.832+0000 7fc95ffff700 1 --2- 172.21.15.36:0/4279062932 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc940053ad0 0x7fc940055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:57.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.832+0000 7fc95ffff700 1 --2- 172.21.15.36:0/4279062932 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc940053ad0 0x7fc940055f90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7fc95000b000 tx=0x7fc950006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:57.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.836+0000 7fc95d7fa700 1 -- 172.21.15.36:0/4279062932 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc954043a90 con 0x7fc9600b3f80 2022-06-17T12:53:57.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.964+0000 7fc965331700 1 -- 172.21.15.36:0/4279062932 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc9600b70b0 con 0x7fc9600b3f80 2022-06-17T12:53:57.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.968+0000 7fc95d7fa700 1 -- 172.21.15.36:0/4279062932 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v75) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc954023070 con 0x7fc9600b3f80 2022-06-17T12:53:57.956 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:53:57.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.968+0000 7fc965331700 1 -- 172.21.15.36:0/4279062932 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc940053ad0 msgr2=0x7fc940055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:57.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.968+0000 7fc965331700 1 --2- 172.21.15.36:0/4279062932 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc940053ad0 0x7fc940055f90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7fc95000b000 tx=0x7fc950006df0).stop 2022-06-17T12:53:57.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.968+0000 7fc965331700 1 -- 172.21.15.36:0/4279062932 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc9600b3f80 msgr2=0x7fc960146b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:57.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.968+0000 7fc965331700 1 --2- 172.21.15.36:0/4279062932 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc9600b3f80 0x7fc960146b60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7fc954002f70 tx=0x7fc954014000).stop 2022-06-17T12:53:57.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.968+0000 7fc965331700 1 -- 172.21.15.36:0/4279062932 shutdown_connections 2022-06-17T12:53:57.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.968+0000 7fc965331700 1 --2- 172.21.15.36:0/4279062932 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc940053ad0 0x7fc940055f90 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.968+0000 7fc965331700 1 --2- 172.21.15.36:0/4279062932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9600b4b90 0x7fc96013eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.968+0000 7fc965331700 1 --2- 172.21.15.36:0/4279062932 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc9600b3f80 0x7fc960146b60 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.968+0000 7fc965331700 1 --2- 172.21.15.36:0/4279062932 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc960060020 0x7fc9601496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:57.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.968+0000 7fc965331700 1 -- 172.21.15.36:0/4279062932 >> 172.21.15.36:0/4279062932 conn(0x7fc96005c550 msgr2=0x7fc9600b64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:57.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.968+0000 7fc965331700 1 -- 172.21.15.36:0/4279062932 shutdown_connections 2022-06-17T12:53:57.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:57.968+0000 7fc965331700 1 -- 172.21.15.36:0/4279062932 wait complete. 2022-06-17T12:53:57.972 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:53:58.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.092+0000 7f9ffffff700 1 Processor -- start 2022-06-17T12:53:58.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffffff700 1 -- start start 2022-06-17T12:53:58.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0000b27b0 0x7fa00014b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:58.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffffff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa000147bd0 0x7fa0001442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:58.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffffff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa00014b870 con 0x7fa000060ca0 2022-06-17T12:53:58.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffffff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0000ac500 con 0x7fa000147bd0 2022-06-17T12:53:58.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffffff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0000ace90 con 0x7fa0000b27b0 2022-06-17T12:53:58.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffe7fc700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fa000060ca0 0x7fa0000ac780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53428/0 (socket says 172.21.15.36:53428) 2022-06-17T12:53:58.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffe7fc700 1 -- 172.21.15.36:0/1870999846 learned_addr learned my addr 172.21.15.36:0/1870999846 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:53:58.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9fff7fe700 1 --2- 172.21.15.36:0/1870999846 >> v2:172.21.15.70:3300/0 conn(0x7fa000147bd0 0x7fa0001442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:58.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffeffd700 1 --2- 172.21.15.36:0/1870999846 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0000b27b0 0x7fa00014b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:58.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffdffb700 1 -- 172.21.15.36:0/1870999846 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 295773204 0 0) 0x7fa00014b870 con 0x7fa000060ca0 2022-06-17T12:53:58.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffdffb700 1 -- 172.21.15.36:0/1870999846 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fe4003260 con 0x7fa000060ca0 2022-06-17T12:53:58.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffdffb700 1 -- 172.21.15.36:0/1870999846 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2152267792 0 0) 0x7f9fe4003260 con 0x7fa000060ca0 2022-06-17T12:53:58.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffdffb700 1 -- 172.21.15.36:0/1870999846 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0000b27b0 msgr2=0x7fa00014b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:58.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffdffb700 1 --2- 172.21.15.36:0/1870999846 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0000b27b0 0x7fa00014b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffdffb700 1 -- 172.21.15.36:0/1870999846 >> v2:172.21.15.70:3300/0 conn(0x7fa000147bd0 msgr2=0x7fa0001442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:58.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffdffb700 1 --2- 172.21.15.36:0/1870999846 >> v2:172.21.15.70:3300/0 conn(0x7fa000147bd0 0x7fa0001442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffdffb700 1 -- 172.21.15.36:0/1870999846 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0000b2b90 con 0x7fa000060ca0 2022-06-17T12:53:58.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffdffb700 1 -- 172.21.15.36:0/1870999846 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f9ff0001680 con 0x7fa000060ca0 2022-06-17T12:53:58.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffdffb700 1 -- 172.21.15.36:0/1870999846 >> v1:172.21.15.36:6789/0 conn(0x7fa000060ca0 legacy=0x7fa0000ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:58.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffdffb700 1 --2- 172.21.15.36:0/1870999846 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0000b27b0 0x7f9fe4003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:58.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffdffb700 1 --2- 172.21.15.36:0/1870999846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fe4003c80 0x7f9fe4004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:58.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffdffb700 1 --2- 172.21.15.36:0/1870999846 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9fe40049f0 0x7f9fe4004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:58.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffdffb700 1 -- 172.21.15.36:0/1870999846 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0000ac500 con 0x7f9fe40049f0 2022-06-17T12:53:58.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffdffb700 1 -- 172.21.15.36:0/1870999846 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0000ace90 con 0x7f9fe4003c80 2022-06-17T12:53:58.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffdffb700 1 -- 172.21.15.36:0/1870999846 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe400b750 con 0x7fa0000b27b0 2022-06-17T12:53:58.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9fff7fe700 1 --2- 172.21.15.36:0/1870999846 >> v2:172.21.15.70:3300/0 conn(0x7fa000147bd0 0x7fa0001442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:53:58.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9fff7fe700 1 --2- 172.21.15.36:0/1870999846 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9fe40049f0 0x7f9fe4004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:58.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffe7fc700 1 --2- 172.21.15.36:0/1870999846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fe4003c80 0x7f9fe4004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:58.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffeffd700 1 --2- 172.21.15.36:0/1870999846 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0000b27b0 0x7f9fe4003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:58.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9fff7fe700 1 -- 172.21.15.36:0/1870999846 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0000b27b0 msgr2=0x7f9fe4003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:58.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9fff7fe700 1 --2- 172.21.15.36:0/1870999846 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0000b27b0 0x7f9fe4003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9fff7fe700 1 -- 172.21.15.36:0/1870999846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fe4003c80 msgr2=0x7f9fe4004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:58.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9fff7fe700 1 --2- 172.21.15.36:0/1870999846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fe4003c80 0x7f9fe4004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9fff7fe700 1 -- 172.21.15.36:0/1870999846 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe4003260 con 0x7f9fe40049f0 2022-06-17T12:53:58.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffeffd700 1 --2- 172.21.15.36:0/1870999846 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0000b27b0 0x7f9fe4003750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:53:58.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9fff7fe700 1 --2- 172.21.15.36:0/1870999846 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9fe40049f0 0x7f9fe4004e70 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f9fe40054d0 tx=0x7f9ff80091d0).ready entity=mon.0 client_cookie=b39ad5ba2622bb2f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:58.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffdffb700 1 -- 172.21.15.36:0/1870999846 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ff800e920 con 0x7f9fe40049f0 2022-06-17T12:53:58.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffdffb700 1 -- 172.21.15.36:0/1870999846 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff800ea80 con 0x7f9fe40049f0 2022-06-17T12:53:58.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffdffb700 1 -- 172.21.15.36:0/1870999846 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ff800ede0 con 0x7f9fe40049f0 2022-06-17T12:53:58.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffffff700 1 -- 172.21.15.36:0/1870999846 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9fe40049f0 msgr2=0x7f9fe4004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:58.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffffff700 1 --2- 172.21.15.36:0/1870999846 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9fe40049f0 0x7f9fe4004e70 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f9fe40054d0 tx=0x7f9ff80091d0).stop 2022-06-17T12:53:58.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9fff7fe700 1 -- 172.21.15.36:0/1870999846 reap_dead start 2022-06-17T12:53:58.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.096+0000 7f9ffffff700 1 -- 172.21.15.36:0/1870999846 shutdown_connections 2022-06-17T12:53:58.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9ffffff700 1 -- 172.21.15.36:0/1870999846 >> 172.21.15.36:0/1870999846 conn(0x7fa0000b5d10 msgr2=0x7fa0000b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:58.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9ffffff700 1 -- 172.21.15.36:0/1870999846 shutdown_connections 2022-06-17T12:53:58.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9ffffff700 1 -- 172.21.15.36:0/1870999846 wait complete. 2022-06-17T12:53:58.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9ffffff700 1 Processor -- start 2022-06-17T12:53:58.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9ffffff700 1 -- start start 2022-06-17T12:53:58.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9ffffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0000a9d90 0x7fa0000aa170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:58.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9ffffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0000668b0 0x7fa000156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:58.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9ffffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa000066d10 0x7fa000156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:58.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9ffffff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0000aa740 con 0x7fa000066d10 2022-06-17T12:53:58.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9ffffff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0000ab8b0 con 0x7fa0000a9d90 2022-06-17T12:53:58.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9ffffff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa000065a10 con 0x7fa0000668b0 2022-06-17T12:53:58.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9fff7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa000066d10 0x7fa000156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:58.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9ffeffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0000a9d90 0x7fa0000aa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:58.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9ffe7fc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0000668b0 0x7fa000156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:58.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9ffeffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0000a9d90 0x7fa0000aa170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:56292/0 (socket says 172.21.15.36:56292) 2022-06-17T12:53:58.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9ffeffd700 1 -- 172.21.15.36:0/3432416343 learned_addr learned my addr 172.21.15.36:0/3432416343 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:58.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9fff7fe700 1 -- 172.21.15.36:0/3432416343 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0000668b0 msgr2=0x7fa000156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:58.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9fff7fe700 1 --2- 172.21.15.36:0/3432416343 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0000668b0 0x7fa000156770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9fff7fe700 1 -- 172.21.15.36:0/3432416343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0000a9d90 msgr2=0x7fa0000aa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:58.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9fff7fe700 1 --2- 172.21.15.36:0/3432416343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0000a9d90 0x7fa0000aa170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9fff7fe700 1 -- 172.21.15.36:0/3432416343 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0000664f0 con 0x7fa000066d10 2022-06-17T12:53:58.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9fff7fe700 1 --2- 172.21.15.36:0/3432416343 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa000066d10 0x7fa000156c60 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f9ff00017e0 tx=0x7f9ff801a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:58.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9feffff700 1 -- 172.21.15.36:0/3432416343 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ff8006ea0 con 0x7fa000066d10 2022-06-17T12:53:58.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9feffff700 1 -- 172.21.15.36:0/3432416343 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff801b850 con 0x7fa000066d10 2022-06-17T12:53:58.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9ffffff700 1 -- 172.21.15.36:0/3432416343 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa000066680 con 0x7fa000066d10 2022-06-17T12:53:58.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9feffff700 1 -- 172.21.15.36:0/3432416343 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ff800e920 con 0x7fa000066d10 2022-06-17T12:53:58.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.100+0000 7f9ffffff700 1 -- 172.21.15.36:0/3432416343 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0000a7300 con 0x7fa000066d10 2022-06-17T12:53:58.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.104+0000 7f9ffffff700 1 -- 172.21.15.36:0/3432416343 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa000057410 con 0x7fa000066d10 2022-06-17T12:53:58.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.104+0000 7f9feffff700 1 -- 172.21.15.36:0/3432416343 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f9ff801bbb0 con 0x7fa000066d10 2022-06-17T12:53:58.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.104+0000 7f9feffff700 1 --2- 172.21.15.36:0/3432416343 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9fdc053940 0x7f9fdc055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:58.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.104+0000 7f9feffff700 1 -- 172.21.15.36:0/3432416343 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f9ff806d940 con 0x7fa000066d10 2022-06-17T12:53:58.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.104+0000 7f9ffeffd700 1 --2- 172.21.15.36:0/3432416343 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9fdc053940 0x7f9fdc055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:58.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.104+0000 7f9ffeffd700 1 --2- 172.21.15.36:0/3432416343 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9fdc053940 0x7f9fdc055e00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f9ff400b000 tx=0x7f9ff4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:58.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.112+0000 7f9feffff700 1 -- 172.21.15.36:0/3432416343 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9ff8044610 con 0x7fa000066d10 2022-06-17T12:53:58.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.240+0000 7f9ffffff700 1 -- 172.21.15.36:0/3432416343 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa0000aba20 con 0x7fa000066d10 2022-06-17T12:53:58.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.240+0000 7f9feffff700 1 -- 172.21.15.36:0/3432416343 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v75) v1 ==== 104+0+10 (secure 0 0 0) 0x7f9ff8047c30 con 0x7fa000066d10 2022-06-17T12:53:58.229 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:53:58.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.240+0000 7f9ffffff700 1 -- 172.21.15.36:0/3432416343 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9fdc053940 msgr2=0x7f9fdc055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:58.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.240+0000 7f9ffffff700 1 --2- 172.21.15.36:0/3432416343 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9fdc053940 0x7f9fdc055e00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f9ff400b000 tx=0x7f9ff4008040).stop 2022-06-17T12:53:58.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.240+0000 7f9ffffff700 1 -- 172.21.15.36:0/3432416343 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa000066d10 msgr2=0x7fa000156c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:58.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.240+0000 7f9ffffff700 1 --2- 172.21.15.36:0/3432416343 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa000066d10 0x7fa000156c60 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f9ff00017e0 tx=0x7f9ff801a000).stop 2022-06-17T12:53:58.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.244+0000 7f9ffffff700 1 -- 172.21.15.36:0/3432416343 shutdown_connections 2022-06-17T12:53:58.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.244+0000 7f9ffffff700 1 --2- 172.21.15.36:0/3432416343 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9fdc053940 0x7f9fdc055e00 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.244+0000 7f9ffffff700 1 --2- 172.21.15.36:0/3432416343 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa000066d10 0x7fa000156c60 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.244+0000 7f9ffffff700 1 --2- 172.21.15.36:0/3432416343 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0000668b0 0x7fa000156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.244+0000 7f9ffffff700 1 --2- 172.21.15.36:0/3432416343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0000a9d90 0x7fa0000aa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.244+0000 7f9ffffff700 1 -- 172.21.15.36:0/3432416343 >> 172.21.15.36:0/3432416343 conn(0x7fa0000b5d10 msgr2=0x7fa0000b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:58.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.244+0000 7f9ffffff700 1 -- 172.21.15.36:0/3432416343 shutdown_connections 2022-06-17T12:53:58.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.244+0000 7f9ffffff700 1 -- 172.21.15.36:0/3432416343 wait complete. 2022-06-17T12:53:58.241 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:53:58.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.364+0000 7f133a40f700 1 Processor -- start 2022-06-17T12:53:58.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.364+0000 7f133a40f700 1 -- start start 2022-06-17T12:53:58.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.364+0000 7f133a40f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f13340b64f0 0x7f1334143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:58.366 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.364+0000 7f133a40f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13340b4120 0x7f1334146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:58.366 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.364+0000 7f133a40f700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1334147270 0x7f1334140100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:58.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.364+0000 7f133a40f700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13340b68d0 con 0x7f13340b64f0 2022-06-17T12:53:58.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.364+0000 7f133a40f700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13341476b0 con 0x7f1334147270 2022-06-17T12:53:58.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.364+0000 7f133a40f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13340b6e20 con 0x7f13340b4120 2022-06-17T12:53:58.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.364+0000 7f1333fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f13340b64f0 0x7f1334143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:58.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.364+0000 7f13389ac700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1334147270 0x7f1334140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:58.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.364+0000 7f13389ac700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1334147270 0x7f1334140100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:56304/0 (socket says 172.21.15.36:56304) 2022-06-17T12:53:58.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.364+0000 7f13337fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13340b4120 0x7f1334146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:58.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.364+0000 7f13389ac700 1 -- 172.21.15.36:0/238886902 learned_addr learned my addr 172.21.15.36:0/238886902 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:58.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.364+0000 7f13389ac700 1 -- 172.21.15.36:0/238886902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13340b4120 msgr2=0x7f1334146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:58.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.364+0000 7f13389ac700 1 --2- 172.21.15.36:0/238886902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13340b4120 0x7f1334146d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.364+0000 7f13389ac700 1 -- 172.21.15.36:0/238886902 >> v2:172.21.15.36:3300/0 conn(0x7f13340b64f0 msgr2=0x7f1334143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:58.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.364+0000 7f13389ac700 1 --2- 172.21.15.36:0/238886902 >> v2:172.21.15.36:3300/0 conn(0x7f13340b64f0 0x7f1334143150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.364+0000 7f13389ac700 1 -- 172.21.15.36:0/238886902 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13341468e0 con 0x7f1334147270 2022-06-17T12:53:58.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.364+0000 7f13389ac700 1 --2- 172.21.15.36:0/238886902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1334147270 0x7f1334140100 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f132c0032f0 tx=0x7f132c009290).ready entity=mon.1 client_cookie=22e5d7d033e07938 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:58.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.368+0000 7f1332ffd700 1 -- 172.21.15.36:0/238886902 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f132c00b680 con 0x7f1334147270 2022-06-17T12:53:58.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.368+0000 7f1332ffd700 1 -- 172.21.15.36:0/238886902 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132c00b7e0 con 0x7f1334147270 2022-06-17T12:53:58.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.368+0000 7f1332ffd700 1 -- 172.21.15.36:0/238886902 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f132c00b950 con 0x7f1334147270 2022-06-17T12:53:58.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.368+0000 7f133a40f700 1 -- 172.21.15.36:0/238886902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1334147270 msgr2=0x7f1334140100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:58.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.368+0000 7f133a40f700 1 --2- 172.21.15.36:0/238886902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1334147270 0x7f1334140100 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f132c0032f0 tx=0x7f132c009290).stop 2022-06-17T12:53:58.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.368+0000 7f133a40f700 1 -- 172.21.15.36:0/238886902 shutdown_connections 2022-06-17T12:53:58.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.368+0000 7f133a40f700 1 --2- 172.21.15.36:0/238886902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1334147270 0x7f1334140100 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.368+0000 7f133a40f700 1 --2- 172.21.15.36:0/238886902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13340b4120 0x7f1334146d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.368+0000 7f133a40f700 1 --2- 172.21.15.36:0/238886902 >> v2:172.21.15.36:3300/0 conn(0x7f13340b64f0 0x7f1334143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.368+0000 7f133a40f700 1 -- 172.21.15.36:0/238886902 >> 172.21.15.36:0/238886902 conn(0x7f13340620f0 msgr2=0x7f1334062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:58.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.368+0000 7f133a40f700 1 -- 172.21.15.36:0/238886902 shutdown_connections 2022-06-17T12:53:58.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.368+0000 7f133a40f700 1 -- 172.21.15.36:0/238886902 wait complete. 2022-06-17T12:53:58.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.368+0000 7f133a40f700 1 Processor -- start 2022-06-17T12:53:58.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.368+0000 7f133a40f700 1 -- start start 2022-06-17T12:53:58.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.372+0000 7f133a40f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13340b4120 0x7f1334140210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:58.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.372+0000 7f133a40f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f13340b64f0 0x7f13341416e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:58.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.372+0000 7f133a40f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1334147270 0x7f1334148970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:58.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.372+0000 7f133a40f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13340b1dc0 con 0x7f13340b64f0 2022-06-17T12:53:58.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.372+0000 7f133a40f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13340b2b90 con 0x7f13340b4120 2022-06-17T12:53:58.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.372+0000 7f133a40f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13340b1680 con 0x7f1334147270 2022-06-17T12:53:58.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.372+0000 7f13337fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f13340b64f0 0x7f13341416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:58.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.372+0000 7f13337fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f13340b64f0 0x7f13341416e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35926/0 (socket says 172.21.15.36:35926) 2022-06-17T12:53:58.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.372+0000 7f13337fe700 1 -- 172.21.15.36:0/2239122547 learned_addr learned my addr 172.21.15.36:0/2239122547 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:58.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.372+0000 7f1333fff700 1 --2- 172.21.15.36:0/2239122547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13340b4120 0x7f1334140210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:58.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.372+0000 7f13389ac700 1 --2- 172.21.15.36:0/2239122547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1334147270 0x7f1334148970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:58.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.372+0000 7f13337fe700 1 -- 172.21.15.36:0/2239122547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1334147270 msgr2=0x7f1334148970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:58.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.372+0000 7f13337fe700 1 --2- 172.21.15.36:0/2239122547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1334147270 0x7f1334148970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.372+0000 7f13337fe700 1 -- 172.21.15.36:0/2239122547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13340b4120 msgr2=0x7f1334140210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:58.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.372+0000 7f13337fe700 1 --2- 172.21.15.36:0/2239122547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13340b4120 0x7f1334140210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.372+0000 7f13337fe700 1 -- 172.21.15.36:0/2239122547 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f132c009530 con 0x7f13340b64f0 2022-06-17T12:53:58.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.372+0000 7f13337fe700 1 --2- 172.21.15.36:0/2239122547 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f13340b64f0 0x7f13341416e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7f132400c000 tx=0x7f1324006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:58.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.372+0000 7f13317fa700 1 -- 172.21.15.36:0/2239122547 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1324010780 con 0x7f13340b64f0 2022-06-17T12:53:58.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.372+0000 7f13317fa700 1 -- 172.21.15.36:0/2239122547 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13240108e0 con 0x7f13340b64f0 2022-06-17T12:53:58.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.372+0000 7f133a40f700 1 -- 172.21.15.36:0/2239122547 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13340b1920 con 0x7f13340b64f0 2022-06-17T12:53:58.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.372+0000 7f13317fa700 1 -- 172.21.15.36:0/2239122547 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1324010c40 con 0x7f13340b64f0 2022-06-17T12:53:58.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.372+0000 7f133a40f700 1 -- 172.21.15.36:0/2239122547 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13341431e0 con 0x7f13340b64f0 2022-06-17T12:53:58.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.372+0000 7f133a40f700 1 -- 172.21.15.36:0/2239122547 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13340b31d0 con 0x7f13340b64f0 2022-06-17T12:53:58.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.372+0000 7f13317fa700 1 -- 172.21.15.36:0/2239122547 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f132401c460 con 0x7f13340b64f0 2022-06-17T12:53:58.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.376+0000 7f13317fa700 1 --2- 172.21.15.36:0/2239122547 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1318053a40 0x7f1318055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:58.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.376+0000 7f13317fa700 1 -- 172.21.15.36:0/2239122547 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f132406c4f0 con 0x7f13340b64f0 2022-06-17T12:53:58.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.376+0000 7f1333fff700 1 --2- 172.21.15.36:0/2239122547 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1318053a40 0x7f1318055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:58.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.376+0000 7f1333fff700 1 --2- 172.21.15.36:0/2239122547 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1318053a40 0x7f1318055f00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f131c00b000 tx=0x7f131c006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:58.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.380+0000 7f13317fa700 1 -- 172.21.15.36:0/2239122547 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f13240430e0 con 0x7f13340b64f0 2022-06-17T12:53:58.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.504+0000 7f133a40f700 1 -- 172.21.15.36:0/2239122547 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f13340b1ab0 con 0x7f13340b64f0 2022-06-17T12:53:58.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.508+0000 7f13317fa700 1 -- 172.21.15.36:0/2239122547 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v75) v1 ==== 100+0+10 (secure 0 0 0) 0x7f1324046700 con 0x7f13340b64f0 2022-06-17T12:53:58.496 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:53:58.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.508+0000 7f133a40f700 1 -- 172.21.15.36:0/2239122547 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1318053a40 msgr2=0x7f1318055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:58.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.508+0000 7f133a40f700 1 --2- 172.21.15.36:0/2239122547 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1318053a40 0x7f1318055f00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f131c00b000 tx=0x7f131c006df0).stop 2022-06-17T12:53:58.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.508+0000 7f133a40f700 1 -- 172.21.15.36:0/2239122547 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f13340b64f0 msgr2=0x7f13341416e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:58.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.508+0000 7f133a40f700 1 --2- 172.21.15.36:0/2239122547 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f13340b64f0 0x7f13341416e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7f132400c000 tx=0x7f1324006df0).stop 2022-06-17T12:53:58.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.508+0000 7f133a40f700 1 -- 172.21.15.36:0/2239122547 shutdown_connections 2022-06-17T12:53:58.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.508+0000 7f133a40f700 1 --2- 172.21.15.36:0/2239122547 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1318053a40 0x7f1318055f00 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.508+0000 7f133a40f700 1 --2- 172.21.15.36:0/2239122547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1334147270 0x7f1334148970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.508+0000 7f133a40f700 1 --2- 172.21.15.36:0/2239122547 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f13340b64f0 0x7f13341416e0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.508+0000 7f133a40f700 1 --2- 172.21.15.36:0/2239122547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13340b4120 0x7f1334140210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.508+0000 7f133a40f700 1 -- 172.21.15.36:0/2239122547 >> 172.21.15.36:0/2239122547 conn(0x7f13340620f0 msgr2=0x7f13340b4840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:58.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.508+0000 7f133a40f700 1 -- 172.21.15.36:0/2239122547 shutdown_connections 2022-06-17T12:53:58.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.508+0000 7f133a40f700 1 -- 172.21.15.36:0/2239122547 wait complete. 2022-06-17T12:53:58.511 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:53:58.511 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:53:58.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bf59e700 1 Processor -- start 2022-06-17T12:53:58.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bf59e700 1 -- start start 2022-06-17T12:53:58.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bf59e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f33c0063730 0x7f33c00b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:58.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bf59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33c0064eb0 0x7f33c013ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:58.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bf59e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f33c01403f0 0x7f33c00b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:58.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bf59e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c0147430 con 0x7f33c01403f0 2022-06-17T12:53:58.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bf59e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c00b1ca0 con 0x7f33c0063730 2022-06-17T12:53:58.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bf59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c0144180 con 0x7f33c0064eb0 2022-06-17T12:53:58.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bed9d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f33c01403f0 0x7f33c00b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:58.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bed9d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f33c01403f0 0x7f33c00b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35936/0 (socket says 172.21.15.36:35936) 2022-06-17T12:53:58.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bed9d700 1 -- 172.21.15.36:0/898683169 learned_addr learned my addr 172.21.15.36:0/898683169 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:58.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33be59c700 1 --2- 172.21.15.36:0/898683169 >> v2:172.21.15.70:3300/0 conn(0x7f33c0063730 0x7f33c00b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:58.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bdd9b700 1 --2- 172.21.15.36:0/898683169 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33c0064eb0 0x7f33c013ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:58.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bed9d700 1 -- 172.21.15.36:0/898683169 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33c0064eb0 msgr2=0x7f33c013ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:58.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bed9d700 1 --2- 172.21.15.36:0/898683169 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33c0064eb0 0x7f33c013ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bed9d700 1 -- 172.21.15.36:0/898683169 >> v2:172.21.15.70:3300/0 conn(0x7f33c0063730 msgr2=0x7f33c00b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:53:58.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bed9d700 1 --2- 172.21.15.36:0/898683169 >> v2:172.21.15.70:3300/0 conn(0x7f33c0063730 0x7f33c00b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bed9d700 1 -- 172.21.15.36:0/898683169 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c0063b10 con 0x7f33c01403f0 2022-06-17T12:53:58.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bed9d700 1 --2- 172.21.15.36:0/898683169 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33c01403f0 0x7f33c00b33c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7f33c0142f30 tx=0x7f33b8009290).ready entity=mon.0 client_cookie=ddae05b0213a6a66 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:58.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bd59a700 1 -- 172.21.15.36:0/898683169 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33b800b830 con 0x7f33c01403f0 2022-06-17T12:53:58.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bd59a700 1 -- 172.21.15.36:0/898683169 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b800b990 con 0x7f33c01403f0 2022-06-17T12:53:58.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bd59a700 1 -- 172.21.15.36:0/898683169 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33b800bb00 con 0x7f33c01403f0 2022-06-17T12:53:58.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bf59e700 1 -- 172.21.15.36:0/898683169 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33c01403f0 msgr2=0x7f33c00b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:58.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bf59e700 1 --2- 172.21.15.36:0/898683169 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33c01403f0 0x7f33c00b33c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7f33c0142f30 tx=0x7f33b8009290).stop 2022-06-17T12:53:58.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bf59e700 1 -- 172.21.15.36:0/898683169 shutdown_connections 2022-06-17T12:53:58.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bf59e700 1 --2- 172.21.15.36:0/898683169 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33c01403f0 0x7f33c00b33c0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bf59e700 1 --2- 172.21.15.36:0/898683169 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33c0064eb0 0x7f33c013ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bf59e700 1 --2- 172.21.15.36:0/898683169 >> v2:172.21.15.70:3300/0 conn(0x7f33c0063730 0x7f33c00b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bf59e700 1 -- 172.21.15.36:0/898683169 >> 172.21.15.36:0/898683169 conn(0x7f33c00614b0 msgr2=0x7f33c00618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:58.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bf59e700 1 -- 172.21.15.36:0/898683169 shutdown_connections 2022-06-17T12:53:58.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.628+0000 7f33bf59e700 1 -- 172.21.15.36:0/898683169 wait complete. 2022-06-17T12:53:58.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.632+0000 7f33bf59e700 1 Processor -- start 2022-06-17T12:53:58.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.632+0000 7f33bf59e700 1 -- start start 2022-06-17T12:53:58.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.632+0000 7f33bf59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33c0063730 0x7f33c0065590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:58.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.632+0000 7f33bf59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33c0064eb0 0x7f33c0066f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:58.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.632+0000 7f33bf59e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33c01403f0 0x7f33c01555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:58.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.632+0000 7f33bf59e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c0149970 con 0x7f33c0064eb0 2022-06-17T12:53:58.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.632+0000 7f33bf59e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c014a0e0 con 0x7f33c01403f0 2022-06-17T12:53:58.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.632+0000 7f33bf59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c0149ba0 con 0x7f33c0063730 2022-06-17T12:53:58.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.632+0000 7f33bdd9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33c0064eb0 0x7f33c0066f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:58.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.632+0000 7f33bdd9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33c0064eb0 0x7f33c0066f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35940/0 (socket says 172.21.15.36:35940) 2022-06-17T12:53:58.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.632+0000 7f33bdd9b700 1 -- 172.21.15.36:0/4144356893 learned_addr learned my addr 172.21.15.36:0/4144356893 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:53:58.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.636+0000 7f33be59c700 1 --2- 172.21.15.36:0/4144356893 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33c0063730 0x7f33c0065590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:58.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.636+0000 7f33bed9d700 1 --2- 172.21.15.36:0/4144356893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33c01403f0 0x7f33c01555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:58.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.636+0000 7f33bdd9b700 1 -- 172.21.15.36:0/4144356893 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33c0063730 msgr2=0x7f33c0065590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:58.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.636+0000 7f33bdd9b700 1 --2- 172.21.15.36:0/4144356893 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33c0063730 0x7f33c0065590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.636+0000 7f33bdd9b700 1 -- 172.21.15.36:0/4144356893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33c01403f0 msgr2=0x7f33c01555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:58.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.636+0000 7f33bdd9b700 1 --2- 172.21.15.36:0/4144356893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33c01403f0 0x7f33c01555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.636+0000 7f33bdd9b700 1 -- 172.21.15.36:0/4144356893 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b8009530 con 0x7f33c0064eb0 2022-06-17T12:53:58.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.636+0000 7f33bdd9b700 1 --2- 172.21.15.36:0/4144356893 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33c0064eb0 0x7f33c0066f00 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f33b40030b0 tx=0x7f33b4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:58.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.636+0000 7f33af7fe700 1 -- 172.21.15.36:0/4144356893 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33b4011780 con 0x7f33c0064eb0 2022-06-17T12:53:58.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.636+0000 7f33af7fe700 1 -- 172.21.15.36:0/4144356893 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b40118e0 con 0x7f33c0064eb0 2022-06-17T12:53:58.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.636+0000 7f33bf59e700 1 -- 172.21.15.36:0/4144356893 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c0149e40 con 0x7f33c0064eb0 2022-06-17T12:53:58.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.636+0000 7f33af7fe700 1 -- 172.21.15.36:0/4144356893 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33b4011c40 con 0x7f33c0064eb0 2022-06-17T12:53:58.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.636+0000 7f33bf59e700 1 -- 172.21.15.36:0/4144356893 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c0066650 con 0x7f33c0064eb0 2022-06-17T12:53:58.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.636+0000 7f33bf59e700 1 -- 172.21.15.36:0/4144356893 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c0057410 con 0x7f33c0064eb0 2022-06-17T12:53:58.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.636+0000 7f33af7fe700 1 -- 172.21.15.36:0/4144356893 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f33b401c460 con 0x7f33c0064eb0 2022-06-17T12:53:58.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.640+0000 7f33af7fe700 1 --2- 172.21.15.36:0/4144356893 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f339c053ad0 0x7f339c055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:53:58.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.640+0000 7f33af7fe700 1 -- 172.21.15.36:0/4144356893 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f33b406c210 con 0x7f33c0064eb0 2022-06-17T12:53:58.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.640+0000 7f33be59c700 1 --2- 172.21.15.36:0/4144356893 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f339c053ad0 0x7f339c055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:53:58.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.640+0000 7f33be59c700 1 --2- 172.21.15.36:0/4144356893 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f339c053ad0 0x7f339c055f90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f33b000c000 tx=0x7f33b0008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:53:58.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.644+0000 7f33af7fe700 1 -- 172.21.15.36:0/4144356893 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f33b4042e80 con 0x7f33c0064eb0 2022-06-17T12:53:58.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.804+0000 7f33bf59e700 1 -- 172.21.15.36:0/4144356893 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33c0142200 con 0x7f33c0064eb0 2022-06-17T12:53:58.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.808+0000 7f33af7fe700 1 -- 172.21.15.36:0/4144356893 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f33b40464a0 con 0x7f33c0064eb0 2022-06-17T12:53:58.795 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:53:58.795 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":36,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/2541998896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2541998896},{"type":"v1","addr":"172.21.15.70:6833","nonce":2541998896}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:53:04.907795+0000","modified":"2022-06-17T12:53:06.511529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6381":{"gid":6381,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/2514005995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2514005995},{"type":"v1","addr":"172.21.15.36:6835","nonce":2514005995}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-06-17T12:53:58.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.808+0000 7f33bf59e700 1 -- 172.21.15.36:0/4144356893 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f339c053ad0 msgr2=0x7f339c055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:58.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.808+0000 7f33bf59e700 1 --2- 172.21.15.36:0/4144356893 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f339c053ad0 0x7f339c055f90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f33b000c000 tx=0x7f33b0008040).stop 2022-06-17T12:53:58.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.808+0000 7f33bf59e700 1 -- 172.21.15.36:0/4144356893 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33c0064eb0 msgr2=0x7f33c0066f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:53:58.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.808+0000 7f33bf59e700 1 --2- 172.21.15.36:0/4144356893 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33c0064eb0 0x7f33c0066f00 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f33b40030b0 tx=0x7f33b4008040).stop 2022-06-17T12:53:58.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.808+0000 7f33bf59e700 1 -- 172.21.15.36:0/4144356893 shutdown_connections 2022-06-17T12:53:58.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.808+0000 7f33bf59e700 1 --2- 172.21.15.36:0/4144356893 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f339c053ad0 0x7f339c055f90 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.808+0000 7f33bf59e700 1 --2- 172.21.15.36:0/4144356893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33c01403f0 0x7f33c01555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.808+0000 7f33bf59e700 1 --2- 172.21.15.36:0/4144356893 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33c0064eb0 0x7f33c0066f00 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.808+0000 7f33bf59e700 1 --2- 172.21.15.36:0/4144356893 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33c0063730 0x7f33c0065590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:53:58.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.808+0000 7f33bf59e700 1 -- 172.21.15.36:0/4144356893 >> 172.21.15.36:0/4144356893 conn(0x7f33c00614b0 msgr2=0x7f33c0061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:53:58.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.808+0000 7f33bf59e700 1 -- 172.21.15.36:0/4144356893 shutdown_connections 2022-06-17T12:53:58.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:53:58.808+0000 7f33bf59e700 1 -- 172.21.15.36:0/4144356893 wait complete. 2022-06-17T12:53:58.800 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 36 2022-06-17T12:53:59.009 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:53:04.907795+0000', 'modified': '2022-06-17T12:53:06.511529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6381}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6381': {'gid': 6381, 'name': 'a', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.36:6835/2514005995', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 2514005995}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 2514005995}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:53:59.010 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:53:59.012 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:53:59.128 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":6381,"want_state":"up:active","state":"up:active","rank_uptime":54.224279523,"mdsmap_epoch":36,"osdmap_epoch":75,"osdmap_epoch_barrier":75,"uptime":58.619167230999999} 2022-06-17T12:53:59.129 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 6381, "mdsmap_epoch": 36, "osdmap_epoch": 75, "osdmap_epoch_barrier": 75, "rank_uptime": 54.224279523, "state": "up:active", "uptime": 58.619167231, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:53:59.132 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:53:59.133 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-17T12:53:59.133 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi107.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-17T12:53:59.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:59.134 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:53:59.143 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:53:59.201 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:53:59.201 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:53:59.215 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:53:59.216 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:53:59.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:59.218 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:53:59.229 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:53:59.230 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:53:59.235 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:53:59.277 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:53:59.280 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:53:59.313 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:2022-06-17T12:53:59.323+0000 7febe9f91080 -1 init, newargv = 0x564e90508690 newargc=9ceph-fuse[22435]: starting ceph client 2022-06-17T12:53:59.313 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr: 2022-06-17T12:53:59.320 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22435]: starting fuse 2022-06-17T12:53:59.328 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:53:59.329 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:53:59.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:59.331 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:53:59.378 INFO:teuthology.orchestra.run.smithi107.stdout:52 2022-06-17T12:53:59.379 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:53:59.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:59.380 DEBUG:teuthology.orchestra.run.smithi107:> sudo python3 -c ' 2022-06-17T12:53:59.380 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2022-06-17T12:53:59.381 DEBUG:teuthology.orchestra.run.smithi107:> import re 2022-06-17T12:53:59.381 DEBUG:teuthology.orchestra.run.smithi107:> import os 2022-06-17T12:53:59.382 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2022-06-17T12:53:59.382 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:53:59.383 DEBUG:teuthology.orchestra.run.smithi107:> def find_socket(client_name): 2022-06-17T12:53:59.383 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-17T12:53:59.384 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2022-06-17T12:53:59.386 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:53:59.386 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2022-06-17T12:53:59.387 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2022-06-17T12:53:59.387 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2022-06-17T12:53:59.388 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2022-06-17T12:53:59.388 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:53:59.389 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2022-06-17T12:53:59.389 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:53:59.389 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:53:59.390 DEBUG:teuthology.orchestra.run.smithi107:> return f 2022-06-17T12:53:59.390 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:53:59.391 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:53:59.391 DEBUG:teuthology.orchestra.run.smithi107:> print(find_socket("client.0")) 2022-06-17T12:53:59.392 DEBUG:teuthology.orchestra.run.smithi107:> ' 2022-06-17T12:53:59.438 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.0.22435.asok 2022-06-17T12:53:59.444 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22435.asok 2022-06-17T12:53:59.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:59.445 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22435.asok status 2022-06-17T12:53:59.529 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2022-06-17T12:53:59.529 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2022-06-17T12:53:59.530 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:53:59.530 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:53:59.530 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "0", 2022-06-17T12:53:59.530 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2022-06-17T12:53:59.531 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-17T12:53:59.532 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "22435", 2022-06-17T12:53:59.532 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/" 2022-06-17T12:53:59.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:53:59.533 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2022-06-17T12:53:59.533 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2022-06-17T12:53:59.533 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 6642, 2022-06-17T12:53:59.534 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2022-06-17T12:53:59.534 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2022-06-17T12:53:59.535 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2022-06-17T12:53:59.535 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 6642 2022-06-17T12:53:59.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:53:59.536 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:53:59.536 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:53:59.537 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:53:59.537 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 433076615 2022-06-17T12:53:59.538 INFO:teuthology.orchestra.run.smithi107.stdout: } 2022-06-17T12:53:59.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:53:59.539 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:53:59.539 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:53:59.540 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:53:59.540 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 433076615 2022-06-17T12:53:59.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:53:59.541 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.6642 172.21.15.107:0/433076615", 2022-06-17T12:53:59.542 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "172.21.15.107:0/433076615", 2022-06-17T12:53:59.542 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 1, 2022-06-17T12:53:59.543 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 36, 2022-06-17T12:53:59.543 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 75, 2022-06-17T12:53:59.543 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:53:59.544 INFO:teuthology.orchestra.run.smithi107.stdout: "blacklisted": false, 2022-06-17T12:53:59.545 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2022-06-17T12:53:59.545 INFO:teuthology.orchestra.run.smithi107.stdout:} 2022-06-17T12:53:59.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:59.548 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:53:59.549 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:53:59.557 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:53:59.558 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:53:59.559 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:59.559 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:53:59.583 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-17T12:53:59.584 DEBUG:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:53:59.607 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:53:59.630 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:53:59.650 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:53:59.672 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:53:59.696 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-17T12:53:59.696 INFO:tasks.cephfs.mount:Terminating background process 2022-06-17T12:53:59.697 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:59.697 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:53:59.698 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:53:59.708 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:53:59.710 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:53:59.710 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2022-06-17T12:53:59.711 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:53:59.712 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:53:59.769 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22435]: fuse finished with error 0 and tester_r 0 2022-06-17T12:53:59.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:59.771 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:53:59.772 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:53:59.779 INFO:teuthology.orchestra.run.smithi107.stdout:ext2/ext3 2022-06-17T12:53:59.781 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:53:59.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:53:59.902 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:53:59.951 INFO:tasks.cephfs.mount:Terminating background process 2022-06-17T12:53:59.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:53:59.952 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:53:59.952 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:53:59.962 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:53:59.963 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:53:59.964 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi136.front.sepia.ceph.com... 2022-06-17T12:53:59.964 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:53:59.965 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:54:00.021 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:ceph-fuse[21883]: fuse finished with error 0 and tester_r 0 2022-06-17T12:54:00.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:00.024 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:00.025 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:54:00.032 INFO:teuthology.orchestra.run.smithi136.stdout:ext2/ext3 2022-06-17T12:54:00.033 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:54:00.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:54:00.151 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:54:00.198 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:00.198 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:00.199 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:54:00.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:54:00.209 INFO:teuthology.orchestra.run.smithi143.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:54:00.210 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-06-17T12:54:00.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:54:00.212 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:54:00.261 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:54:00.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:54:00.267 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-17T12:54:00.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd3a6324700 1 Processor -- start 2022-06-17T12:54:00.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd3a6324700 1 -- start start 2022-06-17T12:54:00.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd3a6324700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd3a00b4e10 0x7fd3a00b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:00.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd3a6324700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3a00b31c0 0x7fd3a00b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:00.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd3a6324700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a00b21d0 con 0x7fd3a00620d0 2022-06-17T12:54:00.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd3a6324700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3a00b2430 con 0x7fd3a00b4e10 2022-06-17T12:54:00.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd3a6324700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a00b26d0 con 0x7fd3a00b31c0 2022-06-17T12:54:00.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd39f7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd3a00b4e10 0x7fd3a00b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:00.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd39ffff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fd3a00620d0 0x7fd3a00b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53476/0 (socket says 172.21.15.36:53476) 2022-06-17T12:54:00.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd39ffff700 1 -- 172.21.15.36:0/1118323660 learned_addr learned my addr 172.21.15.36:0/1118323660 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:00.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd3a48c1700 1 --2- 172.21.15.36:0/1118323660 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3a00b31c0 0x7fd3a00b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:00.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd39f7fe700 1 -- 172.21.15.36:0/1118323660 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3a00b31c0 msgr2=0x7fd3a00b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:00.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd39f7fe700 1 --2- 172.21.15.36:0/1118323660 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3a00b31c0 0x7fd3a00b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:00.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd39f7fe700 1 -- 172.21.15.36:0/1118323660 >> v1:172.21.15.36:6789/0 conn(0x7fd3a00620d0 legacy=0x7fd3a00b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:00.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd39f7fe700 1 -- 172.21.15.36:0/1118323660 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a00b43f0 con 0x7fd3a00b4e10 2022-06-17T12:54:00.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd39f7fe700 1 --2- 172.21.15.36:0/1118323660 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd3a00b4e10 0x7fd3a00b5bc0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7fd3900032f0 tx=0x7fd3900091d0).ready entity=mon.0 client_cookie=6ffd8a8f9d333719 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:00.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd39effd700 1 -- 172.21.15.36:0/1118323660 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd390009ea0 con 0x7fd3a00b4e10 2022-06-17T12:54:00.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd39effd700 1 -- 172.21.15.36:0/1118323660 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39000b8a0 con 0x7fd3a00b4e10 2022-06-17T12:54:00.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd39effd700 1 -- 172.21.15.36:0/1118323660 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd39000ba10 con 0x7fd3a00b4e10 2022-06-17T12:54:00.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd3a6324700 1 -- 172.21.15.36:0/1118323660 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd3a00b4e10 msgr2=0x7fd3a00b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:00.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd3a6324700 1 --2- 172.21.15.36:0/1118323660 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd3a00b4e10 0x7fd3a00b5bc0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7fd3900032f0 tx=0x7fd3900091d0).stop 2022-06-17T12:54:00.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd3a6324700 1 -- 172.21.15.36:0/1118323660 shutdown_connections 2022-06-17T12:54:00.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd3a6324700 1 --2- 172.21.15.36:0/1118323660 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3a00b31c0 0x7fd3a00b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:00.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd3a6324700 1 --2- 172.21.15.36:0/1118323660 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd3a00b4e10 0x7fd3a00b5bc0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:00.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd3a6324700 1 -- 172.21.15.36:0/1118323660 >> 172.21.15.36:0/1118323660 conn(0x7fd3a00b9850 msgr2=0x7fd3a00b9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:00.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd3a6324700 1 -- 172.21.15.36:0/1118323660 shutdown_connections 2022-06-17T12:54:00.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.396+0000 7fd3a6324700 1 -- 172.21.15.36:0/1118323660 wait complete. 2022-06-17T12:54:00.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.400+0000 7fd3a6324700 1 Processor -- start 2022-06-17T12:54:00.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.400+0000 7fd3a6324700 1 -- start start 2022-06-17T12:54:00.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.400+0000 7fd3a6324700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3a00620d0 0x7fd3a0155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:00.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.400+0000 7fd3a6324700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd3a00b31c0 0x7fd3a00b87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:00.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.400+0000 7fd3a6324700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3a00b4e10 0x7fd3a01402c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:00.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.400+0000 7fd3a6324700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a0156310 con 0x7fd3a00b31c0 2022-06-17T12:54:00.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.400+0000 7fd3a6324700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a0156530 con 0x7fd3a00620d0 2022-06-17T12:54:00.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.400+0000 7fd3a6324700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a01567d0 con 0x7fd3a00b4e10 2022-06-17T12:54:00.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.400+0000 7fd39f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd3a00b31c0 0x7fd3a00b87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:00.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.400+0000 7fd39f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd3a00b31c0 0x7fd3a00b87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35954/0 (socket says 172.21.15.36:35954) 2022-06-17T12:54:00.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.400+0000 7fd39f7fe700 1 -- 172.21.15.36:0/3981766002 learned_addr learned my addr 172.21.15.36:0/3981766002 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:00.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.400+0000 7fd39ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3a00620d0 0x7fd3a0155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:00.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.400+0000 7fd3a48c1700 1 --2- 172.21.15.36:0/3981766002 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3a00b4e10 0x7fd3a01402c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:00.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.400+0000 7fd39f7fe700 1 -- 172.21.15.36:0/3981766002 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3a00b4e10 msgr2=0x7fd3a01402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:00.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.400+0000 7fd39f7fe700 1 --2- 172.21.15.36:0/3981766002 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3a00b4e10 0x7fd3a01402c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:00.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.400+0000 7fd39f7fe700 1 -- 172.21.15.36:0/3981766002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3a00620d0 msgr2=0x7fd3a0155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:00.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.400+0000 7fd39f7fe700 1 --2- 172.21.15.36:0/3981766002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3a00620d0 0x7fd3a0155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:00.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.400+0000 7fd39f7fe700 1 -- 172.21.15.36:0/3981766002 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd390009470 con 0x7fd3a00b31c0 2022-06-17T12:54:00.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.400+0000 7fd39ffff700 1 --2- 172.21.15.36:0/3981766002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3a00620d0 0x7fd3a0155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:00.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.404+0000 7fd39f7fe700 1 --2- 172.21.15.36:0/3981766002 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd3a00b31c0 0x7fd3a00b87b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7fd3900032f0 tx=0x7fd390016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:00.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.404+0000 7fd39d7fa700 1 -- 172.21.15.36:0/3981766002 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd390011e20 con 0x7fd3a00b31c0 2022-06-17T12:54:00.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.404+0000 7fd39d7fa700 1 -- 172.21.15.36:0/3981766002 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd390020400 con 0x7fd3a00b31c0 2022-06-17T12:54:00.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.404+0000 7fd3a6324700 1 -- 172.21.15.36:0/3981766002 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a0156a70 con 0x7fd3a00b31c0 2022-06-17T12:54:00.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.404+0000 7fd39d7fa700 1 -- 172.21.15.36:0/3981766002 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd390020760 con 0x7fd3a00b31c0 2022-06-17T12:54:00.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.404+0000 7fd3a6324700 1 -- 172.21.15.36:0/3981766002 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a0112dc0 con 0x7fd3a00b31c0 2022-06-17T12:54:00.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.404+0000 7fd39d7fa700 1 -- 172.21.15.36:0/3981766002 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd390027030 con 0x7fd3a00b31c0 2022-06-17T12:54:00.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.404+0000 7fd3a6324700 1 -- 172.21.15.36:0/3981766002 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a00624b0 con 0x7fd3a00b31c0 2022-06-17T12:54:00.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.404+0000 7fd39d7fa700 1 --2- 172.21.15.36:0/3981766002 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd3780539d0 0x7fd378055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:00.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.404+0000 7fd39d7fa700 1 -- 172.21.15.36:0/3981766002 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6463+0+0 (secure 0 0 0) 0x7fd39006d090 con 0x7fd3a00b31c0 2022-06-17T12:54:00.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.404+0000 7fd39ffff700 1 --2- 172.21.15.36:0/3981766002 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd3780539d0 0x7fd378055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:00.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.404+0000 7fd39ffff700 1 --2- 172.21.15.36:0/3981766002 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd3780539d0 0x7fd378055e90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7fd38800b000 tx=0x7fd388009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:00.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.412+0000 7fd39d7fa700 1 -- 172.21.15.36:0/3981766002 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd390047840 con 0x7fd3a00b31c0 2022-06-17T12:54:00.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.540+0000 7fd3a6324700 1 -- 172.21.15.36:0/3981766002 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd3a0057410 con 0x7fd3a00b31c0 2022-06-17T12:54:00.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.540+0000 7fd39d7fa700 1 -- 172.21.15.36:0/3981766002 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v75) v1 ==== 81+0+34116 (secure 0 0 0) 0x7fd390025090 con 0x7fd3a00b31c0 2022-06-17T12:54:00.531 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:00.531 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:54:00.532 INFO:teuthology.orchestra.run.smithi036.stdout: "epoch": 75, 2022-06-17T12:54:00.532 INFO:teuthology.orchestra.run.smithi036.stdout: "fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", 2022-06-17T12:54:00.532 INFO:teuthology.orchestra.run.smithi036.stdout: "created": "2022-06-17T12:46:30.671408+0000", 2022-06-17T12:54:00.532 INFO:teuthology.orchestra.run.smithi036.stdout: "modified": "2022-06-17T12:53:51.518039+0000", 2022-06-17T12:54:00.533 INFO:teuthology.orchestra.run.smithi036.stdout: "last_up_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:54:00.533 INFO:teuthology.orchestra.run.smithi036.stdout: "last_in_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:54:00.533 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-17T12:54:00.533 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_num": 5799936, 2022-06-17T12:54:00.534 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_set": [ 2022-06-17T12:54:00.534 INFO:teuthology.orchestra.run.smithi036.stdout: "pglog_hardlimit", 2022-06-17T12:54:00.535 INFO:teuthology.orchestra.run.smithi036.stdout: "purged_snapdirs", 2022-06-17T12:54:00.535 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_deletes", 2022-06-17T12:54:00.535 INFO:teuthology.orchestra.run.smithi036.stdout: "sortbitwise" 2022-06-17T12:54:00.535 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:54:00.536 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_version": 6, 2022-06-17T12:54:00.537 INFO:teuthology.orchestra.run.smithi036.stdout: "full_ratio": 0.94999998807907104, 2022-06-17T12:54:00.538 INFO:teuthology.orchestra.run.smithi036.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-17T12:54:00.539 INFO:teuthology.orchestra.run.smithi036.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-17T12:54:00.539 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_snapshot": "", 2022-06-17T12:54:00.539 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_max": 14, 2022-06-17T12:54:00.539 INFO:teuthology.orchestra.run.smithi036.stdout: "max_osd": 8, 2022-06-17T12:54:00.540 INFO:teuthology.orchestra.run.smithi036.stdout: "require_min_compat_client": "jewel", 2022-06-17T12:54:00.540 INFO:teuthology.orchestra.run.smithi036.stdout: "min_compat_client": "jewel", 2022-06-17T12:54:00.540 INFO:teuthology.orchestra.run.smithi036.stdout: "require_osd_release": "octopus", 2022-06-17T12:54:00.540 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:54:00.541 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.541 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 1, 2022-06-17T12:54:00.541 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "device_health_metrics", 2022-06-17T12:54:00.541 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:33.881212+0000", 2022-06-17T12:54:00.542 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:54:00.542 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:54:00.542 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:54:00.542 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:54:00.543 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:54:00.543 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:54:00.543 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:54:00.543 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:54:00.544 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 1, 2022-06-17T12:54:00.544 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 1, 2022-06-17T12:54:00.544 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 1, 2022-06-17T12:54:00.544 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 1, 2022-06-17T12:54:00.545 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 1, 2022-06-17T12:54:00.545 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:54:00.545 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:54:00.546 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:54:00.546 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:54:00.546 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:54:00.546 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:54:00.547 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:54:00.547 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.547 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "16", 2022-06-17T12:54:00.547 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:54:00.548 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:54:00.548 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:54:00.548 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:54:00.548 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:54:00.549 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:54:00.549 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:54:00.549 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:54:00.549 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:54:00.550 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:54:00.550 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:54:00.550 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:54:00.550 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:54:00.551 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:54:00.551 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:54:00.551 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:54:00.551 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:54:00.552 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:54:00.552 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:54:00.552 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:54:00.552 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:54:00.552 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:54:00.553 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:54:00.553 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:54:00.553 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:54:00.553 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:54:00.553 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.554 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:54:00.554 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:54:00.554 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:54:00.554 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:54:00.555 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:54:00.555 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:54:00.555 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:54:00.555 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:54:00.556 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:54:00.556 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:54:00.556 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:54:00.556 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:54:00.557 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 1 2022-06-17T12:54:00.557 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.557 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:54:00.557 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr_devicehealth": {} 2022-06-17T12:54:00.557 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.558 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.558 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.558 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 2, 2022-06-17T12:54:00.558 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "rbd", 2022-06-17T12:54:00.559 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:44.485460+0000", 2022-06-17T12:54:00.559 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:54:00.559 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:54:00.559 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:54:00.560 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:54:00.560 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:54:00.560 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:54:00.560 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:54:00.560 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:54:00.561 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:54:00.561 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:54:00.561 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:54:00.562 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:54:00.562 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:54:00.562 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:54:00.563 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:54:00.563 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:54:00.563 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:54:00.564 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:54:00.564 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:54:00.564 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:54:00.564 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.565 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "19", 2022-06-17T12:54:00.565 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:54:00.565 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:54:00.566 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:54:00.566 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:54:00.566 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:54:00.567 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:54:00.567 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:54:00.567 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:54:00.567 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:54:00.568 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:54:00.568 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:54:00.568 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:54:00.569 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:54:00.569 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:54:00.569 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:54:00.569 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:54:00.570 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:54:00.570 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:54:00.570 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:54:00.570 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:54:00.570 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:54:00.571 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:54:00.571 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:54:00.571 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:54:00.572 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:54:00.572 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:54:00.573 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.574 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:54:00.574 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:54:00.574 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:54:00.575 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:54:00.575 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:54:00.576 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:54:00.576 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:54:00.576 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:54:00.577 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:54:00.577 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:54:00.577 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:54:00.577 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:54:00.577 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:54:00.578 INFO:teuthology.orchestra.run.smithi036.stdout: "rbd": {} 2022-06-17T12:54:00.578 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.578 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.578 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.579 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 13, 2022-06-17T12:54:00.579 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_metadata", 2022-06-17T12:54:00.579 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:53:03.265804+0000", 2022-06-17T12:54:00.579 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:54:00.579 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:54:00.580 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:54:00.580 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:54:00.580 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:54:00.580 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:54:00.581 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:54:00.581 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:54:00.581 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:54:00.581 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:54:00.581 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:54:00.582 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:54:00.582 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:54:00.582 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:54:00.582 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:54:00.583 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:54:00.583 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:54:00.583 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:54:00.583 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:54:00.583 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:54:00.584 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.584 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "73", 2022-06-17T12:54:00.584 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:54:00.584 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:54:00.585 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:54:00.585 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:54:00.585 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:54:00.586 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:54:00.586 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:54:00.586 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:54:00.586 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:54:00.586 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:54:00.587 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:54:00.587 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:54:00.587 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:54:00.587 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:54:00.588 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:54:00.588 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:54:00.588 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:54:00.588 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:54:00.588 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:54:00.589 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:54:00.589 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:54:00.589 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:54:00.589 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:54:00.590 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:54:00.590 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:54:00.590 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:54:00.591 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.591 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:54:00.591 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:54:00.591 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:54:00.592 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:54:00.592 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:54:00.592 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:54:00.592 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:54:00.592 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:54:00.593 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:54:00.593 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:54:00.593 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:54:00.593 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:54:00.594 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_bias": 4, 2022-06-17T12:54:00.594 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 16, 2022-06-17T12:54:00.594 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_priority": 5 2022-06-17T12:54:00.594 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.594 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:54:00.595 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:54:00.595 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": "cephfs" 2022-06-17T12:54:00.595 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.595 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.596 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.596 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.596 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 14, 2022-06-17T12:54:00.596 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_data", 2022-06-17T12:54:00.597 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:53:04.385158+0000", 2022-06-17T12:54:00.597 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:54:00.597 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:54:00.597 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:54:00.597 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:54:00.598 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:54:00.598 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:54:00.598 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:54:00.598 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:54:00.599 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:54:00.599 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:54:00.599 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:54:00.599 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:54:00.599 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:54:00.600 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:54:00.600 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:54:00.600 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:54:00.600 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:54:00.601 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:54:00.601 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:54:00.601 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:54:00.601 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.601 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "74", 2022-06-17T12:54:00.602 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:54:00.602 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:54:00.602 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:54:00.602 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:54:00.603 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:54:00.603 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:54:00.603 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:54:00.604 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:54:00.604 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:54:00.604 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:54:00.604 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:54:00.605 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:54:00.605 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:54:00.605 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:54:00.605 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:54:00.605 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:54:00.606 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:54:00.606 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:54:00.606 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:54:00.606 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:54:00.607 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:54:00.607 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:54:00.607 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:54:00.607 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:54:00.607 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:54:00.608 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:54:00.608 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.608 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:54:00.608 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:54:00.609 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:54:00.609 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:54:00.609 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:54:00.610 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:54:00.610 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:54:00.610 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:54:00.610 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:54:00.611 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:54:00.611 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:54:00.611 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:54:00.611 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:54:00.611 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:54:00.612 INFO:teuthology.orchestra.run.smithi036.stdout: "data": "cephfs" 2022-06-17T12:54:00.612 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.612 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.612 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.613 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:54:00.613 INFO:teuthology.orchestra.run.smithi036.stdout: "osds": [ 2022-06-17T12:54:00.613 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.613 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:54:00.613 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", 2022-06-17T12:54:00.614 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:00.614 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:00.614 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:00.614 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:00.615 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:00.615 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:00.615 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:54:00.615 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 72, 2022-06-17T12:54:00.615 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:00.616 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:00.616 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:00.616 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.616 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.616 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.617 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6802", 2022-06-17T12:54:00.617 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:00.617 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.617 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.618 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.618 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6803", 2022-06-17T12:54:00.618 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:00.618 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.618 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.619 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.619 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:00.619 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.619 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.620 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.620 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6804", 2022-06-17T12:54:00.620 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:00.620 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.621 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.621 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.621 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6805", 2022-06-17T12:54:00.621 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:00.621 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.622 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.622 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.622 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:00.622 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.623 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.623 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.623 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6808", 2022-06-17T12:54:00.623 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:00.623 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.624 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.624 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.624 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6809", 2022-06-17T12:54:00.624 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:00.625 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.625 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.625 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.625 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:00.625 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.626 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.626 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.626 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6806", 2022-06-17T12:54:00.626 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:00.627 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.627 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.627 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.627 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6807", 2022-06-17T12:54:00.627 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:00.628 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.628 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.628 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.629 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6803/22619", 2022-06-17T12:54:00.629 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6805/22619", 2022-06-17T12:54:00.629 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6809/22619", 2022-06-17T12:54:00.629 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6807/22619", 2022-06-17T12:54:00.630 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:00.630 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:00.630 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:00.630 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.631 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.631 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.631 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:54:00.632 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", 2022-06-17T12:54:00.632 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:00.632 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:00.633 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:00.633 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:00.633 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:00.634 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:00.634 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:54:00.634 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 63, 2022-06-17T12:54:00.635 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:00.635 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:00.636 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:00.636 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.636 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.636 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.637 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6810", 2022-06-17T12:54:00.637 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:00.637 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.637 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.638 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.638 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6811", 2022-06-17T12:54:00.638 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:00.638 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.639 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.639 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.639 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:00.639 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.639 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.640 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.640 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6812", 2022-06-17T12:54:00.640 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:00.640 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.640 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.641 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.641 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6813", 2022-06-17T12:54:00.641 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:00.641 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.642 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.642 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.642 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:00.642 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.643 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.643 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.643 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6816", 2022-06-17T12:54:00.643 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:00.643 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.644 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.644 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.644 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6817", 2022-06-17T12:54:00.644 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:00.645 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.645 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.645 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.645 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:00.645 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.646 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.646 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.646 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6814", 2022-06-17T12:54:00.646 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:00.647 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.647 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.647 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.647 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6815", 2022-06-17T12:54:00.647 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:00.648 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.648 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.648 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.648 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6811/22620", 2022-06-17T12:54:00.649 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6813/22620", 2022-06-17T12:54:00.649 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6817/22620", 2022-06-17T12:54:00.649 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6815/22620", 2022-06-17T12:54:00.650 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:00.650 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:00.650 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:00.650 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.650 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.651 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.651 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:54:00.651 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", 2022-06-17T12:54:00.651 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:00.652 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:00.652 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:00.652 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:00.652 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:00.652 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:00.653 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:54:00.653 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 72, 2022-06-17T12:54:00.653 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:00.653 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:00.654 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:00.654 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.654 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.654 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.654 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6826", 2022-06-17T12:54:00.655 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:00.655 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.655 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.655 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.656 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6827", 2022-06-17T12:54:00.656 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:00.656 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.657 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.657 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.658 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:00.658 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.658 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.658 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.659 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6828", 2022-06-17T12:54:00.659 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:00.659 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.659 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.660 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.660 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6829", 2022-06-17T12:54:00.660 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:00.660 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.661 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.661 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.661 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:00.661 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.661 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.662 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.662 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6832", 2022-06-17T12:54:00.662 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:00.662 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.663 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.663 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.663 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6833", 2022-06-17T12:54:00.663 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:00.663 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.664 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.664 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.664 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:00.664 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.665 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.665 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.665 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6830", 2022-06-17T12:54:00.665 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:00.665 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.666 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.666 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.666 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6831", 2022-06-17T12:54:00.666 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:00.667 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.667 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.667 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.667 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6827/22643", 2022-06-17T12:54:00.667 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6829/22643", 2022-06-17T12:54:00.668 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6833/22643", 2022-06-17T12:54:00.668 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6831/22643", 2022-06-17T12:54:00.668 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:00.668 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:00.669 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:00.669 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.669 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.669 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.669 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:54:00.670 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", 2022-06-17T12:54:00.670 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:00.670 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:00.670 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:00.671 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:00.671 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:00.671 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:00.671 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:54:00.671 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 72, 2022-06-17T12:54:00.672 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:00.672 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:00.672 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:00.672 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.673 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.673 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.673 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6818", 2022-06-17T12:54:00.674 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:00.674 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.674 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.674 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.675 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6819", 2022-06-17T12:54:00.675 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:00.675 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.675 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.675 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.676 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:00.676 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.676 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.676 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.677 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6820", 2022-06-17T12:54:00.677 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:00.677 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.677 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.678 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.678 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6821", 2022-06-17T12:54:00.678 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:00.678 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.678 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.679 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.679 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:00.679 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.679 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.680 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.680 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6824", 2022-06-17T12:54:00.680 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:00.680 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.680 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.681 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.681 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6825", 2022-06-17T12:54:00.681 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:00.681 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.682 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.682 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.682 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:00.682 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.683 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.683 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.683 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6822", 2022-06-17T12:54:00.683 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:00.683 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.684 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.684 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.684 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6823", 2022-06-17T12:54:00.684 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:00.685 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.685 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.685 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.685 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6819/22646", 2022-06-17T12:54:00.685 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6821/22646", 2022-06-17T12:54:00.686 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6825/22646", 2022-06-17T12:54:00.686 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6823/22646", 2022-06-17T12:54:00.686 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:00.686 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:00.687 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:00.687 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.687 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.687 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.688 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:54:00.688 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", 2022-06-17T12:54:00.688 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:00.688 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:00.688 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:00.689 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:00.689 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:00.689 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:00.689 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:54:00.690 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 71, 2022-06-17T12:54:00.690 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:00.690 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:00.690 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:00.691 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.691 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.691 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.691 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6808", 2022-06-17T12:54:00.691 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:00.692 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.692 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.692 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.692 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6809", 2022-06-17T12:54:00.693 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:00.693 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.693 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.694 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.694 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:00.694 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.694 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.695 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.695 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6810", 2022-06-17T12:54:00.695 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:00.695 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.695 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.696 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.696 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6811", 2022-06-17T12:54:00.696 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:00.696 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.697 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.697 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.697 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:00.697 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.697 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.698 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.698 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6814", 2022-06-17T12:54:00.698 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:00.698 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.699 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.699 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.699 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6815", 2022-06-17T12:54:00.699 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:00.700 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.700 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.700 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.700 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:00.700 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.701 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.701 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.701 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6812", 2022-06-17T12:54:00.701 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:00.702 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.702 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.702 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.702 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6813", 2022-06-17T12:54:00.703 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:00.703 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.703 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.704 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.704 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6809/22812", 2022-06-17T12:54:00.704 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6811/22812", 2022-06-17T12:54:00.704 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6815/22812", 2022-06-17T12:54:00.705 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6813/22812", 2022-06-17T12:54:00.705 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:00.705 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:00.705 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:00.706 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.706 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.706 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.706 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:54:00.707 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", 2022-06-17T12:54:00.707 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:00.707 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:00.707 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:00.708 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:00.708 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:00.708 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:00.708 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:54:00.708 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 72, 2022-06-17T12:54:00.709 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:00.709 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:00.709 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:00.710 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.710 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.710 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.710 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6800", 2022-06-17T12:54:00.710 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:00.711 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.711 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.711 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.711 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6801", 2022-06-17T12:54:00.712 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:00.712 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.712 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.712 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.712 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:00.713 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.713 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.713 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.713 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6802", 2022-06-17T12:54:00.714 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:00.714 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.714 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.714 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.714 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6803", 2022-06-17T12:54:00.715 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:00.715 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.715 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.715 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.715 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:00.716 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.716 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.716 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.716 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6806", 2022-06-17T12:54:00.717 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:00.717 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.717 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.717 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.718 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6807", 2022-06-17T12:54:00.718 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:00.718 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.718 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.718 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.719 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:00.719 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.719 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.719 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.720 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6804", 2022-06-17T12:54:00.720 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:00.720 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.720 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.720 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.721 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6805", 2022-06-17T12:54:00.721 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:00.721 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.721 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.722 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.722 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6801/22813", 2022-06-17T12:54:00.722 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6803/22813", 2022-06-17T12:54:00.722 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6807/22813", 2022-06-17T12:54:00.723 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6805/22813", 2022-06-17T12:54:00.723 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:00.723 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:00.723 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:00.724 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.724 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.724 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.724 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:54:00.724 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", 2022-06-17T12:54:00.725 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:00.725 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:00.725 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:00.725 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:00.725 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:00.726 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:00.726 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:54:00.726 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 72, 2022-06-17T12:54:00.726 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:00.727 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:00.727 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:00.727 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.727 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.727 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.728 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6816", 2022-06-17T12:54:00.728 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:00.728 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.728 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.729 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.729 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6817", 2022-06-17T12:54:00.729 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:00.730 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.730 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.730 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.730 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:00.730 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.731 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.731 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.731 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6818", 2022-06-17T12:54:00.731 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:00.731 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.732 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.732 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.732 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6819", 2022-06-17T12:54:00.732 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:00.733 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.733 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.733 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.733 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:00.733 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.734 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.734 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.734 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6822", 2022-06-17T12:54:00.734 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:00.734 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.735 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.735 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.735 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6823", 2022-06-17T12:54:00.736 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:00.736 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.736 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.737 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.737 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:00.737 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.737 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.737 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.738 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6820", 2022-06-17T12:54:00.738 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:00.738 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.738 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.739 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.739 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6821", 2022-06-17T12:54:00.739 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:00.739 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.740 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.740 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.740 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6817/22822", 2022-06-17T12:54:00.740 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6819/22822", 2022-06-17T12:54:00.741 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6823/22822", 2022-06-17T12:54:00.741 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6821/22822", 2022-06-17T12:54:00.741 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:00.741 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:00.742 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:00.742 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.742 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.742 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.743 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:54:00.743 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", 2022-06-17T12:54:00.743 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:00.743 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:00.744 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:00.744 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:00.744 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:00.744 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:00.745 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:54:00.745 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 71, 2022-06-17T12:54:00.745 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:00.745 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:00.746 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:00.746 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.746 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.746 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.747 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6824", 2022-06-17T12:54:00.747 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:00.747 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.747 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.747 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.748 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6825", 2022-06-17T12:54:00.748 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:00.748 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.748 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.749 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.749 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:00.749 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.749 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.749 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.750 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6826", 2022-06-17T12:54:00.750 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:00.750 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.751 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.751 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.751 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6827", 2022-06-17T12:54:00.751 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:00.751 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.752 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.752 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.752 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:00.752 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.753 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.753 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.753 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6830", 2022-06-17T12:54:00.753 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:00.753 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.754 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.754 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.754 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6831", 2022-06-17T12:54:00.754 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:00.754 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.755 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.755 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.755 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:00.755 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:00.756 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.756 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:00.756 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6828", 2022-06-17T12:54:00.756 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:00.756 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.757 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.757 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:00.757 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6829", 2022-06-17T12:54:00.757 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:00.758 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.758 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.758 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.758 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6825/22837", 2022-06-17T12:54:00.759 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6827/22837", 2022-06-17T12:54:00.759 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6831/22837", 2022-06-17T12:54:00.759 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6829/22837", 2022-06-17T12:54:00.759 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:00.759 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:00.760 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:00.760 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:00.760 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.760 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:54:00.761 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_xinfo": [ 2022-06-17T12:54:00.761 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.761 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:54:00.761 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:00.761 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:00.762 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:00.762 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:00.762 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:00.762 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.291461+0000", 2022-06-17T12:54:00.762 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:00.763 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.763 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.763 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:54:00.763 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:00.764 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:00.764 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:00.764 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:00.764 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:00.764 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.537912+0000", 2022-06-17T12:54:00.765 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:00.765 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.765 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.765 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:54:00.766 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:00.766 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:00.766 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:00.766 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:00.766 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:00.767 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.606947+0000", 2022-06-17T12:54:00.767 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:00.767 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.767 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.768 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:54:00.768 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:00.768 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:00.768 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:00.768 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:00.769 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:00.769 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.514412+0000", 2022-06-17T12:54:00.769 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:00.769 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.770 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.770 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:54:00.770 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:00.770 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:00.771 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:00.771 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:00.771 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:00.771 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.394645+0000", 2022-06-17T12:54:00.771 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:00.772 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.772 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.772 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:54:00.772 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:00.772 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:00.773 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:00.773 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:00.773 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:00.773 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.343806+0000", 2022-06-17T12:54:00.774 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:00.774 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.774 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.774 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:54:00.774 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:00.775 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:00.775 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:00.775 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:00.775 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:00.776 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.591864+0000", 2022-06-17T12:54:00.776 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:00.776 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.776 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:00.776 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:54:00.777 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:00.777 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:00.777 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:00.777 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:00.778 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:00.778 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.607045+0000", 2022-06-17T12:54:00.778 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:00.778 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.779 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:54:00.779 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap": [], 2022-06-17T12:54:00.779 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap_items": [], 2022-06-17T12:54:00.779 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_temp": [], 2022-06-17T12:54:00.779 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_temp": [], 2022-06-17T12:54:00.780 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklist": { 2022-06-17T12:54:00.780 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.107:0/973797138": "2022-06-17T13:53:50.522507+0000" 2022-06-17T12:54:00.780 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.780 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profiles": { 2022-06-17T12:54:00.780 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs_data": { 2022-06-17T12:54:00.781 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-device-class": "", 2022-06-17T12:54:00.781 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:54:00.781 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-root": "default", 2022-06-17T12:54:00.781 INFO:teuthology.orchestra.run.smithi036.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-17T12:54:00.782 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:54:00.782 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "2", 2022-06-17T12:54:00.782 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:54:00.782 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:54:00.782 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van", 2022-06-17T12:54:00.783 INFO:teuthology.orchestra.run.smithi036.stdout: "w": "8" 2022-06-17T12:54:00.783 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.783 INFO:teuthology.orchestra.run.smithi036.stdout: "default": { 2022-06-17T12:54:00.783 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:54:00.783 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:54:00.784 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "1", 2022-06-17T12:54:00.784 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:54:00.784 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:54:00.784 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van" 2022-06-17T12:54:00.785 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:00.785 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:00.785 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps_queue": [], 2022-06-17T12:54:00.785 INFO:teuthology.orchestra.run.smithi036.stdout: "new_removed_snaps": [], 2022-06-17T12:54:00.785 INFO:teuthology.orchestra.run.smithi036.stdout: "new_purged_snaps": [], 2022-06-17T12:54:00.786 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_node_flags": {}, 2022-06-17T12:54:00.786 INFO:teuthology.orchestra.run.smithi036.stdout: "device_class_flags": {} 2022-06-17T12:54:00.786 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:54:00.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.544+0000 7fd3a6324700 1 -- 172.21.15.36:0/3981766002 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd3780539d0 msgr2=0x7fd378055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:00.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.544+0000 7fd3a6324700 1 --2- 172.21.15.36:0/3981766002 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd3780539d0 0x7fd378055e90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7fd38800b000 tx=0x7fd388009040).stop 2022-06-17T12:54:00.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.544+0000 7fd3a6324700 1 -- 172.21.15.36:0/3981766002 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd3a00b31c0 msgr2=0x7fd3a00b87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:00.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.544+0000 7fd3a6324700 1 --2- 172.21.15.36:0/3981766002 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd3a00b31c0 0x7fd3a00b87b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7fd3900032f0 tx=0x7fd390016000).stop 2022-06-17T12:54:00.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.544+0000 7fd3a6324700 1 -- 172.21.15.36:0/3981766002 shutdown_connections 2022-06-17T12:54:00.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.544+0000 7fd3a6324700 1 --2- 172.21.15.36:0/3981766002 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd3780539d0 0x7fd378055e90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:00.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.544+0000 7fd3a6324700 1 --2- 172.21.15.36:0/3981766002 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3a00b4e10 0x7fd3a01402c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:00.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.544+0000 7fd3a6324700 1 --2- 172.21.15.36:0/3981766002 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd3a00b31c0 0x7fd3a00b87b0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:00.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.544+0000 7fd3a6324700 1 --2- 172.21.15.36:0/3981766002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3a00620d0 0x7fd3a0155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:00.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.544+0000 7fd3a6324700 1 -- 172.21.15.36:0/3981766002 >> 172.21.15.36:0/3981766002 conn(0x7fd3a00b9850 msgr2=0x7fd3a00b4680 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:00.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.548+0000 7fd3a6324700 1 -- 172.21.15.36:0/3981766002 shutdown_connections 2022-06-17T12:54:00.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.548+0000 7fd3a6324700 1 -- 172.21.15.36:0/3981766002 wait complete. 2022-06-17T12:54:00.791 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:54:00.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.916+0000 7f2b8ffff700 1 Processor -- start 2022-06-17T12:54:00.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.916+0000 7f2b8ffff700 1 -- start start 2022-06-17T12:54:00.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.916+0000 7f2b8ffff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2b900b2200 0x7f2b900af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:00.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.916+0000 7f2b8ffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2b900b2cd0 0x7f2b90147600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:00.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.916+0000 7f2b8ffff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b900b25e0 con 0x7f2b900af8d0 2022-06-17T12:54:00.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.916+0000 7f2b8ffff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b900ad790 con 0x7f2b900b2cd0 2022-06-17T12:54:00.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.916+0000 7f2b8ffff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b900aef40 con 0x7f2b900b2200 2022-06-17T12:54:00.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.916+0000 7f2b8e7fc700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f2b900af8d0 0x7f2b9014a8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53490/0 (socket says 172.21.15.36:53490) 2022-06-17T12:54:00.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.916+0000 7f2b8e7fc700 1 -- 172.21.15.36:0/87778264 learned_addr learned my addr 172.21.15.36:0/87778264 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:00.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.916+0000 7f2b8f7fe700 1 --2- 172.21.15.36:0/87778264 >> v2:172.21.15.36:3300/0 conn(0x7f2b900b2cd0 0x7f2b90147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:00.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.916+0000 7f2b8effd700 1 --2- 172.21.15.36:0/87778264 >> v2:172.21.15.70:3300/0 conn(0x7f2b900b2200 0x7f2b900af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:00.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.916+0000 7f2b8dffb700 1 -- 172.21.15.36:0/87778264 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1990240965 0 0) 0x7f2b900b25e0 con 0x7f2b900af8d0 2022-06-17T12:54:00.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.916+0000 7f2b8dffb700 1 -- 172.21.15.36:0/87778264 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b74003260 con 0x7f2b900af8d0 2022-06-17T12:54:00.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.916+0000 7f2b8dffb700 1 -- 172.21.15.36:0/87778264 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4085173855 0 0) 0x7f2b74003260 con 0x7f2b900af8d0 2022-06-17T12:54:00.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.916+0000 7f2b8dffb700 1 -- 172.21.15.36:0/87778264 >> v2:172.21.15.70:3300/0 conn(0x7f2b900b2200 msgr2=0x7f2b900af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:00.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.916+0000 7f2b8dffb700 1 --2- 172.21.15.36:0/87778264 >> v2:172.21.15.70:3300/0 conn(0x7f2b900b2200 0x7f2b900af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:00.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.916+0000 7f2b8dffb700 1 -- 172.21.15.36:0/87778264 >> v2:172.21.15.36:3300/0 conn(0x7f2b900b2cd0 msgr2=0x7f2b90147600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:00.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.916+0000 7f2b8dffb700 1 --2- 172.21.15.36:0/87778264 >> v2:172.21.15.36:3300/0 conn(0x7f2b900b2cd0 0x7f2b90147600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:00.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.916+0000 7f2b8dffb700 1 -- 172.21.15.36:0/87778264 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b900b1950 con 0x7f2b900af8d0 2022-06-17T12:54:00.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.916+0000 7f2b8dffb700 1 -- 172.21.15.36:0/87778264 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f2b800019e0 con 0x7f2b900af8d0 2022-06-17T12:54:00.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.916+0000 7f2b8f7fe700 1 --2- 172.21.15.36:0/87778264 >> v2:172.21.15.36:3300/0 conn(0x7f2b900b2cd0 0x7f2b90147600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:00.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8dffb700 1 -- 172.21.15.36:0/87778264 >> v1:172.21.15.36:6789/0 conn(0x7f2b900af8d0 legacy=0x7f2b9014a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:00.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8dffb700 1 --2- 172.21.15.36:0/87778264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b74003750 0x7f2b74003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:00.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8dffb700 1 --2- 172.21.15.36:0/87778264 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b74004500 0x7f2b74004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:00.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8dffb700 1 --2- 172.21.15.36:0/87778264 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b74004e70 0x7f2b7400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:00.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8dffb700 1 -- 172.21.15.36:0/87778264 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b900ad790 con 0x7f2b74004500 2022-06-17T12:54:00.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8dffb700 1 -- 172.21.15.36:0/87778264 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b900aef40 con 0x7f2b74003750 2022-06-17T12:54:00.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8dffb700 1 -- 172.21.15.36:0/87778264 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7400bc40 con 0x7f2b74004e70 2022-06-17T12:54:00.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8effd700 1 --2- 172.21.15.36:0/87778264 >> v2:172.21.15.70:3300/0 conn(0x7f2b900b2200 0x7f2b900af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:00.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8e7fc700 1 --2- 172.21.15.36:0/87778264 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b74004500 0x7f2b74004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:00.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8f7fe700 1 --2- 172.21.15.36:0/87778264 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b74004e70 0x7f2b7400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:00.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8effd700 1 --2- 172.21.15.36:0/87778264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b74003750 0x7f2b74003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:00.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8e7fc700 1 -- 172.21.15.36:0/87778264 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b74004e70 msgr2=0x7f2b7400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:00.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8e7fc700 1 --2- 172.21.15.36:0/87778264 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b74004e70 0x7f2b7400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:00.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8e7fc700 1 -- 172.21.15.36:0/87778264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b74003750 msgr2=0x7f2b74003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:00.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8e7fc700 1 --2- 172.21.15.36:0/87778264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b74003750 0x7f2b74003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:00.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8e7fc700 1 -- 172.21.15.36:0/87778264 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b74003260 con 0x7f2b74004500 2022-06-17T12:54:00.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8f7fe700 1 -- 172.21.15.36:0/87778264 reap_dead start 2022-06-17T12:54:00.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8e7fc700 1 --2- 172.21.15.36:0/87778264 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b74004500 0x7f2b74004980 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f2b800016f0 tx=0x7f2b800021d0).ready entity=mon.0 client_cookie=cef9f00c3a18ec43 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:00.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8dffb700 1 -- 172.21.15.36:0/87778264 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b8000cba0 con 0x7f2b74004500 2022-06-17T12:54:00.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8dffb700 1 -- 172.21.15.36:0/87778264 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8000cd00 con 0x7f2b74004500 2022-06-17T12:54:00.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8dffb700 1 -- 172.21.15.36:0/87778264 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b80005b80 con 0x7f2b74004500 2022-06-17T12:54:00.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8ffff700 1 -- 172.21.15.36:0/87778264 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b74004500 msgr2=0x7f2b74004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:00.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8ffff700 1 --2- 172.21.15.36:0/87778264 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b74004500 0x7f2b74004980 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f2b800016f0 tx=0x7f2b800021d0).stop 2022-06-17T12:54:00.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8ffff700 1 -- 172.21.15.36:0/87778264 shutdown_connections 2022-06-17T12:54:00.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8ffff700 1 --2- 172.21.15.36:0/87778264 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b74004500 0x7f2b74004980 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:00.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8ffff700 1 -- 172.21.15.36:0/87778264 >> 172.21.15.36:0/87778264 conn(0x7f2b9005f770 msgr2=0x7f2b9005fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:00.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8ffff700 1 -- 172.21.15.36:0/87778264 shutdown_connections 2022-06-17T12:54:00.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.920+0000 7f2b8ffff700 1 -- 172.21.15.36:0/87778264 wait complete. 2022-06-17T12:54:00.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.924+0000 7f2b8ffff700 1 Processor -- start 2022-06-17T12:54:00.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.924+0000 7f2b8ffff700 1 -- start start 2022-06-17T12:54:00.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.924+0000 7f2b8ffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b74004500 0x7f2b90144710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:00.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.924+0000 7f2b8ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b90159c60 0x7f2b901452d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:00.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.924+0000 7f2b8ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b901498c0 0x7f2b90142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:00.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.924+0000 7f2b8ffff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b901457c0 con 0x7f2b90159c60 2022-06-17T12:54:00.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.924+0000 7f2b8ffff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b901459d0 con 0x7f2b901498c0 2022-06-17T12:54:00.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.924+0000 7f2b8ffff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b90145c70 con 0x7f2b74004500 2022-06-17T12:54:00.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.924+0000 7f2b8e7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b90159c60 0x7f2b901452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:00.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.924+0000 7f2b8e7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b90159c60 0x7f2b901452d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35974/0 (socket says 172.21.15.36:35974) 2022-06-17T12:54:00.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.924+0000 7f2b8e7fc700 1 -- 172.21.15.36:0/4118462947 learned_addr learned my addr 172.21.15.36:0/4118462947 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:00.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.924+0000 7f2b8effd700 1 --2- 172.21.15.36:0/4118462947 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b74004500 0x7f2b90144710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:00.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.924+0000 7f2b8f7fe700 1 --2- 172.21.15.36:0/4118462947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b901498c0 0x7f2b90142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:00.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.924+0000 7f2b8e7fc700 1 -- 172.21.15.36:0/4118462947 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b74004500 msgr2=0x7f2b90144710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:00.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.924+0000 7f2b8e7fc700 1 --2- 172.21.15.36:0/4118462947 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b74004500 0x7f2b90144710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:00.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.924+0000 7f2b8e7fc700 1 -- 172.21.15.36:0/4118462947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b901498c0 msgr2=0x7f2b90142250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:00.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.924+0000 7f2b8e7fc700 1 --2- 172.21.15.36:0/4118462947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b901498c0 0x7f2b90142250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:00.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.924+0000 7f2b8e7fc700 1 -- 172.21.15.36:0/4118462947 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b90145f10 con 0x7f2b90159c60 2022-06-17T12:54:00.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.924+0000 7f2b8e7fc700 1 --2- 172.21.15.36:0/4118462947 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b90159c60 0x7f2b901452d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f2b800016f0 tx=0x7f2b80015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:00.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.924+0000 7f2b7ffff700 1 -- 172.21.15.36:0/4118462947 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b800048a0 con 0x7f2b90159c60 2022-06-17T12:54:00.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.924+0000 7f2b7ffff700 1 -- 172.21.15.36:0/4118462947 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b80004a00 con 0x7f2b90159c60 2022-06-17T12:54:00.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.924+0000 7f2b7ffff700 1 -- 172.21.15.36:0/4118462947 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b80004d80 con 0x7f2b90159c60 2022-06-17T12:54:00.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.924+0000 7f2b8ffff700 1 -- 172.21.15.36:0/4118462947 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b90149d70 con 0x7f2b90159c60 2022-06-17T12:54:00.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.924+0000 7f2b8ffff700 1 -- 172.21.15.36:0/4118462947 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b901428e0 con 0x7f2b90159c60 2022-06-17T12:54:00.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.928+0000 7f2b8ffff700 1 -- 172.21.15.36:0/4118462947 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b900ad330 con 0x7f2b90159c60 2022-06-17T12:54:00.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.928+0000 7f2b7ffff700 1 -- 172.21.15.36:0/4118462947 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2b80010070 con 0x7f2b90159c60 2022-06-17T12:54:00.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.928+0000 7f2b7ffff700 1 --2- 172.21.15.36:0/4118462947 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b6c053a60 0x7f2b6c055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:00.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.928+0000 7f2b7ffff700 1 -- 172.21.15.36:0/4118462947 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f2b8006c950 con 0x7f2b90159c60 2022-06-17T12:54:00.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.928+0000 7f2b8effd700 1 --2- 172.21.15.36:0/4118462947 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b6c053a60 0x7f2b6c055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:00.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.928+0000 7f2b8effd700 1 --2- 172.21.15.36:0/4118462947 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b6c053a60 0x7f2b6c055f20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f2b8400b000 tx=0x7f2b84006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:00.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:00.936+0000 7f2b7ffff700 1 -- 172.21.15.36:0/4118462947 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2b80043730 con 0x7f2b90159c60 2022-06-17T12:54:01.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.100+0000 7f2b8ffff700 1 -- 172.21.15.36:0/4118462947 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b90057410 con 0x7f2b90159c60 2022-06-17T12:54:01.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.104+0000 7f2b7ffff700 1 -- 172.21.15.36:0/4118462947 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f2b80023070 con 0x7f2b90159c60 2022-06-17T12:54:01.091 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:01.091 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":36,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/2541998896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2541998896},{"type":"v1","addr":"172.21.15.70:6833","nonce":2541998896}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:53:04.907795+0000","modified":"2022-06-17T12:53:06.511529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6381":{"gid":6381,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/2514005995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2514005995},{"type":"v1","addr":"172.21.15.36:6835","nonce":2514005995}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-06-17T12:54:01.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.104+0000 7f2b8ffff700 1 -- 172.21.15.36:0/4118462947 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b6c053a60 msgr2=0x7f2b6c055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:01.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.104+0000 7f2b8ffff700 1 --2- 172.21.15.36:0/4118462947 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b6c053a60 0x7f2b6c055f20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f2b8400b000 tx=0x7f2b84006df0).stop 2022-06-17T12:54:01.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.104+0000 7f2b8ffff700 1 -- 172.21.15.36:0/4118462947 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b90159c60 msgr2=0x7f2b901452d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:01.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.104+0000 7f2b8ffff700 1 --2- 172.21.15.36:0/4118462947 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b90159c60 0x7f2b901452d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f2b800016f0 tx=0x7f2b80015000).stop 2022-06-17T12:54:01.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.104+0000 7f2b8ffff700 1 -- 172.21.15.36:0/4118462947 shutdown_connections 2022-06-17T12:54:01.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.104+0000 7f2b8ffff700 1 --2- 172.21.15.36:0/4118462947 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b6c053a60 0x7f2b6c055f20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:01.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.104+0000 7f2b8ffff700 1 --2- 172.21.15.36:0/4118462947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b901498c0 0x7f2b90142250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:01.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.104+0000 7f2b8ffff700 1 --2- 172.21.15.36:0/4118462947 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b90159c60 0x7f2b901452d0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:01.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.104+0000 7f2b8ffff700 1 --2- 172.21.15.36:0/4118462947 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b74004500 0x7f2b90144710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:01.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.104+0000 7f2b8ffff700 1 -- 172.21.15.36:0/4118462947 >> 172.21.15.36:0/4118462947 conn(0x7f2b9005f770 msgr2=0x7f2b900b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:01.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.104+0000 7f2b8ffff700 1 -- 172.21.15.36:0/4118462947 shutdown_connections 2022-06-17T12:54:01.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.104+0000 7f2b8ffff700 1 -- 172.21.15.36:0/4118462947 wait complete. 2022-06-17T12:54:01.096 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 36 2022-06-17T12:54:01.104 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-17T12:54:01.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.216+0000 7f6257fff700 1 Processor -- start 2022-06-17T12:54:01.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.216+0000 7f6257fff700 1 -- start start 2022-06-17T12:54:01.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.216+0000 7f6257fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f6258063590 0x7f6258144ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:01.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.216+0000 7f6257fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62580b6d30 0x7f625813a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:01.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.216+0000 7f6257fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f625813ada0 0x7f6258140f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:01.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.216+0000 7f6257fff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f625805f770 con 0x7f6258063590 2022-06-17T12:54:01.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.216+0000 7f6257fff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f625805fa10 con 0x7f625813ada0 2022-06-17T12:54:01.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.216+0000 7f6257fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625805fcb0 con 0x7f62580b6d30 2022-06-17T12:54:01.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.216+0000 7f6256ffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f6258063590 0x7f6258144ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:01.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.216+0000 7f6256ffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f6258063590 0x7f6258144ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35980/0 (socket says 172.21.15.36:35980) 2022-06-17T12:54:01.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.216+0000 7f6256ffd700 1 -- 172.21.15.36:0/539520206 learned_addr learned my addr 172.21.15.36:0/539520206 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:01.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.216+0000 7f62567fc700 1 --2- 172.21.15.36:0/539520206 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62580b6d30 0x7f625813a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:01.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.216+0000 7f62577fe700 1 --2- 172.21.15.36:0/539520206 >> v2:172.21.15.70:3300/0 conn(0x7f625813ada0 0x7f6258140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:01.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.216+0000 7f6256ffd700 1 -- 172.21.15.36:0/539520206 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62580b6d30 msgr2=0x7f625813a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:01.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.216+0000 7f6256ffd700 1 --2- 172.21.15.36:0/539520206 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62580b6d30 0x7f625813a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:01.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.216+0000 7f6256ffd700 1 -- 172.21.15.36:0/539520206 >> v2:172.21.15.70:3300/0 conn(0x7f625813ada0 msgr2=0x7f6258140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:01.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.216+0000 7f6256ffd700 1 --2- 172.21.15.36:0/539520206 >> v2:172.21.15.70:3300/0 conn(0x7f625813ada0 0x7f6258140f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:01.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.216+0000 7f6256ffd700 1 -- 172.21.15.36:0/539520206 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6258142190 con 0x7f6258063590 2022-06-17T12:54:01.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.216+0000 7f62577fe700 1 --2- 172.21.15.36:0/539520206 >> v2:172.21.15.70:3300/0 conn(0x7f625813ada0 0x7f6258140f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:01.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.220+0000 7f6256ffd700 1 --2- 172.21.15.36:0/539520206 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6258063590 0x7f6258144ca0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f6248002f70 tx=0x7f6248006eb0).ready entity=mon.0 client_cookie=e04031762be16645 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:01.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.220+0000 7f6255ffb700 1 -- 172.21.15.36:0/539520206 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f624800d570 con 0x7f6258063590 2022-06-17T12:54:01.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.220+0000 7f6255ffb700 1 -- 172.21.15.36:0/539520206 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6248000e90 con 0x7f6258063590 2022-06-17T12:54:01.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.220+0000 7f6255ffb700 1 -- 172.21.15.36:0/539520206 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f624800d900 con 0x7f6258063590 2022-06-17T12:54:01.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.220+0000 7f6257fff700 1 -- 172.21.15.36:0/539520206 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6258063590 msgr2=0x7f6258144ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:01.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.220+0000 7f6257fff700 1 --2- 172.21.15.36:0/539520206 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6258063590 0x7f6258144ca0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f6248002f70 tx=0x7f6248006eb0).stop 2022-06-17T12:54:01.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.220+0000 7f6257fff700 1 -- 172.21.15.36:0/539520206 shutdown_connections 2022-06-17T12:54:01.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.220+0000 7f6257fff700 1 --2- 172.21.15.36:0/539520206 >> v2:172.21.15.70:3300/0 conn(0x7f625813ada0 0x7f6258140f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:01.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.220+0000 7f6257fff700 1 --2- 172.21.15.36:0/539520206 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62580b6d30 0x7f625813a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:01.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.220+0000 7f6257fff700 1 --2- 172.21.15.36:0/539520206 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6258063590 0x7f6258144ca0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:01.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.220+0000 7f6257fff700 1 -- 172.21.15.36:0/539520206 >> 172.21.15.36:0/539520206 conn(0x7f62580614b0 msgr2=0x7f62580618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:01.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.220+0000 7f6257fff700 1 -- 172.21.15.36:0/539520206 shutdown_connections 2022-06-17T12:54:01.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.220+0000 7f6257fff700 1 -- 172.21.15.36:0/539520206 wait complete. 2022-06-17T12:54:01.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.224+0000 7f6257fff700 1 Processor -- start 2022-06-17T12:54:01.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.224+0000 7f6257fff700 1 -- start start 2022-06-17T12:54:01.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.224+0000 7f6257fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6258063590 0x7f625813c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:01.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.224+0000 7f6257fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62580b6d30 0x7f6258138900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:01.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.224+0000 7f6257fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f625813ada0 0x7f625813ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:01.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.224+0000 7f6257fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62581380b0 con 0x7f625813ada0 2022-06-17T12:54:01.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.224+0000 7f6257fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6258140620 con 0x7f62580b6d30 2022-06-17T12:54:01.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.224+0000 7f6257fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6258140380 con 0x7f6258063590 2022-06-17T12:54:01.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.224+0000 7f62577fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f625813ada0 0x7f625813ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:01.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.224+0000 7f62577fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f625813ada0 0x7f625813ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:35990/0 (socket says 172.21.15.36:35990) 2022-06-17T12:54:01.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.224+0000 7f62577fe700 1 -- 172.21.15.36:0/1184828508 learned_addr learned my addr 172.21.15.36:0/1184828508 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:01.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.224+0000 7f6256ffd700 1 --2- 172.21.15.36:0/1184828508 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6258063590 0x7f625813c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:01.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.224+0000 7f62567fc700 1 --2- 172.21.15.36:0/1184828508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62580b6d30 0x7f6258138900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:01.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.224+0000 7f62577fe700 1 -- 172.21.15.36:0/1184828508 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6258063590 msgr2=0x7f625813c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:01.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.224+0000 7f62577fe700 1 --2- 172.21.15.36:0/1184828508 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6258063590 0x7f625813c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:01.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.224+0000 7f62577fe700 1 -- 172.21.15.36:0/1184828508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62580b6d30 msgr2=0x7f6258138900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:01.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.224+0000 7f62577fe700 1 --2- 172.21.15.36:0/1184828508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62580b6d30 0x7f6258138900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:01.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.224+0000 7f62577fe700 1 -- 172.21.15.36:0/1184828508 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f624800b1e0 con 0x7f625813ada0 2022-06-17T12:54:01.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.224+0000 7f62577fe700 1 --2- 172.21.15.36:0/1184828508 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f625813ada0 0x7f625813ec80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f62581381f0 tx=0x7f6250008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:01.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.224+0000 7f6247fff700 1 -- 172.21.15.36:0/1184828508 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6250011780 con 0x7f625813ada0 2022-06-17T12:54:01.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.224+0000 7f6247fff700 1 -- 172.21.15.36:0/1184828508 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62500118e0 con 0x7f625813ada0 2022-06-17T12:54:01.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.224+0000 7f6257fff700 1 -- 172.21.15.36:0/1184828508 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f625813cba0 con 0x7f625813ada0 2022-06-17T12:54:01.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.224+0000 7f6247fff700 1 -- 172.21.15.36:0/1184828508 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6250011c40 con 0x7f625813ada0 2022-06-17T12:54:01.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.224+0000 7f6257fff700 1 -- 172.21.15.36:0/1184828508 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f625813f170 con 0x7f625813ada0 2022-06-17T12:54:01.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.228+0000 7f6257fff700 1 -- 172.21.15.36:0/1184828508 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f625813cd60 con 0x7f625813ada0 2022-06-17T12:54:01.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.228+0000 7f6247fff700 1 -- 172.21.15.36:0/1184828508 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f625001c460 con 0x7f625813ada0 2022-06-17T12:54:01.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.228+0000 7f6247fff700 1 --2- 172.21.15.36:0/1184828508 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6234053a40 0x7f6234055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:01.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.228+0000 7f6247fff700 1 -- 172.21.15.36:0/1184828508 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f625006c5c0 con 0x7f625813ada0 2022-06-17T12:54:01.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.228+0000 7f6256ffd700 1 --2- 172.21.15.36:0/1184828508 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6234053a40 0x7f6234055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:01.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.228+0000 7f6256ffd700 1 --2- 172.21.15.36:0/1184828508 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6234053a40 0x7f6234055f00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f624801c000 tx=0x7f6248015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:01.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.232+0000 7f6247fff700 1 -- 172.21.15.36:0/1184828508 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f62500431b0 con 0x7f625813ada0 2022-06-17T12:54:01.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.400+0000 7f6257fff700 1 -- 172.21.15.36:0/1184828508 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6258057410 con 0x7f625813ada0 2022-06-17T12:54:01.543 INFO:tasks.ceph.mon.a.smithi036.stderr:2022-06-17T12:54:01.548+0000 7fb7a2649700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-17T12:54:01.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.564+0000 7f6247fff700 1 -- 172.21.15.36:0/1184828508 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) v1 ==== 161+0+0 (secure 0 0 0) 0x7f62500467d0 con 0x7f625813ada0 2022-06-17T12:54:01.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.568+0000 7f6257fff700 1 -- 172.21.15.36:0/1184828508 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6234053a40 msgr2=0x7f6234055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:01.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.568+0000 7f6257fff700 1 --2- 172.21.15.36:0/1184828508 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6234053a40 0x7f6234055f00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f624801c000 tx=0x7f6248015000).stop 2022-06-17T12:54:01.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.568+0000 7f6257fff700 1 -- 172.21.15.36:0/1184828508 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f625813ada0 msgr2=0x7f625813ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:01.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.568+0000 7f6257fff700 1 --2- 172.21.15.36:0/1184828508 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f625813ada0 0x7f625813ec80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f62581381f0 tx=0x7f6250008040).stop 2022-06-17T12:54:01.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.568+0000 7f6257fff700 1 -- 172.21.15.36:0/1184828508 shutdown_connections 2022-06-17T12:54:01.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.568+0000 7f6257fff700 1 --2- 172.21.15.36:0/1184828508 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6234053a40 0x7f6234055f00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:01.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.568+0000 7f6257fff700 1 --2- 172.21.15.36:0/1184828508 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f625813ada0 0x7f625813ec80 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:01.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.568+0000 7f6257fff700 1 --2- 172.21.15.36:0/1184828508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62580b6d30 0x7f6258138900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:01.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.568+0000 7f6257fff700 1 --2- 172.21.15.36:0/1184828508 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6258063590 0x7f625813c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:01.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.568+0000 7f6257fff700 1 -- 172.21.15.36:0/1184828508 >> 172.21.15.36:0/1184828508 conn(0x7f62580614b0 msgr2=0x7f6258140e50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:01.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.568+0000 7f6257fff700 1 -- 172.21.15.36:0/1184828508 shutdown_connections 2022-06-17T12:54:01.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.568+0000 7f6257fff700 1 -- 172.21.15.36:0/1184828508 wait complete. 2022-06-17T12:54:01.572 INFO:teuthology.orchestra.run.smithi036.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-17T12:54:01.575 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:54:01.583 INFO:tasks.ceph.mds.a.smithi036.stdout:starting mds.a at 2022-06-17T12:54:01.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb46d7e1700 1 Processor -- start 2022-06-17T12:54:01.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb46d7e1700 1 -- start start 2022-06-17T12:54:01.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb46d7e1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4680b1ff0 0x7fb46813c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:01.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb46d7e1700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4680b0230 con 0x7fb468060ac0 2022-06-17T12:54:01.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb46d7e1700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb468060ea0 con 0x7fb4680b1890 2022-06-17T12:54:01.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb46d7e1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb468143800 con 0x7fb4680b1ff0 2022-06-17T12:54:01.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb467fff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fb468060ac0 0x7fb46813c2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53536/0 (socket says 172.21.15.36:53536) 2022-06-17T12:54:01.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb467fff700 1 -- 172.21.15.36:0/684659588 learned_addr learned my addr 172.21.15.36:0/684659588 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:01.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb46cfe0700 1 --2- 172.21.15.36:0/684659588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4680b1ff0 0x7fb46813c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:01.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb466ffd700 1 -- 172.21.15.36:0/684659588 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3792115837 0 0) 0x7fb4680b0230 con 0x7fb468060ac0 2022-06-17T12:54:01.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb466ffd700 1 -- 172.21.15.36:0/684659588 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb454003260 con 0x7fb468060ac0 2022-06-17T12:54:01.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb466ffd700 1 -- 172.21.15.36:0/684659588 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443134708 0 0) 0x7fb468060ea0 con 0x7fb4680b1890 2022-06-17T12:54:01.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb466ffd700 1 -- 172.21.15.36:0/684659588 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4680b0230 con 0x7fb4680b1890 2022-06-17T12:54:01.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb46cfe0700 1 -- 172.21.15.36:0/684659588 >> v1:172.21.15.70:6789/0 conn(0x7fb4680b1890 legacy=0x7fb46813f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:01.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb46cfe0700 1 -- 172.21.15.36:0/684659588 >> v1:172.21.15.36:6789/0 conn(0x7fb468060ac0 legacy=0x7fb46813c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:01.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb46cfe0700 1 -- 172.21.15.36:0/684659588 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4680afcf0 con 0x7fb4680b1ff0 2022-06-17T12:54:01.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb46cfe0700 1 --2- 172.21.15.36:0/684659588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4680b1ff0 0x7fb46813c9c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fb460002f70 tx=0x7fb460006df0).ready entity=mon.2 client_cookie=f97a3f222ba8b983 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:01.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb466ffd700 1 -- 172.21.15.36:0/684659588 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb46000d5c0 con 0x7fb4680b1ff0 2022-06-17T12:54:01.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb466ffd700 1 -- 172.21.15.36:0/684659588 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb460000e90 con 0x7fb4680b1ff0 2022-06-17T12:54:01.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb466ffd700 1 -- 172.21.15.36:0/684659588 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb46000d910 con 0x7fb4680b1ff0 2022-06-17T12:54:01.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb46d7e1700 1 -- 172.21.15.36:0/684659588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4680b1ff0 msgr2=0x7fb46813c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:01.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb46d7e1700 1 --2- 172.21.15.36:0/684659588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4680b1ff0 0x7fb46813c9c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fb460002f70 tx=0x7fb460006df0).stop 2022-06-17T12:54:01.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb46d7e1700 1 -- 172.21.15.36:0/684659588 shutdown_connections 2022-06-17T12:54:01.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb46d7e1700 1 --2- 172.21.15.36:0/684659588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4680b1ff0 0x7fb46813c9c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:01.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.692+0000 7fb46d7e1700 1 -- 172.21.15.36:0/684659588 >> 172.21.15.36:0/684659588 conn(0x7fb4680b65f0 msgr2=0x7fb4680b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:01.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb46d7e1700 1 -- 172.21.15.36:0/684659588 shutdown_connections 2022-06-17T12:54:01.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb46d7e1700 1 -- 172.21.15.36:0/684659588 wait complete. 2022-06-17T12:54:01.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb46d7e1700 1 Processor -- start 2022-06-17T12:54:01.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb46d7e1700 1 -- start start 2022-06-17T12:54:01.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb46d7e1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb468060ac0 0x7fb46813fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:01.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb46d7e1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4680b1890 0x7fb468064590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:01.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb46d7e1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4680b1ff0 0x7fb468066320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:01.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb46d7e1700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb468062b70 con 0x7fb468060ac0 2022-06-17T12:54:01.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb46d7e1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4680669e0 con 0x7fb4680b1ff0 2022-06-17T12:54:01.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb46d7e1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb468066c80 con 0x7fb4680b1890 2022-06-17T12:54:01.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb467fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb468060ac0 0x7fb46813fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:01.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb467fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb468060ac0 0x7fb46813fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36012/0 (socket says 172.21.15.36:36012) 2022-06-17T12:54:01.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb467fff700 1 -- 172.21.15.36:0/2685597691 learned_addr learned my addr 172.21.15.36:0/2685597691 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:01.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb4677fe700 1 --2- 172.21.15.36:0/2685597691 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4680b1890 0x7fb468064590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:01.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb46cfe0700 1 --2- 172.21.15.36:0/2685597691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4680b1ff0 0x7fb468066320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:01.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb467fff700 1 -- 172.21.15.36:0/2685597691 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4680b1890 msgr2=0x7fb468064590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:01.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb467fff700 1 --2- 172.21.15.36:0/2685597691 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4680b1890 0x7fb468064590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:01.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb467fff700 1 -- 172.21.15.36:0/2685597691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4680b1ff0 msgr2=0x7fb468066320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:01.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb467fff700 1 --2- 172.21.15.36:0/2685597691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4680b1ff0 0x7fb468066320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:01.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb467fff700 1 -- 172.21.15.36:0/2685597691 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46000b180 con 0x7fb468060ac0 2022-06-17T12:54:01.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb467fff700 1 --2- 172.21.15.36:0/2685597691 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb468060ac0 0x7fb46813fbc0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7fb45c001110 tx=0x7fb45c007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:01.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb4657fa700 1 -- 172.21.15.36:0/2685597691 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb45c0070d0 con 0x7fb468060ac0 2022-06-17T12:54:01.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb4657fa700 1 -- 172.21.15.36:0/2685597691 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45c007230 con 0x7fb468060ac0 2022-06-17T12:54:01.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb4657fa700 1 -- 172.21.15.36:0/2685597691 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb45c005610 con 0x7fb468060ac0 2022-06-17T12:54:01.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb46d7e1700 1 -- 172.21.15.36:0/2685597691 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb468062f60 con 0x7fb468060ac0 2022-06-17T12:54:01.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.696+0000 7fb46d7e1700 1 -- 172.21.15.36:0/2685597691 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb46813d3a0 con 0x7fb468060ac0 2022-06-17T12:54:01.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.700+0000 7fb46d7e1700 1 -- 172.21.15.36:0/2685597691 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb468066810 con 0x7fb468060ac0 2022-06-17T12:54:01.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.700+0000 7fb4657fa700 1 -- 172.21.15.36:0/2685597691 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fb45c010070 con 0x7fb468060ac0 2022-06-17T12:54:01.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.700+0000 7fb4657fa700 1 --2- 172.21.15.36:0/2685597691 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb448053a60 0x7fb448055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:01.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.700+0000 7fb4657fa700 1 -- 172.21.15.36:0/2685597691 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fb45c06b700 con 0x7fb468060ac0 2022-06-17T12:54:01.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.700+0000 7fb4677fe700 1 --2- 172.21.15.36:0/2685597691 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb448053a60 0x7fb448055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:01.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.700+0000 7fb4677fe700 1 --2- 172.21.15.36:0/2685597691 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb448053a60 0x7fb448055f20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7fb45800d000 tx=0x7fb458007d80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:01.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.708+0000 7fb4657fa700 1 -- 172.21.15.36:0/2685597691 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb45c042480 con 0x7fb468060ac0 2022-06-17T12:54:01.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.872+0000 7fb46d7e1700 1 -- 172.21.15.36:0/2685597691 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4680aff90 con 0x7fb468060ac0 2022-06-17T12:54:01.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.872+0000 7fb4657fa700 1 -- 172.21.15.36:0/2685597691 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+1779 (secure 0 0 0) 0x7fb45c018090 con 0x7fb468060ac0 2022-06-17T12:54:01.861 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:01.861 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":37,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/2541998896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2541998896},{"type":"v1","addr":"172.21.15.70:6833","nonce":2541998896}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":37,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:53:04.907795+0000","modified":"2022-06-17T12:54:01.554489+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":76,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-06-17T12:54:01.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.872+0000 7fb46d7e1700 1 -- 172.21.15.36:0/2685597691 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb448053a60 msgr2=0x7fb448055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:01.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.872+0000 7fb46d7e1700 1 --2- 172.21.15.36:0/2685597691 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb448053a60 0x7fb448055f20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7fb45800d000 tx=0x7fb458007d80).stop 2022-06-17T12:54:01.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.872+0000 7fb46d7e1700 1 -- 172.21.15.36:0/2685597691 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb468060ac0 msgr2=0x7fb46813fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:01.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.872+0000 7fb46d7e1700 1 --2- 172.21.15.36:0/2685597691 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb468060ac0 0x7fb46813fbc0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7fb45c001110 tx=0x7fb45c007ad0).stop 2022-06-17T12:54:01.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.872+0000 7fb46d7e1700 1 -- 172.21.15.36:0/2685597691 shutdown_connections 2022-06-17T12:54:01.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.872+0000 7fb46d7e1700 1 --2- 172.21.15.36:0/2685597691 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb448053a60 0x7fb448055f20 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:01.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.872+0000 7fb46d7e1700 1 --2- 172.21.15.36:0/2685597691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4680b1ff0 0x7fb468066320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:01.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.872+0000 7fb46d7e1700 1 --2- 172.21.15.36:0/2685597691 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4680b1890 0x7fb468064590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:01.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.872+0000 7fb46d7e1700 1 --2- 172.21.15.36:0/2685597691 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb468060ac0 0x7fb46813fbc0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:01.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.872+0000 7fb46d7e1700 1 -- 172.21.15.36:0/2685597691 >> 172.21.15.36:0/2685597691 conn(0x7fb4680b65f0 msgr2=0x7fb4680b4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:01.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.872+0000 7fb46d7e1700 1 -- 172.21.15.36:0/2685597691 shutdown_connections 2022-06-17T12:54:01.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.872+0000 7fb46d7e1700 1 -- 172.21.15.36:0/2685597691 wait complete. 2022-06-17T12:54:01.868 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 37 2022-06-17T12:54:01.878 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-17T12:54:01.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.996+0000 7f3f5ca9f700 1 Processor -- start 2022-06-17T12:54:01.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.996+0000 7f3f5ca9f700 1 -- start start 2022-06-17T12:54:01.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.996+0000 7f3f5ca9f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f5814b820 0x7f3f5814dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:01.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.996+0000 7f3f5ca9f700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f580b4160 con 0x7f3f580b9860 2022-06-17T12:54:01.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.996+0000 7f3f5ca9f700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f580b9c40 con 0x7f3f580b45c0 2022-06-17T12:54:01.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.996+0000 7f3f5ca9f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f580b5120 con 0x7f3f5814b820 2022-06-17T12:54:01.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.996+0000 7f3f577fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f3f580b9860 0x7f3f58143840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53550/0 (socket says 172.21.15.36:53550) 2022-06-17T12:54:01.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.996+0000 7f3f577fe700 1 -- 172.21.15.36:0/214255850 learned_addr learned my addr 172.21.15.36:0/214255850 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:01.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.996+0000 7f3f57fff700 1 --2- 172.21.15.36:0/214255850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f5814b820 0x7f3f5814dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:01.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.996+0000 7f3f567fc700 1 -- 172.21.15.36:0/214255850 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2753587635 0 0) 0x7f3f580b4160 con 0x7f3f580b9860 2022-06-17T12:54:01.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.996+0000 7f3f567fc700 1 -- 172.21.15.36:0/214255850 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f38003260 con 0x7f3f580b9860 2022-06-17T12:54:01.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.996+0000 7f3f567fc700 1 -- 172.21.15.36:0/214255850 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1088773371 0 0) 0x7f3f580b9c40 con 0x7f3f580b45c0 2022-06-17T12:54:01.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.996+0000 7f3f567fc700 1 -- 172.21.15.36:0/214255850 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f580b4160 con 0x7f3f580b45c0 2022-06-17T12:54:01.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.996+0000 7f3f567fc700 1 -- 172.21.15.36:0/214255850 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 67656899 0 0) 0x7f3f38003260 con 0x7f3f580b9860 2022-06-17T12:54:01.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.996+0000 7f3f57fff700 1 -- 172.21.15.36:0/214255850 >> v1:172.21.15.70:6789/0 conn(0x7f3f580b45c0 legacy=0x7f3f580b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:01.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.996+0000 7f3f57fff700 1 -- 172.21.15.36:0/214255850 >> v1:172.21.15.36:6789/0 conn(0x7f3f580b9860 legacy=0x7f3f58143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:01.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:01.996+0000 7f3f57fff700 1 -- 172.21.15.36:0/214255850 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f580627d0 con 0x7f3f5814b820 2022-06-17T12:54:01.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.000+0000 7f3f57fff700 1 --2- 172.21.15.36:0/214255850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f5814b820 0x7f3f5814dc10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f3f48001110 tx=0x7f3f500091d0).ready entity=mon.2 client_cookie=72aa059a26052a60 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:01.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.000+0000 7f3f567fc700 1 -- 172.21.15.36:0/214255850 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f48001660 con 0x7f3f5814b820 2022-06-17T12:54:01.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.000+0000 7f3f567fc700 1 -- 172.21.15.36:0/214255850 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5000f510 con 0x7f3f5814b820 2022-06-17T12:54:01.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.000+0000 7f3f567fc700 1 -- 172.21.15.36:0/214255850 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f5000f680 con 0x7f3f5814b820 2022-06-17T12:54:01.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.000+0000 7f3f5ca9f700 1 -- 172.21.15.36:0/214255850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f5814b820 msgr2=0x7f3f5814dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:01.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.000+0000 7f3f5ca9f700 1 --2- 172.21.15.36:0/214255850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f5814b820 0x7f3f5814dc10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f3f48001110 tx=0x7f3f500091d0).stop 2022-06-17T12:54:01.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.000+0000 7f3f5ca9f700 1 -- 172.21.15.36:0/214255850 shutdown_connections 2022-06-17T12:54:01.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.000+0000 7f3f5ca9f700 1 --2- 172.21.15.36:0/214255850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f5814b820 0x7f3f5814dc10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:01.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.000+0000 7f3f5ca9f700 1 -- 172.21.15.36:0/214255850 >> 172.21.15.36:0/214255850 conn(0x7f3f5805c550 msgr2=0x7f3f58060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:01.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.000+0000 7f3f5ca9f700 1 -- 172.21.15.36:0/214255850 shutdown_connections 2022-06-17T12:54:01.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.000+0000 7f3f5ca9f700 1 -- 172.21.15.36:0/214255850 wait complete. 2022-06-17T12:54:01.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.000+0000 7f3f5ca9f700 1 Processor -- start 2022-06-17T12:54:01.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.000+0000 7f3f5ca9f700 1 -- start start 2022-06-17T12:54:01.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.004+0000 7f3f5ca9f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3f580b45c0 0x7f3f58144c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:01.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.004+0000 7f3f5ca9f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f580b9860 0x7f3f58145110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:01.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.004+0000 7f3f5ca9f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f5814b820 0x7f3f5814a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:01.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.004+0000 7f3f5ca9f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f58146c10 con 0x7f3f580b45c0 2022-06-17T12:54:01.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.004+0000 7f3f5ca9f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5813d1a0 con 0x7f3f5814b820 2022-06-17T12:54:01.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.004+0000 7f3f5ca9f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f58146e40 con 0x7f3f580b9860 2022-06-17T12:54:01.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.004+0000 7f3f577fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3f580b45c0 0x7f3f58144c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:02.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.004+0000 7f3f577fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3f580b45c0 0x7f3f58144c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36030/0 (socket says 172.21.15.36:36030) 2022-06-17T12:54:02.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.004+0000 7f3f577fe700 1 -- 172.21.15.36:0/3803135156 learned_addr learned my addr 172.21.15.36:0/3803135156 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:02.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.004+0000 7f3f56ffd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f580b9860 0x7f3f58145110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:02.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.004+0000 7f3f57fff700 1 --2- 172.21.15.36:0/3803135156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f5814b820 0x7f3f5814a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:02.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.004+0000 7f3f577fe700 1 -- 172.21.15.36:0/3803135156 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f580b9860 msgr2=0x7f3f58145110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:02.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.004+0000 7f3f577fe700 1 --2- 172.21.15.36:0/3803135156 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f580b9860 0x7f3f58145110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:02.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.004+0000 7f3f577fe700 1 -- 172.21.15.36:0/3803135156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f5814b820 msgr2=0x7f3f5814a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:02.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.004+0000 7f3f577fe700 1 --2- 172.21.15.36:0/3803135156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f5814b820 0x7f3f5814a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:02.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.004+0000 7f3f577fe700 1 -- 172.21.15.36:0/3803135156 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f50009470 con 0x7f3f580b45c0 2022-06-17T12:54:02.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.004+0000 7f3f577fe700 1 --2- 172.21.15.36:0/3803135156 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3f580b45c0 0x7f3f58144c20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7f3f48000fe0 tx=0x7f3f48001b60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:02.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.004+0000 7f3f54ff9700 1 -- 172.21.15.36:0/3803135156 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f48007170 con 0x7f3f580b45c0 2022-06-17T12:54:02.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.004+0000 7f3f54ff9700 1 -- 172.21.15.36:0/3803135156 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f480072d0 con 0x7f3f580b45c0 2022-06-17T12:54:02.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.004+0000 7f3f5ca9f700 1 -- 172.21.15.36:0/3803135156 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f58140390 con 0x7f3f580b45c0 2022-06-17T12:54:02.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.004+0000 7f3f54ff9700 1 -- 172.21.15.36:0/3803135156 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f48004920 con 0x7f3f580b45c0 2022-06-17T12:54:02.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.004+0000 7f3f5ca9f700 1 -- 172.21.15.36:0/3803135156 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f58145d20 con 0x7f3f580b45c0 2022-06-17T12:54:02.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.008+0000 7f3f5ca9f700 1 -- 172.21.15.36:0/3803135156 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f58062a70 con 0x7f3f580b45c0 2022-06-17T12:54:02.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.008+0000 7f3f54ff9700 1 -- 172.21.15.36:0/3803135156 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f3f4801b070 con 0x7f3f580b45c0 2022-06-17T12:54:02.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.008+0000 7f3f54ff9700 1 --2- 172.21.15.36:0/3803135156 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3f3c053ad0 0x7f3f3c055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:02.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.008+0000 7f3f54ff9700 1 -- 172.21.15.36:0/3803135156 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f3f4806b930 con 0x7f3f580b45c0 2022-06-17T12:54:02.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.008+0000 7f3f56ffd700 1 --2- 172.21.15.36:0/3803135156 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3f3c053ad0 0x7f3f3c055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:02.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.008+0000 7f3f56ffd700 1 --2- 172.21.15.36:0/3803135156 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3f3c053ad0 0x7f3f3c055f90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f3f4000d000 tx=0x7f3f40007a20).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:02.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.012+0000 7f3f54ff9700 1 -- 172.21.15.36:0/3803135156 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3f48042540 con 0x7f3f580b45c0 2022-06-17T12:54:02.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.176+0000 7f3f5ca9f700 1 -- 172.21.15.36:0/3803135156 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3f58057410 con 0x7f3f580b45c0 2022-06-17T12:54:02.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.576+0000 7f3f54ff9700 1 -- 172.21.15.36:0/3803135156 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3f48045b60 con 0x7f3f580b45c0 2022-06-17T12:54:02.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.580+0000 7f3f5ca9f700 1 -- 172.21.15.36:0/3803135156 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3f3c053ad0 msgr2=0x7f3f3c055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:02.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.580+0000 7f3f5ca9f700 1 --2- 172.21.15.36:0/3803135156 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3f3c053ad0 0x7f3f3c055f90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f3f4000d000 tx=0x7f3f40007a20).stop 2022-06-17T12:54:02.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.580+0000 7f3f5ca9f700 1 -- 172.21.15.36:0/3803135156 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3f580b45c0 msgr2=0x7f3f58144c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:02.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.580+0000 7f3f5ca9f700 1 --2- 172.21.15.36:0/3803135156 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3f580b45c0 0x7f3f58144c20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7f3f48000fe0 tx=0x7f3f48001b60).stop 2022-06-17T12:54:02.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.580+0000 7f3f5ca9f700 1 -- 172.21.15.36:0/3803135156 shutdown_connections 2022-06-17T12:54:02.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.580+0000 7f3f5ca9f700 1 --2- 172.21.15.36:0/3803135156 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3f3c053ad0 0x7f3f3c055f90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:02.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.580+0000 7f3f5ca9f700 1 --2- 172.21.15.36:0/3803135156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f5814b820 0x7f3f5814a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:02.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.580+0000 7f3f5ca9f700 1 --2- 172.21.15.36:0/3803135156 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f580b9860 0x7f3f58145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:02.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.580+0000 7f3f5ca9f700 1 --2- 172.21.15.36:0/3803135156 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3f580b45c0 0x7f3f58144c20 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:02.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.580+0000 7f3f5ca9f700 1 -- 172.21.15.36:0/3803135156 >> 172.21.15.36:0/3803135156 conn(0x7f3f5805c550 msgr2=0x7f3f580b2430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:02.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.580+0000 7f3f5ca9f700 1 -- 172.21.15.36:0/3803135156 shutdown_connections 2022-06-17T12:54:02.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.580+0000 7f3f5ca9f700 1 -- 172.21.15.36:0/3803135156 wait complete. 2022-06-17T12:54:02.585 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-17T12:54:02.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.712+0000 7fb910b0f700 1 Processor -- start 2022-06-17T12:54:02.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.712+0000 7fb910b0f700 1 -- start start 2022-06-17T12:54:02.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb910b0f700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb90c0b27b0 0x7fb90c13f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:02.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb910b0f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb90c0b0d70 0x7fb90c13f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:02.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb910b0f700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb90c0b2b90 con 0x7fb90c0b15f0 2022-06-17T12:54:02.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb910b0f700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb90c0610e0 con 0x7fb90c0b27b0 2022-06-17T12:54:02.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb910b0f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb90c0ac190 con 0x7fb90c0b0d70 2022-06-17T12:54:02.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb90bfff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fb90c0b15f0 0x7fb90c0ac780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53568/0 (socket says 172.21.15.36:53568) 2022-06-17T12:54:02.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb90bfff700 1 -- 172.21.15.36:0/2873982105 learned_addr learned my addr 172.21.15.36:0/2873982105 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:02.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb90b7fe700 1 --2- 172.21.15.36:0/2873982105 >> v2:172.21.15.70:3300/0 conn(0x7fb90c0b27b0 0x7fb90c13f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:02.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb902ffd700 1 --2- 172.21.15.36:0/2873982105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb90c0b0d70 0x7fb90c13f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:02.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb90affd700 1 -- 172.21.15.36:0/2873982105 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2560578900 0 0) 0x7fb90c0b2b90 con 0x7fb90c0b15f0 2022-06-17T12:54:02.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb90affd700 1 -- 172.21.15.36:0/2873982105 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8ec003260 con 0x7fb90c0b15f0 2022-06-17T12:54:02.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb90b7fe700 1 -- 172.21.15.36:0/2873982105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb90c0b0d70 msgr2=0x7fb90c13f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:02.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb90b7fe700 1 --2- 172.21.15.36:0/2873982105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb90c0b0d70 0x7fb90c13f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:02.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb90b7fe700 1 -- 172.21.15.36:0/2873982105 >> v1:172.21.15.36:6789/0 conn(0x7fb90c0b15f0 legacy=0x7fb90c0ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:02.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb90b7fe700 1 -- 172.21.15.36:0/2873982105 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb90c0ac430 con 0x7fb90c0b27b0 2022-06-17T12:54:02.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb90b7fe700 1 --2- 172.21.15.36:0/2873982105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb90c0b27b0 0x7fb90c13f110 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7fb8fc002f70 tx=0x7fb8fc006df0).ready entity=mon.1 client_cookie=654ddaac1f68e9e0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:02.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb90affd700 1 -- 172.21.15.36:0/2873982105 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8fc00d4a0 con 0x7fb90c0b27b0 2022-06-17T12:54:02.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb90affd700 1 -- 172.21.15.36:0/2873982105 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8fc000e90 con 0x7fb90c0b27b0 2022-06-17T12:54:02.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb90affd700 1 -- 172.21.15.36:0/2873982105 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8fc00d7f0 con 0x7fb90c0b27b0 2022-06-17T12:54:02.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb910b0f700 1 -- 172.21.15.36:0/2873982105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb90c0b27b0 msgr2=0x7fb90c13f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:02.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb910b0f700 1 --2- 172.21.15.36:0/2873982105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb90c0b27b0 0x7fb90c13f110 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7fb8fc002f70 tx=0x7fb8fc006df0).stop 2022-06-17T12:54:02.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb910b0f700 1 -- 172.21.15.36:0/2873982105 shutdown_connections 2022-06-17T12:54:02.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb910b0f700 1 --2- 172.21.15.36:0/2873982105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb90c0b0d70 0x7fb90c13f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:02.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb910b0f700 1 --2- 172.21.15.36:0/2873982105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb90c0b27b0 0x7fb90c13f110 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:02.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb910b0f700 1 -- 172.21.15.36:0/2873982105 >> 172.21.15.36:0/2873982105 conn(0x7fb90c0b5d10 msgr2=0x7fb90c0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:02.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb910b0f700 1 -- 172.21.15.36:0/2873982105 shutdown_connections 2022-06-17T12:54:02.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.716+0000 7fb910b0f700 1 -- 172.21.15.36:0/2873982105 wait complete. 2022-06-17T12:54:02.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.720+0000 7fb910b0f700 1 Processor -- start 2022-06-17T12:54:02.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.720+0000 7fb910b0f700 1 -- start start 2022-06-17T12:54:02.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.720+0000 7fb910b0f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb90c0b0d70 0x7fb90c0660d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:02.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.720+0000 7fb910b0f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb90c0b15f0 0x7fb90c13da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:02.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.720+0000 7fb910b0f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb90c0b27b0 0x7fb90c063780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:02.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.720+0000 7fb910b0f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90c0665c0 con 0x7fb90c0b27b0 2022-06-17T12:54:02.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.720+0000 7fb910b0f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90c062a70 con 0x7fb90c0b0d70 2022-06-17T12:54:02.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.724+0000 7fb910b0f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb90c0667e0 con 0x7fb90c0b15f0 2022-06-17T12:54:02.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.724+0000 7fb90bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb90c0b27b0 0x7fb90c063780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:02.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.724+0000 7fb90b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb90c0b0d70 0x7fb90c0660d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:02.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.724+0000 7fb90bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb90c0b27b0 0x7fb90c063780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36046/0 (socket says 172.21.15.36:36046) 2022-06-17T12:54:02.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.724+0000 7fb90bfff700 1 -- 172.21.15.36:0/2356229762 learned_addr learned my addr 172.21.15.36:0/2356229762 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:02.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.724+0000 7fb902ffd700 1 --2- 172.21.15.36:0/2356229762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb90c0b15f0 0x7fb90c13da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:02.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.724+0000 7fb90bfff700 1 -- 172.21.15.36:0/2356229762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb90c0b15f0 msgr2=0x7fb90c13da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:02.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.724+0000 7fb90bfff700 1 --2- 172.21.15.36:0/2356229762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb90c0b15f0 0x7fb90c13da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:02.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.724+0000 7fb90bfff700 1 -- 172.21.15.36:0/2356229762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb90c0b0d70 msgr2=0x7fb90c0660d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:02.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.724+0000 7fb90bfff700 1 --2- 172.21.15.36:0/2356229762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb90c0b0d70 0x7fb90c0660d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:02.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.724+0000 7fb90bfff700 1 -- 172.21.15.36:0/2356229762 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8fc00b180 con 0x7fb90c0b27b0 2022-06-17T12:54:02.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.724+0000 7fb902ffd700 1 --2- 172.21.15.36:0/2356229762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb90c0b15f0 0x7fb90c13da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:02.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.724+0000 7fb90bfff700 1 --2- 172.21.15.36:0/2356229762 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb90c0b27b0 0x7fb90c063780 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7fb904001490 tx=0x7fb904007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:02.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.724+0000 7fb9097fa700 1 -- 172.21.15.36:0/2356229762 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb904005ba0 con 0x7fb90c0b27b0 2022-06-17T12:54:02.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.724+0000 7fb9097fa700 1 -- 172.21.15.36:0/2356229762 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb904005d00 con 0x7fb90c0b27b0 2022-06-17T12:54:02.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.724+0000 7fb9097fa700 1 -- 172.21.15.36:0/2356229762 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9040072c0 con 0x7fb90c0b27b0 2022-06-17T12:54:02.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.724+0000 7fb910b0f700 1 -- 172.21.15.36:0/2356229762 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb90c066a80 con 0x7fb90c0b27b0 2022-06-17T12:54:02.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.724+0000 7fb910b0f700 1 -- 172.21.15.36:0/2356229762 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb90c0626f0 con 0x7fb90c0b27b0 2022-06-17T12:54:02.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.724+0000 7fb910b0f700 1 -- 172.21.15.36:0/2356229762 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb90c1407b0 con 0x7fb90c0b27b0 2022-06-17T12:54:02.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.728+0000 7fb9097fa700 1 -- 172.21.15.36:0/2356229762 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fb904010070 con 0x7fb90c0b27b0 2022-06-17T12:54:02.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.728+0000 7fb9097fa700 1 --2- 172.21.15.36:0/2356229762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb8f0053ad0 0x7fb8f0055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:02.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.728+0000 7fb9097fa700 1 -- 172.21.15.36:0/2356229762 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fb90406b9f0 con 0x7fb90c0b27b0 2022-06-17T12:54:02.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.728+0000 7fb90b7fe700 1 --2- 172.21.15.36:0/2356229762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb8f0053ad0 0x7fb8f0055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:02.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.728+0000 7fb90b7fe700 1 --2- 172.21.15.36:0/2356229762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb8f0053ad0 0x7fb8f0055f90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fb8fc01a000 tx=0x7fb8fc014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:02.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.732+0000 7fb9097fa700 1 -- 172.21.15.36:0/2356229762 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb90406de60 con 0x7fb90c0b27b0 2022-06-17T12:54:02.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:02.860+0000 7fb910b0f700 1 -- 172.21.15.36:0/2356229762 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb90c057410 con 0x7fb90c0b27b0 2022-06-17T12:54:03.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.580+0000 7fb9097fa700 1 -- 172.21.15.36:0/2356229762 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v77) v1 ==== 185+0+0 (secure 0 0 0) 0x7fb904042770 con 0x7fb90c0b27b0 2022-06-17T12:54:03.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.584+0000 7fb910b0f700 1 -- 172.21.15.36:0/2356229762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb8f0053ad0 msgr2=0x7fb8f0055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:03.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.584+0000 7fb910b0f700 1 --2- 172.21.15.36:0/2356229762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb8f0053ad0 0x7fb8f0055f90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fb8fc01a000 tx=0x7fb8fc014000).stop 2022-06-17T12:54:03.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.584+0000 7fb910b0f700 1 -- 172.21.15.36:0/2356229762 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb90c0b27b0 msgr2=0x7fb90c063780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:03.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.584+0000 7fb910b0f700 1 --2- 172.21.15.36:0/2356229762 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb90c0b27b0 0x7fb90c063780 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7fb904001490 tx=0x7fb904007e00).stop 2022-06-17T12:54:03.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.584+0000 7fb910b0f700 1 -- 172.21.15.36:0/2356229762 shutdown_connections 2022-06-17T12:54:03.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.584+0000 7fb910b0f700 1 --2- 172.21.15.36:0/2356229762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb8f0053ad0 0x7fb8f0055f90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:03.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.584+0000 7fb910b0f700 1 --2- 172.21.15.36:0/2356229762 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb90c0b27b0 0x7fb90c063780 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:03.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.584+0000 7fb910b0f700 1 --2- 172.21.15.36:0/2356229762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb90c0b15f0 0x7fb90c13da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:03.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.584+0000 7fb910b0f700 1 --2- 172.21.15.36:0/2356229762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb90c0b0d70 0x7fb90c0660d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:03.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.584+0000 7fb910b0f700 1 -- 172.21.15.36:0/2356229762 >> 172.21.15.36:0/2356229762 conn(0x7fb90c0b5d10 msgr2=0x7fb90c134550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:03.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.584+0000 7fb910b0f700 1 -- 172.21.15.36:0/2356229762 shutdown_connections 2022-06-17T12:54:03.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.584+0000 7fb910b0f700 1 -- 172.21.15.36:0/2356229762 wait complete. 2022-06-17T12:54:03.576 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' removed 2022-06-17T12:54:03.590 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-17T12:54:03.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.716+0000 7fbff8ee7700 1 Processor -- start 2022-06-17T12:54:03.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.716+0000 7fbff8ee7700 1 -- start start 2022-06-17T12:54:03.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff8ee7700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbff40620d0 0x7fbff40b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:03.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff8ee7700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbff40b31c0 0x7fbff40b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:03.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff8ee7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbff40b4700 0x7fbff40b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:03.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff8ee7700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff40b21d0 con 0x7fbff40b31c0 2022-06-17T12:54:03.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff8ee7700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff40b5080 con 0x7fbff40620d0 2022-06-17T12:54:03.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff8ee7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff40b2430 con 0x7fbff40b4700 2022-06-17T12:54:03.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff1d9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbff40b31c0 0x7fbff40b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:03.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff1d9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbff40b31c0 0x7fbff40b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36050/0 (socket says 172.21.15.36:36050) 2022-06-17T12:54:03.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff1d9b700 1 -- 172.21.15.36:0/4219670030 learned_addr learned my addr 172.21.15.36:0/4219670030 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:03.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff259c700 1 --2- 172.21.15.36:0/4219670030 >> v2:172.21.15.70:3300/0 conn(0x7fbff40620d0 0x7fbff40b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:03.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff2d9d700 1 --2- 172.21.15.36:0/4219670030 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbff40b4700 0x7fbff40b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:03.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff1d9b700 1 -- 172.21.15.36:0/4219670030 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbff40b4700 msgr2=0x7fbff40b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:03.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff1d9b700 1 --2- 172.21.15.36:0/4219670030 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbff40b4700 0x7fbff40b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:03.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff1d9b700 1 -- 172.21.15.36:0/4219670030 >> v2:172.21.15.70:3300/0 conn(0x7fbff40620d0 msgr2=0x7fbff40b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:03.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff1d9b700 1 --2- 172.21.15.36:0/4219670030 >> v2:172.21.15.70:3300/0 conn(0x7fbff40620d0 0x7fbff40b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:03.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff1d9b700 1 -- 172.21.15.36:0/4219670030 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff40b26d0 con 0x7fbff40b31c0 2022-06-17T12:54:03.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff1d9b700 1 --2- 172.21.15.36:0/4219670030 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbff40b31c0 0x7fbff40b2b70 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7fbfe4002f70 tx=0x7fbfe4006eb0).ready entity=mon.0 client_cookie=5d46cfb5b2c392e3 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:03.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff159a700 1 -- 172.21.15.36:0/4219670030 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfe400d570 con 0x7fbff40b31c0 2022-06-17T12:54:03.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff159a700 1 -- 172.21.15.36:0/4219670030 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe4000e90 con 0x7fbff40b31c0 2022-06-17T12:54:03.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff159a700 1 -- 172.21.15.36:0/4219670030 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfe400d900 con 0x7fbff40b31c0 2022-06-17T12:54:03.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff8ee7700 1 -- 172.21.15.36:0/4219670030 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbff40b31c0 msgr2=0x7fbff40b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:03.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff8ee7700 1 --2- 172.21.15.36:0/4219670030 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbff40b31c0 0x7fbff40b2b70 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7fbfe4002f70 tx=0x7fbfe4006eb0).stop 2022-06-17T12:54:03.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff8ee7700 1 -- 172.21.15.36:0/4219670030 shutdown_connections 2022-06-17T12:54:03.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff8ee7700 1 --2- 172.21.15.36:0/4219670030 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbff40b4700 0x7fbff40b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:03.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff8ee7700 1 --2- 172.21.15.36:0/4219670030 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbff40b31c0 0x7fbff40b2b70 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:03.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff8ee7700 1 --2- 172.21.15.36:0/4219670030 >> v2:172.21.15.70:3300/0 conn(0x7fbff40620d0 0x7fbff40b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:03.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff8ee7700 1 -- 172.21.15.36:0/4219670030 >> 172.21.15.36:0/4219670030 conn(0x7fbff40b9850 msgr2=0x7fbff40b9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:03.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff8ee7700 1 -- 172.21.15.36:0/4219670030 shutdown_connections 2022-06-17T12:54:03.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff8ee7700 1 -- 172.21.15.36:0/4219670030 wait complete. 2022-06-17T12:54:03.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.720+0000 7fbff8ee7700 1 Processor -- start 2022-06-17T12:54:03.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.724+0000 7fbff8ee7700 1 -- start start 2022-06-17T12:54:03.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.724+0000 7fbff8ee7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbff40620d0 0x7fbff4112be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:03.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.724+0000 7fbff8ee7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbff40b31c0 0x7fbff4152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:03.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.724+0000 7fbff8ee7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbff40b4700 0x7fbff4153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:03.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.724+0000 7fbff8ee7700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff4153600 con 0x7fbff40b31c0 2022-06-17T12:54:03.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.724+0000 7fbff8ee7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff40b8380 con 0x7fbff40620d0 2022-06-17T12:54:03.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.724+0000 7fbff8ee7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff40b8620 con 0x7fbff40b4700 2022-06-17T12:54:03.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.724+0000 7fbff1d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbff40b31c0 0x7fbff4152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:03.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.724+0000 7fbff259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbff40620d0 0x7fbff4112be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:03.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.724+0000 7fbff2d9d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbff40b4700 0x7fbff4153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:03.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.724+0000 7fbff1d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbff40b31c0 0x7fbff4152c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36058/0 (socket says 172.21.15.36:36058) 2022-06-17T12:54:03.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.724+0000 7fbff1d9b700 1 -- 172.21.15.36:0/2712737821 learned_addr learned my addr 172.21.15.36:0/2712737821 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:03.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.724+0000 7fbff1d9b700 1 -- 172.21.15.36:0/2712737821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbff40b4700 msgr2=0x7fbff4153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:03.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.724+0000 7fbff1d9b700 1 --2- 172.21.15.36:0/2712737821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbff40b4700 0x7fbff4153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:03.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.724+0000 7fbff1d9b700 1 -- 172.21.15.36:0/2712737821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbff40620d0 msgr2=0x7fbff4112be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:03.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.724+0000 7fbff1d9b700 1 --2- 172.21.15.36:0/2712737821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbff40620d0 0x7fbff4112be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:03.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.724+0000 7fbff1d9b700 1 -- 172.21.15.36:0/2712737821 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe400b1e0 con 0x7fbff40b31c0 2022-06-17T12:54:03.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.724+0000 7fbff1d9b700 1 --2- 172.21.15.36:0/2712737821 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbff40b31c0 0x7fbff4152c20 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7fbfe4002f70 tx=0x7fbfe4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:03.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.724+0000 7fbfeb7fe700 1 -- 172.21.15.36:0/2712737821 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfe40117e0 con 0x7fbff40b31c0 2022-06-17T12:54:03.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.724+0000 7fbfeb7fe700 1 -- 172.21.15.36:0/2712737821 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe4011940 con 0x7fbff40b31c0 2022-06-17T12:54:03.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.724+0000 7fbff8ee7700 1 -- 172.21.15.36:0/2712737821 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff41130d0 con 0x7fbff40b31c0 2022-06-17T12:54:03.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.724+0000 7fbfeb7fe700 1 -- 172.21.15.36:0/2712737821 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfe4011ca0 con 0x7fbff40b31c0 2022-06-17T12:54:03.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.724+0000 7fbff8ee7700 1 -- 172.21.15.36:0/2712737821 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff40b7f50 con 0x7fbff40b31c0 2022-06-17T12:54:03.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.728+0000 7fbff8ee7700 1 -- 172.21.15.36:0/2712737821 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff40b43f0 con 0x7fbff40b31c0 2022-06-17T12:54:03.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.728+0000 7fbfeb7fe700 1 -- 172.21.15.36:0/2712737821 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fbfe4025030 con 0x7fbff40b31c0 2022-06-17T12:54:03.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.728+0000 7fbfeb7fe700 1 --2- 172.21.15.36:0/2712737821 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbfd80539b0 0x7fbfd8055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:03.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.728+0000 7fbfeb7fe700 1 -- 172.21.15.36:0/2712737821 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6111+0+0 (secure 0 0 0) 0x7fbfe406cff0 con 0x7fbff40b31c0 2022-06-17T12:54:03.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.728+0000 7fbff259c700 1 --2- 172.21.15.36:0/2712737821 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbfd80539b0 0x7fbfd8055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:03.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.728+0000 7fbff259c700 1 --2- 172.21.15.36:0/2712737821 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbfd80539b0 0x7fbfd8055e70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7fbfdc00b000 tx=0x7fbfdc006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:03.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.736+0000 7fbfeb7fe700 1 -- 172.21.15.36:0/2712737821 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbfe4043e60 con 0x7fbff40b31c0 2022-06-17T12:54:03.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:03.864+0000 7fbff8ee7700 1 -- 172.21.15.36:0/2712737821 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbff4057410 con 0x7fbff40b31c0 2022-06-17T12:54:04.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.592+0000 7fbfeb7fe700 1 -- 172.21.15.36:0/2712737821 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v78) v1 ==== 173+0+0 (secure 0 0 0) 0x7fbfe4023070 con 0x7fbff40b31c0 2022-06-17T12:54:04.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.596+0000 7fbff8ee7700 1 -- 172.21.15.36:0/2712737821 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbfd80539b0 msgr2=0x7fbfd8055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:04.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.596+0000 7fbff8ee7700 1 --2- 172.21.15.36:0/2712737821 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbfd80539b0 0x7fbfd8055e70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7fbfdc00b000 tx=0x7fbfdc006df0).stop 2022-06-17T12:54:04.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.596+0000 7fbff8ee7700 1 -- 172.21.15.36:0/2712737821 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbff40b31c0 msgr2=0x7fbff4152c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:04.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.596+0000 7fbff8ee7700 1 --2- 172.21.15.36:0/2712737821 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbff40b31c0 0x7fbff4152c20 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7fbfe4002f70 tx=0x7fbfe4015000).stop 2022-06-17T12:54:04.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.596+0000 7fbff8ee7700 1 -- 172.21.15.36:0/2712737821 shutdown_connections 2022-06-17T12:54:04.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.596+0000 7fbff8ee7700 1 --2- 172.21.15.36:0/2712737821 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbfd80539b0 0x7fbfd8055e70 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:04.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.596+0000 7fbff8ee7700 1 --2- 172.21.15.36:0/2712737821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbff40b4700 0x7fbff4153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:04.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.596+0000 7fbff8ee7700 1 --2- 172.21.15.36:0/2712737821 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbff40b31c0 0x7fbff4152c20 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:04.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.596+0000 7fbff8ee7700 1 --2- 172.21.15.36:0/2712737821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbff40620d0 0x7fbff4112be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:04.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.596+0000 7fbff8ee7700 1 -- 172.21.15.36:0/2712737821 >> 172.21.15.36:0/2712737821 conn(0x7fbff40b9850 msgr2=0x7fbff4141670 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:04.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.596+0000 7fbff8ee7700 1 -- 172.21.15.36:0/2712737821 shutdown_connections 2022-06-17T12:54:04.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.596+0000 7fbff8ee7700 1 -- 172.21.15.36:0/2712737821 wait complete. 2022-06-17T12:54:04.589 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' removed 2022-06-17T12:54:04.598 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-06-17T12:54:04.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.724+0000 7f654f59e700 1 Processor -- start 2022-06-17T12:54:04.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.724+0000 7f654f59e700 1 -- start start 2022-06-17T12:54:04.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.724+0000 7f654f59e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f65500b27b0 0x7f6550146520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:04.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.724+0000 7f654f59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6550060ca0 0x7f6550146a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:04.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.724+0000 7f654f59e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f65500aeac0 0x7f6550146f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:04.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.724+0000 7f654f59e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65501473f0 con 0x7f65500b27b0 2022-06-17T12:54:04.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.724+0000 7f654f59e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6550066b90 con 0x7f65500aeac0 2022-06-17T12:54:04.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.724+0000 7f654f59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65500b2b90 con 0x7f6550060ca0 2022-06-17T12:54:04.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.724+0000 7f654e59c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f65500b27b0 0x7f6550146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:04.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.724+0000 7f654e59c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f65500b27b0 0x7f6550146520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36064/0 (socket says 172.21.15.36:36064) 2022-06-17T12:54:04.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.724+0000 7f654e59c700 1 -- 172.21.15.36:0/851893546 learned_addr learned my addr 172.21.15.36:0/851893546 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:04.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.724+0000 7f654dd9b700 1 --2- 172.21.15.36:0/851893546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6550060ca0 0x7f6550146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:04.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.724+0000 7f654ed9d700 1 --2- 172.21.15.36:0/851893546 >> v2:172.21.15.70:3300/0 conn(0x7f65500aeac0 0x7f6550146f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:04.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.724+0000 7f654e59c700 1 -- 172.21.15.36:0/851893546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6550060ca0 msgr2=0x7f6550146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:04.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.724+0000 7f654e59c700 1 --2- 172.21.15.36:0/851893546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6550060ca0 0x7f6550146a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:04.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.724+0000 7f654e59c700 1 -- 172.21.15.36:0/851893546 >> v2:172.21.15.70:3300/0 conn(0x7f65500aeac0 msgr2=0x7f6550146f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:04.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.724+0000 7f654e59c700 1 --2- 172.21.15.36:0/851893546 >> v2:172.21.15.70:3300/0 conn(0x7f65500aeac0 0x7f6550146f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:04.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.724+0000 7f654e59c700 1 -- 172.21.15.36:0/851893546 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65500af610 con 0x7f65500b27b0 2022-06-17T12:54:04.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.724+0000 7f654e59c700 1 --2- 172.21.15.36:0/851893546 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f65500b27b0 0x7f6550146520 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7f6550066cd0 tx=0x7f6540006eb0).ready entity=mon.0 client_cookie=2415d9d19cc7d35f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:04.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.724+0000 7f654d59a700 1 -- 172.21.15.36:0/851893546 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f654000e610 con 0x7f65500b27b0 2022-06-17T12:54:04.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.724+0000 7f654d59a700 1 -- 172.21.15.36:0/851893546 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6540000e90 con 0x7f65500b27b0 2022-06-17T12:54:04.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.724+0000 7f654d59a700 1 -- 172.21.15.36:0/851893546 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f654000e9a0 con 0x7f65500b27b0 2022-06-17T12:54:04.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.728+0000 7f654f59e700 1 -- 172.21.15.36:0/851893546 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f65500b27b0 msgr2=0x7f6550146520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:04.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.728+0000 7f654f59e700 1 --2- 172.21.15.36:0/851893546 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f65500b27b0 0x7f6550146520 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7f6550066cd0 tx=0x7f6540006eb0).stop 2022-06-17T12:54:04.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.728+0000 7f654f59e700 1 -- 172.21.15.36:0/851893546 shutdown_connections 2022-06-17T12:54:04.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.728+0000 7f654f59e700 1 --2- 172.21.15.36:0/851893546 >> v2:172.21.15.70:3300/0 conn(0x7f65500aeac0 0x7f6550146f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:04.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.728+0000 7f654f59e700 1 --2- 172.21.15.36:0/851893546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6550060ca0 0x7f6550146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:04.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.728+0000 7f654f59e700 1 --2- 172.21.15.36:0/851893546 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f65500b27b0 0x7f6550146520 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:04.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.728+0000 7f654f59e700 1 -- 172.21.15.36:0/851893546 >> 172.21.15.36:0/851893546 conn(0x7f65500b5d10 msgr2=0x7f65500b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:04.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.728+0000 7f654f59e700 1 -- 172.21.15.36:0/851893546 shutdown_connections 2022-06-17T12:54:04.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.728+0000 7f654f59e700 1 -- 172.21.15.36:0/851893546 wait complete. 2022-06-17T12:54:04.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.728+0000 7f654f59e700 1 Processor -- start 2022-06-17T12:54:04.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.728+0000 7f654f59e700 1 -- start start 2022-06-17T12:54:04.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.732+0000 7f654f59e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6550060ca0 0x7f65500b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:04.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.732+0000 7f654f59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f65500b27b0 0x7f65500a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:04.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.732+0000 7f654f59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65500a0190 0x7f65500a20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:04.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.732+0000 7f654f59e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65500a1050 con 0x7f65500b27b0 2022-06-17T12:54:04.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.732+0000 7f654f59e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65500a0020 con 0x7f6550060ca0 2022-06-17T12:54:04.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.732+0000 7f654f59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65501465d0 con 0x7f65500a0190 2022-06-17T12:54:04.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.732+0000 7f654dd9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f65500b27b0 0x7f65500a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:04.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.732+0000 7f654dd9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f65500b27b0 0x7f65500a1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36072/0 (socket says 172.21.15.36:36072) 2022-06-17T12:54:04.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.732+0000 7f654dd9b700 1 -- 172.21.15.36:0/2601111844 learned_addr learned my addr 172.21.15.36:0/2601111844 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:04.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.732+0000 7f654ed9d700 1 --2- 172.21.15.36:0/2601111844 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65500a0190 0x7f65500a20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:04.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.732+0000 7f654e59c700 1 --2- 172.21.15.36:0/2601111844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6550060ca0 0x7f65500b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:04.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.732+0000 7f654dd9b700 1 -- 172.21.15.36:0/2601111844 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65500a0190 msgr2=0x7f65500a20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:04.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.732+0000 7f654dd9b700 1 --2- 172.21.15.36:0/2601111844 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65500a0190 0x7f65500a20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:04.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.732+0000 7f654dd9b700 1 -- 172.21.15.36:0/2601111844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6550060ca0 msgr2=0x7f65500b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:04.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.732+0000 7f654dd9b700 1 --2- 172.21.15.36:0/2601111844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6550060ca0 0x7f65500b4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:04.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.732+0000 7f654dd9b700 1 -- 172.21.15.36:0/2601111844 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654000b1e0 con 0x7f65500b27b0 2022-06-17T12:54:04.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.732+0000 7f654ed9d700 1 --2- 172.21.15.36:0/2601111844 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65500a0190 0x7f65500a20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:04.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.732+0000 7f654dd9b700 1 --2- 172.21.15.36:0/2601111844 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f65500b27b0 0x7f65500a1be0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f654400c000 tx=0x7f6544006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:04.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.732+0000 7f653f7fe700 1 -- 172.21.15.36:0/2601111844 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f654400c830 con 0x7f65500b27b0 2022-06-17T12:54:04.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.732+0000 7f653f7fe700 1 -- 172.21.15.36:0/2601111844 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654400c990 con 0x7f65500b27b0 2022-06-17T12:54:04.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.732+0000 7f653f7fe700 1 -- 172.21.15.36:0/2601111844 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f654400ccf0 con 0x7f65500b27b0 2022-06-17T12:54:04.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.732+0000 7f654f59e700 1 -- 172.21.15.36:0/2601111844 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6550146870 con 0x7f65500b27b0 2022-06-17T12:54:04.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.732+0000 7f654f59e700 1 -- 172.21.15.36:0/2601111844 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6550152010 con 0x7f65500b27b0 2022-06-17T12:54:04.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.736+0000 7f654f59e700 1 -- 172.21.15.36:0/2601111844 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f655009ec10 con 0x7f65500b27b0 2022-06-17T12:54:04.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.736+0000 7f653f7fe700 1 -- 172.21.15.36:0/2601111844 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f654401c460 con 0x7f65500b27b0 2022-06-17T12:54:04.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.736+0000 7f653f7fe700 1 --2- 172.21.15.36:0/2601111844 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f652c053a60 0x7f652c055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:04.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.736+0000 7f653f7fe700 1 -- 172.21.15.36:0/2601111844 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f654406c240 con 0x7f65500b27b0 2022-06-17T12:54:04.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.736+0000 7f654e59c700 1 --2- 172.21.15.36:0/2601111844 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f652c053a60 0x7f652c055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:04.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.736+0000 7f654e59c700 1 --2- 172.21.15.36:0/2601111844 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f652c053a60 0x7f652c055f20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f6540019000 tx=0x7f6540015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:04.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.744+0000 7f653f7fe700 1 -- 172.21.15.36:0/2601111844 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f655009ec10 con 0x7f65500b27b0 2022-06-17T12:54:04.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.872+0000 7f654f59e700 1 -- 172.21.15.36:0/2601111844 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f655009ec10 con 0x7f65500b27b0 2022-06-17T12:54:04.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.880+0000 7f653f7fe700 1 -- 172.21.15.36:0/2601111844 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f655009ec10 con 0x7f65500b27b0 2022-06-17T12:54:04.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.884+0000 7f654f59e700 1 -- 172.21.15.36:0/2601111844 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f652c053a60 msgr2=0x7f652c055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:04.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.884+0000 7f654f59e700 1 --2- 172.21.15.36:0/2601111844 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f652c053a60 0x7f652c055f20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f6540019000 tx=0x7f6540015000).stop 2022-06-17T12:54:04.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.884+0000 7f654f59e700 1 -- 172.21.15.36:0/2601111844 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f65500b27b0 msgr2=0x7f65500a1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:04.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.884+0000 7f654f59e700 1 --2- 172.21.15.36:0/2601111844 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f65500b27b0 0x7f65500a1be0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f654400c000 tx=0x7f6544006df0).stop 2022-06-17T12:54:04.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.884+0000 7f654f59e700 1 -- 172.21.15.36:0/2601111844 shutdown_connections 2022-06-17T12:54:04.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.884+0000 7f654f59e700 1 --2- 172.21.15.36:0/2601111844 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f652c053a60 0x7f652c055f20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:04.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.884+0000 7f654f59e700 1 --2- 172.21.15.36:0/2601111844 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65500a0190 0x7f65500a20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:04.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.884+0000 7f654f59e700 1 --2- 172.21.15.36:0/2601111844 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f65500b27b0 0x7f65500a1be0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:04.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.884+0000 7f654f59e700 1 --2- 172.21.15.36:0/2601111844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6550060ca0 0x7f65500b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:04.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.884+0000 7f654f59e700 1 -- 172.21.15.36:0/2601111844 >> 172.21.15.36:0/2601111844 conn(0x7f65500b5d10 msgr2=0x7f65500b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:04.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.884+0000 7f654f59e700 1 -- 172.21.15.36:0/2601111844 shutdown_connections 2022-06-17T12:54:04.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:04.884+0000 7f654f59e700 1 -- 172.21.15.36:0/2601111844 wait complete. 2022-06-17T12:54:04.887 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder' 2022-06-17T12:54:04.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.000+0000 7faffbcb2700 1 Processor -- start 2022-06-17T12:54:04.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.000+0000 7faffbcb2700 1 -- start start 2022-06-17T12:54:04.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.000+0000 7faffbcb2700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7faff40b1220 0x7faff40626e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:04.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.000+0000 7faffbcb2700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faff40ac870 0x7faff4062bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:04.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.000+0000 7faffbcb2700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faff40ad490 con 0x7faff40acd80 2022-06-17T12:54:04.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.000+0000 7faffbcb2700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff40b1600 con 0x7faff40b1220 2022-06-17T12:54:04.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.000+0000 7faffbcb2700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff40abb70 con 0x7faff40ac870 2022-06-17T12:54:04.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.000+0000 7faff9a4e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7faff40b1220 0x7faff40626e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:04.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.000+0000 7faffa24f700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7faff40acd80 0x7faff4142530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53612/0 (socket says 172.21.15.36:53612) 2022-06-17T12:54:04.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.004+0000 7faffa24f700 1 -- 172.21.15.36:0/1926154879 learned_addr learned my addr 172.21.15.36:0/1926154879 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:04.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.004+0000 7faff924d700 1 --2- 172.21.15.36:0/1926154879 >> v2:172.21.15.70:3300/0 conn(0x7faff40ac870 0x7faff4062bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:04.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.004+0000 7faff8a4c700 1 -- 172.21.15.36:0/1926154879 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621388134 0 0) 0x7faff40ad490 con 0x7faff40acd80 2022-06-17T12:54:04.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.004+0000 7faff8a4c700 1 -- 172.21.15.36:0/1926154879 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafdc003260 con 0x7faff40acd80 2022-06-17T12:54:04.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.004+0000 7faff9a4e700 1 -- 172.21.15.36:0/1926154879 >> v2:172.21.15.70:3300/0 conn(0x7faff40ac870 msgr2=0x7faff4062bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:04.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.004+0000 7faff9a4e700 1 --2- 172.21.15.36:0/1926154879 >> v2:172.21.15.70:3300/0 conn(0x7faff40ac870 0x7faff4062bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:04.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.004+0000 7faff9a4e700 1 -- 172.21.15.36:0/1926154879 >> v1:172.21.15.36:6789/0 conn(0x7faff40acd80 legacy=0x7faff4142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:04.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.004+0000 7faff9a4e700 1 -- 172.21.15.36:0/1926154879 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff40abe10 con 0x7faff40b1220 2022-06-17T12:54:05.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.004+0000 7faff9a4e700 1 --2- 172.21.15.36:0/1926154879 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faff40b1220 0x7faff40626e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7faff0002f70 tx=0x7faff0006df0).ready entity=mon.0 client_cookie=6ab0f375e3bd77cb server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:05.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.004+0000 7faff8a4c700 1 -- 172.21.15.36:0/1926154879 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faff000d4f0 con 0x7faff40b1220 2022-06-17T12:54:05.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.004+0000 7faff8a4c700 1 -- 172.21.15.36:0/1926154879 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff0000e90 con 0x7faff40b1220 2022-06-17T12:54:05.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.004+0000 7faff8a4c700 1 -- 172.21.15.36:0/1926154879 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faff000d840 con 0x7faff40b1220 2022-06-17T12:54:05.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.004+0000 7faffbcb2700 1 -- 172.21.15.36:0/1926154879 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faff40b1220 msgr2=0x7faff40626e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:05.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.004+0000 7faffbcb2700 1 --2- 172.21.15.36:0/1926154879 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faff40b1220 0x7faff40626e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7faff0002f70 tx=0x7faff0006df0).stop 2022-06-17T12:54:05.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.004+0000 7faffbcb2700 1 -- 172.21.15.36:0/1926154879 shutdown_connections 2022-06-17T12:54:05.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.004+0000 7faffbcb2700 1 --2- 172.21.15.36:0/1926154879 >> v2:172.21.15.70:3300/0 conn(0x7faff40ac870 0x7faff4062bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:05.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.004+0000 7faffbcb2700 1 --2- 172.21.15.36:0/1926154879 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faff40b1220 0x7faff40626e0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:05.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.004+0000 7faffbcb2700 1 -- 172.21.15.36:0/1926154879 >> 172.21.15.36:0/1926154879 conn(0x7faff40b5430 msgr2=0x7faff40b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:05.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.004+0000 7faffbcb2700 1 -- 172.21.15.36:0/1926154879 shutdown_connections 2022-06-17T12:54:05.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.004+0000 7faffbcb2700 1 -- 172.21.15.36:0/1926154879 wait complete. 2022-06-17T12:54:05.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.008+0000 7faffbcb2700 1 Processor -- start 2022-06-17T12:54:05.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.008+0000 7faffbcb2700 1 -- start start 2022-06-17T12:54:05.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.008+0000 7faffbcb2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faff40ac870 0x7faff413a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:05.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.008+0000 7faffbcb2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faff40acd80 0x7faff413d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:05.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.008+0000 7faffbcb2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faff40b1220 0x7faff413f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:05.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.008+0000 7faffbcb2700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff413bdc0 con 0x7faff40b1220 2022-06-17T12:54:05.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.008+0000 7faffbcb2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff413bff0 con 0x7faff40ac870 2022-06-17T12:54:05.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.008+0000 7faffbcb2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff413e040 con 0x7faff40acd80 2022-06-17T12:54:05.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.008+0000 7faffa24f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faff40b1220 0x7faff413f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:05.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.008+0000 7faff9a4e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faff40ac870 0x7faff413a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:05.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.008+0000 7faffa24f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faff40b1220 0x7faff413f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36088/0 (socket says 172.21.15.36:36088) 2022-06-17T12:54:05.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.008+0000 7faffa24f700 1 -- 172.21.15.36:0/3077600899 learned_addr learned my addr 172.21.15.36:0/3077600899 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:05.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.008+0000 7faff924d700 1 --2- 172.21.15.36:0/3077600899 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faff40acd80 0x7faff413d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:05.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.008+0000 7faffa24f700 1 -- 172.21.15.36:0/3077600899 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faff40acd80 msgr2=0x7faff413d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:05.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.008+0000 7faffa24f700 1 --2- 172.21.15.36:0/3077600899 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faff40acd80 0x7faff413d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:05.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.008+0000 7faffa24f700 1 -- 172.21.15.36:0/3077600899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faff40ac870 msgr2=0x7faff413a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:05.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.008+0000 7faffa24f700 1 --2- 172.21.15.36:0/3077600899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faff40ac870 0x7faff413a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:05.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.008+0000 7faffa24f700 1 -- 172.21.15.36:0/3077600899 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff000b180 con 0x7faff40b1220 2022-06-17T12:54:05.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.008+0000 7faffa24f700 1 --2- 172.21.15.36:0/3077600899 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faff40b1220 0x7faff413f7a0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7fafec001490 tx=0x7fafec007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:05.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.008+0000 7fafe2ffd700 1 -- 172.21.15.36:0/3077600899 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafec005ba0 con 0x7faff40b1220 2022-06-17T12:54:05.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.008+0000 7fafe2ffd700 1 -- 172.21.15.36:0/3077600899 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafec005d00 con 0x7faff40b1220 2022-06-17T12:54:05.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.008+0000 7faffbcb2700 1 -- 172.21.15.36:0/3077600899 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff413c5b0 con 0x7faff40b1220 2022-06-17T12:54:05.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.008+0000 7fafe2ffd700 1 -- 172.21.15.36:0/3077600899 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafec0072c0 con 0x7faff40b1220 2022-06-17T12:54:05.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.008+0000 7faffbcb2700 1 -- 172.21.15.36:0/3077600899 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff4136c60 con 0x7faff40b1220 2022-06-17T12:54:05.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.012+0000 7faffbcb2700 1 -- 172.21.15.36:0/3077600899 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff40ac0b0 con 0x7faff40b1220 2022-06-17T12:54:05.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.012+0000 7fafe2ffd700 1 -- 172.21.15.36:0/3077600899 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fafec010070 con 0x7faff40b1220 2022-06-17T12:54:05.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.012+0000 7fafe2ffd700 1 --2- 172.21.15.36:0/3077600899 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fafd40539b0 0x7fafd4055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:05.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.012+0000 7fafe2ffd700 1 -- 172.21.15.36:0/3077600899 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5729+0+0 (secure 0 0 0) 0x7fafec06b760 con 0x7faff40b1220 2022-06-17T12:54:05.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.012+0000 7faff9a4e700 1 --2- 172.21.15.36:0/3077600899 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fafd40539b0 0x7fafd4055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:05.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.012+0000 7faff9a4e700 1 --2- 172.21.15.36:0/3077600899 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fafd40539b0 0x7fafd4055e70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7faff001a000 tx=0x7faff0014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:05.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.020+0000 7fafe2ffd700 1 -- 172.21.15.36:0/3077600899 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fafec042700 con 0x7faff40b1220 2022-06-17T12:54:05.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.188+0000 7faffbcb2700 1 -- 172.21.15.36:0/3077600899 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]} v 0) v1 -- 0x7faff4057410 con 0x7faff40b1220 2022-06-17T12:54:05.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.592+0000 7fafe2ffd700 1 -- 172.21.15.36:0/3077600899 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]}]=0 v322) v1 ==== 139+0+0 (secure 0 0 0) 0x7fafec045d20 con 0x7faff40b1220 2022-06-17T12:54:05.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.596+0000 7faffbcb2700 1 -- 172.21.15.36:0/3077600899 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fafd40539b0 msgr2=0x7fafd4055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:05.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.596+0000 7faffbcb2700 1 --2- 172.21.15.36:0/3077600899 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fafd40539b0 0x7fafd4055e70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7faff001a000 tx=0x7faff0014000).stop 2022-06-17T12:54:05.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.596+0000 7faffbcb2700 1 -- 172.21.15.36:0/3077600899 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faff40b1220 msgr2=0x7faff413f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:05.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.596+0000 7faffbcb2700 1 --2- 172.21.15.36:0/3077600899 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faff40b1220 0x7faff413f7a0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7fafec001490 tx=0x7fafec007dc0).stop 2022-06-17T12:54:05.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.596+0000 7faffbcb2700 1 -- 172.21.15.36:0/3077600899 shutdown_connections 2022-06-17T12:54:05.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.596+0000 7faffbcb2700 1 --2- 172.21.15.36:0/3077600899 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fafd40539b0 0x7fafd4055e70 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:05.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.596+0000 7faffbcb2700 1 --2- 172.21.15.36:0/3077600899 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faff40b1220 0x7faff413f7a0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:05.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.596+0000 7faffbcb2700 1 --2- 172.21.15.36:0/3077600899 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faff40acd80 0x7faff413d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:05.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.596+0000 7faffbcb2700 1 --2- 172.21.15.36:0/3077600899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faff40ac870 0x7faff413a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:05.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.596+0000 7faffbcb2700 1 -- 172.21.15.36:0/3077600899 >> 172.21.15.36:0/3077600899 conn(0x7faff40b5430 msgr2=0x7faff4063140 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:05.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.596+0000 7faffbcb2700 1 -- 172.21.15.36:0/3077600899 shutdown_connections 2022-06-17T12:54:05.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.596+0000 7faffbcb2700 1 -- 172.21.15.36:0/3077600899 wait complete. 2022-06-17T12:54:05.599 INFO:tasks.cephfs_test_runner:test_cap_revoke_nonresponder (tasks.cephfs.test_misc.TestMisc) ... ok 2022-06-17T12:54:05.600 INFO:tasks.cephfs_test_runner:Starting test: test_dump_inode (tasks.cephfs.test_misc.TestMisc) 2022-06-17T12:54:05.601 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode' 2022-06-17T12:54:05.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.728+0000 7f75af50e700 1 Processor -- start 2022-06-17T12:54:05.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.728+0000 7f75af50e700 1 -- start start 2022-06-17T12:54:05.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.728+0000 7f75af50e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f75a80b27b0 0x7f75a814b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:05.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.728+0000 7f75af50e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75a8060ca0 0x7f75a80ac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:05.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.728+0000 7f75af50e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75a814b870 con 0x7f75a8147bd0 2022-06-17T12:54:05.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.728+0000 7f75af50e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75a80ac500 con 0x7f75a80b27b0 2022-06-17T12:54:05.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.728+0000 7f75af50e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a80b2b90 con 0x7f75a8060ca0 2022-06-17T12:54:05.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.728+0000 7f75aed0d700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f75a8147bd0 0x7f75a81442d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53626/0 (socket says 172.21.15.36:53626) 2022-06-17T12:54:05.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.728+0000 7f75aed0d700 1 -- 172.21.15.36:0/1152688795 learned_addr learned my addr 172.21.15.36:0/1152688795 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:05.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.728+0000 7f75add0b700 1 --2- 172.21.15.36:0/1152688795 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75a8060ca0 0x7f75a80ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:05.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.728+0000 7f75ae50c700 1 --2- 172.21.15.36:0/1152688795 >> v2:172.21.15.70:3300/0 conn(0x7f75a80b27b0 0x7f75a814b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:05.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.728+0000 7f75ad50a700 1 -- 172.21.15.36:0/1152688795 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3600503805 0 0) 0x7f75a814b870 con 0x7f75a8147bd0 2022-06-17T12:54:05.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.728+0000 7f75ad50a700 1 -- 172.21.15.36:0/1152688795 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7590003260 con 0x7f75a8147bd0 2022-06-17T12:54:05.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75add0b700 1 -- 172.21.15.36:0/1152688795 >> v2:172.21.15.70:3300/0 conn(0x7f75a80b27b0 msgr2=0x7f75a814b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:05.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75add0b700 1 --2- 172.21.15.36:0/1152688795 >> v2:172.21.15.70:3300/0 conn(0x7f75a80b27b0 0x7f75a814b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:05.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75add0b700 1 -- 172.21.15.36:0/1152688795 >> v1:172.21.15.36:6789/0 conn(0x7f75a8147bd0 legacy=0x7f75a81442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:05.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75ad50a700 1 -- 172.21.15.36:0/1152688795 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1326532454 0 0) 0x7f7590003260 con 0x7f75a8147bd0 2022-06-17T12:54:05.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75add0b700 1 -- 172.21.15.36:0/1152688795 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a8147490 con 0x7f75a8060ca0 2022-06-17T12:54:05.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75add0b700 1 --2- 172.21.15.36:0/1152688795 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75a8060ca0 0x7f75a80ac780 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f75a0001ba0 tx=0x7f7598006df0).ready entity=mon.2 client_cookie=67c8ac8bb04ef586 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:05.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75ad50a700 1 -- 172.21.15.36:0/1152688795 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75a0001a00 con 0x7f75a8060ca0 2022-06-17T12:54:05.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75ad50a700 1 -- 172.21.15.36:0/1152688795 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7598000e90 con 0x7f75a8060ca0 2022-06-17T12:54:05.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75ad50a700 1 -- 172.21.15.36:0/1152688795 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7598012460 con 0x7f75a8060ca0 2022-06-17T12:54:05.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75af50e700 1 -- 172.21.15.36:0/1152688795 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75a8060ca0 msgr2=0x7f75a80ac780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:05.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75af50e700 1 --2- 172.21.15.36:0/1152688795 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75a8060ca0 0x7f75a80ac780 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f75a0001ba0 tx=0x7f7598006df0).stop 2022-06-17T12:54:05.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75af50e700 1 -- 172.21.15.36:0/1152688795 shutdown_connections 2022-06-17T12:54:05.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75af50e700 1 --2- 172.21.15.36:0/1152688795 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75a8060ca0 0x7f75a80ac780 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:05.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75af50e700 1 --2- 172.21.15.36:0/1152688795 >> v2:172.21.15.70:3300/0 conn(0x7f75a80b27b0 0x7f75a814b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:05.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75af50e700 1 -- 172.21.15.36:0/1152688795 >> 172.21.15.36:0/1152688795 conn(0x7f75a80b5d10 msgr2=0x7f75a80b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:05.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75af50e700 1 -- 172.21.15.36:0/1152688795 shutdown_connections 2022-06-17T12:54:05.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75af50e700 1 -- 172.21.15.36:0/1152688795 wait complete. 2022-06-17T12:54:05.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75af50e700 1 Processor -- start 2022-06-17T12:54:05.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75af50e700 1 -- start start 2022-06-17T12:54:05.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75af50e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f75a8060ca0 0x7f75a814e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:05.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75af50e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75a80b27b0 0x7f75a8065d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:05.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75af50e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75a8147bd0 0x7f75a80a7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:05.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75af50e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a8066f60 con 0x7f75a8060ca0 2022-06-17T12:54:05.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75af50e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a80663a0 con 0x7f75a8147bd0 2022-06-17T12:54:05.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75af50e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a8066640 con 0x7f75a80b27b0 2022-06-17T12:54:05.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75ae50c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f75a8060ca0 0x7f75a814e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:05.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75add0b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75a80b27b0 0x7f75a8065d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:05.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75ae50c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f75a8060ca0 0x7f75a814e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36100/0 (socket says 172.21.15.36:36100) 2022-06-17T12:54:05.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75ae50c700 1 -- 172.21.15.36:0/1772466354 learned_addr learned my addr 172.21.15.36:0/1772466354 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:05.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.732+0000 7f75aed0d700 1 --2- 172.21.15.36:0/1772466354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75a8147bd0 0x7f75a80a7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:05.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.736+0000 7f75add0b700 1 -- 172.21.15.36:0/1772466354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75a8147bd0 msgr2=0x7f75a80a7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:05.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.736+0000 7f75add0b700 1 --2- 172.21.15.36:0/1772466354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75a8147bd0 0x7f75a80a7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:05.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.736+0000 7f75add0b700 1 -- 172.21.15.36:0/1772466354 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f75a8060ca0 msgr2=0x7f75a814e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:05.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.736+0000 7f75add0b700 1 --2- 172.21.15.36:0/1772466354 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f75a8060ca0 0x7f75a814e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:05.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.736+0000 7f75add0b700 1 -- 172.21.15.36:0/1772466354 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759800b180 con 0x7f75a80b27b0 2022-06-17T12:54:05.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.736+0000 7f75add0b700 1 --2- 172.21.15.36:0/1772466354 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75a80b27b0 0x7f75a8065d00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f75a0001ba0 tx=0x7f7598014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:05.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.736+0000 7f75977fe700 1 -- 172.21.15.36:0/1772466354 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7598016030 con 0x7f75a80b27b0 2022-06-17T12:54:05.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.736+0000 7f75977fe700 1 -- 172.21.15.36:0/1772466354 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759800bec0 con 0x7f75a80b27b0 2022-06-17T12:54:05.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.736+0000 7f75af50e700 1 -- 172.21.15.36:0/1772466354 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a80668e0 con 0x7f75a80b27b0 2022-06-17T12:54:05.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.736+0000 7f75977fe700 1 -- 172.21.15.36:0/1772466354 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f759800db50 con 0x7f75a80b27b0 2022-06-17T12:54:05.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.736+0000 7f75af50e700 1 -- 172.21.15.36:0/1772466354 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a8159c20 con 0x7f75a80b27b0 2022-06-17T12:54:05.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.736+0000 7f75af50e700 1 -- 172.21.15.36:0/1772466354 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a8065930 con 0x7f75a80b27b0 2022-06-17T12:54:05.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.736+0000 7f75977fe700 1 -- 172.21.15.36:0/1772466354 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f7598024020 con 0x7f75a80b27b0 2022-06-17T12:54:05.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.736+0000 7f75977fe700 1 --2- 172.21.15.36:0/1772466354 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7588053ad0 0x7f7588055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:05.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.736+0000 7f75977fe700 1 -- 172.21.15.36:0/1772466354 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f7598017070 con 0x7f75a80b27b0 2022-06-17T12:54:05.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.736+0000 7f75ae50c700 1 --2- 172.21.15.36:0/1772466354 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7588053ad0 0x7f7588055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:05.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.740+0000 7f75ae50c700 1 --2- 172.21.15.36:0/1772466354 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7588053ad0 0x7f7588055f90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f75a400b000 tx=0x7f75a4006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:05.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.744+0000 7f75977fe700 1 -- 172.21.15.36:0/1772466354 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7598044330 con 0x7f75a80b27b0 2022-06-17T12:54:05.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:05.920+0000 7f75af50e700 1 -- 172.21.15.36:0/1772466354 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]} v 0) v1 -- 0x7f75a8065930 con 0x7f75a80b27b0 2022-06-17T12:54:06.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.604+0000 7f75977fe700 1 -- 172.21.15.36:0/1772466354 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]}]=0 v323) v1 ==== 129+0+0 (secure 0 0 0) 0x7f75a8065930 con 0x7f75a80b27b0 2022-06-17T12:54:06.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.604+0000 7f75af50e700 1 -- 172.21.15.36:0/1772466354 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7588053ad0 msgr2=0x7f7588055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:06.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.604+0000 7f75af50e700 1 --2- 172.21.15.36:0/1772466354 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7588053ad0 0x7f7588055f90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f75a400b000 tx=0x7f75a4006df0).stop 2022-06-17T12:54:06.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.604+0000 7f75af50e700 1 -- 172.21.15.36:0/1772466354 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75a80b27b0 msgr2=0x7f75a8065d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:06.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.604+0000 7f75af50e700 1 --2- 172.21.15.36:0/1772466354 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75a80b27b0 0x7f75a8065d00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f75a0001ba0 tx=0x7f7598014000).stop 2022-06-17T12:54:06.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.604+0000 7f75af50e700 1 -- 172.21.15.36:0/1772466354 shutdown_connections 2022-06-17T12:54:06.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.604+0000 7f75af50e700 1 --2- 172.21.15.36:0/1772466354 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7588053ad0 0x7f7588055f90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:06.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.604+0000 7f75af50e700 1 --2- 172.21.15.36:0/1772466354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75a8147bd0 0x7f75a80a7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:06.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.604+0000 7f75af50e700 1 --2- 172.21.15.36:0/1772466354 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75a80b27b0 0x7f75a8065d00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:06.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.604+0000 7f75af50e700 1 --2- 172.21.15.36:0/1772466354 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f75a8060ca0 0x7f75a814e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:06.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.604+0000 7f75af50e700 1 -- 172.21.15.36:0/1772466354 >> 172.21.15.36:0/1772466354 conn(0x7f75a80b5d10 msgr2=0x7f75a81466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:06.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.604+0000 7f75af50e700 1 -- 172.21.15.36:0/1772466354 shutdown_connections 2022-06-17T12:54:06.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.604+0000 7f75af50e700 1 -- 172.21.15.36:0/1772466354 wait complete. 2022-06-17T12:54:06.610 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-06-17T12:54:06.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.744+0000 7f9215ccb700 1 Processor -- start 2022-06-17T12:54:06.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.744+0000 7f9215ccb700 1 -- start start 2022-06-17T12:54:06.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.744+0000 7f9215ccb700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f92100b6660 0x7f92100b2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:06.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.744+0000 7f9215ccb700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f92100af540 0x7f921013a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:06.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.744+0000 7f9215ccb700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92100b1190 0x7f921013ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:06.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.744+0000 7f9215ccb700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92100670a0 con 0x7f92100b6660 2022-06-17T12:54:06.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.744+0000 7f9215ccb700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92100b3cf0 con 0x7f92100af540 2022-06-17T12:54:06.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.744+0000 7f9215ccb700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9210144bf0 con 0x7f92100b1190 2022-06-17T12:54:06.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.744+0000 7f9214cc9700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f92100b6660 0x7f92100b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:06.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.744+0000 7f9214cc9700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f92100b6660 0x7f92100b2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36106/0 (socket says 172.21.15.36:36106) 2022-06-17T12:54:06.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.744+0000 7f9214cc9700 1 -- 172.21.15.36:0/3625688276 learned_addr learned my addr 172.21.15.36:0/3625688276 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:06.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.744+0000 7f92154ca700 1 --2- 172.21.15.36:0/3625688276 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92100b1190 0x7f921013ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:06.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.744+0000 7f920ffff700 1 --2- 172.21.15.36:0/3625688276 >> v2:172.21.15.70:3300/0 conn(0x7f92100af540 0x7f921013a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:06.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.748+0000 7f9214cc9700 1 -- 172.21.15.36:0/3625688276 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92100b1190 msgr2=0x7f921013ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:06.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.748+0000 7f9214cc9700 1 --2- 172.21.15.36:0/3625688276 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92100b1190 0x7f921013ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:06.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.748+0000 7f9214cc9700 1 -- 172.21.15.36:0/3625688276 >> v2:172.21.15.70:3300/0 conn(0x7f92100af540 msgr2=0x7f921013a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:06.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.748+0000 7f9214cc9700 1 --2- 172.21.15.36:0/3625688276 >> v2:172.21.15.70:3300/0 conn(0x7f92100af540 0x7f921013a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:06.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.748+0000 7f9214cc9700 1 -- 172.21.15.36:0/3625688276 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9210144e90 con 0x7f92100b6660 2022-06-17T12:54:06.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.748+0000 7f9214cc9700 1 --2- 172.21.15.36:0/3625688276 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f92100b6660 0x7f92100b2860 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f92000032f0 tx=0x7f9200009290).ready entity=mon.0 client_cookie=c41405d46cc884e5 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:06.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.748+0000 7f920f7fe700 1 -- 172.21.15.36:0/3625688276 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f920000b870 con 0x7f92100b6660 2022-06-17T12:54:06.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.748+0000 7f920f7fe700 1 -- 172.21.15.36:0/3625688276 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920000b9d0 con 0x7f92100b6660 2022-06-17T12:54:06.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.748+0000 7f920f7fe700 1 -- 172.21.15.36:0/3625688276 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f920000bb40 con 0x7f92100b6660 2022-06-17T12:54:06.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.748+0000 7f9215ccb700 1 -- 172.21.15.36:0/3625688276 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f92100b6660 msgr2=0x7f92100b2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:06.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.748+0000 7f9215ccb700 1 --2- 172.21.15.36:0/3625688276 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f92100b6660 0x7f92100b2860 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f92000032f0 tx=0x7f9200009290).stop 2022-06-17T12:54:06.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.748+0000 7f9215ccb700 1 -- 172.21.15.36:0/3625688276 shutdown_connections 2022-06-17T12:54:06.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.748+0000 7f9215ccb700 1 --2- 172.21.15.36:0/3625688276 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92100b1190 0x7f921013ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:06.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.748+0000 7f9215ccb700 1 --2- 172.21.15.36:0/3625688276 >> v2:172.21.15.70:3300/0 conn(0x7f92100af540 0x7f921013a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:06.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.748+0000 7f9215ccb700 1 --2- 172.21.15.36:0/3625688276 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f92100b6660 0x7f92100b2860 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:06.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.748+0000 7f9215ccb700 1 -- 172.21.15.36:0/3625688276 >> 172.21.15.36:0/3625688276 conn(0x7f92100614b0 msgr2=0x7f9210061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:06.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.748+0000 7f9215ccb700 1 -- 172.21.15.36:0/3625688276 shutdown_connections 2022-06-17T12:54:06.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.748+0000 7f9215ccb700 1 -- 172.21.15.36:0/3625688276 wait complete. 2022-06-17T12:54:06.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.748+0000 7f9215ccb700 1 Processor -- start 2022-06-17T12:54:06.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.748+0000 7f9215ccb700 1 -- start start 2022-06-17T12:54:06.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.752+0000 7f9215ccb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92100af540 0x7f9210143a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:06.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.752+0000 7f9215ccb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f92100b1190 0x7f9210135ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:06.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.752+0000 7f9215ccb700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92100b6660 0x7f9210136090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:06.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.752+0000 7f9215ccb700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921013ba30 con 0x7f92100b1190 2022-06-17T12:54:06.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.752+0000 7f9215ccb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9210066180 con 0x7f92100af540 2022-06-17T12:54:06.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.752+0000 7f9215ccb700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9210066420 con 0x7f92100b6660 2022-06-17T12:54:06.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.752+0000 7f920ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f92100b1190 0x7f9210135ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:06.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.752+0000 7f920ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f92100b1190 0x7f9210135ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36112/0 (socket says 172.21.15.36:36112) 2022-06-17T12:54:06.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.752+0000 7f920ffff700 1 -- 172.21.15.36:0/1404617555 learned_addr learned my addr 172.21.15.36:0/1404617555 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:06.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.752+0000 7f92154ca700 1 --2- 172.21.15.36:0/1404617555 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92100b6660 0x7f9210136090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:06.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.752+0000 7f9214cc9700 1 --2- 172.21.15.36:0/1404617555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92100af540 0x7f9210143a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:06.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.752+0000 7f920ffff700 1 -- 172.21.15.36:0/1404617555 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92100b6660 msgr2=0x7f9210136090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:06.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.752+0000 7f920ffff700 1 --2- 172.21.15.36:0/1404617555 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92100b6660 0x7f9210136090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:06.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.752+0000 7f920ffff700 1 -- 172.21.15.36:0/1404617555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92100af540 msgr2=0x7f9210143a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:06.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.752+0000 7f920ffff700 1 --2- 172.21.15.36:0/1404617555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92100af540 0x7f9210143a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:06.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.752+0000 7f920ffff700 1 -- 172.21.15.36:0/1404617555 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9200009530 con 0x7f92100b1190 2022-06-17T12:54:06.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.752+0000 7f920ffff700 1 --2- 172.21.15.36:0/1404617555 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f92100b1190 0x7f9210135ba0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f91f800a000 tx=0x7f91f8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:06.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.752+0000 7f920dffb700 1 -- 172.21.15.36:0/1404617555 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91f8011780 con 0x7f92100b1190 2022-06-17T12:54:06.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.752+0000 7f920dffb700 1 -- 172.21.15.36:0/1404617555 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f80118e0 con 0x7f92100b1190 2022-06-17T12:54:06.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.752+0000 7f9215ccb700 1 -- 172.21.15.36:0/1404617555 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92100666c0 con 0x7f92100b1190 2022-06-17T12:54:06.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.752+0000 7f920dffb700 1 -- 172.21.15.36:0/1404617555 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91f8011c40 con 0x7f92100b1190 2022-06-17T12:54:06.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.752+0000 7f9215ccb700 1 -- 172.21.15.36:0/1404617555 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921013b280 con 0x7f92100b1190 2022-06-17T12:54:06.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.756+0000 7f920dffb700 1 -- 172.21.15.36:0/1404617555 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f91f801c460 con 0x7f92100b1190 2022-06-17T12:54:06.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.756+0000 7f9215ccb700 1 -- 172.21.15.36:0/1404617555 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9210066850 con 0x7f92100b1190 2022-06-17T12:54:06.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.756+0000 7f920dffb700 1 --2- 172.21.15.36:0/1404617555 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f91f4053920 0x7f91f4055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:06.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.756+0000 7f920dffb700 1 -- 172.21.15.36:0/1404617555 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f91f806c330 con 0x7f92100b1190 2022-06-17T12:54:06.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.756+0000 7f9214cc9700 1 --2- 172.21.15.36:0/1404617555 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f91f4053920 0x7f91f4055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:06.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.756+0000 7f9214cc9700 1 --2- 172.21.15.36:0/1404617555 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f91f4053920 0x7f91f4055de0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f9200019000 tx=0x7f9200016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:06.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.760+0000 7f920dffb700 1 -- 172.21.15.36:0/1404617555 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f91f8043300 con 0x7f92100b1190 2022-06-17T12:54:06.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.892+0000 7f9215ccb700 1 -- 172.21.15.36:0/1404617555 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9210140b70 con 0x7f92100b1190 2022-06-17T12:54:06.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.900+0000 7f920dffb700 1 -- 172.21.15.36:0/1404617555 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f91f8046920 con 0x7f92100b1190 2022-06-17T12:54:06.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.900+0000 7f9215ccb700 1 -- 172.21.15.36:0/1404617555 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f91f4053920 msgr2=0x7f91f4055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:06.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.900+0000 7f9215ccb700 1 --2- 172.21.15.36:0/1404617555 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f91f4053920 0x7f91f4055de0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f9200019000 tx=0x7f9200016000).stop 2022-06-17T12:54:06.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.900+0000 7f9215ccb700 1 -- 172.21.15.36:0/1404617555 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f92100b1190 msgr2=0x7f9210135ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:06.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.900+0000 7f9215ccb700 1 --2- 172.21.15.36:0/1404617555 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f92100b1190 0x7f9210135ba0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f91f800a000 tx=0x7f91f8008040).stop 2022-06-17T12:54:06.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.900+0000 7f9215ccb700 1 -- 172.21.15.36:0/1404617555 shutdown_connections 2022-06-17T12:54:06.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.900+0000 7f9215ccb700 1 --2- 172.21.15.36:0/1404617555 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f91f4053920 0x7f91f4055de0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:06.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.900+0000 7f9215ccb700 1 --2- 172.21.15.36:0/1404617555 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92100b6660 0x7f9210136090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:06.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.900+0000 7f9215ccb700 1 --2- 172.21.15.36:0/1404617555 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f92100b1190 0x7f9210135ba0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:06.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.900+0000 7f9215ccb700 1 --2- 172.21.15.36:0/1404617555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92100af540 0x7f9210143a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:06.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.900+0000 7f9215ccb700 1 -- 172.21.15.36:0/1404617555 >> 172.21.15.36:0/1404617555 conn(0x7f92100614b0 msgr2=0x7f92100b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:06.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.900+0000 7f9215ccb700 1 -- 172.21.15.36:0/1404617555 shutdown_connections 2022-06-17T12:54:06.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:06.900+0000 7f9215ccb700 1 -- 172.21.15.36:0/1404617555 wait complete. 2022-06-17T12:54:06.912 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-17T12:54:06.913 DEBUG:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:06.934 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:06.956 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:06.978 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:07.000 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:07.022 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-17T12:54:07.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:07.024 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:07.024 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:54:07.032 INFO:teuthology.orchestra.run.smithi107.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-17T12:54:07.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:54:07.035 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-06-17T12:54:07.036 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:07.037 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:07.037 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:54:07.045 INFO:teuthology.orchestra.run.smithi136.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-17T12:54:07.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:54:07.047 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-17T12:54:07.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:07.048 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:07.049 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:54:07.071 INFO:teuthology.orchestra.run.smithi143.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:54:07.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:54:07.073 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-06-17T12:54:07.074 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-17T12:54:07.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.200+0000 7ff11cb03700 1 Processor -- start 2022-06-17T12:54:07.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.200+0000 7ff11cb03700 1 -- start start 2022-06-17T12:54:07.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.200+0000 7ff11cb03700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff1180b6440 0x7ff1180b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:07.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.200+0000 7ff11cb03700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff1180ad910 0x7ff118143300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:07.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.200+0000 7ff11cb03700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1181437f0 con 0x7ff11813fe40 2022-06-17T12:54:07.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.200+0000 7ff11cb03700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff11805cc60 con 0x7ff1180b6440 2022-06-17T12:54:07.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.200+0000 7ff11cb03700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff11813f700 con 0x7ff1180ad910 2022-06-17T12:54:07.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.200+0000 7ff11659c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff1180b6440 0x7ff1180b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:07.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.200+0000 7ff116d9d700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7ff11813fe40 0x7ff11805c550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53652/0 (socket says 172.21.15.36:53652) 2022-06-17T12:54:07.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.200+0000 7ff116d9d700 1 -- 172.21.15.36:0/4036280796 learned_addr learned my addr 172.21.15.36:0/4036280796 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:07.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.200+0000 7ff115d9b700 1 --2- 172.21.15.36:0/4036280796 >> v2:172.21.15.70:3300/0 conn(0x7ff1180ad910 0x7ff118143300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:07.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.200+0000 7ff11559a700 1 -- 172.21.15.36:0/4036280796 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441408358 0 0) 0x7ff1181437f0 con 0x7ff11813fe40 2022-06-17T12:54:07.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.200+0000 7ff11559a700 1 -- 172.21.15.36:0/4036280796 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0fc003260 con 0x7ff11813fe40 2022-06-17T12:54:07.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.204+0000 7ff11659c700 1 -- 172.21.15.36:0/4036280796 >> v2:172.21.15.70:3300/0 conn(0x7ff1180ad910 msgr2=0x7ff118143300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:07.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.204+0000 7ff11659c700 1 --2- 172.21.15.36:0/4036280796 >> v2:172.21.15.70:3300/0 conn(0x7ff1180ad910 0x7ff118143300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:07.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.204+0000 7ff11659c700 1 -- 172.21.15.36:0/4036280796 >> v1:172.21.15.36:6789/0 conn(0x7ff11813fe40 legacy=0x7ff11805c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:07.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.204+0000 7ff11659c700 1 -- 172.21.15.36:0/4036280796 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11813f9a0 con 0x7ff1180b6440 2022-06-17T12:54:07.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.204+0000 7ff11659c700 1 --2- 172.21.15.36:0/4036280796 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1180b6440 0x7ff1180b6820 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7ff108002f70 tx=0x7ff108006df0).ready entity=mon.0 client_cookie=ff7997311187e100 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:07.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.204+0000 7ff11559a700 1 -- 172.21.15.36:0/4036280796 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff10800d4f0 con 0x7ff1180b6440 2022-06-17T12:54:07.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.204+0000 7ff11559a700 1 -- 172.21.15.36:0/4036280796 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff108000e90 con 0x7ff1180b6440 2022-06-17T12:54:07.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.204+0000 7ff11559a700 1 -- 172.21.15.36:0/4036280796 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff10800d840 con 0x7ff1180b6440 2022-06-17T12:54:07.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.204+0000 7ff11cb03700 1 -- 172.21.15.36:0/4036280796 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1180b6440 msgr2=0x7ff1180b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:07.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.204+0000 7ff11cb03700 1 --2- 172.21.15.36:0/4036280796 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1180b6440 0x7ff1180b6820 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7ff108002f70 tx=0x7ff108006df0).stop 2022-06-17T12:54:07.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.204+0000 7ff11cb03700 1 -- 172.21.15.36:0/4036280796 shutdown_connections 2022-06-17T12:54:07.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.204+0000 7ff11cb03700 1 --2- 172.21.15.36:0/4036280796 >> v2:172.21.15.70:3300/0 conn(0x7ff1180ad910 0x7ff118143300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:07.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.204+0000 7ff11cb03700 1 --2- 172.21.15.36:0/4036280796 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1180b6440 0x7ff1180b6820 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:07.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.204+0000 7ff11cb03700 1 -- 172.21.15.36:0/4036280796 >> 172.21.15.36:0/4036280796 conn(0x7ff118063890 msgr2=0x7ff1180b5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:07.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.204+0000 7ff11cb03700 1 -- 172.21.15.36:0/4036280796 shutdown_connections 2022-06-17T12:54:07.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.204+0000 7ff11cb03700 1 -- 172.21.15.36:0/4036280796 wait complete. 2022-06-17T12:54:07.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.204+0000 7ff11cb03700 1 Processor -- start 2022-06-17T12:54:07.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.208+0000 7ff11cb03700 1 -- start start 2022-06-17T12:54:07.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.208+0000 7ff11cb03700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1180ad910 0x7ff1181386c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:07.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.208+0000 7ff11cb03700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1180b6440 0x7ff118064d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:07.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.208+0000 7ff11cb03700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff11813fe40 0x7ff118065240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:07.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.208+0000 7ff11cb03700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11813c9a0 con 0x7ff1180ad910 2022-06-17T12:54:07.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.208+0000 7ff11cb03700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11813cbd0 con 0x7ff1180b6440 2022-06-17T12:54:07.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.208+0000 7ff11cb03700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1180b5920 con 0x7ff11813fe40 2022-06-17T12:54:07.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.208+0000 7ff11659c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1180ad910 0x7ff1181386c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:07.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.208+0000 7ff11659c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1180ad910 0x7ff1181386c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36126/0 (socket says 172.21.15.36:36126) 2022-06-17T12:54:07.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.208+0000 7ff11659c700 1 -- 172.21.15.36:0/491788174 learned_addr learned my addr 172.21.15.36:0/491788174 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:07.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.208+0000 7ff116d9d700 1 --2- 172.21.15.36:0/491788174 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff11813fe40 0x7ff118065240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:07.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.208+0000 7ff115d9b700 1 --2- 172.21.15.36:0/491788174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1180b6440 0x7ff118064d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:07.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.208+0000 7ff11659c700 1 -- 172.21.15.36:0/491788174 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff11813fe40 msgr2=0x7ff118065240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:07.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.208+0000 7ff11659c700 1 --2- 172.21.15.36:0/491788174 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff11813fe40 0x7ff118065240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:07.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.208+0000 7ff11659c700 1 -- 172.21.15.36:0/491788174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1180b6440 msgr2=0x7ff118064d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:07.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.208+0000 7ff11659c700 1 --2- 172.21.15.36:0/491788174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1180b6440 0x7ff118064d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:07.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.208+0000 7ff11659c700 1 -- 172.21.15.36:0/491788174 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10800b180 con 0x7ff1180ad910 2022-06-17T12:54:07.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.208+0000 7ff115d9b700 1 --2- 172.21.15.36:0/491788174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1180b6440 0x7ff118064d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:07.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.208+0000 7ff11659c700 1 --2- 172.21.15.36:0/491788174 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1180ad910 0x7ff1181386c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7ff108002f70 tx=0x7ff108014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:07.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.208+0000 7ff1077fe700 1 -- 172.21.15.36:0/491788174 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff108016070 con 0x7ff1180ad910 2022-06-17T12:54:07.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.208+0000 7ff1077fe700 1 -- 172.21.15.36:0/491788174 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff108013780 con 0x7ff1180ad910 2022-06-17T12:54:07.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.208+0000 7ff1077fe700 1 -- 172.21.15.36:0/491788174 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff108013ae0 con 0x7ff1180ad910 2022-06-17T12:54:07.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.208+0000 7ff11cb03700 1 -- 172.21.15.36:0/491788174 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff118067bd0 con 0x7ff1180ad910 2022-06-17T12:54:07.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.208+0000 7ff11cb03700 1 -- 172.21.15.36:0/491788174 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1180658d0 con 0x7ff1180ad910 2022-06-17T12:54:07.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.212+0000 7ff11cb03700 1 -- 172.21.15.36:0/491788174 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1180b10d0 con 0x7ff1180ad910 2022-06-17T12:54:07.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.212+0000 7ff1077fe700 1 -- 172.21.15.36:0/491788174 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff10800f030 con 0x7ff1180ad910 2022-06-17T12:54:07.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.212+0000 7ff1077fe700 1 --2- 172.21.15.36:0/491788174 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff0f4053ad0 0x7ff0f4055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:07.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.212+0000 7ff1077fe700 1 -- 172.21.15.36:0/491788174 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5729+0+0 (secure 0 0 0) 0x7ff10806c780 con 0x7ff1180ad910 2022-06-17T12:54:07.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.212+0000 7ff115d9b700 1 --2- 172.21.15.36:0/491788174 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff0f4053ad0 0x7ff0f4055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:07.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.212+0000 7ff115d9b700 1 --2- 172.21.15.36:0/491788174 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff0f4053ad0 0x7ff0f4055f90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7ff10c00b000 tx=0x7ff10c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:07.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.220+0000 7ff1077fe700 1 -- 172.21.15.36:0/491788174 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff108043830 con 0x7ff1180ad910 2022-06-17T12:54:07.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.352+0000 7ff11cb03700 1 -- 172.21.15.36:0/491788174 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7ff1180658d0 con 0x7ff1180ad910 2022-06-17T12:54:07.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.352+0000 7ff1077fe700 1 -- 172.21.15.36:0/491788174 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v78) v1 ==== 81+0+29292 (secure 0 0 0) 0x7ff1180658d0 con 0x7ff1180ad910 2022-06-17T12:54:07.342 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:07.342 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:54:07.343 INFO:teuthology.orchestra.run.smithi036.stdout: "epoch": 78, 2022-06-17T12:54:07.343 INFO:teuthology.orchestra.run.smithi036.stdout: "fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", 2022-06-17T12:54:07.343 INFO:teuthology.orchestra.run.smithi036.stdout: "created": "2022-06-17T12:46:30.671408+0000", 2022-06-17T12:54:07.343 INFO:teuthology.orchestra.run.smithi036.stdout: "modified": "2022-06-17T12:54:04.579967+0000", 2022-06-17T12:54:07.344 INFO:teuthology.orchestra.run.smithi036.stdout: "last_up_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:54:07.344 INFO:teuthology.orchestra.run.smithi036.stdout: "last_in_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:54:07.344 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-17T12:54:07.345 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_num": 5799936, 2022-06-17T12:54:07.345 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_set": [ 2022-06-17T12:54:07.345 INFO:teuthology.orchestra.run.smithi036.stdout: "pglog_hardlimit", 2022-06-17T12:54:07.345 INFO:teuthology.orchestra.run.smithi036.stdout: "purged_snapdirs", 2022-06-17T12:54:07.346 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_deletes", 2022-06-17T12:54:07.346 INFO:teuthology.orchestra.run.smithi036.stdout: "sortbitwise" 2022-06-17T12:54:07.346 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:54:07.346 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_version": 6, 2022-06-17T12:54:07.346 INFO:teuthology.orchestra.run.smithi036.stdout: "full_ratio": 0.94999998807907104, 2022-06-17T12:54:07.347 INFO:teuthology.orchestra.run.smithi036.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-17T12:54:07.347 INFO:teuthology.orchestra.run.smithi036.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-17T12:54:07.347 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_snapshot": "", 2022-06-17T12:54:07.348 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_max": 14, 2022-06-17T12:54:07.348 INFO:teuthology.orchestra.run.smithi036.stdout: "max_osd": 8, 2022-06-17T12:54:07.348 INFO:teuthology.orchestra.run.smithi036.stdout: "require_min_compat_client": "jewel", 2022-06-17T12:54:07.349 INFO:teuthology.orchestra.run.smithi036.stdout: "min_compat_client": "jewel", 2022-06-17T12:54:07.349 INFO:teuthology.orchestra.run.smithi036.stdout: "require_osd_release": "octopus", 2022-06-17T12:54:07.349 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:54:07.350 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.350 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 1, 2022-06-17T12:54:07.351 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "device_health_metrics", 2022-06-17T12:54:07.351 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:33.881212+0000", 2022-06-17T12:54:07.351 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:54:07.352 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:54:07.352 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:54:07.353 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:54:07.353 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:54:07.354 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:54:07.354 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:54:07.354 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:54:07.355 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 1, 2022-06-17T12:54:07.355 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 1, 2022-06-17T12:54:07.356 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 1, 2022-06-17T12:54:07.356 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 1, 2022-06-17T12:54:07.357 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 1, 2022-06-17T12:54:07.357 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:54:07.358 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:54:07.358 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:54:07.359 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:54:07.359 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:54:07.359 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:54:07.360 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:54:07.360 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.361 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "16", 2022-06-17T12:54:07.361 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:54:07.362 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:54:07.362 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:54:07.362 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:54:07.362 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:54:07.363 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:54:07.363 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:54:07.363 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:54:07.363 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:54:07.364 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:54:07.364 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:54:07.364 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:54:07.364 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:54:07.365 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:54:07.365 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:54:07.365 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:54:07.365 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:54:07.366 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:54:07.366 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:54:07.367 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:54:07.367 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:54:07.367 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:54:07.367 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:54:07.367 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:54:07.368 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:54:07.368 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:54:07.368 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.369 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:54:07.369 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:54:07.369 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:54:07.369 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:54:07.369 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:54:07.370 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:54:07.370 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:54:07.370 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:54:07.371 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:54:07.371 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:54:07.371 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:54:07.371 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:54:07.372 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 1 2022-06-17T12:54:07.372 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.372 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:54:07.373 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr_devicehealth": {} 2022-06-17T12:54:07.373 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.373 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.374 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.374 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 2, 2022-06-17T12:54:07.374 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "rbd", 2022-06-17T12:54:07.374 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:44.485460+0000", 2022-06-17T12:54:07.375 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:54:07.375 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:54:07.375 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:54:07.376 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:54:07.376 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:54:07.376 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:54:07.377 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:54:07.377 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:54:07.378 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:54:07.378 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:54:07.379 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:54:07.379 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:54:07.379 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:54:07.380 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:54:07.380 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:54:07.381 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:54:07.381 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:54:07.382 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:54:07.383 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:54:07.383 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:54:07.383 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.384 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "19", 2022-06-17T12:54:07.384 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:54:07.385 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:54:07.385 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:54:07.386 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:54:07.386 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:54:07.386 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:54:07.387 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:54:07.387 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:54:07.387 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:54:07.388 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:54:07.388 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:54:07.389 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:54:07.389 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:54:07.390 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:54:07.390 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:54:07.391 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:54:07.391 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:54:07.391 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:54:07.392 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:54:07.392 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:54:07.392 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:54:07.393 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:54:07.393 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:54:07.393 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:54:07.393 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:54:07.394 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:54:07.394 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.394 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:54:07.395 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:54:07.395 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:54:07.395 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:54:07.395 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:54:07.396 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:54:07.396 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:54:07.396 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:54:07.397 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:54:07.397 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:54:07.397 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:54:07.397 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:54:07.398 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:54:07.398 INFO:teuthology.orchestra.run.smithi036.stdout: "rbd": {} 2022-06-17T12:54:07.398 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.398 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.398 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:54:07.399 INFO:teuthology.orchestra.run.smithi036.stdout: "osds": [ 2022-06-17T12:54:07.399 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.399 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:54:07.399 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", 2022-06-17T12:54:07.400 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:07.400 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:07.400 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:07.400 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:07.400 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:07.401 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:07.401 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:54:07.401 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 72, 2022-06-17T12:54:07.401 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:07.402 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:07.402 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:07.402 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.402 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.403 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.403 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6802", 2022-06-17T12:54:07.403 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:07.403 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.403 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.404 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.404 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6803", 2022-06-17T12:54:07.404 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:07.404 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.405 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.405 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.405 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:07.405 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.406 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.406 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.406 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6804", 2022-06-17T12:54:07.406 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:07.406 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.407 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.407 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.407 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6805", 2022-06-17T12:54:07.407 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:07.408 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.408 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.408 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.408 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:07.408 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.409 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.409 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.409 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6808", 2022-06-17T12:54:07.409 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:07.410 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.410 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.410 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.410 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6809", 2022-06-17T12:54:07.410 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:07.411 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.411 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.411 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.411 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:07.412 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.412 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.412 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.412 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6806", 2022-06-17T12:54:07.412 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:07.413 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.413 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.413 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.413 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6807", 2022-06-17T12:54:07.414 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:07.414 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.414 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.414 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.415 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6803/22619", 2022-06-17T12:54:07.415 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6805/22619", 2022-06-17T12:54:07.415 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6809/22619", 2022-06-17T12:54:07.415 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6807/22619", 2022-06-17T12:54:07.415 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:07.416 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:07.416 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:07.416 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.416 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.416 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.417 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:54:07.417 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", 2022-06-17T12:54:07.417 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:07.417 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:07.418 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:07.418 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:07.418 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:07.418 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:07.419 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:54:07.419 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 63, 2022-06-17T12:54:07.419 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:07.419 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:07.420 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:07.420 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.420 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.420 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.421 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6810", 2022-06-17T12:54:07.421 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:07.421 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.421 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.422 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.422 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6811", 2022-06-17T12:54:07.422 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:07.422 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.423 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.423 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.423 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:07.423 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.424 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.424 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.424 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6812", 2022-06-17T12:54:07.424 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:07.424 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.425 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.425 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.425 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6813", 2022-06-17T12:54:07.425 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:07.426 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.426 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.426 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.426 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:07.427 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.427 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.427 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.427 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6816", 2022-06-17T12:54:07.427 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:07.428 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.428 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.428 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.428 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6817", 2022-06-17T12:54:07.429 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:07.429 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.429 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.429 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.429 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:07.430 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.430 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.430 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.431 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6814", 2022-06-17T12:54:07.431 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:07.431 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.431 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.432 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.432 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6815", 2022-06-17T12:54:07.432 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:07.432 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.432 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.433 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.433 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6811/22620", 2022-06-17T12:54:07.433 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6813/22620", 2022-06-17T12:54:07.433 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6817/22620", 2022-06-17T12:54:07.434 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6815/22620", 2022-06-17T12:54:07.434 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:07.434 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:07.434 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:07.435 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.435 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.435 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.435 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:54:07.435 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", 2022-06-17T12:54:07.436 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:07.436 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:07.436 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:07.436 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:07.437 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:07.437 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:07.437 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:54:07.437 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 72, 2022-06-17T12:54:07.438 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:07.438 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:07.438 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:07.438 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.438 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.439 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.439 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6826", 2022-06-17T12:54:07.439 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:07.439 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.440 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.440 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.440 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6827", 2022-06-17T12:54:07.440 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:07.441 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.441 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.441 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.441 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:07.441 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.442 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.442 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.442 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6828", 2022-06-17T12:54:07.442 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:07.443 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.443 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.443 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.443 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6829", 2022-06-17T12:54:07.444 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:07.444 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.444 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.444 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.445 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:07.445 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.445 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.445 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.445 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6832", 2022-06-17T12:54:07.446 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:07.446 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.446 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.446 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.447 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6833", 2022-06-17T12:54:07.447 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:07.447 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.447 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.447 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.448 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:07.448 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.448 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.448 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.449 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6830", 2022-06-17T12:54:07.449 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:07.449 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.449 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.450 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.450 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6831", 2022-06-17T12:54:07.450 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:07.450 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.450 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.451 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.451 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6827/22643", 2022-06-17T12:54:07.451 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6829/22643", 2022-06-17T12:54:07.451 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6833/22643", 2022-06-17T12:54:07.452 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6831/22643", 2022-06-17T12:54:07.452 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:07.452 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:07.452 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:07.453 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.453 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.453 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.453 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:54:07.454 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", 2022-06-17T12:54:07.454 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:07.454 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:07.454 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:07.454 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:07.455 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:07.455 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:07.455 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:54:07.455 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 72, 2022-06-17T12:54:07.456 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:07.456 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:07.456 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:07.456 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.456 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.457 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.457 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6818", 2022-06-17T12:54:07.457 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:07.457 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.458 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.458 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.458 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6819", 2022-06-17T12:54:07.458 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:07.458 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.459 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.459 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.459 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:07.459 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.460 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.460 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.460 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6820", 2022-06-17T12:54:07.460 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:07.461 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.461 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.461 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.461 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6821", 2022-06-17T12:54:07.461 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:07.462 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.462 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.462 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.463 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:07.463 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.463 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.464 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.464 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6824", 2022-06-17T12:54:07.464 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:07.464 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.465 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.465 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.465 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6825", 2022-06-17T12:54:07.465 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:07.466 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.466 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.466 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.467 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:07.467 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.467 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.468 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.468 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6822", 2022-06-17T12:54:07.468 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:07.469 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.469 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.469 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.469 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6823", 2022-06-17T12:54:07.469 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:07.470 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.470 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.471 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.471 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6819/22646", 2022-06-17T12:54:07.471 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6821/22646", 2022-06-17T12:54:07.472 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6825/22646", 2022-06-17T12:54:07.472 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6823/22646", 2022-06-17T12:54:07.472 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:07.473 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:07.473 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:07.473 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.473 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.474 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.474 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:54:07.474 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", 2022-06-17T12:54:07.474 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:07.474 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:07.475 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:07.475 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:07.475 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:07.475 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:07.476 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:54:07.476 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 71, 2022-06-17T12:54:07.476 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:07.476 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:07.477 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:07.477 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.477 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.477 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.478 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6808", 2022-06-17T12:54:07.478 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:07.478 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.478 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.479 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.479 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6809", 2022-06-17T12:54:07.479 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:07.479 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.479 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.480 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.480 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:07.480 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.480 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.481 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.481 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6810", 2022-06-17T12:54:07.481 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:07.481 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.481 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.482 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.482 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6811", 2022-06-17T12:54:07.482 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:07.482 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.483 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.483 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.483 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:07.483 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.484 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.484 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.484 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6814", 2022-06-17T12:54:07.484 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:07.484 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.485 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.485 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.485 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6815", 2022-06-17T12:54:07.485 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:07.486 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.486 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.486 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.486 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:07.486 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.487 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.487 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.487 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6812", 2022-06-17T12:54:07.487 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:07.488 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.488 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.488 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.488 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6813", 2022-06-17T12:54:07.488 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:07.489 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.489 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.489 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.489 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6809/22812", 2022-06-17T12:54:07.490 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6811/22812", 2022-06-17T12:54:07.490 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6815/22812", 2022-06-17T12:54:07.490 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6813/22812", 2022-06-17T12:54:07.490 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:07.491 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:07.491 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:07.491 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.491 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.491 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.492 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:54:07.492 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", 2022-06-17T12:54:07.492 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:07.492 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:07.493 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:07.493 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:07.493 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:07.493 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:07.493 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:54:07.494 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 72, 2022-06-17T12:54:07.494 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:07.494 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:07.495 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:07.495 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.495 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.495 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.495 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6800", 2022-06-17T12:54:07.496 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:07.496 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.496 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.496 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.497 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6801", 2022-06-17T12:54:07.497 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:07.497 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.497 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.498 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.498 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:07.498 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.498 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.498 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.499 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6802", 2022-06-17T12:54:07.499 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:07.499 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.500 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.500 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.500 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6803", 2022-06-17T12:54:07.500 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:07.501 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.501 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.501 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.502 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:07.502 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.502 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.503 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.503 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6806", 2022-06-17T12:54:07.503 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:07.503 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.505 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.505 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.506 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6807", 2022-06-17T12:54:07.507 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:07.507 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.508 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.508 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.508 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:07.509 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.509 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.510 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.510 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6804", 2022-06-17T12:54:07.511 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:07.511 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.511 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.512 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.512 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6805", 2022-06-17T12:54:07.512 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:07.513 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.513 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.514 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.514 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6801/22813", 2022-06-17T12:54:07.514 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6803/22813", 2022-06-17T12:54:07.515 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6807/22813", 2022-06-17T12:54:07.515 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6805/22813", 2022-06-17T12:54:07.515 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:07.516 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:07.516 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:07.516 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.516 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.517 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.517 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:54:07.517 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", 2022-06-17T12:54:07.517 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:07.518 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:07.518 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:07.518 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:07.518 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:07.519 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:07.519 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:54:07.519 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 72, 2022-06-17T12:54:07.520 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:07.520 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:07.521 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:07.521 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.521 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.522 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.522 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6816", 2022-06-17T12:54:07.522 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:07.522 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.523 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.523 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.523 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6817", 2022-06-17T12:54:07.524 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:07.524 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.525 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.525 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.526 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:07.526 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.528 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.529 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.529 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6818", 2022-06-17T12:54:07.529 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:07.530 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.531 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.531 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.532 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6819", 2022-06-17T12:54:07.532 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:07.533 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.533 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.534 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.534 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:07.535 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.535 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.535 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.536 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6822", 2022-06-17T12:54:07.536 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:07.536 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.537 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.537 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.538 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6823", 2022-06-17T12:54:07.538 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:07.539 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.539 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.539 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.540 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:07.540 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.541 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.541 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.542 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6820", 2022-06-17T12:54:07.542 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:07.542 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.543 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.543 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.544 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6821", 2022-06-17T12:54:07.544 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:07.545 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.545 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.545 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.546 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6817/22822", 2022-06-17T12:54:07.546 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6819/22822", 2022-06-17T12:54:07.547 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6823/22822", 2022-06-17T12:54:07.547 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6821/22822", 2022-06-17T12:54:07.548 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:07.548 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:07.549 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:07.549 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.550 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.550 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.550 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:54:07.551 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", 2022-06-17T12:54:07.551 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:07.552 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:07.552 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:07.553 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:07.553 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:07.553 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:07.554 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:54:07.554 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 71, 2022-06-17T12:54:07.555 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:07.555 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:07.556 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:07.556 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.556 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.557 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.557 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6824", 2022-06-17T12:54:07.558 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:07.558 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.559 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.559 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.559 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6825", 2022-06-17T12:54:07.560 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:07.560 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.561 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.561 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.561 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:07.562 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.562 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.563 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.563 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6826", 2022-06-17T12:54:07.563 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:07.564 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.564 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.564 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.565 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6827", 2022-06-17T12:54:07.565 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:07.565 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.566 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.566 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.567 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:07.567 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.568 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.568 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.568 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6830", 2022-06-17T12:54:07.569 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:07.569 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.569 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.570 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.570 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6831", 2022-06-17T12:54:07.571 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:07.571 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.571 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.572 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.572 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:07.573 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:07.573 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.574 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:07.574 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6828", 2022-06-17T12:54:07.575 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:07.575 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.575 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.576 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:07.576 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6829", 2022-06-17T12:54:07.577 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:07.577 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.578 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.578 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.579 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6825/22837", 2022-06-17T12:54:07.579 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6827/22837", 2022-06-17T12:54:07.580 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6831/22837", 2022-06-17T12:54:07.580 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6829/22837", 2022-06-17T12:54:07.580 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:07.581 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:07.581 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:07.582 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:07.582 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.583 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:54:07.583 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_xinfo": [ 2022-06-17T12:54:07.583 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.584 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:54:07.584 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:07.585 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:07.585 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:07.585 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:07.586 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:07.586 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.291461+0000", 2022-06-17T12:54:07.587 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:07.587 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.588 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.588 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:54:07.588 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:07.589 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:07.589 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:07.589 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:07.589 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:07.590 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.537912+0000", 2022-06-17T12:54:07.590 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:07.590 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.591 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.591 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:54:07.591 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:07.591 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:07.592 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:07.592 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:07.592 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:07.592 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.606947+0000", 2022-06-17T12:54:07.593 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:07.593 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.593 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.593 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:54:07.594 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:07.594 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:07.595 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:07.595 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:07.595 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:07.596 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.514412+0000", 2022-06-17T12:54:07.596 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:07.596 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.596 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.597 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:54:07.597 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:07.597 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:07.598 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:07.598 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:07.598 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:07.598 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.394645+0000", 2022-06-17T12:54:07.599 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:07.599 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.599 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.599 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:54:07.599 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:07.600 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:07.600 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:07.600 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:07.600 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:07.601 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.343806+0000", 2022-06-17T12:54:07.601 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:07.601 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.601 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.601 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:54:07.602 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:07.602 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:07.602 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:07.602 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:07.603 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:07.603 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.591864+0000", 2022-06-17T12:54:07.603 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:07.603 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.604 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:07.604 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:54:07.604 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:07.604 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:07.604 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:07.605 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:07.605 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:07.605 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.607045+0000", 2022-06-17T12:54:07.605 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:07.606 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.606 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:54:07.606 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap": [], 2022-06-17T12:54:07.606 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap_items": [], 2022-06-17T12:54:07.607 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_temp": [], 2022-06-17T12:54:07.607 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_temp": [], 2022-06-17T12:54:07.607 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklist": { 2022-06-17T12:54:07.607 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.36:6835/2514005995": "2022-06-18T12:54:01.404304+0000", 2022-06-17T12:54:07.607 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.36:6834/2514005995": "2022-06-18T12:54:01.404304+0000", 2022-06-17T12:54:07.608 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.107:0/973797138": "2022-06-17T13:53:50.522507+0000" 2022-06-17T12:54:07.608 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.608 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profiles": { 2022-06-17T12:54:07.608 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs_data": { 2022-06-17T12:54:07.609 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-device-class": "", 2022-06-17T12:54:07.609 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:54:07.609 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-root": "default", 2022-06-17T12:54:07.609 INFO:teuthology.orchestra.run.smithi036.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-17T12:54:07.609 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:54:07.610 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "2", 2022-06-17T12:54:07.610 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:54:07.610 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:54:07.610 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van", 2022-06-17T12:54:07.611 INFO:teuthology.orchestra.run.smithi036.stdout: "w": "8" 2022-06-17T12:54:07.611 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.612 INFO:teuthology.orchestra.run.smithi036.stdout: "default": { 2022-06-17T12:54:07.612 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:54:07.612 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:54:07.612 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "1", 2022-06-17T12:54:07.613 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:54:07.613 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:54:07.613 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van" 2022-06-17T12:54:07.613 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:07.614 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:07.614 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps_queue": [], 2022-06-17T12:54:07.615 INFO:teuthology.orchestra.run.smithi036.stdout: "new_removed_snaps": [], 2022-06-17T12:54:07.615 INFO:teuthology.orchestra.run.smithi036.stdout: "new_purged_snaps": [], 2022-06-17T12:54:07.615 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_node_flags": {}, 2022-06-17T12:54:07.616 INFO:teuthology.orchestra.run.smithi036.stdout: "device_class_flags": {} 2022-06-17T12:54:07.616 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:54:07.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.356+0000 7ff11cb03700 1 -- 172.21.15.36:0/491788174 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff0f4053ad0 msgr2=0x7ff0f4055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:07.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.356+0000 7ff11cb03700 1 --2- 172.21.15.36:0/491788174 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff0f4053ad0 0x7ff0f4055f90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7ff10c00b000 tx=0x7ff10c008040).stop 2022-06-17T12:54:07.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.356+0000 7ff11cb03700 1 -- 172.21.15.36:0/491788174 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1180ad910 msgr2=0x7ff1181386c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:07.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.356+0000 7ff11cb03700 1 --2- 172.21.15.36:0/491788174 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1180ad910 0x7ff1181386c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7ff108002f70 tx=0x7ff108014000).stop 2022-06-17T12:54:07.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.356+0000 7ff11cb03700 1 -- 172.21.15.36:0/491788174 shutdown_connections 2022-06-17T12:54:07.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.356+0000 7ff11cb03700 1 --2- 172.21.15.36:0/491788174 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff0f4053ad0 0x7ff0f4055f90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:07.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.356+0000 7ff11cb03700 1 --2- 172.21.15.36:0/491788174 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff11813fe40 0x7ff118065240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:07.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.356+0000 7ff11cb03700 1 --2- 172.21.15.36:0/491788174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1180b6440 0x7ff118064d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:07.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.356+0000 7ff11cb03700 1 --2- 172.21.15.36:0/491788174 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1180ad910 0x7ff1181386c0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:07.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.356+0000 7ff11cb03700 1 -- 172.21.15.36:0/491788174 >> 172.21.15.36:0/491788174 conn(0x7ff118063890 msgr2=0x7ff1180b4640 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:07.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.356+0000 7ff11cb03700 1 -- 172.21.15.36:0/491788174 shutdown_connections 2022-06-17T12:54:07.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.356+0000 7ff11cb03700 1 -- 172.21.15.36:0/491788174 wait complete. 2022-06-17T12:54:07.626 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:54:07.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa733418700 1 Processor -- start 2022-06-17T12:54:07.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa733418700 1 -- start start 2022-06-17T12:54:07.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa733418700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa72c066bc0 0x7fa72c143880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:07.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa733418700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72c0b1570 con 0x7fa72c0afe30 2022-06-17T12:54:07.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa733418700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72c066fa0 con 0x7fa72c067510 2022-06-17T12:54:07.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa733418700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c0acb10 con 0x7fa72c066bc0 2022-06-17T12:54:07.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa731c15700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fa72c0afe30 0x7fa72c143d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53664/0 (socket says 172.21.15.36:53664) 2022-06-17T12:54:07.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa731c15700 1 -- 172.21.15.36:0/3446845682 learned_addr learned my addr 172.21.15.36:0/3446845682 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:07.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa732416700 1 --2- 172.21.15.36:0/3446845682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa72c066bc0 0x7fa72c143880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:07.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa731414700 1 -- 172.21.15.36:0/3446845682 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3653560070 0 0) 0x7fa72c0b1570 con 0x7fa72c0afe30 2022-06-17T12:54:07.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa731414700 1 -- 172.21.15.36:0/3446845682 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa714003260 con 0x7fa72c0afe30 2022-06-17T12:54:07.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa731414700 1 -- 172.21.15.36:0/3446845682 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 182316434 0 0) 0x7fa72c066fa0 con 0x7fa72c067510 2022-06-17T12:54:07.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa731414700 1 -- 172.21.15.36:0/3446845682 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa72c0b1570 con 0x7fa72c067510 2022-06-17T12:54:07.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa731414700 1 -- 172.21.15.36:0/3446845682 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4026831869 0 0) 0x7fa714003260 con 0x7fa72c0afe30 2022-06-17T12:54:07.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa731414700 1 -- 172.21.15.36:0/3446845682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa72c066bc0 msgr2=0x7fa72c143880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:07.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa731414700 1 --2- 172.21.15.36:0/3446845682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa72c066bc0 0x7fa72c143880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:07.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa731414700 1 -- 172.21.15.36:0/3446845682 >> v1:172.21.15.70:6789/0 conn(0x7fa72c067510 legacy=0x7fa72c0addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:07.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa731414700 1 -- 172.21.15.36:0/3446845682 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c0b17a0 con 0x7fa72c0afe30 2022-06-17T12:54:07.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa731414700 1 -- 172.21.15.36:0/3446845682 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7fa728001660 con 0x7fa72c0afe30 2022-06-17T12:54:07.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa731414700 1 -- 172.21.15.36:0/3446845682 >> v1:172.21.15.36:6789/0 conn(0x7fa72c0afe30 legacy=0x7fa72c143d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:07.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa731414700 1 --2- 172.21.15.36:0/3446845682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa714003910 0x7fa714003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:07.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa731414700 1 --2- 172.21.15.36:0/3446845682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa714004500 0x7fa714004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:07.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa731414700 1 --2- 172.21.15.36:0/3446845682 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa714004e70 0x7fa71400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:07.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa731414700 1 -- 172.21.15.36:0/3446845682 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71400bc40 con 0x7fa714004e70 2022-06-17T12:54:07.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa731414700 1 -- 172.21.15.36:0/3446845682 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71400bd80 con 0x7fa714003910 2022-06-17T12:54:07.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa731414700 1 -- 172.21.15.36:0/3446845682 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa71400bf40 con 0x7fa714004500 2022-06-17T12:54:07.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa732c17700 1 --2- 172.21.15.36:0/3446845682 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa714004e70 0x7fa71400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:07.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa732416700 1 --2- 172.21.15.36:0/3446845682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa714003910 0x7fa714003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:07.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.752+0000 7fa731c15700 1 --2- 172.21.15.36:0/3446845682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa714004500 0x7fa714004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:07.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.756+0000 7fa732c17700 1 -- 172.21.15.36:0/3446845682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa714004500 msgr2=0x7fa714004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:07.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.756+0000 7fa732c17700 1 --2- 172.21.15.36:0/3446845682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa714004500 0x7fa714004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:07.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.756+0000 7fa732c17700 1 -- 172.21.15.36:0/3446845682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa714003910 msgr2=0x7fa714003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:07.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.756+0000 7fa732c17700 1 --2- 172.21.15.36:0/3446845682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa714003910 0x7fa714003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:07.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.756+0000 7fa732c17700 1 -- 172.21.15.36:0/3446845682 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa714003260 con 0x7fa714004e70 2022-06-17T12:54:07.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.756+0000 7fa731c15700 1 --2- 172.21.15.36:0/3446845682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa714004500 0x7fa714004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:07.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.756+0000 7fa732c17700 1 --2- 172.21.15.36:0/3446845682 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa714004e70 0x7fa71400b750 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7fa724001490 tx=0x7fa724007d80).ready entity=mon.0 client_cookie=5739f030dca9e13c server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:07.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.756+0000 7fa731414700 1 -- 172.21.15.36:0/3446845682 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa724001780 con 0x7fa714004e70 2022-06-17T12:54:07.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.756+0000 7fa731414700 1 -- 172.21.15.36:0/3446845682 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa724006890 con 0x7fa714004e70 2022-06-17T12:54:07.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.756+0000 7fa731414700 1 -- 172.21.15.36:0/3446845682 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa724006bf0 con 0x7fa714004e70 2022-06-17T12:54:07.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.756+0000 7fa733418700 1 -- 172.21.15.36:0/3446845682 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa714004e70 msgr2=0x7fa71400b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:07.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.756+0000 7fa733418700 1 --2- 172.21.15.36:0/3446845682 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa714004e70 0x7fa71400b750 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7fa724001490 tx=0x7fa724007d80).stop 2022-06-17T12:54:07.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.756+0000 7fa732c17700 1 -- 172.21.15.36:0/3446845682 reap_dead start 2022-06-17T12:54:07.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.756+0000 7fa733418700 1 -- 172.21.15.36:0/3446845682 shutdown_connections 2022-06-17T12:54:07.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.756+0000 7fa733418700 1 -- 172.21.15.36:0/3446845682 >> 172.21.15.36:0/3446845682 conn(0x7fa72c05f770 msgr2=0x7fa72c05fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:07.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.756+0000 7fa733418700 1 -- 172.21.15.36:0/3446845682 shutdown_connections 2022-06-17T12:54:07.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.756+0000 7fa733418700 1 -- 172.21.15.36:0/3446845682 wait complete. 2022-06-17T12:54:07.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.760+0000 7fa733418700 1 Processor -- start 2022-06-17T12:54:07.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.760+0000 7fa733418700 1 -- start start 2022-06-17T12:54:07.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.760+0000 7fa733418700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa714004e70 0x7fa72c0a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:07.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.760+0000 7fa733418700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa72c09f1f0 0x7fa72c0ab5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:07.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.760+0000 7fa733418700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa72c0abac0 0x7fa72c152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:07.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.760+0000 7fa733418700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c09f650 con 0x7fa72c0abac0 2022-06-17T12:54:07.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.760+0000 7fa733418700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c0a28c0 con 0x7fa714004e70 2022-06-17T12:54:07.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.760+0000 7fa733418700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c0a2b60 con 0x7fa72c09f1f0 2022-06-17T12:54:07.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.760+0000 7fa732c17700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa72c0abac0 0x7fa72c152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:07.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.760+0000 7fa732c17700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa72c0abac0 0x7fa72c152490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36150/0 (socket says 172.21.15.36:36150) 2022-06-17T12:54:07.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.760+0000 7fa732c17700 1 -- 172.21.15.36:0/2548853970 learned_addr learned my addr 172.21.15.36:0/2548853970 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:07.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.760+0000 7fa732416700 1 --2- 172.21.15.36:0/2548853970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa714004e70 0x7fa72c0a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:07.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.760+0000 7fa732c17700 1 -- 172.21.15.36:0/2548853970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa72c09f1f0 msgr2=0x7fa72c0ab5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:07.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.760+0000 7fa732c17700 1 --2- 172.21.15.36:0/2548853970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa72c09f1f0 0x7fa72c0ab5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-06-17T12:54:07.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.760+0000 7fa732c17700 1 -- 172.21.15.36:0/2548853970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa714004e70 msgr2=0x7fa72c0a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:07.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.760+0000 7fa732c17700 1 --2- 172.21.15.36:0/2548853970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa714004e70 0x7fa72c0a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:07.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.760+0000 7fa732c17700 1 -- 172.21.15.36:0/2548853970 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c0a2e00 con 0x7fa72c0abac0 2022-06-17T12:54:07.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.760+0000 7fa732c17700 1 --2- 172.21.15.36:0/2548853970 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa72c0abac0 0x7fa72c152490 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7fa72c0b6450 tx=0x7fa724016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:07.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.764+0000 7fa71b7fe700 1 -- 172.21.15.36:0/2548853970 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa724001780 con 0x7fa72c0abac0 2022-06-17T12:54:07.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.764+0000 7fa71b7fe700 1 -- 172.21.15.36:0/2548853970 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72400c9e0 con 0x7fa72c0abac0 2022-06-17T12:54:07.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.764+0000 7fa733418700 1 -- 172.21.15.36:0/2548853970 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72c0a32a0 con 0x7fa72c0abac0 2022-06-17T12:54:07.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.764+0000 7fa71b7fe700 1 -- 172.21.15.36:0/2548853970 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa72400cd40 con 0x7fa72c0abac0 2022-06-17T12:54:07.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.764+0000 7fa733418700 1 -- 172.21.15.36:0/2548853970 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72c0a0e90 con 0x7fa72c0abac0 2022-06-17T12:54:07.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.764+0000 7fa733418700 1 -- 172.21.15.36:0/2548853970 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa72c0a26e0 con 0x7fa72c0abac0 2022-06-17T12:54:07.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.764+0000 7fa71b7fe700 1 -- 172.21.15.36:0/2548853970 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fa724025030 con 0x7fa72c0abac0 2022-06-17T12:54:07.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.764+0000 7fa71b7fe700 1 --2- 172.21.15.36:0/2548853970 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa70c053a60 0x7fa70c055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:07.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.764+0000 7fa71b7fe700 1 -- 172.21.15.36:0/2548853970 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5729+0+0 (secure 0 0 0) 0x7fa72406cdf0 con 0x7fa72c0abac0 2022-06-17T12:54:07.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.764+0000 7fa732416700 1 --2- 172.21.15.36:0/2548853970 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa70c053a60 0x7fa70c055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:07.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.764+0000 7fa732416700 1 --2- 172.21.15.36:0/2548853970 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa70c053a60 0x7fa70c055f20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7fa71c00b000 tx=0x7fa71c006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:07.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.768+0000 7fa71b7fe700 1 -- 172.21.15.36:0/2548853970 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa724043d90 con 0x7fa72c0abac0 2022-06-17T12:54:07.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.932+0000 7fa733418700 1 -- 172.21.15.36:0/2548853970 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa72c0a2f90 con 0x7fa72c0abac0 2022-06-17T12:54:07.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.932+0000 7fa71b7fe700 1 -- 172.21.15.36:0/2548853970 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+1209 (secure 0 0 0) 0x7fa724023070 con 0x7fa72c0abac0 2022-06-17T12:54:07.921 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:07.921 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":38,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/2541998896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2541998896},{"type":"v1","addr":"172.21.15.70:6833","nonce":2541998896}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33},{"gid":6487,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/1964171274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":1964171274},{"type":"v1","addr":"172.21.15.36:6835","nonce":1964171274}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38}],"filesystems":[]} 2022-06-17T12:54:07.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.936+0000 7fa733418700 1 -- 172.21.15.36:0/2548853970 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa70c053a60 msgr2=0x7fa70c055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:07.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.936+0000 7fa733418700 1 --2- 172.21.15.36:0/2548853970 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa70c053a60 0x7fa70c055f20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7fa71c00b000 tx=0x7fa71c006df0).stop 2022-06-17T12:54:07.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.936+0000 7fa733418700 1 -- 172.21.15.36:0/2548853970 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa72c0abac0 msgr2=0x7fa72c152490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:07.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.936+0000 7fa733418700 1 --2- 172.21.15.36:0/2548853970 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa72c0abac0 0x7fa72c152490 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7fa72c0b6450 tx=0x7fa724016000).stop 2022-06-17T12:54:07.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.936+0000 7fa733418700 1 -- 172.21.15.36:0/2548853970 shutdown_connections 2022-06-17T12:54:07.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.936+0000 7fa733418700 1 --2- 172.21.15.36:0/2548853970 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa70c053a60 0x7fa70c055f20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:07.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.936+0000 7fa733418700 1 --2- 172.21.15.36:0/2548853970 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa72c0abac0 0x7fa72c152490 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:07.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.936+0000 7fa733418700 1 --2- 172.21.15.36:0/2548853970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa72c09f1f0 0x7fa72c0ab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-06-17T12:54:07.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.936+0000 7fa733418700 1 --2- 172.21.15.36:0/2548853970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa714004e70 0x7fa72c0a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:07.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.936+0000 7fa733418700 1 -- 172.21.15.36:0/2548853970 >> 172.21.15.36:0/2548853970 conn(0x7fa72c05f770 msgr2=0x7fa72c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:07.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.936+0000 7fa733418700 1 -- 172.21.15.36:0/2548853970 shutdown_connections 2022-06-17T12:54:07.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:07.936+0000 7fa733418700 1 -- 172.21.15.36:0/2548853970 wait complete. 2022-06-17T12:54:07.929 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 38 2022-06-17T12:54:07.935 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:54:08.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.052+0000 7f2ec083d700 1 Processor -- start 2022-06-17T12:54:08.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.052+0000 7f2ec083d700 1 -- start start 2022-06-17T12:54:08.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2ec083d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2ebc0b1280 0x7f2ebc13fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:08.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2ec083d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ebc0abe30 0x7f2ebc062ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:08.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2ec083d700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ebc0b1660 con 0x7f2ebc140410 2022-06-17T12:54:08.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2ec083d700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ebc0aef10 con 0x7f2ebc0b1280 2022-06-17T12:54:08.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2ec083d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc140080 con 0x7f2ebc0abe30 2022-06-17T12:54:08.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2ebad9d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2ebc0b1280 0x7f2ebc13fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:08.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2ebb59e700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f2ebc140410 0x7f2ebc1436d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53688/0 (socket says 172.21.15.36:53688) 2022-06-17T12:54:08.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2ebb59e700 1 -- 172.21.15.36:0/379454472 learned_addr learned my addr 172.21.15.36:0/379454472 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:08.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2ebad9d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2ebc0b1280 0x7f2ebc13fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36156/0 (socket says 172.21.15.36:36156) 2022-06-17T12:54:08.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2eb3fff700 1 --2- 172.21.15.36:0/379454472 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ebc0abe30 0x7f2ebc062ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:08.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2eba59c700 1 -- 172.21.15.36:0/379454472 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874238766 0 0) 0x7f2ebc0b1660 con 0x7f2ebc140410 2022-06-17T12:54:08.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2eba59c700 1 -- 172.21.15.36:0/379454472 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e9c003260 con 0x7f2ebc140410 2022-06-17T12:54:08.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2ebad9d700 1 -- 172.21.15.36:0/379454472 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ebc0abe30 msgr2=0x7f2ebc062ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:08.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2ebad9d700 1 --2- 172.21.15.36:0/379454472 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ebc0abe30 0x7f2ebc062ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:08.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2ebad9d700 1 -- 172.21.15.36:0/379454472 >> v1:172.21.15.36:6789/0 conn(0x7f2ebc140410 legacy=0x7f2ebc1436d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:08.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2ebad9d700 1 -- 172.21.15.36:0/379454472 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ebc0ac3b0 con 0x7f2ebc0b1280 2022-06-17T12:54:08.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2eb3fff700 1 --2- 172.21.15.36:0/379454472 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ebc0abe30 0x7f2ebc062ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:08.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2ebad9d700 1 --2- 172.21.15.36:0/379454472 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2ebc0b1280 0x7f2ebc13fb90 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7f2ebc0b17f0 tx=0x7f2eac006df0).ready entity=mon.0 client_cookie=41995893eea85f05 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:08.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2eba59c700 1 -- 172.21.15.36:0/379454472 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2eac00d570 con 0x7f2ebc0b1280 2022-06-17T12:54:08.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2eba59c700 1 -- 172.21.15.36:0/379454472 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eac000e90 con 0x7f2ebc0b1280 2022-06-17T12:54:08.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2eba59c700 1 -- 172.21.15.36:0/379454472 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2eac00d8c0 con 0x7f2ebc0b1280 2022-06-17T12:54:08.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2ec083d700 1 -- 172.21.15.36:0/379454472 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2ebc0b1280 msgr2=0x7f2ebc13fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:08.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2ec083d700 1 --2- 172.21.15.36:0/379454472 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2ebc0b1280 0x7f2ebc13fb90 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7f2ebc0b17f0 tx=0x7f2eac006df0).stop 2022-06-17T12:54:08.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2ec083d700 1 -- 172.21.15.36:0/379454472 shutdown_connections 2022-06-17T12:54:08.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2ec083d700 1 --2- 172.21.15.36:0/379454472 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ebc0abe30 0x7f2ebc062ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:08.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2ec083d700 1 --2- 172.21.15.36:0/379454472 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2ebc0b1280 0x7f2ebc13fb90 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:08.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2ec083d700 1 -- 172.21.15.36:0/379454472 >> 172.21.15.36:0/379454472 conn(0x7f2ebc05f770 msgr2=0x7f2ebc05fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:08.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2ec083d700 1 -- 172.21.15.36:0/379454472 shutdown_connections 2022-06-17T12:54:08.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.056+0000 7f2ec083d700 1 -- 172.21.15.36:0/379454472 wait complete. 2022-06-17T12:54:08.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2ec083d700 1 Processor -- start 2022-06-17T12:54:08.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2ec083d700 1 -- start start 2022-06-17T12:54:08.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2ec083d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2ebc0abe30 0x7f2ebc142550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:08.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2ec083d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ebc0b1280 0x7f2ebc154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:08.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2ec083d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ebc140410 0x7f2ebc155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:08.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2ec083d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc13cc30 con 0x7f2ebc0abe30 2022-06-17T12:54:08.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2ec083d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc13cea0 con 0x7f2ebc140410 2022-06-17T12:54:08.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2ec083d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc13c990 con 0x7f2ebc0b1280 2022-06-17T12:54:08.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2ebad9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2ebc0abe30 0x7f2ebc142550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:08.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2ebad9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2ebc0abe30 0x7f2ebc142550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36160/0 (socket says 172.21.15.36:36160) 2022-06-17T12:54:08.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2ebad9d700 1 -- 172.21.15.36:0/2233972017 learned_addr learned my addr 172.21.15.36:0/2233972017 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:08.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2ebb59e700 1 --2- 172.21.15.36:0/2233972017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ebc140410 0x7f2ebc155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:08.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2eb3fff700 1 --2- 172.21.15.36:0/2233972017 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ebc0b1280 0x7f2ebc154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:08.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2ebad9d700 1 -- 172.21.15.36:0/2233972017 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ebc0b1280 msgr2=0x7f2ebc154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:08.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2ebad9d700 1 --2- 172.21.15.36:0/2233972017 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ebc0b1280 0x7f2ebc154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:08.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2ebad9d700 1 -- 172.21.15.36:0/2233972017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ebc140410 msgr2=0x7f2ebc155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:08.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2ebad9d700 1 --2- 172.21.15.36:0/2233972017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ebc140410 0x7f2ebc155250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:08.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2ebad9d700 1 -- 172.21.15.36:0/2233972017 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eac00b180 con 0x7f2ebc0abe30 2022-06-17T12:54:08.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2eb3fff700 1 --2- 172.21.15.36:0/2233972017 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ebc0b1280 0x7f2ebc154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:08.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2ebb59e700 1 --2- 172.21.15.36:0/2233972017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ebc140410 0x7f2ebc155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:08.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2ebad9d700 1 --2- 172.21.15.36:0/2233972017 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2ebc0abe30 0x7f2ebc142550 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7f2ebc0b17f0 tx=0x7f2eac014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:08.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2eb8d99700 1 -- 172.21.15.36:0/2233972017 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2eac00dea0 con 0x7f2ebc0abe30 2022-06-17T12:54:08.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2eb8d99700 1 -- 172.21.15.36:0/2233972017 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eac013de0 con 0x7f2ebc0abe30 2022-06-17T12:54:08.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2ec083d700 1 -- 172.21.15.36:0/2233972017 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ebc063130 con 0x7f2ebc0abe30 2022-06-17T12:54:08.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2eb8d99700 1 -- 172.21.15.36:0/2233972017 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2eac01e5f0 con 0x7f2ebc0abe30 2022-06-17T12:54:08.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.060+0000 7f2ec083d700 1 -- 172.21.15.36:0/2233972017 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ebc13c590 con 0x7f2ebc0abe30 2022-06-17T12:54:08.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.064+0000 7f2ec083d700 1 -- 172.21.15.36:0/2233972017 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ebc0ac650 con 0x7f2ebc0abe30 2022-06-17T12:54:08.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.064+0000 7f2eb8d99700 1 -- 172.21.15.36:0/2233972017 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2eac01c070 con 0x7f2ebc0abe30 2022-06-17T12:54:08.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.064+0000 7f2eb8d99700 1 --2- 172.21.15.36:0/2233972017 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2ea00539b0 0x7f2ea0055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:08.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.064+0000 7f2eb8d99700 1 -- 172.21.15.36:0/2233972017 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f2eac06c6d0 con 0x7f2ebc0abe30 2022-06-17T12:54:08.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.064+0000 7f2eb3fff700 1 --2- 172.21.15.36:0/2233972017 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2ea00539b0 0x7f2ea0055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:08.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.064+0000 7f2eb3fff700 1 --2- 172.21.15.36:0/2233972017 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2ea00539b0 0x7f2ea0055e70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f2ea400c000 tx=0x7f2ea4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:08.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.072+0000 7f2eb8d99700 1 -- 172.21.15.36:0/2233972017 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2eac043720 con 0x7f2ebc0abe30 2022-06-17T12:54:08.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.236+0000 7f2ec083d700 1 -- 172.21.15.36:0/2233972017 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ebc057410 con 0x7f2ebc0abe30 2022-06-17T12:54:08.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.236+0000 7f2eb8d99700 1 -- 172.21.15.36:0/2233972017 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+1209 (secure 0 0 0) 0x7f2eac023070 con 0x7f2ebc0abe30 2022-06-17T12:54:08.225 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:08.226 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":38,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/2541998896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2541998896},{"type":"v1","addr":"172.21.15.70:6833","nonce":2541998896}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33},{"gid":6487,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/1964171274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":1964171274},{"type":"v1","addr":"172.21.15.36:6835","nonce":1964171274}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38}],"filesystems":[]} 2022-06-17T12:54:08.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.236+0000 7f2ec083d700 1 -- 172.21.15.36:0/2233972017 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2ea00539b0 msgr2=0x7f2ea0055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:08.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.236+0000 7f2ec083d700 1 --2- 172.21.15.36:0/2233972017 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2ea00539b0 0x7f2ea0055e70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f2ea400c000 tx=0x7f2ea4008040).stop 2022-06-17T12:54:08.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.236+0000 7f2ec083d700 1 -- 172.21.15.36:0/2233972017 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2ebc0abe30 msgr2=0x7f2ebc142550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:08.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.236+0000 7f2ec083d700 1 --2- 172.21.15.36:0/2233972017 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2ebc0abe30 0x7f2ebc142550 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7f2ebc0b17f0 tx=0x7f2eac014000).stop 2022-06-17T12:54:08.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.240+0000 7f2ec083d700 1 -- 172.21.15.36:0/2233972017 shutdown_connections 2022-06-17T12:54:08.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.240+0000 7f2ec083d700 1 --2- 172.21.15.36:0/2233972017 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2ea00539b0 0x7f2ea0055e70 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:08.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.240+0000 7f2ec083d700 1 --2- 172.21.15.36:0/2233972017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ebc140410 0x7f2ebc155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:08.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.240+0000 7f2ec083d700 1 --2- 172.21.15.36:0/2233972017 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ebc0b1280 0x7f2ebc154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:08.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.240+0000 7f2ec083d700 1 --2- 172.21.15.36:0/2233972017 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2ebc0abe30 0x7f2ebc142550 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:08.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.240+0000 7f2ec083d700 1 -- 172.21.15.36:0/2233972017 >> 172.21.15.36:0/2233972017 conn(0x7f2ebc05f770 msgr2=0x7f2ebc1435f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:08.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.240+0000 7f2ec083d700 1 -- 172.21.15.36:0/2233972017 shutdown_connections 2022-06-17T12:54:08.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.240+0000 7f2ec083d700 1 -- 172.21.15.36:0/2233972017 wait complete. 2022-06-17T12:54:08.231 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 38 2022-06-17T12:54:08.238 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-17T12:54:08.238 INFO:tasks.ceph.mds.a:Stopping old one... 2022-06-17T12:54:08.239 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-17T12:54:08.239 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T12:54:08.240 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-17T12:54:08.240 INFO:tasks.ceph.mds.b:Stopping old one... 2022-06-17T12:54:08.240 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-17T12:54:08.241 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T12:54:08.253 INFO:tasks.ceph.mds.a:Stopped 2022-06-17T12:54:08.253 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-17T12:54:08.256 INFO:tasks.ceph.mds.a:Started 2022-06-17T12:54:08.256 DEBUG:teuthology.parallel:result is None 2022-06-17T12:54:08.258 INFO:tasks.ceph.mds.b:Stopped 2022-06-17T12:54:08.258 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-17T12:54:08.303 INFO:tasks.ceph.mds.b:Started 2022-06-17T12:54:08.304 DEBUG:teuthology.parallel:result is None 2022-06-17T12:54:08.305 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-06-17T12:54:08.326 INFO:tasks.ceph.mds.a.smithi036.stdout:starting mds.a at 2022-06-17T12:54:08.382 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2022-06-17T12:54:08.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.416+0000 7fac848bd700 1 Processor -- start 2022-06-17T12:54:08.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.416+0000 7fac848bd700 1 -- start start 2022-06-17T12:54:08.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac848bd700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac80060ac0 0x7fac800afe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:08.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac848bd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fac800aee30 0x7fac8013c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:08.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac848bd700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac80060ea0 con 0x7fac800b20f0 2022-06-17T12:54:08.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac848bd700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac800b3060 con 0x7fac800aee30 2022-06-17T12:54:08.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac848bd700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac80067a20 con 0x7fac80060ac0 2022-06-17T12:54:08.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac7e59c700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fac800b20f0 0x7fac800ae720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53710/0 (socket says 172.21.15.36:53710) 2022-06-17T12:54:08.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac7e59c700 1 -- 172.21.15.36:0/709049651 learned_addr learned my addr 172.21.15.36:0/709049651 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:08.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac7f59e700 1 --2- 172.21.15.36:0/709049651 >> v2:172.21.15.36:3300/0 conn(0x7fac800aee30 0x7fac8013c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:08.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac7ed9d700 1 --2- 172.21.15.36:0/709049651 >> v2:172.21.15.70:3300/0 conn(0x7fac80060ac0 0x7fac800afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:08.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac7dd9b700 1 -- 172.21.15.36:0/709049651 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1939239715 0 0) 0x7fac80060ea0 con 0x7fac800b20f0 2022-06-17T12:54:08.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac7dd9b700 1 -- 172.21.15.36:0/709049651 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac64003260 con 0x7fac800b20f0 2022-06-17T12:54:08.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac7f59e700 1 -- 172.21.15.36:0/709049651 >> v2:172.21.15.70:3300/0 conn(0x7fac80060ac0 msgr2=0x7fac800afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:08.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac7f59e700 1 --2- 172.21.15.36:0/709049651 >> v2:172.21.15.70:3300/0 conn(0x7fac80060ac0 0x7fac800afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:08.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac7f59e700 1 -- 172.21.15.36:0/709049651 >> v1:172.21.15.36:6789/0 conn(0x7fac800b20f0 legacy=0x7fac800ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:08.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac7dd9b700 1 -- 172.21.15.36:0/709049651 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 832379276 0 0) 0x7fac64003260 con 0x7fac800b20f0 2022-06-17T12:54:08.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac7f59e700 1 -- 172.21.15.36:0/709049651 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac80143fb0 con 0x7fac800aee30 2022-06-17T12:54:08.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac7f59e700 1 --2- 172.21.15.36:0/709049651 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fac800aee30 0x7fac8013c7e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7fac74001800 tx=0x7fac780091d0).ready entity=mon.0 client_cookie=b6e50ab612b30367 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:08.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac7dd9b700 1 -- 172.21.15.36:0/709049651 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac74001660 con 0x7fac800aee30 2022-06-17T12:54:08.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac7dd9b700 1 -- 172.21.15.36:0/709049651 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac7800f510 con 0x7fac800aee30 2022-06-17T12:54:08.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac7dd9b700 1 -- 172.21.15.36:0/709049651 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac7800f680 con 0x7fac800aee30 2022-06-17T12:54:08.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac848bd700 1 -- 172.21.15.36:0/709049651 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fac800aee30 msgr2=0x7fac8013c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:08.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac848bd700 1 --2- 172.21.15.36:0/709049651 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fac800aee30 0x7fac8013c7e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7fac74001800 tx=0x7fac780091d0).stop 2022-06-17T12:54:08.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac848bd700 1 -- 172.21.15.36:0/709049651 shutdown_connections 2022-06-17T12:54:08.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac848bd700 1 --2- 172.21.15.36:0/709049651 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fac800aee30 0x7fac8013c7e0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:08.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac848bd700 1 --2- 172.21.15.36:0/709049651 >> v2:172.21.15.70:3300/0 conn(0x7fac80060ac0 0x7fac800afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:08.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac848bd700 1 -- 172.21.15.36:0/709049651 >> 172.21.15.36:0/709049651 conn(0x7fac800b65f0 msgr2=0x7fac800b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:08.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac848bd700 1 -- 172.21.15.36:0/709049651 shutdown_connections 2022-06-17T12:54:08.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.420+0000 7fac848bd700 1 -- 172.21.15.36:0/709049651 wait complete. 2022-06-17T12:54:08.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac848bd700 1 Processor -- start 2022-06-17T12:54:08.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac848bd700 1 -- start start 2022-06-17T12:54:08.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac848bd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac80060ac0 0x7fac800b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:08.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac848bd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac800aee30 0x7fac801393a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:08.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac848bd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fac800b20f0 0x7fac80139890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:08.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac848bd700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac801530d0 con 0x7fac800b20f0 2022-06-17T12:54:08.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac848bd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac80152e30 con 0x7fac80060ac0 2022-06-17T12:54:08.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac848bd700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac80152b50 con 0x7fac800aee30 2022-06-17T12:54:08.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac7f59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fac800b20f0 0x7fac80139890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:08.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac7f59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fac800b20f0 0x7fac80139890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36190/0 (socket says 172.21.15.36:36190) 2022-06-17T12:54:08.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac7f59e700 1 -- 172.21.15.36:0/516524669 learned_addr learned my addr 172.21.15.36:0/516524669 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:08.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac7ed9d700 1 --2- 172.21.15.36:0/516524669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac80060ac0 0x7fac800b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:08.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac7e59c700 1 --2- 172.21.15.36:0/516524669 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac800aee30 0x7fac801393a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:08.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac7f59e700 1 -- 172.21.15.36:0/516524669 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac800aee30 msgr2=0x7fac801393a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:08.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac7f59e700 1 --2- 172.21.15.36:0/516524669 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac800aee30 0x7fac801393a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:08.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac7f59e700 1 -- 172.21.15.36:0/516524669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac80060ac0 msgr2=0x7fac800b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:08.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac7f59e700 1 --2- 172.21.15.36:0/516524669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac80060ac0 0x7fac800b0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:08.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac7f59e700 1 -- 172.21.15.36:0/516524669 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac78009470 con 0x7fac800b20f0 2022-06-17T12:54:08.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac7f59e700 1 --2- 172.21.15.36:0/516524669 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fac800b20f0 0x7fac80139890 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7fac74001800 tx=0x7fac78016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:08.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac6ffff700 1 -- 172.21.15.36:0/516524669 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac780032d0 con 0x7fac800b20f0 2022-06-17T12:54:08.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac6ffff700 1 -- 172.21.15.36:0/516524669 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac78011ad0 con 0x7fac800b20f0 2022-06-17T12:54:08.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac848bd700 1 -- 172.21.15.36:0/516524669 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac80139d80 con 0x7fac800b20f0 2022-06-17T12:54:08.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac6ffff700 1 -- 172.21.15.36:0/516524669 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac78011e30 con 0x7fac800b20f0 2022-06-17T12:54:08.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac848bd700 1 -- 172.21.15.36:0/516524669 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac801462c0 con 0x7fac800b20f0 2022-06-17T12:54:08.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.424+0000 7fac848bd700 1 -- 172.21.15.36:0/516524669 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac80142410 con 0x7fac800b20f0 2022-06-17T12:54:08.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.428+0000 7fac6ffff700 1 -- 172.21.15.36:0/516524669 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fac78026030 con 0x7fac800b20f0 2022-06-17T12:54:08.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.428+0000 7fac6ffff700 1 --2- 172.21.15.36:0/516524669 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fac5c053af0 0x7fac5c055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:08.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.428+0000 7fac6ffff700 1 -- 172.21.15.36:0/516524669 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5729+0+0 (secure 0 0 0) 0x7fac7806c640 con 0x7fac800b20f0 2022-06-17T12:54:08.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.428+0000 7fac7ed9d700 1 --2- 172.21.15.36:0/516524669 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fac5c053af0 0x7fac5c055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:08.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.428+0000 7fac7ed9d700 1 --2- 172.21.15.36:0/516524669 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fac5c053af0 0x7fac5c055fb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fac7000b000 tx=0x7fac70006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:08.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.432+0000 7fac6ffff700 1 -- 172.21.15.36:0/516524669 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fac78043650 con 0x7fac800b20f0 2022-06-17T12:54:08.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:08.568+0000 7fac848bd700 1 -- 172.21.15.36:0/516524669 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fac80140ae0 con 0x7fac800b20f0 2022-06-17T12:54:09.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.360+0000 7fac6ffff700 1 -- 172.21.15.36:0/516524669 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v80) v1 ==== 97+0+0 (secure 0 0 0) 0x7fac78015080 con 0x7fac800b20f0 2022-06-17T12:54:09.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.364+0000 7fac848bd700 1 -- 172.21.15.36:0/516524669 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fac5c053af0 msgr2=0x7fac5c055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:09.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.364+0000 7fac848bd700 1 --2- 172.21.15.36:0/516524669 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fac5c053af0 0x7fac5c055fb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fac7000b000 tx=0x7fac70006df0).stop 2022-06-17T12:54:09.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.364+0000 7fac848bd700 1 -- 172.21.15.36:0/516524669 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fac800b20f0 msgr2=0x7fac80139890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:09.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.364+0000 7fac848bd700 1 --2- 172.21.15.36:0/516524669 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fac800b20f0 0x7fac80139890 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7fac74001800 tx=0x7fac78016000).stop 2022-06-17T12:54:09.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.364+0000 7fac848bd700 1 -- 172.21.15.36:0/516524669 shutdown_connections 2022-06-17T12:54:09.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.364+0000 7fac848bd700 1 --2- 172.21.15.36:0/516524669 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fac5c053af0 0x7fac5c055fb0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:09.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.364+0000 7fac848bd700 1 --2- 172.21.15.36:0/516524669 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fac800b20f0 0x7fac80139890 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:09.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.364+0000 7fac848bd700 1 --2- 172.21.15.36:0/516524669 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac800aee30 0x7fac801393a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:09.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.364+0000 7fac848bd700 1 --2- 172.21.15.36:0/516524669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac80060ac0 0x7fac800b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:09.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.364+0000 7fac848bd700 1 -- 172.21.15.36:0/516524669 >> 172.21.15.36:0/516524669 conn(0x7fac800b65f0 msgr2=0x7fac800b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:09.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.364+0000 7fac848bd700 1 -- 172.21.15.36:0/516524669 shutdown_connections 2022-06-17T12:54:09.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.364+0000 7fac848bd700 1 -- 172.21.15.36:0/516524669 wait complete. 2022-06-17T12:54:09.361 INFO:teuthology.orchestra.run.smithi036.stderr: removed all blacklist entries 2022-06-17T12:54:09.375 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2022-06-17T12:54:09.376 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-06-17T12:54:09.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.544+0000 7f4245e55700 1 Processor -- start 2022-06-17T12:54:09.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.544+0000 7f4245e55700 1 -- start start 2022-06-17T12:54:09.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.544+0000 7f4245e55700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42400b1680 0x7f42400630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:09.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.544+0000 7f4245e55700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f42400ae6c0 0x7f424013fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:09.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.544+0000 7f4245e55700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4240062c80 con 0x7f4240060ac0 2022-06-17T12:54:09.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.544+0000 7f4245e55700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4240060ea0 con 0x7f42400ae6c0 2022-06-17T12:54:09.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.544+0000 7f4245e55700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42400b2f50 con 0x7f42400b1680 2022-06-17T12:54:09.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.544+0000 7f4245654700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f42400ae6c0 0x7f424013fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:09.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.544+0000 7f4244e53700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4240060ac0 0x7f42401431a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:49460/0 (socket says 172.21.15.36:49460) 2022-06-17T12:54:09.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.544+0000 7f4245654700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f42400ae6c0 0x7f424013fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36200/0 (socket says 172.21.15.36:36200) 2022-06-17T12:54:09.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.544+0000 7f4244e53700 1 -- 172.21.15.36:0/858999970 learned_addr learned my addr 172.21.15.36:0/858999970 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:09.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.544+0000 7f423ffff700 1 --2- 172.21.15.36:0/858999970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42400b1680 0x7f42400630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:09.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.548+0000 7f4245654700 1 -- 172.21.15.36:0/858999970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42400b1680 msgr2=0x7f42400630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:09.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.548+0000 7f4245654700 1 --2- 172.21.15.36:0/858999970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42400b1680 0x7f42400630e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:09.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.548+0000 7f4245654700 1 -- 172.21.15.36:0/858999970 >> v1:172.21.15.70:6789/0 conn(0x7f4240060ac0 legacy=0x7f42401431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:09.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.548+0000 7f4245654700 1 -- 172.21.15.36:0/858999970 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42400635d0 con 0x7f42400ae6c0 2022-06-17T12:54:09.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.548+0000 7f4245654700 1 --2- 172.21.15.36:0/858999970 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f42400ae6c0 0x7f424013fef0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7f4238002f70 tx=0x7f4238006df0).ready entity=mon.0 client_cookie=1772c15e801f9a85 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:09.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.548+0000 7f423f7fe700 1 -- 172.21.15.36:0/858999970 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f423800d570 con 0x7f42400ae6c0 2022-06-17T12:54:09.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.548+0000 7f423f7fe700 1 -- 172.21.15.36:0/858999970 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4238000e90 con 0x7f42400ae6c0 2022-06-17T12:54:09.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.548+0000 7f423f7fe700 1 -- 172.21.15.36:0/858999970 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f423800d8c0 con 0x7f42400ae6c0 2022-06-17T12:54:09.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.548+0000 7f4245e55700 1 -- 172.21.15.36:0/858999970 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f42400ae6c0 msgr2=0x7f424013fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:09.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.548+0000 7f4245e55700 1 --2- 172.21.15.36:0/858999970 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f42400ae6c0 0x7f424013fef0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7f4238002f70 tx=0x7f4238006df0).stop 2022-06-17T12:54:09.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.548+0000 7f4245e55700 1 -- 172.21.15.36:0/858999970 shutdown_connections 2022-06-17T12:54:09.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.548+0000 7f4245e55700 1 --2- 172.21.15.36:0/858999970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42400b1680 0x7f42400630e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:09.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.548+0000 7f4245e55700 1 --2- 172.21.15.36:0/858999970 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f42400ae6c0 0x7f424013fef0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:09.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.548+0000 7f4245e55700 1 -- 172.21.15.36:0/858999970 >> 172.21.15.36:0/858999970 conn(0x7f42400b64f0 msgr2=0x7f42400b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:09.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.548+0000 7f4245e55700 1 -- 172.21.15.36:0/858999970 shutdown_connections 2022-06-17T12:54:09.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.548+0000 7f4245e55700 1 -- 172.21.15.36:0/858999970 wait complete. 2022-06-17T12:54:09.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.548+0000 7f4245e55700 1 Processor -- start 2022-06-17T12:54:09.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.548+0000 7f4245e55700 1 -- start start 2022-06-17T12:54:09.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.552+0000 7f4245e55700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4240060ac0 0x7f424013e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:09.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.552+0000 7f4245e55700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42400ae6c0 0x7f42401455c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:09.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.552+0000 7f4245e55700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f42400b1680 0x7f42400b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:09.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.552+0000 7f4245e55700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42401453d0 con 0x7f42400b1680 2022-06-17T12:54:09.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.552+0000 7f4245e55700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42401461a0 con 0x7f42400ae6c0 2022-06-17T12:54:09.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.552+0000 7f4245e55700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4240145c60 con 0x7f4240060ac0 2022-06-17T12:54:09.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.552+0000 7f4245654700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f42400b1680 0x7f42400b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:09.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.552+0000 7f423ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42400ae6c0 0x7f42401455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:09.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.552+0000 7f4244e53700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4240060ac0 0x7f424013e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:09.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.552+0000 7f4245654700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f42400b1680 0x7f42400b0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36206/0 (socket says 172.21.15.36:36206) 2022-06-17T12:54:09.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.552+0000 7f4245654700 1 -- 172.21.15.36:0/637643682 learned_addr learned my addr 172.21.15.36:0/637643682 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:09.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.552+0000 7f4245654700 1 -- 172.21.15.36:0/637643682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4240060ac0 msgr2=0x7f424013e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:09.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.552+0000 7f4245654700 1 --2- 172.21.15.36:0/637643682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4240060ac0 0x7f424013e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:09.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.552+0000 7f4245654700 1 -- 172.21.15.36:0/637643682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42400ae6c0 msgr2=0x7f42401455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:09.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.552+0000 7f4245654700 1 --2- 172.21.15.36:0/637643682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42400ae6c0 0x7f42401455c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:09.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.552+0000 7f4245654700 1 -- 172.21.15.36:0/637643682 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423800b180 con 0x7f42400b1680 2022-06-17T12:54:09.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.552+0000 7f423ffff700 1 --2- 172.21.15.36:0/637643682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42400ae6c0 0x7f42401455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:09.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.552+0000 7f4244e53700 1 --2- 172.21.15.36:0/637643682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4240060ac0 0x7f424013e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:09.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.552+0000 7f4245654700 1 --2- 172.21.15.36:0/637643682 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f42400b1680 0x7f42400b0a30 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f4238002f70 tx=0x7f4238014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:09.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.552+0000 7f423dffb700 1 -- 172.21.15.36:0/637643682 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42380139c0 con 0x7f42400b1680 2022-06-17T12:54:09.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.552+0000 7f423dffb700 1 -- 172.21.15.36:0/637643682 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4238013b20 con 0x7f42400b1680 2022-06-17T12:54:09.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.552+0000 7f4245e55700 1 -- 172.21.15.36:0/637643682 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4240145f00 con 0x7f42400b1680 2022-06-17T12:54:09.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.552+0000 7f423dffb700 1 -- 172.21.15.36:0/637643682 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4238013e80 con 0x7f42400b1680 2022-06-17T12:54:09.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.552+0000 7f4245e55700 1 -- 172.21.15.36:0/637643682 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f424009faa0 con 0x7f42400b1680 2022-06-17T12:54:09.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.556+0000 7f4245e55700 1 -- 172.21.15.36:0/637643682 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42400b0670 con 0x7f42400b1680 2022-06-17T12:54:09.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.556+0000 7f423dffb700 1 -- 172.21.15.36:0/637643682 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f4238025030 con 0x7f42400b1680 2022-06-17T12:54:09.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.556+0000 7f423dffb700 1 --2- 172.21.15.36:0/637643682 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f42200539b0 0x7f4220055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:09.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.556+0000 7f423dffb700 1 -- 172.21.15.36:0/637643682 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f423806cac0 con 0x7f42400b1680 2022-06-17T12:54:09.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.556+0000 7f4244e53700 1 --2- 172.21.15.36:0/637643682 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f42200539b0 0x7f4220055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:09.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.556+0000 7f4244e53700 1 --2- 172.21.15.36:0/637643682 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f42200539b0 0x7f4220055e70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f423000d000 tx=0x7f4230007f90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:09.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.564+0000 7f423dffb700 1 -- 172.21.15.36:0/637643682 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4238043af0 con 0x7f42400b1680 2022-06-17T12:54:09.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.732+0000 7f4245e55700 1 -- 172.21.15.36:0/637643682 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4240145ab0 con 0x7f42400b1680 2022-06-17T12:54:09.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.736+0000 7f423dffb700 1 -- 172.21.15.36:0/637643682 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v27) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f4238023070 con 0x7f42400b1680 2022-06-17T12:54:09.722 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:09.722 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:54:09.723 INFO:teuthology.orchestra.run.smithi036.stdout: "auth_dump": [ 2022-06-17T12:54:09.723 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:09.723 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mds.a", 2022-06-17T12:54:09.724 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiyaUgARAAxp7bzOl4uVo2EsaoRBeMRw==", 2022-06-17T12:54:09.724 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:09.725 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:54:09.726 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:09.726 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:09.726 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:09.727 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:09.727 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:09.728 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:09.728 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mds.b", 2022-06-17T12:54:09.729 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xifyxLCRAABI389jmz1zboeDanzraT8Q==", 2022-06-17T12:54:09.729 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:09.729 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:54:09.730 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:09.730 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:09.731 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:09.731 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:09.731 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:09.732 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:09.732 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.0", 2022-06-17T12:54:09.732 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB+d6xibhXGNBAA6AuUT9ITWAZjDFpNsgA4Wg==", 2022-06-17T12:54:09.733 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:09.733 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:09.733 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:09.734 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:09.735 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:09.735 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:09.736 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:09.736 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.1", 2022-06-17T12:54:09.736 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCBd6xiOIPvGxAADkrBoJCy9x91zan6TWU0tQ==", 2022-06-17T12:54:09.737 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:09.737 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:09.738 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:09.738 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:09.738 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:09.739 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:09.739 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:09.740 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.2", 2022-06-17T12:54:09.740 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCEd6xiP4/9AhAAaP3cWXV6vbfGBPfqSLKqUg==", 2022-06-17T12:54:09.741 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:09.741 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:09.741 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:09.742 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:09.743 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:09.743 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:09.743 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:09.744 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.3", 2022-06-17T12:54:09.744 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCGd6xi+LZgJxAAJjGwP5qW6TFDZfR+2rz6ig==", 2022-06-17T12:54:09.745 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:09.745 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:09.746 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:09.746 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:09.747 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:09.747 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:09.748 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:09.748 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.4", 2022-06-17T12:54:09.748 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCKd6xikvrTChAAXIyWSETswDv+tb3zsGyYpA==", 2022-06-17T12:54:09.749 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:09.749 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:09.749 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:09.750 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:09.750 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:09.750 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:09.751 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:09.751 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.5", 2022-06-17T12:54:09.751 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCMd6xiqr2QLhAArhimHuNsVWrDGGreFYxj6A==", 2022-06-17T12:54:09.752 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:09.752 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:09.752 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:09.752 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:09.753 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:09.753 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:09.753 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:09.753 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.6", 2022-06-17T12:54:09.754 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCPd6xiUXcXGxAAdrYz8txdBWMIqF5DF4Aimw==", 2022-06-17T12:54:09.754 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:09.754 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:09.754 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:09.755 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:09.755 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:09.755 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:09.755 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:09.755 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.7", 2022-06-17T12:54:09.756 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCSd6xiBJ2IAhAAUkNfgmlxxWY7+VouPLAvyw==", 2022-06-17T12:54:09.756 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:09.757 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:09.757 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:09.757 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:09.757 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:09.758 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:09.758 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:09.758 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.0", 2022-06-17T12:54:09.758 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiXO3QFBAAsSLtQ0yKKZxhqUT/eYKdtg==", 2022-06-17T12:54:09.759 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:09.759 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:54:09.759 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:54:09.759 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:54:09.760 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:09.760 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:09.760 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:09.761 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.1", 2022-06-17T12:54:09.761 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xi51KdGBAAi7fGZ+Qepg6j3DAhlzoWxw==", 2022-06-17T12:54:09.761 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:09.761 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:54:09.762 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:54:09.762 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:54:09.762 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:09.762 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:09.763 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:09.763 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.2", 2022-06-17T12:54:09.763 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiug2hHBAAqHpLgwMdthSWtJ8Vwu+m1Q==", 2022-06-17T12:54:09.764 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:09.764 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:54:09.764 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:54:09.765 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:54:09.765 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:09.766 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:09.766 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:09.767 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.admin", 2022-06-17T12:54:09.767 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB5d6xiGDODKhAAPZMG/M2nVssdr4FJHcS6XQ==", 2022-06-17T12:54:09.768 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:09.768 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:54:09.768 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:09.769 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:09.769 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:09.770 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:09.770 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:09.771 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:09.771 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mgr.x", 2022-06-17T12:54:09.772 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB6d6xiyH9uORAATJb0chHE1CH0X02w7edVtw==", 2022-06-17T12:54:09.772 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:09.773 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:54:09.773 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile mgr", 2022-06-17T12:54:09.774 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:09.774 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:09.775 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:09.775 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:09.775 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mgr.y", 2022-06-17T12:54:09.776 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB6d6xiHXghMxAAzpsVRCbLWHWZPtrPk1jFlA==", 2022-06-17T12:54:09.776 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:09.776 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:54:09.777 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile mgr", 2022-06-17T12:54:09.777 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:09.778 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:09.779 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:09.779 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:09.779 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:54:09.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.736+0000 7f4245e55700 1 -- 172.21.15.36:0/637643682 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f42200539b0 msgr2=0x7f4220055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:09.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.736+0000 7f4245e55700 1 --2- 172.21.15.36:0/637643682 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f42200539b0 0x7f4220055e70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f423000d000 tx=0x7f4230007f90).stop 2022-06-17T12:54:09.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.736+0000 7f4245e55700 1 -- 172.21.15.36:0/637643682 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f42400b1680 msgr2=0x7f42400b0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:09.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.736+0000 7f4245e55700 1 --2- 172.21.15.36:0/637643682 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f42400b1680 0x7f42400b0a30 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f4238002f70 tx=0x7f4238014000).stop 2022-06-17T12:54:09.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.736+0000 7f4245e55700 1 -- 172.21.15.36:0/637643682 shutdown_connections 2022-06-17T12:54:09.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.736+0000 7f4245e55700 1 --2- 172.21.15.36:0/637643682 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f42200539b0 0x7f4220055e70 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:09.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.736+0000 7f4245e55700 1 --2- 172.21.15.36:0/637643682 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f42400b1680 0x7f42400b0a30 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:09.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.736+0000 7f4245e55700 1 --2- 172.21.15.36:0/637643682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42400ae6c0 0x7f42401455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:09.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.736+0000 7f4245e55700 1 --2- 172.21.15.36:0/637643682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4240060ac0 0x7f424013e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:09.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.736+0000 7f4245e55700 1 -- 172.21.15.36:0/637643682 >> 172.21.15.36:0/637643682 conn(0x7f42400b64f0 msgr2=0x7f424013f180 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:09.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.736+0000 7f4245e55700 1 -- 172.21.15.36:0/637643682 shutdown_connections 2022-06-17T12:54:09.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.736+0000 7f4245e55700 1 -- 172.21.15.36:0/637643682 wait complete. 2022-06-17T12:54:09.796 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:54:09.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.928+0000 7f67d4a84700 1 Processor -- start 2022-06-17T12:54:09.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.928+0000 7f67d4a84700 1 -- start start 2022-06-17T12:54:09.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.928+0000 7f67d4a84700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f67d00b1280 0x7f67d013fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:09.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.928+0000 7f67d4a84700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d00b1660 con 0x7f67d00abe30 2022-06-17T12:54:09.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.928+0000 7f67d4a84700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d00ac3b0 con 0x7f67d0140410 2022-06-17T12:54:09.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.928+0000 7f67d4a84700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d00aef10 con 0x7f67d00b1280 2022-06-17T12:54:09.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67ceffd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f67d00abe30 0x7f67d0062ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53744/0 (socket says 172.21.15.36:53744) 2022-06-17T12:54:09.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67ceffd700 1 -- 172.21.15.36:0/1934906997 learned_addr learned my addr 172.21.15.36:0/1934906997 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:09.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67cf7fe700 1 --2- 172.21.15.36:0/1934906997 >> v2:172.21.15.70:3300/0 conn(0x7f67d00b1280 0x7f67d013fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:09.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67ce7fc700 1 -- 172.21.15.36:0/1934906997 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3194950195 0 0) 0x7f67d00b1660 con 0x7f67d00abe30 2022-06-17T12:54:09.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67ce7fc700 1 -- 172.21.15.36:0/1934906997 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67bc003260 con 0x7f67d00abe30 2022-06-17T12:54:09.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67ce7fc700 1 -- 172.21.15.36:0/1934906997 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1061561177 0 0) 0x7f67d00ac3b0 con 0x7f67d0140410 2022-06-17T12:54:09.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67ce7fc700 1 -- 172.21.15.36:0/1934906997 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67d00b1660 con 0x7f67d0140410 2022-06-17T12:54:09.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67ce7fc700 1 -- 172.21.15.36:0/1934906997 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2204912452 0 0) 0x7f67bc003260 con 0x7f67d00abe30 2022-06-17T12:54:09.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67ce7fc700 1 -- 172.21.15.36:0/1934906997 >> v2:172.21.15.70:3300/0 conn(0x7f67d00b1280 msgr2=0x7f67d013fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:09.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67ce7fc700 1 --2- 172.21.15.36:0/1934906997 >> v2:172.21.15.70:3300/0 conn(0x7f67d00b1280 0x7f67d013fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:09.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67ce7fc700 1 -- 172.21.15.36:0/1934906997 >> v1:172.21.15.70:6789/0 conn(0x7f67d0140410 legacy=0x7f67d01436d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:09.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67ce7fc700 1 -- 172.21.15.36:0/1934906997 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d00ac650 con 0x7f67d00abe30 2022-06-17T12:54:09.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67ce7fc700 1 -- 172.21.15.36:0/1934906997 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f67c4001660 con 0x7f67d00abe30 2022-06-17T12:54:09.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67ce7fc700 1 -- 172.21.15.36:0/1934906997 >> v1:172.21.15.36:6789/0 conn(0x7f67d00abe30 legacy=0x7f67d0062ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:09.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67ce7fc700 1 --2- 172.21.15.36:0/1934906997 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67bc0038e0 0x7f67bc003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:09.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67ce7fc700 1 --2- 172.21.15.36:0/1934906997 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67bc004500 0x7f67bc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:09.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67ce7fc700 1 --2- 172.21.15.36:0/1934906997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67bc004e70 0x7f67bc00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:09.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67ce7fc700 1 -- 172.21.15.36:0/1934906997 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d00aef10 con 0x7f67bc0038e0 2022-06-17T12:54:09.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67ce7fc700 1 -- 172.21.15.36:0/1934906997 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc00bcf0 con 0x7f67bc004e70 2022-06-17T12:54:09.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67ce7fc700 1 -- 172.21.15.36:0/1934906997 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc00beb0 con 0x7f67bc004500 2022-06-17T12:54:09.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67cf7fe700 1 --2- 172.21.15.36:0/1934906997 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67bc0038e0 0x7f67bc003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:09.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67ceffd700 1 --2- 172.21.15.36:0/1934906997 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67bc004500 0x7f67bc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:09.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67cffff700 1 --2- 172.21.15.36:0/1934906997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67bc004e70 0x7f67bc00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:09.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67cf7fe700 1 -- 172.21.15.36:0/1934906997 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67bc004500 msgr2=0x7f67bc004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:09.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67cf7fe700 1 --2- 172.21.15.36:0/1934906997 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67bc004500 0x7f67bc004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:09.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67cf7fe700 1 -- 172.21.15.36:0/1934906997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67bc004e70 msgr2=0x7f67bc00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:09.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67cf7fe700 1 --2- 172.21.15.36:0/1934906997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67bc004e70 0x7f67bc00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:09.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67cf7fe700 1 -- 172.21.15.36:0/1934906997 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc003260 con 0x7f67bc0038e0 2022-06-17T12:54:09.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67cffff700 1 --2- 172.21.15.36:0/1934906997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67bc004e70 0x7f67bc00b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:54:09.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67cffff700 1 -- 172.21.15.36:0/1934906997 reap_dead start 2022-06-17T12:54:09.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67cf7fe700 1 --2- 172.21.15.36:0/1934906997 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67bc0038e0 0x7f67bc003ce0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7f67c000c000 tx=0x7f67c0006df0).ready entity=mon.0 client_cookie=a5ae9a313d625c4d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:09.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67ce7fc700 1 -- 172.21.15.36:0/1934906997 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67c000e800 con 0x7f67bc0038e0 2022-06-17T12:54:09.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67ce7fc700 1 -- 172.21.15.36:0/1934906997 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c000e960 con 0x7f67bc0038e0 2022-06-17T12:54:09.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67ce7fc700 1 -- 172.21.15.36:0/1934906997 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67c000ecc0 con 0x7f67bc0038e0 2022-06-17T12:54:09.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67d4a84700 1 -- 172.21.15.36:0/1934906997 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67bc0038e0 msgr2=0x7f67bc003ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:09.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67d4a84700 1 --2- 172.21.15.36:0/1934906997 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67bc0038e0 0x7f67bc003ce0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7f67c000c000 tx=0x7f67c0006df0).stop 2022-06-17T12:54:09.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67d4a84700 1 -- 172.21.15.36:0/1934906997 shutdown_connections 2022-06-17T12:54:09.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67d4a84700 1 --2- 172.21.15.36:0/1934906997 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67bc0038e0 0x7f67bc003ce0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:09.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67d4a84700 1 -- 172.21.15.36:0/1934906997 >> 172.21.15.36:0/1934906997 conn(0x7f67d005f770 msgr2=0x7f67d005fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:09.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67d4a84700 1 -- 172.21.15.36:0/1934906997 shutdown_connections 2022-06-17T12:54:09.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.932+0000 7f67d4a84700 1 -- 172.21.15.36:0/1934906997 wait complete. 2022-06-17T12:54:09.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.936+0000 7f67d4a84700 1 Processor -- start 2022-06-17T12:54:09.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.936+0000 7f67d4a84700 1 -- start start 2022-06-17T12:54:09.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.940+0000 7f67d4a84700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67bc0038e0 0x7f67d00b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:09.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.940+0000 7f67d4a84700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67d0141090 0x7f67d013c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:09.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.940+0000 7f67d4a84700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67d0141510 0x7f67d013cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:09.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.940+0000 7f67d4a84700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d013b1e0 con 0x7f67d0141510 2022-06-17T12:54:09.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.940+0000 7f67d4a84700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d012d950 con 0x7f67d0141090 2022-06-17T12:54:09.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.940+0000 7f67d4a84700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d00b5a30 con 0x7f67bc0038e0 2022-06-17T12:54:09.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.940+0000 7f67cffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67d0141510 0x7f67d013cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:09.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.940+0000 7f67ceffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67d0141090 0x7f67d013c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:09.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.940+0000 7f67cf7fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67bc0038e0 0x7f67d00b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:09.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.940+0000 7f67ceffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67d0141090 0x7f67d013c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:56606/0 (socket says 172.21.15.36:56606) 2022-06-17T12:54:09.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.940+0000 7f67ceffd700 1 -- 172.21.15.36:0/2807050350 learned_addr learned my addr 172.21.15.36:0/2807050350 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:09.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.940+0000 7f67cf7fe700 1 -- 172.21.15.36:0/2807050350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67d0141090 msgr2=0x7f67d013c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:09.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.940+0000 7f67cf7fe700 1 --2- 172.21.15.36:0/2807050350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67d0141090 0x7f67d013c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:09.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.940+0000 7f67cf7fe700 1 -- 172.21.15.36:0/2807050350 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67d0141510 msgr2=0x7f67d013cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:09.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.940+0000 7f67cf7fe700 1 --2- 172.21.15.36:0/2807050350 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67d0141510 0x7f67d013cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:09.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.940+0000 7f67cf7fe700 1 -- 172.21.15.36:0/2807050350 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d012f0a0 con 0x7f67bc0038e0 2022-06-17T12:54:09.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.940+0000 7f67cf7fe700 1 --2- 172.21.15.36:0/2807050350 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67bc0038e0 0x7f67d00b5ed0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f67c000c000 tx=0x7f67c0019000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:09.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.940+0000 7f67ccff9700 1 -- 172.21.15.36:0/2807050350 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67c001cc30 con 0x7f67bc0038e0 2022-06-17T12:54:09.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.940+0000 7f67ccff9700 1 -- 172.21.15.36:0/2807050350 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c0000e90 con 0x7f67bc0038e0 2022-06-17T12:54:09.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.940+0000 7f67d4a84700 1 -- 172.21.15.36:0/2807050350 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d0142030 con 0x7f67bc0038e0 2022-06-17T12:54:09.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.940+0000 7f67ccff9700 1 -- 172.21.15.36:0/2807050350 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67c000be60 con 0x7f67bc0038e0 2022-06-17T12:54:09.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.940+0000 7f67d4a84700 1 -- 172.21.15.36:0/2807050350 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d012be10 con 0x7f67bc0038e0 2022-06-17T12:54:09.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.940+0000 7f67d4a84700 1 -- 172.21.15.36:0/2807050350 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d00b5bf0 con 0x7f67bc0038e0 2022-06-17T12:54:09.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.944+0000 7f67ccff9700 1 -- 172.21.15.36:0/2807050350 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f67c002d450 con 0x7f67bc0038e0 2022-06-17T12:54:09.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.944+0000 7f67ccff9700 1 --2- 172.21.15.36:0/2807050350 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f67b0053ad0 0x7f67b0055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:09.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.944+0000 7f67ccff9700 1 -- 172.21.15.36:0/2807050350 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f67c006d670 con 0x7f67bc0038e0 2022-06-17T12:54:09.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.944+0000 7f67ceffd700 1 --2- 172.21.15.36:0/2807050350 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f67b0053ad0 0x7f67b0055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:09.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.944+0000 7f67ceffd700 1 --2- 172.21.15.36:0/2807050350 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f67b0053ad0 0x7f67b0055f90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f67c40011d0 tx=0x7f67d00ac2a0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:09.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:09.952+0000 7f67ccff9700 1 -- 172.21.15.36:0/2807050350 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f67c0048180 con 0x7f67bc0038e0 2022-06-17T12:54:10.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.084+0000 7f67d4a84700 1 -- 172.21.15.36:0/2807050350 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67d0143230 con 0x7f67bc0038e0 2022-06-17T12:54:10.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.084+0000 7f67ccff9700 1 -- 172.21.15.36:0/2807050350 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+13415 (secure 0 0 0) 0x7f67c0047d70 con 0x7f67bc0038e0 2022-06-17T12:54:10.074 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:10.074 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":80,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:54:09.357557+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:54:10.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.088+0000 7f67d4a84700 1 -- 172.21.15.36:0/2807050350 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f67b0053ad0 msgr2=0x7f67b0055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:10.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.088+0000 7f67d4a84700 1 --2- 172.21.15.36:0/2807050350 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f67b0053ad0 0x7f67b0055f90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f67c40011d0 tx=0x7f67d00ac2a0).stop 2022-06-17T12:54:10.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.088+0000 7f67d4a84700 1 -- 172.21.15.36:0/2807050350 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67bc0038e0 msgr2=0x7f67d00b5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:10.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.088+0000 7f67d4a84700 1 --2- 172.21.15.36:0/2807050350 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67bc0038e0 0x7f67d00b5ed0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f67c000c000 tx=0x7f67c0019000).stop 2022-06-17T12:54:10.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.088+0000 7f67d4a84700 1 -- 172.21.15.36:0/2807050350 shutdown_connections 2022-06-17T12:54:10.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.088+0000 7f67d4a84700 1 --2- 172.21.15.36:0/2807050350 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f67b0053ad0 0x7f67b0055f90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.088+0000 7f67d4a84700 1 --2- 172.21.15.36:0/2807050350 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67d0141510 0x7f67d013cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.088+0000 7f67d4a84700 1 --2- 172.21.15.36:0/2807050350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67d0141090 0x7f67d013c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.088+0000 7f67d4a84700 1 --2- 172.21.15.36:0/2807050350 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67bc0038e0 0x7f67d00b5ed0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.088+0000 7f67d4a84700 1 -- 172.21.15.36:0/2807050350 >> 172.21.15.36:0/2807050350 conn(0x7f67d005f770 msgr2=0x7f67d01438a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:10.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.088+0000 7f67d4a84700 1 -- 172.21.15.36:0/2807050350 shutdown_connections 2022-06-17T12:54:10.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.088+0000 7f67d4a84700 1 -- 172.21.15.36:0/2807050350 wait complete. 2022-06-17T12:54:10.088 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-17T12:54:10.089 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:54:10.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.216+0000 7f8fd71ab700 1 Processor -- start 2022-06-17T12:54:10.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.216+0000 7f8fd71ab700 1 -- start start 2022-06-17T12:54:10.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.216+0000 7f8fd71ab700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8fd00b6860 0x7f8fd0146d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:10.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.216+0000 7f8fd71ab700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8fd00b18e0 0x7f8fd0143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:10.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.216+0000 7f8fd71ab700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd00b2b70 con 0x7f8fd00b39a0 2022-06-17T12:54:10.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.216+0000 7f8fd71ab700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fd00b6c40 con 0x7f8fd00b6860 2022-06-17T12:54:10.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.216+0000 7f8fd71ab700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fd0142d30 con 0x7f8fd00b18e0 2022-06-17T12:54:10.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.216+0000 7f8fd61a9700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8fd00b6860 0x7f8fd0146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:10.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.216+0000 7f8fd61a9700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8fd00b6860 0x7f8fd0146d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36230/0 (socket says 172.21.15.36:36230) 2022-06-17T12:54:10.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.216+0000 7f8fd59a8700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8fd00b18e0 0x7f8fd0143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:10.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.216+0000 7f8fd69aa700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8fd00b39a0 0x7f8fd0143640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:49498/0 (socket says 172.21.15.36:49498) 2022-06-17T12:54:10.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.216+0000 7f8fd61a9700 1 -- 172.21.15.36:0/1096346774 learned_addr learned my addr 172.21.15.36:0/1096346774 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:10.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.220+0000 7f8fd51a7700 1 -- 172.21.15.36:0/1096346774 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4088247068 0 0) 0x7f8fd00b2b70 con 0x7f8fd00b39a0 2022-06-17T12:54:10.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.220+0000 7f8fd61a9700 1 -- 172.21.15.36:0/1096346774 >> v2:172.21.15.70:3300/0 conn(0x7f8fd00b18e0 msgr2=0x7f8fd0143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:10.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.220+0000 7f8fd61a9700 1 --2- 172.21.15.36:0/1096346774 >> v2:172.21.15.70:3300/0 conn(0x7f8fd00b18e0 0x7f8fd0143150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.220+0000 7f8fd61a9700 1 -- 172.21.15.36:0/1096346774 >> v1:172.21.15.70:6789/0 conn(0x7f8fd00b39a0 legacy=0x7f8fd0143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:10.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.220+0000 7f8fd61a9700 1 -- 172.21.15.36:0/1096346774 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd00b0d30 con 0x7f8fd00b6860 2022-06-17T12:54:10.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.220+0000 7f8fd61a9700 1 --2- 172.21.15.36:0/1096346774 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fd00b6860 0x7f8fd0146d40 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f8fcc002f70 tx=0x7f8fcc006eb0).ready entity=mon.0 client_cookie=a8146fa20b0ae46d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:10.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.220+0000 7f8fd51a7700 1 -- 172.21.15.36:0/1096346774 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fcc00d5c0 con 0x7f8fd00b6860 2022-06-17T12:54:10.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.220+0000 7f8fd51a7700 1 -- 172.21.15.36:0/1096346774 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fcc000e90 con 0x7f8fd00b6860 2022-06-17T12:54:10.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.220+0000 7f8fd51a7700 1 -- 172.21.15.36:0/1096346774 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fcc00d910 con 0x7f8fd00b6860 2022-06-17T12:54:10.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.220+0000 7f8fd71ab700 1 -- 172.21.15.36:0/1096346774 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fd00b6860 msgr2=0x7f8fd0146d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:10.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.220+0000 7f8fd71ab700 1 --2- 172.21.15.36:0/1096346774 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fd00b6860 0x7f8fd0146d40 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f8fcc002f70 tx=0x7f8fcc006eb0).stop 2022-06-17T12:54:10.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.220+0000 7f8fd71ab700 1 -- 172.21.15.36:0/1096346774 shutdown_connections 2022-06-17T12:54:10.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.220+0000 7f8fd71ab700 1 --2- 172.21.15.36:0/1096346774 >> v2:172.21.15.70:3300/0 conn(0x7f8fd00b18e0 0x7f8fd0143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.220+0000 7f8fd71ab700 1 --2- 172.21.15.36:0/1096346774 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fd00b6860 0x7f8fd0146d40 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.220+0000 7f8fd71ab700 1 -- 172.21.15.36:0/1096346774 >> 172.21.15.36:0/1096346774 conn(0x7f8fd00614b0 msgr2=0x7f8fd0061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:10.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.220+0000 7f8fd71ab700 1 -- 172.21.15.36:0/1096346774 shutdown_connections 2022-06-17T12:54:10.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.220+0000 7f8fd71ab700 1 -- 172.21.15.36:0/1096346774 wait complete. 2022-06-17T12:54:10.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.220+0000 7f8fd71ab700 1 Processor -- start 2022-06-17T12:54:10.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.220+0000 7f8fd71ab700 1 -- start start 2022-06-17T12:54:10.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.224+0000 7f8fd71ab700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fd00b18e0 0x7f8fd0152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:10.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.224+0000 7f8fd71ab700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fd00b39a0 0x7f8fd0140bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:10.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.224+0000 7f8fd71ab700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fd00b6860 0x7f8fd01527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:10.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.224+0000 7f8fd71ab700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd00638c0 con 0x7f8fd00b39a0 2022-06-17T12:54:10.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.224+0000 7f8fd71ab700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd0063b60 con 0x7f8fd00b6860 2022-06-17T12:54:10.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.224+0000 7f8fd71ab700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd0155720 con 0x7f8fd00b18e0 2022-06-17T12:54:10.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.224+0000 7f8fd59a8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fd00b39a0 0x7f8fd0140bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:10.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.224+0000 7f8fd59a8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fd00b39a0 0x7f8fd0140bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36238/0 (socket says 172.21.15.36:36238) 2022-06-17T12:54:10.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.224+0000 7f8fd69aa700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fd00b6860 0x7f8fd01527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:10.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.224+0000 7f8fd59a8700 1 -- 172.21.15.36:0/3924055580 learned_addr learned my addr 172.21.15.36:0/3924055580 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:10.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.224+0000 7f8fd61a9700 1 --2- 172.21.15.36:0/3924055580 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fd00b18e0 0x7f8fd0152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:10.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.224+0000 7f8fd59a8700 1 -- 172.21.15.36:0/3924055580 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fd00b18e0 msgr2=0x7f8fd0152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:10.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.224+0000 7f8fd59a8700 1 --2- 172.21.15.36:0/3924055580 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fd00b18e0 0x7f8fd0152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.224+0000 7f8fd59a8700 1 -- 172.21.15.36:0/3924055580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fd00b6860 msgr2=0x7f8fd01527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:10.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.224+0000 7f8fd59a8700 1 --2- 172.21.15.36:0/3924055580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fd00b6860 0x7f8fd01527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.224+0000 7f8fd59a8700 1 -- 172.21.15.36:0/3924055580 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fcc00b1e0 con 0x7f8fd00b39a0 2022-06-17T12:54:10.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.224+0000 7f8fd61a9700 1 --2- 172.21.15.36:0/3924055580 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fd00b18e0 0x7f8fd0152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:10.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.224+0000 7f8fd59a8700 1 --2- 172.21.15.36:0/3924055580 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fd00b39a0 0x7f8fd0140bd0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7f8fc400c000 tx=0x7f8fc4006d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:10.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.224+0000 7f8fc37fe700 1 -- 172.21.15.36:0/3924055580 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fc4011750 con 0x7f8fd00b39a0 2022-06-17T12:54:10.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.224+0000 7f8fc37fe700 1 -- 172.21.15.36:0/3924055580 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc40118b0 con 0x7f8fd00b39a0 2022-06-17T12:54:10.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.224+0000 7f8fd71ab700 1 -- 172.21.15.36:0/3924055580 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd0063580 con 0x7f8fd00b39a0 2022-06-17T12:54:10.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.224+0000 7f8fc37fe700 1 -- 172.21.15.36:0/3924055580 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fc4011c10 con 0x7f8fd00b39a0 2022-06-17T12:54:10.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.224+0000 7f8fd71ab700 1 -- 172.21.15.36:0/3924055580 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd01410c0 con 0x7f8fd00b39a0 2022-06-17T12:54:10.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.224+0000 7f8fd71ab700 1 -- 172.21.15.36:0/3924055580 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd01558e0 con 0x7f8fd00b39a0 2022-06-17T12:54:10.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.228+0000 7f8fc37fe700 1 -- 172.21.15.36:0/3924055580 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8fc4011d70 con 0x7f8fd00b39a0 2022-06-17T12:54:10.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.228+0000 7f8fc37fe700 1 --2- 172.21.15.36:0/3924055580 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8fb0053920 0x7f8fb0055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:10.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.228+0000 7f8fc37fe700 1 -- 172.21.15.36:0/3924055580 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f8fc406bfe0 con 0x7f8fd00b39a0 2022-06-17T12:54:10.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.228+0000 7f8fd61a9700 1 --2- 172.21.15.36:0/3924055580 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8fb0053920 0x7f8fb0055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:10.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.228+0000 7f8fd61a9700 1 --2- 172.21.15.36:0/3924055580 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8fb0053920 0x7f8fb0055de0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f8fcc01a000 tx=0x7f8fcc015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:10.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.232+0000 7f8fc37fe700 1 -- 172.21.15.36:0/3924055580 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8fc4043020 con 0x7f8fd00b39a0 2022-06-17T12:54:10.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.360+0000 7f8fd71ab700 1 -- 172.21.15.36:0/3924055580 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f8fd0057410 con 0x7f8fd00b39a0 2022-06-17T12:54:10.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.360+0000 7f8fc37fe700 1 -- 172.21.15.36:0/3924055580 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v80) v1 ==== 110+0+10 (secure 0 0 0) 0x7f8fc4046640 con 0x7f8fd00b39a0 2022-06-17T12:54:10.351 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:54:10.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.364+0000 7f8fd71ab700 1 -- 172.21.15.36:0/3924055580 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8fb0053920 msgr2=0x7f8fb0055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:10.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.364+0000 7f8fd71ab700 1 --2- 172.21.15.36:0/3924055580 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8fb0053920 0x7f8fb0055de0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f8fcc01a000 tx=0x7f8fcc015000).stop 2022-06-17T12:54:10.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.364+0000 7f8fd71ab700 1 -- 172.21.15.36:0/3924055580 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fd00b39a0 msgr2=0x7f8fd0140bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:10.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.364+0000 7f8fd71ab700 1 --2- 172.21.15.36:0/3924055580 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fd00b39a0 0x7f8fd0140bd0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7f8fc400c000 tx=0x7f8fc4006d90).stop 2022-06-17T12:54:10.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.364+0000 7f8fd71ab700 1 -- 172.21.15.36:0/3924055580 shutdown_connections 2022-06-17T12:54:10.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.364+0000 7f8fd71ab700 1 --2- 172.21.15.36:0/3924055580 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8fb0053920 0x7f8fb0055de0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.364+0000 7f8fd71ab700 1 --2- 172.21.15.36:0/3924055580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fd00b6860 0x7f8fd01527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.364+0000 7f8fd71ab700 1 --2- 172.21.15.36:0/3924055580 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fd00b39a0 0x7f8fd0140bd0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.364+0000 7f8fd71ab700 1 --2- 172.21.15.36:0/3924055580 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fd00b18e0 0x7f8fd0152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.364+0000 7f8fd71ab700 1 -- 172.21.15.36:0/3924055580 >> 172.21.15.36:0/3924055580 conn(0x7f8fd00614b0 msgr2=0x7f8fd013e620 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:10.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.364+0000 7f8fd71ab700 1 -- 172.21.15.36:0/3924055580 shutdown_connections 2022-06-17T12:54:10.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.364+0000 7f8fd71ab700 1 -- 172.21.15.36:0/3924055580 wait complete. 2022-06-17T12:54:10.367 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:54:10.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.492+0000 7fa71359e700 1 Processor -- start 2022-06-17T12:54:10.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.492+0000 7fa71359e700 1 -- start start 2022-06-17T12:54:10.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa71359e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa714060920 0x7fa714146940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:10.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa71359e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7140b31a0 0x7fa714146e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:10.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa71359e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7140b21e0 con 0x7fa7140b2c10 2022-06-17T12:54:10.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa71359e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa714147f30 con 0x7fa714060920 2022-06-17T12:54:10.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa71359e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7141481d0 con 0x7fa7140b31a0 2022-06-17T12:54:10.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa712d9d700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fa7140b2c10 0x7fa71414aa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53778/0 (socket says 172.21.15.36:53778) 2022-06-17T12:54:10.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa712d9d700 1 -- 172.21.15.36:0/332411970 learned_addr learned my addr 172.21.15.36:0/332411970 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:10.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa71259c700 1 --2- 172.21.15.36:0/332411970 >> v2:172.21.15.70:3300/0 conn(0x7fa714060920 0x7fa714146940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:10.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa70bfff700 1 --2- 172.21.15.36:0/332411970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7140b31a0 0x7fa714146e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:10.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa711d9b700 1 -- 172.21.15.36:0/332411970 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 742790597 0 0) 0x7fa7140b21e0 con 0x7fa7140b2c10 2022-06-17T12:54:10.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa711d9b700 1 -- 172.21.15.36:0/332411970 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f4003260 con 0x7fa7140b2c10 2022-06-17T12:54:10.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa70bfff700 1 -- 172.21.15.36:0/332411970 >> v2:172.21.15.70:3300/0 conn(0x7fa714060920 msgr2=0x7fa714146940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:10.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa70bfff700 1 --2- 172.21.15.36:0/332411970 >> v2:172.21.15.70:3300/0 conn(0x7fa714060920 0x7fa714146940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa70bfff700 1 -- 172.21.15.36:0/332411970 >> v1:172.21.15.36:6789/0 conn(0x7fa7140b2c10 legacy=0x7fa71414aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:10.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa70bfff700 1 -- 172.21.15.36:0/332411970 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa714148470 con 0x7fa7140b31a0 2022-06-17T12:54:10.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa70bfff700 1 --2- 172.21.15.36:0/332411970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7140b31a0 0x7fa714146e30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fa6fc002f70 tx=0x7fa6fc006df0).ready entity=mon.2 client_cookie=a69c4b6723c638dd server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:10.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa711d9b700 1 -- 172.21.15.36:0/332411970 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6fc00d4a0 con 0x7fa7140b31a0 2022-06-17T12:54:10.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa711d9b700 1 -- 172.21.15.36:0/332411970 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6fc000e90 con 0x7fa7140b31a0 2022-06-17T12:54:10.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa711d9b700 1 -- 172.21.15.36:0/332411970 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6fc00d7f0 con 0x7fa7140b31a0 2022-06-17T12:54:10.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa71359e700 1 -- 172.21.15.36:0/332411970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7140b31a0 msgr2=0x7fa714146e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:10.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa71359e700 1 --2- 172.21.15.36:0/332411970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7140b31a0 0x7fa714146e30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fa6fc002f70 tx=0x7fa6fc006df0).stop 2022-06-17T12:54:10.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa71359e700 1 -- 172.21.15.36:0/332411970 shutdown_connections 2022-06-17T12:54:10.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa71359e700 1 --2- 172.21.15.36:0/332411970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7140b31a0 0x7fa714146e30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa71359e700 1 --2- 172.21.15.36:0/332411970 >> v2:172.21.15.70:3300/0 conn(0x7fa714060920 0x7fa714146940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa71359e700 1 -- 172.21.15.36:0/332411970 >> 172.21.15.36:0/332411970 conn(0x7fa7140b64f0 msgr2=0x7fa7140b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:10.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa71359e700 1 -- 172.21.15.36:0/332411970 shutdown_connections 2022-06-17T12:54:10.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.496+0000 7fa71359e700 1 -- 172.21.15.36:0/332411970 wait complete. 2022-06-17T12:54:10.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa71359e700 1 Processor -- start 2022-06-17T12:54:10.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa71359e700 1 -- start start 2022-06-17T12:54:10.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa71359e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa714060920 0x7fa714156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:10.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa71359e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7140b2c10 0x7fa7140655b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:10.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa71359e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa7140b31a0 0x7fa714063760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:10.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa71359e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7141469e0 con 0x7fa7140b31a0 2022-06-17T12:54:10.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa71359e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7140649a0 con 0x7fa7140b2c10 2022-06-17T12:54:10.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa71359e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa714064bd0 con 0x7fa714060920 2022-06-17T12:54:10.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa712d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa7140b31a0 0x7fa714063760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:10.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa70bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7140b2c10 0x7fa7140655b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:10.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa712d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa7140b31a0 0x7fa714063760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36254/0 (socket says 172.21.15.36:36254) 2022-06-17T12:54:10.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa712d9d700 1 -- 172.21.15.36:0/342961271 learned_addr learned my addr 172.21.15.36:0/342961271 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:10.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa71259c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa714060920 0x7fa714156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:10.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa712d9d700 1 -- 172.21.15.36:0/342961271 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa714060920 msgr2=0x7fa714156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:10.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa712d9d700 1 --2- 172.21.15.36:0/342961271 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa714060920 0x7fa714156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa712d9d700 1 -- 172.21.15.36:0/342961271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7140b2c10 msgr2=0x7fa7140655b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:10.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa712d9d700 1 --2- 172.21.15.36:0/342961271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7140b2c10 0x7fa7140655b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa712d9d700 1 -- 172.21.15.36:0/342961271 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc00b180 con 0x7fa7140b31a0 2022-06-17T12:54:10.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa71259c700 1 --2- 172.21.15.36:0/342961271 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa714060920 0x7fa714156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:10.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa712d9d700 1 --2- 172.21.15.36:0/342961271 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa7140b31a0 0x7fa714063760 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7fa70c001490 tx=0x7fa70c007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:10.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa70b7fe700 1 -- 172.21.15.36:0/342961271 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa70c005ba0 con 0x7fa7140b31a0 2022-06-17T12:54:10.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa70b7fe700 1 -- 172.21.15.36:0/342961271 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa70c005d00 con 0x7fa7140b31a0 2022-06-17T12:54:10.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa71359e700 1 -- 172.21.15.36:0/342961271 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa714064e70 con 0x7fa7140b31a0 2022-06-17T12:54:10.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa70b7fe700 1 -- 172.21.15.36:0/342961271 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa70c0072c0 con 0x7fa7140b31a0 2022-06-17T12:54:10.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.500+0000 7fa71359e700 1 -- 172.21.15.36:0/342961271 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa714063c50 con 0x7fa7140b31a0 2022-06-17T12:54:10.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.504+0000 7fa71359e700 1 -- 172.21.15.36:0/342961271 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa714148650 con 0x7fa7140b31a0 2022-06-17T12:54:10.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.504+0000 7fa70b7fe700 1 -- 172.21.15.36:0/342961271 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fa70c010070 con 0x7fa7140b31a0 2022-06-17T12:54:10.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.504+0000 7fa70b7fe700 1 --2- 172.21.15.36:0/342961271 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa6f8053a60 0x7fa6f8055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:10.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.504+0000 7fa70b7fe700 1 -- 172.21.15.36:0/342961271 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fa70c06b650 con 0x7fa7140b31a0 2022-06-17T12:54:10.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.504+0000 7fa71259c700 1 --2- 172.21.15.36:0/342961271 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa6f8053a60 0x7fa6f8055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:10.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.504+0000 7fa71259c700 1 --2- 172.21.15.36:0/342961271 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa6f8053a60 0x7fa6f8055f20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7fa70400b000 tx=0x7fa704006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:10.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.512+0000 7fa70b7fe700 1 -- 172.21.15.36:0/342961271 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa70c042680 con 0x7fa7140b31a0 2022-06-17T12:54:10.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.636+0000 7fa71359e700 1 -- 172.21.15.36:0/342961271 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa714057410 con 0x7fa7140b31a0 2022-06-17T12:54:10.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.640+0000 7fa70b7fe700 1 -- 172.21.15.36:0/342961271 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v80) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa70c045ca0 con 0x7fa7140b31a0 2022-06-17T12:54:10.627 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:54:10.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.640+0000 7fa71359e700 1 -- 172.21.15.36:0/342961271 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa6f8053a60 msgr2=0x7fa6f8055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:10.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.640+0000 7fa71359e700 1 --2- 172.21.15.36:0/342961271 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa6f8053a60 0x7fa6f8055f20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7fa70400b000 tx=0x7fa704006df0).stop 2022-06-17T12:54:10.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.640+0000 7fa71359e700 1 -- 172.21.15.36:0/342961271 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa7140b31a0 msgr2=0x7fa714063760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:10.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.640+0000 7fa71359e700 1 --2- 172.21.15.36:0/342961271 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa7140b31a0 0x7fa714063760 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7fa70c001490 tx=0x7fa70c007e00).stop 2022-06-17T12:54:10.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.640+0000 7fa71359e700 1 -- 172.21.15.36:0/342961271 shutdown_connections 2022-06-17T12:54:10.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.640+0000 7fa71359e700 1 --2- 172.21.15.36:0/342961271 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa6f8053a60 0x7fa6f8055f20 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.640+0000 7fa71359e700 1 --2- 172.21.15.36:0/342961271 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa7140b31a0 0x7fa714063760 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.640+0000 7fa71359e700 1 --2- 172.21.15.36:0/342961271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7140b2c10 0x7fa7140655b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.640+0000 7fa71359e700 1 --2- 172.21.15.36:0/342961271 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa714060920 0x7fa714156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.640+0000 7fa71359e700 1 -- 172.21.15.36:0/342961271 >> 172.21.15.36:0/342961271 conn(0x7fa7140b64f0 msgr2=0x7fa714067360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:10.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.640+0000 7fa71359e700 1 -- 172.21.15.36:0/342961271 shutdown_connections 2022-06-17T12:54:10.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.640+0000 7fa71359e700 1 -- 172.21.15.36:0/342961271 wait complete. 2022-06-17T12:54:10.640 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-17T12:54:10.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d8b2d6700 1 Processor -- start 2022-06-17T12:54:10.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d8b2d6700 1 -- start start 2022-06-17T12:54:10.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d8b2d6700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f6d84144c00 0x7f6d84062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:10.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d8b2d6700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d840ad330 0x7f6d840ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:10.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d8b2d6700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d840b0ba0 con 0x7f6d84062770 2022-06-17T12:54:10.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d8b2d6700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d840ae820 con 0x7f6d84144c00 2022-06-17T12:54:10.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d8b2d6700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d84061030 con 0x7f6d840ad330 2022-06-17T12:54:10.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d88871700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f6d84144c00 0x7f6d84062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:10.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d88871700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f6d84144c00 0x7f6d84062b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36258/0 (socket says 172.21.15.36:36258) 2022-06-17T12:54:10.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d88871700 1 -- 172.21.15.36:0/2407169441 learned_addr learned my addr 172.21.15.36:0/2407169441 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:10.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d89873700 1 --2- 172.21.15.36:0/2407169441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d840ad330 0x7f6d840ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:10.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d88871700 1 -- 172.21.15.36:0/2407169441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d840ad330 msgr2=0x7f6d840ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:10.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d88871700 1 --2- 172.21.15.36:0/2407169441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d840ad330 0x7f6d840ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d88871700 1 -- 172.21.15.36:0/2407169441 >> v1:172.21.15.70:6789/0 conn(0x7f6d84062770 legacy=0x7f6d841444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:10.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d88871700 1 -- 172.21.15.36:0/2407169441 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d84147e90 con 0x7f6d84144c00 2022-06-17T12:54:10.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d89873700 1 --2- 172.21.15.36:0/2407169441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d840ad330 0x7f6d840ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:54:10.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d88871700 1 --2- 172.21.15.36:0/2407169441 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d84144c00 0x7f6d84062b50 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f6d8405d270 tx=0x7f6d80009290).ready entity=mon.0 client_cookie=d533db0e2a18e73a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:10.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d7bfff700 1 -- 172.21.15.36:0/2407169441 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d8000b820 con 0x7f6d84144c00 2022-06-17T12:54:10.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d7bfff700 1 -- 172.21.15.36:0/2407169441 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8000b980 con 0x7f6d84144c00 2022-06-17T12:54:10.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d7bfff700 1 -- 172.21.15.36:0/2407169441 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d8000bb30 con 0x7f6d84144c00 2022-06-17T12:54:10.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d8b2d6700 1 -- 172.21.15.36:0/2407169441 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d84144c00 msgr2=0x7f6d84062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:10.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d8b2d6700 1 --2- 172.21.15.36:0/2407169441 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d84144c00 0x7f6d84062b50 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f6d8405d270 tx=0x7f6d80009290).stop 2022-06-17T12:54:10.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d8b2d6700 1 -- 172.21.15.36:0/2407169441 shutdown_connections 2022-06-17T12:54:10.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d8b2d6700 1 --2- 172.21.15.36:0/2407169441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d840ad330 0x7f6d840ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d8b2d6700 1 --2- 172.21.15.36:0/2407169441 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d84144c00 0x7f6d84062b50 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d8b2d6700 1 -- 172.21.15.36:0/2407169441 >> 172.21.15.36:0/2407169441 conn(0x7f6d840b53e0 msgr2=0x7f6d840b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:10.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d8b2d6700 1 -- 172.21.15.36:0/2407169441 shutdown_connections 2022-06-17T12:54:10.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.760+0000 7f6d8b2d6700 1 -- 172.21.15.36:0/2407169441 wait complete. 2022-06-17T12:54:10.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.764+0000 7f6d8b2d6700 1 Processor -- start 2022-06-17T12:54:10.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.764+0000 7f6d8b2d6700 1 -- start start 2022-06-17T12:54:10.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.764+0000 7f6d8b2d6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d84062770 0x7f6d84146580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:10.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.764+0000 7f6d8b2d6700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d840ad330 0x7f6d84066a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:10.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.764+0000 7f6d8b2d6700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d84144c00 0x7f6d84066f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:10.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.764+0000 7f6d8b2d6700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d84064bd0 con 0x7f6d84144c00 2022-06-17T12:54:10.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.764+0000 7f6d8b2d6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d84064e70 con 0x7f6d84062770 2022-06-17T12:54:10.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.764+0000 7f6d8b2d6700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d84065110 con 0x7f6d840ad330 2022-06-17T12:54:10.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.764+0000 7f6d89873700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d84144c00 0x7f6d84066f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:10.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.764+0000 7f6d89873700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d84144c00 0x7f6d84066f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36268/0 (socket says 172.21.15.36:36268) 2022-06-17T12:54:10.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.764+0000 7f6d89873700 1 -- 172.21.15.36:0/974 learned_addr learned my addr 172.21.15.36:0/974 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:10.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.764+0000 7f6d88871700 1 --2- 172.21.15.36:0/974 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d840ad330 0x7f6d84066a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:10.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.764+0000 7f6d89072700 1 --2- 172.21.15.36:0/974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d84062770 0x7f6d84146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:10.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.764+0000 7f6d89873700 1 -- 172.21.15.36:0/974 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d840ad330 msgr2=0x7f6d84066a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:10.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.764+0000 7f6d89873700 1 --2- 172.21.15.36:0/974 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d840ad330 0x7f6d84066a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.764+0000 7f6d89873700 1 -- 172.21.15.36:0/974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d84062770 msgr2=0x7f6d84146580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:10.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.764+0000 7f6d89873700 1 --2- 172.21.15.36:0/974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d84062770 0x7f6d84146580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.764+0000 7f6d89873700 1 -- 172.21.15.36:0/974 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d80009530 con 0x7f6d84144c00 2022-06-17T12:54:10.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.764+0000 7f6d89873700 1 --2- 172.21.15.36:0/974 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d84144c00 0x7f6d84066f00 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7f6d7c007000 tx=0x7f6d7c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:10.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.764+0000 7f6d7a7fc700 1 -- 172.21.15.36:0/974 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d7c015070 con 0x7f6d84144c00 2022-06-17T12:54:10.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.764+0000 7f6d7a7fc700 1 -- 172.21.15.36:0/974 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7c00a160 con 0x7f6d84144c00 2022-06-17T12:54:10.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.764+0000 7f6d8b2d6700 1 -- 172.21.15.36:0/974 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8414b640 con 0x7f6d84144c00 2022-06-17T12:54:10.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.764+0000 7f6d7a7fc700 1 -- 172.21.15.36:0/974 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d7c010770 con 0x7f6d84144c00 2022-06-17T12:54:10.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.764+0000 7f6d8b2d6700 1 -- 172.21.15.36:0/974 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d840647a0 con 0x7f6d84144c00 2022-06-17T12:54:10.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.768+0000 7f6d8b2d6700 1 -- 172.21.15.36:0/974 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d840667b0 con 0x7f6d84144c00 2022-06-17T12:54:10.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.768+0000 7f6d7a7fc700 1 -- 172.21.15.36:0/974 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6d7c0108d0 con 0x7f6d84144c00 2022-06-17T12:54:10.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.768+0000 7f6d7a7fc700 1 --2- 172.21.15.36:0/974 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6d6c053a60 0x7f6d6c055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:10.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.768+0000 7f6d7a7fc700 1 -- 172.21.15.36:0/974 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f6d7c06b630 con 0x7f6d84144c00 2022-06-17T12:54:10.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.768+0000 7f6d89072700 1 --2- 172.21.15.36:0/974 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6d6c053a60 0x7f6d6c055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:10.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.768+0000 7f6d89072700 1 --2- 172.21.15.36:0/974 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6d6c053a60 0x7f6d6c055f20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f6d7400d000 tx=0x7f6d74007b70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:10.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.776+0000 7f6d7a7fc700 1 -- 172.21.15.36:0/974 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6d7c042770 con 0x7f6d84144c00 2022-06-17T12:54:10.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.904+0000 7f6d8b2d6700 1 -- 172.21.15.36:0/974 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6d840b1c20 con 0x7f6d84144c00 2022-06-17T12:54:10.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.904+0000 7f6d7a7fc700 1 -- 172.21.15.36:0/974 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+143 (secure 0 0 0) 0x7f6d7c045d90 con 0x7f6d84144c00 2022-06-17T12:54:10.895 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:10.896 INFO:teuthology.orchestra.run.smithi036.stdout:[ 2022-06-17T12:54:10.896 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:10.896 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 1, 2022-06-17T12:54:10.897 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "device_health_metrics" 2022-06-17T12:54:10.898 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:10.899 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:10.899 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 2, 2022-06-17T12:54:10.899 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "rbd" 2022-06-17T12:54:10.900 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:10.900 INFO:teuthology.orchestra.run.smithi036.stdout:] 2022-06-17T12:54:10.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.908+0000 7f6d8b2d6700 1 -- 172.21.15.36:0/974 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6d6c053a60 msgr2=0x7f6d6c055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:10.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.908+0000 7f6d8b2d6700 1 --2- 172.21.15.36:0/974 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6d6c053a60 0x7f6d6c055f20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f6d7400d000 tx=0x7f6d74007b70).stop 2022-06-17T12:54:10.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.908+0000 7f6d8b2d6700 1 -- 172.21.15.36:0/974 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d84144c00 msgr2=0x7f6d84066f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:10.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.908+0000 7f6d8b2d6700 1 --2- 172.21.15.36:0/974 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d84144c00 0x7f6d84066f00 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7f6d7c007000 tx=0x7f6d7c008040).stop 2022-06-17T12:54:10.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.908+0000 7f6d8b2d6700 1 -- 172.21.15.36:0/974 shutdown_connections 2022-06-17T12:54:10.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.908+0000 7f6d8b2d6700 1 --2- 172.21.15.36:0/974 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6d6c053a60 0x7f6d6c055f20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.908+0000 7f6d8b2d6700 1 --2- 172.21.15.36:0/974 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d84144c00 0x7f6d84066f00 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.908+0000 7f6d8b2d6700 1 --2- 172.21.15.36:0/974 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d840ad330 0x7f6d84066a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.908+0000 7f6d8b2d6700 1 --2- 172.21.15.36:0/974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d84062770 0x7f6d84146580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:10.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.908+0000 7f6d8b2d6700 1 -- 172.21.15.36:0/974 >> 172.21.15.36:0/974 conn(0x7f6d840b53e0 msgr2=0x7f6d8405f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:10.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.908+0000 7f6d8b2d6700 1 -- 172.21.15.36:0/974 shutdown_connections 2022-06-17T12:54:10.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:10.908+0000 7f6d8b2d6700 1 -- 172.21.15.36:0/974 wait complete. 2022-06-17T12:54:10.909 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-17T12:54:10.909 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-17T12:54:11.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.032+0000 7f7571fbc700 1 Processor -- start 2022-06-17T12:54:11.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.032+0000 7f7571fbc700 1 -- start start 2022-06-17T12:54:11.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.032+0000 7f7571fbc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f756c0b18e0 0x7f756c143860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:11.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.032+0000 7f7571fbc700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f756c0b39a0 0x7f756c146d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:11.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.032+0000 7f7571fbc700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f756c0b2b70 con 0x7f756c0b6860 2022-06-17T12:54:11.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.032+0000 7f7571fbc700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f756c0b6c40 con 0x7f756c0b39a0 2022-06-17T12:54:11.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.032+0000 7f7571fbc700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756c142d30 con 0x7f756c0b18e0 2022-06-17T12:54:11.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.032+0000 7f7570fba700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f756c0b6860 0x7f756c143150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53802/0 (socket says 172.21.15.36:53802) 2022-06-17T12:54:11.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.032+0000 7f7570fba700 1 -- 172.21.15.36:0/4292524391 learned_addr learned my addr 172.21.15.36:0/4292524391 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:11.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.032+0000 7f75717bb700 1 --2- 172.21.15.36:0/4292524391 >> v2:172.21.15.70:3300/0 conn(0x7f756c0b39a0 0x7f756c146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:11.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.032+0000 7f756bfff700 1 --2- 172.21.15.36:0/4292524391 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f756c0b18e0 0x7f756c143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:11.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.032+0000 7f756b7fe700 1 -- 172.21.15.36:0/4292524391 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 724123814 0 0) 0x7f756c0b2b70 con 0x7f756c0b6860 2022-06-17T12:54:11.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.032+0000 7f756b7fe700 1 -- 172.21.15.36:0/4292524391 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7550003260 con 0x7f756c0b6860 2022-06-17T12:54:11.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.036+0000 7f75717bb700 1 -- 172.21.15.36:0/4292524391 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f756c0b18e0 msgr2=0x7f756c143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:11.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.036+0000 7f75717bb700 1 --2- 172.21.15.36:0/4292524391 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f756c0b18e0 0x7f756c143860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:11.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.036+0000 7f75717bb700 1 -- 172.21.15.36:0/4292524391 >> v1:172.21.15.36:6789/0 conn(0x7f756c0b6860 legacy=0x7f756c143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:11.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.036+0000 7f75717bb700 1 -- 172.21.15.36:0/4292524391 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c0b0d30 con 0x7f756c0b39a0 2022-06-17T12:54:11.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.036+0000 7f75717bb700 1 --2- 172.21.15.36:0/4292524391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f756c0b39a0 0x7f756c146d40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f75640032f0 tx=0x7f75640091d0).ready entity=mon.1 client_cookie=6409261a1926005f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:11.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.036+0000 7f756b7fe700 1 -- 172.21.15.36:0/4292524391 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f756400b600 con 0x7f756c0b39a0 2022-06-17T12:54:11.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.036+0000 7f756b7fe700 1 -- 172.21.15.36:0/4292524391 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756400b760 con 0x7f756c0b39a0 2022-06-17T12:54:11.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.036+0000 7f756b7fe700 1 -- 172.21.15.36:0/4292524391 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f756400b8d0 con 0x7f756c0b39a0 2022-06-17T12:54:11.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.036+0000 7f7571fbc700 1 -- 172.21.15.36:0/4292524391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f756c0b39a0 msgr2=0x7f756c146d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:11.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.036+0000 7f7571fbc700 1 --2- 172.21.15.36:0/4292524391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f756c0b39a0 0x7f756c146d40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f75640032f0 tx=0x7f75640091d0).stop 2022-06-17T12:54:11.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.036+0000 7f7571fbc700 1 -- 172.21.15.36:0/4292524391 shutdown_connections 2022-06-17T12:54:11.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.036+0000 7f7571fbc700 1 --2- 172.21.15.36:0/4292524391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f756c0b39a0 0x7f756c146d40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:11.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.036+0000 7f7571fbc700 1 --2- 172.21.15.36:0/4292524391 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f756c0b18e0 0x7f756c143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:11.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.036+0000 7f7571fbc700 1 -- 172.21.15.36:0/4292524391 >> 172.21.15.36:0/4292524391 conn(0x7f756c0614b0 msgr2=0x7f756c061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:11.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.036+0000 7f7571fbc700 1 -- 172.21.15.36:0/4292524391 shutdown_connections 2022-06-17T12:54:11.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.036+0000 7f7571fbc700 1 -- 172.21.15.36:0/4292524391 wait complete. 2022-06-17T12:54:11.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.036+0000 7f7571fbc700 1 Processor -- start 2022-06-17T12:54:11.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.036+0000 7f7571fbc700 1 -- start start 2022-06-17T12:54:11.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.040+0000 7f7571fbc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f756c0b18e0 0x7f756c0653a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:11.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.040+0000 7f7571fbc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f756c0b39a0 0x7f756c155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:11.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.040+0000 7f7571fbc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f756c0b6860 0x7f756c0659d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:11.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.040+0000 7f7571fbc700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c066be0 con 0x7f756c0b6860 2022-06-17T12:54:11.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.040+0000 7f7571fbc700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c066e50 con 0x7f756c0b39a0 2022-06-17T12:54:11.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.040+0000 7f7571fbc700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756c0670f0 con 0x7f756c0b18e0 2022-06-17T12:54:11.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.040+0000 7f75717bb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f756c0b6860 0x7f756c0659d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:11.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.040+0000 7f756bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f756c0b39a0 0x7f756c155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:11.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.040+0000 7f75717bb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f756c0b6860 0x7f756c0659d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36282/0 (socket says 172.21.15.36:36282) 2022-06-17T12:54:11.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.040+0000 7f7570fba700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f756c0b18e0 0x7f756c0653a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:11.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.040+0000 7f75717bb700 1 -- 172.21.15.36:0/2582041003 learned_addr learned my addr 172.21.15.36:0/2582041003 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:11.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.040+0000 7f756bfff700 1 -- 172.21.15.36:0/2582041003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f756c0b18e0 msgr2=0x7f756c0653a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:11.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.040+0000 7f756bfff700 1 --2- 172.21.15.36:0/2582041003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f756c0b18e0 0x7f756c0653a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:11.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.040+0000 7f756bfff700 1 -- 172.21.15.36:0/2582041003 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f756c0b6860 msgr2=0x7f756c0659d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:11.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.040+0000 7f756bfff700 1 --2- 172.21.15.36:0/2582041003 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f756c0b6860 0x7f756c0659d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:11.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.040+0000 7f756bfff700 1 -- 172.21.15.36:0/2582041003 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7564009470 con 0x7f756c0b39a0 2022-06-17T12:54:11.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.040+0000 7f756bfff700 1 --2- 172.21.15.36:0/2582041003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f756c0b39a0 0x7f756c155b70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f755800c000 tx=0x7f7558006d90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:11.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.040+0000 7f7569ffb700 1 -- 172.21.15.36:0/2582041003 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75580104c0 con 0x7f756c0b39a0 2022-06-17T12:54:11.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.040+0000 7f7569ffb700 1 -- 172.21.15.36:0/2582041003 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7558010620 con 0x7f756c0b39a0 2022-06-17T12:54:11.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.040+0000 7f7571fbc700 1 -- 172.21.15.36:0/2582041003 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f756c156320 con 0x7f756c0b39a0 2022-06-17T12:54:11.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.040+0000 7f7569ffb700 1 -- 172.21.15.36:0/2582041003 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7558010980 con 0x7f756c0b39a0 2022-06-17T12:54:11.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.040+0000 7f7571fbc700 1 -- 172.21.15.36:0/2582041003 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756c0633f0 con 0x7f756c0b39a0 2022-06-17T12:54:11.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.040+0000 7f7571fbc700 1 -- 172.21.15.36:0/2582041003 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f756c156060 con 0x7f756c0b39a0 2022-06-17T12:54:11.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.044+0000 7f7569ffb700 1 -- 172.21.15.36:0/2582041003 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f7558010ae0 con 0x7f756c0b39a0 2022-06-17T12:54:11.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.044+0000 7f7569ffb700 1 --2- 172.21.15.36:0/2582041003 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f754c0539b0 0x7f754c055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:11.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.044+0000 7f7569ffb700 1 -- 172.21.15.36:0/2582041003 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f755806b6d0 con 0x7f756c0b39a0 2022-06-17T12:54:11.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.044+0000 7f7570fba700 1 --2- 172.21.15.36:0/2582041003 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f754c0539b0 0x7f754c055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:11.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.044+0000 7f7570fba700 1 --2- 172.21.15.36:0/2582041003 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f754c0539b0 0x7f754c055e70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f756000d000 tx=0x7f7560009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:11.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.052+0000 7f7569ffb700 1 -- 172.21.15.36:0/2582041003 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f756c156060 con 0x7f756c0b39a0 2022-06-17T12:54:11.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.176+0000 7f7571fbc700 1 -- 172.21.15.36:0/2582041003 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f756c156060 con 0x7f756c0b39a0 2022-06-17T12:54:11.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.636+0000 7f7569ffb700 1 -- 172.21.15.36:0/2582041003 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v81) v1 ==== 133+0+0 (secure 0 0 0) 0x7f756c156060 con 0x7f756c0b39a0 2022-06-17T12:54:11.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.644+0000 7f7571fbc700 1 -- 172.21.15.36:0/2582041003 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f754c0539b0 msgr2=0x7f754c055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:11.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.644+0000 7f7571fbc700 1 --2- 172.21.15.36:0/2582041003 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f754c0539b0 0x7f754c055e70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f756000d000 tx=0x7f7560009040).stop 2022-06-17T12:54:11.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.644+0000 7f7571fbc700 1 -- 172.21.15.36:0/2582041003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f756c0b39a0 msgr2=0x7f756c155b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:11.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.644+0000 7f7571fbc700 1 --2- 172.21.15.36:0/2582041003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f756c0b39a0 0x7f756c155b70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f755800c000 tx=0x7f7558006d90).stop 2022-06-17T12:54:11.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.644+0000 7f7571fbc700 1 -- 172.21.15.36:0/2582041003 shutdown_connections 2022-06-17T12:54:11.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.644+0000 7f7571fbc700 1 --2- 172.21.15.36:0/2582041003 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f754c0539b0 0x7f754c055e70 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:11.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.644+0000 7f7571fbc700 1 --2- 172.21.15.36:0/2582041003 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f756c0b6860 0x7f756c0659d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:11.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.644+0000 7f7571fbc700 1 --2- 172.21.15.36:0/2582041003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f756c0b39a0 0x7f756c155b70 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:11.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.644+0000 7f7571fbc700 1 --2- 172.21.15.36:0/2582041003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f756c0b18e0 0x7f756c0653a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:11.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.644+0000 7f7571fbc700 1 -- 172.21.15.36:0/2582041003 >> 172.21.15.36:0/2582041003 conn(0x7f756c0614b0 msgr2=0x7f756c13ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:11.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.644+0000 7f7571fbc700 1 -- 172.21.15.36:0/2582041003 shutdown_connections 2022-06-17T12:54:11.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.644+0000 7f7571fbc700 1 -- 172.21.15.36:0/2582041003 wait complete. 2022-06-17T12:54:11.637 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' created 2022-06-17T12:54:11.644 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-06-17T12:54:11.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.772+0000 7fc1d9780700 1 Processor -- start 2022-06-17T12:54:11.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.772+0000 7fc1d9780700 1 -- start start 2022-06-17T12:54:11.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.776+0000 7fc1d9780700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc1d40ae360 0x7fc1d4144ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:11.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.776+0000 7fc1d9780700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d40b2b60 con 0x7fc1d40601c0 2022-06-17T12:54:11.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.776+0000 7fc1d9780700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d40b3230 con 0x7fc1d4144fd0 2022-06-17T12:54:11.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.776+0000 7fc1d9780700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1d40605a0 con 0x7fc1d40ae360 2022-06-17T12:54:11.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.776+0000 7fc1d3fff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fc1d40601c0 0x7fc1d41443d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53816/0 (socket says 172.21.15.36:53816) 2022-06-17T12:54:11.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.776+0000 7fc1d3fff700 1 -- 172.21.15.36:0/3272602698 learned_addr learned my addr 172.21.15.36:0/3272602698 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:11.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.776+0000 7fc1cb7fe700 1 --2- 172.21.15.36:0/3272602698 >> v2:172.21.15.70:3300/0 conn(0x7fc1d40ae360 0x7fc1d4144ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:11.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.776+0000 7fc1d37fe700 1 -- 172.21.15.36:0/3272602698 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2581543083 0 0) 0x7fc1d40b2b60 con 0x7fc1d40601c0 2022-06-17T12:54:11.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.776+0000 7fc1d37fe700 1 -- 172.21.15.36:0/3272602698 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1b4003260 con 0x7fc1d40601c0 2022-06-17T12:54:11.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.776+0000 7fc1cb7fe700 1 -- 172.21.15.36:0/3272602698 >> v1:172.21.15.70:6789/0 conn(0x7fc1d4144fd0 legacy=0x7fc1d414aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:11.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.776+0000 7fc1d37fe700 1 -- 172.21.15.36:0/3272602698 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2330931659 0 0) 0x7fc1cc001490 con 0x7fc1d4144fd0 2022-06-17T12:54:11.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.776+0000 7fc1cb7fe700 1 -- 172.21.15.36:0/3272602698 >> v1:172.21.15.36:6789/0 conn(0x7fc1d40601c0 legacy=0x7fc1d41443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:11.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.776+0000 7fc1cb7fe700 1 -- 172.21.15.36:0/3272602698 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d40662f0 con 0x7fc1d40ae360 2022-06-17T12:54:11.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.776+0000 7fc1cb7fe700 1 --2- 172.21.15.36:0/3272602698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1d40ae360 0x7fc1d4144ae0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fc1bc002f70 tx=0x7fc1bc006df0).ready entity=mon.1 client_cookie=514bcdfbca52d8d5 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:11.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.776+0000 7fc1d37fe700 1 -- 172.21.15.36:0/3272602698 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1bc00d610 con 0x7fc1d40ae360 2022-06-17T12:54:11.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.776+0000 7fc1d37fe700 1 -- 172.21.15.36:0/3272602698 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1bc000e90 con 0x7fc1d40ae360 2022-06-17T12:54:11.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.776+0000 7fc1d37fe700 1 -- 172.21.15.36:0/3272602698 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1bc00d960 con 0x7fc1d40ae360 2022-06-17T12:54:11.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.776+0000 7fc1d9780700 1 -- 172.21.15.36:0/3272602698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1d40ae360 msgr2=0x7fc1d4144ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:11.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.776+0000 7fc1d9780700 1 --2- 172.21.15.36:0/3272602698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1d40ae360 0x7fc1d4144ae0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fc1bc002f70 tx=0x7fc1bc006df0).stop 2022-06-17T12:54:11.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.776+0000 7fc1d9780700 1 -- 172.21.15.36:0/3272602698 shutdown_connections 2022-06-17T12:54:11.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.776+0000 7fc1d9780700 1 --2- 172.21.15.36:0/3272602698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1d40ae360 0x7fc1d4144ae0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:11.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.776+0000 7fc1d9780700 1 -- 172.21.15.36:0/3272602698 >> 172.21.15.36:0/3272602698 conn(0x7fc1d4060990 msgr2=0x7fc1d405f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:11.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.776+0000 7fc1d9780700 1 -- 172.21.15.36:0/3272602698 shutdown_connections 2022-06-17T12:54:11.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.776+0000 7fc1d9780700 1 -- 172.21.15.36:0/3272602698 wait complete. 2022-06-17T12:54:11.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.780+0000 7fc1d9780700 1 Processor -- start 2022-06-17T12:54:11.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.780+0000 7fc1d9780700 1 -- start start 2022-06-17T12:54:11.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.780+0000 7fc1d9780700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc1d40601c0 0x7fc1d40b5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:11.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.780+0000 7fc1d9780700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1d40ae360 0x7fc1d4155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:11.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.780+0000 7fc1d9780700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc1d4144fd0 0x7fc1d41562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:11.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.780+0000 7fc1d9780700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d41567b0 con 0x7fc1d40601c0 2022-06-17T12:54:11.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.780+0000 7fc1d9780700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d4114a10 con 0x7fc1d40ae360 2022-06-17T12:54:11.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.780+0000 7fc1d9780700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d4114cb0 con 0x7fc1d4144fd0 2022-06-17T12:54:11.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.780+0000 7fc1d3fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc1d40601c0 0x7fc1d40b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:11.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.780+0000 7fc1d3fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc1d40601c0 0x7fc1d40b5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36292/0 (socket says 172.21.15.36:36292) 2022-06-17T12:54:11.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.780+0000 7fc1d3fff700 1 -- 172.21.15.36:0/3135649348 learned_addr learned my addr 172.21.15.36:0/3135649348 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:11.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.780+0000 7fc1cb7fe700 1 --2- 172.21.15.36:0/3135649348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1d40ae360 0x7fc1d4155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:11.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.780+0000 7fc1d8f7f700 1 --2- 172.21.15.36:0/3135649348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc1d4144fd0 0x7fc1d41562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:11.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.780+0000 7fc1d3fff700 1 -- 172.21.15.36:0/3135649348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc1d4144fd0 msgr2=0x7fc1d41562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:11.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.780+0000 7fc1d3fff700 1 --2- 172.21.15.36:0/3135649348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc1d4144fd0 0x7fc1d41562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:11.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.780+0000 7fc1d3fff700 1 -- 172.21.15.36:0/3135649348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1d40ae360 msgr2=0x7fc1d4155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:11.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.780+0000 7fc1d3fff700 1 --2- 172.21.15.36:0/3135649348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1d40ae360 0x7fc1d4155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:11.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.780+0000 7fc1d3fff700 1 -- 172.21.15.36:0/3135649348 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc00b180 con 0x7fc1d40601c0 2022-06-17T12:54:11.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.784+0000 7fc1d3fff700 1 --2- 172.21.15.36:0/3135649348 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc1d40601c0 0x7fc1d40b5740 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7fc1c4001110 tx=0x7fc1c4007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:11.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.784+0000 7fc1d1ffb700 1 -- 172.21.15.36:0/3135649348 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1c40070d0 con 0x7fc1d40601c0 2022-06-17T12:54:11.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.784+0000 7fc1d1ffb700 1 -- 172.21.15.36:0/3135649348 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c4007230 con 0x7fc1d40601c0 2022-06-17T12:54:11.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.784+0000 7fc1d9780700 1 -- 172.21.15.36:0/3135649348 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d4114f50 con 0x7fc1d40601c0 2022-06-17T12:54:11.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.784+0000 7fc1d1ffb700 1 -- 172.21.15.36:0/3135649348 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1c4005610 con 0x7fc1d40601c0 2022-06-17T12:54:11.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.784+0000 7fc1d9780700 1 -- 172.21.15.36:0/3135649348 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d40b5c30 con 0x7fc1d40601c0 2022-06-17T12:54:11.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.784+0000 7fc1d9780700 1 -- 172.21.15.36:0/3135649348 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1d4066590 con 0x7fc1d40601c0 2022-06-17T12:54:11.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.784+0000 7fc1d1ffb700 1 -- 172.21.15.36:0/3135649348 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fc1c4010070 con 0x7fc1d40601c0 2022-06-17T12:54:11.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.784+0000 7fc1d1ffb700 1 --2- 172.21.15.36:0/3135649348 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc1b8053940 0x7fc1b8055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:11.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.784+0000 7fc1cb7fe700 1 --2- 172.21.15.36:0/3135649348 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc1b8053940 0x7fc1b8055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:11.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.784+0000 7fc1d1ffb700 1 -- 172.21.15.36:0/3135649348 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5954+0+0 (secure 0 0 0) 0x7fc1c406b420 con 0x7fc1d40601c0 2022-06-17T12:54:11.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.788+0000 7fc1cb7fe700 1 --2- 172.21.15.36:0/3135649348 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc1b8053940 0x7fc1b8055e00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fc1bc018000 tx=0x7fc1bc014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:11.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.792+0000 7fc1d1ffb700 1 -- 172.21.15.36:0/3135649348 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc1c40423f0 con 0x7fc1d40601c0 2022-06-17T12:54:11.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:11.924+0000 7fc1d9780700 1 -- 172.21.15.36:0/3135649348 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fc1d4057410 con 0x7fc1d40601c0 2022-06-17T12:54:12.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.412+0000 7fc1d1ffb700 1 -- 172.21.15.36:0/3135649348 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v82) v1 ==== 125+0+0 (secure 0 0 0) 0x7fc1c4045a10 con 0x7fc1d40601c0 2022-06-17T12:54:12.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.424+0000 7fc1d9780700 1 -- 172.21.15.36:0/3135649348 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc1b8053940 msgr2=0x7fc1b8055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:12.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.424+0000 7fc1d9780700 1 --2- 172.21.15.36:0/3135649348 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc1b8053940 0x7fc1b8055e00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fc1bc018000 tx=0x7fc1bc014000).stop 2022-06-17T12:54:12.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.424+0000 7fc1d9780700 1 -- 172.21.15.36:0/3135649348 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc1d40601c0 msgr2=0x7fc1d40b5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:12.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.424+0000 7fc1d9780700 1 --2- 172.21.15.36:0/3135649348 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc1d40601c0 0x7fc1d40b5740 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7fc1c4001110 tx=0x7fc1c4007aa0).stop 2022-06-17T12:54:12.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.424+0000 7fc1d9780700 1 -- 172.21.15.36:0/3135649348 shutdown_connections 2022-06-17T12:54:12.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.424+0000 7fc1d9780700 1 --2- 172.21.15.36:0/3135649348 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc1b8053940 0x7fc1b8055e00 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:12.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.424+0000 7fc1d9780700 1 --2- 172.21.15.36:0/3135649348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc1d4144fd0 0x7fc1d41562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:12.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.424+0000 7fc1d9780700 1 --2- 172.21.15.36:0/3135649348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1d40ae360 0x7fc1d4155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:12.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.424+0000 7fc1d9780700 1 --2- 172.21.15.36:0/3135649348 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc1d40601c0 0x7fc1d40b5740 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:12.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.424+0000 7fc1d9780700 1 -- 172.21.15.36:0/3135649348 >> 172.21.15.36:0/3135649348 conn(0x7fc1d4060990 msgr2=0x7fc1d413f410 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:12.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.424+0000 7fc1d9780700 1 -- 172.21.15.36:0/3135649348 shutdown_connections 2022-06-17T12:54:12.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.424+0000 7fc1d9780700 1 -- 172.21.15.36:0/3135649348 wait complete. 2022-06-17T12:54:12.417 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' created 2022-06-17T12:54:12.428 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-17T12:54:12.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.552+0000 7f19c1d96700 1 Processor -- start 2022-06-17T12:54:12.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.552+0000 7f19c1d96700 1 -- start start 2022-06-17T12:54:12.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.552+0000 7f19c1d96700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19bc142f20 0x7f19bc143e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:12.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.552+0000 7f19c1d96700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19bc0b44b0 con 0x7f19bc0b0cb0 2022-06-17T12:54:12.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.552+0000 7f19c1d96700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19bc062650 con 0x7f19bc062270 2022-06-17T12:54:12.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.552+0000 7f19c1d96700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc0b6650 con 0x7f19bc142f20 2022-06-17T12:54:12.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.552+0000 7f19baffd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f19bc0b0cb0 0x7f19bc146970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53832/0 (socket says 172.21.15.36:53832) 2022-06-17T12:54:12.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.552+0000 7f19baffd700 1 -- 172.21.15.36:0/2022653739 learned_addr learned my addr 172.21.15.36:0/2022653739 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:12.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.552+0000 7f19bbfff700 1 --2- 172.21.15.36:0/2022653739 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19bc142f20 0x7f19bc143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:12.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.552+0000 7f19ba7fc700 1 -- 172.21.15.36:0/2022653739 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638956280 0 0) 0x7f19bc0b44b0 con 0x7f19bc0b0cb0 2022-06-17T12:54:12.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.552+0000 7f19ba7fc700 1 -- 172.21.15.36:0/2022653739 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19a8003260 con 0x7f19bc0b0cb0 2022-06-17T12:54:12.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.552+0000 7f19ba7fc700 1 -- 172.21.15.36:0/2022653739 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536376206 0 0) 0x7f19bc062650 con 0x7f19bc062270 2022-06-17T12:54:12.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.552+0000 7f19ba7fc700 1 -- 172.21.15.36:0/2022653739 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19bc0b44b0 con 0x7f19bc062270 2022-06-17T12:54:12.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.552+0000 7f19bbfff700 1 -- 172.21.15.36:0/2022653739 >> v1:172.21.15.70:6789/0 conn(0x7f19bc062270 legacy=0x7f19bc143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:12.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.552+0000 7f19bbfff700 1 -- 172.21.15.36:0/2022653739 >> v1:172.21.15.36:6789/0 conn(0x7f19bc0b0cb0 legacy=0x7f19bc146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:12.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.552+0000 7f19ba7fc700 1 -- 172.21.15.36:0/2022653739 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3827076511 0 0) 0x7f19a8003260 con 0x7f19bc0b0cb0 2022-06-17T12:54:12.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.552+0000 7f19bbfff700 1 -- 172.21.15.36:0/2022653739 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19bc0b4210 con 0x7f19bc142f20 2022-06-17T12:54:12.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.552+0000 7f19bbfff700 1 --2- 172.21.15.36:0/2022653739 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19bc142f20 0x7f19bc143e80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f19b40032f0 tx=0x7f19b40091d0).ready entity=mon.2 client_cookie=81850ea086f4ac4b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:12.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.552+0000 7f19ba7fc700 1 -- 172.21.15.36:0/2022653739 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19b400b640 con 0x7f19bc142f20 2022-06-17T12:54:12.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.552+0000 7f19ba7fc700 1 -- 172.21.15.36:0/2022653739 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b400b7a0 con 0x7f19bc142f20 2022-06-17T12:54:12.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.552+0000 7f19ba7fc700 1 -- 172.21.15.36:0/2022653739 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19b400b910 con 0x7f19bc142f20 2022-06-17T12:54:12.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.556+0000 7f19c1d96700 1 -- 172.21.15.36:0/2022653739 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19bc142f20 msgr2=0x7f19bc143e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:12.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.556+0000 7f19c1d96700 1 --2- 172.21.15.36:0/2022653739 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19bc142f20 0x7f19bc143e80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f19b40032f0 tx=0x7f19b40091d0).stop 2022-06-17T12:54:12.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.556+0000 7f19c1d96700 1 -- 172.21.15.36:0/2022653739 shutdown_connections 2022-06-17T12:54:12.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.556+0000 7f19c1d96700 1 --2- 172.21.15.36:0/2022653739 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19bc142f20 0x7f19bc143e80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:12.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.556+0000 7f19c1d96700 1 -- 172.21.15.36:0/2022653739 >> 172.21.15.36:0/2022653739 conn(0x7f19bc0b9870 msgr2=0x7f19bc0b9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:12.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.556+0000 7f19c1d96700 1 -- 172.21.15.36:0/2022653739 shutdown_connections 2022-06-17T12:54:12.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.556+0000 7f19c1d96700 1 -- 172.21.15.36:0/2022653739 wait complete. 2022-06-17T12:54:12.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.556+0000 7f19c1d96700 1 Processor -- start 2022-06-17T12:54:12.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.556+0000 7f19c1d96700 1 -- start start 2022-06-17T12:54:12.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.560+0000 7f19c1d96700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19bc062270 0x7f19bc144870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:12.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.560+0000 7f19c1d96700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19bc142f20 0x7f19bc1528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:12.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.560+0000 7f19c1d96700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19bc152da0 0x7f19bc155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:12.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.560+0000 7f19c1d96700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc1468e0 con 0x7f19bc062270 2022-06-17T12:54:12.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.560+0000 7f19c1d96700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc0af440 con 0x7f19bc152da0 2022-06-17T12:54:12.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.560+0000 7f19c1d96700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc152220 con 0x7f19bc142f20 2022-06-17T12:54:12.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.560+0000 7f19bb7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19bc062270 0x7f19bc144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:12.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.560+0000 7f19bb7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19bc062270 0x7f19bc144870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36306/0 (socket says 172.21.15.36:36306) 2022-06-17T12:54:12.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.560+0000 7f19bb7fe700 1 -- 172.21.15.36:0/1888212973 learned_addr learned my addr 172.21.15.36:0/1888212973 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:12.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.560+0000 7f19bbfff700 1 --2- 172.21.15.36:0/1888212973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19bc152da0 0x7f19bc155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:12.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.560+0000 7f19baffd700 1 --2- 172.21.15.36:0/1888212973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19bc142f20 0x7f19bc1528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:12.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.560+0000 7f19bb7fe700 1 -- 172.21.15.36:0/1888212973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19bc142f20 msgr2=0x7f19bc1528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:12.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.560+0000 7f19bb7fe700 1 --2- 172.21.15.36:0/1888212973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19bc142f20 0x7f19bc1528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:12.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.560+0000 7f19bb7fe700 1 -- 172.21.15.36:0/1888212973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19bc152da0 msgr2=0x7f19bc155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:12.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.560+0000 7f19bb7fe700 1 --2- 172.21.15.36:0/1888212973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19bc152da0 0x7f19bc155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:12.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.560+0000 7f19bb7fe700 1 -- 172.21.15.36:0/1888212973 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b4009470 con 0x7f19bc062270 2022-06-17T12:54:12.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.560+0000 7f19baffd700 1 --2- 172.21.15.36:0/1888212973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19bc142f20 0x7f19bc1528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:12.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.560+0000 7f19bb7fe700 1 --2- 172.21.15.36:0/1888212973 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19bc062270 0x7f19bc144870 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f19ac001110 tx=0x7f19ac007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:12.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.560+0000 7f19b8ff9700 1 -- 172.21.15.36:0/1888212973 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19ac00ce50 con 0x7f19bc062270 2022-06-17T12:54:12.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.560+0000 7f19b8ff9700 1 -- 172.21.15.36:0/1888212973 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ac00f470 con 0x7f19bc062270 2022-06-17T12:54:12.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.560+0000 7f19c1d96700 1 -- 172.21.15.36:0/1888212973 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19bc152450 con 0x7f19bc062270 2022-06-17T12:54:12.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.560+0000 7f19b8ff9700 1 -- 172.21.15.36:0/1888212973 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19ac00f7d0 con 0x7f19bc062270 2022-06-17T12:54:12.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.560+0000 7f19c1d96700 1 -- 172.21.15.36:0/1888212973 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19bc0af1a0 con 0x7f19bc062270 2022-06-17T12:54:12.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.564+0000 7f19c1d96700 1 -- 172.21.15.36:0/1888212973 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19bc0b63b0 con 0x7f19bc062270 2022-06-17T12:54:12.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.564+0000 7f19b8ff9700 1 -- 172.21.15.36:0/1888212973 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f19ac019030 con 0x7f19bc062270 2022-06-17T12:54:12.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.564+0000 7f19b8ff9700 1 --2- 172.21.15.36:0/1888212973 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f199c053920 0x7f199c055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:12.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.564+0000 7f19b8ff9700 1 -- 172.21.15.36:0/1888212973 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f19ac06b390 con 0x7f19bc062270 2022-06-17T12:54:12.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.564+0000 7f19baffd700 1 --2- 172.21.15.36:0/1888212973 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f199c053920 0x7f199c055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:12.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.564+0000 7f19baffd700 1 --2- 172.21.15.36:0/1888212973 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f199c053920 0x7f199c055de0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f19b000b000 tx=0x7f19b0001660).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:12.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.568+0000 7f19b8ff9700 1 -- 172.21.15.36:0/1888212973 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f19ac042210 con 0x7f19bc062270 2022-06-17T12:54:12.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.736+0000 7f19c1d96700 1 -- 172.21.15.36:0/1888212973 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f19bc057410 con 0x7f19bc062270 2022-06-17T12:54:12.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.744+0000 7f19b8ff9700 1 -- 172.21.15.36:0/1888212973 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 15 and data pool 16 v41) v1 ==== 189+0+0 (secure 0 0 0) 0x7f19ac045830 con 0x7f19bc062270 2022-06-17T12:54:12.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.748+0000 7f19c1d96700 1 -- 172.21.15.36:0/1888212973 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f199c053920 msgr2=0x7f199c055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:12.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.748+0000 7f19c1d96700 1 --2- 172.21.15.36:0/1888212973 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f199c053920 0x7f199c055de0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f19b000b000 tx=0x7f19b0001660).stop 2022-06-17T12:54:12.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.752+0000 7f19c1d96700 1 -- 172.21.15.36:0/1888212973 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19bc062270 msgr2=0x7f19bc144870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:12.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.752+0000 7f19c1d96700 1 --2- 172.21.15.36:0/1888212973 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19bc062270 0x7f19bc144870 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f19ac001110 tx=0x7f19ac007ae0).stop 2022-06-17T12:54:12.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.752+0000 7f19c1d96700 1 -- 172.21.15.36:0/1888212973 shutdown_connections 2022-06-17T12:54:12.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.752+0000 7f19c1d96700 1 --2- 172.21.15.36:0/1888212973 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f199c053920 0x7f199c055de0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:12.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.752+0000 7f19c1d96700 1 --2- 172.21.15.36:0/1888212973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19bc152da0 0x7f19bc155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:12.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.752+0000 7f19c1d96700 1 --2- 172.21.15.36:0/1888212973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19bc142f20 0x7f19bc1528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:12.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.752+0000 7f19c1d96700 1 --2- 172.21.15.36:0/1888212973 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19bc062270 0x7f19bc144870 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:12.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.752+0000 7f19c1d96700 1 -- 172.21.15.36:0/1888212973 >> 172.21.15.36:0/1888212973 conn(0x7f19bc0b9870 msgr2=0x7f19bc0b8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:12.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.752+0000 7f19c1d96700 1 -- 172.21.15.36:0/1888212973 shutdown_connections 2022-06-17T12:54:12.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.752+0000 7f19c1d96700 1 -- 172.21.15.36:0/1888212973 wait complete. 2022-06-17T12:54:12.742 INFO:teuthology.orchestra.run.smithi036.stderr:new fs with metadata pool 15 and data pool 16 2022-06-17T12:54:12.756 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:54:12.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.860+0000 7f00bd3be700 1 Processor -- start 2022-06-17T12:54:12.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.860+0000 7f00bd3be700 1 -- start start 2022-06-17T12:54:12.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.860+0000 7f00bd3be700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00b813cb70 0x7f00b8143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:12.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.860+0000 7f00bd3be700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f00b813cf50 0x7f00b81442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:12.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.860+0000 7f00bd3be700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b80af4b0 con 0x7f00b80aef60 2022-06-17T12:54:12.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.860+0000 7f00bd3be700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00b80ba500 con 0x7f00b813cf50 2022-06-17T12:54:12.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.860+0000 7f00bd3be700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b8142f50 con 0x7f00b813cb70 2022-06-17T12:54:12.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.860+0000 7f00b7fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00b813cb70 0x7f00b8143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:12.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.860+0000 7f00b7fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00b813cb70 0x7f00b8143690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:51114/0 (socket says 172.21.15.36:51114) 2022-06-17T12:54:12.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.860+0000 7f00b7fff700 1 -- 172.21.15.36:0/2446714657 learned_addr learned my addr 172.21.15.36:0/2446714657 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:12.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.860+0000 7f00bcbbd700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f00b813cf50 0x7f00b81442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:12.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.860+0000 7f00b7fff700 1 -- 172.21.15.36:0/2446714657 >> v2:172.21.15.70:3300/0 conn(0x7f00b813cf50 msgr2=0x7f00b81442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:12.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.860+0000 7f00b7fff700 1 --2- 172.21.15.36:0/2446714657 >> v2:172.21.15.70:3300/0 conn(0x7f00b813cf50 0x7f00b81442a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:12.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.860+0000 7f00b7fff700 1 -- 172.21.15.36:0/2446714657 >> v1:172.21.15.70:6789/0 conn(0x7f00b80aef60 legacy=0x7f00b80b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:12.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.860+0000 7f00b7fff700 1 -- 172.21.15.36:0/2446714657 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b81431f0 con 0x7f00b813cb70 2022-06-17T12:54:12.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.860+0000 7f00b7fff700 1 --2- 172.21.15.36:0/2446714657 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00b813cb70 0x7f00b8143690 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f00a80032f0 tx=0x7f00a8009290).ready entity=mon.2 client_cookie=c1d96390c6cd1099 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:12.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.860+0000 7f00b6ffd700 1 -- 172.21.15.36:0/2446714657 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00a800b8a0 con 0x7f00b813cb70 2022-06-17T12:54:12.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.860+0000 7f00b6ffd700 1 -- 172.21.15.36:0/2446714657 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a800ba00 con 0x7f00b813cb70 2022-06-17T12:54:12.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.860+0000 7f00b6ffd700 1 -- 172.21.15.36:0/2446714657 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00a800bb70 con 0x7f00b813cb70 2022-06-17T12:54:12.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.864+0000 7f00bd3be700 1 -- 172.21.15.36:0/2446714657 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00b813cb70 msgr2=0x7f00b8143690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:12.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.864+0000 7f00bd3be700 1 --2- 172.21.15.36:0/2446714657 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00b813cb70 0x7f00b8143690 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f00a80032f0 tx=0x7f00a8009290).stop 2022-06-17T12:54:12.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.864+0000 7f00bd3be700 1 -- 172.21.15.36:0/2446714657 shutdown_connections 2022-06-17T12:54:12.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.864+0000 7f00bd3be700 1 --2- 172.21.15.36:0/2446714657 >> v2:172.21.15.70:3300/0 conn(0x7f00b813cf50 0x7f00b81442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:12.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.864+0000 7f00bd3be700 1 --2- 172.21.15.36:0/2446714657 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00b813cb70 0x7f00b8143690 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:12.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.864+0000 7f00bd3be700 1 -- 172.21.15.36:0/2446714657 >> 172.21.15.36:0/2446714657 conn(0x7f00b805c550 msgr2=0x7f00b80b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:12.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.864+0000 7f00bd3be700 1 -- 172.21.15.36:0/2446714657 shutdown_connections 2022-06-17T12:54:12.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.864+0000 7f00bd3be700 1 -- 172.21.15.36:0/2446714657 wait complete. 2022-06-17T12:54:12.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.864+0000 7f00bd3be700 1 Processor -- start 2022-06-17T12:54:12.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.864+0000 7f00bd3be700 1 -- start start 2022-06-17T12:54:12.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.864+0000 7f00bd3be700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00b80aef60 0x7f00b8155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:12.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.864+0000 7f00bd3be700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00b813cb70 0x7f00b8141e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:12.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.864+0000 7f00bd3be700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f00b813cf50 0x7f00b80afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:12.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.864+0000 7f00bd3be700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b80b2820 con 0x7f00b813cf50 2022-06-17T12:54:12.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.864+0000 7f00bd3be700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b80b2580 con 0x7f00b813cb70 2022-06-17T12:54:12.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.864+0000 7f00bd3be700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b80af790 con 0x7f00b80aef60 2022-06-17T12:54:12.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.864+0000 7f00b7fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00b80aef60 0x7f00b8155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:12.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.864+0000 7f00bcbbd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f00b813cf50 0x7f00b80afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:12.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.864+0000 7f00b7fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00b80aef60 0x7f00b8155920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:51120/0 (socket says 172.21.15.36:51120) 2022-06-17T12:54:12.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.864+0000 7f00b7fff700 1 -- 172.21.15.36:0/880761787 learned_addr learned my addr 172.21.15.36:0/880761787 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:12.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.864+0000 7f00b77fe700 1 --2- 172.21.15.36:0/880761787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00b813cb70 0x7f00b8141e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:12.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.868+0000 7f00bcbbd700 1 -- 172.21.15.36:0/880761787 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00b80aef60 msgr2=0x7f00b8155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:12.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.868+0000 7f00bcbbd700 1 --2- 172.21.15.36:0/880761787 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00b80aef60 0x7f00b8155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:12.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.868+0000 7f00bcbbd700 1 -- 172.21.15.36:0/880761787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00b813cb70 msgr2=0x7f00b8141e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:12.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.868+0000 7f00bcbbd700 1 --2- 172.21.15.36:0/880761787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00b813cb70 0x7f00b8141e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:12.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.868+0000 7f00bcbbd700 1 -- 172.21.15.36:0/880761787 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a8009530 con 0x7f00b813cf50 2022-06-17T12:54:12.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.868+0000 7f00b7fff700 1 --2- 172.21.15.36:0/880761787 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00b80aef60 0x7f00b8155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:12.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.868+0000 7f00b77fe700 1 --2- 172.21.15.36:0/880761787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00b813cb70 0x7f00b8141e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:12.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.868+0000 7f00bcbbd700 1 --2- 172.21.15.36:0/880761787 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f00b813cf50 0x7f00b80afc30 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7f00b000c000 tx=0x7f00b0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:12.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.868+0000 7f00b57fa700 1 -- 172.21.15.36:0/880761787 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00b0016070 con 0x7f00b813cf50 2022-06-17T12:54:12.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.868+0000 7f00b57fa700 1 -- 172.21.15.36:0/880761787 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b0011750 con 0x7f00b813cf50 2022-06-17T12:54:12.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.868+0000 7f00b57fa700 1 -- 172.21.15.36:0/880761787 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00b0011ab0 con 0x7f00b813cf50 2022-06-17T12:54:12.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.868+0000 7f00bd3be700 1 -- 172.21.15.36:0/880761787 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b8142350 con 0x7f00b813cf50 2022-06-17T12:54:12.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.868+0000 7f00bd3be700 1 -- 172.21.15.36:0/880761787 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b8140490 con 0x7f00b813cf50 2022-06-17T12:54:12.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.868+0000 7f00bd3be700 1 -- 172.21.15.36:0/880761787 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b813f490 con 0x7f00b813cf50 2022-06-17T12:54:12.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.868+0000 7f00b57fa700 1 -- 172.21.15.36:0/880761787 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f00b0011c10 con 0x7f00b813cf50 2022-06-17T12:54:12.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.868+0000 7f00b57fa700 1 --2- 172.21.15.36:0/880761787 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0098053940 0x7f0098055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:12.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.868+0000 7f00b57fa700 1 -- 172.21.15.36:0/880761787 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f00b006be10 con 0x7f00b813cf50 2022-06-17T12:54:12.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.868+0000 7f00b7fff700 1 --2- 172.21.15.36:0/880761787 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0098053940 0x7f0098055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:12.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.872+0000 7f00b7fff700 1 --2- 172.21.15.36:0/880761787 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0098053940 0x7f0098055e00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f00a801b000 tx=0x7f00a8016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:12.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:12.876+0000 7f00b57fa700 1 -- 172.21.15.36:0/880761787 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f00b813f490 con 0x7f00b813cf50 2022-06-17T12:54:12.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.000+0000 7f00bd3be700 1 -- 172.21.15.36:0/880761787 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00b813f490 con 0x7f00b813cf50 2022-06-17T12:54:12.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.004+0000 7f00b57fa700 1 -- 172.21.15.36:0/880761787 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f00b813f490 con 0x7f00b813cf50 2022-06-17T12:54:12.991 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:12.991 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":83,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:54:12.740133+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:54:11.237500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-06-17T12:54:11.985387+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:54:13.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.004+0000 7f00bd3be700 1 -- 172.21.15.36:0/880761787 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0098053940 msgr2=0x7f0098055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:13.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.004+0000 7f00bd3be700 1 --2- 172.21.15.36:0/880761787 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0098053940 0x7f0098055e00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f00a801b000 tx=0x7f00a8016000).stop 2022-06-17T12:54:13.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.004+0000 7f00bd3be700 1 -- 172.21.15.36:0/880761787 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f00b813cf50 msgr2=0x7f00b80afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:13.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.004+0000 7f00bd3be700 1 --2- 172.21.15.36:0/880761787 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f00b813cf50 0x7f00b80afc30 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7f00b000c000 tx=0x7f00b0006df0).stop 2022-06-17T12:54:13.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.004+0000 7f00bd3be700 1 -- 172.21.15.36:0/880761787 shutdown_connections 2022-06-17T12:54:13.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.004+0000 7f00bd3be700 1 --2- 172.21.15.36:0/880761787 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0098053940 0x7f0098055e00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:13.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.004+0000 7f00bd3be700 1 --2- 172.21.15.36:0/880761787 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f00b813cf50 0x7f00b80afc30 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:13.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.004+0000 7f00bd3be700 1 --2- 172.21.15.36:0/880761787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00b813cb70 0x7f00b8141e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:13.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.004+0000 7f00bd3be700 1 --2- 172.21.15.36:0/880761787 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00b80aef60 0x7f00b8155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:13.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.004+0000 7f00bd3be700 1 -- 172.21.15.36:0/880761787 >> 172.21.15.36:0/880761787 conn(0x7f00b805c550 msgr2=0x7f00b80b77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:13.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.004+0000 7f00bd3be700 1 -- 172.21.15.36:0/880761787 shutdown_connections 2022-06-17T12:54:13.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.004+0000 7f00bd3be700 1 -- 172.21.15.36:0/880761787 wait complete. 2022-06-17T12:54:13.010 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:54:13.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.132+0000 7fd10ef4a700 1 Processor -- start 2022-06-17T12:54:13.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.132+0000 7fd10ef4a700 1 -- start start 2022-06-17T12:54:13.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.132+0000 7fd10ef4a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd108062770 0x7fd108062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:13.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd10ef4a700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd1081444f0 0x7fd108144960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:13.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd10ef4a700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1080b0ba0 con 0x7fd1080ad330 2022-06-17T12:54:13.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd10ef4a700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1080ae820 con 0x7fd108062770 2022-06-17T12:54:13.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd10ef4a700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd108061030 con 0x7fd1081444f0 2022-06-17T12:54:13.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd10d4e7700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fd1080ad330 0x7fd1080ad7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53870/0 (socket says 172.21.15.36:53870) 2022-06-17T12:54:13.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd10d4e7700 1 -- 172.21.15.36:0/1628782468 learned_addr learned my addr 172.21.15.36:0/1628782468 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:13.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd10cce6700 1 --2- 172.21.15.36:0/1628782468 >> v2:172.21.15.36:3300/0 conn(0x7fd108062770 0x7fd108062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:13.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd107fff700 1 --2- 172.21.15.36:0/1628782468 >> v2:172.21.15.70:3300/0 conn(0x7fd1081444f0 0x7fd108144960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:13.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd1077fe700 1 -- 172.21.15.36:0/1628782468 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3752919127 0 0) 0x7fd1080b0ba0 con 0x7fd1080ad330 2022-06-17T12:54:13.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd1077fe700 1 -- 172.21.15.36:0/1628782468 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0e8003260 con 0x7fd1080ad330 2022-06-17T12:54:13.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd107fff700 1 -- 172.21.15.36:0/1628782468 >> v2:172.21.15.36:3300/0 conn(0x7fd108062770 msgr2=0x7fd108062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:13.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd107fff700 1 --2- 172.21.15.36:0/1628782468 >> v2:172.21.15.36:3300/0 conn(0x7fd108062770 0x7fd108062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:13.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd107fff700 1 -- 172.21.15.36:0/1628782468 >> v1:172.21.15.36:6789/0 conn(0x7fd1080ad330 legacy=0x7fd1080ad7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:13.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd107fff700 1 -- 172.21.15.36:0/1628782468 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd108147e90 con 0x7fd1081444f0 2022-06-17T12:54:13.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd10cce6700 1 --2- 172.21.15.36:0/1628782468 >> v2:172.21.15.36:3300/0 conn(0x7fd108062770 0x7fd108062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:13.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd107fff700 1 --2- 172.21.15.36:0/1628782468 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1081444f0 0x7fd108144960 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7fd0f0002f70 tx=0x7fd0f0006df0).ready entity=mon.1 client_cookie=38d0b084b6a7d306 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:13.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd1077fe700 1 -- 172.21.15.36:0/1628782468 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0f000d4f0 con 0x7fd1081444f0 2022-06-17T12:54:13.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd1077fe700 1 -- 172.21.15.36:0/1628782468 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f0000e90 con 0x7fd1081444f0 2022-06-17T12:54:13.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd1077fe700 1 -- 172.21.15.36:0/1628782468 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0f000d840 con 0x7fd1081444f0 2022-06-17T12:54:13.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd10ef4a700 1 -- 172.21.15.36:0/1628782468 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1081444f0 msgr2=0x7fd108144960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:13.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd10ef4a700 1 --2- 172.21.15.36:0/1628782468 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1081444f0 0x7fd108144960 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7fd0f0002f70 tx=0x7fd0f0006df0).stop 2022-06-17T12:54:13.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd10ef4a700 1 -- 172.21.15.36:0/1628782468 shutdown_connections 2022-06-17T12:54:13.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd10ef4a700 1 --2- 172.21.15.36:0/1628782468 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1081444f0 0x7fd108144960 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:13.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd10ef4a700 1 --2- 172.21.15.36:0/1628782468 >> v2:172.21.15.36:3300/0 conn(0x7fd108062770 0x7fd108062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:13.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd10ef4a700 1 -- 172.21.15.36:0/1628782468 >> 172.21.15.36:0/1628782468 conn(0x7fd1080b53e0 msgr2=0x7fd1080b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:13.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd10ef4a700 1 -- 172.21.15.36:0/1628782468 shutdown_connections 2022-06-17T12:54:13.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd10ef4a700 1 -- 172.21.15.36:0/1628782468 wait complete. 2022-06-17T12:54:13.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd10ef4a700 1 Processor -- start 2022-06-17T12:54:13.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.136+0000 7fd10ef4a700 1 -- start start 2022-06-17T12:54:13.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.140+0000 7fd10ef4a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd108062770 0x7fd108064520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:13.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.140+0000 7fd10ef4a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd1080ad330 0x7fd108143610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:13.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.140+0000 7fd10ef4a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1081444f0 0x7fd108143b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:13.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.140+0000 7fd10ef4a700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd108063090 con 0x7fd1081444f0 2022-06-17T12:54:13.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.140+0000 7fd10ef4a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd108064e00 con 0x7fd108062770 2022-06-17T12:54:13.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.140+0000 7fd10ef4a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1080632c0 con 0x7fd1080ad330 2022-06-17T12:54:13.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.140+0000 7fd10d4e7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1081444f0 0x7fd108143b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:13.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.140+0000 7fd10cce6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd108062770 0x7fd108064520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:13.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.140+0000 7fd10d4e7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1081444f0 0x7fd108143b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36348/0 (socket says 172.21.15.36:36348) 2022-06-17T12:54:13.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.140+0000 7fd107fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd1080ad330 0x7fd108143610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:13.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.140+0000 7fd10d4e7700 1 -- 172.21.15.36:0/4166734365 learned_addr learned my addr 172.21.15.36:0/4166734365 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:13.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.140+0000 7fd10d4e7700 1 -- 172.21.15.36:0/4166734365 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd1080ad330 msgr2=0x7fd108143610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:13.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.140+0000 7fd10d4e7700 1 --2- 172.21.15.36:0/4166734365 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd1080ad330 0x7fd108143610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:13.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.140+0000 7fd10d4e7700 1 -- 172.21.15.36:0/4166734365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd108062770 msgr2=0x7fd108064520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:13.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.140+0000 7fd10d4e7700 1 --2- 172.21.15.36:0/4166734365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd108062770 0x7fd108064520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:13.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.140+0000 7fd10d4e7700 1 -- 172.21.15.36:0/4166734365 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f800b180 con 0x7fd1081444f0 2022-06-17T12:54:13.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.140+0000 7fd10d4e7700 1 --2- 172.21.15.36:0/4166734365 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1081444f0 0x7fd108143b00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7fd100001490 tx=0x7fd100007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:13.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.140+0000 7fd105ffb700 1 -- 172.21.15.36:0/4166734365 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd100005ba0 con 0x7fd1081444f0 2022-06-17T12:54:13.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.140+0000 7fd105ffb700 1 -- 172.21.15.36:0/4166734365 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd100005d00 con 0x7fd1081444f0 2022-06-17T12:54:13.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.140+0000 7fd105ffb700 1 -- 172.21.15.36:0/4166734365 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1000072c0 con 0x7fd1081444f0 2022-06-17T12:54:13.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.140+0000 7fd10ef4a700 1 -- 172.21.15.36:0/4166734365 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f000b180 con 0x7fd1081444f0 2022-06-17T12:54:13.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.140+0000 7fd10ef4a700 1 -- 172.21.15.36:0/4166734365 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd108143ff0 con 0x7fd1081444f0 2022-06-17T12:54:13.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.140+0000 7fd10ef4a700 1 -- 172.21.15.36:0/4166734365 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd108067680 con 0x7fd1081444f0 2022-06-17T12:54:13.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.144+0000 7fd105ffb700 1 -- 172.21.15.36:0/4166734365 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd100010070 con 0x7fd1081444f0 2022-06-17T12:54:13.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.144+0000 7fd105ffb700 1 --2- 172.21.15.36:0/4166734365 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd0ec053ad0 0x7fd0ec055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:13.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.144+0000 7fd105ffb700 1 -- 172.21.15.36:0/4166734365 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd10006b980 con 0x7fd1081444f0 2022-06-17T12:54:13.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.144+0000 7fd10cce6700 1 --2- 172.21.15.36:0/4166734365 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd0ec053ad0 0x7fd0ec055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:13.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.144+0000 7fd10cce6700 1 --2- 172.21.15.36:0/4166734365 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd0ec053ad0 0x7fd0ec055f90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7fd0f800b000 tx=0x7fd0f8006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:13.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.148+0000 7fd105ffb700 1 -- 172.21.15.36:0/4166734365 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd1000425f0 con 0x7fd1081444f0 2022-06-17T12:54:13.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.272+0000 7fd10ef4a700 1 -- 172.21.15.36:0/4166734365 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd108064f70 con 0x7fd1081444f0 2022-06-17T12:54:13.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.276+0000 7fd105ffb700 1 -- 172.21.15.36:0/4166734365 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16287 (secure 0 0 0) 0x7fd100018070 con 0x7fd1081444f0 2022-06-17T12:54:13.264 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:13.264 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":83,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:54:12.740133+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:54:11.237500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-06-17T12:54:11.985387+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:54:13.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.276+0000 7fd10ef4a700 1 -- 172.21.15.36:0/4166734365 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd0ec053ad0 msgr2=0x7fd0ec055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:13.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.276+0000 7fd10ef4a700 1 --2- 172.21.15.36:0/4166734365 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd0ec053ad0 0x7fd0ec055f90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7fd0f800b000 tx=0x7fd0f8006df0).stop 2022-06-17T12:54:13.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.276+0000 7fd10ef4a700 1 -- 172.21.15.36:0/4166734365 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1081444f0 msgr2=0x7fd108143b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:13.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.276+0000 7fd10ef4a700 1 --2- 172.21.15.36:0/4166734365 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1081444f0 0x7fd108143b00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7fd100001490 tx=0x7fd100007e00).stop 2022-06-17T12:54:13.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.276+0000 7fd10ef4a700 1 -- 172.21.15.36:0/4166734365 shutdown_connections 2022-06-17T12:54:13.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.276+0000 7fd10ef4a700 1 --2- 172.21.15.36:0/4166734365 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd0ec053ad0 0x7fd0ec055f90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:13.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.276+0000 7fd10ef4a700 1 --2- 172.21.15.36:0/4166734365 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd1081444f0 0x7fd108143b00 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:13.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.276+0000 7fd10ef4a700 1 --2- 172.21.15.36:0/4166734365 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd1080ad330 0x7fd108143610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:13.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.276+0000 7fd10ef4a700 1 --2- 172.21.15.36:0/4166734365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd108062770 0x7fd108064520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:13.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.276+0000 7fd10ef4a700 1 -- 172.21.15.36:0/4166734365 >> 172.21.15.36:0/4166734365 conn(0x7fd1080b53e0 msgr2=0x7fd1080ad7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:13.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.276+0000 7fd10ef4a700 1 -- 172.21.15.36:0/4166734365 shutdown_connections 2022-06-17T12:54:13.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.276+0000 7fd10ef4a700 1 -- 172.21.15.36:0/4166734365 wait complete. 2022-06-17T12:54:13.293 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-17T12:54:13.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.420+0000 7f1aa1120700 1 Processor -- start 2022-06-17T12:54:13.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.420+0000 7f1aa1120700 1 -- start start 2022-06-17T12:54:13.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.420+0000 7f1aa1120700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a9c0b2ee0 0x7f1a9c0b1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:13.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.420+0000 7f1aa1120700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a9c0ad4c0 con 0x7f1a9c142d10 2022-06-17T12:54:13.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.420+0000 7f1aa1120700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a9c0b32c0 con 0x7f1a9c13eb10 2022-06-17T12:54:13.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.420+0000 7f1aa1120700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c0ad720 con 0x7f1a9c0b2ee0 2022-06-17T12:54:13.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.420+0000 7f1a9b59e700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f1a9c142d10 0x7f1a9c143180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53884/0 (socket says 172.21.15.36:53884) 2022-06-17T12:54:13.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.420+0000 7f1a9b59e700 1 -- 172.21.15.36:0/4262843407 learned_addr learned my addr 172.21.15.36:0/4262843407 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:13.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.420+0000 7f1a9ad9d700 1 --2- 172.21.15.36:0/4262843407 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a9c0b2ee0 0x7f1a9c0b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:13.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.420+0000 7f1a99d9b700 1 -- 172.21.15.36:0/4262843407 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1603725473 0 0) 0x7f1a9c0ad4c0 con 0x7f1a9c142d10 2022-06-17T12:54:13.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.420+0000 7f1a99d9b700 1 -- 172.21.15.36:0/4262843407 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a80003260 con 0x7f1a9c142d10 2022-06-17T12:54:13.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.420+0000 7f1a99d9b700 1 -- 172.21.15.36:0/4262843407 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7224619 0 0) 0x7f1a9c0b32c0 con 0x7f1a9c13eb10 2022-06-17T12:54:13.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.420+0000 7f1a99d9b700 1 -- 172.21.15.36:0/4262843407 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a9c0ad4c0 con 0x7f1a9c13eb10 2022-06-17T12:54:13.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.420+0000 7f1a99d9b700 1 -- 172.21.15.36:0/4262843407 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3586667412 0 0) 0x7f1a80003260 con 0x7f1a9c142d10 2022-06-17T12:54:13.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.420+0000 7f1a9ad9d700 1 -- 172.21.15.36:0/4262843407 >> v1:172.21.15.70:6789/0 conn(0x7f1a9c13eb10 legacy=0x7f1a9c13ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:13.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.420+0000 7f1a9ad9d700 1 -- 172.21.15.36:0/4262843407 >> v1:172.21.15.36:6789/0 conn(0x7f1a9c142d10 legacy=0x7f1a9c143180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:13.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.420+0000 7f1a9ad9d700 1 -- 172.21.15.36:0/4262843407 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9c0ad9c0 con 0x7f1a9c0b2ee0 2022-06-17T12:54:13.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.420+0000 7f1a9ad9d700 1 --2- 172.21.15.36:0/4262843407 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a9c0b2ee0 0x7f1a9c0b1120 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f1a94001490 tx=0x7f1a8c006df0).ready entity=mon.2 client_cookie=505163c4dd95feee server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:13.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.420+0000 7f1a99d9b700 1 -- 172.21.15.36:0/4262843407 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a94001a00 con 0x7f1a9c0b2ee0 2022-06-17T12:54:13.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.420+0000 7f1a99d9b700 1 -- 172.21.15.36:0/4262843407 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c000e90 con 0x7f1a9c0b2ee0 2022-06-17T12:54:13.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.420+0000 7f1a99d9b700 1 -- 172.21.15.36:0/4262843407 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a8c012460 con 0x7f1a9c0b2ee0 2022-06-17T12:54:13.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.424+0000 7f1aa1120700 1 -- 172.21.15.36:0/4262843407 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a9c0b2ee0 msgr2=0x7f1a9c0b1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:13.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.424+0000 7f1aa1120700 1 --2- 172.21.15.36:0/4262843407 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a9c0b2ee0 0x7f1a9c0b1120 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f1a94001490 tx=0x7f1a8c006df0).stop 2022-06-17T12:54:13.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.424+0000 7f1aa1120700 1 -- 172.21.15.36:0/4262843407 shutdown_connections 2022-06-17T12:54:13.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.424+0000 7f1aa1120700 1 --2- 172.21.15.36:0/4262843407 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a9c0b2ee0 0x7f1a9c0b1120 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:13.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.424+0000 7f1aa1120700 1 -- 172.21.15.36:0/4262843407 >> 172.21.15.36:0/4262843407 conn(0x7f1a9c063020 msgr2=0x7f1a9c063430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:13.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.424+0000 7f1aa1120700 1 -- 172.21.15.36:0/4262843407 shutdown_connections 2022-06-17T12:54:13.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.424+0000 7f1aa1120700 1 -- 172.21.15.36:0/4262843407 wait complete. 2022-06-17T12:54:13.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.424+0000 7f1aa1120700 1 Processor -- start 2022-06-17T12:54:13.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.424+0000 7f1aa1120700 1 -- start start 2022-06-17T12:54:13.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.428+0000 7f1aa1120700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a9c0b2ee0 0x7f1a9c0aa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:13.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.428+0000 7f1aa1120700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a9c13eb10 0x7f1a9c0ab520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:13.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.428+0000 7f1aa1120700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a9c142d10 0x7f1a9c0abf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:13.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.428+0000 7f1aa1120700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c13dac0 con 0x7f1a9c13eb10 2022-06-17T12:54:13.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.428+0000 7f1aa1120700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c0658c0 con 0x7f1a9c0b2ee0 2022-06-17T12:54:13.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.428+0000 7f1aa1120700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c065b60 con 0x7f1a9c142d10 2022-06-17T12:54:13.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.428+0000 7f1a9a59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a9c13eb10 0x7f1a9c0ab520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:13.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.428+0000 7f1a9a59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a9c13eb10 0x7f1a9c0ab520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36358/0 (socket says 172.21.15.36:36358) 2022-06-17T12:54:13.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.428+0000 7f1a9a59c700 1 -- 172.21.15.36:0/1983250558 learned_addr learned my addr 172.21.15.36:0/1983250558 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:13.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.428+0000 7f1a9ad9d700 1 --2- 172.21.15.36:0/1983250558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a9c0b2ee0 0x7f1a9c0aa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:13.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.428+0000 7f1a9b59e700 1 --2- 172.21.15.36:0/1983250558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a9c142d10 0x7f1a9c0abf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:13.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.428+0000 7f1a9a59c700 1 -- 172.21.15.36:0/1983250558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a9c142d10 msgr2=0x7f1a9c0abf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:13.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.428+0000 7f1a9a59c700 1 --2- 172.21.15.36:0/1983250558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a9c142d10 0x7f1a9c0abf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:13.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.428+0000 7f1a9a59c700 1 -- 172.21.15.36:0/1983250558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a9c0b2ee0 msgr2=0x7f1a9c0aa7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:13.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.428+0000 7f1a9a59c700 1 --2- 172.21.15.36:0/1983250558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a9c0b2ee0 0x7f1a9c0aa7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:13.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.428+0000 7f1a9a59c700 1 -- 172.21.15.36:0/1983250558 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a8c00b180 con 0x7f1a9c13eb10 2022-06-17T12:54:13.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.428+0000 7f1a9a59c700 1 --2- 172.21.15.36:0/1983250558 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a9c13eb10 0x7f1a9c0ab520 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7f1a90001110 tx=0x7f1a90007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:13.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.428+0000 7f1a8bfff700 1 -- 172.21.15.36:0/1983250558 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a90012400 con 0x7f1a9c13eb10 2022-06-17T12:54:13.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.428+0000 7f1a8bfff700 1 -- 172.21.15.36:0/1983250558 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a90012560 con 0x7f1a9c13eb10 2022-06-17T12:54:13.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.428+0000 7f1a8bfff700 1 -- 172.21.15.36:0/1983250558 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a900128c0 con 0x7f1a9c13eb10 2022-06-17T12:54:13.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.428+0000 7f1aa1120700 1 -- 172.21.15.36:0/1983250558 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a9c0aa170 con 0x7f1a9c13eb10 2022-06-17T12:54:13.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.428+0000 7f1aa1120700 1 -- 172.21.15.36:0/1983250558 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a9c0ac460 con 0x7f1a9c13eb10 2022-06-17T12:54:13.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.428+0000 7f1aa1120700 1 -- 172.21.15.36:0/1983250558 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a9c0aa300 con 0x7f1a9c13eb10 2022-06-17T12:54:13.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.432+0000 7f1a8bfff700 1 -- 172.21.15.36:0/1983250558 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f1a90019030 con 0x7f1a9c13eb10 2022-06-17T12:54:13.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.432+0000 7f1a8bfff700 1 --2- 172.21.15.36:0/1983250558 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1a780539d0 0x7f1a78055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:13.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.432+0000 7f1a8bfff700 1 -- 172.21.15.36:0/1983250558 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1a9006b360 con 0x7f1a9c13eb10 2022-06-17T12:54:13.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.432+0000 7f1a9ad9d700 1 --2- 172.21.15.36:0/1983250558 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1a780539d0 0x7f1a78055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:13.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.432+0000 7f1a9ad9d700 1 --2- 172.21.15.36:0/1983250558 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1a780539d0 0x7f1a78055e90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f1a8c01a000 tx=0x7f1a8c014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:13.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.436+0000 7f1a8bfff700 1 -- 172.21.15.36:0/1983250558 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1a90042160 con 0x7f1a9c13eb10 2022-06-17T12:54:13.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.600+0000 7f1aa1120700 1 -- 172.21.15.36:0/1983250558 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1a9c0b2120 con 0x7f1a9c13eb10 2022-06-17T12:54:13.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.748+0000 7f1a8bfff700 1 -- 172.21.15.36:0/1983250558 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v42) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1a90045780 con 0x7f1a9c13eb10 2022-06-17T12:54:13.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.752+0000 7f1aa1120700 1 -- 172.21.15.36:0/1983250558 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1a780539d0 msgr2=0x7f1a78055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:13.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.752+0000 7f1aa1120700 1 --2- 172.21.15.36:0/1983250558 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1a780539d0 0x7f1a78055e90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f1a8c01a000 tx=0x7f1a8c014000).stop 2022-06-17T12:54:13.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.752+0000 7f1aa1120700 1 -- 172.21.15.36:0/1983250558 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a9c13eb10 msgr2=0x7f1a9c0ab520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:13.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.752+0000 7f1aa1120700 1 --2- 172.21.15.36:0/1983250558 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a9c13eb10 0x7f1a9c0ab520 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7f1a90001110 tx=0x7f1a90007ae0).stop 2022-06-17T12:54:13.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.752+0000 7f1aa1120700 1 -- 172.21.15.36:0/1983250558 shutdown_connections 2022-06-17T12:54:13.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.752+0000 7f1aa1120700 1 --2- 172.21.15.36:0/1983250558 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1a780539d0 0x7f1a78055e90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:13.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.752+0000 7f1aa1120700 1 --2- 172.21.15.36:0/1983250558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a9c142d10 0x7f1a9c0abf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:13.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.752+0000 7f1aa1120700 1 --2- 172.21.15.36:0/1983250558 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1a9c13eb10 0x7f1a9c0ab520 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:13.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.752+0000 7f1aa1120700 1 --2- 172.21.15.36:0/1983250558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a9c0b2ee0 0x7f1a9c0aa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:13.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.752+0000 7f1aa1120700 1 -- 172.21.15.36:0/1983250558 >> 172.21.15.36:0/1983250558 conn(0x7f1a9c063020 msgr2=0x7f1a9c13fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:13.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.752+0000 7f1aa1120700 1 -- 172.21.15.36:0/1983250558 shutdown_connections 2022-06-17T12:54:13.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.752+0000 7f1aa1120700 1 -- 172.21.15.36:0/1983250558 wait complete. 2022-06-17T12:54:13.755 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:54:13.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.872+0000 7fde28cb7700 1 Processor -- start 2022-06-17T12:54:13.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.872+0000 7fde28cb7700 1 -- start start 2022-06-17T12:54:13.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.872+0000 7fde28cb7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde240aef60 0x7fde240b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:13.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.872+0000 7fde28cb7700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde240af4b0 con 0x7fde2413cb70 2022-06-17T12:54:13.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.872+0000 7fde28cb7700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde24142f50 con 0x7fde240ba2e0 2022-06-17T12:54:13.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.872+0000 7fde28cb7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde24143da0 con 0x7fde240aef60 2022-06-17T12:54:13.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.872+0000 7fde237fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fde2413cb70 0x7fde24143690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53896/0 (socket says 172.21.15.36:53896) 2022-06-17T12:54:13.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.872+0000 7fde237fe700 1 -- 172.21.15.36:0/2123199932 learned_addr learned my addr 172.21.15.36:0/2123199932 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:13.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.872+0000 7fde22ffd700 1 --2- 172.21.15.36:0/2123199932 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde240aef60 0x7fde240b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:13.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.872+0000 7fde227fc700 1 -- 172.21.15.36:0/2123199932 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 496605052 0 0) 0x7fde240af4b0 con 0x7fde2413cb70 2022-06-17T12:54:13.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.872+0000 7fde227fc700 1 -- 172.21.15.36:0/2123199932 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde10003260 con 0x7fde2413cb70 2022-06-17T12:54:13.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.872+0000 7fde227fc700 1 -- 172.21.15.36:0/2123199932 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 260027515 0 0) 0x7fde24142f50 con 0x7fde240ba2e0 2022-06-17T12:54:13.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.872+0000 7fde227fc700 1 -- 172.21.15.36:0/2123199932 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde240af4b0 con 0x7fde240ba2e0 2022-06-17T12:54:13.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.872+0000 7fde22ffd700 1 -- 172.21.15.36:0/2123199932 >> v1:172.21.15.70:6789/0 conn(0x7fde240ba2e0 legacy=0x7fde241442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:13.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.872+0000 7fde22ffd700 1 -- 172.21.15.36:0/2123199932 >> v1:172.21.15.36:6789/0 conn(0x7fde2413cb70 legacy=0x7fde24143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:13.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.872+0000 7fde22ffd700 1 -- 172.21.15.36:0/2123199932 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde241431f0 con 0x7fde240aef60 2022-06-17T12:54:13.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.876+0000 7fde22ffd700 1 --2- 172.21.15.36:0/2123199932 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde240aef60 0x7fde240b9df0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fde14002f70 tx=0x7fde14006df0).ready entity=mon.2 client_cookie=ab9b704e25691dc3 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:13.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.876+0000 7fde227fc700 1 -- 172.21.15.36:0/2123199932 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde1400d5c0 con 0x7fde240aef60 2022-06-17T12:54:13.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.876+0000 7fde227fc700 1 -- 172.21.15.36:0/2123199932 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde14000e90 con 0x7fde240aef60 2022-06-17T12:54:13.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.876+0000 7fde227fc700 1 -- 172.21.15.36:0/2123199932 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde1400d910 con 0x7fde240aef60 2022-06-17T12:54:13.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.876+0000 7fde28cb7700 1 -- 172.21.15.36:0/2123199932 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde240aef60 msgr2=0x7fde240b9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:13.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.876+0000 7fde28cb7700 1 --2- 172.21.15.36:0/2123199932 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde240aef60 0x7fde240b9df0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fde14002f70 tx=0x7fde14006df0).stop 2022-06-17T12:54:13.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.876+0000 7fde28cb7700 1 -- 172.21.15.36:0/2123199932 shutdown_connections 2022-06-17T12:54:13.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.876+0000 7fde28cb7700 1 --2- 172.21.15.36:0/2123199932 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde240aef60 0x7fde240b9df0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:13.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.876+0000 7fde28cb7700 1 -- 172.21.15.36:0/2123199932 >> 172.21.15.36:0/2123199932 conn(0x7fde2405c550 msgr2=0x7fde240b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:13.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.876+0000 7fde28cb7700 1 -- 172.21.15.36:0/2123199932 shutdown_connections 2022-06-17T12:54:13.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.876+0000 7fde28cb7700 1 -- 172.21.15.36:0/2123199932 wait complete. 2022-06-17T12:54:13.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.876+0000 7fde28cb7700 1 Processor -- start 2022-06-17T12:54:13.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.876+0000 7fde28cb7700 1 -- start start 2022-06-17T12:54:13.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.876+0000 7fde28cb7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde240aef60 0x7fde240afe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:13.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.876+0000 7fde28cb7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde240ba2e0 0x7fde240b5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:13.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.876+0000 7fde28cb7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde2413cb70 0x7fde24155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:13.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.876+0000 7fde28cb7700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde240b3be0 con 0x7fde240aef60 2022-06-17T12:54:13.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.876+0000 7fde28cb7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde240b36a0 con 0x7fde240ba2e0 2022-06-17T12:54:13.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.876+0000 7fde28cb7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde240b3940 con 0x7fde2413cb70 2022-06-17T12:54:13.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.880+0000 7fde237fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde240aef60 0x7fde240afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:13.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.880+0000 7fde22ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde240ba2e0 0x7fde240b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:13.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.880+0000 7fde237fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde240aef60 0x7fde240afe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36374/0 (socket says 172.21.15.36:36374) 2022-06-17T12:54:13.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.880+0000 7fde23fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde2413cb70 0x7fde24155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:13.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.880+0000 7fde22ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde240ba2e0 0x7fde240b5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:56754/0 (socket says 172.21.15.36:56754) 2022-06-17T12:54:13.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.880+0000 7fde237fe700 1 -- 172.21.15.36:0/2940448024 learned_addr learned my addr 172.21.15.36:0/2940448024 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:13.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.880+0000 7fde237fe700 1 -- 172.21.15.36:0/2940448024 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde2413cb70 msgr2=0x7fde24155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:13.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.880+0000 7fde237fe700 1 --2- 172.21.15.36:0/2940448024 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde2413cb70 0x7fde24155780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:13.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.880+0000 7fde237fe700 1 -- 172.21.15.36:0/2940448024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde240ba2e0 msgr2=0x7fde240b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:13.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.880+0000 7fde237fe700 1 --2- 172.21.15.36:0/2940448024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde240ba2e0 0x7fde240b5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:13.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.880+0000 7fde237fe700 1 -- 172.21.15.36:0/2940448024 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1400b180 con 0x7fde240aef60 2022-06-17T12:54:13.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.880+0000 7fde237fe700 1 --2- 172.21.15.36:0/2940448024 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde240aef60 0x7fde240afe40 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7fde180011d0 tx=0x7fde18007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:13.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.880+0000 7fde20ff9700 1 -- 172.21.15.36:0/2940448024 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde1800ce70 con 0x7fde240aef60 2022-06-17T12:54:13.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.880+0000 7fde20ff9700 1 -- 172.21.15.36:0/2940448024 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde180070d0 con 0x7fde240aef60 2022-06-17T12:54:13.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.880+0000 7fde28cb7700 1 -- 172.21.15.36:0/2940448024 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde241402a0 con 0x7fde240aef60 2022-06-17T12:54:13.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.880+0000 7fde20ff9700 1 -- 172.21.15.36:0/2940448024 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde18007430 con 0x7fde240aef60 2022-06-17T12:54:13.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.880+0000 7fde28cb7700 1 -- 172.21.15.36:0/2940448024 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde240b0f50 con 0x7fde240aef60 2022-06-17T12:54:13.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.880+0000 7fde28cb7700 1 -- 172.21.15.36:0/2940448024 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde240afb30 con 0x7fde240aef60 2022-06-17T12:54:13.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.880+0000 7fde20ff9700 1 -- 172.21.15.36:0/2940448024 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fde18010070 con 0x7fde240aef60 2022-06-17T12:54:13.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.884+0000 7fde20ff9700 1 --2- 172.21.15.36:0/2940448024 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde04053a60 0x7fde04055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:13.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.884+0000 7fde20ff9700 1 -- 172.21.15.36:0/2940448024 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fde1806b5b0 con 0x7fde240aef60 2022-06-17T12:54:13.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.884+0000 7fde22ffd700 1 --2- 172.21.15.36:0/2940448024 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde04053a60 0x7fde04055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:13.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.884+0000 7fde22ffd700 1 --2- 172.21.15.36:0/2940448024 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde04053a60 0x7fde04055f20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fde14018000 tx=0x7fde14014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:13.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:13.888+0000 7fde20ff9700 1 -- 172.21.15.36:0/2940448024 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fde180423b0 con 0x7fde240aef60 2022-06-17T12:54:14.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.060+0000 7fde28cb7700 1 -- 172.21.15.36:0/2940448024 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde241449b0 con 0x7fde240aef60 2022-06-17T12:54:14.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.060+0000 7fde20ff9700 1 -- 172.21.15.36:0/2940448024 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+2140 (secure 0 0 0) 0x7fde180459d0 con 0x7fde240aef60 2022-06-17T12:54:14.048 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:14.048 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":42,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/3068692336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3068692336},{"type":"v1","addr":"172.21.15.70:6833","nonce":3068692336}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:54:12.740483+0000","modified":"2022-06-17T12:54:13.744294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6753},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6753":{"gid":6753,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/2728662009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2728662009},{"type":"v1","addr":"172.21.15.36:6835","nonce":2728662009}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-06-17T12:54:14.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.064+0000 7fde28cb7700 1 -- 172.21.15.36:0/2940448024 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde04053a60 msgr2=0x7fde04055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:14.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.064+0000 7fde28cb7700 1 --2- 172.21.15.36:0/2940448024 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde04053a60 0x7fde04055f20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fde14018000 tx=0x7fde14014000).stop 2022-06-17T12:54:14.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.064+0000 7fde28cb7700 1 -- 172.21.15.36:0/2940448024 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde240aef60 msgr2=0x7fde240afe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:14.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.064+0000 7fde28cb7700 1 --2- 172.21.15.36:0/2940448024 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde240aef60 0x7fde240afe40 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7fde180011d0 tx=0x7fde18007b50).stop 2022-06-17T12:54:14.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.064+0000 7fde28cb7700 1 -- 172.21.15.36:0/2940448024 shutdown_connections 2022-06-17T12:54:14.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.064+0000 7fde28cb7700 1 --2- 172.21.15.36:0/2940448024 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde04053a60 0x7fde04055f20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.064+0000 7fde28cb7700 1 --2- 172.21.15.36:0/2940448024 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde2413cb70 0x7fde24155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.064+0000 7fde28cb7700 1 --2- 172.21.15.36:0/2940448024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde240ba2e0 0x7fde240b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.064+0000 7fde28cb7700 1 --2- 172.21.15.36:0/2940448024 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde240aef60 0x7fde240afe40 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.064+0000 7fde28cb7700 1 -- 172.21.15.36:0/2940448024 >> 172.21.15.36:0/2940448024 conn(0x7fde2405c550 msgr2=0x7fde241456c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:14.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.064+0000 7fde28cb7700 1 -- 172.21.15.36:0/2940448024 shutdown_connections 2022-06-17T12:54:14.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.064+0000 7fde28cb7700 1 -- 172.21.15.36:0/2940448024 wait complete. 2022-06-17T12:54:14.053 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 42 2022-06-17T12:54:14.061 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:54:14.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.172+0000 7f3d9b59e700 1 Processor -- start 2022-06-17T12:54:14.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.172+0000 7f3d9b59e700 1 -- start start 2022-06-17T12:54:14.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d9b59e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3d9c0b2200 0x7f3d9c0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:14.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d9b59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d9c0af8d0 0x7f3d9c14a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:14.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d9b59e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d9c0b25e0 con 0x7f3d9c0b2cd0 2022-06-17T12:54:14.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d9b59e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d9c0ad790 con 0x7f3d9c0b2200 2022-06-17T12:54:14.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d9b59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c0aef40 con 0x7f3d9c0af8d0 2022-06-17T12:54:14.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d99d9b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d9c0af8d0 0x7f3d9c14a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:14.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d99d9b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d9c0af8d0 0x7f3d9c14a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:51170/0 (socket says 172.21.15.36:51170) 2022-06-17T12:54:14.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d99d9b700 1 -- 172.21.15.36:0/921694065 learned_addr learned my addr 172.21.15.36:0/921694065 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:14.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d9ad9d700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3d9c0b2cd0 0x7f3d9c14adc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:49646/0 (socket says 172.21.15.36:49646) 2022-06-17T12:54:14.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d9a59c700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3d9c0b2200 0x7f3d9c0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:14.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d99d9b700 1 -- 172.21.15.36:0/921694065 >> v2:172.21.15.70:3300/0 conn(0x7f3d9c0b2200 msgr2=0x7f3d9c0af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:14.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d99d9b700 1 --2- 172.21.15.36:0/921694065 >> v2:172.21.15.70:3300/0 conn(0x7f3d9c0b2200 0x7f3d9c0af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d99d9b700 1 -- 172.21.15.36:0/921694065 >> v1:172.21.15.70:6789/0 conn(0x7f3d9c0b2cd0 legacy=0x7f3d9c14adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:14.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d99d9b700 1 -- 172.21.15.36:0/921694065 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c0b1950 con 0x7f3d9c0af8d0 2022-06-17T12:54:14.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d99d9b700 1 --2- 172.21.15.36:0/921694065 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d9c0af8d0 0x7f3d9c14a8d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f3d8c002f70 tx=0x7f3d8c006eb0).ready entity=mon.2 client_cookie=27ea7c3c875b8b3b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:14.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d9959a700 1 -- 172.21.15.36:0/921694065 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d8c00d570 con 0x7f3d9c0af8d0 2022-06-17T12:54:14.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d9959a700 1 -- 172.21.15.36:0/921694065 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8c000e90 con 0x7f3d9c0af8d0 2022-06-17T12:54:14.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d9959a700 1 -- 172.21.15.36:0/921694065 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d8c00d900 con 0x7f3d9c0af8d0 2022-06-17T12:54:14.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d9b59e700 1 -- 172.21.15.36:0/921694065 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d9c0af8d0 msgr2=0x7f3d9c14a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:14.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d9b59e700 1 --2- 172.21.15.36:0/921694065 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d9c0af8d0 0x7f3d9c14a8d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f3d8c002f70 tx=0x7f3d8c006eb0).stop 2022-06-17T12:54:14.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d9b59e700 1 -- 172.21.15.36:0/921694065 shutdown_connections 2022-06-17T12:54:14.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d9b59e700 1 --2- 172.21.15.36:0/921694065 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d9c0af8d0 0x7f3d9c14a8d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d9b59e700 1 --2- 172.21.15.36:0/921694065 >> v2:172.21.15.70:3300/0 conn(0x7f3d9c0b2200 0x7f3d9c0af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d9b59e700 1 -- 172.21.15.36:0/921694065 >> 172.21.15.36:0/921694065 conn(0x7f3d9c05f770 msgr2=0x7f3d9c05fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:14.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d9b59e700 1 -- 172.21.15.36:0/921694065 shutdown_connections 2022-06-17T12:54:14.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.176+0000 7f3d9b59e700 1 -- 172.21.15.36:0/921694065 wait complete. 2022-06-17T12:54:14.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.180+0000 7f3d9b59e700 1 Processor -- start 2022-06-17T12:54:14.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.180+0000 7f3d9b59e700 1 -- start start 2022-06-17T12:54:14.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.180+0000 7f3d9b59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3d9c0af8d0 0x7f3d9c159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:14.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.180+0000 7f3d9b59e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d9c0b2200 0x7f3d9c149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:14.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.180+0000 7f3d9b59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d9c0b2cd0 0x7f3d9c062390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:14.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.180+0000 7f3d9b59e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c0a8420 con 0x7f3d9c0af8d0 2022-06-17T12:54:14.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.180+0000 7f3d9b59e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c0a8db0 con 0x7f3d9c0b2200 2022-06-17T12:54:14.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.180+0000 7f3d9b59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c0644d0 con 0x7f3d9c0b2cd0 2022-06-17T12:54:14.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.180+0000 7f3d9a59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3d9c0af8d0 0x7f3d9c159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:14.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.184+0000 7f3d9a59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3d9c0af8d0 0x7f3d9c159db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36386/0 (socket says 172.21.15.36:36386) 2022-06-17T12:54:14.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.184+0000 7f3d99d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d9c0b2200 0x7f3d9c149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:14.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.184+0000 7f3d9a59c700 1 -- 172.21.15.36:0/1321765592 learned_addr learned my addr 172.21.15.36:0/1321765592 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:14.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.184+0000 7f3d9ad9d700 1 --2- 172.21.15.36:0/1321765592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d9c0b2cd0 0x7f3d9c062390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:14.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.184+0000 7f3d9a59c700 1 -- 172.21.15.36:0/1321765592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d9c0b2cd0 msgr2=0x7f3d9c062390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:14.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.184+0000 7f3d9a59c700 1 --2- 172.21.15.36:0/1321765592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d9c0b2cd0 0x7f3d9c062390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.184+0000 7f3d9a59c700 1 -- 172.21.15.36:0/1321765592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d9c0b2200 msgr2=0x7f3d9c149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:14.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.184+0000 7f3d9a59c700 1 --2- 172.21.15.36:0/1321765592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d9c0b2200 0x7f3d9c149c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.184+0000 7f3d9a59c700 1 -- 172.21.15.36:0/1321765592 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8c00b1e0 con 0x7f3d9c0af8d0 2022-06-17T12:54:14.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.184+0000 7f3d9a59c700 1 --2- 172.21.15.36:0/1321765592 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3d9c0af8d0 0x7f3d9c159db0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7f3d9c064a70 tx=0x7f3d90006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:14.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.184+0000 7f3d8b7fe700 1 -- 172.21.15.36:0/1321765592 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d90010780 con 0x7f3d9c0af8d0 2022-06-17T12:54:14.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.184+0000 7f3d8b7fe700 1 -- 172.21.15.36:0/1321765592 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d900108e0 con 0x7f3d9c0af8d0 2022-06-17T12:54:14.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.184+0000 7f3d9b59e700 1 -- 172.21.15.36:0/1321765592 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d9c064770 con 0x7f3d9c0af8d0 2022-06-17T12:54:14.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.184+0000 7f3d8b7fe700 1 -- 172.21.15.36:0/1321765592 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d90010c40 con 0x7f3d9c0af8d0 2022-06-17T12:54:14.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.184+0000 7f3d9b59e700 1 -- 172.21.15.36:0/1321765592 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d9c0a9480 con 0x7f3d9c0af8d0 2022-06-17T12:54:14.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.184+0000 7f3d8b7fe700 1 -- 172.21.15.36:0/1321765592 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f3d9001c460 con 0x7f3d9c0af8d0 2022-06-17T12:54:14.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.184+0000 7f3d9b59e700 1 -- 172.21.15.36:0/1321765592 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d9c0ad330 con 0x7f3d9c0af8d0 2022-06-17T12:54:14.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.188+0000 7f3d8b7fe700 1 --2- 172.21.15.36:0/1321765592 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3d78053a40 0x7f3d78055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:14.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.188+0000 7f3d8b7fe700 1 -- 172.21.15.36:0/1321765592 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3d9006c560 con 0x7f3d9c0af8d0 2022-06-17T12:54:14.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.188+0000 7f3d99d9b700 1 --2- 172.21.15.36:0/1321765592 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3d78053a40 0x7f3d78055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:14.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.188+0000 7f3d99d9b700 1 --2- 172.21.15.36:0/1321765592 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3d78053a40 0x7f3d78055f00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f3d8c01c000 tx=0x7f3d8c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:14.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.192+0000 7f3d8b7fe700 1 -- 172.21.15.36:0/1321765592 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3d90043170 con 0x7f3d9c0af8d0 2022-06-17T12:54:14.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.320+0000 7f3d9b59e700 1 -- 172.21.15.36:0/1321765592 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d9c057410 con 0x7f3d9c0af8d0 2022-06-17T12:54:14.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.320+0000 7f3d8b7fe700 1 -- 172.21.15.36:0/1321765592 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f3d90046790 con 0x7f3d9c0af8d0 2022-06-17T12:54:14.309 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:14.310 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":84,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:54:13.745105+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:54:11.237500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-06-17T12:54:11.985387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:54:14.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.324+0000 7f3d9b59e700 1 -- 172.21.15.36:0/1321765592 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3d78053a40 msgr2=0x7f3d78055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:14.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.324+0000 7f3d9b59e700 1 --2- 172.21.15.36:0/1321765592 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3d78053a40 0x7f3d78055f00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f3d8c01c000 tx=0x7f3d8c015000).stop 2022-06-17T12:54:14.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.324+0000 7f3d9b59e700 1 -- 172.21.15.36:0/1321765592 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3d9c0af8d0 msgr2=0x7f3d9c159db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:14.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.324+0000 7f3d9b59e700 1 --2- 172.21.15.36:0/1321765592 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3d9c0af8d0 0x7f3d9c159db0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7f3d9c064a70 tx=0x7f3d90006df0).stop 2022-06-17T12:54:14.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.324+0000 7f3d9b59e700 1 -- 172.21.15.36:0/1321765592 shutdown_connections 2022-06-17T12:54:14.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.324+0000 7f3d9b59e700 1 --2- 172.21.15.36:0/1321765592 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3d78053a40 0x7f3d78055f00 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.324+0000 7f3d9b59e700 1 --2- 172.21.15.36:0/1321765592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d9c0b2cd0 0x7f3d9c062390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.324+0000 7f3d9b59e700 1 --2- 172.21.15.36:0/1321765592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d9c0b2200 0x7f3d9c149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.324+0000 7f3d9b59e700 1 --2- 172.21.15.36:0/1321765592 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3d9c0af8d0 0x7f3d9c159db0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.324+0000 7f3d9b59e700 1 -- 172.21.15.36:0/1321765592 >> 172.21.15.36:0/1321765592 conn(0x7f3d9c05f770 msgr2=0x7f3d9c0b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:14.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.324+0000 7f3d9b59e700 1 -- 172.21.15.36:0/1321765592 shutdown_connections 2022-06-17T12:54:14.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.324+0000 7f3d9b59e700 1 -- 172.21.15.36:0/1321765592 wait complete. 2022-06-17T12:54:14.323 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:54:14.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.444+0000 7fe19de6c700 1 Processor -- start 2022-06-17T12:54:14.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.444+0000 7fe19de6c700 1 -- start start 2022-06-17T12:54:14.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.448+0000 7fe19de6c700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe1980b80c0 0x7fe198140c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:14.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.448+0000 7fe19de6c700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe198142160 con 0x7fe1980b7ce0 2022-06-17T12:54:14.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.448+0000 7fe19de6c700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1980b4f30 con 0x7fe198142d70 2022-06-17T12:54:14.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.448+0000 7fe19de6c700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe198142390 con 0x7fe1980b80c0 2022-06-17T12:54:14.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.448+0000 7fe19ce6a700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fe1980b7ce0 0x7fe19813aa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53924/0 (socket says 172.21.15.36:53924) 2022-06-17T12:54:14.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.448+0000 7fe19ce6a700 1 -- 172.21.15.36:0/410185378 learned_addr learned my addr 172.21.15.36:0/410185378 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:14.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.448+0000 7fe197fff700 1 --2- 172.21.15.36:0/410185378 >> v2:172.21.15.70:3300/0 conn(0x7fe1980b80c0 0x7fe198140c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:14.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.448+0000 7fe1977fe700 1 -- 172.21.15.36:0/410185378 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2967564514 0 0) 0x7fe198142160 con 0x7fe1980b7ce0 2022-06-17T12:54:14.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.448+0000 7fe1977fe700 1 -- 172.21.15.36:0/410185378 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe184003260 con 0x7fe1980b7ce0 2022-06-17T12:54:14.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.448+0000 7fe197fff700 1 -- 172.21.15.36:0/410185378 >> v1:172.21.15.70:6789/0 conn(0x7fe198142d70 legacy=0x7fe198143220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:14.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.448+0000 7fe197fff700 1 -- 172.21.15.36:0/410185378 >> v1:172.21.15.36:6789/0 conn(0x7fe1980b7ce0 legacy=0x7fe19813aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:14.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.448+0000 7fe197fff700 1 -- 172.21.15.36:0/410185378 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe198142630 con 0x7fe1980b80c0 2022-06-17T12:54:14.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.448+0000 7fe197fff700 1 --2- 172.21.15.36:0/410185378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1980b80c0 0x7fe198140c90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7fe188002f70 tx=0x7fe188006df0).ready entity=mon.1 client_cookie=b2ad8b8774937023 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:14.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.448+0000 7fe1977fe700 1 -- 172.21.15.36:0/410185378 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe18800d610 con 0x7fe1980b80c0 2022-06-17T12:54:14.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.448+0000 7fe1977fe700 1 -- 172.21.15.36:0/410185378 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe188000e90 con 0x7fe1980b80c0 2022-06-17T12:54:14.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.448+0000 7fe1977fe700 1 -- 172.21.15.36:0/410185378 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe18800d960 con 0x7fe1980b80c0 2022-06-17T12:54:14.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.448+0000 7fe19de6c700 1 -- 172.21.15.36:0/410185378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1980b80c0 msgr2=0x7fe198140c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:14.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.448+0000 7fe19de6c700 1 --2- 172.21.15.36:0/410185378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1980b80c0 0x7fe198140c90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7fe188002f70 tx=0x7fe188006df0).stop 2022-06-17T12:54:14.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.448+0000 7fe19de6c700 1 -- 172.21.15.36:0/410185378 shutdown_connections 2022-06-17T12:54:14.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.448+0000 7fe19de6c700 1 --2- 172.21.15.36:0/410185378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1980b80c0 0x7fe198140c90 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.448+0000 7fe19de6c700 1 -- 172.21.15.36:0/410185378 >> 172.21.15.36:0/410185378 conn(0x7fe19805c550 msgr2=0x7fe1980b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:14.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.448+0000 7fe19de6c700 1 -- 172.21.15.36:0/410185378 shutdown_connections 2022-06-17T12:54:14.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.448+0000 7fe19de6c700 1 -- 172.21.15.36:0/410185378 wait complete. 2022-06-17T12:54:14.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe19de6c700 1 Processor -- start 2022-06-17T12:54:14.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe19de6c700 1 -- start start 2022-06-17T12:54:14.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe19de6c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1980b7ce0 0x7fe1981415d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:14.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe19de6c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1980b80c0 0x7fe1980b2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:14.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe19de6c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe198142d70 0x7fe198066a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:14.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe19de6c700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1981410b0 con 0x7fe198142d70 2022-06-17T12:54:14.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe19de6c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19813fc80 con 0x7fe1980b7ce0 2022-06-17T12:54:14.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe19de6c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19813ff20 con 0x7fe1980b80c0 2022-06-17T12:54:14.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe19d66b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe198142d70 0x7fe198066a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:14.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe19d66b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe198142d70 0x7fe198066a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36402/0 (socket says 172.21.15.36:36402) 2022-06-17T12:54:14.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe19d66b700 1 -- 172.21.15.36:0/1987184482 learned_addr learned my addr 172.21.15.36:0/1987184482 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:14.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe197fff700 1 --2- 172.21.15.36:0/1987184482 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1980b80c0 0x7fe1980b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:14.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe19ce6a700 1 --2- 172.21.15.36:0/1987184482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1980b7ce0 0x7fe1981415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:14.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe19d66b700 1 -- 172.21.15.36:0/1987184482 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1980b80c0 msgr2=0x7fe1980b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:14.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe19d66b700 1 --2- 172.21.15.36:0/1987184482 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1980b80c0 0x7fe1980b2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe19d66b700 1 -- 172.21.15.36:0/1987184482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1980b7ce0 msgr2=0x7fe1981415d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:14.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe19d66b700 1 --2- 172.21.15.36:0/1987184482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1980b7ce0 0x7fe1981415d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe19d66b700 1 -- 172.21.15.36:0/1987184482 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18800b180 con 0x7fe198142d70 2022-06-17T12:54:14.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe197fff700 1 --2- 172.21.15.36:0/1987184482 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1980b80c0 0x7fe1980b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:54:14.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe19ce6a700 1 --2- 172.21.15.36:0/1987184482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1980b7ce0 0x7fe1981415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:54:14.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe19d66b700 1 --2- 172.21.15.36:0/1987184482 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe198142d70 0x7fe198066a00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7fe198142c50 tx=0x7fe1900013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:14.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe195ffb700 1 -- 172.21.15.36:0/1987184482 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe190006aa0 con 0x7fe198142d70 2022-06-17T12:54:14.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe195ffb700 1 -- 172.21.15.36:0/1987184482 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe190006c00 con 0x7fe198142d70 2022-06-17T12:54:14.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe19de6c700 1 -- 172.21.15.36:0/1987184482 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe198140d30 con 0x7fe198142d70 2022-06-17T12:54:14.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe195ffb700 1 -- 172.21.15.36:0/1987184482 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe190007030 con 0x7fe198142d70 2022-06-17T12:54:14.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.452+0000 7fe19de6c700 1 -- 172.21.15.36:0/1987184482 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe19813f950 con 0x7fe198142d70 2022-06-17T12:54:14.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.456+0000 7fe19de6c700 1 -- 172.21.15.36:0/1987184482 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1981428d0 con 0x7fe198142d70 2022-06-17T12:54:14.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.456+0000 7fe195ffb700 1 -- 172.21.15.36:0/1987184482 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe190010030 con 0x7fe198142d70 2022-06-17T12:54:14.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.456+0000 7fe195ffb700 1 --2- 172.21.15.36:0/1987184482 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe178053a40 0x7fe178055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:14.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.456+0000 7fe195ffb700 1 -- 172.21.15.36:0/1987184482 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe19006bf00 con 0x7fe198142d70 2022-06-17T12:54:14.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.456+0000 7fe19ce6a700 1 --2- 172.21.15.36:0/1987184482 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe178053a40 0x7fe178055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:14.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.456+0000 7fe19ce6a700 1 --2- 172.21.15.36:0/1987184482 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe178053a40 0x7fe178055f00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7fe18c008000 tx=0x7fe18c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:14.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.464+0000 7fe195ffb700 1 -- 172.21.15.36:0/1987184482 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe190042d00 con 0x7fe198142d70 2022-06-17T12:54:14.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.632+0000 7fe19de6c700 1 -- 172.21.15.36:0/1987184482 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fe198140090 con 0x7fe198142d70 2022-06-17T12:54:14.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.640+0000 7fe195ffb700 1 -- 172.21.15.36:0/1987184482 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v28) v1 ==== 184+0+0 (secure 0 0 0) 0x7fe198140090 con 0x7fe198142d70 2022-06-17T12:54:14.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.640+0000 7fe19de6c700 1 -- 172.21.15.36:0/1987184482 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe178053a40 msgr2=0x7fe178055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:14.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.640+0000 7fe19de6c700 1 --2- 172.21.15.36:0/1987184482 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe178053a40 0x7fe178055f00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7fe18c008000 tx=0x7fe18c009040).stop 2022-06-17T12:54:14.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.640+0000 7fe19de6c700 1 -- 172.21.15.36:0/1987184482 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe198142d70 msgr2=0x7fe198066a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:14.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.640+0000 7fe19de6c700 1 --2- 172.21.15.36:0/1987184482 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe198142d70 0x7fe198066a00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7fe198142c50 tx=0x7fe1900013a0).stop 2022-06-17T12:54:14.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.640+0000 7fe19de6c700 1 -- 172.21.15.36:0/1987184482 shutdown_connections 2022-06-17T12:54:14.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.640+0000 7fe19de6c700 1 --2- 172.21.15.36:0/1987184482 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe178053a40 0x7fe178055f00 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.640+0000 7fe19de6c700 1 --2- 172.21.15.36:0/1987184482 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe198142d70 0x7fe198066a00 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.640+0000 7fe19de6c700 1 --2- 172.21.15.36:0/1987184482 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1980b80c0 0x7fe1980b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.640+0000 7fe19de6c700 1 --2- 172.21.15.36:0/1987184482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1980b7ce0 0x7fe1981415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.640+0000 7fe19de6c700 1 -- 172.21.15.36:0/1987184482 >> 172.21.15.36:0/1987184482 conn(0x7fe19805c550 msgr2=0x7fe1980b5600 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:14.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.640+0000 7fe19de6c700 1 -- 172.21.15.36:0/1987184482 shutdown_connections 2022-06-17T12:54:14.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.644+0000 7fe19de6c700 1 -- 172.21.15.36:0/1987184482 wait complete. 2022-06-17T12:54:14.633 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.0 2022-06-17T12:54:14.642 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:54:14.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.764+0000 7f211bfff700 1 Processor -- start 2022-06-17T12:54:14.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.764+0000 7f211bfff700 1 -- start start 2022-06-17T12:54:14.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211bfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f211c0b80c0 0x7f211c140c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:14.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211bfff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f211c142160 con 0x7f211c0b7ce0 2022-06-17T12:54:14.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211bfff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f211c0b4f30 con 0x7f211c142d70 2022-06-17T12:54:14.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211bfff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211c142390 con 0x7f211c0b80c0 2022-06-17T12:54:14.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211affd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f211c0b7ce0 0x7f211c13aa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53938/0 (socket says 172.21.15.36:53938) 2022-06-17T12:54:14.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211affd700 1 -- 172.21.15.36:0/379271958 learned_addr learned my addr 172.21.15.36:0/379271958 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:14.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f2113fff700 1 --2- 172.21.15.36:0/379271958 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f211c0b80c0 0x7f211c140c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:14.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211a7fc700 1 -- 172.21.15.36:0/379271958 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972925999 0 0) 0x7f211c142160 con 0x7f211c0b7ce0 2022-06-17T12:54:14.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211a7fc700 1 -- 172.21.15.36:0/379271958 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20fc003260 con 0x7f211c0b7ce0 2022-06-17T12:54:14.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211a7fc700 1 -- 172.21.15.36:0/379271958 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1293751288 0 0) 0x7f211c0b4f30 con 0x7f211c142d70 2022-06-17T12:54:14.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211a7fc700 1 -- 172.21.15.36:0/379271958 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f211c142160 con 0x7f211c142d70 2022-06-17T12:54:14.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211a7fc700 1 -- 172.21.15.36:0/379271958 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1678075056 0 0) 0x7f20fc003260 con 0x7f211c0b7ce0 2022-06-17T12:54:14.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211a7fc700 1 -- 172.21.15.36:0/379271958 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f211c0b80c0 msgr2=0x7f211c140c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:14.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211a7fc700 1 --2- 172.21.15.36:0/379271958 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f211c0b80c0 0x7f211c140c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211a7fc700 1 -- 172.21.15.36:0/379271958 >> v1:172.21.15.70:6789/0 conn(0x7f211c142d70 legacy=0x7f211c1431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:14.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211a7fc700 1 -- 172.21.15.36:0/379271958 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211c142630 con 0x7f211c0b7ce0 2022-06-17T12:54:14.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211a7fc700 1 -- 172.21.15.36:0/379271958 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f210c001660 con 0x7f211c0b7ce0 2022-06-17T12:54:14.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f2113fff700 1 --2- 172.21.15.36:0/379271958 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f211c0b80c0 0x7f211c140c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:14.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211a7fc700 1 -- 172.21.15.36:0/379271958 >> v1:172.21.15.36:6789/0 conn(0x7f211c0b7ce0 legacy=0x7f211c13aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:14.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211a7fc700 1 --2- 172.21.15.36:0/379271958 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f211c0b80c0 0x7f20fc003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:14.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211a7fc700 1 --2- 172.21.15.36:0/379271958 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f20fc003e00 0x7f20fc004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:14.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211a7fc700 1 --2- 172.21.15.36:0/379271958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20fc0049f0 0x7f20fc004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:14.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211a7fc700 1 -- 172.21.15.36:0/379271958 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c142390 con 0x7f20fc003e00 2022-06-17T12:54:14.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211a7fc700 1 -- 172.21.15.36:0/379271958 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc00b7b0 con 0x7f20fc0049f0 2022-06-17T12:54:14.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f2113fff700 1 --2- 172.21.15.36:0/379271958 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f20fc003e00 0x7f20fc004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:14.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211a7fc700 1 -- 172.21.15.36:0/379271958 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc00b970 con 0x7f211c0b80c0 2022-06-17T12:54:14.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211b7fe700 1 --2- 172.21.15.36:0/379271958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20fc0049f0 0x7f20fc004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:14.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211affd700 1 --2- 172.21.15.36:0/379271958 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f211c0b80c0 0x7f20fc003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:14.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f2113fff700 1 -- 172.21.15.36:0/379271958 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f211c0b80c0 msgr2=0x7f20fc003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:14.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f2113fff700 1 --2- 172.21.15.36:0/379271958 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f211c0b80c0 0x7f20fc003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f2113fff700 1 -- 172.21.15.36:0/379271958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20fc0049f0 msgr2=0x7f20fc004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:14.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f2113fff700 1 --2- 172.21.15.36:0/379271958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20fc0049f0 0x7f20fc004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f2113fff700 1 -- 172.21.15.36:0/379271958 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210400b180 con 0x7f20fc003e00 2022-06-17T12:54:14.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f211b7fe700 1 --2- 172.21.15.36:0/379271958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20fc0049f0 0x7f20fc004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:14.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.768+0000 7f2113fff700 1 --2- 172.21.15.36:0/379271958 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f20fc003e00 0x7f20fc004500 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f210400c000 tx=0x7f2104006df0).ready entity=mon.0 client_cookie=76146bc3950df260 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:14.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.772+0000 7f211a7fc700 1 -- 172.21.15.36:0/379271958 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f210400e830 con 0x7f20fc003e00 2022-06-17T12:54:14.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.772+0000 7f211a7fc700 1 -- 172.21.15.36:0/379271958 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210400e990 con 0x7f20fc003e00 2022-06-17T12:54:14.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.772+0000 7f211a7fc700 1 -- 172.21.15.36:0/379271958 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f210400ecf0 con 0x7f20fc003e00 2022-06-17T12:54:14.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.772+0000 7f211bfff700 1 -- 172.21.15.36:0/379271958 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f20fc003e00 msgr2=0x7f20fc004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:14.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.772+0000 7f211bfff700 1 --2- 172.21.15.36:0/379271958 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f20fc003e00 0x7f20fc004500 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f210400c000 tx=0x7f2104006df0).stop 2022-06-17T12:54:14.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.772+0000 7f211b7fe700 1 -- 172.21.15.36:0/379271958 reap_dead start 2022-06-17T12:54:14.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.772+0000 7f211bfff700 1 -- 172.21.15.36:0/379271958 shutdown_connections 2022-06-17T12:54:14.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.772+0000 7f211bfff700 1 -- 172.21.15.36:0/379271958 >> 172.21.15.36:0/379271958 conn(0x7f211c05c550 msgr2=0x7f211c0b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:14.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.772+0000 7f211bfff700 1 -- 172.21.15.36:0/379271958 shutdown_connections 2022-06-17T12:54:14.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.772+0000 7f211bfff700 1 -- 172.21.15.36:0/379271958 wait complete. 2022-06-17T12:54:14.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.772+0000 7f211bfff700 1 Processor -- start 2022-06-17T12:54:14.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.772+0000 7f211bfff700 1 -- start start 2022-06-17T12:54:14.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.772+0000 7f211bfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f20fc003e00 0x7f211c1509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:14.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.772+0000 7f211bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f211c066eb0 0x7f211c140030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:14.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.772+0000 7f211bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f211c1440d0 0x7f211c1399f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:14.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.776+0000 7f211bfff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c140520 con 0x7f211c066eb0 2022-06-17T12:54:14.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.776+0000 7f211bfff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c150010 con 0x7f211c1440d0 2022-06-17T12:54:14.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.776+0000 7f211bfff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211c1502b0 con 0x7f20fc003e00 2022-06-17T12:54:14.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.776+0000 7f211b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f211c1440d0 0x7f211c1399f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:14.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.776+0000 7f211b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f211c1440d0 0x7f211c1399f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:56802/0 (socket says 172.21.15.36:56802) 2022-06-17T12:54:14.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.776+0000 7f211b7fe700 1 -- 172.21.15.36:0/1005189611 learned_addr learned my addr 172.21.15.36:0/1005189611 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:14.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.776+0000 7f2113fff700 1 --2- 172.21.15.36:0/1005189611 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f211c066eb0 0x7f211c140030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:14.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.776+0000 7f211affd700 1 --2- 172.21.15.36:0/1005189611 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f20fc003e00 0x7f211c1509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:14.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.776+0000 7f2113fff700 1 -- 172.21.15.36:0/1005189611 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f20fc003e00 msgr2=0x7f211c1509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:14.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.776+0000 7f2113fff700 1 --2- 172.21.15.36:0/1005189611 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f20fc003e00 0x7f211c1509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.776+0000 7f2113fff700 1 -- 172.21.15.36:0/1005189611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f211c1440d0 msgr2=0x7f211c1399f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:14.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.776+0000 7f2113fff700 1 --2- 172.21.15.36:0/1005189611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f211c1440d0 0x7f211c1399f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.776+0000 7f2113fff700 1 -- 172.21.15.36:0/1005189611 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211c150550 con 0x7f211c066eb0 2022-06-17T12:54:14.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.776+0000 7f211affd700 1 --2- 172.21.15.36:0/1005189611 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f20fc003e00 0x7f211c1509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:14.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.776+0000 7f211b7fe700 1 --2- 172.21.15.36:0/1005189611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f211c1440d0 0x7f211c1399f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:14.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.776+0000 7f2113fff700 1 --2- 172.21.15.36:0/1005189611 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f211c066eb0 0x7f211c140030 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7f210400c000 tx=0x7f2104019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:14.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.776+0000 7f2118ff9700 1 -- 172.21.15.36:0/1005189611 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f210401b820 con 0x7f211c066eb0 2022-06-17T12:54:14.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.776+0000 7f2118ff9700 1 -- 172.21.15.36:0/1005189611 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2104000e90 con 0x7f211c066eb0 2022-06-17T12:54:14.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.776+0000 7f211bfff700 1 -- 172.21.15.36:0/1005189611 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211c144550 con 0x7f211c066eb0 2022-06-17T12:54:14.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.776+0000 7f2118ff9700 1 -- 172.21.15.36:0/1005189611 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f210401bb70 con 0x7f211c066eb0 2022-06-17T12:54:14.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.776+0000 7f211bfff700 1 -- 172.21.15.36:0/1005189611 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211c13a080 con 0x7f211c066eb0 2022-06-17T12:54:14.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.776+0000 7f2118ff9700 1 -- 172.21.15.36:0/1005189611 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f210402c070 con 0x7f211c066eb0 2022-06-17T12:54:14.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.776+0000 7f211bfff700 1 -- 172.21.15.36:0/1005189611 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f211c1506e0 con 0x7f211c066eb0 2022-06-17T12:54:14.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.780+0000 7f2118ff9700 1 --2- 172.21.15.36:0/1005189611 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2100053a40 0x7f2100055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:14.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.780+0000 7f2118ff9700 1 -- 172.21.15.36:0/1005189611 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f210406d0f0 con 0x7f211c066eb0 2022-06-17T12:54:14.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.780+0000 7f211affd700 1 --2- 172.21.15.36:0/1005189611 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2100053a40 0x7f2100055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:14.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.780+0000 7f211affd700 1 --2- 172.21.15.36:0/1005189611 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2100053a40 0x7f2100055f00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f210c005000 tx=0x7f210c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:14.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.784+0000 7f2118ff9700 1 -- 172.21.15.36:0/1005189611 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2104043e20 con 0x7f211c066eb0 2022-06-17T12:54:14.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.952+0000 7f211bfff700 1 -- 172.21.15.36:0/1005189611 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f211c1428d0 con 0x7f211c066eb0 2022-06-17T12:54:14.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.960+0000 7f2118ff9700 1 -- 172.21.15.36:0/1005189611 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v29) v1 ==== 184+0+0 (secure 0 0 0) 0x7f2104047440 con 0x7f211c066eb0 2022-06-17T12:54:14.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.960+0000 7f211bfff700 1 -- 172.21.15.36:0/1005189611 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2100053a40 msgr2=0x7f2100055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:14.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.960+0000 7f211bfff700 1 --2- 172.21.15.36:0/1005189611 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2100053a40 0x7f2100055f00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f210c005000 tx=0x7f210c009040).stop 2022-06-17T12:54:14.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.960+0000 7f211bfff700 1 -- 172.21.15.36:0/1005189611 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f211c066eb0 msgr2=0x7f211c140030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:14.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.960+0000 7f211bfff700 1 --2- 172.21.15.36:0/1005189611 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f211c066eb0 0x7f211c140030 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7f210400c000 tx=0x7f2104019000).stop 2022-06-17T12:54:14.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.960+0000 7f211bfff700 1 -- 172.21.15.36:0/1005189611 shutdown_connections 2022-06-17T12:54:14.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.960+0000 7f211bfff700 1 --2- 172.21.15.36:0/1005189611 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2100053a40 0x7f2100055f00 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.960+0000 7f211bfff700 1 --2- 172.21.15.36:0/1005189611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f211c1440d0 0x7f211c1399f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.960+0000 7f211bfff700 1 --2- 172.21.15.36:0/1005189611 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f211c066eb0 0x7f211c140030 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.960+0000 7f211bfff700 1 --2- 172.21.15.36:0/1005189611 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f20fc003e00 0x7f211c1509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:14.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.960+0000 7f211bfff700 1 -- 172.21.15.36:0/1005189611 >> 172.21.15.36:0/1005189611 conn(0x7f211c05c550 msgr2=0x7f211c13ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:14.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.964+0000 7f211bfff700 1 -- 172.21.15.36:0/1005189611 shutdown_connections 2022-06-17T12:54:14.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:14.964+0000 7f211bfff700 1 -- 172.21.15.36:0/1005189611 wait complete. 2022-06-17T12:54:14.955 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.1 2022-06-17T12:54:14.963 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:54:15.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.092+0000 7ff36039b700 1 Processor -- start 2022-06-17T12:54:15.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.092+0000 7ff36039b700 1 -- start start 2022-06-17T12:54:15.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.092+0000 7ff36039b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3580601c0 0x7ff3580b2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:15.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.092+0000 7ff36039b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff358141cd0 0x7ff3580b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:15.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.092+0000 7ff36039b700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3580605a0 con 0x7ff3580b3890 2022-06-17T12:54:15.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.092+0000 7ff36039b700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff358145220 con 0x7ff358141cd0 2022-06-17T12:54:15.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.092+0000 7ff36039b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff358141120 con 0x7ff3580601c0 2022-06-17T12:54:15.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.092+0000 7ff35e938700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff358141cd0 0x7ff3580b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:15.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.092+0000 7ff35d936700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7ff3580b3890 0x7ff3581415c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53958/0 (socket says 172.21.15.36:53958) 2022-06-17T12:54:15.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.092+0000 7ff35d936700 1 -- 172.21.15.36:0/2152039044 learned_addr learned my addr 172.21.15.36:0/2152039044 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:15.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.092+0000 7ff35e137700 1 --2- 172.21.15.36:0/2152039044 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3580601c0 0x7ff3580b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:15.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.092+0000 7ff35e938700 1 -- 172.21.15.36:0/2152039044 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3580601c0 msgr2=0x7ff3580b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:15.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.092+0000 7ff35e938700 1 --2- 172.21.15.36:0/2152039044 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3580601c0 0x7ff3580b2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:15.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.092+0000 7ff35e938700 1 -- 172.21.15.36:0/2152039044 >> v1:172.21.15.36:6789/0 conn(0x7ff3580b3890 legacy=0x7ff3581415c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:15.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.092+0000 7ff35e938700 1 -- 172.21.15.36:0/2152039044 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff358140e10 con 0x7ff358141cd0 2022-06-17T12:54:15.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.092+0000 7ff35d135700 1 -- 172.21.15.36:0/2152039044 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3615706702 0 0) 0x7ff3580605a0 con 0x7ff3580b3890 2022-06-17T12:54:15.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff35e938700 1 --2- 172.21.15.36:0/2152039044 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff358141cd0 0x7ff3580b4010 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7ff3500032f0 tx=0x7ff3500091d0).ready entity=mon.0 client_cookie=3595a64b115b2cf1 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:15.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff35d135700 1 -- 172.21.15.36:0/2152039044 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff35000b6c0 con 0x7ff358141cd0 2022-06-17T12:54:15.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff35d135700 1 -- 172.21.15.36:0/2152039044 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff35000b820 con 0x7ff358141cd0 2022-06-17T12:54:15.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff35d135700 1 -- 172.21.15.36:0/2152039044 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff35000b990 con 0x7ff358141cd0 2022-06-17T12:54:15.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff36039b700 1 -- 172.21.15.36:0/2152039044 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff358141cd0 msgr2=0x7ff3580b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:15.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff36039b700 1 --2- 172.21.15.36:0/2152039044 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff358141cd0 0x7ff3580b4010 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7ff3500032f0 tx=0x7ff3500091d0).stop 2022-06-17T12:54:15.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff36039b700 1 -- 172.21.15.36:0/2152039044 shutdown_connections 2022-06-17T12:54:15.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff36039b700 1 --2- 172.21.15.36:0/2152039044 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff358141cd0 0x7ff3580b4010 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:15.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff36039b700 1 --2- 172.21.15.36:0/2152039044 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3580601c0 0x7ff3580b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:15.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff36039b700 1 -- 172.21.15.36:0/2152039044 >> 172.21.15.36:0/2152039044 conn(0x7ff358060990 msgr2=0x7ff35805f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:15.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff36039b700 1 -- 172.21.15.36:0/2152039044 shutdown_connections 2022-06-17T12:54:15.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff36039b700 1 -- 172.21.15.36:0/2152039044 wait complete. 2022-06-17T12:54:15.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff36039b700 1 Processor -- start 2022-06-17T12:54:15.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff36039b700 1 -- start start 2022-06-17T12:54:15.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff36039b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3580601c0 0x7ff3581435b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:15.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff36039b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff3580b3890 0x7ff3580614b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:15.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff36039b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff358141cd0 0x7ff35812f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:15.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff36039b700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff358144230 con 0x7ff3580b3890 2022-06-17T12:54:15.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff36039b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3580af5a0 con 0x7ff3580601c0 2022-06-17T12:54:15.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff36039b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3580af060 con 0x7ff358141cd0 2022-06-17T12:54:15.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff35d936700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff3580b3890 0x7ff3580614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:15.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff35d936700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff3580b3890 0x7ff3580614b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36436/0 (socket says 172.21.15.36:36436) 2022-06-17T12:54:15.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff35d936700 1 -- 172.21.15.36:0/3616962374 learned_addr learned my addr 172.21.15.36:0/3616962374 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:15.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff35e137700 1 --2- 172.21.15.36:0/3616962374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3580601c0 0x7ff3581435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:15.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff35e938700 1 --2- 172.21.15.36:0/3616962374 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff358141cd0 0x7ff35812f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:15.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff35d936700 1 -- 172.21.15.36:0/3616962374 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff358141cd0 msgr2=0x7ff35812f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:15.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff35d936700 1 --2- 172.21.15.36:0/3616962374 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff358141cd0 0x7ff35812f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:15.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff35d936700 1 -- 172.21.15.36:0/3616962374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3580601c0 msgr2=0x7ff3581435b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:15.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff35d936700 1 --2- 172.21.15.36:0/3616962374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3580601c0 0x7ff3581435b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:15.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff35d936700 1 -- 172.21.15.36:0/3616962374 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff350009470 con 0x7ff3580b3890 2022-06-17T12:54:15.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff35e137700 1 --2- 172.21.15.36:0/3616962374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3580601c0 0x7ff3581435b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:15.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff35d936700 1 --2- 172.21.15.36:0/3616962374 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff3580b3890 0x7ff3580614b0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7ff358144500 tx=0x7ff34c007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:15.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff3577fe700 1 -- 172.21.15.36:0/3616962374 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff34c00ce40 con 0x7ff3580b3890 2022-06-17T12:54:15.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.096+0000 7ff3577fe700 1 -- 172.21.15.36:0/3616962374 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34c0072e0 con 0x7ff3580b3890 2022-06-17T12:54:15.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.100+0000 7ff36039b700 1 -- 172.21.15.36:0/3616962374 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3580af300 con 0x7ff3580b3890 2022-06-17T12:54:15.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.100+0000 7ff3577fe700 1 -- 172.21.15.36:0/3616962374 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff34c005610 con 0x7ff3580b3890 2022-06-17T12:54:15.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.100+0000 7ff36039b700 1 -- 172.21.15.36:0/3616962374 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3580aeae0 con 0x7ff3580b3890 2022-06-17T12:54:15.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.100+0000 7ff36039b700 1 -- 172.21.15.36:0/3616962374 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3580aeed0 con 0x7ff3580b3890 2022-06-17T12:54:15.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.100+0000 7ff3577fe700 1 -- 172.21.15.36:0/3616962374 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff34c01a060 con 0x7ff3580b3890 2022-06-17T12:54:15.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.100+0000 7ff3577fe700 1 --2- 172.21.15.36:0/3616962374 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff3380539d0 0x7ff338055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:15.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.100+0000 7ff3577fe700 1 -- 172.21.15.36:0/3616962374 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff34c06b450 con 0x7ff3580b3890 2022-06-17T12:54:15.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.100+0000 7ff35e137700 1 --2- 172.21.15.36:0/3616962374 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff3380539d0 0x7ff338055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:15.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.100+0000 7ff35e137700 1 --2- 172.21.15.36:0/3616962374 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff3380539d0 0x7ff338055e90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7ff34800b000 tx=0x7ff348008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:15.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.104+0000 7ff3577fe700 1 -- 172.21.15.36:0/3616962374 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff34c042250 con 0x7ff3580b3890 2022-06-17T12:54:15.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.268+0000 7ff36039b700 1 -- 172.21.15.36:0/3616962374 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7ff35812efc0 con 0x7ff3580b3890 2022-06-17T12:54:15.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.276+0000 7ff3577fe700 1 -- 172.21.15.36:0/3616962374 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v30) v1 ==== 184+0+0 (secure 0 0 0) 0x7ff34c018020 con 0x7ff3580b3890 2022-06-17T12:54:15.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.280+0000 7ff36039b700 1 -- 172.21.15.36:0/3616962374 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff3380539d0 msgr2=0x7ff338055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:15.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.280+0000 7ff36039b700 1 --2- 172.21.15.36:0/3616962374 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff3380539d0 0x7ff338055e90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7ff34800b000 tx=0x7ff348008040).stop 2022-06-17T12:54:15.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.280+0000 7ff36039b700 1 -- 172.21.15.36:0/3616962374 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff3580b3890 msgr2=0x7ff3580614b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:15.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.280+0000 7ff36039b700 1 --2- 172.21.15.36:0/3616962374 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff3580b3890 0x7ff3580614b0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7ff358144500 tx=0x7ff34c007ac0).stop 2022-06-17T12:54:15.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.280+0000 7ff36039b700 1 -- 172.21.15.36:0/3616962374 shutdown_connections 2022-06-17T12:54:15.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.280+0000 7ff36039b700 1 --2- 172.21.15.36:0/3616962374 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff3380539d0 0x7ff338055e90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:15.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.280+0000 7ff36039b700 1 --2- 172.21.15.36:0/3616962374 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff358141cd0 0x7ff35812f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:15.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.280+0000 7ff36039b700 1 --2- 172.21.15.36:0/3616962374 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff3580b3890 0x7ff3580614b0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:15.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.280+0000 7ff36039b700 1 --2- 172.21.15.36:0/3616962374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3580601c0 0x7ff3581435b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:15.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.280+0000 7ff36039b700 1 -- 172.21.15.36:0/3616962374 >> 172.21.15.36:0/3616962374 conn(0x7ff358060990 msgr2=0x7ff358065520 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:15.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.280+0000 7ff36039b700 1 -- 172.21.15.36:0/3616962374 shutdown_connections 2022-06-17T12:54:15.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.280+0000 7ff36039b700 1 -- 172.21.15.36:0/3616962374 wait complete. 2022-06-17T12:54:15.275 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.2 2022-06-17T12:54:15.283 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:54:15.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.396+0000 7f9e88a6e700 1 Processor -- start 2022-06-17T12:54:15.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.396+0000 7f9e88a6e700 1 -- start start 2022-06-17T12:54:15.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.396+0000 7f9e88a6e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9e84060ac0 0x7f9e840ae970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:15.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.396+0000 7f9e88a6e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e84067290 0x7f9e840ad640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:15.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.396+0000 7f9e88a6e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9e840b07c0 0x7f9e840b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:15.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.396+0000 7f9e88a6e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e84143060 con 0x7f9e84060ac0 2022-06-17T12:54:15.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.396+0000 7f9e88a6e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e84060ea0 con 0x7f9e840b07c0 2022-06-17T12:54:15.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.396+0000 7f9e88a6e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e840aee60 con 0x7f9e84067290 2022-06-17T12:54:15.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.396+0000 7f9e8259c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9e84060ac0 0x7f9e840ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:15.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.396+0000 7f9e8259c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9e84060ac0 0x7f9e840ae970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36442/0 (socket says 172.21.15.36:36442) 2022-06-17T12:54:15.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.396+0000 7f9e8259c700 1 -- 172.21.15.36:0/1464448085 learned_addr learned my addr 172.21.15.36:0/1464448085 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:15.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.396+0000 7f9e81d9b700 1 --2- 172.21.15.36:0/1464448085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e84067290 0x7f9e840ad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:15.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.400+0000 7f9e82d9d700 1 --2- 172.21.15.36:0/1464448085 >> v2:172.21.15.70:3300/0 conn(0x7f9e840b07c0 0x7f9e840b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:15.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.400+0000 7f9e8259c700 1 -- 172.21.15.36:0/1464448085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e84067290 msgr2=0x7f9e840ad640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:15.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.400+0000 7f9e8259c700 1 --2- 172.21.15.36:0/1464448085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e84067290 0x7f9e840ad640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:15.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.400+0000 7f9e8259c700 1 -- 172.21.15.36:0/1464448085 >> v2:172.21.15.70:3300/0 conn(0x7f9e840b07c0 msgr2=0x7f9e840b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:15.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.400+0000 7f9e8259c700 1 --2- 172.21.15.36:0/1464448085 >> v2:172.21.15.70:3300/0 conn(0x7f9e840b07c0 0x7f9e840b0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:15.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.400+0000 7f9e8259c700 1 -- 172.21.15.36:0/1464448085 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e840b36a0 con 0x7f9e84060ac0 2022-06-17T12:54:15.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.400+0000 7f9e8259c700 1 --2- 172.21.15.36:0/1464448085 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9e84060ac0 0x7f9e840ae970 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7f9e74002f70 tx=0x7f9e74006eb0).ready entity=mon.0 client_cookie=a333e7193c454cd2 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:15.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.400+0000 7f9e8159a700 1 -- 172.21.15.36:0/1464448085 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e7400d570 con 0x7f9e84060ac0 2022-06-17T12:54:15.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.400+0000 7f9e8159a700 1 -- 172.21.15.36:0/1464448085 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e74000e90 con 0x7f9e84060ac0 2022-06-17T12:54:15.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.400+0000 7f9e8159a700 1 -- 172.21.15.36:0/1464448085 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e7400d900 con 0x7f9e84060ac0 2022-06-17T12:54:15.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.400+0000 7f9e88a6e700 1 -- 172.21.15.36:0/1464448085 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9e84060ac0 msgr2=0x7f9e840ae970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:15.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.400+0000 7f9e88a6e700 1 --2- 172.21.15.36:0/1464448085 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9e84060ac0 0x7f9e840ae970 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7f9e74002f70 tx=0x7f9e74006eb0).stop 2022-06-17T12:54:15.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.400+0000 7f9e88a6e700 1 -- 172.21.15.36:0/1464448085 shutdown_connections 2022-06-17T12:54:15.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.400+0000 7f9e88a6e700 1 --2- 172.21.15.36:0/1464448085 >> v2:172.21.15.70:3300/0 conn(0x7f9e840b07c0 0x7f9e840b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:15.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.400+0000 7f9e88a6e700 1 --2- 172.21.15.36:0/1464448085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e84067290 0x7f9e840ad640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:15.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.400+0000 7f9e88a6e700 1 --2- 172.21.15.36:0/1464448085 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9e84060ac0 0x7f9e840ae970 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:15.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.400+0000 7f9e88a6e700 1 -- 172.21.15.36:0/1464448085 >> 172.21.15.36:0/1464448085 conn(0x7f9e840b6870 msgr2=0x7f9e840b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:15.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.400+0000 7f9e88a6e700 1 -- 172.21.15.36:0/1464448085 shutdown_connections 2022-06-17T12:54:15.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.400+0000 7f9e88a6e700 1 -- 172.21.15.36:0/1464448085 wait complete. 2022-06-17T12:54:15.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.404+0000 7f9e88a6e700 1 Processor -- start 2022-06-17T12:54:15.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.404+0000 7f9e88a6e700 1 -- start start 2022-06-17T12:54:15.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.404+0000 7f9e88a6e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e84060ac0 0x7f9e8409f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:15.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.404+0000 7f9e88a6e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e84067290 0x7f9e8409fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:15.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.404+0000 7f9e88a6e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9e840b07c0 0x7f9e8413e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:15.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.404+0000 7f9e88a6e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e84148410 con 0x7f9e840b07c0 2022-06-17T12:54:15.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.404+0000 7f9e88a6e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e841477e0 con 0x7f9e84067290 2022-06-17T12:54:15.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.404+0000 7f9e88a6e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e84147cd0 con 0x7f9e84060ac0 2022-06-17T12:54:15.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.404+0000 7f9e82d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9e840b07c0 0x7f9e8413e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:15.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.404+0000 7f9e82d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9e840b07c0 0x7f9e8413e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36448/0 (socket says 172.21.15.36:36448) 2022-06-17T12:54:15.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.404+0000 7f9e82d9d700 1 -- 172.21.15.36:0/1177403412 learned_addr learned my addr 172.21.15.36:0/1177403412 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:15.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.404+0000 7f9e8259c700 1 --2- 172.21.15.36:0/1177403412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e84060ac0 0x7f9e8409f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:15.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.404+0000 7f9e81d9b700 1 --2- 172.21.15.36:0/1177403412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e84067290 0x7f9e8409fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:15.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.404+0000 7f9e82d9d700 1 -- 172.21.15.36:0/1177403412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e84060ac0 msgr2=0x7f9e8409f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:15.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.404+0000 7f9e82d9d700 1 --2- 172.21.15.36:0/1177403412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e84060ac0 0x7f9e8409f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:15.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.404+0000 7f9e82d9d700 1 -- 172.21.15.36:0/1177403412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e84067290 msgr2=0x7f9e8409fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:15.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.404+0000 7f9e82d9d700 1 --2- 172.21.15.36:0/1177403412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e84067290 0x7f9e8409fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:15.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.404+0000 7f9e82d9d700 1 -- 172.21.15.36:0/1177403412 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e7400b1e0 con 0x7f9e840b07c0 2022-06-17T12:54:15.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.404+0000 7f9e82d9d700 1 --2- 172.21.15.36:0/1177403412 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9e840b07c0 0x7f9e8413e200 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7f9e7c006e00 tx=0x7f9e7c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:15.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.404+0000 7f9e737fe700 1 -- 172.21.15.36:0/1177403412 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e7c00c670 con 0x7f9e840b07c0 2022-06-17T12:54:15.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.404+0000 7f9e737fe700 1 -- 172.21.15.36:0/1177403412 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7c00c7d0 con 0x7f9e840b07c0 2022-06-17T12:54:15.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.404+0000 7f9e88a6e700 1 -- 172.21.15.36:0/1177403412 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e84147f70 con 0x7f9e840b07c0 2022-06-17T12:54:15.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.404+0000 7f9e737fe700 1 -- 172.21.15.36:0/1177403412 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e7c00cb30 con 0x7f9e840b07c0 2022-06-17T12:54:15.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.404+0000 7f9e88a6e700 1 -- 172.21.15.36:0/1177403412 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e84147340 con 0x7f9e840b07c0 2022-06-17T12:54:15.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.408+0000 7f9e88a6e700 1 -- 172.21.15.36:0/1177403412 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e84147650 con 0x7f9e840b07c0 2022-06-17T12:54:15.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.408+0000 7f9e737fe700 1 -- 172.21.15.36:0/1177403412 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f9e7c00cc90 con 0x7f9e840b07c0 2022-06-17T12:54:15.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.408+0000 7f9e737fe700 1 --2- 172.21.15.36:0/1177403412 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9e600539b0 0x7f9e60055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:15.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.408+0000 7f9e8259c700 1 --2- 172.21.15.36:0/1177403412 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9e600539b0 0x7f9e60055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:15.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.408+0000 7f9e737fe700 1 -- 172.21.15.36:0/1177403412 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9e7c06c1d0 con 0x7f9e840b07c0 2022-06-17T12:54:15.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.408+0000 7f9e8259c700 1 --2- 172.21.15.36:0/1177403412 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9e600539b0 0x7f9e60055e70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f9e7401c000 tx=0x7f9e74015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:15.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.416+0000 7f9e737fe700 1 -- 172.21.15.36:0/1177403412 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9e7c042fd0 con 0x7f9e840b07c0 2022-06-17T12:54:15.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.580+0000 7f9e88a6e700 1 -- 172.21.15.36:0/1177403412 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e84148100 con 0x7f9e840b07c0 2022-06-17T12:54:15.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.580+0000 7f9e737fe700 1 -- 172.21.15.36:0/1177403412 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f9e7c0465f0 con 0x7f9e840b07c0 2022-06-17T12:54:15.569 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:15.569 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":42,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/3068692336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3068692336},{"type":"v1","addr":"172.21.15.70:6833","nonce":3068692336}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:54:12.740483+0000","modified":"2022-06-17T12:54:13.744294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6753},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6753":{"gid":6753,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/2728662009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2728662009},{"type":"v1","addr":"172.21.15.36:6835","nonce":2728662009}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-06-17T12:54:15.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.584+0000 7f9e88a6e700 1 -- 172.21.15.36:0/1177403412 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9e600539b0 msgr2=0x7f9e60055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:15.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.584+0000 7f9e88a6e700 1 --2- 172.21.15.36:0/1177403412 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9e600539b0 0x7f9e60055e70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f9e7401c000 tx=0x7f9e74015000).stop 2022-06-17T12:54:15.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.584+0000 7f9e88a6e700 1 -- 172.21.15.36:0/1177403412 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9e840b07c0 msgr2=0x7f9e8413e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:15.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.584+0000 7f9e88a6e700 1 --2- 172.21.15.36:0/1177403412 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9e840b07c0 0x7f9e8413e200 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7f9e7c006e00 tx=0x7f9e7c0091d0).stop 2022-06-17T12:54:15.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.584+0000 7f9e88a6e700 1 -- 172.21.15.36:0/1177403412 shutdown_connections 2022-06-17T12:54:15.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.584+0000 7f9e88a6e700 1 --2- 172.21.15.36:0/1177403412 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9e600539b0 0x7f9e60055e70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:15.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.584+0000 7f9e88a6e700 1 --2- 172.21.15.36:0/1177403412 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9e840b07c0 0x7f9e8413e200 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:15.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.584+0000 7f9e88a6e700 1 --2- 172.21.15.36:0/1177403412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e84067290 0x7f9e8409fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:15.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.584+0000 7f9e88a6e700 1 --2- 172.21.15.36:0/1177403412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e84060ac0 0x7f9e8409f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:15.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.584+0000 7f9e88a6e700 1 -- 172.21.15.36:0/1177403412 >> 172.21.15.36:0/1177403412 conn(0x7f9e840b6870 msgr2=0x7f9e840af250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:15.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.584+0000 7f9e88a6e700 1 -- 172.21.15.36:0/1177403412 shutdown_connections 2022-06-17T12:54:15.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:15.584+0000 7f9e88a6e700 1 -- 172.21.15.36:0/1177403412 wait complete. 2022-06-17T12:54:15.574 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 42 2022-06-17T12:54:15.784 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:54:12.740483+0000', 'modified': '2022-06-17T12:54:13.744294+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6753}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6753': {'gid': 6753, 'name': 'a', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.36:6835/2728662009', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 2728662009}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 2728662009}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:54:15.785 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:54:15.786 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:54:15.909 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":6753,"want_state":"up:active","state":"up:active","rank_uptime":3.1681156100000001,"mdsmap_epoch":42,"osdmap_epoch":84,"osdmap_epoch_barrier":83,"uptime":7.5684263610000002} 2022-06-17T12:54:15.910 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 6753, "mdsmap_epoch": 42, "osdmap_epoch": 84, "osdmap_epoch_barrier": 83, "rank_uptime": 3.16811561, "state": "up:active", "uptime": 7.568426361, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:54:15.911 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:54:15.986 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:54:15.993 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:54:15.996 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:54:16.000 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:54:16.003 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:54:16.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.024+0000 7f57de174700 1 Processor -- start 2022-06-17T12:54:16.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.024+0000 7f57de174700 1 -- start start 2022-06-17T12:54:16.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.024+0000 7f57de174700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f57d8060ac0 0x7f57d80ae970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:16.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.024+0000 7f57de174700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57d8060ea0 con 0x7f57d8067290 2022-06-17T12:54:16.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.024+0000 7f57de174700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57d80aee60 con 0x7f57d8141f70 2022-06-17T12:54:16.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.028+0000 7f57de174700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d8143060 con 0x7f57d8060ac0 2022-06-17T12:54:16.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.028+0000 7f57d6ffd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f57d8067290 0x7f57d80ad640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:53984/0 (socket says 172.21.15.36:53984) 2022-06-17T12:54:16.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.028+0000 7f57d6ffd700 1 -- 172.21.15.36:0/1097344588 learned_addr learned my addr 172.21.15.36:0/1097344588 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:16.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.028+0000 7f57d77fe700 1 --2- 172.21.15.36:0/1097344588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f57d8060ac0 0x7f57d80ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:16.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.028+0000 7f57d67fc700 1 -- 172.21.15.36:0/1097344588 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1877836144 0 0) 0x7f57d8060ea0 con 0x7f57d8067290 2022-06-17T12:54:16.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.028+0000 7f57d67fc700 1 -- 172.21.15.36:0/1097344588 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57bc003260 con 0x7f57d8067290 2022-06-17T12:54:16.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.028+0000 7f57d77fe700 1 -- 172.21.15.36:0/1097344588 >> v1:172.21.15.70:6789/0 conn(0x7f57d8141f70 legacy=0x7f57d81423c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.028+0000 7f57d77fe700 1 -- 172.21.15.36:0/1097344588 >> v1:172.21.15.36:6789/0 conn(0x7f57d8067290 legacy=0x7f57d80ad640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.028+0000 7f57d67fc700 1 -- 172.21.15.36:0/1097344588 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1817248416 0 0) 0x7f57bc003260 con 0x7f57d8067290 2022-06-17T12:54:16.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.028+0000 7f57d77fe700 1 -- 172.21.15.36:0/1097344588 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d80b36a0 con 0x7f57d8060ac0 2022-06-17T12:54:16.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.028+0000 7f57d77fe700 1 --2- 172.21.15.36:0/1097344588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f57d8060ac0 0x7f57d80ae970 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f57c8001800 tx=0x7f57d00091d0).ready entity=mon.2 client_cookie=e87533692b3357ee server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:16.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.028+0000 7f57d67fc700 1 -- 172.21.15.36:0/1097344588 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57c8001660 con 0x7f57d8060ac0 2022-06-17T12:54:16.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.028+0000 7f57d67fc700 1 -- 172.21.15.36:0/1097344588 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d000b580 con 0x7f57d8060ac0 2022-06-17T12:54:16.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.028+0000 7f57d67fc700 1 -- 172.21.15.36:0/1097344588 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57d000b6f0 con 0x7f57d8060ac0 2022-06-17T12:54:16.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.028+0000 7f57de174700 1 -- 172.21.15.36:0/1097344588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f57d8060ac0 msgr2=0x7f57d80ae970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.028+0000 7f57de174700 1 --2- 172.21.15.36:0/1097344588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f57d8060ac0 0x7f57d80ae970 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f57c8001800 tx=0x7f57d00091d0).stop 2022-06-17T12:54:16.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.028+0000 7f57de174700 1 -- 172.21.15.36:0/1097344588 shutdown_connections 2022-06-17T12:54:16.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.028+0000 7f57de174700 1 --2- 172.21.15.36:0/1097344588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f57d8060ac0 0x7f57d80ae970 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.028+0000 7f57de174700 1 -- 172.21.15.36:0/1097344588 >> 172.21.15.36:0/1097344588 conn(0x7f57d80b6870 msgr2=0x7f57d80b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:16.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.028+0000 7f57de174700 1 -- 172.21.15.36:0/1097344588 shutdown_connections 2022-06-17T12:54:16.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.028+0000 7f57de174700 1 -- 172.21.15.36:0/1097344588 wait complete. 2022-06-17T12:54:16.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.028+0000 7f57de174700 1 Processor -- start 2022-06-17T12:54:16.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.028+0000 7f57de174700 1 -- start start 2022-06-17T12:54:16.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57de174700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57d8060ac0 0x7f57d8140400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:16.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57de174700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f57d8067290 0x7f57d80a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:16.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57de174700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f57d8141f70 0x7f57d8152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:16.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57de174700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d80a1c40 con 0x7f57d8141f70 2022-06-17T12:54:16.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57de174700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d8147a50 con 0x7f57d8060ac0 2022-06-17T12:54:16.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57de174700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d8147510 con 0x7f57d8067290 2022-06-17T12:54:16.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57d7fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f57d8141f70 0x7f57d8152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:16.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57d6ffd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f57d8067290 0x7f57d80a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:16.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57d7fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f57d8141f70 0x7f57d8152a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36464/0 (socket says 172.21.15.36:36464) 2022-06-17T12:54:16.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57d7fff700 1 -- 172.21.15.36:0/3046555111 learned_addr learned my addr 172.21.15.36:0/3046555111 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:16.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57d77fe700 1 --2- 172.21.15.36:0/3046555111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57d8060ac0 0x7f57d8140400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:16.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57d7fff700 1 -- 172.21.15.36:0/3046555111 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f57d8067290 msgr2=0x7f57d80a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57d7fff700 1 --2- 172.21.15.36:0/3046555111 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f57d8067290 0x7f57d80a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57d7fff700 1 -- 172.21.15.36:0/3046555111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57d8060ac0 msgr2=0x7f57d8140400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57d7fff700 1 --2- 172.21.15.36:0/3046555111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57d8060ac0 0x7f57d8140400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57d7fff700 1 -- 172.21.15.36:0/3046555111 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d0009470 con 0x7f57d8141f70 2022-06-17T12:54:16.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57d6ffd700 1 --2- 172.21.15.36:0/3046555111 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f57d8067290 0x7f57d80a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:16.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57d7fff700 1 --2- 172.21.15.36:0/3046555111 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f57d8141f70 0x7f57d8152a40 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7f57d8147be0 tx=0x7f57c0007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:16.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57d4ff9700 1 -- 172.21.15.36:0/3046555111 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57c0007030 con 0x7f57d8141f70 2022-06-17T12:54:16.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57d4ff9700 1 -- 172.21.15.36:0/3046555111 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c0007190 con 0x7f57d8141f70 2022-06-17T12:54:16.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57de174700 1 -- 172.21.15.36:0/3046555111 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d81477b0 con 0x7f57d8141f70 2022-06-17T12:54:16.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57d4ff9700 1 -- 172.21.15.36:0/3046555111 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57c0006510 con 0x7f57d8141f70 2022-06-17T12:54:16.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57de174700 1 -- 172.21.15.36:0/3046555111 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d81486c0 con 0x7f57d8141f70 2022-06-17T12:54:16.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57d4ff9700 1 -- 172.21.15.36:0/3046555111 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f57c0010030 con 0x7f57d8141f70 2022-06-17T12:54:16.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.032+0000 7f57de174700 1 -- 172.21.15.36:0/3046555111 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d80b1b60 con 0x7f57d8141f70 2022-06-17T12:54:16.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.036+0000 7f57d4ff9700 1 --2- 172.21.15.36:0/3046555111 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f57b0053ad0 0x7f57b0055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:16.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.036+0000 7f57d4ff9700 1 -- 172.21.15.36:0/3046555111 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f57c006ba90 con 0x7f57d8141f70 2022-06-17T12:54:16.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.036+0000 7f57d77fe700 1 --2- 172.21.15.36:0/3046555111 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f57b0053ad0 0x7f57b0055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:16.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.036+0000 7f57d77fe700 1 --2- 172.21.15.36:0/3046555111 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f57b0053ad0 0x7f57b0055f90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f57d001b000 tx=0x7f57d0016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:16.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.040+0000 7f57d4ff9700 1 -- 172.21.15.36:0/3046555111 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f57c0042890 con 0x7f57d8141f70 2022-06-17T12:54:16.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.168+0000 7f57de174700 1 -- 172.21.15.36:0/3046555111 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57d8057410 con 0x7f57d8141f70 2022-06-17T12:54:16.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.168+0000 7f57d4ff9700 1 -- 172.21.15.36:0/3046555111 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f57c0045eb0 con 0x7f57d8141f70 2022-06-17T12:54:16.157 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:16.158 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":84,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:54:13.745105+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:54:11.237500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-06-17T12:54:11.985387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:54:16.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.172+0000 7f57de174700 1 -- 172.21.15.36:0/3046555111 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f57b0053ad0 msgr2=0x7f57b0055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.172+0000 7f57de174700 1 --2- 172.21.15.36:0/3046555111 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f57b0053ad0 0x7f57b0055f90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f57d001b000 tx=0x7f57d0016000).stop 2022-06-17T12:54:16.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.172+0000 7f57de174700 1 -- 172.21.15.36:0/3046555111 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f57d8141f70 msgr2=0x7f57d8152a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.172+0000 7f57de174700 1 --2- 172.21.15.36:0/3046555111 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f57d8141f70 0x7f57d8152a40 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7f57d8147be0 tx=0x7f57c0007c20).stop 2022-06-17T12:54:16.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.172+0000 7f57de174700 1 -- 172.21.15.36:0/3046555111 shutdown_connections 2022-06-17T12:54:16.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.172+0000 7f57de174700 1 --2- 172.21.15.36:0/3046555111 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f57b0053ad0 0x7f57b0055f90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.172+0000 7f57de174700 1 --2- 172.21.15.36:0/3046555111 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f57d8141f70 0x7f57d8152a40 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.172+0000 7f57de174700 1 --2- 172.21.15.36:0/3046555111 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f57d8067290 0x7f57d80a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.172+0000 7f57de174700 1 --2- 172.21.15.36:0/3046555111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57d8060ac0 0x7f57d8140400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.172+0000 7f57de174700 1 -- 172.21.15.36:0/3046555111 >> 172.21.15.36:0/3046555111 conn(0x7f57d80b6870 msgr2=0x7f57d80b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:16.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.172+0000 7f57de174700 1 -- 172.21.15.36:0/3046555111 shutdown_connections 2022-06-17T12:54:16.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.172+0000 7f57de174700 1 -- 172.21.15.36:0/3046555111 wait complete. 2022-06-17T12:54:16.170 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:54:11.237500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:54:11.985387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:54:16.172 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:54:16.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.292+0000 7f9c67fff700 1 Processor -- start 2022-06-17T12:54:16.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.292+0000 7f9c67fff700 1 -- start start 2022-06-17T12:54:16.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.292+0000 7f9c67fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9c68060020 0x7f9c680626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:16.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.292+0000 7f9c67fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c6813ca70 0x7f9c6813cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:16.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.292+0000 7f9c67fff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c681432c0 con 0x7f9c680b3f80 2022-06-17T12:54:16.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.292+0000 7f9c67fff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c68143df0 con 0x7f9c68060020 2022-06-17T12:54:16.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.292+0000 7f9c67fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c68061f60 con 0x7f9c6813ca70 2022-06-17T12:54:16.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.292+0000 7f9c66ffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9c68060020 0x7f9c680626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:16.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.292+0000 7f9c667fc700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f9c680b3f80 0x7f9c680b46a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54000/0 (socket says 172.21.15.36:54000) 2022-06-17T12:54:16.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.292+0000 7f9c667fc700 1 -- 172.21.15.36:0/240042092 learned_addr learned my addr 172.21.15.36:0/240042092 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:16.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.292+0000 7f9c677fe700 1 --2- 172.21.15.36:0/240042092 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c6813ca70 0x7f9c6813cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:16.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.292+0000 7f9c66ffd700 1 -- 172.21.15.36:0/240042092 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c6813ca70 msgr2=0x7f9c6813cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:16.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.292+0000 7f9c65ffb700 1 -- 172.21.15.36:0/240042092 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201075817 0 0) 0x7f9c681432c0 con 0x7f9c680b3f80 2022-06-17T12:54:16.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.292+0000 7f9c66ffd700 1 --2- 172.21.15.36:0/240042092 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c6813ca70 0x7f9c6813cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.292+0000 7f9c66ffd700 1 -- 172.21.15.36:0/240042092 >> v1:172.21.15.36:6789/0 conn(0x7f9c680b3f80 legacy=0x7f9c680b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.292+0000 7f9c66ffd700 1 -- 172.21.15.36:0/240042092 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c68062200 con 0x7f9c68060020 2022-06-17T12:54:16.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.292+0000 7f9c677fe700 1 --2- 172.21.15.36:0/240042092 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c6813ca70 0x7f9c6813cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:54:16.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.296+0000 7f9c66ffd700 1 --2- 172.21.15.36:0/240042092 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9c68060020 0x7f9c680626a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7f9c58002f70 tx=0x7f9c58006df0).ready entity=mon.0 client_cookie=ca6d705704225e47 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:16.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.296+0000 7f9c65ffb700 1 -- 172.21.15.36:0/240042092 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c5800d570 con 0x7f9c68060020 2022-06-17T12:54:16.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.296+0000 7f9c65ffb700 1 -- 172.21.15.36:0/240042092 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c58000e90 con 0x7f9c68060020 2022-06-17T12:54:16.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.296+0000 7f9c65ffb700 1 -- 172.21.15.36:0/240042092 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c5800d8c0 con 0x7f9c68060020 2022-06-17T12:54:16.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.296+0000 7f9c67fff700 1 -- 172.21.15.36:0/240042092 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9c68060020 msgr2=0x7f9c680626a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.296+0000 7f9c67fff700 1 --2- 172.21.15.36:0/240042092 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9c68060020 0x7f9c680626a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7f9c58002f70 tx=0x7f9c58006df0).stop 2022-06-17T12:54:16.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.296+0000 7f9c67fff700 1 -- 172.21.15.36:0/240042092 shutdown_connections 2022-06-17T12:54:16.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.296+0000 7f9c67fff700 1 --2- 172.21.15.36:0/240042092 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c6813ca70 0x7f9c6813cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.296+0000 7f9c67fff700 1 --2- 172.21.15.36:0/240042092 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9c68060020 0x7f9c680626a0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.296+0000 7f9c67fff700 1 -- 172.21.15.36:0/240042092 >> 172.21.15.36:0/240042092 conn(0x7f9c6805c550 msgr2=0x7f9c6805f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:16.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.296+0000 7f9c67fff700 1 -- 172.21.15.36:0/240042092 shutdown_connections 2022-06-17T12:54:16.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.296+0000 7f9c67fff700 1 -- 172.21.15.36:0/240042092 wait complete. 2022-06-17T12:54:16.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.296+0000 7f9c67fff700 1 Processor -- start 2022-06-17T12:54:16.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.296+0000 7f9c67fff700 1 -- start start 2022-06-17T12:54:16.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.300+0000 7f9c67fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c68060020 0x7f9c68141110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:16.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.300+0000 7f9c67fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c680b3f80 0x7f9c680aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:16.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.300+0000 7f9c67fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9c6813ca70 0x7f9c680af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:16.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.300+0000 7f9c67fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c68144b80 con 0x7f9c6813ca70 2022-06-17T12:54:16.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.300+0000 7f9c67fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c681467c0 con 0x7f9c680b3f80 2022-06-17T12:54:16.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.300+0000 7f9c67fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c68144440 con 0x7f9c68060020 2022-06-17T12:54:16.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.300+0000 7f9c677fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9c6813ca70 0x7f9c680af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:16.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.300+0000 7f9c677fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9c6813ca70 0x7f9c680af1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36478/0 (socket says 172.21.15.36:36478) 2022-06-17T12:54:16.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.300+0000 7f9c677fe700 1 -- 172.21.15.36:0/3617888321 learned_addr learned my addr 172.21.15.36:0/3617888321 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:16.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.300+0000 7f9c667fc700 1 --2- 172.21.15.36:0/3617888321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c680b3f80 0x7f9c680aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:16.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.300+0000 7f9c66ffd700 1 --2- 172.21.15.36:0/3617888321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c68060020 0x7f9c68141110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:16.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.300+0000 7f9c677fe700 1 -- 172.21.15.36:0/3617888321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c68060020 msgr2=0x7f9c68141110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.300+0000 7f9c677fe700 1 --2- 172.21.15.36:0/3617888321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c68060020 0x7f9c68141110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.300+0000 7f9c677fe700 1 -- 172.21.15.36:0/3617888321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c680b3f80 msgr2=0x7f9c680aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.300+0000 7f9c677fe700 1 --2- 172.21.15.36:0/3617888321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c680b3f80 0x7f9c680aecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.300+0000 7f9c677fe700 1 -- 172.21.15.36:0/3617888321 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5800b180 con 0x7f9c6813ca70 2022-06-17T12:54:16.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.300+0000 7f9c677fe700 1 --2- 172.21.15.36:0/3617888321 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9c6813ca70 0x7f9c680af1b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7f9c60007000 tx=0x7f9c60008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:16.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.300+0000 7f9c57fff700 1 -- 172.21.15.36:0/3617888321 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c6000e510 con 0x7f9c6813ca70 2022-06-17T12:54:16.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.300+0000 7f9c57fff700 1 -- 172.21.15.36:0/3617888321 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6000e670 con 0x7f9c6813ca70 2022-06-17T12:54:16.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.300+0000 7f9c67fff700 1 -- 172.21.15.36:0/3617888321 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c681446e0 con 0x7f9c6813ca70 2022-06-17T12:54:16.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.300+0000 7f9c57fff700 1 -- 172.21.15.36:0/3617888321 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c6000e9d0 con 0x7f9c6813ca70 2022-06-17T12:54:16.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.300+0000 7f9c67fff700 1 -- 172.21.15.36:0/3617888321 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c68146570 con 0x7f9c6813ca70 2022-06-17T12:54:16.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.300+0000 7f9c67fff700 1 -- 172.21.15.36:0/3617888321 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c681448a0 con 0x7f9c6813ca70 2022-06-17T12:54:16.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.304+0000 7f9c57fff700 1 -- 172.21.15.36:0/3617888321 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f9c60019030 con 0x7f9c6813ca70 2022-06-17T12:54:16.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.304+0000 7f9c57fff700 1 --2- 172.21.15.36:0/3617888321 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9c440539b0 0x7f9c44055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:16.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.304+0000 7f9c66ffd700 1 --2- 172.21.15.36:0/3617888321 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9c440539b0 0x7f9c44055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:16.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.304+0000 7f9c57fff700 1 -- 172.21.15.36:0/3617888321 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9c6006beb0 con 0x7f9c6813ca70 2022-06-17T12:54:16.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.304+0000 7f9c66ffd700 1 --2- 172.21.15.36:0/3617888321 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9c440539b0 0x7f9c44055e70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f9c5801c000 tx=0x7f9c58015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:16.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.312+0000 7f9c57fff700 1 -- 172.21.15.36:0/3617888321 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9c60042c10 con 0x7f9c6813ca70 2022-06-17T12:54:16.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.440+0000 7f9c67fff700 1 -- 172.21.15.36:0/3617888321 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9c68057410 con 0x7f9c6813ca70 2022-06-17T12:54:16.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.440+0000 7f9c57fff700 1 -- 172.21.15.36:0/3617888321 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v84) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9c60046230 con 0x7f9c6813ca70 2022-06-17T12:54:16.431 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:54:16.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.444+0000 7f9c67fff700 1 -- 172.21.15.36:0/3617888321 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9c440539b0 msgr2=0x7f9c44055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.444+0000 7f9c67fff700 1 --2- 172.21.15.36:0/3617888321 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9c440539b0 0x7f9c44055e70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f9c5801c000 tx=0x7f9c58015000).stop 2022-06-17T12:54:16.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.444+0000 7f9c67fff700 1 -- 172.21.15.36:0/3617888321 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9c6813ca70 msgr2=0x7f9c680af1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.444+0000 7f9c67fff700 1 --2- 172.21.15.36:0/3617888321 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9c6813ca70 0x7f9c680af1b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7f9c60007000 tx=0x7f9c60008040).stop 2022-06-17T12:54:16.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.444+0000 7f9c67fff700 1 -- 172.21.15.36:0/3617888321 shutdown_connections 2022-06-17T12:54:16.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.444+0000 7f9c67fff700 1 --2- 172.21.15.36:0/3617888321 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9c440539b0 0x7f9c44055e70 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.444+0000 7f9c67fff700 1 --2- 172.21.15.36:0/3617888321 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9c6813ca70 0x7f9c680af1b0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.444+0000 7f9c67fff700 1 --2- 172.21.15.36:0/3617888321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c680b3f80 0x7f9c680aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.444+0000 7f9c67fff700 1 --2- 172.21.15.36:0/3617888321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c68060020 0x7f9c68141110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.444+0000 7f9c67fff700 1 -- 172.21.15.36:0/3617888321 >> 172.21.15.36:0/3617888321 conn(0x7f9c6805c550 msgr2=0x7f9c680b5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:16.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.444+0000 7f9c67fff700 1 -- 172.21.15.36:0/3617888321 shutdown_connections 2022-06-17T12:54:16.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.444+0000 7f9c67fff700 1 -- 172.21.15.36:0/3617888321 wait complete. 2022-06-17T12:54:16.447 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:54:16.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.572+0000 7fdf3e00e700 1 Processor -- start 2022-06-17T12:54:16.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.572+0000 7fdf3e00e700 1 -- start start 2022-06-17T12:54:16.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf3e00e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf38062270 0x7fdf38143d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:16.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf3e00e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fdf380b4fc0 0x7fdf381471d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:16.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf3e00e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf38143130 con 0x7fdf380b4190 2022-06-17T12:54:16.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf3e00e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf380b6080 con 0x7fdf380b4fc0 2022-06-17T12:54:16.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf3e00e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf38062650 con 0x7fdf38062270 2022-06-17T12:54:16.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf37fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fdf380b4fc0 0x7fdf381471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:16.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf37fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fdf380b4fc0 0x7fdf381471d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36486/0 (socket says 172.21.15.36:36486) 2022-06-17T12:54:16.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf37fff700 1 -- 172.21.15.36:0/2189156168 learned_addr learned my addr 172.21.15.36:0/2189156168 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:16.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf377fe700 1 --2- 172.21.15.36:0/2189156168 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf38062270 0x7fdf38143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:16.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf37fff700 1 -- 172.21.15.36:0/2189156168 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf38062270 msgr2=0x7fdf38143d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:16.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf36ffd700 1 -- 172.21.15.36:0/2189156168 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3964200389 0 0) 0x7fdf38143130 con 0x7fdf380b4190 2022-06-17T12:54:16.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf37fff700 1 --2- 172.21.15.36:0/2189156168 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf38062270 0x7fdf38143d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf37fff700 1 -- 172.21.15.36:0/2189156168 >> v1:172.21.15.70:6789/0 conn(0x7fdf380b4190 legacy=0x7fdf380b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf37fff700 1 -- 172.21.15.36:0/2189156168 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf38143360 con 0x7fdf380b4fc0 2022-06-17T12:54:16.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf377fe700 1 --2- 172.21.15.36:0/2189156168 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf38062270 0x7fdf38143d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:16.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf37fff700 1 --2- 172.21.15.36:0/2189156168 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf380b4fc0 0x7fdf381471d0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7fdf300032f0 tx=0x7fdf30009290).ready entity=mon.0 client_cookie=44117f80214ea76b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:16.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf36ffd700 1 -- 172.21.15.36:0/2189156168 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf3000b8a0 con 0x7fdf380b4fc0 2022-06-17T12:54:16.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf36ffd700 1 -- 172.21.15.36:0/2189156168 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3000ba00 con 0x7fdf380b4fc0 2022-06-17T12:54:16.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf36ffd700 1 -- 172.21.15.36:0/2189156168 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf3000bb70 con 0x7fdf380b4fc0 2022-06-17T12:54:16.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf3e00e700 1 -- 172.21.15.36:0/2189156168 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf380b4fc0 msgr2=0x7fdf381471d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf3e00e700 1 --2- 172.21.15.36:0/2189156168 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf380b4fc0 0x7fdf381471d0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7fdf300032f0 tx=0x7fdf30009290).stop 2022-06-17T12:54:16.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf3e00e700 1 -- 172.21.15.36:0/2189156168 shutdown_connections 2022-06-17T12:54:16.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf3e00e700 1 --2- 172.21.15.36:0/2189156168 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf380b4fc0 0x7fdf381471d0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf3e00e700 1 --2- 172.21.15.36:0/2189156168 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf38062270 0x7fdf38143d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf3e00e700 1 -- 172.21.15.36:0/2189156168 >> 172.21.15.36:0/2189156168 conn(0x7fdf380b97f0 msgr2=0x7fdf380b9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:16.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf3e00e700 1 -- 172.21.15.36:0/2189156168 shutdown_connections 2022-06-17T12:54:16.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.576+0000 7fdf3e00e700 1 -- 172.21.15.36:0/2189156168 wait complete. 2022-06-17T12:54:16.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.580+0000 7fdf3e00e700 1 Processor -- start 2022-06-17T12:54:16.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.580+0000 7fdf3e00e700 1 -- start start 2022-06-17T12:54:16.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.580+0000 7fdf3e00e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf38062270 0x7fdf380b6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:16.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.580+0000 7fdf3e00e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf380b4190 0x7fdf38142260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:16.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.580+0000 7fdf3e00e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf380b4fc0 0x7fdf38142750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:16.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.580+0000 7fdf3e00e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380b23d0 con 0x7fdf380b4fc0 2022-06-17T12:54:16.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.580+0000 7fdf3e00e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380b2670 con 0x7fdf38062270 2022-06-17T12:54:16.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.580+0000 7fdf3e00e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380b2910 con 0x7fdf380b4190 2022-06-17T12:54:16.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.580+0000 7fdf37fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf380b4fc0 0x7fdf38142750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:16.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.580+0000 7fdf37fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf380b4fc0 0x7fdf38142750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36492/0 (socket says 172.21.15.36:36492) 2022-06-17T12:54:16.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.580+0000 7fdf37fff700 1 -- 172.21.15.36:0/1205808279 learned_addr learned my addr 172.21.15.36:0/1205808279 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:16.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.580+0000 7fdf377fe700 1 --2- 172.21.15.36:0/1205808279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf38062270 0x7fdf380b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:16.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.580+0000 7fdf2effd700 1 --2- 172.21.15.36:0/1205808279 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf380b4190 0x7fdf38142260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:16.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.580+0000 7fdf37fff700 1 -- 172.21.15.36:0/1205808279 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf380b4190 msgr2=0x7fdf38142260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.580+0000 7fdf37fff700 1 --2- 172.21.15.36:0/1205808279 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf380b4190 0x7fdf38142260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.580+0000 7fdf37fff700 1 -- 172.21.15.36:0/1205808279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf38062270 msgr2=0x7fdf380b6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.580+0000 7fdf37fff700 1 --2- 172.21.15.36:0/1205808279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf38062270 0x7fdf380b6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.580+0000 7fdf37fff700 1 -- 172.21.15.36:0/1205808279 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2800b180 con 0x7fdf380b4fc0 2022-06-17T12:54:16.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.580+0000 7fdf377fe700 1 --2- 172.21.15.36:0/1205808279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf38062270 0x7fdf380b6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:16.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.580+0000 7fdf2effd700 1 --2- 172.21.15.36:0/1205808279 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf380b4190 0x7fdf38142260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:16.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.580+0000 7fdf37fff700 1 --2- 172.21.15.36:0/1205808279 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf380b4fc0 0x7fdf38142750 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7fdf300032f0 tx=0x7fdf30016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:16.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.584+0000 7fdf357fa700 1 -- 172.21.15.36:0/1205808279 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf3000b8a0 con 0x7fdf380b4fc0 2022-06-17T12:54:16.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.584+0000 7fdf357fa700 1 -- 172.21.15.36:0/1205808279 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf30021630 con 0x7fdf380b4fc0 2022-06-17T12:54:16.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.584+0000 7fdf357fa700 1 -- 172.21.15.36:0/1205808279 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf30021990 con 0x7fdf380b4fc0 2022-06-17T12:54:16.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.584+0000 7fdf3e00e700 1 -- 172.21.15.36:0/1205808279 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf30009530 con 0x7fdf380b4fc0 2022-06-17T12:54:16.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.584+0000 7fdf3e00e700 1 -- 172.21.15.36:0/1205808279 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3812d760 con 0x7fdf380b4fc0 2022-06-17T12:54:16.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.584+0000 7fdf3e00e700 1 -- 172.21.15.36:0/1205808279 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf38143600 con 0x7fdf380b4fc0 2022-06-17T12:54:16.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.584+0000 7fdf357fa700 1 -- 172.21.15.36:0/1205808279 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fdf30029070 con 0x7fdf380b4fc0 2022-06-17T12:54:16.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.584+0000 7fdf357fa700 1 --2- 172.21.15.36:0/1205808279 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdf1c053ad0 0x7fdf1c055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:16.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.584+0000 7fdf357fa700 1 -- 172.21.15.36:0/1205808279 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdf3006cdb0 con 0x7fdf380b4fc0 2022-06-17T12:54:16.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.584+0000 7fdf377fe700 1 --2- 172.21.15.36:0/1205808279 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdf1c053ad0 0x7fdf1c055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:16.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.584+0000 7fdf377fe700 1 --2- 172.21.15.36:0/1205808279 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdf1c053ad0 0x7fdf1c055f90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fdf2800b000 tx=0x7fdf28006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:16.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.592+0000 7fdf357fa700 1 -- 172.21.15.36:0/1205808279 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdf30043bb0 con 0x7fdf380b4fc0 2022-06-17T12:54:16.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.720+0000 7fdf3e00e700 1 -- 172.21.15.36:0/1205808279 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdf38142de0 con 0x7fdf380b4fc0 2022-06-17T12:54:16.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.720+0000 7fdf357fa700 1 -- 172.21.15.36:0/1205808279 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v84) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdf30015070 con 0x7fdf380b4fc0 2022-06-17T12:54:16.710 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:54:16.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.724+0000 7fdf3e00e700 1 -- 172.21.15.36:0/1205808279 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdf1c053ad0 msgr2=0x7fdf1c055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.724+0000 7fdf3e00e700 1 --2- 172.21.15.36:0/1205808279 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdf1c053ad0 0x7fdf1c055f90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fdf2800b000 tx=0x7fdf28006df0).stop 2022-06-17T12:54:16.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.724+0000 7fdf3e00e700 1 -- 172.21.15.36:0/1205808279 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf380b4fc0 msgr2=0x7fdf38142750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.724+0000 7fdf3e00e700 1 --2- 172.21.15.36:0/1205808279 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf380b4fc0 0x7fdf38142750 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7fdf300032f0 tx=0x7fdf30016000).stop 2022-06-17T12:54:16.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.724+0000 7fdf3e00e700 1 -- 172.21.15.36:0/1205808279 shutdown_connections 2022-06-17T12:54:16.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.724+0000 7fdf3e00e700 1 --2- 172.21.15.36:0/1205808279 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdf1c053ad0 0x7fdf1c055f90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.724+0000 7fdf3e00e700 1 --2- 172.21.15.36:0/1205808279 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdf380b4fc0 0x7fdf38142750 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.724+0000 7fdf3e00e700 1 --2- 172.21.15.36:0/1205808279 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf380b4190 0x7fdf38142260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.724+0000 7fdf3e00e700 1 --2- 172.21.15.36:0/1205808279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf38062270 0x7fdf380b6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.724+0000 7fdf3e00e700 1 -- 172.21.15.36:0/1205808279 >> 172.21.15.36:0/1205808279 conn(0x7fdf380b97f0 msgr2=0x7fdf380b8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:16.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.724+0000 7fdf3e00e700 1 -- 172.21.15.36:0/1205808279 shutdown_connections 2022-06-17T12:54:16.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.724+0000 7fdf3e00e700 1 -- 172.21.15.36:0/1205808279 wait complete. 2022-06-17T12:54:16.732 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:54:16.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.864+0000 7f2bd931a700 1 Processor -- start 2022-06-17T12:54:16.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.864+0000 7f2bd931a700 1 -- start start 2022-06-17T12:54:16.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.864+0000 7f2bd931a700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2bd4060920 0x7f2bd4138ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:16.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.864+0000 7f2bd931a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2bd413fe30 0x7f2bd41394e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:16.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.864+0000 7f2bd931a700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd40af610 con 0x7f2bd40b11a0 2022-06-17T12:54:16.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.864+0000 7f2bd931a700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd40ae190 con 0x7f2bd4060920 2022-06-17T12:54:16.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.864+0000 7f2bd931a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd40ae9d0 con 0x7f2bd413fe30 2022-06-17T12:54:16.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.864+0000 7f2bd8b19700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f2bd40b11a0 0x7f2bd40ab9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54030/0 (socket says 172.21.15.36:54030) 2022-06-17T12:54:16.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.864+0000 7f2bd8b19700 1 -- 172.21.15.36:0/3610193003 learned_addr learned my addr 172.21.15.36:0/3610193003 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:16.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.864+0000 7f2bcbfff700 1 --2- 172.21.15.36:0/3610193003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2bd413fe30 0x7f2bd41394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:16.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.864+0000 7f2bd3fff700 1 --2- 172.21.15.36:0/3610193003 >> v2:172.21.15.70:3300/0 conn(0x7f2bd4060920 0x7f2bd4138ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:16.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.864+0000 7f2bd37fe700 1 -- 172.21.15.36:0/3610193003 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4109754220 0 0) 0x7f2bd40af610 con 0x7f2bd40b11a0 2022-06-17T12:54:16.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.864+0000 7f2bd37fe700 1 -- 172.21.15.36:0/3610193003 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bb4003260 con 0x7f2bd40b11a0 2022-06-17T12:54:16.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.864+0000 7f2bcbfff700 1 -- 172.21.15.36:0/3610193003 >> v2:172.21.15.70:3300/0 conn(0x7f2bd4060920 msgr2=0x7f2bd4138ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:16.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.864+0000 7f2bcbfff700 1 --2- 172.21.15.36:0/3610193003 >> v2:172.21.15.70:3300/0 conn(0x7f2bd4060920 0x7f2bd4138ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.864+0000 7f2bcbfff700 1 -- 172.21.15.36:0/3610193003 >> v1:172.21.15.36:6789/0 conn(0x7f2bd40b11a0 legacy=0x7f2bd40ab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.864+0000 7f2bcbfff700 1 -- 172.21.15.36:0/3610193003 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd40af8b0 con 0x7f2bd413fe30 2022-06-17T12:54:16.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.868+0000 7f2bcbfff700 1 --2- 172.21.15.36:0/3610193003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2bd413fe30 0x7f2bd41394e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f2bbc002f70 tx=0x7f2bbc006e80).ready entity=mon.2 client_cookie=df63fe48be0a51bc server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:16.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.868+0000 7f2bd37fe700 1 -- 172.21.15.36:0/3610193003 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bbc014070 con 0x7f2bd413fe30 2022-06-17T12:54:16.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.868+0000 7f2bd37fe700 1 -- 172.21.15.36:0/3610193003 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bbc000e90 con 0x7f2bd413fe30 2022-06-17T12:54:16.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.868+0000 7f2bd37fe700 1 -- 172.21.15.36:0/3610193003 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bbc0107f0 con 0x7f2bd413fe30 2022-06-17T12:54:16.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.868+0000 7f2bd931a700 1 -- 172.21.15.36:0/3610193003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2bd413fe30 msgr2=0x7f2bd41394e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.868+0000 7f2bd931a700 1 --2- 172.21.15.36:0/3610193003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2bd413fe30 0x7f2bd41394e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f2bbc002f70 tx=0x7f2bbc006e80).stop 2022-06-17T12:54:16.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.868+0000 7f2bd931a700 1 -- 172.21.15.36:0/3610193003 shutdown_connections 2022-06-17T12:54:16.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.868+0000 7f2bd931a700 1 --2- 172.21.15.36:0/3610193003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2bd413fe30 0x7f2bd41394e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.868+0000 7f2bd931a700 1 --2- 172.21.15.36:0/3610193003 >> v2:172.21.15.70:3300/0 conn(0x7f2bd4060920 0x7f2bd4138ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.868+0000 7f2bd931a700 1 -- 172.21.15.36:0/3610193003 >> 172.21.15.36:0/3610193003 conn(0x7f2bd40b5d10 msgr2=0x7f2bd40b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:16.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.868+0000 7f2bd931a700 1 -- 172.21.15.36:0/3610193003 shutdown_connections 2022-06-17T12:54:16.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.868+0000 7f2bd931a700 1 -- 172.21.15.36:0/3610193003 wait complete. 2022-06-17T12:54:16.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.868+0000 7f2bd931a700 1 Processor -- start 2022-06-17T12:54:16.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.868+0000 7f2bd931a700 1 -- start start 2022-06-17T12:54:16.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bd931a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bd4060920 0x7f2bd4062530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:16.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bd931a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2bd40b11a0 0x7f2bd413b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:16.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bd931a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2bd413fe30 0x7f2bd413bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:16.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bd931a700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd4062f20 con 0x7f2bd40b11a0 2022-06-17T12:54:16.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bd931a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd4063cc0 con 0x7f2bd4060920 2022-06-17T12:54:16.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bd931a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd4063f60 con 0x7f2bd413fe30 2022-06-17T12:54:16.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bcbfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2bd40b11a0 0x7f2bd413b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:16.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bcbfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2bd40b11a0 0x7f2bd413b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36504/0 (socket says 172.21.15.36:36504) 2022-06-17T12:54:16.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bcbfff700 1 -- 172.21.15.36:0/2877298021 learned_addr learned my addr 172.21.15.36:0/2877298021 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:16.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bd3fff700 1 --2- 172.21.15.36:0/2877298021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bd4060920 0x7f2bd4062530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:16.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bd8b19700 1 --2- 172.21.15.36:0/2877298021 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2bd413fe30 0x7f2bd413bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:16.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bcbfff700 1 -- 172.21.15.36:0/2877298021 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2bd413fe30 msgr2=0x7f2bd413bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bcbfff700 1 --2- 172.21.15.36:0/2877298021 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2bd413fe30 0x7f2bd413bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bcbfff700 1 -- 172.21.15.36:0/2877298021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bd4060920 msgr2=0x7f2bd4062530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bcbfff700 1 --2- 172.21.15.36:0/2877298021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bd4060920 0x7f2bd4062530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bcbfff700 1 -- 172.21.15.36:0/2877298021 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc00c310 con 0x7f2bd40b11a0 2022-06-17T12:54:16.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bcbfff700 1 --2- 172.21.15.36:0/2877298021 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2bd40b11a0 0x7f2bd413b830 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7f2bbc002f70 tx=0x7f2bd413cf10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:16.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bd1ffb700 1 -- 172.21.15.36:0/2877298021 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bbc020030 con 0x7f2bd40b11a0 2022-06-17T12:54:16.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bd1ffb700 1 -- 172.21.15.36:0/2877298021 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bbc01bb50 con 0x7f2bd40b11a0 2022-06-17T12:54:16.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bd1ffb700 1 -- 172.21.15.36:0/2877298021 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bbc01c440 con 0x7f2bd40b11a0 2022-06-17T12:54:16.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bd931a700 1 -- 172.21.15.36:0/2877298021 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd4064200 con 0x7f2bd40b11a0 2022-06-17T12:54:16.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bd931a700 1 -- 172.21.15.36:0/2877298021 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd4062a20 con 0x7f2bd40b11a0 2022-06-17T12:54:16.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bd931a700 1 -- 172.21.15.36:0/2877298021 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd40643c0 con 0x7f2bd40b11a0 2022-06-17T12:54:16.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bd1ffb700 1 -- 172.21.15.36:0/2877298021 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2bbc00e040 con 0x7f2bd40b11a0 2022-06-17T12:54:16.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.872+0000 7f2bd1ffb700 1 --2- 172.21.15.36:0/2877298021 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2bb8053a40 0x7f2bb8055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:16.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.876+0000 7f2bd1ffb700 1 -- 172.21.15.36:0/2877298021 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2bbc021070 con 0x7f2bd40b11a0 2022-06-17T12:54:16.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.876+0000 7f2bd3fff700 1 --2- 172.21.15.36:0/2877298021 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2bb8053a40 0x7f2bb8055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:16.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.876+0000 7f2bd3fff700 1 --2- 172.21.15.36:0/2877298021 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2bb8053a40 0x7f2bb8055f00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f2bc400b000 tx=0x7f2bc4006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:16.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:16.880+0000 7f2bd1ffb700 1 -- 172.21.15.36:0/2877298021 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2bbc048a70 con 0x7f2bd40b11a0 2022-06-17T12:54:16.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.004+0000 7f2bd931a700 1 -- 172.21.15.36:0/2877298021 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2bd40ac780 con 0x7f2bd40b11a0 2022-06-17T12:54:16.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.008+0000 7f2bd1ffb700 1 -- 172.21.15.36:0/2877298021 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+10 (secure 0 0 0) 0x7f2bbc048660 con 0x7f2bd40b11a0 2022-06-17T12:54:16.995 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:54:16.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.008+0000 7f2bd931a700 1 -- 172.21.15.36:0/2877298021 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2bb8053a40 msgr2=0x7f2bb8055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.008+0000 7f2bd931a700 1 --2- 172.21.15.36:0/2877298021 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2bb8053a40 0x7f2bb8055f00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f2bc400b000 tx=0x7f2bc4006df0).stop 2022-06-17T12:54:16.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.008+0000 7f2bd931a700 1 -- 172.21.15.36:0/2877298021 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2bd40b11a0 msgr2=0x7f2bd413b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:16.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.008+0000 7f2bd931a700 1 --2- 172.21.15.36:0/2877298021 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2bd40b11a0 0x7f2bd413b830 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7f2bbc002f70 tx=0x7f2bd413cf10).stop 2022-06-17T12:54:16.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.008+0000 7f2bd931a700 1 -- 172.21.15.36:0/2877298021 shutdown_connections 2022-06-17T12:54:16.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.008+0000 7f2bd931a700 1 --2- 172.21.15.36:0/2877298021 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2bb8053a40 0x7f2bb8055f00 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.008+0000 7f2bd931a700 1 --2- 172.21.15.36:0/2877298021 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2bd413fe30 0x7f2bd413bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.008+0000 7f2bd931a700 1 --2- 172.21.15.36:0/2877298021 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2bd40b11a0 0x7f2bd413b830 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:16.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.008+0000 7f2bd931a700 1 --2- 172.21.15.36:0/2877298021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bd4060920 0x7f2bd4062530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:17.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.008+0000 7f2bd931a700 1 -- 172.21.15.36:0/2877298021 >> 172.21.15.36:0/2877298021 conn(0x7f2bd40b5d10 msgr2=0x7f2bd40ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:17.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.008+0000 7f2bd931a700 1 -- 172.21.15.36:0/2877298021 shutdown_connections 2022-06-17T12:54:17.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.008+0000 7f2bd931a700 1 -- 172.21.15.36:0/2877298021 wait complete. 2022-06-17T12:54:17.007 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:54:17.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad23fff700 1 Processor -- start 2022-06-17T12:54:17.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad23fff700 1 -- start start 2022-06-17T12:54:17.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad23fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fad240adf10 0x7fad240b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:17.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad23fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fad240b15b0 0x7fad240af630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:17.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad23fff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad2413fdf0 con 0x7fad240b65f0 2022-06-17T12:54:17.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad23fff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad240ae8e0 con 0x7fad240b15b0 2022-06-17T12:54:17.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad23fff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad24140090 con 0x7fad240adf10 2022-06-17T12:54:17.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad237fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fad240b15b0 0x7fad240af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:17.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad237fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fad240b15b0 0x7fad240af630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36514/0 (socket says 172.21.15.36:36514) 2022-06-17T12:54:17.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad237fe700 1 -- 172.21.15.36:0/2009561801 learned_addr learned my addr 172.21.15.36:0/2009561801 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:17.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad1a7fc700 1 --2- 172.21.15.36:0/2009561801 >> v2:172.21.15.70:3300/0 conn(0x7fad240adf10 0x7fad240b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:17.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad237fe700 1 -- 172.21.15.36:0/2009561801 >> v2:172.21.15.70:3300/0 conn(0x7fad240adf10 msgr2=0x7fad240b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:17.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad237fe700 1 --2- 172.21.15.36:0/2009561801 >> v2:172.21.15.70:3300/0 conn(0x7fad240adf10 0x7fad240b69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:17.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad237fe700 1 -- 172.21.15.36:0/2009561801 >> v1:172.21.15.70:6789/0 conn(0x7fad240b65f0 legacy=0x7fad24140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:17.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad237fe700 1 -- 172.21.15.36:0/2009561801 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad240b1be0 con 0x7fad240b15b0 2022-06-17T12:54:17.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad237fe700 1 --2- 172.21.15.36:0/2009561801 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fad240b15b0 0x7fad240af630 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7fad1c0032f0 tx=0x7fad1c009290).ready entity=mon.0 client_cookie=39f77f07678c4d6d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:17.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad227fc700 1 -- 172.21.15.36:0/2009561801 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad1c00b870 con 0x7fad240b15b0 2022-06-17T12:54:17.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad227fc700 1 -- 172.21.15.36:0/2009561801 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1c00b9d0 con 0x7fad240b15b0 2022-06-17T12:54:17.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad227fc700 1 -- 172.21.15.36:0/2009561801 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad1c00bb40 con 0x7fad240b15b0 2022-06-17T12:54:17.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad23fff700 1 -- 172.21.15.36:0/2009561801 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fad240b15b0 msgr2=0x7fad240af630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:17.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad23fff700 1 --2- 172.21.15.36:0/2009561801 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fad240b15b0 0x7fad240af630 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7fad1c0032f0 tx=0x7fad1c009290).stop 2022-06-17T12:54:17.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad23fff700 1 -- 172.21.15.36:0/2009561801 shutdown_connections 2022-06-17T12:54:17.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad23fff700 1 --2- 172.21.15.36:0/2009561801 >> v2:172.21.15.70:3300/0 conn(0x7fad240adf10 0x7fad240b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:17.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad23fff700 1 --2- 172.21.15.36:0/2009561801 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fad240b15b0 0x7fad240af630 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:17.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad23fff700 1 -- 172.21.15.36:0/2009561801 >> 172.21.15.36:0/2009561801 conn(0x7fad240614b0 msgr2=0x7fad24061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:17.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad23fff700 1 -- 172.21.15.36:0/2009561801 shutdown_connections 2022-06-17T12:54:17.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.132+0000 7fad23fff700 1 -- 172.21.15.36:0/2009561801 wait complete. 2022-06-17T12:54:17.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.136+0000 7fad23fff700 1 Processor -- start 2022-06-17T12:54:17.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.136+0000 7fad23fff700 1 -- start start 2022-06-17T12:54:17.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.136+0000 7fad23fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad240adf10 0x7fad24138b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:17.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.136+0000 7fad23fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fad240b15b0 0x7fad2412ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:17.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.136+0000 7fad23fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad240b65f0 0x7fad2412f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:17.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.136+0000 7fad23fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad241434a0 con 0x7fad240b15b0 2022-06-17T12:54:17.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.136+0000 7fad23fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad24066b70 con 0x7fad240b65f0 2022-06-17T12:54:17.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.136+0000 7fad23fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad24143ca0 con 0x7fad240adf10 2022-06-17T12:54:17.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.136+0000 7fad1a7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fad240b15b0 0x7fad2412ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:17.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.136+0000 7fad1a7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fad240b15b0 0x7fad2412ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36518/0 (socket says 172.21.15.36:36518) 2022-06-17T12:54:17.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.136+0000 7fad1a7fc700 1 -- 172.21.15.36:0/1999715170 learned_addr learned my addr 172.21.15.36:0/1999715170 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:17.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.136+0000 7fad22ffd700 1 --2- 172.21.15.36:0/1999715170 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad240adf10 0x7fad24138b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:17.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.136+0000 7fad237fe700 1 --2- 172.21.15.36:0/1999715170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad240b65f0 0x7fad2412f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:17.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.136+0000 7fad1a7fc700 1 -- 172.21.15.36:0/1999715170 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad240adf10 msgr2=0x7fad24138b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:17.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.136+0000 7fad1a7fc700 1 --2- 172.21.15.36:0/1999715170 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad240adf10 0x7fad24138b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:17.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.136+0000 7fad1a7fc700 1 -- 172.21.15.36:0/1999715170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad240b65f0 msgr2=0x7fad2412f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:17.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.136+0000 7fad1a7fc700 1 --2- 172.21.15.36:0/1999715170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad240b65f0 0x7fad2412f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:17.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.136+0000 7fad1a7fc700 1 -- 172.21.15.36:0/1999715170 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c009530 con 0x7fad240b15b0 2022-06-17T12:54:17.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.136+0000 7fad1a7fc700 1 --2- 172.21.15.36:0/1999715170 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fad240b15b0 0x7fad2412ed60 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7fad0c00a000 tx=0x7fad0c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:17.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.136+0000 7fad20ff9700 1 -- 172.21.15.36:0/1999715170 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad0c0106e0 con 0x7fad240b15b0 2022-06-17T12:54:17.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.136+0000 7fad20ff9700 1 -- 172.21.15.36:0/1999715170 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0c010840 con 0x7fad240b15b0 2022-06-17T12:54:17.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.136+0000 7fad23fff700 1 -- 172.21.15.36:0/1999715170 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad24143160 con 0x7fad240b15b0 2022-06-17T12:54:17.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.136+0000 7fad20ff9700 1 -- 172.21.15.36:0/1999715170 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad0c010ba0 con 0x7fad240b15b0 2022-06-17T12:54:17.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.136+0000 7fad23fff700 1 -- 172.21.15.36:0/1999715170 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad240670d0 con 0x7fad240b15b0 2022-06-17T12:54:17.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.140+0000 7fad23fff700 1 -- 172.21.15.36:0/1999715170 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad2412f8e0 con 0x7fad240b15b0 2022-06-17T12:54:17.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.140+0000 7fad20ff9700 1 -- 172.21.15.36:0/1999715170 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fad0c01a030 con 0x7fad240b15b0 2022-06-17T12:54:17.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.140+0000 7fad20ff9700 1 --2- 172.21.15.36:0/1999715170 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7facfc053ad0 0x7facfc055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:17.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.140+0000 7fad20ff9700 1 -- 172.21.15.36:0/1999715170 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fad0c06bcd0 con 0x7fad240b15b0 2022-06-17T12:54:17.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.140+0000 7fad22ffd700 1 --2- 172.21.15.36:0/1999715170 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7facfc053ad0 0x7facfc055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:17.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.140+0000 7fad22ffd700 1 --2- 172.21.15.36:0/1999715170 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7facfc053ad0 0x7facfc055f90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fad1400d000 tx=0x7fad14009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:17.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.148+0000 7fad20ff9700 1 -- 172.21.15.36:0/1999715170 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fad0c042a70 con 0x7fad240b15b0 2022-06-17T12:54:17.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.268+0000 7fad23fff700 1 -- 172.21.15.36:0/1999715170 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fad2413fb20 con 0x7fad240b15b0 2022-06-17T12:54:17.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.268+0000 7fad20ff9700 1 -- 172.21.15.36:0/1999715170 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+10 (secure 0 0 0) 0x7fad0c046090 con 0x7fad240b15b0 2022-06-17T12:54:17.257 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:54:17.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.268+0000 7fad23fff700 1 -- 172.21.15.36:0/1999715170 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7facfc053ad0 msgr2=0x7facfc055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:17.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.268+0000 7fad23fff700 1 --2- 172.21.15.36:0/1999715170 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7facfc053ad0 0x7facfc055f90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fad1400d000 tx=0x7fad14009040).stop 2022-06-17T12:54:17.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.268+0000 7fad23fff700 1 -- 172.21.15.36:0/1999715170 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fad240b15b0 msgr2=0x7fad2412ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:17.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.268+0000 7fad23fff700 1 --2- 172.21.15.36:0/1999715170 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fad240b15b0 0x7fad2412ed60 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7fad0c00a000 tx=0x7fad0c008040).stop 2022-06-17T12:54:17.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.268+0000 7fad23fff700 1 -- 172.21.15.36:0/1999715170 shutdown_connections 2022-06-17T12:54:17.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.268+0000 7fad23fff700 1 --2- 172.21.15.36:0/1999715170 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7facfc053ad0 0x7facfc055f90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:17.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.268+0000 7fad23fff700 1 --2- 172.21.15.36:0/1999715170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad240b65f0 0x7fad2412f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:17.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.268+0000 7fad23fff700 1 --2- 172.21.15.36:0/1999715170 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fad240b15b0 0x7fad2412ed60 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:17.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.268+0000 7fad23fff700 1 --2- 172.21.15.36:0/1999715170 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad240adf10 0x7fad24138b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:17.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.268+0000 7fad23fff700 1 -- 172.21.15.36:0/1999715170 >> 172.21.15.36:0/1999715170 conn(0x7fad240614b0 msgr2=0x7fad24061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:17.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.268+0000 7fad23fff700 1 -- 172.21.15.36:0/1999715170 shutdown_connections 2022-06-17T12:54:17.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.268+0000 7fad23fff700 1 -- 172.21.15.36:0/1999715170 wait complete. 2022-06-17T12:54:17.275 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:54:17.276 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:54:17.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.392+0000 7f9ceb3c6700 1 Processor -- start 2022-06-17T12:54:17.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.392+0000 7f9ceb3c6700 1 -- start start 2022-06-17T12:54:17.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.392+0000 7f9ceb3c6700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9ce40b7b60 0x7f9ce40b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:17.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.392+0000 7f9ceb3c6700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9ce40616c0 0x7f9ce413cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:17.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.392+0000 7f9ceb3c6700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ce40b7f40 con 0x7f9ce4061eb0 2022-06-17T12:54:17.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.392+0000 7f9ceb3c6700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ce40b3670 con 0x7f9ce40b7b60 2022-06-17T12:54:17.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.392+0000 7f9ceb3c6700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ce41433b0 con 0x7f9ce40616c0 2022-06-17T12:54:17.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.392+0000 7f9ce9162700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9ce40b7b60 0x7f9ce40b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:17.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.392+0000 7f9ce9162700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9ce40b7b60 0x7f9ce40b3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36524/0 (socket says 172.21.15.36:36524) 2022-06-17T12:54:17.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.392+0000 7f9ce9162700 1 -- 172.21.15.36:0/462174676 learned_addr learned my addr 172.21.15.36:0/462174676 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:17.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.392+0000 7f9ce9963700 1 --2- 172.21.15.36:0/462174676 >> v2:172.21.15.70:3300/0 conn(0x7f9ce40616c0 0x7f9ce413cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:17.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.392+0000 7f9ce9162700 1 -- 172.21.15.36:0/462174676 >> v2:172.21.15.70:3300/0 conn(0x7f9ce40616c0 msgr2=0x7f9ce413cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:17.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.392+0000 7f9ce9162700 1 --2- 172.21.15.36:0/462174676 >> v2:172.21.15.70:3300/0 conn(0x7f9ce40616c0 0x7f9ce413cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:17.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.392+0000 7f9ce9162700 1 -- 172.21.15.36:0/462174676 >> v1:172.21.15.36:6789/0 conn(0x7f9ce4061eb0 legacy=0x7f9ce4143af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:17.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.392+0000 7f9ce9162700 1 -- 172.21.15.36:0/462174676 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce4143650 con 0x7f9ce40b7b60 2022-06-17T12:54:17.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.392+0000 7f9ce9963700 1 --2- 172.21.15.36:0/462174676 >> v2:172.21.15.70:3300/0 conn(0x7f9ce40616c0 0x7f9ce413cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:17.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.396+0000 7f9ce9162700 1 --2- 172.21.15.36:0/462174676 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9ce40b7b60 0x7f9ce40b3e50 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7f9ce0002f70 tx=0x7f9ce0006eb0).ready entity=mon.0 client_cookie=5bc5698b8c14498e server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:17.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.396+0000 7f9cdbfff700 1 -- 172.21.15.36:0/462174676 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ce000d610 con 0x7f9ce40b7b60 2022-06-17T12:54:17.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.396+0000 7f9cdbfff700 1 -- 172.21.15.36:0/462174676 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce0000e90 con 0x7f9ce40b7b60 2022-06-17T12:54:17.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.396+0000 7f9cdbfff700 1 -- 172.21.15.36:0/462174676 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ce000d9a0 con 0x7f9ce40b7b60 2022-06-17T12:54:17.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.396+0000 7f9ceb3c6700 1 -- 172.21.15.36:0/462174676 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9ce40b7b60 msgr2=0x7f9ce40b3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:17.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.396+0000 7f9ceb3c6700 1 --2- 172.21.15.36:0/462174676 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9ce40b7b60 0x7f9ce40b3e50 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7f9ce0002f70 tx=0x7f9ce0006eb0).stop 2022-06-17T12:54:17.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.396+0000 7f9ceb3c6700 1 -- 172.21.15.36:0/462174676 shutdown_connections 2022-06-17T12:54:17.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.396+0000 7f9ceb3c6700 1 --2- 172.21.15.36:0/462174676 >> v2:172.21.15.70:3300/0 conn(0x7f9ce40616c0 0x7f9ce413cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:17.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.396+0000 7f9ceb3c6700 1 --2- 172.21.15.36:0/462174676 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9ce40b7b60 0x7f9ce40b3e50 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:17.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.396+0000 7f9ceb3c6700 1 -- 172.21.15.36:0/462174676 >> 172.21.15.36:0/462174676 conn(0x7f9ce4060c60 msgr2=0x7f9ce405c550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:17.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.396+0000 7f9ceb3c6700 1 -- 172.21.15.36:0/462174676 shutdown_connections 2022-06-17T12:54:17.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.396+0000 7f9ceb3c6700 1 -- 172.21.15.36:0/462174676 wait complete. 2022-06-17T12:54:17.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9ceb3c6700 1 Processor -- start 2022-06-17T12:54:17.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9ceb3c6700 1 -- start start 2022-06-17T12:54:17.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9ceb3c6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ce40616c0 0x7f9ce4064e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:17.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9ceb3c6700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ce4061eb0 0x7f9ce4155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:17.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9ceb3c6700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9ce40b7b60 0x7f9ce4156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:17.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9ceb3c6700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce40655e0 con 0x7f9ce40b7b60 2022-06-17T12:54:17.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9ceb3c6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce413f190 con 0x7f9ce40616c0 2022-06-17T12:54:17.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9ceb3c6700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce413f430 con 0x7f9ce4061eb0 2022-06-17T12:54:17.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9ce9963700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9ce40b7b60 0x7f9ce4156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:17.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9ce9963700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9ce40b7b60 0x7f9ce4156000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36532/0 (socket says 172.21.15.36:36532) 2022-06-17T12:54:17.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9ce9963700 1 -- 172.21.15.36:0/1364264712 learned_addr learned my addr 172.21.15.36:0/1364264712 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:17.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9ce9162700 1 --2- 172.21.15.36:0/1364264712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ce40616c0 0x7f9ce4064e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:17.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9ce8961700 1 --2- 172.21.15.36:0/1364264712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ce4061eb0 0x7f9ce4155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:17.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9ce9963700 1 -- 172.21.15.36:0/1364264712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ce4061eb0 msgr2=0x7f9ce4155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:17.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9ce9963700 1 --2- 172.21.15.36:0/1364264712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ce4061eb0 0x7f9ce4155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:17.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9ce9963700 1 -- 172.21.15.36:0/1364264712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ce40616c0 msgr2=0x7f9ce4064e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:17.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9ce9963700 1 --2- 172.21.15.36:0/1364264712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ce40616c0 0x7f9ce4064e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:17.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9ce9963700 1 -- 172.21.15.36:0/1364264712 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce000b1e0 con 0x7f9ce40b7b60 2022-06-17T12:54:17.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9ce8961700 1 --2- 172.21.15.36:0/1364264712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ce4061eb0 0x7f9ce4155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:54:17.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9ce9963700 1 --2- 172.21.15.36:0/1364264712 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9ce40b7b60 0x7f9ce4156000 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f9cdc006cc0 tx=0x7f9cdc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:17.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9cda7fc700 1 -- 172.21.15.36:0/1364264712 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cdc00c770 con 0x7f9ce40b7b60 2022-06-17T12:54:17.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9ceb3c6700 1 -- 172.21.15.36:0/1364264712 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce413f6d0 con 0x7f9ce40b7b60 2022-06-17T12:54:17.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9cda7fc700 1 -- 172.21.15.36:0/1364264712 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cdc00c8d0 con 0x7f9ce40b7b60 2022-06-17T12:54:17.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9cda7fc700 1 -- 172.21.15.36:0/1364264712 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cdc00cc30 con 0x7f9ce40b7b60 2022-06-17T12:54:17.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9ceb3c6700 1 -- 172.21.15.36:0/1364264712 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce413ee30 con 0x7f9ce40b7b60 2022-06-17T12:54:17.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.400+0000 7f9ceb3c6700 1 -- 172.21.15.36:0/1364264712 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce413ff60 con 0x7f9ce40b7b60 2022-06-17T12:54:17.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.404+0000 7f9cda7fc700 1 -- 172.21.15.36:0/1364264712 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f9cdc013430 con 0x7f9ce40b7b60 2022-06-17T12:54:17.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.404+0000 7f9cda7fc700 1 --2- 172.21.15.36:0/1364264712 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9ccc053a40 0x7f9ccc055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:17.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.404+0000 7f9cda7fc700 1 -- 172.21.15.36:0/1364264712 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9cdc06c400 con 0x7f9ce40b7b60 2022-06-17T12:54:17.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.404+0000 7f9ce9162700 1 --2- 172.21.15.36:0/1364264712 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9ccc053a40 0x7f9ccc055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:17.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.404+0000 7f9ce9162700 1 --2- 172.21.15.36:0/1364264712 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9ccc053a40 0x7f9ccc055f00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f9ce001a000 tx=0x7f9ce0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:17.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.408+0000 7f9cda7fc700 1 -- 172.21.15.36:0/1364264712 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9ce413ff60 con 0x7f9ce40b7b60 2022-06-17T12:54:17.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.568+0000 7f9ceb3c6700 1 -- 172.21.15.36:0/1364264712 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ce413ff60 con 0x7f9ce40b7b60 2022-06-17T12:54:17.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.568+0000 7f9cda7fc700 1 -- 172.21.15.36:0/1364264712 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f9ce413ff60 con 0x7f9ce40b7b60 2022-06-17T12:54:17.557 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:17.557 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":42,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/3068692336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3068692336},{"type":"v1","addr":"172.21.15.70:6833","nonce":3068692336}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:54:12.740483+0000","modified":"2022-06-17T12:54:13.744294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6753},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6753":{"gid":6753,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/2728662009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2728662009},{"type":"v1","addr":"172.21.15.36:6835","nonce":2728662009}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-06-17T12:54:17.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.572+0000 7f9ceb3c6700 1 -- 172.21.15.36:0/1364264712 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9ccc053a40 msgr2=0x7f9ccc055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:17.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.572+0000 7f9ceb3c6700 1 --2- 172.21.15.36:0/1364264712 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9ccc053a40 0x7f9ccc055f00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f9ce001a000 tx=0x7f9ce0015000).stop 2022-06-17T12:54:17.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.572+0000 7f9ceb3c6700 1 -- 172.21.15.36:0/1364264712 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9ce40b7b60 msgr2=0x7f9ce4156000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:17.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.572+0000 7f9ceb3c6700 1 --2- 172.21.15.36:0/1364264712 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9ce40b7b60 0x7f9ce4156000 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f9cdc006cc0 tx=0x7f9cdc008040).stop 2022-06-17T12:54:17.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.572+0000 7f9ceb3c6700 1 -- 172.21.15.36:0/1364264712 shutdown_connections 2022-06-17T12:54:17.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.572+0000 7f9ceb3c6700 1 --2- 172.21.15.36:0/1364264712 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9ccc053a40 0x7f9ccc055f00 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:17.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.572+0000 7f9ceb3c6700 1 --2- 172.21.15.36:0/1364264712 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9ce40b7b60 0x7f9ce4156000 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:17.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.572+0000 7f9ceb3c6700 1 --2- 172.21.15.36:0/1364264712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ce4061eb0 0x7f9ce4155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:17.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.572+0000 7f9ceb3c6700 1 --2- 172.21.15.36:0/1364264712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ce40616c0 0x7f9ce4064e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:17.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.572+0000 7f9ceb3c6700 1 -- 172.21.15.36:0/1364264712 >> 172.21.15.36:0/1364264712 conn(0x7f9ce4060c60 msgr2=0x7f9ce4140ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:17.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.572+0000 7f9ceb3c6700 1 -- 172.21.15.36:0/1364264712 shutdown_connections 2022-06-17T12:54:17.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:17.572+0000 7f9ceb3c6700 1 -- 172.21.15.36:0/1364264712 wait complete. 2022-06-17T12:54:17.562 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 42 2022-06-17T12:54:17.772 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:54:12.740483+0000', 'modified': '2022-06-17T12:54:13.744294+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6753}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6753': {'gid': 6753, 'name': 'a', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.36:6835/2728662009', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 2728662009}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 2728662009}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:54:17.773 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:54:17.774 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:54:17.894 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":6753,"want_state":"up:active","state":"up:active","rank_uptime":5.1558061210000004,"mdsmap_epoch":42,"osdmap_epoch":84,"osdmap_epoch_barrier":83,"uptime":9.5561174819999994} 2022-06-17T12:54:17.895 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 6753, "mdsmap_epoch": 42, "osdmap_epoch": 84, "osdmap_epoch_barrier": 83, "rank_uptime": 5.155806121, "state": "up:active", "uptime": 9.556117482, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:54:17.896 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:54:17.896 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-17T12:54:17.896 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi107.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-17T12:54:17.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:17.897 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:54:17.907 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:54:17.926 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:54:17.927 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:54:17.938 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:54:17.939 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:54:17.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:17.942 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:54:17.990 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:54:17.991 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:54:17.994 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:54:18.011 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:54:18.011 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:54:18.023 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:54:18.025 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:54:18.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:18.026 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:54:18.071 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:2022-06-17T12:54:18.083+0000 7f4da1da1080 -1 init, newargv = 0x5598373f9420 newargc=9 2022-06-17T12:54:18.071 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22516]: starting ceph client 2022-06-17T12:54:18.080 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22516]: starting fuse 2022-06-17T12:54:19.178 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:54:19.198 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:54:19.199 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:54:19.256 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:54:19.256 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:54:19.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:19.258 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:54:19.265 INFO:teuthology.orchestra.run.smithi107.stdout:52 2022-06-17T12:54:19.266 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:54:19.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:19.268 DEBUG:teuthology.orchestra.run.smithi107:> sudo python3 -c ' 2022-06-17T12:54:19.268 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2022-06-17T12:54:19.269 DEBUG:teuthology.orchestra.run.smithi107:> import re 2022-06-17T12:54:19.270 DEBUG:teuthology.orchestra.run.smithi107:> import os 2022-06-17T12:54:19.270 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2022-06-17T12:54:19.271 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:54:19.271 DEBUG:teuthology.orchestra.run.smithi107:> def find_socket(client_name): 2022-06-17T12:54:19.272 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-17T12:54:19.273 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2022-06-17T12:54:19.274 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:54:19.275 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2022-06-17T12:54:19.275 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2022-06-17T12:54:19.276 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2022-06-17T12:54:19.277 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2022-06-17T12:54:19.279 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:54:19.280 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2022-06-17T12:54:19.280 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:54:19.281 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:54:19.282 DEBUG:teuthology.orchestra.run.smithi107:> return f 2022-06-17T12:54:19.283 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:54:19.284 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:54:19.284 DEBUG:teuthology.orchestra.run.smithi107:> print(find_socket("client.0")) 2022-06-17T12:54:19.286 DEBUG:teuthology.orchestra.run.smithi107:> ' 2022-06-17T12:54:19.330 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.0.22516.asok 2022-06-17T12:54:19.335 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22516.asok 2022-06-17T12:54:19.336 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:19.336 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22516.asok status 2022-06-17T12:54:19.464 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2022-06-17T12:54:19.464 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2022-06-17T12:54:19.464 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:54:19.464 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:54:19.465 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "0", 2022-06-17T12:54:19.465 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2022-06-17T12:54:19.465 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-17T12:54:19.465 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "22516", 2022-06-17T12:54:19.466 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/" 2022-06-17T12:54:19.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:54:19.466 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2022-06-17T12:54:19.466 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2022-06-17T12:54:19.467 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 6921, 2022-06-17T12:54:19.467 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2022-06-17T12:54:19.467 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2022-06-17T12:54:19.468 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2022-06-17T12:54:19.468 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 6921 2022-06-17T12:54:19.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:54:19.468 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:54:19.468 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:54:19.469 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:54:19.469 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3204152170 2022-06-17T12:54:19.469 INFO:teuthology.orchestra.run.smithi107.stdout: } 2022-06-17T12:54:19.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:54:19.470 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:54:19.470 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:54:19.470 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:54:19.470 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3204152170 2022-06-17T12:54:19.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:54:19.471 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.6921 172.21.15.107:0/3204152170", 2022-06-17T12:54:19.471 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "172.21.15.107:0/3204152170", 2022-06-17T12:54:19.471 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 1, 2022-06-17T12:54:19.471 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 42, 2022-06-17T12:54:19.472 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 84, 2022-06-17T12:54:19.472 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:54:19.472 INFO:teuthology.orchestra.run.smithi107.stdout: "blacklisted": false, 2022-06-17T12:54:19.472 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2022-06-17T12:54:19.473 INFO:teuthology.orchestra.run.smithi107.stdout:} 2022-06-17T12:54:19.474 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:19.475 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:19.475 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:54:19.483 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:54:19.484 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:54:19.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:19.486 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:54:19.503 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:54:19.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.628+0000 7f3fcea03700 1 Processor -- start 2022-06-17T12:54:19.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.628+0000 7f3fcea03700 1 -- start start 2022-06-17T12:54:19.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.628+0000 7f3fcea03700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f3fc8063730 0x7f3fc8065e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:19.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.628+0000 7f3fcea03700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc8063b10 con 0x7f3fc80b1930 2022-06-17T12:54:19.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.628+0000 7f3fcea03700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc8065430 con 0x7f3fc80b2900 2022-06-17T12:54:19.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.628+0000 7f3fcea03700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fc80656d0 con 0x7f3fc8063730 2022-06-17T12:54:19.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.628+0000 7f3fcda01700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f3fc8063730 0x7f3fc8065e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:19.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.628+0000 7f3fcda01700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f3fc8063730 0x7f3fc8065e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36538/0 (socket says 172.21.15.36:36538) 2022-06-17T12:54:19.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.628+0000 7f3fcda01700 1 -- 172.21.15.36:0/1359165547 learned_addr learned my addr 172.21.15.36:0/1359165547 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:19.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.628+0000 7f3fcd200700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f3fc80b1930 0x7f3fc81431f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54072/0 (socket says 172.21.15.36:54072) 2022-06-17T12:54:19.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.628+0000 7f3fcda01700 1 -- 172.21.15.36:0/1359165547 >> v1:172.21.15.70:6789/0 conn(0x7f3fc80b2900 legacy=0x7f3fc813eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:19.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.628+0000 7f3fcda01700 1 -- 172.21.15.36:0/1359165547 >> v1:172.21.15.36:6789/0 conn(0x7f3fc80b1930 legacy=0x7f3fc81431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:19.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.628+0000 7f3fcda01700 1 -- 172.21.15.36:0/1359165547 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc8065970 con 0x7f3fc8063730 2022-06-17T12:54:19.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.628+0000 7f3fcc9ff700 1 -- 172.21.15.36:0/1359165547 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2316881351 0 0) 0x7f3fc8063b10 con 0x7f3fc80b1930 2022-06-17T12:54:19.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.628+0000 7f3fcda01700 1 --2- 172.21.15.36:0/1359165547 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3fc8063730 0x7f3fc8065e10 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7f3fc4002f70 tx=0x7f3fc4006eb0).ready entity=mon.0 client_cookie=9ad3a9dc911d5998 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:19.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.628+0000 7f3fcc9ff700 1 -- 172.21.15.36:0/1359165547 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fc400d660 con 0x7f3fc8063730 2022-06-17T12:54:19.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.628+0000 7f3fcc9ff700 1 -- 172.21.15.36:0/1359165547 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc4000e90 con 0x7f3fc8063730 2022-06-17T12:54:19.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.628+0000 7f3fcc9ff700 1 -- 172.21.15.36:0/1359165547 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fc400d9b0 con 0x7f3fc8063730 2022-06-17T12:54:19.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.628+0000 7f3fcea03700 1 -- 172.21.15.36:0/1359165547 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3fc8063730 msgr2=0x7f3fc8065e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:19.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.628+0000 7f3fcea03700 1 --2- 172.21.15.36:0/1359165547 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3fc8063730 0x7f3fc8065e10 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7f3fc4002f70 tx=0x7f3fc4006eb0).stop 2022-06-17T12:54:19.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.632+0000 7f3fcea03700 1 -- 172.21.15.36:0/1359165547 shutdown_connections 2022-06-17T12:54:19.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.632+0000 7f3fcea03700 1 --2- 172.21.15.36:0/1359165547 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3fc8063730 0x7f3fc8065e10 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:19.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.632+0000 7f3fcea03700 1 -- 172.21.15.36:0/1359165547 >> 172.21.15.36:0/1359165547 conn(0x7f3fc80614b0 msgr2=0x7f3fc80618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:19.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.632+0000 7f3fcea03700 1 -- 172.21.15.36:0/1359165547 shutdown_connections 2022-06-17T12:54:19.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.632+0000 7f3fcea03700 1 -- 172.21.15.36:0/1359165547 wait complete. 2022-06-17T12:54:19.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.632+0000 7f3fcea03700 1 Processor -- start 2022-06-17T12:54:19.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.632+0000 7f3fcea03700 1 -- start start 2022-06-17T12:54:19.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.632+0000 7f3fcea03700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3fc8063730 0x7f3fc80b5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:19.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.632+0000 7f3fcea03700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3fc80b1930 0x7f3fc81389f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:19.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.632+0000 7f3fcea03700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fc80b2900 0x7f3fc80ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:19.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.632+0000 7f3fcea03700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc80b04a0 con 0x7f3fc80b1930 2022-06-17T12:54:19.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.632+0000 7f3fcea03700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc80b0740 con 0x7f3fc80b2900 2022-06-17T12:54:19.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.632+0000 7f3fcea03700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc80b09e0 con 0x7f3fc8063730 2022-06-17T12:54:19.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.632+0000 7f3fcd200700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3fc80b1930 0x7f3fc81389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:19.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.632+0000 7f3fcd200700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3fc80b1930 0x7f3fc81389f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36546/0 (socket says 172.21.15.36:36546) 2022-06-17T12:54:19.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.632+0000 7f3fcd200700 1 -- 172.21.15.36:0/3223953543 learned_addr learned my addr 172.21.15.36:0/3223953543 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:19.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.632+0000 7f3fce202700 1 --2- 172.21.15.36:0/3223953543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fc80b2900 0x7f3fc80ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:19.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.632+0000 7f3fcda01700 1 --2- 172.21.15.36:0/3223953543 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3fc8063730 0x7f3fc80b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:19.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.632+0000 7f3fcd200700 1 -- 172.21.15.36:0/3223953543 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3fc8063730 msgr2=0x7f3fc80b5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:19.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.632+0000 7f3fcd200700 1 --2- 172.21.15.36:0/3223953543 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3fc8063730 0x7f3fc80b5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:19.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.632+0000 7f3fcd200700 1 -- 172.21.15.36:0/3223953543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fc80b2900 msgr2=0x7f3fc80ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:19.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.632+0000 7f3fcd200700 1 --2- 172.21.15.36:0/3223953543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fc80b2900 0x7f3fc80ae2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:19.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.632+0000 7f3fcd200700 1 -- 172.21.15.36:0/3223953543 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc400b1e0 con 0x7f3fc80b1930 2022-06-17T12:54:19.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.636+0000 7f3fcd200700 1 --2- 172.21.15.36:0/3223953543 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3fc80b1930 0x7f3fc81389f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7f3fb8000bc0 tx=0x7f3fb8007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:19.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.636+0000 7f3fb6ffd700 1 -- 172.21.15.36:0/3223953543 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fb80070d0 con 0x7f3fc80b1930 2022-06-17T12:54:19.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.636+0000 7f3fb6ffd700 1 -- 172.21.15.36:0/3223953543 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb8007230 con 0x7f3fc80b1930 2022-06-17T12:54:19.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.636+0000 7f3fcea03700 1 -- 172.21.15.36:0/3223953543 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc8138ee0 con 0x7f3fc80b1930 2022-06-17T12:54:19.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.636+0000 7f3fb6ffd700 1 -- 172.21.15.36:0/3223953543 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fb8005740 con 0x7f3fc80b1930 2022-06-17T12:54:19.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.636+0000 7f3fcea03700 1 -- 172.21.15.36:0/3223953543 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc8133450 con 0x7f3fc80b1930 2022-06-17T12:54:19.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.636+0000 7f3fcea03700 1 -- 172.21.15.36:0/3223953543 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc81387f0 con 0x7f3fc80b1930 2022-06-17T12:54:19.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.636+0000 7f3fb6ffd700 1 -- 172.21.15.36:0/3223953543 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f3fb801a060 con 0x7f3fc80b1930 2022-06-17T12:54:19.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.636+0000 7f3fb6ffd700 1 --2- 172.21.15.36:0/3223953543 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3fa8053ad0 0x7f3fa8055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:19.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.636+0000 7f3fb6ffd700 1 -- 172.21.15.36:0/3223953543 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3fb806b3d0 con 0x7f3fc80b1930 2022-06-17T12:54:19.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.636+0000 7f3fcda01700 1 --2- 172.21.15.36:0/3223953543 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3fa8053ad0 0x7f3fa8055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:19.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.640+0000 7f3fcda01700 1 --2- 172.21.15.36:0/3223953543 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3fa8053ad0 0x7f3fa8055f90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f3fc401a000 tx=0x7f3fc4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:19.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.640+0000 7f3fb6ffd700 1 -- 172.21.15.36:0/3223953543 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3fb80421d0 con 0x7f3fc80b1930 2022-06-17T12:54:19.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.768+0000 7f3fcea03700 1 -- 172.21.15.36:0/3223953543 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3fc80ae0a0 con 0x7f3fc80b1930 2022-06-17T12:54:19.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.772+0000 7f3fb6ffd700 1 -- 172.21.15.36:0/3223953543 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f3fb8018020 con 0x7f3fc80b1930 2022-06-17T12:54:19.762 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:19.763 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":84,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:54:13.745105+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:54:11.237500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-06-17T12:54:11.985387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:54:19.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.776+0000 7f3fcea03700 1 -- 172.21.15.36:0/3223953543 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3fa8053ad0 msgr2=0x7f3fa8055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:19.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.776+0000 7f3fcea03700 1 --2- 172.21.15.36:0/3223953543 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3fa8053ad0 0x7f3fa8055f90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f3fc401a000 tx=0x7f3fc4015000).stop 2022-06-17T12:54:19.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.776+0000 7f3fcea03700 1 -- 172.21.15.36:0/3223953543 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3fc80b1930 msgr2=0x7f3fc81389f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:19.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.776+0000 7f3fcea03700 1 --2- 172.21.15.36:0/3223953543 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3fc80b1930 0x7f3fc81389f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7f3fb8000bc0 tx=0x7f3fb8007a60).stop 2022-06-17T12:54:19.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.776+0000 7f3fcea03700 1 -- 172.21.15.36:0/3223953543 shutdown_connections 2022-06-17T12:54:19.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.776+0000 7f3fcea03700 1 --2- 172.21.15.36:0/3223953543 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3fa8053ad0 0x7f3fa8055f90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:19.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.776+0000 7f3fcea03700 1 --2- 172.21.15.36:0/3223953543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fc80b2900 0x7f3fc80ae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:19.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.776+0000 7f3fcea03700 1 --2- 172.21.15.36:0/3223953543 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3fc80b1930 0x7f3fc81389f0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:19.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.776+0000 7f3fcea03700 1 --2- 172.21.15.36:0/3223953543 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3fc8063730 0x7f3fc80b5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:19.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.776+0000 7f3fcea03700 1 -- 172.21.15.36:0/3223953543 >> 172.21.15.36:0/3223953543 conn(0x7f3fc80614b0 msgr2=0x7f3fc80b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:19.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.776+0000 7f3fcea03700 1 -- 172.21.15.36:0/3223953543 shutdown_connections 2022-06-17T12:54:19.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.776+0000 7f3fcea03700 1 -- 172.21.15.36:0/3223953543 wait complete. 2022-06-17T12:54:19.775 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:54:11.237500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:54:11.985387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:54:19.777 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:54:19.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.904+0000 7f269b51c700 1 Processor -- start 2022-06-17T12:54:19.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.904+0000 7f269b51c700 1 -- start start 2022-06-17T12:54:19.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.908+0000 7f269b51c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2694060ac0 0x7f2694140260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:19.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.908+0000 7f269b51c700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2694060ea0 con 0x7f26940b2c80 2022-06-17T12:54:19.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.908+0000 7f269b51c700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26940aebb0 con 0x7f2694067260 2022-06-17T12:54:19.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.908+0000 7f269b51c700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26940b2210 con 0x7f2694060ac0 2022-06-17T12:54:19.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.908+0000 7f269a51a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2694060ac0 0x7f2694140260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:19.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.908+0000 7f2699d19700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f26940b2c80 0x7f26940aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54082/0 (socket says 172.21.15.36:54082) 2022-06-17T12:54:19.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.908+0000 7f2699d19700 1 -- 172.21.15.36:0/2920540642 learned_addr learned my addr 172.21.15.36:0/2920540642 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:19.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.908+0000 7f2699518700 1 -- 172.21.15.36:0/2920540642 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3812907400 0 0) 0x7f2694060ea0 con 0x7f26940b2c80 2022-06-17T12:54:19.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.908+0000 7f2699518700 1 -- 172.21.15.36:0/2920540642 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f267c003260 con 0x7f26940b2c80 2022-06-17T12:54:19.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.908+0000 7f269a51a700 1 -- 172.21.15.36:0/2920540642 >> v1:172.21.15.70:6789/0 conn(0x7f2694067260 legacy=0x7f26940af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:19.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.908+0000 7f269a51a700 1 -- 172.21.15.36:0/2920540642 >> v1:172.21.15.36:6789/0 conn(0x7f26940b2c80 legacy=0x7f26940aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:19.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.908+0000 7f269a51a700 1 -- 172.21.15.36:0/2920540642 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26940b1ad0 con 0x7f2694060ac0 2022-06-17T12:54:19.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.908+0000 7f269a51a700 1 --2- 172.21.15.36:0/2920540642 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2694060ac0 0x7f2694140260 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7f2690002f70 tx=0x7f2690006df0).ready entity=mon.0 client_cookie=254a84e0e989f3f1 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:19.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.908+0000 7f2699518700 1 -- 172.21.15.36:0/2920540642 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f269000d610 con 0x7f2694060ac0 2022-06-17T12:54:19.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.908+0000 7f2699518700 1 -- 172.21.15.36:0/2920540642 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2690000e90 con 0x7f2694060ac0 2022-06-17T12:54:19.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.908+0000 7f2699518700 1 -- 172.21.15.36:0/2920540642 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f269000d960 con 0x7f2694060ac0 2022-06-17T12:54:19.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.908+0000 7f269b51c700 1 -- 172.21.15.36:0/2920540642 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2694060ac0 msgr2=0x7f2694140260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:19.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.908+0000 7f269b51c700 1 --2- 172.21.15.36:0/2920540642 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2694060ac0 0x7f2694140260 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7f2690002f70 tx=0x7f2690006df0).stop 2022-06-17T12:54:19.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.908+0000 7f269b51c700 1 -- 172.21.15.36:0/2920540642 shutdown_connections 2022-06-17T12:54:19.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.908+0000 7f269b51c700 1 --2- 172.21.15.36:0/2920540642 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2694060ac0 0x7f2694140260 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:19.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.908+0000 7f269b51c700 1 -- 172.21.15.36:0/2920540642 >> 172.21.15.36:0/2920540642 conn(0x7f26940b64f0 msgr2=0x7f26940b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:19.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.908+0000 7f269b51c700 1 -- 172.21.15.36:0/2920540642 shutdown_connections 2022-06-17T12:54:19.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.908+0000 7f269b51c700 1 -- 172.21.15.36:0/2920540642 wait complete. 2022-06-17T12:54:19.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.912+0000 7f269b51c700 1 Processor -- start 2022-06-17T12:54:19.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.912+0000 7f269b51c700 1 -- start start 2022-06-17T12:54:19.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.912+0000 7f269b51c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2694060ac0 0x7f269413bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:19.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.912+0000 7f269b51c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2694067260 0x7f269413bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:19.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.912+0000 7f269b51c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26940b2c80 0x7f26941475a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:19.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.912+0000 7f269b51c700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26941439f0 con 0x7f2694060ac0 2022-06-17T12:54:19.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.912+0000 7f269b51c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2694152e30 con 0x7f26940b2c80 2022-06-17T12:54:19.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.912+0000 7f269b51c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26941530d0 con 0x7f2694067260 2022-06-17T12:54:19.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.912+0000 7f269a51a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2694060ac0 0x7f269413bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:19.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.912+0000 7f2699d19700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2694067260 0x7f269413bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:19.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.912+0000 7f269a51a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2694060ac0 0x7f269413bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36560/0 (socket says 172.21.15.36:36560) 2022-06-17T12:54:19.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.912+0000 7f269a51a700 1 -- 172.21.15.36:0/1106562298 learned_addr learned my addr 172.21.15.36:0/1106562298 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:19.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.912+0000 7f269ad1b700 1 --2- 172.21.15.36:0/1106562298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26940b2c80 0x7f26941475a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:19.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.912+0000 7f269a51a700 1 -- 172.21.15.36:0/1106562298 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2694067260 msgr2=0x7f269413bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:19.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.912+0000 7f269a51a700 1 --2- 172.21.15.36:0/1106562298 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2694067260 0x7f269413bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:19.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.912+0000 7f269a51a700 1 -- 172.21.15.36:0/1106562298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26940b2c80 msgr2=0x7f26941475a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:19.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.912+0000 7f269a51a700 1 --2- 172.21.15.36:0/1106562298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26940b2c80 0x7f26941475a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:19.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.912+0000 7f269a51a700 1 -- 172.21.15.36:0/1106562298 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269000b180 con 0x7f2694060ac0 2022-06-17T12:54:19.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.912+0000 7f269a51a700 1 --2- 172.21.15.36:0/1106562298 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2694060ac0 0x7f269413bac0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7f2690002f70 tx=0x7f2690014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:19.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.912+0000 7f26877fe700 1 -- 172.21.15.36:0/1106562298 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2690013de0 con 0x7f2694060ac0 2022-06-17T12:54:19.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.912+0000 7f26877fe700 1 -- 172.21.15.36:0/1106562298 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269000d290 con 0x7f2694060ac0 2022-06-17T12:54:19.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.912+0000 7f269b51c700 1 -- 172.21.15.36:0/1106562298 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2694142bc0 con 0x7f2694060ac0 2022-06-17T12:54:19.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.912+0000 7f26877fe700 1 -- 172.21.15.36:0/1106562298 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f269001e5f0 con 0x7f2694060ac0 2022-06-17T12:54:19.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.912+0000 7f269b51c700 1 -- 172.21.15.36:0/1106562298 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2694143de0 con 0x7f2694060ac0 2022-06-17T12:54:19.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.916+0000 7f269b51c700 1 -- 172.21.15.36:0/1106562298 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2694152ca0 con 0x7f2694060ac0 2022-06-17T12:54:19.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.916+0000 7f26877fe700 1 -- 172.21.15.36:0/1106562298 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2690025030 con 0x7f2694060ac0 2022-06-17T12:54:19.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.916+0000 7f26877fe700 1 --2- 172.21.15.36:0/1106562298 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26740539b0 0x7f2674055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:19.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.916+0000 7f26877fe700 1 -- 172.21.15.36:0/1106562298 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f269006ccf0 con 0x7f2694060ac0 2022-06-17T12:54:19.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.916+0000 7f2699d19700 1 --2- 172.21.15.36:0/1106562298 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26740539b0 0x7f2674055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:19.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.916+0000 7f2699d19700 1 --2- 172.21.15.36:0/1106562298 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26740539b0 0x7f2674055e70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f268800d000 tx=0x7f2688009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:19.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:19.924+0000 7f26877fe700 1 -- 172.21.15.36:0/1106562298 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2690043af0 con 0x7f2694060ac0 2022-06-17T12:54:20.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.044+0000 7f269b51c700 1 -- 172.21.15.36:0/1106562298 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f2694153290 con 0x7f2694060ac0 2022-06-17T12:54:20.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.048+0000 7f26877fe700 1 -- 172.21.15.36:0/1106562298 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v84) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2690023070 con 0x7f2694060ac0 2022-06-17T12:54:20.034 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:54:20.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.048+0000 7f269b51c700 1 -- 172.21.15.36:0/1106562298 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26740539b0 msgr2=0x7f2674055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.048+0000 7f269b51c700 1 --2- 172.21.15.36:0/1106562298 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26740539b0 0x7f2674055e70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f268800d000 tx=0x7f2688009040).stop 2022-06-17T12:54:20.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.048+0000 7f269b51c700 1 -- 172.21.15.36:0/1106562298 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2694060ac0 msgr2=0x7f269413bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.048+0000 7f269b51c700 1 --2- 172.21.15.36:0/1106562298 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2694060ac0 0x7f269413bac0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7f2690002f70 tx=0x7f2690014000).stop 2022-06-17T12:54:20.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.048+0000 7f269b51c700 1 -- 172.21.15.36:0/1106562298 shutdown_connections 2022-06-17T12:54:20.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.048+0000 7f269b51c700 1 --2- 172.21.15.36:0/1106562298 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26740539b0 0x7f2674055e70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.048+0000 7f269b51c700 1 --2- 172.21.15.36:0/1106562298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26940b2c80 0x7f26941475a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.048+0000 7f269b51c700 1 --2- 172.21.15.36:0/1106562298 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2694067260 0x7f269413bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.048+0000 7f269b51c700 1 --2- 172.21.15.36:0/1106562298 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2694060ac0 0x7f269413bac0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.048+0000 7f269b51c700 1 -- 172.21.15.36:0/1106562298 >> 172.21.15.36:0/1106562298 conn(0x7f26940b64f0 msgr2=0x7f26940b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:20.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.048+0000 7f269b51c700 1 -- 172.21.15.36:0/1106562298 shutdown_connections 2022-06-17T12:54:20.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.048+0000 7f269b51c700 1 -- 172.21.15.36:0/1106562298 wait complete. 2022-06-17T12:54:20.048 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:54:20.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.168+0000 7ff8f5cb5700 1 Processor -- start 2022-06-17T12:54:20.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.168+0000 7ff8f5cb5700 1 -- start start 2022-06-17T12:54:20.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8f5cb5700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff8f00b27b0 0x7ff8f00b1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8f5cb5700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f00ac0b0 con 0x7ff8f0140940 2022-06-17T12:54:20.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8f5cb5700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f00b2b90 con 0x7ff8f0144d20 2022-06-17T12:54:20.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8f5cb5700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f0140d90 con 0x7ff8f00b27b0 2022-06-17T12:54:20.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8f54b4700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7ff8f0140940 0x7ff8f0141180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54100/0 (socket says 172.21.15.36:54100) 2022-06-17T12:54:20.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8f54b4700 1 -- 172.21.15.36:0/1844827764 learned_addr learned my addr 172.21.15.36:0/1844827764 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:20.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8f4cb3700 1 --2- 172.21.15.36:0/1844827764 >> v2:172.21.15.36:3300/0 conn(0x7ff8f00b27b0 0x7ff8f00b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8ef7fe700 1 -- 172.21.15.36:0/1844827764 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1872844981 0 0) 0x7ff8f00ac0b0 con 0x7ff8f0140940 2022-06-17T12:54:20.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8ef7fe700 1 -- 172.21.15.36:0/1844827764 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8dc003260 con 0x7ff8f0140940 2022-06-17T12:54:20.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8ef7fe700 1 -- 172.21.15.36:0/1844827764 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3121036529 0 0) 0x7ff8f00b2b90 con 0x7ff8f0144d20 2022-06-17T12:54:20.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8ef7fe700 1 -- 172.21.15.36:0/1844827764 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8f00ac0b0 con 0x7ff8f0144d20 2022-06-17T12:54:20.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8ef7fe700 1 -- 172.21.15.36:0/1844827764 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1338170921 0 0) 0x7ff8dc003260 con 0x7ff8f0140940 2022-06-17T12:54:20.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8f4cb3700 1 -- 172.21.15.36:0/1844827764 >> v1:172.21.15.70:6789/0 conn(0x7ff8f0144d20 legacy=0x7ff8f0140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8f4cb3700 1 -- 172.21.15.36:0/1844827764 >> v1:172.21.15.36:6789/0 conn(0x7ff8f0140940 legacy=0x7ff8f0141180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8f4cb3700 1 -- 172.21.15.36:0/1844827764 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f00afc60 con 0x7ff8f00b27b0 2022-06-17T12:54:20.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8f4cb3700 1 --2- 172.21.15.36:0/1844827764 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8f00b27b0 0x7ff8f00b1830 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7ff8e8001b80 tx=0x7ff8e4006df0).ready entity=mon.0 client_cookie=5ce27ec240579199 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:20.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8ef7fe700 1 -- 172.21.15.36:0/1844827764 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8e80019e0 con 0x7ff8f00b27b0 2022-06-17T12:54:20.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8ef7fe700 1 -- 172.21.15.36:0/1844827764 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e4000e90 con 0x7ff8f00b27b0 2022-06-17T12:54:20.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8ef7fe700 1 -- 172.21.15.36:0/1844827764 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8e4012460 con 0x7ff8f00b27b0 2022-06-17T12:54:20.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8f5cb5700 1 -- 172.21.15.36:0/1844827764 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8f00b27b0 msgr2=0x7ff8f00b1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8f5cb5700 1 --2- 172.21.15.36:0/1844827764 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8f00b27b0 0x7ff8f00b1830 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7ff8e8001b80 tx=0x7ff8e4006df0).stop 2022-06-17T12:54:20.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8f5cb5700 1 -- 172.21.15.36:0/1844827764 shutdown_connections 2022-06-17T12:54:20.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8f5cb5700 1 --2- 172.21.15.36:0/1844827764 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8f00b27b0 0x7ff8f00b1830 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8f5cb5700 1 -- 172.21.15.36:0/1844827764 >> 172.21.15.36:0/1844827764 conn(0x7ff8f00b5d10 msgr2=0x7ff8f00b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:20.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8f5cb5700 1 -- 172.21.15.36:0/1844827764 shutdown_connections 2022-06-17T12:54:20.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.172+0000 7ff8f5cb5700 1 -- 172.21.15.36:0/1844827764 wait complete. 2022-06-17T12:54:20.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8f5cb5700 1 Processor -- start 2022-06-17T12:54:20.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8f5cb5700 1 -- start start 2022-06-17T12:54:20.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8f5cb5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8f00b27b0 0x7ff8f0064a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8f5cb5700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8f0140940 0x7ff8f01539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8f5cb5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8f0144d20 0x7ff8f009e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8f5cb5700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f013edb0 con 0x7ff8f00b27b0 2022-06-17T12:54:20.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8f5cb5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f0066c50 con 0x7ff8f0144d20 2022-06-17T12:54:20.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8f5cb5700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f0064f30 con 0x7ff8f0140940 2022-06-17T12:54:20.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8f4cb3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8f00b27b0 0x7ff8f0064a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8f4cb3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8f00b27b0 0x7ff8f0064a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36572/0 (socket says 172.21.15.36:36572) 2022-06-17T12:54:20.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8f4cb3700 1 -- 172.21.15.36:0/3698665786 learned_addr learned my addr 172.21.15.36:0/3698665786 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:20.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8effff700 1 --2- 172.21.15.36:0/3698665786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8f0140940 0x7ff8f01539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8f54b4700 1 --2- 172.21.15.36:0/3698665786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8f0144d20 0x7ff8f009e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8f4cb3700 1 -- 172.21.15.36:0/3698665786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8f0140940 msgr2=0x7ff8f01539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8f4cb3700 1 --2- 172.21.15.36:0/3698665786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8f0140940 0x7ff8f01539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8f4cb3700 1 -- 172.21.15.36:0/3698665786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8f0144d20 msgr2=0x7ff8f009e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8f4cb3700 1 --2- 172.21.15.36:0/3698665786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8f0144d20 0x7ff8f009e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8f4cb3700 1 -- 172.21.15.36:0/3698665786 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e400b180 con 0x7ff8f00b27b0 2022-06-17T12:54:20.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8f54b4700 1 --2- 172.21.15.36:0/3698665786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8f0144d20 0x7ff8f009e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:20.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8f4cb3700 1 --2- 172.21.15.36:0/3698665786 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8f00b27b0 0x7ff8f0064a40 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7ff8f013eef0 tx=0x7ff8e4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:20.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8edffb700 1 -- 172.21.15.36:0/3698665786 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8e4016070 con 0x7ff8f00b27b0 2022-06-17T12:54:20.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8edffb700 1 -- 172.21.15.36:0/3698665786 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e400db80 con 0x7ff8f00b27b0 2022-06-17T12:54:20.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8f5cb5700 1 -- 172.21.15.36:0/3698665786 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f0066ed0 con 0x7ff8f00b27b0 2022-06-17T12:54:20.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8edffb700 1 -- 172.21.15.36:0/3698665786 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8e4012460 con 0x7ff8f00b27b0 2022-06-17T12:54:20.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.176+0000 7ff8f5cb5700 1 -- 172.21.15.36:0/3698665786 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f0154400 con 0x7ff8f00b27b0 2022-06-17T12:54:20.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.180+0000 7ff8edffb700 1 -- 172.21.15.36:0/3698665786 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff8e4024020 con 0x7ff8f00b27b0 2022-06-17T12:54:20.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.180+0000 7ff8edffb700 1 --2- 172.21.15.36:0/3698665786 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff8d00539b0 0x7ff8d0055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.180+0000 7ff8f5cb5700 1 -- 172.21.15.36:0/3698665786 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f00ac310 con 0x7ff8f00b27b0 2022-06-17T12:54:20.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.180+0000 7ff8effff700 1 --2- 172.21.15.36:0/3698665786 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff8d00539b0 0x7ff8d0055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.180+0000 7ff8edffb700 1 -- 172.21.15.36:0/3698665786 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff8e406cfc0 con 0x7ff8f00b27b0 2022-06-17T12:54:20.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.180+0000 7ff8effff700 1 --2- 172.21.15.36:0/3698665786 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff8d00539b0 0x7ff8d0055e70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7ff8e000d000 tx=0x7ff8e0007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:20.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.188+0000 7ff8edffb700 1 -- 172.21.15.36:0/3698665786 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff8e4047790 con 0x7ff8f00b27b0 2022-06-17T12:54:20.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.312+0000 7ff8f5cb5700 1 -- 172.21.15.36:0/3698665786 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff8f0067200 con 0x7ff8f00b27b0 2022-06-17T12:54:20.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.316+0000 7ff8edffb700 1 -- 172.21.15.36:0/3698665786 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v84) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff8e405de60 con 0x7ff8f00b27b0 2022-06-17T12:54:20.303 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:54:20.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.316+0000 7ff8f5cb5700 1 -- 172.21.15.36:0/3698665786 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff8d00539b0 msgr2=0x7ff8d0055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.316+0000 7ff8f5cb5700 1 --2- 172.21.15.36:0/3698665786 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff8d00539b0 0x7ff8d0055e70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7ff8e000d000 tx=0x7ff8e0007a00).stop 2022-06-17T12:54:20.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.316+0000 7ff8f5cb5700 1 -- 172.21.15.36:0/3698665786 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8f00b27b0 msgr2=0x7ff8f0064a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.316+0000 7ff8f5cb5700 1 --2- 172.21.15.36:0/3698665786 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8f00b27b0 0x7ff8f0064a40 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7ff8f013eef0 tx=0x7ff8e4014000).stop 2022-06-17T12:54:20.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.316+0000 7ff8f5cb5700 1 -- 172.21.15.36:0/3698665786 shutdown_connections 2022-06-17T12:54:20.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.316+0000 7ff8f5cb5700 1 --2- 172.21.15.36:0/3698665786 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff8d00539b0 0x7ff8d0055e70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.316+0000 7ff8f5cb5700 1 --2- 172.21.15.36:0/3698665786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8f0144d20 0x7ff8f009e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.316+0000 7ff8f5cb5700 1 --2- 172.21.15.36:0/3698665786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8f0140940 0x7ff8f01539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.316+0000 7ff8f5cb5700 1 --2- 172.21.15.36:0/3698665786 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8f00b27b0 0x7ff8f0064a40 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.316+0000 7ff8f5cb5700 1 -- 172.21.15.36:0/3698665786 >> 172.21.15.36:0/3698665786 conn(0x7ff8f00b5d10 msgr2=0x7ff8f00b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:20.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.316+0000 7ff8f5cb5700 1 -- 172.21.15.36:0/3698665786 shutdown_connections 2022-06-17T12:54:20.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.316+0000 7ff8f5cb5700 1 -- 172.21.15.36:0/3698665786 wait complete. 2022-06-17T12:54:20.316 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:54:20.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd193245700 1 Processor -- start 2022-06-17T12:54:20.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd193245700 1 -- start start 2022-06-17T12:54:20.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd193245700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd18c0acee0 0x7fd18c0b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd193245700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd18c144240 0x7fd18c1446d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd193245700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18c0b1900 con 0x7fd18c0acb00 2022-06-17T12:54:20.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd193245700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd18c0b0ad0 con 0x7fd18c0acee0 2022-06-17T12:54:20.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd193245700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c0af370 con 0x7fd18c144240 2022-06-17T12:54:20.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd190fe1700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fd18c0acb00 0x7fd18c147480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54110/0 (socket says 172.21.15.36:54110) 2022-06-17T12:54:20.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd190fe1700 1 -- 172.21.15.36:0/360242586 learned_addr learned my addr 172.21.15.36:0/360242586 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:20.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd1917e2700 1 --2- 172.21.15.36:0/360242586 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd18c144240 0x7fd18c1446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd18bfff700 1 --2- 172.21.15.36:0/360242586 >> v2:172.21.15.70:3300/0 conn(0x7fd18c0acee0 0x7fd18c0b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd18b7fe700 1 -- 172.21.15.36:0/360242586 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1960703314 0 0) 0x7fd18c0b1900 con 0x7fd18c0acb00 2022-06-17T12:54:20.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd18b7fe700 1 -- 172.21.15.36:0/360242586 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd170003260 con 0x7fd18c0acb00 2022-06-17T12:54:20.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd18b7fe700 1 -- 172.21.15.36:0/360242586 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1147702301 0 0) 0x7fd170003260 con 0x7fd18c0acb00 2022-06-17T12:54:20.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd18b7fe700 1 -- 172.21.15.36:0/360242586 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd18c144240 msgr2=0x7fd18c1446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:20.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd18b7fe700 1 --2- 172.21.15.36:0/360242586 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd18c144240 0x7fd18c1446d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd18b7fe700 1 -- 172.21.15.36:0/360242586 >> v2:172.21.15.70:3300/0 conn(0x7fd18c0acee0 msgr2=0x7fd18c0b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:20.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd18b7fe700 1 --2- 172.21.15.36:0/360242586 >> v2:172.21.15.70:3300/0 conn(0x7fd18c0acee0 0x7fd18c0b05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd18b7fe700 1 -- 172.21.15.36:0/360242586 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c0b1b30 con 0x7fd18c0acb00 2022-06-17T12:54:20.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd18b7fe700 1 -- 172.21.15.36:0/360242586 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7fd180001660 con 0x7fd18c0acb00 2022-06-17T12:54:20.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd18bfff700 1 --2- 172.21.15.36:0/360242586 >> v2:172.21.15.70:3300/0 conn(0x7fd18c0acee0 0x7fd18c0b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:20.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd18b7fe700 1 -- 172.21.15.36:0/360242586 >> v1:172.21.15.36:6789/0 conn(0x7fd18c0acb00 legacy=0x7fd18c147480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd18b7fe700 1 --2- 172.21.15.36:0/360242586 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd170003750 0x7fd170003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd18b7fe700 1 --2- 172.21.15.36:0/360242586 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd18c144240 0x7fd170004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd18b7fe700 1 --2- 172.21.15.36:0/360242586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1700049f0 0x7fd170004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd18b7fe700 1 -- 172.21.15.36:0/360242586 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c0b0ad0 con 0x7fd170003750 2022-06-17T12:54:20.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd18b7fe700 1 -- 172.21.15.36:0/360242586 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c0af370 con 0x7fd1700049f0 2022-06-17T12:54:20.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd18b7fe700 1 -- 172.21.15.36:0/360242586 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17000b750 con 0x7fd18c144240 2022-06-17T12:54:20.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd190fe1700 1 --2- 172.21.15.36:0/360242586 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd170003750 0x7fd170003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd18bfff700 1 --2- 172.21.15.36:0/360242586 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd18c144240 0x7fd170004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd1917e2700 1 --2- 172.21.15.36:0/360242586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1700049f0 0x7fd170004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd190fe1700 1 -- 172.21.15.36:0/360242586 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd18c144240 msgr2=0x7fd170004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:20.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd190fe1700 1 --2- 172.21.15.36:0/360242586 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd18c144240 0x7fd170004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd190fe1700 1 -- 172.21.15.36:0/360242586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1700049f0 msgr2=0x7fd170004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:20.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd190fe1700 1 --2- 172.21.15.36:0/360242586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1700049f0 0x7fd170004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd190fe1700 1 -- 172.21.15.36:0/360242586 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd170003260 con 0x7fd170003750 2022-06-17T12:54:20.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd190fe1700 1 --2- 172.21.15.36:0/360242586 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd170003750 0x7fd170003b70 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7fd180001370 tx=0x7fd18c0b6390).ready entity=mon.0 client_cookie=c4c102f8a4535fa4 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:20.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd18b7fe700 1 -- 172.21.15.36:0/360242586 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1800070d0 con 0x7fd170003750 2022-06-17T12:54:20.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd18b7fe700 1 -- 172.21.15.36:0/360242586 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd180007230 con 0x7fd170003750 2022-06-17T12:54:20.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.440+0000 7fd18b7fe700 1 -- 172.21.15.36:0/360242586 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd18000c8f0 con 0x7fd170003750 2022-06-17T12:54:20.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd193245700 1 -- 172.21.15.36:0/360242586 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd170003750 msgr2=0x7fd170003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd193245700 1 --2- 172.21.15.36:0/360242586 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd170003750 0x7fd170003b70 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7fd180001370 tx=0x7fd18c0b6390).stop 2022-06-17T12:54:20.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd1917e2700 1 -- 172.21.15.36:0/360242586 reap_dead start 2022-06-17T12:54:20.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd193245700 1 -- 172.21.15.36:0/360242586 shutdown_connections 2022-06-17T12:54:20.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd193245700 1 -- 172.21.15.36:0/360242586 >> 172.21.15.36:0/360242586 conn(0x7fd18c05c550 msgr2=0x7fd18c0b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:20.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd193245700 1 -- 172.21.15.36:0/360242586 shutdown_connections 2022-06-17T12:54:20.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd193245700 1 -- 172.21.15.36:0/360242586 wait complete. 2022-06-17T12:54:20.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd193245700 1 Processor -- start 2022-06-17T12:54:20.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd193245700 1 -- start start 2022-06-17T12:54:20.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd193245700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd18c1429b0 0x7fd18c159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd193245700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd18c146650 0x7fd18c063f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd193245700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd18c064460 0x7fd18c066b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd193245700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c064910 con 0x7fd18c064460 2022-06-17T12:54:20.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd193245700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c147b20 con 0x7fd18c146650 2022-06-17T12:54:20.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd193245700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c145870 con 0x7fd18c1429b0 2022-06-17T12:54:20.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd1917e2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd18c064460 0x7fd18c066b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd1917e2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd18c064460 0x7fd18c066b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36596/0 (socket says 172.21.15.36:36596) 2022-06-17T12:54:20.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd1917e2700 1 -- 172.21.15.36:0/1614244832 learned_addr learned my addr 172.21.15.36:0/1614244832 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:20.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd190fe1700 1 --2- 172.21.15.36:0/1614244832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd18c1429b0 0x7fd18c159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd18bfff700 1 --2- 172.21.15.36:0/1614244832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd18c146650 0x7fd18c063f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd1917e2700 1 -- 172.21.15.36:0/1614244832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd18c1429b0 msgr2=0x7fd18c159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd1917e2700 1 --2- 172.21.15.36:0/1614244832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd18c1429b0 0x7fd18c159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd1917e2700 1 -- 172.21.15.36:0/1614244832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd18c146650 msgr2=0x7fd18c063f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd1917e2700 1 --2- 172.21.15.36:0/1614244832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd18c146650 0x7fd18c063f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd1917e2700 1 -- 172.21.15.36:0/1614244832 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c142d90 con 0x7fd18c064460 2022-06-17T12:54:20.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd1917e2700 1 --2- 172.21.15.36:0/1614244832 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd18c064460 0x7fd18c066b00 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7fd18400b000 tx=0x7fd1840091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:20.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd189ffb700 1 -- 172.21.15.36:0/1614244832 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd184019070 con 0x7fd18c064460 2022-06-17T12:54:20.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd189ffb700 1 -- 172.21.15.36:0/1614244832 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd184014720 con 0x7fd18c064460 2022-06-17T12:54:20.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd189ffb700 1 -- 172.21.15.36:0/1614244832 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd184014a80 con 0x7fd18c064460 2022-06-17T12:54:20.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd193245700 1 -- 172.21.15.36:0/1614244832 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18c147f30 con 0x7fd18c064460 2022-06-17T12:54:20.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.444+0000 7fd193245700 1 -- 172.21.15.36:0/1614244832 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18c148470 con 0x7fd18c064460 2022-06-17T12:54:20.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.448+0000 7fd193245700 1 -- 172.21.15.36:0/1614244832 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd18c063c70 con 0x7fd18c064460 2022-06-17T12:54:20.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.448+0000 7fd189ffb700 1 -- 172.21.15.36:0/1614244832 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd184014be0 con 0x7fd18c064460 2022-06-17T12:54:20.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.448+0000 7fd189ffb700 1 --2- 172.21.15.36:0/1614244832 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd16c053a60 0x7fd16c055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.448+0000 7fd189ffb700 1 -- 172.21.15.36:0/1614244832 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd18406c8d0 con 0x7fd18c064460 2022-06-17T12:54:20.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.448+0000 7fd190fe1700 1 --2- 172.21.15.36:0/1614244832 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd16c053a60 0x7fd16c055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.448+0000 7fd190fe1700 1 --2- 172.21.15.36:0/1614244832 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd16c053a60 0x7fd16c055f20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7fd180019000 tx=0x7fd180015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:20.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.452+0000 7fd189ffb700 1 -- 172.21.15.36:0/1614244832 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd184043560 con 0x7fd18c064460 2022-06-17T12:54:20.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.584+0000 7fd193245700 1 -- 172.21.15.36:0/1614244832 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd18c057410 con 0x7fd18c064460 2022-06-17T12:54:20.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.584+0000 7fd189ffb700 1 -- 172.21.15.36:0/1614244832 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+10 (secure 0 0 0) 0x7fd184046b80 con 0x7fd18c064460 2022-06-17T12:54:20.572 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:54:20.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.584+0000 7fd193245700 1 -- 172.21.15.36:0/1614244832 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd16c053a60 msgr2=0x7fd16c055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.584+0000 7fd193245700 1 --2- 172.21.15.36:0/1614244832 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd16c053a60 0x7fd16c055f20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7fd180019000 tx=0x7fd180015000).stop 2022-06-17T12:54:20.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.584+0000 7fd193245700 1 -- 172.21.15.36:0/1614244832 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd18c064460 msgr2=0x7fd18c066b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.584+0000 7fd193245700 1 --2- 172.21.15.36:0/1614244832 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd18c064460 0x7fd18c066b00 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7fd18400b000 tx=0x7fd1840091d0).stop 2022-06-17T12:54:20.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.584+0000 7fd193245700 1 -- 172.21.15.36:0/1614244832 shutdown_connections 2022-06-17T12:54:20.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.584+0000 7fd193245700 1 --2- 172.21.15.36:0/1614244832 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd16c053a60 0x7fd16c055f20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.584+0000 7fd193245700 1 --2- 172.21.15.36:0/1614244832 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd18c064460 0x7fd18c066b00 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.584+0000 7fd193245700 1 --2- 172.21.15.36:0/1614244832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd18c146650 0x7fd18c063f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.584+0000 7fd193245700 1 --2- 172.21.15.36:0/1614244832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd18c1429b0 0x7fd18c159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.584+0000 7fd193245700 1 -- 172.21.15.36:0/1614244832 >> 172.21.15.36:0/1614244832 conn(0x7fd18c05c550 msgr2=0x7fd18c060910 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:20.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.588+0000 7fd193245700 1 -- 172.21.15.36:0/1614244832 shutdown_connections 2022-06-17T12:54:20.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.588+0000 7fd193245700 1 -- 172.21.15.36:0/1614244832 wait complete. 2022-06-17T12:54:20.585 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:54:20.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.704+0000 7fc373366700 1 Processor -- start 2022-06-17T12:54:20.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.704+0000 7fc373366700 1 -- start start 2022-06-17T12:54:20.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc373366700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc36c0afe40 0x7fc36c0b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc373366700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc36c13b130 0x7fc36c140f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc373366700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc36c0a9b90 con 0x7fc36c0b3440 2022-06-17T12:54:20.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc373366700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc36c0b0220 con 0x7fc36c13b130 2022-06-17T12:54:20.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc373366700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc36c0a9dc0 con 0x7fc36c0afe40 2022-06-17T12:54:20.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc371903700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc36c13b130 0x7fc36c140f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc371903700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc36c13b130 0x7fc36c140f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36602/0 (socket says 172.21.15.36:36602) 2022-06-17T12:54:20.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc371903700 1 -- 172.21.15.36:0/134398304 learned_addr learned my addr 172.21.15.36:0/134398304 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:20.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc371102700 1 --2- 172.21.15.36:0/134398304 >> v2:172.21.15.70:3300/0 conn(0x7fc36c0afe40 0x7fc36c0b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc371903700 1 -- 172.21.15.36:0/134398304 >> v2:172.21.15.70:3300/0 conn(0x7fc36c0afe40 msgr2=0x7fc36c0b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:20.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc371903700 1 --2- 172.21.15.36:0/134398304 >> v2:172.21.15.70:3300/0 conn(0x7fc36c0afe40 0x7fc36c0b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc371903700 1 -- 172.21.15.36:0/134398304 >> v1:172.21.15.70:6789/0 conn(0x7fc36c0b3440 legacy=0x7fc36c13aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc371903700 1 -- 172.21.15.36:0/134398304 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc36c144f40 con 0x7fc36c13b130 2022-06-17T12:54:20.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc371903700 1 --2- 172.21.15.36:0/134398304 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc36c13b130 0x7fc36c140f60 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7fc36c0b3870 tx=0x7fc364006eb0).ready entity=mon.0 client_cookie=337975108804005e server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:20.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc35ffff700 1 -- 172.21.15.36:0/134398304 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc36400d610 con 0x7fc36c13b130 2022-06-17T12:54:20.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc35ffff700 1 -- 172.21.15.36:0/134398304 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc364000e90 con 0x7fc36c13b130 2022-06-17T12:54:20.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc35ffff700 1 -- 172.21.15.36:0/134398304 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc36400d960 con 0x7fc36c13b130 2022-06-17T12:54:20.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc373366700 1 -- 172.21.15.36:0/134398304 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc36c13b130 msgr2=0x7fc36c140f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc373366700 1 --2- 172.21.15.36:0/134398304 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc36c13b130 0x7fc36c140f60 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7fc36c0b3870 tx=0x7fc364006eb0).stop 2022-06-17T12:54:20.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc373366700 1 -- 172.21.15.36:0/134398304 shutdown_connections 2022-06-17T12:54:20.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc373366700 1 --2- 172.21.15.36:0/134398304 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc36c13b130 0x7fc36c140f60 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc373366700 1 --2- 172.21.15.36:0/134398304 >> v2:172.21.15.70:3300/0 conn(0x7fc36c0afe40 0x7fc36c0b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc373366700 1 -- 172.21.15.36:0/134398304 >> 172.21.15.36:0/134398304 conn(0x7fc36c05c550 msgr2=0x7fc36c05c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:20.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc373366700 1 -- 172.21.15.36:0/134398304 shutdown_connections 2022-06-17T12:54:20.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.708+0000 7fc373366700 1 -- 172.21.15.36:0/134398304 wait complete. 2022-06-17T12:54:20.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc373366700 1 Processor -- start 2022-06-17T12:54:20.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc373366700 1 -- start start 2022-06-17T12:54:20.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc373366700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc36c0afe40 0x7fc36c0ab410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc373366700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc36c0b3440 0x7fc36c0accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc373366700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc36c13b130 0x7fc36c13c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc373366700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c0ac580 con 0x7fc36c0b3440 2022-06-17T12:54:20.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc373366700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c1515b0 con 0x7fc36c0afe40 2022-06-17T12:54:20.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc373366700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c151850 con 0x7fc36c13b130 2022-06-17T12:54:20.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc370901700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc36c0b3440 0x7fc36c0accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc370901700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc36c0b3440 0x7fc36c0accf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36608/0 (socket says 172.21.15.36:36608) 2022-06-17T12:54:20.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc370901700 1 -- 172.21.15.36:0/612409476 learned_addr learned my addr 172.21.15.36:0/612409476 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:20.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc371903700 1 --2- 172.21.15.36:0/612409476 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc36c13b130 0x7fc36c13c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc371102700 1 --2- 172.21.15.36:0/612409476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc36c0afe40 0x7fc36c0ab410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc370901700 1 -- 172.21.15.36:0/612409476 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc36c13b130 msgr2=0x7fc36c13c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc370901700 1 --2- 172.21.15.36:0/612409476 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc36c13b130 0x7fc36c13c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc370901700 1 -- 172.21.15.36:0/612409476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc36c0afe40 msgr2=0x7fc36c0ab410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc370901700 1 --2- 172.21.15.36:0/612409476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc36c0afe40 0x7fc36c0ab410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc370901700 1 -- 172.21.15.36:0/612409476 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc36400b1e0 con 0x7fc36c0b3440 2022-06-17T12:54:20.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc371903700 1 --2- 172.21.15.36:0/612409476 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc36c13b130 0x7fc36c13c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:54:20.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc371102700 1 --2- 172.21.15.36:0/612409476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc36c0afe40 0x7fc36c0ab410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:54:20.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc370901700 1 --2- 172.21.15.36:0/612409476 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc36c0b3440 0x7fc36c0accf0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7fc36c062eb0 tx=0x7fc360007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:20.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc35e7fc700 1 -- 172.21.15.36:0/612409476 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc360006aa0 con 0x7fc36c0b3440 2022-06-17T12:54:20.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc35e7fc700 1 -- 172.21.15.36:0/612409476 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc360006c00 con 0x7fc36c0b3440 2022-06-17T12:54:20.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc373366700 1 -- 172.21.15.36:0/612409476 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36c0ac200 con 0x7fc36c0b3440 2022-06-17T12:54:20.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc35e7fc700 1 -- 172.21.15.36:0/612409476 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc360007030 con 0x7fc36c0b3440 2022-06-17T12:54:20.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc373366700 1 -- 172.21.15.36:0/612409476 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36c0ad1e0 con 0x7fc36c0b3440 2022-06-17T12:54:20.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.712+0000 7fc373366700 1 -- 172.21.15.36:0/612409476 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc36c0ac3e0 con 0x7fc36c0b3440 2022-06-17T12:54:20.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.716+0000 7fc35e7fc700 1 -- 172.21.15.36:0/612409476 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fc360019030 con 0x7fc36c0b3440 2022-06-17T12:54:20.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.716+0000 7fc35e7fc700 1 --2- 172.21.15.36:0/612409476 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc354053920 0x7fc354055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.716+0000 7fc35e7fc700 1 -- 172.21.15.36:0/612409476 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc36006bb70 con 0x7fc36c0b3440 2022-06-17T12:54:20.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.716+0000 7fc371102700 1 --2- 172.21.15.36:0/612409476 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc354053920 0x7fc354055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.716+0000 7fc371102700 1 --2- 172.21.15.36:0/612409476 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc354053920 0x7fc354055de0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7fc368008000 tx=0x7fc368008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:20.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.720+0000 7fc35e7fc700 1 -- 172.21.15.36:0/612409476 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc360042860 con 0x7fc36c0b3440 2022-06-17T12:54:20.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.844+0000 7fc373366700 1 -- 172.21.15.36:0/612409476 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc36c0ac390 con 0x7fc36c0b3440 2022-06-17T12:54:20.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.844+0000 7fc35e7fc700 1 -- 172.21.15.36:0/612409476 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc360017020 con 0x7fc36c0b3440 2022-06-17T12:54:20.835 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:54:20.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.848+0000 7fc373366700 1 -- 172.21.15.36:0/612409476 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc354053920 msgr2=0x7fc354055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.848+0000 7fc373366700 1 --2- 172.21.15.36:0/612409476 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc354053920 0x7fc354055de0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7fc368008000 tx=0x7fc368008040).stop 2022-06-17T12:54:20.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.848+0000 7fc373366700 1 -- 172.21.15.36:0/612409476 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc36c0b3440 msgr2=0x7fc36c0accf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.848+0000 7fc373366700 1 --2- 172.21.15.36:0/612409476 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc36c0b3440 0x7fc36c0accf0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7fc36c062eb0 tx=0x7fc360007f90).stop 2022-06-17T12:54:20.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.848+0000 7fc373366700 1 -- 172.21.15.36:0/612409476 shutdown_connections 2022-06-17T12:54:20.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.848+0000 7fc373366700 1 --2- 172.21.15.36:0/612409476 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc354053920 0x7fc354055de0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.848+0000 7fc373366700 1 --2- 172.21.15.36:0/612409476 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc36c13b130 0x7fc36c13c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.848+0000 7fc373366700 1 --2- 172.21.15.36:0/612409476 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc36c0b3440 0x7fc36c0accf0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.848+0000 7fc373366700 1 --2- 172.21.15.36:0/612409476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc36c0afe40 0x7fc36c0ab410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.848+0000 7fc373366700 1 -- 172.21.15.36:0/612409476 >> 172.21.15.36:0/612409476 conn(0x7fc36c05c550 msgr2=0x7fc36c0b48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:20.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.848+0000 7fc373366700 1 -- 172.21.15.36:0/612409476 shutdown_connections 2022-06-17T12:54:20.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.848+0000 7fc373366700 1 -- 172.21.15.36:0/612409476 wait complete. 2022-06-17T12:54:20.846 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:54:20.847 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:54:20.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.956+0000 7fb73cb7e700 1 Processor -- start 2022-06-17T12:54:20.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.956+0000 7fb73cb7e700 1 -- start start 2022-06-17T12:54:20.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb73cb7e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb73813cb70 0x7fb738143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb73cb7e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb7380aef60 0x7fb7380b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb73cb7e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7380af4b0 con 0x7fb7380ba2e0 2022-06-17T12:54:20.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb73cb7e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb738142f50 con 0x7fb7380aef60 2022-06-17T12:54:20.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb73cb7e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7381431f0 con 0x7fb73813cb70 2022-06-17T12:54:20.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb737fff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fb7380ba2e0 0x7fb7381442a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54148/0 (socket says 172.21.15.36:54148) 2022-06-17T12:54:20.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb737fff700 1 -- 172.21.15.36:0/1943905513 learned_addr learned my addr 172.21.15.36:0/1943905513 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:20.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb7377fe700 1 --2- 172.21.15.36:0/1943905513 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb73813cb70 0x7fb738143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb736ffd700 1 --2- 172.21.15.36:0/1943905513 >> v2:172.21.15.70:3300/0 conn(0x7fb7380aef60 0x7fb7380b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb7367fc700 1 -- 172.21.15.36:0/1943905513 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 939257689 0 0) 0x7fb7380af4b0 con 0x7fb7380ba2e0 2022-06-17T12:54:20.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb7367fc700 1 -- 172.21.15.36:0/1943905513 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb724003260 con 0x7fb7380ba2e0 2022-06-17T12:54:20.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb736ffd700 1 -- 172.21.15.36:0/1943905513 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb73813cb70 msgr2=0x7fb738143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:20.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb736ffd700 1 --2- 172.21.15.36:0/1943905513 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb73813cb70 0x7fb738143690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb736ffd700 1 -- 172.21.15.36:0/1943905513 >> v1:172.21.15.36:6789/0 conn(0x7fb7380ba2e0 legacy=0x7fb7381442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb736ffd700 1 -- 172.21.15.36:0/1943905513 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7381449b0 con 0x7fb7380aef60 2022-06-17T12:54:20.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb7377fe700 1 --2- 172.21.15.36:0/1943905513 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb73813cb70 0x7fb738143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:20.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb736ffd700 1 --2- 172.21.15.36:0/1943905513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7380aef60 0x7fb7380b9df0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7fb72c002f70 tx=0x7fb72c006df0).ready entity=mon.1 client_cookie=b57021bed3b14583 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:20.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb7367fc700 1 -- 172.21.15.36:0/1943905513 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb72c00d4a0 con 0x7fb7380aef60 2022-06-17T12:54:20.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb7367fc700 1 -- 172.21.15.36:0/1943905513 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72c000e90 con 0x7fb7380aef60 2022-06-17T12:54:20.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb7367fc700 1 -- 172.21.15.36:0/1943905513 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb72c00d7f0 con 0x7fb7380aef60 2022-06-17T12:54:20.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb73cb7e700 1 -- 172.21.15.36:0/1943905513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7380aef60 msgr2=0x7fb7380b9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb73cb7e700 1 --2- 172.21.15.36:0/1943905513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7380aef60 0x7fb7380b9df0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7fb72c002f70 tx=0x7fb72c006df0).stop 2022-06-17T12:54:20.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb73cb7e700 1 -- 172.21.15.36:0/1943905513 shutdown_connections 2022-06-17T12:54:20.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb73cb7e700 1 --2- 172.21.15.36:0/1943905513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7380aef60 0x7fb7380b9df0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb73cb7e700 1 --2- 172.21.15.36:0/1943905513 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb73813cb70 0x7fb738143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb73cb7e700 1 -- 172.21.15.36:0/1943905513 >> 172.21.15.36:0/1943905513 conn(0x7fb73805c550 msgr2=0x7fb7380b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:20.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb73cb7e700 1 -- 172.21.15.36:0/1943905513 shutdown_connections 2022-06-17T12:54:20.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.960+0000 7fb73cb7e700 1 -- 172.21.15.36:0/1943905513 wait complete. 2022-06-17T12:54:20.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.964+0000 7fb73cb7e700 1 Processor -- start 2022-06-17T12:54:20.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.964+0000 7fb73cb7e700 1 -- start start 2022-06-17T12:54:20.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.964+0000 7fb73cb7e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7380aef60 0x7fb7380b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.964+0000 7fb73cb7e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7380ba2e0 0x7fb7380b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.964+0000 7fb73cb7e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb73813cb70 0x7fb738152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.964+0000 7fb73cb7e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73813fd90 con 0x7fb73813cb70 2022-06-17T12:54:20.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.964+0000 7fb73cb7e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7381557c0 con 0x7fb7380ba2e0 2022-06-17T12:54:20.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.964+0000 7fb73cb7e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7380b7ee0 con 0x7fb7380aef60 2022-06-17T12:54:20.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.964+0000 7fb737fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb73813cb70 0x7fb738152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.964+0000 7fb736ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7380ba2e0 0x7fb7380b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.964+0000 7fb7377fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7380aef60 0x7fb7380b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.964+0000 7fb737fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb73813cb70 0x7fb738152150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36624/0 (socket says 172.21.15.36:36624) 2022-06-17T12:54:20.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.964+0000 7fb737fff700 1 -- 172.21.15.36:0/1956385812 learned_addr learned my addr 172.21.15.36:0/1956385812 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:20.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.964+0000 7fb737fff700 1 -- 172.21.15.36:0/1956385812 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7380aef60 msgr2=0x7fb7380b8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.964+0000 7fb737fff700 1 --2- 172.21.15.36:0/1956385812 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7380aef60 0x7fb7380b8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.964+0000 7fb737fff700 1 -- 172.21.15.36:0/1956385812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7380ba2e0 msgr2=0x7fb7380b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:20.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.964+0000 7fb737fff700 1 --2- 172.21.15.36:0/1956385812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7380ba2e0 0x7fb7380b2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:20.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.964+0000 7fb737fff700 1 -- 172.21.15.36:0/1956385812 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72800b180 con 0x7fb73813cb70 2022-06-17T12:54:20.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.964+0000 7fb737fff700 1 --2- 172.21.15.36:0/1956385812 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb73813cb70 0x7fb738152150 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7fb730001490 tx=0x7fb730007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:20.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.964+0000 7fb734ff9700 1 -- 172.21.15.36:0/1956385812 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb730005ba0 con 0x7fb73813cb70 2022-06-17T12:54:20.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.964+0000 7fb734ff9700 1 -- 172.21.15.36:0/1956385812 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb730005d00 con 0x7fb73813cb70 2022-06-17T12:54:20.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.964+0000 7fb73cb7e700 1 -- 172.21.15.36:0/1956385812 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb72c00b180 con 0x7fb73813cb70 2022-06-17T12:54:20.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.964+0000 7fb734ff9700 1 -- 172.21.15.36:0/1956385812 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7300072c0 con 0x7fb73813cb70 2022-06-17T12:54:20.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.964+0000 7fb73cb7e700 1 -- 172.21.15.36:0/1956385812 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb73813fa10 con 0x7fb73813cb70 2022-06-17T12:54:20.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.968+0000 7fb73cb7e700 1 -- 172.21.15.36:0/1956385812 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb738057410 con 0x7fb73813cb70 2022-06-17T12:54:20.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.968+0000 7fb734ff9700 1 -- 172.21.15.36:0/1956385812 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fb730010070 con 0x7fb73813cb70 2022-06-17T12:54:20.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.968+0000 7fb734ff9700 1 --2- 172.21.15.36:0/1956385812 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb7180539b0 0x7fb718055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:20.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.968+0000 7fb734ff9700 1 -- 172.21.15.36:0/1956385812 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb73006b950 con 0x7fb73813cb70 2022-06-17T12:54:20.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.968+0000 7fb7377fe700 1 --2- 172.21.15.36:0/1956385812 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb7180539b0 0x7fb718055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:20.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.968+0000 7fb7377fe700 1 --2- 172.21.15.36:0/1956385812 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb7180539b0 0x7fb718055e70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fb72800b000 tx=0x7fb728006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:20.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:20.976+0000 7fb734ff9700 1 -- 172.21.15.36:0/1956385812 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb7300426b0 con 0x7fb73813cb70 2022-06-17T12:54:21.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:21.140+0000 7fb73cb7e700 1 -- 172.21.15.36:0/1956385812 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb738067bf0 con 0x7fb73813cb70 2022-06-17T12:54:21.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:21.140+0000 7fb734ff9700 1 -- 172.21.15.36:0/1956385812 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+2140 (secure 0 0 0) 0x7fb730018070 con 0x7fb73813cb70 2022-06-17T12:54:21.129 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:21.129 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":42,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/3068692336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3068692336},{"type":"v1","addr":"172.21.15.70:6833","nonce":3068692336}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:54:12.740483+0000","modified":"2022-06-17T12:54:13.744294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6753},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6753":{"gid":6753,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/2728662009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2728662009},{"type":"v1","addr":"172.21.15.36:6835","nonce":2728662009}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-06-17T12:54:21.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:21.144+0000 7fb73cb7e700 1 -- 172.21.15.36:0/1956385812 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb7180539b0 msgr2=0x7fb718055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:21.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:21.144+0000 7fb73cb7e700 1 --2- 172.21.15.36:0/1956385812 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb7180539b0 0x7fb718055e70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fb72800b000 tx=0x7fb728006df0).stop 2022-06-17T12:54:21.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:21.144+0000 7fb73cb7e700 1 -- 172.21.15.36:0/1956385812 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb73813cb70 msgr2=0x7fb738152150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:21.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:21.144+0000 7fb73cb7e700 1 --2- 172.21.15.36:0/1956385812 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb73813cb70 0x7fb738152150 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7fb730001490 tx=0x7fb730007e00).stop 2022-06-17T12:54:21.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:21.144+0000 7fb73cb7e700 1 -- 172.21.15.36:0/1956385812 shutdown_connections 2022-06-17T12:54:21.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:21.144+0000 7fb73cb7e700 1 --2- 172.21.15.36:0/1956385812 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb7180539b0 0x7fb718055e70 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:21.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:21.144+0000 7fb73cb7e700 1 --2- 172.21.15.36:0/1956385812 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb73813cb70 0x7fb738152150 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:21.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:21.144+0000 7fb73cb7e700 1 --2- 172.21.15.36:0/1956385812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7380ba2e0 0x7fb7380b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:21.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:21.144+0000 7fb73cb7e700 1 --2- 172.21.15.36:0/1956385812 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7380aef60 0x7fb7380b8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:21.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:21.144+0000 7fb73cb7e700 1 -- 172.21.15.36:0/1956385812 >> 172.21.15.36:0/1956385812 conn(0x7fb73805c550 msgr2=0x7fb7381462f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:21.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:21.144+0000 7fb73cb7e700 1 -- 172.21.15.36:0/1956385812 shutdown_connections 2022-06-17T12:54:21.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:21.144+0000 7fb73cb7e700 1 -- 172.21.15.36:0/1956385812 wait complete. 2022-06-17T12:54:21.139 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 42 2022-06-17T12:54:21.344 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:54:12.740483+0000', 'modified': '2022-06-17T12:54:13.744294+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6753}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6753': {'gid': 6753, 'name': 'a', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.36:6835/2728662009', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 2728662009}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 2728662009}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:54:21.345 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:54:21.346 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:54:21.467 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":6753,"want_state":"up:active","state":"up:active","rank_uptime":8.7296462960000003,"mdsmap_epoch":42,"osdmap_epoch":84,"osdmap_epoch_barrier":83,"uptime":13.129955943000001} 2022-06-17T12:54:21.468 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 6753, "mdsmap_epoch": 42, "osdmap_epoch": 84, "osdmap_epoch_barrier": 83, "rank_uptime": 8.729646296, "state": "up:active", "uptime": 13.129955943, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:54:21.468 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:54:21.469 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-06-17T12:54:21.469 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi136.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-06-17T12:54:21.470 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:21.470 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:54:21.480 DEBUG:teuthology.orchestra.run.smithi136:> sudo modprobe fuse 2022-06-17T12:54:21.496 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:54:21.497 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:54:21.554 INFO:teuthology.orchestra.run.smithi136.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:54:21.555 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:54:21.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:21.557 DEBUG:teuthology.orchestra.run.smithi136:> ls /sys/fs/fuse/connections 2022-06-17T12:54:21.565 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:54:21.566 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:54:21.568 DEBUG:teuthology.orchestra.run.smithi136:> sudo modprobe fuse 2022-06-17T12:54:21.587 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:54:21.589 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:54:21.641 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:2022-06-17T12:54:21.649+0000 7f821862c080 -1 init, newargv = 0x55574cae71d0 newargc=9 2022-06-17T12:54:21.642 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:ceph-fuse[21979]: starting ceph client 2022-06-17T12:54:21.647 INFO:teuthology.orchestra.run.smithi136.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:54:21.649 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:54:21.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:21.651 DEBUG:teuthology.orchestra.run.smithi136:> ls /sys/fs/fuse/connections 2022-06-17T12:54:21.652 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:ceph-fuse[21979]: starting fuse 2022-06-17T12:54:21.658 INFO:teuthology.orchestra.run.smithi136.stdout:52 2022-06-17T12:54:21.659 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:54:21.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:21.660 DEBUG:teuthology.orchestra.run.smithi136:> sudo python3 -c ' 2022-06-17T12:54:21.660 DEBUG:teuthology.orchestra.run.smithi136:> import glob 2022-06-17T12:54:21.661 DEBUG:teuthology.orchestra.run.smithi136:> import re 2022-06-17T12:54:21.661 DEBUG:teuthology.orchestra.run.smithi136:> import os 2022-06-17T12:54:21.661 DEBUG:teuthology.orchestra.run.smithi136:> import subprocess 2022-06-17T12:54:21.662 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:54:21.663 DEBUG:teuthology.orchestra.run.smithi136:> def find_socket(client_name): 2022-06-17T12:54:21.663 DEBUG:teuthology.orchestra.run.smithi136:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-06-17T12:54:21.664 DEBUG:teuthology.orchestra.run.smithi136:> files = glob.glob(asok_path) 2022-06-17T12:54:21.664 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:54:21.664 DEBUG:teuthology.orchestra.run.smithi136:> # Given a non-glob path, it better be there 2022-06-17T12:54:21.665 DEBUG:teuthology.orchestra.run.smithi136:> if "*" not in asok_path: 2022-06-17T12:54:21.666 DEBUG:teuthology.orchestra.run.smithi136:> assert(len(files) == 1) 2022-06-17T12:54:21.666 DEBUG:teuthology.orchestra.run.smithi136:> return files[0] 2022-06-17T12:54:21.667 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:54:21.667 DEBUG:teuthology.orchestra.run.smithi136:> for f in files: 2022-06-17T12:54:21.668 DEBUG:teuthology.orchestra.run.smithi136:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:54:21.668 DEBUG:teuthology.orchestra.run.smithi136:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:54:21.668 DEBUG:teuthology.orchestra.run.smithi136:> return f 2022-06-17T12:54:21.669 DEBUG:teuthology.orchestra.run.smithi136:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:54:21.670 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:54:21.670 DEBUG:teuthology.orchestra.run.smithi136:> print(find_socket("client.1")) 2022-06-17T12:54:21.671 DEBUG:teuthology.orchestra.run.smithi136:> ' 2022-06-17T12:54:21.714 INFO:teuthology.orchestra.run.smithi136.stdout:/var/run/ceph/ceph-client.1.21979.asok 2022-06-17T12:54:21.719 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.21979.asok 2022-06-17T12:54:21.720 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:21.721 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.21979.asok status 2022-06-17T12:54:21.856 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2022-06-17T12:54:21.857 INFO:teuthology.orchestra.run.smithi136.stdout: "metadata": { 2022-06-17T12:54:21.857 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:54:21.857 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:54:21.858 INFO:teuthology.orchestra.run.smithi136.stdout: "entity_id": "1", 2022-06-17T12:54:21.858 INFO:teuthology.orchestra.run.smithi136.stdout: "hostname": "smithi136", 2022-06-17T12:54:21.858 INFO:teuthology.orchestra.run.smithi136.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-06-17T12:54:21.858 INFO:teuthology.orchestra.run.smithi136.stdout: "pid": "21979", 2022-06-17T12:54:21.859 INFO:teuthology.orchestra.run.smithi136.stdout: "root": "/" 2022-06-17T12:54:21.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:54:21.859 INFO:teuthology.orchestra.run.smithi136.stdout: "dentry_count": 0, 2022-06-17T12:54:21.859 INFO:teuthology.orchestra.run.smithi136.stdout: "dentry_pinned_count": 0, 2022-06-17T12:54:21.860 INFO:teuthology.orchestra.run.smithi136.stdout: "id": 6766, 2022-06-17T12:54:21.860 INFO:teuthology.orchestra.run.smithi136.stdout: "inst": { 2022-06-17T12:54:21.860 INFO:teuthology.orchestra.run.smithi136.stdout: "name": { 2022-06-17T12:54:21.860 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "client", 2022-06-17T12:54:21.860 INFO:teuthology.orchestra.run.smithi136.stdout: "num": 6766 2022-06-17T12:54:21.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:54:21.861 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": { 2022-06-17T12:54:21.861 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "v1", 2022-06-17T12:54:21.863 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": "172.21.15.136:0", 2022-06-17T12:54:21.863 INFO:teuthology.orchestra.run.smithi136.stdout: "nonce": 1471553432 2022-06-17T12:54:21.863 INFO:teuthology.orchestra.run.smithi136.stdout: } 2022-06-17T12:54:21.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:54:21.864 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": { 2022-06-17T12:54:21.864 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "v1", 2022-06-17T12:54:21.864 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": "172.21.15.136:0", 2022-06-17T12:54:21.864 INFO:teuthology.orchestra.run.smithi136.stdout: "nonce": 1471553432 2022-06-17T12:54:21.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:54:21.865 INFO:teuthology.orchestra.run.smithi136.stdout: "inst_str": "client.6766 172.21.15.136:0/1471553432", 2022-06-17T12:54:21.865 INFO:teuthology.orchestra.run.smithi136.stdout: "addr_str": "172.21.15.136:0/1471553432", 2022-06-17T12:54:21.865 INFO:teuthology.orchestra.run.smithi136.stdout: "inode_count": 1, 2022-06-17T12:54:21.865 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_epoch": 42, 2022-06-17T12:54:21.866 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_epoch": 84, 2022-06-17T12:54:21.866 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:54:21.868 INFO:teuthology.orchestra.run.smithi136.stdout: "blacklisted": false, 2022-06-17T12:54:21.868 INFO:teuthology.orchestra.run.smithi136.stdout: "fs_name": "cephfs" 2022-06-17T12:54:21.869 INFO:teuthology.orchestra.run.smithi136.stdout:} 2022-06-17T12:54:21.870 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:21.871 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:21.871 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:54:21.879 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:54:21.880 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:54:21.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:21.881 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:54:21.944 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:54:22.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.072+0000 7fc576759700 1 Processor -- start 2022-06-17T12:54:22.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.072+0000 7fc576759700 1 -- start start 2022-06-17T12:54:22.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.072+0000 7fc576759700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc5700af930 0x7fc570064230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:22.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.072+0000 7fc576759700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5700af280 con 0x7fc5700afe30 2022-06-17T12:54:22.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.072+0000 7fc576759700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5700b14f0 con 0x7fc5700aeea0 2022-06-17T12:54:22.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.072+0000 7fc576759700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc570067560 con 0x7fc5700af930 2022-06-17T12:54:22.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.072+0000 7fc575f58700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fc5700afe30 0x7fc5700ab9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54162/0 (socket says 172.21.15.36:54162) 2022-06-17T12:54:22.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.072+0000 7fc575f58700 1 -- 172.21.15.36:0/866809234 learned_addr learned my addr 172.21.15.36:0/866809234 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:22.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.072+0000 7fc574f56700 1 --2- 172.21.15.36:0/866809234 >> v2:172.21.15.70:3300/0 conn(0x7fc5700af930 0x7fc570064230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:22.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.072+0000 7fc567fff700 1 -- 172.21.15.36:0/866809234 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741543792 0 0) 0x7fc5700af280 con 0x7fc5700afe30 2022-06-17T12:54:22.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.072+0000 7fc567fff700 1 -- 172.21.15.36:0/866809234 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc554003260 con 0x7fc5700afe30 2022-06-17T12:54:22.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.072+0000 7fc574f56700 1 -- 172.21.15.36:0/866809234 >> v1:172.21.15.70:6789/0 conn(0x7fc5700aeea0 legacy=0x7fc5700b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:22.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.072+0000 7fc567fff700 1 -- 172.21.15.36:0/866809234 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4151492866 0 0) 0x7fc5700b14f0 con 0x7fc5700aeea0 2022-06-17T12:54:22.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.072+0000 7fc574f56700 1 -- 172.21.15.36:0/866809234 >> v1:172.21.15.36:6789/0 conn(0x7fc5700afe30 legacy=0x7fc5700ab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:22.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.072+0000 7fc574f56700 1 -- 172.21.15.36:0/866809234 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5700b20f0 con 0x7fc5700af930 2022-06-17T12:54:22.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.072+0000 7fc574f56700 1 --2- 172.21.15.36:0/866809234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5700af930 0x7fc570064230 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fc5700b0700 tx=0x7fc56c0091d0).ready entity=mon.1 client_cookie=577b74ccf6760a97 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:22.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.072+0000 7fc567fff700 1 -- 172.21.15.36:0/866809234 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc56c00b670 con 0x7fc5700af930 2022-06-17T12:54:22.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.076+0000 7fc567fff700 1 -- 172.21.15.36:0/866809234 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc56c00b7d0 con 0x7fc5700af930 2022-06-17T12:54:22.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.076+0000 7fc567fff700 1 -- 172.21.15.36:0/866809234 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc56c00b940 con 0x7fc5700af930 2022-06-17T12:54:22.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.076+0000 7fc576759700 1 -- 172.21.15.36:0/866809234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5700af930 msgr2=0x7fc570064230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:22.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.076+0000 7fc576759700 1 --2- 172.21.15.36:0/866809234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5700af930 0x7fc570064230 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fc5700b0700 tx=0x7fc56c0091d0).stop 2022-06-17T12:54:22.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.076+0000 7fc576759700 1 -- 172.21.15.36:0/866809234 shutdown_connections 2022-06-17T12:54:22.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.076+0000 7fc576759700 1 --2- 172.21.15.36:0/866809234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5700af930 0x7fc570064230 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:22.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.076+0000 7fc576759700 1 -- 172.21.15.36:0/866809234 >> 172.21.15.36:0/866809234 conn(0x7fc57005f770 msgr2=0x7fc57005fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:22.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.076+0000 7fc576759700 1 -- 172.21.15.36:0/866809234 shutdown_connections 2022-06-17T12:54:22.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.076+0000 7fc576759700 1 -- 172.21.15.36:0/866809234 wait complete. 2022-06-17T12:54:22.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.076+0000 7fc576759700 1 Processor -- start 2022-06-17T12:54:22.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.076+0000 7fc576759700 1 -- start start 2022-06-17T12:54:22.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.080+0000 7fc576759700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc5700aeea0 0x7fc5701501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:22.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.080+0000 7fc576759700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5700af930 0x7fc5701506b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:22.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.080+0000 7fc576759700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc5700afe30 0x7fc5700a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:22.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.080+0000 7fc576759700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5700a66e0 con 0x7fc5700afe30 2022-06-17T12:54:22.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.080+0000 7fc576759700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc570150d20 con 0x7fc5700af930 2022-06-17T12:54:22.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.080+0000 7fc576759700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc570150fc0 con 0x7fc5700aeea0 2022-06-17T12:54:22.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.080+0000 7fc575f58700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc5700afe30 0x7fc5700a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:22.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.080+0000 7fc575f58700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc5700afe30 0x7fc5700a5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36638/0 (socket says 172.21.15.36:36638) 2022-06-17T12:54:22.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.080+0000 7fc575f58700 1 -- 172.21.15.36:0/2157293403 learned_addr learned my addr 172.21.15.36:0/2157293403 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:22.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.080+0000 7fc574f56700 1 --2- 172.21.15.36:0/2157293403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5700af930 0x7fc5701506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:22.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.080+0000 7fc575757700 1 --2- 172.21.15.36:0/2157293403 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc5700aeea0 0x7fc5701501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:22.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.080+0000 7fc575f58700 1 -- 172.21.15.36:0/2157293403 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc5700aeea0 msgr2=0x7fc5701501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:22.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.080+0000 7fc575f58700 1 --2- 172.21.15.36:0/2157293403 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc5700aeea0 0x7fc5701501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:22.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.080+0000 7fc575f58700 1 -- 172.21.15.36:0/2157293403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5700af930 msgr2=0x7fc5701506b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:22.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.080+0000 7fc575f58700 1 --2- 172.21.15.36:0/2157293403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5700af930 0x7fc5701506b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:22.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.080+0000 7fc575f58700 1 -- 172.21.15.36:0/2157293403 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56c009470 con 0x7fc5700afe30 2022-06-17T12:54:22.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.080+0000 7fc575757700 1 --2- 172.21.15.36:0/2157293403 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc5700aeea0 0x7fc5701501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:22.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.080+0000 7fc575f58700 1 --2- 172.21.15.36:0/2157293403 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc5700afe30 0x7fc5700a5ae0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7fc568001110 tx=0x7fc5680079e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:22.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.080+0000 7fc5667fc700 1 -- 172.21.15.36:0/2157293403 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc56800cde0 con 0x7fc5700afe30 2022-06-17T12:54:22.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.080+0000 7fc5667fc700 1 -- 172.21.15.36:0/2157293403 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5680070d0 con 0x7fc5700afe30 2022-06-17T12:54:22.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.080+0000 7fc576759700 1 -- 172.21.15.36:0/2157293403 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5700a2080 con 0x7fc5700afe30 2022-06-17T12:54:22.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.080+0000 7fc5667fc700 1 -- 172.21.15.36:0/2157293403 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc568007430 con 0x7fc5700afe30 2022-06-17T12:54:22.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.080+0000 7fc576759700 1 -- 172.21.15.36:0/2157293403 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5700a26a0 con 0x7fc5700afe30 2022-06-17T12:54:22.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.080+0000 7fc576759700 1 -- 172.21.15.36:0/2157293403 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc570151180 con 0x7fc5700afe30 2022-06-17T12:54:22.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.084+0000 7fc5667fc700 1 -- 172.21.15.36:0/2157293403 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fc568010070 con 0x7fc5700afe30 2022-06-17T12:54:22.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.084+0000 7fc5667fc700 1 --2- 172.21.15.36:0/2157293403 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc5580539b0 0x7fc558055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:22.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.084+0000 7fc575757700 1 --2- 172.21.15.36:0/2157293403 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc5580539b0 0x7fc558055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:22.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.084+0000 7fc5667fc700 1 -- 172.21.15.36:0/2157293403 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc56806b660 con 0x7fc5700afe30 2022-06-17T12:54:22.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.084+0000 7fc575757700 1 --2- 172.21.15.36:0/2157293403 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc5580539b0 0x7fc558055e70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7fc56000b000 tx=0x7fc560009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:22.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.088+0000 7fc5667fc700 1 -- 172.21.15.36:0/2157293403 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc568042350 con 0x7fc5700afe30 2022-06-17T12:54:22.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.252+0000 7fc576759700 1 -- 172.21.15.36:0/2157293403 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc570057410 con 0x7fc5700afe30 2022-06-17T12:54:22.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.252+0000 7fc5667fc700 1 -- 172.21.15.36:0/2157293403 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+2140 (secure 0 0 0) 0x7fc568045970 con 0x7fc5700afe30 2022-06-17T12:54:22.240 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:22.240 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":42,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/3068692336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3068692336},{"type":"v1","addr":"172.21.15.70:6833","nonce":3068692336}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:54:12.740483+0000","modified":"2022-06-17T12:54:13.744294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6753},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6753":{"gid":6753,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/2728662009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2728662009},{"type":"v1","addr":"172.21.15.36:6835","nonce":2728662009}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-06-17T12:54:22.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.252+0000 7fc576759700 1 -- 172.21.15.36:0/2157293403 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc5580539b0 msgr2=0x7fc558055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:22.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.252+0000 7fc576759700 1 --2- 172.21.15.36:0/2157293403 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc5580539b0 0x7fc558055e70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7fc56000b000 tx=0x7fc560009040).stop 2022-06-17T12:54:22.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.252+0000 7fc576759700 1 -- 172.21.15.36:0/2157293403 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc5700afe30 msgr2=0x7fc5700a5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:22.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.252+0000 7fc576759700 1 --2- 172.21.15.36:0/2157293403 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc5700afe30 0x7fc5700a5ae0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7fc568001110 tx=0x7fc5680079e0).stop 2022-06-17T12:54:22.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.252+0000 7fc576759700 1 -- 172.21.15.36:0/2157293403 shutdown_connections 2022-06-17T12:54:22.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.252+0000 7fc576759700 1 --2- 172.21.15.36:0/2157293403 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc5580539b0 0x7fc558055e70 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:22.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.252+0000 7fc576759700 1 --2- 172.21.15.36:0/2157293403 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc5700afe30 0x7fc5700a5ae0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:22.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.252+0000 7fc576759700 1 --2- 172.21.15.36:0/2157293403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5700af930 0x7fc5701506b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:22.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.252+0000 7fc576759700 1 --2- 172.21.15.36:0/2157293403 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc5700aeea0 0x7fc5701501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:22.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.252+0000 7fc576759700 1 -- 172.21.15.36:0/2157293403 >> 172.21.15.36:0/2157293403 conn(0x7fc57005f770 msgr2=0x7fc5700b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:22.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.256+0000 7fc576759700 1 -- 172.21.15.36:0/2157293403 shutdown_connections 2022-06-17T12:54:22.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.256+0000 7fc576759700 1 -- 172.21.15.36:0/2157293403 wait complete. 2022-06-17T12:54:22.245 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 42 2022-06-17T12:54:22.253 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump inode 1 2022-06-17T12:54:22.358 INFO:teuthology.orchestra.run.smithi036.stdout:{"path":"/","ino":1,"rdev":0,"ctime":"2022-06-17T12:54:19.517457+0000","btime":"2022-06-17T12:54:12.745444+0000","mode":17407,"uid":0,"gid":0,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":4194304,"stripe_count":1,"object_size":4194304,"pool_id":16,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":0,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2022-06-17T12:54:12.745444+0000","atime":"0.000000","time_warp_seq":0,"change_attr":1,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"2022-06-17T12:54:19.517457+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"2022-06-17T12:54:12.745444+0000"},"version":3,"file_data_version":0,"xattr_version":1,"backtrace_version":3,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"request":1,"lock":3,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1,"authpin":1},"nref":9,"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":1,"num_xlocks":0,"xlock_by":{}},"authlock":{"gather_set":[],"state":"xlockdone","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":1,"xlock_by":{}},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{"gather_set":[],"state":"sync","is_leased":false,"num_rdlocks":1,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"nestlock":{},"flocklock":{},"policylock":{},"states":["auth","dirty","dirtyparent"],"client_caps":[{"client_id":6766,"pending":"pLsXsFs","issued":"pLsXsFs","wanted":"-","last_sent":2},{"client_id":6921,"pending":"pLsXsFs","issued":"pLsXsFs","wanted":"-","last_sent":2}],"loner":-1,"want_loner":-1,"mds_caps_wanted":[]} 2022-06-17T12:54:22.359 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 0, "rctime": "2022-06-17T12:54:12.745444+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "0.000000", "auth_pins": 2, "auth_state": { "replicas": {} }, "authlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "state": "xlockdone", "xlock_by": {} }, "backtrace_version": 3, "btime": "2022-06-17T12:54:12.745444+0000", "change_attr": 1, "client_caps": [ { "client_id": 6766, "issued": "pLsXsFs", "last_sent": 2, "pending": "pLsXsFs", "wanted": "-" }, { "client_id": 6921, "issued": "pLsXsFs", "last_sent": 2, "pending": "pLsXsFs", "wanted": "-" } ], "client_ranges": [], "ctime": "2022-06-17T12:54:19.517457+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 0, "ino": 1, "is_auth": true, "is_freezing": false, "is_frozen": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 4194304, "pool_id": 16, "pool_ns": "", "stripe_count": 1, "stripe_unit": 4194304 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 17407, "mtime": "2022-06-17T12:54:12.745444+0000", "nestlock": {}, "nlink": 1, "nref": 9, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/", "pins": { "authpin": 1, "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "lock": 3, "request": 1 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2022-06-17T12:54:19.517457+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": { "gather_set": [], "is_leased": false, "num_rdlocks": 1, "num_wrlocks": 0, "num_xlocks": 0, "state": "sync", "xlock_by": {} }, "states": [ "auth", "dirty", "dirtyparent" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 0, "truncate_size": 18446744073709551615, "uid": 0, "version": 3, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 1, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } 2022-06-17T12:54:22.360 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-17T12:54:22.360 DEBUG:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:22.375 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:22.396 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:22.418 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:22.439 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:22.460 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-17T12:54:22.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:22.461 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:22.461 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:54:22.469 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:54:22.471 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:54:22.471 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2022-06-17T12:54:22.472 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:54:22.472 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:54:22.527 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22516]: fuse finished with error 0 and tester_r 0 2022-06-17T12:54:22.529 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:22.530 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:22.530 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:54:22.538 INFO:teuthology.orchestra.run.smithi107.stdout:ext2/ext3 2022-06-17T12:54:22.540 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:54:22.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:54:22.657 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:54:22.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:22.710 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:22.711 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:54:22.723 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:54:22.724 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:54:22.724 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi136.front.sepia.ceph.com... 2022-06-17T12:54:22.725 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:54:22.726 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:54:22.743 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:ceph-fuse[21979]: fuse finished with error 0 and tester_r 0 2022-06-17T12:54:22.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:22.745 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:22.746 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:54:22.795 INFO:teuthology.orchestra.run.smithi136.stdout:ext2/ext3 2022-06-17T12:54:22.795 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:54:22.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:54:22.835 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:54:22.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:22.844 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:22.845 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:54:22.853 INFO:teuthology.orchestra.run.smithi143.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:54:22.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:54:22.855 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-06-17T12:54:22.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:54:22.856 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:54:22.863 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:54:22.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:54:22.866 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-17T12:54:22.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.988+0000 7f319e55d700 1 Processor -- start 2022-06-17T12:54:22.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.988+0000 7f319e55d700 1 -- start start 2022-06-17T12:54:22.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.988+0000 7f319e55d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f31980646e0 0x7f31980a9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:22.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.988+0000 7f319e55d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3198140ae0 0x7f31980b2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:22.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.988+0000 7f319e55d700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31980b6c10 con 0x7f31980b6830 2022-06-17T12:54:22.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.988+0000 7f319e55d700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3198140f00 con 0x7f31980646e0 2022-06-17T12:54:22.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.988+0000 7f319e55d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31980aa1c0 con 0x7f3198140ae0 2022-06-17T12:54:22.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.988+0000 7f31977fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f31980646e0 0x7f31980a9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:22.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.988+0000 7f31977fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f31980646e0 0x7f31980a9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36642/0 (socket says 172.21.15.36:36642) 2022-06-17T12:54:22.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.988+0000 7f31977fe700 1 -- 172.21.15.36:0/1241888284 learned_addr learned my addr 172.21.15.36:0/1241888284 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:22.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.988+0000 7f319cafa700 1 --2- 172.21.15.36:0/1241888284 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3198140ae0 0x7f31980b2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:22.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.988+0000 7f31977fe700 1 -- 172.21.15.36:0/1241888284 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3198140ae0 msgr2=0x7f31980b2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:22.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.988+0000 7f31977fe700 1 --2- 172.21.15.36:0/1241888284 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3198140ae0 0x7f31980b2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:22.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.988+0000 7f31977fe700 1 -- 172.21.15.36:0/1241888284 >> v1:172.21.15.70:6789/0 conn(0x7f31980b6830 legacy=0x7f31981403d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:22.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.988+0000 7f31977fe700 1 -- 172.21.15.36:0/1241888284 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31980b2d70 con 0x7f31980646e0 2022-06-17T12:54:22.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.992+0000 7f31977fe700 1 --2- 172.21.15.36:0/1241888284 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f31980646e0 0x7f31980a9b90 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7f3180002f70 tx=0x7f3180006eb0).ready entity=mon.0 client_cookie=f9f7c8820d3e4e1e server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:22.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.992+0000 7f3196ffd700 1 -- 172.21.15.36:0/1241888284 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f318000d5c0 con 0x7f31980646e0 2022-06-17T12:54:22.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.992+0000 7f3196ffd700 1 -- 172.21.15.36:0/1241888284 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3180000e90 con 0x7f31980646e0 2022-06-17T12:54:22.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.992+0000 7f3196ffd700 1 -- 172.21.15.36:0/1241888284 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f318000d950 con 0x7f31980646e0 2022-06-17T12:54:22.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.992+0000 7f319e55d700 1 -- 172.21.15.36:0/1241888284 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f31980646e0 msgr2=0x7f31980a9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:22.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.992+0000 7f319e55d700 1 --2- 172.21.15.36:0/1241888284 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f31980646e0 0x7f31980a9b90 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7f3180002f70 tx=0x7f3180006eb0).stop 2022-06-17T12:54:22.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.992+0000 7f319e55d700 1 -- 172.21.15.36:0/1241888284 shutdown_connections 2022-06-17T12:54:22.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.992+0000 7f319e55d700 1 --2- 172.21.15.36:0/1241888284 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3198140ae0 0x7f31980b2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:22.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.992+0000 7f319e55d700 1 --2- 172.21.15.36:0/1241888284 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f31980646e0 0x7f31980a9b90 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:22.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.992+0000 7f319e55d700 1 -- 172.21.15.36:0/1241888284 >> 172.21.15.36:0/1241888284 conn(0x7f31980636d0 msgr2=0x7f3198063ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:22.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.992+0000 7f319e55d700 1 -- 172.21.15.36:0/1241888284 shutdown_connections 2022-06-17T12:54:22.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.992+0000 7f319e55d700 1 -- 172.21.15.36:0/1241888284 wait complete. 2022-06-17T12:54:22.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.992+0000 7f319e55d700 1 Processor -- start 2022-06-17T12:54:22.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.992+0000 7f319e55d700 1 -- start start 2022-06-17T12:54:22.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.996+0000 7f319e55d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31980646e0 0x7f319813dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:22.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.996+0000 7f319e55d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f31980b6830 0x7f3198153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:22.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.996+0000 7f319e55d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3198140ae0 0x7f31980667d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:22.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.996+0000 7f319e55d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31980b62f0 con 0x7f31980b6830 2022-06-17T12:54:22.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.996+0000 7f319e55d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3198153e00 con 0x7f31980646e0 2022-06-17T12:54:22.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.996+0000 7f319e55d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3198064e30 con 0x7f3198140ae0 2022-06-17T12:54:22.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.996+0000 7f31977fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f31980b6830 0x7f3198153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:22.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.996+0000 7f3197fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31980646e0 0x7f319813dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:22.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.996+0000 7f31977fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f31980b6830 0x7f3198153910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36650/0 (socket says 172.21.15.36:36650) 2022-06-17T12:54:22.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.996+0000 7f31977fe700 1 -- 172.21.15.36:0/3019369980 learned_addr learned my addr 172.21.15.36:0/3019369980 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:22.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.996+0000 7f319cafa700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3198140ae0 0x7f31980667d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:22.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.996+0000 7f31977fe700 1 -- 172.21.15.36:0/3019369980 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3198140ae0 msgr2=0x7f31980667d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:22.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.996+0000 7f31977fe700 1 --2- 172.21.15.36:0/3019369980 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3198140ae0 0x7f31980667d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:22.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.996+0000 7f31977fe700 1 -- 172.21.15.36:0/3019369980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31980646e0 msgr2=0x7f319813dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:22.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.996+0000 7f31977fe700 1 --2- 172.21.15.36:0/3019369980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31980646e0 0x7f319813dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:22.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.996+0000 7f31977fe700 1 -- 172.21.15.36:0/3019369980 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318000b1e0 con 0x7f31980b6830 2022-06-17T12:54:22.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.996+0000 7f31977fe700 1 --2- 172.21.15.36:0/3019369980 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f31980b6830 0x7f3198153910 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7f3180002f70 tx=0x7f3180015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:22.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.996+0000 7f31957fa700 1 -- 172.21.15.36:0/3019369980 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31800117e0 con 0x7f31980b6830 2022-06-17T12:54:22.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.996+0000 7f31957fa700 1 -- 172.21.15.36:0/3019369980 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3180011940 con 0x7f31980b6830 2022-06-17T12:54:22.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.996+0000 7f319e55d700 1 -- 172.21.15.36:0/3019369980 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31980650d0 con 0x7f31980b6830 2022-06-17T12:54:22.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.996+0000 7f31957fa700 1 -- 172.21.15.36:0/3019369980 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3180011ca0 con 0x7f31980b6830 2022-06-17T12:54:22.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.996+0000 7f319e55d700 1 -- 172.21.15.36:0/3019369980 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3198064b20 con 0x7f31980b6830 2022-06-17T12:54:22.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:22.996+0000 7f319e55d700 1 -- 172.21.15.36:0/3019369980 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31980b46a0 con 0x7f31980b6830 2022-06-17T12:54:22.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.000+0000 7f31957fa700 1 -- 172.21.15.36:0/3019369980 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f3180025030 con 0x7f31980b6830 2022-06-17T12:54:22.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.000+0000 7f31957fa700 1 --2- 172.21.15.36:0/3019369980 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f317c053ad0 0x7f317c055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:22.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.000+0000 7f31957fa700 1 -- 172.21.15.36:0/3019369980 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f318006d1e0 con 0x7f31980b6830 2022-06-17T12:54:22.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.000+0000 7f3197fff700 1 --2- 172.21.15.36:0/3019369980 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f317c053ad0 0x7f317c055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:22.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.000+0000 7f3197fff700 1 --2- 172.21.15.36:0/3019369980 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f317c053ad0 0x7f317c055f90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f318800d000 tx=0x7f3188007b50).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:22.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.004+0000 7f31957fa700 1 -- 172.21.15.36:0/3019369980 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3180043f50 con 0x7f31980b6830 2022-06-17T12:54:23.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.136+0000 7f319e55d700 1 -- 172.21.15.36:0/3019369980 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f31980ac7a0 con 0x7f31980b6830 2022-06-17T12:54:23.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.136+0000 7f31957fa700 1 -- 172.21.15.36:0/3019369980 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v84) v1 ==== 81+0+34040 (secure 0 0 0) 0x7f3180023090 con 0x7f31980b6830 2022-06-17T12:54:23.125 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:23.126 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:54:23.127 INFO:teuthology.orchestra.run.smithi036.stdout: "epoch": 84, 2022-06-17T12:54:23.127 INFO:teuthology.orchestra.run.smithi036.stdout: "fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", 2022-06-17T12:54:23.127 INFO:teuthology.orchestra.run.smithi036.stdout: "created": "2022-06-17T12:46:30.671408+0000", 2022-06-17T12:54:23.128 INFO:teuthology.orchestra.run.smithi036.stdout: "modified": "2022-06-17T12:54:13.745105+0000", 2022-06-17T12:54:23.128 INFO:teuthology.orchestra.run.smithi036.stdout: "last_up_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:54:23.129 INFO:teuthology.orchestra.run.smithi036.stdout: "last_in_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:54:23.129 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-17T12:54:23.129 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_num": 5799936, 2022-06-17T12:54:23.134 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_set": [ 2022-06-17T12:54:23.134 INFO:teuthology.orchestra.run.smithi036.stdout: "pglog_hardlimit", 2022-06-17T12:54:23.134 INFO:teuthology.orchestra.run.smithi036.stdout: "purged_snapdirs", 2022-06-17T12:54:23.135 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_deletes", 2022-06-17T12:54:23.135 INFO:teuthology.orchestra.run.smithi036.stdout: "sortbitwise" 2022-06-17T12:54:23.135 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:54:23.137 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_version": 6, 2022-06-17T12:54:23.138 INFO:teuthology.orchestra.run.smithi036.stdout: "full_ratio": 0.94999998807907104, 2022-06-17T12:54:23.138 INFO:teuthology.orchestra.run.smithi036.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-17T12:54:23.138 INFO:teuthology.orchestra.run.smithi036.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-17T12:54:23.139 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_snapshot": "", 2022-06-17T12:54:23.139 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_max": 16, 2022-06-17T12:54:23.139 INFO:teuthology.orchestra.run.smithi036.stdout: "max_osd": 8, 2022-06-17T12:54:23.139 INFO:teuthology.orchestra.run.smithi036.stdout: "require_min_compat_client": "jewel", 2022-06-17T12:54:23.140 INFO:teuthology.orchestra.run.smithi036.stdout: "min_compat_client": "jewel", 2022-06-17T12:54:23.140 INFO:teuthology.orchestra.run.smithi036.stdout: "require_osd_release": "octopus", 2022-06-17T12:54:23.140 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:54:23.141 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.141 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 1, 2022-06-17T12:54:23.141 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "device_health_metrics", 2022-06-17T12:54:23.142 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:33.881212+0000", 2022-06-17T12:54:23.142 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:54:23.142 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:54:23.143 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:54:23.143 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:54:23.143 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:54:23.143 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:54:23.144 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:54:23.144 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:54:23.144 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 1, 2022-06-17T12:54:23.145 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 1, 2022-06-17T12:54:23.145 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 1, 2022-06-17T12:54:23.145 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 1, 2022-06-17T12:54:23.146 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 1, 2022-06-17T12:54:23.146 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:54:23.146 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:54:23.146 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:54:23.147 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:54:23.147 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:54:23.147 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:54:23.148 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:54:23.148 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.148 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "16", 2022-06-17T12:54:23.149 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:54:23.149 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:54:23.149 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:54:23.150 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:54:23.150 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:54:23.150 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:54:23.151 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:54:23.151 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:54:23.151 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:54:23.151 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:54:23.152 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:54:23.152 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:54:23.152 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:54:23.153 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:54:23.153 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:54:23.153 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:54:23.154 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:54:23.154 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:54:23.154 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:54:23.154 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:54:23.155 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:54:23.155 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:54:23.156 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:54:23.156 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:54:23.156 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:54:23.157 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:54:23.158 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.158 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:54:23.159 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:54:23.159 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:54:23.160 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:54:23.160 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:54:23.160 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:54:23.161 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:54:23.161 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:54:23.162 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:54:23.162 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:54:23.163 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:54:23.163 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:54:23.164 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 1 2022-06-17T12:54:23.165 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.166 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:54:23.166 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr_devicehealth": {} 2022-06-17T12:54:23.167 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.167 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.167 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.167 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 2, 2022-06-17T12:54:23.168 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "rbd", 2022-06-17T12:54:23.168 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:44.485460+0000", 2022-06-17T12:54:23.168 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:54:23.168 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:54:23.169 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:54:23.169 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:54:23.169 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:54:23.169 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:54:23.170 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:54:23.170 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:54:23.170 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:54:23.171 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:54:23.171 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:54:23.171 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:54:23.172 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:54:23.172 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:54:23.172 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:54:23.172 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:54:23.173 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:54:23.173 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:54:23.173 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:54:23.173 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:54:23.174 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.174 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "19", 2022-06-17T12:54:23.174 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:54:23.174 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:54:23.175 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:54:23.175 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:54:23.176 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:54:23.176 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:54:23.176 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:54:23.176 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:54:23.177 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:54:23.177 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:54:23.177 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:54:23.177 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:54:23.178 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:54:23.178 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:54:23.178 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:54:23.178 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:54:23.179 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:54:23.179 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:54:23.179 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:54:23.180 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:54:23.180 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:54:23.181 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:54:23.181 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:54:23.182 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:54:23.182 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:54:23.183 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:54:23.183 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.184 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:54:23.184 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:54:23.184 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:54:23.185 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:54:23.186 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:54:23.186 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:54:23.187 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:54:23.187 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:54:23.187 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:54:23.188 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:54:23.188 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:54:23.189 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:54:23.189 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:54:23.190 INFO:teuthology.orchestra.run.smithi036.stdout: "rbd": {} 2022-06-17T12:54:23.190 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.191 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.191 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.192 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 15, 2022-06-17T12:54:23.192 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_metadata", 2022-06-17T12:54:23.193 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:54:11.237500+0000", 2022-06-17T12:54:23.193 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:54:23.194 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:54:23.195 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:54:23.195 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:54:23.196 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:54:23.196 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:54:23.196 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:54:23.197 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:54:23.197 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:54:23.198 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:54:23.198 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:54:23.199 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:54:23.199 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:54:23.200 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:54:23.200 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:54:23.201 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:54:23.201 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:54:23.202 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:54:23.202 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:54:23.203 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:54:23.203 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.203 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "83", 2022-06-17T12:54:23.204 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:54:23.204 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:54:23.205 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:54:23.205 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:54:23.206 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:54:23.206 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:54:23.206 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:54:23.207 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:54:23.207 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:54:23.208 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:54:23.208 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:54:23.209 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:54:23.209 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:54:23.210 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:54:23.210 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:54:23.211 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:54:23.211 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:54:23.211 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:54:23.212 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:54:23.212 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:54:23.213 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:54:23.213 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:54:23.214 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:54:23.214 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:54:23.215 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:54:23.215 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:54:23.215 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.216 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:54:23.216 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:54:23.217 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:54:23.217 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:54:23.218 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:54:23.218 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:54:23.219 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:54:23.219 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:54:23.220 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:54:23.220 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:54:23.220 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:54:23.221 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:54:23.221 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_bias": 4, 2022-06-17T12:54:23.221 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 16, 2022-06-17T12:54:23.221 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_priority": 5 2022-06-17T12:54:23.222 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.222 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:54:23.223 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:54:23.223 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": "cephfs" 2022-06-17T12:54:23.223 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.224 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.224 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.225 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.225 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 16, 2022-06-17T12:54:23.226 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_data", 2022-06-17T12:54:23.226 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:54:11.985387+0000", 2022-06-17T12:54:23.227 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:54:23.227 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:54:23.227 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:54:23.228 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:54:23.228 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:54:23.228 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:54:23.228 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:54:23.229 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:54:23.229 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:54:23.229 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:54:23.229 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:54:23.229 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:54:23.230 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:54:23.230 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:54:23.230 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:54:23.230 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:54:23.231 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:54:23.231 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:54:23.231 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:54:23.231 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:54:23.231 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.232 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "84", 2022-06-17T12:54:23.232 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:54:23.232 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:54:23.232 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:54:23.233 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:54:23.233 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:54:23.233 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:54:23.233 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:54:23.234 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:54:23.234 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:54:23.234 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:54:23.234 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:54:23.235 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:54:23.235 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:54:23.235 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:54:23.235 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:54:23.235 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:54:23.236 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:54:23.236 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:54:23.236 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:54:23.236 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:54:23.237 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:54:23.237 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:54:23.237 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:54:23.237 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:54:23.238 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:54:23.238 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:54:23.238 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.238 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:54:23.238 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:54:23.239 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:54:23.239 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:54:23.239 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:54:23.239 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:54:23.240 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:54:23.240 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:54:23.240 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:54:23.240 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:54:23.241 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:54:23.241 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:54:23.241 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:54:23.241 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:54:23.242 INFO:teuthology.orchestra.run.smithi036.stdout: "data": "cephfs" 2022-06-17T12:54:23.242 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.242 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.243 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.243 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:54:23.244 INFO:teuthology.orchestra.run.smithi036.stdout: "osds": [ 2022-06-17T12:54:23.244 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.244 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:54:23.245 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", 2022-06-17T12:54:23.245 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:23.245 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:23.246 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:23.247 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:23.247 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:23.248 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:23.248 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:54:23.249 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 82, 2022-06-17T12:54:23.249 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:23.250 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:23.250 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:23.251 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.251 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.252 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.252 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6802", 2022-06-17T12:54:23.252 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:23.253 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.253 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.253 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.254 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6803", 2022-06-17T12:54:23.254 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:23.255 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.255 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.255 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.255 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:23.256 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.256 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.256 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.257 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6804", 2022-06-17T12:54:23.257 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:23.257 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.258 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.258 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.258 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6805", 2022-06-17T12:54:23.259 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:23.259 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.260 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.260 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.261 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:23.261 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.261 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.262 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.262 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6808", 2022-06-17T12:54:23.263 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:23.263 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.264 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.264 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.264 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6809", 2022-06-17T12:54:23.265 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:23.265 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.265 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.266 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.266 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:23.266 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.266 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.267 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.267 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6806", 2022-06-17T12:54:23.267 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:23.267 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.268 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.268 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.268 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6807", 2022-06-17T12:54:23.269 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:23.269 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.269 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.269 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.270 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6803/22619", 2022-06-17T12:54:23.270 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6805/22619", 2022-06-17T12:54:23.270 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6809/22619", 2022-06-17T12:54:23.271 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6807/22619", 2022-06-17T12:54:23.271 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:23.272 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:23.272 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:23.273 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.273 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.274 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.274 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:54:23.275 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", 2022-06-17T12:54:23.275 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:23.275 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:23.276 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:23.276 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:23.277 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:23.277 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:23.277 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:54:23.278 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 81, 2022-06-17T12:54:23.278 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:23.278 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:23.279 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:23.279 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.280 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.280 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.280 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6810", 2022-06-17T12:54:23.281 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:23.281 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.281 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.282 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.282 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6811", 2022-06-17T12:54:23.282 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:23.282 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.283 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.283 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.283 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:23.284 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.284 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.284 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.285 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6812", 2022-06-17T12:54:23.285 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:23.285 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.285 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.286 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.286 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6813", 2022-06-17T12:54:23.286 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:23.286 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.287 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.287 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.287 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:23.287 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.288 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.288 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.288 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6816", 2022-06-17T12:54:23.288 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:23.288 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.289 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.289 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.289 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6817", 2022-06-17T12:54:23.289 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:23.290 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.290 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.290 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.290 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:23.291 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.291 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.291 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.291 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6814", 2022-06-17T12:54:23.291 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:23.292 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.292 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.292 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.292 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6815", 2022-06-17T12:54:23.293 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:23.293 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.294 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.294 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.294 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6811/22620", 2022-06-17T12:54:23.295 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6813/22620", 2022-06-17T12:54:23.296 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6817/22620", 2022-06-17T12:54:23.296 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6815/22620", 2022-06-17T12:54:23.296 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:23.296 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:23.297 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:23.297 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.297 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.297 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.298 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:54:23.298 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", 2022-06-17T12:54:23.298 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:23.298 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:23.299 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:23.299 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:23.299 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:23.299 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:23.300 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:54:23.300 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 82, 2022-06-17T12:54:23.300 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:23.300 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:23.300 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:23.301 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.301 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.301 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.301 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6826", 2022-06-17T12:54:23.302 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:23.302 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.302 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.303 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.309 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6827", 2022-06-17T12:54:23.310 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:23.311 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.311 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.311 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.312 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:23.312 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.312 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.313 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.313 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6828", 2022-06-17T12:54:23.313 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:23.314 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.314 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.315 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.315 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6829", 2022-06-17T12:54:23.315 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:23.316 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.316 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.316 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.317 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:23.317 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.317 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.318 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.318 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6832", 2022-06-17T12:54:23.318 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:23.319 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.319 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.320 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.320 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6833", 2022-06-17T12:54:23.320 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:23.321 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.321 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.322 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.322 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:23.323 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.323 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.323 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.324 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6830", 2022-06-17T12:54:23.324 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:23.325 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.325 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.325 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.326 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6831", 2022-06-17T12:54:23.326 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:23.327 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.327 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.327 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.327 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6827/22643", 2022-06-17T12:54:23.328 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6829/22643", 2022-06-17T12:54:23.328 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6833/22643", 2022-06-17T12:54:23.328 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6831/22643", 2022-06-17T12:54:23.328 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:23.328 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:23.329 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:23.329 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.329 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.329 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.330 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:54:23.330 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", 2022-06-17T12:54:23.330 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:23.330 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:23.331 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:23.331 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:23.331 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:23.331 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:23.332 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:54:23.332 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 82, 2022-06-17T12:54:23.332 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:23.332 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:23.333 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:23.333 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.333 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.333 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.333 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6818", 2022-06-17T12:54:23.334 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:23.334 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.334 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.335 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.335 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6819", 2022-06-17T12:54:23.335 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:23.335 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.336 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.336 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.336 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:23.337 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.338 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.338 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.338 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6820", 2022-06-17T12:54:23.339 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:23.339 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.339 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.340 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.340 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6821", 2022-06-17T12:54:23.340 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:23.340 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.341 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.341 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.341 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:23.341 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.342 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.342 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.342 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6824", 2022-06-17T12:54:23.342 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:23.342 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.343 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.343 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.343 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6825", 2022-06-17T12:54:23.343 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:23.344 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.344 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.345 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.345 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:23.345 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.345 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.345 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.346 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6822", 2022-06-17T12:54:23.346 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:23.346 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.346 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.347 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.347 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6823", 2022-06-17T12:54:23.347 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:23.347 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.348 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.348 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.348 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6819/22646", 2022-06-17T12:54:23.348 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6821/22646", 2022-06-17T12:54:23.349 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6825/22646", 2022-06-17T12:54:23.349 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6823/22646", 2022-06-17T12:54:23.349 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:23.349 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:23.350 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:23.350 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.350 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.350 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.351 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:54:23.351 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", 2022-06-17T12:54:23.351 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:23.351 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:23.352 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:23.352 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:23.352 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:23.352 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:23.352 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:54:23.353 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 82, 2022-06-17T12:54:23.353 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:23.353 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:23.353 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:23.354 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.354 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.354 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.354 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6808", 2022-06-17T12:54:23.355 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:23.355 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.355 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.355 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.355 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6809", 2022-06-17T12:54:23.356 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:23.356 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.356 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.356 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.357 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:23.357 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.357 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.357 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.358 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6810", 2022-06-17T12:54:23.358 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:23.358 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.358 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.359 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.359 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6811", 2022-06-17T12:54:23.359 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:23.359 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.359 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.360 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.360 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:23.360 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.360 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.361 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.361 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6814", 2022-06-17T12:54:23.361 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:23.361 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.362 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.362 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.362 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6815", 2022-06-17T12:54:23.362 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:23.363 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.363 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.363 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.364 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:23.364 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.364 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.365 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.365 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6812", 2022-06-17T12:54:23.365 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:23.365 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.366 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.366 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.366 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6813", 2022-06-17T12:54:23.366 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:23.367 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.367 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.367 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.367 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6809/22812", 2022-06-17T12:54:23.368 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6811/22812", 2022-06-17T12:54:23.368 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6815/22812", 2022-06-17T12:54:23.368 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6813/22812", 2022-06-17T12:54:23.368 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:23.368 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:23.369 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:23.369 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.369 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.369 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.370 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:54:23.370 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", 2022-06-17T12:54:23.370 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:23.370 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:23.370 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:23.371 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:23.371 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:23.371 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:23.372 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:54:23.372 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 82, 2022-06-17T12:54:23.372 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:23.372 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:23.373 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:23.373 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.373 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.373 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.373 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6800", 2022-06-17T12:54:23.374 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:23.374 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.374 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.375 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.375 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6801", 2022-06-17T12:54:23.375 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:23.375 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.376 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.376 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.376 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:23.377 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.377 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.377 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.377 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6802", 2022-06-17T12:54:23.378 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:23.378 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.378 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.379 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.379 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6803", 2022-06-17T12:54:23.379 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:23.379 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.380 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.380 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.380 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:23.380 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.380 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.381 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.381 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6806", 2022-06-17T12:54:23.381 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:23.381 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.382 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.382 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.382 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6807", 2022-06-17T12:54:23.382 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:23.383 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.383 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.383 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.383 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:23.384 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.384 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.384 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.384 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6804", 2022-06-17T12:54:23.385 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:23.385 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.385 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.385 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.386 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6805", 2022-06-17T12:54:23.386 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:23.386 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.386 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.387 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.387 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6801/22813", 2022-06-17T12:54:23.387 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6803/22813", 2022-06-17T12:54:23.387 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6807/22813", 2022-06-17T12:54:23.387 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6805/22813", 2022-06-17T12:54:23.388 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:23.388 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:23.389 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:23.389 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.389 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.390 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.390 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:54:23.390 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", 2022-06-17T12:54:23.390 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:23.391 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:23.391 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:23.391 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:23.391 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:23.392 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:23.392 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:54:23.392 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 82, 2022-06-17T12:54:23.392 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:23.393 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:23.393 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:23.393 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.393 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.394 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.394 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6816", 2022-06-17T12:54:23.394 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:23.394 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.394 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.395 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.395 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6817", 2022-06-17T12:54:23.395 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:23.395 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.396 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.396 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.396 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:23.397 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.397 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.397 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.397 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6818", 2022-06-17T12:54:23.397 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:23.398 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.398 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.398 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.398 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6819", 2022-06-17T12:54:23.399 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:23.399 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.399 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.399 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.399 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:23.400 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.400 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.400 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.400 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6822", 2022-06-17T12:54:23.401 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:23.401 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.401 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.401 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.401 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6823", 2022-06-17T12:54:23.402 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:23.402 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.402 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.402 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.403 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:23.403 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.403 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.403 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.404 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6820", 2022-06-17T12:54:23.404 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:23.404 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.404 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.405 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.405 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6821", 2022-06-17T12:54:23.405 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:23.405 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.406 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.406 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.406 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6817/22822", 2022-06-17T12:54:23.406 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6819/22822", 2022-06-17T12:54:23.407 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6823/22822", 2022-06-17T12:54:23.407 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6821/22822", 2022-06-17T12:54:23.407 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:23.407 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:23.408 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:23.408 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.408 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.408 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.408 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:54:23.409 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", 2022-06-17T12:54:23.409 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:23.409 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:23.409 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:23.410 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:23.410 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:23.410 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:23.410 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:54:23.411 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 71, 2022-06-17T12:54:23.411 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:23.411 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:23.411 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:23.412 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.412 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.412 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.412 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6824", 2022-06-17T12:54:23.412 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:23.413 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.413 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.413 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.413 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6825", 2022-06-17T12:54:23.413 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:23.414 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.414 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.414 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.414 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:23.415 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.415 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.415 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.415 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6826", 2022-06-17T12:54:23.415 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:23.416 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.416 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.416 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.416 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6827", 2022-06-17T12:54:23.417 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:23.417 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.417 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.417 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.418 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:23.418 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.418 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.419 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.419 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6830", 2022-06-17T12:54:23.419 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:23.419 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.420 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.420 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.420 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6831", 2022-06-17T12:54:23.420 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:23.421 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.421 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.421 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.421 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:23.422 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:23.422 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.422 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:23.422 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6828", 2022-06-17T12:54:23.423 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:23.423 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.423 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.423 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:23.424 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6829", 2022-06-17T12:54:23.424 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:23.424 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.424 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.425 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.425 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6825/22837", 2022-06-17T12:54:23.425 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6827/22837", 2022-06-17T12:54:23.425 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6831/22837", 2022-06-17T12:54:23.425 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6829/22837", 2022-06-17T12:54:23.426 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:23.426 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:23.426 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:23.426 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:23.427 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.427 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:54:23.427 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_xinfo": [ 2022-06-17T12:54:23.427 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.428 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:54:23.428 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:23.428 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:23.429 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:23.429 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:23.429 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:23.429 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.291461+0000", 2022-06-17T12:54:23.429 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:23.430 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.430 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.430 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:54:23.431 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:23.431 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:23.431 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:23.431 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:23.432 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:23.432 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.537912+0000", 2022-06-17T12:54:23.432 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:23.433 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.433 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.433 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:54:23.433 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:23.433 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:23.434 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:23.434 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:23.434 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:23.434 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.606947+0000", 2022-06-17T12:54:23.435 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:23.435 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.435 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.435 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:54:23.436 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:23.436 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:23.436 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:23.436 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:23.437 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:23.437 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.514412+0000", 2022-06-17T12:54:23.437 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:23.437 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.437 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.438 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:54:23.438 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:23.438 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:23.439 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:23.439 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:23.439 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:23.439 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.394645+0000", 2022-06-17T12:54:23.440 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:23.440 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.440 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.440 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:54:23.440 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:23.441 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:23.441 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:23.441 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:23.441 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:23.442 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.343806+0000", 2022-06-17T12:54:23.442 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:23.442 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.442 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.443 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:54:23.443 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:23.443 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:23.443 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:23.443 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:23.444 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:23.444 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.591864+0000", 2022-06-17T12:54:23.444 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:23.444 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.445 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:23.445 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:54:23.445 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:23.446 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:23.446 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:23.446 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:23.446 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:23.447 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.607045+0000", 2022-06-17T12:54:23.447 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:23.447 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.447 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:54:23.448 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap": [], 2022-06-17T12:54:23.448 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap_items": [], 2022-06-17T12:54:23.448 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_temp": [], 2022-06-17T12:54:23.449 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_temp": [], 2022-06-17T12:54:23.449 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklist": {}, 2022-06-17T12:54:23.449 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profiles": { 2022-06-17T12:54:23.449 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs_data": { 2022-06-17T12:54:23.450 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-device-class": "", 2022-06-17T12:54:23.450 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:54:23.450 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-root": "default", 2022-06-17T12:54:23.450 INFO:teuthology.orchestra.run.smithi036.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-17T12:54:23.451 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:54:23.451 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "2", 2022-06-17T12:54:23.451 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:54:23.451 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:54:23.451 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van", 2022-06-17T12:54:23.452 INFO:teuthology.orchestra.run.smithi036.stdout: "w": "8" 2022-06-17T12:54:23.452 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.452 INFO:teuthology.orchestra.run.smithi036.stdout: "default": { 2022-06-17T12:54:23.452 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:54:23.453 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:54:23.453 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "1", 2022-06-17T12:54:23.453 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:54:23.453 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:54:23.454 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van" 2022-06-17T12:54:23.454 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:23.454 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:23.454 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps_queue": [], 2022-06-17T12:54:23.455 INFO:teuthology.orchestra.run.smithi036.stdout: "new_removed_snaps": [], 2022-06-17T12:54:23.455 INFO:teuthology.orchestra.run.smithi036.stdout: "new_purged_snaps": [], 2022-06-17T12:54:23.455 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_node_flags": {}, 2022-06-17T12:54:23.455 INFO:teuthology.orchestra.run.smithi036.stdout: "device_class_flags": {} 2022-06-17T12:54:23.455 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:54:23.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.136+0000 7f319e55d700 1 -- 172.21.15.36:0/3019369980 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f317c053ad0 msgr2=0x7f317c055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:23.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.136+0000 7f319e55d700 1 --2- 172.21.15.36:0/3019369980 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f317c053ad0 0x7f317c055f90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f318800d000 tx=0x7f3188007b50).stop 2022-06-17T12:54:23.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.136+0000 7f319e55d700 1 -- 172.21.15.36:0/3019369980 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f31980b6830 msgr2=0x7f3198153910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:23.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.136+0000 7f319e55d700 1 --2- 172.21.15.36:0/3019369980 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f31980b6830 0x7f3198153910 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7f3180002f70 tx=0x7f3180015000).stop 2022-06-17T12:54:23.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.136+0000 7f319e55d700 1 -- 172.21.15.36:0/3019369980 shutdown_connections 2022-06-17T12:54:23.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.136+0000 7f319e55d700 1 --2- 172.21.15.36:0/3019369980 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f317c053ad0 0x7f317c055f90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:23.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.136+0000 7f319e55d700 1 --2- 172.21.15.36:0/3019369980 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3198140ae0 0x7f31980667d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:23.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.136+0000 7f319e55d700 1 --2- 172.21.15.36:0/3019369980 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f31980b6830 0x7f3198153910 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:23.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.136+0000 7f319e55d700 1 --2- 172.21.15.36:0/3019369980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31980646e0 0x7f319813dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:23.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.136+0000 7f319e55d700 1 -- 172.21.15.36:0/3019369980 >> 172.21.15.36:0/3019369980 conn(0x7f31980636d0 msgr2=0x7f319805ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:23.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.136+0000 7f319e55d700 1 -- 172.21.15.36:0/3019369980 shutdown_connections 2022-06-17T12:54:23.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.140+0000 7f319e55d700 1 -- 172.21.15.36:0/3019369980 wait complete. 2022-06-17T12:54:23.461 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:54:23.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.588+0000 7f637ea28700 1 Processor -- start 2022-06-17T12:54:23.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.588+0000 7f637ea28700 1 -- start start 2022-06-17T12:54:23.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637ea28700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6378062470 0x7f63780b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:23.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637ea28700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6378143a90 0x7f6378144390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:23.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637ea28700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63780b83f0 con 0x7f63780b4820 2022-06-17T12:54:23.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637ea28700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6378142ec0 con 0x7f6378143a90 2022-06-17T12:54:23.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637ea28700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6378143ec0 con 0x7f6378062470 2022-06-17T12:54:23.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637da26700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f63780b4820 0x7f63780b7ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54188/0 (socket says 172.21.15.36:54188) 2022-06-17T12:54:23.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637da26700 1 -- 172.21.15.36:0/1216912670 learned_addr learned my addr 172.21.15.36:0/1216912670 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:23.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637d225700 1 --2- 172.21.15.36:0/1216912670 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6378062470 0x7f63780b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:23.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637e227700 1 --2- 172.21.15.36:0/1216912670 >> v2:172.21.15.70:3300/0 conn(0x7f6378143a90 0x7f6378144390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:23.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637d225700 1 -- 172.21.15.36:0/1216912670 >> v2:172.21.15.70:3300/0 conn(0x7f6378143a90 msgr2=0x7f6378144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:23.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637d225700 1 --2- 172.21.15.36:0/1216912670 >> v2:172.21.15.70:3300/0 conn(0x7f6378143a90 0x7f6378144390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:23.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637d225700 1 -- 172.21.15.36:0/1216912670 >> v1:172.21.15.36:6789/0 conn(0x7f63780b4820 legacy=0x7f63780b7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:23.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637d225700 1 -- 172.21.15.36:0/1216912670 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63780b1ff0 con 0x7f6378062470 2022-06-17T12:54:23.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637d225700 1 --2- 172.21.15.36:0/1216912670 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6378062470 0x7f63780b4c00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f6374002f70 tx=0x7f6374006df0).ready entity=mon.2 client_cookie=ec6d0f94a6834313 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:23.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637ca24700 1 -- 172.21.15.36:0/1216912670 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f637400d570 con 0x7f6378062470 2022-06-17T12:54:23.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637ca24700 1 -- 172.21.15.36:0/1216912670 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6374000e90 con 0x7f6378062470 2022-06-17T12:54:23.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637ca24700 1 -- 172.21.15.36:0/1216912670 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f637400d8c0 con 0x7f6378062470 2022-06-17T12:54:23.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637ea28700 1 -- 172.21.15.36:0/1216912670 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6378062470 msgr2=0x7f63780b4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:23.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637ea28700 1 --2- 172.21.15.36:0/1216912670 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6378062470 0x7f63780b4c00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f6374002f70 tx=0x7f6374006df0).stop 2022-06-17T12:54:23.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637ea28700 1 -- 172.21.15.36:0/1216912670 shutdown_connections 2022-06-17T12:54:23.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637ea28700 1 --2- 172.21.15.36:0/1216912670 >> v2:172.21.15.70:3300/0 conn(0x7f6378143a90 0x7f6378144390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:23.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637ea28700 1 --2- 172.21.15.36:0/1216912670 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6378062470 0x7f63780b4c00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:23.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637ea28700 1 -- 172.21.15.36:0/1216912670 >> 172.21.15.36:0/1216912670 conn(0x7f637805c550 msgr2=0x7f63780b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:23.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637ea28700 1 -- 172.21.15.36:0/1216912670 shutdown_connections 2022-06-17T12:54:23.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.592+0000 7f637ea28700 1 -- 172.21.15.36:0/1216912670 wait complete. 2022-06-17T12:54:23.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f637ea28700 1 Processor -- start 2022-06-17T12:54:23.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f637ea28700 1 -- start start 2022-06-17T12:54:23.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f637ea28700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6378062470 0x7f637813e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:23.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f637ea28700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f63780b4820 0x7f6378142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:23.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f637ea28700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6378143a90 0x7f6378142740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:23.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f637ea28700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6378148240 con 0x7f63780b4820 2022-06-17T12:54:23.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f637ea28700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6378067150 con 0x7f6378143a90 2022-06-17T12:54:23.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f637ea28700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6378152d90 con 0x7f6378062470 2022-06-17T12:54:23.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f637d225700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f63780b4820 0x7f6378142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:23.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f637d225700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f63780b4820 0x7f6378142250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36662/0 (socket says 172.21.15.36:36662) 2022-06-17T12:54:23.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f637d225700 1 -- 172.21.15.36:0/3933657865 learned_addr learned my addr 172.21.15.36:0/3933657865 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:23.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f637e227700 1 --2- 172.21.15.36:0/3933657865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6378143a90 0x7f6378142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:23.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f637da26700 1 --2- 172.21.15.36:0/3933657865 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6378062470 0x7f637813e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:23.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f637d225700 1 -- 172.21.15.36:0/3933657865 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6378062470 msgr2=0x7f637813e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:23.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f637d225700 1 --2- 172.21.15.36:0/3933657865 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6378062470 0x7f637813e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:23.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f637d225700 1 -- 172.21.15.36:0/3933657865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6378143a90 msgr2=0x7f6378142740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:23.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f637d225700 1 --2- 172.21.15.36:0/3933657865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6378143a90 0x7f6378142740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:23.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f637d225700 1 -- 172.21.15.36:0/3933657865 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637400b180 con 0x7f63780b4820 2022-06-17T12:54:23.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f637e227700 1 --2- 172.21.15.36:0/3933657865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6378143a90 0x7f6378142740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:23.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f637d225700 1 --2- 172.21.15.36:0/3933657865 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f63780b4820 0x7f6378142250 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7f6374002f70 tx=0x7f6374014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:23.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f6366ffd700 1 -- 172.21.15.36:0/3933657865 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6374016070 con 0x7f63780b4820 2022-06-17T12:54:23.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f6366ffd700 1 -- 172.21.15.36:0/3933657865 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637400d8f0 con 0x7f63780b4820 2022-06-17T12:54:23.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f637ea28700 1 -- 172.21.15.36:0/3933657865 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f637813ec10 con 0x7f63780b4820 2022-06-17T12:54:23.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f6366ffd700 1 -- 172.21.15.36:0/3933657865 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6374013de0 con 0x7f63780b4820 2022-06-17T12:54:23.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.596+0000 7f637ea28700 1 -- 172.21.15.36:0/3933657865 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637813e970 con 0x7f63780b4820 2022-06-17T12:54:23.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.600+0000 7f637ea28700 1 -- 172.21.15.36:0/3933657865 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63781430b0 con 0x7f63780b4820 2022-06-17T12:54:23.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.600+0000 7f6366ffd700 1 -- 172.21.15.36:0/3933657865 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6374025030 con 0x7f63780b4820 2022-06-17T12:54:23.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.600+0000 7f6366ffd700 1 --2- 172.21.15.36:0/3933657865 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6358053a60 0x7f6358055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:23.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.600+0000 7f6366ffd700 1 -- 172.21.15.36:0/3933657865 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f637406ce30 con 0x7f63780b4820 2022-06-17T12:54:23.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.600+0000 7f637da26700 1 --2- 172.21.15.36:0/3933657865 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6358053a60 0x7f6358055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:23.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.600+0000 7f637da26700 1 --2- 172.21.15.36:0/3933657865 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6358053a60 0x7f6358055f20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f636800d000 tx=0x7f6368009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:23.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.604+0000 7f6366ffd700 1 -- 172.21.15.36:0/3933657865 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6374043c30 con 0x7f63780b4820 2022-06-17T12:54:23.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.768+0000 7f637ea28700 1 -- 172.21.15.36:0/3933657865 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6378057410 con 0x7f63780b4820 2022-06-17T12:54:23.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.768+0000 7f6366ffd700 1 -- 172.21.15.36:0/3933657865 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f6374023020 con 0x7f63780b4820 2022-06-17T12:54:23.757 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:23.757 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":42,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/3068692336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3068692336},{"type":"v1","addr":"172.21.15.70:6833","nonce":3068692336}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:54:12.740483+0000","modified":"2022-06-17T12:54:13.744294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6753},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6753":{"gid":6753,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/2728662009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2728662009},{"type":"v1","addr":"172.21.15.36:6835","nonce":2728662009}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-06-17T12:54:23.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.772+0000 7f637ea28700 1 -- 172.21.15.36:0/3933657865 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6358053a60 msgr2=0x7f6358055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:23.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.772+0000 7f637ea28700 1 --2- 172.21.15.36:0/3933657865 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6358053a60 0x7f6358055f20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f636800d000 tx=0x7f6368009040).stop 2022-06-17T12:54:23.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.772+0000 7f637ea28700 1 -- 172.21.15.36:0/3933657865 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f63780b4820 msgr2=0x7f6378142250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:23.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.772+0000 7f637ea28700 1 --2- 172.21.15.36:0/3933657865 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f63780b4820 0x7f6378142250 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7f6374002f70 tx=0x7f6374014000).stop 2022-06-17T12:54:23.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.772+0000 7f637ea28700 1 -- 172.21.15.36:0/3933657865 shutdown_connections 2022-06-17T12:54:23.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.772+0000 7f637ea28700 1 --2- 172.21.15.36:0/3933657865 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6358053a60 0x7f6358055f20 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:23.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.772+0000 7f637ea28700 1 --2- 172.21.15.36:0/3933657865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6378143a90 0x7f6378142740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:23.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.772+0000 7f637ea28700 1 --2- 172.21.15.36:0/3933657865 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f63780b4820 0x7f6378142250 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:23.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.772+0000 7f637ea28700 1 --2- 172.21.15.36:0/3933657865 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6378062470 0x7f637813e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:23.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.772+0000 7f637ea28700 1 -- 172.21.15.36:0/3933657865 >> 172.21.15.36:0/3933657865 conn(0x7f637805c550 msgr2=0x7f6378144390 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:23.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.772+0000 7f637ea28700 1 -- 172.21.15.36:0/3933657865 shutdown_connections 2022-06-17T12:54:23.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.772+0000 7f637ea28700 1 -- 172.21.15.36:0/3933657865 wait complete. 2022-06-17T12:54:23.766 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 42 2022-06-17T12:54:23.778 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-17T12:54:23.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.900+0000 7f3f022ed700 1 Processor -- start 2022-06-17T12:54:23.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.900+0000 7f3f022ed700 1 -- start start 2022-06-17T12:54:23.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.900+0000 7f3f022ed700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3efc0b4820 0x7f3efc0b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:23.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.900+0000 7f3f022ed700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3efc0b83f0 con 0x7f3efc143a90 2022-06-17T12:54:23.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.900+0000 7f3f022ed700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3efc0b1ff0 con 0x7f3efc062470 2022-06-17T12:54:23.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.900+0000 7f3f022ed700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc142ec0 con 0x7f3efc0b4820 2022-06-17T12:54:23.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.900+0000 7f3f01aec700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f3efc143a90 0x7f3efc144390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54202/0 (socket says 172.21.15.36:54202) 2022-06-17T12:54:23.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.900+0000 7f3f01aec700 1 -- 172.21.15.36:0/102072101 learned_addr learned my addr 172.21.15.36:0/102072101 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:23.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.900+0000 7f3efbfff700 1 -- 172.21.15.36:0/102072101 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1661751588 0 0) 0x7f3efc0b83f0 con 0x7f3efc143a90 2022-06-17T12:54:23.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.900+0000 7f3efbfff700 1 -- 172.21.15.36:0/102072101 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ee8003260 con 0x7f3efc143a90 2022-06-17T12:54:23.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.900+0000 7f3f012eb700 1 --2- 172.21.15.36:0/102072101 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3efc0b4820 0x7f3efc0b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:23.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.900+0000 7f3efbfff700 1 -- 172.21.15.36:0/102072101 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2157107592 0 0) 0x7f3ee8003260 con 0x7f3efc143a90 2022-06-17T12:54:23.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.900+0000 7f3efbfff700 1 -- 172.21.15.36:0/102072101 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3efc0b4820 msgr2=0x7f3efc0b4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:23.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.900+0000 7f3efbfff700 1 --2- 172.21.15.36:0/102072101 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3efc0b4820 0x7f3efc0b4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:23.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.900+0000 7f3efbfff700 1 -- 172.21.15.36:0/102072101 >> v1:172.21.15.70:6789/0 conn(0x7f3efc062470 legacy=0x7f3efc0b7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:23.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.900+0000 7f3efbfff700 1 -- 172.21.15.36:0/102072101 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3efc1430b0 con 0x7f3efc143a90 2022-06-17T12:54:23.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.900+0000 7f3efbfff700 1 -- 172.21.15.36:0/102072101 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f3eec001660 con 0x7f3efc143a90 2022-06-17T12:54:23.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.900+0000 7f3efbfff700 1 -- 172.21.15.36:0/102072101 >> v1:172.21.15.36:6789/0 conn(0x7f3efc143a90 legacy=0x7f3efc144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:23.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.900+0000 7f3efbfff700 1 --2- 172.21.15.36:0/102072101 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3ee8003780 0x7f3ee8003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:23.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.900+0000 7f3efbfff700 1 --2- 172.21.15.36:0/102072101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ee8004500 0x7f3ee8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:23.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.900+0000 7f3efbfff700 1 --2- 172.21.15.36:0/102072101 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3efc0b4820 0x7f3ee8004f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:23.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.900+0000 7f3efbfff700 1 -- 172.21.15.36:0/102072101 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee800b800 con 0x7f3ee8003780 2022-06-17T12:54:23.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.900+0000 7f3efbfff700 1 -- 172.21.15.36:0/102072101 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee800b970 con 0x7f3ee8004500 2022-06-17T12:54:23.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.900+0000 7f3efbfff700 1 -- 172.21.15.36:0/102072101 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee800bb30 con 0x7f3efc0b4820 2022-06-17T12:54:23.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.904+0000 7f3f012eb700 1 --2- 172.21.15.36:0/102072101 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3ee8003780 0x7f3ee8003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:23.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.904+0000 7f3f01aec700 1 --2- 172.21.15.36:0/102072101 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3efc0b4820 0x7f3ee8004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:23.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.904+0000 7f3f00aea700 1 --2- 172.21.15.36:0/102072101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ee8004500 0x7f3ee8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:23.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.904+0000 7f3f012eb700 1 -- 172.21.15.36:0/102072101 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3efc0b4820 msgr2=0x7f3ee8004f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:23.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.904+0000 7f3f012eb700 1 --2- 172.21.15.36:0/102072101 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3efc0b4820 0x7f3ee8004f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:23.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.904+0000 7f3f012eb700 1 -- 172.21.15.36:0/102072101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ee8004500 msgr2=0x7f3ee8004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:23.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.904+0000 7f3f012eb700 1 --2- 172.21.15.36:0/102072101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ee8004500 0x7f3ee8004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:23.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.904+0000 7f3f012eb700 1 -- 172.21.15.36:0/102072101 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee8003260 con 0x7f3ee8003780 2022-06-17T12:54:23.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.904+0000 7f3f012eb700 1 --2- 172.21.15.36:0/102072101 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3ee8003780 0x7f3ee8003ba0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f3ef400a000 tx=0x7f3ef4008040).ready entity=mon.0 client_cookie=8bfb11abf0a07ad9 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:23.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.904+0000 7f3efbfff700 1 -- 172.21.15.36:0/102072101 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ef400c070 con 0x7f3ee8003780 2022-06-17T12:54:23.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.904+0000 7f3efbfff700 1 -- 172.21.15.36:0/102072101 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef400e620 con 0x7f3ee8003780 2022-06-17T12:54:23.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.904+0000 7f3efbfff700 1 -- 172.21.15.36:0/102072101 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ef400e980 con 0x7f3ee8003780 2022-06-17T12:54:23.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.904+0000 7f3f022ed700 1 -- 172.21.15.36:0/102072101 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3ee8003780 msgr2=0x7f3ee8003ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:23.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.904+0000 7f3f022ed700 1 --2- 172.21.15.36:0/102072101 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3ee8003780 0x7f3ee8003ba0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f3ef400a000 tx=0x7f3ef4008040).stop 2022-06-17T12:54:23.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.904+0000 7f3f01aec700 1 -- 172.21.15.36:0/102072101 reap_dead start 2022-06-17T12:54:23.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.904+0000 7f3f022ed700 1 -- 172.21.15.36:0/102072101 shutdown_connections 2022-06-17T12:54:23.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.904+0000 7f3f022ed700 1 -- 172.21.15.36:0/102072101 >> 172.21.15.36:0/102072101 conn(0x7f3efc05c550 msgr2=0x7f3efc0b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:23.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.904+0000 7f3f022ed700 1 -- 172.21.15.36:0/102072101 shutdown_connections 2022-06-17T12:54:23.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.904+0000 7f3f022ed700 1 -- 172.21.15.36:0/102072101 wait complete. 2022-06-17T12:54:23.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.904+0000 7f3f022ed700 1 Processor -- start 2022-06-17T12:54:23.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.904+0000 7f3f022ed700 1 -- start start 2022-06-17T12:54:23.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.908+0000 7f3f022ed700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ee8003780 0x7f3efc152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:23.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.908+0000 7f3f022ed700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3efc064c30 0x7f3efc1453b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:23.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.908+0000 7f3f022ed700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3efc0b0ab0 0x7f3efc0b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:23.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.908+0000 7f3f022ed700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc142b70 con 0x7f3efc064c30 2022-06-17T12:54:23.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.908+0000 7f3f022ed700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc145a70 con 0x7f3ee8003780 2022-06-17T12:54:23.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.908+0000 7f3f022ed700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc145d10 con 0x7f3efc0b0ab0 2022-06-17T12:54:23.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.908+0000 7f3f00aea700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3efc064c30 0x7f3efc1453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:23.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.908+0000 7f3f00aea700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3efc064c30 0x7f3efc1453b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36684/0 (socket says 172.21.15.36:36684) 2022-06-17T12:54:23.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.908+0000 7f3f00aea700 1 -- 172.21.15.36:0/493055894 learned_addr learned my addr 172.21.15.36:0/493055894 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:23.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.908+0000 7f3f01aec700 1 --2- 172.21.15.36:0/493055894 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3efc0b0ab0 0x7f3efc0b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:23.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.908+0000 7f3f012eb700 1 --2- 172.21.15.36:0/493055894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ee8003780 0x7f3efc152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:23.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.908+0000 7f3f00aea700 1 -- 172.21.15.36:0/493055894 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3efc0b0ab0 msgr2=0x7f3efc0b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:23.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.908+0000 7f3f00aea700 1 --2- 172.21.15.36:0/493055894 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3efc0b0ab0 0x7f3efc0b0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:23.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.908+0000 7f3f00aea700 1 -- 172.21.15.36:0/493055894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ee8003780 msgr2=0x7f3efc152ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:23.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.908+0000 7f3f00aea700 1 --2- 172.21.15.36:0/493055894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ee8003780 0x7f3efc152ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:23.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.908+0000 7f3f00aea700 1 -- 172.21.15.36:0/493055894 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3efc0b1430 con 0x7f3efc064c30 2022-06-17T12:54:23.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.908+0000 7f3f00aea700 1 --2- 172.21.15.36:0/493055894 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3efc064c30 0x7f3efc1453b0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7f3ef000b000 tx=0x7f3ef0009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:23.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.908+0000 7f3efa7fc700 1 -- 172.21.15.36:0/493055894 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ef0010640 con 0x7f3efc064c30 2022-06-17T12:54:23.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.908+0000 7f3efa7fc700 1 -- 172.21.15.36:0/493055894 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef00107a0 con 0x7f3efc064c30 2022-06-17T12:54:23.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.908+0000 7f3f022ed700 1 -- 172.21.15.36:0/493055894 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3efc1427f0 con 0x7f3efc064c30 2022-06-17T12:54:23.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.908+0000 7f3efa7fc700 1 -- 172.21.15.36:0/493055894 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ef0010b00 con 0x7f3efc064c30 2022-06-17T12:54:23.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.908+0000 7f3f022ed700 1 -- 172.21.15.36:0/493055894 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3efc0649f0 con 0x7f3efc064c30 2022-06-17T12:54:23.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.908+0000 7f3f022ed700 1 -- 172.21.15.36:0/493055894 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3efc1458a0 con 0x7f3efc064c30 2022-06-17T12:54:23.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.908+0000 7f3efa7fc700 1 -- 172.21.15.36:0/493055894 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f3ef0010c60 con 0x7f3efc064c30 2022-06-17T12:54:23.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.912+0000 7f3efa7fc700 1 --2- 172.21.15.36:0/493055894 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3edc053940 0x7f3edc055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:23.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.912+0000 7f3efa7fc700 1 -- 172.21.15.36:0/493055894 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3ef006bfe0 con 0x7f3efc064c30 2022-06-17T12:54:23.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.912+0000 7f3f012eb700 1 --2- 172.21.15.36:0/493055894 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3edc053940 0x7f3edc055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:23.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.912+0000 7f3f012eb700 1 --2- 172.21.15.36:0/493055894 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3edc053940 0x7f3edc055e00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f3ef400a000 tx=0x7f3ef4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:23.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:23.916+0000 7f3efa7fc700 1 -- 172.21.15.36:0/493055894 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3ef0042cb0 con 0x7f3efc064c30 2022-06-17T12:54:24.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.080+0000 7f3f022ed700 1 -- 172.21.15.36:0/493055894 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3efc143350 con 0x7f3efc064c30 2022-06-17T12:54:24.780 INFO:tasks.ceph.mon.a.smithi036.stderr:2022-06-17T12:54:24.792+0000 7fb7a2649700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-17T12:54:24.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.804+0000 7f3efa7fc700 1 -- 172.21.15.36:0/493055894 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3ef00462d0 con 0x7f3efc064c30 2022-06-17T12:54:24.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.808+0000 7f3f022ed700 1 -- 172.21.15.36:0/493055894 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3edc053940 msgr2=0x7f3edc055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:24.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.808+0000 7f3f022ed700 1 --2- 172.21.15.36:0/493055894 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3edc053940 0x7f3edc055e00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f3ef400a000 tx=0x7f3ef4008040).stop 2022-06-17T12:54:24.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.808+0000 7f3f022ed700 1 -- 172.21.15.36:0/493055894 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3efc064c30 msgr2=0x7f3efc1453b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:24.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.808+0000 7f3f022ed700 1 --2- 172.21.15.36:0/493055894 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3efc064c30 0x7f3efc1453b0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7f3ef000b000 tx=0x7f3ef0009040).stop 2022-06-17T12:54:24.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.812+0000 7f3f022ed700 1 -- 172.21.15.36:0/493055894 shutdown_connections 2022-06-17T12:54:24.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.812+0000 7f3f022ed700 1 --2- 172.21.15.36:0/493055894 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3edc053940 0x7f3edc055e00 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:24.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.812+0000 7f3f022ed700 1 --2- 172.21.15.36:0/493055894 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3efc0b0ab0 0x7f3efc0b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:24.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.812+0000 7f3f022ed700 1 --2- 172.21.15.36:0/493055894 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3efc064c30 0x7f3efc1453b0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:24.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.812+0000 7f3f022ed700 1 --2- 172.21.15.36:0/493055894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ee8003780 0x7f3efc152ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:24.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.812+0000 7f3f022ed700 1 -- 172.21.15.36:0/493055894 >> 172.21.15.36:0/493055894 conn(0x7f3efc05c550 msgr2=0x7f3efc05f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:24.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.812+0000 7f3f022ed700 1 -- 172.21.15.36:0/493055894 shutdown_connections 2022-06-17T12:54:24.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.812+0000 7f3f022ed700 1 -- 172.21.15.36:0/493055894 wait complete. 2022-06-17T12:54:24.801 INFO:teuthology.orchestra.run.smithi036.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-17T12:54:24.815 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:54:24.821 INFO:tasks.ceph.mds.a.smithi036.stdout:starting mds.a at 2022-06-17T12:54:24.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70cffff700 1 Processor -- start 2022-06-17T12:54:24.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70cffff700 1 -- start start 2022-06-17T12:54:24.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70cffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f70d00b4820 0x7f70d00b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:24.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70cffff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f70d0143a90 0x7f70d0144390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:24.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70cffff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d00b83f0 con 0x7f70d0062470 2022-06-17T12:54:24.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70cffff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d0143f00 con 0x7f70d00b4820 2022-06-17T12:54:24.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70cffff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d0142ec0 con 0x7f70d0143a90 2022-06-17T12:54:24.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70ceffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f70d00b4820 0x7f70d00b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:24.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70ce7fc700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f70d0062470 0x7f70d00b7ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54232/0 (socket says 172.21.15.36:54232) 2022-06-17T12:54:24.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70ce7fc700 1 -- 172.21.15.36:0/1859892395 learned_addr learned my addr 172.21.15.36:0/1859892395 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:24.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70cf7fe700 1 --2- 172.21.15.36:0/1859892395 >> v2:172.21.15.70:3300/0 conn(0x7f70d0143a90 0x7f70d0144390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:24.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70cdffb700 1 -- 172.21.15.36:0/1859892395 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423578560 0 0) 0x7f70d00b83f0 con 0x7f70d0062470 2022-06-17T12:54:24.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70cdffb700 1 -- 172.21.15.36:0/1859892395 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70b4003260 con 0x7f70d0062470 2022-06-17T12:54:24.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70ceffd700 1 -- 172.21.15.36:0/1859892395 >> v2:172.21.15.70:3300/0 conn(0x7f70d0143a90 msgr2=0x7f70d0144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:24.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70ceffd700 1 --2- 172.21.15.36:0/1859892395 >> v2:172.21.15.70:3300/0 conn(0x7f70d0143a90 0x7f70d0144390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:24.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70ceffd700 1 -- 172.21.15.36:0/1859892395 >> v1:172.21.15.36:6789/0 conn(0x7f70d0062470 legacy=0x7f70d00b7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:24.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70ceffd700 1 -- 172.21.15.36:0/1859892395 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d00b1ff0 con 0x7f70d00b4820 2022-06-17T12:54:24.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70ceffd700 1 --2- 172.21.15.36:0/1859892395 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f70d00b4820 0x7f70d00b4c00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7f70c0002f70 tx=0x7f70c0006df0).ready entity=mon.0 client_cookie=9a459e98b84a8d04 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:24.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70cdffb700 1 -- 172.21.15.36:0/1859892395 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70c000d4f0 con 0x7f70d00b4820 2022-06-17T12:54:24.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70cdffb700 1 -- 172.21.15.36:0/1859892395 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c0000e90 con 0x7f70d00b4820 2022-06-17T12:54:24.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70cdffb700 1 -- 172.21.15.36:0/1859892395 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70c000d840 con 0x7f70d00b4820 2022-06-17T12:54:24.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70cffff700 1 -- 172.21.15.36:0/1859892395 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f70d00b4820 msgr2=0x7f70d00b4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:24.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70cffff700 1 --2- 172.21.15.36:0/1859892395 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f70d00b4820 0x7f70d00b4c00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7f70c0002f70 tx=0x7f70c0006df0).stop 2022-06-17T12:54:24.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70cffff700 1 -- 172.21.15.36:0/1859892395 shutdown_connections 2022-06-17T12:54:24.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70cffff700 1 --2- 172.21.15.36:0/1859892395 >> v2:172.21.15.70:3300/0 conn(0x7f70d0143a90 0x7f70d0144390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:24.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70cffff700 1 --2- 172.21.15.36:0/1859892395 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f70d00b4820 0x7f70d00b4c00 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:24.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70cffff700 1 -- 172.21.15.36:0/1859892395 >> 172.21.15.36:0/1859892395 conn(0x7f70d005c550 msgr2=0x7f70d00b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:24.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70cffff700 1 -- 172.21.15.36:0/1859892395 shutdown_connections 2022-06-17T12:54:24.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.928+0000 7f70cffff700 1 -- 172.21.15.36:0/1859892395 wait complete. 2022-06-17T12:54:24.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70cffff700 1 Processor -- start 2022-06-17T12:54:24.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70cffff700 1 -- start start 2022-06-17T12:54:24.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70cffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70d0062470 0x7f70d0065dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:24.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70cffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f70d00b4820 0x7f70d0066710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:24.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70cffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70d0143a90 0x7f70d0066c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:24.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70cffff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d0153430 con 0x7f70d00b4820 2022-06-17T12:54:24.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70cffff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d0064eb0 con 0x7f70d0143a90 2022-06-17T12:54:24.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70cffff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d013ec90 con 0x7f70d0062470 2022-06-17T12:54:24.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70ce7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f70d00b4820 0x7f70d0066710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:24.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70ce7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f70d00b4820 0x7f70d0066710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36710/0 (socket says 172.21.15.36:36710) 2022-06-17T12:54:24.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70ce7fc700 1 -- 172.21.15.36:0/2373508540 learned_addr learned my addr 172.21.15.36:0/2373508540 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:24.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70cf7fe700 1 --2- 172.21.15.36:0/2373508540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70d0143a90 0x7f70d0066c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:24.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70ceffd700 1 --2- 172.21.15.36:0/2373508540 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70d0062470 0x7f70d0065dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:24.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70ce7fc700 1 -- 172.21.15.36:0/2373508540 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70d0062470 msgr2=0x7f70d0065dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:24.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70ce7fc700 1 --2- 172.21.15.36:0/2373508540 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70d0062470 0x7f70d0065dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:24.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70ce7fc700 1 -- 172.21.15.36:0/2373508540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70d0143a90 msgr2=0x7f70d0066c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:24.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70ce7fc700 1 --2- 172.21.15.36:0/2373508540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70d0143a90 0x7f70d0066c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:24.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70ce7fc700 1 -- 172.21.15.36:0/2373508540 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c000b180 con 0x7f70d00b4820 2022-06-17T12:54:24.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70ceffd700 1 --2- 172.21.15.36:0/2373508540 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70d0062470 0x7f70d0065dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:54:24.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70ce7fc700 1 --2- 172.21.15.36:0/2373508540 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f70d00b4820 0x7f70d0066710 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7f70c4001110 tx=0x7f70c4007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:24.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70bffff700 1 -- 172.21.15.36:0/2373508540 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70c40070d0 con 0x7f70d00b4820 2022-06-17T12:54:24.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70bffff700 1 -- 172.21.15.36:0/2373508540 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c4007230 con 0x7f70d00b4820 2022-06-17T12:54:24.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70cffff700 1 -- 172.21.15.36:0/2373508540 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d013ef30 con 0x7f70d00b4820 2022-06-17T12:54:24.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70bffff700 1 -- 172.21.15.36:0/2373508540 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70c4005610 con 0x7f70d00b4820 2022-06-17T12:54:24.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.932+0000 7f70cffff700 1 -- 172.21.15.36:0/2373508540 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d0146cb0 con 0x7f70d00b4820 2022-06-17T12:54:24.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.936+0000 7f70bffff700 1 -- 172.21.15.36:0/2373508540 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f70c4010070 con 0x7f70d00b4820 2022-06-17T12:54:24.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.936+0000 7f70cffff700 1 -- 172.21.15.36:0/2373508540 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d013f0c0 con 0x7f70d00b4820 2022-06-17T12:54:24.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.936+0000 7f70bffff700 1 --2- 172.21.15.36:0/2373508540 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f70ac053a40 0x7f70ac055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:24.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.936+0000 7f70ceffd700 1 --2- 172.21.15.36:0/2373508540 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f70ac053a40 0x7f70ac055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:24.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.936+0000 7f70bffff700 1 -- 172.21.15.36:0/2373508540 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f70c406b7f0 con 0x7f70d00b4820 2022-06-17T12:54:24.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.936+0000 7f70ceffd700 1 --2- 172.21.15.36:0/2373508540 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f70ac053a40 0x7f70ac055f00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f70c0015000 tx=0x7f70c0014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:24.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:24.940+0000 7f70bffff700 1 -- 172.21.15.36:0/2373508540 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f70c4042430 con 0x7f70d00b4820 2022-06-17T12:54:25.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.104+0000 7f70cffff700 1 -- 172.21.15.36:0/2373508540 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70d01430b0 con 0x7f70d00b4820 2022-06-17T12:54:25.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.104+0000 7f70bffff700 1 -- 172.21.15.36:0/2373508540 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+1779 (secure 0 0 0) 0x7f70c4045a50 con 0x7f70d00b4820 2022-06-17T12:54:25.094 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:25.095 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":43,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/3068692336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3068692336},{"type":"v1","addr":"172.21.15.70:6833","nonce":3068692336}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40}],"filesystems":[{"mdsmap":{"epoch":43,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:54:12.740483+0000","modified":"2022-06-17T12:54:24.797016+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-06-17T12:54:25.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.108+0000 7f70cffff700 1 -- 172.21.15.36:0/2373508540 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f70ac053a40 msgr2=0x7f70ac055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:25.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.108+0000 7f70cffff700 1 --2- 172.21.15.36:0/2373508540 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f70ac053a40 0x7f70ac055f00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f70c0015000 tx=0x7f70c0014000).stop 2022-06-17T12:54:25.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.108+0000 7f70cffff700 1 -- 172.21.15.36:0/2373508540 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f70d00b4820 msgr2=0x7f70d0066710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:25.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.108+0000 7f70cffff700 1 --2- 172.21.15.36:0/2373508540 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f70d00b4820 0x7f70d0066710 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7f70c4001110 tx=0x7f70c4007aa0).stop 2022-06-17T12:54:25.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.108+0000 7f70cffff700 1 -- 172.21.15.36:0/2373508540 shutdown_connections 2022-06-17T12:54:25.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.108+0000 7f70cffff700 1 --2- 172.21.15.36:0/2373508540 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f70ac053a40 0x7f70ac055f00 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:25.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.108+0000 7f70cffff700 1 --2- 172.21.15.36:0/2373508540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70d0143a90 0x7f70d0066c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:25.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.108+0000 7f70cffff700 1 --2- 172.21.15.36:0/2373508540 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f70d00b4820 0x7f70d0066710 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:25.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.108+0000 7f70cffff700 1 --2- 172.21.15.36:0/2373508540 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70d0062470 0x7f70d0065dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:25.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.108+0000 7f70cffff700 1 -- 172.21.15.36:0/2373508540 >> 172.21.15.36:0/2373508540 conn(0x7f70d005c550 msgr2=0x7f70d01475b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:25.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.108+0000 7f70cffff700 1 -- 172.21.15.36:0/2373508540 shutdown_connections 2022-06-17T12:54:25.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.108+0000 7f70cffff700 1 -- 172.21.15.36:0/2373508540 wait complete. 2022-06-17T12:54:25.104 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 43 2022-06-17T12:54:25.107 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-17T12:54:25.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.228+0000 7f966eb79700 1 Processor -- start 2022-06-17T12:54:25.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.228+0000 7f966eb79700 1 -- start start 2022-06-17T12:54:25.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.228+0000 7f966eb79700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f96680b4120 0x7f9668146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:25.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.228+0000 7f966eb79700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9668147270 0x7f9668140100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:25.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.228+0000 7f966eb79700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96681476e0 con 0x7f96680b64f0 2022-06-17T12:54:25.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.228+0000 7f966eb79700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96680b68d0 con 0x7f96680b4120 2022-06-17T12:54:25.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.228+0000 7f966eb79700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96680b6e20 con 0x7f9668147270 2022-06-17T12:54:25.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.228+0000 7f9667fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f96680b4120 0x7f9668146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:25.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.228+0000 7f966d116700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9668147270 0x7f9668140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:25.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.228+0000 7f966d116700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9668147270 0x7f9668140100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:51510/0 (socket says 172.21.15.36:51510) 2022-06-17T12:54:25.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.228+0000 7f9667fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f96680b4120 0x7f9668146d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:57098/0 (socket says 172.21.15.36:57098) 2022-06-17T12:54:25.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.228+0000 7f966c915700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f96680b64f0 0x7f9668143150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:49980/0 (socket says 172.21.15.36:49980) 2022-06-17T12:54:25.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f966d116700 1 -- 172.21.15.36:0/395901992 learned_addr learned my addr 172.21.15.36:0/395901992 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:25.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f966d116700 1 -- 172.21.15.36:0/395901992 >> v2:172.21.15.70:3300/0 conn(0x7f96680b4120 msgr2=0x7f9668146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:25.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f966d116700 1 --2- 172.21.15.36:0/395901992 >> v2:172.21.15.70:3300/0 conn(0x7f96680b4120 0x7f9668146d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:25.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f966d116700 1 -- 172.21.15.36:0/395901992 >> v1:172.21.15.70:6789/0 conn(0x7f96680b64f0 legacy=0x7f9668143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:25.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f966d116700 1 -- 172.21.15.36:0/395901992 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96681468e0 con 0x7f9668147270 2022-06-17T12:54:25.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f9667fff700 1 --2- 172.21.15.36:0/395901992 >> v2:172.21.15.70:3300/0 conn(0x7f96680b4120 0x7f9668146d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:25.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f966d116700 1 --2- 172.21.15.36:0/395901992 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9668147270 0x7f9668140100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f96600032f0 tx=0x7f9660009290).ready entity=mon.2 client_cookie=446908ce05df4251 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:25.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f96677fe700 1 -- 172.21.15.36:0/395901992 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f966000b8a0 con 0x7f9668147270 2022-06-17T12:54:25.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f96677fe700 1 -- 172.21.15.36:0/395901992 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f966000ba00 con 0x7f9668147270 2022-06-17T12:54:25.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f96677fe700 1 -- 172.21.15.36:0/395901992 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f966000bb70 con 0x7f9668147270 2022-06-17T12:54:25.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f966eb79700 1 -- 172.21.15.36:0/395901992 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9668147270 msgr2=0x7f9668140100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:25.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f966eb79700 1 --2- 172.21.15.36:0/395901992 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9668147270 0x7f9668140100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f96600032f0 tx=0x7f9660009290).stop 2022-06-17T12:54:25.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f966eb79700 1 -- 172.21.15.36:0/395901992 shutdown_connections 2022-06-17T12:54:25.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f966eb79700 1 --2- 172.21.15.36:0/395901992 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9668147270 0x7f9668140100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:25.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f966eb79700 1 --2- 172.21.15.36:0/395901992 >> v2:172.21.15.70:3300/0 conn(0x7f96680b4120 0x7f9668146d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:25.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f966eb79700 1 -- 172.21.15.36:0/395901992 >> 172.21.15.36:0/395901992 conn(0x7f96680620f0 msgr2=0x7f9668062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:25.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f966eb79700 1 -- 172.21.15.36:0/395901992 shutdown_connections 2022-06-17T12:54:25.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f966eb79700 1 -- 172.21.15.36:0/395901992 wait complete. 2022-06-17T12:54:25.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f966eb79700 1 Processor -- start 2022-06-17T12:54:25.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f966eb79700 1 -- start start 2022-06-17T12:54:25.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f966eb79700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96680b4120 0x7f966813d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:25.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f966eb79700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96680b64f0 0x7f96680b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:25.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f966eb79700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9668147270 0x7f96681132d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:25.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f966eb79700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96681438f0 con 0x7f96680b64f0 2022-06-17T12:54:25.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f966eb79700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96681408d0 con 0x7f9668147270 2022-06-17T12:54:25.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f966eb79700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9668140b70 con 0x7f96680b4120 2022-06-17T12:54:25.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f9667fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96680b64f0 0x7f96680b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:25.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.232+0000 7f9667fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96680b64f0 0x7f96680b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36722/0 (socket says 172.21.15.36:36722) 2022-06-17T12:54:25.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.236+0000 7f9667fff700 1 -- 172.21.15.36:0/1383711682 learned_addr learned my addr 172.21.15.36:0/1383711682 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:25.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.236+0000 7f966c915700 1 --2- 172.21.15.36:0/1383711682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96680b4120 0x7f966813d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:25.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.236+0000 7f966d116700 1 --2- 172.21.15.36:0/1383711682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9668147270 0x7f96681132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:25.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.236+0000 7f9667fff700 1 -- 172.21.15.36:0/1383711682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96680b4120 msgr2=0x7f966813d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:25.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.236+0000 7f9667fff700 1 --2- 172.21.15.36:0/1383711682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96680b4120 0x7f966813d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:25.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.236+0000 7f9667fff700 1 -- 172.21.15.36:0/1383711682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9668147270 msgr2=0x7f96681132d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:25.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.236+0000 7f9667fff700 1 --2- 172.21.15.36:0/1383711682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9668147270 0x7f96681132d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:25.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.236+0000 7f9667fff700 1 -- 172.21.15.36:0/1383711682 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965000b180 con 0x7f96680b64f0 2022-06-17T12:54:25.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.236+0000 7f9667fff700 1 --2- 172.21.15.36:0/1383711682 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96680b64f0 0x7f96680b4d50 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7f965000c000 tx=0x7f9650006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:25.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.236+0000 7f9665ffb700 1 -- 172.21.15.36:0/1383711682 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f965000c7e0 con 0x7f96680b64f0 2022-06-17T12:54:25.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.236+0000 7f9665ffb700 1 -- 172.21.15.36:0/1383711682 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965000c940 con 0x7f96680b64f0 2022-06-17T12:54:25.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.236+0000 7f966eb79700 1 -- 172.21.15.36:0/1383711682 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9660009530 con 0x7f96680b64f0 2022-06-17T12:54:25.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.236+0000 7f9665ffb700 1 -- 172.21.15.36:0/1383711682 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f965000cca0 con 0x7f96680b64f0 2022-06-17T12:54:25.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.236+0000 7f966eb79700 1 -- 172.21.15.36:0/1383711682 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96681411e0 con 0x7f96680b64f0 2022-06-17T12:54:25.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.236+0000 7f966eb79700 1 -- 172.21.15.36:0/1383711682 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96680b3a60 con 0x7f96680b64f0 2022-06-17T12:54:25.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.236+0000 7f9665ffb700 1 -- 172.21.15.36:0/1383711682 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f965001c460 con 0x7f96680b64f0 2022-06-17T12:54:25.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.240+0000 7f9665ffb700 1 --2- 172.21.15.36:0/1383711682 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f964c053a40 0x7f964c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:25.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.240+0000 7f966c915700 1 --2- 172.21.15.36:0/1383711682 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f964c053a40 0x7f964c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:25.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.240+0000 7f9665ffb700 1 -- 172.21.15.36:0/1383711682 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f965006c620 con 0x7f96680b64f0 2022-06-17T12:54:25.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.240+0000 7f966c915700 1 --2- 172.21.15.36:0/1383711682 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f964c053a40 0x7f964c055f00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f965800d000 tx=0x7f9658009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:25.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.244+0000 7f9665ffb700 1 -- 172.21.15.36:0/1383711682 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f96500432f0 con 0x7f96680b64f0 2022-06-17T12:54:25.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.404+0000 7f966eb79700 1 -- 172.21.15.36:0/1383711682 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9668057410 con 0x7f96680b64f0 2022-06-17T12:54:25.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.808+0000 7f9665ffb700 1 -- 172.21.15.36:0/1383711682 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9650046910 con 0x7f96680b64f0 2022-06-17T12:54:25.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.812+0000 7f966eb79700 1 -- 172.21.15.36:0/1383711682 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f964c053a40 msgr2=0x7f964c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:25.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.812+0000 7f966eb79700 1 --2- 172.21.15.36:0/1383711682 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f964c053a40 0x7f964c055f00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f965800d000 tx=0x7f9658009040).stop 2022-06-17T12:54:25.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.812+0000 7f966eb79700 1 -- 172.21.15.36:0/1383711682 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96680b64f0 msgr2=0x7f96680b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:25.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.812+0000 7f966eb79700 1 --2- 172.21.15.36:0/1383711682 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96680b64f0 0x7f96680b4d50 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7f965000c000 tx=0x7f9650006df0).stop 2022-06-17T12:54:25.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.812+0000 7f966eb79700 1 -- 172.21.15.36:0/1383711682 shutdown_connections 2022-06-17T12:54:25.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.812+0000 7f966eb79700 1 --2- 172.21.15.36:0/1383711682 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f964c053a40 0x7f964c055f00 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:25.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.812+0000 7f966eb79700 1 --2- 172.21.15.36:0/1383711682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9668147270 0x7f96681132d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:25.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.812+0000 7f966eb79700 1 --2- 172.21.15.36:0/1383711682 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f96680b64f0 0x7f96680b4d50 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:25.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.812+0000 7f966eb79700 1 --2- 172.21.15.36:0/1383711682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96680b4120 0x7f966813d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:25.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.812+0000 7f966eb79700 1 -- 172.21.15.36:0/1383711682 >> 172.21.15.36:0/1383711682 conn(0x7f96680620f0 msgr2=0x7f966813e660 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:25.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.812+0000 7f966eb79700 1 -- 172.21.15.36:0/1383711682 shutdown_connections 2022-06-17T12:54:25.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.812+0000 7f966eb79700 1 -- 172.21.15.36:0/1383711682 wait complete. 2022-06-17T12:54:25.819 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-17T12:54:25.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.948+0000 7f98560d7700 1 Processor -- start 2022-06-17T12:54:25.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.948+0000 7f98560d7700 1 -- start start 2022-06-17T12:54:25.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f98560d7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98500b6860 0x7f985013ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:25.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f98560d7700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f98500b34c0 0x7f98501404a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:25.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f98560d7700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9850063b10 con 0x7f9850063730 2022-06-17T12:54:25.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f98560d7700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9850144280 con 0x7f98500b34c0 2022-06-17T12:54:25.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f98560d7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9850144520 con 0x7f98500b6860 2022-06-17T12:54:25.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f98558d6700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f98500b34c0 0x7f98501404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:25.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f98558d6700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f98500b34c0 0x7f98501404a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36736/0 (socket says 172.21.15.36:36736) 2022-06-17T12:54:25.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f98558d6700 1 -- 172.21.15.36:0/2845559464 learned_addr learned my addr 172.21.15.36:0/2845559464 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:25.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f98548d4700 1 --2- 172.21.15.36:0/2845559464 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98500b6860 0x7f985013ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:25.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f98558d6700 1 -- 172.21.15.36:0/2845559464 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98500b6860 msgr2=0x7f985013ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:25.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f98558d6700 1 --2- 172.21.15.36:0/2845559464 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98500b6860 0x7f985013ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:25.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f98558d6700 1 -- 172.21.15.36:0/2845559464 >> v1:172.21.15.70:6789/0 conn(0x7f9850063730 legacy=0x7f9850144c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:25.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f98558d6700 1 -- 172.21.15.36:0/2845559464 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98501447c0 con 0x7f98500b34c0 2022-06-17T12:54:25.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f98558d6700 1 --2- 172.21.15.36:0/2845559464 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98500b34c0 0x7f98501404a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f98480032f0 tx=0x7f9848009290).ready entity=mon.0 client_cookie=a285a427dda45dd0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:25.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f983ffff700 1 -- 172.21.15.36:0/2845559464 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f984800b8a0 con 0x7f98500b34c0 2022-06-17T12:54:25.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f983ffff700 1 -- 172.21.15.36:0/2845559464 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984800ba00 con 0x7f98500b34c0 2022-06-17T12:54:25.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f983ffff700 1 -- 172.21.15.36:0/2845559464 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f984800bb70 con 0x7f98500b34c0 2022-06-17T12:54:25.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f98560d7700 1 -- 172.21.15.36:0/2845559464 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98500b34c0 msgr2=0x7f98501404a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:25.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f98560d7700 1 --2- 172.21.15.36:0/2845559464 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98500b34c0 0x7f98501404a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f98480032f0 tx=0x7f9848009290).stop 2022-06-17T12:54:25.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f98560d7700 1 -- 172.21.15.36:0/2845559464 shutdown_connections 2022-06-17T12:54:25.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f98560d7700 1 --2- 172.21.15.36:0/2845559464 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98500b6860 0x7f985013ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:25.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f98560d7700 1 --2- 172.21.15.36:0/2845559464 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f98500b34c0 0x7f98501404a0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:25.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f98560d7700 1 -- 172.21.15.36:0/2845559464 >> 172.21.15.36:0/2845559464 conn(0x7f9850060880 msgr2=0x7f98500614b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:25.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f98560d7700 1 -- 172.21.15.36:0/2845559464 shutdown_connections 2022-06-17T12:54:25.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.952+0000 7f98560d7700 1 -- 172.21.15.36:0/2845559464 wait complete. 2022-06-17T12:54:25.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.956+0000 7f98560d7700 1 Processor -- start 2022-06-17T12:54:25.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.956+0000 7f98560d7700 1 -- start start 2022-06-17T12:54:25.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.956+0000 7f98560d7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9850063730 0x7f9850135cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:25.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.956+0000 7f98560d7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98500b34c0 0x7f9850131930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:25.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.956+0000 7f98560d7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98500b6860 0x7f9850150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:25.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.956+0000 7f98560d7700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9850150530 con 0x7f9850063730 2022-06-17T12:54:25.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.956+0000 7f98560d7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98501536a0 con 0x7f98500b34c0 2022-06-17T12:54:25.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.956+0000 7f98560d7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9850132270 con 0x7f98500b6860 2022-06-17T12:54:25.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.956+0000 7f98550d5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9850063730 0x7f9850135cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:25.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.956+0000 7f98550d5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9850063730 0x7f9850135cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36738/0 (socket says 172.21.15.36:36738) 2022-06-17T12:54:25.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.956+0000 7f98550d5700 1 -- 172.21.15.36:0/2286625207 learned_addr learned my addr 172.21.15.36:0/2286625207 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:25.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.956+0000 7f98548d4700 1 --2- 172.21.15.36:0/2286625207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98500b34c0 0x7f9850131930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:25.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.956+0000 7f98558d6700 1 --2- 172.21.15.36:0/2286625207 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98500b6860 0x7f9850150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:25.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.956+0000 7f98550d5700 1 -- 172.21.15.36:0/2286625207 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98500b6860 msgr2=0x7f9850150040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:25.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.956+0000 7f98550d5700 1 --2- 172.21.15.36:0/2286625207 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98500b6860 0x7f9850150040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:25.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.956+0000 7f98550d5700 1 -- 172.21.15.36:0/2286625207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98500b34c0 msgr2=0x7f9850131930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:25.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.956+0000 7f98550d5700 1 --2- 172.21.15.36:0/2286625207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98500b34c0 0x7f9850131930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:25.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.956+0000 7f98550d5700 1 -- 172.21.15.36:0/2286625207 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9848009530 con 0x7f9850063730 2022-06-17T12:54:25.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.956+0000 7f98548d4700 1 --2- 172.21.15.36:0/2286625207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98500b34c0 0x7f9850131930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:25.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.956+0000 7f98558d6700 1 --2- 172.21.15.36:0/2286625207 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98500b6860 0x7f9850150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:25.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.956+0000 7f98550d5700 1 --2- 172.21.15.36:0/2286625207 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9850063730 0x7f9850135cf0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0x7f98500b6620 tx=0x7f984c007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:25.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.960+0000 7f983e7fc700 1 -- 172.21.15.36:0/2286625207 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f984c00ce30 con 0x7f9850063730 2022-06-17T12:54:25.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.960+0000 7f983e7fc700 1 -- 172.21.15.36:0/2286625207 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984c00f470 con 0x7f9850063730 2022-06-17T12:54:25.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.960+0000 7f983e7fc700 1 -- 172.21.15.36:0/2286625207 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f984c00f7d0 con 0x7f9850063730 2022-06-17T12:54:25.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.960+0000 7f98560d7700 1 -- 172.21.15.36:0/2286625207 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9850131fd0 con 0x7f9850063730 2022-06-17T12:54:25.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.960+0000 7f98560d7700 1 -- 172.21.15.36:0/2286625207 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9850150d10 con 0x7f9850063730 2022-06-17T12:54:25.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.960+0000 7f98560d7700 1 -- 172.21.15.36:0/2286625207 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9850131e20 con 0x7f9850063730 2022-06-17T12:54:25.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.960+0000 7f983e7fc700 1 -- 172.21.15.36:0/2286625207 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f984c010070 con 0x7f9850063730 2022-06-17T12:54:25.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.960+0000 7f983e7fc700 1 --2- 172.21.15.36:0/2286625207 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9830053940 0x7f9830055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:25.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.960+0000 7f983e7fc700 1 -- 172.21.15.36:0/2286625207 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f984c06b0d0 con 0x7f9850063730 2022-06-17T12:54:25.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.960+0000 7f98548d4700 1 --2- 172.21.15.36:0/2286625207 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9830053940 0x7f9830055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:25.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.964+0000 7f98548d4700 1 --2- 172.21.15.36:0/2286625207 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9830053940 0x7f9830055e00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f984000b000 tx=0x7f9840008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:25.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:25.968+0000 7f983e7fc700 1 -- 172.21.15.36:0/2286625207 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f984c041e80 con 0x7f9850063730 2022-06-17T12:54:26.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.096+0000 7f98560d7700 1 -- 172.21.15.36:0/2286625207 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f985005fa80 con 0x7f9850063730 2022-06-17T12:54:26.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.812+0000 7f983e7fc700 1 -- 172.21.15.36:0/2286625207 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v86) v1 ==== 185+0+0 (secure 0 0 0) 0x7f984c0454a0 con 0x7f9850063730 2022-06-17T12:54:26.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.816+0000 7f98560d7700 1 -- 172.21.15.36:0/2286625207 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9830053940 msgr2=0x7f9830055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:26.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.816+0000 7f98560d7700 1 --2- 172.21.15.36:0/2286625207 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9830053940 0x7f9830055e00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f984000b000 tx=0x7f9840008040).stop 2022-06-17T12:54:26.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.816+0000 7f98560d7700 1 -- 172.21.15.36:0/2286625207 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9850063730 msgr2=0x7f9850135cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:26.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.816+0000 7f98560d7700 1 --2- 172.21.15.36:0/2286625207 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9850063730 0x7f9850135cf0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0x7f98500b6620 tx=0x7f984c007c50).stop 2022-06-17T12:54:26.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.816+0000 7f98560d7700 1 -- 172.21.15.36:0/2286625207 shutdown_connections 2022-06-17T12:54:26.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.816+0000 7f98560d7700 1 --2- 172.21.15.36:0/2286625207 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9830053940 0x7f9830055e00 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:26.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.816+0000 7f98560d7700 1 --2- 172.21.15.36:0/2286625207 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98500b6860 0x7f9850150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:26.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.816+0000 7f98560d7700 1 --2- 172.21.15.36:0/2286625207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98500b34c0 0x7f9850131930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:26.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.816+0000 7f98560d7700 1 --2- 172.21.15.36:0/2286625207 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9850063730 0x7f9850135cf0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:26.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.816+0000 7f98560d7700 1 -- 172.21.15.36:0/2286625207 >> 172.21.15.36:0/2286625207 conn(0x7f9850060880 msgr2=0x7f98500b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:26.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.816+0000 7f98560d7700 1 -- 172.21.15.36:0/2286625207 shutdown_connections 2022-06-17T12:54:26.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.816+0000 7f98560d7700 1 -- 172.21.15.36:0/2286625207 wait complete. 2022-06-17T12:54:26.810 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' removed 2022-06-17T12:54:26.822 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-17T12:54:26.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.956+0000 7fa501528700 1 Processor -- start 2022-06-17T12:54:26.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.956+0000 7fa501528700 1 -- start start 2022-06-17T12:54:26.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa501528700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4fc0b9860 0x7fa4fc143840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:26.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa501528700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa4fc0b45c0 0x7fa4fc0b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:26.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa501528700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa4fc0b4ed0 0x7fa4fc146490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:26.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa501528700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4fc0b4160 con 0x7fa4fc0b4ed0 2022-06-17T12:54:26.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa501528700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4fc0b9c40 con 0x7fa4fc0b45c0 2022-06-17T12:54:26.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa501528700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc0627d0 con 0x7fa4fc0b9860 2022-06-17T12:54:26.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa500d27700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa4fc0b4ed0 0x7fa4fc146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:26.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa500d27700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa4fc0b4ed0 0x7fa4fc146490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36748/0 (socket says 172.21.15.36:36748) 2022-06-17T12:54:26.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa500d27700 1 -- 172.21.15.36:0/1008222041 learned_addr learned my addr 172.21.15.36:0/1008222041 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:26.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa4f3fff700 1 --2- 172.21.15.36:0/1008222041 >> v2:172.21.15.70:3300/0 conn(0x7fa4fc0b45c0 0x7fa4fc0b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:26.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa4fbfff700 1 --2- 172.21.15.36:0/1008222041 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4fc0b9860 0x7fa4fc143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:26.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa500d27700 1 -- 172.21.15.36:0/1008222041 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4fc0b9860 msgr2=0x7fa4fc143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:26.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa500d27700 1 --2- 172.21.15.36:0/1008222041 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4fc0b9860 0x7fa4fc143840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:26.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa500d27700 1 -- 172.21.15.36:0/1008222041 >> v2:172.21.15.70:3300/0 conn(0x7fa4fc0b45c0 msgr2=0x7fa4fc0b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:26.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa500d27700 1 --2- 172.21.15.36:0/1008222041 >> v2:172.21.15.70:3300/0 conn(0x7fa4fc0b45c0 0x7fa4fc0b49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:26.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa500d27700 1 -- 172.21.15.36:0/1008222041 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4fc062a70 con 0x7fa4fc0b4ed0 2022-06-17T12:54:26.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa4f3fff700 1 --2- 172.21.15.36:0/1008222041 >> v2:172.21.15.70:3300/0 conn(0x7fa4fc0b45c0 0x7fa4fc0b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:26.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa500d27700 1 --2- 172.21.15.36:0/1008222041 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4fc0b4ed0 0x7fa4fc146490 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7fa4f40032f0 tx=0x7fa4f4009290).ready entity=mon.0 client_cookie=d7b1d7d8dadfdd77 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:26.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa4fb7fe700 1 -- 172.21.15.36:0/1008222041 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4f400b680 con 0x7fa4fc0b4ed0 2022-06-17T12:54:26.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa4fb7fe700 1 -- 172.21.15.36:0/1008222041 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f400b7e0 con 0x7fa4fc0b4ed0 2022-06-17T12:54:26.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa4fb7fe700 1 -- 172.21.15.36:0/1008222041 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4f400b950 con 0x7fa4fc0b4ed0 2022-06-17T12:54:26.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa501528700 1 -- 172.21.15.36:0/1008222041 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4fc0b4ed0 msgr2=0x7fa4fc146490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:26.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa501528700 1 --2- 172.21.15.36:0/1008222041 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4fc0b4ed0 0x7fa4fc146490 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7fa4f40032f0 tx=0x7fa4f4009290).stop 2022-06-17T12:54:26.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa501528700 1 -- 172.21.15.36:0/1008222041 shutdown_connections 2022-06-17T12:54:26.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa501528700 1 --2- 172.21.15.36:0/1008222041 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4fc0b4ed0 0x7fa4fc146490 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:26.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa501528700 1 --2- 172.21.15.36:0/1008222041 >> v2:172.21.15.70:3300/0 conn(0x7fa4fc0b45c0 0x7fa4fc0b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:26.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa501528700 1 --2- 172.21.15.36:0/1008222041 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4fc0b9860 0x7fa4fc143840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:26.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa501528700 1 -- 172.21.15.36:0/1008222041 >> 172.21.15.36:0/1008222041 conn(0x7fa4fc05c550 msgr2=0x7fa4fc060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:26.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa501528700 1 -- 172.21.15.36:0/1008222041 shutdown_connections 2022-06-17T12:54:26.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.960+0000 7fa501528700 1 -- 172.21.15.36:0/1008222041 wait complete. 2022-06-17T12:54:26.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.964+0000 7fa501528700 1 Processor -- start 2022-06-17T12:54:26.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.964+0000 7fa501528700 1 -- start start 2022-06-17T12:54:26.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.964+0000 7fa501528700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4fc0b45c0 0x7fa4fc1456b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:26.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.964+0000 7fa501528700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4fc0b4ed0 0x7fa4fc1524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:26.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.964+0000 7fa501528700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4fc0b9860 0x7fa4fc1529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:26.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.964+0000 7fa501528700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc152080 con 0x7fa4fc0b4ed0 2022-06-17T12:54:26.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.964+0000 7fa501528700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc152ec0 con 0x7fa4fc0b9860 2022-06-17T12:54:26.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.964+0000 7fa501528700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc13d280 con 0x7fa4fc0b45c0 2022-06-17T12:54:26.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.964+0000 7fa4f3fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4fc0b4ed0 0x7fa4fc1524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:26.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.964+0000 7fa500d27700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4fc0b9860 0x7fa4fc1529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:26.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.964+0000 7fa4f3fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4fc0b4ed0 0x7fa4fc1524e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36754/0 (socket says 172.21.15.36:36754) 2022-06-17T12:54:26.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.964+0000 7fa4f3fff700 1 -- 172.21.15.36:0/764316051 learned_addr learned my addr 172.21.15.36:0/764316051 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:26.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.964+0000 7fa4fbfff700 1 --2- 172.21.15.36:0/764316051 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4fc0b45c0 0x7fa4fc1456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:26.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.964+0000 7fa4f3fff700 1 -- 172.21.15.36:0/764316051 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4fc0b45c0 msgr2=0x7fa4fc1456b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:26.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.964+0000 7fa4f3fff700 1 --2- 172.21.15.36:0/764316051 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4fc0b45c0 0x7fa4fc1456b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:26.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.964+0000 7fa4f3fff700 1 -- 172.21.15.36:0/764316051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4fc0b9860 msgr2=0x7fa4fc1529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:26.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.964+0000 7fa4f3fff700 1 --2- 172.21.15.36:0/764316051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4fc0b9860 0x7fa4fc1529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:26.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.964+0000 7fa4f3fff700 1 -- 172.21.15.36:0/764316051 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e400b180 con 0x7fa4fc0b4ed0 2022-06-17T12:54:26.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.964+0000 7fa4f3fff700 1 --2- 172.21.15.36:0/764316051 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4fc0b4ed0 0x7fa4fc1524e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7fa4e400c000 tx=0x7fa4e4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:26.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.964+0000 7fa4f9ffb700 1 -- 172.21.15.36:0/764316051 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4e400c890 con 0x7fa4fc0b4ed0 2022-06-17T12:54:26.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.964+0000 7fa4f9ffb700 1 -- 172.21.15.36:0/764316051 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e400c9f0 con 0x7fa4fc0b4ed0 2022-06-17T12:54:26.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.964+0000 7fa501528700 1 -- 172.21.15.36:0/764316051 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f4009530 con 0x7fa4fc0b4ed0 2022-06-17T12:54:26.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.964+0000 7fa4f9ffb700 1 -- 172.21.15.36:0/764316051 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4e400cd50 con 0x7fa4fc0b4ed0 2022-06-17T12:54:26.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.964+0000 7fa501528700 1 -- 172.21.15.36:0/764316051 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4fc12d5d0 con 0x7fa4fc0b4ed0 2022-06-17T12:54:26.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.968+0000 7fa501528700 1 -- 172.21.15.36:0/764316051 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4fc062eb0 con 0x7fa4fc0b4ed0 2022-06-17T12:54:26.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.968+0000 7fa4f9ffb700 1 -- 172.21.15.36:0/764316051 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fa4e401c460 con 0x7fa4fc0b4ed0 2022-06-17T12:54:26.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.968+0000 7fa4f9ffb700 1 --2- 172.21.15.36:0/764316051 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa4e0053a40 0x7fa4e0055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:26.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.968+0000 7fa4f9ffb700 1 -- 172.21.15.36:0/764316051 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6068+0+0 (secure 0 0 0) 0x7fa4e406c450 con 0x7fa4fc0b4ed0 2022-06-17T12:54:26.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.968+0000 7fa4fbfff700 1 --2- 172.21.15.36:0/764316051 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa4e0053a40 0x7fa4e0055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:26.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.968+0000 7fa4fbfff700 1 --2- 172.21.15.36:0/764316051 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa4e0053a40 0x7fa4e0055f00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7fa4ec00b000 tx=0x7fa4ec006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:26.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:26.972+0000 7fa4f9ffb700 1 -- 172.21.15.36:0/764316051 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa4e4043350 con 0x7fa4fc0b4ed0 2022-06-17T12:54:27.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.096+0000 7fa501528700 1 -- 172.21.15.36:0/764316051 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa4fc057410 con 0x7fa4fc0b4ed0 2022-06-17T12:54:27.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.824+0000 7fa4f9ffb700 1 -- 172.21.15.36:0/764316051 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v87) v1 ==== 173+0+0 (secure 0 0 0) 0x7fa4e4046970 con 0x7fa4fc0b4ed0 2022-06-17T12:54:27.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.828+0000 7fa501528700 1 -- 172.21.15.36:0/764316051 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa4e0053a40 msgr2=0x7fa4e0055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:27.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.828+0000 7fa501528700 1 --2- 172.21.15.36:0/764316051 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa4e0053a40 0x7fa4e0055f00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7fa4ec00b000 tx=0x7fa4ec006df0).stop 2022-06-17T12:54:27.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.828+0000 7fa501528700 1 -- 172.21.15.36:0/764316051 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4fc0b4ed0 msgr2=0x7fa4fc1524e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:27.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.828+0000 7fa501528700 1 --2- 172.21.15.36:0/764316051 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4fc0b4ed0 0x7fa4fc1524e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7fa4e400c000 tx=0x7fa4e4006df0).stop 2022-06-17T12:54:27.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.828+0000 7fa501528700 1 -- 172.21.15.36:0/764316051 shutdown_connections 2022-06-17T12:54:27.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.828+0000 7fa501528700 1 --2- 172.21.15.36:0/764316051 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa4e0053a40 0x7fa4e0055f00 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:27.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.828+0000 7fa501528700 1 --2- 172.21.15.36:0/764316051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4fc0b9860 0x7fa4fc1529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:27.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.828+0000 7fa501528700 1 --2- 172.21.15.36:0/764316051 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4fc0b4ed0 0x7fa4fc1524e0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:27.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.828+0000 7fa501528700 1 --2- 172.21.15.36:0/764316051 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4fc0b45c0 0x7fa4fc1456b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:27.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.828+0000 7fa501528700 1 -- 172.21.15.36:0/764316051 >> 172.21.15.36:0/764316051 conn(0x7fa4fc05c550 msgr2=0x7fa4fc146850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:27.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.828+0000 7fa501528700 1 -- 172.21.15.36:0/764316051 shutdown_connections 2022-06-17T12:54:27.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.828+0000 7fa501528700 1 -- 172.21.15.36:0/764316051 wait complete. 2022-06-17T12:54:27.821 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' removed 2022-06-17T12:54:27.831 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-06-17T12:54:27.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.952+0000 7fd821a2f700 1 Processor -- start 2022-06-17T12:54:27.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.952+0000 7fd821a2f700 1 -- start start 2022-06-17T12:54:27.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.952+0000 7fd821a2f700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd81c060ac0 0x7fd81c140260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:27.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.952+0000 7fd821a2f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd81c0b2c80 0x7fd81c140750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:27.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.952+0000 7fd821a2f700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd81c060ea0 con 0x7fd81c067260 2022-06-17T12:54:27.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.952+0000 7fd821a2f700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd81c0b2210 con 0x7fd81c0b2c80 2022-06-17T12:54:27.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.952+0000 7fd821a2f700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd81c0aebb0 con 0x7fd81c060ac0 2022-06-17T12:54:27.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.952+0000 7fd81bfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd81c0b2c80 0x7fd81c140750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:27.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.952+0000 7fd81bfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd81c0b2c80 0x7fd81c140750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36762/0 (socket says 172.21.15.36:36762) 2022-06-17T12:54:27.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.952+0000 7fd820a2d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd81c060ac0 0x7fd81c140260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:27.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.952+0000 7fd81bfff700 1 -- 172.21.15.36:0/3434549802 learned_addr learned my addr 172.21.15.36:0/3434549802 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:27.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.952+0000 7fd82122e700 1 --1- 172.21.15.36:0/3434549802 >> v1:172.21.15.70:6789/0 conn(0x7fd81c067260 0x7fd81c0aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:50028/0 (socket says 172.21.15.36:50028) 2022-06-17T12:54:27.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.952+0000 7fd81bfff700 1 -- 172.21.15.36:0/3434549802 >> v2:172.21.15.70:3300/0 conn(0x7fd81c060ac0 msgr2=0x7fd81c140260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:27.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.952+0000 7fd81bfff700 1 --2- 172.21.15.36:0/3434549802 >> v2:172.21.15.70:3300/0 conn(0x7fd81c060ac0 0x7fd81c140260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:27.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.952+0000 7fd81bfff700 1 -- 172.21.15.36:0/3434549802 >> v1:172.21.15.70:6789/0 conn(0x7fd81c067260 legacy=0x7fd81c0aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:27.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.952+0000 7fd81bfff700 1 -- 172.21.15.36:0/3434549802 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c0b1ad0 con 0x7fd81c0b2c80 2022-06-17T12:54:27.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.952+0000 7fd820a2d700 1 --2- 172.21.15.36:0/3434549802 >> v2:172.21.15.70:3300/0 conn(0x7fd81c060ac0 0x7fd81c140260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:27.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.956+0000 7fd81bfff700 1 --2- 172.21.15.36:0/3434549802 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd81c0b2c80 0x7fd81c140750 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0x7fd810002f70 tx=0x7fd810006eb0).ready entity=mon.0 client_cookie=19e9b8f317342b79 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:27.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.956+0000 7fd81b7fe700 1 -- 172.21.15.36:0/3434549802 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd81000d5c0 con 0x7fd81c0b2c80 2022-06-17T12:54:27.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.956+0000 7fd81b7fe700 1 -- 172.21.15.36:0/3434549802 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd810000e90 con 0x7fd81c0b2c80 2022-06-17T12:54:27.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.956+0000 7fd81b7fe700 1 -- 172.21.15.36:0/3434549802 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd81000d950 con 0x7fd81c0b2c80 2022-06-17T12:54:27.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.956+0000 7fd821a2f700 1 -- 172.21.15.36:0/3434549802 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd81c0b2c80 msgr2=0x7fd81c140750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:27.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.956+0000 7fd821a2f700 1 --2- 172.21.15.36:0/3434549802 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd81c0b2c80 0x7fd81c140750 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0x7fd810002f70 tx=0x7fd810006eb0).stop 2022-06-17T12:54:27.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.956+0000 7fd821a2f700 1 -- 172.21.15.36:0/3434549802 shutdown_connections 2022-06-17T12:54:27.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.956+0000 7fd821a2f700 1 --2- 172.21.15.36:0/3434549802 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd81c0b2c80 0x7fd81c140750 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:27.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.956+0000 7fd821a2f700 1 --2- 172.21.15.36:0/3434549802 >> v2:172.21.15.70:3300/0 conn(0x7fd81c060ac0 0x7fd81c140260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:27.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.956+0000 7fd821a2f700 1 -- 172.21.15.36:0/3434549802 >> 172.21.15.36:0/3434549802 conn(0x7fd81c0b64f0 msgr2=0x7fd81c0b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:27.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.956+0000 7fd821a2f700 1 -- 172.21.15.36:0/3434549802 shutdown_connections 2022-06-17T12:54:27.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.956+0000 7fd821a2f700 1 -- 172.21.15.36:0/3434549802 wait complete. 2022-06-17T12:54:27.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.956+0000 7fd821a2f700 1 Processor -- start 2022-06-17T12:54:27.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.960+0000 7fd821a2f700 1 -- start start 2022-06-17T12:54:27.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.960+0000 7fd821a2f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd81c060ac0 0x7fd81c143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:27.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.960+0000 7fd821a2f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd81c067260 0x7fd81c1479c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:27.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.960+0000 7fd821a2f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd81c0b2c80 0x7fd81c13b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:27.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.960+0000 7fd821a2f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c144cd0 con 0x7fd81c060ac0 2022-06-17T12:54:27.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.960+0000 7fd821a2f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c147280 con 0x7fd81c067260 2022-06-17T12:54:27.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.960+0000 7fd821a2f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c147520 con 0x7fd81c0b2c80 2022-06-17T12:54:27.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.960+0000 7fd820a2d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd81c060ac0 0x7fd81c143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:27.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.960+0000 7fd820a2d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd81c060ac0 0x7fd81c143ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36766/0 (socket says 172.21.15.36:36766) 2022-06-17T12:54:27.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.960+0000 7fd820a2d700 1 -- 172.21.15.36:0/1530460682 learned_addr learned my addr 172.21.15.36:0/1530460682 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:27.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.960+0000 7fd81bfff700 1 --2- 172.21.15.36:0/1530460682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd81c067260 0x7fd81c1479c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:27.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.960+0000 7fd82122e700 1 --2- 172.21.15.36:0/1530460682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd81c0b2c80 0x7fd81c13b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:27.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.960+0000 7fd820a2d700 1 -- 172.21.15.36:0/1530460682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd81c0b2c80 msgr2=0x7fd81c13b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:27.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.960+0000 7fd820a2d700 1 --2- 172.21.15.36:0/1530460682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd81c0b2c80 0x7fd81c13b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:27.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.960+0000 7fd820a2d700 1 -- 172.21.15.36:0/1530460682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd81c067260 msgr2=0x7fd81c1479c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:27.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.960+0000 7fd820a2d700 1 --2- 172.21.15.36:0/1530460682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd81c067260 0x7fd81c1479c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:27.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.960+0000 7fd820a2d700 1 -- 172.21.15.36:0/1530460682 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80c00b180 con 0x7fd81c060ac0 2022-06-17T12:54:27.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.960+0000 7fd81bfff700 1 --2- 172.21.15.36:0/1530460682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd81c067260 0x7fd81c1479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:27.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.960+0000 7fd820a2d700 1 --2- 172.21.15.36:0/1530460682 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd81c060ac0 0x7fd81c143ab0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7fd80c00c000 tx=0x7fd80c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:27.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.960+0000 7fd819ffb700 1 -- 172.21.15.36:0/1530460682 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd80c00c7e0 con 0x7fd81c060ac0 2022-06-17T12:54:27.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.960+0000 7fd819ffb700 1 -- 172.21.15.36:0/1530460682 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80c00c940 con 0x7fd81c060ac0 2022-06-17T12:54:27.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.960+0000 7fd821a2f700 1 -- 172.21.15.36:0/1530460682 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd81000b1e0 con 0x7fd81c060ac0 2022-06-17T12:54:27.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.960+0000 7fd819ffb700 1 -- 172.21.15.36:0/1530460682 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd80c00cca0 con 0x7fd81c060ac0 2022-06-17T12:54:27.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.960+0000 7fd821a2f700 1 -- 172.21.15.36:0/1530460682 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd81c147eb0 con 0x7fd81c060ac0 2022-06-17T12:54:27.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.964+0000 7fd821a2f700 1 -- 172.21.15.36:0/1530460682 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd81c144af0 con 0x7fd81c060ac0 2022-06-17T12:54:27.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.964+0000 7fd819ffb700 1 -- 172.21.15.36:0/1530460682 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd80c01c460 con 0x7fd81c060ac0 2022-06-17T12:54:27.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.964+0000 7fd819ffb700 1 --2- 172.21.15.36:0/1530460682 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7fc053a60 0x7fd7fc055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:27.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.964+0000 7fd819ffb700 1 -- 172.21.15.36:0/1530460682 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fd80c06c2b0 con 0x7fd81c060ac0 2022-06-17T12:54:27.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.964+0000 7fd81bfff700 1 --2- 172.21.15.36:0/1530460682 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7fc053a60 0x7fd7fc055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:27.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.964+0000 7fd81bfff700 1 --2- 172.21.15.36:0/1530460682 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7fc053a60 0x7fd7fc055f20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7fd81001c000 tx=0x7fd810015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:27.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:27.968+0000 7fd819ffb700 1 -- 172.21.15.36:0/1530460682 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd80c043330 con 0x7fd81c060ac0 2022-06-17T12:54:28.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.096+0000 7fd821a2f700 1 -- 172.21.15.36:0/1530460682 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd81c057410 con 0x7fd81c060ac0 2022-06-17T12:54:28.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.104+0000 7fd819ffb700 1 -- 172.21.15.36:0/1530460682 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd80c046950 con 0x7fd81c060ac0 2022-06-17T12:54:28.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.108+0000 7fd821a2f700 1 -- 172.21.15.36:0/1530460682 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7fc053a60 msgr2=0x7fd7fc055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:28.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.108+0000 7fd821a2f700 1 --2- 172.21.15.36:0/1530460682 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7fc053a60 0x7fd7fc055f20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7fd81001c000 tx=0x7fd810015000).stop 2022-06-17T12:54:28.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.108+0000 7fd821a2f700 1 -- 172.21.15.36:0/1530460682 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd81c060ac0 msgr2=0x7fd81c143ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:28.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.108+0000 7fd821a2f700 1 --2- 172.21.15.36:0/1530460682 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd81c060ac0 0x7fd81c143ab0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7fd80c00c000 tx=0x7fd80c006df0).stop 2022-06-17T12:54:28.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.108+0000 7fd821a2f700 1 -- 172.21.15.36:0/1530460682 shutdown_connections 2022-06-17T12:54:28.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.108+0000 7fd821a2f700 1 --2- 172.21.15.36:0/1530460682 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7fc053a60 0x7fd7fc055f20 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:28.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.108+0000 7fd821a2f700 1 --2- 172.21.15.36:0/1530460682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd81c0b2c80 0x7fd81c13b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:28.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.108+0000 7fd821a2f700 1 --2- 172.21.15.36:0/1530460682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd81c067260 0x7fd81c1479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:28.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.108+0000 7fd821a2f700 1 --2- 172.21.15.36:0/1530460682 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd81c060ac0 0x7fd81c143ab0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:28.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.108+0000 7fd821a2f700 1 -- 172.21.15.36:0/1530460682 >> 172.21.15.36:0/1530460682 conn(0x7fd81c0b64f0 msgr2=0x7fd81c0b3860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:28.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.108+0000 7fd821a2f700 1 -- 172.21.15.36:0/1530460682 shutdown_connections 2022-06-17T12:54:28.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.108+0000 7fd821a2f700 1 -- 172.21.15.36:0/1530460682 wait complete. 2022-06-17T12:54:28.114 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode' 2022-06-17T12:54:28.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.232+0000 7f467ffff700 1 Processor -- start 2022-06-17T12:54:28.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.232+0000 7f467ffff700 1 -- start start 2022-06-17T12:54:28.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.232+0000 7f467ffff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f468014b2b0 0x7f4680060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:28.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.232+0000 7f467ffff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f468014b870 con 0x7f46800b27b0 2022-06-17T12:54:28.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.232+0000 7f467ffff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46800ac500 con 0x7f4680147bd0 2022-06-17T12:54:28.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.232+0000 7f467ffff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46800ace90 con 0x7f468014b2b0 2022-06-17T12:54:28.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.232+0000 7f467effd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f46800b27b0 0x7f46800ac780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54304/0 (socket says 172.21.15.36:54304) 2022-06-17T12:54:28.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.232+0000 7f467effd700 1 -- 172.21.15.36:0/2575738330 learned_addr learned my addr 172.21.15.36:0/2575738330 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:28.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.236+0000 7f467e7fc700 1 --2- 172.21.15.36:0/2575738330 >> v2:172.21.15.70:3300/0 conn(0x7f468014b2b0 0x7f4680060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:28.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.236+0000 7f467dffb700 1 -- 172.21.15.36:0/2575738330 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307669707 0 0) 0x7f468014b870 con 0x7f46800b27b0 2022-06-17T12:54:28.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.236+0000 7f467dffb700 1 -- 172.21.15.36:0/2575738330 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4664003260 con 0x7f46800b27b0 2022-06-17T12:54:28.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.236+0000 7f467dffb700 1 -- 172.21.15.36:0/2575738330 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765626334 0 0) 0x7f46800ac500 con 0x7f4680147bd0 2022-06-17T12:54:28.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.236+0000 7f467dffb700 1 -- 172.21.15.36:0/2575738330 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f468014b870 con 0x7f4680147bd0 2022-06-17T12:54:28.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.236+0000 7f467e7fc700 1 -- 172.21.15.36:0/2575738330 >> v1:172.21.15.70:6789/0 conn(0x7f4680147bd0 legacy=0x7f46801442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:28.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.236+0000 7f467e7fc700 1 -- 172.21.15.36:0/2575738330 >> v1:172.21.15.36:6789/0 conn(0x7f46800b27b0 legacy=0x7f46800ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:28.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.236+0000 7f467e7fc700 1 -- 172.21.15.36:0/2575738330 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46800b2b90 con 0x7f468014b2b0 2022-06-17T12:54:28.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.236+0000 7f467e7fc700 1 --2- 172.21.15.36:0/2575738330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f468014b2b0 0x7f4680060ca0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f46700032f0 tx=0x7f46700091d0).ready entity=mon.1 client_cookie=132fca82007164e9 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:28.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.236+0000 7f467dffb700 1 -- 172.21.15.36:0/2575738330 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f467000b600 con 0x7f468014b2b0 2022-06-17T12:54:28.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.236+0000 7f467dffb700 1 -- 172.21.15.36:0/2575738330 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467000b760 con 0x7f468014b2b0 2022-06-17T12:54:28.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.236+0000 7f467dffb700 1 -- 172.21.15.36:0/2575738330 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f467000b8d0 con 0x7f468014b2b0 2022-06-17T12:54:28.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.236+0000 7f467ffff700 1 -- 172.21.15.36:0/2575738330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f468014b2b0 msgr2=0x7f4680060ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:28.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.236+0000 7f467ffff700 1 --2- 172.21.15.36:0/2575738330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f468014b2b0 0x7f4680060ca0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f46700032f0 tx=0x7f46700091d0).stop 2022-06-17T12:54:28.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.236+0000 7f467ffff700 1 -- 172.21.15.36:0/2575738330 shutdown_connections 2022-06-17T12:54:28.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.236+0000 7f467ffff700 1 --2- 172.21.15.36:0/2575738330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f468014b2b0 0x7f4680060ca0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:28.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.236+0000 7f467ffff700 1 -- 172.21.15.36:0/2575738330 >> 172.21.15.36:0/2575738330 conn(0x7f46800b5d10 msgr2=0x7f46800b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:28.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.236+0000 7f467ffff700 1 -- 172.21.15.36:0/2575738330 shutdown_connections 2022-06-17T12:54:28.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.236+0000 7f467ffff700 1 -- 172.21.15.36:0/2575738330 wait complete. 2022-06-17T12:54:28.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.240+0000 7f467ffff700 1 Processor -- start 2022-06-17T12:54:28.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.240+0000 7f467ffff700 1 -- start start 2022-06-17T12:54:28.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.240+0000 7f467ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46800b27b0 0x7f4680148b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:28.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.240+0000 7f467ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4680147bd0 0x7f4680143cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:28.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.240+0000 7f467ffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f468014b2b0 0x7f4680142070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:28.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.240+0000 7f467ffff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4680141030 con 0x7f4680147bd0 2022-06-17T12:54:28.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.240+0000 7f467ffff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468015ad00 con 0x7f46800b27b0 2022-06-17T12:54:28.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.240+0000 7f467ffff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4680149330 con 0x7f468014b2b0 2022-06-17T12:54:28.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.240+0000 7f467e7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4680147bd0 0x7f4680143cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:28.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.240+0000 7f467e7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4680147bd0 0x7f4680143cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36782/0 (socket says 172.21.15.36:36782) 2022-06-17T12:54:28.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.240+0000 7f467e7fc700 1 -- 172.21.15.36:0/3093470738 learned_addr learned my addr 172.21.15.36:0/3093470738 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:28.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.240+0000 7f467effd700 1 --2- 172.21.15.36:0/3093470738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46800b27b0 0x7f4680148b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:28.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.240+0000 7f467f7fe700 1 --2- 172.21.15.36:0/3093470738 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f468014b2b0 0x7f4680142070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:28.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.240+0000 7f467e7fc700 1 -- 172.21.15.36:0/3093470738 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f468014b2b0 msgr2=0x7f4680142070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:28.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.240+0000 7f467e7fc700 1 --2- 172.21.15.36:0/3093470738 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f468014b2b0 0x7f4680142070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:28.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.240+0000 7f467e7fc700 1 -- 172.21.15.36:0/3093470738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46800b27b0 msgr2=0x7f4680148b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:28.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.240+0000 7f467e7fc700 1 --2- 172.21.15.36:0/3093470738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46800b27b0 0x7f4680148b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:28.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.240+0000 7f467e7fc700 1 -- 172.21.15.36:0/3093470738 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4670009470 con 0x7f4680147bd0 2022-06-17T12:54:28.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.240+0000 7f467e7fc700 1 --2- 172.21.15.36:0/3093470738 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4680147bd0 0x7f4680143cb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7f46700032f0 tx=0x7f4670016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:28.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.240+0000 7f466ffff700 1 -- 172.21.15.36:0/3093470738 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f467000b600 con 0x7f4680147bd0 2022-06-17T12:54:28.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.240+0000 7f466ffff700 1 -- 172.21.15.36:0/3093470738 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4670011e60 con 0x7f4680147bd0 2022-06-17T12:54:28.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.240+0000 7f467ffff700 1 -- 172.21.15.36:0/3093470738 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f468015a820 con 0x7f4680147bd0 2022-06-17T12:54:28.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.240+0000 7f466ffff700 1 -- 172.21.15.36:0/3093470738 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f467001f5e0 con 0x7f4680147bd0 2022-06-17T12:54:28.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.244+0000 7f467ffff700 1 -- 172.21.15.36:0/3093470738 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4680149090 con 0x7f4680147bd0 2022-06-17T12:54:28.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.244+0000 7f467ffff700 1 -- 172.21.15.36:0/3093470738 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4680057410 con 0x7f4680147bd0 2022-06-17T12:54:28.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.244+0000 7f466ffff700 1 -- 172.21.15.36:0/3093470738 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f467000f070 con 0x7f4680147bd0 2022-06-17T12:54:28.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.244+0000 7f466ffff700 1 --2- 172.21.15.36:0/3093470738 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f465c0539d0 0x7f465c055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:28.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.244+0000 7f467effd700 1 --2- 172.21.15.36:0/3093470738 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f465c0539d0 0x7f465c055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:28.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.244+0000 7f466ffff700 1 -- 172.21.15.36:0/3093470738 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f467006c360 con 0x7f4680147bd0 2022-06-17T12:54:28.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.244+0000 7f467effd700 1 --2- 172.21.15.36:0/3093470738 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f465c0539d0 0x7f465c055e90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f467400d000 tx=0x7f4674009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:28.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.252+0000 7f466ffff700 1 -- 172.21.15.36:0/3093470738 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f46700433a0 con 0x7f4680147bd0 2022-06-17T12:54:28.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.420+0000 7f467ffff700 1 -- 172.21.15.36:0/3093470738 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]} v 0) v1 -- 0x7f468014af60 con 0x7f4680147bd0 2022-06-17T12:54:28.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.824+0000 7f466ffff700 1 -- 172.21.15.36:0/3093470738 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]}]=0 v344) v1 ==== 126+0+0 (secure 0 0 0) 0x7f46700150c0 con 0x7f4680147bd0 2022-06-17T12:54:28.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.828+0000 7f467ffff700 1 -- 172.21.15.36:0/3093470738 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f465c0539d0 msgr2=0x7f465c055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:28.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.828+0000 7f467ffff700 1 --2- 172.21.15.36:0/3093470738 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f465c0539d0 0x7f465c055e90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f467400d000 tx=0x7f4674009040).stop 2022-06-17T12:54:28.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.828+0000 7f467ffff700 1 -- 172.21.15.36:0/3093470738 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4680147bd0 msgr2=0x7f4680143cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:28.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.828+0000 7f467ffff700 1 --2- 172.21.15.36:0/3093470738 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4680147bd0 0x7f4680143cb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7f46700032f0 tx=0x7f4670016000).stop 2022-06-17T12:54:28.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.828+0000 7f467ffff700 1 -- 172.21.15.36:0/3093470738 shutdown_connections 2022-06-17T12:54:28.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.828+0000 7f467ffff700 1 --2- 172.21.15.36:0/3093470738 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f465c0539d0 0x7f465c055e90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:28.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.828+0000 7f467ffff700 1 --2- 172.21.15.36:0/3093470738 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f468014b2b0 0x7f4680142070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:28.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.828+0000 7f467ffff700 1 --2- 172.21.15.36:0/3093470738 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4680147bd0 0x7f4680143cb0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:28.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.828+0000 7f467ffff700 1 --2- 172.21.15.36:0/3093470738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46800b27b0 0x7f4680148b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:28.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.828+0000 7f467ffff700 1 -- 172.21.15.36:0/3093470738 >> 172.21.15.36:0/3093470738 conn(0x7f46800b5d10 msgr2=0x7f46800acfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:28.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.828+0000 7f467ffff700 1 -- 172.21.15.36:0/3093470738 shutdown_connections 2022-06-17T12:54:28.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.828+0000 7f467ffff700 1 -- 172.21.15.36:0/3093470738 wait complete. 2022-06-17T12:54:28.835 INFO:tasks.cephfs_test_runner:test_dump_inode (tasks.cephfs.test_misc.TestMisc) ... ok 2022-06-17T12:54:28.836 INFO:tasks.cephfs_test_runner:Starting test: test_dump_inode_hexademical (tasks.cephfs.test_misc.TestMisc) 2022-06-17T12:54:28.837 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical' 2022-06-17T12:54:28.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.956+0000 7fd502561700 1 Processor -- start 2022-06-17T12:54:28.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.956+0000 7fd502561700 1 -- start start 2022-06-17T12:54:28.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.960+0000 7fd502561700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd4fc0b45c0 0x7fd4fc0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:28.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.960+0000 7fd502561700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4fc0b4f00 0x7fd4fc0b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:28.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.960+0000 7fd502561700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4fc0b4160 con 0x7fd4fc0b9860 2022-06-17T12:54:28.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.960+0000 7fd502561700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4fc0b9c40 con 0x7fd4fc0b45c0 2022-06-17T12:54:28.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.960+0000 7fd502561700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc0627d0 con 0x7fd4fc0b4f00 2022-06-17T12:54:28.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.960+0000 7fd50155f700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fd4fc0b9860 0x7fd4fc143840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54318/0 (socket says 172.21.15.36:54318) 2022-06-17T12:54:28.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.960+0000 7fd50155f700 1 -- 172.21.15.36:0/735075379 learned_addr learned my addr 172.21.15.36:0/735075379 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:28.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.960+0000 7fd501d60700 1 --2- 172.21.15.36:0/735075379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4fc0b4f00 0x7fd4fc0b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:28.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.960+0000 7fd500d5e700 1 --2- 172.21.15.36:0/735075379 >> v2:172.21.15.70:3300/0 conn(0x7fd4fc0b45c0 0x7fd4fc0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:28.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.960+0000 7fd4ebfff700 1 -- 172.21.15.36:0/735075379 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 613268602 0 0) 0x7fd4fc0b4160 con 0x7fd4fc0b9860 2022-06-17T12:54:28.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.960+0000 7fd4ebfff700 1 -- 172.21.15.36:0/735075379 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4e4003260 con 0x7fd4fc0b9860 2022-06-17T12:54:28.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.960+0000 7fd501d60700 1 -- 172.21.15.36:0/735075379 >> v2:172.21.15.70:3300/0 conn(0x7fd4fc0b45c0 msgr2=0x7fd4fc0b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:28.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.960+0000 7fd501d60700 1 --2- 172.21.15.36:0/735075379 >> v2:172.21.15.70:3300/0 conn(0x7fd4fc0b45c0 0x7fd4fc0b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:28.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.960+0000 7fd501d60700 1 -- 172.21.15.36:0/735075379 >> v1:172.21.15.36:6789/0 conn(0x7fd4fc0b9860 legacy=0x7fd4fc143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:28.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.960+0000 7fd501d60700 1 -- 172.21.15.36:0/735075379 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4fc062a70 con 0x7fd4fc0b4f00 2022-06-17T12:54:28.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.960+0000 7fd501d60700 1 --2- 172.21.15.36:0/735075379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4fc0b4f00 0x7fd4fc0b5dc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fd4f40032f0 tx=0x7fd4f40091d0).ready entity=mon.2 client_cookie=91b504403f829d26 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:28.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.960+0000 7fd4ebfff700 1 -- 172.21.15.36:0/735075379 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4f400b600 con 0x7fd4fc0b4f00 2022-06-17T12:54:28.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.960+0000 7fd4ebfff700 1 -- 172.21.15.36:0/735075379 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f400b760 con 0x7fd4fc0b4f00 2022-06-17T12:54:28.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.960+0000 7fd4ebfff700 1 -- 172.21.15.36:0/735075379 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4f400b8d0 con 0x7fd4fc0b4f00 2022-06-17T12:54:28.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd502561700 1 -- 172.21.15.36:0/735075379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4fc0b4f00 msgr2=0x7fd4fc0b5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:28.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd502561700 1 --2- 172.21.15.36:0/735075379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4fc0b4f00 0x7fd4fc0b5dc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fd4f40032f0 tx=0x7fd4f40091d0).stop 2022-06-17T12:54:28.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd502561700 1 -- 172.21.15.36:0/735075379 shutdown_connections 2022-06-17T12:54:28.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd502561700 1 --2- 172.21.15.36:0/735075379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4fc0b4f00 0x7fd4fc0b5dc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:28.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd502561700 1 --2- 172.21.15.36:0/735075379 >> v2:172.21.15.70:3300/0 conn(0x7fd4fc0b45c0 0x7fd4fc0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:28.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd502561700 1 -- 172.21.15.36:0/735075379 >> 172.21.15.36:0/735075379 conn(0x7fd4fc05c550 msgr2=0x7fd4fc060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:28.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd502561700 1 -- 172.21.15.36:0/735075379 shutdown_connections 2022-06-17T12:54:28.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd502561700 1 -- 172.21.15.36:0/735075379 wait complete. 2022-06-17T12:54:28.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd502561700 1 Processor -- start 2022-06-17T12:54:28.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd502561700 1 -- start start 2022-06-17T12:54:28.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd502561700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4fc0b45c0 0x7fd4fc0ae090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:28.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd502561700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd4fc0b4f00 0x7fd4fc141f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:28.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd502561700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4fc0b9860 0x7fd4fc13d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:28.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd502561700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc13d250 con 0x7fd4fc0b4f00 2022-06-17T12:54:28.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd502561700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc0ad950 con 0x7fd4fc0b9860 2022-06-17T12:54:28.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd502561700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc0adbf0 con 0x7fd4fc0b45c0 2022-06-17T12:54:28.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd500d5e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd4fc0b4f00 0x7fd4fc141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:28.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd500d5e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd4fc0b4f00 0x7fd4fc141f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36796/0 (socket says 172.21.15.36:36796) 2022-06-17T12:54:28.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd500d5e700 1 -- 172.21.15.36:0/2738627093 learned_addr learned my addr 172.21.15.36:0/2738627093 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:28.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd501d60700 1 --2- 172.21.15.36:0/2738627093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4fc0b9860 0x7fd4fc13d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:28.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd50155f700 1 --2- 172.21.15.36:0/2738627093 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4fc0b45c0 0x7fd4fc0ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:28.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd500d5e700 1 -- 172.21.15.36:0/2738627093 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4fc0b45c0 msgr2=0x7fd4fc0ae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:28.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd500d5e700 1 --2- 172.21.15.36:0/2738627093 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4fc0b45c0 0x7fd4fc0ae090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:28.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd500d5e700 1 -- 172.21.15.36:0/2738627093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4fc0b9860 msgr2=0x7fd4fc13d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:28.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd500d5e700 1 --2- 172.21.15.36:0/2738627093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4fc0b9860 0x7fd4fc13d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:28.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd500d5e700 1 -- 172.21.15.36:0/2738627093 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f4009470 con 0x7fd4fc0b4f00 2022-06-17T12:54:28.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd501d60700 1 --2- 172.21.15.36:0/2738627093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4fc0b9860 0x7fd4fc13d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:54:28.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd50155f700 1 --2- 172.21.15.36:0/2738627093 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4fc0b45c0 0x7fd4fc0ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:54:28.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd500d5e700 1 --2- 172.21.15.36:0/2738627093 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd4fc0b4f00 0x7fd4fc141f50 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7fd4ec00c000 tx=0x7fd4ec006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:28.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.964+0000 7fd4ea7fc700 1 -- 172.21.15.36:0/2738627093 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4ec010780 con 0x7fd4fc0b4f00 2022-06-17T12:54:28.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.968+0000 7fd4ea7fc700 1 -- 172.21.15.36:0/2738627093 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4ec0108e0 con 0x7fd4fc0b4f00 2022-06-17T12:54:28.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.968+0000 7fd502561700 1 -- 172.21.15.36:0/2738627093 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4fc13ced0 con 0x7fd4fc0b4f00 2022-06-17T12:54:28.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.968+0000 7fd4ea7fc700 1 -- 172.21.15.36:0/2738627093 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4ec010c40 con 0x7fd4fc0b4f00 2022-06-17T12:54:28.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.968+0000 7fd502561700 1 -- 172.21.15.36:0/2738627093 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4fc0ad330 con 0x7fd4fc0b4f00 2022-06-17T12:54:28.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.968+0000 7fd4ea7fc700 1 -- 172.21.15.36:0/2738627093 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd4ec01c460 con 0x7fd4fc0b4f00 2022-06-17T12:54:28.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.968+0000 7fd502561700 1 -- 172.21.15.36:0/2738627093 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4fc0addb0 con 0x7fd4fc0b4f00 2022-06-17T12:54:28.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.968+0000 7fd4ea7fc700 1 --2- 172.21.15.36:0/2738627093 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd4dc053940 0x7fd4dc055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:28.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.968+0000 7fd4ea7fc700 1 -- 172.21.15.36:0/2738627093 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fd4ec06c310 con 0x7fd4fc0b4f00 2022-06-17T12:54:28.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.968+0000 7fd50155f700 1 --2- 172.21.15.36:0/2738627093 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd4dc053940 0x7fd4dc055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:28.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.968+0000 7fd50155f700 1 --2- 172.21.15.36:0/2738627093 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd4dc053940 0x7fd4dc055e00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7fd4f8008000 tx=0x7fd4f8009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:28.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:28.976+0000 7fd4ea7fc700 1 -- 172.21.15.36:0/2738627093 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd4ec069e90 con 0x7fd4fc0b4f00 2022-06-17T12:54:29.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.148+0000 7fd502561700 1 -- 172.21.15.36:0/2738627093 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]} v 0) v1 -- 0x7fd4fc062eb0 con 0x7fd4fc0b4f00 2022-06-17T12:54:29.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.836+0000 7fd4ea7fc700 1 -- 172.21.15.36:0/2738627093 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]}]=0 v345) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd4ec043280 con 0x7fd4fc0b4f00 2022-06-17T12:54:29.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.836+0000 7fd502561700 1 -- 172.21.15.36:0/2738627093 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd4dc053940 msgr2=0x7fd4dc055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:29.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.836+0000 7fd502561700 1 --2- 172.21.15.36:0/2738627093 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd4dc053940 0x7fd4dc055e00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7fd4f8008000 tx=0x7fd4f8009040).stop 2022-06-17T12:54:29.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.836+0000 7fd502561700 1 -- 172.21.15.36:0/2738627093 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd4fc0b4f00 msgr2=0x7fd4fc141f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:29.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.836+0000 7fd502561700 1 --2- 172.21.15.36:0/2738627093 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd4fc0b4f00 0x7fd4fc141f50 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7fd4ec00c000 tx=0x7fd4ec006df0).stop 2022-06-17T12:54:29.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.836+0000 7fd502561700 1 -- 172.21.15.36:0/2738627093 shutdown_connections 2022-06-17T12:54:29.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.836+0000 7fd502561700 1 --2- 172.21.15.36:0/2738627093 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd4dc053940 0x7fd4dc055e00 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:29.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.836+0000 7fd502561700 1 --2- 172.21.15.36:0/2738627093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4fc0b9860 0x7fd4fc13d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:29.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.836+0000 7fd502561700 1 --2- 172.21.15.36:0/2738627093 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd4fc0b4f00 0x7fd4fc141f50 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:29.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.840+0000 7fd502561700 1 --2- 172.21.15.36:0/2738627093 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4fc0b45c0 0x7fd4fc0ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:29.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.840+0000 7fd502561700 1 -- 172.21.15.36:0/2738627093 >> 172.21.15.36:0/2738627093 conn(0x7fd4fc05c550 msgr2=0x7fd4fc0b8840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:29.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.840+0000 7fd502561700 1 -- 172.21.15.36:0/2738627093 shutdown_connections 2022-06-17T12:54:29.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.840+0000 7fd502561700 1 -- 172.21.15.36:0/2738627093 wait complete. 2022-06-17T12:54:29.841 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-06-17T12:54:29.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.972+0000 7f1687147700 1 Processor -- start 2022-06-17T12:54:29.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.972+0000 7f1687147700 1 -- start start 2022-06-17T12:54:29.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.972+0000 7f1687147700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f16800ae360 0x7f1680144ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:29.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.972+0000 7f1687147700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f1680144fd0 0x7f168014aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:29.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.972+0000 7f1687147700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16800b2b60 con 0x7f16800601c0 2022-06-17T12:54:29.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.972+0000 7f1687147700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16800b3230 con 0x7f1680144fd0 2022-06-17T12:54:29.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.972+0000 7f1687147700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16800605a0 con 0x7f16800ae360 2022-06-17T12:54:29.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.972+0000 7f1686946700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f1680144fd0 0x7f168014aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:29.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.972+0000 7f1686145700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f16800601c0 0x7f16801443d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54332/0 (socket says 172.21.15.36:54332) 2022-06-17T12:54:29.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.972+0000 7f1686946700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f1680144fd0 0x7f168014aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36806/0 (socket says 172.21.15.36:36806) 2022-06-17T12:54:29.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.972+0000 7f1686145700 1 -- 172.21.15.36:0/923508344 learned_addr learned my addr 172.21.15.36:0/923508344 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:29.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.972+0000 7f1685944700 1 --2- 172.21.15.36:0/923508344 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f16800ae360 0x7f1680144ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:29.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.976+0000 7f1686946700 1 -- 172.21.15.36:0/923508344 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f16800ae360 msgr2=0x7f1680144ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:29.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.976+0000 7f1686946700 1 --2- 172.21.15.36:0/923508344 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f16800ae360 0x7f1680144ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:29.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.976+0000 7f1686946700 1 -- 172.21.15.36:0/923508344 >> v1:172.21.15.36:6789/0 conn(0x7f16800601c0 legacy=0x7f16801443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:29.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.976+0000 7f1686946700 1 -- 172.21.15.36:0/923508344 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16800662f0 con 0x7f1680144fd0 2022-06-17T12:54:29.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.976+0000 7f1686946700 1 --2- 172.21.15.36:0/923508344 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1680144fd0 0x7f168014aa00 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7f1678002f70 tx=0x7f1678006df0).ready entity=mon.0 client_cookie=8d1974ee503434b8 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:29.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.976+0000 7f1685143700 1 -- 172.21.15.36:0/923508344 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f167800d570 con 0x7f1680144fd0 2022-06-17T12:54:29.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.976+0000 7f1685143700 1 -- 172.21.15.36:0/923508344 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1678000e90 con 0x7f1680144fd0 2022-06-17T12:54:29.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.976+0000 7f1685143700 1 -- 172.21.15.36:0/923508344 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f167800d8c0 con 0x7f1680144fd0 2022-06-17T12:54:29.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.976+0000 7f1687147700 1 -- 172.21.15.36:0/923508344 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1680144fd0 msgr2=0x7f168014aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:29.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.976+0000 7f1687147700 1 --2- 172.21.15.36:0/923508344 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1680144fd0 0x7f168014aa00 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7f1678002f70 tx=0x7f1678006df0).stop 2022-06-17T12:54:29.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.976+0000 7f1687147700 1 -- 172.21.15.36:0/923508344 shutdown_connections 2022-06-17T12:54:29.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.976+0000 7f1687147700 1 --2- 172.21.15.36:0/923508344 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1680144fd0 0x7f168014aa00 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:29.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.976+0000 7f1687147700 1 --2- 172.21.15.36:0/923508344 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f16800ae360 0x7f1680144ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:29.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.976+0000 7f1687147700 1 -- 172.21.15.36:0/923508344 >> 172.21.15.36:0/923508344 conn(0x7f1680060990 msgr2=0x7f168005f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:29.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.976+0000 7f1687147700 1 -- 172.21.15.36:0/923508344 shutdown_connections 2022-06-17T12:54:29.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.976+0000 7f1687147700 1 -- 172.21.15.36:0/923508344 wait complete. 2022-06-17T12:54:29.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.976+0000 7f1687147700 1 Processor -- start 2022-06-17T12:54:29.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.976+0000 7f1687147700 1 -- start start 2022-06-17T12:54:29.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f1687147700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f16800601c0 0x7f1680110930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:29.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f1687147700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f16800ae360 0x7f1680114720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:29.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f1687147700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1680144fd0 0x7f1680114c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:29.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f1687147700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168013ea10 con 0x7f16800601c0 2022-06-17T12:54:29.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f1687147700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1680150690 con 0x7f1680144fd0 2022-06-17T12:54:29.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f1687147700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1680150930 con 0x7f16800ae360 2022-06-17T12:54:29.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f1686145700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f16800601c0 0x7f1680110930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:29.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f1686145700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f16800601c0 0x7f1680110930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36808/0 (socket says 172.21.15.36:36808) 2022-06-17T12:54:29.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f1686145700 1 -- 172.21.15.36:0/133823159 learned_addr learned my addr 172.21.15.36:0/133823159 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:29.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f1686946700 1 --2- 172.21.15.36:0/133823159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1680144fd0 0x7f1680114c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:29.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f1685944700 1 --2- 172.21.15.36:0/133823159 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f16800ae360 0x7f1680114720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:29.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f1686145700 1 -- 172.21.15.36:0/133823159 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f16800ae360 msgr2=0x7f1680114720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:29.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f1686145700 1 --2- 172.21.15.36:0/133823159 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f16800ae360 0x7f1680114720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:29.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f1686145700 1 -- 172.21.15.36:0/133823159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1680144fd0 msgr2=0x7f1680114c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:29.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f1686145700 1 --2- 172.21.15.36:0/133823159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1680144fd0 0x7f1680114c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:29.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f1686145700 1 -- 172.21.15.36:0/133823159 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f167800b180 con 0x7f16800601c0 2022-06-17T12:54:29.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f1685944700 1 --2- 172.21.15.36:0/133823159 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f16800ae360 0x7f1680114720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:29.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f1686145700 1 --2- 172.21.15.36:0/133823159 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f16800601c0 0x7f1680110930 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7f1680061080 tx=0x7f1670007ca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:29.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f16777fe700 1 -- 172.21.15.36:0/133823159 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16700070d0 con 0x7f16800601c0 2022-06-17T12:54:29.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f16777fe700 1 -- 172.21.15.36:0/133823159 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1670007230 con 0x7f16800601c0 2022-06-17T12:54:29.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f1687147700 1 -- 172.21.15.36:0/133823159 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1680150bd0 con 0x7f16800601c0 2022-06-17T12:54:29.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f16777fe700 1 -- 172.21.15.36:0/133823159 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1670005610 con 0x7f16800601c0 2022-06-17T12:54:29.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f1687147700 1 -- 172.21.15.36:0/133823159 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1680111160 con 0x7f16800601c0 2022-06-17T12:54:29.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f1687147700 1 -- 172.21.15.36:0/133823159 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1680066590 con 0x7f16800601c0 2022-06-17T12:54:29.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.980+0000 7f16777fe700 1 -- 172.21.15.36:0/133823159 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f1670010070 con 0x7f16800601c0 2022-06-17T12:54:29.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.984+0000 7f16777fe700 1 --2- 172.21.15.36:0/133823159 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1660053a40 0x7f1660055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:29.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.984+0000 7f16777fe700 1 -- 172.21.15.36:0/133823159 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f167006b390 con 0x7f16800601c0 2022-06-17T12:54:29.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.984+0000 7f1685944700 1 --2- 172.21.15.36:0/133823159 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1660053a40 0x7f1660055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:29.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.984+0000 7f1685944700 1 --2- 172.21.15.36:0/133823159 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1660053a40 0x7f1660055f00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f167c00b000 tx=0x7f167c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:29.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:29.988+0000 7f16777fe700 1 -- 172.21.15.36:0/133823159 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f16700423d0 con 0x7f16800601c0 2022-06-17T12:54:30.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.120+0000 7f1687147700 1 -- 172.21.15.36:0/133823159 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1680057410 con 0x7f16800601c0 2022-06-17T12:54:30.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.124+0000 7f16777fe700 1 -- 172.21.15.36:0/133823159 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1670018020 con 0x7f16800601c0 2022-06-17T12:54:30.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.128+0000 7f1687147700 1 -- 172.21.15.36:0/133823159 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1660053a40 msgr2=0x7f1660055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:30.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.128+0000 7f1687147700 1 --2- 172.21.15.36:0/133823159 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1660053a40 0x7f1660055f00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f167c00b000 tx=0x7f167c008040).stop 2022-06-17T12:54:30.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.128+0000 7f1687147700 1 -- 172.21.15.36:0/133823159 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f16800601c0 msgr2=0x7f1680110930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:30.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.128+0000 7f1687147700 1 --2- 172.21.15.36:0/133823159 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f16800601c0 0x7f1680110930 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7f1680061080 tx=0x7f1670007ca0).stop 2022-06-17T12:54:30.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.128+0000 7f1687147700 1 -- 172.21.15.36:0/133823159 shutdown_connections 2022-06-17T12:54:30.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.128+0000 7f1687147700 1 --2- 172.21.15.36:0/133823159 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1660053a40 0x7f1660055f00 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:30.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.128+0000 7f1687147700 1 --2- 172.21.15.36:0/133823159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1680144fd0 0x7f1680114c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:30.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.128+0000 7f1687147700 1 --2- 172.21.15.36:0/133823159 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f16800ae360 0x7f1680114720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:30.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.128+0000 7f1687147700 1 --2- 172.21.15.36:0/133823159 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f16800601c0 0x7f1680110930 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:30.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.128+0000 7f1687147700 1 -- 172.21.15.36:0/133823159 >> 172.21.15.36:0/133823159 conn(0x7f1680060990 msgr2=0x7f16800b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:30.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.128+0000 7f1687147700 1 -- 172.21.15.36:0/133823159 shutdown_connections 2022-06-17T12:54:30.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.128+0000 7f1687147700 1 -- 172.21.15.36:0/133823159 wait complete. 2022-06-17T12:54:30.129 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-17T12:54:30.130 DEBUG:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:30.151 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:30.174 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:30.205 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:30.229 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:30.252 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-17T12:54:30.252 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:30.253 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:30.253 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:54:30.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:54:30.263 INFO:teuthology.orchestra.run.smithi107.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-17T12:54:30.264 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-06-17T12:54:30.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:30.265 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:30.265 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:54:30.272 INFO:teuthology.orchestra.run.smithi136.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-17T12:54:30.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:54:30.274 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-17T12:54:30.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:30.276 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:30.276 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:54:30.299 INFO:teuthology.orchestra.run.smithi143.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:54:30.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:54:30.301 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-06-17T12:54:30.301 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-17T12:54:30.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.424+0000 7f5371893700 1 Processor -- start 2022-06-17T12:54:30.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.424+0000 7f5371893700 1 -- start start 2022-06-17T12:54:30.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.428+0000 7f5371893700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f536c0b0d70 0x7f536c0ac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:30.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.428+0000 7f5371893700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f536c0b15f0 0x7f536c0acc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:30.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.428+0000 7f5371893700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f536c0b2b90 con 0x7f536c0b27b0 2022-06-17T12:54:30.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.428+0000 7f5371893700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f536c0610e0 con 0x7f536c0b0d70 2022-06-17T12:54:30.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.428+0000 7f5371893700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536c0ac190 con 0x7f536c0b15f0 2022-06-17T12:54:30.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.428+0000 7f5370891700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f536c0b27b0 0x7f536c13f110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54346/0 (socket says 172.21.15.36:54346) 2022-06-17T12:54:30.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.428+0000 7f5370891700 1 -- 172.21.15.36:0/367896779 learned_addr learned my addr 172.21.15.36:0/367896779 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:30.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.428+0000 7f536bfff700 1 --2- 172.21.15.36:0/367896779 >> v2:172.21.15.70:3300/0 conn(0x7f536c0b0d70 0x7f536c0ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:30.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.428+0000 7f5371092700 1 --2- 172.21.15.36:0/367896779 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f536c0b15f0 0x7f536c0acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:30.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.428+0000 7f536b7fe700 1 -- 172.21.15.36:0/367896779 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3166294137 0 0) 0x7f536c0b2b90 con 0x7f536c0b27b0 2022-06-17T12:54:30.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.428+0000 7f536b7fe700 1 -- 172.21.15.36:0/367896779 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5358003260 con 0x7f536c0b27b0 2022-06-17T12:54:30.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.428+0000 7f5371092700 1 -- 172.21.15.36:0/367896779 >> v2:172.21.15.70:3300/0 conn(0x7f536c0b0d70 msgr2=0x7f536c0ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:30.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.428+0000 7f5371092700 1 --2- 172.21.15.36:0/367896779 >> v2:172.21.15.70:3300/0 conn(0x7f536c0b0d70 0x7f536c0ac780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:30.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.428+0000 7f5371092700 1 -- 172.21.15.36:0/367896779 >> v1:172.21.15.36:6789/0 conn(0x7f536c0b27b0 legacy=0x7f536c13f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:30.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.428+0000 7f5371092700 1 -- 172.21.15.36:0/367896779 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c0ac430 con 0x7f536c0b15f0 2022-06-17T12:54:30.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.428+0000 7f536bfff700 1 --2- 172.21.15.36:0/367896779 >> v2:172.21.15.70:3300/0 conn(0x7f536c0b0d70 0x7f536c0ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:30.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.428+0000 7f5371092700 1 --2- 172.21.15.36:0/367896779 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f536c0b15f0 0x7f536c0acc70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f53640032f0 tx=0x7f53640091d0).ready entity=mon.2 client_cookie=b9b924e9ed6b3556 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:30.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.428+0000 7f536b7fe700 1 -- 172.21.15.36:0/367896779 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f536400b600 con 0x7f536c0b15f0 2022-06-17T12:54:30.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.428+0000 7f536b7fe700 1 -- 172.21.15.36:0/367896779 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536400b760 con 0x7f536c0b15f0 2022-06-17T12:54:30.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.428+0000 7f536b7fe700 1 -- 172.21.15.36:0/367896779 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f536400b8d0 con 0x7f536c0b15f0 2022-06-17T12:54:30.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.432+0000 7f5371893700 1 -- 172.21.15.36:0/367896779 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f536c0b15f0 msgr2=0x7f536c0acc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:30.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.432+0000 7f5371893700 1 --2- 172.21.15.36:0/367896779 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f536c0b15f0 0x7f536c0acc70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f53640032f0 tx=0x7f53640091d0).stop 2022-06-17T12:54:30.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.432+0000 7f5371893700 1 -- 172.21.15.36:0/367896779 shutdown_connections 2022-06-17T12:54:30.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.432+0000 7f5371893700 1 --2- 172.21.15.36:0/367896779 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f536c0b15f0 0x7f536c0acc70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:30.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.432+0000 7f5371893700 1 --2- 172.21.15.36:0/367896779 >> v2:172.21.15.70:3300/0 conn(0x7f536c0b0d70 0x7f536c0ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:30.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.432+0000 7f5371893700 1 -- 172.21.15.36:0/367896779 >> 172.21.15.36:0/367896779 conn(0x7f536c0b5d10 msgr2=0x7f536c0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:30.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.432+0000 7f5371893700 1 -- 172.21.15.36:0/367896779 shutdown_connections 2022-06-17T12:54:30.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.432+0000 7f5371893700 1 -- 172.21.15.36:0/367896779 wait complete. 2022-06-17T12:54:30.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.432+0000 7f5371893700 1 Processor -- start 2022-06-17T12:54:30.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.432+0000 7f5371893700 1 -- start start 2022-06-17T12:54:30.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.432+0000 7f5371893700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f536c0b0d70 0x7f536c13c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:30.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.432+0000 7f5371893700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f536c0b15f0 0x7f536c142e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:30.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.432+0000 7f5371893700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f536c0b27b0 0x7f536c1525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:30.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.432+0000 7f5371893700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c144210 con 0x7f536c0b15f0 2022-06-17T12:54:30.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.432+0000 7f5371893700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c1427c0 con 0x7f536c0b27b0 2022-06-17T12:54:30.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.432+0000 7f5371893700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536c153180 con 0x7f536c0b0d70 2022-06-17T12:54:30.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.436+0000 7f536bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f536c0b15f0 0x7f536c142e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:30.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.436+0000 7f5371092700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f536c0b27b0 0x7f536c1525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:30.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.436+0000 7f536bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f536c0b15f0 0x7f536c142e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36826/0 (socket says 172.21.15.36:36826) 2022-06-17T12:54:30.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.436+0000 7f536bfff700 1 -- 172.21.15.36:0/1400879664 learned_addr learned my addr 172.21.15.36:0/1400879664 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:30.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.436+0000 7f5370891700 1 --2- 172.21.15.36:0/1400879664 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f536c0b0d70 0x7f536c13c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:30.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.436+0000 7f536bfff700 1 -- 172.21.15.36:0/1400879664 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f536c0b0d70 msgr2=0x7f536c13c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:30.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.436+0000 7f536bfff700 1 --2- 172.21.15.36:0/1400879664 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f536c0b0d70 0x7f536c13c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:30.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.436+0000 7f536bfff700 1 -- 172.21.15.36:0/1400879664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f536c0b27b0 msgr2=0x7f536c1525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:30.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.436+0000 7f536bfff700 1 --2- 172.21.15.36:0/1400879664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f536c0b27b0 0x7f536c1525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:30.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.436+0000 7f536bfff700 1 -- 172.21.15.36:0/1400879664 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c00b180 con 0x7f536c0b15f0 2022-06-17T12:54:30.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.436+0000 7f536bfff700 1 --2- 172.21.15.36:0/1400879664 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f536c0b15f0 0x7f536c142e90 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7f535c00c000 tx=0x7f535c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:30.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.436+0000 7f5369ffb700 1 -- 172.21.15.36:0/1400879664 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f535c00c7e0 con 0x7f536c0b15f0 2022-06-17T12:54:30.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.436+0000 7f5369ffb700 1 -- 172.21.15.36:0/1400879664 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535c00c940 con 0x7f536c0b15f0 2022-06-17T12:54:30.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.436+0000 7f5369ffb700 1 -- 172.21.15.36:0/1400879664 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f535c00cca0 con 0x7f536c0b15f0 2022-06-17T12:54:30.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.436+0000 7f5371893700 1 -- 172.21.15.36:0/1400879664 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5364009470 con 0x7f536c0b15f0 2022-06-17T12:54:30.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.436+0000 7f5371893700 1 -- 172.21.15.36:0/1400879664 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f536c13cc40 con 0x7f536c0b15f0 2022-06-17T12:54:30.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.436+0000 7f5371893700 1 -- 172.21.15.36:0/1400879664 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f536c1436c0 con 0x7f536c0b15f0 2022-06-17T12:54:30.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.440+0000 7f5369ffb700 1 -- 172.21.15.36:0/1400879664 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f535c01c460 con 0x7f536c0b15f0 2022-06-17T12:54:30.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.440+0000 7f5369ffb700 1 --2- 172.21.15.36:0/1400879664 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f534c0539b0 0x7f534c055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:30.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.440+0000 7f5369ffb700 1 -- 172.21.15.36:0/1400879664 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f535c06c270 con 0x7f536c0b15f0 2022-06-17T12:54:30.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.440+0000 7f5370891700 1 --2- 172.21.15.36:0/1400879664 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f534c0539b0 0x7f534c055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:30.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.440+0000 7f5370891700 1 --2- 172.21.15.36:0/1400879664 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f534c0539b0 0x7f534c055e70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f536000d000 tx=0x7f5360009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:30.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.444+0000 7f5369ffb700 1 -- 172.21.15.36:0/1400879664 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f536c1436c0 con 0x7f536c0b15f0 2022-06-17T12:54:30.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.568+0000 7f5371893700 1 -- 172.21.15.36:0/1400879664 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f536c1436c0 con 0x7f536c0b15f0 2022-06-17T12:54:30.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.572+0000 7f5369ffb700 1 -- 172.21.15.36:0/1400879664 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v87) v1 ==== 81+0+29220 (secure 0 0 0) 0x7f536c1436c0 con 0x7f536c0b15f0 2022-06-17T12:54:30.561 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:30.561 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:54:30.561 INFO:teuthology.orchestra.run.smithi036.stdout: "epoch": 87, 2022-06-17T12:54:30.561 INFO:teuthology.orchestra.run.smithi036.stdout: "fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", 2022-06-17T12:54:30.562 INFO:teuthology.orchestra.run.smithi036.stdout: "created": "2022-06-17T12:46:30.671408+0000", 2022-06-17T12:54:30.562 INFO:teuthology.orchestra.run.smithi036.stdout: "modified": "2022-06-17T12:54:27.812957+0000", 2022-06-17T12:54:30.562 INFO:teuthology.orchestra.run.smithi036.stdout: "last_up_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:54:30.562 INFO:teuthology.orchestra.run.smithi036.stdout: "last_in_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:54:30.563 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-17T12:54:30.563 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_num": 5799936, 2022-06-17T12:54:30.563 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_set": [ 2022-06-17T12:54:30.567 INFO:teuthology.orchestra.run.smithi036.stdout: "pglog_hardlimit", 2022-06-17T12:54:30.567 INFO:teuthology.orchestra.run.smithi036.stdout: "purged_snapdirs", 2022-06-17T12:54:30.567 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_deletes", 2022-06-17T12:54:30.567 INFO:teuthology.orchestra.run.smithi036.stdout: "sortbitwise" 2022-06-17T12:54:30.568 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:54:30.568 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_version": 6, 2022-06-17T12:54:30.568 INFO:teuthology.orchestra.run.smithi036.stdout: "full_ratio": 0.94999998807907104, 2022-06-17T12:54:30.568 INFO:teuthology.orchestra.run.smithi036.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-17T12:54:30.569 INFO:teuthology.orchestra.run.smithi036.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-17T12:54:30.569 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_snapshot": "", 2022-06-17T12:54:30.569 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_max": 16, 2022-06-17T12:54:30.569 INFO:teuthology.orchestra.run.smithi036.stdout: "max_osd": 8, 2022-06-17T12:54:30.570 INFO:teuthology.orchestra.run.smithi036.stdout: "require_min_compat_client": "jewel", 2022-06-17T12:54:30.570 INFO:teuthology.orchestra.run.smithi036.stdout: "min_compat_client": "jewel", 2022-06-17T12:54:30.570 INFO:teuthology.orchestra.run.smithi036.stdout: "require_osd_release": "octopus", 2022-06-17T12:54:30.571 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:54:30.571 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.571 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 1, 2022-06-17T12:54:30.571 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "device_health_metrics", 2022-06-17T12:54:30.572 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:33.881212+0000", 2022-06-17T12:54:30.572 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:54:30.572 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:54:30.572 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:54:30.572 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:54:30.573 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:54:30.573 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:54:30.573 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:54:30.573 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:54:30.574 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 1, 2022-06-17T12:54:30.574 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 1, 2022-06-17T12:54:30.574 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 1, 2022-06-17T12:54:30.574 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 1, 2022-06-17T12:54:30.575 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 1, 2022-06-17T12:54:30.575 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:54:30.575 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:54:30.575 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:54:30.576 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:54:30.576 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:54:30.576 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:54:30.576 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:54:30.576 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.577 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "16", 2022-06-17T12:54:30.577 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:54:30.577 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:54:30.577 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:54:30.578 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:54:30.578 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:54:30.578 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:54:30.578 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:54:30.578 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:54:30.579 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:54:30.579 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:54:30.579 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:54:30.579 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:54:30.580 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:54:30.580 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:54:30.580 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:54:30.580 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:54:30.581 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:54:30.581 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:54:30.581 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:54:30.581 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:54:30.581 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:54:30.582 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:54:30.582 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:54:30.582 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:54:30.582 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:54:30.583 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:54:30.583 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.583 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:54:30.584 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:54:30.584 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:54:30.584 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:54:30.584 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:54:30.585 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:54:30.585 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:54:30.585 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:54:30.585 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:54:30.585 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:54:30.586 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:54:30.586 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:54:30.586 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 1 2022-06-17T12:54:30.586 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.587 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:54:30.587 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr_devicehealth": {} 2022-06-17T12:54:30.587 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.587 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.587 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.588 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 2, 2022-06-17T12:54:30.588 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "rbd", 2022-06-17T12:54:30.588 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:44.485460+0000", 2022-06-17T12:54:30.588 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:54:30.589 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:54:30.589 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:54:30.589 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:54:30.589 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:54:30.589 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:54:30.590 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:54:30.590 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:54:30.590 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:54:30.590 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:54:30.591 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:54:30.591 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:54:30.591 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:54:30.591 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:54:30.592 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:54:30.592 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:54:30.592 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:54:30.592 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:54:30.593 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:54:30.593 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:54:30.593 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.594 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "19", 2022-06-17T12:54:30.594 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:54:30.594 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:54:30.594 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:54:30.595 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:54:30.595 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:54:30.595 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:54:30.598 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:54:30.599 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:54:30.599 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:54:30.599 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:54:30.600 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:54:30.600 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:54:30.601 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:54:30.601 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:54:30.602 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:54:30.602 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:54:30.603 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:54:30.603 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:54:30.603 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:54:30.604 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:54:30.604 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:54:30.605 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:54:30.606 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:54:30.606 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:54:30.607 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:54:30.607 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:54:30.607 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.608 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:54:30.609 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:54:30.609 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:54:30.609 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:54:30.609 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:54:30.610 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:54:30.610 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:54:30.610 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:54:30.610 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:54:30.611 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:54:30.611 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:54:30.611 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:54:30.611 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:54:30.612 INFO:teuthology.orchestra.run.smithi036.stdout: "rbd": {} 2022-06-17T12:54:30.612 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.612 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.612 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:54:30.613 INFO:teuthology.orchestra.run.smithi036.stdout: "osds": [ 2022-06-17T12:54:30.613 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.613 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:54:30.613 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", 2022-06-17T12:54:30.613 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:30.614 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:30.614 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:30.614 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:30.614 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:30.615 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:30.615 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:54:30.615 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 82, 2022-06-17T12:54:30.615 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:30.615 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:30.616 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:30.616 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.616 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.616 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.617 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6802", 2022-06-17T12:54:30.617 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:30.617 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.617 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.617 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.618 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6803", 2022-06-17T12:54:30.618 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:30.618 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.618 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.619 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.619 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:30.619 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.619 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.620 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.620 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6804", 2022-06-17T12:54:30.620 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:30.621 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.621 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.621 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.622 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6805", 2022-06-17T12:54:30.622 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:30.622 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.623 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.623 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.624 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:30.624 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.624 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.625 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.625 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6808", 2022-06-17T12:54:30.625 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:30.626 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.626 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.627 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.627 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6809", 2022-06-17T12:54:30.627 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:30.628 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.628 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.628 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.629 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:30.629 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.629 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.630 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.630 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6806", 2022-06-17T12:54:30.630 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:30.630 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.631 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.631 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.631 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6807", 2022-06-17T12:54:30.631 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:30.631 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.632 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.632 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.632 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6803/22619", 2022-06-17T12:54:30.632 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6805/22619", 2022-06-17T12:54:30.633 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6809/22619", 2022-06-17T12:54:30.633 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6807/22619", 2022-06-17T12:54:30.633 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:30.633 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:30.634 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:30.634 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.634 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.634 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.635 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:54:30.635 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", 2022-06-17T12:54:30.635 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:30.635 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:30.635 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:30.636 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:30.636 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:30.636 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:30.636 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:54:30.637 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 81, 2022-06-17T12:54:30.637 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:30.637 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:30.637 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:30.637 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.638 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.638 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.638 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6810", 2022-06-17T12:54:30.638 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:30.639 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.639 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.639 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.639 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6811", 2022-06-17T12:54:30.640 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:30.640 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.640 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.640 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.641 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:30.641 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.641 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.642 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.642 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6812", 2022-06-17T12:54:30.642 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:30.642 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.643 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.643 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.643 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6813", 2022-06-17T12:54:30.643 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:30.644 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.644 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.644 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.644 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:30.645 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.645 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.645 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.645 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6816", 2022-06-17T12:54:30.646 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:30.646 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.646 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.646 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.647 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6817", 2022-06-17T12:54:30.647 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:30.647 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.647 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.647 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.648 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:30.648 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.648 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.648 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.648 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6814", 2022-06-17T12:54:30.649 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:30.649 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.649 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.649 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.650 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6815", 2022-06-17T12:54:30.650 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:30.650 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.650 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.650 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.651 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6811/22620", 2022-06-17T12:54:30.651 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6813/22620", 2022-06-17T12:54:30.651 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6817/22620", 2022-06-17T12:54:30.651 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6815/22620", 2022-06-17T12:54:30.652 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:30.652 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:30.652 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:30.652 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.653 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.653 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.653 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:54:30.653 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", 2022-06-17T12:54:30.653 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:30.654 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:30.654 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:30.654 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:30.654 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:30.655 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:30.655 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:54:30.655 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 82, 2022-06-17T12:54:30.655 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:30.656 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:30.656 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:30.656 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.656 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.656 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.657 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6826", 2022-06-17T12:54:30.657 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:30.657 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.657 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.657 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.658 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6827", 2022-06-17T12:54:30.658 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:30.658 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.658 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.659 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.659 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:30.659 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.659 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.659 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.660 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6828", 2022-06-17T12:54:30.660 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:30.660 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.660 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.661 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.661 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6829", 2022-06-17T12:54:30.661 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:30.661 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.662 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.662 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.662 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:30.662 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.663 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.663 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.663 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6832", 2022-06-17T12:54:30.663 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:30.663 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.664 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.664 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.664 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6833", 2022-06-17T12:54:30.664 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:30.665 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.665 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.665 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.665 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:30.665 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.666 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.666 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.666 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6830", 2022-06-17T12:54:30.666 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:30.667 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.667 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.667 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.667 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6831", 2022-06-17T12:54:30.667 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:30.668 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.668 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.668 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.668 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6827/22643", 2022-06-17T12:54:30.669 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6829/22643", 2022-06-17T12:54:30.669 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6833/22643", 2022-06-17T12:54:30.669 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6831/22643", 2022-06-17T12:54:30.670 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:30.670 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:30.670 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:30.670 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.671 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.671 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.671 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:54:30.671 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", 2022-06-17T12:54:30.672 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:30.672 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:30.672 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:30.672 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:30.672 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:30.673 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:30.673 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:54:30.673 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 82, 2022-06-17T12:54:30.673 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:30.674 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:30.674 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:30.674 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.675 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.675 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.675 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6818", 2022-06-17T12:54:30.676 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:30.676 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.676 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.676 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.677 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6819", 2022-06-17T12:54:30.677 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:30.677 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.677 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.677 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.678 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:30.678 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.678 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.678 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.679 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6820", 2022-06-17T12:54:30.679 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:30.679 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.679 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.679 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.680 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6821", 2022-06-17T12:54:30.680 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:30.680 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.680 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.681 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.681 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:30.682 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.682 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.682 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.683 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6824", 2022-06-17T12:54:30.683 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:30.684 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.684 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.685 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.685 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6825", 2022-06-17T12:54:30.685 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:30.686 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.686 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.687 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.687 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:30.688 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.688 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.688 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.689 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6822", 2022-06-17T12:54:30.689 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:30.689 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.690 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.690 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.691 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6823", 2022-06-17T12:54:30.691 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:30.691 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.691 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.692 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.692 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6819/22646", 2022-06-17T12:54:30.692 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6821/22646", 2022-06-17T12:54:30.692 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6825/22646", 2022-06-17T12:54:30.692 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6823/22646", 2022-06-17T12:54:30.693 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:30.693 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:30.693 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:30.693 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.694 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.694 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.694 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:54:30.695 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", 2022-06-17T12:54:30.695 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:30.695 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:30.695 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:30.696 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:30.696 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:30.696 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:30.696 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:54:30.696 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 82, 2022-06-17T12:54:30.697 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:30.697 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:30.697 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:30.698 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.698 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.698 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.698 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6808", 2022-06-17T12:54:30.699 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:30.699 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.699 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.699 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.699 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6809", 2022-06-17T12:54:30.700 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:30.700 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.700 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.700 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.701 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:30.701 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.701 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.701 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.702 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6810", 2022-06-17T12:54:30.702 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:30.702 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.702 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.703 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.703 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6811", 2022-06-17T12:54:30.703 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:30.703 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.703 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.704 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.704 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:30.704 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.704 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.705 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.705 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6814", 2022-06-17T12:54:30.705 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:30.705 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.706 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.706 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.706 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6815", 2022-06-17T12:54:30.706 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:30.707 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.707 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.707 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.707 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:30.708 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.708 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.708 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.708 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6812", 2022-06-17T12:54:30.708 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:30.709 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.709 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.709 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.709 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6813", 2022-06-17T12:54:30.709 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:30.710 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.710 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.710 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.711 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6809/22812", 2022-06-17T12:54:30.711 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6811/22812", 2022-06-17T12:54:30.711 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6815/22812", 2022-06-17T12:54:30.711 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6813/22812", 2022-06-17T12:54:30.711 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:30.712 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:30.712 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:30.712 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.712 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.713 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.713 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:54:30.713 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", 2022-06-17T12:54:30.713 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:30.713 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:30.714 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:30.714 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:30.714 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:30.714 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:30.715 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:54:30.715 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 82, 2022-06-17T12:54:30.715 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:30.715 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:30.716 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:30.716 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.716 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.716 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.717 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6800", 2022-06-17T12:54:30.717 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:30.717 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.717 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.718 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.718 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6801", 2022-06-17T12:54:30.718 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:30.719 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.719 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.719 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.719 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:30.720 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.720 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.720 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.720 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6802", 2022-06-17T12:54:30.720 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:30.721 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.721 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.721 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.721 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6803", 2022-06-17T12:54:30.722 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:30.722 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.722 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.722 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.723 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:30.723 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.723 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.723 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.724 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6806", 2022-06-17T12:54:30.724 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:30.725 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.725 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.725 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.726 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6807", 2022-06-17T12:54:30.726 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:30.727 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.727 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.728 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.728 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:30.728 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.729 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.729 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.730 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6804", 2022-06-17T12:54:30.730 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:30.730 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.731 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.731 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.732 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6805", 2022-06-17T12:54:30.732 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:30.733 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.733 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.734 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.734 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6801/22813", 2022-06-17T12:54:30.734 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6803/22813", 2022-06-17T12:54:30.735 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6807/22813", 2022-06-17T12:54:30.735 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6805/22813", 2022-06-17T12:54:30.736 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:30.736 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:30.737 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:30.737 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.737 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.738 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.738 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:54:30.739 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", 2022-06-17T12:54:30.739 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:30.740 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:30.740 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:30.740 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:30.741 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:30.741 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:30.742 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:54:30.742 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 82, 2022-06-17T12:54:30.743 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:30.743 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:30.744 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:30.744 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.744 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.745 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.745 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6816", 2022-06-17T12:54:30.746 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:30.746 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.746 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.747 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.747 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6817", 2022-06-17T12:54:30.747 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:30.748 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.748 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.748 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.749 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:30.749 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.749 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.749 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.750 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6818", 2022-06-17T12:54:30.750 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:30.750 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.751 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.751 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.751 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6819", 2022-06-17T12:54:30.752 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:30.752 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.752 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.752 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.753 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:30.753 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.753 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.753 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.754 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6822", 2022-06-17T12:54:30.754 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:30.754 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.754 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.755 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.755 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6823", 2022-06-17T12:54:30.755 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:30.755 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.755 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.756 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.756 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:30.756 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.756 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.757 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.757 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6820", 2022-06-17T12:54:30.757 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:30.757 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.758 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.758 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.758 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6821", 2022-06-17T12:54:30.758 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:30.759 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.759 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.759 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.759 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6817/22822", 2022-06-17T12:54:30.760 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6819/22822", 2022-06-17T12:54:30.760 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6823/22822", 2022-06-17T12:54:30.760 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6821/22822", 2022-06-17T12:54:30.760 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:30.760 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:30.761 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:30.761 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.761 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.762 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.762 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:54:30.762 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", 2022-06-17T12:54:30.762 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:30.763 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:30.763 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:30.763 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:30.763 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:30.764 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:30.764 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:54:30.764 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 71, 2022-06-17T12:54:30.764 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:30.764 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:30.765 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:30.765 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.765 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.765 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.766 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6824", 2022-06-17T12:54:30.766 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:30.766 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.766 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.767 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.767 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6825", 2022-06-17T12:54:30.767 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:30.767 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.768 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.768 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.768 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:30.768 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.769 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.769 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.769 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6826", 2022-06-17T12:54:30.769 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:30.770 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.770 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.770 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.770 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6827", 2022-06-17T12:54:30.770 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:30.771 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.771 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.771 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.771 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:30.772 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.772 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.772 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.772 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6830", 2022-06-17T12:54:30.773 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:30.773 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.773 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.773 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.774 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6831", 2022-06-17T12:54:30.774 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:30.774 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.774 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.774 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.775 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:30.775 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:30.775 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.775 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:30.776 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6828", 2022-06-17T12:54:30.776 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:30.776 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.776 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.777 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:30.777 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6829", 2022-06-17T12:54:30.777 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:30.777 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.777 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.778 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.778 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6825/22837", 2022-06-17T12:54:30.778 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6827/22837", 2022-06-17T12:54:30.778 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6831/22837", 2022-06-17T12:54:30.779 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6829/22837", 2022-06-17T12:54:30.779 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:30.779 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:30.779 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:30.780 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:30.780 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.780 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:54:30.780 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_xinfo": [ 2022-06-17T12:54:30.780 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.781 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:54:30.781 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:30.781 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:30.781 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:30.782 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:30.782 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:30.782 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.291461+0000", 2022-06-17T12:54:30.782 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:30.782 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.783 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.783 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:54:30.783 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:30.783 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:30.784 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:30.784 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:30.784 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:30.784 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.537912+0000", 2022-06-17T12:54:30.785 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:30.785 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.785 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.785 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:54:30.785 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:30.786 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:30.786 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:30.786 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:30.786 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:30.787 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.606947+0000", 2022-06-17T12:54:30.787 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:30.787 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.787 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.788 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:54:30.788 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:30.788 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:30.788 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:30.788 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:30.789 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:30.789 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.514412+0000", 2022-06-17T12:54:30.789 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:30.789 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.790 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.790 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:54:30.790 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:30.790 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:30.790 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:30.791 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:30.791 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:30.791 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.394645+0000", 2022-06-17T12:54:30.791 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:30.792 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.792 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.792 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:54:30.792 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:30.792 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:30.793 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:30.793 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:30.793 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:30.793 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.343806+0000", 2022-06-17T12:54:30.794 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:30.794 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.794 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.795 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:54:30.795 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:30.795 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:30.795 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:30.795 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:30.796 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:30.796 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.591864+0000", 2022-06-17T12:54:30.796 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:30.796 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.797 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:30.797 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:54:30.797 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:30.797 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:30.797 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:30.798 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:30.798 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:30.798 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.607045+0000", 2022-06-17T12:54:30.798 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:30.799 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.799 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:54:30.799 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap": [], 2022-06-17T12:54:30.799 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap_items": [], 2022-06-17T12:54:30.800 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_temp": [], 2022-06-17T12:54:30.800 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_temp": [], 2022-06-17T12:54:30.800 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklist": { 2022-06-17T12:54:30.800 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.36:6835/2728662009": "2022-06-18T12:54:24.085641+0000", 2022-06-17T12:54:30.800 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.36:6834/2728662009": "2022-06-18T12:54:24.085641+0000" 2022-06-17T12:54:30.801 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.801 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profiles": { 2022-06-17T12:54:30.801 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs_data": { 2022-06-17T12:54:30.801 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-device-class": "", 2022-06-17T12:54:30.801 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:54:30.802 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-root": "default", 2022-06-17T12:54:30.802 INFO:teuthology.orchestra.run.smithi036.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-17T12:54:30.802 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:54:30.802 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "2", 2022-06-17T12:54:30.803 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:54:30.803 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:54:30.803 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van", 2022-06-17T12:54:30.803 INFO:teuthology.orchestra.run.smithi036.stdout: "w": "8" 2022-06-17T12:54:30.804 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.804 INFO:teuthology.orchestra.run.smithi036.stdout: "default": { 2022-06-17T12:54:30.804 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:54:30.804 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:54:30.804 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "1", 2022-06-17T12:54:30.805 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:54:30.805 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:54:30.805 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van" 2022-06-17T12:54:30.805 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:30.805 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:30.806 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps_queue": [], 2022-06-17T12:54:30.806 INFO:teuthology.orchestra.run.smithi036.stdout: "new_removed_snaps": [], 2022-06-17T12:54:30.806 INFO:teuthology.orchestra.run.smithi036.stdout: "new_purged_snaps": [], 2022-06-17T12:54:30.806 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_node_flags": {}, 2022-06-17T12:54:30.807 INFO:teuthology.orchestra.run.smithi036.stdout: "device_class_flags": {} 2022-06-17T12:54:30.807 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:54:30.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.576+0000 7f5371893700 1 -- 172.21.15.36:0/1400879664 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f534c0539b0 msgr2=0x7f534c055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:30.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.576+0000 7f5371893700 1 --2- 172.21.15.36:0/1400879664 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f534c0539b0 0x7f534c055e70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f536000d000 tx=0x7f5360009040).stop 2022-06-17T12:54:30.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.576+0000 7f5371893700 1 -- 172.21.15.36:0/1400879664 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f536c0b15f0 msgr2=0x7f536c142e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:30.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.576+0000 7f5371893700 1 --2- 172.21.15.36:0/1400879664 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f536c0b15f0 0x7f536c142e90 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7f535c00c000 tx=0x7f535c006df0).stop 2022-06-17T12:54:30.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.576+0000 7f5371893700 1 -- 172.21.15.36:0/1400879664 shutdown_connections 2022-06-17T12:54:30.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.576+0000 7f5371893700 1 --2- 172.21.15.36:0/1400879664 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f534c0539b0 0x7f534c055e70 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:30.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.576+0000 7f5371893700 1 --2- 172.21.15.36:0/1400879664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f536c0b27b0 0x7f536c1525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:30.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.576+0000 7f5371893700 1 --2- 172.21.15.36:0/1400879664 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f536c0b15f0 0x7f536c142e90 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:30.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.576+0000 7f5371893700 1 --2- 172.21.15.36:0/1400879664 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f536c0b0d70 0x7f536c13c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:30.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.576+0000 7f5371893700 1 -- 172.21.15.36:0/1400879664 >> 172.21.15.36:0/1400879664 conn(0x7f536c0b5d10 msgr2=0x7f536c066d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:30.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.576+0000 7f5371893700 1 -- 172.21.15.36:0/1400879664 shutdown_connections 2022-06-17T12:54:30.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.576+0000 7f5371893700 1 -- 172.21.15.36:0/1400879664 wait complete. 2022-06-17T12:54:30.813 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:54:30.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.936+0000 7fba14e9d700 1 Processor -- start 2022-06-17T12:54:30.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.936+0000 7fba14e9d700 1 -- start start 2022-06-17T12:54:30.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba14e9d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba10144c00 0x7fba10062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:30.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba14e9d700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba100b0ba0 con 0x7fba100ad330 2022-06-17T12:54:30.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba14e9d700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba10147e90 con 0x7fba10062770 2022-06-17T12:54:30.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba14e9d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba100ae820 con 0x7fba10144c00 2022-06-17T12:54:30.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0ed9d700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fba100ad330 0x7fba100ad780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54364/0 (socket says 172.21.15.36:54364) 2022-06-17T12:54:30.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0ed9d700 1 -- 172.21.15.36:0/2852352341 learned_addr learned my addr 172.21.15.36:0/2852352341 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:30.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0dd9b700 1 --2- 172.21.15.36:0/2852352341 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba10144c00 0x7fba10062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:30.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0d59a700 1 -- 172.21.15.36:0/2852352341 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1209823666 0 0) 0x7fba100b0ba0 con 0x7fba100ad330 2022-06-17T12:54:30.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0d59a700 1 -- 172.21.15.36:0/2852352341 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9f8003260 con 0x7fba100ad330 2022-06-17T12:54:30.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0d59a700 1 -- 172.21.15.36:0/2852352341 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2352580665 0 0) 0x7fb9f8003260 con 0x7fba100ad330 2022-06-17T12:54:30.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0d59a700 1 -- 172.21.15.36:0/2852352341 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba10144c00 msgr2=0x7fba10062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:30.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0d59a700 1 --2- 172.21.15.36:0/2852352341 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba10144c00 0x7fba10062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:30.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0d59a700 1 -- 172.21.15.36:0/2852352341 >> v1:172.21.15.70:6789/0 conn(0x7fba10062770 legacy=0x7fba101444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:30.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0d59a700 1 -- 172.21.15.36:0/2852352341 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba100b0dd0 con 0x7fba100ad330 2022-06-17T12:54:30.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0d59a700 1 -- 172.21.15.36:0/2852352341 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7fba08001660 con 0x7fba100ad330 2022-06-17T12:54:30.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0d59a700 1 -- 172.21.15.36:0/2852352341 >> v1:172.21.15.36:6789/0 conn(0x7fba100ad330 legacy=0x7fba100ad780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:30.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0d59a700 1 --2- 172.21.15.36:0/2852352341 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb9f8003780 0x7fb9f8003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:30.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0d59a700 1 --2- 172.21.15.36:0/2852352341 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba10144c00 0x7fb9f8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:30.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0d59a700 1 --2- 172.21.15.36:0/2852352341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9f80049f0 0x7fb9f8004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:30.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0d59a700 1 -- 172.21.15.36:0/2852352341 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba100ae820 con 0x7fb9f8003780 2022-06-17T12:54:30.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0d59a700 1 -- 172.21.15.36:0/2852352341 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f800b780 con 0x7fb9f80049f0 2022-06-17T12:54:30.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0d59a700 1 -- 172.21.15.36:0/2852352341 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f800b940 con 0x7fba10144c00 2022-06-17T12:54:30.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0e59c700 1 --2- 172.21.15.36:0/2852352341 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb9f8003780 0x7fb9f8003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:30.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0dd9b700 1 --2- 172.21.15.36:0/2852352341 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba10144c00 0x7fb9f8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:30.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0ed9d700 1 --2- 172.21.15.36:0/2852352341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9f80049f0 0x7fb9f8004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:30.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0e59c700 1 -- 172.21.15.36:0/2852352341 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba10144c00 msgr2=0x7fb9f8004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:30.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0e59c700 1 --2- 172.21.15.36:0/2852352341 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba10144c00 0x7fb9f8004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:30.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0e59c700 1 -- 172.21.15.36:0/2852352341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9f80049f0 msgr2=0x7fb9f8004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:30.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0e59c700 1 --2- 172.21.15.36:0/2852352341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9f80049f0 0x7fb9f8004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:30.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0e59c700 1 -- 172.21.15.36:0/2852352341 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f8003260 con 0x7fb9f8003780 2022-06-17T12:54:30.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0dd9b700 1 --2- 172.21.15.36:0/2852352341 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba10144c00 0x7fb9f8004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:30.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0e59c700 1 --2- 172.21.15.36:0/2852352341 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb9f8003780 0x7fb9f8003ba0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7fba1005d2b0 tx=0x7fb9fc007a00).ready entity=mon.0 client_cookie=590ae0f9ce465833 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:30.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0d59a700 1 -- 172.21.15.36:0/2852352341 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9fc007d10 con 0x7fb9f8003780 2022-06-17T12:54:30.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0d59a700 1 -- 172.21.15.36:0/2852352341 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9fc007030 con 0x7fb9f8003780 2022-06-17T12:54:30.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.940+0000 7fba0d59a700 1 -- 172.21.15.36:0/2852352341 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9fc007390 con 0x7fb9f8003780 2022-06-17T12:54:30.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba14e9d700 1 -- 172.21.15.36:0/2852352341 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb9f8003780 msgr2=0x7fb9f8003ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:30.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba14e9d700 1 --2- 172.21.15.36:0/2852352341 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb9f8003780 0x7fb9f8003ba0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7fba1005d2b0 tx=0x7fb9fc007a00).stop 2022-06-17T12:54:30.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba0ed9d700 1 -- 172.21.15.36:0/2852352341 reap_dead start 2022-06-17T12:54:30.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba14e9d700 1 -- 172.21.15.36:0/2852352341 shutdown_connections 2022-06-17T12:54:30.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba14e9d700 1 -- 172.21.15.36:0/2852352341 >> 172.21.15.36:0/2852352341 conn(0x7fba100b53e0 msgr2=0x7fba100b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:30.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba14e9d700 1 -- 172.21.15.36:0/2852352341 shutdown_connections 2022-06-17T12:54:30.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba14e9d700 1 -- 172.21.15.36:0/2852352341 wait complete. 2022-06-17T12:54:30.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba14e9d700 1 Processor -- start 2022-06-17T12:54:30.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba14e9d700 1 -- start start 2022-06-17T12:54:30.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba14e9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9f8003780 0x7fba1014ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:30.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba14e9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba10065670 0x7fba10065af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:30.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba14e9d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba10065fe0 0x7fba1012ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:30.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba14e9d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1014b840 con 0x7fba10065670 2022-06-17T12:54:30.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba14e9d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba10066bd0 con 0x7fb9f8003780 2022-06-17T12:54:30.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba14e9d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba100631f0 con 0x7fba10065fe0 2022-06-17T12:54:30.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba0dd9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba10065670 0x7fba10065af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:30.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba0dd9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba10065670 0x7fba10065af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36844/0 (socket says 172.21.15.36:36844) 2022-06-17T12:54:30.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba0dd9b700 1 -- 172.21.15.36:0/2938286598 learned_addr learned my addr 172.21.15.36:0/2938286598 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:30.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba0e59c700 1 --2- 172.21.15.36:0/2938286598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9f8003780 0x7fba1014ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:30.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba0ed9d700 1 --2- 172.21.15.36:0/2938286598 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba10065fe0 0x7fba1012ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:30.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba0dd9b700 1 -- 172.21.15.36:0/2938286598 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba10065fe0 msgr2=0x7fba1012ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:30.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba0dd9b700 1 --2- 172.21.15.36:0/2938286598 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba10065fe0 0x7fba1012ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:30.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba0dd9b700 1 -- 172.21.15.36:0/2938286598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9f8003780 msgr2=0x7fba1014ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:30.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba0dd9b700 1 --2- 172.21.15.36:0/2938286598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9f8003780 0x7fba1014ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:30.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba0dd9b700 1 -- 172.21.15.36:0/2938286598 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba10062f50 con 0x7fba10065670 2022-06-17T12:54:30.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.944+0000 7fba0ed9d700 1 --2- 172.21.15.36:0/2938286598 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba10065fe0 0x7fba1012ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:30.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.948+0000 7fba0dd9b700 1 --2- 172.21.15.36:0/2938286598 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba10065670 0x7fba10065af0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7fba0000b000 tx=0x7fba000091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:30.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.948+0000 7fba077fe700 1 -- 172.21.15.36:0/2938286598 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba0000fcd0 con 0x7fba10065670 2022-06-17T12:54:30.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.948+0000 7fba077fe700 1 -- 172.21.15.36:0/2938286598 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0000fe30 con 0x7fba10065670 2022-06-17T12:54:30.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.948+0000 7fba14e9d700 1 -- 172.21.15.36:0/2938286598 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba10157870 con 0x7fba10065670 2022-06-17T12:54:30.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.948+0000 7fba077fe700 1 -- 172.21.15.36:0/2938286598 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba00014630 con 0x7fba10065670 2022-06-17T12:54:30.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.948+0000 7fba14e9d700 1 -- 172.21.15.36:0/2938286598 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba10062b50 con 0x7fba10065670 2022-06-17T12:54:30.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.948+0000 7fba14e9d700 1 -- 172.21.15.36:0/2938286598 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba10066d40 con 0x7fba10065670 2022-06-17T12:54:30.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.948+0000 7fba077fe700 1 -- 172.21.15.36:0/2938286598 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fba00024070 con 0x7fba10065670 2022-06-17T12:54:30.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.948+0000 7fba077fe700 1 --2- 172.21.15.36:0/2938286598 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb9ec0539d0 0x7fb9ec055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:30.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.948+0000 7fba077fe700 1 -- 172.21.15.36:0/2938286598 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fba0006cad0 con 0x7fba10065670 2022-06-17T12:54:30.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.948+0000 7fba0e59c700 1 --2- 172.21.15.36:0/2938286598 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb9ec0539d0 0x7fb9ec055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:30.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.948+0000 7fba0e59c700 1 --2- 172.21.15.36:0/2938286598 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb9ec0539d0 0x7fb9ec055e90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7fb9fc019000 tx=0x7fb9fc015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:30.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:30.956+0000 7fba077fe700 1 -- 172.21.15.36:0/2938286598 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fba00043aa0 con 0x7fba10065670 2022-06-17T12:54:31.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.120+0000 7fba14e9d700 1 -- 172.21.15.36:0/2938286598 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba100b1c20 con 0x7fba10065670 2022-06-17T12:54:31.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.120+0000 7fba077fe700 1 -- 172.21.15.36:0/2938286598 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+1209 (secure 0 0 0) 0x7fba000470c0 con 0x7fba10065670 2022-06-17T12:54:31.109 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:31.110 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":44,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/3068692336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3068692336},{"type":"v1","addr":"172.21.15.70:6833","nonce":3068692336}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40},{"gid":6999,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/1548464171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":1548464171},{"type":"v1","addr":"172.21.15.36:6835","nonce":1548464171}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44}],"filesystems":[]} 2022-06-17T12:54:31.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.124+0000 7fba14e9d700 1 -- 172.21.15.36:0/2938286598 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb9ec0539d0 msgr2=0x7fb9ec055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:31.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.124+0000 7fba14e9d700 1 --2- 172.21.15.36:0/2938286598 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb9ec0539d0 0x7fb9ec055e90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7fb9fc019000 tx=0x7fb9fc015000).stop 2022-06-17T12:54:31.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.124+0000 7fba14e9d700 1 -- 172.21.15.36:0/2938286598 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba10065670 msgr2=0x7fba10065af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:31.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.124+0000 7fba14e9d700 1 --2- 172.21.15.36:0/2938286598 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba10065670 0x7fba10065af0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7fba0000b000 tx=0x7fba000091d0).stop 2022-06-17T12:54:31.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.124+0000 7fba14e9d700 1 -- 172.21.15.36:0/2938286598 shutdown_connections 2022-06-17T12:54:31.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.124+0000 7fba14e9d700 1 --2- 172.21.15.36:0/2938286598 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb9ec0539d0 0x7fb9ec055e90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:31.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.124+0000 7fba14e9d700 1 --2- 172.21.15.36:0/2938286598 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba10065fe0 0x7fba1012ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:31.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.124+0000 7fba14e9d700 1 --2- 172.21.15.36:0/2938286598 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba10065670 0x7fba10065af0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:31.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.124+0000 7fba14e9d700 1 --2- 172.21.15.36:0/2938286598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9f8003780 0x7fba1014ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:31.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.124+0000 7fba14e9d700 1 -- 172.21.15.36:0/2938286598 >> 172.21.15.36:0/2938286598 conn(0x7fba100b53e0 msgr2=0x7fba1005f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:31.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.124+0000 7fba14e9d700 1 -- 172.21.15.36:0/2938286598 shutdown_connections 2022-06-17T12:54:31.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.124+0000 7fba14e9d700 1 -- 172.21.15.36:0/2938286598 wait complete. 2022-06-17T12:54:31.119 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 44 2022-06-17T12:54:31.122 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:54:31.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.236+0000 7f9af42fe700 1 Processor -- start 2022-06-17T12:54:31.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.236+0000 7f9af42fe700 1 -- start start 2022-06-17T12:54:31.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.236+0000 7f9af42fe700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9aec062770 0x7f9aec062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:31.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.236+0000 7f9af42fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9aec0ad330 0x7f9aec0ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:31.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.236+0000 7f9af42fe700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aec0b0ba0 con 0x7f9aec1444f0 2022-06-17T12:54:31.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.236+0000 7f9af42fe700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aec0ae820 con 0x7f9aec062770 2022-06-17T12:54:31.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.236+0000 7f9af42fe700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec061030 con 0x7f9aec0ad330 2022-06-17T12:54:31.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.240+0000 7f9af1899700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f9aec1444f0 0x7f9aec144960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54380/0 (socket says 172.21.15.36:54380) 2022-06-17T12:54:31.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.240+0000 7f9af1899700 1 -- 172.21.15.36:0/1574236935 learned_addr learned my addr 172.21.15.36:0/1574236935 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:31.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.240+0000 7f9af209a700 1 --2- 172.21.15.36:0/1574236935 >> v2:172.21.15.70:3300/0 conn(0x7f9aec062770 0x7f9aec062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:31.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.240+0000 7f9af289b700 1 --2- 172.21.15.36:0/1574236935 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9aec0ad330 0x7f9aec0ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:31.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.240+0000 7f9af1098700 1 -- 172.21.15.36:0/1574236935 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3945573475 0 0) 0x7f9aec0b0ba0 con 0x7f9aec1444f0 2022-06-17T12:54:31.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.240+0000 7f9af1098700 1 -- 172.21.15.36:0/1574236935 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ad8003260 con 0x7f9aec1444f0 2022-06-17T12:54:31.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.240+0000 7f9af289b700 1 -- 172.21.15.36:0/1574236935 >> v2:172.21.15.70:3300/0 conn(0x7f9aec062770 msgr2=0x7f9aec062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:31.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.240+0000 7f9af289b700 1 --2- 172.21.15.36:0/1574236935 >> v2:172.21.15.70:3300/0 conn(0x7f9aec062770 0x7f9aec062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:31.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.240+0000 7f9af289b700 1 -- 172.21.15.36:0/1574236935 >> v1:172.21.15.36:6789/0 conn(0x7f9aec1444f0 legacy=0x7f9aec144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:31.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.240+0000 7f9af289b700 1 -- 172.21.15.36:0/1574236935 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aec147e90 con 0x7f9aec0ad330 2022-06-17T12:54:31.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.240+0000 7f9af289b700 1 --2- 172.21.15.36:0/1574236935 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9aec0ad330 0x7f9aec0ad7c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f9ae40032f0 tx=0x7f9ae40091d0).ready entity=mon.2 client_cookie=87e2c03c4fec9c45 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:31.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.240+0000 7f9af1098700 1 -- 172.21.15.36:0/1574236935 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ae400b600 con 0x7f9aec0ad330 2022-06-17T12:54:31.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.240+0000 7f9af1098700 1 -- 172.21.15.36:0/1574236935 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae400b760 con 0x7f9aec0ad330 2022-06-17T12:54:31.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.240+0000 7f9af1098700 1 -- 172.21.15.36:0/1574236935 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ae400b8d0 con 0x7f9aec0ad330 2022-06-17T12:54:31.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.240+0000 7f9af42fe700 1 -- 172.21.15.36:0/1574236935 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9aec0ad330 msgr2=0x7f9aec0ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:31.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.240+0000 7f9af42fe700 1 --2- 172.21.15.36:0/1574236935 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9aec0ad330 0x7f9aec0ad7c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f9ae40032f0 tx=0x7f9ae40091d0).stop 2022-06-17T12:54:31.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.240+0000 7f9af42fe700 1 -- 172.21.15.36:0/1574236935 shutdown_connections 2022-06-17T12:54:31.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.240+0000 7f9af42fe700 1 --2- 172.21.15.36:0/1574236935 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9aec0ad330 0x7f9aec0ad7c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:31.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.240+0000 7f9af42fe700 1 --2- 172.21.15.36:0/1574236935 >> v2:172.21.15.70:3300/0 conn(0x7f9aec062770 0x7f9aec062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:31.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.240+0000 7f9af42fe700 1 -- 172.21.15.36:0/1574236935 >> 172.21.15.36:0/1574236935 conn(0x7f9aec0b53e0 msgr2=0x7f9aec0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:31.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.240+0000 7f9af42fe700 1 -- 172.21.15.36:0/1574236935 shutdown_connections 2022-06-17T12:54:31.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.240+0000 7f9af42fe700 1 -- 172.21.15.36:0/1574236935 wait complete. 2022-06-17T12:54:31.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.244+0000 7f9af42fe700 1 Processor -- start 2022-06-17T12:54:31.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.244+0000 7f9af42fe700 1 -- start start 2022-06-17T12:54:31.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.244+0000 7f9af42fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9aec062770 0x7f9aec063fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:31.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.244+0000 7f9af42fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9aec0ad330 0x7f9aec148ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:31.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.244+0000 7f9af42fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9aec1444f0 0x7f9aec0669c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:31.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.244+0000 7f9af42fe700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec063130 con 0x7f9aec1444f0 2022-06-17T12:54:31.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.244+0000 7f9af42fe700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec063390 con 0x7f9aec062770 2022-06-17T12:54:31.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.244+0000 7f9af42fe700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec064610 con 0x7f9aec0ad330 2022-06-17T12:54:31.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.244+0000 7f9af289b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9aec1444f0 0x7f9aec0669c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:31.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.244+0000 7f9af289b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9aec1444f0 0x7f9aec0669c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36860/0 (socket says 172.21.15.36:36860) 2022-06-17T12:54:31.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.244+0000 7f9af289b700 1 -- 172.21.15.36:0/1934114076 learned_addr learned my addr 172.21.15.36:0/1934114076 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:31.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.244+0000 7f9af1899700 1 --2- 172.21.15.36:0/1934114076 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9aec0ad330 0x7f9aec148ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:31.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.244+0000 7f9af209a700 1 --2- 172.21.15.36:0/1934114076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9aec062770 0x7f9aec063fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:31.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.248+0000 7f9af289b700 1 -- 172.21.15.36:0/1934114076 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9aec0ad330 msgr2=0x7f9aec148ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:31.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.248+0000 7f9af289b700 1 --2- 172.21.15.36:0/1934114076 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9aec0ad330 0x7f9aec148ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:31.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.248+0000 7f9af289b700 1 -- 172.21.15.36:0/1934114076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9aec062770 msgr2=0x7f9aec063fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:31.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.248+0000 7f9af289b700 1 --2- 172.21.15.36:0/1934114076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9aec062770 0x7f9aec063fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:31.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.248+0000 7f9af289b700 1 -- 172.21.15.36:0/1934114076 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae4009470 con 0x7f9aec1444f0 2022-06-17T12:54:31.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.248+0000 7f9af209a700 1 --2- 172.21.15.36:0/1934114076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9aec062770 0x7f9aec063fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:31.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.248+0000 7f9af289b700 1 --2- 172.21.15.36:0/1934114076 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9aec1444f0 0x7f9aec0669c0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7f9ae40032f0 tx=0x7f9ae4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:31.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.248+0000 7f9ae37fe700 1 -- 172.21.15.36:0/1934114076 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ae400b8d0 con 0x7f9aec1444f0 2022-06-17T12:54:31.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.248+0000 7f9ae37fe700 1 -- 172.21.15.36:0/1934114076 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae4021630 con 0x7f9aec1444f0 2022-06-17T12:54:31.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.248+0000 7f9af42fe700 1 -- 172.21.15.36:0/1934114076 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aec14b670 con 0x7f9aec1444f0 2022-06-17T12:54:31.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.248+0000 7f9ae37fe700 1 -- 172.21.15.36:0/1934114076 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ae400b600 con 0x7f9aec1444f0 2022-06-17T12:54:31.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.248+0000 7f9af42fe700 1 -- 172.21.15.36:0/1934114076 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aec159d00 con 0x7f9aec1444f0 2022-06-17T12:54:31.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.248+0000 7f9af42fe700 1 -- 172.21.15.36:0/1934114076 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aec063590 con 0x7f9aec1444f0 2022-06-17T12:54:31.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.248+0000 7f9ae37fe700 1 -- 172.21.15.36:0/1934114076 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f9ae4029070 con 0x7f9aec1444f0 2022-06-17T12:54:31.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.248+0000 7f9ae37fe700 1 --2- 172.21.15.36:0/1934114076 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9acc053ad0 0x7f9acc055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:31.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.248+0000 7f9ae37fe700 1 -- 172.21.15.36:0/1934114076 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f9ae406cf70 con 0x7f9aec1444f0 2022-06-17T12:54:31.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.248+0000 7f9af209a700 1 --2- 172.21.15.36:0/1934114076 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9acc053ad0 0x7f9acc055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:31.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.252+0000 7f9af209a700 1 --2- 172.21.15.36:0/1934114076 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9acc053ad0 0x7f9acc055f90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f9adc00b000 tx=0x7f9adc008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:31.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.256+0000 7f9ae37fe700 1 -- 172.21.15.36:0/1934114076 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9ae4043fb0 con 0x7f9aec1444f0 2022-06-17T12:54:31.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.424+0000 7f9af42fe700 1 -- 172.21.15.36:0/1934114076 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9aec0666b0 con 0x7f9aec1444f0 2022-06-17T12:54:31.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.424+0000 7f9ae37fe700 1 -- 172.21.15.36:0/1934114076 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+1209 (secure 0 0 0) 0x7f9ae4027090 con 0x7f9aec1444f0 2022-06-17T12:54:31.412 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:31.413 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":44,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/3068692336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3068692336},{"type":"v1","addr":"172.21.15.70:6833","nonce":3068692336}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40},{"gid":6999,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/1548464171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":1548464171},{"type":"v1","addr":"172.21.15.36:6835","nonce":1548464171}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44}],"filesystems":[]} 2022-06-17T12:54:31.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.424+0000 7f9af42fe700 1 -- 172.21.15.36:0/1934114076 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9acc053ad0 msgr2=0x7f9acc055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:31.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.424+0000 7f9af42fe700 1 --2- 172.21.15.36:0/1934114076 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9acc053ad0 0x7f9acc055f90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f9adc00b000 tx=0x7f9adc008040).stop 2022-06-17T12:54:31.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.424+0000 7f9af42fe700 1 -- 172.21.15.36:0/1934114076 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9aec1444f0 msgr2=0x7f9aec0669c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:31.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.424+0000 7f9af42fe700 1 --2- 172.21.15.36:0/1934114076 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9aec1444f0 0x7f9aec0669c0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7f9ae40032f0 tx=0x7f9ae4016000).stop 2022-06-17T12:54:31.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.424+0000 7f9af42fe700 1 -- 172.21.15.36:0/1934114076 shutdown_connections 2022-06-17T12:54:31.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.424+0000 7f9af42fe700 1 --2- 172.21.15.36:0/1934114076 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9acc053ad0 0x7f9acc055f90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:31.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.424+0000 7f9af42fe700 1 --2- 172.21.15.36:0/1934114076 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9aec1444f0 0x7f9aec0669c0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:31.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.424+0000 7f9af42fe700 1 --2- 172.21.15.36:0/1934114076 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9aec0ad330 0x7f9aec148ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:31.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.424+0000 7f9af42fe700 1 --2- 172.21.15.36:0/1934114076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9aec062770 0x7f9aec063fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:31.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.424+0000 7f9af42fe700 1 -- 172.21.15.36:0/1934114076 >> 172.21.15.36:0/1934114076 conn(0x7f9aec0b53e0 msgr2=0x7f9aec14df50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:31.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.424+0000 7f9af42fe700 1 -- 172.21.15.36:0/1934114076 shutdown_connections 2022-06-17T12:54:31.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.424+0000 7f9af42fe700 1 -- 172.21.15.36:0/1934114076 wait complete. 2022-06-17T12:54:31.418 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 44 2022-06-17T12:54:31.424 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-17T12:54:31.425 INFO:tasks.ceph.mds.a:Stopping old one... 2022-06-17T12:54:31.425 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-17T12:54:31.426 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T12:54:31.427 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-17T12:54:31.427 INFO:tasks.ceph.mds.b:Stopping old one... 2022-06-17T12:54:31.427 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-17T12:54:31.428 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T12:54:31.440 INFO:tasks.ceph.mds.a:Stopped 2022-06-17T12:54:31.440 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-17T12:54:31.443 INFO:tasks.ceph.mds.a:Started 2022-06-17T12:54:31.443 DEBUG:teuthology.parallel:result is None 2022-06-17T12:54:31.444 INFO:tasks.ceph.mds.b:Stopped 2022-06-17T12:54:31.445 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-17T12:54:31.494 INFO:tasks.ceph.mds.b:Started 2022-06-17T12:54:31.495 DEBUG:teuthology.parallel:result is None 2022-06-17T12:54:31.496 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-06-17T12:54:31.517 INFO:tasks.ceph.mds.a.smithi036.stdout:starting mds.a at 2022-06-17T12:54:31.567 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2022-06-17T12:54:31.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.612+0000 7fbf0df64700 1 Processor -- start 2022-06-17T12:54:31.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.612+0000 7fbf0df64700 1 -- start start 2022-06-17T12:54:31.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.612+0000 7fbf0df64700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbf080b2cd0 0x7fbf081465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:31.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.612+0000 7fbf0df64700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf080b25e0 con 0x7fbf080b2200 2022-06-17T12:54:31.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.612+0000 7fbf0df64700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf080ad790 con 0x7fbf08146ae0 2022-06-17T12:54:31.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.612+0000 7fbf0df64700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf080aef40 con 0x7fbf080b2cd0 2022-06-17T12:54:31.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.612+0000 7fbf0cf62700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fbf080b2200 0x7fbf080af3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54406/0 (socket says 172.21.15.36:54406) 2022-06-17T12:54:31.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.612+0000 7fbf0cf62700 1 -- 172.21.15.36:0/792785486 learned_addr learned my addr 172.21.15.36:0/792785486 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:31.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.612+0000 7fbf07fff700 1 --2- 172.21.15.36:0/792785486 >> v2:172.21.15.70:3300/0 conn(0x7fbf080b2cd0 0x7fbf081465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:31.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.612+0000 7fbf077fe700 1 -- 172.21.15.36:0/792785486 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3764891915 0 0) 0x7fbf080b25e0 con 0x7fbf080b2200 2022-06-17T12:54:31.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.612+0000 7fbf077fe700 1 -- 172.21.15.36:0/792785486 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbeec003260 con 0x7fbf080b2200 2022-06-17T12:54:31.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf077fe700 1 -- 172.21.15.36:0/792785486 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2506711983 0 0) 0x7fbeec003260 con 0x7fbf080b2200 2022-06-17T12:54:31.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf077fe700 1 -- 172.21.15.36:0/792785486 >> v2:172.21.15.70:3300/0 conn(0x7fbf080b2cd0 msgr2=0x7fbf081465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:31.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf077fe700 1 --2- 172.21.15.36:0/792785486 >> v2:172.21.15.70:3300/0 conn(0x7fbf080b2cd0 0x7fbf081465f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:31.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf077fe700 1 -- 172.21.15.36:0/792785486 >> v1:172.21.15.70:6789/0 conn(0x7fbf08146ae0 legacy=0x7fbf08146f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:31.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf077fe700 1 -- 172.21.15.36:0/792785486 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf080afaf0 con 0x7fbf080b2200 2022-06-17T12:54:31.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf077fe700 1 -- 172.21.15.36:0/792785486 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7fbef80019e0 con 0x7fbf080b2200 2022-06-17T12:54:31.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf077fe700 1 -- 172.21.15.36:0/792785486 >> v1:172.21.15.36:6789/0 conn(0x7fbf080b2200 legacy=0x7fbf080af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:31.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf077fe700 1 --2- 172.21.15.36:0/792785486 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbeec003750 0x7fbeec003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:31.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf077fe700 1 --2- 172.21.15.36:0/792785486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbeec004500 0x7fbeec004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:31.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf077fe700 1 --2- 172.21.15.36:0/792785486 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbeec004e70 0x7fbeec00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:31.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf077fe700 1 -- 172.21.15.36:0/792785486 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf080aef40 con 0x7fbeec003750 2022-06-17T12:54:31.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf077fe700 1 -- 172.21.15.36:0/792785486 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeec00bd20 con 0x7fbeec004500 2022-06-17T12:54:31.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf0cf62700 1 --2- 172.21.15.36:0/792785486 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbeec003750 0x7fbeec003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:31.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf077fe700 1 -- 172.21.15.36:0/792785486 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeec00bee0 con 0x7fbeec004e70 2022-06-17T12:54:31.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf07fff700 1 --2- 172.21.15.36:0/792785486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbeec004500 0x7fbeec004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:31.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf0d763700 1 --2- 172.21.15.36:0/792785486 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbeec004e70 0x7fbeec00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:31.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf0cf62700 1 -- 172.21.15.36:0/792785486 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbeec004e70 msgr2=0x7fbeec00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:31.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf0cf62700 1 --2- 172.21.15.36:0/792785486 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbeec004e70 0x7fbeec00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:31.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf0cf62700 1 -- 172.21.15.36:0/792785486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbeec004500 msgr2=0x7fbeec004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:31.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf0cf62700 1 --2- 172.21.15.36:0/792785486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbeec004500 0x7fbeec004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:31.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf0d763700 1 -- 172.21.15.36:0/792785486 reap_dead start 2022-06-17T12:54:31.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf0cf62700 1 -- 172.21.15.36:0/792785486 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeec003260 con 0x7fbeec003750 2022-06-17T12:54:31.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf07fff700 1 --2- 172.21.15.36:0/792785486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbeec004500 0x7fbeec004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:31.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf0cf62700 1 --2- 172.21.15.36:0/792785486 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbeec003750 0x7fbeec003b70 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7fbef80016f0 tx=0x7fbf08060500).ready entity=mon.0 client_cookie=41a80c4bf8e0c29 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:31.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf077fe700 1 -- 172.21.15.36:0/792785486 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbef800cca0 con 0x7fbeec003750 2022-06-17T12:54:31.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf077fe700 1 -- 172.21.15.36:0/792785486 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef800ce00 con 0x7fbeec003750 2022-06-17T12:54:31.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf077fe700 1 -- 172.21.15.36:0/792785486 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbef8005b80 con 0x7fbeec003750 2022-06-17T12:54:31.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf0df64700 1 -- 172.21.15.36:0/792785486 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbeec003750 msgr2=0x7fbeec003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:31.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf0df64700 1 --2- 172.21.15.36:0/792785486 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbeec003750 0x7fbeec003b70 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7fbef80016f0 tx=0x7fbf08060500).stop 2022-06-17T12:54:31.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf0df64700 1 -- 172.21.15.36:0/792785486 shutdown_connections 2022-06-17T12:54:31.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf0df64700 1 --2- 172.21.15.36:0/792785486 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbeec003750 0x7fbeec003b70 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:31.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf0df64700 1 -- 172.21.15.36:0/792785486 >> 172.21.15.36:0/792785486 conn(0x7fbf0805f770 msgr2=0x7fbf0805fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:31.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf0df64700 1 -- 172.21.15.36:0/792785486 shutdown_connections 2022-06-17T12:54:31.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.616+0000 7fbf0df64700 1 -- 172.21.15.36:0/792785486 wait complete. 2022-06-17T12:54:31.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf0df64700 1 Processor -- start 2022-06-17T12:54:31.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf0df64700 1 -- start start 2022-06-17T12:54:31.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf0df64700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbeec003750 0x7fbf08145310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:31.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf0df64700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf08145800 0x7fbf0815a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:31.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf0df64700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbf08145d00 0x7fbf08142770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:31.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf0df64700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf080a7b60 con 0x7fbf08145d00 2022-06-17T12:54:31.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf0df64700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf080a7180 con 0x7fbeec003750 2022-06-17T12:54:31.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf0df64700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf080a7420 con 0x7fbf08145800 2022-06-17T12:54:31.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf0d763700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbf08145d00 0x7fbf08142770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:31.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf07fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf08145800 0x7fbf0815a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:31.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf0d763700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbf08145d00 0x7fbf08142770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36892/0 (socket says 172.21.15.36:36892) 2022-06-17T12:54:31.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf0cf62700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbeec003750 0x7fbf08145310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:31.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf07fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf08145800 0x7fbf0815a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:51680/0 (socket says 172.21.15.36:51680) 2022-06-17T12:54:31.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf0d763700 1 -- 172.21.15.36:0/4512936 learned_addr learned my addr 172.21.15.36:0/4512936 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:31.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf0d763700 1 -- 172.21.15.36:0/4512936 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf08145800 msgr2=0x7fbf0815a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:31.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf0d763700 1 --2- 172.21.15.36:0/4512936 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf08145800 0x7fbf0815a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:31.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf0d763700 1 -- 172.21.15.36:0/4512936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbeec003750 msgr2=0x7fbf08145310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:31.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf0d763700 1 --2- 172.21.15.36:0/4512936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbeec003750 0x7fbf08145310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:31.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf0d763700 1 -- 172.21.15.36:0/4512936 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf080a76c0 con 0x7fbf08145d00 2022-06-17T12:54:31.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf0cf62700 1 --2- 172.21.15.36:0/4512936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbeec003750 0x7fbf08145310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:31.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf0d763700 1 --2- 172.21.15.36:0/4512936 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbf08145d00 0x7fbf08142770 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7fbf00000e90 tx=0x7fbf00009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:31.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf05ffb700 1 -- 172.21.15.36:0/4512936 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf0000e970 con 0x7fbf08145d00 2022-06-17T12:54:31.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf05ffb700 1 -- 172.21.15.36:0/4512936 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0000ead0 con 0x7fbf08145d00 2022-06-17T12:54:31.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf0df64700 1 -- 172.21.15.36:0/4512936 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf08143690 con 0x7fbf08145d00 2022-06-17T12:54:31.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf05ffb700 1 -- 172.21.15.36:0/4512936 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf0000ee30 con 0x7fbf08145d00 2022-06-17T12:54:31.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.620+0000 7fbf0df64700 1 -- 172.21.15.36:0/4512936 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf0815ac30 con 0x7fbf08145d00 2022-06-17T12:54:31.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.624+0000 7fbf05ffb700 1 -- 172.21.15.36:0/4512936 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fbf0001f070 con 0x7fbf08145d00 2022-06-17T12:54:31.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.624+0000 7fbf0df64700 1 -- 172.21.15.36:0/4512936 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf08144930 con 0x7fbf08145d00 2022-06-17T12:54:31.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.624+0000 7fbf05ffb700 1 --2- 172.21.15.36:0/4512936 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbee00539b0 0x7fbee0055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:31.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.624+0000 7fbf05ffb700 1 -- 172.21.15.36:0/4512936 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fbf0006b270 con 0x7fbf08145d00 2022-06-17T12:54:31.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.624+0000 7fbf0cf62700 1 --2- 172.21.15.36:0/4512936 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbee00539b0 0x7fbee0055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:31.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.624+0000 7fbf0cf62700 1 --2- 172.21.15.36:0/4512936 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbee00539b0 0x7fbee0055e70 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7fbef8019000 tx=0x7fbef8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:31.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.632+0000 7fbf05ffb700 1 -- 172.21.15.36:0/4512936 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbf08144930 con 0x7fbf08145d00 2022-06-17T12:54:31.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:31.768+0000 7fbf0df64700 1 -- 172.21.15.36:0/4512936 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fbf08144930 con 0x7fbf08145d00 2022-06-17T12:54:32.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.576+0000 7fbf05ffb700 1 -- 172.21.15.36:0/4512936 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v89) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbf08144930 con 0x7fbf08145d00 2022-06-17T12:54:32.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.580+0000 7fbf0df64700 1 -- 172.21.15.36:0/4512936 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbee00539b0 msgr2=0x7fbee0055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:32.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.580+0000 7fbf0df64700 1 --2- 172.21.15.36:0/4512936 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbee00539b0 0x7fbee0055e70 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7fbef8019000 tx=0x7fbef8015000).stop 2022-06-17T12:54:32.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.580+0000 7fbf0df64700 1 -- 172.21.15.36:0/4512936 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbf08145d00 msgr2=0x7fbf08142770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:32.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.580+0000 7fbf0df64700 1 --2- 172.21.15.36:0/4512936 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbf08145d00 0x7fbf08142770 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7fbf00000e90 tx=0x7fbf00009040).stop 2022-06-17T12:54:32.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.580+0000 7fbf0df64700 1 -- 172.21.15.36:0/4512936 shutdown_connections 2022-06-17T12:54:32.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.580+0000 7fbf0df64700 1 --2- 172.21.15.36:0/4512936 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbee00539b0 0x7fbee0055e70 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:32.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.580+0000 7fbf0df64700 1 --2- 172.21.15.36:0/4512936 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbf08145d00 0x7fbf08142770 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:32.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.580+0000 7fbf0df64700 1 --2- 172.21.15.36:0/4512936 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf08145800 0x7fbf0815a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:32.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.580+0000 7fbf0df64700 1 --2- 172.21.15.36:0/4512936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbeec003750 0x7fbf08145310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:32.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.580+0000 7fbf0df64700 1 -- 172.21.15.36:0/4512936 >> 172.21.15.36:0/4512936 conn(0x7fbf0805f770 msgr2=0x7fbf080b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:32.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.580+0000 7fbf0df64700 1 -- 172.21.15.36:0/4512936 shutdown_connections 2022-06-17T12:54:32.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.580+0000 7fbf0df64700 1 -- 172.21.15.36:0/4512936 wait complete. 2022-06-17T12:54:32.578 INFO:teuthology.orchestra.run.smithi036.stderr: removed all blacklist entries 2022-06-17T12:54:32.583 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2022-06-17T12:54:32.583 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-06-17T12:54:32.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.752+0000 7fd0e669a700 1 Processor -- start 2022-06-17T12:54:32.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.752+0000 7fd0e669a700 1 -- start start 2022-06-17T12:54:32.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0e669a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd0e00aedc0 0x7fd0e00b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:32.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0e669a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0e0064340 0x7fd0e0066b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:32.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0e669a700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e00af1a0 con 0x7fd0e00afdd0 2022-06-17T12:54:32.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0e669a700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e0063960 con 0x7fd0e00aedc0 2022-06-17T12:54:32.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0e669a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e0063c00 con 0x7fd0e0064340 2022-06-17T12:54:32.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0dffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd0e00aedc0 0x7fd0e00b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:32.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0dffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd0e00aedc0 0x7fd0e00b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36900/0 (socket says 172.21.15.36:36900) 2022-06-17T12:54:32.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0dffff700 1 -- 172.21.15.36:0/2403781377 learned_addr learned my addr 172.21.15.36:0/2403781377 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:32.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0df7fe700 1 --2- 172.21.15.36:0/2403781377 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0e0064340 0x7fd0e0066b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:32.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0dffff700 1 -- 172.21.15.36:0/2403781377 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0e0064340 msgr2=0x7fd0e0066b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:32.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0dffff700 1 --2- 172.21.15.36:0/2403781377 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0e0064340 0x7fd0e0066b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:32.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0dffff700 1 -- 172.21.15.36:0/2403781377 >> v1:172.21.15.70:6789/0 conn(0x7fd0e00afdd0 legacy=0x7fd0e005f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:32.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0deffd700 1 -- 172.21.15.36:0/2403781377 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2376696235 0 0) 0x7fd0e00af1a0 con 0x7fd0e00afdd0 2022-06-17T12:54:32.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0dffff700 1 -- 172.21.15.36:0/2403781377 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e0063ea0 con 0x7fd0e00aedc0 2022-06-17T12:54:32.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0df7fe700 1 --2- 172.21.15.36:0/2403781377 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0e0064340 0x7fd0e0066b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:32.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0dffff700 1 --2- 172.21.15.36:0/2403781377 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0e00aedc0 0x7fd0e00b0560 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7fd0c8002f70 tx=0x7fd0c8006eb0).ready entity=mon.0 client_cookie=868669062a9ac344 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:32.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0deffd700 1 -- 172.21.15.36:0/2403781377 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0c800d570 con 0x7fd0e00aedc0 2022-06-17T12:54:32.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0deffd700 1 -- 172.21.15.36:0/2403781377 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c8000e90 con 0x7fd0e00aedc0 2022-06-17T12:54:32.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0deffd700 1 -- 172.21.15.36:0/2403781377 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0c800d8c0 con 0x7fd0e00aedc0 2022-06-17T12:54:32.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0e669a700 1 -- 172.21.15.36:0/2403781377 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0e00aedc0 msgr2=0x7fd0e00b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:32.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0e669a700 1 --2- 172.21.15.36:0/2403781377 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0e00aedc0 0x7fd0e00b0560 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7fd0c8002f70 tx=0x7fd0c8006eb0).stop 2022-06-17T12:54:32.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0e669a700 1 -- 172.21.15.36:0/2403781377 shutdown_connections 2022-06-17T12:54:32.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0e669a700 1 --2- 172.21.15.36:0/2403781377 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0e0064340 0x7fd0e0066b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:32.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0e669a700 1 --2- 172.21.15.36:0/2403781377 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0e00aedc0 0x7fd0e00b0560 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:32.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0e669a700 1 -- 172.21.15.36:0/2403781377 >> 172.21.15.36:0/2403781377 conn(0x7fd0e00b5430 msgr2=0x7fd0e00b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:32.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0e669a700 1 -- 172.21.15.36:0/2403781377 shutdown_connections 2022-06-17T12:54:32.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.756+0000 7fd0e669a700 1 -- 172.21.15.36:0/2403781377 wait complete. 2022-06-17T12:54:32.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0e669a700 1 Processor -- start 2022-06-17T12:54:32.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0e669a700 1 -- start start 2022-06-17T12:54:32.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0e669a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0e0064340 0x7fd0e0148650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:32.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0e669a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0e00afdd0 0x7fd0e00a3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:32.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0e669a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0e0155dd0 0x7fd0e00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:32.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0e669a700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e00a6210 con 0x7fd0e0064340 2022-06-17T12:54:32.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0e669a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e00a5f70 con 0x7fd0e00afdd0 2022-06-17T12:54:32.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0e669a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e00ae390 con 0x7fd0e0155dd0 2022-06-17T12:54:32.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0dffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0e0064340 0x7fd0e0148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:32.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0dffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0e0064340 0x7fd0e0148650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36906/0 (socket says 172.21.15.36:36906) 2022-06-17T12:54:32.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0e4c37700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0e0155dd0 0x7fd0e00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:32.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0dffff700 1 -- 172.21.15.36:0/2884704037 learned_addr learned my addr 172.21.15.36:0/2884704037 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:32.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0df7fe700 1 --2- 172.21.15.36:0/2884704037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0e00afdd0 0x7fd0e00a3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:32.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0e4c37700 1 -- 172.21.15.36:0/2884704037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0e00afdd0 msgr2=0x7fd0e00a3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:32.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0e4c37700 1 --2- 172.21.15.36:0/2884704037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0e00afdd0 0x7fd0e00a3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:32.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0e4c37700 1 -- 172.21.15.36:0/2884704037 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0e0064340 msgr2=0x7fd0e0148650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:32.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0e4c37700 1 --2- 172.21.15.36:0/2884704037 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0e0064340 0x7fd0e0148650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:32.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0e4c37700 1 -- 172.21.15.36:0/2884704037 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d8009470 con 0x7fd0e0155dd0 2022-06-17T12:54:32.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0dffff700 1 --2- 172.21.15.36:0/2884704037 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0e0064340 0x7fd0e0148650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:32.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0e4c37700 1 --2- 172.21.15.36:0/2884704037 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0e0155dd0 0x7fd0e00a36f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fd0e00ad000 tx=0x7fd0d0007b70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:32.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0dd7fa700 1 -- 172.21.15.36:0/2884704037 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0d000cce0 con 0x7fd0e0155dd0 2022-06-17T12:54:32.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0dd7fa700 1 -- 172.21.15.36:0/2884704037 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d000ce40 con 0x7fd0e0155dd0 2022-06-17T12:54:32.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0e669a700 1 -- 172.21.15.36:0/2884704037 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c800b1e0 con 0x7fd0e0155dd0 2022-06-17T12:54:32.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0dd7fa700 1 -- 172.21.15.36:0/2884704037 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0d0007220 con 0x7fd0e0155dd0 2022-06-17T12:54:32.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.760+0000 7fd0e669a700 1 -- 172.21.15.36:0/2884704037 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e00a3be0 con 0x7fd0e0155dd0 2022-06-17T12:54:32.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.764+0000 7fd0e669a700 1 -- 172.21.15.36:0/2884704037 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e00ae6b0 con 0x7fd0e0155dd0 2022-06-17T12:54:32.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.764+0000 7fd0dd7fa700 1 -- 172.21.15.36:0/2884704037 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd0d0010030 con 0x7fd0e0155dd0 2022-06-17T12:54:32.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.764+0000 7fd0dd7fa700 1 --2- 172.21.15.36:0/2884704037 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd0c4053ad0 0x7fd0c4055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:32.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.764+0000 7fd0dd7fa700 1 -- 172.21.15.36:0/2884704037 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fd0d006b950 con 0x7fd0e0155dd0 2022-06-17T12:54:32.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.764+0000 7fd0dffff700 1 --2- 172.21.15.36:0/2884704037 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd0c4053ad0 0x7fd0c4055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:32.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.768+0000 7fd0dffff700 1 --2- 172.21.15.36:0/2884704037 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd0c4053ad0 0x7fd0c4055f90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7fd0c801c000 tx=0x7fd0c8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:32.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.776+0000 7fd0dd7fa700 1 -- 172.21.15.36:0/2884704037 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd0d0042980 con 0x7fd0e0155dd0 2022-06-17T12:54:32.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.948+0000 7fd0e669a700 1 -- 172.21.15.36:0/2884704037 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd0e0057410 con 0x7fd0e0155dd0 2022-06-17T12:54:32.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.948+0000 7fd0dd7fa700 1 -- 172.21.15.36:0/2884704037 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v30) v1 ==== 80+0+4242 (secure 0 0 0) 0x7fd0d0045fa0 con 0x7fd0e0155dd0 2022-06-17T12:54:32.937 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:32.937 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:54:32.937 INFO:teuthology.orchestra.run.smithi036.stdout: "auth_dump": [ 2022-06-17T12:54:32.937 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:32.938 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mds.a", 2022-06-17T12:54:32.938 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiyaUgARAAxp7bzOl4uVo2EsaoRBeMRw==", 2022-06-17T12:54:32.938 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:32.938 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:54:32.939 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:32.939 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:32.939 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:32.939 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:32.940 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:32.940 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:32.940 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mds.b", 2022-06-17T12:54:32.940 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xifyxLCRAABI389jmz1zboeDanzraT8Q==", 2022-06-17T12:54:32.940 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:32.941 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:54:32.941 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:32.941 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:32.941 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:32.942 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:32.942 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:32.942 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:32.943 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.0", 2022-06-17T12:54:32.943 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB+d6xibhXGNBAA6AuUT9ITWAZjDFpNsgA4Wg==", 2022-06-17T12:54:32.943 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:32.943 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:32.943 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:32.944 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:32.944 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:32.944 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:32.944 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:32.945 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.1", 2022-06-17T12:54:32.945 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCBd6xiOIPvGxAADkrBoJCy9x91zan6TWU0tQ==", 2022-06-17T12:54:32.945 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:32.945 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:32.945 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:32.946 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:32.946 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:32.946 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:32.946 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:32.946 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.2", 2022-06-17T12:54:32.947 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCEd6xiP4/9AhAAaP3cWXV6vbfGBPfqSLKqUg==", 2022-06-17T12:54:32.947 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:32.947 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:32.947 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:32.948 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:32.948 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:32.948 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:32.948 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:32.948 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.3", 2022-06-17T12:54:32.949 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCGd6xi+LZgJxAAJjGwP5qW6TFDZfR+2rz6ig==", 2022-06-17T12:54:32.949 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:32.949 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:32.949 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:32.949 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:32.950 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:32.950 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:32.950 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:32.950 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.4", 2022-06-17T12:54:32.951 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCKd6xikvrTChAAXIyWSETswDv+tb3zsGyYpA==", 2022-06-17T12:54:32.951 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:32.951 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:32.951 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:32.951 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:32.952 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:32.952 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:32.952 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:32.952 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.5", 2022-06-17T12:54:32.952 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCMd6xiqr2QLhAArhimHuNsVWrDGGreFYxj6A==", 2022-06-17T12:54:32.953 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:32.953 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:32.953 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:32.953 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:32.953 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:32.954 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:32.954 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:32.954 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.6", 2022-06-17T12:54:32.954 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCPd6xiUXcXGxAAdrYz8txdBWMIqF5DF4Aimw==", 2022-06-17T12:54:32.955 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:32.955 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:32.955 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:32.955 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:32.955 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:32.956 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:32.956 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:32.956 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.7", 2022-06-17T12:54:32.956 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCSd6xiBJ2IAhAAUkNfgmlxxWY7+VouPLAvyw==", 2022-06-17T12:54:32.956 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:32.957 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:32.957 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:32.957 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:32.957 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:32.958 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:32.958 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:32.958 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.0", 2022-06-17T12:54:32.958 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiXO3QFBAAsSLtQ0yKKZxhqUT/eYKdtg==", 2022-06-17T12:54:32.958 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:32.959 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:54:32.959 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:54:32.959 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:54:32.959 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:32.960 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:32.960 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:32.960 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.1", 2022-06-17T12:54:32.960 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xi51KdGBAAi7fGZ+Qepg6j3DAhlzoWxw==", 2022-06-17T12:54:32.961 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:32.961 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:54:32.961 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:54:32.961 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:54:32.961 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:32.962 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:32.962 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:32.962 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.2", 2022-06-17T12:54:32.962 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiug2hHBAAqHpLgwMdthSWtJ8Vwu+m1Q==", 2022-06-17T12:54:32.963 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:32.963 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:54:32.963 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:54:32.963 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:54:32.963 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:32.964 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:32.964 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:32.964 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.admin", 2022-06-17T12:54:32.964 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB5d6xiGDODKhAAPZMG/M2nVssdr4FJHcS6XQ==", 2022-06-17T12:54:32.964 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:32.965 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:54:32.965 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:32.965 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:32.965 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:32.965 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:32.966 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:32.966 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:32.966 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mgr.x", 2022-06-17T12:54:32.966 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB6d6xiyH9uORAATJb0chHE1CH0X02w7edVtw==", 2022-06-17T12:54:32.967 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:32.967 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:54:32.967 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile mgr", 2022-06-17T12:54:32.967 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:32.967 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:32.968 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:32.968 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:32.968 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mgr.y", 2022-06-17T12:54:32.968 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB6d6xiHXghMxAAzpsVRCbLWHWZPtrPk1jFlA==", 2022-06-17T12:54:32.969 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:32.969 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:54:32.969 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile mgr", 2022-06-17T12:54:32.969 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:32.969 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:32.970 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:32.970 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:32.970 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:54:32.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.952+0000 7fd0e669a700 1 -- 172.21.15.36:0/2884704037 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd0c4053ad0 msgr2=0x7fd0c4055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:32.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.952+0000 7fd0e669a700 1 --2- 172.21.15.36:0/2884704037 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd0c4053ad0 0x7fd0c4055f90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7fd0c801c000 tx=0x7fd0c8015000).stop 2022-06-17T12:54:32.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.952+0000 7fd0e669a700 1 -- 172.21.15.36:0/2884704037 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0e0155dd0 msgr2=0x7fd0e00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:32.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.952+0000 7fd0e669a700 1 --2- 172.21.15.36:0/2884704037 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0e0155dd0 0x7fd0e00a36f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fd0e00ad000 tx=0x7fd0d0007b70).stop 2022-06-17T12:54:32.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.952+0000 7fd0e669a700 1 -- 172.21.15.36:0/2884704037 shutdown_connections 2022-06-17T12:54:32.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.952+0000 7fd0e669a700 1 --2- 172.21.15.36:0/2884704037 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd0c4053ad0 0x7fd0c4055f90 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:32.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.952+0000 7fd0e669a700 1 --2- 172.21.15.36:0/2884704037 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0e0155dd0 0x7fd0e00a36f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:32.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.952+0000 7fd0e669a700 1 --2- 172.21.15.36:0/2884704037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0e00afdd0 0x7fd0e00a3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:32.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.952+0000 7fd0e669a700 1 --2- 172.21.15.36:0/2884704037 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0e0064340 0x7fd0e0148650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:32.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.952+0000 7fd0e669a700 1 -- 172.21.15.36:0/2884704037 >> 172.21.15.36:0/2884704037 conn(0x7fd0e00b5430 msgr2=0x7fd0e0060b10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:32.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.952+0000 7fd0e669a700 1 -- 172.21.15.36:0/2884704037 shutdown_connections 2022-06-17T12:54:32.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:32.952+0000 7fd0e669a700 1 -- 172.21.15.36:0/2884704037 wait complete. 2022-06-17T12:54:32.976 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:54:33.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.096+0000 7fe263087700 1 Processor -- start 2022-06-17T12:54:33.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.096+0000 7fe263087700 1 -- start start 2022-06-17T12:54:33.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.096+0000 7fe263087700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe25c146d00 0x7fe25c147150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:33.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.096+0000 7fe263087700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe25c0b25e0 con 0x7fe25c0b2200 2022-06-17T12:54:33.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.096+0000 7fe263087700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe25c0ad790 con 0x7fe25c0b2cd0 2022-06-17T12:54:33.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.096+0000 7fe263087700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe25c0aef40 con 0x7fe25c146d00 2022-06-17T12:54:33.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.096+0000 7fe262085700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fe25c0b2200 0x7fe25c0af3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54444/0 (socket says 172.21.15.36:54444) 2022-06-17T12:54:33.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.096+0000 7fe262085700 1 -- 172.21.15.36:0/2814548689 learned_addr learned my addr 172.21.15.36:0/2814548689 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:33.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.096+0000 7fe262886700 1 --2- 172.21.15.36:0/2814548689 >> v2:172.21.15.70:3300/0 conn(0x7fe25c146d00 0x7fe25c147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:33.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.096+0000 7fe261083700 1 -- 172.21.15.36:0/2814548689 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3736499455 0 0) 0x7fe25c0b25e0 con 0x7fe25c0b2200 2022-06-17T12:54:33.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.096+0000 7fe261083700 1 -- 172.21.15.36:0/2814548689 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe244003260 con 0x7fe25c0b2200 2022-06-17T12:54:33.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.096+0000 7fe262886700 1 -- 172.21.15.36:0/2814548689 >> v1:172.21.15.70:6789/0 conn(0x7fe25c0b2cd0 legacy=0x7fe25c1465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:33.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.096+0000 7fe262886700 1 -- 172.21.15.36:0/2814548689 >> v1:172.21.15.36:6789/0 conn(0x7fe25c0b2200 legacy=0x7fe25c0af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:33.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.096+0000 7fe262886700 1 -- 172.21.15.36:0/2814548689 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe25c0afaf0 con 0x7fe25c146d00 2022-06-17T12:54:33.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.096+0000 7fe262886700 1 --2- 172.21.15.36:0/2814548689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe25c146d00 0x7fe25c147150 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7fe2540032f0 tx=0x7fe2540091d0).ready entity=mon.1 client_cookie=1f7bc526efaa5438 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:33.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.096+0000 7fe261083700 1 -- 172.21.15.36:0/2814548689 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe25400b6c0 con 0x7fe25c146d00 2022-06-17T12:54:33.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.096+0000 7fe261083700 1 -- 172.21.15.36:0/2814548689 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe25400b820 con 0x7fe25c146d00 2022-06-17T12:54:33.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.096+0000 7fe261083700 1 -- 172.21.15.36:0/2814548689 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe25400b990 con 0x7fe25c146d00 2022-06-17T12:54:33.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe263087700 1 -- 172.21.15.36:0/2814548689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe25c146d00 msgr2=0x7fe25c147150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:33.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe263087700 1 --2- 172.21.15.36:0/2814548689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe25c146d00 0x7fe25c147150 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7fe2540032f0 tx=0x7fe2540091d0).stop 2022-06-17T12:54:33.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe263087700 1 -- 172.21.15.36:0/2814548689 shutdown_connections 2022-06-17T12:54:33.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe263087700 1 --2- 172.21.15.36:0/2814548689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe25c146d00 0x7fe25c147150 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe263087700 1 -- 172.21.15.36:0/2814548689 >> 172.21.15.36:0/2814548689 conn(0x7fe25c05f770 msgr2=0x7fe25c05fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:33.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe263087700 1 -- 172.21.15.36:0/2814548689 shutdown_connections 2022-06-17T12:54:33.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe263087700 1 -- 172.21.15.36:0/2814548689 wait complete. 2022-06-17T12:54:33.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe263087700 1 Processor -- start 2022-06-17T12:54:33.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe263087700 1 -- start start 2022-06-17T12:54:33.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe263087700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe25c0b2200 0x7fe25c1567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:33.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe263087700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe25c0b2cd0 0x7fe25c149090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:33.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe263087700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe25c146d00 0x7fe25c0a9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:33.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe263087700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c0a7af0 con 0x7fe25c0b2cd0 2022-06-17T12:54:33.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe263087700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c0a7d50 con 0x7fe25c146d00 2022-06-17T12:54:33.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe263087700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c0a88e0 con 0x7fe25c0b2200 2022-06-17T12:54:33.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe261884700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe25c0b2cd0 0x7fe25c149090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:33.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe262886700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe25c146d00 0x7fe25c0a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:33.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe261884700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe25c0b2cd0 0x7fe25c149090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36922/0 (socket says 172.21.15.36:36922) 2022-06-17T12:54:33.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe261884700 1 -- 172.21.15.36:0/4167429949 learned_addr learned my addr 172.21.15.36:0/4167429949 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:33.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe262085700 1 --2- 172.21.15.36:0/4167429949 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe25c0b2200 0x7fe25c1567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:33.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe261884700 1 -- 172.21.15.36:0/4167429949 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe25c0b2200 msgr2=0x7fe25c1567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:33.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe261884700 1 --2- 172.21.15.36:0/4167429949 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe25c0b2200 0x7fe25c1567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe261884700 1 -- 172.21.15.36:0/4167429949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe25c146d00 msgr2=0x7fe25c0a9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:33.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe261884700 1 --2- 172.21.15.36:0/4167429949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe25c146d00 0x7fe25c0a9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe261884700 1 -- 172.21.15.36:0/4167429949 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe254009470 con 0x7fe25c0b2cd0 2022-06-17T12:54:33.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.100+0000 7fe262886700 1 --2- 172.21.15.36:0/4167429949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe25c146d00 0x7fe25c0a9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:33.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.104+0000 7fe261884700 1 --2- 172.21.15.36:0/4167429949 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe25c0b2cd0 0x7fe25c149090 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7fe25c06ad60 tx=0x7fe258007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:33.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.104+0000 7fe24b7fe700 1 -- 172.21.15.36:0/4167429949 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe258007030 con 0x7fe25c0b2cd0 2022-06-17T12:54:33.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.104+0000 7fe24b7fe700 1 -- 172.21.15.36:0/4167429949 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe258007190 con 0x7fe25c0b2cd0 2022-06-17T12:54:33.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.104+0000 7fe24b7fe700 1 -- 172.21.15.36:0/4167429949 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe258006510 con 0x7fe25c0b2cd0 2022-06-17T12:54:33.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.104+0000 7fe263087700 1 -- 172.21.15.36:0/4167429949 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe25c0a9360 con 0x7fe25c0b2cd0 2022-06-17T12:54:33.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.104+0000 7fe263087700 1 -- 172.21.15.36:0/4167429949 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe25c0a8540 con 0x7fe25c0b2cd0 2022-06-17T12:54:33.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.104+0000 7fe263087700 1 -- 172.21.15.36:0/4167429949 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe25c0b1950 con 0x7fe25c0b2cd0 2022-06-17T12:54:33.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.104+0000 7fe24b7fe700 1 -- 172.21.15.36:0/4167429949 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe258010030 con 0x7fe25c0b2cd0 2022-06-17T12:54:33.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.104+0000 7fe24b7fe700 1 --2- 172.21.15.36:0/4167429949 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe23c053ad0 0x7fe23c055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:33.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.104+0000 7fe24b7fe700 1 -- 172.21.15.36:0/4167429949 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fe25806b7d0 con 0x7fe25c0b2cd0 2022-06-17T12:54:33.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.104+0000 7fe262085700 1 --2- 172.21.15.36:0/4167429949 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe23c053ad0 0x7fe23c055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:33.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.108+0000 7fe262085700 1 --2- 172.21.15.36:0/4167429949 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe23c053ad0 0x7fe23c055f90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7fe24c00d000 tx=0x7fe24c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:33.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.112+0000 7fe24b7fe700 1 -- 172.21.15.36:0/4167429949 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe258042910 con 0x7fe25c0b2cd0 2022-06-17T12:54:33.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.236+0000 7fe263087700 1 -- 172.21.15.36:0/4167429949 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe25c057410 con 0x7fe25c0b2cd0 2022-06-17T12:54:33.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.236+0000 7fe24b7fe700 1 -- 172.21.15.36:0/4167429949 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+13415 (secure 0 0 0) 0x7fe258045f30 con 0x7fe25c0b2cd0 2022-06-17T12:54:33.228 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:33.228 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":89,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:54:32.571420+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:54:33.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.240+0000 7fe263087700 1 -- 172.21.15.36:0/4167429949 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe23c053ad0 msgr2=0x7fe23c055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:33.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.240+0000 7fe263087700 1 --2- 172.21.15.36:0/4167429949 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe23c053ad0 0x7fe23c055f90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7fe24c00d000 tx=0x7fe24c009040).stop 2022-06-17T12:54:33.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.240+0000 7fe263087700 1 -- 172.21.15.36:0/4167429949 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe25c0b2cd0 msgr2=0x7fe25c149090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:33.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.240+0000 7fe263087700 1 --2- 172.21.15.36:0/4167429949 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe25c0b2cd0 0x7fe25c149090 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7fe25c06ad60 tx=0x7fe258007c10).stop 2022-06-17T12:54:33.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.240+0000 7fe263087700 1 -- 172.21.15.36:0/4167429949 shutdown_connections 2022-06-17T12:54:33.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.240+0000 7fe263087700 1 --2- 172.21.15.36:0/4167429949 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe23c053ad0 0x7fe23c055f90 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.240+0000 7fe263087700 1 --2- 172.21.15.36:0/4167429949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe25c146d00 0x7fe25c0a9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.240+0000 7fe263087700 1 --2- 172.21.15.36:0/4167429949 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe25c0b2cd0 0x7fe25c149090 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.240+0000 7fe263087700 1 --2- 172.21.15.36:0/4167429949 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe25c0b2200 0x7fe25c1567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.240+0000 7fe263087700 1 -- 172.21.15.36:0/4167429949 >> 172.21.15.36:0/4167429949 conn(0x7fe25c05f770 msgr2=0x7fe25c0abaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:33.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.240+0000 7fe263087700 1 -- 172.21.15.36:0/4167429949 shutdown_connections 2022-06-17T12:54:33.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.240+0000 7fe263087700 1 -- 172.21.15.36:0/4167429949 wait complete. 2022-06-17T12:54:33.239 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-17T12:54:33.241 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:54:33.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.364+0000 7f0db096c700 1 Processor -- start 2022-06-17T12:54:33.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.364+0000 7f0db096c700 1 -- start start 2022-06-17T12:54:33.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.364+0000 7f0db096c700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0dac0b45c0 0x7f0dac0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:33.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.364+0000 7f0db096c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0dac0b4f00 0x7f0dac0b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:33.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.364+0000 7f0db096c700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dac0b4160 con 0x7f0dac0b9860 2022-06-17T12:54:33.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.364+0000 7f0db096c700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dac0b9c40 con 0x7f0dac0b45c0 2022-06-17T12:54:33.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.364+0000 7f0db096c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac0627d0 con 0x7f0dac0b4f00 2022-06-17T12:54:33.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.364+0000 7f0dab59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0dac0b4f00 0x7f0dac0b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:33.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.364+0000 7f0daa59c700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0dac0b45c0 0x7f0dac0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:33.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.364+0000 7f0daa59c700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0dac0b45c0 0x7f0dac0b4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:57310/0 (socket says 172.21.15.36:57310) 2022-06-17T12:54:33.366 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.364+0000 7f0dab59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0dac0b4f00 0x7f0dac0b5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:51720/0 (socket says 172.21.15.36:51720) 2022-06-17T12:54:33.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.364+0000 7f0daad9d700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0dac0b9860 0x7f0dac143840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:50190/0 (socket says 172.21.15.36:50190) 2022-06-17T12:54:33.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.364+0000 7f0daa59c700 1 -- 172.21.15.36:0/274446831 learned_addr learned my addr 172.21.15.36:0/274446831 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:33.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.368+0000 7f0daa59c700 1 -- 172.21.15.36:0/274446831 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0dac0b4f00 msgr2=0x7f0dac0b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:33.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.368+0000 7f0daa59c700 1 --2- 172.21.15.36:0/274446831 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0dac0b4f00 0x7f0dac0b5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.368+0000 7f0daa59c700 1 -- 172.21.15.36:0/274446831 >> v1:172.21.15.70:6789/0 conn(0x7f0dac0b9860 legacy=0x7f0dac143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:33.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.368+0000 7f0daa59c700 1 -- 172.21.15.36:0/274446831 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dac062a70 con 0x7f0dac0b45c0 2022-06-17T12:54:33.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.368+0000 7f0dab59e700 1 --2- 172.21.15.36:0/274446831 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0dac0b4f00 0x7f0dac0b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:33.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.368+0000 7f0daa59c700 1 --2- 172.21.15.36:0/274446831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dac0b45c0 0x7f0dac0b4a10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f0d94002f70 tx=0x7f0d94006eb0).ready entity=mon.1 client_cookie=8ec7b277760f1b0b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:33.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.368+0000 7f0da9d9b700 1 -- 172.21.15.36:0/274446831 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d9400d570 con 0x7f0dac0b45c0 2022-06-17T12:54:33.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.368+0000 7f0da9d9b700 1 -- 172.21.15.36:0/274446831 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d94000e90 con 0x7f0dac0b45c0 2022-06-17T12:54:33.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.368+0000 7f0da9d9b700 1 -- 172.21.15.36:0/274446831 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d9400d900 con 0x7f0dac0b45c0 2022-06-17T12:54:33.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.368+0000 7f0db096c700 1 -- 172.21.15.36:0/274446831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dac0b45c0 msgr2=0x7f0dac0b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:33.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.368+0000 7f0db096c700 1 --2- 172.21.15.36:0/274446831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dac0b45c0 0x7f0dac0b4a10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f0d94002f70 tx=0x7f0d94006eb0).stop 2022-06-17T12:54:33.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.368+0000 7f0db096c700 1 -- 172.21.15.36:0/274446831 shutdown_connections 2022-06-17T12:54:33.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.368+0000 7f0db096c700 1 --2- 172.21.15.36:0/274446831 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0dac0b4f00 0x7f0dac0b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.368+0000 7f0db096c700 1 --2- 172.21.15.36:0/274446831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dac0b45c0 0x7f0dac0b4a10 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.368+0000 7f0db096c700 1 -- 172.21.15.36:0/274446831 >> 172.21.15.36:0/274446831 conn(0x7f0dac05c550 msgr2=0x7f0dac060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:33.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.368+0000 7f0db096c700 1 -- 172.21.15.36:0/274446831 shutdown_connections 2022-06-17T12:54:33.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.368+0000 7f0db096c700 1 -- 172.21.15.36:0/274446831 wait complete. 2022-06-17T12:54:33.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.368+0000 7f0db096c700 1 Processor -- start 2022-06-17T12:54:33.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.368+0000 7f0db096c700 1 -- start start 2022-06-17T12:54:33.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.372+0000 7f0db096c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0dac0b45c0 0x7f0dac0ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:33.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.372+0000 7f0db096c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0dac0b4f00 0x7f0dac13dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:33.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.372+0000 7f0db096c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dac0b9860 0x7f0dac13e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:33.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.372+0000 7f0db096c700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac0af5b0 con 0x7f0dac0b4f00 2022-06-17T12:54:33.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.372+0000 7f0db096c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac0afdf0 con 0x7f0dac0b9860 2022-06-17T12:54:33.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.372+0000 7f0db096c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac0aeab0 con 0x7f0dac0b45c0 2022-06-17T12:54:33.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.372+0000 7f0daa59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0dac0b4f00 0x7f0dac13dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:33.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.372+0000 7f0daa59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0dac0b4f00 0x7f0dac13dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36934/0 (socket says 172.21.15.36:36934) 2022-06-17T12:54:33.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.372+0000 7f0daa59c700 1 -- 172.21.15.36:0/875218796 learned_addr learned my addr 172.21.15.36:0/875218796 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:33.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.372+0000 7f0daad9d700 1 --2- 172.21.15.36:0/875218796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0dac0b45c0 0x7f0dac0ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:33.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.372+0000 7f0dab59e700 1 --2- 172.21.15.36:0/875218796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dac0b9860 0x7f0dac13e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:33.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.372+0000 7f0daa59c700 1 -- 172.21.15.36:0/875218796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0dac0b45c0 msgr2=0x7f0dac0ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:33.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.372+0000 7f0daa59c700 1 --2- 172.21.15.36:0/875218796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0dac0b45c0 0x7f0dac0ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.372+0000 7f0daa59c700 1 -- 172.21.15.36:0/875218796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dac0b9860 msgr2=0x7f0dac13e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:33.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.372+0000 7f0daa59c700 1 --2- 172.21.15.36:0/875218796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dac0b9860 0x7f0dac13e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.372+0000 7f0daa59c700 1 -- 172.21.15.36:0/875218796 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da4009470 con 0x7f0dac0b4f00 2022-06-17T12:54:33.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.372+0000 7f0daa59c700 1 --2- 172.21.15.36:0/875218796 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0dac0b4f00 0x7f0dac13dd50 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7f0d94002f70 tx=0x7f0d94015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:33.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.372+0000 7f0da3fff700 1 -- 172.21.15.36:0/875218796 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d940068b0 con 0x7f0dac0b4f00 2022-06-17T12:54:33.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.372+0000 7f0db096c700 1 -- 172.21.15.36:0/875218796 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d9400b1e0 con 0x7f0dac0b4f00 2022-06-17T12:54:33.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.372+0000 7f0db096c700 1 -- 172.21.15.36:0/875218796 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dac0ae380 con 0x7f0dac0b4f00 2022-06-17T12:54:33.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.372+0000 7f0da3fff700 1 -- 172.21.15.36:0/875218796 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9400de30 con 0x7f0dac0b4f00 2022-06-17T12:54:33.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.372+0000 7f0da3fff700 1 -- 172.21.15.36:0/875218796 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d94011d70 con 0x7f0dac0b4f00 2022-06-17T12:54:33.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.372+0000 7f0db096c700 1 -- 172.21.15.36:0/875218796 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dac062eb0 con 0x7f0dac0b4f00 2022-06-17T12:54:33.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.376+0000 7f0da3fff700 1 -- 172.21.15.36:0/875218796 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f0d9401a040 con 0x7f0dac0b4f00 2022-06-17T12:54:33.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.376+0000 7f0da3fff700 1 --2- 172.21.15.36:0/875218796 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0d90053ad0 0x7f0d90055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:33.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.376+0000 7f0da3fff700 1 -- 172.21.15.36:0/875218796 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f0d9406cb50 con 0x7f0dac0b4f00 2022-06-17T12:54:33.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.376+0000 7f0daad9d700 1 --2- 172.21.15.36:0/875218796 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0d90053ad0 0x7f0d90055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:33.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.376+0000 7f0daad9d700 1 --2- 172.21.15.36:0/875218796 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0d90053ad0 0x7f0d90055f90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f0d9c00d000 tx=0x7f0d9c000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:33.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.384+0000 7f0da3fff700 1 -- 172.21.15.36:0/875218796 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0d94043c30 con 0x7f0dac0b4f00 2022-06-17T12:54:33.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.504+0000 7f0db096c700 1 -- 172.21.15.36:0/875218796 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0dac060e30 con 0x7f0dac0b4f00 2022-06-17T12:54:33.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.504+0000 7f0da3fff700 1 -- 172.21.15.36:0/875218796 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v89) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0d94023020 con 0x7f0dac0b4f00 2022-06-17T12:54:33.494 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:54:33.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.508+0000 7f0db096c700 1 -- 172.21.15.36:0/875218796 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0d90053ad0 msgr2=0x7f0d90055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:33.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.508+0000 7f0db096c700 1 --2- 172.21.15.36:0/875218796 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0d90053ad0 0x7f0d90055f90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f0d9c00d000 tx=0x7f0d9c000bc0).stop 2022-06-17T12:54:33.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.508+0000 7f0db096c700 1 -- 172.21.15.36:0/875218796 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0dac0b4f00 msgr2=0x7f0dac13dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:33.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.508+0000 7f0db096c700 1 --2- 172.21.15.36:0/875218796 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0dac0b4f00 0x7f0dac13dd50 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7f0d94002f70 tx=0x7f0d94015000).stop 2022-06-17T12:54:33.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.508+0000 7f0db096c700 1 -- 172.21.15.36:0/875218796 shutdown_connections 2022-06-17T12:54:33.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.508+0000 7f0db096c700 1 --2- 172.21.15.36:0/875218796 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0d90053ad0 0x7f0d90055f90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.508+0000 7f0db096c700 1 --2- 172.21.15.36:0/875218796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dac0b9860 0x7f0dac13e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.508+0000 7f0db096c700 1 --2- 172.21.15.36:0/875218796 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0dac0b4f00 0x7f0dac13dd50 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.508+0000 7f0db096c700 1 --2- 172.21.15.36:0/875218796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0dac0b45c0 0x7f0dac0ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.508+0000 7f0db096c700 1 -- 172.21.15.36:0/875218796 >> 172.21.15.36:0/875218796 conn(0x7f0dac05c550 msgr2=0x7f0dac0b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:33.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.508+0000 7f0db096c700 1 -- 172.21.15.36:0/875218796 shutdown_connections 2022-06-17T12:54:33.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.508+0000 7f0db096c700 1 -- 172.21.15.36:0/875218796 wait complete. 2022-06-17T12:54:33.507 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:54:33.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.632+0000 7f049359e700 1 Processor -- start 2022-06-17T12:54:33.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.632+0000 7f049359e700 1 -- start start 2022-06-17T12:54:33.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.632+0000 7f049359e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f049405fe80 0x7f0494060260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:33.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.632+0000 7f049359e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f04940ae360 0x7f04941444f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:33.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.632+0000 7f049359e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f04940b1db0 0x7f04941437d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:33.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.632+0000 7f049359e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04940633c0 con 0x7f04940ae360 2022-06-17T12:54:33.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.632+0000 7f049359e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0494143330 con 0x7f04940b1db0 2022-06-17T12:54:33.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.632+0000 7f049359e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0494063100 con 0x7f049405fe80 2022-06-17T12:54:33.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.632+0000 7f0489d9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f04940ae360 0x7f04941444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:33.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.636+0000 7f0489d9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f04940ae360 0x7f04941444f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36942/0 (socket says 172.21.15.36:36942) 2022-06-17T12:54:33.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.636+0000 7f0489d9b700 1 -- 172.21.15.36:0/1367502185 learned_addr learned my addr 172.21.15.36:0/1367502185 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:33.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.636+0000 7f0492d9d700 1 --2- 172.21.15.36:0/1367502185 >> v2:172.21.15.70:3300/0 conn(0x7f04940b1db0 0x7f04941437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:33.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.636+0000 7f049259c700 1 --2- 172.21.15.36:0/1367502185 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f049405fe80 0x7f0494060260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:33.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.636+0000 7f0489d9b700 1 -- 172.21.15.36:0/1367502185 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f049405fe80 msgr2=0x7f0494060260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:33.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.636+0000 7f0489d9b700 1 --2- 172.21.15.36:0/1367502185 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f049405fe80 0x7f0494060260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.636+0000 7f0489d9b700 1 -- 172.21.15.36:0/1367502185 >> v2:172.21.15.70:3300/0 conn(0x7f04940b1db0 msgr2=0x7f04941437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:33.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.636+0000 7f0489d9b700 1 --2- 172.21.15.36:0/1367502185 >> v2:172.21.15.70:3300/0 conn(0x7f04940b1db0 0x7f04941437d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.636+0000 7f0489d9b700 1 -- 172.21.15.36:0/1367502185 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0494143020 con 0x7f04940ae360 2022-06-17T12:54:33.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.636+0000 7f0489d9b700 1 --2- 172.21.15.36:0/1367502185 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04940ae360 0x7f04941444f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7f047c002f70 tx=0x7f047c006eb0).ready entity=mon.0 client_cookie=bf4e8a59fd82dd70 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:33.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.636+0000 7f0491d9b700 1 -- 172.21.15.36:0/1367502185 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f047c00d570 con 0x7f04940ae360 2022-06-17T12:54:33.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.636+0000 7f0491d9b700 1 -- 172.21.15.36:0/1367502185 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047c000e90 con 0x7f04940ae360 2022-06-17T12:54:33.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.636+0000 7f0491d9b700 1 -- 172.21.15.36:0/1367502185 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f047c00d900 con 0x7f04940ae360 2022-06-17T12:54:33.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.636+0000 7f049359e700 1 -- 172.21.15.36:0/1367502185 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04940ae360 msgr2=0x7f04941444f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:33.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.636+0000 7f049359e700 1 --2- 172.21.15.36:0/1367502185 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04940ae360 0x7f04941444f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7f047c002f70 tx=0x7f047c006eb0).stop 2022-06-17T12:54:33.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.636+0000 7f049359e700 1 -- 172.21.15.36:0/1367502185 shutdown_connections 2022-06-17T12:54:33.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.636+0000 7f049359e700 1 --2- 172.21.15.36:0/1367502185 >> v2:172.21.15.70:3300/0 conn(0x7f04940b1db0 0x7f04941437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.636+0000 7f049359e700 1 --2- 172.21.15.36:0/1367502185 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04940ae360 0x7f04941444f0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.636+0000 7f049359e700 1 --2- 172.21.15.36:0/1367502185 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f049405fe80 0x7f0494060260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.636+0000 7f049359e700 1 -- 172.21.15.36:0/1367502185 >> 172.21.15.36:0/1367502185 conn(0x7f0494060990 msgr2=0x7f049405f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:33.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.636+0000 7f049359e700 1 -- 172.21.15.36:0/1367502185 shutdown_connections 2022-06-17T12:54:33.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.636+0000 7f049359e700 1 -- 172.21.15.36:0/1367502185 wait complete. 2022-06-17T12:54:33.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.636+0000 7f049359e700 1 Processor -- start 2022-06-17T12:54:33.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.636+0000 7f049359e700 1 -- start start 2022-06-17T12:54:33.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.640+0000 7f049359e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f049405fe80 0x7f0494153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:33.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.640+0000 7f049359e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04940ae360 0x7f04941484c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:33.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.640+0000 7f049359e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04940b1db0 0x7f0494147870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:33.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.640+0000 7f049359e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04941494f0 con 0x7f04940b1db0 2022-06-17T12:54:33.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.640+0000 7f049359e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049414a9c0 con 0x7f049405fe80 2022-06-17T12:54:33.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.640+0000 7f049359e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0494147600 con 0x7f04940ae360 2022-06-17T12:54:33.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.640+0000 7f0492d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04940b1db0 0x7f0494147870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:33.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.640+0000 7f0492d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04940b1db0 0x7f0494147870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36950/0 (socket says 172.21.15.36:36950) 2022-06-17T12:54:33.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.640+0000 7f0492d9d700 1 -- 172.21.15.36:0/3266027199 learned_addr learned my addr 172.21.15.36:0/3266027199 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:33.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.640+0000 7f049259c700 1 --2- 172.21.15.36:0/3266027199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f049405fe80 0x7f0494153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:33.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.640+0000 7f0489d9b700 1 --2- 172.21.15.36:0/3266027199 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04940ae360 0x7f04941484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:33.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.640+0000 7f0492d9d700 1 -- 172.21.15.36:0/3266027199 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04940ae360 msgr2=0x7f04941484c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:33.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.640+0000 7f0492d9d700 1 --2- 172.21.15.36:0/3266027199 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04940ae360 0x7f04941484c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.640+0000 7f0492d9d700 1 -- 172.21.15.36:0/3266027199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f049405fe80 msgr2=0x7f0494153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:33.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.640+0000 7f0492d9d700 1 --2- 172.21.15.36:0/3266027199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f049405fe80 0x7f0494153490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.640+0000 7f0492d9d700 1 -- 172.21.15.36:0/3266027199 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c00b1e0 con 0x7f04940b1db0 2022-06-17T12:54:33.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.640+0000 7f049259c700 1 --2- 172.21.15.36:0/3266027199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f049405fe80 0x7f0494153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:33.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.640+0000 7f0492d9d700 1 --2- 172.21.15.36:0/3266027199 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04940b1db0 0x7f0494147870 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7f0494147790 tx=0x7f048c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:33.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.640+0000 7f048bfff700 1 -- 172.21.15.36:0/3266027199 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f048c012600 con 0x7f04940b1db0 2022-06-17T12:54:33.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.640+0000 7f048bfff700 1 -- 172.21.15.36:0/3266027199 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048c012760 con 0x7f04940b1db0 2022-06-17T12:54:33.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.640+0000 7f048bfff700 1 -- 172.21.15.36:0/3266027199 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f048c012ac0 con 0x7f04940b1db0 2022-06-17T12:54:33.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.640+0000 7f049359e700 1 -- 172.21.15.36:0/3266027199 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0494153980 con 0x7f04940b1db0 2022-06-17T12:54:33.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.640+0000 7f049359e700 1 -- 172.21.15.36:0/3266027199 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0494146660 con 0x7f04940b1db0 2022-06-17T12:54:33.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.640+0000 7f049359e700 1 -- 172.21.15.36:0/3266027199 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04941449e0 con 0x7f04940b1db0 2022-06-17T12:54:33.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.644+0000 7f048bfff700 1 -- 172.21.15.36:0/3266027199 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f048c022070 con 0x7f04940b1db0 2022-06-17T12:54:33.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.644+0000 7f048bfff700 1 --2- 172.21.15.36:0/3266027199 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0478053a60 0x7f0478055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:33.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.644+0000 7f048bfff700 1 -- 172.21.15.36:0/3266027199 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f048c06c2a0 con 0x7f04940b1db0 2022-06-17T12:54:33.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.644+0000 7f049259c700 1 --2- 172.21.15.36:0/3266027199 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0478053a60 0x7f0478055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:33.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.644+0000 7f049259c700 1 --2- 172.21.15.36:0/3266027199 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0478053a60 0x7f0478055f20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f048400c000 tx=0x7f0484008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:33.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.652+0000 7f048bfff700 1 -- 172.21.15.36:0/3266027199 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f048c0432d0 con 0x7f04940b1db0 2022-06-17T12:54:33.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.780+0000 7f049359e700 1 -- 172.21.15.36:0/3266027199 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0494057410 con 0x7f04940b1db0 2022-06-17T12:54:33.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.780+0000 7f048bfff700 1 -- 172.21.15.36:0/3266027199 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v89) v1 ==== 92+0+10 (secure 0 0 0) 0x7f048c0160a0 con 0x7f04940b1db0 2022-06-17T12:54:33.773 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:54:33.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.780+0000 7f049359e700 1 -- 172.21.15.36:0/3266027199 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0478053a60 msgr2=0x7f0478055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:33.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.780+0000 7f049359e700 1 --2- 172.21.15.36:0/3266027199 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0478053a60 0x7f0478055f20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f048400c000 tx=0x7f0484008040).stop 2022-06-17T12:54:33.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.780+0000 7f049359e700 1 -- 172.21.15.36:0/3266027199 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04940b1db0 msgr2=0x7f0494147870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:33.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.780+0000 7f049359e700 1 --2- 172.21.15.36:0/3266027199 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04940b1db0 0x7f0494147870 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7f0494147790 tx=0x7f048c0091d0).stop 2022-06-17T12:54:33.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.780+0000 7f049359e700 1 -- 172.21.15.36:0/3266027199 shutdown_connections 2022-06-17T12:54:33.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.780+0000 7f049359e700 1 --2- 172.21.15.36:0/3266027199 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0478053a60 0x7f0478055f20 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.780+0000 7f049359e700 1 --2- 172.21.15.36:0/3266027199 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04940b1db0 0x7f0494147870 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.780+0000 7f049359e700 1 --2- 172.21.15.36:0/3266027199 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04940ae360 0x7f04941484c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.780+0000 7f049359e700 1 --2- 172.21.15.36:0/3266027199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f049405fe80 0x7f0494153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.780+0000 7f049359e700 1 -- 172.21.15.36:0/3266027199 >> 172.21.15.36:0/3266027199 conn(0x7f0494060990 msgr2=0x7f04940b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:33.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.780+0000 7f049359e700 1 -- 172.21.15.36:0/3266027199 shutdown_connections 2022-06-17T12:54:33.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.784+0000 7f049359e700 1 -- 172.21.15.36:0/3266027199 wait complete. 2022-06-17T12:54:33.781 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-17T12:54:33.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.900+0000 7fd0bffff700 1 Processor -- start 2022-06-17T12:54:33.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.900+0000 7fd0bffff700 1 -- start start 2022-06-17T12:54:33.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.900+0000 7fd0bffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd0c00b2200 0x7fd0c00af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:33.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.900+0000 7fd0bffff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd0c00b2cd0 0x7fd0c0147600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:33.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.900+0000 7fd0bffff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c00b25e0 con 0x7fd0c00af8d0 2022-06-17T12:54:33.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.900+0000 7fd0bffff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0c00ad790 con 0x7fd0c00b2200 2022-06-17T12:54:33.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.900+0000 7fd0bffff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0c00aef40 con 0x7fd0c00b2cd0 2022-06-17T12:54:33.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.900+0000 7fd0beffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd0c00b2200 0x7fd0c00af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:33.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.900+0000 7fd0beffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd0c00b2200 0x7fd0c00af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36954/0 (socket says 172.21.15.36:36954) 2022-06-17T12:54:33.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.900+0000 7fd0beffd700 1 -- 172.21.15.36:0/734131875 learned_addr learned my addr 172.21.15.36:0/734131875 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:33.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.900+0000 7fd0bf7fe700 1 --2- 172.21.15.36:0/734131875 >> v2:172.21.15.70:3300/0 conn(0x7fd0c00b2cd0 0x7fd0c0147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:33.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.900+0000 7fd0beffd700 1 -- 172.21.15.36:0/734131875 >> v2:172.21.15.70:3300/0 conn(0x7fd0c00b2cd0 msgr2=0x7fd0c0147600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:33.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.900+0000 7fd0beffd700 1 --2- 172.21.15.36:0/734131875 >> v2:172.21.15.70:3300/0 conn(0x7fd0c00b2cd0 0x7fd0c0147600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.900+0000 7fd0beffd700 1 -- 172.21.15.36:0/734131875 >> v1:172.21.15.70:6789/0 conn(0x7fd0c00af8d0 legacy=0x7fd0c014a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:33.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.900+0000 7fd0beffd700 1 -- 172.21.15.36:0/734131875 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c00b1950 con 0x7fd0c00b2200 2022-06-17T12:54:33.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.900+0000 7fd0beffd700 1 --2- 172.21.15.36:0/734131875 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0c00b2200 0x7fd0c00af3e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7fd0b80032f0 tx=0x7fd0b8009290).ready entity=mon.0 client_cookie=ffc5ffd0c127bde9 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:33.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.900+0000 7fd0bdffb700 1 -- 172.21.15.36:0/734131875 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0b800b870 con 0x7fd0c00b2200 2022-06-17T12:54:33.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.904+0000 7fd0bdffb700 1 -- 172.21.15.36:0/734131875 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b800b9d0 con 0x7fd0c00b2200 2022-06-17T12:54:33.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.904+0000 7fd0bdffb700 1 -- 172.21.15.36:0/734131875 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0b800bb40 con 0x7fd0c00b2200 2022-06-17T12:54:33.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.904+0000 7fd0bffff700 1 -- 172.21.15.36:0/734131875 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0c00b2200 msgr2=0x7fd0c00af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:33.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.904+0000 7fd0bffff700 1 --2- 172.21.15.36:0/734131875 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0c00b2200 0x7fd0c00af3e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7fd0b80032f0 tx=0x7fd0b8009290).stop 2022-06-17T12:54:33.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.904+0000 7fd0bffff700 1 -- 172.21.15.36:0/734131875 shutdown_connections 2022-06-17T12:54:33.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.904+0000 7fd0bffff700 1 --2- 172.21.15.36:0/734131875 >> v2:172.21.15.70:3300/0 conn(0x7fd0c00b2cd0 0x7fd0c0147600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.904+0000 7fd0bffff700 1 --2- 172.21.15.36:0/734131875 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0c00b2200 0x7fd0c00af3e0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.904+0000 7fd0bffff700 1 -- 172.21.15.36:0/734131875 >> 172.21.15.36:0/734131875 conn(0x7fd0c005f770 msgr2=0x7fd0c005fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:33.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.904+0000 7fd0bffff700 1 -- 172.21.15.36:0/734131875 shutdown_connections 2022-06-17T12:54:33.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.904+0000 7fd0bffff700 1 -- 172.21.15.36:0/734131875 wait complete. 2022-06-17T12:54:33.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.904+0000 7fd0bffff700 1 Processor -- start 2022-06-17T12:54:33.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.904+0000 7fd0bffff700 1 -- start start 2022-06-17T12:54:33.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.908+0000 7fd0bffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0c00af8d0 0x7fd0c014d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:33.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.908+0000 7fd0bffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0c00b2200 0x7fd0c00aa560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:33.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.908+0000 7fd0bffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0c00b2cd0 0x7fd0c00aaa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:33.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.908+0000 7fd0bffff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c00aa100 con 0x7fd0c00b2cd0 2022-06-17T12:54:33.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.908+0000 7fd0bffff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c0062c10 con 0x7fd0c00b2200 2022-06-17T12:54:33.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.908+0000 7fd0bffff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c0062970 con 0x7fd0c00af8d0 2022-06-17T12:54:33.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.908+0000 7fd0bf7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0c00b2cd0 0x7fd0c00aaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:33.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.908+0000 7fd0bf7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0c00b2cd0 0x7fd0c00aaa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36962/0 (socket says 172.21.15.36:36962) 2022-06-17T12:54:33.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.908+0000 7fd0bf7fe700 1 -- 172.21.15.36:0/1307686614 learned_addr learned my addr 172.21.15.36:0/1307686614 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:33.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.908+0000 7fd0beffd700 1 --2- 172.21.15.36:0/1307686614 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0c00af8d0 0x7fd0c014d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:33.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.908+0000 7fd0be7fc700 1 --2- 172.21.15.36:0/1307686614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0c00b2200 0x7fd0c00aa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:33.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.908+0000 7fd0bf7fe700 1 -- 172.21.15.36:0/1307686614 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0c00af8d0 msgr2=0x7fd0c014d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:33.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.908+0000 7fd0bf7fe700 1 --2- 172.21.15.36:0/1307686614 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0c00af8d0 0x7fd0c014d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.908+0000 7fd0bf7fe700 1 -- 172.21.15.36:0/1307686614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0c00b2200 msgr2=0x7fd0c00aa560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:33.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.908+0000 7fd0bf7fe700 1 --2- 172.21.15.36:0/1307686614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0c00b2200 0x7fd0c00aa560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:33.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.908+0000 7fd0bf7fe700 1 -- 172.21.15.36:0/1307686614 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b8009530 con 0x7fd0c00b2cd0 2022-06-17T12:54:33.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.908+0000 7fd0bf7fe700 1 --2- 172.21.15.36:0/1307686614 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0c00b2cd0 0x7fd0c00aaa50 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7fd0b000c000 tx=0x7fd0b0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:33.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.908+0000 7fd0b7fff700 1 -- 172.21.15.36:0/1307686614 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0b0011750 con 0x7fd0c00b2cd0 2022-06-17T12:54:33.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.908+0000 7fd0b7fff700 1 -- 172.21.15.36:0/1307686614 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b00118b0 con 0x7fd0c00b2cd0 2022-06-17T12:54:33.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.908+0000 7fd0bffff700 1 -- 172.21.15.36:0/1307686614 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c00a8c40 con 0x7fd0c00b2cd0 2022-06-17T12:54:33.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.908+0000 7fd0b7fff700 1 -- 172.21.15.36:0/1307686614 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0b0011c10 con 0x7fd0c00b2cd0 2022-06-17T12:54:33.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.908+0000 7fd0bffff700 1 -- 172.21.15.36:0/1307686614 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c0067400 con 0x7fd0c00b2cd0 2022-06-17T12:54:33.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.908+0000 7fd0b7fff700 1 -- 172.21.15.36:0/1307686614 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd0b0011d70 con 0x7fd0c00b2cd0 2022-06-17T12:54:33.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.912+0000 7fd0bffff700 1 -- 172.21.15.36:0/1307686614 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c00ad330 con 0x7fd0c00b2cd0 2022-06-17T12:54:33.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.912+0000 7fd0b7fff700 1 --2- 172.21.15.36:0/1307686614 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd0a4053a60 0x7fd0a4055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:33.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.912+0000 7fd0beffd700 1 --2- 172.21.15.36:0/1307686614 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd0a4053a60 0x7fd0a4055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:33.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.912+0000 7fd0b7fff700 1 -- 172.21.15.36:0/1307686614 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fd0b006bf30 con 0x7fd0c00b2cd0 2022-06-17T12:54:33.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.912+0000 7fd0beffd700 1 --2- 172.21.15.36:0/1307686614 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd0a4053a60 0x7fd0a4055f20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fd0b801b000 tx=0x7fd0b8016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:33.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:33.920+0000 7fd0b7fff700 1 -- 172.21.15.36:0/1307686614 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd0b0043070 con 0x7fd0c00b2cd0 2022-06-17T12:54:34.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.048+0000 7fd0bffff700 1 -- 172.21.15.36:0/1307686614 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd0c0057410 con 0x7fd0c00b2cd0 2022-06-17T12:54:34.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.048+0000 7fd0b7fff700 1 -- 172.21.15.36:0/1307686614 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v89) v1 ==== 84+0+143 (secure 0 0 0) 0x7fd0b0046690 con 0x7fd0c00b2cd0 2022-06-17T12:54:34.038 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:34.038 INFO:teuthology.orchestra.run.smithi036.stdout:[ 2022-06-17T12:54:34.038 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:34.038 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 1, 2022-06-17T12:54:34.039 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "device_health_metrics" 2022-06-17T12:54:34.039 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:34.039 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:34.039 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 2, 2022-06-17T12:54:34.040 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "rbd" 2022-06-17T12:54:34.041 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:34.041 INFO:teuthology.orchestra.run.smithi036.stdout:] 2022-06-17T12:54:34.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.052+0000 7fd0bffff700 1 -- 172.21.15.36:0/1307686614 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd0a4053a60 msgr2=0x7fd0a4055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:34.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.052+0000 7fd0bffff700 1 --2- 172.21.15.36:0/1307686614 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd0a4053a60 0x7fd0a4055f20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fd0b801b000 tx=0x7fd0b8016000).stop 2022-06-17T12:54:34.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.052+0000 7fd0bffff700 1 -- 172.21.15.36:0/1307686614 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0c00b2cd0 msgr2=0x7fd0c00aaa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:34.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.052+0000 7fd0bffff700 1 --2- 172.21.15.36:0/1307686614 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0c00b2cd0 0x7fd0c00aaa50 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7fd0b000c000 tx=0x7fd0b0006df0).stop 2022-06-17T12:54:34.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.052+0000 7fd0bffff700 1 -- 172.21.15.36:0/1307686614 shutdown_connections 2022-06-17T12:54:34.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.052+0000 7fd0bffff700 1 --2- 172.21.15.36:0/1307686614 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd0a4053a60 0x7fd0a4055f20 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:34.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.052+0000 7fd0bffff700 1 --2- 172.21.15.36:0/1307686614 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd0c00b2cd0 0x7fd0c00aaa50 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:34.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.052+0000 7fd0bffff700 1 --2- 172.21.15.36:0/1307686614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0c00b2200 0x7fd0c00aa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:34.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.052+0000 7fd0bffff700 1 --2- 172.21.15.36:0/1307686614 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0c00af8d0 0x7fd0c014d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:34.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.052+0000 7fd0bffff700 1 -- 172.21.15.36:0/1307686614 >> 172.21.15.36:0/1307686614 conn(0x7fd0c005f770 msgr2=0x7fd0c00b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:34.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.052+0000 7fd0bffff700 1 -- 172.21.15.36:0/1307686614 shutdown_connections 2022-06-17T12:54:34.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.052+0000 7fd0bffff700 1 -- 172.21.15.36:0/1307686614 wait complete. 2022-06-17T12:54:34.058 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-17T12:54:34.058 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-17T12:54:34.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.180+0000 7f737f6cb700 1 Processor -- start 2022-06-17T12:54:34.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f737f6cb700 1 -- start start 2022-06-17T12:54:34.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f737f6cb700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f73780afe40 0x7f73780b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:34.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f737f6cb700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f737813b130 0x7f7378140f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:34.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f737f6cb700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73780a9b90 con 0x7f73780b3440 2022-06-17T12:54:34.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f737f6cb700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73780b0220 con 0x7f73780afe40 2022-06-17T12:54:34.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f737f6cb700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73780a9dc0 con 0x7f737813b130 2022-06-17T12:54:34.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f737cc66700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f73780b3440 0x7f737813aa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54498/0 (socket says 172.21.15.36:54498) 2022-06-17T12:54:34.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f737cc66700 1 -- 172.21.15.36:0/1829680121 learned_addr learned my addr 172.21.15.36:0/1829680121 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:34.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f737dc68700 1 --2- 172.21.15.36:0/1829680121 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f737813b130 0x7f7378140f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:34.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f737d467700 1 --2- 172.21.15.36:0/1829680121 >> v2:172.21.15.70:3300/0 conn(0x7f73780afe40 0x7f73780b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:34.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f737dc68700 1 -- 172.21.15.36:0/1829680121 >> v2:172.21.15.70:3300/0 conn(0x7f73780afe40 msgr2=0x7f73780b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:34.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f737dc68700 1 --2- 172.21.15.36:0/1829680121 >> v2:172.21.15.70:3300/0 conn(0x7f73780afe40 0x7f73780b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:34.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f737dc68700 1 -- 172.21.15.36:0/1829680121 >> v1:172.21.15.36:6789/0 conn(0x7f73780b3440 legacy=0x7f737813aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:34.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f7367fff700 1 -- 172.21.15.36:0/1829680121 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1446305662 0 0) 0x7f73780a9b90 con 0x7f73780b3440 2022-06-17T12:54:34.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f737dc68700 1 -- 172.21.15.36:0/1829680121 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7378144f40 con 0x7f737813b130 2022-06-17T12:54:34.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f737dc68700 1 --2- 172.21.15.36:0/1829680121 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f737813b130 0x7f7378140f60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f73780b3870 tx=0x7f7370006df0).ready entity=mon.2 client_cookie=d0da53b797f68043 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:34.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f7367fff700 1 -- 172.21.15.36:0/1829680121 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f737000d570 con 0x7f737813b130 2022-06-17T12:54:34.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f7367fff700 1 -- 172.21.15.36:0/1829680121 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7370000e90 con 0x7f737813b130 2022-06-17T12:54:34.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f7367fff700 1 -- 172.21.15.36:0/1829680121 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f737000d8c0 con 0x7f737813b130 2022-06-17T12:54:34.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f737f6cb700 1 -- 172.21.15.36:0/1829680121 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f737813b130 msgr2=0x7f7378140f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:34.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f737f6cb700 1 --2- 172.21.15.36:0/1829680121 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f737813b130 0x7f7378140f60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f73780b3870 tx=0x7f7370006df0).stop 2022-06-17T12:54:34.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f737f6cb700 1 -- 172.21.15.36:0/1829680121 shutdown_connections 2022-06-17T12:54:34.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f737f6cb700 1 --2- 172.21.15.36:0/1829680121 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f737813b130 0x7f7378140f60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:34.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f737f6cb700 1 --2- 172.21.15.36:0/1829680121 >> v2:172.21.15.70:3300/0 conn(0x7f73780afe40 0x7f73780b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:34.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f737f6cb700 1 -- 172.21.15.36:0/1829680121 >> 172.21.15.36:0/1829680121 conn(0x7f737805c550 msgr2=0x7f737805c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:34.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.184+0000 7f737f6cb700 1 -- 172.21.15.36:0/1829680121 shutdown_connections 2022-06-17T12:54:34.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.188+0000 7f737f6cb700 1 -- 172.21.15.36:0/1829680121 wait complete. 2022-06-17T12:54:34.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.188+0000 7f737f6cb700 1 Processor -- start 2022-06-17T12:54:34.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.188+0000 7f737f6cb700 1 -- start start 2022-06-17T12:54:34.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.188+0000 7f737f6cb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f73780afe40 0x7f73780ab980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:34.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.188+0000 7f737f6cb700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73780b3440 0x7f7378065030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:34.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.188+0000 7f737f6cb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f737813b130 0x7f7378150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:34.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.188+0000 7f737f6cb700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737813d880 con 0x7f73780afe40 2022-06-17T12:54:34.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.188+0000 7f737f6cb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73780648f0 con 0x7f737813b130 2022-06-17T12:54:34.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.188+0000 7f737f6cb700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7378064b90 con 0x7f73780b3440 2022-06-17T12:54:34.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.188+0000 7f737d467700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f73780afe40 0x7f73780ab980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:34.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.188+0000 7f737d467700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f73780afe40 0x7f73780ab980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36970/0 (socket says 172.21.15.36:36970) 2022-06-17T12:54:34.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.188+0000 7f737d467700 1 -- 172.21.15.36:0/2133804832 learned_addr learned my addr 172.21.15.36:0/2133804832 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:34.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.188+0000 7f737dc68700 1 --2- 172.21.15.36:0/2133804832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f737813b130 0x7f7378150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:34.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.188+0000 7f737cc66700 1 --2- 172.21.15.36:0/2133804832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73780b3440 0x7f7378065030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:34.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.192+0000 7f737dc68700 1 -- 172.21.15.36:0/2133804832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73780b3440 msgr2=0x7f7378065030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:34.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.192+0000 7f737dc68700 1 --2- 172.21.15.36:0/2133804832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73780b3440 0x7f7378065030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:34.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.192+0000 7f737dc68700 1 -- 172.21.15.36:0/2133804832 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f73780afe40 msgr2=0x7f73780ab980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:34.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.192+0000 7f737dc68700 1 --2- 172.21.15.36:0/2133804832 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f73780afe40 0x7f73780ab980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:34.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.192+0000 7f737dc68700 1 -- 172.21.15.36:0/2133804832 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737000b180 con 0x7f737813b130 2022-06-17T12:54:34.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.192+0000 7f737d467700 1 --2- 172.21.15.36:0/2133804832 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f73780afe40 0x7f73780ab980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:34.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.192+0000 7f737dc68700 1 --2- 172.21.15.36:0/2133804832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f737813b130 0x7f7378150650 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f73780b3870 tx=0x7f7370014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:34.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.192+0000 7f73667fc700 1 -- 172.21.15.36:0/2133804832 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7370016070 con 0x7f737813b130 2022-06-17T12:54:34.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.192+0000 7f73667fc700 1 -- 172.21.15.36:0/2133804832 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7370002f70 con 0x7f737813b130 2022-06-17T12:54:34.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.192+0000 7f737f6cb700 1 -- 172.21.15.36:0/2133804832 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7378065520 con 0x7f737813b130 2022-06-17T12:54:34.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.192+0000 7f73667fc700 1 -- 172.21.15.36:0/2133804832 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7370013bf0 con 0x7f737813b130 2022-06-17T12:54:34.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.192+0000 7f737f6cb700 1 -- 172.21.15.36:0/2133804832 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737813d500 con 0x7f737813b130 2022-06-17T12:54:34.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.192+0000 7f737f6cb700 1 -- 172.21.15.36:0/2133804832 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7378064d50 con 0x7f737813b130 2022-06-17T12:54:34.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.192+0000 7f73667fc700 1 -- 172.21.15.36:0/2133804832 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f737000f030 con 0x7f737813b130 2022-06-17T12:54:34.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.192+0000 7f73667fc700 1 --2- 172.21.15.36:0/2133804832 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f73580539b0 0x7f7358055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:34.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.192+0000 7f73667fc700 1 -- 172.21.15.36:0/2133804832 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f737006c910 con 0x7f737813b130 2022-06-17T12:54:34.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.192+0000 7f737d467700 1 --2- 172.21.15.36:0/2133804832 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f73580539b0 0x7f7358055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:34.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.192+0000 7f737d467700 1 --2- 172.21.15.36:0/2133804832 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f73580539b0 0x7f7358055e70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f7374006e00 tx=0x7f73740091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:34.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.204+0000 7f73667fc700 1 -- 172.21.15.36:0/2133804832 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f73700439b0 con 0x7f737813b130 2022-06-17T12:54:34.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.328+0000 7f737f6cb700 1 -- 172.21.15.36:0/2133804832 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f73781451e0 con 0x7f737813b130 2022-06-17T12:54:34.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.872+0000 7f73667fc700 1 -- 172.21.15.36:0/2133804832 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v90) v1 ==== 133+0+0 (secure 0 0 0) 0x7f7370023070 con 0x7f737813b130 2022-06-17T12:54:34.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.876+0000 7f737f6cb700 1 -- 172.21.15.36:0/2133804832 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f73580539b0 msgr2=0x7f7358055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:34.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.876+0000 7f737f6cb700 1 --2- 172.21.15.36:0/2133804832 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f73580539b0 0x7f7358055e70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f7374006e00 tx=0x7f73740091d0).stop 2022-06-17T12:54:34.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.876+0000 7f737f6cb700 1 -- 172.21.15.36:0/2133804832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f737813b130 msgr2=0x7f7378150650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:34.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.876+0000 7f737f6cb700 1 --2- 172.21.15.36:0/2133804832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f737813b130 0x7f7378150650 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f73780b3870 tx=0x7f7370014000).stop 2022-06-17T12:54:34.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.876+0000 7f737f6cb700 1 -- 172.21.15.36:0/2133804832 shutdown_connections 2022-06-17T12:54:34.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.876+0000 7f737f6cb700 1 --2- 172.21.15.36:0/2133804832 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f73580539b0 0x7f7358055e70 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:34.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.876+0000 7f737f6cb700 1 --2- 172.21.15.36:0/2133804832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f737813b130 0x7f7378150650 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:34.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.876+0000 7f737f6cb700 1 --2- 172.21.15.36:0/2133804832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73780b3440 0x7f7378065030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:34.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.876+0000 7f737f6cb700 1 --2- 172.21.15.36:0/2133804832 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f73780afe40 0x7f73780ab980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:34.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.876+0000 7f737f6cb700 1 -- 172.21.15.36:0/2133804832 >> 172.21.15.36:0/2133804832 conn(0x7f737805c550 msgr2=0x7f73781443d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:34.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.876+0000 7f737f6cb700 1 -- 172.21.15.36:0/2133804832 shutdown_connections 2022-06-17T12:54:34.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.876+0000 7f737f6cb700 1 -- 172.21.15.36:0/2133804832 wait complete. 2022-06-17T12:54:34.867 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' created 2022-06-17T12:54:34.879 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-06-17T12:54:34.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.996+0000 7f7801c83700 1 Processor -- start 2022-06-17T12:54:34.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:34.996+0000 7f7801c83700 1 -- start start 2022-06-17T12:54:35.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.000+0000 7f7801c83700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77fc0b1680 0x7f77fc0630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:35.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.000+0000 7f7801c83700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f77fc0ae6c0 0x7f77fc13fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:35.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.000+0000 7f7801c83700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77fc062c80 con 0x7f77fc060ac0 2022-06-17T12:54:35.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.000+0000 7f7801c83700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77fc060ea0 con 0x7f77fc0ae6c0 2022-06-17T12:54:35.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.000+0000 7f7801c83700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc0b2f50 con 0x7f77fc0b1680 2022-06-17T12:54:35.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.000+0000 7f7801482700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f77fc0ae6c0 0x7f77fc13fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:35.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.000+0000 7f7800c81700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f77fc060ac0 0x7f77fc1431a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54508/0 (socket says 172.21.15.36:54508) 2022-06-17T12:54:35.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.000+0000 7f7800c81700 1 -- 172.21.15.36:0/1868306017 learned_addr learned my addr 172.21.15.36:0/1868306017 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:35.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.000+0000 7f77fbfff700 1 --2- 172.21.15.36:0/1868306017 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77fc0b1680 0x7f77fc0630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:35.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.000+0000 7f7801482700 1 -- 172.21.15.36:0/1868306017 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77fc0b1680 msgr2=0x7f77fc0630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:35.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.000+0000 7f7801482700 1 --2- 172.21.15.36:0/1868306017 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77fc0b1680 0x7f77fc0630e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:35.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.000+0000 7f7801482700 1 -- 172.21.15.36:0/1868306017 >> v1:172.21.15.36:6789/0 conn(0x7f77fc060ac0 legacy=0x7f77fc1431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:35.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.000+0000 7f7801482700 1 -- 172.21.15.36:0/1868306017 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77fc0635d0 con 0x7f77fc0ae6c0 2022-06-17T12:54:35.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.000+0000 7f7801482700 1 --2- 172.21.15.36:0/1868306017 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77fc0ae6c0 0x7f77fc13fef0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7f77f40032f0 tx=0x7f77f40091d0).ready entity=mon.0 client_cookie=9ddd2354cb03230b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:35.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.000+0000 7f77fb7fe700 1 -- 172.21.15.36:0/1868306017 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77f400b8b0 con 0x7f77fc0ae6c0 2022-06-17T12:54:35.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.000+0000 7f77fb7fe700 1 -- 172.21.15.36:0/1868306017 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f400ba10 con 0x7f77fc0ae6c0 2022-06-17T12:54:35.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.000+0000 7f77fb7fe700 1 -- 172.21.15.36:0/1868306017 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77f400bd70 con 0x7f77fc0ae6c0 2022-06-17T12:54:35.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.000+0000 7f7801c83700 1 -- 172.21.15.36:0/1868306017 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77fc0ae6c0 msgr2=0x7f77fc13fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:35.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.000+0000 7f7801c83700 1 --2- 172.21.15.36:0/1868306017 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77fc0ae6c0 0x7f77fc13fef0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7f77f40032f0 tx=0x7f77f40091d0).stop 2022-06-17T12:54:35.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.000+0000 7f7801c83700 1 -- 172.21.15.36:0/1868306017 shutdown_connections 2022-06-17T12:54:35.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.000+0000 7f7801c83700 1 --2- 172.21.15.36:0/1868306017 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77fc0ae6c0 0x7f77fc13fef0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:35.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.000+0000 7f7801c83700 1 --2- 172.21.15.36:0/1868306017 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77fc0b1680 0x7f77fc0630e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:35.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.000+0000 7f7801c83700 1 -- 172.21.15.36:0/1868306017 >> 172.21.15.36:0/1868306017 conn(0x7f77fc0b64f0 msgr2=0x7f77fc0b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:35.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.004+0000 7f7801c83700 1 -- 172.21.15.36:0/1868306017 shutdown_connections 2022-06-17T12:54:35.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.004+0000 7f7801c83700 1 -- 172.21.15.36:0/1868306017 wait complete. 2022-06-17T12:54:35.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.004+0000 7f7801c83700 1 Processor -- start 2022-06-17T12:54:35.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.004+0000 7f7801c83700 1 -- start start 2022-06-17T12:54:35.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.008+0000 7f7801c83700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77fc060ac0 0x7f77fc0b4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:35.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.008+0000 7f7801c83700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77fc0ae6c0 0x7f77fc13bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:35.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.008+0000 7f7801c83700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77fc0b1680 0x7f77fc14dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:35.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.008+0000 7f7801c83700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc13c800 con 0x7f77fc060ac0 2022-06-17T12:54:35.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.008+0000 7f7801c83700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc0b3930 con 0x7f77fc0b1680 2022-06-17T12:54:35.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.008+0000 7f7801c83700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc0b3bd0 con 0x7f77fc0ae6c0 2022-06-17T12:54:35.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.008+0000 7f7800c81700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77fc060ac0 0x7f77fc0b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:35.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.008+0000 7f7800c81700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77fc060ac0 0x7f77fc0b4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36984/0 (socket says 172.21.15.36:36984) 2022-06-17T12:54:35.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.008+0000 7f7800c81700 1 -- 172.21.15.36:0/1288757327 learned_addr learned my addr 172.21.15.36:0/1288757327 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:35.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.008+0000 7f77fbfff700 1 --2- 172.21.15.36:0/1288757327 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77fc0ae6c0 0x7f77fc13bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:35.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.008+0000 7f7801482700 1 --2- 172.21.15.36:0/1288757327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77fc0b1680 0x7f77fc14dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:35.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.008+0000 7f77fbfff700 1 -- 172.21.15.36:0/1288757327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77fc0b1680 msgr2=0x7f77fc14dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:35.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.008+0000 7f77fbfff700 1 --2- 172.21.15.36:0/1288757327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77fc0b1680 0x7f77fc14dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:35.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.008+0000 7f77fbfff700 1 -- 172.21.15.36:0/1288757327 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77fc060ac0 msgr2=0x7f77fc0b4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:35.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.008+0000 7f77fbfff700 1 --2- 172.21.15.36:0/1288757327 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77fc060ac0 0x7f77fc0b4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:35.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.008+0000 7f77fbfff700 1 -- 172.21.15.36:0/1288757327 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f4009470 con 0x7f77fc0ae6c0 2022-06-17T12:54:35.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.008+0000 7f7800c81700 1 --2- 172.21.15.36:0/1288757327 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77fc060ac0 0x7f77fc0b4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:35.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.008+0000 7f77fbfff700 1 --2- 172.21.15.36:0/1288757327 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77fc0ae6c0 0x7f77fc13bb00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f77ec00c000 tx=0x7f77ec006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:35.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.008+0000 7f77f9ffb700 1 -- 172.21.15.36:0/1288757327 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77ec011750 con 0x7f77fc0ae6c0 2022-06-17T12:54:35.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.008+0000 7f77f9ffb700 1 -- 172.21.15.36:0/1288757327 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ec0118b0 con 0x7f77fc0ae6c0 2022-06-17T12:54:35.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.008+0000 7f7801c83700 1 -- 172.21.15.36:0/1288757327 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77fc0b3e70 con 0x7f77fc0ae6c0 2022-06-17T12:54:35.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.008+0000 7f77f9ffb700 1 -- 172.21.15.36:0/1288757327 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77ec011c10 con 0x7f77fc0ae6c0 2022-06-17T12:54:35.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.008+0000 7f7801c83700 1 -- 172.21.15.36:0/1288757327 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77fc067450 con 0x7f77fc0ae6c0 2022-06-17T12:54:35.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.008+0000 7f7801c83700 1 -- 172.21.15.36:0/1288757327 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77fc063b60 con 0x7f77fc0ae6c0 2022-06-17T12:54:35.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.012+0000 7f77f9ffb700 1 -- 172.21.15.36:0/1288757327 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f77ec011d70 con 0x7f77fc0ae6c0 2022-06-17T12:54:35.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.012+0000 7f77f9ffb700 1 --2- 172.21.15.36:0/1288757327 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f77dc053a60 0x7f77dc055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:35.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.012+0000 7f77f9ffb700 1 -- 172.21.15.36:0/1288757327 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f77ec06c0e0 con 0x7f77fc0ae6c0 2022-06-17T12:54:35.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.012+0000 7f7800c81700 1 --2- 172.21.15.36:0/1288757327 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f77dc053a60 0x7f77dc055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:35.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.012+0000 7f7800c81700 1 --2- 172.21.15.36:0/1288757327 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f77dc053a60 0x7f77dc055f20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f77f000d000 tx=0x7f77f0007c40).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:35.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.020+0000 7f77f9ffb700 1 -- 172.21.15.36:0/1288757327 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f77fc063b60 con 0x7f77fc0ae6c0 2022-06-17T12:54:35.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.144+0000 7f7801c83700 1 -- 172.21.15.36:0/1288757327 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f77fc063b60 con 0x7f77fc0ae6c0 2022-06-17T12:54:35.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.636+0000 7f77f9ffb700 1 -- 172.21.15.36:0/1288757327 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v91) v1 ==== 125+0+0 (secure 0 0 0) 0x7f77fc063b60 con 0x7f77fc0ae6c0 2022-06-17T12:54:35.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.640+0000 7f7801c83700 1 -- 172.21.15.36:0/1288757327 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f77dc053a60 msgr2=0x7f77dc055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:35.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.640+0000 7f7801c83700 1 --2- 172.21.15.36:0/1288757327 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f77dc053a60 0x7f77dc055f20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f77f000d000 tx=0x7f77f0007c40).stop 2022-06-17T12:54:35.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.640+0000 7f7801c83700 1 -- 172.21.15.36:0/1288757327 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77fc0ae6c0 msgr2=0x7f77fc13bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:35.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.640+0000 7f7801c83700 1 --2- 172.21.15.36:0/1288757327 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77fc0ae6c0 0x7f77fc13bb00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f77ec00c000 tx=0x7f77ec006df0).stop 2022-06-17T12:54:35.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.644+0000 7f7801c83700 1 -- 172.21.15.36:0/1288757327 shutdown_connections 2022-06-17T12:54:35.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.644+0000 7f7801c83700 1 --2- 172.21.15.36:0/1288757327 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f77dc053a60 0x7f77dc055f20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:35.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.644+0000 7f7801c83700 1 --2- 172.21.15.36:0/1288757327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77fc0b1680 0x7f77fc14dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:35.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.644+0000 7f7801c83700 1 --2- 172.21.15.36:0/1288757327 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77fc0ae6c0 0x7f77fc13bb00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:35.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.644+0000 7f7801c83700 1 --2- 172.21.15.36:0/1288757327 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77fc060ac0 0x7f77fc0b4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:35.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.644+0000 7f7801c83700 1 -- 172.21.15.36:0/1288757327 >> 172.21.15.36:0/1288757327 conn(0x7f77fc0b64f0 msgr2=0x7f77fc13f880 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:35.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.644+0000 7f7801c83700 1 -- 172.21.15.36:0/1288757327 shutdown_connections 2022-06-17T12:54:35.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.644+0000 7f7801c83700 1 -- 172.21.15.36:0/1288757327 wait complete. 2022-06-17T12:54:35.633 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' created 2022-06-17T12:54:35.643 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-17T12:54:35.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.760+0000 7f4c3d326700 1 Processor -- start 2022-06-17T12:54:35.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.760+0000 7f4c3d326700 1 -- start start 2022-06-17T12:54:35.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.760+0000 7f4c3d326700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4c380b6600 0x7f4c38146da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:35.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.760+0000 7f4c3d326700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4c380b56f0 0x7f4c381420a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:35.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.760+0000 7f4c3d326700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c380b48e0 con 0x7f4c381469c0 2022-06-17T12:54:35.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.760+0000 7f4c3d326700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c38147290 con 0x7f4c380b56f0 2022-06-17T12:54:35.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.760+0000 7f4c3d326700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c380b4b40 con 0x7f4c380b6600 2022-06-17T12:54:35.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.764+0000 7f4c377fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4c380b56f0 0x7f4c381420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:35.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.764+0000 7f4c377fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4c380b56f0 0x7f4c381420a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:36996/0 (socket says 172.21.15.36:36996) 2022-06-17T12:54:35.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.764+0000 7f4c377fe700 1 -- 172.21.15.36:0/3174397098 learned_addr learned my addr 172.21.15.36:0/3174397098 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:35.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.764+0000 7f4c367fc700 1 --2- 172.21.15.36:0/3174397098 >> v2:172.21.15.70:3300/0 conn(0x7f4c380b6600 0x7f4c38146da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:35.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.764+0000 7f4c377fe700 1 -- 172.21.15.36:0/3174397098 >> v2:172.21.15.70:3300/0 conn(0x7f4c380b6600 msgr2=0x7f4c38146da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:35.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.764+0000 7f4c377fe700 1 --2- 172.21.15.36:0/3174397098 >> v2:172.21.15.70:3300/0 conn(0x7f4c380b6600 0x7f4c38146da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:35.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.764+0000 7f4c377fe700 1 -- 172.21.15.36:0/3174397098 >> v1:172.21.15.70:6789/0 conn(0x7f4c381469c0 legacy=0x7f4c380b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:35.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.764+0000 7f4c377fe700 1 -- 172.21.15.36:0/3174397098 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c380b6aa0 con 0x7f4c380b56f0 2022-06-17T12:54:35.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.764+0000 7f4c377fe700 1 --2- 172.21.15.36:0/3174397098 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4c380b56f0 0x7f4c381420a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7f4c380b1060 tx=0x7f4c30009290).ready entity=mon.0 client_cookie=6bac2a70b97d8cb5 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:35.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.764+0000 7f4c35ffb700 1 -- 172.21.15.36:0/3174397098 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c3000b860 con 0x7f4c380b56f0 2022-06-17T12:54:35.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.764+0000 7f4c35ffb700 1 -- 172.21.15.36:0/3174397098 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3000b9c0 con 0x7f4c380b56f0 2022-06-17T12:54:35.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.764+0000 7f4c35ffb700 1 -- 172.21.15.36:0/3174397098 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c3000bb30 con 0x7f4c380b56f0 2022-06-17T12:54:35.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.764+0000 7f4c3d326700 1 -- 172.21.15.36:0/3174397098 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4c380b56f0 msgr2=0x7f4c381420a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:35.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.764+0000 7f4c3d326700 1 --2- 172.21.15.36:0/3174397098 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4c380b56f0 0x7f4c381420a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7f4c380b1060 tx=0x7f4c30009290).stop 2022-06-17T12:54:35.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.764+0000 7f4c3d326700 1 -- 172.21.15.36:0/3174397098 shutdown_connections 2022-06-17T12:54:35.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.764+0000 7f4c3d326700 1 --2- 172.21.15.36:0/3174397098 >> v2:172.21.15.70:3300/0 conn(0x7f4c380b6600 0x7f4c38146da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:35.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.764+0000 7f4c3d326700 1 --2- 172.21.15.36:0/3174397098 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4c380b56f0 0x7f4c381420a0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:35.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.764+0000 7f4c3d326700 1 -- 172.21.15.36:0/3174397098 >> 172.21.15.36:0/3174397098 conn(0x7f4c3805f770 msgr2=0x7f4c3805fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:35.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.764+0000 7f4c3d326700 1 -- 172.21.15.36:0/3174397098 shutdown_connections 2022-06-17T12:54:35.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.764+0000 7f4c3d326700 1 -- 172.21.15.36:0/3174397098 wait complete. 2022-06-17T12:54:35.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c3d326700 1 Processor -- start 2022-06-17T12:54:35.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c3d326700 1 -- start start 2022-06-17T12:54:35.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c3d326700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c380b56f0 0x7f4c380afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:35.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c3d326700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c380b6600 0x7f4c380b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:35.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c3d326700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4c381469c0 0x7f4c3813e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:35.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c3d326700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c380b3b40 con 0x7f4c381469c0 2022-06-17T12:54:35.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c3d326700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c38112b70 con 0x7f4c380b56f0 2022-06-17T12:54:35.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c3d326700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c38112e10 con 0x7f4c380b6600 2022-06-17T12:54:35.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c377fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4c381469c0 0x7f4c3813e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:35.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c377fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4c381469c0 0x7f4c3813e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37002/0 (socket says 172.21.15.36:37002) 2022-06-17T12:54:35.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c377fe700 1 -- 172.21.15.36:0/3072606625 learned_addr learned my addr 172.21.15.36:0/3072606625 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:35.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c367fc700 1 --2- 172.21.15.36:0/3072606625 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c380b6600 0x7f4c380b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:35.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c36ffd700 1 --2- 172.21.15.36:0/3072606625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c380b56f0 0x7f4c380afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:35.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c377fe700 1 -- 172.21.15.36:0/3072606625 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c380b6600 msgr2=0x7f4c380b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:35.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c377fe700 1 --2- 172.21.15.36:0/3072606625 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c380b6600 0x7f4c380b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:35.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c377fe700 1 -- 172.21.15.36:0/3072606625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c380b56f0 msgr2=0x7f4c380afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:35.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c377fe700 1 --2- 172.21.15.36:0/3072606625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c380b56f0 0x7f4c380afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:35.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c377fe700 1 -- 172.21.15.36:0/3072606625 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c30009530 con 0x7f4c381469c0 2022-06-17T12:54:35.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c36ffd700 1 --2- 172.21.15.36:0/3072606625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c380b56f0 0x7f4c380afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:35.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c377fe700 1 --2- 172.21.15.36:0/3072606625 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4c381469c0 0x7f4c3813e460 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7f4c380b1060 tx=0x7f4c30016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:35.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c27fff700 1 -- 172.21.15.36:0/3072606625 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c30021440 con 0x7f4c381469c0 2022-06-17T12:54:35.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c27fff700 1 -- 172.21.15.36:0/3072606625 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c300215a0 con 0x7f4c381469c0 2022-06-17T12:54:35.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c3d326700 1 -- 172.21.15.36:0/3072606625 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c380b0060 con 0x7f4c381469c0 2022-06-17T12:54:35.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c27fff700 1 -- 172.21.15.36:0/3072606625 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c30021900 con 0x7f4c381469c0 2022-06-17T12:54:35.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.768+0000 7f4c3d326700 1 -- 172.21.15.36:0/3072606625 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c380ae420 con 0x7f4c381469c0 2022-06-17T12:54:35.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.772+0000 7f4c3d326700 1 -- 172.21.15.36:0/3072606625 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c38112fd0 con 0x7f4c381469c0 2022-06-17T12:54:35.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.772+0000 7f4c27fff700 1 -- 172.21.15.36:0/3072606625 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f4c30029070 con 0x7f4c381469c0 2022-06-17T12:54:35.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.772+0000 7f4c27fff700 1 --2- 172.21.15.36:0/3072606625 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4c14053a60 0x7f4c14055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:35.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.772+0000 7f4c27fff700 1 -- 172.21.15.36:0/3072606625 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f4c3006d460 con 0x7f4c381469c0 2022-06-17T12:54:35.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.772+0000 7f4c36ffd700 1 --2- 172.21.15.36:0/3072606625 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4c14053a60 0x7f4c14055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:35.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.772+0000 7f4c36ffd700 1 --2- 172.21.15.36:0/3072606625 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4c14053a60 0x7f4c14055f20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f4c2800b000 tx=0x7f4c28009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:35.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.776+0000 7f4c27fff700 1 -- 172.21.15.36:0/3072606625 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4c300442e0 con 0x7f4c381469c0 2022-06-17T12:54:35.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.944+0000 7f4c3d326700 1 -- 172.21.15.36:0/3072606625 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f4c380b3960 con 0x7f4c381469c0 2022-06-17T12:54:35.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.956+0000 7f4c27fff700 1 -- 172.21.15.36:0/3072606625 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 17 and data pool 18 v46) v1 ==== 189+0+0 (secure 0 0 0) 0x7f4c30015070 con 0x7f4c381469c0 2022-06-17T12:54:35.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.964+0000 7f4c3d326700 1 -- 172.21.15.36:0/3072606625 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4c14053a60 msgr2=0x7f4c14055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:35.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.964+0000 7f4c3d326700 1 --2- 172.21.15.36:0/3072606625 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4c14053a60 0x7f4c14055f20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f4c2800b000 tx=0x7f4c28009040).stop 2022-06-17T12:54:35.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.964+0000 7f4c3d326700 1 -- 172.21.15.36:0/3072606625 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4c381469c0 msgr2=0x7f4c3813e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:35.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.964+0000 7f4c3d326700 1 --2- 172.21.15.36:0/3072606625 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4c381469c0 0x7f4c3813e460 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7f4c380b1060 tx=0x7f4c30016000).stop 2022-06-17T12:54:35.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.964+0000 7f4c3d326700 1 -- 172.21.15.36:0/3072606625 shutdown_connections 2022-06-17T12:54:35.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.964+0000 7f4c3d326700 1 --2- 172.21.15.36:0/3072606625 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4c14053a60 0x7f4c14055f20 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:35.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.964+0000 7f4c3d326700 1 --2- 172.21.15.36:0/3072606625 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4c381469c0 0x7f4c3813e460 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:35.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.964+0000 7f4c3d326700 1 --2- 172.21.15.36:0/3072606625 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c380b6600 0x7f4c380b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:35.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.964+0000 7f4c3d326700 1 --2- 172.21.15.36:0/3072606625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c380b56f0 0x7f4c380afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:35.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.964+0000 7f4c3d326700 1 -- 172.21.15.36:0/3072606625 >> 172.21.15.36:0/3072606625 conn(0x7f4c3805f770 msgr2=0x7f4c381488d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:35.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.964+0000 7f4c3d326700 1 -- 172.21.15.36:0/3072606625 shutdown_connections 2022-06-17T12:54:35.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:35.964+0000 7f4c3d326700 1 -- 172.21.15.36:0/3072606625 wait complete. 2022-06-17T12:54:35.957 INFO:teuthology.orchestra.run.smithi036.stderr:new fs with metadata pool 17 and data pool 18 2022-06-17T12:54:35.964 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:54:36.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.072+0000 7f8d94630700 1 Processor -- start 2022-06-17T12:54:36.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.072+0000 7f8d94630700 1 -- start start 2022-06-17T12:54:36.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.072+0000 7f8d94630700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8d8c141860 0x7f8d8c13da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:36.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.072+0000 7f8d94630700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d8c05fe90 con 0x7f8d8c13df40 2022-06-17T12:54:36.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.072+0000 7f8d94630700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d8c144fa0 con 0x7f8d8c05fab0 2022-06-17T12:54:36.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.072+0000 7f8d94630700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d8c145240 con 0x7f8d8c141860 2022-06-17T12:54:36.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.072+0000 7f8d92bcd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f8d8c13df40 0x7f8d8c13e370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54548/0 (socket says 172.21.15.36:54548) 2022-06-17T12:54:36.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.072+0000 7f8d92bcd700 1 -- 172.21.15.36:0/1260099203 learned_addr learned my addr 172.21.15.36:0/1260099203 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:36.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.072+0000 7f8d91bcb700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8d8c141860 0x7f8d8c13da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:36.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.072+0000 7f8d913ca700 1 -- 172.21.15.36:0/1260099203 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714892975 0 0) 0x7f8d8c144fa0 con 0x7f8d8c05fab0 2022-06-17T12:54:36.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.072+0000 7f8d913ca700 1 -- 172.21.15.36:0/1260099203 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d74003260 con 0x7f8d8c05fab0 2022-06-17T12:54:36.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.072+0000 7f8d91bcb700 1 -- 172.21.15.36:0/1260099203 >> v1:172.21.15.70:6789/0 conn(0x7f8d8c05fab0 legacy=0x7f8d8c141150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:36.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.072+0000 7f8d91bcb700 1 -- 172.21.15.36:0/1260099203 >> v1:172.21.15.36:6789/0 conn(0x7f8d8c13df40 legacy=0x7f8d8c13e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:36.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.072+0000 7f8d913ca700 1 -- 172.21.15.36:0/1260099203 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22152567 0 0) 0x7f8d8c05fe90 con 0x7f8d8c13df40 2022-06-17T12:54:36.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.072+0000 7f8d91bcb700 1 -- 172.21.15.36:0/1260099203 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8c0afbd0 con 0x7f8d8c141860 2022-06-17T12:54:36.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.072+0000 7f8d91bcb700 1 --2- 172.21.15.36:0/1260099203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d8c141860 0x7f8d8c13da50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f8d80002f70 tx=0x7f8d80006df0).ready entity=mon.1 client_cookie=6381b7f924a5474a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:36.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.072+0000 7f8d913ca700 1 -- 172.21.15.36:0/1260099203 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d8000d610 con 0x7f8d8c141860 2022-06-17T12:54:36.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.072+0000 7f8d913ca700 1 -- 172.21.15.36:0/1260099203 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d80000e90 con 0x7f8d8c141860 2022-06-17T12:54:36.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.072+0000 7f8d913ca700 1 -- 172.21.15.36:0/1260099203 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d8000d960 con 0x7f8d8c141860 2022-06-17T12:54:36.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.076+0000 7f8d94630700 1 -- 172.21.15.36:0/1260099203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d8c141860 msgr2=0x7f8d8c13da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:36.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.076+0000 7f8d94630700 1 --2- 172.21.15.36:0/1260099203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d8c141860 0x7f8d8c13da50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f8d80002f70 tx=0x7f8d80006df0).stop 2022-06-17T12:54:36.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.076+0000 7f8d94630700 1 -- 172.21.15.36:0/1260099203 shutdown_connections 2022-06-17T12:54:36.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.076+0000 7f8d94630700 1 --2- 172.21.15.36:0/1260099203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d8c141860 0x7f8d8c13da50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.076+0000 7f8d94630700 1 -- 172.21.15.36:0/1260099203 >> 172.21.15.36:0/1260099203 conn(0x7f8d8c05c550 msgr2=0x7f8d8c05c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:36.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.076+0000 7f8d94630700 1 -- 172.21.15.36:0/1260099203 shutdown_connections 2022-06-17T12:54:36.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.076+0000 7f8d94630700 1 -- 172.21.15.36:0/1260099203 wait complete. 2022-06-17T12:54:36.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.076+0000 7f8d94630700 1 Processor -- start 2022-06-17T12:54:36.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.076+0000 7f8d94630700 1 -- start start 2022-06-17T12:54:36.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.076+0000 7f8d94630700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d8c05fab0 0x7f8d8c0b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:36.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.076+0000 7f8d94630700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d8c13df40 0x7f8d8c155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:36.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.076+0000 7f8d94630700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8d8c141860 0x7f8d8c156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:36.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.076+0000 7f8d94630700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c153220 con 0x7f8d8c141860 2022-06-17T12:54:36.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.076+0000 7f8d94630700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c152200 con 0x7f8d8c13df40 2022-06-17T12:54:36.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.076+0000 7f8d94630700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c152450 con 0x7f8d8c05fab0 2022-06-17T12:54:36.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.076+0000 7f8d92bcd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8d8c141860 0x7f8d8c156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:36.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.076+0000 7f8d92bcd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8d8c141860 0x7f8d8c156060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37024/0 (socket says 172.21.15.36:37024) 2022-06-17T12:54:36.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.076+0000 7f8d91bcb700 1 --2- 172.21.15.36:0/109484716 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d8c13df40 0x7f8d8c155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:36.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.076+0000 7f8d92bcd700 1 -- 172.21.15.36:0/109484716 learned_addr learned my addr 172.21.15.36:0/109484716 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:36.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.076+0000 7f8d923cc700 1 --2- 172.21.15.36:0/109484716 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d8c05fab0 0x7f8d8c0b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:36.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.080+0000 7f8d91bcb700 1 -- 172.21.15.36:0/109484716 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d8c05fab0 msgr2=0x7f8d8c0b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:36.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.080+0000 7f8d91bcb700 1 --2- 172.21.15.36:0/109484716 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d8c05fab0 0x7f8d8c0b46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.080+0000 7f8d91bcb700 1 -- 172.21.15.36:0/109484716 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8d8c141860 msgr2=0x7f8d8c156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:36.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.080+0000 7f8d91bcb700 1 --2- 172.21.15.36:0/109484716 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8d8c141860 0x7f8d8c156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.080+0000 7f8d91bcb700 1 -- 172.21.15.36:0/109484716 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8000b180 con 0x7f8d8c13df40 2022-06-17T12:54:36.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.080+0000 7f8d92bcd700 1 --2- 172.21.15.36:0/109484716 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8d8c141860 0x7f8d8c156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:36.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.080+0000 7f8d91bcb700 1 --2- 172.21.15.36:0/109484716 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d8c13df40 0x7f8d8c155b70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f8d8c152720 tx=0x7f8d80014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:36.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.080+0000 7f8d7f7fe700 1 -- 172.21.15.36:0/109484716 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d80013da0 con 0x7f8d8c13df40 2022-06-17T12:54:36.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.080+0000 7f8d7f7fe700 1 -- 172.21.15.36:0/109484716 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d8000d290 con 0x7f8d8c13df40 2022-06-17T12:54:36.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.080+0000 7f8d94630700 1 -- 172.21.15.36:0/109484716 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8c145480 con 0x7f8d8c13df40 2022-06-17T12:54:36.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.080+0000 7f8d7f7fe700 1 -- 172.21.15.36:0/109484716 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d8001e5f0 con 0x7f8d8c13df40 2022-06-17T12:54:36.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.080+0000 7f8d94630700 1 -- 172.21.15.36:0/109484716 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8c1460d0 con 0x7f8d8c13df40 2022-06-17T12:54:36.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.080+0000 7f8d94630700 1 -- 172.21.15.36:0/109484716 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d8c141cf0 con 0x7f8d8c13df40 2022-06-17T12:54:36.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.080+0000 7f8d7f7fe700 1 -- 172.21.15.36:0/109484716 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8d80025030 con 0x7f8d8c13df40 2022-06-17T12:54:36.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.080+0000 7f8d7f7fe700 1 --2- 172.21.15.36:0/109484716 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8d6c053920 0x7f8d6c055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:36.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.080+0000 7f8d7f7fe700 1 -- 172.21.15.36:0/109484716 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8d8006cbc0 con 0x7f8d8c13df40 2022-06-17T12:54:36.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.080+0000 7f8d923cc700 1 --2- 172.21.15.36:0/109484716 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8d6c053920 0x7f8d6c055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:36.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.084+0000 7f8d923cc700 1 --2- 172.21.15.36:0/109484716 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8d6c053920 0x7f8d6c055de0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f8d8400d000 tx=0x7f8d84007d80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:36.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.092+0000 7f8d7f7fe700 1 -- 172.21.15.36:0/109484716 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8d800439c0 con 0x7f8d8c13df40 2022-06-17T12:54:36.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.220+0000 7f8d94630700 1 -- 172.21.15.36:0/109484716 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d8c057410 con 0x7f8d8c13df40 2022-06-17T12:54:36.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.224+0000 7f8d7f7fe700 1 -- 172.21.15.36:0/109484716 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16300 (secure 0 0 0) 0x7f8d80023020 con 0x7f8d8c13df40 2022-06-17T12:54:36.212 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:36.212 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":92,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:54:35.949366+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:54:34.381157+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-06-17T12:54:35.204631+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:54:36.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.224+0000 7f8d94630700 1 -- 172.21.15.36:0/109484716 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8d6c053920 msgr2=0x7f8d6c055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:36.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.224+0000 7f8d94630700 1 --2- 172.21.15.36:0/109484716 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8d6c053920 0x7f8d6c055de0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f8d8400d000 tx=0x7f8d84007d80).stop 2022-06-17T12:54:36.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.224+0000 7f8d94630700 1 -- 172.21.15.36:0/109484716 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d8c13df40 msgr2=0x7f8d8c155b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:36.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.224+0000 7f8d94630700 1 --2- 172.21.15.36:0/109484716 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d8c13df40 0x7f8d8c155b70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f8d8c152720 tx=0x7f8d80014000).stop 2022-06-17T12:54:36.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.224+0000 7f8d94630700 1 -- 172.21.15.36:0/109484716 shutdown_connections 2022-06-17T12:54:36.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.224+0000 7f8d94630700 1 --2- 172.21.15.36:0/109484716 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8d6c053920 0x7f8d6c055de0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.224+0000 7f8d94630700 1 --2- 172.21.15.36:0/109484716 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8d8c141860 0x7f8d8c156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.224+0000 7f8d94630700 1 --2- 172.21.15.36:0/109484716 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d8c13df40 0x7f8d8c155b70 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.224+0000 7f8d94630700 1 --2- 172.21.15.36:0/109484716 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d8c05fab0 0x7f8d8c0b46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.224+0000 7f8d94630700 1 -- 172.21.15.36:0/109484716 >> 172.21.15.36:0/109484716 conn(0x7f8d8c05c550 msgr2=0x7f8d8c0afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:36.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.224+0000 7f8d94630700 1 -- 172.21.15.36:0/109484716 shutdown_connections 2022-06-17T12:54:36.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.224+0000 7f8d94630700 1 -- 172.21.15.36:0/109484716 wait complete. 2022-06-17T12:54:36.225 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:54:36.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.336+0000 7f9ccd083700 1 Processor -- start 2022-06-17T12:54:36.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.336+0000 7f9ccd083700 1 -- start start 2022-06-17T12:54:36.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9ccd083700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cc8062770 0x7f9cc8062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:36.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9ccd083700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9cc81444f0 0x7f9cc8144930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:36.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9ccd083700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cc80b0ba0 con 0x7f9cc80ad330 2022-06-17T12:54:36.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9ccd083700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cc80ae820 con 0x7f9cc81444f0 2022-06-17T12:54:36.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9ccd083700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc8061030 con 0x7f9cc8062770 2022-06-17T12:54:36.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9cc759e700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f9cc80ad330 0x7f9cc80ad770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54562/0 (socket says 172.21.15.36:54562) 2022-06-17T12:54:36.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9cc759e700 1 -- 172.21.15.36:0/2001295930 learned_addr learned my addr 172.21.15.36:0/2001295930 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:36.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9cc659c700 1 --2- 172.21.15.36:0/2001295930 >> v2:172.21.15.70:3300/0 conn(0x7f9cc81444f0 0x7f9cc8144930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:36.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9cc6d9d700 1 --2- 172.21.15.36:0/2001295930 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cc8062770 0x7f9cc8062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:36.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9cc5d9b700 1 -- 172.21.15.36:0/2001295930 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 417130468 0 0) 0x7f9cc80b0ba0 con 0x7f9cc80ad330 2022-06-17T12:54:36.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9cc5d9b700 1 -- 172.21.15.36:0/2001295930 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cac003260 con 0x7f9cc80ad330 2022-06-17T12:54:36.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9cc659c700 1 -- 172.21.15.36:0/2001295930 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cc8062770 msgr2=0x7f9cc8062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:36.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9cc659c700 1 --2- 172.21.15.36:0/2001295930 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cc8062770 0x7f9cc8062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9cc659c700 1 -- 172.21.15.36:0/2001295930 >> v1:172.21.15.36:6789/0 conn(0x7f9cc80ad330 legacy=0x7f9cc80ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:36.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9cc659c700 1 -- 172.21.15.36:0/2001295930 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc8147e90 con 0x7f9cc81444f0 2022-06-17T12:54:36.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9cc659c700 1 --2- 172.21.15.36:0/2001295930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cc81444f0 0x7f9cc8144930 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f9cc8061170 tx=0x7f9cbc006df0).ready entity=mon.1 client_cookie=2a0150de50c3a88e server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:36.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9cc5d9b700 1 -- 172.21.15.36:0/2001295930 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cbc00d570 con 0x7f9cc81444f0 2022-06-17T12:54:36.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9cc5d9b700 1 -- 172.21.15.36:0/2001295930 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cbc000e90 con 0x7f9cc81444f0 2022-06-17T12:54:36.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9cc5d9b700 1 -- 172.21.15.36:0/2001295930 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cbc00d8c0 con 0x7f9cc81444f0 2022-06-17T12:54:36.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9ccd083700 1 -- 172.21.15.36:0/2001295930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cc81444f0 msgr2=0x7f9cc8144930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:36.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9ccd083700 1 --2- 172.21.15.36:0/2001295930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cc81444f0 0x7f9cc8144930 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f9cc8061170 tx=0x7f9cbc006df0).stop 2022-06-17T12:54:36.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9ccd083700 1 -- 172.21.15.36:0/2001295930 shutdown_connections 2022-06-17T12:54:36.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9ccd083700 1 --2- 172.21.15.36:0/2001295930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cc81444f0 0x7f9cc8144930 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9ccd083700 1 --2- 172.21.15.36:0/2001295930 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cc8062770 0x7f9cc8062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9ccd083700 1 -- 172.21.15.36:0/2001295930 >> 172.21.15.36:0/2001295930 conn(0x7f9cc80b53e0 msgr2=0x7f9cc80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:36.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9ccd083700 1 -- 172.21.15.36:0/2001295930 shutdown_connections 2022-06-17T12:54:36.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.340+0000 7f9ccd083700 1 -- 172.21.15.36:0/2001295930 wait complete. 2022-06-17T12:54:36.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.344+0000 7f9ccd083700 1 Processor -- start 2022-06-17T12:54:36.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.344+0000 7f9ccd083700 1 -- start start 2022-06-17T12:54:36.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.344+0000 7f9ccd083700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9cc8062770 0x7f9cc8159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:36.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.344+0000 7f9ccd083700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cc80ad330 0x7f9cc8146680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:36.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.344+0000 7f9ccd083700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cc81444f0 0x7f9cc8065430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:36.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.344+0000 7f9ccd083700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc814ab00 con 0x7f9cc8062770 2022-06-17T12:54:36.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.344+0000 7f9ccd083700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc8063680 con 0x7f9cc80ad330 2022-06-17T12:54:36.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.344+0000 7f9ccd083700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc814a400 con 0x7f9cc81444f0 2022-06-17T12:54:36.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.344+0000 7f9cc6d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9cc8062770 0x7f9cc8159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:36.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.344+0000 7f9cc6d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9cc8062770 0x7f9cc8159ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37034/0 (socket says 172.21.15.36:37034) 2022-06-17T12:54:36.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.344+0000 7f9cc6d9d700 1 -- 172.21.15.36:0/3138819505 learned_addr learned my addr 172.21.15.36:0/3138819505 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:36.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.344+0000 7f9cc659c700 1 --2- 172.21.15.36:0/3138819505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cc80ad330 0x7f9cc8146680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:36.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.344+0000 7f9cc759e700 1 --2- 172.21.15.36:0/3138819505 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cc81444f0 0x7f9cc8065430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:36.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.344+0000 7f9cc6d9d700 1 -- 172.21.15.36:0/3138819505 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cc81444f0 msgr2=0x7f9cc8065430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:36.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.344+0000 7f9cc6d9d700 1 --2- 172.21.15.36:0/3138819505 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cc81444f0 0x7f9cc8065430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.344+0000 7f9cc6d9d700 1 -- 172.21.15.36:0/3138819505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cc80ad330 msgr2=0x7f9cc8146680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:36.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.344+0000 7f9cc6d9d700 1 --2- 172.21.15.36:0/3138819505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cc80ad330 0x7f9cc8146680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.344+0000 7f9cc6d9d700 1 -- 172.21.15.36:0/3138819505 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cbc00b180 con 0x7f9cc8062770 2022-06-17T12:54:36.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.344+0000 7f9cc6d9d700 1 --2- 172.21.15.36:0/3138819505 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9cc8062770 0x7f9cc8159ab0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7f9cb800c000 tx=0x7f9cb8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:36.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.344+0000 7f9cb7fff700 1 -- 172.21.15.36:0/3138819505 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cb8010780 con 0x7f9cc8062770 2022-06-17T12:54:36.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.344+0000 7f9cb7fff700 1 -- 172.21.15.36:0/3138819505 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb80108e0 con 0x7f9cc8062770 2022-06-17T12:54:36.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.344+0000 7f9cb7fff700 1 -- 172.21.15.36:0/3138819505 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cb8010c40 con 0x7f9cc8062770 2022-06-17T12:54:36.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.344+0000 7f9ccd083700 1 -- 172.21.15.36:0/3138819505 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc814a040 con 0x7f9cc8062770 2022-06-17T12:54:36.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.344+0000 7f9ccd083700 1 -- 172.21.15.36:0/3138819505 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc8063bf0 con 0x7f9cc8062770 2022-06-17T12:54:36.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.348+0000 7f9ccd083700 1 -- 172.21.15.36:0/3138819505 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc814a230 con 0x7f9cc8062770 2022-06-17T12:54:36.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.348+0000 7f9cb7fff700 1 -- 172.21.15.36:0/3138819505 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f9cb801c460 con 0x7f9cc8062770 2022-06-17T12:54:36.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.348+0000 7f9cb7fff700 1 --2- 172.21.15.36:0/3138819505 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9ca4053ad0 0x7f9ca4055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:36.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.348+0000 7f9cb7fff700 1 -- 172.21.15.36:0/3138819505 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9cb806c5e0 con 0x7f9cc8062770 2022-06-17T12:54:36.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.348+0000 7f9cc659c700 1 --2- 172.21.15.36:0/3138819505 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9ca4053ad0 0x7f9ca4055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:36.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.348+0000 7f9cc659c700 1 --2- 172.21.15.36:0/3138819505 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9ca4053ad0 0x7f9ca4055f90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f9cbc018000 tx=0x7f9cbc014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:36.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.356+0000 7f9cb7fff700 1 -- 172.21.15.36:0/3138819505 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9cb8043270 con 0x7f9cc8062770 2022-06-17T12:54:36.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.484+0000 7f9ccd083700 1 -- 172.21.15.36:0/3138819505 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9cc80b1c20 con 0x7f9cc8062770 2022-06-17T12:54:36.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.484+0000 7f9cb7fff700 1 -- 172.21.15.36:0/3138819505 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16300 (secure 0 0 0) 0x7f9cb8046890 con 0x7f9cc8062770 2022-06-17T12:54:36.473 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:36.473 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":92,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:54:35.949366+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:54:34.381157+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-06-17T12:54:35.204631+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:54:36.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.488+0000 7f9ccd083700 1 -- 172.21.15.36:0/3138819505 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9ca4053ad0 msgr2=0x7f9ca4055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:36.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.488+0000 7f9ccd083700 1 --2- 172.21.15.36:0/3138819505 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9ca4053ad0 0x7f9ca4055f90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f9cbc018000 tx=0x7f9cbc014000).stop 2022-06-17T12:54:36.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.488+0000 7f9ccd083700 1 -- 172.21.15.36:0/3138819505 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9cc8062770 msgr2=0x7f9cc8159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:36.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.488+0000 7f9ccd083700 1 --2- 172.21.15.36:0/3138819505 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9cc8062770 0x7f9cc8159ab0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7f9cb800c000 tx=0x7f9cb8006df0).stop 2022-06-17T12:54:36.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.488+0000 7f9ccd083700 1 -- 172.21.15.36:0/3138819505 shutdown_connections 2022-06-17T12:54:36.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.488+0000 7f9ccd083700 1 --2- 172.21.15.36:0/3138819505 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9ca4053ad0 0x7f9ca4055f90 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.488+0000 7f9ccd083700 1 --2- 172.21.15.36:0/3138819505 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cc81444f0 0x7f9cc8065430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.488+0000 7f9ccd083700 1 --2- 172.21.15.36:0/3138819505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cc80ad330 0x7f9cc8146680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.488+0000 7f9ccd083700 1 --2- 172.21.15.36:0/3138819505 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9cc8062770 0x7f9cc8159ab0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.488+0000 7f9ccd083700 1 -- 172.21.15.36:0/3138819505 >> 172.21.15.36:0/3138819505 conn(0x7f9cc80b53e0 msgr2=0x7f9cc80af400 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:36.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.488+0000 7f9ccd083700 1 -- 172.21.15.36:0/3138819505 shutdown_connections 2022-06-17T12:54:36.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.488+0000 7f9ccd083700 1 -- 172.21.15.36:0/3138819505 wait complete. 2022-06-17T12:54:36.487 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-17T12:54:36.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.612+0000 7f514f332700 1 Processor -- start 2022-06-17T12:54:36.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.612+0000 7f514f332700 1 -- start start 2022-06-17T12:54:36.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.612+0000 7f514f332700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51480b3f80 0x7f51480b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:36.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.612+0000 7f514f332700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f51480b4b90 0x7f514813ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:36.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.612+0000 7f514f332700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51481432c0 con 0x7f5148060020 2022-06-17T12:54:36.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.612+0000 7f514f332700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5148062db0 con 0x7f51480b4b90 2022-06-17T12:54:36.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.612+0000 7f514f332700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5148143df0 con 0x7f51480b3f80 2022-06-17T12:54:36.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.616+0000 7f514eb31700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f51480b4b90 0x7f514813ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:36.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.616+0000 7f514eb31700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f51480b4b90 0x7f514813ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37046/0 (socket says 172.21.15.36:37046) 2022-06-17T12:54:36.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.616+0000 7f514eb31700 1 -- 172.21.15.36:0/1467218373 learned_addr learned my addr 172.21.15.36:0/1467218373 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:36.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.616+0000 7f514db2f700 1 --2- 172.21.15.36:0/1467218373 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51480b3f80 0x7f51480b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:36.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.616+0000 7f514eb31700 1 -- 172.21.15.36:0/1467218373 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51480b3f80 msgr2=0x7f51480b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:36.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.616+0000 7f514eb31700 1 --2- 172.21.15.36:0/1467218373 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51480b3f80 0x7f51480b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.616+0000 7f514eb31700 1 -- 172.21.15.36:0/1467218373 >> v1:172.21.15.70:6789/0 conn(0x7f5148060020 legacy=0x7f51480626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:36.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.616+0000 7f514eb31700 1 -- 172.21.15.36:0/1467218373 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5148061f60 con 0x7f51480b4b90 2022-06-17T12:54:36.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.616+0000 7f514eb31700 1 --2- 172.21.15.36:0/1467218373 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f51480b4b90 0x7f514813ca70 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0x7f51400032f0 tx=0x7f5140009290).ready entity=mon.0 client_cookie=a61e4d9486782203 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:36.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.616+0000 7f514d32e700 1 -- 172.21.15.36:0/1467218373 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f514000b8a0 con 0x7f51480b4b90 2022-06-17T12:54:36.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.616+0000 7f514d32e700 1 -- 172.21.15.36:0/1467218373 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514000ba00 con 0x7f51480b4b90 2022-06-17T12:54:36.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.616+0000 7f514d32e700 1 -- 172.21.15.36:0/1467218373 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f514000bb70 con 0x7f51480b4b90 2022-06-17T12:54:36.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.616+0000 7f514f332700 1 -- 172.21.15.36:0/1467218373 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f51480b4b90 msgr2=0x7f514813ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:36.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.616+0000 7f514f332700 1 --2- 172.21.15.36:0/1467218373 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f51480b4b90 0x7f514813ca70 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0x7f51400032f0 tx=0x7f5140009290).stop 2022-06-17T12:54:36.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.616+0000 7f514f332700 1 -- 172.21.15.36:0/1467218373 shutdown_connections 2022-06-17T12:54:36.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.616+0000 7f514f332700 1 --2- 172.21.15.36:0/1467218373 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51480b3f80 0x7f51480b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.616+0000 7f514f332700 1 --2- 172.21.15.36:0/1467218373 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f51480b4b90 0x7f514813ca70 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.616+0000 7f514f332700 1 -- 172.21.15.36:0/1467218373 >> 172.21.15.36:0/1467218373 conn(0x7f514805c550 msgr2=0x7f514805f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:36.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.616+0000 7f514f332700 1 -- 172.21.15.36:0/1467218373 shutdown_connections 2022-06-17T12:54:36.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.616+0000 7f514f332700 1 -- 172.21.15.36:0/1467218373 wait complete. 2022-06-17T12:54:36.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.616+0000 7f514f332700 1 Processor -- start 2022-06-17T12:54:36.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f514f332700 1 -- start start 2022-06-17T12:54:36.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f514f332700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5148060020 0x7f5148152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:36.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f514f332700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51480b3f80 0x7f51480ad330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:36.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f514f332700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51480b4b90 0x7f5148152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:36.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f514f332700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5148116150 con 0x7f5148060020 2022-06-17T12:54:36.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f514f332700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51481527f0 con 0x7f51480b3f80 2022-06-17T12:54:36.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f514f332700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5148115eb0 con 0x7f51480b4b90 2022-06-17T12:54:36.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f514e330700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5148060020 0x7f5148152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:36.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f514e330700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5148060020 0x7f5148152300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37048/0 (socket says 172.21.15.36:37048) 2022-06-17T12:54:36.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f514e330700 1 -- 172.21.15.36:0/4139286568 learned_addr learned my addr 172.21.15.36:0/4139286568 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:36.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f514db2f700 1 --2- 172.21.15.36:0/4139286568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51480b3f80 0x7f51480ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:36.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f514eb31700 1 --2- 172.21.15.36:0/4139286568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51480b4b90 0x7f5148152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:36.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f514e330700 1 -- 172.21.15.36:0/4139286568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51480b4b90 msgr2=0x7f5148152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:36.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f514e330700 1 --2- 172.21.15.36:0/4139286568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51480b4b90 0x7f5148152a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f514e330700 1 -- 172.21.15.36:0/4139286568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51480b3f80 msgr2=0x7f51480ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:36.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f514e330700 1 --2- 172.21.15.36:0/4139286568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51480b3f80 0x7f51480ad330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f514e330700 1 -- 172.21.15.36:0/4139286568 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5140009530 con 0x7f5148060020 2022-06-17T12:54:36.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f514db2f700 1 --2- 172.21.15.36:0/4139286568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51480b3f80 0x7f51480ad330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:54:36.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f514eb31700 1 --2- 172.21.15.36:0/4139286568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51480b4b90 0x7f5148152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:54:36.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f514e330700 1 --2- 172.21.15.36:0/4139286568 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5148060020 0x7f5148152300 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7f51480b4380 tx=0x7f5144007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:36.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f513b7fe700 1 -- 172.21.15.36:0/4139286568 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5144012400 con 0x7f5148060020 2022-06-17T12:54:36.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f513b7fe700 1 -- 172.21.15.36:0/4139286568 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5144012560 con 0x7f5148060020 2022-06-17T12:54:36.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f514f332700 1 -- 172.21.15.36:0/4139286568 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51481559e0 con 0x7f5148060020 2022-06-17T12:54:36.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f513b7fe700 1 -- 172.21.15.36:0/4139286568 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51440128c0 con 0x7f5148060020 2022-06-17T12:54:36.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.620+0000 7f514f332700 1 -- 172.21.15.36:0/4139286568 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51480afe80 con 0x7f5148060020 2022-06-17T12:54:36.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.624+0000 7f514f332700 1 -- 172.21.15.36:0/4139286568 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5148115cf0 con 0x7f5148060020 2022-06-17T12:54:36.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.624+0000 7f513b7fe700 1 -- 172.21.15.36:0/4139286568 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f5144019030 con 0x7f5148060020 2022-06-17T12:54:36.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.624+0000 7f513b7fe700 1 --2- 172.21.15.36:0/4139286568 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f51280539b0 0x7f5128055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:36.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.624+0000 7f513b7fe700 1 -- 172.21.15.36:0/4139286568 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f514406b4f0 con 0x7f5148060020 2022-06-17T12:54:36.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.624+0000 7f514db2f700 1 --2- 172.21.15.36:0/4139286568 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f51280539b0 0x7f5128055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:36.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.624+0000 7f514db2f700 1 --2- 172.21.15.36:0/4139286568 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f51280539b0 0x7f5128055e70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f513c00b000 tx=0x7f513c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:36.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.628+0000 7f513b7fe700 1 -- 172.21.15.36:0/4139286568 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f51440421e0 con 0x7f5148060020 2022-06-17T12:54:36.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.792+0000 7f514f332700 1 -- 172.21.15.36:0/4139286568 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5148057410 con 0x7f5148060020 2022-06-17T12:54:36.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.964+0000 7f513b7fe700 1 -- 172.21.15.36:0/4139286568 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v47) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5144045800 con 0x7f5148060020 2022-06-17T12:54:36.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.968+0000 7f514f332700 1 -- 172.21.15.36:0/4139286568 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f51280539b0 msgr2=0x7f5128055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:36.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.968+0000 7f514f332700 1 --2- 172.21.15.36:0/4139286568 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f51280539b0 0x7f5128055e70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f513c00b000 tx=0x7f513c008040).stop 2022-06-17T12:54:36.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.968+0000 7f514f332700 1 -- 172.21.15.36:0/4139286568 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5148060020 msgr2=0x7f5148152300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:36.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.968+0000 7f514f332700 1 --2- 172.21.15.36:0/4139286568 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5148060020 0x7f5148152300 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7f51480b4380 tx=0x7f5144007c50).stop 2022-06-17T12:54:36.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.968+0000 7f514f332700 1 -- 172.21.15.36:0/4139286568 shutdown_connections 2022-06-17T12:54:36.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.968+0000 7f514f332700 1 --2- 172.21.15.36:0/4139286568 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f51280539b0 0x7f5128055e70 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.968+0000 7f514f332700 1 --2- 172.21.15.36:0/4139286568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51480b4b90 0x7f5148152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.968+0000 7f514f332700 1 --2- 172.21.15.36:0/4139286568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51480b3f80 0x7f51480ad330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.968+0000 7f514f332700 1 --2- 172.21.15.36:0/4139286568 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5148060020 0x7f5148152300 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:36.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.968+0000 7f514f332700 1 -- 172.21.15.36:0/4139286568 >> 172.21.15.36:0/4139286568 conn(0x7f514805c550 msgr2=0x7f51481465d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:36.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.968+0000 7f514f332700 1 -- 172.21.15.36:0/4139286568 shutdown_connections 2022-06-17T12:54:36.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:36.968+0000 7f514f332700 1 -- 172.21.15.36:0/4139286568 wait complete. 2022-06-17T12:54:36.969 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:54:37.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.092+0000 7f16ecf09700 1 Processor -- start 2022-06-17T12:54:37.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.092+0000 7f16ecf09700 1 -- start start 2022-06-17T12:54:37.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16ecf09700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f16e813cb70 0x7f16e8143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:37.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16ecf09700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f16e80aef60 0x7f16e80b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:37.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16ecf09700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f16e80ba2e0 0x7f16e81442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:37.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16ecf09700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16e80af4b0 con 0x7f16e80ba2e0 2022-06-17T12:54:37.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16ecf09700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16e8142f50 con 0x7f16e813cb70 2022-06-17T12:54:37.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16ecf09700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e81431f0 con 0x7f16e80aef60 2022-06-17T12:54:37.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16e7fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f16e80ba2e0 0x7f16e81442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:37.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16e7fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f16e80ba2e0 0x7f16e81442a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37060/0 (socket says 172.21.15.36:37060) 2022-06-17T12:54:37.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16e7fff700 1 -- 172.21.15.36:0/1854776282 learned_addr learned my addr 172.21.15.36:0/1854776282 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:37.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16dffff700 1 --2- 172.21.15.36:0/1854776282 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f16e80aef60 0x7f16e80b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:37.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16e77fe700 1 --2- 172.21.15.36:0/1854776282 >> v2:172.21.15.70:3300/0 conn(0x7f16e813cb70 0x7f16e8143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:37.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16e7fff700 1 -- 172.21.15.36:0/1854776282 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f16e80aef60 msgr2=0x7f16e80b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:37.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16e7fff700 1 --2- 172.21.15.36:0/1854776282 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f16e80aef60 0x7f16e80b9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16e7fff700 1 -- 172.21.15.36:0/1854776282 >> v2:172.21.15.70:3300/0 conn(0x7f16e813cb70 msgr2=0x7f16e8143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:37.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16e7fff700 1 --2- 172.21.15.36:0/1854776282 >> v2:172.21.15.70:3300/0 conn(0x7f16e813cb70 0x7f16e8143690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16e7fff700 1 -- 172.21.15.36:0/1854776282 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e8062e30 con 0x7f16e80ba2e0 2022-06-17T12:54:37.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16dffff700 1 --2- 172.21.15.36:0/1854776282 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f16e80aef60 0x7f16e80b9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:37.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16e7fff700 1 --2- 172.21.15.36:0/1854776282 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f16e80ba2e0 0x7f16e81442a0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7f16e00032f0 tx=0x7f16e0009290).ready entity=mon.0 client_cookie=b91408103fed914e server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:37.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16e6ffd700 1 -- 172.21.15.36:0/1854776282 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16e000b680 con 0x7f16e80ba2e0 2022-06-17T12:54:37.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16e6ffd700 1 -- 172.21.15.36:0/1854776282 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e000b7e0 con 0x7f16e80ba2e0 2022-06-17T12:54:37.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16e6ffd700 1 -- 172.21.15.36:0/1854776282 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16e000b950 con 0x7f16e80ba2e0 2022-06-17T12:54:37.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16ecf09700 1 -- 172.21.15.36:0/1854776282 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f16e80ba2e0 msgr2=0x7f16e81442a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:37.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16ecf09700 1 --2- 172.21.15.36:0/1854776282 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f16e80ba2e0 0x7f16e81442a0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7f16e00032f0 tx=0x7f16e0009290).stop 2022-06-17T12:54:37.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16ecf09700 1 -- 172.21.15.36:0/1854776282 shutdown_connections 2022-06-17T12:54:37.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16ecf09700 1 --2- 172.21.15.36:0/1854776282 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f16e80ba2e0 0x7f16e81442a0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16ecf09700 1 --2- 172.21.15.36:0/1854776282 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f16e80aef60 0x7f16e80b9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16ecf09700 1 --2- 172.21.15.36:0/1854776282 >> v2:172.21.15.70:3300/0 conn(0x7f16e813cb70 0x7f16e8143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16ecf09700 1 -- 172.21.15.36:0/1854776282 >> 172.21.15.36:0/1854776282 conn(0x7f16e805c550 msgr2=0x7f16e80b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:37.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16ecf09700 1 -- 172.21.15.36:0/1854776282 shutdown_connections 2022-06-17T12:54:37.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.096+0000 7f16ecf09700 1 -- 172.21.15.36:0/1854776282 wait complete. 2022-06-17T12:54:37.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.100+0000 7f16ecf09700 1 Processor -- start 2022-06-17T12:54:37.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.100+0000 7f16ecf09700 1 -- start start 2022-06-17T12:54:37.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.104+0000 7f16ecf09700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f16e80aef60 0x7f16e8144440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:37.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.104+0000 7f16ecf09700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f16e80ba2e0 0x7f16e81514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:37.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.104+0000 7f16ecf09700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16e813cb70 0x7f16e81519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:37.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.104+0000 7f16ecf09700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e8112a50 con 0x7f16e80ba2e0 2022-06-17T12:54:37.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.104+0000 7f16ecf09700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e8112cf0 con 0x7f16e813cb70 2022-06-17T12:54:37.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.104+0000 7f16ecf09700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e8112f90 con 0x7f16e80aef60 2022-06-17T12:54:37.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.104+0000 7f16dffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f16e80ba2e0 0x7f16e81514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:37.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.104+0000 7f16dffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f16e80ba2e0 0x7f16e81514f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37064/0 (socket says 172.21.15.36:37064) 2022-06-17T12:54:37.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.104+0000 7f16dffff700 1 -- 172.21.15.36:0/1249494684 learned_addr learned my addr 172.21.15.36:0/1249494684 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:37.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.104+0000 7f16e77fe700 1 --2- 172.21.15.36:0/1249494684 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f16e80aef60 0x7f16e8144440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:37.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.104+0000 7f16e7fff700 1 --2- 172.21.15.36:0/1249494684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16e813cb70 0x7f16e81519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:37.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.104+0000 7f16dffff700 1 -- 172.21.15.36:0/1249494684 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f16e80aef60 msgr2=0x7f16e8144440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:37.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.104+0000 7f16dffff700 1 --2- 172.21.15.36:0/1249494684 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f16e80aef60 0x7f16e8144440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.104+0000 7f16dffff700 1 -- 172.21.15.36:0/1249494684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16e813cb70 msgr2=0x7f16e81519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:37.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.104+0000 7f16dffff700 1 --2- 172.21.15.36:0/1249494684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16e813cb70 0x7f16e81519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.104+0000 7f16dffff700 1 -- 172.21.15.36:0/1249494684 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d000b180 con 0x7f16e80ba2e0 2022-06-17T12:54:37.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.104+0000 7f16dffff700 1 --2- 172.21.15.36:0/1249494684 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f16e80ba2e0 0x7f16e81514f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7f16d000c000 tx=0x7f16d0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:37.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.104+0000 7f16e57fa700 1 -- 172.21.15.36:0/1249494684 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16d000c7e0 con 0x7f16e80ba2e0 2022-06-17T12:54:37.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.104+0000 7f16e57fa700 1 -- 172.21.15.36:0/1249494684 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d000c940 con 0x7f16e80ba2e0 2022-06-17T12:54:37.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.104+0000 7f16e57fa700 1 -- 172.21.15.36:0/1249494684 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16d000cca0 con 0x7f16e80ba2e0 2022-06-17T12:54:37.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.104+0000 7f16ecf09700 1 -- 172.21.15.36:0/1249494684 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e0009530 con 0x7f16e80ba2e0 2022-06-17T12:54:37.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.104+0000 7f16ecf09700 1 -- 172.21.15.36:0/1249494684 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e8144930 con 0x7f16e80ba2e0 2022-06-17T12:54:37.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.104+0000 7f16ecf09700 1 -- 172.21.15.36:0/1249494684 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16e80b07c0 con 0x7f16e80ba2e0 2022-06-17T12:54:37.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.108+0000 7f16e57fa700 1 -- 172.21.15.36:0/1249494684 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f16d001c460 con 0x7f16e80ba2e0 2022-06-17T12:54:37.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.108+0000 7f16e57fa700 1 --2- 172.21.15.36:0/1249494684 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f16cc053a60 0x7f16cc055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:37.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.108+0000 7f16e57fa700 1 -- 172.21.15.36:0/1249494684 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f16d006c520 con 0x7f16e80ba2e0 2022-06-17T12:54:37.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.108+0000 7f16e77fe700 1 --2- 172.21.15.36:0/1249494684 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f16cc053a60 0x7f16cc055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:37.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.108+0000 7f16e77fe700 1 --2- 172.21.15.36:0/1249494684 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f16cc053a60 0x7f16cc055f20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f16d800b000 tx=0x7f16d8006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:37.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.116+0000 7f16e57fa700 1 -- 172.21.15.36:0/1249494684 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f16d00432c0 con 0x7f16e80ba2e0 2022-06-17T12:54:37.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.272+0000 7f16ecf09700 1 -- 172.21.15.36:0/1249494684 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16e8057410 con 0x7f16e80ba2e0 2022-06-17T12:54:37.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.276+0000 7f16e57fa700 1 -- 172.21.15.36:0/1249494684 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f16d000ce00 con 0x7f16e80ba2e0 2022-06-17T12:54:37.263 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:37.264 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":47,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/138643404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":138643404},{"type":"v1","addr":"172.21.15.70:6833","nonce":138643404}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":46}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:54:35.949730+0000","modified":"2022-06-17T12:54:36.953427+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7074},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7074":{"gid":7074,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/3267343301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3267343301},{"type":"v1","addr":"172.21.15.36:6835","nonce":3267343301}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-06-17T12:54:37.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.276+0000 7f16ecf09700 1 -- 172.21.15.36:0/1249494684 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f16cc053a60 msgr2=0x7f16cc055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:37.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.276+0000 7f16ecf09700 1 --2- 172.21.15.36:0/1249494684 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f16cc053a60 0x7f16cc055f20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f16d800b000 tx=0x7f16d8006df0).stop 2022-06-17T12:54:37.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.276+0000 7f16ecf09700 1 -- 172.21.15.36:0/1249494684 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f16e80ba2e0 msgr2=0x7f16e81514f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:37.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.276+0000 7f16ecf09700 1 --2- 172.21.15.36:0/1249494684 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f16e80ba2e0 0x7f16e81514f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7f16d000c000 tx=0x7f16d0006df0).stop 2022-06-17T12:54:37.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.276+0000 7f16ecf09700 1 -- 172.21.15.36:0/1249494684 shutdown_connections 2022-06-17T12:54:37.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.276+0000 7f16ecf09700 1 --2- 172.21.15.36:0/1249494684 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f16cc053a60 0x7f16cc055f20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.276+0000 7f16ecf09700 1 --2- 172.21.15.36:0/1249494684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16e813cb70 0x7f16e81519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.276+0000 7f16ecf09700 1 --2- 172.21.15.36:0/1249494684 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f16e80ba2e0 0x7f16e81514f0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.276+0000 7f16ecf09700 1 --2- 172.21.15.36:0/1249494684 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f16e80aef60 0x7f16e8144440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.280+0000 7f16ecf09700 1 -- 172.21.15.36:0/1249494684 >> 172.21.15.36:0/1249494684 conn(0x7f16e805c550 msgr2=0x7f16e81400b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:37.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.280+0000 7f16ecf09700 1 -- 172.21.15.36:0/1249494684 shutdown_connections 2022-06-17T12:54:37.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.280+0000 7f16ecf09700 1 -- 172.21.15.36:0/1249494684 wait complete. 2022-06-17T12:54:37.272 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 47 2022-06-17T12:54:37.283 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:54:37.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.400+0000 7f8fe67ee700 1 Processor -- start 2022-06-17T12:54:37.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.400+0000 7f8fe67ee700 1 -- start start 2022-06-17T12:54:37.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.400+0000 7f8fe67ee700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fe0063730 0x7f8fe00b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:37.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.400+0000 7f8fe67ee700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fe014ad50 con 0x7f8fe014a8f0 2022-06-17T12:54:37.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.400+0000 7f8fe67ee700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fe0063b10 con 0x7f8fe0064f70 2022-06-17T12:54:37.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.400+0000 7f8fe67ee700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe00b2e80 con 0x7f8fe0063730 2022-06-17T12:54:37.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.400+0000 7f8fe4feb700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f8fe014a8f0 0x7f8fe0064860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54602/0 (socket says 172.21.15.36:54602) 2022-06-17T12:54:37.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.400+0000 7f8fe4feb700 1 -- 172.21.15.36:0/426116801 learned_addr learned my addr 172.21.15.36:0/426116801 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:37.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.400+0000 7f8fe57ec700 1 --2- 172.21.15.36:0/426116801 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fe0063730 0x7f8fe00b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:37.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.400+0000 7f8fd3fff700 1 -- 172.21.15.36:0/426116801 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106322365 0 0) 0x7f8fe014ad50 con 0x7f8fe014a8f0 2022-06-17T12:54:37.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.400+0000 7f8fd3fff700 1 -- 172.21.15.36:0/426116801 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fc4003260 con 0x7f8fe014a8f0 2022-06-17T12:54:37.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.400+0000 7f8fe57ec700 1 -- 172.21.15.36:0/426116801 >> v1:172.21.15.70:6789/0 conn(0x7f8fe0064f70 legacy=0x7f8fe0146790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:37.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.400+0000 7f8fe57ec700 1 -- 172.21.15.36:0/426116801 >> v1:172.21.15.36:6789/0 conn(0x7f8fe014a8f0 legacy=0x7f8fe0064860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:37.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.400+0000 7f8fe57ec700 1 -- 172.21.15.36:0/426116801 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe005ffb0 con 0x7f8fe0063730 2022-06-17T12:54:37.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.400+0000 7f8fe57ec700 1 --2- 172.21.15.36:0/426116801 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fe0063730 0x7f8fe00b3bf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f8fdc0032f0 tx=0x7f8fdc0091d0).ready entity=mon.2 client_cookie=9060eb5254719cfd server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:37.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.400+0000 7f8fd3fff700 1 -- 172.21.15.36:0/426116801 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fdc00b6c0 con 0x7f8fe0063730 2022-06-17T12:54:37.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.404+0000 7f8fd3fff700 1 -- 172.21.15.36:0/426116801 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fdc00b820 con 0x7f8fe0063730 2022-06-17T12:54:37.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.404+0000 7f8fd3fff700 1 -- 172.21.15.36:0/426116801 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fdc00b990 con 0x7f8fe0063730 2022-06-17T12:54:37.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.404+0000 7f8fe67ee700 1 -- 172.21.15.36:0/426116801 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fe0063730 msgr2=0x7f8fe00b3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:37.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.404+0000 7f8fe67ee700 1 --2- 172.21.15.36:0/426116801 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fe0063730 0x7f8fe00b3bf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f8fdc0032f0 tx=0x7f8fdc0091d0).stop 2022-06-17T12:54:37.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.404+0000 7f8fe67ee700 1 -- 172.21.15.36:0/426116801 shutdown_connections 2022-06-17T12:54:37.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.404+0000 7f8fe67ee700 1 --2- 172.21.15.36:0/426116801 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fe0063730 0x7f8fe00b3bf0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.404+0000 7f8fe67ee700 1 -- 172.21.15.36:0/426116801 >> 172.21.15.36:0/426116801 conn(0x7f8fe00614b0 msgr2=0x7f8fe00618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:37.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.404+0000 7f8fe67ee700 1 -- 172.21.15.36:0/426116801 shutdown_connections 2022-06-17T12:54:37.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.404+0000 7f8fe67ee700 1 -- 172.21.15.36:0/426116801 wait complete. 2022-06-17T12:54:37.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.404+0000 7f8fe67ee700 1 Processor -- start 2022-06-17T12:54:37.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.404+0000 7f8fe67ee700 1 -- start start 2022-06-17T12:54:37.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.404+0000 7f8fe67ee700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fe0063730 0x7f8fe00b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:37.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.404+0000 7f8fe67ee700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fe0064f70 0x7f8fe00aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:37.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.404+0000 7f8fe67ee700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fe014a8f0 0x7f8fe015a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:37.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.404+0000 7f8fe67ee700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe00b0340 con 0x7f8fe014a8f0 2022-06-17T12:54:37.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.404+0000 7f8fe67ee700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe00b05e0 con 0x7f8fe0063730 2022-06-17T12:54:37.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.404+0000 7f8fe67ee700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe00b0880 con 0x7f8fe0064f70 2022-06-17T12:54:37.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.404+0000 7f8fe5fed700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fe014a8f0 0x7f8fe015a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:37.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.408+0000 7f8fe5fed700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fe014a8f0 0x7f8fe015a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37078/0 (socket says 172.21.15.36:37078) 2022-06-17T12:54:37.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.408+0000 7f8fe5fed700 1 -- 172.21.15.36:0/2434918645 learned_addr learned my addr 172.21.15.36:0/2434918645 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:37.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.408+0000 7f8fe57ec700 1 --2- 172.21.15.36:0/2434918645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fe0063730 0x7f8fe00b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:37.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.408+0000 7f8fe4feb700 1 --2- 172.21.15.36:0/2434918645 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fe0064f70 0x7f8fe00aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:37.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.408+0000 7f8fe5fed700 1 -- 172.21.15.36:0/2434918645 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fe0064f70 msgr2=0x7f8fe00aeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:37.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.408+0000 7f8fe5fed700 1 --2- 172.21.15.36:0/2434918645 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fe0064f70 0x7f8fe00aeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.408+0000 7f8fe5fed700 1 -- 172.21.15.36:0/2434918645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fe0063730 msgr2=0x7f8fe00b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:37.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.408+0000 7f8fe5fed700 1 --2- 172.21.15.36:0/2434918645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fe0063730 0x7f8fe00b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.408+0000 7f8fe5fed700 1 -- 172.21.15.36:0/2434918645 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fdc009470 con 0x7f8fe014a8f0 2022-06-17T12:54:37.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.408+0000 7f8fe5fed700 1 --2- 172.21.15.36:0/2434918645 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fe014a8f0 0x7f8fe015a260 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7f8fe0063e30 tx=0x7f8fd8007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:37.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.408+0000 7f8fd2ffd700 1 -- 172.21.15.36:0/2434918645 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fd8013450 con 0x7f8fe014a8f0 2022-06-17T12:54:37.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.408+0000 7f8fd2ffd700 1 -- 172.21.15.36:0/2434918645 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd80135b0 con 0x7f8fe014a8f0 2022-06-17T12:54:37.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.408+0000 7f8fe67ee700 1 -- 172.21.15.36:0/2434918645 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe00b7000 con 0x7f8fe014a8f0 2022-06-17T12:54:37.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.408+0000 7f8fd2ffd700 1 -- 172.21.15.36:0/2434918645 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fd8013910 con 0x7f8fe014a8f0 2022-06-17T12:54:37.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.408+0000 7f8fe67ee700 1 -- 172.21.15.36:0/2434918645 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe0159cb0 con 0x7f8fe014a8f0 2022-06-17T12:54:37.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.408+0000 7f8fe67ee700 1 -- 172.21.15.36:0/2434918645 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fe0057410 con 0x7f8fe014a8f0 2022-06-17T12:54:37.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.408+0000 7f8fd2ffd700 1 -- 172.21.15.36:0/2434918645 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8fd8010070 con 0x7f8fe014a8f0 2022-06-17T12:54:37.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.408+0000 7f8fd2ffd700 1 --2- 172.21.15.36:0/2434918645 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8fbc053ad0 0x7f8fbc055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:37.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.412+0000 7f8fd2ffd700 1 -- 172.21.15.36:0/2434918645 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8fd806b680 con 0x7f8fe014a8f0 2022-06-17T12:54:37.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.412+0000 7f8fe57ec700 1 --2- 172.21.15.36:0/2434918645 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8fbc053ad0 0x7f8fbc055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:37.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.412+0000 7f8fe57ec700 1 --2- 172.21.15.36:0/2434918645 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8fbc053ad0 0x7f8fbc055f90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f8fdc01b000 tx=0x7f8fdc016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:37.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.420+0000 7f8fd2ffd700 1 -- 172.21.15.36:0/2434918645 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8fd80423e0 con 0x7f8fe014a8f0 2022-06-17T12:54:37.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.540+0000 7f8fe67ee700 1 -- 172.21.15.36:0/2434918645 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8fe005d820 con 0x7f8fe014a8f0 2022-06-17T12:54:37.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.540+0000 7f8fd2ffd700 1 -- 172.21.15.36:0/2434918645 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f8fd8045a00 con 0x7f8fe014a8f0 2022-06-17T12:54:37.531 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:37.531 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":93,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:54:36.952445+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:54:34.381157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-06-17T12:54:35.204631+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:54:37.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.544+0000 7f8fe67ee700 1 -- 172.21.15.36:0/2434918645 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8fbc053ad0 msgr2=0x7f8fbc055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:37.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.544+0000 7f8fe67ee700 1 --2- 172.21.15.36:0/2434918645 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8fbc053ad0 0x7f8fbc055f90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f8fdc01b000 tx=0x7f8fdc016000).stop 2022-06-17T12:54:37.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.544+0000 7f8fe67ee700 1 -- 172.21.15.36:0/2434918645 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fe014a8f0 msgr2=0x7f8fe015a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:37.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.544+0000 7f8fe67ee700 1 --2- 172.21.15.36:0/2434918645 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fe014a8f0 0x7f8fe015a260 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7f8fe0063e30 tx=0x7f8fd8007b90).stop 2022-06-17T12:54:37.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.544+0000 7f8fe67ee700 1 -- 172.21.15.36:0/2434918645 shutdown_connections 2022-06-17T12:54:37.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.544+0000 7f8fe67ee700 1 --2- 172.21.15.36:0/2434918645 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8fbc053ad0 0x7f8fbc055f90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.544+0000 7f8fe67ee700 1 --2- 172.21.15.36:0/2434918645 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fe014a8f0 0x7f8fe015a260 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.544+0000 7f8fe67ee700 1 --2- 172.21.15.36:0/2434918645 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fe0064f70 0x7f8fe00aeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.544+0000 7f8fe67ee700 1 --2- 172.21.15.36:0/2434918645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fe0063730 0x7f8fe00b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.544+0000 7f8fe67ee700 1 -- 172.21.15.36:0/2434918645 >> 172.21.15.36:0/2434918645 conn(0x7f8fe00614b0 msgr2=0x7f8fe0065c60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:37.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.544+0000 7f8fe67ee700 1 -- 172.21.15.36:0/2434918645 shutdown_connections 2022-06-17T12:54:37.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.544+0000 7f8fe67ee700 1 -- 172.21.15.36:0/2434918645 wait complete. 2022-06-17T12:54:37.544 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:54:37.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.668+0000 7f76361e3700 1 Processor -- start 2022-06-17T12:54:37.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.668+0000 7f76361e3700 1 -- start start 2022-06-17T12:54:37.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f76361e3700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7630062270 0x7f7630142f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:37.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f76361e3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76300b0cb0 0x7f7630143770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:37.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f76361e3700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7630143c60 0x7f7630146970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:37.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f76361e3700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76300b6650 con 0x7f7630062270 2022-06-17T12:54:37.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f76361e3700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76300b44b0 con 0x7f7630143c60 2022-06-17T12:54:37.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f76361e3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7630062650 con 0x7f76300b0cb0 2022-06-17T12:54:37.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f762f7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7630062270 0x7f7630142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:37.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f762f7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7630062270 0x7f7630142f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37086/0 (socket says 172.21.15.36:37086) 2022-06-17T12:54:37.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f762f7fe700 1 -- 172.21.15.36:0/2219299427 learned_addr learned my addr 172.21.15.36:0/2219299427 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:37.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f762effd700 1 --2- 172.21.15.36:0/2219299427 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76300b0cb0 0x7f7630143770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:37.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f762ffff700 1 --2- 172.21.15.36:0/2219299427 >> v2:172.21.15.70:3300/0 conn(0x7f7630143c60 0x7f7630146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:37.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f762f7fe700 1 -- 172.21.15.36:0/2219299427 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76300b0cb0 msgr2=0x7f7630143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:37.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f762f7fe700 1 --2- 172.21.15.36:0/2219299427 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76300b0cb0 0x7f7630143770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f762f7fe700 1 -- 172.21.15.36:0/2219299427 >> v2:172.21.15.70:3300/0 conn(0x7f7630143c60 msgr2=0x7f7630146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:37.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f762f7fe700 1 --2- 172.21.15.36:0/2219299427 >> v2:172.21.15.70:3300/0 conn(0x7f7630143c60 0x7f7630146970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f762f7fe700 1 -- 172.21.15.36:0/2219299427 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76300b4210 con 0x7f7630062270 2022-06-17T12:54:37.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f762ffff700 1 --2- 172.21.15.36:0/2219299427 >> v2:172.21.15.70:3300/0 conn(0x7f7630143c60 0x7f7630146970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:37.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f762f7fe700 1 --2- 172.21.15.36:0/2219299427 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7630062270 0x7f7630142f20 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7f76300ba8b0 tx=0x7f7624006eb0).ready entity=mon.0 client_cookie=93d52b3250bbb748 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:37.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f762e7fc700 1 -- 172.21.15.36:0/2219299427 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f762400e610 con 0x7f7630062270 2022-06-17T12:54:37.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f762e7fc700 1 -- 172.21.15.36:0/2219299427 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7624000e90 con 0x7f7630062270 2022-06-17T12:54:37.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f762e7fc700 1 -- 172.21.15.36:0/2219299427 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f762400e9a0 con 0x7f7630062270 2022-06-17T12:54:37.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f76361e3700 1 -- 172.21.15.36:0/2219299427 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7630062270 msgr2=0x7f7630142f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:37.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f76361e3700 1 --2- 172.21.15.36:0/2219299427 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7630062270 0x7f7630142f20 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7f76300ba8b0 tx=0x7f7624006eb0).stop 2022-06-17T12:54:37.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f76361e3700 1 -- 172.21.15.36:0/2219299427 shutdown_connections 2022-06-17T12:54:37.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f76361e3700 1 --2- 172.21.15.36:0/2219299427 >> v2:172.21.15.70:3300/0 conn(0x7f7630143c60 0x7f7630146970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f76361e3700 1 --2- 172.21.15.36:0/2219299427 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76300b0cb0 0x7f7630143770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f76361e3700 1 --2- 172.21.15.36:0/2219299427 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7630062270 0x7f7630142f20 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f76361e3700 1 -- 172.21.15.36:0/2219299427 >> 172.21.15.36:0/2219299427 conn(0x7f76300b9870 msgr2=0x7f76300b9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:37.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f76361e3700 1 -- 172.21.15.36:0/2219299427 shutdown_connections 2022-06-17T12:54:37.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.672+0000 7f76361e3700 1 -- 172.21.15.36:0/2219299427 wait complete. 2022-06-17T12:54:37.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.676+0000 7f76361e3700 1 Processor -- start 2022-06-17T12:54:37.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.676+0000 7f76361e3700 1 -- start start 2022-06-17T12:54:37.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.676+0000 7f76361e3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7630062270 0x7f763013cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:37.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.676+0000 7f76361e3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7630143c60 0x7f763013e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:37.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.676+0000 7f76361e3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7630146c50 0x7f7630145070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:37.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.676+0000 7f76361e3700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763013e9e0 con 0x7f7630146c50 2022-06-17T12:54:37.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.676+0000 7f76361e3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763013ec70 con 0x7f7630062270 2022-06-17T12:54:37.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.676+0000 7f76361e3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7630144bd0 con 0x7f7630143c60 2022-06-17T12:54:37.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.676+0000 7f762ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7630146c50 0x7f7630145070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:37.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.676+0000 7f762ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7630146c50 0x7f7630145070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37092/0 (socket says 172.21.15.36:37092) 2022-06-17T12:54:37.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.676+0000 7f762ffff700 1 -- 172.21.15.36:0/159723348 learned_addr learned my addr 172.21.15.36:0/159723348 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:37.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.676+0000 7f762f7fe700 1 --2- 172.21.15.36:0/159723348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7630062270 0x7f763013cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:37.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.676+0000 7f762effd700 1 --2- 172.21.15.36:0/159723348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7630143c60 0x7f763013e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:37.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.676+0000 7f762ffff700 1 -- 172.21.15.36:0/159723348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7630143c60 msgr2=0x7f763013e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:37.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.676+0000 7f762ffff700 1 --2- 172.21.15.36:0/159723348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7630143c60 0x7f763013e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.676+0000 7f762ffff700 1 -- 172.21.15.36:0/159723348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7630062270 msgr2=0x7f763013cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:37.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.676+0000 7f762ffff700 1 --2- 172.21.15.36:0/159723348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7630062270 0x7f763013cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.676+0000 7f762ffff700 1 -- 172.21.15.36:0/159723348 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7628009470 con 0x7f7630146c50 2022-06-17T12:54:37.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.680+0000 7f762ffff700 1 --2- 172.21.15.36:0/159723348 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7630146c50 0x7f7630145070 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7f762800c000 tx=0x7f76280091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:37.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.680+0000 7f762cff9700 1 -- 172.21.15.36:0/159723348 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f762800c850 con 0x7f7630146c50 2022-06-17T12:54:37.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.680+0000 7f762cff9700 1 -- 172.21.15.36:0/159723348 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f762800c9b0 con 0x7f7630146c50 2022-06-17T12:54:37.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.680+0000 7f76361e3700 1 -- 172.21.15.36:0/159723348 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f762400b1e0 con 0x7f7630146c50 2022-06-17T12:54:37.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.680+0000 7f762cff9700 1 -- 172.21.15.36:0/159723348 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f762800cd10 con 0x7f7630146c50 2022-06-17T12:54:37.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.680+0000 7f76361e3700 1 -- 172.21.15.36:0/159723348 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763013e290 con 0x7f7630146c50 2022-06-17T12:54:37.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.680+0000 7f76361e3700 1 -- 172.21.15.36:0/159723348 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7630144a30 con 0x7f7630146c50 2022-06-17T12:54:37.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.680+0000 7f762cff9700 1 -- 172.21.15.36:0/159723348 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f7628022070 con 0x7f7630146c50 2022-06-17T12:54:37.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.680+0000 7f762cff9700 1 --2- 172.21.15.36:0/159723348 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7610053a40 0x7f7610055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:37.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.680+0000 7f762cff9700 1 -- 172.21.15.36:0/159723348 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f762806c910 con 0x7f7630146c50 2022-06-17T12:54:37.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.680+0000 7f762f7fe700 1 --2- 172.21.15.36:0/159723348 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7610053a40 0x7f7610055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:37.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.684+0000 7f762f7fe700 1 --2- 172.21.15.36:0/159723348 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7610053a40 0x7f7610055f00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f762401a000 tx=0x7f7624015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:37.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.688+0000 7f762cff9700 1 -- 172.21.15.36:0/159723348 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7630144a30 con 0x7f7630146c50 2022-06-17T12:54:37.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.856+0000 7f76361e3700 1 -- 172.21.15.36:0/159723348 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f7630144a30 con 0x7f7630146c50 2022-06-17T12:54:37.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.864+0000 7f762cff9700 1 -- 172.21.15.36:0/159723348 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v32) v1 ==== 184+0+0 (secure 0 0 0) 0x7f7630144a30 con 0x7f7630146c50 2022-06-17T12:54:37.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.864+0000 7f76361e3700 1 -- 172.21.15.36:0/159723348 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7610053a40 msgr2=0x7f7610055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:37.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.864+0000 7f76361e3700 1 --2- 172.21.15.36:0/159723348 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7610053a40 0x7f7610055f00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f762401a000 tx=0x7f7624015000).stop 2022-06-17T12:54:37.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.864+0000 7f76361e3700 1 -- 172.21.15.36:0/159723348 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7630146c50 msgr2=0x7f7630145070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:37.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.864+0000 7f76361e3700 1 --2- 172.21.15.36:0/159723348 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7630146c50 0x7f7630145070 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7f762800c000 tx=0x7f76280091d0).stop 2022-06-17T12:54:37.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.864+0000 7f76361e3700 1 -- 172.21.15.36:0/159723348 shutdown_connections 2022-06-17T12:54:37.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.864+0000 7f76361e3700 1 --2- 172.21.15.36:0/159723348 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7610053a40 0x7f7610055f00 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.864+0000 7f76361e3700 1 --2- 172.21.15.36:0/159723348 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7630146c50 0x7f7630145070 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.864+0000 7f76361e3700 1 --2- 172.21.15.36:0/159723348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7630143c60 0x7f763013e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.864+0000 7f76361e3700 1 --2- 172.21.15.36:0/159723348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7630062270 0x7f763013cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.864+0000 7f76361e3700 1 -- 172.21.15.36:0/159723348 >> 172.21.15.36:0/159723348 conn(0x7f76300b9870 msgr2=0x7f76300b76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:37.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.864+0000 7f76361e3700 1 -- 172.21.15.36:0/159723348 shutdown_connections 2022-06-17T12:54:37.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.864+0000 7f76361e3700 1 -- 172.21.15.36:0/159723348 wait complete. 2022-06-17T12:54:37.858 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.0 2022-06-17T12:54:37.867 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:54:37.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.992+0000 7f0ee07c2700 1 Processor -- start 2022-06-17T12:54:37.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.992+0000 7f0ee07c2700 1 -- start start 2022-06-17T12:54:37.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0ee07c2700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0ed8142a20 0x7f0ed8060780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:37.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0ee07c2700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0ed8060c70 0x7f0ed813fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:37.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0ee07c2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed80b2340 0x7f0ed8140340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:37.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0ee07c2700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed80b3520 con 0x7f0ed8142a20 2022-06-17T12:54:37.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0ee07c2700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed80b37c0 con 0x7f0ed8060c70 2022-06-17T12:54:37.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0ee07c2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed8062f20 con 0x7f0ed80b2340 2022-06-17T12:54:37.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0ede55e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0ed8142a20 0x7f0ed8060780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:37.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0ede55e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0ed8142a20 0x7f0ed8060780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37098/0 (socket says 172.21.15.36:37098) 2022-06-17T12:54:37.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0ede55e700 1 -- 172.21.15.36:0/143380677 learned_addr learned my addr 172.21.15.36:0/143380677 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:37.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0eded5f700 1 --2- 172.21.15.36:0/143380677 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed80b2340 0x7f0ed8140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:37.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0eddd5d700 1 --2- 172.21.15.36:0/143380677 >> v2:172.21.15.70:3300/0 conn(0x7f0ed8060c70 0x7f0ed813fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:37.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0eded5f700 1 -- 172.21.15.36:0/143380677 >> v2:172.21.15.70:3300/0 conn(0x7f0ed8060c70 msgr2=0x7f0ed813fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:37.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0eded5f700 1 --2- 172.21.15.36:0/143380677 >> v2:172.21.15.70:3300/0 conn(0x7f0ed8060c70 0x7f0ed813fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0eded5f700 1 -- 172.21.15.36:0/143380677 >> v2:172.21.15.36:3300/0 conn(0x7f0ed8142a20 msgr2=0x7f0ed8060780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:37.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0eded5f700 1 --2- 172.21.15.36:0/143380677 >> v2:172.21.15.36:3300/0 conn(0x7f0ed8142a20 0x7f0ed8060780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0eded5f700 1 -- 172.21.15.36:0/143380677 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed80acf00 con 0x7f0ed80b2340 2022-06-17T12:54:37.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0ede55e700 1 --2- 172.21.15.36:0/143380677 >> v2:172.21.15.36:3300/0 conn(0x7f0ed8142a20 0x7f0ed8060780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:37.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0eded5f700 1 --2- 172.21.15.36:0/143380677 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed80b2340 0x7f0ed8140340 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f0ed0002f70 tx=0x7f0ed0006df0).ready entity=mon.2 client_cookie=cf4930c35f1f8820 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:37.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0edd55c700 1 -- 172.21.15.36:0/143380677 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ed000d4a0 con 0x7f0ed80b2340 2022-06-17T12:54:37.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0edd55c700 1 -- 172.21.15.36:0/143380677 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed0000e90 con 0x7f0ed80b2340 2022-06-17T12:54:37.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0edd55c700 1 -- 172.21.15.36:0/143380677 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ed000d7f0 con 0x7f0ed80b2340 2022-06-17T12:54:37.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0ee07c2700 1 -- 172.21.15.36:0/143380677 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed80b2340 msgr2=0x7f0ed8140340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:37.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0ee07c2700 1 --2- 172.21.15.36:0/143380677 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed80b2340 0x7f0ed8140340 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f0ed0002f70 tx=0x7f0ed0006df0).stop 2022-06-17T12:54:37.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0ee07c2700 1 -- 172.21.15.36:0/143380677 shutdown_connections 2022-06-17T12:54:37.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0ee07c2700 1 --2- 172.21.15.36:0/143380677 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed80b2340 0x7f0ed8140340 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0ee07c2700 1 --2- 172.21.15.36:0/143380677 >> v2:172.21.15.70:3300/0 conn(0x7f0ed8060c70 0x7f0ed813fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0ee07c2700 1 --2- 172.21.15.36:0/143380677 >> v2:172.21.15.36:3300/0 conn(0x7f0ed8142a20 0x7f0ed8060780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:37.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0ee07c2700 1 -- 172.21.15.36:0/143380677 >> 172.21.15.36:0/143380677 conn(0x7f0ed80b6870 msgr2=0x7f0ed80b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:38.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0ee07c2700 1 -- 172.21.15.36:0/143380677 shutdown_connections 2022-06-17T12:54:38.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0ee07c2700 1 -- 172.21.15.36:0/143380677 wait complete. 2022-06-17T12:54:38.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0ee07c2700 1 Processor -- start 2022-06-17T12:54:38.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:37.996+0000 7f0ee07c2700 1 -- start start 2022-06-17T12:54:38.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0ee07c2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ed8060c70 0x7f0ed80ae610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:38.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0ee07c2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed80b2340 0x7f0ed813a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:38.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0ee07c2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ed8142a20 0x7f0ed80a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:38.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0ee07c2700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed809fa50 con 0x7f0ed8060c70 2022-06-17T12:54:38.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0ee07c2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed813c3d0 con 0x7f0ed8142a20 2022-06-17T12:54:38.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0ee07c2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed813c130 con 0x7f0ed80b2340 2022-06-17T12:54:38.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0ede55e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ed8060c70 0x7f0ed80ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:38.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0ede55e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ed8060c70 0x7f0ed80ae610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37104/0 (socket says 172.21.15.36:37104) 2022-06-17T12:54:38.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0ede55e700 1 -- 172.21.15.36:0/1901643315 learned_addr learned my addr 172.21.15.36:0/1901643315 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:38.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0eddd5d700 1 --2- 172.21.15.36:0/1901643315 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed80b2340 0x7f0ed813a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:38.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0eded5f700 1 --2- 172.21.15.36:0/1901643315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ed8142a20 0x7f0ed80a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:38.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0ede55e700 1 -- 172.21.15.36:0/1901643315 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed80b2340 msgr2=0x7f0ed813a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:38.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0ede55e700 1 --2- 172.21.15.36:0/1901643315 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed80b2340 0x7f0ed813a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:38.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0ede55e700 1 -- 172.21.15.36:0/1901643315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ed8142a20 msgr2=0x7f0ed80a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:38.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0ede55e700 1 --2- 172.21.15.36:0/1901643315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ed8142a20 0x7f0ed80a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:38.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0ede55e700 1 -- 172.21.15.36:0/1901643315 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec8009530 con 0x7f0ed8060c70 2022-06-17T12:54:38.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0ede55e700 1 --2- 172.21.15.36:0/1901643315 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ed8060c70 0x7f0ed80ae610 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7f0ec8006b60 tx=0x7f0ec8009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:38.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0ed77fe700 1 -- 172.21.15.36:0/1901643315 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ec800c9d0 con 0x7f0ed8060c70 2022-06-17T12:54:38.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0ed77fe700 1 -- 172.21.15.36:0/1901643315 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec800cb30 con 0x7f0ed8060c70 2022-06-17T12:54:38.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0ee07c2700 1 -- 172.21.15.36:0/1901643315 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed000b180 con 0x7f0ed8060c70 2022-06-17T12:54:38.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0ed77fe700 1 -- 172.21.15.36:0/1901643315 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ec800ce90 con 0x7f0ed8060c70 2022-06-17T12:54:38.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0ee07c2700 1 -- 172.21.15.36:0/1901643315 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed813a7e0 con 0x7f0ed8060c70 2022-06-17T12:54:38.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0ee07c2700 1 -- 172.21.15.36:0/1901643315 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed8057410 con 0x7f0ed8060c70 2022-06-17T12:54:38.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0ed77fe700 1 -- 172.21.15.36:0/1901643315 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f0ec8022070 con 0x7f0ed8060c70 2022-06-17T12:54:38.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.000+0000 7f0ed77fe700 1 --2- 172.21.15.36:0/1901643315 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0eb8053920 0x7f0eb8055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:38.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.004+0000 7f0ed77fe700 1 -- 172.21.15.36:0/1901643315 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0ec806c940 con 0x7f0ed8060c70 2022-06-17T12:54:38.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.004+0000 7f0eddd5d700 1 --2- 172.21.15.36:0/1901643315 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0eb8053920 0x7f0eb8055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:38.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.004+0000 7f0eddd5d700 1 --2- 172.21.15.36:0/1901643315 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0eb8053920 0x7f0eb8055de0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f0ecc00a000 tx=0x7f0ecc00a040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:38.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.008+0000 7f0ed77fe700 1 -- 172.21.15.36:0/1901643315 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0ec8043630 con 0x7f0ed8060c70 2022-06-17T12:54:38.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.176+0000 7f0ee07c2700 1 -- 172.21.15.36:0/1901643315 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f0ed805f220 con 0x7f0ed8060c70 2022-06-17T12:54:38.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.180+0000 7f0ed77fe700 1 -- 172.21.15.36:0/1901643315 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v33) v1 ==== 184+0+0 (secure 0 0 0) 0x7f0ec8046c50 con 0x7f0ed8060c70 2022-06-17T12:54:38.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.184+0000 7f0ee07c2700 1 -- 172.21.15.36:0/1901643315 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0eb8053920 msgr2=0x7f0eb8055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:38.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.184+0000 7f0ee07c2700 1 --2- 172.21.15.36:0/1901643315 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0eb8053920 0x7f0eb8055de0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f0ecc00a000 tx=0x7f0ecc00a040).stop 2022-06-17T12:54:38.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.184+0000 7f0ee07c2700 1 -- 172.21.15.36:0/1901643315 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ed8060c70 msgr2=0x7f0ed80ae610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:38.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.184+0000 7f0ee07c2700 1 --2- 172.21.15.36:0/1901643315 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ed8060c70 0x7f0ed80ae610 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7f0ec8006b60 tx=0x7f0ec8009290).stop 2022-06-17T12:54:38.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.184+0000 7f0ee07c2700 1 -- 172.21.15.36:0/1901643315 shutdown_connections 2022-06-17T12:54:38.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.184+0000 7f0ee07c2700 1 --2- 172.21.15.36:0/1901643315 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0eb8053920 0x7f0eb8055de0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:38.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.184+0000 7f0ee07c2700 1 --2- 172.21.15.36:0/1901643315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ed8142a20 0x7f0ed80a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:38.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.184+0000 7f0ee07c2700 1 --2- 172.21.15.36:0/1901643315 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed80b2340 0x7f0ed813a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:38.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.184+0000 7f0ee07c2700 1 --2- 172.21.15.36:0/1901643315 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ed8060c70 0x7f0ed80ae610 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:38.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.184+0000 7f0ee07c2700 1 -- 172.21.15.36:0/1901643315 >> 172.21.15.36:0/1901643315 conn(0x7f0ed80b6870 msgr2=0x7f0ed80b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:38.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.184+0000 7f0ee07c2700 1 -- 172.21.15.36:0/1901643315 shutdown_connections 2022-06-17T12:54:38.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.184+0000 7f0ee07c2700 1 -- 172.21.15.36:0/1901643315 wait complete. 2022-06-17T12:54:38.181 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.1 2022-06-17T12:54:38.188 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:54:38.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.312+0000 7f97cdf98700 1 Processor -- start 2022-06-17T12:54:38.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.312+0000 7f97cdf98700 1 -- start start 2022-06-17T12:54:38.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.312+0000 7f97cdf98700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97c80aef60 0x7f97c80b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:38.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.312+0000 7f97cdf98700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f97c80ba2e0 0x7f97c81442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:38.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.312+0000 7f97cdf98700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c80af4b0 con 0x7f97c813cb70 2022-06-17T12:54:38.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.312+0000 7f97cdf98700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c8143da0 con 0x7f97c80ba2e0 2022-06-17T12:54:38.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.312+0000 7f97cdf98700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c8142f50 con 0x7f97c80aef60 2022-06-17T12:54:38.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.312+0000 7f97cd797700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f97c80ba2e0 0x7f97c81442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:38.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.312+0000 7f97cd797700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f97c80ba2e0 0x7f97c81442a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37116/0 (socket says 172.21.15.36:37116) 2022-06-17T12:54:38.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.312+0000 7f97cd797700 1 -- 172.21.15.36:0/2865321548 learned_addr learned my addr 172.21.15.36:0/2865321548 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:38.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.312+0000 7f97bffff700 1 --2- 172.21.15.36:0/2865321548 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97c80aef60 0x7f97c80b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:38.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.312+0000 7f97cd797700 1 -- 172.21.15.36:0/2865321548 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97c80aef60 msgr2=0x7f97c80b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:38.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.312+0000 7f97cd797700 1 --2- 172.21.15.36:0/2865321548 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97c80aef60 0x7f97c80b9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:38.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.312+0000 7f97cd797700 1 -- 172.21.15.36:0/2865321548 >> v1:172.21.15.70:6789/0 conn(0x7f97c813cb70 legacy=0x7f97c8143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:38.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.312+0000 7f97cd797700 1 -- 172.21.15.36:0/2865321548 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c81431f0 con 0x7f97c80ba2e0 2022-06-17T12:54:38.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.312+0000 7f97cd797700 1 --2- 172.21.15.36:0/2865321548 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97c80ba2e0 0x7f97c81442a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7f97c00032f0 tx=0x7f97c0009290).ready entity=mon.0 client_cookie=4a742bedf810cfc4 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:38.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.312+0000 7f97bf7fe700 1 -- 172.21.15.36:0/2865321548 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97c000b8a0 con 0x7f97c80ba2e0 2022-06-17T12:54:38.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.312+0000 7f97bf7fe700 1 -- 172.21.15.36:0/2865321548 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c000ba00 con 0x7f97c80ba2e0 2022-06-17T12:54:38.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.312+0000 7f97bf7fe700 1 -- 172.21.15.36:0/2865321548 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97c000bb70 con 0x7f97c80ba2e0 2022-06-17T12:54:38.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.316+0000 7f97cdf98700 1 -- 172.21.15.36:0/2865321548 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97c80ba2e0 msgr2=0x7f97c81442a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:38.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.316+0000 7f97cdf98700 1 --2- 172.21.15.36:0/2865321548 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97c80ba2e0 0x7f97c81442a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7f97c00032f0 tx=0x7f97c0009290).stop 2022-06-17T12:54:38.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.316+0000 7f97cdf98700 1 -- 172.21.15.36:0/2865321548 shutdown_connections 2022-06-17T12:54:38.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.316+0000 7f97cdf98700 1 --2- 172.21.15.36:0/2865321548 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97c80aef60 0x7f97c80b9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:38.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.316+0000 7f97cdf98700 1 --2- 172.21.15.36:0/2865321548 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97c80ba2e0 0x7f97c81442a0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:38.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.316+0000 7f97cdf98700 1 -- 172.21.15.36:0/2865321548 >> 172.21.15.36:0/2865321548 conn(0x7f97c805c550 msgr2=0x7f97c80b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:38.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.316+0000 7f97cdf98700 1 -- 172.21.15.36:0/2865321548 shutdown_connections 2022-06-17T12:54:38.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.316+0000 7f97cdf98700 1 -- 172.21.15.36:0/2865321548 wait complete. 2022-06-17T12:54:38.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.316+0000 7f97cdf98700 1 Processor -- start 2022-06-17T12:54:38.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.316+0000 7f97cdf98700 1 -- start start 2022-06-17T12:54:38.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.316+0000 7f97cdf98700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97c80aef60 0x7f97c8112a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:38.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.316+0000 7f97cdf98700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97c80ba2e0 0x7f97c8155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:38.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.316+0000 7f97cdf98700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97c813cb70 0x7f97c8156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:38.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.316+0000 7f97cdf98700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c8155900 con 0x7f97c813cb70 2022-06-17T12:54:38.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.316+0000 7f97cdf98700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c8156710 con 0x7f97c80aef60 2022-06-17T12:54:38.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.316+0000 7f97cdf98700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c80b36e0 con 0x7f97c80ba2e0 2022-06-17T12:54:38.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.316+0000 7f97cd797700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97c813cb70 0x7f97c8156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:38.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.316+0000 7f97cd797700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97c813cb70 0x7f97c8156220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37122/0 (socket says 172.21.15.36:37122) 2022-06-17T12:54:38.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.316+0000 7f97cd797700 1 -- 172.21.15.36:0/4155980933 learned_addr learned my addr 172.21.15.36:0/4155980933 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:38.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.316+0000 7f97bffff700 1 --2- 172.21.15.36:0/4155980933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97c80ba2e0 0x7f97c8155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:38.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.316+0000 7f97ccf96700 1 --2- 172.21.15.36:0/4155980933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97c80aef60 0x7f97c8112a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:38.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.320+0000 7f97cd797700 1 -- 172.21.15.36:0/4155980933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97c80ba2e0 msgr2=0x7f97c8155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:38.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.320+0000 7f97cd797700 1 --2- 172.21.15.36:0/4155980933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97c80ba2e0 0x7f97c8155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:38.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.320+0000 7f97cd797700 1 -- 172.21.15.36:0/4155980933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97c80aef60 msgr2=0x7f97c8112a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:38.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.320+0000 7f97cd797700 1 --2- 172.21.15.36:0/4155980933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97c80aef60 0x7f97c8112a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:38.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.320+0000 7f97cd797700 1 -- 172.21.15.36:0/4155980933 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c0009530 con 0x7f97c813cb70 2022-06-17T12:54:38.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.320+0000 7f97cd797700 1 --2- 172.21.15.36:0/4155980933 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97c813cb70 0x7f97c8156220 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7f97c00032f0 tx=0x7f97c0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:38.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.320+0000 7f97bdffb700 1 -- 172.21.15.36:0/4155980933 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97c000b8a0 con 0x7f97c813cb70 2022-06-17T12:54:38.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.320+0000 7f97bdffb700 1 -- 172.21.15.36:0/4155980933 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c0021440 con 0x7f97c813cb70 2022-06-17T12:54:38.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.320+0000 7f97cdf98700 1 -- 172.21.15.36:0/4155980933 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c8112f60 con 0x7f97c813cb70 2022-06-17T12:54:38.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.320+0000 7f97bdffb700 1 -- 172.21.15.36:0/4155980933 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97c00217a0 con 0x7f97c813cb70 2022-06-17T12:54:38.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.320+0000 7f97cdf98700 1 -- 172.21.15.36:0/4155980933 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c80afc10 con 0x7f97c813cb70 2022-06-17T12:54:38.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.320+0000 7f97cdf98700 1 -- 172.21.15.36:0/4155980933 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c8155a70 con 0x7f97c813cb70 2022-06-17T12:54:38.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.320+0000 7f97bdffb700 1 -- 172.21.15.36:0/4155980933 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f97c000f070 con 0x7f97c813cb70 2022-06-17T12:54:38.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.320+0000 7f97bdffb700 1 --2- 172.21.15.36:0/4155980933 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f97a8053af0 0x7f97a8055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:38.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.320+0000 7f97bdffb700 1 -- 172.21.15.36:0/4155980933 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f97c006d520 con 0x7f97c813cb70 2022-06-17T12:54:38.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.320+0000 7f97ccf96700 1 --2- 172.21.15.36:0/4155980933 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f97a8053af0 0x7f97a8055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:38.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.324+0000 7f97ccf96700 1 --2- 172.21.15.36:0/4155980933 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f97a8053af0 0x7f97a8055fb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f97c400d000 tx=0x7f97c4007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:38.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.328+0000 7f97bdffb700 1 -- 172.21.15.36:0/4155980933 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f97c8155a70 con 0x7f97c813cb70 2022-06-17T12:54:38.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.500+0000 7f97cdf98700 1 -- 172.21.15.36:0/4155980933 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f97c8155a40 con 0x7f97c813cb70 2022-06-17T12:54:38.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.508+0000 7f97bdffb700 1 -- 172.21.15.36:0/4155980933 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v34) v1 ==== 184+0+0 (secure 0 0 0) 0x7f97c0015020 con 0x7f97c813cb70 2022-06-17T12:54:38.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.508+0000 7f97cdf98700 1 -- 172.21.15.36:0/4155980933 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f97a8053af0 msgr2=0x7f97a8055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:38.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.508+0000 7f97cdf98700 1 --2- 172.21.15.36:0/4155980933 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f97a8053af0 0x7f97a8055fb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f97c400d000 tx=0x7f97c4007b90).stop 2022-06-17T12:54:38.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.508+0000 7f97cdf98700 1 -- 172.21.15.36:0/4155980933 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97c813cb70 msgr2=0x7f97c8156220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:38.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.508+0000 7f97cdf98700 1 --2- 172.21.15.36:0/4155980933 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97c813cb70 0x7f97c8156220 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7f97c00032f0 tx=0x7f97c0016000).stop 2022-06-17T12:54:38.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.508+0000 7f97cdf98700 1 -- 172.21.15.36:0/4155980933 shutdown_connections 2022-06-17T12:54:38.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.508+0000 7f97cdf98700 1 --2- 172.21.15.36:0/4155980933 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f97a8053af0 0x7f97a8055fb0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:38.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.508+0000 7f97cdf98700 1 --2- 172.21.15.36:0/4155980933 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f97c813cb70 0x7f97c8156220 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:38.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.508+0000 7f97cdf98700 1 --2- 172.21.15.36:0/4155980933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97c80ba2e0 0x7f97c8155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:38.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.508+0000 7f97cdf98700 1 --2- 172.21.15.36:0/4155980933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97c80aef60 0x7f97c8112a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:38.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.508+0000 7f97cdf98700 1 -- 172.21.15.36:0/4155980933 >> 172.21.15.36:0/4155980933 conn(0x7f97c805c550 msgr2=0x7f97c80b6550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:38.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.508+0000 7f97cdf98700 1 -- 172.21.15.36:0/4155980933 shutdown_connections 2022-06-17T12:54:38.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.508+0000 7f97cdf98700 1 -- 172.21.15.36:0/4155980933 wait complete. 2022-06-17T12:54:38.505 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.2 2022-06-17T12:54:38.509 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:54:38.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.624+0000 7f4e9692f700 1 Processor -- start 2022-06-17T12:54:38.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.624+0000 7f4e9692f700 1 -- start start 2022-06-17T12:54:38.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.628+0000 7f4e9692f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4e90063730 0x7f4e90065e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:38.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.628+0000 7f4e9692f700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e90063b10 con 0x7f4e900b1930 2022-06-17T12:54:38.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.628+0000 7f4e9692f700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e90065430 con 0x7f4e900b2900 2022-06-17T12:54:38.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.628+0000 7f4e9692f700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e900656d0 con 0x7f4e90063730 2022-06-17T12:54:38.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.628+0000 7f4e9512c700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f4e900b1930 0x7f4e901431f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54656/0 (socket says 172.21.15.36:54656) 2022-06-17T12:54:38.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.628+0000 7f4e9512c700 1 -- 172.21.15.36:0/906473687 learned_addr learned my addr 172.21.15.36:0/906473687 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:38.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.628+0000 7f4e9592d700 1 --2- 172.21.15.36:0/906473687 >> v2:172.21.15.36:3300/0 conn(0x7f4e90063730 0x7f4e90065e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:38.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.628+0000 7f4e9492b700 1 -- 172.21.15.36:0/906473687 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3795163244 0 0) 0x7f4e90063b10 con 0x7f4e900b1930 2022-06-17T12:54:38.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.628+0000 7f4e9492b700 1 -- 172.21.15.36:0/906473687 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e78003260 con 0x7f4e900b1930 2022-06-17T12:54:38.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.628+0000 7f4e9492b700 1 -- 172.21.15.36:0/906473687 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 468596687 0 0) 0x7f4e78003260 con 0x7f4e900b1930 2022-06-17T12:54:38.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.628+0000 7f4e9592d700 1 -- 172.21.15.36:0/906473687 >> v1:172.21.15.70:6789/0 conn(0x7f4e900b2900 legacy=0x7f4e9013eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:38.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.628+0000 7f4e9592d700 1 -- 172.21.15.36:0/906473687 >> v1:172.21.15.36:6789/0 conn(0x7f4e900b1930 legacy=0x7f4e901431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:38.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.628+0000 7f4e9592d700 1 -- 172.21.15.36:0/906473687 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e90065970 con 0x7f4e90063730 2022-06-17T12:54:38.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.628+0000 7f4e9592d700 1 --2- 172.21.15.36:0/906473687 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4e90063730 0x7f4e90065e10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7f4e84001800 tx=0x7f4e8c006df0).ready entity=mon.0 client_cookie=c86d2469c931e112 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:38.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.628+0000 7f4e9492b700 1 -- 172.21.15.36:0/906473687 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e84001660 con 0x7f4e90063730 2022-06-17T12:54:38.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.628+0000 7f4e9492b700 1 -- 172.21.15.36:0/906473687 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8c000e90 con 0x7f4e90063730 2022-06-17T12:54:38.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.628+0000 7f4e9492b700 1 -- 172.21.15.36:0/906473687 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e8c00d4f0 con 0x7f4e90063730 2022-06-17T12:54:38.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.628+0000 7f4e9692f700 1 -- 172.21.15.36:0/906473687 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4e90063730 msgr2=0x7f4e90065e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:38.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.628+0000 7f4e9692f700 1 --2- 172.21.15.36:0/906473687 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4e90063730 0x7f4e90065e10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7f4e84001800 tx=0x7f4e8c006df0).stop 2022-06-17T12:54:38.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.628+0000 7f4e9692f700 1 -- 172.21.15.36:0/906473687 shutdown_connections 2022-06-17T12:54:38.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.628+0000 7f4e9692f700 1 --2- 172.21.15.36:0/906473687 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4e90063730 0x7f4e90065e10 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:38.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.628+0000 7f4e9692f700 1 -- 172.21.15.36:0/906473687 >> 172.21.15.36:0/906473687 conn(0x7f4e900614b0 msgr2=0x7f4e900618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:38.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.628+0000 7f4e9692f700 1 -- 172.21.15.36:0/906473687 shutdown_connections 2022-06-17T12:54:38.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.628+0000 7f4e9692f700 1 -- 172.21.15.36:0/906473687 wait complete. 2022-06-17T12:54:38.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.632+0000 7f4e9692f700 1 Processor -- start 2022-06-17T12:54:38.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.632+0000 7f4e9692f700 1 -- start start 2022-06-17T12:54:38.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.632+0000 7f4e9692f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4e90063730 0x7f4e901334a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:38.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.632+0000 7f4e9692f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e900b1930 0x7f4e90138a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:38.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.632+0000 7f4e9692f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4e900b2900 0x7f4e900ae050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:38.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.632+0000 7f4e9692f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e900ad8f0 con 0x7f4e90063730 2022-06-17T12:54:38.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.632+0000 7f4e9692f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e900ae540 con 0x7f4e900b1930 2022-06-17T12:54:38.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.632+0000 7f4e9692f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e900add20 con 0x7f4e900b2900 2022-06-17T12:54:38.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.632+0000 7f4e9592d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4e90063730 0x7f4e901334a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:38.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.632+0000 7f4e9592d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4e90063730 0x7f4e901334a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37132/0 (socket says 172.21.15.36:37132) 2022-06-17T12:54:38.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.632+0000 7f4e9592d700 1 -- 172.21.15.36:0/1183418218 learned_addr learned my addr 172.21.15.36:0/1183418218 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:38.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.632+0000 7f4e9512c700 1 --2- 172.21.15.36:0/1183418218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e900b1930 0x7f4e90138a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:38.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.636+0000 7f4e9612e700 1 --2- 172.21.15.36:0/1183418218 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4e900b2900 0x7f4e900ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:38.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.636+0000 7f4e9592d700 1 -- 172.21.15.36:0/1183418218 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4e900b2900 msgr2=0x7f4e900ae050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:38.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.636+0000 7f4e9592d700 1 --2- 172.21.15.36:0/1183418218 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4e900b2900 0x7f4e900ae050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:38.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.636+0000 7f4e9592d700 1 -- 172.21.15.36:0/1183418218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e900b1930 msgr2=0x7f4e90138a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:38.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.636+0000 7f4e9592d700 1 --2- 172.21.15.36:0/1183418218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e900b1930 0x7f4e90138a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:38.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.636+0000 7f4e9592d700 1 -- 172.21.15.36:0/1183418218 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8c00b180 con 0x7f4e90063730 2022-06-17T12:54:38.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.636+0000 7f4e9592d700 1 --2- 172.21.15.36:0/1183418218 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4e90063730 0x7f4e901334a0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7f4e84001800 tx=0x7f4e8c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:38.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.636+0000 7f4e82ffd700 1 -- 172.21.15.36:0/1183418218 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e8c016070 con 0x7f4e90063730 2022-06-17T12:54:38.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.636+0000 7f4e82ffd700 1 -- 172.21.15.36:0/1183418218 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8c002f50 con 0x7f4e90063730 2022-06-17T12:54:38.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.636+0000 7f4e9692f700 1 -- 172.21.15.36:0/1183418218 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e900b05e0 con 0x7f4e90063730 2022-06-17T12:54:38.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.636+0000 7f4e82ffd700 1 -- 172.21.15.36:0/1183418218 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e8c00d4f0 con 0x7f4e90063730 2022-06-17T12:54:38.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.636+0000 7f4e9692f700 1 -- 172.21.15.36:0/1183418218 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e90142bf0 con 0x7f4e90063730 2022-06-17T12:54:38.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.636+0000 7f4e9692f700 1 -- 172.21.15.36:0/1183418218 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e900ada30 con 0x7f4e90063730 2022-06-17T12:54:38.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.636+0000 7f4e82ffd700 1 -- 172.21.15.36:0/1183418218 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f4e8c025030 con 0x7f4e90063730 2022-06-17T12:54:38.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.636+0000 7f4e82ffd700 1 --2- 172.21.15.36:0/1183418218 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4e70053ad0 0x7f4e70055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:38.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.636+0000 7f4e82ffd700 1 -- 172.21.15.36:0/1183418218 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4e8c06cd50 con 0x7f4e90063730 2022-06-17T12:54:38.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.636+0000 7f4e9512c700 1 --2- 172.21.15.36:0/1183418218 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4e70053ad0 0x7f4e70055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:38.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.640+0000 7f4e9512c700 1 --2- 172.21.15.36:0/1183418218 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4e70053ad0 0x7f4e70055f90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f4e8400d000 tx=0x7f4e84001790).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:38.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.644+0000 7f4e82ffd700 1 -- 172.21.15.36:0/1183418218 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4e8c043b50 con 0x7f4e90063730 2022-06-17T12:54:38.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.808+0000 7f4e9692f700 1 -- 172.21.15.36:0/1183418218 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e900b3b30 con 0x7f4e90063730 2022-06-17T12:54:38.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.812+0000 7f4e82ffd700 1 -- 172.21.15.36:0/1183418218 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f4e8c023020 con 0x7f4e90063730 2022-06-17T12:54:38.798 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:38.799 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":47,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/138643404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":138643404},{"type":"v1","addr":"172.21.15.70:6833","nonce":138643404}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":46}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:54:35.949730+0000","modified":"2022-06-17T12:54:36.953427+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7074},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7074":{"gid":7074,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/3267343301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3267343301},{"type":"v1","addr":"172.21.15.36:6835","nonce":3267343301}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-06-17T12:54:38.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.812+0000 7f4e9692f700 1 -- 172.21.15.36:0/1183418218 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4e70053ad0 msgr2=0x7f4e70055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:38.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.812+0000 7f4e9692f700 1 --2- 172.21.15.36:0/1183418218 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4e70053ad0 0x7f4e70055f90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f4e8400d000 tx=0x7f4e84001790).stop 2022-06-17T12:54:38.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.812+0000 7f4e9692f700 1 -- 172.21.15.36:0/1183418218 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4e90063730 msgr2=0x7f4e901334a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:38.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.812+0000 7f4e9692f700 1 --2- 172.21.15.36:0/1183418218 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4e90063730 0x7f4e901334a0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7f4e84001800 tx=0x7f4e8c014000).stop 2022-06-17T12:54:38.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.812+0000 7f4e9692f700 1 -- 172.21.15.36:0/1183418218 shutdown_connections 2022-06-17T12:54:38.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.812+0000 7f4e9692f700 1 --2- 172.21.15.36:0/1183418218 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4e70053ad0 0x7f4e70055f90 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:38.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.812+0000 7f4e9692f700 1 --2- 172.21.15.36:0/1183418218 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4e900b2900 0x7f4e900ae050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:38.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.812+0000 7f4e9692f700 1 --2- 172.21.15.36:0/1183418218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e900b1930 0x7f4e90138a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:38.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.812+0000 7f4e9692f700 1 --2- 172.21.15.36:0/1183418218 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4e90063730 0x7f4e901334a0 secure :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 rx=0x7f4e84001800 tx=0x7f4e8c014000).stop 2022-06-17T12:54:38.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.812+0000 7f4e9692f700 1 -- 172.21.15.36:0/1183418218 >> 172.21.15.36:0/1183418218 conn(0x7f4e900614b0 msgr2=0x7f4e9005bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:38.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.812+0000 7f4e9692f700 1 -- 172.21.15.36:0/1183418218 shutdown_connections 2022-06-17T12:54:38.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:38.812+0000 7f4e9692f700 1 -- 172.21.15.36:0/1183418218 wait complete. 2022-06-17T12:54:38.807 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 47 2022-06-17T12:54:39.017 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:54:35.949730+0000', 'modified': '2022-06-17T12:54:36.953427+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7074}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7074': {'gid': 7074, 'name': 'a', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.36:6835/3267343301', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 3267343301}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 3267343301}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:54:39.018 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:54:39.019 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:54:39.139 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":7074,"want_state":"up:active","state":"up:active","rank_uptime":3.1938265659999998,"mdsmap_epoch":47,"osdmap_epoch":94,"osdmap_epoch_barrier":93,"uptime":7.6103868200000004} 2022-06-17T12:54:39.139 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 7074, "mdsmap_epoch": 47, "osdmap_epoch": 94, "osdmap_epoch_barrier": 93, "rank_uptime": 3.193826566, "state": "up:active", "uptime": 7.61038682, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:54:39.140 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:54:39.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb5317aa700 1 Processor -- start 2022-06-17T12:54:39.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb5317aa700 1 -- start start 2022-06-17T12:54:39.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb5317aa700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb52c0b2200 0x7fb52c0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:39.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb5317aa700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb52c0b2cd0 0x7fb52c147600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:39.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb5317aa700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52c0b25e0 con 0x7fb52c0af8d0 2022-06-17T12:54:39.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb5317aa700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb52c0ad790 con 0x7fb52c0b2cd0 2022-06-17T12:54:39.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb5317aa700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0aef40 con 0x7fb52c0b2200 2022-06-17T12:54:39.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb530fa9700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb52c0b2cd0 0x7fb52c147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:39.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb52bfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb52c0b2200 0x7fb52c0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:39.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb530fa9700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb52c0b2cd0 0x7fb52c147600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:57526/0 (socket says 172.21.15.36:57526) 2022-06-17T12:54:39.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb530fa9700 1 -- 172.21.15.36:0/3799436042 learned_addr learned my addr 172.21.15.36:0/3799436042 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:39.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb52b7fe700 1 --1- 172.21.15.36:0/3799436042 >> v1:172.21.15.70:6789/0 conn(0x7fb52c0af8d0 0x7fb52c14a8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:50404/0 (socket says 172.21.15.36:50404) 2022-06-17T12:54:39.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb52affd700 1 -- 172.21.15.36:0/3799436042 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3722571016 0 0) 0x7fb52c0b25e0 con 0x7fb52c0af8d0 2022-06-17T12:54:39.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb52affd700 1 -- 172.21.15.36:0/3799436042 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb518003260 con 0x7fb52c0af8d0 2022-06-17T12:54:39.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb530fa9700 1 -- 172.21.15.36:0/3799436042 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb52c0b2200 msgr2=0x7fb52c0af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:39.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb530fa9700 1 --2- 172.21.15.36:0/3799436042 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb52c0b2200 0x7fb52c0af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb530fa9700 1 -- 172.21.15.36:0/3799436042 >> v1:172.21.15.70:6789/0 conn(0x7fb52c0af8d0 legacy=0x7fb52c14a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:39.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb530fa9700 1 -- 172.21.15.36:0/3799436042 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52c0b1950 con 0x7fb52c0b2cd0 2022-06-17T12:54:39.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb530fa9700 1 --2- 172.21.15.36:0/3799436042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb52c0b2cd0 0x7fb52c147600 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fb5240032f0 tx=0x7fb524009290).ready entity=mon.1 client_cookie=ca715dbd07c02fc7 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:39.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb52affd700 1 -- 172.21.15.36:0/3799436042 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb52400b6c0 con 0x7fb52c0b2cd0 2022-06-17T12:54:39.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb52affd700 1 -- 172.21.15.36:0/3799436042 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52400b820 con 0x7fb52c0b2cd0 2022-06-17T12:54:39.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb52affd700 1 -- 172.21.15.36:0/3799436042 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb52400b990 con 0x7fb52c0b2cd0 2022-06-17T12:54:39.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb5317aa700 1 -- 172.21.15.36:0/3799436042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb52c0b2cd0 msgr2=0x7fb52c147600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:39.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb5317aa700 1 --2- 172.21.15.36:0/3799436042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb52c0b2cd0 0x7fb52c147600 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fb5240032f0 tx=0x7fb524009290).stop 2022-06-17T12:54:39.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb5317aa700 1 -- 172.21.15.36:0/3799436042 shutdown_connections 2022-06-17T12:54:39.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb5317aa700 1 --2- 172.21.15.36:0/3799436042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb52c0b2cd0 0x7fb52c147600 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb5317aa700 1 --2- 172.21.15.36:0/3799436042 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb52c0b2200 0x7fb52c0af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb5317aa700 1 -- 172.21.15.36:0/3799436042 >> 172.21.15.36:0/3799436042 conn(0x7fb52c05f770 msgr2=0x7fb52c05fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:39.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb5317aa700 1 -- 172.21.15.36:0/3799436042 shutdown_connections 2022-06-17T12:54:39.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.252+0000 7fb5317aa700 1 -- 172.21.15.36:0/3799436042 wait complete. 2022-06-17T12:54:39.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.256+0000 7fb5317aa700 1 Processor -- start 2022-06-17T12:54:39.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.256+0000 7fb5317aa700 1 -- start start 2022-06-17T12:54:39.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.256+0000 7fb5317aa700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb52c0af8d0 0x7fb52c062dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:39.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.256+0000 7fb5317aa700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb52c0b2200 0x7fb52c146560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:39.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.256+0000 7fb5317aa700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb52c0b2cd0 0x7fb52c157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:39.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.256+0000 7fb5317aa700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0632b0 con 0x7fb52c0b2200 2022-06-17T12:54:39.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.256+0000 7fb5317aa700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c156820 con 0x7fb52c0af8d0 2022-06-17T12:54:39.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.256+0000 7fb5317aa700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c156ac0 con 0x7fb52c0b2cd0 2022-06-17T12:54:39.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.256+0000 7fb52b7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb52c0b2200 0x7fb52c146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:39.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.256+0000 7fb52b7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb52c0b2200 0x7fb52c146560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37148/0 (socket says 172.21.15.36:37148) 2022-06-17T12:54:39.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.256+0000 7fb52b7fe700 1 -- 172.21.15.36:0/2358074231 learned_addr learned my addr 172.21.15.36:0/2358074231 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:39.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.256+0000 7fb52bfff700 1 --2- 172.21.15.36:0/2358074231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb52c0af8d0 0x7fb52c062dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:39.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.256+0000 7fb530fa9700 1 --2- 172.21.15.36:0/2358074231 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb52c0b2cd0 0x7fb52c157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:39.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.256+0000 7fb52b7fe700 1 -- 172.21.15.36:0/2358074231 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb52c0b2cd0 msgr2=0x7fb52c157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:39.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.256+0000 7fb52b7fe700 1 --2- 172.21.15.36:0/2358074231 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb52c0b2cd0 0x7fb52c157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.256+0000 7fb52b7fe700 1 -- 172.21.15.36:0/2358074231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb52c0af8d0 msgr2=0x7fb52c062dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:39.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.256+0000 7fb52b7fe700 1 --2- 172.21.15.36:0/2358074231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb52c0af8d0 0x7fb52c062dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.256+0000 7fb52b7fe700 1 -- 172.21.15.36:0/2358074231 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb524009530 con 0x7fb52c0b2200 2022-06-17T12:54:39.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.256+0000 7fb52b7fe700 1 --2- 172.21.15.36:0/2358074231 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb52c0b2200 0x7fb52c146560 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7fb51c001110 tx=0x7fb51c007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:39.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.256+0000 7fb5297fa700 1 -- 172.21.15.36:0/2358074231 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb51c0070d0 con 0x7fb52c0b2200 2022-06-17T12:54:39.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.256+0000 7fb5297fa700 1 -- 172.21.15.36:0/2358074231 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51c007230 con 0x7fb52c0b2200 2022-06-17T12:54:39.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.256+0000 7fb5317aa700 1 -- 172.21.15.36:0/2358074231 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52c156d60 con 0x7fb52c0b2200 2022-06-17T12:54:39.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.256+0000 7fb5297fa700 1 -- 172.21.15.36:0/2358074231 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb51c005740 con 0x7fb52c0b2200 2022-06-17T12:54:39.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.256+0000 7fb5317aa700 1 -- 172.21.15.36:0/2358074231 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52c1576f0 con 0x7fb52c0b2200 2022-06-17T12:54:39.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.260+0000 7fb5317aa700 1 -- 172.21.15.36:0/2358074231 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52c0672e0 con 0x7fb52c0b2200 2022-06-17T12:54:39.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.260+0000 7fb5297fa700 1 -- 172.21.15.36:0/2358074231 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fb51c010070 con 0x7fb52c0b2200 2022-06-17T12:54:39.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.260+0000 7fb5297fa700 1 --2- 172.21.15.36:0/2358074231 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb50c053ad0 0x7fb50c055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:39.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.260+0000 7fb5297fa700 1 -- 172.21.15.36:0/2358074231 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb51c06b6e0 con 0x7fb52c0b2200 2022-06-17T12:54:39.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.260+0000 7fb52bfff700 1 --2- 172.21.15.36:0/2358074231 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb50c053ad0 0x7fb50c055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:39.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.260+0000 7fb52bfff700 1 --2- 172.21.15.36:0/2358074231 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb50c053ad0 0x7fb50c055f90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fb52000b000 tx=0x7fb520006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:39.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.264+0000 7fb5297fa700 1 -- 172.21.15.36:0/2358074231 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb52c0672e0 con 0x7fb52c0b2200 2022-06-17T12:54:39.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.392+0000 7fb5317aa700 1 -- 172.21.15.36:0/2358074231 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb52c0672e0 con 0x7fb52c0b2200 2022-06-17T12:54:39.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.396+0000 7fb5297fa700 1 -- 172.21.15.36:0/2358074231 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16274 (secure 0 0 0) 0x7fb52c0672e0 con 0x7fb52c0b2200 2022-06-17T12:54:39.384 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:39.384 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":94,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:54:37.964330+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:54:34.381157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-06-17T12:54:35.204631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:54:39.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.396+0000 7fb5317aa700 1 -- 172.21.15.36:0/2358074231 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb50c053ad0 msgr2=0x7fb50c055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:39.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.396+0000 7fb5317aa700 1 --2- 172.21.15.36:0/2358074231 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb50c053ad0 0x7fb50c055f90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fb52000b000 tx=0x7fb520006df0).stop 2022-06-17T12:54:39.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.396+0000 7fb5317aa700 1 -- 172.21.15.36:0/2358074231 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb52c0b2200 msgr2=0x7fb52c146560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:39.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.396+0000 7fb5317aa700 1 --2- 172.21.15.36:0/2358074231 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb52c0b2200 0x7fb52c146560 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7fb51c001110 tx=0x7fb51c007a60).stop 2022-06-17T12:54:39.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.396+0000 7fb5317aa700 1 -- 172.21.15.36:0/2358074231 shutdown_connections 2022-06-17T12:54:39.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.396+0000 7fb5317aa700 1 --2- 172.21.15.36:0/2358074231 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb50c053ad0 0x7fb50c055f90 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.396+0000 7fb5317aa700 1 --2- 172.21.15.36:0/2358074231 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb52c0b2cd0 0x7fb52c157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.396+0000 7fb5317aa700 1 --2- 172.21.15.36:0/2358074231 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb52c0b2200 0x7fb52c146560 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.396+0000 7fb5317aa700 1 --2- 172.21.15.36:0/2358074231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb52c0af8d0 0x7fb52c062dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.396+0000 7fb5317aa700 1 -- 172.21.15.36:0/2358074231 >> 172.21.15.36:0/2358074231 conn(0x7fb52c05f770 msgr2=0x7fb52c147730 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:39.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.396+0000 7fb5317aa700 1 -- 172.21.15.36:0/2358074231 shutdown_connections 2022-06-17T12:54:39.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.396+0000 7fb5317aa700 1 -- 172.21.15.36:0/2358074231 wait complete. 2022-06-17T12:54:39.397 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:54:34.381157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:54:35.204631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:54:39.398 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:54:39.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.528+0000 7fab13c15700 1 Processor -- start 2022-06-17T12:54:39.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.528+0000 7fab13c15700 1 -- start start 2022-06-17T12:54:39.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.528+0000 7fab13c15700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fab0c0b4120 0x7fab0c146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:39.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.528+0000 7fab13c15700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fab0c147270 0x7fab0c140100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:39.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.528+0000 7fab13c15700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab0c1476e0 con 0x7fab0c0b64f0 2022-06-17T12:54:39.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.528+0000 7fab13c15700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab0c0b68d0 con 0x7fab0c0b4120 2022-06-17T12:54:39.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.528+0000 7fab13c15700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab0c0b6e20 con 0x7fab0c147270 2022-06-17T12:54:39.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.528+0000 7fab111b0700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fab0c0b4120 0x7fab0c146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:39.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.528+0000 7fab111b0700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fab0c0b4120 0x7fab0c146d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37156/0 (socket says 172.21.15.36:37156) 2022-06-17T12:54:39.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.528+0000 7fab111b0700 1 -- 172.21.15.36:0/1224091681 learned_addr learned my addr 172.21.15.36:0/1224091681 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:39.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.528+0000 7fab119b1700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fab0c0b64f0 0x7fab0c143150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54684/0 (socket says 172.21.15.36:54684) 2022-06-17T12:54:39.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.528+0000 7fab121b2700 1 --2- 172.21.15.36:0/1224091681 >> v2:172.21.15.70:3300/0 conn(0x7fab0c147270 0x7fab0c140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:39.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.528+0000 7fab111b0700 1 -- 172.21.15.36:0/1224091681 >> v2:172.21.15.70:3300/0 conn(0x7fab0c147270 msgr2=0x7fab0c140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:39.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.528+0000 7fab111b0700 1 --2- 172.21.15.36:0/1224091681 >> v2:172.21.15.70:3300/0 conn(0x7fab0c147270 0x7fab0c140100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.528+0000 7fab111b0700 1 -- 172.21.15.36:0/1224091681 >> v1:172.21.15.36:6789/0 conn(0x7fab0c0b64f0 legacy=0x7fab0c143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:39.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.528+0000 7fab111b0700 1 -- 172.21.15.36:0/1224091681 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0c1468e0 con 0x7fab0c0b4120 2022-06-17T12:54:39.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.528+0000 7fab109af700 1 -- 172.21.15.36:0/1224091681 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3216010189 0 0) 0x7fab0c1476e0 con 0x7fab0c0b64f0 2022-06-17T12:54:39.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.528+0000 7fab111b0700 1 --2- 172.21.15.36:0/1224091681 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fab0c0b4120 0x7fab0c146d80 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7faafc002f70 tx=0x7faafc006eb0).ready entity=mon.0 client_cookie=4eae42f9eaa6a192 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:39.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.528+0000 7fab109af700 1 -- 172.21.15.36:0/1224091681 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faafc00d5c0 con 0x7fab0c0b4120 2022-06-17T12:54:39.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.528+0000 7fab109af700 1 -- 172.21.15.36:0/1224091681 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faafc000e90 con 0x7fab0c0b4120 2022-06-17T12:54:39.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.528+0000 7fab109af700 1 -- 172.21.15.36:0/1224091681 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faafc00d910 con 0x7fab0c0b4120 2022-06-17T12:54:39.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.532+0000 7fab13c15700 1 -- 172.21.15.36:0/1224091681 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fab0c0b4120 msgr2=0x7fab0c146d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:39.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.532+0000 7fab13c15700 1 --2- 172.21.15.36:0/1224091681 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fab0c0b4120 0x7fab0c146d80 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7faafc002f70 tx=0x7faafc006eb0).stop 2022-06-17T12:54:39.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.532+0000 7fab13c15700 1 -- 172.21.15.36:0/1224091681 shutdown_connections 2022-06-17T12:54:39.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.532+0000 7fab13c15700 1 --2- 172.21.15.36:0/1224091681 >> v2:172.21.15.70:3300/0 conn(0x7fab0c147270 0x7fab0c140100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.532+0000 7fab13c15700 1 --2- 172.21.15.36:0/1224091681 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fab0c0b4120 0x7fab0c146d80 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.532+0000 7fab13c15700 1 -- 172.21.15.36:0/1224091681 >> 172.21.15.36:0/1224091681 conn(0x7fab0c0620f0 msgr2=0x7fab0c062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:39.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.532+0000 7fab13c15700 1 -- 172.21.15.36:0/1224091681 shutdown_connections 2022-06-17T12:54:39.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.532+0000 7fab13c15700 1 -- 172.21.15.36:0/1224091681 wait complete. 2022-06-17T12:54:39.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.532+0000 7fab13c15700 1 Processor -- start 2022-06-17T12:54:39.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.532+0000 7fab13c15700 1 -- start start 2022-06-17T12:54:39.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.532+0000 7fab13c15700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab0c0b4120 0x7fab0c148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:39.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.532+0000 7fab13c15700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab0c0b64f0 0x7fab0c0b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:39.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.532+0000 7fab13c15700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fab0c147270 0x7fab0c13e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:39.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.532+0000 7fab13c15700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c1416b0 con 0x7fab0c147270 2022-06-17T12:54:39.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.532+0000 7fab13c15700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c1491c0 con 0x7fab0c0b4120 2022-06-17T12:54:39.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.532+0000 7fab13c15700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c149460 con 0x7fab0c0b64f0 2022-06-17T12:54:39.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.532+0000 7fab121b2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fab0c147270 0x7fab0c13e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:39.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.532+0000 7fab121b2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fab0c147270 0x7fab0c13e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37164/0 (socket says 172.21.15.36:37164) 2022-06-17T12:54:39.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.532+0000 7fab121b2700 1 -- 172.21.15.36:0/502682441 learned_addr learned my addr 172.21.15.36:0/502682441 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:39.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.532+0000 7fab111b0700 1 --2- 172.21.15.36:0/502682441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab0c0b64f0 0x7fab0c0b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:39.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.532+0000 7fab119b1700 1 --2- 172.21.15.36:0/502682441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab0c0b4120 0x7fab0c148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:39.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.536+0000 7fab121b2700 1 -- 172.21.15.36:0/502682441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab0c0b64f0 msgr2=0x7fab0c0b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:39.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.536+0000 7fab121b2700 1 --2- 172.21.15.36:0/502682441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab0c0b64f0 0x7fab0c0b28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.536+0000 7fab121b2700 1 -- 172.21.15.36:0/502682441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab0c0b4120 msgr2=0x7fab0c148bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:39.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.536+0000 7fab121b2700 1 --2- 172.21.15.36:0/502682441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab0c0b4120 0x7fab0c148bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.536+0000 7fab121b2700 1 -- 172.21.15.36:0/502682441 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faafc00b1e0 con 0x7fab0c147270 2022-06-17T12:54:39.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.536+0000 7fab121b2700 1 --2- 172.21.15.36:0/502682441 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fab0c147270 0x7fab0c13e060 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7fab04006e00 tx=0x7fab040091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:39.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.536+0000 7fab02ffd700 1 -- 172.21.15.36:0/502682441 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab04012600 con 0x7fab0c147270 2022-06-17T12:54:39.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.536+0000 7fab02ffd700 1 -- 172.21.15.36:0/502682441 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab04012760 con 0x7fab0c147270 2022-06-17T12:54:39.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.536+0000 7fab13c15700 1 -- 172.21.15.36:0/502682441 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab0c0b2dd0 con 0x7fab0c147270 2022-06-17T12:54:39.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.536+0000 7fab02ffd700 1 -- 172.21.15.36:0/502682441 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab04012ac0 con 0x7fab0c147270 2022-06-17T12:54:39.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.536+0000 7fab13c15700 1 -- 172.21.15.36:0/502682441 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab0c0ae8d0 con 0x7fab0c147270 2022-06-17T12:54:39.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.536+0000 7fab02ffd700 1 -- 172.21.15.36:0/502682441 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fab04022070 con 0x7fab0c147270 2022-06-17T12:54:39.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.536+0000 7fab13c15700 1 -- 172.21.15.36:0/502682441 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab0c149620 con 0x7fab0c147270 2022-06-17T12:54:39.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.536+0000 7fab02ffd700 1 --2- 172.21.15.36:0/502682441 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faaf4053af0 0x7faaf4055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:39.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.536+0000 7fab119b1700 1 --2- 172.21.15.36:0/502682441 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faaf4053af0 0x7faaf4055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:39.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.536+0000 7fab02ffd700 1 -- 172.21.15.36:0/502682441 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fab0406c620 con 0x7fab0c147270 2022-06-17T12:54:39.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.540+0000 7fab119b1700 1 --2- 172.21.15.36:0/502682441 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faaf4053af0 0x7faaf4055fb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7fab0800d000 tx=0x7fab08007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:39.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.544+0000 7fab02ffd700 1 -- 172.21.15.36:0/502682441 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fab04043290 con 0x7fab0c147270 2022-06-17T12:54:39.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.668+0000 7fab13c15700 1 -- 172.21.15.36:0/502682441 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fab0c0b3a60 con 0x7fab0c147270 2022-06-17T12:54:39.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.668+0000 7fab02ffd700 1 -- 172.21.15.36:0/502682441 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v94) v1 ==== 110+0+10 (secure 0 0 0) 0x7fab040160b0 con 0x7fab0c147270 2022-06-17T12:54:39.657 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:54:39.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.672+0000 7fab13c15700 1 -- 172.21.15.36:0/502682441 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faaf4053af0 msgr2=0x7faaf4055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:39.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.672+0000 7fab13c15700 1 --2- 172.21.15.36:0/502682441 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faaf4053af0 0x7faaf4055fb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7fab0800d000 tx=0x7fab08007a00).stop 2022-06-17T12:54:39.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.672+0000 7fab13c15700 1 -- 172.21.15.36:0/502682441 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fab0c147270 msgr2=0x7fab0c13e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:39.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.672+0000 7fab13c15700 1 --2- 172.21.15.36:0/502682441 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fab0c147270 0x7fab0c13e060 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7fab04006e00 tx=0x7fab040091d0).stop 2022-06-17T12:54:39.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.672+0000 7fab13c15700 1 -- 172.21.15.36:0/502682441 shutdown_connections 2022-06-17T12:54:39.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.672+0000 7fab13c15700 1 --2- 172.21.15.36:0/502682441 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faaf4053af0 0x7faaf4055fb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.672+0000 7fab13c15700 1 --2- 172.21.15.36:0/502682441 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fab0c147270 0x7fab0c13e060 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.672+0000 7fab13c15700 1 --2- 172.21.15.36:0/502682441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab0c0b64f0 0x7fab0c0b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.672+0000 7fab13c15700 1 --2- 172.21.15.36:0/502682441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab0c0b4120 0x7fab0c148bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.672+0000 7fab13c15700 1 -- 172.21.15.36:0/502682441 >> 172.21.15.36:0/502682441 conn(0x7fab0c0620f0 msgr2=0x7fab0c0b7810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:39.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.672+0000 7fab13c15700 1 -- 172.21.15.36:0/502682441 shutdown_connections 2022-06-17T12:54:39.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.672+0000 7fab13c15700 1 -- 172.21.15.36:0/502682441 wait complete. 2022-06-17T12:54:39.670 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:54:39.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.792+0000 7f5a522fb700 1 Processor -- start 2022-06-17T12:54:39.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.792+0000 7f5a522fb700 1 -- start start 2022-06-17T12:54:39.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.792+0000 7f5a522fb700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5a4c0b45c0 0x7f5a4c0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:39.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.792+0000 7f5a522fb700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5a4c0b4f00 0x7f5a4c0b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:39.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.792+0000 7f5a522fb700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a4c0b4160 con 0x7f5a4c0b9860 2022-06-17T12:54:39.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.792+0000 7f5a522fb700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a4c0b9c40 con 0x7f5a4c0b4f00 2022-06-17T12:54:39.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.792+0000 7f5a522fb700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a4c0627d0 con 0x7f5a4c0b45c0 2022-06-17T12:54:39.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.792+0000 7f5a51afa700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5a4c0b4f00 0x7f5a4c0b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:39.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.792+0000 7f5a51afa700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5a4c0b4f00 0x7f5a4c0b5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37172/0 (socket says 172.21.15.36:37172) 2022-06-17T12:54:39.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.792+0000 7f5a51afa700 1 -- 172.21.15.36:0/4037117109 learned_addr learned my addr 172.21.15.36:0/4037117109 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:39.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.792+0000 7f5a50af8700 1 --2- 172.21.15.36:0/4037117109 >> v2:172.21.15.70:3300/0 conn(0x7f5a4c0b45c0 0x7f5a4c0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:39.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.792+0000 7f5a51afa700 1 -- 172.21.15.36:0/4037117109 >> v2:172.21.15.70:3300/0 conn(0x7f5a4c0b45c0 msgr2=0x7f5a4c0b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:39.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.792+0000 7f5a51afa700 1 --2- 172.21.15.36:0/4037117109 >> v2:172.21.15.70:3300/0 conn(0x7f5a4c0b45c0 0x7f5a4c0b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.792+0000 7f5a51afa700 1 -- 172.21.15.36:0/4037117109 >> v1:172.21.15.70:6789/0 conn(0x7f5a4c0b9860 legacy=0x7f5a4c143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:39.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.792+0000 7f5a51afa700 1 -- 172.21.15.36:0/4037117109 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4c062a70 con 0x7f5a4c0b4f00 2022-06-17T12:54:39.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.796+0000 7f5a51afa700 1 --2- 172.21.15.36:0/4037117109 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a4c0b4f00 0x7f5a4c0b5dc0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7f5a4c0ba5d0 tx=0x7f5a44009290).ready entity=mon.0 client_cookie=72affe6d3613520a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:39.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.796+0000 7f5a37fff700 1 -- 172.21.15.36:0/4037117109 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a4400b860 con 0x7f5a4c0b4f00 2022-06-17T12:54:39.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.796+0000 7f5a37fff700 1 -- 172.21.15.36:0/4037117109 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4400b9c0 con 0x7f5a4c0b4f00 2022-06-17T12:54:39.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.796+0000 7f5a37fff700 1 -- 172.21.15.36:0/4037117109 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a4400bb30 con 0x7f5a4c0b4f00 2022-06-17T12:54:39.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.796+0000 7f5a522fb700 1 -- 172.21.15.36:0/4037117109 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a4c0b4f00 msgr2=0x7f5a4c0b5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:39.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.796+0000 7f5a522fb700 1 --2- 172.21.15.36:0/4037117109 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a4c0b4f00 0x7f5a4c0b5dc0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7f5a4c0ba5d0 tx=0x7f5a44009290).stop 2022-06-17T12:54:39.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.796+0000 7f5a522fb700 1 -- 172.21.15.36:0/4037117109 shutdown_connections 2022-06-17T12:54:39.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.796+0000 7f5a522fb700 1 --2- 172.21.15.36:0/4037117109 >> v2:172.21.15.70:3300/0 conn(0x7f5a4c0b45c0 0x7f5a4c0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.796+0000 7f5a522fb700 1 --2- 172.21.15.36:0/4037117109 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a4c0b4f00 0x7f5a4c0b5dc0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.796+0000 7f5a522fb700 1 -- 172.21.15.36:0/4037117109 >> 172.21.15.36:0/4037117109 conn(0x7f5a4c05c550 msgr2=0x7f5a4c060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:39.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.796+0000 7f5a522fb700 1 -- 172.21.15.36:0/4037117109 shutdown_connections 2022-06-17T12:54:39.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.796+0000 7f5a522fb700 1 -- 172.21.15.36:0/4037117109 wait complete. 2022-06-17T12:54:39.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.796+0000 7f5a522fb700 1 Processor -- start 2022-06-17T12:54:39.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.796+0000 7f5a522fb700 1 -- start start 2022-06-17T12:54:39.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.800+0000 7f5a522fb700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5a4c0b45c0 0x7f5a4c152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:39.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.800+0000 7f5a522fb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a4c0b4f00 0x7f5a4c141f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:39.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.800+0000 7f5a522fb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a4c0b9860 0x7f5a4c152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:39.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.800+0000 7f5a522fb700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c1488a0 con 0x7f5a4c0b4f00 2022-06-17T12:54:39.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.800+0000 7f5a522fb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c1485c0 con 0x7f5a4c0b9860 2022-06-17T12:54:39.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.800+0000 7f5a522fb700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c146c70 con 0x7f5a4c0b45c0 2022-06-17T12:54:39.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.800+0000 7f5a50af8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a4c0b4f00 0x7f5a4c141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:39.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.800+0000 7f5a50af8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a4c0b4f00 0x7f5a4c141f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37176/0 (socket says 172.21.15.36:37176) 2022-06-17T12:54:39.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.800+0000 7f5a50af8700 1 -- 172.21.15.36:0/3951763978 learned_addr learned my addr 172.21.15.36:0/3951763978 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:39.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.800+0000 7f5a512f9700 1 --2- 172.21.15.36:0/3951763978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5a4c0b45c0 0x7f5a4c152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:39.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.800+0000 7f5a51afa700 1 --2- 172.21.15.36:0/3951763978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a4c0b9860 0x7f5a4c152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:39.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.800+0000 7f5a512f9700 1 -- 172.21.15.36:0/3951763978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a4c0b9860 msgr2=0x7f5a4c152930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:39.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.800+0000 7f5a512f9700 1 --2- 172.21.15.36:0/3951763978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a4c0b9860 0x7f5a4c152930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.800+0000 7f5a512f9700 1 -- 172.21.15.36:0/3951763978 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a4c0b4f00 msgr2=0x7f5a4c141f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:39.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.800+0000 7f5a512f9700 1 --2- 172.21.15.36:0/3951763978 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a4c0b4f00 0x7f5a4c141f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.800+0000 7f5a512f9700 1 -- 172.21.15.36:0/3951763978 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a44009530 con 0x7f5a4c0b45c0 2022-06-17T12:54:39.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.800+0000 7f5a50af8700 1 --2- 172.21.15.36:0/3951763978 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a4c0b4f00 0x7f5a4c141f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:39.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.800+0000 7f5a512f9700 1 --2- 172.21.15.36:0/3951763978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5a4c0b45c0 0x7f5a4c152200 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f5a4c142ef0 tx=0x7f5a40007b70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:39.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.800+0000 7f5a3effd700 1 -- 172.21.15.36:0/3951763978 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a4000ce80 con 0x7f5a4c0b45c0 2022-06-17T12:54:39.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.800+0000 7f5a3effd700 1 -- 172.21.15.36:0/3951763978 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a40007030 con 0x7f5a4c0b45c0 2022-06-17T12:54:39.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.800+0000 7f5a522fb700 1 -- 172.21.15.36:0/3951763978 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a4c148e80 con 0x7f5a4c0b45c0 2022-06-17T12:54:39.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.800+0000 7f5a3effd700 1 -- 172.21.15.36:0/3951763978 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a40007390 con 0x7f5a4c0b45c0 2022-06-17T12:54:39.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.800+0000 7f5a522fb700 1 -- 172.21.15.36:0/3951763978 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a4c1470d0 con 0x7f5a4c0b45c0 2022-06-17T12:54:39.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.800+0000 7f5a522fb700 1 -- 172.21.15.36:0/3951763978 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a4c1491e0 con 0x7f5a4c0b45c0 2022-06-17T12:54:39.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.804+0000 7f5a3effd700 1 -- 172.21.15.36:0/3951763978 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f5a40006510 con 0x7f5a4c0b45c0 2022-06-17T12:54:39.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.804+0000 7f5a3effd700 1 --2- 172.21.15.36:0/3951763978 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5a28053a40 0x7f5a28055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:39.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.804+0000 7f5a3effd700 1 -- 172.21.15.36:0/3951763978 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5a4006c460 con 0x7f5a4c0b45c0 2022-06-17T12:54:39.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.804+0000 7f5a50af8700 1 --2- 172.21.15.36:0/3951763978 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5a28053a40 0x7f5a28055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:39.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.804+0000 7f5a50af8700 1 --2- 172.21.15.36:0/3951763978 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5a28053a40 0x7f5a28055f00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f5a4800c000 tx=0x7f5a48008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:39.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.812+0000 7f5a3effd700 1 -- 172.21.15.36:0/3951763978 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5a40043200 con 0x7f5a4c0b45c0 2022-06-17T12:54:39.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.940+0000 7f5a522fb700 1 -- 172.21.15.36:0/3951763978 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5a4c149010 con 0x7f5a4c0b45c0 2022-06-17T12:54:39.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.940+0000 7f5a3effd700 1 -- 172.21.15.36:0/3951763978 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v94) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5a4c149010 con 0x7f5a4c0b45c0 2022-06-17T12:54:39.929 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:54:39.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.944+0000 7f5a522fb700 1 -- 172.21.15.36:0/3951763978 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5a28053a40 msgr2=0x7f5a28055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:39.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.944+0000 7f5a522fb700 1 --2- 172.21.15.36:0/3951763978 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5a28053a40 0x7f5a28055f00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f5a4800c000 tx=0x7f5a48008040).stop 2022-06-17T12:54:39.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.944+0000 7f5a522fb700 1 -- 172.21.15.36:0/3951763978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5a4c0b45c0 msgr2=0x7f5a4c152200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:39.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.944+0000 7f5a522fb700 1 --2- 172.21.15.36:0/3951763978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5a4c0b45c0 0x7f5a4c152200 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f5a4c142ef0 tx=0x7f5a40007b70).stop 2022-06-17T12:54:39.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.944+0000 7f5a522fb700 1 -- 172.21.15.36:0/3951763978 shutdown_connections 2022-06-17T12:54:39.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.944+0000 7f5a522fb700 1 --2- 172.21.15.36:0/3951763978 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5a28053a40 0x7f5a28055f00 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.944+0000 7f5a522fb700 1 --2- 172.21.15.36:0/3951763978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a4c0b9860 0x7f5a4c152930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.944+0000 7f5a522fb700 1 --2- 172.21.15.36:0/3951763978 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5a4c0b4f00 0x7f5a4c141f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.944+0000 7f5a522fb700 1 --2- 172.21.15.36:0/3951763978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5a4c0b45c0 0x7f5a4c152200 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:39.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.944+0000 7f5a522fb700 1 -- 172.21.15.36:0/3951763978 >> 172.21.15.36:0/3951763978 conn(0x7f5a4c05c550 msgr2=0x7f5a4c0b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:39.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.944+0000 7f5a522fb700 1 -- 172.21.15.36:0/3951763978 shutdown_connections 2022-06-17T12:54:39.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:39.944+0000 7f5a522fb700 1 -- 172.21.15.36:0/3951763978 wait complete. 2022-06-17T12:54:39.942 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:54:40.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.060+0000 7fdabe9c9700 1 Processor -- start 2022-06-17T12:54:40.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.060+0000 7fdabe9c9700 1 -- start start 2022-06-17T12:54:40.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.060+0000 7fdabe9c9700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fdab80b4120 0x7fdab8146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:40.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.060+0000 7fdabe9c9700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdab8147270 0x7fdab8140100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:40.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.060+0000 7fdabe9c9700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab81476e0 con 0x7fdab80b64f0 2022-06-17T12:54:40.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.060+0000 7fdabe9c9700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdab80b68d0 con 0x7fdab80b4120 2022-06-17T12:54:40.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.060+0000 7fdabe9c9700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab80b6e20 con 0x7fdab8147270 2022-06-17T12:54:40.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdab77fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fdab80b4120 0x7fdab8146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:40.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdab7fff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fdab80b64f0 0x7fdab8143150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54712/0 (socket says 172.21.15.36:54712) 2022-06-17T12:54:40.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdab7fff700 1 -- 172.21.15.36:0/1715900884 learned_addr learned my addr 172.21.15.36:0/1715900884 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:40.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdabcf66700 1 --2- 172.21.15.36:0/1715900884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdab8147270 0x7fdab8140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:40.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdab77fe700 1 -- 172.21.15.36:0/1715900884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdab8147270 msgr2=0x7fdab8140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:40.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdab77fe700 1 --2- 172.21.15.36:0/1715900884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdab8147270 0x7fdab8140100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdab77fe700 1 -- 172.21.15.36:0/1715900884 >> v1:172.21.15.36:6789/0 conn(0x7fdab80b64f0 legacy=0x7fdab8143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:40.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdab77fe700 1 -- 172.21.15.36:0/1715900884 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab81468e0 con 0x7fdab80b4120 2022-06-17T12:54:40.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdab77fe700 1 --2- 172.21.15.36:0/1715900884 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdab80b4120 0x7fdab8146d80 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7fdaa8002f70 tx=0x7fdaa8006df0).ready entity=mon.0 client_cookie=51767a332cc8c5e6 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:40.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdab6ffd700 1 -- 172.21.15.36:0/1715900884 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdaa800d570 con 0x7fdab80b4120 2022-06-17T12:54:40.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdab6ffd700 1 -- 172.21.15.36:0/1715900884 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa8000e90 con 0x7fdab80b4120 2022-06-17T12:54:40.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdab6ffd700 1 -- 172.21.15.36:0/1715900884 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdaa800d8c0 con 0x7fdab80b4120 2022-06-17T12:54:40.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdabe9c9700 1 -- 172.21.15.36:0/1715900884 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdab80b4120 msgr2=0x7fdab8146d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:40.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdabe9c9700 1 --2- 172.21.15.36:0/1715900884 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdab80b4120 0x7fdab8146d80 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7fdaa8002f70 tx=0x7fdaa8006df0).stop 2022-06-17T12:54:40.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdabe9c9700 1 -- 172.21.15.36:0/1715900884 shutdown_connections 2022-06-17T12:54:40.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdabe9c9700 1 --2- 172.21.15.36:0/1715900884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdab8147270 0x7fdab8140100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdabe9c9700 1 --2- 172.21.15.36:0/1715900884 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdab80b4120 0x7fdab8146d80 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdabe9c9700 1 -- 172.21.15.36:0/1715900884 >> 172.21.15.36:0/1715900884 conn(0x7fdab80620f0 msgr2=0x7fdab8062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:40.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdabe9c9700 1 -- 172.21.15.36:0/1715900884 shutdown_connections 2022-06-17T12:54:40.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdabe9c9700 1 -- 172.21.15.36:0/1715900884 wait complete. 2022-06-17T12:54:40.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdabe9c9700 1 Processor -- start 2022-06-17T12:54:40.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdabe9c9700 1 -- start start 2022-06-17T12:54:40.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdabe9c9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdab80b4120 0x7fdab8141b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:40.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdabe9c9700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdab80b64f0 0x7fdab81449c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:40.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.064+0000 7fdabe9c9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdab8147270 0x7fdab813cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:40.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.068+0000 7fdabe9c9700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab80b1380 con 0x7fdab80b4120 2022-06-17T12:54:40.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.068+0000 7fdabe9c9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab80b1620 con 0x7fdab8147270 2022-06-17T12:54:40.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.068+0000 7fdabe9c9700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab80b18c0 con 0x7fdab80b64f0 2022-06-17T12:54:40.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.068+0000 7fdab7fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdab80b4120 0x7fdab8141b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:40.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.068+0000 7fdab7fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdab80b4120 0x7fdab8141b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37188/0 (socket says 172.21.15.36:37188) 2022-06-17T12:54:40.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.068+0000 7fdab7fff700 1 -- 172.21.15.36:0/1714447201 learned_addr learned my addr 172.21.15.36:0/1714447201 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:40.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.068+0000 7fdab77fe700 1 --2- 172.21.15.36:0/1714447201 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdab80b64f0 0x7fdab81449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:40.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.068+0000 7fdabcf66700 1 --2- 172.21.15.36:0/1714447201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdab8147270 0x7fdab813cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:40.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.068+0000 7fdab7fff700 1 -- 172.21.15.36:0/1714447201 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdab80b64f0 msgr2=0x7fdab81449c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:40.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.068+0000 7fdab7fff700 1 --2- 172.21.15.36:0/1714447201 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdab80b64f0 0x7fdab81449c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.068+0000 7fdab7fff700 1 -- 172.21.15.36:0/1714447201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdab8147270 msgr2=0x7fdab813cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:40.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.068+0000 7fdab7fff700 1 --2- 172.21.15.36:0/1714447201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdab8147270 0x7fdab813cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.068+0000 7fdab7fff700 1 -- 172.21.15.36:0/1714447201 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa800b180 con 0x7fdab80b4120 2022-06-17T12:54:40.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.068+0000 7fdabcf66700 1 --2- 172.21.15.36:0/1714447201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdab8147270 0x7fdab813cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:40.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.068+0000 7fdab7fff700 1 --2- 172.21.15.36:0/1714447201 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdab80b4120 0x7fdab8141b40 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7fdab80ae4b0 tx=0x7fdaa0007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:40.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.068+0000 7fdab57fa700 1 -- 172.21.15.36:0/1714447201 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdaa000ce30 con 0x7fdab80b4120 2022-06-17T12:54:40.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.068+0000 7fdab57fa700 1 -- 172.21.15.36:0/1714447201 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa00072e0 con 0x7fdab80b4120 2022-06-17T12:54:40.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.068+0000 7fdabe9c9700 1 -- 172.21.15.36:0/1714447201 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab80b1070 con 0x7fdab80b4120 2022-06-17T12:54:40.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.068+0000 7fdab57fa700 1 -- 172.21.15.36:0/1714447201 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdaa00057c0 con 0x7fdab80b4120 2022-06-17T12:54:40.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.068+0000 7fdabe9c9700 1 -- 172.21.15.36:0/1714447201 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab813d460 con 0x7fdab80b4120 2022-06-17T12:54:40.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.068+0000 7fdabe9c9700 1 -- 172.21.15.36:0/1714447201 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab80b1a80 con 0x7fdab80b4120 2022-06-17T12:54:40.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.068+0000 7fdab57fa700 1 -- 172.21.15.36:0/1714447201 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fdaa0010070 con 0x7fdab80b4120 2022-06-17T12:54:40.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.072+0000 7fdab57fa700 1 --2- 172.21.15.36:0/1714447201 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fda9c053af0 0x7fda9c055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:40.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.072+0000 7fdab57fa700 1 -- 172.21.15.36:0/1714447201 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdaa006b620 con 0x7fdab80b4120 2022-06-17T12:54:40.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.072+0000 7fdab77fe700 1 --2- 172.21.15.36:0/1714447201 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fda9c053af0 0x7fda9c055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:40.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.072+0000 7fdab77fe700 1 --2- 172.21.15.36:0/1714447201 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fda9c053af0 0x7fda9c055fb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7fdaa801a000 tx=0x7fdaa8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:40.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.076+0000 7fdab57fa700 1 -- 172.21.15.36:0/1714447201 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdaa0042380 con 0x7fdab80b4120 2022-06-17T12:54:40.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.204+0000 7fdabe9c9700 1 -- 172.21.15.36:0/1714447201 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdab80b3a60 con 0x7fdab80b4120 2022-06-17T12:54:40.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.208+0000 7fdab57fa700 1 -- 172.21.15.36:0/1714447201 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+10 (secure 0 0 0) 0x7fdaa0018020 con 0x7fdab80b4120 2022-06-17T12:54:40.195 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:54:40.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.208+0000 7fdabe9c9700 1 -- 172.21.15.36:0/1714447201 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fda9c053af0 msgr2=0x7fda9c055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:40.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.208+0000 7fdabe9c9700 1 --2- 172.21.15.36:0/1714447201 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fda9c053af0 0x7fda9c055fb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7fdaa801a000 tx=0x7fdaa8015000).stop 2022-06-17T12:54:40.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.208+0000 7fdabe9c9700 1 -- 172.21.15.36:0/1714447201 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdab80b4120 msgr2=0x7fdab8141b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:40.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.208+0000 7fdabe9c9700 1 --2- 172.21.15.36:0/1714447201 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdab80b4120 0x7fdab8141b40 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7fdab80ae4b0 tx=0x7fdaa0007c30).stop 2022-06-17T12:54:40.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.208+0000 7fdabe9c9700 1 -- 172.21.15.36:0/1714447201 shutdown_connections 2022-06-17T12:54:40.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.208+0000 7fdabe9c9700 1 --2- 172.21.15.36:0/1714447201 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fda9c053af0 0x7fda9c055fb0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.208+0000 7fdabe9c9700 1 --2- 172.21.15.36:0/1714447201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdab8147270 0x7fdab813cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.208+0000 7fdabe9c9700 1 --2- 172.21.15.36:0/1714447201 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdab80b64f0 0x7fdab81449c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.208+0000 7fdabe9c9700 1 --2- 172.21.15.36:0/1714447201 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdab80b4120 0x7fdab8141b40 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.208+0000 7fdabe9c9700 1 -- 172.21.15.36:0/1714447201 >> 172.21.15.36:0/1714447201 conn(0x7fdab80620f0 msgr2=0x7fdab813f300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:40.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.208+0000 7fdabe9c9700 1 -- 172.21.15.36:0/1714447201 shutdown_connections 2022-06-17T12:54:40.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.208+0000 7fdabe9c9700 1 -- 172.21.15.36:0/1714447201 wait complete. 2022-06-17T12:54:40.207 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:54:40.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.332+0000 7f8bb2621700 1 Processor -- start 2022-06-17T12:54:40.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.332+0000 7f8bb2621700 1 -- start start 2022-06-17T12:54:40.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.332+0000 7f8bb2621700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8bac147490 0x7f8bac140100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:40.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.332+0000 7f8bb2621700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bac0b68d0 con 0x7f8bac0b4120 2022-06-17T12:54:40.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.332+0000 7f8bb2621700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bac0b6e20 con 0x7f8bac0b64f0 2022-06-17T12:54:40.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.332+0000 7f8bb2621700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bac1468e0 con 0x7f8bac147490 2022-06-17T12:54:40.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.332+0000 7f8bb0bbe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8bac147490 0x7f8bac140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:40.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.332+0000 7f8bab7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f8bac0b4120 0x7f8bac146d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54728/0 (socket says 172.21.15.36:54728) 2022-06-17T12:54:40.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.332+0000 7f8bab7fe700 1 -- 172.21.15.36:0/2712390783 learned_addr learned my addr 172.21.15.36:0/2712390783 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:40.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.332+0000 7f8bb0bbe700 1 -- 172.21.15.36:0/2712390783 >> v1:172.21.15.70:6789/0 conn(0x7f8bac0b64f0 legacy=0x7f8bac143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:40.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.332+0000 7f8bb0bbe700 1 -- 172.21.15.36:0/2712390783 >> v1:172.21.15.36:6789/0 conn(0x7f8bac0b4120 legacy=0x7f8bac146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:40.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.332+0000 7f8bb0bbe700 1 -- 172.21.15.36:0/2712390783 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bac0b3a60 con 0x7f8bac147490 2022-06-17T12:54:40.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.332+0000 7f8bb0bbe700 1 --2- 172.21.15.36:0/2712390783 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8bac147490 0x7f8bac140100 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7f8ba40032f0 tx=0x7f8ba40091d0).ready entity=mon.0 client_cookie=c8097da12e0b1137 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:40.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.332+0000 7f8baaffd700 1 -- 172.21.15.36:0/2712390783 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ba4009ea0 con 0x7f8bac147490 2022-06-17T12:54:40.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.332+0000 7f8baaffd700 1 -- 172.21.15.36:0/2712390783 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba400b8a0 con 0x7f8bac147490 2022-06-17T12:54:40.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.332+0000 7f8baaffd700 1 -- 172.21.15.36:0/2712390783 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ba400ba10 con 0x7f8bac147490 2022-06-17T12:54:40.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.332+0000 7f8bb2621700 1 -- 172.21.15.36:0/2712390783 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8bac147490 msgr2=0x7f8bac140100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:40.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.332+0000 7f8bb2621700 1 --2- 172.21.15.36:0/2712390783 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8bac147490 0x7f8bac140100 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7f8ba40032f0 tx=0x7f8ba40091d0).stop 2022-06-17T12:54:40.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8bb2621700 1 -- 172.21.15.36:0/2712390783 shutdown_connections 2022-06-17T12:54:40.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8bb2621700 1 --2- 172.21.15.36:0/2712390783 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8bac147490 0x7f8bac140100 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8bb2621700 1 -- 172.21.15.36:0/2712390783 >> 172.21.15.36:0/2712390783 conn(0x7f8bac0620f0 msgr2=0x7f8bac062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:40.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8bb2621700 1 -- 172.21.15.36:0/2712390783 shutdown_connections 2022-06-17T12:54:40.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8bb2621700 1 -- 172.21.15.36:0/2712390783 wait complete. 2022-06-17T12:54:40.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8bb2621700 1 Processor -- start 2022-06-17T12:54:40.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8bb2621700 1 -- start start 2022-06-17T12:54:40.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8bb2621700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8bac0b4120 0x7f8bac0b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:40.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8bb2621700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8bac0b64f0 0x7f8bac13d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:40.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8bb2621700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bac147490 0x7f8bac13d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:40.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8bb2621700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac152bd0 con 0x7f8bac0b4120 2022-06-17T12:54:40.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8bb2621700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac152e70 con 0x7f8bac147490 2022-06-17T12:54:40.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8bb2621700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac153110 con 0x7f8bac0b64f0 2022-06-17T12:54:40.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8babfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8bac0b4120 0x7f8bac0b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:40.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8bb0bbe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bac147490 0x7f8bac13d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:40.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8babfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8bac0b4120 0x7f8bac0b20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37202/0 (socket says 172.21.15.36:37202) 2022-06-17T12:54:40.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8babfff700 1 -- 172.21.15.36:0/897365352 learned_addr learned my addr 172.21.15.36:0/897365352 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:40.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8bab7fe700 1 --2- 172.21.15.36:0/897365352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8bac0b64f0 0x7f8bac13d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:40.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8bb0bbe700 1 -- 172.21.15.36:0/897365352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8bac0b64f0 msgr2=0x7f8bac13d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:40.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8bb0bbe700 1 --2- 172.21.15.36:0/897365352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8bac0b64f0 0x7f8bac13d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8bb0bbe700 1 -- 172.21.15.36:0/897365352 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8bac0b4120 msgr2=0x7f8bac0b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:40.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8bb0bbe700 1 --2- 172.21.15.36:0/897365352 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8bac0b4120 0x7f8bac0b20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8bb0bbe700 1 -- 172.21.15.36:0/897365352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba4009470 con 0x7f8bac147490 2022-06-17T12:54:40.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8babfff700 1 --2- 172.21.15.36:0/897365352 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8bac0b4120 0x7f8bac0b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:40.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.336+0000 7f8bb0bbe700 1 --2- 172.21.15.36:0/897365352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bac147490 0x7f8bac13d5f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f8ba40032f0 tx=0x7f8ba4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:40.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.340+0000 7f8ba97fa700 1 -- 172.21.15.36:0/897365352 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ba4009ea0 con 0x7f8bac147490 2022-06-17T12:54:40.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.340+0000 7f8ba97fa700 1 -- 172.21.15.36:0/897365352 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba401f3f0 con 0x7f8bac147490 2022-06-17T12:54:40.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.340+0000 7f8bb2621700 1 -- 172.21.15.36:0/897365352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bac1533b0 con 0x7f8bac147490 2022-06-17T12:54:40.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.340+0000 7f8ba97fa700 1 -- 172.21.15.36:0/897365352 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ba400bee0 con 0x7f8bac147490 2022-06-17T12:54:40.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.340+0000 7f8bb2621700 1 -- 172.21.15.36:0/897365352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bac153850 con 0x7f8bac147490 2022-06-17T12:54:40.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.340+0000 7f8bb2621700 1 -- 172.21.15.36:0/897365352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bac057410 con 0x7f8bac147490 2022-06-17T12:54:40.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.340+0000 7f8ba97fa700 1 -- 172.21.15.36:0/897365352 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8ba4026030 con 0x7f8bac147490 2022-06-17T12:54:40.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.340+0000 7f8ba97fa700 1 --2- 172.21.15.36:0/897365352 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b8c053a40 0x7f8b8c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:40.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.340+0000 7f8ba97fa700 1 -- 172.21.15.36:0/897365352 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8ba406cc20 con 0x7f8bac147490 2022-06-17T12:54:40.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.340+0000 7f8babfff700 1 --2- 172.21.15.36:0/897365352 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b8c053a40 0x7f8b8c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:40.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.340+0000 7f8babfff700 1 --2- 172.21.15.36:0/897365352 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b8c053a40 0x7f8b8c055f00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f8b9c00d000 tx=0x7f8b9c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:40.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.348+0000 7f8ba97fa700 1 -- 172.21.15.36:0/897365352 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8ba4043910 con 0x7f8bac147490 2022-06-17T12:54:40.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.476+0000 7f8bb2621700 1 -- 172.21.15.36:0/897365352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8bac141cf0 con 0x7f8bac147490 2022-06-17T12:54:40.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.476+0000 7f8ba97fa700 1 -- 172.21.15.36:0/897365352 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+10 (secure 0 0 0) 0x7f8ba40240a0 con 0x7f8bac147490 2022-06-17T12:54:40.463 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:54:40.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.480+0000 7f8bb2621700 1 -- 172.21.15.36:0/897365352 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b8c053a40 msgr2=0x7f8b8c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:40.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.480+0000 7f8bb2621700 1 --2- 172.21.15.36:0/897365352 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b8c053a40 0x7f8b8c055f00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f8b9c00d000 tx=0x7f8b9c009040).stop 2022-06-17T12:54:40.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.480+0000 7f8bb2621700 1 -- 172.21.15.36:0/897365352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bac147490 msgr2=0x7f8bac13d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:40.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.480+0000 7f8bb2621700 1 --2- 172.21.15.36:0/897365352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bac147490 0x7f8bac13d5f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f8ba40032f0 tx=0x7f8ba4016000).stop 2022-06-17T12:54:40.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.480+0000 7f8bb2621700 1 -- 172.21.15.36:0/897365352 shutdown_connections 2022-06-17T12:54:40.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.480+0000 7f8bb2621700 1 --2- 172.21.15.36:0/897365352 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8b8c053a40 0x7f8b8c055f00 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.480+0000 7f8bb2621700 1 --2- 172.21.15.36:0/897365352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bac147490 0x7f8bac13d5f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.480+0000 7f8bb2621700 1 --2- 172.21.15.36:0/897365352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8bac0b64f0 0x7f8bac13d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.480+0000 7f8bb2621700 1 --2- 172.21.15.36:0/897365352 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8bac0b4120 0x7f8bac0b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.480+0000 7f8bb2621700 1 -- 172.21.15.36:0/897365352 >> 172.21.15.36:0/897365352 conn(0x7f8bac0620f0 msgr2=0x7f8bac0b7680 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:40.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.480+0000 7f8bb2621700 1 -- 172.21.15.36:0/897365352 shutdown_connections 2022-06-17T12:54:40.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.480+0000 7f8bb2621700 1 -- 172.21.15.36:0/897365352 wait complete. 2022-06-17T12:54:40.480 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:54:40.481 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:54:40.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.600+0000 7f2a3670f700 1 Processor -- start 2022-06-17T12:54:40.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.600+0000 7f2a3670f700 1 -- start start 2022-06-17T12:54:40.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a3670f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2a30060ac0 0x7f2a3013c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:40.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a3670f700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2a300b1890 0x7f2a3013c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:40.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a3670f700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a300b0230 con 0x7f2a300b1ff0 2022-06-17T12:54:40.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a3670f700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a30143800 con 0x7f2a30060ac0 2022-06-17T12:54:40.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a3670f700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a30060ea0 con 0x7f2a300b1890 2022-06-17T12:54:40.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a3570d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2a30060ac0 0x7f2a3013c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:40.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a3570d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2a30060ac0 0x7f2a3013c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37210/0 (socket says 172.21.15.36:37210) 2022-06-17T12:54:40.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a3570d700 1 -- 172.21.15.36:0/857300969 learned_addr learned my addr 172.21.15.36:0/857300969 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:40.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a34f0c700 1 --2- 172.21.15.36:0/857300969 >> v2:172.21.15.70:3300/0 conn(0x7f2a300b1890 0x7f2a3013c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:40.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a3570d700 1 -- 172.21.15.36:0/857300969 >> v2:172.21.15.70:3300/0 conn(0x7f2a300b1890 msgr2=0x7f2a3013c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:40.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a3570d700 1 --2- 172.21.15.36:0/857300969 >> v2:172.21.15.70:3300/0 conn(0x7f2a300b1890 0x7f2a3013c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a3570d700 1 -- 172.21.15.36:0/857300969 >> v1:172.21.15.70:6789/0 conn(0x7f2a300b1ff0 legacy=0x7f2a300b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:40.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a3570d700 1 -- 172.21.15.36:0/857300969 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a300afcf0 con 0x7f2a30060ac0 2022-06-17T12:54:40.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a34f0c700 1 --2- 172.21.15.36:0/857300969 >> v2:172.21.15.70:3300/0 conn(0x7f2a300b1890 0x7f2a3013c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:40.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a3570d700 1 --2- 172.21.15.36:0/857300969 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2a30060ac0 0x7f2a3013c2b0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7f2a2c002f70 tx=0x7f2a2c006eb0).ready entity=mon.0 client_cookie=cbb5f75b003072c0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:40.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a23fff700 1 -- 172.21.15.36:0/857300969 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a2c00d610 con 0x7f2a30060ac0 2022-06-17T12:54:40.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a23fff700 1 -- 172.21.15.36:0/857300969 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2c000e90 con 0x7f2a30060ac0 2022-06-17T12:54:40.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a23fff700 1 -- 172.21.15.36:0/857300969 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a2c00d9a0 con 0x7f2a30060ac0 2022-06-17T12:54:40.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a3670f700 1 -- 172.21.15.36:0/857300969 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2a30060ac0 msgr2=0x7f2a3013c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:40.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a3670f700 1 --2- 172.21.15.36:0/857300969 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2a30060ac0 0x7f2a3013c2b0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7f2a2c002f70 tx=0x7f2a2c006eb0).stop 2022-06-17T12:54:40.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a3670f700 1 -- 172.21.15.36:0/857300969 shutdown_connections 2022-06-17T12:54:40.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a3670f700 1 --2- 172.21.15.36:0/857300969 >> v2:172.21.15.70:3300/0 conn(0x7f2a300b1890 0x7f2a3013c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a3670f700 1 --2- 172.21.15.36:0/857300969 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2a30060ac0 0x7f2a3013c2b0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a3670f700 1 -- 172.21.15.36:0/857300969 >> 172.21.15.36:0/857300969 conn(0x7f2a300b65f0 msgr2=0x7f2a300b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:40.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a3670f700 1 -- 172.21.15.36:0/857300969 shutdown_connections 2022-06-17T12:54:40.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.604+0000 7f2a3670f700 1 -- 172.21.15.36:0/857300969 wait complete. 2022-06-17T12:54:40.602 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a3670f700 1 Processor -- start 2022-06-17T12:54:40.602 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a3670f700 1 -- start start 2022-06-17T12:54:40.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a3670f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2a30060ac0 0x7f2a300adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:40.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a3670f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a300b1890 0x7f2a301557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:40.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a3670f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2a300b1ff0 0x7f2a30155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:40.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a3670f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3013c200 con 0x7f2a30060ac0 2022-06-17T12:54:40.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a3670f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3013c4a0 con 0x7f2a300b1890 2022-06-17T12:54:40.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a3670f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3013c940 con 0x7f2a300b1ff0 2022-06-17T12:54:40.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a3570d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2a30060ac0 0x7f2a300adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:40.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a3570d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2a30060ac0 0x7f2a300adf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37216/0 (socket says 172.21.15.36:37216) 2022-06-17T12:54:40.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a3570d700 1 -- 172.21.15.36:0/4236604788 learned_addr learned my addr 172.21.15.36:0/4236604788 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:40.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a34f0c700 1 --2- 172.21.15.36:0/4236604788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a300b1890 0x7f2a301557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:40.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a35f0e700 1 --2- 172.21.15.36:0/4236604788 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2a300b1ff0 0x7f2a30155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:40.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a3570d700 1 -- 172.21.15.36:0/4236604788 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2a300b1ff0 msgr2=0x7f2a30155ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:40.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a3570d700 1 --2- 172.21.15.36:0/4236604788 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2a300b1ff0 0x7f2a30155ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a3570d700 1 -- 172.21.15.36:0/4236604788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a300b1890 msgr2=0x7f2a301557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:40.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a3570d700 1 --2- 172.21.15.36:0/4236604788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a300b1890 0x7f2a301557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a3570d700 1 -- 172.21.15.36:0/4236604788 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c00b1e0 con 0x7f2a30060ac0 2022-06-17T12:54:40.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a35f0e700 1 --2- 172.21.15.36:0/4236604788 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2a300b1ff0 0x7f2a30155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:40.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a3570d700 1 --2- 172.21.15.36:0/4236604788 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2a30060ac0 0x7f2a300adf70 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0x7f2a2c002f70 tx=0x7f2a2c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:40.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a227fc700 1 -- 172.21.15.36:0/4236604788 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a2c011710 con 0x7f2a30060ac0 2022-06-17T12:54:40.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a227fc700 1 -- 172.21.15.36:0/4236604788 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2c011870 con 0x7f2a30060ac0 2022-06-17T12:54:40.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a3670f700 1 -- 172.21.15.36:0/4236604788 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a30064bc0 con 0x7f2a30060ac0 2022-06-17T12:54:40.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a227fc700 1 -- 172.21.15.36:0/4236604788 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a2c011bd0 con 0x7f2a30060ac0 2022-06-17T12:54:40.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.608+0000 7f2a3670f700 1 -- 172.21.15.36:0/4236604788 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3013e6c0 con 0x7f2a30060ac0 2022-06-17T12:54:40.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.612+0000 7f2a3670f700 1 -- 172.21.15.36:0/4236604788 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3013e8b0 con 0x7f2a30060ac0 2022-06-17T12:54:40.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.612+0000 7f2a227fc700 1 -- 172.21.15.36:0/4236604788 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2a2c025030 con 0x7f2a30060ac0 2022-06-17T12:54:40.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.612+0000 7f2a227fc700 1 --2- 172.21.15.36:0/4236604788 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2a10053ad0 0x7f2a10055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:40.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.612+0000 7f2a227fc700 1 -- 172.21.15.36:0/4236604788 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2a2c06d0f0 con 0x7f2a30060ac0 2022-06-17T12:54:40.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.612+0000 7f2a34f0c700 1 --2- 172.21.15.36:0/4236604788 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2a10053ad0 0x7f2a10055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:40.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.612+0000 7f2a34f0c700 1 --2- 172.21.15.36:0/4236604788 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2a10053ad0 0x7f2a10055f90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f2a2400c000 tx=0x7f2a24008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:40.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.620+0000 7f2a227fc700 1 -- 172.21.15.36:0/4236604788 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2a2c043ef0 con 0x7f2a30060ac0 2022-06-17T12:54:40.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.784+0000 7f2a3670f700 1 -- 172.21.15.36:0/4236604788 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a300aff90 con 0x7f2a30060ac0 2022-06-17T12:54:40.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.784+0000 7f2a227fc700 1 -- 172.21.15.36:0/4236604788 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f2a2c023020 con 0x7f2a30060ac0 2022-06-17T12:54:40.773 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:40.773 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":47,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/138643404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":138643404},{"type":"v1","addr":"172.21.15.70:6833","nonce":138643404}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":46}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:54:35.949730+0000","modified":"2022-06-17T12:54:36.953427+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7074},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7074":{"gid":7074,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/3267343301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3267343301},{"type":"v1","addr":"172.21.15.36:6835","nonce":3267343301}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-06-17T12:54:40.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.788+0000 7f2a3670f700 1 -- 172.21.15.36:0/4236604788 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2a10053ad0 msgr2=0x7f2a10055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:40.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.788+0000 7f2a3670f700 1 --2- 172.21.15.36:0/4236604788 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2a10053ad0 0x7f2a10055f90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f2a2400c000 tx=0x7f2a24008040).stop 2022-06-17T12:54:40.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.788+0000 7f2a3670f700 1 -- 172.21.15.36:0/4236604788 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2a30060ac0 msgr2=0x7f2a300adf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:40.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.788+0000 7f2a3670f700 1 --2- 172.21.15.36:0/4236604788 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2a30060ac0 0x7f2a300adf70 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0x7f2a2c002f70 tx=0x7f2a2c015000).stop 2022-06-17T12:54:40.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.788+0000 7f2a3670f700 1 -- 172.21.15.36:0/4236604788 shutdown_connections 2022-06-17T12:54:40.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.788+0000 7f2a3670f700 1 --2- 172.21.15.36:0/4236604788 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2a10053ad0 0x7f2a10055f90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.788+0000 7f2a3670f700 1 --2- 172.21.15.36:0/4236604788 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2a300b1ff0 0x7f2a30155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.788+0000 7f2a3670f700 1 --2- 172.21.15.36:0/4236604788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a300b1890 0x7f2a301557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.788+0000 7f2a3670f700 1 --2- 172.21.15.36:0/4236604788 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2a30060ac0 0x7f2a300adf70 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:40.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.788+0000 7f2a3670f700 1 -- 172.21.15.36:0/4236604788 >> 172.21.15.36:0/4236604788 conn(0x7f2a300b65f0 msgr2=0x7f2a30139be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:40.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.788+0000 7f2a3670f700 1 -- 172.21.15.36:0/4236604788 shutdown_connections 2022-06-17T12:54:40.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:40.788+0000 7f2a3670f700 1 -- 172.21.15.36:0/4236604788 wait complete. 2022-06-17T12:54:40.780 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 47 2022-06-17T12:54:40.989 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:54:35.949730+0000', 'modified': '2022-06-17T12:54:36.953427+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7074}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7074': {'gid': 7074, 'name': 'a', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.36:6835/3267343301', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 3267343301}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 3267343301}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:54:40.990 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:54:40.990 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:54:41.108 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":7074,"want_state":"up:active","state":"up:active","rank_uptime":5.1633786199999996,"mdsmap_epoch":47,"osdmap_epoch":94,"osdmap_epoch_barrier":93,"uptime":9.5799373039999995} 2022-06-17T12:54:41.109 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 7074, "mdsmap_epoch": 47, "osdmap_epoch": 94, "osdmap_epoch_barrier": 93, "rank_uptime": 5.16337862, "state": "up:active", "uptime": 9.579937304, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:54:41.109 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:54:41.110 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-17T12:54:41.110 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi107.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-17T12:54:41.111 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:41.111 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:54:41.121 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:54:41.183 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:54:41.183 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:54:41.196 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:54:41.197 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:54:41.198 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:41.199 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:54:41.204 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:54:41.204 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:54:41.250 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:54:41.269 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:54:41.270 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:54:41.282 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:54:41.283 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:54:41.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:41.284 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:54:41.330 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:2022-06-17T12:54:41.339+0000 7f4d4e77b080 -1 init, newargv = 0x55e8fa3939f0 newargc=9 2022-06-17T12:54:41.330 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22600]: starting ceph client 2022-06-17T12:54:41.338 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22600]: starting fuse 2022-06-17T12:54:42.438 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:54:42.456 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:54:42.457 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:54:42.514 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:54:42.514 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:54:42.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:42.516 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:54:42.522 INFO:teuthology.orchestra.run.smithi107.stdout:52 2022-06-17T12:54:42.522 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:54:42.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:42.523 DEBUG:teuthology.orchestra.run.smithi107:> sudo python3 -c ' 2022-06-17T12:54:42.524 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2022-06-17T12:54:42.524 DEBUG:teuthology.orchestra.run.smithi107:> import re 2022-06-17T12:54:42.525 DEBUG:teuthology.orchestra.run.smithi107:> import os 2022-06-17T12:54:42.525 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2022-06-17T12:54:42.526 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:54:42.526 DEBUG:teuthology.orchestra.run.smithi107:> def find_socket(client_name): 2022-06-17T12:54:42.527 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-17T12:54:42.527 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2022-06-17T12:54:42.528 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:54:42.528 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2022-06-17T12:54:42.528 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2022-06-17T12:54:42.529 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2022-06-17T12:54:42.529 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2022-06-17T12:54:42.530 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:54:42.530 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2022-06-17T12:54:42.530 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:54:42.531 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:54:42.531 DEBUG:teuthology.orchestra.run.smithi107:> return f 2022-06-17T12:54:42.532 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:54:42.532 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:54:42.533 DEBUG:teuthology.orchestra.run.smithi107:> print(find_socket("client.0")) 2022-06-17T12:54:42.533 DEBUG:teuthology.orchestra.run.smithi107:> ' 2022-06-17T12:54:42.608 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.0.22600.asok 2022-06-17T12:54:42.614 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22600.asok 2022-06-17T12:54:42.614 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:42.615 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22600.asok status 2022-06-17T12:54:42.706 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2022-06-17T12:54:42.710 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2022-06-17T12:54:42.710 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:54:42.711 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:54:42.711 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "0", 2022-06-17T12:54:42.711 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2022-06-17T12:54:42.711 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-17T12:54:42.712 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "22600", 2022-06-17T12:54:42.712 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/" 2022-06-17T12:54:42.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:54:42.712 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2022-06-17T12:54:42.712 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2022-06-17T12:54:42.713 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 6263, 2022-06-17T12:54:42.713 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2022-06-17T12:54:42.713 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2022-06-17T12:54:42.713 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2022-06-17T12:54:42.714 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 6263 2022-06-17T12:54:42.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:54:42.714 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:54:42.715 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:54:42.715 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:54:42.716 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 2106877740 2022-06-17T12:54:42.716 INFO:teuthology.orchestra.run.smithi107.stdout: } 2022-06-17T12:54:42.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:54:42.717 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:54:42.718 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:54:42.718 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:54:42.719 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 2106877740 2022-06-17T12:54:42.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:54:42.720 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.6263 172.21.15.107:0/2106877740", 2022-06-17T12:54:42.720 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "172.21.15.107:0/2106877740", 2022-06-17T12:54:42.720 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 1, 2022-06-17T12:54:42.721 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 47, 2022-06-17T12:54:42.721 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 94, 2022-06-17T12:54:42.721 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:54:42.722 INFO:teuthology.orchestra.run.smithi107.stdout: "blacklisted": false, 2022-06-17T12:54:42.722 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2022-06-17T12:54:42.723 INFO:teuthology.orchestra.run.smithi107.stdout:} 2022-06-17T12:54:42.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:42.726 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:42.727 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:54:42.744 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:54:42.744 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:54:42.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:42.746 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:54:42.805 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:54:42.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.932+0000 7f6f92092700 1 Processor -- start 2022-06-17T12:54:42.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.932+0000 7f6f92092700 1 -- start start 2022-06-17T12:54:42.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.932+0000 7f6f92092700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f8c063730 0x7f6f8c065e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:42.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.932+0000 7f6f92092700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6f8c0b1930 0x7f6f8c1431f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:42.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.932+0000 7f6f92092700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f8c063b10 con 0x7f6f8c0b2900 2022-06-17T12:54:42.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.932+0000 7f6f92092700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f8c065430 con 0x7f6f8c0b1930 2022-06-17T12:54:42.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.932+0000 7f6f92092700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c0656d0 con 0x7f6f8c063730 2022-06-17T12:54:42.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.932+0000 7f6f9088f700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6f8c0b1930 0x7f6f8c1431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:42.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.932+0000 7f6f9088f700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6f8c0b1930 0x7f6f8c1431f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:57606/0 (socket says 172.21.15.36:57606) 2022-06-17T12:54:42.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.932+0000 7f6f91090700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f8c063730 0x7f6f8c065e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:42.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.932+0000 7f6f9088f700 1 -- 172.21.15.36:0/193322941 learned_addr learned my addr 172.21.15.36:0/193322941 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:42.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.932+0000 7f6f91090700 1 -- 172.21.15.36:0/193322941 >> v2:172.21.15.70:3300/0 conn(0x7f6f8c0b1930 msgr2=0x7f6f8c1431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:42.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.932+0000 7f6f91090700 1 --2- 172.21.15.36:0/193322941 >> v2:172.21.15.70:3300/0 conn(0x7f6f8c0b1930 0x7f6f8c1431f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:42.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.932+0000 7f6f91090700 1 -- 172.21.15.36:0/193322941 >> v1:172.21.15.70:6789/0 conn(0x7f6f8c0b2900 legacy=0x7f6f8c0b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:42.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.932+0000 7f6f91090700 1 -- 172.21.15.36:0/193322941 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8c065970 con 0x7f6f8c063730 2022-06-17T12:54:42.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.932+0000 7f6f91090700 1 --2- 172.21.15.36:0/193322941 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f8c063730 0x7f6f8c065e10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f6f7c002f70 tx=0x7f6f7c006df0).ready entity=mon.2 client_cookie=c030d24b89bcc1f4 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:42.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.936+0000 7f6f83fff700 1 -- 172.21.15.36:0/193322941 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f7c00d570 con 0x7f6f8c063730 2022-06-17T12:54:42.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.936+0000 7f6f83fff700 1 -- 172.21.15.36:0/193322941 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7c000e90 con 0x7f6f8c063730 2022-06-17T12:54:42.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.936+0000 7f6f83fff700 1 -- 172.21.15.36:0/193322941 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f7c00d8c0 con 0x7f6f8c063730 2022-06-17T12:54:42.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.936+0000 7f6f92092700 1 -- 172.21.15.36:0/193322941 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f8c063730 msgr2=0x7f6f8c065e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:42.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.936+0000 7f6f92092700 1 --2- 172.21.15.36:0/193322941 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f8c063730 0x7f6f8c065e10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f6f7c002f70 tx=0x7f6f7c006df0).stop 2022-06-17T12:54:42.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.936+0000 7f6f92092700 1 -- 172.21.15.36:0/193322941 shutdown_connections 2022-06-17T12:54:42.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.936+0000 7f6f92092700 1 --2- 172.21.15.36:0/193322941 >> v2:172.21.15.70:3300/0 conn(0x7f6f8c0b1930 0x7f6f8c1431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:42.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.936+0000 7f6f92092700 1 --2- 172.21.15.36:0/193322941 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f8c063730 0x7f6f8c065e10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:42.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.936+0000 7f6f92092700 1 -- 172.21.15.36:0/193322941 >> 172.21.15.36:0/193322941 conn(0x7f6f8c0614b0 msgr2=0x7f6f8c0618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:42.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.936+0000 7f6f92092700 1 -- 172.21.15.36:0/193322941 shutdown_connections 2022-06-17T12:54:42.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.936+0000 7f6f92092700 1 -- 172.21.15.36:0/193322941 wait complete. 2022-06-17T12:54:42.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.936+0000 7f6f92092700 1 Processor -- start 2022-06-17T12:54:42.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.936+0000 7f6f92092700 1 -- start start 2022-06-17T12:54:42.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.936+0000 7f6f92092700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f8c063730 0x7f6f8c1158e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:42.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.936+0000 7f6f92092700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f8c0b1930 0x7f6f8c1425b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:42.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.936+0000 7f6f92092700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6f8c0b2900 0x7f6f8c14f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:42.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.936+0000 7f6f92092700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c1151e0 con 0x7f6f8c0b2900 2022-06-17T12:54:42.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.936+0000 7f6f92092700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c115440 con 0x7f6f8c063730 2022-06-17T12:54:42.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.936+0000 7f6f92092700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c0b0380 con 0x7f6f8c0b1930 2022-06-17T12:54:42.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.936+0000 7f6f91090700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f8c063730 0x7f6f8c1158e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:42.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.936+0000 7f6f9088f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f8c0b1930 0x7f6f8c1425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:42.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.936+0000 7f6f91090700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f8c063730 0x7f6f8c1158e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:57612/0 (socket says 172.21.15.36:57612) 2022-06-17T12:54:42.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.936+0000 7f6f91090700 1 -- 172.21.15.36:0/17958041 learned_addr learned my addr 172.21.15.36:0/17958041 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:42.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.936+0000 7f6f91891700 1 --2- 172.21.15.36:0/17958041 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6f8c0b2900 0x7f6f8c14f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:42.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.940+0000 7f6f9088f700 1 -- 172.21.15.36:0/17958041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f8c063730 msgr2=0x7f6f8c1158e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:42.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.940+0000 7f6f9088f700 1 --2- 172.21.15.36:0/17958041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f8c063730 0x7f6f8c1158e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:42.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.940+0000 7f6f9088f700 1 -- 172.21.15.36:0/17958041 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6f8c0b2900 msgr2=0x7f6f8c14f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:42.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.940+0000 7f6f9088f700 1 --2- 172.21.15.36:0/17958041 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6f8c0b2900 0x7f6f8c14f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:42.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.940+0000 7f6f9088f700 1 -- 172.21.15.36:0/17958041 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7c00b180 con 0x7f6f8c0b1930 2022-06-17T12:54:42.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.940+0000 7f6f91090700 1 --2- 172.21.15.36:0/17958041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f8c063730 0x7f6f8c1158e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:42.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.940+0000 7f6f9088f700 1 --2- 172.21.15.36:0/17958041 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f8c0b1930 0x7f6f8c1425b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f6f8800e000 tx=0x7f6f8800b530).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:42.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.940+0000 7f6f827fc700 1 -- 172.21.15.36:0/17958041 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f8800dce0 con 0x7f6f8c0b1930 2022-06-17T12:54:42.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.940+0000 7f6f827fc700 1 -- 172.21.15.36:0/17958041 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8800de40 con 0x7f6f8c0b1930 2022-06-17T12:54:42.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.940+0000 7f6f92092700 1 -- 172.21.15.36:0/17958041 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8c115dd0 con 0x7f6f8c0b1930 2022-06-17T12:54:42.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.940+0000 7f6f827fc700 1 -- 172.21.15.36:0/17958041 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f88010630 con 0x7f6f8c0b1930 2022-06-17T12:54:42.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.940+0000 7f6f92092700 1 -- 172.21.15.36:0/17958041 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f8c0af090 con 0x7f6f8c0b1930 2022-06-17T12:54:42.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.940+0000 7f6f92092700 1 -- 172.21.15.36:0/17958041 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f8c0b3b30 con 0x7f6f8c0b1930 2022-06-17T12:54:42.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.940+0000 7f6f827fc700 1 -- 172.21.15.36:0/17958041 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6f88020070 con 0x7f6f8c0b1930 2022-06-17T12:54:42.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.940+0000 7f6f827fc700 1 --2- 172.21.15.36:0/17958041 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6f6c053a40 0x7f6f6c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:42.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.944+0000 7f6f827fc700 1 -- 172.21.15.36:0/17958041 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6f8806cad0 con 0x7f6f8c0b1930 2022-06-17T12:54:42.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.944+0000 7f6f91090700 1 --2- 172.21.15.36:0/17958041 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6f6c053a40 0x7f6f6c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:42.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.944+0000 7f6f91090700 1 --2- 172.21.15.36:0/17958041 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6f6c053a40 0x7f6f6c055f00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f6f7c01a000 tx=0x7f6f7c014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:42.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:42.952+0000 7f6f827fc700 1 -- 172.21.15.36:0/17958041 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6f88043740 con 0x7f6f8c0b1930 2022-06-17T12:54:43.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.080+0000 7f6f92092700 1 -- 172.21.15.36:0/17958041 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f8c115610 con 0x7f6f8c0b1930 2022-06-17T12:54:43.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.080+0000 7f6f827fc700 1 -- 172.21.15.36:0/17958041 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f6f88046d60 con 0x7f6f8c0b1930 2022-06-17T12:54:43.068 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:43.069 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":94,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:54:37.964330+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:54:34.381157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-06-17T12:54:35.204631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:54:43.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.084+0000 7f6f92092700 1 -- 172.21.15.36:0/17958041 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6f6c053a40 msgr2=0x7f6f6c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:43.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.084+0000 7f6f92092700 1 --2- 172.21.15.36:0/17958041 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6f6c053a40 0x7f6f6c055f00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f6f7c01a000 tx=0x7f6f7c014000).stop 2022-06-17T12:54:43.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.084+0000 7f6f92092700 1 -- 172.21.15.36:0/17958041 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f8c0b1930 msgr2=0x7f6f8c1425b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:43.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.084+0000 7f6f92092700 1 --2- 172.21.15.36:0/17958041 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f8c0b1930 0x7f6f8c1425b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f6f8800e000 tx=0x7f6f8800b530).stop 2022-06-17T12:54:43.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.084+0000 7f6f92092700 1 -- 172.21.15.36:0/17958041 shutdown_connections 2022-06-17T12:54:43.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.084+0000 7f6f92092700 1 --2- 172.21.15.36:0/17958041 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6f6c053a40 0x7f6f6c055f00 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.084+0000 7f6f92092700 1 --2- 172.21.15.36:0/17958041 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6f8c0b2900 0x7f6f8c14f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.084+0000 7f6f92092700 1 --2- 172.21.15.36:0/17958041 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f8c0b1930 0x7f6f8c1425b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.084+0000 7f6f92092700 1 --2- 172.21.15.36:0/17958041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f8c063730 0x7f6f8c1158e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.084+0000 7f6f92092700 1 -- 172.21.15.36:0/17958041 >> 172.21.15.36:0/17958041 conn(0x7f6f8c0614b0 msgr2=0x7f6f8c13d160 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:43.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.084+0000 7f6f92092700 1 -- 172.21.15.36:0/17958041 shutdown_connections 2022-06-17T12:54:43.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.084+0000 7f6f92092700 1 -- 172.21.15.36:0/17958041 wait complete. 2022-06-17T12:54:43.082 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:54:34.381157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:54:35.204631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:54:43.083 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:54:43.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27d39b6700 1 Processor -- start 2022-06-17T12:54:43.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27d39b6700 1 -- start start 2022-06-17T12:54:43.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27d39b6700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f27cc0b64f0 0x7f27cc143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:43.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27d39b6700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f27cc0b4120 0x7f27cc146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:43.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27d39b6700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27cc1476e0 con 0x7f27cc147270 2022-06-17T12:54:43.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27d39b6700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27cc0b68d0 con 0x7f27cc0b4120 2022-06-17T12:54:43.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27d39b6700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27cc0b6e20 con 0x7f27cc0b64f0 2022-06-17T12:54:43.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27d0f51700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f27cc0b4120 0x7f27cc146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:43.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27d1f53700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f27cc147270 0x7f27cc140100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54768/0 (socket says 172.21.15.36:54768) 2022-06-17T12:54:43.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27d1f53700 1 -- 172.21.15.36:0/3540787204 learned_addr learned my addr 172.21.15.36:0/3540787204 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:43.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27d1752700 1 --2- 172.21.15.36:0/3540787204 >> v2:172.21.15.70:3300/0 conn(0x7f27cc0b64f0 0x7f27cc143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:43.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27d0f51700 1 -- 172.21.15.36:0/3540787204 >> v2:172.21.15.70:3300/0 conn(0x7f27cc0b64f0 msgr2=0x7f27cc143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:43.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27d0f51700 1 --2- 172.21.15.36:0/3540787204 >> v2:172.21.15.70:3300/0 conn(0x7f27cc0b64f0 0x7f27cc143150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27d0f51700 1 -- 172.21.15.36:0/3540787204 >> v1:172.21.15.36:6789/0 conn(0x7f27cc147270 legacy=0x7f27cc140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:43.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27d0f51700 1 -- 172.21.15.36:0/3540787204 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc1468e0 con 0x7f27cc0b4120 2022-06-17T12:54:43.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27d0f51700 1 --2- 172.21.15.36:0/3540787204 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f27cc0b4120 0x7f27cc146d80 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7f27bc002f70 tx=0x7f27bc006df0).ready entity=mon.0 client_cookie=698a1a83ad7ea903 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:43.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27bbfff700 1 -- 172.21.15.36:0/3540787204 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27bc00d5c0 con 0x7f27cc0b4120 2022-06-17T12:54:43.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27bbfff700 1 -- 172.21.15.36:0/3540787204 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27bc000e90 con 0x7f27cc0b4120 2022-06-17T12:54:43.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27bbfff700 1 -- 172.21.15.36:0/3540787204 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27bc00d910 con 0x7f27cc0b4120 2022-06-17T12:54:43.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27d39b6700 1 -- 172.21.15.36:0/3540787204 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f27cc0b4120 msgr2=0x7f27cc146d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:43.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27d39b6700 1 --2- 172.21.15.36:0/3540787204 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f27cc0b4120 0x7f27cc146d80 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7f27bc002f70 tx=0x7f27bc006df0).stop 2022-06-17T12:54:43.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27d39b6700 1 -- 172.21.15.36:0/3540787204 shutdown_connections 2022-06-17T12:54:43.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27d39b6700 1 --2- 172.21.15.36:0/3540787204 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f27cc0b4120 0x7f27cc146d80 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27d39b6700 1 --2- 172.21.15.36:0/3540787204 >> v2:172.21.15.70:3300/0 conn(0x7f27cc0b64f0 0x7f27cc143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27d39b6700 1 -- 172.21.15.36:0/3540787204 >> 172.21.15.36:0/3540787204 conn(0x7f27cc0620f0 msgr2=0x7f27cc062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:43.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27d39b6700 1 -- 172.21.15.36:0/3540787204 shutdown_connections 2022-06-17T12:54:43.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.208+0000 7f27d39b6700 1 -- 172.21.15.36:0/3540787204 wait complete. 2022-06-17T12:54:43.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.212+0000 7f27d39b6700 1 Processor -- start 2022-06-17T12:54:43.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.212+0000 7f27d39b6700 1 -- start start 2022-06-17T12:54:43.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.212+0000 7f27d39b6700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f27cc0b4120 0x7f27cc14a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:43.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.212+0000 7f27d39b6700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f27cc0b64f0 0x7f27cc155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:43.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.212+0000 7f27d39b6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27cc147270 0x7f27cc155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:43.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.212+0000 7f27d39b6700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc0afca0 con 0x7f27cc0b4120 2022-06-17T12:54:43.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.212+0000 7f27d39b6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc0aff40 con 0x7f27cc147270 2022-06-17T12:54:43.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.212+0000 7f27d39b6700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc0b01e0 con 0x7f27cc0b64f0 2022-06-17T12:54:43.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.212+0000 7f27d1752700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f27cc0b4120 0x7f27cc14a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:43.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.212+0000 7f27d1752700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f27cc0b4120 0x7f27cc14a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37244/0 (socket says 172.21.15.36:37244) 2022-06-17T12:54:43.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.212+0000 7f27d1752700 1 -- 172.21.15.36:0/3942038845 learned_addr learned my addr 172.21.15.36:0/3942038845 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:43.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.212+0000 7f27d0f51700 1 --2- 172.21.15.36:0/3942038845 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f27cc0b64f0 0x7f27cc155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:43.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.212+0000 7f27d1f53700 1 --2- 172.21.15.36:0/3942038845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27cc147270 0x7f27cc155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:43.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.212+0000 7f27d1752700 1 -- 172.21.15.36:0/3942038845 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f27cc0b64f0 msgr2=0x7f27cc155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:43.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.212+0000 7f27d1752700 1 --2- 172.21.15.36:0/3942038845 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f27cc0b64f0 0x7f27cc155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.212+0000 7f27d1752700 1 -- 172.21.15.36:0/3942038845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27cc147270 msgr2=0x7f27cc155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:43.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.212+0000 7f27d1752700 1 --2- 172.21.15.36:0/3942038845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27cc147270 0x7f27cc155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.212+0000 7f27d1752700 1 -- 172.21.15.36:0/3942038845 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27bc00b180 con 0x7f27cc0b4120 2022-06-17T12:54:43.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.212+0000 7f27d1752700 1 --2- 172.21.15.36:0/3942038845 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f27cc0b4120 0x7f27cc14a570 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7f27c80030b0 tx=0x7f27c8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:43.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.216+0000 7f27ba7fc700 1 -- 172.21.15.36:0/3942038845 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27c8011780 con 0x7f27cc0b4120 2022-06-17T12:54:43.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.216+0000 7f27ba7fc700 1 -- 172.21.15.36:0/3942038845 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c80118e0 con 0x7f27cc0b4120 2022-06-17T12:54:43.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.216+0000 7f27d39b6700 1 -- 172.21.15.36:0/3942038845 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27cc152370 con 0x7f27cc0b4120 2022-06-17T12:54:43.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.216+0000 7f27ba7fc700 1 -- 172.21.15.36:0/3942038845 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27c8011c40 con 0x7f27cc0b4120 2022-06-17T12:54:43.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.216+0000 7f27d39b6700 1 -- 172.21.15.36:0/3942038845 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27cc142af0 con 0x7f27cc0b4120 2022-06-17T12:54:43.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.216+0000 7f27d39b6700 1 -- 172.21.15.36:0/3942038845 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27cc143640 con 0x7f27cc0b4120 2022-06-17T12:54:43.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.216+0000 7f27ba7fc700 1 -- 172.21.15.36:0/3942038845 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f27c8012420 con 0x7f27cc0b4120 2022-06-17T12:54:43.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.216+0000 7f27ba7fc700 1 --2- 172.21.15.36:0/3942038845 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f27ac053a40 0x7f27ac055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:43.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.216+0000 7f27ba7fc700 1 -- 172.21.15.36:0/3942038845 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f27c806bd80 con 0x7f27cc0b4120 2022-06-17T12:54:43.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.216+0000 7f27d0f51700 1 --2- 172.21.15.36:0/3942038845 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f27ac053a40 0x7f27ac055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:43.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.216+0000 7f27d0f51700 1 --2- 172.21.15.36:0/3942038845 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f27ac053a40 0x7f27ac055f00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f27bc01a000 tx=0x7f27bc014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:43.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.224+0000 7f27ba7fc700 1 -- 172.21.15.36:0/3942038845 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f27c8042b20 con 0x7f27cc0b4120 2022-06-17T12:54:43.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.348+0000 7f27d39b6700 1 -- 172.21.15.36:0/3942038845 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f27cc057410 con 0x7f27cc0b4120 2022-06-17T12:54:43.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.348+0000 7f27ba7fc700 1 -- 172.21.15.36:0/3942038845 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v94) v1 ==== 110+0+10 (secure 0 0 0) 0x7f27c806d070 con 0x7f27cc0b4120 2022-06-17T12:54:43.337 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:54:43.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.352+0000 7f27d39b6700 1 -- 172.21.15.36:0/3942038845 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f27ac053a40 msgr2=0x7f27ac055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:43.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.352+0000 7f27d39b6700 1 --2- 172.21.15.36:0/3942038845 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f27ac053a40 0x7f27ac055f00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f27bc01a000 tx=0x7f27bc014000).stop 2022-06-17T12:54:43.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.352+0000 7f27d39b6700 1 -- 172.21.15.36:0/3942038845 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f27cc0b4120 msgr2=0x7f27cc14a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:43.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.352+0000 7f27d39b6700 1 --2- 172.21.15.36:0/3942038845 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f27cc0b4120 0x7f27cc14a570 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7f27c80030b0 tx=0x7f27c8008040).stop 2022-06-17T12:54:43.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.352+0000 7f27d39b6700 1 -- 172.21.15.36:0/3942038845 shutdown_connections 2022-06-17T12:54:43.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.352+0000 7f27d39b6700 1 --2- 172.21.15.36:0/3942038845 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f27ac053a40 0x7f27ac055f00 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.352+0000 7f27d39b6700 1 --2- 172.21.15.36:0/3942038845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27cc147270 0x7f27cc155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.352+0000 7f27d39b6700 1 --2- 172.21.15.36:0/3942038845 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f27cc0b64f0 0x7f27cc155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.352+0000 7f27d39b6700 1 --2- 172.21.15.36:0/3942038845 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f27cc0b4120 0x7f27cc14a570 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.352+0000 7f27d39b6700 1 -- 172.21.15.36:0/3942038845 >> 172.21.15.36:0/3942038845 conn(0x7f27cc0620f0 msgr2=0x7f27cc0b9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:43.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.352+0000 7f27d39b6700 1 -- 172.21.15.36:0/3942038845 shutdown_connections 2022-06-17T12:54:43.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.352+0000 7f27d39b6700 1 -- 172.21.15.36:0/3942038845 wait complete. 2022-06-17T12:54:43.363 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:54:43.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.488+0000 7ff6ef0a0700 1 Processor -- start 2022-06-17T12:54:43.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.488+0000 7ff6ef0a0700 1 -- start start 2022-06-17T12:54:43.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.488+0000 7ff6ef0a0700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff6e80601c0 0x7ff6e81443d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:43.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.488+0000 7ff6ef0a0700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6e80ae360 0x7ff6e81448c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:43.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.488+0000 7ff6ef0a0700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e80b2b60 con 0x7ff6e80b1db0 2022-06-17T12:54:43.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.488+0000 7ff6ef0a0700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e80b3230 con 0x7ff6e80601c0 2022-06-17T12:54:43.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.488+0000 7ff6ef0a0700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e80605a0 con 0x7ff6e80ae360 2022-06-17T12:54:43.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.488+0000 7ff6ed89d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6e80ae360 0x7ff6e81448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:43.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.488+0000 7ff6ee09e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff6e80601c0 0x7ff6e81443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:43.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.488+0000 7ff6ee89f700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff6e80b1db0 0x7ff6e81499f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:50518/0 (socket says 172.21.15.36:50518) 2022-06-17T12:54:43.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.488+0000 7ff6ed89d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6e80ae360 0x7ff6e81448c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:52044/0 (socket says 172.21.15.36:52044) 2022-06-17T12:54:43.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.488+0000 7ff6ee09e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff6e80601c0 0x7ff6e81443d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:57634/0 (socket says 172.21.15.36:57634) 2022-06-17T12:54:43.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.488+0000 7ff6ed89d700 1 -- 172.21.15.36:0/3209205979 learned_addr learned my addr 172.21.15.36:0/3209205979 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:43.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.488+0000 7ff6ed89d700 1 -- 172.21.15.36:0/3209205979 >> v2:172.21.15.70:3300/0 conn(0x7ff6e80601c0 msgr2=0x7ff6e81443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:43.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.488+0000 7ff6ed89d700 1 --2- 172.21.15.36:0/3209205979 >> v2:172.21.15.70:3300/0 conn(0x7ff6e80601c0 0x7ff6e81443d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.488+0000 7ff6ed89d700 1 -- 172.21.15.36:0/3209205979 >> v1:172.21.15.70:6789/0 conn(0x7ff6e80b1db0 legacy=0x7ff6e81499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:43.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.492+0000 7ff6ed89d700 1 -- 172.21.15.36:0/3209205979 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e80662f0 con 0x7ff6e80ae360 2022-06-17T12:54:43.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.492+0000 7ff6ed89d700 1 --2- 172.21.15.36:0/3209205979 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6e80ae360 0x7ff6e81448c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7ff6d8002f70 tx=0x7ff6d8006eb0).ready entity=mon.2 client_cookie=e902656e149c5cf1 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:43.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.492+0000 7ff6ed09c700 1 -- 172.21.15.36:0/3209205979 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6d800d570 con 0x7ff6e80ae360 2022-06-17T12:54:43.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.492+0000 7ff6ed09c700 1 -- 172.21.15.36:0/3209205979 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d8000e90 con 0x7ff6e80ae360 2022-06-17T12:54:43.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.492+0000 7ff6ed09c700 1 -- 172.21.15.36:0/3209205979 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6d800d900 con 0x7ff6e80ae360 2022-06-17T12:54:43.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.492+0000 7ff6ef0a0700 1 -- 172.21.15.36:0/3209205979 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6e80ae360 msgr2=0x7ff6e81448c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:43.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.492+0000 7ff6ef0a0700 1 --2- 172.21.15.36:0/3209205979 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6e80ae360 0x7ff6e81448c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7ff6d8002f70 tx=0x7ff6d8006eb0).stop 2022-06-17T12:54:43.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.492+0000 7ff6ef0a0700 1 -- 172.21.15.36:0/3209205979 shutdown_connections 2022-06-17T12:54:43.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.492+0000 7ff6ef0a0700 1 --2- 172.21.15.36:0/3209205979 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6e80ae360 0x7ff6e81448c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.492+0000 7ff6ef0a0700 1 --2- 172.21.15.36:0/3209205979 >> v2:172.21.15.70:3300/0 conn(0x7ff6e80601c0 0x7ff6e81443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.492+0000 7ff6ef0a0700 1 -- 172.21.15.36:0/3209205979 >> 172.21.15.36:0/3209205979 conn(0x7ff6e8060990 msgr2=0x7ff6e805f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:43.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.492+0000 7ff6ef0a0700 1 -- 172.21.15.36:0/3209205979 shutdown_connections 2022-06-17T12:54:43.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.492+0000 7ff6ef0a0700 1 -- 172.21.15.36:0/3209205979 wait complete. 2022-06-17T12:54:43.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6ef0a0700 1 Processor -- start 2022-06-17T12:54:43.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6ef0a0700 1 -- start start 2022-06-17T12:54:43.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6ef0a0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff6e80601c0 0x7ff6e8143400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:43.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6ef0a0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6e80ae360 0x7ff6e8155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:43.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6ef0a0700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6e80b1db0 0x7ff6e8156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:43.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6ef0a0700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e8155980 con 0x7ff6e80601c0 2022-06-17T12:54:43.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6ef0a0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e80b0eb0 con 0x7ff6e80ae360 2022-06-17T12:54:43.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6ef0a0700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e80b1150 con 0x7ff6e80b1db0 2022-06-17T12:54:43.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6ee09e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff6e80601c0 0x7ff6e8143400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:43.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6ee09e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff6e80601c0 0x7ff6e8143400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37258/0 (socket says 172.21.15.36:37258) 2022-06-17T12:54:43.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6ee09e700 1 -- 172.21.15.36:0/1043045856 learned_addr learned my addr 172.21.15.36:0/1043045856 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:43.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6ed89d700 1 --2- 172.21.15.36:0/1043045856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6e80ae360 0x7ff6e8155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:43.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6ee89f700 1 --2- 172.21.15.36:0/1043045856 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6e80b1db0 0x7ff6e8156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:43.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6ee09e700 1 -- 172.21.15.36:0/1043045856 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6e80b1db0 msgr2=0x7ff6e8156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:43.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6ee09e700 1 --2- 172.21.15.36:0/1043045856 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6e80b1db0 0x7ff6e8156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6ee09e700 1 -- 172.21.15.36:0/1043045856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6e80ae360 msgr2=0x7ff6e8155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:43.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6ee09e700 1 --2- 172.21.15.36:0/1043045856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6e80ae360 0x7ff6e8155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6ee09e700 1 -- 172.21.15.36:0/1043045856 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d800b1e0 con 0x7ff6e80601c0 2022-06-17T12:54:43.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6ed89d700 1 --2- 172.21.15.36:0/1043045856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6e80ae360 0x7ff6e8155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:43.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6ee09e700 1 --2- 172.21.15.36:0/1043045856 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff6e80601c0 0x7ff6e8143400 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0x7ff6e400c000 tx=0x7ff6e4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:43.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6d77fe700 1 -- 172.21.15.36:0/1043045856 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6e4010780 con 0x7ff6e80601c0 2022-06-17T12:54:43.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6d77fe700 1 -- 172.21.15.36:0/1043045856 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e40108e0 con 0x7ff6e80601c0 2022-06-17T12:54:43.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6ef0a0700 1 -- 172.21.15.36:0/1043045856 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e80b13f0 con 0x7ff6e80601c0 2022-06-17T12:54:43.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6d77fe700 1 -- 172.21.15.36:0/1043045856 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6e4010c40 con 0x7ff6e80601c0 2022-06-17T12:54:43.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.496+0000 7ff6ef0a0700 1 -- 172.21.15.36:0/1043045856 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e8144090 con 0x7ff6e80601c0 2022-06-17T12:54:43.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.500+0000 7ff6ef0a0700 1 -- 172.21.15.36:0/1043045856 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e8066590 con 0x7ff6e80601c0 2022-06-17T12:54:43.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.500+0000 7ff6d77fe700 1 -- 172.21.15.36:0/1043045856 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff6e401c460 con 0x7ff6e80601c0 2022-06-17T12:54:43.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.500+0000 7ff6d77fe700 1 --2- 172.21.15.36:0/1043045856 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff6c8053a40 0x7ff6c8055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:43.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.500+0000 7ff6d77fe700 1 -- 172.21.15.36:0/1043045856 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff6e406c520 con 0x7ff6e80601c0 2022-06-17T12:54:43.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.500+0000 7ff6ed89d700 1 --2- 172.21.15.36:0/1043045856 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff6c8053a40 0x7ff6c8055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:43.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.500+0000 7ff6ed89d700 1 --2- 172.21.15.36:0/1043045856 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff6c8053a40 0x7ff6c8055f00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7ff6d801a000 tx=0x7ff6d8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:43.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.504+0000 7ff6d77fe700 1 -- 172.21.15.36:0/1043045856 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff6e40432c0 con 0x7ff6e80601c0 2022-06-17T12:54:43.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.632+0000 7ff6ef0a0700 1 -- 172.21.15.36:0/1043045856 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff6e8057410 con 0x7ff6e80601c0 2022-06-17T12:54:43.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.632+0000 7ff6d77fe700 1 -- 172.21.15.36:0/1043045856 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v94) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff6e40468e0 con 0x7ff6e80601c0 2022-06-17T12:54:43.620 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:54:43.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.636+0000 7ff6ef0a0700 1 -- 172.21.15.36:0/1043045856 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff6c8053a40 msgr2=0x7ff6c8055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:43.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.636+0000 7ff6ef0a0700 1 --2- 172.21.15.36:0/1043045856 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff6c8053a40 0x7ff6c8055f00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7ff6d801a000 tx=0x7ff6d8015000).stop 2022-06-17T12:54:43.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.636+0000 7ff6ef0a0700 1 -- 172.21.15.36:0/1043045856 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff6e80601c0 msgr2=0x7ff6e8143400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:43.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.636+0000 7ff6ef0a0700 1 --2- 172.21.15.36:0/1043045856 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff6e80601c0 0x7ff6e8143400 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0x7ff6e400c000 tx=0x7ff6e4006df0).stop 2022-06-17T12:54:43.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.636+0000 7ff6ef0a0700 1 -- 172.21.15.36:0/1043045856 shutdown_connections 2022-06-17T12:54:43.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.636+0000 7ff6ef0a0700 1 --2- 172.21.15.36:0/1043045856 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff6c8053a40 0x7ff6c8055f00 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.636+0000 7ff6ef0a0700 1 --2- 172.21.15.36:0/1043045856 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6e80b1db0 0x7ff6e8156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.636+0000 7ff6ef0a0700 1 --2- 172.21.15.36:0/1043045856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6e80ae360 0x7ff6e8155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.636+0000 7ff6ef0a0700 1 --2- 172.21.15.36:0/1043045856 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff6e80601c0 0x7ff6e8143400 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.636+0000 7ff6ef0a0700 1 -- 172.21.15.36:0/1043045856 >> 172.21.15.36:0/1043045856 conn(0x7ff6e8060990 msgr2=0x7ff6e8062fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:43.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.636+0000 7ff6ef0a0700 1 -- 172.21.15.36:0/1043045856 shutdown_connections 2022-06-17T12:54:43.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.636+0000 7ff6ef0a0700 1 -- 172.21.15.36:0/1043045856 wait complete. 2022-06-17T12:54:43.637 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:54:43.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.764+0000 7f1d0a397700 1 Processor -- start 2022-06-17T12:54:43.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.764+0000 7f1d0a397700 1 -- start start 2022-06-17T12:54:43.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1d0a397700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1d04060ac0 0x7f1d040630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:43.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1d0a397700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f1d040ae6c0 0x7f1d0413fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:43.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1d0a397700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d04062c80 con 0x7f1d040b1680 2022-06-17T12:54:43.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1d0a397700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d04060ea0 con 0x7f1d040ae6c0 2022-06-17T12:54:43.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1d0a397700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d040b2f50 con 0x7f1d04060ac0 2022-06-17T12:54:43.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1d09b96700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f1d040ae6c0 0x7f1d0413fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:43.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1d09b96700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f1d040ae6c0 0x7f1d0413fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37270/0 (socket says 172.21.15.36:37270) 2022-06-17T12:54:43.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1d09b96700 1 -- 172.21.15.36:0/585351303 learned_addr learned my addr 172.21.15.36:0/585351303 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:43.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1d09395700 1 --2- 172.21.15.36:0/585351303 >> v2:172.21.15.70:3300/0 conn(0x7f1d04060ac0 0x7f1d040630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:43.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1d09395700 1 -- 172.21.15.36:0/585351303 >> v2:172.21.15.36:3300/0 conn(0x7f1d040ae6c0 msgr2=0x7f1d0413fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:43.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1d09395700 1 --2- 172.21.15.36:0/585351303 >> v2:172.21.15.36:3300/0 conn(0x7f1d040ae6c0 0x7f1d0413fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1d09395700 1 -- 172.21.15.36:0/585351303 >> v1:172.21.15.70:6789/0 conn(0x7f1d040b1680 legacy=0x7f1d041431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:43.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1d09395700 1 -- 172.21.15.36:0/585351303 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d040635d0 con 0x7f1d04060ac0 2022-06-17T12:54:43.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1d09395700 1 --2- 172.21.15.36:0/585351303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d04060ac0 0x7f1d040630e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f1d00002f70 tx=0x7f1d00006df0).ready entity=mon.1 client_cookie=4d006f72fb2965d3 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:43.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1cf3fff700 1 -- 172.21.15.36:0/585351303 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d0000d570 con 0x7f1d04060ac0 2022-06-17T12:54:43.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1cf3fff700 1 -- 172.21.15.36:0/585351303 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d00000e90 con 0x7f1d04060ac0 2022-06-17T12:54:43.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1cf3fff700 1 -- 172.21.15.36:0/585351303 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d0000d8c0 con 0x7f1d04060ac0 2022-06-17T12:54:43.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1d0a397700 1 -- 172.21.15.36:0/585351303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d04060ac0 msgr2=0x7f1d040630e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:43.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1d0a397700 1 --2- 172.21.15.36:0/585351303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d04060ac0 0x7f1d040630e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f1d00002f70 tx=0x7f1d00006df0).stop 2022-06-17T12:54:43.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1d0a397700 1 -- 172.21.15.36:0/585351303 shutdown_connections 2022-06-17T12:54:43.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1d0a397700 1 --2- 172.21.15.36:0/585351303 >> v2:172.21.15.36:3300/0 conn(0x7f1d040ae6c0 0x7f1d0413fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1d0a397700 1 --2- 172.21.15.36:0/585351303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d04060ac0 0x7f1d040630e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1d0a397700 1 -- 172.21.15.36:0/585351303 >> 172.21.15.36:0/585351303 conn(0x7f1d040b64f0 msgr2=0x7f1d040b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:43.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1d0a397700 1 -- 172.21.15.36:0/585351303 shutdown_connections 2022-06-17T12:54:43.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.768+0000 7f1d0a397700 1 -- 172.21.15.36:0/585351303 wait complete. 2022-06-17T12:54:43.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.772+0000 7f1d0a397700 1 Processor -- start 2022-06-17T12:54:43.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.772+0000 7f1d0a397700 1 -- start start 2022-06-17T12:54:43.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.772+0000 7f1d0a397700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d04060ac0 0x7f1d040b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:43.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.776+0000 7f1d0a397700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d040ae6c0 0x7f1d0413e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:43.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.776+0000 7f1d0a397700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d040b1680 0x7f1d040a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:43.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.776+0000 7f1d0a397700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d041463d0 con 0x7f1d040b1680 2022-06-17T12:54:43.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.776+0000 7f1d0a397700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d04146600 con 0x7f1d04060ac0 2022-06-17T12:54:43.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.776+0000 7f1d0a397700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0409e8d0 con 0x7f1d040ae6c0 2022-06-17T12:54:43.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.776+0000 7f1d09b96700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d040b1680 0x7f1d040a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:43.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.776+0000 7f1d09b96700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d040b1680 0x7f1d040a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37276/0 (socket says 172.21.15.36:37276) 2022-06-17T12:54:43.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.776+0000 7f1d09b96700 1 -- 172.21.15.36:0/2458852023 learned_addr learned my addr 172.21.15.36:0/2458852023 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:43.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.776+0000 7f1d09395700 1 --2- 172.21.15.36:0/2458852023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d04060ac0 0x7f1d040b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:43.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.776+0000 7f1d08b94700 1 --2- 172.21.15.36:0/2458852023 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d040ae6c0 0x7f1d0413e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:43.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.776+0000 7f1d09b96700 1 -- 172.21.15.36:0/2458852023 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d040ae6c0 msgr2=0x7f1d0413e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:43.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.776+0000 7f1d09b96700 1 --2- 172.21.15.36:0/2458852023 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d040ae6c0 0x7f1d0413e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.776+0000 7f1d09b96700 1 -- 172.21.15.36:0/2458852023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d04060ac0 msgr2=0x7f1d040b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:43.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.776+0000 7f1d09b96700 1 --2- 172.21.15.36:0/2458852023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d04060ac0 0x7f1d040b0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.776+0000 7f1d09b96700 1 -- 172.21.15.36:0/2458852023 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0000b180 con 0x7f1d040b1680 2022-06-17T12:54:43.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.776+0000 7f1d08b94700 1 --2- 172.21.15.36:0/2458852023 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d040ae6c0 0x7f1d0413e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:43.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.776+0000 7f1d09b96700 1 --2- 172.21.15.36:0/2458852023 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d040b1680 0x7f1d040a11b0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f1cfc00c000 tx=0x7f1cfc009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:43.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.776+0000 7f1cf27fc700 1 -- 172.21.15.36:0/2458852023 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cfc00c850 con 0x7f1d040b1680 2022-06-17T12:54:43.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.776+0000 7f1cf27fc700 1 -- 172.21.15.36:0/2458852023 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cfc00c9b0 con 0x7f1d040b1680 2022-06-17T12:54:43.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.776+0000 7f1d0a397700 1 -- 172.21.15.36:0/2458852023 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d041471a0 con 0x7f1d040b1680 2022-06-17T12:54:43.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.776+0000 7f1cf27fc700 1 -- 172.21.15.36:0/2458852023 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cfc00cd10 con 0x7f1d040b1680 2022-06-17T12:54:43.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.776+0000 7f1d0a397700 1 -- 172.21.15.36:0/2458852023 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d04146d40 con 0x7f1d040b1680 2022-06-17T12:54:43.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.776+0000 7f1d0a397700 1 -- 172.21.15.36:0/2458852023 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d040b2cb0 con 0x7f1d040b1680 2022-06-17T12:54:43.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.776+0000 7f1cf27fc700 1 -- 172.21.15.36:0/2458852023 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f1cfc022070 con 0x7f1d040b1680 2022-06-17T12:54:43.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.780+0000 7f1cf27fc700 1 --2- 172.21.15.36:0/2458852023 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1ce4053ad0 0x7f1ce4055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:43.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.780+0000 7f1cf27fc700 1 -- 172.21.15.36:0/2458852023 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1cfc06c6b0 con 0x7f1d040b1680 2022-06-17T12:54:43.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.780+0000 7f1d09395700 1 --2- 172.21.15.36:0/2458852023 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1ce4053ad0 0x7f1ce4055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:43.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.780+0000 7f1d09395700 1 --2- 172.21.15.36:0/2458852023 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1ce4053ad0 0x7f1ce4055f90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f1d0001a000 tx=0x7f1d00014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:43.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.780+0000 7f1cf27fc700 1 -- 172.21.15.36:0/2458852023 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1cfc0434b0 con 0x7f1d040b1680 2022-06-17T12:54:43.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.904+0000 7f1d0a397700 1 -- 172.21.15.36:0/2458852023 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1d04057410 con 0x7f1d040b1680 2022-06-17T12:54:43.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.908+0000 7f1cf27fc700 1 -- 172.21.15.36:0/2458852023 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1cfc016020 con 0x7f1d040b1680 2022-06-17T12:54:43.896 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:54:43.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.908+0000 7f1d0a397700 1 -- 172.21.15.36:0/2458852023 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1ce4053ad0 msgr2=0x7f1ce4055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:43.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.908+0000 7f1d0a397700 1 --2- 172.21.15.36:0/2458852023 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1ce4053ad0 0x7f1ce4055f90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f1d0001a000 tx=0x7f1d00014000).stop 2022-06-17T12:54:43.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.908+0000 7f1d0a397700 1 -- 172.21.15.36:0/2458852023 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d040b1680 msgr2=0x7f1d040a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:43.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.908+0000 7f1d0a397700 1 --2- 172.21.15.36:0/2458852023 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d040b1680 0x7f1d040a11b0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f1cfc00c000 tx=0x7f1cfc009290).stop 2022-06-17T12:54:43.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.908+0000 7f1d0a397700 1 -- 172.21.15.36:0/2458852023 shutdown_connections 2022-06-17T12:54:43.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.908+0000 7f1d0a397700 1 --2- 172.21.15.36:0/2458852023 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1ce4053ad0 0x7f1ce4055f90 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.908+0000 7f1d0a397700 1 --2- 172.21.15.36:0/2458852023 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d040b1680 0x7f1d040a11b0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.908+0000 7f1d0a397700 1 --2- 172.21.15.36:0/2458852023 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d040ae6c0 0x7f1d0413e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.908+0000 7f1d0a397700 1 --2- 172.21.15.36:0/2458852023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d04060ac0 0x7f1d040b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:43.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.908+0000 7f1d0a397700 1 -- 172.21.15.36:0/2458852023 >> 172.21.15.36:0/2458852023 conn(0x7f1d040b64f0 msgr2=0x7f1d041430b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:43.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.908+0000 7f1d0a397700 1 -- 172.21.15.36:0/2458852023 shutdown_connections 2022-06-17T12:54:43.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:43.908+0000 7f1d0a397700 1 -- 172.21.15.36:0/2458852023 wait complete. 2022-06-17T12:54:43.907 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:54:44.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.036+0000 7f7ec321e700 1 Processor -- start 2022-06-17T12:54:44.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.036+0000 7f7ec321e700 1 -- start start 2022-06-17T12:54:44.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.036+0000 7f7ec321e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7ebc062b40 0x7f7ebc143080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:44.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.036+0000 7f7ec321e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ebc0b2450 con 0x7f7ebc0ae150 2022-06-17T12:54:44.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.036+0000 7f7ec321e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ebc0b26f0 con 0x7f7ebc0b7930 2022-06-17T12:54:44.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.036+0000 7f7ec321e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ebc061970 con 0x7f7ebc062b40 2022-06-17T12:54:44.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.040+0000 7f7eb3fff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f7ebc0ae150 0x7f7ebc062430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54810/0 (socket says 172.21.15.36:54810) 2022-06-17T12:54:44.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.040+0000 7f7ec17bb700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7ebc062b40 0x7f7ebc143080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:44.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.040+0000 7f7eb3fff700 1 -- 172.21.15.36:0/283024309 learned_addr learned my addr 172.21.15.36:0/283024309 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:44.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.040+0000 7f7ec17bb700 1 -- 172.21.15.36:0/283024309 >> v1:172.21.15.70:6789/0 conn(0x7f7ebc0b7930 legacy=0x7f7ebc13ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:44.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.040+0000 7f7ec17bb700 1 -- 172.21.15.36:0/283024309 >> v1:172.21.15.36:6789/0 conn(0x7f7ebc0ae150 legacy=0x7f7ebc062430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:44.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.040+0000 7f7ec17bb700 1 -- 172.21.15.36:0/283024309 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ebc0b2990 con 0x7f7ebc062b40 2022-06-17T12:54:44.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.040+0000 7f7eb37fe700 1 -- 172.21.15.36:0/283024309 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 750014289 0 0) 0x7f7ebc0b2450 con 0x7f7ebc0ae150 2022-06-17T12:54:44.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.040+0000 7f7ec17bb700 1 --2- 172.21.15.36:0/283024309 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ebc062b40 0x7f7ebc143080 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0x7f7eb40032f0 tx=0x7f7eb40091d0).ready entity=mon.0 client_cookie=5a6b3bde02a34b84 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:44.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.040+0000 7f7eb37fe700 1 -- 172.21.15.36:0/283024309 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7eb4009ea0 con 0x7f7ebc062b40 2022-06-17T12:54:44.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.040+0000 7f7eb37fe700 1 -- 172.21.15.36:0/283024309 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb400b8a0 con 0x7f7ebc062b40 2022-06-17T12:54:44.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.040+0000 7f7eb37fe700 1 -- 172.21.15.36:0/283024309 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7eb400ba10 con 0x7f7ebc062b40 2022-06-17T12:54:44.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.040+0000 7f7ec321e700 1 -- 172.21.15.36:0/283024309 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ebc062b40 msgr2=0x7f7ebc143080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:44.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.040+0000 7f7ec321e700 1 --2- 172.21.15.36:0/283024309 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ebc062b40 0x7f7ebc143080 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0x7f7eb40032f0 tx=0x7f7eb40091d0).stop 2022-06-17T12:54:44.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.040+0000 7f7ec321e700 1 -- 172.21.15.36:0/283024309 shutdown_connections 2022-06-17T12:54:44.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.040+0000 7f7ec321e700 1 --2- 172.21.15.36:0/283024309 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ebc062b40 0x7f7ebc143080 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:44.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.040+0000 7f7ec321e700 1 -- 172.21.15.36:0/283024309 >> 172.21.15.36:0/283024309 conn(0x7f7ebc060990 msgr2=0x7f7ebc05c550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:44.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.040+0000 7f7ec321e700 1 -- 172.21.15.36:0/283024309 shutdown_connections 2022-06-17T12:54:44.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.040+0000 7f7ec321e700 1 -- 172.21.15.36:0/283024309 wait complete. 2022-06-17T12:54:44.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.040+0000 7f7ec321e700 1 Processor -- start 2022-06-17T12:54:44.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.040+0000 7f7ec321e700 1 -- start start 2022-06-17T12:54:44.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.044+0000 7f7ec321e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ebc062b40 0x7f7ebc146540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:44.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.044+0000 7f7ec321e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ebc0ae150 0x7f7ebc065fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:44.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.044+0000 7f7ec321e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ebc0b7930 0x7f7ebc146ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:44.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.044+0000 7f7ec321e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc0af080 con 0x7f7ebc0b7930 2022-06-17T12:54:44.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.044+0000 7f7ec321e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc0b0240 con 0x7f7ebc0ae150 2022-06-17T12:54:44.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.044+0000 7f7ec321e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc0664d0 con 0x7f7ebc062b40 2022-06-17T12:54:44.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.044+0000 7f7ec17bb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ebc0b7930 0x7f7ebc146ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:44.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.044+0000 7f7ec17bb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ebc0b7930 0x7f7ebc146ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37290/0 (socket says 172.21.15.36:37290) 2022-06-17T12:54:44.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.044+0000 7f7ec17bb700 1 -- 172.21.15.36:0/1888310501 learned_addr learned my addr 172.21.15.36:0/1888310501 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:44.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.044+0000 7f7eb3fff700 1 --2- 172.21.15.36:0/1888310501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ebc0ae150 0x7f7ebc065fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:44.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.044+0000 7f7ec0fba700 1 --2- 172.21.15.36:0/1888310501 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ebc062b40 0x7f7ebc146540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:44.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.044+0000 7f7ec17bb700 1 -- 172.21.15.36:0/1888310501 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ebc062b40 msgr2=0x7f7ebc146540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:44.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.044+0000 7f7ec17bb700 1 --2- 172.21.15.36:0/1888310501 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ebc062b40 0x7f7ebc146540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:44.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.044+0000 7f7ec17bb700 1 -- 172.21.15.36:0/1888310501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ebc0ae150 msgr2=0x7f7ebc065fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:44.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.044+0000 7f7ec17bb700 1 --2- 172.21.15.36:0/1888310501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ebc0ae150 0x7f7ebc065fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:44.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.044+0000 7f7ec17bb700 1 -- 172.21.15.36:0/1888310501 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb4009470 con 0x7f7ebc0b7930 2022-06-17T12:54:44.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.044+0000 7f7ec17bb700 1 --2- 172.21.15.36:0/1888310501 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ebc0b7930 0x7f7ebc146ab0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f7eb40032f0 tx=0x7f7eb4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:44.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.044+0000 7f7eb1ffb700 1 -- 172.21.15.36:0/1888310501 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7eb4009ea0 con 0x7f7ebc0b7930 2022-06-17T12:54:44.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.044+0000 7f7eb1ffb700 1 -- 172.21.15.36:0/1888310501 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb401f5e0 con 0x7f7ebc0b7930 2022-06-17T12:54:44.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.044+0000 7f7ec321e700 1 -- 172.21.15.36:0/1888310501 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ebc0af2b0 con 0x7f7ebc0b7930 2022-06-17T12:54:44.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.044+0000 7f7eb1ffb700 1 -- 172.21.15.36:0/1888310501 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7eb401f940 con 0x7f7ebc0b7930 2022-06-17T12:54:44.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.044+0000 7f7ec321e700 1 -- 172.21.15.36:0/1888310501 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ebc0af550 con 0x7f7ebc0b7930 2022-06-17T12:54:44.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.044+0000 7f7ec321e700 1 -- 172.21.15.36:0/1888310501 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ebc0b03b0 con 0x7f7ebc0b7930 2022-06-17T12:54:44.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.048+0000 7f7eb1ffb700 1 -- 172.21.15.36:0/1888310501 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f7eb4026030 con 0x7f7ebc0b7930 2022-06-17T12:54:44.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.048+0000 7f7eb1ffb700 1 --2- 172.21.15.36:0/1888310501 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7e9c053a40 0x7f7e9c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:44.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.048+0000 7f7eb1ffb700 1 -- 172.21.15.36:0/1888310501 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7eb406ce50 con 0x7f7ebc0b7930 2022-06-17T12:54:44.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.048+0000 7f7ec0fba700 1 --2- 172.21.15.36:0/1888310501 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7e9c053a40 0x7f7e9c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:44.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.048+0000 7f7ec0fba700 1 --2- 172.21.15.36:0/1888310501 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7e9c053a40 0x7f7e9c055f00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f7eb800b000 tx=0x7f7eb8009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:44.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.056+0000 7f7eb1ffb700 1 -- 172.21.15.36:0/1888310501 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7eb4043c50 con 0x7f7ebc0b7930 2022-06-17T12:54:44.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.184+0000 7f7ec321e700 1 -- 172.21.15.36:0/1888310501 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7ebc0b6520 con 0x7f7ebc0b7930 2022-06-17T12:54:44.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.184+0000 7f7eb1ffb700 1 -- 172.21.15.36:0/1888310501 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7eb40150a0 con 0x7f7ebc0b7930 2022-06-17T12:54:44.171 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:54:44.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.184+0000 7f7ec321e700 1 -- 172.21.15.36:0/1888310501 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7e9c053a40 msgr2=0x7f7e9c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:44.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.184+0000 7f7ec321e700 1 --2- 172.21.15.36:0/1888310501 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7e9c053a40 0x7f7e9c055f00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f7eb800b000 tx=0x7f7eb8009040).stop 2022-06-17T12:54:44.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.184+0000 7f7ec321e700 1 -- 172.21.15.36:0/1888310501 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ebc0b7930 msgr2=0x7f7ebc146ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:44.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.184+0000 7f7ec321e700 1 --2- 172.21.15.36:0/1888310501 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ebc0b7930 0x7f7ebc146ab0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f7eb40032f0 tx=0x7f7eb4016000).stop 2022-06-17T12:54:44.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.184+0000 7f7ec321e700 1 -- 172.21.15.36:0/1888310501 shutdown_connections 2022-06-17T12:54:44.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.184+0000 7f7ec321e700 1 --2- 172.21.15.36:0/1888310501 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7e9c053a40 0x7f7e9c055f00 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:44.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.184+0000 7f7ec321e700 1 --2- 172.21.15.36:0/1888310501 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ebc0b7930 0x7f7ebc146ab0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:44.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.184+0000 7f7ec321e700 1 --2- 172.21.15.36:0/1888310501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ebc0ae150 0x7f7ebc065fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:44.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.184+0000 7f7ec321e700 1 --2- 172.21.15.36:0/1888310501 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ebc062b40 0x7f7ebc146540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:44.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.184+0000 7f7ec321e700 1 -- 172.21.15.36:0/1888310501 >> 172.21.15.36:0/1888310501 conn(0x7f7ebc060990 msgr2=0x7f7ebc0b6740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:44.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.184+0000 7f7ec321e700 1 -- 172.21.15.36:0/1888310501 shutdown_connections 2022-06-17T12:54:44.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.184+0000 7f7ec321e700 1 -- 172.21.15.36:0/1888310501 wait complete. 2022-06-17T12:54:44.184 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:54:44.185 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:54:44.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.300+0000 7f1f0bd57700 1 Processor -- start 2022-06-17T12:54:44.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.300+0000 7f1f0bd57700 1 -- start start 2022-06-17T12:54:44.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.300+0000 7f1f0bd57700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1f04060ac0 0x7f1f04064840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:44.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.300+0000 7f1f0bd57700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1f04140e70 0x7f1f04141300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:44.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.300+0000 7f1f0bd57700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f04060ea0 con 0x7f1f040b22b0 2022-06-17T12:54:44.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.300+0000 7f1f0bd57700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f041445f0 con 0x7f1f04140e70 2022-06-17T12:54:44.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.300+0000 7f1f0bd57700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f04144890 con 0x7f1f04060ac0 2022-06-17T12:54:44.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.300+0000 7f1f092f2700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f1f040b22b0 0x7f1f04144d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54826/0 (socket says 172.21.15.36:54826) 2022-06-17T12:54:44.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.300+0000 7f1f092f2700 1 -- 172.21.15.36:0/1220388248 learned_addr learned my addr 172.21.15.36:0/1220388248 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:44.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.300+0000 7f1f09af3700 1 --2- 172.21.15.36:0/1220388248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1f04060ac0 0x7f1f04064840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:44.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.300+0000 7f1f0a2f4700 1 --2- 172.21.15.36:0/1220388248 >> v2:172.21.15.70:3300/0 conn(0x7f1f04140e70 0x7f1f04141300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:44.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.300+0000 7f1f08af1700 1 -- 172.21.15.36:0/1220388248 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3211666163 0 0) 0x7f1f04060ea0 con 0x7f1f040b22b0 2022-06-17T12:54:44.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.300+0000 7f1f08af1700 1 -- 172.21.15.36:0/1220388248 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1eec003260 con 0x7f1f040b22b0 2022-06-17T12:54:44.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.300+0000 7f1f0a2f4700 1 -- 172.21.15.36:0/1220388248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1f04060ac0 msgr2=0x7f1f04064840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:44.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.300+0000 7f1f0a2f4700 1 --2- 172.21.15.36:0/1220388248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1f04060ac0 0x7f1f04064840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:44.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.300+0000 7f1f0a2f4700 1 -- 172.21.15.36:0/1220388248 >> v1:172.21.15.36:6789/0 conn(0x7f1f040b22b0 legacy=0x7f1f04144d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:44.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.300+0000 7f1f0a2f4700 1 -- 172.21.15.36:0/1220388248 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f040b0910 con 0x7f1f04140e70 2022-06-17T12:54:44.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.300+0000 7f1f09af3700 1 --2- 172.21.15.36:0/1220388248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1f04060ac0 0x7f1f04064840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:44.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.300+0000 7f1f0a2f4700 1 --2- 172.21.15.36:0/1220388248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f04140e70 0x7f1f04141300 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f1f04063bd0 tx=0x7f1efc0091d0).ready entity=mon.1 client_cookie=b5b83ed0cf69e301 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:44.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.304+0000 7f1f08af1700 1 -- 172.21.15.36:0/1220388248 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1efc00b600 con 0x7f1f04140e70 2022-06-17T12:54:44.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.304+0000 7f1f08af1700 1 -- 172.21.15.36:0/1220388248 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1efc00b760 con 0x7f1f04140e70 2022-06-17T12:54:44.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.304+0000 7f1f08af1700 1 -- 172.21.15.36:0/1220388248 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1efc00b8d0 con 0x7f1f04140e70 2022-06-17T12:54:44.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.304+0000 7f1f0bd57700 1 -- 172.21.15.36:0/1220388248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f04140e70 msgr2=0x7f1f04141300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:44.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.304+0000 7f1f0bd57700 1 --2- 172.21.15.36:0/1220388248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f04140e70 0x7f1f04141300 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f1f04063bd0 tx=0x7f1efc0091d0).stop 2022-06-17T12:54:44.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.304+0000 7f1f0bd57700 1 -- 172.21.15.36:0/1220388248 shutdown_connections 2022-06-17T12:54:44.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.304+0000 7f1f0bd57700 1 --2- 172.21.15.36:0/1220388248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f04140e70 0x7f1f04141300 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:44.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.304+0000 7f1f0bd57700 1 --2- 172.21.15.36:0/1220388248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1f04060ac0 0x7f1f04064840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:44.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.304+0000 7f1f0bd57700 1 -- 172.21.15.36:0/1220388248 >> 172.21.15.36:0/1220388248 conn(0x7f1f040b6870 msgr2=0x7f1f040b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:44.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.304+0000 7f1f0bd57700 1 -- 172.21.15.36:0/1220388248 shutdown_connections 2022-06-17T12:54:44.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.304+0000 7f1f0bd57700 1 -- 172.21.15.36:0/1220388248 wait complete. 2022-06-17T12:54:44.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.304+0000 7f1f0bd57700 1 Processor -- start 2022-06-17T12:54:44.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.304+0000 7f1f0bd57700 1 -- start start 2022-06-17T12:54:44.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.304+0000 7f1f0bd57700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f04060ac0 0x7f1f040a28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:44.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.304+0000 7f1f0bd57700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1f040b22b0 0x7f1f040a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:44.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.304+0000 7f1f0bd57700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1f04140e70 0x7f1f041464e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:44.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.304+0000 7f1f0bd57700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f040a1540 con 0x7f1f040b22b0 2022-06-17T12:54:44.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.304+0000 7f1f0bd57700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0409f5a0 con 0x7f1f04060ac0 2022-06-17T12:54:44.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.304+0000 7f1f0bd57700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f040a0820 con 0x7f1f04140e70 2022-06-17T12:54:44.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.304+0000 7f1f092f2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1f040b22b0 0x7f1f040a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:44.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.308+0000 7f1f092f2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1f040b22b0 0x7f1f040a2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37302/0 (socket says 172.21.15.36:37302) 2022-06-17T12:54:44.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.308+0000 7f1f092f2700 1 -- 172.21.15.36:0/1985284243 learned_addr learned my addr 172.21.15.36:0/1985284243 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:44.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.308+0000 7f1f09af3700 1 --2- 172.21.15.36:0/1985284243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f04060ac0 0x7f1f040a28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:44.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.308+0000 7f1f0a2f4700 1 --2- 172.21.15.36:0/1985284243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1f04140e70 0x7f1f041464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:44.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.308+0000 7f1f092f2700 1 -- 172.21.15.36:0/1985284243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1f04140e70 msgr2=0x7f1f041464e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:44.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.308+0000 7f1f092f2700 1 --2- 172.21.15.36:0/1985284243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1f04140e70 0x7f1f041464e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:44.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.308+0000 7f1f092f2700 1 -- 172.21.15.36:0/1985284243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f04060ac0 msgr2=0x7f1f040a28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:44.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.308+0000 7f1f092f2700 1 --2- 172.21.15.36:0/1985284243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f04060ac0 0x7f1f040a28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:44.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.308+0000 7f1f092f2700 1 -- 172.21.15.36:0/1985284243 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0000b180 con 0x7f1f040b22b0 2022-06-17T12:54:44.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.308+0000 7f1f0a2f4700 1 --2- 172.21.15.36:0/1985284243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1f04140e70 0x7f1f041464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:44.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.308+0000 7f1f092f2700 1 --2- 172.21.15.36:0/1985284243 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1f040b22b0 0x7f1f040a2dd0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7f1f040a09b0 tx=0x7f1ef4007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:44.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.308+0000 7f1ef2ffd700 1 -- 172.21.15.36:0/1985284243 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ef40072e0 con 0x7f1f040b22b0 2022-06-17T12:54:44.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.308+0000 7f1ef2ffd700 1 -- 172.21.15.36:0/1985284243 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef4007440 con 0x7f1f040b22b0 2022-06-17T12:54:44.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.308+0000 7f1f0bd57700 1 -- 172.21.15.36:0/1985284243 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1efc009470 con 0x7f1f040b22b0 2022-06-17T12:54:44.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.308+0000 7f1ef2ffd700 1 -- 172.21.15.36:0/1985284243 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ef4005800 con 0x7f1f040b22b0 2022-06-17T12:54:44.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.308+0000 7f1f0bd57700 1 -- 172.21.15.36:0/1985284243 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f040a0f30 con 0x7f1f040b22b0 2022-06-17T12:54:44.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.308+0000 7f1f0bd57700 1 -- 172.21.15.36:0/1985284243 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f04057410 con 0x7f1f040b22b0 2022-06-17T12:54:44.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.308+0000 7f1ef2ffd700 1 -- 172.21.15.36:0/1985284243 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f1ef4010070 con 0x7f1f040b22b0 2022-06-17T12:54:44.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.312+0000 7f1ef2ffd700 1 --2- 172.21.15.36:0/1985284243 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1ee4053a40 0x7f1ee4055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:44.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.312+0000 7f1ef2ffd700 1 -- 172.21.15.36:0/1985284243 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1ef406b8e0 con 0x7f1f040b22b0 2022-06-17T12:54:44.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.312+0000 7f1f09af3700 1 --2- 172.21.15.36:0/1985284243 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1ee4053a40 0x7f1ee4055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:44.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.312+0000 7f1f09af3700 1 --2- 172.21.15.36:0/1985284243 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1ee4053a40 0x7f1ee4055f00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f1f0000b000 tx=0x7f1f00008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:44.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.316+0000 7f1ef2ffd700 1 -- 172.21.15.36:0/1985284243 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1ef4042550 con 0x7f1f040b22b0 2022-06-17T12:54:44.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.480+0000 7f1f0bd57700 1 -- 172.21.15.36:0/1985284243 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f0409ee80 con 0x7f1f040b22b0 2022-06-17T12:54:44.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.484+0000 7f1ef2ffd700 1 -- 172.21.15.36:0/1985284243 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f1ef4018020 con 0x7f1f040b22b0 2022-06-17T12:54:44.470 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:44.470 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":47,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/138643404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":138643404},{"type":"v1","addr":"172.21.15.70:6833","nonce":138643404}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":46}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:54:35.949730+0000","modified":"2022-06-17T12:54:36.953427+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7074},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7074":{"gid":7074,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/3267343301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3267343301},{"type":"v1","addr":"172.21.15.36:6835","nonce":3267343301}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-06-17T12:54:44.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.484+0000 7f1f0bd57700 1 -- 172.21.15.36:0/1985284243 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1ee4053a40 msgr2=0x7f1ee4055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:44.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.484+0000 7f1f0bd57700 1 --2- 172.21.15.36:0/1985284243 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1ee4053a40 0x7f1ee4055f00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f1f0000b000 tx=0x7f1f00008040).stop 2022-06-17T12:54:44.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.484+0000 7f1f0bd57700 1 -- 172.21.15.36:0/1985284243 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1f040b22b0 msgr2=0x7f1f040a2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:44.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.484+0000 7f1f0bd57700 1 --2- 172.21.15.36:0/1985284243 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1f040b22b0 0x7f1f040a2dd0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7f1f040a09b0 tx=0x7f1ef4007ad0).stop 2022-06-17T12:54:44.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.484+0000 7f1f0bd57700 1 -- 172.21.15.36:0/1985284243 shutdown_connections 2022-06-17T12:54:44.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.484+0000 7f1f0bd57700 1 --2- 172.21.15.36:0/1985284243 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1ee4053a40 0x7f1ee4055f00 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:44.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.484+0000 7f1f0bd57700 1 --2- 172.21.15.36:0/1985284243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1f04140e70 0x7f1f041464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:44.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.484+0000 7f1f0bd57700 1 --2- 172.21.15.36:0/1985284243 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1f040b22b0 0x7f1f040a2dd0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:44.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.484+0000 7f1f0bd57700 1 --2- 172.21.15.36:0/1985284243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f04060ac0 0x7f1f040a28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:44.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.484+0000 7f1f0bd57700 1 -- 172.21.15.36:0/1985284243 >> 172.21.15.36:0/1985284243 conn(0x7f1f040b6870 msgr2=0x7f1f04067800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:44.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.484+0000 7f1f0bd57700 1 -- 172.21.15.36:0/1985284243 shutdown_connections 2022-06-17T12:54:44.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:44.484+0000 7f1f0bd57700 1 -- 172.21.15.36:0/1985284243 wait complete. 2022-06-17T12:54:44.476 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 47 2022-06-17T12:54:44.684 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:54:35.949730+0000', 'modified': '2022-06-17T12:54:36.953427+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7074}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7074': {'gid': 7074, 'name': 'a', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.36:6835/3267343301', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 3267343301}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 3267343301}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:54:44.685 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:54:44.686 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:54:44.801 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":7074,"want_state":"up:active","state":"up:active","rank_uptime":8.8563487110000008,"mdsmap_epoch":47,"osdmap_epoch":94,"osdmap_epoch_barrier":93,"uptime":13.272899774000001} 2022-06-17T12:54:44.801 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 7074, "mdsmap_epoch": 47, "osdmap_epoch": 94, "osdmap_epoch_barrier": 93, "rank_uptime": 8.856348711, "state": "up:active", "uptime": 13.272899774, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:54:44.802 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:54:44.802 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-06-17T12:54:44.803 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi136.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-06-17T12:54:44.803 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:44.804 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:54:44.813 DEBUG:teuthology.orchestra.run.smithi136:> sudo modprobe fuse 2022-06-17T12:54:44.871 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:54:44.874 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:54:44.888 INFO:teuthology.orchestra.run.smithi136.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:54:44.889 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:54:44.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:44.892 DEBUG:teuthology.orchestra.run.smithi136:> ls /sys/fs/fuse/connections 2022-06-17T12:54:44.902 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:54:44.903 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:54:44.949 DEBUG:teuthology.orchestra.run.smithi136:> sudo modprobe fuse 2022-06-17T12:54:44.974 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:54:44.975 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:54:45.027 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:2022-06-17T12:54:45.038+0000 7f9d0d5aa080 -1 init, newargv = 0x55abfec70640 newargc=9 2022-06-17T12:54:45.027 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:ceph-fuse[22057]: starting ceph client 2022-06-17T12:54:45.033 INFO:teuthology.orchestra.run.smithi136.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:54:45.035 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:54:45.036 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:45.037 DEBUG:teuthology.orchestra.run.smithi136:> ls /sys/fs/fuse/connections 2022-06-17T12:54:45.039 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:ceph-fuse[22057]: starting fuse 2022-06-17T12:54:45.045 INFO:teuthology.orchestra.run.smithi136.stdout:52 2022-06-17T12:54:45.046 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:54:45.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:45.047 DEBUG:teuthology.orchestra.run.smithi136:> sudo python3 -c ' 2022-06-17T12:54:45.048 DEBUG:teuthology.orchestra.run.smithi136:> import glob 2022-06-17T12:54:45.049 DEBUG:teuthology.orchestra.run.smithi136:> import re 2022-06-17T12:54:45.049 DEBUG:teuthology.orchestra.run.smithi136:> import os 2022-06-17T12:54:45.056 DEBUG:teuthology.orchestra.run.smithi136:> import subprocess 2022-06-17T12:54:45.057 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:54:45.057 DEBUG:teuthology.orchestra.run.smithi136:> def find_socket(client_name): 2022-06-17T12:54:45.058 DEBUG:teuthology.orchestra.run.smithi136:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-06-17T12:54:45.059 DEBUG:teuthology.orchestra.run.smithi136:> files = glob.glob(asok_path) 2022-06-17T12:54:45.060 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:54:45.061 DEBUG:teuthology.orchestra.run.smithi136:> # Given a non-glob path, it better be there 2022-06-17T12:54:45.062 DEBUG:teuthology.orchestra.run.smithi136:> if "*" not in asok_path: 2022-06-17T12:54:45.063 DEBUG:teuthology.orchestra.run.smithi136:> assert(len(files) == 1) 2022-06-17T12:54:45.063 DEBUG:teuthology.orchestra.run.smithi136:> return files[0] 2022-06-17T12:54:45.064 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:54:45.064 DEBUG:teuthology.orchestra.run.smithi136:> for f in files: 2022-06-17T12:54:45.065 DEBUG:teuthology.orchestra.run.smithi136:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:54:45.065 DEBUG:teuthology.orchestra.run.smithi136:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:54:45.066 DEBUG:teuthology.orchestra.run.smithi136:> return f 2022-06-17T12:54:45.066 DEBUG:teuthology.orchestra.run.smithi136:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:54:45.067 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:54:45.067 DEBUG:teuthology.orchestra.run.smithi136:> print(find_socket("client.1")) 2022-06-17T12:54:45.068 DEBUG:teuthology.orchestra.run.smithi136:> ' 2022-06-17T12:54:45.133 INFO:teuthology.orchestra.run.smithi136.stdout:/var/run/ceph/ceph-client.1.22057.asok 2022-06-17T12:54:45.138 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22057.asok 2022-06-17T12:54:45.139 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:45.140 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22057.asok status 2022-06-17T12:54:45.226 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2022-06-17T12:54:45.227 INFO:teuthology.orchestra.run.smithi136.stdout: "metadata": { 2022-06-17T12:54:45.227 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:54:45.228 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:54:45.228 INFO:teuthology.orchestra.run.smithi136.stdout: "entity_id": "1", 2022-06-17T12:54:45.229 INFO:teuthology.orchestra.run.smithi136.stdout: "hostname": "smithi136", 2022-06-17T12:54:45.229 INFO:teuthology.orchestra.run.smithi136.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-06-17T12:54:45.230 INFO:teuthology.orchestra.run.smithi136.stdout: "pid": "22057", 2022-06-17T12:54:45.230 INFO:teuthology.orchestra.run.smithi136.stdout: "root": "/" 2022-06-17T12:54:45.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:54:45.231 INFO:teuthology.orchestra.run.smithi136.stdout: "dentry_count": 0, 2022-06-17T12:54:45.231 INFO:teuthology.orchestra.run.smithi136.stdout: "dentry_pinned_count": 0, 2022-06-17T12:54:45.232 INFO:teuthology.orchestra.run.smithi136.stdout: "id": 6296, 2022-06-17T12:54:45.232 INFO:teuthology.orchestra.run.smithi136.stdout: "inst": { 2022-06-17T12:54:45.233 INFO:teuthology.orchestra.run.smithi136.stdout: "name": { 2022-06-17T12:54:45.233 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "client", 2022-06-17T12:54:45.234 INFO:teuthology.orchestra.run.smithi136.stdout: "num": 6296 2022-06-17T12:54:45.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:54:45.234 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": { 2022-06-17T12:54:45.235 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "v1", 2022-06-17T12:54:45.235 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": "172.21.15.136:0", 2022-06-17T12:54:45.239 INFO:teuthology.orchestra.run.smithi136.stdout: "nonce": 4112281177 2022-06-17T12:54:45.239 INFO:teuthology.orchestra.run.smithi136.stdout: } 2022-06-17T12:54:45.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:54:45.240 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": { 2022-06-17T12:54:45.241 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "v1", 2022-06-17T12:54:45.243 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": "172.21.15.136:0", 2022-06-17T12:54:45.243 INFO:teuthology.orchestra.run.smithi136.stdout: "nonce": 4112281177 2022-06-17T12:54:45.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:54:45.244 INFO:teuthology.orchestra.run.smithi136.stdout: "inst_str": "client.6296 172.21.15.136:0/4112281177", 2022-06-17T12:54:45.244 INFO:teuthology.orchestra.run.smithi136.stdout: "addr_str": "172.21.15.136:0/4112281177", 2022-06-17T12:54:45.245 INFO:teuthology.orchestra.run.smithi136.stdout: "inode_count": 1, 2022-06-17T12:54:45.245 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_epoch": 47, 2022-06-17T12:54:45.245 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_epoch": 94, 2022-06-17T12:54:45.246 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:54:45.247 INFO:teuthology.orchestra.run.smithi136.stdout: "blacklisted": false, 2022-06-17T12:54:45.247 INFO:teuthology.orchestra.run.smithi136.stdout: "fs_name": "cephfs" 2022-06-17T12:54:45.248 INFO:teuthology.orchestra.run.smithi136.stdout:} 2022-06-17T12:54:45.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:45.252 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:45.252 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:54:45.261 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:54:45.263 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:54:45.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:45.265 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:54:45.324 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-06-17T12:54:45.325 DEBUG:teuthology.orchestra.run.smithi107:> cd /home/ubuntu/cephtest/mnt.0 && sudo mkdir -p foo 2022-06-17T12:54:45.355 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-06-17T12:54:45.356 DEBUG:teuthology.orchestra.run.smithi107:> import os 2022-06-17T12:54:45.356 DEBUG:teuthology.orchestra.run.smithi107:> import stat 2022-06-17T12:54:45.357 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:54:45.358 DEBUG:teuthology.orchestra.run.smithi107:> print(os.stat("/home/ubuntu/cephtest/mnt.0/foo").st_ino) 2022-06-17T12:54:45.359 DEBUG:teuthology.orchestra.run.smithi107:> ' 2022-06-17T12:54:45.478 INFO:teuthology.orchestra.run.smithi107.stdout:1099511627776 2022-06-17T12:54:45.671 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:54:45.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.796+0000 7ff4ef2cb700 1 Processor -- start 2022-06-17T12:54:45.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.796+0000 7ff4ef2cb700 1 -- start start 2022-06-17T12:54:45.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.796+0000 7ff4ef2cb700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff4e80b80c0 0x7ff4e8140c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:45.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.796+0000 7ff4ef2cb700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff4e8142d70 0x7ff4e8143220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:45.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.796+0000 7ff4ef2cb700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4e8142160 con 0x7ff4e80b7ce0 2022-06-17T12:54:45.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.796+0000 7ff4ef2cb700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e80b4f30 con 0x7ff4e80b80c0 2022-06-17T12:54:45.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.796+0000 7ff4ef2cb700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e8142390 con 0x7ff4e8142d70 2022-06-17T12:54:45.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.796+0000 7ff4edac8700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff4e80b80c0 0x7ff4e8140c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:45.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.800+0000 7ff4edac8700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff4e80b80c0 0x7ff4e8140c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37310/0 (socket says 172.21.15.36:37310) 2022-06-17T12:54:45.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.800+0000 7ff4edac8700 1 -- 172.21.15.36:0/2986598511 learned_addr learned my addr 172.21.15.36:0/2986598511 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:45.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.800+0000 7ff4eeaca700 1 --2- 172.21.15.36:0/2986598511 >> v2:172.21.15.70:3300/0 conn(0x7ff4e8142d70 0x7ff4e8143220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:45.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.800+0000 7ff4edac8700 1 -- 172.21.15.36:0/2986598511 >> v2:172.21.15.70:3300/0 conn(0x7ff4e8142d70 msgr2=0x7ff4e8143220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:45.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.800+0000 7ff4edac8700 1 --2- 172.21.15.36:0/2986598511 >> v2:172.21.15.70:3300/0 conn(0x7ff4e8142d70 0x7ff4e8143220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:45.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.800+0000 7ff4edac8700 1 -- 172.21.15.36:0/2986598511 >> v1:172.21.15.70:6789/0 conn(0x7ff4e80b7ce0 legacy=0x7ff4e813aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:45.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.800+0000 7ff4edac8700 1 -- 172.21.15.36:0/2986598511 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e8142630 con 0x7ff4e80b80c0 2022-06-17T12:54:45.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.800+0000 7ff4edac8700 1 --2- 172.21.15.36:0/2986598511 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff4e80b80c0 0x7ff4e8140c90 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7ff4d8002f70 tx=0x7ff4d8006eb0).ready entity=mon.0 client_cookie=7f1bc03f7167db7b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:45.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.800+0000 7ff4ed2c7700 1 -- 172.21.15.36:0/2986598511 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4d800d5c0 con 0x7ff4e80b80c0 2022-06-17T12:54:45.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.800+0000 7ff4ed2c7700 1 -- 172.21.15.36:0/2986598511 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d8000e90 con 0x7ff4e80b80c0 2022-06-17T12:54:45.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.800+0000 7ff4ed2c7700 1 -- 172.21.15.36:0/2986598511 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4d800d950 con 0x7ff4e80b80c0 2022-06-17T12:54:45.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.800+0000 7ff4ef2cb700 1 -- 172.21.15.36:0/2986598511 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff4e80b80c0 msgr2=0x7ff4e8140c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:45.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.800+0000 7ff4ef2cb700 1 --2- 172.21.15.36:0/2986598511 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff4e80b80c0 0x7ff4e8140c90 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7ff4d8002f70 tx=0x7ff4d8006eb0).stop 2022-06-17T12:54:45.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.800+0000 7ff4ef2cb700 1 -- 172.21.15.36:0/2986598511 shutdown_connections 2022-06-17T12:54:45.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.800+0000 7ff4ef2cb700 1 --2- 172.21.15.36:0/2986598511 >> v2:172.21.15.70:3300/0 conn(0x7ff4e8142d70 0x7ff4e8143220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:45.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.800+0000 7ff4ef2cb700 1 --2- 172.21.15.36:0/2986598511 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff4e80b80c0 0x7ff4e8140c90 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:45.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.800+0000 7ff4ef2cb700 1 -- 172.21.15.36:0/2986598511 >> 172.21.15.36:0/2986598511 conn(0x7ff4e805c550 msgr2=0x7ff4e80b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:45.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.800+0000 7ff4ef2cb700 1 -- 172.21.15.36:0/2986598511 shutdown_connections 2022-06-17T12:54:45.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.800+0000 7ff4ef2cb700 1 -- 172.21.15.36:0/2986598511 wait complete. 2022-06-17T12:54:45.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.804+0000 7ff4ef2cb700 1 Processor -- start 2022-06-17T12:54:45.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.804+0000 7ff4ef2cb700 1 -- start start 2022-06-17T12:54:45.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.804+0000 7ff4ef2cb700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4e80b7ce0 0x7ff4e8066400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:45.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.804+0000 7ff4ef2cb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff4e80b80c0 0x7ff4e8064900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:45.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.804+0000 7ff4ef2cb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4e8142d70 0x7ff4e8143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:45.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.804+0000 7ff4ef2cb700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e8067440 con 0x7ff4e80b80c0 2022-06-17T12:54:45.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.804+0000 7ff4ef2cb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e8065600 con 0x7ff4e8142d70 2022-06-17T12:54:45.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.804+0000 7ff4ef2cb700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e80668f0 con 0x7ff4e80b7ce0 2022-06-17T12:54:45.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.804+0000 7ff4edac8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff4e80b80c0 0x7ff4e8064900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:45.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.804+0000 7ff4edac8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff4e80b80c0 0x7ff4e8064900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37316/0 (socket says 172.21.15.36:37316) 2022-06-17T12:54:45.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.804+0000 7ff4edac8700 1 -- 172.21.15.36:0/1894347374 learned_addr learned my addr 172.21.15.36:0/1894347374 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:45.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.804+0000 7ff4eeaca700 1 --2- 172.21.15.36:0/1894347374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4e8142d70 0x7ff4e8143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:45.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.804+0000 7ff4ee2c9700 1 --2- 172.21.15.36:0/1894347374 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4e80b7ce0 0x7ff4e8066400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:45.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.804+0000 7ff4edac8700 1 -- 172.21.15.36:0/1894347374 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4e80b7ce0 msgr2=0x7ff4e8066400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:45.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.804+0000 7ff4edac8700 1 --2- 172.21.15.36:0/1894347374 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4e80b7ce0 0x7ff4e8066400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:45.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.804+0000 7ff4edac8700 1 -- 172.21.15.36:0/1894347374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4e8142d70 msgr2=0x7ff4e8143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:45.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.804+0000 7ff4edac8700 1 --2- 172.21.15.36:0/1894347374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4e8142d70 0x7ff4e8143bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:45.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.804+0000 7ff4edac8700 1 -- 172.21.15.36:0/1894347374 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d800b1e0 con 0x7ff4e80b80c0 2022-06-17T12:54:45.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.804+0000 7ff4edac8700 1 --2- 172.21.15.36:0/1894347374 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff4e80b80c0 0x7ff4e8064900 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7ff4e8065f60 tx=0x7ff4d8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:45.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.804+0000 7ff4d77fe700 1 -- 172.21.15.36:0/1894347374 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4d8011760 con 0x7ff4e80b80c0 2022-06-17T12:54:45.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.804+0000 7ff4d77fe700 1 -- 172.21.15.36:0/1894347374 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d80118c0 con 0x7ff4e80b80c0 2022-06-17T12:54:45.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.804+0000 7ff4d77fe700 1 -- 172.21.15.36:0/1894347374 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4d8011c20 con 0x7ff4e80b80c0 2022-06-17T12:54:45.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.804+0000 7ff4ef2cb700 1 -- 172.21.15.36:0/1894347374 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e80b29c0 con 0x7ff4e80b80c0 2022-06-17T12:54:45.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.808+0000 7ff4ef2cb700 1 -- 172.21.15.36:0/1894347374 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e80b2c20 con 0x7ff4e80b80c0 2022-06-17T12:54:45.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.808+0000 7ff4ef2cb700 1 -- 172.21.15.36:0/1894347374 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e8066a30 con 0x7ff4e80b80c0 2022-06-17T12:54:45.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.808+0000 7ff4d77fe700 1 -- 172.21.15.36:0/1894347374 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff4d801a040 con 0x7ff4e80b80c0 2022-06-17T12:54:45.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.808+0000 7ff4d77fe700 1 --2- 172.21.15.36:0/1894347374 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff4c8053a40 0x7ff4c8055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:45.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.808+0000 7ff4d77fe700 1 -- 172.21.15.36:0/1894347374 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff4d806ccb0 con 0x7ff4e80b80c0 2022-06-17T12:54:45.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.808+0000 7ff4ee2c9700 1 --2- 172.21.15.36:0/1894347374 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff4c8053a40 0x7ff4c8055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:45.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.808+0000 7ff4ee2c9700 1 --2- 172.21.15.36:0/1894347374 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff4c8053a40 0x7ff4c8055f00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7ff4e400d000 tx=0x7ff4e4009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:45.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.816+0000 7ff4d77fe700 1 -- 172.21.15.36:0/1894347374 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff4d8043b70 con 0x7ff4e80b80c0 2022-06-17T12:54:45.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.980+0000 7ff4ef2cb700 1 -- 172.21.15.36:0/1894347374 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4e81428d0 con 0x7ff4e80b80c0 2022-06-17T12:54:45.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.980+0000 7ff4d77fe700 1 -- 172.21.15.36:0/1894347374 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+2137 (secure 0 0 0) 0x7ff4d8023020 con 0x7ff4e80b80c0 2022-06-17T12:54:45.969 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:45.970 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":47,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/138643404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":138643404},{"type":"v1","addr":"172.21.15.70:6833","nonce":138643404}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":46}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:54:35.949730+0000","modified":"2022-06-17T12:54:36.953427+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7074},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7074":{"gid":7074,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/3267343301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3267343301},{"type":"v1","addr":"172.21.15.36:6835","nonce":3267343301}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-06-17T12:54:45.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.984+0000 7ff4ef2cb700 1 -- 172.21.15.36:0/1894347374 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff4c8053a40 msgr2=0x7ff4c8055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:45.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.984+0000 7ff4ef2cb700 1 --2- 172.21.15.36:0/1894347374 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff4c8053a40 0x7ff4c8055f00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7ff4e400d000 tx=0x7ff4e4009040).stop 2022-06-17T12:54:45.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.984+0000 7ff4ef2cb700 1 -- 172.21.15.36:0/1894347374 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff4e80b80c0 msgr2=0x7ff4e8064900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:45.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.984+0000 7ff4ef2cb700 1 --2- 172.21.15.36:0/1894347374 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff4e80b80c0 0x7ff4e8064900 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7ff4e8065f60 tx=0x7ff4d8015000).stop 2022-06-17T12:54:45.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.984+0000 7ff4ef2cb700 1 -- 172.21.15.36:0/1894347374 shutdown_connections 2022-06-17T12:54:45.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.984+0000 7ff4ef2cb700 1 --2- 172.21.15.36:0/1894347374 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff4c8053a40 0x7ff4c8055f00 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:45.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.984+0000 7ff4ef2cb700 1 --2- 172.21.15.36:0/1894347374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4e8142d70 0x7ff4e8143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:45.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.984+0000 7ff4ef2cb700 1 --2- 172.21.15.36:0/1894347374 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff4e80b80c0 0x7ff4e8064900 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:45.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.984+0000 7ff4ef2cb700 1 --2- 172.21.15.36:0/1894347374 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4e80b7ce0 0x7ff4e8066400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:45.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.984+0000 7ff4ef2cb700 1 -- 172.21.15.36:0/1894347374 >> 172.21.15.36:0/1894347374 conn(0x7ff4e805c550 msgr2=0x7ff4e80b6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:45.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.984+0000 7ff4ef2cb700 1 -- 172.21.15.36:0/1894347374 shutdown_connections 2022-06-17T12:54:45.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:45.984+0000 7ff4ef2cb700 1 -- 172.21.15.36:0/1894347374 wait complete. 2022-06-17T12:54:45.979 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 47 2022-06-17T12:54:45.981 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump inode 0x10000000000 2022-06-17T12:54:46.064 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:54:46.068 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:54:46.071 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:54:46.076 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:54:46.079 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:54:46.093 INFO:teuthology.orchestra.run.smithi036.stdout:{"path":"/foo","ino":1099511627776,"rdev":0,"ctime":"2022-06-17T12:54:45.364141+0000","btime":"2022-06-17T12:54:45.364141+0000","mode":16877,"uid":0,"gid":0,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2022-06-17T12:54:45.364141+0000","atime":"2022-06-17T12:54:45.364141+0000","time_warp_seq":0,"change_attr":0,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"version":2,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1},"nref":4,"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"linklock":{},"dirfragtreelock":{},"filelock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"xattrlock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":6263,"pending":"pAsxLsXsxFsx","issued":"pAsxLsXsxFsx","wanted":"-","last_sent":1}],"loner":6263,"want_loner":6263,"mds_caps_wanted":[]} 2022-06-17T12:54:46.094 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2022-06-17T12:54:45.364141+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "backtrace_version": 2, "btime": "2022-06-17T12:54:45.364141+0000", "change_attr": 0, "client_caps": [ { "client_id": 6263, "issued": "pAsxLsXsxFsx", "last_sent": 1, "pending": "pAsxLsXsxFsx", "wanted": "-" } ], "client_ranges": [], "ctime": "2022-06-17T12:54:45.364141+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "flocklock": {}, "gid": 0, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": 6263, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2022-06-17T12:54:45.364141+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "nlink": 1, "nref": 4, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/foo", "pins": { "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "request": 0 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 0, "version": 2, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "want_loner": 6263, "xattr_version": 1, "xattrlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "xattrs": [] } 2022-06-17T12:54:46.094 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-17T12:54:46.095 DEBUG:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:46.176 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:46.206 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:46.229 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:46.284 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:46.308 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-17T12:54:46.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:46.310 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:46.310 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:54:46.321 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:54:46.322 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:54:46.322 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2022-06-17T12:54:46.323 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:54:46.323 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:54:46.341 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22600]: fuse finished with error 0 and tester_r 0 2022-06-17T12:54:46.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:46.344 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:46.344 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:54:46.351 INFO:teuthology.orchestra.run.smithi107.stdout:ext2/ext3 2022-06-17T12:54:46.352 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:54:46.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:54:46.470 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:54:46.478 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:46.479 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:46.480 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:54:46.489 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:54:46.491 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:54:46.491 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi136.front.sepia.ceph.com... 2022-06-17T12:54:46.492 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:54:46.493 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:54:46.509 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:ceph-fuse[22057]: fuse finished with error 0 and tester_r 0 2022-06-17T12:54:46.511 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:46.512 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:46.513 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:54:46.560 INFO:teuthology.orchestra.run.smithi136.stdout:ext2/ext3 2022-06-17T12:54:46.562 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:54:46.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:54:46.620 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:54:46.627 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:46.628 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:46.628 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:54:46.636 INFO:teuthology.orchestra.run.smithi143.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:54:46.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:54:46.639 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-06-17T12:54:46.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:54:46.640 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:54:46.686 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:54:46.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:54:46.688 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-17T12:54:46.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8e0f734700 1 Processor -- start 2022-06-17T12:54:46.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8e0f734700 1 -- start start 2022-06-17T12:54:46.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8e0f734700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8e080ac340 0x7f8e0814b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:46.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8e0f734700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e080ac870 con 0x7f8e080ad140 2022-06-17T12:54:46.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8e0f734700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e080acaa0 con 0x7f8e080af7f0 2022-06-17T12:54:46.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8e0f734700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e08148960 con 0x7f8e080ac340 2022-06-17T12:54:46.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8e0d4d0700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f8e080ad140 0x7f8e0814aca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54852/0 (socket says 172.21.15.36:54852) 2022-06-17T12:54:46.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8e0d4d0700 1 -- 172.21.15.36:0/971647306 learned_addr learned my addr 172.21.15.36:0/971647306 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:46.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8e0cccf700 1 --2- 172.21.15.36:0/971647306 >> v2:172.21.15.70:3300/0 conn(0x7f8e080ac340 0x7f8e0814b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:46.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8dfffff700 1 -- 172.21.15.36:0/971647306 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3083619414 0 0) 0x7f8e080ac870 con 0x7f8e080ad140 2022-06-17T12:54:46.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8dfffff700 1 -- 172.21.15.36:0/971647306 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dec003260 con 0x7f8e080ad140 2022-06-17T12:54:46.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8e0cccf700 1 -- 172.21.15.36:0/971647306 >> v1:172.21.15.70:6789/0 conn(0x7f8e080af7f0 legacy=0x7f8e08147b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:46.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8e0cccf700 1 -- 172.21.15.36:0/971647306 >> v1:172.21.15.36:6789/0 conn(0x7f8e080ad140 legacy=0x7f8e0814aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:46.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8dfffff700 1 -- 172.21.15.36:0/971647306 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2161091111 0 0) 0x7f8e080acaa0 con 0x7f8e080af7f0 2022-06-17T12:54:46.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8e0cccf700 1 -- 172.21.15.36:0/971647306 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e080b0bf0 con 0x7f8e080ac340 2022-06-17T12:54:46.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8e0cccf700 1 --2- 172.21.15.36:0/971647306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e080ac340 0x7f8e0814b3b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f8df8002f70 tx=0x7f8df8006df0).ready entity=mon.1 client_cookie=d257d92bc189b2f8 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:46.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8dfffff700 1 -- 172.21.15.36:0/971647306 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8df800d610 con 0x7f8e080ac340 2022-06-17T12:54:46.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8dfffff700 1 -- 172.21.15.36:0/971647306 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df8000e90 con 0x7f8e080ac340 2022-06-17T12:54:46.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8dfffff700 1 -- 172.21.15.36:0/971647306 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8df800d960 con 0x7f8e080ac340 2022-06-17T12:54:46.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8e0f734700 1 -- 172.21.15.36:0/971647306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e080ac340 msgr2=0x7f8e0814b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:46.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8e0f734700 1 --2- 172.21.15.36:0/971647306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e080ac340 0x7f8e0814b3b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f8df8002f70 tx=0x7f8df8006df0).stop 2022-06-17T12:54:46.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8e0f734700 1 -- 172.21.15.36:0/971647306 shutdown_connections 2022-06-17T12:54:46.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8e0f734700 1 --2- 172.21.15.36:0/971647306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e080ac340 0x7f8e0814b3b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:46.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8e0f734700 1 -- 172.21.15.36:0/971647306 >> 172.21.15.36:0/971647306 conn(0x7f8e080b5430 msgr2=0x7f8e080b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:46.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8e0f734700 1 -- 172.21.15.36:0/971647306 shutdown_connections 2022-06-17T12:54:46.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.820+0000 7f8e0f734700 1 -- 172.21.15.36:0/971647306 wait complete. 2022-06-17T12:54:46.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.824+0000 7f8e0f734700 1 Processor -- start 2022-06-17T12:54:46.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.824+0000 7f8e0f734700 1 -- start start 2022-06-17T12:54:46.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.824+0000 7f8e0f734700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e080ac340 0x7f8e0805fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:46.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.824+0000 7f8e0f734700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e080ad140 0x7f8e080a9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:46.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.824+0000 7f8e0f734700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e080af7f0 0x7f8e080a9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:46.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.824+0000 7f8e0f734700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e08060860 con 0x7f8e080ac340 2022-06-17T12:54:46.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.824+0000 7f8e0f734700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e080a85c0 con 0x7f8e080af7f0 2022-06-17T12:54:46.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.824+0000 7f8e0f734700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0814a690 con 0x7f8e080ad140 2022-06-17T12:54:46.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.824+0000 7f8e0d4d0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e080ac340 0x7f8e0805fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:46.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.824+0000 7f8e0d4d0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e080ac340 0x7f8e0805fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37328/0 (socket says 172.21.15.36:37328) 2022-06-17T12:54:46.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.824+0000 7f8e0d4d0700 1 -- 172.21.15.36:0/4100873735 learned_addr learned my addr 172.21.15.36:0/4100873735 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:46.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.824+0000 7f8e0cccf700 1 --2- 172.21.15.36:0/4100873735 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e080ad140 0x7f8e080a9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:46.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.824+0000 7f8e0dcd1700 1 --2- 172.21.15.36:0/4100873735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e080af7f0 0x7f8e080a9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:46.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.824+0000 7f8e0d4d0700 1 -- 172.21.15.36:0/4100873735 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e080ad140 msgr2=0x7f8e080a9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:46.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.824+0000 7f8e0d4d0700 1 --2- 172.21.15.36:0/4100873735 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e080ad140 0x7f8e080a9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:46.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.824+0000 7f8e0d4d0700 1 -- 172.21.15.36:0/4100873735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e080af7f0 msgr2=0x7f8e080a9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:46.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.824+0000 7f8e0d4d0700 1 --2- 172.21.15.36:0/4100873735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e080af7f0 0x7f8e080a9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:46.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.824+0000 7f8e0d4d0700 1 -- 172.21.15.36:0/4100873735 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df800b180 con 0x7f8e080ac340 2022-06-17T12:54:46.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.828+0000 7f8e0d4d0700 1 --2- 172.21.15.36:0/4100873735 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e080ac340 0x7f8e0805fd70 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f8e04001110 tx=0x7f8e04007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:46.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.828+0000 7f8dfe7fc700 1 -- 172.21.15.36:0/4100873735 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e0400ce80 con 0x7f8e080ac340 2022-06-17T12:54:46.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.828+0000 7f8dfe7fc700 1 -- 172.21.15.36:0/4100873735 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e040070d0 con 0x7f8e080ac340 2022-06-17T12:54:46.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.828+0000 7f8e0f734700 1 -- 172.21.15.36:0/4100873735 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e0814b5b0 con 0x7f8e080ac340 2022-06-17T12:54:46.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.828+0000 7f8dfe7fc700 1 -- 172.21.15.36:0/4100873735 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e04007430 con 0x7f8e080ac340 2022-06-17T12:54:46.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.828+0000 7f8e0f734700 1 -- 172.21.15.36:0/4100873735 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e0814a180 con 0x7f8e080ac340 2022-06-17T12:54:46.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.828+0000 7f8e0f734700 1 -- 172.21.15.36:0/4100873735 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e0814a850 con 0x7f8e080ac340 2022-06-17T12:54:46.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.828+0000 7f8dfe7fc700 1 -- 172.21.15.36:0/4100873735 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8e04010070 con 0x7f8e080ac340 2022-06-17T12:54:46.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.828+0000 7f8dfe7fc700 1 --2- 172.21.15.36:0/4100873735 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8df0053af0 0x7f8df0055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:46.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.828+0000 7f8dfe7fc700 1 -- 172.21.15.36:0/4100873735 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8e0406b500 con 0x7f8e080ac340 2022-06-17T12:54:46.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.828+0000 7f8e0cccf700 1 --2- 172.21.15.36:0/4100873735 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8df0053af0 0x7f8df0055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:46.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.832+0000 7f8e0cccf700 1 --2- 172.21.15.36:0/4100873735 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8df0053af0 0x7f8df0055fb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f8df8018000 tx=0x7f8df8014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:46.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.832+0000 7f8dfe7fc700 1 -- 172.21.15.36:0/4100873735 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8e04042300 con 0x7f8e080ac340 2022-06-17T12:54:46.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.964+0000 7f8e0f734700 1 -- 172.21.15.36:0/4100873735 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f8e0814b740 con 0x7f8e080ac340 2022-06-17T12:54:46.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.964+0000 7f8dfe7fc700 1 -- 172.21.15.36:0/4100873735 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v94) v1 ==== 81+0+34040 (secure 0 0 0) 0x7f8e04045920 con 0x7f8e080ac340 2022-06-17T12:54:46.955 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:46.955 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:54:46.955 INFO:teuthology.orchestra.run.smithi036.stdout: "epoch": 94, 2022-06-17T12:54:46.956 INFO:teuthology.orchestra.run.smithi036.stdout: "fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", 2022-06-17T12:54:46.956 INFO:teuthology.orchestra.run.smithi036.stdout: "created": "2022-06-17T12:46:30.671408+0000", 2022-06-17T12:54:46.956 INFO:teuthology.orchestra.run.smithi036.stdout: "modified": "2022-06-17T12:54:37.964330+0000", 2022-06-17T12:54:46.956 INFO:teuthology.orchestra.run.smithi036.stdout: "last_up_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:54:46.957 INFO:teuthology.orchestra.run.smithi036.stdout: "last_in_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:54:46.957 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-17T12:54:46.957 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_num": 5799936, 2022-06-17T12:54:46.957 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_set": [ 2022-06-17T12:54:46.957 INFO:teuthology.orchestra.run.smithi036.stdout: "pglog_hardlimit", 2022-06-17T12:54:46.958 INFO:teuthology.orchestra.run.smithi036.stdout: "purged_snapdirs", 2022-06-17T12:54:46.958 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_deletes", 2022-06-17T12:54:46.958 INFO:teuthology.orchestra.run.smithi036.stdout: "sortbitwise" 2022-06-17T12:54:46.958 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:54:46.959 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_version": 6, 2022-06-17T12:54:46.959 INFO:teuthology.orchestra.run.smithi036.stdout: "full_ratio": 0.94999998807907104, 2022-06-17T12:54:46.959 INFO:teuthology.orchestra.run.smithi036.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-17T12:54:46.960 INFO:teuthology.orchestra.run.smithi036.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-17T12:54:46.960 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_snapshot": "", 2022-06-17T12:54:46.960 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_max": 18, 2022-06-17T12:54:46.960 INFO:teuthology.orchestra.run.smithi036.stdout: "max_osd": 8, 2022-06-17T12:54:46.961 INFO:teuthology.orchestra.run.smithi036.stdout: "require_min_compat_client": "jewel", 2022-06-17T12:54:46.962 INFO:teuthology.orchestra.run.smithi036.stdout: "min_compat_client": "jewel", 2022-06-17T12:54:46.962 INFO:teuthology.orchestra.run.smithi036.stdout: "require_osd_release": "octopus", 2022-06-17T12:54:46.962 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:54:46.962 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:46.963 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 1, 2022-06-17T12:54:46.963 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "device_health_metrics", 2022-06-17T12:54:46.963 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:33.881212+0000", 2022-06-17T12:54:46.963 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:54:46.964 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:54:46.964 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:54:46.964 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:54:46.964 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:54:46.965 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:54:46.965 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:54:46.965 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:54:46.965 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 1, 2022-06-17T12:54:46.966 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 1, 2022-06-17T12:54:46.966 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 1, 2022-06-17T12:54:46.967 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 1, 2022-06-17T12:54:46.967 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 1, 2022-06-17T12:54:46.967 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:54:46.967 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:54:46.967 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:54:46.968 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:54:46.968 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:54:46.968 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:54:46.968 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:54:46.969 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:46.969 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "16", 2022-06-17T12:54:46.969 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:54:46.970 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:54:46.970 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:54:46.970 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:54:46.971 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:54:46.972 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:54:46.972 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:54:46.973 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:54:46.973 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:54:46.974 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:54:46.975 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:54:46.975 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:54:46.975 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:54:46.976 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:54:46.976 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:54:46.977 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:54:46.977 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:54:46.977 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:54:46.978 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:54:46.978 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:54:46.979 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:54:46.979 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:54:46.979 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:54:46.980 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:54:46.980 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:54:46.981 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:54:46.981 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:46.982 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:54:46.982 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:54:46.983 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:54:46.983 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:54:46.983 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:54:46.984 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:54:46.984 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:54:46.985 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:54:46.985 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:54:46.985 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:54:46.986 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:54:46.986 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:54:46.987 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 1 2022-06-17T12:54:46.987 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:46.988 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:54:46.988 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr_devicehealth": {} 2022-06-17T12:54:46.989 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:46.989 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:46.990 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:46.990 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 2, 2022-06-17T12:54:46.991 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "rbd", 2022-06-17T12:54:46.991 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:44.485460+0000", 2022-06-17T12:54:46.991 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:54:46.992 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:54:46.992 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:54:46.993 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:54:46.993 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:54:46.993 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:54:46.994 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:54:46.994 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:54:46.994 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:54:46.995 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:54:46.995 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:54:46.996 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:54:46.996 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:54:46.997 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:54:46.997 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:54:46.997 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:54:46.998 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:54:46.998 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:54:46.999 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:54:46.999 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:54:46.999 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.000 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "19", 2022-06-17T12:54:47.000 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:54:47.001 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:54:47.001 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:54:47.002 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:54:47.002 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:54:47.003 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:54:47.003 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:54:47.003 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:54:47.004 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:54:47.004 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:54:47.004 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:54:47.005 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:54:47.005 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:54:47.006 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:54:47.006 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:54:47.007 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:54:47.007 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:54:47.008 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:54:47.008 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:54:47.009 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:54:47.009 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:54:47.009 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:54:47.010 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:54:47.010 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:54:47.011 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:54:47.011 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:54:47.011 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.012 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:54:47.012 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:54:47.012 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:54:47.013 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:54:47.013 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:54:47.013 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:54:47.014 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:54:47.014 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:54:47.015 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:54:47.015 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:54:47.016 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:54:47.016 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:54:47.017 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:54:47.017 INFO:teuthology.orchestra.run.smithi036.stdout: "rbd": {} 2022-06-17T12:54:47.017 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.018 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.018 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.018 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 17, 2022-06-17T12:54:47.018 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_metadata", 2022-06-17T12:54:47.019 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:54:34.381157+0000", 2022-06-17T12:54:47.019 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:54:47.019 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:54:47.019 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:54:47.020 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:54:47.020 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:54:47.020 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:54:47.021 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:54:47.021 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:54:47.022 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:54:47.022 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:54:47.022 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:54:47.023 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:54:47.023 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:54:47.023 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:54:47.024 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:54:47.024 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:54:47.025 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:54:47.025 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:54:47.025 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:54:47.025 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:54:47.026 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.026 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "93", 2022-06-17T12:54:47.026 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:54:47.027 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:54:47.027 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:54:47.027 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:54:47.027 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:54:47.028 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:54:47.028 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:54:47.028 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:54:47.028 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:54:47.029 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:54:47.029 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:54:47.029 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:54:47.030 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:54:47.030 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:54:47.030 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:54:47.031 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:54:47.031 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:54:47.031 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:54:47.032 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:54:47.032 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:54:47.032 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:54:47.032 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:54:47.033 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:54:47.033 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:54:47.033 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:54:47.033 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:54:47.034 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.034 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:54:47.034 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:54:47.034 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:54:47.035 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:54:47.035 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:54:47.035 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:54:47.035 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:54:47.036 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:54:47.036 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:54:47.036 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:54:47.037 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:54:47.037 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:54:47.037 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_bias": 4, 2022-06-17T12:54:47.037 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 16, 2022-06-17T12:54:47.038 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_priority": 5 2022-06-17T12:54:47.038 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.038 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:54:47.038 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:54:47.039 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": "cephfs" 2022-06-17T12:54:47.039 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.039 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.039 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.040 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.040 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 18, 2022-06-17T12:54:47.040 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_data", 2022-06-17T12:54:47.040 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:54:35.204631+0000", 2022-06-17T12:54:47.041 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:54:47.041 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:54:47.041 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:54:47.042 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:54:47.042 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:54:47.042 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:54:47.042 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:54:47.043 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:54:47.043 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:54:47.043 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:54:47.044 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:54:47.044 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:54:47.044 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:54:47.044 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:54:47.045 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:54:47.045 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:54:47.045 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:54:47.045 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:54:47.046 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:54:47.046 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:54:47.046 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.046 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "94", 2022-06-17T12:54:47.046 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:54:47.047 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:54:47.047 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:54:47.047 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:54:47.048 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:54:47.048 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:54:47.048 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:54:47.048 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:54:47.049 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:54:47.049 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:54:47.049 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:54:47.049 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:54:47.049 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:54:47.050 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:54:47.050 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:54:47.050 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:54:47.050 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:54:47.051 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:54:47.051 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:54:47.051 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:54:47.051 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:54:47.052 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:54:47.052 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:54:47.052 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:54:47.052 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:54:47.052 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:54:47.053 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.053 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:54:47.053 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:54:47.053 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:54:47.053 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:54:47.054 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:54:47.054 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:54:47.054 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:54:47.054 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:54:47.055 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:54:47.055 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:54:47.055 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:54:47.055 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:54:47.056 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:54:47.056 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:54:47.056 INFO:teuthology.orchestra.run.smithi036.stdout: "data": "cephfs" 2022-06-17T12:54:47.056 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.057 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.057 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.057 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:54:47.057 INFO:teuthology.orchestra.run.smithi036.stdout: "osds": [ 2022-06-17T12:54:47.058 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.059 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:54:47.059 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", 2022-06-17T12:54:47.060 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:47.060 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:47.061 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:47.061 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:47.062 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:47.062 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:47.062 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:54:47.063 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 91, 2022-06-17T12:54:47.063 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:47.063 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:47.063 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:47.064 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.064 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.064 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.064 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6802", 2022-06-17T12:54:47.065 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:47.065 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.065 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.065 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.065 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6803", 2022-06-17T12:54:47.066 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:47.066 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.066 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.066 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.067 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:47.067 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.067 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.067 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.068 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6804", 2022-06-17T12:54:47.068 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:47.068 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.068 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.069 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.069 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6805", 2022-06-17T12:54:47.069 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:47.069 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.070 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.070 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.070 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:47.071 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.071 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.071 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.072 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6808", 2022-06-17T12:54:47.072 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:47.073 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.073 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.073 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.074 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6809", 2022-06-17T12:54:47.074 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:47.075 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.075 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.075 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.076 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:47.076 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.076 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.077 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.077 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6806", 2022-06-17T12:54:47.077 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:47.078 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.078 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.079 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.079 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6807", 2022-06-17T12:54:47.079 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:47.080 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.080 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.080 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.081 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6803/22619", 2022-06-17T12:54:47.081 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6805/22619", 2022-06-17T12:54:47.081 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6809/22619", 2022-06-17T12:54:47.082 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6807/22619", 2022-06-17T12:54:47.082 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:47.083 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:47.083 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:47.084 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.084 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.084 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.085 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:54:47.085 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", 2022-06-17T12:54:47.086 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:47.086 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:47.087 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:47.087 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:47.087 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:47.088 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:47.088 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:54:47.088 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 91, 2022-06-17T12:54:47.089 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:47.089 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:47.089 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:47.090 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.090 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.091 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.091 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6810", 2022-06-17T12:54:47.091 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:47.092 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.092 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.092 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.093 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6811", 2022-06-17T12:54:47.093 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:47.094 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.094 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.094 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.095 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:47.095 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.095 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.096 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.096 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6812", 2022-06-17T12:54:47.097 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:47.097 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.097 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.098 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.098 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6813", 2022-06-17T12:54:47.099 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:47.099 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.099 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.100 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.100 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:47.100 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.101 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.101 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.101 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6816", 2022-06-17T12:54:47.101 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:47.102 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.102 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.102 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.103 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6817", 2022-06-17T12:54:47.103 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:47.103 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.103 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.104 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.104 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:47.104 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.104 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.105 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.105 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6814", 2022-06-17T12:54:47.105 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:47.105 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.105 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.106 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.106 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6815", 2022-06-17T12:54:47.106 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:47.106 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.107 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.107 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.107 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6811/22620", 2022-06-17T12:54:47.107 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6813/22620", 2022-06-17T12:54:47.108 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6817/22620", 2022-06-17T12:54:47.108 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6815/22620", 2022-06-17T12:54:47.108 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:47.108 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:47.109 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:47.109 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.109 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.109 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.109 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:54:47.110 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", 2022-06-17T12:54:47.110 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:47.111 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:47.111 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:47.111 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:47.112 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:47.112 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:47.112 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:54:47.113 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 92, 2022-06-17T12:54:47.113 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:47.114 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:47.114 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:47.115 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.115 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.116 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.116 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6826", 2022-06-17T12:54:47.116 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:47.117 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.117 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.118 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.118 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6827", 2022-06-17T12:54:47.118 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:47.119 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.119 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.120 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.120 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:47.120 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.121 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.121 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.122 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6828", 2022-06-17T12:54:47.122 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:47.122 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.123 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.123 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.124 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6829", 2022-06-17T12:54:47.124 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:47.125 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.125 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.126 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.126 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:47.127 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.127 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.127 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.128 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6832", 2022-06-17T12:54:47.128 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:47.129 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.129 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.130 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.130 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6833", 2022-06-17T12:54:47.130 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:47.131 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.131 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.132 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.132 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:47.133 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.133 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.134 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.134 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6830", 2022-06-17T12:54:47.134 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:47.135 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.135 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.136 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.136 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6831", 2022-06-17T12:54:47.136 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:47.137 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.137 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.137 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.137 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6827/22643", 2022-06-17T12:54:47.138 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6829/22643", 2022-06-17T12:54:47.138 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6833/22643", 2022-06-17T12:54:47.138 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6831/22643", 2022-06-17T12:54:47.138 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:47.139 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:47.139 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:47.139 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.139 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.140 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.140 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:54:47.140 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", 2022-06-17T12:54:47.141 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:47.141 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:47.141 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:47.141 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:47.142 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:47.142 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:47.142 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:54:47.142 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 92, 2022-06-17T12:54:47.143 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:47.143 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:47.143 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:47.144 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.144 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.144 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.144 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6818", 2022-06-17T12:54:47.144 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:47.145 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.145 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.145 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.145 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6819", 2022-06-17T12:54:47.146 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:47.146 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.146 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.146 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.147 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:47.147 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.147 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.147 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.147 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6820", 2022-06-17T12:54:47.148 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:47.148 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.148 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.148 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.149 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6821", 2022-06-17T12:54:47.149 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:47.149 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.149 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.150 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.150 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:47.150 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.150 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.151 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.151 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6824", 2022-06-17T12:54:47.151 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:47.151 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.152 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.152 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.152 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6825", 2022-06-17T12:54:47.152 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:47.153 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.153 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.153 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.154 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:47.154 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.154 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.155 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.155 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6822", 2022-06-17T12:54:47.155 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:47.155 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.156 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.156 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.156 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6823", 2022-06-17T12:54:47.156 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:47.157 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.157 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.157 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.157 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6819/22646", 2022-06-17T12:54:47.158 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6821/22646", 2022-06-17T12:54:47.158 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6825/22646", 2022-06-17T12:54:47.158 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6823/22646", 2022-06-17T12:54:47.158 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:47.159 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:47.159 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:47.159 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.159 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.160 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.160 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:54:47.160 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", 2022-06-17T12:54:47.160 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:47.160 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:47.161 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:47.161 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:47.161 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:47.161 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:47.162 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:54:47.162 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 90, 2022-06-17T12:54:47.163 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:47.163 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:47.163 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:47.164 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.164 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.164 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.164 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6808", 2022-06-17T12:54:47.165 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:47.165 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.165 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.165 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.166 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6809", 2022-06-17T12:54:47.166 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:47.166 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.167 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.167 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.167 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:47.168 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.168 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.168 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.169 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6810", 2022-06-17T12:54:47.169 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:47.169 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.169 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.170 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.170 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6811", 2022-06-17T12:54:47.170 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:47.170 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.171 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.171 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.171 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:47.171 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.171 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.172 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.172 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6814", 2022-06-17T12:54:47.172 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:47.172 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.173 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.173 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.173 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6815", 2022-06-17T12:54:47.174 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:47.174 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.175 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.175 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.176 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:47.176 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.177 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.177 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.177 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6812", 2022-06-17T12:54:47.178 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:47.178 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.179 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.179 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.180 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6813", 2022-06-17T12:54:47.180 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:47.180 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.181 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.181 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.182 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6809/22812", 2022-06-17T12:54:47.182 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6811/22812", 2022-06-17T12:54:47.182 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6815/22812", 2022-06-17T12:54:47.183 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6813/22812", 2022-06-17T12:54:47.183 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:47.183 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:47.184 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:47.184 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.184 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.185 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.185 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:54:47.185 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", 2022-06-17T12:54:47.186 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:47.186 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:47.187 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:47.187 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:47.188 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:47.188 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:47.189 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:54:47.189 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 82, 2022-06-17T12:54:47.190 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:47.190 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:47.190 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:47.191 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.191 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.192 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.192 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6800", 2022-06-17T12:54:47.193 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:47.193 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.193 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.194 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.194 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6801", 2022-06-17T12:54:47.195 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:47.195 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.195 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.196 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.196 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:47.196 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.196 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.197 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.197 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6802", 2022-06-17T12:54:47.197 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:47.197 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.198 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.198 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.198 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6803", 2022-06-17T12:54:47.198 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:47.199 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.199 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.199 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.200 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:47.200 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.200 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.200 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.201 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6806", 2022-06-17T12:54:47.201 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:47.202 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.202 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.202 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.203 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6807", 2022-06-17T12:54:47.203 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:47.203 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.204 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.204 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.204 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:47.205 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.205 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.205 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.205 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6804", 2022-06-17T12:54:47.206 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:47.206 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.206 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.207 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.207 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6805", 2022-06-17T12:54:47.207 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:47.208 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.208 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.208 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.209 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6801/22813", 2022-06-17T12:54:47.209 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6803/22813", 2022-06-17T12:54:47.209 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6807/22813", 2022-06-17T12:54:47.210 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6805/22813", 2022-06-17T12:54:47.210 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:47.210 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:47.211 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:47.211 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.211 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.212 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.212 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:54:47.212 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", 2022-06-17T12:54:47.213 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:47.213 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:47.213 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:47.214 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:47.214 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:47.214 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:47.215 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:54:47.215 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 90, 2022-06-17T12:54:47.215 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:47.216 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:47.216 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:47.216 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.216 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.217 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.217 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6816", 2022-06-17T12:54:47.218 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:47.218 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.218 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.219 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.219 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6817", 2022-06-17T12:54:47.219 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:47.220 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.220 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.220 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.220 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:47.221 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.221 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.221 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.222 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6818", 2022-06-17T12:54:47.222 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:47.222 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.223 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.223 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.223 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6819", 2022-06-17T12:54:47.224 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:47.224 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.224 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.225 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.225 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:47.225 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.226 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.226 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.226 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6822", 2022-06-17T12:54:47.227 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:47.227 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.227 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.228 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.228 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6823", 2022-06-17T12:54:47.229 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:47.229 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.229 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.230 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.230 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:47.230 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.231 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.231 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.231 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6820", 2022-06-17T12:54:47.232 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:47.232 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.232 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.233 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.233 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6821", 2022-06-17T12:54:47.233 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:47.234 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.234 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.235 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.235 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6817/22822", 2022-06-17T12:54:47.235 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6819/22822", 2022-06-17T12:54:47.236 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6823/22822", 2022-06-17T12:54:47.236 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6821/22822", 2022-06-17T12:54:47.236 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:47.237 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:47.237 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:47.237 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.238 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.238 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.238 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:54:47.239 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", 2022-06-17T12:54:47.239 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:47.239 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:47.240 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:47.240 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:47.240 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:47.241 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:47.241 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:54:47.242 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 91, 2022-06-17T12:54:47.242 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:47.242 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:47.243 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:47.244 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.244 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.244 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.245 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6824", 2022-06-17T12:54:47.246 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:47.246 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.246 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.247 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.247 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6825", 2022-06-17T12:54:47.247 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:47.248 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.249 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.249 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.249 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:47.250 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.250 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.251 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.251 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6826", 2022-06-17T12:54:47.251 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:47.252 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.252 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.252 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.253 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6827", 2022-06-17T12:54:47.253 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:47.253 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.254 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.254 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.255 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:47.255 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.255 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.256 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.256 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6830", 2022-06-17T12:54:47.257 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:47.257 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.258 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.258 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.259 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6831", 2022-06-17T12:54:47.259 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:47.259 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.260 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.260 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.260 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:47.261 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:47.261 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.262 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:47.262 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6828", 2022-06-17T12:54:47.262 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:47.263 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.263 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.263 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:47.264 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6829", 2022-06-17T12:54:47.264 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:47.265 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.265 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.266 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.266 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6825/22837", 2022-06-17T12:54:47.266 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6827/22837", 2022-06-17T12:54:47.267 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6831/22837", 2022-06-17T12:54:47.267 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6829/22837", 2022-06-17T12:54:47.267 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:47.268 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:47.269 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:47.269 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:47.270 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.270 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:54:47.271 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_xinfo": [ 2022-06-17T12:54:47.271 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.272 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:54:47.272 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:47.273 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:47.273 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:47.274 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:47.274 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:47.275 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.291461+0000", 2022-06-17T12:54:47.275 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:47.276 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.276 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.277 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:54:47.277 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:47.278 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:47.278 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:47.279 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:47.279 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:47.280 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.537912+0000", 2022-06-17T12:54:47.280 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:47.280 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.282 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.282 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:54:47.284 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:47.285 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:47.285 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:47.286 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:47.286 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:47.287 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.606947+0000", 2022-06-17T12:54:47.287 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:47.288 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.288 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.288 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:54:47.289 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:47.289 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:47.291 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:47.291 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:47.292 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:47.293 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.514412+0000", 2022-06-17T12:54:47.293 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:47.295 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.295 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.295 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:54:47.296 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:47.296 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:47.297 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:47.297 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:47.298 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:47.298 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.394645+0000", 2022-06-17T12:54:47.299 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:47.299 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.299 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.300 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:54:47.300 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:47.301 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:47.301 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:47.302 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:47.303 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:47.303 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.343806+0000", 2022-06-17T12:54:47.304 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:47.304 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.304 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.305 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:54:47.305 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:47.306 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:47.306 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:47.307 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:47.307 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:47.308 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.591864+0000", 2022-06-17T12:54:47.308 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:47.308 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.309 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:47.309 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:54:47.310 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:47.310 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:47.311 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:47.311 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:47.311 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:47.312 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.607045+0000", 2022-06-17T12:54:47.312 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:47.312 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.312 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:54:47.313 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap": [], 2022-06-17T12:54:47.313 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap_items": [], 2022-06-17T12:54:47.313 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_temp": [], 2022-06-17T12:54:47.313 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_temp": [], 2022-06-17T12:54:47.313 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklist": {}, 2022-06-17T12:54:47.314 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profiles": { 2022-06-17T12:54:47.314 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs_data": { 2022-06-17T12:54:47.315 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-device-class": "", 2022-06-17T12:54:47.315 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:54:47.316 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-root": "default", 2022-06-17T12:54:47.316 INFO:teuthology.orchestra.run.smithi036.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-17T12:54:47.317 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:54:47.317 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "2", 2022-06-17T12:54:47.317 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:54:47.318 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:54:47.318 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van", 2022-06-17T12:54:47.319 INFO:teuthology.orchestra.run.smithi036.stdout: "w": "8" 2022-06-17T12:54:47.319 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.320 INFO:teuthology.orchestra.run.smithi036.stdout: "default": { 2022-06-17T12:54:47.320 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:54:47.321 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:54:47.321 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "1", 2022-06-17T12:54:47.321 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:54:47.322 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:54:47.322 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van" 2022-06-17T12:54:47.322 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:47.323 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:47.323 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps_queue": [], 2022-06-17T12:54:47.324 INFO:teuthology.orchestra.run.smithi036.stdout: "new_removed_snaps": [], 2022-06-17T12:54:47.324 INFO:teuthology.orchestra.run.smithi036.stdout: "new_purged_snaps": [], 2022-06-17T12:54:47.324 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_node_flags": {}, 2022-06-17T12:54:47.325 INFO:teuthology.orchestra.run.smithi036.stdout: "device_class_flags": {} 2022-06-17T12:54:47.325 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:54:47.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.968+0000 7f8e0f734700 1 -- 172.21.15.36:0/4100873735 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8df0053af0 msgr2=0x7f8df0055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:47.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.968+0000 7f8e0f734700 1 --2- 172.21.15.36:0/4100873735 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8df0053af0 0x7f8df0055fb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f8df8018000 tx=0x7f8df8014000).stop 2022-06-17T12:54:47.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.968+0000 7f8e0f734700 1 -- 172.21.15.36:0/4100873735 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e080ac340 msgr2=0x7f8e0805fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:47.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.968+0000 7f8e0f734700 1 --2- 172.21.15.36:0/4100873735 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e080ac340 0x7f8e0805fd70 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f8e04001110 tx=0x7f8e04007aa0).stop 2022-06-17T12:54:47.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.968+0000 7f8e0f734700 1 -- 172.21.15.36:0/4100873735 shutdown_connections 2022-06-17T12:54:47.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.968+0000 7f8e0f734700 1 --2- 172.21.15.36:0/4100873735 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8df0053af0 0x7f8df0055fb0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:47.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.968+0000 7f8e0f734700 1 --2- 172.21.15.36:0/4100873735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e080af7f0 0x7f8e080a9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:47.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.968+0000 7f8e0f734700 1 --2- 172.21.15.36:0/4100873735 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e080ad140 0x7f8e080a9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:47.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.968+0000 7f8e0f734700 1 --2- 172.21.15.36:0/4100873735 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e080ac340 0x7f8e0805fd70 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:47.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.968+0000 7f8e0f734700 1 -- 172.21.15.36:0/4100873735 >> 172.21.15.36:0/4100873735 conn(0x7f8e080b5430 msgr2=0x7f8e080b3260 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:47.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.968+0000 7f8e0f734700 1 -- 172.21.15.36:0/4100873735 shutdown_connections 2022-06-17T12:54:47.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:46.968+0000 7f8e0f734700 1 -- 172.21.15.36:0/4100873735 wait complete. 2022-06-17T12:54:47.337 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:54:47.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.464+0000 7efd074cf700 1 Processor -- start 2022-06-17T12:54:47.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.464+0000 7efd074cf700 1 -- start start 2022-06-17T12:54:47.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.464+0000 7efd074cf700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7efd00062470 0x7efd000b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:47.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.464+0000 7efd074cf700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd000b83f0 con 0x7efd000b4820 2022-06-17T12:54:47.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.464+0000 7efd074cf700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd000b1ff0 con 0x7efd00143a90 2022-06-17T12:54:47.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.464+0000 7efd074cf700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd00143f00 con 0x7efd00062470 2022-06-17T12:54:47.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.464+0000 7efd05ccc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7efd00062470 0x7efd000b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:47.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.464+0000 7efd064cd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7efd000b4820 0x7efd000b7ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54866/0 (socket says 172.21.15.36:54866) 2022-06-17T12:54:47.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.464+0000 7efd05ccc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7efd00062470 0x7efd000b4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37336/0 (socket says 172.21.15.36:37336) 2022-06-17T12:54:47.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.464+0000 7efd064cd700 1 -- 172.21.15.36:0/782163367 learned_addr learned my addr 172.21.15.36:0/782163367 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:47.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.464+0000 7efd05ccc700 1 -- 172.21.15.36:0/782163367 >> v1:172.21.15.70:6789/0 conn(0x7efd00143a90 legacy=0x7efd00144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:47.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.464+0000 7efd05ccc700 1 -- 172.21.15.36:0/782163367 >> v1:172.21.15.36:6789/0 conn(0x7efd000b4820 legacy=0x7efd000b7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:47.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.464+0000 7efd05ccc700 1 -- 172.21.15.36:0/782163367 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd001430b0 con 0x7efd00062470 2022-06-17T12:54:47.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.468+0000 7efd05ccc700 1 --2- 172.21.15.36:0/782163367 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd00062470 0x7efd000b4c00 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 rx=0x7efcfc002f70 tx=0x7efcfc006df0).ready entity=mon.0 client_cookie=57736142d220c6ab server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:47.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.468+0000 7efd054cb700 1 -- 172.21.15.36:0/782163367 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efcfc00d610 con 0x7efd00062470 2022-06-17T12:54:47.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.468+0000 7efd054cb700 1 -- 172.21.15.36:0/782163367 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcfc000e90 con 0x7efd00062470 2022-06-17T12:54:47.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.468+0000 7efd054cb700 1 -- 172.21.15.36:0/782163367 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efcfc00d960 con 0x7efd00062470 2022-06-17T12:54:47.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.468+0000 7efd074cf700 1 -- 172.21.15.36:0/782163367 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd00062470 msgr2=0x7efd000b4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:47.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.468+0000 7efd074cf700 1 --2- 172.21.15.36:0/782163367 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd00062470 0x7efd000b4c00 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 rx=0x7efcfc002f70 tx=0x7efcfc006df0).stop 2022-06-17T12:54:47.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.468+0000 7efd074cf700 1 -- 172.21.15.36:0/782163367 shutdown_connections 2022-06-17T12:54:47.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.468+0000 7efd074cf700 1 --2- 172.21.15.36:0/782163367 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd00062470 0x7efd000b4c00 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:47.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.468+0000 7efd074cf700 1 -- 172.21.15.36:0/782163367 >> 172.21.15.36:0/782163367 conn(0x7efd0005c550 msgr2=0x7efd000b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:47.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.468+0000 7efd074cf700 1 -- 172.21.15.36:0/782163367 shutdown_connections 2022-06-17T12:54:47.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.468+0000 7efd074cf700 1 -- 172.21.15.36:0/782163367 wait complete. 2022-06-17T12:54:47.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.472+0000 7efd074cf700 1 Processor -- start 2022-06-17T12:54:47.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.472+0000 7efd074cf700 1 -- start start 2022-06-17T12:54:47.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.472+0000 7efd074cf700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efd00062470 0x7efd0013df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:47.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.472+0000 7efd074cf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd000b4820 0x7efd0013e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:47.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.472+0000 7efd074cf700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd00143a90 0x7efd00151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:47.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.472+0000 7efd074cf700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd000669b0 con 0x7efd00143a90 2022-06-17T12:54:47.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.472+0000 7efd074cf700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd00144550 con 0x7efd000b4820 2022-06-17T12:54:47.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.472+0000 7efd074cf700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd00152f10 con 0x7efd00062470 2022-06-17T12:54:47.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.472+0000 7efd06cce700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd00143a90 0x7efd00151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:47.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.472+0000 7efd064cd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efd00062470 0x7efd0013df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:47.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.472+0000 7efd05ccc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd000b4820 0x7efd0013e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:47.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.472+0000 7efd06cce700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd00143a90 0x7efd00151650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37346/0 (socket says 172.21.15.36:37346) 2022-06-17T12:54:47.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.472+0000 7efd06cce700 1 -- 172.21.15.36:0/2965999272 learned_addr learned my addr 172.21.15.36:0/2965999272 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:47.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.472+0000 7efd06cce700 1 -- 172.21.15.36:0/2965999272 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efd00062470 msgr2=0x7efd0013df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:47.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.472+0000 7efd06cce700 1 --2- 172.21.15.36:0/2965999272 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efd00062470 0x7efd0013df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:47.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.472+0000 7efd06cce700 1 -- 172.21.15.36:0/2965999272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd000b4820 msgr2=0x7efd0013e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:47.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.472+0000 7efd06cce700 1 --2- 172.21.15.36:0/2965999272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd000b4820 0x7efd0013e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:47.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.472+0000 7efd06cce700 1 -- 172.21.15.36:0/2965999272 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc00b180 con 0x7efd00143a90 2022-06-17T12:54:47.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.472+0000 7efd06cce700 1 --2- 172.21.15.36:0/2965999272 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd00143a90 0x7efd00151650 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7efd0005ce00 tx=0x7efcf8001330).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:47.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.472+0000 7efcef7fe700 1 -- 172.21.15.36:0/2965999272 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efcf800f470 con 0x7efd00143a90 2022-06-17T12:54:47.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.472+0000 7efcef7fe700 1 -- 172.21.15.36:0/2965999272 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf800f5d0 con 0x7efd00143a90 2022-06-17T12:54:47.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.472+0000 7efd074cf700 1 -- 172.21.15.36:0/2965999272 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd00153180 con 0x7efd00143a90 2022-06-17T12:54:47.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.476+0000 7efcef7fe700 1 -- 172.21.15.36:0/2965999272 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efcf800f930 con 0x7efd00143a90 2022-06-17T12:54:47.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.476+0000 7efd074cf700 1 -- 172.21.15.36:0/2965999272 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd00144a30 con 0x7efd00143a90 2022-06-17T12:54:47.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.476+0000 7efd074cf700 1 -- 172.21.15.36:0/2965999272 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd001446c0 con 0x7efd00143a90 2022-06-17T12:54:47.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.476+0000 7efcef7fe700 1 -- 172.21.15.36:0/2965999272 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7efcf8019030 con 0x7efd00143a90 2022-06-17T12:54:47.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.476+0000 7efcef7fe700 1 --2- 172.21.15.36:0/2965999272 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efce00539b0 0x7efce0055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:47.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.476+0000 7efd064cd700 1 --2- 172.21.15.36:0/2965999272 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efce00539b0 0x7efce0055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:47.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.476+0000 7efcef7fe700 1 -- 172.21.15.36:0/2965999272 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6420+0+0 (secure 0 0 0) 0x7efcf806b670 con 0x7efd00143a90 2022-06-17T12:54:47.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.480+0000 7efd064cd700 1 --2- 172.21.15.36:0/2965999272 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efce00539b0 0x7efce0055e70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7efcf000d000 tx=0x7efcf0009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:47.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.484+0000 7efcef7fe700 1 -- 172.21.15.36:0/2965999272 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efcf8042360 con 0x7efd00143a90 2022-06-17T12:54:47.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.652+0000 7efd074cf700 1 -- 172.21.15.36:0/2965999272 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd00143350 con 0x7efd00143a90 2022-06-17T12:54:47.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.652+0000 7efcef7fe700 1 -- 172.21.15.36:0/2965999272 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+2137 (secure 0 0 0) 0x7efcf8045980 con 0x7efd00143a90 2022-06-17T12:54:47.640 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:47.641 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":47,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/138643404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":138643404},{"type":"v1","addr":"172.21.15.70:6833","nonce":138643404}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":46}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:54:35.949730+0000","modified":"2022-06-17T12:54:36.953427+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7074},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7074":{"gid":7074,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/3267343301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3267343301},{"type":"v1","addr":"172.21.15.36:6835","nonce":3267343301}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-06-17T12:54:47.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.652+0000 7efd074cf700 1 -- 172.21.15.36:0/2965999272 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efce00539b0 msgr2=0x7efce0055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:47.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.652+0000 7efd074cf700 1 --2- 172.21.15.36:0/2965999272 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efce00539b0 0x7efce0055e70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7efcf000d000 tx=0x7efcf0009040).stop 2022-06-17T12:54:47.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.652+0000 7efd074cf700 1 -- 172.21.15.36:0/2965999272 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd00143a90 msgr2=0x7efd00151650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:47.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.652+0000 7efd074cf700 1 --2- 172.21.15.36:0/2965999272 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd00143a90 0x7efd00151650 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7efd0005ce00 tx=0x7efcf8001330).stop 2022-06-17T12:54:47.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.652+0000 7efd074cf700 1 -- 172.21.15.36:0/2965999272 shutdown_connections 2022-06-17T12:54:47.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.652+0000 7efd074cf700 1 --2- 172.21.15.36:0/2965999272 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efce00539b0 0x7efce0055e70 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:47.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.652+0000 7efd074cf700 1 --2- 172.21.15.36:0/2965999272 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efd00143a90 0x7efd00151650 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:47.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.652+0000 7efd074cf700 1 --2- 172.21.15.36:0/2965999272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd000b4820 0x7efd0013e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:47.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.652+0000 7efd074cf700 1 --2- 172.21.15.36:0/2965999272 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efd00062470 0x7efd0013df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:47.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.652+0000 7efd074cf700 1 -- 172.21.15.36:0/2965999272 >> 172.21.15.36:0/2965999272 conn(0x7efd0005c550 msgr2=0x7efd0013a630 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:47.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.652+0000 7efd074cf700 1 -- 172.21.15.36:0/2965999272 shutdown_connections 2022-06-17T12:54:47.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.652+0000 7efd074cf700 1 -- 172.21.15.36:0/2965999272 wait complete. 2022-06-17T12:54:47.648 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 47 2022-06-17T12:54:47.652 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-17T12:54:47.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.768+0000 7feeb096a700 1 Processor -- start 2022-06-17T12:54:47.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.768+0000 7feeb096a700 1 -- start start 2022-06-17T12:54:47.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.768+0000 7feeb096a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeac0b2200 0x7feeac0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:47.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.768+0000 7feeb096a700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeac0b25e0 con 0x7feeac0af8d0 2022-06-17T12:54:47.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.768+0000 7feeb096a700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeac14b6f0 con 0x7feeac0b2cd0 2022-06-17T12:54:47.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.768+0000 7feeb096a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeac0ad790 con 0x7feeac0b2200 2022-06-17T12:54:47.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.768+0000 7feeaa59c700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7feeac0af8d0 0x7feeac14a8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54882/0 (socket says 172.21.15.36:54882) 2022-06-17T12:54:47.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.768+0000 7feeaa59c700 1 -- 172.21.15.36:0/3934603953 learned_addr learned my addr 172.21.15.36:0/3934603953 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:47.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.768+0000 7feeaad9d700 1 --2- 172.21.15.36:0/3934603953 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeac0b2200 0x7feeac0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:47.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.768+0000 7feea9d9b700 1 -- 172.21.15.36:0/3934603953 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688136007 0 0) 0x7feeac0b25e0 con 0x7feeac0af8d0 2022-06-17T12:54:47.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.768+0000 7feea9d9b700 1 -- 172.21.15.36:0/3934603953 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee8c003260 con 0x7feeac0af8d0 2022-06-17T12:54:47.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.768+0000 7feea9d9b700 1 -- 172.21.15.36:0/3934603953 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1371586928 0 0) 0x7feeac14b6f0 con 0x7feeac0b2cd0 2022-06-17T12:54:47.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.768+0000 7feea9d9b700 1 -- 172.21.15.36:0/3934603953 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feeac0b25e0 con 0x7feeac0b2cd0 2022-06-17T12:54:47.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.772+0000 7feeaad9d700 1 -- 172.21.15.36:0/3934603953 >> v1:172.21.15.70:6789/0 conn(0x7feeac0b2cd0 legacy=0x7feeac14afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:47.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.772+0000 7feeaad9d700 1 -- 172.21.15.36:0/3934603953 >> v1:172.21.15.36:6789/0 conn(0x7feeac0af8d0 legacy=0x7feeac14a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:47.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.772+0000 7feeaad9d700 1 -- 172.21.15.36:0/3934603953 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac0aef40 con 0x7feeac0b2200 2022-06-17T12:54:47.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.772+0000 7feeaad9d700 1 --2- 172.21.15.36:0/3934603953 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeac0b2200 0x7feeac0af3e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fee9c002f70 tx=0x7fee9c006df0).ready entity=mon.2 client_cookie=799381c1a00d58a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:47.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.772+0000 7feea9d9b700 1 -- 172.21.15.36:0/3934603953 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee9c00d5c0 con 0x7feeac0b2200 2022-06-17T12:54:47.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.772+0000 7feea9d9b700 1 -- 172.21.15.36:0/3934603953 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9c000e90 con 0x7feeac0b2200 2022-06-17T12:54:47.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.772+0000 7feea9d9b700 1 -- 172.21.15.36:0/3934603953 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee9c00d910 con 0x7feeac0b2200 2022-06-17T12:54:47.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.772+0000 7feeb096a700 1 -- 172.21.15.36:0/3934603953 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeac0b2200 msgr2=0x7feeac0af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:47.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.772+0000 7feeb096a700 1 --2- 172.21.15.36:0/3934603953 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeac0b2200 0x7feeac0af3e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fee9c002f70 tx=0x7fee9c006df0).stop 2022-06-17T12:54:47.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.772+0000 7feeb096a700 1 -- 172.21.15.36:0/3934603953 shutdown_connections 2022-06-17T12:54:47.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.772+0000 7feeb096a700 1 --2- 172.21.15.36:0/3934603953 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeac0b2200 0x7feeac0af3e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:47.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.772+0000 7feeb096a700 1 -- 172.21.15.36:0/3934603953 >> 172.21.15.36:0/3934603953 conn(0x7feeac05f770 msgr2=0x7feeac05fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:47.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.772+0000 7feeb096a700 1 -- 172.21.15.36:0/3934603953 shutdown_connections 2022-06-17T12:54:47.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.772+0000 7feeb096a700 1 -- 172.21.15.36:0/3934603953 wait complete. 2022-06-17T12:54:47.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.772+0000 7feeb096a700 1 Processor -- start 2022-06-17T12:54:47.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.772+0000 7feeb096a700 1 -- start start 2022-06-17T12:54:47.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.776+0000 7feeb096a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feeac0af8d0 0x7feeac0abb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:47.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.776+0000 7feeb096a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeac0b2200 0x7feeac0aa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:47.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.776+0000 7feeb096a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feeac0b2cd0 0x7feeac144270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:47.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.776+0000 7feeb096a700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac14a6e0 con 0x7feeac0b2cd0 2022-06-17T12:54:47.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.776+0000 7feeb096a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac0aaf90 con 0x7feeac0af8d0 2022-06-17T12:54:47.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.776+0000 7feeb096a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeac14a330 con 0x7feeac0b2200 2022-06-17T12:54:47.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.776+0000 7feeab59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feeac0b2cd0 0x7feeac144270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:47.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.776+0000 7feeab59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feeac0b2cd0 0x7feeac144270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37358/0 (socket says 172.21.15.36:37358) 2022-06-17T12:54:47.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.776+0000 7feeab59e700 1 -- 172.21.15.36:0/2898244813 learned_addr learned my addr 172.21.15.36:0/2898244813 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:47.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.776+0000 7feeaad9d700 1 --2- 172.21.15.36:0/2898244813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feeac0af8d0 0x7feeac0abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:47.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.776+0000 7feeaa59c700 1 --2- 172.21.15.36:0/2898244813 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeac0b2200 0x7feeac0aa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:47.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.776+0000 7feeab59e700 1 -- 172.21.15.36:0/2898244813 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeac0b2200 msgr2=0x7feeac0aa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:47.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.776+0000 7feeab59e700 1 --2- 172.21.15.36:0/2898244813 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeac0b2200 0x7feeac0aa5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:47.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.776+0000 7feeab59e700 1 -- 172.21.15.36:0/2898244813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feeac0af8d0 msgr2=0x7feeac0abb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:47.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.776+0000 7feeab59e700 1 --2- 172.21.15.36:0/2898244813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feeac0af8d0 0x7feeac0abb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:47.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.776+0000 7feeab59e700 1 -- 172.21.15.36:0/2898244813 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9c00b180 con 0x7feeac0b2cd0 2022-06-17T12:54:47.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.776+0000 7feeaa59c700 1 --2- 172.21.15.36:0/2898244813 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeac0b2200 0x7feeac0aa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:47.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.776+0000 7feeab59e700 1 --2- 172.21.15.36:0/2898244813 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feeac0b2cd0 0x7feeac144270 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 rx=0x7feea4001490 tx=0x7feea4007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:47.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.776+0000 7feea3fff700 1 -- 172.21.15.36:0/2898244813 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feea4002310 con 0x7feeac0b2cd0 2022-06-17T12:54:47.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.776+0000 7feea3fff700 1 -- 172.21.15.36:0/2898244813 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea4002470 con 0x7feeac0b2cd0 2022-06-17T12:54:47.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.776+0000 7feeb096a700 1 -- 172.21.15.36:0/2898244813 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeac14b2b0 con 0x7feeac0b2cd0 2022-06-17T12:54:47.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.776+0000 7feea3fff700 1 -- 172.21.15.36:0/2898244813 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feea40017a0 con 0x7feeac0b2cd0 2022-06-17T12:54:47.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.776+0000 7feeb096a700 1 -- 172.21.15.36:0/2898244813 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeac0aacf0 con 0x7feeac0b2cd0 2022-06-17T12:54:47.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.776+0000 7feeb096a700 1 -- 172.21.15.36:0/2898244813 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeac0ab150 con 0x7feeac0b2cd0 2022-06-17T12:54:47.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.780+0000 7feea3fff700 1 -- 172.21.15.36:0/2898244813 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7feea4010030 con 0x7feeac0b2cd0 2022-06-17T12:54:47.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.780+0000 7feea3fff700 1 --2- 172.21.15.36:0/2898244813 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fee90053a60 0x7fee90055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:47.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.780+0000 7feea3fff700 1 -- 172.21.15.36:0/2898244813 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6420+0+0 (secure 0 0 0) 0x7feea406be90 con 0x7feeac0b2cd0 2022-06-17T12:54:47.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.780+0000 7feeaad9d700 1 --2- 172.21.15.36:0/2898244813 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fee90053a60 0x7fee90055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:47.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.780+0000 7feeaad9d700 1 --2- 172.21.15.36:0/2898244813 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fee90053a60 0x7fee90055f20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7fee9c01a000 tx=0x7fee9c014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:47.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.784+0000 7feea3fff700 1 -- 172.21.15.36:0/2898244813 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7feea4042b00 con 0x7feeac0b2cd0 2022-06-17T12:54:47.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:47.948+0000 7feeb096a700 1 -- 172.21.15.36:0/2898244813 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7feeac0b1950 con 0x7feeac0b2cd0 2022-06-17T12:54:48.535 INFO:tasks.ceph.mon.a.smithi036.stderr:2022-06-17T12:54:48.548+0000 7fb7a2649700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-17T12:54:48.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.560+0000 7feea3fff700 1 -- 172.21.15.36:0/2898244813 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) v1 ==== 161+0+0 (secure 0 0 0) 0x7feea4046120 con 0x7feeac0b2cd0 2022-06-17T12:54:48.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.564+0000 7feeb096a700 1 -- 172.21.15.36:0/2898244813 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fee90053a60 msgr2=0x7fee90055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:48.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.564+0000 7feeb096a700 1 --2- 172.21.15.36:0/2898244813 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fee90053a60 0x7fee90055f20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7fee9c01a000 tx=0x7fee9c014000).stop 2022-06-17T12:54:48.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.564+0000 7feeb096a700 1 -- 172.21.15.36:0/2898244813 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feeac0b2cd0 msgr2=0x7feeac144270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:48.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.564+0000 7feeb096a700 1 --2- 172.21.15.36:0/2898244813 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feeac0b2cd0 0x7feeac144270 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 rx=0x7feea4001490 tx=0x7feea4007dd0).stop 2022-06-17T12:54:48.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.564+0000 7feeb096a700 1 -- 172.21.15.36:0/2898244813 shutdown_connections 2022-06-17T12:54:48.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.568+0000 7feeb096a700 1 --2- 172.21.15.36:0/2898244813 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fee90053a60 0x7fee90055f20 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:48.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.568+0000 7feeb096a700 1 --2- 172.21.15.36:0/2898244813 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feeac0b2cd0 0x7feeac144270 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:48.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.568+0000 7feeb096a700 1 --2- 172.21.15.36:0/2898244813 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeac0b2200 0x7feeac0aa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:48.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.568+0000 7feeb096a700 1 --2- 172.21.15.36:0/2898244813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feeac0af8d0 0x7feeac0abb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:48.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.568+0000 7feeb096a700 1 -- 172.21.15.36:0/2898244813 >> 172.21.15.36:0/2898244813 conn(0x7feeac05f770 msgr2=0x7feeac0ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:48.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.568+0000 7feeb096a700 1 -- 172.21.15.36:0/2898244813 shutdown_connections 2022-06-17T12:54:48.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.568+0000 7feeb096a700 1 -- 172.21.15.36:0/2898244813 wait complete. 2022-06-17T12:54:48.564 INFO:teuthology.orchestra.run.smithi036.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-17T12:54:48.574 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:54:48.584 INFO:tasks.ceph.mds.a.smithi036.stdout:starting mds.a at 2022-06-17T12:54:48.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.692+0000 7f796519a700 1 Processor -- start 2022-06-17T12:54:48.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.692+0000 7f796519a700 1 -- start start 2022-06-17T12:54:48.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.692+0000 7f796519a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79600aef60 0x7f79600b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:48.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.692+0000 7f796519a700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f79600ba2e0 0x7f79601442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:48.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f796519a700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79600af4b0 con 0x7f796013cb70 2022-06-17T12:54:48.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f796519a700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7960143da0 con 0x7f79600ba2e0 2022-06-17T12:54:48.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f796519a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7960142f50 con 0x7f79600aef60 2022-06-17T12:54:48.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f795ffff700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f796013cb70 0x7f7960143690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:50638/0 (socket says 172.21.15.36:50638) 2022-06-17T12:54:48.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f795ffff700 1 -- 172.21.15.36:0/3113822753 learned_addr learned my addr 172.21.15.36:0/3113822753 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:48.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f7964999700 1 --2- 172.21.15.36:0/3113822753 >> v2:172.21.15.70:3300/0 conn(0x7f79600ba2e0 0x7f79601442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:48.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f795f7fe700 1 --2- 172.21.15.36:0/3113822753 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79600aef60 0x7f79600b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:48.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f795effd700 1 -- 172.21.15.36:0/3113822753 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 602009658 0 0) 0x7f79600af4b0 con 0x7f796013cb70 2022-06-17T12:54:48.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f795effd700 1 -- 172.21.15.36:0/3113822753 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f794c003260 con 0x7f796013cb70 2022-06-17T12:54:48.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f7964999700 1 -- 172.21.15.36:0/3113822753 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79600aef60 msgr2=0x7f79600b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:48.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f7964999700 1 --2- 172.21.15.36:0/3113822753 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79600aef60 0x7f79600b9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:48.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f7964999700 1 -- 172.21.15.36:0/3113822753 >> v1:172.21.15.70:6789/0 conn(0x7f796013cb70 legacy=0x7f7960143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:48.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f7964999700 1 -- 172.21.15.36:0/3113822753 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79601431f0 con 0x7f79600ba2e0 2022-06-17T12:54:48.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f7964999700 1 --2- 172.21.15.36:0/3113822753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79600ba2e0 0x7f79601442a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f7950002f70 tx=0x7f7950006df0).ready entity=mon.1 client_cookie=a65c9af2fa11db14 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:48.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f795effd700 1 -- 172.21.15.36:0/3113822753 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f795000d4a0 con 0x7f79600ba2e0 2022-06-17T12:54:48.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f795effd700 1 -- 172.21.15.36:0/3113822753 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7950000e90 con 0x7f79600ba2e0 2022-06-17T12:54:48.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f795effd700 1 -- 172.21.15.36:0/3113822753 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f795000d7f0 con 0x7f79600ba2e0 2022-06-17T12:54:48.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f796519a700 1 -- 172.21.15.36:0/3113822753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79600ba2e0 msgr2=0x7f79601442a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:48.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f796519a700 1 --2- 172.21.15.36:0/3113822753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79600ba2e0 0x7f79601442a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f7950002f70 tx=0x7f7950006df0).stop 2022-06-17T12:54:48.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f796519a700 1 -- 172.21.15.36:0/3113822753 shutdown_connections 2022-06-17T12:54:48.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f796519a700 1 --2- 172.21.15.36:0/3113822753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79600ba2e0 0x7f79601442a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:48.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f796519a700 1 --2- 172.21.15.36:0/3113822753 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79600aef60 0x7f79600b9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:48.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f796519a700 1 -- 172.21.15.36:0/3113822753 >> 172.21.15.36:0/3113822753 conn(0x7f796005c550 msgr2=0x7f79600b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:48.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f796519a700 1 -- 172.21.15.36:0/3113822753 shutdown_connections 2022-06-17T12:54:48.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f796519a700 1 -- 172.21.15.36:0/3113822753 wait complete. 2022-06-17T12:54:48.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f796519a700 1 Processor -- start 2022-06-17T12:54:48.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.696+0000 7f796519a700 1 -- start start 2022-06-17T12:54:48.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.700+0000 7f796519a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f79600aef60 0x7f79600afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:48.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.700+0000 7f796519a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79600ba2e0 0x7f79601453b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:48.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.700+0000 7f796519a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f796013cb70 0x7f79600b2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:48.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.700+0000 7f796519a700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79600b1530 con 0x7f79600aef60 2022-06-17T12:54:48.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.700+0000 7f796519a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79600af680 con 0x7f796013cb70 2022-06-17T12:54:48.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.700+0000 7f796519a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79600b0a80 con 0x7f79600ba2e0 2022-06-17T12:54:48.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.700+0000 7f795ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f79600aef60 0x7f79600afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:48.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.700+0000 7f795ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f79600aef60 0x7f79600afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37380/0 (socket says 172.21.15.36:37380) 2022-06-17T12:54:48.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.700+0000 7f795ffff700 1 -- 172.21.15.36:0/1108894156 learned_addr learned my addr 172.21.15.36:0/1108894156 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:48.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.700+0000 7f795f7fe700 1 --2- 172.21.15.36:0/1108894156 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79600ba2e0 0x7f79601453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:48.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.700+0000 7f7964999700 1 --2- 172.21.15.36:0/1108894156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f796013cb70 0x7f79600b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:48.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.700+0000 7f795ffff700 1 -- 172.21.15.36:0/1108894156 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79600ba2e0 msgr2=0x7f79601453b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:48.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.700+0000 7f795ffff700 1 --2- 172.21.15.36:0/1108894156 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79600ba2e0 0x7f79601453b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:48.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.700+0000 7f795ffff700 1 -- 172.21.15.36:0/1108894156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f796013cb70 msgr2=0x7f79600b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:48.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.700+0000 7f795ffff700 1 --2- 172.21.15.36:0/1108894156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f796013cb70 0x7f79600b2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:48.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.700+0000 7f795ffff700 1 -- 172.21.15.36:0/1108894156 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795000b180 con 0x7f79600aef60 2022-06-17T12:54:48.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.700+0000 7f795ffff700 1 --2- 172.21.15.36:0/1108894156 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f79600aef60 0x7f79600afab0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 rx=0x7f79600b0c10 tx=0x7f7958007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:48.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.700+0000 7f795d7fa700 1 -- 172.21.15.36:0/1108894156 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7958005ba0 con 0x7f79600aef60 2022-06-17T12:54:48.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.700+0000 7f795d7fa700 1 -- 172.21.15.36:0/1108894156 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7958005d00 con 0x7f79600aef60 2022-06-17T12:54:48.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.700+0000 7f796519a700 1 -- 172.21.15.36:0/1108894156 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79600b0cf0 con 0x7f79600aef60 2022-06-17T12:54:48.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.700+0000 7f795d7fa700 1 -- 172.21.15.36:0/1108894156 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79580072c0 con 0x7f79600aef60 2022-06-17T12:54:48.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.700+0000 7f796519a700 1 -- 172.21.15.36:0/1108894156 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f796013fb10 con 0x7f79600aef60 2022-06-17T12:54:48.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.704+0000 7f796519a700 1 -- 172.21.15.36:0/1108894156 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7960057410 con 0x7f79600aef60 2022-06-17T12:54:48.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.704+0000 7f795d7fa700 1 -- 172.21.15.36:0/1108894156 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f795801a070 con 0x7f79600aef60 2022-06-17T12:54:48.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.704+0000 7f795d7fa700 1 --2- 172.21.15.36:0/1108894156 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7940053a40 0x7f7940055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:48.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.704+0000 7f795d7fa700 1 -- 172.21.15.36:0/1108894156 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f795806b6f0 con 0x7f79600aef60 2022-06-17T12:54:48.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.704+0000 7f795f7fe700 1 --2- 172.21.15.36:0/1108894156 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7940053a40 0x7f7940055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:48.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.704+0000 7f795f7fe700 1 --2- 172.21.15.36:0/1108894156 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7940053a40 0x7f7940055f00 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f795400b000 tx=0x7f7954006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:48.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.708+0000 7f795d7fa700 1 -- 172.21.15.36:0/1108894156 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7958042400 con 0x7f79600aef60 2022-06-17T12:54:48.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.868+0000 7f796519a700 1 -- 172.21.15.36:0/1108894156 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7960067bf0 con 0x7f79600aef60 2022-06-17T12:54:48.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.868+0000 7f795d7fa700 1 -- 172.21.15.36:0/1108894156 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+1776 (secure 0 0 0) 0x7f7958045a20 con 0x7f79600aef60 2022-06-17T12:54:48.858 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:48.858 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":48,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/138643404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":138643404},{"type":"v1","addr":"172.21.15.70:6833","nonce":138643404}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":46}],"filesystems":[{"mdsmap":{"epoch":48,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:54:35.949730+0000","modified":"2022-06-17T12:54:48.552932+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-06-17T12:54:48.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.872+0000 7f796519a700 1 -- 172.21.15.36:0/1108894156 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7940053a40 msgr2=0x7f7940055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:48.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.872+0000 7f796519a700 1 --2- 172.21.15.36:0/1108894156 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7940053a40 0x7f7940055f00 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f795400b000 tx=0x7f7954006df0).stop 2022-06-17T12:54:48.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.872+0000 7f796519a700 1 -- 172.21.15.36:0/1108894156 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f79600aef60 msgr2=0x7f79600afab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:48.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.872+0000 7f796519a700 1 --2- 172.21.15.36:0/1108894156 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f79600aef60 0x7f79600afab0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 rx=0x7f79600b0c10 tx=0x7f7958007e50).stop 2022-06-17T12:54:48.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.872+0000 7f796519a700 1 -- 172.21.15.36:0/1108894156 shutdown_connections 2022-06-17T12:54:48.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.872+0000 7f796519a700 1 --2- 172.21.15.36:0/1108894156 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7940053a40 0x7f7940055f00 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:48.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.872+0000 7f796519a700 1 --2- 172.21.15.36:0/1108894156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f796013cb70 0x7f79600b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:48.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.872+0000 7f796519a700 1 --2- 172.21.15.36:0/1108894156 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79600ba2e0 0x7f79601453b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:48.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.872+0000 7f796519a700 1 --2- 172.21.15.36:0/1108894156 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f79600aef60 0x7f79600afab0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:48.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.872+0000 7f796519a700 1 -- 172.21.15.36:0/1108894156 >> 172.21.15.36:0/1108894156 conn(0x7f796005c550 msgr2=0x7f796013c440 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:48.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.872+0000 7f796519a700 1 -- 172.21.15.36:0/1108894156 shutdown_connections 2022-06-17T12:54:48.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:48.872+0000 7f796519a700 1 -- 172.21.15.36:0/1108894156 wait complete. 2022-06-17T12:54:48.864 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 48 2022-06-17T12:54:48.872 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-17T12:54:49.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.048+0000 7f9b44e24700 1 Processor -- start 2022-06-17T12:54:49.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.048+0000 7f9b44e24700 1 -- start start 2022-06-17T12:54:49.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.048+0000 7f9b44e24700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9b40060e40 0x7f9b401451e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:49.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.048+0000 7f9b44e24700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b400b1830 con 0x7f9b400aed10 2022-06-17T12:54:49.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.048+0000 7f9b44e24700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b400b0440 con 0x7f9b400b1360 2022-06-17T12:54:49.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.048+0000 7f9b44e24700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b401423e0 con 0x7f9b40060e40 2022-06-17T12:54:49.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.048+0000 7f9b3e59c700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f9b400aed10 0x7f9b401425f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54918/0 (socket says 172.21.15.36:54918) 2022-06-17T12:54:49.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.048+0000 7f9b3e59c700 1 -- 172.21.15.36:0/2235272574 learned_addr learned my addr 172.21.15.36:0/2235272574 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:49.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.048+0000 7f9b3dd9b700 1 --2- 172.21.15.36:0/2235272574 >> v2:172.21.15.70:3300/0 conn(0x7f9b40060e40 0x7f9b401451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:49.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.048+0000 7f9b3d59a700 1 -- 172.21.15.36:0/2235272574 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2100097076 0 0) 0x7f9b400b1830 con 0x7f9b400aed10 2022-06-17T12:54:49.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.048+0000 7f9b3d59a700 1 -- 172.21.15.36:0/2235272574 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b24003260 con 0x7f9b400aed10 2022-06-17T12:54:49.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.048+0000 7f9b3dd9b700 1 -- 172.21.15.36:0/2235272574 >> v1:172.21.15.70:6789/0 conn(0x7f9b400b1360 legacy=0x7f9b40066a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:49.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.048+0000 7f9b3d59a700 1 -- 172.21.15.36:0/2235272574 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4191806606 0 0) 0x7f9b24003260 con 0x7f9b400aed10 2022-06-17T12:54:49.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.048+0000 7f9b3dd9b700 1 -- 172.21.15.36:0/2235272574 >> v1:172.21.15.36:6789/0 conn(0x7f9b400aed10 legacy=0x7f9b401425f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:49.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.048+0000 7f9b3dd9b700 1 -- 172.21.15.36:0/2235272574 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b400af0f0 con 0x7f9b40060e40 2022-06-17T12:54:49.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.048+0000 7f9b3dd9b700 1 --2- 172.21.15.36:0/2235272574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b40060e40 0x7f9b401451e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f9b34001800 tx=0x7f9b30006df0).ready entity=mon.1 client_cookie=5d7ba3924a72a656 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:49.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.052+0000 7f9b3d59a700 1 -- 172.21.15.36:0/2235272574 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b34001660 con 0x7f9b40060e40 2022-06-17T12:54:49.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.052+0000 7f9b3d59a700 1 -- 172.21.15.36:0/2235272574 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b30000e90 con 0x7f9b40060e40 2022-06-17T12:54:49.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.052+0000 7f9b3d59a700 1 -- 172.21.15.36:0/2235272574 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b3000d4f0 con 0x7f9b40060e40 2022-06-17T12:54:49.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.052+0000 7f9b44e24700 1 -- 172.21.15.36:0/2235272574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b40060e40 msgr2=0x7f9b401451e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:49.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.052+0000 7f9b44e24700 1 --2- 172.21.15.36:0/2235272574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b40060e40 0x7f9b401451e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f9b34001800 tx=0x7f9b30006df0).stop 2022-06-17T12:54:49.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.052+0000 7f9b44e24700 1 -- 172.21.15.36:0/2235272574 shutdown_connections 2022-06-17T12:54:49.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.052+0000 7f9b44e24700 1 --2- 172.21.15.36:0/2235272574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b40060e40 0x7f9b401451e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:49.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.052+0000 7f9b44e24700 1 -- 172.21.15.36:0/2235272574 >> 172.21.15.36:0/2235272574 conn(0x7f9b4005c550 msgr2=0x7f9b400b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:49.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.052+0000 7f9b44e24700 1 -- 172.21.15.36:0/2235272574 shutdown_connections 2022-06-17T12:54:49.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.052+0000 7f9b44e24700 1 -- 172.21.15.36:0/2235272574 wait complete. 2022-06-17T12:54:49.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.052+0000 7f9b44e24700 1 Processor -- start 2022-06-17T12:54:49.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.052+0000 7f9b44e24700 1 -- start start 2022-06-17T12:54:49.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.052+0000 7f9b44e24700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b40060e40 0x7f9b400a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:49.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.052+0000 7f9b44e24700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9b400aed10 0x7f9b40065ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:49.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.052+0000 7f9b44e24700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9b400b1360 0x7f9b40150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:49.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.052+0000 7f9b44e24700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b401412b0 con 0x7f9b400b1360 2022-06-17T12:54:49.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.056+0000 7f9b44e24700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b400a0450 con 0x7f9b40060e40 2022-06-17T12:54:49.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.056+0000 7f9b44e24700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4009ff10 con 0x7f9b400aed10 2022-06-17T12:54:49.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.056+0000 7f9b3ed9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9b400b1360 0x7f9b40150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:49.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.056+0000 7f9b3ed9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9b400b1360 0x7f9b40150880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37398/0 (socket says 172.21.15.36:37398) 2022-06-17T12:54:49.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.056+0000 7f9b3ed9d700 1 -- 172.21.15.36:0/2585946743 learned_addr learned my addr 172.21.15.36:0/2585946743 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:49.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.056+0000 7f9b3e59c700 1 --2- 172.21.15.36:0/2585946743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b40060e40 0x7f9b400a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:49.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.056+0000 7f9b3dd9b700 1 --2- 172.21.15.36:0/2585946743 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9b400aed10 0x7f9b40065ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:49.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.056+0000 7f9b3ed9d700 1 -- 172.21.15.36:0/2585946743 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9b400aed10 msgr2=0x7f9b40065ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:49.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.056+0000 7f9b3ed9d700 1 --2- 172.21.15.36:0/2585946743 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9b400aed10 0x7f9b40065ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:49.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.056+0000 7f9b3ed9d700 1 -- 172.21.15.36:0/2585946743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b40060e40 msgr2=0x7f9b400a4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:49.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.056+0000 7f9b3ed9d700 1 --2- 172.21.15.36:0/2585946743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b40060e40 0x7f9b400a4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:49.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.056+0000 7f9b3ed9d700 1 -- 172.21.15.36:0/2585946743 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3000b180 con 0x7f9b400b1360 2022-06-17T12:54:49.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.056+0000 7f9b3ed9d700 1 --2- 172.21.15.36:0/2585946743 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9b400b1360 0x7f9b40150880 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 rx=0x7f9b400af4f0 tx=0x7f9b380013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:49.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.056+0000 7f9b2f7fe700 1 -- 172.21.15.36:0/2585946743 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b38006aa0 con 0x7f9b400b1360 2022-06-17T12:54:49.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.056+0000 7f9b2f7fe700 1 -- 172.21.15.36:0/2585946743 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b38006c00 con 0x7f9b400b1360 2022-06-17T12:54:49.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.056+0000 7f9b44e24700 1 -- 172.21.15.36:0/2585946743 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b400a01b0 con 0x7f9b400b1360 2022-06-17T12:54:49.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.056+0000 7f9b2f7fe700 1 -- 172.21.15.36:0/2585946743 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b38007030 con 0x7f9b400b1360 2022-06-17T12:54:49.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.056+0000 7f9b44e24700 1 -- 172.21.15.36:0/2585946743 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b40150470 con 0x7f9b400b1360 2022-06-17T12:54:49.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.056+0000 7f9b44e24700 1 -- 172.21.15.36:0/2585946743 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b40153910 con 0x7f9b400b1360 2022-06-17T12:54:49.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.056+0000 7f9b2f7fe700 1 -- 172.21.15.36:0/2585946743 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f9b38010030 con 0x7f9b400b1360 2022-06-17T12:54:49.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.056+0000 7f9b2f7fe700 1 --2- 172.21.15.36:0/2585946743 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9b1c053ad0 0x7f9b1c055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:49.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.056+0000 7f9b2f7fe700 1 -- 172.21.15.36:0/2585946743 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f9b3806bda0 con 0x7f9b400b1360 2022-06-17T12:54:49.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.060+0000 7f9b3e59c700 1 --2- 172.21.15.36:0/2585946743 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9b1c053ad0 0x7f9b1c055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:49.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.060+0000 7f9b3e59c700 1 --2- 172.21.15.36:0/2585946743 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9b1c053ad0 0x7f9b1c055f90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f9b3400d000 tx=0x7f9b34000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:49.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.060+0000 7f9b2f7fe700 1 -- 172.21.15.36:0/2585946743 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9b38042b50 con 0x7f9b400b1360 2022-06-17T12:54:49.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.228+0000 7f9b44e24700 1 -- 172.21.15.36:0/2585946743 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9b400b1a60 con 0x7f9b400b1360 2022-06-17T12:54:49.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.564+0000 7f9b2f7fe700 1 -- 172.21.15.36:0/2585946743 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9b38046170 con 0x7f9b400b1360 2022-06-17T12:54:49.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.568+0000 7f9b44e24700 1 -- 172.21.15.36:0/2585946743 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9b1c053ad0 msgr2=0x7f9b1c055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:49.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.568+0000 7f9b44e24700 1 --2- 172.21.15.36:0/2585946743 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9b1c053ad0 0x7f9b1c055f90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f9b3400d000 tx=0x7f9b34000bc0).stop 2022-06-17T12:54:49.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.568+0000 7f9b44e24700 1 -- 172.21.15.36:0/2585946743 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9b400b1360 msgr2=0x7f9b40150880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:49.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.568+0000 7f9b44e24700 1 --2- 172.21.15.36:0/2585946743 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9b400b1360 0x7f9b40150880 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 rx=0x7f9b400af4f0 tx=0x7f9b380013a0).stop 2022-06-17T12:54:49.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.568+0000 7f9b44e24700 1 -- 172.21.15.36:0/2585946743 shutdown_connections 2022-06-17T12:54:49.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.568+0000 7f9b44e24700 1 --2- 172.21.15.36:0/2585946743 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9b1c053ad0 0x7f9b1c055f90 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:49.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.568+0000 7f9b44e24700 1 --2- 172.21.15.36:0/2585946743 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9b400b1360 0x7f9b40150880 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:49.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.568+0000 7f9b44e24700 1 --2- 172.21.15.36:0/2585946743 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9b400aed10 0x7f9b40065ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:49.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.568+0000 7f9b44e24700 1 --2- 172.21.15.36:0/2585946743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b40060e40 0x7f9b400a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:49.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.568+0000 7f9b44e24700 1 -- 172.21.15.36:0/2585946743 >> 172.21.15.36:0/2585946743 conn(0x7f9b4005c550 msgr2=0x7f9b400b47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:49.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.568+0000 7f9b44e24700 1 -- 172.21.15.36:0/2585946743 shutdown_connections 2022-06-17T12:54:49.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.568+0000 7f9b44e24700 1 -- 172.21.15.36:0/2585946743 wait complete. 2022-06-17T12:54:49.573 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-17T12:54:49.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.720+0000 7f6c2994b700 1 Processor -- start 2022-06-17T12:54:49.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.720+0000 7f6c2994b700 1 -- start start 2022-06-17T12:54:49.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.720+0000 7f6c2994b700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6c240b8020 0x7f6c2413ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:49.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.720+0000 7f6c2994b700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c240ae5b0 con 0x7f6c240b3c60 2022-06-17T12:54:49.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.720+0000 7f6c2994b700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c240b8400 con 0x7f6c240b2d20 2022-06-17T12:54:49.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.720+0000 7f6c2994b700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c240aef80 con 0x7f6c240b8020 2022-06-17T12:54:49.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.724+0000 7f6c23fff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f6c240b3c60 0x7f6c240afe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54936/0 (socket says 172.21.15.36:54936) 2022-06-17T12:54:49.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.724+0000 7f6c23fff700 1 -- 172.21.15.36:0/4212181445 learned_addr learned my addr 172.21.15.36:0/4212181445 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:49.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.724+0000 7f6c28949700 1 --2- 172.21.15.36:0/4212181445 >> v2:172.21.15.70:3300/0 conn(0x7f6c240b8020 0x7f6c2413ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:49.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.724+0000 7f6c237fe700 1 -- 172.21.15.36:0/4212181445 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 390409800 0 0) 0x7f6c240ae5b0 con 0x7f6c240b3c60 2022-06-17T12:54:49.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.724+0000 7f6c237fe700 1 -- 172.21.15.36:0/4212181445 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c10003260 con 0x7f6c240b3c60 2022-06-17T12:54:49.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.724+0000 7f6c237fe700 1 -- 172.21.15.36:0/4212181445 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3553988895 0 0) 0x7f6c240b8400 con 0x7f6c240b2d20 2022-06-17T12:54:49.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.724+0000 7f6c237fe700 1 -- 172.21.15.36:0/4212181445 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c240ae5b0 con 0x7f6c240b2d20 2022-06-17T12:54:49.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.724+0000 7f6c237fe700 1 -- 172.21.15.36:0/4212181445 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3739737685 0 0) 0x7f6c10003260 con 0x7f6c240b3c60 2022-06-17T12:54:49.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.724+0000 7f6c28949700 1 -- 172.21.15.36:0/4212181445 >> v1:172.21.15.70:6789/0 conn(0x7f6c240b2d20 legacy=0x7f6c240b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:49.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.724+0000 7f6c28949700 1 -- 172.21.15.36:0/4212181445 >> v1:172.21.15.36:6789/0 conn(0x7f6c240b3c60 legacy=0x7f6c240afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:49.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.724+0000 7f6c28949700 1 -- 172.21.15.36:0/4212181445 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c240b2100 con 0x7f6c240b8020 2022-06-17T12:54:49.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.724+0000 7f6c28949700 1 --2- 172.21.15.36:0/4212181445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c240b8020 0x7f6c2413ded0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f6c14001110 tx=0x7f6c18006df0).ready entity=mon.1 client_cookie=aa68c7bb60816ab0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:49.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.724+0000 7f6c237fe700 1 -- 172.21.15.36:0/4212181445 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c14001660 con 0x7f6c240b8020 2022-06-17T12:54:49.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.724+0000 7f6c237fe700 1 -- 172.21.15.36:0/4212181445 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c18000e90 con 0x7f6c240b8020 2022-06-17T12:54:49.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.724+0000 7f6c237fe700 1 -- 172.21.15.36:0/4212181445 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c18012460 con 0x7f6c240b8020 2022-06-17T12:54:49.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.724+0000 7f6c2994b700 1 -- 172.21.15.36:0/4212181445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c240b8020 msgr2=0x7f6c2413ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:49.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.724+0000 7f6c2994b700 1 --2- 172.21.15.36:0/4212181445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c240b8020 0x7f6c2413ded0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f6c14001110 tx=0x7f6c18006df0).stop 2022-06-17T12:54:49.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.724+0000 7f6c2994b700 1 -- 172.21.15.36:0/4212181445 shutdown_connections 2022-06-17T12:54:49.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.724+0000 7f6c2994b700 1 --2- 172.21.15.36:0/4212181445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c240b8020 0x7f6c2413ded0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:49.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.724+0000 7f6c2994b700 1 -- 172.21.15.36:0/4212181445 >> 172.21.15.36:0/4212181445 conn(0x7f6c2405c550 msgr2=0x7f6c240b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:49.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.724+0000 7f6c2994b700 1 -- 172.21.15.36:0/4212181445 shutdown_connections 2022-06-17T12:54:49.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.724+0000 7f6c2994b700 1 -- 172.21.15.36:0/4212181445 wait complete. 2022-06-17T12:54:49.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.728+0000 7f6c2994b700 1 Processor -- start 2022-06-17T12:54:49.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.728+0000 7f6c2994b700 1 -- start start 2022-06-17T12:54:49.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.728+0000 7f6c2994b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c240b2d20 0x7f6c24110aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:49.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.728+0000 7f6c2994b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c240b8020 0x7f6c24140580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:49.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.728+0000 7f6c2994b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c2413f680 0x7f6c2413a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:49.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.728+0000 7f6c2994b700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c24140a70 con 0x7f6c240b8020 2022-06-17T12:54:49.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.728+0000 7f6c2994b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c241110f0 con 0x7f6c2413f680 2022-06-17T12:54:49.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.728+0000 7f6c2994b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2413e6e0 con 0x7f6c240b2d20 2022-06-17T12:54:49.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.728+0000 7f6c23fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c240b8020 0x7f6c24140580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:49.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.728+0000 7f6c23fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c240b8020 0x7f6c24140580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37410/0 (socket says 172.21.15.36:37410) 2022-06-17T12:54:49.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.728+0000 7f6c23fff700 1 -- 172.21.15.36:0/3969243542 learned_addr learned my addr 172.21.15.36:0/3969243542 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:49.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.728+0000 7f6c28949700 1 --2- 172.21.15.36:0/3969243542 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c240b2d20 0x7f6c24110aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:49.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.728+0000 7f6c2914a700 1 --2- 172.21.15.36:0/3969243542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c2413f680 0x7f6c2413a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:49.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.728+0000 7f6c23fff700 1 -- 172.21.15.36:0/3969243542 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c240b2d20 msgr2=0x7f6c24110aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:49.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.728+0000 7f6c23fff700 1 --2- 172.21.15.36:0/3969243542 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c240b2d20 0x7f6c24110aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:49.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.728+0000 7f6c23fff700 1 -- 172.21.15.36:0/3969243542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c2413f680 msgr2=0x7f6c2413a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:49.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.728+0000 7f6c23fff700 1 --2- 172.21.15.36:0/3969243542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c2413f680 0x7f6c2413a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:49.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.728+0000 7f6c23fff700 1 -- 172.21.15.36:0/3969243542 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1800b180 con 0x7f6c240b8020 2022-06-17T12:54:49.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.728+0000 7f6c23fff700 1 --2- 172.21.15.36:0/3969243542 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c240b8020 0x7f6c24140580 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7f6c2413e9f0 tx=0x7f6c14001970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:49.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.728+0000 7f6c21ffb700 1 -- 172.21.15.36:0/3969243542 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c14007170 con 0x7f6c240b8020 2022-06-17T12:54:49.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.728+0000 7f6c21ffb700 1 -- 172.21.15.36:0/3969243542 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c140072d0 con 0x7f6c240b8020 2022-06-17T12:54:49.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.728+0000 7f6c2994b700 1 -- 172.21.15.36:0/3969243542 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2413e480 con 0x7f6c240b8020 2022-06-17T12:54:49.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.728+0000 7f6c21ffb700 1 -- 172.21.15.36:0/3969243542 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c14004710 con 0x7f6c240b8020 2022-06-17T12:54:49.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.732+0000 7f6c2994b700 1 -- 172.21.15.36:0/3969243542 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2413eb10 con 0x7f6c240b8020 2022-06-17T12:54:49.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.732+0000 7f6c2994b700 1 -- 172.21.15.36:0/3969243542 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c240b23a0 con 0x7f6c240b8020 2022-06-17T12:54:49.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.732+0000 7f6c21ffb700 1 -- 172.21.15.36:0/3969243542 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6c1401b070 con 0x7f6c240b8020 2022-06-17T12:54:49.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.732+0000 7f6c21ffb700 1 --2- 172.21.15.36:0/3969243542 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c04053af0 0x7f6c04055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:49.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.732+0000 7f6c21ffb700 1 -- 172.21.15.36:0/3969243542 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f6c1406b5e0 con 0x7f6c240b8020 2022-06-17T12:54:49.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.732+0000 7f6c28949700 1 --2- 172.21.15.36:0/3969243542 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c04053af0 0x7f6c04055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:49.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.736+0000 7f6c28949700 1 --2- 172.21.15.36:0/3969243542 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c04053af0 0x7f6c04055fb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f6c1801a000 tx=0x7f6c18014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:49.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.740+0000 7f6c21ffb700 1 -- 172.21.15.36:0/3969243542 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6c14042390 con 0x7f6c240b8020 2022-06-17T12:54:49.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:49.868+0000 7f6c2994b700 1 -- 172.21.15.36:0/3969243542 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6c2413e820 con 0x7f6c240b8020 2022-06-17T12:54:50.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.572+0000 7f6c21ffb700 1 -- 172.21.15.36:0/3969243542 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) v1 ==== 185+0+0 (secure 0 0 0) 0x7f6c140459b0 con 0x7f6c240b8020 2022-06-17T12:54:50.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.576+0000 7f6c2994b700 1 -- 172.21.15.36:0/3969243542 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c04053af0 msgr2=0x7f6c04055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:50.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.576+0000 7f6c2994b700 1 --2- 172.21.15.36:0/3969243542 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c04053af0 0x7f6c04055fb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f6c1801a000 tx=0x7f6c18014000).stop 2022-06-17T12:54:50.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.576+0000 7f6c2994b700 1 -- 172.21.15.36:0/3969243542 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c240b8020 msgr2=0x7f6c24140580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:50.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.576+0000 7f6c2994b700 1 --2- 172.21.15.36:0/3969243542 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c240b8020 0x7f6c24140580 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7f6c2413e9f0 tx=0x7f6c14001970).stop 2022-06-17T12:54:50.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.576+0000 7f6c2994b700 1 -- 172.21.15.36:0/3969243542 shutdown_connections 2022-06-17T12:54:50.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.576+0000 7f6c2994b700 1 --2- 172.21.15.36:0/3969243542 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c04053af0 0x7f6c04055fb0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:50.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.576+0000 7f6c2994b700 1 --2- 172.21.15.36:0/3969243542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c2413f680 0x7f6c2413a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:50.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.576+0000 7f6c2994b700 1 --2- 172.21.15.36:0/3969243542 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c240b8020 0x7f6c24140580 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:50.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.576+0000 7f6c2994b700 1 --2- 172.21.15.36:0/3969243542 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c240b2d20 0x7f6c24110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:50.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.576+0000 7f6c2994b700 1 -- 172.21.15.36:0/3969243542 >> 172.21.15.36:0/3969243542 conn(0x7f6c2405c550 msgr2=0x7f6c2405f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:50.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.576+0000 7f6c2994b700 1 -- 172.21.15.36:0/3969243542 shutdown_connections 2022-06-17T12:54:50.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.576+0000 7f6c2994b700 1 -- 172.21.15.36:0/3969243542 wait complete. 2022-06-17T12:54:50.584 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' removed 2022-06-17T12:54:50.587 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-17T12:54:50.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.712+0000 7fd9c1d37700 1 Processor -- start 2022-06-17T12:54:50.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.716+0000 7fd9c1d37700 1 -- start start 2022-06-17T12:54:50.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.716+0000 7fd9c1d37700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd9bc05fe80 0x7fd9bc060260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:50.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.716+0000 7fd9c1d37700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd9bc0adf40 0x7fd9bc0667f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:50.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.716+0000 7fd9c1d37700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9bc066210 con 0x7fd9bc13aa80 2022-06-17T12:54:50.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.716+0000 7fd9c1d37700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9bc142070 con 0x7fd9bc0adf40 2022-06-17T12:54:50.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.716+0000 7fd9c1d37700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc0664b0 con 0x7fd9bc05fe80 2022-06-17T12:54:50.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.716+0000 7fd9baffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd9bc0adf40 0x7fd9bc0667f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:50.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.716+0000 7fd9bbfff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fd9bc13aa80 0x7fd9bc141530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54950/0 (socket says 172.21.15.36:54950) 2022-06-17T12:54:50.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.716+0000 7fd9bbfff700 1 -- 172.21.15.36:0/3916188986 learned_addr learned my addr 172.21.15.36:0/3916188986 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:50.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.716+0000 7fd9bb7fe700 1 --2- 172.21.15.36:0/3916188986 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd9bc05fe80 0x7fd9bc060260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:50.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.716+0000 7fd9baffd700 1 -- 172.21.15.36:0/3916188986 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd9bc05fe80 msgr2=0x7fd9bc060260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:50.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.716+0000 7fd9baffd700 1 --2- 172.21.15.36:0/3916188986 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd9bc05fe80 0x7fd9bc060260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:50.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.716+0000 7fd9baffd700 1 -- 172.21.15.36:0/3916188986 >> v1:172.21.15.36:6789/0 conn(0x7fd9bc13aa80 legacy=0x7fd9bc141530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:50.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.716+0000 7fd9baffd700 1 -- 172.21.15.36:0/3916188986 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9bc141090 con 0x7fd9bc0adf40 2022-06-17T12:54:50.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.716+0000 7fd9baffd700 1 --2- 172.21.15.36:0/3916188986 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd9bc0adf40 0x7fd9bc0667f0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7fd9b0002f70 tx=0x7fd9b0006df0).ready entity=mon.0 client_cookie=13c144319bc6749 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:50.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.716+0000 7fd9ba7fc700 1 -- 172.21.15.36:0/3916188986 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9b000d570 con 0x7fd9bc0adf40 2022-06-17T12:54:50.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.716+0000 7fd9ba7fc700 1 -- 172.21.15.36:0/3916188986 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b0000e90 con 0x7fd9bc0adf40 2022-06-17T12:54:50.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.716+0000 7fd9ba7fc700 1 -- 172.21.15.36:0/3916188986 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9b000d8c0 con 0x7fd9bc0adf40 2022-06-17T12:54:50.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9c1d37700 1 -- 172.21.15.36:0/3916188986 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd9bc0adf40 msgr2=0x7fd9bc0667f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:50.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9c1d37700 1 --2- 172.21.15.36:0/3916188986 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd9bc0adf40 0x7fd9bc0667f0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7fd9b0002f70 tx=0x7fd9b0006df0).stop 2022-06-17T12:54:50.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9c1d37700 1 -- 172.21.15.36:0/3916188986 shutdown_connections 2022-06-17T12:54:50.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9c1d37700 1 --2- 172.21.15.36:0/3916188986 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd9bc0adf40 0x7fd9bc0667f0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:50.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9c1d37700 1 --2- 172.21.15.36:0/3916188986 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd9bc05fe80 0x7fd9bc060260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:50.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9c1d37700 1 -- 172.21.15.36:0/3916188986 >> 172.21.15.36:0/3916188986 conn(0x7fd9bc060990 msgr2=0x7fd9bc05f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:50.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9c1d37700 1 -- 172.21.15.36:0/3916188986 shutdown_connections 2022-06-17T12:54:50.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9c1d37700 1 -- 172.21.15.36:0/3916188986 wait complete. 2022-06-17T12:54:50.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9c1d37700 1 Processor -- start 2022-06-17T12:54:50.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9c1d37700 1 -- start start 2022-06-17T12:54:50.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9c1d37700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9bc05fe80 0x7fd9bc13a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:50.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9c1d37700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd9bc0adf40 0x7fd9bc143250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:50.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9c1d37700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd9bc13aa80 0x7fd9bc0b5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:50.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9c1d37700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc061a00 con 0x7fd9bc13aa80 2022-06-17T12:54:50.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9c1d37700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc062090 con 0x7fd9bc05fe80 2022-06-17T12:54:50.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9c1d37700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc062ca0 con 0x7fd9bc0adf40 2022-06-17T12:54:50.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9bbfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd9bc13aa80 0x7fd9bc0b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:50.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9bbfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd9bc13aa80 0x7fd9bc0b5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37426/0 (socket says 172.21.15.36:37426) 2022-06-17T12:54:50.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9bbfff700 1 -- 172.21.15.36:0/3998566023 learned_addr learned my addr 172.21.15.36:0/3998566023 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:50.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9bb7fe700 1 --2- 172.21.15.36:0/3998566023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9bc05fe80 0x7fd9bc13a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:50.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9baffd700 1 --2- 172.21.15.36:0/3998566023 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd9bc0adf40 0x7fd9bc143250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:50.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9bbfff700 1 -- 172.21.15.36:0/3998566023 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd9bc0adf40 msgr2=0x7fd9bc143250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:50.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9bbfff700 1 --2- 172.21.15.36:0/3998566023 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd9bc0adf40 0x7fd9bc143250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:50.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9bbfff700 1 -- 172.21.15.36:0/3998566023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9bc05fe80 msgr2=0x7fd9bc13a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:50.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9bbfff700 1 --2- 172.21.15.36:0/3998566023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9bc05fe80 0x7fd9bc13a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:50.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.720+0000 7fd9bbfff700 1 -- 172.21.15.36:0/3998566023 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b000b180 con 0x7fd9bc13aa80 2022-06-17T12:54:50.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.724+0000 7fd9bbfff700 1 --2- 172.21.15.36:0/3998566023 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd9bc13aa80 0x7fd9bc0b5760 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x7fd9bc0add00 tx=0x7fd9b40013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:50.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.724+0000 7fd9b8ff9700 1 -- 172.21.15.36:0/3998566023 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9b4005ba0 con 0x7fd9bc13aa80 2022-06-17T12:54:50.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.724+0000 7fd9b8ff9700 1 -- 172.21.15.36:0/3998566023 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b4005d00 con 0x7fd9bc13aa80 2022-06-17T12:54:50.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.724+0000 7fd9b8ff9700 1 -- 172.21.15.36:0/3998566023 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9b40072c0 con 0x7fd9bc13aa80 2022-06-17T12:54:50.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.724+0000 7fd9c1d37700 1 -- 172.21.15.36:0/3998566023 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9bc061bf0 con 0x7fd9bc13aa80 2022-06-17T12:54:50.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.724+0000 7fd9c1d37700 1 -- 172.21.15.36:0/3998566023 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9bc0b5f20 con 0x7fd9bc13aa80 2022-06-17T12:54:50.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.724+0000 7fd9c1d37700 1 -- 172.21.15.36:0/3998566023 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9bc061d80 con 0x7fd9bc13aa80 2022-06-17T12:54:50.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.724+0000 7fd9b8ff9700 1 -- 172.21.15.36:0/3998566023 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd9b4010070 con 0x7fd9bc13aa80 2022-06-17T12:54:50.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.724+0000 7fd9b8ff9700 1 --2- 172.21.15.36:0/3998566023 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd99c053ad0 0x7fd99c055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:50.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.724+0000 7fd9b8ff9700 1 -- 172.21.15.36:0/3998566023 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6068+0+0 (secure 0 0 0) 0x7fd9b406b750 con 0x7fd9bc13aa80 2022-06-17T12:54:50.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.724+0000 7fd9bb7fe700 1 --2- 172.21.15.36:0/3998566023 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd99c053ad0 0x7fd99c055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:50.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.724+0000 7fd9bb7fe700 1 --2- 172.21.15.36:0/3998566023 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd99c053ad0 0x7fd99c055f90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7fd9ac00b000 tx=0x7fd9ac006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:50.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.732+0000 7fd9b8ff9700 1 -- 172.21.15.36:0/3998566023 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd9b40426b0 con 0x7fd9bc13aa80 2022-06-17T12:54:50.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:50.856+0000 7fd9c1d37700 1 -- 172.21.15.36:0/3998566023 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd9bc057410 con 0x7fd9bc13aa80 2022-06-17T12:54:51.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.584+0000 7fd9b8ff9700 1 -- 172.21.15.36:0/3998566023 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) v1 ==== 173+0+0 (secure 0 0 0) 0x7fd9b4045cd0 con 0x7fd9bc13aa80 2022-06-17T12:54:51.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.588+0000 7fd9c1d37700 1 -- 172.21.15.36:0/3998566023 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd99c053ad0 msgr2=0x7fd99c055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:51.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.588+0000 7fd9c1d37700 1 --2- 172.21.15.36:0/3998566023 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd99c053ad0 0x7fd99c055f90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7fd9ac00b000 tx=0x7fd9ac006df0).stop 2022-06-17T12:54:51.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.588+0000 7fd9c1d37700 1 -- 172.21.15.36:0/3998566023 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd9bc13aa80 msgr2=0x7fd9bc0b5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:51.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.588+0000 7fd9c1d37700 1 --2- 172.21.15.36:0/3998566023 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd9bc13aa80 0x7fd9bc0b5760 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x7fd9bc0add00 tx=0x7fd9b40013a0).stop 2022-06-17T12:54:51.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.588+0000 7fd9c1d37700 1 -- 172.21.15.36:0/3998566023 shutdown_connections 2022-06-17T12:54:51.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.588+0000 7fd9c1d37700 1 --2- 172.21.15.36:0/3998566023 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd99c053ad0 0x7fd99c055f90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:51.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.588+0000 7fd9c1d37700 1 --2- 172.21.15.36:0/3998566023 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd9bc13aa80 0x7fd9bc0b5760 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:51.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.588+0000 7fd9c1d37700 1 --2- 172.21.15.36:0/3998566023 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd9bc0adf40 0x7fd9bc143250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:51.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.588+0000 7fd9c1d37700 1 --2- 172.21.15.36:0/3998566023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9bc05fe80 0x7fd9bc13a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:51.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.588+0000 7fd9c1d37700 1 -- 172.21.15.36:0/3998566023 >> 172.21.15.36:0/3998566023 conn(0x7fd9bc060990 msgr2=0x7fd9bc0667f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:51.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.588+0000 7fd9c1d37700 1 -- 172.21.15.36:0/3998566023 shutdown_connections 2022-06-17T12:54:51.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.588+0000 7fd9c1d37700 1 -- 172.21.15.36:0/3998566023 wait complete. 2022-06-17T12:54:51.585 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' removed 2022-06-17T12:54:51.592 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-06-17T12:54:51.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.712+0000 7f2b22d06700 1 Processor -- start 2022-06-17T12:54:51.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.712+0000 7f2b22d06700 1 -- start start 2022-06-17T12:54:51.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b22d06700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2b1c0b27b0 0x7f2b1c14b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:51.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b22d06700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b1c14b870 con 0x7f2b1c147bd0 2022-06-17T12:54:51.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b22d06700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b1c0ac500 con 0x7f2b1c060ca0 2022-06-17T12:54:51.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b22d06700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b1c0ace90 con 0x7f2b1c0b27b0 2022-06-17T12:54:51.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b22505700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f2b1c147bd0 0x7f2b1c1442d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54964/0 (socket says 172.21.15.36:54964) 2022-06-17T12:54:51.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b22505700 1 -- 172.21.15.36:0/3609287661 learned_addr learned my addr 172.21.15.36:0/3609287661 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:51.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b21d04700 1 --2- 172.21.15.36:0/3609287661 >> v2:172.21.15.36:3300/0 conn(0x7f2b1c0b27b0 0x7f2b1c14b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:51.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b20d02700 1 -- 172.21.15.36:0/3609287661 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1414767882 0 0) 0x7f2b1c14b870 con 0x7f2b1c147bd0 2022-06-17T12:54:51.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b20d02700 1 -- 172.21.15.36:0/3609287661 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b04003260 con 0x7f2b1c147bd0 2022-06-17T12:54:51.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b20d02700 1 -- 172.21.15.36:0/3609287661 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1815272147 0 0) 0x7f2b04003260 con 0x7f2b1c147bd0 2022-06-17T12:54:51.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b20d02700 1 -- 172.21.15.36:0/3609287661 >> v2:172.21.15.36:3300/0 conn(0x7f2b1c0b27b0 msgr2=0x7f2b1c14b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:51.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b20d02700 1 --2- 172.21.15.36:0/3609287661 >> v2:172.21.15.36:3300/0 conn(0x7f2b1c0b27b0 0x7f2b1c14b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:51.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b20d02700 1 -- 172.21.15.36:0/3609287661 >> v1:172.21.15.70:6789/0 conn(0x7f2b1c060ca0 legacy=0x7f2b1c0ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:51.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b20d02700 1 -- 172.21.15.36:0/3609287661 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1c0b2b90 con 0x7f2b1c147bd0 2022-06-17T12:54:51.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b20d02700 1 -- 172.21.15.36:0/3609287661 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f2b140019e0 con 0x7f2b1c147bd0 2022-06-17T12:54:51.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b20d02700 1 -- 172.21.15.36:0/3609287661 >> v1:172.21.15.36:6789/0 conn(0x7f2b1c147bd0 legacy=0x7f2b1c1442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:51.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b20d02700 1 --2- 172.21.15.36:0/3609287661 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b04003750 0x7f2b04003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:51.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b20d02700 1 --2- 172.21.15.36:0/3609287661 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b04004500 0x7f2b04004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:51.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b20d02700 1 --2- 172.21.15.36:0/3609287661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b04004e70 0x7f2b0400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:51.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b20d02700 1 -- 172.21.15.36:0/3609287661 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c0ace90 con 0x7f2b04004500 2022-06-17T12:54:51.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b20d02700 1 -- 172.21.15.36:0/3609287661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0400bc70 con 0x7f2b04004e70 2022-06-17T12:54:51.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b20d02700 1 -- 172.21.15.36:0/3609287661 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0400be30 con 0x7f2b04003750 2022-06-17T12:54:51.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b21503700 1 --2- 172.21.15.36:0/3609287661 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b04004500 0x7f2b04004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:51.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b22505700 1 --2- 172.21.15.36:0/3609287661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b04004e70 0x7f2b0400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:51.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b21d04700 1 --2- 172.21.15.36:0/3609287661 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b04003750 0x7f2b04003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:51.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b21503700 1 -- 172.21.15.36:0/3609287661 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b04003750 msgr2=0x7f2b04003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:51.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b21503700 1 --2- 172.21.15.36:0/3609287661 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b04003750 0x7f2b04003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:51.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b21503700 1 -- 172.21.15.36:0/3609287661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b04004e70 msgr2=0x7f2b0400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:51.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b21503700 1 --2- 172.21.15.36:0/3609287661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b04004e70 0x7f2b0400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:51.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b21503700 1 -- 172.21.15.36:0/3609287661 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b04003260 con 0x7f2b04004500 2022-06-17T12:54:51.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b22505700 1 -- 172.21.15.36:0/3609287661 reap_dead start 2022-06-17T12:54:51.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b21503700 1 --2- 172.21.15.36:0/3609287661 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b04004500 0x7f2b04004980 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7f2b18001020 tx=0x7f2b180079e0).ready entity=mon.0 client_cookie=1fbb8998239f31c9 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:51.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b20d02700 1 -- 172.21.15.36:0/3609287661 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b18007cf0 con 0x7f2b04004500 2022-06-17T12:54:51.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b20d02700 1 -- 172.21.15.36:0/3609287661 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b18007030 con 0x7f2b04004500 2022-06-17T12:54:51.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.716+0000 7f2b20d02700 1 -- 172.21.15.36:0/3609287661 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b18007390 con 0x7f2b04004500 2022-06-17T12:54:51.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b22d06700 1 -- 172.21.15.36:0/3609287661 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b04004500 msgr2=0x7f2b04004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:51.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b22d06700 1 --2- 172.21.15.36:0/3609287661 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b04004500 0x7f2b04004980 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7f2b18001020 tx=0x7f2b180079e0).stop 2022-06-17T12:54:51.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b22d06700 1 -- 172.21.15.36:0/3609287661 shutdown_connections 2022-06-17T12:54:51.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b22d06700 1 --2- 172.21.15.36:0/3609287661 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b04004500 0x7f2b04004980 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:51.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b22d06700 1 -- 172.21.15.36:0/3609287661 >> 172.21.15.36:0/3609287661 conn(0x7f2b1c0b5d10 msgr2=0x7f2b1c0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:51.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b22d06700 1 -- 172.21.15.36:0/3609287661 shutdown_connections 2022-06-17T12:54:51.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b22d06700 1 -- 172.21.15.36:0/3609287661 wait complete. 2022-06-17T12:54:51.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b22d06700 1 Processor -- start 2022-06-17T12:54:51.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b22d06700 1 -- start start 2022-06-17T12:54:51.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b22d06700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b04004500 0x7f2b1c14e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:51.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b22d06700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b1c14cc40 0x7f2b1c15a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:51.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b22d06700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b1c1566a0 0x7f2b1c156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:51.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b22d06700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c0659c0 con 0x7f2b04004500 2022-06-17T12:54:51.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b22d06700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c159d50 con 0x7f2b1c14cc40 2022-06-17T12:54:51.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b22d06700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c159ff0 con 0x7f2b1c1566a0 2022-06-17T12:54:51.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b21d04700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b04004500 0x7f2b1c14e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:51.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b22505700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b1c1566a0 0x7f2b1c156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:51.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b21503700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b1c14cc40 0x7f2b1c15a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:51.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b21d04700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b04004500 0x7f2b1c14e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37444/0 (socket says 172.21.15.36:37444) 2022-06-17T12:54:51.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b21d04700 1 -- 172.21.15.36:0/2065968562 learned_addr learned my addr 172.21.15.36:0/2065968562 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:51.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b21d04700 1 -- 172.21.15.36:0/2065968562 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b1c1566a0 msgr2=0x7f2b1c156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:51.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b21d04700 1 --2- 172.21.15.36:0/2065968562 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b1c1566a0 0x7f2b1c156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:51.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b21d04700 1 -- 172.21.15.36:0/2065968562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b1c14cc40 msgr2=0x7f2b1c15a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:51.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b21d04700 1 --2- 172.21.15.36:0/2065968562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b1c14cc40 0x7f2b1c15a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:51.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b21d04700 1 -- 172.21.15.36:0/2065968562 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1c14d850 con 0x7f2b04004500 2022-06-17T12:54:51.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.720+0000 7f2b21d04700 1 --2- 172.21.15.36:0/2065968562 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b04004500 0x7f2b1c14e4b0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7f2b0c00b000 tx=0x7f2b0c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:51.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.724+0000 7f2b0affd700 1 -- 172.21.15.36:0/2065968562 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b0c00f730 con 0x7f2b04004500 2022-06-17T12:54:51.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.724+0000 7f2b0affd700 1 -- 172.21.15.36:0/2065968562 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0c000e90 con 0x7f2b04004500 2022-06-17T12:54:51.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.724+0000 7f2b22d06700 1 -- 172.21.15.36:0/2065968562 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b1c15a980 con 0x7f2b04004500 2022-06-17T12:54:51.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.724+0000 7f2b0affd700 1 -- 172.21.15.36:0/2065968562 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b0c00fa80 con 0x7f2b04004500 2022-06-17T12:54:51.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.724+0000 7f2b22d06700 1 -- 172.21.15.36:0/2065968562 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b1c064cf0 con 0x7f2b04004500 2022-06-17T12:54:51.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.724+0000 7f2b22d06700 1 -- 172.21.15.36:0/2065968562 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b1c15a1b0 con 0x7f2b04004500 2022-06-17T12:54:51.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.724+0000 7f2b0affd700 1 -- 172.21.15.36:0/2065968562 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2b0c00fbe0 con 0x7f2b04004500 2022-06-17T12:54:51.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.724+0000 7f2b0affd700 1 --2- 172.21.15.36:0/2065968562 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2afc053af0 0x7f2afc055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:51.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.724+0000 7f2b0affd700 1 -- 172.21.15.36:0/2065968562 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f2b0c06c440 con 0x7f2b04004500 2022-06-17T12:54:51.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.724+0000 7f2b21503700 1 --2- 172.21.15.36:0/2065968562 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2afc053af0 0x7f2afc055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:51.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.724+0000 7f2b21503700 1 --2- 172.21.15.36:0/2065968562 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2afc053af0 0x7f2afc055fb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f2b1801b000 tx=0x7f2b18015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:51.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.732+0000 7f2b0affd700 1 -- 172.21.15.36:0/2065968562 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2b0c043520 con 0x7f2b04004500 2022-06-17T12:54:51.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.863+0000 7f2b22d06700 1 -- 172.21.15.36:0/2065968562 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2b1c057410 con 0x7f2b04004500 2022-06-17T12:54:51.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.871+0000 7f2b0affd700 1 -- 172.21.15.36:0/2065968562 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2b0c0180a0 con 0x7f2b04004500 2022-06-17T12:54:51.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.871+0000 7f2b22d06700 1 -- 172.21.15.36:0/2065968562 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2afc053af0 msgr2=0x7f2afc055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:51.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.871+0000 7f2b22d06700 1 --2- 172.21.15.36:0/2065968562 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2afc053af0 0x7f2afc055fb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f2b1801b000 tx=0x7f2b18015000).stop 2022-06-17T12:54:51.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.871+0000 7f2b22d06700 1 -- 172.21.15.36:0/2065968562 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b04004500 msgr2=0x7f2b1c14e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:51.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.871+0000 7f2b22d06700 1 --2- 172.21.15.36:0/2065968562 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b04004500 0x7f2b1c14e4b0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7f2b0c00b000 tx=0x7f2b0c006df0).stop 2022-06-17T12:54:51.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.871+0000 7f2b22d06700 1 -- 172.21.15.36:0/2065968562 shutdown_connections 2022-06-17T12:54:51.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.871+0000 7f2b22d06700 1 --2- 172.21.15.36:0/2065968562 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2afc053af0 0x7f2afc055fb0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:51.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.871+0000 7f2b22d06700 1 --2- 172.21.15.36:0/2065968562 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b1c1566a0 0x7f2b1c156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:51.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.875+0000 7f2b22d06700 1 --2- 172.21.15.36:0/2065968562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b1c14cc40 0x7f2b1c15a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:51.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.875+0000 7f2b22d06700 1 --2- 172.21.15.36:0/2065968562 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b04004500 0x7f2b1c14e4b0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:51.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.875+0000 7f2b22d06700 1 -- 172.21.15.36:0/2065968562 >> 172.21.15.36:0/2065968562 conn(0x7f2b1c0b5d10 msgr2=0x7f2b1c146710 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:51.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.875+0000 7f2b22d06700 1 -- 172.21.15.36:0/2065968562 shutdown_connections 2022-06-17T12:54:51.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.875+0000 7f2b22d06700 1 -- 172.21.15.36:0/2065968562 wait complete. 2022-06-17T12:54:51.876 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical' 2022-06-17T12:54:51.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.999+0000 7f8e2f59e700 1 Processor -- start 2022-06-17T12:54:51.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.999+0000 7f8e2f59e700 1 -- start start 2022-06-17T12:54:51.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.999+0000 7f8e2f59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e300b1e00 0x7f8e30067610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:51.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.999+0000 7f8e2f59e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8e3005c550 0x7f8e30149cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:51.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.999+0000 7f8e2f59e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e30067b00 con 0x7f8e30060920 2022-06-17T12:54:51.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.999+0000 7f8e2f59e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e300aff80 con 0x7f8e3005c550 2022-06-17T12:54:51.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:51.999+0000 7f8e2f59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e300ad020 con 0x7f8e300b1e00 2022-06-17T12:54:51.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.003+0000 7f8e2e59c700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f8e30060920 0x7f8e300ad760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:54980/0 (socket says 172.21.15.36:54980) 2022-06-17T12:54:51.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.003+0000 7f8e2e59c700 1 -- 172.21.15.36:0/2110438324 learned_addr learned my addr 172.21.15.36:0/2110438324 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:51.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.003+0000 7f8e2ed9d700 1 --2- 172.21.15.36:0/2110438324 >> v2:172.21.15.70:3300/0 conn(0x7f8e3005c550 0x7f8e30149cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:51.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.003+0000 7f8e2dd9b700 1 --2- 172.21.15.36:0/2110438324 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e300b1e00 0x7f8e30067610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:51.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.003+0000 7f8e2d59a700 1 -- 172.21.15.36:0/2110438324 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4134030396 0 0) 0x7f8e30067b00 con 0x7f8e30060920 2022-06-17T12:54:51.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.003+0000 7f8e2d59a700 1 -- 172.21.15.36:0/2110438324 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e10003260 con 0x7f8e30060920 2022-06-17T12:54:52.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.003+0000 7f8e2ed9d700 1 -- 172.21.15.36:0/2110438324 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e300b1e00 msgr2=0x7f8e30067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:52.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.003+0000 7f8e2ed9d700 1 --2- 172.21.15.36:0/2110438324 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e300b1e00 0x7f8e30067610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:52.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.003+0000 7f8e2ed9d700 1 -- 172.21.15.36:0/2110438324 >> v1:172.21.15.36:6789/0 conn(0x7f8e30060920 legacy=0x7f8e300ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:52.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.003+0000 7f8e2ed9d700 1 -- 172.21.15.36:0/2110438324 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e300ad2c0 con 0x7f8e3005c550 2022-06-17T12:54:52.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.003+0000 7f8e2dd9b700 1 --2- 172.21.15.36:0/2110438324 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e300b1e00 0x7f8e30067610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:52.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.003+0000 7f8e2ed9d700 1 --2- 172.21.15.36:0/2110438324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e3005c550 0x7f8e30149cc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f8e28002f70 tx=0x7f8e28006df0).ready entity=mon.1 client_cookie=bd9bbd1980cb858d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:52.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.003+0000 7f8e2d59a700 1 -- 172.21.15.36:0/2110438324 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e28014070 con 0x7f8e3005c550 2022-06-17T12:54:52.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.003+0000 7f8e2d59a700 1 -- 172.21.15.36:0/2110438324 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e28000e90 con 0x7f8e3005c550 2022-06-17T12:54:52.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.003+0000 7f8e2d59a700 1 -- 172.21.15.36:0/2110438324 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e2800fa90 con 0x7f8e3005c550 2022-06-17T12:54:52.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.003+0000 7f8e2f59e700 1 -- 172.21.15.36:0/2110438324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e3005c550 msgr2=0x7f8e30149cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:52.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.003+0000 7f8e2f59e700 1 --2- 172.21.15.36:0/2110438324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e3005c550 0x7f8e30149cc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f8e28002f70 tx=0x7f8e28006df0).stop 2022-06-17T12:54:52.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.003+0000 7f8e2f59e700 1 -- 172.21.15.36:0/2110438324 shutdown_connections 2022-06-17T12:54:52.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.003+0000 7f8e2f59e700 1 --2- 172.21.15.36:0/2110438324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e3005c550 0x7f8e30149cc0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:52.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.003+0000 7f8e2f59e700 1 --2- 172.21.15.36:0/2110438324 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e300b1e00 0x7f8e30067610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:52.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.003+0000 7f8e2f59e700 1 -- 172.21.15.36:0/2110438324 >> 172.21.15.36:0/2110438324 conn(0x7f8e300b64f0 msgr2=0x7f8e300b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:52.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.003+0000 7f8e2f59e700 1 -- 172.21.15.36:0/2110438324 shutdown_connections 2022-06-17T12:54:52.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.003+0000 7f8e2f59e700 1 -- 172.21.15.36:0/2110438324 wait complete. 2022-06-17T12:54:52.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.007+0000 7f8e2f59e700 1 Processor -- start 2022-06-17T12:54:52.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.007+0000 7f8e2f59e700 1 -- start start 2022-06-17T12:54:52.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.007+0000 7f8e2f59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e3005c550 0x7f8e3009fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:52.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.007+0000 7f8e2f59e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e30060920 0x7f8e3014dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:52.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.007+0000 7f8e2f59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e300b1e00 0x7f8e300a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:52.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.007+0000 7f8e2f59e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e300a3fd0 con 0x7f8e300b1e00 2022-06-17T12:54:52.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.007+0000 7f8e2f59e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e300a0be0 con 0x7f8e30060920 2022-06-17T12:54:52.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.007+0000 7f8e2f59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e300a3d30 con 0x7f8e3005c550 2022-06-17T12:54:52.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.007+0000 7f8e2ed9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e300b1e00 0x7f8e300a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:52.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.007+0000 7f8e2e59c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e3005c550 0x7f8e3009fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:52.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.007+0000 7f8e2e59c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e3005c550 0x7f8e3009fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:52246/0 (socket says 172.21.15.36:52246) 2022-06-17T12:54:52.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.007+0000 7f8e2dd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e30060920 0x7f8e3014dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:52.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.007+0000 7f8e2e59c700 1 -- 172.21.15.36:0/2722054248 learned_addr learned my addr 172.21.15.36:0/2722054248 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:52.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.007+0000 7f8e2ed9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e300b1e00 0x7f8e300a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37460/0 (socket says 172.21.15.36:37460) 2022-06-17T12:54:52.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.011+0000 7f8e2e59c700 1 -- 172.21.15.36:0/2722054248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e30060920 msgr2=0x7f8e3014dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:52.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.011+0000 7f8e2e59c700 1 --2- 172.21.15.36:0/2722054248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e30060920 0x7f8e3014dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:52.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.011+0000 7f8e2e59c700 1 -- 172.21.15.36:0/2722054248 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e300b1e00 msgr2=0x7f8e300a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:52.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.011+0000 7f8e2e59c700 1 --2- 172.21.15.36:0/2722054248 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e300b1e00 0x7f8e300a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:52.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.011+0000 7f8e2e59c700 1 -- 172.21.15.36:0/2722054248 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e18009470 con 0x7f8e3005c550 2022-06-17T12:54:52.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.011+0000 7f8e2e59c700 1 --2- 172.21.15.36:0/2722054248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e3005c550 0x7f8e3009fba0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f8e300a3c50 tx=0x7f8e20007aa0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:52.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.011+0000 7f8e277fe700 1 -- 172.21.15.36:0/2722054248 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e2000ce70 con 0x7f8e3005c550 2022-06-17T12:54:52.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.011+0000 7f8e277fe700 1 -- 172.21.15.36:0/2722054248 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e200070d0 con 0x7f8e3005c550 2022-06-17T12:54:52.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.011+0000 7f8e2f59e700 1 -- 172.21.15.36:0/2722054248 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e2800b180 con 0x7f8e3005c550 2022-06-17T12:54:52.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.011+0000 7f8e277fe700 1 -- 172.21.15.36:0/2722054248 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e20007430 con 0x7f8e3005c550 2022-06-17T12:54:52.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.011+0000 7f8e2f59e700 1 -- 172.21.15.36:0/2722054248 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e300a2420 con 0x7f8e3005c550 2022-06-17T12:54:52.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.011+0000 7f8e2f59e700 1 -- 172.21.15.36:0/2722054248 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e300a0d50 con 0x7f8e3005c550 2022-06-17T12:54:52.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.011+0000 7f8e277fe700 1 -- 172.21.15.36:0/2722054248 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8e20010070 con 0x7f8e3005c550 2022-06-17T12:54:52.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.011+0000 7f8e277fe700 1 --2- 172.21.15.36:0/2722054248 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8e14053920 0x7f8e14055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:52.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.011+0000 7f8e277fe700 1 -- 172.21.15.36:0/2722054248 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f8e2006b330 con 0x7f8e3005c550 2022-06-17T12:54:52.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.011+0000 7f8e2dd9b700 1 --2- 172.21.15.36:0/2722054248 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8e14053920 0x7f8e14055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:52.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.015+0000 7f8e2dd9b700 1 --2- 172.21.15.36:0/2722054248 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8e14053920 0x7f8e14055de0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f8e1800b000 tx=0x7f8e180091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:52.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.023+0000 7f8e277fe700 1 -- 172.21.15.36:0/2722054248 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8e20042300 con 0x7f8e3005c550 2022-06-17T12:54:52.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.199+0000 7f8e2f59e700 1 -- 172.21.15.36:0/2722054248 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]} v 0) v1 -- 0x7f8e300ade70 con 0x7f8e3005c550 2022-06-17T12:54:52.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.595+0000 7f8e277fe700 1 -- 172.21.15.36:0/2722054248 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]}]=0 v367) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8e20045920 con 0x7f8e3005c550 2022-06-17T12:54:52.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.595+0000 7f8e2f59e700 1 -- 172.21.15.36:0/2722054248 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8e14053920 msgr2=0x7f8e14055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:52.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.595+0000 7f8e2f59e700 1 --2- 172.21.15.36:0/2722054248 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8e14053920 0x7f8e14055de0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f8e1800b000 tx=0x7f8e180091d0).stop 2022-06-17T12:54:52.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.595+0000 7f8e2f59e700 1 -- 172.21.15.36:0/2722054248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e3005c550 msgr2=0x7f8e3009fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:52.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.595+0000 7f8e2f59e700 1 --2- 172.21.15.36:0/2722054248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e3005c550 0x7f8e3009fba0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f8e300a3c50 tx=0x7f8e20007aa0).stop 2022-06-17T12:54:52.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.595+0000 7f8e2f59e700 1 -- 172.21.15.36:0/2722054248 shutdown_connections 2022-06-17T12:54:52.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.595+0000 7f8e2f59e700 1 --2- 172.21.15.36:0/2722054248 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8e14053920 0x7f8e14055de0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:52.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.595+0000 7f8e2f59e700 1 --2- 172.21.15.36:0/2722054248 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e300b1e00 0x7f8e300a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:52.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.595+0000 7f8e2f59e700 1 --2- 172.21.15.36:0/2722054248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e30060920 0x7f8e3014dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:52.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.595+0000 7f8e2f59e700 1 --2- 172.21.15.36:0/2722054248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e3005c550 0x7f8e3009fba0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:52.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.599+0000 7f8e2f59e700 1 -- 172.21.15.36:0/2722054248 >> 172.21.15.36:0/2722054248 conn(0x7f8e300b64f0 msgr2=0x7f8e300b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:52.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.599+0000 7f8e2f59e700 1 -- 172.21.15.36:0/2722054248 shutdown_connections 2022-06-17T12:54:52.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.599+0000 7f8e2f59e700 1 -- 172.21.15.36:0/2722054248 wait complete. 2022-06-17T12:54:52.603 INFO:tasks.cephfs_test_runner:test_dump_inode_hexademical (tasks.cephfs.test_misc.TestMisc) ... ok 2022-06-17T12:54:52.604 INFO:tasks.cephfs_test_runner:Starting test: test_filtered_df (tasks.cephfs.test_misc.TestMisc) 2022-06-17T12:54:52.604 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df' 2022-06-17T12:54:52.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.715+0000 7f03f759e700 1 Processor -- start 2022-06-17T12:54:52.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.715+0000 7f03f759e700 1 -- start start 2022-06-17T12:54:52.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f759e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f03f80b13c0 0x7f03f8140d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:52.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f759e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f03f80b2c10 0x7f03f81299a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:52.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f759e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f80b05e0 con 0x7f03f8060ac0 2022-06-17T12:54:52.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f759e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03f8060ea0 con 0x7f03f80b13c0 2022-06-17T12:54:52.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f759e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f81450f0 con 0x7f03f80b2c10 2022-06-17T12:54:52.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f5d9b700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f03f80b13c0 0x7f03f8140d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:52.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f5d9b700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f03f80b13c0 0x7f03f8140d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:57848/0 (socket says 172.21.15.36:57848) 2022-06-17T12:54:52.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f5d9b700 1 -- 172.21.15.36:0/8137475 learned_addr learned my addr 172.21.15.36:0/8137475 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:52.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f6d9d700 1 --2- 172.21.15.36:0/8137475 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f03f80b2c10 0x7f03f81299a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:52.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f5d9b700 1 -- 172.21.15.36:0/8137475 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f03f80b2c10 msgr2=0x7f03f81299a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:52.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f5d9b700 1 --2- 172.21.15.36:0/8137475 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f03f80b2c10 0x7f03f81299a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:52.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f5d9b700 1 -- 172.21.15.36:0/8137475 >> v1:172.21.15.70:6789/0 conn(0x7f03f8060ac0 legacy=0x7f03f80b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:52.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f5d9b700 1 -- 172.21.15.36:0/8137475 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f80b0810 con 0x7f03f80b13c0 2022-06-17T12:54:52.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f5d9b700 1 --2- 172.21.15.36:0/8137475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03f80b13c0 0x7f03f8140d30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f03e8002f70 tx=0x7f03e8006eb0).ready entity=mon.1 client_cookie=7565fcf51fee803 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:52.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f559a700 1 -- 172.21.15.36:0/8137475 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03e800d570 con 0x7f03f80b13c0 2022-06-17T12:54:52.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f559a700 1 -- 172.21.15.36:0/8137475 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e8000e90 con 0x7f03f80b13c0 2022-06-17T12:54:52.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f559a700 1 -- 172.21.15.36:0/8137475 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03e800d900 con 0x7f03f80b13c0 2022-06-17T12:54:52.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f759e700 1 -- 172.21.15.36:0/8137475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03f80b13c0 msgr2=0x7f03f8140d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:52.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f759e700 1 --2- 172.21.15.36:0/8137475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03f80b13c0 0x7f03f8140d30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f03e8002f70 tx=0x7f03e8006eb0).stop 2022-06-17T12:54:52.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f759e700 1 -- 172.21.15.36:0/8137475 shutdown_connections 2022-06-17T12:54:52.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f759e700 1 --2- 172.21.15.36:0/8137475 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f03f80b2c10 0x7f03f81299a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:52.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f759e700 1 --2- 172.21.15.36:0/8137475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03f80b13c0 0x7f03f8140d30 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:52.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f759e700 1 -- 172.21.15.36:0/8137475 >> 172.21.15.36:0/8137475 conn(0x7f03f80b64f0 msgr2=0x7f03f80b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:52.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f759e700 1 -- 172.21.15.36:0/8137475 shutdown_connections 2022-06-17T12:54:52.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.719+0000 7f03f759e700 1 -- 172.21.15.36:0/8137475 wait complete. 2022-06-17T12:54:52.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.723+0000 7f03f759e700 1 Processor -- start 2022-06-17T12:54:52.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.723+0000 7f03f759e700 1 -- start start 2022-06-17T12:54:52.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.723+0000 7f03f759e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03f8060ac0 0x7f03f8149060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:52.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.723+0000 7f03f759e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f03f80b13c0 0x7f03f80670a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:52.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.723+0000 7f03f759e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f03f80b2c10 0x7f03f8140160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:52.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.723+0000 7f03f759e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f80b4440 con 0x7f03f80b2c10 2022-06-17T12:54:52.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.723+0000 7f03f759e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f80b46e0 con 0x7f03f8060ac0 2022-06-17T12:54:52.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.723+0000 7f03f759e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f8148bc0 con 0x7f03f80b13c0 2022-06-17T12:54:52.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.723+0000 7f03f6d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f03f80b2c10 0x7f03f8140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:52.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.723+0000 7f03f6d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f03f80b2c10 0x7f03f8140160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37474/0 (socket says 172.21.15.36:37474) 2022-06-17T12:54:52.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.723+0000 7f03f6d9d700 1 -- 172.21.15.36:0/1853655935 learned_addr learned my addr 172.21.15.36:0/1853655935 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:52.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.723+0000 7f03f5d9b700 1 --2- 172.21.15.36:0/1853655935 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f03f80b13c0 0x7f03f80670a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:52.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.723+0000 7f03f659c700 1 --2- 172.21.15.36:0/1853655935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03f8060ac0 0x7f03f8149060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:52.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.723+0000 7f03f6d9d700 1 -- 172.21.15.36:0/1853655935 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f03f80b13c0 msgr2=0x7f03f80670a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:52.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.723+0000 7f03f6d9d700 1 --2- 172.21.15.36:0/1853655935 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f03f80b13c0 0x7f03f80670a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:52.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.723+0000 7f03f6d9d700 1 -- 172.21.15.36:0/1853655935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03f8060ac0 msgr2=0x7f03f8149060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:52.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.723+0000 7f03f6d9d700 1 --2- 172.21.15.36:0/1853655935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03f8060ac0 0x7f03f8149060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:52.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.723+0000 7f03f6d9d700 1 -- 172.21.15.36:0/1853655935 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e800b1e0 con 0x7f03f80b2c10 2022-06-17T12:54:52.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.723+0000 7f03f6d9d700 1 --2- 172.21.15.36:0/1853655935 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f03f80b2c10 0x7f03f8140160 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7f03f0006e00 tx=0x7f03f00091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:52.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.727+0000 7f03e77fe700 1 -- 172.21.15.36:0/1853655935 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03f0012600 con 0x7f03f80b2c10 2022-06-17T12:54:52.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.727+0000 7f03e77fe700 1 -- 172.21.15.36:0/1853655935 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f0012760 con 0x7f03f80b2c10 2022-06-17T12:54:52.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.727+0000 7f03e77fe700 1 -- 172.21.15.36:0/1853655935 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03f0012ac0 con 0x7f03f80b2c10 2022-06-17T12:54:52.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.727+0000 7f03f759e700 1 -- 172.21.15.36:0/1853655935 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f80afd90 con 0x7f03f80b2c10 2022-06-17T12:54:52.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.727+0000 7f03f759e700 1 -- 172.21.15.36:0/1853655935 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f8064880 con 0x7f03f80b2c10 2022-06-17T12:54:52.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.727+0000 7f03f759e700 1 -- 172.21.15.36:0/1853655935 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f8148d80 con 0x7f03f80b2c10 2022-06-17T12:54:52.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.727+0000 7f03e77fe700 1 -- 172.21.15.36:0/1853655935 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f03f0022070 con 0x7f03f80b2c10 2022-06-17T12:54:52.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.727+0000 7f03e77fe700 1 --2- 172.21.15.36:0/1853655935 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f03d4053920 0x7f03d4055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:52.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.731+0000 7f03e77fe700 1 -- 172.21.15.36:0/1853655935 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f03f006c420 con 0x7f03f80b2c10 2022-06-17T12:54:52.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.731+0000 7f03f659c700 1 --2- 172.21.15.36:0/1853655935 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f03d4053920 0x7f03d4055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:52.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.731+0000 7f03f659c700 1 --2- 172.21.15.36:0/1853655935 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f03d4053920 0x7f03d4055de0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f03ec00d000 tx=0x7f03ec009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:52.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.739+0000 7f03e77fe700 1 -- 172.21.15.36:0/1853655935 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f03f0043420 con 0x7f03f80b2c10 2022-06-17T12:54:52.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:52.907+0000 7f03f759e700 1 -- 172.21.15.36:0/1853655935 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]} v 0) v1 -- 0x7f03f8141220 con 0x7f03f80b2c10 2022-06-17T12:54:53.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.603+0000 7f03e77fe700 1 -- 172.21.15.36:0/1853655935 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]}]=0 v368) v1 ==== 130+0+0 (secure 0 0 0) 0x7f03f0016020 con 0x7f03f80b2c10 2022-06-17T12:54:53.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.607+0000 7f03f759e700 1 -- 172.21.15.36:0/1853655935 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f03d4053920 msgr2=0x7f03d4055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:53.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.607+0000 7f03f759e700 1 --2- 172.21.15.36:0/1853655935 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f03d4053920 0x7f03d4055de0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f03ec00d000 tx=0x7f03ec009040).stop 2022-06-17T12:54:53.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.607+0000 7f03f759e700 1 -- 172.21.15.36:0/1853655935 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f03f80b2c10 msgr2=0x7f03f8140160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:53.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.607+0000 7f03f759e700 1 --2- 172.21.15.36:0/1853655935 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f03f80b2c10 0x7f03f8140160 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7f03f0006e00 tx=0x7f03f00091d0).stop 2022-06-17T12:54:53.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.607+0000 7f03f759e700 1 -- 172.21.15.36:0/1853655935 shutdown_connections 2022-06-17T12:54:53.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.607+0000 7f03f759e700 1 --2- 172.21.15.36:0/1853655935 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f03d4053920 0x7f03d4055de0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:53.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.607+0000 7f03f759e700 1 --2- 172.21.15.36:0/1853655935 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f03f80b2c10 0x7f03f8140160 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:53.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.607+0000 7f03f759e700 1 --2- 172.21.15.36:0/1853655935 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f03f80b13c0 0x7f03f80670a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:53.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.607+0000 7f03f759e700 1 --2- 172.21.15.36:0/1853655935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03f8060ac0 0x7f03f8149060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:53.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.607+0000 7f03f759e700 1 -- 172.21.15.36:0/1853655935 >> 172.21.15.36:0/1853655935 conn(0x7f03f80b64f0 msgr2=0x7f03f81477f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:53.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.607+0000 7f03f759e700 1 -- 172.21.15.36:0/1853655935 shutdown_connections 2022-06-17T12:54:53.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.607+0000 7f03f759e700 1 -- 172.21.15.36:0/1853655935 wait complete. 2022-06-17T12:54:53.612 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-06-17T12:54:53.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.747+0000 7fe140183700 1 Processor -- start 2022-06-17T12:54:53.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.747+0000 7fe140183700 1 -- start start 2022-06-17T12:54:53.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.747+0000 7fe140183700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe138060e20 0x7fe13814dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:53.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.747+0000 7fe140183700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1380b1780 con 0x7fe138066ca0 2022-06-17T12:54:53.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.747+0000 7fe140183700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1380b19b0 con 0x7fe1380afd20 2022-06-17T12:54:53.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.747+0000 7fe140183700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe138067080 con 0x7fe138060e20 2022-06-17T12:54:53.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.747+0000 7fe13df1f700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fe138066ca0 0x7fe1380ada50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55010/0 (socket says 172.21.15.36:55010) 2022-06-17T12:54:53.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.747+0000 7fe13df1f700 1 -- 172.21.15.36:0/3063627785 learned_addr learned my addr 172.21.15.36:0/3063627785 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:53.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.747+0000 7fe13e720700 1 --2- 172.21.15.36:0/3063627785 >> v2:172.21.15.36:3300/0 conn(0x7fe138060e20 0x7fe13814dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:53.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.747+0000 7fe13cf1d700 1 -- 172.21.15.36:0/3063627785 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441073266 0 0) 0x7fe1380b1780 con 0x7fe138066ca0 2022-06-17T12:54:53.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.747+0000 7fe13cf1d700 1 -- 172.21.15.36:0/3063627785 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe120003260 con 0x7fe138066ca0 2022-06-17T12:54:53.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.747+0000 7fe13e720700 1 -- 172.21.15.36:0/3063627785 >> v1:172.21.15.70:6789/0 conn(0x7fe1380afd20 legacy=0x7fe1380ae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:53.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.747+0000 7fe13cf1d700 1 -- 172.21.15.36:0/3063627785 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3562853254 0 0) 0x7fe1380b19b0 con 0x7fe1380afd20 2022-06-17T12:54:53.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.747+0000 7fe13e720700 1 -- 172.21.15.36:0/3063627785 >> v1:172.21.15.36:6789/0 conn(0x7fe138066ca0 legacy=0x7fe1380ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:53.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.747+0000 7fe13e720700 1 -- 172.21.15.36:0/3063627785 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1380b0820 con 0x7fe138060e20 2022-06-17T12:54:53.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.747+0000 7fe13e720700 1 --2- 172.21.15.36:0/3063627785 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe138060e20 0x7fe13814dc60 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7fe1300032f0 tx=0x7fe1300091d0).ready entity=mon.0 client_cookie=812d0cbd648cb5ad server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:53.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.747+0000 7fe13cf1d700 1 -- 172.21.15.36:0/3063627785 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe13000b6c0 con 0x7fe138060e20 2022-06-17T12:54:53.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.747+0000 7fe13cf1d700 1 -- 172.21.15.36:0/3063627785 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13000b820 con 0x7fe138060e20 2022-06-17T12:54:53.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.747+0000 7fe13cf1d700 1 -- 172.21.15.36:0/3063627785 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe13000b990 con 0x7fe138060e20 2022-06-17T12:54:53.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.751+0000 7fe140183700 1 -- 172.21.15.36:0/3063627785 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe138060e20 msgr2=0x7fe13814dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:53.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.751+0000 7fe140183700 1 --2- 172.21.15.36:0/3063627785 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe138060e20 0x7fe13814dc60 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7fe1300032f0 tx=0x7fe1300091d0).stop 2022-06-17T12:54:53.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.751+0000 7fe140183700 1 -- 172.21.15.36:0/3063627785 shutdown_connections 2022-06-17T12:54:53.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.751+0000 7fe140183700 1 --2- 172.21.15.36:0/3063627785 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe138060e20 0x7fe13814dc60 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:53.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.751+0000 7fe140183700 1 -- 172.21.15.36:0/3063627785 >> 172.21.15.36:0/3063627785 conn(0x7fe13805c550 msgr2=0x7fe1380b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:53.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.751+0000 7fe140183700 1 -- 172.21.15.36:0/3063627785 shutdown_connections 2022-06-17T12:54:53.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.751+0000 7fe140183700 1 -- 172.21.15.36:0/3063627785 wait complete. 2022-06-17T12:54:53.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.751+0000 7fe140183700 1 Processor -- start 2022-06-17T12:54:53.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.751+0000 7fe140183700 1 -- start start 2022-06-17T12:54:53.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.751+0000 7fe140183700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe138060e20 0x7fe1380a2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:53.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.751+0000 7fe140183700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe138066ca0 0x7fe1380a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:53.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.751+0000 7fe140183700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1380afd20 0x7fe138152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:53.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.751+0000 7fe140183700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1380a2410 con 0x7fe138060e20 2022-06-17T12:54:53.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.751+0000 7fe140183700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1380a5490 con 0x7fe138066ca0 2022-06-17T12:54:53.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.751+0000 7fe140183700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1380a51f0 con 0x7fe1380afd20 2022-06-17T12:54:53.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.755+0000 7fe13df1f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe138060e20 0x7fe1380a2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:53.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.755+0000 7fe13df1f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe138060e20 0x7fe1380a2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37486/0 (socket says 172.21.15.36:37486) 2022-06-17T12:54:53.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.755+0000 7fe13df1f700 1 -- 172.21.15.36:0/127496834 learned_addr learned my addr 172.21.15.36:0/127496834 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:53.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.755+0000 7fe13d71e700 1 --2- 172.21.15.36:0/127496834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe138066ca0 0x7fe1380a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:53.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.755+0000 7fe13e720700 1 --2- 172.21.15.36:0/127496834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1380afd20 0x7fe138152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:53.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.755+0000 7fe13df1f700 1 -- 172.21.15.36:0/127496834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1380afd20 msgr2=0x7fe138152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:53.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.755+0000 7fe13df1f700 1 --2- 172.21.15.36:0/127496834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1380afd20 0x7fe138152370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:53.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.755+0000 7fe13df1f700 1 -- 172.21.15.36:0/127496834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe138066ca0 msgr2=0x7fe1380a2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:53.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.755+0000 7fe13df1f700 1 --2- 172.21.15.36:0/127496834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe138066ca0 0x7fe1380a2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:53.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.755+0000 7fe13df1f700 1 -- 172.21.15.36:0/127496834 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe130009470 con 0x7fe138060e20 2022-06-17T12:54:53.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.755+0000 7fe13d71e700 1 --2- 172.21.15.36:0/127496834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe138066ca0 0x7fe1380a2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:53.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.755+0000 7fe13df1f700 1 --2- 172.21.15.36:0/127496834 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe138060e20 0x7fe1380a2890 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7fe134001110 tx=0x7fe134007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:53.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.755+0000 7fe126ffd700 1 -- 172.21.15.36:0/127496834 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1340070d0 con 0x7fe138060e20 2022-06-17T12:54:53.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.755+0000 7fe126ffd700 1 -- 172.21.15.36:0/127496834 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe134007230 con 0x7fe138060e20 2022-06-17T12:54:53.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.755+0000 7fe140183700 1 -- 172.21.15.36:0/127496834 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1380a0d20 con 0x7fe138060e20 2022-06-17T12:54:53.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.755+0000 7fe126ffd700 1 -- 172.21.15.36:0/127496834 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe134005610 con 0x7fe138060e20 2022-06-17T12:54:53.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.755+0000 7fe140183700 1 -- 172.21.15.36:0/127496834 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1380a2190 con 0x7fe138060e20 2022-06-17T12:54:53.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.759+0000 7fe140183700 1 -- 172.21.15.36:0/127496834 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1380af630 con 0x7fe138060e20 2022-06-17T12:54:53.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.759+0000 7fe126ffd700 1 -- 172.21.15.36:0/127496834 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe13401a070 con 0x7fe138060e20 2022-06-17T12:54:53.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.759+0000 7fe126ffd700 1 --2- 172.21.15.36:0/127496834 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe1180539b0 0x7fe118055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:53.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.759+0000 7fe126ffd700 1 -- 172.21.15.36:0/127496834 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fe13406afe0 con 0x7fe138060e20 2022-06-17T12:54:53.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.759+0000 7fe13d71e700 1 --2- 172.21.15.36:0/127496834 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe1180539b0 0x7fe118055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:53.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.759+0000 7fe13d71e700 1 --2- 172.21.15.36:0/127496834 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe1180539b0 0x7fe118055e70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7fe12800b000 tx=0x7fe128009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:53.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.763+0000 7fe126ffd700 1 -- 172.21.15.36:0/127496834 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe134042060 con 0x7fe138060e20 2022-06-17T12:54:53.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.891+0000 7fe140183700 1 -- 172.21.15.36:0/127496834 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe138057410 con 0x7fe138060e20 2022-06-17T12:54:53.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.899+0000 7fe126ffd700 1 -- 172.21.15.36:0/127496834 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe134018020 con 0x7fe138060e20 2022-06-17T12:54:53.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.903+0000 7fe140183700 1 -- 172.21.15.36:0/127496834 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe1180539b0 msgr2=0x7fe118055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:53.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.903+0000 7fe140183700 1 --2- 172.21.15.36:0/127496834 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe1180539b0 0x7fe118055e70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7fe12800b000 tx=0x7fe128009040).stop 2022-06-17T12:54:53.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.903+0000 7fe140183700 1 -- 172.21.15.36:0/127496834 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe138060e20 msgr2=0x7fe1380a2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:53.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.903+0000 7fe140183700 1 --2- 172.21.15.36:0/127496834 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe138060e20 0x7fe1380a2890 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7fe134001110 tx=0x7fe134007aa0).stop 2022-06-17T12:54:53.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.903+0000 7fe140183700 1 -- 172.21.15.36:0/127496834 shutdown_connections 2022-06-17T12:54:53.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.903+0000 7fe140183700 1 --2- 172.21.15.36:0/127496834 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe1180539b0 0x7fe118055e70 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:53.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.903+0000 7fe140183700 1 --2- 172.21.15.36:0/127496834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1380afd20 0x7fe138152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:53.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.903+0000 7fe140183700 1 --2- 172.21.15.36:0/127496834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe138066ca0 0x7fe1380a2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:53.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.903+0000 7fe140183700 1 --2- 172.21.15.36:0/127496834 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe138060e20 0x7fe1380a2890 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:53.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.903+0000 7fe140183700 1 -- 172.21.15.36:0/127496834 >> 172.21.15.36:0/127496834 conn(0x7fe13805c550 msgr2=0x7fe1380b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:53.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.903+0000 7fe140183700 1 -- 172.21.15.36:0/127496834 shutdown_connections 2022-06-17T12:54:53.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:53.903+0000 7fe140183700 1 -- 172.21.15.36:0/127496834 wait complete. 2022-06-17T12:54:53.906 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-17T12:54:53.907 DEBUG:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:53.935 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:53.957 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:53.980 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:54.002 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:54:54.028 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-17T12:54:54.029 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:54.030 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:54.031 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:54:54.039 INFO:teuthology.orchestra.run.smithi107.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-17T12:54:54.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:54:54.043 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-06-17T12:54:54.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:54.045 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:54.046 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:54:54.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:54:54.059 INFO:teuthology.orchestra.run.smithi136.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-17T12:54:54.059 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-17T12:54:54.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:54:54.061 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:54:54.062 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:54:54.078 INFO:teuthology.orchestra.run.smithi143.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:54:54.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:54:54.081 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-06-17T12:54:54.083 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-17T12:54:54.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.207+0000 7f4f289fb700 1 Processor -- start 2022-06-17T12:54:54.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.207+0000 7f4f289fb700 1 -- start start 2022-06-17T12:54:54.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.207+0000 7f4f289fb700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4f240ac340 0x7f4f2414b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:54.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.207+0000 7f4f289fb700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f240ac870 con 0x7f4f240af7f0 2022-06-17T12:54:54.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.207+0000 7f4f289fb700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f240acaa0 con 0x7f4f240ad140 2022-06-17T12:54:54.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.207+0000 7f4f289fb700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f24148960 con 0x7f4f240ac340 2022-06-17T12:54:54.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.207+0000 7f4f22d9d700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f4f240af7f0 0x7f4f24147b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55028/0 (socket says 172.21.15.36:55028) 2022-06-17T12:54:54.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.207+0000 7f4f22d9d700 1 -- 172.21.15.36:0/3867269223 learned_addr learned my addr 172.21.15.36:0/3867269223 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:54.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.207+0000 7f4f21d9b700 1 --2- 172.21.15.36:0/3867269223 >> v2:172.21.15.70:3300/0 conn(0x7f4f240ac340 0x7f4f2414b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:54.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.207+0000 7f4f21d9b700 1 -- 172.21.15.36:0/3867269223 >> v1:172.21.15.70:6789/0 conn(0x7f4f240ad140 legacy=0x7f4f2414aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:54.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.207+0000 7f4f21d9b700 1 -- 172.21.15.36:0/3867269223 >> v1:172.21.15.36:6789/0 conn(0x7f4f240af7f0 legacy=0x7f4f24147b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:54.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.207+0000 7f4f2159a700 1 -- 172.21.15.36:0/3867269223 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2302920382 0 0) 0x7f4f240ac870 con 0x7f4f240af7f0 2022-06-17T12:54:54.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.207+0000 7f4f21d9b700 1 -- 172.21.15.36:0/3867269223 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f240b0bf0 con 0x7f4f240ac340 2022-06-17T12:54:54.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.207+0000 7f4f21d9b700 1 --2- 172.21.15.36:0/3867269223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f240ac340 0x7f4f2414b3b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f4f1c0032f0 tx=0x7f4f1c0091d0).ready entity=mon.1 client_cookie=71f7e649cc822c8a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:54.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.207+0000 7f4f2159a700 1 -- 172.21.15.36:0/3867269223 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f1c009ea0 con 0x7f4f240ac340 2022-06-17T12:54:54.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.207+0000 7f4f2159a700 1 -- 172.21.15.36:0/3867269223 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f1c00b8a0 con 0x7f4f240ac340 2022-06-17T12:54:54.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.207+0000 7f4f2159a700 1 -- 172.21.15.36:0/3867269223 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f1c00ba10 con 0x7f4f240ac340 2022-06-17T12:54:54.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.207+0000 7f4f289fb700 1 -- 172.21.15.36:0/3867269223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f240ac340 msgr2=0x7f4f2414b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:54.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.207+0000 7f4f289fb700 1 --2- 172.21.15.36:0/3867269223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f240ac340 0x7f4f2414b3b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f4f1c0032f0 tx=0x7f4f1c0091d0).stop 2022-06-17T12:54:54.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.207+0000 7f4f289fb700 1 -- 172.21.15.36:0/3867269223 shutdown_connections 2022-06-17T12:54:54.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.207+0000 7f4f289fb700 1 --2- 172.21.15.36:0/3867269223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f240ac340 0x7f4f2414b3b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:54.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.207+0000 7f4f289fb700 1 -- 172.21.15.36:0/3867269223 >> 172.21.15.36:0/3867269223 conn(0x7f4f240b5430 msgr2=0x7f4f240b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:54.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.207+0000 7f4f289fb700 1 -- 172.21.15.36:0/3867269223 shutdown_connections 2022-06-17T12:54:54.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.207+0000 7f4f289fb700 1 -- 172.21.15.36:0/3867269223 wait complete. 2022-06-17T12:54:54.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.211+0000 7f4f289fb700 1 Processor -- start 2022-06-17T12:54:54.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.211+0000 7f4f289fb700 1 -- start start 2022-06-17T12:54:54.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.211+0000 7f4f289fb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4f240ac340 0x7f4f240636e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:54.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.211+0000 7f4f289fb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f240ad140 0x7f4f24063bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:54.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.211+0000 7f4f289fb700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f240af7f0 0x7f4f240aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:54.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.211+0000 7f4f289fb700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f241435b0 con 0x7f4f240ac340 2022-06-17T12:54:54.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.211+0000 7f4f289fb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2415af90 con 0x7f4f240ad140 2022-06-17T12:54:54.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.211+0000 7f4f289fb700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f240a9480 con 0x7f4f240af7f0 2022-06-17T12:54:54.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.211+0000 7f4f21d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f240ad140 0x7f4f24063bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:54.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.211+0000 7f4f2259c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4f240ac340 0x7f4f240636e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:54.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.211+0000 7f4f21d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f240ad140 0x7f4f24063bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:57882/0 (socket says 172.21.15.36:57882) 2022-06-17T12:54:54.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.211+0000 7f4f21d9b700 1 -- 172.21.15.36:0/2512911608 learned_addr learned my addr 172.21.15.36:0/2512911608 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:54.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.211+0000 7f4f22d9d700 1 --2- 172.21.15.36:0/2512911608 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f240af7f0 0x7f4f240aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:54.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.211+0000 7f4f21d9b700 1 -- 172.21.15.36:0/2512911608 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f240af7f0 msgr2=0x7f4f240aa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:54.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.211+0000 7f4f21d9b700 1 --2- 172.21.15.36:0/2512911608 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f240af7f0 0x7f4f240aa5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:54.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.211+0000 7f4f21d9b700 1 -- 172.21.15.36:0/2512911608 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4f240ac340 msgr2=0x7f4f240636e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:54.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.211+0000 7f4f21d9b700 1 --2- 172.21.15.36:0/2512911608 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4f240ac340 0x7f4f240636e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:54.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.211+0000 7f4f21d9b700 1 -- 172.21.15.36:0/2512911608 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c009470 con 0x7f4f240ad140 2022-06-17T12:54:54.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.211+0000 7f4f21d9b700 1 --2- 172.21.15.36:0/2512911608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f240ad140 0x7f4f24063bd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f4f1c0032f0 tx=0x7f4f1c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:54.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.211+0000 7f4f1b7fe700 1 -- 172.21.15.36:0/2512911608 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f1c009ea0 con 0x7f4f240ad140 2022-06-17T12:54:54.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.211+0000 7f4f1b7fe700 1 -- 172.21.15.36:0/2512911608 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f1c01f3f0 con 0x7f4f240ad140 2022-06-17T12:54:54.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.211+0000 7f4f289fb700 1 -- 172.21.15.36:0/2512911608 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f240a9720 con 0x7f4f240ad140 2022-06-17T12:54:54.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.211+0000 7f4f1b7fe700 1 -- 172.21.15.36:0/2512911608 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f1c011ea0 con 0x7f4f240ad140 2022-06-17T12:54:54.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.211+0000 7f4f289fb700 1 -- 172.21.15.36:0/2512911608 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f2415aab0 con 0x7f4f240ad140 2022-06-17T12:54:54.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.215+0000 7f4f289fb700 1 -- 172.21.15.36:0/2512911608 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f240b1500 con 0x7f4f240ad140 2022-06-17T12:54:54.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.215+0000 7f4f1b7fe700 1 -- 172.21.15.36:0/2512911608 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f4f1c026030 con 0x7f4f240ad140 2022-06-17T12:54:54.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.215+0000 7f4f1b7fe700 1 --2- 172.21.15.36:0/2512911608 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4efc053ad0 0x7f4efc055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:54.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.215+0000 7f4f1b7fe700 1 -- 172.21.15.36:0/2512911608 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f4f1c06c990 con 0x7f4f240ad140 2022-06-17T12:54:54.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.215+0000 7f4f2259c700 1 --2- 172.21.15.36:0/2512911608 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4efc053ad0 0x7f4efc055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:54.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.215+0000 7f4f2259c700 1 --2- 172.21.15.36:0/2512911608 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4efc053ad0 0x7f4efc055f90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f4f0c00d000 tx=0x7f4f0c007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:54.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.223+0000 7f4f1b7fe700 1 -- 172.21.15.36:0/2512911608 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4f1c043960 con 0x7f4f240ad140 2022-06-17T12:54:54.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.351+0000 7f4f289fb700 1 -- 172.21.15.36:0/2512911608 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f4f24057410 con 0x7f4f240ad140 2022-06-17T12:54:54.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.355+0000 7f4f1b7fe700 1 -- 172.21.15.36:0/2512911608 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v97) v1 ==== 81+0+29220 (secure 0 0 0) 0x7f4f1c024090 con 0x7f4f240ad140 2022-06-17T12:54:54.348 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:54.361 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:54:54.361 INFO:teuthology.orchestra.run.smithi036.stdout: "epoch": 97, 2022-06-17T12:54:54.361 INFO:teuthology.orchestra.run.smithi036.stdout: "fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", 2022-06-17T12:54:54.362 INFO:teuthology.orchestra.run.smithi036.stdout: "created": "2022-06-17T12:46:30.671408+0000", 2022-06-17T12:54:54.362 INFO:teuthology.orchestra.run.smithi036.stdout: "modified": "2022-06-17T12:54:51.574663+0000", 2022-06-17T12:54:54.362 INFO:teuthology.orchestra.run.smithi036.stdout: "last_up_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:54:54.362 INFO:teuthology.orchestra.run.smithi036.stdout: "last_in_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:54:54.363 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-17T12:54:54.363 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_num": 5799936, 2022-06-17T12:54:54.363 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_set": [ 2022-06-17T12:54:54.363 INFO:teuthology.orchestra.run.smithi036.stdout: "pglog_hardlimit", 2022-06-17T12:54:54.364 INFO:teuthology.orchestra.run.smithi036.stdout: "purged_snapdirs", 2022-06-17T12:54:54.364 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_deletes", 2022-06-17T12:54:54.364 INFO:teuthology.orchestra.run.smithi036.stdout: "sortbitwise" 2022-06-17T12:54:54.364 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:54:54.364 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_version": 6, 2022-06-17T12:54:54.365 INFO:teuthology.orchestra.run.smithi036.stdout: "full_ratio": 0.94999998807907104, 2022-06-17T12:54:54.365 INFO:teuthology.orchestra.run.smithi036.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-17T12:54:54.365 INFO:teuthology.orchestra.run.smithi036.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-17T12:54:54.366 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_snapshot": "", 2022-06-17T12:54:54.366 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_max": 18, 2022-06-17T12:54:54.367 INFO:teuthology.orchestra.run.smithi036.stdout: "max_osd": 8, 2022-06-17T12:54:54.367 INFO:teuthology.orchestra.run.smithi036.stdout: "require_min_compat_client": "jewel", 2022-06-17T12:54:54.367 INFO:teuthology.orchestra.run.smithi036.stdout: "min_compat_client": "jewel", 2022-06-17T12:54:54.367 INFO:teuthology.orchestra.run.smithi036.stdout: "require_osd_release": "octopus", 2022-06-17T12:54:54.367 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:54:54.368 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.368 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 1, 2022-06-17T12:54:54.368 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "device_health_metrics", 2022-06-17T12:54:54.368 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:33.881212+0000", 2022-06-17T12:54:54.369 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:54:54.369 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:54:54.369 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:54:54.369 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:54:54.370 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:54:54.370 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:54:54.370 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:54:54.370 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:54:54.371 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 1, 2022-06-17T12:54:54.371 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 1, 2022-06-17T12:54:54.371 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 1, 2022-06-17T12:54:54.371 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 1, 2022-06-17T12:54:54.372 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 1, 2022-06-17T12:54:54.372 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:54:54.372 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:54:54.372 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:54:54.373 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:54:54.373 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:54:54.373 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:54:54.373 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:54:54.374 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.374 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "16", 2022-06-17T12:54:54.374 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:54:54.374 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:54:54.374 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:54:54.375 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:54:54.375 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:54:54.375 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:54:54.375 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:54:54.376 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:54:54.376 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:54:54.376 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:54:54.377 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:54:54.377 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:54:54.377 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:54:54.377 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:54:54.378 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:54:54.378 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:54:54.378 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:54:54.378 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:54:54.379 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:54:54.379 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:54:54.379 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:54:54.379 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:54:54.380 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:54:54.380 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:54:54.380 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:54:54.380 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:54:54.380 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.381 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:54:54.381 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:54:54.381 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:54:54.381 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:54:54.381 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:54:54.382 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:54:54.382 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:54:54.382 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:54:54.382 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:54:54.383 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:54:54.383 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:54:54.383 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:54:54.383 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 1 2022-06-17T12:54:54.384 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.384 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:54:54.384 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr_devicehealth": {} 2022-06-17T12:54:54.385 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.385 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.385 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.385 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 2, 2022-06-17T12:54:54.386 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "rbd", 2022-06-17T12:54:54.386 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:44.485460+0000", 2022-06-17T12:54:54.386 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:54:54.386 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:54:54.387 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:54:54.387 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:54:54.387 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:54:54.387 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:54:54.387 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:54:54.388 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:54:54.388 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:54:54.388 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:54:54.388 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:54:54.389 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:54:54.389 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:54:54.389 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:54:54.389 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:54:54.390 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:54:54.390 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:54:54.390 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:54:54.390 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:54:54.390 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:54:54.391 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.391 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "19", 2022-06-17T12:54:54.391 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:54:54.391 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:54:54.392 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:54:54.392 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:54:54.392 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:54:54.392 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:54:54.392 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:54:54.393 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:54:54.393 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:54:54.393 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:54:54.393 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:54:54.394 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:54:54.394 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:54:54.394 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:54:54.394 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:54:54.395 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:54:54.395 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:54:54.395 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:54:54.395 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:54:54.395 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:54:54.396 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:54:54.396 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:54:54.403 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:54:54.403 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:54:54.404 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:54:54.404 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:54:54.404 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.404 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:54:54.405 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:54:54.405 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:54:54.405 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:54:54.406 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:54:54.406 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:54:54.406 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:54:54.406 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:54:54.406 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:54:54.407 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:54:54.407 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:54:54.407 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:54:54.407 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:54:54.408 INFO:teuthology.orchestra.run.smithi036.stdout: "rbd": {} 2022-06-17T12:54:54.408 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.408 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.408 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:54:54.408 INFO:teuthology.orchestra.run.smithi036.stdout: "osds": [ 2022-06-17T12:54:54.409 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.409 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:54:54.409 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", 2022-06-17T12:54:54.409 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:54.409 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:54.410 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:54.410 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:54.410 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:54.410 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:54.411 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:54:54.411 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 91, 2022-06-17T12:54:54.411 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:54.411 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:54.412 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:54.412 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.412 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.412 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.412 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6802", 2022-06-17T12:54:54.413 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:54.413 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.413 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.413 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.414 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6803", 2022-06-17T12:54:54.414 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:54.414 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.414 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.415 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.415 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:54.415 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.415 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.415 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.416 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6804", 2022-06-17T12:54:54.416 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:54.416 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.416 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.417 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.417 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6805", 2022-06-17T12:54:54.417 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:54.417 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.417 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.418 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.418 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:54.418 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.418 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.419 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.419 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6808", 2022-06-17T12:54:54.419 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:54.419 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.419 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.420 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.420 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6809", 2022-06-17T12:54:54.421 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:54.421 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.422 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.422 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.423 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:54.423 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.423 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.424 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.424 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6806", 2022-06-17T12:54:54.425 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:54.425 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.425 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.426 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.426 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6807", 2022-06-17T12:54:54.426 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:54:54.427 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.427 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.427 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.428 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6803/22619", 2022-06-17T12:54:54.428 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6805/22619", 2022-06-17T12:54:54.429 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6809/22619", 2022-06-17T12:54:54.429 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6807/22619", 2022-06-17T12:54:54.429 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:54.430 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:54.430 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:54.430 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.430 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.431 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.431 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:54:54.431 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", 2022-06-17T12:54:54.431 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:54.432 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:54.432 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:54.432 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:54.432 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:54.433 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:54.433 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:54:54.433 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 91, 2022-06-17T12:54:54.433 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:54.433 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:54.434 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:54.434 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.435 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.435 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.436 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6810", 2022-06-17T12:54:54.436 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:54.436 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.437 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.437 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.438 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6811", 2022-06-17T12:54:54.438 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:54.439 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.439 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.440 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.440 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:54.440 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.441 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.441 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.441 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6812", 2022-06-17T12:54:54.442 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:54.442 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.443 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.443 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.444 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6813", 2022-06-17T12:54:54.444 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:54.444 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.444 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.445 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.445 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:54.445 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.445 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.446 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.446 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6816", 2022-06-17T12:54:54.446 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:54.446 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.447 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.447 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.447 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6817", 2022-06-17T12:54:54.447 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:54.448 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.448 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.448 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.448 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:54.449 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.449 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.449 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.449 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6814", 2022-06-17T12:54:54.449 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:54.450 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.450 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.450 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.450 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6815", 2022-06-17T12:54:54.451 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:54:54.451 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.451 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.451 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.452 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6811/22620", 2022-06-17T12:54:54.452 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6813/22620", 2022-06-17T12:54:54.452 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6817/22620", 2022-06-17T12:54:54.452 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6815/22620", 2022-06-17T12:54:54.452 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:54.453 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:54.453 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:54.453 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.453 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.454 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.454 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:54:54.454 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", 2022-06-17T12:54:54.454 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:54.455 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:54.455 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:54.455 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:54.456 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:54.456 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:54.456 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:54:54.457 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 92, 2022-06-17T12:54:54.457 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:54.458 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:54.458 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:54.458 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.459 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.459 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.459 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6826", 2022-06-17T12:54:54.460 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:54.460 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.461 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.462 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.462 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6827", 2022-06-17T12:54:54.463 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:54.463 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.464 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.464 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.464 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:54.465 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.465 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.466 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.466 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6828", 2022-06-17T12:54:54.467 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:54.467 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.467 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.468 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.468 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6829", 2022-06-17T12:54:54.469 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:54.469 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.469 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.470 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.470 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:54.471 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.471 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.472 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.472 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6832", 2022-06-17T12:54:54.472 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:54.472 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.473 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.473 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.473 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6833", 2022-06-17T12:54:54.473 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:54.474 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.474 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.475 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.475 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:54.476 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.476 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.476 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.477 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6830", 2022-06-17T12:54:54.477 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:54.478 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.478 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.479 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.479 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6831", 2022-06-17T12:54:54.479 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:54:54.479 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.480 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.480 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.481 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6827/22643", 2022-06-17T12:54:54.481 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6829/22643", 2022-06-17T12:54:54.482 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6833/22643", 2022-06-17T12:54:54.482 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6831/22643", 2022-06-17T12:54:54.483 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:54.483 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:54.484 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:54.484 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.484 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.485 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.485 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:54:54.486 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", 2022-06-17T12:54:54.486 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:54.486 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:54.486 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:54.487 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:54.487 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:54.487 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:54.487 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:54:54.487 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 92, 2022-06-17T12:54:54.488 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:54.488 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:54.488 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:54.488 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.489 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.489 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.489 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6818", 2022-06-17T12:54:54.489 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:54.489 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.490 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.490 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.490 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6819", 2022-06-17T12:54:54.490 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:54.491 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.491 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.491 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.491 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:54.492 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.492 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.492 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.492 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6820", 2022-06-17T12:54:54.492 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:54.493 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.493 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.493 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.493 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6821", 2022-06-17T12:54:54.494 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:54.494 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.494 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.494 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.495 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:54.495 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.495 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.495 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.495 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6824", 2022-06-17T12:54:54.496 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:54.496 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.496 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.497 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.497 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6825", 2022-06-17T12:54:54.497 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:54.497 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.497 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.498 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.498 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:54.498 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.498 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.499 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.499 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6822", 2022-06-17T12:54:54.499 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:54.499 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.499 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.500 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.500 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6823", 2022-06-17T12:54:54.500 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:54:54.500 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.501 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.501 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.501 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6819/22646", 2022-06-17T12:54:54.501 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6821/22646", 2022-06-17T12:54:54.501 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6825/22646", 2022-06-17T12:54:54.502 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6823/22646", 2022-06-17T12:54:54.502 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:54.502 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:54.502 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:54.503 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.503 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.503 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.503 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:54:54.503 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", 2022-06-17T12:54:54.504 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:54.504 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:54.504 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:54.504 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:54.505 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:54.505 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:54.505 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:54:54.505 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 90, 2022-06-17T12:54:54.505 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:54.506 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:54.506 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:54.506 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.506 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.507 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.507 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6808", 2022-06-17T12:54:54.507 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:54.507 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.508 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.508 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.508 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6809", 2022-06-17T12:54:54.508 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:54.508 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.509 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.509 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.509 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:54.509 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.510 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.510 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.510 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6810", 2022-06-17T12:54:54.510 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:54.510 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.511 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.511 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.511 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6811", 2022-06-17T12:54:54.511 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:54.512 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.512 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.512 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.512 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:54.513 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.513 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.513 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.513 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6814", 2022-06-17T12:54:54.514 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:54.514 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.514 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.514 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.515 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6815", 2022-06-17T12:54:54.515 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:54.515 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.515 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.515 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.516 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:54.516 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.516 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.516 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.517 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6812", 2022-06-17T12:54:54.517 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:54.517 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.517 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.518 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.518 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6813", 2022-06-17T12:54:54.518 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:54:54.518 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.518 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.519 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.519 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6809/22812", 2022-06-17T12:54:54.519 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6811/22812", 2022-06-17T12:54:54.519 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6815/22812", 2022-06-17T12:54:54.520 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6813/22812", 2022-06-17T12:54:54.520 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:54.520 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:54.520 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:54.520 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.521 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.521 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.521 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:54:54.521 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", 2022-06-17T12:54:54.522 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:54.522 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:54.522 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:54.522 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:54.523 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:54.523 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:54.523 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:54:54.523 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 82, 2022-06-17T12:54:54.524 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:54.524 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:54.524 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:54.524 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.525 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.525 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.525 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6800", 2022-06-17T12:54:54.525 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:54.526 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.526 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.526 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.526 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6801", 2022-06-17T12:54:54.527 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:54.527 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.527 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.527 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.527 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:54.528 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.528 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.528 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.528 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6802", 2022-06-17T12:54:54.529 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:54.529 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.529 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.529 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.529 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6803", 2022-06-17T12:54:54.530 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:54.530 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.530 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.530 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.531 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:54.531 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.531 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.531 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.532 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6806", 2022-06-17T12:54:54.532 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:54.532 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.532 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.532 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.533 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6807", 2022-06-17T12:54:54.533 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:54.533 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.533 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.533 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.534 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:54.534 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.535 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.535 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.535 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6804", 2022-06-17T12:54:54.536 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:54.536 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.536 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.536 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.537 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6805", 2022-06-17T12:54:54.537 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:54:54.537 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.537 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.537 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.538 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6801/22813", 2022-06-17T12:54:54.538 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6803/22813", 2022-06-17T12:54:54.538 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6807/22813", 2022-06-17T12:54:54.538 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6805/22813", 2022-06-17T12:54:54.539 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:54.539 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:54.539 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:54.539 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.540 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.540 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.540 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:54:54.540 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", 2022-06-17T12:54:54.540 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:54.541 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:54.541 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:54.541 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:54.541 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:54.542 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:54.542 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:54:54.542 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 90, 2022-06-17T12:54:54.542 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:54.542 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:54.543 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:54.543 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.543 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.543 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.544 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6816", 2022-06-17T12:54:54.544 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:54.544 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.544 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.544 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.545 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6817", 2022-06-17T12:54:54.545 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:54.545 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.545 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.546 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.546 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:54.546 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.546 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.546 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.547 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6818", 2022-06-17T12:54:54.547 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:54.547 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.547 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.548 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.548 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6819", 2022-06-17T12:54:54.548 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:54.548 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.549 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.549 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.549 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:54.549 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.549 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.550 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.550 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6822", 2022-06-17T12:54:54.550 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:54.550 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.551 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.551 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.551 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6823", 2022-06-17T12:54:54.551 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:54.551 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.552 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.552 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.552 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:54.552 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.553 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.553 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.553 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6820", 2022-06-17T12:54:54.553 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:54.553 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.554 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.554 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.554 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6821", 2022-06-17T12:54:54.554 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:54:54.555 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.555 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.555 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.556 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6817/22822", 2022-06-17T12:54:54.556 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6819/22822", 2022-06-17T12:54:54.557 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6823/22822", 2022-06-17T12:54:54.557 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6821/22822", 2022-06-17T12:54:54.557 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:54.557 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:54.558 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:54.558 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.559 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.559 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.559 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:54:54.560 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", 2022-06-17T12:54:54.560 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:54:54.561 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:54:54.562 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:54:54.562 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:54:54.563 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:54:54.563 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:54:54.563 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:54:54.564 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 91, 2022-06-17T12:54:54.564 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:54:54.565 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:54:54.565 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:54:54.566 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.566 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.567 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.567 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6824", 2022-06-17T12:54:54.567 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:54.568 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.568 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.569 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.569 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6825", 2022-06-17T12:54:54.570 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:54.570 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.570 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.571 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.571 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:54:54.572 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.572 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.573 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.573 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6826", 2022-06-17T12:54:54.574 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:54.574 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.575 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.575 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.576 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6827", 2022-06-17T12:54:54.576 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:54.576 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.577 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.577 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.578 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:54:54.578 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.579 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.579 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.579 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6830", 2022-06-17T12:54:54.580 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:54.580 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.581 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.581 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.582 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6831", 2022-06-17T12:54:54.582 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:54.583 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.583 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.583 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.584 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:54:54.584 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:54:54.585 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.585 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:54:54.585 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6828", 2022-06-17T12:54:54.586 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:54.586 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.587 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.587 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:54:54.588 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6829", 2022-06-17T12:54:54.588 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:54:54.588 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.589 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.589 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.590 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6825/22837", 2022-06-17T12:54:54.590 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6827/22837", 2022-06-17T12:54:54.591 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6831/22837", 2022-06-17T12:54:54.591 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6829/22837", 2022-06-17T12:54:54.591 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:54:54.592 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:54:54.592 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:54:54.593 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:54.602 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.603 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:54:54.603 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_xinfo": [ 2022-06-17T12:54:54.603 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.604 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:54:54.605 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:54.605 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:54.606 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:54.606 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:54.607 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:54.607 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.291461+0000", 2022-06-17T12:54:54.608 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:54.608 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.609 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.609 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:54:54.609 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:54.610 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:54.610 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:54.611 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:54.611 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:54.612 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.537912+0000", 2022-06-17T12:54:54.612 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:54.612 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.613 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.613 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:54:54.614 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:54.614 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:54.615 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:54.615 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:54.616 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:54.616 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.606947+0000", 2022-06-17T12:54:54.617 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:54.617 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.617 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.618 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:54:54.618 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:54.619 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:54.619 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:54.620 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:54.620 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:54.621 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.514412+0000", 2022-06-17T12:54:54.621 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:54.621 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.622 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.622 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:54:54.622 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:54.623 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:54.623 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:54.623 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:54.624 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:54.624 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.394645+0000", 2022-06-17T12:54:54.625 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:54.625 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.626 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.626 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:54:54.627 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:54.627 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:54.627 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:54.627 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:54.628 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:54.628 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.343806+0000", 2022-06-17T12:54:54.628 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:54.628 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.629 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.629 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:54:54.629 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:54.630 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:54.630 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:54.631 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:54.631 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:54.631 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.591864+0000", 2022-06-17T12:54:54.632 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:54.632 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.632 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:54.633 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:54:54.633 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:54:54.634 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:54:54.634 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:54:54.634 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:54:54.635 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:54:54.635 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.607045+0000", 2022-06-17T12:54:54.635 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:54:54.635 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.636 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:54:54.636 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap": [], 2022-06-17T12:54:54.636 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap_items": [], 2022-06-17T12:54:54.636 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_temp": [], 2022-06-17T12:54:54.637 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_temp": [], 2022-06-17T12:54:54.637 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklist": { 2022-06-17T12:54:54.637 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.36:6835/3267343301": "2022-06-18T12:54:47.952040+0000", 2022-06-17T12:54:54.638 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.36:6834/3267343301": "2022-06-18T12:54:47.952040+0000" 2022-06-17T12:54:54.638 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.639 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profiles": { 2022-06-17T12:54:54.639 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs_data": { 2022-06-17T12:54:54.639 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-device-class": "", 2022-06-17T12:54:54.640 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:54:54.640 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-root": "default", 2022-06-17T12:54:54.640 INFO:teuthology.orchestra.run.smithi036.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-17T12:54:54.641 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:54:54.641 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "2", 2022-06-17T12:54:54.642 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:54:54.642 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:54:54.643 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van", 2022-06-17T12:54:54.643 INFO:teuthology.orchestra.run.smithi036.stdout: "w": "8" 2022-06-17T12:54:54.643 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.644 INFO:teuthology.orchestra.run.smithi036.stdout: "default": { 2022-06-17T12:54:54.644 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:54:54.644 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:54:54.645 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "1", 2022-06-17T12:54:54.646 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:54:54.646 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:54:54.646 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van" 2022-06-17T12:54:54.647 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:54.647 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:54.647 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps_queue": [], 2022-06-17T12:54:54.648 INFO:teuthology.orchestra.run.smithi036.stdout: "new_removed_snaps": [], 2022-06-17T12:54:54.648 INFO:teuthology.orchestra.run.smithi036.stdout: "new_purged_snaps": [], 2022-06-17T12:54:54.648 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_node_flags": {}, 2022-06-17T12:54:54.649 INFO:teuthology.orchestra.run.smithi036.stdout: "device_class_flags": {} 2022-06-17T12:54:54.649 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:54:54.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.355+0000 7f4f289fb700 1 -- 172.21.15.36:0/2512911608 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4efc053ad0 msgr2=0x7f4efc055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:54.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.355+0000 7f4f289fb700 1 --2- 172.21.15.36:0/2512911608 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4efc053ad0 0x7f4efc055f90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f4f0c00d000 tx=0x7f4f0c007b90).stop 2022-06-17T12:54:54.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.355+0000 7f4f289fb700 1 -- 172.21.15.36:0/2512911608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f240ad140 msgr2=0x7f4f24063bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:54.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.355+0000 7f4f289fb700 1 --2- 172.21.15.36:0/2512911608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f240ad140 0x7f4f24063bd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f4f1c0032f0 tx=0x7f4f1c016000).stop 2022-06-17T12:54:54.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.355+0000 7f4f289fb700 1 -- 172.21.15.36:0/2512911608 shutdown_connections 2022-06-17T12:54:54.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.355+0000 7f4f289fb700 1 --2- 172.21.15.36:0/2512911608 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4efc053ad0 0x7f4efc055f90 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:54.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.355+0000 7f4f289fb700 1 --2- 172.21.15.36:0/2512911608 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f240af7f0 0x7f4f240aa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:54.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.355+0000 7f4f289fb700 1 --2- 172.21.15.36:0/2512911608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f240ad140 0x7f4f24063bd0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:54.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.355+0000 7f4f289fb700 1 --2- 172.21.15.36:0/2512911608 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4f240ac340 0x7f4f240636e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:54.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.355+0000 7f4f289fb700 1 -- 172.21.15.36:0/2512911608 >> 172.21.15.36:0/2512911608 conn(0x7f4f240b5430 msgr2=0x7f4f24145fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:54.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.355+0000 7f4f289fb700 1 -- 172.21.15.36:0/2512911608 shutdown_connections 2022-06-17T12:54:54.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.355+0000 7f4f289fb700 1 -- 172.21.15.36:0/2512911608 wait complete. 2022-06-17T12:54:54.659 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:54:54.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.795+0000 7fa6d4ea0700 1 Processor -- start 2022-06-17T12:54:54.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.795+0000 7fa6d4ea0700 1 -- start start 2022-06-17T12:54:54.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.795+0000 7fa6d4ea0700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa6d00b7b60 0x7fa6d0062580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:54.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.795+0000 7fa6d4ea0700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa6d0064e60 0x7fa6d00652f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:54.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.795+0000 7fa6d4ea0700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6d00b35d0 con 0x7fa6d014afc0 2022-06-17T12:54:54.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.795+0000 7fa6d4ea0700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d0067250 con 0x7fa6d00b7b60 2022-06-17T12:54:54.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.795+0000 7fa6d4ea0700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d00674f0 con 0x7fa6d0064e60 2022-06-17T12:54:54.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.795+0000 7fa6ce59c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa6d00b7b60 0x7fa6d0062580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:54.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.795+0000 7fa6ce59c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa6d00b7b60 0x7fa6d0062580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37508/0 (socket says 172.21.15.36:37508) 2022-06-17T12:54:54.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.795+0000 7fa6ce59c700 1 -- 172.21.15.36:0/1722002343 learned_addr learned my addr 172.21.15.36:0/1722002343 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:54.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.795+0000 7fa6ced9d700 1 --2- 172.21.15.36:0/1722002343 >> v2:172.21.15.70:3300/0 conn(0x7fa6d0064e60 0x7fa6d00652f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:54.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.795+0000 7fa6ce59c700 1 -- 172.21.15.36:0/1722002343 >> v2:172.21.15.70:3300/0 conn(0x7fa6d0064e60 msgr2=0x7fa6d00652f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:54.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.795+0000 7fa6ce59c700 1 --2- 172.21.15.36:0/1722002343 >> v2:172.21.15.70:3300/0 conn(0x7fa6d0064e60 0x7fa6d00652f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:54.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.795+0000 7fa6ce59c700 1 -- 172.21.15.36:0/1722002343 >> v1:172.21.15.36:6789/0 conn(0x7fa6d014afc0 legacy=0x7fa6d014b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:54.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.795+0000 7fa6ce59c700 1 -- 172.21.15.36:0/1722002343 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d00b7f40 con 0x7fa6d00b7b60 2022-06-17T12:54:54.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.795+0000 7fa6ce59c700 1 --2- 172.21.15.36:0/1722002343 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa6d00b7b60 0x7fa6d0062580 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7fa6b8002f70 tx=0x7fa6b8006eb0).ready entity=mon.0 client_cookie=73c52fccde777741 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:54.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.795+0000 7fa6cd59a700 1 -- 172.21.15.36:0/1722002343 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6b800d610 con 0x7fa6d00b7b60 2022-06-17T12:54:54.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.795+0000 7fa6cd59a700 1 -- 172.21.15.36:0/1722002343 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b8000e90 con 0x7fa6d00b7b60 2022-06-17T12:54:54.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.795+0000 7fa6cd59a700 1 -- 172.21.15.36:0/1722002343 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6b800d9a0 con 0x7fa6d00b7b60 2022-06-17T12:54:54.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.795+0000 7fa6d4ea0700 1 -- 172.21.15.36:0/1722002343 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa6d00b7b60 msgr2=0x7fa6d0062580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:54.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.795+0000 7fa6d4ea0700 1 --2- 172.21.15.36:0/1722002343 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa6d00b7b60 0x7fa6d0062580 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7fa6b8002f70 tx=0x7fa6b8006eb0).stop 2022-06-17T12:54:54.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.795+0000 7fa6d4ea0700 1 -- 172.21.15.36:0/1722002343 shutdown_connections 2022-06-17T12:54:54.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.795+0000 7fa6d4ea0700 1 --2- 172.21.15.36:0/1722002343 >> v2:172.21.15.70:3300/0 conn(0x7fa6d0064e60 0x7fa6d00652f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:54.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.799+0000 7fa6d4ea0700 1 --2- 172.21.15.36:0/1722002343 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa6d00b7b60 0x7fa6d0062580 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:54.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.799+0000 7fa6d4ea0700 1 -- 172.21.15.36:0/1722002343 >> 172.21.15.36:0/1722002343 conn(0x7fa6d0060c60 msgr2=0x7fa6d005c550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:54.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.799+0000 7fa6d4ea0700 1 -- 172.21.15.36:0/1722002343 shutdown_connections 2022-06-17T12:54:54.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.799+0000 7fa6d4ea0700 1 -- 172.21.15.36:0/1722002343 wait complete. 2022-06-17T12:54:54.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.799+0000 7fa6d4ea0700 1 Processor -- start 2022-06-17T12:54:54.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.799+0000 7fa6d4ea0700 1 -- start start 2022-06-17T12:54:54.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.799+0000 7fa6d4ea0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6d0064e60 0x7fa6d00b4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:54.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.799+0000 7fa6d4ea0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa6d00b7b60 0x7fa6d00af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:54.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.799+0000 7fa6d4ea0700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6d014afc0 0x7fa6d00b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:54.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.799+0000 7fa6d4ea0700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d00b0600 con 0x7fa6d00b7b60 2022-06-17T12:54:54.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.799+0000 7fa6d4ea0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d00ad670 con 0x7fa6d0064e60 2022-06-17T12:54:54.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.799+0000 7fa6d4ea0700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d014ccf0 con 0x7fa6d014afc0 2022-06-17T12:54:54.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.799+0000 7fa6cdd9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa6d00b7b60 0x7fa6d00af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:54.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.799+0000 7fa6cdd9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa6d00b7b60 0x7fa6d00af4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37516/0 (socket says 172.21.15.36:37516) 2022-06-17T12:54:54.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.799+0000 7fa6cdd9b700 1 -- 172.21.15.36:0/3488893443 learned_addr learned my addr 172.21.15.36:0/3488893443 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:54.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.799+0000 7fa6ce59c700 1 --2- 172.21.15.36:0/3488893443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6d0064e60 0x7fa6d00b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:54.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.799+0000 7fa6ced9d700 1 --2- 172.21.15.36:0/3488893443 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6d014afc0 0x7fa6d00b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:54.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.803+0000 7fa6cdd9b700 1 -- 172.21.15.36:0/3488893443 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6d014afc0 msgr2=0x7fa6d00b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:54.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.803+0000 7fa6cdd9b700 1 --2- 172.21.15.36:0/3488893443 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6d014afc0 0x7fa6d00b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:54.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.803+0000 7fa6cdd9b700 1 -- 172.21.15.36:0/3488893443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6d0064e60 msgr2=0x7fa6d00b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:54.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.803+0000 7fa6cdd9b700 1 --2- 172.21.15.36:0/3488893443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6d0064e60 0x7fa6d00b4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:54.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.803+0000 7fa6cdd9b700 1 -- 172.21.15.36:0/3488893443 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b800b1e0 con 0x7fa6d00b7b60 2022-06-17T12:54:54.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.803+0000 7fa6cdd9b700 1 --2- 172.21.15.36:0/3488893443 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa6d00b7b60 0x7fa6d00af4e0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 rx=0x7fa6d0067870 tx=0x7fa6c0001400).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:54.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.803+0000 7fa6c77fe700 1 -- 172.21.15.36:0/3488893443 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6c000e4a0 con 0x7fa6d00b7b60 2022-06-17T12:54:54.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.803+0000 7fa6c77fe700 1 -- 172.21.15.36:0/3488893443 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c000e600 con 0x7fa6d00b7b60 2022-06-17T12:54:54.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.803+0000 7fa6d4ea0700 1 -- 172.21.15.36:0/3488893443 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d00ad3d0 con 0x7fa6d00b7b60 2022-06-17T12:54:54.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.803+0000 7fa6c77fe700 1 -- 172.21.15.36:0/3488893443 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6c000e960 con 0x7fa6d00b7b60 2022-06-17T12:54:54.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.803+0000 7fa6d4ea0700 1 -- 172.21.15.36:0/3488893443 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d014cf30 con 0x7fa6d00b7b60 2022-06-17T12:54:54.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.803+0000 7fa6d4ea0700 1 -- 172.21.15.36:0/3488893443 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d0057410 con 0x7fa6d00b7b60 2022-06-17T12:54:54.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.803+0000 7fa6c77fe700 1 -- 172.21.15.36:0/3488893443 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fa6c0019030 con 0x7fa6d00b7b60 2022-06-17T12:54:54.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.803+0000 7fa6c77fe700 1 --2- 172.21.15.36:0/3488893443 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa6a80539b0 0x7fa6a8055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:54.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.803+0000 7fa6ce59c700 1 --2- 172.21.15.36:0/3488893443 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa6a80539b0 0x7fa6a8055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:54.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.803+0000 7fa6c77fe700 1 -- 172.21.15.36:0/3488893443 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fa6c006b480 con 0x7fa6d00b7b60 2022-06-17T12:54:54.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.807+0000 7fa6ce59c700 1 --2- 172.21.15.36:0/3488893443 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa6a80539b0 0x7fa6a8055e70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7fa6b801c000 tx=0x7fa6b8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:54.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.811+0000 7fa6c77fe700 1 -- 172.21.15.36:0/3488893443 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa6c00424c0 con 0x7fa6d00b7b60 2022-06-17T12:54:54.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.971+0000 7fa6d4ea0700 1 -- 172.21.15.36:0/3488893443 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6d00b84d0 con 0x7fa6d00b7b60 2022-06-17T12:54:54.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.975+0000 7fa6c77fe700 1 -- 172.21.15.36:0/3488893443 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+1206 (secure 0 0 0) 0x7fa6c0045ae0 con 0x7fa6d00b7b60 2022-06-17T12:54:54.963 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:54.963 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":50,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/138643404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":138643404},{"type":"v1","addr":"172.21.15.70:6833","nonce":138643404}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":46},{"gid":7308,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.36:6835/1732238859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":1732238859},{"type":"v1","addr":"172.21.15.36:6835","nonce":1732238859}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":50}],"filesystems":[]} 2022-06-17T12:54:54.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.975+0000 7fa6d4ea0700 1 -- 172.21.15.36:0/3488893443 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa6a80539b0 msgr2=0x7fa6a8055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:54.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.975+0000 7fa6d4ea0700 1 --2- 172.21.15.36:0/3488893443 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa6a80539b0 0x7fa6a8055e70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7fa6b801c000 tx=0x7fa6b8015000).stop 2022-06-17T12:54:54.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.975+0000 7fa6d4ea0700 1 -- 172.21.15.36:0/3488893443 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa6d00b7b60 msgr2=0x7fa6d00af4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:54.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.975+0000 7fa6d4ea0700 1 --2- 172.21.15.36:0/3488893443 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa6d00b7b60 0x7fa6d00af4e0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 rx=0x7fa6d0067870 tx=0x7fa6c0001400).stop 2022-06-17T12:54:54.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.975+0000 7fa6d4ea0700 1 -- 172.21.15.36:0/3488893443 shutdown_connections 2022-06-17T12:54:54.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.975+0000 7fa6d4ea0700 1 --2- 172.21.15.36:0/3488893443 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa6a80539b0 0x7fa6a8055e70 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:54.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.975+0000 7fa6d4ea0700 1 --2- 172.21.15.36:0/3488893443 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6d014afc0 0x7fa6d00b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:54.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.975+0000 7fa6d4ea0700 1 --2- 172.21.15.36:0/3488893443 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa6d00b7b60 0x7fa6d00af4e0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:54.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.975+0000 7fa6d4ea0700 1 --2- 172.21.15.36:0/3488893443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6d0064e60 0x7fa6d00b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:54.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.975+0000 7fa6d4ea0700 1 -- 172.21.15.36:0/3488893443 >> 172.21.15.36:0/3488893443 conn(0x7fa6d0060c60 msgr2=0x7fa6d0147580 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:54.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.975+0000 7fa6d4ea0700 1 -- 172.21.15.36:0/3488893443 shutdown_connections 2022-06-17T12:54:54.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:54.975+0000 7fa6d4ea0700 1 -- 172.21.15.36:0/3488893443 wait complete. 2022-06-17T12:54:54.977 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 50 2022-06-17T12:54:54.979 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:54:55.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.099+0000 7fa80660b700 1 Processor -- start 2022-06-17T12:54:55.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.099+0000 7fa80660b700 1 -- start start 2022-06-17T12:54:55.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.099+0000 7fa80660b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa8000b2200 0x7fa8000af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:55.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.099+0000 7fa80660b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa8000b2cd0 0x7fa800147600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:55.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.099+0000 7fa80660b700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8000b25e0 con 0x7fa8000af8d0 2022-06-17T12:54:55.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.099+0000 7fa80660b700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8000ad790 con 0x7fa8000b2cd0 2022-06-17T12:54:55.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.099+0000 7fa80660b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8000aef40 con 0x7fa8000b2200 2022-06-17T12:54:55.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.099+0000 7fa804e08700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fa8000af8d0 0x7fa80014a8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55052/0 (socket says 172.21.15.36:55052) 2022-06-17T12:54:55.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.099+0000 7fa804e08700 1 -- 172.21.15.36:0/1070973299 learned_addr learned my addr 172.21.15.36:0/1070973299 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:55.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.099+0000 7fa805e0a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa8000b2cd0 0x7fa800147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:55.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.099+0000 7fa805609700 1 --2- 172.21.15.36:0/1070973299 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa8000b2200 0x7fa8000af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:55.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.099+0000 7fa7f7fff700 1 -- 172.21.15.36:0/1070973299 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2935640479 0 0) 0x7fa8000b25e0 con 0x7fa8000af8d0 2022-06-17T12:54:55.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.099+0000 7fa7f7fff700 1 -- 172.21.15.36:0/1070973299 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7e4003260 con 0x7fa8000af8d0 2022-06-17T12:54:55.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.099+0000 7fa805e0a700 1 -- 172.21.15.36:0/1070973299 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa8000b2200 msgr2=0x7fa8000af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:55.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.099+0000 7fa805e0a700 1 --2- 172.21.15.36:0/1070973299 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa8000b2200 0x7fa8000af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:55.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.099+0000 7fa805e0a700 1 -- 172.21.15.36:0/1070973299 >> v1:172.21.15.36:6789/0 conn(0x7fa8000af8d0 legacy=0x7fa80014a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:55.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.099+0000 7fa805e0a700 1 -- 172.21.15.36:0/1070973299 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8000b1950 con 0x7fa8000b2cd0 2022-06-17T12:54:55.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.099+0000 7fa805e0a700 1 --2- 172.21.15.36:0/1070973299 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa8000b2cd0 0x7fa800147600 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7fa7f80032f0 tx=0x7fa7f80091d0).ready entity=mon.0 client_cookie=d4e0ecdca23cb816 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:55.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.099+0000 7fa7f7fff700 1 -- 172.21.15.36:0/1070973299 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7f800b600 con 0x7fa8000b2cd0 2022-06-17T12:54:55.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.099+0000 7fa7f7fff700 1 -- 172.21.15.36:0/1070973299 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f800b760 con 0x7fa8000b2cd0 2022-06-17T12:54:55.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.099+0000 7fa7f7fff700 1 -- 172.21.15.36:0/1070973299 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7f800b8d0 con 0x7fa8000b2cd0 2022-06-17T12:54:55.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.099+0000 7fa80660b700 1 -- 172.21.15.36:0/1070973299 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa8000b2cd0 msgr2=0x7fa800147600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:55.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.099+0000 7fa80660b700 1 --2- 172.21.15.36:0/1070973299 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa8000b2cd0 0x7fa800147600 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7fa7f80032f0 tx=0x7fa7f80091d0).stop 2022-06-17T12:54:55.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa80660b700 1 -- 172.21.15.36:0/1070973299 shutdown_connections 2022-06-17T12:54:55.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa80660b700 1 --2- 172.21.15.36:0/1070973299 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa8000b2cd0 0x7fa800147600 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:55.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa80660b700 1 --2- 172.21.15.36:0/1070973299 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa8000b2200 0x7fa8000af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:55.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa80660b700 1 -- 172.21.15.36:0/1070973299 >> 172.21.15.36:0/1070973299 conn(0x7fa80005f770 msgr2=0x7fa80005fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:55.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa80660b700 1 -- 172.21.15.36:0/1070973299 shutdown_connections 2022-06-17T12:54:55.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa80660b700 1 -- 172.21.15.36:0/1070973299 wait complete. 2022-06-17T12:54:55.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa80660b700 1 Processor -- start 2022-06-17T12:54:55.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa80660b700 1 -- start start 2022-06-17T12:54:55.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa80660b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8000af8d0 0x7fa8000a8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:55.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa80660b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa8000b2200 0x7fa800156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:55.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa80660b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa8000b2cd0 0x7fa8000aac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:55.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa80660b700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8000a9400 con 0x7fa8000b2cd0 2022-06-17T12:54:55.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa80660b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa800062390 con 0x7fa8000af8d0 2022-06-17T12:54:55.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa80660b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8000625c0 con 0x7fa8000b2200 2022-06-17T12:54:55.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa805e0a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa8000b2cd0 0x7fa8000aac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:55.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa805e0a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa8000b2cd0 0x7fa8000aac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37530/0 (socket says 172.21.15.36:37530) 2022-06-17T12:54:55.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa805e0a700 1 -- 172.21.15.36:0/2307896696 learned_addr learned my addr 172.21.15.36:0/2307896696 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:55.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa805609700 1 --2- 172.21.15.36:0/2307896696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8000af8d0 0x7fa8000a8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:55.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa804e08700 1 --2- 172.21.15.36:0/2307896696 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa8000b2200 0x7fa800156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:55.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa805e0a700 1 -- 172.21.15.36:0/2307896696 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa8000b2200 msgr2=0x7fa800156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:55.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa805e0a700 1 --2- 172.21.15.36:0/2307896696 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa8000b2200 0x7fa800156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:55.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa805e0a700 1 -- 172.21.15.36:0/2307896696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8000af8d0 msgr2=0x7fa8000a8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:55.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa805e0a700 1 --2- 172.21.15.36:0/2307896696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8000af8d0 0x7fa8000a8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:55.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa805e0a700 1 -- 172.21.15.36:0/2307896696 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f8009470 con 0x7fa8000b2cd0 2022-06-17T12:54:55.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa805609700 1 --2- 172.21.15.36:0/2307896696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8000af8d0 0x7fa8000a8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:55.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa805e0a700 1 --2- 172.21.15.36:0/2307896696 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa8000b2cd0 0x7fa8000aac20 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 rx=0x7fa7f80032f0 tx=0x7fa7f8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:55.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa7f67fc700 1 -- 172.21.15.36:0/2307896696 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7f800b8d0 con 0x7fa8000b2cd0 2022-06-17T12:54:55.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa7f67fc700 1 -- 172.21.15.36:0/2307896696 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f801f3f0 con 0x7fa8000b2cd0 2022-06-17T12:54:55.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa7f67fc700 1 -- 172.21.15.36:0/2307896696 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7f800b600 con 0x7fa8000b2cd0 2022-06-17T12:54:55.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa80660b700 1 -- 172.21.15.36:0/2307896696 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa800156f30 con 0x7fa8000b2cd0 2022-06-17T12:54:55.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.103+0000 7fa80660b700 1 -- 172.21.15.36:0/2307896696 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8000a7ad0 con 0x7fa8000b2cd0 2022-06-17T12:54:55.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.107+0000 7fa80660b700 1 -- 172.21.15.36:0/2307896696 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8000ad330 con 0x7fa8000b2cd0 2022-06-17T12:54:55.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.107+0000 7fa7f67fc700 1 -- 172.21.15.36:0/2307896696 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fa7f8026030 con 0x7fa8000b2cd0 2022-06-17T12:54:55.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.107+0000 7fa7f67fc700 1 --2- 172.21.15.36:0/2307896696 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa7e8053af0 0x7fa7e8055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:55.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.107+0000 7fa7f67fc700 1 -- 172.21.15.36:0/2307896696 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fa7f806c9b0 con 0x7fa8000b2cd0 2022-06-17T12:54:55.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.107+0000 7fa805609700 1 --2- 172.21.15.36:0/2307896696 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa7e8053af0 0x7fa7e8055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:55.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.107+0000 7fa805609700 1 --2- 172.21.15.36:0/2307896696 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa7e8053af0 0x7fa7e8055fb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7fa7fc00b000 tx=0x7fa7fc008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:55.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.115+0000 7fa7f67fc700 1 -- 172.21.15.36:0/2307896696 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa7f8047350 con 0x7fa8000b2cd0 2022-06-17T12:54:55.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.279+0000 7fa80660b700 1 -- 172.21.15.36:0/2307896696 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa800057410 con 0x7fa8000b2cd0 2022-06-17T12:54:55.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.279+0000 7fa7f67fc700 1 -- 172.21.15.36:0/2307896696 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+1206 (secure 0 0 0) 0x7fa7f8024090 con 0x7fa8000b2cd0 2022-06-17T12:54:55.268 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:55.269 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":50,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/138643404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":138643404},{"type":"v1","addr":"172.21.15.70:6833","nonce":138643404}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":46},{"gid":7308,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.36:6835/1732238859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":1732238859},{"type":"v1","addr":"172.21.15.36:6835","nonce":1732238859}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":50}],"filesystems":[]} 2022-06-17T12:54:55.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.283+0000 7fa80660b700 1 -- 172.21.15.36:0/2307896696 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa7e8053af0 msgr2=0x7fa7e8055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:55.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.283+0000 7fa80660b700 1 --2- 172.21.15.36:0/2307896696 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa7e8053af0 0x7fa7e8055fb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7fa7fc00b000 tx=0x7fa7fc008040).stop 2022-06-17T12:54:55.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.283+0000 7fa80660b700 1 -- 172.21.15.36:0/2307896696 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa8000b2cd0 msgr2=0x7fa8000aac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:55.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.283+0000 7fa80660b700 1 --2- 172.21.15.36:0/2307896696 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa8000b2cd0 0x7fa8000aac20 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 rx=0x7fa7f80032f0 tx=0x7fa7f8016000).stop 2022-06-17T12:54:55.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.283+0000 7fa80660b700 1 -- 172.21.15.36:0/2307896696 shutdown_connections 2022-06-17T12:54:55.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.283+0000 7fa80660b700 1 --2- 172.21.15.36:0/2307896696 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa7e8053af0 0x7fa7e8055fb0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:55.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.283+0000 7fa80660b700 1 --2- 172.21.15.36:0/2307896696 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa8000b2cd0 0x7fa8000aac20 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:55.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.283+0000 7fa80660b700 1 --2- 172.21.15.36:0/2307896696 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa8000b2200 0x7fa800156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:55.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.283+0000 7fa80660b700 1 --2- 172.21.15.36:0/2307896696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8000af8d0 0x7fa8000a8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:55.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.283+0000 7fa80660b700 1 -- 172.21.15.36:0/2307896696 >> 172.21.15.36:0/2307896696 conn(0x7fa80005f770 msgr2=0x7fa8000ab480 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:55.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.283+0000 7fa80660b700 1 -- 172.21.15.36:0/2307896696 shutdown_connections 2022-06-17T12:54:55.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.283+0000 7fa80660b700 1 -- 172.21.15.36:0/2307896696 wait complete. 2022-06-17T12:54:55.274 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 50 2022-06-17T12:54:55.285 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-17T12:54:55.285 INFO:tasks.ceph.mds.a:Stopping old one... 2022-06-17T12:54:55.285 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-17T12:54:55.286 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T12:54:55.287 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-17T12:54:55.287 INFO:tasks.ceph.mds.b:Stopping old one... 2022-06-17T12:54:55.287 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-17T12:54:55.288 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T12:54:55.303 INFO:tasks.ceph.mds.a:Stopped 2022-06-17T12:54:55.304 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-17T12:54:55.311 INFO:tasks.ceph.mds.b:Stopped 2022-06-17T12:54:55.311 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-17T12:54:55.315 INFO:tasks.ceph.mds.b:Started 2022-06-17T12:54:55.316 DEBUG:teuthology.parallel:result is None 2022-06-17T12:54:55.348 INFO:tasks.ceph.mds.a:Started 2022-06-17T12:54:55.349 DEBUG:teuthology.parallel:result is None 2022-06-17T12:54:55.350 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-06-17T12:54:55.391 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2022-06-17T12:54:55.423 INFO:tasks.ceph.mds.a.smithi036.stdout:starting mds.a at 2022-06-17T12:54:55.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe067fff700 1 Processor -- start 2022-06-17T12:54:55.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe067fff700 1 -- start start 2022-06-17T12:54:55.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe067fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe0680b20f0 0x7fe0680aee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:55.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe067fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe0680afe50 0x7fe06813c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:55.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe067fff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe068060ea0 con 0x7fe068060ac0 2022-06-17T12:54:55.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe067fff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0680b3060 con 0x7fe0680b20f0 2022-06-17T12:54:55.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe067fff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe068067a20 con 0x7fe0680afe50 2022-06-17T12:54:55.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe066ffd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fe068060ac0 0x7fe0680ae720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55078/0 (socket says 172.21.15.36:55078) 2022-06-17T12:54:55.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe066ffd700 1 -- 172.21.15.36:0/2754842491 learned_addr learned my addr 172.21.15.36:0/2754842491 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:55.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe0667fc700 1 --2- 172.21.15.36:0/2754842491 >> v2:172.21.15.36:3300/0 conn(0x7fe0680b20f0 0x7fe0680aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:55.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe0677fe700 1 --2- 172.21.15.36:0/2754842491 >> v2:172.21.15.70:3300/0 conn(0x7fe0680afe50 0x7fe06813c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:55.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe065ffb700 1 -- 172.21.15.36:0/2754842491 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1039991515 0 0) 0x7fe068060ea0 con 0x7fe068060ac0 2022-06-17T12:54:55.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe065ffb700 1 -- 172.21.15.36:0/2754842491 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe04c003260 con 0x7fe068060ac0 2022-06-17T12:54:55.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe0667fc700 1 -- 172.21.15.36:0/2754842491 >> v2:172.21.15.70:3300/0 conn(0x7fe0680afe50 msgr2=0x7fe06813c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:55.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe0667fc700 1 --2- 172.21.15.36:0/2754842491 >> v2:172.21.15.70:3300/0 conn(0x7fe0680afe50 0x7fe06813c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:55.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe0667fc700 1 -- 172.21.15.36:0/2754842491 >> v1:172.21.15.36:6789/0 conn(0x7fe068060ac0 legacy=0x7fe0680ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:55.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe0667fc700 1 -- 172.21.15.36:0/2754842491 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe068143fb0 con 0x7fe0680b20f0 2022-06-17T12:54:55.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe0667fc700 1 --2- 172.21.15.36:0/2754842491 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0680b20f0 0x7fe0680aee30 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7fe058002f70 tx=0x7fe058006df0).ready entity=mon.0 client_cookie=e482f1ce0f9a0c14 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:55.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe065ffb700 1 -- 172.21.15.36:0/2754842491 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe05800d4f0 con 0x7fe0680b20f0 2022-06-17T12:54:55.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe065ffb700 1 -- 172.21.15.36:0/2754842491 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe058000e90 con 0x7fe0680b20f0 2022-06-17T12:54:55.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe065ffb700 1 -- 172.21.15.36:0/2754842491 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe05800d840 con 0x7fe0680b20f0 2022-06-17T12:54:55.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe067fff700 1 -- 172.21.15.36:0/2754842491 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0680b20f0 msgr2=0x7fe0680aee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:55.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe067fff700 1 --2- 172.21.15.36:0/2754842491 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0680b20f0 0x7fe0680aee30 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7fe058002f70 tx=0x7fe058006df0).stop 2022-06-17T12:54:55.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe067fff700 1 -- 172.21.15.36:0/2754842491 shutdown_connections 2022-06-17T12:54:55.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe067fff700 1 --2- 172.21.15.36:0/2754842491 >> v2:172.21.15.70:3300/0 conn(0x7fe0680afe50 0x7fe06813c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:55.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe067fff700 1 --2- 172.21.15.36:0/2754842491 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0680b20f0 0x7fe0680aee30 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:55.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe067fff700 1 -- 172.21.15.36:0/2754842491 >> 172.21.15.36:0/2754842491 conn(0x7fe0680b65f0 msgr2=0x7fe0680b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:55.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.475+0000 7fe067fff700 1 -- 172.21.15.36:0/2754842491 shutdown_connections 2022-06-17T12:54:55.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe067fff700 1 -- 172.21.15.36:0/2754842491 wait complete. 2022-06-17T12:54:55.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe067fff700 1 Processor -- start 2022-06-17T12:54:55.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe067fff700 1 -- start start 2022-06-17T12:54:55.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe067fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe068060ac0 0x7fe068148d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:55.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe067fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0680afe50 0x7fe068142740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:55.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe067fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0680b20f0 0x7fe068151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:55.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe067fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe068141dd0 con 0x7fe0680b20f0 2022-06-17T12:54:55.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe067fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe068142000 con 0x7fe0680afe50 2022-06-17T12:54:55.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe067fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0681422a0 con 0x7fe068060ac0 2022-06-17T12:54:55.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe0677fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0680b20f0 0x7fe068151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:55.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe0677fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0680b20f0 0x7fe068151ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37558/0 (socket says 172.21.15.36:37558) 2022-06-17T12:54:55.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe0677fe700 1 -- 172.21.15.36:0/3596053918 learned_addr learned my addr 172.21.15.36:0/3596053918 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:55.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe0667fc700 1 --2- 172.21.15.36:0/3596053918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0680afe50 0x7fe068142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:55.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe066ffd700 1 --2- 172.21.15.36:0/3596053918 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe068060ac0 0x7fe068148d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:55.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe0677fe700 1 -- 172.21.15.36:0/3596053918 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe068060ac0 msgr2=0x7fe068148d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:55.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe0677fe700 1 --2- 172.21.15.36:0/3596053918 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe068060ac0 0x7fe068148d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:55.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe0677fe700 1 -- 172.21.15.36:0/3596053918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0680afe50 msgr2=0x7fe068142740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:55.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe0677fe700 1 --2- 172.21.15.36:0/3596053918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0680afe50 0x7fe068142740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:55.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe0677fe700 1 -- 172.21.15.36:0/3596053918 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05800b180 con 0x7fe0680b20f0 2022-06-17T12:54:55.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe0677fe700 1 --2- 172.21.15.36:0/3596053918 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0680b20f0 0x7fe068151ec0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7fe060006e00 tx=0x7fe0600091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:55.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe057fff700 1 -- 172.21.15.36:0/3596053918 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe06000c850 con 0x7fe0680b20f0 2022-06-17T12:54:55.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe057fff700 1 -- 172.21.15.36:0/3596053918 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06000c9b0 con 0x7fe0680b20f0 2022-06-17T12:54:55.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe067fff700 1 -- 172.21.15.36:0/3596053918 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe06813ecf0 con 0x7fe0680b20f0 2022-06-17T12:54:55.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe057fff700 1 -- 172.21.15.36:0/3596053918 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe06000cd10 con 0x7fe0680b20f0 2022-06-17T12:54:55.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.479+0000 7fe067fff700 1 -- 172.21.15.36:0/3596053918 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe06813fe60 con 0x7fe0680b20f0 2022-06-17T12:54:55.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.483+0000 7fe067fff700 1 -- 172.21.15.36:0/3596053918 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe06813fa60 con 0x7fe0680b20f0 2022-06-17T12:54:55.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.483+0000 7fe057fff700 1 -- 172.21.15.36:0/3596053918 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe0600123f0 con 0x7fe0680b20f0 2022-06-17T12:54:55.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.483+0000 7fe057fff700 1 --2- 172.21.15.36:0/3596053918 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe044053a40 0x7fe044055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:55.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.483+0000 7fe057fff700 1 -- 172.21.15.36:0/3596053918 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fe06006c1c0 con 0x7fe0680b20f0 2022-06-17T12:54:55.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.483+0000 7fe066ffd700 1 --2- 172.21.15.36:0/3596053918 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe044053a40 0x7fe044055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:55.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.483+0000 7fe066ffd700 1 --2- 172.21.15.36:0/3596053918 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe044053a40 0x7fe044055f00 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7fe05c00b000 tx=0x7fe05c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:55.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.487+0000 7fe057fff700 1 -- 172.21.15.36:0/3596053918 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe06813fa60 con 0x7fe0680b20f0 2022-06-17T12:54:55.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:55.627+0000 7fe067fff700 1 -- 172.21.15.36:0/3596053918 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fe06813fa60 con 0x7fe0680b20f0 2022-06-17T12:54:56.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.635+0000 7fe057fff700 1 -- 172.21.15.36:0/3596053918 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v100) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe06813fa60 con 0x7fe0680b20f0 2022-06-17T12:54:56.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.639+0000 7fe067fff700 1 -- 172.21.15.36:0/3596053918 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe044053a40 msgr2=0x7fe044055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:56.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.639+0000 7fe067fff700 1 --2- 172.21.15.36:0/3596053918 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe044053a40 0x7fe044055f00 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7fe05c00b000 tx=0x7fe05c009040).stop 2022-06-17T12:54:56.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.639+0000 7fe067fff700 1 -- 172.21.15.36:0/3596053918 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0680b20f0 msgr2=0x7fe068151ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:56.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.639+0000 7fe067fff700 1 --2- 172.21.15.36:0/3596053918 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0680b20f0 0x7fe068151ec0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7fe060006e00 tx=0x7fe0600091d0).stop 2022-06-17T12:54:56.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.639+0000 7fe067fff700 1 -- 172.21.15.36:0/3596053918 shutdown_connections 2022-06-17T12:54:56.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.639+0000 7fe067fff700 1 --2- 172.21.15.36:0/3596053918 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe044053a40 0x7fe044055f00 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:56.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.639+0000 7fe067fff700 1 --2- 172.21.15.36:0/3596053918 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe0680b20f0 0x7fe068151ec0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:56.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.639+0000 7fe067fff700 1 --2- 172.21.15.36:0/3596053918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0680afe50 0x7fe068142740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:56.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.639+0000 7fe067fff700 1 --2- 172.21.15.36:0/3596053918 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe068060ac0 0x7fe068148d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:56.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.639+0000 7fe067fff700 1 -- 172.21.15.36:0/3596053918 >> 172.21.15.36:0/3596053918 conn(0x7fe0680b65f0 msgr2=0x7fe0680b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:56.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.639+0000 7fe067fff700 1 -- 172.21.15.36:0/3596053918 shutdown_connections 2022-06-17T12:54:56.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.639+0000 7fe067fff700 1 -- 172.21.15.36:0/3596053918 wait complete. 2022-06-17T12:54:56.643 INFO:teuthology.orchestra.run.smithi036.stderr: removed all blacklist entries 2022-06-17T12:54:56.648 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2022-06-17T12:54:56.649 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-06-17T12:54:56.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.771+0000 7f034d2da700 1 Processor -- start 2022-06-17T12:54:56.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.775+0000 7f034d2da700 1 -- start start 2022-06-17T12:54:56.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.775+0000 7f034d2da700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f03480b18e0 0x7f0348143860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:56.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.775+0000 7f034d2da700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03480b2b70 con 0x7f03480b39a0 2022-06-17T12:54:56.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.775+0000 7f034d2da700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03480b0d30 con 0x7f03480b6860 2022-06-17T12:54:56.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.775+0000 7f034d2da700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03480b6c40 con 0x7f03480b18e0 2022-06-17T12:54:56.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.775+0000 7f034cad9700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f03480b39a0 0x7f034813fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55098/0 (socket says 172.21.15.36:55098) 2022-06-17T12:54:56.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.775+0000 7f034cad9700 1 -- 172.21.15.36:0/973890270 learned_addr learned my addr 172.21.15.36:0/973890270 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:56.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.775+0000 7f03477fe700 1 --2- 172.21.15.36:0/973890270 >> v2:172.21.15.70:3300/0 conn(0x7f03480b18e0 0x7f0348143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:56.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.775+0000 7f0346ffd700 1 -- 172.21.15.36:0/973890270 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1436373168 0 0) 0x7f03480b2b70 con 0x7f03480b39a0 2022-06-17T12:54:56.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.775+0000 7f0346ffd700 1 -- 172.21.15.36:0/973890270 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0334003260 con 0x7f03480b39a0 2022-06-17T12:54:56.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.775+0000 7f0346ffd700 1 -- 172.21.15.36:0/973890270 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2604735974 0 0) 0x7f03480b0d30 con 0x7f03480b6860 2022-06-17T12:54:56.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.775+0000 7f0346ffd700 1 -- 172.21.15.36:0/973890270 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03480b2b70 con 0x7f03480b6860 2022-06-17T12:54:56.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.775+0000 7f03477fe700 1 -- 172.21.15.36:0/973890270 >> v1:172.21.15.70:6789/0 conn(0x7f03480b6860 legacy=0x7f0348143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:56.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.775+0000 7f03477fe700 1 -- 172.21.15.36:0/973890270 >> v1:172.21.15.36:6789/0 conn(0x7f03480b39a0 legacy=0x7f034813fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:56.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.775+0000 7f03477fe700 1 -- 172.21.15.36:0/973890270 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03480b0fd0 con 0x7f03480b18e0 2022-06-17T12:54:56.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.775+0000 7f03477fe700 1 --2- 172.21.15.36:0/973890270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03480b18e0 0x7f0348143860 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f033c002f70 tx=0x7f033c006d90).ready entity=mon.1 client_cookie=55ff2fb6f12fceb5 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:56.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.779+0000 7f0346ffd700 1 -- 172.21.15.36:0/973890270 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f033c00e630 con 0x7f03480b18e0 2022-06-17T12:54:56.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.779+0000 7f0346ffd700 1 -- 172.21.15.36:0/973890270 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033c000e90 con 0x7f03480b18e0 2022-06-17T12:54:56.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.779+0000 7f0346ffd700 1 -- 172.21.15.36:0/973890270 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f033c00e980 con 0x7f03480b18e0 2022-06-17T12:54:56.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.779+0000 7f034d2da700 1 -- 172.21.15.36:0/973890270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03480b18e0 msgr2=0x7f0348143860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:56.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.779+0000 7f034d2da700 1 --2- 172.21.15.36:0/973890270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03480b18e0 0x7f0348143860 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f033c002f70 tx=0x7f033c006d90).stop 2022-06-17T12:54:56.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.779+0000 7f034d2da700 1 -- 172.21.15.36:0/973890270 shutdown_connections 2022-06-17T12:54:56.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.779+0000 7f034d2da700 1 --2- 172.21.15.36:0/973890270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03480b18e0 0x7f0348143860 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:56.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.779+0000 7f034d2da700 1 -- 172.21.15.36:0/973890270 >> 172.21.15.36:0/973890270 conn(0x7f03480614b0 msgr2=0x7f0348061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:56.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.779+0000 7f034d2da700 1 -- 172.21.15.36:0/973890270 shutdown_connections 2022-06-17T12:54:56.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.779+0000 7f034d2da700 1 -- 172.21.15.36:0/973890270 wait complete. 2022-06-17T12:54:56.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.783+0000 7f034d2da700 1 Processor -- start 2022-06-17T12:54:56.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.783+0000 7f034d2da700 1 -- start start 2022-06-17T12:54:56.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.783+0000 7f034d2da700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03480b18e0 0x7f0348065e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:56.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.783+0000 7f034d2da700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f03480b39a0 0x7f0348141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:56.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.783+0000 7f034d2da700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f03480b6860 0x7f0348064a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:56.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.783+0000 7f034d2da700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0348065110 con 0x7f03480b39a0 2022-06-17T12:54:56.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.783+0000 7f034d2da700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03480653b0 con 0x7f03480b18e0 2022-06-17T12:54:56.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.783+0000 7f034d2da700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0348155c10 con 0x7f03480b6860 2022-06-17T12:54:56.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.783+0000 7f03477fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f03480b39a0 0x7f0348141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:56.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.783+0000 7f03477fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f03480b39a0 0x7f0348141ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37572/0 (socket says 172.21.15.36:37572) 2022-06-17T12:54:56.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.783+0000 7f03477fe700 1 -- 172.21.15.36:0/3580893326 learned_addr learned my addr 172.21.15.36:0/3580893326 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:56.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.783+0000 7f0347fff700 1 --2- 172.21.15.36:0/3580893326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03480b18e0 0x7f0348065e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:56.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.783+0000 7f034cad9700 1 --2- 172.21.15.36:0/3580893326 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f03480b6860 0x7f0348064a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:56.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.783+0000 7f03477fe700 1 -- 172.21.15.36:0/3580893326 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f03480b6860 msgr2=0x7f0348064a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:56.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.783+0000 7f03477fe700 1 --2- 172.21.15.36:0/3580893326 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f03480b6860 0x7f0348064a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:56.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.783+0000 7f03477fe700 1 -- 172.21.15.36:0/3580893326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03480b18e0 msgr2=0x7f0348065e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:56.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.783+0000 7f03477fe700 1 --2- 172.21.15.36:0/3580893326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03480b18e0 0x7f0348065e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:56.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.783+0000 7f03477fe700 1 -- 172.21.15.36:0/3580893326 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033c00b120 con 0x7f03480b39a0 2022-06-17T12:54:56.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.783+0000 7f03477fe700 1 --2- 172.21.15.36:0/3580893326 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f03480b39a0 0x7f0348141ae0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7f033c002f70 tx=0x7f033c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:56.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.783+0000 7f03457fa700 1 -- 172.21.15.36:0/3580893326 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f033c0116c0 con 0x7f03480b39a0 2022-06-17T12:54:56.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.783+0000 7f03457fa700 1 -- 172.21.15.36:0/3580893326 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033c011820 con 0x7f03480b39a0 2022-06-17T12:54:56.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.783+0000 7f034d2da700 1 -- 172.21.15.36:0/3580893326 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0348155e90 con 0x7f03480b39a0 2022-06-17T12:54:56.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.783+0000 7f03457fa700 1 -- 172.21.15.36:0/3580893326 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f033c011b80 con 0x7f03480b39a0 2022-06-17T12:54:56.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.783+0000 7f034d2da700 1 -- 172.21.15.36:0/3580893326 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0348114e50 con 0x7f03480b39a0 2022-06-17T12:54:56.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.787+0000 7f034d2da700 1 -- 172.21.15.36:0/3580893326 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0348064f60 con 0x7f03480b39a0 2022-06-17T12:54:56.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.787+0000 7f03457fa700 1 -- 172.21.15.36:0/3580893326 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f033c00f070 con 0x7f03480b39a0 2022-06-17T12:54:56.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.787+0000 7f03457fa700 1 --2- 172.21.15.36:0/3580893326 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0328053920 0x7f0328055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:56.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.787+0000 7f03457fa700 1 -- 172.21.15.36:0/3580893326 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f033c06cf20 con 0x7f03480b39a0 2022-06-17T12:54:56.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.787+0000 7f0347fff700 1 --2- 172.21.15.36:0/3580893326 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0328053920 0x7f0328055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:56.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.787+0000 7f0347fff700 1 --2- 172.21.15.36:0/3580893326 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0328053920 0x7f0328055de0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f033800d000 tx=0x7f0338007a20).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:56.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.791+0000 7f03457fa700 1 -- 172.21.15.36:0/3580893326 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f033c044060 con 0x7f03480b39a0 2022-06-17T12:54:56.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.963+0000 7f034d2da700 1 -- 172.21.15.36:0/3580893326 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0348057410 con 0x7f03480b39a0 2022-06-17T12:54:56.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.963+0000 7f03457fa700 1 -- 172.21.15.36:0/3580893326 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v34) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f033c047680 con 0x7f03480b39a0 2022-06-17T12:54:56.953 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:56.953 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:54:56.954 INFO:teuthology.orchestra.run.smithi036.stdout: "auth_dump": [ 2022-06-17T12:54:56.954 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:56.954 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mds.a", 2022-06-17T12:54:56.954 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiyaUgARAAxp7bzOl4uVo2EsaoRBeMRw==", 2022-06-17T12:54:56.955 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:56.955 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:54:56.955 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:56.955 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:56.955 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:56.956 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:56.956 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:56.956 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:56.956 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mds.b", 2022-06-17T12:54:56.957 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xifyxLCRAABI389jmz1zboeDanzraT8Q==", 2022-06-17T12:54:56.957 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:56.957 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:54:56.957 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:56.958 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:56.958 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:56.958 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:56.958 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:56.958 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:56.959 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.0", 2022-06-17T12:54:56.959 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB+d6xibhXGNBAA6AuUT9ITWAZjDFpNsgA4Wg==", 2022-06-17T12:54:56.959 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:56.959 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:56.960 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:56.960 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:56.960 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:56.960 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:56.960 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:56.961 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.1", 2022-06-17T12:54:56.961 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCBd6xiOIPvGxAADkrBoJCy9x91zan6TWU0tQ==", 2022-06-17T12:54:56.961 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:56.961 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:56.961 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:56.962 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:56.962 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:56.962 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:56.962 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:56.963 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.2", 2022-06-17T12:54:56.963 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCEd6xiP4/9AhAAaP3cWXV6vbfGBPfqSLKqUg==", 2022-06-17T12:54:56.963 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:56.963 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:56.963 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:56.964 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:56.964 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:56.964 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:56.964 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:56.964 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.3", 2022-06-17T12:54:56.965 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCGd6xi+LZgJxAAJjGwP5qW6TFDZfR+2rz6ig==", 2022-06-17T12:54:56.965 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:56.965 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:56.965 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:56.966 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:56.966 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:56.966 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:56.966 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:56.967 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.4", 2022-06-17T12:54:56.967 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCKd6xikvrTChAAXIyWSETswDv+tb3zsGyYpA==", 2022-06-17T12:54:56.967 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:56.967 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:56.967 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:56.968 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:56.968 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:56.968 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:56.968 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:56.968 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.5", 2022-06-17T12:54:56.969 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCMd6xiqr2QLhAArhimHuNsVWrDGGreFYxj6A==", 2022-06-17T12:54:56.969 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:56.969 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:56.969 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:56.970 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:56.970 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:56.970 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:56.970 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:56.970 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.6", 2022-06-17T12:54:56.971 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCPd6xiUXcXGxAAdrYz8txdBWMIqF5DF4Aimw==", 2022-06-17T12:54:56.971 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:56.971 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:56.971 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:56.971 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:56.972 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:56.972 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:56.972 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:56.972 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.7", 2022-06-17T12:54:56.973 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCSd6xiBJ2IAhAAUkNfgmlxxWY7+VouPLAvyw==", 2022-06-17T12:54:56.973 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:56.973 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:56.973 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:56.973 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:56.974 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:56.974 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:56.974 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:56.974 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.0", 2022-06-17T12:54:56.974 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiXO3QFBAAsSLtQ0yKKZxhqUT/eYKdtg==", 2022-06-17T12:54:56.975 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:56.975 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:54:56.975 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:54:56.975 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:54:56.976 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:56.976 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:56.976 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:56.976 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.1", 2022-06-17T12:54:56.976 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xi51KdGBAAi7fGZ+Qepg6j3DAhlzoWxw==", 2022-06-17T12:54:56.977 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:56.977 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:54:56.977 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:54:56.977 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:54:56.977 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:56.978 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:56.978 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:56.978 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.2", 2022-06-17T12:54:56.978 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiug2hHBAAqHpLgwMdthSWtJ8Vwu+m1Q==", 2022-06-17T12:54:56.979 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:56.979 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:54:56.979 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:54:56.979 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:54:56.979 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:56.980 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:56.980 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:56.980 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.admin", 2022-06-17T12:54:56.980 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB5d6xiGDODKhAAPZMG/M2nVssdr4FJHcS6XQ==", 2022-06-17T12:54:56.980 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:56.981 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:54:56.981 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:54:56.981 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:54:56.981 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:56.982 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:56.982 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:56.982 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:56.982 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mgr.x", 2022-06-17T12:54:56.982 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB6d6xiyH9uORAATJb0chHE1CH0X02w7edVtw==", 2022-06-17T12:54:56.983 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:56.983 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:54:56.983 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile mgr", 2022-06-17T12:54:56.983 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:56.983 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:56.984 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:56.984 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:56.984 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mgr.y", 2022-06-17T12:54:56.984 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB6d6xiHXghMxAAzpsVRCbLWHWZPtrPk1jFlA==", 2022-06-17T12:54:56.985 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:54:56.985 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:54:56.985 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile mgr", 2022-06-17T12:54:56.986 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:54:56.986 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:56.987 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:56.987 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:54:56.987 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:54:56.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.967+0000 7f034d2da700 1 -- 172.21.15.36:0/3580893326 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0328053920 msgr2=0x7f0328055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:56.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.967+0000 7f034d2da700 1 --2- 172.21.15.36:0/3580893326 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0328053920 0x7f0328055de0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f033800d000 tx=0x7f0338007a20).stop 2022-06-17T12:54:56.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.967+0000 7f034d2da700 1 -- 172.21.15.36:0/3580893326 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f03480b39a0 msgr2=0x7f0348141ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:56.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.967+0000 7f034d2da700 1 --2- 172.21.15.36:0/3580893326 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f03480b39a0 0x7f0348141ae0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7f033c002f70 tx=0x7f033c014000).stop 2022-06-17T12:54:56.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.967+0000 7f034d2da700 1 -- 172.21.15.36:0/3580893326 shutdown_connections 2022-06-17T12:54:56.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.967+0000 7f034d2da700 1 --2- 172.21.15.36:0/3580893326 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0328053920 0x7f0328055de0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:56.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.967+0000 7f034d2da700 1 --2- 172.21.15.36:0/3580893326 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f03480b6860 0x7f0348064a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:56.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.967+0000 7f034d2da700 1 --2- 172.21.15.36:0/3580893326 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f03480b39a0 0x7f0348141ae0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:56.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.967+0000 7f034d2da700 1 --2- 172.21.15.36:0/3580893326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03480b18e0 0x7f0348065e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:56.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.967+0000 7f034d2da700 1 -- 172.21.15.36:0/3580893326 >> 172.21.15.36:0/3580893326 conn(0x7f03480614b0 msgr2=0x7f034813f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:56.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.967+0000 7f034d2da700 1 -- 172.21.15.36:0/3580893326 shutdown_connections 2022-06-17T12:54:56.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:56.967+0000 7f034d2da700 1 -- 172.21.15.36:0/3580893326 wait complete. 2022-06-17T12:54:56.992 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:54:57.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f705e9ee700 1 Processor -- start 2022-06-17T12:54:57.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f705e9ee700 1 -- start start 2022-06-17T12:54:57.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f705e9ee700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7058062770 0x7f7058062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f705e9ee700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70581444f0 0x7f7058144930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f705e9ee700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70580b0ba0 con 0x7f70580ad330 2022-06-17T12:54:57.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f705e9ee700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70580ae820 con 0x7f7058062770 2022-06-17T12:54:57.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f705e9ee700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7058061030 con 0x7f70581444f0 2022-06-17T12:54:57.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f7057fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7058062770 0x7f7058062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f7057fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7058062770 0x7f7058062b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37578/0 (socket says 172.21.15.36:37578) 2022-06-17T12:54:57.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f7057fff700 1 -- 172.21.15.36:0/1121015950 learned_addr learned my addr 172.21.15.36:0/1121015950 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:57.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f705cf8b700 1 --1- 172.21.15.36:0/1121015950 >> v1:172.21.15.36:6789/0 conn(0x7f70580ad330 0x7f70580ad770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55112/0 (socket says 172.21.15.36:55112) 2022-06-17T12:54:57.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f704f7fe700 1 --2- 172.21.15.36:0/1121015950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70581444f0 0x7f7058144930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f7057fff700 1 -- 172.21.15.36:0/1121015950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70581444f0 msgr2=0x7f7058144930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:57.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f7057fff700 1 --2- 172.21.15.36:0/1121015950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70581444f0 0x7f7058144930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f7057fff700 1 -- 172.21.15.36:0/1121015950 >> v1:172.21.15.36:6789/0 conn(0x7f70580ad330 legacy=0x7f70580ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:57.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f7057fff700 1 -- 172.21.15.36:0/1121015950 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7058147e90 con 0x7f7058062770 2022-06-17T12:54:57.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f7057fff700 1 --2- 172.21.15.36:0/1121015950 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7058062770 0x7f7058062b50 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7f7058061170 tx=0x7f7048006eb0).ready entity=mon.0 client_cookie=e824ac165eacc09d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:57.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f70577fe700 1 -- 172.21.15.36:0/1121015950 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f704800d5c0 con 0x7f7058062770 2022-06-17T12:54:57.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f70577fe700 1 -- 172.21.15.36:0/1121015950 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7048000e90 con 0x7f7058062770 2022-06-17T12:54:57.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f70577fe700 1 -- 172.21.15.36:0/1121015950 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f704800d950 con 0x7f7058062770 2022-06-17T12:54:57.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f705e9ee700 1 -- 172.21.15.36:0/1121015950 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7058062770 msgr2=0x7f7058062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:57.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f705e9ee700 1 --2- 172.21.15.36:0/1121015950 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7058062770 0x7f7058062b50 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7f7058061170 tx=0x7f7048006eb0).stop 2022-06-17T12:54:57.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f705e9ee700 1 -- 172.21.15.36:0/1121015950 shutdown_connections 2022-06-17T12:54:57.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f705e9ee700 1 --2- 172.21.15.36:0/1121015950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70581444f0 0x7f7058144930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f705e9ee700 1 --2- 172.21.15.36:0/1121015950 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7058062770 0x7f7058062b50 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f705e9ee700 1 -- 172.21.15.36:0/1121015950 >> 172.21.15.36:0/1121015950 conn(0x7f70580b53e0 msgr2=0x7f70580b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:57.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f705e9ee700 1 -- 172.21.15.36:0/1121015950 shutdown_connections 2022-06-17T12:54:57.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.111+0000 7f705e9ee700 1 -- 172.21.15.36:0/1121015950 wait complete. 2022-06-17T12:54:57.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.115+0000 7f705e9ee700 1 Processor -- start 2022-06-17T12:54:57.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.115+0000 7f705e9ee700 1 -- start start 2022-06-17T12:54:57.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.115+0000 7f705e9ee700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7058062770 0x7f7058062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.115+0000 7f705e9ee700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70580ad330 0x7f70580adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.115+0000 7f705e9ee700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70581444f0 0x7f7058063d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.115+0000 7f705e9ee700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7058159e70 con 0x7f7058062770 2022-06-17T12:54:57.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.115+0000 7f705e9ee700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705815a110 con 0x7f70581444f0 2022-06-17T12:54:57.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.115+0000 7f705e9ee700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7058067040 con 0x7f70580ad330 2022-06-17T12:54:57.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.115+0000 7f7057fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7058062770 0x7f7058062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.115+0000 7f7057fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7058062770 0x7f7058062b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37584/0 (socket says 172.21.15.36:37584) 2022-06-17T12:54:57.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.115+0000 7f7057fff700 1 -- 172.21.15.36:0/1661857526 learned_addr learned my addr 172.21.15.36:0/1661857526 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:57.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.119+0000 7f704f7fe700 1 --2- 172.21.15.36:0/1661857526 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70580ad330 0x7f70580adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.119+0000 7f705cf8b700 1 --2- 172.21.15.36:0/1661857526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70581444f0 0x7f7058063d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.119+0000 7f7057fff700 1 -- 172.21.15.36:0/1661857526 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70580ad330 msgr2=0x7f70580adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:57.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.119+0000 7f7057fff700 1 --2- 172.21.15.36:0/1661857526 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70580ad330 0x7f70580adb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.119+0000 7f7057fff700 1 -- 172.21.15.36:0/1661857526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70581444f0 msgr2=0x7f7058063d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:57.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.119+0000 7f7057fff700 1 --2- 172.21.15.36:0/1661857526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70581444f0 0x7f7058063d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.119+0000 7f7057fff700 1 -- 172.21.15.36:0/1661857526 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704800b1e0 con 0x7f7058062770 2022-06-17T12:54:57.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.119+0000 7f7057fff700 1 --2- 172.21.15.36:0/1661857526 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7058062770 0x7f7058062b50 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0x7f7058061170 tx=0x7f7048015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:57.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.119+0000 7f7055ffb700 1 -- 172.21.15.36:0/1661857526 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7048002f50 con 0x7f7058062770 2022-06-17T12:54:57.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.119+0000 7f7055ffb700 1 -- 172.21.15.36:0/1661857526 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704800de00 con 0x7f7058062770 2022-06-17T12:54:57.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.119+0000 7f705e9ee700 1 -- 172.21.15.36:0/1661857526 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7058066cc0 con 0x7f7058062770 2022-06-17T12:54:57.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.119+0000 7f7055ffb700 1 -- 172.21.15.36:0/1661857526 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7048011d70 con 0x7f7058062770 2022-06-17T12:54:57.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.119+0000 7f705e9ee700 1 -- 172.21.15.36:0/1661857526 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7058064240 con 0x7f7058062770 2022-06-17T12:54:57.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.119+0000 7f705e9ee700 1 -- 172.21.15.36:0/1661857526 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70580b1c20 con 0x7f7058062770 2022-06-17T12:54:57.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.119+0000 7f7055ffb700 1 -- 172.21.15.36:0/1661857526 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f7048025030 con 0x7f7058062770 2022-06-17T12:54:57.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.119+0000 7f7055ffb700 1 --2- 172.21.15.36:0/1661857526 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f703c053af0 0x7f703c055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.119+0000 7f7055ffb700 1 -- 172.21.15.36:0/1661857526 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f704806ce30 con 0x7f7058062770 2022-06-17T12:54:57.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.119+0000 7f704f7fe700 1 --2- 172.21.15.36:0/1661857526 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f703c053af0 0x7f703c055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.123+0000 7f704f7fe700 1 --2- 172.21.15.36:0/1661857526 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f703c053af0 0x7f703c055fb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f704000b000 tx=0x7f7040006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:57.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.127+0000 7f7055ffb700 1 -- 172.21.15.36:0/1661857526 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7048047e60 con 0x7f7058062770 2022-06-17T12:54:57.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.255+0000 7f705e9ee700 1 -- 172.21.15.36:0/1661857526 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7058057410 con 0x7f7058062770 2022-06-17T12:54:57.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.255+0000 7f7055ffb700 1 -- 172.21.15.36:0/1661857526 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+13416 (secure 0 0 0) 0x7f7048023070 con 0x7f7058062770 2022-06-17T12:54:57.245 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:57.245 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":100,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:54:56.625246+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:54:57.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.259+0000 7f705e9ee700 1 -- 172.21.15.36:0/1661857526 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f703c053af0 msgr2=0x7f703c055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:57.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.259+0000 7f705e9ee700 1 --2- 172.21.15.36:0/1661857526 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f703c053af0 0x7f703c055fb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f704000b000 tx=0x7f7040006df0).stop 2022-06-17T12:54:57.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.259+0000 7f705e9ee700 1 -- 172.21.15.36:0/1661857526 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7058062770 msgr2=0x7f7058062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:57.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.259+0000 7f705e9ee700 1 --2- 172.21.15.36:0/1661857526 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7058062770 0x7f7058062b50 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0x7f7058061170 tx=0x7f7048015000).stop 2022-06-17T12:54:57.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.259+0000 7f705e9ee700 1 -- 172.21.15.36:0/1661857526 shutdown_connections 2022-06-17T12:54:57.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.259+0000 7f705e9ee700 1 --2- 172.21.15.36:0/1661857526 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f703c053af0 0x7f703c055fb0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.259+0000 7f705e9ee700 1 --2- 172.21.15.36:0/1661857526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70581444f0 0x7f7058063d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.259+0000 7f705e9ee700 1 --2- 172.21.15.36:0/1661857526 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70580ad330 0x7f70580adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.259+0000 7f705e9ee700 1 --2- 172.21.15.36:0/1661857526 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7058062770 0x7f7058062b50 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.259+0000 7f705e9ee700 1 -- 172.21.15.36:0/1661857526 >> 172.21.15.36:0/1661857526 conn(0x7f70580b53e0 msgr2=0x7f70581453b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:57.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.259+0000 7f705e9ee700 1 -- 172.21.15.36:0/1661857526 shutdown_connections 2022-06-17T12:54:57.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.259+0000 7f705e9ee700 1 -- 172.21.15.36:0/1661857526 wait complete. 2022-06-17T12:54:57.260 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-17T12:54:57.261 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:54:57.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.387+0000 7fdebc5ac700 1 Processor -- start 2022-06-17T12:54:57.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.387+0000 7fdebc5ac700 1 -- start start 2022-06-17T12:54:57.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdebc5ac700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdeb40b4190 0x7fdeb40b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdebc5ac700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fdeb4143d40 0x7fdeb41471d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdebc5ac700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeb4143130 con 0x7fdeb4062270 2022-06-17T12:54:57.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdebc5ac700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb4144140 con 0x7fdeb4143d40 2022-06-17T12:54:57.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdebc5ac700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb40b6080 con 0x7fdeb40b4190 2022-06-17T12:54:57.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdebab49700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fdeb4143d40 0x7fdeb41471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdebab49700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fdeb4143d40 0x7fdeb41471d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37594/0 (socket says 172.21.15.36:37594) 2022-06-17T12:54:57.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdebab49700 1 -- 172.21.15.36:0/2012537114 learned_addr learned my addr 172.21.15.36:0/2012537114 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:57.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdeb9b47700 1 --2- 172.21.15.36:0/2012537114 >> v2:172.21.15.70:3300/0 conn(0x7fdeb40b4190 0x7fdeb40b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdebab49700 1 -- 172.21.15.36:0/2012537114 >> v2:172.21.15.70:3300/0 conn(0x7fdeb40b4190 msgr2=0x7fdeb40b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:57.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdebab49700 1 --2- 172.21.15.36:0/2012537114 >> v2:172.21.15.70:3300/0 conn(0x7fdeb40b4190 0x7fdeb40b4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdebab49700 1 -- 172.21.15.36:0/2012537114 >> v1:172.21.15.36:6789/0 conn(0x7fdeb4062270 legacy=0x7fdeb40b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:57.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdebab49700 1 -- 172.21.15.36:0/2012537114 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb4062650 con 0x7fdeb4143d40 2022-06-17T12:54:57.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdebab49700 1 --2- 172.21.15.36:0/2012537114 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdeb4143d40 0x7fdeb41471d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 rx=0x7fdeac0032f0 tx=0x7fdeac009290).ready entity=mon.0 client_cookie=dc5d096e4ebbb7f8 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:57.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdeb9346700 1 -- 172.21.15.36:0/2012537114 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdeac00b870 con 0x7fdeb4143d40 2022-06-17T12:54:57.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdeb9346700 1 -- 172.21.15.36:0/2012537114 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeac00b9d0 con 0x7fdeb4143d40 2022-06-17T12:54:57.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdeb9346700 1 -- 172.21.15.36:0/2012537114 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdeac00bb40 con 0x7fdeb4143d40 2022-06-17T12:54:57.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdebc5ac700 1 -- 172.21.15.36:0/2012537114 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdeb4143d40 msgr2=0x7fdeb41471d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:57.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdebc5ac700 1 --2- 172.21.15.36:0/2012537114 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdeb4143d40 0x7fdeb41471d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 rx=0x7fdeac0032f0 tx=0x7fdeac009290).stop 2022-06-17T12:54:57.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdebc5ac700 1 -- 172.21.15.36:0/2012537114 shutdown_connections 2022-06-17T12:54:57.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdebc5ac700 1 --2- 172.21.15.36:0/2012537114 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdeb4143d40 0x7fdeb41471d0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdebc5ac700 1 --2- 172.21.15.36:0/2012537114 >> v2:172.21.15.70:3300/0 conn(0x7fdeb40b4190 0x7fdeb40b4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdebc5ac700 1 -- 172.21.15.36:0/2012537114 >> 172.21.15.36:0/2012537114 conn(0x7fdeb40b97f0 msgr2=0x7fdeb40b9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:57.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdebc5ac700 1 -- 172.21.15.36:0/2012537114 shutdown_connections 2022-06-17T12:54:57.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.391+0000 7fdebc5ac700 1 -- 172.21.15.36:0/2012537114 wait complete. 2022-06-17T12:54:57.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.395+0000 7fdebc5ac700 1 Processor -- start 2022-06-17T12:54:57.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.395+0000 7fdebc5ac700 1 -- start start 2022-06-17T12:54:57.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.395+0000 7fdebc5ac700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdeb4062270 0x7fdeb40b7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.395+0000 7fdebc5ac700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdeb40b4190 0x7fdeb4128b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.395+0000 7fdebc5ac700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdeb4143d40 0x7fdeb4129030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.395+0000 7fdebc5ac700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb4146a00 con 0x7fdeb4062270 2022-06-17T12:54:57.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.395+0000 7fdebc5ac700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb4146ca0 con 0x7fdeb4143d40 2022-06-17T12:54:57.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.395+0000 7fdebc5ac700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb4146f40 con 0x7fdeb40b4190 2022-06-17T12:54:57.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.395+0000 7fdeba348700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdeb4062270 0x7fdeb40b7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.395+0000 7fdeba348700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdeb4062270 0x7fdeb40b7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37598/0 (socket says 172.21.15.36:37598) 2022-06-17T12:54:57.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.395+0000 7fdeba348700 1 -- 172.21.15.36:0/3901844988 learned_addr learned my addr 172.21.15.36:0/3901844988 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:57.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.395+0000 7fdeb9b47700 1 --2- 172.21.15.36:0/3901844988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdeb40b4190 0x7fdeb4128b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.395+0000 7fdebab49700 1 --2- 172.21.15.36:0/3901844988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdeb4143d40 0x7fdeb4129030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.395+0000 7fdeba348700 1 -- 172.21.15.36:0/3901844988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdeb40b4190 msgr2=0x7fdeb4128b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:57.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.395+0000 7fdeba348700 1 --2- 172.21.15.36:0/3901844988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdeb40b4190 0x7fdeb4128b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.395+0000 7fdeba348700 1 -- 172.21.15.36:0/3901844988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdeb4143d40 msgr2=0x7fdeb4129030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:57.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.395+0000 7fdeba348700 1 --2- 172.21.15.36:0/3901844988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdeb4143d40 0x7fdeb4129030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.395+0000 7fdeba348700 1 -- 172.21.15.36:0/3901844988 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac009530 con 0x7fdeb4062270 2022-06-17T12:54:57.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.395+0000 7fdeba348700 1 --2- 172.21.15.36:0/3901844988 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdeb4062270 0x7fdeb40b7ff0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7fdeb4063130 tx=0x7fdeb0007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:57.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.395+0000 7fdea37fe700 1 -- 172.21.15.36:0/3901844988 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdeb000ce30 con 0x7fdeb4062270 2022-06-17T12:54:57.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.395+0000 7fdea37fe700 1 -- 172.21.15.36:0/3901844988 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb0007240 con 0x7fdeb4062270 2022-06-17T12:54:57.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.395+0000 7fdebc5ac700 1 -- 172.21.15.36:0/3901844988 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb4146680 con 0x7fdeb4062270 2022-06-17T12:54:57.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.395+0000 7fdea37fe700 1 -- 172.21.15.36:0/3901844988 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdeb0006700 con 0x7fdeb4062270 2022-06-17T12:54:57.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.395+0000 7fdebc5ac700 1 -- 172.21.15.36:0/3901844988 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb40b3900 con 0x7fdeb4062270 2022-06-17T12:54:57.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.399+0000 7fdebc5ac700 1 -- 172.21.15.36:0/3901844988 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeb4147100 con 0x7fdeb4062270 2022-06-17T12:54:57.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.399+0000 7fdea37fe700 1 -- 172.21.15.36:0/3901844988 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fdeb0010030 con 0x7fdeb4062270 2022-06-17T12:54:57.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.399+0000 7fdea37fe700 1 --2- 172.21.15.36:0/3901844988 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde94053ad0 0x7fde94055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.399+0000 7fdeb9b47700 1 --2- 172.21.15.36:0/3901844988 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde94053ad0 0x7fde94055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.399+0000 7fdea37fe700 1 -- 172.21.15.36:0/3901844988 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fdeb006baa0 con 0x7fdeb4062270 2022-06-17T12:54:57.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.399+0000 7fdeb9b47700 1 --2- 172.21.15.36:0/3901844988 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde94053ad0 0x7fde94055f90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7fdea400b000 tx=0x7fdea4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:57.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.407+0000 7fdea37fe700 1 -- 172.21.15.36:0/3901844988 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdeb0042ad0 con 0x7fdeb4062270 2022-06-17T12:54:57.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.531+0000 7fdebc5ac700 1 -- 172.21.15.36:0/3901844988 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fdeb4146810 con 0x7fdeb4062270 2022-06-17T12:54:57.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.531+0000 7fdea37fe700 1 -- 172.21.15.36:0/3901844988 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v100) v1 ==== 110+0+10 (secure 0 0 0) 0x7fdeb00460f0 con 0x7fdeb4062270 2022-06-17T12:54:57.523 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:54:57.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.535+0000 7fdebc5ac700 1 -- 172.21.15.36:0/3901844988 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde94053ad0 msgr2=0x7fde94055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:57.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.535+0000 7fdebc5ac700 1 --2- 172.21.15.36:0/3901844988 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde94053ad0 0x7fde94055f90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7fdea400b000 tx=0x7fdea4008040).stop 2022-06-17T12:54:57.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.535+0000 7fdebc5ac700 1 -- 172.21.15.36:0/3901844988 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdeb4062270 msgr2=0x7fdeb40b7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:57.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.535+0000 7fdebc5ac700 1 --2- 172.21.15.36:0/3901844988 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdeb4062270 0x7fdeb40b7ff0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7fdeb4063130 tx=0x7fdeb0007c10).stop 2022-06-17T12:54:57.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.535+0000 7fdebc5ac700 1 -- 172.21.15.36:0/3901844988 shutdown_connections 2022-06-17T12:54:57.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.535+0000 7fdebc5ac700 1 --2- 172.21.15.36:0/3901844988 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde94053ad0 0x7fde94055f90 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.535+0000 7fdebc5ac700 1 --2- 172.21.15.36:0/3901844988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdeb4143d40 0x7fdeb4129030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.535+0000 7fdebc5ac700 1 --2- 172.21.15.36:0/3901844988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdeb40b4190 0x7fdeb4128b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.535+0000 7fdebc5ac700 1 --2- 172.21.15.36:0/3901844988 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdeb4062270 0x7fdeb40b7ff0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.535+0000 7fdebc5ac700 1 -- 172.21.15.36:0/3901844988 >> 172.21.15.36:0/3901844988 conn(0x7fdeb40b97f0 msgr2=0x7fdeb40b5250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:57.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.535+0000 7fdebc5ac700 1 -- 172.21.15.36:0/3901844988 shutdown_connections 2022-06-17T12:54:57.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.535+0000 7fdebc5ac700 1 -- 172.21.15.36:0/3901844988 wait complete. 2022-06-17T12:54:57.534 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:54:57.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.651+0000 7f2dc7c9b700 1 Processor -- start 2022-06-17T12:54:57.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.651+0000 7f2dc7c9b700 1 -- start start 2022-06-17T12:54:57.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc7c9b700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2dc0060e40 0x7f2dc01451e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc7c9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2dc00b1360 0x7f2dc0066a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc7c9b700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc00b1830 con 0x7f2dc00aed10 2022-06-17T12:54:57.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc7c9b700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dc01423e0 con 0x7f2dc00b1360 2022-06-17T12:54:57.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc7c9b700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dc0062b30 con 0x7f2dc0060e40 2022-06-17T12:54:57.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc6238700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2dc00b1360 0x7f2dc0066a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc6238700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2dc00b1360 0x7f2dc0066a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37610/0 (socket says 172.21.15.36:37610) 2022-06-17T12:54:57.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc6238700 1 -- 172.21.15.36:0/1429612384 learned_addr learned my addr 172.21.15.36:0/1429612384 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:57.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc5236700 1 --2- 172.21.15.36:0/1429612384 >> v2:172.21.15.70:3300/0 conn(0x7f2dc0060e40 0x7f2dc01451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc6238700 1 -- 172.21.15.36:0/1429612384 >> v2:172.21.15.70:3300/0 conn(0x7f2dc0060e40 msgr2=0x7f2dc01451e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:57.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc6238700 1 --2- 172.21.15.36:0/1429612384 >> v2:172.21.15.70:3300/0 conn(0x7f2dc0060e40 0x7f2dc01451e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc6238700 1 -- 172.21.15.36:0/1429612384 >> v1:172.21.15.70:6789/0 conn(0x7f2dc00aed10 legacy=0x7f2dc01425f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:57.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc6238700 1 -- 172.21.15.36:0/1429612384 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc00b0440 con 0x7f2dc00b1360 2022-06-17T12:54:57.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc6238700 1 --2- 172.21.15.36:0/1429612384 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2dc00b1360 0x7f2dc0066a80 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0x7f2db80032f0 tx=0x7f2db8009290).ready entity=mon.0 client_cookie=e86c474517c8a97d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:57.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc4a35700 1 -- 172.21.15.36:0/1429612384 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2db800b870 con 0x7f2dc00b1360 2022-06-17T12:54:57.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc4a35700 1 -- 172.21.15.36:0/1429612384 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db800b9d0 con 0x7f2dc00b1360 2022-06-17T12:54:57.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc4a35700 1 -- 172.21.15.36:0/1429612384 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2db800bb40 con 0x7f2dc00b1360 2022-06-17T12:54:57.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc7c9b700 1 -- 172.21.15.36:0/1429612384 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2dc00b1360 msgr2=0x7f2dc0066a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:57.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc7c9b700 1 --2- 172.21.15.36:0/1429612384 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2dc00b1360 0x7f2dc0066a80 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0x7f2db80032f0 tx=0x7f2db8009290).stop 2022-06-17T12:54:57.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc7c9b700 1 -- 172.21.15.36:0/1429612384 shutdown_connections 2022-06-17T12:54:57.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc7c9b700 1 --2- 172.21.15.36:0/1429612384 >> v2:172.21.15.70:3300/0 conn(0x7f2dc0060e40 0x7f2dc01451e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc7c9b700 1 --2- 172.21.15.36:0/1429612384 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2dc00b1360 0x7f2dc0066a80 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc7c9b700 1 -- 172.21.15.36:0/1429612384 >> 172.21.15.36:0/1429612384 conn(0x7f2dc005c550 msgr2=0x7f2dc00b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:57.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc7c9b700 1 -- 172.21.15.36:0/1429612384 shutdown_connections 2022-06-17T12:54:57.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.655+0000 7f2dc7c9b700 1 -- 172.21.15.36:0/1429612384 wait complete. 2022-06-17T12:54:57.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2dc7c9b700 1 Processor -- start 2022-06-17T12:54:57.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2dc7c9b700 1 -- start start 2022-06-17T12:54:57.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2dc7c9b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2dc0060e40 0x7f2dc00a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2dc7c9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dc00aed10 0x7f2dc00ab570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2dc7c9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2dc00b1360 0x7f2dc00aba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2dc7c9b700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc00a22d0 con 0x7f2dc00b1360 2022-06-17T12:54:57.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2dc7c9b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc009ee50 con 0x7f2dc00aed10 2022-06-17T12:54:57.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2dc7c9b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc00a40f0 con 0x7f2dc0060e40 2022-06-17T12:54:57.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2dc6238700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2dc00b1360 0x7f2dc00aba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2dc5236700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dc00aed10 0x7f2dc00ab570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2dc6238700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2dc00b1360 0x7f2dc00aba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37616/0 (socket says 172.21.15.36:37616) 2022-06-17T12:54:57.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2dc5236700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dc00aed10 0x7f2dc00ab570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:57996/0 (socket says 172.21.15.36:57996) 2022-06-17T12:54:57.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2dc5a37700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2dc0060e40 0x7f2dc00a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2dc5236700 1 -- 172.21.15.36:0/3730377332 learned_addr learned my addr 172.21.15.36:0/3730377332 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:57.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2dc6238700 1 -- 172.21.15.36:0/3730377332 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2dc0060e40 msgr2=0x7f2dc00a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:57.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2dc6238700 1 --2- 172.21.15.36:0/3730377332 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2dc0060e40 0x7f2dc00a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2dc6238700 1 -- 172.21.15.36:0/3730377332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dc00aed10 msgr2=0x7f2dc00ab570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:57.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2dc6238700 1 --2- 172.21.15.36:0/3730377332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dc00aed10 0x7f2dc00ab570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2dc6238700 1 -- 172.21.15.36:0/3730377332 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db8009530 con 0x7f2dc00b1360 2022-06-17T12:54:57.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2dc5236700 1 --2- 172.21.15.36:0/3730377332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dc00aed10 0x7f2dc00ab570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:57.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2dc6238700 1 --2- 172.21.15.36:0/3730377332 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2dc00b1360 0x7f2dc00aba60 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7f2dc009ef90 tx=0x7f2db8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:57.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2daeffd700 1 -- 172.21.15.36:0/3730377332 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2db800b870 con 0x7f2dc00b1360 2022-06-17T12:54:57.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2daeffd700 1 -- 172.21.15.36:0/3730377332 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db8021440 con 0x7f2dc00b1360 2022-06-17T12:54:57.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2dc7c9b700 1 -- 172.21.15.36:0/3730377332 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc00a3e50 con 0x7f2dc00b1360 2022-06-17T12:54:57.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2daeffd700 1 -- 172.21.15.36:0/3730377332 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2db80217a0 con 0x7f2dc00b1360 2022-06-17T12:54:57.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.659+0000 7f2dc7c9b700 1 -- 172.21.15.36:0/3730377332 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc00a3b60 con 0x7f2dc00b1360 2022-06-17T12:54:57.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.663+0000 7f2dc7c9b700 1 -- 172.21.15.36:0/3730377332 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dc00b1a60 con 0x7f2dc00b1360 2022-06-17T12:54:57.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.663+0000 7f2daeffd700 1 -- 172.21.15.36:0/3730377332 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2db8027020 con 0x7f2dc00b1360 2022-06-17T12:54:57.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.663+0000 7f2daeffd700 1 --2- 172.21.15.36:0/3730377332 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2da0053af0 0x7f2da0055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.663+0000 7f2daeffd700 1 -- 172.21.15.36:0/3730377332 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f2db806d190 con 0x7f2dc00b1360 2022-06-17T12:54:57.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.663+0000 7f2dc5a37700 1 --2- 172.21.15.36:0/3730377332 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2da0053af0 0x7f2da0055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.663+0000 7f2dc5a37700 1 --2- 172.21.15.36:0/3730377332 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2da0053af0 0x7f2da0055fb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f2db000d000 tx=0x7f2db0000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:57.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.667+0000 7f2daeffd700 1 -- 172.21.15.36:0/3730377332 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2db8044240 con 0x7f2dc00b1360 2022-06-17T12:54:57.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.795+0000 7f2dc7c9b700 1 -- 172.21.15.36:0/3730377332 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2dc0057410 con 0x7f2dc00b1360 2022-06-17T12:54:57.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.795+0000 7f2daeffd700 1 -- 172.21.15.36:0/3730377332 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v100) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2db80150a0 con 0x7f2dc00b1360 2022-06-17T12:54:57.786 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:54:57.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.799+0000 7f2dc7c9b700 1 -- 172.21.15.36:0/3730377332 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2da0053af0 msgr2=0x7f2da0055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:57.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.799+0000 7f2dc7c9b700 1 --2- 172.21.15.36:0/3730377332 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2da0053af0 0x7f2da0055fb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f2db000d000 tx=0x7f2db0000bc0).stop 2022-06-17T12:54:57.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.799+0000 7f2dc7c9b700 1 -- 172.21.15.36:0/3730377332 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2dc00b1360 msgr2=0x7f2dc00aba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:57.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.799+0000 7f2dc7c9b700 1 --2- 172.21.15.36:0/3730377332 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2dc00b1360 0x7f2dc00aba60 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7f2dc009ef90 tx=0x7f2db8016000).stop 2022-06-17T12:54:57.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.799+0000 7f2dc7c9b700 1 -- 172.21.15.36:0/3730377332 shutdown_connections 2022-06-17T12:54:57.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.799+0000 7f2dc7c9b700 1 --2- 172.21.15.36:0/3730377332 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2da0053af0 0x7f2da0055fb0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.799+0000 7f2dc7c9b700 1 --2- 172.21.15.36:0/3730377332 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2dc00b1360 0x7f2dc00aba60 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.799+0000 7f2dc7c9b700 1 --2- 172.21.15.36:0/3730377332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dc00aed10 0x7f2dc00ab570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.799+0000 7f2dc7c9b700 1 --2- 172.21.15.36:0/3730377332 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2dc0060e40 0x7f2dc00a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.799+0000 7f2dc7c9b700 1 -- 172.21.15.36:0/3730377332 >> 172.21.15.36:0/3730377332 conn(0x7f2dc005c550 msgr2=0x7f2dc005f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:57.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.799+0000 7f2dc7c9b700 1 -- 172.21.15.36:0/3730377332 shutdown_connections 2022-06-17T12:54:57.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.799+0000 7f2dc7c9b700 1 -- 172.21.15.36:0/3730377332 wait complete. 2022-06-17T12:54:57.798 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-17T12:54:57.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.907+0000 7f589d25c700 1 Processor -- start 2022-06-17T12:54:57.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.907+0000 7f589d25c700 1 -- start start 2022-06-17T12:54:57.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.907+0000 7f589d25c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5898140c90 0x7f5898143480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.907+0000 7f589d25c700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5898142160 con 0x7f58980b80c0 2022-06-17T12:54:57.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.907+0000 7f589d25c700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58980b4f30 con 0x7f58980b7ce0 2022-06-17T12:54:57.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.907+0000 7f589d25c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5898142390 con 0x7f5898140c90 2022-06-17T12:54:57.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.907+0000 7f58977fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f58980b80c0 0x7f5898142d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55152/0 (socket says 172.21.15.36:55152) 2022-06-17T12:54:57.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.907+0000 7f58977fe700 1 -- 172.21.15.36:0/1611719865 learned_addr learned my addr 172.21.15.36:0/1611719865 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:57.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f589ca5b700 1 --2- 172.21.15.36:0/1611719865 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5898140c90 0x7f5898143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5896ffd700 1 -- 172.21.15.36:0/1611719865 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 504022587 0 0) 0x7f5898142160 con 0x7f58980b80c0 2022-06-17T12:54:57.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5896ffd700 1 -- 172.21.15.36:0/1611719865 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5884003260 con 0x7f58980b80c0 2022-06-17T12:54:57.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5896ffd700 1 -- 172.21.15.36:0/1611719865 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3741442970 0 0) 0x7f5884003260 con 0x7f58980b80c0 2022-06-17T12:54:57.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5896ffd700 1 -- 172.21.15.36:0/1611719865 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5898140c90 msgr2=0x7f5898143480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:57.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5896ffd700 1 --2- 172.21.15.36:0/1611719865 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5898140c90 0x7f5898143480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5896ffd700 1 -- 172.21.15.36:0/1611719865 >> v1:172.21.15.70:6789/0 conn(0x7f58980b7ce0 legacy=0x7f589813aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:57.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5896ffd700 1 -- 172.21.15.36:0/1611719865 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5898142630 con 0x7f58980b80c0 2022-06-17T12:54:57.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5896ffd700 1 -- 172.21.15.36:0/1611719865 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f588c001660 con 0x7f58980b80c0 2022-06-17T12:54:57.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5896ffd700 1 -- 172.21.15.36:0/1611719865 >> v1:172.21.15.36:6789/0 conn(0x7f58980b80c0 legacy=0x7f5898142d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:57.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5896ffd700 1 --2- 172.21.15.36:0/1611719865 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5884003780 0x7f5884003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5896ffd700 1 --2- 172.21.15.36:0/1611719865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5884004500 0x7f5884004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5896ffd700 1 --2- 172.21.15.36:0/1611719865 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5898140c90 0x7f5884004f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5896ffd700 1 -- 172.21.15.36:0/1611719865 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5898142390 con 0x7f5884003780 2022-06-17T12:54:57.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5896ffd700 1 -- 172.21.15.36:0/1611719865 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588400b830 con 0x7f5884004500 2022-06-17T12:54:57.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5896ffd700 1 -- 172.21.15.36:0/1611719865 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588400b9f0 con 0x7f5898140c90 2022-06-17T12:54:57.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5897fff700 1 --2- 172.21.15.36:0/1611719865 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5884003780 0x7f5884003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f58977fe700 1 --2- 172.21.15.36:0/1611719865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5884004500 0x7f5884004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f589ca5b700 1 --2- 172.21.15.36:0/1611719865 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5898140c90 0x7f5884004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5897fff700 1 -- 172.21.15.36:0/1611719865 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5898140c90 msgr2=0x7f5884004f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:57.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5897fff700 1 --2- 172.21.15.36:0/1611719865 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5898140c90 0x7f5884004f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5897fff700 1 -- 172.21.15.36:0/1611719865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5884004500 msgr2=0x7f5884004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:57.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5897fff700 1 --2- 172.21.15.36:0/1611719865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5884004500 0x7f5884004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5897fff700 1 -- 172.21.15.36:0/1611719865 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5884003260 con 0x7f5884003780 2022-06-17T12:54:57.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5897fff700 1 --2- 172.21.15.36:0/1611719865 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5884003780 0x7f5884003ba0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7f589805cd80 tx=0x7f5888007a00).ready entity=mon.0 client_cookie=a3c1ced302de7728 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:57.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5896ffd700 1 -- 172.21.15.36:0/1611719865 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5888007d10 con 0x7f5884003780 2022-06-17T12:54:57.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5896ffd700 1 -- 172.21.15.36:0/1611719865 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5888007030 con 0x7f5884003780 2022-06-17T12:54:57.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f5896ffd700 1 -- 172.21.15.36:0/1611719865 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5888007390 con 0x7f5884003780 2022-06-17T12:54:57.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f589d25c700 1 -- 172.21.15.36:0/1611719865 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5884003780 msgr2=0x7f5884003ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:57.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f589d25c700 1 --2- 172.21.15.36:0/1611719865 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5884003780 0x7f5884003ba0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7f589805cd80 tx=0x7f5888007a00).stop 2022-06-17T12:54:57.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f589ca5b700 1 -- 172.21.15.36:0/1611719865 reap_dead start 2022-06-17T12:54:57.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f589d25c700 1 -- 172.21.15.36:0/1611719865 shutdown_connections 2022-06-17T12:54:57.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f589d25c700 1 -- 172.21.15.36:0/1611719865 >> 172.21.15.36:0/1611719865 conn(0x7f589805c550 msgr2=0x7f58980b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:57.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f589d25c700 1 -- 172.21.15.36:0/1611719865 shutdown_connections 2022-06-17T12:54:57.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.911+0000 7f589d25c700 1 -- 172.21.15.36:0/1611719865 wait complete. 2022-06-17T12:54:57.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.915+0000 7f589d25c700 1 Processor -- start 2022-06-17T12:54:57.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.915+0000 7f589d25c700 1 -- start start 2022-06-17T12:54:57.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.915+0000 7f589d25c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5884003780 0x7f5898064740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.915+0000 7f589d25c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5898065920 0x7f5898065da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.915+0000 7f589d25c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5898066290 0x7f58981153f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.915+0000 7f589d25c700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58980670f0 con 0x7f5884003780 2022-06-17T12:54:57.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.915+0000 7f589d25c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58981525a0 con 0x7f5898066290 2022-06-17T12:54:57.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.915+0000 7f589d25c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5898152840 con 0x7f5898065920 2022-06-17T12:54:57.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.915+0000 7f5897fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5884003780 0x7f5898064740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.915+0000 7f5897fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5884003780 0x7f5898064740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37632/0 (socket says 172.21.15.36:37632) 2022-06-17T12:54:57.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.915+0000 7f5897fff700 1 -- 172.21.15.36:0/650657378 learned_addr learned my addr 172.21.15.36:0/650657378 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:57.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.915+0000 7f589ca5b700 1 --2- 172.21.15.36:0/650657378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5898066290 0x7f58981153f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.915+0000 7f58977fe700 1 --2- 172.21.15.36:0/650657378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5898065920 0x7f5898065da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.915+0000 7f5897fff700 1 -- 172.21.15.36:0/650657378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5898065920 msgr2=0x7f5898065da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:57.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.915+0000 7f5897fff700 1 --2- 172.21.15.36:0/650657378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5898065920 0x7f5898065da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.915+0000 7f5897fff700 1 -- 172.21.15.36:0/650657378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5898066290 msgr2=0x7f58981153f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:57.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.915+0000 7f5897fff700 1 --2- 172.21.15.36:0/650657378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5898066290 0x7f58981153f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:57.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.915+0000 7f5897fff700 1 -- 172.21.15.36:0/650657378 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5898152ae0 con 0x7f5884003780 2022-06-17T12:54:57.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.915+0000 7f5897fff700 1 --2- 172.21.15.36:0/650657378 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5884003780 0x7f5898064740 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7f589805cd80 tx=0x7f5888015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:57.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.915+0000 7f58957fa700 1 -- 172.21.15.36:0/650657378 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58880127b0 con 0x7f5884003780 2022-06-17T12:54:57.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.915+0000 7f58957fa700 1 -- 172.21.15.36:0/650657378 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5888012910 con 0x7f5884003780 2022-06-17T12:54:57.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.915+0000 7f589d25c700 1 -- 172.21.15.36:0/650657378 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5898064c30 con 0x7f5884003780 2022-06-17T12:54:57.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.915+0000 7f58957fa700 1 -- 172.21.15.36:0/650657378 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5888012c70 con 0x7f5884003780 2022-06-17T12:54:57.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.915+0000 7f589d25c700 1 -- 172.21.15.36:0/650657378 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5898152f80 con 0x7f5884003780 2022-06-17T12:54:57.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.919+0000 7f589d25c700 1 -- 172.21.15.36:0/650657378 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58981523d0 con 0x7f5884003780 2022-06-17T12:54:57.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.919+0000 7f58957fa700 1 -- 172.21.15.36:0/650657378 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f588801d070 con 0x7f5884003780 2022-06-17T12:54:57.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.919+0000 7f58957fa700 1 --2- 172.21.15.36:0/650657378 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5878053a60 0x7f5878055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:57.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.919+0000 7f58957fa700 1 -- 172.21.15.36:0/650657378 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f5888001e40 con 0x7f5884003780 2022-06-17T12:54:57.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.919+0000 7f58977fe700 1 --2- 172.21.15.36:0/650657378 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5878053a60 0x7f5878055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:57.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.919+0000 7f58977fe700 1 --2- 172.21.15.36:0/650657378 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5878053a60 0x7f5878055f20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f588c005000 tx=0x7f588c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:57.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:57.927+0000 7f58957fa700 1 -- 172.21.15.36:0/650657378 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5888043c60 con 0x7f5884003780 2022-06-17T12:54:58.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.055+0000 7f589d25c700 1 -- 172.21.15.36:0/650657378 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5898152c70 con 0x7f5884003780 2022-06-17T12:54:58.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.055+0000 7f58957fa700 1 -- 172.21.15.36:0/650657378 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v100) v1 ==== 84+0+143 (secure 0 0 0) 0x7f5888047240 con 0x7f5884003780 2022-06-17T12:54:58.052 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:54:58.053 INFO:teuthology.orchestra.run.smithi036.stdout:[ 2022-06-17T12:54:58.054 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:58.054 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 1, 2022-06-17T12:54:58.054 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "device_health_metrics" 2022-06-17T12:54:58.054 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:54:58.055 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:54:58.055 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 2, 2022-06-17T12:54:58.055 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "rbd" 2022-06-17T12:54:58.056 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:54:58.056 INFO:teuthology.orchestra.run.smithi036.stdout:] 2022-06-17T12:54:58.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.059+0000 7f589d25c700 1 -- 172.21.15.36:0/650657378 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5878053a60 msgr2=0x7f5878055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:58.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.059+0000 7f589d25c700 1 --2- 172.21.15.36:0/650657378 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5878053a60 0x7f5878055f20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f588c005000 tx=0x7f588c009040).stop 2022-06-17T12:54:58.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.059+0000 7f589d25c700 1 -- 172.21.15.36:0/650657378 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5884003780 msgr2=0x7f5898064740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:58.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.059+0000 7f589d25c700 1 --2- 172.21.15.36:0/650657378 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5884003780 0x7f5898064740 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7f589805cd80 tx=0x7f5888015000).stop 2022-06-17T12:54:58.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.059+0000 7f589d25c700 1 -- 172.21.15.36:0/650657378 shutdown_connections 2022-06-17T12:54:58.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.059+0000 7f589d25c700 1 --2- 172.21.15.36:0/650657378 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5878053a60 0x7f5878055f20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:58.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.059+0000 7f589d25c700 1 --2- 172.21.15.36:0/650657378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5898066290 0x7f58981153f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:58.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.059+0000 7f589d25c700 1 --2- 172.21.15.36:0/650657378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5898065920 0x7f5898065da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:58.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.059+0000 7f589d25c700 1 --2- 172.21.15.36:0/650657378 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5884003780 0x7f5898064740 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:58.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.059+0000 7f589d25c700 1 -- 172.21.15.36:0/650657378 >> 172.21.15.36:0/650657378 conn(0x7f589805c550 msgr2=0x7f589813e210 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:58.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.059+0000 7f589d25c700 1 -- 172.21.15.36:0/650657378 shutdown_connections 2022-06-17T12:54:58.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.059+0000 7f589d25c700 1 -- 172.21.15.36:0/650657378 wait complete. 2022-06-17T12:54:58.065 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-17T12:54:58.065 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-17T12:54:58.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba18d59700 1 Processor -- start 2022-06-17T12:54:58.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba18d59700 1 -- start start 2022-06-17T12:54:58.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba18d59700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba1405f770 0x7fba1413eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:58.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba18d59700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fba14067120 0x7fba1405ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:58.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba18d59700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba140abf90 con 0x7fba140b1770 2022-06-17T12:54:58.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba18d59700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba1413eb20 con 0x7fba14067120 2022-06-17T12:54:58.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba18d59700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba140ac1f0 con 0x7fba1405f770 2022-06-17T12:54:58.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba11d9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fba14067120 0x7fba1405ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:58.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba12d9d700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fba140b1770 0x7fba1413e410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55174/0 (socket says 172.21.15.36:55174) 2022-06-17T12:54:58.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba12d9d700 1 -- 172.21.15.36:0/124319105 learned_addr learned my addr 172.21.15.36:0/124319105 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:58.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba1259c700 1 --2- 172.21.15.36:0/124319105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba1405f770 0x7fba1413eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:58.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba11d9b700 1 -- 172.21.15.36:0/124319105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba1405f770 msgr2=0x7fba1413eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:58.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba11d9b700 1 --2- 172.21.15.36:0/124319105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba1405f770 0x7fba1413eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:58.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba11d9b700 1 -- 172.21.15.36:0/124319105 >> v1:172.21.15.36:6789/0 conn(0x7fba140b1770 legacy=0x7fba1413e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:58.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba1159a700 1 -- 172.21.15.36:0/124319105 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2155959860 0 0) 0x7fba140abf90 con 0x7fba140b1770 2022-06-17T12:54:58.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba11d9b700 1 -- 172.21.15.36:0/124319105 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba140ac490 con 0x7fba14067120 2022-06-17T12:54:58.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba11d9b700 1 --2- 172.21.15.36:0/124319105 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba14067120 0x7fba1405ff00 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 rx=0x7fba08002f70 tx=0x7fba08006df0).ready entity=mon.0 client_cookie=1f9fdb1af15488e5 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:58.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba1159a700 1 -- 172.21.15.36:0/124319105 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba0800d570 con 0x7fba14067120 2022-06-17T12:54:58.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba1159a700 1 -- 172.21.15.36:0/124319105 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba08000e90 con 0x7fba14067120 2022-06-17T12:54:58.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba1159a700 1 -- 172.21.15.36:0/124319105 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba0800d8c0 con 0x7fba14067120 2022-06-17T12:54:58.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba18d59700 1 -- 172.21.15.36:0/124319105 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba14067120 msgr2=0x7fba1405ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:58.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba18d59700 1 --2- 172.21.15.36:0/124319105 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba14067120 0x7fba1405ff00 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 rx=0x7fba08002f70 tx=0x7fba08006df0).stop 2022-06-17T12:54:58.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba18d59700 1 -- 172.21.15.36:0/124319105 shutdown_connections 2022-06-17T12:54:58.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba18d59700 1 --2- 172.21.15.36:0/124319105 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba14067120 0x7fba1405ff00 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:58.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba18d59700 1 --2- 172.21.15.36:0/124319105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba1405f770 0x7fba1413eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:58.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba18d59700 1 -- 172.21.15.36:0/124319105 >> 172.21.15.36:0/124319105 conn(0x7fba1405c550 msgr2=0x7fba140b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:58.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba18d59700 1 -- 172.21.15.36:0/124319105 shutdown_connections 2022-06-17T12:54:58.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.191+0000 7fba18d59700 1 -- 172.21.15.36:0/124319105 wait complete. 2022-06-17T12:54:58.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba18d59700 1 Processor -- start 2022-06-17T12:54:58.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba18d59700 1 -- start start 2022-06-17T12:54:58.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba18d59700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba1405f770 0x7fba14143b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:58.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba18d59700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba14067120 0x7fba1413c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:58.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba18d59700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba140b1770 0x7fba1413ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:58.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba18d59700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1413abc0 con 0x7fba1405f770 2022-06-17T12:54:58.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba18d59700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1413cf70 con 0x7fba140b1770 2022-06-17T12:54:58.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba18d59700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1413adf0 con 0x7fba14067120 2022-06-17T12:54:58.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba1259c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba1405f770 0x7fba14143b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:58.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba1259c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba1405f770 0x7fba14143b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37646/0 (socket says 172.21.15.36:37646) 2022-06-17T12:54:58.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba1259c700 1 -- 172.21.15.36:0/1835864958 learned_addr learned my addr 172.21.15.36:0/1835864958 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:58.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba12d9d700 1 --2- 172.21.15.36:0/1835864958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba140b1770 0x7fba1413ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:58.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba11d9b700 1 --2- 172.21.15.36:0/1835864958 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba14067120 0x7fba1413c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:58.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba1259c700 1 -- 172.21.15.36:0/1835864958 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba14067120 msgr2=0x7fba1413c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:58.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba1259c700 1 --2- 172.21.15.36:0/1835864958 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba14067120 0x7fba1413c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:58.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba1259c700 1 -- 172.21.15.36:0/1835864958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba140b1770 msgr2=0x7fba1413ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:58.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba1259c700 1 --2- 172.21.15.36:0/1835864958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba140b1770 0x7fba1413ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:58.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba1259c700 1 -- 172.21.15.36:0/1835864958 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0800b180 con 0x7fba1405f770 2022-06-17T12:54:58.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba12d9d700 1 --2- 172.21.15.36:0/1835864958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba140b1770 0x7fba1413ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:54:58.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba1259c700 1 --2- 172.21.15.36:0/1835864958 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba1405f770 0x7fba14143b30 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7fba0400c000 tx=0x7fba04006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:58.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba037fe700 1 -- 172.21.15.36:0/1835864958 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba04010780 con 0x7fba1405f770 2022-06-17T12:54:58.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba037fe700 1 -- 172.21.15.36:0/1835864958 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba040108e0 con 0x7fba1405f770 2022-06-17T12:54:58.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba18d59700 1 -- 172.21.15.36:0/1835864958 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1413b090 con 0x7fba1405f770 2022-06-17T12:54:58.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba037fe700 1 -- 172.21.15.36:0/1835864958 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba04010c40 con 0x7fba1405f770 2022-06-17T12:54:58.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.195+0000 7fba18d59700 1 -- 172.21.15.36:0/1835864958 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1413e560 con 0x7fba1405f770 2022-06-17T12:54:58.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.199+0000 7fba18d59700 1 -- 172.21.15.36:0/1835864958 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba140b1c00 con 0x7fba1405f770 2022-06-17T12:54:58.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.199+0000 7fba037fe700 1 -- 172.21.15.36:0/1835864958 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fba0401c460 con 0x7fba1405f770 2022-06-17T12:54:58.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.199+0000 7fba037fe700 1 --2- 172.21.15.36:0/1835864958 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb9f0053ad0 0x7fb9f0055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:58.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.199+0000 7fba037fe700 1 -- 172.21.15.36:0/1835864958 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fba0406c1e0 con 0x7fba1405f770 2022-06-17T12:54:58.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.199+0000 7fba11d9b700 1 --2- 172.21.15.36:0/1835864958 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb9f0053ad0 0x7fb9f0055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:58.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.199+0000 7fba11d9b700 1 --2- 172.21.15.36:0/1835864958 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb9f0053ad0 0x7fb9f0055f90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fba08018000 tx=0x7fba08014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:58.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.207+0000 7fba037fe700 1 -- 172.21.15.36:0/1835864958 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fba040432c0 con 0x7fba1405f770 2022-06-17T12:54:58.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.335+0000 7fba18d59700 1 -- 172.21.15.36:0/1835864958 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fba14057410 con 0x7fba1405f770 2022-06-17T12:54:58.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.651+0000 7fba037fe700 1 -- 172.21.15.36:0/1835864958 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v101) v1 ==== 133+0+0 (secure 0 0 0) 0x7fba040468e0 con 0x7fba1405f770 2022-06-17T12:54:58.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.655+0000 7fba18d59700 1 -- 172.21.15.36:0/1835864958 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb9f0053ad0 msgr2=0x7fb9f0055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:58.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.655+0000 7fba18d59700 1 --2- 172.21.15.36:0/1835864958 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb9f0053ad0 0x7fb9f0055f90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fba08018000 tx=0x7fba08014000).stop 2022-06-17T12:54:58.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.655+0000 7fba18d59700 1 -- 172.21.15.36:0/1835864958 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba1405f770 msgr2=0x7fba14143b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:58.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.655+0000 7fba18d59700 1 --2- 172.21.15.36:0/1835864958 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba1405f770 0x7fba14143b30 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7fba0400c000 tx=0x7fba04006df0).stop 2022-06-17T12:54:58.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.655+0000 7fba18d59700 1 -- 172.21.15.36:0/1835864958 shutdown_connections 2022-06-17T12:54:58.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.655+0000 7fba18d59700 1 --2- 172.21.15.36:0/1835864958 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb9f0053ad0 0x7fb9f0055f90 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:58.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.655+0000 7fba18d59700 1 --2- 172.21.15.36:0/1835864958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba140b1770 0x7fba1413ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:58.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.655+0000 7fba18d59700 1 --2- 172.21.15.36:0/1835864958 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba14067120 0x7fba1413c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:58.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.655+0000 7fba18d59700 1 --2- 172.21.15.36:0/1835864958 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba1405f770 0x7fba14143b30 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:58.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.655+0000 7fba18d59700 1 -- 172.21.15.36:0/1835864958 >> 172.21.15.36:0/1835864958 conn(0x7fba1405c550 msgr2=0x7fba14142db0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:58.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.655+0000 7fba18d59700 1 -- 172.21.15.36:0/1835864958 shutdown_connections 2022-06-17T12:54:58.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.655+0000 7fba18d59700 1 -- 172.21.15.36:0/1835864958 wait complete. 2022-06-17T12:54:58.653 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' created 2022-06-17T12:54:58.659 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-06-17T12:54:58.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.787+0000 7fcd0759e700 1 Processor -- start 2022-06-17T12:54:58.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.787+0000 7fcd0759e700 1 -- start start 2022-06-17T12:54:58.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd0759e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd08060ca0 0x7fcd081442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:58.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd0759e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd0814b870 con 0x7fcd080b27b0 2022-06-17T12:54:58.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd0759e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd080ac500 con 0x7fcd0814b2b0 2022-06-17T12:54:58.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd0759e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd080ace90 con 0x7fcd08060ca0 2022-06-17T12:54:58.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd0659c700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fcd080b27b0 0x7fcd080ac780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55184/0 (socket says 172.21.15.36:55184) 2022-06-17T12:54:58.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd0659c700 1 -- 172.21.15.36:0/349562348 learned_addr learned my addr 172.21.15.36:0/349562348 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:58.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd06d9d700 1 --2- 172.21.15.36:0/349562348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd08060ca0 0x7fcd081442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:58.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd0559a700 1 -- 172.21.15.36:0/349562348 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 989793831 0 0) 0x7fcd0814b870 con 0x7fcd080b27b0 2022-06-17T12:54:58.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd0559a700 1 -- 172.21.15.36:0/349562348 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccec003260 con 0x7fcd080b27b0 2022-06-17T12:54:58.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd0559a700 1 -- 172.21.15.36:0/349562348 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2363620050 0 0) 0x7fcd080ac500 con 0x7fcd0814b2b0 2022-06-17T12:54:58.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd0559a700 1 -- 172.21.15.36:0/349562348 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd0814b870 con 0x7fcd0814b2b0 2022-06-17T12:54:58.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd06d9d700 1 -- 172.21.15.36:0/349562348 >> v1:172.21.15.70:6789/0 conn(0x7fcd0814b2b0 legacy=0x7fcd08147bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:58.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd06d9d700 1 -- 172.21.15.36:0/349562348 >> v1:172.21.15.36:6789/0 conn(0x7fcd080b27b0 legacy=0x7fcd080ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:58.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd06d9d700 1 -- 172.21.15.36:0/349562348 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd081482e0 con 0x7fcd08060ca0 2022-06-17T12:54:58.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd06d9d700 1 --2- 172.21.15.36:0/349562348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd08060ca0 0x7fcd081442d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fcd000032f0 tx=0x7fcd000091d0).ready entity=mon.2 client_cookie=3373339252b197ec server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:58.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd0559a700 1 -- 172.21.15.36:0/349562348 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd0000b640 con 0x7fcd08060ca0 2022-06-17T12:54:58.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd0559a700 1 -- 172.21.15.36:0/349562348 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0000b7a0 con 0x7fcd08060ca0 2022-06-17T12:54:58.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd0559a700 1 -- 172.21.15.36:0/349562348 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd0000b910 con 0x7fcd08060ca0 2022-06-17T12:54:58.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd0759e700 1 -- 172.21.15.36:0/349562348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd08060ca0 msgr2=0x7fcd081442d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:58.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd0759e700 1 --2- 172.21.15.36:0/349562348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd08060ca0 0x7fcd081442d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fcd000032f0 tx=0x7fcd000091d0).stop 2022-06-17T12:54:58.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd0759e700 1 -- 172.21.15.36:0/349562348 shutdown_connections 2022-06-17T12:54:58.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd0759e700 1 --2- 172.21.15.36:0/349562348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd08060ca0 0x7fcd081442d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:58.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd0759e700 1 -- 172.21.15.36:0/349562348 >> 172.21.15.36:0/349562348 conn(0x7fcd080b5d10 msgr2=0x7fcd080b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:58.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd0759e700 1 -- 172.21.15.36:0/349562348 shutdown_connections 2022-06-17T12:54:58.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.791+0000 7fcd0759e700 1 -- 172.21.15.36:0/349562348 wait complete. 2022-06-17T12:54:58.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.795+0000 7fcd0759e700 1 Processor -- start 2022-06-17T12:54:58.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.795+0000 7fcd0759e700 1 -- start start 2022-06-17T12:54:58.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.795+0000 7fcd0759e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd08060ca0 0x7fcd080aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:58.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.795+0000 7fcd0759e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd080b27b0 0x7fcd081564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:58.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.795+0000 7fcd0759e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd0814b2b0 0x7fcd08065a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:58.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.795+0000 7fcd0759e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd08065340 con 0x7fcd08060ca0 2022-06-17T12:54:58.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.795+0000 7fcd0759e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd080655e0 con 0x7fcd080b27b0 2022-06-17T12:54:58.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.795+0000 7fcd0759e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd08063f60 con 0x7fcd0814b2b0 2022-06-17T12:54:58.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.795+0000 7fcd0659c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd08060ca0 0x7fcd080aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:58.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.795+0000 7fcd0659c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd08060ca0 0x7fcd080aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37660/0 (socket says 172.21.15.36:37660) 2022-06-17T12:54:58.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.795+0000 7fcd0659c700 1 -- 172.21.15.36:0/1642659942 learned_addr learned my addr 172.21.15.36:0/1642659942 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:58.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.795+0000 7fcd06d9d700 1 --2- 172.21.15.36:0/1642659942 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd0814b2b0 0x7fcd08065a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:58.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.795+0000 7fcd05d9b700 1 --2- 172.21.15.36:0/1642659942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd080b27b0 0x7fcd081564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:58.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.799+0000 7fcd0659c700 1 -- 172.21.15.36:0/1642659942 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd0814b2b0 msgr2=0x7fcd08065a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:58.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.799+0000 7fcd0659c700 1 --2- 172.21.15.36:0/1642659942 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd0814b2b0 0x7fcd08065a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:58.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.799+0000 7fcd0659c700 1 -- 172.21.15.36:0/1642659942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd080b27b0 msgr2=0x7fcd081564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:58.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.799+0000 7fcd0659c700 1 --2- 172.21.15.36:0/1642659942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd080b27b0 0x7fcd081564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:58.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.799+0000 7fcd0659c700 1 -- 172.21.15.36:0/1642659942 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd00009470 con 0x7fcd08060ca0 2022-06-17T12:54:58.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.799+0000 7fcd05d9b700 1 --2- 172.21.15.36:0/1642659942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd080b27b0 0x7fcd081564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:58.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.799+0000 7fcd0659c700 1 --2- 172.21.15.36:0/1642659942 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd08060ca0 0x7fcd080aede0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0x7fccfc001110 tx=0x7fccfc007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:58.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.799+0000 7fccf77fe700 1 -- 172.21.15.36:0/1642659942 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccfc00ce80 con 0x7fcd08060ca0 2022-06-17T12:54:58.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.799+0000 7fccf77fe700 1 -- 172.21.15.36:0/1642659942 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccfc0070d0 con 0x7fcd08060ca0 2022-06-17T12:54:58.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.799+0000 7fcd0759e700 1 -- 172.21.15.36:0/1642659942 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd081569c0 con 0x7fcd08060ca0 2022-06-17T12:54:58.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.799+0000 7fccf77fe700 1 -- 172.21.15.36:0/1642659942 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccfc007470 con 0x7fcd08060ca0 2022-06-17T12:54:58.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.799+0000 7fcd0759e700 1 -- 172.21.15.36:0/1642659942 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd080a80f0 con 0x7fcd08060ca0 2022-06-17T12:54:58.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.799+0000 7fcd0759e700 1 -- 172.21.15.36:0/1642659942 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd080662b0 con 0x7fcd08060ca0 2022-06-17T12:54:58.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.803+0000 7fccf77fe700 1 -- 172.21.15.36:0/1642659942 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fccfc010070 con 0x7fcd08060ca0 2022-06-17T12:54:58.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.803+0000 7fccf77fe700 1 --2- 172.21.15.36:0/1642659942 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcce4053a40 0x7fcce4055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:58.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.803+0000 7fccf77fe700 1 -- 172.21.15.36:0/1642659942 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5954+0+0 (secure 0 0 0) 0x7fccfc06b3c0 con 0x7fcd08060ca0 2022-06-17T12:54:58.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.803+0000 7fcd05d9b700 1 --2- 172.21.15.36:0/1642659942 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcce4053a40 0x7fcce4055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:58.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.803+0000 7fcd05d9b700 1 --2- 172.21.15.36:0/1642659942 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcce4053a40 0x7fcce4055f00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7fccf800d000 tx=0x7fccf8007a20).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:58.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.807+0000 7fccf77fe700 1 -- 172.21.15.36:0/1642659942 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcd080662b0 con 0x7fcd08060ca0 2022-06-17T12:54:58.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:58.939+0000 7fcd0759e700 1 -- 172.21.15.36:0/1642659942 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fcd080662b0 con 0x7fcd08060ca0 2022-06-17T12:54:59.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.663+0000 7fccf77fe700 1 -- 172.21.15.36:0/1642659942 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v102) v1 ==== 125+0+0 (secure 0 0 0) 0x7fcd080662b0 con 0x7fcd08060ca0 2022-06-17T12:54:59.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.667+0000 7fcd0759e700 1 -- 172.21.15.36:0/1642659942 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcce4053a40 msgr2=0x7fcce4055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:59.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.667+0000 7fcd0759e700 1 --2- 172.21.15.36:0/1642659942 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcce4053a40 0x7fcce4055f00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7fccf800d000 tx=0x7fccf8007a20).stop 2022-06-17T12:54:59.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.667+0000 7fcd0759e700 1 -- 172.21.15.36:0/1642659942 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd08060ca0 msgr2=0x7fcd080aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:59.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.667+0000 7fcd0759e700 1 --2- 172.21.15.36:0/1642659942 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd08060ca0 0x7fcd080aede0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0x7fccfc001110 tx=0x7fccfc007aa0).stop 2022-06-17T12:54:59.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.667+0000 7fcd0759e700 1 -- 172.21.15.36:0/1642659942 shutdown_connections 2022-06-17T12:54:59.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.667+0000 7fcd0759e700 1 --2- 172.21.15.36:0/1642659942 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcce4053a40 0x7fcce4055f00 secure :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0x7fccf800d000 tx=0x7fccf8007a20).stop 2022-06-17T12:54:59.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.667+0000 7fcd0759e700 1 --2- 172.21.15.36:0/1642659942 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd0814b2b0 0x7fcd08065a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:59.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.667+0000 7fcd0759e700 1 --2- 172.21.15.36:0/1642659942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd080b27b0 0x7fcd081564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:59.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.667+0000 7fcd0759e700 1 --2- 172.21.15.36:0/1642659942 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd08060ca0 0x7fcd080aede0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:59.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.667+0000 7fcd0759e700 1 -- 172.21.15.36:0/1642659942 >> 172.21.15.36:0/1642659942 conn(0x7fcd080b5d10 msgr2=0x7fcd080afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:59.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.671+0000 7fcd0759e700 1 -- 172.21.15.36:0/1642659942 shutdown_connections 2022-06-17T12:54:59.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.671+0000 7fcd0759e700 1 -- 172.21.15.36:0/1642659942 wait complete. 2022-06-17T12:54:59.662 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' created 2022-06-17T12:54:59.676 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-17T12:54:59.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.803+0000 7f8f7a1f6700 1 Processor -- start 2022-06-17T12:54:59.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.803+0000 7f8f7a1f6700 1 -- start start 2022-06-17T12:54:59.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f7a1f6700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f740b1890 0x7f8f7413c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:59.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f7a1f6700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8f740b1ff0 0x7f8f7413f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:59.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f7a1f6700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f740b0230 con 0x7f8f74060ac0 2022-06-17T12:54:59.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f7a1f6700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f74143800 con 0x7f8f740b1ff0 2022-06-17T12:54:59.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f7a1f6700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f74060ea0 con 0x7f8f740b1890 2022-06-17T12:54:59.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f799f5700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8f740b1ff0 0x7f8f7413f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:59.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f791f4700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8f74060ac0 0x7f8f7413c2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:50932/0 (socket says 172.21.15.36:50932) 2022-06-17T12:54:59.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f799f5700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8f740b1ff0 0x7f8f7413f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:58054/0 (socket says 172.21.15.36:58054) 2022-06-17T12:54:59.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f789f3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f740b1890 0x7f8f7413c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:59.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f791f4700 1 -- 172.21.15.36:0/1649787436 learned_addr learned my addr 172.21.15.36:0/1649787436 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:54:59.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f799f5700 1 -- 172.21.15.36:0/1649787436 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f740b1890 msgr2=0x7f8f7413c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:54:59.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f799f5700 1 --2- 172.21.15.36:0/1649787436 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f740b1890 0x7f8f7413c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:59.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f799f5700 1 -- 172.21.15.36:0/1649787436 >> v1:172.21.15.70:6789/0 conn(0x7f8f74060ac0 legacy=0x7f8f7413c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:59.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f799f5700 1 -- 172.21.15.36:0/1649787436 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f740afcf0 con 0x7f8f740b1ff0 2022-06-17T12:54:59.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f799f5700 1 --2- 172.21.15.36:0/1649787436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f740b1ff0 0x7f8f7413f270 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f8f6c0032f0 tx=0x7f8f6c0091d0).ready entity=mon.1 client_cookie=2ebe2340cf043ae server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:59.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f63fff700 1 -- 172.21.15.36:0/1649787436 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f6c00b6c0 con 0x7f8f740b1ff0 2022-06-17T12:54:59.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f63fff700 1 -- 172.21.15.36:0/1649787436 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6c00b820 con 0x7f8f740b1ff0 2022-06-17T12:54:59.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f63fff700 1 -- 172.21.15.36:0/1649787436 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f6c00b990 con 0x7f8f740b1ff0 2022-06-17T12:54:59.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f7a1f6700 1 -- 172.21.15.36:0/1649787436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f740b1ff0 msgr2=0x7f8f7413f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:59.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f7a1f6700 1 --2- 172.21.15.36:0/1649787436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f740b1ff0 0x7f8f7413f270 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f8f6c0032f0 tx=0x7f8f6c0091d0).stop 2022-06-17T12:54:59.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f7a1f6700 1 -- 172.21.15.36:0/1649787436 shutdown_connections 2022-06-17T12:54:59.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f7a1f6700 1 --2- 172.21.15.36:0/1649787436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f740b1ff0 0x7f8f7413f270 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:59.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f7a1f6700 1 --2- 172.21.15.36:0/1649787436 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f740b1890 0x7f8f7413c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:59.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f7a1f6700 1 -- 172.21.15.36:0/1649787436 >> 172.21.15.36:0/1649787436 conn(0x7f8f740b65f0 msgr2=0x7f8f740b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:54:59.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f7a1f6700 1 -- 172.21.15.36:0/1649787436 shutdown_connections 2022-06-17T12:54:59.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.807+0000 7f8f7a1f6700 1 -- 172.21.15.36:0/1649787436 wait complete. 2022-06-17T12:54:59.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.811+0000 7f8f7a1f6700 1 Processor -- start 2022-06-17T12:54:59.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.811+0000 7f8f7a1f6700 1 -- start start 2022-06-17T12:54:59.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.811+0000 7f8f7a1f6700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f74060ac0 0x7f8f740654d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:59.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.811+0000 7f8f7a1f6700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f740b1890 0x7f8f741376f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:59.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.811+0000 7f8f7a1f6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f740b1ff0 0x7f8f741385e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:59.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.811+0000 7f8f7a1f6700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f74064af0 con 0x7f8f74060ac0 2022-06-17T12:54:59.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.811+0000 7f8f7a1f6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f74064d90 con 0x7f8f740b1ff0 2022-06-17T12:54:59.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.811+0000 7f8f7a1f6700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f74065030 con 0x7f8f740b1890 2022-06-17T12:54:59.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.811+0000 7f8f791f4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f74060ac0 0x7f8f740654d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:59.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.811+0000 7f8f791f4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f74060ac0 0x7f8f740654d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37676/0 (socket says 172.21.15.36:37676) 2022-06-17T12:54:59.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.811+0000 7f8f791f4700 1 -- 172.21.15.36:0/3480828576 learned_addr learned my addr 172.21.15.36:0/3480828576 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:54:59.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.811+0000 7f8f789f3700 1 --2- 172.21.15.36:0/3480828576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f740b1890 0x7f8f741376f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:59.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.811+0000 7f8f799f5700 1 --2- 172.21.15.36:0/3480828576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f740b1ff0 0x7f8f741385e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:59.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.815+0000 7f8f791f4700 1 -- 172.21.15.36:0/3480828576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f740b1890 msgr2=0x7f8f741376f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:59.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.815+0000 7f8f791f4700 1 --2- 172.21.15.36:0/3480828576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f740b1890 0x7f8f741376f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:59.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.815+0000 7f8f791f4700 1 -- 172.21.15.36:0/3480828576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f740b1ff0 msgr2=0x7f8f741385e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:54:59.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.815+0000 7f8f791f4700 1 --2- 172.21.15.36:0/3480828576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f740b1ff0 0x7f8f741385e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:54:59.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.815+0000 7f8f791f4700 1 -- 172.21.15.36:0/3480828576 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6c009470 con 0x7f8f74060ac0 2022-06-17T12:54:59.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.815+0000 7f8f799f5700 1 --2- 172.21.15.36:0/3480828576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f740b1ff0 0x7f8f741385e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:54:59.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.815+0000 7f8f791f4700 1 --2- 172.21.15.36:0/3480828576 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f74060ac0 0x7f8f740654d0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7f8f740626d0 tx=0x7f8f70007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:59.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.815+0000 7f8f627fc700 1 -- 172.21.15.36:0/3480828576 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f700070d0 con 0x7f8f74060ac0 2022-06-17T12:54:59.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.815+0000 7f8f7a1f6700 1 -- 172.21.15.36:0/3480828576 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f74136680 con 0x7f8f74060ac0 2022-06-17T12:54:59.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.815+0000 7f8f7a1f6700 1 -- 172.21.15.36:0/3480828576 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f74138ad0 con 0x7f8f74060ac0 2022-06-17T12:54:59.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.815+0000 7f8f627fc700 1 -- 172.21.15.36:0/3480828576 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f70007230 con 0x7f8f74060ac0 2022-06-17T12:54:59.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.815+0000 7f8f627fc700 1 -- 172.21.15.36:0/3480828576 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f700055d0 con 0x7f8f74060ac0 2022-06-17T12:54:59.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.815+0000 7f8f627fc700 1 -- 172.21.15.36:0/3480828576 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8f70010070 con 0x7f8f74060ac0 2022-06-17T12:54:59.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.815+0000 7f8f7a1f6700 1 -- 172.21.15.36:0/3480828576 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f50003410 con 0x7f8f74060ac0 2022-06-17T12:54:59.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.819+0000 7f8f627fc700 1 --2- 172.21.15.36:0/3480828576 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8f54053a40 0x7f8f54055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:54:59.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.819+0000 7f8f627fc700 1 -- 172.21.15.36:0/3480828576 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f8f7006b530 con 0x7f8f74060ac0 2022-06-17T12:54:59.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.819+0000 7f8f789f3700 1 --2- 172.21.15.36:0/3480828576 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8f54053a40 0x7f8f54055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:54:59.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.819+0000 7f8f789f3700 1 --2- 172.21.15.36:0/3480828576 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8f54053a40 0x7f8f54055f00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f8f64008000 tx=0x7f8f64008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:54:59.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.827+0000 7f8f627fc700 1 -- 172.21.15.36:0/3480828576 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8f70042220 con 0x7f8f74060ac0 2022-06-17T12:54:59.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:54:59.987+0000 7f8f7a1f6700 1 -- 172.21.15.36:0/3480828576 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f8f50003c40 con 0x7f8f74060ac0 2022-06-17T12:55:00.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.011+0000 7f8f627fc700 1 -- 172.21.15.36:0/3480828576 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 19 and data pool 20 v52) v1 ==== 189+0+0 (secure 0 0 0) 0x7f8f70018070 con 0x7f8f74060ac0 2022-06-17T12:55:00.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.015+0000 7f8f7a1f6700 1 -- 172.21.15.36:0/3480828576 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8f54053a40 msgr2=0x7f8f54055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:00.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.015+0000 7f8f7a1f6700 1 --2- 172.21.15.36:0/3480828576 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8f54053a40 0x7f8f54055f00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f8f64008000 tx=0x7f8f64008040).stop 2022-06-17T12:55:00.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.015+0000 7f8f7a1f6700 1 -- 172.21.15.36:0/3480828576 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f74060ac0 msgr2=0x7f8f740654d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:00.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.015+0000 7f8f7a1f6700 1 --2- 172.21.15.36:0/3480828576 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f74060ac0 0x7f8f740654d0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7f8f740626d0 tx=0x7f8f70007c30).stop 2022-06-17T12:55:00.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.015+0000 7f8f7a1f6700 1 -- 172.21.15.36:0/3480828576 shutdown_connections 2022-06-17T12:55:00.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.015+0000 7f8f7a1f6700 1 --2- 172.21.15.36:0/3480828576 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8f54053a40 0x7f8f54055f00 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.015+0000 7f8f7a1f6700 1 --2- 172.21.15.36:0/3480828576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f740b1ff0 0x7f8f741385e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.015+0000 7f8f7a1f6700 1 --2- 172.21.15.36:0/3480828576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f740b1890 0x7f8f741376f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.015+0000 7f8f7a1f6700 1 --2- 172.21.15.36:0/3480828576 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f74060ac0 0x7f8f740654d0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.015+0000 7f8f7a1f6700 1 -- 172.21.15.36:0/3480828576 >> 172.21.15.36:0/3480828576 conn(0x7f8f740b65f0 msgr2=0x7f8f7413a810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:00.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.015+0000 7f8f7a1f6700 1 -- 172.21.15.36:0/3480828576 shutdown_connections 2022-06-17T12:55:00.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.015+0000 7f8f7a1f6700 1 -- 172.21.15.36:0/3480828576 wait complete. 2022-06-17T12:55:00.016 INFO:teuthology.orchestra.run.smithi036.stderr:new fs with metadata pool 19 and data pool 20 2022-06-17T12:55:00.018 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:55:00.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.135+0000 7f542922b700 1 Processor -- start 2022-06-17T12:55:00.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.135+0000 7f542922b700 1 -- start start 2022-06-17T12:55:00.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.139+0000 7f542922b700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f54240b1280 0x7f542413fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:00.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.139+0000 7f542922b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5424140410 0x7f54241436d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:00.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.139+0000 7f542922b700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54240b1660 con 0x7f54240abe30 2022-06-17T12:55:00.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.139+0000 7f542922b700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54240aef10 con 0x7f54240b1280 2022-06-17T12:55:00.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.139+0000 7f542922b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5424140080 con 0x7f5424140410 2022-06-17T12:55:00.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.139+0000 7f5428a2a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5424140410 0x7f54241436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:00.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.139+0000 7f5423fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f54240b1280 0x7f542413fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:00.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.139+0000 7f541bfff700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f54240abe30 0x7f5424062ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:50958/0 (socket says 172.21.15.36:50958) 2022-06-17T12:55:00.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.139+0000 7f541bfff700 1 -- 172.21.15.36:0/1134991117 learned_addr learned my addr 172.21.15.36:0/1134991117 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:00.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.139+0000 7f54237fe700 1 -- 172.21.15.36:0/1134991117 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1982312856 0 0) 0x7f54240b1660 con 0x7f54240abe30 2022-06-17T12:55:00.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.139+0000 7f54237fe700 1 -- 172.21.15.36:0/1134991117 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5404003260 con 0x7f54240abe30 2022-06-17T12:55:00.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.139+0000 7f5428a2a700 1 -- 172.21.15.36:0/1134991117 >> v2:172.21.15.70:3300/0 conn(0x7f54240b1280 msgr2=0x7f542413fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:00.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.139+0000 7f5428a2a700 1 --2- 172.21.15.36:0/1134991117 >> v2:172.21.15.70:3300/0 conn(0x7f54240b1280 0x7f542413fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.139+0000 7f5428a2a700 1 -- 172.21.15.36:0/1134991117 >> v1:172.21.15.70:6789/0 conn(0x7f54240abe30 legacy=0x7f5424062ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:00.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.139+0000 7f5428a2a700 1 -- 172.21.15.36:0/1134991117 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54240ac3b0 con 0x7f5424140410 2022-06-17T12:55:00.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.139+0000 7f5428a2a700 1 --2- 172.21.15.36:0/1134991117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5424140410 0x7f54241436d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f541c0032f0 tx=0x7f541c0091d0).ready entity=mon.2 client_cookie=90ab10625ad783e9 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:00.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.139+0000 7f54237fe700 1 -- 172.21.15.36:0/1134991117 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f541c00b600 con 0x7f5424140410 2022-06-17T12:55:00.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.139+0000 7f54237fe700 1 -- 172.21.15.36:0/1134991117 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541c00b760 con 0x7f5424140410 2022-06-17T12:55:00.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.139+0000 7f54237fe700 1 -- 172.21.15.36:0/1134991117 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f541c00b8d0 con 0x7f5424140410 2022-06-17T12:55:00.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.139+0000 7f542922b700 1 -- 172.21.15.36:0/1134991117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5424140410 msgr2=0x7f54241436d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:00.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.143+0000 7f542922b700 1 --2- 172.21.15.36:0/1134991117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5424140410 0x7f54241436d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f541c0032f0 tx=0x7f541c0091d0).stop 2022-06-17T12:55:00.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.143+0000 7f542922b700 1 -- 172.21.15.36:0/1134991117 shutdown_connections 2022-06-17T12:55:00.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.143+0000 7f542922b700 1 --2- 172.21.15.36:0/1134991117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5424140410 0x7f54241436d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.143+0000 7f542922b700 1 --2- 172.21.15.36:0/1134991117 >> v2:172.21.15.70:3300/0 conn(0x7f54240b1280 0x7f542413fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.143+0000 7f542922b700 1 -- 172.21.15.36:0/1134991117 >> 172.21.15.36:0/1134991117 conn(0x7f542405f770 msgr2=0x7f542405fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:00.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.143+0000 7f542922b700 1 -- 172.21.15.36:0/1134991117 shutdown_connections 2022-06-17T12:55:00.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.143+0000 7f542922b700 1 -- 172.21.15.36:0/1134991117 wait complete. 2022-06-17T12:55:00.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.143+0000 7f542922b700 1 Processor -- start 2022-06-17T12:55:00.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.143+0000 7f542922b700 1 -- start start 2022-06-17T12:55:00.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.143+0000 7f542922b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54240abe30 0x7f542413c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:00.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.143+0000 7f542922b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f54240b1280 0x7f5424065a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:00.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.147+0000 7f542922b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5424140410 0x7f5424066cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:00.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.147+0000 7f542922b700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54240643d0 con 0x7f54240b1280 2022-06-17T12:55:00.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.147+0000 7f542922b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54240b50b0 con 0x7f5424140410 2022-06-17T12:55:00.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.147+0000 7f542922b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5424064130 con 0x7f54240abe30 2022-06-17T12:55:00.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.147+0000 7f541bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f54240b1280 0x7f5424065a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:00.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.147+0000 7f5423fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54240abe30 0x7f542413c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:00.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.147+0000 7f541bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f54240b1280 0x7f5424065a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37702/0 (socket says 172.21.15.36:37702) 2022-06-17T12:55:00.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.147+0000 7f541bfff700 1 -- 172.21.15.36:0/821347415 learned_addr learned my addr 172.21.15.36:0/821347415 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:00.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.147+0000 7f5428a2a700 1 --2- 172.21.15.36:0/821347415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5424140410 0x7f5424066cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:00.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.147+0000 7f541bfff700 1 -- 172.21.15.36:0/821347415 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54240abe30 msgr2=0x7f542413c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:00.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.147+0000 7f541bfff700 1 --2- 172.21.15.36:0/821347415 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54240abe30 0x7f542413c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.147+0000 7f541bfff700 1 -- 172.21.15.36:0/821347415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5424140410 msgr2=0x7f5424066cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:00.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.147+0000 7f541bfff700 1 --2- 172.21.15.36:0/821347415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5424140410 0x7f5424066cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.147+0000 7f541bfff700 1 -- 172.21.15.36:0/821347415 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c009470 con 0x7f54240b1280 2022-06-17T12:55:00.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.147+0000 7f541bfff700 1 --2- 172.21.15.36:0/821347415 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f54240b1280 0x7f5424065a10 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7f540c001110 tx=0x7f540c007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:00.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.147+0000 7f5421ffb700 1 -- 172.21.15.36:0/821347415 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f540c0070d0 con 0x7f54240b1280 2022-06-17T12:55:00.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.147+0000 7f5421ffb700 1 -- 172.21.15.36:0/821347415 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540c007230 con 0x7f54240b1280 2022-06-17T12:55:00.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.147+0000 7f542922b700 1 -- 172.21.15.36:0/821347415 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54240b4de0 con 0x7f54240b1280 2022-06-17T12:55:00.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.147+0000 7f5421ffb700 1 -- 172.21.15.36:0/821347415 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f540c005610 con 0x7f54240b1280 2022-06-17T12:55:00.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.147+0000 7f542922b700 1 -- 172.21.15.36:0/821347415 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54240b3df0 con 0x7f54240b1280 2022-06-17T12:55:00.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.147+0000 7f542922b700 1 -- 172.21.15.36:0/821347415 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5424066a00 con 0x7f54240b1280 2022-06-17T12:55:00.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.151+0000 7f5421ffb700 1 -- 172.21.15.36:0/821347415 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f540c010070 con 0x7f54240b1280 2022-06-17T12:55:00.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.151+0000 7f5421ffb700 1 --2- 172.21.15.36:0/821347415 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5408053920 0x7f5408055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:00.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.151+0000 7f5421ffb700 1 -- 172.21.15.36:0/821347415 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f540c06b630 con 0x7f54240b1280 2022-06-17T12:55:00.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.151+0000 7f5423fff700 1 --2- 172.21.15.36:0/821347415 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5408053920 0x7f5408055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:00.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.151+0000 7f5423fff700 1 --2- 172.21.15.36:0/821347415 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5408053920 0x7f5408055de0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f541400b000 tx=0x7f5414006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:00.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.155+0000 7f5421ffb700 1 -- 172.21.15.36:0/821347415 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5424066a00 con 0x7f54240b1280 2022-06-17T12:55:00.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.283+0000 7f542922b700 1 -- 172.21.15.36:0/821347415 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5424066a00 con 0x7f54240b1280 2022-06-17T12:55:00.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.283+0000 7f5421ffb700 1 -- 172.21.15.36:0/821347415 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f5424066a00 con 0x7f54240b1280 2022-06-17T12:55:00.272 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:00.272 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":103,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:54:59.995055+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:54:58.393546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-06-17T12:54:58.997835+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:55:00.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.287+0000 7f542922b700 1 -- 172.21.15.36:0/821347415 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5408053920 msgr2=0x7f5408055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:00.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.287+0000 7f542922b700 1 --2- 172.21.15.36:0/821347415 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5408053920 0x7f5408055de0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f541400b000 tx=0x7f5414006df0).stop 2022-06-17T12:55:00.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.287+0000 7f542922b700 1 -- 172.21.15.36:0/821347415 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f54240b1280 msgr2=0x7f5424065a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:00.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.287+0000 7f542922b700 1 --2- 172.21.15.36:0/821347415 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f54240b1280 0x7f5424065a10 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7f540c001110 tx=0x7f540c007ad0).stop 2022-06-17T12:55:00.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.287+0000 7f542922b700 1 -- 172.21.15.36:0/821347415 shutdown_connections 2022-06-17T12:55:00.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.287+0000 7f542922b700 1 --2- 172.21.15.36:0/821347415 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5408053920 0x7f5408055de0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.287+0000 7f542922b700 1 --2- 172.21.15.36:0/821347415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5424140410 0x7f5424066cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.287+0000 7f542922b700 1 --2- 172.21.15.36:0/821347415 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f54240b1280 0x7f5424065a10 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.287+0000 7f542922b700 1 --2- 172.21.15.36:0/821347415 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54240abe30 0x7f542413c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.287+0000 7f542922b700 1 -- 172.21.15.36:0/821347415 >> 172.21.15.36:0/821347415 conn(0x7f542405f770 msgr2=0x7f54240acde0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:00.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.287+0000 7f542922b700 1 -- 172.21.15.36:0/821347415 shutdown_connections 2022-06-17T12:55:00.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.287+0000 7f542922b700 1 -- 172.21.15.36:0/821347415 wait complete. 2022-06-17T12:55:00.286 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:55:00.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.407+0000 7fcb80b17700 1 Processor -- start 2022-06-17T12:55:00.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.407+0000 7fcb80b17700 1 -- start start 2022-06-17T12:55:00.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.407+0000 7fcb80b17700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcb7c13cb70 0x7fcb7c143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:00.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.407+0000 7fcb80b17700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb7c13cf50 0x7fcb7c1442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:00.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.407+0000 7fcb80b17700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb7c0af4b0 con 0x7fcb7c0aef60 2022-06-17T12:55:00.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.407+0000 7fcb80b17700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb7c0ba500 con 0x7fcb7c13cb70 2022-06-17T12:55:00.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.407+0000 7fcb80b17700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c142f50 con 0x7fcb7c13cf50 2022-06-17T12:55:00.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.407+0000 7fcb7bfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb7c13cf50 0x7fcb7c1442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:00.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.407+0000 7fcb7b7fe700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcb7c13cb70 0x7fcb7c143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:00.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.407+0000 7fcb7affd700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcb7c0aef60 0x7fcb7c0b9df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:50974/0 (socket says 172.21.15.36:50974) 2022-06-17T12:55:00.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.407+0000 7fcb7affd700 1 -- 172.21.15.36:0/3493091010 learned_addr learned my addr 172.21.15.36:0/3493091010 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:00.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.407+0000 7fcb7bfff700 1 -- 172.21.15.36:0/3493091010 >> v2:172.21.15.70:3300/0 conn(0x7fcb7c13cb70 msgr2=0x7fcb7c143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:00.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.407+0000 7fcb7bfff700 1 --2- 172.21.15.36:0/3493091010 >> v2:172.21.15.70:3300/0 conn(0x7fcb7c13cb70 0x7fcb7c143690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.407+0000 7fcb7bfff700 1 -- 172.21.15.36:0/3493091010 >> v1:172.21.15.70:6789/0 conn(0x7fcb7c0aef60 legacy=0x7fcb7c0b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:00.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.407+0000 7fcb7bfff700 1 -- 172.21.15.36:0/3493091010 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7c1431f0 con 0x7fcb7c13cf50 2022-06-17T12:55:00.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.407+0000 7fcb7bfff700 1 --2- 172.21.15.36:0/3493091010 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb7c13cf50 0x7fcb7c1442a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fcb740032f0 tx=0x7fcb740091d0).ready entity=mon.2 client_cookie=b071e1454737c64 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:00.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.407+0000 7fcb7a7fc700 1 -- 172.21.15.36:0/3493091010 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb7400b6c0 con 0x7fcb7c13cf50 2022-06-17T12:55:00.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.407+0000 7fcb7a7fc700 1 -- 172.21.15.36:0/3493091010 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7400b820 con 0x7fcb7c13cf50 2022-06-17T12:55:00.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.407+0000 7fcb7a7fc700 1 -- 172.21.15.36:0/3493091010 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb7400b990 con 0x7fcb7c13cf50 2022-06-17T12:55:00.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb80b17700 1 -- 172.21.15.36:0/3493091010 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb7c13cf50 msgr2=0x7fcb7c1442a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:00.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb80b17700 1 --2- 172.21.15.36:0/3493091010 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb7c13cf50 0x7fcb7c1442a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fcb740032f0 tx=0x7fcb740091d0).stop 2022-06-17T12:55:00.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb80b17700 1 -- 172.21.15.36:0/3493091010 shutdown_connections 2022-06-17T12:55:00.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb80b17700 1 --2- 172.21.15.36:0/3493091010 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb7c13cf50 0x7fcb7c1442a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb80b17700 1 --2- 172.21.15.36:0/3493091010 >> v2:172.21.15.70:3300/0 conn(0x7fcb7c13cb70 0x7fcb7c143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb80b17700 1 -- 172.21.15.36:0/3493091010 >> 172.21.15.36:0/3493091010 conn(0x7fcb7c05c550 msgr2=0x7fcb7c0b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:00.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb80b17700 1 -- 172.21.15.36:0/3493091010 shutdown_connections 2022-06-17T12:55:00.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb80b17700 1 -- 172.21.15.36:0/3493091010 wait complete. 2022-06-17T12:55:00.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb80b17700 1 Processor -- start 2022-06-17T12:55:00.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb80b17700 1 -- start start 2022-06-17T12:55:00.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb80b17700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb7c0aef60 0x7fcb7c1521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:00.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb80b17700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb7c13cb70 0x7fcb7c0b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:00.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb80b17700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb7c13cf50 0x7fcb7c1529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:00.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb80b17700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c13fcf0 con 0x7fcb7c0aef60 2022-06-17T12:55:00.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb80b17700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c13ff90 con 0x7fcb7c13cf50 2022-06-17T12:55:00.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb80b17700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c0b64b0 con 0x7fcb7c13cb70 2022-06-17T12:55:00.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb7b7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb7c0aef60 0x7fcb7c1521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:00.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb7b7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb7c0aef60 0x7fcb7c1521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37714/0 (socket says 172.21.15.36:37714) 2022-06-17T12:55:00.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb7b7fe700 1 -- 172.21.15.36:0/1671572885 learned_addr learned my addr 172.21.15.36:0/1671572885 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:00.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb7affd700 1 --2- 172.21.15.36:0/1671572885 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb7c13cb70 0x7fcb7c0b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:00.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb7bfff700 1 --2- 172.21.15.36:0/1671572885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb7c13cf50 0x7fcb7c1529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:00.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb7b7fe700 1 -- 172.21.15.36:0/1671572885 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb7c13cb70 msgr2=0x7fcb7c0b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:00.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb7b7fe700 1 --2- 172.21.15.36:0/1671572885 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb7c13cb70 0x7fcb7c0b5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb7b7fe700 1 -- 172.21.15.36:0/1671572885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb7c13cf50 msgr2=0x7fcb7c1529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:00.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb7b7fe700 1 --2- 172.21.15.36:0/1671572885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb7c13cf50 0x7fcb7c1529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb7b7fe700 1 -- 172.21.15.36:0/1671572885 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb74009470 con 0x7fcb7c0aef60 2022-06-17T12:55:00.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb7b7fe700 1 --2- 172.21.15.36:0/1671572885 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb7c0aef60 0x7fcb7c1521b0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0x7fcb6c00c000 tx=0x7fcb6c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:00.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb78ff9700 1 -- 172.21.15.36:0/1671572885 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb6c010780 con 0x7fcb7c0aef60 2022-06-17T12:55:00.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb78ff9700 1 -- 172.21.15.36:0/1671572885 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6c0108e0 con 0x7fcb7c0aef60 2022-06-17T12:55:00.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb78ff9700 1 -- 172.21.15.36:0/1671572885 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb6c010c40 con 0x7fcb7c0aef60 2022-06-17T12:55:00.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb80b17700 1 -- 172.21.15.36:0/1671572885 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb7c13f970 con 0x7fcb7c0aef60 2022-06-17T12:55:00.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.411+0000 7fcb80b17700 1 -- 172.21.15.36:0/1671572885 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb7c0b01f0 con 0x7fcb7c0aef60 2022-06-17T12:55:00.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.415+0000 7fcb80b17700 1 -- 172.21.15.36:0/1671572885 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb7c057410 con 0x7fcb7c0aef60 2022-06-17T12:55:00.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.415+0000 7fcb78ff9700 1 -- 172.21.15.36:0/1671572885 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fcb6c01c460 con 0x7fcb7c0aef60 2022-06-17T12:55:00.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.415+0000 7fcb78ff9700 1 --2- 172.21.15.36:0/1671572885 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcb60053af0 0x7fcb60055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:00.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.415+0000 7fcb78ff9700 1 -- 172.21.15.36:0/1671572885 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcb6c06c540 con 0x7fcb7c0aef60 2022-06-17T12:55:00.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.415+0000 7fcb7affd700 1 --2- 172.21.15.36:0/1671572885 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcb60053af0 0x7fcb60055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:00.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.415+0000 7fcb7affd700 1 --2- 172.21.15.36:0/1671572885 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcb60053af0 0x7fcb60055fb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7fcb6400d000 tx=0x7fcb64009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:00.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.423+0000 7fcb78ff9700 1 -- 172.21.15.36:0/1671572885 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcb6c043150 con 0x7fcb7c0aef60 2022-06-17T12:55:00.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.551+0000 7fcb80b17700 1 -- 172.21.15.36:0/1671572885 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb7c13fb60 con 0x7fcb7c0aef60 2022-06-17T12:55:00.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.551+0000 7fcb78ff9700 1 -- 172.21.15.36:0/1671572885 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16298 (secure 0 0 0) 0x7fcb6c046770 con 0x7fcb7c0aef60 2022-06-17T12:55:00.541 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:00.542 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":103,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:54:59.995055+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:54:58.393546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-06-17T12:54:58.997835+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:55:00.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.551+0000 7fcb80b17700 1 -- 172.21.15.36:0/1671572885 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcb60053af0 msgr2=0x7fcb60055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:00.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.551+0000 7fcb80b17700 1 --2- 172.21.15.36:0/1671572885 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcb60053af0 0x7fcb60055fb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7fcb6400d000 tx=0x7fcb64009040).stop 2022-06-17T12:55:00.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.551+0000 7fcb80b17700 1 -- 172.21.15.36:0/1671572885 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb7c0aef60 msgr2=0x7fcb7c1521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:00.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.551+0000 7fcb80b17700 1 --2- 172.21.15.36:0/1671572885 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb7c0aef60 0x7fcb7c1521b0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0x7fcb6c00c000 tx=0x7fcb6c006df0).stop 2022-06-17T12:55:00.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.551+0000 7fcb80b17700 1 -- 172.21.15.36:0/1671572885 shutdown_connections 2022-06-17T12:55:00.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.551+0000 7fcb80b17700 1 --2- 172.21.15.36:0/1671572885 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcb60053af0 0x7fcb60055fb0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.551+0000 7fcb80b17700 1 --2- 172.21.15.36:0/1671572885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb7c13cf50 0x7fcb7c1529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.551+0000 7fcb80b17700 1 --2- 172.21.15.36:0/1671572885 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb7c13cb70 0x7fcb7c0b5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.551+0000 7fcb80b17700 1 --2- 172.21.15.36:0/1671572885 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb7c0aef60 0x7fcb7c1521b0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.551+0000 7fcb80b17700 1 -- 172.21.15.36:0/1671572885 >> 172.21.15.36:0/1671572885 conn(0x7fcb7c05c550 msgr2=0x7fcb7c061f30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:00.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.551+0000 7fcb80b17700 1 -- 172.21.15.36:0/1671572885 shutdown_connections 2022-06-17T12:55:00.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.551+0000 7fcb80b17700 1 -- 172.21.15.36:0/1671572885 wait complete. 2022-06-17T12:55:00.557 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-17T12:55:00.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.675+0000 7fb12aca7700 1 Processor -- start 2022-06-17T12:55:00.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.675+0000 7fb12aca7700 1 -- start start 2022-06-17T12:55:00.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.675+0000 7fb12aca7700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb1240b64f0 0x7fb124143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:00.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.675+0000 7fb12aca7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1240b4120 0x7fb124146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:00.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.675+0000 7fb12aca7700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb124147270 0x7fb124140100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:00.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.675+0000 7fb12aca7700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1240b68d0 con 0x7fb1240b64f0 2022-06-17T12:55:00.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.675+0000 7fb12aca7700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1241476b0 con 0x7fb124147270 2022-06-17T12:55:00.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.675+0000 7fb12aca7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1240b6e20 con 0x7fb1240b4120 2022-06-17T12:55:00.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.675+0000 7fb128a43700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb1240b64f0 0x7fb124143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:00.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.675+0000 7fb128a43700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb1240b64f0 0x7fb124143150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37722/0 (socket says 172.21.15.36:37722) 2022-06-17T12:55:00.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.675+0000 7fb128a43700 1 -- 172.21.15.36:0/867570314 learned_addr learned my addr 172.21.15.36:0/867570314 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:00.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.675+0000 7fb123fff700 1 --2- 172.21.15.36:0/867570314 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1240b4120 0x7fb124146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:00.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.675+0000 7fb129244700 1 --2- 172.21.15.36:0/867570314 >> v2:172.21.15.70:3300/0 conn(0x7fb124147270 0x7fb124140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:00.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.675+0000 7fb128a43700 1 -- 172.21.15.36:0/867570314 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1240b4120 msgr2=0x7fb124146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:00.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.675+0000 7fb128a43700 1 --2- 172.21.15.36:0/867570314 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1240b4120 0x7fb124146d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.675+0000 7fb128a43700 1 -- 172.21.15.36:0/867570314 >> v2:172.21.15.70:3300/0 conn(0x7fb124147270 msgr2=0x7fb124140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:00.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.675+0000 7fb128a43700 1 --2- 172.21.15.36:0/867570314 >> v2:172.21.15.70:3300/0 conn(0x7fb124147270 0x7fb124140100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.675+0000 7fb128a43700 1 -- 172.21.15.36:0/867570314 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1241468e0 con 0x7fb1240b64f0 2022-06-17T12:55:00.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.675+0000 7fb129244700 1 --2- 172.21.15.36:0/867570314 >> v2:172.21.15.70:3300/0 conn(0x7fb124147270 0x7fb124140100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:55:00.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.675+0000 7fb128a43700 1 --2- 172.21.15.36:0/867570314 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb1240b64f0 0x7fb124143150 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7fb114002f70 tx=0x7fb114006eb0).ready entity=mon.0 client_cookie=180bf92893f61986 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:00.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.675+0000 7fb1237fe700 1 -- 172.21.15.36:0/867570314 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb11400d570 con 0x7fb1240b64f0 2022-06-17T12:55:00.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.675+0000 7fb1237fe700 1 -- 172.21.15.36:0/867570314 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb114000e90 con 0x7fb1240b64f0 2022-06-17T12:55:00.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.675+0000 7fb1237fe700 1 -- 172.21.15.36:0/867570314 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb11400d900 con 0x7fb1240b64f0 2022-06-17T12:55:00.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb12aca7700 1 -- 172.21.15.36:0/867570314 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb1240b64f0 msgr2=0x7fb124143150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:00.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb12aca7700 1 --2- 172.21.15.36:0/867570314 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb1240b64f0 0x7fb124143150 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7fb114002f70 tx=0x7fb114006eb0).stop 2022-06-17T12:55:00.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb12aca7700 1 -- 172.21.15.36:0/867570314 shutdown_connections 2022-06-17T12:55:00.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb12aca7700 1 --2- 172.21.15.36:0/867570314 >> v2:172.21.15.70:3300/0 conn(0x7fb124147270 0x7fb124140100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb12aca7700 1 --2- 172.21.15.36:0/867570314 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1240b4120 0x7fb124146d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb12aca7700 1 --2- 172.21.15.36:0/867570314 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb1240b64f0 0x7fb124143150 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb12aca7700 1 -- 172.21.15.36:0/867570314 >> 172.21.15.36:0/867570314 conn(0x7fb1240620f0 msgr2=0x7fb124062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:00.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb12aca7700 1 -- 172.21.15.36:0/867570314 shutdown_connections 2022-06-17T12:55:00.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb12aca7700 1 -- 172.21.15.36:0/867570314 wait complete. 2022-06-17T12:55:00.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb12aca7700 1 Processor -- start 2022-06-17T12:55:00.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb12aca7700 1 -- start start 2022-06-17T12:55:00.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb12aca7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1240b4120 0x7fb124148170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:00.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb12aca7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb1240b64f0 0x7fb1240b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:00.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb12aca7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb124147270 0x7fb1240b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:00.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb12aca7700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1241499c0 con 0x7fb1240b64f0 2022-06-17T12:55:00.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb12aca7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb124142060 con 0x7fb1240b4120 2022-06-17T12:55:00.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb12aca7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb124142300 con 0x7fb124147270 2022-06-17T12:55:00.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb123fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb1240b64f0 0x7fb1240b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:00.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb123fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb1240b64f0 0x7fb1240b19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37730/0 (socket says 172.21.15.36:37730) 2022-06-17T12:55:00.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb123fff700 1 -- 172.21.15.36:0/1126600237 learned_addr learned my addr 172.21.15.36:0/1126600237 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:00.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb128a43700 1 --2- 172.21.15.36:0/1126600237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1240b4120 0x7fb124148170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:00.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb129244700 1 --2- 172.21.15.36:0/1126600237 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb124147270 0x7fb1240b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:00.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb123fff700 1 -- 172.21.15.36:0/1126600237 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb124147270 msgr2=0x7fb1240b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:00.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb123fff700 1 --2- 172.21.15.36:0/1126600237 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb124147270 0x7fb1240b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb123fff700 1 -- 172.21.15.36:0/1126600237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1240b4120 msgr2=0x7fb124148170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:00.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb123fff700 1 --2- 172.21.15.36:0/1126600237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1240b4120 0x7fb124148170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:00.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb123fff700 1 -- 172.21.15.36:0/1126600237 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11400b1e0 con 0x7fb1240b64f0 2022-06-17T12:55:00.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb129244700 1 --2- 172.21.15.36:0/1126600237 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb124147270 0x7fb1240b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:00.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb123fff700 1 --2- 172.21.15.36:0/1126600237 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb1240b64f0 0x7fb1240b19b0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7fb11800a000 tx=0x7fb118008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:00.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb121ffb700 1 -- 172.21.15.36:0/1126600237 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb118011780 con 0x7fb1240b64f0 2022-06-17T12:55:00.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb121ffb700 1 -- 172.21.15.36:0/1126600237 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1180118e0 con 0x7fb1240b64f0 2022-06-17T12:55:00.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb12aca7700 1 -- 172.21.15.36:0/1126600237 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1240b2390 con 0x7fb1240b64f0 2022-06-17T12:55:00.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.679+0000 7fb121ffb700 1 -- 172.21.15.36:0/1126600237 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb118011c40 con 0x7fb1240b64f0 2022-06-17T12:55:00.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.683+0000 7fb12aca7700 1 -- 172.21.15.36:0/1126600237 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1240b3380 con 0x7fb1240b64f0 2022-06-17T12:55:00.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.683+0000 7fb12aca7700 1 -- 172.21.15.36:0/1126600237 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1241424c0 con 0x7fb1240b64f0 2022-06-17T12:55:00.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.683+0000 7fb121ffb700 1 -- 172.21.15.36:0/1126600237 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fb118012420 con 0x7fb1240b64f0 2022-06-17T12:55:00.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.683+0000 7fb121ffb700 1 --2- 172.21.15.36:0/1126600237 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb104053920 0x7fb104055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:00.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.683+0000 7fb121ffb700 1 -- 172.21.15.36:0/1126600237 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb11806be80 con 0x7fb1240b64f0 2022-06-17T12:55:00.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.683+0000 7fb128a43700 1 --2- 172.21.15.36:0/1126600237 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb104053920 0x7fb104055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:00.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.683+0000 7fb128a43700 1 --2- 172.21.15.36:0/1126600237 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb104053920 0x7fb104055de0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7fb11401c000 tx=0x7fb114015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:00.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.691+0000 7fb121ffb700 1 -- 172.21.15.36:0/1126600237 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb118042c20 con 0x7fb1240b64f0 2022-06-17T12:55:00.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:00.855+0000 7fb12aca7700 1 -- 172.21.15.36:0/1126600237 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb124143640 con 0x7fb1240b64f0 2022-06-17T12:55:01.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.675+0000 7fb121ffb700 1 -- 172.21.15.36:0/1126600237 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v54) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb118046240 con 0x7fb1240b64f0 2022-06-17T12:55:01.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.679+0000 7fb12aca7700 1 -- 172.21.15.36:0/1126600237 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb104053920 msgr2=0x7fb104055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:01.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.679+0000 7fb12aca7700 1 --2- 172.21.15.36:0/1126600237 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb104053920 0x7fb104055de0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7fb11401c000 tx=0x7fb114015000).stop 2022-06-17T12:55:01.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.679+0000 7fb12aca7700 1 -- 172.21.15.36:0/1126600237 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb1240b64f0 msgr2=0x7fb1240b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:01.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.679+0000 7fb12aca7700 1 --2- 172.21.15.36:0/1126600237 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb1240b64f0 0x7fb1240b19b0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7fb11800a000 tx=0x7fb118008040).stop 2022-06-17T12:55:01.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.679+0000 7fb12aca7700 1 -- 172.21.15.36:0/1126600237 shutdown_connections 2022-06-17T12:55:01.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.679+0000 7fb12aca7700 1 --2- 172.21.15.36:0/1126600237 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb104053920 0x7fb104055de0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:01.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.679+0000 7fb12aca7700 1 --2- 172.21.15.36:0/1126600237 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb124147270 0x7fb1240b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:01.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.679+0000 7fb12aca7700 1 --2- 172.21.15.36:0/1126600237 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb1240b64f0 0x7fb1240b19b0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:01.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.679+0000 7fb12aca7700 1 --2- 172.21.15.36:0/1126600237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1240b4120 0x7fb124148170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:01.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.679+0000 7fb12aca7700 1 -- 172.21.15.36:0/1126600237 >> 172.21.15.36:0/1126600237 conn(0x7fb1240620f0 msgr2=0x7fb12413f230 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:01.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.679+0000 7fb12aca7700 1 -- 172.21.15.36:0/1126600237 shutdown_connections 2022-06-17T12:55:01.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.679+0000 7fb12aca7700 1 -- 172.21.15.36:0/1126600237 wait complete. 2022-06-17T12:55:01.681 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:55:01.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.795+0000 7f07e1abd700 1 Processor -- start 2022-06-17T12:55:01.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.795+0000 7f07e1abd700 1 -- start start 2022-06-17T12:55:01.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.799+0000 7f07e1abd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f07dc1429c0 0x7f07dc0b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:01.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.799+0000 7f07e1abd700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07dc13fb70 con 0x7f07dc0b6440 2022-06-17T12:55:01.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.799+0000 7f07e1abd700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07dc13fe10 con 0x7f07dc139160 2022-06-17T12:55:01.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.799+0000 7f07e1abd700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07dc1400b0 con 0x7f07dc1429c0 2022-06-17T12:55:01.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.799+0000 7f07daffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f07dc1429c0 0x7f07dc0b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:01.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.799+0000 7f07daffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f07dc1429c0 0x7f07dc0b6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37736/0 (socket says 172.21.15.36:37736) 2022-06-17T12:55:01.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.799+0000 7f07daffd700 1 -- 172.21.15.36:0/260224864 learned_addr learned my addr 172.21.15.36:0/260224864 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:01.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.799+0000 7f07daffd700 1 -- 172.21.15.36:0/260224864 >> v1:172.21.15.70:6789/0 conn(0x7f07dc139160 legacy=0x7f07dc0aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:01.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.799+0000 7f07daffd700 1 -- 172.21.15.36:0/260224864 >> v1:172.21.15.36:6789/0 conn(0x7f07dc0b6440 legacy=0x7f07dc13f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:01.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.799+0000 7f07daffd700 1 -- 172.21.15.36:0/260224864 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07dc1395b0 con 0x7f07dc1429c0 2022-06-17T12:55:01.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.799+0000 7f07da7fc700 1 -- 172.21.15.36:0/260224864 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2140230732 0 0) 0x7f07dc13fb70 con 0x7f07dc0b6440 2022-06-17T12:55:01.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.799+0000 7f07daffd700 1 --2- 172.21.15.36:0/260224864 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f07dc1429c0 0x7f07dc0b6820 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 rx=0x7f07cc002f70 tx=0x7f07cc006eb0).ready entity=mon.0 client_cookie=390a2270ac91995 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:01.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.799+0000 7f07da7fc700 1 -- 172.21.15.36:0/260224864 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07cc00d660 con 0x7f07dc1429c0 2022-06-17T12:55:01.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.799+0000 7f07da7fc700 1 -- 172.21.15.36:0/260224864 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07cc000e90 con 0x7f07dc1429c0 2022-06-17T12:55:01.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.799+0000 7f07da7fc700 1 -- 172.21.15.36:0/260224864 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07cc00d9b0 con 0x7f07dc1429c0 2022-06-17T12:55:01.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.799+0000 7f07e1abd700 1 -- 172.21.15.36:0/260224864 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f07dc1429c0 msgr2=0x7f07dc0b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:01.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.799+0000 7f07e1abd700 1 --2- 172.21.15.36:0/260224864 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f07dc1429c0 0x7f07dc0b6820 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 rx=0x7f07cc002f70 tx=0x7f07cc006eb0).stop 2022-06-17T12:55:01.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.799+0000 7f07e1abd700 1 -- 172.21.15.36:0/260224864 shutdown_connections 2022-06-17T12:55:01.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.799+0000 7f07e1abd700 1 --2- 172.21.15.36:0/260224864 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f07dc1429c0 0x7f07dc0b6820 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:01.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.799+0000 7f07e1abd700 1 -- 172.21.15.36:0/260224864 >> 172.21.15.36:0/260224864 conn(0x7f07dc0638d0 msgr2=0x7f07dc0b5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:01.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.799+0000 7f07e1abd700 1 -- 172.21.15.36:0/260224864 shutdown_connections 2022-06-17T12:55:01.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.799+0000 7f07e1abd700 1 -- 172.21.15.36:0/260224864 wait complete. 2022-06-17T12:55:01.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.799+0000 7f07e1abd700 1 Processor -- start 2022-06-17T12:55:01.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.803+0000 7f07e1abd700 1 -- start start 2022-06-17T12:55:01.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.803+0000 7f07e1abd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f07dc0b6440 0x7f07dc13e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:01.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.803+0000 7f07e1abd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07dc139160 0x7f07dc152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:01.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.803+0000 7f07e1abd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f07dc1429c0 0x7f07dc0668e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:01.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.803+0000 7f07e1abd700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc0b6890 con 0x7f07dc1429c0 2022-06-17T12:55:01.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.803+0000 7f07e1abd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc1533e0 con 0x7f07dc139160 2022-06-17T12:55:01.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.803+0000 7f07e1abd700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc152340 con 0x7f07dc0b6440 2022-06-17T12:55:01.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.803+0000 7f07dbfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f07dc1429c0 0x7f07dc0668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:01.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.803+0000 7f07dbfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f07dc1429c0 0x7f07dc0668e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37744/0 (socket says 172.21.15.36:37744) 2022-06-17T12:55:01.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.803+0000 7f07dbfff700 1 -- 172.21.15.36:0/3542973257 learned_addr learned my addr 172.21.15.36:0/3542973257 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:01.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.803+0000 7f07db7fe700 1 --2- 172.21.15.36:0/3542973257 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f07dc0b6440 0x7f07dc13e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:01.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.803+0000 7f07daffd700 1 --2- 172.21.15.36:0/3542973257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07dc139160 0x7f07dc152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:01.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.803+0000 7f07dbfff700 1 -- 172.21.15.36:0/3542973257 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f07dc0b6440 msgr2=0x7f07dc13e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:01.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.803+0000 7f07dbfff700 1 --2- 172.21.15.36:0/3542973257 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f07dc0b6440 0x7f07dc13e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:01.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.803+0000 7f07dbfff700 1 -- 172.21.15.36:0/3542973257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07dc139160 msgr2=0x7f07dc152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:01.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.803+0000 7f07dbfff700 1 --2- 172.21.15.36:0/3542973257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07dc139160 0x7f07dc152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:01.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.803+0000 7f07dbfff700 1 -- 172.21.15.36:0/3542973257 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc00b1e0 con 0x7f07dc1429c0 2022-06-17T12:55:01.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.803+0000 7f07dbfff700 1 --2- 172.21.15.36:0/3542973257 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f07dc1429c0 0x7f07dc0668e0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f07dc0b3860 tx=0x7f07d4007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:01.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.803+0000 7f07d8ff9700 1 -- 172.21.15.36:0/3542973257 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07d400e4a0 con 0x7f07dc1429c0 2022-06-17T12:55:01.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.803+0000 7f07d8ff9700 1 -- 172.21.15.36:0/3542973257 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d400e600 con 0x7f07dc1429c0 2022-06-17T12:55:01.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.803+0000 7f07e1abd700 1 -- 172.21.15.36:0/3542973257 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07dc1525e0 con 0x7f07dc1429c0 2022-06-17T12:55:01.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.803+0000 7f07d8ff9700 1 -- 172.21.15.36:0/3542973257 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07d400e960 con 0x7f07dc1429c0 2022-06-17T12:55:01.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.803+0000 7f07e1abd700 1 -- 172.21.15.36:0/3542973257 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07dc10eec0 con 0x7f07dc1429c0 2022-06-17T12:55:01.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.807+0000 7f07d8ff9700 1 -- 172.21.15.36:0/3542973257 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f07d4019030 con 0x7f07dc1429c0 2022-06-17T12:55:01.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.807+0000 7f07e1abd700 1 -- 172.21.15.36:0/3542973257 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07dc057410 con 0x7f07dc1429c0 2022-06-17T12:55:01.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.807+0000 7f07d8ff9700 1 --2- 172.21.15.36:0/3542973257 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f07bc0539b0 0x7f07bc055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:01.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.807+0000 7f07d8ff9700 1 -- 172.21.15.36:0/3542973257 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f07d406b720 con 0x7f07dc1429c0 2022-06-17T12:55:01.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.807+0000 7f07db7fe700 1 --2- 172.21.15.36:0/3542973257 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f07bc0539b0 0x7f07bc055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:01.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.807+0000 7f07db7fe700 1 --2- 172.21.15.36:0/3542973257 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f07bc0539b0 0x7f07bc055e70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f07d000d000 tx=0x7f07d0009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:01.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.815+0000 7f07d8ff9700 1 -- 172.21.15.36:0/3542973257 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f07d4042420 con 0x7f07dc1429c0 2022-06-17T12:55:01.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.983+0000 7f07e1abd700 1 -- 172.21.15.36:0/3542973257 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07dc05d820 con 0x7f07dc1429c0 2022-06-17T12:55:01.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.983+0000 7f07d8ff9700 1 -- 172.21.15.36:0/3542973257 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f07d4045a40 con 0x7f07dc1429c0 2022-06-17T12:55:01.974 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:01.975 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":54,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/676934608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":676934608},{"type":"v1","addr":"172.21.15.70:6833","nonce":676934608}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:54:59.995522+0000","modified":"2022-06-17T12:55:01.664559+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7395":{"gid":7395,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/233328233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":233328233},{"type":"v1","addr":"172.21.15.36:6835","nonce":233328233}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-06-17T12:55:01.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.987+0000 7f07e1abd700 1 -- 172.21.15.36:0/3542973257 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f07bc0539b0 msgr2=0x7f07bc055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:01.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.987+0000 7f07e1abd700 1 --2- 172.21.15.36:0/3542973257 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f07bc0539b0 0x7f07bc055e70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f07d000d000 tx=0x7f07d0009040).stop 2022-06-17T12:55:01.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.987+0000 7f07e1abd700 1 -- 172.21.15.36:0/3542973257 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f07dc1429c0 msgr2=0x7f07dc0668e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:01.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.987+0000 7f07e1abd700 1 --2- 172.21.15.36:0/3542973257 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f07dc1429c0 0x7f07dc0668e0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f07dc0b3860 tx=0x7f07d4007f40).stop 2022-06-17T12:55:01.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.987+0000 7f07e1abd700 1 -- 172.21.15.36:0/3542973257 shutdown_connections 2022-06-17T12:55:01.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.987+0000 7f07e1abd700 1 --2- 172.21.15.36:0/3542973257 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f07bc0539b0 0x7f07bc055e70 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:01.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.987+0000 7f07e1abd700 1 --2- 172.21.15.36:0/3542973257 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f07dc1429c0 0x7f07dc0668e0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:01.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.987+0000 7f07e1abd700 1 --2- 172.21.15.36:0/3542973257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07dc139160 0x7f07dc152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:01.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.987+0000 7f07e1abd700 1 --2- 172.21.15.36:0/3542973257 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f07dc0b6440 0x7f07dc13e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:01.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.987+0000 7f07e1abd700 1 -- 172.21.15.36:0/3542973257 >> 172.21.15.36:0/3542973257 conn(0x7f07dc0638d0 msgr2=0x7f07dc0b12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:01.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.987+0000 7f07e1abd700 1 -- 172.21.15.36:0/3542973257 shutdown_connections 2022-06-17T12:55:01.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:01.987+0000 7f07e1abd700 1 -- 172.21.15.36:0/3542973257 wait complete. 2022-06-17T12:55:01.989 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 54 2022-06-17T12:55:01.991 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:55:02.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.103+0000 7f4285ac3700 1 Processor -- start 2022-06-17T12:55:02.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.103+0000 7f4285ac3700 1 -- start start 2022-06-17T12:55:02.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.103+0000 7f4285ac3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4280060ac0 0x7f4280064840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:02.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.103+0000 7f4285ac3700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4280140e70 0x7f4280141300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:02.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.103+0000 7f4285ac3700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4280060ea0 con 0x7f42800b22b0 2022-06-17T12:55:02.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.103+0000 7f4285ac3700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42801445f0 con 0x7f4280140e70 2022-06-17T12:55:02.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.103+0000 7f4285ac3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4280144890 con 0x7f4280060ac0 2022-06-17T12:55:02.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.103+0000 7f427ffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4280140e70 0x7f4280141300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:02.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.103+0000 7f427f7fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4280060ac0 0x7f4280064840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:02.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.107+0000 7f427ffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4280140e70 0x7f4280141300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37754/0 (socket says 172.21.15.36:37754) 2022-06-17T12:55:02.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.107+0000 7f427f7fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4280060ac0 0x7f4280064840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:52540/0 (socket says 172.21.15.36:52540) 2022-06-17T12:55:02.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.107+0000 7f427ffff700 1 -- 172.21.15.36:0/496189978 learned_addr learned my addr 172.21.15.36:0/496189978 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:02.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.107+0000 7f4276ffd700 1 --1- 172.21.15.36:0/496189978 >> v1:172.21.15.70:6789/0 conn(0x7f42800b22b0 0x7f4280144d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:51014/0 (socket says 172.21.15.36:51014) 2022-06-17T12:55:02.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.107+0000 7f427f7fe700 1 -- 172.21.15.36:0/496189978 >> v2:172.21.15.36:3300/0 conn(0x7f4280140e70 msgr2=0x7f4280141300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:02.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.107+0000 7f427f7fe700 1 --2- 172.21.15.36:0/496189978 >> v2:172.21.15.36:3300/0 conn(0x7f4280140e70 0x7f4280141300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.107+0000 7f427f7fe700 1 -- 172.21.15.36:0/496189978 >> v1:172.21.15.70:6789/0 conn(0x7f42800b22b0 legacy=0x7f4280144d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:02.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.107+0000 7f427effd700 1 -- 172.21.15.36:0/496189978 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034218964 0 0) 0x7f4280060ea0 con 0x7f42800b22b0 2022-06-17T12:55:02.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.107+0000 7f427f7fe700 1 -- 172.21.15.36:0/496189978 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42800b0910 con 0x7f4280060ac0 2022-06-17T12:55:02.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.107+0000 7f427f7fe700 1 --2- 172.21.15.36:0/496189978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4280060ac0 0x7f4280064840 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f4280063bd0 tx=0x7f4270006df0).ready entity=mon.2 client_cookie=4dbccbaa94cc751f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:02.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.107+0000 7f427effd700 1 -- 172.21.15.36:0/496189978 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f427000d570 con 0x7f4280060ac0 2022-06-17T12:55:02.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.107+0000 7f427effd700 1 -- 172.21.15.36:0/496189978 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4270000e90 con 0x7f4280060ac0 2022-06-17T12:55:02.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.107+0000 7f427effd700 1 -- 172.21.15.36:0/496189978 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f427000d8c0 con 0x7f4280060ac0 2022-06-17T12:55:02.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.107+0000 7f4285ac3700 1 -- 172.21.15.36:0/496189978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4280060ac0 msgr2=0x7f4280064840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:02.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.107+0000 7f4285ac3700 1 --2- 172.21.15.36:0/496189978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4280060ac0 0x7f4280064840 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f4280063bd0 tx=0x7f4270006df0).stop 2022-06-17T12:55:02.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.107+0000 7f4285ac3700 1 -- 172.21.15.36:0/496189978 shutdown_connections 2022-06-17T12:55:02.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.107+0000 7f4285ac3700 1 --2- 172.21.15.36:0/496189978 >> v2:172.21.15.36:3300/0 conn(0x7f4280140e70 0x7f4280141300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.107+0000 7f4285ac3700 1 --2- 172.21.15.36:0/496189978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4280060ac0 0x7f4280064840 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.107+0000 7f4285ac3700 1 -- 172.21.15.36:0/496189978 >> 172.21.15.36:0/496189978 conn(0x7f42800b6870 msgr2=0x7f42800b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:02.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.107+0000 7f4285ac3700 1 -- 172.21.15.36:0/496189978 shutdown_connections 2022-06-17T12:55:02.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.107+0000 7f4285ac3700 1 -- 172.21.15.36:0/496189978 wait complete. 2022-06-17T12:55:02.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f4285ac3700 1 Processor -- start 2022-06-17T12:55:02.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f4285ac3700 1 -- start start 2022-06-17T12:55:02.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f4285ac3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4280060ac0 0x7f428013fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:02.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f4285ac3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42800b22b0 0x7f428013cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:02.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f4285ac3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4280140e70 0x7f428013d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:02.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f4285ac3700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428013f470 con 0x7f4280060ac0 2022-06-17T12:55:02.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f4285ac3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428013de40 con 0x7f42800b22b0 2022-06-17T12:55:02.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f4285ac3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f428013f6a0 con 0x7f4280140e70 2022-06-17T12:55:02.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f427f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4280060ac0 0x7f428013fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:02.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f427f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4280060ac0 0x7f428013fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37756/0 (socket says 172.21.15.36:37756) 2022-06-17T12:55:02.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f427f7fe700 1 -- 172.21.15.36:0/3523125658 learned_addr learned my addr 172.21.15.36:0/3523125658 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:02.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f427ffff700 1 --2- 172.21.15.36:0/3523125658 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4280140e70 0x7f428013d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:02.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f4276ffd700 1 --2- 172.21.15.36:0/3523125658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42800b22b0 0x7f428013cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:02.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f427f7fe700 1 -- 172.21.15.36:0/3523125658 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4280140e70 msgr2=0x7f428013d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:02.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f427f7fe700 1 --2- 172.21.15.36:0/3523125658 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4280140e70 0x7f428013d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f427f7fe700 1 -- 172.21.15.36:0/3523125658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42800b22b0 msgr2=0x7f428013cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:02.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f427f7fe700 1 --2- 172.21.15.36:0/3523125658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42800b22b0 0x7f428013cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f427f7fe700 1 -- 172.21.15.36:0/3523125658 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427000b180 con 0x7f4280060ac0 2022-06-17T12:55:02.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f4276ffd700 1 --2- 172.21.15.36:0/3523125658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42800b22b0 0x7f428013cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:55:02.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f427f7fe700 1 --2- 172.21.15.36:0/3523125658 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4280060ac0 0x7f428013fc60 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f4280063bd0 tx=0x7f4270014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:02.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f427d7fa700 1 -- 172.21.15.36:0/3523125658 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f427000dea0 con 0x7f4280060ac0 2022-06-17T12:55:02.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f427d7fa700 1 -- 172.21.15.36:0/3523125658 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4270002f70 con 0x7f4280060ac0 2022-06-17T12:55:02.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f427d7fa700 1 -- 172.21.15.36:0/3523125658 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f427001e5f0 con 0x7f4280060ac0 2022-06-17T12:55:02.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f4285ac3700 1 -- 172.21.15.36:0/3523125658 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f428013e0e0 con 0x7f4280060ac0 2022-06-17T12:55:02.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.111+0000 7f4285ac3700 1 -- 172.21.15.36:0/3523125658 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4280146840 con 0x7f4280060ac0 2022-06-17T12:55:02.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.115+0000 7f4285ac3700 1 -- 172.21.15.36:0/3523125658 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4280057410 con 0x7f4280060ac0 2022-06-17T12:55:02.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.115+0000 7f427d7fa700 1 -- 172.21.15.36:0/3523125658 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f427001c070 con 0x7f4280060ac0 2022-06-17T12:55:02.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.115+0000 7f427d7fa700 1 --2- 172.21.15.36:0/3523125658 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4264053920 0x7f4264055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:02.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.115+0000 7f427d7fa700 1 -- 172.21.15.36:0/3523125658 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f427006cb50 con 0x7f4280060ac0 2022-06-17T12:55:02.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.115+0000 7f4276ffd700 1 --2- 172.21.15.36:0/3523125658 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4264053920 0x7f4264055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:02.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.115+0000 7f4276ffd700 1 --2- 172.21.15.36:0/3523125658 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4264053920 0x7f4264055de0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f4268008000 tx=0x7f4268009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:02.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.123+0000 7f427d7fa700 1 -- 172.21.15.36:0/3523125658 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4270043880 con 0x7f4280060ac0 2022-06-17T12:55:02.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.247+0000 7f4285ac3700 1 -- 172.21.15.36:0/3523125658 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42801524c0 con 0x7f4280060ac0 2022-06-17T12:55:02.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.247+0000 7f427d7fa700 1 -- 172.21.15.36:0/3523125658 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f4270023070 con 0x7f4280060ac0 2022-06-17T12:55:02.238 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:02.239 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":104,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:55:00.998590+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:54:58.393546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-06-17T12:54:58.997835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:55:02.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.251+0000 7f4285ac3700 1 -- 172.21.15.36:0/3523125658 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4264053920 msgr2=0x7f4264055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:02.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.251+0000 7f4285ac3700 1 --2- 172.21.15.36:0/3523125658 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4264053920 0x7f4264055de0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f4268008000 tx=0x7f4268009040).stop 2022-06-17T12:55:02.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.251+0000 7f4285ac3700 1 -- 172.21.15.36:0/3523125658 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4280060ac0 msgr2=0x7f428013fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:02.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.251+0000 7f4285ac3700 1 --2- 172.21.15.36:0/3523125658 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4280060ac0 0x7f428013fc60 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f4280063bd0 tx=0x7f4270014000).stop 2022-06-17T12:55:02.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.251+0000 7f4285ac3700 1 -- 172.21.15.36:0/3523125658 shutdown_connections 2022-06-17T12:55:02.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.251+0000 7f4285ac3700 1 --2- 172.21.15.36:0/3523125658 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4264053920 0x7f4264055de0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.251+0000 7f4285ac3700 1 --2- 172.21.15.36:0/3523125658 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4280140e70 0x7f428013d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.251+0000 7f4285ac3700 1 --2- 172.21.15.36:0/3523125658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42800b22b0 0x7f428013cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.251+0000 7f4285ac3700 1 --2- 172.21.15.36:0/3523125658 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4280060ac0 0x7f428013fc60 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.251+0000 7f4285ac3700 1 -- 172.21.15.36:0/3523125658 >> 172.21.15.36:0/3523125658 conn(0x7f42800b6870 msgr2=0x7f428005bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:02.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.251+0000 7f4285ac3700 1 -- 172.21.15.36:0/3523125658 shutdown_connections 2022-06-17T12:55:02.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.251+0000 7f4285ac3700 1 -- 172.21.15.36:0/3523125658 wait complete. 2022-06-17T12:55:02.252 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:55:02.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.379+0000 7fa6e974f700 1 Processor -- start 2022-06-17T12:55:02.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.379+0000 7fa6e974f700 1 -- start start 2022-06-17T12:55:02.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.379+0000 7fa6e974f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6e405fab0 0x7fa6e4143280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:02.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.379+0000 7fa6e974f700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa6e40aea60 0x7fa6e413fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:02.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.379+0000 7fa6e974f700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6e405fe90 con 0x7fa6e4140260 2022-06-17T12:55:02.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.379+0000 7fa6e974f700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6e40b0b60 con 0x7fa6e40aea60 2022-06-17T12:55:02.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.379+0000 7fa6e974f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e4143770 con 0x7fa6e405fab0 2022-06-17T12:55:02.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.379+0000 7fa6e37fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fa6e4140260 0x7fa6e4146690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55298/0 (socket says 172.21.15.36:55298) 2022-06-17T12:55:02.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.379+0000 7fa6e37fe700 1 -- 172.21.15.36:0/2848343254 learned_addr learned my addr 172.21.15.36:0/2848343254 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:02.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.379+0000 7fa6e27fc700 1 --2- 172.21.15.36:0/2848343254 >> v2:172.21.15.70:3300/0 conn(0x7fa6e40aea60 0x7fa6e413fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:02.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.379+0000 7fa6e2ffd700 1 --2- 172.21.15.36:0/2848343254 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6e405fab0 0x7fa6e4143280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:02.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.379+0000 7fa6e1ffb700 1 -- 172.21.15.36:0/2848343254 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956299956 0 0) 0x7fa6e405fe90 con 0x7fa6e4140260 2022-06-17T12:55:02.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.379+0000 7fa6e1ffb700 1 -- 172.21.15.36:0/2848343254 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6c4003260 con 0x7fa6e4140260 2022-06-17T12:55:02.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.379+0000 7fa6e27fc700 1 -- 172.21.15.36:0/2848343254 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6e405fab0 msgr2=0x7fa6e4143280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:02.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.379+0000 7fa6e27fc700 1 --2- 172.21.15.36:0/2848343254 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6e405fab0 0x7fa6e4143280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.379+0000 7fa6e27fc700 1 -- 172.21.15.36:0/2848343254 >> v1:172.21.15.36:6789/0 conn(0x7fa6e4140260 legacy=0x7fa6e4146690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:02.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.379+0000 7fa6e27fc700 1 -- 172.21.15.36:0/2848343254 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e4143990 con 0x7fa6e40aea60 2022-06-17T12:55:02.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.379+0000 7fa6e27fc700 1 --2- 172.21.15.36:0/2848343254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6e40aea60 0x7fa6e413fd70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fa6d4002f70 tx=0x7fa6d4006df0).ready entity=mon.1 client_cookie=1e133006031f543c server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:02.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.383+0000 7fa6e1ffb700 1 -- 172.21.15.36:0/2848343254 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6d400d4a0 con 0x7fa6e40aea60 2022-06-17T12:55:02.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.383+0000 7fa6e1ffb700 1 -- 172.21.15.36:0/2848343254 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d4000e90 con 0x7fa6e40aea60 2022-06-17T12:55:02.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.383+0000 7fa6e1ffb700 1 -- 172.21.15.36:0/2848343254 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6d400d7f0 con 0x7fa6e40aea60 2022-06-17T12:55:02.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.383+0000 7fa6e974f700 1 -- 172.21.15.36:0/2848343254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6e40aea60 msgr2=0x7fa6e413fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:02.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.383+0000 7fa6e974f700 1 --2- 172.21.15.36:0/2848343254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6e40aea60 0x7fa6e413fd70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fa6d4002f70 tx=0x7fa6d4006df0).stop 2022-06-17T12:55:02.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.383+0000 7fa6e974f700 1 -- 172.21.15.36:0/2848343254 shutdown_connections 2022-06-17T12:55:02.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.383+0000 7fa6e974f700 1 --2- 172.21.15.36:0/2848343254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6e40aea60 0x7fa6e413fd70 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.383+0000 7fa6e974f700 1 --2- 172.21.15.36:0/2848343254 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6e405fab0 0x7fa6e4143280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.383+0000 7fa6e974f700 1 -- 172.21.15.36:0/2848343254 >> 172.21.15.36:0/2848343254 conn(0x7fa6e405c550 msgr2=0x7fa6e405c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:02.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.383+0000 7fa6e974f700 1 -- 172.21.15.36:0/2848343254 shutdown_connections 2022-06-17T12:55:02.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.383+0000 7fa6e974f700 1 -- 172.21.15.36:0/2848343254 wait complete. 2022-06-17T12:55:02.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.383+0000 7fa6e974f700 1 Processor -- start 2022-06-17T12:55:02.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.383+0000 7fa6e974f700 1 -- start start 2022-06-17T12:55:02.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.387+0000 7fa6e974f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6e405fab0 0x7fa6e4152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:02.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.387+0000 7fa6e974f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa6e40aea60 0x7fa6e412d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:02.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.387+0000 7fa6e974f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6e4140260 0x7fa6e412d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:02.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.387+0000 7fa6e974f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e4142c60 con 0x7fa6e40aea60 2022-06-17T12:55:02.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.387+0000 7fa6e974f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e412dd50 con 0x7fa6e4140260 2022-06-17T12:55:02.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.387+0000 7fa6e974f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e412dac0 con 0x7fa6e405fab0 2022-06-17T12:55:02.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.387+0000 7fa6e27fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa6e40aea60 0x7fa6e412d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:02.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.387+0000 7fa6e27fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa6e40aea60 0x7fa6e412d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37772/0 (socket says 172.21.15.36:37772) 2022-06-17T12:55:02.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.387+0000 7fa6e27fc700 1 -- 172.21.15.36:0/1871493582 learned_addr learned my addr 172.21.15.36:0/1871493582 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:02.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.387+0000 7fa6e37fe700 1 --2- 172.21.15.36:0/1871493582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6e4140260 0x7fa6e412d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:02.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.387+0000 7fa6e2ffd700 1 --2- 172.21.15.36:0/1871493582 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6e405fab0 0x7fa6e4152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:02.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.387+0000 7fa6e27fc700 1 -- 172.21.15.36:0/1871493582 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6e405fab0 msgr2=0x7fa6e4152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:02.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.387+0000 7fa6e27fc700 1 --2- 172.21.15.36:0/1871493582 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6e405fab0 0x7fa6e4152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.387+0000 7fa6e27fc700 1 -- 172.21.15.36:0/1871493582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6e4140260 msgr2=0x7fa6e412d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:02.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.387+0000 7fa6e27fc700 1 --2- 172.21.15.36:0/1871493582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6e4140260 0x7fa6e412d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.387+0000 7fa6e27fc700 1 -- 172.21.15.36:0/1871493582 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d400b180 con 0x7fa6e40aea60 2022-06-17T12:55:02.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.387+0000 7fa6e27fc700 1 --2- 172.21.15.36:0/1871493582 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa6e40aea60 0x7fa6e412d0e0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7fa6d4002f70 tx=0x7fa6d4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:02.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.387+0000 7fa6dbfff700 1 -- 172.21.15.36:0/1871493582 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6d40139c0 con 0x7fa6e40aea60 2022-06-17T12:55:02.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.387+0000 7fa6dbfff700 1 -- 172.21.15.36:0/1871493582 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d4013b20 con 0x7fa6e40aea60 2022-06-17T12:55:02.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.387+0000 7fa6dbfff700 1 -- 172.21.15.36:0/1871493582 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6d4013e80 con 0x7fa6e40aea60 2022-06-17T12:55:02.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.387+0000 7fa6e974f700 1 -- 172.21.15.36:0/1871493582 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e413f660 con 0x7fa6e40aea60 2022-06-17T12:55:02.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.387+0000 7fa6e974f700 1 -- 172.21.15.36:0/1871493582 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e41531b0 con 0x7fa6e40aea60 2022-06-17T12:55:02.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.391+0000 7fa6e974f700 1 -- 172.21.15.36:0/1871493582 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e413f820 con 0x7fa6e40aea60 2022-06-17T12:55:02.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.391+0000 7fa6dbfff700 1 -- 172.21.15.36:0/1871493582 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fa6d4025030 con 0x7fa6e40aea60 2022-06-17T12:55:02.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.391+0000 7fa6dbfff700 1 --2- 172.21.15.36:0/1871493582 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa6c8053ad0 0x7fa6c8055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:02.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.391+0000 7fa6dbfff700 1 -- 172.21.15.36:0/1871493582 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa6d406ce20 con 0x7fa6e40aea60 2022-06-17T12:55:02.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.391+0000 7fa6e2ffd700 1 --2- 172.21.15.36:0/1871493582 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa6c8053ad0 0x7fa6c8055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:02.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.391+0000 7fa6e2ffd700 1 --2- 172.21.15.36:0/1871493582 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa6c8053ad0 0x7fa6c8055f90 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7fa6cc00b000 tx=0x7fa6cc006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:02.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.399+0000 7fa6dbfff700 1 -- 172.21.15.36:0/1871493582 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa6e413f820 con 0x7fa6e40aea60 2022-06-17T12:55:02.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.563+0000 7fa6e974f700 1 -- 172.21.15.36:0/1871493582 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fa6e413f820 con 0x7fa6e40aea60 2022-06-17T12:55:02.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.571+0000 7fa6dbfff700 1 -- 172.21.15.36:0/1871493582 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v35) v1 ==== 184+0+0 (secure 0 0 0) 0x7fa6e413f820 con 0x7fa6e40aea60 2022-06-17T12:55:02.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.575+0000 7fa6e974f700 1 -- 172.21.15.36:0/1871493582 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa6c8053ad0 msgr2=0x7fa6c8055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:02.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.575+0000 7fa6e974f700 1 --2- 172.21.15.36:0/1871493582 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa6c8053ad0 0x7fa6c8055f90 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7fa6cc00b000 tx=0x7fa6cc006df0).stop 2022-06-17T12:55:02.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.575+0000 7fa6e974f700 1 -- 172.21.15.36:0/1871493582 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa6e40aea60 msgr2=0x7fa6e412d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:02.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.575+0000 7fa6e974f700 1 --2- 172.21.15.36:0/1871493582 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa6e40aea60 0x7fa6e412d0e0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7fa6d4002f70 tx=0x7fa6d4014000).stop 2022-06-17T12:55:02.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.575+0000 7fa6e974f700 1 -- 172.21.15.36:0/1871493582 shutdown_connections 2022-06-17T12:55:02.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.575+0000 7fa6e974f700 1 --2- 172.21.15.36:0/1871493582 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa6c8053ad0 0x7fa6c8055f90 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.575+0000 7fa6e974f700 1 --2- 172.21.15.36:0/1871493582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6e4140260 0x7fa6e412d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.575+0000 7fa6e974f700 1 --2- 172.21.15.36:0/1871493582 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa6e40aea60 0x7fa6e412d0e0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.575+0000 7fa6e974f700 1 --2- 172.21.15.36:0/1871493582 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6e405fab0 0x7fa6e4152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.575+0000 7fa6e974f700 1 -- 172.21.15.36:0/1871493582 >> 172.21.15.36:0/1871493582 conn(0x7fa6e405c550 msgr2=0x7fa6e40b64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:02.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.575+0000 7fa6e974f700 1 -- 172.21.15.36:0/1871493582 shutdown_connections 2022-06-17T12:55:02.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.575+0000 7fa6e974f700 1 -- 172.21.15.36:0/1871493582 wait complete. 2022-06-17T12:55:02.571 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.0 2022-06-17T12:55:02.579 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:55:02.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.703+0000 7f0565d03700 1 Processor -- start 2022-06-17T12:55:02.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.703+0000 7f0565d03700 1 -- start start 2022-06-17T12:55:02.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f0565d03700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f056013cf50 0x7f05601442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:02.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f0565d03700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05600af4b0 con 0x7f056013cb70 2022-06-17T12:55:02.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f0565d03700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05600ba500 con 0x7f05600aef60 2022-06-17T12:55:02.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f0565d03700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0560143da0 con 0x7f056013cf50 2022-06-17T12:55:02.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f0564d01700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f056013cb70 0x7f0560143690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55308/0 (socket says 172.21.15.36:55308) 2022-06-17T12:55:02.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f0564d01700 1 -- 172.21.15.36:0/973022627 learned_addr learned my addr 172.21.15.36:0/973022627 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:02.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f0565502700 1 --2- 172.21.15.36:0/973022627 >> v2:172.21.15.70:3300/0 conn(0x7f056013cf50 0x7f05601442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:02.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f05577fe700 1 -- 172.21.15.36:0/973022627 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 364670449 0 0) 0x7f05600af4b0 con 0x7f056013cb70 2022-06-17T12:55:02.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f05577fe700 1 -- 172.21.15.36:0/973022627 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f054c003260 con 0x7f056013cb70 2022-06-17T12:55:02.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f05577fe700 1 -- 172.21.15.36:0/973022627 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1115305404 0 0) 0x7f05600ba500 con 0x7f05600aef60 2022-06-17T12:55:02.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f05577fe700 1 -- 172.21.15.36:0/973022627 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05600af4b0 con 0x7f05600aef60 2022-06-17T12:55:02.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f05577fe700 1 -- 172.21.15.36:0/973022627 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2481386367 0 0) 0x7f054c003260 con 0x7f056013cb70 2022-06-17T12:55:02.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f05577fe700 1 -- 172.21.15.36:0/973022627 >> v2:172.21.15.70:3300/0 conn(0x7f056013cf50 msgr2=0x7f05601442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:02.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f05577fe700 1 --2- 172.21.15.36:0/973022627 >> v2:172.21.15.70:3300/0 conn(0x7f056013cf50 0x7f05601442a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f05577fe700 1 -- 172.21.15.36:0/973022627 >> v1:172.21.15.70:6789/0 conn(0x7f05600aef60 legacy=0x7f05600b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:02.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f05577fe700 1 -- 172.21.15.36:0/973022627 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0560142f50 con 0x7f056013cb70 2022-06-17T12:55:02.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f05577fe700 1 -- 172.21.15.36:0/973022627 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f055c001660 con 0x7f056013cb70 2022-06-17T12:55:02.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f05577fe700 1 -- 172.21.15.36:0/973022627 >> v1:172.21.15.36:6789/0 conn(0x7f056013cb70 legacy=0x7f0560143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:02.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f05577fe700 1 --2- 172.21.15.36:0/973022627 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f054c003930 0x7f054c003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:02.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f05577fe700 1 --2- 172.21.15.36:0/973022627 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f054c004500 0x7f054c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:02.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f05577fe700 1 --2- 172.21.15.36:0/973022627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f054c004e70 0x7f054c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:02.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f05577fe700 1 -- 172.21.15.36:0/973022627 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0560143da0 con 0x7f054c004500 2022-06-17T12:55:02.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f05577fe700 1 -- 172.21.15.36:0/973022627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c00bd20 con 0x7f054c004e70 2022-06-17T12:55:02.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f05577fe700 1 -- 172.21.15.36:0/973022627 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054c00bee0 con 0x7f054c003930 2022-06-17T12:55:02.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f0557fff700 1 --2- 172.21.15.36:0/973022627 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f054c004500 0x7f054c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:02.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f0565502700 1 --2- 172.21.15.36:0/973022627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f054c004e70 0x7f054c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:02.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f0564d01700 1 --2- 172.21.15.36:0/973022627 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f054c003930 0x7f054c003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:02.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f0557fff700 1 -- 172.21.15.36:0/973022627 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f054c003930 msgr2=0x7f054c003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:02.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f0557fff700 1 --2- 172.21.15.36:0/973022627 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f054c003930 0x7f054c003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f0557fff700 1 -- 172.21.15.36:0/973022627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f054c004e70 msgr2=0x7f054c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:02.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f0557fff700 1 --2- 172.21.15.36:0/973022627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f054c004e70 0x7f054c00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f0557fff700 1 -- 172.21.15.36:0/973022627 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c003260 con 0x7f054c004500 2022-06-17T12:55:02.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f0565502700 1 --2- 172.21.15.36:0/973022627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f054c004e70 0x7f054c00b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:02.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f0565502700 1 -- 172.21.15.36:0/973022627 reap_dead start 2022-06-17T12:55:02.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f0557fff700 1 --2- 172.21.15.36:0/973022627 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f054c004500 0x7f054c004980 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 rx=0x7f0550001110 tx=0x7f0550007a00).ready entity=mon.0 client_cookie=e06e55a555bbe90a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:02.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f05577fe700 1 -- 172.21.15.36:0/973022627 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0550007d10 con 0x7f054c004500 2022-06-17T12:55:02.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f05577fe700 1 -- 172.21.15.36:0/973022627 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0550007030 con 0x7f054c004500 2022-06-17T12:55:02.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f05577fe700 1 -- 172.21.15.36:0/973022627 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0550007390 con 0x7f054c004500 2022-06-17T12:55:02.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f0565d03700 1 -- 172.21.15.36:0/973022627 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f054c004500 msgr2=0x7f054c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:02.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.707+0000 7f0565d03700 1 --2- 172.21.15.36:0/973022627 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f054c004500 0x7f054c004980 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 rx=0x7f0550001110 tx=0x7f0550007a00).stop 2022-06-17T12:55:02.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0565d03700 1 -- 172.21.15.36:0/973022627 shutdown_connections 2022-06-17T12:55:02.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0565d03700 1 --2- 172.21.15.36:0/973022627 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f054c004500 0x7f054c004980 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0565d03700 1 -- 172.21.15.36:0/973022627 >> 172.21.15.36:0/973022627 conn(0x7f056005c550 msgr2=0x7f05600b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:02.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0565d03700 1 -- 172.21.15.36:0/973022627 shutdown_connections 2022-06-17T12:55:02.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0565d03700 1 -- 172.21.15.36:0/973022627 wait complete. 2022-06-17T12:55:02.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0565d03700 1 Processor -- start 2022-06-17T12:55:02.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0565d03700 1 -- start start 2022-06-17T12:55:02.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0565d03700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f054c004500 0x7f05600af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:02.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0565d03700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0560141d80 0x7f056013f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:02.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0565d03700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0560155d30 0x7f05601561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:02.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0565d03700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05601566d0 con 0x7f0560141d80 2022-06-17T12:55:02.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0565d03700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05601425f0 con 0x7f0560155d30 2022-06-17T12:55:02.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0565d03700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05600afc70 con 0x7f054c004500 2022-06-17T12:55:02.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0557fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0560141d80 0x7f056013f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:02.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0557fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0560141d80 0x7f056013f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37792/0 (socket says 172.21.15.36:37792) 2022-06-17T12:55:02.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0557fff700 1 -- 172.21.15.36:0/2289608408 learned_addr learned my addr 172.21.15.36:0/2289608408 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:02.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0565502700 1 --2- 172.21.15.36:0/2289608408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0560155d30 0x7f05601561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:02.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0564d01700 1 --2- 172.21.15.36:0/2289608408 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f054c004500 0x7f05600af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:02.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0557fff700 1 -- 172.21.15.36:0/2289608408 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f054c004500 msgr2=0x7f05600af640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:02.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0557fff700 1 --2- 172.21.15.36:0/2289608408 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f054c004500 0x7f05600af640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0557fff700 1 -- 172.21.15.36:0/2289608408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0560155d30 msgr2=0x7f05601561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:02.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0557fff700 1 --2- 172.21.15.36:0/2289608408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0560155d30 0x7f05601561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0557fff700 1 -- 172.21.15.36:0/2289608408 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05600aff10 con 0x7f0560141d80 2022-06-17T12:55:02.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0564d01700 1 --2- 172.21.15.36:0/2289608408 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f054c004500 0x7f05600af640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:55:02.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.711+0000 7f0557fff700 1 --2- 172.21.15.36:0/2289608408 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0560141d80 0x7f056013f3c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f0550001110 tx=0x7f0550015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:02.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.715+0000 7f0555ffb700 1 -- 172.21.15.36:0/2289608408 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0550013850 con 0x7f0560141d80 2022-06-17T12:55:02.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.715+0000 7f0555ffb700 1 -- 172.21.15.36:0/2289608408 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05500139b0 con 0x7f0560141d80 2022-06-17T12:55:02.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.715+0000 7f0555ffb700 1 -- 172.21.15.36:0/2289608408 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0550013d10 con 0x7f0560141d80 2022-06-17T12:55:02.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.715+0000 7f0565d03700 1 -- 172.21.15.36:0/2289608408 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05600b01b0 con 0x7f0560141d80 2022-06-17T12:55:02.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.715+0000 7f0565d03700 1 -- 172.21.15.36:0/2289608408 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05600b1900 con 0x7f0560141d80 2022-06-17T12:55:02.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.715+0000 7f0565d03700 1 -- 172.21.15.36:0/2289608408 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05600b0340 con 0x7f0560141d80 2022-06-17T12:55:02.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.715+0000 7f0555ffb700 1 -- 172.21.15.36:0/2289608408 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f0550010070 con 0x7f0560141d80 2022-06-17T12:55:02.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.715+0000 7f0555ffb700 1 --2- 172.21.15.36:0/2289608408 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0540053a40 0x7f0540055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:02.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.715+0000 7f0555ffb700 1 -- 172.21.15.36:0/2289608408 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f055006cfb0 con 0x7f0560141d80 2022-06-17T12:55:02.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.715+0000 7f0564d01700 1 --2- 172.21.15.36:0/2289608408 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0540053a40 0x7f0540055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:02.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.719+0000 7f0564d01700 1 --2- 172.21.15.36:0/2289608408 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0540053a40 0x7f0540055f00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f055c006000 tx=0x7f055c001650).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:02.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.723+0000 7f0555ffb700 1 -- 172.21.15.36:0/2289608408 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0550043db0 con 0x7f0560141d80 2022-06-17T12:55:02.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.891+0000 7f0565d03700 1 -- 172.21.15.36:0/2289608408 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f05601431f0 con 0x7f0560141d80 2022-06-17T12:55:02.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.899+0000 7f0555ffb700 1 -- 172.21.15.36:0/2289608408 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v36) v1 ==== 184+0+0 (secure 0 0 0) 0x7f0550025030 con 0x7f0560141d80 2022-06-17T12:55:02.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.899+0000 7f0565d03700 1 -- 172.21.15.36:0/2289608408 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0540053a40 msgr2=0x7f0540055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:02.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.899+0000 7f0565d03700 1 --2- 172.21.15.36:0/2289608408 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0540053a40 0x7f0540055f00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f055c006000 tx=0x7f055c001650).stop 2022-06-17T12:55:02.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.899+0000 7f0565d03700 1 -- 172.21.15.36:0/2289608408 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0560141d80 msgr2=0x7f056013f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:02.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.899+0000 7f0565d03700 1 --2- 172.21.15.36:0/2289608408 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0560141d80 0x7f056013f3c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f0550001110 tx=0x7f0550015000).stop 2022-06-17T12:55:02.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.899+0000 7f0565d03700 1 -- 172.21.15.36:0/2289608408 shutdown_connections 2022-06-17T12:55:02.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.899+0000 7f0565d03700 1 --2- 172.21.15.36:0/2289608408 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0540053a40 0x7f0540055f00 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.899+0000 7f0565d03700 1 --2- 172.21.15.36:0/2289608408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0560155d30 0x7f05601561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.899+0000 7f0565d03700 1 --2- 172.21.15.36:0/2289608408 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0560141d80 0x7f056013f3c0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.899+0000 7f0565d03700 1 --2- 172.21.15.36:0/2289608408 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f054c004500 0x7f05600af640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:02.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.899+0000 7f0565d03700 1 -- 172.21.15.36:0/2289608408 >> 172.21.15.36:0/2289608408 conn(0x7f056005c550 msgr2=0x7f05600b74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:02.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.899+0000 7f0565d03700 1 -- 172.21.15.36:0/2289608408 shutdown_connections 2022-06-17T12:55:02.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:02.899+0000 7f0565d03700 1 -- 172.21.15.36:0/2289608408 wait complete. 2022-06-17T12:55:02.895 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.1 2022-06-17T12:55:02.903 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:55:03.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.035+0000 7fe31f59e700 1 Processor -- start 2022-06-17T12:55:03.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.035+0000 7fe31f59e700 1 -- start start 2022-06-17T12:55:03.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.035+0000 7fe31f59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe32013cb70 0x7fe320143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:03.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.035+0000 7fe31f59e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3200af4b0 con 0x7fe3200aef60 2022-06-17T12:55:03.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.035+0000 7fe31f59e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3200ba500 con 0x7fe32013cf50 2022-06-17T12:55:03.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.035+0000 7fe31f59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe320142f50 con 0x7fe32013cb70 2022-06-17T12:55:03.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.035+0000 7fe315d9b700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fe3200aef60 0x7fe3200b9df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55330/0 (socket says 172.21.15.36:55330) 2022-06-17T12:55:03.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.035+0000 7fe315d9b700 1 -- 172.21.15.36:0/847852678 learned_addr learned my addr 172.21.15.36:0/847852678 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:03.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.035+0000 7fe31e59c700 1 --2- 172.21.15.36:0/847852678 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe32013cb70 0x7fe320143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:03.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.035+0000 7fe31dd9b700 1 -- 172.21.15.36:0/847852678 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4153890225 0 0) 0x7fe3200af4b0 con 0x7fe3200aef60 2022-06-17T12:55:03.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.035+0000 7fe31dd9b700 1 -- 172.21.15.36:0/847852678 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe300003260 con 0x7fe3200aef60 2022-06-17T12:55:03.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.039+0000 7fe31dd9b700 1 -- 172.21.15.36:0/847852678 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 489366933 0 0) 0x7fe3200ba500 con 0x7fe32013cf50 2022-06-17T12:55:03.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.039+0000 7fe31dd9b700 1 -- 172.21.15.36:0/847852678 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3200af4b0 con 0x7fe32013cf50 2022-06-17T12:55:03.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.039+0000 7fe31e59c700 1 -- 172.21.15.36:0/847852678 >> v1:172.21.15.70:6789/0 conn(0x7fe32013cf50 legacy=0x7fe3201442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:03.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.039+0000 7fe31e59c700 1 -- 172.21.15.36:0/847852678 >> v1:172.21.15.36:6789/0 conn(0x7fe3200aef60 legacy=0x7fe3200b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:03.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.039+0000 7fe31dd9b700 1 -- 172.21.15.36:0/847852678 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2190821647 0 0) 0x7fe300003260 con 0x7fe3200aef60 2022-06-17T12:55:03.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.039+0000 7fe31e59c700 1 -- 172.21.15.36:0/847852678 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3201431f0 con 0x7fe32013cb70 2022-06-17T12:55:03.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.039+0000 7fe31e59c700 1 --2- 172.21.15.36:0/847852678 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe32013cb70 0x7fe320143690 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fe308001110 tx=0x7fe310006df0).ready entity=mon.2 client_cookie=53da908f152852f2 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:03.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.039+0000 7fe31dd9b700 1 -- 172.21.15.36:0/847852678 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe308001660 con 0x7fe32013cb70 2022-06-17T12:55:03.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.039+0000 7fe31dd9b700 1 -- 172.21.15.36:0/847852678 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe310000e90 con 0x7fe32013cb70 2022-06-17T12:55:03.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.039+0000 7fe31dd9b700 1 -- 172.21.15.36:0/847852678 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe310012460 con 0x7fe32013cb70 2022-06-17T12:55:03.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.039+0000 7fe31f59e700 1 -- 172.21.15.36:0/847852678 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe32013cb70 msgr2=0x7fe320143690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:03.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.039+0000 7fe31f59e700 1 --2- 172.21.15.36:0/847852678 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe32013cb70 0x7fe320143690 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fe308001110 tx=0x7fe310006df0).stop 2022-06-17T12:55:03.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.039+0000 7fe31f59e700 1 -- 172.21.15.36:0/847852678 shutdown_connections 2022-06-17T12:55:03.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.039+0000 7fe31f59e700 1 --2- 172.21.15.36:0/847852678 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe32013cb70 0x7fe320143690 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:03.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.039+0000 7fe31f59e700 1 -- 172.21.15.36:0/847852678 >> 172.21.15.36:0/847852678 conn(0x7fe32005c550 msgr2=0x7fe3200b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:03.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.039+0000 7fe31f59e700 1 -- 172.21.15.36:0/847852678 shutdown_connections 2022-06-17T12:55:03.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.039+0000 7fe31f59e700 1 -- 172.21.15.36:0/847852678 wait complete. 2022-06-17T12:55:03.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.039+0000 7fe31f59e700 1 Processor -- start 2022-06-17T12:55:03.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.039+0000 7fe31f59e700 1 -- start start 2022-06-17T12:55:03.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.043+0000 7fe31f59e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3200aef60 0x7fe320145400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:03.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.043+0000 7fe31f59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe32013cb70 0x7fe320156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:03.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.043+0000 7fe31f59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe32013cf50 0x7fe32013f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:03.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.043+0000 7fe31f59e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3201407a0 con 0x7fe32013cb70 2022-06-17T12:55:03.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.043+0000 7fe31f59e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe320140260 con 0x7fe3200aef60 2022-06-17T12:55:03.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.043+0000 7fe31f59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe320140500 con 0x7fe32013cf50 2022-06-17T12:55:03.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.043+0000 7fe31e59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3200aef60 0x7fe320145400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:03.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.043+0000 7fe315d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe32013cb70 0x7fe320156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:03.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.043+0000 7fe31e59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3200aef60 0x7fe320145400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:58186/0 (socket says 172.21.15.36:58186) 2022-06-17T12:55:03.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.043+0000 7fe31e59c700 1 -- 172.21.15.36:0/763725186 learned_addr learned my addr 172.21.15.36:0/763725186 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:03.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.043+0000 7fe31ed9d700 1 --2- 172.21.15.36:0/763725186 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe32013cf50 0x7fe32013f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:03.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.043+0000 7fe315d9b700 1 -- 172.21.15.36:0/763725186 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe32013cf50 msgr2=0x7fe32013f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:03.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.043+0000 7fe315d9b700 1 --2- 172.21.15.36:0/763725186 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe32013cf50 0x7fe32013f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:03.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.043+0000 7fe315d9b700 1 -- 172.21.15.36:0/763725186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3200aef60 msgr2=0x7fe320145400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:03.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.043+0000 7fe315d9b700 1 --2- 172.21.15.36:0/763725186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3200aef60 0x7fe320145400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:03.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.043+0000 7fe315d9b700 1 -- 172.21.15.36:0/763725186 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31000b180 con 0x7fe32013cb70 2022-06-17T12:55:03.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.043+0000 7fe315d9b700 1 --2- 172.21.15.36:0/763725186 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe32013cb70 0x7fe320156210 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 rx=0x7fe308001840 tx=0x7fe308001660).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:03.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.043+0000 7fe317fff700 1 -- 172.21.15.36:0/763725186 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe308007170 con 0x7fe32013cb70 2022-06-17T12:55:03.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.043+0000 7fe317fff700 1 -- 172.21.15.36:0/763725186 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3080072d0 con 0x7fe32013cb70 2022-06-17T12:55:03.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.043+0000 7fe31f59e700 1 -- 172.21.15.36:0/763725186 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe320156700 con 0x7fe32013cb70 2022-06-17T12:55:03.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.043+0000 7fe317fff700 1 -- 172.21.15.36:0/763725186 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe308004710 con 0x7fe32013cb70 2022-06-17T12:55:03.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.043+0000 7fe31f59e700 1 -- 172.21.15.36:0/763725186 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32013df40 con 0x7fe32013cb70 2022-06-17T12:55:03.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.047+0000 7fe31f59e700 1 -- 172.21.15.36:0/763725186 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe320152300 con 0x7fe32013cb70 2022-06-17T12:55:03.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.047+0000 7fe317fff700 1 -- 172.21.15.36:0/763725186 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe30801b070 con 0x7fe32013cb70 2022-06-17T12:55:03.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.047+0000 7fe317fff700 1 --2- 172.21.15.36:0/763725186 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe304053af0 0x7fe304055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:03.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.047+0000 7fe317fff700 1 -- 172.21.15.36:0/763725186 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe30806b710 con 0x7fe32013cb70 2022-06-17T12:55:03.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.047+0000 7fe31e59c700 1 --2- 172.21.15.36:0/763725186 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe304053af0 0x7fe304055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:03.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.047+0000 7fe31e59c700 1 --2- 172.21.15.36:0/763725186 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe304053af0 0x7fe304055fb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7fe31001a000 tx=0x7fe310014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:03.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.055+0000 7fe317fff700 1 -- 172.21.15.36:0/763725186 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe308042400 con 0x7fe32013cb70 2022-06-17T12:55:03.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.223+0000 7fe31f59e700 1 -- 172.21.15.36:0/763725186 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fe3201449b0 con 0x7fe32013cb70 2022-06-17T12:55:03.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.231+0000 7fe317fff700 1 -- 172.21.15.36:0/763725186 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v37) v1 ==== 184+0+0 (secure 0 0 0) 0x7fe308045a20 con 0x7fe32013cb70 2022-06-17T12:55:03.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.231+0000 7fe31f59e700 1 -- 172.21.15.36:0/763725186 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe304053af0 msgr2=0x7fe304055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:03.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.231+0000 7fe31f59e700 1 --2- 172.21.15.36:0/763725186 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe304053af0 0x7fe304055fb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7fe31001a000 tx=0x7fe310014000).stop 2022-06-17T12:55:03.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.231+0000 7fe31f59e700 1 -- 172.21.15.36:0/763725186 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe32013cb70 msgr2=0x7fe320156210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:03.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.231+0000 7fe31f59e700 1 --2- 172.21.15.36:0/763725186 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe32013cb70 0x7fe320156210 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 rx=0x7fe308001840 tx=0x7fe308001660).stop 2022-06-17T12:55:03.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.231+0000 7fe31f59e700 1 -- 172.21.15.36:0/763725186 shutdown_connections 2022-06-17T12:55:03.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.231+0000 7fe31f59e700 1 --2- 172.21.15.36:0/763725186 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe304053af0 0x7fe304055fb0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:03.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.231+0000 7fe31f59e700 1 --2- 172.21.15.36:0/763725186 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe32013cf50 0x7fe32013f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:03.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.231+0000 7fe31f59e700 1 --2- 172.21.15.36:0/763725186 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe32013cb70 0x7fe320156210 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:03.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.231+0000 7fe31f59e700 1 --2- 172.21.15.36:0/763725186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3200aef60 0x7fe320145400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:03.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.231+0000 7fe31f59e700 1 -- 172.21.15.36:0/763725186 >> 172.21.15.36:0/763725186 conn(0x7fe32005c550 msgr2=0x7fe320139fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:03.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.231+0000 7fe31f59e700 1 -- 172.21.15.36:0/763725186 shutdown_connections 2022-06-17T12:55:03.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.231+0000 7fe31f59e700 1 -- 172.21.15.36:0/763725186 wait complete. 2022-06-17T12:55:03.233 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.2 2022-06-17T12:55:03.235 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:55:03.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.359+0000 7fe4780e0700 1 Processor -- start 2022-06-17T12:55:03.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.359+0000 7fe4780e0700 1 -- start start 2022-06-17T12:55:03.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe4780e0700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe470062770 0x7fe470062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:03.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe4780e0700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe4701444f0 0x7fe470144930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:03.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe4780e0700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4700b0ba0 con 0x7fe4700ad330 2022-06-17T12:55:03.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe4780e0700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4700ae820 con 0x7fe4701444f0 2022-06-17T12:55:03.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe4780e0700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe470061030 con 0x7fe470062770 2022-06-17T12:55:03.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe47567b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe4701444f0 0x7fe470144930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:03.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe47567b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe4701444f0 0x7fe470144930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37812/0 (socket says 172.21.15.36:37812) 2022-06-17T12:55:03.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe47567b700 1 -- 172.21.15.36:0/3817427789 learned_addr learned my addr 172.21.15.36:0/3817427789 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:03.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe475e7c700 1 --2- 172.21.15.36:0/3817427789 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe470062770 0x7fe470062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:03.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe47567b700 1 -- 172.21.15.36:0/3817427789 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe470062770 msgr2=0x7fe470062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:03.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe47567b700 1 --2- 172.21.15.36:0/3817427789 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe470062770 0x7fe470062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:03.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe47567b700 1 -- 172.21.15.36:0/3817427789 >> v1:172.21.15.70:6789/0 conn(0x7fe4700ad330 legacy=0x7fe4700ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:03.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe47567b700 1 -- 172.21.15.36:0/3817427789 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe470147e90 con 0x7fe4701444f0 2022-06-17T12:55:03.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe47567b700 1 --2- 172.21.15.36:0/3817427789 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4701444f0 0x7fe470144930 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7fe470061170 tx=0x7fe464009290).ready entity=mon.0 client_cookie=afb523247b5a9779 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:03.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe474e7a700 1 -- 172.21.15.36:0/3817427789 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe46400b860 con 0x7fe4701444f0 2022-06-17T12:55:03.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe474e7a700 1 -- 172.21.15.36:0/3817427789 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46400b9c0 con 0x7fe4701444f0 2022-06-17T12:55:03.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe474e7a700 1 -- 172.21.15.36:0/3817427789 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe46400bb30 con 0x7fe4701444f0 2022-06-17T12:55:03.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe4780e0700 1 -- 172.21.15.36:0/3817427789 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4701444f0 msgr2=0x7fe470144930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:03.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe4780e0700 1 --2- 172.21.15.36:0/3817427789 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4701444f0 0x7fe470144930 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7fe470061170 tx=0x7fe464009290).stop 2022-06-17T12:55:03.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe4780e0700 1 -- 172.21.15.36:0/3817427789 shutdown_connections 2022-06-17T12:55:03.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe4780e0700 1 --2- 172.21.15.36:0/3817427789 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4701444f0 0x7fe470144930 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:03.366 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe4780e0700 1 --2- 172.21.15.36:0/3817427789 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe470062770 0x7fe470062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:03.366 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe4780e0700 1 -- 172.21.15.36:0/3817427789 >> 172.21.15.36:0/3817427789 conn(0x7fe4700b53e0 msgr2=0x7fe4700b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:03.366 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe4780e0700 1 -- 172.21.15.36:0/3817427789 shutdown_connections 2022-06-17T12:55:03.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.363+0000 7fe4780e0700 1 -- 172.21.15.36:0/3817427789 wait complete. 2022-06-17T12:55:03.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.367+0000 7fe4780e0700 1 Processor -- start 2022-06-17T12:55:03.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.367+0000 7fe4780e0700 1 -- start start 2022-06-17T12:55:03.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.367+0000 7fe4780e0700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe470062770 0x7fe4701574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:03.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.367+0000 7fe4780e0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4700ad330 0x7fe47014ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:03.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.367+0000 7fe4780e0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4701444f0 0x7fe4701453b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:03.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.367+0000 7fe4780e0700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe470149c50 con 0x7fe4701444f0 2022-06-17T12:55:03.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.367+0000 7fe4780e0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47014b2f0 con 0x7fe4700ad330 2022-06-17T12:55:03.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.367+0000 7fe4780e0700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe47014b530 con 0x7fe470062770 2022-06-17T12:55:03.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.367+0000 7fe47667d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4701444f0 0x7fe4701453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:03.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.367+0000 7fe47567b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4700ad330 0x7fe47014ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:03.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.367+0000 7fe475e7c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe470062770 0x7fe4701574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:03.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.367+0000 7fe47667d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4701444f0 0x7fe4701453b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37822/0 (socket says 172.21.15.36:37822) 2022-06-17T12:55:03.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.367+0000 7fe47667d700 1 -- 172.21.15.36:0/2425665434 learned_addr learned my addr 172.21.15.36:0/2425665434 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:03.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.371+0000 7fe47667d700 1 -- 172.21.15.36:0/2425665434 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe470062770 msgr2=0x7fe4701574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:03.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.371+0000 7fe47667d700 1 --2- 172.21.15.36:0/2425665434 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe470062770 0x7fe4701574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:03.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.371+0000 7fe47667d700 1 -- 172.21.15.36:0/2425665434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4700ad330 msgr2=0x7fe47014ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:03.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.371+0000 7fe47667d700 1 --2- 172.21.15.36:0/2425665434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4700ad330 0x7fe47014ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:03.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.371+0000 7fe47667d700 1 -- 172.21.15.36:0/2425665434 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe464009530 con 0x7fe4701444f0 2022-06-17T12:55:03.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.371+0000 7fe47667d700 1 --2- 172.21.15.36:0/2425665434 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4701444f0 0x7fe4701453b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7fe47005d270 tx=0x7fe468007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:03.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.371+0000 7fe46effd700 1 -- 172.21.15.36:0/2425665434 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe468012400 con 0x7fe4701444f0 2022-06-17T12:55:03.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.371+0000 7fe46effd700 1 -- 172.21.15.36:0/2425665434 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe468012560 con 0x7fe4701444f0 2022-06-17T12:55:03.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.371+0000 7fe4780e0700 1 -- 172.21.15.36:0/2425665434 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe47014a5d0 con 0x7fe4701444f0 2022-06-17T12:55:03.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.371+0000 7fe46effd700 1 -- 172.21.15.36:0/2425665434 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4680128c0 con 0x7fe4701444f0 2022-06-17T12:55:03.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.371+0000 7fe4780e0700 1 -- 172.21.15.36:0/2425665434 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe470149630 con 0x7fe4701444f0 2022-06-17T12:55:03.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.371+0000 7fe46effd700 1 -- 172.21.15.36:0/2425665434 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe468019030 con 0x7fe4701444f0 2022-06-17T12:55:03.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.371+0000 7fe4780e0700 1 -- 172.21.15.36:0/2425665434 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4700b1c20 con 0x7fe4701444f0 2022-06-17T12:55:03.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.371+0000 7fe46effd700 1 --2- 172.21.15.36:0/2425665434 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe450053ad0 0x7fe450055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:03.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.371+0000 7fe46effd700 1 -- 172.21.15.36:0/2425665434 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe46806b360 con 0x7fe4701444f0 2022-06-17T12:55:03.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.371+0000 7fe475e7c700 1 --2- 172.21.15.36:0/2425665434 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe450053ad0 0x7fe450055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:03.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.375+0000 7fe475e7c700 1 --2- 172.21.15.36:0/2425665434 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe450053ad0 0x7fe450055f90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7fe46000b000 tx=0x7fe460006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:03.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.379+0000 7fe46effd700 1 -- 172.21.15.36:0/2425665434 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe468042080 con 0x7fe4701444f0 2022-06-17T12:55:03.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.543+0000 7fe4780e0700 1 -- 172.21.15.36:0/2425665434 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe470057410 con 0x7fe4701444f0 2022-06-17T12:55:03.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.547+0000 7fe46effd700 1 -- 172.21.15.36:0/2425665434 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+2134 (secure 0 0 0) 0x7fe4680456a0 con 0x7fe4701444f0 2022-06-17T12:55:03.535 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:03.536 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":54,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/676934608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":676934608},{"type":"v1","addr":"172.21.15.70:6833","nonce":676934608}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:54:59.995522+0000","modified":"2022-06-17T12:55:01.664559+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7395":{"gid":7395,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/233328233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":233328233},{"type":"v1","addr":"172.21.15.36:6835","nonce":233328233}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-06-17T12:55:03.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.547+0000 7fe4780e0700 1 -- 172.21.15.36:0/2425665434 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe450053ad0 msgr2=0x7fe450055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:03.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.547+0000 7fe4780e0700 1 --2- 172.21.15.36:0/2425665434 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe450053ad0 0x7fe450055f90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7fe46000b000 tx=0x7fe460006df0).stop 2022-06-17T12:55:03.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.547+0000 7fe4780e0700 1 -- 172.21.15.36:0/2425665434 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4701444f0 msgr2=0x7fe4701453b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:03.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.547+0000 7fe4780e0700 1 --2- 172.21.15.36:0/2425665434 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4701444f0 0x7fe4701453b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7fe47005d270 tx=0x7fe468007bc0).stop 2022-06-17T12:55:03.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.547+0000 7fe4780e0700 1 -- 172.21.15.36:0/2425665434 shutdown_connections 2022-06-17T12:55:03.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.547+0000 7fe4780e0700 1 --2- 172.21.15.36:0/2425665434 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe450053ad0 0x7fe450055f90 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:03.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.547+0000 7fe4780e0700 1 --2- 172.21.15.36:0/2425665434 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4701444f0 0x7fe4701453b0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:03.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.547+0000 7fe4780e0700 1 --2- 172.21.15.36:0/2425665434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4700ad330 0x7fe47014ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:03.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.547+0000 7fe4780e0700 1 --2- 172.21.15.36:0/2425665434 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe470062770 0x7fe4701574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:03.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.547+0000 7fe4780e0700 1 -- 172.21.15.36:0/2425665434 >> 172.21.15.36:0/2425665434 conn(0x7fe4700b53e0 msgr2=0x7fe4700b4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:03.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.547+0000 7fe4780e0700 1 -- 172.21.15.36:0/2425665434 shutdown_connections 2022-06-17T12:55:03.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:03.547+0000 7fe4780e0700 1 -- 172.21.15.36:0/2425665434 wait complete. 2022-06-17T12:55:03.544 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 54 2022-06-17T12:55:03.755 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:54:59.995522+0000', 'modified': '2022-06-17T12:55:01.664559+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7395}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7395': {'gid': 7395, 'name': 'a', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.36:6835/233328233', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 233328233}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 233328233}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:55:03.756 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:55:03.757 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:55:03.879 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":7395,"want_state":"up:active","state":"up:active","rank_uptime":3.887137997,"mdsmap_epoch":54,"osdmap_epoch":104,"osdmap_epoch_barrier":104,"uptime":8.4461004430000006} 2022-06-17T12:55:03.880 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 7395, "mdsmap_epoch": 54, "osdmap_epoch": 104, "osdmap_epoch_barrier": 104, "rank_uptime": 3.887137997, "state": "up:active", "uptime": 8.446100443, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:55:03.881 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:55:04.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc363552700 1 Processor -- start 2022-06-17T12:55:04.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc363552700 1 -- start start 2022-06-17T12:55:04.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc363552700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc35c060340 0x7fc35c13ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc363552700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc35c13f100 0x7fc35c142aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc363552700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc35c0ad330 0x7fc35c142f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc363552700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc35c060720 con 0x7fc35c0ad330 2022-06-17T12:55:04.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc363552700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc35c062d30 con 0x7fc35c13f100 2022-06-17T12:55:04.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc363552700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c062fd0 con 0x7fc35c060340 2022-06-17T12:55:04.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc361aef700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc35c0ad330 0x7fc35c142f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc361aef700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc35c0ad330 0x7fc35c142f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37828/0 (socket says 172.21.15.36:37828) 2022-06-17T12:55:04.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc361aef700 1 -- 172.21.15.36:0/1124613493 learned_addr learned my addr 172.21.15.36:0/1124613493 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:04.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc3612ee700 1 --2- 172.21.15.36:0/1124613493 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc35c060340 0x7fc35c13ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc360aed700 1 --2- 172.21.15.36:0/1124613493 >> v2:172.21.15.70:3300/0 conn(0x7fc35c13f100 0x7fc35c142aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc361aef700 1 -- 172.21.15.36:0/1124613493 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc35c060340 msgr2=0x7fc35c13ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:04.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc361aef700 1 --2- 172.21.15.36:0/1124613493 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc35c060340 0x7fc35c13ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc361aef700 1 -- 172.21.15.36:0/1124613493 >> v2:172.21.15.70:3300/0 conn(0x7fc35c13f100 msgr2=0x7fc35c142aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:04.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc361aef700 1 --2- 172.21.15.36:0/1124613493 >> v2:172.21.15.70:3300/0 conn(0x7fc35c13f100 0x7fc35c142aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc361aef700 1 -- 172.21.15.36:0/1124613493 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c063710 con 0x7fc35c0ad330 2022-06-17T12:55:04.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc360aed700 1 --2- 172.21.15.36:0/1124613493 >> v2:172.21.15.70:3300/0 conn(0x7fc35c13f100 0x7fc35c142aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:55:04.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc361aef700 1 --2- 172.21.15.36:0/1124613493 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc35c0ad330 0x7fc35c142f90 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7fc354002f70 tx=0x7fc354006eb0).ready entity=mon.0 client_cookie=55ed892ee3de1353 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:04.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc35bfff700 1 -- 172.21.15.36:0/1124613493 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc35400d570 con 0x7fc35c0ad330 2022-06-17T12:55:04.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc35bfff700 1 -- 172.21.15.36:0/1124613493 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc354000e90 con 0x7fc35c0ad330 2022-06-17T12:55:04.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc35bfff700 1 -- 172.21.15.36:0/1124613493 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc35400d900 con 0x7fc35c0ad330 2022-06-17T12:55:04.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc363552700 1 -- 172.21.15.36:0/1124613493 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc35c0ad330 msgr2=0x7fc35c142f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:04.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc363552700 1 --2- 172.21.15.36:0/1124613493 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc35c0ad330 0x7fc35c142f90 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7fc354002f70 tx=0x7fc354006eb0).stop 2022-06-17T12:55:04.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc363552700 1 -- 172.21.15.36:0/1124613493 shutdown_connections 2022-06-17T12:55:04.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc363552700 1 --2- 172.21.15.36:0/1124613493 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc35c0ad330 0x7fc35c142f90 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc363552700 1 --2- 172.21.15.36:0/1124613493 >> v2:172.21.15.70:3300/0 conn(0x7fc35c13f100 0x7fc35c142aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc363552700 1 --2- 172.21.15.36:0/1124613493 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc35c060340 0x7fc35c13ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc363552700 1 -- 172.21.15.36:0/1124613493 >> 172.21.15.36:0/1124613493 conn(0x7fc35c0b53e0 msgr2=0x7fc35c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:04.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc363552700 1 -- 172.21.15.36:0/1124613493 shutdown_connections 2022-06-17T12:55:04.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.007+0000 7fc363552700 1 -- 172.21.15.36:0/1124613493 wait complete. 2022-06-17T12:55:04.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc363552700 1 Processor -- start 2022-06-17T12:55:04.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc363552700 1 -- start start 2022-06-17T12:55:04.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc363552700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc35c060340 0x7fc35c142df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc363552700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc35c0ad330 0x7fc35c0639a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc363552700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc35c13f100 0x7fc35c066360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc363552700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c13eef0 con 0x7fc35c0ad330 2022-06-17T12:55:04.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc363552700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c139260 con 0x7fc35c13f100 2022-06-17T12:55:04.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc363552700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c139450 con 0x7fc35c060340 2022-06-17T12:55:04.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc360aed700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc35c0ad330 0x7fc35c0639a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc360aed700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc35c0ad330 0x7fc35c0639a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37834/0 (socket says 172.21.15.36:37834) 2022-06-17T12:55:04.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc360aed700 1 -- 172.21.15.36:0/883664195 learned_addr learned my addr 172.21.15.36:0/883664195 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:04.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc3612ee700 1 --2- 172.21.15.36:0/883664195 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc35c060340 0x7fc35c142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc361aef700 1 --2- 172.21.15.36:0/883664195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc35c13f100 0x7fc35c066360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc360aed700 1 -- 172.21.15.36:0/883664195 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc35c060340 msgr2=0x7fc35c142df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:04.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc360aed700 1 --2- 172.21.15.36:0/883664195 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc35c060340 0x7fc35c142df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc360aed700 1 -- 172.21.15.36:0/883664195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc35c13f100 msgr2=0x7fc35c066360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:04.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc360aed700 1 --2- 172.21.15.36:0/883664195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc35c13f100 0x7fc35c066360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc360aed700 1 -- 172.21.15.36:0/883664195 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35400b1e0 con 0x7fc35c0ad330 2022-06-17T12:55:04.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc360aed700 1 --2- 172.21.15.36:0/883664195 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc35c0ad330 0x7fc35c0639a0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 rx=0x7fc344007000 tx=0x7fc344008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:04.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc35a7fc700 1 -- 172.21.15.36:0/883664195 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc344010580 con 0x7fc35c0ad330 2022-06-17T12:55:04.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc35a7fc700 1 -- 172.21.15.36:0/883664195 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3440106e0 con 0x7fc35c0ad330 2022-06-17T12:55:04.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc363552700 1 -- 172.21.15.36:0/883664195 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc35c065ec0 con 0x7fc35c0ad330 2022-06-17T12:55:04.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc35a7fc700 1 -- 172.21.15.36:0/883664195 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc344010a40 con 0x7fc35c0ad330 2022-06-17T12:55:04.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc363552700 1 -- 172.21.15.36:0/883664195 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc35c064030 con 0x7fc35c0ad330 2022-06-17T12:55:04.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.011+0000 7fc363552700 1 -- 172.21.15.36:0/883664195 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc35c066050 con 0x7fc35c0ad330 2022-06-17T12:55:04.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.015+0000 7fc35a7fc700 1 -- 172.21.15.36:0/883664195 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fc344010ba0 con 0x7fc35c0ad330 2022-06-17T12:55:04.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.015+0000 7fc35a7fc700 1 --2- 172.21.15.36:0/883664195 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc340053a60 0x7fc340055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.015+0000 7fc35a7fc700 1 -- 172.21.15.36:0/883664195 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc34406bea0 con 0x7fc35c0ad330 2022-06-17T12:55:04.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.015+0000 7fc3612ee700 1 --2- 172.21.15.36:0/883664195 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc340053a60 0x7fc340055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.015+0000 7fc3612ee700 1 --2- 172.21.15.36:0/883664195 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc340053a60 0x7fc340055f20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7fc34c00d000 tx=0x7fc34c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:04.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.019+0000 7fc35a7fc700 1 -- 172.21.15.36:0/883664195 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc344042ba0 con 0x7fc35c0ad330 2022-06-17T12:55:04.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.143+0000 7fc363552700 1 -- 172.21.15.36:0/883664195 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc35c057410 con 0x7fc35c0ad330 2022-06-17T12:55:04.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.147+0000 7fc35a7fc700 1 -- 172.21.15.36:0/883664195 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+16285 (secure 0 0 0) 0x7fc3440461c0 con 0x7fc35c0ad330 2022-06-17T12:55:04.135 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:04.135 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":104,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:55:00.998590+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:54:58.393546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-06-17T12:54:58.997835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:55:04.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.147+0000 7fc363552700 1 -- 172.21.15.36:0/883664195 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc340053a60 msgr2=0x7fc340055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:04.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.147+0000 7fc363552700 1 --2- 172.21.15.36:0/883664195 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc340053a60 0x7fc340055f20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7fc34c00d000 tx=0x7fc34c008040).stop 2022-06-17T12:55:04.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.147+0000 7fc363552700 1 -- 172.21.15.36:0/883664195 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc35c0ad330 msgr2=0x7fc35c0639a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:04.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.147+0000 7fc363552700 1 --2- 172.21.15.36:0/883664195 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc35c0ad330 0x7fc35c0639a0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 rx=0x7fc344007000 tx=0x7fc344008040).stop 2022-06-17T12:55:04.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.147+0000 7fc363552700 1 -- 172.21.15.36:0/883664195 shutdown_connections 2022-06-17T12:55:04.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.147+0000 7fc363552700 1 --2- 172.21.15.36:0/883664195 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc340053a60 0x7fc340055f20 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.147+0000 7fc363552700 1 --2- 172.21.15.36:0/883664195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc35c13f100 0x7fc35c066360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.147+0000 7fc363552700 1 --2- 172.21.15.36:0/883664195 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc35c0ad330 0x7fc35c0639a0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.147+0000 7fc363552700 1 --2- 172.21.15.36:0/883664195 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc35c060340 0x7fc35c142df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.147+0000 7fc363552700 1 -- 172.21.15.36:0/883664195 >> 172.21.15.36:0/883664195 conn(0x7fc35c0b53e0 msgr2=0x7fc35c0b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:04.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.147+0000 7fc363552700 1 -- 172.21.15.36:0/883664195 shutdown_connections 2022-06-17T12:55:04.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.147+0000 7fc363552700 1 -- 172.21.15.36:0/883664195 wait complete. 2022-06-17T12:55:04.150 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:54:58.393546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:54:58.997835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:55:04.151 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:55:04.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.279+0000 7fcb5ad0d700 1 Processor -- start 2022-06-17T12:55:04.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.279+0000 7fcb5ad0d700 1 -- start start 2022-06-17T12:55:04.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.279+0000 7fcb5ad0d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcb540b9860 0x7fcb54143840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.279+0000 7fcb5ad0d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb540b45c0 0x7fcb540b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.279+0000 7fcb5ad0d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcb540b4f00 0x7fcb54146490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.279+0000 7fcb5ad0d700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb540b4160 con 0x7fcb540b4f00 2022-06-17T12:55:04.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.279+0000 7fcb5ad0d700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb540b9c40 con 0x7fcb540b9860 2022-06-17T12:55:04.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.279+0000 7fcb5ad0d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb540627d0 con 0x7fcb540b45c0 2022-06-17T12:55:04.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.279+0000 7fcb5a50c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcb540b4f00 0x7fcb54146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.279+0000 7fcb5a50c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcb540b4f00 0x7fcb54146490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37844/0 (socket says 172.21.15.36:37844) 2022-06-17T12:55:04.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.279+0000 7fcb5a50c700 1 -- 172.21.15.36:0/823837620 learned_addr learned my addr 172.21.15.36:0/823837620 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:04.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.279+0000 7fcb59d0b700 1 --2- 172.21.15.36:0/823837620 >> v2:172.21.15.70:3300/0 conn(0x7fcb540b9860 0x7fcb54143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.279+0000 7fcb5950a700 1 --2- 172.21.15.36:0/823837620 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb540b45c0 0x7fcb540b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.279+0000 7fcb5a50c700 1 -- 172.21.15.36:0/823837620 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb540b45c0 msgr2=0x7fcb540b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:04.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.279+0000 7fcb5a50c700 1 --2- 172.21.15.36:0/823837620 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb540b45c0 0x7fcb540b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.279+0000 7fcb5a50c700 1 -- 172.21.15.36:0/823837620 >> v2:172.21.15.70:3300/0 conn(0x7fcb540b9860 msgr2=0x7fcb54143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:04.278 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.279+0000 7fcb5a50c700 1 --2- 172.21.15.36:0/823837620 >> v2:172.21.15.70:3300/0 conn(0x7fcb540b9860 0x7fcb54143840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.278 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.279+0000 7fcb5a50c700 1 -- 172.21.15.36:0/823837620 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb54062a70 con 0x7fcb540b4f00 2022-06-17T12:55:04.279 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.279+0000 7fcb5a50c700 1 --2- 172.21.15.36:0/823837620 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb540b4f00 0x7fcb54146490 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 rx=0x7fcb4c002f70 tx=0x7fcb4c006eb0).ready entity=mon.0 client_cookie=5999e37d25e65d49 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:04.279 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.279+0000 7fcb58d09700 1 -- 172.21.15.36:0/823837620 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb4c00d570 con 0x7fcb540b4f00 2022-06-17T12:55:04.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.279+0000 7fcb58d09700 1 -- 172.21.15.36:0/823837620 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c000e90 con 0x7fcb540b4f00 2022-06-17T12:55:04.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.279+0000 7fcb58d09700 1 -- 172.21.15.36:0/823837620 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb4c00d900 con 0x7fcb540b4f00 2022-06-17T12:55:04.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.283+0000 7fcb5ad0d700 1 -- 172.21.15.36:0/823837620 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb540b4f00 msgr2=0x7fcb54146490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:04.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.283+0000 7fcb5ad0d700 1 --2- 172.21.15.36:0/823837620 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb540b4f00 0x7fcb54146490 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 rx=0x7fcb4c002f70 tx=0x7fcb4c006eb0).stop 2022-06-17T12:55:04.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.283+0000 7fcb5ad0d700 1 -- 172.21.15.36:0/823837620 shutdown_connections 2022-06-17T12:55:04.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.283+0000 7fcb5ad0d700 1 --2- 172.21.15.36:0/823837620 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb540b4f00 0x7fcb54146490 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.283+0000 7fcb5ad0d700 1 --2- 172.21.15.36:0/823837620 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb540b45c0 0x7fcb540b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.283+0000 7fcb5ad0d700 1 --2- 172.21.15.36:0/823837620 >> v2:172.21.15.70:3300/0 conn(0x7fcb540b9860 0x7fcb54143840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.283+0000 7fcb5ad0d700 1 -- 172.21.15.36:0/823837620 >> 172.21.15.36:0/823837620 conn(0x7fcb5405c550 msgr2=0x7fcb54060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:04.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.283+0000 7fcb5ad0d700 1 -- 172.21.15.36:0/823837620 shutdown_connections 2022-06-17T12:55:04.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.283+0000 7fcb5ad0d700 1 -- 172.21.15.36:0/823837620 wait complete. 2022-06-17T12:55:04.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.283+0000 7fcb5ad0d700 1 Processor -- start 2022-06-17T12:55:04.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.283+0000 7fcb5ad0d700 1 -- start start 2022-06-17T12:55:04.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.287+0000 7fcb5ad0d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb540b45c0 0x7fcb5412ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.287+0000 7fcb5ad0d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb540b4f00 0x7fcb541557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.287+0000 7fcb5ad0d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb540b9860 0x7fcb54155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.287+0000 7fcb5ad0d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5413f860 con 0x7fcb540b4f00 2022-06-17T12:55:04.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.287+0000 7fcb5ad0d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb540aef40 con 0x7fcb540b45c0 2022-06-17T12:55:04.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.287+0000 7fcb5ad0d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5412d670 con 0x7fcb540b9860 2022-06-17T12:55:04.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.287+0000 7fcb5950a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb540b4f00 0x7fcb541557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.287+0000 7fcb5950a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb540b4f00 0x7fcb541557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37848/0 (socket says 172.21.15.36:37848) 2022-06-17T12:55:04.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.287+0000 7fcb5950a700 1 -- 172.21.15.36:0/753442961 learned_addr learned my addr 172.21.15.36:0/753442961 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:04.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.287+0000 7fcb59d0b700 1 --2- 172.21.15.36:0/753442961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb540b45c0 0x7fcb5412ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.287+0000 7fcb5a50c700 1 --2- 172.21.15.36:0/753442961 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb540b9860 0x7fcb54155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.287+0000 7fcb5950a700 1 -- 172.21.15.36:0/753442961 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb540b9860 msgr2=0x7fcb54155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:04.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.287+0000 7fcb5950a700 1 --2- 172.21.15.36:0/753442961 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb540b9860 0x7fcb54155ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.287+0000 7fcb5950a700 1 -- 172.21.15.36:0/753442961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb540b45c0 msgr2=0x7fcb5412ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:04.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.287+0000 7fcb5950a700 1 --2- 172.21.15.36:0/753442961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb540b45c0 0x7fcb5412ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.287+0000 7fcb5950a700 1 -- 172.21.15.36:0/753442961 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4c00b1e0 con 0x7fcb540b4f00 2022-06-17T12:55:04.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.287+0000 7fcb59d0b700 1 --2- 172.21.15.36:0/753442961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb540b45c0 0x7fcb5412ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:04.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.287+0000 7fcb5950a700 1 --2- 172.21.15.36:0/753442961 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb540b4f00 0x7fcb541557b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0x7fcb4400c000 tx=0x7fcb44006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:04.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.287+0000 7fcb42ffd700 1 -- 172.21.15.36:0/753442961 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb4400c830 con 0x7fcb540b4f00 2022-06-17T12:55:04.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.287+0000 7fcb42ffd700 1 -- 172.21.15.36:0/753442961 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4400c990 con 0x7fcb540b4f00 2022-06-17T12:55:04.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.287+0000 7fcb5ad0d700 1 -- 172.21.15.36:0/753442961 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb5412d910 con 0x7fcb540b4f00 2022-06-17T12:55:04.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.287+0000 7fcb42ffd700 1 -- 172.21.15.36:0/753442961 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb4400ccf0 con 0x7fcb540b4f00 2022-06-17T12:55:04.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.287+0000 7fcb5ad0d700 1 -- 172.21.15.36:0/753442961 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5412d1e0 con 0x7fcb540b4f00 2022-06-17T12:55:04.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.291+0000 7fcb5ad0d700 1 -- 172.21.15.36:0/753442961 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb5412d490 con 0x7fcb540b4f00 2022-06-17T12:55:04.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.291+0000 7fcb42ffd700 1 -- 172.21.15.36:0/753442961 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fcb4401c460 con 0x7fcb540b4f00 2022-06-17T12:55:04.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.291+0000 7fcb42ffd700 1 --2- 172.21.15.36:0/753442961 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcb34053ad0 0x7fcb34055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.291+0000 7fcb42ffd700 1 -- 172.21.15.36:0/753442961 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcb4406c490 con 0x7fcb540b4f00 2022-06-17T12:55:04.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.291+0000 7fcb59d0b700 1 --2- 172.21.15.36:0/753442961 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcb34053ad0 0x7fcb34055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.291+0000 7fcb59d0b700 1 --2- 172.21.15.36:0/753442961 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcb34053ad0 0x7fcb34055f90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7fcb5000b000 tx=0x7fcb50008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:04.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.299+0000 7fcb42ffd700 1 -- 172.21.15.36:0/753442961 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcb44043190 con 0x7fcb540b4f00 2022-06-17T12:55:04.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.423+0000 7fcb5ad0d700 1 -- 172.21.15.36:0/753442961 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fcb54062eb0 con 0x7fcb540b4f00 2022-06-17T12:55:04.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.423+0000 7fcb42ffd700 1 -- 172.21.15.36:0/753442961 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v104) v1 ==== 110+0+10 (secure 0 0 0) 0x7fcb440467b0 con 0x7fcb540b4f00 2022-06-17T12:55:04.411 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:55:04.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.423+0000 7fcb5ad0d700 1 -- 172.21.15.36:0/753442961 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcb34053ad0 msgr2=0x7fcb34055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:04.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.423+0000 7fcb5ad0d700 1 --2- 172.21.15.36:0/753442961 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcb34053ad0 0x7fcb34055f90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7fcb5000b000 tx=0x7fcb50008040).stop 2022-06-17T12:55:04.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.423+0000 7fcb5ad0d700 1 -- 172.21.15.36:0/753442961 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb540b4f00 msgr2=0x7fcb541557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:04.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.423+0000 7fcb5ad0d700 1 --2- 172.21.15.36:0/753442961 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb540b4f00 0x7fcb541557b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0x7fcb4400c000 tx=0x7fcb44006df0).stop 2022-06-17T12:55:04.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.423+0000 7fcb5ad0d700 1 -- 172.21.15.36:0/753442961 shutdown_connections 2022-06-17T12:55:04.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.423+0000 7fcb5ad0d700 1 --2- 172.21.15.36:0/753442961 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcb34053ad0 0x7fcb34055f90 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.423+0000 7fcb5ad0d700 1 --2- 172.21.15.36:0/753442961 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb540b9860 0x7fcb54155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.423+0000 7fcb5ad0d700 1 --2- 172.21.15.36:0/753442961 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb540b4f00 0x7fcb541557b0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.423+0000 7fcb5ad0d700 1 --2- 172.21.15.36:0/753442961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb540b45c0 0x7fcb5412ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.423+0000 7fcb5ad0d700 1 -- 172.21.15.36:0/753442961 >> 172.21.15.36:0/753442961 conn(0x7fcb5405c550 msgr2=0x7fcb540b2880 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:04.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.423+0000 7fcb5ad0d700 1 -- 172.21.15.36:0/753442961 shutdown_connections 2022-06-17T12:55:04.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.423+0000 7fcb5ad0d700 1 -- 172.21.15.36:0/753442961 wait complete. 2022-06-17T12:55:04.426 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:55:04.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.547+0000 7fe6171fc700 1 Processor -- start 2022-06-17T12:55:04.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.547+0000 7fe6171fc700 1 -- start start 2022-06-17T12:55:04.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe6171fc700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe610062270 0x7fe6101481f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe6171fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe61014b780 0x7fe610146960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe6171fc700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe610062650 con 0x7fe6100b4190 2022-06-17T12:55:04.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe6171fc700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe610147810 con 0x7fe61014b780 2022-06-17T12:55:04.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe6171fc700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe610147ab0 con 0x7fe610062270 2022-06-17T12:55:04.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe615799700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe61014b780 0x7fe610146960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe615799700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe61014b780 0x7fe610146960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37858/0 (socket says 172.21.15.36:37858) 2022-06-17T12:55:04.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe615799700 1 -- 172.21.15.36:0/714371395 learned_addr learned my addr 172.21.15.36:0/714371395 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:04.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe614f98700 1 --2- 172.21.15.36:0/714371395 >> v2:172.21.15.70:3300/0 conn(0x7fe610062270 0x7fe6101481f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe60f7fe700 1 -- 172.21.15.36:0/714371395 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3447606141 0 0) 0x7fe610062650 con 0x7fe6100b4190 2022-06-17T12:55:04.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe615799700 1 -- 172.21.15.36:0/714371395 >> v2:172.21.15.70:3300/0 conn(0x7fe610062270 msgr2=0x7fe6101481f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:04.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe615799700 1 --2- 172.21.15.36:0/714371395 >> v2:172.21.15.70:3300/0 conn(0x7fe610062270 0x7fe6101481f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe615799700 1 -- 172.21.15.36:0/714371395 >> v1:172.21.15.70:6789/0 conn(0x7fe6100b4190 legacy=0x7fe6100b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:04.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe615799700 1 -- 172.21.15.36:0/714371395 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe610147d50 con 0x7fe61014b780 2022-06-17T12:55:04.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe615799700 1 --2- 172.21.15.36:0/714371395 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe61014b780 0x7fe610146960 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0x7fe6080032f0 tx=0x7fe608009290).ready entity=mon.0 client_cookie=ae5f4351a6feea1c server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:04.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe60f7fe700 1 -- 172.21.15.36:0/714371395 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe60800b870 con 0x7fe61014b780 2022-06-17T12:55:04.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe60f7fe700 1 -- 172.21.15.36:0/714371395 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60800b9d0 con 0x7fe61014b780 2022-06-17T12:55:04.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe60f7fe700 1 -- 172.21.15.36:0/714371395 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe60800bb40 con 0x7fe61014b780 2022-06-17T12:55:04.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe6171fc700 1 -- 172.21.15.36:0/714371395 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe61014b780 msgr2=0x7fe610146960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:04.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe6171fc700 1 --2- 172.21.15.36:0/714371395 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe61014b780 0x7fe610146960 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0x7fe6080032f0 tx=0x7fe608009290).stop 2022-06-17T12:55:04.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe6171fc700 1 -- 172.21.15.36:0/714371395 shutdown_connections 2022-06-17T12:55:04.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe6171fc700 1 --2- 172.21.15.36:0/714371395 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe61014b780 0x7fe610146960 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe6171fc700 1 --2- 172.21.15.36:0/714371395 >> v2:172.21.15.70:3300/0 conn(0x7fe610062270 0x7fe6101481f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe6171fc700 1 -- 172.21.15.36:0/714371395 >> 172.21.15.36:0/714371395 conn(0x7fe6100b97f0 msgr2=0x7fe6100b9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:04.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.551+0000 7fe6171fc700 1 -- 172.21.15.36:0/714371395 shutdown_connections 2022-06-17T12:55:04.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.555+0000 7fe6171fc700 1 -- 172.21.15.36:0/714371395 wait complete. 2022-06-17T12:55:04.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.555+0000 7fe6171fc700 1 Processor -- start 2022-06-17T12:55:04.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.555+0000 7fe6171fc700 1 -- start start 2022-06-17T12:55:04.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.555+0000 7fe6171fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe610062270 0x7fe6101567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.555+0000 7fe6171fc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6100b4190 0x7fe6101411a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.555+0000 7fe6171fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe61014b780 0x7fe61013fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.555+0000 7fe6171fc700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61013e5e0 con 0x7fe610062270 2022-06-17T12:55:04.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.555+0000 7fe6171fc700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe610157780 con 0x7fe61014b780 2022-06-17T12:55:04.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.555+0000 7fe6171fc700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe610156da0 con 0x7fe6100b4190 2022-06-17T12:55:04.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.555+0000 7fe614f98700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe610062270 0x7fe6101567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.555+0000 7fe60ffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6100b4190 0x7fe6101411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.555+0000 7fe614f98700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe610062270 0x7fe6101567b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37862/0 (socket says 172.21.15.36:37862) 2022-06-17T12:55:04.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.555+0000 7fe614f98700 1 -- 172.21.15.36:0/1479574425 learned_addr learned my addr 172.21.15.36:0/1479574425 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:04.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.555+0000 7fe615799700 1 --2- 172.21.15.36:0/1479574425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe61014b780 0x7fe61013fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.559+0000 7fe614f98700 1 -- 172.21.15.36:0/1479574425 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6100b4190 msgr2=0x7fe6101411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:04.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.559+0000 7fe614f98700 1 --2- 172.21.15.36:0/1479574425 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6100b4190 0x7fe6101411a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.559+0000 7fe614f98700 1 -- 172.21.15.36:0/1479574425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe61014b780 msgr2=0x7fe61013fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:04.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.559+0000 7fe614f98700 1 --2- 172.21.15.36:0/1479574425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe61014b780 0x7fe61013fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.559+0000 7fe614f98700 1 -- 172.21.15.36:0/1479574425 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe608009530 con 0x7fe610062270 2022-06-17T12:55:04.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.559+0000 7fe614f98700 1 --2- 172.21.15.36:0/1479574425 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe610062270 0x7fe6101567b0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7fe60400c000 tx=0x7fe604006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:04.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.559+0000 7fe60dffb700 1 -- 172.21.15.36:0/1479574425 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe604010780 con 0x7fe610062270 2022-06-17T12:55:04.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.559+0000 7fe60dffb700 1 -- 172.21.15.36:0/1479574425 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6040108e0 con 0x7fe610062270 2022-06-17T12:55:04.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.559+0000 7fe60dffb700 1 -- 172.21.15.36:0/1479574425 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe604010c40 con 0x7fe610062270 2022-06-17T12:55:04.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.559+0000 7fe6171fc700 1 -- 172.21.15.36:0/1479574425 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6101570a0 con 0x7fe610062270 2022-06-17T12:55:04.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.559+0000 7fe6171fc700 1 -- 172.21.15.36:0/1479574425 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6100b1c60 con 0x7fe610062270 2022-06-17T12:55:04.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.559+0000 7fe6171fc700 1 -- 172.21.15.36:0/1479574425 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6101486e0 con 0x7fe610062270 2022-06-17T12:55:04.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.559+0000 7fe60dffb700 1 -- 172.21.15.36:0/1479574425 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe60401c460 con 0x7fe610062270 2022-06-17T12:55:04.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.559+0000 7fe60dffb700 1 --2- 172.21.15.36:0/1479574425 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe5f0053af0 0x7fe5f0055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.559+0000 7fe60dffb700 1 -- 172.21.15.36:0/1479574425 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe60406c230 con 0x7fe610062270 2022-06-17T12:55:04.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.559+0000 7fe60ffff700 1 --2- 172.21.15.36:0/1479574425 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe5f0053af0 0x7fe5f0055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.563+0000 7fe60ffff700 1 --2- 172.21.15.36:0/1479574425 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe5f0053af0 0x7fe5f0055fb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7fe5fc00d000 tx=0x7fe5fc0079b0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:04.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.567+0000 7fe60dffb700 1 -- 172.21.15.36:0/1479574425 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe604042ec0 con 0x7fe610062270 2022-06-17T12:55:04.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.695+0000 7fe6171fc700 1 -- 172.21.15.36:0/1479574425 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe610057410 con 0x7fe610062270 2022-06-17T12:55:04.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.695+0000 7fe60dffb700 1 -- 172.21.15.36:0/1479574425 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v104) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe6040464e0 con 0x7fe610062270 2022-06-17T12:55:04.687 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:55:04.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.699+0000 7fe6171fc700 1 -- 172.21.15.36:0/1479574425 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe5f0053af0 msgr2=0x7fe5f0055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:04.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.699+0000 7fe6171fc700 1 --2- 172.21.15.36:0/1479574425 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe5f0053af0 0x7fe5f0055fb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7fe5fc00d000 tx=0x7fe5fc0079b0).stop 2022-06-17T12:55:04.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.699+0000 7fe6171fc700 1 -- 172.21.15.36:0/1479574425 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe610062270 msgr2=0x7fe6101567b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:04.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.699+0000 7fe6171fc700 1 --2- 172.21.15.36:0/1479574425 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe610062270 0x7fe6101567b0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7fe60400c000 tx=0x7fe604006df0).stop 2022-06-17T12:55:04.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.699+0000 7fe6171fc700 1 -- 172.21.15.36:0/1479574425 shutdown_connections 2022-06-17T12:55:04.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.699+0000 7fe6171fc700 1 --2- 172.21.15.36:0/1479574425 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe5f0053af0 0x7fe5f0055fb0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.699+0000 7fe6171fc700 1 --2- 172.21.15.36:0/1479574425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe61014b780 0x7fe61013fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.699+0000 7fe6171fc700 1 --2- 172.21.15.36:0/1479574425 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6100b4190 0x7fe6101411a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.699+0000 7fe6171fc700 1 --2- 172.21.15.36:0/1479574425 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe610062270 0x7fe6101567b0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.699+0000 7fe6171fc700 1 -- 172.21.15.36:0/1479574425 >> 172.21.15.36:0/1479574425 conn(0x7fe6100b97f0 msgr2=0x7fe610143450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:04.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.699+0000 7fe6171fc700 1 -- 172.21.15.36:0/1479574425 shutdown_connections 2022-06-17T12:55:04.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.699+0000 7fe6171fc700 1 -- 172.21.15.36:0/1479574425 wait complete. 2022-06-17T12:55:04.702 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:55:04.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.831+0000 7fbfd90e7700 1 Processor -- start 2022-06-17T12:55:04.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.835+0000 7fbfd90e7700 1 -- start start 2022-06-17T12:55:04.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.835+0000 7fbfd90e7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbfd40b7bc0 0x7fbfd4140e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.835+0000 7fbfd90e7700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbfd40ae150 0x7fbfd40b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.835+0000 7fbfd90e7700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfd40b7fa0 con 0x7fbfd40b2fc0 2022-06-17T12:55:04.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.835+0000 7fbfd90e7700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd40b20f0 con 0x7fbfd40ae150 2022-06-17T12:55:04.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.835+0000 7fbfd90e7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40b2390 con 0x7fbfd40b7bc0 2022-06-17T12:55:04.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.835+0000 7fbfd2d9d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbfd40b7bc0 0x7fbfd4140e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.835+0000 7fbfca59c700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbfd40ae150 0x7fbfd40b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.835+0000 7fbfd359e700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbfd40b2fc0 0x7fbfd4144450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:51136/0 (socket says 172.21.15.36:51136) 2022-06-17T12:55:04.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.835+0000 7fbfd359e700 1 -- 172.21.15.36:0/1702896428 learned_addr learned my addr 172.21.15.36:0/1702896428 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:04.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.835+0000 7fbfca59c700 1 -- 172.21.15.36:0/1702896428 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbfd40b7bc0 msgr2=0x7fbfd4140e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:04.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.835+0000 7fbfca59c700 1 --2- 172.21.15.36:0/1702896428 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbfd40b7bc0 0x7fbfd4140e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.835+0000 7fbfca59c700 1 -- 172.21.15.36:0/1702896428 >> v1:172.21.15.70:6789/0 conn(0x7fbfd40b2fc0 legacy=0x7fbfd4144450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:04.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.835+0000 7fbfca59c700 1 -- 172.21.15.36:0/1702896428 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd40b2630 con 0x7fbfd40ae150 2022-06-17T12:55:04.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.835+0000 7fbfd259c700 1 -- 172.21.15.36:0/1702896428 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1138329187 0 0) 0x7fbfd40b7fa0 con 0x7fbfd40b2fc0 2022-06-17T12:55:04.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.835+0000 7fbfd2d9d700 1 --2- 172.21.15.36:0/1702896428 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbfd40b7bc0 0x7fbfd4140e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:04.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.835+0000 7fbfca59c700 1 --2- 172.21.15.36:0/1702896428 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfd40ae150 0x7fbfd40b2ad0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7fbfbc002f70 tx=0x7fbfbc006df0).ready entity=mon.1 client_cookie=35c5eb39d2956739 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:04.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.835+0000 7fbfd259c700 1 -- 172.21.15.36:0/1702896428 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfbc00d570 con 0x7fbfd40ae150 2022-06-17T12:55:04.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.835+0000 7fbfd259c700 1 -- 172.21.15.36:0/1702896428 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfbc000e90 con 0x7fbfd40ae150 2022-06-17T12:55:04.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.835+0000 7fbfd259c700 1 -- 172.21.15.36:0/1702896428 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfbc00d8c0 con 0x7fbfd40ae150 2022-06-17T12:55:04.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.839+0000 7fbfd90e7700 1 -- 172.21.15.36:0/1702896428 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfd40ae150 msgr2=0x7fbfd40b2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:04.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.839+0000 7fbfd90e7700 1 --2- 172.21.15.36:0/1702896428 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfd40ae150 0x7fbfd40b2ad0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7fbfbc002f70 tx=0x7fbfbc006df0).stop 2022-06-17T12:55:04.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.839+0000 7fbfd90e7700 1 -- 172.21.15.36:0/1702896428 shutdown_connections 2022-06-17T12:55:04.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.839+0000 7fbfd90e7700 1 --2- 172.21.15.36:0/1702896428 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfd40ae150 0x7fbfd40b2ad0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.839+0000 7fbfd90e7700 1 --2- 172.21.15.36:0/1702896428 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbfd40b7bc0 0x7fbfd4140e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.839+0000 7fbfd90e7700 1 -- 172.21.15.36:0/1702896428 >> 172.21.15.36:0/1702896428 conn(0x7fbfd4060990 msgr2=0x7fbfd405c550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:04.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.839+0000 7fbfd90e7700 1 -- 172.21.15.36:0/1702896428 shutdown_connections 2022-06-17T12:55:04.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.839+0000 7fbfd90e7700 1 -- 172.21.15.36:0/1702896428 wait complete. 2022-06-17T12:55:04.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.839+0000 7fbfd90e7700 1 Processor -- start 2022-06-17T12:55:04.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.839+0000 7fbfd90e7700 1 -- start start 2022-06-17T12:55:04.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.839+0000 7fbfd90e7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbfd40ae150 0x7fbfd4067100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.839+0000 7fbfd90e7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfd40b2fc0 0x7fbfd40b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.839+0000 7fbfd90e7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfd40b7bc0 0x7fbfd4142a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.839+0000 7fbfd90e7700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd41452a0 con 0x7fbfd40b2fc0 2022-06-17T12:55:04.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.839+0000 7fbfd90e7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40b00d0 con 0x7fbfd40b7bc0 2022-06-17T12:55:04.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.839+0000 7fbfd90e7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40b0370 con 0x7fbfd40ae150 2022-06-17T12:55:04.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.839+0000 7fbfca59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfd40b2fc0 0x7fbfd40b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.843+0000 7fbfca59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfd40b2fc0 0x7fbfd40b0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37876/0 (socket says 172.21.15.36:37876) 2022-06-17T12:55:04.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.843+0000 7fbfca59c700 1 -- 172.21.15.36:0/3982452490 learned_addr learned my addr 172.21.15.36:0/3982452490 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:04.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.843+0000 7fbfd2d9d700 1 --2- 172.21.15.36:0/3982452490 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbfd40ae150 0x7fbfd4067100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.843+0000 7fbfd359e700 1 --2- 172.21.15.36:0/3982452490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfd40b7bc0 0x7fbfd4142a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.843+0000 7fbfca59c700 1 -- 172.21.15.36:0/3982452490 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbfd40ae150 msgr2=0x7fbfd4067100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:04.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.843+0000 7fbfca59c700 1 --2- 172.21.15.36:0/3982452490 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbfd40ae150 0x7fbfd4067100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.843+0000 7fbfca59c700 1 -- 172.21.15.36:0/3982452490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfd40b7bc0 msgr2=0x7fbfd4142a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:04.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.843+0000 7fbfca59c700 1 --2- 172.21.15.36:0/3982452490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfd40b7bc0 0x7fbfd4142a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.843+0000 7fbfca59c700 1 -- 172.21.15.36:0/3982452490 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc400b180 con 0x7fbfd40b2fc0 2022-06-17T12:55:04.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.843+0000 7fbfd2d9d700 1 --2- 172.21.15.36:0/3982452490 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbfd40ae150 0x7fbfd4067100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:04.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.843+0000 7fbfca59c700 1 --2- 172.21.15.36:0/3982452490 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfd40b2fc0 0x7fbfd40b0810 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7fbfbc002f70 tx=0x7fbfbc014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:04.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.843+0000 7fbfd0d99700 1 -- 172.21.15.36:0/3982452490 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfbc00dea0 con 0x7fbfd40b2fc0 2022-06-17T12:55:04.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.843+0000 7fbfd0d99700 1 -- 172.21.15.36:0/3982452490 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfbc01e400 con 0x7fbfd40b2fc0 2022-06-17T12:55:04.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.843+0000 7fbfd90e7700 1 -- 172.21.15.36:0/3982452490 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfbc00b180 con 0x7fbfd40b2fc0 2022-06-17T12:55:04.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.843+0000 7fbfd0d99700 1 -- 172.21.15.36:0/3982452490 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfbc01e760 con 0x7fbfd40b2fc0 2022-06-17T12:55:04.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.843+0000 7fbfd90e7700 1 -- 172.21.15.36:0/3982452490 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd4144e70 con 0x7fbfd40b2fc0 2022-06-17T12:55:04.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.843+0000 7fbfd90e7700 1 -- 172.21.15.36:0/3982452490 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd40b0530 con 0x7fbfd40b2fc0 2022-06-17T12:55:04.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.843+0000 7fbfd0d99700 1 -- 172.21.15.36:0/3982452490 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fbfbc025030 con 0x7fbfd40b2fc0 2022-06-17T12:55:04.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.843+0000 7fbfd0d99700 1 --2- 172.21.15.36:0/3982452490 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbfb8053ad0 0x7fbfb8055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:04.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.843+0000 7fbfd0d99700 1 -- 172.21.15.36:0/3982452490 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbfbc06cdc0 con 0x7fbfd40b2fc0 2022-06-17T12:55:04.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.847+0000 7fbfd2d9d700 1 --2- 172.21.15.36:0/3982452490 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbfb8053ad0 0x7fbfb8055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:04.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.847+0000 7fbfd2d9d700 1 --2- 172.21.15.36:0/3982452490 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbfb8053ad0 0x7fbfb8055f90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7fbfc400b000 tx=0x7fbfc4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:04.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.851+0000 7fbfd0d99700 1 -- 172.21.15.36:0/3982452490 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbfbc043b40 con 0x7fbfd40b2fc0 2022-06-17T12:55:04.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.979+0000 7fbfd90e7700 1 -- 172.21.15.36:0/3982452490 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbfd4057410 con 0x7fbfd40b2fc0 2022-06-17T12:55:04.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.979+0000 7fbfd0d99700 1 -- 172.21.15.36:0/3982452490 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v104) v1 ==== 104+0+10 (secure 0 0 0) 0x7fbfbc023020 con 0x7fbfd40b2fc0 2022-06-17T12:55:04.969 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:55:04.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.983+0000 7fbfd90e7700 1 -- 172.21.15.36:0/3982452490 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbfb8053ad0 msgr2=0x7fbfb8055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:04.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.983+0000 7fbfd90e7700 1 --2- 172.21.15.36:0/3982452490 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbfb8053ad0 0x7fbfb8055f90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7fbfc400b000 tx=0x7fbfc4008040).stop 2022-06-17T12:55:04.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.983+0000 7fbfd90e7700 1 -- 172.21.15.36:0/3982452490 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfd40b2fc0 msgr2=0x7fbfd40b0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:04.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.983+0000 7fbfd90e7700 1 --2- 172.21.15.36:0/3982452490 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfd40b2fc0 0x7fbfd40b0810 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7fbfbc002f70 tx=0x7fbfbc014000).stop 2022-06-17T12:55:04.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.983+0000 7fbfd90e7700 1 -- 172.21.15.36:0/3982452490 shutdown_connections 2022-06-17T12:55:04.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.983+0000 7fbfd90e7700 1 --2- 172.21.15.36:0/3982452490 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbfb8053ad0 0x7fbfb8055f90 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.983+0000 7fbfd90e7700 1 --2- 172.21.15.36:0/3982452490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfd40b7bc0 0x7fbfd4142a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.983+0000 7fbfd90e7700 1 --2- 172.21.15.36:0/3982452490 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfd40b2fc0 0x7fbfd40b0810 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.983+0000 7fbfd90e7700 1 --2- 172.21.15.36:0/3982452490 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbfd40ae150 0x7fbfd4067100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:04.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.983+0000 7fbfd90e7700 1 -- 172.21.15.36:0/3982452490 >> 172.21.15.36:0/3982452490 conn(0x7fbfd4060990 msgr2=0x7fbfd4065ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:04.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.983+0000 7fbfd90e7700 1 -- 172.21.15.36:0/3982452490 shutdown_connections 2022-06-17T12:55:04.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:04.983+0000 7fbfd90e7700 1 -- 172.21.15.36:0/3982452490 wait complete. 2022-06-17T12:55:04.982 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:55:05.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.103+0000 7f846ffff700 1 Processor -- start 2022-06-17T12:55:05.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.103+0000 7f846ffff700 1 -- start start 2022-06-17T12:55:05.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.103+0000 7f846ffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f84700b27b0 0x7f847014b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:05.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.103+0000 7f846ffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8470060ca0 0x7f84700ac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:05.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.103+0000 7f846ffff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f847014b870 con 0x7f8470147bd0 2022-06-17T12:55:05.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.103+0000 7f846ffff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84700ac500 con 0x7f84700b27b0 2022-06-17T12:55:05.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.103+0000 7f846ffff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84700b2b90 con 0x7f8470060ca0 2022-06-17T12:55:05.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.103+0000 7f846f7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f8470147bd0 0x7f84701442d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55412/0 (socket says 172.21.15.36:55412) 2022-06-17T12:55:05.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.103+0000 7f846f7fe700 1 -- 172.21.15.36:0/1758938591 learned_addr learned my addr 172.21.15.36:0/1758938591 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:05.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.103+0000 7f846effd700 1 --2- 172.21.15.36:0/1758938591 >> v2:172.21.15.36:3300/0 conn(0x7f84700b27b0 0x7f847014b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:05.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.103+0000 7f846e7fc700 1 --2- 172.21.15.36:0/1758938591 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8470060ca0 0x7f84700ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:05.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.103+0000 7f846dffb700 1 -- 172.21.15.36:0/1758938591 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335700762 0 0) 0x7f847014b870 con 0x7f8470147bd0 2022-06-17T12:55:05.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.103+0000 7f846dffb700 1 -- 172.21.15.36:0/1758938591 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8454003260 con 0x7f8470147bd0 2022-06-17T12:55:05.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.107+0000 7f846effd700 1 -- 172.21.15.36:0/1758938591 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8470060ca0 msgr2=0x7f84700ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:05.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.107+0000 7f846effd700 1 --2- 172.21.15.36:0/1758938591 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8470060ca0 0x7f84700ac780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:05.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.107+0000 7f846effd700 1 -- 172.21.15.36:0/1758938591 >> v1:172.21.15.36:6789/0 conn(0x7f8470147bd0 legacy=0x7f84701442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:05.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.107+0000 7f846effd700 1 -- 172.21.15.36:0/1758938591 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8470147490 con 0x7f84700b27b0 2022-06-17T12:55:05.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.107+0000 7f846effd700 1 --2- 172.21.15.36:0/1758938591 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f84700b27b0 0x7f847014b2b0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7f8464002f70 tx=0x7f8464006df0).ready entity=mon.0 client_cookie=2d8c71188878ec9a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:05.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.107+0000 7f846dffb700 1 -- 172.21.15.36:0/1758938591 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f846400d4a0 con 0x7f84700b27b0 2022-06-17T12:55:05.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.107+0000 7f846dffb700 1 -- 172.21.15.36:0/1758938591 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8464000e90 con 0x7f84700b27b0 2022-06-17T12:55:05.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.107+0000 7f846dffb700 1 -- 172.21.15.36:0/1758938591 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f846400d7f0 con 0x7f84700b27b0 2022-06-17T12:55:05.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.107+0000 7f846ffff700 1 -- 172.21.15.36:0/1758938591 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f84700b27b0 msgr2=0x7f847014b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:05.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.107+0000 7f846ffff700 1 --2- 172.21.15.36:0/1758938591 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f84700b27b0 0x7f847014b2b0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7f8464002f70 tx=0x7f8464006df0).stop 2022-06-17T12:55:05.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.107+0000 7f846ffff700 1 -- 172.21.15.36:0/1758938591 shutdown_connections 2022-06-17T12:55:05.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.107+0000 7f846ffff700 1 --2- 172.21.15.36:0/1758938591 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8470060ca0 0x7f84700ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:05.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.107+0000 7f846ffff700 1 --2- 172.21.15.36:0/1758938591 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f84700b27b0 0x7f847014b2b0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:05.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.107+0000 7f846ffff700 1 -- 172.21.15.36:0/1758938591 >> 172.21.15.36:0/1758938591 conn(0x7f84700b5d10 msgr2=0x7f84700b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:05.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.107+0000 7f846ffff700 1 -- 172.21.15.36:0/1758938591 shutdown_connections 2022-06-17T12:55:05.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.107+0000 7f846ffff700 1 -- 172.21.15.36:0/1758938591 wait complete. 2022-06-17T12:55:05.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.111+0000 7f846ffff700 1 Processor -- start 2022-06-17T12:55:05.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.111+0000 7f846ffff700 1 -- start start 2022-06-17T12:55:05.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.111+0000 7f846ffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8470060ca0 0x7f8470159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:05.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.111+0000 7f846ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84700b27b0 0x7f847015a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:05.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.111+0000 7f846ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8470147bd0 0x7f8470063320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:05.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.111+0000 7f846ffff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84700ac310 con 0x7f8470147bd0 2022-06-17T12:55:05.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.111+0000 7f846ffff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8470066fc0 con 0x7f84700b27b0 2022-06-17T12:55:05.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.111+0000 7f846ffff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8470065f50 con 0x7f8470060ca0 2022-06-17T12:55:05.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.111+0000 7f846f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8470147bd0 0x7f8470063320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:05.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.111+0000 7f846effd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8470060ca0 0x7f8470159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:05.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.111+0000 7f846effd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8470060ca0 0x7f8470159e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:52676/0 (socket says 172.21.15.36:52676) 2022-06-17T12:55:05.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.111+0000 7f846e7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84700b27b0 0x7f847015a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:05.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.111+0000 7f846effd700 1 -- 172.21.15.36:0/1088810312 learned_addr learned my addr 172.21.15.36:0/1088810312 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:05.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.111+0000 7f846f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8470147bd0 0x7f8470063320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37890/0 (socket says 172.21.15.36:37890) 2022-06-17T12:55:05.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.111+0000 7f846effd700 1 -- 172.21.15.36:0/1088810312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84700b27b0 msgr2=0x7f847015a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:05.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.111+0000 7f846effd700 1 --2- 172.21.15.36:0/1088810312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84700b27b0 0x7f847015a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:05.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.111+0000 7f846effd700 1 -- 172.21.15.36:0/1088810312 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8470147bd0 msgr2=0x7f8470063320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:05.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.111+0000 7f846effd700 1 --2- 172.21.15.36:0/1088810312 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8470147bd0 0x7f8470063320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:05.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.111+0000 7f846effd700 1 -- 172.21.15.36:0/1088810312 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846400b180 con 0x7f8470060ca0 2022-06-17T12:55:05.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.111+0000 7f846e7fc700 1 --2- 172.21.15.36:0/1088810312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84700b27b0 0x7f847015a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:05.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.111+0000 7f846effd700 1 --2- 172.21.15.36:0/1088810312 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8470060ca0 0x7f8470159e90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f8464002f70 tx=0x7f8464014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:05.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.115+0000 7f845ffff700 1 -- 172.21.15.36:0/1088810312 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8464016070 con 0x7f8470060ca0 2022-06-17T12:55:05.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.115+0000 7f845ffff700 1 -- 172.21.15.36:0/1088810312 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84640137d0 con 0x7f8470060ca0 2022-06-17T12:55:05.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.115+0000 7f846ffff700 1 -- 172.21.15.36:0/1088810312 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8470064360 con 0x7f8470060ca0 2022-06-17T12:55:05.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.115+0000 7f845ffff700 1 -- 172.21.15.36:0/1088810312 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8464013b30 con 0x7f8470060ca0 2022-06-17T12:55:05.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.115+0000 7f846ffff700 1 -- 172.21.15.36:0/1088810312 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84700640d0 con 0x7f8470060ca0 2022-06-17T12:55:05.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.115+0000 7f846ffff700 1 -- 172.21.15.36:0/1088810312 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8470057410 con 0x7f8470060ca0 2022-06-17T12:55:05.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.115+0000 7f845ffff700 1 -- 172.21.15.36:0/1088810312 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8464025030 con 0x7f8470060ca0 2022-06-17T12:55:05.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.115+0000 7f845ffff700 1 --2- 172.21.15.36:0/1088810312 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f844c053a60 0x7f844c055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:05.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.115+0000 7f845ffff700 1 -- 172.21.15.36:0/1088810312 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f846406cd60 con 0x7f8470060ca0 2022-06-17T12:55:05.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.115+0000 7f846e7fc700 1 --2- 172.21.15.36:0/1088810312 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f844c053a60 0x7f844c055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:05.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.119+0000 7f846e7fc700 1 --2- 172.21.15.36:0/1088810312 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f844c053a60 0x7f844c055f20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f846000b000 tx=0x7f84600091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:05.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.127+0000 7f845ffff700 1 -- 172.21.15.36:0/1088810312 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8464043b60 con 0x7f8470060ca0 2022-06-17T12:55:05.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.255+0000 7f846ffff700 1 -- 172.21.15.36:0/1088810312 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8470066090 con 0x7f8470060ca0 2022-06-17T12:55:05.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.255+0000 7f845ffff700 1 -- 172.21.15.36:0/1088810312 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v104) v1 ==== 100+0+10 (secure 0 0 0) 0x7f84640230a0 con 0x7f8470060ca0 2022-06-17T12:55:05.244 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:55:05.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.259+0000 7f846ffff700 1 -- 172.21.15.36:0/1088810312 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f844c053a60 msgr2=0x7f844c055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:05.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.259+0000 7f846ffff700 1 --2- 172.21.15.36:0/1088810312 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f844c053a60 0x7f844c055f20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f846000b000 tx=0x7f84600091d0).stop 2022-06-17T12:55:05.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.259+0000 7f846ffff700 1 -- 172.21.15.36:0/1088810312 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8470060ca0 msgr2=0x7f8470159e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:05.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.259+0000 7f846ffff700 1 --2- 172.21.15.36:0/1088810312 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8470060ca0 0x7f8470159e90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f8464002f70 tx=0x7f8464014000).stop 2022-06-17T12:55:05.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.259+0000 7f846ffff700 1 -- 172.21.15.36:0/1088810312 shutdown_connections 2022-06-17T12:55:05.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.259+0000 7f846ffff700 1 --2- 172.21.15.36:0/1088810312 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f844c053a60 0x7f844c055f20 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:05.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.259+0000 7f846ffff700 1 --2- 172.21.15.36:0/1088810312 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8470147bd0 0x7f8470063320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:05.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.259+0000 7f846ffff700 1 --2- 172.21.15.36:0/1088810312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84700b27b0 0x7f847015a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:05.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.259+0000 7f846ffff700 1 --2- 172.21.15.36:0/1088810312 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8470060ca0 0x7f8470159e90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:05.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.259+0000 7f846ffff700 1 -- 172.21.15.36:0/1088810312 >> 172.21.15.36:0/1088810312 conn(0x7f84700b5d10 msgr2=0x7f84700affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:05.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.259+0000 7f846ffff700 1 -- 172.21.15.36:0/1088810312 shutdown_connections 2022-06-17T12:55:05.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.259+0000 7f846ffff700 1 -- 172.21.15.36:0/1088810312 wait complete. 2022-06-17T12:55:05.264 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:55:05.264 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:55:05.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.379+0000 7f154d70f700 1 Processor -- start 2022-06-17T12:55:05.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.379+0000 7f154d70f700 1 -- start start 2022-06-17T12:55:05.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.379+0000 7f154d70f700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f154813cb70 0x7f1548143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:05.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.379+0000 7f154d70f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f15480aef60 0x7f15480b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:05.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.379+0000 7f154d70f700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15480af4b0 con 0x7f15480ba2e0 2022-06-17T12:55:05.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.379+0000 7f154d70f700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1548142f50 con 0x7f15480aef60 2022-06-17T12:55:05.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.379+0000 7f154d70f700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15481431f0 con 0x7f154813cb70 2022-06-17T12:55:05.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.379+0000 7f15477fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f15480aef60 0x7f15480b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:05.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.379+0000 7f15477fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f15480aef60 0x7f15480b9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37894/0 (socket says 172.21.15.36:37894) 2022-06-17T12:55:05.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.379+0000 7f15477fe700 1 -- 172.21.15.36:0/349840547 learned_addr learned my addr 172.21.15.36:0/349840547 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:05.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.383+0000 7f1547fff700 1 --2- 172.21.15.36:0/349840547 >> v2:172.21.15.70:3300/0 conn(0x7f154813cb70 0x7f1548143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:05.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.383+0000 7f15477fe700 1 -- 172.21.15.36:0/349840547 >> v2:172.21.15.70:3300/0 conn(0x7f154813cb70 msgr2=0x7f1548143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:05.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.383+0000 7f15477fe700 1 --2- 172.21.15.36:0/349840547 >> v2:172.21.15.70:3300/0 conn(0x7f154813cb70 0x7f1548143690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:05.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.383+0000 7f15477fe700 1 -- 172.21.15.36:0/349840547 >> v1:172.21.15.70:6789/0 conn(0x7f15480ba2e0 legacy=0x7f15481442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:05.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.383+0000 7f15477fe700 1 -- 172.21.15.36:0/349840547 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15481449b0 con 0x7f15480aef60 2022-06-17T12:55:05.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.383+0000 7f15477fe700 1 --2- 172.21.15.36:0/349840547 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f15480aef60 0x7f15480b9df0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 rx=0x7f153c002f70 tx=0x7f153c006eb0).ready entity=mon.0 client_cookie=18cffbd899fef12f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:05.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.383+0000 7f1546ffd700 1 -- 172.21.15.36:0/349840547 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f153c00d610 con 0x7f15480aef60 2022-06-17T12:55:05.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.383+0000 7f1546ffd700 1 -- 172.21.15.36:0/349840547 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153c000e90 con 0x7f15480aef60 2022-06-17T12:55:05.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.383+0000 7f1546ffd700 1 -- 172.21.15.36:0/349840547 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f153c00d9a0 con 0x7f15480aef60 2022-06-17T12:55:05.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.383+0000 7f154d70f700 1 -- 172.21.15.36:0/349840547 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f15480aef60 msgr2=0x7f15480b9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:05.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.383+0000 7f154d70f700 1 --2- 172.21.15.36:0/349840547 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f15480aef60 0x7f15480b9df0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 rx=0x7f153c002f70 tx=0x7f153c006eb0).stop 2022-06-17T12:55:05.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.383+0000 7f154d70f700 1 -- 172.21.15.36:0/349840547 shutdown_connections 2022-06-17T12:55:05.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.383+0000 7f154d70f700 1 --2- 172.21.15.36:0/349840547 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f15480aef60 0x7f15480b9df0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:05.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.383+0000 7f154d70f700 1 --2- 172.21.15.36:0/349840547 >> v2:172.21.15.70:3300/0 conn(0x7f154813cb70 0x7f1548143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:05.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.383+0000 7f154d70f700 1 -- 172.21.15.36:0/349840547 >> 172.21.15.36:0/349840547 conn(0x7f154805c550 msgr2=0x7f15480b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:05.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.383+0000 7f154d70f700 1 -- 172.21.15.36:0/349840547 shutdown_connections 2022-06-17T12:55:05.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.383+0000 7f154d70f700 1 -- 172.21.15.36:0/349840547 wait complete. 2022-06-17T12:55:05.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.383+0000 7f154d70f700 1 Processor -- start 2022-06-17T12:55:05.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.383+0000 7f154d70f700 1 -- start start 2022-06-17T12:55:05.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f154d70f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f15480aef60 0x7f15481423f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:05.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f154d70f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15480ba2e0 0x7f15480afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:05.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f154d70f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f154813cb70 0x7f15480b0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:05.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f154d70f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15480b1710 con 0x7f15480aef60 2022-06-17T12:55:05.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f154d70f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15480af680 con 0x7f154813cb70 2022-06-17T12:55:05.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f154d70f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15480b3860 con 0x7f15480ba2e0 2022-06-17T12:55:05.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f1547fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f15480aef60 0x7f15481423f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:05.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f1547fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f15480aef60 0x7f15481423f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37900/0 (socket says 172.21.15.36:37900) 2022-06-17T12:55:05.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f1547fff700 1 -- 172.21.15.36:0/4219906318 learned_addr learned my addr 172.21.15.36:0/4219906318 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:05.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f15477fe700 1 --2- 172.21.15.36:0/4219906318 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15480ba2e0 0x7f15480afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:05.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f154cf0e700 1 --2- 172.21.15.36:0/4219906318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f154813cb70 0x7f15480b0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:05.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f1547fff700 1 -- 172.21.15.36:0/4219906318 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15480ba2e0 msgr2=0x7f15480afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:05.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f1547fff700 1 --2- 172.21.15.36:0/4219906318 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15480ba2e0 0x7f15480afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:05.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f1547fff700 1 -- 172.21.15.36:0/4219906318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f154813cb70 msgr2=0x7f15480b0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:05.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f1547fff700 1 --2- 172.21.15.36:0/4219906318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f154813cb70 0x7f15480b0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:05.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f1547fff700 1 -- 172.21.15.36:0/4219906318 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c00b1e0 con 0x7f15480aef60 2022-06-17T12:55:05.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f154cf0e700 1 --2- 172.21.15.36:0/4219906318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f154813cb70 0x7f15480b0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:05.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f1547fff700 1 --2- 172.21.15.36:0/4219906318 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f15480aef60 0x7f15481423f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7f15380030b0 tx=0x7f1538008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:05.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f15457fa700 1 -- 172.21.15.36:0/4219906318 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1538011780 con 0x7f15480aef60 2022-06-17T12:55:05.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f15457fa700 1 -- 172.21.15.36:0/4219906318 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15380118e0 con 0x7f15480aef60 2022-06-17T12:55:05.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f154d70f700 1 -- 172.21.15.36:0/4219906318 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15480b3430 con 0x7f15480aef60 2022-06-17T12:55:05.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f15457fa700 1 -- 172.21.15.36:0/4219906318 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1538011c40 con 0x7f15480aef60 2022-06-17T12:55:05.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f154d70f700 1 -- 172.21.15.36:0/4219906318 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15480b1390 con 0x7f15480aef60 2022-06-17T12:55:05.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f154d70f700 1 -- 172.21.15.36:0/4219906318 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15480b35c0 con 0x7f15480aef60 2022-06-17T12:55:05.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.387+0000 7f15457fa700 1 -- 172.21.15.36:0/4219906318 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f1538006020 con 0x7f15480aef60 2022-06-17T12:55:05.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.391+0000 7f15457fa700 1 --2- 172.21.15.36:0/4219906318 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f15280539b0 0x7f1528055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:05.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.391+0000 7f15457fa700 1 -- 172.21.15.36:0/4219906318 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f153806c330 con 0x7f15480aef60 2022-06-17T12:55:05.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.391+0000 7f15477fe700 1 --2- 172.21.15.36:0/4219906318 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f15280539b0 0x7f1528055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:05.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.391+0000 7f15477fe700 1 --2- 172.21.15.36:0/4219906318 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f15280539b0 0x7f1528055e70 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f153c01c000 tx=0x7f153c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:05.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.395+0000 7f15457fa700 1 -- 172.21.15.36:0/4219906318 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f15380430d0 con 0x7f15480aef60 2022-06-17T12:55:05.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.555+0000 7f154d70f700 1 -- 172.21.15.36:0/4219906318 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15480b35c0 con 0x7f15480aef60 2022-06-17T12:55:05.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.555+0000 7f15457fa700 1 -- 172.21.15.36:0/4219906318 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f15480b35c0 con 0x7f15480aef60 2022-06-17T12:55:05.545 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:05.546 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":54,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/676934608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":676934608},{"type":"v1","addr":"172.21.15.70:6833","nonce":676934608}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:54:59.995522+0000","modified":"2022-06-17T12:55:01.664559+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7395":{"gid":7395,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/233328233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":233328233},{"type":"v1","addr":"172.21.15.36:6835","nonce":233328233}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-06-17T12:55:05.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.559+0000 7f154d70f700 1 -- 172.21.15.36:0/4219906318 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f15280539b0 msgr2=0x7f1528055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:05.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.559+0000 7f154d70f700 1 --2- 172.21.15.36:0/4219906318 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f15280539b0 0x7f1528055e70 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f153c01c000 tx=0x7f153c015000).stop 2022-06-17T12:55:05.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.559+0000 7f154d70f700 1 -- 172.21.15.36:0/4219906318 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f15480aef60 msgr2=0x7f15481423f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:05.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.559+0000 7f154d70f700 1 --2- 172.21.15.36:0/4219906318 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f15480aef60 0x7f15481423f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7f15380030b0 tx=0x7f1538008040).stop 2022-06-17T12:55:05.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.559+0000 7f154d70f700 1 -- 172.21.15.36:0/4219906318 shutdown_connections 2022-06-17T12:55:05.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.559+0000 7f154d70f700 1 --2- 172.21.15.36:0/4219906318 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f15280539b0 0x7f1528055e70 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:05.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.559+0000 7f154d70f700 1 --2- 172.21.15.36:0/4219906318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f154813cb70 0x7f15480b0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:05.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.559+0000 7f154d70f700 1 --2- 172.21.15.36:0/4219906318 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15480ba2e0 0x7f15480afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:05.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.559+0000 7f154d70f700 1 --2- 172.21.15.36:0/4219906318 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f15480aef60 0x7f15481423f0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:05.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.559+0000 7f154d70f700 1 -- 172.21.15.36:0/4219906318 >> 172.21.15.36:0/4219906318 conn(0x7f154805c550 msgr2=0x7f15480b8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:05.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.559+0000 7f154d70f700 1 -- 172.21.15.36:0/4219906318 shutdown_connections 2022-06-17T12:55:05.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:05.559+0000 7f154d70f700 1 -- 172.21.15.36:0/4219906318 wait complete. 2022-06-17T12:55:05.555 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 54 2022-06-17T12:55:05.759 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:54:59.995522+0000', 'modified': '2022-06-17T12:55:01.664559+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7395}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7395': {'gid': 7395, 'name': 'a', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.36:6835/233328233', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 233328233}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 233328233}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:55:05.761 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:55:05.762 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:55:05.882 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":7395,"want_state":"up:active","state":"up:active","rank_uptime":5.8898909540000002,"mdsmap_epoch":54,"osdmap_epoch":104,"osdmap_epoch_barrier":104,"uptime":10.448853310000001} 2022-06-17T12:55:05.883 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 7395, "mdsmap_epoch": 54, "osdmap_epoch": 104, "osdmap_epoch_barrier": 104, "rank_uptime": 5.889890954, "state": "up:active", "uptime": 10.44885331, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:55:05.884 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:55:05.885 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-17T12:55:05.886 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi107.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-17T12:55:05.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:05.888 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:55:05.905 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:55:05.965 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:55:05.966 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:55:05.980 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:55:05.981 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:55:05.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:05.984 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:55:05.991 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:55:05.993 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:55:06.038 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:55:06.056 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:55:06.057 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:55:06.069 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:55:06.071 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:55:06.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:06.074 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:55:06.111 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:2022-06-17T12:55:06.122+0000 7fd6a927c080 -1 init, newargv = 0x5557f979d050 newargc=9ceph-fuse[22695 2022-06-17T12:55:06.112 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:]: starting ceph client 2022-06-17T12:55:06.122 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22695]: starting fuse 2022-06-17T12:55:07.223 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:55:07.242 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:55:07.243 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:55:07.298 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:55:07.300 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:55:07.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:07.301 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:55:07.309 INFO:teuthology.orchestra.run.smithi107.stdout:52 2022-06-17T12:55:07.309 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:55:07.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:07.311 DEBUG:teuthology.orchestra.run.smithi107:> sudo python3 -c ' 2022-06-17T12:55:07.311 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2022-06-17T12:55:07.312 DEBUG:teuthology.orchestra.run.smithi107:> import re 2022-06-17T12:55:07.312 DEBUG:teuthology.orchestra.run.smithi107:> import os 2022-06-17T12:55:07.313 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2022-06-17T12:55:07.314 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:55:07.315 DEBUG:teuthology.orchestra.run.smithi107:> def find_socket(client_name): 2022-06-17T12:55:07.316 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-17T12:55:07.317 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2022-06-17T12:55:07.317 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:55:07.318 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2022-06-17T12:55:07.318 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2022-06-17T12:55:07.319 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2022-06-17T12:55:07.319 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2022-06-17T12:55:07.320 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:55:07.320 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2022-06-17T12:55:07.321 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:55:07.322 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:55:07.323 DEBUG:teuthology.orchestra.run.smithi107:> return f 2022-06-17T12:55:07.323 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:55:07.324 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:55:07.324 DEBUG:teuthology.orchestra.run.smithi107:> print(find_socket("client.0")) 2022-06-17T12:55:07.325 DEBUG:teuthology.orchestra.run.smithi107:> ' 2022-06-17T12:55:07.370 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.0.22695.asok 2022-06-17T12:55:07.375 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22695.asok 2022-06-17T12:55:07.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:07.377 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22695.asok status 2022-06-17T12:55:07.513 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2022-06-17T12:55:07.514 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2022-06-17T12:55:07.514 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:55:07.515 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:55:07.515 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "0", 2022-06-17T12:55:07.515 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2022-06-17T12:55:07.515 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-17T12:55:07.516 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "22695", 2022-06-17T12:55:07.516 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/" 2022-06-17T12:55:07.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:55:07.516 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2022-06-17T12:55:07.516 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2022-06-17T12:55:07.517 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 7554, 2022-06-17T12:55:07.517 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2022-06-17T12:55:07.517 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2022-06-17T12:55:07.517 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2022-06-17T12:55:07.518 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 7554 2022-06-17T12:55:07.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:55:07.518 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:55:07.518 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:55:07.519 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:55:07.519 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 202615733 2022-06-17T12:55:07.519 INFO:teuthology.orchestra.run.smithi107.stdout: } 2022-06-17T12:55:07.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:55:07.520 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:55:07.520 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:55:07.520 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:55:07.521 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 202615733 2022-06-17T12:55:07.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:55:07.522 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.7554 172.21.15.107:0/202615733", 2022-06-17T12:55:07.522 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "172.21.15.107:0/202615733", 2022-06-17T12:55:07.522 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 1, 2022-06-17T12:55:07.522 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 54, 2022-06-17T12:55:07.523 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 104, 2022-06-17T12:55:07.523 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:55:07.523 INFO:teuthology.orchestra.run.smithi107.stdout: "blacklisted": false, 2022-06-17T12:55:07.523 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2022-06-17T12:55:07.524 INFO:teuthology.orchestra.run.smithi107.stdout:} 2022-06-17T12:55:07.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:07.527 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:55:07.528 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:55:07.539 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:55:07.539 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:55:07.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:07.540 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:55:07.557 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:55:07.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.683+0000 7f6c64ddd700 1 Processor -- start 2022-06-17T12:55:07.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.683+0000 7f6c64ddd700 1 -- start start 2022-06-17T12:55:07.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.683+0000 7f6c64ddd700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6c60060ac0 0x7f6c6013c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:07.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.683+0000 7f6c64ddd700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c600b0230 con 0x7f6c600b1ff0 2022-06-17T12:55:07.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.683+0000 7f6c64ddd700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c60060ea0 con 0x7f6c600b1890 2022-06-17T12:55:07.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.683+0000 7f6c64ddd700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c60143800 con 0x7f6c60060ac0 2022-06-17T12:55:07.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.683+0000 7f6c5ffff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f6c600b1ff0 0x7f6c6013f270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55442/0 (socket says 172.21.15.36:55442) 2022-06-17T12:55:07.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.683+0000 7f6c5ffff700 1 -- 172.21.15.36:0/556188738 learned_addr learned my addr 172.21.15.36:0/556188738 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:07.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.683+0000 7f6c5f7fe700 1 --2- 172.21.15.36:0/556188738 >> v2:172.21.15.70:3300/0 conn(0x7f6c60060ac0 0x7f6c6013c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:07.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.687+0000 7f6c5f7fe700 1 -- 172.21.15.36:0/556188738 >> v1:172.21.15.70:6789/0 conn(0x7f6c600b1890 legacy=0x7f6c6013c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:07.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.687+0000 7f6c5f7fe700 1 -- 172.21.15.36:0/556188738 >> v1:172.21.15.36:6789/0 conn(0x7f6c600b1ff0 legacy=0x7f6c6013f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:07.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.687+0000 7f6c5f7fe700 1 -- 172.21.15.36:0/556188738 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c600afcf0 con 0x7f6c60060ac0 2022-06-17T12:55:07.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.687+0000 7f6c5f7fe700 1 --2- 172.21.15.36:0/556188738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c60060ac0 0x7f6c6013c2b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f6c50002f70 tx=0x7f6c50006e80).ready entity=mon.1 client_cookie=c227f63c129138fd server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:07.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.691+0000 7f6c5e7fc700 1 -- 172.21.15.36:0/556188738 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c50014070 con 0x7f6c60060ac0 2022-06-17T12:55:07.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.691+0000 7f6c5e7fc700 1 -- 172.21.15.36:0/556188738 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c50000e90 con 0x7f6c60060ac0 2022-06-17T12:55:07.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.691+0000 7f6c5e7fc700 1 -- 172.21.15.36:0/556188738 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c5000a860 con 0x7f6c60060ac0 2022-06-17T12:55:07.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.691+0000 7f6c64ddd700 1 -- 172.21.15.36:0/556188738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c60060ac0 msgr2=0x7f6c6013c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:07.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.691+0000 7f6c64ddd700 1 --2- 172.21.15.36:0/556188738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c60060ac0 0x7f6c6013c2b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f6c50002f70 tx=0x7f6c50006e80).stop 2022-06-17T12:55:07.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.691+0000 7f6c64ddd700 1 -- 172.21.15.36:0/556188738 shutdown_connections 2022-06-17T12:55:07.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.691+0000 7f6c64ddd700 1 --2- 172.21.15.36:0/556188738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c60060ac0 0x7f6c6013c2b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:07.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.691+0000 7f6c64ddd700 1 -- 172.21.15.36:0/556188738 >> 172.21.15.36:0/556188738 conn(0x7f6c600b65f0 msgr2=0x7f6c600b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:07.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.691+0000 7f6c64ddd700 1 -- 172.21.15.36:0/556188738 shutdown_connections 2022-06-17T12:55:07.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.691+0000 7f6c64ddd700 1 -- 172.21.15.36:0/556188738 wait complete. 2022-06-17T12:55:07.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.691+0000 7f6c64ddd700 1 Processor -- start 2022-06-17T12:55:07.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.695+0000 7f6c64ddd700 1 -- start start 2022-06-17T12:55:07.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.695+0000 7f6c64ddd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c60060ac0 0x7f6c6013c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:07.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.695+0000 7f6c64ddd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c600b1890 0x7f6c60143980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:07.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.695+0000 7f6c64ddd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c600b1ff0 0x7f6c60143e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:07.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.695+0000 7f6c64ddd700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6013dd40 con 0x7f6c60060ac0 2022-06-17T12:55:07.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.695+0000 7f6c64ddd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c60065e10 con 0x7f6c600b1ff0 2022-06-17T12:55:07.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.695+0000 7f6c64ddd700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6013ecc0 con 0x7f6c600b1890 2022-06-17T12:55:07.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.695+0000 7f6c5f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c60060ac0 0x7f6c6013c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:07.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.695+0000 7f6c5f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c60060ac0 0x7f6c6013c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37914/0 (socket says 172.21.15.36:37914) 2022-06-17T12:55:07.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.695+0000 7f6c5f7fe700 1 -- 172.21.15.36:0/3768554064 learned_addr learned my addr 172.21.15.36:0/3768554064 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:07.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.695+0000 7f6c5effd700 1 --2- 172.21.15.36:0/3768554064 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c600b1890 0x7f6c60143980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:07.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.695+0000 7f6c5ffff700 1 --2- 172.21.15.36:0/3768554064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c600b1ff0 0x7f6c60143e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:07.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.695+0000 7f6c5f7fe700 1 -- 172.21.15.36:0/3768554064 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c600b1890 msgr2=0x7f6c60143980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:07.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.695+0000 7f6c5f7fe700 1 --2- 172.21.15.36:0/3768554064 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c600b1890 0x7f6c60143980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:07.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.695+0000 7f6c5f7fe700 1 -- 172.21.15.36:0/3768554064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c600b1ff0 msgr2=0x7f6c60143e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:07.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.695+0000 7f6c5f7fe700 1 --2- 172.21.15.36:0/3768554064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c600b1ff0 0x7f6c60143e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:07.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.695+0000 7f6c5f7fe700 1 -- 172.21.15.36:0/3768554064 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5000c310 con 0x7f6c60060ac0 2022-06-17T12:55:07.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.695+0000 7f6c5ffff700 1 --2- 172.21.15.36:0/3768554064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c600b1ff0 0x7f6c60143e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:07.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.695+0000 7f6c5f7fe700 1 --2- 172.21.15.36:0/3768554064 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c60060ac0 0x7f6c6013c920 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7f6c50002f70 tx=0x7f6c600afb60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:07.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.695+0000 7f6c5cff9700 1 -- 172.21.15.36:0/3768554064 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c50010b90 con 0x7f6c60060ac0 2022-06-17T12:55:07.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.695+0000 7f6c5cff9700 1 -- 172.21.15.36:0/3768554064 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c50020070 con 0x7f6c60060ac0 2022-06-17T12:55:07.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.695+0000 7f6c64ddd700 1 -- 172.21.15.36:0/3768554064 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c6013daa0 con 0x7f6c60060ac0 2022-06-17T12:55:07.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.695+0000 7f6c5cff9700 1 -- 172.21.15.36:0/3768554064 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c5001c440 con 0x7f6c60060ac0 2022-06-17T12:55:07.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.695+0000 7f6c64ddd700 1 -- 172.21.15.36:0/3768554064 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c60066270 con 0x7f6c60060ac0 2022-06-17T12:55:07.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.699+0000 7f6c64ddd700 1 -- 172.21.15.36:0/3768554064 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c600aff90 con 0x7f6c60060ac0 2022-06-17T12:55:07.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.699+0000 7f6c5cff9700 1 -- 172.21.15.36:0/3768554064 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6c5000e040 con 0x7f6c60060ac0 2022-06-17T12:55:07.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.699+0000 7f6c5cff9700 1 --2- 172.21.15.36:0/3768554064 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c40053a40 0x7f6c40055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:07.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.699+0000 7f6c5cff9700 1 -- 172.21.15.36:0/3768554064 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6c5006d930 con 0x7f6c60060ac0 2022-06-17T12:55:07.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.699+0000 7f6c5effd700 1 --2- 172.21.15.36:0/3768554064 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c40053a40 0x7f6c40055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:07.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.699+0000 7f6c5effd700 1 --2- 172.21.15.36:0/3768554064 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c40053a40 0x7f6c40055f00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f6c5400d000 tx=0x7f6c54007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:07.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.707+0000 7f6c5cff9700 1 -- 172.21.15.36:0/3768554064 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6c50044730 con 0x7f6c60060ac0 2022-06-17T12:55:07.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.835+0000 7f6c64ddd700 1 -- 172.21.15.36:0/3768554064 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c60057410 con 0x7f6c60060ac0 2022-06-17T12:55:07.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.835+0000 7f6c5cff9700 1 -- 172.21.15.36:0/3768554064 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f6c50047d50 con 0x7f6c60060ac0 2022-06-17T12:55:07.829 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:07.829 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":104,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:55:00.998590+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:54:58.393546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-06-17T12:54:58.997835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:55:07.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.839+0000 7f6c64ddd700 1 -- 172.21.15.36:0/3768554064 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c40053a40 msgr2=0x7f6c40055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:07.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.839+0000 7f6c64ddd700 1 --2- 172.21.15.36:0/3768554064 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c40053a40 0x7f6c40055f00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f6c5400d000 tx=0x7f6c54007b90).stop 2022-06-17T12:55:07.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.839+0000 7f6c64ddd700 1 -- 172.21.15.36:0/3768554064 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c60060ac0 msgr2=0x7f6c6013c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:07.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.839+0000 7f6c64ddd700 1 --2- 172.21.15.36:0/3768554064 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c60060ac0 0x7f6c6013c920 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7f6c50002f70 tx=0x7f6c600afb60).stop 2022-06-17T12:55:07.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.839+0000 7f6c64ddd700 1 -- 172.21.15.36:0/3768554064 shutdown_connections 2022-06-17T12:55:07.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.839+0000 7f6c64ddd700 1 --2- 172.21.15.36:0/3768554064 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c40053a40 0x7f6c40055f00 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:07.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.839+0000 7f6c64ddd700 1 --2- 172.21.15.36:0/3768554064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c600b1ff0 0x7f6c60143e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:07.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.839+0000 7f6c64ddd700 1 --2- 172.21.15.36:0/3768554064 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c600b1890 0x7f6c60143980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:07.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.839+0000 7f6c64ddd700 1 --2- 172.21.15.36:0/3768554064 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c60060ac0 0x7f6c6013c920 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:07.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.839+0000 7f6c64ddd700 1 -- 172.21.15.36:0/3768554064 >> 172.21.15.36:0/3768554064 conn(0x7f6c600b65f0 msgr2=0x7f6c600b4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:07.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.839+0000 7f6c64ddd700 1 -- 172.21.15.36:0/3768554064 shutdown_connections 2022-06-17T12:55:07.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.839+0000 7f6c64ddd700 1 -- 172.21.15.36:0/3768554064 wait complete. 2022-06-17T12:55:07.841 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:54:58.393546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:54:58.997835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:55:07.843 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:55:07.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.963+0000 7f6ca76c3700 1 Processor -- start 2022-06-17T12:55:07.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.963+0000 7f6ca76c3700 1 -- start start 2022-06-17T12:55:07.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6ca76c3700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6ca00b64f0 0x7f6ca0143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:07.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6ca76c3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ca00b4120 0x7f6ca0146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:07.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6ca76c3700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca00b68d0 con 0x7f6ca0147270 2022-06-17T12:55:07.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6ca76c3700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca01476b0 con 0x7f6ca00b64f0 2022-06-17T12:55:07.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6ca76c3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca00b6e20 con 0x7f6ca00b4120 2022-06-17T12:55:07.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6ca5c60700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f6ca0147270 0x7f6ca0140100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55456/0 (socket says 172.21.15.36:55456) 2022-06-17T12:55:07.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6ca5c60700 1 -- 172.21.15.36:0/3390973299 learned_addr learned my addr 172.21.15.36:0/3390973299 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:07.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6ca4c5e700 1 --2- 172.21.15.36:0/3390973299 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ca00b4120 0x7f6ca0146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:07.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6ca545f700 1 --2- 172.21.15.36:0/3390973299 >> v2:172.21.15.70:3300/0 conn(0x7f6ca00b64f0 0x7f6ca0143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:07.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6c93fff700 1 -- 172.21.15.36:0/3390973299 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3026108399 0 0) 0x7f6ca00b68d0 con 0x7f6ca0147270 2022-06-17T12:55:07.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6c93fff700 1 -- 172.21.15.36:0/3390973299 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c84003260 con 0x7f6ca0147270 2022-06-17T12:55:07.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6c93fff700 1 -- 172.21.15.36:0/3390973299 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 919012128 0 0) 0x7f6c84003260 con 0x7f6ca0147270 2022-06-17T12:55:07.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6c93fff700 1 -- 172.21.15.36:0/3390973299 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ca00b4120 msgr2=0x7f6ca0146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:07.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6c93fff700 1 --2- 172.21.15.36:0/3390973299 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ca00b4120 0x7f6ca0146d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:07.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6c93fff700 1 -- 172.21.15.36:0/3390973299 >> v2:172.21.15.70:3300/0 conn(0x7f6ca00b64f0 msgr2=0x7f6ca0143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:07.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6c93fff700 1 --2- 172.21.15.36:0/3390973299 >> v2:172.21.15.70:3300/0 conn(0x7f6ca00b64f0 0x7f6ca0143150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:07.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6c93fff700 1 -- 172.21.15.36:0/3390973299 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca01468e0 con 0x7f6ca0147270 2022-06-17T12:55:07.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6c93fff700 1 -- 172.21.15.36:0/3390973299 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f6c980019e0 con 0x7f6ca0147270 2022-06-17T12:55:07.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6ca4c5e700 1 --2- 172.21.15.36:0/3390973299 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ca00b4120 0x7f6ca0146d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:07.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6ca545f700 1 --2- 172.21.15.36:0/3390973299 >> v2:172.21.15.70:3300/0 conn(0x7f6ca00b64f0 0x7f6ca0143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:07.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6c93fff700 1 -- 172.21.15.36:0/3390973299 >> v1:172.21.15.36:6789/0 conn(0x7f6ca0147270 legacy=0x7f6ca0140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:07.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6c93fff700 1 --2- 172.21.15.36:0/3390973299 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ca00b4120 0x7f6c84003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:07.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6c93fff700 1 --2- 172.21.15.36:0/3390973299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c84003c60 0x7f6c84004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:07.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6c93fff700 1 --2- 172.21.15.36:0/3390973299 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c840049f0 0x7f6c84004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:07.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6c93fff700 1 -- 172.21.15.36:0/3390973299 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca01476b0 con 0x7f6c840049f0 2022-06-17T12:55:07.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6c93fff700 1 -- 172.21.15.36:0/3390973299 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca00b6e20 con 0x7f6c84003c60 2022-06-17T12:55:07.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6c93fff700 1 -- 172.21.15.36:0/3390973299 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8400b750 con 0x7f6ca00b4120 2022-06-17T12:55:07.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6ca5c60700 1 --2- 172.21.15.36:0/3390973299 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c840049f0 0x7f6c84004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:07.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6ca545f700 1 --2- 172.21.15.36:0/3390973299 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ca00b4120 0x7f6c84003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:07.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6ca4c5e700 1 --2- 172.21.15.36:0/3390973299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c84003c60 0x7f6c84004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:07.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6ca5c60700 1 -- 172.21.15.36:0/3390973299 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ca00b4120 msgr2=0x7f6c84003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:07.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6ca5c60700 1 --2- 172.21.15.36:0/3390973299 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ca00b4120 0x7f6c84003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:07.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6ca5c60700 1 -- 172.21.15.36:0/3390973299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c84003c60 msgr2=0x7f6c84004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:07.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6ca5c60700 1 --2- 172.21.15.36:0/3390973299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c84003c60 0x7f6c84004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:07.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6ca5c60700 1 -- 172.21.15.36:0/3390973299 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9400b180 con 0x7f6c840049f0 2022-06-17T12:55:07.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6ca5c60700 1 --2- 172.21.15.36:0/3390973299 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c840049f0 0x7f6c84004e70 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7f6c84004470 tx=0x7f6ca00ae600).ready entity=mon.0 client_cookie=9faef3fdaf6ff0c3 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:07.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6c93fff700 1 -- 172.21.15.36:0/3390973299 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c9800cc40 con 0x7f6c840049f0 2022-06-17T12:55:07.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6c93fff700 1 -- 172.21.15.36:0/3390973299 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9800cda0 con 0x7f6c840049f0 2022-06-17T12:55:07.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.967+0000 7f6c93fff700 1 -- 172.21.15.36:0/3390973299 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c98005b80 con 0x7f6c840049f0 2022-06-17T12:55:07.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.971+0000 7f6ca76c3700 1 -- 172.21.15.36:0/3390973299 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c840049f0 msgr2=0x7f6c84004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:07.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.971+0000 7f6ca76c3700 1 --2- 172.21.15.36:0/3390973299 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6c840049f0 0x7f6c84004e70 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7f6c84004470 tx=0x7f6ca00ae600).stop 2022-06-17T12:55:07.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.971+0000 7f6ca5c60700 1 -- 172.21.15.36:0/3390973299 reap_dead start 2022-06-17T12:55:07.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.971+0000 7f6ca76c3700 1 -- 172.21.15.36:0/3390973299 shutdown_connections 2022-06-17T12:55:07.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.971+0000 7f6ca76c3700 1 -- 172.21.15.36:0/3390973299 >> 172.21.15.36:0/3390973299 conn(0x7f6ca00620f0 msgr2=0x7f6ca0062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:07.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.971+0000 7f6ca76c3700 1 -- 172.21.15.36:0/3390973299 shutdown_connections 2022-06-17T12:55:07.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.971+0000 7f6ca76c3700 1 -- 172.21.15.36:0/3390973299 wait complete. 2022-06-17T12:55:07.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.971+0000 7f6ca76c3700 1 Processor -- start 2022-06-17T12:55:07.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.971+0000 7f6ca76c3700 1 -- start start 2022-06-17T12:55:07.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.975+0000 7f6ca76c3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c840049f0 0x7f6ca00b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:07.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.975+0000 7f6ca76c3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ca00b2f00 0x7f6ca013f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:07.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.975+0000 7f6ca76c3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca013f640 0x7f6ca00aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:07.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.975+0000 7f6ca76c3700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca0142bc0 con 0x7f6ca013f640 2022-06-17T12:55:07.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.975+0000 7f6ca76c3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca0153030 con 0x7f6ca00b2f00 2022-06-17T12:55:07.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.975+0000 7f6ca76c3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca0141cf0 con 0x7f6c840049f0 2022-06-17T12:55:07.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.975+0000 7f6ca5c60700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca013f640 0x7f6ca00aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:07.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.975+0000 7f6ca5c60700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca013f640 0x7f6ca00aff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37938/0 (socket says 172.21.15.36:37938) 2022-06-17T12:55:07.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.975+0000 7f6ca5c60700 1 -- 172.21.15.36:0/1790982465 learned_addr learned my addr 172.21.15.36:0/1790982465 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:07.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.975+0000 7f6ca545f700 1 --2- 172.21.15.36:0/1790982465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c840049f0 0x7f6ca00b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:07.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.975+0000 7f6ca4c5e700 1 --2- 172.21.15.36:0/1790982465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ca00b2f00 0x7f6ca013f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:07.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.975+0000 7f6ca5c60700 1 -- 172.21.15.36:0/1790982465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c840049f0 msgr2=0x7f6ca00b2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:07.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.975+0000 7f6ca5c60700 1 --2- 172.21.15.36:0/1790982465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c840049f0 0x7f6ca00b2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:07.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.975+0000 7f6ca5c60700 1 -- 172.21.15.36:0/1790982465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ca00b2f00 msgr2=0x7f6ca013f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:07.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.975+0000 7f6ca5c60700 1 --2- 172.21.15.36:0/1790982465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ca00b2f00 0x7f6ca013f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:07.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.975+0000 7f6ca5c60700 1 -- 172.21.15.36:0/1790982465 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca013fac0 con 0x7f6ca013f640 2022-06-17T12:55:07.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.975+0000 7f6ca4c5e700 1 --2- 172.21.15.36:0/1790982465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ca00b2f00 0x7f6ca013f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:07.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.975+0000 7f6ca5c60700 1 --2- 172.21.15.36:0/1790982465 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca013f640 0x7f6ca00aff90 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7f6ca0069e20 tx=0x7f6c98015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:07.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.975+0000 7f6c927fc700 1 -- 172.21.15.36:0/1790982465 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c980072c0 con 0x7f6ca013f640 2022-06-17T12:55:07.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.975+0000 7f6c927fc700 1 -- 172.21.15.36:0/1790982465 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c98007420 con 0x7f6ca013f640 2022-06-17T12:55:07.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.975+0000 7f6ca76c3700 1 -- 172.21.15.36:0/1790982465 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca01532d0 con 0x7f6ca013f640 2022-06-17T12:55:07.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.975+0000 7f6c927fc700 1 -- 172.21.15.36:0/1790982465 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c98007780 con 0x7f6ca013f640 2022-06-17T12:55:07.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.975+0000 7f6ca76c3700 1 -- 172.21.15.36:0/1790982465 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca0142900 con 0x7f6ca013f640 2022-06-17T12:55:07.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.979+0000 7f6c927fc700 1 -- 172.21.15.36:0/1790982465 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6c98010070 con 0x7f6ca013f640 2022-06-17T12:55:07.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.979+0000 7f6ca76c3700 1 -- 172.21.15.36:0/1790982465 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca0152e60 con 0x7f6ca013f640 2022-06-17T12:55:07.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.979+0000 7f6c927fc700 1 --2- 172.21.15.36:0/1790982465 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c88053af0 0x7f6c88055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:07.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.979+0000 7f6c927fc700 1 -- 172.21.15.36:0/1790982465 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6c98019070 con 0x7f6ca013f640 2022-06-17T12:55:07.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.979+0000 7f6ca545f700 1 --2- 172.21.15.36:0/1790982465 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c88053af0 0x7f6c88055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:07.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.979+0000 7f6ca545f700 1 --2- 172.21.15.36:0/1790982465 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c88053af0 0x7f6c88055fb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f6c9c00b000 tx=0x7f6c9c006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:07.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:07.987+0000 7f6c927fc700 1 -- 172.21.15.36:0/1790982465 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6c98047910 con 0x7f6ca013f640 2022-06-17T12:55:08.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.107+0000 7f6ca76c3700 1 -- 172.21.15.36:0/1790982465 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6ca0143640 con 0x7f6ca013f640 2022-06-17T12:55:08.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.111+0000 7f6c927fc700 1 -- 172.21.15.36:0/1790982465 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v104) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6c98047500 con 0x7f6ca013f640 2022-06-17T12:55:08.099 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:55:08.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.111+0000 7f6ca76c3700 1 -- 172.21.15.36:0/1790982465 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c88053af0 msgr2=0x7f6c88055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:08.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.111+0000 7f6ca76c3700 1 --2- 172.21.15.36:0/1790982465 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c88053af0 0x7f6c88055fb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f6c9c00b000 tx=0x7f6c9c006df0).stop 2022-06-17T12:55:08.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.111+0000 7f6ca76c3700 1 -- 172.21.15.36:0/1790982465 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca013f640 msgr2=0x7f6ca00aff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:08.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.111+0000 7f6ca76c3700 1 --2- 172.21.15.36:0/1790982465 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca013f640 0x7f6ca00aff90 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7f6ca0069e20 tx=0x7f6c98015000).stop 2022-06-17T12:55:08.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.111+0000 7f6ca76c3700 1 -- 172.21.15.36:0/1790982465 shutdown_connections 2022-06-17T12:55:08.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.111+0000 7f6ca76c3700 1 --2- 172.21.15.36:0/1790982465 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c88053af0 0x7f6c88055fb0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.111+0000 7f6ca76c3700 1 --2- 172.21.15.36:0/1790982465 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca013f640 0x7f6ca00aff90 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.111+0000 7f6ca76c3700 1 --2- 172.21.15.36:0/1790982465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ca00b2f00 0x7f6ca013f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.111+0000 7f6ca76c3700 1 --2- 172.21.15.36:0/1790982465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c840049f0 0x7f6ca00b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.111+0000 7f6ca76c3700 1 -- 172.21.15.36:0/1790982465 >> 172.21.15.36:0/1790982465 conn(0x7f6ca00620f0 msgr2=0x7f6ca005f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:08.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.111+0000 7f6ca76c3700 1 -- 172.21.15.36:0/1790982465 shutdown_connections 2022-06-17T12:55:08.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.111+0000 7f6ca76c3700 1 -- 172.21.15.36:0/1790982465 wait complete. 2022-06-17T12:55:08.113 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:55:08.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.239+0000 7f56fd9bc700 1 Processor -- start 2022-06-17T12:55:08.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.239+0000 7f56fd9bc700 1 -- start start 2022-06-17T12:55:08.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.239+0000 7f56fd9bc700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f56f80b27b0 0x7f56f8146520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:08.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.239+0000 7f56fd9bc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56f8060ca0 0x7f56f8146a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:08.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.239+0000 7f56fd9bc700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f80b2b90 con 0x7f56f80aeac0 2022-06-17T12:55:08.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.239+0000 7f56fd9bc700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f8066b90 con 0x7f56f80b27b0 2022-06-17T12:55:08.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.239+0000 7f56fd9bc700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f80af610 con 0x7f56f8060ca0 2022-06-17T12:55:08.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.239+0000 7f56fd1bb700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f56f80aeac0 0x7f56f8142350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55472/0 (socket says 172.21.15.36:55472) 2022-06-17T12:55:08.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.239+0000 7f56fd1bb700 1 -- 172.21.15.36:0/3200171029 learned_addr learned my addr 172.21.15.36:0/3200171029 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:08.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.239+0000 7f56fc9ba700 1 --2- 172.21.15.36:0/3200171029 >> v2:172.21.15.70:3300/0 conn(0x7f56f80b27b0 0x7f56f8146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:08.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.239+0000 7f56f7fff700 1 --2- 172.21.15.36:0/3200171029 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56f8060ca0 0x7f56f8146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:08.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.239+0000 7f56f77fe700 1 -- 172.21.15.36:0/3200171029 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1357517271 0 0) 0x7f56f80b2b90 con 0x7f56f80aeac0 2022-06-17T12:55:08.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.239+0000 7f56f77fe700 1 -- 172.21.15.36:0/3200171029 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56d8003260 con 0x7f56f80aeac0 2022-06-17T12:55:08.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.239+0000 7f56fc9ba700 1 -- 172.21.15.36:0/3200171029 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56f8060ca0 msgr2=0x7f56f8146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:08.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.239+0000 7f56fc9ba700 1 --2- 172.21.15.36:0/3200171029 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56f8060ca0 0x7f56f8146a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.239+0000 7f56fc9ba700 1 -- 172.21.15.36:0/3200171029 >> v1:172.21.15.36:6789/0 conn(0x7f56f80aeac0 legacy=0x7f56f8142350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:08.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.239+0000 7f56fc9ba700 1 -- 172.21.15.36:0/3200171029 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f80620f0 con 0x7f56f80b27b0 2022-06-17T12:55:08.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.239+0000 7f56fc9ba700 1 --2- 172.21.15.36:0/3200171029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f80b27b0 0x7f56f8146520 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f56e8002f70 tx=0x7f56e8006df0).ready entity=mon.1 client_cookie=c33236d39e956c59 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:08.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.239+0000 7f56f77fe700 1 -- 172.21.15.36:0/3200171029 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56e800d4a0 con 0x7f56f80b27b0 2022-06-17T12:55:08.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.239+0000 7f56f77fe700 1 -- 172.21.15.36:0/3200171029 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e8000e90 con 0x7f56f80b27b0 2022-06-17T12:55:08.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.239+0000 7f56f77fe700 1 -- 172.21.15.36:0/3200171029 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56e800d7f0 con 0x7f56f80b27b0 2022-06-17T12:55:08.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.243+0000 7f56fd9bc700 1 -- 172.21.15.36:0/3200171029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f80b27b0 msgr2=0x7f56f8146520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:08.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.243+0000 7f56fd9bc700 1 --2- 172.21.15.36:0/3200171029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f80b27b0 0x7f56f8146520 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f56e8002f70 tx=0x7f56e8006df0).stop 2022-06-17T12:55:08.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.243+0000 7f56fd9bc700 1 -- 172.21.15.36:0/3200171029 shutdown_connections 2022-06-17T12:55:08.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.243+0000 7f56fd9bc700 1 --2- 172.21.15.36:0/3200171029 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56f8060ca0 0x7f56f8146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.243+0000 7f56fd9bc700 1 --2- 172.21.15.36:0/3200171029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f80b27b0 0x7f56f8146520 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.243+0000 7f56fd9bc700 1 -- 172.21.15.36:0/3200171029 >> 172.21.15.36:0/3200171029 conn(0x7f56f80b5d10 msgr2=0x7f56f80b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:08.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.243+0000 7f56fd9bc700 1 -- 172.21.15.36:0/3200171029 shutdown_connections 2022-06-17T12:55:08.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.243+0000 7f56fd9bc700 1 -- 172.21.15.36:0/3200171029 wait complete. 2022-06-17T12:55:08.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.243+0000 7f56fd9bc700 1 Processor -- start 2022-06-17T12:55:08.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.243+0000 7f56fd9bc700 1 -- start start 2022-06-17T12:55:08.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.243+0000 7f56fd9bc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f8060ca0 0x7f56f813d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:08.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.243+0000 7f56fd9bc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56f80b27b0 0x7f56f813d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:08.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.243+0000 7f56fd9bc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f56f813f560 0x7f56f813de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:08.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.243+0000 7f56fd9bc700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f8152d90 con 0x7f56f813f560 2022-06-17T12:55:08.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.243+0000 7f56fd9bc700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f8152850 con 0x7f56f8060ca0 2022-06-17T12:55:08.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.243+0000 7f56fd9bc700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f8152af0 con 0x7f56f80b27b0 2022-06-17T12:55:08.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.243+0000 7f56fd1bb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f56f813f560 0x7f56f813de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:08.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.243+0000 7f56fd1bb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f56f813f560 0x7f56f813de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37952/0 (socket says 172.21.15.36:37952) 2022-06-17T12:55:08.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.243+0000 7f56fd1bb700 1 -- 172.21.15.36:0/386418327 learned_addr learned my addr 172.21.15.36:0/386418327 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:08.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.243+0000 7f56f7fff700 1 --2- 172.21.15.36:0/386418327 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56f80b27b0 0x7f56f813d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:08.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.243+0000 7f56fc9ba700 1 --2- 172.21.15.36:0/386418327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f8060ca0 0x7f56f813d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:08.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.247+0000 7f56fd1bb700 1 -- 172.21.15.36:0/386418327 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56f80b27b0 msgr2=0x7f56f813d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:08.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.247+0000 7f56fd1bb700 1 --2- 172.21.15.36:0/386418327 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56f80b27b0 0x7f56f813d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.247+0000 7f56fd1bb700 1 -- 172.21.15.36:0/386418327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f8060ca0 msgr2=0x7f56f813d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:08.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.247+0000 7f56fd1bb700 1 --2- 172.21.15.36:0/386418327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f8060ca0 0x7f56f813d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.247+0000 7f56fd1bb700 1 -- 172.21.15.36:0/386418327 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e800b180 con 0x7f56f813f560 2022-06-17T12:55:08.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.247+0000 7f56fc9ba700 1 --2- 172.21.15.36:0/386418327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f8060ca0 0x7f56f813d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:08.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.247+0000 7f56fd1bb700 1 --2- 172.21.15.36:0/386418327 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f56f813f560 0x7f56f813de80 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7f56f0001490 tx=0x7f56f0007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:08.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.247+0000 7f56f5ffb700 1 -- 172.21.15.36:0/386418327 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56f0005c70 con 0x7f56f813f560 2022-06-17T12:55:08.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.247+0000 7f56f5ffb700 1 -- 172.21.15.36:0/386418327 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f0005dd0 con 0x7f56f813f560 2022-06-17T12:55:08.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.247+0000 7f56fd9bc700 1 -- 172.21.15.36:0/386418327 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f813db80 con 0x7f56f813f560 2022-06-17T12:55:08.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.247+0000 7f56f5ffb700 1 -- 172.21.15.36:0/386418327 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56f00072c0 con 0x7f56f813f560 2022-06-17T12:55:08.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.247+0000 7f56fd9bc700 1 -- 172.21.15.36:0/386418327 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f8146de0 con 0x7f56f813f560 2022-06-17T12:55:08.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.247+0000 7f56fd9bc700 1 -- 172.21.15.36:0/386418327 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f813f2a0 con 0x7f56f813f560 2022-06-17T12:55:08.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.247+0000 7f56f5ffb700 1 -- 172.21.15.36:0/386418327 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f56f0010070 con 0x7f56f813f560 2022-06-17T12:55:08.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.247+0000 7f56f5ffb700 1 --2- 172.21.15.36:0/386418327 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f56dc053a40 0x7f56dc055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:08.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.251+0000 7f56f5ffb700 1 -- 172.21.15.36:0/386418327 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f56f006bba0 con 0x7f56f813f560 2022-06-17T12:55:08.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.251+0000 7f56fc9ba700 1 --2- 172.21.15.36:0/386418327 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f56dc053a40 0x7f56dc055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:08.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.251+0000 7f56fc9ba700 1 --2- 172.21.15.36:0/386418327 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f56dc053a40 0x7f56dc055f00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f56e8018000 tx=0x7f56e8014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:08.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.259+0000 7f56f5ffb700 1 -- 172.21.15.36:0/386418327 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f56f813f2a0 con 0x7f56f813f560 2022-06-17T12:55:08.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.375+0000 7f56fd9bc700 1 -- 172.21.15.36:0/386418327 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f56f813f2a0 con 0x7f56f813f560 2022-06-17T12:55:08.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.375+0000 7f56f5ffb700 1 -- 172.21.15.36:0/386418327 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v104) v1 ==== 92+0+10 (secure 0 0 0) 0x7f56f813f2a0 con 0x7f56f813f560 2022-06-17T12:55:08.368 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:55:08.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.379+0000 7f56fd9bc700 1 -- 172.21.15.36:0/386418327 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f56dc053a40 msgr2=0x7f56dc055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:08.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.379+0000 7f56fd9bc700 1 --2- 172.21.15.36:0/386418327 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f56dc053a40 0x7f56dc055f00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f56e8018000 tx=0x7f56e8014000).stop 2022-06-17T12:55:08.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.379+0000 7f56fd9bc700 1 -- 172.21.15.36:0/386418327 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f56f813f560 msgr2=0x7f56f813de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:08.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.379+0000 7f56fd9bc700 1 --2- 172.21.15.36:0/386418327 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f56f813f560 0x7f56f813de80 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7f56f0001490 tx=0x7f56f0007dc0).stop 2022-06-17T12:55:08.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.379+0000 7f56fd9bc700 1 -- 172.21.15.36:0/386418327 shutdown_connections 2022-06-17T12:55:08.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.379+0000 7f56fd9bc700 1 --2- 172.21.15.36:0/386418327 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f56dc053a40 0x7f56dc055f00 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.379+0000 7f56fd9bc700 1 --2- 172.21.15.36:0/386418327 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f56f813f560 0x7f56f813de80 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.379+0000 7f56fd9bc700 1 --2- 172.21.15.36:0/386418327 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56f80b27b0 0x7f56f813d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.379+0000 7f56fd9bc700 1 --2- 172.21.15.36:0/386418327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f8060ca0 0x7f56f813d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.379+0000 7f56fd9bc700 1 -- 172.21.15.36:0/386418327 >> 172.21.15.36:0/386418327 conn(0x7f56f80b5d10 msgr2=0x7f56f8141700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:08.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.379+0000 7f56fd9bc700 1 -- 172.21.15.36:0/386418327 shutdown_connections 2022-06-17T12:55:08.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.379+0000 7f56fd9bc700 1 -- 172.21.15.36:0/386418327 wait complete. 2022-06-17T12:55:08.383 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:55:08.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.527+0000 7fadf0cf9700 1 Processor -- start 2022-06-17T12:55:08.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.527+0000 7fadf0cf9700 1 -- start start 2022-06-17T12:55:08.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadf0cf9700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fadec05fe80 0x7fadec060260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:08.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadf0cf9700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadec0b1db0 0x7fadec1437d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:08.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadf0cf9700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadec0633c0 con 0x7fadec0ae360 2022-06-17T12:55:08.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadf0cf9700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadec143330 con 0x7fadec05fe80 2022-06-17T12:55:08.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadf0cf9700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadec063100 con 0x7fadec0b1db0 2022-06-17T12:55:08.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadeaffd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fadec0ae360 0x7fadec1444f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55486/0 (socket says 172.21.15.36:55486) 2022-06-17T12:55:08.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadeaffd700 1 -- 172.21.15.36:0/3705374783 learned_addr learned my addr 172.21.15.36:0/3705374783 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:08.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadebfff700 1 --2- 172.21.15.36:0/3705374783 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadec0b1db0 0x7fadec1437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:08.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadeb7fe700 1 --2- 172.21.15.36:0/3705374783 >> v2:172.21.15.70:3300/0 conn(0x7fadec05fe80 0x7fadec060260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:08.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadea7fc700 1 -- 172.21.15.36:0/3705374783 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2442077716 0 0) 0x7fadec0633c0 con 0x7fadec0ae360 2022-06-17T12:55:08.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadea7fc700 1 -- 172.21.15.36:0/3705374783 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadcc003260 con 0x7fadec0ae360 2022-06-17T12:55:08.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadebfff700 1 -- 172.21.15.36:0/3705374783 >> v2:172.21.15.70:3300/0 conn(0x7fadec05fe80 msgr2=0x7fadec060260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:08.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadebfff700 1 --2- 172.21.15.36:0/3705374783 >> v2:172.21.15.70:3300/0 conn(0x7fadec05fe80 0x7fadec060260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadebfff700 1 -- 172.21.15.36:0/3705374783 >> v1:172.21.15.36:6789/0 conn(0x7fadec0ae360 legacy=0x7fadec1444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:08.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadea7fc700 1 -- 172.21.15.36:0/3705374783 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2311739950 0 0) 0x7fadcc003260 con 0x7fadec0ae360 2022-06-17T12:55:08.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadebfff700 1 -- 172.21.15.36:0/3705374783 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadec143020 con 0x7fadec0b1db0 2022-06-17T12:55:08.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadeb7fe700 1 --2- 172.21.15.36:0/3705374783 >> v2:172.21.15.70:3300/0 conn(0x7fadec05fe80 0x7fadec060260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:08.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadebfff700 1 --2- 172.21.15.36:0/3705374783 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadec0b1db0 0x7fadec1437d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fade40032f0 tx=0x7fade40091d0).ready entity=mon.2 client_cookie=956e49fda1c08099 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:08.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadea7fc700 1 -- 172.21.15.36:0/3705374783 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fade400b600 con 0x7fadec0b1db0 2022-06-17T12:55:08.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadea7fc700 1 -- 172.21.15.36:0/3705374783 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade400b760 con 0x7fadec0b1db0 2022-06-17T12:55:08.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadea7fc700 1 -- 172.21.15.36:0/3705374783 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fade400b8d0 con 0x7fadec0b1db0 2022-06-17T12:55:08.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadf0cf9700 1 -- 172.21.15.36:0/3705374783 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadec0b1db0 msgr2=0x7fadec1437d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:08.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadf0cf9700 1 --2- 172.21.15.36:0/3705374783 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadec0b1db0 0x7fadec1437d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fade40032f0 tx=0x7fade40091d0).stop 2022-06-17T12:55:08.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadf0cf9700 1 -- 172.21.15.36:0/3705374783 shutdown_connections 2022-06-17T12:55:08.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadf0cf9700 1 --2- 172.21.15.36:0/3705374783 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadec0b1db0 0x7fadec1437d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadf0cf9700 1 --2- 172.21.15.36:0/3705374783 >> v2:172.21.15.70:3300/0 conn(0x7fadec05fe80 0x7fadec060260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadf0cf9700 1 -- 172.21.15.36:0/3705374783 >> 172.21.15.36:0/3705374783 conn(0x7fadec060990 msgr2=0x7fadec05f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:08.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadf0cf9700 1 -- 172.21.15.36:0/3705374783 shutdown_connections 2022-06-17T12:55:08.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.531+0000 7fadf0cf9700 1 -- 172.21.15.36:0/3705374783 wait complete. 2022-06-17T12:55:08.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.535+0000 7fadf0cf9700 1 Processor -- start 2022-06-17T12:55:08.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.535+0000 7fadf0cf9700 1 -- start start 2022-06-17T12:55:08.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.535+0000 7fadf0cf9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadec05fe80 0x7fadec0b0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:08.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.535+0000 7fadf0cf9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadec0ae360 0x7fadec0b5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:08.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.535+0000 7fadf0cf9700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadec0b1db0 0x7fadec0ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:08.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.535+0000 7fadf0cf9700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec064dd0 con 0x7fadec05fe80 2022-06-17T12:55:08.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.535+0000 7fadf0cf9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec066540 con 0x7fadec0ae360 2022-06-17T12:55:08.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.535+0000 7fadf0cf9700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadec0aed80 con 0x7fadec0b1db0 2022-06-17T12:55:08.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.535+0000 7fadeb7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadec05fe80 0x7fadec0b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:08.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.535+0000 7fadeb7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadec05fe80 0x7fadec0b0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37962/0 (socket says 172.21.15.36:37962) 2022-06-17T12:55:08.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.535+0000 7fadeb7fe700 1 -- 172.21.15.36:0/1540226285 learned_addr learned my addr 172.21.15.36:0/1540226285 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:08.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.535+0000 7fadeaffd700 1 --2- 172.21.15.36:0/1540226285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadec0ae360 0x7fadec0b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:08.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.535+0000 7fadebfff700 1 --2- 172.21.15.36:0/1540226285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadec0b1db0 0x7fadec0ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:08.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.535+0000 7fadeb7fe700 1 -- 172.21.15.36:0/1540226285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadec0b1db0 msgr2=0x7fadec0ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:08.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.535+0000 7fadeb7fe700 1 --2- 172.21.15.36:0/1540226285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadec0b1db0 0x7fadec0ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.535+0000 7fadeb7fe700 1 -- 172.21.15.36:0/1540226285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadec0ae360 msgr2=0x7fadec0b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:08.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.535+0000 7fadeb7fe700 1 --2- 172.21.15.36:0/1540226285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadec0ae360 0x7fadec0b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.535+0000 7fadeb7fe700 1 -- 172.21.15.36:0/1540226285 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd400b180 con 0x7fadec05fe80 2022-06-17T12:55:08.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.535+0000 7fadeaffd700 1 --2- 172.21.15.36:0/1540226285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadec0ae360 0x7fadec0b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:08.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.535+0000 7fadeb7fe700 1 --2- 172.21.15.36:0/1540226285 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadec05fe80 0x7fadec0b0740 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 rx=0x7fadd400c000 tx=0x7fadd4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:08.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.539+0000 7fade8ff9700 1 -- 172.21.15.36:0/1540226285 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadd400ca40 con 0x7fadec05fe80 2022-06-17T12:55:08.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.539+0000 7fadf0cf9700 1 -- 172.21.15.36:0/1540226285 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade4009470 con 0x7fadec05fe80 2022-06-17T12:55:08.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.539+0000 7fade8ff9700 1 -- 172.21.15.36:0/1540226285 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd400cba0 con 0x7fadec05fe80 2022-06-17T12:55:08.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.539+0000 7fadf0cf9700 1 -- 172.21.15.36:0/1540226285 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadec0b1580 con 0x7fadec05fe80 2022-06-17T12:55:08.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.539+0000 7fade8ff9700 1 -- 172.21.15.36:0/1540226285 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadd401c460 con 0x7fadec05fe80 2022-06-17T12:55:08.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.539+0000 7fadf0cf9700 1 -- 172.21.15.36:0/1540226285 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadec057410 con 0x7fadec05fe80 2022-06-17T12:55:08.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.539+0000 7fade8ff9700 1 -- 172.21.15.36:0/1540226285 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fadd401c5c0 con 0x7fadec05fe80 2022-06-17T12:55:08.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.539+0000 7fade8ff9700 1 --2- 172.21.15.36:0/1540226285 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fadd0053a40 0x7fadd0055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:08.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.539+0000 7fade8ff9700 1 -- 172.21.15.36:0/1540226285 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fadd406c7e0 con 0x7fadec05fe80 2022-06-17T12:55:08.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.539+0000 7fadeaffd700 1 --2- 172.21.15.36:0/1540226285 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fadd0053a40 0x7fadd0055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:08.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.539+0000 7fadeaffd700 1 --2- 172.21.15.36:0/1540226285 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fadd0053a40 0x7fadd0055f00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7faddc00b000 tx=0x7faddc001660).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:08.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.547+0000 7fade8ff9700 1 -- 172.21.15.36:0/1540226285 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fadd40434e0 con 0x7fadec05fe80 2022-06-17T12:55:08.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.675+0000 7fadf0cf9700 1 -- 172.21.15.36:0/1540226285 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fadec0669a0 con 0x7fadec05fe80 2022-06-17T12:55:08.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.675+0000 7fade8ff9700 1 -- 172.21.15.36:0/1540226285 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v104) v1 ==== 104+0+10 (secure 0 0 0) 0x7fadd4046b00 con 0x7fadec05fe80 2022-06-17T12:55:08.666 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:55:08.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.679+0000 7fadf0cf9700 1 -- 172.21.15.36:0/1540226285 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fadd0053a40 msgr2=0x7fadd0055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:08.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.679+0000 7fadf0cf9700 1 --2- 172.21.15.36:0/1540226285 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fadd0053a40 0x7fadd0055f00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7faddc00b000 tx=0x7faddc001660).stop 2022-06-17T12:55:08.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.679+0000 7fadf0cf9700 1 -- 172.21.15.36:0/1540226285 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadec05fe80 msgr2=0x7fadec0b0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:08.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.679+0000 7fadf0cf9700 1 --2- 172.21.15.36:0/1540226285 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadec05fe80 0x7fadec0b0740 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 rx=0x7fadd400c000 tx=0x7fadd4006df0).stop 2022-06-17T12:55:08.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.679+0000 7fadf0cf9700 1 -- 172.21.15.36:0/1540226285 shutdown_connections 2022-06-17T12:55:08.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.679+0000 7fadf0cf9700 1 --2- 172.21.15.36:0/1540226285 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fadd0053a40 0x7fadd0055f00 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.679+0000 7fadf0cf9700 1 --2- 172.21.15.36:0/1540226285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fadec0b1db0 0x7fadec0ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.679+0000 7fadf0cf9700 1 --2- 172.21.15.36:0/1540226285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadec0ae360 0x7fadec0b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.679+0000 7fadf0cf9700 1 --2- 172.21.15.36:0/1540226285 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fadec05fe80 0x7fadec0b0740 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.679+0000 7fadf0cf9700 1 -- 172.21.15.36:0/1540226285 >> 172.21.15.36:0/1540226285 conn(0x7fadec060990 msgr2=0x7fadec145380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:08.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.679+0000 7fadf0cf9700 1 -- 172.21.15.36:0/1540226285 shutdown_connections 2022-06-17T12:55:08.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.679+0000 7fadf0cf9700 1 -- 172.21.15.36:0/1540226285 wait complete. 2022-06-17T12:55:08.679 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:55:08.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.799+0000 7f9cf63f6700 1 Processor -- start 2022-06-17T12:55:08.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9cf63f6700 1 -- start start 2022-06-17T12:55:08.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9cf63f6700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cf0062270 0x7f9cf0143d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:08.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9cf63f6700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf0143130 con 0x7f9cf00b4fc0 2022-06-17T12:55:08.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9cf63f6700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf0062650 con 0x7f9cf00b4190 2022-06-17T12:55:08.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9cf63f6700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf00b6080 con 0x7f9cf0062270 2022-06-17T12:55:08.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9cf4993700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f9cf00b4fc0 0x7f9cf01400f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55504/0 (socket says 172.21.15.36:55504) 2022-06-17T12:55:08.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9cf4993700 1 -- 172.21.15.36:0/2294185956 learned_addr learned my addr 172.21.15.36:0/2294185956 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:08.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9ceffff700 1 --2- 172.21.15.36:0/2294185956 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cf0062270 0x7f9cf0143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:08.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9ceeffd700 1 -- 172.21.15.36:0/2294185956 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2577481086 0 0) 0x7f9cf0143130 con 0x7f9cf00b4fc0 2022-06-17T12:55:08.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9ceeffd700 1 -- 172.21.15.36:0/2294185956 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cdc003260 con 0x7f9cf00b4fc0 2022-06-17T12:55:08.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9ceeffd700 1 -- 172.21.15.36:0/2294185956 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1224392630 0 0) 0x7f9cf0062650 con 0x7f9cf00b4190 2022-06-17T12:55:08.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9ceeffd700 1 -- 172.21.15.36:0/2294185956 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cf0143130 con 0x7f9cf00b4190 2022-06-17T12:55:08.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9ceeffd700 1 -- 172.21.15.36:0/2294185956 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1036481542 0 0) 0x7f9cdc003260 con 0x7f9cf00b4fc0 2022-06-17T12:55:08.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9ceeffd700 1 -- 172.21.15.36:0/2294185956 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cf0062270 msgr2=0x7f9cf0143d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:08.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9ceeffd700 1 --2- 172.21.15.36:0/2294185956 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cf0062270 0x7f9cf0143d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9ceeffd700 1 -- 172.21.15.36:0/2294185956 >> v1:172.21.15.70:6789/0 conn(0x7f9cf00b4190 legacy=0x7f9cf00b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:08.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9ceeffd700 1 -- 172.21.15.36:0/2294185956 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf0143360 con 0x7f9cf00b4fc0 2022-06-17T12:55:08.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9ceeffd700 1 -- 172.21.15.36:0/2294185956 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f9ce80019e0 con 0x7f9cf00b4fc0 2022-06-17T12:55:08.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9ceffff700 1 --2- 172.21.15.36:0/2294185956 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cf0062270 0x7f9cf0143d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:08.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9ceeffd700 1 -- 172.21.15.36:0/2294185956 >> v1:172.21.15.36:6789/0 conn(0x7f9cf00b4fc0 legacy=0x7f9cf01400f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:08.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9ceeffd700 1 --2- 172.21.15.36:0/2294185956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cdc003910 0x7f9cdc003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:08.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9ceeffd700 1 --2- 172.21.15.36:0/2294185956 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cf0062270 0x7f9cdc004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:08.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9ceeffd700 1 --2- 172.21.15.36:0/2294185956 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9cdc0049f0 0x7f9cdc004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:08.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9ceeffd700 1 -- 172.21.15.36:0/2294185956 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf00b6080 con 0x7f9cdc0049f0 2022-06-17T12:55:08.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9ceeffd700 1 -- 172.21.15.36:0/2294185956 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc00b750 con 0x7f9cdc003910 2022-06-17T12:55:08.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9ceeffd700 1 -- 172.21.15.36:0/2294185956 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc00b890 con 0x7f9cf0062270 2022-06-17T12:55:08.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9cf4993700 1 --2- 172.21.15.36:0/2294185956 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9cdc0049f0 0x7f9cdc004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:08.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9ceffff700 1 --2- 172.21.15.36:0/2294185956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cdc003910 0x7f9cdc003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:08.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9cef7fe700 1 --2- 172.21.15.36:0/2294185956 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cf0062270 0x7f9cdc004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:08.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9cf4993700 1 -- 172.21.15.36:0/2294185956 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cf0062270 msgr2=0x7f9cdc004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:08.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9cf4993700 1 --2- 172.21.15.36:0/2294185956 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cf0062270 0x7f9cdc004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9cf4993700 1 -- 172.21.15.36:0/2294185956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cdc003910 msgr2=0x7f9cdc003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:08.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9cf4993700 1 --2- 172.21.15.36:0/2294185956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cdc003910 0x7f9cdc003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9cf4993700 1 -- 172.21.15.36:0/2294185956 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce400b180 con 0x7f9cdc0049f0 2022-06-17T12:55:08.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9cf4993700 1 --2- 172.21.15.36:0/2294185956 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9cdc0049f0 0x7f9cdc004e70 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f9cdc004470 tx=0x7f9ce8008e40).ready entity=mon.0 client_cookie=ac5babcfbb6d9677 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:08.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9ceeffd700 1 -- 172.21.15.36:0/2294185956 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ce8005990 con 0x7f9cdc0049f0 2022-06-17T12:55:08.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9ceeffd700 1 -- 172.21.15.36:0/2294185956 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce8005af0 con 0x7f9cdc0049f0 2022-06-17T12:55:08.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.803+0000 7f9ceeffd700 1 -- 172.21.15.36:0/2294185956 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ce8005e50 con 0x7f9cdc0049f0 2022-06-17T12:55:08.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.807+0000 7f9cf63f6700 1 -- 172.21.15.36:0/2294185956 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9cdc0049f0 msgr2=0x7f9cdc004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:08.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.807+0000 7f9cf63f6700 1 --2- 172.21.15.36:0/2294185956 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9cdc0049f0 0x7f9cdc004e70 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f9cdc004470 tx=0x7f9ce8008e40).stop 2022-06-17T12:55:08.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.807+0000 7f9cf4993700 1 -- 172.21.15.36:0/2294185956 reap_dead start 2022-06-17T12:55:08.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.807+0000 7f9cf63f6700 1 -- 172.21.15.36:0/2294185956 shutdown_connections 2022-06-17T12:55:08.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.807+0000 7f9cf63f6700 1 -- 172.21.15.36:0/2294185956 >> 172.21.15.36:0/2294185956 conn(0x7f9cf00b97f0 msgr2=0x7f9cf00b9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:08.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.807+0000 7f9cf63f6700 1 -- 172.21.15.36:0/2294185956 shutdown_connections 2022-06-17T12:55:08.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.807+0000 7f9cf63f6700 1 -- 172.21.15.36:0/2294185956 wait complete. 2022-06-17T12:55:08.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.807+0000 7f9cf63f6700 1 Processor -- start 2022-06-17T12:55:08.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.807+0000 7f9cf63f6700 1 -- start start 2022-06-17T12:55:08.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.807+0000 7f9cf63f6700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cdc0049f0 0x7f9cf012af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:08.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.807+0000 7f9cf63f6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cf0145e30 0x7f9cf01471e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:08.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.807+0000 7f9cf63f6700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9cf0152ba0 0x7f9cf00b7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:08.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.807+0000 7f9cf63f6700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf0156720 con 0x7f9cf0152ba0 2022-06-17T12:55:08.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.807+0000 7f9cf63f6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf0152460 con 0x7f9cf0145e30 2022-06-17T12:55:08.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.807+0000 7f9cf63f6700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf0152700 con 0x7f9cdc0049f0 2022-06-17T12:55:08.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.811+0000 7f9cf4993700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9cf0152ba0 0x7f9cf00b7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:08.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.811+0000 7f9ceffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cdc0049f0 0x7f9cf012af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:08.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.811+0000 7f9cf4993700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9cf0152ba0 0x7f9cf00b7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37986/0 (socket says 172.21.15.36:37986) 2022-06-17T12:55:08.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.811+0000 7f9cf4993700 1 -- 172.21.15.36:0/363919168 learned_addr learned my addr 172.21.15.36:0/363919168 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:08.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.811+0000 7f9cef7fe700 1 --2- 172.21.15.36:0/363919168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cf0145e30 0x7f9cf01471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:08.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.811+0000 7f9ceffff700 1 -- 172.21.15.36:0/363919168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cf0145e30 msgr2=0x7f9cf01471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:08.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.811+0000 7f9ceffff700 1 --2- 172.21.15.36:0/363919168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cf0145e30 0x7f9cf01471e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.811+0000 7f9ceffff700 1 -- 172.21.15.36:0/363919168 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9cf0152ba0 msgr2=0x7f9cf00b7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:08.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.811+0000 7f9ceffff700 1 --2- 172.21.15.36:0/363919168 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9cf0152ba0 0x7f9cf00b7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.811+0000 7f9ceffff700 1 -- 172.21.15.36:0/363919168 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf01466d0 con 0x7f9cdc0049f0 2022-06-17T12:55:08.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.811+0000 7f9cf4993700 1 --2- 172.21.15.36:0/363919168 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9cf0152ba0 0x7f9cf00b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:08.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.811+0000 7f9ceffff700 1 --2- 172.21.15.36:0/363919168 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cdc0049f0 0x7f9cf012af70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f9ce400b000 tx=0x7f9ce4008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:08.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.811+0000 7f9ced7fa700 1 -- 172.21.15.36:0/363919168 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ce4014870 con 0x7f9cdc0049f0 2022-06-17T12:55:08.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.811+0000 7f9ced7fa700 1 -- 172.21.15.36:0/363919168 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce40149d0 con 0x7f9cdc0049f0 2022-06-17T12:55:08.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.811+0000 7f9cf63f6700 1 -- 172.21.15.36:0/363919168 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf01476d0 con 0x7f9cdc0049f0 2022-06-17T12:55:08.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.811+0000 7f9ced7fa700 1 -- 172.21.15.36:0/363919168 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ce4014d30 con 0x7f9cdc0049f0 2022-06-17T12:55:08.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.811+0000 7f9cf63f6700 1 -- 172.21.15.36:0/363919168 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf0156450 con 0x7f9cdc0049f0 2022-06-17T12:55:08.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.811+0000 7f9cf63f6700 1 -- 172.21.15.36:0/363919168 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf01528c0 con 0x7f9cdc0049f0 2022-06-17T12:55:08.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.811+0000 7f9ced7fa700 1 -- 172.21.15.36:0/363919168 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f9ce4006030 con 0x7f9cdc0049f0 2022-06-17T12:55:08.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.811+0000 7f9ced7fa700 1 --2- 172.21.15.36:0/363919168 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9cd00539b0 0x7f9cd0055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:08.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.811+0000 7f9ced7fa700 1 -- 172.21.15.36:0/363919168 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9ce406c930 con 0x7f9cdc0049f0 2022-06-17T12:55:08.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.811+0000 7f9cef7fe700 1 --2- 172.21.15.36:0/363919168 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9cd00539b0 0x7f9cd0055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:08.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.815+0000 7f9cef7fe700 1 --2- 172.21.15.36:0/363919168 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9cd00539b0 0x7f9cd0055e70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f9ce000d000 tx=0x7f9ce000c040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:08.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.823+0000 7f9ced7fa700 1 -- 172.21.15.36:0/363919168 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9ce4043730 con 0x7f9cdc0049f0 2022-06-17T12:55:08.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.947+0000 7f9cf63f6700 1 -- 172.21.15.36:0/363919168 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9cf0143600 con 0x7f9cdc0049f0 2022-06-17T12:55:08.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.947+0000 7f9ced7fa700 1 -- 172.21.15.36:0/363919168 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v104) v1 ==== 100+0+10 (secure 0 0 0) 0x7f9ce40180a0 con 0x7f9cdc0049f0 2022-06-17T12:55:08.938 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:55:08.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.951+0000 7f9cf63f6700 1 -- 172.21.15.36:0/363919168 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9cd00539b0 msgr2=0x7f9cd0055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:08.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.951+0000 7f9cf63f6700 1 --2- 172.21.15.36:0/363919168 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9cd00539b0 0x7f9cd0055e70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f9ce000d000 tx=0x7f9ce000c040).stop 2022-06-17T12:55:08.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.951+0000 7f9cf63f6700 1 -- 172.21.15.36:0/363919168 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cdc0049f0 msgr2=0x7f9cf012af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:08.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.951+0000 7f9cf63f6700 1 --2- 172.21.15.36:0/363919168 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cdc0049f0 0x7f9cf012af70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f9ce400b000 tx=0x7f9ce4008040).stop 2022-06-17T12:55:08.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.951+0000 7f9cf63f6700 1 -- 172.21.15.36:0/363919168 shutdown_connections 2022-06-17T12:55:08.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.951+0000 7f9cf63f6700 1 --2- 172.21.15.36:0/363919168 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9cd00539b0 0x7f9cd0055e70 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.951+0000 7f9cf63f6700 1 --2- 172.21.15.36:0/363919168 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9cf0152ba0 0x7f9cf00b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.951+0000 7f9cf63f6700 1 --2- 172.21.15.36:0/363919168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cf0145e30 0x7f9cf01471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.951+0000 7f9cf63f6700 1 --2- 172.21.15.36:0/363919168 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cdc0049f0 0x7f9cf012af70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:08.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.951+0000 7f9cf63f6700 1 -- 172.21.15.36:0/363919168 >> 172.21.15.36:0/363919168 conn(0x7f9cf00b97f0 msgr2=0x7f9cf00b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:08.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.951+0000 7f9cf63f6700 1 -- 172.21.15.36:0/363919168 shutdown_connections 2022-06-17T12:55:08.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:08.951+0000 7f9cf63f6700 1 -- 172.21.15.36:0/363919168 wait complete. 2022-06-17T12:55:08.951 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:55:08.952 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:55:09.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.071+0000 7fce4937d700 1 Processor -- start 2022-06-17T12:55:09.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.071+0000 7fce4937d700 1 -- start start 2022-06-17T12:55:09.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.071+0000 7fce4937d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fce440b7ce0 0x7fce440b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:09.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.071+0000 7fce4937d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce44140c90 0x7fce4413aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:09.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.071+0000 7fce4937d700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce44142160 con 0x7fce4413aef0 2022-06-17T12:55:09.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.071+0000 7fce4937d700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce44143d80 con 0x7fce440b7ce0 2022-06-17T12:55:09.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.071+0000 7fce4937d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce440b4f30 con 0x7fce44140c90 2022-06-17T12:55:09.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.071+0000 7fce437fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce44140c90 0x7fce4413aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:09.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.071+0000 7fce43fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fce440b7ce0 0x7fce440b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:09.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.071+0000 7fce48b7c700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fce4413aef0 0x7fce44144260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:51258/0 (socket says 172.21.15.36:51258) 2022-06-17T12:55:09.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.071+0000 7fce48b7c700 1 -- 172.21.15.36:0/1232427977 learned_addr learned my addr 172.21.15.36:0/1232427977 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:09.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.071+0000 7fce43fff700 1 -- 172.21.15.36:0/1232427977 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce44140c90 msgr2=0x7fce4413aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:09.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.071+0000 7fce43fff700 1 --2- 172.21.15.36:0/1232427977 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce44140c90 0x7fce4413aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:09.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.071+0000 7fce43fff700 1 -- 172.21.15.36:0/1232427977 >> v1:172.21.15.70:6789/0 conn(0x7fce4413aef0 legacy=0x7fce44144260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:09.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.071+0000 7fce42ffd700 1 -- 172.21.15.36:0/1232427977 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 6430074 0 0) 0x7fce44142160 con 0x7fce4413aef0 2022-06-17T12:55:09.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.071+0000 7fce43fff700 1 -- 172.21.15.36:0/1232427977 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce44142390 con 0x7fce440b7ce0 2022-06-17T12:55:09.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.071+0000 7fce437fe700 1 --2- 172.21.15.36:0/1232427977 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce44140c90 0x7fce4413aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:09.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.071+0000 7fce43fff700 1 --2- 172.21.15.36:0/1232427977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce440b7ce0 0x7fce440b80c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7fce34002f70 tx=0x7fce34006df0).ready entity=mon.1 client_cookie=55d162d1741923af server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:09.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce42ffd700 1 -- 172.21.15.36:0/1232427977 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce3400d570 con 0x7fce440b7ce0 2022-06-17T12:55:09.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce42ffd700 1 -- 172.21.15.36:0/1232427977 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce34000e90 con 0x7fce440b7ce0 2022-06-17T12:55:09.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce42ffd700 1 -- 172.21.15.36:0/1232427977 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce3400d8c0 con 0x7fce440b7ce0 2022-06-17T12:55:09.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce4937d700 1 -- 172.21.15.36:0/1232427977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce440b7ce0 msgr2=0x7fce440b80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:09.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce4937d700 1 --2- 172.21.15.36:0/1232427977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce440b7ce0 0x7fce440b80c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7fce34002f70 tx=0x7fce34006df0).stop 2022-06-17T12:55:09.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce4937d700 1 -- 172.21.15.36:0/1232427977 shutdown_connections 2022-06-17T12:55:09.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce4937d700 1 --2- 172.21.15.36:0/1232427977 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce44140c90 0x7fce4413aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:09.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce4937d700 1 --2- 172.21.15.36:0/1232427977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce440b7ce0 0x7fce440b80c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:09.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce4937d700 1 -- 172.21.15.36:0/1232427977 >> 172.21.15.36:0/1232427977 conn(0x7fce4405c550 msgr2=0x7fce440b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:09.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce4937d700 1 -- 172.21.15.36:0/1232427977 shutdown_connections 2022-06-17T12:55:09.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce4937d700 1 -- 172.21.15.36:0/1232427977 wait complete. 2022-06-17T12:55:09.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce4937d700 1 Processor -- start 2022-06-17T12:55:09.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce4937d700 1 -- start start 2022-06-17T12:55:09.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce4937d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce440b7ce0 0x7fce440b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:09.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce4937d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce4413aef0 0x7fce44064840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:09.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce4937d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce44140c90 0x7fce44155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:09.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce4937d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce44061b90 con 0x7fce440b7ce0 2022-06-17T12:55:09.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce4937d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce44066750 con 0x7fce44140c90 2022-06-17T12:55:09.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce4937d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce440669f0 con 0x7fce4413aef0 2022-06-17T12:55:09.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce43fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce440b7ce0 0x7fce440b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:09.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce43fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce440b7ce0 0x7fce440b1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:37996/0 (socket says 172.21.15.36:37996) 2022-06-17T12:55:09.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce43fff700 1 -- 172.21.15.36:0/139121484 learned_addr learned my addr 172.21.15.36:0/139121484 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:09.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce48b7c700 1 --2- 172.21.15.36:0/139121484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce44140c90 0x7fce44155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:09.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce437fe700 1 --2- 172.21.15.36:0/139121484 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce4413aef0 0x7fce44064840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:09.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce43fff700 1 -- 172.21.15.36:0/139121484 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce4413aef0 msgr2=0x7fce44064840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:09.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce43fff700 1 --2- 172.21.15.36:0/139121484 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce4413aef0 0x7fce44064840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:09.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce43fff700 1 -- 172.21.15.36:0/139121484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce44140c90 msgr2=0x7fce44155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:09.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce43fff700 1 --2- 172.21.15.36:0/139121484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce44140c90 0x7fce44155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:09.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce43fff700 1 -- 172.21.15.36:0/139121484 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c00b180 con 0x7fce440b7ce0 2022-06-17T12:55:09.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.075+0000 7fce43fff700 1 --2- 172.21.15.36:0/139121484 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce440b7ce0 0x7fce440b1330 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 rx=0x7fce34002f70 tx=0x7fce34014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:09.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.079+0000 7fce417fa700 1 -- 172.21.15.36:0/139121484 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce34016070 con 0x7fce440b7ce0 2022-06-17T12:55:09.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.079+0000 7fce417fa700 1 -- 172.21.15.36:0/139121484 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce340138d0 con 0x7fce440b7ce0 2022-06-17T12:55:09.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.079+0000 7fce4937d700 1 -- 172.21.15.36:0/139121484 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3400b180 con 0x7fce440b7ce0 2022-06-17T12:55:09.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.079+0000 7fce417fa700 1 -- 172.21.15.36:0/139121484 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce34013c30 con 0x7fce440b7ce0 2022-06-17T12:55:09.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.079+0000 7fce4937d700 1 -- 172.21.15.36:0/139121484 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce440afde0 con 0x7fce440b7ce0 2022-06-17T12:55:09.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.079+0000 7fce4937d700 1 -- 172.21.15.36:0/139121484 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce44142630 con 0x7fce440b7ce0 2022-06-17T12:55:09.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.079+0000 7fce417fa700 1 -- 172.21.15.36:0/139121484 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fce34025030 con 0x7fce440b7ce0 2022-06-17T12:55:09.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.079+0000 7fce417fa700 1 --2- 172.21.15.36:0/139121484 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce28053a40 0x7fce28055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:09.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.079+0000 7fce417fa700 1 -- 172.21.15.36:0/139121484 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fce3406ce70 con 0x7fce440b7ce0 2022-06-17T12:55:09.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.079+0000 7fce437fe700 1 --2- 172.21.15.36:0/139121484 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce28053a40 0x7fce28055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:09.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.083+0000 7fce437fe700 1 --2- 172.21.15.36:0/139121484 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce28053a40 0x7fce28055f00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7fce2c00b000 tx=0x7fce2c006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:09.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.087+0000 7fce417fa700 1 -- 172.21.15.36:0/139121484 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fce34043c70 con 0x7fce440b7ce0 2022-06-17T12:55:09.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.251+0000 7fce4937d700 1 -- 172.21.15.36:0/139121484 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce44057410 con 0x7fce440b7ce0 2022-06-17T12:55:09.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.255+0000 7fce417fa700 1 -- 172.21.15.36:0/139121484 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+2134 (secure 0 0 0) 0x7fce34023070 con 0x7fce440b7ce0 2022-06-17T12:55:09.243 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:09.244 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":54,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/676934608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":676934608},{"type":"v1","addr":"172.21.15.70:6833","nonce":676934608}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:54:59.995522+0000","modified":"2022-06-17T12:55:01.664559+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7395":{"gid":7395,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/233328233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":233328233},{"type":"v1","addr":"172.21.15.36:6835","nonce":233328233}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-06-17T12:55:09.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.255+0000 7fce4937d700 1 -- 172.21.15.36:0/139121484 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce28053a40 msgr2=0x7fce28055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:09.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.255+0000 7fce4937d700 1 --2- 172.21.15.36:0/139121484 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce28053a40 0x7fce28055f00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7fce2c00b000 tx=0x7fce2c006df0).stop 2022-06-17T12:55:09.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.255+0000 7fce4937d700 1 -- 172.21.15.36:0/139121484 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce440b7ce0 msgr2=0x7fce440b1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:09.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.255+0000 7fce4937d700 1 --2- 172.21.15.36:0/139121484 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce440b7ce0 0x7fce440b1330 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 rx=0x7fce34002f70 tx=0x7fce34014000).stop 2022-06-17T12:55:09.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.255+0000 7fce4937d700 1 -- 172.21.15.36:0/139121484 shutdown_connections 2022-06-17T12:55:09.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.255+0000 7fce4937d700 1 --2- 172.21.15.36:0/139121484 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce28053a40 0x7fce28055f00 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:09.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.255+0000 7fce4937d700 1 --2- 172.21.15.36:0/139121484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce44140c90 0x7fce44155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:09.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.255+0000 7fce4937d700 1 --2- 172.21.15.36:0/139121484 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce4413aef0 0x7fce44064840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:09.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.255+0000 7fce4937d700 1 --2- 172.21.15.36:0/139121484 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce440b7ce0 0x7fce440b1330 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:09.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.255+0000 7fce4937d700 1 -- 172.21.15.36:0/139121484 >> 172.21.15.36:0/139121484 conn(0x7fce4405c550 msgr2=0x7fce440b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:09.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.255+0000 7fce4937d700 1 -- 172.21.15.36:0/139121484 shutdown_connections 2022-06-17T12:55:09.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:09.255+0000 7fce4937d700 1 -- 172.21.15.36:0/139121484 wait complete. 2022-06-17T12:55:09.249 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 54 2022-06-17T12:55:09.462 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:54:59.995522+0000', 'modified': '2022-06-17T12:55:01.664559+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7395}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7395': {'gid': 7395, 'name': 'a', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.36:6835/233328233', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 233328233}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 233328233}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:55:09.462 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:55:09.463 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:55:09.582 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":7395,"want_state":"up:active","state":"up:active","rank_uptime":9.5901331970000001,"mdsmap_epoch":54,"osdmap_epoch":104,"osdmap_epoch_barrier":104,"uptime":14.149087024} 2022-06-17T12:55:09.584 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 7395, "mdsmap_epoch": 54, "osdmap_epoch": 104, "osdmap_epoch_barrier": 104, "rank_uptime": 9.590133197, "state": "up:active", "uptime": 14.149087024, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:55:09.585 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:55:09.585 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-06-17T12:55:09.586 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi136.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-06-17T12:55:09.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:09.588 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:55:09.599 DEBUG:teuthology.orchestra.run.smithi136:> sudo modprobe fuse 2022-06-17T12:55:09.659 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:55:09.660 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:55:09.672 INFO:teuthology.orchestra.run.smithi136.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:55:09.674 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:55:09.676 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:09.676 DEBUG:teuthology.orchestra.run.smithi136:> ls /sys/fs/fuse/connections 2022-06-17T12:55:09.685 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:55:09.686 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:55:09.728 DEBUG:teuthology.orchestra.run.smithi136:> sudo modprobe fuse 2022-06-17T12:55:09.748 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:55:09.748 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:55:09.759 INFO:teuthology.orchestra.run.smithi136.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:55:09.761 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:55:09.763 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:09.763 DEBUG:teuthology.orchestra.run.smithi136:> ls /sys/fs/fuse/connections 2022-06-17T12:55:09.802 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:2022-06-17T12:55:09.809+0000 7f1b4b105080 -1 init, newargv = 0x5628807166e0 newargc=9ceph-fuse[22143 2022-06-17T12:55:09.803 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:]: starting ceph client 2022-06-17T12:55:09.812 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:ceph-fuse[22143]: starting fuse 2022-06-17T12:55:10.872 DEBUG:teuthology.orchestra.run.smithi136:> sudo modprobe fuse 2022-06-17T12:55:10.891 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:55:10.891 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:55:10.948 INFO:teuthology.orchestra.run.smithi136.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:55:10.949 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:55:10.952 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:10.953 DEBUG:teuthology.orchestra.run.smithi136:> ls /sys/fs/fuse/connections 2022-06-17T12:55:10.959 INFO:teuthology.orchestra.run.smithi136.stdout:52 2022-06-17T12:55:10.960 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:55:10.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:10.961 DEBUG:teuthology.orchestra.run.smithi136:> sudo python3 -c ' 2022-06-17T12:55:10.961 DEBUG:teuthology.orchestra.run.smithi136:> import glob 2022-06-17T12:55:10.962 DEBUG:teuthology.orchestra.run.smithi136:> import re 2022-06-17T12:55:10.963 DEBUG:teuthology.orchestra.run.smithi136:> import os 2022-06-17T12:55:10.964 DEBUG:teuthology.orchestra.run.smithi136:> import subprocess 2022-06-17T12:55:10.965 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:55:10.965 DEBUG:teuthology.orchestra.run.smithi136:> def find_socket(client_name): 2022-06-17T12:55:10.966 DEBUG:teuthology.orchestra.run.smithi136:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-06-17T12:55:10.967 DEBUG:teuthology.orchestra.run.smithi136:> files = glob.glob(asok_path) 2022-06-17T12:55:10.967 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:55:10.968 DEBUG:teuthology.orchestra.run.smithi136:> # Given a non-glob path, it better be there 2022-06-17T12:55:10.969 DEBUG:teuthology.orchestra.run.smithi136:> if "*" not in asok_path: 2022-06-17T12:55:10.969 DEBUG:teuthology.orchestra.run.smithi136:> assert(len(files) == 1) 2022-06-17T12:55:10.970 DEBUG:teuthology.orchestra.run.smithi136:> return files[0] 2022-06-17T12:55:10.971 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:55:10.971 DEBUG:teuthology.orchestra.run.smithi136:> for f in files: 2022-06-17T12:55:10.972 DEBUG:teuthology.orchestra.run.smithi136:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:55:10.973 DEBUG:teuthology.orchestra.run.smithi136:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:55:10.973 DEBUG:teuthology.orchestra.run.smithi136:> return f 2022-06-17T12:55:10.974 DEBUG:teuthology.orchestra.run.smithi136:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:55:10.975 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:55:10.976 DEBUG:teuthology.orchestra.run.smithi136:> print(find_socket("client.1")) 2022-06-17T12:55:10.976 DEBUG:teuthology.orchestra.run.smithi136:> ' 2022-06-17T12:55:11.019 INFO:teuthology.orchestra.run.smithi136.stdout:/var/run/ceph/ceph-client.1.22143.asok 2022-06-17T12:55:11.025 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22143.asok 2022-06-17T12:55:11.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:11.027 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22143.asok status 2022-06-17T12:55:11.160 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2022-06-17T12:55:11.160 INFO:teuthology.orchestra.run.smithi136.stdout: "metadata": { 2022-06-17T12:55:11.160 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:55:11.161 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:55:11.161 INFO:teuthology.orchestra.run.smithi136.stdout: "entity_id": "1", 2022-06-17T12:55:11.161 INFO:teuthology.orchestra.run.smithi136.stdout: "hostname": "smithi136", 2022-06-17T12:55:11.161 INFO:teuthology.orchestra.run.smithi136.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-06-17T12:55:11.162 INFO:teuthology.orchestra.run.smithi136.stdout: "pid": "22143", 2022-06-17T12:55:11.162 INFO:teuthology.orchestra.run.smithi136.stdout: "root": "/" 2022-06-17T12:55:11.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:55:11.162 INFO:teuthology.orchestra.run.smithi136.stdout: "dentry_count": 0, 2022-06-17T12:55:11.163 INFO:teuthology.orchestra.run.smithi136.stdout: "dentry_pinned_count": 0, 2022-06-17T12:55:11.163 INFO:teuthology.orchestra.run.smithi136.stdout: "id": 6533, 2022-06-17T12:55:11.163 INFO:teuthology.orchestra.run.smithi136.stdout: "inst": { 2022-06-17T12:55:11.164 INFO:teuthology.orchestra.run.smithi136.stdout: "name": { 2022-06-17T12:55:11.165 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "client", 2022-06-17T12:55:11.165 INFO:teuthology.orchestra.run.smithi136.stdout: "num": 6533 2022-06-17T12:55:11.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:55:11.167 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": { 2022-06-17T12:55:11.167 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "v1", 2022-06-17T12:55:11.168 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": "172.21.15.136:0", 2022-06-17T12:55:11.168 INFO:teuthology.orchestra.run.smithi136.stdout: "nonce": 3877048751 2022-06-17T12:55:11.168 INFO:teuthology.orchestra.run.smithi136.stdout: } 2022-06-17T12:55:11.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:55:11.169 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": { 2022-06-17T12:55:11.169 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "v1", 2022-06-17T12:55:11.170 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": "172.21.15.136:0", 2022-06-17T12:55:11.170 INFO:teuthology.orchestra.run.smithi136.stdout: "nonce": 3877048751 2022-06-17T12:55:11.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:55:11.171 INFO:teuthology.orchestra.run.smithi136.stdout: "inst_str": "client.6533 172.21.15.136:0/3877048751", 2022-06-17T12:55:11.171 INFO:teuthology.orchestra.run.smithi136.stdout: "addr_str": "172.21.15.136:0/3877048751", 2022-06-17T12:55:11.172 INFO:teuthology.orchestra.run.smithi136.stdout: "inode_count": 1, 2022-06-17T12:55:11.172 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_epoch": 54, 2022-06-17T12:55:11.172 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_epoch": 104, 2022-06-17T12:55:11.173 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:55:11.173 INFO:teuthology.orchestra.run.smithi136.stdout: "blacklisted": false, 2022-06-17T12:55:11.174 INFO:teuthology.orchestra.run.smithi136.stdout: "fs_name": "cephfs" 2022-06-17T12:55:11.176 INFO:teuthology.orchestra.run.smithi136.stdout:} 2022-06-17T12:55:11.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:11.179 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:55:11.180 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:55:11.189 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:55:11.190 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:55:11.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:11.191 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:55:11.209 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2022-06-17T12:55:11.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b2c7b700 1 Processor -- start 2022-06-17T12:55:11.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b2c7b700 1 -- start start 2022-06-17T12:55:11.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b2c7b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39ac0b27b0 0x7f39ac14b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:11.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b2c7b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f39ac060ca0 0x7f39ac0ac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:11.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b2c7b700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39ac14b870 con 0x7f39ac147bd0 2022-06-17T12:55:11.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b2c7b700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39ac0ac500 con 0x7f39ac060ca0 2022-06-17T12:55:11.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b2c7b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac0b2b90 con 0x7f39ac0b27b0 2022-06-17T12:55:11.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b1478700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f39ac060ca0 0x7f39ac0ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:11.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b1478700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f39ac060ca0 0x7f39ac0ac780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38004/0 (socket says 172.21.15.36:38004) 2022-06-17T12:55:11.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b1478700 1 -- 172.21.15.36:0/1821576640 learned_addr learned my addr 172.21.15.36:0/1821576640 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:11.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b1c79700 1 --2- 172.21.15.36:0/1821576640 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39ac0b27b0 0x7f39ac14b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:11.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b1478700 1 -- 172.21.15.36:0/1821576640 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39ac0b27b0 msgr2=0x7f39ac14b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:11.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b1478700 1 --2- 172.21.15.36:0/1821576640 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39ac0b27b0 0x7f39ac14b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:11.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b1478700 1 -- 172.21.15.36:0/1821576640 >> v1:172.21.15.70:6789/0 conn(0x7f39ac147bd0 legacy=0x7f39ac1442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:11.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b1478700 1 -- 172.21.15.36:0/1821576640 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ac147490 con 0x7f39ac060ca0 2022-06-17T12:55:11.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b1c79700 1 --2- 172.21.15.36:0/1821576640 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39ac0b27b0 0x7f39ac14b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:55:11.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b1478700 1 --2- 172.21.15.36:0/1821576640 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f39ac060ca0 0x7f39ac0ac780 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7f39ac147ab0 tx=0x7f399c006eb0).ready entity=mon.0 client_cookie=2c9fca8daad7a679 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:11.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b0c77700 1 -- 172.21.15.36:0/1821576640 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f399c00d570 con 0x7f39ac060ca0 2022-06-17T12:55:11.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b0c77700 1 -- 172.21.15.36:0/1821576640 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399c000e90 con 0x7f39ac060ca0 2022-06-17T12:55:11.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b0c77700 1 -- 172.21.15.36:0/1821576640 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f399c00d8c0 con 0x7f39ac060ca0 2022-06-17T12:55:11.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b2c7b700 1 -- 172.21.15.36:0/1821576640 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f39ac060ca0 msgr2=0x7f39ac0ac780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:11.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b2c7b700 1 --2- 172.21.15.36:0/1821576640 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f39ac060ca0 0x7f39ac0ac780 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7f39ac147ab0 tx=0x7f399c006eb0).stop 2022-06-17T12:55:11.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b2c7b700 1 -- 172.21.15.36:0/1821576640 shutdown_connections 2022-06-17T12:55:11.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b2c7b700 1 --2- 172.21.15.36:0/1821576640 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f39ac060ca0 0x7f39ac0ac780 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:11.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b2c7b700 1 --2- 172.21.15.36:0/1821576640 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39ac0b27b0 0x7f39ac14b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:11.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b2c7b700 1 -- 172.21.15.36:0/1821576640 >> 172.21.15.36:0/1821576640 conn(0x7f39ac0b5d10 msgr2=0x7f39ac0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:11.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b2c7b700 1 -- 172.21.15.36:0/1821576640 shutdown_connections 2022-06-17T12:55:11.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.335+0000 7f39b2c7b700 1 -- 172.21.15.36:0/1821576640 wait complete. 2022-06-17T12:55:11.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.339+0000 7f39b2c7b700 1 Processor -- start 2022-06-17T12:55:11.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.339+0000 7f39b2c7b700 1 -- start start 2022-06-17T12:55:11.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.339+0000 7f39b2c7b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39ac060ca0 0x7f39ac066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:11.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.339+0000 7f39b2c7b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39ac0b27b0 0x7f39ac066fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:11.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.339+0000 7f39b2c7b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f39ac147bd0 0x7f39ac144400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:11.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.339+0000 7f39b2c7b700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac0641d0 con 0x7f39ac147bd0 2022-06-17T12:55:11.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.339+0000 7f39b2c7b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac064470 con 0x7f39ac0b27b0 2022-06-17T12:55:11.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.339+0000 7f39b2c7b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac062e00 con 0x7f39ac060ca0 2022-06-17T12:55:11.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.339+0000 7f39b247a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f39ac147bd0 0x7f39ac144400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:11.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.339+0000 7f39b247a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f39ac147bd0 0x7f39ac144400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38014/0 (socket says 172.21.15.36:38014) 2022-06-17T12:55:11.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.339+0000 7f39b247a700 1 -- 172.21.15.36:0/3036741229 learned_addr learned my addr 172.21.15.36:0/3036741229 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:11.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.339+0000 7f39b1c79700 1 --2- 172.21.15.36:0/3036741229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39ac060ca0 0x7f39ac066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:11.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.339+0000 7f39b1478700 1 --2- 172.21.15.36:0/3036741229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39ac0b27b0 0x7f39ac066fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:11.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.343+0000 7f39b247a700 1 -- 172.21.15.36:0/3036741229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39ac060ca0 msgr2=0x7f39ac066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:11.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.343+0000 7f39b247a700 1 --2- 172.21.15.36:0/3036741229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39ac060ca0 0x7f39ac066ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:11.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.343+0000 7f39b247a700 1 -- 172.21.15.36:0/3036741229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39ac0b27b0 msgr2=0x7f39ac066fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:11.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.343+0000 7f39b247a700 1 --2- 172.21.15.36:0/3036741229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39ac0b27b0 0x7f39ac066fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:11.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.343+0000 7f39b247a700 1 -- 172.21.15.36:0/3036741229 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c00b1e0 con 0x7f39ac147bd0 2022-06-17T12:55:11.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.343+0000 7f39b247a700 1 --2- 172.21.15.36:0/3036741229 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f39ac147bd0 0x7f39ac144400 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0x7f39a8001fa0 tx=0x7f39a8007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:11.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.343+0000 7f399affd700 1 -- 172.21.15.36:0/3036741229 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39a8002da0 con 0x7f39ac147bd0 2022-06-17T12:55:11.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.343+0000 7f399affd700 1 -- 172.21.15.36:0/3036741229 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a8007030 con 0x7f39ac147bd0 2022-06-17T12:55:11.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.343+0000 7f39b2c7b700 1 -- 172.21.15.36:0/3036741229 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39ac063120 con 0x7f39ac147bd0 2022-06-17T12:55:11.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.343+0000 7f399affd700 1 -- 172.21.15.36:0/3036741229 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39a8007390 con 0x7f39ac147bd0 2022-06-17T12:55:11.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.343+0000 7f39b2c7b700 1 -- 172.21.15.36:0/3036741229 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ac144a90 con 0x7f39ac147bd0 2022-06-17T12:55:11.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.343+0000 7f39b2c7b700 1 -- 172.21.15.36:0/3036741229 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39ac065400 con 0x7f39ac147bd0 2022-06-17T12:55:11.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.343+0000 7f399affd700 1 -- 172.21.15.36:0/3036741229 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f39a8010030 con 0x7f39ac147bd0 2022-06-17T12:55:11.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.347+0000 7f399affd700 1 --2- 172.21.15.36:0/3036741229 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f398c0539b0 0x7f398c055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:11.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.347+0000 7f399affd700 1 -- 172.21.15.36:0/3036741229 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f39a806bb00 con 0x7f39ac147bd0 2022-06-17T12:55:11.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.347+0000 7f39b1c79700 1 --2- 172.21.15.36:0/3036741229 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f398c0539b0 0x7f398c055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:11.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.347+0000 7f39b1c79700 1 --2- 172.21.15.36:0/3036741229 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f398c0539b0 0x7f398c055e70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f39a400d000 tx=0x7f39a4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:11.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.351+0000 7f399affd700 1 -- 172.21.15.36:0/3036741229 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f39a8042900 con 0x7f39ac147bd0 2022-06-17T12:55:11.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.519+0000 7f39b2c7b700 1 -- 172.21.15.36:0/3036741229 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f39ac057410 con 0x7f39ac147bd0 2022-06-17T12:55:11.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.519+0000 7f399affd700 1 -- 172.21.15.36:0/3036741229 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1936 (secure 0 0 0) 0x7f39a8045f20 con 0x7f39ac147bd0 2022-06-17T12:55:11.509 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:11.509 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:55:11.509 INFO:teuthology.orchestra.run.smithi036.stdout: "stats": { 2022-06-17T12:55:11.510 INFO:teuthology.orchestra.run.smithi036.stdout: "total_bytes": 773094113280, 2022-06-17T12:55:11.510 INFO:teuthology.orchestra.run.smithi036.stdout: "total_avail_bytes": 764493660160, 2022-06-17T12:55:11.510 INFO:teuthology.orchestra.run.smithi036.stdout: "total_used_bytes": 10518528, 2022-06-17T12:55:11.511 INFO:teuthology.orchestra.run.smithi036.stdout: "total_used_raw_bytes": 8600453120, 2022-06-17T12:55:11.511 INFO:teuthology.orchestra.run.smithi036.stdout: "total_used_raw_ratio": 0.011124717071652412, 2022-06-17T12:55:11.512 INFO:teuthology.orchestra.run.smithi036.stdout: "num_osds": 8, 2022-06-17T12:55:11.512 INFO:teuthology.orchestra.run.smithi036.stdout: "num_per_pool_osds": 8, 2022-06-17T12:55:11.512 INFO:teuthology.orchestra.run.smithi036.stdout: "num_per_pool_omap_osds": 8 2022-06-17T12:55:11.513 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:11.513 INFO:teuthology.orchestra.run.smithi036.stdout: "stats_by_class": { 2022-06-17T12:55:11.513 INFO:teuthology.orchestra.run.smithi036.stdout: "ssd": { 2022-06-17T12:55:11.514 INFO:teuthology.orchestra.run.smithi036.stdout: "total_bytes": 773094113280, 2022-06-17T12:55:11.514 INFO:teuthology.orchestra.run.smithi036.stdout: "total_avail_bytes": 764493660160, 2022-06-17T12:55:11.515 INFO:teuthology.orchestra.run.smithi036.stdout: "total_used_bytes": 10518528, 2022-06-17T12:55:11.515 INFO:teuthology.orchestra.run.smithi036.stdout: "total_used_raw_bytes": 8600453120, 2022-06-17T12:55:11.515 INFO:teuthology.orchestra.run.smithi036.stdout: "total_used_raw_ratio": 0.011124717071652412 2022-06-17T12:55:11.516 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:11.516 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:11.517 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:55:11.517 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:11.517 INFO:teuthology.orchestra.run.smithi036.stdout: "name": "device_health_metrics", 2022-06-17T12:55:11.518 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 1, 2022-06-17T12:55:11.518 INFO:teuthology.orchestra.run.smithi036.stdout: "stats": { 2022-06-17T12:55:11.518 INFO:teuthology.orchestra.run.smithi036.stdout: "stored": 0, 2022-06-17T12:55:11.519 INFO:teuthology.orchestra.run.smithi036.stdout: "objects": 0, 2022-06-17T12:55:11.519 INFO:teuthology.orchestra.run.smithi036.stdout: "kb_used": 0, 2022-06-17T12:55:11.519 INFO:teuthology.orchestra.run.smithi036.stdout: "bytes_used": 0, 2022-06-17T12:55:11.520 INFO:teuthology.orchestra.run.smithi036.stdout: "percent_used": 0, 2022-06-17T12:55:11.520 INFO:teuthology.orchestra.run.smithi036.stdout: "max_avail": 362919428096 2022-06-17T12:55:11.521 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:11.521 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:11.521 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:11.522 INFO:teuthology.orchestra.run.smithi036.stdout: "name": "rbd", 2022-06-17T12:55:11.522 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 2, 2022-06-17T12:55:11.523 INFO:teuthology.orchestra.run.smithi036.stdout: "stats": { 2022-06-17T12:55:11.523 INFO:teuthology.orchestra.run.smithi036.stdout: "stored": 0, 2022-06-17T12:55:11.523 INFO:teuthology.orchestra.run.smithi036.stdout: "objects": 0, 2022-06-17T12:55:11.524 INFO:teuthology.orchestra.run.smithi036.stdout: "kb_used": 0, 2022-06-17T12:55:11.524 INFO:teuthology.orchestra.run.smithi036.stdout: "bytes_used": 0, 2022-06-17T12:55:11.524 INFO:teuthology.orchestra.run.smithi036.stdout: "percent_used": 0, 2022-06-17T12:55:11.525 INFO:teuthology.orchestra.run.smithi036.stdout: "max_avail": 362919428096 2022-06-17T12:55:11.525 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:11.525 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:11.526 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:11.526 INFO:teuthology.orchestra.run.smithi036.stdout: "name": "cephfs_metadata", 2022-06-17T12:55:11.527 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 19, 2022-06-17T12:55:11.527 INFO:teuthology.orchestra.run.smithi036.stdout: "stats": { 2022-06-17T12:55:11.527 INFO:teuthology.orchestra.run.smithi036.stdout: "stored": 2523, 2022-06-17T12:55:11.528 INFO:teuthology.orchestra.run.smithi036.stdout: "objects": 22, 2022-06-17T12:55:11.529 INFO:teuthology.orchestra.run.smithi036.stdout: "kb_used": 64, 2022-06-17T12:55:11.529 INFO:teuthology.orchestra.run.smithi036.stdout: "bytes_used": 65536, 2022-06-17T12:55:11.529 INFO:teuthology.orchestra.run.smithi036.stdout: "percent_used": 9.0290008358806517e-08, 2022-06-17T12:55:11.530 INFO:teuthology.orchestra.run.smithi036.stdout: "max_avail": 362919428096 2022-06-17T12:55:11.530 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:11.530 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:11.531 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:11.531 INFO:teuthology.orchestra.run.smithi036.stdout: "name": "cephfs_data", 2022-06-17T12:55:11.531 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 20, 2022-06-17T12:55:11.531 INFO:teuthology.orchestra.run.smithi036.stdout: "stats": { 2022-06-17T12:55:11.532 INFO:teuthology.orchestra.run.smithi036.stdout: "stored": 0, 2022-06-17T12:55:11.532 INFO:teuthology.orchestra.run.smithi036.stdout: "objects": 0, 2022-06-17T12:55:11.532 INFO:teuthology.orchestra.run.smithi036.stdout: "kb_used": 0, 2022-06-17T12:55:11.533 INFO:teuthology.orchestra.run.smithi036.stdout: "bytes_used": 0, 2022-06-17T12:55:11.533 INFO:teuthology.orchestra.run.smithi036.stdout: "percent_used": 0, 2022-06-17T12:55:11.533 INFO:teuthology.orchestra.run.smithi036.stdout: "max_avail": 362919428096 2022-06-17T12:55:11.534 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:11.534 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:11.534 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:11.535 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:55:11.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.523+0000 7f39b2c7b700 1 -- 172.21.15.36:0/3036741229 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f398c0539b0 msgr2=0x7f398c055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:11.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.523+0000 7f39b2c7b700 1 --2- 172.21.15.36:0/3036741229 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f398c0539b0 0x7f398c055e70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f39a400d000 tx=0x7f39a4008040).stop 2022-06-17T12:55:11.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.523+0000 7f39b2c7b700 1 -- 172.21.15.36:0/3036741229 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f39ac147bd0 msgr2=0x7f39ac144400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:11.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.523+0000 7f39b2c7b700 1 --2- 172.21.15.36:0/3036741229 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f39ac147bd0 0x7f39ac144400 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0x7f39a8001fa0 tx=0x7f39a8007c30).stop 2022-06-17T12:55:11.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.523+0000 7f39b2c7b700 1 -- 172.21.15.36:0/3036741229 shutdown_connections 2022-06-17T12:55:11.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.523+0000 7f39b2c7b700 1 --2- 172.21.15.36:0/3036741229 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f398c0539b0 0x7f398c055e70 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:11.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.523+0000 7f39b2c7b700 1 --2- 172.21.15.36:0/3036741229 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f39ac147bd0 0x7f39ac144400 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:11.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.523+0000 7f39b2c7b700 1 --2- 172.21.15.36:0/3036741229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39ac0b27b0 0x7f39ac066fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:11.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.523+0000 7f39b2c7b700 1 --2- 172.21.15.36:0/3036741229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39ac060ca0 0x7f39ac066ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:11.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.523+0000 7f39b2c7b700 1 -- 172.21.15.36:0/3036741229 >> 172.21.15.36:0/3036741229 conn(0x7f39ac0b5d10 msgr2=0x7f39ac146930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:11.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.523+0000 7f39b2c7b700 1 -- 172.21.15.36:0/3036741229 shutdown_connections 2022-06-17T12:55:11.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.523+0000 7f39b2c7b700 1 -- 172.21.15.36:0/3036741229 wait complete. 2022-06-17T12:55:11.545 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data size -f json-pretty 2022-06-17T12:55:11.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.675+0000 7f4fd21a2700 1 Processor -- start 2022-06-17T12:55:11.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.675+0000 7f4fd21a2700 1 -- start start 2022-06-17T12:55:11.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.675+0000 7f4fd21a2700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4fcc0b9860 0x7f4fcc143840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:11.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.675+0000 7f4fd21a2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fcc0b45c0 0x7f4fcc0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:11.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.675+0000 7f4fd21a2700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4fcc0b4f00 0x7f4fcc146490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:11.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.675+0000 7f4fd21a2700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fcc0b4160 con 0x7f4fcc0b4f00 2022-06-17T12:55:11.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.675+0000 7f4fd21a2700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fcc0b9c40 con 0x7f4fcc0b9860 2022-06-17T12:55:11.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.675+0000 7f4fd21a2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc0627d0 con 0x7f4fcc0b45c0 2022-06-17T12:55:11.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.675+0000 7f4fd19a1700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4fcc0b4f00 0x7f4fcc146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:11.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.675+0000 7f4fd19a1700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4fcc0b4f00 0x7f4fcc146490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38022/0 (socket says 172.21.15.36:38022) 2022-06-17T12:55:11.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.675+0000 7f4fd19a1700 1 -- 172.21.15.36:0/3624779254 learned_addr learned my addr 172.21.15.36:0/3624779254 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:11.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.675+0000 7f4fd11a0700 1 --2- 172.21.15.36:0/3624779254 >> v2:172.21.15.70:3300/0 conn(0x7f4fcc0b9860 0x7f4fcc143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:11.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.675+0000 7f4fd099f700 1 --2- 172.21.15.36:0/3624779254 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fcc0b45c0 0x7f4fcc0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:11.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd099f700 1 -- 172.21.15.36:0/3624779254 >> v2:172.21.15.70:3300/0 conn(0x7f4fcc0b9860 msgr2=0x7f4fcc143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:11.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd099f700 1 --2- 172.21.15.36:0/3624779254 >> v2:172.21.15.70:3300/0 conn(0x7f4fcc0b9860 0x7f4fcc143840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:11.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd099f700 1 -- 172.21.15.36:0/3624779254 >> v2:172.21.15.36:3300/0 conn(0x7f4fcc0b4f00 msgr2=0x7f4fcc146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:11.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd099f700 1 --2- 172.21.15.36:0/3624779254 >> v2:172.21.15.36:3300/0 conn(0x7f4fcc0b4f00 0x7f4fcc146490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:11.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd099f700 1 -- 172.21.15.36:0/3624779254 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc062a70 con 0x7f4fcc0b45c0 2022-06-17T12:55:11.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd19a1700 1 --2- 172.21.15.36:0/3624779254 >> v2:172.21.15.36:3300/0 conn(0x7f4fcc0b4f00 0x7f4fcc146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:11.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd099f700 1 --2- 172.21.15.36:0/3624779254 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fcc0b45c0 0x7f4fcc0b4a10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f4fc0002f70 tx=0x7f4fc0006df0).ready entity=mon.2 client_cookie=d1a441e3a7359c3 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:11.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fb7fff700 1 -- 172.21.15.36:0/3624779254 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fc000d4a0 con 0x7f4fcc0b45c0 2022-06-17T12:55:11.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fb7fff700 1 -- 172.21.15.36:0/3624779254 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc0000e90 con 0x7f4fcc0b45c0 2022-06-17T12:55:11.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fb7fff700 1 -- 172.21.15.36:0/3624779254 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fc000d7f0 con 0x7f4fcc0b45c0 2022-06-17T12:55:11.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd21a2700 1 -- 172.21.15.36:0/3624779254 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fcc0b45c0 msgr2=0x7f4fcc0b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:11.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd21a2700 1 --2- 172.21.15.36:0/3624779254 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fcc0b45c0 0x7f4fcc0b4a10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f4fc0002f70 tx=0x7f4fc0006df0).stop 2022-06-17T12:55:11.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd21a2700 1 -- 172.21.15.36:0/3624779254 shutdown_connections 2022-06-17T12:55:11.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd21a2700 1 --2- 172.21.15.36:0/3624779254 >> v2:172.21.15.36:3300/0 conn(0x7f4fcc0b4f00 0x7f4fcc146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:11.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd21a2700 1 --2- 172.21.15.36:0/3624779254 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fcc0b45c0 0x7f4fcc0b4a10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:11.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd21a2700 1 --2- 172.21.15.36:0/3624779254 >> v2:172.21.15.70:3300/0 conn(0x7f4fcc0b9860 0x7f4fcc143840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:11.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd21a2700 1 -- 172.21.15.36:0/3624779254 >> 172.21.15.36:0/3624779254 conn(0x7f4fcc05c550 msgr2=0x7f4fcc060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:11.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd21a2700 1 -- 172.21.15.36:0/3624779254 shutdown_connections 2022-06-17T12:55:11.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd21a2700 1 -- 172.21.15.36:0/3624779254 wait complete. 2022-06-17T12:55:11.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd21a2700 1 Processor -- start 2022-06-17T12:55:11.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd21a2700 1 -- start start 2022-06-17T12:55:11.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd21a2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fcc0b45c0 0x7f4fcc155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:11.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd21a2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fcc0b4f00 0x7f4fcc1560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:11.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd21a2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fcc0b9860 0x7f4fcc153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:11.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd21a2700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc156770 con 0x7f4fcc0b9860 2022-06-17T12:55:11.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd21a2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc0ae2e0 con 0x7f4fcc0b4f00 2022-06-17T12:55:11.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd21a2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc0ae580 con 0x7f4fcc0b45c0 2022-06-17T12:55:11.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd19a1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fcc0b9860 0x7f4fcc153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:11.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd19a1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fcc0b9860 0x7f4fcc153790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38028/0 (socket says 172.21.15.36:38028) 2022-06-17T12:55:11.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd19a1700 1 -- 172.21.15.36:0/1750854004 learned_addr learned my addr 172.21.15.36:0/1750854004 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:11.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd099f700 1 --2- 172.21.15.36:0/1750854004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fcc0b4f00 0x7f4fcc1560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:11.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.679+0000 7f4fd11a0700 1 --2- 172.21.15.36:0/1750854004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fcc0b45c0 0x7f4fcc155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:11.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.683+0000 7f4fd19a1700 1 -- 172.21.15.36:0/1750854004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fcc0b45c0 msgr2=0x7f4fcc155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:11.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.683+0000 7f4fd19a1700 1 --2- 172.21.15.36:0/1750854004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fcc0b45c0 0x7f4fcc155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:11.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.683+0000 7f4fd19a1700 1 -- 172.21.15.36:0/1750854004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fcc0b4f00 msgr2=0x7f4fcc1560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:11.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.683+0000 7f4fd19a1700 1 --2- 172.21.15.36:0/1750854004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fcc0b4f00 0x7f4fcc1560c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:11.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.683+0000 7f4fd19a1700 1 -- 172.21.15.36:0/1750854004 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc4009530 con 0x7f4fcc0b9860 2022-06-17T12:55:11.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.683+0000 7f4fd19a1700 1 --2- 172.21.15.36:0/1750854004 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fcc0b9860 0x7f4fcc153790 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7f4fc400c000 tx=0x7f4fc4009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:11.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.683+0000 7f4fbeffd700 1 -- 172.21.15.36:0/1750854004 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fc400c850 con 0x7f4fcc0b9860 2022-06-17T12:55:11.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.683+0000 7f4fbeffd700 1 -- 172.21.15.36:0/1750854004 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc400c9b0 con 0x7f4fcc0b9860 2022-06-17T12:55:11.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.683+0000 7f4fd21a2700 1 -- 172.21.15.36:0/1750854004 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc000b180 con 0x7f4fcc0b9860 2022-06-17T12:55:11.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.683+0000 7f4fbeffd700 1 -- 172.21.15.36:0/1750854004 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fc400cd10 con 0x7f4fcc0b9860 2022-06-17T12:55:11.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.683+0000 7f4fd21a2700 1 -- 172.21.15.36:0/1750854004 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fcc0aecc0 con 0x7f4fcc0b9860 2022-06-17T12:55:11.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.683+0000 7f4fd21a2700 1 -- 172.21.15.36:0/1750854004 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fcc062eb0 con 0x7f4fcc0b9860 2022-06-17T12:55:11.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.683+0000 7f4fbeffd700 1 -- 172.21.15.36:0/1750854004 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f4fc4022070 con 0x7f4fcc0b9860 2022-06-17T12:55:11.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.683+0000 7f4fbeffd700 1 --2- 172.21.15.36:0/1750854004 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4fa80539b0 0x7f4fa8055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:11.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.687+0000 7f4fbeffd700 1 -- 172.21.15.36:0/1750854004 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4fc406c750 con 0x7f4fcc0b9860 2022-06-17T12:55:11.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.687+0000 7f4fd11a0700 1 --2- 172.21.15.36:0/1750854004 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4fa80539b0 0x7f4fa8055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:11.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.687+0000 7f4fd11a0700 1 --2- 172.21.15.36:0/1750854004 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4fa80539b0 0x7f4fa8055e70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f4fc800b000 tx=0x7f4fc8006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:11.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.691+0000 7f4fbeffd700 1 -- 172.21.15.36:0/1750854004 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4fc4043550 con 0x7f4fcc0b9860 2022-06-17T12:55:11.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.823+0000 7f4fd21a2700 1 -- 172.21.15.36:0/1750854004 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "size", "format": "json-pretty"} v 0) v1 -- 0x7f4fcc05f220 con 0x7f4fcc0b9860 2022-06-17T12:55:11.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.823+0000 7f4fbeffd700 1 -- 172.21.15.36:0/1750854004 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "size", "format": "json-pretty"}]=0 v104) v1 ==== 123+0+64 (secure 0 0 0) 0x7f4fc40160c0 con 0x7f4fcc0b9860 2022-06-17T12:55:11.811 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:11.811 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:55:11.811 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": "cephfs_data", 2022-06-17T12:55:11.812 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_id": 20, 2022-06-17T12:55:11.812 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2 2022-06-17T12:55:11.812 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:55:11.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.823+0000 7f4fd21a2700 1 -- 172.21.15.36:0/1750854004 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4fa80539b0 msgr2=0x7f4fa8055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:11.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.823+0000 7f4fd21a2700 1 --2- 172.21.15.36:0/1750854004 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4fa80539b0 0x7f4fa8055e70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f4fc800b000 tx=0x7f4fc8006df0).stop 2022-06-17T12:55:11.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.823+0000 7f4fd21a2700 1 -- 172.21.15.36:0/1750854004 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fcc0b9860 msgr2=0x7f4fcc153790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:11.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.823+0000 7f4fd21a2700 1 --2- 172.21.15.36:0/1750854004 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fcc0b9860 0x7f4fcc153790 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7f4fc400c000 tx=0x7f4fc4009290).stop 2022-06-17T12:55:11.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.823+0000 7f4fd21a2700 1 -- 172.21.15.36:0/1750854004 shutdown_connections 2022-06-17T12:55:11.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.823+0000 7f4fd21a2700 1 --2- 172.21.15.36:0/1750854004 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4fa80539b0 0x7f4fa8055e70 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:11.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.823+0000 7f4fd21a2700 1 --2- 172.21.15.36:0/1750854004 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4fcc0b9860 0x7f4fcc153790 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:11.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.823+0000 7f4fd21a2700 1 --2- 172.21.15.36:0/1750854004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fcc0b4f00 0x7f4fcc1560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:11.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.823+0000 7f4fd21a2700 1 --2- 172.21.15.36:0/1750854004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fcc0b45c0 0x7f4fcc155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:11.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.823+0000 7f4fd21a2700 1 -- 172.21.15.36:0/1750854004 >> 172.21.15.36:0/1750854004 conn(0x7f4fcc05c550 msgr2=0x7f4fcc060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:11.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.823+0000 7f4fd21a2700 1 -- 172.21.15.36:0/1750854004 shutdown_connections 2022-06-17T12:55:11.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:11.823+0000 7f4fd21a2700 1 -- 172.21.15.36:0/1750854004 wait complete. 2022-06-17T12:55:11.829 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-06-17T12:55:11.830 DEBUG:teuthology.orchestra.run.smithi107:> cd /home/ubuntu/cephtest/mnt.0 && sudo df . 2022-06-17T12:55:11.874 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 1K-blocks Used Available Use% Mounted on 2022-06-17T12:55:11.875 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-fuse 354410496 0 354410496 0% /home/ubuntu/cephtest/mnt.0 2022-06-17T12:55:11.877 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-17T12:55:11.879 DEBUG:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:55:11.904 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:55:11.934 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:55:11.958 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:55:11.987 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:55:12.010 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-17T12:55:12.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:12.012 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:55:12.012 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:55:12.043 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:55:12.044 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:55:12.045 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2022-06-17T12:55:12.045 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:55:12.045 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:55:12.079 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22695]: fuse finished with error 0 and tester_r 0 2022-06-17T12:55:12.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:12.082 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:55:12.083 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:55:12.179 INFO:teuthology.orchestra.run.smithi107.stdout:ext2/ext3 2022-06-17T12:55:12.179 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:55:12.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:55:12.181 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:55:12.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:12.198 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:55:12.198 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:55:12.214 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:55:12.222 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:55:12.224 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi136.front.sepia.ceph.com... 2022-06-17T12:55:12.225 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:55:12.226 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:55:12.282 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:ceph-fuse[22143]: fuse finished with error 0 and tester_r 0 2022-06-17T12:55:12.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:12.285 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:55:12.286 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:55:12.336 INFO:teuthology.orchestra.run.smithi136.stdout:ext2/ext3 2022-06-17T12:55:12.338 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:55:12.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:55:12.400 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:55:12.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:12.408 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:55:12.409 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:55:12.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:55:12.421 INFO:teuthology.orchestra.run.smithi143.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:55:12.422 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-06-17T12:55:12.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:55:12.423 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:55:12.431 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:55:12.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:55:12.435 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-17T12:55:12.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.559+0000 7f91e8a61700 1 Processor -- start 2022-06-17T12:55:12.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.559+0000 7f91e8a61700 1 -- start start 2022-06-17T12:55:12.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e8a61700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f91e4066ca0 0x7f91e4142770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:12.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e8a61700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f91e40acde0 0x7f91e40adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:12.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e8a61700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e40ac870 con 0x7f91e40b1440 2022-06-17T12:55:12.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e8a61700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e40b0200 con 0x7f91e40acde0 2022-06-17T12:55:12.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e8a61700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e4067080 con 0x7f91e4066ca0 2022-06-17T12:55:12.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91d9d9b700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f91e40b1440 0x7f91e4142c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55562/0 (socket says 172.21.15.36:55562) 2022-06-17T12:55:12.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91d9d9b700 1 -- 172.21.15.36:0/2585024184 learned_addr learned my addr 172.21.15.36:0/2585024184 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:12.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e2d9d700 1 --2- 172.21.15.36:0/2585024184 >> v2:172.21.15.36:3300/0 conn(0x7f91e40acde0 0x7f91e40adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:12.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e259c700 1 --2- 172.21.15.36:0/2585024184 >> v2:172.21.15.70:3300/0 conn(0x7f91e4066ca0 0x7f91e4142770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:12.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e1d9b700 1 -- 172.21.15.36:0/2585024184 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1310990072 0 0) 0x7f91e40ac870 con 0x7f91e40b1440 2022-06-17T12:55:12.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e1d9b700 1 -- 172.21.15.36:0/2585024184 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91c8003260 con 0x7f91e40b1440 2022-06-17T12:55:12.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e2d9d700 1 -- 172.21.15.36:0/2585024184 >> v2:172.21.15.70:3300/0 conn(0x7f91e4066ca0 msgr2=0x7f91e4142770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:12.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e2d9d700 1 --2- 172.21.15.36:0/2585024184 >> v2:172.21.15.70:3300/0 conn(0x7f91e4066ca0 0x7f91e4142770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:12.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e2d9d700 1 -- 172.21.15.36:0/2585024184 >> v1:172.21.15.36:6789/0 conn(0x7f91e40b1440 legacy=0x7f91e4142c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:12.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e1d9b700 1 -- 172.21.15.36:0/2585024184 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1369618410 0 0) 0x7f91c8003260 con 0x7f91e40b1440 2022-06-17T12:55:12.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e2d9d700 1 -- 172.21.15.36:0/2585024184 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e40afdd0 con 0x7f91e40acde0 2022-06-17T12:55:12.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e259c700 1 --2- 172.21.15.36:0/2585024184 >> v2:172.21.15.70:3300/0 conn(0x7f91e4066ca0 0x7f91e4142770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:12.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e2d9d700 1 --2- 172.21.15.36:0/2585024184 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91e40acde0 0x7f91e40adfe0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 rx=0x7f91d4001800 tx=0x7f91dc0091d0).ready entity=mon.0 client_cookie=5294b6827862e487 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:12.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e1d9b700 1 -- 172.21.15.36:0/2585024184 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91d4001660 con 0x7f91e40acde0 2022-06-17T12:55:12.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e1d9b700 1 -- 172.21.15.36:0/2585024184 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91dc00b520 con 0x7f91e40acde0 2022-06-17T12:55:12.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e1d9b700 1 -- 172.21.15.36:0/2585024184 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91dc00b690 con 0x7f91e40acde0 2022-06-17T12:55:12.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e8a61700 1 -- 172.21.15.36:0/2585024184 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91e40acde0 msgr2=0x7f91e40adfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:12.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e8a61700 1 --2- 172.21.15.36:0/2585024184 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91e40acde0 0x7f91e40adfe0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 rx=0x7f91d4001800 tx=0x7f91dc0091d0).stop 2022-06-17T12:55:12.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e8a61700 1 -- 172.21.15.36:0/2585024184 shutdown_connections 2022-06-17T12:55:12.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e8a61700 1 --2- 172.21.15.36:0/2585024184 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91e40acde0 0x7f91e40adfe0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:12.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e8a61700 1 --2- 172.21.15.36:0/2585024184 >> v2:172.21.15.70:3300/0 conn(0x7f91e4066ca0 0x7f91e4142770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:12.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e8a61700 1 -- 172.21.15.36:0/2585024184 >> 172.21.15.36:0/2585024184 conn(0x7f91e40b5430 msgr2=0x7f91e40b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:12.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e8a61700 1 -- 172.21.15.36:0/2585024184 shutdown_connections 2022-06-17T12:55:12.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.563+0000 7f91e8a61700 1 -- 172.21.15.36:0/2585024184 wait complete. 2022-06-17T12:55:12.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.567+0000 7f91e8a61700 1 Processor -- start 2022-06-17T12:55:12.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.567+0000 7f91e8a61700 1 -- start start 2022-06-17T12:55:12.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.567+0000 7f91e8a61700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91e4066ca0 0x7f91e40a34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:12.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.567+0000 7f91e8a61700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91e40acde0 0x7f91e40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:12.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.567+0000 7f91e8a61700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91e40b1440 0x7f91e40ab440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:12.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.567+0000 7f91e8a61700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e409fac0 con 0x7f91e40b1440 2022-06-17T12:55:12.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.567+0000 7f91e8a61700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e4153230 con 0x7f91e4066ca0 2022-06-17T12:55:12.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.567+0000 7f91e8a61700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e41534d0 con 0x7f91e40acde0 2022-06-17T12:55:12.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.567+0000 7f91e2d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91e40b1440 0x7f91e40ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:12.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.567+0000 7f91e2d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91e40b1440 0x7f91e40ab440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38042/0 (socket says 172.21.15.36:38042) 2022-06-17T12:55:12.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.567+0000 7f91e2d9d700 1 -- 172.21.15.36:0/2504947099 learned_addr learned my addr 172.21.15.36:0/2504947099 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:12.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.567+0000 7f91e259c700 1 --2- 172.21.15.36:0/2504947099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91e4066ca0 0x7f91e40a34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:12.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.567+0000 7f91d9d9b700 1 --2- 172.21.15.36:0/2504947099 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91e40acde0 0x7f91e40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:12.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.567+0000 7f91e2d9d700 1 -- 172.21.15.36:0/2504947099 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91e40acde0 msgr2=0x7f91e40a39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:12.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.567+0000 7f91e2d9d700 1 --2- 172.21.15.36:0/2504947099 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91e40acde0 0x7f91e40a39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:12.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.567+0000 7f91e2d9d700 1 -- 172.21.15.36:0/2504947099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91e4066ca0 msgr2=0x7f91e40a34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:12.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.567+0000 7f91e2d9d700 1 --2- 172.21.15.36:0/2504947099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91e4066ca0 0x7f91e40a34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:12.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.567+0000 7f91e2d9d700 1 -- 172.21.15.36:0/2504947099 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91dc009470 con 0x7f91e40b1440 2022-06-17T12:55:12.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.567+0000 7f91e2d9d700 1 --2- 172.21.15.36:0/2504947099 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91e40b1440 0x7f91e40ab440 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7f91d4001800 tx=0x7f91dc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:12.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.567+0000 7f91dbfff700 1 -- 172.21.15.36:0/2504947099 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91dc0032d0 con 0x7f91e40b1440 2022-06-17T12:55:12.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.567+0000 7f91dbfff700 1 -- 172.21.15.36:0/2504947099 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91dc011970 con 0x7f91e40b1440 2022-06-17T12:55:12.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.567+0000 7f91e8a61700 1 -- 172.21.15.36:0/2504947099 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e409ec90 con 0x7f91e40b1440 2022-06-17T12:55:12.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.567+0000 7f91e8a61700 1 -- 172.21.15.36:0/2504947099 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e409f700 con 0x7f91e40b1440 2022-06-17T12:55:12.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.567+0000 7f91dbfff700 1 -- 172.21.15.36:0/2504947099 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91dc011cd0 con 0x7f91e40b1440 2022-06-17T12:55:12.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.571+0000 7f91e8a61700 1 -- 172.21.15.36:0/2504947099 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e409fc30 con 0x7f91e40b1440 2022-06-17T12:55:12.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.571+0000 7f91dbfff700 1 -- 172.21.15.36:0/2504947099 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f91dc026030 con 0x7f91e40b1440 2022-06-17T12:55:12.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.571+0000 7f91dbfff700 1 --2- 172.21.15.36:0/2504947099 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f91bc053a60 0x7f91bc055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:12.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.571+0000 7f91dbfff700 1 -- 172.21.15.36:0/2504947099 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f91dc06cbd0 con 0x7f91e40b1440 2022-06-17T12:55:12.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.571+0000 7f91e259c700 1 --2- 172.21.15.36:0/2504947099 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f91bc053a60 0x7f91bc055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:12.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.571+0000 7f91e259c700 1 --2- 172.21.15.36:0/2504947099 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f91bc053a60 0x7f91bc055f20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f91cc00c000 tx=0x7f91cc008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:12.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.579+0000 7f91dbfff700 1 -- 172.21.15.36:0/2504947099 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f91e409fc30 con 0x7f91e40b1440 2022-06-17T12:55:12.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.707+0000 7f91e8a61700 1 -- 172.21.15.36:0/2504947099 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f91e409fc00 con 0x7f91e40b1440 2022-06-17T12:55:12.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.707+0000 7f91dbfff700 1 -- 172.21.15.36:0/2504947099 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v104) v1 ==== 81+0+34051 (secure 0 0 0) 0x7f91dc024090 con 0x7f91e40b1440 2022-06-17T12:55:12.699 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:12.700 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:55:12.700 INFO:teuthology.orchestra.run.smithi036.stdout: "epoch": 104, 2022-06-17T12:55:12.700 INFO:teuthology.orchestra.run.smithi036.stdout: "fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", 2022-06-17T12:55:12.701 INFO:teuthology.orchestra.run.smithi036.stdout: "created": "2022-06-17T12:46:30.671408+0000", 2022-06-17T12:55:12.701 INFO:teuthology.orchestra.run.smithi036.stdout: "modified": "2022-06-17T12:55:00.998590+0000", 2022-06-17T12:55:12.702 INFO:teuthology.orchestra.run.smithi036.stdout: "last_up_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:55:12.702 INFO:teuthology.orchestra.run.smithi036.stdout: "last_in_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:55:12.702 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-17T12:55:12.703 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_num": 5799936, 2022-06-17T12:55:12.703 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_set": [ 2022-06-17T12:55:12.704 INFO:teuthology.orchestra.run.smithi036.stdout: "pglog_hardlimit", 2022-06-17T12:55:12.704 INFO:teuthology.orchestra.run.smithi036.stdout: "purged_snapdirs", 2022-06-17T12:55:12.704 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_deletes", 2022-06-17T12:55:12.704 INFO:teuthology.orchestra.run.smithi036.stdout: "sortbitwise" 2022-06-17T12:55:12.705 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:55:12.705 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_version": 6, 2022-06-17T12:55:12.705 INFO:teuthology.orchestra.run.smithi036.stdout: "full_ratio": 0.94999998807907104, 2022-06-17T12:55:12.705 INFO:teuthology.orchestra.run.smithi036.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-17T12:55:12.706 INFO:teuthology.orchestra.run.smithi036.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-17T12:55:12.706 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_snapshot": "", 2022-06-17T12:55:12.707 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_max": 20, 2022-06-17T12:55:12.707 INFO:teuthology.orchestra.run.smithi036.stdout: "max_osd": 8, 2022-06-17T12:55:12.707 INFO:teuthology.orchestra.run.smithi036.stdout: "require_min_compat_client": "jewel", 2022-06-17T12:55:12.707 INFO:teuthology.orchestra.run.smithi036.stdout: "min_compat_client": "jewel", 2022-06-17T12:55:12.708 INFO:teuthology.orchestra.run.smithi036.stdout: "require_osd_release": "octopus", 2022-06-17T12:55:12.708 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:55:12.708 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.708 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 1, 2022-06-17T12:55:12.708 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "device_health_metrics", 2022-06-17T12:55:12.709 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:33.881212+0000", 2022-06-17T12:55:12.709 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:55:12.709 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:55:12.709 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:55:12.710 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:55:12.710 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:55:12.710 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:55:12.710 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:55:12.711 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:55:12.711 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 1, 2022-06-17T12:55:12.711 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 1, 2022-06-17T12:55:12.711 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 1, 2022-06-17T12:55:12.712 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 1, 2022-06-17T12:55:12.712 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 1, 2022-06-17T12:55:12.712 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:55:12.712 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:55:12.713 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:55:12.713 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:55:12.713 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:55:12.713 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:55:12.714 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:55:12.714 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.715 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "16", 2022-06-17T12:55:12.715 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:55:12.715 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:55:12.716 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:55:12.716 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:55:12.716 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:55:12.716 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:55:12.717 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:55:12.717 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:55:12.717 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:55:12.718 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:55:12.718 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:55:12.718 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:55:12.718 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:55:12.719 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:55:12.719 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:55:12.719 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:55:12.720 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:55:12.720 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:55:12.720 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:55:12.720 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:55:12.720 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:55:12.721 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:55:12.721 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:55:12.721 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:55:12.721 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:55:12.722 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:55:12.722 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.722 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:55:12.722 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:55:12.723 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:55:12.723 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:55:12.723 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:55:12.723 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:55:12.723 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:55:12.724 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:55:12.724 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:55:12.724 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:55:12.724 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:55:12.724 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:55:12.725 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 1 2022-06-17T12:55:12.725 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.725 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:55:12.725 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr_devicehealth": {} 2022-06-17T12:55:12.726 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.726 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.726 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.726 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 2, 2022-06-17T12:55:12.727 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "rbd", 2022-06-17T12:55:12.727 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:44.485460+0000", 2022-06-17T12:55:12.727 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:55:12.727 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:55:12.727 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:55:12.728 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:55:12.728 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:55:12.728 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:55:12.728 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:55:12.729 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:55:12.729 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:55:12.729 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:55:12.730 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:55:12.730 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:55:12.730 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:55:12.730 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:55:12.731 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:55:12.731 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:55:12.731 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:55:12.731 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:55:12.731 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:55:12.732 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:55:12.732 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.732 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "19", 2022-06-17T12:55:12.732 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:55:12.733 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:55:12.733 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:55:12.733 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:55:12.733 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:55:12.733 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:55:12.734 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:55:12.734 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:55:12.734 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:55:12.734 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:55:12.735 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:55:12.735 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:55:12.735 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:55:12.735 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:55:12.735 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:55:12.736 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:55:12.736 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:55:12.736 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:55:12.736 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:55:12.737 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:55:12.737 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:55:12.737 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:55:12.737 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:55:12.737 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:55:12.738 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:55:12.738 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:55:12.738 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.738 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:55:12.739 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:55:12.739 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:55:12.739 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:55:12.739 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:55:12.740 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:55:12.740 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:55:12.740 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:55:12.740 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:55:12.741 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:55:12.741 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:55:12.741 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:55:12.741 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:55:12.741 INFO:teuthology.orchestra.run.smithi036.stdout: "rbd": {} 2022-06-17T12:55:12.742 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.742 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.742 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.742 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 19, 2022-06-17T12:55:12.742 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_metadata", 2022-06-17T12:55:12.743 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:54:58.393546+0000", 2022-06-17T12:55:12.743 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:55:12.743 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:55:12.743 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:55:12.744 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:55:12.744 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:55:12.744 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:55:12.744 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:55:12.745 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:55:12.745 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:55:12.745 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:55:12.745 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:55:12.746 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:55:12.746 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:55:12.746 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:55:12.746 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:55:12.747 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:55:12.747 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:55:12.747 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:55:12.747 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:55:12.748 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:55:12.748 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.748 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "103", 2022-06-17T12:55:12.748 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:55:12.749 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:55:12.749 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:55:12.749 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:55:12.749 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:55:12.749 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:55:12.750 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:55:12.750 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:55:12.750 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:55:12.750 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:55:12.751 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:55:12.751 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:55:12.751 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:55:12.751 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:55:12.751 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:55:12.752 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:55:12.752 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:55:12.752 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:55:12.752 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:55:12.753 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:55:12.753 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:55:12.753 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:55:12.753 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:55:12.753 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:55:12.754 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:55:12.754 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:55:12.754 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.754 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:55:12.755 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:55:12.755 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:55:12.755 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:55:12.755 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:55:12.755 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:55:12.756 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:55:12.756 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:55:12.756 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:55:12.756 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:55:12.757 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:55:12.757 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:55:12.757 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_bias": 4, 2022-06-17T12:55:12.757 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 16, 2022-06-17T12:55:12.757 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_priority": 5 2022-06-17T12:55:12.758 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.758 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:55:12.758 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:55:12.759 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": "cephfs" 2022-06-17T12:55:12.759 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.759 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.759 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.759 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.760 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 20, 2022-06-17T12:55:12.760 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_data", 2022-06-17T12:55:12.760 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:54:58.997835+0000", 2022-06-17T12:55:12.760 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:55:12.761 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:55:12.761 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:55:12.761 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:55:12.761 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:55:12.761 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:55:12.762 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:55:12.762 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:55:12.762 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:55:12.762 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:55:12.762 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:55:12.763 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:55:12.763 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:55:12.763 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:55:12.763 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:55:12.764 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:55:12.764 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:55:12.764 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:55:12.764 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:55:12.764 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:55:12.765 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.765 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "104", 2022-06-17T12:55:12.765 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:55:12.765 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:55:12.766 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:55:12.766 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:55:12.766 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:55:12.766 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:55:12.766 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:55:12.767 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:55:12.767 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:55:12.767 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:55:12.767 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:55:12.768 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:55:12.768 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:55:12.768 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:55:12.768 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:55:12.768 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:55:12.769 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:55:12.769 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:55:12.769 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:55:12.769 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:55:12.770 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:55:12.770 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:55:12.770 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:55:12.770 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:55:12.771 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:55:12.771 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:55:12.771 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.771 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:55:12.772 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:55:12.772 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:55:12.772 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:55:12.773 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:55:12.773 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:55:12.773 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:55:12.774 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:55:12.774 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:55:12.774 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:55:12.774 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:55:12.775 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:55:12.775 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:55:12.775 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:55:12.775 INFO:teuthology.orchestra.run.smithi036.stdout: "data": "cephfs" 2022-06-17T12:55:12.776 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.776 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.776 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.776 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:55:12.776 INFO:teuthology.orchestra.run.smithi036.stdout: "osds": [ 2022-06-17T12:55:12.777 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.777 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:55:12.777 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", 2022-06-17T12:55:12.777 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:55:12.778 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:55:12.778 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:55:12.778 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:55:12.778 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:55:12.779 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:55:12.779 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:55:12.779 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 102, 2022-06-17T12:55:12.779 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:55:12.779 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:55:12.780 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:55:12.780 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.780 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.780 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.780 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6802", 2022-06-17T12:55:12.781 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:55:12.781 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.781 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.781 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.782 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6803", 2022-06-17T12:55:12.782 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:55:12.782 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.782 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.782 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.783 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:55:12.783 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.783 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.783 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.784 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6804", 2022-06-17T12:55:12.784 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:55:12.784 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.784 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.784 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.785 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6805", 2022-06-17T12:55:12.785 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:55:12.785 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.785 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.786 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.786 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:55:12.786 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.786 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.787 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.787 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6808", 2022-06-17T12:55:12.787 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:55:12.787 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.787 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.788 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.788 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6809", 2022-06-17T12:55:12.788 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:55:12.788 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.789 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.789 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.789 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:55:12.789 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.790 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.790 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.790 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6806", 2022-06-17T12:55:12.790 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:55:12.790 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.791 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.791 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.791 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6807", 2022-06-17T12:55:12.791 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:55:12.792 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.792 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.792 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.792 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6803/22619", 2022-06-17T12:55:12.793 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6805/22619", 2022-06-17T12:55:12.793 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6809/22619", 2022-06-17T12:55:12.793 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6807/22619", 2022-06-17T12:55:12.793 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:55:12.793 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:55:12.794 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:55:12.794 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.794 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.794 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.795 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:55:12.795 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", 2022-06-17T12:55:12.795 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:55:12.795 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:55:12.795 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:55:12.796 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:55:12.796 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:55:12.796 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:55:12.796 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:55:12.797 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 102, 2022-06-17T12:55:12.797 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:55:12.797 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:55:12.797 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:55:12.797 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.798 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.798 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.798 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6810", 2022-06-17T12:55:12.799 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:55:12.799 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.799 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.800 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.800 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6811", 2022-06-17T12:55:12.800 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:55:12.800 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.800 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.801 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.801 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:55:12.801 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.801 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.802 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.802 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6812", 2022-06-17T12:55:12.802 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:55:12.802 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.803 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.803 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.803 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6813", 2022-06-17T12:55:12.804 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:55:12.804 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.804 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.805 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.805 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:55:12.805 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.805 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.805 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.806 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6816", 2022-06-17T12:55:12.806 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:55:12.806 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.806 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.807 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.807 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6817", 2022-06-17T12:55:12.807 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:55:12.807 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.807 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.808 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.808 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:55:12.808 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.808 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.809 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.809 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6814", 2022-06-17T12:55:12.809 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:55:12.809 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.809 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.810 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.810 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6815", 2022-06-17T12:55:12.810 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:55:12.810 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.811 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.811 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.811 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6811/22620", 2022-06-17T12:55:12.811 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6813/22620", 2022-06-17T12:55:12.811 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6817/22620", 2022-06-17T12:55:12.812 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6815/22620", 2022-06-17T12:55:12.812 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:55:12.812 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:55:12.812 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:55:12.813 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.813 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.813 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.813 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:55:12.813 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", 2022-06-17T12:55:12.814 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:55:12.814 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:55:12.814 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:55:12.814 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:55:12.815 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:55:12.815 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:55:12.815 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:55:12.815 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 101, 2022-06-17T12:55:12.815 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:55:12.816 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:55:12.816 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:55:12.816 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.816 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.817 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.817 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6826", 2022-06-17T12:55:12.817 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:55:12.818 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.818 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.818 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.818 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6827", 2022-06-17T12:55:12.818 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:55:12.819 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.819 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.819 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.819 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:55:12.820 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.820 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.820 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.820 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6828", 2022-06-17T12:55:12.821 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:55:12.821 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.821 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.821 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.822 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6829", 2022-06-17T12:55:12.822 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:55:12.822 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.822 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.822 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.823 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:55:12.823 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.823 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.823 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.824 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6832", 2022-06-17T12:55:12.824 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:55:12.824 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.824 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.824 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.825 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6833", 2022-06-17T12:55:12.825 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:55:12.825 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.825 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.826 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.826 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:55:12.826 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.826 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.826 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.827 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6830", 2022-06-17T12:55:12.827 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:55:12.827 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.828 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.828 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.828 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6831", 2022-06-17T12:55:12.828 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:55:12.829 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.829 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.829 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.829 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6827/22643", 2022-06-17T12:55:12.829 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6829/22643", 2022-06-17T12:55:12.830 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6833/22643", 2022-06-17T12:55:12.830 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6831/22643", 2022-06-17T12:55:12.830 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:55:12.830 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:55:12.830 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:55:12.831 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.831 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.831 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.831 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:55:12.832 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", 2022-06-17T12:55:12.832 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:55:12.832 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:55:12.832 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:55:12.833 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:55:12.833 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:55:12.833 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:55:12.833 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:55:12.833 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 102, 2022-06-17T12:55:12.834 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:55:12.834 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:55:12.834 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:55:12.834 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.835 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.835 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.835 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6818", 2022-06-17T12:55:12.835 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:55:12.835 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.836 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.836 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.836 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6819", 2022-06-17T12:55:12.836 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:55:12.836 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.837 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.837 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.837 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:55:12.837 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.838 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.838 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.838 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6820", 2022-06-17T12:55:12.838 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:55:12.838 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.839 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.839 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.839 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6821", 2022-06-17T12:55:12.839 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:55:12.840 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.840 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.840 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.840 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:55:12.840 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.841 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.841 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.841 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6824", 2022-06-17T12:55:12.841 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:55:12.842 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.842 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.842 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.842 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6825", 2022-06-17T12:55:12.842 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:55:12.843 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.843 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.843 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.843 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:55:12.844 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.844 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.844 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.844 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6822", 2022-06-17T12:55:12.845 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:55:12.845 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.845 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.846 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.846 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6823", 2022-06-17T12:55:12.847 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:55:12.847 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.848 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.848 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.848 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6819/22646", 2022-06-17T12:55:12.849 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6821/22646", 2022-06-17T12:55:12.849 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6825/22646", 2022-06-17T12:55:12.850 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6823/22646", 2022-06-17T12:55:12.850 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:55:12.851 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:55:12.851 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:55:12.851 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.852 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.852 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.853 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:55:12.853 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", 2022-06-17T12:55:12.854 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:55:12.854 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:55:12.854 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:55:12.855 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:55:12.855 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:55:12.856 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:55:12.856 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:55:12.856 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 101, 2022-06-17T12:55:12.857 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:55:12.857 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:55:12.858 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:55:12.858 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.859 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.859 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.860 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6808", 2022-06-17T12:55:12.860 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:55:12.860 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.861 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.861 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.862 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6809", 2022-06-17T12:55:12.862 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:55:12.863 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.863 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.863 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.864 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:55:12.864 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.865 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.865 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.866 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6810", 2022-06-17T12:55:12.866 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:55:12.866 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.867 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.867 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.868 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6811", 2022-06-17T12:55:12.868 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:55:12.869 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.869 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.869 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.870 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:55:12.870 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.871 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.871 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.872 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6814", 2022-06-17T12:55:12.872 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:55:12.872 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.873 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.873 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.874 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6815", 2022-06-17T12:55:12.874 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:55:12.875 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.875 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.875 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.876 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:55:12.876 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.877 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.877 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.877 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6812", 2022-06-17T12:55:12.878 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:55:12.878 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.879 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.879 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.880 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6813", 2022-06-17T12:55:12.880 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:55:12.881 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.881 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.881 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.882 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6809/22812", 2022-06-17T12:55:12.882 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6811/22812", 2022-06-17T12:55:12.883 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6815/22812", 2022-06-17T12:55:12.883 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6813/22812", 2022-06-17T12:55:12.884 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:55:12.884 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:55:12.884 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:55:12.885 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.885 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.886 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.886 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:55:12.886 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", 2022-06-17T12:55:12.887 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:55:12.887 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:55:12.888 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:55:12.888 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:55:12.889 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:55:12.889 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:55:12.889 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:55:12.890 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 101, 2022-06-17T12:55:12.890 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:55:12.891 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:55:12.891 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:55:12.892 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.892 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.893 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.893 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6800", 2022-06-17T12:55:12.893 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:55:12.894 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.894 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.895 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.895 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6801", 2022-06-17T12:55:12.896 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:55:12.896 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.896 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.897 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.897 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:55:12.898 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.898 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.899 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.899 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6802", 2022-06-17T12:55:12.900 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:55:12.900 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.900 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.901 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.901 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6803", 2022-06-17T12:55:12.902 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:55:12.902 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.903 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.903 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.903 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:55:12.904 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.904 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.905 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.905 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6806", 2022-06-17T12:55:12.905 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:55:12.906 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.906 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.907 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.907 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6807", 2022-06-17T12:55:12.908 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:55:12.908 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.909 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.909 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.909 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:55:12.910 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.910 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.911 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.911 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6804", 2022-06-17T12:55:12.911 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:55:12.911 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.911 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.912 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.912 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6805", 2022-06-17T12:55:12.912 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:55:12.912 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.913 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.913 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.913 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6801/22813", 2022-06-17T12:55:12.913 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6803/22813", 2022-06-17T12:55:12.914 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6807/22813", 2022-06-17T12:55:12.914 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6805/22813", 2022-06-17T12:55:12.914 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:55:12.914 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:55:12.914 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:55:12.915 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.915 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.915 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.915 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:55:12.916 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", 2022-06-17T12:55:12.916 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:55:12.916 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:55:12.916 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:55:12.916 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:55:12.917 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:55:12.917 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:55:12.917 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:55:12.917 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 102, 2022-06-17T12:55:12.917 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:55:12.918 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:55:12.918 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:55:12.918 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.918 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.919 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.919 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6816", 2022-06-17T12:55:12.919 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:55:12.919 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.919 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.920 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.920 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6817", 2022-06-17T12:55:12.920 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:55:12.920 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.921 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.921 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.921 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:55:12.921 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.921 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.922 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.922 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6818", 2022-06-17T12:55:12.922 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:55:12.922 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.923 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.923 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.923 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6819", 2022-06-17T12:55:12.923 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:55:12.924 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.924 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.924 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.924 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:55:12.924 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.925 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.925 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.925 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6822", 2022-06-17T12:55:12.925 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:55:12.926 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.927 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.928 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.928 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6823", 2022-06-17T12:55:12.929 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:55:12.929 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.930 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.930 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.931 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:55:12.931 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.932 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.932 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.932 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6820", 2022-06-17T12:55:12.933 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:55:12.933 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.934 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.934 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.935 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6821", 2022-06-17T12:55:12.935 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:55:12.935 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.936 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.936 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.936 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6817/22822", 2022-06-17T12:55:12.937 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6819/22822", 2022-06-17T12:55:12.937 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6823/22822", 2022-06-17T12:55:12.938 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6821/22822", 2022-06-17T12:55:12.938 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:55:12.939 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:55:12.939 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:55:12.939 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.940 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.940 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.940 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:55:12.941 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", 2022-06-17T12:55:12.941 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:55:12.941 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:55:12.941 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:55:12.942 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:55:12.942 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:55:12.943 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:55:12.943 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:55:12.943 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 102, 2022-06-17T12:55:12.944 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:55:12.944 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:55:12.944 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:55:12.944 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.945 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.945 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.945 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6824", 2022-06-17T12:55:12.946 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:55:12.946 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.946 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.947 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.947 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6825", 2022-06-17T12:55:12.947 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:55:12.947 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.948 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.948 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.948 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:55:12.949 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.949 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.950 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.950 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6826", 2022-06-17T12:55:12.950 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:55:12.951 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.951 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.951 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.951 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6827", 2022-06-17T12:55:12.952 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:55:12.952 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.952 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.952 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.953 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:55:12.953 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.953 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.953 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.954 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6830", 2022-06-17T12:55:12.954 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:55:12.954 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.954 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.955 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.955 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6831", 2022-06-17T12:55:12.955 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:55:12.955 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.956 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.956 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.956 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:55:12.957 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:12.957 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.957 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:12.957 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6828", 2022-06-17T12:55:12.958 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:55:12.958 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.958 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.959 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:12.959 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6829", 2022-06-17T12:55:12.959 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:55:12.959 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.960 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.960 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.960 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6825/22837", 2022-06-17T12:55:12.960 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6827/22837", 2022-06-17T12:55:12.961 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6831/22837", 2022-06-17T12:55:12.961 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6829/22837", 2022-06-17T12:55:12.961 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:55:12.961 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:55:12.962 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:55:12.962 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:12.963 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.963 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:55:12.963 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_xinfo": [ 2022-06-17T12:55:12.964 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.964 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:55:12.964 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:55:12.964 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:55:12.965 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:55:12.965 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:55:12.965 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:55:12.965 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.291461+0000", 2022-06-17T12:55:12.966 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:55:12.966 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.966 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.966 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:55:12.967 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:55:12.967 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:55:12.967 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:55:12.968 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:55:12.968 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:55:12.968 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.537912+0000", 2022-06-17T12:55:12.969 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:55:12.969 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.970 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.970 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:55:12.971 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:55:12.971 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:55:12.971 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:55:12.972 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:55:12.972 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:55:12.972 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.606947+0000", 2022-06-17T12:55:12.973 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:55:12.973 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.974 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.974 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:55:12.974 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:55:12.975 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:55:12.975 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:55:12.976 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:55:12.976 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:55:12.976 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.514412+0000", 2022-06-17T12:55:12.977 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:55:12.977 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.977 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.978 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:55:12.978 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:55:12.979 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:55:12.979 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:55:12.979 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:55:12.980 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:55:12.980 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.394645+0000", 2022-06-17T12:55:12.980 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:55:12.981 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.981 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.982 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:55:12.983 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:55:12.983 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:55:12.983 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:55:12.984 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:55:12.984 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:55:12.984 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.343806+0000", 2022-06-17T12:55:12.985 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:55:12.986 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.986 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.986 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:55:12.987 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:55:12.987 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:55:12.988 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:55:12.988 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:55:12.989 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:55:12.989 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.591864+0000", 2022-06-17T12:55:12.990 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:55:12.990 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:12.991 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:12.991 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:55:12.992 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:55:12.992 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:55:12.993 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:55:12.993 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:55:12.994 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:55:12.994 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.607045+0000", 2022-06-17T12:55:12.995 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:55:12.995 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:12.996 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:55:12.996 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap": [], 2022-06-17T12:55:12.997 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap_items": [], 2022-06-17T12:55:12.997 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_temp": [], 2022-06-17T12:55:12.998 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_temp": [], 2022-06-17T12:55:12.998 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklist": {}, 2022-06-17T12:55:12.998 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profiles": { 2022-06-17T12:55:12.999 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs_data": { 2022-06-17T12:55:12.999 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-device-class": "", 2022-06-17T12:55:13.000 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:55:13.000 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-root": "default", 2022-06-17T12:55:13.001 INFO:teuthology.orchestra.run.smithi036.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-17T12:55:13.001 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:55:13.002 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "2", 2022-06-17T12:55:13.002 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:55:13.003 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:55:13.003 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van", 2022-06-17T12:55:13.004 INFO:teuthology.orchestra.run.smithi036.stdout: "w": "8" 2022-06-17T12:55:13.004 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:13.005 INFO:teuthology.orchestra.run.smithi036.stdout: "default": { 2022-06-17T12:55:13.005 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:55:13.006 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:55:13.006 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "1", 2022-06-17T12:55:13.007 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:55:13.007 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:55:13.007 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van" 2022-06-17T12:55:13.008 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:13.008 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:13.009 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps_queue": [], 2022-06-17T12:55:13.009 INFO:teuthology.orchestra.run.smithi036.stdout: "new_removed_snaps": [], 2022-06-17T12:55:13.010 INFO:teuthology.orchestra.run.smithi036.stdout: "new_purged_snaps": [], 2022-06-17T12:55:13.010 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_node_flags": {}, 2022-06-17T12:55:13.011 INFO:teuthology.orchestra.run.smithi036.stdout: "device_class_flags": {} 2022-06-17T12:55:13.011 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:55:13.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.711+0000 7f91e8a61700 1 -- 172.21.15.36:0/2504947099 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f91bc053a60 msgr2=0x7f91bc055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:13.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.711+0000 7f91e8a61700 1 --2- 172.21.15.36:0/2504947099 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f91bc053a60 0x7f91bc055f20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f91cc00c000 tx=0x7f91cc008040).stop 2022-06-17T12:55:13.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.711+0000 7f91e8a61700 1 -- 172.21.15.36:0/2504947099 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91e40b1440 msgr2=0x7f91e40ab440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:13.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.711+0000 7f91e8a61700 1 --2- 172.21.15.36:0/2504947099 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91e40b1440 0x7f91e40ab440 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7f91d4001800 tx=0x7f91dc016000).stop 2022-06-17T12:55:13.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.711+0000 7f91e8a61700 1 -- 172.21.15.36:0/2504947099 shutdown_connections 2022-06-17T12:55:13.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.711+0000 7f91e8a61700 1 --2- 172.21.15.36:0/2504947099 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f91bc053a60 0x7f91bc055f20 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.711+0000 7f91e8a61700 1 --2- 172.21.15.36:0/2504947099 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91e40b1440 0x7f91e40ab440 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.711+0000 7f91e8a61700 1 --2- 172.21.15.36:0/2504947099 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91e40acde0 0x7f91e40a39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.711+0000 7f91e8a61700 1 --2- 172.21.15.36:0/2504947099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91e4066ca0 0x7f91e40a34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.711+0000 7f91e8a61700 1 -- 172.21.15.36:0/2504947099 >> 172.21.15.36:0/2504947099 conn(0x7f91e40b5430 msgr2=0x7f91e4142730 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:13.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.711+0000 7f91e8a61700 1 -- 172.21.15.36:0/2504947099 shutdown_connections 2022-06-17T12:55:13.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:12.711+0000 7f91e8a61700 1 -- 172.21.15.36:0/2504947099 wait complete. 2022-06-17T12:55:13.021 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:55:13.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.147+0000 7f4090d94700 1 Processor -- start 2022-06-17T12:55:13.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.147+0000 7f4090d94700 1 -- start start 2022-06-17T12:55:13.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.147+0000 7f4090d94700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f408c0b2200 0x7f408c0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:13.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.147+0000 7f4090d94700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f408c0b2cd0 0x7f408c147600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:13.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.147+0000 7f4090d94700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f408c0b25e0 con 0x7f408c0af8d0 2022-06-17T12:55:13.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.147+0000 7f4090d94700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f408c0ad790 con 0x7f408c0b2cd0 2022-06-17T12:55:13.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.147+0000 7f4090d94700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f408c0aef40 con 0x7f408c0b2200 2022-06-17T12:55:13.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.147+0000 7f408affd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f408c0af8d0 0x7f408c14a8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55576/0 (socket says 172.21.15.36:55576) 2022-06-17T12:55:13.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.147+0000 7f408affd700 1 -- 172.21.15.36:0/1597204548 learned_addr learned my addr 172.21.15.36:0/1597204548 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:13.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.147+0000 7f408bfff700 1 --2- 172.21.15.36:0/1597204548 >> v2:172.21.15.70:3300/0 conn(0x7f408c0b2cd0 0x7f408c147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:13.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.147+0000 7f408b7fe700 1 --2- 172.21.15.36:0/1597204548 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f408c0b2200 0x7f408c0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:13.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.147+0000 7f408a7fc700 1 -- 172.21.15.36:0/1597204548 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4161540714 0 0) 0x7f408c0b25e0 con 0x7f408c0af8d0 2022-06-17T12:55:13.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.147+0000 7f408a7fc700 1 -- 172.21.15.36:0/1597204548 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4078003260 con 0x7f408c0af8d0 2022-06-17T12:55:13.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408a7fc700 1 -- 172.21.15.36:0/1597204548 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1495128993 0 0) 0x7f4078003260 con 0x7f408c0af8d0 2022-06-17T12:55:13.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408a7fc700 1 -- 172.21.15.36:0/1597204548 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f408c0b2200 msgr2=0x7f408c0af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:13.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408a7fc700 1 --2- 172.21.15.36:0/1597204548 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f408c0b2200 0x7f408c0af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408a7fc700 1 -- 172.21.15.36:0/1597204548 >> v2:172.21.15.70:3300/0 conn(0x7f408c0b2cd0 msgr2=0x7f408c147600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:13.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408a7fc700 1 --2- 172.21.15.36:0/1597204548 >> v2:172.21.15.70:3300/0 conn(0x7f408c0b2cd0 0x7f408c147600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408a7fc700 1 -- 172.21.15.36:0/1597204548 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408c0b1950 con 0x7f408c0af8d0 2022-06-17T12:55:13.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408a7fc700 1 -- 172.21.15.36:0/1597204548 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f407c001680 con 0x7f408c0af8d0 2022-06-17T12:55:13.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408bfff700 1 --2- 172.21.15.36:0/1597204548 >> v2:172.21.15.70:3300/0 conn(0x7f408c0b2cd0 0x7f408c147600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:13.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408a7fc700 1 -- 172.21.15.36:0/1597204548 >> v1:172.21.15.36:6789/0 conn(0x7f408c0af8d0 legacy=0x7f408c14a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:13.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408a7fc700 1 --2- 172.21.15.36:0/1597204548 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f408c0b2200 0x7f4078003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:13.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408a7fc700 1 --2- 172.21.15.36:0/1597204548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4078003c80 0x7f4078004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:13.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408a7fc700 1 --2- 172.21.15.36:0/1597204548 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f40780049f0 0x7f4078004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:13.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408a7fc700 1 -- 172.21.15.36:0/1597204548 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408c0ad790 con 0x7f40780049f0 2022-06-17T12:55:13.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408a7fc700 1 -- 172.21.15.36:0/1597204548 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408c0aef40 con 0x7f4078003c80 2022-06-17T12:55:13.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408a7fc700 1 -- 172.21.15.36:0/1597204548 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407800b750 con 0x7f408c0b2200 2022-06-17T12:55:13.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408bfff700 1 --2- 172.21.15.36:0/1597204548 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f40780049f0 0x7f4078004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:13.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408b7fe700 1 --2- 172.21.15.36:0/1597204548 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f408c0b2200 0x7f4078003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:13.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408affd700 1 --2- 172.21.15.36:0/1597204548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4078003c80 0x7f4078004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:13.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408bfff700 1 -- 172.21.15.36:0/1597204548 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f408c0b2200 msgr2=0x7f4078003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:13.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408bfff700 1 --2- 172.21.15.36:0/1597204548 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f408c0b2200 0x7f4078003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408bfff700 1 -- 172.21.15.36:0/1597204548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4078003c80 msgr2=0x7f4078004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:13.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408bfff700 1 --2- 172.21.15.36:0/1597204548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4078003c80 0x7f4078004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408bfff700 1 -- 172.21.15.36:0/1597204548 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4078003260 con 0x7f40780049f0 2022-06-17T12:55:13.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408affd700 1 --2- 172.21.15.36:0/1597204548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4078003c80 0x7f4078004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:13.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408bfff700 1 --2- 172.21.15.36:0/1597204548 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f40780049f0 0x7f4078004e70 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7f40780054d0 tx=0x7f40840091d0).ready entity=mon.0 client_cookie=ebd65353589b94af server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:13.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408a7fc700 1 -- 172.21.15.36:0/1597204548 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f408400e920 con 0x7f40780049f0 2022-06-17T12:55:13.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408a7fc700 1 -- 172.21.15.36:0/1597204548 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408400ea80 con 0x7f40780049f0 2022-06-17T12:55:13.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408a7fc700 1 -- 172.21.15.36:0/1597204548 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f408400ede0 con 0x7f40780049f0 2022-06-17T12:55:13.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f4090d94700 1 -- 172.21.15.36:0/1597204548 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f40780049f0 msgr2=0x7f4078004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:13.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f4090d94700 1 --2- 172.21.15.36:0/1597204548 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f40780049f0 0x7f4078004e70 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7f40780054d0 tx=0x7f40840091d0).stop 2022-06-17T12:55:13.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f408bfff700 1 -- 172.21.15.36:0/1597204548 reap_dead start 2022-06-17T12:55:13.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f4090d94700 1 -- 172.21.15.36:0/1597204548 shutdown_connections 2022-06-17T12:55:13.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f4090d94700 1 -- 172.21.15.36:0/1597204548 >> 172.21.15.36:0/1597204548 conn(0x7f408c05f770 msgr2=0x7f408c05fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:13.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f4090d94700 1 -- 172.21.15.36:0/1597204548 shutdown_connections 2022-06-17T12:55:13.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.151+0000 7f4090d94700 1 -- 172.21.15.36:0/1597204548 wait complete. 2022-06-17T12:55:13.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f4090d94700 1 Processor -- start 2022-06-17T12:55:13.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f4090d94700 1 -- start start 2022-06-17T12:55:13.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f4090d94700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40780049f0 0x7f408c149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:13.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f4090d94700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f408c0a8610 0x7f408c159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:13.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f4090d94700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f408c062b00 0x7f408c062f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:13.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f4090d94700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408c0aa960 con 0x7f408c0a8610 2022-06-17T12:55:13.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f4090d94700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408c0a8c70 con 0x7f40780049f0 2022-06-17T12:55:13.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f4090d94700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f408c0a9940 con 0x7f408c062b00 2022-06-17T12:55:13.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f408affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f408c0a8610 0x7f408c159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:13.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f408affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f408c0a8610 0x7f408c159d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38058/0 (socket says 172.21.15.36:38058) 2022-06-17T12:55:13.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f408affd700 1 -- 172.21.15.36:0/3798539055 learned_addr learned my addr 172.21.15.36:0/3798539055 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:13.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f408b7fe700 1 --2- 172.21.15.36:0/3798539055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40780049f0 0x7f408c149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:13.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f408bfff700 1 --2- 172.21.15.36:0/3798539055 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f408c062b00 0x7f408c062f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:13.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f408affd700 1 -- 172.21.15.36:0/3798539055 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f408c062b00 msgr2=0x7f408c062f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:13.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f408affd700 1 --2- 172.21.15.36:0/3798539055 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f408c062b00 0x7f408c062f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f408affd700 1 -- 172.21.15.36:0/3798539055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40780049f0 msgr2=0x7f408c149aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:13.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f408affd700 1 --2- 172.21.15.36:0/3798539055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40780049f0 0x7f408c149aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f408affd700 1 -- 172.21.15.36:0/3798539055 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408c0626a0 con 0x7f408c0a8610 2022-06-17T12:55:13.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f408affd700 1 --2- 172.21.15.36:0/3798539055 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f408c0a8610 0x7f408c159d30 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 rx=0x7f407c00b000 tx=0x7f407c001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:13.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f4088ff9700 1 -- 172.21.15.36:0/3798539055 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f407c010680 con 0x7f408c0a8610 2022-06-17T12:55:13.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f4088ff9700 1 -- 172.21.15.36:0/3798539055 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407c010040 con 0x7f408c0a8610 2022-06-17T12:55:13.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f4088ff9700 1 -- 172.21.15.36:0/3798539055 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f407c0109d0 con 0x7f408c0a8610 2022-06-17T12:55:13.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f4090d94700 1 -- 172.21.15.36:0/3798539055 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f408c0a96a0 con 0x7f408c0a8610 2022-06-17T12:55:13.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f4090d94700 1 -- 172.21.15.36:0/3798539055 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f408c15a220 con 0x7f408c0a8610 2022-06-17T12:55:13.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.155+0000 7f4090d94700 1 -- 172.21.15.36:0/3798539055 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f408c0ad330 con 0x7f408c0a8610 2022-06-17T12:55:13.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.159+0000 7f4088ff9700 1 -- 172.21.15.36:0/3798539055 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f407c010b30 con 0x7f408c0a8610 2022-06-17T12:55:13.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.159+0000 7f4088ff9700 1 --2- 172.21.15.36:0/3798539055 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f406c053ad0 0x7f406c055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:13.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.159+0000 7f4088ff9700 1 -- 172.21.15.36:0/3798539055 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f407c06bd00 con 0x7f408c0a8610 2022-06-17T12:55:13.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.159+0000 7f408b7fe700 1 --2- 172.21.15.36:0/3798539055 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f406c053ad0 0x7f406c055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:13.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.159+0000 7f408b7fe700 1 --2- 172.21.15.36:0/3798539055 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f406c053ad0 0x7f406c055f90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f408000b000 tx=0x7f4080006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:13.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.167+0000 7f4088ff9700 1 -- 172.21.15.36:0/3798539055 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f407c042970 con 0x7f408c0a8610 2022-06-17T12:55:13.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.323+0000 7f4090d94700 1 -- 172.21.15.36:0/3798539055 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f408c062830 con 0x7f408c0a8610 2022-06-17T12:55:13.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.327+0000 7f4088ff9700 1 -- 172.21.15.36:0/3798539055 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f407c01a030 con 0x7f408c0a8610 2022-06-17T12:55:13.316 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:13.316 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":54,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/676934608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":676934608},{"type":"v1","addr":"172.21.15.70:6833","nonce":676934608}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:54:59.995522+0000","modified":"2022-06-17T12:55:01.664559+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7395":{"gid":7395,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/233328233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":233328233},{"type":"v1","addr":"172.21.15.36:6835","nonce":233328233}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-06-17T12:55:13.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.327+0000 7f4090d94700 1 -- 172.21.15.36:0/3798539055 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f406c053ad0 msgr2=0x7f406c055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:13.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.327+0000 7f4090d94700 1 --2- 172.21.15.36:0/3798539055 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f406c053ad0 0x7f406c055f90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f408000b000 tx=0x7f4080006df0).stop 2022-06-17T12:55:13.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.327+0000 7f4090d94700 1 -- 172.21.15.36:0/3798539055 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f408c0a8610 msgr2=0x7f408c159d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:13.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.327+0000 7f4090d94700 1 --2- 172.21.15.36:0/3798539055 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f408c0a8610 0x7f408c159d30 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 rx=0x7f407c00b000 tx=0x7f407c001510).stop 2022-06-17T12:55:13.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.327+0000 7f4090d94700 1 -- 172.21.15.36:0/3798539055 shutdown_connections 2022-06-17T12:55:13.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.327+0000 7f4090d94700 1 --2- 172.21.15.36:0/3798539055 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f406c053ad0 0x7f406c055f90 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.327+0000 7f4090d94700 1 --2- 172.21.15.36:0/3798539055 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f408c062b00 0x7f408c062f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.327+0000 7f4090d94700 1 --2- 172.21.15.36:0/3798539055 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f408c0a8610 0x7f408c159d30 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.327+0000 7f4090d94700 1 --2- 172.21.15.36:0/3798539055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40780049f0 0x7f408c149aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.327+0000 7f4090d94700 1 -- 172.21.15.36:0/3798539055 >> 172.21.15.36:0/3798539055 conn(0x7f408c05f770 msgr2=0x7f408c14d580 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:13.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.327+0000 7f4090d94700 1 -- 172.21.15.36:0/3798539055 shutdown_connections 2022-06-17T12:55:13.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.327+0000 7f4090d94700 1 -- 172.21.15.36:0/3798539055 wait complete. 2022-06-17T12:55:13.323 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 54 2022-06-17T12:55:13.327 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-17T12:55:13.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.443+0000 7f666720e700 1 Processor -- start 2022-06-17T12:55:13.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.443+0000 7f666720e700 1 -- start start 2022-06-17T12:55:13.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f666720e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f66600b2cd0 0x7f66601465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:13.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f666720e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6660146ae0 0x7f6660146f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:13.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f666720e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66600b25e0 con 0x7f66600b2200 2022-06-17T12:55:13.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f666720e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66600ad790 con 0x7f66600b2cd0 2022-06-17T12:55:13.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f666720e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66600aef40 con 0x7f6660146ae0 2022-06-17T12:55:13.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f6665a0b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f66600b2cd0 0x7f66601465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:13.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f6665a0b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f66600b2cd0 0x7f66601465f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38066/0 (socket says 172.21.15.36:38066) 2022-06-17T12:55:13.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f6665a0b700 1 -- 172.21.15.36:0/2998738264 learned_addr learned my addr 172.21.15.36:0/2998738264 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:13.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f6666a0d700 1 --2- 172.21.15.36:0/2998738264 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6660146ae0 0x7f6660146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:13.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f6665a0b700 1 -- 172.21.15.36:0/2998738264 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6660146ae0 msgr2=0x7f6660146f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:13.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f6665a0b700 1 --2- 172.21.15.36:0/2998738264 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6660146ae0 0x7f6660146f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f6665a0b700 1 -- 172.21.15.36:0/2998738264 >> v1:172.21.15.70:6789/0 conn(0x7f66600b2200 legacy=0x7f66600af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:13.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f6665a0b700 1 -- 172.21.15.36:0/2998738264 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66600afaf0 con 0x7f66600b2cd0 2022-06-17T12:55:13.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f6665a0b700 1 --2- 172.21.15.36:0/2998738264 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f66600b2cd0 0x7f66601465f0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7f665c002f70 tx=0x7f665c006eb0).ready entity=mon.0 client_cookie=97b820a993f294aa server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:13.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f666520a700 1 -- 172.21.15.36:0/2998738264 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f665c00d570 con 0x7f66600b2cd0 2022-06-17T12:55:13.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f666520a700 1 -- 172.21.15.36:0/2998738264 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665c000e90 con 0x7f66600b2cd0 2022-06-17T12:55:13.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f666520a700 1 -- 172.21.15.36:0/2998738264 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f665c00d900 con 0x7f66600b2cd0 2022-06-17T12:55:13.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f666720e700 1 -- 172.21.15.36:0/2998738264 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f66600b2cd0 msgr2=0x7f66601465f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:13.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f666720e700 1 --2- 172.21.15.36:0/2998738264 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f66600b2cd0 0x7f66601465f0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7f665c002f70 tx=0x7f665c006eb0).stop 2022-06-17T12:55:13.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f666720e700 1 -- 172.21.15.36:0/2998738264 shutdown_connections 2022-06-17T12:55:13.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f666720e700 1 --2- 172.21.15.36:0/2998738264 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6660146ae0 0x7f6660146f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f666720e700 1 --2- 172.21.15.36:0/2998738264 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f66600b2cd0 0x7f66601465f0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f666720e700 1 -- 172.21.15.36:0/2998738264 >> 172.21.15.36:0/2998738264 conn(0x7f666005f770 msgr2=0x7f666005fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:13.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f666720e700 1 -- 172.21.15.36:0/2998738264 shutdown_connections 2022-06-17T12:55:13.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.447+0000 7f666720e700 1 -- 172.21.15.36:0/2998738264 wait complete. 2022-06-17T12:55:13.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.451+0000 7f666720e700 1 Processor -- start 2022-06-17T12:55:13.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.451+0000 7f666720e700 1 -- start start 2022-06-17T12:55:13.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.451+0000 7f666720e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66600b2200 0x7f666014d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:13.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.451+0000 7f666720e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66600b2cd0 0x7f66600afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:13.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.451+0000 7f666720e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6660146ae0 0x7f6660157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:13.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.451+0000 7f666720e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66600a8130 con 0x7f6660146ae0 2022-06-17T12:55:13.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.451+0000 7f666720e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6660062390 con 0x7f66600b2200 2022-06-17T12:55:13.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.451+0000 7f666720e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66600625f0 con 0x7f66600b2cd0 2022-06-17T12:55:13.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.451+0000 7f6666a0d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6660146ae0 0x7f6660157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:13.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.451+0000 7f6666a0d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6660146ae0 0x7f6660157000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38076/0 (socket says 172.21.15.36:38076) 2022-06-17T12:55:13.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.451+0000 7f6666a0d700 1 -- 172.21.15.36:0/1796662270 learned_addr learned my addr 172.21.15.36:0/1796662270 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:13.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.451+0000 7f666620c700 1 --2- 172.21.15.36:0/1796662270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66600b2200 0x7f666014d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:13.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.451+0000 7f6665a0b700 1 --2- 172.21.15.36:0/1796662270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66600b2cd0 0x7f66600afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:13.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.455+0000 7f6666a0d700 1 -- 172.21.15.36:0/1796662270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66600b2cd0 msgr2=0x7f66600afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:13.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.455+0000 7f6666a0d700 1 --2- 172.21.15.36:0/1796662270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66600b2cd0 0x7f66600afd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.455+0000 7f6666a0d700 1 -- 172.21.15.36:0/1796662270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66600b2200 msgr2=0x7f666014d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:13.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.455+0000 7f6666a0d700 1 --2- 172.21.15.36:0/1796662270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66600b2200 0x7f666014d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.455+0000 7f6666a0d700 1 -- 172.21.15.36:0/1796662270 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665c00b1e0 con 0x7f6660146ae0 2022-06-17T12:55:13.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.455+0000 7f6666a0d700 1 --2- 172.21.15.36:0/1796662270 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6660146ae0 0x7f6660157000 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 rx=0x7f6658006e00 tx=0x7f66580091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:13.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.455+0000 7f664f7fe700 1 -- 172.21.15.36:0/1796662270 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f665800c670 con 0x7f6660146ae0 2022-06-17T12:55:13.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.455+0000 7f664f7fe700 1 -- 172.21.15.36:0/1796662270 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665800c7d0 con 0x7f6660146ae0 2022-06-17T12:55:13.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.455+0000 7f666720e700 1 -- 172.21.15.36:0/1796662270 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6660156b60 con 0x7f6660146ae0 2022-06-17T12:55:13.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.455+0000 7f664f7fe700 1 -- 172.21.15.36:0/1796662270 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f665800cb30 con 0x7f6660146ae0 2022-06-17T12:55:13.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.455+0000 7f666720e700 1 -- 172.21.15.36:0/1796662270 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66600b0250 con 0x7f6660146ae0 2022-06-17T12:55:13.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.455+0000 7f664f7fe700 1 -- 172.21.15.36:0/1796662270 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f665800cc90 con 0x7f6660146ae0 2022-06-17T12:55:13.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.455+0000 7f666720e700 1 -- 172.21.15.36:0/1796662270 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6660156cf0 con 0x7f6660146ae0 2022-06-17T12:55:13.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.455+0000 7f664f7fe700 1 --2- 172.21.15.36:0/1796662270 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6640053a40 0x7f6640055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:13.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.455+0000 7f666620c700 1 --2- 172.21.15.36:0/1796662270 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6640053a40 0x7f6640055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:13.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.455+0000 7f664f7fe700 1 -- 172.21.15.36:0/1796662270 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f665806c280 con 0x7f6660146ae0 2022-06-17T12:55:13.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.455+0000 7f666620c700 1 --2- 172.21.15.36:0/1796662270 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6640053a40 0x7f6640055f00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f665000d000 tx=0x7f6650007b50).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:13.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.463+0000 7f664f7fe700 1 -- 172.21.15.36:0/1796662270 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6658043020 con 0x7f6660146ae0 2022-06-17T12:55:13.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.627+0000 7f666720e700 1 -- 172.21.15.36:0/1796662270 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f66600b1950 con 0x7f6660146ae0 2022-06-17T12:55:13.701 INFO:tasks.ceph.mon.a.smithi036.stderr:2022-06-17T12:55:13.715+0000 7fb7a2649700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-17T12:55:13.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.719+0000 7f664f7fe700 1 -- 172.21.15.36:0/1796662270 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v55) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6658046640 con 0x7f6660146ae0 2022-06-17T12:55:13.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.723+0000 7f666720e700 1 -- 172.21.15.36:0/1796662270 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6640053a40 msgr2=0x7f6640055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:13.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.723+0000 7f666720e700 1 --2- 172.21.15.36:0/1796662270 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6640053a40 0x7f6640055f00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f665000d000 tx=0x7f6650007b50).stop 2022-06-17T12:55:13.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.723+0000 7f666720e700 1 -- 172.21.15.36:0/1796662270 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6660146ae0 msgr2=0x7f6660157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:13.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.723+0000 7f666720e700 1 --2- 172.21.15.36:0/1796662270 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6660146ae0 0x7f6660157000 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 rx=0x7f6658006e00 tx=0x7f66580091d0).stop 2022-06-17T12:55:13.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.723+0000 7f666720e700 1 -- 172.21.15.36:0/1796662270 shutdown_connections 2022-06-17T12:55:13.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.723+0000 7f666720e700 1 --2- 172.21.15.36:0/1796662270 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6640053a40 0x7f6640055f00 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.727+0000 7f666720e700 1 --2- 172.21.15.36:0/1796662270 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6660146ae0 0x7f6660157000 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.727+0000 7f666720e700 1 --2- 172.21.15.36:0/1796662270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66600b2cd0 0x7f66600afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.727+0000 7f666720e700 1 --2- 172.21.15.36:0/1796662270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66600b2200 0x7f666014d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.727+0000 7f666720e700 1 -- 172.21.15.36:0/1796662270 >> 172.21.15.36:0/1796662270 conn(0x7f666005f770 msgr2=0x7f66600b4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:13.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.727+0000 7f666720e700 1 -- 172.21.15.36:0/1796662270 shutdown_connections 2022-06-17T12:55:13.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.727+0000 7f666720e700 1 -- 172.21.15.36:0/1796662270 wait complete. 2022-06-17T12:55:13.720 INFO:teuthology.orchestra.run.smithi036.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-17T12:55:13.730 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:55:13.734 INFO:tasks.ceph.mds.a.smithi036.stdout:starting mds.a at 2022-06-17T12:55:13.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.839+0000 7f3f793fe700 1 Processor -- start 2022-06-17T12:55:13.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.839+0000 7f3f793fe700 1 -- start start 2022-06-17T12:55:13.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.839+0000 7f3f793fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f740afec0 0x7f3f74060b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:13.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.839+0000 7f3f793fe700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f740b15e0 con 0x7f3f74060780 2022-06-17T12:55:13.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.839+0000 7f3f793fe700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f74143990 con 0x7f3f741434f0 2022-06-17T12:55:13.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.839+0000 7f3f793fe700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f740b1a80 con 0x7f3f740afec0 2022-06-17T12:55:13.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.839+0000 7f3f72ffd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f3f74060780 0x7f3f740b07c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55622/0 (socket says 172.21.15.36:55622) 2022-06-17T12:55:13.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.839+0000 7f3f72ffd700 1 -- 172.21.15.36:0/3241157849 learned_addr learned my addr 172.21.15.36:0/3241157849 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:13.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.839+0000 7f3f737fe700 1 --2- 172.21.15.36:0/3241157849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f740afec0 0x7f3f74060b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:13.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.843+0000 7f3f71ffb700 1 -- 172.21.15.36:0/3241157849 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091075915 0 0) 0x7f3f740b15e0 con 0x7f3f74060780 2022-06-17T12:55:13.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.843+0000 7f3f71ffb700 1 -- 172.21.15.36:0/3241157849 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f58003260 con 0x7f3f74060780 2022-06-17T12:55:13.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.843+0000 7f3f71ffb700 1 -- 172.21.15.36:0/3241157849 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3586688803 0 0) 0x7f3f74143990 con 0x7f3f741434f0 2022-06-17T12:55:13.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.843+0000 7f3f71ffb700 1 -- 172.21.15.36:0/3241157849 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f740b15e0 con 0x7f3f741434f0 2022-06-17T12:55:13.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.843+0000 7f3f737fe700 1 -- 172.21.15.36:0/3241157849 >> v1:172.21.15.70:6789/0 conn(0x7f3f741434f0 legacy=0x7f3f740b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:13.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.843+0000 7f3f737fe700 1 -- 172.21.15.36:0/3241157849 >> v1:172.21.15.36:6789/0 conn(0x7f3f74060780 legacy=0x7f3f740b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:13.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.843+0000 7f3f737fe700 1 -- 172.21.15.36:0/3241157849 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f740b1d20 con 0x7f3f740afec0 2022-06-17T12:55:13.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.843+0000 7f3f737fe700 1 --2- 172.21.15.36:0/3241157849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f740afec0 0x7f3f74060b60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f3f6c0032f0 tx=0x7f3f6c00a370).ready entity=mon.2 client_cookie=289ed8a286c67370 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:13.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.843+0000 7f3f71ffb700 1 -- 172.21.15.36:0/3241157849 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f6c00cba0 con 0x7f3f740afec0 2022-06-17T12:55:13.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.843+0000 7f3f71ffb700 1 -- 172.21.15.36:0/3241157849 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6c00cd00 con 0x7f3f740afec0 2022-06-17T12:55:13.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.843+0000 7f3f71ffb700 1 -- 172.21.15.36:0/3241157849 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f6c010600 con 0x7f3f740afec0 2022-06-17T12:55:13.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.843+0000 7f3f793fe700 1 -- 172.21.15.36:0/3241157849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f740afec0 msgr2=0x7f3f74060b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:13.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.843+0000 7f3f793fe700 1 --2- 172.21.15.36:0/3241157849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f740afec0 0x7f3f74060b60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f3f6c0032f0 tx=0x7f3f6c00a370).stop 2022-06-17T12:55:13.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.843+0000 7f3f793fe700 1 -- 172.21.15.36:0/3241157849 shutdown_connections 2022-06-17T12:55:13.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.843+0000 7f3f793fe700 1 --2- 172.21.15.36:0/3241157849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f740afec0 0x7f3f74060b60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.843+0000 7f3f793fe700 1 -- 172.21.15.36:0/3241157849 >> 172.21.15.36:0/3241157849 conn(0x7f3f740b6870 msgr2=0x7f3f740b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:13.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.843+0000 7f3f793fe700 1 -- 172.21.15.36:0/3241157849 shutdown_connections 2022-06-17T12:55:13.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.843+0000 7f3f793fe700 1 -- 172.21.15.36:0/3241157849 wait complete. 2022-06-17T12:55:13.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.847+0000 7f3f793fe700 1 Processor -- start 2022-06-17T12:55:13.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.847+0000 7f3f793fe700 1 -- start start 2022-06-17T12:55:13.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.847+0000 7f3f793fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f74060780 0x7f3f74066a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:13.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.847+0000 7f3f793fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3f740afec0 0x7f3f740ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:13.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.847+0000 7f3f793fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f741434f0 0x7f3f740a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:13.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.847+0000 7f3f793fe700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f740662c0 con 0x7f3f740afec0 2022-06-17T12:55:13.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.847+0000 7f3f793fe700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7409f1b0 con 0x7f3f74060780 2022-06-17T12:55:13.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.847+0000 7f3f793fe700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7409fe50 con 0x7f3f741434f0 2022-06-17T12:55:13.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.847+0000 7f3f727fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3f740afec0 0x7f3f740ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:13.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.847+0000 7f3f72ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f74060780 0x7f3f74066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:13.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.847+0000 7f3f72ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f74060780 0x7f3f74066a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:58480/0 (socket says 172.21.15.36:58480) 2022-06-17T12:55:13.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.847+0000 7f3f737fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f741434f0 0x7f3f740a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:13.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.847+0000 7f3f72ffd700 1 -- 172.21.15.36:0/1244002064 learned_addr learned my addr 172.21.15.36:0/1244002064 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:13.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.851+0000 7f3f727fc700 1 -- 172.21.15.36:0/1244002064 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f741434f0 msgr2=0x7f3f740a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:13.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.851+0000 7f3f727fc700 1 --2- 172.21.15.36:0/1244002064 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f741434f0 0x7f3f740a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.851+0000 7f3f727fc700 1 -- 172.21.15.36:0/1244002064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f74060780 msgr2=0x7f3f74066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:13.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.851+0000 7f3f727fc700 1 --2- 172.21.15.36:0/1244002064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f74060780 0x7f3f74066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:13.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.851+0000 7f3f727fc700 1 -- 172.21.15.36:0/1244002064 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6c00a610 con 0x7f3f740afec0 2022-06-17T12:55:13.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.851+0000 7f3f727fc700 1 --2- 172.21.15.36:0/1244002064 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3f740afec0 0x7f3f740ae5e0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 rx=0x7f3f7409f2f0 tx=0x7f3f68007ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:13.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.851+0000 7f3f63fff700 1 -- 172.21.15.36:0/1244002064 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f68011470 con 0x7f3f740afec0 2022-06-17T12:55:13.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.851+0000 7f3f63fff700 1 -- 172.21.15.36:0/1244002064 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f680115d0 con 0x7f3f740afec0 2022-06-17T12:55:13.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.851+0000 7f3f793fe700 1 -- 172.21.15.36:0/1244002064 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f740664b0 con 0x7f3f740afec0 2022-06-17T12:55:13.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.851+0000 7f3f63fff700 1 -- 172.21.15.36:0/1244002064 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f68011930 con 0x7f3f740afec0 2022-06-17T12:55:13.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.851+0000 7f3f793fe700 1 -- 172.21.15.36:0/1244002064 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f74065ec0 con 0x7f3f740afec0 2022-06-17T12:55:13.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.851+0000 7f3f793fe700 1 -- 172.21.15.36:0/1244002064 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f74066640 con 0x7f3f740afec0 2022-06-17T12:55:13.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.851+0000 7f3f63fff700 1 -- 172.21.15.36:0/1244002064 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f3f68021070 con 0x7f3f740afec0 2022-06-17T12:55:13.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.851+0000 7f3f63fff700 1 --2- 172.21.15.36:0/1244002064 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3f50053a40 0x7f3f50055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:13.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.851+0000 7f3f63fff700 1 -- 172.21.15.36:0/1244002064 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f3f6806e600 con 0x7f3f740afec0 2022-06-17T12:55:13.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.851+0000 7f3f72ffd700 1 --2- 172.21.15.36:0/1244002064 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3f50053a40 0x7f3f50055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:13.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.851+0000 7f3f72ffd700 1 --2- 172.21.15.36:0/1244002064 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3f50053a40 0x7f3f50055f00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f3f6400d000 tx=0x7f3f64009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:13.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:13.859+0000 7f3f63fff700 1 -- 172.21.15.36:0/1244002064 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3f68045330 con 0x7f3f740afec0 2022-06-17T12:55:14.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.023+0000 7f3f793fe700 1 -- 172.21.15.36:0/1244002064 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f74057410 con 0x7f3f740afec0 2022-06-17T12:55:14.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.023+0000 7f3f63fff700 1 -- 172.21.15.36:0/1244002064 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+1777 (secure 0 0 0) 0x7f3f68015020 con 0x7f3f740afec0 2022-06-17T12:55:14.011 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:14.011 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":55,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/676934608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":676934608},{"type":"v1","addr":"172.21.15.70:6833","nonce":676934608}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":51}],"filesystems":[{"mdsmap":{"epoch":55,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:54:59.995522+0000","modified":"2022-06-17T12:55:13.718398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-06-17T12:55:14.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.023+0000 7f3f793fe700 1 -- 172.21.15.36:0/1244002064 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3f50053a40 msgr2=0x7f3f50055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:14.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.023+0000 7f3f793fe700 1 --2- 172.21.15.36:0/1244002064 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3f50053a40 0x7f3f50055f00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f3f6400d000 tx=0x7f3f64009040).stop 2022-06-17T12:55:14.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.023+0000 7f3f793fe700 1 -- 172.21.15.36:0/1244002064 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3f740afec0 msgr2=0x7f3f740ae5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:14.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.023+0000 7f3f793fe700 1 --2- 172.21.15.36:0/1244002064 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3f740afec0 0x7f3f740ae5e0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 rx=0x7f3f7409f2f0 tx=0x7f3f68007ab0).stop 2022-06-17T12:55:14.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.023+0000 7f3f793fe700 1 -- 172.21.15.36:0/1244002064 shutdown_connections 2022-06-17T12:55:14.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.023+0000 7f3f793fe700 1 --2- 172.21.15.36:0/1244002064 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3f50053a40 0x7f3f50055f00 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:14.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.023+0000 7f3f793fe700 1 --2- 172.21.15.36:0/1244002064 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f741434f0 0x7f3f740a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:14.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.023+0000 7f3f793fe700 1 --2- 172.21.15.36:0/1244002064 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3f740afec0 0x7f3f740ae5e0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:14.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.023+0000 7f3f793fe700 1 --2- 172.21.15.36:0/1244002064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f74060780 0x7f3f74066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:14.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.023+0000 7f3f793fe700 1 -- 172.21.15.36:0/1244002064 >> 172.21.15.36:0/1244002064 conn(0x7f3f740b6870 msgr2=0x7f3f740b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:14.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.023+0000 7f3f793fe700 1 -- 172.21.15.36:0/1244002064 shutdown_connections 2022-06-17T12:55:14.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.023+0000 7f3f793fe700 1 -- 172.21.15.36:0/1244002064 wait complete. 2022-06-17T12:55:14.016 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 55 2022-06-17T12:55:14.024 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-17T12:55:14.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.147+0000 7f19c7f06700 1 Processor -- start 2022-06-17T12:55:14.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.147+0000 7f19c7f06700 1 -- start start 2022-06-17T12:55:14.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.147+0000 7f19c7f06700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f19c00acb00 0x7f19c00acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:14.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.147+0000 7f19c7f06700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19c00b05e0 0x7f19c0147480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:14.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c7f06700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c00b1900 con 0x7f19c0147970 2022-06-17T12:55:14.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c7f06700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c0147e00 con 0x7f19c00acb00 2022-06-17T12:55:14.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c7f06700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c00af370 con 0x7f19c00b05e0 2022-06-17T12:55:14.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c5ca2700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f19c00acb00 0x7f19c00acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:14.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c64a3700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f19c0147970 0x7f19c0144240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55640/0 (socket says 172.21.15.36:55640) 2022-06-17T12:55:14.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c64a3700 1 -- 172.21.15.36:0/2648033626 learned_addr learned my addr 172.21.15.36:0/2648033626 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:14.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c54a1700 1 --2- 172.21.15.36:0/2648033626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19c00b05e0 0x7f19c0147480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:14.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c5ca2700 1 -- 172.21.15.36:0/2648033626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19c00b05e0 msgr2=0x7f19c0147480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:14.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c5ca2700 1 --2- 172.21.15.36:0/2648033626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19c00b05e0 0x7f19c0147480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:14.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c5ca2700 1 -- 172.21.15.36:0/2648033626 >> v1:172.21.15.36:6789/0 conn(0x7f19c0147970 legacy=0x7f19c0144240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:14.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c5ca2700 1 -- 172.21.15.36:0/2648033626 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c00b0a60 con 0x7f19c00acb00 2022-06-17T12:55:14.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c4ca0700 1 -- 172.21.15.36:0/2648033626 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4205187065 0 0) 0x7f19c00b1900 con 0x7f19c0147970 2022-06-17T12:55:14.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c5ca2700 1 --2- 172.21.15.36:0/2648033626 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19c00acb00 0x7f19c00acee0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 rx=0x7f19b0002f70 tx=0x7f19b0006df0).ready entity=mon.0 client_cookie=d675ebca55c23482 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:14.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c4ca0700 1 -- 172.21.15.36:0/2648033626 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19b000d570 con 0x7f19c00acb00 2022-06-17T12:55:14.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c4ca0700 1 -- 172.21.15.36:0/2648033626 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b0000e90 con 0x7f19c00acb00 2022-06-17T12:55:14.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c4ca0700 1 -- 172.21.15.36:0/2648033626 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19b000d8c0 con 0x7f19c00acb00 2022-06-17T12:55:14.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c7f06700 1 -- 172.21.15.36:0/2648033626 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19c00acb00 msgr2=0x7f19c00acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:14.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c7f06700 1 --2- 172.21.15.36:0/2648033626 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19c00acb00 0x7f19c00acee0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 rx=0x7f19b0002f70 tx=0x7f19b0006df0).stop 2022-06-17T12:55:14.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c7f06700 1 -- 172.21.15.36:0/2648033626 shutdown_connections 2022-06-17T12:55:14.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c7f06700 1 --2- 172.21.15.36:0/2648033626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19c00b05e0 0x7f19c0147480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:14.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c7f06700 1 --2- 172.21.15.36:0/2648033626 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19c00acb00 0x7f19c00acee0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:14.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c7f06700 1 -- 172.21.15.36:0/2648033626 >> 172.21.15.36:0/2648033626 conn(0x7f19c005c550 msgr2=0x7f19c00b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:14.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c7f06700 1 -- 172.21.15.36:0/2648033626 shutdown_connections 2022-06-17T12:55:14.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c7f06700 1 -- 172.21.15.36:0/2648033626 wait complete. 2022-06-17T12:55:14.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c7f06700 1 Processor -- start 2022-06-17T12:55:14.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.151+0000 7f19c7f06700 1 -- start start 2022-06-17T12:55:14.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.155+0000 7f19c7f06700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19c00acb00 0x7f19c00b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:14.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.155+0000 7f19c7f06700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19c00b05e0 0x7f19c0065f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:14.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.155+0000 7f19c7f06700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19c0147970 0x7f19c00643d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:14.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.155+0000 7f19c7f06700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c0063d60 con 0x7f19c0147970 2022-06-17T12:55:14.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.155+0000 7f19c7f06700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c0066fc0 con 0x7f19c00b05e0 2022-06-17T12:55:14.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.155+0000 7f19c7f06700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c0067260 con 0x7f19c00acb00 2022-06-17T12:55:14.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.155+0000 7f19c64a3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19c0147970 0x7f19c00643d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:14.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.155+0000 7f19c64a3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19c0147970 0x7f19c00643d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38116/0 (socket says 172.21.15.36:38116) 2022-06-17T12:55:14.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.155+0000 7f19c64a3700 1 -- 172.21.15.36:0/2514439099 learned_addr learned my addr 172.21.15.36:0/2514439099 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:14.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.155+0000 7f19c54a1700 1 --2- 172.21.15.36:0/2514439099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19c00b05e0 0x7f19c0065f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:14.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.155+0000 7f19c5ca2700 1 --2- 172.21.15.36:0/2514439099 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19c00acb00 0x7f19c00b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:14.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.155+0000 7f19c64a3700 1 -- 172.21.15.36:0/2514439099 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19c00acb00 msgr2=0x7f19c00b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:14.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.155+0000 7f19c64a3700 1 --2- 172.21.15.36:0/2514439099 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19c00acb00 0x7f19c00b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:14.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.155+0000 7f19c64a3700 1 -- 172.21.15.36:0/2514439099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19c00b05e0 msgr2=0x7f19c0065f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:14.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.155+0000 7f19c64a3700 1 --2- 172.21.15.36:0/2514439099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19c00b05e0 0x7f19c0065f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:14.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.155+0000 7f19c64a3700 1 -- 172.21.15.36:0/2514439099 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b000b180 con 0x7f19c0147970 2022-06-17T12:55:14.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.155+0000 7f19c54a1700 1 --2- 172.21.15.36:0/2514439099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19c00b05e0 0x7f19c0065f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:14.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.155+0000 7f19c64a3700 1 --2- 172.21.15.36:0/2514439099 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19c0147970 0x7f19c00643d0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 rx=0x7f19b8001250 tx=0x7f19b8001360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:14.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.155+0000 7f19aeffd700 1 -- 172.21.15.36:0/2514439099 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19b8005c70 con 0x7f19c0147970 2022-06-17T12:55:14.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.155+0000 7f19aeffd700 1 -- 172.21.15.36:0/2514439099 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b8005dd0 con 0x7f19c0147970 2022-06-17T12:55:14.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.155+0000 7f19c7f06700 1 -- 172.21.15.36:0/2514439099 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c0067500 con 0x7f19c0147970 2022-06-17T12:55:14.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.155+0000 7f19aeffd700 1 -- 172.21.15.36:0/2514439099 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19b8004a90 con 0x7f19c0147970 2022-06-17T12:55:14.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.155+0000 7f19c7f06700 1 -- 172.21.15.36:0/2514439099 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c00625a0 con 0x7f19c0147970 2022-06-17T12:55:14.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.155+0000 7f19c7f06700 1 -- 172.21.15.36:0/2514439099 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c0057410 con 0x7f19c0147970 2022-06-17T12:55:14.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.159+0000 7f19aeffd700 1 -- 172.21.15.36:0/2514439099 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f19b8004bf0 con 0x7f19c0147970 2022-06-17T12:55:14.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.159+0000 7f19aeffd700 1 --2- 172.21.15.36:0/2514439099 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f19a0053a40 0x7f19a0055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:14.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.159+0000 7f19aeffd700 1 -- 172.21.15.36:0/2514439099 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f19b806bcb0 con 0x7f19c0147970 2022-06-17T12:55:14.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.159+0000 7f19c5ca2700 1 --2- 172.21.15.36:0/2514439099 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f19a0053a40 0x7f19a0055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:14.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.159+0000 7f19c5ca2700 1 --2- 172.21.15.36:0/2514439099 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f19a0053a40 0x7f19a0055f00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f19b001a000 tx=0x7f19b0014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:14.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.163+0000 7f19aeffd700 1 -- 172.21.15.36:0/2514439099 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f19b8042a60 con 0x7f19c0147970 2022-06-17T12:55:14.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.327+0000 7f19c7f06700 1 -- 172.21.15.36:0/2514439099 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f19c005f340 con 0x7f19c0147970 2022-06-17T12:55:14.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.739+0000 7f19aeffd700 1 -- 172.21.15.36:0/2514439099 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v56) v1 ==== 104+0+0 (secure 0 0 0) 0x7f19b8046080 con 0x7f19c0147970 2022-06-17T12:55:14.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.743+0000 7f19c7f06700 1 -- 172.21.15.36:0/2514439099 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f19a0053a40 msgr2=0x7f19a0055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:14.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.743+0000 7f19c7f06700 1 --2- 172.21.15.36:0/2514439099 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f19a0053a40 0x7f19a0055f00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f19b001a000 tx=0x7f19b0014000).stop 2022-06-17T12:55:14.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.743+0000 7f19c7f06700 1 -- 172.21.15.36:0/2514439099 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19c0147970 msgr2=0x7f19c00643d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:14.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.743+0000 7f19c7f06700 1 --2- 172.21.15.36:0/2514439099 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19c0147970 0x7f19c00643d0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 rx=0x7f19b8001250 tx=0x7f19b8001360).stop 2022-06-17T12:55:14.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.743+0000 7f19c7f06700 1 -- 172.21.15.36:0/2514439099 shutdown_connections 2022-06-17T12:55:14.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.743+0000 7f19c7f06700 1 --2- 172.21.15.36:0/2514439099 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f19a0053a40 0x7f19a0055f00 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:14.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.743+0000 7f19c7f06700 1 --2- 172.21.15.36:0/2514439099 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f19c0147970 0x7f19c00643d0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:14.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.743+0000 7f19c7f06700 1 --2- 172.21.15.36:0/2514439099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19c00b05e0 0x7f19c0065f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:14.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.743+0000 7f19c7f06700 1 --2- 172.21.15.36:0/2514439099 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19c00acb00 0x7f19c00b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:14.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.743+0000 7f19c7f06700 1 -- 172.21.15.36:0/2514439099 >> 172.21.15.36:0/2514439099 conn(0x7f19c005c550 msgr2=0x7f19c014def0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:14.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.743+0000 7f19c7f06700 1 -- 172.21.15.36:0/2514439099 shutdown_connections 2022-06-17T12:55:14.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.743+0000 7f19c7f06700 1 -- 172.21.15.36:0/2514439099 wait complete. 2022-06-17T12:55:14.746 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-17T12:55:14.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.879+0000 7f81312d4700 1 Processor -- start 2022-06-17T12:55:14.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.879+0000 7f81312d4700 1 -- start start 2022-06-17T12:55:14.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f81312d4700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f812c0620d0 0x7f812c0b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:14.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f81312d4700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f812c0b31c0 0x7f812c0b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:14.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f81312d4700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f812c0b21d0 con 0x7f812c0b4700 2022-06-17T12:55:14.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f81312d4700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f812c0b2430 con 0x7f812c0620d0 2022-06-17T12:55:14.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f81312d4700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812c0b26d0 con 0x7f812c0b31c0 2022-06-17T12:55:14.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f812affd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f812c0620d0 0x7f812c0b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:14.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f812b7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f812c0b4700 0x7f812c146660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55656/0 (socket says 172.21.15.36:55656) 2022-06-17T12:55:14.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f812b7fe700 1 -- 172.21.15.36:0/1883623877 learned_addr learned my addr 172.21.15.36:0/1883623877 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:14.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f8123fff700 1 --2- 172.21.15.36:0/1883623877 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f812c0b31c0 0x7f812c0b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:14.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f812affd700 1 -- 172.21.15.36:0/1883623877 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f812c0b31c0 msgr2=0x7f812c0b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:14.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f812affd700 1 --2- 172.21.15.36:0/1883623877 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f812c0b31c0 0x7f812c0b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:14.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f812affd700 1 -- 172.21.15.36:0/1883623877 >> v1:172.21.15.36:6789/0 conn(0x7f812c0b4700 legacy=0x7f812c146660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:14.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f812a7fc700 1 -- 172.21.15.36:0/1883623877 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122883158 0 0) 0x7f812c0b21d0 con 0x7f812c0b4700 2022-06-17T12:55:14.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f812affd700 1 -- 172.21.15.36:0/1883623877 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812c0b43f0 con 0x7f812c0620d0 2022-06-17T12:55:14.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f812affd700 1 --2- 172.21.15.36:0/1883623877 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f812c0620d0 0x7f812c0b5bc0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7f811c002f70 tx=0x7f811c006df0).ready entity=mon.0 client_cookie=f125778a61043707 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:14.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f812a7fc700 1 -- 172.21.15.36:0/1883623877 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f811c00d570 con 0x7f812c0620d0 2022-06-17T12:55:14.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f812a7fc700 1 -- 172.21.15.36:0/1883623877 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811c000e90 con 0x7f812c0620d0 2022-06-17T12:55:14.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f812a7fc700 1 -- 172.21.15.36:0/1883623877 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f811c00d8c0 con 0x7f812c0620d0 2022-06-17T12:55:14.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f81312d4700 1 -- 172.21.15.36:0/1883623877 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f812c0620d0 msgr2=0x7f812c0b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:14.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f81312d4700 1 --2- 172.21.15.36:0/1883623877 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f812c0620d0 0x7f812c0b5bc0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7f811c002f70 tx=0x7f811c006df0).stop 2022-06-17T12:55:14.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f81312d4700 1 -- 172.21.15.36:0/1883623877 shutdown_connections 2022-06-17T12:55:14.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f81312d4700 1 --2- 172.21.15.36:0/1883623877 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f812c0b31c0 0x7f812c0b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:14.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f81312d4700 1 --2- 172.21.15.36:0/1883623877 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f812c0620d0 0x7f812c0b5bc0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:14.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f81312d4700 1 -- 172.21.15.36:0/1883623877 >> 172.21.15.36:0/1883623877 conn(0x7f812c0b9850 msgr2=0x7f812c0b9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:14.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f81312d4700 1 -- 172.21.15.36:0/1883623877 shutdown_connections 2022-06-17T12:55:14.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.883+0000 7f81312d4700 1 -- 172.21.15.36:0/1883623877 wait complete. 2022-06-17T12:55:14.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.887+0000 7f81312d4700 1 Processor -- start 2022-06-17T12:55:14.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.887+0000 7f81312d4700 1 -- start start 2022-06-17T12:55:14.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.887+0000 7f81312d4700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f812c0620d0 0x7f812c146710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:14.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.887+0000 7f81312d4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f812c0b31c0 0x7f812c151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:14.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.887+0000 7f81312d4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f812c0b4700 0x7f812c151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:14.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.887+0000 7f81312d4700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c152480 con 0x7f812c0b31c0 2022-06-17T12:55:14.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.887+0000 7f81312d4700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c0afcf0 con 0x7f812c0b4700 2022-06-17T12:55:14.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.887+0000 7f81312d4700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812c0aff90 con 0x7f812c0620d0 2022-06-17T12:55:14.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.887+0000 7f8123fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f812c0b31c0 0x7f812c151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:14.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.887+0000 7f8123fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f812c0b31c0 0x7f812c151aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38130/0 (socket says 172.21.15.36:38130) 2022-06-17T12:55:14.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.887+0000 7f8123fff700 1 -- 172.21.15.36:0/3547585981 learned_addr learned my addr 172.21.15.36:0/3547585981 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:14.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.887+0000 7f812b7fe700 1 --2- 172.21.15.36:0/3547585981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f812c0b4700 0x7f812c151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:14.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.887+0000 7f812affd700 1 --2- 172.21.15.36:0/3547585981 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f812c0620d0 0x7f812c146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:14.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.887+0000 7f8123fff700 1 -- 172.21.15.36:0/3547585981 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f812c0620d0 msgr2=0x7f812c146710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:14.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.887+0000 7f8123fff700 1 --2- 172.21.15.36:0/3547585981 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f812c0620d0 0x7f812c146710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:14.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.887+0000 7f8123fff700 1 -- 172.21.15.36:0/3547585981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f812c0b4700 msgr2=0x7f812c151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:14.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.887+0000 7f8123fff700 1 --2- 172.21.15.36:0/3547585981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f812c0b4700 0x7f812c151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:14.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.887+0000 7f8123fff700 1 -- 172.21.15.36:0/3547585981 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811c00b180 con 0x7f812c0b31c0 2022-06-17T12:55:14.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.887+0000 7f8123fff700 1 --2- 172.21.15.36:0/3547585981 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f812c0b31c0 0x7f812c151aa0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0x7f811400c000 tx=0x7f8114006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:14.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.887+0000 7f8128ff9700 1 -- 172.21.15.36:0/3547585981 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8114010780 con 0x7f812c0b31c0 2022-06-17T12:55:14.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.887+0000 7f8128ff9700 1 -- 172.21.15.36:0/3547585981 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81140108e0 con 0x7f812c0b31c0 2022-06-17T12:55:14.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.887+0000 7f81312d4700 1 -- 172.21.15.36:0/3547585981 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f812c0b0230 con 0x7f812c0b31c0 2022-06-17T12:55:14.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.887+0000 7f8128ff9700 1 -- 172.21.15.36:0/3547585981 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8114010c40 con 0x7f812c0b31c0 2022-06-17T12:55:14.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.887+0000 7f81312d4700 1 -- 172.21.15.36:0/3547585981 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812c1144c0 con 0x7f812c0b31c0 2022-06-17T12:55:14.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.891+0000 7f8128ff9700 1 -- 172.21.15.36:0/3547585981 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f811401c460 con 0x7f812c0b31c0 2022-06-17T12:55:14.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.891+0000 7f81312d4700 1 -- 172.21.15.36:0/3547585981 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f812c115a90 con 0x7f812c0b31c0 2022-06-17T12:55:14.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.891+0000 7f8128ff9700 1 --2- 172.21.15.36:0/3547585981 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8110053a40 0x7f8110055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:14.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.891+0000 7f8128ff9700 1 -- 172.21.15.36:0/3547585981 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f811406c480 con 0x7f812c0b31c0 2022-06-17T12:55:14.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.891+0000 7f812affd700 1 --2- 172.21.15.36:0/3547585981 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8110053a40 0x7f8110055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:14.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.891+0000 7f812affd700 1 --2- 172.21.15.36:0/3547585981 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8110053a40 0x7f8110055f00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f811c01a000 tx=0x7f811c014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:14.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:14.899+0000 7f8128ff9700 1 -- 172.21.15.36:0/3547585981 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8114043130 con 0x7f812c0b31c0 2022-06-17T12:55:15.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.027+0000 7f81312d4700 1 -- 172.21.15.36:0/3547585981 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f812c0624b0 con 0x7f812c0b31c0 2022-06-17T12:55:15.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.743+0000 7f8128ff9700 1 -- 172.21.15.36:0/3547585981 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v106) v1 ==== 185+0+0 (secure 0 0 0) 0x7f8114046750 con 0x7f812c0b31c0 2022-06-17T12:55:15.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.751+0000 7f81312d4700 1 -- 172.21.15.36:0/3547585981 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8110053a40 msgr2=0x7f8110055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:15.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.751+0000 7f81312d4700 1 --2- 172.21.15.36:0/3547585981 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8110053a40 0x7f8110055f00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f811c01a000 tx=0x7f811c014000).stop 2022-06-17T12:55:15.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.751+0000 7f81312d4700 1 -- 172.21.15.36:0/3547585981 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f812c0b31c0 msgr2=0x7f812c151aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:15.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.751+0000 7f81312d4700 1 --2- 172.21.15.36:0/3547585981 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f812c0b31c0 0x7f812c151aa0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0x7f811400c000 tx=0x7f8114006df0).stop 2022-06-17T12:55:15.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.751+0000 7f81312d4700 1 -- 172.21.15.36:0/3547585981 shutdown_connections 2022-06-17T12:55:15.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.751+0000 7f81312d4700 1 --2- 172.21.15.36:0/3547585981 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8110053a40 0x7f8110055f00 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:15.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.751+0000 7f81312d4700 1 --2- 172.21.15.36:0/3547585981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f812c0b4700 0x7f812c151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:15.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.751+0000 7f81312d4700 1 --2- 172.21.15.36:0/3547585981 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f812c0b31c0 0x7f812c151aa0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:15.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.751+0000 7f81312d4700 1 --2- 172.21.15.36:0/3547585981 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f812c0620d0 0x7f812c146710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:15.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.751+0000 7f81312d4700 1 -- 172.21.15.36:0/3547585981 >> 172.21.15.36:0/3547585981 conn(0x7f812c0b9850 msgr2=0x7f812c142700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:15.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.751+0000 7f81312d4700 1 -- 172.21.15.36:0/3547585981 shutdown_connections 2022-06-17T12:55:15.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.751+0000 7f81312d4700 1 -- 172.21.15.36:0/3547585981 wait complete. 2022-06-17T12:55:15.744 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' removed 2022-06-17T12:55:15.763 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-17T12:55:15.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.895+0000 7f89c8067700 1 Processor -- start 2022-06-17T12:55:15.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.895+0000 7f89c8067700 1 -- start start 2022-06-17T12:55:15.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.895+0000 7f89c8067700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89c00620d0 0x7f89c00b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:15.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.895+0000 7f89c8067700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f89c00b31c0 0x7f89c00b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:15.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.895+0000 7f89c8067700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89c00b21d0 con 0x7f89c00b4700 2022-06-17T12:55:15.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.895+0000 7f89c8067700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c00b2430 con 0x7f89c00b31c0 2022-06-17T12:55:15.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.895+0000 7f89c8067700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c00b26d0 con 0x7f89c00620d0 2022-06-17T12:55:15.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.895+0000 7f89c6604700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f89c00b4700 0x7f89c0146660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55670/0 (socket says 172.21.15.36:55670) 2022-06-17T12:55:15.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.895+0000 7f89c6604700 1 -- 172.21.15.36:0/2127727951 learned_addr learned my addr 172.21.15.36:0/2127727951 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:15.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c5602700 1 --2- 172.21.15.36:0/2127727951 >> v2:172.21.15.70:3300/0 conn(0x7f89c00b31c0 0x7f89c00b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:15.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c5e03700 1 --2- 172.21.15.36:0/2127727951 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89c00620d0 0x7f89c00b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:15.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c4e01700 1 -- 172.21.15.36:0/2127727951 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 551415386 0 0) 0x7f89c00b21d0 con 0x7f89c00b4700 2022-06-17T12:55:15.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c4e01700 1 -- 172.21.15.36:0/2127727951 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89a8003260 con 0x7f89c00b4700 2022-06-17T12:55:15.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c4e01700 1 -- 172.21.15.36:0/2127727951 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1770996462 0 0) 0x7f89a8003260 con 0x7f89c00b4700 2022-06-17T12:55:15.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c4e01700 1 -- 172.21.15.36:0/2127727951 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89c00620d0 msgr2=0x7f89c00b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:15.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c4e01700 1 --2- 172.21.15.36:0/2127727951 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89c00620d0 0x7f89c00b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:15.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c4e01700 1 -- 172.21.15.36:0/2127727951 >> v2:172.21.15.70:3300/0 conn(0x7f89c00b31c0 msgr2=0x7f89c00b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:15.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c4e01700 1 --2- 172.21.15.36:0/2127727951 >> v2:172.21.15.70:3300/0 conn(0x7f89c00b31c0 0x7f89c00b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:15.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c4e01700 1 -- 172.21.15.36:0/2127727951 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c00b43f0 con 0x7f89c00b4700 2022-06-17T12:55:15.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c4e01700 1 -- 172.21.15.36:0/2127727951 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f89b80019e0 con 0x7f89c00b4700 2022-06-17T12:55:15.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c4e01700 1 -- 172.21.15.36:0/2127727951 >> v1:172.21.15.36:6789/0 conn(0x7f89c00b4700 legacy=0x7f89c0146660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:15.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c4e01700 1 --2- 172.21.15.36:0/2127727951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89a8003750 0x7f89a8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:15.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c4e01700 1 --2- 172.21.15.36:0/2127727951 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89c00620d0 0x7f89a8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:15.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c4e01700 1 --2- 172.21.15.36:0/2127727951 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f89a80049f0 0x7f89a8004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:15.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c4e01700 1 -- 172.21.15.36:0/2127727951 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c00b2430 con 0x7f89a80049f0 2022-06-17T12:55:15.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c4e01700 1 -- 172.21.15.36:0/2127727951 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c00b26d0 con 0x7f89a8003750 2022-06-17T12:55:15.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c4e01700 1 -- 172.21.15.36:0/2127727951 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a800b750 con 0x7f89c00620d0 2022-06-17T12:55:15.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c6604700 1 --2- 172.21.15.36:0/2127727951 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f89a80049f0 0x7f89a8004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:15.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c5e03700 1 --2- 172.21.15.36:0/2127727951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89a8003750 0x7f89a8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:15.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c5602700 1 --2- 172.21.15.36:0/2127727951 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89c00620d0 0x7f89a8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:15.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c5602700 1 -- 172.21.15.36:0/2127727951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89a8003750 msgr2=0x7f89a8003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:15.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c5602700 1 --2- 172.21.15.36:0/2127727951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89a8003750 0x7f89a8003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:15.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c5602700 1 -- 172.21.15.36:0/2127727951 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f89a80049f0 msgr2=0x7f89a8004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:15.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c5602700 1 --2- 172.21.15.36:0/2127727951 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f89a80049f0 0x7f89a8004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:15.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c5602700 1 -- 172.21.15.36:0/2127727951 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a8003260 con 0x7f89c00620d0 2022-06-17T12:55:15.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c6604700 1 --2- 172.21.15.36:0/2127727951 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f89a80049f0 0x7f89a8004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:15.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c5602700 1 --2- 172.21.15.36:0/2127727951 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89c00620d0 0x7f89a8004500 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f89bc00c000 tx=0x7f89bc006df0).ready entity=mon.2 client_cookie=39a92439b86ee536 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:15.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c4e01700 1 -- 172.21.15.36:0/2127727951 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89bc00e800 con 0x7f89c00620d0 2022-06-17T12:55:15.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c4e01700 1 -- 172.21.15.36:0/2127727951 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89bc00e960 con 0x7f89c00620d0 2022-06-17T12:55:15.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c4e01700 1 -- 172.21.15.36:0/2127727951 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89bc00ecc0 con 0x7f89c00620d0 2022-06-17T12:55:15.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c8067700 1 -- 172.21.15.36:0/2127727951 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89c00620d0 msgr2=0x7f89a8004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:15.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c8067700 1 --2- 172.21.15.36:0/2127727951 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89c00620d0 0x7f89a8004500 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f89bc00c000 tx=0x7f89bc006df0).stop 2022-06-17T12:55:15.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c6604700 1 -- 172.21.15.36:0/2127727951 reap_dead start 2022-06-17T12:55:15.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c8067700 1 -- 172.21.15.36:0/2127727951 shutdown_connections 2022-06-17T12:55:15.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c8067700 1 -- 172.21.15.36:0/2127727951 >> 172.21.15.36:0/2127727951 conn(0x7f89c00b9850 msgr2=0x7f89c00b9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:15.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c8067700 1 -- 172.21.15.36:0/2127727951 shutdown_connections 2022-06-17T12:55:15.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.899+0000 7f89c8067700 1 -- 172.21.15.36:0/2127727951 wait complete. 2022-06-17T12:55:15.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.903+0000 7f89c8067700 1 Processor -- start 2022-06-17T12:55:15.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.903+0000 7f89c8067700 1 -- start start 2022-06-17T12:55:15.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.903+0000 7f89c8067700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89c00620d0 0x7f89c00af020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:15.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.903+0000 7f89c8067700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89c00b8af0 0x7f89c01401e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:15.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.903+0000 7f89c8067700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f89c01406d0 0x7f89c0155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:15.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.903+0000 7f89c8067700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c00ad3d0 con 0x7f89c01406d0 2022-06-17T12:55:15.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.903+0000 7f89c8067700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c00ad600 con 0x7f89c00620d0 2022-06-17T12:55:15.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.903+0000 7f89c8067700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c00ae150 con 0x7f89c00b8af0 2022-06-17T12:55:15.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.903+0000 7f89c6604700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f89c01406d0 0x7f89c0155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:15.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.903+0000 7f89c6604700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f89c01406d0 0x7f89c0155e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38152/0 (socket says 172.21.15.36:38152) 2022-06-17T12:55:15.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.903+0000 7f89c5602700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89c00b8af0 0x7f89c01401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:15.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.903+0000 7f89c6604700 1 -- 172.21.15.36:0/2624916378 learned_addr learned my addr 172.21.15.36:0/2624916378 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:15.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.903+0000 7f89c5e03700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89c00620d0 0x7f89c00af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:15.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.903+0000 7f89c6604700 1 -- 172.21.15.36:0/2624916378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89c00b8af0 msgr2=0x7f89c01401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:15.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.903+0000 7f89c6604700 1 --2- 172.21.15.36:0/2624916378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89c00b8af0 0x7f89c01401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:15.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.903+0000 7f89c6604700 1 -- 172.21.15.36:0/2624916378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89c00620d0 msgr2=0x7f89c00af020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:15.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.903+0000 7f89c6604700 1 --2- 172.21.15.36:0/2624916378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89c00620d0 0x7f89c00af020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:15.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.903+0000 7f89c6604700 1 -- 172.21.15.36:0/2624916378 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c0152750 con 0x7f89c01406d0 2022-06-17T12:55:15.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.903+0000 7f89c6604700 1 --2- 172.21.15.36:0/2624916378 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f89c01406d0 0x7f89c0155e30 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7f89b8000b60 tx=0x7f89b80021d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:15.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.907+0000 7f89b2ffd700 1 -- 172.21.15.36:0/2624916378 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89b800f870 con 0x7f89c01406d0 2022-06-17T12:55:15.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.907+0000 7f89b2ffd700 1 -- 172.21.15.36:0/2624916378 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b800f9d0 con 0x7f89c01406d0 2022-06-17T12:55:15.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.907+0000 7f89c8067700 1 -- 172.21.15.36:0/2624916378 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c0140b80 con 0x7f89c01406d0 2022-06-17T12:55:15.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.907+0000 7f89b2ffd700 1 -- 172.21.15.36:0/2624916378 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89b800fd30 con 0x7f89c01406d0 2022-06-17T12:55:15.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.907+0000 7f89c8067700 1 -- 172.21.15.36:0/2624916378 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c00ad9e0 con 0x7f89c01406d0 2022-06-17T12:55:15.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.907+0000 7f89c8067700 1 -- 172.21.15.36:0/2624916378 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c00b8ed0 con 0x7f89c01406d0 2022-06-17T12:55:15.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.907+0000 7f89b2ffd700 1 -- 172.21.15.36:0/2624916378 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f89b801c360 con 0x7f89c01406d0 2022-06-17T12:55:15.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.907+0000 7f89b2ffd700 1 --2- 172.21.15.36:0/2624916378 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f89a0053a40 0x7f89a0055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:15.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.907+0000 7f89b2ffd700 1 -- 172.21.15.36:0/2624916378 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f89b806ba60 con 0x7f89c01406d0 2022-06-17T12:55:15.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.907+0000 7f89c5e03700 1 --2- 172.21.15.36:0/2624916378 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f89a0053a40 0x7f89a0055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:15.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.911+0000 7f89c5e03700 1 --2- 172.21.15.36:0/2624916378 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f89a0053a40 0x7f89a0055f00 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f89b400b000 tx=0x7f89b4006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:15.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:15.915+0000 7f89b2ffd700 1 -- 172.21.15.36:0/2624916378 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f89b8042890 con 0x7f89c01406d0 2022-06-17T12:55:16.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.043+0000 7f89c8067700 1 -- 172.21.15.36:0/2624916378 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f89c00624b0 con 0x7f89c01406d0 2022-06-17T12:55:16.192 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:55:16.196 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:55:16.202 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:55:16.206 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:55:16.210 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:55:16.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.751+0000 7f89b2ffd700 1 -- 172.21.15.36:0/2624916378 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v107) v1 ==== 173+0+0 (secure 0 0 0) 0x7f89b8019020 con 0x7f89c01406d0 2022-06-17T12:55:16.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.755+0000 7f89c8067700 1 -- 172.21.15.36:0/2624916378 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f89a0053a40 msgr2=0x7f89a0055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:16.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.755+0000 7f89c8067700 1 --2- 172.21.15.36:0/2624916378 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f89a0053a40 0x7f89a0055f00 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f89b400b000 tx=0x7f89b4006df0).stop 2022-06-17T12:55:16.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.755+0000 7f89c8067700 1 -- 172.21.15.36:0/2624916378 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f89c01406d0 msgr2=0x7f89c0155e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:16.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.755+0000 7f89c8067700 1 --2- 172.21.15.36:0/2624916378 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f89c01406d0 0x7f89c0155e30 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7f89b8000b60 tx=0x7f89b80021d0).stop 2022-06-17T12:55:16.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.755+0000 7f89c8067700 1 -- 172.21.15.36:0/2624916378 shutdown_connections 2022-06-17T12:55:16.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.755+0000 7f89c8067700 1 --2- 172.21.15.36:0/2624916378 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f89a0053a40 0x7f89a0055f00 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:16.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.755+0000 7f89c8067700 1 --2- 172.21.15.36:0/2624916378 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f89c01406d0 0x7f89c0155e30 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:16.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.755+0000 7f89c8067700 1 --2- 172.21.15.36:0/2624916378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f89c00b8af0 0x7f89c01401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:16.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.755+0000 7f89c8067700 1 --2- 172.21.15.36:0/2624916378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89c00620d0 0x7f89c00af020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:16.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.755+0000 7f89c8067700 1 -- 172.21.15.36:0/2624916378 >> 172.21.15.36:0/2624916378 conn(0x7f89c00b9850 msgr2=0x7f89c0143ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:16.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.755+0000 7f89c8067700 1 -- 172.21.15.36:0/2624916378 shutdown_connections 2022-06-17T12:55:16.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.755+0000 7f89c8067700 1 -- 172.21.15.36:0/2624916378 wait complete. 2022-06-17T12:55:16.763 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' removed 2022-06-17T12:55:16.766 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-06-17T12:55:16.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.891+0000 7fecefb32700 1 Processor -- start 2022-06-17T12:55:16.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.891+0000 7fecefb32700 1 -- start start 2022-06-17T12:55:16.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.891+0000 7fecefb32700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fece8067230 0x7fece80650d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:16.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.891+0000 7fecefb32700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fece8141050 0x7fece81414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:16.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.891+0000 7fecefb32700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fece80676c0 con 0x7fece80b7d40 2022-06-17T12:55:16.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.891+0000 7fecefb32700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fece8067e40 con 0x7fece8141050 2022-06-17T12:55:16.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.891+0000 7fecefb32700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece8066790 con 0x7fece8067230 2022-06-17T12:55:16.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.891+0000 7feced8ce700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fece80b7d40 0x7fece80649c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:51420/0 (socket says 172.21.15.36:51420) 2022-06-17T12:55:16.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.891+0000 7feced0cd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fece8067230 0x7fece80650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:16.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.891+0000 7feced8ce700 1 -- 172.21.15.36:0/2948952515 learned_addr learned my addr 172.21.15.36:0/2948952515 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:16.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.891+0000 7fecee0cf700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fece8141050 0x7fece81414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:16.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.891+0000 7fecec8cc700 1 -- 172.21.15.36:0/2948952515 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2178671960 0 0) 0x7fece4001490 con 0x7fece80b7d40 2022-06-17T12:55:16.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.891+0000 7fecec8cc700 1 -- 172.21.15.36:0/2948952515 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feccc003260 con 0x7fece80b7d40 2022-06-17T12:55:16.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.891+0000 7feced0cd700 1 -- 172.21.15.36:0/2948952515 >> v2:172.21.15.70:3300/0 conn(0x7fece8141050 msgr2=0x7fece81414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:16.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.891+0000 7feced0cd700 1 --2- 172.21.15.36:0/2948952515 >> v2:172.21.15.70:3300/0 conn(0x7fece8141050 0x7fece81414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:16.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.891+0000 7feced0cd700 1 -- 172.21.15.36:0/2948952515 >> v1:172.21.15.70:6789/0 conn(0x7fece80b7d40 legacy=0x7fece80649c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:16.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.891+0000 7feced0cd700 1 -- 172.21.15.36:0/2948952515 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece80b8120 con 0x7fece8067230 2022-06-17T12:55:16.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.891+0000 7feced0cd700 1 --2- 172.21.15.36:0/2948952515 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fece8067230 0x7fece80650d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fecdc002f70 tx=0x7fecdc006df0).ready entity=mon.2 client_cookie=14d4d0ceec5ee19 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:16.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.895+0000 7fecec8cc700 1 -- 172.21.15.36:0/2948952515 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecdc00d4a0 con 0x7fece8067230 2022-06-17T12:55:16.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.895+0000 7fecec8cc700 1 -- 172.21.15.36:0/2948952515 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecdc000e90 con 0x7fece8067230 2022-06-17T12:55:16.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.895+0000 7fecec8cc700 1 -- 172.21.15.36:0/2948952515 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecdc00d7f0 con 0x7fece8067230 2022-06-17T12:55:16.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.895+0000 7fecefb32700 1 -- 172.21.15.36:0/2948952515 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fece8067230 msgr2=0x7fece80650d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:16.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.895+0000 7fecefb32700 1 --2- 172.21.15.36:0/2948952515 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fece8067230 0x7fece80650d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fecdc002f70 tx=0x7fecdc006df0).stop 2022-06-17T12:55:16.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.895+0000 7fecefb32700 1 -- 172.21.15.36:0/2948952515 shutdown_connections 2022-06-17T12:55:16.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.895+0000 7fecefb32700 1 --2- 172.21.15.36:0/2948952515 >> v2:172.21.15.70:3300/0 conn(0x7fece8141050 0x7fece81414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:16.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.895+0000 7fecefb32700 1 --2- 172.21.15.36:0/2948952515 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fece8067230 0x7fece80650d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:16.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.895+0000 7fecefb32700 1 -- 172.21.15.36:0/2948952515 >> 172.21.15.36:0/2948952515 conn(0x7fece805c920 msgr2=0x7fece8060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:16.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.895+0000 7fecefb32700 1 -- 172.21.15.36:0/2948952515 shutdown_connections 2022-06-17T12:55:16.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.895+0000 7fecefb32700 1 -- 172.21.15.36:0/2948952515 wait complete. 2022-06-17T12:55:16.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.895+0000 7fecefb32700 1 Processor -- start 2022-06-17T12:55:16.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.895+0000 7fecefb32700 1 -- start start 2022-06-17T12:55:16.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.899+0000 7fecefb32700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fece8067230 0x7fece81441c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:16.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.899+0000 7fecefb32700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fece80b7d40 0x7fece813d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:16.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.899+0000 7fecefb32700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fece8141050 0x7fece813ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:16.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.899+0000 7fecefb32700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece813ddf0 con 0x7fece8141050 2022-06-17T12:55:16.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.899+0000 7fecefb32700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece813d0c0 con 0x7fece80b7d40 2022-06-17T12:55:16.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.899+0000 7fecefb32700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece813ce20 con 0x7fece8067230 2022-06-17T12:55:16.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.899+0000 7fecee0cf700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fece8141050 0x7fece813ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:16.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.899+0000 7feced0cd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fece80b7d40 0x7fece813d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:16.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.899+0000 7fecee0cf700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fece8141050 0x7fece813ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38166/0 (socket says 172.21.15.36:38166) 2022-06-17T12:55:16.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.899+0000 7fecee0cf700 1 -- 172.21.15.36:0/4225415863 learned_addr learned my addr 172.21.15.36:0/4225415863 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:16.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.899+0000 7feced8ce700 1 --2- 172.21.15.36:0/4225415863 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fece8067230 0x7fece81441c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:16.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.899+0000 7fecee0cf700 1 -- 172.21.15.36:0/4225415863 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fece8067230 msgr2=0x7fece81441c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:16.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.899+0000 7fecee0cf700 1 --2- 172.21.15.36:0/4225415863 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fece8067230 0x7fece81441c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:16.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.899+0000 7fecee0cf700 1 -- 172.21.15.36:0/4225415863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fece80b7d40 msgr2=0x7fece813d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:16.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.899+0000 7fecee0cf700 1 --2- 172.21.15.36:0/4225415863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fece80b7d40 0x7fece813d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:16.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.899+0000 7fecee0cf700 1 -- 172.21.15.36:0/4225415863 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecdc00b180 con 0x7fece8141050 2022-06-17T12:55:16.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.899+0000 7fecee0cf700 1 --2- 172.21.15.36:0/4225415863 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fece8141050 0x7fece813ff20 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7fece000c000 tx=0x7fece0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:16.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.899+0000 7fecdaffd700 1 -- 172.21.15.36:0/4225415863 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fece0011750 con 0x7fece8141050 2022-06-17T12:55:16.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.899+0000 7fecdaffd700 1 -- 172.21.15.36:0/4225415863 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece00118b0 con 0x7fece8141050 2022-06-17T12:55:16.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.899+0000 7fecefb32700 1 -- 172.21.15.36:0/4225415863 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece813d460 con 0x7fece8141050 2022-06-17T12:55:16.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.899+0000 7fecdaffd700 1 -- 172.21.15.36:0/4225415863 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fece0011c10 con 0x7fece8141050 2022-06-17T12:55:16.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.899+0000 7fecefb32700 1 -- 172.21.15.36:0/4225415863 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece813c470 con 0x7fece8141050 2022-06-17T12:55:16.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.903+0000 7fecdaffd700 1 -- 172.21.15.36:0/4225415863 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fece0011d70 con 0x7fece8141050 2022-06-17T12:55:16.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.903+0000 7fecefb32700 1 -- 172.21.15.36:0/4225415863 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece813d5f0 con 0x7fece8141050 2022-06-17T12:55:16.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.903+0000 7fecdaffd700 1 --2- 172.21.15.36:0/4225415863 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fecd00539d0 0x7fecd0055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:16.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.903+0000 7fecdaffd700 1 -- 172.21.15.36:0/4225415863 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fece006bfb0 con 0x7fece8141050 2022-06-17T12:55:16.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.903+0000 7feced8ce700 1 --2- 172.21.15.36:0/4225415863 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fecd00539d0 0x7fecd0055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:16.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.903+0000 7feced8ce700 1 --2- 172.21.15.36:0/4225415863 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fecd00539d0 0x7fecd0055e90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7fece400d000 tx=0x7fece4007c20).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:16.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:16.907+0000 7fecdaffd700 1 -- 172.21.15.36:0/4225415863 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fece0043090 con 0x7fece8141050 2022-06-17T12:55:17.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.035+0000 7fecefb32700 1 -- 172.21.15.36:0/4225415863 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fece8067900 con 0x7fece8141050 2022-06-17T12:55:17.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.047+0000 7fecdaffd700 1 -- 172.21.15.36:0/4225415863 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fece00466b0 con 0x7fece8141050 2022-06-17T12:55:17.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.047+0000 7fecefb32700 1 -- 172.21.15.36:0/4225415863 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fecd00539d0 msgr2=0x7fecd0055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:17.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.047+0000 7fecefb32700 1 --2- 172.21.15.36:0/4225415863 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fecd00539d0 0x7fecd0055e90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7fece400d000 tx=0x7fece4007c20).stop 2022-06-17T12:55:17.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.047+0000 7fecefb32700 1 -- 172.21.15.36:0/4225415863 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fece8141050 msgr2=0x7fece813ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:17.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.047+0000 7fecefb32700 1 --2- 172.21.15.36:0/4225415863 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fece8141050 0x7fece813ff20 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7fece000c000 tx=0x7fece0006df0).stop 2022-06-17T12:55:17.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.047+0000 7fecefb32700 1 -- 172.21.15.36:0/4225415863 shutdown_connections 2022-06-17T12:55:17.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.047+0000 7fecefb32700 1 --2- 172.21.15.36:0/4225415863 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fecd00539d0 0x7fecd0055e90 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:17.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.047+0000 7fecefb32700 1 --2- 172.21.15.36:0/4225415863 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fece8141050 0x7fece813ff20 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:17.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.047+0000 7fecefb32700 1 --2- 172.21.15.36:0/4225415863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fece80b7d40 0x7fece813d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:17.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.047+0000 7fecefb32700 1 --2- 172.21.15.36:0/4225415863 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fece8067230 0x7fece81441c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:17.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.047+0000 7fecefb32700 1 -- 172.21.15.36:0/4225415863 >> 172.21.15.36:0/4225415863 conn(0x7fece805c920 msgr2=0x7fece8144f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:17.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.047+0000 7fecefb32700 1 -- 172.21.15.36:0/4225415863 shutdown_connections 2022-06-17T12:55:17.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.047+0000 7fecefb32700 1 -- 172.21.15.36:0/4225415863 wait complete. 2022-06-17T12:55:17.050 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df' 2022-06-17T12:55:17.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.171+0000 7f8f9995e700 1 Processor -- start 2022-06-17T12:55:17.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.171+0000 7f8f9995e700 1 -- start start 2022-06-17T12:55:17.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.171+0000 7f8f9995e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f940b65f0 0x7f8f940b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:17.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.171+0000 7f8f9995e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8f940b15b0 0x7f8f940af630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:17.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.171+0000 7f8f9995e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f9413fdf0 con 0x7f8f940adf10 2022-06-17T12:55:17.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.171+0000 7f8f9995e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f940ae8e0 con 0x7f8f940b15b0 2022-06-17T12:55:17.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.171+0000 7f8f9995e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f94140090 con 0x7f8f940b65f0 2022-06-17T12:55:17.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.171+0000 7f8f93fff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f8f940adf10 0x7f8f94140530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55700/0 (socket says 172.21.15.36:55700) 2022-06-17T12:55:17.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.171+0000 7f8f93fff700 1 -- 172.21.15.36:0/3200306279 learned_addr learned my addr 172.21.15.36:0/3200306279 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:17.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.171+0000 7f8f9895c700 1 --2- 172.21.15.36:0/3200306279 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f940b65f0 0x7f8f940b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:17.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.171+0000 7f8f9915d700 1 --2- 172.21.15.36:0/3200306279 >> v2:172.21.15.70:3300/0 conn(0x7f8f940b15b0 0x7f8f940af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:17.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.175+0000 7f8f937fe700 1 -- 172.21.15.36:0/3200306279 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4127325178 0 0) 0x7f8f9413fdf0 con 0x7f8f940adf10 2022-06-17T12:55:17.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.175+0000 7f8f937fe700 1 -- 172.21.15.36:0/3200306279 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f74003260 con 0x7f8f940adf10 2022-06-17T12:55:17.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.175+0000 7f8f9895c700 1 -- 172.21.15.36:0/3200306279 >> v2:172.21.15.70:3300/0 conn(0x7f8f940b15b0 msgr2=0x7f8f940af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:17.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.175+0000 7f8f9895c700 1 --2- 172.21.15.36:0/3200306279 >> v2:172.21.15.70:3300/0 conn(0x7f8f940b15b0 0x7f8f940af630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:17.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.175+0000 7f8f9895c700 1 -- 172.21.15.36:0/3200306279 >> v1:172.21.15.36:6789/0 conn(0x7f8f940adf10 legacy=0x7f8f94140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:17.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.175+0000 7f8f9895c700 1 -- 172.21.15.36:0/3200306279 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f940b1be0 con 0x7f8f940b65f0 2022-06-17T12:55:17.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.175+0000 7f8f9915d700 1 --2- 172.21.15.36:0/3200306279 >> v2:172.21.15.70:3300/0 conn(0x7f8f940b15b0 0x7f8f940af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:17.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.175+0000 7f8f9895c700 1 --2- 172.21.15.36:0/3200306279 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f940b65f0 0x7f8f940b69d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f8f7c002f70 tx=0x7f8f7c006df0).ready entity=mon.2 client_cookie=fea267f03625f459 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:17.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.175+0000 7f8f937fe700 1 -- 172.21.15.36:0/3200306279 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f7c00d4a0 con 0x7f8f940b65f0 2022-06-17T12:55:17.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.175+0000 7f8f937fe700 1 -- 172.21.15.36:0/3200306279 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7c000e90 con 0x7f8f940b65f0 2022-06-17T12:55:17.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.175+0000 7f8f937fe700 1 -- 172.21.15.36:0/3200306279 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f7c00d7f0 con 0x7f8f940b65f0 2022-06-17T12:55:17.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.175+0000 7f8f9995e700 1 -- 172.21.15.36:0/3200306279 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f940b65f0 msgr2=0x7f8f940b69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:17.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.175+0000 7f8f9995e700 1 --2- 172.21.15.36:0/3200306279 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f940b65f0 0x7f8f940b69d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f8f7c002f70 tx=0x7f8f7c006df0).stop 2022-06-17T12:55:17.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.175+0000 7f8f9995e700 1 -- 172.21.15.36:0/3200306279 shutdown_connections 2022-06-17T12:55:17.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.175+0000 7f8f9995e700 1 --2- 172.21.15.36:0/3200306279 >> v2:172.21.15.70:3300/0 conn(0x7f8f940b15b0 0x7f8f940af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:17.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.175+0000 7f8f9995e700 1 --2- 172.21.15.36:0/3200306279 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f940b65f0 0x7f8f940b69d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:17.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.175+0000 7f8f9995e700 1 -- 172.21.15.36:0/3200306279 >> 172.21.15.36:0/3200306279 conn(0x7f8f940614b0 msgr2=0x7f8f94061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:17.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.175+0000 7f8f9995e700 1 -- 172.21.15.36:0/3200306279 shutdown_connections 2022-06-17T12:55:17.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.175+0000 7f8f9995e700 1 -- 172.21.15.36:0/3200306279 wait complete. 2022-06-17T12:55:17.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.179+0000 7f8f9995e700 1 Processor -- start 2022-06-17T12:55:17.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.179+0000 7f8f9995e700 1 -- start start 2022-06-17T12:55:17.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.179+0000 7f8f9995e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f940adf10 0x7f8f94063d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:17.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.179+0000 7f8f9995e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f940b15b0 0x7f8f9414db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:17.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.179+0000 7f8f9995e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f940b65f0 0x7f8f9414e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:17.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.179+0000 7f8f9995e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f94143d00 con 0x7f8f940b65f0 2022-06-17T12:55:17.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.179+0000 7f8f9995e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9414ea90 con 0x7f8f940b15b0 2022-06-17T12:55:17.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.179+0000 7f8f9995e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9414e630 con 0x7f8f940adf10 2022-06-17T12:55:17.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.179+0000 7f8f9915d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f940b65f0 0x7f8f9414e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:17.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.179+0000 7f8f9915d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f940b65f0 0x7f8f9414e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38180/0 (socket says 172.21.15.36:38180) 2022-06-17T12:55:17.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.179+0000 7f8f9915d700 1 -- 172.21.15.36:0/1096354135 learned_addr learned my addr 172.21.15.36:0/1096354135 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:17.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.179+0000 7f8f9895c700 1 --2- 172.21.15.36:0/1096354135 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f940adf10 0x7f8f94063d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:17.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.179+0000 7f8f93fff700 1 --2- 172.21.15.36:0/1096354135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f940b15b0 0x7f8f9414db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:17.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.179+0000 7f8f9915d700 1 -- 172.21.15.36:0/1096354135 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f940adf10 msgr2=0x7f8f94063d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:17.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.179+0000 7f8f9915d700 1 --2- 172.21.15.36:0/1096354135 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f940adf10 0x7f8f94063d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:17.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.179+0000 7f8f9915d700 1 -- 172.21.15.36:0/1096354135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f940b15b0 msgr2=0x7f8f9414db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:17.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.179+0000 7f8f9915d700 1 --2- 172.21.15.36:0/1096354135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f940b15b0 0x7f8f9414db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:17.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.179+0000 7f8f9915d700 1 -- 172.21.15.36:0/1096354135 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8c009470 con 0x7f8f940b65f0 2022-06-17T12:55:17.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.179+0000 7f8f9915d700 1 --2- 172.21.15.36:0/1096354135 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f940b65f0 0x7f8f9414e040 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7f8f8c00c000 tx=0x7f8f8c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:17.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.179+0000 7f8f91ffb700 1 -- 172.21.15.36:0/1096354135 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f8c00c850 con 0x7f8f940b65f0 2022-06-17T12:55:17.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.179+0000 7f8f91ffb700 1 -- 172.21.15.36:0/1096354135 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8c00c9b0 con 0x7f8f940b65f0 2022-06-17T12:55:17.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.179+0000 7f8f91ffb700 1 -- 172.21.15.36:0/1096354135 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f8c00cd10 con 0x7f8f940b65f0 2022-06-17T12:55:17.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.179+0000 7f8f9995e700 1 -- 172.21.15.36:0/1096354135 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f7c00b180 con 0x7f8f940b65f0 2022-06-17T12:55:17.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.179+0000 7f8f9995e700 1 -- 172.21.15.36:0/1096354135 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f94064280 con 0x7f8f940b65f0 2022-06-17T12:55:17.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.183+0000 7f8f9995e700 1 -- 172.21.15.36:0/1096354135 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f9414e7f0 con 0x7f8f940b65f0 2022-06-17T12:55:17.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.183+0000 7f8f91ffb700 1 -- 172.21.15.36:0/1096354135 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8f8c022070 con 0x7f8f940b65f0 2022-06-17T12:55:17.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.183+0000 7f8f91ffb700 1 --2- 172.21.15.36:0/1096354135 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8f78053a40 0x7f8f78055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:17.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.183+0000 7f8f91ffb700 1 -- 172.21.15.36:0/1096354135 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f8f8c06c510 con 0x7f8f940b65f0 2022-06-17T12:55:17.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.183+0000 7f8f9895c700 1 --2- 172.21.15.36:0/1096354135 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8f78053a40 0x7f8f78055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:17.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.183+0000 7f8f9895c700 1 --2- 172.21.15.36:0/1096354135 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8f78053a40 0x7f8f78055f00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f8f7c01a000 tx=0x7f8f7c014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:17.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.187+0000 7f8f91ffb700 1 -- 172.21.15.36:0/1096354135 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8f9414e7f0 con 0x7f8f940b65f0 2022-06-17T12:55:17.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.359+0000 7f8f9995e700 1 -- 172.21.15.36:0/1096354135 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]} v 0) v1 -- 0x7f8f9414e7f0 con 0x7f8f940b65f0 2022-06-17T12:55:17.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.759+0000 7f8f91ffb700 1 -- 172.21.15.36:0/1096354135 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]}]=0 v391) v1 ==== 127+0+0 (secure 0 0 0) 0x7f8f9414e7f0 con 0x7f8f940b65f0 2022-06-17T12:55:17.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.763+0000 7f8f9995e700 1 -- 172.21.15.36:0/1096354135 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8f78053a40 msgr2=0x7f8f78055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:17.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.763+0000 7f8f9995e700 1 --2- 172.21.15.36:0/1096354135 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8f78053a40 0x7f8f78055f00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f8f7c01a000 tx=0x7f8f7c014000).stop 2022-06-17T12:55:17.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.763+0000 7f8f9995e700 1 -- 172.21.15.36:0/1096354135 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f940b65f0 msgr2=0x7f8f9414e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:17.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.763+0000 7f8f9995e700 1 --2- 172.21.15.36:0/1096354135 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f940b65f0 0x7f8f9414e040 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7f8f8c00c000 tx=0x7f8f8c0091d0).stop 2022-06-17T12:55:17.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.763+0000 7f8f9995e700 1 -- 172.21.15.36:0/1096354135 shutdown_connections 2022-06-17T12:55:17.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.763+0000 7f8f9995e700 1 --2- 172.21.15.36:0/1096354135 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8f78053a40 0x7f8f78055f00 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:17.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.763+0000 7f8f9995e700 1 --2- 172.21.15.36:0/1096354135 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8f940b65f0 0x7f8f9414e040 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:17.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.763+0000 7f8f9995e700 1 --2- 172.21.15.36:0/1096354135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f940b15b0 0x7f8f9414db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:17.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.763+0000 7f8f9995e700 1 --2- 172.21.15.36:0/1096354135 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f940adf10 0x7f8f94063d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:17.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.763+0000 7f8f9995e700 1 -- 172.21.15.36:0/1096354135 >> 172.21.15.36:0/1096354135 conn(0x7f8f940614b0 msgr2=0x7f8f940b51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:17.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.767+0000 7f8f9995e700 1 -- 172.21.15.36:0/1096354135 shutdown_connections 2022-06-17T12:55:17.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.767+0000 7f8f9995e700 1 -- 172.21.15.36:0/1096354135 wait complete. 2022-06-17T12:55:17.771 INFO:tasks.cephfs_test_runner:test_filtered_df (tasks.cephfs.test_misc.TestMisc) ... ok 2022-06-17T12:55:17.771 INFO:tasks.cephfs_test_runner:Starting test: test_fs_new (tasks.cephfs.test_misc.TestMisc) 2022-06-17T12:55:17.773 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new' 2022-06-17T12:55:17.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f628be15700 1 Processor -- start 2022-06-17T12:55:17.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f628be15700 1 -- start start 2022-06-17T12:55:17.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f628be15700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6284062b50 0x7f62840ada40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:17.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f628be15700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62840b0ba0 con 0x7f6284062770 2022-06-17T12:55:17.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f628be15700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6284147e90 con 0x7f6284144c00 2022-06-17T12:55:17.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f628be15700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62840ae820 con 0x7f6284062b50 2022-06-17T12:55:17.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f6289bb1700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f6284062770 0x7f62841444f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55718/0 (socket says 172.21.15.36:55718) 2022-06-17T12:55:17.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f6289bb1700 1 -- 172.21.15.36:0/1021344480 learned_addr learned my addr 172.21.15.36:0/1021344480 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:17.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f628a3b2700 1 --2- 172.21.15.36:0/1021344480 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6284062b50 0x7f62840ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:17.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f6288baf700 1 -- 172.21.15.36:0/1021344480 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2418463653 0 0) 0x7f62840b0ba0 con 0x7f6284062770 2022-06-17T12:55:17.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f6288baf700 1 -- 172.21.15.36:0/1021344480 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f626c003260 con 0x7f6284062770 2022-06-17T12:55:17.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f6288baf700 1 -- 172.21.15.36:0/1021344480 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888212154 0 0) 0x7f6284147e90 con 0x7f6284144c00 2022-06-17T12:55:17.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f628a3b2700 1 -- 172.21.15.36:0/1021344480 >> v1:172.21.15.70:6789/0 conn(0x7f6284144c00 legacy=0x7f62840ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:17.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f628a3b2700 1 -- 172.21.15.36:0/1021344480 >> v1:172.21.15.36:6789/0 conn(0x7f6284062770 legacy=0x7f62841444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:17.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f628a3b2700 1 -- 172.21.15.36:0/1021344480 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62840b0dd0 con 0x7f6284062b50 2022-06-17T12:55:17.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f628a3b2700 1 --2- 172.21.15.36:0/1021344480 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6284062b50 0x7f62840ada40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f627c0032f0 tx=0x7f627c0091d0).ready entity=mon.2 client_cookie=7c186e78df99a183 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:17.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f6288baf700 1 -- 172.21.15.36:0/1021344480 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f627c00b6c0 con 0x7f6284062b50 2022-06-17T12:55:17.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f6288baf700 1 -- 172.21.15.36:0/1021344480 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627c00b820 con 0x7f6284062b50 2022-06-17T12:55:17.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f6288baf700 1 -- 172.21.15.36:0/1021344480 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f627c00b990 con 0x7f6284062b50 2022-06-17T12:55:17.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f628be15700 1 -- 172.21.15.36:0/1021344480 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6284062b50 msgr2=0x7f62840ada40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:17.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f628be15700 1 --2- 172.21.15.36:0/1021344480 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6284062b50 0x7f62840ada40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f627c0032f0 tx=0x7f627c0091d0).stop 2022-06-17T12:55:17.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f628be15700 1 -- 172.21.15.36:0/1021344480 shutdown_connections 2022-06-17T12:55:17.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f628be15700 1 --2- 172.21.15.36:0/1021344480 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6284062b50 0x7f62840ada40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:17.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f628be15700 1 -- 172.21.15.36:0/1021344480 >> 172.21.15.36:0/1021344480 conn(0x7f62840b53e0 msgr2=0x7f62840b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:17.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f628be15700 1 -- 172.21.15.36:0/1021344480 shutdown_connections 2022-06-17T12:55:17.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.899+0000 7f628be15700 1 -- 172.21.15.36:0/1021344480 wait complete. 2022-06-17T12:55:17.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.903+0000 7f628be15700 1 Processor -- start 2022-06-17T12:55:17.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.903+0000 7f628be15700 1 -- start start 2022-06-17T12:55:17.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.903+0000 7f628be15700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6284062770 0x7f62840b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:17.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.903+0000 7f628be15700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6284062b50 0x7f62840655e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:17.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.903+0000 7f628be15700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6284144c00 0x7f6284063170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:17.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.903+0000 7f628be15700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6284065ad0 con 0x7f6284144c00 2022-06-17T12:55:17.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.903+0000 7f628be15700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6284063dc0 con 0x7f6284062b50 2022-06-17T12:55:17.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.903+0000 7f628be15700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628414aad0 con 0x7f6284062770 2022-06-17T12:55:17.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.903+0000 7f628a3b2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6284144c00 0x7f6284063170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:17.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.903+0000 7f6289bb1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6284062770 0x7f62840b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:17.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.903+0000 7f628a3b2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6284144c00 0x7f6284063170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38194/0 (socket says 172.21.15.36:38194) 2022-06-17T12:55:17.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.903+0000 7f628a3b2700 1 -- 172.21.15.36:0/2420028000 learned_addr learned my addr 172.21.15.36:0/2420028000 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:17.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.903+0000 7f62893b0700 1 --2- 172.21.15.36:0/2420028000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6284062b50 0x7f62840655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:17.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.903+0000 7f628a3b2700 1 -- 172.21.15.36:0/2420028000 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6284062770 msgr2=0x7f62840b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:17.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.903+0000 7f628a3b2700 1 --2- 172.21.15.36:0/2420028000 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6284062770 0x7f62840b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:17.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.903+0000 7f628a3b2700 1 -- 172.21.15.36:0/2420028000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6284062b50 msgr2=0x7f62840655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:17.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.903+0000 7f628a3b2700 1 --2- 172.21.15.36:0/2420028000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6284062b50 0x7f62840655e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:17.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.903+0000 7f628a3b2700 1 -- 172.21.15.36:0/2420028000 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627c009470 con 0x7f6284144c00 2022-06-17T12:55:17.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.903+0000 7f628a3b2700 1 --2- 172.21.15.36:0/2420028000 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6284144c00 0x7f6284063170 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 rx=0x7f627c0032f0 tx=0x7f627c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:17.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.903+0000 7f6272ffd700 1 -- 172.21.15.36:0/2420028000 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f627c00b990 con 0x7f6284144c00 2022-06-17T12:55:17.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.903+0000 7f6272ffd700 1 -- 172.21.15.36:0/2420028000 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627c021630 con 0x7f6284144c00 2022-06-17T12:55:17.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.903+0000 7f628be15700 1 -- 172.21.15.36:0/2420028000 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628414a650 con 0x7f6284144c00 2022-06-17T12:55:17.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.903+0000 7f6272ffd700 1 -- 172.21.15.36:0/2420028000 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f627c00b6c0 con 0x7f6284144c00 2022-06-17T12:55:17.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.907+0000 7f628be15700 1 -- 172.21.15.36:0/2420028000 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628414b920 con 0x7f6284144c00 2022-06-17T12:55:17.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.907+0000 7f628be15700 1 -- 172.21.15.36:0/2420028000 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f628414acd0 con 0x7f6284144c00 2022-06-17T12:55:17.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.907+0000 7f6272ffd700 1 -- 172.21.15.36:0/2420028000 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f627c00f070 con 0x7f6284144c00 2022-06-17T12:55:17.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.907+0000 7f6272ffd700 1 --2- 172.21.15.36:0/2420028000 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6264053a40 0x7f6264055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:17.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.907+0000 7f6272ffd700 1 -- 172.21.15.36:0/2420028000 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f627c06cf20 con 0x7f6284144c00 2022-06-17T12:55:17.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.907+0000 7f6289bb1700 1 --2- 172.21.15.36:0/2420028000 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6264053a40 0x7f6264055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:17.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.907+0000 7f6289bb1700 1 --2- 172.21.15.36:0/2420028000 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6264053a40 0x7f6264055f00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f628000d000 tx=0x7f6280009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:17.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:17.915+0000 7f6272ffd700 1 -- 172.21.15.36:0/2420028000 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f627c044000 con 0x7f6284144c00 2022-06-17T12:55:18.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.087+0000 7f628be15700 1 -- 172.21.15.36:0/2420028000 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new"]} v 0) v1 -- 0x7f628414b920 con 0x7f6284144c00 2022-06-17T12:55:18.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.763+0000 7f6272ffd700 1 -- 172.21.15.36:0/2420028000 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new"]}]=0 v392) v1 ==== 125+0+0 (secure 0 0 0) 0x7f628414b920 con 0x7f6284144c00 2022-06-17T12:55:18.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.767+0000 7f628be15700 1 -- 172.21.15.36:0/2420028000 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6264053a40 msgr2=0x7f6264055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:18.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.767+0000 7f628be15700 1 --2- 172.21.15.36:0/2420028000 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6264053a40 0x7f6264055f00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f628000d000 tx=0x7f6280009040).stop 2022-06-17T12:55:18.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.767+0000 7f628be15700 1 -- 172.21.15.36:0/2420028000 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6284144c00 msgr2=0x7f6284063170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:18.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.767+0000 7f628be15700 1 --2- 172.21.15.36:0/2420028000 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6284144c00 0x7f6284063170 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 rx=0x7f627c0032f0 tx=0x7f627c016000).stop 2022-06-17T12:55:18.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.767+0000 7f628be15700 1 -- 172.21.15.36:0/2420028000 shutdown_connections 2022-06-17T12:55:18.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.767+0000 7f628be15700 1 --2- 172.21.15.36:0/2420028000 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6264053a40 0x7f6264055f00 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:18.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.767+0000 7f628be15700 1 --2- 172.21.15.36:0/2420028000 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6284144c00 0x7f6284063170 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:18.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.767+0000 7f628be15700 1 --2- 172.21.15.36:0/2420028000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6284062b50 0x7f62840655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:18.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.767+0000 7f628be15700 1 --2- 172.21.15.36:0/2420028000 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6284062770 0x7f62840b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:18.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.767+0000 7f628be15700 1 -- 172.21.15.36:0/2420028000 >> 172.21.15.36:0/2420028000 conn(0x7f62840b53e0 msgr2=0x7f628405f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:18.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.767+0000 7f628be15700 1 -- 172.21.15.36:0/2420028000 shutdown_connections 2022-06-17T12:55:18.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.767+0000 7f628be15700 1 -- 172.21.15.36:0/2420028000 wait complete. 2022-06-17T12:55:18.773 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-06-17T12:55:18.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.903+0000 7f93fb01d700 1 Processor -- start 2022-06-17T12:55:18.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.903+0000 7f93fb01d700 1 -- start start 2022-06-17T12:55:18.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93fb01d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f93f4067610 0x7f93f413efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:18.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93fb01d700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93f40ad020 con 0x7f93f40b1e00 2022-06-17T12:55:18.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93fb01d700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93f40ad2c0 con 0x7f93f4060920 2022-06-17T12:55:18.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93fb01d700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93f40aff80 con 0x7f93f4067610 2022-06-17T12:55:18.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f981a700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f93f40b1e00 0x7f93f413e8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55730/0 (socket says 172.21.15.36:55730) 2022-06-17T12:55:18.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f981a700 1 -- 172.21.15.36:0/2122010233 learned_addr learned my addr 172.21.15.36:0/2122010233 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:18.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93fa81c700 1 --2- 172.21.15.36:0/2122010233 >> v2:172.21.15.70:3300/0 conn(0x7f93f4067610 0x7f93f413efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:18.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f9019700 1 -- 172.21.15.36:0/2122010233 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473348583 0 0) 0x7f93f40ad020 con 0x7f93f40b1e00 2022-06-17T12:55:18.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f9019700 1 -- 172.21.15.36:0/2122010233 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93dc004370 con 0x7f93f40b1e00 2022-06-17T12:55:18.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f9019700 1 -- 172.21.15.36:0/2122010233 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983034805 0 0) 0x7f93f40ad2c0 con 0x7f93f4060920 2022-06-17T12:55:18.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f9019700 1 -- 172.21.15.36:0/2122010233 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93f40ad020 con 0x7f93f4060920 2022-06-17T12:55:18.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f9019700 1 -- 172.21.15.36:0/2122010233 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 170905652 0 0) 0x7f93dc004370 con 0x7f93f40b1e00 2022-06-17T12:55:18.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f9019700 1 -- 172.21.15.36:0/2122010233 >> v2:172.21.15.70:3300/0 conn(0x7f93f4067610 msgr2=0x7f93f413efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:18.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f9019700 1 --2- 172.21.15.36:0/2122010233 >> v2:172.21.15.70:3300/0 conn(0x7f93f4067610 0x7f93f413efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:18.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f9019700 1 -- 172.21.15.36:0/2122010233 >> v1:172.21.15.70:6789/0 conn(0x7f93f4060920 legacy=0x7f93f40ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:18.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f9019700 1 -- 172.21.15.36:0/2122010233 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f4067a50 con 0x7f93f40b1e00 2022-06-17T12:55:18.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f9019700 1 -- 172.21.15.36:0/2122010233 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f93e4001660 con 0x7f93f40b1e00 2022-06-17T12:55:18.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f9019700 1 -- 172.21.15.36:0/2122010233 >> v1:172.21.15.36:6789/0 conn(0x7f93f40b1e00 legacy=0x7f93f413e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:18.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f9019700 1 --2- 172.21.15.36:0/2122010233 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93dc005aa0 0x7f93dc005ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:18.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f9019700 1 --2- 172.21.15.36:0/2122010233 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93dc0045b0 0x7f93dc004a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:18.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f9019700 1 --2- 172.21.15.36:0/2122010233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93dc004f20 0x7f93dc00c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:18.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f9019700 1 -- 172.21.15.36:0/2122010233 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f40aff80 con 0x7f93dc0045b0 2022-06-17T12:55:18.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f9019700 1 -- 172.21.15.36:0/2122010233 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93dc00ce30 con 0x7f93dc004f20 2022-06-17T12:55:18.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f9019700 1 -- 172.21.15.36:0/2122010233 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93dc00cff0 con 0x7f93dc005aa0 2022-06-17T12:55:18.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f981a700 1 --2- 172.21.15.36:0/2122010233 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93dc0045b0 0x7f93dc004a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:18.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93fa01b700 1 --2- 172.21.15.36:0/2122010233 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93dc005aa0 0x7f93dc005ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:18.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93fa81c700 1 --2- 172.21.15.36:0/2122010233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93dc004f20 0x7f93dc00c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:18.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f981a700 1 -- 172.21.15.36:0/2122010233 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93dc005aa0 msgr2=0x7f93dc005ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:18.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f981a700 1 --2- 172.21.15.36:0/2122010233 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93dc005aa0 0x7f93dc005ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:18.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f981a700 1 -- 172.21.15.36:0/2122010233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93dc004f20 msgr2=0x7f93dc00c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:18.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f981a700 1 --2- 172.21.15.36:0/2122010233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93dc004f20 0x7f93dc00c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:18.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f981a700 1 -- 172.21.15.36:0/2122010233 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93dc004370 con 0x7f93dc0045b0 2022-06-17T12:55:18.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93fa01b700 1 --2- 172.21.15.36:0/2122010233 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93dc005aa0 0x7f93dc005ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:18.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93fa81c700 1 --2- 172.21.15.36:0/2122010233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93dc004f20 0x7f93dc00c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:18.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93fa81c700 1 -- 172.21.15.36:0/2122010233 reap_dead start 2022-06-17T12:55:18.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f981a700 1 --2- 172.21.15.36:0/2122010233 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93dc0045b0 0x7f93dc004a30 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7f93e4001430 tx=0x7f93e4001dd0).ready entity=mon.0 client_cookie=ad0ef3c8ad169bc server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:18.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f9019700 1 -- 172.21.15.36:0/2122010233 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93e40070d0 con 0x7f93dc0045b0 2022-06-17T12:55:18.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f9019700 1 -- 172.21.15.36:0/2122010233 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e4007230 con 0x7f93dc0045b0 2022-06-17T12:55:18.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.907+0000 7f93f9019700 1 -- 172.21.15.36:0/2122010233 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93e400c8f0 con 0x7f93dc0045b0 2022-06-17T12:55:18.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.911+0000 7f93fb01d700 1 -- 172.21.15.36:0/2122010233 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93dc0045b0 msgr2=0x7f93dc004a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:18.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.911+0000 7f93fb01d700 1 --2- 172.21.15.36:0/2122010233 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93dc0045b0 0x7f93dc004a30 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7f93e4001430 tx=0x7f93e4001dd0).stop 2022-06-17T12:55:18.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.911+0000 7f93fb01d700 1 -- 172.21.15.36:0/2122010233 shutdown_connections 2022-06-17T12:55:18.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.911+0000 7f93fb01d700 1 --2- 172.21.15.36:0/2122010233 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93dc0045b0 0x7f93dc004a30 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:18.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.911+0000 7f93fb01d700 1 -- 172.21.15.36:0/2122010233 >> 172.21.15.36:0/2122010233 conn(0x7f93f40b64f0 msgr2=0x7f93f40b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:18.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.911+0000 7f93fb01d700 1 -- 172.21.15.36:0/2122010233 shutdown_connections 2022-06-17T12:55:18.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.911+0000 7f93fb01d700 1 -- 172.21.15.36:0/2122010233 wait complete. 2022-06-17T12:55:18.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.911+0000 7f93fb01d700 1 Processor -- start 2022-06-17T12:55:18.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.911+0000 7f93fb01d700 1 -- start start 2022-06-17T12:55:18.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93fb01d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93dc0045b0 0x7f93f4152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:18.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93fb01d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93f40a7ef0 0x7f93f409fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:18.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93fb01d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93f40a0260 0x7f93f413e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:18.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93fb01d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f40a9440 con 0x7f93dc0045b0 2022-06-17T12:55:18.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93fb01d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f40a6430 con 0x7f93f40a0260 2022-06-17T12:55:18.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93fb01d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f40a66d0 con 0x7f93f40a7ef0 2022-06-17T12:55:18.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93fa01b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93dc0045b0 0x7f93f4152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:18.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93fa01b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93dc0045b0 0x7f93f4152030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38210/0 (socket says 172.21.15.36:38210) 2022-06-17T12:55:18.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93fa01b700 1 -- 172.21.15.36:0/1112150161 learned_addr learned my addr 172.21.15.36:0/1112150161 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:18.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93f981a700 1 --2- 172.21.15.36:0/1112150161 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93f40a7ef0 0x7f93f409fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:18.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93fa81c700 1 --2- 172.21.15.36:0/1112150161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93f40a0260 0x7f93f413e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:18.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93fa01b700 1 -- 172.21.15.36:0/1112150161 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93f40a7ef0 msgr2=0x7f93f409fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:18.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93fa01b700 1 --2- 172.21.15.36:0/1112150161 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93f40a7ef0 0x7f93f409fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:18.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93fa01b700 1 -- 172.21.15.36:0/1112150161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93f40a0260 msgr2=0x7f93f413e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:18.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93fa01b700 1 --2- 172.21.15.36:0/1112150161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93f40a0260 0x7f93f413e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:18.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93fa01b700 1 -- 172.21.15.36:0/1112150161 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f40a6970 con 0x7f93dc0045b0 2022-06-17T12:55:18.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93fa81c700 1 --2- 172.21.15.36:0/1112150161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93f40a0260 0x7f93f413e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:18.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93fa01b700 1 --2- 172.21.15.36:0/1112150161 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93dc0045b0 0x7f93f4152030 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 rx=0x7f93f000d000 tx=0x7f93f0007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:18.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93e37fe700 1 -- 172.21.15.36:0/1112150161 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93f00116b0 con 0x7f93dc0045b0 2022-06-17T12:55:18.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93e37fe700 1 -- 172.21.15.36:0/1112150161 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f0011810 con 0x7f93dc0045b0 2022-06-17T12:55:18.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93fb01d700 1 -- 172.21.15.36:0/1112150161 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f4153190 con 0x7f93dc0045b0 2022-06-17T12:55:18.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93e37fe700 1 -- 172.21.15.36:0/1112150161 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93f0011b70 con 0x7f93dc0045b0 2022-06-17T12:55:18.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93fb01d700 1 -- 172.21.15.36:0/1112150161 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f40a78b0 con 0x7f93dc0045b0 2022-06-17T12:55:18.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93fb01d700 1 -- 172.21.15.36:0/1112150161 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f4060d00 con 0x7f93dc0045b0 2022-06-17T12:55:18.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.915+0000 7f93e37fe700 1 -- 172.21.15.36:0/1112150161 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f93f001b030 con 0x7f93dc0045b0 2022-06-17T12:55:18.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.919+0000 7f93e37fe700 1 --2- 172.21.15.36:0/1112150161 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f93d4053a40 0x7f93d4055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:18.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.919+0000 7f93e37fe700 1 -- 172.21.15.36:0/1112150161 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f93f006baf0 con 0x7f93dc0045b0 2022-06-17T12:55:18.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.919+0000 7f93f981a700 1 --2- 172.21.15.36:0/1112150161 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f93d4053a40 0x7f93d4055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:18.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.919+0000 7f93f981a700 1 --2- 172.21.15.36:0/1112150161 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f93d4053a40 0x7f93d4055f00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f93e4019000 tx=0x7f93e4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:18.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:18.923+0000 7f93e37fe700 1 -- 172.21.15.36:0/1112150161 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f93f0042bd0 con 0x7f93dc0045b0 2022-06-17T12:55:19.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.055+0000 7f93fb01d700 1 -- 172.21.15.36:0/1112150161 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f93f4057410 con 0x7f93dc0045b0 2022-06-17T12:55:19.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.059+0000 7f93e37fe700 1 -- 172.21.15.36:0/1112150161 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f93f00461f0 con 0x7f93dc0045b0 2022-06-17T12:55:19.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.059+0000 7f93fb01d700 1 -- 172.21.15.36:0/1112150161 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f93d4053a40 msgr2=0x7f93d4055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:19.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.059+0000 7f93fb01d700 1 --2- 172.21.15.36:0/1112150161 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f93d4053a40 0x7f93d4055f00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f93e4019000 tx=0x7f93e4015000).stop 2022-06-17T12:55:19.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.059+0000 7f93fb01d700 1 -- 172.21.15.36:0/1112150161 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93dc0045b0 msgr2=0x7f93f4152030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:19.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.059+0000 7f93fb01d700 1 --2- 172.21.15.36:0/1112150161 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93dc0045b0 0x7f93f4152030 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 rx=0x7f93f000d000 tx=0x7f93f0007a00).stop 2022-06-17T12:55:19.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.059+0000 7f93fb01d700 1 -- 172.21.15.36:0/1112150161 shutdown_connections 2022-06-17T12:55:19.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.059+0000 7f93fb01d700 1 --2- 172.21.15.36:0/1112150161 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f93d4053a40 0x7f93d4055f00 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:19.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.059+0000 7f93fb01d700 1 --2- 172.21.15.36:0/1112150161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93f40a0260 0x7f93f413e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:19.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.059+0000 7f93fb01d700 1 --2- 172.21.15.36:0/1112150161 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93f40a7ef0 0x7f93f409fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:19.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.059+0000 7f93fb01d700 1 --2- 172.21.15.36:0/1112150161 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93dc0045b0 0x7f93f4152030 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:19.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.059+0000 7f93fb01d700 1 -- 172.21.15.36:0/1112150161 >> 172.21.15.36:0/1112150161 conn(0x7f93f40b64f0 msgr2=0x7f93f40b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:19.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.063+0000 7f93fb01d700 1 -- 172.21.15.36:0/1112150161 shutdown_connections 2022-06-17T12:55:19.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.063+0000 7f93fb01d700 1 -- 172.21.15.36:0/1112150161 wait complete. 2022-06-17T12:55:19.065 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-17T12:55:19.066 DEBUG:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:55:19.089 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:55:19.111 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:55:19.133 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:55:19.154 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:55:19.176 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-17T12:55:19.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:19.178 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:55:19.179 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:55:19.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:55:19.199 INFO:teuthology.orchestra.run.smithi107.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-17T12:55:19.200 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-06-17T12:55:19.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:19.201 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:55:19.202 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:55:19.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:55:19.212 INFO:teuthology.orchestra.run.smithi136.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-17T12:55:19.213 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-17T12:55:19.213 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:19.214 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:55:19.214 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:55:19.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:55:19.224 INFO:teuthology.orchestra.run.smithi143.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:55:19.225 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-06-17T12:55:19.226 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-17T12:55:19.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0efaa7700 1 Processor -- start 2022-06-17T12:55:19.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0efaa7700 1 -- start start 2022-06-17T12:55:19.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0efaa7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb0e805c870 0x7fb0e8143910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:19.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0efaa7700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e8067660 con 0x7fb0e80b6780 2022-06-17T12:55:19.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0efaa7700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e80b6b60 con 0x7fb0e8143e00 2022-06-17T12:55:19.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0efaa7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e8067890 con 0x7fb0e805c870 2022-06-17T12:55:19.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0ed843700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fb0e80b6780 0x7fb0e8143200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55748/0 (socket says 172.21.15.36:55748) 2022-06-17T12:55:19.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0ed843700 1 -- 172.21.15.36:0/287015310 learned_addr learned my addr 172.21.15.36:0/287015310 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:19.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0ed042700 1 --2- 172.21.15.36:0/287015310 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb0e805c870 0x7fb0e8143910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:19.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0ec841700 1 -- 172.21.15.36:0/287015310 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 341297146 0 0) 0x7fb0e8067660 con 0x7fb0e80b6780 2022-06-17T12:55:19.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0ec841700 1 -- 172.21.15.36:0/287015310 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0d0003260 con 0x7fb0e80b6780 2022-06-17T12:55:19.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0ec841700 1 -- 172.21.15.36:0/287015310 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3611403336 0 0) 0x7fb0d0003260 con 0x7fb0e80b6780 2022-06-17T12:55:19.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0ec841700 1 -- 172.21.15.36:0/287015310 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb0e805c870 msgr2=0x7fb0e8143910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:19.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0ec841700 1 --2- 172.21.15.36:0/287015310 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb0e805c870 0x7fb0e8143910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:19.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0ec841700 1 -- 172.21.15.36:0/287015310 >> v1:172.21.15.70:6789/0 conn(0x7fb0e8143e00 legacy=0x7fb0e813fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:19.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0ec841700 1 -- 172.21.15.36:0/287015310 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e8067b30 con 0x7fb0e80b6780 2022-06-17T12:55:19.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0ec841700 1 -- 172.21.15.36:0/287015310 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7fb0d8001660 con 0x7fb0e80b6780 2022-06-17T12:55:19.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0ec841700 1 -- 172.21.15.36:0/287015310 >> v1:172.21.15.36:6789/0 conn(0x7fb0e80b6780 legacy=0x7fb0e8143200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:19.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0ec841700 1 --2- 172.21.15.36:0/287015310 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb0e805c870 0x7fb0d0003780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:19.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0ec841700 1 --2- 172.21.15.36:0/287015310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0d0003c70 0x7fb0d0004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:19.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0ec841700 1 --2- 172.21.15.36:0/287015310 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb0d00049f0 0x7fb0d0004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:19.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0ec841700 1 -- 172.21.15.36:0/287015310 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e8067890 con 0x7fb0d00049f0 2022-06-17T12:55:19.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0ec841700 1 -- 172.21.15.36:0/287015310 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d000d960 con 0x7fb0d0003c70 2022-06-17T12:55:19.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0ec841700 1 -- 172.21.15.36:0/287015310 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d000db20 con 0x7fb0e805c870 2022-06-17T12:55:19.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0ed843700 1 --2- 172.21.15.36:0/287015310 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb0e805c870 0x7fb0d0003780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:19.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0ee044700 1 --2- 172.21.15.36:0/287015310 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb0d00049f0 0x7fb0d0004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:19.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.351+0000 7fb0ed042700 1 --2- 172.21.15.36:0/287015310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0d0003c70 0x7fb0d0004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:19.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.355+0000 7fb0ed843700 1 -- 172.21.15.36:0/287015310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0d0003c70 msgr2=0x7fb0d0004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:19.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.355+0000 7fb0ed843700 1 --2- 172.21.15.36:0/287015310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0d0003c70 0x7fb0d0004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:19.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.355+0000 7fb0ed843700 1 -- 172.21.15.36:0/287015310 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb0d00049f0 msgr2=0x7fb0d0004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:19.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.355+0000 7fb0ed843700 1 --2- 172.21.15.36:0/287015310 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb0d00049f0 0x7fb0d0004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:19.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.355+0000 7fb0ed843700 1 -- 172.21.15.36:0/287015310 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d0003260 con 0x7fb0e805c870 2022-06-17T12:55:19.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.355+0000 7fb0ee044700 1 --2- 172.21.15.36:0/287015310 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb0d00049f0 0x7fb0d0004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:19.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.355+0000 7fb0ed843700 1 --2- 172.21.15.36:0/287015310 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb0e805c870 0x7fb0d0003780 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fb0d8001370 tx=0x7fb0e8067cc0).ready entity=mon.2 client_cookie=53c62b869f8fc4e8 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:19.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.355+0000 7fb0ec841700 1 -- 172.21.15.36:0/287015310 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0d8007270 con 0x7fb0e805c870 2022-06-17T12:55:19.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.355+0000 7fb0ec841700 1 -- 172.21.15.36:0/287015310 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d80073d0 con 0x7fb0e805c870 2022-06-17T12:55:19.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.355+0000 7fb0ec841700 1 -- 172.21.15.36:0/287015310 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0d800cae0 con 0x7fb0e805c870 2022-06-17T12:55:19.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.355+0000 7fb0efaa7700 1 -- 172.21.15.36:0/287015310 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb0e805c870 msgr2=0x7fb0d0003780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:19.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.355+0000 7fb0efaa7700 1 --2- 172.21.15.36:0/287015310 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb0e805c870 0x7fb0d0003780 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fb0d8001370 tx=0x7fb0e8067cc0).stop 2022-06-17T12:55:19.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.355+0000 7fb0ee044700 1 -- 172.21.15.36:0/287015310 reap_dead start 2022-06-17T12:55:19.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.355+0000 7fb0efaa7700 1 -- 172.21.15.36:0/287015310 shutdown_connections 2022-06-17T12:55:19.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.355+0000 7fb0efaa7700 1 -- 172.21.15.36:0/287015310 >> 172.21.15.36:0/287015310 conn(0x7fb0e8063890 msgr2=0x7fb0e80b5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:19.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.355+0000 7fb0efaa7700 1 -- 172.21.15.36:0/287015310 shutdown_connections 2022-06-17T12:55:19.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.355+0000 7fb0efaa7700 1 -- 172.21.15.36:0/287015310 wait complete. 2022-06-17T12:55:19.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.355+0000 7fb0efaa7700 1 Processor -- start 2022-06-17T12:55:19.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.355+0000 7fb0efaa7700 1 -- start start 2022-06-17T12:55:19.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.359+0000 7fb0efaa7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb0e805c870 0x7fb0e81519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:19.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.359+0000 7fb0efaa7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb0e813a320 0x7fb0e813b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:19.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.359+0000 7fb0efaa7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0e8063e00 0x7fb0e8064280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:19.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.359+0000 7fb0efaa7700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e80668c0 con 0x7fb0e813a320 2022-06-17T12:55:19.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.359+0000 7fb0efaa7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e813be10 con 0x7fb0e8063e00 2022-06-17T12:55:19.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.359+0000 7fb0efaa7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e8151ee0 con 0x7fb0e805c870 2022-06-17T12:55:19.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.359+0000 7fb0ed042700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb0e813a320 0x7fb0e813b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:19.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.359+0000 7fb0ed042700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb0e813a320 0x7fb0e813b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38228/0 (socket says 172.21.15.36:38228) 2022-06-17T12:55:19.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.359+0000 7fb0ed042700 1 -- 172.21.15.36:0/593235267 learned_addr learned my addr 172.21.15.36:0/593235267 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:19.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.359+0000 7fb0ee044700 1 --2- 172.21.15.36:0/593235267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0e8063e00 0x7fb0e8064280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:19.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.359+0000 7fb0ed843700 1 --2- 172.21.15.36:0/593235267 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb0e805c870 0x7fb0e81519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:19.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.359+0000 7fb0ed042700 1 -- 172.21.15.36:0/593235267 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb0e805c870 msgr2=0x7fb0e81519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:19.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.359+0000 7fb0ed042700 1 --2- 172.21.15.36:0/593235267 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb0e805c870 0x7fb0e81519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:19.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.359+0000 7fb0ed042700 1 -- 172.21.15.36:0/593235267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0e8063e00 msgr2=0x7fb0e8064280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:19.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.359+0000 7fb0ed042700 1 --2- 172.21.15.36:0/593235267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0e8063e00 0x7fb0e8064280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:19.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.359+0000 7fb0ed042700 1 -- 172.21.15.36:0/593235267 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e8065bf0 con 0x7fb0e813a320 2022-06-17T12:55:19.366 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.359+0000 7fb0ed042700 1 --2- 172.21.15.36:0/593235267 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb0e813a320 0x7fb0e813b3e0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7fb0e400b000 tx=0x7fb0e4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:19.366 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.359+0000 7fb0d6ffd700 1 -- 172.21.15.36:0/593235267 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0e4011ed0 con 0x7fb0e813a320 2022-06-17T12:55:19.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.359+0000 7fb0d6ffd700 1 -- 172.21.15.36:0/593235267 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e4000e90 con 0x7fb0e813a320 2022-06-17T12:55:19.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.359+0000 7fb0d6ffd700 1 -- 172.21.15.36:0/593235267 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0e400fb30 con 0x7fb0e813a320 2022-06-17T12:55:19.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.359+0000 7fb0efaa7700 1 -- 172.21.15.36:0/593235267 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e813b8d0 con 0x7fb0e813a320 2022-06-17T12:55:19.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.359+0000 7fb0efaa7700 1 -- 172.21.15.36:0/593235267 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e8139760 con 0x7fb0e813a320 2022-06-17T12:55:19.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.363+0000 7fb0efaa7700 1 -- 172.21.15.36:0/593235267 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e8066a30 con 0x7fb0e813a320 2022-06-17T12:55:19.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.363+0000 7fb0d6ffd700 1 -- 172.21.15.36:0/593235267 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fb0e400fc90 con 0x7fb0e813a320 2022-06-17T12:55:19.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.363+0000 7fb0d6ffd700 1 --2- 172.21.15.36:0/593235267 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb0c8053ad0 0x7fb0c8055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:19.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.363+0000 7fb0d6ffd700 1 -- 172.21.15.36:0/593235267 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fb0e406c500 con 0x7fb0e813a320 2022-06-17T12:55:19.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.363+0000 7fb0ed843700 1 --2- 172.21.15.36:0/593235267 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb0c8053ad0 0x7fb0c8055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:19.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.363+0000 7fb0ed843700 1 --2- 172.21.15.36:0/593235267 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb0c8053ad0 0x7fb0c8055f90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fb0d801a000 tx=0x7fb0d8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:19.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.367+0000 7fb0d6ffd700 1 -- 172.21.15.36:0/593235267 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb0e8066a30 con 0x7fb0e813a320 2022-06-17T12:55:19.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.499+0000 7fb0efaa7700 1 -- 172.21.15.36:0/593235267 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb0e813bfd0 con 0x7fb0e813a320 2022-06-17T12:55:19.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.499+0000 7fb0d6ffd700 1 -- 172.21.15.36:0/593235267 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v107) v1 ==== 81+0+29227 (secure 0 0 0) 0x7fb0e4018020 con 0x7fb0e813a320 2022-06-17T12:55:19.490 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:19.490 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:55:19.491 INFO:teuthology.orchestra.run.smithi036.stdout: "epoch": 107, 2022-06-17T12:55:19.491 INFO:teuthology.orchestra.run.smithi036.stdout: "fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", 2022-06-17T12:55:19.492 INFO:teuthology.orchestra.run.smithi036.stdout: "created": "2022-06-17T12:46:30.671408+0000", 2022-06-17T12:55:19.492 INFO:teuthology.orchestra.run.smithi036.stdout: "modified": "2022-06-17T12:55:16.744886+0000", 2022-06-17T12:55:19.492 INFO:teuthology.orchestra.run.smithi036.stdout: "last_up_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:55:19.493 INFO:teuthology.orchestra.run.smithi036.stdout: "last_in_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:55:19.496 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-17T12:55:19.497 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_num": 5799936, 2022-06-17T12:55:19.498 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_set": [ 2022-06-17T12:55:19.498 INFO:teuthology.orchestra.run.smithi036.stdout: "pglog_hardlimit", 2022-06-17T12:55:19.499 INFO:teuthology.orchestra.run.smithi036.stdout: "purged_snapdirs", 2022-06-17T12:55:19.499 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_deletes", 2022-06-17T12:55:19.499 INFO:teuthology.orchestra.run.smithi036.stdout: "sortbitwise" 2022-06-17T12:55:19.499 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:55:19.500 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_version": 6, 2022-06-17T12:55:19.500 INFO:teuthology.orchestra.run.smithi036.stdout: "full_ratio": 0.94999998807907104, 2022-06-17T12:55:19.500 INFO:teuthology.orchestra.run.smithi036.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-17T12:55:19.501 INFO:teuthology.orchestra.run.smithi036.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-17T12:55:19.502 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_snapshot": "", 2022-06-17T12:55:19.503 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_max": 20, 2022-06-17T12:55:19.503 INFO:teuthology.orchestra.run.smithi036.stdout: "max_osd": 8, 2022-06-17T12:55:19.504 INFO:teuthology.orchestra.run.smithi036.stdout: "require_min_compat_client": "jewel", 2022-06-17T12:55:19.504 INFO:teuthology.orchestra.run.smithi036.stdout: "min_compat_client": "jewel", 2022-06-17T12:55:19.505 INFO:teuthology.orchestra.run.smithi036.stdout: "require_osd_release": "octopus", 2022-06-17T12:55:19.505 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:55:19.506 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.506 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 1, 2022-06-17T12:55:19.507 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "device_health_metrics", 2022-06-17T12:55:19.507 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:33.881212+0000", 2022-06-17T12:55:19.507 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:55:19.508 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:55:19.508 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:55:19.509 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:55:19.509 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:55:19.509 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:55:19.510 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:55:19.510 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:55:19.511 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 1, 2022-06-17T12:55:19.511 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 1, 2022-06-17T12:55:19.511 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 1, 2022-06-17T12:55:19.512 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 1, 2022-06-17T12:55:19.512 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 1, 2022-06-17T12:55:19.512 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:55:19.513 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:55:19.513 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:55:19.514 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:55:19.514 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:55:19.515 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:55:19.515 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:55:19.516 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.516 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "16", 2022-06-17T12:55:19.517 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:55:19.517 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:55:19.517 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:55:19.518 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:55:19.518 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:55:19.519 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:55:19.519 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:55:19.520 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:55:19.520 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:55:19.521 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:55:19.521 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:55:19.521 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:55:19.522 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:55:19.522 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:55:19.522 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:55:19.523 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:55:19.524 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:55:19.524 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:55:19.524 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:55:19.525 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:55:19.525 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:55:19.525 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:55:19.526 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:55:19.526 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:55:19.526 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:55:19.526 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:55:19.527 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.527 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:55:19.527 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:55:19.528 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:55:19.528 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:55:19.528 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:55:19.528 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:55:19.529 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:55:19.529 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:55:19.529 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:55:19.529 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:55:19.530 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:55:19.530 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:55:19.530 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 1 2022-06-17T12:55:19.530 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.531 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:55:19.531 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr_devicehealth": {} 2022-06-17T12:55:19.531 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.531 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.532 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.532 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 2, 2022-06-17T12:55:19.532 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "rbd", 2022-06-17T12:55:19.532 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:44.485460+0000", 2022-06-17T12:55:19.533 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:55:19.533 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:55:19.533 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:55:19.533 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:55:19.534 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:55:19.534 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:55:19.534 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:55:19.535 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:55:19.535 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:55:19.535 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:55:19.535 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:55:19.536 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:55:19.536 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:55:19.536 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:55:19.537 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:55:19.537 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:55:19.537 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:55:19.537 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:55:19.538 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:55:19.538 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:55:19.538 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.538 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "19", 2022-06-17T12:55:19.539 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:55:19.539 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:55:19.539 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:55:19.539 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:55:19.540 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:55:19.540 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:55:19.541 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:55:19.541 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:55:19.541 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:55:19.541 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:55:19.542 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:55:19.542 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:55:19.542 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:55:19.543 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:55:19.543 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:55:19.543 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:55:19.543 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:55:19.544 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:55:19.544 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:55:19.544 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:55:19.545 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:55:19.545 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:55:19.545 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:55:19.545 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:55:19.546 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:55:19.546 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:55:19.546 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.546 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:55:19.547 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:55:19.547 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:55:19.547 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:55:19.547 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:55:19.548 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:55:19.548 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:55:19.548 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:55:19.548 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:55:19.549 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:55:19.549 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:55:19.549 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:55:19.549 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:55:19.550 INFO:teuthology.orchestra.run.smithi036.stdout: "rbd": {} 2022-06-17T12:55:19.550 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.550 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.550 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:55:19.551 INFO:teuthology.orchestra.run.smithi036.stdout: "osds": [ 2022-06-17T12:55:19.551 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.551 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:55:19.551 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", 2022-06-17T12:55:19.552 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:55:19.552 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:55:19.552 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:55:19.552 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:55:19.553 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:55:19.553 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:55:19.553 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:55:19.553 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 102, 2022-06-17T12:55:19.554 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:55:19.554 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:55:19.554 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:55:19.554 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.554 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.555 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.555 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6802", 2022-06-17T12:55:19.555 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:55:19.555 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.556 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.556 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.556 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6803", 2022-06-17T12:55:19.556 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:55:19.557 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.557 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.557 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.558 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:55:19.558 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.558 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.558 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.559 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6804", 2022-06-17T12:55:19.559 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:55:19.559 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.559 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.560 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.560 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6805", 2022-06-17T12:55:19.560 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:55:19.560 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.560 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.561 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.561 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:55:19.561 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.561 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.562 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.562 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6808", 2022-06-17T12:55:19.562 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:55:19.562 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.563 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.563 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.563 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6809", 2022-06-17T12:55:19.564 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:55:19.564 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.565 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.565 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.566 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:55:19.566 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.567 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.567 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.567 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6806", 2022-06-17T12:55:19.568 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:55:19.568 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.569 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.569 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.570 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6807", 2022-06-17T12:55:19.570 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:55:19.571 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.571 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.571 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.572 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6803/22619", 2022-06-17T12:55:19.572 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6805/22619", 2022-06-17T12:55:19.573 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6809/22619", 2022-06-17T12:55:19.573 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6807/22619", 2022-06-17T12:55:19.573 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:55:19.574 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:55:19.574 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:55:19.575 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.575 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.576 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.576 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:55:19.577 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", 2022-06-17T12:55:19.577 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:55:19.577 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:55:19.578 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:55:19.578 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:55:19.579 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:55:19.579 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:55:19.579 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:55:19.580 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 102, 2022-06-17T12:55:19.580 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:55:19.580 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:55:19.580 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:55:19.581 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.581 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.581 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.581 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6810", 2022-06-17T12:55:19.582 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:55:19.582 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.582 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.582 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.583 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6811", 2022-06-17T12:55:19.583 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:55:19.583 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.584 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.584 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.584 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:55:19.584 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.585 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.585 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.585 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6812", 2022-06-17T12:55:19.585 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:55:19.585 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.586 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.586 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.586 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6813", 2022-06-17T12:55:19.586 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:55:19.587 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.587 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.587 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.587 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:55:19.588 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.588 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.588 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.588 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6816", 2022-06-17T12:55:19.588 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:55:19.589 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.589 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.589 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.589 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6817", 2022-06-17T12:55:19.590 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:55:19.590 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.590 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.590 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.591 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:55:19.591 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.591 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.591 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.592 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6814", 2022-06-17T12:55:19.592 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:55:19.592 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.592 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.592 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.593 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6815", 2022-06-17T12:55:19.593 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:55:19.593 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.593 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.594 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.594 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6811/22620", 2022-06-17T12:55:19.594 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6813/22620", 2022-06-17T12:55:19.594 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6817/22620", 2022-06-17T12:55:19.595 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6815/22620", 2022-06-17T12:55:19.595 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:55:19.595 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:55:19.595 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:55:19.596 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.596 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.596 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.596 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:55:19.597 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", 2022-06-17T12:55:19.597 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:55:19.597 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:55:19.597 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:55:19.597 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:55:19.598 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:55:19.598 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:55:19.598 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:55:19.598 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 101, 2022-06-17T12:55:19.599 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:55:19.599 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:55:19.599 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:55:19.599 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.599 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.600 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.600 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6826", 2022-06-17T12:55:19.600 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:55:19.600 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.601 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.601 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.601 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6827", 2022-06-17T12:55:19.601 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:55:19.602 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.602 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.602 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.602 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:55:19.603 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.603 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.603 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.603 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6828", 2022-06-17T12:55:19.604 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:55:19.604 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.604 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.604 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.604 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6829", 2022-06-17T12:55:19.605 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:55:19.605 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.605 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.605 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.606 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:55:19.606 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.606 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.607 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.607 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6832", 2022-06-17T12:55:19.607 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:55:19.607 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.607 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.608 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.608 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6833", 2022-06-17T12:55:19.608 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:55:19.608 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.609 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.609 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.609 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:55:19.609 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.609 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.610 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.610 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6830", 2022-06-17T12:55:19.610 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:55:19.610 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.611 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.611 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.611 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6831", 2022-06-17T12:55:19.611 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:55:19.611 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.612 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.612 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.612 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6827/22643", 2022-06-17T12:55:19.612 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6829/22643", 2022-06-17T12:55:19.613 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6833/22643", 2022-06-17T12:55:19.613 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6831/22643", 2022-06-17T12:55:19.613 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:55:19.613 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:55:19.613 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:55:19.614 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.614 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.614 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.614 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:55:19.615 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", 2022-06-17T12:55:19.615 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:55:19.615 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:55:19.615 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:55:19.615 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:55:19.616 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:55:19.616 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:55:19.616 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:55:19.616 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 102, 2022-06-17T12:55:19.617 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:55:19.617 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:55:19.617 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:55:19.617 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.617 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.618 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.618 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6818", 2022-06-17T12:55:19.618 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:55:19.618 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.619 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.619 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.619 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6819", 2022-06-17T12:55:19.619 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:55:19.619 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.620 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.620 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.620 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:55:19.620 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.621 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.621 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.621 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6820", 2022-06-17T12:55:19.621 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:55:19.621 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.622 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.622 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.622 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6821", 2022-06-17T12:55:19.622 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:55:19.623 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.623 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.623 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.623 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:55:19.623 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.624 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.624 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.624 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6824", 2022-06-17T12:55:19.624 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:55:19.625 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.625 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.625 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.625 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6825", 2022-06-17T12:55:19.625 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:55:19.626 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.626 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.626 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.626 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:55:19.627 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.627 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.627 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.627 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6822", 2022-06-17T12:55:19.627 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:55:19.628 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.628 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.628 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.628 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6823", 2022-06-17T12:55:19.629 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:55:19.629 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.629 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.629 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.629 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6819/22646", 2022-06-17T12:55:19.630 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6821/22646", 2022-06-17T12:55:19.630 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6825/22646", 2022-06-17T12:55:19.630 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6823/22646", 2022-06-17T12:55:19.630 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:55:19.631 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:55:19.631 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:55:19.631 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.631 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.632 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.632 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:55:19.632 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", 2022-06-17T12:55:19.632 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:55:19.632 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:55:19.633 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:55:19.633 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:55:19.633 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:55:19.633 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:55:19.633 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:55:19.634 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 101, 2022-06-17T12:55:19.634 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:55:19.634 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:55:19.634 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:55:19.635 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.635 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.635 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.635 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6808", 2022-06-17T12:55:19.635 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:55:19.636 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.636 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.636 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.636 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6809", 2022-06-17T12:55:19.637 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:55:19.637 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.637 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.637 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.638 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:55:19.638 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.638 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.638 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.638 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6810", 2022-06-17T12:55:19.639 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:55:19.639 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.639 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.639 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.640 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6811", 2022-06-17T12:55:19.640 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:55:19.640 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.640 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.640 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.641 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:55:19.641 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.641 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.641 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.642 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6814", 2022-06-17T12:55:19.642 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:55:19.642 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.642 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.642 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.643 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6815", 2022-06-17T12:55:19.643 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:55:19.643 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.643 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.644 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.644 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:55:19.644 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.644 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.645 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.645 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6812", 2022-06-17T12:55:19.645 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:55:19.645 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.645 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.646 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.646 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6813", 2022-06-17T12:55:19.646 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:55:19.646 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.647 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.647 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.647 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6809/22812", 2022-06-17T12:55:19.647 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6811/22812", 2022-06-17T12:55:19.647 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6815/22812", 2022-06-17T12:55:19.648 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6813/22812", 2022-06-17T12:55:19.648 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:55:19.648 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:55:19.648 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:55:19.649 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.649 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.649 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.649 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:55:19.649 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", 2022-06-17T12:55:19.650 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:55:19.650 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:55:19.650 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:55:19.650 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:55:19.651 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:55:19.651 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:55:19.651 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:55:19.651 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 101, 2022-06-17T12:55:19.652 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:55:19.652 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:55:19.652 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:55:19.652 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.652 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.653 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.653 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6800", 2022-06-17T12:55:19.653 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:55:19.653 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.654 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.654 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.654 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6801", 2022-06-17T12:55:19.654 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:55:19.655 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.655 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.655 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.655 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:55:19.656 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.656 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.656 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.656 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6802", 2022-06-17T12:55:19.656 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:55:19.657 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.657 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.657 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.658 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6803", 2022-06-17T12:55:19.658 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:55:19.658 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.658 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.658 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.659 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:55:19.659 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.659 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.660 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.660 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6806", 2022-06-17T12:55:19.660 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:55:19.660 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.661 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.661 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.661 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6807", 2022-06-17T12:55:19.661 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:55:19.661 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.662 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.662 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.662 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:55:19.662 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.663 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.663 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.663 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6804", 2022-06-17T12:55:19.663 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:55:19.663 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.664 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.664 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.664 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6805", 2022-06-17T12:55:19.664 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:55:19.665 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.665 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.665 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.665 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6801/22813", 2022-06-17T12:55:19.665 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6803/22813", 2022-06-17T12:55:19.666 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6807/22813", 2022-06-17T12:55:19.666 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6805/22813", 2022-06-17T12:55:19.666 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:55:19.666 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:55:19.667 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:55:19.667 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.667 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.667 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.668 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:55:19.668 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", 2022-06-17T12:55:19.668 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:55:19.668 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:55:19.668 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:55:19.669 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:55:19.669 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:55:19.669 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:55:19.669 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:55:19.670 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 102, 2022-06-17T12:55:19.670 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:55:19.670 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:55:19.670 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:55:19.670 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.671 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.671 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.671 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6816", 2022-06-17T12:55:19.671 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:55:19.672 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.672 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.672 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.672 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6817", 2022-06-17T12:55:19.672 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:55:19.673 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.673 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.673 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.673 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:55:19.674 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.674 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.674 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.674 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6818", 2022-06-17T12:55:19.675 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:55:19.675 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.675 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.675 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.675 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6819", 2022-06-17T12:55:19.676 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:55:19.676 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.676 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.676 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.677 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:55:19.677 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.677 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.677 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.677 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6822", 2022-06-17T12:55:19.678 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:55:19.678 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.678 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.678 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.679 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6823", 2022-06-17T12:55:19.679 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:55:19.679 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.679 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.680 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.680 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:55:19.680 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.680 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.680 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.681 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6820", 2022-06-17T12:55:19.681 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:55:19.681 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.681 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.681 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.682 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6821", 2022-06-17T12:55:19.682 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:55:19.682 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.682 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.683 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.683 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6817/22822", 2022-06-17T12:55:19.683 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6819/22822", 2022-06-17T12:55:19.684 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6823/22822", 2022-06-17T12:55:19.684 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6821/22822", 2022-06-17T12:55:19.684 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:55:19.684 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:55:19.685 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:55:19.685 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.685 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.685 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.686 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:55:19.686 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", 2022-06-17T12:55:19.686 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:55:19.686 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:55:19.686 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:55:19.687 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:55:19.687 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:55:19.687 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:55:19.687 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:55:19.688 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 102, 2022-06-17T12:55:19.688 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:55:19.688 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:55:19.688 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:55:19.688 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.689 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.689 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.689 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6824", 2022-06-17T12:55:19.689 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:55:19.690 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.690 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.690 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.690 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6825", 2022-06-17T12:55:19.691 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:55:19.691 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.691 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.691 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.691 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:55:19.692 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.692 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.692 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.692 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6826", 2022-06-17T12:55:19.693 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:55:19.693 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.693 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.693 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.693 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6827", 2022-06-17T12:55:19.694 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:55:19.694 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.694 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.694 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.695 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:55:19.695 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.695 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.695 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.695 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6830", 2022-06-17T12:55:19.696 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:55:19.696 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.696 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.696 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.697 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6831", 2022-06-17T12:55:19.697 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:55:19.697 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.697 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.697 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.698 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:55:19.698 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:19.698 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.698 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:19.699 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6828", 2022-06-17T12:55:19.699 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:55:19.699 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.699 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.699 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:19.700 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6829", 2022-06-17T12:55:19.700 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:55:19.700 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.700 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.701 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.701 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6825/22837", 2022-06-17T12:55:19.701 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6827/22837", 2022-06-17T12:55:19.701 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6831/22837", 2022-06-17T12:55:19.701 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6829/22837", 2022-06-17T12:55:19.702 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:55:19.702 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:55:19.702 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:55:19.702 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:19.703 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.703 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:55:19.703 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_xinfo": [ 2022-06-17T12:55:19.703 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.704 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:55:19.704 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:55:19.704 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:55:19.704 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:55:19.704 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:55:19.705 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:55:19.705 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.291461+0000", 2022-06-17T12:55:19.705 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:55:19.705 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.706 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.706 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:55:19.706 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:55:19.706 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:55:19.706 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:55:19.707 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:55:19.707 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:55:19.707 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.537912+0000", 2022-06-17T12:55:19.708 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:55:19.708 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.708 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.708 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:55:19.709 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:55:19.709 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:55:19.709 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:55:19.709 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:55:19.709 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:55:19.710 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.606947+0000", 2022-06-17T12:55:19.710 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:55:19.710 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.710 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.711 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:55:19.711 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:55:19.711 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:55:19.711 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:55:19.711 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:55:19.712 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:55:19.712 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.514412+0000", 2022-06-17T12:55:19.712 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:55:19.712 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.713 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.713 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:55:19.713 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:55:19.713 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:55:19.713 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:55:19.714 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:55:19.714 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:55:19.714 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.394645+0000", 2022-06-17T12:55:19.714 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:55:19.715 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.715 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.715 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:55:19.715 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:55:19.715 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:55:19.716 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:55:19.716 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:55:19.716 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:55:19.716 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.343806+0000", 2022-06-17T12:55:19.717 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:55:19.717 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.717 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.717 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:55:19.717 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:55:19.718 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:55:19.718 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:55:19.718 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:55:19.718 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:55:19.719 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.591864+0000", 2022-06-17T12:55:19.719 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:55:19.719 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.719 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:19.719 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:55:19.720 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:55:19.720 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:55:19.720 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:55:19.720 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:55:19.721 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:55:19.721 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.607045+0000", 2022-06-17T12:55:19.721 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:55:19.721 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.721 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:55:19.722 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap": [], 2022-06-17T12:55:19.722 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap_items": [], 2022-06-17T12:55:19.722 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_temp": [], 2022-06-17T12:55:19.722 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_temp": [], 2022-06-17T12:55:19.723 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklist": { 2022-06-17T12:55:19.723 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.36:6835/233328233": "2022-06-18T12:55:13.632666+0000", 2022-06-17T12:55:19.723 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.36:6834/233328233": "2022-06-18T12:55:13.632666+0000" 2022-06-17T12:55:19.723 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.723 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profiles": { 2022-06-17T12:55:19.724 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs_data": { 2022-06-17T12:55:19.724 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-device-class": "", 2022-06-17T12:55:19.724 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:55:19.724 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-root": "default", 2022-06-17T12:55:19.725 INFO:teuthology.orchestra.run.smithi036.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-17T12:55:19.725 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:55:19.725 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "2", 2022-06-17T12:55:19.725 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:55:19.725 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:55:19.726 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van", 2022-06-17T12:55:19.726 INFO:teuthology.orchestra.run.smithi036.stdout: "w": "8" 2022-06-17T12:55:19.726 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.727 INFO:teuthology.orchestra.run.smithi036.stdout: "default": { 2022-06-17T12:55:19.727 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:55:19.728 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:55:19.728 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "1", 2022-06-17T12:55:19.729 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:55:19.729 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:55:19.729 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van" 2022-06-17T12:55:19.730 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:19.730 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:19.731 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps_queue": [], 2022-06-17T12:55:19.731 INFO:teuthology.orchestra.run.smithi036.stdout: "new_removed_snaps": [], 2022-06-17T12:55:19.732 INFO:teuthology.orchestra.run.smithi036.stdout: "new_purged_snaps": [], 2022-06-17T12:55:19.732 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_node_flags": {}, 2022-06-17T12:55:19.732 INFO:teuthology.orchestra.run.smithi036.stdout: "device_class_flags": {} 2022-06-17T12:55:19.733 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:55:19.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.503+0000 7fb0efaa7700 1 -- 172.21.15.36:0/593235267 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb0c8053ad0 msgr2=0x7fb0c8055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:19.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.503+0000 7fb0efaa7700 1 --2- 172.21.15.36:0/593235267 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb0c8053ad0 0x7fb0c8055f90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fb0d801a000 tx=0x7fb0d8015000).stop 2022-06-17T12:55:19.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.503+0000 7fb0efaa7700 1 -- 172.21.15.36:0/593235267 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb0e813a320 msgr2=0x7fb0e813b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:19.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.503+0000 7fb0efaa7700 1 --2- 172.21.15.36:0/593235267 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb0e813a320 0x7fb0e813b3e0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7fb0e400b000 tx=0x7fb0e4006df0).stop 2022-06-17T12:55:19.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.503+0000 7fb0efaa7700 1 -- 172.21.15.36:0/593235267 shutdown_connections 2022-06-17T12:55:19.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.503+0000 7fb0efaa7700 1 --2- 172.21.15.36:0/593235267 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb0c8053ad0 0x7fb0c8055f90 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:19.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.503+0000 7fb0efaa7700 1 --2- 172.21.15.36:0/593235267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0e8063e00 0x7fb0e8064280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:19.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.503+0000 7fb0efaa7700 1 --2- 172.21.15.36:0/593235267 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb0e813a320 0x7fb0e813b3e0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:19.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.503+0000 7fb0efaa7700 1 --2- 172.21.15.36:0/593235267 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb0e805c870 0x7fb0e81519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:19.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.503+0000 7fb0efaa7700 1 -- 172.21.15.36:0/593235267 >> 172.21.15.36:0/593235267 conn(0x7fb0e8063890 msgr2=0x7fb0e8140650 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:19.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.503+0000 7fb0efaa7700 1 -- 172.21.15.36:0/593235267 shutdown_connections 2022-06-17T12:55:19.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.503+0000 7fb0efaa7700 1 -- 172.21.15.36:0/593235267 wait complete. 2022-06-17T12:55:19.743 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:55:19.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.863+0000 7f1d3128b700 1 Processor -- start 2022-06-17T12:55:19.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d3128b700 1 -- start start 2022-06-17T12:55:19.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d3128b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f1d2c0b7c10 0x7f1d2c145070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:19.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d3128b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d2c066fd0 0x7f1d2c13df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:19.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d3128b700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d2c1417a0 con 0x7f1d2c0ad330 2022-06-17T12:55:19.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d3128b700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d2c1422e0 con 0x7f1d2c0b7c10 2022-06-17T12:55:19.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d3128b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c0b7ff0 con 0x7f1d2c066fd0 2022-06-17T12:55:19.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d2affd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f1d2c0b7c10 0x7f1d2c145070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:19.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d2a7fc700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f1d2c0ad330 0x7f1d2c141090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55766/0 (socket says 172.21.15.36:55766) 2022-06-17T12:55:19.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d2a7fc700 1 -- 172.21.15.36:0/2195656460 learned_addr learned my addr 172.21.15.36:0/2195656460 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:19.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d2b7fe700 1 --2- 172.21.15.36:0/2195656460 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d2c066fd0 0x7f1d2c13df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:19.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d29ffb700 1 -- 172.21.15.36:0/2195656460 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2866591347 0 0) 0x7f1d2c1417a0 con 0x7f1d2c0ad330 2022-06-17T12:55:19.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d29ffb700 1 -- 172.21.15.36:0/2195656460 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d0c003260 con 0x7f1d2c0ad330 2022-06-17T12:55:19.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d2affd700 1 -- 172.21.15.36:0/2195656460 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d2c066fd0 msgr2=0x7f1d2c13df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:19.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d2affd700 1 --2- 172.21.15.36:0/2195656460 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d2c066fd0 0x7f1d2c13df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:19.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d2affd700 1 -- 172.21.15.36:0/2195656460 >> v1:172.21.15.36:6789/0 conn(0x7f1d2c0ad330 legacy=0x7f1d2c141090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:19.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d2affd700 1 -- 172.21.15.36:0/2195656460 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2c0b30a0 con 0x7f1d2c0b7c10 2022-06-17T12:55:19.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d2b7fe700 1 --2- 172.21.15.36:0/2195656460 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d2c066fd0 0x7f1d2c13df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:19.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d2affd700 1 --2- 172.21.15.36:0/2195656460 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d2c0b7c10 0x7f1d2c145070 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7f1d1c002f70 tx=0x7f1d1c006df0).ready entity=mon.0 client_cookie=7008fc6f7bc7c1e5 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:19.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d29ffb700 1 -- 172.21.15.36:0/2195656460 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d1c00d4a0 con 0x7f1d2c0b7c10 2022-06-17T12:55:19.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d29ffb700 1 -- 172.21.15.36:0/2195656460 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1c000e90 con 0x7f1d2c0b7c10 2022-06-17T12:55:19.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d29ffb700 1 -- 172.21.15.36:0/2195656460 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d1c00d7f0 con 0x7f1d2c0b7c10 2022-06-17T12:55:19.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d3128b700 1 -- 172.21.15.36:0/2195656460 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d2c0b7c10 msgr2=0x7f1d2c145070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:19.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d3128b700 1 --2- 172.21.15.36:0/2195656460 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d2c0b7c10 0x7f1d2c145070 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7f1d1c002f70 tx=0x7f1d1c006df0).stop 2022-06-17T12:55:19.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d3128b700 1 -- 172.21.15.36:0/2195656460 shutdown_connections 2022-06-17T12:55:19.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d3128b700 1 --2- 172.21.15.36:0/2195656460 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d2c066fd0 0x7f1d2c13df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:19.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d3128b700 1 --2- 172.21.15.36:0/2195656460 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d2c0b7c10 0x7f1d2c145070 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:19.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d3128b700 1 -- 172.21.15.36:0/2195656460 >> 172.21.15.36:0/2195656460 conn(0x7f1d2c060c60 msgr2=0x7f1d2c05c550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:19.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d3128b700 1 -- 172.21.15.36:0/2195656460 shutdown_connections 2022-06-17T12:55:19.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.867+0000 7f1d3128b700 1 -- 172.21.15.36:0/2195656460 wait complete. 2022-06-17T12:55:19.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.871+0000 7f1d3128b700 1 Processor -- start 2022-06-17T12:55:19.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.871+0000 7f1d3128b700 1 -- start start 2022-06-17T12:55:19.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.871+0000 7f1d3128b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d2c066fd0 0x7f1d2c0aef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:19.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.871+0000 7f1d3128b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d2c0ad330 0x7f1d2c0b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:19.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.871+0000 7f1d3128b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d2c0b7c10 0x7f1d2c13a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:19.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.871+0000 7f1d3128b700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c144380 con 0x7f1d2c066fd0 2022-06-17T12:55:19.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.871+0000 7f1d3128b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c1445b0 con 0x7f1d2c0ad330 2022-06-17T12:55:19.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.871+0000 7f1d3128b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c0b18d0 con 0x7f1d2c0b7c10 2022-06-17T12:55:19.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.871+0000 7f1d2affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d2c066fd0 0x7f1d2c0aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:19.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.871+0000 7f1d2affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d2c066fd0 0x7f1d2c0aef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38242/0 (socket says 172.21.15.36:38242) 2022-06-17T12:55:19.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.871+0000 7f1d2affd700 1 -- 172.21.15.36:0/2799845401 learned_addr learned my addr 172.21.15.36:0/2799845401 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:19.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.871+0000 7f1d2b7fe700 1 --2- 172.21.15.36:0/2799845401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d2c0b7c10 0x7f1d2c13a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:19.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.871+0000 7f1d2a7fc700 1 --2- 172.21.15.36:0/2799845401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d2c0ad330 0x7f1d2c0b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:19.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.875+0000 7f1d2affd700 1 -- 172.21.15.36:0/2799845401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d2c0b7c10 msgr2=0x7f1d2c13a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:19.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.875+0000 7f1d2affd700 1 --2- 172.21.15.36:0/2799845401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d2c0b7c10 0x7f1d2c13a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:19.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.875+0000 7f1d2affd700 1 -- 172.21.15.36:0/2799845401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d2c0ad330 msgr2=0x7f1d2c0b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:19.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.875+0000 7f1d2affd700 1 --2- 172.21.15.36:0/2799845401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d2c0ad330 0x7f1d2c0b13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:19.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.875+0000 7f1d2affd700 1 -- 172.21.15.36:0/2799845401 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d24009470 con 0x7f1d2c066fd0 2022-06-17T12:55:19.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.875+0000 7f1d2a7fc700 1 --2- 172.21.15.36:0/2799845401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d2c0ad330 0x7f1d2c0b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:19.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.875+0000 7f1d2affd700 1 --2- 172.21.15.36:0/2799845401 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d2c066fd0 0x7f1d2c0aef20 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7f1d1c002f70 tx=0x7f1d1c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:19.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.875+0000 7f1d23fff700 1 -- 172.21.15.36:0/2799845401 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d1c016070 con 0x7f1d2c066fd0 2022-06-17T12:55:19.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.875+0000 7f1d23fff700 1 -- 172.21.15.36:0/2799845401 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1c013a40 con 0x7f1d2c066fd0 2022-06-17T12:55:19.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.875+0000 7f1d23fff700 1 -- 172.21.15.36:0/2799845401 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d1c013da0 con 0x7f1d2c066fd0 2022-06-17T12:55:19.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.875+0000 7f1d3128b700 1 -- 172.21.15.36:0/2799845401 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d1c00b180 con 0x7f1d2c066fd0 2022-06-17T12:55:19.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.875+0000 7f1d3128b700 1 -- 172.21.15.36:0/2799845401 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d2c1539f0 con 0x7f1d2c066fd0 2022-06-17T12:55:19.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.875+0000 7f1d23fff700 1 -- 172.21.15.36:0/2799845401 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f1d1c00f030 con 0x7f1d2c066fd0 2022-06-17T12:55:19.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.875+0000 7f1d3128b700 1 -- 172.21.15.36:0/2799845401 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d2c144720 con 0x7f1d2c066fd0 2022-06-17T12:55:19.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.875+0000 7f1d23fff700 1 --2- 172.21.15.36:0/2799845401 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1d10053ad0 0x7f1d10055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:19.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.875+0000 7f1d23fff700 1 -- 172.21.15.36:0/2799845401 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f1d1c06ca30 con 0x7f1d2c066fd0 2022-06-17T12:55:19.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.875+0000 7f1d2a7fc700 1 --2- 172.21.15.36:0/2799845401 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1d10053ad0 0x7f1d10055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:19.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.875+0000 7f1d2a7fc700 1 --2- 172.21.15.36:0/2799845401 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1d10053ad0 0x7f1d10055f90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f1d1400b000 tx=0x7f1d14009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:19.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:19.883+0000 7f1d23fff700 1 -- 172.21.15.36:0/2799845401 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1d2c144720 con 0x7f1d2c066fd0 2022-06-17T12:55:20.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.043+0000 7f1d3128b700 1 -- 172.21.15.36:0/2799845401 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d2c144720 con 0x7f1d2c066fd0 2022-06-17T12:55:20.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.047+0000 7f1d23fff700 1 -- 172.21.15.36:0/2799845401 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+1206 (secure 0 0 0) 0x7f1d2c144720 con 0x7f1d2c066fd0 2022-06-17T12:55:20.034 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:20.034 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":56,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/676934608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":676934608},{"type":"v1","addr":"172.21.15.70:6833","nonce":676934608}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":51},{"gid":7635,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/1915322959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":1915322959},{"type":"v1","addr":"172.21.15.36:6835","nonce":1915322959}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":56}],"filesystems":[]} 2022-06-17T12:55:20.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.047+0000 7f1d3128b700 1 -- 172.21.15.36:0/2799845401 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1d10053ad0 msgr2=0x7f1d10055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:20.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.047+0000 7f1d3128b700 1 --2- 172.21.15.36:0/2799845401 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1d10053ad0 0x7f1d10055f90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f1d1400b000 tx=0x7f1d14009040).stop 2022-06-17T12:55:20.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.047+0000 7f1d3128b700 1 -- 172.21.15.36:0/2799845401 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d2c066fd0 msgr2=0x7f1d2c0aef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:20.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.047+0000 7f1d3128b700 1 --2- 172.21.15.36:0/2799845401 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d2c066fd0 0x7f1d2c0aef20 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7f1d1c002f70 tx=0x7f1d1c014000).stop 2022-06-17T12:55:20.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.047+0000 7f1d3128b700 1 -- 172.21.15.36:0/2799845401 shutdown_connections 2022-06-17T12:55:20.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.047+0000 7f1d3128b700 1 --2- 172.21.15.36:0/2799845401 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1d10053ad0 0x7f1d10055f90 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:20.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.047+0000 7f1d3128b700 1 --2- 172.21.15.36:0/2799845401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d2c0b7c10 0x7f1d2c13a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:20.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.047+0000 7f1d3128b700 1 --2- 172.21.15.36:0/2799845401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d2c0ad330 0x7f1d2c0b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:20.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.047+0000 7f1d3128b700 1 --2- 172.21.15.36:0/2799845401 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d2c066fd0 0x7f1d2c0aef20 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:20.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.047+0000 7f1d3128b700 1 -- 172.21.15.36:0/2799845401 >> 172.21.15.36:0/2799845401 conn(0x7f1d2c060c60 msgr2=0x7f1d2c13aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:20.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.047+0000 7f1d3128b700 1 -- 172.21.15.36:0/2799845401 shutdown_connections 2022-06-17T12:55:20.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.047+0000 7f1d3128b700 1 -- 172.21.15.36:0/2799845401 wait complete. 2022-06-17T12:55:20.040 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 56 2022-06-17T12:55:20.049 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:55:20.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.163+0000 7fd800121700 1 Processor -- start 2022-06-17T12:55:20.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.163+0000 7fd800121700 1 -- start start 2022-06-17T12:55:20.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.167+0000 7fd800121700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7f80ad9c0 0x7fd7f80b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:20.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.167+0000 7fd800121700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f80afc30 con 0x7fd7f80ae270 2022-06-17T12:55:20.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.167+0000 7fd800121700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f80ae650 con 0x7fd7f813fde0 2022-06-17T12:55:20.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.167+0000 7fd800121700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f80b0cb0 con 0x7fd7f80ad9c0 2022-06-17T12:55:20.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.167+0000 7fd7fdebd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fd7f80ae270 0x7fd7f80b05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55782/0 (socket says 172.21.15.36:55782) 2022-06-17T12:55:20.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.167+0000 7fd7fdebd700 1 -- 172.21.15.36:0/992909065 learned_addr learned my addr 172.21.15.36:0/992909065 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:20.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.167+0000 7fd7fd6bc700 1 --2- 172.21.15.36:0/992909065 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7f80ad9c0 0x7fd7f80b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:20.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.167+0000 7fd7fcebb700 1 -- 172.21.15.36:0/992909065 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1035951340 0 0) 0x7fd7f80afc30 con 0x7fd7f80ae270 2022-06-17T12:55:20.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.167+0000 7fd7fcebb700 1 -- 172.21.15.36:0/992909065 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7e0003260 con 0x7fd7f80ae270 2022-06-17T12:55:20.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.167+0000 7fd7fd6bc700 1 -- 172.21.15.36:0/992909065 >> v1:172.21.15.70:6789/0 conn(0x7fd7f813fde0 legacy=0x7fd7f813edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:20.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.167+0000 7fd7fd6bc700 1 -- 172.21.15.36:0/992909065 >> v1:172.21.15.36:6789/0 conn(0x7fd7f80ae270 legacy=0x7fd7f80b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:20.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.167+0000 7fd7fd6bc700 1 -- 172.21.15.36:0/992909065 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f80afe60 con 0x7fd7f80ad9c0 2022-06-17T12:55:20.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.167+0000 7fd7fd6bc700 1 --2- 172.21.15.36:0/992909065 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7f80ad9c0 0x7fd7f80b39c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fd7ec002f70 tx=0x7fd7ec006df0).ready entity=mon.2 client_cookie=9cf413cbc4b18982 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:20.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.167+0000 7fd7fcebb700 1 -- 172.21.15.36:0/992909065 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7ec014070 con 0x7fd7f80ad9c0 2022-06-17T12:55:20.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.167+0000 7fd7fcebb700 1 -- 172.21.15.36:0/992909065 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ec000e90 con 0x7fd7f80ad9c0 2022-06-17T12:55:20.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.167+0000 7fd7fcebb700 1 -- 172.21.15.36:0/992909065 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7ec00fb60 con 0x7fd7f80ad9c0 2022-06-17T12:55:20.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.167+0000 7fd800121700 1 -- 172.21.15.36:0/992909065 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7f80ad9c0 msgr2=0x7fd7f80b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:20.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.167+0000 7fd800121700 1 --2- 172.21.15.36:0/992909065 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7f80ad9c0 0x7fd7f80b39c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fd7ec002f70 tx=0x7fd7ec006df0).stop 2022-06-17T12:55:20.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.167+0000 7fd800121700 1 -- 172.21.15.36:0/992909065 shutdown_connections 2022-06-17T12:55:20.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.167+0000 7fd800121700 1 --2- 172.21.15.36:0/992909065 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7f80ad9c0 0x7fd7f80b39c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:20.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.167+0000 7fd800121700 1 -- 172.21.15.36:0/992909065 >> 172.21.15.36:0/992909065 conn(0x7fd7f805c550 msgr2=0x7fd7f805c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:20.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.167+0000 7fd800121700 1 -- 172.21.15.36:0/992909065 shutdown_connections 2022-06-17T12:55:20.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.167+0000 7fd800121700 1 -- 172.21.15.36:0/992909065 wait complete. 2022-06-17T12:55:20.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.171+0000 7fd800121700 1 Processor -- start 2022-06-17T12:55:20.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.171+0000 7fd800121700 1 -- start start 2022-06-17T12:55:20.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.171+0000 7fd800121700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7f80ad9c0 0x7fd7f8145a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:20.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.171+0000 7fd800121700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7f80ae270 0x7fd7f813ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:20.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.171+0000 7fd800121700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7f813fde0 0x7fd7f813b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:20.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.171+0000 7fd800121700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f8144700 con 0x7fd7f813fde0 2022-06-17T12:55:20.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.171+0000 7fd800121700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f8143050 con 0x7fd7f80ae270 2022-06-17T12:55:20.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.171+0000 7fd800121700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f8142230 con 0x7fd7f80ad9c0 2022-06-17T12:55:20.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.171+0000 7fd7fe6be700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7f813fde0 0x7fd7f813b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:20.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.171+0000 7fd7fdebd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7f80ad9c0 0x7fd7f8145a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:20.169 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.171+0000 7fd7fd6bc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7f80ae270 0x7fd7f813ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:20.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.171+0000 7fd7fe6be700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7f813fde0 0x7fd7f813b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38260/0 (socket says 172.21.15.36:38260) 2022-06-17T12:55:20.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.171+0000 7fd7fe6be700 1 -- 172.21.15.36:0/3132069572 learned_addr learned my addr 172.21.15.36:0/3132069572 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:20.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.171+0000 7fd7fe6be700 1 -- 172.21.15.36:0/3132069572 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7f80ad9c0 msgr2=0x7fd7f8145a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:20.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.171+0000 7fd7fe6be700 1 --2- 172.21.15.36:0/3132069572 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7f80ad9c0 0x7fd7f8145a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:20.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.171+0000 7fd7fe6be700 1 -- 172.21.15.36:0/3132069572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7f80ae270 msgr2=0x7fd7f813ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:20.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.171+0000 7fd7fe6be700 1 --2- 172.21.15.36:0/3132069572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7f80ae270 0x7fd7f813ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:20.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.171+0000 7fd7fe6be700 1 -- 172.21.15.36:0/3132069572 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ec00b180 con 0x7fd7f813fde0 2022-06-17T12:55:20.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.171+0000 7fd7fd6bc700 1 --2- 172.21.15.36:0/3132069572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7f80ae270 0x7fd7f813ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:20.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.171+0000 7fd7fdebd700 1 --2- 172.21.15.36:0/3132069572 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7f80ad9c0 0x7fd7f8145a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:20.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.171+0000 7fd7fe6be700 1 --2- 172.21.15.36:0/3132069572 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7f813fde0 0x7fd7f813b140 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7fd7f8063350 tx=0x7fd7f0007f50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:20.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.175+0000 7fd7eaffd700 1 -- 172.21.15.36:0/3132069572 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7f000ce80 con 0x7fd7f813fde0 2022-06-17T12:55:20.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.175+0000 7fd7eaffd700 1 -- 172.21.15.36:0/3132069572 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f000f470 con 0x7fd7f813fde0 2022-06-17T12:55:20.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.175+0000 7fd7eaffd700 1 -- 172.21.15.36:0/3132069572 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7f000f7d0 con 0x7fd7f813fde0 2022-06-17T12:55:20.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.175+0000 7fd800121700 1 -- 172.21.15.36:0/3132069572 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f8143280 con 0x7fd7f813fde0 2022-06-17T12:55:20.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.175+0000 7fd800121700 1 -- 172.21.15.36:0/3132069572 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f813b6b0 con 0x7fd7f813fde0 2022-06-17T12:55:20.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.175+0000 7fd800121700 1 -- 172.21.15.36:0/3132069572 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f8142710 con 0x7fd7f813fde0 2022-06-17T12:55:20.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.175+0000 7fd7eaffd700 1 -- 172.21.15.36:0/3132069572 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd7f0010070 con 0x7fd7f813fde0 2022-06-17T12:55:20.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.175+0000 7fd7eaffd700 1 --2- 172.21.15.36:0/3132069572 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7d8053ad0 0x7fd7d8055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:20.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.175+0000 7fd7eaffd700 1 -- 172.21.15.36:0/3132069572 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fd7f006b130 con 0x7fd7f813fde0 2022-06-17T12:55:20.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.175+0000 7fd7fdebd700 1 --2- 172.21.15.36:0/3132069572 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7d8053ad0 0x7fd7d8055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:20.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.179+0000 7fd7fdebd700 1 --2- 172.21.15.36:0/3132069572 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7d8053ad0 0x7fd7d8055f90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7fd7f400d000 tx=0x7fd7f4009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:20.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.183+0000 7fd7eaffd700 1 -- 172.21.15.36:0/3132069572 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd7f0042210 con 0x7fd7f813fde0 2022-06-17T12:55:20.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.347+0000 7fd800121700 1 -- 172.21.15.36:0/3132069572 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7f8141e50 con 0x7fd7f813fde0 2022-06-17T12:55:20.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.347+0000 7fd7eaffd700 1 -- 172.21.15.36:0/3132069572 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+1206 (secure 0 0 0) 0x7fd7f0045830 con 0x7fd7f813fde0 2022-06-17T12:55:20.338 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:20.339 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":56,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/676934608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":676934608},{"type":"v1","addr":"172.21.15.70:6833","nonce":676934608}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":51},{"gid":7635,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/1915322959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":1915322959},{"type":"v1","addr":"172.21.15.36:6835","nonce":1915322959}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":56}],"filesystems":[]} 2022-06-17T12:55:20.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.351+0000 7fd800121700 1 -- 172.21.15.36:0/3132069572 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7d8053ad0 msgr2=0x7fd7d8055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:20.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.351+0000 7fd800121700 1 --2- 172.21.15.36:0/3132069572 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7d8053ad0 0x7fd7d8055f90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7fd7f400d000 tx=0x7fd7f4009040).stop 2022-06-17T12:55:20.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.351+0000 7fd800121700 1 -- 172.21.15.36:0/3132069572 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7f813fde0 msgr2=0x7fd7f813b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:20.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.351+0000 7fd800121700 1 --2- 172.21.15.36:0/3132069572 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7f813fde0 0x7fd7f813b140 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7fd7f8063350 tx=0x7fd7f0007f50).stop 2022-06-17T12:55:20.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.351+0000 7fd800121700 1 -- 172.21.15.36:0/3132069572 shutdown_connections 2022-06-17T12:55:20.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.351+0000 7fd800121700 1 --2- 172.21.15.36:0/3132069572 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7d8053ad0 0x7fd7d8055f90 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:20.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.351+0000 7fd800121700 1 --2- 172.21.15.36:0/3132069572 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7f813fde0 0x7fd7f813b140 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:20.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.351+0000 7fd800121700 1 --2- 172.21.15.36:0/3132069572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7f80ae270 0x7fd7f813ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:20.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.351+0000 7fd800121700 1 --2- 172.21.15.36:0/3132069572 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7f80ad9c0 0x7fd7f8145a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:20.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.351+0000 7fd800121700 1 -- 172.21.15.36:0/3132069572 >> 172.21.15.36:0/3132069572 conn(0x7fd7f805c550 msgr2=0x7fd7f80b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:20.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.351+0000 7fd800121700 1 -- 172.21.15.36:0/3132069572 shutdown_connections 2022-06-17T12:55:20.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.351+0000 7fd800121700 1 -- 172.21.15.36:0/3132069572 wait complete. 2022-06-17T12:55:20.346 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 56 2022-06-17T12:55:20.350 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-17T12:55:20.350 INFO:tasks.ceph.mds.a:Stopping old one... 2022-06-17T12:55:20.351 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-17T12:55:20.351 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T12:55:20.353 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-17T12:55:20.353 INFO:tasks.ceph.mds.b:Stopping old one... 2022-06-17T12:55:20.353 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-17T12:55:20.354 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T12:55:20.366 INFO:tasks.ceph.mds.a:Stopped 2022-06-17T12:55:20.367 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-17T12:55:20.370 INFO:tasks.ceph.mds.a:Started 2022-06-17T12:55:20.371 DEBUG:teuthology.parallel:result is None 2022-06-17T12:55:20.374 INFO:tasks.ceph.mds.b:Stopped 2022-06-17T12:55:20.374 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-17T12:55:20.378 INFO:tasks.ceph.mds.b:Started 2022-06-17T12:55:20.378 DEBUG:teuthology.parallel:result is None 2022-06-17T12:55:20.379 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-06-17T12:55:20.443 INFO:tasks.ceph.mds.a.smithi036.stdout:starting mds.a at 2022-06-17T12:55:20.456 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2022-06-17T12:55:20.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.503+0000 7fcc7b0ec700 1 Processor -- start 2022-06-17T12:55:20.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.503+0000 7fcc7b0ec700 1 -- start start 2022-06-17T12:55:20.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.503+0000 7fcc7b0ec700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcc740b2200 0x7fcc740af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:20.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.503+0000 7fcc7b0ec700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcc740b2cd0 0x7fcc74147600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:20.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.503+0000 7fcc7b0ec700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc740b25e0 con 0x7fcc740af8d0 2022-06-17T12:55:20.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.503+0000 7fcc7b0ec700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc740ad790 con 0x7fcc740b2cd0 2022-06-17T12:55:20.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.503+0000 7fcc7b0ec700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc740aef40 con 0x7fcc740b2200 2022-06-17T12:55:20.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.503+0000 7fcc7a8eb700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcc740b2cd0 0x7fcc74147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:20.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.503+0000 7fcc7a8eb700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcc740b2cd0 0x7fcc74147600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38276/0 (socket says 172.21.15.36:38276) 2022-06-17T12:55:20.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.503+0000 7fcc7a8eb700 1 -- 172.21.15.36:0/2140986717 learned_addr learned my addr 172.21.15.36:0/2140986717 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:20.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.503+0000 7fcc7a0ea700 1 --2- 172.21.15.36:0/2140986717 >> v2:172.21.15.70:3300/0 conn(0x7fcc740b2200 0x7fcc740af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:20.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.507+0000 7fcc7a8eb700 1 -- 172.21.15.36:0/2140986717 >> v2:172.21.15.70:3300/0 conn(0x7fcc740b2200 msgr2=0x7fcc740af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:20.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.507+0000 7fcc7a8eb700 1 --2- 172.21.15.36:0/2140986717 >> v2:172.21.15.70:3300/0 conn(0x7fcc740b2200 0x7fcc740af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:20.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.507+0000 7fcc7a8eb700 1 -- 172.21.15.36:0/2140986717 >> v1:172.21.15.36:6789/0 conn(0x7fcc740af8d0 legacy=0x7fcc7414a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:20.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.507+0000 7fcc7a8eb700 1 -- 172.21.15.36:0/2140986717 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc740b1950 con 0x7fcc740b2cd0 2022-06-17T12:55:20.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.507+0000 7fcc7a8eb700 1 --2- 172.21.15.36:0/2140986717 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcc740b2cd0 0x7fcc74147600 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7fcc6c0032f0 tx=0x7fcc6c009290).ready entity=mon.0 client_cookie=79f9ef3727aa51e8 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:20.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.507+0000 7fcc790e8700 1 -- 172.21.15.36:0/2140986717 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc6c00b870 con 0x7fcc740b2cd0 2022-06-17T12:55:20.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.507+0000 7fcc790e8700 1 -- 172.21.15.36:0/2140986717 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6c00b9d0 con 0x7fcc740b2cd0 2022-06-17T12:55:20.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.507+0000 7fcc790e8700 1 -- 172.21.15.36:0/2140986717 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc6c00bb40 con 0x7fcc740b2cd0 2022-06-17T12:55:20.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.507+0000 7fcc7b0ec700 1 -- 172.21.15.36:0/2140986717 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcc740b2cd0 msgr2=0x7fcc74147600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:20.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.507+0000 7fcc7b0ec700 1 --2- 172.21.15.36:0/2140986717 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcc740b2cd0 0x7fcc74147600 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7fcc6c0032f0 tx=0x7fcc6c009290).stop 2022-06-17T12:55:20.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.507+0000 7fcc7b0ec700 1 -- 172.21.15.36:0/2140986717 shutdown_connections 2022-06-17T12:55:20.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.507+0000 7fcc7b0ec700 1 --2- 172.21.15.36:0/2140986717 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcc740b2cd0 0x7fcc74147600 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:20.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.507+0000 7fcc7b0ec700 1 --2- 172.21.15.36:0/2140986717 >> v2:172.21.15.70:3300/0 conn(0x7fcc740b2200 0x7fcc740af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:20.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.507+0000 7fcc7b0ec700 1 -- 172.21.15.36:0/2140986717 >> 172.21.15.36:0/2140986717 conn(0x7fcc7405f770 msgr2=0x7fcc7405fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:20.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.507+0000 7fcc7b0ec700 1 -- 172.21.15.36:0/2140986717 shutdown_connections 2022-06-17T12:55:20.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.507+0000 7fcc7b0ec700 1 -- 172.21.15.36:0/2140986717 wait complete. 2022-06-17T12:55:20.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.507+0000 7fcc7b0ec700 1 Processor -- start 2022-06-17T12:55:20.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.507+0000 7fcc7b0ec700 1 -- start start 2022-06-17T12:55:20.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.507+0000 7fcc7b0ec700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc740af8d0 0x7fcc740626d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:20.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.507+0000 7fcc7b0ec700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcc740b2200 0x7fcc7413fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:20.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.507+0000 7fcc7b0ec700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc740b2cd0 0x7fcc74140170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:20.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.507+0000 7fcc7b0ec700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc74144500 con 0x7fcc740b2200 2022-06-17T12:55:20.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.507+0000 7fcc7b0ec700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc74143fc0 con 0x7fcc740af8d0 2022-06-17T12:55:20.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.507+0000 7fcc7b0ec700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc74144260 con 0x7fcc740b2cd0 2022-06-17T12:55:20.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.511+0000 7fcc798e9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcc740b2200 0x7fcc7413fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:20.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.511+0000 7fcc798e9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcc740b2200 0x7fcc7413fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38284/0 (socket says 172.21.15.36:38284) 2022-06-17T12:55:20.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.511+0000 7fcc798e9700 1 -- 172.21.15.36:0/1150185619 learned_addr learned my addr 172.21.15.36:0/1150185619 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:20.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.511+0000 7fcc7a0ea700 1 --2- 172.21.15.36:0/1150185619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc740af8d0 0x7fcc740626d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:20.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.511+0000 7fcc7a8eb700 1 --2- 172.21.15.36:0/1150185619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc740b2cd0 0x7fcc74140170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:20.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.511+0000 7fcc798e9700 1 -- 172.21.15.36:0/1150185619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc740b2cd0 msgr2=0x7fcc74140170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:20.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.511+0000 7fcc798e9700 1 --2- 172.21.15.36:0/1150185619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc740b2cd0 0x7fcc74140170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:20.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.511+0000 7fcc798e9700 1 -- 172.21.15.36:0/1150185619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc740af8d0 msgr2=0x7fcc740626d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:20.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.511+0000 7fcc798e9700 1 --2- 172.21.15.36:0/1150185619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc740af8d0 0x7fcc740626d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:20.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.511+0000 7fcc798e9700 1 -- 172.21.15.36:0/1150185619 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c009530 con 0x7fcc740b2200 2022-06-17T12:55:20.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.511+0000 7fcc798e9700 1 --2- 172.21.15.36:0/1150185619 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcc740b2200 0x7fcc7413fc80 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 rx=0x7fcc7406ad20 tx=0x7fcc64007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:20.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.511+0000 7fcc637fe700 1 -- 172.21.15.36:0/1150185619 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc64012400 con 0x7fcc740b2200 2022-06-17T12:55:20.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.511+0000 7fcc637fe700 1 -- 172.21.15.36:0/1150185619 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc64012560 con 0x7fcc740b2200 2022-06-17T12:55:20.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.511+0000 7fcc7b0ec700 1 -- 172.21.15.36:0/1150185619 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc74140c20 con 0x7fcc740b2200 2022-06-17T12:55:20.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.511+0000 7fcc637fe700 1 -- 172.21.15.36:0/1150185619 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc640128c0 con 0x7fcc740b2200 2022-06-17T12:55:20.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.511+0000 7fcc7b0ec700 1 -- 172.21.15.36:0/1150185619 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc74145c70 con 0x7fcc740b2200 2022-06-17T12:55:20.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.511+0000 7fcc7b0ec700 1 -- 172.21.15.36:0/1150185619 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc74140a60 con 0x7fcc740b2200 2022-06-17T12:55:20.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.511+0000 7fcc637fe700 1 -- 172.21.15.36:0/1150185619 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fcc64010070 con 0x7fcc740b2200 2022-06-17T12:55:20.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.511+0000 7fcc637fe700 1 --2- 172.21.15.36:0/1150185619 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcc54053ad0 0x7fcc54055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:20.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.511+0000 7fcc637fe700 1 -- 172.21.15.36:0/1150185619 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fcc6406acf0 con 0x7fcc740b2200 2022-06-17T12:55:20.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.511+0000 7fcc7a0ea700 1 --2- 172.21.15.36:0/1150185619 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcc54053ad0 0x7fcc54055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:20.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.511+0000 7fcc7a0ea700 1 --2- 172.21.15.36:0/1150185619 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcc54053ad0 0x7fcc54055f90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7fcc7000b000 tx=0x7fcc70006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:20.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.515+0000 7fcc637fe700 1 -- 172.21.15.36:0/1150185619 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcc64041dd0 con 0x7fcc740b2200 2022-06-17T12:55:20.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:20.651+0000 7fcc7b0ec700 1 -- 172.21.15.36:0/1150185619 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fcc740ad330 con 0x7fcc740b2200 2022-06-17T12:55:21.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.567+0000 7fcc637fe700 1 -- 172.21.15.36:0/1150185619 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v109) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcc640453f0 con 0x7fcc740b2200 2022-06-17T12:55:21.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.571+0000 7fcc7b0ec700 1 -- 172.21.15.36:0/1150185619 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcc54053ad0 msgr2=0x7fcc54055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:21.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.571+0000 7fcc7b0ec700 1 --2- 172.21.15.36:0/1150185619 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcc54053ad0 0x7fcc54055f90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7fcc7000b000 tx=0x7fcc70006df0).stop 2022-06-17T12:55:21.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.571+0000 7fcc7b0ec700 1 -- 172.21.15.36:0/1150185619 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcc740b2200 msgr2=0x7fcc7413fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:21.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.571+0000 7fcc7b0ec700 1 --2- 172.21.15.36:0/1150185619 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcc740b2200 0x7fcc7413fc80 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 rx=0x7fcc7406ad20 tx=0x7fcc64007c50).stop 2022-06-17T12:55:21.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.571+0000 7fcc7b0ec700 1 -- 172.21.15.36:0/1150185619 shutdown_connections 2022-06-17T12:55:21.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.571+0000 7fcc7b0ec700 1 --2- 172.21.15.36:0/1150185619 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcc54053ad0 0x7fcc54055f90 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:21.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.571+0000 7fcc7b0ec700 1 --2- 172.21.15.36:0/1150185619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc740b2cd0 0x7fcc74140170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:21.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.571+0000 7fcc7b0ec700 1 --2- 172.21.15.36:0/1150185619 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcc740b2200 0x7fcc7413fc80 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:21.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.571+0000 7fcc7b0ec700 1 --2- 172.21.15.36:0/1150185619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc740af8d0 0x7fcc740626d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:21.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.571+0000 7fcc7b0ec700 1 -- 172.21.15.36:0/1150185619 >> 172.21.15.36:0/1150185619 conn(0x7fcc7405f770 msgr2=0x7fcc740b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:21.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.571+0000 7fcc7b0ec700 1 -- 172.21.15.36:0/1150185619 shutdown_connections 2022-06-17T12:55:21.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.571+0000 7fcc7b0ec700 1 -- 172.21.15.36:0/1150185619 wait complete. 2022-06-17T12:55:21.566 INFO:teuthology.orchestra.run.smithi036.stderr: removed all blacklist entries 2022-06-17T12:55:21.578 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2022-06-17T12:55:21.579 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-06-17T12:55:21.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.703+0000 7f673759e700 1 Processor -- start 2022-06-17T12:55:21.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.703+0000 7f673759e700 1 -- start start 2022-06-17T12:55:21.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.703+0000 7f673759e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67380b5be0 0x7f6738144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:21.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.703+0000 7f673759e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67380b2c80 con 0x7f6738147270 2022-06-17T12:55:21.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.703+0000 7f673759e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67380605a0 con 0x7f67380601c0 2022-06-17T12:55:21.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.703+0000 7f673759e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67380b72d0 con 0x7f67380b5be0 2022-06-17T12:55:21.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.703+0000 7f6736d9d700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f6738147270 0x7f673814ab80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55824/0 (socket says 172.21.15.36:55824) 2022-06-17T12:55:21.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.703+0000 7f6736d9d700 1 -- 172.21.15.36:0/2869208442 learned_addr learned my addr 172.21.15.36:0/2869208442 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:21.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.703+0000 7f6735d9b700 1 --2- 172.21.15.36:0/2869208442 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67380b5be0 0x7f6738144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:21.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.703+0000 7f673559a700 1 -- 172.21.15.36:0/2869208442 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1943485814 0 0) 0x7f67380b2c80 con 0x7f6738147270 2022-06-17T12:55:21.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.703+0000 7f673559a700 1 -- 172.21.15.36:0/2869208442 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f671c003260 con 0x7f6738147270 2022-06-17T12:55:21.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.703+0000 7f673559a700 1 -- 172.21.15.36:0/2869208442 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1607696787 0 0) 0x7f67380605a0 con 0x7f67380601c0 2022-06-17T12:55:21.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.703+0000 7f673559a700 1 -- 172.21.15.36:0/2869208442 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67380b2c80 con 0x7f67380601c0 2022-06-17T12:55:21.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.703+0000 7f6735d9b700 1 -- 172.21.15.36:0/2869208442 >> v1:172.21.15.70:6789/0 conn(0x7f67380601c0 legacy=0x7f6738144270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:21.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.703+0000 7f6735d9b700 1 -- 172.21.15.36:0/2869208442 >> v1:172.21.15.36:6789/0 conn(0x7f6738147270 legacy=0x7f673814ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:21.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.703+0000 7f6735d9b700 1 -- 172.21.15.36:0/2869208442 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67380b29e0 con 0x7f67380b5be0 2022-06-17T12:55:21.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.703+0000 7f6735d9b700 1 --2- 172.21.15.36:0/2869208442 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67380b5be0 0x7f6738144980 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f672c0032f0 tx=0x7f672c0091d0).ready entity=mon.2 client_cookie=90a2c61d1a707b49 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:21.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.707+0000 7f673559a700 1 -- 172.21.15.36:0/2869208442 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f672c00b640 con 0x7f67380b5be0 2022-06-17T12:55:21.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.707+0000 7f673559a700 1 -- 172.21.15.36:0/2869208442 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672c00b7a0 con 0x7f67380b5be0 2022-06-17T12:55:21.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.707+0000 7f673559a700 1 -- 172.21.15.36:0/2869208442 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f672c00b910 con 0x7f67380b5be0 2022-06-17T12:55:21.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.707+0000 7f673759e700 1 -- 172.21.15.36:0/2869208442 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67380b5be0 msgr2=0x7f6738144980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:21.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.707+0000 7f673759e700 1 --2- 172.21.15.36:0/2869208442 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67380b5be0 0x7f6738144980 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f672c0032f0 tx=0x7f672c0091d0).stop 2022-06-17T12:55:21.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.707+0000 7f673759e700 1 -- 172.21.15.36:0/2869208442 shutdown_connections 2022-06-17T12:55:21.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.707+0000 7f673759e700 1 --2- 172.21.15.36:0/2869208442 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67380b5be0 0x7f6738144980 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:21.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.707+0000 7f673759e700 1 -- 172.21.15.36:0/2869208442 >> 172.21.15.36:0/2869208442 conn(0x7f673805c550 msgr2=0x7f673805f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:21.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.707+0000 7f673759e700 1 -- 172.21.15.36:0/2869208442 shutdown_connections 2022-06-17T12:55:21.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.707+0000 7f673759e700 1 -- 172.21.15.36:0/2869208442 wait complete. 2022-06-17T12:55:21.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.707+0000 7f673759e700 1 Processor -- start 2022-06-17T12:55:21.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.707+0000 7f673759e700 1 -- start start 2022-06-17T12:55:21.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.707+0000 7f673759e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67380601c0 0x7f673814e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:21.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.707+0000 7f673759e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67380b5be0 0x7f67381571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:21.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.707+0000 7f673759e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6738147270 0x7f67380ad330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:21.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.707+0000 7f673759e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6738156800 con 0x7f67380b5be0 2022-06-17T12:55:21.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.707+0000 7f673759e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6738156aa0 con 0x7f67380601c0 2022-06-17T12:55:21.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.707+0000 7f673759e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6738156d40 con 0x7f6738147270 2022-06-17T12:55:21.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.707+0000 7f6735d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67380b5be0 0x7f67381571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:21.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.707+0000 7f6735d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67380b5be0 0x7f67381571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38296/0 (socket says 172.21.15.36:38296) 2022-06-17T12:55:21.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.707+0000 7f6735d9b700 1 -- 172.21.15.36:0/2137397656 learned_addr learned my addr 172.21.15.36:0/2137397656 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:21.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.707+0000 7f673659c700 1 --2- 172.21.15.36:0/2137397656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67380601c0 0x7f673814e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:21.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.711+0000 7f6736d9d700 1 --2- 172.21.15.36:0/2137397656 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6738147270 0x7f67380ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:21.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.711+0000 7f6735d9b700 1 -- 172.21.15.36:0/2137397656 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6738147270 msgr2=0x7f67380ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:21.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.711+0000 7f6735d9b700 1 --2- 172.21.15.36:0/2137397656 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6738147270 0x7f67380ad330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:21.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.711+0000 7f6735d9b700 1 -- 172.21.15.36:0/2137397656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67380601c0 msgr2=0x7f673814e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:21.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.711+0000 7f6735d9b700 1 --2- 172.21.15.36:0/2137397656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67380601c0 0x7f673814e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:21.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.711+0000 7f6735d9b700 1 -- 172.21.15.36:0/2137397656 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672c009470 con 0x7f67380b5be0 2022-06-17T12:55:21.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.711+0000 7f673659c700 1 --2- 172.21.15.36:0/2137397656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67380601c0 0x7f673814e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:21.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.711+0000 7f6735d9b700 1 --2- 172.21.15.36:0/2137397656 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67380b5be0 0x7f67381571e0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7f672c0032f0 tx=0x7f672c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:21.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.711+0000 7f67277fe700 1 -- 172.21.15.36:0/2137397656 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f672c00b910 con 0x7f67380b5be0 2022-06-17T12:55:21.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.711+0000 7f67277fe700 1 -- 172.21.15.36:0/2137397656 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672c01f5e0 con 0x7f67380b5be0 2022-06-17T12:55:21.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.711+0000 7f673759e700 1 -- 172.21.15.36:0/2137397656 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6738156490 con 0x7f67380b5be0 2022-06-17T12:55:21.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.711+0000 7f67277fe700 1 -- 172.21.15.36:0/2137397656 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f672c00b640 con 0x7f67380b5be0 2022-06-17T12:55:21.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.711+0000 7f673759e700 1 -- 172.21.15.36:0/2137397656 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6738144c40 con 0x7f67380b5be0 2022-06-17T12:55:21.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.711+0000 7f673759e700 1 -- 172.21.15.36:0/2137397656 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6738159da0 con 0x7f67380b5be0 2022-06-17T12:55:21.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.711+0000 7f67277fe700 1 -- 172.21.15.36:0/2137397656 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f672c00f070 con 0x7f67380b5be0 2022-06-17T12:55:21.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.711+0000 7f67277fe700 1 --2- 172.21.15.36:0/2137397656 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6714053920 0x7f6714055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:21.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.711+0000 7f67277fe700 1 -- 172.21.15.36:0/2137397656 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f672c06c850 con 0x7f67380b5be0 2022-06-17T12:55:21.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.711+0000 7f673659c700 1 --2- 172.21.15.36:0/2137397656 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6714053920 0x7f6714055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:21.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.711+0000 7f673659c700 1 --2- 172.21.15.36:0/2137397656 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6714053920 0x7f6714055de0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f672800b000 tx=0x7f6728009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:21.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.719+0000 7f67277fe700 1 -- 172.21.15.36:0/2137397656 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6738159da0 con 0x7f67380b5be0 2022-06-17T12:55:21.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.887+0000 7f673759e700 1 -- 172.21.15.36:0/2137397656 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6738159da0 con 0x7f67380b5be0 2022-06-17T12:55:21.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.887+0000 7f67277fe700 1 -- 172.21.15.36:0/2137397656 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v37) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f6738159da0 con 0x7f67380b5be0 2022-06-17T12:55:21.876 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:21.876 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:55:21.877 INFO:teuthology.orchestra.run.smithi036.stdout: "auth_dump": [ 2022-06-17T12:55:21.877 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:21.877 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mds.a", 2022-06-17T12:55:21.877 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiyaUgARAAxp7bzOl4uVo2EsaoRBeMRw==", 2022-06-17T12:55:21.878 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:55:21.879 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:55:21.879 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:55:21.880 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:55:21.880 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:55:21.880 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:21.880 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:21.881 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:21.881 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mds.b", 2022-06-17T12:55:21.881 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xifyxLCRAABI389jmz1zboeDanzraT8Q==", 2022-06-17T12:55:21.882 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:55:21.882 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:55:21.882 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:55:21.883 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:55:21.883 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:55:21.883 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:21.884 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:21.884 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:21.884 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.0", 2022-06-17T12:55:21.885 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB+d6xibhXGNBAA6AuUT9ITWAZjDFpNsgA4Wg==", 2022-06-17T12:55:21.885 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:55:21.885 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:55:21.885 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:55:21.885 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:55:21.886 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:21.886 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:21.886 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:21.886 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.1", 2022-06-17T12:55:21.887 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCBd6xiOIPvGxAADkrBoJCy9x91zan6TWU0tQ==", 2022-06-17T12:55:21.887 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:55:21.887 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:55:21.888 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:55:21.888 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:55:21.888 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:21.888 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:21.888 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:21.889 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.2", 2022-06-17T12:55:21.889 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCEd6xiP4/9AhAAaP3cWXV6vbfGBPfqSLKqUg==", 2022-06-17T12:55:21.889 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:55:21.889 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:55:21.890 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:55:21.890 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:55:21.890 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:21.890 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:21.891 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:21.891 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.3", 2022-06-17T12:55:21.891 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCGd6xi+LZgJxAAJjGwP5qW6TFDZfR+2rz6ig==", 2022-06-17T12:55:21.891 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:55:21.892 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:55:21.892 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:55:21.892 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:55:21.892 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:21.893 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:21.893 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:21.893 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.4", 2022-06-17T12:55:21.893 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCKd6xikvrTChAAXIyWSETswDv+tb3zsGyYpA==", 2022-06-17T12:55:21.894 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:55:21.894 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:55:21.894 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:55:21.894 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:55:21.895 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:21.895 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:21.895 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:21.895 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.5", 2022-06-17T12:55:21.895 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCMd6xiqr2QLhAArhimHuNsVWrDGGreFYxj6A==", 2022-06-17T12:55:21.896 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:55:21.896 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:55:21.897 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:55:21.898 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:55:21.898 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:21.898 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:21.899 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:21.899 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.6", 2022-06-17T12:55:21.899 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCPd6xiUXcXGxAAdrYz8txdBWMIqF5DF4Aimw==", 2022-06-17T12:55:21.899 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:55:21.900 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:55:21.900 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:55:21.900 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:55:21.900 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:21.901 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:21.901 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:21.901 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.7", 2022-06-17T12:55:21.901 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCSd6xiBJ2IAhAAUkNfgmlxxWY7+VouPLAvyw==", 2022-06-17T12:55:21.902 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:55:21.902 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:55:21.902 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:55:21.902 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:55:21.903 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:21.903 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:21.903 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:21.903 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.0", 2022-06-17T12:55:21.903 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiXO3QFBAAsSLtQ0yKKZxhqUT/eYKdtg==", 2022-06-17T12:55:21.904 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:55:21.904 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:55:21.904 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:55:21.904 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:55:21.905 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:21.905 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:21.905 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:21.905 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.1", 2022-06-17T12:55:21.905 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xi51KdGBAAi7fGZ+Qepg6j3DAhlzoWxw==", 2022-06-17T12:55:21.906 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:55:21.906 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:55:21.906 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:55:21.906 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:55:21.906 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:21.907 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:21.907 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:21.907 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.2", 2022-06-17T12:55:21.907 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiug2hHBAAqHpLgwMdthSWtJ8Vwu+m1Q==", 2022-06-17T12:55:21.908 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:55:21.908 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:55:21.908 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:55:21.908 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:55:21.909 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:21.909 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:21.909 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:21.909 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.admin", 2022-06-17T12:55:21.911 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB5d6xiGDODKhAAPZMG/M2nVssdr4FJHcS6XQ==", 2022-06-17T12:55:21.911 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:55:21.911 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:55:21.911 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:55:21.912 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:55:21.912 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:55:21.912 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:21.912 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:21.913 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:21.913 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mgr.x", 2022-06-17T12:55:21.913 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB6d6xiyH9uORAATJb0chHE1CH0X02w7edVtw==", 2022-06-17T12:55:21.913 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:55:21.914 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:55:21.914 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile mgr", 2022-06-17T12:55:21.914 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:55:21.914 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:21.915 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:21.915 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:21.915 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mgr.y", 2022-06-17T12:55:21.915 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB6d6xiHXghMxAAzpsVRCbLWHWZPtrPk1jFlA==", 2022-06-17T12:55:21.916 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:55:21.916 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:55:21.916 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile mgr", 2022-06-17T12:55:21.916 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:55:21.917 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:21.917 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:21.917 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:21.917 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:55:21.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.891+0000 7f673759e700 1 -- 172.21.15.36:0/2137397656 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6714053920 msgr2=0x7f6714055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:21.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.891+0000 7f673759e700 1 --2- 172.21.15.36:0/2137397656 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6714053920 0x7f6714055de0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f672800b000 tx=0x7f6728009040).stop 2022-06-17T12:55:21.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.891+0000 7f673759e700 1 -- 172.21.15.36:0/2137397656 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67380b5be0 msgr2=0x7f67381571e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:21.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.891+0000 7f673759e700 1 --2- 172.21.15.36:0/2137397656 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67380b5be0 0x7f67381571e0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7f672c0032f0 tx=0x7f672c016000).stop 2022-06-17T12:55:21.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.891+0000 7f673759e700 1 -- 172.21.15.36:0/2137397656 shutdown_connections 2022-06-17T12:55:21.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.891+0000 7f673759e700 1 --2- 172.21.15.36:0/2137397656 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6714053920 0x7f6714055de0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:21.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.891+0000 7f673759e700 1 --2- 172.21.15.36:0/2137397656 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6738147270 0x7f67380ad330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:21.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.891+0000 7f673759e700 1 --2- 172.21.15.36:0/2137397656 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67380b5be0 0x7f67381571e0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:21.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.891+0000 7f673759e700 1 --2- 172.21.15.36:0/2137397656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67380601c0 0x7f673814e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:21.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.891+0000 7f673759e700 1 -- 172.21.15.36:0/2137397656 >> 172.21.15.36:0/2137397656 conn(0x7f673805c550 msgr2=0x7f673814ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:21.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.891+0000 7f673759e700 1 -- 172.21.15.36:0/2137397656 shutdown_connections 2022-06-17T12:55:21.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:21.891+0000 7f673759e700 1 -- 172.21.15.36:0/2137397656 wait complete. 2022-06-17T12:55:21.923 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:55:22.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.039+0000 7f700e820700 1 Processor -- start 2022-06-17T12:55:22.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.039+0000 7f700e820700 1 -- start start 2022-06-17T12:55:22.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.039+0000 7f700e820700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7008066bc0 0x7f7008143880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.039+0000 7f700e820700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f70080afe30 0x7f7008143d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.039+0000 7f700e820700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70080b1570 con 0x7f7008067510 2022-06-17T12:55:22.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.039+0000 7f700e820700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70080acb10 con 0x7f7008066bc0 2022-06-17T12:55:22.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.039+0000 7f700e820700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7008066fa0 con 0x7f70080afe30 2022-06-17T12:55:22.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.039+0000 7f700d81e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7008066bc0 0x7f7008143880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.043+0000 7f700d81e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7008066bc0 0x7f7008143880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38308/0 (socket says 172.21.15.36:38308) 2022-06-17T12:55:22.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.043+0000 7f700d81e700 1 -- 172.21.15.36:0/380329324 learned_addr learned my addr 172.21.15.36:0/380329324 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:22.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.043+0000 7f700d01d700 1 --2- 172.21.15.36:0/380329324 >> v2:172.21.15.70:3300/0 conn(0x7f70080afe30 0x7f7008143d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.043+0000 7f700d01d700 1 -- 172.21.15.36:0/380329324 >> v2:172.21.15.36:3300/0 conn(0x7f7008066bc0 msgr2=0x7f7008143880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:22.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.043+0000 7f700d01d700 1 --2- 172.21.15.36:0/380329324 >> v2:172.21.15.36:3300/0 conn(0x7f7008066bc0 0x7f7008143880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.043+0000 7f700d01d700 1 -- 172.21.15.36:0/380329324 >> v1:172.21.15.70:6789/0 conn(0x7f7008067510 legacy=0x7f70080addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:22.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.043+0000 7f700d01d700 1 -- 172.21.15.36:0/380329324 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70080b17a0 con 0x7f70080afe30 2022-06-17T12:55:22.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.043+0000 7f700d01d700 1 --2- 172.21.15.36:0/380329324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70080afe30 0x7f7008143d70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f6ff8002f70 tx=0x7f6ff8006df0).ready entity=mon.1 client_cookie=89e62f70a6d4ae4 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:22.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.043+0000 7f700c81c700 1 -- 172.21.15.36:0/380329324 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ff800d570 con 0x7f70080afe30 2022-06-17T12:55:22.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.043+0000 7f700c81c700 1 -- 172.21.15.36:0/380329324 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff8000e90 con 0x7f70080afe30 2022-06-17T12:55:22.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.043+0000 7f700c81c700 1 -- 172.21.15.36:0/380329324 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ff800d8c0 con 0x7f70080afe30 2022-06-17T12:55:22.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.043+0000 7f700e820700 1 -- 172.21.15.36:0/380329324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70080afe30 msgr2=0x7f7008143d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:22.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.043+0000 7f700e820700 1 --2- 172.21.15.36:0/380329324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70080afe30 0x7f7008143d70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f6ff8002f70 tx=0x7f6ff8006df0).stop 2022-06-17T12:55:22.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.043+0000 7f700e820700 1 -- 172.21.15.36:0/380329324 shutdown_connections 2022-06-17T12:55:22.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.043+0000 7f700e820700 1 --2- 172.21.15.36:0/380329324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70080afe30 0x7f7008143d70 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.043+0000 7f700e820700 1 --2- 172.21.15.36:0/380329324 >> v2:172.21.15.36:3300/0 conn(0x7f7008066bc0 0x7f7008143880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.043+0000 7f700e820700 1 -- 172.21.15.36:0/380329324 >> 172.21.15.36:0/380329324 conn(0x7f700805f770 msgr2=0x7f700805fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:22.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.043+0000 7f700e820700 1 -- 172.21.15.36:0/380329324 shutdown_connections 2022-06-17T12:55:22.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.043+0000 7f700e820700 1 -- 172.21.15.36:0/380329324 wait complete. 2022-06-17T12:55:22.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.047+0000 7f700e820700 1 Processor -- start 2022-06-17T12:55:22.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.047+0000 7f700e820700 1 -- start start 2022-06-17T12:55:22.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.047+0000 7f700e820700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7008066bc0 0x7f70080650c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.047+0000 7f700e820700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7008067510 0x7f70080a7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.047+0000 7f700e820700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70080afe30 0x7f70080a7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.047+0000 7f700e820700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70080a9a50 con 0x7f7008067510 2022-06-17T12:55:22.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.047+0000 7f700e820700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70080a9ed0 con 0x7f7008066bc0 2022-06-17T12:55:22.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.047+0000 7f700e820700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70080a9c30 con 0x7f70080afe30 2022-06-17T12:55:22.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.047+0000 7f700d01d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7008067510 0x7f70080a7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.047+0000 7f700d01d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7008067510 0x7f70080a7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38312/0 (socket says 172.21.15.36:38312) 2022-06-17T12:55:22.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.047+0000 7f700d01d700 1 -- 172.21.15.36:0/1290189466 learned_addr learned my addr 172.21.15.36:0/1290189466 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:22.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.047+0000 7f700d81e700 1 --2- 172.21.15.36:0/1290189466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7008066bc0 0x7f70080650c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.047+0000 7f700e01f700 1 --2- 172.21.15.36:0/1290189466 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70080afe30 0x7f70080a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.047+0000 7f700d01d700 1 -- 172.21.15.36:0/1290189466 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70080afe30 msgr2=0x7f70080a7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:22.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.047+0000 7f700d01d700 1 --2- 172.21.15.36:0/1290189466 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70080afe30 0x7f70080a7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.047+0000 7f700d01d700 1 -- 172.21.15.36:0/1290189466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7008066bc0 msgr2=0x7f70080650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:22.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.047+0000 7f700d01d700 1 --2- 172.21.15.36:0/1290189466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7008066bc0 0x7f70080650c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.047+0000 7f700d01d700 1 -- 172.21.15.36:0/1290189466 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff800b180 con 0x7f7008067510 2022-06-17T12:55:22.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.047+0000 7f700d81e700 1 --2- 172.21.15.36:0/1290189466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7008066bc0 0x7f70080650c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:55:22.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.047+0000 7f700e01f700 1 --2- 172.21.15.36:0/1290189466 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70080afe30 0x7f70080a7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:55:22.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.047+0000 7f700d01d700 1 --2- 172.21.15.36:0/1290189466 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7008067510 0x7f70080a7420 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 rx=0x7f7008064de0 tx=0x7f6ff8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:22.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.047+0000 7f6ff6ffd700 1 -- 172.21.15.36:0/1290189466 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ff8016070 con 0x7f7008067510 2022-06-17T12:55:22.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.047+0000 7f6ff6ffd700 1 -- 172.21.15.36:0/1290189466 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff800d8f0 con 0x7f7008067510 2022-06-17T12:55:22.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.051+0000 7f700e820700 1 -- 172.21.15.36:0/1290189466 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7008065730 con 0x7f7008067510 2022-06-17T12:55:22.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.051+0000 7f6ff6ffd700 1 -- 172.21.15.36:0/1290189466 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ff801e450 con 0x7f7008067510 2022-06-17T12:55:22.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.051+0000 7f700e820700 1 -- 172.21.15.36:0/1290189466 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7008156790 con 0x7f7008067510 2022-06-17T12:55:22.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.051+0000 7f700e820700 1 -- 172.21.15.36:0/1290189466 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70080b0550 con 0x7f7008067510 2022-06-17T12:55:22.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.051+0000 7f6ff6ffd700 1 -- 172.21.15.36:0/1290189466 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6ff800f030 con 0x7f7008067510 2022-06-17T12:55:22.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.051+0000 7f6ff6ffd700 1 --2- 172.21.15.36:0/1290189466 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6fe8053a60 0x7f6fe8055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.051+0000 7f6ff6ffd700 1 -- 172.21.15.36:0/1290189466 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f6ff806cf20 con 0x7f7008067510 2022-06-17T12:55:22.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.051+0000 7f700d81e700 1 --2- 172.21.15.36:0/1290189466 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6fe8053a60 0x7f6fe8055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.055+0000 7f700d81e700 1 --2- 172.21.15.36:0/1290189466 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6fe8053a60 0x7f6fe8055f20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f7004008000 tx=0x7f7004008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:22.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.055+0000 7f6ff6ffd700 1 -- 172.21.15.36:0/1290189466 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6ff8043f50 con 0x7f7008067510 2022-06-17T12:55:22.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.183+0000 7f700e820700 1 -- 172.21.15.36:0/1290189466 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7008057410 con 0x7f7008067510 2022-06-17T12:55:22.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.187+0000 7f6ff6ffd700 1 -- 172.21.15.36:0/1290189466 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+13424 (secure 0 0 0) 0x7f6ff8023030 con 0x7f7008067510 2022-06-17T12:55:22.175 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:22.176 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":109,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:55:21.565803+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:55:22.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.187+0000 7f700e820700 1 -- 172.21.15.36:0/1290189466 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6fe8053a60 msgr2=0x7f6fe8055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:22.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.187+0000 7f700e820700 1 --2- 172.21.15.36:0/1290189466 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6fe8053a60 0x7f6fe8055f20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f7004008000 tx=0x7f7004008040).stop 2022-06-17T12:55:22.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.187+0000 7f700e820700 1 -- 172.21.15.36:0/1290189466 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7008067510 msgr2=0x7f70080a7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:22.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.187+0000 7f700e820700 1 --2- 172.21.15.36:0/1290189466 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7008067510 0x7f70080a7420 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 rx=0x7f7008064de0 tx=0x7f6ff8014000).stop 2022-06-17T12:55:22.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.187+0000 7f700e820700 1 -- 172.21.15.36:0/1290189466 shutdown_connections 2022-06-17T12:55:22.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.187+0000 7f700e820700 1 --2- 172.21.15.36:0/1290189466 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6fe8053a60 0x7f6fe8055f20 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.187+0000 7f700e820700 1 --2- 172.21.15.36:0/1290189466 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70080afe30 0x7f70080a7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.187+0000 7f700e820700 1 --2- 172.21.15.36:0/1290189466 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7008067510 0x7f70080a7420 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.187+0000 7f700e820700 1 --2- 172.21.15.36:0/1290189466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7008066bc0 0x7f70080650c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.187+0000 7f700e820700 1 -- 172.21.15.36:0/1290189466 >> 172.21.15.36:0/1290189466 conn(0x7f700805f770 msgr2=0x7f7008140970 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:22.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.187+0000 7f700e820700 1 -- 172.21.15.36:0/1290189466 shutdown_connections 2022-06-17T12:55:22.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.187+0000 7f700e820700 1 -- 172.21.15.36:0/1290189466 wait complete. 2022-06-17T12:55:22.188 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-17T12:55:22.189 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:55:22.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.307+0000 7f215cc09700 1 Processor -- start 2022-06-17T12:55:22.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.307+0000 7f215cc09700 1 -- start start 2022-06-17T12:55:22.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.307+0000 7f215cc09700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f21580b0d70 0x7f21580ac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.307+0000 7f215cc09700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f21580b15f0 0x7f21580acc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.307+0000 7f215cc09700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21580b2b90 con 0x7f21580b27b0 2022-06-17T12:55:22.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.307+0000 7f215cc09700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21580610e0 con 0x7f21580b15f0 2022-06-17T12:55:22.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.307+0000 7f215cc09700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21580ac190 con 0x7f21580b0d70 2022-06-17T12:55:22.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.307+0000 7f2157fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f21580b15f0 0x7f21580acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.307+0000 7f2156ffd700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f21580b0d70 0x7f21580ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.307+0000 7f2157fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f21580b15f0 0x7f21580acc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38322/0 (socket says 172.21.15.36:38322) 2022-06-17T12:55:22.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.307+0000 7f2157fff700 1 -- 172.21.15.36:0/1488541189 learned_addr learned my addr 172.21.15.36:0/1488541189 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:22.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.307+0000 7f2156ffd700 1 -- 172.21.15.36:0/1488541189 >> v2:172.21.15.36:3300/0 conn(0x7f21580b15f0 msgr2=0x7f21580acc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:22.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.307+0000 7f2156ffd700 1 --2- 172.21.15.36:0/1488541189 >> v2:172.21.15.36:3300/0 conn(0x7f21580b15f0 0x7f21580acc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.307+0000 7f2156ffd700 1 -- 172.21.15.36:0/1488541189 >> v1:172.21.15.70:6789/0 conn(0x7f21580b27b0 legacy=0x7f215813f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:22.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.307+0000 7f2156ffd700 1 -- 172.21.15.36:0/1488541189 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21580ac430 con 0x7f21580b0d70 2022-06-17T12:55:22.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.307+0000 7f2157fff700 1 --2- 172.21.15.36:0/1488541189 >> v2:172.21.15.36:3300/0 conn(0x7f21580b15f0 0x7f21580acc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:22.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.311+0000 7f2156ffd700 1 --2- 172.21.15.36:0/1488541189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21580b0d70 0x7f21580ac780 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f21480032f0 tx=0x7f21480091d0).ready entity=mon.1 client_cookie=a618d303372a3f6e server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:22.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.311+0000 7f21567fc700 1 -- 172.21.15.36:0/1488541189 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f214800f8a0 con 0x7f21580b0d70 2022-06-17T12:55:22.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.311+0000 7f21567fc700 1 -- 172.21.15.36:0/1488541189 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214800fa00 con 0x7f21580b0d70 2022-06-17T12:55:22.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.311+0000 7f21567fc700 1 -- 172.21.15.36:0/1488541189 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f214800fd60 con 0x7f21580b0d70 2022-06-17T12:55:22.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.311+0000 7f215cc09700 1 -- 172.21.15.36:0/1488541189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21580b0d70 msgr2=0x7f21580ac780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:22.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.311+0000 7f215cc09700 1 --2- 172.21.15.36:0/1488541189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21580b0d70 0x7f21580ac780 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f21480032f0 tx=0x7f21480091d0).stop 2022-06-17T12:55:22.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.311+0000 7f215cc09700 1 -- 172.21.15.36:0/1488541189 shutdown_connections 2022-06-17T12:55:22.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.311+0000 7f215cc09700 1 --2- 172.21.15.36:0/1488541189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21580b0d70 0x7f21580ac780 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.311+0000 7f215cc09700 1 --2- 172.21.15.36:0/1488541189 >> v2:172.21.15.36:3300/0 conn(0x7f21580b15f0 0x7f21580acc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.311+0000 7f215cc09700 1 -- 172.21.15.36:0/1488541189 >> 172.21.15.36:0/1488541189 conn(0x7f21580b5d10 msgr2=0x7f21580b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:22.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.311+0000 7f215cc09700 1 -- 172.21.15.36:0/1488541189 shutdown_connections 2022-06-17T12:55:22.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.311+0000 7f215cc09700 1 -- 172.21.15.36:0/1488541189 wait complete. 2022-06-17T12:55:22.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f215cc09700 1 Processor -- start 2022-06-17T12:55:22.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f215cc09700 1 -- start start 2022-06-17T12:55:22.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f215cc09700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21580b0d70 0x7f215813b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f215cc09700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f21580b15f0 0x7f2158139630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f215cc09700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21580b27b0 0x7f2158151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f215cc09700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2158062710 con 0x7f21580b15f0 2022-06-17T12:55:22.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f215cc09700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215813bbd0 con 0x7f21580b0d70 2022-06-17T12:55:22.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f2156ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f21580b15f0 0x7f2158139630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f215cc09700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2158062940 con 0x7f21580b27b0 2022-06-17T12:55:22.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f2156ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f21580b15f0 0x7f2158139630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38326/0 (socket says 172.21.15.36:38326) 2022-06-17T12:55:22.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f2156ffd700 1 -- 172.21.15.36:0/914152868 learned_addr learned my addr 172.21.15.36:0/914152868 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:22.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f21577fe700 1 --2- 172.21.15.36:0/914152868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21580b0d70 0x7f215813b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f2157fff700 1 --2- 172.21.15.36:0/914152868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21580b27b0 0x7f2158151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f2156ffd700 1 -- 172.21.15.36:0/914152868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21580b27b0 msgr2=0x7f2158151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:22.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f2156ffd700 1 --2- 172.21.15.36:0/914152868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21580b27b0 0x7f2158151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f2156ffd700 1 -- 172.21.15.36:0/914152868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21580b0d70 msgr2=0x7f215813b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:22.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f2156ffd700 1 --2- 172.21.15.36:0/914152868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21580b0d70 0x7f215813b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f2156ffd700 1 -- 172.21.15.36:0/914152868 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215000b1e0 con 0x7f21580b15f0 2022-06-17T12:55:22.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f21577fe700 1 --2- 172.21.15.36:0/914152868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21580b0d70 0x7f215813b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:22.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f2156ffd700 1 --2- 172.21.15.36:0/914152868 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f21580b15f0 0x7f2158139630 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f21480032f0 tx=0x7f2148018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:22.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f2154ff9700 1 -- 172.21.15.36:0/914152868 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2148021070 con 0x7f21580b15f0 2022-06-17T12:55:22.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f2154ff9700 1 -- 172.21.15.36:0/914152868 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2148010870 con 0x7f21580b15f0 2022-06-17T12:55:22.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f215cc09700 1 -- 172.21.15.36:0/914152868 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2148009470 con 0x7f21580b15f0 2022-06-17T12:55:22.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f2154ff9700 1 -- 172.21.15.36:0/914152868 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f214801cea0 con 0x7f21580b15f0 2022-06-17T12:55:22.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.315+0000 7f215cc09700 1 -- 172.21.15.36:0/914152868 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2158151a10 con 0x7f21580b15f0 2022-06-17T12:55:22.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.319+0000 7f215cc09700 1 -- 172.21.15.36:0/914152868 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21581407b0 con 0x7f21580b15f0 2022-06-17T12:55:22.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.319+0000 7f2154ff9700 1 -- 172.21.15.36:0/914152868 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2148010bd0 con 0x7f21580b15f0 2022-06-17T12:55:22.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.319+0000 7f2154ff9700 1 --2- 172.21.15.36:0/914152868 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f213c0539d0 0x7f213c055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.319+0000 7f21577fe700 1 --2- 172.21.15.36:0/914152868 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f213c0539d0 0x7f213c055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.319+0000 7f2154ff9700 1 -- 172.21.15.36:0/914152868 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f2148071b70 con 0x7f21580b15f0 2022-06-17T12:55:22.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.319+0000 7f21577fe700 1 --2- 172.21.15.36:0/914152868 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f213c0539d0 0x7f213c055e90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f214000b000 tx=0x7f2140009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:22.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.327+0000 7f2154ff9700 1 -- 172.21.15.36:0/914152868 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2148048be0 con 0x7f21580b15f0 2022-06-17T12:55:22.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.455+0000 7f215cc09700 1 -- 172.21.15.36:0/914152868 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f2158057410 con 0x7f21580b15f0 2022-06-17T12:55:22.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.455+0000 7f2154ff9700 1 -- 172.21.15.36:0/914152868 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v109) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2148006d00 con 0x7f21580b15f0 2022-06-17T12:55:22.443 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:55:22.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.455+0000 7f215cc09700 1 -- 172.21.15.36:0/914152868 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f213c0539d0 msgr2=0x7f213c055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:22.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.455+0000 7f215cc09700 1 --2- 172.21.15.36:0/914152868 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f213c0539d0 0x7f213c055e90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f214000b000 tx=0x7f2140009040).stop 2022-06-17T12:55:22.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.455+0000 7f215cc09700 1 -- 172.21.15.36:0/914152868 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f21580b15f0 msgr2=0x7f2158139630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:22.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.455+0000 7f215cc09700 1 --2- 172.21.15.36:0/914152868 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f21580b15f0 0x7f2158139630 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f21480032f0 tx=0x7f2148018000).stop 2022-06-17T12:55:22.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.455+0000 7f215cc09700 1 -- 172.21.15.36:0/914152868 shutdown_connections 2022-06-17T12:55:22.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.455+0000 7f215cc09700 1 --2- 172.21.15.36:0/914152868 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f213c0539d0 0x7f213c055e90 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.455+0000 7f215cc09700 1 --2- 172.21.15.36:0/914152868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21580b27b0 0x7f2158151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.455+0000 7f215cc09700 1 --2- 172.21.15.36:0/914152868 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f21580b15f0 0x7f2158139630 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.455+0000 7f215cc09700 1 --2- 172.21.15.36:0/914152868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21580b0d70 0x7f215813b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.455+0000 7f215cc09700 1 -- 172.21.15.36:0/914152868 >> 172.21.15.36:0/914152868 conn(0x7f21580b5d10 msgr2=0x7f21581442e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:22.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.455+0000 7f215cc09700 1 -- 172.21.15.36:0/914152868 shutdown_connections 2022-06-17T12:55:22.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.455+0000 7f215cc09700 1 -- 172.21.15.36:0/914152868 wait complete. 2022-06-17T12:55:22.457 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:55:22.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faedef8d700 1 Processor -- start 2022-06-17T12:55:22.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faedef8d700 1 -- start start 2022-06-17T12:55:22.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faedef8d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faed80b7bc0 0x7faed8140e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faedef8d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7faed80ae150 0x7faed80b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faedef8d700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed80b7fa0 con 0x7faed80b2fc0 2022-06-17T12:55:22.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faedef8d700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed80b20f0 con 0x7faed80ae150 2022-06-17T12:55:22.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faedef8d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed80b2390 con 0x7faed80b7bc0 2022-06-17T12:55:22.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faedd52a700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7faed80b2fc0 0x7faed8144450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55864/0 (socket says 172.21.15.36:55864) 2022-06-17T12:55:22.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faed7fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7faed80ae150 0x7faed80b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faedd52a700 1 -- 172.21.15.36:0/1645979029 learned_addr learned my addr 172.21.15.36:0/1645979029 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:22.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faedcd29700 1 --2- 172.21.15.36:0/1645979029 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faed80b7bc0 0x7faed8140e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faed77fe700 1 -- 172.21.15.36:0/1645979029 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507928253 0 0) 0x7faed80b7fa0 con 0x7faed80b2fc0 2022-06-17T12:55:22.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faed77fe700 1 -- 172.21.15.36:0/1645979029 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faeb8003260 con 0x7faed80b2fc0 2022-06-17T12:55:22.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faed7fff700 1 -- 172.21.15.36:0/1645979029 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faed80b7bc0 msgr2=0x7faed8140e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:22.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faed77fe700 1 -- 172.21.15.36:0/1645979029 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3151425649 0 0) 0x7faeb8003260 con 0x7faed80b2fc0 2022-06-17T12:55:22.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faed7fff700 1 --2- 172.21.15.36:0/1645979029 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faed80b7bc0 0x7faed8140e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faed7fff700 1 -- 172.21.15.36:0/1645979029 >> v1:172.21.15.36:6789/0 conn(0x7faed80b2fc0 legacy=0x7faed8144450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:22.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faed7fff700 1 -- 172.21.15.36:0/1645979029 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed80b2630 con 0x7faed80ae150 2022-06-17T12:55:22.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faedcd29700 1 --2- 172.21.15.36:0/1645979029 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faed80b7bc0 0x7faed8140e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:22.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faed7fff700 1 --2- 172.21.15.36:0/1645979029 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faed80ae150 0x7faed80b2ad0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7faec8001800 tx=0x7faec0006df0).ready entity=mon.0 client_cookie=2bbc7a68971f8bde server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:22.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faed77fe700 1 -- 172.21.15.36:0/1645979029 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faec8001660 con 0x7faed80ae150 2022-06-17T12:55:22.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faed77fe700 1 -- 172.21.15.36:0/1645979029 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec0000e90 con 0x7faed80ae150 2022-06-17T12:55:22.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faed77fe700 1 -- 172.21.15.36:0/1645979029 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faec0012460 con 0x7faed80ae150 2022-06-17T12:55:22.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faedef8d700 1 -- 172.21.15.36:0/1645979029 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faed80ae150 msgr2=0x7faed80b2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:22.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faedef8d700 1 --2- 172.21.15.36:0/1645979029 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faed80ae150 0x7faed80b2ad0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7faec8001800 tx=0x7faec0006df0).stop 2022-06-17T12:55:22.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faedef8d700 1 -- 172.21.15.36:0/1645979029 shutdown_connections 2022-06-17T12:55:22.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faedef8d700 1 --2- 172.21.15.36:0/1645979029 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faed80ae150 0x7faed80b2ad0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faedef8d700 1 --2- 172.21.15.36:0/1645979029 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faed80b7bc0 0x7faed8140e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.575+0000 7faedef8d700 1 -- 172.21.15.36:0/1645979029 >> 172.21.15.36:0/1645979029 conn(0x7faed8060990 msgr2=0x7faed805c550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:22.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.579+0000 7faedef8d700 1 -- 172.21.15.36:0/1645979029 shutdown_connections 2022-06-17T12:55:22.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.579+0000 7faedef8d700 1 -- 172.21.15.36:0/1645979029 wait complete. 2022-06-17T12:55:22.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.579+0000 7faedef8d700 1 Processor -- start 2022-06-17T12:55:22.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.579+0000 7faedef8d700 1 -- start start 2022-06-17T12:55:22.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.579+0000 7faedef8d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faed80ae150 0x7faed81538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.579+0000 7faedef8d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faed80b2fc0 0x7faed8153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.579+0000 7faedef8d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faed80b7bc0 0x7faed813ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.579+0000 7faedef8d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed81542b0 con 0x7faed80b7bc0 2022-06-17T12:55:22.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.579+0000 7faedef8d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed80b15a0 con 0x7faed80ae150 2022-06-17T12:55:22.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.579+0000 7faedef8d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed80ae9d0 con 0x7faed80b2fc0 2022-06-17T12:55:22.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.579+0000 7faedd52a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faed80b7bc0 0x7faed813ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.579+0000 7faedd52a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faed80b7bc0 0x7faed813ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38342/0 (socket says 172.21.15.36:38342) 2022-06-17T12:55:22.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.579+0000 7faedd52a700 1 -- 172.21.15.36:0/3722373094 learned_addr learned my addr 172.21.15.36:0/3722373094 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:22.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.579+0000 7faedcd29700 1 --2- 172.21.15.36:0/3722373094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faed80ae150 0x7faed81538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.579+0000 7faed7fff700 1 --2- 172.21.15.36:0/3722373094 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faed80b2fc0 0x7faed8153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.579+0000 7faedd52a700 1 -- 172.21.15.36:0/3722373094 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faed80b2fc0 msgr2=0x7faed8153dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:22.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.579+0000 7faedd52a700 1 --2- 172.21.15.36:0/3722373094 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faed80b2fc0 0x7faed8153dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.579+0000 7faedd52a700 1 -- 172.21.15.36:0/3722373094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faed80ae150 msgr2=0x7faed81538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:22.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.579+0000 7faedd52a700 1 --2- 172.21.15.36:0/3722373094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faed80ae150 0x7faed81538d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.579+0000 7faedd52a700 1 -- 172.21.15.36:0/3722373094 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed0009470 con 0x7faed80b7bc0 2022-06-17T12:55:22.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.579+0000 7faed7fff700 1 --2- 172.21.15.36:0/3722373094 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faed80b2fc0 0x7faed8153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:22.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.579+0000 7faedd52a700 1 --2- 172.21.15.36:0/3722373094 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faed80b7bc0 0x7faed813ce80 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7faec80018c0 tx=0x7faec8001e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:22.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.583+0000 7faed5ffb700 1 -- 172.21.15.36:0/3722373094 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faec8007170 con 0x7faed80b7bc0 2022-06-17T12:55:22.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.583+0000 7faed5ffb700 1 -- 172.21.15.36:0/3722373094 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec80072d0 con 0x7faed80b7bc0 2022-06-17T12:55:22.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.583+0000 7faedef8d700 1 -- 172.21.15.36:0/3722373094 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec000b180 con 0x7faed80b7bc0 2022-06-17T12:55:22.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.583+0000 7faed5ffb700 1 -- 172.21.15.36:0/3722373094 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faec8004710 con 0x7faed80b7bc0 2022-06-17T12:55:22.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.583+0000 7faedef8d700 1 -- 172.21.15.36:0/3722373094 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed80b0d30 con 0x7faed80b7bc0 2022-06-17T12:55:22.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.583+0000 7faedef8d700 1 -- 172.21.15.36:0/3722373094 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed8145220 con 0x7faed80b7bc0 2022-06-17T12:55:22.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.583+0000 7faed5ffb700 1 -- 172.21.15.36:0/3722373094 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7faec8004870 con 0x7faed80b7bc0 2022-06-17T12:55:22.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.583+0000 7faed5ffb700 1 --2- 172.21.15.36:0/3722373094 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faebc053a60 0x7faebc055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.583+0000 7faed5ffb700 1 -- 172.21.15.36:0/3722373094 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5600+0+0 (secure 0 0 0) 0x7faec806b5d0 con 0x7faed80b7bc0 2022-06-17T12:55:22.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.583+0000 7faedcd29700 1 --2- 172.21.15.36:0/3722373094 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faebc053a60 0x7faebc055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.583+0000 7faedcd29700 1 --2- 172.21.15.36:0/3722373094 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faebc053a60 0x7faebc055f20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7faed000b000 tx=0x7faed00091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:22.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.591+0000 7faed5ffb700 1 -- 172.21.15.36:0/3722373094 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faec8042710 con 0x7faed80b7bc0 2022-06-17T12:55:22.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.715+0000 7faedef8d700 1 -- 172.21.15.36:0/3722373094 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7faed8057410 con 0x7faed80b7bc0 2022-06-17T12:55:22.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.715+0000 7faed5ffb700 1 -- 172.21.15.36:0/3722373094 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v109) v1 ==== 92+0+10 (secure 0 0 0) 0x7faec8045d30 con 0x7faed80b7bc0 2022-06-17T12:55:22.705 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:55:22.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.719+0000 7faedef8d700 1 -- 172.21.15.36:0/3722373094 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faebc053a60 msgr2=0x7faebc055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:22.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.719+0000 7faedef8d700 1 --2- 172.21.15.36:0/3722373094 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faebc053a60 0x7faebc055f20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7faed000b000 tx=0x7faed00091d0).stop 2022-06-17T12:55:22.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.719+0000 7faedef8d700 1 -- 172.21.15.36:0/3722373094 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faed80b7bc0 msgr2=0x7faed813ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:22.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.719+0000 7faedef8d700 1 --2- 172.21.15.36:0/3722373094 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faed80b7bc0 0x7faed813ce80 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7faec80018c0 tx=0x7faec8001e40).stop 2022-06-17T12:55:22.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.719+0000 7faedef8d700 1 -- 172.21.15.36:0/3722373094 shutdown_connections 2022-06-17T12:55:22.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.719+0000 7faedef8d700 1 --2- 172.21.15.36:0/3722373094 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faebc053a60 0x7faebc055f20 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.719+0000 7faedef8d700 1 --2- 172.21.15.36:0/3722373094 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faed80b7bc0 0x7faed813ce80 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.719+0000 7faedef8d700 1 --2- 172.21.15.36:0/3722373094 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faed80b2fc0 0x7faed8153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.719+0000 7faedef8d700 1 --2- 172.21.15.36:0/3722373094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faed80ae150 0x7faed81538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.719+0000 7faedef8d700 1 -- 172.21.15.36:0/3722373094 >> 172.21.15.36:0/3722373094 conn(0x7faed8060990 msgr2=0x7faed81290d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:22.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.719+0000 7faedef8d700 1 -- 172.21.15.36:0/3722373094 shutdown_connections 2022-06-17T12:55:22.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.719+0000 7faedef8d700 1 -- 172.21.15.36:0/3722373094 wait complete. 2022-06-17T12:55:22.723 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-17T12:55:22.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.839+0000 7f0ed4f3f700 1 Processor -- start 2022-06-17T12:55:22.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.839+0000 7f0ed4f3f700 1 -- start start 2022-06-17T12:55:22.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ed4f3f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed00b1280 0x7f0ed013fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ed4f3f700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0ed00abe30 0x7f0ed0062ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ed4f3f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0ed0140410 0x7f0ed01436d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ed4f3f700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed00aef10 con 0x7f0ed0140410 2022-06-17T12:55:22.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ed4f3f700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed0140080 con 0x7f0ed00abe30 2022-06-17T12:55:22.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ed4f3f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed00b1660 con 0x7f0ed00b1280 2022-06-17T12:55:22.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ecffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0ed0140410 0x7f0ed01436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ecffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0ed0140410 0x7f0ed01436d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38350/0 (socket says 172.21.15.36:38350) 2022-06-17T12:55:22.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ecffff700 1 -- 172.21.15.36:0/95613847 learned_addr learned my addr 172.21.15.36:0/95613847 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:22.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0eceffd700 1 --2- 172.21.15.36:0/95613847 >> v2:172.21.15.70:3300/0 conn(0x7f0ed00abe30 0x7f0ed0062ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ecf7fe700 1 --2- 172.21.15.36:0/95613847 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed00b1280 0x7f0ed013fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ecffff700 1 -- 172.21.15.36:0/95613847 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed00b1280 msgr2=0x7f0ed013fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:22.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ecffff700 1 --2- 172.21.15.36:0/95613847 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed00b1280 0x7f0ed013fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ecffff700 1 -- 172.21.15.36:0/95613847 >> v2:172.21.15.70:3300/0 conn(0x7f0ed00abe30 msgr2=0x7f0ed0062ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:22.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ecffff700 1 --2- 172.21.15.36:0/95613847 >> v2:172.21.15.70:3300/0 conn(0x7f0ed00abe30 0x7f0ed0062ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ecffff700 1 -- 172.21.15.36:0/95613847 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed00ac3b0 con 0x7f0ed0140410 2022-06-17T12:55:22.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0eceffd700 1 --2- 172.21.15.36:0/95613847 >> v2:172.21.15.70:3300/0 conn(0x7f0ed00abe30 0x7f0ed0062ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:22.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ecffff700 1 --2- 172.21.15.36:0/95613847 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ed0140410 0x7f0ed01436d0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7f0ed0062510 tx=0x7f0ec8009290).ready entity=mon.0 client_cookie=63e06a5b354cfdff server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:22.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ece7fc700 1 -- 172.21.15.36:0/95613847 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ec800b680 con 0x7f0ed0140410 2022-06-17T12:55:22.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ece7fc700 1 -- 172.21.15.36:0/95613847 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec800b7e0 con 0x7f0ed0140410 2022-06-17T12:55:22.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ece7fc700 1 -- 172.21.15.36:0/95613847 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ec800b950 con 0x7f0ed0140410 2022-06-17T12:55:22.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ed4f3f700 1 -- 172.21.15.36:0/95613847 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ed0140410 msgr2=0x7f0ed01436d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:22.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ed4f3f700 1 --2- 172.21.15.36:0/95613847 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ed0140410 0x7f0ed01436d0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7f0ed0062510 tx=0x7f0ec8009290).stop 2022-06-17T12:55:22.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ed4f3f700 1 -- 172.21.15.36:0/95613847 shutdown_connections 2022-06-17T12:55:22.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ed4f3f700 1 --2- 172.21.15.36:0/95613847 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ed0140410 0x7f0ed01436d0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ed4f3f700 1 --2- 172.21.15.36:0/95613847 >> v2:172.21.15.70:3300/0 conn(0x7f0ed00abe30 0x7f0ed0062ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ed4f3f700 1 --2- 172.21.15.36:0/95613847 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed00b1280 0x7f0ed013fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ed4f3f700 1 -- 172.21.15.36:0/95613847 >> 172.21.15.36:0/95613847 conn(0x7f0ed005f770 msgr2=0x7f0ed005fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:22.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ed4f3f700 1 -- 172.21.15.36:0/95613847 shutdown_connections 2022-06-17T12:55:22.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.843+0000 7f0ed4f3f700 1 -- 172.21.15.36:0/95613847 wait complete. 2022-06-17T12:55:22.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.847+0000 7f0ed4f3f700 1 Processor -- start 2022-06-17T12:55:22.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.847+0000 7f0ed4f3f700 1 -- start start 2022-06-17T12:55:22.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.847+0000 7f0ed4f3f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ed00abe30 0x7f0ed013ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.847+0000 7f0ed4f3f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ed00b1280 0x7f0ed00b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.847+0000 7f0ed4f3f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed0140410 0x7f0ed00b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.847+0000 7f0ed4f3f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed012b880 con 0x7f0ed00abe30 2022-06-17T12:55:22.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.847+0000 7f0ed4f3f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed0142750 con 0x7f0ed00b1280 2022-06-17T12:55:22.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.847+0000 7f0ed4f3f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed01399e0 con 0x7f0ed0140410 2022-06-17T12:55:22.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.851+0000 7f0ecf7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ed00abe30 0x7f0ed013ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.851+0000 7f0ecf7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ed00abe30 0x7f0ed013ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38352/0 (socket says 172.21.15.36:38352) 2022-06-17T12:55:22.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.851+0000 7f0ecf7fe700 1 -- 172.21.15.36:0/2485778836 learned_addr learned my addr 172.21.15.36:0/2485778836 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:22.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.851+0000 7f0eceffd700 1 --2- 172.21.15.36:0/2485778836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ed00b1280 0x7f0ed00b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.851+0000 7f0ecffff700 1 --2- 172.21.15.36:0/2485778836 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed0140410 0x7f0ed00b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.851+0000 7f0ecf7fe700 1 -- 172.21.15.36:0/2485778836 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed0140410 msgr2=0x7f0ed00b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:22.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.851+0000 7f0ecf7fe700 1 --2- 172.21.15.36:0/2485778836 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed0140410 0x7f0ed00b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.851+0000 7f0ecf7fe700 1 -- 172.21.15.36:0/2485778836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ed00b1280 msgr2=0x7f0ed00b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:22.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.851+0000 7f0ecf7fe700 1 --2- 172.21.15.36:0/2485778836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ed00b1280 0x7f0ed00b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.851+0000 7f0ecf7fe700 1 -- 172.21.15.36:0/2485778836 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec400b180 con 0x7f0ed00abe30 2022-06-17T12:55:22.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.851+0000 7f0eceffd700 1 --2- 172.21.15.36:0/2485778836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ed00b1280 0x7f0ed00b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:22.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.851+0000 7f0ecf7fe700 1 --2- 172.21.15.36:0/2485778836 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ed00abe30 0x7f0ed013ecd0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 rx=0x7f0ec000c000 tx=0x7f0ec0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:22.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.851+0000 7f0eccff9700 1 -- 172.21.15.36:0/2485778836 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ec0010780 con 0x7f0ed00abe30 2022-06-17T12:55:22.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.851+0000 7f0eccff9700 1 -- 172.21.15.36:0/2485778836 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec00108e0 con 0x7f0ed00abe30 2022-06-17T12:55:22.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.851+0000 7f0eccff9700 1 -- 172.21.15.36:0/2485778836 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ec0010c40 con 0x7f0ed00abe30 2022-06-17T12:55:22.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.851+0000 7f0ed4f3f700 1 -- 172.21.15.36:0/2485778836 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec8009530 con 0x7f0ed00abe30 2022-06-17T12:55:22.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.851+0000 7f0ed4f3f700 1 -- 172.21.15.36:0/2485778836 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed0139c40 con 0x7f0ed00abe30 2022-06-17T12:55:22.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.851+0000 7f0eccff9700 1 -- 172.21.15.36:0/2485778836 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f0ec001c460 con 0x7f0ed00abe30 2022-06-17T12:55:22.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.851+0000 7f0ed4f3f700 1 -- 172.21.15.36:0/2485778836 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed00ac650 con 0x7f0ed00abe30 2022-06-17T12:55:22.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.851+0000 7f0eccff9700 1 --2- 172.21.15.36:0/2485778836 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0eb0053a40 0x7f0eb0055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:22.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.855+0000 7f0eceffd700 1 --2- 172.21.15.36:0/2485778836 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0eb0053a40 0x7f0eb0055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:22.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.855+0000 7f0eccff9700 1 -- 172.21.15.36:0/2485778836 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f0ec006c260 con 0x7f0ed00abe30 2022-06-17T12:55:22.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.855+0000 7f0eceffd700 1 --2- 172.21.15.36:0/2485778836 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0eb0053a40 0x7f0eb0055f00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f0ec400b000 tx=0x7f0ec4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:22.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.859+0000 7f0eccff9700 1 -- 172.21.15.36:0/2485778836 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0ec0043270 con 0x7f0ed00abe30 2022-06-17T12:55:22.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.987+0000 7f0ed4f3f700 1 -- 172.21.15.36:0/2485778836 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0ed0057410 con 0x7f0ed00abe30 2022-06-17T12:55:22.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.987+0000 7f0eccff9700 1 -- 172.21.15.36:0/2485778836 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v109) v1 ==== 84+0+143 (secure 0 0 0) 0x7f0ec0046890 con 0x7f0ed00abe30 2022-06-17T12:55:22.977 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:22.978 INFO:teuthology.orchestra.run.smithi036.stdout:[ 2022-06-17T12:55:22.978 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:22.978 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 1, 2022-06-17T12:55:22.979 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "device_health_metrics" 2022-06-17T12:55:22.979 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:22.979 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:22.979 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 2, 2022-06-17T12:55:22.979 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "rbd" 2022-06-17T12:55:22.980 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:22.980 INFO:teuthology.orchestra.run.smithi036.stdout:] 2022-06-17T12:55:22.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.991+0000 7f0ed4f3f700 1 -- 172.21.15.36:0/2485778836 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0eb0053a40 msgr2=0x7f0eb0055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:22.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.991+0000 7f0ed4f3f700 1 --2- 172.21.15.36:0/2485778836 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0eb0053a40 0x7f0eb0055f00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f0ec400b000 tx=0x7f0ec4008040).stop 2022-06-17T12:55:22.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.991+0000 7f0ed4f3f700 1 -- 172.21.15.36:0/2485778836 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ed00abe30 msgr2=0x7f0ed013ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:22.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.991+0000 7f0ed4f3f700 1 --2- 172.21.15.36:0/2485778836 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ed00abe30 0x7f0ed013ecd0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 rx=0x7f0ec000c000 tx=0x7f0ec0006df0).stop 2022-06-17T12:55:22.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.991+0000 7f0ed4f3f700 1 -- 172.21.15.36:0/2485778836 shutdown_connections 2022-06-17T12:55:22.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.991+0000 7f0ed4f3f700 1 --2- 172.21.15.36:0/2485778836 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0eb0053a40 0x7f0eb0055f00 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.991+0000 7f0ed4f3f700 1 --2- 172.21.15.36:0/2485778836 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0ed0140410 0x7f0ed00b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.991+0000 7f0ed4f3f700 1 --2- 172.21.15.36:0/2485778836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ed00b1280 0x7f0ed00b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.991+0000 7f0ed4f3f700 1 --2- 172.21.15.36:0/2485778836 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0ed00abe30 0x7f0ed013ecd0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:22.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.991+0000 7f0ed4f3f700 1 -- 172.21.15.36:0/2485778836 >> 172.21.15.36:0/2485778836 conn(0x7f0ed005f770 msgr2=0x7f0ed005fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:22.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.991+0000 7f0ed4f3f700 1 -- 172.21.15.36:0/2485778836 shutdown_connections 2022-06-17T12:55:22.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:22.991+0000 7f0ed4f3f700 1 -- 172.21.15.36:0/2485778836 wait complete. 2022-06-17T12:55:22.990 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-17T12:55:22.991 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-17T12:55:23.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.115+0000 7fd22b59e700 1 Processor -- start 2022-06-17T12:55:23.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.115+0000 7fd22b59e700 1 -- start start 2022-06-17T12:55:23.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd22b59e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd22c0ae360 0x7fd22c144ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:23.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd22b59e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd22c144fd0 0x7fd22c14aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:23.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd22b59e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22c0b2b60 con 0x7fd22c0601c0 2022-06-17T12:55:23.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd22b59e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd22c0b3230 con 0x7fd22c144fd0 2022-06-17T12:55:23.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd22b59e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd22c0605a0 con 0x7fd22c0ae360 2022-06-17T12:55:23.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd22ad9d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd22c144fd0 0x7fd22c14aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:23.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd223fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd22c0ae360 0x7fd22c144ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:23.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd22ad9d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd22c144fd0 0x7fd22c14aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38364/0 (socket says 172.21.15.36:38364) 2022-06-17T12:55:23.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd22ad9d700 1 -- 172.21.15.36:0/477352085 learned_addr learned my addr 172.21.15.36:0/477352085 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:23.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd22a59c700 1 --1- 172.21.15.36:0/477352085 >> v1:172.21.15.70:6789/0 conn(0x7fd22c0601c0 0x7fd22c1443d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:51624/0 (socket says 172.21.15.36:51624) 2022-06-17T12:55:23.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd22ad9d700 1 -- 172.21.15.36:0/477352085 >> v2:172.21.15.70:3300/0 conn(0x7fd22c0ae360 msgr2=0x7fd22c144ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:23.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd22ad9d700 1 --2- 172.21.15.36:0/477352085 >> v2:172.21.15.70:3300/0 conn(0x7fd22c0ae360 0x7fd22c144ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:23.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd22ad9d700 1 -- 172.21.15.36:0/477352085 >> v1:172.21.15.70:6789/0 conn(0x7fd22c0601c0 legacy=0x7fd22c1443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:23.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd22ad9d700 1 -- 172.21.15.36:0/477352085 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22c0662f0 con 0x7fd22c144fd0 2022-06-17T12:55:23.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd223fff700 1 --2- 172.21.15.36:0/477352085 >> v2:172.21.15.70:3300/0 conn(0x7fd22c0ae360 0x7fd22c144ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:23.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd22ad9d700 1 --2- 172.21.15.36:0/477352085 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd22c144fd0 0x7fd22c14aa00 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0x7fd2240032f0 tx=0x7fd224009290).ready entity=mon.0 client_cookie=4961cc8ec1f35e8f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:23.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd229d9b700 1 -- 172.21.15.36:0/477352085 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd22400b870 con 0x7fd22c144fd0 2022-06-17T12:55:23.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd229d9b700 1 -- 172.21.15.36:0/477352085 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22400b9d0 con 0x7fd22c144fd0 2022-06-17T12:55:23.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd229d9b700 1 -- 172.21.15.36:0/477352085 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd22400bb40 con 0x7fd22c144fd0 2022-06-17T12:55:23.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd22b59e700 1 -- 172.21.15.36:0/477352085 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd22c144fd0 msgr2=0x7fd22c14aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:23.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd22b59e700 1 --2- 172.21.15.36:0/477352085 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd22c144fd0 0x7fd22c14aa00 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0x7fd2240032f0 tx=0x7fd224009290).stop 2022-06-17T12:55:23.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd22b59e700 1 -- 172.21.15.36:0/477352085 shutdown_connections 2022-06-17T12:55:23.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd22b59e700 1 --2- 172.21.15.36:0/477352085 >> v2:172.21.15.70:3300/0 conn(0x7fd22c0ae360 0x7fd22c144ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:23.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd22b59e700 1 --2- 172.21.15.36:0/477352085 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd22c144fd0 0x7fd22c14aa00 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:23.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd22b59e700 1 -- 172.21.15.36:0/477352085 >> 172.21.15.36:0/477352085 conn(0x7fd22c060990 msgr2=0x7fd22c05f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:23.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd22b59e700 1 -- 172.21.15.36:0/477352085 shutdown_connections 2022-06-17T12:55:23.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.119+0000 7fd22b59e700 1 -- 172.21.15.36:0/477352085 wait complete. 2022-06-17T12:55:23.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.123+0000 7fd22b59e700 1 Processor -- start 2022-06-17T12:55:23.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.123+0000 7fd22b59e700 1 -- start start 2022-06-17T12:55:23.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.123+0000 7fd22b59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd22c0601c0 0x7fd22c066c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:23.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.123+0000 7fd22b59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd22c0ae360 0x7fd22c155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:23.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.123+0000 7fd22b59e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd22c144fd0 0x7fd22c1561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:23.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.123+0000 7fd22b59e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c13ea30 con 0x7fd22c0ae360 2022-06-17T12:55:23.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.123+0000 7fd22b59e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c0626f0 con 0x7fd22c144fd0 2022-06-17T12:55:23.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.123+0000 7fd22b59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c1566e0 con 0x7fd22c0601c0 2022-06-17T12:55:23.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.123+0000 7fd223fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd22c0ae360 0x7fd22c155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:23.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.123+0000 7fd223fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd22c0ae360 0x7fd22c155d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38368/0 (socket says 172.21.15.36:38368) 2022-06-17T12:55:23.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.123+0000 7fd223fff700 1 -- 172.21.15.36:0/925227593 learned_addr learned my addr 172.21.15.36:0/925227593 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:23.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.123+0000 7fd22a59c700 1 --2- 172.21.15.36:0/925227593 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd22c0601c0 0x7fd22c066c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:23.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.123+0000 7fd22ad9d700 1 --2- 172.21.15.36:0/925227593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd22c144fd0 0x7fd22c1561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:23.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.127+0000 7fd223fff700 1 -- 172.21.15.36:0/925227593 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd22c0601c0 msgr2=0x7fd22c066c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:23.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.127+0000 7fd223fff700 1 --2- 172.21.15.36:0/925227593 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd22c0601c0 0x7fd22c066c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:23.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.127+0000 7fd223fff700 1 -- 172.21.15.36:0/925227593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd22c144fd0 msgr2=0x7fd22c1561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:23.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.127+0000 7fd223fff700 1 --2- 172.21.15.36:0/925227593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd22c144fd0 0x7fd22c1561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:23.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.127+0000 7fd223fff700 1 -- 172.21.15.36:0/925227593 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd21400b180 con 0x7fd22c0ae360 2022-06-17T12:55:23.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.127+0000 7fd223fff700 1 --2- 172.21.15.36:0/925227593 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd22c0ae360 0x7fd22c155d00 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7fd21400c000 tx=0x7fd214006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:23.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.127+0000 7fd2237fe700 1 -- 172.21.15.36:0/925227593 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd21400c7e0 con 0x7fd22c0ae360 2022-06-17T12:55:23.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.127+0000 7fd2237fe700 1 -- 172.21.15.36:0/925227593 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21400c940 con 0x7fd22c0ae360 2022-06-17T12:55:23.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.127+0000 7fd22b59e700 1 -- 172.21.15.36:0/925227593 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd224009530 con 0x7fd22c0ae360 2022-06-17T12:55:23.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.127+0000 7fd2237fe700 1 -- 172.21.15.36:0/925227593 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd21400cca0 con 0x7fd22c0ae360 2022-06-17T12:55:23.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.127+0000 7fd22b59e700 1 -- 172.21.15.36:0/925227593 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22c062e60 con 0x7fd22c0ae360 2022-06-17T12:55:23.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.127+0000 7fd22b59e700 1 -- 172.21.15.36:0/925227593 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd22c062880 con 0x7fd22c0ae360 2022-06-17T12:55:23.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.127+0000 7fd2237fe700 1 -- 172.21.15.36:0/925227593 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd21401c460 con 0x7fd22c0ae360 2022-06-17T12:55:23.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.127+0000 7fd2237fe700 1 --2- 172.21.15.36:0/925227593 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd210053940 0x7fd210055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:23.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.127+0000 7fd2237fe700 1 -- 172.21.15.36:0/925227593 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fd21406c1c0 con 0x7fd22c0ae360 2022-06-17T12:55:23.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.127+0000 7fd22a59c700 1 --2- 172.21.15.36:0/925227593 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd210053940 0x7fd210055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:23.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.131+0000 7fd22a59c700 1 --2- 172.21.15.36:0/925227593 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd210053940 0x7fd210055e00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7fd21c00d000 tx=0x7fd21c007b80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:23.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.135+0000 7fd2237fe700 1 -- 172.21.15.36:0/925227593 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd2140432a0 con 0x7fd22c0ae360 2022-06-17T12:55:23.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:23.263+0000 7fd22b59e700 1 -- 172.21.15.36:0/925227593 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fd22c062ff0 con 0x7fd22c0ae360 2022-06-17T12:55:24.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.083+0000 7fd2237fe700 1 -- 172.21.15.36:0/925227593 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v110) v1 ==== 133+0+0 (secure 0 0 0) 0x7fd21400ce00 con 0x7fd22c0ae360 2022-06-17T12:55:24.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.087+0000 7fd22b59e700 1 -- 172.21.15.36:0/925227593 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd210053940 msgr2=0x7fd210055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:24.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.087+0000 7fd22b59e700 1 --2- 172.21.15.36:0/925227593 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd210053940 0x7fd210055e00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7fd21c00d000 tx=0x7fd21c007b80).stop 2022-06-17T12:55:24.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.087+0000 7fd22b59e700 1 -- 172.21.15.36:0/925227593 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd22c0ae360 msgr2=0x7fd22c155d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:24.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.087+0000 7fd22b59e700 1 --2- 172.21.15.36:0/925227593 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd22c0ae360 0x7fd22c155d00 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7fd21400c000 tx=0x7fd214006df0).stop 2022-06-17T12:55:24.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.087+0000 7fd22b59e700 1 -- 172.21.15.36:0/925227593 shutdown_connections 2022-06-17T12:55:24.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.087+0000 7fd22b59e700 1 --2- 172.21.15.36:0/925227593 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd210053940 0x7fd210055e00 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.087+0000 7fd22b59e700 1 --2- 172.21.15.36:0/925227593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd22c144fd0 0x7fd22c1561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.087+0000 7fd22b59e700 1 --2- 172.21.15.36:0/925227593 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd22c0ae360 0x7fd22c155d00 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.087+0000 7fd22b59e700 1 --2- 172.21.15.36:0/925227593 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd22c0601c0 0x7fd22c066c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.087+0000 7fd22b59e700 1 -- 172.21.15.36:0/925227593 >> 172.21.15.36:0/925227593 conn(0x7fd22c060990 msgr2=0x7fd22c13f410 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:24.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.087+0000 7fd22b59e700 1 -- 172.21.15.36:0/925227593 shutdown_connections 2022-06-17T12:55:24.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.087+0000 7fd22b59e700 1 -- 172.21.15.36:0/925227593 wait complete. 2022-06-17T12:55:24.077 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' created 2022-06-17T12:55:24.088 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-06-17T12:55:24.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.207+0000 7fcd5e39d700 1 Processor -- start 2022-06-17T12:55:24.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.207+0000 7fcd5e39d700 1 -- start start 2022-06-17T12:55:24.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.207+0000 7fcd5e39d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd580b6860 0x7fcd58146d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:24.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.207+0000 7fcd5e39d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcd580b18e0 0x7fcd58143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:24.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.207+0000 7fcd5e39d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcd580b39a0 0x7fcd58143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:24.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.207+0000 7fcd5e39d700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd580b6c40 con 0x7fcd580b39a0 2022-06-17T12:55:24.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.207+0000 7fcd5e39d700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd58142d30 con 0x7fcd580b18e0 2022-06-17T12:55:24.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.207+0000 7fcd5e39d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd58147230 con 0x7fcd580b6860 2022-06-17T12:55:24.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.207+0000 7fcd5db9c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcd580b39a0 0x7fcd58143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:24.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.207+0000 7fcd5db9c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcd580b39a0 0x7fcd58143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38378/0 (socket says 172.21.15.36:38378) 2022-06-17T12:55:24.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.207+0000 7fcd5cb9a700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcd580b18e0 0x7fcd58143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:24.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.207+0000 7fcd5db9c700 1 -- 172.21.15.36:0/1714119858 learned_addr learned my addr 172.21.15.36:0/1714119858 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:24.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.207+0000 7fcd5d39b700 1 --2- 172.21.15.36:0/1714119858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd580b6860 0x7fcd58146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:24.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.211+0000 7fcd5db9c700 1 -- 172.21.15.36:0/1714119858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd580b6860 msgr2=0x7fcd58146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:24.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.211+0000 7fcd5db9c700 1 --2- 172.21.15.36:0/1714119858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd580b6860 0x7fcd58146d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.211+0000 7fcd5db9c700 1 -- 172.21.15.36:0/1714119858 >> v2:172.21.15.70:3300/0 conn(0x7fcd580b18e0 msgr2=0x7fcd58143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:24.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.211+0000 7fcd5db9c700 1 --2- 172.21.15.36:0/1714119858 >> v2:172.21.15.70:3300/0 conn(0x7fcd580b18e0 0x7fcd58143150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.211+0000 7fcd5db9c700 1 -- 172.21.15.36:0/1714119858 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd580b2b70 con 0x7fcd580b39a0 2022-06-17T12:55:24.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.211+0000 7fcd5db9c700 1 --2- 172.21.15.36:0/1714119858 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd580b39a0 0x7fcd58143640 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7fcd58142e70 tx=0x7fcd50009290).ready entity=mon.0 client_cookie=2f233191a28a7c35 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:24.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.211+0000 7fcd4ffff700 1 -- 172.21.15.36:0/1714119858 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd5000b680 con 0x7fcd580b39a0 2022-06-17T12:55:24.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.211+0000 7fcd4ffff700 1 -- 172.21.15.36:0/1714119858 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5000b7e0 con 0x7fcd580b39a0 2022-06-17T12:55:24.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.211+0000 7fcd4ffff700 1 -- 172.21.15.36:0/1714119858 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd5000b950 con 0x7fcd580b39a0 2022-06-17T12:55:24.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.211+0000 7fcd5e39d700 1 -- 172.21.15.36:0/1714119858 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd580b39a0 msgr2=0x7fcd58143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:24.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.211+0000 7fcd5e39d700 1 --2- 172.21.15.36:0/1714119858 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd580b39a0 0x7fcd58143640 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7fcd58142e70 tx=0x7fcd50009290).stop 2022-06-17T12:55:24.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.211+0000 7fcd5e39d700 1 -- 172.21.15.36:0/1714119858 shutdown_connections 2022-06-17T12:55:24.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.211+0000 7fcd5e39d700 1 --2- 172.21.15.36:0/1714119858 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd580b39a0 0x7fcd58143640 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.211+0000 7fcd5e39d700 1 --2- 172.21.15.36:0/1714119858 >> v2:172.21.15.70:3300/0 conn(0x7fcd580b18e0 0x7fcd58143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.211+0000 7fcd5e39d700 1 --2- 172.21.15.36:0/1714119858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd580b6860 0x7fcd58146d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.211+0000 7fcd5e39d700 1 -- 172.21.15.36:0/1714119858 >> 172.21.15.36:0/1714119858 conn(0x7fcd580614b0 msgr2=0x7fcd58061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:24.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.211+0000 7fcd5e39d700 1 -- 172.21.15.36:0/1714119858 shutdown_connections 2022-06-17T12:55:24.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.211+0000 7fcd5e39d700 1 -- 172.21.15.36:0/1714119858 wait complete. 2022-06-17T12:55:24.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.215+0000 7fcd5e39d700 1 Processor -- start 2022-06-17T12:55:24.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.215+0000 7fcd5e39d700 1 -- start start 2022-06-17T12:55:24.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.215+0000 7fcd5e39d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd580b18e0 0x7fcd581520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:24.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.215+0000 7fcd5e39d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd580b39a0 0x7fcd5813f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:24.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.215+0000 7fcd5e39d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd580b6860 0x7fcd580ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:24.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.215+0000 7fcd5e39d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd58115d70 con 0x7fcd580b6860 2022-06-17T12:55:24.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.215+0000 7fcd5e39d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd58115fa0 con 0x7fcd580b18e0 2022-06-17T12:55:24.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.215+0000 7fcd5e39d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd580653d0 con 0x7fcd580b39a0 2022-06-17T12:55:24.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.215+0000 7fcd5db9c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd580b6860 0x7fcd580ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:24.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.215+0000 7fcd5d39b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd580b18e0 0x7fcd581520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:24.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.215+0000 7fcd5cb9a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd580b39a0 0x7fcd5813f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:24.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.215+0000 7fcd5db9c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd580b6860 0x7fcd580ae5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38384/0 (socket says 172.21.15.36:38384) 2022-06-17T12:55:24.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.215+0000 7fcd5db9c700 1 -- 172.21.15.36:0/2209634801 learned_addr learned my addr 172.21.15.36:0/2209634801 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:24.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.215+0000 7fcd5d39b700 1 --2- 172.21.15.36:0/2209634801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd580b18e0 0x7fcd581520e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:58762/0 (socket says 172.21.15.36:58762) 2022-06-17T12:55:24.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.215+0000 7fcd5db9c700 1 -- 172.21.15.36:0/2209634801 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd580b39a0 msgr2=0x7fcd5813f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:24.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.215+0000 7fcd5db9c700 1 --2- 172.21.15.36:0/2209634801 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd580b39a0 0x7fcd5813f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.215+0000 7fcd5db9c700 1 -- 172.21.15.36:0/2209634801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd580b18e0 msgr2=0x7fcd581520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:24.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.215+0000 7fcd5db9c700 1 --2- 172.21.15.36:0/2209634801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd580b18e0 0x7fcd581520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.215+0000 7fcd5db9c700 1 -- 172.21.15.36:0/2209634801 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd50009530 con 0x7fcd580b6860 2022-06-17T12:55:24.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.215+0000 7fcd5d39b700 1 --2- 172.21.15.36:0/2209634801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd580b18e0 0x7fcd581520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:24.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.215+0000 7fcd5db9c700 1 --2- 172.21.15.36:0/2209634801 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd580b6860 0x7fcd580ae5c0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7fcd58142e70 tx=0x7fcd50016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:24.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.215+0000 7fcd4e7fc700 1 -- 172.21.15.36:0/2209634801 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd5000b680 con 0x7fcd580b6860 2022-06-17T12:55:24.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.219+0000 7fcd4e7fc700 1 -- 172.21.15.36:0/2209634801 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd50021670 con 0x7fcd580b6860 2022-06-17T12:55:24.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.219+0000 7fcd5e39d700 1 -- 172.21.15.36:0/2209634801 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5813faf0 con 0x7fcd580b6860 2022-06-17T12:55:24.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.219+0000 7fcd4e7fc700 1 -- 172.21.15.36:0/2209634801 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd500219d0 con 0x7fcd580b6860 2022-06-17T12:55:24.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.219+0000 7fcd5e39d700 1 -- 172.21.15.36:0/2209634801 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd580aec50 con 0x7fcd580b6860 2022-06-17T12:55:24.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.219+0000 7fcd5e39d700 1 -- 172.21.15.36:0/2209634801 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd58114030 con 0x7fcd580b6860 2022-06-17T12:55:24.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.219+0000 7fcd4e7fc700 1 -- 172.21.15.36:0/2209634801 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fcd50028070 con 0x7fcd580b6860 2022-06-17T12:55:24.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.219+0000 7fcd4e7fc700 1 --2- 172.21.15.36:0/2209634801 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcd40053a40 0x7fcd40055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:24.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.219+0000 7fcd4e7fc700 1 -- 172.21.15.36:0/2209634801 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5954+0+0 (secure 0 0 0) 0x7fcd5006c900 con 0x7fcd580b6860 2022-06-17T12:55:24.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.219+0000 7fcd5d39b700 1 --2- 172.21.15.36:0/2209634801 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcd40053a40 0x7fcd40055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:24.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.219+0000 7fcd5d39b700 1 --2- 172.21.15.36:0/2209634801 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcd40053a40 0x7fcd40055f00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7fcd4800b000 tx=0x7fcd48008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:24.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.227+0000 7fcd4e7fc700 1 -- 172.21.15.36:0/2209634801 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcd500438d0 con 0x7fcd580b6860 2022-06-17T12:55:24.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.355+0000 7fcd5e39d700 1 -- 172.21.15.36:0/2209634801 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fcd580b0d30 con 0x7fcd580b6860 2022-06-17T12:55:24.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.511+0000 7fcd4e7fc700 1 -- 172.21.15.36:0/2209634801 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v111) v1 ==== 125+0+0 (secure 0 0 0) 0x7fcd50015030 con 0x7fcd580b6860 2022-06-17T12:55:24.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.515+0000 7fcd5e39d700 1 -- 172.21.15.36:0/2209634801 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcd40053a40 msgr2=0x7fcd40055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:24.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.515+0000 7fcd5e39d700 1 --2- 172.21.15.36:0/2209634801 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcd40053a40 0x7fcd40055f00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7fcd4800b000 tx=0x7fcd48008040).stop 2022-06-17T12:55:24.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.515+0000 7fcd5e39d700 1 -- 172.21.15.36:0/2209634801 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd580b6860 msgr2=0x7fcd580ae5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:24.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.515+0000 7fcd5e39d700 1 --2- 172.21.15.36:0/2209634801 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd580b6860 0x7fcd580ae5c0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7fcd58142e70 tx=0x7fcd50016000).stop 2022-06-17T12:55:24.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.515+0000 7fcd5e39d700 1 -- 172.21.15.36:0/2209634801 shutdown_connections 2022-06-17T12:55:24.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.515+0000 7fcd5e39d700 1 --2- 172.21.15.36:0/2209634801 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcd40053a40 0x7fcd40055f00 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.515+0000 7fcd5e39d700 1 --2- 172.21.15.36:0/2209634801 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd580b6860 0x7fcd580ae5c0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.515+0000 7fcd5e39d700 1 --2- 172.21.15.36:0/2209634801 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd580b39a0 0x7fcd5813f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.515+0000 7fcd5e39d700 1 --2- 172.21.15.36:0/2209634801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd580b18e0 0x7fcd581520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.515+0000 7fcd5e39d700 1 -- 172.21.15.36:0/2209634801 >> 172.21.15.36:0/2209634801 conn(0x7fcd580614b0 msgr2=0x7fcd58061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:24.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.515+0000 7fcd5e39d700 1 -- 172.21.15.36:0/2209634801 shutdown_connections 2022-06-17T12:55:24.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.515+0000 7fcd5e39d700 1 -- 172.21.15.36:0/2209634801 wait complete. 2022-06-17T12:55:24.511 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' created 2022-06-17T12:55:24.517 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-17T12:55:24.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.639+0000 7fcb2f59e700 1 Processor -- start 2022-06-17T12:55:24.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.639+0000 7fcb2f59e700 1 -- start start 2022-06-17T12:55:24.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2f59e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcb30063730 0x7fcb300b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:24.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2f59e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcb30064eb0 0x7fcb3013ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:24.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2f59e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb30063b10 con 0x7fcb301403f0 2022-06-17T12:55:24.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2f59e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb30147430 con 0x7fcb30063730 2022-06-17T12:55:24.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2f59e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb300b1ca0 con 0x7fcb30064eb0 2022-06-17T12:55:24.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2e59c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcb30063730 0x7fcb300b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:24.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2e59c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcb30063730 0x7fcb300b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38388/0 (socket says 172.21.15.36:38388) 2022-06-17T12:55:24.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2e59c700 1 -- 172.21.15.36:0/133458857 learned_addr learned my addr 172.21.15.36:0/133458857 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:24.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2dd9b700 1 --2- 172.21.15.36:0/133458857 >> v2:172.21.15.70:3300/0 conn(0x7fcb30064eb0 0x7fcb3013ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:24.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2e59c700 1 -- 172.21.15.36:0/133458857 >> v2:172.21.15.70:3300/0 conn(0x7fcb30064eb0 msgr2=0x7fcb3013ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:24.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2e59c700 1 --2- 172.21.15.36:0/133458857 >> v2:172.21.15.70:3300/0 conn(0x7fcb30064eb0 0x7fcb3013ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2e59c700 1 -- 172.21.15.36:0/133458857 >> v1:172.21.15.70:6789/0 conn(0x7fcb301403f0 legacy=0x7fcb300b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:24.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2e59c700 1 -- 172.21.15.36:0/133458857 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb301464d0 con 0x7fcb30063730 2022-06-17T12:55:24.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2e59c700 1 --2- 172.21.15.36:0/133458857 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb30063730 0x7fcb300b2e10 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0x7fcb20002f70 tx=0x7fcb20006eb0).ready entity=mon.0 client_cookie=73e0d6d48cee9608 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:24.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2d59a700 1 -- 172.21.15.36:0/133458857 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb2000d5c0 con 0x7fcb30063730 2022-06-17T12:55:24.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2d59a700 1 -- 172.21.15.36:0/133458857 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb20000e90 con 0x7fcb30063730 2022-06-17T12:55:24.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2d59a700 1 -- 172.21.15.36:0/133458857 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb2000d950 con 0x7fcb30063730 2022-06-17T12:55:24.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2f59e700 1 -- 172.21.15.36:0/133458857 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb30063730 msgr2=0x7fcb300b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:24.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2f59e700 1 --2- 172.21.15.36:0/133458857 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb30063730 0x7fcb300b2e10 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0x7fcb20002f70 tx=0x7fcb20006eb0).stop 2022-06-17T12:55:24.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2f59e700 1 -- 172.21.15.36:0/133458857 shutdown_connections 2022-06-17T12:55:24.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2f59e700 1 --2- 172.21.15.36:0/133458857 >> v2:172.21.15.70:3300/0 conn(0x7fcb30064eb0 0x7fcb3013ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2f59e700 1 --2- 172.21.15.36:0/133458857 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb30063730 0x7fcb300b2e10 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2f59e700 1 -- 172.21.15.36:0/133458857 >> 172.21.15.36:0/133458857 conn(0x7fcb300614b0 msgr2=0x7fcb300618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:24.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2f59e700 1 -- 172.21.15.36:0/133458857 shutdown_connections 2022-06-17T12:55:24.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.643+0000 7fcb2f59e700 1 -- 172.21.15.36:0/133458857 wait complete. 2022-06-17T12:55:24.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.647+0000 7fcb2f59e700 1 Processor -- start 2022-06-17T12:55:24.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.647+0000 7fcb2f59e700 1 -- start start 2022-06-17T12:55:24.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.647+0000 7fcb2f59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb30063730 0x7fcb3013e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:24.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.647+0000 7fcb2f59e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb30064eb0 0x7fcb30156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:24.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.647+0000 7fcb2f59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb301403f0 0x7fcb3013be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:24.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.647+0000 7fcb2f59e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb30156870 con 0x7fcb30063730 2022-06-17T12:55:24.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.647+0000 7fcb2f59e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb30142cf0 con 0x7fcb30064eb0 2022-06-17T12:55:24.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.647+0000 7fcb2f59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb30148e80 con 0x7fcb301403f0 2022-06-17T12:55:24.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.647+0000 7fcb2e59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb30063730 0x7fcb3013e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:24.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.647+0000 7fcb2e59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb30063730 0x7fcb3013e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38394/0 (socket says 172.21.15.36:38394) 2022-06-17T12:55:24.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.647+0000 7fcb2e59c700 1 -- 172.21.15.36:0/2334771384 learned_addr learned my addr 172.21.15.36:0/2334771384 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:24.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.647+0000 7fcb2dd9b700 1 --2- 172.21.15.36:0/2334771384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb30064eb0 0x7fcb30156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:24.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.647+0000 7fcb2ed9d700 1 --2- 172.21.15.36:0/2334771384 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb301403f0 0x7fcb3013be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:24.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.647+0000 7fcb2e59c700 1 -- 172.21.15.36:0/2334771384 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb301403f0 msgr2=0x7fcb3013be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:24.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.647+0000 7fcb2e59c700 1 --2- 172.21.15.36:0/2334771384 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb301403f0 0x7fcb3013be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.647+0000 7fcb2e59c700 1 -- 172.21.15.36:0/2334771384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb30064eb0 msgr2=0x7fcb30156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:24.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.647+0000 7fcb2e59c700 1 --2- 172.21.15.36:0/2334771384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb30064eb0 0x7fcb30156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.647+0000 7fcb2e59c700 1 -- 172.21.15.36:0/2334771384 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2000b1e0 con 0x7fcb30063730 2022-06-17T12:55:24.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.647+0000 7fcb2e59c700 1 --2- 172.21.15.36:0/2334771384 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb30063730 0x7fcb3013e3f0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 rx=0x7fcb20002f70 tx=0x7fcb20015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:24.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.647+0000 7fcb277fe700 1 -- 172.21.15.36:0/2334771384 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb20011860 con 0x7fcb30063730 2022-06-17T12:55:24.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.647+0000 7fcb277fe700 1 -- 172.21.15.36:0/2334771384 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb200119c0 con 0x7fcb30063730 2022-06-17T12:55:24.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.647+0000 7fcb277fe700 1 -- 172.21.15.36:0/2334771384 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb20011d20 con 0x7fcb30063730 2022-06-17T12:55:24.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.647+0000 7fcb2f59e700 1 -- 172.21.15.36:0/2334771384 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb30149120 con 0x7fcb30063730 2022-06-17T12:55:24.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.647+0000 7fcb2f59e700 1 -- 172.21.15.36:0/2334771384 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb30148b70 con 0x7fcb30063730 2022-06-17T12:55:24.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.651+0000 7fcb2f59e700 1 -- 172.21.15.36:0/2334771384 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb30057410 con 0x7fcb30063730 2022-06-17T12:55:24.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.651+0000 7fcb277fe700 1 -- 172.21.15.36:0/2334771384 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fcb20025030 con 0x7fcb30063730 2022-06-17T12:55:24.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.651+0000 7fcb277fe700 1 --2- 172.21.15.36:0/2334771384 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcb14053970 0x7fcb14055e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:24.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.651+0000 7fcb2dd9b700 1 --2- 172.21.15.36:0/2334771384 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcb14053970 0x7fcb14055e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:24.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.651+0000 7fcb277fe700 1 -- 172.21.15.36:0/2334771384 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fcb2006d2f0 con 0x7fcb30063730 2022-06-17T12:55:24.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.651+0000 7fcb2dd9b700 1 --2- 172.21.15.36:0/2334771384 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcb14053970 0x7fcb14055e30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7fcb1800b000 tx=0x7fcb18006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:24.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.655+0000 7fcb277fe700 1 -- 172.21.15.36:0/2334771384 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcb200440a0 con 0x7fcb30063730 2022-06-17T12:55:24.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.819+0000 7fcb2f59e700 1 -- 172.21.15.36:0/2334771384 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fcb300b28e0 con 0x7fcb30063730 2022-06-17T12:55:24.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.831+0000 7fcb277fe700 1 -- 172.21.15.36:0/2334771384 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 21 and data pool 22 v58) v1 ==== 189+0+0 (secure 0 0 0) 0x7fcb200230d0 con 0x7fcb30063730 2022-06-17T12:55:24.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.839+0000 7fcb2f59e700 1 -- 172.21.15.36:0/2334771384 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcb14053970 msgr2=0x7fcb14055e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:24.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.839+0000 7fcb2f59e700 1 --2- 172.21.15.36:0/2334771384 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcb14053970 0x7fcb14055e30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7fcb1800b000 tx=0x7fcb18006df0).stop 2022-06-17T12:55:24.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.839+0000 7fcb2f59e700 1 -- 172.21.15.36:0/2334771384 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb30063730 msgr2=0x7fcb3013e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:24.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.839+0000 7fcb2f59e700 1 --2- 172.21.15.36:0/2334771384 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb30063730 0x7fcb3013e3f0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 rx=0x7fcb20002f70 tx=0x7fcb20015000).stop 2022-06-17T12:55:24.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.839+0000 7fcb2f59e700 1 -- 172.21.15.36:0/2334771384 shutdown_connections 2022-06-17T12:55:24.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.839+0000 7fcb2f59e700 1 --2- 172.21.15.36:0/2334771384 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcb14053970 0x7fcb14055e30 secure :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0x7fcb1800b000 tx=0x7fcb18006df0).stop 2022-06-17T12:55:24.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.839+0000 7fcb2f59e700 1 --2- 172.21.15.36:0/2334771384 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb301403f0 0x7fcb3013be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.839+0000 7fcb2f59e700 1 --2- 172.21.15.36:0/2334771384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb30064eb0 0x7fcb30156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.839+0000 7fcb2f59e700 1 --2- 172.21.15.36:0/2334771384 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcb30063730 0x7fcb3013e3f0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.839+0000 7fcb2f59e700 1 -- 172.21.15.36:0/2334771384 >> 172.21.15.36:0/2334771384 conn(0x7fcb300614b0 msgr2=0x7fcb3013f950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:24.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.839+0000 7fcb2f59e700 1 -- 172.21.15.36:0/2334771384 shutdown_connections 2022-06-17T12:55:24.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.839+0000 7fcb2f59e700 1 -- 172.21.15.36:0/2334771384 wait complete. 2022-06-17T12:55:24.838 INFO:teuthology.orchestra.run.smithi036.stderr:new fs with metadata pool 21 and data pool 22 2022-06-17T12:55:24.844 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:55:24.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.951+0000 7fe78e752700 1 Processor -- start 2022-06-17T12:55:24.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.951+0000 7fe78e752700 1 -- start start 2022-06-17T12:55:24.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78e752700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe78813cb70 0x7fe788143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:24.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78e752700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7880aef60 0x7fe7880b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:24.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78e752700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe7880ba2e0 0x7fe7881442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:24.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78e752700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7880af4b0 con 0x7fe78813cb70 2022-06-17T12:55:24.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78e752700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe788142f50 con 0x7fe7880ba2e0 2022-06-17T12:55:24.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78e752700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7881431f0 con 0x7fe7880aef60 2022-06-17T12:55:24.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78d750700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe78813cb70 0x7fe788143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:24.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78d750700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe78813cb70 0x7fe788143690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38418/0 (socket says 172.21.15.36:38418) 2022-06-17T12:55:24.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78df51700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe7880ba2e0 0x7fe7881442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:24.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78d750700 1 -- 172.21.15.36:0/3932626265 learned_addr learned my addr 172.21.15.36:0/3932626265 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:24.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78cf4f700 1 --2- 172.21.15.36:0/3932626265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7880aef60 0x7fe7880b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:24.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78d750700 1 -- 172.21.15.36:0/3932626265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7880aef60 msgr2=0x7fe7880b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:24.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78d750700 1 --2- 172.21.15.36:0/3932626265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7880aef60 0x7fe7880b9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78d750700 1 -- 172.21.15.36:0/3932626265 >> v2:172.21.15.70:3300/0 conn(0x7fe7880ba2e0 msgr2=0x7fe7881442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:24.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78d750700 1 --2- 172.21.15.36:0/3932626265 >> v2:172.21.15.70:3300/0 conn(0x7fe7880ba2e0 0x7fe7881442a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78d750700 1 -- 172.21.15.36:0/3932626265 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe788062e30 con 0x7fe78813cb70 2022-06-17T12:55:24.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78d750700 1 --2- 172.21.15.36:0/3932626265 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe78813cb70 0x7fe788143690 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 rx=0x7fe784002f70 tx=0x7fe784006eb0).ready entity=mon.0 client_cookie=7abf11f0c16b48d1 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:24.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe777fff700 1 -- 172.21.15.36:0/3932626265 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe78400d570 con 0x7fe78813cb70 2022-06-17T12:55:24.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe777fff700 1 -- 172.21.15.36:0/3932626265 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe784000e90 con 0x7fe78813cb70 2022-06-17T12:55:24.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe777fff700 1 -- 172.21.15.36:0/3932626265 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe78400d900 con 0x7fe78813cb70 2022-06-17T12:55:24.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78e752700 1 -- 172.21.15.36:0/3932626265 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe78813cb70 msgr2=0x7fe788143690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:24.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78e752700 1 --2- 172.21.15.36:0/3932626265 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe78813cb70 0x7fe788143690 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 rx=0x7fe784002f70 tx=0x7fe784006eb0).stop 2022-06-17T12:55:24.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78e752700 1 -- 172.21.15.36:0/3932626265 shutdown_connections 2022-06-17T12:55:24.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78e752700 1 --2- 172.21.15.36:0/3932626265 >> v2:172.21.15.70:3300/0 conn(0x7fe7880ba2e0 0x7fe7881442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78e752700 1 --2- 172.21.15.36:0/3932626265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7880aef60 0x7fe7880b9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78e752700 1 --2- 172.21.15.36:0/3932626265 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe78813cb70 0x7fe788143690 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78e752700 1 -- 172.21.15.36:0/3932626265 >> 172.21.15.36:0/3932626265 conn(0x7fe78805c550 msgr2=0x7fe7880b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:24.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78e752700 1 -- 172.21.15.36:0/3932626265 shutdown_connections 2022-06-17T12:55:24.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.955+0000 7fe78e752700 1 -- 172.21.15.36:0/3932626265 wait complete. 2022-06-17T12:55:24.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.959+0000 7fe78e752700 1 Processor -- start 2022-06-17T12:55:24.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.959+0000 7fe78e752700 1 -- start start 2022-06-17T12:55:24.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.959+0000 7fe78e752700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7880aef60 0x7fe7880b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:24.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.959+0000 7fe78e752700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7880ba2e0 0x7fe7880b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:24.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.959+0000 7fe78e752700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe78813cb70 0x7fe7880b2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:24.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.959+0000 7fe78e752700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7880afe40 con 0x7fe78813cb70 2022-06-17T12:55:24.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.959+0000 7fe78e752700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78813fbc0 con 0x7fe7880aef60 2022-06-17T12:55:24.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.959+0000 7fe78e752700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe788140060 con 0x7fe7880ba2e0 2022-06-17T12:55:24.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.959+0000 7fe78df51700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe78813cb70 0x7fe7880b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:24.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.959+0000 7fe78df51700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe78813cb70 0x7fe7880b2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38426/0 (socket says 172.21.15.36:38426) 2022-06-17T12:55:24.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.959+0000 7fe78df51700 1 -- 172.21.15.36:0/2132735913 learned_addr learned my addr 172.21.15.36:0/2132735913 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:24.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.959+0000 7fe78cf4f700 1 --2- 172.21.15.36:0/2132735913 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7880ba2e0 0x7fe7880b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:24.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.959+0000 7fe78d750700 1 --2- 172.21.15.36:0/2132735913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7880aef60 0x7fe7880b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:24.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.959+0000 7fe78df51700 1 -- 172.21.15.36:0/2132735913 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7880ba2e0 msgr2=0x7fe7880b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:24.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.959+0000 7fe78df51700 1 --2- 172.21.15.36:0/2132735913 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7880ba2e0 0x7fe7880b2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.959+0000 7fe78df51700 1 -- 172.21.15.36:0/2132735913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7880aef60 msgr2=0x7fe7880b6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:24.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.959+0000 7fe78df51700 1 --2- 172.21.15.36:0/2132735913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7880aef60 0x7fe7880b6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:24.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.959+0000 7fe78df51700 1 -- 172.21.15.36:0/2132735913 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78400b1e0 con 0x7fe78813cb70 2022-06-17T12:55:24.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.959+0000 7fe78df51700 1 --2- 172.21.15.36:0/2132735913 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe78813cb70 0x7fe7880b2710 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7fe78000c000 tx=0x7fe7800091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:24.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.959+0000 7fe7767fc700 1 -- 172.21.15.36:0/2132735913 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe78000c6a0 con 0x7fe78813cb70 2022-06-17T12:55:24.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.959+0000 7fe7767fc700 1 -- 172.21.15.36:0/2132735913 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78000c800 con 0x7fe78813cb70 2022-06-17T12:55:24.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.959+0000 7fe78e752700 1 -- 172.21.15.36:0/2132735913 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7880b0d30 con 0x7fe78813cb70 2022-06-17T12:55:24.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.959+0000 7fe7767fc700 1 -- 172.21.15.36:0/2132735913 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe78000cb60 con 0x7fe78813cb70 2022-06-17T12:55:24.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.959+0000 7fe78e752700 1 -- 172.21.15.36:0/2132735913 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7880afbc0 con 0x7fe78813cb70 2022-06-17T12:55:24.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.963+0000 7fe78e752700 1 -- 172.21.15.36:0/2132735913 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7880b3570 con 0x7fe78813cb70 2022-06-17T12:55:24.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.963+0000 7fe7767fc700 1 -- 172.21.15.36:0/2132735913 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe780022070 con 0x7fe78813cb70 2022-06-17T12:55:24.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.963+0000 7fe7767fc700 1 --2- 172.21.15.36:0/2132735913 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe768053a40 0x7fe768055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:24.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.963+0000 7fe7767fc700 1 -- 172.21.15.36:0/2132735913 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe78006c680 con 0x7fe78813cb70 2022-06-17T12:55:24.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.963+0000 7fe78d750700 1 --2- 172.21.15.36:0/2132735913 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe768053a40 0x7fe768055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:24.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.963+0000 7fe78d750700 1 --2- 172.21.15.36:0/2132735913 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe768053a40 0x7fe768055f00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7fe78401c000 tx=0x7fe784015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:24.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:24.971+0000 7fe7767fc700 1 -- 172.21.15.36:0/2132735913 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe7880b3570 con 0x7fe78813cb70 2022-06-17T12:55:25.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.099+0000 7fe78e752700 1 -- 172.21.15.36:0/2132735913 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7880b3570 con 0x7fe78813cb70 2022-06-17T12:55:25.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.103+0000 7fe7767fc700 1 -- 172.21.15.36:0/2132735913 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16298 (secure 0 0 0) 0x7fe7880b3570 con 0x7fe78813cb70 2022-06-17T12:55:25.090 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:25.091 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":112,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:55:24.825487+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:55:23.324881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-06-17T12:55:24.417060+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:55:25.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.103+0000 7fe78e752700 1 -- 172.21.15.36:0/2132735913 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe768053a40 msgr2=0x7fe768055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:25.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.103+0000 7fe78e752700 1 --2- 172.21.15.36:0/2132735913 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe768053a40 0x7fe768055f00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7fe78401c000 tx=0x7fe784015000).stop 2022-06-17T12:55:25.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.103+0000 7fe78e752700 1 -- 172.21.15.36:0/2132735913 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe78813cb70 msgr2=0x7fe7880b2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:25.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.103+0000 7fe78e752700 1 --2- 172.21.15.36:0/2132735913 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe78813cb70 0x7fe7880b2710 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7fe78000c000 tx=0x7fe7800091d0).stop 2022-06-17T12:55:25.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.103+0000 7fe78e752700 1 -- 172.21.15.36:0/2132735913 shutdown_connections 2022-06-17T12:55:25.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.103+0000 7fe78e752700 1 --2- 172.21.15.36:0/2132735913 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe768053a40 0x7fe768055f00 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:25.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.103+0000 7fe78e752700 1 --2- 172.21.15.36:0/2132735913 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe78813cb70 0x7fe7880b2710 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:25.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.103+0000 7fe78e752700 1 --2- 172.21.15.36:0/2132735913 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7880ba2e0 0x7fe7880b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:25.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.103+0000 7fe78e752700 1 --2- 172.21.15.36:0/2132735913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7880aef60 0x7fe7880b6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:25.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.103+0000 7fe78e752700 1 -- 172.21.15.36:0/2132735913 >> 172.21.15.36:0/2132735913 conn(0x7fe78805c550 msgr2=0x7fe788145920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:25.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.103+0000 7fe78e752700 1 -- 172.21.15.36:0/2132735913 shutdown_connections 2022-06-17T12:55:25.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.103+0000 7fe78e752700 1 -- 172.21.15.36:0/2132735913 wait complete. 2022-06-17T12:55:25.105 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:55:25.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.231+0000 7f997803c700 1 Processor -- start 2022-06-17T12:55:25.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.231+0000 7f997803c700 1 -- start start 2022-06-17T12:55:25.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f997803c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f99700ae970 0x7f9970142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:25.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f997803c700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9970060ea0 con 0x7f9970060ac0 2022-06-17T12:55:25.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f997803c700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99700b36a0 con 0x7f9970067290 2022-06-17T12:55:25.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f997803c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9970143060 con 0x7f99700ae970 2022-06-17T12:55:25.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f9975dd8700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f9970060ac0 0x7f99700ad640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55960/0 (socket says 172.21.15.36:55960) 2022-06-17T12:55:25.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f9975dd8700 1 -- 172.21.15.36:0/1949244436 learned_addr learned my addr 172.21.15.36:0/1949244436 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:25.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f99765d9700 1 --2- 172.21.15.36:0/1949244436 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f99700ae970 0x7f9970142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:25.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f9974dd6700 1 -- 172.21.15.36:0/1949244436 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3150916683 0 0) 0x7f9970060ea0 con 0x7f9970060ac0 2022-06-17T12:55:25.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f9974dd6700 1 -- 172.21.15.36:0/1949244436 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9958003260 con 0x7f9970060ac0 2022-06-17T12:55:25.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f9974dd6700 1 -- 172.21.15.36:0/1949244436 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819916632 0 0) 0x7f99700b36a0 con 0x7f9970067290 2022-06-17T12:55:25.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f9974dd6700 1 -- 172.21.15.36:0/1949244436 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9970060ea0 con 0x7f9970067290 2022-06-17T12:55:25.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f99765d9700 1 -- 172.21.15.36:0/1949244436 >> v1:172.21.15.70:6789/0 conn(0x7f9970067290 legacy=0x7f9970141f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:25.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f99765d9700 1 -- 172.21.15.36:0/1949244436 >> v1:172.21.15.36:6789/0 conn(0x7f9970060ac0 legacy=0x7f99700ad640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:25.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f99765d9700 1 -- 172.21.15.36:0/1949244436 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99700b1b60 con 0x7f99700ae970 2022-06-17T12:55:25.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f99765d9700 1 --2- 172.21.15.36:0/1949244436 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f99700ae970 0x7f9970142680 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f99680032f0 tx=0x7f99680091d0).ready entity=mon.2 client_cookie=4445ea1002fd2541 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:25.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f9974dd6700 1 -- 172.21.15.36:0/1949244436 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f996800b640 con 0x7f99700ae970 2022-06-17T12:55:25.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f9974dd6700 1 -- 172.21.15.36:0/1949244436 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996800b7a0 con 0x7f99700ae970 2022-06-17T12:55:25.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f9974dd6700 1 -- 172.21.15.36:0/1949244436 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f996800b910 con 0x7f99700ae970 2022-06-17T12:55:25.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f997803c700 1 -- 172.21.15.36:0/1949244436 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f99700ae970 msgr2=0x7f9970142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:25.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f997803c700 1 --2- 172.21.15.36:0/1949244436 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f99700ae970 0x7f9970142680 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f99680032f0 tx=0x7f99680091d0).stop 2022-06-17T12:55:25.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f997803c700 1 -- 172.21.15.36:0/1949244436 shutdown_connections 2022-06-17T12:55:25.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f997803c700 1 --2- 172.21.15.36:0/1949244436 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f99700ae970 0x7f9970142680 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:25.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f997803c700 1 -- 172.21.15.36:0/1949244436 >> 172.21.15.36:0/1949244436 conn(0x7f99700b6870 msgr2=0x7f99700b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:25.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f997803c700 1 -- 172.21.15.36:0/1949244436 shutdown_connections 2022-06-17T12:55:25.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.235+0000 7f997803c700 1 -- 172.21.15.36:0/1949244436 wait complete. 2022-06-17T12:55:25.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.239+0000 7f997803c700 1 Processor -- start 2022-06-17T12:55:25.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.239+0000 7f997803c700 1 -- start start 2022-06-17T12:55:25.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.239+0000 7f997803c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9970060ac0 0x7f9970143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:25.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.239+0000 7f997803c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9970067290 0x7f99700a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:25.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.239+0000 7f997803c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f99700ae970 0x7f99701528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:25.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.239+0000 7f997803c700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9970152450 con 0x7f99700ae970 2022-06-17T12:55:25.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.239+0000 7f997803c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997009f780 con 0x7f9970067290 2022-06-17T12:55:25.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.239+0000 7f997803c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997009fa20 con 0x7f9970060ac0 2022-06-17T12:55:25.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.239+0000 7f99765d9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f99700ae970 0x7f99701528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:25.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.239+0000 7f99765d9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f99700ae970 0x7f99701528e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38440/0 (socket says 172.21.15.36:38440) 2022-06-17T12:55:25.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.239+0000 7f99765d9700 1 -- 172.21.15.36:0/1217674549 learned_addr learned my addr 172.21.15.36:0/1217674549 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:25.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.239+0000 7f9975dd8700 1 --2- 172.21.15.36:0/1217674549 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9970060ac0 0x7f9970143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:25.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.239+0000 7f99755d7700 1 --2- 172.21.15.36:0/1217674549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9970067290 0x7f99700a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:25.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.239+0000 7f99765d9700 1 -- 172.21.15.36:0/1217674549 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9970060ac0 msgr2=0x7f9970143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:25.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.239+0000 7f99765d9700 1 --2- 172.21.15.36:0/1217674549 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9970060ac0 0x7f9970143510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:25.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.239+0000 7f99765d9700 1 -- 172.21.15.36:0/1217674549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9970067290 msgr2=0x7f99700a3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:25.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.239+0000 7f99765d9700 1 --2- 172.21.15.36:0/1217674549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9970067290 0x7f99700a3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:25.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.239+0000 7f99765d9700 1 -- 172.21.15.36:0/1217674549 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9968009470 con 0x7f99700ae970 2022-06-17T12:55:25.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.239+0000 7f9975dd8700 1 --2- 172.21.15.36:0/1217674549 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9970060ac0 0x7f9970143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:25.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.239+0000 7f99755d7700 1 --2- 172.21.15.36:0/1217674549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9970067290 0x7f99700a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:25.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.243+0000 7f99765d9700 1 --2- 172.21.15.36:0/1217674549 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f99700ae970 0x7f99701528e0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7f99680032f0 tx=0x7f9968016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:25.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.243+0000 7f995effd700 1 -- 172.21.15.36:0/1217674549 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f996800b910 con 0x7f99700ae970 2022-06-17T12:55:25.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.243+0000 7f995effd700 1 -- 172.21.15.36:0/1217674549 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99680205e0 con 0x7f99700ae970 2022-06-17T12:55:25.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.243+0000 7f997803c700 1 -- 172.21.15.36:0/1217674549 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997009fcc0 con 0x7f99700ae970 2022-06-17T12:55:25.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.243+0000 7f995effd700 1 -- 172.21.15.36:0/1217674549 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f996800b640 con 0x7f99700ae970 2022-06-17T12:55:25.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.243+0000 7f997803c700 1 -- 172.21.15.36:0/1217674549 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9970152dd0 con 0x7f99700ae970 2022-06-17T12:55:25.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.243+0000 7f997803c700 1 -- 172.21.15.36:0/1217674549 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9970152620 con 0x7f99700ae970 2022-06-17T12:55:25.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.243+0000 7f995effd700 1 -- 172.21.15.36:0/1217674549 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f996801e070 con 0x7f99700ae970 2022-06-17T12:55:25.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.243+0000 7f995effd700 1 --2- 172.21.15.36:0/1217674549 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9950053a60 0x7f9950055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:25.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.243+0000 7f995effd700 1 -- 172.21.15.36:0/1217674549 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f996806cdd0 con 0x7f99700ae970 2022-06-17T12:55:25.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.243+0000 7f9975dd8700 1 --2- 172.21.15.36:0/1217674549 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9950053a60 0x7f9950055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:25.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.247+0000 7f9975dd8700 1 --2- 172.21.15.36:0/1217674549 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9950053a60 0x7f9950055f20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f996c00b000 tx=0x7f996c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:25.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.251+0000 7f995effd700 1 -- 172.21.15.36:0/1217674549 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9970152620 con 0x7f99700ae970 2022-06-17T12:55:25.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.379+0000 7f997803c700 1 -- 172.21.15.36:0/1217674549 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99700aedc0 con 0x7f99700ae970 2022-06-17T12:55:25.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.383+0000 7f995effd700 1 -- 172.21.15.36:0/1217674549 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f996801b020 con 0x7f99700ae970 2022-06-17T12:55:25.371 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:25.371 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":112,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:55:24.825487+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:55:23.324881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-06-17T12:55:24.417060+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:55:25.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.383+0000 7f997803c700 1 -- 172.21.15.36:0/1217674549 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9950053a60 msgr2=0x7f9950055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:25.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.383+0000 7f997803c700 1 --2- 172.21.15.36:0/1217674549 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9950053a60 0x7f9950055f20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f996c00b000 tx=0x7f996c009040).stop 2022-06-17T12:55:25.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.383+0000 7f997803c700 1 -- 172.21.15.36:0/1217674549 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f99700ae970 msgr2=0x7f99701528e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:25.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.383+0000 7f997803c700 1 --2- 172.21.15.36:0/1217674549 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f99700ae970 0x7f99701528e0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7f99680032f0 tx=0x7f9968016000).stop 2022-06-17T12:55:25.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.383+0000 7f997803c700 1 -- 172.21.15.36:0/1217674549 shutdown_connections 2022-06-17T12:55:25.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.383+0000 7f997803c700 1 --2- 172.21.15.36:0/1217674549 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9950053a60 0x7f9950055f20 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:25.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.383+0000 7f997803c700 1 --2- 172.21.15.36:0/1217674549 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f99700ae970 0x7f99701528e0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:25.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.383+0000 7f997803c700 1 --2- 172.21.15.36:0/1217674549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9970067290 0x7f99700a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:25.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.383+0000 7f997803c700 1 --2- 172.21.15.36:0/1217674549 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9970060ac0 0x7f9970143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:25.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.383+0000 7f997803c700 1 -- 172.21.15.36:0/1217674549 >> 172.21.15.36:0/1217674549 conn(0x7f99700b6870 msgr2=0x7f99701492c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:25.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.383+0000 7f997803c700 1 -- 172.21.15.36:0/1217674549 shutdown_connections 2022-06-17T12:55:25.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.383+0000 7f997803c700 1 -- 172.21.15.36:0/1217674549 wait complete. 2022-06-17T12:55:25.386 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-17T12:55:25.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.515+0000 7f1d6ffff700 1 Processor -- start 2022-06-17T12:55:25.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.515+0000 7f1d6ffff700 1 -- start start 2022-06-17T12:55:25.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.515+0000 7f1d6ffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d700ae360 0x7f1d70144ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:25.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.515+0000 7f1d6ffff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d700b2b60 con 0x7f1d700601c0 2022-06-17T12:55:25.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.515+0000 7f1d6ffff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d700b3230 con 0x7f1d70144fd0 2022-06-17T12:55:25.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.515+0000 7f1d6ffff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d700605a0 con 0x7f1d700ae360 2022-06-17T12:55:25.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.515+0000 7f1d6effd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f1d700601c0 0x7f1d701443d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:55974/0 (socket says 172.21.15.36:55974) 2022-06-17T12:55:25.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.515+0000 7f1d6effd700 1 -- 172.21.15.36:0/834999382 learned_addr learned my addr 172.21.15.36:0/834999382 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:25.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.515+0000 7f1d67fff700 1 --2- 172.21.15.36:0/834999382 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d700ae360 0x7f1d70144ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:25.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.515+0000 7f1d6e7fc700 1 -- 172.21.15.36:0/834999382 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4139368575 0 0) 0x7f1d700b2b60 con 0x7f1d700601c0 2022-06-17T12:55:25.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.515+0000 7f1d6e7fc700 1 -- 172.21.15.36:0/834999382 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d50003260 con 0x7f1d700601c0 2022-06-17T12:55:25.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.515+0000 7f1d6e7fc700 1 -- 172.21.15.36:0/834999382 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2883483942 0 0) 0x7f1d68001490 con 0x7f1d70144fd0 2022-06-17T12:55:25.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.515+0000 7f1d6e7fc700 1 -- 172.21.15.36:0/834999382 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d700b2b60 con 0x7f1d70144fd0 2022-06-17T12:55:25.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.515+0000 7f1d6e7fc700 1 -- 172.21.15.36:0/834999382 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 938901818 0 0) 0x7f1d50003260 con 0x7f1d700601c0 2022-06-17T12:55:25.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.515+0000 7f1d67fff700 1 -- 172.21.15.36:0/834999382 >> v1:172.21.15.70:6789/0 conn(0x7f1d70144fd0 legacy=0x7f1d7014aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:25.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.515+0000 7f1d67fff700 1 -- 172.21.15.36:0/834999382 >> v1:172.21.15.36:6789/0 conn(0x7f1d700601c0 legacy=0x7f1d701443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:25.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.515+0000 7f1d67fff700 1 -- 172.21.15.36:0/834999382 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d700662f0 con 0x7f1d700ae360 2022-06-17T12:55:25.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.515+0000 7f1d67fff700 1 --2- 172.21.15.36:0/834999382 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d700ae360 0x7f1d70144ae0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f1d60001110 tx=0x7f1d58006df0).ready entity=mon.2 client_cookie=afe47999c247b621 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:25.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.519+0000 7f1d6e7fc700 1 -- 172.21.15.36:0/834999382 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d60001660 con 0x7f1d700ae360 2022-06-17T12:55:25.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.519+0000 7f1d6e7fc700 1 -- 172.21.15.36:0/834999382 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d58000e90 con 0x7f1d700ae360 2022-06-17T12:55:25.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.519+0000 7f1d6e7fc700 1 -- 172.21.15.36:0/834999382 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d58012460 con 0x7f1d700ae360 2022-06-17T12:55:25.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.519+0000 7f1d6ffff700 1 -- 172.21.15.36:0/834999382 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d700ae360 msgr2=0x7f1d70144ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:25.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.519+0000 7f1d6ffff700 1 --2- 172.21.15.36:0/834999382 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d700ae360 0x7f1d70144ae0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f1d60001110 tx=0x7f1d58006df0).stop 2022-06-17T12:55:25.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.519+0000 7f1d6ffff700 1 -- 172.21.15.36:0/834999382 shutdown_connections 2022-06-17T12:55:25.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.519+0000 7f1d6ffff700 1 --2- 172.21.15.36:0/834999382 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d700ae360 0x7f1d70144ae0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:25.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.519+0000 7f1d6ffff700 1 -- 172.21.15.36:0/834999382 >> 172.21.15.36:0/834999382 conn(0x7f1d70060990 msgr2=0x7f1d7005f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:25.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.519+0000 7f1d6ffff700 1 -- 172.21.15.36:0/834999382 shutdown_connections 2022-06-17T12:55:25.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.519+0000 7f1d6ffff700 1 -- 172.21.15.36:0/834999382 wait complete. 2022-06-17T12:55:25.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.519+0000 7f1d6ffff700 1 Processor -- start 2022-06-17T12:55:25.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.519+0000 7f1d6ffff700 1 -- start start 2022-06-17T12:55:25.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d6ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d700601c0 0x7f1d70155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:25.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d6ffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d700ae360 0x7f1d70062660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:25.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d6ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d70144fd0 0x7f1d70140550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:25.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d6ffff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d700b0eb0 con 0x7f1d70144fd0 2022-06-17T12:55:25.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d6ffff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7013e850 con 0x7f1d700601c0 2022-06-17T12:55:25.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d6ffff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d700614b0 con 0x7f1d700ae360 2022-06-17T12:55:25.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d6f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d70144fd0 0x7f1d70140550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:25.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d6effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d700601c0 0x7f1d70155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:25.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d6effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d700601c0 0x7f1d70155b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:58832/0 (socket says 172.21.15.36:58832) 2022-06-17T12:55:25.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d67fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d700ae360 0x7f1d70062660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:25.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d6effd700 1 -- 172.21.15.36:0/1577721447 learned_addr learned my addr 172.21.15.36:0/1577721447 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:25.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d6effd700 1 -- 172.21.15.36:0/1577721447 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d700ae360 msgr2=0x7f1d70062660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:25.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d6effd700 1 --2- 172.21.15.36:0/1577721447 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d700ae360 0x7f1d70062660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:25.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d6effd700 1 -- 172.21.15.36:0/1577721447 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d70144fd0 msgr2=0x7f1d70140550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:25.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d6effd700 1 --2- 172.21.15.36:0/1577721447 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d70144fd0 0x7f1d70140550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:25.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d6effd700 1 -- 172.21.15.36:0/1577721447 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5800b180 con 0x7f1d700601c0 2022-06-17T12:55:25.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d67fff700 1 --2- 172.21.15.36:0/1577721447 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d700ae360 0x7f1d70062660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:25.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d6f7fe700 1 --2- 172.21.15.36:0/1577721447 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d70144fd0 0x7f1d70140550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:25.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d6effd700 1 --2- 172.21.15.36:0/1577721447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d700601c0 0x7f1d70155b30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f1d70116170 tx=0x7f1d60001720).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:25.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d6cff9700 1 -- 172.21.15.36:0/1577721447 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d60007170 con 0x7f1d700601c0 2022-06-17T12:55:25.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d6cff9700 1 -- 172.21.15.36:0/1577721447 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d600072d0 con 0x7f1d700601c0 2022-06-17T12:55:25.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d6ffff700 1 -- 172.21.15.36:0/1577721447 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d700616e0 con 0x7f1d700601c0 2022-06-17T12:55:25.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d6cff9700 1 -- 172.21.15.36:0/1577721447 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d60004710 con 0x7f1d700601c0 2022-06-17T12:55:25.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d6ffff700 1 -- 172.21.15.36:0/1577721447 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d70062d80 con 0x7f1d700601c0 2022-06-17T12:55:25.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d6ffff700 1 -- 172.21.15.36:0/1577721447 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d70061b80 con 0x7f1d700601c0 2022-06-17T12:55:25.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.523+0000 7f1d6cff9700 1 -- 172.21.15.36:0/1577721447 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f1d6001b070 con 0x7f1d700601c0 2022-06-17T12:55:25.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.527+0000 7f1d6cff9700 1 --2- 172.21.15.36:0/1577721447 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1d54053a40 0x7f1d54055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:25.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.527+0000 7f1d6cff9700 1 -- 172.21.15.36:0/1577721447 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1d6006b680 con 0x7f1d700601c0 2022-06-17T12:55:25.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.527+0000 7f1d67fff700 1 --2- 172.21.15.36:0/1577721447 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1d54053a40 0x7f1d54055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:25.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.527+0000 7f1d67fff700 1 --2- 172.21.15.36:0/1577721447 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1d54053a40 0x7f1d54055f00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f1d5801a000 tx=0x7f1d58014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:25.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.535+0000 7f1d6cff9700 1 -- 172.21.15.36:0/1577721447 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1d600423e0 con 0x7f1d700601c0 2022-06-17T12:55:25.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:25.699+0000 7f1d6ffff700 1 -- 172.21.15.36:0/1577721447 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1d70066590 con 0x7f1d700601c0 2022-06-17T12:55:26.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.583+0000 7f1d6cff9700 1 -- 172.21.15.36:0/1577721447 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v60) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1d60045a00 con 0x7f1d700601c0 2022-06-17T12:55:26.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.587+0000 7f1d6ffff700 1 -- 172.21.15.36:0/1577721447 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1d54053a40 msgr2=0x7f1d54055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:26.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.587+0000 7f1d6ffff700 1 --2- 172.21.15.36:0/1577721447 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1d54053a40 0x7f1d54055f00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f1d5801a000 tx=0x7f1d58014000).stop 2022-06-17T12:55:26.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.587+0000 7f1d6ffff700 1 -- 172.21.15.36:0/1577721447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d700601c0 msgr2=0x7f1d70155b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:26.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.587+0000 7f1d6ffff700 1 --2- 172.21.15.36:0/1577721447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d700601c0 0x7f1d70155b30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f1d70116170 tx=0x7f1d60001720).stop 2022-06-17T12:55:26.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.587+0000 7f1d6ffff700 1 -- 172.21.15.36:0/1577721447 shutdown_connections 2022-06-17T12:55:26.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.587+0000 7f1d6ffff700 1 --2- 172.21.15.36:0/1577721447 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1d54053a40 0x7f1d54055f00 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:26.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.587+0000 7f1d6ffff700 1 --2- 172.21.15.36:0/1577721447 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d70144fd0 0x7f1d70140550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:26.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.587+0000 7f1d6ffff700 1 --2- 172.21.15.36:0/1577721447 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d700ae360 0x7f1d70062660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:26.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.587+0000 7f1d6ffff700 1 --2- 172.21.15.36:0/1577721447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d700601c0 0x7f1d70155b30 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:26.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.587+0000 7f1d6ffff700 1 -- 172.21.15.36:0/1577721447 >> 172.21.15.36:0/1577721447 conn(0x7f1d70060990 msgr2=0x7f1d700b6390 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:26.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.587+0000 7f1d6ffff700 1 -- 172.21.15.36:0/1577721447 shutdown_connections 2022-06-17T12:55:26.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.587+0000 7f1d6ffff700 1 -- 172.21.15.36:0/1577721447 wait complete. 2022-06-17T12:55:26.598 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:55:26.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.719+0000 7fe47b942700 1 Processor -- start 2022-06-17T12:55:26.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.719+0000 7fe47b942700 1 -- start start 2022-06-17T12:55:26.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe47b942700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe474062770 0x7fe474062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:26.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe47b942700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe4740ad330 0x7fe4740ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:26.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe47b942700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4740b0ba0 con 0x7fe4741444f0 2022-06-17T12:55:26.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe47b942700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4740ae820 con 0x7fe4740ad330 2022-06-17T12:55:26.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe47b942700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe474061030 con 0x7fe474062770 2022-06-17T12:55:26.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe479edf700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe4740ad330 0x7fe4740ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:26.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe479edf700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe4740ad330 0x7fe4740ad7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38462/0 (socket says 172.21.15.36:38462) 2022-06-17T12:55:26.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe479edf700 1 -- 172.21.15.36:0/4210153746 learned_addr learned my addr 172.21.15.36:0/4210153746 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:26.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe4796de700 1 --2- 172.21.15.36:0/4210153746 >> v2:172.21.15.70:3300/0 conn(0x7fe474062770 0x7fe474062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:26.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe479edf700 1 -- 172.21.15.36:0/4210153746 >> v2:172.21.15.70:3300/0 conn(0x7fe474062770 msgr2=0x7fe474062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:26.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe479edf700 1 --2- 172.21.15.36:0/4210153746 >> v2:172.21.15.70:3300/0 conn(0x7fe474062770 0x7fe474062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:26.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe479edf700 1 -- 172.21.15.36:0/4210153746 >> v1:172.21.15.70:6789/0 conn(0x7fe4741444f0 legacy=0x7fe474144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:26.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe479edf700 1 -- 172.21.15.36:0/4210153746 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe474147e90 con 0x7fe4740ad330 2022-06-17T12:55:26.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe4796de700 1 --2- 172.21.15.36:0/4210153746 >> v2:172.21.15.70:3300/0 conn(0x7fe474062770 0x7fe474062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:26.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe479edf700 1 --2- 172.21.15.36:0/4210153746 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4740ad330 0x7fe4740ad7c0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 rx=0x7fe474061170 tx=0x7fe46c009290).ready entity=mon.0 client_cookie=9087e0764f25e58d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:26.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe467fff700 1 -- 172.21.15.36:0/4210153746 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe46c00b830 con 0x7fe4740ad330 2022-06-17T12:55:26.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe467fff700 1 -- 172.21.15.36:0/4210153746 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46c00b990 con 0x7fe4740ad330 2022-06-17T12:55:26.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe467fff700 1 -- 172.21.15.36:0/4210153746 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe46c00bb00 con 0x7fe4740ad330 2022-06-17T12:55:26.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe47b942700 1 -- 172.21.15.36:0/4210153746 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4740ad330 msgr2=0x7fe4740ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:26.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe47b942700 1 --2- 172.21.15.36:0/4210153746 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4740ad330 0x7fe4740ad7c0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 rx=0x7fe474061170 tx=0x7fe46c009290).stop 2022-06-17T12:55:26.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe47b942700 1 -- 172.21.15.36:0/4210153746 shutdown_connections 2022-06-17T12:55:26.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe47b942700 1 --2- 172.21.15.36:0/4210153746 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4740ad330 0x7fe4740ad7c0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:26.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe47b942700 1 --2- 172.21.15.36:0/4210153746 >> v2:172.21.15.70:3300/0 conn(0x7fe474062770 0x7fe474062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:26.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe47b942700 1 -- 172.21.15.36:0/4210153746 >> 172.21.15.36:0/4210153746 conn(0x7fe4740b53e0 msgr2=0x7fe4740b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:26.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe47b942700 1 -- 172.21.15.36:0/4210153746 shutdown_connections 2022-06-17T12:55:26.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.723+0000 7fe47b942700 1 -- 172.21.15.36:0/4210153746 wait complete. 2022-06-17T12:55:26.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.727+0000 7fe47b942700 1 Processor -- start 2022-06-17T12:55:26.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.727+0000 7fe47b942700 1 -- start start 2022-06-17T12:55:26.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.727+0000 7fe47b942700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe474062770 0x7fe47412e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:26.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.727+0000 7fe47b942700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4740ad330 0x7fe47414e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:26.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.727+0000 7fe47b942700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4741444f0 0x7fe47412f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:26.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.727+0000 7fe47b942700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47414deb0 con 0x7fe4741444f0 2022-06-17T12:55:26.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.727+0000 7fe47b942700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4740635c0 con 0x7fe474062770 2022-06-17T12:55:26.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.727+0000 7fe47b942700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe474063860 con 0x7fe4740ad330 2022-06-17T12:55:26.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.727+0000 7fe479edf700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4741444f0 0x7fe47412f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:26.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.727+0000 7fe479edf700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4741444f0 0x7fe47412f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38466/0 (socket says 172.21.15.36:38466) 2022-06-17T12:55:26.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.727+0000 7fe479edf700 1 -- 172.21.15.36:0/3681935439 learned_addr learned my addr 172.21.15.36:0/3681935439 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:26.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.727+0000 7fe4796de700 1 --2- 172.21.15.36:0/3681935439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe474062770 0x7fe47412e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:26.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.727+0000 7fe478edd700 1 --2- 172.21.15.36:0/3681935439 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4740ad330 0x7fe47414e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:26.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.727+0000 7fe479edf700 1 -- 172.21.15.36:0/3681935439 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4740ad330 msgr2=0x7fe47414e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:26.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.727+0000 7fe479edf700 1 --2- 172.21.15.36:0/3681935439 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4740ad330 0x7fe47414e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:26.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.727+0000 7fe479edf700 1 -- 172.21.15.36:0/3681935439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe474062770 msgr2=0x7fe47412e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:26.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.727+0000 7fe479edf700 1 --2- 172.21.15.36:0/3681935439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe474062770 0x7fe47412e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:26.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.727+0000 7fe479edf700 1 -- 172.21.15.36:0/3681935439 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46c009530 con 0x7fe4741444f0 2022-06-17T12:55:26.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.727+0000 7fe479edf700 1 --2- 172.21.15.36:0/3681935439 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4741444f0 0x7fe47412f9d0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7fe474061170 tx=0x7fe46c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:26.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.727+0000 7fe4667fc700 1 -- 172.21.15.36:0/3681935439 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe46c021630 con 0x7fe4741444f0 2022-06-17T12:55:26.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.727+0000 7fe4667fc700 1 -- 172.21.15.36:0/3681935439 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46c021790 con 0x7fe4741444f0 2022-06-17T12:55:26.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.727+0000 7fe47b942700 1 -- 172.21.15.36:0/3681935439 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe47414db70 con 0x7fe4741444f0 2022-06-17T12:55:26.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.727+0000 7fe4667fc700 1 -- 172.21.15.36:0/3681935439 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe46c021af0 con 0x7fe4741444f0 2022-06-17T12:55:26.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.727+0000 7fe47b942700 1 -- 172.21.15.36:0/3681935439 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe47414bf80 con 0x7fe4741444f0 2022-06-17T12:55:26.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.731+0000 7fe47b942700 1 -- 172.21.15.36:0/3681935439 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4740b1c20 con 0x7fe4741444f0 2022-06-17T12:55:26.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.731+0000 7fe4667fc700 1 -- 172.21.15.36:0/3681935439 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe46c027020 con 0x7fe4741444f0 2022-06-17T12:55:26.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.731+0000 7fe4667fc700 1 --2- 172.21.15.36:0/3681935439 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe45c0539d0 0x7fe45c055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:26.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.731+0000 7fe4667fc700 1 -- 172.21.15.36:0/3681935439 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe46c06d8b0 con 0x7fe4741444f0 2022-06-17T12:55:26.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.731+0000 7fe4796de700 1 --2- 172.21.15.36:0/3681935439 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe45c0539d0 0x7fe45c055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:26.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.731+0000 7fe4796de700 1 --2- 172.21.15.36:0/3681935439 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe45c0539d0 0x7fe45c055e90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7fe46800c000 tx=0x7fe468008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:26.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.739+0000 7fe4667fc700 1 -- 172.21.15.36:0/3681935439 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe46c0445a0 con 0x7fe4741444f0 2022-06-17T12:55:26.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.903+0000 7fe47b942700 1 -- 172.21.15.36:0/3681935439 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe474057410 con 0x7fe4741444f0 2022-06-17T12:55:26.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.903+0000 7fe4667fc700 1 -- 172.21.15.36:0/3681935439 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+2142 (secure 0 0 0) 0x7fe46c015070 con 0x7fe4741444f0 2022-06-17T12:55:26.900 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:26.900 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":60,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1161585109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1161585109},{"type":"v1","addr":"172.21.15.70:6833","nonce":1161585109}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":57}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:55:24.825976+0000","modified":"2022-06-17T12:55:26.574711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7531":{"gid":7531,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/1342585509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":1342585509},{"type":"v1","addr":"172.21.15.36:6835","nonce":1342585509}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2022-06-17T12:55:26.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.903+0000 7fe47b942700 1 -- 172.21.15.36:0/3681935439 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe45c0539d0 msgr2=0x7fe45c055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:26.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.903+0000 7fe47b942700 1 --2- 172.21.15.36:0/3681935439 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe45c0539d0 0x7fe45c055e90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7fe46800c000 tx=0x7fe468008040).stop 2022-06-17T12:55:26.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.903+0000 7fe47b942700 1 -- 172.21.15.36:0/3681935439 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4741444f0 msgr2=0x7fe47412f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:26.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.903+0000 7fe47b942700 1 --2- 172.21.15.36:0/3681935439 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4741444f0 0x7fe47412f9d0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7fe474061170 tx=0x7fe46c016000).stop 2022-06-17T12:55:26.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.903+0000 7fe47b942700 1 -- 172.21.15.36:0/3681935439 shutdown_connections 2022-06-17T12:55:26.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.903+0000 7fe47b942700 1 --2- 172.21.15.36:0/3681935439 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe45c0539d0 0x7fe45c055e90 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:26.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.903+0000 7fe47b942700 1 --2- 172.21.15.36:0/3681935439 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4741444f0 0x7fe47412f9d0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:26.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.903+0000 7fe47b942700 1 --2- 172.21.15.36:0/3681935439 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4740ad330 0x7fe47414e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:26.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.903+0000 7fe47b942700 1 --2- 172.21.15.36:0/3681935439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe474062770 0x7fe47412e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:26.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.903+0000 7fe47b942700 1 -- 172.21.15.36:0/3681935439 >> 172.21.15.36:0/3681935439 conn(0x7fe4740b53e0 msgr2=0x7fe4741305c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:26.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.907+0000 7fe47b942700 1 -- 172.21.15.36:0/3681935439 shutdown_connections 2022-06-17T12:55:26.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:26.907+0000 7fe47b942700 1 -- 172.21.15.36:0/3681935439 wait complete. 2022-06-17T12:55:26.912 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 60 2022-06-17T12:55:26.920 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:55:27.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.047+0000 7f4306749700 1 Processor -- start 2022-06-17T12:55:27.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.047+0000 7f4306749700 1 -- start start 2022-06-17T12:55:27.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.047+0000 7f4306749700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4300063730 0x7f4300065e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:27.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.047+0000 7f4306749700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4300063b10 con 0x7f43000b2900 2022-06-17T12:55:27.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.047+0000 7f4306749700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4300065430 con 0x7f43000b1930 2022-06-17T12:55:27.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.047+0000 7f4306749700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43000656d0 con 0x7f4300063730 2022-06-17T12:55:27.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.047+0000 7f4305f48700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f43000b2900 0x7f430013eae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56006/0 (socket says 172.21.15.36:56006) 2022-06-17T12:55:27.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.047+0000 7f4305f48700 1 -- 172.21.15.36:0/2948052547 learned_addr learned my addr 172.21.15.36:0/2948052547 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:27.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.047+0000 7f4305747700 1 --2- 172.21.15.36:0/2948052547 >> v2:172.21.15.70:3300/0 conn(0x7f4300063730 0x7f4300065e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:27.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.047+0000 7f42fffff700 1 -- 172.21.15.36:0/2948052547 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1299559313 0 0) 0x7f4300063b10 con 0x7f43000b2900 2022-06-17T12:55:27.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.047+0000 7f42fffff700 1 -- 172.21.15.36:0/2948052547 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42e0003260 con 0x7f43000b2900 2022-06-17T12:55:27.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.047+0000 7f4305747700 1 -- 172.21.15.36:0/2948052547 >> v1:172.21.15.70:6789/0 conn(0x7f43000b1930 legacy=0x7f43001431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:27.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.047+0000 7f4305747700 1 -- 172.21.15.36:0/2948052547 >> v1:172.21.15.36:6789/0 conn(0x7f43000b2900 legacy=0x7f430013eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:27.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.047+0000 7f4305747700 1 -- 172.21.15.36:0/2948052547 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4300065970 con 0x7f4300063730 2022-06-17T12:55:27.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.047+0000 7f4305747700 1 --2- 172.21.15.36:0/2948052547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4300063730 0x7f4300065e10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f42e80032f0 tx=0x7f42e80091d0).ready entity=mon.1 client_cookie=c6ba111c288da26d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:27.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.047+0000 7f42fffff700 1 -- 172.21.15.36:0/2948052547 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42e800b710 con 0x7f4300063730 2022-06-17T12:55:27.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.047+0000 7f42fffff700 1 -- 172.21.15.36:0/2948052547 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e800b870 con 0x7f4300063730 2022-06-17T12:55:27.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.047+0000 7f42fffff700 1 -- 172.21.15.36:0/2948052547 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42e800b9e0 con 0x7f4300063730 2022-06-17T12:55:27.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4306749700 1 -- 172.21.15.36:0/2948052547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4300063730 msgr2=0x7f4300065e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:27.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4306749700 1 --2- 172.21.15.36:0/2948052547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4300063730 0x7f4300065e10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f42e80032f0 tx=0x7f42e80091d0).stop 2022-06-17T12:55:27.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4306749700 1 -- 172.21.15.36:0/2948052547 shutdown_connections 2022-06-17T12:55:27.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4306749700 1 --2- 172.21.15.36:0/2948052547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4300063730 0x7f4300065e10 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:27.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4306749700 1 -- 172.21.15.36:0/2948052547 >> 172.21.15.36:0/2948052547 conn(0x7f43000614b0 msgr2=0x7f43000618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:27.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4306749700 1 -- 172.21.15.36:0/2948052547 shutdown_connections 2022-06-17T12:55:27.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4306749700 1 -- 172.21.15.36:0/2948052547 wait complete. 2022-06-17T12:55:27.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4306749700 1 Processor -- start 2022-06-17T12:55:27.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4306749700 1 -- start start 2022-06-17T12:55:27.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4306749700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4300063730 0x7f430010f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:27.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4306749700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f43000b1930 0x7f43000aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:27.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4306749700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f43000b2900 0x7f43001486c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:27.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4306749700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430014c0b0 con 0x7f43000b1930 2022-06-17T12:55:27.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4306749700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430014ad50 con 0x7f4300063730 2022-06-17T12:55:27.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4306749700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f430014aaf0 con 0x7f43000b2900 2022-06-17T12:55:27.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4304f46700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f43000b1930 0x7f43000aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:27.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4304f46700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f43000b1930 0x7f43000aeee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38478/0 (socket says 172.21.15.36:38478) 2022-06-17T12:55:27.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4304f46700 1 -- 172.21.15.36:0/766667499 learned_addr learned my addr 172.21.15.36:0/766667499 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:27.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4305747700 1 --2- 172.21.15.36:0/766667499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4300063730 0x7f430010f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:27.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4305f48700 1 --2- 172.21.15.36:0/766667499 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f43000b2900 0x7f43001486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:27.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4304f46700 1 -- 172.21.15.36:0/766667499 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f43000b2900 msgr2=0x7f43001486c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:27.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4304f46700 1 --2- 172.21.15.36:0/766667499 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f43000b2900 0x7f43001486c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:27.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4304f46700 1 -- 172.21.15.36:0/766667499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4300063730 msgr2=0x7f430010f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:27.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4304f46700 1 --2- 172.21.15.36:0/766667499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4300063730 0x7f430010f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:27.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4304f46700 1 -- 172.21.15.36:0/766667499 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e8009470 con 0x7f43000b1930 2022-06-17T12:55:27.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4304f46700 1 --2- 172.21.15.36:0/766667499 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f43000b1930 0x7f43000aeee0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 rx=0x7f43000688a0 tx=0x7f42f0007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:27.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f42fe7fc700 1 -- 172.21.15.36:0/766667499 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42f000ce30 con 0x7f43000b1930 2022-06-17T12:55:27.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f42fe7fc700 1 -- 172.21.15.36:0/766667499 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f000f470 con 0x7f43000b1930 2022-06-17T12:55:27.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f42fe7fc700 1 -- 172.21.15.36:0/766667499 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42f000f7d0 con 0x7f43000b1930 2022-06-17T12:55:27.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4306749700 1 -- 172.21.15.36:0/766667499 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430014af70 con 0x7f43000b1930 2022-06-17T12:55:27.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.051+0000 7f4306749700 1 -- 172.21.15.36:0/766667499 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43000afe00 con 0x7f43000b1930 2022-06-17T12:55:27.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.055+0000 7f4306749700 1 -- 172.21.15.36:0/766667499 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f430014d3d0 con 0x7f43000b1930 2022-06-17T12:55:27.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.055+0000 7f42fe7fc700 1 -- 172.21.15.36:0/766667499 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f42f0019030 con 0x7f43000b1930 2022-06-17T12:55:27.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.055+0000 7f42fe7fc700 1 --2- 172.21.15.36:0/766667499 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f42e4053920 0x7f42e4055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:27.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.055+0000 7f42fe7fc700 1 -- 172.21.15.36:0/766667499 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f42f006b550 con 0x7f43000b1930 2022-06-17T12:55:27.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.055+0000 7f4305747700 1 --2- 172.21.15.36:0/766667499 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f42e4053920 0x7f42e4055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:27.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.055+0000 7f4305747700 1 --2- 172.21.15.36:0/766667499 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f42e4053920 0x7f42e4055de0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f42e8019000 tx=0x7f42e8016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:27.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.063+0000 7f42fe7fc700 1 -- 172.21.15.36:0/766667499 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f42f00421e0 con 0x7f43000b1930 2022-06-17T12:55:27.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.191+0000 7f4306749700 1 -- 172.21.15.36:0/766667499 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43000af570 con 0x7f43000b1930 2022-06-17T12:55:27.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.191+0000 7f42fe7fc700 1 -- 172.21.15.36:0/766667499 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f42f0045800 con 0x7f43000b1930 2022-06-17T12:55:27.180 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:27.181 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":113,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:55:25.828833+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:55:23.324881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-06-17T12:55:24.417060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:55:27.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.195+0000 7f4306749700 1 -- 172.21.15.36:0/766667499 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f42e4053920 msgr2=0x7f42e4055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:27.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.195+0000 7f4306749700 1 --2- 172.21.15.36:0/766667499 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f42e4053920 0x7f42e4055de0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f42e8019000 tx=0x7f42e8016000).stop 2022-06-17T12:55:27.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.195+0000 7f4306749700 1 -- 172.21.15.36:0/766667499 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f43000b1930 msgr2=0x7f43000aeee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:27.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.195+0000 7f4306749700 1 --2- 172.21.15.36:0/766667499 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f43000b1930 0x7f43000aeee0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 rx=0x7f43000688a0 tx=0x7f42f0007c50).stop 2022-06-17T12:55:27.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.195+0000 7f4306749700 1 -- 172.21.15.36:0/766667499 shutdown_connections 2022-06-17T12:55:27.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.195+0000 7f4306749700 1 --2- 172.21.15.36:0/766667499 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f42e4053920 0x7f42e4055de0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:27.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.195+0000 7f4306749700 1 --2- 172.21.15.36:0/766667499 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f43000b2900 0x7f43001486c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:27.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.195+0000 7f4306749700 1 --2- 172.21.15.36:0/766667499 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f43000b1930 0x7f43000aeee0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:27.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.195+0000 7f4306749700 1 --2- 172.21.15.36:0/766667499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4300063730 0x7f430010f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:27.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.195+0000 7f4306749700 1 -- 172.21.15.36:0/766667499 >> 172.21.15.36:0/766667499 conn(0x7f43000614b0 msgr2=0x7f4300143100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:27.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.195+0000 7f4306749700 1 -- 172.21.15.36:0/766667499 shutdown_connections 2022-06-17T12:55:27.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.195+0000 7f4306749700 1 -- 172.21.15.36:0/766667499 wait complete. 2022-06-17T12:55:27.198 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:55:27.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.331+0000 7f42551ee700 1 Processor -- start 2022-06-17T12:55:27.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.331+0000 7f42551ee700 1 -- start start 2022-06-17T12:55:27.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.331+0000 7f42551ee700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4250062470 0x7f42500b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:27.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.331+0000 7f42551ee700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42500b83f0 con 0x7f42500b4820 2022-06-17T12:55:27.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.331+0000 7f42551ee700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42500b1ff0 con 0x7f4250143a90 2022-06-17T12:55:27.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.331+0000 7f42551ee700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4250143f00 con 0x7f4250062470 2022-06-17T12:55:27.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.331+0000 7f424ffff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f42500b4820 0x7f42500b7ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56014/0 (socket says 172.21.15.36:56014) 2022-06-17T12:55:27.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.331+0000 7f424ffff700 1 -- 172.21.15.36:0/292256956 learned_addr learned my addr 172.21.15.36:0/292256956 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:27.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.331+0000 7f4247fff700 1 --2- 172.21.15.36:0/292256956 >> v2:172.21.15.70:3300/0 conn(0x7f4250062470 0x7f42500b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:27.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.331+0000 7f424f7fe700 1 -- 172.21.15.36:0/292256956 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1178873241 0 0) 0x7f42500b83f0 con 0x7f42500b4820 2022-06-17T12:55:27.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.331+0000 7f424f7fe700 1 -- 172.21.15.36:0/292256956 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4230003260 con 0x7f42500b4820 2022-06-17T12:55:27.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.331+0000 7f424f7fe700 1 -- 172.21.15.36:0/292256956 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4092068246 0 0) 0x7f42500b1ff0 con 0x7f4250143a90 2022-06-17T12:55:27.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.331+0000 7f424f7fe700 1 -- 172.21.15.36:0/292256956 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42500b83f0 con 0x7f4250143a90 2022-06-17T12:55:27.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.331+0000 7f4247fff700 1 -- 172.21.15.36:0/292256956 >> v1:172.21.15.70:6789/0 conn(0x7f4250143a90 legacy=0x7f4250144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:27.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.331+0000 7f4247fff700 1 -- 172.21.15.36:0/292256956 >> v1:172.21.15.36:6789/0 conn(0x7f42500b4820 legacy=0x7f42500b7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:27.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.331+0000 7f4247fff700 1 -- 172.21.15.36:0/292256956 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42501430b0 con 0x7f4250062470 2022-06-17T12:55:27.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.331+0000 7f4247fff700 1 --2- 172.21.15.36:0/292256956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4250062470 0x7f42500b4c00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f4238002f70 tx=0x7f4238006df0).ready entity=mon.1 client_cookie=ef89c2e2bc9004d0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:27.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f424f7fe700 1 -- 172.21.15.36:0/292256956 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f423800d540 con 0x7f4250062470 2022-06-17T12:55:27.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f424f7fe700 1 -- 172.21.15.36:0/292256956 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4238000e90 con 0x7f4250062470 2022-06-17T12:55:27.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f424f7fe700 1 -- 172.21.15.36:0/292256956 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f423800d890 con 0x7f4250062470 2022-06-17T12:55:27.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f42551ee700 1 -- 172.21.15.36:0/292256956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4250062470 msgr2=0x7f42500b4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:27.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f42551ee700 1 --2- 172.21.15.36:0/292256956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4250062470 0x7f42500b4c00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f4238002f70 tx=0x7f4238006df0).stop 2022-06-17T12:55:27.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f42551ee700 1 -- 172.21.15.36:0/292256956 shutdown_connections 2022-06-17T12:55:27.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f42551ee700 1 --2- 172.21.15.36:0/292256956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4250062470 0x7f42500b4c00 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:27.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f42551ee700 1 -- 172.21.15.36:0/292256956 >> 172.21.15.36:0/292256956 conn(0x7f425005c550 msgr2=0x7f42500b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:27.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f42551ee700 1 -- 172.21.15.36:0/292256956 shutdown_connections 2022-06-17T12:55:27.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f42551ee700 1 -- 172.21.15.36:0/292256956 wait complete. 2022-06-17T12:55:27.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f42551ee700 1 Processor -- start 2022-06-17T12:55:27.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f42551ee700 1 -- start start 2022-06-17T12:55:27.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f42551ee700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4250062470 0x7f425013f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:27.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f42551ee700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42500b4820 0x7f42501467e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:27.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f42551ee700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4250143a90 0x7f4250132010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:27.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f42551ee700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4250065b70 con 0x7f4250062470 2022-06-17T12:55:27.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f42551ee700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4250065e10 con 0x7f4250143a90 2022-06-17T12:55:27.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f42551ee700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4250155ad0 con 0x7f42500b4820 2022-06-17T12:55:27.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f424ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4250062470 0x7f425013f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:27.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f424ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4250062470 0x7f425013f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38490/0 (socket says 172.21.15.36:38490) 2022-06-17T12:55:27.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f424ffff700 1 -- 172.21.15.36:0/3391414546 learned_addr learned my addr 172.21.15.36:0/3391414546 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:27.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f42549ed700 1 --2- 172.21.15.36:0/3391414546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4250143a90 0x7f4250132010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:27.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f4247fff700 1 --2- 172.21.15.36:0/3391414546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42500b4820 0x7f42501467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:27.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f424ffff700 1 -- 172.21.15.36:0/3391414546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42500b4820 msgr2=0x7f42501467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:27.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f424ffff700 1 --2- 172.21.15.36:0/3391414546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42500b4820 0x7f42501467e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:27.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f424ffff700 1 -- 172.21.15.36:0/3391414546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4250143a90 msgr2=0x7f4250132010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:27.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f424ffff700 1 --2- 172.21.15.36:0/3391414546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4250143a90 0x7f4250132010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:27.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f424ffff700 1 -- 172.21.15.36:0/3391414546 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423800b180 con 0x7f4250062470 2022-06-17T12:55:27.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.335+0000 7f42549ed700 1 --2- 172.21.15.36:0/3391414546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4250143a90 0x7f4250132010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:27.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.339+0000 7f424ffff700 1 --2- 172.21.15.36:0/3391414546 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4250062470 0x7f425013f5c0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7f42400011d0 tx=0x7f4240007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:27.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.339+0000 7f424dffb700 1 -- 172.21.15.36:0/3391414546 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f424000ce70 con 0x7f4250062470 2022-06-17T12:55:27.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.339+0000 7f424dffb700 1 -- 172.21.15.36:0/3391414546 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42400072e0 con 0x7f4250062470 2022-06-17T12:55:27.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.339+0000 7f42551ee700 1 -- 172.21.15.36:0/3391414546 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42500655f0 con 0x7f4250062470 2022-06-17T12:55:27.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.339+0000 7f424dffb700 1 -- 172.21.15.36:0/3391414546 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4240005800 con 0x7f4250062470 2022-06-17T12:55:27.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.339+0000 7f42551ee700 1 -- 172.21.15.36:0/3391414546 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4250155f70 con 0x7f4250062470 2022-06-17T12:55:27.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.339+0000 7f42551ee700 1 -- 172.21.15.36:0/3391414546 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4250155c90 con 0x7f4250062470 2022-06-17T12:55:27.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.339+0000 7f424dffb700 1 -- 172.21.15.36:0/3391414546 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f424001a060 con 0x7f4250062470 2022-06-17T12:55:27.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.339+0000 7f424dffb700 1 --2- 172.21.15.36:0/3391414546 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4234053920 0x7f4234055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:27.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.339+0000 7f424dffb700 1 -- 172.21.15.36:0/3391414546 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f424006b870 con 0x7f4250062470 2022-06-17T12:55:27.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.339+0000 7f4247fff700 1 --2- 172.21.15.36:0/3391414546 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4234053920 0x7f4234055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:27.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.343+0000 7f4247fff700 1 --2- 172.21.15.36:0/3391414546 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4234053920 0x7f4234055de0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f4238018000 tx=0x7f4238014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:27.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.347+0000 7f424dffb700 1 -- 172.21.15.36:0/3391414546 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4240042670 con 0x7f4250062470 2022-06-17T12:55:27.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.515+0000 7f42551ee700 1 -- 172.21.15.36:0/3391414546 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f4250143350 con 0x7f4250062470 2022-06-17T12:55:27.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.523+0000 7f424dffb700 1 -- 172.21.15.36:0/3391414546 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v38) v1 ==== 184+0+0 (secure 0 0 0) 0x7f4240018020 con 0x7f4250062470 2022-06-17T12:55:27.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.523+0000 7f42551ee700 1 -- 172.21.15.36:0/3391414546 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4234053920 msgr2=0x7f4234055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:27.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.523+0000 7f42551ee700 1 --2- 172.21.15.36:0/3391414546 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4234053920 0x7f4234055de0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f4238018000 tx=0x7f4238014000).stop 2022-06-17T12:55:27.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.523+0000 7f42551ee700 1 -- 172.21.15.36:0/3391414546 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4250062470 msgr2=0x7f425013f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:27.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.523+0000 7f42551ee700 1 --2- 172.21.15.36:0/3391414546 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4250062470 0x7f425013f5c0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7f42400011d0 tx=0x7f4240007b50).stop 2022-06-17T12:55:27.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.523+0000 7f42551ee700 1 -- 172.21.15.36:0/3391414546 shutdown_connections 2022-06-17T12:55:27.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.523+0000 7f42551ee700 1 --2- 172.21.15.36:0/3391414546 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4234053920 0x7f4234055de0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:27.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.523+0000 7f42551ee700 1 --2- 172.21.15.36:0/3391414546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4250143a90 0x7f4250132010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:27.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.523+0000 7f42551ee700 1 --2- 172.21.15.36:0/3391414546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42500b4820 0x7f42501467e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:27.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.523+0000 7f42551ee700 1 --2- 172.21.15.36:0/3391414546 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4250062470 0x7f425013f5c0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:27.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.523+0000 7f42551ee700 1 -- 172.21.15.36:0/3391414546 >> 172.21.15.36:0/3391414546 conn(0x7f425005c550 msgr2=0x7f42500b75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:27.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.523+0000 7f42551ee700 1 -- 172.21.15.36:0/3391414546 shutdown_connections 2022-06-17T12:55:27.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.523+0000 7f42551ee700 1 -- 172.21.15.36:0/3391414546 wait complete. 2022-06-17T12:55:27.518 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.0 2022-06-17T12:55:27.527 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:55:27.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.647+0000 7ff129470700 1 Processor -- start 2022-06-17T12:55:27.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.647+0000 7ff129470700 1 -- start start 2022-06-17T12:55:27.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.647+0000 7ff129470700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff124063730 0x7ff1240b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:27.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.647+0000 7ff129470700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff124064eb0 0x7ff12413ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:27.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.647+0000 7ff129470700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff124063b10 con 0x7ff1241403f0 2022-06-17T12:55:27.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.647+0000 7ff129470700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff124147430 con 0x7ff124064eb0 2022-06-17T12:55:27.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.647+0000 7ff129470700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1240b1ca0 con 0x7ff124063730 2022-06-17T12:55:27.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.647+0000 7ff1237fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff124064eb0 0x7ff12413ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:27.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.647+0000 7ff1237fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff124064eb0 0x7ff12413ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38500/0 (socket says 172.21.15.36:38500) 2022-06-17T12:55:27.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.647+0000 7ff1237fe700 1 -- 172.21.15.36:0/1652924718 learned_addr learned my addr 172.21.15.36:0/1652924718 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:27.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.647+0000 7ff123fff700 1 --2- 172.21.15.36:0/1652924718 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff124063730 0x7ff1240b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:27.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.651+0000 7ff1237fe700 1 -- 172.21.15.36:0/1652924718 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff124063730 msgr2=0x7ff1240b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:27.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.651+0000 7ff1237fe700 1 --2- 172.21.15.36:0/1652924718 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff124063730 0x7ff1240b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:27.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.651+0000 7ff1237fe700 1 -- 172.21.15.36:0/1652924718 >> v1:172.21.15.36:6789/0 conn(0x7ff1241403f0 legacy=0x7ff1240b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:27.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.651+0000 7ff1237fe700 1 -- 172.21.15.36:0/1652924718 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1241464d0 con 0x7ff124064eb0 2022-06-17T12:55:27.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.651+0000 7ff122ffd700 1 -- 172.21.15.36:0/1652924718 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2855838850 0 0) 0x7ff124063b10 con 0x7ff1241403f0 2022-06-17T12:55:27.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.651+0000 7ff1237fe700 1 --2- 172.21.15.36:0/1652924718 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff124064eb0 0x7ff12413ff00 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7ff124063ca0 tx=0x7ff114006eb0).ready entity=mon.0 client_cookie=1d5a24989cc93efd server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:27.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.651+0000 7ff122ffd700 1 -- 172.21.15.36:0/1652924718 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff11400d5c0 con 0x7ff124064eb0 2022-06-17T12:55:27.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.651+0000 7ff122ffd700 1 -- 172.21.15.36:0/1652924718 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff114000e90 con 0x7ff124064eb0 2022-06-17T12:55:27.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.651+0000 7ff122ffd700 1 -- 172.21.15.36:0/1652924718 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff11400d910 con 0x7ff124064eb0 2022-06-17T12:55:27.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.651+0000 7ff129470700 1 -- 172.21.15.36:0/1652924718 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff124064eb0 msgr2=0x7ff12413ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:27.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.651+0000 7ff129470700 1 --2- 172.21.15.36:0/1652924718 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff124064eb0 0x7ff12413ff00 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7ff124063ca0 tx=0x7ff114006eb0).stop 2022-06-17T12:55:27.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.651+0000 7ff129470700 1 -- 172.21.15.36:0/1652924718 shutdown_connections 2022-06-17T12:55:27.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.651+0000 7ff129470700 1 --2- 172.21.15.36:0/1652924718 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff124064eb0 0x7ff12413ff00 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:27.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.651+0000 7ff129470700 1 --2- 172.21.15.36:0/1652924718 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff124063730 0x7ff1240b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:27.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.651+0000 7ff129470700 1 -- 172.21.15.36:0/1652924718 >> 172.21.15.36:0/1652924718 conn(0x7ff1240614b0 msgr2=0x7ff1240618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:27.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.651+0000 7ff129470700 1 -- 172.21.15.36:0/1652924718 shutdown_connections 2022-06-17T12:55:27.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.651+0000 7ff129470700 1 -- 172.21.15.36:0/1652924718 wait complete. 2022-06-17T12:55:27.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.651+0000 7ff129470700 1 Processor -- start 2022-06-17T12:55:27.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.651+0000 7ff129470700 1 -- start start 2022-06-17T12:55:27.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.655+0000 7ff129470700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff124063730 0x7ff1240656b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:27.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.655+0000 7ff129470700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff124064eb0 0x7ff124152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:27.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.655+0000 7ff129470700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1241403f0 0x7ff1240b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:27.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.655+0000 7ff129470700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1240afd00 con 0x7ff124063730 2022-06-17T12:55:27.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.655+0000 7ff129470700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1240affa0 con 0x7ff1241403f0 2022-06-17T12:55:27.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.655+0000 7ff129470700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1240b0240 con 0x7ff124064eb0 2022-06-17T12:55:27.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.655+0000 7ff123fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff124063730 0x7ff1240656b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:27.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.655+0000 7ff123fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff124063730 0x7ff1240656b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38504/0 (socket says 172.21.15.36:38504) 2022-06-17T12:55:27.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.655+0000 7ff123fff700 1 -- 172.21.15.36:0/3955038833 learned_addr learned my addr 172.21.15.36:0/3955038833 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:27.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.655+0000 7ff1237fe700 1 --2- 172.21.15.36:0/3955038833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff124064eb0 0x7ff124152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:27.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.655+0000 7ff128c6f700 1 --2- 172.21.15.36:0/3955038833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1241403f0 0x7ff1240b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:27.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.655+0000 7ff123fff700 1 -- 172.21.15.36:0/3955038833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff124064eb0 msgr2=0x7ff124152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:27.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.655+0000 7ff123fff700 1 --2- 172.21.15.36:0/3955038833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff124064eb0 0x7ff124152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:27.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.655+0000 7ff123fff700 1 -- 172.21.15.36:0/3955038833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1241403f0 msgr2=0x7ff1240b67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:27.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.655+0000 7ff123fff700 1 --2- 172.21.15.36:0/3955038833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1241403f0 0x7ff1240b67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:27.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.655+0000 7ff123fff700 1 -- 172.21.15.36:0/3955038833 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11400b1e0 con 0x7ff124063730 2022-06-17T12:55:27.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.655+0000 7ff123fff700 1 --2- 172.21.15.36:0/3955038833 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff124063730 0x7ff1240656b0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7ff10c00c000 tx=0x7ff10c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:27.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.655+0000 7ff1217fa700 1 -- 172.21.15.36:0/3955038833 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff10c010780 con 0x7ff124063730 2022-06-17T12:55:27.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.655+0000 7ff1217fa700 1 -- 172.21.15.36:0/3955038833 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10c0108e0 con 0x7ff124063730 2022-06-17T12:55:27.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.655+0000 7ff129470700 1 -- 172.21.15.36:0/3955038833 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1240b06e0 con 0x7ff124063730 2022-06-17T12:55:27.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.655+0000 7ff1217fa700 1 -- 172.21.15.36:0/3955038833 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff10c010c40 con 0x7ff124063730 2022-06-17T12:55:27.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.655+0000 7ff129470700 1 -- 172.21.15.36:0/3955038833 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff124152fa0 con 0x7ff124063730 2022-06-17T12:55:27.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.655+0000 7ff129470700 1 -- 172.21.15.36:0/3955038833 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1240b0400 con 0x7ff124063730 2022-06-17T12:55:27.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.655+0000 7ff1217fa700 1 -- 172.21.15.36:0/3955038833 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff10c01c460 con 0x7ff124063730 2022-06-17T12:55:27.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.659+0000 7ff1217fa700 1 --2- 172.21.15.36:0/3955038833 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff1080539b0 0x7ff108055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:27.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.659+0000 7ff1217fa700 1 -- 172.21.15.36:0/3955038833 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff10c06c3a0 con 0x7ff124063730 2022-06-17T12:55:27.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.659+0000 7ff1237fe700 1 --2- 172.21.15.36:0/3955038833 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff1080539b0 0x7ff108055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:27.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.659+0000 7ff1237fe700 1 --2- 172.21.15.36:0/3955038833 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff1080539b0 0x7ff108055e70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7ff11401b000 tx=0x7ff114015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:27.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.663+0000 7ff1217fa700 1 -- 172.21.15.36:0/3955038833 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff10c043140 con 0x7ff124063730 2022-06-17T12:55:27.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.835+0000 7ff129470700 1 -- 172.21.15.36:0/3955038833 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7ff124057410 con 0x7ff124063730 2022-06-17T12:55:27.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.843+0000 7ff1217fa700 1 -- 172.21.15.36:0/3955038833 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v39) v1 ==== 184+0+0 (secure 0 0 0) 0x7ff10c046760 con 0x7ff124063730 2022-06-17T12:55:27.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.843+0000 7ff129470700 1 -- 172.21.15.36:0/3955038833 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff1080539b0 msgr2=0x7ff108055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:27.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.843+0000 7ff129470700 1 --2- 172.21.15.36:0/3955038833 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff1080539b0 0x7ff108055e70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7ff11401b000 tx=0x7ff114015000).stop 2022-06-17T12:55:27.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.843+0000 7ff129470700 1 -- 172.21.15.36:0/3955038833 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff124063730 msgr2=0x7ff1240656b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:27.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.843+0000 7ff129470700 1 --2- 172.21.15.36:0/3955038833 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff124063730 0x7ff1240656b0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7ff10c00c000 tx=0x7ff10c006df0).stop 2022-06-17T12:55:27.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.843+0000 7ff129470700 1 -- 172.21.15.36:0/3955038833 shutdown_connections 2022-06-17T12:55:27.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.843+0000 7ff129470700 1 --2- 172.21.15.36:0/3955038833 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff1080539b0 0x7ff108055e70 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:27.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.843+0000 7ff129470700 1 --2- 172.21.15.36:0/3955038833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1241403f0 0x7ff1240b67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:27.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.843+0000 7ff129470700 1 --2- 172.21.15.36:0/3955038833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff124064eb0 0x7ff124152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:27.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.843+0000 7ff129470700 1 --2- 172.21.15.36:0/3955038833 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff124063730 0x7ff1240656b0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:27.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.843+0000 7ff129470700 1 -- 172.21.15.36:0/3955038833 >> 172.21.15.36:0/3955038833 conn(0x7ff1240614b0 msgr2=0x7ff124061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:27.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.843+0000 7ff129470700 1 -- 172.21.15.36:0/3955038833 shutdown_connections 2022-06-17T12:55:27.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.843+0000 7ff129470700 1 -- 172.21.15.36:0/3955038833 wait complete. 2022-06-17T12:55:27.849 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.1 2022-06-17T12:55:27.849 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:55:27.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.979+0000 7fd60bb37700 1 Processor -- start 2022-06-17T12:55:27.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.979+0000 7fd60bb37700 1 -- start start 2022-06-17T12:55:27.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.979+0000 7fd60bb37700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd6040b0cb0 0x7fd604142f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:27.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.979+0000 7fd60bb37700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6040b44b0 con 0x7fd604143e80 2022-06-17T12:55:27.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.979+0000 7fd60bb37700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd604062650 con 0x7fd604062270 2022-06-17T12:55:27.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.979+0000 7fd60bb37700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6040b6650 con 0x7fd6040b0cb0 2022-06-17T12:55:27.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.979+0000 7fd6090d2700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd6040b0cb0 0x7fd604142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:27.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.979+0000 7fd60a0d4700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fd604143e80 0x7fd604146970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56046/0 (socket says 172.21.15.36:56046) 2022-06-17T12:55:27.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.979+0000 7fd60a0d4700 1 -- 172.21.15.36:0/3243180557 learned_addr learned my addr 172.21.15.36:0/3243180557 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:27.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.979+0000 7fd6090d2700 1 -- 172.21.15.36:0/3243180557 >> v1:172.21.15.70:6789/0 conn(0x7fd604062270 legacy=0x7fd604143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:27.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.979+0000 7fd6090d2700 1 -- 172.21.15.36:0/3243180557 >> v1:172.21.15.36:6789/0 conn(0x7fd604143e80 legacy=0x7fd604146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:27.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.979+0000 7fd6090d2700 1 -- 172.21.15.36:0/3243180557 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6040b4210 con 0x7fd6040b0cb0 2022-06-17T12:55:27.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.979+0000 7fd6090d2700 1 --2- 172.21.15.36:0/3243180557 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd6040b0cb0 0x7fd604142f20 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7fd5f4002f70 tx=0x7fd5f4006df0).ready entity=mon.0 client_cookie=311ec9e0f438479 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:27.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.979+0000 7fd6088d1700 1 -- 172.21.15.36:0/3243180557 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5f400d610 con 0x7fd6040b0cb0 2022-06-17T12:55:27.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.979+0000 7fd6088d1700 1 -- 172.21.15.36:0/3243180557 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f4000e90 con 0x7fd6040b0cb0 2022-06-17T12:55:27.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.979+0000 7fd6088d1700 1 -- 172.21.15.36:0/3243180557 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5f400d960 con 0x7fd6040b0cb0 2022-06-17T12:55:27.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.983+0000 7fd60bb37700 1 -- 172.21.15.36:0/3243180557 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd6040b0cb0 msgr2=0x7fd604142f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:27.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.983+0000 7fd60bb37700 1 --2- 172.21.15.36:0/3243180557 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd6040b0cb0 0x7fd604142f20 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7fd5f4002f70 tx=0x7fd5f4006df0).stop 2022-06-17T12:55:27.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.983+0000 7fd60bb37700 1 -- 172.21.15.36:0/3243180557 shutdown_connections 2022-06-17T12:55:27.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.983+0000 7fd60bb37700 1 --2- 172.21.15.36:0/3243180557 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd6040b0cb0 0x7fd604142f20 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:27.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.983+0000 7fd60bb37700 1 -- 172.21.15.36:0/3243180557 >> 172.21.15.36:0/3243180557 conn(0x7fd6040b9870 msgr2=0x7fd6040b9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:27.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.983+0000 7fd60bb37700 1 -- 172.21.15.36:0/3243180557 shutdown_connections 2022-06-17T12:55:27.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.983+0000 7fd60bb37700 1 -- 172.21.15.36:0/3243180557 wait complete. 2022-06-17T12:55:27.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.983+0000 7fd60bb37700 1 Processor -- start 2022-06-17T12:55:27.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.983+0000 7fd60bb37700 1 -- start start 2022-06-17T12:55:27.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.983+0000 7fd60bb37700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd604062270 0x7fd604152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:27.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.983+0000 7fd60bb37700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd604143e80 0x7fd6040b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:27.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.983+0000 7fd60bb37700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6040afff0 0x7fd604155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:27.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.983+0000 7fd60bb37700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6041534b0 con 0x7fd604062270 2022-06-17T12:55:27.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.983+0000 7fd60bb37700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd604152880 con 0x7fd6040afff0 2022-06-17T12:55:27.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.983+0000 7fd60bb37700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd604152b20 con 0x7fd604143e80 2022-06-17T12:55:27.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.983+0000 7fd6098d3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd604062270 0x7fd604152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:28.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.983+0000 7fd6098d3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd604062270 0x7fd604152fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38518/0 (socket says 172.21.15.36:38518) 2022-06-17T12:55:28.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.983+0000 7fd6098d3700 1 -- 172.21.15.36:0/2951871950 learned_addr learned my addr 172.21.15.36:0/2951871950 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:28.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.983+0000 7fd6090d2700 1 --2- 172.21.15.36:0/2951871950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd604143e80 0x7fd6040b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:28.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.987+0000 7fd60a0d4700 1 --2- 172.21.15.36:0/2951871950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6040afff0 0x7fd604155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:28.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.987+0000 7fd6098d3700 1 -- 172.21.15.36:0/2951871950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd604143e80 msgr2=0x7fd6040b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:28.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.987+0000 7fd6098d3700 1 --2- 172.21.15.36:0/2951871950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd604143e80 0x7fd6040b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:28.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.987+0000 7fd6098d3700 1 -- 172.21.15.36:0/2951871950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6040afff0 msgr2=0x7fd604155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:28.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.987+0000 7fd6098d3700 1 --2- 172.21.15.36:0/2951871950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6040afff0 0x7fd604155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:28.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.987+0000 7fd6098d3700 1 -- 172.21.15.36:0/2951871950 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f400b180 con 0x7fd604062270 2022-06-17T12:55:28.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.987+0000 7fd6098d3700 1 --2- 172.21.15.36:0/2951871950 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd604062270 0x7fd604152fc0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 rx=0x7fd604063520 tx=0x7fd600007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:28.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.987+0000 7fd5f2ffd700 1 -- 172.21.15.36:0/2951871950 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd600012400 con 0x7fd604062270 2022-06-17T12:55:28.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.987+0000 7fd5f2ffd700 1 -- 172.21.15.36:0/2951871950 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd600012560 con 0x7fd604062270 2022-06-17T12:55:28.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.987+0000 7fd5f2ffd700 1 -- 172.21.15.36:0/2951871950 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6000128c0 con 0x7fd604062270 2022-06-17T12:55:28.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.987+0000 7fd60bb37700 1 -- 172.21.15.36:0/2951871950 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6040b27d0 con 0x7fd604062270 2022-06-17T12:55:28.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.987+0000 7fd60bb37700 1 -- 172.21.15.36:0/2951871950 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd604152570 con 0x7fd604062270 2022-06-17T12:55:28.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.987+0000 7fd60bb37700 1 -- 172.21.15.36:0/2951871950 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6040b63b0 con 0x7fd604062270 2022-06-17T12:55:28.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.987+0000 7fd5f2ffd700 1 -- 172.21.15.36:0/2951871950 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd600019030 con 0x7fd604062270 2022-06-17T12:55:28.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.987+0000 7fd5f2ffd700 1 --2- 172.21.15.36:0/2951871950 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd5e40539b0 0x7fd5e4055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:28.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.987+0000 7fd5f2ffd700 1 -- 172.21.15.36:0/2951871950 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd60006b4a0 con 0x7fd604062270 2022-06-17T12:55:28.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.987+0000 7fd6090d2700 1 --2- 172.21.15.36:0/2951871950 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd5e40539b0 0x7fd5e4055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:28.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.991+0000 7fd6090d2700 1 --2- 172.21.15.36:0/2951871950 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd5e40539b0 0x7fd5e4055e70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7fd5f401c000 tx=0x7fd5f4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:28.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:27.995+0000 7fd5f2ffd700 1 -- 172.21.15.36:0/2951871950 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd600042140 con 0x7fd604062270 2022-06-17T12:55:28.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.163+0000 7fd60bb37700 1 -- 172.21.15.36:0/2951871950 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fd604057410 con 0x7fd604062270 2022-06-17T12:55:28.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.171+0000 7fd5f2ffd700 1 -- 172.21.15.36:0/2951871950 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v40) v1 ==== 184+0+0 (secure 0 0 0) 0x7fd600045760 con 0x7fd604062270 2022-06-17T12:55:28.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.171+0000 7fd60bb37700 1 -- 172.21.15.36:0/2951871950 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd5e40539b0 msgr2=0x7fd5e4055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:28.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.171+0000 7fd60bb37700 1 --2- 172.21.15.36:0/2951871950 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd5e40539b0 0x7fd5e4055e70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7fd5f401c000 tx=0x7fd5f4015000).stop 2022-06-17T12:55:28.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.171+0000 7fd60bb37700 1 -- 172.21.15.36:0/2951871950 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd604062270 msgr2=0x7fd604152fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:28.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.171+0000 7fd60bb37700 1 --2- 172.21.15.36:0/2951871950 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd604062270 0x7fd604152fc0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 rx=0x7fd604063520 tx=0x7fd600007c50).stop 2022-06-17T12:55:28.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.171+0000 7fd60bb37700 1 -- 172.21.15.36:0/2951871950 shutdown_connections 2022-06-17T12:55:28.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.171+0000 7fd60bb37700 1 --2- 172.21.15.36:0/2951871950 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd5e40539b0 0x7fd5e4055e70 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:28.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.171+0000 7fd60bb37700 1 --2- 172.21.15.36:0/2951871950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6040afff0 0x7fd604155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:28.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.171+0000 7fd60bb37700 1 --2- 172.21.15.36:0/2951871950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd604143e80 0x7fd6040b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:28.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.171+0000 7fd60bb37700 1 --2- 172.21.15.36:0/2951871950 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd604062270 0x7fd604152fc0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:28.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.171+0000 7fd60bb37700 1 -- 172.21.15.36:0/2951871950 >> 172.21.15.36:0/2951871950 conn(0x7fd6040b9870 msgr2=0x7fd60405f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:28.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.171+0000 7fd60bb37700 1 -- 172.21.15.36:0/2951871950 shutdown_connections 2022-06-17T12:55:28.168 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.171+0000 7fd60bb37700 1 -- 172.21.15.36:0/2951871950 wait complete. 2022-06-17T12:55:28.169 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.2 2022-06-17T12:55:28.174 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:55:28.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.287+0000 7f3c4dd28700 1 Processor -- start 2022-06-17T12:55:28.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.287+0000 7f3c4dd28700 1 -- start start 2022-06-17T12:55:28.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c4dd28700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3c480b1fb0 0x7f3c4814db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:28.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c4dd28700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c48143430 con 0x7f3c48142f90 2022-06-17T12:55:28.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c4dd28700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c480605a0 con 0x7f3c480601c0 2022-06-17T12:55:28.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c4dd28700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c48063520 con 0x7f3c480b1fb0 2022-06-17T12:55:28.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c46ffd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f3c48142f90 0x7f3c481465d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56056/0 (socket says 172.21.15.36:56056) 2022-06-17T12:55:28.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c46ffd700 1 -- 172.21.15.36:0/1765415142 learned_addr learned my addr 172.21.15.36:0/1765415142 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:28.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c47fff700 1 --2- 172.21.15.36:0/1765415142 >> v2:172.21.15.70:3300/0 conn(0x7f3c480b1fb0 0x7f3c4814db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:28.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c467fc700 1 -- 172.21.15.36:0/1765415142 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1745220909 0 0) 0x7f3c48143430 con 0x7f3c48142f90 2022-06-17T12:55:28.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c467fc700 1 -- 172.21.15.36:0/1765415142 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c34003260 con 0x7f3c48142f90 2022-06-17T12:55:28.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c467fc700 1 -- 172.21.15.36:0/1765415142 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130597814 0 0) 0x7f3c480605a0 con 0x7f3c480601c0 2022-06-17T12:55:28.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c467fc700 1 -- 172.21.15.36:0/1765415142 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c48143430 con 0x7f3c480601c0 2022-06-17T12:55:28.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c467fc700 1 -- 172.21.15.36:0/1765415142 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2890128194 0 0) 0x7f3c34003260 con 0x7f3c48142f90 2022-06-17T12:55:28.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c467fc700 1 -- 172.21.15.36:0/1765415142 >> v2:172.21.15.70:3300/0 conn(0x7f3c480b1fb0 msgr2=0x7f3c4814db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:28.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c467fc700 1 --2- 172.21.15.36:0/1765415142 >> v2:172.21.15.70:3300/0 conn(0x7f3c480b1fb0 0x7f3c4814db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:28.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c467fc700 1 -- 172.21.15.36:0/1765415142 >> v1:172.21.15.70:6789/0 conn(0x7f3c480601c0 legacy=0x7f3c48062e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:28.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c467fc700 1 -- 172.21.15.36:0/1765415142 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c480626d0 con 0x7f3c48142f90 2022-06-17T12:55:28.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c467fc700 1 -- 172.21.15.36:0/1765415142 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f3c38001680 con 0x7f3c48142f90 2022-06-17T12:55:28.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c467fc700 1 -- 172.21.15.36:0/1765415142 >> v1:172.21.15.36:6789/0 conn(0x7f3c48142f90 legacy=0x7f3c481465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:28.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c467fc700 1 --2- 172.21.15.36:0/1765415142 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3c340038e0 0x7f3c34003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:28.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c467fc700 1 --2- 172.21.15.36:0/1765415142 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c34004500 0x7f3c34004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:28.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c467fc700 1 --2- 172.21.15.36:0/1765415142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c34004e70 0x7f3c3400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:28.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c467fc700 1 -- 172.21.15.36:0/1765415142 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c48063520 con 0x7f3c340038e0 2022-06-17T12:55:28.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c467fc700 1 -- 172.21.15.36:0/1765415142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3400bcf0 con 0x7f3c34004e70 2022-06-17T12:55:28.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c467fc700 1 -- 172.21.15.36:0/1765415142 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3400beb0 con 0x7f3c34004500 2022-06-17T12:55:28.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c477fe700 1 --2- 172.21.15.36:0/1765415142 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3c340038e0 0x7f3c34003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:28.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c46ffd700 1 --2- 172.21.15.36:0/1765415142 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c34004500 0x7f3c34004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:28.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c47fff700 1 --2- 172.21.15.36:0/1765415142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c34004e70 0x7f3c3400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:28.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c477fe700 1 -- 172.21.15.36:0/1765415142 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c34004500 msgr2=0x7f3c34004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:28.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c477fe700 1 --2- 172.21.15.36:0/1765415142 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c34004500 0x7f3c34004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:28.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c477fe700 1 -- 172.21.15.36:0/1765415142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c34004e70 msgr2=0x7f3c3400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:28.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c477fe700 1 --2- 172.21.15.36:0/1765415142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c34004e70 0x7f3c3400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:28.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c47fff700 1 -- 172.21.15.36:0/1765415142 reap_dead start 2022-06-17T12:55:28.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c477fe700 1 -- 172.21.15.36:0/1765415142 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c34003260 con 0x7f3c340038e0 2022-06-17T12:55:28.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c46ffd700 1 --2- 172.21.15.36:0/1765415142 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c34004500 0x7f3c34004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:28.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c477fe700 1 --2- 172.21.15.36:0/1765415142 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3c340038e0 0x7f3c34003ce0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0x7f3c3c001110 tx=0x7f3c3c007a00).ready entity=mon.0 client_cookie=26e3178df2b3243c server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:28.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c467fc700 1 -- 172.21.15.36:0/1765415142 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c3c007d10 con 0x7f3c340038e0 2022-06-17T12:55:28.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c467fc700 1 -- 172.21.15.36:0/1765415142 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3c007070 con 0x7f3c340038e0 2022-06-17T12:55:28.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.291+0000 7f3c467fc700 1 -- 172.21.15.36:0/1765415142 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c3c0073d0 con 0x7f3c340038e0 2022-06-17T12:55:28.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.295+0000 7f3c4dd28700 1 -- 172.21.15.36:0/1765415142 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3c340038e0 msgr2=0x7f3c34003ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:28.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.295+0000 7f3c4dd28700 1 --2- 172.21.15.36:0/1765415142 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3c340038e0 0x7f3c34003ce0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0x7f3c3c001110 tx=0x7f3c3c007a00).stop 2022-06-17T12:55:28.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.295+0000 7f3c4dd28700 1 -- 172.21.15.36:0/1765415142 shutdown_connections 2022-06-17T12:55:28.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.295+0000 7f3c4dd28700 1 --2- 172.21.15.36:0/1765415142 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3c340038e0 0x7f3c34003ce0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:28.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.295+0000 7f3c4dd28700 1 -- 172.21.15.36:0/1765415142 >> 172.21.15.36:0/1765415142 conn(0x7f3c48060990 msgr2=0x7f3c4805f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:28.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.295+0000 7f3c4dd28700 1 -- 172.21.15.36:0/1765415142 shutdown_connections 2022-06-17T12:55:28.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.295+0000 7f3c4dd28700 1 -- 172.21.15.36:0/1765415142 wait complete. 2022-06-17T12:55:28.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.295+0000 7f3c4dd28700 1 Processor -- start 2022-06-17T12:55:28.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.295+0000 7f3c4dd28700 1 -- start start 2022-06-17T12:55:28.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.295+0000 7f3c4dd28700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c340038e0 0x7f3c480652b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:28.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.295+0000 7f3c4dd28700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3c480ae540 0x7f3c480658f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:28.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.295+0000 7f3c4dd28700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c48152890 0x7f3c48152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:28.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.295+0000 7f3c4dd28700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c48152430 con 0x7f3c480ae540 2022-06-17T12:55:28.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.295+0000 7f3c4dd28700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c48115180 con 0x7f3c340038e0 2022-06-17T12:55:28.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.295+0000 7f3c4dd28700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c48114ee0 con 0x7f3c48152890 2022-06-17T12:55:28.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.295+0000 7f3c46ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3c480ae540 0x7f3c480658f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:28.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.295+0000 7f3c46ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3c480ae540 0x7f3c480658f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38540/0 (socket says 172.21.15.36:38540) 2022-06-17T12:55:28.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.295+0000 7f3c46ffd700 1 -- 172.21.15.36:0/4058020568 learned_addr learned my addr 172.21.15.36:0/4058020568 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:28.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.295+0000 7f3c47fff700 1 --2- 172.21.15.36:0/4058020568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c48152890 0x7f3c48152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:28.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.295+0000 7f3c477fe700 1 --2- 172.21.15.36:0/4058020568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c340038e0 0x7f3c480652b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:28.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.299+0000 7f3c47fff700 1 -- 172.21.15.36:0/4058020568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c340038e0 msgr2=0x7f3c480652b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:28.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.299+0000 7f3c47fff700 1 --2- 172.21.15.36:0/4058020568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c340038e0 0x7f3c480652b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:28.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.299+0000 7f3c47fff700 1 -- 172.21.15.36:0/4058020568 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3c480ae540 msgr2=0x7f3c480658f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:28.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.299+0000 7f3c47fff700 1 --2- 172.21.15.36:0/4058020568 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3c480ae540 0x7f3c480658f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:28.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.299+0000 7f3c47fff700 1 -- 172.21.15.36:0/4058020568 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c48153200 con 0x7f3c48152890 2022-06-17T12:55:28.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.299+0000 7f3c47fff700 1 --2- 172.21.15.36:0/4058020568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c48152890 0x7f3c48152d10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f3c48115310 tx=0x7f3c400091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:28.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.299+0000 7f3c44ff9700 1 -- 172.21.15.36:0/4058020568 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c40019070 con 0x7f3c48152890 2022-06-17T12:55:28.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.299+0000 7f3c44ff9700 1 -- 172.21.15.36:0/4058020568 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c40013750 con 0x7f3c48152890 2022-06-17T12:55:28.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.299+0000 7f3c4dd28700 1 -- 172.21.15.36:0/4058020568 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c480aea50 con 0x7f3c48152890 2022-06-17T12:55:28.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.299+0000 7f3c44ff9700 1 -- 172.21.15.36:0/4058020568 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c40013ab0 con 0x7f3c48152890 2022-06-17T12:55:28.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.299+0000 7f3c4dd28700 1 -- 172.21.15.36:0/4058020568 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c48067310 con 0x7f3c48152890 2022-06-17T12:55:28.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.299+0000 7f3c4dd28700 1 -- 172.21.15.36:0/4058020568 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c480aec40 con 0x7f3c48152890 2022-06-17T12:55:28.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.299+0000 7f3c44ff9700 1 -- 172.21.15.36:0/4058020568 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f3c40013c10 con 0x7f3c48152890 2022-06-17T12:55:28.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.299+0000 7f3c44ff9700 1 --2- 172.21.15.36:0/4058020568 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3c28053ad0 0x7f3c28055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:28.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.299+0000 7f3c44ff9700 1 -- 172.21.15.36:0/4058020568 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3c4006cb30 con 0x7f3c48152890 2022-06-17T12:55:28.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.299+0000 7f3c477fe700 1 --2- 172.21.15.36:0/4058020568 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3c28053ad0 0x7f3c28055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:28.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.299+0000 7f3c477fe700 1 --2- 172.21.15.36:0/4058020568 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3c28053ad0 0x7f3c28055f90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f3c3c01b000 tx=0x7f3c3c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:28.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.307+0000 7f3c44ff9700 1 -- 172.21.15.36:0/4058020568 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3c480aec40 con 0x7f3c48152890 2022-06-17T12:55:28.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.471+0000 7f3c4dd28700 1 -- 172.21.15.36:0/4058020568 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c480aec40 con 0x7f3c48152890 2022-06-17T12:55:28.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.471+0000 7f3c44ff9700 1 -- 172.21.15.36:0/4058020568 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f3c480aec40 con 0x7f3c48152890 2022-06-17T12:55:28.464 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:28.464 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":60,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1161585109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1161585109},{"type":"v1","addr":"172.21.15.70:6833","nonce":1161585109}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":57}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:55:24.825976+0000","modified":"2022-06-17T12:55:26.574711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7531":{"gid":7531,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/1342585509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":1342585509},{"type":"v1","addr":"172.21.15.36:6835","nonce":1342585509}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2022-06-17T12:55:28.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.475+0000 7f3c4dd28700 1 -- 172.21.15.36:0/4058020568 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3c28053ad0 msgr2=0x7f3c28055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:28.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.475+0000 7f3c4dd28700 1 --2- 172.21.15.36:0/4058020568 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3c28053ad0 0x7f3c28055f90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f3c3c01b000 tx=0x7f3c3c015000).stop 2022-06-17T12:55:28.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.475+0000 7f3c4dd28700 1 -- 172.21.15.36:0/4058020568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c48152890 msgr2=0x7f3c48152d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:28.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.475+0000 7f3c4dd28700 1 --2- 172.21.15.36:0/4058020568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c48152890 0x7f3c48152d10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f3c48115310 tx=0x7f3c400091d0).stop 2022-06-17T12:55:28.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.475+0000 7f3c4dd28700 1 -- 172.21.15.36:0/4058020568 shutdown_connections 2022-06-17T12:55:28.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.475+0000 7f3c4dd28700 1 --2- 172.21.15.36:0/4058020568 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3c28053ad0 0x7f3c28055f90 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:28.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.475+0000 7f3c4dd28700 1 --2- 172.21.15.36:0/4058020568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c48152890 0x7f3c48152d10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:28.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.475+0000 7f3c4dd28700 1 --2- 172.21.15.36:0/4058020568 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3c480ae540 0x7f3c480658f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:28.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.475+0000 7f3c4dd28700 1 --2- 172.21.15.36:0/4058020568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c340038e0 0x7f3c480652b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:28.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.475+0000 7f3c4dd28700 1 -- 172.21.15.36:0/4058020568 >> 172.21.15.36:0/4058020568 conn(0x7f3c48060990 msgr2=0x7f3c480b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:28.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.475+0000 7f3c4dd28700 1 -- 172.21.15.36:0/4058020568 shutdown_connections 2022-06-17T12:55:28.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.475+0000 7f3c4dd28700 1 -- 172.21.15.36:0/4058020568 wait complete. 2022-06-17T12:55:28.472 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 60 2022-06-17T12:55:28.678 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:55:24.825976+0000', 'modified': '2022-06-17T12:55:26.574711+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7531}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7531': {'gid': 7531, 'name': 'a', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.36:6835/1342585509', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 1342585509}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 1342585509}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:55:28.679 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:55:28.679 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:55:28.809 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":7531,"want_state":"up:active","state":"up:active","rank_uptime":3.977084611,"mdsmap_epoch":60,"osdmap_epoch":113,"osdmap_epoch_barrier":113,"uptime":8.3463019470000006} 2022-06-17T12:55:28.810 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 7531, "mdsmap_epoch": 60, "osdmap_epoch": 113, "osdmap_epoch_barrier": 113, "rank_uptime": 3.977084611, "state": "up:active", "uptime": 8.346301947, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:55:28.811 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:55:28.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.931+0000 7f46a4137700 1 Processor -- start 2022-06-17T12:55:28.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.931+0000 7f46a4137700 1 -- start start 2022-06-17T12:55:28.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a4137700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f469c142aa0 0x7f469c142f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:28.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a4137700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469c060720 con 0x7f469c0ad330 2022-06-17T12:55:28.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a4137700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469c062d30 con 0x7f469c060340 2022-06-17T12:55:28.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a4137700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469c05fe80 con 0x7f469c142aa0 2022-06-17T12:55:28.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a16d2700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f469c0ad330 0x7f469c05f770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56076/0 (socket says 172.21.15.36:56076) 2022-06-17T12:55:28.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a16d2700 1 -- 172.21.15.36:0/3071671941 learned_addr learned my addr 172.21.15.36:0/3071671941 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:28.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a26d4700 1 --2- 172.21.15.36:0/3071671941 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f469c142aa0 0x7f469c142f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:28.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a0ed1700 1 -- 172.21.15.36:0/3071671941 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 710722418 0 0) 0x7f469c060720 con 0x7f469c0ad330 2022-06-17T12:55:28.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a0ed1700 1 -- 172.21.15.36:0/3071671941 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4684003260 con 0x7f469c0ad330 2022-06-17T12:55:28.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a0ed1700 1 -- 172.21.15.36:0/3071671941 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 337477756 0 0) 0x7f469c062d30 con 0x7f469c060340 2022-06-17T12:55:28.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a0ed1700 1 -- 172.21.15.36:0/3071671941 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f469c060720 con 0x7f469c060340 2022-06-17T12:55:28.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a0ed1700 1 -- 172.21.15.36:0/3071671941 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1327571988 0 0) 0x7f4684003260 con 0x7f469c0ad330 2022-06-17T12:55:28.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a0ed1700 1 -- 172.21.15.36:0/3071671941 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f469c142aa0 msgr2=0x7f469c142f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:28.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a0ed1700 1 --2- 172.21.15.36:0/3071671941 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f469c142aa0 0x7f469c142f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:28.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a0ed1700 1 -- 172.21.15.36:0/3071671941 >> v1:172.21.15.70:6789/0 conn(0x7f469c060340 legacy=0x7f469c13ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:28.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a0ed1700 1 -- 172.21.15.36:0/3071671941 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469c062fd0 con 0x7f469c0ad330 2022-06-17T12:55:28.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a0ed1700 1 -- 172.21.15.36:0/3071671941 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f468c001680 con 0x7f469c0ad330 2022-06-17T12:55:28.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a26d4700 1 --2- 172.21.15.36:0/3071671941 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f469c142aa0 0x7f469c142f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:28.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a0ed1700 1 -- 172.21.15.36:0/3071671941 >> v1:172.21.15.36:6789/0 conn(0x7f469c0ad330 legacy=0x7f469c05f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:28.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a0ed1700 1 --2- 172.21.15.36:0/3071671941 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f469c142aa0 0x7f4684003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:28.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a0ed1700 1 --2- 172.21.15.36:0/3071671941 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4684003e50 0x7f4684004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:28.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a0ed1700 1 --2- 172.21.15.36:0/3071671941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46840049f0 0x7f4684004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:28.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a0ed1700 1 -- 172.21.15.36:0/3071671941 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469c05fe80 con 0x7f4684003e50 2022-06-17T12:55:28.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a0ed1700 1 -- 172.21.15.36:0/3071671941 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468400b800 con 0x7f46840049f0 2022-06-17T12:55:28.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a0ed1700 1 -- 172.21.15.36:0/3071671941 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468400b9c0 con 0x7f469c142aa0 2022-06-17T12:55:28.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a16d2700 1 --2- 172.21.15.36:0/3071671941 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4684003e50 0x7f4684004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:28.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a1ed3700 1 --2- 172.21.15.36:0/3071671941 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f469c142aa0 0x7f4684003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:28.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a26d4700 1 --2- 172.21.15.36:0/3071671941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46840049f0 0x7f4684004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:28.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a16d2700 1 -- 172.21.15.36:0/3071671941 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f469c142aa0 msgr2=0x7f4684003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:28.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a16d2700 1 --2- 172.21.15.36:0/3071671941 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f469c142aa0 0x7f4684003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:28.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a16d2700 1 -- 172.21.15.36:0/3071671941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46840049f0 msgr2=0x7f4684004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:28.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a16d2700 1 --2- 172.21.15.36:0/3071671941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46840049f0 0x7f4684004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:28.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a16d2700 1 -- 172.21.15.36:0/3071671941 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4694009470 con 0x7f4684003e50 2022-06-17T12:55:28.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a1ed3700 1 --2- 172.21.15.36:0/3071671941 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f469c142aa0 0x7f4684003910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:28.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a16d2700 1 --2- 172.21.15.36:0/3071671941 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4684003e50 0x7f4684004500 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7f468c001450 tx=0x7f468c001d20).ready entity=mon.0 client_cookie=f85999bfa3978262 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:28.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a0ed1700 1 -- 172.21.15.36:0/3071671941 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f468c001d90 con 0x7f4684003e50 2022-06-17T12:55:28.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a0ed1700 1 -- 172.21.15.36:0/3071671941 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468c007170 con 0x7f4684003e50 2022-06-17T12:55:28.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a0ed1700 1 -- 172.21.15.36:0/3071671941 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f468c0074d0 con 0x7f4684003e50 2022-06-17T12:55:28.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a4137700 1 -- 172.21.15.36:0/3071671941 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4684003e50 msgr2=0x7f4684004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:28.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a4137700 1 --2- 172.21.15.36:0/3071671941 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4684003e50 0x7f4684004500 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7f468c001450 tx=0x7f468c001d20).stop 2022-06-17T12:55:28.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a26d4700 1 -- 172.21.15.36:0/3071671941 reap_dead start 2022-06-17T12:55:28.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a4137700 1 -- 172.21.15.36:0/3071671941 shutdown_connections 2022-06-17T12:55:28.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a4137700 1 -- 172.21.15.36:0/3071671941 >> 172.21.15.36:0/3071671941 conn(0x7f469c0b53e0 msgr2=0x7f469c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:28.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a4137700 1 -- 172.21.15.36:0/3071671941 shutdown_connections 2022-06-17T12:55:28.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.935+0000 7f46a4137700 1 -- 172.21.15.36:0/3071671941 wait complete. 2022-06-17T12:55:28.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.939+0000 7f46a4137700 1 Processor -- start 2022-06-17T12:55:28.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.939+0000 7f46a4137700 1 -- start start 2022-06-17T12:55:28.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.939+0000 7f46a4137700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f469c112d20 0x7f469c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:28.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.939+0000 7f46a4137700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f469c063b40 0x7f469c1564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:28.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.939+0000 7f46a4137700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f469c063630 0x7f469c13bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:28.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.939+0000 7f46a4137700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469c1137a0 con 0x7f469c063630 2022-06-17T12:55:28.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.939+0000 7f46a4137700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469c065640 con 0x7f469c112d20 2022-06-17T12:55:28.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.939+0000 7f46a4137700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469c113500 con 0x7f469c063b40 2022-06-17T12:55:28.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.939+0000 7f46a26d4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f469c063630 0x7f469c13bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:28.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.939+0000 7f46a1ed3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f469c112d20 0x7f469c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:28.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.939+0000 7f46a26d4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f469c063630 0x7f469c13bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38560/0 (socket says 172.21.15.36:38560) 2022-06-17T12:55:28.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.939+0000 7f46a26d4700 1 -- 172.21.15.36:0/2984856520 learned_addr learned my addr 172.21.15.36:0/2984856520 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:28.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.939+0000 7f46a16d2700 1 --2- 172.21.15.36:0/2984856520 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f469c063b40 0x7f469c1564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:28.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.939+0000 7f46a26d4700 1 -- 172.21.15.36:0/2984856520 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f469c063b40 msgr2=0x7f469c1564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:28.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.939+0000 7f46a26d4700 1 --2- 172.21.15.36:0/2984856520 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f469c063b40 0x7f469c1564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:28.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.939+0000 7f46a26d4700 1 -- 172.21.15.36:0/2984856520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f469c112d20 msgr2=0x7f469c065150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:28.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.939+0000 7f46a26d4700 1 --2- 172.21.15.36:0/2984856520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f469c112d20 0x7f469c065150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:28.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.939+0000 7f46a26d4700 1 -- 172.21.15.36:0/2984856520 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469c064990 con 0x7f469c063630 2022-06-17T12:55:28.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.939+0000 7f46a26d4700 1 --2- 172.21.15.36:0/2984856520 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f469c063630 0x7f469c13bb60 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7f469c064010 tx=0x7f4694008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:28.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.939+0000 7f468affd700 1 -- 172.21.15.36:0/2984856520 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4694013550 con 0x7f469c063630 2022-06-17T12:55:28.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.939+0000 7f468affd700 1 -- 172.21.15.36:0/2984856520 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46940136b0 con 0x7f469c063630 2022-06-17T12:55:28.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.939+0000 7f46a4137700 1 -- 172.21.15.36:0/2984856520 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469c064700 con 0x7f469c063630 2022-06-17T12:55:28.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.939+0000 7f468affd700 1 -- 172.21.15.36:0/2984856520 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4694013a10 con 0x7f469c063630 2022-06-17T12:55:28.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.943+0000 7f46a4137700 1 -- 172.21.15.36:0/2984856520 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469c145a70 con 0x7f469c063630 2022-06-17T12:55:28.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.943+0000 7f46a4137700 1 -- 172.21.15.36:0/2984856520 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f469c13f320 con 0x7f469c063630 2022-06-17T12:55:28.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.943+0000 7f468affd700 1 -- 172.21.15.36:0/2984856520 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f469401d030 con 0x7f469c063630 2022-06-17T12:55:28.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.943+0000 7f468affd700 1 --2- 172.21.15.36:0/2984856520 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f467c053a60 0x7f467c055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:28.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.943+0000 7f468affd700 1 -- 172.21.15.36:0/2984856520 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f469406bea0 con 0x7f469c063630 2022-06-17T12:55:28.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.943+0000 7f46a1ed3700 1 --2- 172.21.15.36:0/2984856520 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f467c053a60 0x7f467c055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:28.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.943+0000 7f46a1ed3700 1 --2- 172.21.15.36:0/2984856520 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f467c053a60 0x7f467c055f20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f469800d000 tx=0x7f4698009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:28.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:28.951+0000 7f468affd700 1 -- 172.21.15.36:0/2984856520 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4694042ca0 con 0x7f469c063630 2022-06-17T12:55:29.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.075+0000 7f46a4137700 1 -- 172.21.15.36:0/2984856520 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f469c057410 con 0x7f469c063630 2022-06-17T12:55:29.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.079+0000 7f468affd700 1 -- 172.21.15.36:0/2984856520 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f46940462c0 con 0x7f469c063630 2022-06-17T12:55:29.066 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:29.067 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":113,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:55:25.828833+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:55:23.324881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-06-17T12:55:24.417060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:55:29.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.079+0000 7f46a4137700 1 -- 172.21.15.36:0/2984856520 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f467c053a60 msgr2=0x7f467c055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:29.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.079+0000 7f46a4137700 1 --2- 172.21.15.36:0/2984856520 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f467c053a60 0x7f467c055f20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f469800d000 tx=0x7f4698009040).stop 2022-06-17T12:55:29.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.079+0000 7f46a4137700 1 -- 172.21.15.36:0/2984856520 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f469c063630 msgr2=0x7f469c13bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:29.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.079+0000 7f46a4137700 1 --2- 172.21.15.36:0/2984856520 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f469c063630 0x7f469c13bb60 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7f469c064010 tx=0x7f4694008040).stop 2022-06-17T12:55:29.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.079+0000 7f46a4137700 1 -- 172.21.15.36:0/2984856520 shutdown_connections 2022-06-17T12:55:29.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.079+0000 7f46a4137700 1 --2- 172.21.15.36:0/2984856520 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f467c053a60 0x7f467c055f20 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.079+0000 7f46a4137700 1 --2- 172.21.15.36:0/2984856520 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f469c063630 0x7f469c13bb60 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.079+0000 7f46a4137700 1 --2- 172.21.15.36:0/2984856520 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f469c063b40 0x7f469c1564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.079+0000 7f46a4137700 1 --2- 172.21.15.36:0/2984856520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f469c112d20 0x7f469c065150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.079+0000 7f46a4137700 1 -- 172.21.15.36:0/2984856520 >> 172.21.15.36:0/2984856520 conn(0x7f469c0b53e0 msgr2=0x7f469c13fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:29.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.079+0000 7f46a4137700 1 -- 172.21.15.36:0/2984856520 shutdown_connections 2022-06-17T12:55:29.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.079+0000 7f46a4137700 1 -- 172.21.15.36:0/2984856520 wait complete. 2022-06-17T12:55:29.080 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:55:23.324881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:55:24.417060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:55:29.082 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:55:29.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.219+0000 7f7378ec1700 1 Processor -- start 2022-06-17T12:55:29.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.219+0000 7f7378ec1700 1 -- start start 2022-06-17T12:55:29.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.219+0000 7f7378ec1700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f73740b64f0 0x7f7374143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:29.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.219+0000 7f7378ec1700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f73740b4120 0x7f7374146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:29.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.219+0000 7f7378ec1700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73741476e0 con 0x7f7374147270 2022-06-17T12:55:29.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.219+0000 7f7378ec1700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73740b68d0 con 0x7f73740b4120 2022-06-17T12:55:29.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.219+0000 7f7378ec1700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73740b6e20 con 0x7f73740b64f0 2022-06-17T12:55:29.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.219+0000 7f7371d9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f73740b4120 0x7f7374146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:29.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.219+0000 7f7371d9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f73740b4120 0x7f7374146d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38568/0 (socket says 172.21.15.36:38568) 2022-06-17T12:55:29.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.219+0000 7f7371d9b700 1 -- 172.21.15.36:0/1275232377 learned_addr learned my addr 172.21.15.36:0/1275232377 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:29.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.219+0000 7f737259c700 1 --2- 172.21.15.36:0/1275232377 >> v2:172.21.15.70:3300/0 conn(0x7f73740b64f0 0x7f7374143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:29.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.223+0000 7f737159a700 1 -- 172.21.15.36:0/1275232377 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857793751 0 0) 0x7f73741476e0 con 0x7f7374147270 2022-06-17T12:55:29.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.223+0000 7f737159a700 1 -- 172.21.15.36:0/1275232377 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7354003260 con 0x7f7374147270 2022-06-17T12:55:29.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.223+0000 7f7371d9b700 1 -- 172.21.15.36:0/1275232377 >> v2:172.21.15.70:3300/0 conn(0x7f73740b64f0 msgr2=0x7f7374143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:29.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.223+0000 7f7371d9b700 1 --2- 172.21.15.36:0/1275232377 >> v2:172.21.15.70:3300/0 conn(0x7f73740b64f0 0x7f7374143150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.223+0000 7f7371d9b700 1 -- 172.21.15.36:0/1275232377 >> v1:172.21.15.70:6789/0 conn(0x7f7374147270 legacy=0x7f7374140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:29.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.223+0000 7f7371d9b700 1 -- 172.21.15.36:0/1275232377 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73741468e0 con 0x7f73740b4120 2022-06-17T12:55:29.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.223+0000 7f7371d9b700 1 --2- 172.21.15.36:0/1275232377 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f73740b4120 0x7f7374146d80 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7f7364002f70 tx=0x7f7364006eb0).ready entity=mon.0 client_cookie=3b9d72084cde1068 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:29.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.223+0000 7f737159a700 1 -- 172.21.15.36:0/1275232377 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f736400d610 con 0x7f73740b4120 2022-06-17T12:55:29.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.223+0000 7f737159a700 1 -- 172.21.15.36:0/1275232377 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7364000e90 con 0x7f73740b4120 2022-06-17T12:55:29.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.223+0000 7f737159a700 1 -- 172.21.15.36:0/1275232377 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f736400d9a0 con 0x7f73740b4120 2022-06-17T12:55:29.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.223+0000 7f7378ec1700 1 -- 172.21.15.36:0/1275232377 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f73740b4120 msgr2=0x7f7374146d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:29.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.223+0000 7f7378ec1700 1 --2- 172.21.15.36:0/1275232377 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f73740b4120 0x7f7374146d80 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7f7364002f70 tx=0x7f7364006eb0).stop 2022-06-17T12:55:29.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.223+0000 7f7378ec1700 1 -- 172.21.15.36:0/1275232377 shutdown_connections 2022-06-17T12:55:29.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.223+0000 7f7378ec1700 1 --2- 172.21.15.36:0/1275232377 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f73740b4120 0x7f7374146d80 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.223+0000 7f7378ec1700 1 --2- 172.21.15.36:0/1275232377 >> v2:172.21.15.70:3300/0 conn(0x7f73740b64f0 0x7f7374143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.223+0000 7f7378ec1700 1 -- 172.21.15.36:0/1275232377 >> 172.21.15.36:0/1275232377 conn(0x7f73740620f0 msgr2=0x7f7374062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:29.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.223+0000 7f7378ec1700 1 -- 172.21.15.36:0/1275232377 shutdown_connections 2022-06-17T12:55:29.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.223+0000 7f7378ec1700 1 -- 172.21.15.36:0/1275232377 wait complete. 2022-06-17T12:55:29.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.223+0000 7f7378ec1700 1 Processor -- start 2022-06-17T12:55:29.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.223+0000 7f7378ec1700 1 -- start start 2022-06-17T12:55:29.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f7378ec1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f73740b4120 0x7f73740b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:29.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f7378ec1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73740b64f0 0x7f73741428d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:29.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f7378ec1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7374147270 0x7f73740b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:29.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f7378ec1700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7374140f90 con 0x7f73740b4120 2022-06-17T12:55:29.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f7378ec1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73741411f0 con 0x7f73740b64f0 2022-06-17T12:55:29.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f7378ec1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7374141490 con 0x7f7374147270 2022-06-17T12:55:29.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f737259c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f73740b4120 0x7f73740b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:29.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f737259c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f73740b4120 0x7f73740b05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38572/0 (socket says 172.21.15.36:38572) 2022-06-17T12:55:29.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f737259c700 1 -- 172.21.15.36:0/1781997212 learned_addr learned my addr 172.21.15.36:0/1781997212 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:29.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f7371d9b700 1 --2- 172.21.15.36:0/1781997212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73740b64f0 0x7f73741428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:29.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f737259c700 1 -- 172.21.15.36:0/1781997212 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7374147270 msgr2=0x7f73740b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:29.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f7372d9d700 1 --2- 172.21.15.36:0/1781997212 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7374147270 0x7f73740b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:29.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f737259c700 1 --2- 172.21.15.36:0/1781997212 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7374147270 0x7f73740b1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f737259c700 1 -- 172.21.15.36:0/1781997212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73740b64f0 msgr2=0x7f73741428d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:29.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f737259c700 1 --2- 172.21.15.36:0/1781997212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73740b64f0 0x7f73741428d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f737259c700 1 -- 172.21.15.36:0/1781997212 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736400b1e0 con 0x7f73740b4120 2022-06-17T12:55:29.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f7371d9b700 1 --2- 172.21.15.36:0/1781997212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73740b64f0 0x7f73741428d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:29.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f7372d9d700 1 --2- 172.21.15.36:0/1781997212 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7374147270 0x7f73740b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:55:29.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f737259c700 1 --2- 172.21.15.36:0/1781997212 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f73740b4120 0x7f73740b05b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 rx=0x7f735c00c000 tx=0x7f735c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:29.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f736b7fe700 1 -- 172.21.15.36:0/1781997212 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f735c010780 con 0x7f73740b4120 2022-06-17T12:55:29.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f736b7fe700 1 -- 172.21.15.36:0/1781997212 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735c0108e0 con 0x7f73740b4120 2022-06-17T12:55:29.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f736b7fe700 1 -- 172.21.15.36:0/1781997212 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f735c010c40 con 0x7f73740b4120 2022-06-17T12:55:29.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f7378ec1700 1 -- 172.21.15.36:0/1781997212 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73740b1f40 con 0x7f73740b4120 2022-06-17T12:55:29.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f7378ec1700 1 -- 172.21.15.36:0/1781997212 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73741423d0 con 0x7f73740b4120 2022-06-17T12:55:29.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f7378ec1700 1 -- 172.21.15.36:0/1781997212 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7374143640 con 0x7f73740b4120 2022-06-17T12:55:29.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f736b7fe700 1 -- 172.21.15.36:0/1781997212 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f735c01c460 con 0x7f73740b4120 2022-06-17T12:55:29.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.227+0000 7f736b7fe700 1 --2- 172.21.15.36:0/1781997212 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7358053a40 0x7f7358055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:29.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.231+0000 7f736b7fe700 1 -- 172.21.15.36:0/1781997212 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f735c06c480 con 0x7f73740b4120 2022-06-17T12:55:29.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.231+0000 7f7371d9b700 1 --2- 172.21.15.36:0/1781997212 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7358053a40 0x7f7358055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:29.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.231+0000 7f7371d9b700 1 --2- 172.21.15.36:0/1781997212 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7358053a40 0x7f7358055f00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f736401a000 tx=0x7f7364015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:29.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.235+0000 7f736b7fe700 1 -- 172.21.15.36:0/1781997212 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f735c043220 con 0x7f73740b4120 2022-06-17T12:55:29.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.359+0000 7f7378ec1700 1 -- 172.21.15.36:0/1781997212 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7374057410 con 0x7f73740b4120 2022-06-17T12:55:29.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.363+0000 7f736b7fe700 1 -- 172.21.15.36:0/1781997212 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v113) v1 ==== 110+0+10 (secure 0 0 0) 0x7f735c046840 con 0x7f73740b4120 2022-06-17T12:55:29.351 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:55:29.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.363+0000 7f7378ec1700 1 -- 172.21.15.36:0/1781997212 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7358053a40 msgr2=0x7f7358055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:29.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.363+0000 7f7378ec1700 1 --2- 172.21.15.36:0/1781997212 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7358053a40 0x7f7358055f00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f736401a000 tx=0x7f7364015000).stop 2022-06-17T12:55:29.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.363+0000 7f7378ec1700 1 -- 172.21.15.36:0/1781997212 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f73740b4120 msgr2=0x7f73740b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:29.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.363+0000 7f7378ec1700 1 --2- 172.21.15.36:0/1781997212 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f73740b4120 0x7f73740b05b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 rx=0x7f735c00c000 tx=0x7f735c006df0).stop 2022-06-17T12:55:29.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.363+0000 7f7378ec1700 1 -- 172.21.15.36:0/1781997212 shutdown_connections 2022-06-17T12:55:29.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.363+0000 7f7378ec1700 1 --2- 172.21.15.36:0/1781997212 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7358053a40 0x7f7358055f00 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.363+0000 7f7378ec1700 1 --2- 172.21.15.36:0/1781997212 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7374147270 0x7f73740b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.363+0000 7f7378ec1700 1 --2- 172.21.15.36:0/1781997212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73740b64f0 0x7f73741428d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.363+0000 7f7378ec1700 1 --2- 172.21.15.36:0/1781997212 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f73740b4120 0x7f73740b05b0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.363+0000 7f7378ec1700 1 -- 172.21.15.36:0/1781997212 >> 172.21.15.36:0/1781997212 conn(0x7f73740620f0 msgr2=0x7f73740b9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:29.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.363+0000 7f7378ec1700 1 -- 172.21.15.36:0/1781997212 shutdown_connections 2022-06-17T12:55:29.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.363+0000 7f7378ec1700 1 -- 172.21.15.36:0/1781997212 wait complete. 2022-06-17T12:55:29.370 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:55:29.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.491+0000 7fcea22c9700 1 Processor -- start 2022-06-17T12:55:29.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.491+0000 7fcea22c9700 1 -- start start 2022-06-17T12:55:29.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fcea22c9700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fce9c0b18e0 0x7fce9c143860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:29.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fcea22c9700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce9c0b39a0 0x7fce9c146d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:29.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fcea22c9700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce9c0b2b70 con 0x7fce9c0b6860 2022-06-17T12:55:29.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fcea22c9700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce9c0b6c40 con 0x7fce9c0b18e0 2022-06-17T12:55:29.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fcea22c9700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c142d30 con 0x7fce9c0b39a0 2022-06-17T12:55:29.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fcea12c7700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fce9c0b6860 0x7fce9c143150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56110/0 (socket says 172.21.15.36:56110) 2022-06-17T12:55:29.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fcea12c7700 1 -- 172.21.15.36:0/3090942979 learned_addr learned my addr 172.21.15.36:0/3090942979 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:29.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fcea0ac6700 1 --2- 172.21.15.36:0/3090942979 >> v2:172.21.15.70:3300/0 conn(0x7fce9c0b18e0 0x7fce9c143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:29.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fcea1ac8700 1 --2- 172.21.15.36:0/3090942979 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce9c0b39a0 0x7fce9c146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:29.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fce8bfff700 1 -- 172.21.15.36:0/3090942979 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3014819910 0 0) 0x7fce9c0b2b70 con 0x7fce9c0b6860 2022-06-17T12:55:29.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fce8bfff700 1 -- 172.21.15.36:0/3090942979 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce84003260 con 0x7fce9c0b6860 2022-06-17T12:55:29.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fcea0ac6700 1 -- 172.21.15.36:0/3090942979 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce9c0b39a0 msgr2=0x7fce9c146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:29.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fcea0ac6700 1 --2- 172.21.15.36:0/3090942979 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce9c0b39a0 0x7fce9c146d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fcea0ac6700 1 -- 172.21.15.36:0/3090942979 >> v1:172.21.15.36:6789/0 conn(0x7fce9c0b6860 legacy=0x7fce9c143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:29.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fcea0ac6700 1 -- 172.21.15.36:0/3090942979 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9c0b0d30 con 0x7fce9c0b18e0 2022-06-17T12:55:29.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fcea1ac8700 1 --2- 172.21.15.36:0/3090942979 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce9c0b39a0 0x7fce9c146d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:29.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fcea0ac6700 1 --2- 172.21.15.36:0/3090942979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce9c0b18e0 0x7fce9c143860 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7fce98002f70 tx=0x7fce98006d90).ready entity=mon.1 client_cookie=2581cdec43e77df server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:29.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fce8bfff700 1 -- 172.21.15.36:0/3090942979 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce9800e590 con 0x7fce9c0b18e0 2022-06-17T12:55:29.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fce8bfff700 1 -- 172.21.15.36:0/3090942979 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce98000e90 con 0x7fce9c0b18e0 2022-06-17T12:55:29.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fce8bfff700 1 -- 172.21.15.36:0/3090942979 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce9800e8e0 con 0x7fce9c0b18e0 2022-06-17T12:55:29.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fcea22c9700 1 -- 172.21.15.36:0/3090942979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce9c0b18e0 msgr2=0x7fce9c143860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:29.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fcea22c9700 1 --2- 172.21.15.36:0/3090942979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce9c0b18e0 0x7fce9c143860 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7fce98002f70 tx=0x7fce98006d90).stop 2022-06-17T12:55:29.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fcea22c9700 1 -- 172.21.15.36:0/3090942979 shutdown_connections 2022-06-17T12:55:29.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fcea22c9700 1 --2- 172.21.15.36:0/3090942979 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce9c0b39a0 0x7fce9c146d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fcea22c9700 1 --2- 172.21.15.36:0/3090942979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce9c0b18e0 0x7fce9c143860 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fcea22c9700 1 -- 172.21.15.36:0/3090942979 >> 172.21.15.36:0/3090942979 conn(0x7fce9c0614b0 msgr2=0x7fce9c061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:29.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fcea22c9700 1 -- 172.21.15.36:0/3090942979 shutdown_connections 2022-06-17T12:55:29.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.495+0000 7fcea22c9700 1 -- 172.21.15.36:0/3090942979 wait complete. 2022-06-17T12:55:29.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.499+0000 7fcea22c9700 1 Processor -- start 2022-06-17T12:55:29.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.499+0000 7fcea22c9700 1 -- start start 2022-06-17T12:55:29.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.499+0000 7fcea22c9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce9c0b18e0 0x7fce9c146950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:29.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.499+0000 7fcea22c9700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce9c0b39a0 0x7fce9c0666c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:29.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.499+0000 7fcea22c9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce9c0b6860 0x7fce9c066bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:29.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.499+0000 7fcea22c9700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c0639e0 con 0x7fce9c0b6860 2022-06-17T12:55:29.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.499+0000 7fcea22c9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c063c80 con 0x7fce9c0b18e0 2022-06-17T12:55:29.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.499+0000 7fcea22c9700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c063f20 con 0x7fce9c0b39a0 2022-06-17T12:55:29.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.499+0000 7fcea1ac8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce9c0b6860 0x7fce9c066bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:29.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.499+0000 7fcea0ac6700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce9c0b39a0 0x7fce9c0666c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:29.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.499+0000 7fcea12c7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce9c0b18e0 0x7fce9c146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:29.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.499+0000 7fcea1ac8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce9c0b6860 0x7fce9c066bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38588/0 (socket says 172.21.15.36:38588) 2022-06-17T12:55:29.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.499+0000 7fcea1ac8700 1 -- 172.21.15.36:0/2624466142 learned_addr learned my addr 172.21.15.36:0/2624466142 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:29.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.499+0000 7fcea1ac8700 1 -- 172.21.15.36:0/2624466142 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce9c0b39a0 msgr2=0x7fce9c0666c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:29.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.499+0000 7fcea1ac8700 1 --2- 172.21.15.36:0/2624466142 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce9c0b39a0 0x7fce9c0666c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.499+0000 7fcea1ac8700 1 -- 172.21.15.36:0/2624466142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce9c0b18e0 msgr2=0x7fce9c146950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:29.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.499+0000 7fcea1ac8700 1 --2- 172.21.15.36:0/2624466142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce9c0b18e0 0x7fce9c146950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.499+0000 7fcea1ac8700 1 -- 172.21.15.36:0/2624466142 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce94009470 con 0x7fce9c0b6860 2022-06-17T12:55:29.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.499+0000 7fcea12c7700 1 --2- 172.21.15.36:0/2624466142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce9c0b18e0 0x7fce9c146950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:29.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.503+0000 7fcea1ac8700 1 --2- 172.21.15.36:0/2624466142 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce9c0b6860 0x7fce9c066bb0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7fce9400c000 tx=0x7fce940091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:29.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.503+0000 7fce8a7fc700 1 -- 172.21.15.36:0/2624466142 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce9400c850 con 0x7fce9c0b6860 2022-06-17T12:55:29.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.503+0000 7fce8a7fc700 1 -- 172.21.15.36:0/2624466142 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9400c9b0 con 0x7fce9c0b6860 2022-06-17T12:55:29.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.503+0000 7fcea22c9700 1 -- 172.21.15.36:0/2624466142 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce9800b120 con 0x7fce9c0b6860 2022-06-17T12:55:29.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.503+0000 7fce8a7fc700 1 -- 172.21.15.36:0/2624466142 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce9400cd10 con 0x7fce9c0b6860 2022-06-17T12:55:29.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.503+0000 7fcea22c9700 1 -- 172.21.15.36:0/2624466142 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce9c0670a0 con 0x7fce9c0b6860 2022-06-17T12:55:29.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.503+0000 7fcea22c9700 1 -- 172.21.15.36:0/2624466142 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce9c0640e0 con 0x7fce9c0b6860 2022-06-17T12:55:29.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.503+0000 7fce8a7fc700 1 -- 172.21.15.36:0/2624466142 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fce94022070 con 0x7fce9c0b6860 2022-06-17T12:55:29.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.503+0000 7fce8a7fc700 1 --2- 172.21.15.36:0/2624466142 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce7c053a40 0x7fce7c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:29.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.503+0000 7fce8a7fc700 1 -- 172.21.15.36:0/2624466142 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fce9406c930 con 0x7fce9c0b6860 2022-06-17T12:55:29.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.503+0000 7fcea12c7700 1 --2- 172.21.15.36:0/2624466142 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce7c053a40 0x7fce7c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:29.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.507+0000 7fcea12c7700 1 --2- 172.21.15.36:0/2624466142 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce7c053a40 0x7fce7c055f00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fce8c00d000 tx=0x7fce8c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:29.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.511+0000 7fce8a7fc700 1 -- 172.21.15.36:0/2624466142 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fce940435a0 con 0x7fce9c0b6860 2022-06-17T12:55:29.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.635+0000 7fcea22c9700 1 -- 172.21.15.36:0/2624466142 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fce9c057410 con 0x7fce9c0b6860 2022-06-17T12:55:29.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.635+0000 7fce8a7fc700 1 -- 172.21.15.36:0/2624466142 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v113) v1 ==== 92+0+10 (secure 0 0 0) 0x7fce94016020 con 0x7fce9c0b6860 2022-06-17T12:55:29.625 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:55:29.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.639+0000 7fcea22c9700 1 -- 172.21.15.36:0/2624466142 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce7c053a40 msgr2=0x7fce7c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:29.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.639+0000 7fcea22c9700 1 --2- 172.21.15.36:0/2624466142 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce7c053a40 0x7fce7c055f00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fce8c00d000 tx=0x7fce8c009040).stop 2022-06-17T12:55:29.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.639+0000 7fcea22c9700 1 -- 172.21.15.36:0/2624466142 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce9c0b6860 msgr2=0x7fce9c066bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:29.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.639+0000 7fcea22c9700 1 --2- 172.21.15.36:0/2624466142 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce9c0b6860 0x7fce9c066bb0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7fce9400c000 tx=0x7fce940091d0).stop 2022-06-17T12:55:29.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.639+0000 7fcea22c9700 1 -- 172.21.15.36:0/2624466142 shutdown_connections 2022-06-17T12:55:29.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.639+0000 7fcea22c9700 1 --2- 172.21.15.36:0/2624466142 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce7c053a40 0x7fce7c055f00 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.639+0000 7fcea22c9700 1 --2- 172.21.15.36:0/2624466142 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce9c0b6860 0x7fce9c066bb0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.639+0000 7fcea22c9700 1 --2- 172.21.15.36:0/2624466142 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce9c0b39a0 0x7fce9c0666c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.639+0000 7fcea22c9700 1 --2- 172.21.15.36:0/2624466142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce9c0b18e0 0x7fce9c146950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.639+0000 7fcea22c9700 1 -- 172.21.15.36:0/2624466142 >> 172.21.15.36:0/2624466142 conn(0x7fce9c0614b0 msgr2=0x7fce9c0ae440 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:29.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.639+0000 7fcea22c9700 1 -- 172.21.15.36:0/2624466142 shutdown_connections 2022-06-17T12:55:29.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.639+0000 7fcea22c9700 1 -- 172.21.15.36:0/2624466142 wait complete. 2022-06-17T12:55:29.639 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:55:29.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.759+0000 7f3de359e700 1 Processor -- start 2022-06-17T12:55:29.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.759+0000 7f3de359e700 1 -- start start 2022-06-17T12:55:29.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.759+0000 7f3de359e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f3de40b3f80 0x7f3de40b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:29.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.759+0000 7f3de359e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3de40b4b90 0x7f3de413ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:29.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.759+0000 7f3de359e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de41432c0 con 0x7f3de4060020 2022-06-17T12:55:29.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.759+0000 7f3de359e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de4062db0 con 0x7f3de40b3f80 2022-06-17T12:55:29.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.759+0000 7f3de359e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de4143df0 con 0x7f3de40b4b90 2022-06-17T12:55:29.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.759+0000 7f3de1d9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f3de40b3f80 0x7f3de40b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:29.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.759+0000 7f3de1d9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f3de40b3f80 0x7f3de40b46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38596/0 (socket says 172.21.15.36:38596) 2022-06-17T12:55:29.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.759+0000 7f3de1d9b700 1 -- 172.21.15.36:0/1422084636 learned_addr learned my addr 172.21.15.36:0/1422084636 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:29.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.759+0000 7f3de2d9d700 1 --2- 172.21.15.36:0/1422084636 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3de40b4b90 0x7f3de413ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:29.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.763+0000 7f3de159a700 1 -- 172.21.15.36:0/1422084636 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1505125685 0 0) 0x7f3de41432c0 con 0x7f3de4060020 2022-06-17T12:55:29.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.763+0000 7f3de1d9b700 1 -- 172.21.15.36:0/1422084636 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3de40b4b90 msgr2=0x7f3de413ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:29.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.763+0000 7f3de1d9b700 1 --2- 172.21.15.36:0/1422084636 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3de40b4b90 0x7f3de413ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.763+0000 7f3de1d9b700 1 -- 172.21.15.36:0/1422084636 >> v1:172.21.15.70:6789/0 conn(0x7f3de4060020 legacy=0x7f3de40626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:29.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.763+0000 7f3de1d9b700 1 -- 172.21.15.36:0/1422084636 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de4061f60 con 0x7f3de40b3f80 2022-06-17T12:55:29.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.763+0000 7f3de1d9b700 1 --2- 172.21.15.36:0/1422084636 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3de40b3f80 0x7f3de40b46a0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 rx=0x7f3dcc002f70 tx=0x7f3dcc006eb0).ready entity=mon.0 client_cookie=6d7c538644af6f48 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:29.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.763+0000 7f3de159a700 1 -- 172.21.15.36:0/1422084636 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dcc00d570 con 0x7f3de40b3f80 2022-06-17T12:55:29.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.763+0000 7f3de159a700 1 -- 172.21.15.36:0/1422084636 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dcc000e90 con 0x7f3de40b3f80 2022-06-17T12:55:29.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.763+0000 7f3de159a700 1 -- 172.21.15.36:0/1422084636 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dcc00d8c0 con 0x7f3de40b3f80 2022-06-17T12:55:29.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.763+0000 7f3de359e700 1 -- 172.21.15.36:0/1422084636 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3de40b3f80 msgr2=0x7f3de40b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:29.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.763+0000 7f3de359e700 1 --2- 172.21.15.36:0/1422084636 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3de40b3f80 0x7f3de40b46a0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 rx=0x7f3dcc002f70 tx=0x7f3dcc006eb0).stop 2022-06-17T12:55:29.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.763+0000 7f3de359e700 1 -- 172.21.15.36:0/1422084636 shutdown_connections 2022-06-17T12:55:29.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.763+0000 7f3de359e700 1 --2- 172.21.15.36:0/1422084636 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3de40b4b90 0x7f3de413ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.763+0000 7f3de359e700 1 --2- 172.21.15.36:0/1422084636 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3de40b3f80 0x7f3de40b46a0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.763+0000 7f3de359e700 1 -- 172.21.15.36:0/1422084636 >> 172.21.15.36:0/1422084636 conn(0x7f3de405c550 msgr2=0x7f3de405f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:29.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.763+0000 7f3de359e700 1 -- 172.21.15.36:0/1422084636 shutdown_connections 2022-06-17T12:55:29.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.763+0000 7f3de359e700 1 -- 172.21.15.36:0/1422084636 wait complete. 2022-06-17T12:55:29.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.763+0000 7f3de359e700 1 Processor -- start 2022-06-17T12:55:29.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.763+0000 7f3de359e700 1 -- start start 2022-06-17T12:55:29.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3de359e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3de4060020 0x7f3de40add00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:29.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3de359e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3de40b3f80 0x7f3de4140190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:29.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3de359e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3de40b4b90 0x7f3de413edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:29.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3de359e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de4146330 con 0x7f3de40b4b90 2022-06-17T12:55:29.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3de359e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de4145db0 con 0x7f3de40b3f80 2022-06-17T12:55:29.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3de359e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de4146f60 con 0x7f3de4060020 2022-06-17T12:55:29.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3de2d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3de40b4b90 0x7f3de413edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:29.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3de2d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3de40b4b90 0x7f3de413edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38602/0 (socket says 172.21.15.36:38602) 2022-06-17T12:55:29.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3de2d9d700 1 -- 172.21.15.36:0/3069331645 learned_addr learned my addr 172.21.15.36:0/3069331645 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:29.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3de259c700 1 --2- 172.21.15.36:0/3069331645 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3de4060020 0x7f3de40add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:29.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3de1d9b700 1 --2- 172.21.15.36:0/3069331645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3de40b3f80 0x7f3de4140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:29.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3de2d9d700 1 -- 172.21.15.36:0/3069331645 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3de4060020 msgr2=0x7f3de40add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:29.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3de2d9d700 1 --2- 172.21.15.36:0/3069331645 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3de4060020 0x7f3de40add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3de2d9d700 1 -- 172.21.15.36:0/3069331645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3de40b3f80 msgr2=0x7f3de4140190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:29.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3de2d9d700 1 --2- 172.21.15.36:0/3069331645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3de40b3f80 0x7f3de4140190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3de2d9d700 1 -- 172.21.15.36:0/3069331645 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc00b1e0 con 0x7f3de40b4b90 2022-06-17T12:55:29.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3de2d9d700 1 --2- 172.21.15.36:0/3069331645 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3de40b4b90 0x7f3de413edf0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7f3ddc00c000 tx=0x7f3ddc0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:29.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3ddb7fe700 1 -- 172.21.15.36:0/3069331645 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ddc012600 con 0x7f3de40b4b90 2022-06-17T12:55:29.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3ddb7fe700 1 -- 172.21.15.36:0/3069331645 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ddc012760 con 0x7f3de40b4b90 2022-06-17T12:55:29.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3de359e700 1 -- 172.21.15.36:0/3069331645 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de4152180 con 0x7f3de40b4b90 2022-06-17T12:55:29.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3ddb7fe700 1 -- 172.21.15.36:0/3069331645 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ddc012ac0 con 0x7f3de40b4b90 2022-06-17T12:55:29.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3de359e700 1 -- 172.21.15.36:0/3069331645 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de413f480 con 0x7f3de40b4b90 2022-06-17T12:55:29.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3ddb7fe700 1 -- 172.21.15.36:0/3069331645 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f3ddc012c20 con 0x7f3de40b4b90 2022-06-17T12:55:29.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3de359e700 1 -- 172.21.15.36:0/3069331645 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de41464a0 con 0x7f3de40b4b90 2022-06-17T12:55:29.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3ddb7fe700 1 --2- 172.21.15.36:0/3069331645 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3dc80539b0 0x7f3dc8055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:29.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3ddb7fe700 1 -- 172.21.15.36:0/3069331645 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3ddc06c250 con 0x7f3de40b4b90 2022-06-17T12:55:29.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.767+0000 7f3de259c700 1 --2- 172.21.15.36:0/3069331645 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3dc80539b0 0x7f3dc8055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:29.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.771+0000 7f3de259c700 1 --2- 172.21.15.36:0/3069331645 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3dc80539b0 0x7f3dc8055e70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f3dd400d000 tx=0x7f3dd4007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:29.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.775+0000 7f3ddb7fe700 1 -- 172.21.15.36:0/3069331645 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3de41464a0 con 0x7f3de40b4b90 2022-06-17T12:55:29.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.899+0000 7f3de359e700 1 -- 172.21.15.36:0/3069331645 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3de41466b0 con 0x7f3de40b4b90 2022-06-17T12:55:29.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.903+0000 7f3ddb7fe700 1 -- 172.21.15.36:0/3069331645 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3ddc016020 con 0x7f3de40b4b90 2022-06-17T12:55:29.890 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:55:29.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.903+0000 7f3de359e700 1 -- 172.21.15.36:0/3069331645 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3dc80539b0 msgr2=0x7f3dc8055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:29.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.903+0000 7f3de359e700 1 --2- 172.21.15.36:0/3069331645 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3dc80539b0 0x7f3dc8055e70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f3dd400d000 tx=0x7f3dd4007a00).stop 2022-06-17T12:55:29.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.903+0000 7f3de359e700 1 -- 172.21.15.36:0/3069331645 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3de40b4b90 msgr2=0x7f3de413edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:29.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.903+0000 7f3de359e700 1 --2- 172.21.15.36:0/3069331645 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3de40b4b90 0x7f3de413edf0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7f3ddc00c000 tx=0x7f3ddc0091d0).stop 2022-06-17T12:55:29.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.903+0000 7f3de359e700 1 -- 172.21.15.36:0/3069331645 shutdown_connections 2022-06-17T12:55:29.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.903+0000 7f3de359e700 1 --2- 172.21.15.36:0/3069331645 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3dc80539b0 0x7f3dc8055e70 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.903+0000 7f3de359e700 1 --2- 172.21.15.36:0/3069331645 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3de40b4b90 0x7f3de413edf0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.903+0000 7f3de359e700 1 --2- 172.21.15.36:0/3069331645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3de40b3f80 0x7f3de4140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.903+0000 7f3de359e700 1 --2- 172.21.15.36:0/3069331645 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3de4060020 0x7f3de40add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:29.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.903+0000 7f3de359e700 1 -- 172.21.15.36:0/3069331645 >> 172.21.15.36:0/3069331645 conn(0x7f3de405c550 msgr2=0x7f3de405bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:29.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.903+0000 7f3de359e700 1 -- 172.21.15.36:0/3069331645 shutdown_connections 2022-06-17T12:55:29.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:29.903+0000 7f3de359e700 1 -- 172.21.15.36:0/3069331645 wait complete. 2022-06-17T12:55:29.904 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:55:30.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.027+0000 7fbb9f247700 1 Processor -- start 2022-06-17T12:55:30.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.027+0000 7fbb9f247700 1 -- start start 2022-06-17T12:55:30.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb9f247700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbb980b0610 0x7fbb98146730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:30.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb9f247700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbb98146c20 0x7fbb981424c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:30.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb9f247700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb98060d40 con 0x7fbb980b2df0 2022-06-17T12:55:30.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb9f247700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb98144170 con 0x7fbb98146c20 2022-06-17T12:55:30.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb9f247700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb980b31d0 con 0x7fbb980b0610 2022-06-17T12:55:30.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb9d7e4700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbb98146c20 0x7fbb981424c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:30.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb9d7e4700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbb98146c20 0x7fbb981424c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38608/0 (socket says 172.21.15.36:38608) 2022-06-17T12:55:30.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb9d7e4700 1 -- 172.21.15.36:0/139096542 learned_addr learned my addr 172.21.15.36:0/139096542 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:30.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb9cfe3700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fbb980b2df0 0x7fbb98143a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56136/0 (socket says 172.21.15.36:56136) 2022-06-17T12:55:30.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb97fff700 1 --2- 172.21.15.36:0/139096542 >> v2:172.21.15.70:3300/0 conn(0x7fbb980b0610 0x7fbb98146730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:30.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb9d7e4700 1 -- 172.21.15.36:0/139096542 >> v2:172.21.15.70:3300/0 conn(0x7fbb980b0610 msgr2=0x7fbb98146730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:30.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb9d7e4700 1 --2- 172.21.15.36:0/139096542 >> v2:172.21.15.70:3300/0 conn(0x7fbb980b0610 0x7fbb98146730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:30.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb9d7e4700 1 -- 172.21.15.36:0/139096542 >> v1:172.21.15.36:6789/0 conn(0x7fbb980b2df0 legacy=0x7fbb98143a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:30.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb9d7e4700 1 -- 172.21.15.36:0/139096542 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb98143320 con 0x7fbb98146c20 2022-06-17T12:55:30.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb977fe700 1 -- 172.21.15.36:0/139096542 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3588824276 0 0) 0x7fbb98060d40 con 0x7fbb980b2df0 2022-06-17T12:55:30.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb9d7e4700 1 --2- 172.21.15.36:0/139096542 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb98146c20 0x7fbb981424c0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7fbb90002f70 tx=0x7fbb90006eb0).ready entity=mon.0 client_cookie=9a956876a3a78182 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:30.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb977fe700 1 -- 172.21.15.36:0/139096542 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb9000d610 con 0x7fbb98146c20 2022-06-17T12:55:30.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb977fe700 1 -- 172.21.15.36:0/139096542 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb90000e90 con 0x7fbb98146c20 2022-06-17T12:55:30.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb977fe700 1 -- 172.21.15.36:0/139096542 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb9000d960 con 0x7fbb98146c20 2022-06-17T12:55:30.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb9f247700 1 -- 172.21.15.36:0/139096542 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb98146c20 msgr2=0x7fbb981424c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:30.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb9f247700 1 --2- 172.21.15.36:0/139096542 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb98146c20 0x7fbb981424c0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7fbb90002f70 tx=0x7fbb90006eb0).stop 2022-06-17T12:55:30.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb9f247700 1 -- 172.21.15.36:0/139096542 shutdown_connections 2022-06-17T12:55:30.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb9f247700 1 --2- 172.21.15.36:0/139096542 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb98146c20 0x7fbb981424c0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:30.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb9f247700 1 --2- 172.21.15.36:0/139096542 >> v2:172.21.15.70:3300/0 conn(0x7fbb980b0610 0x7fbb98146730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:30.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb9f247700 1 -- 172.21.15.36:0/139096542 >> 172.21.15.36:0/139096542 conn(0x7fbb98063020 msgr2=0x7fbb98063430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:30.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb9f247700 1 -- 172.21.15.36:0/139096542 shutdown_connections 2022-06-17T12:55:30.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.031+0000 7fbb9f247700 1 -- 172.21.15.36:0/139096542 wait complete. 2022-06-17T12:55:30.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb9f247700 1 Processor -- start 2022-06-17T12:55:30.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb9f247700 1 -- start start 2022-06-17T12:55:30.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb9f247700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb980b0610 0x7fbb981563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:30.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb9f247700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb980b2df0 0x7fbb980b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:30.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb9f247700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb98146c20 0x7fbb9813ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:30.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb9f247700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb980ac330 con 0x7fbb98146c20 2022-06-17T12:55:30.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb9f247700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb980aa310 con 0x7fbb980b2df0 2022-06-17T12:55:30.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb9f247700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb980a9dd0 con 0x7fbb980b0610 2022-06-17T12:55:30.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb9d7e4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb98146c20 0x7fbb9813ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:30.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb9d7e4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb98146c20 0x7fbb9813ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38616/0 (socket says 172.21.15.36:38616) 2022-06-17T12:55:30.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb9d7e4700 1 -- 172.21.15.36:0/187375236 learned_addr learned my addr 172.21.15.36:0/187375236 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:30.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb97fff700 1 --2- 172.21.15.36:0/187375236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb980b2df0 0x7fbb980b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:30.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb9cfe3700 1 --2- 172.21.15.36:0/187375236 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb980b0610 0x7fbb981563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:30.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb9d7e4700 1 -- 172.21.15.36:0/187375236 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb980b0610 msgr2=0x7fbb981563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:30.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb9d7e4700 1 --2- 172.21.15.36:0/187375236 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb980b0610 0x7fbb981563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:30.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb9d7e4700 1 -- 172.21.15.36:0/187375236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb980b2df0 msgr2=0x7fbb980b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:30.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb9d7e4700 1 --2- 172.21.15.36:0/187375236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb980b2df0 0x7fbb980b1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:30.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb9d7e4700 1 -- 172.21.15.36:0/187375236 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb9000b1e0 con 0x7fbb98146c20 2022-06-17T12:55:30.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb9d7e4700 1 --2- 172.21.15.36:0/187375236 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb98146c20 0x7fbb9813ba00 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7fbb98145f10 tx=0x7fbb90015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:30.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb95ffb700 1 -- 172.21.15.36:0/187375236 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb90006b40 con 0x7fbb98146c20 2022-06-17T12:55:30.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb95ffb700 1 -- 172.21.15.36:0/187375236 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9000d990 con 0x7fbb98146c20 2022-06-17T12:55:30.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb9f247700 1 -- 172.21.15.36:0/187375236 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb980aa070 con 0x7fbb98146c20 2022-06-17T12:55:30.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb95ffb700 1 -- 172.21.15.36:0/187375236 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb90011d30 con 0x7fbb98146c20 2022-06-17T12:55:30.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb9f247700 1 -- 172.21.15.36:0/187375236 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb980ab9a0 con 0x7fbb98146c20 2022-06-17T12:55:30.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.035+0000 7fbb9f247700 1 -- 172.21.15.36:0/187375236 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb980aa4c0 con 0x7fbb98146c20 2022-06-17T12:55:30.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.039+0000 7fbb95ffb700 1 -- 172.21.15.36:0/187375236 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fbb90025030 con 0x7fbb98146c20 2022-06-17T12:55:30.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.039+0000 7fbb95ffb700 1 --2- 172.21.15.36:0/187375236 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb7c053a60 0x7fbb7c055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:30.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.039+0000 7fbb95ffb700 1 -- 172.21.15.36:0/187375236 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbb9006d4e0 con 0x7fbb98146c20 2022-06-17T12:55:30.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.039+0000 7fbb9cfe3700 1 --2- 172.21.15.36:0/187375236 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb7c053a60 0x7fbb7c055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:30.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.039+0000 7fbb9cfe3700 1 --2- 172.21.15.36:0/187375236 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb7c053a60 0x7fbb7c055f20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7fbb8800d000 tx=0x7fbb88007d80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:30.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.043+0000 7fbb95ffb700 1 -- 172.21.15.36:0/187375236 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbb900442e0 con 0x7fbb98146c20 2022-06-17T12:55:30.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.167+0000 7fbb9f247700 1 -- 172.21.15.36:0/187375236 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbb981435c0 con 0x7fbb98146c20 2022-06-17T12:55:30.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.167+0000 7fbb95ffb700 1 -- 172.21.15.36:0/187375236 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+10 (secure 0 0 0) 0x7fbb90023070 con 0x7fbb98146c20 2022-06-17T12:55:30.158 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:55:30.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.171+0000 7fbb9f247700 1 -- 172.21.15.36:0/187375236 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb7c053a60 msgr2=0x7fbb7c055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:30.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.171+0000 7fbb9f247700 1 --2- 172.21.15.36:0/187375236 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb7c053a60 0x7fbb7c055f20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7fbb8800d000 tx=0x7fbb88007d80).stop 2022-06-17T12:55:30.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.171+0000 7fbb9f247700 1 -- 172.21.15.36:0/187375236 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb98146c20 msgr2=0x7fbb9813ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:30.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.171+0000 7fbb9f247700 1 --2- 172.21.15.36:0/187375236 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb98146c20 0x7fbb9813ba00 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7fbb98145f10 tx=0x7fbb90015000).stop 2022-06-17T12:55:30.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.171+0000 7fbb9f247700 1 -- 172.21.15.36:0/187375236 shutdown_connections 2022-06-17T12:55:30.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.171+0000 7fbb9f247700 1 --2- 172.21.15.36:0/187375236 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb7c053a60 0x7fbb7c055f20 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:30.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.171+0000 7fbb9f247700 1 --2- 172.21.15.36:0/187375236 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb98146c20 0x7fbb9813ba00 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:30.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.171+0000 7fbb9f247700 1 --2- 172.21.15.36:0/187375236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb980b2df0 0x7fbb980b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:30.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.171+0000 7fbb9f247700 1 --2- 172.21.15.36:0/187375236 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb980b0610 0x7fbb981563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:30.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.171+0000 7fbb9f247700 1 -- 172.21.15.36:0/187375236 >> 172.21.15.36:0/187375236 conn(0x7fbb98063020 msgr2=0x7fbb981492b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:30.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.171+0000 7fbb9f247700 1 -- 172.21.15.36:0/187375236 shutdown_connections 2022-06-17T12:55:30.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.171+0000 7fbb9f247700 1 -- 172.21.15.36:0/187375236 wait complete. 2022-06-17T12:55:30.173 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:55:30.174 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:55:30.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.291+0000 7fb413fff700 1 Processor -- start 2022-06-17T12:55:30.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.291+0000 7fb413fff700 1 -- start start 2022-06-17T12:55:30.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.291+0000 7fb413fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb4140b1280 0x7fb41413fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:30.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.291+0000 7fb413fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb4140abe30 0x7fb414062ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:30.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.291+0000 7fb413fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb414140410 0x7fb4141436d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:30.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.291+0000 7fb413fff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4140aef10 con 0x7fb4140abe30 2022-06-17T12:55:30.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.291+0000 7fb413fff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb414140080 con 0x7fb4140b1280 2022-06-17T12:55:30.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.291+0000 7fb413fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4140b1660 con 0x7fb414140410 2022-06-17T12:55:30.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.291+0000 7fb40bfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb4140abe30 0x7fb414062ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:30.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.291+0000 7fb40bfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb4140abe30 0x7fb414062ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38622/0 (socket says 172.21.15.36:38622) 2022-06-17T12:55:30.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.291+0000 7fb40bfff700 1 -- 172.21.15.36:0/2019744558 learned_addr learned my addr 172.21.15.36:0/2019744558 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:30.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.291+0000 7fb412ffd700 1 --2- 172.21.15.36:0/2019744558 >> v2:172.21.15.70:3300/0 conn(0x7fb4140b1280 0x7fb41413fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:30.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.291+0000 7fb4137fe700 1 --2- 172.21.15.36:0/2019744558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb414140410 0x7fb4141436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:30.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.295+0000 7fb40bfff700 1 -- 172.21.15.36:0/2019744558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb414140410 msgr2=0x7fb4141436d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:30.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.295+0000 7fb40bfff700 1 --2- 172.21.15.36:0/2019744558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb414140410 0x7fb4141436d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:30.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.295+0000 7fb40bfff700 1 -- 172.21.15.36:0/2019744558 >> v2:172.21.15.70:3300/0 conn(0x7fb4140b1280 msgr2=0x7fb41413fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:30.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.295+0000 7fb40bfff700 1 --2- 172.21.15.36:0/2019744558 >> v2:172.21.15.70:3300/0 conn(0x7fb4140b1280 0x7fb41413fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:30.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.295+0000 7fb40bfff700 1 -- 172.21.15.36:0/2019744558 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4140ac3b0 con 0x7fb4140abe30 2022-06-17T12:55:30.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.295+0000 7fb412ffd700 1 --2- 172.21.15.36:0/2019744558 >> v2:172.21.15.70:3300/0 conn(0x7fb4140b1280 0x7fb41413fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:30.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.295+0000 7fb40bfff700 1 --2- 172.21.15.36:0/2019744558 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb4140abe30 0x7fb414062ef0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7fb3fc002f70 tx=0x7fb3fc006f40).ready entity=mon.0 client_cookie=8a82b7ceedda6875 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:30.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.295+0000 7fb4127fc700 1 -- 172.21.15.36:0/2019744558 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3fc00d9a0 con 0x7fb4140abe30 2022-06-17T12:55:30.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.295+0000 7fb4127fc700 1 -- 172.21.15.36:0/2019744558 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3fc000e90 con 0x7fb4140abe30 2022-06-17T12:55:30.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.295+0000 7fb4127fc700 1 -- 172.21.15.36:0/2019744558 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3fc00dd30 con 0x7fb4140abe30 2022-06-17T12:55:30.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.295+0000 7fb413fff700 1 -- 172.21.15.36:0/2019744558 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb4140abe30 msgr2=0x7fb414062ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:30.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.295+0000 7fb413fff700 1 --2- 172.21.15.36:0/2019744558 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb4140abe30 0x7fb414062ef0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7fb3fc002f70 tx=0x7fb3fc006f40).stop 2022-06-17T12:55:30.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.295+0000 7fb413fff700 1 -- 172.21.15.36:0/2019744558 shutdown_connections 2022-06-17T12:55:30.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.295+0000 7fb413fff700 1 --2- 172.21.15.36:0/2019744558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb414140410 0x7fb4141436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:30.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.295+0000 7fb413fff700 1 --2- 172.21.15.36:0/2019744558 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb4140abe30 0x7fb414062ef0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:30.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.295+0000 7fb413fff700 1 --2- 172.21.15.36:0/2019744558 >> v2:172.21.15.70:3300/0 conn(0x7fb4140b1280 0x7fb41413fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:30.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.295+0000 7fb413fff700 1 -- 172.21.15.36:0/2019744558 >> 172.21.15.36:0/2019744558 conn(0x7fb41405f770 msgr2=0x7fb41405fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:30.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.295+0000 7fb413fff700 1 -- 172.21.15.36:0/2019744558 shutdown_connections 2022-06-17T12:55:30.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.295+0000 7fb413fff700 1 -- 172.21.15.36:0/2019744558 wait complete. 2022-06-17T12:55:30.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.295+0000 7fb413fff700 1 Processor -- start 2022-06-17T12:55:30.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.295+0000 7fb413fff700 1 -- start start 2022-06-17T12:55:30.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.299+0000 7fb413fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4140abe30 0x7fb4140b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:30.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.299+0000 7fb413fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb4140b1280 0x7fb414139000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:30.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.299+0000 7fb413fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb414140410 0x7fb4140b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:30.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.299+0000 7fb413fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41413b530 con 0x7fb4140b1280 2022-06-17T12:55:30.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.299+0000 7fb413fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4140b3870 con 0x7fb4140abe30 2022-06-17T12:55:30.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.299+0000 7fb413fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41413b790 con 0x7fb414140410 2022-06-17T12:55:30.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.299+0000 7fb40bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb4140b1280 0x7fb414139000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:30.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.299+0000 7fb4137fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb414140410 0x7fb4140b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:30.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.299+0000 7fb412ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4140abe30 0x7fb4140b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:30.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.299+0000 7fb4137fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb414140410 0x7fb4140b42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:53418/0 (socket says 172.21.15.36:53418) 2022-06-17T12:55:30.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.299+0000 7fb4137fe700 1 -- 172.21.15.36:0/402860282 learned_addr learned my addr 172.21.15.36:0/402860282 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:30.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.299+0000 7fb40bfff700 1 -- 172.21.15.36:0/402860282 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb414140410 msgr2=0x7fb4140b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:30.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.299+0000 7fb40bfff700 1 --2- 172.21.15.36:0/402860282 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb414140410 0x7fb4140b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:30.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.299+0000 7fb40bfff700 1 -- 172.21.15.36:0/402860282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4140abe30 msgr2=0x7fb4140b3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:30.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.299+0000 7fb40bfff700 1 --2- 172.21.15.36:0/402860282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4140abe30 0x7fb4140b3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:30.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.299+0000 7fb40bfff700 1 -- 172.21.15.36:0/402860282 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40400b180 con 0x7fb4140b1280 2022-06-17T12:55:30.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.299+0000 7fb4137fe700 1 --2- 172.21.15.36:0/402860282 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb414140410 0x7fb4140b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:30.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.299+0000 7fb40bfff700 1 --2- 172.21.15.36:0/402860282 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb4140b1280 0x7fb414139000 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0x7fb3fc002f70 tx=0x7fb4140624e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:30.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.299+0000 7fb410ff9700 1 -- 172.21.15.36:0/402860282 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3fc010da0 con 0x7fb4140b1280 2022-06-17T12:55:30.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.299+0000 7fb410ff9700 1 -- 172.21.15.36:0/402860282 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3fc01e070 con 0x7fb4140b1280 2022-06-17T12:55:30.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.299+0000 7fb413fff700 1 -- 172.21.15.36:0/402860282 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3fc00b460 con 0x7fb4140b1280 2022-06-17T12:55:30.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.299+0000 7fb410ff9700 1 -- 172.21.15.36:0/402860282 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3fc023610 con 0x7fb4140b1280 2022-06-17T12:55:30.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.299+0000 7fb413fff700 1 -- 172.21.15.36:0/402860282 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41413d110 con 0x7fb4140b1280 2022-06-17T12:55:30.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.299+0000 7fb413fff700 1 -- 172.21.15.36:0/402860282 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb41413b950 con 0x7fb4140b1280 2022-06-17T12:55:30.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.303+0000 7fb410ff9700 1 -- 172.21.15.36:0/402860282 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fb3fc02a070 con 0x7fb4140b1280 2022-06-17T12:55:30.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.303+0000 7fb410ff9700 1 --2- 172.21.15.36:0/402860282 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb3ec053920 0x7fb3ec055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:30.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.303+0000 7fb410ff9700 1 -- 172.21.15.36:0/402860282 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb3fc06d1a0 con 0x7fb4140b1280 2022-06-17T12:55:30.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.303+0000 7fb412ffd700 1 --2- 172.21.15.36:0/402860282 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb3ec053920 0x7fb3ec055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:30.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.303+0000 7fb412ffd700 1 --2- 172.21.15.36:0/402860282 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb3ec053920 0x7fb3ec055de0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7fb40400b000 tx=0x7fb404012040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:30.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.311+0000 7fb410ff9700 1 -- 172.21.15.36:0/402860282 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb3fc043fa0 con 0x7fb4140b1280 2022-06-17T12:55:30.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.475+0000 7fb413fff700 1 -- 172.21.15.36:0/402860282 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4140ac650 con 0x7fb4140b1280 2022-06-17T12:55:30.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.475+0000 7fb410ff9700 1 -- 172.21.15.36:0/402860282 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+2142 (secure 0 0 0) 0x7fb3fc0475c0 con 0x7fb4140b1280 2022-06-17T12:55:30.472 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:30.472 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":60,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1161585109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1161585109},{"type":"v1","addr":"172.21.15.70:6833","nonce":1161585109}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":57}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:55:24.825976+0000","modified":"2022-06-17T12:55:26.574711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7531":{"gid":7531,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/1342585509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":1342585509},{"type":"v1","addr":"172.21.15.36:6835","nonce":1342585509}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2022-06-17T12:55:30.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.479+0000 7fb413fff700 1 -- 172.21.15.36:0/402860282 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb3ec053920 msgr2=0x7fb3ec055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:30.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.479+0000 7fb413fff700 1 --2- 172.21.15.36:0/402860282 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb3ec053920 0x7fb3ec055de0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7fb40400b000 tx=0x7fb404012040).stop 2022-06-17T12:55:30.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.479+0000 7fb413fff700 1 -- 172.21.15.36:0/402860282 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb4140b1280 msgr2=0x7fb414139000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:30.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.479+0000 7fb413fff700 1 --2- 172.21.15.36:0/402860282 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb4140b1280 0x7fb414139000 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0x7fb3fc002f70 tx=0x7fb4140624e0).stop 2022-06-17T12:55:30.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.479+0000 7fb413fff700 1 -- 172.21.15.36:0/402860282 shutdown_connections 2022-06-17T12:55:30.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.479+0000 7fb413fff700 1 --2- 172.21.15.36:0/402860282 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb3ec053920 0x7fb3ec055de0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:30.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.479+0000 7fb413fff700 1 --2- 172.21.15.36:0/402860282 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb414140410 0x7fb4140b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:30.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.479+0000 7fb413fff700 1 --2- 172.21.15.36:0/402860282 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb4140b1280 0x7fb414139000 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:30.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.479+0000 7fb413fff700 1 --2- 172.21.15.36:0/402860282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4140abe30 0x7fb4140b3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:30.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.479+0000 7fb413fff700 1 -- 172.21.15.36:0/402860282 >> 172.21.15.36:0/402860282 conn(0x7fb41405f770 msgr2=0x7fb4140afe30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:30.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.479+0000 7fb413fff700 1 -- 172.21.15.36:0/402860282 shutdown_connections 2022-06-17T12:55:30.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:30.479+0000 7fb413fff700 1 -- 172.21.15.36:0/402860282 wait complete. 2022-06-17T12:55:30.481 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 60 2022-06-17T12:55:30.686 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:55:24.825976+0000', 'modified': '2022-06-17T12:55:26.574711+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7531}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7531': {'gid': 7531, 'name': 'a', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.36:6835/1342585509', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 1342585509}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 1342585509}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:55:30.687 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:55:30.694 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:55:30.826 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":7531,"want_state":"up:active","state":"up:active","rank_uptime":5.9994073129999999,"mdsmap_epoch":60,"osdmap_epoch":113,"osdmap_epoch_barrier":113,"uptime":10.368628001999999} 2022-06-17T12:55:30.828 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 7531, "mdsmap_epoch": 60, "osdmap_epoch": 113, "osdmap_epoch_barrier": 113, "rank_uptime": 5.999407313, "state": "up:active", "uptime": 10.368628002, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:55:30.829 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:55:30.830 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-17T12:55:30.831 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi107.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-17T12:55:30.831 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:30.832 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:55:30.842 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:55:30.872 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:55:30.873 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:55:30.941 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:55:30.942 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:55:30.943 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:30.944 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:55:30.955 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:55:30.959 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:55:31.003 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:55:31.029 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:55:31.029 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:55:31.080 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:2022-06-17T12:55:31.086+0000 7fc7b96fb080 -1 init, newargv = 0x55bbbdf87ec0 newargc=9 2022-06-17T12:55:31.082 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22781]: starting ceph client 2022-06-17T12:55:31.087 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:55:31.088 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22781]: starting fuse 2022-06-17T12:55:31.089 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:55:31.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:31.090 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:55:31.104 INFO:teuthology.orchestra.run.smithi107.stdout:52 2022-06-17T12:55:31.105 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:55:31.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:31.116 DEBUG:teuthology.orchestra.run.smithi107:> sudo python3 -c ' 2022-06-17T12:55:31.117 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2022-06-17T12:55:31.118 DEBUG:teuthology.orchestra.run.smithi107:> import re 2022-06-17T12:55:31.118 DEBUG:teuthology.orchestra.run.smithi107:> import os 2022-06-17T12:55:31.119 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2022-06-17T12:55:31.119 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:55:31.120 DEBUG:teuthology.orchestra.run.smithi107:> def find_socket(client_name): 2022-06-17T12:55:31.120 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-17T12:55:31.120 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2022-06-17T12:55:31.121 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:55:31.121 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2022-06-17T12:55:31.122 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2022-06-17T12:55:31.122 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2022-06-17T12:55:31.123 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2022-06-17T12:55:31.123 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:55:31.123 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2022-06-17T12:55:31.124 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:55:31.124 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:55:31.125 DEBUG:teuthology.orchestra.run.smithi107:> return f 2022-06-17T12:55:31.125 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:55:31.125 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:55:31.126 DEBUG:teuthology.orchestra.run.smithi107:> print(find_socket("client.0")) 2022-06-17T12:55:31.126 DEBUG:teuthology.orchestra.run.smithi107:> ' 2022-06-17T12:55:31.197 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.0.22781.asok 2022-06-17T12:55:31.203 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22781.asok 2022-06-17T12:55:31.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:31.206 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22781.asok status 2022-06-17T12:55:31.291 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2022-06-17T12:55:31.292 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2022-06-17T12:55:31.292 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:55:31.293 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:55:31.293 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "0", 2022-06-17T12:55:31.293 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2022-06-17T12:55:31.293 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-17T12:55:31.294 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "22781", 2022-06-17T12:55:31.294 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/" 2022-06-17T12:55:31.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:55:31.294 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2022-06-17T12:55:31.295 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2022-06-17T12:55:31.295 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 6740, 2022-06-17T12:55:31.295 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2022-06-17T12:55:31.296 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2022-06-17T12:55:31.297 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2022-06-17T12:55:31.297 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 6740 2022-06-17T12:55:31.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:55:31.298 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:55:31.298 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:55:31.299 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:55:31.299 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 2600543451 2022-06-17T12:55:31.299 INFO:teuthology.orchestra.run.smithi107.stdout: } 2022-06-17T12:55:31.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:55:31.299 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:55:31.300 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:55:31.300 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:55:31.300 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 2600543451 2022-06-17T12:55:31.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:55:31.301 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.6740 172.21.15.107:0/2600543451", 2022-06-17T12:55:31.301 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "172.21.15.107:0/2600543451", 2022-06-17T12:55:31.301 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 1, 2022-06-17T12:55:31.301 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 60, 2022-06-17T12:55:31.302 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 113, 2022-06-17T12:55:31.302 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:55:31.302 INFO:teuthology.orchestra.run.smithi107.stdout: "blacklisted": false, 2022-06-17T12:55:31.302 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2022-06-17T12:55:31.302 INFO:teuthology.orchestra.run.smithi107.stdout:} 2022-06-17T12:55:31.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:31.305 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:55:31.305 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:55:31.316 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:55:31.316 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:55:31.317 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:31.317 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:55:31.379 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:55:31.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f06716e3700 1 Processor -- start 2022-06-17T12:55:31.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f06716e3700 1 -- start start 2022-06-17T12:55:31.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f06716e3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f066c066c60 0x7f066c0ad880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:31.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f06716e3700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f066c0afd80 0x7f066c146640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:31.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f06716e3700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f066c0ad330 0x7f066c141fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:31.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f06716e3700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f066c0add70 con 0x7f066c0ad330 2022-06-17T12:55:31.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f06716e3700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f066c067040 con 0x7f066c0afd80 2022-06-17T12:55:31.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f06716e3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066c0b0940 con 0x7f066c066c60 2022-06-17T12:55:31.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f066b7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f066c0ad330 0x7f066c141fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:31.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f066b7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f066c0ad330 0x7f066c141fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38638/0 (socket says 172.21.15.36:38638) 2022-06-17T12:55:31.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f066b7fe700 1 -- 172.21.15.36:0/4245464882 learned_addr learned my addr 172.21.15.36:0/4245464882 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:31.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f06627fc700 1 --2- 172.21.15.36:0/4245464882 >> v2:172.21.15.70:3300/0 conn(0x7f066c0afd80 0x7f066c146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:31.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f066affd700 1 --2- 172.21.15.36:0/4245464882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f066c066c60 0x7f066c0ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:31.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f066b7fe700 1 -- 172.21.15.36:0/4245464882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f066c066c60 msgr2=0x7f066c0ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:31.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f066b7fe700 1 --2- 172.21.15.36:0/4245464882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f066c066c60 0x7f066c0ad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:31.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f066b7fe700 1 -- 172.21.15.36:0/4245464882 >> v2:172.21.15.70:3300/0 conn(0x7f066c0afd80 msgr2=0x7f066c146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:31.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f066b7fe700 1 --2- 172.21.15.36:0/4245464882 >> v2:172.21.15.70:3300/0 conn(0x7f066c0afd80 0x7f066c146640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:31.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f066b7fe700 1 -- 172.21.15.36:0/4245464882 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066c0b1300 con 0x7f066c0ad330 2022-06-17T12:55:31.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f066b7fe700 1 --2- 172.21.15.36:0/4245464882 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f066c0ad330 0x7f066c141fc0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 rx=0x7f06640032f0 tx=0x7f0664009290).ready entity=mon.0 client_cookie=fe8091025b62096a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:31.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f066a7fc700 1 -- 172.21.15.36:0/4245464882 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f066400b870 con 0x7f066c0ad330 2022-06-17T12:55:31.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f066a7fc700 1 -- 172.21.15.36:0/4245464882 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066400b9d0 con 0x7f066c0ad330 2022-06-17T12:55:31.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f066a7fc700 1 -- 172.21.15.36:0/4245464882 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f066400bb40 con 0x7f066c0ad330 2022-06-17T12:55:31.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f06716e3700 1 -- 172.21.15.36:0/4245464882 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f066c0ad330 msgr2=0x7f066c141fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:31.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f06716e3700 1 --2- 172.21.15.36:0/4245464882 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f066c0ad330 0x7f066c141fc0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 rx=0x7f06640032f0 tx=0x7f0664009290).stop 2022-06-17T12:55:31.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f06716e3700 1 -- 172.21.15.36:0/4245464882 shutdown_connections 2022-06-17T12:55:31.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f06716e3700 1 --2- 172.21.15.36:0/4245464882 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f066c0ad330 0x7f066c141fc0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:31.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f06716e3700 1 --2- 172.21.15.36:0/4245464882 >> v2:172.21.15.70:3300/0 conn(0x7f066c0afd80 0x7f066c146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:31.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f06716e3700 1 --2- 172.21.15.36:0/4245464882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f066c066c60 0x7f066c0ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:31.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f06716e3700 1 -- 172.21.15.36:0/4245464882 >> 172.21.15.36:0/4245464882 conn(0x7f066c0b53e0 msgr2=0x7f066c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:31.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f06716e3700 1 -- 172.21.15.36:0/4245464882 shutdown_connections 2022-06-17T12:55:31.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.507+0000 7f06716e3700 1 -- 172.21.15.36:0/4245464882 wait complete. 2022-06-17T12:55:31.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f06716e3700 1 Processor -- start 2022-06-17T12:55:31.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f06716e3700 1 -- start start 2022-06-17T12:55:31.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f06716e3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f066c066c60 0x7f066c13c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:31.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f06716e3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f066c0ad330 0x7f066c1566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:31.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f06716e3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f066c0afd80 0x7f066c152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:31.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f06716e3700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066c138ce0 con 0x7f066c066c60 2022-06-17T12:55:31.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f06716e3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066c13f7f0 con 0x7f066c0ad330 2022-06-17T12:55:31.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f06716e3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066c155f80 con 0x7f066c0afd80 2022-06-17T12:55:31.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f066affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f066c066c60 0x7f066c13c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:31.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f066affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f066c066c60 0x7f066c13c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38640/0 (socket says 172.21.15.36:38640) 2022-06-17T12:55:31.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f066affd700 1 -- 172.21.15.36:0/19085629 learned_addr learned my addr 172.21.15.36:0/19085629 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:31.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f06627fc700 1 --2- 172.21.15.36:0/19085629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f066c0ad330 0x7f066c1566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:31.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f066affd700 1 -- 172.21.15.36:0/19085629 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f066c0afd80 msgr2=0x7f066c152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:31.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f066b7fe700 1 --2- 172.21.15.36:0/19085629 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f066c0afd80 0x7f066c152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:31.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f066affd700 1 --2- 172.21.15.36:0/19085629 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f066c0afd80 0x7f066c152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:31.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f066affd700 1 -- 172.21.15.36:0/19085629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f066c0ad330 msgr2=0x7f066c1566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:31.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f066affd700 1 --2- 172.21.15.36:0/19085629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f066c0ad330 0x7f066c1566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:31.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f066affd700 1 -- 172.21.15.36:0/19085629 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0664009530 con 0x7f066c066c60 2022-06-17T12:55:31.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f06627fc700 1 --2- 172.21.15.36:0/19085629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f066c0ad330 0x7f066c1566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:31.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f066b7fe700 1 --2- 172.21.15.36:0/19085629 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f066c0afd80 0x7f066c152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:55:31.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f066affd700 1 --2- 172.21.15.36:0/19085629 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f066c066c60 0x7f066c13c580 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7f065c0030b0 tx=0x7f065c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:31.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f0668ff9700 1 -- 172.21.15.36:0/19085629 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f065c011780 con 0x7f066c066c60 2022-06-17T12:55:31.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f0668ff9700 1 -- 172.21.15.36:0/19085629 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065c0118e0 con 0x7f066c066c60 2022-06-17T12:55:31.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f06716e3700 1 -- 172.21.15.36:0/19085629 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f066c156220 con 0x7f066c066c60 2022-06-17T12:55:31.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f0668ff9700 1 -- 172.21.15.36:0/19085629 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f065c011c40 con 0x7f066c066c60 2022-06-17T12:55:31.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.511+0000 7f06716e3700 1 -- 172.21.15.36:0/19085629 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066c149510 con 0x7f066c066c60 2022-06-17T12:55:31.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.515+0000 7f06716e3700 1 -- 172.21.15.36:0/19085629 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066c1563b0 con 0x7f066c066c60 2022-06-17T12:55:31.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.515+0000 7f0668ff9700 1 -- 172.21.15.36:0/19085629 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f065c012420 con 0x7f066c066c60 2022-06-17T12:55:31.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.515+0000 7f0668ff9700 1 --2- 172.21.15.36:0/19085629 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0650053a60 0x7f0650055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:31.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.515+0000 7f06627fc700 1 --2- 172.21.15.36:0/19085629 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0650053a60 0x7f0650055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:31.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.515+0000 7f0668ff9700 1 -- 172.21.15.36:0/19085629 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f065c06bfc0 con 0x7f066c066c60 2022-06-17T12:55:31.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.515+0000 7f06627fc700 1 --2- 172.21.15.36:0/19085629 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0650053a60 0x7f0650055f20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f065400b000 tx=0x7f0654008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:31.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.519+0000 7f0668ff9700 1 -- 172.21.15.36:0/19085629 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f065c042c50 con 0x7f066c066c60 2022-06-17T12:55:31.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.647+0000 7f06716e3700 1 -- 172.21.15.36:0/19085629 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f066c0af470 con 0x7f066c066c60 2022-06-17T12:55:31.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.651+0000 7f0668ff9700 1 -- 172.21.15.36:0/19085629 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f065c046270 con 0x7f066c066c60 2022-06-17T12:55:31.639 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:31.639 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":113,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:55:25.828833+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:55:23.324881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-06-17T12:55:24.417060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:55:31.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.651+0000 7f06716e3700 1 -- 172.21.15.36:0/19085629 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0650053a60 msgr2=0x7f0650055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:31.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.651+0000 7f06716e3700 1 --2- 172.21.15.36:0/19085629 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0650053a60 0x7f0650055f20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f065400b000 tx=0x7f0654008040).stop 2022-06-17T12:55:31.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.651+0000 7f06716e3700 1 -- 172.21.15.36:0/19085629 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f066c066c60 msgr2=0x7f066c13c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:31.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.651+0000 7f06716e3700 1 --2- 172.21.15.36:0/19085629 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f066c066c60 0x7f066c13c580 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7f065c0030b0 tx=0x7f065c008040).stop 2022-06-17T12:55:31.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.651+0000 7f06716e3700 1 -- 172.21.15.36:0/19085629 shutdown_connections 2022-06-17T12:55:31.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.651+0000 7f06716e3700 1 --2- 172.21.15.36:0/19085629 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0650053a60 0x7f0650055f20 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:31.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.651+0000 7f06716e3700 1 --2- 172.21.15.36:0/19085629 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f066c0afd80 0x7f066c152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:31.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.651+0000 7f06716e3700 1 --2- 172.21.15.36:0/19085629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f066c0ad330 0x7f066c1566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:31.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.651+0000 7f06716e3700 1 --2- 172.21.15.36:0/19085629 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f066c066c60 0x7f066c13c580 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:31.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.651+0000 7f06716e3700 1 -- 172.21.15.36:0/19085629 >> 172.21.15.36:0/19085629 conn(0x7f066c0b53e0 msgr2=0x7f066c13d840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:31.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.651+0000 7f06716e3700 1 -- 172.21.15.36:0/19085629 shutdown_connections 2022-06-17T12:55:31.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.651+0000 7f06716e3700 1 -- 172.21.15.36:0/19085629 wait complete. 2022-06-17T12:55:31.652 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:55:23.324881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:55:24.417060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:55:31.654 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:55:31.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.783+0000 7f1b47461700 1 Processor -- start 2022-06-17T12:55:31.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.783+0000 7f1b47461700 1 -- start start 2022-06-17T12:55:31.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.783+0000 7f1b47461700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1b400b27b0 0x7f1b400b1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:31.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.783+0000 7f1b47461700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b40144d20 0x7f1b40140230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:31.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.783+0000 7f1b47461700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b400ac0b0 con 0x7f1b40140720 2022-06-17T12:55:31.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.783+0000 7f1b47461700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b400b2b90 con 0x7f1b400b27b0 2022-06-17T12:55:31.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.783+0000 7f1b47461700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b400afc60 con 0x7f1b40144d20 2022-06-17T12:55:31.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.783+0000 7f1b46c60700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f1b40140720 0x7f1b40141180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56182/0 (socket says 172.21.15.36:56182) 2022-06-17T12:55:31.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.783+0000 7f1b46c60700 1 -- 172.21.15.36:0/979121571 learned_addr learned my addr 172.21.15.36:0/979121571 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:31.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.783+0000 7f1b4645f700 1 --2- 172.21.15.36:0/979121571 >> v2:172.21.15.70:3300/0 conn(0x7f1b400b27b0 0x7f1b400b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:31.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.783+0000 7f1b45c5e700 1 --2- 172.21.15.36:0/979121571 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b40144d20 0x7f1b40140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:31.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.783+0000 7f1b4545d700 1 -- 172.21.15.36:0/979121571 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 9196030 0 0) 0x7f1b400ac0b0 con 0x7f1b40140720 2022-06-17T12:55:31.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.783+0000 7f1b4545d700 1 -- 172.21.15.36:0/979121571 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b28003260 con 0x7f1b40140720 2022-06-17T12:55:31.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.783+0000 7f1b45c5e700 1 -- 172.21.15.36:0/979121571 >> v2:172.21.15.70:3300/0 conn(0x7f1b400b27b0 msgr2=0x7f1b400b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:31.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.783+0000 7f1b45c5e700 1 --2- 172.21.15.36:0/979121571 >> v2:172.21.15.70:3300/0 conn(0x7f1b400b27b0 0x7f1b400b1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:31.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.783+0000 7f1b45c5e700 1 -- 172.21.15.36:0/979121571 >> v1:172.21.15.36:6789/0 conn(0x7f1b40140720 legacy=0x7f1b40141180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:31.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.783+0000 7f1b45c5e700 1 -- 172.21.15.36:0/979121571 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b400ac310 con 0x7f1b40144d20 2022-06-17T12:55:31.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.783+0000 7f1b45c5e700 1 --2- 172.21.15.36:0/979121571 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b40144d20 0x7f1b40140230 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f1b30002f70 tx=0x7f1b30006df0).ready entity=mon.2 client_cookie=f7b64c77e3d752ea server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:31.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.783+0000 7f1b4545d700 1 -- 172.21.15.36:0/979121571 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b3000d4a0 con 0x7f1b40144d20 2022-06-17T12:55:31.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.787+0000 7f1b4545d700 1 -- 172.21.15.36:0/979121571 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b30000e90 con 0x7f1b40144d20 2022-06-17T12:55:31.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.787+0000 7f1b4545d700 1 -- 172.21.15.36:0/979121571 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b3000d7f0 con 0x7f1b40144d20 2022-06-17T12:55:31.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.787+0000 7f1b47461700 1 -- 172.21.15.36:0/979121571 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b40144d20 msgr2=0x7f1b40140230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:31.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.787+0000 7f1b47461700 1 --2- 172.21.15.36:0/979121571 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b40144d20 0x7f1b40140230 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f1b30002f70 tx=0x7f1b30006df0).stop 2022-06-17T12:55:31.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.787+0000 7f1b47461700 1 -- 172.21.15.36:0/979121571 shutdown_connections 2022-06-17T12:55:31.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.787+0000 7f1b47461700 1 --2- 172.21.15.36:0/979121571 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b40144d20 0x7f1b40140230 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:31.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.787+0000 7f1b47461700 1 --2- 172.21.15.36:0/979121571 >> v2:172.21.15.70:3300/0 conn(0x7f1b400b27b0 0x7f1b400b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:31.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.787+0000 7f1b47461700 1 -- 172.21.15.36:0/979121571 >> 172.21.15.36:0/979121571 conn(0x7f1b400b5d10 msgr2=0x7f1b400b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:31.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.787+0000 7f1b47461700 1 -- 172.21.15.36:0/979121571 shutdown_connections 2022-06-17T12:55:31.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.787+0000 7f1b47461700 1 -- 172.21.15.36:0/979121571 wait complete. 2022-06-17T12:55:31.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.787+0000 7f1b47461700 1 Processor -- start 2022-06-17T12:55:31.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.787+0000 7f1b47461700 1 -- start start 2022-06-17T12:55:31.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.791+0000 7f1b47461700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b400b27b0 0x7f1b400af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:31.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.791+0000 7f1b47461700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1b40140720 0x7f1b4013ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:31.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.791+0000 7f1b47461700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b40144d20 0x7f1b40066d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:31.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.791+0000 7f1b47461700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4013eca0 con 0x7f1b40140720 2022-06-17T12:55:31.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.791+0000 7f1b47461700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b400664d0 con 0x7f1b400b27b0 2022-06-17T12:55:31.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.791+0000 7f1b47461700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b40066770 con 0x7f1b40144d20 2022-06-17T12:55:31.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.791+0000 7f1b45c5e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1b40140720 0x7f1b4013ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:31.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.791+0000 7f1b45c5e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1b40140720 0x7f1b4013ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38656/0 (socket says 172.21.15.36:38656) 2022-06-17T12:55:31.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.791+0000 7f1b45c5e700 1 -- 172.21.15.36:0/3462498399 learned_addr learned my addr 172.21.15.36:0/3462498399 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:31.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.791+0000 7f1b4645f700 1 --2- 172.21.15.36:0/3462498399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b400b27b0 0x7f1b400af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:31.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.791+0000 7f1b46c60700 1 --2- 172.21.15.36:0/3462498399 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b40144d20 0x7f1b40066d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:31.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.791+0000 7f1b4645f700 1 -- 172.21.15.36:0/3462498399 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b40144d20 msgr2=0x7f1b40066d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:31.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.791+0000 7f1b4645f700 1 --2- 172.21.15.36:0/3462498399 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b40144d20 0x7f1b40066d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:31.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.791+0000 7f1b4645f700 1 -- 172.21.15.36:0/3462498399 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1b40140720 msgr2=0x7f1b4013ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:31.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.791+0000 7f1b4645f700 1 --2- 172.21.15.36:0/3462498399 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1b40140720 0x7f1b4013ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:31.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.791+0000 7f1b4645f700 1 -- 172.21.15.36:0/3462498399 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b3000b180 con 0x7f1b400b27b0 2022-06-17T12:55:31.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.791+0000 7f1b45c5e700 1 --2- 172.21.15.36:0/3462498399 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1b40140720 0x7f1b4013ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:31.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.791+0000 7f1b4645f700 1 --2- 172.21.15.36:0/3462498399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b400b27b0 0x7f1b400af160 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f1b3c00c000 tx=0x7f1b3c006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:31.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.791+0000 7f1b2f7fe700 1 -- 172.21.15.36:0/3462498399 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b3c015070 con 0x7f1b400b27b0 2022-06-17T12:55:31.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.791+0000 7f1b2f7fe700 1 -- 172.21.15.36:0/3462498399 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3c00a7e0 con 0x7f1b400b27b0 2022-06-17T12:55:31.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.791+0000 7f1b47461700 1 -- 172.21.15.36:0/3462498399 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b400aecc0 con 0x7f1b400b27b0 2022-06-17T12:55:31.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.791+0000 7f1b2f7fe700 1 -- 172.21.15.36:0/3462498399 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b3c00ab40 con 0x7f1b400b27b0 2022-06-17T12:55:31.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.791+0000 7f1b47461700 1 -- 172.21.15.36:0/3462498399 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4013f760 con 0x7f1b400b27b0 2022-06-17T12:55:31.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.795+0000 7f1b47461700 1 -- 172.21.15.36:0/3462498399 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b40145160 con 0x7f1b400b27b0 2022-06-17T12:55:31.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.795+0000 7f1b2f7fe700 1 -- 172.21.15.36:0/3462498399 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f1b3c021070 con 0x7f1b400b27b0 2022-06-17T12:55:31.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.795+0000 7f1b2f7fe700 1 --2- 172.21.15.36:0/3462498399 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1b20053a60 0x7f1b20055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:31.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.795+0000 7f1b2f7fe700 1 -- 172.21.15.36:0/3462498399 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1b3c06f760 con 0x7f1b400b27b0 2022-06-17T12:55:31.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.795+0000 7f1b45c5e700 1 --2- 172.21.15.36:0/3462498399 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1b20053a60 0x7f1b20055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:31.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.795+0000 7f1b45c5e700 1 --2- 172.21.15.36:0/3462498399 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1b20053a60 0x7f1b20055f20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f1b3001a000 tx=0x7f1b30014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:31.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.803+0000 7f1b2f7fe700 1 -- 172.21.15.36:0/3462498399 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1b3c046450 con 0x7f1b400b27b0 2022-06-17T12:55:31.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.927+0000 7f1b47461700 1 -- 172.21.15.36:0/3462498399 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f1b40057410 con 0x7f1b400b27b0 2022-06-17T12:55:31.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.931+0000 7f1b2f7fe700 1 -- 172.21.15.36:0/3462498399 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v113) v1 ==== 110+0+10 (secure 0 0 0) 0x7f1b3c049a70 con 0x7f1b400b27b0 2022-06-17T12:55:31.920 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:55:31.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.931+0000 7f1b47461700 1 -- 172.21.15.36:0/3462498399 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1b20053a60 msgr2=0x7f1b20055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:31.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.931+0000 7f1b47461700 1 --2- 172.21.15.36:0/3462498399 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1b20053a60 0x7f1b20055f20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f1b3001a000 tx=0x7f1b30014000).stop 2022-06-17T12:55:31.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.931+0000 7f1b47461700 1 -- 172.21.15.36:0/3462498399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b400b27b0 msgr2=0x7f1b400af160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:31.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.931+0000 7f1b47461700 1 --2- 172.21.15.36:0/3462498399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b400b27b0 0x7f1b400af160 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f1b3c00c000 tx=0x7f1b3c006df0).stop 2022-06-17T12:55:31.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.931+0000 7f1b47461700 1 -- 172.21.15.36:0/3462498399 shutdown_connections 2022-06-17T12:55:31.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.931+0000 7f1b47461700 1 --2- 172.21.15.36:0/3462498399 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1b20053a60 0x7f1b20055f20 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:31.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.931+0000 7f1b47461700 1 --2- 172.21.15.36:0/3462498399 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b40144d20 0x7f1b40066d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:31.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.931+0000 7f1b47461700 1 --2- 172.21.15.36:0/3462498399 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1b40140720 0x7f1b4013ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:31.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.931+0000 7f1b47461700 1 --2- 172.21.15.36:0/3462498399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b400b27b0 0x7f1b400af160 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:31.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.931+0000 7f1b47461700 1 -- 172.21.15.36:0/3462498399 >> 172.21.15.36:0/3462498399 conn(0x7f1b400b5d10 msgr2=0x7f1b400b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:31.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.931+0000 7f1b47461700 1 -- 172.21.15.36:0/3462498399 shutdown_connections 2022-06-17T12:55:31.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:31.931+0000 7f1b47461700 1 -- 172.21.15.36:0/3462498399 wait complete. 2022-06-17T12:55:31.932 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:55:32.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.051+0000 7fc8015a9700 1 Processor -- start 2022-06-17T12:55:32.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.051+0000 7fc8015a9700 1 -- start start 2022-06-17T12:55:32.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.051+0000 7fc8015a9700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc7fc0b45c0 0x7fc7fc0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:32.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.051+0000 7fc8015a9700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc7fc0b4f00 0x7fc7fc0b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:32.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.051+0000 7fc8015a9700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7fc0b4160 con 0x7fc7fc0b9860 2022-06-17T12:55:32.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.051+0000 7fc8015a9700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7fc0b9c40 con 0x7fc7fc0b45c0 2022-06-17T12:55:32.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.051+0000 7fc8015a9700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7fc0627d0 con 0x7fc7fc0b4f00 2022-06-17T12:55:32.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.051+0000 7fc7fb7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc7fc0b45c0 0x7fc7fc0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:32.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.051+0000 7fc7fb7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc7fc0b45c0 0x7fc7fc0b4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38664/0 (socket says 172.21.15.36:38664) 2022-06-17T12:55:32.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.051+0000 7fc7fb7fe700 1 -- 172.21.15.36:0/2453228259 learned_addr learned my addr 172.21.15.36:0/2453228259 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:32.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.051+0000 7fc800da8700 1 --2- 172.21.15.36:0/2453228259 >> v2:172.21.15.70:3300/0 conn(0x7fc7fc0b4f00 0x7fc7fc0b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:32.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.055+0000 7fc7fb7fe700 1 -- 172.21.15.36:0/2453228259 >> v2:172.21.15.70:3300/0 conn(0x7fc7fc0b4f00 msgr2=0x7fc7fc0b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:32.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.055+0000 7fc7fb7fe700 1 --2- 172.21.15.36:0/2453228259 >> v2:172.21.15.70:3300/0 conn(0x7fc7fc0b4f00 0x7fc7fc0b5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.055+0000 7fc7fb7fe700 1 -- 172.21.15.36:0/2453228259 >> v1:172.21.15.70:6789/0 conn(0x7fc7fc0b9860 legacy=0x7fc7fc143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:32.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.055+0000 7fc7fb7fe700 1 -- 172.21.15.36:0/2453228259 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7fc062a70 con 0x7fc7fc0b45c0 2022-06-17T12:55:32.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.055+0000 7fc7fb7fe700 1 --2- 172.21.15.36:0/2453228259 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc7fc0b45c0 0x7fc7fc0b4a10 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7fc7fc0ba5d0 tx=0x7fc7f0006eb0).ready entity=mon.0 client_cookie=a708acd3b8f4e829 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:32.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.055+0000 7fc7faffd700 1 -- 172.21.15.36:0/2453228259 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7f000d610 con 0x7fc7fc0b45c0 2022-06-17T12:55:32.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.055+0000 7fc7faffd700 1 -- 172.21.15.36:0/2453228259 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f0000e90 con 0x7fc7fc0b45c0 2022-06-17T12:55:32.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.055+0000 7fc7faffd700 1 -- 172.21.15.36:0/2453228259 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7f000d9a0 con 0x7fc7fc0b45c0 2022-06-17T12:55:32.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.055+0000 7fc8015a9700 1 -- 172.21.15.36:0/2453228259 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc7fc0b45c0 msgr2=0x7fc7fc0b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:32.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.055+0000 7fc8015a9700 1 --2- 172.21.15.36:0/2453228259 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc7fc0b45c0 0x7fc7fc0b4a10 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7fc7fc0ba5d0 tx=0x7fc7f0006eb0).stop 2022-06-17T12:55:32.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.055+0000 7fc8015a9700 1 -- 172.21.15.36:0/2453228259 shutdown_connections 2022-06-17T12:55:32.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.055+0000 7fc8015a9700 1 --2- 172.21.15.36:0/2453228259 >> v2:172.21.15.70:3300/0 conn(0x7fc7fc0b4f00 0x7fc7fc0b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.055+0000 7fc8015a9700 1 --2- 172.21.15.36:0/2453228259 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc7fc0b45c0 0x7fc7fc0b4a10 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.055+0000 7fc8015a9700 1 -- 172.21.15.36:0/2453228259 >> 172.21.15.36:0/2453228259 conn(0x7fc7fc05c550 msgr2=0x7fc7fc060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:32.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.055+0000 7fc8015a9700 1 -- 172.21.15.36:0/2453228259 shutdown_connections 2022-06-17T12:55:32.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.055+0000 7fc8015a9700 1 -- 172.21.15.36:0/2453228259 wait complete. 2022-06-17T12:55:32.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.055+0000 7fc8015a9700 1 Processor -- start 2022-06-17T12:55:32.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.059+0000 7fc8015a9700 1 -- start start 2022-06-17T12:55:32.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.059+0000 7fc8015a9700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7fc0b45c0 0x7fc7fc13d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:32.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.059+0000 7fc8015a9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc7fc0b4f00 0x7fc7fc141a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:32.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.059+0000 7fc8015a9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7fc0b9860 0x7fc7fc152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:32.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.059+0000 7fc8015a9700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7fc13e130 con 0x7fc7fc0b4f00 2022-06-17T12:55:32.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.059+0000 7fc8015a9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7fc1522d0 con 0x7fc7fc0b9860 2022-06-17T12:55:32.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.059+0000 7fc8015a9700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7fc141f10 con 0x7fc7fc0b45c0 2022-06-17T12:55:32.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.059+0000 7fc7fb7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc7fc0b4f00 0x7fc7fc141a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:32.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.059+0000 7fc7fb7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc7fc0b4f00 0x7fc7fc141a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38668/0 (socket says 172.21.15.36:38668) 2022-06-17T12:55:32.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.059+0000 7fc7fb7fe700 1 -- 172.21.15.36:0/3705645363 learned_addr learned my addr 172.21.15.36:0/3705645363 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:32.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.059+0000 7fc800da8700 1 --2- 172.21.15.36:0/3705645363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7fc0b9860 0x7fc7fc152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:32.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.059+0000 7fc7fbfff700 1 --2- 172.21.15.36:0/3705645363 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7fc0b45c0 0x7fc7fc13d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:32.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.059+0000 7fc7fb7fe700 1 -- 172.21.15.36:0/3705645363 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7fc0b45c0 msgr2=0x7fc7fc13d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:32.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.059+0000 7fc7fb7fe700 1 --2- 172.21.15.36:0/3705645363 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7fc0b45c0 0x7fc7fc13d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.059+0000 7fc7fb7fe700 1 -- 172.21.15.36:0/3705645363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7fc0b9860 msgr2=0x7fc7fc152960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:32.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.059+0000 7fc7fb7fe700 1 --2- 172.21.15.36:0/3705645363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7fc0b9860 0x7fc7fc152960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.059+0000 7fc7fb7fe700 1 -- 172.21.15.36:0/3705645363 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f000b1e0 con 0x7fc7fc0b4f00 2022-06-17T12:55:32.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.059+0000 7fc7fb7fe700 1 --2- 172.21.15.36:0/3705645363 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc7fc0b4f00 0x7fc7fc141a20 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 rx=0x7fc7fc0ba5d0 tx=0x7fc7f0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:32.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.059+0000 7fc7f97fa700 1 -- 172.21.15.36:0/3705645363 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7f0011860 con 0x7fc7fc0b4f00 2022-06-17T12:55:32.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.059+0000 7fc8015a9700 1 -- 172.21.15.36:0/3705645363 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7fc1421a0 con 0x7fc7fc0b4f00 2022-06-17T12:55:32.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.059+0000 7fc7f97fa700 1 -- 172.21.15.36:0/3705645363 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f00119c0 con 0x7fc7fc0b4f00 2022-06-17T12:55:32.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.059+0000 7fc8015a9700 1 -- 172.21.15.36:0/3705645363 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7fc0afe60 con 0x7fc7fc0b4f00 2022-06-17T12:55:32.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.059+0000 7fc7f97fa700 1 -- 172.21.15.36:0/3705645363 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7f0011d20 con 0x7fc7fc0b4f00 2022-06-17T12:55:32.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.059+0000 7fc8015a9700 1 -- 172.21.15.36:0/3705645363 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7fc062eb0 con 0x7fc7fc0b4f00 2022-06-17T12:55:32.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.063+0000 7fc7f97fa700 1 -- 172.21.15.36:0/3705645363 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fc7f0025030 con 0x7fc7fc0b4f00 2022-06-17T12:55:32.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.063+0000 7fc7f97fa700 1 --2- 172.21.15.36:0/3705645363 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc7dc053af0 0x7fc7dc055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:32.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.063+0000 7fc7f97fa700 1 -- 172.21.15.36:0/3705645363 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc7f006d240 con 0x7fc7fc0b4f00 2022-06-17T12:55:32.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.063+0000 7fc7fbfff700 1 --2- 172.21.15.36:0/3705645363 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc7dc053af0 0x7fc7dc055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:32.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.063+0000 7fc7fbfff700 1 --2- 172.21.15.36:0/3705645363 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc7dc053af0 0x7fc7dc055fb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7fc7ec00b000 tx=0x7fc7ec009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:32.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.071+0000 7fc7f97fa700 1 -- 172.21.15.36:0/3705645363 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc7f0043fe0 con 0x7fc7fc0b4f00 2022-06-17T12:55:32.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.195+0000 7fc8015a9700 1 -- 172.21.15.36:0/3705645363 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc7fc057410 con 0x7fc7fc0b4f00 2022-06-17T12:55:32.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.195+0000 7fc7f97fa700 1 -- 172.21.15.36:0/3705645363 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v113) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc7f00230a0 con 0x7fc7fc0b4f00 2022-06-17T12:55:32.187 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:55:32.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.199+0000 7fc8015a9700 1 -- 172.21.15.36:0/3705645363 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc7dc053af0 msgr2=0x7fc7dc055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:32.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.199+0000 7fc8015a9700 1 --2- 172.21.15.36:0/3705645363 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc7dc053af0 0x7fc7dc055fb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7fc7ec00b000 tx=0x7fc7ec009040).stop 2022-06-17T12:55:32.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.199+0000 7fc8015a9700 1 -- 172.21.15.36:0/3705645363 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc7fc0b4f00 msgr2=0x7fc7fc141a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:32.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.199+0000 7fc8015a9700 1 --2- 172.21.15.36:0/3705645363 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc7fc0b4f00 0x7fc7fc141a20 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 rx=0x7fc7fc0ba5d0 tx=0x7fc7f0015000).stop 2022-06-17T12:55:32.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.199+0000 7fc8015a9700 1 -- 172.21.15.36:0/3705645363 shutdown_connections 2022-06-17T12:55:32.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.199+0000 7fc8015a9700 1 --2- 172.21.15.36:0/3705645363 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc7dc053af0 0x7fc7dc055fb0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.199+0000 7fc8015a9700 1 --2- 172.21.15.36:0/3705645363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7fc0b9860 0x7fc7fc152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.199+0000 7fc8015a9700 1 --2- 172.21.15.36:0/3705645363 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc7fc0b4f00 0x7fc7fc141a20 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.199+0000 7fc8015a9700 1 --2- 172.21.15.36:0/3705645363 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7fc0b45c0 0x7fc7fc13d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.199+0000 7fc8015a9700 1 -- 172.21.15.36:0/3705645363 >> 172.21.15.36:0/3705645363 conn(0x7fc7fc05c550 msgr2=0x7fc7fc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:32.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.199+0000 7fc8015a9700 1 -- 172.21.15.36:0/3705645363 shutdown_connections 2022-06-17T12:55:32.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.199+0000 7fc8015a9700 1 -- 172.21.15.36:0/3705645363 wait complete. 2022-06-17T12:55:32.198 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:55:32.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.327+0000 7f9bb1d0d700 1 Processor -- start 2022-06-17T12:55:32.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.327+0000 7f9bb1d0d700 1 -- start start 2022-06-17T12:55:32.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.327+0000 7f9bb1d0d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9bac0b45c0 0x7f9bac0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:32.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.327+0000 7f9bb1d0d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9bac0b4f00 0x7f9bac0b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:32.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.327+0000 7f9bb1d0d700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bac0b4160 con 0x7f9bac0b9860 2022-06-17T12:55:32.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.327+0000 7f9bb1d0d700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bac0b9c40 con 0x7f9bac0b4f00 2022-06-17T12:55:32.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.327+0000 7f9bb1d0d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac0627d0 con 0x7f9bac0b45c0 2022-06-17T12:55:32.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.327+0000 7f9bb150c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9bac0b4f00 0x7f9bac0b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:32.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.327+0000 7f9bb0d0b700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f9bac0b9860 0x7f9bac143840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56206/0 (socket says 172.21.15.36:56206) 2022-06-17T12:55:32.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.327+0000 7f9bb0d0b700 1 -- 172.21.15.36:0/1534043735 learned_addr learned my addr 172.21.15.36:0/1534043735 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:32.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.327+0000 7f9babfff700 1 --2- 172.21.15.36:0/1534043735 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9bac0b45c0 0x7f9bac0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:32.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.327+0000 7f9bb150c700 1 -- 172.21.15.36:0/1534043735 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9bac0b45c0 msgr2=0x7f9bac0b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:32.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.327+0000 7f9bb150c700 1 --2- 172.21.15.36:0/1534043735 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9bac0b45c0 0x7f9bac0b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.327+0000 7f9bb150c700 1 -- 172.21.15.36:0/1534043735 >> v1:172.21.15.36:6789/0 conn(0x7f9bac0b9860 legacy=0x7f9bac143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:32.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.327+0000 7f9bb150c700 1 -- 172.21.15.36:0/1534043735 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bac062a70 con 0x7f9bac0b4f00 2022-06-17T12:55:32.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.327+0000 7f9bb150c700 1 --2- 172.21.15.36:0/1534043735 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9bac0b4f00 0x7f9bac0b5dc0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7f9ba40032f0 tx=0x7f9ba40091d0).ready entity=mon.0 client_cookie=482d190b70db7075 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:32.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.327+0000 7f9bab7fe700 1 -- 172.21.15.36:0/1534043735 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ba4009ea0 con 0x7f9bac0b4f00 2022-06-17T12:55:32.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.327+0000 7f9bab7fe700 1 -- 172.21.15.36:0/1534043735 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba400b8a0 con 0x7f9bac0b4f00 2022-06-17T12:55:32.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.327+0000 7f9bab7fe700 1 -- 172.21.15.36:0/1534043735 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ba400ba10 con 0x7f9bac0b4f00 2022-06-17T12:55:32.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.331+0000 7f9bb1d0d700 1 -- 172.21.15.36:0/1534043735 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9bac0b4f00 msgr2=0x7f9bac0b5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:32.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.331+0000 7f9bb1d0d700 1 --2- 172.21.15.36:0/1534043735 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9bac0b4f00 0x7f9bac0b5dc0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7f9ba40032f0 tx=0x7f9ba40091d0).stop 2022-06-17T12:55:32.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.331+0000 7f9bb1d0d700 1 -- 172.21.15.36:0/1534043735 shutdown_connections 2022-06-17T12:55:32.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.331+0000 7f9bb1d0d700 1 --2- 172.21.15.36:0/1534043735 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9bac0b4f00 0x7f9bac0b5dc0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.331+0000 7f9bb1d0d700 1 --2- 172.21.15.36:0/1534043735 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9bac0b45c0 0x7f9bac0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.331+0000 7f9bb1d0d700 1 -- 172.21.15.36:0/1534043735 >> 172.21.15.36:0/1534043735 conn(0x7f9bac05c550 msgr2=0x7f9bac060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:32.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.331+0000 7f9bb1d0d700 1 -- 172.21.15.36:0/1534043735 shutdown_connections 2022-06-17T12:55:32.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.331+0000 7f9bb1d0d700 1 -- 172.21.15.36:0/1534043735 wait complete. 2022-06-17T12:55:32.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.331+0000 7f9bb1d0d700 1 Processor -- start 2022-06-17T12:55:32.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.331+0000 7f9bb1d0d700 1 -- start start 2022-06-17T12:55:32.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.331+0000 7f9bb1d0d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bac0b45c0 0x7f9bac0afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:32.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.331+0000 7f9bb1d0d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9bac0b4f00 0x7f9bac155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:32.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.331+0000 7f9bb1d0d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9bac0b9860 0x7f9bac155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:32.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.331+0000 7f9bb1d0d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac1421f0 con 0x7f9bac0b4f00 2022-06-17T12:55:32.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.331+0000 7f9bb1d0d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac13f720 con 0x7f9bac0b45c0 2022-06-17T12:55:32.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.331+0000 7f9bb1d0d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac13f9c0 con 0x7f9bac0b9860 2022-06-17T12:55:32.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.331+0000 7f9babfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9bac0b4f00 0x7f9bac155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:32.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.331+0000 7f9babfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9bac0b4f00 0x7f9bac155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38684/0 (socket says 172.21.15.36:38684) 2022-06-17T12:55:32.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.331+0000 7f9babfff700 1 -- 172.21.15.36:0/3294430564 learned_addr learned my addr 172.21.15.36:0/3294430564 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:32.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.335+0000 7f9bb0d0b700 1 --2- 172.21.15.36:0/3294430564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bac0b45c0 0x7f9bac0afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:32.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.335+0000 7f9bb150c700 1 --2- 172.21.15.36:0/3294430564 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9bac0b9860 0x7f9bac155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:32.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.335+0000 7f9babfff700 1 -- 172.21.15.36:0/3294430564 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9bac0b9860 msgr2=0x7f9bac155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:32.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.335+0000 7f9babfff700 1 --2- 172.21.15.36:0/3294430564 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9bac0b9860 0x7f9bac155f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.335+0000 7f9babfff700 1 -- 172.21.15.36:0/3294430564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bac0b45c0 msgr2=0x7f9bac0afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:32.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.335+0000 7f9babfff700 1 --2- 172.21.15.36:0/3294430564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bac0b45c0 0x7f9bac0afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.335+0000 7f9babfff700 1 -- 172.21.15.36:0/3294430564 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba4009470 con 0x7f9bac0b4f00 2022-06-17T12:55:32.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.335+0000 7f9babfff700 1 --2- 172.21.15.36:0/3294430564 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9bac0b4f00 0x7f9bac155aa0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7f9b9c0030b0 tx=0x7f9b9c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:32.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.335+0000 7f9ba9ffb700 1 -- 172.21.15.36:0/3294430564 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b9c011780 con 0x7f9bac0b4f00 2022-06-17T12:55:32.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.335+0000 7f9ba9ffb700 1 -- 172.21.15.36:0/3294430564 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9c0118e0 con 0x7f9bac0b4f00 2022-06-17T12:55:32.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.335+0000 7f9bb1d0d700 1 -- 172.21.15.36:0/3294430564 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bac0aff40 con 0x7f9bac0b4f00 2022-06-17T12:55:32.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.335+0000 7f9ba9ffb700 1 -- 172.21.15.36:0/3294430564 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b9c011c40 con 0x7f9bac0b4f00 2022-06-17T12:55:32.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.335+0000 7f9bb1d0d700 1 -- 172.21.15.36:0/3294430564 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bac156620 con 0x7f9bac0b4f00 2022-06-17T12:55:32.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.335+0000 7f9bb1d0d700 1 -- 172.21.15.36:0/3294430564 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bac13f550 con 0x7f9bac0b4f00 2022-06-17T12:55:32.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.335+0000 7f9ba9ffb700 1 -- 172.21.15.36:0/3294430564 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f9b9c012420 con 0x7f9bac0b4f00 2022-06-17T12:55:32.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.335+0000 7f9ba9ffb700 1 --2- 172.21.15.36:0/3294430564 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9b90053a60 0x7f9b90055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:32.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.335+0000 7f9ba9ffb700 1 -- 172.21.15.36:0/3294430564 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9b9c06bbe0 con 0x7f9bac0b4f00 2022-06-17T12:55:32.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.335+0000 7f9bb0d0b700 1 --2- 172.21.15.36:0/3294430564 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9b90053a60 0x7f9b90055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:32.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.339+0000 7f9bb0d0b700 1 --2- 172.21.15.36:0/3294430564 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9b90053a60 0x7f9b90055f20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f9b9400b000 tx=0x7f9b94009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:32.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.343+0000 7f9ba9ffb700 1 -- 172.21.15.36:0/3294430564 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9b9c042980 con 0x7f9bac0b4f00 2022-06-17T12:55:32.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.467+0000 7f9bb1d0d700 1 -- 172.21.15.36:0/3294430564 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9bac156620 con 0x7f9bac0b4f00 2022-06-17T12:55:32.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.471+0000 7f9ba9ffb700 1 -- 172.21.15.36:0/3294430564 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+10 (secure 0 0 0) 0x7f9bac156620 con 0x7f9bac0b4f00 2022-06-17T12:55:32.459 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:55:32.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.471+0000 7f9bb1d0d700 1 -- 172.21.15.36:0/3294430564 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9b90053a60 msgr2=0x7f9b90055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:32.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.471+0000 7f9bb1d0d700 1 --2- 172.21.15.36:0/3294430564 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9b90053a60 0x7f9b90055f20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f9b9400b000 tx=0x7f9b94009040).stop 2022-06-17T12:55:32.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.471+0000 7f9bb1d0d700 1 -- 172.21.15.36:0/3294430564 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9bac0b4f00 msgr2=0x7f9bac155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:32.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.471+0000 7f9bb1d0d700 1 --2- 172.21.15.36:0/3294430564 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9bac0b4f00 0x7f9bac155aa0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7f9b9c0030b0 tx=0x7f9b9c008040).stop 2022-06-17T12:55:32.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.471+0000 7f9bb1d0d700 1 -- 172.21.15.36:0/3294430564 shutdown_connections 2022-06-17T12:55:32.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.471+0000 7f9bb1d0d700 1 --2- 172.21.15.36:0/3294430564 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9b90053a60 0x7f9b90055f20 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.471+0000 7f9bb1d0d700 1 --2- 172.21.15.36:0/3294430564 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9bac0b9860 0x7f9bac155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.471+0000 7f9bb1d0d700 1 --2- 172.21.15.36:0/3294430564 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9bac0b4f00 0x7f9bac155aa0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.471+0000 7f9bb1d0d700 1 --2- 172.21.15.36:0/3294430564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bac0b45c0 0x7f9bac0afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.471+0000 7f9bb1d0d700 1 -- 172.21.15.36:0/3294430564 >> 172.21.15.36:0/3294430564 conn(0x7f9bac05c550 msgr2=0x7f9bac0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:32.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.471+0000 7f9bb1d0d700 1 -- 172.21.15.36:0/3294430564 shutdown_connections 2022-06-17T12:55:32.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.471+0000 7f9bb1d0d700 1 -- 172.21.15.36:0/3294430564 wait complete. 2022-06-17T12:55:32.473 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:55:32.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.607+0000 7f724ea10700 1 Processor -- start 2022-06-17T12:55:32.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.607+0000 7f724ea10700 1 -- start start 2022-06-17T12:55:32.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.607+0000 7f724ea10700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f72480b6860 0x7f7248146d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:32.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.607+0000 7f724ea10700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f72480b39a0 0x7f7248143860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:32.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.607+0000 7f724ea10700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72480b2b70 con 0x7f72480b18e0 2022-06-17T12:55:32.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.607+0000 7f724ea10700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72480b6c40 con 0x7f72480b39a0 2022-06-17T12:55:32.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.607+0000 7f724ea10700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7248142d30 con 0x7f72480b6860 2022-06-17T12:55:32.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.607+0000 7f724e20f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f72480b39a0 0x7f7248143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:32.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.607+0000 7f724e20f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f72480b39a0 0x7f7248143860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38692/0 (socket says 172.21.15.36:38692) 2022-06-17T12:55:32.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.607+0000 7f724e20f700 1 -- 172.21.15.36:0/1660060507 learned_addr learned my addr 172.21.15.36:0/1660060507 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:32.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.607+0000 7f724da0e700 1 --2- 172.21.15.36:0/1660060507 >> v2:172.21.15.70:3300/0 conn(0x7f72480b6860 0x7f7248146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:32.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.611+0000 7f724e20f700 1 -- 172.21.15.36:0/1660060507 >> v2:172.21.15.70:3300/0 conn(0x7f72480b6860 msgr2=0x7f7248146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:32.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.611+0000 7f724e20f700 1 --2- 172.21.15.36:0/1660060507 >> v2:172.21.15.70:3300/0 conn(0x7f72480b6860 0x7f7248146d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.611+0000 7f724e20f700 1 -- 172.21.15.36:0/1660060507 >> v1:172.21.15.36:6789/0 conn(0x7f72480b18e0 legacy=0x7f7248143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:32.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.611+0000 7f724ca0c700 1 -- 172.21.15.36:0/1660060507 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4914887 0 0) 0x7f72480b2b70 con 0x7f72480b18e0 2022-06-17T12:55:32.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.611+0000 7f724e20f700 1 -- 172.21.15.36:0/1660060507 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72480b0d30 con 0x7f72480b39a0 2022-06-17T12:55:32.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.611+0000 7f724da0e700 1 --2- 172.21.15.36:0/1660060507 >> v2:172.21.15.70:3300/0 conn(0x7f72480b6860 0x7f7248146d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:32.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.611+0000 7f724e20f700 1 --2- 172.21.15.36:0/1660060507 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f72480b39a0 0x7f7248143860 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0x7f72400032f0 tx=0x7f7240009290).ready entity=mon.0 client_cookie=17ae3e80825a2c96 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:32.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.611+0000 7f724ca0c700 1 -- 172.21.15.36:0/1660060507 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f724000b870 con 0x7f72480b39a0 2022-06-17T12:55:32.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.611+0000 7f724ca0c700 1 -- 172.21.15.36:0/1660060507 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724000b9d0 con 0x7f72480b39a0 2022-06-17T12:55:32.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.611+0000 7f724ca0c700 1 -- 172.21.15.36:0/1660060507 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f724000bb40 con 0x7f72480b39a0 2022-06-17T12:55:32.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.611+0000 7f724ea10700 1 -- 172.21.15.36:0/1660060507 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f72480b39a0 msgr2=0x7f7248143860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:32.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.611+0000 7f724ea10700 1 --2- 172.21.15.36:0/1660060507 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f72480b39a0 0x7f7248143860 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0x7f72400032f0 tx=0x7f7240009290).stop 2022-06-17T12:55:32.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.611+0000 7f724ea10700 1 -- 172.21.15.36:0/1660060507 shutdown_connections 2022-06-17T12:55:32.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.611+0000 7f724ea10700 1 --2- 172.21.15.36:0/1660060507 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f72480b39a0 0x7f7248143860 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.611+0000 7f724ea10700 1 --2- 172.21.15.36:0/1660060507 >> v2:172.21.15.70:3300/0 conn(0x7f72480b6860 0x7f7248146d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.611+0000 7f724ea10700 1 -- 172.21.15.36:0/1660060507 >> 172.21.15.36:0/1660060507 conn(0x7f72480614b0 msgr2=0x7f7248061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:32.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.611+0000 7f724ea10700 1 -- 172.21.15.36:0/1660060507 shutdown_connections 2022-06-17T12:55:32.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.611+0000 7f724ea10700 1 -- 172.21.15.36:0/1660060507 wait complete. 2022-06-17T12:55:32.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.615+0000 7f724ea10700 1 Processor -- start 2022-06-17T12:55:32.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.615+0000 7f724ea10700 1 -- start start 2022-06-17T12:55:32.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.615+0000 7f724ea10700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f72480b18e0 0x7f72481413e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:32.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.615+0000 7f724ea10700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72480b39a0 0x7f72480639e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:32.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.615+0000 7f724ea10700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f72480b6860 0x7f7248063ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:32.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.615+0000 7f724ea10700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7248140a50 con 0x7f72480b18e0 2022-06-17T12:55:32.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.615+0000 7f724ea10700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72481402d0 con 0x7f72480b39a0 2022-06-17T12:55:32.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.615+0000 7f724ea10700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7248140c80 con 0x7f72480b6860 2022-06-17T12:55:32.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.615+0000 7f724da0e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f72480b18e0 0x7f72481413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:32.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.615+0000 7f724da0e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f72480b18e0 0x7f72481413e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38696/0 (socket says 172.21.15.36:38696) 2022-06-17T12:55:32.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.615+0000 7f724da0e700 1 -- 172.21.15.36:0/3600279326 learned_addr learned my addr 172.21.15.36:0/3600279326 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:32.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.615+0000 7f724d20d700 1 --2- 172.21.15.36:0/3600279326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72480b39a0 0x7f72480639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:32.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.615+0000 7f724e20f700 1 --2- 172.21.15.36:0/3600279326 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f72480b6860 0x7f7248063ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:32.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.615+0000 7f724da0e700 1 -- 172.21.15.36:0/3600279326 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f72480b6860 msgr2=0x7f7248063ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:32.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.615+0000 7f724da0e700 1 --2- 172.21.15.36:0/3600279326 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f72480b6860 0x7f7248063ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.615+0000 7f724da0e700 1 -- 172.21.15.36:0/3600279326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72480b39a0 msgr2=0x7f72480639e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:32.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.615+0000 7f724da0e700 1 --2- 172.21.15.36:0/3600279326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72480b39a0 0x7f72480639e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.615+0000 7f724da0e700 1 -- 172.21.15.36:0/3600279326 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7240009530 con 0x7f72480b18e0 2022-06-17T12:55:32.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.615+0000 7f724e20f700 1 --2- 172.21.15.36:0/3600279326 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f72480b6860 0x7f7248063ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:32.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.615+0000 7f724da0e700 1 --2- 172.21.15.36:0/3600279326 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f72480b18e0 0x7f72481413e0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7f723c006940 tx=0x7f723c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:32.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.615+0000 7f723affd700 1 -- 172.21.15.36:0/3600279326 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f723c011850 con 0x7f72480b18e0 2022-06-17T12:55:32.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.615+0000 7f723affd700 1 -- 172.21.15.36:0/3600279326 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723c0119b0 con 0x7f72480b18e0 2022-06-17T12:55:32.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.615+0000 7f724ea10700 1 -- 172.21.15.36:0/3600279326 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72481418d0 con 0x7f72480b18e0 2022-06-17T12:55:32.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.615+0000 7f723affd700 1 -- 172.21.15.36:0/3600279326 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f723c011d10 con 0x7f72480b18e0 2022-06-17T12:55:32.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.619+0000 7f724ea10700 1 -- 172.21.15.36:0/3600279326 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72480b0390 con 0x7f72480b18e0 2022-06-17T12:55:32.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.619+0000 7f724ea10700 1 -- 172.21.15.36:0/3600279326 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7248057410 con 0x7f72480b18e0 2022-06-17T12:55:32.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.619+0000 7f723affd700 1 -- 172.21.15.36:0/3600279326 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f723c012420 con 0x7f72480b18e0 2022-06-17T12:55:32.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.619+0000 7f723affd700 1 --2- 172.21.15.36:0/3600279326 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f72280539b0 0x7f7228055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:32.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.619+0000 7f723affd700 1 -- 172.21.15.36:0/3600279326 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f723c06be90 con 0x7f72480b18e0 2022-06-17T12:55:32.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.619+0000 7f724d20d700 1 --2- 172.21.15.36:0/3600279326 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f72280539b0 0x7f7228055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:32.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.619+0000 7f724d20d700 1 --2- 172.21.15.36:0/3600279326 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f72280539b0 0x7f7228055e70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f724400d000 tx=0x7f7244009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:32.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.627+0000 7f723affd700 1 -- 172.21.15.36:0/3600279326 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f723c042bb0 con 0x7f72480b18e0 2022-06-17T12:55:32.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.755+0000 7f724ea10700 1 -- 172.21.15.36:0/3600279326 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7248061890 con 0x7f72480b18e0 2022-06-17T12:55:32.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.755+0000 7f723affd700 1 -- 172.21.15.36:0/3600279326 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+10 (secure 0 0 0) 0x7f723c0461d0 con 0x7f72480b18e0 2022-06-17T12:55:32.746 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:55:32.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.755+0000 7f724ea10700 1 -- 172.21.15.36:0/3600279326 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f72280539b0 msgr2=0x7f7228055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:32.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.755+0000 7f724ea10700 1 --2- 172.21.15.36:0/3600279326 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f72280539b0 0x7f7228055e70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f724400d000 tx=0x7f7244009040).stop 2022-06-17T12:55:32.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.755+0000 7f724ea10700 1 -- 172.21.15.36:0/3600279326 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f72480b18e0 msgr2=0x7f72481413e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:32.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.755+0000 7f724ea10700 1 --2- 172.21.15.36:0/3600279326 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f72480b18e0 0x7f72481413e0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7f723c006940 tx=0x7f723c008040).stop 2022-06-17T12:55:32.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.755+0000 7f724ea10700 1 -- 172.21.15.36:0/3600279326 shutdown_connections 2022-06-17T12:55:32.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.755+0000 7f724ea10700 1 --2- 172.21.15.36:0/3600279326 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f72280539b0 0x7f7228055e70 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.755+0000 7f724ea10700 1 --2- 172.21.15.36:0/3600279326 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f72480b6860 0x7f7248063ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.755+0000 7f724ea10700 1 --2- 172.21.15.36:0/3600279326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72480b39a0 0x7f72480639e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.755+0000 7f724ea10700 1 --2- 172.21.15.36:0/3600279326 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f72480b18e0 0x7f72481413e0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.755+0000 7f724ea10700 1 -- 172.21.15.36:0/3600279326 >> 172.21.15.36:0/3600279326 conn(0x7f72480614b0 msgr2=0x7f724813f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:32.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.759+0000 7f724ea10700 1 -- 172.21.15.36:0/3600279326 shutdown_connections 2022-06-17T12:55:32.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.759+0000 7f724ea10700 1 -- 172.21.15.36:0/3600279326 wait complete. 2022-06-17T12:55:32.757 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:55:32.757 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:55:32.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.875+0000 7fda53fff700 1 Processor -- start 2022-06-17T12:55:32.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.875+0000 7fda53fff700 1 -- start start 2022-06-17T12:55:32.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda53fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fda54063730 0x7fda54065e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:32.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda53fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda540b1930 0x7fda541431f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:32.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda53fff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda54063b10 con 0x7fda540b2900 2022-06-17T12:55:32.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda53fff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda54065430 con 0x7fda54063730 2022-06-17T12:55:32.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda53fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda540656d0 con 0x7fda540b1930 2022-06-17T12:55:32.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda52ffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fda54063730 0x7fda54065e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:32.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda537fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fda540b2900 0x7fda540b6380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56236/0 (socket says 172.21.15.36:56236) 2022-06-17T12:55:32.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda537fe700 1 -- 172.21.15.36:0/1714041542 learned_addr learned my addr 172.21.15.36:0/1714041542 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:32.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda4bfff700 1 --2- 172.21.15.36:0/1714041542 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda540b1930 0x7fda541431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:32.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda527fc700 1 -- 172.21.15.36:0/1714041542 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1023362884 0 0) 0x7fda54063b10 con 0x7fda540b2900 2022-06-17T12:55:32.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda527fc700 1 -- 172.21.15.36:0/1714041542 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda34003260 con 0x7fda540b2900 2022-06-17T12:55:32.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda52ffd700 1 -- 172.21.15.36:0/1714041542 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda540b1930 msgr2=0x7fda541431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:32.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda52ffd700 1 --2- 172.21.15.36:0/1714041542 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda540b1930 0x7fda541431f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda52ffd700 1 -- 172.21.15.36:0/1714041542 >> v1:172.21.15.36:6789/0 conn(0x7fda540b2900 legacy=0x7fda540b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:32.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda52ffd700 1 -- 172.21.15.36:0/1714041542 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda54065970 con 0x7fda54063730 2022-06-17T12:55:32.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda52ffd700 1 --2- 172.21.15.36:0/1714041542 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda54063730 0x7fda54065e10 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0x7fda54063ca0 tx=0x7fda44006df0).ready entity=mon.0 client_cookie=d3185564f8bec737 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:32.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda527fc700 1 -- 172.21.15.36:0/1714041542 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda4400d570 con 0x7fda54063730 2022-06-17T12:55:32.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda527fc700 1 -- 172.21.15.36:0/1714041542 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda44000e90 con 0x7fda54063730 2022-06-17T12:55:32.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda527fc700 1 -- 172.21.15.36:0/1714041542 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda4400d8c0 con 0x7fda54063730 2022-06-17T12:55:32.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda53fff700 1 -- 172.21.15.36:0/1714041542 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda54063730 msgr2=0x7fda54065e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:32.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda53fff700 1 --2- 172.21.15.36:0/1714041542 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda54063730 0x7fda54065e10 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0x7fda54063ca0 tx=0x7fda44006df0).stop 2022-06-17T12:55:32.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda53fff700 1 -- 172.21.15.36:0/1714041542 shutdown_connections 2022-06-17T12:55:32.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda53fff700 1 --2- 172.21.15.36:0/1714041542 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda540b1930 0x7fda541431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda53fff700 1 --2- 172.21.15.36:0/1714041542 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda54063730 0x7fda54065e10 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda53fff700 1 -- 172.21.15.36:0/1714041542 >> 172.21.15.36:0/1714041542 conn(0x7fda540614b0 msgr2=0x7fda540618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:32.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda53fff700 1 -- 172.21.15.36:0/1714041542 shutdown_connections 2022-06-17T12:55:32.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.879+0000 7fda53fff700 1 -- 172.21.15.36:0/1714041542 wait complete. 2022-06-17T12:55:32.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda53fff700 1 Processor -- start 2022-06-17T12:55:32.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda53fff700 1 -- start start 2022-06-17T12:55:32.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda53fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda54063730 0x7fda5413f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:32.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda53fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda540b1930 0x7fda54142e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:32.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda53fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda540b2900 0x7fda540b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:32.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda53fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda54143380 con 0x7fda540b2900 2022-06-17T12:55:32.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda53fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda54142330 con 0x7fda540b1930 2022-06-17T12:55:32.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda53fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda541425d0 con 0x7fda54063730 2022-06-17T12:55:32.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda537fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda540b2900 0x7fda540b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:32.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda537fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda540b2900 0x7fda540b0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38714/0 (socket says 172.21.15.36:38714) 2022-06-17T12:55:32.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda537fe700 1 -- 172.21.15.36:0/2028150563 learned_addr learned my addr 172.21.15.36:0/2028150563 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:32.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda52ffd700 1 --2- 172.21.15.36:0/2028150563 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda54063730 0x7fda5413f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:32.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda4bfff700 1 --2- 172.21.15.36:0/2028150563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda540b1930 0x7fda54142e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:32.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda537fe700 1 -- 172.21.15.36:0/2028150563 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda54063730 msgr2=0x7fda5413f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:32.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda537fe700 1 --2- 172.21.15.36:0/2028150563 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda54063730 0x7fda5413f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda537fe700 1 -- 172.21.15.36:0/2028150563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda540b1930 msgr2=0x7fda54142e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:32.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda537fe700 1 --2- 172.21.15.36:0/2028150563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda540b1930 0x7fda54142e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:32.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda537fe700 1 -- 172.21.15.36:0/2028150563 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4400b180 con 0x7fda540b2900 2022-06-17T12:55:32.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda4bfff700 1 --2- 172.21.15.36:0/2028150563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda540b1930 0x7fda54142e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:32.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda537fe700 1 --2- 172.21.15.36:0/2028150563 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda540b2900 0x7fda540b0050 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7fda4c001490 tx=0x7fda4c007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:32.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda50ff9700 1 -- 172.21.15.36:0/2028150563 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda4c005ba0 con 0x7fda540b2900 2022-06-17T12:55:32.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda50ff9700 1 -- 172.21.15.36:0/2028150563 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4c005d00 con 0x7fda540b2900 2022-06-17T12:55:32.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda53fff700 1 -- 172.21.15.36:0/2028150563 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda540ae580 con 0x7fda540b2900 2022-06-17T12:55:32.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda50ff9700 1 -- 172.21.15.36:0/2028150563 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda4c0072c0 con 0x7fda540b2900 2022-06-17T12:55:32.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.883+0000 7fda53fff700 1 -- 172.21.15.36:0/2028150563 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda540b0540 con 0x7fda540b2900 2022-06-17T12:55:32.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.887+0000 7fda53fff700 1 -- 172.21.15.36:0/2028150563 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda54142950 con 0x7fda540b2900 2022-06-17T12:55:32.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.887+0000 7fda50ff9700 1 -- 172.21.15.36:0/2028150563 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fda4c01a060 con 0x7fda540b2900 2022-06-17T12:55:32.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.887+0000 7fda50ff9700 1 --2- 172.21.15.36:0/2028150563 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fda38053920 0x7fda38055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:32.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.887+0000 7fda50ff9700 1 -- 172.21.15.36:0/2028150563 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fda4c06b9f0 con 0x7fda540b2900 2022-06-17T12:55:32.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.887+0000 7fda52ffd700 1 --2- 172.21.15.36:0/2028150563 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fda38053920 0x7fda38055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:32.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.887+0000 7fda52ffd700 1 --2- 172.21.15.36:0/2028150563 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fda38053920 0x7fda38055de0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7fda4401a000 tx=0x7fda44014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:32.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:32.891+0000 7fda50ff9700 1 -- 172.21.15.36:0/2028150563 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fda4c06de60 con 0x7fda540b2900 2022-06-17T12:55:33.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:33.051+0000 7fda53fff700 1 -- 172.21.15.36:0/2028150563 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda540b3b30 con 0x7fda540b2900 2022-06-17T12:55:33.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:33.051+0000 7fda50ff9700 1 -- 172.21.15.36:0/2028150563 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+2142 (secure 0 0 0) 0x7fda4c018070 con 0x7fda540b2900 2022-06-17T12:55:33.042 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:33.042 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":60,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1161585109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1161585109},{"type":"v1","addr":"172.21.15.70:6833","nonce":1161585109}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":57}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:55:24.825976+0000","modified":"2022-06-17T12:55:26.574711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7531":{"gid":7531,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/1342585509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":1342585509},{"type":"v1","addr":"172.21.15.36:6835","nonce":1342585509}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2022-06-17T12:55:33.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:33.055+0000 7fda53fff700 1 -- 172.21.15.36:0/2028150563 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fda38053920 msgr2=0x7fda38055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:33.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:33.055+0000 7fda53fff700 1 --2- 172.21.15.36:0/2028150563 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fda38053920 0x7fda38055de0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7fda4401a000 tx=0x7fda44014000).stop 2022-06-17T12:55:33.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:33.055+0000 7fda53fff700 1 -- 172.21.15.36:0/2028150563 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda540b2900 msgr2=0x7fda540b0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:33.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:33.055+0000 7fda53fff700 1 --2- 172.21.15.36:0/2028150563 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda540b2900 0x7fda540b0050 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7fda4c001490 tx=0x7fda4c007e00).stop 2022-06-17T12:55:33.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:33.055+0000 7fda53fff700 1 -- 172.21.15.36:0/2028150563 shutdown_connections 2022-06-17T12:55:33.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:33.055+0000 7fda53fff700 1 --2- 172.21.15.36:0/2028150563 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fda38053920 0x7fda38055de0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:33.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:33.055+0000 7fda53fff700 1 --2- 172.21.15.36:0/2028150563 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda540b2900 0x7fda540b0050 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:33.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:33.055+0000 7fda53fff700 1 --2- 172.21.15.36:0/2028150563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda540b1930 0x7fda54142e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:33.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:33.055+0000 7fda53fff700 1 --2- 172.21.15.36:0/2028150563 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda54063730 0x7fda5413f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:33.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:33.055+0000 7fda53fff700 1 -- 172.21.15.36:0/2028150563 >> 172.21.15.36:0/2028150563 conn(0x7fda540614b0 msgr2=0x7fda54061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:33.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:33.055+0000 7fda53fff700 1 -- 172.21.15.36:0/2028150563 shutdown_connections 2022-06-17T12:55:33.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:33.055+0000 7fda53fff700 1 -- 172.21.15.36:0/2028150563 wait complete. 2022-06-17T12:55:33.048 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 60 2022-06-17T12:55:33.258 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:55:24.825976+0000', 'modified': '2022-06-17T12:55:26.574711+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7531}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7531': {'gid': 7531, 'name': 'a', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.36:6835/1342585509', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 1342585509}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 1342585509}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:55:33.259 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:55:33.260 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:55:33.382 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":7531,"want_state":"up:active","state":"up:active","rank_uptime":8.5555207410000005,"mdsmap_epoch":60,"osdmap_epoch":113,"osdmap_epoch_barrier":113,"uptime":12.924741077} 2022-06-17T12:55:33.383 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 7531, "mdsmap_epoch": 60, "osdmap_epoch": 113, "osdmap_epoch_barrier": 113, "rank_uptime": 8.555520741, "state": "up:active", "uptime": 12.924741077, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:55:33.384 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:55:33.384 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-06-17T12:55:33.385 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi136.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-06-17T12:55:33.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:33.391 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:55:33.402 DEBUG:teuthology.orchestra.run.smithi136:> sudo modprobe fuse 2022-06-17T12:55:33.427 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:55:33.428 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:55:33.485 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:55:33.486 INFO:teuthology.orchestra.run.smithi136.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:55:33.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:33.487 DEBUG:teuthology.orchestra.run.smithi136:> ls /sys/fs/fuse/connections 2022-06-17T12:55:33.499 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:55:33.500 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:55:33.504 DEBUG:teuthology.orchestra.run.smithi136:> sudo modprobe fuse 2022-06-17T12:55:33.521 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:55:33.522 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:55:33.580 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:2022-06-17T12:55:33.589+0000 7f87a63a3080 -1 init, newargv = 0x55e0e307f2e0 newargc=9ceph-fuse[22225]: starting ceph client 2022-06-17T12:55:33.580 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr: 2022-06-17T12:55:33.581 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:55:33.582 INFO:teuthology.orchestra.run.smithi136.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:55:33.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:33.583 DEBUG:teuthology.orchestra.run.smithi136:> ls /sys/fs/fuse/connections 2022-06-17T12:55:33.590 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:ceph-fuse[22225]: starting fuse 2022-06-17T12:55:34.691 DEBUG:teuthology.orchestra.run.smithi136:> sudo modprobe fuse 2022-06-17T12:55:34.711 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:55:34.712 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:55:34.771 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:55:34.772 INFO:teuthology.orchestra.run.smithi136.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:55:34.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:34.774 DEBUG:teuthology.orchestra.run.smithi136:> ls /sys/fs/fuse/connections 2022-06-17T12:55:34.780 INFO:teuthology.orchestra.run.smithi136.stdout:52 2022-06-17T12:55:34.781 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:55:34.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:34.782 DEBUG:teuthology.orchestra.run.smithi136:> sudo python3 -c ' 2022-06-17T12:55:34.783 DEBUG:teuthology.orchestra.run.smithi136:> import glob 2022-06-17T12:55:34.783 DEBUG:teuthology.orchestra.run.smithi136:> import re 2022-06-17T12:55:34.784 DEBUG:teuthology.orchestra.run.smithi136:> import os 2022-06-17T12:55:34.784 DEBUG:teuthology.orchestra.run.smithi136:> import subprocess 2022-06-17T12:55:34.785 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:55:34.785 DEBUG:teuthology.orchestra.run.smithi136:> def find_socket(client_name): 2022-06-17T12:55:34.786 DEBUG:teuthology.orchestra.run.smithi136:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-06-17T12:55:34.786 DEBUG:teuthology.orchestra.run.smithi136:> files = glob.glob(asok_path) 2022-06-17T12:55:34.787 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:55:34.787 DEBUG:teuthology.orchestra.run.smithi136:> # Given a non-glob path, it better be there 2022-06-17T12:55:34.787 DEBUG:teuthology.orchestra.run.smithi136:> if "*" not in asok_path: 2022-06-17T12:55:34.788 DEBUG:teuthology.orchestra.run.smithi136:> assert(len(files) == 1) 2022-06-17T12:55:34.788 DEBUG:teuthology.orchestra.run.smithi136:> return files[0] 2022-06-17T12:55:34.789 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:55:34.789 DEBUG:teuthology.orchestra.run.smithi136:> for f in files: 2022-06-17T12:55:34.790 DEBUG:teuthology.orchestra.run.smithi136:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:55:34.790 DEBUG:teuthology.orchestra.run.smithi136:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:55:34.791 DEBUG:teuthology.orchestra.run.smithi136:> return f 2022-06-17T12:55:34.791 DEBUG:teuthology.orchestra.run.smithi136:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:55:34.792 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:55:34.792 DEBUG:teuthology.orchestra.run.smithi136:> print(find_socket("client.1")) 2022-06-17T12:55:34.793 DEBUG:teuthology.orchestra.run.smithi136:> ' 2022-06-17T12:55:34.873 INFO:teuthology.orchestra.run.smithi136.stdout:/var/run/ceph/ceph-client.1.22225.asok 2022-06-17T12:55:34.876 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22225.asok 2022-06-17T12:55:34.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:34.877 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22225.asok status 2022-06-17T12:55:34.965 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2022-06-17T12:55:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: "metadata": { 2022-06-17T12:55:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:55:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:55:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: "entity_id": "1", 2022-06-17T12:55:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: "hostname": "smithi136", 2022-06-17T12:55:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-06-17T12:55:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: "pid": "22225", 2022-06-17T12:55:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: "root": "/" 2022-06-17T12:55:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:55:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: "dentry_count": 0, 2022-06-17T12:55:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: "dentry_pinned_count": 0, 2022-06-17T12:55:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: "id": 7744, 2022-06-17T12:55:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: "inst": { 2022-06-17T12:55:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: "name": { 2022-06-17T12:55:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "client", 2022-06-17T12:55:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: "num": 7744 2022-06-17T12:55:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:55:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": { 2022-06-17T12:55:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "v1", 2022-06-17T12:55:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": "172.21.15.136:0", 2022-06-17T12:55:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: "nonce": 2576442562 2022-06-17T12:55:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: } 2022-06-17T12:55:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:55:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": { 2022-06-17T12:55:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "v1", 2022-06-17T12:55:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": "172.21.15.136:0", 2022-06-17T12:55:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: "nonce": 2576442562 2022-06-17T12:55:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:55:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: "inst_str": "client.7744 172.21.15.136:0/2576442562", 2022-06-17T12:55:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: "addr_str": "172.21.15.136:0/2576442562", 2022-06-17T12:55:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: "inode_count": 1, 2022-06-17T12:55:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_epoch": 60, 2022-06-17T12:55:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_epoch": 113, 2022-06-17T12:55:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:55:34.977 INFO:teuthology.orchestra.run.smithi136.stdout: "blacklisted": false, 2022-06-17T12:55:34.977 INFO:teuthology.orchestra.run.smithi136.stdout: "fs_name": "cephfs" 2022-06-17T12:55:34.977 INFO:teuthology.orchestra.run.smithi136.stdout:} 2022-06-17T12:55:34.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:34.979 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:55:34.980 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:55:34.988 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:55:34.989 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:55:34.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:34.990 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:55:35.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:35.053 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:55:35.053 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:55:35.062 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:55:35.063 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:55:35.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:35.064 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:55:35.064 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:55:35.117 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:55:35.119 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:55:35.123 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2022-06-17T12:55:35.124 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:55:35.124 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:55:35.142 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22781]: fuse finished with error 0 and tester_r 0 2022-06-17T12:55:35.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:35.152 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:55:35.153 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:55:35.194 INFO:teuthology.orchestra.run.smithi107.stdout:ext2/ext3 2022-06-17T12:55:35.197 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:55:35.198 INFO:teuthology.orchestra.run:waiting for 900 2022-06-17T12:55:35.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:55:35.276 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:55:35.285 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:35.287 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:55:35.287 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:55:35.296 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:55:35.297 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:55:35.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:35.298 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:55:35.299 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:55:35.348 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:55:35.350 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:55:35.350 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi136.front.sepia.ceph.com... 2022-06-17T12:55:35.351 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:55:35.351 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:55:35.368 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:ceph-fuse[22225]: fuse finished with error 0 and tester_r 0 2022-06-17T12:55:35.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:35.372 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:55:35.372 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:55:35.380 INFO:teuthology.orchestra.run.smithi136.stdout:ext2/ext3 2022-06-17T12:55:35.382 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:55:35.383 INFO:teuthology.orchestra.run:waiting for 900 2022-06-17T12:55:41.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:55:41.486 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:55:41.496 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-17T12:55:41.496 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T12:55:41.497 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-17T12:55:41.497 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T12:55:41.513 INFO:tasks.ceph.mds.a:Stopped 2022-06-17T12:55:41.514 DEBUG:teuthology.parallel:result is None 2022-06-17T12:55:41.516 INFO:tasks.ceph.mds.b:Stopped 2022-06-17T12:55:41.517 DEBUG:teuthology.parallel:result is None 2022-06-17T12:55:41.517 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2022-06-17T12:55:41.518 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2022-06-17T12:55:41.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffabc8f1700 1 Processor -- start 2022-06-17T12:55:41.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7f9c32f13700 1 Processor -- start 2022-06-17T12:55:41.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffabc8f1700 1 -- start start 2022-06-17T12:55:41.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7f9c32f13700 1 -- start start 2022-06-17T12:55:41.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffabc8f1700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ffab8060c40 0x7ffab805fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:41.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffabc8f1700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffab8060370 0x7ffab813c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:41.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffabc8f1700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab8146ff0 con 0x7ffab80b6780 2022-06-17T12:55:41.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffabc8f1700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab8147420 con 0x7ffab8060c40 2022-06-17T12:55:41.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffabc8f1700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab80b6b60 con 0x7ffab8060370 2022-06-17T12:55:41.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffab5d9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ffab8060c40 0x7ffab805fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:41.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffab5d9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ffab8060c40 0x7ffab805fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38720/0 (socket says 172.21.15.36:38720) 2022-06-17T12:55:41.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffab5d9b700 1 -- 172.21.15.36:0/1751224060 learned_addr learned my addr 172.21.15.36:0/1751224060 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:41.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7f9c32f13700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9c2c0acf00 0x7f9c2c0b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:41.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffab6d9d700 1 --2- 172.21.15.36:0/1751224060 >> v2:172.21.15.70:3300/0 conn(0x7ffab8060370 0x7ffab813c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:41.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffab5d9b700 1 -- 172.21.15.36:0/1751224060 >> v2:172.21.15.70:3300/0 conn(0x7ffab8060370 msgr2=0x7ffab813c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:41.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffab5d9b700 1 --2- 172.21.15.36:0/1751224060 >> v2:172.21.15.70:3300/0 conn(0x7ffab8060370 0x7ffab813c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:41.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffab5d9b700 1 -- 172.21.15.36:0/1751224060 >> v1:172.21.15.36:6789/0 conn(0x7ffab80b6780 legacy=0x7ffab805f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:41.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffab5d9b700 1 -- 172.21.15.36:0/1751224060 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab80afd40 con 0x7ffab8060c40 2022-06-17T12:55:41.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffab559a700 1 -- 172.21.15.36:0/1751224060 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769278875 0 0) 0x7ffab8146ff0 con 0x7ffab80b6780 2022-06-17T12:55:41.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffab6d9d700 1 --2- 172.21.15.36:0/1751224060 >> v2:172.21.15.70:3300/0 conn(0x7ffab8060370 0x7ffab813c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:55:41.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7f9c32f13700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c2c0abbb0 0x7f9c2c0ad2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:41.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7f9c32f13700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c2c0aa060 con 0x7f9c2c0b1d70 2022-06-17T12:55:41.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7f9c32f13700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c2c0b64f0 con 0x7f9c2c0acf00 2022-06-17T12:55:41.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7f9c32f13700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c0b66e0 con 0x7f9c2c0abbb0 2022-06-17T12:55:41.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7f9c30caf700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9c2c0acf00 0x7f9c2c0b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:41.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffab5d9b700 1 --2- 172.21.15.36:0/1751224060 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffab8060c40 0x7ffab805fe80 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7ffaa0002f70 tx=0x7ffaa0006eb0).ready entity=mon.0 client_cookie=45c0079ded695bd1 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:41.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7f9c30caf700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9c2c0acf00 0x7f9c2c0b0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38724/0 (socket says 172.21.15.36:38724) 2022-06-17T12:55:41.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7f9c30caf700 1 -- 172.21.15.36:0/857185500 learned_addr learned my addr 172.21.15.36:0/857185500 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:41.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7f9c2bfff700 1 --2- 172.21.15.36:0/857185500 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c2c0abbb0 0x7f9c2c0ad2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:41.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffab559a700 1 -- 172.21.15.36:0/1751224060 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffaa000d610 con 0x7ffab8060c40 2022-06-17T12:55:41.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffab559a700 1 -- 172.21.15.36:0/1751224060 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa0000e90 con 0x7ffab8060c40 2022-06-17T12:55:41.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffab559a700 1 -- 172.21.15.36:0/1751224060 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffaa000d960 con 0x7ffab8060c40 2022-06-17T12:55:41.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7f9c30caf700 1 -- 172.21.15.36:0/857185500 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c2c0abbb0 msgr2=0x7f9c2c0ad2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:41.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7f9c30caf700 1 --2- 172.21.15.36:0/857185500 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c2c0abbb0 0x7f9c2c0ad2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:41.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7f9c30caf700 1 -- 172.21.15.36:0/857185500 >> v1:172.21.15.70:6789/0 conn(0x7f9c2c0b1d70 legacy=0x7f9c2c1407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:41.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7f9c30caf700 1 -- 172.21.15.36:0/857185500 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2c0a9ce0 con 0x7f9c2c0acf00 2022-06-17T12:55:41.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7f9c2bfff700 1 --2- 172.21.15.36:0/857185500 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c2c0abbb0 0x7f9c2c0ad2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:41.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7f9c30caf700 1 --2- 172.21.15.36:0/857185500 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9c2c0acf00 0x7f9c2c0b0f00 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0x7f9c1c002f70 tx=0x7f9c1c006eb0).ready entity=mon.0 client_cookie=43d36c4e4b8a84b7 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:41.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7f9c2b7fe700 1 -- 172.21.15.36:0/857185500 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c1c00d5c0 con 0x7f9c2c0acf00 2022-06-17T12:55:41.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7f9c2b7fe700 1 -- 172.21.15.36:0/857185500 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1c000e90 con 0x7f9c2c0acf00 2022-06-17T12:55:41.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7f9c2b7fe700 1 -- 172.21.15.36:0/857185500 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c1c00d950 con 0x7f9c2c0acf00 2022-06-17T12:55:41.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffabc8f1700 1 -- 172.21.15.36:0/1751224060 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffab8060c40 msgr2=0x7ffab805fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:41.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffabc8f1700 1 --2- 172.21.15.36:0/1751224060 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffab8060c40 0x7ffab805fe80 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7ffaa0002f70 tx=0x7ffaa0006eb0).stop 2022-06-17T12:55:41.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffabc8f1700 1 -- 172.21.15.36:0/1751224060 shutdown_connections 2022-06-17T12:55:41.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffabc8f1700 1 --2- 172.21.15.36:0/1751224060 >> v2:172.21.15.70:3300/0 conn(0x7ffab8060370 0x7ffab813c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:41.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffabc8f1700 1 --2- 172.21.15.36:0/1751224060 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffab8060c40 0x7ffab805fe80 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:41.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffabc8f1700 1 -- 172.21.15.36:0/1751224060 >> 172.21.15.36:0/1751224060 conn(0x7ffab8063890 msgr2=0x7ffab80b5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:41.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffabc8f1700 1 -- 172.21.15.36:0/1751224060 shutdown_connections 2022-06-17T12:55:41.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.635+0000 7ffabc8f1700 1 -- 172.21.15.36:0/1751224060 wait complete. 2022-06-17T12:55:41.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7f9c32f13700 1 -- 172.21.15.36:0/857185500 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9c2c0acf00 msgr2=0x7f9c2c0b0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:41.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7f9c32f13700 1 --2- 172.21.15.36:0/857185500 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9c2c0acf00 0x7f9c2c0b0f00 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0x7f9c1c002f70 tx=0x7f9c1c006eb0).stop 2022-06-17T12:55:41.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7ffabc8f1700 1 Processor -- start 2022-06-17T12:55:41.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7f9c32f13700 1 -- 172.21.15.36:0/857185500 shutdown_connections 2022-06-17T12:55:41.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7f9c32f13700 1 --2- 172.21.15.36:0/857185500 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c2c0abbb0 0x7f9c2c0ad2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:41.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7f9c32f13700 1 --2- 172.21.15.36:0/857185500 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9c2c0acf00 0x7f9c2c0b0f00 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:41.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7f9c32f13700 1 -- 172.21.15.36:0/857185500 >> 172.21.15.36:0/857185500 conn(0x7f9c2c0636d0 msgr2=0x7f9c2c063ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:41.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7ffabc8f1700 1 -- start start 2022-06-17T12:55:41.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7f9c32f13700 1 -- 172.21.15.36:0/857185500 shutdown_connections 2022-06-17T12:55:41.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7f9c32f13700 1 -- 172.21.15.36:0/857185500 wait complete. 2022-06-17T12:55:41.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7ffabc8f1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffab8060370 0x7ffab8143810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:41.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7ffabc8f1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffab8060c40 0x7ffab8145b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:41.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7ffabc8f1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffab80b6780 0x7ffab8066190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:41.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7ffabc8f1700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab8063ea0 con 0x7ffab8060370 2022-06-17T12:55:41.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7ffabc8f1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab8142a50 con 0x7ffab80b6780 2022-06-17T12:55:41.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7ffabc8f1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab8066680 con 0x7ffab8060c40 2022-06-17T12:55:41.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7ffab659c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffab8060370 0x7ffab8143810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:41.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7ffab659c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffab8060370 0x7ffab8143810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38730/0 (socket says 172.21.15.36:38730) 2022-06-17T12:55:41.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7ffab659c700 1 -- 172.21.15.36:0/728682983 learned_addr learned my addr 172.21.15.36:0/728682983 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:41.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7ffab5d9b700 1 --2- 172.21.15.36:0/728682983 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffab8060c40 0x7ffab8145b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:41.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7ffab6d9d700 1 --2- 172.21.15.36:0/728682983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffab80b6780 0x7ffab8066190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:41.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7ffab659c700 1 -- 172.21.15.36:0/728682983 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffab8060c40 msgr2=0x7ffab8145b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:41.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7ffab659c700 1 --2- 172.21.15.36:0/728682983 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffab8060c40 0x7ffab8145b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:41.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7f9c32f13700 1 Processor -- start 2022-06-17T12:55:41.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7ffab659c700 1 -- 172.21.15.36:0/728682983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffab80b6780 msgr2=0x7ffab8066190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:41.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7ffab659c700 1 --2- 172.21.15.36:0/728682983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffab80b6780 0x7ffab8066190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:41.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7ffab659c700 1 -- 172.21.15.36:0/728682983 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa000b1e0 con 0x7ffab8060370 2022-06-17T12:55:41.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7f9c32f13700 1 -- start start 2022-06-17T12:55:41.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7ffab659c700 1 --2- 172.21.15.36:0/728682983 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffab8060370 0x7ffab8143810 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7ffab0001250 tx=0x7ffab00013f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:41.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7ffaaf7fe700 1 -- 172.21.15.36:0/728682983 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffab000f470 con 0x7ffab8060370 2022-06-17T12:55:41.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7ffaaf7fe700 1 -- 172.21.15.36:0/728682983 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab000f5d0 con 0x7ffab8060370 2022-06-17T12:55:41.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7ffabc8f1700 1 -- 172.21.15.36:0/728682983 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab8067c90 con 0x7ffab8060370 2022-06-17T12:55:41.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.639+0000 7ffaaf7fe700 1 -- 172.21.15.36:0/728682983 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffab000f930 con 0x7ffab8060370 2022-06-17T12:55:41.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7ffabc8f1700 1 -- 172.21.15.36:0/728682983 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab81426d0 con 0x7ffab8060370 2022-06-17T12:55:41.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7f9c32f13700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c2c0abbb0 0x7f9c2c0b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:41.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7f9c32f13700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9c2c0acf00 0x7f9c2c0b4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:41.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7f9c32f13700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c2c0b1d70 0x7f9c2c1356f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:41.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7f9c32f13700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c13fd70 con 0x7f9c2c0acf00 2022-06-17T12:55:41.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7f9c32f13700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c140240 con 0x7f9c2c0b1d70 2022-06-17T12:55:41.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7f9c32f13700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c064e50 con 0x7f9c2c0abbb0 2022-06-17T12:55:41.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7f9c2bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9c2c0acf00 0x7f9c2c0b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:41.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7f9c2bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9c2c0acf00 0x7f9c2c0b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38738/0 (socket says 172.21.15.36:38738) 2022-06-17T12:55:41.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7f9c30caf700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c2c0abbb0 0x7f9c2c0b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:41.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7f9c2bfff700 1 -- 172.21.15.36:0/2475630609 learned_addr learned my addr 172.21.15.36:0/2475630609 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:41.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7ffabc8f1700 1 -- 172.21.15.36:0/728682983 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab8064010 con 0x7ffab8060370 2022-06-17T12:55:41.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7f9c314b0700 1 --2- 172.21.15.36:0/2475630609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c2c0b1d70 0x7f9c2c1356f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:41.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7f9c2bfff700 1 -- 172.21.15.36:0/2475630609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c2c0abbb0 msgr2=0x7f9c2c0b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:41.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7f9c2bfff700 1 --2- 172.21.15.36:0/2475630609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c2c0abbb0 0x7f9c2c0b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:41.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7f9c2bfff700 1 -- 172.21.15.36:0/2475630609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c2c0b1d70 msgr2=0x7f9c2c1356f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:41.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7f9c2bfff700 1 --2- 172.21.15.36:0/2475630609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c2c0b1d70 0x7f9c2c1356f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:41.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7f9c2bfff700 1 -- 172.21.15.36:0/2475630609 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1c00b1e0 con 0x7f9c2c0acf00 2022-06-17T12:55:41.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7f9c2bfff700 1 --2- 172.21.15.36:0/2475630609 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9c2c0acf00 0x7f9c2c0b4360 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0x7f9c2000a000 tx=0x7f9c20008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:41.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7ffaaf7fe700 1 -- 172.21.15.36:0/728682983 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ffab0019030 con 0x7ffab8060370 2022-06-17T12:55:41.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7f9c29ffb700 1 -- 172.21.15.36:0/2475630609 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c20011780 con 0x7f9c2c0acf00 2022-06-17T12:55:41.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7f9c32f13700 1 -- 172.21.15.36:0/2475630609 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c2c13ffa0 con 0x7f9c2c0acf00 2022-06-17T12:55:41.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7f9c32f13700 1 -- 172.21.15.36:0/2475630609 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c2c0af9c0 con 0x7f9c2c0acf00 2022-06-17T12:55:41.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7ffaaf7fe700 1 --2- 172.21.15.36:0/728682983 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffa9c0539b0 0x7ffa9c055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:41.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7ffaaf7fe700 1 -- 172.21.15.36:0/728682983 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ffab006b780 con 0x7ffab8060370 2022-06-17T12:55:41.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7ffab5d9b700 1 --2- 172.21.15.36:0/728682983 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffa9c0539b0 0x7ffa9c055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:41.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.643+0000 7ffab5d9b700 1 --2- 172.21.15.36:0/728682983 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffa9c0539b0 0x7ffa9c055e70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7ffaa001c000 tx=0x7ffaa0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:41.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.647+0000 7f9c32f13700 1 -- 172.21.15.36:0/2475630609 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c2c057410 con 0x7f9c2c0acf00 2022-06-17T12:55:41.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.651+0000 7f9c29ffb700 1 -- 172.21.15.36:0/2475630609 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c200118e0 con 0x7f9c2c0acf00 2022-06-17T12:55:41.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.651+0000 7f9c29ffb700 1 -- 172.21.15.36:0/2475630609 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c20011c40 con 0x7f9c2c0acf00 2022-06-17T12:55:41.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.651+0000 7ffaaf7fe700 1 -- 172.21.15.36:0/728682983 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffab0042470 con 0x7ffab8060370 2022-06-17T12:55:41.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.651+0000 7f9c29ffb700 1 -- 172.21.15.36:0/2475630609 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f9c20012420 con 0x7f9c2c0acf00 2022-06-17T12:55:41.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.651+0000 7f9c29ffb700 1 --2- 172.21.15.36:0/2475630609 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9c0c0539b0 0x7f9c0c055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:41.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.651+0000 7f9c29ffb700 1 -- 172.21.15.36:0/2475630609 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9c2006ce10 con 0x7f9c2c0acf00 2022-06-17T12:55:41.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.651+0000 7f9c30caf700 1 --2- 172.21.15.36:0/2475630609 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9c0c0539b0 0x7f9c0c055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:41.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.651+0000 7f9c30caf700 1 --2- 172.21.15.36:0/2475630609 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9c0c0539b0 0x7f9c0c055e70 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f9c1c01c000 tx=0x7f9c1c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:41.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.655+0000 7f9c29ffb700 1 -- 172.21.15.36:0/2475630609 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9c20043aa0 con 0x7f9c2c0acf00 2022-06-17T12:55:41.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.819+0000 7ffabc8f1700 1 -- 172.21.15.36:0/728682983 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7ffab80affe0 con 0x7ffab8060370 2022-06-17T12:55:41.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:41.819+0000 7f9c32f13700 1 -- 172.21.15.36:0/2475630609 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7f9c2c060660 con 0x7f9c2c0acf00 2022-06-17T12:55:42.703 INFO:tasks.ceph.mon.a.smithi036.stderr:2022-06-17T12:55:42.711+0000 7fb7a2649700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-17T12:55:42.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.719+0000 7ffaaf7fe700 1 -- 172.21.15.36:0/728682983 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 6632 v61) v1 ==== 95+0+0 (secure 0 0 0) 0x7ffab0017020 con 0x7ffab8060370 2022-06-17T12:55:42.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.719+0000 7f9c29ffb700 1 -- 172.21.15.36:0/2475630609 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 7531 v61) v1 ==== 95+0+0 (secure 0 0 0) 0x7f9c200470c0 con 0x7f9c2c0acf00 2022-06-17T12:55:42.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.723+0000 7ffabc8f1700 1 -- 172.21.15.36:0/728682983 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffa9c0539b0 msgr2=0x7ffa9c055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:42.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.723+0000 7ffabc8f1700 1 --2- 172.21.15.36:0/728682983 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffa9c0539b0 0x7ffa9c055e70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7ffaa001c000 tx=0x7ffaa0015000).stop 2022-06-17T12:55:42.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.723+0000 7f9c32f13700 1 -- 172.21.15.36:0/2475630609 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9c0c0539b0 msgr2=0x7f9c0c055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:42.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.723+0000 7ffabc8f1700 1 -- 172.21.15.36:0/728682983 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffab8060370 msgr2=0x7ffab8143810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:42.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.723+0000 7ffabc8f1700 1 --2- 172.21.15.36:0/728682983 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffab8060370 0x7ffab8143810 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7ffab0001250 tx=0x7ffab00013f0).stop 2022-06-17T12:55:42.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.723+0000 7f9c32f13700 1 --2- 172.21.15.36:0/2475630609 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9c0c0539b0 0x7f9c0c055e70 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f9c1c01c000 tx=0x7f9c1c015000).stop 2022-06-17T12:55:42.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.723+0000 7f9c32f13700 1 -- 172.21.15.36:0/2475630609 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9c2c0acf00 msgr2=0x7f9c2c0b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:42.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.723+0000 7f9c32f13700 1 --2- 172.21.15.36:0/2475630609 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9c2c0acf00 0x7f9c2c0b4360 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0x7f9c2000a000 tx=0x7f9c20008040).stop 2022-06-17T12:55:42.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.723+0000 7ffabc8f1700 1 -- 172.21.15.36:0/728682983 shutdown_connections 2022-06-17T12:55:42.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.723+0000 7ffabc8f1700 1 --2- 172.21.15.36:0/728682983 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ffa9c0539b0 0x7ffa9c055e70 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:42.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.723+0000 7ffabc8f1700 1 --2- 172.21.15.36:0/728682983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffab80b6780 0x7ffab8066190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:42.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.723+0000 7ffabc8f1700 1 --2- 172.21.15.36:0/728682983 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffab8060c40 0x7ffab8145b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:42.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.723+0000 7ffabc8f1700 1 --2- 172.21.15.36:0/728682983 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ffab8060370 0x7ffab8143810 secure :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 rx=0x7ffab0001250 tx=0x7ffab00013f0).stop 2022-06-17T12:55:42.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.723+0000 7ffabc8f1700 1 -- 172.21.15.36:0/728682983 >> 172.21.15.36:0/728682983 conn(0x7ffab8063890 msgr2=0x7ffab805bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:42.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.723+0000 7ffabc8f1700 1 -- 172.21.15.36:0/728682983 shutdown_connections 2022-06-17T12:55:42.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.723+0000 7ffabc8f1700 1 -- 172.21.15.36:0/728682983 wait complete. 2022-06-17T12:55:42.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.727+0000 7f9c32f13700 1 -- 172.21.15.36:0/2475630609 shutdown_connections 2022-06-17T12:55:42.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.727+0000 7f9c32f13700 1 --2- 172.21.15.36:0/2475630609 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9c0c0539b0 0x7f9c0c055e70 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:42.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.727+0000 7f9c32f13700 1 --2- 172.21.15.36:0/2475630609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c2c0b1d70 0x7f9c2c1356f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:42.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.727+0000 7f9c32f13700 1 --2- 172.21.15.36:0/2475630609 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9c2c0acf00 0x7f9c2c0b4360 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:42.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.727+0000 7f9c32f13700 1 --2- 172.21.15.36:0/2475630609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c2c0abbb0 0x7f9c2c0b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:42.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.727+0000 7f9c32f13700 1 -- 172.21.15.36:0/2475630609 >> 172.21.15.36:0/2475630609 conn(0x7f9c2c0636d0 msgr2=0x7f9c2c0ac050 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:42.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.727+0000 7f9c32f13700 1 -- 172.21.15.36:0/2475630609 shutdown_connections 2022-06-17T12:55:42.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.727+0000 7f9c32f13700 1 -- 172.21.15.36:0/2475630609 wait complete. 2022-06-17T12:55:42.731 INFO:teuthology.orchestra.run.smithi036.stderr:failed mds gid 6632 2022-06-17T12:55:42.732 INFO:teuthology.orchestra.run.smithi036.stderr:failed mds gid 7531 2022-06-17T12:55:42.735 DEBUG:teuthology.parallel:result is '' 2022-06-17T12:55:42.736 DEBUG:teuthology.parallel:result is '' 2022-06-17T12:55:42.737 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-17T12:55:42.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.863+0000 7fdfa6d5b700 1 Processor -- start 2022-06-17T12:55:42.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.863+0000 7fdfa6d5b700 1 -- start start 2022-06-17T12:55:42.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa6d5b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfa00b9860 0x7fdfa0143840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:42.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa6d5b700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdfa00b45c0 0x7fdfa00b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:42.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa6d5b700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa00b4160 con 0x7fdfa00b4ed0 2022-06-17T12:55:42.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa6d5b700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfa00b9c40 con 0x7fdfa00b45c0 2022-06-17T12:55:42.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa6d5b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa00627d0 con 0x7fdfa00b9860 2022-06-17T12:55:42.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa5558700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdfa00b45c0 0x7fdfa00b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:42.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa5d59700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfa00b9860 0x7fdfa0143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:42.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa655a700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdfa00b4ed0 0x7fdfa0146490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:52014/0 (socket says 172.21.15.36:52014) 2022-06-17T12:55:42.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa655a700 1 -- 172.21.15.36:0/2730489152 learned_addr learned my addr 172.21.15.36:0/2730489152 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:42.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa5558700 1 -- 172.21.15.36:0/2730489152 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfa00b9860 msgr2=0x7fdfa0143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:42.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa5558700 1 --2- 172.21.15.36:0/2730489152 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfa00b9860 0x7fdfa0143840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:42.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa5558700 1 -- 172.21.15.36:0/2730489152 >> v1:172.21.15.70:6789/0 conn(0x7fdfa00b4ed0 legacy=0x7fdfa0146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:42.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa5558700 1 -- 172.21.15.36:0/2730489152 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa0062a70 con 0x7fdfa00b45c0 2022-06-17T12:55:42.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa5558700 1 --2- 172.21.15.36:0/2730489152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa00b45c0 0x7fdfa00b49e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7fdf94002f70 tx=0x7fdf94006df0).ready entity=mon.1 client_cookie=a3aaa46aa178bb5b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:42.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa4d57700 1 -- 172.21.15.36:0/2730489152 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf9400d570 con 0x7fdfa00b45c0 2022-06-17T12:55:42.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa4d57700 1 -- 172.21.15.36:0/2730489152 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf94000e90 con 0x7fdfa00b45c0 2022-06-17T12:55:42.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa4d57700 1 -- 172.21.15.36:0/2730489152 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf9400d8c0 con 0x7fdfa00b45c0 2022-06-17T12:55:42.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa6d5b700 1 -- 172.21.15.36:0/2730489152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa00b45c0 msgr2=0x7fdfa00b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:42.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa6d5b700 1 --2- 172.21.15.36:0/2730489152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa00b45c0 0x7fdfa00b49e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7fdf94002f70 tx=0x7fdf94006df0).stop 2022-06-17T12:55:42.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa6d5b700 1 -- 172.21.15.36:0/2730489152 shutdown_connections 2022-06-17T12:55:42.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa6d5b700 1 --2- 172.21.15.36:0/2730489152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa00b45c0 0x7fdfa00b49e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:42.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa6d5b700 1 --2- 172.21.15.36:0/2730489152 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfa00b9860 0x7fdfa0143840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:42.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa6d5b700 1 -- 172.21.15.36:0/2730489152 >> 172.21.15.36:0/2730489152 conn(0x7fdfa005c550 msgr2=0x7fdfa0060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:42.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa6d5b700 1 -- 172.21.15.36:0/2730489152 shutdown_connections 2022-06-17T12:55:42.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.867+0000 7fdfa6d5b700 1 -- 172.21.15.36:0/2730489152 wait complete. 2022-06-17T12:55:42.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.871+0000 7fdfa6d5b700 1 Processor -- start 2022-06-17T12:55:42.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.871+0000 7fdfa6d5b700 1 -- start start 2022-06-17T12:55:42.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.871+0000 7fdfa6d5b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfa00b45c0 0x7fdfa013d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:42.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.871+0000 7fdfa6d5b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa00b4ed0 0x7fdfa013db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:42.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.871+0000 7fdfa6d5b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfa00b9860 0x7fdfa0155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:42.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.871+0000 7fdfa6d5b700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa0152c60 con 0x7fdfa00b9860 2022-06-17T12:55:42.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.871+0000 7fdfa6d5b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa0146f00 con 0x7fdfa00b4ed0 2022-06-17T12:55:42.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.871+0000 7fdfa6d5b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa0153380 con 0x7fdfa00b45c0 2022-06-17T12:55:42.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.871+0000 7fdfa5d59700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfa00b45c0 0x7fdfa013d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:42.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.871+0000 7fdfa5d59700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfa00b45c0 0x7fdfa013d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:53542/0 (socket says 172.21.15.36:53542) 2022-06-17T12:55:42.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.871+0000 7fdfa655a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfa00b9860 0x7fdfa0155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:42.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.871+0000 7fdfa5d59700 1 -- 172.21.15.36:0/386948832 learned_addr learned my addr 172.21.15.36:0/386948832 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:42.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.871+0000 7fdfa5558700 1 --2- 172.21.15.36:0/386948832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa00b4ed0 0x7fdfa013db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:42.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.871+0000 7fdfa5d59700 1 -- 172.21.15.36:0/386948832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa00b4ed0 msgr2=0x7fdfa013db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:42.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.871+0000 7fdfa5d59700 1 --2- 172.21.15.36:0/386948832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa00b4ed0 0x7fdfa013db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:42.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.871+0000 7fdfa5d59700 1 -- 172.21.15.36:0/386948832 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfa00b9860 msgr2=0x7fdfa0155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:42.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.871+0000 7fdfa5d59700 1 --2- 172.21.15.36:0/386948832 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfa00b9860 0x7fdfa0155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:42.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.871+0000 7fdfa5d59700 1 -- 172.21.15.36:0/386948832 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9400b180 con 0x7fdfa00b45c0 2022-06-17T12:55:42.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.871+0000 7fdfa5558700 1 --2- 172.21.15.36:0/386948832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa00b4ed0 0x7fdfa013db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:42.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.871+0000 7fdfa655a700 1 --2- 172.21.15.36:0/386948832 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfa00b9860 0x7fdfa0155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:42.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.871+0000 7fdfa5d59700 1 --2- 172.21.15.36:0/386948832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfa00b45c0 0x7fdfa013d690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fdf9c00c000 tx=0x7fdf9c006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:42.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.875+0000 7fdf92ffd700 1 -- 172.21.15.36:0/386948832 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf9c011750 con 0x7fdfa00b45c0 2022-06-17T12:55:42.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.875+0000 7fdf92ffd700 1 -- 172.21.15.36:0/386948832 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9c0118b0 con 0x7fdfa00b45c0 2022-06-17T12:55:42.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.875+0000 7fdf92ffd700 1 -- 172.21.15.36:0/386948832 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf9c011c10 con 0x7fdfa00b45c0 2022-06-17T12:55:42.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.875+0000 7fdfa6d5b700 1 -- 172.21.15.36:0/386948832 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa0145e30 con 0x7fdfa00b45c0 2022-06-17T12:55:42.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.875+0000 7fdfa6d5b700 1 -- 172.21.15.36:0/386948832 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa0146980 con 0x7fdfa00b45c0 2022-06-17T12:55:42.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.875+0000 7fdfa6d5b700 1 -- 172.21.15.36:0/386948832 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa013e0c0 con 0x7fdfa00b45c0 2022-06-17T12:55:42.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.875+0000 7fdf92ffd700 1 -- 172.21.15.36:0/386948832 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fdf9c011d70 con 0x7fdfa00b45c0 2022-06-17T12:55:42.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.875+0000 7fdf92ffd700 1 --2- 172.21.15.36:0/386948832 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdf80053ad0 0x7fdf80055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:42.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.875+0000 7fdf92ffd700 1 -- 172.21.15.36:0/386948832 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fdf9c06c450 con 0x7fdfa00b45c0 2022-06-17T12:55:42.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.875+0000 7fdfa5558700 1 --2- 172.21.15.36:0/386948832 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdf80053ad0 0x7fdf80055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:42.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.879+0000 7fdfa5558700 1 --2- 172.21.15.36:0/386948832 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdf80053ad0 0x7fdf80055f90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7fdf9401a000 tx=0x7fdf94014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:42.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:42.887+0000 7fdf92ffd700 1 -- 172.21.15.36:0/386948832 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdf9c043200 con 0x7fdfa00b45c0 2022-06-17T12:55:43.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.055+0000 7fdfa6d5b700 1 -- 172.21.15.36:0/386948832 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdfa0062eb0 con 0x7fdfa00b45c0 2022-06-17T12:55:43.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.735+0000 7fdf92ffd700 1 -- 172.21.15.36:0/386948832 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v62) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdf9c046820 con 0x7fdfa00b45c0 2022-06-17T12:55:43.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.739+0000 7fdfa6d5b700 1 -- 172.21.15.36:0/386948832 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdf80053ad0 msgr2=0x7fdf80055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:43.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.739+0000 7fdfa6d5b700 1 --2- 172.21.15.36:0/386948832 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdf80053ad0 0x7fdf80055f90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7fdf9401a000 tx=0x7fdf94014000).stop 2022-06-17T12:55:43.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.739+0000 7fdfa6d5b700 1 -- 172.21.15.36:0/386948832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfa00b45c0 msgr2=0x7fdfa013d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:43.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.739+0000 7fdfa6d5b700 1 --2- 172.21.15.36:0/386948832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfa00b45c0 0x7fdfa013d690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fdf9c00c000 tx=0x7fdf9c006df0).stop 2022-06-17T12:55:43.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.739+0000 7fdfa6d5b700 1 -- 172.21.15.36:0/386948832 shutdown_connections 2022-06-17T12:55:43.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.739+0000 7fdfa6d5b700 1 --2- 172.21.15.36:0/386948832 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdf80053ad0 0x7fdf80055f90 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:43.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.739+0000 7fdfa6d5b700 1 --2- 172.21.15.36:0/386948832 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdfa00b9860 0x7fdfa0155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:43.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.739+0000 7fdfa6d5b700 1 --2- 172.21.15.36:0/386948832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa00b4ed0 0x7fdfa013db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:43.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.739+0000 7fdfa6d5b700 1 --2- 172.21.15.36:0/386948832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdfa00b45c0 0x7fdfa013d690 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:43.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.739+0000 7fdfa6d5b700 1 -- 172.21.15.36:0/386948832 >> 172.21.15.36:0/386948832 conn(0x7fdfa005c550 msgr2=0x7fdfa00b76b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:43.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.739+0000 7fdfa6d5b700 1 -- 172.21.15.36:0/386948832 shutdown_connections 2022-06-17T12:55:43.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.739+0000 7fdfa6d5b700 1 -- 172.21.15.36:0/386948832 wait complete. 2022-06-17T12:55:43.743 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-17T12:55:43.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.871+0000 7fda52a7d700 1 Processor -- start 2022-06-17T12:55:43.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.871+0000 7fda52a7d700 1 -- start start 2022-06-17T12:55:43.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda52a7d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda4c0b4e10 0x7fda4c0b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:43.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda52a7d700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda4c0b21d0 con 0x7fda4c0620d0 2022-06-17T12:55:43.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda52a7d700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda4c0b2430 con 0x7fda4c0b31c0 2022-06-17T12:55:43.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda52a7d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c0b26d0 con 0x7fda4c0b4e10 2022-06-17T12:55:43.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda50819700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fda4c0620d0 0x7fda4c0b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56290/0 (socket says 172.21.15.36:56290) 2022-06-17T12:55:43.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda50819700 1 -- 172.21.15.36:0/855333185 learned_addr learned my addr 172.21.15.36:0/855333185 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:43.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda4bfff700 1 --2- 172.21.15.36:0/855333185 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda4c0b4e10 0x7fda4c0b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:43.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda4b7fe700 1 -- 172.21.15.36:0/855333185 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 291695566 0 0) 0x7fda4c0b21d0 con 0x7fda4c0620d0 2022-06-17T12:55:43.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda4b7fe700 1 -- 172.21.15.36:0/855333185 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda30003260 con 0x7fda4c0620d0 2022-06-17T12:55:43.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda4b7fe700 1 -- 172.21.15.36:0/855333185 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147707895 0 0) 0x7fda4c0b2430 con 0x7fda4c0b31c0 2022-06-17T12:55:43.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda4b7fe700 1 -- 172.21.15.36:0/855333185 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda4c0b21d0 con 0x7fda4c0b31c0 2022-06-17T12:55:43.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda4bfff700 1 -- 172.21.15.36:0/855333185 >> v1:172.21.15.70:6789/0 conn(0x7fda4c0b31c0 legacy=0x7fda4c141ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:43.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda4bfff700 1 -- 172.21.15.36:0/855333185 >> v1:172.21.15.36:6789/0 conn(0x7fda4c0620d0 legacy=0x7fda4c0b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:43.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda4b7fe700 1 -- 172.21.15.36:0/855333185 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3177246200 0 0) 0x7fda30003260 con 0x7fda4c0620d0 2022-06-17T12:55:43.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda4bfff700 1 -- 172.21.15.36:0/855333185 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4c0b43f0 con 0x7fda4c0b4e10 2022-06-17T12:55:43.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda4bfff700 1 --2- 172.21.15.36:0/855333185 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda4c0b4e10 0x7fda4c0b5bc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fda40001110 tx=0x7fda38006df0).ready entity=mon.2 client_cookie=2afb381c7a557d20 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:43.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda4b7fe700 1 -- 172.21.15.36:0/855333185 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda40001680 con 0x7fda4c0b4e10 2022-06-17T12:55:43.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda4b7fe700 1 -- 172.21.15.36:0/855333185 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda38000e90 con 0x7fda4c0b4e10 2022-06-17T12:55:43.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda4b7fe700 1 -- 172.21.15.36:0/855333185 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda38012460 con 0x7fda4c0b4e10 2022-06-17T12:55:43.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda52a7d700 1 -- 172.21.15.36:0/855333185 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda4c0b4e10 msgr2=0x7fda4c0b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:43.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda52a7d700 1 --2- 172.21.15.36:0/855333185 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda4c0b4e10 0x7fda4c0b5bc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fda40001110 tx=0x7fda38006df0).stop 2022-06-17T12:55:43.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda52a7d700 1 -- 172.21.15.36:0/855333185 shutdown_connections 2022-06-17T12:55:43.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda52a7d700 1 --2- 172.21.15.36:0/855333185 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda4c0b4e10 0x7fda4c0b5bc0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:43.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda52a7d700 1 -- 172.21.15.36:0/855333185 >> 172.21.15.36:0/855333185 conn(0x7fda4c0b9850 msgr2=0x7fda4c0b9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:43.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda52a7d700 1 -- 172.21.15.36:0/855333185 shutdown_connections 2022-06-17T12:55:43.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.875+0000 7fda52a7d700 1 -- 172.21.15.36:0/855333185 wait complete. 2022-06-17T12:55:43.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.879+0000 7fda52a7d700 1 Processor -- start 2022-06-17T12:55:43.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.879+0000 7fda52a7d700 1 -- start start 2022-06-17T12:55:43.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.879+0000 7fda52a7d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda4c0620d0 0x7fda4c146e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:43.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.879+0000 7fda52a7d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda4c0b31c0 0x7fda4c152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:43.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.879+0000 7fda52a7d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda4c0b4e10 0x7fda4c0af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:43.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.879+0000 7fda52a7d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c13ef40 con 0x7fda4c0b4e10 2022-06-17T12:55:43.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.879+0000 7fda52a7d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c112ae0 con 0x7fda4c0b31c0 2022-06-17T12:55:43.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.879+0000 7fda52a7d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c112d80 con 0x7fda4c0620d0 2022-06-17T12:55:43.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.879+0000 7fda4bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda4c0b31c0 0x7fda4c152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:43.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.879+0000 7fda5101a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda4c0b4e10 0x7fda4c0af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:43.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.879+0000 7fda50819700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda4c0620d0 0x7fda4c146e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:43.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.879+0000 7fda4bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda4c0b31c0 0x7fda4c152880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:59146/0 (socket says 172.21.15.36:59146) 2022-06-17T12:55:43.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.879+0000 7fda4bfff700 1 -- 172.21.15.36:0/1174226252 learned_addr learned my addr 172.21.15.36:0/1174226252 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:43.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.879+0000 7fda5101a700 1 -- 172.21.15.36:0/1174226252 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda4c0620d0 msgr2=0x7fda4c146e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:43.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.879+0000 7fda5101a700 1 --2- 172.21.15.36:0/1174226252 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda4c0620d0 0x7fda4c146e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:43.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.879+0000 7fda5101a700 1 -- 172.21.15.36:0/1174226252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda4c0b31c0 msgr2=0x7fda4c152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:43.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.879+0000 7fda5101a700 1 --2- 172.21.15.36:0/1174226252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda4c0b31c0 0x7fda4c152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:43.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.879+0000 7fda5101a700 1 -- 172.21.15.36:0/1174226252 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3800b180 con 0x7fda4c0b4e10 2022-06-17T12:55:43.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.879+0000 7fda4bfff700 1 --2- 172.21.15.36:0/1174226252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda4c0b31c0 0x7fda4c152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:43.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.879+0000 7fda5101a700 1 --2- 172.21.15.36:0/1174226252 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda4c0b4e10 0x7fda4c0af780 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7fda44001490 tx=0x7fda44007d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:43.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.879+0000 7fda49ffb700 1 -- 172.21.15.36:0/1174226252 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda4400f470 con 0x7fda4c0b4e10 2022-06-17T12:55:43.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.883+0000 7fda49ffb700 1 -- 172.21.15.36:0/1174226252 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4400f5d0 con 0x7fda4c0b4e10 2022-06-17T12:55:43.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.883+0000 7fda52a7d700 1 -- 172.21.15.36:0/1174226252 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda4c1452f0 con 0x7fda4c0b4e10 2022-06-17T12:55:43.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.883+0000 7fda49ffb700 1 -- 172.21.15.36:0/1174226252 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda4400f930 con 0x7fda4c0b4e10 2022-06-17T12:55:43.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.883+0000 7fda52a7d700 1 -- 172.21.15.36:0/1174226252 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda4c113260 con 0x7fda4c0b4e10 2022-06-17T12:55:43.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.883+0000 7fda52a7d700 1 -- 172.21.15.36:0/1174226252 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda4c057410 con 0x7fda4c0b4e10 2022-06-17T12:55:43.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.883+0000 7fda49ffb700 1 -- 172.21.15.36:0/1174226252 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fda44019030 con 0x7fda4c0b4e10 2022-06-17T12:55:43.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.883+0000 7fda49ffb700 1 --2- 172.21.15.36:0/1174226252 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fda2c053920 0x7fda2c055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:43.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.883+0000 7fda49ffb700 1 -- 172.21.15.36:0/1174226252 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fda4406b770 con 0x7fda4c0b4e10 2022-06-17T12:55:43.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.883+0000 7fda50819700 1 --2- 172.21.15.36:0/1174226252 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fda2c053920 0x7fda2c055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:43.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.887+0000 7fda50819700 1 --2- 172.21.15.36:0/1174226252 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fda2c053920 0x7fda2c055de0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fda4000d000 tx=0x7fda40001e20).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:43.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:43.891+0000 7fda49ffb700 1 -- 172.21.15.36:0/1174226252 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fda440424c0 con 0x7fda4c0b4e10 2022-06-17T12:55:44.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.023+0000 7fda52a7d700 1 -- 172.21.15.36:0/1174226252 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fda4c0b0640 con 0x7fda4c0b4e10 2022-06-17T12:55:44.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.739+0000 7fda49ffb700 1 -- 172.21.15.36:0/1174226252 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v116) v1 ==== 185+0+0 (secure 0 0 0) 0x7fda44045ae0 con 0x7fda4c0b4e10 2022-06-17T12:55:44.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.747+0000 7fda52a7d700 1 -- 172.21.15.36:0/1174226252 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fda2c053920 msgr2=0x7fda2c055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:44.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.747+0000 7fda52a7d700 1 --2- 172.21.15.36:0/1174226252 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fda2c053920 0x7fda2c055de0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fda4000d000 tx=0x7fda40001e20).stop 2022-06-17T12:55:44.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.747+0000 7fda52a7d700 1 -- 172.21.15.36:0/1174226252 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda4c0b4e10 msgr2=0x7fda4c0af780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:44.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.747+0000 7fda52a7d700 1 --2- 172.21.15.36:0/1174226252 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda4c0b4e10 0x7fda4c0af780 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7fda44001490 tx=0x7fda44007d40).stop 2022-06-17T12:55:44.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.747+0000 7fda52a7d700 1 -- 172.21.15.36:0/1174226252 shutdown_connections 2022-06-17T12:55:44.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.747+0000 7fda52a7d700 1 --2- 172.21.15.36:0/1174226252 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fda2c053920 0x7fda2c055de0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:44.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.747+0000 7fda52a7d700 1 --2- 172.21.15.36:0/1174226252 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fda4c0b4e10 0x7fda4c0af780 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:44.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.747+0000 7fda52a7d700 1 --2- 172.21.15.36:0/1174226252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda4c0b31c0 0x7fda4c152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:44.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.747+0000 7fda52a7d700 1 --2- 172.21.15.36:0/1174226252 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda4c0620d0 0x7fda4c146e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:44.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.747+0000 7fda52a7d700 1 -- 172.21.15.36:0/1174226252 >> 172.21.15.36:0/1174226252 conn(0x7fda4c0b9850 msgr2=0x7fda4c1416f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:44.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.747+0000 7fda52a7d700 1 -- 172.21.15.36:0/1174226252 shutdown_connections 2022-06-17T12:55:44.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.747+0000 7fda52a7d700 1 -- 172.21.15.36:0/1174226252 wait complete. 2022-06-17T12:55:44.741 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' removed 2022-06-17T12:55:44.749 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-17T12:55:44.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.875+0000 7f76766ca700 1 Processor -- start 2022-06-17T12:55:44.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.875+0000 7f76766ca700 1 -- start start 2022-06-17T12:55:44.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.875+0000 7f76766ca700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7670060ac0 0x7f7670140260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:44.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.875+0000 7f76766ca700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76700b2c80 0x7f7670140750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:44.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.875+0000 7f76766ca700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7670060ea0 con 0x7f7670067260 2022-06-17T12:55:44.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.875+0000 7f76766ca700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76700b2210 con 0x7f7670060ac0 2022-06-17T12:55:44.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.875+0000 7f76766ca700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76700aebb0 con 0x7f76700b2c80 2022-06-17T12:55:44.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.875+0000 7f76756c8700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7670060ac0 0x7f7670140260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:44.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.875+0000 7f7675ec9700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f7670067260 0x7f76700aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56304/0 (socket says 172.21.15.36:56304) 2022-06-17T12:55:44.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.875+0000 7f7675ec9700 1 -- 172.21.15.36:0/2142684134 learned_addr learned my addr 172.21.15.36:0/2142684134 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:44.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.875+0000 7f7674ec7700 1 --2- 172.21.15.36:0/2142684134 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76700b2c80 0x7f7670140750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:44.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.875+0000 7f76756c8700 1 -- 172.21.15.36:0/2142684134 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76700b2c80 msgr2=0x7f7670140750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:44.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.875+0000 7f76756c8700 1 --2- 172.21.15.36:0/2142684134 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76700b2c80 0x7f7670140750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:44.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.875+0000 7f76756c8700 1 -- 172.21.15.36:0/2142684134 >> v1:172.21.15.36:6789/0 conn(0x7f7670067260 legacy=0x7f76700aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:44.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.875+0000 7f76756c8700 1 -- 172.21.15.36:0/2142684134 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76700b1ad0 con 0x7f7670060ac0 2022-06-17T12:55:44.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.875+0000 7f7674ec7700 1 --2- 172.21.15.36:0/2142684134 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76700b2c80 0x7f7670140750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:44.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.875+0000 7f76756c8700 1 --2- 172.21.15.36:0/2142684134 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7670060ac0 0x7f7670140260 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7f766c002f70 tx=0x7f766c006df0).ready entity=mon.0 client_cookie=3d335ac3f84b4abb server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:44.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.875+0000 7f765ffff700 1 -- 172.21.15.36:0/2142684134 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f766c00d570 con 0x7f7670060ac0 2022-06-17T12:55:44.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.875+0000 7f765ffff700 1 -- 172.21.15.36:0/2142684134 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766c000e90 con 0x7f7670060ac0 2022-06-17T12:55:44.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.875+0000 7f765ffff700 1 -- 172.21.15.36:0/2142684134 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f766c00d8c0 con 0x7f7670060ac0 2022-06-17T12:55:44.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f76766ca700 1 -- 172.21.15.36:0/2142684134 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7670060ac0 msgr2=0x7f7670140260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:44.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f76766ca700 1 --2- 172.21.15.36:0/2142684134 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7670060ac0 0x7f7670140260 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7f766c002f70 tx=0x7f766c006df0).stop 2022-06-17T12:55:44.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f76766ca700 1 -- 172.21.15.36:0/2142684134 shutdown_connections 2022-06-17T12:55:44.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f76766ca700 1 --2- 172.21.15.36:0/2142684134 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76700b2c80 0x7f7670140750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:44.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f76766ca700 1 --2- 172.21.15.36:0/2142684134 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7670060ac0 0x7f7670140260 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:44.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f76766ca700 1 -- 172.21.15.36:0/2142684134 >> 172.21.15.36:0/2142684134 conn(0x7f76700b64f0 msgr2=0x7f76700b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:44.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f76766ca700 1 -- 172.21.15.36:0/2142684134 shutdown_connections 2022-06-17T12:55:44.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f76766ca700 1 -- 172.21.15.36:0/2142684134 wait complete. 2022-06-17T12:55:44.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f76766ca700 1 Processor -- start 2022-06-17T12:55:44.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f76766ca700 1 -- start start 2022-06-17T12:55:44.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f76766ca700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7670060ac0 0x7f767013cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:44.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f76766ca700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7670067260 0x7f76701560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:44.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f76766ca700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f76700b2c80 0x7f76701522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:44.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f76766ca700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767014fd70 con 0x7f76700b2c80 2022-06-17T12:55:44.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f76766ca700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76700a0070 con 0x7f7670060ac0 2022-06-17T12:55:44.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f76766ca700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7670155970 con 0x7f7670067260 2022-06-17T12:55:44.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f7675ec9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f76700b2c80 0x7f76701522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:44.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f7675ec9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f76700b2c80 0x7f76701522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38782/0 (socket says 172.21.15.36:38782) 2022-06-17T12:55:44.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f7675ec9700 1 -- 172.21.15.36:0/261128024 learned_addr learned my addr 172.21.15.36:0/261128024 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:44.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f76756c8700 1 --2- 172.21.15.36:0/261128024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7670060ac0 0x7f767013cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:44.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f7674ec7700 1 --2- 172.21.15.36:0/261128024 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7670067260 0x7f76701560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:44.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f7675ec9700 1 -- 172.21.15.36:0/261128024 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7670067260 msgr2=0x7f76701560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:44.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f7675ec9700 1 --2- 172.21.15.36:0/261128024 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7670067260 0x7f76701560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:44.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f7675ec9700 1 -- 172.21.15.36:0/261128024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7670060ac0 msgr2=0x7f767013cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:44.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f7675ec9700 1 --2- 172.21.15.36:0/261128024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7670060ac0 0x7f767013cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:44.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f7675ec9700 1 -- 172.21.15.36:0/261128024 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766c00b180 con 0x7f76700b2c80 2022-06-17T12:55:44.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f7674ec7700 1 --2- 172.21.15.36:0/261128024 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7670067260 0x7f76701560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:44.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f7675ec9700 1 --2- 172.21.15.36:0/261128024 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f76700b2c80 0x7f76701522c0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7f76700b3820 tx=0x7f7668007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:44.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f765e7fc700 1 -- 172.21.15.36:0/261128024 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7668005ba0 con 0x7f76700b2c80 2022-06-17T12:55:44.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f765e7fc700 1 -- 172.21.15.36:0/261128024 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7668005d00 con 0x7f76700b2c80 2022-06-17T12:55:44.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f76766ca700 1 -- 172.21.15.36:0/261128024 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7670155c10 con 0x7f76700b2c80 2022-06-17T12:55:44.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f765e7fc700 1 -- 172.21.15.36:0/261128024 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76680072c0 con 0x7f76700b2c80 2022-06-17T12:55:44.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.879+0000 7f76766ca700 1 -- 172.21.15.36:0/261128024 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7670146c40 con 0x7f76700b2c80 2022-06-17T12:55:44.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.883+0000 7f765e7fc700 1 -- 172.21.15.36:0/261128024 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f766801a060 con 0x7f76700b2c80 2022-06-17T12:55:44.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.883+0000 7f76766ca700 1 -- 172.21.15.36:0/261128024 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7670057410 con 0x7f76700b2c80 2022-06-17T12:55:44.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.883+0000 7f765e7fc700 1 --2- 172.21.15.36:0/261128024 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7650053920 0x7f7650055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:44.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.883+0000 7f765e7fc700 1 -- 172.21.15.36:0/261128024 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f766806b970 con 0x7f76700b2c80 2022-06-17T12:55:44.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.883+0000 7f76756c8700 1 --2- 172.21.15.36:0/261128024 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7650053920 0x7f7650055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:44.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.883+0000 7f76756c8700 1 --2- 172.21.15.36:0/261128024 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7650053920 0x7f7650055de0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f766c01a000 tx=0x7f766c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:44.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:44.891+0000 7f765e7fc700 1 -- 172.21.15.36:0/261128024 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f76680428d0 con 0x7f76700b2c80 2022-06-17T12:55:45.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:45.019+0000 7f76766ca700 1 -- 172.21.15.36:0/261128024 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f767005d820 con 0x7f76700b2c80 2022-06-17T12:55:45.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:45.759+0000 7f765e7fc700 1 -- 172.21.15.36:0/261128024 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v117) v1 ==== 133+0+0 (secure 0 0 0) 0x7f7668018030 con 0x7f76700b2c80 2022-06-17T12:55:45.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:45.763+0000 7f76766ca700 1 -- 172.21.15.36:0/261128024 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7650053920 msgr2=0x7f7650055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:45.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:45.763+0000 7f76766ca700 1 --2- 172.21.15.36:0/261128024 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7650053920 0x7f7650055de0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f766c01a000 tx=0x7f766c015000).stop 2022-06-17T12:55:45.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:45.763+0000 7f76766ca700 1 -- 172.21.15.36:0/261128024 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f76700b2c80 msgr2=0x7f76701522c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:45.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:45.763+0000 7f76766ca700 1 --2- 172.21.15.36:0/261128024 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f76700b2c80 0x7f76701522c0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7f76700b3820 tx=0x7f7668007f10).stop 2022-06-17T12:55:45.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:45.763+0000 7f76766ca700 1 -- 172.21.15.36:0/261128024 shutdown_connections 2022-06-17T12:55:45.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:45.763+0000 7f76766ca700 1 --2- 172.21.15.36:0/261128024 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7650053920 0x7f7650055de0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:45.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:45.763+0000 7f76766ca700 1 --2- 172.21.15.36:0/261128024 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f76700b2c80 0x7f76701522c0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:45.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:45.763+0000 7f76766ca700 1 --2- 172.21.15.36:0/261128024 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7670067260 0x7f76701560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:45.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:45.763+0000 7f76766ca700 1 --2- 172.21.15.36:0/261128024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7670060ac0 0x7f767013cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:45.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:45.763+0000 7f76766ca700 1 -- 172.21.15.36:0/261128024 >> 172.21.15.36:0/261128024 conn(0x7f76700b64f0 msgr2=0x7f76700b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:45.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:45.763+0000 7f76766ca700 1 -- 172.21.15.36:0/261128024 shutdown_connections 2022-06-17T12:55:45.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:45.767+0000 7f76766ca700 1 -- 172.21.15.36:0/261128024 wait complete. 2022-06-17T12:55:45.770 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' created 2022-06-17T12:55:45.773 DEBUG:teuthology.orchestra.run.smithi107:> sudo rados -p cephfs_metadata put key /etc/fstab 2022-06-17T12:55:45.833 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.830+0000 7ff9b11e4b80 1 Processor -- start 2022-06-17T12:55:45.834 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.830+0000 7ff9b11e4b80 1 -- start start 2022-06-17T12:55:45.834 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.830+0000 7ff9b11e4b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4df9addf0 0x55d4df9a5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:45.834 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.830+0000 7ff9b11e4b80 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x55d4df9b0910 0x55d4df9a6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:45.834 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.830+0000 7ff9b11e4b80 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d4df9b3250 0x55d4df9a6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:45.835 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.830+0000 7ff9b11e4b80 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4df8a7480 con 0x55d4df9b0910 2022-06-17T12:55:45.835 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.830+0000 7ff9b11e4b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4df897460 con 0x55d4df9addf0 2022-06-17T12:55:45.835 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.830+0000 7ff9b11e4b80 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4df9bd2c0 con 0x55d4df9b3250 2022-06-17T12:55:45.835 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.830+0000 7ff9afe22700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d4df9b3250 0x55d4df9a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:45.836 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.830+0000 7ff9afe22700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d4df9b3250 0x55d4df9a6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.107:39956/0 (socket says 172.21.15.107:39956) 2022-06-17T12:55:45.836 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.830+0000 7ff9afe22700 1 -- 172.21.15.107:0/973903829 learned_addr learned my addr 172.21.15.107:0/973903829 (peer_addr_for_me v2:172.21.15.107:0/0) 2022-06-17T12:55:45.837 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.830+0000 7ff9aee20700 1 --2- 172.21.15.107:0/973903829 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x55d4df9b0910 0x55d4df9a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:45.837 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.830+0000 7ff9af621700 1 --2- 172.21.15.107:0/973903829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4df9addf0 0x55d4df9a5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:45.838 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.830+0000 7ff9afe22700 1 -- 172.21.15.107:0/973903829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4df9addf0 msgr2=0x55d4df9a5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:45.838 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.830+0000 7ff9afe22700 1 --2- 172.21.15.107:0/973903829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4df9addf0 0x55d4df9a5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:45.839 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.830+0000 7ff9afe22700 1 -- 172.21.15.107:0/973903829 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x55d4df9b0910 msgr2=0x55d4df9a6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:45.839 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.830+0000 7ff9afe22700 1 --2- 172.21.15.107:0/973903829 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x55d4df9b0910 0x55d4df9a6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:45.840 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.830+0000 7ff9afe22700 1 -- 172.21.15.107:0/973903829 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff998005cd0 con 0x55d4df9b3250 2022-06-17T12:55:45.840 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9afe22700 1 --2- 172.21.15.107:0/973903829 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d4df9b3250 0x55d4df9a6910 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x55d4df9a8fb0 tx=0x7ff9a0006af0).ready entity=mon.2 client_cookie=64596d0b44a7d55b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:45.840 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9ad61d700 1 -- 172.21.15.107:0/973903829 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9a00020a0 con 0x55d4df9b3250 2022-06-17T12:55:45.840 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9ad61d700 1 -- 172.21.15.107:0/973903829 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a0006110 con 0x55d4df9b3250 2022-06-17T12:55:45.841 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9ad61d700 1 -- 172.21.15.107:0/973903829 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9a0004540 con 0x55d4df9b3250 2022-06-17T12:55:45.841 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9b11e4b80 1 -- 172.21.15.107:0/973903829 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d4df9b3250 msgr2=0x55d4df9a6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:45.842 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9b11e4b80 1 --2- 172.21.15.107:0/973903829 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d4df9b3250 0x55d4df9a6910 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x55d4df9a8fb0 tx=0x7ff9a0006af0).stop 2022-06-17T12:55:45.843 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9b11e4b80 1 -- 172.21.15.107:0/973903829 shutdown_connections 2022-06-17T12:55:45.843 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9b11e4b80 1 --2- 172.21.15.107:0/973903829 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d4df9b3250 0x55d4df9a6910 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:45.843 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9b11e4b80 1 --2- 172.21.15.107:0/973903829 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x55d4df9b0910 0x55d4df9a6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:45.844 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9b11e4b80 1 --2- 172.21.15.107:0/973903829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4df9addf0 0x55d4df9a5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:45.844 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9b11e4b80 1 -- 172.21.15.107:0/973903829 >> 172.21.15.107:0/973903829 conn(0x55d4df91b110 msgr2=0x55d4df9b56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:45.844 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9b11e4b80 1 -- 172.21.15.107:0/973903829 shutdown_connections 2022-06-17T12:55:45.845 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9b11e4b80 1 -- 172.21.15.107:0/973903829 wait complete. 2022-06-17T12:55:45.845 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9b11e4b80 1 Processor -- start 2022-06-17T12:55:45.849 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9b11e4b80 1 -- start start 2022-06-17T12:55:45.850 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9b11e4b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4df9addf0 0x55d4df9acf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:45.850 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9b11e4b80 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d4df9b0910 0x55d4df9bba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:45.850 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9b11e4b80 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x55d4df9b3250 0x55d4df9bbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:45.851 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9b11e4b80 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4df9b07d0 con 0x55d4df9b3250 2022-06-17T12:55:45.851 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9b11e4b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4df9bc4f0 con 0x55d4df9addf0 2022-06-17T12:55:45.852 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9b11e4b80 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4df9bc6b0 con 0x55d4df9b0910 2022-06-17T12:55:45.852 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9aee20700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d4df9b0910 0x55d4df9bba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:45.852 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9af621700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4df9addf0 0x55d4df9acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:45.853 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9aee20700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d4df9b0910 0x55d4df9bba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.107:39960/0 (socket says 172.21.15.107:39960) 2022-06-17T12:55:45.853 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9aee20700 1 -- 172.21.15.107:0/4123269611 learned_addr learned my addr 172.21.15.107:0/4123269611 (peer_addr_for_me v2:172.21.15.107:0/0) 2022-06-17T12:55:45.854 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9afe22700 1 --2- 172.21.15.107:0/4123269611 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x55d4df9b3250 0x55d4df9bbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:45.854 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9aee20700 1 -- 172.21.15.107:0/4123269611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4df9addf0 msgr2=0x55d4df9acf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:45.854 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9aee20700 1 --2- 172.21.15.107:0/4123269611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d4df9addf0 0x55d4df9acf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:45.855 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9aee20700 1 -- 172.21.15.107:0/4123269611 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x55d4df9b3250 msgr2=0x55d4df9bbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:45.855 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9aee20700 1 --2- 172.21.15.107:0/4123269611 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x55d4df9b3250 0x55d4df9bbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:45.856 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9aee20700 1 -- 172.21.15.107:0/4123269611 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4df8bf720 con 0x55d4df9b0910 2022-06-17T12:55:45.856 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9aee20700 1 --2- 172.21.15.107:0/4123269611 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d4df9b0910 0x55d4df9bba20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7ff99801a000 tx=0x7ff998013000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:45.856 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9ace1c700 1 -- 172.21.15.107:0/4123269611 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff99801e620 con 0x55d4df9b0910 2022-06-17T12:55:45.857 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9ace1c700 1 -- 172.21.15.107:0/4123269611 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99801e780 con 0x55d4df9b0910 2022-06-17T12:55:45.859 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9b11e4b80 1 -- 172.21.15.107:0/4123269611 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d4df88ffe0 con 0x55d4df9b0910 2022-06-17T12:55:45.860 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9ace1c700 1 -- 172.21.15.107:0/4123269611 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff99801eae0 con 0x55d4df9b0910 2022-06-17T12:55:45.860 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.834+0000 7ff9b11e4b80 1 -- 172.21.15.107:0/4123269611 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d4df9c4b90 con 0x55d4df9b0910 2022-06-17T12:55:45.860 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.838+0000 7ff9ace1c700 1 -- 172.21.15.107:0/4123269611 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff998024030 con 0x55d4df9b0910 2022-06-17T12:55:45.861 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.838+0000 7ff9ace1c700 1 --2- 172.21.15.107:0/4123269611 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff9900525c0 0x7ff990054a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:45.861 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.838+0000 7ff9ace1c700 1 -- 172.21.15.107:0/4123269611 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6422+0+0 (secure 0 0 0) 0x7ff99806cda0 con 0x55d4df9b0910 2022-06-17T12:55:45.862 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.838+0000 7ff9af621700 1 --2- 172.21.15.107:0/4123269611 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff9900525c0 0x7ff990054a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:45.862 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.838+0000 7ff9b11e4b80 1 --2- 172.21.15.107:0/4123269611 >> [v2:172.21.15.36:6802/22619,v1:172.21.15.36:6803/22619] conn(0x55d4df9c5870 0x55d4df9c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:45.862 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.838+0000 7ff9b11e4b80 1 -- 172.21.15.107:0/4123269611 --> [v2:172.21.15.36:6802/22619,v1:172.21.15.36:6803/22619] -- osd_op(unknown.0.0:1 23.0 23:0620e6f9:::key:head [writefull 0~617 in=617b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55d4df9c8190 con 0x55d4df9c5870 2022-06-17T12:55:45.863 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.838+0000 7ff9afe22700 1 --2- 172.21.15.107:0/4123269611 >> [v2:172.21.15.36:6802/22619,v1:172.21.15.36:6803/22619] conn(0x55d4df9c5870 0x55d4df9c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:45.863 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.838+0000 7ff9af621700 1 --2- 172.21.15.107:0/4123269611 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff9900525c0 0x7ff990054a80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7ff9a8008000 tx=0x7ff9a8000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:45.864 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:45.838+0000 7ff9afe22700 1 --2- 172.21.15.107:0/4123269611 >> [v2:172.21.15.36:6802/22619,v1:172.21.15.36:6803/22619] conn(0x55d4df9c5870 0x55d4df9c7c60 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:46.259 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:55:46.271 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:55:46.274 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:55:46.278 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:55:46.281 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:55:46.767 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:46.781+0000 7ff9ace1c700 1 -- 172.21.15.107:0/4123269611 <== osd.0 v2:172.21.15.36:6802/22619 1 ==== osd_map(118..118 src has 1..118) v4 ==== 320+0+0 (crc 0 0 0) 0x7ff9a001b040 con 0x55d4df9c5870 2022-06-17T12:55:46.774 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:46.785+0000 7ff9afe22700 1 -- 172.21.15.107:0/4123269611 <== osd.0 v2:172.21.15.36:6802/22619 2 ==== osd_op_reply(1 key [writefull 0~617] v118'1 uv1 ondisk = 0) v8 ==== 147+0+0 (crc 0 0 0) 0x7ff9a00044f0 con 0x55d4df9c5870 2022-06-17T12:55:46.775 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:46.785+0000 7ff9b11e4b80 1 -- 172.21.15.107:0/4123269611 >> [v2:172.21.15.36:6802/22619,v1:172.21.15.36:6803/22619] conn(0x55d4df9c5870 msgr2=0x55d4df9c7c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:46.775 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:46.785+0000 7ff9b11e4b80 1 --2- 172.21.15.107:0/4123269611 >> [v2:172.21.15.36:6802/22619,v1:172.21.15.36:6803/22619] conn(0x55d4df9c5870 0x55d4df9c7c60 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:46.776 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:46.785+0000 7ff9b11e4b80 1 -- 172.21.15.107:0/4123269611 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff9900525c0 msgr2=0x7ff990054a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:46.776 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:46.785+0000 7ff9b11e4b80 1 --2- 172.21.15.107:0/4123269611 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff9900525c0 0x7ff990054a80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7ff9a8008000 tx=0x7ff9a8000bc0).stop 2022-06-17T12:55:46.776 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:46.785+0000 7ff9b11e4b80 1 -- 172.21.15.107:0/4123269611 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d4df9b0910 msgr2=0x55d4df9bba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:46.776 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:46.785+0000 7ff9b11e4b80 1 --2- 172.21.15.107:0/4123269611 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d4df9b0910 0x55d4df9bba20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7ff99801a000 tx=0x7ff998013000).stop 2022-06-17T12:55:46.777 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:46.785+0000 7ff9afe22700 1 -- 172.21.15.107:0/4123269611 reap_dead start 2022-06-17T12:55:46.777 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:46.785+0000 7ff9b11e4b80 1 -- 172.21.15.107:0/4123269611 shutdown_connections 2022-06-17T12:55:46.777 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:46.785+0000 7ff9b11e4b80 1 -- 172.21.15.107:0/4123269611 >> 172.21.15.107:0/4123269611 conn(0x55d4df91b110 msgr2=0x55d4df9b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:46.777 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:46.785+0000 7ff9b11e4b80 1 -- 172.21.15.107:0/4123269611 shutdown_connections 2022-06-17T12:55:46.778 INFO:teuthology.orchestra.run.smithi107.stderr:2022-06-17T12:55:46.785+0000 7ff9b11e4b80 1 -- 172.21.15.107:0/4123269611 wait complete. 2022-06-17T12:55:46.785 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2022-06-17T12:55:46.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.903+0000 7fead02e4700 1 Processor -- start 2022-06-17T12:55:46.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.903+0000 7fead02e4700 1 -- start start 2022-06-17T12:55:46.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7fead02e4700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feac80b2c50 0x7feac80aee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:46.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7fead02e4700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7feac8146ab0 0x7feac8146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:46.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7fead02e4700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac80b3030 con 0x7feac80b0610 2022-06-17T12:55:46.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7fead02e4700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac8060db0 con 0x7feac8146ab0 2022-06-17T12:55:46.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7fead02e4700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac80b14e0 con 0x7feac80b2c50 2022-06-17T12:55:46.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7feace881700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7feac8146ab0 0x7feac8146ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:46.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7feace881700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7feac8146ab0 0x7feac8146ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38790/0 (socket says 172.21.15.36:38790) 2022-06-17T12:55:46.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7feace881700 1 -- 172.21.15.36:0/987369841 learned_addr learned my addr 172.21.15.36:0/987369841 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:46.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7feace080700 1 --2- 172.21.15.36:0/987369841 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feac80b2c50 0x7feac80aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:46.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7feace881700 1 -- 172.21.15.36:0/987369841 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feac80b2c50 msgr2=0x7feac80aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:46.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7feace881700 1 --2- 172.21.15.36:0/987369841 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feac80b2c50 0x7feac80aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:46.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7feace881700 1 -- 172.21.15.36:0/987369841 >> v1:172.21.15.70:6789/0 conn(0x7feac80b0610 legacy=0x7feac80b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:46.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7feace881700 1 -- 172.21.15.36:0/987369841 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac80b1780 con 0x7feac8146ab0 2022-06-17T12:55:46.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7feace881700 1 --2- 172.21.15.36:0/987369841 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feac8146ab0 0x7feac8146ef0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7feac00032f0 tx=0x7feac0009290).ready entity=mon.0 client_cookie=43f763c7bd933a7b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:46.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7feacd07e700 1 -- 172.21.15.36:0/987369841 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feac000b8a0 con 0x7feac8146ab0 2022-06-17T12:55:46.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7feacd07e700 1 -- 172.21.15.36:0/987369841 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac000ba00 con 0x7feac8146ab0 2022-06-17T12:55:46.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7feacd07e700 1 -- 172.21.15.36:0/987369841 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feac000bb70 con 0x7feac8146ab0 2022-06-17T12:55:46.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7fead02e4700 1 -- 172.21.15.36:0/987369841 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feac8146ab0 msgr2=0x7feac8146ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:46.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7fead02e4700 1 --2- 172.21.15.36:0/987369841 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feac8146ab0 0x7feac8146ef0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7feac00032f0 tx=0x7feac0009290).stop 2022-06-17T12:55:46.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7fead02e4700 1 -- 172.21.15.36:0/987369841 shutdown_connections 2022-06-17T12:55:46.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7fead02e4700 1 --2- 172.21.15.36:0/987369841 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feac8146ab0 0x7feac8146ef0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:46.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7fead02e4700 1 --2- 172.21.15.36:0/987369841 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feac80b2c50 0x7feac80aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:46.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7fead02e4700 1 -- 172.21.15.36:0/987369841 >> 172.21.15.36:0/987369841 conn(0x7feac8062e80 msgr2=0x7feac8063260 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:46.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7fead02e4700 1 -- 172.21.15.36:0/987369841 shutdown_connections 2022-06-17T12:55:46.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.907+0000 7fead02e4700 1 -- 172.21.15.36:0/987369841 wait complete. 2022-06-17T12:55:46.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.911+0000 7fead02e4700 1 Processor -- start 2022-06-17T12:55:46.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.911+0000 7fead02e4700 1 -- start start 2022-06-17T12:55:46.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.911+0000 7fead02e4700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feac80b0610 0x7feac8148f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:46.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.911+0000 7fead02e4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feac80b2c50 0x7feac8152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:46.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.911+0000 7fead02e4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feac8146ab0 0x7feac80aa580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:46.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.911+0000 7fead02e4700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac8152a10 con 0x7feac8146ab0 2022-06-17T12:55:46.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.911+0000 7fead02e4700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac80acba0 con 0x7feac80b2c50 2022-06-17T12:55:46.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.911+0000 7fead02e4700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac80aa0e0 con 0x7feac80b0610 2022-06-17T12:55:46.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.911+0000 7feace881700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feac8146ab0 0x7feac80aa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:46.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.911+0000 7feace881700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feac8146ab0 0x7feac80aa580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38796/0 (socket says 172.21.15.36:38796) 2022-06-17T12:55:46.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.911+0000 7feace881700 1 -- 172.21.15.36:0/549732665 learned_addr learned my addr 172.21.15.36:0/549732665 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:46.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.911+0000 7feacd87f700 1 --2- 172.21.15.36:0/549732665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feac80b2c50 0x7feac8152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:46.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.911+0000 7feace080700 1 --2- 172.21.15.36:0/549732665 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feac80b0610 0x7feac8148f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:46.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.911+0000 7feace881700 1 -- 172.21.15.36:0/549732665 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feac80b0610 msgr2=0x7feac8148f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:46.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.911+0000 7feace881700 1 --2- 172.21.15.36:0/549732665 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feac80b0610 0x7feac8148f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:46.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.911+0000 7feace881700 1 -- 172.21.15.36:0/549732665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feac80b2c50 msgr2=0x7feac8152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:46.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.911+0000 7feace881700 1 --2- 172.21.15.36:0/549732665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feac80b2c50 0x7feac8152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:46.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.911+0000 7feace881700 1 -- 172.21.15.36:0/549732665 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac0009530 con 0x7feac8146ab0 2022-06-17T12:55:46.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.911+0000 7feacd87f700 1 --2- 172.21.15.36:0/549732665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feac80b2c50 0x7feac8152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:46.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.911+0000 7feace881700 1 --2- 172.21.15.36:0/549732665 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feac8146ab0 0x7feac80aa580 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7feac00032f0 tx=0x7feac0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:46.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.915+0000 7feab77fe700 1 -- 172.21.15.36:0/549732665 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feac0021440 con 0x7feac8146ab0 2022-06-17T12:55:46.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.915+0000 7feab77fe700 1 -- 172.21.15.36:0/549732665 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac00215a0 con 0x7feac8146ab0 2022-06-17T12:55:46.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.915+0000 7fead02e4700 1 -- 172.21.15.36:0/549732665 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac80ab8c0 con 0x7feac8146ab0 2022-06-17T12:55:46.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.915+0000 7feab77fe700 1 -- 172.21.15.36:0/549732665 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feac0021900 con 0x7feac8146ab0 2022-06-17T12:55:46.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.915+0000 7fead02e4700 1 -- 172.21.15.36:0/549732665 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac8064ac0 con 0x7feac8146ab0 2022-06-17T12:55:46.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.915+0000 7fead02e4700 1 -- 172.21.15.36:0/549732665 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac80aa2a0 con 0x7feac8146ab0 2022-06-17T12:55:46.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.915+0000 7feab77fe700 1 -- 172.21.15.36:0/549732665 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7feac0028070 con 0x7feac8146ab0 2022-06-17T12:55:46.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.915+0000 7feab77fe700 1 --2- 172.21.15.36:0/549732665 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7feaa8053ad0 0x7feaa8055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:46.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.915+0000 7feab77fe700 1 -- 172.21.15.36:0/549732665 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6422+0+0 (secure 0 0 0) 0x7feac006d4d0 con 0x7feac8146ab0 2022-06-17T12:55:46.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.915+0000 7feace080700 1 --2- 172.21.15.36:0/549732665 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7feaa8053ad0 0x7feaa8055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:46.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.919+0000 7feace080700 1 --2- 172.21.15.36:0/549732665 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7feaa8053ad0 0x7feaa8055f90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7feac400c000 tx=0x7feac4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:46.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:46.923+0000 7feab77fe700 1 -- 172.21.15.36:0/549732665 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7feac0044230 con 0x7feac8146ab0 2022-06-17T12:55:47.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:47.095+0000 7fead02e4700 1 -- 172.21.15.36:0/549732665 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7feac80b24d0 con 0x7feac8146ab0 2022-06-17T12:55:47.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:47.095+0000 7feab77fe700 1 -- 172.21.15.36:0/549732665 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1588 (secure 0 0 0) 0x7feac0015070 con 0x7feac8146ab0 2022-06-17T12:55:47.083 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:47.084 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:55:47.084 INFO:teuthology.orchestra.run.smithi036.stdout: "stats": { 2022-06-17T12:55:47.084 INFO:teuthology.orchestra.run.smithi036.stdout: "total_bytes": 773094113280, 2022-06-17T12:55:47.084 INFO:teuthology.orchestra.run.smithi036.stdout: "total_avail_bytes": 764492603392, 2022-06-17T12:55:47.085 INFO:teuthology.orchestra.run.smithi036.stdout: "total_used_bytes": 11575296, 2022-06-17T12:55:47.085 INFO:teuthology.orchestra.run.smithi036.stdout: "total_used_raw_bytes": 8601509888, 2022-06-17T12:55:47.085 INFO:teuthology.orchestra.run.smithi036.stdout: "total_used_raw_ratio": 0.011126084253191948, 2022-06-17T12:55:47.085 INFO:teuthology.orchestra.run.smithi036.stdout: "num_osds": 8, 2022-06-17T12:55:47.086 INFO:teuthology.orchestra.run.smithi036.stdout: "num_per_pool_osds": 8, 2022-06-17T12:55:47.086 INFO:teuthology.orchestra.run.smithi036.stdout: "num_per_pool_omap_osds": 8 2022-06-17T12:55:47.086 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:47.086 INFO:teuthology.orchestra.run.smithi036.stdout: "stats_by_class": { 2022-06-17T12:55:47.087 INFO:teuthology.orchestra.run.smithi036.stdout: "ssd": { 2022-06-17T12:55:47.087 INFO:teuthology.orchestra.run.smithi036.stdout: "total_bytes": 773094113280, 2022-06-17T12:55:47.088 INFO:teuthology.orchestra.run.smithi036.stdout: "total_avail_bytes": 764492603392, 2022-06-17T12:55:47.089 INFO:teuthology.orchestra.run.smithi036.stdout: "total_used_bytes": 11575296, 2022-06-17T12:55:47.089 INFO:teuthology.orchestra.run.smithi036.stdout: "total_used_raw_bytes": 8601509888, 2022-06-17T12:55:47.090 INFO:teuthology.orchestra.run.smithi036.stdout: "total_used_raw_ratio": 0.011126084253191948 2022-06-17T12:55:47.090 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:47.090 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:47.091 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:55:47.091 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:47.092 INFO:teuthology.orchestra.run.smithi036.stdout: "name": "device_health_metrics", 2022-06-17T12:55:47.092 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 1, 2022-06-17T12:55:47.092 INFO:teuthology.orchestra.run.smithi036.stdout: "stats": { 2022-06-17T12:55:47.093 INFO:teuthology.orchestra.run.smithi036.stdout: "stored": 0, 2022-06-17T12:55:47.093 INFO:teuthology.orchestra.run.smithi036.stdout: "objects": 0, 2022-06-17T12:55:47.093 INFO:teuthology.orchestra.run.smithi036.stdout: "kb_used": 0, 2022-06-17T12:55:47.094 INFO:teuthology.orchestra.run.smithi036.stdout: "bytes_used": 0, 2022-06-17T12:55:47.094 INFO:teuthology.orchestra.run.smithi036.stdout: "percent_used": 0, 2022-06-17T12:55:47.095 INFO:teuthology.orchestra.run.smithi036.stdout: "max_avail": 362918805504 2022-06-17T12:55:47.095 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:47.095 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:47.096 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:47.096 INFO:teuthology.orchestra.run.smithi036.stdout: "name": "rbd", 2022-06-17T12:55:47.097 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 2, 2022-06-17T12:55:47.097 INFO:teuthology.orchestra.run.smithi036.stdout: "stats": { 2022-06-17T12:55:47.102 INFO:teuthology.orchestra.run.smithi036.stdout: "stored": 0, 2022-06-17T12:55:47.103 INFO:teuthology.orchestra.run.smithi036.stdout: "objects": 0, 2022-06-17T12:55:47.103 INFO:teuthology.orchestra.run.smithi036.stdout: "kb_used": 0, 2022-06-17T12:55:47.104 INFO:teuthology.orchestra.run.smithi036.stdout: "bytes_used": 0, 2022-06-17T12:55:47.104 INFO:teuthology.orchestra.run.smithi036.stdout: "percent_used": 0, 2022-06-17T12:55:47.104 INFO:teuthology.orchestra.run.smithi036.stdout: "max_avail": 362918805504 2022-06-17T12:55:47.104 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:47.104 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:47.105 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:47.105 INFO:teuthology.orchestra.run.smithi036.stdout: "name": "cephfs_data", 2022-06-17T12:55:47.105 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 22, 2022-06-17T12:55:47.105 INFO:teuthology.orchestra.run.smithi036.stdout: "stats": { 2022-06-17T12:55:47.106 INFO:teuthology.orchestra.run.smithi036.stdout: "stored": 0, 2022-06-17T12:55:47.106 INFO:teuthology.orchestra.run.smithi036.stdout: "objects": 0, 2022-06-17T12:55:47.106 INFO:teuthology.orchestra.run.smithi036.stdout: "kb_used": 0, 2022-06-17T12:55:47.106 INFO:teuthology.orchestra.run.smithi036.stdout: "bytes_used": 0, 2022-06-17T12:55:47.106 INFO:teuthology.orchestra.run.smithi036.stdout: "percent_used": 0, 2022-06-17T12:55:47.107 INFO:teuthology.orchestra.run.smithi036.stdout: "max_avail": 362918805504 2022-06-17T12:55:47.107 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:47.108 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:47.108 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:47.108 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:55:47.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:47.095+0000 7fead02e4700 1 -- 172.21.15.36:0/549732665 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7feaa8053ad0 msgr2=0x7feaa8055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:47.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:47.095+0000 7fead02e4700 1 --2- 172.21.15.36:0/549732665 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7feaa8053ad0 0x7feaa8055f90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7feac400c000 tx=0x7feac4008040).stop 2022-06-17T12:55:47.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:47.095+0000 7fead02e4700 1 -- 172.21.15.36:0/549732665 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feac8146ab0 msgr2=0x7feac80aa580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:47.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:47.099+0000 7fead02e4700 1 --2- 172.21.15.36:0/549732665 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feac8146ab0 0x7feac80aa580 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7feac00032f0 tx=0x7feac0016000).stop 2022-06-17T12:55:47.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:47.099+0000 7fead02e4700 1 -- 172.21.15.36:0/549732665 shutdown_connections 2022-06-17T12:55:47.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:47.099+0000 7fead02e4700 1 --2- 172.21.15.36:0/549732665 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7feaa8053ad0 0x7feaa8055f90 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:47.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:47.099+0000 7fead02e4700 1 --2- 172.21.15.36:0/549732665 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feac8146ab0 0x7feac80aa580 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:47.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:47.099+0000 7fead02e4700 1 --2- 172.21.15.36:0/549732665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feac80b2c50 0x7feac8152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:47.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:47.099+0000 7fead02e4700 1 --2- 172.21.15.36:0/549732665 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feac80b0610 0x7feac8148f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:47.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:47.099+0000 7fead02e4700 1 -- 172.21.15.36:0/549732665 >> 172.21.15.36:0/549732665 conn(0x7feac8062e80 msgr2=0x7feac80b4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:47.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:47.099+0000 7fead02e4700 1 -- 172.21.15.36:0/549732665 shutdown_connections 2022-06-17T12:55:47.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:47.099+0000 7fead02e4700 1 -- 172.21.15.36:0/549732665 wait complete. 2022-06-17T12:55:47.116 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30)... 2022-06-17T12:55:52.117 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2022-06-17T12:55:52.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.239+0000 7fbb38ad2700 1 Processor -- start 2022-06-17T12:55:52.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.239+0000 7fbb38ad2700 1 -- start start 2022-06-17T12:55:52.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb38ad2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb340605f0 0x7fbb341436e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:52.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb38ad2700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbb34066df0 0x7fbb340ace80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:52.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb38ad2700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb340af780 con 0x7fbb340b17e0 2022-06-17T12:55:52.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb38ad2700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb34067280 con 0x7fbb34066df0 2022-06-17T12:55:52.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb38ad2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb340af240 con 0x7fbb340605f0 2022-06-17T12:55:52.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb32d9d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbb34066df0 0x7fbb340ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:52.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb3259c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb340605f0 0x7fbb341436e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:52.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb3259c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb340605f0 0x7fbb341436e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:53590/0 (socket says 172.21.15.36:53590) 2022-06-17T12:55:52.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb3259c700 1 -- 172.21.15.36:0/294568568 learned_addr learned my addr 172.21.15.36:0/294568568 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:52.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb32d9d700 1 -- 172.21.15.36:0/294568568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb340605f0 msgr2=0x7fbb341436e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:52.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb32d9d700 1 --2- 172.21.15.36:0/294568568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb340605f0 0x7fbb341436e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:52.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb32d9d700 1 -- 172.21.15.36:0/294568568 >> v1:172.21.15.70:6789/0 conn(0x7fbb340b17e0 legacy=0x7fbb340666e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:52.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb32d9d700 1 -- 172.21.15.36:0/294568568 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb340af4e0 con 0x7fbb34066df0 2022-06-17T12:55:52.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb3259c700 1 --2- 172.21.15.36:0/294568568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb340605f0 0x7fbb341436e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:52.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb32d9d700 1 --2- 172.21.15.36:0/294568568 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb34066df0 0x7fbb340ace80 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0x7fbb2c0032f0 tx=0x7fbb2c0091d0).ready entity=mon.0 client_cookie=e69cd689eb226094 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:52.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb31d9b700 1 -- 172.21.15.36:0/294568568 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb2c00b6c0 con 0x7fbb34066df0 2022-06-17T12:55:52.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb31d9b700 1 -- 172.21.15.36:0/294568568 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2c00b820 con 0x7fbb34066df0 2022-06-17T12:55:52.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb31d9b700 1 -- 172.21.15.36:0/294568568 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb2c00b990 con 0x7fbb34066df0 2022-06-17T12:55:52.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb38ad2700 1 -- 172.21.15.36:0/294568568 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb34066df0 msgr2=0x7fbb340ace80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:52.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb38ad2700 1 --2- 172.21.15.36:0/294568568 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb34066df0 0x7fbb340ace80 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0x7fbb2c0032f0 tx=0x7fbb2c0091d0).stop 2022-06-17T12:55:52.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb38ad2700 1 -- 172.21.15.36:0/294568568 shutdown_connections 2022-06-17T12:55:52.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb38ad2700 1 --2- 172.21.15.36:0/294568568 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb34066df0 0x7fbb340ace80 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:52.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb38ad2700 1 --2- 172.21.15.36:0/294568568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb340605f0 0x7fbb341436e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:52.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb38ad2700 1 -- 172.21.15.36:0/294568568 >> 172.21.15.36:0/294568568 conn(0x7fbb3405c550 msgr2=0x7fbb340b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:52.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb38ad2700 1 -- 172.21.15.36:0/294568568 shutdown_connections 2022-06-17T12:55:52.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.243+0000 7fbb38ad2700 1 -- 172.21.15.36:0/294568568 wait complete. 2022-06-17T12:55:52.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.247+0000 7fbb38ad2700 1 Processor -- start 2022-06-17T12:55:52.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.247+0000 7fbb38ad2700 1 -- start start 2022-06-17T12:55:52.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.247+0000 7fbb38ad2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb340605f0 0x7fbb3409ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:52.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.247+0000 7fbb38ad2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb34066df0 0x7fbb340644e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:52.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.247+0000 7fbb38ad2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb340b17e0 0x7fbb340649d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:52.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.247+0000 7fbb38ad2700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb340653e0 con 0x7fbb340605f0 2022-06-17T12:55:52.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.247+0000 7fbb38ad2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb34066080 con 0x7fbb34066df0 2022-06-17T12:55:52.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.247+0000 7fbb38ad2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb34066320 con 0x7fbb340b17e0 2022-06-17T12:55:52.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.247+0000 7fbb3259c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb340605f0 0x7fbb3409ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:52.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.247+0000 7fbb3259c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb340605f0 0x7fbb3409ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38806/0 (socket says 172.21.15.36:38806) 2022-06-17T12:55:52.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.247+0000 7fbb3259c700 1 -- 172.21.15.36:0/2049006383 learned_addr learned my addr 172.21.15.36:0/2049006383 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:52.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.247+0000 7fbb32d9d700 1 --2- 172.21.15.36:0/2049006383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb340b17e0 0x7fbb340649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:52.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.247+0000 7fbb2bfff700 1 --2- 172.21.15.36:0/2049006383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb34066df0 0x7fbb340644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:52.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.247+0000 7fbb32d9d700 1 -- 172.21.15.36:0/2049006383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb34066df0 msgr2=0x7fbb340644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:52.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.247+0000 7fbb32d9d700 1 --2- 172.21.15.36:0/2049006383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb34066df0 0x7fbb340644e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:52.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.247+0000 7fbb32d9d700 1 -- 172.21.15.36:0/2049006383 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb340605f0 msgr2=0x7fbb3409ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:52.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.247+0000 7fbb32d9d700 1 --2- 172.21.15.36:0/2049006383 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb340605f0 0x7fbb3409ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:52.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.247+0000 7fbb32d9d700 1 -- 172.21.15.36:0/2049006383 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2400d4d0 con 0x7fbb340b17e0 2022-06-17T12:55:52.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.247+0000 7fbb3259c700 1 --2- 172.21.15.36:0/2049006383 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb340605f0 0x7fbb3409ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:52.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.247+0000 7fbb32d9d700 1 --2- 172.21.15.36:0/2049006383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb340b17e0 0x7fbb340649d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fbb2c0032f0 tx=0x7fbb2c016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:52.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.247+0000 7fbb2b7fe700 1 -- 172.21.15.36:0/2049006383 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb2c011d70 con 0x7fbb340b17e0 2022-06-17T12:55:52.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.247+0000 7fbb2b7fe700 1 -- 172.21.15.36:0/2049006383 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2c021440 con 0x7fbb340b17e0 2022-06-17T12:55:52.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.247+0000 7fbb38ad2700 1 -- 172.21.15.36:0/2049006383 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb2c009470 con 0x7fbb340b17e0 2022-06-17T12:55:52.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.247+0000 7fbb2b7fe700 1 -- 172.21.15.36:0/2049006383 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb2c0217a0 con 0x7fbb340b17e0 2022-06-17T12:55:52.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.251+0000 7fbb38ad2700 1 -- 172.21.15.36:0/2049006383 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3409f170 con 0x7fbb340b17e0 2022-06-17T12:55:52.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.251+0000 7fbb38ad2700 1 -- 172.21.15.36:0/2049006383 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb34057410 con 0x7fbb340b17e0 2022-06-17T12:55:52.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.251+0000 7fbb2b7fe700 1 -- 172.21.15.36:0/2049006383 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fbb2c029070 con 0x7fbb340b17e0 2022-06-17T12:55:52.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.251+0000 7fbb2b7fe700 1 --2- 172.21.15.36:0/2049006383 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb18053920 0x7fbb18055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:52.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.251+0000 7fbb2b7fe700 1 -- 172.21.15.36:0/2049006383 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6422+0+0 (secure 0 0 0) 0x7fbb2c06d0a0 con 0x7fbb340b17e0 2022-06-17T12:55:52.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.251+0000 7fbb3259c700 1 --2- 172.21.15.36:0/2049006383 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb18053920 0x7fbb18055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:52.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.251+0000 7fbb3259c700 1 --2- 172.21.15.36:0/2049006383 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb18053920 0x7fbb18055de0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fbb24008000 tx=0x7fbb2400f040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:52.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.263+0000 7fbb2b7fe700 1 -- 172.21.15.36:0/2049006383 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbb2c043cf0 con 0x7fbb340b17e0 2022-06-17T12:55:52.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.431+0000 7fbb38ad2700 1 -- 172.21.15.36:0/2049006383 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7fbb3405f340 con 0x7fbb340b17e0 2022-06-17T12:55:52.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.435+0000 7fbb2b7fe700 1 -- 172.21.15.36:0/2049006383 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1932 (secure 0 0 0) 0x7fbb2c015070 con 0x7fbb340b17e0 2022-06-17T12:55:52.423 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:52.423 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:55:52.424 INFO:teuthology.orchestra.run.smithi036.stdout: "stats": { 2022-06-17T12:55:52.424 INFO:teuthology.orchestra.run.smithi036.stdout: "total_bytes": 773094113280, 2022-06-17T12:55:52.425 INFO:teuthology.orchestra.run.smithi036.stdout: "total_avail_bytes": 764492226560, 2022-06-17T12:55:52.425 INFO:teuthology.orchestra.run.smithi036.stdout: "total_used_bytes": 11952128, 2022-06-17T12:55:52.425 INFO:teuthology.orchestra.run.smithi036.stdout: "total_used_raw_bytes": 8601886720, 2022-06-17T12:55:52.426 INFO:teuthology.orchestra.run.smithi036.stdout: "total_used_raw_ratio": 0.011126571334898472, 2022-06-17T12:55:52.426 INFO:teuthology.orchestra.run.smithi036.stdout: "num_osds": 8, 2022-06-17T12:55:52.427 INFO:teuthology.orchestra.run.smithi036.stdout: "num_per_pool_osds": 8, 2022-06-17T12:55:52.427 INFO:teuthology.orchestra.run.smithi036.stdout: "num_per_pool_omap_osds": 8 2022-06-17T12:55:52.428 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:52.428 INFO:teuthology.orchestra.run.smithi036.stdout: "stats_by_class": { 2022-06-17T12:55:52.428 INFO:teuthology.orchestra.run.smithi036.stdout: "ssd": { 2022-06-17T12:55:52.429 INFO:teuthology.orchestra.run.smithi036.stdout: "total_bytes": 773094113280, 2022-06-17T12:55:52.429 INFO:teuthology.orchestra.run.smithi036.stdout: "total_avail_bytes": 764492226560, 2022-06-17T12:55:52.430 INFO:teuthology.orchestra.run.smithi036.stdout: "total_used_bytes": 11952128, 2022-06-17T12:55:52.430 INFO:teuthology.orchestra.run.smithi036.stdout: "total_used_raw_bytes": 8601886720, 2022-06-17T12:55:52.430 INFO:teuthology.orchestra.run.smithi036.stdout: "total_used_raw_ratio": 0.011126571334898472 2022-06-17T12:55:52.431 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:52.431 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:52.431 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:55:52.432 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:52.432 INFO:teuthology.orchestra.run.smithi036.stdout: "name": "device_health_metrics", 2022-06-17T12:55:52.432 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 1, 2022-06-17T12:55:52.433 INFO:teuthology.orchestra.run.smithi036.stdout: "stats": { 2022-06-17T12:55:52.433 INFO:teuthology.orchestra.run.smithi036.stdout: "stored": 0, 2022-06-17T12:55:52.434 INFO:teuthology.orchestra.run.smithi036.stdout: "objects": 0, 2022-06-17T12:55:52.434 INFO:teuthology.orchestra.run.smithi036.stdout: "kb_used": 0, 2022-06-17T12:55:52.434 INFO:teuthology.orchestra.run.smithi036.stdout: "bytes_used": 0, 2022-06-17T12:55:52.435 INFO:teuthology.orchestra.run.smithi036.stdout: "percent_used": 0, 2022-06-17T12:55:52.435 INFO:teuthology.orchestra.run.smithi036.stdout: "max_avail": 362918707200 2022-06-17T12:55:52.435 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:52.436 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:52.436 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:52.436 INFO:teuthology.orchestra.run.smithi036.stdout: "name": "rbd", 2022-06-17T12:55:52.437 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 2, 2022-06-17T12:55:52.437 INFO:teuthology.orchestra.run.smithi036.stdout: "stats": { 2022-06-17T12:55:52.437 INFO:teuthology.orchestra.run.smithi036.stdout: "stored": 0, 2022-06-17T12:55:52.437 INFO:teuthology.orchestra.run.smithi036.stdout: "objects": 0, 2022-06-17T12:55:52.438 INFO:teuthology.orchestra.run.smithi036.stdout: "kb_used": 0, 2022-06-17T12:55:52.438 INFO:teuthology.orchestra.run.smithi036.stdout: "bytes_used": 0, 2022-06-17T12:55:52.438 INFO:teuthology.orchestra.run.smithi036.stdout: "percent_used": 0, 2022-06-17T12:55:52.438 INFO:teuthology.orchestra.run.smithi036.stdout: "max_avail": 362918707200 2022-06-17T12:55:52.439 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:52.439 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:52.439 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:52.439 INFO:teuthology.orchestra.run.smithi036.stdout: "name": "cephfs_data", 2022-06-17T12:55:52.440 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 22, 2022-06-17T12:55:52.440 INFO:teuthology.orchestra.run.smithi036.stdout: "stats": { 2022-06-17T12:55:52.440 INFO:teuthology.orchestra.run.smithi036.stdout: "stored": 0, 2022-06-17T12:55:52.440 INFO:teuthology.orchestra.run.smithi036.stdout: "objects": 0, 2022-06-17T12:55:52.440 INFO:teuthology.orchestra.run.smithi036.stdout: "kb_used": 0, 2022-06-17T12:55:52.441 INFO:teuthology.orchestra.run.smithi036.stdout: "bytes_used": 0, 2022-06-17T12:55:52.441 INFO:teuthology.orchestra.run.smithi036.stdout: "percent_used": 0, 2022-06-17T12:55:52.441 INFO:teuthology.orchestra.run.smithi036.stdout: "max_avail": 362918707200 2022-06-17T12:55:52.441 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:52.442 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:52.442 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:52.442 INFO:teuthology.orchestra.run.smithi036.stdout: "name": "cephfs_metadata", 2022-06-17T12:55:52.443 INFO:teuthology.orchestra.run.smithi036.stdout: "id": 23, 2022-06-17T12:55:52.443 INFO:teuthology.orchestra.run.smithi036.stdout: "stats": { 2022-06-17T12:55:52.443 INFO:teuthology.orchestra.run.smithi036.stdout: "stored": 308, 2022-06-17T12:55:52.443 INFO:teuthology.orchestra.run.smithi036.stdout: "objects": 1, 2022-06-17T12:55:52.444 INFO:teuthology.orchestra.run.smithi036.stdout: "kb_used": 4, 2022-06-17T12:55:52.444 INFO:teuthology.orchestra.run.smithi036.stdout: "bytes_used": 4096, 2022-06-17T12:55:52.444 INFO:teuthology.orchestra.run.smithi036.stdout: "percent_used": 5.6431370687448634e-09, 2022-06-17T12:55:52.445 INFO:teuthology.orchestra.run.smithi036.stdout: "max_avail": 362918707200 2022-06-17T12:55:52.445 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:52.445 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:52.446 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:52.446 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:55:52.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.435+0000 7fbb38ad2700 1 -- 172.21.15.36:0/2049006383 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb18053920 msgr2=0x7fbb18055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:52.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.435+0000 7fbb38ad2700 1 --2- 172.21.15.36:0/2049006383 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb18053920 0x7fbb18055de0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fbb24008000 tx=0x7fbb2400f040).stop 2022-06-17T12:55:52.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.435+0000 7fbb38ad2700 1 -- 172.21.15.36:0/2049006383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb340b17e0 msgr2=0x7fbb340649d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:52.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.435+0000 7fbb38ad2700 1 --2- 172.21.15.36:0/2049006383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb340b17e0 0x7fbb340649d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fbb2c0032f0 tx=0x7fbb2c016000).stop 2022-06-17T12:55:52.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.435+0000 7fbb38ad2700 1 -- 172.21.15.36:0/2049006383 shutdown_connections 2022-06-17T12:55:52.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.435+0000 7fbb38ad2700 1 --2- 172.21.15.36:0/2049006383 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb18053920 0x7fbb18055de0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:52.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.435+0000 7fbb38ad2700 1 --2- 172.21.15.36:0/2049006383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb340b17e0 0x7fbb340649d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:52.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.435+0000 7fbb38ad2700 1 --2- 172.21.15.36:0/2049006383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb34066df0 0x7fbb340644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:52.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.435+0000 7fbb38ad2700 1 --2- 172.21.15.36:0/2049006383 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb340605f0 0x7fbb3409ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:52.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.435+0000 7fbb38ad2700 1 -- 172.21.15.36:0/2049006383 >> 172.21.15.36:0/2049006383 conn(0x7fbb3405c550 msgr2=0x7fbb340b5360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:52.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.435+0000 7fbb38ad2700 1 -- 172.21.15.36:0/2049006383 shutdown_connections 2022-06-17T12:55:52.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.435+0000 7fbb38ad2700 1 -- 172.21.15.36:0/2049006383 wait complete. 2022-06-17T12:55:52.454 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s 2022-06-17T12:55:52.455 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2022-06-17T12:55:52.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.575+0000 7f87766fe700 1 Processor -- start 2022-06-17T12:55:52.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.575+0000 7f87766fe700 1 -- start start 2022-06-17T12:55:52.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f87766fe700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f87700b27b0 0x7f877013f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:52.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f87766fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87700b0d70 0x7f877013f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:52.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f87766fe700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87700b2b90 con 0x7f87700b15f0 2022-06-17T12:55:52.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f87766fe700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87700610e0 con 0x7f87700b27b0 2022-06-17T12:55:52.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f87766fe700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87700ac190 con 0x7f87700b0d70 2022-06-17T12:55:52.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f8774efb700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87700b0d70 0x7f877013f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:52.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f87756fc700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f87700b27b0 0x7f877013f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:52.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f8774efb700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87700b0d70 0x7f877013f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:53606/0 (socket says 172.21.15.36:53606) 2022-06-17T12:55:52.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f8774efb700 1 -- 172.21.15.36:0/1991835479 learned_addr learned my addr 172.21.15.36:0/1991835479 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:52.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f8775efd700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f87700b15f0 0x7f87700ac780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:52080/0 (socket says 172.21.15.36:52080) 2022-06-17T12:55:52.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f8774efb700 1 -- 172.21.15.36:0/1991835479 >> v2:172.21.15.70:3300/0 conn(0x7f87700b27b0 msgr2=0x7f877013f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:52.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f8774efb700 1 --2- 172.21.15.36:0/1991835479 >> v2:172.21.15.70:3300/0 conn(0x7f87700b27b0 0x7f877013f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:52.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f8774efb700 1 -- 172.21.15.36:0/1991835479 >> v1:172.21.15.70:6789/0 conn(0x7f87700b15f0 legacy=0x7f87700ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:52.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f8774efb700 1 -- 172.21.15.36:0/1991835479 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87700ac430 con 0x7f87700b0d70 2022-06-17T12:55:52.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f8774efb700 1 --2- 172.21.15.36:0/1991835479 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87700b0d70 0x7f877013f600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f8764002f70 tx=0x7f8764006f40).ready entity=mon.2 client_cookie=b49840bc9d25f26c server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:52.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f875bfff700 1 -- 172.21.15.36:0/1991835479 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f876400da80 con 0x7f87700b0d70 2022-06-17T12:55:52.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f875bfff700 1 -- 172.21.15.36:0/1991835479 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8764000e90 con 0x7f87700b0d70 2022-06-17T12:55:52.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f875bfff700 1 -- 172.21.15.36:0/1991835479 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f876400de10 con 0x7f87700b0d70 2022-06-17T12:55:52.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f87766fe700 1 -- 172.21.15.36:0/1991835479 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87700b0d70 msgr2=0x7f877013f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:52.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f87766fe700 1 --2- 172.21.15.36:0/1991835479 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87700b0d70 0x7f877013f600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f8764002f70 tx=0x7f8764006f40).stop 2022-06-17T12:55:52.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f87766fe700 1 -- 172.21.15.36:0/1991835479 shutdown_connections 2022-06-17T12:55:52.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f87766fe700 1 --2- 172.21.15.36:0/1991835479 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87700b0d70 0x7f877013f600 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:52.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f87766fe700 1 --2- 172.21.15.36:0/1991835479 >> v2:172.21.15.70:3300/0 conn(0x7f87700b27b0 0x7f877013f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:52.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f87766fe700 1 -- 172.21.15.36:0/1991835479 >> 172.21.15.36:0/1991835479 conn(0x7f87700b5d10 msgr2=0x7f87700b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:52.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f87766fe700 1 -- 172.21.15.36:0/1991835479 shutdown_connections 2022-06-17T12:55:52.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.579+0000 7f87766fe700 1 -- 172.21.15.36:0/1991835479 wait complete. 2022-06-17T12:55:52.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.583+0000 7f87766fe700 1 Processor -- start 2022-06-17T12:55:52.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.583+0000 7f87766fe700 1 -- start start 2022-06-17T12:55:52.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.583+0000 7f87766fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87700b0d70 0x7f87701513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:52.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.583+0000 7f87766fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87700b15f0 0x7f877012d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:52.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.583+0000 7f87766fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f87700b27b0 0x7f8770062390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:52.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.583+0000 7f87766fe700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87701444d0 con 0x7f87700b27b0 2022-06-17T12:55:52.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.583+0000 7f87766fe700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8770143e00 con 0x7f87700b15f0 2022-06-17T12:55:52.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.583+0000 7f87766fe700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8770143300 con 0x7f87700b0d70 2022-06-17T12:55:52.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.583+0000 7f8775efd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f87700b27b0 0x7f8770062390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:52.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.583+0000 7f87756fc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87700b0d70 0x7f87701513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:52.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.583+0000 7f8775efd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f87700b27b0 0x7f8770062390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38824/0 (socket says 172.21.15.36:38824) 2022-06-17T12:55:52.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.583+0000 7f8775efd700 1 -- 172.21.15.36:0/768976738 learned_addr learned my addr 172.21.15.36:0/768976738 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:52.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.583+0000 7f8774efb700 1 --2- 172.21.15.36:0/768976738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87700b15f0 0x7f877012d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:52.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.583+0000 7f8775efd700 1 -- 172.21.15.36:0/768976738 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87700b0d70 msgr2=0x7f87701513a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:52.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.583+0000 7f8775efd700 1 --2- 172.21.15.36:0/768976738 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87700b0d70 0x7f87701513a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:52.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.583+0000 7f8775efd700 1 -- 172.21.15.36:0/768976738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87700b15f0 msgr2=0x7f877012d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:52.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.583+0000 7f8775efd700 1 --2- 172.21.15.36:0/768976738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87700b15f0 0x7f877012d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:52.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.583+0000 7f8775efd700 1 -- 172.21.15.36:0/768976738 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f876400b460 con 0x7f87700b27b0 2022-06-17T12:55:52.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.583+0000 7f8775efd700 1 --2- 172.21.15.36:0/768976738 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f87700b27b0 0x7f8770062390 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7f8770144610 tx=0x7f8768007f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:52.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.583+0000 7f8762ffd700 1 -- 172.21.15.36:0/768976738 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8768005ae0 con 0x7f87700b27b0 2022-06-17T12:55:52.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.583+0000 7f8762ffd700 1 -- 172.21.15.36:0/768976738 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8768005c40 con 0x7f87700b27b0 2022-06-17T12:55:52.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.583+0000 7f87766fe700 1 -- 172.21.15.36:0/768976738 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87701435a0 con 0x7f87700b27b0 2022-06-17T12:55:52.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.583+0000 7f8762ffd700 1 -- 172.21.15.36:0/768976738 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87680070d0 con 0x7f87700b27b0 2022-06-17T12:55:52.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.583+0000 7f87766fe700 1 -- 172.21.15.36:0/768976738 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f877012dc20 con 0x7f87700b27b0 2022-06-17T12:55:52.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.587+0000 7f87766fe700 1 -- 172.21.15.36:0/768976738 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87701407b0 con 0x7f87700b27b0 2022-06-17T12:55:52.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.587+0000 7f8762ffd700 1 -- 172.21.15.36:0/768976738 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8768010070 con 0x7f87700b27b0 2022-06-17T12:55:52.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.587+0000 7f8762ffd700 1 --2- 172.21.15.36:0/768976738 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f874c053a40 0x7f874c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:52.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.587+0000 7f8762ffd700 1 -- 172.21.15.36:0/768976738 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f876806b860 con 0x7f87700b27b0 2022-06-17T12:55:52.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.587+0000 7f87756fc700 1 --2- 172.21.15.36:0/768976738 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f874c053a40 0x7f874c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:52.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.587+0000 7f87756fc700 1 --2- 172.21.15.36:0/768976738 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f874c053a40 0x7f874c055f00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f876c00b000 tx=0x7f876c006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:52.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.591+0000 7f8762ffd700 1 -- 172.21.15.36:0/768976738 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8768042600 con 0x7f87700b27b0 2022-06-17T12:55:52.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.755+0000 7f87766fe700 1 -- 172.21.15.36:0/768976738 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8770057410 con 0x7f87700b27b0 2022-06-17T12:55:52.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.755+0000 7f8762ffd700 1 -- 172.21.15.36:0/768976738 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=-22 pool 'cephfs_metadata' already contains some objects. Use an empty pool instead. v62) v1 ==== 209+0+0 (secure 0 0 0) 0x7f8768045c20 con 0x7f87700b27b0 2022-06-17T12:55:52.745 INFO:teuthology.orchestra.run.smithi036.stderr:Error EINVAL: pool 'cephfs_metadata' already contains some objects. Use an empty pool instead. 2022-06-17T12:55:52.750 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-06-17T12:55:52.752 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-17T12:55:52.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.871+0000 7f71e083e700 1 Processor -- start 2022-06-17T12:55:52.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.871+0000 7f71e083e700 1 -- start start 2022-06-17T12:55:52.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.871+0000 7f71e083e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f71dc0b7ce0 0x7f71dc0b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:52.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.871+0000 7f71e083e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71dc140c90 0x7f71dc13aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:52.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.871+0000 7f71e083e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71dc142160 con 0x7f71dc13aef0 2022-06-17T12:55:52.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.871+0000 7f71e083e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71dc143d80 con 0x7f71dc0b7ce0 2022-06-17T12:55:52.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.871+0000 7f71e083e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc0b4f30 con 0x7f71dc140c90 2022-06-17T12:55:52.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.871+0000 7f71dad9d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f71dc0b7ce0 0x7f71dc0b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:52.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.871+0000 7f71da59c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71dc140c90 0x7f71dc13aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:52.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.871+0000 7f71dad9d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f71dc0b7ce0 0x7f71dc0b80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:59210/0 (socket says 172.21.15.36:59210) 2022-06-17T12:55:52.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.871+0000 7f71dad9d700 1 -- 172.21.15.36:0/3864911919 learned_addr learned my addr 172.21.15.36:0/3864911919 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:52.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.871+0000 7f71db59e700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f71dc13aef0 0x7f71dc144260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:52096/0 (socket says 172.21.15.36:52096) 2022-06-17T12:55:52.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.871+0000 7f71dad9d700 1 -- 172.21.15.36:0/3864911919 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71dc140c90 msgr2=0x7f71dc13aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:52.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.871+0000 7f71dad9d700 1 --2- 172.21.15.36:0/3864911919 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71dc140c90 0x7f71dc13aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:52.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.871+0000 7f71dad9d700 1 -- 172.21.15.36:0/3864911919 >> v1:172.21.15.70:6789/0 conn(0x7f71dc13aef0 legacy=0x7f71dc144260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:52.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.871+0000 7f71dad9d700 1 -- 172.21.15.36:0/3864911919 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71dc142390 con 0x7f71dc0b7ce0 2022-06-17T12:55:52.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.871+0000 7f71da59c700 1 --2- 172.21.15.36:0/3864911919 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71dc140c90 0x7f71dc13aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:52.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.871+0000 7f71dad9d700 1 --2- 172.21.15.36:0/3864911919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71dc0b7ce0 0x7f71dc0b80c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f71cc002f70 tx=0x7f71cc006eb0).ready entity=mon.1 client_cookie=b392612350080426 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:52.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.871+0000 7f71d9d9b700 1 -- 172.21.15.36:0/3864911919 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71cc00d570 con 0x7f71dc0b7ce0 2022-06-17T12:55:52.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.871+0000 7f71d9d9b700 1 -- 172.21.15.36:0/3864911919 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71cc000e90 con 0x7f71dc0b7ce0 2022-06-17T12:55:52.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.871+0000 7f71d9d9b700 1 -- 172.21.15.36:0/3864911919 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71cc00d900 con 0x7f71dc0b7ce0 2022-06-17T12:55:52.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71e083e700 1 -- 172.21.15.36:0/3864911919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71dc0b7ce0 msgr2=0x7f71dc0b80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:52.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71e083e700 1 --2- 172.21.15.36:0/3864911919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71dc0b7ce0 0x7f71dc0b80c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f71cc002f70 tx=0x7f71cc006eb0).stop 2022-06-17T12:55:52.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71e083e700 1 -- 172.21.15.36:0/3864911919 shutdown_connections 2022-06-17T12:55:52.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71e083e700 1 --2- 172.21.15.36:0/3864911919 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71dc140c90 0x7f71dc13aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:52.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71e083e700 1 --2- 172.21.15.36:0/3864911919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71dc0b7ce0 0x7f71dc0b80c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:52.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71e083e700 1 -- 172.21.15.36:0/3864911919 >> 172.21.15.36:0/3864911919 conn(0x7f71dc05c550 msgr2=0x7f71dc0b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:52.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71e083e700 1 -- 172.21.15.36:0/3864911919 shutdown_connections 2022-06-17T12:55:52.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71e083e700 1 -- 172.21.15.36:0/3864911919 wait complete. 2022-06-17T12:55:52.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71e083e700 1 Processor -- start 2022-06-17T12:55:52.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71e083e700 1 -- start start 2022-06-17T12:55:52.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71e083e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71dc0b7ce0 0x7f71dc0614b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:52.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71e083e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71dc13aef0 0x7f71dc13faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:52.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71e083e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f71dc140c90 0x7f71dc152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:52.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71e083e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc1535d0 con 0x7f71dc140c90 2022-06-17T12:55:52.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71e083e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc0631a0 con 0x7f71dc0b7ce0 2022-06-17T12:55:52.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71e083e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc063440 con 0x7f71dc13aef0 2022-06-17T12:55:52.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71db59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f71dc140c90 0x7f71dc152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:52.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71db59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f71dc140c90 0x7f71dc152970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38838/0 (socket says 172.21.15.36:38838) 2022-06-17T12:55:52.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71db59e700 1 -- 172.21.15.36:0/1646210687 learned_addr learned my addr 172.21.15.36:0/1646210687 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:52.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71dad9d700 1 --2- 172.21.15.36:0/1646210687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71dc0b7ce0 0x7f71dc0614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:52.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71da59c700 1 --2- 172.21.15.36:0/1646210687 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71dc13aef0 0x7f71dc13faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:52.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71db59e700 1 -- 172.21.15.36:0/1646210687 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71dc13aef0 msgr2=0x7f71dc13faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:52.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71db59e700 1 --2- 172.21.15.36:0/1646210687 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71dc13aef0 0x7f71dc13faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:52.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71db59e700 1 -- 172.21.15.36:0/1646210687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71dc0b7ce0 msgr2=0x7f71dc0614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:52.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71db59e700 1 --2- 172.21.15.36:0/1646210687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71dc0b7ce0 0x7f71dc0614b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:52.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71db59e700 1 -- 172.21.15.36:0/1646210687 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d000b180 con 0x7f71dc140c90 2022-06-17T12:55:52.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71db59e700 1 --2- 172.21.15.36:0/1646210687 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f71dc140c90 0x7f71dc152970 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7f71dc0b5210 tx=0x7f71d4007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:52.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.875+0000 7f71cbfff700 1 -- 172.21.15.36:0/1646210687 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71d4005ae0 con 0x7f71dc140c90 2022-06-17T12:55:52.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.879+0000 7f71cbfff700 1 -- 172.21.15.36:0/1646210687 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d4005c40 con 0x7f71dc140c90 2022-06-17T12:55:52.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.879+0000 7f71e083e700 1 -- 172.21.15.36:0/1646210687 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71cc00b1e0 con 0x7f71dc140c90 2022-06-17T12:55:52.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.879+0000 7f71cbfff700 1 -- 172.21.15.36:0/1646210687 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71d40070d0 con 0x7f71dc140c90 2022-06-17T12:55:52.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.879+0000 7f71e083e700 1 -- 172.21.15.36:0/1646210687 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71dc140330 con 0x7f71dc140c90 2022-06-17T12:55:52.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.879+0000 7f71e083e700 1 -- 172.21.15.36:0/1646210687 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71dc142630 con 0x7f71dc140c90 2022-06-17T12:55:52.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.879+0000 7f71cbfff700 1 -- 172.21.15.36:0/1646210687 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f71d401a060 con 0x7f71dc140c90 2022-06-17T12:55:52.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.879+0000 7f71cbfff700 1 --2- 172.21.15.36:0/1646210687 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f71b8053a40 0x7f71b8055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:52.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.879+0000 7f71cbfff700 1 -- 172.21.15.36:0/1646210687 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f71d406b9c0 con 0x7f71dc140c90 2022-06-17T12:55:52.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.879+0000 7f71dad9d700 1 --2- 172.21.15.36:0/1646210687 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f71b8053a40 0x7f71b8055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:52.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.879+0000 7f71dad9d700 1 --2- 172.21.15.36:0/1646210687 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f71b8053a40 0x7f71b8055f00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f71cc01c000 tx=0x7f71cc015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:52.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:52.887+0000 7f71cbfff700 1 -- 172.21.15.36:0/1646210687 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f71d40427c0 con 0x7f71dc140c90 2022-06-17T12:55:53.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.047+0000 7f71e083e700 1 -- 172.21.15.36:0/1646210687 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f71dc057410 con 0x7f71dc140c90 2022-06-17T12:55:53.039 INFO:tasks.ceph.mon.a.smithi036.stderr:2022-06-17T12:55:53.047+0000 7fb79fe44700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-17T12:55:53.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.055+0000 7f71cbfff700 1 -- 172.21.15.36:0/1646210687 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 23 and data pool 22 v63) v1 ==== 189+0+0 (secure 0 0 0) 0x7f71d4018070 con 0x7f71dc140c90 2022-06-17T12:55:53.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.055+0000 7f71e083e700 1 -- 172.21.15.36:0/1646210687 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f71b8053a40 msgr2=0x7f71b8055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:53.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.055+0000 7f71e083e700 1 --2- 172.21.15.36:0/1646210687 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f71b8053a40 0x7f71b8055f00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f71cc01c000 tx=0x7f71cc015000).stop 2022-06-17T12:55:53.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.055+0000 7f71e083e700 1 -- 172.21.15.36:0/1646210687 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f71dc140c90 msgr2=0x7f71dc152970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:53.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.055+0000 7f71e083e700 1 --2- 172.21.15.36:0/1646210687 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f71dc140c90 0x7f71dc152970 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7f71dc0b5210 tx=0x7f71d4007f40).stop 2022-06-17T12:55:53.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.055+0000 7f71e083e700 1 -- 172.21.15.36:0/1646210687 shutdown_connections 2022-06-17T12:55:53.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.055+0000 7f71e083e700 1 --2- 172.21.15.36:0/1646210687 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f71b8053a40 0x7f71b8055f00 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:53.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.055+0000 7f71e083e700 1 --2- 172.21.15.36:0/1646210687 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f71dc140c90 0x7f71dc152970 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:53.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.055+0000 7f71e083e700 1 --2- 172.21.15.36:0/1646210687 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71dc13aef0 0x7f71dc13faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:53.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.055+0000 7f71e083e700 1 --2- 172.21.15.36:0/1646210687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71dc0b7ce0 0x7f71dc0614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:53.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.055+0000 7f71e083e700 1 -- 172.21.15.36:0/1646210687 >> 172.21.15.36:0/1646210687 conn(0x7f71dc05c550 msgr2=0x7f71dc0b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:53.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.059+0000 7f71e083e700 1 -- 172.21.15.36:0/1646210687 shutdown_connections 2022-06-17T12:55:53.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.059+0000 7f71e083e700 1 -- 172.21.15.36:0/1646210687 wait complete. 2022-06-17T12:55:53.053 INFO:teuthology.orchestra.run.smithi036.stderr:new fs with metadata pool 23 and data pool 22 2022-06-17T12:55:53.057 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-17T12:55:53.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7b00a0b700 1 Processor -- start 2022-06-17T12:55:53.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7b00a0b700 1 -- start start 2022-06-17T12:55:53.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7b00a0b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7afc0b27b0 0x7f7afc14b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:53.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7b00a0b700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7afc147bd0 0x7f7afc1442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:53.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7b00a0b700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7afc14b870 con 0x7f7afc060ca0 2022-06-17T12:55:53.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7b00a0b700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7afc0ac500 con 0x7f7afc0b27b0 2022-06-17T12:55:53.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7b00a0b700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7afc0ace90 con 0x7f7afc147bd0 2022-06-17T12:55:53.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7afb7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7afc0b27b0 0x7f7afc14b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:53.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7afb7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7afc0b27b0 0x7f7afc14b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38842/0 (socket says 172.21.15.36:38842) 2022-06-17T12:55:53.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7afb7fe700 1 -- 172.21.15.36:0/900546169 learned_addr learned my addr 172.21.15.36:0/900546169 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:53.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7afbfff700 1 --2- 172.21.15.36:0/900546169 >> v2:172.21.15.70:3300/0 conn(0x7f7afc147bd0 0x7f7afc1442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:53.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7afb7fe700 1 -- 172.21.15.36:0/900546169 >> v2:172.21.15.70:3300/0 conn(0x7f7afc147bd0 msgr2=0x7f7afc1442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:53.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7afb7fe700 1 --2- 172.21.15.36:0/900546169 >> v2:172.21.15.70:3300/0 conn(0x7f7afc147bd0 0x7f7afc1442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:53.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7afb7fe700 1 -- 172.21.15.36:0/900546169 >> v1:172.21.15.36:6789/0 conn(0x7f7afc060ca0 legacy=0x7f7afc0ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:53.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7afb7fe700 1 -- 172.21.15.36:0/900546169 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc0b2b90 con 0x7f7afc0b27b0 2022-06-17T12:55:53.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7afa7fc700 1 -- 172.21.15.36:0/900546169 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 657494331 0 0) 0x7f7afc14b870 con 0x7f7afc060ca0 2022-06-17T12:55:53.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7afbfff700 1 --2- 172.21.15.36:0/900546169 >> v2:172.21.15.70:3300/0 conn(0x7f7afc147bd0 0x7f7afc1442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:53.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7afb7fe700 1 --2- 172.21.15.36:0/900546169 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7afc0b27b0 0x7f7afc14b2b0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7f7af0002f70 tx=0x7f7af0006eb0).ready entity=mon.0 client_cookie=bf63b0a0d636430c server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:53.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7afa7fc700 1 -- 172.21.15.36:0/900546169 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7af000d610 con 0x7f7afc0b27b0 2022-06-17T12:55:53.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7afa7fc700 1 -- 172.21.15.36:0/900546169 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af0000e90 con 0x7f7afc0b27b0 2022-06-17T12:55:53.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7afa7fc700 1 -- 172.21.15.36:0/900546169 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7af000d960 con 0x7f7afc0b27b0 2022-06-17T12:55:53.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7b00a0b700 1 -- 172.21.15.36:0/900546169 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7afc0b27b0 msgr2=0x7f7afc14b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:53.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7b00a0b700 1 --2- 172.21.15.36:0/900546169 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7afc0b27b0 0x7f7afc14b2b0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7f7af0002f70 tx=0x7f7af0006eb0).stop 2022-06-17T12:55:53.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7b00a0b700 1 -- 172.21.15.36:0/900546169 shutdown_connections 2022-06-17T12:55:53.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7b00a0b700 1 --2- 172.21.15.36:0/900546169 >> v2:172.21.15.70:3300/0 conn(0x7f7afc147bd0 0x7f7afc1442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:53.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7b00a0b700 1 --2- 172.21.15.36:0/900546169 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7afc0b27b0 0x7f7afc14b2b0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:53.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7b00a0b700 1 -- 172.21.15.36:0/900546169 >> 172.21.15.36:0/900546169 conn(0x7f7afc0b5d10 msgr2=0x7f7afc0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:53.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7b00a0b700 1 -- 172.21.15.36:0/900546169 shutdown_connections 2022-06-17T12:55:53.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.179+0000 7f7b00a0b700 1 -- 172.21.15.36:0/900546169 wait complete. 2022-06-17T12:55:53.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.183+0000 7f7b00a0b700 1 Processor -- start 2022-06-17T12:55:53.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.183+0000 7f7b00a0b700 1 -- start start 2022-06-17T12:55:53.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.183+0000 7f7b00a0b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7afc060ca0 0x7f7afc0a70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:53.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.183+0000 7f7b00a0b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7afc0b27b0 0x7f7afc0648a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:53.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.183+0000 7f7b00a0b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7afc147bd0 0x7f7afc144a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:53.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.183+0000 7f7b00a0b700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc0a7860 con 0x7f7afc147bd0 2022-06-17T12:55:53.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.183+0000 7f7b00a0b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc0aa9a0 con 0x7f7afc060ca0 2022-06-17T12:55:53.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.183+0000 7f7b00a0b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc0aac40 con 0x7f7afc0b27b0 2022-06-17T12:55:53.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.183+0000 7f7afbfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7afc147bd0 0x7f7afc144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:53.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.183+0000 7f7afbfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7afc147bd0 0x7f7afc144a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38852/0 (socket says 172.21.15.36:38852) 2022-06-17T12:55:53.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.183+0000 7f7afbfff700 1 -- 172.21.15.36:0/4201350493 learned_addr learned my addr 172.21.15.36:0/4201350493 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:53.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.183+0000 7f7afaffd700 1 --2- 172.21.15.36:0/4201350493 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7afc0b27b0 0x7f7afc0648a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:53.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.183+0000 7f7afb7fe700 1 --2- 172.21.15.36:0/4201350493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7afc060ca0 0x7f7afc0a70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:53.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.183+0000 7f7afbfff700 1 -- 172.21.15.36:0/4201350493 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7afc0b27b0 msgr2=0x7f7afc0648a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:53.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.183+0000 7f7afbfff700 1 --2- 172.21.15.36:0/4201350493 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7afc0b27b0 0x7f7afc0648a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:53.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.183+0000 7f7afbfff700 1 -- 172.21.15.36:0/4201350493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7afc060ca0 msgr2=0x7f7afc0a70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:53.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.183+0000 7f7afbfff700 1 --2- 172.21.15.36:0/4201350493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7afc060ca0 0x7f7afc0a70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:53.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.183+0000 7f7afbfff700 1 -- 172.21.15.36:0/4201350493 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af000b1e0 con 0x7f7afc147bd0 2022-06-17T12:55:53.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.183+0000 7f7afbfff700 1 --2- 172.21.15.36:0/4201350493 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7afc147bd0 0x7f7afc144a90 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 rx=0x7f7af4006cc0 tx=0x7f7af4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:53.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.183+0000 7f7af8ff9700 1 -- 172.21.15.36:0/4201350493 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7af400c770 con 0x7f7afc147bd0 2022-06-17T12:55:53.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.183+0000 7f7af8ff9700 1 -- 172.21.15.36:0/4201350493 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af400c8d0 con 0x7f7afc147bd0 2022-06-17T12:55:53.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.183+0000 7f7b00a0b700 1 -- 172.21.15.36:0/4201350493 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7afc0aaee0 con 0x7f7afc147bd0 2022-06-17T12:55:53.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.183+0000 7f7af8ff9700 1 -- 172.21.15.36:0/4201350493 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7af400cc30 con 0x7f7afc147bd0 2022-06-17T12:55:53.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.183+0000 7f7b00a0b700 1 -- 172.21.15.36:0/4201350493 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7afc0ab380 con 0x7f7afc147bd0 2022-06-17T12:55:53.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.187+0000 7f7b00a0b700 1 -- 172.21.15.36:0/4201350493 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7afc0ab070 con 0x7f7afc147bd0 2022-06-17T12:55:53.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.187+0000 7f7af8ff9700 1 -- 172.21.15.36:0/4201350493 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f7af401d460 con 0x7f7afc147bd0 2022-06-17T12:55:53.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.187+0000 7f7af8ff9700 1 --2- 172.21.15.36:0/4201350493 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7adc0539b0 0x7f7adc055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:53.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.187+0000 7f7af8ff9700 1 -- 172.21.15.36:0/4201350493 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f7af406c590 con 0x7f7afc147bd0 2022-06-17T12:55:53.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.187+0000 7f7afb7fe700 1 --2- 172.21.15.36:0/4201350493 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7adc0539b0 0x7f7adc055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:53.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.187+0000 7f7afb7fe700 1 --2- 172.21.15.36:0/4201350493 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7adc0539b0 0x7f7adc055e70 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f7af001a000 tx=0x7f7af0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:53.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.195+0000 7f7af8ff9700 1 -- 172.21.15.36:0/4201350493 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7af40432e0 con 0x7f7afc147bd0 2022-06-17T12:55:53.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:53.359+0000 7f7b00a0b700 1 -- 172.21.15.36:0/4201350493 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7afc057410 con 0x7f7afc147bd0 2022-06-17T12:55:54.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.063+0000 7f7af8ff9700 1 -- 172.21.15.36:0/4201350493 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v64) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7af400ee60 con 0x7f7afc147bd0 2022-06-17T12:55:54.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.067+0000 7f7b00a0b700 1 -- 172.21.15.36:0/4201350493 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7adc0539b0 msgr2=0x7f7adc055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:54.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.067+0000 7f7b00a0b700 1 --2- 172.21.15.36:0/4201350493 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7adc0539b0 0x7f7adc055e70 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f7af001a000 tx=0x7f7af0015000).stop 2022-06-17T12:55:54.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.067+0000 7f7b00a0b700 1 -- 172.21.15.36:0/4201350493 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7afc147bd0 msgr2=0x7f7afc144a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:54.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.067+0000 7f7b00a0b700 1 --2- 172.21.15.36:0/4201350493 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7afc147bd0 0x7f7afc144a90 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 rx=0x7f7af4006cc0 tx=0x7f7af4008040).stop 2022-06-17T12:55:54.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.067+0000 7f7b00a0b700 1 -- 172.21.15.36:0/4201350493 shutdown_connections 2022-06-17T12:55:54.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.067+0000 7f7b00a0b700 1 --2- 172.21.15.36:0/4201350493 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7adc0539b0 0x7f7adc055e70 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:54.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.067+0000 7f7b00a0b700 1 --2- 172.21.15.36:0/4201350493 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7afc147bd0 0x7f7afc144a90 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:54.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.067+0000 7f7b00a0b700 1 --2- 172.21.15.36:0/4201350493 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7afc0b27b0 0x7f7afc0648a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:54.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.067+0000 7f7b00a0b700 1 --2- 172.21.15.36:0/4201350493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7afc060ca0 0x7f7afc0a70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:54.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.067+0000 7f7b00a0b700 1 -- 172.21.15.36:0/4201350493 >> 172.21.15.36:0/4201350493 conn(0x7f7afc0b5d10 msgr2=0x7f7afc0b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:54.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.067+0000 7f7b00a0b700 1 -- 172.21.15.36:0/4201350493 shutdown_connections 2022-06-17T12:55:54.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.067+0000 7f7b00a0b700 1 -- 172.21.15.36:0/4201350493 wait complete. 2022-06-17T12:55:54.072 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-17T12:55:54.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.199+0000 7f75bbfff700 1 Processor -- start 2022-06-17T12:55:54.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.199+0000 7f75bbfff700 1 -- start start 2022-06-17T12:55:54.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.199+0000 7f75bbfff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f75bc0b1920 0x7f75bc0afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:54.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.199+0000 7f75bbfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75bc143500 0x7f75bc13f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:54.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.199+0000 7f75bbfff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75bc060ea0 con 0x7f75bc060ac0 2022-06-17T12:55:54.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.199+0000 7f75bbfff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75bc142880 con 0x7f75bc0b1920 2022-06-17T12:55:54.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.199+0000 7f75bbfff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc142b20 con 0x7f75bc143500 2022-06-17T12:55:54.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75baffd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f75bc060ac0 0x7f75bc13c320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56386/0 (socket says 172.21.15.36:56386) 2022-06-17T12:55:54.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75baffd700 1 -- 172.21.15.36:0/2884196363 learned_addr learned my addr 172.21.15.36:0/2884196363 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:54.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75ba7fc700 1 --2- 172.21.15.36:0/2884196363 >> v2:172.21.15.70:3300/0 conn(0x7f75bc0b1920 0x7f75bc0afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:54.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75bb7fe700 1 --2- 172.21.15.36:0/2884196363 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75bc143500 0x7f75bc13f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:54.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75b9ffb700 1 -- 172.21.15.36:0/2884196363 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1370298938 0 0) 0x7f75bc060ea0 con 0x7f75bc060ac0 2022-06-17T12:55:54.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75b9ffb700 1 -- 172.21.15.36:0/2884196363 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75a0004370 con 0x7f75bc060ac0 2022-06-17T12:55:54.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75bb7fe700 1 -- 172.21.15.36:0/2884196363 >> v2:172.21.15.70:3300/0 conn(0x7f75bc0b1920 msgr2=0x7f75bc0afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:54.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75bb7fe700 1 --2- 172.21.15.36:0/2884196363 >> v2:172.21.15.70:3300/0 conn(0x7f75bc0b1920 0x7f75bc0afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:54.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75b9ffb700 1 -- 172.21.15.36:0/2884196363 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 759371015 0 0) 0x7f75a0004370 con 0x7f75bc060ac0 2022-06-17T12:55:54.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75bb7fe700 1 -- 172.21.15.36:0/2884196363 >> v1:172.21.15.36:6789/0 conn(0x7f75bc060ac0 legacy=0x7f75bc13c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:54.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75bb7fe700 1 -- 172.21.15.36:0/2884196363 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75bc142dc0 con 0x7f75bc143500 2022-06-17T12:55:54.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75ba7fc700 1 --2- 172.21.15.36:0/2884196363 >> v2:172.21.15.70:3300/0 conn(0x7f75bc0b1920 0x7f75bc0afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:54.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75bb7fe700 1 --2- 172.21.15.36:0/2884196363 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75bc143500 0x7f75bc13f2b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f75ac001800 tx=0x7f75b40091d0).ready entity=mon.2 client_cookie=ccfbd0f39cba63a3 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:54.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75b9ffb700 1 -- 172.21.15.36:0/2884196363 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75ac001660 con 0x7f75bc143500 2022-06-17T12:55:54.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75b9ffb700 1 -- 172.21.15.36:0/2884196363 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b400f510 con 0x7f75bc143500 2022-06-17T12:55:54.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75b9ffb700 1 -- 172.21.15.36:0/2884196363 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75b400f680 con 0x7f75bc143500 2022-06-17T12:55:54.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75bbfff700 1 -- 172.21.15.36:0/2884196363 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75bc143500 msgr2=0x7f75bc13f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:54.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75bbfff700 1 --2- 172.21.15.36:0/2884196363 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75bc143500 0x7f75bc13f2b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f75ac001800 tx=0x7f75b40091d0).stop 2022-06-17T12:55:54.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75bbfff700 1 -- 172.21.15.36:0/2884196363 shutdown_connections 2022-06-17T12:55:54.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75bbfff700 1 --2- 172.21.15.36:0/2884196363 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75bc143500 0x7f75bc13f2b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:54.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75bbfff700 1 --2- 172.21.15.36:0/2884196363 >> v2:172.21.15.70:3300/0 conn(0x7f75bc0b1920 0x7f75bc0afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:54.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75bbfff700 1 -- 172.21.15.36:0/2884196363 >> 172.21.15.36:0/2884196363 conn(0x7f75bc0b65d0 msgr2=0x7f75bc0b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:54.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75bbfff700 1 -- 172.21.15.36:0/2884196363 shutdown_connections 2022-06-17T12:55:54.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75bbfff700 1 -- 172.21.15.36:0/2884196363 wait complete. 2022-06-17T12:55:54.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75bbfff700 1 Processor -- start 2022-06-17T12:55:54.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.203+0000 7f75bbfff700 1 -- start start 2022-06-17T12:55:54.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.207+0000 7f75bbfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f75bc060ac0 0x7f75bc150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:54.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.207+0000 7f75bbfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75bc0b1920 0x7f75bc137f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:54.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.207+0000 7f75bbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75bc143500 0x7f75bc144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:54.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.207+0000 7f75bbfff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc1442a0 con 0x7f75bc060ac0 2022-06-17T12:55:54.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.207+0000 7f75bbfff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc151140 con 0x7f75bc143500 2022-06-17T12:55:54.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.207+0000 7f75bbfff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc138470 con 0x7f75bc0b1920 2022-06-17T12:55:54.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.207+0000 7f75baffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f75bc060ac0 0x7f75bc150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:54.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.207+0000 7f75baffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f75bc060ac0 0x7f75bc150c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38860/0 (socket says 172.21.15.36:38860) 2022-06-17T12:55:54.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.207+0000 7f75baffd700 1 -- 172.21.15.36:0/248972216 learned_addr learned my addr 172.21.15.36:0/248972216 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:54.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.207+0000 7f75ba7fc700 1 --2- 172.21.15.36:0/248972216 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75bc0b1920 0x7f75bc137f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:54.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.207+0000 7f75bb7fe700 1 --2- 172.21.15.36:0/248972216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75bc143500 0x7f75bc144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:54.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.207+0000 7f75ba7fc700 1 -- 172.21.15.36:0/248972216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75bc143500 msgr2=0x7f75bc144e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:54.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.207+0000 7f75ba7fc700 1 --2- 172.21.15.36:0/248972216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75bc143500 0x7f75bc144e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:54.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.207+0000 7f75ba7fc700 1 -- 172.21.15.36:0/248972216 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f75bc060ac0 msgr2=0x7f75bc150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:54.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.207+0000 7f75ba7fc700 1 --2- 172.21.15.36:0/248972216 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f75bc060ac0 0x7f75bc150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:54.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.207+0000 7f75ba7fc700 1 -- 172.21.15.36:0/248972216 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b000b180 con 0x7f75bc0b1920 2022-06-17T12:55:54.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.207+0000 7f75ba7fc700 1 --2- 172.21.15.36:0/248972216 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75bc0b1920 0x7f75bc137f80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f75b000c000 tx=0x7f75b0006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:54.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.207+0000 7f75abfff700 1 -- 172.21.15.36:0/248972216 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75b0010780 con 0x7f75bc0b1920 2022-06-17T12:55:54.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.207+0000 7f75abfff700 1 -- 172.21.15.36:0/248972216 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b00108e0 con 0x7f75bc0b1920 2022-06-17T12:55:54.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.207+0000 7f75bbfff700 1 -- 172.21.15.36:0/248972216 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b4009470 con 0x7f75bc0b1920 2022-06-17T12:55:54.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.207+0000 7f75abfff700 1 -- 172.21.15.36:0/248972216 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75b0010c40 con 0x7f75bc0b1920 2022-06-17T12:55:54.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.207+0000 7f75bbfff700 1 -- 172.21.15.36:0/248972216 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75bc1446a0 con 0x7f75bc0b1920 2022-06-17T12:55:54.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.207+0000 7f75bbfff700 1 -- 172.21.15.36:0/248972216 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75bc143060 con 0x7f75bc0b1920 2022-06-17T12:55:54.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.207+0000 7f75abfff700 1 -- 172.21.15.36:0/248972216 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f75b001c460 con 0x7f75bc0b1920 2022-06-17T12:55:54.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.211+0000 7f75abfff700 1 --2- 172.21.15.36:0/248972216 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7598053a60 0x7f7598055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:54.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.211+0000 7f75baffd700 1 --2- 172.21.15.36:0/248972216 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7598053a60 0x7f7598055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:54.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.211+0000 7f75abfff700 1 -- 172.21.15.36:0/248972216 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f75b006c510 con 0x7f75bc0b1920 2022-06-17T12:55:54.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.211+0000 7f75baffd700 1 --2- 172.21.15.36:0/248972216 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7598053a60 0x7f7598055f20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f75ac00d000 tx=0x7f75ac0011a0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:54.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.219+0000 7f75abfff700 1 -- 172.21.15.36:0/248972216 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f75b0043150 con 0x7f75bc0b1920 2022-06-17T12:55:54.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:54.351+0000 7f75bbfff700 1 -- 172.21.15.36:0/248972216 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f75bc143f30 con 0x7f75bc0b1920 2022-06-17T12:55:55.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.071+0000 7f75abfff700 1 -- 172.21.15.36:0/248972216 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v121) v1 ==== 185+0+0 (secure 0 0 0) 0x7f75b0046770 con 0x7f75bc0b1920 2022-06-17T12:55:55.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.075+0000 7f75bbfff700 1 -- 172.21.15.36:0/248972216 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7598053a60 msgr2=0x7f7598055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:55.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.075+0000 7f75bbfff700 1 --2- 172.21.15.36:0/248972216 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7598053a60 0x7f7598055f20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f75ac00d000 tx=0x7f75ac0011a0).stop 2022-06-17T12:55:55.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.075+0000 7f75bbfff700 1 -- 172.21.15.36:0/248972216 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75bc0b1920 msgr2=0x7f75bc137f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:55.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.075+0000 7f75bbfff700 1 --2- 172.21.15.36:0/248972216 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75bc0b1920 0x7f75bc137f80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f75b000c000 tx=0x7f75b0006df0).stop 2022-06-17T12:55:55.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.075+0000 7f75bbfff700 1 -- 172.21.15.36:0/248972216 shutdown_connections 2022-06-17T12:55:55.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.075+0000 7f75bbfff700 1 --2- 172.21.15.36:0/248972216 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7598053a60 0x7f7598055f20 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:55.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.075+0000 7f75bbfff700 1 --2- 172.21.15.36:0/248972216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75bc143500 0x7f75bc144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:55.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.075+0000 7f75bbfff700 1 --2- 172.21.15.36:0/248972216 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75bc0b1920 0x7f75bc137f80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:55.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.075+0000 7f75bbfff700 1 --2- 172.21.15.36:0/248972216 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f75bc060ac0 0x7f75bc150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:55.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.075+0000 7f75bbfff700 1 -- 172.21.15.36:0/248972216 >> 172.21.15.36:0/248972216 conn(0x7f75bc0b65d0 msgr2=0x7f75bc05bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:55.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.075+0000 7f75bbfff700 1 -- 172.21.15.36:0/248972216 shutdown_connections 2022-06-17T12:55:55.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.075+0000 7f75bbfff700 1 -- 172.21.15.36:0/248972216 wait complete. 2022-06-17T12:55:55.074 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' removed 2022-06-17T12:55:55.077 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-17T12:55:55.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.211+0000 7f91be446700 1 Processor -- start 2022-06-17T12:55:55.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.211+0000 7f91be446700 1 -- start start 2022-06-17T12:55:55.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.211+0000 7f91be446700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91b8060920 0x7f91b8138ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:55.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.211+0000 7f91be446700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b80af610 con 0x7f91b813fe30 2022-06-17T12:55:55.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.211+0000 7f91be446700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b80af8b0 con 0x7f91b80b11a0 2022-06-17T12:55:55.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.211+0000 7f91be446700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b80ae190 con 0x7f91b8060920 2022-06-17T12:55:55.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.211+0000 7f91bcc43700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f91b813fe30 0x7f91b80ab9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56402/0 (socket says 172.21.15.36:56402) 2022-06-17T12:55:55.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.211+0000 7f91bcc43700 1 -- 172.21.15.36:0/1724641321 learned_addr learned my addr 172.21.15.36:0/1724641321 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:55.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.211+0000 7f91bd444700 1 --2- 172.21.15.36:0/1724641321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91b8060920 0x7f91b8138ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:55.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.211+0000 7f91affff700 1 -- 172.21.15.36:0/1724641321 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3031449646 0 0) 0x7f91b80af610 con 0x7f91b813fe30 2022-06-17T12:55:55.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.211+0000 7f91affff700 1 -- 172.21.15.36:0/1724641321 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f919c003260 con 0x7f91b813fe30 2022-06-17T12:55:55.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.211+0000 7f91affff700 1 -- 172.21.15.36:0/1724641321 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3439072523 0 0) 0x7f91b80af8b0 con 0x7f91b80b11a0 2022-06-17T12:55:55.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.211+0000 7f91affff700 1 -- 172.21.15.36:0/1724641321 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91b80af610 con 0x7f91b80b11a0 2022-06-17T12:55:55.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.211+0000 7f91affff700 1 -- 172.21.15.36:0/1724641321 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3547960536 0 0) 0x7f919c003260 con 0x7f91b813fe30 2022-06-17T12:55:55.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.211+0000 7f91bd444700 1 -- 172.21.15.36:0/1724641321 >> v1:172.21.15.70:6789/0 conn(0x7f91b80b11a0 legacy=0x7f91b80afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:55.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.211+0000 7f91bd444700 1 -- 172.21.15.36:0/1724641321 >> v1:172.21.15.36:6789/0 conn(0x7f91b813fe30 legacy=0x7f91b80ab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:55.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.211+0000 7f91bd444700 1 -- 172.21.15.36:0/1724641321 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b80ac780 con 0x7f91b8060920 2022-06-17T12:55:55.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.211+0000 7f91bd444700 1 --2- 172.21.15.36:0/1724641321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91b8060920 0x7f91b8138ff0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f91b4001490 tx=0x7f91a8006df0).ready entity=mon.2 client_cookie=aacb4172e14180a0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:55.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.215+0000 7f91affff700 1 -- 172.21.15.36:0/1724641321 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91b40019e0 con 0x7f91b8060920 2022-06-17T12:55:55.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.215+0000 7f91affff700 1 -- 172.21.15.36:0/1724641321 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a8000e90 con 0x7f91b8060920 2022-06-17T12:55:55.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.215+0000 7f91affff700 1 -- 172.21.15.36:0/1724641321 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91a8012460 con 0x7f91b8060920 2022-06-17T12:55:55.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.215+0000 7f91be446700 1 -- 172.21.15.36:0/1724641321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91b8060920 msgr2=0x7f91b8138ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:55.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.215+0000 7f91be446700 1 --2- 172.21.15.36:0/1724641321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91b8060920 0x7f91b8138ff0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f91b4001490 tx=0x7f91a8006df0).stop 2022-06-17T12:55:55.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.215+0000 7f91be446700 1 -- 172.21.15.36:0/1724641321 shutdown_connections 2022-06-17T12:55:55.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.215+0000 7f91be446700 1 --2- 172.21.15.36:0/1724641321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91b8060920 0x7f91b8138ff0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:55.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.215+0000 7f91be446700 1 -- 172.21.15.36:0/1724641321 >> 172.21.15.36:0/1724641321 conn(0x7f91b80b5d10 msgr2=0x7f91b80b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:55.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.215+0000 7f91be446700 1 -- 172.21.15.36:0/1724641321 shutdown_connections 2022-06-17T12:55:55.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.215+0000 7f91be446700 1 -- 172.21.15.36:0/1724641321 wait complete. 2022-06-17T12:55:55.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.219+0000 7f91be446700 1 Processor -- start 2022-06-17T12:55:55.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.219+0000 7f91be446700 1 -- start start 2022-06-17T12:55:55.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.219+0000 7f91be446700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b8060920 0x7f91b813ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:55.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.219+0000 7f91be446700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91b80b11a0 0x7f91b8140880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:55.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.219+0000 7f91be446700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91b813fe30 0x7f91b8140d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:55.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.219+0000 7f91be446700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b80653a0 con 0x7f91b813fe30 2022-06-17T12:55:55.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.219+0000 7f91be446700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b8142fd0 con 0x7f91b8060920 2022-06-17T12:55:55.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.219+0000 7f91be446700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b8143270 con 0x7f91b80b11a0 2022-06-17T12:55:55.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.219+0000 7f91bdc45700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91b813fe30 0x7f91b8140d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:55.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.219+0000 7f91bcc43700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91b80b11a0 0x7f91b8140880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:55.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.219+0000 7f91bd444700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b8060920 0x7f91b813ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:55.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.219+0000 7f91bdc45700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91b813fe30 0x7f91b8140d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38878/0 (socket says 172.21.15.36:38878) 2022-06-17T12:55:55.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.219+0000 7f91bdc45700 1 -- 172.21.15.36:0/3154695374 learned_addr learned my addr 172.21.15.36:0/3154695374 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:55.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.219+0000 7f91bdc45700 1 -- 172.21.15.36:0/3154695374 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91b80b11a0 msgr2=0x7f91b8140880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:55.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.219+0000 7f91bdc45700 1 --2- 172.21.15.36:0/3154695374 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91b80b11a0 0x7f91b8140880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:55.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.219+0000 7f91bdc45700 1 -- 172.21.15.36:0/3154695374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b8060920 msgr2=0x7f91b813ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:55.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.219+0000 7f91bdc45700 1 --2- 172.21.15.36:0/3154695374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b8060920 0x7f91b813ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:55.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.219+0000 7f91bdc45700 1 -- 172.21.15.36:0/3154695374 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a800b180 con 0x7f91b813fe30 2022-06-17T12:55:55.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.219+0000 7f91bdc45700 1 --2- 172.21.15.36:0/3154695374 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91b813fe30 0x7f91b8140d70 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0x7f91b0001110 tx=0x7f91b0007a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:55.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.219+0000 7f91ae7fc700 1 -- 172.21.15.36:0/3154695374 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91b0012400 con 0x7f91b813fe30 2022-06-17T12:55:55.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.219+0000 7f91ae7fc700 1 -- 172.21.15.36:0/3154695374 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b0012560 con 0x7f91b813fe30 2022-06-17T12:55:55.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.219+0000 7f91be446700 1 -- 172.21.15.36:0/3154695374 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b8143510 con 0x7f91b813fe30 2022-06-17T12:55:55.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.219+0000 7f91ae7fc700 1 -- 172.21.15.36:0/3154695374 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91b00128c0 con 0x7f91b813fe30 2022-06-17T12:55:55.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.219+0000 7f91be446700 1 -- 172.21.15.36:0/3154695374 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b8142340 con 0x7f91b813fe30 2022-06-17T12:55:55.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.223+0000 7f91be446700 1 -- 172.21.15.36:0/3154695374 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b81436a0 con 0x7f91b813fe30 2022-06-17T12:55:55.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.223+0000 7f91ae7fc700 1 -- 172.21.15.36:0/3154695374 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f91b0019030 con 0x7f91b813fe30 2022-06-17T12:55:55.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.223+0000 7f91ae7fc700 1 --2- 172.21.15.36:0/3154695374 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f91a00539b0 0x7f91a0055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:55.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.223+0000 7f91ae7fc700 1 -- 172.21.15.36:0/3154695374 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f91b006b190 con 0x7f91b813fe30 2022-06-17T12:55:55.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.223+0000 7f91bd444700 1 --2- 172.21.15.36:0/3154695374 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f91a00539b0 0x7f91a0055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:55.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.223+0000 7f91bd444700 1 --2- 172.21.15.36:0/3154695374 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f91a00539b0 0x7f91a0055e70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f91a801a000 tx=0x7f91a8014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:55.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.231+0000 7f91ae7fc700 1 -- 172.21.15.36:0/3154695374 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f91b00420f0 con 0x7f91b813fe30 2022-06-17T12:55:55.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:55.355+0000 7f91be446700 1 -- 172.21.15.36:0/3154695374 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f91b8057410 con 0x7f91b813fe30 2022-06-17T12:55:56.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.071+0000 7f91ae7fc700 1 -- 172.21.15.36:0/3154695374 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v122) v1 ==== 133+0+0 (secure 0 0 0) 0x7f91b0045710 con 0x7f91b813fe30 2022-06-17T12:55:56.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.075+0000 7f91be446700 1 -- 172.21.15.36:0/3154695374 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f91a00539b0 msgr2=0x7f91a0055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:56.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.075+0000 7f91be446700 1 --2- 172.21.15.36:0/3154695374 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f91a00539b0 0x7f91a0055e70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f91a801a000 tx=0x7f91a8014000).stop 2022-06-17T12:55:56.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.075+0000 7f91be446700 1 -- 172.21.15.36:0/3154695374 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91b813fe30 msgr2=0x7f91b8140d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:56.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.075+0000 7f91be446700 1 --2- 172.21.15.36:0/3154695374 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91b813fe30 0x7f91b8140d70 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0x7f91b0001110 tx=0x7f91b0007a50).stop 2022-06-17T12:55:56.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.075+0000 7f91be446700 1 -- 172.21.15.36:0/3154695374 shutdown_connections 2022-06-17T12:55:56.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.075+0000 7f91be446700 1 --2- 172.21.15.36:0/3154695374 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f91a00539b0 0x7f91a0055e70 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:56.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.075+0000 7f91be446700 1 --2- 172.21.15.36:0/3154695374 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91b813fe30 0x7f91b8140d70 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:56.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.075+0000 7f91be446700 1 --2- 172.21.15.36:0/3154695374 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91b80b11a0 0x7f91b8140880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:56.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.075+0000 7f91be446700 1 --2- 172.21.15.36:0/3154695374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b8060920 0x7f91b813ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:56.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.075+0000 7f91be446700 1 -- 172.21.15.36:0/3154695374 >> 172.21.15.36:0/3154695374 conn(0x7f91b80b5d10 msgr2=0x7f91b8134970 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:56.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.075+0000 7f91be446700 1 -- 172.21.15.36:0/3154695374 shutdown_connections 2022-06-17T12:55:56.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.075+0000 7f91be446700 1 -- 172.21.15.36:0/3154695374 wait complete. 2022-06-17T12:55:56.080 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' created 2022-06-17T12:55:56.083 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2022-06-17T12:55:56.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.207+0000 7faa35766700 1 Processor -- start 2022-06-17T12:55:56.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.207+0000 7faa35766700 1 -- start start 2022-06-17T12:55:56.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.207+0000 7faa35766700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa300b3440 0x7faa300b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:56.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.207+0000 7faa35766700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faa3013b130 0x7faa30140f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:56.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.207+0000 7faa35766700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa300a9b90 con 0x7faa300afe40 2022-06-17T12:55:56.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.207+0000 7faa35766700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa300b0220 con 0x7faa3013b130 2022-06-17T12:55:56.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.207+0000 7faa35766700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa300a9dc0 con 0x7faa300b3440 2022-06-17T12:55:56.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.207+0000 7faa2e7fc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa300b3440 0x7faa300b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:56.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.207+0000 7faa2e7fc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa300b3440 0x7faa300b2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:53674/0 (socket says 172.21.15.36:53674) 2022-06-17T12:55:56.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.207+0000 7faa2e7fc700 1 -- 172.21.15.36:0/86519214 learned_addr learned my addr 172.21.15.36:0/86519214 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:56.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.207+0000 7faa2f7fe700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faa3013b130 0x7faa30140f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:56.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.207+0000 7faa2e7fc700 1 -- 172.21.15.36:0/86519214 >> v2:172.21.15.70:3300/0 conn(0x7faa3013b130 msgr2=0x7faa30140f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:56.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.207+0000 7faa2e7fc700 1 --2- 172.21.15.36:0/86519214 >> v2:172.21.15.70:3300/0 conn(0x7faa3013b130 0x7faa30140f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:56.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.207+0000 7faa2e7fc700 1 -- 172.21.15.36:0/86519214 >> v1:172.21.15.70:6789/0 conn(0x7faa300afe40 legacy=0x7faa3013aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:56.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.207+0000 7faa2e7fc700 1 -- 172.21.15.36:0/86519214 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa30144f40 con 0x7faa300b3440 2022-06-17T12:55:56.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.207+0000 7faa2e7fc700 1 --2- 172.21.15.36:0/86519214 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa300b3440 0x7faa300b2e90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7faa200032f0 tx=0x7faa20009290).ready entity=mon.2 client_cookie=8c0bfcf11de8cbdc server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:56.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.207+0000 7faa2dffb700 1 -- 172.21.15.36:0/86519214 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa2000b8a0 con 0x7faa300b3440 2022-06-17T12:55:56.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.207+0000 7faa2dffb700 1 -- 172.21.15.36:0/86519214 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2000ba00 con 0x7faa300b3440 2022-06-17T12:55:56.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.207+0000 7faa2dffb700 1 -- 172.21.15.36:0/86519214 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa2000bb70 con 0x7faa300b3440 2022-06-17T12:55:56.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa35766700 1 -- 172.21.15.36:0/86519214 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa300b3440 msgr2=0x7faa300b2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:56.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa35766700 1 --2- 172.21.15.36:0/86519214 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa300b3440 0x7faa300b2e90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7faa200032f0 tx=0x7faa20009290).stop 2022-06-17T12:55:56.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa35766700 1 -- 172.21.15.36:0/86519214 shutdown_connections 2022-06-17T12:55:56.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa35766700 1 --2- 172.21.15.36:0/86519214 >> v2:172.21.15.70:3300/0 conn(0x7faa3013b130 0x7faa30140f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:56.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa35766700 1 --2- 172.21.15.36:0/86519214 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa300b3440 0x7faa300b2e90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:56.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa35766700 1 -- 172.21.15.36:0/86519214 >> 172.21.15.36:0/86519214 conn(0x7faa3005c550 msgr2=0x7faa3005c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:56.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa35766700 1 -- 172.21.15.36:0/86519214 shutdown_connections 2022-06-17T12:55:56.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa35766700 1 -- 172.21.15.36:0/86519214 wait complete. 2022-06-17T12:55:56.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa35766700 1 Processor -- start 2022-06-17T12:55:56.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa35766700 1 -- start start 2022-06-17T12:55:56.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa35766700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa300afe40 0x7faa300a91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:56.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa35766700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa300b3440 0x7faa30138970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:56.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa35766700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa3013b130 0x7faa30138e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:56.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa35766700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa300ad820 con 0x7faa300b3440 2022-06-17T12:55:56.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa35766700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa300ab960 con 0x7faa300afe40 2022-06-17T12:55:56.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa35766700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa300ab450 con 0x7faa3013b130 2022-06-17T12:55:56.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa2e7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa300b3440 0x7faa30138970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:56.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa2e7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa300b3440 0x7faa30138970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38890/0 (socket says 172.21.15.36:38890) 2022-06-17T12:55:56.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa2e7fc700 1 -- 172.21.15.36:0/1701890281 learned_addr learned my addr 172.21.15.36:0/1701890281 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:56.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa2f7fe700 1 --2- 172.21.15.36:0/1701890281 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa3013b130 0x7faa30138e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:56.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa2effd700 1 --2- 172.21.15.36:0/1701890281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa300afe40 0x7faa300a91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:56.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa2e7fc700 1 -- 172.21.15.36:0/1701890281 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa3013b130 msgr2=0x7faa30138e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:56.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa2e7fc700 1 --2- 172.21.15.36:0/1701890281 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa3013b130 0x7faa30138e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:56.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa2e7fc700 1 -- 172.21.15.36:0/1701890281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa300afe40 msgr2=0x7faa300a91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:56.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa2e7fc700 1 --2- 172.21.15.36:0/1701890281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa300afe40 0x7faa300a91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:56.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa2e7fc700 1 -- 172.21.15.36:0/1701890281 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa20009530 con 0x7faa300b3440 2022-06-17T12:55:56.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa2effd700 1 --2- 172.21.15.36:0/1701890281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa300afe40 0x7faa300a91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:56.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa2e7fc700 1 --2- 172.21.15.36:0/1701890281 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa300b3440 0x7faa30138970 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0x7faa3013c4e0 tx=0x7faa20016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:56.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa27fff700 1 -- 172.21.15.36:0/1701890281 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa20021480 con 0x7faa300b3440 2022-06-17T12:55:56.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa35766700 1 -- 172.21.15.36:0/1701890281 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa300ab6c0 con 0x7faa300b3440 2022-06-17T12:55:56.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.211+0000 7faa27fff700 1 -- 172.21.15.36:0/1701890281 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa200215e0 con 0x7faa300b3440 2022-06-17T12:55:56.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.215+0000 7faa27fff700 1 -- 172.21.15.36:0/1701890281 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa20021940 con 0x7faa300b3440 2022-06-17T12:55:56.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.215+0000 7faa35766700 1 -- 172.21.15.36:0/1701890281 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa30139350 con 0x7faa300b3440 2022-06-17T12:55:56.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.215+0000 7faa35766700 1 -- 172.21.15.36:0/1701890281 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa301451e0 con 0x7faa300b3440 2022-06-17T12:55:56.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.215+0000 7faa27fff700 1 -- 172.21.15.36:0/1701890281 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7faa2000f070 con 0x7faa300b3440 2022-06-17T12:55:56.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.215+0000 7faa27fff700 1 --2- 172.21.15.36:0/1701890281 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa080539b0 0x7faa08055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:56.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.215+0000 7faa27fff700 1 -- 172.21.15.36:0/1701890281 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6422+0+0 (secure 0 0 0) 0x7faa2006d4b0 con 0x7faa300b3440 2022-06-17T12:55:56.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.215+0000 7faa2effd700 1 --2- 172.21.15.36:0/1701890281 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa080539b0 0x7faa08055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:56.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.215+0000 7faa2effd700 1 --2- 172.21.15.36:0/1701890281 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa080539b0 0x7faa08055e70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7faa1800b000 tx=0x7faa18009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:56.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.223+0000 7faa27fff700 1 -- 172.21.15.36:0/1701890281 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faa20044250 con 0x7faa300b3440 2022-06-17T12:55:56.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.391+0000 7faa35766700 1 -- 172.21.15.36:0/1701890281 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7faa30057410 con 0x7faa300b3440 2022-06-17T12:55:56.379 INFO:tasks.ceph.mon.a.smithi036.stderr:2022-06-17T12:55:56.391+0000 7fb79fe44700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-17T12:55:56.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.399+0000 7faa27fff700 1 -- 172.21.15.36:0/1701890281 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 22 v65) v1 ==== 174+0+0 (secure 0 0 0) 0x7faa20015030 con 0x7faa300b3440 2022-06-17T12:55:56.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.407+0000 7faa35766700 1 -- 172.21.15.36:0/1701890281 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa080539b0 msgr2=0x7faa08055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:56.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.407+0000 7faa35766700 1 --2- 172.21.15.36:0/1701890281 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa080539b0 0x7faa08055e70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7faa1800b000 tx=0x7faa18009040).stop 2022-06-17T12:55:56.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.407+0000 7faa35766700 1 -- 172.21.15.36:0/1701890281 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa300b3440 msgr2=0x7faa30138970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:56.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.407+0000 7faa35766700 1 --2- 172.21.15.36:0/1701890281 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa300b3440 0x7faa30138970 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0x7faa3013c4e0 tx=0x7faa20016000).stop 2022-06-17T12:55:56.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.407+0000 7faa35766700 1 -- 172.21.15.36:0/1701890281 shutdown_connections 2022-06-17T12:55:56.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.407+0000 7faa35766700 1 --2- 172.21.15.36:0/1701890281 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faa080539b0 0x7faa08055e70 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:56.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.407+0000 7faa35766700 1 --2- 172.21.15.36:0/1701890281 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa3013b130 0x7faa30138e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:56.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.407+0000 7faa35766700 1 --2- 172.21.15.36:0/1701890281 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faa300b3440 0x7faa30138970 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:56.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.407+0000 7faa35766700 1 --2- 172.21.15.36:0/1701890281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa300afe40 0x7faa300a91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:56.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.407+0000 7faa35766700 1 -- 172.21.15.36:0/1701890281 >> 172.21.15.36:0/1701890281 conn(0x7faa3005c550 msgr2=0x7faa30139f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:56.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.407+0000 7faa35766700 1 -- 172.21.15.36:0/1701890281 shutdown_connections 2022-06-17T12:55:56.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.407+0000 7faa35766700 1 -- 172.21.15.36:0/1701890281 wait complete. 2022-06-17T12:55:56.397 INFO:teuthology.orchestra.run.smithi036.stderr:new fs with metadata pool 24 and data pool 22 2022-06-17T12:55:56.406 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-17T12:55:56.407 DEBUG:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:55:56.469 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:55:56.503 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:55:56.536 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:55:56.565 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:55:56.593 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-17T12:55:56.595 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:56.596 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:55:56.597 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:55:56.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:55:56.610 INFO:teuthology.orchestra.run.smithi107.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-17T12:55:56.611 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-06-17T12:55:56.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:55:56.612 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:55:56.658 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-17T12:55:56.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:55:56.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:56.663 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:55:56.663 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:55:56.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:55:56.677 INFO:teuthology.orchestra.run.smithi136.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-17T12:55:56.677 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-17T12:55:56.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:55:56.680 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:55:56.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:55:56.690 INFO:teuthology.orchestra.run.smithi136.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-17T12:55:56.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:55:56.691 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:55:56.692 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:55:56.699 INFO:teuthology.orchestra.run.smithi143.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:55:56.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:55:56.702 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-06-17T12:55:56.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:55:56.703 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:55:56.710 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:55:56.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:55:56.713 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-17T12:55:56.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.839+0000 7fe1149a5700 1 Processor -- start 2022-06-17T12:55:56.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.839+0000 7fe1149a5700 1 -- start start 2022-06-17T12:55:56.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.843+0000 7fe1149a5700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe11005fe80 0x7fe110060260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:56.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.843+0000 7fe1149a5700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe110066210 con 0x7fe11013aa80 2022-06-17T12:55:56.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.843+0000 7fe1149a5700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1100664b0 con 0x7fe1100adf40 2022-06-17T12:55:56.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.843+0000 7fe1149a5700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe110142070 con 0x7fe11005fe80 2022-06-17T12:55:56.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.843+0000 7fe10e59c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe11005fe80 0x7fe110060260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:56.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.843+0000 7fe10ed9d700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fe11013aa80 0x7fe110141530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56430/0 (socket says 172.21.15.36:56430) 2022-06-17T12:55:56.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.843+0000 7fe10ed9d700 1 -- 172.21.15.36:0/2177098731 learned_addr learned my addr 172.21.15.36:0/2177098731 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:56.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.843+0000 7fe10d59a700 1 -- 172.21.15.36:0/2177098731 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2114302680 0 0) 0x7fe110066210 con 0x7fe11013aa80 2022-06-17T12:55:56.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.843+0000 7fe10d59a700 1 -- 172.21.15.36:0/2177098731 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0f4003260 con 0x7fe11013aa80 2022-06-17T12:55:56.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.843+0000 7fe10e59c700 1 -- 172.21.15.36:0/2177098731 >> v1:172.21.15.70:6789/0 conn(0x7fe1100adf40 legacy=0x7fe1100667f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:56.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.843+0000 7fe10e59c700 1 -- 172.21.15.36:0/2177098731 >> v1:172.21.15.36:6789/0 conn(0x7fe11013aa80 legacy=0x7fe110141530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:56.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.843+0000 7fe10e59c700 1 -- 172.21.15.36:0/2177098731 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe110141090 con 0x7fe11005fe80 2022-06-17T12:55:56.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.843+0000 7fe10e59c700 1 --2- 172.21.15.36:0/2177098731 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe11005fe80 0x7fe110060260 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 rx=0x7fe100002f70 tx=0x7fe100006df0).ready entity=mon.0 client_cookie=5b7c14a5f0704e6c server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:56.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.843+0000 7fe10d59a700 1 -- 172.21.15.36:0/2177098731 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe10000d610 con 0x7fe11005fe80 2022-06-17T12:55:56.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.843+0000 7fe10d59a700 1 -- 172.21.15.36:0/2177098731 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe100000e90 con 0x7fe11005fe80 2022-06-17T12:55:56.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.843+0000 7fe10d59a700 1 -- 172.21.15.36:0/2177098731 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe10000d960 con 0x7fe11005fe80 2022-06-17T12:55:56.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.843+0000 7fe1149a5700 1 -- 172.21.15.36:0/2177098731 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe11005fe80 msgr2=0x7fe110060260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:56.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.843+0000 7fe1149a5700 1 --2- 172.21.15.36:0/2177098731 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe11005fe80 0x7fe110060260 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 rx=0x7fe100002f70 tx=0x7fe100006df0).stop 2022-06-17T12:55:56.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.843+0000 7fe1149a5700 1 -- 172.21.15.36:0/2177098731 shutdown_connections 2022-06-17T12:55:56.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.843+0000 7fe1149a5700 1 --2- 172.21.15.36:0/2177098731 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe11005fe80 0x7fe110060260 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:56.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.843+0000 7fe1149a5700 1 -- 172.21.15.36:0/2177098731 >> 172.21.15.36:0/2177098731 conn(0x7fe110060990 msgr2=0x7fe11005f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:56.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.843+0000 7fe1149a5700 1 -- 172.21.15.36:0/2177098731 shutdown_connections 2022-06-17T12:55:56.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.843+0000 7fe1149a5700 1 -- 172.21.15.36:0/2177098731 wait complete. 2022-06-17T12:55:56.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.847+0000 7fe1149a5700 1 Processor -- start 2022-06-17T12:55:56.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.847+0000 7fe1149a5700 1 -- start start 2022-06-17T12:55:56.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.847+0000 7fe1149a5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe11005fe80 0x7fe1100b5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:56.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.847+0000 7fe1149a5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1100adf40 0x7fe110143170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:56.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.847+0000 7fe1149a5700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe11013aa80 0x7fe110144e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:56.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.847+0000 7fe1149a5700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11013a320 con 0x7fe11005fe80 2022-06-17T12:55:56.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.847+0000 7fe1149a5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1100b0530 con 0x7fe1100adf40 2022-06-17T12:55:56.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.847+0000 7fe1149a5700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1100b07d0 con 0x7fe11013aa80 2022-06-17T12:55:56.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.847+0000 7fe10e59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe11005fe80 0x7fe1100b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:56.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.847+0000 7fe10e59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe11005fe80 0x7fe1100b5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38904/0 (socket says 172.21.15.36:38904) 2022-06-17T12:55:56.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.847+0000 7fe10e59c700 1 -- 172.21.15.36:0/3339918026 learned_addr learned my addr 172.21.15.36:0/3339918026 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:56.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.847+0000 7fe10ed9d700 1 --2- 172.21.15.36:0/3339918026 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe11013aa80 0x7fe110144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:56.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.847+0000 7fe10dd9b700 1 --2- 172.21.15.36:0/3339918026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1100adf40 0x7fe110143170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:56.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.847+0000 7fe10e59c700 1 -- 172.21.15.36:0/3339918026 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe11013aa80 msgr2=0x7fe110144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:56.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.847+0000 7fe10e59c700 1 --2- 172.21.15.36:0/3339918026 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe11013aa80 0x7fe110144e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:56.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.847+0000 7fe10e59c700 1 -- 172.21.15.36:0/3339918026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1100adf40 msgr2=0x7fe110143170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:56.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.847+0000 7fe10e59c700 1 --2- 172.21.15.36:0/3339918026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1100adf40 0x7fe110143170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:56.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.847+0000 7fe10e59c700 1 -- 172.21.15.36:0/3339918026 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10000b180 con 0x7fe11005fe80 2022-06-17T12:55:56.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.847+0000 7fe10e59c700 1 --2- 172.21.15.36:0/3339918026 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe11005fe80 0x7fe1100b5420 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 rx=0x7fe100002f70 tx=0x7fe100014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:56.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.847+0000 7fe0ff7fe700 1 -- 172.21.15.36:0/3339918026 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe100013de0 con 0x7fe11005fe80 2022-06-17T12:55:56.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.847+0000 7fe0ff7fe700 1 -- 172.21.15.36:0/3339918026 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10000d290 con 0x7fe11005fe80 2022-06-17T12:55:56.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.847+0000 7fe1149a5700 1 -- 172.21.15.36:0/3339918026 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe110143810 con 0x7fe11005fe80 2022-06-17T12:55:56.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.851+0000 7fe0ff7fe700 1 -- 172.21.15.36:0/3339918026 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe10001e5f0 con 0x7fe11005fe80 2022-06-17T12:55:56.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.851+0000 7fe1149a5700 1 -- 172.21.15.36:0/3339918026 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1100b0240 con 0x7fe11005fe80 2022-06-17T12:55:56.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.851+0000 7fe1149a5700 1 -- 172.21.15.36:0/3339918026 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe110057410 con 0x7fe11005fe80 2022-06-17T12:55:56.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.851+0000 7fe0ff7fe700 1 -- 172.21.15.36:0/3339918026 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe100025030 con 0x7fe11005fe80 2022-06-17T12:55:56.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.851+0000 7fe0ff7fe700 1 --2- 172.21.15.36:0/3339918026 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe0ec0539d0 0x7fe0ec055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:56.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.851+0000 7fe0ff7fe700 1 -- 172.21.15.36:0/3339918026 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fe10006cd50 con 0x7fe11005fe80 2022-06-17T12:55:56.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.851+0000 7fe10dd9b700 1 --2- 172.21.15.36:0/3339918026 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe0ec0539d0 0x7fe0ec055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:56.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.855+0000 7fe10dd9b700 1 --2- 172.21.15.36:0/3339918026 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe0ec0539d0 0x7fe0ec055e90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fe10400d000 tx=0x7fe104007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:56.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.859+0000 7fe0ff7fe700 1 -- 172.21.15.36:0/3339918026 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe100043b00 con 0x7fe11005fe80 2022-06-17T12:55:56.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.983+0000 7fe1149a5700 1 -- 172.21.15.36:0/3339918026 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fe11005da60 con 0x7fe11005fe80 2022-06-17T12:55:56.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.987+0000 7fe0ff7fe700 1 -- 172.21.15.36:0/3339918026 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v123) v1 ==== 81+0+34218 (secure 0 0 0) 0x7fe100023070 con 0x7fe11005fe80 2022-06-17T12:55:56.977 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:56.977 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:55:56.977 INFO:teuthology.orchestra.run.smithi036.stdout: "epoch": 123, 2022-06-17T12:55:56.977 INFO:teuthology.orchestra.run.smithi036.stdout: "fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", 2022-06-17T12:55:56.978 INFO:teuthology.orchestra.run.smithi036.stdout: "created": "2022-06-17T12:46:30.671408+0000", 2022-06-17T12:55:56.978 INFO:teuthology.orchestra.run.smithi036.stdout: "modified": "2022-06-17T12:55:56.396522+0000", 2022-06-17T12:55:56.978 INFO:teuthology.orchestra.run.smithi036.stdout: "last_up_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:55:56.978 INFO:teuthology.orchestra.run.smithi036.stdout: "last_in_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:55:56.979 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-17T12:55:56.979 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_num": 5799936, 2022-06-17T12:55:56.980 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_set": [ 2022-06-17T12:55:56.980 INFO:teuthology.orchestra.run.smithi036.stdout: "pglog_hardlimit", 2022-06-17T12:55:56.981 INFO:teuthology.orchestra.run.smithi036.stdout: "purged_snapdirs", 2022-06-17T12:55:56.981 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_deletes", 2022-06-17T12:55:56.981 INFO:teuthology.orchestra.run.smithi036.stdout: "sortbitwise" 2022-06-17T12:55:56.982 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:55:56.982 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_version": 6, 2022-06-17T12:55:56.983 INFO:teuthology.orchestra.run.smithi036.stdout: "full_ratio": 0.94999998807907104, 2022-06-17T12:55:56.983 INFO:teuthology.orchestra.run.smithi036.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-17T12:55:56.983 INFO:teuthology.orchestra.run.smithi036.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-17T12:55:56.983 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_snapshot": "", 2022-06-17T12:55:56.985 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_max": 24, 2022-06-17T12:55:56.985 INFO:teuthology.orchestra.run.smithi036.stdout: "max_osd": 8, 2022-06-17T12:55:56.985 INFO:teuthology.orchestra.run.smithi036.stdout: "require_min_compat_client": "jewel", 2022-06-17T12:55:56.985 INFO:teuthology.orchestra.run.smithi036.stdout: "min_compat_client": "jewel", 2022-06-17T12:55:56.986 INFO:teuthology.orchestra.run.smithi036.stdout: "require_osd_release": "octopus", 2022-06-17T12:55:56.986 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:55:56.986 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:56.986 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 1, 2022-06-17T12:55:56.987 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "device_health_metrics", 2022-06-17T12:55:56.987 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:33.881212+0000", 2022-06-17T12:55:56.987 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:55:56.987 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:55:56.988 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:55:56.988 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:55:56.988 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:55:56.988 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:55:56.988 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:55:56.989 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:55:56.989 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 1, 2022-06-17T12:55:56.990 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 1, 2022-06-17T12:55:56.992 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 1, 2022-06-17T12:55:56.992 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 1, 2022-06-17T12:55:56.992 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 1, 2022-06-17T12:55:56.992 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:55:56.993 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:55:56.993 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:55:56.993 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:55:56.994 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:55:56.994 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:55:56.994 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:55:56.995 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:56.996 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "16", 2022-06-17T12:55:56.996 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:55:56.996 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:55:56.996 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:55:56.997 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:55:56.997 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:55:56.997 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:55:56.997 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:55:56.998 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:55:56.998 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:55:56.998 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:55:56.998 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:55:56.999 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:55:57.000 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:55:57.000 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:55:57.001 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:55:57.002 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:55:57.002 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:55:57.003 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:55:57.003 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:55:57.003 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:55:57.004 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:55:57.004 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:55:57.005 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:55:57.005 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:55:57.005 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:55:57.009 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:55:57.009 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.010 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:55:57.010 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:55:57.011 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:55:57.012 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:55:57.012 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:55:57.012 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:55:57.012 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:55:57.023 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:55:57.023 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:55:57.024 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:55:57.024 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:55:57.024 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:55:57.028 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 1 2022-06-17T12:55:57.028 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.028 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:55:57.029 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr_devicehealth": {} 2022-06-17T12:55:57.029 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.029 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.030 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.030 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 2, 2022-06-17T12:55:57.032 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "rbd", 2022-06-17T12:55:57.033 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:44.485460+0000", 2022-06-17T12:55:57.033 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:55:57.034 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:55:57.034 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:55:57.034 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:55:57.034 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:55:57.035 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:55:57.035 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:55:57.035 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:55:57.035 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:55:57.035 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:55:57.036 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:55:57.036 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:55:57.036 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:55:57.036 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:55:57.040 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:55:57.041 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:55:57.041 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:55:57.042 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:55:57.043 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:55:57.044 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:55:57.045 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.045 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "19", 2022-06-17T12:55:57.046 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:55:57.046 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:55:57.046 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:55:57.047 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:55:57.047 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:55:57.048 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:55:57.048 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:55:57.049 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:55:57.049 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:55:57.049 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:55:57.050 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:55:57.050 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:55:57.051 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:55:57.051 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:55:57.052 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:55:57.052 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:55:57.053 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:55:57.053 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:55:57.053 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:55:57.055 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:55:57.055 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:55:57.055 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:55:57.056 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:55:57.056 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:55:57.057 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:55:57.058 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:55:57.059 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.063 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:55:57.065 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:55:57.066 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:55:57.067 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:55:57.067 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:55:57.068 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:55:57.069 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:55:57.070 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:55:57.070 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:55:57.072 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:55:57.072 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:55:57.074 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:55:57.074 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:55:57.077 INFO:teuthology.orchestra.run.smithi036.stdout: "rbd": {} 2022-06-17T12:55:57.078 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.079 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.080 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.080 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 22, 2022-06-17T12:55:57.080 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_data", 2022-06-17T12:55:57.081 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:55:24.417060+0000", 2022-06-17T12:55:57.081 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:55:57.082 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:55:57.082 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:55:57.083 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:55:57.083 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:55:57.084 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:55:57.084 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:55:57.085 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:55:57.085 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:55:57.086 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:55:57.086 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:55:57.087 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:55:57.087 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:55:57.088 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:55:57.088 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:55:57.088 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:55:57.089 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:55:57.090 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:55:57.091 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:55:57.091 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:55:57.091 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.092 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "123", 2022-06-17T12:55:57.093 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:55:57.093 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:55:57.093 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:55:57.094 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:55:57.094 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:55:57.095 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:55:57.095 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:55:57.096 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:55:57.096 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:55:57.097 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:55:57.097 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:55:57.098 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:55:57.098 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:55:57.099 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:55:57.099 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:55:57.099 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:55:57.101 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:55:57.101 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:55:57.102 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:55:57.102 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:55:57.103 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:55:57.103 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:55:57.104 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:55:57.104 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:55:57.105 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:55:57.105 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:55:57.106 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.106 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:55:57.106 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:55:57.107 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:55:57.107 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:55:57.108 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:55:57.108 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:55:57.108 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:55:57.109 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:55:57.109 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:55:57.109 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:55:57.110 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:55:57.110 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:55:57.111 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:55:57.112 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:55:57.112 INFO:teuthology.orchestra.run.smithi036.stdout: "data": "cephfs" 2022-06-17T12:55:57.113 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.113 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.115 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.115 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.116 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 24, 2022-06-17T12:55:57.116 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_metadata", 2022-06-17T12:55:57.117 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:55:55.417088+0000", 2022-06-17T12:55:57.117 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 32769, 2022-06-17T12:55:57.117 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool,creating", 2022-06-17T12:55:57.118 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:55:57.118 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:55:57.119 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:55:57.119 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:55:57.120 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:55:57.120 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:55:57.120 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:55:57.121 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:55:57.122 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:55:57.123 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:55:57.123 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:55:57.124 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:55:57.124 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:55:57.125 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:55:57.125 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:55:57.126 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:55:57.127 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:55:57.127 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:55:57.127 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.128 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "123", 2022-06-17T12:55:57.128 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:55:57.129 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:55:57.129 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:55:57.130 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:55:57.130 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:55:57.131 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:55:57.131 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:55:57.133 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:55:57.133 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:55:57.134 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:55:57.134 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:55:57.135 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:55:57.135 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:55:57.136 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:55:57.136 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:55:57.136 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:55:57.137 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:55:57.137 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:55:57.137 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:55:57.138 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:55:57.139 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:55:57.139 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:55:57.140 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:55:57.140 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:55:57.142 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:55:57.144 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:55:57.144 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.145 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:55:57.145 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:55:57.146 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:55:57.147 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:55:57.147 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:55:57.148 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:55:57.148 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:55:57.149 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:55:57.149 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:55:57.150 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:55:57.150 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:55:57.151 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:55:57.151 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_bias": 4, 2022-06-17T12:55:57.152 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 16, 2022-06-17T12:55:57.152 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_priority": 5 2022-06-17T12:55:57.153 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.154 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:55:57.155 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:55:57.155 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": "cephfs" 2022-06-17T12:55:57.155 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.156 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.156 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.156 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:55:57.156 INFO:teuthology.orchestra.run.smithi036.stdout: "osds": [ 2022-06-17T12:55:57.157 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.157 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:55:57.157 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", 2022-06-17T12:55:57.157 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:55:57.158 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:55:57.158 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:55:57.159 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:55:57.159 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:55:57.159 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:55:57.159 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:55:57.160 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 122, 2022-06-17T12:55:57.160 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:55:57.160 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:55:57.160 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:55:57.161 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.161 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.161 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.161 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6802", 2022-06-17T12:55:57.162 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:55:57.162 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.162 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.162 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.163 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6803", 2022-06-17T12:55:57.163 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:55:57.163 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.163 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.164 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.164 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:55:57.164 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.164 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.165 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.165 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6804", 2022-06-17T12:55:57.165 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:55:57.166 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.166 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.166 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.166 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6805", 2022-06-17T12:55:57.167 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:55:57.167 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.167 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.167 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.168 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:55:57.168 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.168 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.169 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.169 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6808", 2022-06-17T12:55:57.169 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:55:57.169 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.169 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.170 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.172 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6809", 2022-06-17T12:55:57.172 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:55:57.174 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.174 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.175 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.175 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:55:57.175 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.176 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.176 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.176 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6806", 2022-06-17T12:55:57.177 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:55:57.177 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.178 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.178 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.178 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6807", 2022-06-17T12:55:57.189 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:55:57.189 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.190 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.190 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.191 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6803/22619", 2022-06-17T12:55:57.192 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6805/22619", 2022-06-17T12:55:57.192 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6809/22619", 2022-06-17T12:55:57.192 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6807/22619", 2022-06-17T12:55:57.193 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:55:57.193 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:55:57.194 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:55:57.195 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.195 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.196 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.196 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:55:57.197 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", 2022-06-17T12:55:57.197 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:55:57.198 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:55:57.198 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:55:57.199 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:55:57.199 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:55:57.199 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:55:57.200 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:55:57.200 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 122, 2022-06-17T12:55:57.200 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:55:57.201 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:55:57.201 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:55:57.202 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.202 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.203 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.203 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6810", 2022-06-17T12:55:57.204 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:55:57.204 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.204 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.205 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.205 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6811", 2022-06-17T12:55:57.206 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:55:57.206 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.207 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.207 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.207 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:55:57.208 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.208 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.209 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.209 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6812", 2022-06-17T12:55:57.210 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:55:57.210 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.211 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.211 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.212 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6813", 2022-06-17T12:55:57.212 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:55:57.213 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.213 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.214 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.214 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:55:57.215 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.215 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.215 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.216 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6816", 2022-06-17T12:55:57.216 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:55:57.217 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.217 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.218 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.219 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6817", 2022-06-17T12:55:57.219 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:55:57.220 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.220 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.221 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.221 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:55:57.228 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.228 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.229 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.229 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6814", 2022-06-17T12:55:57.229 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:55:57.230 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.230 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.231 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.231 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6815", 2022-06-17T12:55:57.232 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:55:57.232 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.232 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.233 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.233 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6811/22620", 2022-06-17T12:55:57.234 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6813/22620", 2022-06-17T12:55:57.235 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6817/22620", 2022-06-17T12:55:57.235 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6815/22620", 2022-06-17T12:55:57.236 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:55:57.236 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:55:57.237 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:55:57.237 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.238 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.239 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.239 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:55:57.239 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", 2022-06-17T12:55:57.240 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:55:57.240 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:55:57.241 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:55:57.241 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:55:57.242 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:55:57.242 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:55:57.242 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:55:57.243 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 117, 2022-06-17T12:55:57.243 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:55:57.244 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:55:57.244 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:55:57.245 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.245 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.246 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.246 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6826", 2022-06-17T12:55:57.247 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:55:57.247 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.248 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.251 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.252 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6827", 2022-06-17T12:55:57.253 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:55:57.253 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.254 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.254 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.255 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:55:57.255 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.255 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.256 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.256 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6828", 2022-06-17T12:55:57.257 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:55:57.257 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.258 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.258 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.259 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6829", 2022-06-17T12:55:57.259 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:55:57.260 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.260 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.261 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.261 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:55:57.262 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.263 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.263 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.264 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6832", 2022-06-17T12:55:57.264 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:55:57.265 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.265 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.266 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.266 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6833", 2022-06-17T12:55:57.267 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:55:57.267 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.268 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.268 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.269 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:55:57.269 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.270 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.270 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.271 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6830", 2022-06-17T12:55:57.271 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:55:57.271 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.272 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.272 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.273 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6831", 2022-06-17T12:55:57.273 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:55:57.275 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.275 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.276 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.276 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6827/22643", 2022-06-17T12:55:57.276 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6829/22643", 2022-06-17T12:55:57.277 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6833/22643", 2022-06-17T12:55:57.277 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6831/22643", 2022-06-17T12:55:57.278 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:55:57.278 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:55:57.279 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:55:57.279 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.279 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.280 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.280 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:55:57.280 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", 2022-06-17T12:55:57.281 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:55:57.281 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:55:57.282 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:55:57.282 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:55:57.283 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:55:57.283 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:55:57.284 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:55:57.285 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 122, 2022-06-17T12:55:57.286 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:55:57.286 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:55:57.286 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:55:57.287 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.288 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.288 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.289 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6818", 2022-06-17T12:55:57.290 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:55:57.290 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.291 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.291 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.292 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6819", 2022-06-17T12:55:57.292 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:55:57.292 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.293 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.293 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.294 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:55:57.295 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.295 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.297 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.297 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6820", 2022-06-17T12:55:57.303 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:55:57.304 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.304 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.305 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.305 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6821", 2022-06-17T12:55:57.306 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:55:57.306 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.306 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.307 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.307 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:55:57.308 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.308 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.309 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.309 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6824", 2022-06-17T12:55:57.310 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:55:57.311 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.311 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.312 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.312 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6825", 2022-06-17T12:55:57.312 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:55:57.313 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.313 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.313 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.313 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:55:57.314 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.314 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.315 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.316 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6822", 2022-06-17T12:55:57.316 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:55:57.317 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.317 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.317 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.318 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6823", 2022-06-17T12:55:57.318 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:55:57.318 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.319 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.319 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.319 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6819/22646", 2022-06-17T12:55:57.319 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6821/22646", 2022-06-17T12:55:57.320 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6825/22646", 2022-06-17T12:55:57.321 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6823/22646", 2022-06-17T12:55:57.321 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:55:57.322 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:55:57.322 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:55:57.322 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.323 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.323 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.324 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:55:57.324 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", 2022-06-17T12:55:57.324 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:55:57.325 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:55:57.325 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:55:57.326 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:55:57.326 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:55:57.327 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:55:57.327 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:55:57.327 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 117, 2022-06-17T12:55:57.328 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:55:57.328 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:55:57.328 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:55:57.329 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.329 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.330 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.330 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6808", 2022-06-17T12:55:57.337 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:55:57.337 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.337 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.338 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.338 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6809", 2022-06-17T12:55:57.338 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:55:57.338 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.339 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.339 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.339 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:55:57.339 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.340 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.340 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.340 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6810", 2022-06-17T12:55:57.340 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:55:57.340 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.341 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.341 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.341 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6811", 2022-06-17T12:55:57.341 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:55:57.342 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.343 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.343 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.343 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:55:57.343 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.343 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.344 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.344 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6814", 2022-06-17T12:55:57.344 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:55:57.344 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.345 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.345 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.345 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6815", 2022-06-17T12:55:57.345 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:55:57.346 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.346 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.346 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.346 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:55:57.346 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.348 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.348 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.349 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6812", 2022-06-17T12:55:57.349 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:55:57.349 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.349 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.350 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.350 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6813", 2022-06-17T12:55:57.350 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:55:57.350 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.350 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.351 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.351 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6809/22812", 2022-06-17T12:55:57.351 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6811/22812", 2022-06-17T12:55:57.351 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6815/22812", 2022-06-17T12:55:57.352 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6813/22812", 2022-06-17T12:55:57.352 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:55:57.352 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:55:57.352 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:55:57.353 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.353 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.353 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.353 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:55:57.354 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", 2022-06-17T12:55:57.354 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:55:57.354 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:55:57.354 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:55:57.355 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:55:57.355 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:55:57.355 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:55:57.355 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:55:57.355 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 122, 2022-06-17T12:55:57.356 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:55:57.356 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:55:57.356 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:55:57.356 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.357 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.357 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.357 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6800", 2022-06-17T12:55:57.357 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:55:57.357 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.360 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.361 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.362 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6801", 2022-06-17T12:55:57.362 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:55:57.363 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.363 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.364 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.364 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:55:57.365 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.365 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.366 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.366 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6802", 2022-06-17T12:55:57.367 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:55:57.367 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.369 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.369 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.369 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6803", 2022-06-17T12:55:57.370 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:55:57.371 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.371 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.372 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.372 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:55:57.373 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.377 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.378 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.379 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6806", 2022-06-17T12:55:57.380 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:55:57.380 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.381 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.381 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.382 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6807", 2022-06-17T12:55:57.382 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:55:57.383 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.383 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.384 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.384 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:55:57.385 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.385 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.386 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.387 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6804", 2022-06-17T12:55:57.387 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:55:57.387 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.388 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.388 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.388 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6805", 2022-06-17T12:55:57.390 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:55:57.390 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.391 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.393 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.393 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6801/22813", 2022-06-17T12:55:57.393 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6803/22813", 2022-06-17T12:55:57.393 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6807/22813", 2022-06-17T12:55:57.394 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6805/22813", 2022-06-17T12:55:57.394 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:55:57.394 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:55:57.395 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:55:57.395 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.395 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.395 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.396 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:55:57.396 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", 2022-06-17T12:55:57.396 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:55:57.396 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:55:57.397 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:55:57.397 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:55:57.397 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:55:57.397 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:55:57.398 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:55:57.398 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 122, 2022-06-17T12:55:57.398 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:55:57.398 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:55:57.399 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:55:57.399 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.399 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.399 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.399 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6816", 2022-06-17T12:55:57.400 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:55:57.400 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.400 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.401 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.401 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6817", 2022-06-17T12:55:57.402 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:55:57.402 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.402 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.403 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.403 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:55:57.403 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.403 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.403 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.404 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6818", 2022-06-17T12:55:57.404 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:55:57.404 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.404 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.405 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.405 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6819", 2022-06-17T12:55:57.405 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:55:57.405 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.406 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.406 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.406 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:55:57.406 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.406 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.407 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.407 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6822", 2022-06-17T12:55:57.407 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:55:57.407 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.407 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.408 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.408 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6823", 2022-06-17T12:55:57.408 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:55:57.408 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.409 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.409 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.409 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:55:57.409 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.409 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.410 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.410 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6820", 2022-06-17T12:55:57.410 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:55:57.410 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.411 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.411 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.412 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6821", 2022-06-17T12:55:57.412 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:55:57.413 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.413 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.413 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.413 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6817/22822", 2022-06-17T12:55:57.414 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6819/22822", 2022-06-17T12:55:57.414 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6823/22822", 2022-06-17T12:55:57.414 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6821/22822", 2022-06-17T12:55:57.415 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:55:57.415 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:55:57.415 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:55:57.415 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.416 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.417 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.420 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:55:57.420 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", 2022-06-17T12:55:57.420 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:55:57.422 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:55:57.422 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:55:57.422 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:55:57.423 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:55:57.423 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:55:57.424 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:55:57.424 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 122, 2022-06-17T12:55:57.424 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:55:57.424 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:55:57.425 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:55:57.425 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.425 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.426 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.426 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6824", 2022-06-17T12:55:57.426 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:55:57.427 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.427 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.428 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.428 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6825", 2022-06-17T12:55:57.428 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:55:57.428 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.428 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.429 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.429 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:55:57.429 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.429 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.430 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.430 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6826", 2022-06-17T12:55:57.430 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:55:57.430 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.431 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.431 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.431 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6827", 2022-06-17T12:55:57.431 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:55:57.432 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.432 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.433 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.433 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:55:57.434 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.435 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.435 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.435 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6830", 2022-06-17T12:55:57.435 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:55:57.435 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.436 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.436 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.436 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6831", 2022-06-17T12:55:57.436 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:55:57.437 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.437 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.437 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.437 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:55:57.438 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:55:57.438 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.438 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:55:57.439 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6828", 2022-06-17T12:55:57.439 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:55:57.439 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.440 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.440 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:55:57.441 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6829", 2022-06-17T12:55:57.441 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:55:57.442 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.443 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.444 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.445 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6825/22837", 2022-06-17T12:55:57.445 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6827/22837", 2022-06-17T12:55:57.446 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6831/22837", 2022-06-17T12:55:57.446 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6829/22837", 2022-06-17T12:55:57.447 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:55:57.447 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:55:57.447 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:55:57.448 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:55:57.448 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.449 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:55:57.449 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_xinfo": [ 2022-06-17T12:55:57.449 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.450 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:55:57.452 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:55:57.452 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:55:57.453 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:55:57.453 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:55:57.454 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:55:57.455 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.291461+0000", 2022-06-17T12:55:57.456 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:55:57.456 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.457 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.457 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:55:57.457 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:55:57.458 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:55:57.458 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:55:57.459 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:55:57.460 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:55:57.460 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.537912+0000", 2022-06-17T12:55:57.461 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:55:57.461 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.461 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.461 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:55:57.462 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:55:57.462 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:55:57.462 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:55:57.462 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:55:57.465 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:55:57.466 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.606947+0000", 2022-06-17T12:55:57.467 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:55:57.467 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.467 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.467 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:55:57.468 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:55:57.468 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:55:57.468 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:55:57.468 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:55:57.469 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:55:57.474 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.514412+0000", 2022-06-17T12:55:57.474 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:55:57.475 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.475 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.478 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:55:57.478 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:55:57.479 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:55:57.479 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:55:57.479 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:55:57.480 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:55:57.480 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.394645+0000", 2022-06-17T12:55:57.481 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:55:57.481 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.482 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.482 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:55:57.483 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:55:57.483 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:55:57.483 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:55:57.483 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:55:57.484 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:55:57.484 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.343806+0000", 2022-06-17T12:55:57.484 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:55:57.485 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.485 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.485 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:55:57.485 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:55:57.486 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:55:57.486 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:55:57.486 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:55:57.486 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:55:57.487 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.591864+0000", 2022-06-17T12:55:57.487 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:55:57.487 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.487 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:55:57.488 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:55:57.488 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:55:57.488 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:55:57.488 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:55:57.489 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:55:57.489 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:55:57.489 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.607045+0000", 2022-06-17T12:55:57.489 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:55:57.490 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.490 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:55:57.490 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap": [], 2022-06-17T12:55:57.491 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap_items": [], 2022-06-17T12:55:57.491 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_temp": [], 2022-06-17T12:55:57.492 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_temp": [], 2022-06-17T12:55:57.492 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklist": { 2022-06-17T12:55:57.492 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.36:6835/1342585509": "2022-06-18T12:55:41.828746+0000", 2022-06-17T12:55:57.492 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.36:6834/1342585509": "2022-06-18T12:55:41.828746+0000" 2022-06-17T12:55:57.493 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.493 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profiles": { 2022-06-17T12:55:57.493 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs_data": { 2022-06-17T12:55:57.493 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-device-class": "", 2022-06-17T12:55:57.494 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:55:57.494 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-root": "default", 2022-06-17T12:55:57.494 INFO:teuthology.orchestra.run.smithi036.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-17T12:55:57.495 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:55:57.495 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "2", 2022-06-17T12:55:57.495 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:55:57.495 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:55:57.496 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van", 2022-06-17T12:55:57.496 INFO:teuthology.orchestra.run.smithi036.stdout: "w": "8" 2022-06-17T12:55:57.496 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.497 INFO:teuthology.orchestra.run.smithi036.stdout: "default": { 2022-06-17T12:55:57.497 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:55:57.497 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:55:57.497 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "1", 2022-06-17T12:55:57.498 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:55:57.498 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:55:57.499 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van" 2022-06-17T12:55:57.499 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:55:57.499 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:55:57.499 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps_queue": [], 2022-06-17T12:55:57.500 INFO:teuthology.orchestra.run.smithi036.stdout: "new_removed_snaps": [], 2022-06-17T12:55:57.500 INFO:teuthology.orchestra.run.smithi036.stdout: "new_purged_snaps": [], 2022-06-17T12:55:57.500 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_node_flags": {}, 2022-06-17T12:55:57.501 INFO:teuthology.orchestra.run.smithi036.stdout: "device_class_flags": {} 2022-06-17T12:55:57.501 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:55:57.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.987+0000 7fe1149a5700 1 -- 172.21.15.36:0/3339918026 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe0ec0539d0 msgr2=0x7fe0ec055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:57.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.987+0000 7fe1149a5700 1 --2- 172.21.15.36:0/3339918026 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe0ec0539d0 0x7fe0ec055e90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fe10400d000 tx=0x7fe104007b90).stop 2022-06-17T12:55:57.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.987+0000 7fe1149a5700 1 -- 172.21.15.36:0/3339918026 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe11005fe80 msgr2=0x7fe1100b5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:57.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.987+0000 7fe1149a5700 1 --2- 172.21.15.36:0/3339918026 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe11005fe80 0x7fe1100b5420 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 rx=0x7fe100002f70 tx=0x7fe100014000).stop 2022-06-17T12:55:57.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.987+0000 7fe1149a5700 1 -- 172.21.15.36:0/3339918026 shutdown_connections 2022-06-17T12:55:57.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.987+0000 7fe1149a5700 1 --2- 172.21.15.36:0/3339918026 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe0ec0539d0 0x7fe0ec055e90 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:57.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.987+0000 7fe1149a5700 1 --2- 172.21.15.36:0/3339918026 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe11013aa80 0x7fe110144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:57.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.987+0000 7fe1149a5700 1 --2- 172.21.15.36:0/3339918026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1100adf40 0x7fe110143170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:57.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.987+0000 7fe1149a5700 1 --2- 172.21.15.36:0/3339918026 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe11005fe80 0x7fe1100b5420 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:57.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.987+0000 7fe1149a5700 1 -- 172.21.15.36:0/3339918026 >> 172.21.15.36:0/3339918026 conn(0x7fe110060990 msgr2=0x7fe11013f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:57.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.987+0000 7fe1149a5700 1 -- 172.21.15.36:0/3339918026 shutdown_connections 2022-06-17T12:55:57.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:56.987+0000 7fe1149a5700 1 -- 172.21.15.36:0/3339918026 wait complete. 2022-06-17T12:55:57.507 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:55:57.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdda17c700 1 Processor -- start 2022-06-17T12:55:57.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdda17c700 1 -- start start 2022-06-17T12:55:57.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdda17c700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbdd4060340 0x7fbdd413ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:57.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdda17c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbdd413f100 0x7fbdd4142aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:57.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdda17c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdd40ad330 0x7fbdd4142f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:57.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdda17c700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdd4060720 con 0x7fbdd413f100 2022-06-17T12:55:57.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdda17c700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdd4062d30 con 0x7fbdd4060340 2022-06-17T12:55:57.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdda17c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd4062fd0 con 0x7fbdd40ad330 2022-06-17T12:55:57.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdd2ffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbdd413f100 0x7fbdd4142aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:57.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdd2ffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbdd413f100 0x7fbdd4142aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38910/0 (socket says 172.21.15.36:38910) 2022-06-17T12:55:57.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdd2ffd700 1 -- 172.21.15.36:0/3318167150 learned_addr learned my addr 172.21.15.36:0/3318167150 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:57.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdd3fff700 1 --2- 172.21.15.36:0/3318167150 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdd40ad330 0x7fbdd4142f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:57.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdd37fe700 1 --2- 172.21.15.36:0/3318167150 >> v2:172.21.15.70:3300/0 conn(0x7fbdd4060340 0x7fbdd413ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:57.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdd2ffd700 1 -- 172.21.15.36:0/3318167150 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdd40ad330 msgr2=0x7fbdd4142f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:57.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdd2ffd700 1 --2- 172.21.15.36:0/3318167150 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdd40ad330 0x7fbdd4142f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:57.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdd2ffd700 1 -- 172.21.15.36:0/3318167150 >> v2:172.21.15.70:3300/0 conn(0x7fbdd4060340 msgr2=0x7fbdd413ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:57.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdd2ffd700 1 --2- 172.21.15.36:0/3318167150 >> v2:172.21.15.70:3300/0 conn(0x7fbdd4060340 0x7fbdd413ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:57.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdd2ffd700 1 -- 172.21.15.36:0/3318167150 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd4063710 con 0x7fbdd413f100 2022-06-17T12:55:57.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdd3fff700 1 --2- 172.21.15.36:0/3318167150 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdd40ad330 0x7fbdd4142f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:55:57.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdd2ffd700 1 --2- 172.21.15.36:0/3318167150 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbdd413f100 0x7fbdd4142aa0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7fbdbc002f70 tx=0x7fbdbc006eb0).ready entity=mon.0 client_cookie=24de227c53dc14f5 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:57.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdd27fc700 1 -- 172.21.15.36:0/3318167150 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdbc00d570 con 0x7fbdd413f100 2022-06-17T12:55:57.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdd27fc700 1 -- 172.21.15.36:0/3318167150 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdbc000e90 con 0x7fbdd413f100 2022-06-17T12:55:57.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdd27fc700 1 -- 172.21.15.36:0/3318167150 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdbc00d900 con 0x7fbdd413f100 2022-06-17T12:55:57.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdda17c700 1 -- 172.21.15.36:0/3318167150 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbdd413f100 msgr2=0x7fbdd4142aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:57.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdda17c700 1 --2- 172.21.15.36:0/3318167150 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbdd413f100 0x7fbdd4142aa0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7fbdbc002f70 tx=0x7fbdbc006eb0).stop 2022-06-17T12:55:57.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdda17c700 1 -- 172.21.15.36:0/3318167150 shutdown_connections 2022-06-17T12:55:57.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdda17c700 1 --2- 172.21.15.36:0/3318167150 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdd40ad330 0x7fbdd4142f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:57.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdda17c700 1 --2- 172.21.15.36:0/3318167150 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbdd413f100 0x7fbdd4142aa0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:57.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdda17c700 1 --2- 172.21.15.36:0/3318167150 >> v2:172.21.15.70:3300/0 conn(0x7fbdd4060340 0x7fbdd413ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:57.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdda17c700 1 -- 172.21.15.36:0/3318167150 >> 172.21.15.36:0/3318167150 conn(0x7fbdd40b53e0 msgr2=0x7fbdd40b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:57.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdda17c700 1 -- 172.21.15.36:0/3318167150 shutdown_connections 2022-06-17T12:55:57.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.635+0000 7fbdda17c700 1 -- 172.21.15.36:0/3318167150 wait complete. 2022-06-17T12:55:57.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.639+0000 7fbdda17c700 1 Processor -- start 2022-06-17T12:55:57.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.639+0000 7fbdda17c700 1 -- start start 2022-06-17T12:55:57.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.639+0000 7fbdda17c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbdd4060340 0x7fbdd413b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:57.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.639+0000 7fbdda17c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdd40ad330 0x7fbdd413d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:57.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.639+0000 7fbdda17c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdd413f100 0x7fbdd413d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:57.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.639+0000 7fbdda17c700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd413da20 con 0x7fbdd4060340 2022-06-17T12:55:57.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.639+0000 7fbdda17c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd40661a0 con 0x7fbdd413f100 2022-06-17T12:55:57.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.639+0000 7fbdda17c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd4066440 con 0x7fbdd40ad330 2022-06-17T12:55:57.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.639+0000 7fbdd37fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbdd4060340 0x7fbdd413b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:57.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.639+0000 7fbdd37fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbdd4060340 0x7fbdd413b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38918/0 (socket says 172.21.15.36:38918) 2022-06-17T12:55:57.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.639+0000 7fbdd37fe700 1 -- 172.21.15.36:0/967020174 learned_addr learned my addr 172.21.15.36:0/967020174 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:57.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.639+0000 7fbdd2ffd700 1 --2- 172.21.15.36:0/967020174 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdd40ad330 0x7fbdd413d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:57.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.639+0000 7fbdd3fff700 1 --2- 172.21.15.36:0/967020174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdd413f100 0x7fbdd413d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:57.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.639+0000 7fbdd37fe700 1 -- 172.21.15.36:0/967020174 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdd40ad330 msgr2=0x7fbdd413d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:57.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.639+0000 7fbdd37fe700 1 --2- 172.21.15.36:0/967020174 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdd40ad330 0x7fbdd413d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:57.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.639+0000 7fbdd37fe700 1 -- 172.21.15.36:0/967020174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdd413f100 msgr2=0x7fbdd413d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:57.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.639+0000 7fbdd37fe700 1 --2- 172.21.15.36:0/967020174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdd413f100 0x7fbdd413d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:57.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.639+0000 7fbdd37fe700 1 -- 172.21.15.36:0/967020174 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdbc00b1e0 con 0x7fbdd4060340 2022-06-17T12:55:57.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.643+0000 7fbdd37fe700 1 --2- 172.21.15.36:0/967020174 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbdd4060340 0x7fbdd413b720 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 rx=0x7fbdc40030b0 tx=0x7fbdc4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:57.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.643+0000 7fbdd0ff9700 1 -- 172.21.15.36:0/967020174 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdc4011780 con 0x7fbdd4060340 2022-06-17T12:55:57.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.643+0000 7fbdd0ff9700 1 -- 172.21.15.36:0/967020174 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc40118e0 con 0x7fbdd4060340 2022-06-17T12:55:57.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.643+0000 7fbdda17c700 1 -- 172.21.15.36:0/967020174 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd40666e0 con 0x7fbdd4060340 2022-06-17T12:55:57.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.643+0000 7fbdd0ff9700 1 -- 172.21.15.36:0/967020174 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdc4011c40 con 0x7fbdd4060340 2022-06-17T12:55:57.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.643+0000 7fbdda17c700 1 -- 172.21.15.36:0/967020174 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd413ca80 con 0x7fbdd4060340 2022-06-17T12:55:57.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.643+0000 7fbdda17c700 1 -- 172.21.15.36:0/967020174 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd413ce40 con 0x7fbdd4060340 2022-06-17T12:55:57.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.643+0000 7fbdd0ff9700 1 -- 172.21.15.36:0/967020174 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fbdc4012420 con 0x7fbdd4060340 2022-06-17T12:55:57.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.643+0000 7fbdd0ff9700 1 --2- 172.21.15.36:0/967020174 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbdb8053940 0x7fbdb8055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:57.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.643+0000 7fbdd0ff9700 1 -- 172.21.15.36:0/967020174 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fbdc406bd50 con 0x7fbdd4060340 2022-06-17T12:55:57.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.643+0000 7fbdd2ffd700 1 --2- 172.21.15.36:0/967020174 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbdb8053940 0x7fbdb8055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:57.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.647+0000 7fbdd2ffd700 1 --2- 172.21.15.36:0/967020174 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbdb8053940 0x7fbdb8055e00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fbdbc01c000 tx=0x7fbdbc015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:57.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.647+0000 7fbdd0ff9700 1 -- 172.21.15.36:0/967020174 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbdc4042aa0 con 0x7fbdd4060340 2022-06-17T12:55:57.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.815+0000 7fbdda17c700 1 -- 172.21.15.36:0/967020174 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbdd4062960 con 0x7fbdd4060340 2022-06-17T12:55:57.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.815+0000 7fbdd0ff9700 1 -- 172.21.15.36:0/967020174 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+1426 (secure 0 0 0) 0x7fbdc40460c0 con 0x7fbdd4060340 2022-06-17T12:55:57.806 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:57.807 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":65,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:55:56.396693+0000","modified":"2022-06-17T12:55:56.396763+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[22],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2022-06-17T12:55:57.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.815+0000 7fbdda17c700 1 -- 172.21.15.36:0/967020174 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbdb8053940 msgr2=0x7fbdb8055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:57.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.815+0000 7fbdda17c700 1 --2- 172.21.15.36:0/967020174 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbdb8053940 0x7fbdb8055e00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fbdbc01c000 tx=0x7fbdbc015000).stop 2022-06-17T12:55:57.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.815+0000 7fbdda17c700 1 -- 172.21.15.36:0/967020174 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbdd4060340 msgr2=0x7fbdd413b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:57.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.815+0000 7fbdda17c700 1 --2- 172.21.15.36:0/967020174 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbdd4060340 0x7fbdd413b720 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 rx=0x7fbdc40030b0 tx=0x7fbdc4008040).stop 2022-06-17T12:55:57.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.819+0000 7fbdda17c700 1 -- 172.21.15.36:0/967020174 shutdown_connections 2022-06-17T12:55:57.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.819+0000 7fbdda17c700 1 --2- 172.21.15.36:0/967020174 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbdb8053940 0x7fbdb8055e00 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:57.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.819+0000 7fbdda17c700 1 --2- 172.21.15.36:0/967020174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdd413f100 0x7fbdd413d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:57.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.819+0000 7fbdda17c700 1 --2- 172.21.15.36:0/967020174 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdd40ad330 0x7fbdd413d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:57.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.819+0000 7fbdda17c700 1 --2- 172.21.15.36:0/967020174 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbdd4060340 0x7fbdd413b720 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:57.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.819+0000 7fbdda17c700 1 -- 172.21.15.36:0/967020174 >> 172.21.15.36:0/967020174 conn(0x7fbdd40b53e0 msgr2=0x7fbdd40608f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:57.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.819+0000 7fbdda17c700 1 -- 172.21.15.36:0/967020174 shutdown_connections 2022-06-17T12:55:57.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.819+0000 7fbdda17c700 1 -- 172.21.15.36:0/967020174 wait complete. 2022-06-17T12:55:57.813 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 65 2022-06-17T12:55:57.817 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-17T12:55:57.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.939+0000 7f8833fff700 1 Processor -- start 2022-06-17T12:55:57.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.939+0000 7f8833fff700 1 -- start start 2022-06-17T12:55:57.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.939+0000 7f8833fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88340b2cd0 0x7f88341465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:57.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.939+0000 7f8833fff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88340b25e0 con 0x7f8834146ae0 2022-06-17T12:55:57.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.939+0000 7f8833fff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88340ad790 con 0x7f88340b2200 2022-06-17T12:55:57.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.939+0000 7f8833fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88340aef40 con 0x7f88340b2cd0 2022-06-17T12:55:57.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.939+0000 7f88337fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f8834146ae0 0x7f8834146f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56458/0 (socket says 172.21.15.36:56458) 2022-06-17T12:55:57.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.939+0000 7f88337fe700 1 -- 172.21.15.36:0/506428987 learned_addr learned my addr 172.21.15.36:0/506428987 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:57.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.939+0000 7f88327fc700 1 --2- 172.21.15.36:0/506428987 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88340b2cd0 0x7f88341465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:57.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.939+0000 7f8831ffb700 1 -- 172.21.15.36:0/506428987 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2568258790 0 0) 0x7f88340b25e0 con 0x7f8834146ae0 2022-06-17T12:55:57.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.939+0000 7f8831ffb700 1 -- 172.21.15.36:0/506428987 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8814003260 con 0x7f8834146ae0 2022-06-17T12:55:57.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.939+0000 7f88327fc700 1 -- 172.21.15.36:0/506428987 >> v1:172.21.15.70:6789/0 conn(0x7f88340b2200 legacy=0x7f88340af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:57.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.939+0000 7f88327fc700 1 -- 172.21.15.36:0/506428987 >> v1:172.21.15.36:6789/0 conn(0x7f8834146ae0 legacy=0x7f8834146f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:57.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.939+0000 7f8831ffb700 1 -- 172.21.15.36:0/506428987 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2759823639 0 0) 0x7f88340ad790 con 0x7f88340b2200 2022-06-17T12:55:57.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.939+0000 7f88327fc700 1 -- 172.21.15.36:0/506428987 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88340afaf0 con 0x7f88340b2cd0 2022-06-17T12:55:57.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.939+0000 7f88327fc700 1 --2- 172.21.15.36:0/506428987 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88340b2cd0 0x7f88341465f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f88240032f0 tx=0x7f88240091d0).ready entity=mon.2 client_cookie=5cc260afc7ab689f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:57.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.939+0000 7f8831ffb700 1 -- 172.21.15.36:0/506428987 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f882400b6c0 con 0x7f88340b2cd0 2022-06-17T12:55:57.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.939+0000 7f8831ffb700 1 -- 172.21.15.36:0/506428987 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882400b820 con 0x7f88340b2cd0 2022-06-17T12:55:57.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.939+0000 7f8831ffb700 1 -- 172.21.15.36:0/506428987 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f882400b990 con 0x7f88340b2cd0 2022-06-17T12:55:57.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.939+0000 7f8833fff700 1 -- 172.21.15.36:0/506428987 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88340b2cd0 msgr2=0x7f88341465f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:57.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.939+0000 7f8833fff700 1 --2- 172.21.15.36:0/506428987 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88340b2cd0 0x7f88341465f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f88240032f0 tx=0x7f88240091d0).stop 2022-06-17T12:55:57.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.939+0000 7f8833fff700 1 -- 172.21.15.36:0/506428987 shutdown_connections 2022-06-17T12:55:57.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.939+0000 7f8833fff700 1 --2- 172.21.15.36:0/506428987 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88340b2cd0 0x7f88341465f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:57.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.939+0000 7f8833fff700 1 -- 172.21.15.36:0/506428987 >> 172.21.15.36:0/506428987 conn(0x7f883405f770 msgr2=0x7f883405fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:57.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f8833fff700 1 -- 172.21.15.36:0/506428987 shutdown_connections 2022-06-17T12:55:57.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f8833fff700 1 -- 172.21.15.36:0/506428987 wait complete. 2022-06-17T12:55:57.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f8833fff700 1 Processor -- start 2022-06-17T12:55:57.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f8833fff700 1 -- start start 2022-06-17T12:55:57.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f8833fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88340b2200 0x7f88340a9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:57.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f8833fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88340b2cd0 0x7f88341458d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:57.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f8833fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8834146ae0 0x7f8834142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:57.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f8833fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8834145dc0 con 0x7f88340b2cd0 2022-06-17T12:55:57.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f8833fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88340a8e60 con 0x7f8834146ae0 2022-06-17T12:55:57.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f8833fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88340a9100 con 0x7f88340b2200 2022-06-17T12:55:57.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f88327fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88340b2cd0 0x7f88341458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:57.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f8832ffd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88340b2200 0x7f88340a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:57.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f88327fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88340b2cd0 0x7f88341458d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38932/0 (socket says 172.21.15.36:38932) 2022-06-17T12:55:57.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f88327fc700 1 -- 172.21.15.36:0/2963159463 learned_addr learned my addr 172.21.15.36:0/2963159463 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:57.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f88337fe700 1 --2- 172.21.15.36:0/2963159463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8834146ae0 0x7f8834142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:57.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f8832ffd700 1 -- 172.21.15.36:0/2963159463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8834146ae0 msgr2=0x7f8834142250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:57.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f8832ffd700 1 --2- 172.21.15.36:0/2963159463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8834146ae0 0x7f8834142250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:57.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f8832ffd700 1 -- 172.21.15.36:0/2963159463 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88340b2cd0 msgr2=0x7f88341458d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:57.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f8832ffd700 1 --2- 172.21.15.36:0/2963159463 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88340b2cd0 0x7f88341458d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:57.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f8832ffd700 1 -- 172.21.15.36:0/2963159463 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8824009470 con 0x7f88340b2200 2022-06-17T12:55:57.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f8832ffd700 1 --2- 172.21.15.36:0/2963159463 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88340b2200 0x7f88340a9840 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f881c000ed0 tx=0x7f881c007a00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:57.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f882bfff700 1 -- 172.21.15.36:0/2963159463 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f881c012400 con 0x7f88340b2200 2022-06-17T12:55:57.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f882bfff700 1 -- 172.21.15.36:0/2963159463 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881c012560 con 0x7f88340b2200 2022-06-17T12:55:57.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f8833fff700 1 -- 172.21.15.36:0/2963159463 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88340a93a0 con 0x7f88340b2200 2022-06-17T12:55:57.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f882bfff700 1 -- 172.21.15.36:0/2963159463 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f881c0128c0 con 0x7f88340b2200 2022-06-17T12:55:57.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.943+0000 7f8833fff700 1 -- 172.21.15.36:0/2963159463 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88341428e0 con 0x7f88340b2200 2022-06-17T12:55:57.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.947+0000 7f8833fff700 1 -- 172.21.15.36:0/2963159463 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88340a9530 con 0x7f88340b2200 2022-06-17T12:55:57.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.947+0000 7f882bfff700 1 -- 172.21.15.36:0/2963159463 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f881c019030 con 0x7f88340b2200 2022-06-17T12:55:57.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.947+0000 7f882bfff700 1 --2- 172.21.15.36:0/2963159463 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8818053a40 0x7f8818055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:57.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.947+0000 7f882bfff700 1 -- 172.21.15.36:0/2963159463 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f881c06b230 con 0x7f88340b2200 2022-06-17T12:55:57.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.947+0000 7f88327fc700 1 --2- 172.21.15.36:0/2963159463 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8818053a40 0x7f8818055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:57.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.947+0000 7f88327fc700 1 --2- 172.21.15.36:0/2963159463 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8818053a40 0x7f8818055f00 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f882401b000 tx=0x7f8824016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:57.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:57.955+0000 7f882bfff700 1 -- 172.21.15.36:0/2963159463 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f881c041ff0 con 0x7f88340b2200 2022-06-17T12:55:58.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.123+0000 7f8833fff700 1 -- 172.21.15.36:0/2963159463 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f88340b1950 con 0x7f88340b2200 2022-06-17T12:55:58.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.411+0000 7f882bfff700 1 -- 172.21.15.36:0/2963159463 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v66) v1 ==== 161+0+0 (secure 0 0 0) 0x7f881c045610 con 0x7f88340b2200 2022-06-17T12:55:58.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.415+0000 7f8833fff700 1 -- 172.21.15.36:0/2963159463 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8818053a40 msgr2=0x7f8818055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:58.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.415+0000 7f8833fff700 1 --2- 172.21.15.36:0/2963159463 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8818053a40 0x7f8818055f00 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f882401b000 tx=0x7f8824016000).stop 2022-06-17T12:55:58.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.415+0000 7f8833fff700 1 -- 172.21.15.36:0/2963159463 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88340b2200 msgr2=0x7f88340a9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:58.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.415+0000 7f8833fff700 1 --2- 172.21.15.36:0/2963159463 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88340b2200 0x7f88340a9840 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f881c000ed0 tx=0x7f881c007a00).stop 2022-06-17T12:55:58.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.415+0000 7f8833fff700 1 -- 172.21.15.36:0/2963159463 shutdown_connections 2022-06-17T12:55:58.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.415+0000 7f8833fff700 1 --2- 172.21.15.36:0/2963159463 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8818053a40 0x7f8818055f00 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:58.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.415+0000 7f8833fff700 1 --2- 172.21.15.36:0/2963159463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8834146ae0 0x7f8834142250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:58.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.415+0000 7f8833fff700 1 --2- 172.21.15.36:0/2963159463 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88340b2cd0 0x7f88341458d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:58.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.415+0000 7f8833fff700 1 --2- 172.21.15.36:0/2963159463 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88340b2200 0x7f88340a9840 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:58.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.415+0000 7f8833fff700 1 -- 172.21.15.36:0/2963159463 >> 172.21.15.36:0/2963159463 conn(0x7f883405f770 msgr2=0x7f883414cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:58.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.415+0000 7f8833fff700 1 -- 172.21.15.36:0/2963159463 shutdown_connections 2022-06-17T12:55:58.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.415+0000 7f8833fff700 1 -- 172.21.15.36:0/2963159463 wait complete. 2022-06-17T12:55:58.413 INFO:teuthology.orchestra.run.smithi036.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-17T12:55:58.430 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:55:58.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.551+0000 7f6d8091d700 1 Processor -- start 2022-06-17T12:55:58.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.551+0000 7f6d8091d700 1 -- start start 2022-06-17T12:55:58.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d8091d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d7c13cb70 0x7f6d7c143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:58.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d8091d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6d7c0aef60 0x7f6d7c0b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:58.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d8091d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f6d7c0ba2e0 0x7f6d7c1442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:58.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d8091d700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d7c0af4b0 con 0x7f6d7c0ba2e0 2022-06-17T12:55:58.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d8091d700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d7c142f50 con 0x7f6d7c0aef60 2022-06-17T12:55:58.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d8091d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c1431f0 con 0x7f6d7c13cb70 2022-06-17T12:55:58.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d7bfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f6d7c0ba2e0 0x7f6d7c1442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:58.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d7bfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f6d7c0ba2e0 0x7f6d7c1442a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38942/0 (socket says 172.21.15.36:38942) 2022-06-17T12:55:58.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d7bfff700 1 -- 172.21.15.36:0/3852037763 learned_addr learned my addr 172.21.15.36:0/3852037763 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:58.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d7b7fe700 1 --2- 172.21.15.36:0/3852037763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d7c13cb70 0x7f6d7c143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:58.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d7affd700 1 --2- 172.21.15.36:0/3852037763 >> v2:172.21.15.70:3300/0 conn(0x7f6d7c0aef60 0x7f6d7c0b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:58.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d7bfff700 1 -- 172.21.15.36:0/3852037763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d7c13cb70 msgr2=0x7f6d7c143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:58.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d7bfff700 1 --2- 172.21.15.36:0/3852037763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d7c13cb70 0x7f6d7c143690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:58.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d7bfff700 1 -- 172.21.15.36:0/3852037763 >> v2:172.21.15.70:3300/0 conn(0x7f6d7c0aef60 msgr2=0x7f6d7c0b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:58.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d7bfff700 1 --2- 172.21.15.36:0/3852037763 >> v2:172.21.15.70:3300/0 conn(0x7f6d7c0aef60 0x7f6d7c0b9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:58.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d7bfff700 1 -- 172.21.15.36:0/3852037763 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7c062e30 con 0x7f6d7c0ba2e0 2022-06-17T12:55:58.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d7affd700 1 --2- 172.21.15.36:0/3852037763 >> v2:172.21.15.70:3300/0 conn(0x7f6d7c0aef60 0x7f6d7c0b9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:55:58.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d7bfff700 1 --2- 172.21.15.36:0/3852037763 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d7c0ba2e0 0x7f6d7c1442a0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7f6d740032f0 tx=0x7f6d74009290).ready entity=mon.0 client_cookie=e3e21511102585cd server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:58.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d7a7fc700 1 -- 172.21.15.36:0/3852037763 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d7400b680 con 0x7f6d7c0ba2e0 2022-06-17T12:55:58.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d7a7fc700 1 -- 172.21.15.36:0/3852037763 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7400b7e0 con 0x7f6d7c0ba2e0 2022-06-17T12:55:58.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d7a7fc700 1 -- 172.21.15.36:0/3852037763 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d7400b950 con 0x7f6d7c0ba2e0 2022-06-17T12:55:58.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d8091d700 1 -- 172.21.15.36:0/3852037763 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d7c0ba2e0 msgr2=0x7f6d7c1442a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:58.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d8091d700 1 --2- 172.21.15.36:0/3852037763 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d7c0ba2e0 0x7f6d7c1442a0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7f6d740032f0 tx=0x7f6d74009290).stop 2022-06-17T12:55:58.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d8091d700 1 -- 172.21.15.36:0/3852037763 shutdown_connections 2022-06-17T12:55:58.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d8091d700 1 --2- 172.21.15.36:0/3852037763 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d7c0ba2e0 0x7f6d7c1442a0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:58.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d8091d700 1 --2- 172.21.15.36:0/3852037763 >> v2:172.21.15.70:3300/0 conn(0x7f6d7c0aef60 0x7f6d7c0b9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:58.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d8091d700 1 --2- 172.21.15.36:0/3852037763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d7c13cb70 0x7f6d7c143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:58.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d8091d700 1 -- 172.21.15.36:0/3852037763 >> 172.21.15.36:0/3852037763 conn(0x7f6d7c05c550 msgr2=0x7f6d7c0b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:58.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d8091d700 1 -- 172.21.15.36:0/3852037763 shutdown_connections 2022-06-17T12:55:58.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.555+0000 7f6d8091d700 1 -- 172.21.15.36:0/3852037763 wait complete. 2022-06-17T12:55:58.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.559+0000 7f6d8091d700 1 Processor -- start 2022-06-17T12:55:58.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.559+0000 7f6d8091d700 1 -- start start 2022-06-17T12:55:58.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.559+0000 7f6d8091d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d7c0aef60 0x7f6d7c1530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:58.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.559+0000 7f6d8091d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d7c0ba2e0 0x7f6d7c152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:58.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.559+0000 7f6d8091d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d7c13cb70 0x7f6d7c152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:58.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.559+0000 7f6d8091d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c152d00 con 0x7f6d7c0ba2e0 2022-06-17T12:55:58.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.559+0000 7f6d8091d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c13fb90 con 0x7f6d7c13cb70 2022-06-17T12:55:58.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.559+0000 7f6d8091d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c13db30 con 0x7f6d7c0aef60 2022-06-17T12:55:58.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.559+0000 7f6d7affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d7c0ba2e0 0x7f6d7c152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:58.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.559+0000 7f6d7affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d7c0ba2e0 0x7f6d7c152320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38946/0 (socket says 172.21.15.36:38946) 2022-06-17T12:55:58.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.559+0000 7f6d7b7fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d7c0aef60 0x7f6d7c1530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:58.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.559+0000 7f6d7affd700 1 -- 172.21.15.36:0/4287727609 learned_addr learned my addr 172.21.15.36:0/4287727609 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:58.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.559+0000 7f6d7bfff700 1 --2- 172.21.15.36:0/4287727609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d7c13cb70 0x7f6d7c152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:58.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.559+0000 7f6d7affd700 1 -- 172.21.15.36:0/4287727609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d7c0aef60 msgr2=0x7f6d7c1530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:58.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.559+0000 7f6d7affd700 1 --2- 172.21.15.36:0/4287727609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d7c0aef60 0x7f6d7c1530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:58.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.559+0000 7f6d7affd700 1 -- 172.21.15.36:0/4287727609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d7c13cb70 msgr2=0x7f6d7c152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:58.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.559+0000 7f6d7affd700 1 --2- 172.21.15.36:0/4287727609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d7c13cb70 0x7f6d7c152810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:58.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.559+0000 7f6d7affd700 1 -- 172.21.15.36:0/4287727609 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6400b180 con 0x7f6d7c0ba2e0 2022-06-17T12:55:58.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.559+0000 7f6d7affd700 1 --2- 172.21.15.36:0/4287727609 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d7c0ba2e0 0x7f6d7c152320 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7f6d7c13fcd0 tx=0x7f6d64006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:58.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.563+0000 7f6d78ff9700 1 -- 172.21.15.36:0/4287727609 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d6400c800 con 0x7f6d7c0ba2e0 2022-06-17T12:55:58.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.563+0000 7f6d78ff9700 1 -- 172.21.15.36:0/4287727609 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6400c960 con 0x7f6d7c0ba2e0 2022-06-17T12:55:58.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.563+0000 7f6d8091d700 1 -- 172.21.15.36:0/4287727609 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d74009530 con 0x7f6d7c0ba2e0 2022-06-17T12:55:58.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.563+0000 7f6d78ff9700 1 -- 172.21.15.36:0/4287727609 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d6400ccc0 con 0x7f6d7c0ba2e0 2022-06-17T12:55:58.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.563+0000 7f6d8091d700 1 -- 172.21.15.36:0/4287727609 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d7c13d890 con 0x7f6d7c0ba2e0 2022-06-17T12:55:58.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.563+0000 7f6d8091d700 1 -- 172.21.15.36:0/4287727609 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d7c152e40 con 0x7f6d7c0ba2e0 2022-06-17T12:55:58.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.563+0000 7f6d78ff9700 1 -- 172.21.15.36:0/4287727609 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6d6401c460 con 0x7f6d7c0ba2e0 2022-06-17T12:55:58.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.563+0000 7f6d78ff9700 1 --2- 172.21.15.36:0/4287727609 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6d60053a60 0x7f6d60055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:58.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.563+0000 7f6d78ff9700 1 -- 172.21.15.36:0/4287727609 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f6d6406c650 con 0x7f6d7c0ba2e0 2022-06-17T12:55:58.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.563+0000 7f6d7b7fe700 1 --2- 172.21.15.36:0/4287727609 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6d60053a60 0x7f6d60055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:58.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.567+0000 7f6d7b7fe700 1 --2- 172.21.15.36:0/4287727609 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6d60053a60 0x7f6d60055f20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f6d6c00b000 tx=0x7f6d6c006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:58.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.571+0000 7f6d78ff9700 1 -- 172.21.15.36:0/4287727609 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6d64043290 con 0x7f6d7c0ba2e0 2022-06-17T12:55:58.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.735+0000 7f6d8091d700 1 -- 172.21.15.36:0/4287727609 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d7c057410 con 0x7f6d7c0ba2e0 2022-06-17T12:55:58.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.739+0000 7f6d78ff9700 1 -- 172.21.15.36:0/4287727609 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+1426 (secure 0 0 0) 0x7f6d640468b0 con 0x7f6d7c0ba2e0 2022-06-17T12:55:58.727 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:55:58.728 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":66,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":66,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:55:56.396693+0000","modified":"2022-06-17T12:55:58.403091+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[22],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2022-06-17T12:55:58.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.739+0000 7f6d8091d700 1 -- 172.21.15.36:0/4287727609 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6d60053a60 msgr2=0x7f6d60055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:58.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.739+0000 7f6d8091d700 1 --2- 172.21.15.36:0/4287727609 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6d60053a60 0x7f6d60055f20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f6d6c00b000 tx=0x7f6d6c006df0).stop 2022-06-17T12:55:58.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.739+0000 7f6d8091d700 1 -- 172.21.15.36:0/4287727609 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d7c0ba2e0 msgr2=0x7f6d7c152320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:58.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.739+0000 7f6d8091d700 1 --2- 172.21.15.36:0/4287727609 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d7c0ba2e0 0x7f6d7c152320 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7f6d7c13fcd0 tx=0x7f6d64006df0).stop 2022-06-17T12:55:58.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.739+0000 7f6d8091d700 1 -- 172.21.15.36:0/4287727609 shutdown_connections 2022-06-17T12:55:58.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.739+0000 7f6d8091d700 1 --2- 172.21.15.36:0/4287727609 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6d60053a60 0x7f6d60055f20 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:58.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.739+0000 7f6d8091d700 1 --2- 172.21.15.36:0/4287727609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d7c13cb70 0x7f6d7c152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:58.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.739+0000 7f6d8091d700 1 --2- 172.21.15.36:0/4287727609 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6d7c0ba2e0 0x7f6d7c152320 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:58.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.739+0000 7f6d8091d700 1 --2- 172.21.15.36:0/4287727609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d7c0aef60 0x7f6d7c1530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:58.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.739+0000 7f6d8091d700 1 -- 172.21.15.36:0/4287727609 >> 172.21.15.36:0/4287727609 conn(0x7f6d7c05c550 msgr2=0x7f6d7c0b4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:58.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.739+0000 7f6d8091d700 1 -- 172.21.15.36:0/4287727609 shutdown_connections 2022-06-17T12:55:58.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.739+0000 7f6d8091d700 1 -- 172.21.15.36:0/4287727609 wait complete. 2022-06-17T12:55:58.740 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 66 2022-06-17T12:55:58.742 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-17T12:55:58.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.859+0000 7f21c25c4700 1 Processor -- start 2022-06-17T12:55:58.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.859+0000 7f21c25c4700 1 -- start start 2022-06-17T12:55:58.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.859+0000 7f21c25c4700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f21bc0acb00 0x7f21bc0acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:58.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.859+0000 7f21c25c4700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f21bc144240 0x7f21bc1446b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:58.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.859+0000 7f21c25c4700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21bc0b1900 con 0x7f21bc0b05e0 2022-06-17T12:55:58.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.859+0000 7f21c25c4700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21bc0b0a90 con 0x7f21bc0acb00 2022-06-17T12:55:58.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.859+0000 7f21c25c4700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21bc0af370 con 0x7f21bc144240 2022-06-17T12:55:58.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.863+0000 7f21bbfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f21bc0acb00 0x7f21bc0acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:58.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.863+0000 7f21bbfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f21bc0acb00 0x7f21bc0acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38954/0 (socket says 172.21.15.36:38954) 2022-06-17T12:55:58.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.863+0000 7f21bbfff700 1 -- 172.21.15.36:0/849343169 learned_addr learned my addr 172.21.15.36:0/849343169 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:58.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.863+0000 7f21c0b61700 1 --2- 172.21.15.36:0/849343169 >> v2:172.21.15.70:3300/0 conn(0x7f21bc144240 0x7f21bc1446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:58.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.863+0000 7f21bbfff700 1 -- 172.21.15.36:0/849343169 >> v2:172.21.15.70:3300/0 conn(0x7f21bc144240 msgr2=0x7f21bc1446b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:58.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.863+0000 7f21bbfff700 1 --2- 172.21.15.36:0/849343169 >> v2:172.21.15.70:3300/0 conn(0x7f21bc144240 0x7f21bc1446b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:58.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.863+0000 7f21bbfff700 1 -- 172.21.15.36:0/849343169 >> v1:172.21.15.70:6789/0 conn(0x7f21bc0b05e0 legacy=0x7f21bc147480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:58.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.863+0000 7f21bbfff700 1 -- 172.21.15.36:0/849343169 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21bc0b1b30 con 0x7f21bc0acb00 2022-06-17T12:55:58.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.863+0000 7f21bbfff700 1 --2- 172.21.15.36:0/849343169 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f21bc0acb00 0x7f21bc0acee0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0x7f21b0002f70 tx=0x7f21b0006eb0).ready entity=mon.0 client_cookie=71fb6856991229aa server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:58.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.863+0000 7f21baffd700 1 -- 172.21.15.36:0/849343169 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21b000d5c0 con 0x7f21bc0acb00 2022-06-17T12:55:58.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.863+0000 7f21baffd700 1 -- 172.21.15.36:0/849343169 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b0000e90 con 0x7f21bc0acb00 2022-06-17T12:55:58.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.863+0000 7f21baffd700 1 -- 172.21.15.36:0/849343169 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21b000d950 con 0x7f21bc0acb00 2022-06-17T12:55:58.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.863+0000 7f21c25c4700 1 -- 172.21.15.36:0/849343169 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f21bc0acb00 msgr2=0x7f21bc0acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:58.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.863+0000 7f21c25c4700 1 --2- 172.21.15.36:0/849343169 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f21bc0acb00 0x7f21bc0acee0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0x7f21b0002f70 tx=0x7f21b0006eb0).stop 2022-06-17T12:55:58.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.863+0000 7f21c25c4700 1 -- 172.21.15.36:0/849343169 shutdown_connections 2022-06-17T12:55:58.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.863+0000 7f21c25c4700 1 --2- 172.21.15.36:0/849343169 >> v2:172.21.15.70:3300/0 conn(0x7f21bc144240 0x7f21bc1446b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:58.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.863+0000 7f21c25c4700 1 --2- 172.21.15.36:0/849343169 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f21bc0acb00 0x7f21bc0acee0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:58.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.863+0000 7f21c25c4700 1 -- 172.21.15.36:0/849343169 >> 172.21.15.36:0/849343169 conn(0x7f21bc05c550 msgr2=0x7f21bc0b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:58.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.863+0000 7f21c25c4700 1 -- 172.21.15.36:0/849343169 shutdown_connections 2022-06-17T12:55:58.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.863+0000 7f21c25c4700 1 -- 172.21.15.36:0/849343169 wait complete. 2022-06-17T12:55:58.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.867+0000 7f21c25c4700 1 Processor -- start 2022-06-17T12:55:58.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.867+0000 7f21c25c4700 1 -- start start 2022-06-17T12:55:58.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.867+0000 7f21c25c4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f21bc0acb00 0x7f21bc15a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:58.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.867+0000 7f21c25c4700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21bc0b05e0 0x7f21bc140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:58.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.867+0000 7f21c25c4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21bc144240 0x7f21bc1566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:58.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.867+0000 7f21c25c4700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc140930 con 0x7f21bc0acb00 2022-06-17T12:55:58.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.867+0000 7f21c25c4700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc159d90 con 0x7f21bc144240 2022-06-17T12:55:58.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.867+0000 7f21c25c4700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc15a030 con 0x7f21bc0b05e0 2022-06-17T12:55:58.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.867+0000 7f21bbfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f21bc0acb00 0x7f21bc15a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:58.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.867+0000 7f21bbfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f21bc0acb00 0x7f21bc15a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38958/0 (socket says 172.21.15.36:38958) 2022-06-17T12:55:58.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.867+0000 7f21bbfff700 1 -- 172.21.15.36:0/2765789397 learned_addr learned my addr 172.21.15.36:0/2765789397 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:58.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.867+0000 7f21bb7fe700 1 --2- 172.21.15.36:0/2765789397 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21bc0b05e0 0x7f21bc140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:58.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.867+0000 7f21c0b61700 1 --2- 172.21.15.36:0/2765789397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21bc144240 0x7f21bc1566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:58.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.867+0000 7f21bbfff700 1 -- 172.21.15.36:0/2765789397 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21bc0b05e0 msgr2=0x7f21bc140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:58.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.867+0000 7f21bbfff700 1 --2- 172.21.15.36:0/2765789397 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21bc0b05e0 0x7f21bc140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:58.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.867+0000 7f21bbfff700 1 -- 172.21.15.36:0/2765789397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21bc144240 msgr2=0x7f21bc1566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:58.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.867+0000 7f21bbfff700 1 --2- 172.21.15.36:0/2765789397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21bc144240 0x7f21bc1566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:58.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.867+0000 7f21bbfff700 1 -- 172.21.15.36:0/2765789397 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b000b1e0 con 0x7f21bc0acb00 2022-06-17T12:55:58.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.867+0000 7f21bbfff700 1 --2- 172.21.15.36:0/2765789397 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f21bc0acb00 0x7f21bc15a770 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7f21b0002f70 tx=0x7f21b0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:58.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.871+0000 7f21b97fa700 1 -- 172.21.15.36:0/2765789397 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21b0011860 con 0x7f21bc0acb00 2022-06-17T12:55:58.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.871+0000 7f21b97fa700 1 -- 172.21.15.36:0/2765789397 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b00119c0 con 0x7f21bc0acb00 2022-06-17T12:55:58.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.871+0000 7f21c25c4700 1 -- 172.21.15.36:0/2765789397 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21bc15a2d0 con 0x7f21bc0acb00 2022-06-17T12:55:58.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.871+0000 7f21b97fa700 1 -- 172.21.15.36:0/2765789397 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21b0011d20 con 0x7f21bc0acb00 2022-06-17T12:55:58.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.871+0000 7f21c25c4700 1 -- 172.21.15.36:0/2765789397 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21bc1414e0 con 0x7f21bc0acb00 2022-06-17T12:55:58.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.871+0000 7f21c25c4700 1 -- 172.21.15.36:0/2765789397 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21bc145210 con 0x7f21bc0acb00 2022-06-17T12:55:58.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.871+0000 7f21b97fa700 1 -- 172.21.15.36:0/2765789397 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f21b0025030 con 0x7f21bc0acb00 2022-06-17T12:55:58.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.871+0000 7f21b97fa700 1 --2- 172.21.15.36:0/2765789397 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f219c053a40 0x7f219c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:58.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.871+0000 7f21b97fa700 1 -- 172.21.15.36:0/2765789397 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f21b006d220 con 0x7f21bc0acb00 2022-06-17T12:55:58.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.871+0000 7f21bb7fe700 1 --2- 172.21.15.36:0/2765789397 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f219c053a40 0x7f219c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:58.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.871+0000 7f21bb7fe700 1 --2- 172.21.15.36:0/2765789397 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f219c053a40 0x7f219c055f00 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f21ac00b000 tx=0x7f21ac009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:58.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:58.875+0000 7f21b97fa700 1 -- 172.21.15.36:0/2765789397 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f21bc145210 con 0x7f21bc0acb00 2022-06-17T12:55:59.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.039+0000 7f21c25c4700 1 -- 172.21.15.36:0/2765789397 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f21bc145210 con 0x7f21bc0acb00 2022-06-17T12:55:59.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.415+0000 7f21b97fa700 1 -- 172.21.15.36:0/2765789397 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v67) v1 ==== 104+0+0 (secure 0 0 0) 0x7f21bc145210 con 0x7f21bc0acb00 2022-06-17T12:55:59.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.419+0000 7f21c25c4700 1 -- 172.21.15.36:0/2765789397 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f219c053a40 msgr2=0x7f219c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:59.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.419+0000 7f21c25c4700 1 --2- 172.21.15.36:0/2765789397 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f219c053a40 0x7f219c055f00 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f21ac00b000 tx=0x7f21ac009040).stop 2022-06-17T12:55:59.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.419+0000 7f21c25c4700 1 -- 172.21.15.36:0/2765789397 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f21bc0acb00 msgr2=0x7f21bc15a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:59.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.419+0000 7f21c25c4700 1 --2- 172.21.15.36:0/2765789397 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f21bc0acb00 0x7f21bc15a770 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7f21b0002f70 tx=0x7f21b0015000).stop 2022-06-17T12:55:59.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.419+0000 7f21c25c4700 1 -- 172.21.15.36:0/2765789397 shutdown_connections 2022-06-17T12:55:59.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.419+0000 7f21c25c4700 1 --2- 172.21.15.36:0/2765789397 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f219c053a40 0x7f219c055f00 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:59.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.419+0000 7f21c25c4700 1 --2- 172.21.15.36:0/2765789397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21bc144240 0x7f21bc1566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:59.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.419+0000 7f21c25c4700 1 --2- 172.21.15.36:0/2765789397 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21bc0b05e0 0x7f21bc140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:59.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.419+0000 7f21c25c4700 1 --2- 172.21.15.36:0/2765789397 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f21bc0acb00 0x7f21bc15a770 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:59.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.419+0000 7f21c25c4700 1 -- 172.21.15.36:0/2765789397 >> 172.21.15.36:0/2765789397 conn(0x7f21bc05c550 msgr2=0x7f21bc1436a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:59.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.419+0000 7f21c25c4700 1 -- 172.21.15.36:0/2765789397 shutdown_connections 2022-06-17T12:55:59.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.419+0000 7f21c25c4700 1 -- 172.21.15.36:0/2765789397 wait complete. 2022-06-17T12:55:59.423 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-17T12:55:59.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.555+0000 7f6a5c830700 1 Processor -- start 2022-06-17T12:55:59.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.555+0000 7f6a5c830700 1 -- start start 2022-06-17T12:55:59.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a5c830700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a5814b820 0x7f6a5814dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:59.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a5c830700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a580b4160 con 0x7f6a580b45c0 2022-06-17T12:55:59.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a5c830700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a580b9c40 con 0x7f6a580b9860 2022-06-17T12:55:59.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a5c830700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a580b5120 con 0x7f6a5814b820 2022-06-17T12:55:59.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a4e59c700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f6a580b45c0 0x7f6a580b4a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56498/0 (socket says 172.21.15.36:56498) 2022-06-17T12:55:59.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a4e59c700 1 -- 172.21.15.36:0/3024737992 learned_addr learned my addr 172.21.15.36:0/3024737992 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:55:59.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a5759e700 1 --2- 172.21.15.36:0/3024737992 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a5814b820 0x7f6a5814dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:59.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a5659c700 1 -- 172.21.15.36:0/3024737992 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 459752217 0 0) 0x7f6a580b4160 con 0x7f6a580b45c0 2022-06-17T12:55:59.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a5659c700 1 -- 172.21.15.36:0/3024737992 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a38003260 con 0x7f6a580b45c0 2022-06-17T12:55:59.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a5659c700 1 -- 172.21.15.36:0/3024737992 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4094149351 0 0) 0x7f6a38003260 con 0x7f6a580b45c0 2022-06-17T12:55:59.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a5659c700 1 -- 172.21.15.36:0/3024737992 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a5814b820 msgr2=0x7f6a5814dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:59.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a5659c700 1 --2- 172.21.15.36:0/3024737992 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a5814b820 0x7f6a5814dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:59.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a5659c700 1 -- 172.21.15.36:0/3024737992 >> v1:172.21.15.70:6789/0 conn(0x7f6a580b9860 legacy=0x7f6a58143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:59.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a5659c700 1 -- 172.21.15.36:0/3024737992 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a580627d0 con 0x7f6a580b45c0 2022-06-17T12:55:59.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a5659c700 1 -- 172.21.15.36:0/3024737992 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f6a40001660 con 0x7f6a580b45c0 2022-06-17T12:55:59.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a5659c700 1 -- 172.21.15.36:0/3024737992 >> v1:172.21.15.36:6789/0 conn(0x7f6a580b45c0 legacy=0x7f6a580b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:59.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a5659c700 1 --2- 172.21.15.36:0/3024737992 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6a38003780 0x7f6a38003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:59.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a5659c700 1 --2- 172.21.15.36:0/3024737992 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a5814b820 0x7f6a38004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:59.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a5659c700 1 --2- 172.21.15.36:0/3024737992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a380049f0 0x7f6a38004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:59.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a5659c700 1 -- 172.21.15.36:0/3024737992 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a580b5120 con 0x7f6a38003780 2022-06-17T12:55:59.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a5659c700 1 -- 172.21.15.36:0/3024737992 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3800b780 con 0x7f6a380049f0 2022-06-17T12:55:59.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a5659c700 1 -- 172.21.15.36:0/3024737992 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3800b940 con 0x7f6a5814b820 2022-06-17T12:55:59.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a56d9d700 1 --2- 172.21.15.36:0/3024737992 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6a38003780 0x7f6a38003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:59.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a4e59c700 1 --2- 172.21.15.36:0/3024737992 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a5814b820 0x7f6a38004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:59.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a5759e700 1 --2- 172.21.15.36:0/3024737992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a380049f0 0x7f6a38004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:59.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a56d9d700 1 -- 172.21.15.36:0/3024737992 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a5814b820 msgr2=0x7f6a38004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:59.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a56d9d700 1 --2- 172.21.15.36:0/3024737992 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a5814b820 0x7f6a38004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:59.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a56d9d700 1 -- 172.21.15.36:0/3024737992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a380049f0 msgr2=0x7f6a38004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:55:59.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a56d9d700 1 --2- 172.21.15.36:0/3024737992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a380049f0 0x7f6a38004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:59.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a56d9d700 1 -- 172.21.15.36:0/3024737992 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a38003260 con 0x7f6a38003780 2022-06-17T12:55:59.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a56d9d700 1 --2- 172.21.15.36:0/3024737992 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6a38003780 0x7f6a38003ba0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7f6a580b16d0 tx=0x7f6a48007a00).ready entity=mon.0 client_cookie=434ed9994e257cdd server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:59.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a5659c700 1 -- 172.21.15.36:0/3024737992 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a48007d10 con 0x7f6a38003780 2022-06-17T12:55:59.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a5659c700 1 -- 172.21.15.36:0/3024737992 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a48007030 con 0x7f6a38003780 2022-06-17T12:55:59.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.559+0000 7f6a5659c700 1 -- 172.21.15.36:0/3024737992 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a48007390 con 0x7f6a38003780 2022-06-17T12:55:59.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.563+0000 7f6a5c830700 1 -- 172.21.15.36:0/3024737992 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6a38003780 msgr2=0x7f6a38003ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:59.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.563+0000 7f6a5c830700 1 --2- 172.21.15.36:0/3024737992 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6a38003780 0x7f6a38003ba0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7f6a580b16d0 tx=0x7f6a48007a00).stop 2022-06-17T12:55:59.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.563+0000 7f6a5759e700 1 -- 172.21.15.36:0/3024737992 reap_dead start 2022-06-17T12:55:59.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.563+0000 7f6a5c830700 1 -- 172.21.15.36:0/3024737992 shutdown_connections 2022-06-17T12:55:59.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.563+0000 7f6a5c830700 1 -- 172.21.15.36:0/3024737992 >> 172.21.15.36:0/3024737992 conn(0x7f6a5805c550 msgr2=0x7f6a58060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:55:59.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.563+0000 7f6a5c830700 1 -- 172.21.15.36:0/3024737992 shutdown_connections 2022-06-17T12:55:59.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.563+0000 7f6a5c830700 1 -- 172.21.15.36:0/3024737992 wait complete. 2022-06-17T12:55:59.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.563+0000 7f6a5c830700 1 Processor -- start 2022-06-17T12:55:59.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.563+0000 7f6a5c830700 1 -- start start 2022-06-17T12:55:59.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.567+0000 7f6a5c830700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6a581467e0 0x7f6a5813d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:59.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.567+0000 7f6a5c830700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a580ae9b0 0x7f6a5812af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:59.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.567+0000 7f6a5c830700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a5812b410 0x7f6a5813f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:59.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.567+0000 7f6a5c830700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5813d590 con 0x7f6a581467e0 2022-06-17T12:55:59.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.567+0000 7f6a5c830700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a58146bc0 con 0x7f6a5812b410 2022-06-17T12:55:59.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.567+0000 7f6a5c830700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a58140560 con 0x7f6a580ae9b0 2022-06-17T12:55:59.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.567+0000 7f6a56d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6a581467e0 0x7f6a5813d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:59.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.567+0000 7f6a56d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6a581467e0 0x7f6a5813d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38980/0 (socket says 172.21.15.36:38980) 2022-06-17T12:55:59.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.567+0000 7f6a56d9d700 1 -- 172.21.15.36:0/1576042265 learned_addr learned my addr 172.21.15.36:0/1576042265 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:55:59.570 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.567+0000 7f6a4e59c700 1 --2- 172.21.15.36:0/1576042265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a580ae9b0 0x7f6a5812af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:59.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.567+0000 7f6a5759e700 1 --2- 172.21.15.36:0/1576042265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a5812b410 0x7f6a5813f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:59.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.567+0000 7f6a56d9d700 1 -- 172.21.15.36:0/1576042265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a580ae9b0 msgr2=0x7f6a5812af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:59.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.567+0000 7f6a56d9d700 1 --2- 172.21.15.36:0/1576042265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a580ae9b0 0x7f6a5812af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:59.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.567+0000 7f6a56d9d700 1 -- 172.21.15.36:0/1576042265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a5812b410 msgr2=0x7f6a5813f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:55:59.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.567+0000 7f6a56d9d700 1 --2- 172.21.15.36:0/1576042265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a5812b410 0x7f6a5813f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:55:59.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.567+0000 7f6a56d9d700 1 -- 172.21.15.36:0/1576042265 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a58140800 con 0x7f6a581467e0 2022-06-17T12:55:59.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.567+0000 7f6a56d9d700 1 --2- 172.21.15.36:0/1576042265 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6a581467e0 0x7f6a5813d9c0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 rx=0x7f6a580b16d0 tx=0x7f6a48015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:59.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.567+0000 7f6a54d99700 1 -- 172.21.15.36:0/1576042265 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a48003510 con 0x7f6a581467e0 2022-06-17T12:55:59.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.567+0000 7f6a54d99700 1 -- 172.21.15.36:0/1576042265 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a48003670 con 0x7f6a581467e0 2022-06-17T12:55:59.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.567+0000 7f6a5c830700 1 -- 172.21.15.36:0/1576042265 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a580aff80 con 0x7f6a581467e0 2022-06-17T12:55:59.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.567+0000 7f6a54d99700 1 -- 172.21.15.36:0/1576042265 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a480039d0 con 0x7f6a581467e0 2022-06-17T12:55:59.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.567+0000 7f6a5c830700 1 -- 172.21.15.36:0/1576042265 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a581402b0 con 0x7f6a581467e0 2022-06-17T12:55:59.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.567+0000 7f6a5c830700 1 -- 172.21.15.36:0/1576042265 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a58062a70 con 0x7f6a581467e0 2022-06-17T12:55:59.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.571+0000 7f6a54d99700 1 -- 172.21.15.36:0/1576042265 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6a48028030 con 0x7f6a581467e0 2022-06-17T12:55:59.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.571+0000 7f6a54d99700 1 --2- 172.21.15.36:0/1576042265 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6a3c053ad0 0x7f6a3c055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:55:59.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.571+0000 7f6a54d99700 1 -- 172.21.15.36:0/1576042265 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f6a4806d340 con 0x7f6a581467e0 2022-06-17T12:55:59.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.571+0000 7f6a4e59c700 1 --2- 172.21.15.36:0/1576042265 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6a3c053ad0 0x7f6a3c055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:55:59.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.571+0000 7f6a4e59c700 1 --2- 172.21.15.36:0/1576042265 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6a3c053ad0 0x7f6a3c055f90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f6a40005000 tx=0x7f6a580ba5a0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:55:59.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.575+0000 7f6a54d99700 1 -- 172.21.15.36:0/1576042265 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6a48043fe0 con 0x7f6a581467e0 2022-06-17T12:55:59.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:55:59.707+0000 7f6a5c830700 1 -- 172.21.15.36:0/1576042265 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6a58057410 con 0x7f6a581467e0 2022-06-17T12:56:00.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.431+0000 7f6a54d99700 1 -- 172.21.15.36:0/1576042265 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v125) v1 ==== 185+0+0 (secure 0 0 0) 0x7f6a48047600 con 0x7f6a581467e0 2022-06-17T12:56:00.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.435+0000 7f6a5c830700 1 -- 172.21.15.36:0/1576042265 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6a3c053ad0 msgr2=0x7f6a3c055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:00.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.435+0000 7f6a5c830700 1 --2- 172.21.15.36:0/1576042265 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6a3c053ad0 0x7f6a3c055f90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f6a40005000 tx=0x7f6a580ba5a0).stop 2022-06-17T12:56:00.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.435+0000 7f6a5c830700 1 -- 172.21.15.36:0/1576042265 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6a581467e0 msgr2=0x7f6a5813d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:00.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.435+0000 7f6a5c830700 1 --2- 172.21.15.36:0/1576042265 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6a581467e0 0x7f6a5813d9c0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 rx=0x7f6a580b16d0 tx=0x7f6a48015000).stop 2022-06-17T12:56:00.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.435+0000 7f6a5c830700 1 -- 172.21.15.36:0/1576042265 shutdown_connections 2022-06-17T12:56:00.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.435+0000 7f6a5c830700 1 --2- 172.21.15.36:0/1576042265 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6a3c053ad0 0x7f6a3c055f90 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:00.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.435+0000 7f6a5c830700 1 --2- 172.21.15.36:0/1576042265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a5812b410 0x7f6a5813f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:00.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.435+0000 7f6a5c830700 1 --2- 172.21.15.36:0/1576042265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a580ae9b0 0x7f6a5812af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:00.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.435+0000 7f6a5c830700 1 --2- 172.21.15.36:0/1576042265 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6a581467e0 0x7f6a5813d9c0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:00.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.435+0000 7f6a5c830700 1 -- 172.21.15.36:0/1576042265 >> 172.21.15.36:0/1576042265 conn(0x7f6a5805c550 msgr2=0x7f6a580b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:00.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.435+0000 7f6a5c830700 1 -- 172.21.15.36:0/1576042265 shutdown_connections 2022-06-17T12:56:00.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.435+0000 7f6a5c830700 1 -- 172.21.15.36:0/1576042265 wait complete. 2022-06-17T12:56:00.437 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' removed 2022-06-17T12:56:00.441 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-17T12:56:00.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.619+0000 7f2d1dae3700 1 Processor -- start 2022-06-17T12:56:00.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.619+0000 7f2d1dae3700 1 -- start start 2022-06-17T12:56:00.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.623+0000 7f2d1dae3700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2d180b8020 0x7f2d18062830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:00.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.623+0000 7f2d1dae3700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d180b8400 con 0x7f2d180b3a40 2022-06-17T12:56:00.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.623+0000 7f2d1dae3700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d180b21b0 con 0x7f2d180b2af0 2022-06-17T12:56:00.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.623+0000 7f2d1dae3700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d18063430 con 0x7f2d180b8020 2022-06-17T12:56:00.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.623+0000 7f2d1cae1700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2d180b8020 0x7f2d18062830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:00.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.623+0000 7f2d17fff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f2d180b3a40 0x7f2d18062d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56516/0 (socket says 172.21.15.36:56516) 2022-06-17T12:56:00.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.623+0000 7f2d1cae1700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2d180b8020 0x7f2d18062830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38986/0 (socket says 172.21.15.36:38986) 2022-06-17T12:56:00.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.623+0000 7f2d17fff700 1 -- 172.21.15.36:0/1266050502 learned_addr learned my addr 172.21.15.36:0/1266050502 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:00.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.623+0000 7f2d1cae1700 1 -- 172.21.15.36:0/1266050502 >> v1:172.21.15.70:6789/0 conn(0x7f2d180b2af0 legacy=0x7f2d18141e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:00.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.623+0000 7f2d1cae1700 1 -- 172.21.15.36:0/1266050502 >> v1:172.21.15.36:6789/0 conn(0x7f2d180b3a40 legacy=0x7f2d18062d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:00.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.623+0000 7f2d1cae1700 1 -- 172.21.15.36:0/1266050502 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d18067a90 con 0x7f2d180b8020 2022-06-17T12:56:00.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.623+0000 7f2d177fe700 1 -- 172.21.15.36:0/1266050502 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1819279057 0 0) 0x7f2d180b8400 con 0x7f2d180b3a40 2022-06-17T12:56:00.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.623+0000 7f2d1cae1700 1 --2- 172.21.15.36:0/1266050502 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2d180b8020 0x7f2d18062830 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x7f2d0c002f70 tx=0x7f2d0c006df0).ready entity=mon.0 client_cookie=3d350b4256190ace server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:00.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.623+0000 7f2d177fe700 1 -- 172.21.15.36:0/1266050502 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d0c00d610 con 0x7f2d180b8020 2022-06-17T12:56:00.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.623+0000 7f2d177fe700 1 -- 172.21.15.36:0/1266050502 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0c000e90 con 0x7f2d180b8020 2022-06-17T12:56:00.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.623+0000 7f2d177fe700 1 -- 172.21.15.36:0/1266050502 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d0c00d960 con 0x7f2d180b8020 2022-06-17T12:56:00.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.623+0000 7f2d1dae3700 1 -- 172.21.15.36:0/1266050502 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2d180b8020 msgr2=0x7f2d18062830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:00.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.623+0000 7f2d1dae3700 1 --2- 172.21.15.36:0/1266050502 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2d180b8020 0x7f2d18062830 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x7f2d0c002f70 tx=0x7f2d0c006df0).stop 2022-06-17T12:56:00.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.623+0000 7f2d1dae3700 1 -- 172.21.15.36:0/1266050502 shutdown_connections 2022-06-17T12:56:00.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.623+0000 7f2d1dae3700 1 --2- 172.21.15.36:0/1266050502 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2d180b8020 0x7f2d18062830 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:00.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.623+0000 7f2d1dae3700 1 -- 172.21.15.36:0/1266050502 >> 172.21.15.36:0/1266050502 conn(0x7f2d1805c550 msgr2=0x7f2d180b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:00.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.623+0000 7f2d1dae3700 1 -- 172.21.15.36:0/1266050502 shutdown_connections 2022-06-17T12:56:00.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.623+0000 7f2d1dae3700 1 -- 172.21.15.36:0/1266050502 wait complete. 2022-06-17T12:56:00.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.627+0000 7f2d1dae3700 1 Processor -- start 2022-06-17T12:56:00.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.627+0000 7f2d1dae3700 1 -- start start 2022-06-17T12:56:00.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.627+0000 7f2d1dae3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d180b2af0 0x7f2d18156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:00.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.627+0000 7f2d1dae3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2d180b3a40 0x7f2d180b4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:00.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.627+0000 7f2d1dae3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d180b8020 0x7f2d181405b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:00.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.627+0000 7f2d1dae3700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d18145120 con 0x7f2d180b3a40 2022-06-17T12:56:00.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.627+0000 7f2d1dae3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d18159ab0 con 0x7f2d180b8020 2022-06-17T12:56:00.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.627+0000 7f2d1dae3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d18145350 con 0x7f2d180b2af0 2022-06-17T12:56:00.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.627+0000 7f2d17fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2d180b3a40 0x7f2d180b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:00.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.627+0000 7f2d17fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2d180b3a40 0x7f2d180b4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:38994/0 (socket says 172.21.15.36:38994) 2022-06-17T12:56:00.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.627+0000 7f2d17fff700 1 -- 172.21.15.36:0/2458468762 learned_addr learned my addr 172.21.15.36:0/2458468762 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:00.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.627+0000 7f2d1d2e2700 1 --2- 172.21.15.36:0/2458468762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d180b8020 0x7f2d181405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:00.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.627+0000 7f2d1cae1700 1 --2- 172.21.15.36:0/2458468762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d180b2af0 0x7f2d18156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:00.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.631+0000 7f2d17fff700 1 -- 172.21.15.36:0/2458468762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d180b2af0 msgr2=0x7f2d18156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:00.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.631+0000 7f2d17fff700 1 --2- 172.21.15.36:0/2458468762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d180b2af0 0x7f2d18156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:00.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.631+0000 7f2d17fff700 1 -- 172.21.15.36:0/2458468762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d180b8020 msgr2=0x7f2d181405b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:00.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.631+0000 7f2d17fff700 1 --2- 172.21.15.36:0/2458468762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d180b8020 0x7f2d181405b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:00.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.631+0000 7f2d17fff700 1 -- 172.21.15.36:0/2458468762 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c00b180 con 0x7f2d180b3a40 2022-06-17T12:56:00.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.631+0000 7f2d1cae1700 1 --2- 172.21.15.36:0/2458468762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d180b2af0 0x7f2d18156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:56:00.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.631+0000 7f2d17fff700 1 --2- 172.21.15.36:0/2458468762 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2d180b3a40 0x7f2d180b4030 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7f2d04000bc0 tx=0x7f2d04007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:00.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.631+0000 7f2d15ffb700 1 -- 172.21.15.36:0/2458468762 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d040070d0 con 0x7f2d180b3a40 2022-06-17T12:56:00.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.631+0000 7f2d15ffb700 1 -- 172.21.15.36:0/2458468762 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d04007230 con 0x7f2d180b3a40 2022-06-17T12:56:00.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.631+0000 7f2d1dae3700 1 -- 172.21.15.36:0/2458468762 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d18159d10 con 0x7f2d180b3a40 2022-06-17T12:56:00.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.631+0000 7f2d15ffb700 1 -- 172.21.15.36:0/2458468762 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d04005610 con 0x7f2d180b3a40 2022-06-17T12:56:00.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.631+0000 7f2d1dae3700 1 -- 172.21.15.36:0/2458468762 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d18144d20 con 0x7f2d180b3a40 2022-06-17T12:56:00.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.631+0000 7f2d1dae3700 1 -- 172.21.15.36:0/2458468762 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d18159ea0 con 0x7f2d180b3a40 2022-06-17T12:56:00.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.635+0000 7f2d15ffb700 1 -- 172.21.15.36:0/2458468762 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2d0401a060 con 0x7f2d180b3a40 2022-06-17T12:56:00.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.635+0000 7f2d15ffb700 1 --2- 172.21.15.36:0/2458468762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2cf8053ad0 0x7f2cf8055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:00.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.635+0000 7f2d15ffb700 1 -- 172.21.15.36:0/2458468762 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f2d0406bca0 con 0x7f2d180b3a40 2022-06-17T12:56:00.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.635+0000 7f2d1cae1700 1 --2- 172.21.15.36:0/2458468762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2cf8053ad0 0x7f2cf8055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:00.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.635+0000 7f2d1cae1700 1 --2- 172.21.15.36:0/2458468762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2cf8053ad0 0x7f2cf8055f90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f2d0c01a000 tx=0x7f2d0c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:00.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.639+0000 7f2d15ffb700 1 -- 172.21.15.36:0/2458468762 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2d04042c00 con 0x7f2d180b3a40 2022-06-17T12:56:00.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:00.767+0000 7f2d1dae3700 1 -- 172.21.15.36:0/2458468762 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2d18057410 con 0x7f2d180b3a40 2022-06-17T12:56:01.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.587+0000 7f2d15ffb700 1 -- 172.21.15.36:0/2458468762 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v126) v1 ==== 173+0+0 (secure 0 0 0) 0x7f2d04018030 con 0x7f2d180b3a40 2022-06-17T12:56:01.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.591+0000 7f2d1dae3700 1 -- 172.21.15.36:0/2458468762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2cf8053ad0 msgr2=0x7f2cf8055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:01.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.591+0000 7f2d1dae3700 1 --2- 172.21.15.36:0/2458468762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2cf8053ad0 0x7f2cf8055f90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f2d0c01a000 tx=0x7f2d0c015000).stop 2022-06-17T12:56:01.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.591+0000 7f2d1dae3700 1 -- 172.21.15.36:0/2458468762 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2d180b3a40 msgr2=0x7f2d180b4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:01.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.591+0000 7f2d1dae3700 1 --2- 172.21.15.36:0/2458468762 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2d180b3a40 0x7f2d180b4030 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7f2d04000bc0 tx=0x7f2d04007a40).stop 2022-06-17T12:56:01.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.591+0000 7f2d1dae3700 1 -- 172.21.15.36:0/2458468762 shutdown_connections 2022-06-17T12:56:01.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.591+0000 7f2d1dae3700 1 --2- 172.21.15.36:0/2458468762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2cf8053ad0 0x7f2cf8055f90 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:01.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.591+0000 7f2d1dae3700 1 --2- 172.21.15.36:0/2458468762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d180b8020 0x7f2d181405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:01.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.591+0000 7f2d1dae3700 1 --2- 172.21.15.36:0/2458468762 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2d180b3a40 0x7f2d180b4030 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:01.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.591+0000 7f2d1dae3700 1 --2- 172.21.15.36:0/2458468762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d180b2af0 0x7f2d18156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:01.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.591+0000 7f2d1dae3700 1 -- 172.21.15.36:0/2458468762 >> 172.21.15.36:0/2458468762 conn(0x7f2d1805c550 msgr2=0x7f2d180ad410 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:01.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.591+0000 7f2d1dae3700 1 -- 172.21.15.36:0/2458468762 shutdown_connections 2022-06-17T12:56:01.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.591+0000 7f2d1dae3700 1 -- 172.21.15.36:0/2458468762 wait complete. 2022-06-17T12:56:01.587 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' removed 2022-06-17T12:56:01.602 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-06-17T12:56:01.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.739+0000 7fd42759e700 1 Processor -- start 2022-06-17T12:56:01.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.739+0000 7fd42759e700 1 -- start start 2022-06-17T12:56:01.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.743+0000 7fd42759e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4280b65f0 0x7fd4280b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:01.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.743+0000 7fd42759e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd4280adf10 0x7fd428140530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:01.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.743+0000 7fd42759e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd42813fdf0 con 0x7fd4280b15b0 2022-06-17T12:56:01.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.743+0000 7fd42759e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4280ae8e0 con 0x7fd4280adf10 2022-06-17T12:56:01.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.743+0000 7fd42759e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd428140090 con 0x7fd4280b65f0 2022-06-17T12:56:01.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.743+0000 7fd425d9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd4280adf10 0x7fd428140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:01.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.743+0000 7fd425d9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd4280adf10 0x7fd428140530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39000/0 (socket says 172.21.15.36:39000) 2022-06-17T12:56:01.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.743+0000 7fd425d9b700 1 -- 172.21.15.36:0/1082776317 learned_addr learned my addr 172.21.15.36:0/1082776317 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:01.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.743+0000 7fd42659c700 1 --2- 172.21.15.36:0/1082776317 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4280b65f0 0x7fd4280b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:01.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.743+0000 7fd425d9b700 1 -- 172.21.15.36:0/1082776317 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4280b65f0 msgr2=0x7fd4280b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:01.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.743+0000 7fd425d9b700 1 --2- 172.21.15.36:0/1082776317 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4280b65f0 0x7fd4280b69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:01.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.743+0000 7fd42559a700 1 -- 172.21.15.36:0/1082776317 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930796500 0 0) 0x7fd42813fdf0 con 0x7fd4280b15b0 2022-06-17T12:56:01.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.743+0000 7fd425d9b700 1 -- 172.21.15.36:0/1082776317 >> v1:172.21.15.36:6789/0 conn(0x7fd4280b15b0 legacy=0x7fd4280af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:01.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.743+0000 7fd425d9b700 1 -- 172.21.15.36:0/1082776317 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd428140a20 con 0x7fd4280adf10 2022-06-17T12:56:01.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.743+0000 7fd425d9b700 1 --2- 172.21.15.36:0/1082776317 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd4280adf10 0x7fd428140530 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 rx=0x7fd41c002f70 tx=0x7fd41c006e50).ready entity=mon.0 client_cookie=1e6b3ef1ac393b0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:01.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.743+0000 7fd42559a700 1 -- 172.21.15.36:0/1082776317 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd41c00d5c0 con 0x7fd4280adf10 2022-06-17T12:56:01.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.743+0000 7fd42559a700 1 -- 172.21.15.36:0/1082776317 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41c000e90 con 0x7fd4280adf10 2022-06-17T12:56:01.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.743+0000 7fd42559a700 1 -- 172.21.15.36:0/1082776317 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd41c00d910 con 0x7fd4280adf10 2022-06-17T12:56:01.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.743+0000 7fd42759e700 1 -- 172.21.15.36:0/1082776317 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd4280adf10 msgr2=0x7fd428140530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:01.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.743+0000 7fd42759e700 1 --2- 172.21.15.36:0/1082776317 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd4280adf10 0x7fd428140530 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 rx=0x7fd41c002f70 tx=0x7fd41c006e50).stop 2022-06-17T12:56:01.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.743+0000 7fd42759e700 1 -- 172.21.15.36:0/1082776317 shutdown_connections 2022-06-17T12:56:01.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.743+0000 7fd42759e700 1 --2- 172.21.15.36:0/1082776317 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd4280adf10 0x7fd428140530 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:01.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.743+0000 7fd42759e700 1 --2- 172.21.15.36:0/1082776317 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4280b65f0 0x7fd4280b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:01.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.743+0000 7fd42759e700 1 -- 172.21.15.36:0/1082776317 >> 172.21.15.36:0/1082776317 conn(0x7fd4280614b0 msgr2=0x7fd428061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:01.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.747+0000 7fd42759e700 1 -- 172.21.15.36:0/1082776317 shutdown_connections 2022-06-17T12:56:01.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.747+0000 7fd42759e700 1 -- 172.21.15.36:0/1082776317 wait complete. 2022-06-17T12:56:01.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.747+0000 7fd42759e700 1 Processor -- start 2022-06-17T12:56:01.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.747+0000 7fd42759e700 1 -- start start 2022-06-17T12:56:01.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.747+0000 7fd42759e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4280adf10 0x7fd4281375a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:01.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.747+0000 7fd42759e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd4280b15b0 0x7fd42814df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:01.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.747+0000 7fd42759e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4280b65f0 0x7fd42814e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:01.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.747+0000 7fd42759e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42814db40 con 0x7fd4280b15b0 2022-06-17T12:56:01.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.747+0000 7fd42759e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd428141cb0 con 0x7fd4280b65f0 2022-06-17T12:56:01.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.747+0000 7fd42759e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd428151ba0 con 0x7fd4280adf10 2022-06-17T12:56:01.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.747+0000 7fd425d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd4280b15b0 0x7fd42814df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:01.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.747+0000 7fd425d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd4280b15b0 0x7fd42814df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39008/0 (socket says 172.21.15.36:39008) 2022-06-17T12:56:01.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.747+0000 7fd425d9b700 1 -- 172.21.15.36:0/2777688334 learned_addr learned my addr 172.21.15.36:0/2777688334 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:01.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.747+0000 7fd42659c700 1 --2- 172.21.15.36:0/2777688334 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4280adf10 0x7fd4281375a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:01.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.747+0000 7fd426d9d700 1 --2- 172.21.15.36:0/2777688334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4280b65f0 0x7fd42814e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:01.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.747+0000 7fd425d9b700 1 -- 172.21.15.36:0/2777688334 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4280adf10 msgr2=0x7fd4281375a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:01.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.747+0000 7fd425d9b700 1 --2- 172.21.15.36:0/2777688334 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4280adf10 0x7fd4281375a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:01.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.747+0000 7fd425d9b700 1 -- 172.21.15.36:0/2777688334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4280b65f0 msgr2=0x7fd42814e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:01.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.747+0000 7fd425d9b700 1 --2- 172.21.15.36:0/2777688334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4280b65f0 0x7fd42814e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:01.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.747+0000 7fd425d9b700 1 -- 172.21.15.36:0/2777688334 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41c00b1e0 con 0x7fd4280b15b0 2022-06-17T12:56:01.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.751+0000 7fd425d9b700 1 --2- 172.21.15.36:0/2777688334 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd4280b15b0 0x7fd42814df90 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7fd41c002f70 tx=0x7fd41c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:01.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.751+0000 7fd4177fe700 1 -- 172.21.15.36:0/2777688334 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd41c013de0 con 0x7fd4280b15b0 2022-06-17T12:56:01.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.751+0000 7fd42759e700 1 -- 172.21.15.36:0/2777688334 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd428151e40 con 0x7fd4280b15b0 2022-06-17T12:56:01.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.751+0000 7fd4177fe700 1 -- 172.21.15.36:0/2777688334 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41c01e400 con 0x7fd4280b15b0 2022-06-17T12:56:01.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.751+0000 7fd4177fe700 1 -- 172.21.15.36:0/2777688334 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd41c01e760 con 0x7fd4280b15b0 2022-06-17T12:56:01.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.751+0000 7fd42759e700 1 -- 172.21.15.36:0/2777688334 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd42813f700 con 0x7fd4280b15b0 2022-06-17T12:56:01.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.751+0000 7fd42759e700 1 -- 172.21.15.36:0/2777688334 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4281412f0 con 0x7fd4280b15b0 2022-06-17T12:56:01.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.751+0000 7fd4177fe700 1 -- 172.21.15.36:0/2777688334 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd41c025030 con 0x7fd4280b15b0 2022-06-17T12:56:01.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.751+0000 7fd4177fe700 1 --2- 172.21.15.36:0/2777688334 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd404053af0 0x7fd404055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:01.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.751+0000 7fd42659c700 1 --2- 172.21.15.36:0/2777688334 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd404053af0 0x7fd404055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:01.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.751+0000 7fd4177fe700 1 -- 172.21.15.36:0/2777688334 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fd41c06cad0 con 0x7fd4280b15b0 2022-06-17T12:56:01.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.751+0000 7fd42659c700 1 --2- 172.21.15.36:0/2777688334 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd404053af0 0x7fd404055fb0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7fd41800b000 tx=0x7fd418006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:01.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.759+0000 7fd4177fe700 1 -- 172.21.15.36:0/2777688334 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd4281412f0 con 0x7fd4280b15b0 2022-06-17T12:56:01.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.891+0000 7fd42759e700 1 -- 172.21.15.36:0/2777688334 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd4281412f0 con 0x7fd4280b15b0 2022-06-17T12:56:01.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.899+0000 7fd4177fe700 1 -- 172.21.15.36:0/2777688334 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd4281412f0 con 0x7fd4280b15b0 2022-06-17T12:56:01.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.899+0000 7fd42759e700 1 -- 172.21.15.36:0/2777688334 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd404053af0 msgr2=0x7fd404055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:01.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.899+0000 7fd42759e700 1 --2- 172.21.15.36:0/2777688334 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd404053af0 0x7fd404055fb0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7fd41800b000 tx=0x7fd418006df0).stop 2022-06-17T12:56:01.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.899+0000 7fd42759e700 1 -- 172.21.15.36:0/2777688334 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd4280b15b0 msgr2=0x7fd42814df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:01.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.899+0000 7fd42759e700 1 --2- 172.21.15.36:0/2777688334 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd4280b15b0 0x7fd42814df90 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7fd41c002f70 tx=0x7fd41c014000).stop 2022-06-17T12:56:01.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.899+0000 7fd42759e700 1 -- 172.21.15.36:0/2777688334 shutdown_connections 2022-06-17T12:56:01.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.899+0000 7fd42759e700 1 --2- 172.21.15.36:0/2777688334 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd404053af0 0x7fd404055fb0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:01.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.899+0000 7fd42759e700 1 --2- 172.21.15.36:0/2777688334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4280b65f0 0x7fd42814e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:01.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.899+0000 7fd42759e700 1 --2- 172.21.15.36:0/2777688334 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd4280b15b0 0x7fd42814df90 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:01.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.899+0000 7fd42759e700 1 --2- 172.21.15.36:0/2777688334 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4280adf10 0x7fd4281375a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:01.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.899+0000 7fd42759e700 1 -- 172.21.15.36:0/2777688334 >> 172.21.15.36:0/2777688334 conn(0x7fd4280614b0 msgr2=0x7fd4280b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:01.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.899+0000 7fd42759e700 1 -- 172.21.15.36:0/2777688334 shutdown_connections 2022-06-17T12:56:01.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:01.899+0000 7fd42759e700 1 -- 172.21.15.36:0/2777688334 wait complete. 2022-06-17T12:56:01.914 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new' 2022-06-17T12:56:02.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.043+0000 7f619e723700 1 Processor -- start 2022-06-17T12:56:02.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.043+0000 7f619e723700 1 -- start start 2022-06-17T12:56:02.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.047+0000 7f619e723700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f61980aea60 0x7f6198143280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:02.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.047+0000 7f619e723700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f619805fe90 con 0x7f6198140480 2022-06-17T12:56:02.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.047+0000 7f619e723700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6198143990 con 0x7f619805fab0 2022-06-17T12:56:02.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.047+0000 7f619e723700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6198143770 con 0x7f61980aea60 2022-06-17T12:56:02.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.047+0000 7f61977fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f61980aea60 0x7f6198143280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:02.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.047+0000 7f619ccc0700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f6198140480 0x7f61980afb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56548/0 (socket says 172.21.15.36:56548) 2022-06-17T12:56:02.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.047+0000 7f619ccc0700 1 -- 172.21.15.36:0/2075612985 learned_addr learned my addr 172.21.15.36:0/2075612985 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:02.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.047+0000 7f61977fe700 1 -- 172.21.15.36:0/2075612985 >> v1:172.21.15.70:6789/0 conn(0x7f619805fab0 legacy=0x7f619813fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:02.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.047+0000 7f61977fe700 1 -- 172.21.15.36:0/2075612985 >> v1:172.21.15.36:6789/0 conn(0x7f6198140480 legacy=0x7f61980afb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:02.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.047+0000 7f6196ffd700 1 -- 172.21.15.36:0/2075612985 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 167354211 0 0) 0x7f619805fe90 con 0x7f6198140480 2022-06-17T12:56:02.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.047+0000 7f61977fe700 1 -- 172.21.15.36:0/2075612985 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6198143c30 con 0x7f61980aea60 2022-06-17T12:56:02.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.047+0000 7f61977fe700 1 --2- 172.21.15.36:0/2075612985 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61980aea60 0x7f6198143280 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7f618c002f70 tx=0x7f618c006df0).ready entity=mon.0 client_cookie=352125ef51ff3f87 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:02.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.047+0000 7f6196ffd700 1 -- 172.21.15.36:0/2075612985 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f618c00d610 con 0x7f61980aea60 2022-06-17T12:56:02.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.047+0000 7f6196ffd700 1 -- 172.21.15.36:0/2075612985 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618c000e90 con 0x7f61980aea60 2022-06-17T12:56:02.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.047+0000 7f6196ffd700 1 -- 172.21.15.36:0/2075612985 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f618c00d960 con 0x7f61980aea60 2022-06-17T12:56:02.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.047+0000 7f619e723700 1 -- 172.21.15.36:0/2075612985 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61980aea60 msgr2=0x7f6198143280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:02.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.047+0000 7f619e723700 1 --2- 172.21.15.36:0/2075612985 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61980aea60 0x7f6198143280 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7f618c002f70 tx=0x7f618c006df0).stop 2022-06-17T12:56:02.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.047+0000 7f619e723700 1 -- 172.21.15.36:0/2075612985 shutdown_connections 2022-06-17T12:56:02.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.047+0000 7f619e723700 1 --2- 172.21.15.36:0/2075612985 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61980aea60 0x7f6198143280 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:02.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.047+0000 7f619e723700 1 -- 172.21.15.36:0/2075612985 >> 172.21.15.36:0/2075612985 conn(0x7f619805c550 msgr2=0x7f619805c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:02.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.051+0000 7f619e723700 1 -- 172.21.15.36:0/2075612985 shutdown_connections 2022-06-17T12:56:02.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.051+0000 7f619e723700 1 -- 172.21.15.36:0/2075612985 wait complete. 2022-06-17T12:56:02.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.051+0000 7f619e723700 1 Processor -- start 2022-06-17T12:56:02.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.051+0000 7f619e723700 1 -- start start 2022-06-17T12:56:02.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.051+0000 7f619e723700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f619805fab0 0x7f6198145c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:02.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.051+0000 7f619e723700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61980aea60 0x7f6198155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:02.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.051+0000 7f619e723700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6198140480 0x7f61981561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:02.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.051+0000 7f619e723700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6198128cd0 con 0x7f61980aea60 2022-06-17T12:56:02.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.051+0000 7f619e723700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6198145030 con 0x7f6198140480 2022-06-17T12:56:02.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.051+0000 7f619e723700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6198146ee0 con 0x7f619805fab0 2022-06-17T12:56:02.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.051+0000 7f61977fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61980aea60 0x7f6198155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:02.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.051+0000 7f61977fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61980aea60 0x7f6198155cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39020/0 (socket says 172.21.15.36:39020) 2022-06-17T12:56:02.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.051+0000 7f61977fe700 1 -- 172.21.15.36:0/3708846955 learned_addr learned my addr 172.21.15.36:0/3708846955 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:02.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.051+0000 7f619ccc0700 1 --2- 172.21.15.36:0/3708846955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6198140480 0x7f61981561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:02.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.051+0000 7f6197fff700 1 --2- 172.21.15.36:0/3708846955 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f619805fab0 0x7f6198145c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:02.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.051+0000 7f61977fe700 1 -- 172.21.15.36:0/3708846955 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f619805fab0 msgr2=0x7f6198145c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:02.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.051+0000 7f61977fe700 1 --2- 172.21.15.36:0/3708846955 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f619805fab0 0x7f6198145c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:02.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.051+0000 7f61977fe700 1 -- 172.21.15.36:0/3708846955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6198140480 msgr2=0x7f61981561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:02.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.051+0000 7f61977fe700 1 --2- 172.21.15.36:0/3708846955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6198140480 0x7f61981561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:02.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.051+0000 7f61977fe700 1 -- 172.21.15.36:0/3708846955 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c00b180 con 0x7f61980aea60 2022-06-17T12:56:02.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.051+0000 7f61977fe700 1 --2- 172.21.15.36:0/3708846955 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61980aea60 0x7f6198155cb0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f618c002f70 tx=0x7f618c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:02.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.051+0000 7f61957fa700 1 -- 172.21.15.36:0/3708846955 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f618c00d290 con 0x7f61980aea60 2022-06-17T12:56:02.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.055+0000 7f61957fa700 1 -- 172.21.15.36:0/3708846955 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618c011ce0 con 0x7f61980aea60 2022-06-17T12:56:02.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.055+0000 7f619e723700 1 -- 172.21.15.36:0/3708846955 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6198155850 con 0x7f61980aea60 2022-06-17T12:56:02.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.055+0000 7f61957fa700 1 -- 172.21.15.36:0/3708846955 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f618c01e650 con 0x7f61980aea60 2022-06-17T12:56:02.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.055+0000 7f619e723700 1 -- 172.21.15.36:0/3708846955 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6198156690 con 0x7f61980aea60 2022-06-17T12:56:02.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.055+0000 7f619e723700 1 -- 172.21.15.36:0/3708846955 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6198146d40 con 0x7f61980aea60 2022-06-17T12:56:02.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.055+0000 7f61957fa700 1 -- 172.21.15.36:0/3708846955 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f618c025030 con 0x7f61980aea60 2022-06-17T12:56:02.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.055+0000 7f61957fa700 1 --2- 172.21.15.36:0/3708846955 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6178053940 0x7f6178055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:02.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.055+0000 7f61957fa700 1 -- 172.21.15.36:0/3708846955 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f618c06d120 con 0x7f61980aea60 2022-06-17T12:56:02.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.055+0000 7f6197fff700 1 --2- 172.21.15.36:0/3708846955 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6178053940 0x7f6178055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:02.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.055+0000 7f6197fff700 1 --2- 172.21.15.36:0/3708846955 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6178053940 0x7f6178055e00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f618800b000 tx=0x7f6188009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:02.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.063+0000 7f61957fa700 1 -- 172.21.15.36:0/3708846955 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6198146d40 con 0x7f61980aea60 2022-06-17T12:56:02.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.231+0000 7f619e723700 1 -- 172.21.15.36:0/3708846955 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new"]} v 0) v1 -- 0x7f6198146d40 con 0x7f61980aea60 2022-06-17T12:56:02.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.587+0000 7f61957fa700 1 -- 172.21.15.36:0/3708846955 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new"]}]=0 v429) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6198146d40 con 0x7f61980aea60 2022-06-17T12:56:02.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.591+0000 7f619e723700 1 -- 172.21.15.36:0/3708846955 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6178053940 msgr2=0x7f6178055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:02.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.591+0000 7f619e723700 1 --2- 172.21.15.36:0/3708846955 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6178053940 0x7f6178055e00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f618800b000 tx=0x7f6188009040).stop 2022-06-17T12:56:02.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.591+0000 7f619e723700 1 -- 172.21.15.36:0/3708846955 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61980aea60 msgr2=0x7f6198155cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:02.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.591+0000 7f619e723700 1 --2- 172.21.15.36:0/3708846955 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61980aea60 0x7f6198155cb0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f618c002f70 tx=0x7f618c015000).stop 2022-06-17T12:56:02.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.591+0000 7f619e723700 1 -- 172.21.15.36:0/3708846955 shutdown_connections 2022-06-17T12:56:02.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.591+0000 7f619e723700 1 --2- 172.21.15.36:0/3708846955 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6178053940 0x7f6178055e00 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:02.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.591+0000 7f619e723700 1 --2- 172.21.15.36:0/3708846955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6198140480 0x7f61981561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:02.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.591+0000 7f619e723700 1 --2- 172.21.15.36:0/3708846955 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61980aea60 0x7f6198155cb0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:02.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.591+0000 7f619e723700 1 --2- 172.21.15.36:0/3708846955 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f619805fab0 0x7f6198145c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:02.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.591+0000 7f619e723700 1 -- 172.21.15.36:0/3708846955 >> 172.21.15.36:0/3708846955 conn(0x7f619805c550 msgr2=0x7f61980afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:02.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.591+0000 7f619e723700 1 -- 172.21.15.36:0/3708846955 shutdown_connections 2022-06-17T12:56:02.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.591+0000 7f619e723700 1 -- 172.21.15.36:0/3708846955 wait complete. 2022-06-17T12:56:02.596 INFO:tasks.cephfs_test_runner:test_fs_new (tasks.cephfs.test_misc.TestMisc) ... ok 2022-06-17T12:56:02.597 INFO:tasks.cephfs_test_runner:Starting test: test_getattr_caps (tasks.cephfs.test_misc.TestMisc) 2022-06-17T12:56:02.597 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps' 2022-06-17T12:56:02.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b8a50c700 1 Processor -- start 2022-06-17T12:56:02.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b8a50c700 1 -- start start 2022-06-17T12:56:02.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b8a50c700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2b840620d0 0x7f2b840b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:02.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b8a50c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b840b4e10 0x7f2b840b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:02.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b8a50c700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b840b21d0 con 0x7f2b840b31c0 2022-06-17T12:56:02.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b8a50c700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b840b2430 con 0x7f2b840620d0 2022-06-17T12:56:02.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b8a50c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b840b26d0 con 0x7f2b840b4e10 2022-06-17T12:56:02.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b837fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f2b840b31c0 0x7f2b840b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56558/0 (socket says 172.21.15.36:56558) 2022-06-17T12:56:02.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b837fe700 1 -- 172.21.15.36:0/213669096 learned_addr learned my addr 172.21.15.36:0/213669096 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:02.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b88aa9700 1 --2- 172.21.15.36:0/213669096 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b840b4e10 0x7f2b840b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:02.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b83fff700 1 --2- 172.21.15.36:0/213669096 >> v2:172.21.15.70:3300/0 conn(0x7f2b840620d0 0x7f2b840b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:02.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b82ffd700 1 -- 172.21.15.36:0/213669096 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3902882585 0 0) 0x7f2b840b21d0 con 0x7f2b840b31c0 2022-06-17T12:56:02.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b82ffd700 1 -- 172.21.15.36:0/213669096 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b64003260 con 0x7f2b840b31c0 2022-06-17T12:56:02.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b82ffd700 1 -- 172.21.15.36:0/213669096 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2346804033 0 0) 0x7f2b64003260 con 0x7f2b840b31c0 2022-06-17T12:56:02.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b82ffd700 1 -- 172.21.15.36:0/213669096 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b840b4e10 msgr2=0x7f2b840b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:02.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b82ffd700 1 --2- 172.21.15.36:0/213669096 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b840b4e10 0x7f2b840b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:02.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b82ffd700 1 -- 172.21.15.36:0/213669096 >> v2:172.21.15.70:3300/0 conn(0x7f2b840620d0 msgr2=0x7f2b840b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:02.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b82ffd700 1 --2- 172.21.15.36:0/213669096 >> v2:172.21.15.70:3300/0 conn(0x7f2b840620d0 0x7f2b840b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:02.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b82ffd700 1 -- 172.21.15.36:0/213669096 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b840b43f0 con 0x7f2b840b31c0 2022-06-17T12:56:02.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b82ffd700 1 -- 172.21.15.36:0/213669096 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f2b6c001660 con 0x7f2b840b31c0 2022-06-17T12:56:02.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b83fff700 1 --2- 172.21.15.36:0/213669096 >> v2:172.21.15.70:3300/0 conn(0x7f2b840620d0 0x7f2b840b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:02.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b82ffd700 1 -- 172.21.15.36:0/213669096 >> v1:172.21.15.36:6789/0 conn(0x7f2b840b31c0 legacy=0x7f2b840b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:02.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b82ffd700 1 --2- 172.21.15.36:0/213669096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b64003750 0x7f2b64003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:02.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b82ffd700 1 --2- 172.21.15.36:0/213669096 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b64004500 0x7f2b64004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:02.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b82ffd700 1 --2- 172.21.15.36:0/213669096 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b64004e70 0x7f2b6400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:02.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b82ffd700 1 -- 172.21.15.36:0/213669096 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b840b2430 con 0x7f2b64004500 2022-06-17T12:56:02.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b82ffd700 1 -- 172.21.15.36:0/213669096 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b840b26d0 con 0x7f2b64003750 2022-06-17T12:56:02.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b82ffd700 1 -- 172.21.15.36:0/213669096 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6400bc40 con 0x7f2b64004e70 2022-06-17T12:56:02.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b88aa9700 1 --2- 172.21.15.36:0/213669096 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b840b4e10 0x7f2b840b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:02.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b83fff700 1 --2- 172.21.15.36:0/213669096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b64003750 0x7f2b64003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:02.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b837fe700 1 --2- 172.21.15.36:0/213669096 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b64004500 0x7f2b64004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:02.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.739+0000 7f2b88aa9700 1 --2- 172.21.15.36:0/213669096 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b64004e70 0x7f2b6400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:02.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.743+0000 7f2b83fff700 1 -- 172.21.15.36:0/213669096 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b64004e70 msgr2=0x7f2b6400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:02.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.743+0000 7f2b83fff700 1 --2- 172.21.15.36:0/213669096 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b64004e70 0x7f2b6400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:02.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.743+0000 7f2b83fff700 1 -- 172.21.15.36:0/213669096 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b64004500 msgr2=0x7f2b64004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:02.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.743+0000 7f2b83fff700 1 --2- 172.21.15.36:0/213669096 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b64004500 0x7f2b64004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:02.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.743+0000 7f2b83fff700 1 -- 172.21.15.36:0/213669096 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b64003260 con 0x7f2b64003750 2022-06-17T12:56:02.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.743+0000 7f2b88aa9700 1 --2- 172.21.15.36:0/213669096 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b64004e70 0x7f2b6400b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:56:02.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.743+0000 7f2b83fff700 1 --2- 172.21.15.36:0/213669096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b64003750 0x7f2b64003b70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f2b64004470 tx=0x7f2b74006df0).ready entity=mon.1 client_cookie=24590a9d122c10f7 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:02.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.743+0000 7f2b82ffd700 1 -- 172.21.15.36:0/213669096 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b7400e830 con 0x7f2b64003750 2022-06-17T12:56:02.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.743+0000 7f2b82ffd700 1 -- 172.21.15.36:0/213669096 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b7400e990 con 0x7f2b64003750 2022-06-17T12:56:02.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.743+0000 7f2b82ffd700 1 -- 172.21.15.36:0/213669096 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b7400ecf0 con 0x7f2b64003750 2022-06-17T12:56:02.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.743+0000 7f2b8a50c700 1 -- 172.21.15.36:0/213669096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b64003750 msgr2=0x7f2b64003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:02.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.743+0000 7f2b8a50c700 1 --2- 172.21.15.36:0/213669096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b64003750 0x7f2b64003b70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f2b64004470 tx=0x7f2b74006df0).stop 2022-06-17T12:56:02.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.743+0000 7f2b88aa9700 1 -- 172.21.15.36:0/213669096 reap_dead start 2022-06-17T12:56:02.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.743+0000 7f2b8a50c700 1 -- 172.21.15.36:0/213669096 shutdown_connections 2022-06-17T12:56:02.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.743+0000 7f2b8a50c700 1 -- 172.21.15.36:0/213669096 >> 172.21.15.36:0/213669096 conn(0x7f2b840b9850 msgr2=0x7f2b840b9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:02.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.743+0000 7f2b8a50c700 1 -- 172.21.15.36:0/213669096 shutdown_connections 2022-06-17T12:56:02.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.743+0000 7f2b8a50c700 1 -- 172.21.15.36:0/213669096 wait complete. 2022-06-17T12:56:02.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.743+0000 7f2b8a50c700 1 Processor -- start 2022-06-17T12:56:02.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.743+0000 7f2b8a50c700 1 -- start start 2022-06-17T12:56:02.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.747+0000 7f2b8a50c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b64003750 0x7f2b84117a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:02.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.747+0000 7f2b8a50c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b840afb70 0x7f2b8414f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:02.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.747+0000 7f2b8a50c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b8414fd30 0x7f2b84119380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:02.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.747+0000 7f2b8a50c700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b84115b90 con 0x7f2b64003750 2022-06-17T12:56:02.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.747+0000 7f2b8a50c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b84115e30 con 0x7f2b840afb70 2022-06-17T12:56:02.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.747+0000 7f2b8a50c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b841160d0 con 0x7f2b8414fd30 2022-06-17T12:56:02.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.747+0000 7f2b83fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b64003750 0x7f2b84117a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:02.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.747+0000 7f2b83fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b64003750 0x7f2b84117a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39040/0 (socket says 172.21.15.36:39040) 2022-06-17T12:56:02.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.747+0000 7f2b83fff700 1 -- 172.21.15.36:0/1224384073 learned_addr learned my addr 172.21.15.36:0/1224384073 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:02.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.747+0000 7f2b88aa9700 1 --2- 172.21.15.36:0/1224384073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b8414fd30 0x7f2b84119380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:02.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.747+0000 7f2b837fe700 1 --2- 172.21.15.36:0/1224384073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b840afb70 0x7f2b8414f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:02.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.747+0000 7f2b83fff700 1 -- 172.21.15.36:0/1224384073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b8414fd30 msgr2=0x7f2b84119380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:02.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.747+0000 7f2b83fff700 1 --2- 172.21.15.36:0/1224384073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b8414fd30 0x7f2b84119380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:02.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.747+0000 7f2b83fff700 1 -- 172.21.15.36:0/1224384073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b840afb70 msgr2=0x7f2b8414f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:02.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.747+0000 7f2b83fff700 1 --2- 172.21.15.36:0/1224384073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b840afb70 0x7f2b8414f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:02.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.747+0000 7f2b83fff700 1 -- 172.21.15.36:0/1224384073 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b84155e80 con 0x7f2b64003750 2022-06-17T12:56:02.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.747+0000 7f2b88aa9700 1 --2- 172.21.15.36:0/1224384073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b8414fd30 0x7f2b84119380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:56:02.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.747+0000 7f2b83fff700 1 --2- 172.21.15.36:0/1224384073 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b64003750 0x7f2b84117a90 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7f2b64004470 tx=0x7f2b74019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:02.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.747+0000 7f2b817fa700 1 -- 172.21.15.36:0/1224384073 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b74012c50 con 0x7f2b64003750 2022-06-17T12:56:02.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.747+0000 7f2b817fa700 1 -- 172.21.15.36:0/1224384073 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b74000e90 con 0x7f2b64003750 2022-06-17T12:56:02.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.747+0000 7f2b8a50c700 1 -- 172.21.15.36:0/1224384073 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b840afff0 con 0x7f2b64003750 2022-06-17T12:56:02.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.747+0000 7f2b817fa700 1 -- 172.21.15.36:0/1224384073 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b7401d410 con 0x7f2b64003750 2022-06-17T12:56:02.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.747+0000 7f2b8a50c700 1 -- 172.21.15.36:0/1224384073 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b84115810 con 0x7f2b64003750 2022-06-17T12:56:02.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.751+0000 7f2b817fa700 1 -- 172.21.15.36:0/1224384073 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2b7402d070 con 0x7f2b64003750 2022-06-17T12:56:02.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.751+0000 7f2b8a50c700 1 -- 172.21.15.36:0/1224384073 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b84116290 con 0x7f2b64003750 2022-06-17T12:56:02.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.751+0000 7f2b817fa700 1 --2- 172.21.15.36:0/1224384073 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b68053af0 0x7f2b68055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:02.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.751+0000 7f2b817fa700 1 -- 172.21.15.36:0/1224384073 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f2b7406d940 con 0x7f2b64003750 2022-06-17T12:56:02.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.751+0000 7f2b837fe700 1 --2- 172.21.15.36:0/1224384073 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b68053af0 0x7f2b68055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:02.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.751+0000 7f2b837fe700 1 --2- 172.21.15.36:0/1224384073 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b68053af0 0x7f2b68055fb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f2b6c005000 tx=0x7f2b6c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:02.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.755+0000 7f2b817fa700 1 -- 172.21.15.36:0/1224384073 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2b74044a20 con 0x7f2b64003750 2022-06-17T12:56:02.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:02.923+0000 7f2b8a50c700 1 -- 172.21.15.36:0/1224384073 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]} v 0) v1 -- 0x7f2b84057410 con 0x7f2b64003750 2022-06-17T12:56:03.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.591+0000 7f2b817fa700 1 -- 172.21.15.36:0/1224384073 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]}]=0 v430) v1 ==== 131+0+0 (secure 0 0 0) 0x7f2b74048040 con 0x7f2b64003750 2022-06-17T12:56:03.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.591+0000 7f2b8a50c700 1 -- 172.21.15.36:0/1224384073 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b68053af0 msgr2=0x7f2b68055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:03.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.591+0000 7f2b8a50c700 1 --2- 172.21.15.36:0/1224384073 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b68053af0 0x7f2b68055fb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f2b6c005000 tx=0x7f2b6c009040).stop 2022-06-17T12:56:03.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.591+0000 7f2b8a50c700 1 -- 172.21.15.36:0/1224384073 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b64003750 msgr2=0x7f2b84117a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:03.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.591+0000 7f2b8a50c700 1 --2- 172.21.15.36:0/1224384073 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b64003750 0x7f2b84117a90 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7f2b64004470 tx=0x7f2b74019000).stop 2022-06-17T12:56:03.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.595+0000 7f2b8a50c700 1 -- 172.21.15.36:0/1224384073 shutdown_connections 2022-06-17T12:56:03.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.595+0000 7f2b8a50c700 1 --2- 172.21.15.36:0/1224384073 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2b68053af0 0x7f2b68055fb0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:03.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.595+0000 7f2b8a50c700 1 --2- 172.21.15.36:0/1224384073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b8414fd30 0x7f2b84119380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:03.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.595+0000 7f2b8a50c700 1 --2- 172.21.15.36:0/1224384073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b840afb70 0x7f2b8414f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:03.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.595+0000 7f2b8a50c700 1 --2- 172.21.15.36:0/1224384073 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2b64003750 0x7f2b84117a90 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:03.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.595+0000 7f2b8a50c700 1 -- 172.21.15.36:0/1224384073 >> 172.21.15.36:0/1224384073 conn(0x7f2b840b9850 msgr2=0x7f2b84143140 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:03.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.595+0000 7f2b8a50c700 1 -- 172.21.15.36:0/1224384073 shutdown_connections 2022-06-17T12:56:03.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.595+0000 7f2b8a50c700 1 -- 172.21.15.36:0/1224384073 wait complete. 2022-06-17T12:56:03.653 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-06-17T12:56:03.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.783+0000 7f58d7091700 1 Processor -- start 2022-06-17T12:56:03.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.783+0000 7f58d7091700 1 -- start start 2022-06-17T12:56:03.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.783+0000 7f58d7091700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f58d014b2b0 0x7f58d0060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:03.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.783+0000 7f58d7091700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f58d0147bd0 0x7f58d01442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:03.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.783+0000 7f58d7091700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d014b870 con 0x7f58d00b27b0 2022-06-17T12:56:03.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.783+0000 7f58d7091700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d00ac500 con 0x7f58d014b2b0 2022-06-17T12:56:03.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.783+0000 7f58d7091700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d00ace90 con 0x7f58d0147bd0 2022-06-17T12:56:03.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.783+0000 7f58d588e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f58d014b2b0 0x7f58d0060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:03.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.783+0000 7f58d6890700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f58d0147bd0 0x7f58d01442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:03.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.783+0000 7f58d588e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f58d014b2b0 0x7f58d0060ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:59432/0 (socket says 172.21.15.36:59432) 2022-06-17T12:56:03.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.783+0000 7f58d588e700 1 -- 172.21.15.36:0/545845602 learned_addr learned my addr 172.21.15.36:0/545845602 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:03.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.783+0000 7f58d588e700 1 -- 172.21.15.36:0/545845602 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f58d0147bd0 msgr2=0x7f58d01442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:03.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.783+0000 7f58d588e700 1 --2- 172.21.15.36:0/545845602 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f58d0147bd0 0x7f58d01442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:03.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.783+0000 7f58d588e700 1 -- 172.21.15.36:0/545845602 >> v1:172.21.15.70:6789/0 conn(0x7f58d00b27b0 legacy=0x7f58d00ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:03.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.783+0000 7f58d588e700 1 -- 172.21.15.36:0/545845602 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d00b2b90 con 0x7f58d014b2b0 2022-06-17T12:56:03.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.783+0000 7f58d588e700 1 --2- 172.21.15.36:0/545845602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58d014b2b0 0x7f58d0060ca0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f58c80032f0 tx=0x7f58c8009290).ready entity=mon.1 client_cookie=3212d1671a98f36f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:03.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.783+0000 7f58d508d700 1 -- 172.21.15.36:0/545845602 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58c800b8a0 con 0x7f58d014b2b0 2022-06-17T12:56:03.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.783+0000 7f58d508d700 1 -- 172.21.15.36:0/545845602 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c800ba00 con 0x7f58d014b2b0 2022-06-17T12:56:03.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.783+0000 7f58d508d700 1 -- 172.21.15.36:0/545845602 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58c800bb70 con 0x7f58d014b2b0 2022-06-17T12:56:03.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d7091700 1 -- 172.21.15.36:0/545845602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58d014b2b0 msgr2=0x7f58d0060ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:03.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d7091700 1 --2- 172.21.15.36:0/545845602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58d014b2b0 0x7f58d0060ca0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f58c80032f0 tx=0x7f58c8009290).stop 2022-06-17T12:56:03.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d7091700 1 -- 172.21.15.36:0/545845602 shutdown_connections 2022-06-17T12:56:03.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d7091700 1 --2- 172.21.15.36:0/545845602 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f58d0147bd0 0x7f58d01442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:03.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d7091700 1 --2- 172.21.15.36:0/545845602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58d014b2b0 0x7f58d0060ca0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:03.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d7091700 1 -- 172.21.15.36:0/545845602 >> 172.21.15.36:0/545845602 conn(0x7f58d00b5d10 msgr2=0x7f58d00b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:03.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d7091700 1 -- 172.21.15.36:0/545845602 shutdown_connections 2022-06-17T12:56:03.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d7091700 1 -- 172.21.15.36:0/545845602 wait complete. 2022-06-17T12:56:03.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d7091700 1 Processor -- start 2022-06-17T12:56:03.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d7091700 1 -- start start 2022-06-17T12:56:03.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d7091700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f58d00b27b0 0x7f58d0149910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:03.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d7091700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58d0147bd0 0x7f58d0064dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:03.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d7091700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f58d014b2b0 0x7f58d0156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:03.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d7091700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d0064390 con 0x7f58d014b2b0 2022-06-17T12:56:03.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d7091700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d0064630 con 0x7f58d0147bd0 2022-06-17T12:56:03.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d7091700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d0064900 con 0x7f58d00b27b0 2022-06-17T12:56:03.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d6890700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f58d014b2b0 0x7f58d0156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:03.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d6890700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f58d014b2b0 0x7f58d0156530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39058/0 (socket says 172.21.15.36:39058) 2022-06-17T12:56:03.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d6890700 1 -- 172.21.15.36:0/980844331 learned_addr learned my addr 172.21.15.36:0/980844331 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:03.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d608f700 1 --2- 172.21.15.36:0/980844331 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f58d00b27b0 0x7f58d0149910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:03.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d588e700 1 --2- 172.21.15.36:0/980844331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58d0147bd0 0x7f58d0064dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:03.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d6890700 1 -- 172.21.15.36:0/980844331 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f58d00b27b0 msgr2=0x7f58d0149910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:03.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d6890700 1 --2- 172.21.15.36:0/980844331 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f58d00b27b0 0x7f58d0149910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:03.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d6890700 1 -- 172.21.15.36:0/980844331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58d0147bd0 msgr2=0x7f58d0064dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:03.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d6890700 1 --2- 172.21.15.36:0/980844331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58d0147bd0 0x7f58d0064dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:03.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.787+0000 7f58d6890700 1 -- 172.21.15.36:0/980844331 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c8009530 con 0x7f58d014b2b0 2022-06-17T12:56:03.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.791+0000 7f58d6890700 1 --2- 172.21.15.36:0/980844331 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f58d014b2b0 0x7f58d0156530 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 rx=0x7f58cc00c000 tx=0x7f58cc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:03.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.791+0000 7f58c37fe700 1 -- 172.21.15.36:0/980844331 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58cc011750 con 0x7f58d014b2b0 2022-06-17T12:56:03.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.791+0000 7f58c37fe700 1 -- 172.21.15.36:0/980844331 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58cc0118b0 con 0x7f58d014b2b0 2022-06-17T12:56:03.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.791+0000 7f58d7091700 1 -- 172.21.15.36:0/980844331 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d0149e60 con 0x7f58d014b2b0 2022-06-17T12:56:03.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.791+0000 7f58c37fe700 1 -- 172.21.15.36:0/980844331 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58cc011c10 con 0x7f58d014b2b0 2022-06-17T12:56:03.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.791+0000 7f58d7091700 1 -- 172.21.15.36:0/980844331 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d014a910 con 0x7f58d014b2b0 2022-06-17T12:56:03.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.791+0000 7f58d7091700 1 -- 172.21.15.36:0/980844331 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d0057410 con 0x7f58d014b2b0 2022-06-17T12:56:03.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.791+0000 7f58c37fe700 1 -- 172.21.15.36:0/980844331 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f58cc011d70 con 0x7f58d014b2b0 2022-06-17T12:56:03.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.791+0000 7f58c37fe700 1 --2- 172.21.15.36:0/980844331 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f58b0053a60 0x7f58b0055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:03.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.791+0000 7f58c37fe700 1 -- 172.21.15.36:0/980844331 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f58cc06c150 con 0x7f58d014b2b0 2022-06-17T12:56:03.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.795+0000 7f58d608f700 1 --2- 172.21.15.36:0/980844331 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f58b0053a60 0x7f58b0055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:03.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.795+0000 7f58d608f700 1 --2- 172.21.15.36:0/980844331 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f58b0053a60 0x7f58b0055f20 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f58c400d000 tx=0x7f58c4007b50).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:03.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.799+0000 7f58c37fe700 1 -- 172.21.15.36:0/980844331 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f58cc043120 con 0x7f58d014b2b0 2022-06-17T12:56:03.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.923+0000 7f58d7091700 1 -- 172.21.15.36:0/980844331 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f58d005d9b0 con 0x7f58d014b2b0 2022-06-17T12:56:03.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.931+0000 7f58c37fe700 1 -- 172.21.15.36:0/980844331 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f58cc046740 con 0x7f58d014b2b0 2022-06-17T12:56:03.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.935+0000 7f58d7091700 1 -- 172.21.15.36:0/980844331 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f58b0053a60 msgr2=0x7f58b0055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:03.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.935+0000 7f58d7091700 1 --2- 172.21.15.36:0/980844331 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f58b0053a60 0x7f58b0055f20 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f58c400d000 tx=0x7f58c4007b50).stop 2022-06-17T12:56:03.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.935+0000 7f58d7091700 1 -- 172.21.15.36:0/980844331 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f58d014b2b0 msgr2=0x7f58d0156530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:03.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.935+0000 7f58d7091700 1 --2- 172.21.15.36:0/980844331 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f58d014b2b0 0x7f58d0156530 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 rx=0x7f58cc00c000 tx=0x7f58cc006df0).stop 2022-06-17T12:56:03.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.935+0000 7f58d7091700 1 -- 172.21.15.36:0/980844331 shutdown_connections 2022-06-17T12:56:03.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.935+0000 7f58d7091700 1 --2- 172.21.15.36:0/980844331 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f58b0053a60 0x7f58b0055f20 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:03.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.935+0000 7f58d7091700 1 --2- 172.21.15.36:0/980844331 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f58d014b2b0 0x7f58d0156530 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:03.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.935+0000 7f58d7091700 1 --2- 172.21.15.36:0/980844331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58d0147bd0 0x7f58d0064dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:03.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.935+0000 7f58d7091700 1 --2- 172.21.15.36:0/980844331 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f58d00b27b0 0x7f58d0149910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:03.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.935+0000 7f58d7091700 1 -- 172.21.15.36:0/980844331 >> 172.21.15.36:0/980844331 conn(0x7f58d00b5d10 msgr2=0x7f58d0146b70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:03.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.935+0000 7f58d7091700 1 -- 172.21.15.36:0/980844331 shutdown_connections 2022-06-17T12:56:03.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:03.935+0000 7f58d7091700 1 -- 172.21.15.36:0/980844331 wait complete. 2022-06-17T12:56:03.957 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-17T12:56:03.958 DEBUG:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:56:04.032 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:56:04.063 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:56:04.091 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:56:04.116 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:56:04.159 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-17T12:56:04.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:04.162 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:04.165 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:56:04.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:56:04.187 INFO:teuthology.orchestra.run.smithi107.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-17T12:56:04.187 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-06-17T12:56:04.188 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:04.189 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:04.190 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:04.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:56:04.206 INFO:teuthology.orchestra.run.smithi136.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-17T12:56:04.207 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-17T12:56:04.208 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:04.209 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:04.209 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:56:04.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:56:04.222 INFO:teuthology.orchestra.run.smithi143.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:56:04.223 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-06-17T12:56:04.224 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-17T12:56:04.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.347+0000 7fbd252af700 1 Processor -- start 2022-06-17T12:56:04.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.347+0000 7fbd252af700 1 -- start start 2022-06-17T12:56:04.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.347+0000 7fbd252af700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd200620d0 0x7fbd200b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:04.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.347+0000 7fbd252af700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbd200b4e10 0x7fbd200b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:04.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.347+0000 7fbd252af700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd200b21d0 con 0x7fbd200b31c0 2022-06-17T12:56:04.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.347+0000 7fbd252af700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd200b2430 con 0x7fbd200b4e10 2022-06-17T12:56:04.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.347+0000 7fbd252af700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd200b26d0 con 0x7fbd200620d0 2022-06-17T12:56:04.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.347+0000 7fbd1e7fc700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fbd200b31c0 0x7fbd200b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56594/0 (socket says 172.21.15.36:56594) 2022-06-17T12:56:04.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.347+0000 7fbd1e7fc700 1 -- 172.21.15.36:0/3442453441 learned_addr learned my addr 172.21.15.36:0/3442453441 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:04.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.347+0000 7fbd1effd700 1 --2- 172.21.15.36:0/3442453441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd200620d0 0x7fbd200b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:04.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.347+0000 7fbd1f7fe700 1 --2- 172.21.15.36:0/3442453441 >> v2:172.21.15.70:3300/0 conn(0x7fbd200b4e10 0x7fbd200b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:04.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.347+0000 7fbd1dffb700 1 -- 172.21.15.36:0/3442453441 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2690721623 0 0) 0x7fbd200b21d0 con 0x7fbd200b31c0 2022-06-17T12:56:04.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.347+0000 7fbd1dffb700 1 -- 172.21.15.36:0/3442453441 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd00003260 con 0x7fbd200b31c0 2022-06-17T12:56:04.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.347+0000 7fbd1effd700 1 -- 172.21.15.36:0/3442453441 >> v2:172.21.15.70:3300/0 conn(0x7fbd200b4e10 msgr2=0x7fbd200b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:04.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.347+0000 7fbd1effd700 1 --2- 172.21.15.36:0/3442453441 >> v2:172.21.15.70:3300/0 conn(0x7fbd200b4e10 0x7fbd200b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:04.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.347+0000 7fbd1effd700 1 -- 172.21.15.36:0/3442453441 >> v1:172.21.15.36:6789/0 conn(0x7fbd200b31c0 legacy=0x7fbd200b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:04.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.347+0000 7fbd1effd700 1 -- 172.21.15.36:0/3442453441 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd200b43f0 con 0x7fbd200620d0 2022-06-17T12:56:04.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.347+0000 7fbd1effd700 1 --2- 172.21.15.36:0/3442453441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd200620d0 0x7fbd200b5bc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fbd08002f70 tx=0x7fbd08006df0).ready entity=mon.2 client_cookie=4207692ce0f83bb server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:04.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.347+0000 7fbd1dffb700 1 -- 172.21.15.36:0/3442453441 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd0800d4a0 con 0x7fbd200620d0 2022-06-17T12:56:04.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.347+0000 7fbd1dffb700 1 -- 172.21.15.36:0/3442453441 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd08000e90 con 0x7fbd200620d0 2022-06-17T12:56:04.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.347+0000 7fbd1dffb700 1 -- 172.21.15.36:0/3442453441 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd0800d7f0 con 0x7fbd200620d0 2022-06-17T12:56:04.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.351+0000 7fbd252af700 1 -- 172.21.15.36:0/3442453441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd200620d0 msgr2=0x7fbd200b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:04.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.351+0000 7fbd252af700 1 --2- 172.21.15.36:0/3442453441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd200620d0 0x7fbd200b5bc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fbd08002f70 tx=0x7fbd08006df0).stop 2022-06-17T12:56:04.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.351+0000 7fbd252af700 1 -- 172.21.15.36:0/3442453441 shutdown_connections 2022-06-17T12:56:04.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.351+0000 7fbd252af700 1 --2- 172.21.15.36:0/3442453441 >> v2:172.21.15.70:3300/0 conn(0x7fbd200b4e10 0x7fbd200b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:04.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.351+0000 7fbd252af700 1 --2- 172.21.15.36:0/3442453441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd200620d0 0x7fbd200b5bc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:04.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.351+0000 7fbd252af700 1 -- 172.21.15.36:0/3442453441 >> 172.21.15.36:0/3442453441 conn(0x7fbd200b9850 msgr2=0x7fbd200b9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:04.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.351+0000 7fbd252af700 1 -- 172.21.15.36:0/3442453441 shutdown_connections 2022-06-17T12:56:04.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.351+0000 7fbd252af700 1 -- 172.21.15.36:0/3442453441 wait complete. 2022-06-17T12:56:04.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.351+0000 7fbd252af700 1 Processor -- start 2022-06-17T12:56:04.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.351+0000 7fbd252af700 1 -- start start 2022-06-17T12:56:04.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.355+0000 7fbd252af700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbd200620d0 0x7fbd201136b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:04.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.355+0000 7fbd252af700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd200b31c0 0x7fbd200b8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:04.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.355+0000 7fbd252af700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd200b4e10 0x7fbd201140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:04.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.355+0000 7fbd252af700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd20153110 con 0x7fbd200620d0 2022-06-17T12:56:04.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.355+0000 7fbd252af700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd20153610 con 0x7fbd200b4e10 2022-06-17T12:56:04.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.355+0000 7fbd252af700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd20153370 con 0x7fbd200b31c0 2022-06-17T12:56:04.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.355+0000 7fbd1effd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbd200620d0 0x7fbd201136b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:04.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.355+0000 7fbd1e7fc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd200b31c0 0x7fbd200b8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:04.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.355+0000 7fbd1effd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbd200620d0 0x7fbd201136b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39068/0 (socket says 172.21.15.36:39068) 2022-06-17T12:56:04.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.355+0000 7fbd1e7fc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd200b31c0 0x7fbd200b8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:53860/0 (socket says 172.21.15.36:53860) 2022-06-17T12:56:04.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.355+0000 7fbd1effd700 1 -- 172.21.15.36:0/2349617641 learned_addr learned my addr 172.21.15.36:0/2349617641 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:04.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.355+0000 7fbd1f7fe700 1 --2- 172.21.15.36:0/2349617641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd200b4e10 0x7fbd201140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:04.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.355+0000 7fbd1e7fc700 1 -- 172.21.15.36:0/2349617641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd200b4e10 msgr2=0x7fbd201140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:04.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.355+0000 7fbd1e7fc700 1 --2- 172.21.15.36:0/2349617641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd200b4e10 0x7fbd201140e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:04.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.355+0000 7fbd1e7fc700 1 -- 172.21.15.36:0/2349617641 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbd200620d0 msgr2=0x7fbd201136b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:04.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.355+0000 7fbd1e7fc700 1 --2- 172.21.15.36:0/2349617641 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbd200620d0 0x7fbd201136b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:04.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.355+0000 7fbd1e7fc700 1 -- 172.21.15.36:0/2349617641 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0800b180 con 0x7fbd200b31c0 2022-06-17T12:56:04.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.355+0000 7fbd1e7fc700 1 --2- 172.21.15.36:0/2349617641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd200b31c0 0x7fbd200b8670 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fbd10001110 tx=0x7fbd100079e0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:04.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.355+0000 7fbd17fff700 1 -- 172.21.15.36:0/2349617641 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd1000cda0 con 0x7fbd200b31c0 2022-06-17T12:56:04.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.355+0000 7fbd17fff700 1 -- 172.21.15.36:0/2349617641 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd10007110 con 0x7fbd200b31c0 2022-06-17T12:56:04.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.355+0000 7fbd17fff700 1 -- 172.21.15.36:0/2349617641 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd10007470 con 0x7fbd200b31c0 2022-06-17T12:56:04.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.355+0000 7fbd252af700 1 -- 172.21.15.36:0/2349617641 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd200ae5c0 con 0x7fbd200b31c0 2022-06-17T12:56:04.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.355+0000 7fbd252af700 1 -- 172.21.15.36:0/2349617641 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd20112a40 con 0x7fbd200b31c0 2022-06-17T12:56:04.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.359+0000 7fbd252af700 1 -- 172.21.15.36:0/2349617641 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd200b0190 con 0x7fbd200b31c0 2022-06-17T12:56:04.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.359+0000 7fbd17fff700 1 -- 172.21.15.36:0/2349617641 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fbd1001b070 con 0x7fbd200b31c0 2022-06-17T12:56:04.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.359+0000 7fbd17fff700 1 --2- 172.21.15.36:0/2349617641 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbd04053a60 0x7fbd04055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:04.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.359+0000 7fbd17fff700 1 -- 172.21.15.36:0/2349617641 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fbd1006b160 con 0x7fbd200b31c0 2022-06-17T12:56:04.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.359+0000 7fbd1effd700 1 --2- 172.21.15.36:0/2349617641 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbd04053a60 0x7fbd04055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:04.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.359+0000 7fbd1effd700 1 --2- 172.21.15.36:0/2349617641 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbd04053a60 0x7fbd04055f20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fbd0801a000 tx=0x7fbd08014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:04.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.367+0000 7fbd17fff700 1 -- 172.21.15.36:0/2349617641 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbd200b0190 con 0x7fbd200b31c0 2022-06-17T12:56:04.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.495+0000 7fbd252af700 1 -- 172.21.15.36:0/2349617641 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fbd200b0190 con 0x7fbd200b31c0 2022-06-17T12:56:04.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.499+0000 7fbd17fff700 1 -- 172.21.15.36:0/2349617641 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v126) v1 ==== 81+0+29229 (secure 0 0 0) 0x7fbd200b0190 con 0x7fbd200b31c0 2022-06-17T12:56:04.493 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:04.499 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:56:04.499 INFO:teuthology.orchestra.run.smithi036.stdout: "epoch": 126, 2022-06-17T12:56:04.500 INFO:teuthology.orchestra.run.smithi036.stdout: "fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", 2022-06-17T12:56:04.500 INFO:teuthology.orchestra.run.smithi036.stdout: "created": "2022-06-17T12:46:30.671408+0000", 2022-06-17T12:56:04.501 INFO:teuthology.orchestra.run.smithi036.stdout: "modified": "2022-06-17T12:56:01.581150+0000", 2022-06-17T12:56:04.501 INFO:teuthology.orchestra.run.smithi036.stdout: "last_up_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:56:04.501 INFO:teuthology.orchestra.run.smithi036.stdout: "last_in_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:56:04.503 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-17T12:56:04.503 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_num": 5799936, 2022-06-17T12:56:04.504 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_set": [ 2022-06-17T12:56:04.504 INFO:teuthology.orchestra.run.smithi036.stdout: "pglog_hardlimit", 2022-06-17T12:56:04.504 INFO:teuthology.orchestra.run.smithi036.stdout: "purged_snapdirs", 2022-06-17T12:56:04.505 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_deletes", 2022-06-17T12:56:04.506 INFO:teuthology.orchestra.run.smithi036.stdout: "sortbitwise" 2022-06-17T12:56:04.506 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:56:04.507 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_version": 6, 2022-06-17T12:56:04.507 INFO:teuthology.orchestra.run.smithi036.stdout: "full_ratio": 0.94999998807907104, 2022-06-17T12:56:04.508 INFO:teuthology.orchestra.run.smithi036.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-17T12:56:04.508 INFO:teuthology.orchestra.run.smithi036.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-17T12:56:04.509 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_snapshot": "", 2022-06-17T12:56:04.509 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_max": 24, 2022-06-17T12:56:04.509 INFO:teuthology.orchestra.run.smithi036.stdout: "max_osd": 8, 2022-06-17T12:56:04.510 INFO:teuthology.orchestra.run.smithi036.stdout: "require_min_compat_client": "jewel", 2022-06-17T12:56:04.510 INFO:teuthology.orchestra.run.smithi036.stdout: "min_compat_client": "jewel", 2022-06-17T12:56:04.511 INFO:teuthology.orchestra.run.smithi036.stdout: "require_osd_release": "octopus", 2022-06-17T12:56:04.511 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:56:04.512 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.512 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 1, 2022-06-17T12:56:04.513 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "device_health_metrics", 2022-06-17T12:56:04.513 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:33.881212+0000", 2022-06-17T12:56:04.514 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:56:04.515 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:56:04.515 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:56:04.516 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:56:04.516 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:56:04.517 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:56:04.519 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:56:04.519 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:56:04.520 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 1, 2022-06-17T12:56:04.520 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 1, 2022-06-17T12:56:04.525 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 1, 2022-06-17T12:56:04.526 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 1, 2022-06-17T12:56:04.526 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 1, 2022-06-17T12:56:04.527 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:56:04.530 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:56:04.530 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:56:04.531 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:56:04.531 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:56:04.532 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:56:04.532 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:56:04.533 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.533 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "16", 2022-06-17T12:56:04.534 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:56:04.534 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:56:04.535 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:56:04.536 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:56:04.536 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:56:04.537 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:56:04.538 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:56:04.538 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:56:04.539 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:56:04.539 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:56:04.540 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:56:04.541 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:56:04.541 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:56:04.542 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:56:04.542 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:56:04.542 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:56:04.543 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:56:04.543 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:56:04.544 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:56:04.544 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:56:04.544 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:56:04.545 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:56:04.545 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:56:04.546 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:56:04.546 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:56:04.548 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:56:04.549 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.549 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:56:04.551 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:56:04.551 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:56:04.552 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:56:04.552 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:56:04.552 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:56:04.553 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:56:04.554 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:56:04.556 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:56:04.557 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:56:04.557 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:56:04.558 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:56:04.558 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 1 2022-06-17T12:56:04.559 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.574 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:56:04.575 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr_devicehealth": {} 2022-06-17T12:56:04.575 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.576 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.577 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.577 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 2, 2022-06-17T12:56:04.578 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "rbd", 2022-06-17T12:56:04.578 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:44.485460+0000", 2022-06-17T12:56:04.579 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:56:04.579 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:56:04.580 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:56:04.580 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:56:04.581 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:56:04.581 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:56:04.582 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:56:04.582 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:56:04.583 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:56:04.587 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:56:04.588 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:56:04.588 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:56:04.588 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:56:04.589 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:56:04.590 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:56:04.590 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:56:04.591 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:56:04.591 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:56:04.594 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:56:04.595 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:56:04.595 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.596 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "19", 2022-06-17T12:56:04.597 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:56:04.597 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:56:04.598 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:56:04.598 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:56:04.599 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:56:04.601 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:56:04.602 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:56:04.621 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:56:04.621 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:56:04.622 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:56:04.622 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:56:04.622 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:56:04.623 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:56:04.623 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:56:04.624 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:56:04.624 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:56:04.624 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:56:04.625 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:56:04.625 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:56:04.625 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:56:04.625 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:56:04.633 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:56:04.634 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:56:04.634 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:56:04.635 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:56:04.635 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:56:04.635 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.636 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:56:04.636 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:56:04.637 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:56:04.637 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:56:04.638 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:56:04.638 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:56:04.638 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:56:04.639 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:56:04.639 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:56:04.640 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:56:04.640 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:56:04.640 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:56:04.641 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:56:04.641 INFO:teuthology.orchestra.run.smithi036.stdout: "rbd": {} 2022-06-17T12:56:04.641 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.642 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.642 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:56:04.642 INFO:teuthology.orchestra.run.smithi036.stdout: "osds": [ 2022-06-17T12:56:04.643 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.643 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:56:04.645 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", 2022-06-17T12:56:04.645 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:56:04.646 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:56:04.647 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:56:04.647 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:56:04.647 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:56:04.648 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:56:04.648 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:56:04.648 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 122, 2022-06-17T12:56:04.649 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:56:04.649 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:56:04.650 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:56:04.650 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:04.651 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.651 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:04.652 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6802", 2022-06-17T12:56:04.652 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:56:04.652 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.653 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.653 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:04.653 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6803", 2022-06-17T12:56:04.654 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:56:04.654 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.654 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.655 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.655 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:56:04.655 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:04.656 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.656 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:04.657 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6804", 2022-06-17T12:56:04.657 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:56:04.657 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.658 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.660 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:04.660 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6805", 2022-06-17T12:56:04.660 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:56:04.661 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.662 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.662 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.663 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:56:04.663 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:04.664 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.665 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:04.666 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6808", 2022-06-17T12:56:04.667 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:56:04.667 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.668 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.668 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:04.669 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6809", 2022-06-17T12:56:04.669 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:56:04.670 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.670 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.671 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.671 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:56:04.672 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:04.672 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.672 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:04.673 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6806", 2022-06-17T12:56:04.673 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:56:04.674 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.674 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.676 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:04.677 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6807", 2022-06-17T12:56:04.678 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:56:04.678 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.679 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.679 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.680 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6803/22619", 2022-06-17T12:56:04.680 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6805/22619", 2022-06-17T12:56:04.680 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6809/22619", 2022-06-17T12:56:04.681 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6807/22619", 2022-06-17T12:56:04.681 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:56:04.682 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:56:04.682 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:56:04.683 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.683 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.684 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.684 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:56:04.685 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", 2022-06-17T12:56:04.685 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:56:04.686 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:56:04.687 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:56:04.688 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:56:04.689 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:56:04.689 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:56:04.690 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:56:04.690 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 122, 2022-06-17T12:56:04.691 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:56:04.691 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:56:04.692 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:56:04.692 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:04.693 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.693 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:04.693 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6810", 2022-06-17T12:56:04.694 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:56:04.695 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.695 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.695 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:04.696 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6811", 2022-06-17T12:56:04.696 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:56:04.697 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.697 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.697 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.699 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:56:04.699 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:04.700 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.701 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:04.701 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6812", 2022-06-17T12:56:04.702 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:56:04.703 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.704 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.704 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:04.705 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6813", 2022-06-17T12:56:04.706 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:56:04.706 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.706 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.708 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.708 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:56:04.708 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:04.709 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.709 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:04.712 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6816", 2022-06-17T12:56:04.712 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:56:04.712 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.713 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.713 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:04.714 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6817", 2022-06-17T12:56:04.714 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:56:04.715 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.715 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.716 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.716 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:56:04.717 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:04.717 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.717 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:04.718 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6814", 2022-06-17T12:56:04.718 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:56:04.719 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.719 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.719 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:04.719 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6815", 2022-06-17T12:56:04.720 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:56:04.720 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.720 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.721 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.721 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6811/22620", 2022-06-17T12:56:04.722 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6813/22620", 2022-06-17T12:56:04.722 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6817/22620", 2022-06-17T12:56:04.723 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6815/22620", 2022-06-17T12:56:04.723 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:56:04.723 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:56:04.723 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:56:04.724 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.724 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.724 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.724 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:56:04.725 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", 2022-06-17T12:56:04.725 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:56:04.725 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:56:04.725 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:56:04.726 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:56:04.726 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:56:04.726 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:56:04.726 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:56:04.727 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 117, 2022-06-17T12:56:04.727 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:56:04.727 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:56:04.727 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:56:04.728 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:04.728 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.728 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:04.728 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6826", 2022-06-17T12:56:04.729 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:56:04.729 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.729 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.729 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:04.730 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6827", 2022-06-17T12:56:04.730 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:56:04.730 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.731 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.731 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.731 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:56:04.731 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:04.732 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.732 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:04.732 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6828", 2022-06-17T12:56:04.732 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:56:04.733 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.733 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.733 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:04.733 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6829", 2022-06-17T12:56:04.734 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:56:04.734 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.734 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.734 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.735 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:56:04.735 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:04.735 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.736 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:04.736 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6832", 2022-06-17T12:56:04.736 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:56:04.737 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.737 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.737 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:04.738 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6833", 2022-06-17T12:56:04.738 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:56:04.739 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.739 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.740 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.740 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:56:04.740 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:04.741 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.742 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:04.742 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6830", 2022-06-17T12:56:04.743 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:56:04.743 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.744 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.745 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:04.745 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6831", 2022-06-17T12:56:04.746 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:56:04.746 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.746 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.747 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.747 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6827/22643", 2022-06-17T12:56:04.747 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6829/22643", 2022-06-17T12:56:04.747 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6833/22643", 2022-06-17T12:56:04.747 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6831/22643", 2022-06-17T12:56:04.748 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:56:04.749 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:56:04.749 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:56:04.749 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.750 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.750 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.750 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:56:04.751 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", 2022-06-17T12:56:04.751 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:56:04.751 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:56:04.752 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:56:04.752 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:56:04.752 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:56:04.753 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:56:04.753 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:56:04.754 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 122, 2022-06-17T12:56:04.754 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:56:04.755 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:56:04.755 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:56:04.755 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:04.756 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.756 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:04.756 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6818", 2022-06-17T12:56:04.757 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:56:04.757 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.758 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.758 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:04.759 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6819", 2022-06-17T12:56:04.759 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:56:04.760 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.760 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.761 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.761 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:56:04.762 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:04.762 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.763 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:04.763 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6820", 2022-06-17T12:56:04.763 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:56:04.764 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.764 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.764 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:04.765 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6821", 2022-06-17T12:56:04.765 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:56:04.766 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.766 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.766 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.767 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:56:04.767 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:04.768 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.768 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:04.769 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6824", 2022-06-17T12:56:04.769 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:56:04.770 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.770 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.771 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:04.771 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6825", 2022-06-17T12:56:04.771 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:56:04.772 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.772 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.773 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.773 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:56:04.774 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:04.774 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.774 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:04.775 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6822", 2022-06-17T12:56:04.775 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:56:04.776 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.776 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.777 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:04.777 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6823", 2022-06-17T12:56:04.779 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:56:04.779 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.780 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.780 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.780 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6819/22646", 2022-06-17T12:56:04.781 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6821/22646", 2022-06-17T12:56:04.781 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6825/22646", 2022-06-17T12:56:04.782 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6823/22646", 2022-06-17T12:56:04.782 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:56:04.783 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:56:04.783 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:56:04.784 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.784 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.785 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.785 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:56:04.786 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", 2022-06-17T12:56:04.786 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:56:04.809 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:56:04.810 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:56:04.810 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:56:04.811 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:56:04.813 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:56:04.813 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:56:04.814 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 117, 2022-06-17T12:56:04.814 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:56:04.815 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:56:04.825 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:56:04.825 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:04.826 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.826 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:04.826 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6808", 2022-06-17T12:56:04.827 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:56:04.827 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.828 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.828 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:04.828 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6809", 2022-06-17T12:56:04.829 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:56:04.829 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.829 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.830 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.831 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:56:04.831 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:04.831 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.832 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:04.834 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6810", 2022-06-17T12:56:04.834 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:56:04.835 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.835 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.836 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:04.837 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6811", 2022-06-17T12:56:04.838 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:56:04.838 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.839 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.839 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.839 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:56:04.840 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:04.840 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.841 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:04.841 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6814", 2022-06-17T12:56:04.841 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:56:04.842 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.843 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.843 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:04.844 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6815", 2022-06-17T12:56:04.860 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:56:04.860 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.860 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.861 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.861 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:56:04.862 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:04.863 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.863 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:04.864 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6812", 2022-06-17T12:56:04.864 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:56:04.865 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.865 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.865 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:04.866 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6813", 2022-06-17T12:56:04.868 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:56:04.868 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.869 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.869 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.869 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6809/22812", 2022-06-17T12:56:04.870 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6811/22812", 2022-06-17T12:56:04.870 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6815/22812", 2022-06-17T12:56:04.871 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6813/22812", 2022-06-17T12:56:04.871 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:56:04.871 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:56:04.872 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:56:04.872 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.872 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.873 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.873 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:56:04.873 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", 2022-06-17T12:56:04.874 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:56:04.875 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:56:04.875 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:56:04.875 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:56:04.876 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:56:04.876 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:56:04.877 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:56:04.877 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 122, 2022-06-17T12:56:04.878 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:56:04.878 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:56:04.879 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:56:04.880 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:04.881 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.881 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:04.882 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6800", 2022-06-17T12:56:04.882 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:56:04.883 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.883 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.883 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:04.884 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6801", 2022-06-17T12:56:04.898 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:56:04.898 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.899 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.899 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.900 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:56:04.900 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:04.900 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.901 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:04.901 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6802", 2022-06-17T12:56:04.903 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:56:04.903 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.904 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.904 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:04.904 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6803", 2022-06-17T12:56:04.905 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:56:04.905 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.905 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.907 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.919 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:56:04.919 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:04.921 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.922 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:04.923 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6806", 2022-06-17T12:56:04.923 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:56:04.924 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.925 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.925 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:04.926 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6807", 2022-06-17T12:56:04.926 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:56:04.927 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.927 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.938 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.939 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:56:04.939 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:04.940 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.940 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:04.941 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6804", 2022-06-17T12:56:04.941 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:56:04.949 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.949 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:04.950 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:04.950 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6805", 2022-06-17T12:56:04.951 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:56:04.951 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:04.952 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.953 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.953 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6801/22813", 2022-06-17T12:56:04.970 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6803/22813", 2022-06-17T12:56:04.995 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6807/22813", 2022-06-17T12:56:04.996 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6805/22813", 2022-06-17T12:56:04.996 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:56:04.996 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:56:04.997 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:56:04.997 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:04.997 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:04.998 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.000 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:56:05.001 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", 2022-06-17T12:56:05.001 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:56:05.002 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:56:05.003 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:56:05.004 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:56:05.004 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:56:05.005 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:56:05.005 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:56:05.007 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 122, 2022-06-17T12:56:05.008 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:56:05.011 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:56:05.015 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:56:05.015 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:05.016 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.016 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:05.017 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6816", 2022-06-17T12:56:05.017 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:56:05.018 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.019 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.019 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:05.022 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6817", 2022-06-17T12:56:05.022 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:56:05.023 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:05.023 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:05.024 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.024 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:56:05.025 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:05.025 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.026 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:05.026 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6818", 2022-06-17T12:56:05.026 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:56:05.027 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.027 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.028 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:05.029 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6819", 2022-06-17T12:56:05.029 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:56:05.031 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:05.032 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:05.032 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.032 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:56:05.033 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:05.033 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.033 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:05.035 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6822", 2022-06-17T12:56:05.035 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:56:05.044 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.049 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.050 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:05.050 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6823", 2022-06-17T12:56:05.053 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:56:05.053 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:05.054 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:05.054 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.055 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:56:05.055 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:05.067 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.067 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:05.068 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6820", 2022-06-17T12:56:05.069 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:56:05.069 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.070 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.070 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:05.071 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6821", 2022-06-17T12:56:05.071 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:56:05.072 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:05.072 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:05.073 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.074 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6817/22822", 2022-06-17T12:56:05.075 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6819/22822", 2022-06-17T12:56:05.075 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6823/22822", 2022-06-17T12:56:05.076 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6821/22822", 2022-06-17T12:56:05.077 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:56:05.077 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:56:05.091 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:56:05.091 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:05.092 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.092 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.093 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:56:05.093 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", 2022-06-17T12:56:05.094 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:56:05.094 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:56:05.095 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:56:05.096 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:56:05.096 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:56:05.097 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:56:05.097 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:56:05.097 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 122, 2022-06-17T12:56:05.098 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:56:05.098 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:56:05.099 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:56:05.099 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:05.100 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.100 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:05.101 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6824", 2022-06-17T12:56:05.102 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:56:05.102 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.103 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.103 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:05.104 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6825", 2022-06-17T12:56:05.104 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:56:05.105 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:05.105 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:05.106 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.106 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:56:05.106 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:05.107 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.107 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:05.108 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6826", 2022-06-17T12:56:05.108 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:56:05.109 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.121 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.121 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:05.121 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6827", 2022-06-17T12:56:05.122 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:56:05.122 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:05.122 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:05.123 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.130 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:56:05.130 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:05.131 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.131 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:05.132 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6830", 2022-06-17T12:56:05.132 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:56:05.133 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.133 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.134 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:05.134 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6831", 2022-06-17T12:56:05.134 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:56:05.135 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:05.135 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:05.136 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.159 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:56:05.159 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:05.159 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.159 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:05.160 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6828", 2022-06-17T12:56:05.160 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:56:05.160 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.161 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.161 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:05.162 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6829", 2022-06-17T12:56:05.162 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:56:05.170 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:05.170 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:05.170 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.171 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6825/22837", 2022-06-17T12:56:05.171 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6827/22837", 2022-06-17T12:56:05.172 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6831/22837", 2022-06-17T12:56:05.172 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6829/22837", 2022-06-17T12:56:05.173 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:56:05.173 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:56:05.174 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:56:05.174 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:05.175 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:05.175 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:56:05.176 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_xinfo": [ 2022-06-17T12:56:05.176 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.177 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:56:05.177 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:56:05.178 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:56:05.178 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:56:05.179 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:56:05.191 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:56:05.191 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.291461+0000", 2022-06-17T12:56:05.192 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:56:05.192 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.193 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.193 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:56:05.193 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:56:05.194 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:56:05.195 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:56:05.195 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:56:05.196 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:56:05.196 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.537912+0000", 2022-06-17T12:56:05.197 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:56:05.197 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.198 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.198 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:56:05.199 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:56:05.199 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:56:05.199 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:56:05.200 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:56:05.201 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:56:05.201 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.606947+0000", 2022-06-17T12:56:05.202 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:56:05.203 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.203 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.204 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:56:05.204 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:56:05.204 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:56:05.204 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:56:05.205 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:56:05.205 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:56:05.205 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.514412+0000", 2022-06-17T12:56:05.205 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:56:05.206 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.206 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.206 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:56:05.206 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:56:05.207 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:56:05.207 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:56:05.215 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:56:05.215 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:56:05.215 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.394645+0000", 2022-06-17T12:56:05.215 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:56:05.216 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.216 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.216 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:56:05.216 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:56:05.217 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:56:05.217 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:56:05.217 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:56:05.221 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:56:05.221 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.343806+0000", 2022-06-17T12:56:05.221 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:56:05.222 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.222 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.222 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:56:05.223 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:56:05.224 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:56:05.225 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:56:05.226 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:56:05.226 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:56:05.226 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.591864+0000", 2022-06-17T12:56:05.227 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:56:05.227 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.228 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:05.228 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:56:05.229 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:56:05.229 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:56:05.230 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:56:05.230 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:56:05.230 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:56:05.231 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.607045+0000", 2022-06-17T12:56:05.232 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:56:05.232 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:05.233 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:56:05.234 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap": [], 2022-06-17T12:56:05.235 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap_items": [], 2022-06-17T12:56:05.235 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_temp": [], 2022-06-17T12:56:05.236 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_temp": [], 2022-06-17T12:56:05.236 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklist": { 2022-06-17T12:56:05.237 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.36:6835/1342585509": "2022-06-18T12:55:41.828746+0000", 2022-06-17T12:56:05.237 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.36:6834/1342585509": "2022-06-18T12:55:41.828746+0000" 2022-06-17T12:56:05.238 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.238 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profiles": { 2022-06-17T12:56:05.239 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs_data": { 2022-06-17T12:56:05.239 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-device-class": "", 2022-06-17T12:56:05.240 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:56:05.240 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-root": "default", 2022-06-17T12:56:05.241 INFO:teuthology.orchestra.run.smithi036.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-17T12:56:05.241 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:56:05.242 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "2", 2022-06-17T12:56:05.242 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:56:05.243 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:56:05.244 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van", 2022-06-17T12:56:05.244 INFO:teuthology.orchestra.run.smithi036.stdout: "w": "8" 2022-06-17T12:56:05.245 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.245 INFO:teuthology.orchestra.run.smithi036.stdout: "default": { 2022-06-17T12:56:05.246 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:56:05.246 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:56:05.247 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "1", 2022-06-17T12:56:05.247 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:56:05.248 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:56:05.248 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van" 2022-06-17T12:56:05.248 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:05.249 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:05.249 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps_queue": [], 2022-06-17T12:56:05.250 INFO:teuthology.orchestra.run.smithi036.stdout: "new_removed_snaps": [], 2022-06-17T12:56:05.250 INFO:teuthology.orchestra.run.smithi036.stdout: "new_purged_snaps": [], 2022-06-17T12:56:05.251 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_node_flags": {}, 2022-06-17T12:56:05.251 INFO:teuthology.orchestra.run.smithi036.stdout: "device_class_flags": {} 2022-06-17T12:56:05.252 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:56:05.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.503+0000 7fbd252af700 1 -- 172.21.15.36:0/2349617641 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbd04053a60 msgr2=0x7fbd04055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:05.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.503+0000 7fbd252af700 1 --2- 172.21.15.36:0/2349617641 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbd04053a60 0x7fbd04055f20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fbd0801a000 tx=0x7fbd08014000).stop 2022-06-17T12:56:05.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.503+0000 7fbd252af700 1 -- 172.21.15.36:0/2349617641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd200b31c0 msgr2=0x7fbd200b8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:05.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.503+0000 7fbd252af700 1 --2- 172.21.15.36:0/2349617641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd200b31c0 0x7fbd200b8670 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fbd10001110 tx=0x7fbd100079e0).stop 2022-06-17T12:56:05.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.503+0000 7fbd252af700 1 -- 172.21.15.36:0/2349617641 shutdown_connections 2022-06-17T12:56:05.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.503+0000 7fbd252af700 1 --2- 172.21.15.36:0/2349617641 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbd04053a60 0x7fbd04055f20 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:05.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.503+0000 7fbd252af700 1 --2- 172.21.15.36:0/2349617641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd200b4e10 0x7fbd201140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:05.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.503+0000 7fbd252af700 1 --2- 172.21.15.36:0/2349617641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd200b31c0 0x7fbd200b8670 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:05.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.503+0000 7fbd252af700 1 --2- 172.21.15.36:0/2349617641 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbd200620d0 0x7fbd201136b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:05.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.503+0000 7fbd252af700 1 -- 172.21.15.36:0/2349617641 >> 172.21.15.36:0/2349617641 conn(0x7fbd200b9850 msgr2=0x7fbd2005f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:05.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.503+0000 7fbd252af700 1 -- 172.21.15.36:0/2349617641 shutdown_connections 2022-06-17T12:56:05.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:04.503+0000 7fbd252af700 1 -- 172.21.15.36:0/2349617641 wait complete. 2022-06-17T12:56:05.263 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:56:05.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334fdcc700 1 Processor -- start 2022-06-17T12:56:05.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334fdcc700 1 -- start start 2022-06-17T12:56:05.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334fdcc700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3348144c00 0x7f3348062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:05.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334fdcc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33480ad330 0x7f33480ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:05.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334fdcc700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33480b0ba0 con 0x7f3348062770 2022-06-17T12:56:05.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334fdcc700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33480ae820 con 0x7f3348144c00 2022-06-17T12:56:05.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334fdcc700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3348061030 con 0x7f33480ad330 2022-06-17T12:56:05.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334d367700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3348144c00 0x7f3348062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:05.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334e369700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33480ad330 0x7f33480ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:05.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334d367700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3348144c00 0x7f3348062b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:59460/0 (socket says 172.21.15.36:59460) 2022-06-17T12:56:05.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334d367700 1 -- 172.21.15.36:0/3801167141 learned_addr learned my addr 172.21.15.36:0/3801167141 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:05.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334d367700 1 -- 172.21.15.36:0/3801167141 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33480ad330 msgr2=0x7f33480ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:05.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334d367700 1 --2- 172.21.15.36:0/3801167141 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33480ad330 0x7f33480ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:05.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334d367700 1 -- 172.21.15.36:0/3801167141 >> v1:172.21.15.70:6789/0 conn(0x7f3348062770 legacy=0x7f33481444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:05.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334d367700 1 -- 172.21.15.36:0/3801167141 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3348147e90 con 0x7f3348144c00 2022-06-17T12:56:05.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334d367700 1 --2- 172.21.15.36:0/3801167141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3348144c00 0x7f3348062b50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f3348061170 tx=0x7f3338009290).ready entity=mon.1 client_cookie=ef76bb3dfa26a1b0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:05.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334cb66700 1 -- 172.21.15.36:0/3801167141 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f333800b860 con 0x7f3348144c00 2022-06-17T12:56:05.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334cb66700 1 -- 172.21.15.36:0/3801167141 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333800b9c0 con 0x7f3348144c00 2022-06-17T12:56:05.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334cb66700 1 -- 172.21.15.36:0/3801167141 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f333800bb30 con 0x7f3348144c00 2022-06-17T12:56:05.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334fdcc700 1 -- 172.21.15.36:0/3801167141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3348144c00 msgr2=0x7f3348062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:05.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334fdcc700 1 --2- 172.21.15.36:0/3801167141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3348144c00 0x7f3348062b50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f3348061170 tx=0x7f3338009290).stop 2022-06-17T12:56:05.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334fdcc700 1 -- 172.21.15.36:0/3801167141 shutdown_connections 2022-06-17T12:56:05.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334fdcc700 1 --2- 172.21.15.36:0/3801167141 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33480ad330 0x7f33480ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:05.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334fdcc700 1 --2- 172.21.15.36:0/3801167141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3348144c00 0x7f3348062b50 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:05.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334fdcc700 1 -- 172.21.15.36:0/3801167141 >> 172.21.15.36:0/3801167141 conn(0x7f33480b53e0 msgr2=0x7f33480b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:05.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334fdcc700 1 -- 172.21.15.36:0/3801167141 shutdown_connections 2022-06-17T12:56:05.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.391+0000 7f334fdcc700 1 -- 172.21.15.36:0/3801167141 wait complete. 2022-06-17T12:56:05.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.395+0000 7f334fdcc700 1 Processor -- start 2022-06-17T12:56:05.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.395+0000 7f334fdcc700 1 -- start start 2022-06-17T12:56:05.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.395+0000 7f334fdcc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3348062770 0x7f33481314d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:05.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.395+0000 7f334fdcc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33480ad330 0x7f3348143fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:05.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.395+0000 7f334fdcc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3348144c00 0x7f33481444a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:05.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.395+0000 7f334fdcc700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33481571c0 con 0x7f33480ad330 2022-06-17T12:56:05.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.395+0000 7f334fdcc700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3348067390 con 0x7f3348144c00 2022-06-17T12:56:05.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.395+0000 7f334fdcc700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3348144990 con 0x7f3348062770 2022-06-17T12:56:05.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.395+0000 7f334d367700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33480ad330 0x7f3348143fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:05.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.395+0000 7f334db68700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3348062770 0x7f33481314d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:05.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.395+0000 7f334e369700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3348144c00 0x7f33481444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:05.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.395+0000 7f334d367700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33480ad330 0x7f3348143fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39084/0 (socket says 172.21.15.36:39084) 2022-06-17T12:56:05.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.395+0000 7f334db68700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3348062770 0x7f33481314d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:53872/0 (socket says 172.21.15.36:53872) 2022-06-17T12:56:05.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.395+0000 7f334d367700 1 -- 172.21.15.36:0/483644309 learned_addr learned my addr 172.21.15.36:0/483644309 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:05.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.395+0000 7f334d367700 1 -- 172.21.15.36:0/483644309 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3348062770 msgr2=0x7f33481314d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:05.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.399+0000 7f334d367700 1 --2- 172.21.15.36:0/483644309 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3348062770 0x7f33481314d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:05.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.399+0000 7f334d367700 1 -- 172.21.15.36:0/483644309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3348144c00 msgr2=0x7f33481444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:05.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.399+0000 7f334d367700 1 --2- 172.21.15.36:0/483644309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3348144c00 0x7f33481444a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:05.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.399+0000 7f334d367700 1 -- 172.21.15.36:0/483644309 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3338009530 con 0x7f33480ad330 2022-06-17T12:56:05.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.399+0000 7f334db68700 1 --2- 172.21.15.36:0/483644309 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3348062770 0x7f33481314d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:05.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.399+0000 7f334d367700 1 --2- 172.21.15.36:0/483644309 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33480ad330 0x7f3348143fb0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f3348061170 tx=0x7f3338016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:05.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.399+0000 7f3346ffd700 1 -- 172.21.15.36:0/483644309 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3338021630 con 0x7f33480ad330 2022-06-17T12:56:05.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.399+0000 7f3346ffd700 1 -- 172.21.15.36:0/483644309 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3338021790 con 0x7f33480ad330 2022-06-17T12:56:05.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.399+0000 7f334fdcc700 1 -- 172.21.15.36:0/483644309 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334814b5a0 con 0x7f33480ad330 2022-06-17T12:56:05.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.399+0000 7f3346ffd700 1 -- 172.21.15.36:0/483644309 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3338021af0 con 0x7f33480ad330 2022-06-17T12:56:05.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.399+0000 7f334fdcc700 1 -- 172.21.15.36:0/483644309 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33480640c0 con 0x7f33480ad330 2022-06-17T12:56:05.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.399+0000 7f334fdcc700 1 -- 172.21.15.36:0/483644309 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33480b1c20 con 0x7f33480ad330 2022-06-17T12:56:05.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.399+0000 7f3346ffd700 1 -- 172.21.15.36:0/483644309 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f333800f070 con 0x7f33480ad330 2022-06-17T12:56:05.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.399+0000 7f3346ffd700 1 --2- 172.21.15.36:0/483644309 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f332c053ad0 0x7f332c055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:05.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.399+0000 7f334db68700 1 --2- 172.21.15.36:0/483644309 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f332c053ad0 0x7f332c055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:05.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.399+0000 7f3346ffd700 1 -- 172.21.15.36:0/483644309 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f333806d860 con 0x7f33480ad330 2022-06-17T12:56:05.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.403+0000 7f334db68700 1 --2- 172.21.15.36:0/483644309 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f332c053ad0 0x7f332c055f90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f333000d000 tx=0x7f3330009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:05.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.407+0000 7f3346ffd700 1 -- 172.21.15.36:0/483644309 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f33380448e0 con 0x7f33480ad330 2022-06-17T12:56:05.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.567+0000 7f334fdcc700 1 -- 172.21.15.36:0/483644309 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3348057410 con 0x7f33480ad330 2022-06-17T12:56:05.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.567+0000 7f3346ffd700 1 -- 172.21.15.36:0/483644309 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+504 (secure 0 0 0) 0x7f3338015030 con 0x7f33480ad330 2022-06-17T12:56:05.568 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:05.569 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":67,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[]} 2022-06-17T12:56:05.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.571+0000 7f334fdcc700 1 -- 172.21.15.36:0/483644309 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f332c053ad0 msgr2=0x7f332c055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:05.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.571+0000 7f334fdcc700 1 --2- 172.21.15.36:0/483644309 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f332c053ad0 0x7f332c055f90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f333000d000 tx=0x7f3330009040).stop 2022-06-17T12:56:05.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.571+0000 7f334fdcc700 1 -- 172.21.15.36:0/483644309 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33480ad330 msgr2=0x7f3348143fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:05.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.571+0000 7f334fdcc700 1 --2- 172.21.15.36:0/483644309 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33480ad330 0x7f3348143fb0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f3348061170 tx=0x7f3338016000).stop 2022-06-17T12:56:05.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.571+0000 7f334fdcc700 1 -- 172.21.15.36:0/483644309 shutdown_connections 2022-06-17T12:56:05.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.571+0000 7f334fdcc700 1 --2- 172.21.15.36:0/483644309 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f332c053ad0 0x7f332c055f90 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:05.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.571+0000 7f334fdcc700 1 --2- 172.21.15.36:0/483644309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3348144c00 0x7f33481444a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:05.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.571+0000 7f334fdcc700 1 --2- 172.21.15.36:0/483644309 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33480ad330 0x7f3348143fb0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:05.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.571+0000 7f334fdcc700 1 --2- 172.21.15.36:0/483644309 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3348062770 0x7f33481314d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:05.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.571+0000 7f334fdcc700 1 -- 172.21.15.36:0/483644309 >> 172.21.15.36:0/483644309 conn(0x7f33480b53e0 msgr2=0x7f33480b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:05.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.571+0000 7f334fdcc700 1 -- 172.21.15.36:0/483644309 shutdown_connections 2022-06-17T12:56:05.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.571+0000 7f334fdcc700 1 -- 172.21.15.36:0/483644309 wait complete. 2022-06-17T12:56:05.581 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 67 2022-06-17T12:56:05.582 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:56:05.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.723+0000 7fa0da609700 1 Processor -- start 2022-06-17T12:56:05.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.723+0000 7fa0da609700 1 -- start start 2022-06-17T12:56:05.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.723+0000 7fa0da609700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa0d413cb70 0x7fa0d4143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:05.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.723+0000 7fa0da609700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0d40aef60 0x7fa0d40b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:05.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.723+0000 7fa0da609700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0d40af4b0 con 0x7fa0d40ba2e0 2022-06-17T12:56:05.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.723+0000 7fa0da609700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0d4142f50 con 0x7fa0d413cb70 2022-06-17T12:56:05.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.723+0000 7fa0da609700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d41431f0 con 0x7fa0d40aef60 2022-06-17T12:56:05.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.723+0000 7fa0d9607700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa0d413cb70 0x7fa0d4143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:05.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.727+0000 7fa0d9607700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa0d413cb70 0x7fa0d4143690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39092/0 (socket says 172.21.15.36:39092) 2022-06-17T12:56:05.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.727+0000 7fa0d9607700 1 -- 172.21.15.36:0/1592031592 learned_addr learned my addr 172.21.15.36:0/1592031592 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:05.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.727+0000 7fa0d8e06700 1 --2- 172.21.15.36:0/1592031592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0d40aef60 0x7fa0d40b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:05.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.727+0000 7fa0c3fff700 1 -- 172.21.15.36:0/1592031592 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 391670517 0 0) 0x7fa0d40af4b0 con 0x7fa0d40ba2e0 2022-06-17T12:56:05.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.727+0000 7fa0d9607700 1 -- 172.21.15.36:0/1592031592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0d40aef60 msgr2=0x7fa0d40b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:05.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.727+0000 7fa0d9607700 1 --2- 172.21.15.36:0/1592031592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0d40aef60 0x7fa0d40b9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:05.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.727+0000 7fa0d9607700 1 -- 172.21.15.36:0/1592031592 >> v1:172.21.15.70:6789/0 conn(0x7fa0d40ba2e0 legacy=0x7fa0d41442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:05.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.727+0000 7fa0d9607700 1 -- 172.21.15.36:0/1592031592 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d41449b0 con 0x7fa0d413cb70 2022-06-17T12:56:05.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.727+0000 7fa0d9607700 1 --2- 172.21.15.36:0/1592031592 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa0d413cb70 0x7fa0d4143690 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7fa0d0002f70 tx=0x7fa0d0006eb0).ready entity=mon.0 client_cookie=d2d472d728309fe5 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:05.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.727+0000 7fa0c3fff700 1 -- 172.21.15.36:0/1592031592 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0d000d570 con 0x7fa0d413cb70 2022-06-17T12:56:05.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.727+0000 7fa0c3fff700 1 -- 172.21.15.36:0/1592031592 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d0000e90 con 0x7fa0d413cb70 2022-06-17T12:56:05.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.727+0000 7fa0c3fff700 1 -- 172.21.15.36:0/1592031592 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0d000d8c0 con 0x7fa0d413cb70 2022-06-17T12:56:05.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.727+0000 7fa0da609700 1 -- 172.21.15.36:0/1592031592 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa0d413cb70 msgr2=0x7fa0d4143690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:05.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.727+0000 7fa0da609700 1 --2- 172.21.15.36:0/1592031592 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa0d413cb70 0x7fa0d4143690 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7fa0d0002f70 tx=0x7fa0d0006eb0).stop 2022-06-17T12:56:05.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.727+0000 7fa0da609700 1 -- 172.21.15.36:0/1592031592 shutdown_connections 2022-06-17T12:56:05.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.727+0000 7fa0da609700 1 --2- 172.21.15.36:0/1592031592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0d40aef60 0x7fa0d40b9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:05.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.727+0000 7fa0da609700 1 --2- 172.21.15.36:0/1592031592 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa0d413cb70 0x7fa0d4143690 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:05.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.727+0000 7fa0da609700 1 -- 172.21.15.36:0/1592031592 >> 172.21.15.36:0/1592031592 conn(0x7fa0d405c550 msgr2=0x7fa0d40b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:05.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.727+0000 7fa0da609700 1 -- 172.21.15.36:0/1592031592 shutdown_connections 2022-06-17T12:56:05.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.727+0000 7fa0da609700 1 -- 172.21.15.36:0/1592031592 wait complete. 2022-06-17T12:56:05.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.727+0000 7fa0da609700 1 Processor -- start 2022-06-17T12:56:05.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.727+0000 7fa0da609700 1 -- start start 2022-06-17T12:56:05.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.731+0000 7fa0da609700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa0d40aef60 0x7fa0d413d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:05.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.731+0000 7fa0da609700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0d40ba2e0 0x7fa0d4155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:05.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.731+0000 7fa0da609700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0d413cb70 0x7fa0d4148470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:05.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.731+0000 7fa0da609700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d4155ca0 con 0x7fa0d40aef60 2022-06-17T12:56:05.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.731+0000 7fa0da609700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d4155f40 con 0x7fa0d40ba2e0 2022-06-17T12:56:05.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.731+0000 7fa0da609700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d41561e0 con 0x7fa0d413cb70 2022-06-17T12:56:05.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.731+0000 7fa0d9607700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa0d40aef60 0x7fa0d413d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:05.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.731+0000 7fa0d9607700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa0d40aef60 0x7fa0d413d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39098/0 (socket says 172.21.15.36:39098) 2022-06-17T12:56:05.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.731+0000 7fa0d9607700 1 -- 172.21.15.36:0/3139145344 learned_addr learned my addr 172.21.15.36:0/3139145344 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:05.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.731+0000 7fa0d8e06700 1 --2- 172.21.15.36:0/3139145344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0d40ba2e0 0x7fa0d4155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:05.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.731+0000 7fa0d9e08700 1 --2- 172.21.15.36:0/3139145344 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0d413cb70 0x7fa0d4148470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:05.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.731+0000 7fa0d9607700 1 -- 172.21.15.36:0/3139145344 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0d413cb70 msgr2=0x7fa0d4148470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:05.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.731+0000 7fa0d9607700 1 --2- 172.21.15.36:0/3139145344 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0d413cb70 0x7fa0d4148470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:05.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.731+0000 7fa0d9607700 1 -- 172.21.15.36:0/3139145344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0d40ba2e0 msgr2=0x7fa0d4155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:05.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.731+0000 7fa0d9607700 1 --2- 172.21.15.36:0/3139145344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0d40ba2e0 0x7fa0d4155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:05.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.731+0000 7fa0d9607700 1 -- 172.21.15.36:0/3139145344 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d000b1e0 con 0x7fa0d40aef60 2022-06-17T12:56:05.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.731+0000 7fa0d9607700 1 --2- 172.21.15.36:0/3139145344 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa0d40aef60 0x7fa0d413d4d0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7fa0d0002f70 tx=0x7fa0d0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:05.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.731+0000 7fa0c27fc700 1 -- 172.21.15.36:0/3139145344 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0d00118a0 con 0x7fa0d40aef60 2022-06-17T12:56:05.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.731+0000 7fa0c27fc700 1 -- 172.21.15.36:0/3139145344 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d0011a00 con 0x7fa0d40aef60 2022-06-17T12:56:05.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.731+0000 7fa0da609700 1 -- 172.21.15.36:0/3139145344 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0d40af6b0 con 0x7fa0d40aef60 2022-06-17T12:56:05.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.731+0000 7fa0c27fc700 1 -- 172.21.15.36:0/3139145344 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0d0011d60 con 0x7fa0d40aef60 2022-06-17T12:56:05.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.731+0000 7fa0da609700 1 -- 172.21.15.36:0/3139145344 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0d4148960 con 0x7fa0d40aef60 2022-06-17T12:56:05.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.735+0000 7fa0da609700 1 -- 172.21.15.36:0/3139145344 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0d4155af0 con 0x7fa0d40aef60 2022-06-17T12:56:05.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.735+0000 7fa0c27fc700 1 -- 172.21.15.36:0/3139145344 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fa0d0025030 con 0x7fa0d40aef60 2022-06-17T12:56:05.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.735+0000 7fa0c27fc700 1 --2- 172.21.15.36:0/3139145344 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa0b4053ad0 0x7fa0b4055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:05.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.735+0000 7fa0c27fc700 1 -- 172.21.15.36:0/3139145344 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fa0d006cfe0 con 0x7fa0d40aef60 2022-06-17T12:56:05.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.735+0000 7fa0d8e06700 1 --2- 172.21.15.36:0/3139145344 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa0b4053ad0 0x7fa0b4055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:05.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.735+0000 7fa0d8e06700 1 --2- 172.21.15.36:0/3139145344 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa0b4053ad0 0x7fa0b4055f90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7fa0c400b000 tx=0x7fa0c4006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:05.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.739+0000 7fa0c27fc700 1 -- 172.21.15.36:0/3139145344 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa0d0043fd0 con 0x7fa0d40aef60 2022-06-17T12:56:05.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.903+0000 7fa0da609700 1 -- 172.21.15.36:0/3139145344 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0d41563a0 con 0x7fa0d40aef60 2022-06-17T12:56:05.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.903+0000 7fa0c27fc700 1 -- 172.21.15.36:0/3139145344 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+504 (secure 0 0 0) 0x7fa0d0023090 con 0x7fa0d40aef60 2022-06-17T12:56:05.893 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:05.893 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":67,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[]} 2022-06-17T12:56:05.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.903+0000 7fa0da609700 1 -- 172.21.15.36:0/3139145344 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa0b4053ad0 msgr2=0x7fa0b4055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:05.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.903+0000 7fa0da609700 1 --2- 172.21.15.36:0/3139145344 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa0b4053ad0 0x7fa0b4055f90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7fa0c400b000 tx=0x7fa0c4006df0).stop 2022-06-17T12:56:05.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.903+0000 7fa0da609700 1 -- 172.21.15.36:0/3139145344 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa0d40aef60 msgr2=0x7fa0d413d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:05.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.903+0000 7fa0da609700 1 --2- 172.21.15.36:0/3139145344 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa0d40aef60 0x7fa0d413d4d0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7fa0d0002f70 tx=0x7fa0d0015000).stop 2022-06-17T12:56:05.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.907+0000 7fa0da609700 1 -- 172.21.15.36:0/3139145344 shutdown_connections 2022-06-17T12:56:05.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.907+0000 7fa0da609700 1 --2- 172.21.15.36:0/3139145344 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa0b4053ad0 0x7fa0b4055f90 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:05.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.907+0000 7fa0da609700 1 --2- 172.21.15.36:0/3139145344 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0d413cb70 0x7fa0d4148470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:05.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.907+0000 7fa0da609700 1 --2- 172.21.15.36:0/3139145344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0d40ba2e0 0x7fa0d4155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:05.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.907+0000 7fa0da609700 1 --2- 172.21.15.36:0/3139145344 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa0d40aef60 0x7fa0d413d4d0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:05.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.907+0000 7fa0da609700 1 -- 172.21.15.36:0/3139145344 >> 172.21.15.36:0/3139145344 conn(0x7fa0d405c550 msgr2=0x7fa0d40b74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:05.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.907+0000 7fa0da609700 1 -- 172.21.15.36:0/3139145344 shutdown_connections 2022-06-17T12:56:05.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:05.907+0000 7fa0da609700 1 -- 172.21.15.36:0/3139145344 wait complete. 2022-06-17T12:56:05.917 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 67 2022-06-17T12:56:05.920 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-17T12:56:05.922 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-17T12:56:05.930 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-17T12:56:05.931 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-17T12:56:05.945 INFO:tasks.ceph.mds.a:Started 2022-06-17T12:56:05.946 DEBUG:teuthology.parallel:result is None 2022-06-17T12:56:05.948 INFO:tasks.ceph.mds.b:Started 2022-06-17T12:56:05.949 DEBUG:teuthology.parallel:result is None 2022-06-17T12:56:05.950 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-06-17T12:56:06.023 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2022-06-17T12:56:06.024 INFO:tasks.ceph.mds.a.smithi036.stdout:starting mds.a at 2022-06-17T12:56:06.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.067+0000 7fe8720a7700 1 Processor -- start 2022-06-17T12:56:06.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.071+0000 7fe8720a7700 1 -- start start 2022-06-17T12:56:06.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.071+0000 7fe8720a7700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe86c0ad450 0x7fe86c143180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:06.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.071+0000 7fe8720a7700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe86c062820 con 0x7fe86c0ac950 2022-06-17T12:56:06.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.071+0000 7fe8720a7700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe86c062ac0 con 0x7fe86c0ad070 2022-06-17T12:56:06.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.071+0000 7fe8720a7700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe86c0aefa0 con 0x7fe86c0ad450 2022-06-17T12:56:06.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.071+0000 7fe8708a4700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fe86c0ac950 0x7fe86c140230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56648/0 (socket says 172.21.15.36:56648) 2022-06-17T12:56:06.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.071+0000 7fe8708a4700 1 -- 172.21.15.36:0/4238632383 learned_addr learned my addr 172.21.15.36:0/4238632383 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:06.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.071+0000 7fe8718a6700 1 --2- 172.21.15.36:0/4238632383 >> v2:172.21.15.36:3300/0 conn(0x7fe86c0ad450 0x7fe86c143180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:06.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.071+0000 7fe86bfff700 1 -- 172.21.15.36:0/4238632383 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3865495130 0 0) 0x7fe86c062820 con 0x7fe86c0ac950 2022-06-17T12:56:06.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.071+0000 7fe86bfff700 1 -- 172.21.15.36:0/4238632383 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe858003260 con 0x7fe86c0ac950 2022-06-17T12:56:06.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.071+0000 7fe86bfff700 1 -- 172.21.15.36:0/4238632383 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3470501287 0 0) 0x7fe86c062ac0 con 0x7fe86c0ad070 2022-06-17T12:56:06.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.071+0000 7fe86bfff700 1 -- 172.21.15.36:0/4238632383 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe86c062820 con 0x7fe86c0ad070 2022-06-17T12:56:06.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.071+0000 7fe8718a6700 1 -- 172.21.15.36:0/4238632383 >> v1:172.21.15.70:6789/0 conn(0x7fe86c0ad070 legacy=0x7fe86c062f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:06.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.071+0000 7fe86bfff700 1 -- 172.21.15.36:0/4238632383 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2134505567 0 0) 0x7fe858003260 con 0x7fe86c0ac950 2022-06-17T12:56:06.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.071+0000 7fe8718a6700 1 -- 172.21.15.36:0/4238632383 >> v1:172.21.15.36:6789/0 conn(0x7fe86c0ac950 legacy=0x7fe86c140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:06.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.071+0000 7fe8718a6700 1 -- 172.21.15.36:0/4238632383 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86c140940 con 0x7fe86c0ad450 2022-06-17T12:56:06.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.071+0000 7fe8718a6700 1 --2- 172.21.15.36:0/4238632383 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe86c0ad450 0x7fe86c143180 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7fe860001110 tx=0x7fe85c006df0).ready entity=mon.0 client_cookie=993abd7259eec351 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:06.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.071+0000 7fe86bfff700 1 -- 172.21.15.36:0/4238632383 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe860001660 con 0x7fe86c0ad450 2022-06-17T12:56:06.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.071+0000 7fe86bfff700 1 -- 172.21.15.36:0/4238632383 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85c000e90 con 0x7fe86c0ad450 2022-06-17T12:56:06.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.071+0000 7fe86bfff700 1 -- 172.21.15.36:0/4238632383 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe85c012460 con 0x7fe86c0ad450 2022-06-17T12:56:06.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.075+0000 7fe8720a7700 1 -- 172.21.15.36:0/4238632383 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe86c0ad450 msgr2=0x7fe86c143180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:06.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.075+0000 7fe8720a7700 1 --2- 172.21.15.36:0/4238632383 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe86c0ad450 0x7fe86c143180 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7fe860001110 tx=0x7fe85c006df0).stop 2022-06-17T12:56:06.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.075+0000 7fe8720a7700 1 -- 172.21.15.36:0/4238632383 shutdown_connections 2022-06-17T12:56:06.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.075+0000 7fe8720a7700 1 --2- 172.21.15.36:0/4238632383 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe86c0ad450 0x7fe86c143180 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:06.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.075+0000 7fe8720a7700 1 -- 172.21.15.36:0/4238632383 >> 172.21.15.36:0/4238632383 conn(0x7fe86c060990 msgr2=0x7fe86c05f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:06.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.075+0000 7fe8720a7700 1 -- 172.21.15.36:0/4238632383 shutdown_connections 2022-06-17T12:56:06.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.075+0000 7fe8720a7700 1 -- 172.21.15.36:0/4238632383 wait complete. 2022-06-17T12:56:06.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.075+0000 7fe8720a7700 1 Processor -- start 2022-06-17T12:56:06.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.075+0000 7fe8720a7700 1 -- start start 2022-06-17T12:56:06.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.075+0000 7fe8720a7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe86c0ac950 0x7fe86c065a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:06.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.075+0000 7fe8720a7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe86c0ad070 0x7fe86c0aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:06.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.075+0000 7fe8720a7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe86c0ad450 0x7fe86c153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:06.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.075+0000 7fe8720a7700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c09f2a0 con 0x7fe86c0ac950 2022-06-17T12:56:06.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.075+0000 7fe8720a7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c1537e0 con 0x7fe86c0ad070 2022-06-17T12:56:06.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.075+0000 7fe8720a7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c09f000 con 0x7fe86c0ad450 2022-06-17T12:56:06.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.075+0000 7fe8710a5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe86c0ac950 0x7fe86c065a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:06.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.079+0000 7fe8710a5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe86c0ac950 0x7fe86c065a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39122/0 (socket says 172.21.15.36:39122) 2022-06-17T12:56:06.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.079+0000 7fe8710a5700 1 -- 172.21.15.36:0/287288704 learned_addr learned my addr 172.21.15.36:0/287288704 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:06.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.079+0000 7fe8708a4700 1 --2- 172.21.15.36:0/287288704 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe86c0ad070 0x7fe86c0aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:06.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.079+0000 7fe8718a6700 1 --2- 172.21.15.36:0/287288704 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe86c0ad450 0x7fe86c153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:06.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.079+0000 7fe8710a5700 1 -- 172.21.15.36:0/287288704 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe86c0ad450 msgr2=0x7fe86c153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:06.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.079+0000 7fe8710a5700 1 --2- 172.21.15.36:0/287288704 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe86c0ad450 0x7fe86c153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:06.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.079+0000 7fe8710a5700 1 -- 172.21.15.36:0/287288704 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe86c0ad070 msgr2=0x7fe86c0aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:06.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.079+0000 7fe8710a5700 1 --2- 172.21.15.36:0/287288704 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe86c0ad070 0x7fe86c0aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:06.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.079+0000 7fe8710a5700 1 -- 172.21.15.36:0/287288704 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85c00b180 con 0x7fe86c0ac950 2022-06-17T12:56:06.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.079+0000 7fe8718a6700 1 --2- 172.21.15.36:0/287288704 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe86c0ad450 0x7fe86c153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:56:06.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.079+0000 7fe8710a5700 1 --2- 172.21.15.36:0/287288704 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe86c0ac950 0x7fe86c065a80 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7fe864001490 tx=0x7fe864007e60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:06.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.079+0000 7fe86a7fc700 1 -- 172.21.15.36:0/287288704 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe864002310 con 0x7fe86c0ac950 2022-06-17T12:56:06.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.079+0000 7fe86a7fc700 1 -- 172.21.15.36:0/287288704 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe864002470 con 0x7fe86c0ac950 2022-06-17T12:56:06.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.079+0000 7fe8720a7700 1 -- 172.21.15.36:0/287288704 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe86c142f30 con 0x7fe86c0ac950 2022-06-17T12:56:06.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.079+0000 7fe86a7fc700 1 -- 172.21.15.36:0/287288704 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe864001840 con 0x7fe86c0ac950 2022-06-17T12:56:06.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.079+0000 7fe8720a7700 1 -- 172.21.15.36:0/287288704 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe86c142540 con 0x7fe86c0ac950 2022-06-17T12:56:06.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.079+0000 7fe8720a7700 1 -- 172.21.15.36:0/287288704 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe86c057410 con 0x7fe86c0ac950 2022-06-17T12:56:06.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.079+0000 7fe86a7fc700 1 -- 172.21.15.36:0/287288704 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe864010030 con 0x7fe86c0ac950 2022-06-17T12:56:06.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.079+0000 7fe86a7fc700 1 --2- 172.21.15.36:0/287288704 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe84c053a40 0x7fe84c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:06.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.079+0000 7fe86a7fc700 1 -- 172.21.15.36:0/287288704 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fe86406bd10 con 0x7fe86c0ac950 2022-06-17T12:56:06.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.079+0000 7fe8708a4700 1 --2- 172.21.15.36:0/287288704 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe84c053a40 0x7fe84c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:06.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.083+0000 7fe8708a4700 1 --2- 172.21.15.36:0/287288704 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe84c053a40 0x7fe84c055f00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7fe86000d000 tx=0x7fe860000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:06.126 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.087+0000 7fe86a7fc700 1 -- 172.21.15.36:0/287288704 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe864042df0 con 0x7fe86c0ac950 2022-06-17T12:56:06.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:06.227+0000 7fe8720a7700 1 -- 172.21.15.36:0/287288704 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fe86c1539a0 con 0x7fe86c0ac950 2022-06-17T12:56:07.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.079+0000 7fe86a7fc700 1 -- 172.21.15.36:0/287288704 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v127) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe864046410 con 0x7fe86c0ac950 2022-06-17T12:56:07.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.083+0000 7fe8720a7700 1 -- 172.21.15.36:0/287288704 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe84c053a40 msgr2=0x7fe84c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:07.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.083+0000 7fe8720a7700 1 --2- 172.21.15.36:0/287288704 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe84c053a40 0x7fe84c055f00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7fe86000d000 tx=0x7fe860000bc0).stop 2022-06-17T12:56:07.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.083+0000 7fe8720a7700 1 -- 172.21.15.36:0/287288704 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe86c0ac950 msgr2=0x7fe86c065a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:07.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.083+0000 7fe8720a7700 1 --2- 172.21.15.36:0/287288704 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe86c0ac950 0x7fe86c065a80 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7fe864001490 tx=0x7fe864007e60).stop 2022-06-17T12:56:07.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.083+0000 7fe8720a7700 1 -- 172.21.15.36:0/287288704 shutdown_connections 2022-06-17T12:56:07.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.083+0000 7fe8720a7700 1 --2- 172.21.15.36:0/287288704 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe84c053a40 0x7fe84c055f00 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.083+0000 7fe8720a7700 1 --2- 172.21.15.36:0/287288704 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe86c0ad450 0x7fe86c153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.083+0000 7fe8720a7700 1 --2- 172.21.15.36:0/287288704 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe86c0ad070 0x7fe86c0aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.083+0000 7fe8720a7700 1 --2- 172.21.15.36:0/287288704 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe86c0ac950 0x7fe86c065a80 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.083+0000 7fe8720a7700 1 -- 172.21.15.36:0/287288704 >> 172.21.15.36:0/287288704 conn(0x7fe86c060990 msgr2=0x7fe86c140120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:07.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.083+0000 7fe8720a7700 1 -- 172.21.15.36:0/287288704 shutdown_connections 2022-06-17T12:56:07.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.083+0000 7fe8720a7700 1 -- 172.21.15.36:0/287288704 wait complete. 2022-06-17T12:56:07.080 INFO:teuthology.orchestra.run.smithi036.stderr: removed all blacklist entries 2022-06-17T12:56:07.087 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2022-06-17T12:56:07.088 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-06-17T12:56:07.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac8563700 1 Processor -- start 2022-06-17T12:56:07.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac8563700 1 -- start start 2022-06-17T12:56:07.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac8563700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faac014b3b0 0x7faac00af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:07.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac8563700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac00ac870 con 0x7faac00ac340 2022-06-17T12:56:07.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac8563700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac00acaa0 con 0x7faac00ad140 2022-06-17T12:56:07.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac8563700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac0148960 con 0x7faac014b3b0 2022-06-17T12:56:07.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac5afe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7faac00ac340 0x7faac0147b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56662/0 (socket says 172.21.15.36:56662) 2022-06-17T12:56:07.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac5afe700 1 -- 172.21.15.36:0/969680516 learned_addr learned my addr 172.21.15.36:0/969680516 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:07.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac6b00700 1 --2- 172.21.15.36:0/969680516 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faac014b3b0 0x7faac00af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:07.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac52fd700 1 -- 172.21.15.36:0/969680516 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903626489 0 0) 0x7faac00ac870 con 0x7faac00ac340 2022-06-17T12:56:07.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac52fd700 1 -- 172.21.15.36:0/969680516 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaa0003260 con 0x7faac00ac340 2022-06-17T12:56:07.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac6b00700 1 -- 172.21.15.36:0/969680516 >> v1:172.21.15.70:6789/0 conn(0x7faac00ad140 legacy=0x7faac014aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:07.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac6b00700 1 -- 172.21.15.36:0/969680516 >> v1:172.21.15.36:6789/0 conn(0x7faac00ac340 legacy=0x7faac0147b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:07.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac52fd700 1 -- 172.21.15.36:0/969680516 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 580666250 0 0) 0x7faac00acaa0 con 0x7faac00ad140 2022-06-17T12:56:07.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac6b00700 1 -- 172.21.15.36:0/969680516 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac00b0bf0 con 0x7faac014b3b0 2022-06-17T12:56:07.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac6b00700 1 --2- 172.21.15.36:0/969680516 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faac014b3b0 0x7faac00af7f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7faab8002f70 tx=0x7faab8006df0).ready entity=mon.2 client_cookie=1f5d75ec645c7db0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:07.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac52fd700 1 -- 172.21.15.36:0/969680516 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faab800d570 con 0x7faac014b3b0 2022-06-17T12:56:07.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac52fd700 1 -- 172.21.15.36:0/969680516 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab8000e90 con 0x7faac014b3b0 2022-06-17T12:56:07.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac52fd700 1 -- 172.21.15.36:0/969680516 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faab800d8c0 con 0x7faac014b3b0 2022-06-17T12:56:07.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac8563700 1 -- 172.21.15.36:0/969680516 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faac014b3b0 msgr2=0x7faac00af7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:07.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac8563700 1 --2- 172.21.15.36:0/969680516 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faac014b3b0 0x7faac00af7f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7faab8002f70 tx=0x7faab8006df0).stop 2022-06-17T12:56:07.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac8563700 1 -- 172.21.15.36:0/969680516 shutdown_connections 2022-06-17T12:56:07.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac8563700 1 --2- 172.21.15.36:0/969680516 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faac014b3b0 0x7faac00af7f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac8563700 1 -- 172.21.15.36:0/969680516 >> 172.21.15.36:0/969680516 conn(0x7faac00b5430 msgr2=0x7faac00b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:07.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac8563700 1 -- 172.21.15.36:0/969680516 shutdown_connections 2022-06-17T12:56:07.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.211+0000 7faac8563700 1 -- 172.21.15.36:0/969680516 wait complete. 2022-06-17T12:56:07.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faac8563700 1 Processor -- start 2022-06-17T12:56:07.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faac8563700 1 -- start start 2022-06-17T12:56:07.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faac8563700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faac00ac340 0x7faac0060a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:07.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faac8563700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac00ad140 0x7faac0143640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:07.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faac8563700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faac014b3b0 0x7faac0143b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:07.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faac8563700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac00a7ed0 con 0x7faac00ac340 2022-06-17T12:56:07.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faac8563700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac00a71f0 con 0x7faac00ad140 2022-06-17T12:56:07.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faac8563700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac00a7490 con 0x7faac014b3b0 2022-06-17T12:56:07.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faac62ff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faac00ac340 0x7faac0060a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:07.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faac62ff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faac00ac340 0x7faac0060a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39136/0 (socket says 172.21.15.36:39136) 2022-06-17T12:56:07.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faac62ff700 1 -- 172.21.15.36:0/2592475405 learned_addr learned my addr 172.21.15.36:0/2592475405 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:07.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faac6b00700 1 --2- 172.21.15.36:0/2592475405 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faac014b3b0 0x7faac0143b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:07.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faac5afe700 1 --2- 172.21.15.36:0/2592475405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac00ad140 0x7faac0143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:07.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faac62ff700 1 -- 172.21.15.36:0/2592475405 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faac014b3b0 msgr2=0x7faac0143b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:07.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faac62ff700 1 --2- 172.21.15.36:0/2592475405 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faac014b3b0 0x7faac0143b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faac62ff700 1 -- 172.21.15.36:0/2592475405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac00ad140 msgr2=0x7faac0143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:07.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faac62ff700 1 --2- 172.21.15.36:0/2592475405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac00ad140 0x7faac0143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faac62ff700 1 -- 172.21.15.36:0/2592475405 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab800b180 con 0x7faac00ac340 2022-06-17T12:56:07.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faac5afe700 1 --2- 172.21.15.36:0/2592475405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac00ad140 0x7faac0143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:56:07.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faac62ff700 1 --2- 172.21.15.36:0/2592475405 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faac00ac340 0x7faac0060a40 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0x7faaa8000ed0 tx=0x7faaa8007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:07.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faabf7fe700 1 -- 172.21.15.36:0/2592475405 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faaa8012400 con 0x7faac00ac340 2022-06-17T12:56:07.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faabf7fe700 1 -- 172.21.15.36:0/2592475405 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa8012560 con 0x7faac00ac340 2022-06-17T12:56:07.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faac8563700 1 -- 172.21.15.36:0/2592475405 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac00a7b50 con 0x7faac00ac340 2022-06-17T12:56:07.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faabf7fe700 1 -- 172.21.15.36:0/2592475405 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faaa80128c0 con 0x7faac00ac340 2022-06-17T12:56:07.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.215+0000 7faac8563700 1 -- 172.21.15.36:0/2592475405 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac00a8c50 con 0x7faac00ac340 2022-06-17T12:56:07.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.219+0000 7faabf7fe700 1 -- 172.21.15.36:0/2592475405 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7faaa8019030 con 0x7faac00ac340 2022-06-17T12:56:07.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.219+0000 7faac8563700 1 -- 172.21.15.36:0/2592475405 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac00a7650 con 0x7faac00ac340 2022-06-17T12:56:07.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.219+0000 7faabf7fe700 1 --2- 172.21.15.36:0/2592475405 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faaa4053ad0 0x7faaa4055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:07.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.219+0000 7faac5afe700 1 --2- 172.21.15.36:0/2592475405 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faaa4053ad0 0x7faaa4055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:07.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.219+0000 7faabf7fe700 1 -- 172.21.15.36:0/2592475405 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5600+0+0 (secure 0 0 0) 0x7faaa806b080 con 0x7faac00ac340 2022-06-17T12:56:07.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.219+0000 7faac5afe700 1 --2- 172.21.15.36:0/2592475405 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faaa4053ad0 0x7faaa4055f90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7faab000b000 tx=0x7faab0009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:07.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.227+0000 7faabf7fe700 1 -- 172.21.15.36:0/2592475405 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faaa8042160 con 0x7faac00ac340 2022-06-17T12:56:07.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.391+0000 7faac8563700 1 -- 172.21.15.36:0/2592475405 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7faac00a7d40 con 0x7faac00ac340 2022-06-17T12:56:07.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.391+0000 7faabf7fe700 1 -- 172.21.15.36:0/2592475405 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v40) v1 ==== 80+0+4242 (secure 0 0 0) 0x7faaa8045780 con 0x7faac00ac340 2022-06-17T12:56:07.383 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:07.384 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:56:07.384 INFO:teuthology.orchestra.run.smithi036.stdout: "auth_dump": [ 2022-06-17T12:56:07.385 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:07.386 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mds.a", 2022-06-17T12:56:07.393 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiyaUgARAAxp7bzOl4uVo2EsaoRBeMRw==", 2022-06-17T12:56:07.394 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:07.395 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:56:07.395 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:56:07.395 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:56:07.396 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:07.397 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:07.397 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:07.398 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:07.400 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mds.b", 2022-06-17T12:56:07.401 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xifyxLCRAABI389jmz1zboeDanzraT8Q==", 2022-06-17T12:56:07.401 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:07.402 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:56:07.403 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:56:07.404 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:56:07.404 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:07.405 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:07.405 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:07.405 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:07.406 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.0", 2022-06-17T12:56:07.406 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB+d6xibhXGNBAA6AuUT9ITWAZjDFpNsgA4Wg==", 2022-06-17T12:56:07.407 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:07.407 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:56:07.408 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:56:07.408 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:07.409 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:07.409 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:07.410 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:07.411 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.1", 2022-06-17T12:56:07.411 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCBd6xiOIPvGxAADkrBoJCy9x91zan6TWU0tQ==", 2022-06-17T12:56:07.412 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:07.412 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:56:07.412 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:56:07.413 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:07.413 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:07.414 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:07.414 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:07.414 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.2", 2022-06-17T12:56:07.415 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCEd6xiP4/9AhAAaP3cWXV6vbfGBPfqSLKqUg==", 2022-06-17T12:56:07.415 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:07.416 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:56:07.416 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:56:07.417 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:07.417 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:07.418 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:07.418 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:07.419 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.3", 2022-06-17T12:56:07.419 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCGd6xi+LZgJxAAJjGwP5qW6TFDZfR+2rz6ig==", 2022-06-17T12:56:07.419 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:07.420 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:56:07.421 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:56:07.421 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:07.422 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:07.422 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:07.423 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:07.423 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.4", 2022-06-17T12:56:07.424 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCKd6xikvrTChAAXIyWSETswDv+tb3zsGyYpA==", 2022-06-17T12:56:07.424 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:07.425 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:56:07.425 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:56:07.426 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:07.426 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:07.427 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:07.427 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:07.428 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.5", 2022-06-17T12:56:07.432 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCMd6xiqr2QLhAArhimHuNsVWrDGGreFYxj6A==", 2022-06-17T12:56:07.433 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:07.433 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:56:07.434 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:56:07.434 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:07.435 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:07.435 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:07.436 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:07.437 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.6", 2022-06-17T12:56:07.438 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCPd6xiUXcXGxAAdrYz8txdBWMIqF5DF4Aimw==", 2022-06-17T12:56:07.438 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:07.439 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:56:07.439 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:56:07.440 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:07.440 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:07.441 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:07.441 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:07.442 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.7", 2022-06-17T12:56:07.442 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCSd6xiBJ2IAhAAUkNfgmlxxWY7+VouPLAvyw==", 2022-06-17T12:56:07.442 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:07.443 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:56:07.444 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:56:07.444 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:07.445 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:07.445 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:07.445 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:07.445 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.0", 2022-06-17T12:56:07.446 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiXO3QFBAAsSLtQ0yKKZxhqUT/eYKdtg==", 2022-06-17T12:56:07.446 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:07.446 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:56:07.446 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:56:07.446 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:56:07.447 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:07.447 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:07.447 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:07.447 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.1", 2022-06-17T12:56:07.448 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xi51KdGBAAi7fGZ+Qepg6j3DAhlzoWxw==", 2022-06-17T12:56:07.448 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:07.448 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:56:07.448 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:56:07.449 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:56:07.449 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:07.449 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:07.449 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:07.450 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.2", 2022-06-17T12:56:07.457 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiug2hHBAAqHpLgwMdthSWtJ8Vwu+m1Q==", 2022-06-17T12:56:07.457 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:07.459 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:56:07.459 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:56:07.459 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:56:07.460 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:07.460 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:07.461 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:07.461 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.admin", 2022-06-17T12:56:07.462 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB5d6xiGDODKhAAPZMG/M2nVssdr4FJHcS6XQ==", 2022-06-17T12:56:07.462 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:07.463 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:56:07.463 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:56:07.464 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:56:07.464 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:07.464 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:07.465 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:07.465 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:07.466 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mgr.x", 2022-06-17T12:56:07.467 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB6d6xiyH9uORAATJb0chHE1CH0X02w7edVtw==", 2022-06-17T12:56:07.468 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:07.469 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:56:07.470 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile mgr", 2022-06-17T12:56:07.471 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:07.471 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:07.472 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:07.472 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:07.473 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mgr.y", 2022-06-17T12:56:07.473 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB6d6xiHXghMxAAzpsVRCbLWHWZPtrPk1jFlA==", 2022-06-17T12:56:07.473 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:07.473 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:56:07.474 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile mgr", 2022-06-17T12:56:07.474 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:07.474 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:07.474 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:07.475 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:07.475 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:56:07.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.395+0000 7faac8563700 1 -- 172.21.15.36:0/2592475405 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faaa4053ad0 msgr2=0x7faaa4055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:07.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.395+0000 7faac8563700 1 --2- 172.21.15.36:0/2592475405 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faaa4053ad0 0x7faaa4055f90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7faab000b000 tx=0x7faab0009040).stop 2022-06-17T12:56:07.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.395+0000 7faac8563700 1 -- 172.21.15.36:0/2592475405 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faac00ac340 msgr2=0x7faac0060a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:07.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.395+0000 7faac8563700 1 --2- 172.21.15.36:0/2592475405 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faac00ac340 0x7faac0060a40 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0x7faaa8000ed0 tx=0x7faaa8007ac0).stop 2022-06-17T12:56:07.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.395+0000 7faac8563700 1 -- 172.21.15.36:0/2592475405 shutdown_connections 2022-06-17T12:56:07.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.395+0000 7faac8563700 1 --2- 172.21.15.36:0/2592475405 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faaa4053ad0 0x7faaa4055f90 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.395+0000 7faac8563700 1 --2- 172.21.15.36:0/2592475405 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faac014b3b0 0x7faac0143b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.395+0000 7faac8563700 1 --2- 172.21.15.36:0/2592475405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac00ad140 0x7faac0143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.395+0000 7faac8563700 1 --2- 172.21.15.36:0/2592475405 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faac00ac340 0x7faac0060a40 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.395+0000 7faac8563700 1 -- 172.21.15.36:0/2592475405 >> 172.21.15.36:0/2592475405 conn(0x7faac00b5430 msgr2=0x7faac0144da0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:07.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.395+0000 7faac8563700 1 -- 172.21.15.36:0/2592475405 shutdown_connections 2022-06-17T12:56:07.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.395+0000 7faac8563700 1 -- 172.21.15.36:0/2592475405 wait complete. 2022-06-17T12:56:07.480 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:56:07.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.603+0000 7f5e808b0700 1 Processor -- start 2022-06-17T12:56:07.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.603+0000 7f5e808b0700 1 -- start start 2022-06-17T12:56:07.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e808b0700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5e7c0b2200 0x7f5e7c0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:07.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e808b0700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5e7c0af8d0 0x7f5e7c14a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:07.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e808b0700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e7c0b25e0 con 0x7f5e7c0b2cd0 2022-06-17T12:56:07.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e808b0700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e7c0ad790 con 0x7f5e7c0b2200 2022-06-17T12:56:07.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e808b0700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e7c0aef40 con 0x7f5e7c0af8d0 2022-06-17T12:56:07.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e7b7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5e7c0b2200 0x7f5e7c0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:07.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e7b7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5e7c0b2200 0x7f5e7c0af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39144/0 (socket says 172.21.15.36:39144) 2022-06-17T12:56:07.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e7b7fe700 1 -- 172.21.15.36:0/3776328316 learned_addr learned my addr 172.21.15.36:0/3776328316 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:07.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e7affd700 1 --2- 172.21.15.36:0/3776328316 >> v2:172.21.15.70:3300/0 conn(0x7f5e7c0af8d0 0x7f5e7c14a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:07.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e7b7fe700 1 -- 172.21.15.36:0/3776328316 >> v2:172.21.15.70:3300/0 conn(0x7f5e7c0af8d0 msgr2=0x7f5e7c14a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:07.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e7b7fe700 1 --2- 172.21.15.36:0/3776328316 >> v2:172.21.15.70:3300/0 conn(0x7f5e7c0af8d0 0x7f5e7c14a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e7b7fe700 1 -- 172.21.15.36:0/3776328316 >> v1:172.21.15.70:6789/0 conn(0x7f5e7c0b2cd0 legacy=0x7f5e7c14adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:07.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e7b7fe700 1 -- 172.21.15.36:0/3776328316 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7c0b1950 con 0x7f5e7c0b2200 2022-06-17T12:56:07.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e7b7fe700 1 --2- 172.21.15.36:0/3776328316 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e7c0b2200 0x7f5e7c0af3e0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 rx=0x7f5e740032f0 tx=0x7f5e74009290).ready entity=mon.0 client_cookie=1b84a3d1542becf0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:07.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e7a7fc700 1 -- 172.21.15.36:0/3776328316 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e7400b870 con 0x7f5e7c0b2200 2022-06-17T12:56:07.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e7a7fc700 1 -- 172.21.15.36:0/3776328316 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7400b9d0 con 0x7f5e7c0b2200 2022-06-17T12:56:07.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e7a7fc700 1 -- 172.21.15.36:0/3776328316 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e7400bb40 con 0x7f5e7c0b2200 2022-06-17T12:56:07.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e808b0700 1 -- 172.21.15.36:0/3776328316 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e7c0b2200 msgr2=0x7f5e7c0af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:07.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e808b0700 1 --2- 172.21.15.36:0/3776328316 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e7c0b2200 0x7f5e7c0af3e0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 rx=0x7f5e740032f0 tx=0x7f5e74009290).stop 2022-06-17T12:56:07.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e808b0700 1 -- 172.21.15.36:0/3776328316 shutdown_connections 2022-06-17T12:56:07.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e808b0700 1 --2- 172.21.15.36:0/3776328316 >> v2:172.21.15.70:3300/0 conn(0x7f5e7c0af8d0 0x7f5e7c14a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e808b0700 1 --2- 172.21.15.36:0/3776328316 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e7c0b2200 0x7f5e7c0af3e0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e808b0700 1 -- 172.21.15.36:0/3776328316 >> 172.21.15.36:0/3776328316 conn(0x7f5e7c05f770 msgr2=0x7f5e7c05fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:07.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e808b0700 1 -- 172.21.15.36:0/3776328316 shutdown_connections 2022-06-17T12:56:07.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.607+0000 7f5e808b0700 1 -- 172.21.15.36:0/3776328316 wait complete. 2022-06-17T12:56:07.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.611+0000 7f5e808b0700 1 Processor -- start 2022-06-17T12:56:07.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.611+0000 7f5e808b0700 1 -- start start 2022-06-17T12:56:07.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.611+0000 7f5e808b0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e7c0af8d0 0x7f5e7c14e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:07.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.611+0000 7f5e808b0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e7c0b2200 0x7f5e7c0a8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:07.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.611+0000 7f5e808b0700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5e7c0b2cd0 0x7f5e7c062f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:07.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.611+0000 7f5e808b0700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c062860 con 0x7f5e7c0b2200 2022-06-17T12:56:07.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.611+0000 7f5e808b0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c0a94a0 con 0x7f5e7c0af8d0 2022-06-17T12:56:07.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.611+0000 7f5e808b0700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c062a90 con 0x7f5e7c0b2cd0 2022-06-17T12:56:07.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.611+0000 7f5e7affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e7c0b2200 0x7f5e7c0a8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:07.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.611+0000 7f5e7affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e7c0b2200 0x7f5e7c0a8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39152/0 (socket says 172.21.15.36:39152) 2022-06-17T12:56:07.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.611+0000 7f5e7b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e7c0af8d0 0x7f5e7c14e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:07.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.611+0000 7f5e7affd700 1 -- 172.21.15.36:0/1939147592 learned_addr learned my addr 172.21.15.36:0/1939147592 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:07.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.611+0000 7f5e7bfff700 1 --2- 172.21.15.36:0/1939147592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5e7c0b2cd0 0x7f5e7c062f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:07.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.611+0000 7f5e7affd700 1 -- 172.21.15.36:0/1939147592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5e7c0b2cd0 msgr2=0x7f5e7c062f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:07.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.611+0000 7f5e7affd700 1 --2- 172.21.15.36:0/1939147592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5e7c0b2cd0 0x7f5e7c062f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.611+0000 7f5e7affd700 1 -- 172.21.15.36:0/1939147592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e7c0af8d0 msgr2=0x7f5e7c14e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:07.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.611+0000 7f5e7affd700 1 --2- 172.21.15.36:0/1939147592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e7c0af8d0 0x7f5e7c14e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.611+0000 7f5e7affd700 1 -- 172.21.15.36:0/1939147592 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e74009530 con 0x7f5e7c0b2200 2022-06-17T12:56:07.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.611+0000 7f5e7bfff700 1 --2- 172.21.15.36:0/1939147592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5e7c0b2cd0 0x7f5e7c062f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:56:07.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.611+0000 7f5e7affd700 1 --2- 172.21.15.36:0/1939147592 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e7c0b2200 0x7f5e7c0a8c40 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f5e700030b0 tx=0x7f5e70008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:07.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.615+0000 7f5e78ff9700 1 -- 172.21.15.36:0/1939147592 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e70011780 con 0x7f5e7c0b2200 2022-06-17T12:56:07.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.615+0000 7f5e78ff9700 1 -- 172.21.15.36:0/1939147592 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e700118e0 con 0x7f5e7c0b2200 2022-06-17T12:56:07.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.615+0000 7f5e808b0700 1 -- 172.21.15.36:0/1939147592 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7c14e860 con 0x7f5e7c0b2200 2022-06-17T12:56:07.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.615+0000 7f5e78ff9700 1 -- 172.21.15.36:0/1939147592 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e70011c40 con 0x7f5e7c0b2200 2022-06-17T12:56:07.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.615+0000 7f5e808b0700 1 -- 172.21.15.36:0/1939147592 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7c0a9270 con 0x7f5e7c0b2200 2022-06-17T12:56:07.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.615+0000 7f5e78ff9700 1 -- 172.21.15.36:0/1939147592 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f5e70012420 con 0x7f5e7c0b2200 2022-06-17T12:56:07.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.615+0000 7f5e808b0700 1 -- 172.21.15.36:0/1939147592 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e7c062c50 con 0x7f5e7c0b2200 2022-06-17T12:56:07.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.615+0000 7f5e78ff9700 1 --2- 172.21.15.36:0/1939147592 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5e5c053a40 0x7f5e5c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:07.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.615+0000 7f5e78ff9700 1 -- 172.21.15.36:0/1939147592 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f5e7006bbd0 con 0x7f5e7c0b2200 2022-06-17T12:56:07.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.615+0000 7f5e7b7fe700 1 --2- 172.21.15.36:0/1939147592 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5e5c053a40 0x7f5e5c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:07.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.619+0000 7f5e7b7fe700 1 --2- 172.21.15.36:0/1939147592 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5e5c053a40 0x7f5e5c055f00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f5e7401b000 tx=0x7f5e74016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:07.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.619+0000 7f5e78ff9700 1 -- 172.21.15.36:0/1939147592 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5e70042ba0 con 0x7f5e7c0b2200 2022-06-17T12:56:07.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.743+0000 7f5e808b0700 1 -- 172.21.15.36:0/1939147592 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e7c0ad330 con 0x7f5e7c0b2200 2022-06-17T12:56:07.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.747+0000 7f5e78ff9700 1 -- 172.21.15.36:0/1939147592 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+13424 (secure 0 0 0) 0x7f5e700461c0 con 0x7f5e7c0b2200 2022-06-17T12:56:07.751 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:07.751 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":127,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:56:07.072378+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:56:07.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.747+0000 7f5e808b0700 1 -- 172.21.15.36:0/1939147592 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5e5c053a40 msgr2=0x7f5e5c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:07.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.747+0000 7f5e808b0700 1 --2- 172.21.15.36:0/1939147592 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5e5c053a40 0x7f5e5c055f00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f5e7401b000 tx=0x7f5e74016000).stop 2022-06-17T12:56:07.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.747+0000 7f5e808b0700 1 -- 172.21.15.36:0/1939147592 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e7c0b2200 msgr2=0x7f5e7c0a8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:07.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.747+0000 7f5e808b0700 1 --2- 172.21.15.36:0/1939147592 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e7c0b2200 0x7f5e7c0a8c40 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f5e700030b0 tx=0x7f5e70008040).stop 2022-06-17T12:56:07.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.747+0000 7f5e808b0700 1 -- 172.21.15.36:0/1939147592 shutdown_connections 2022-06-17T12:56:07.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.747+0000 7f5e808b0700 1 --2- 172.21.15.36:0/1939147592 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5e5c053a40 0x7f5e5c055f00 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.747+0000 7f5e808b0700 1 --2- 172.21.15.36:0/1939147592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5e7c0b2cd0 0x7f5e7c062f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.747+0000 7f5e808b0700 1 --2- 172.21.15.36:0/1939147592 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5e7c0b2200 0x7f5e7c0a8c40 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.747+0000 7f5e808b0700 1 --2- 172.21.15.36:0/1939147592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e7c0af8d0 0x7f5e7c14e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.747+0000 7f5e808b0700 1 -- 172.21.15.36:0/1939147592 >> 172.21.15.36:0/1939147592 conn(0x7f5e7c05f770 msgr2=0x7f5e7c0b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:07.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.747+0000 7f5e808b0700 1 -- 172.21.15.36:0/1939147592 shutdown_connections 2022-06-17T12:56:07.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.747+0000 7f5e808b0700 1 -- 172.21.15.36:0/1939147592 wait complete. 2022-06-17T12:56:07.760 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-17T12:56:07.761 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:56:07.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.915+0000 7f9ca3fff700 1 Processor -- start 2022-06-17T12:56:07.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca3fff700 1 -- start start 2022-06-17T12:56:07.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca3fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ca414b2b0 0x7f9ca4060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:07.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca3fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9ca4147bd0 0x7f9ca41442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:07.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca3fff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca414b870 con 0x7f9ca40b27b0 2022-06-17T12:56:07.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca3fff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca40ac500 con 0x7f9ca4147bd0 2022-06-17T12:56:07.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca3fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca40ace90 con 0x7f9ca414b2b0 2022-06-17T12:56:07.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca37fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9ca4147bd0 0x7f9ca41442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:07.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca37fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9ca4147bd0 0x7f9ca41442d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39162/0 (socket says 172.21.15.36:39162) 2022-06-17T12:56:07.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca37fe700 1 -- 172.21.15.36:0/3943130821 learned_addr learned my addr 172.21.15.36:0/3943130821 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:07.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca27fc700 1 --2- 172.21.15.36:0/3943130821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ca414b2b0 0x7f9ca4060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:07.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca1ffb700 1 -- 172.21.15.36:0/3943130821 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1990673411 0 0) 0x7f9ca414b870 con 0x7f9ca40b27b0 2022-06-17T12:56:07.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca37fe700 1 -- 172.21.15.36:0/3943130821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ca414b2b0 msgr2=0x7f9ca4060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:07.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca37fe700 1 --2- 172.21.15.36:0/3943130821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ca414b2b0 0x7f9ca4060ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca37fe700 1 -- 172.21.15.36:0/3943130821 >> v1:172.21.15.70:6789/0 conn(0x7f9ca40b27b0 legacy=0x7f9ca40ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:07.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca37fe700 1 -- 172.21.15.36:0/3943130821 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca40b2b90 con 0x7f9ca4147bd0 2022-06-17T12:56:07.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca37fe700 1 --2- 172.21.15.36:0/3943130821 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9ca4147bd0 0x7f9ca41442d0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7f9c94002f70 tx=0x7f9c94006eb0).ready entity=mon.0 client_cookie=ff9400ce1d35bf7d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:07.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca1ffb700 1 -- 172.21.15.36:0/3943130821 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c9400d570 con 0x7f9ca4147bd0 2022-06-17T12:56:07.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca1ffb700 1 -- 172.21.15.36:0/3943130821 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c94000e90 con 0x7f9ca4147bd0 2022-06-17T12:56:07.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca1ffb700 1 -- 172.21.15.36:0/3943130821 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c9400d900 con 0x7f9ca4147bd0 2022-06-17T12:56:07.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca3fff700 1 -- 172.21.15.36:0/3943130821 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9ca4147bd0 msgr2=0x7f9ca41442d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:07.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca3fff700 1 --2- 172.21.15.36:0/3943130821 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9ca4147bd0 0x7f9ca41442d0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7f9c94002f70 tx=0x7f9c94006eb0).stop 2022-06-17T12:56:07.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca3fff700 1 -- 172.21.15.36:0/3943130821 shutdown_connections 2022-06-17T12:56:07.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca3fff700 1 --2- 172.21.15.36:0/3943130821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ca414b2b0 0x7f9ca4060ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca3fff700 1 --2- 172.21.15.36:0/3943130821 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9ca4147bd0 0x7f9ca41442d0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca3fff700 1 -- 172.21.15.36:0/3943130821 >> 172.21.15.36:0/3943130821 conn(0x7f9ca40b5d10 msgr2=0x7f9ca40b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:07.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca3fff700 1 -- 172.21.15.36:0/3943130821 shutdown_connections 2022-06-17T12:56:07.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.919+0000 7f9ca3fff700 1 -- 172.21.15.36:0/3943130821 wait complete. 2022-06-17T12:56:07.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9ca3fff700 1 Processor -- start 2022-06-17T12:56:07.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9ca3fff700 1 -- start start 2022-06-17T12:56:07.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9ca3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ca40b27b0 0x7f9ca40af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:07.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9ca3fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ca4147bd0 0x7f9ca40a70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:07.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9ca3fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9ca414b2b0 0x7f9ca4064c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:07.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9ca3fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca4067270 con 0x7f9ca414b2b0 2022-06-17T12:56:07.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9ca3fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca4064810 con 0x7f9ca40b27b0 2022-06-17T12:56:07.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9ca3fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca4062d20 con 0x7f9ca4147bd0 2022-06-17T12:56:07.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9ca37fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9ca414b2b0 0x7f9ca4064c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:07.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9ca37fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9ca414b2b0 0x7f9ca4064c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39168/0 (socket says 172.21.15.36:39168) 2022-06-17T12:56:07.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9ca37fe700 1 -- 172.21.15.36:0/274619760 learned_addr learned my addr 172.21.15.36:0/274619760 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:07.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9ca27fc700 1 --2- 172.21.15.36:0/274619760 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ca4147bd0 0x7f9ca40a70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:07.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9ca2ffd700 1 --2- 172.21.15.36:0/274619760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ca40b27b0 0x7f9ca40af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:07.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9ca37fe700 1 -- 172.21.15.36:0/274619760 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ca4147bd0 msgr2=0x7f9ca40a70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:07.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9ca37fe700 1 --2- 172.21.15.36:0/274619760 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ca4147bd0 0x7f9ca40a70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9ca37fe700 1 -- 172.21.15.36:0/274619760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ca40b27b0 msgr2=0x7f9ca40af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:07.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9ca37fe700 1 --2- 172.21.15.36:0/274619760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ca40b27b0 0x7f9ca40af160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:07.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9ca37fe700 1 -- 172.21.15.36:0/274619760 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9400b1e0 con 0x7f9ca414b2b0 2022-06-17T12:56:07.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9ca2ffd700 1 --2- 172.21.15.36:0/274619760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ca40b27b0 0x7f9ca40af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:56:07.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9ca37fe700 1 --2- 172.21.15.36:0/274619760 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9ca414b2b0 0x7f9ca4064c00 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 rx=0x7f9c94002f70 tx=0x7f9c94015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:07.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9c9bfff700 1 -- 172.21.15.36:0/274619760 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c94011860 con 0x7f9ca414b2b0 2022-06-17T12:56:07.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9c9bfff700 1 -- 172.21.15.36:0/274619760 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c940119c0 con 0x7f9ca414b2b0 2022-06-17T12:56:07.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9ca3fff700 1 -- 172.21.15.36:0/274619760 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca4062fc0 con 0x7f9ca414b2b0 2022-06-17T12:56:07.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9c9bfff700 1 -- 172.21.15.36:0/274619760 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c94011d20 con 0x7f9ca414b2b0 2022-06-17T12:56:07.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.923+0000 7f9ca3fff700 1 -- 172.21.15.36:0/274619760 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca40636b0 con 0x7f9ca414b2b0 2022-06-17T12:56:07.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.927+0000 7f9ca3fff700 1 -- 172.21.15.36:0/274619760 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca4057410 con 0x7f9ca414b2b0 2022-06-17T12:56:07.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.927+0000 7f9c9bfff700 1 -- 172.21.15.36:0/274619760 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f9c94025030 con 0x7f9ca414b2b0 2022-06-17T12:56:07.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.927+0000 7f9c9bfff700 1 --2- 172.21.15.36:0/274619760 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9c88053920 0x7f9c88055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:07.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.927+0000 7f9ca2ffd700 1 --2- 172.21.15.36:0/274619760 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9c88053920 0x7f9c88055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:07.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.927+0000 7f9c9bfff700 1 -- 172.21.15.36:0/274619760 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f9c9406d0b0 con 0x7f9ca414b2b0 2022-06-17T12:56:07.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.927+0000 7f9ca2ffd700 1 --2- 172.21.15.36:0/274619760 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9c88053920 0x7f9c88055de0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f9c9c00b000 tx=0x7f9c9c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:07.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:07.931+0000 7f9c9bfff700 1 -- 172.21.15.36:0/274619760 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9c940440e0 con 0x7f9ca414b2b0 2022-06-17T12:56:08.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.055+0000 7f9ca3fff700 1 -- 172.21.15.36:0/274619760 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9ca405d9b0 con 0x7f9ca414b2b0 2022-06-17T12:56:08.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.055+0000 7f9c9bfff700 1 -- 172.21.15.36:0/274619760 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v127) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9c94023020 con 0x7f9ca414b2b0 2022-06-17T12:56:08.047 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:56:08.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.059+0000 7f9ca3fff700 1 -- 172.21.15.36:0/274619760 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9c88053920 msgr2=0x7f9c88055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:08.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.059+0000 7f9ca3fff700 1 --2- 172.21.15.36:0/274619760 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9c88053920 0x7f9c88055de0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f9c9c00b000 tx=0x7f9c9c009040).stop 2022-06-17T12:56:08.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.059+0000 7f9ca3fff700 1 -- 172.21.15.36:0/274619760 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9ca414b2b0 msgr2=0x7f9ca4064c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:08.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.059+0000 7f9ca3fff700 1 --2- 172.21.15.36:0/274619760 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9ca414b2b0 0x7f9ca4064c00 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 rx=0x7f9c94002f70 tx=0x7f9c94015000).stop 2022-06-17T12:56:08.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.059+0000 7f9ca3fff700 1 -- 172.21.15.36:0/274619760 shutdown_connections 2022-06-17T12:56:08.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.059+0000 7f9ca3fff700 1 --2- 172.21.15.36:0/274619760 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9c88053920 0x7f9c88055de0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:08.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.059+0000 7f9ca3fff700 1 --2- 172.21.15.36:0/274619760 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9ca414b2b0 0x7f9ca4064c00 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:08.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.059+0000 7f9ca3fff700 1 --2- 172.21.15.36:0/274619760 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ca4147bd0 0x7f9ca40a70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:08.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.059+0000 7f9ca3fff700 1 --2- 172.21.15.36:0/274619760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ca40b27b0 0x7f9ca40af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:08.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.059+0000 7f9ca3fff700 1 -- 172.21.15.36:0/274619760 >> 172.21.15.36:0/274619760 conn(0x7f9ca40b5d10 msgr2=0x7f9ca4146710 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:08.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.059+0000 7f9ca3fff700 1 -- 172.21.15.36:0/274619760 shutdown_connections 2022-06-17T12:56:08.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.059+0000 7f9ca3fff700 1 -- 172.21.15.36:0/274619760 wait complete. 2022-06-17T12:56:08.060 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:56:08.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.191+0000 7fc84896d700 1 Processor -- start 2022-06-17T12:56:08.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.191+0000 7fc84896d700 1 -- start start 2022-06-17T12:56:08.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.191+0000 7fc84896d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc8440b9860 0x7fc844143840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:08.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.191+0000 7fc84896d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8440b45c0 0x7fc8440b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:08.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.191+0000 7fc84896d700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8440b4160 con 0x7fc8440b4f00 2022-06-17T12:56:08.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.191+0000 7fc84896d700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8440b9c40 con 0x7fc8440b9860 2022-06-17T12:56:08.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.191+0000 7fc84896d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8440627d0 con 0x7fc8440b45c0 2022-06-17T12:56:08.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.191+0000 7fc842ffd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8440b45c0 0x7fc8440b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:08.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.191+0000 7fc842ffd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8440b45c0 0x7fc8440b4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:53964/0 (socket says 172.21.15.36:53964) 2022-06-17T12:56:08.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.191+0000 7fc842ffd700 1 -- 172.21.15.36:0/2617543348 learned_addr learned my addr 172.21.15.36:0/2617543348 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:08.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.191+0000 7fc8437fe700 1 --2- 172.21.15.36:0/2617543348 >> v2:172.21.15.70:3300/0 conn(0x7fc8440b9860 0x7fc844143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:08.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.191+0000 7fc842ffd700 1 -- 172.21.15.36:0/2617543348 >> v2:172.21.15.70:3300/0 conn(0x7fc8440b9860 msgr2=0x7fc844143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:08.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.191+0000 7fc842ffd700 1 --2- 172.21.15.36:0/2617543348 >> v2:172.21.15.70:3300/0 conn(0x7fc8440b9860 0x7fc844143840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:08.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.191+0000 7fc842ffd700 1 -- 172.21.15.36:0/2617543348 >> v1:172.21.15.70:6789/0 conn(0x7fc8440b4f00 legacy=0x7fc844146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:08.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.191+0000 7fc842ffd700 1 -- 172.21.15.36:0/2617543348 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc844062a70 con 0x7fc8440b45c0 2022-06-17T12:56:08.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc842ffd700 1 --2- 172.21.15.36:0/2617543348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8440b45c0 0x7fc8440b4a10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fc838002f70 tx=0x7fc838006eb0).ready entity=mon.2 client_cookie=adb2f6ceca0a0433 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:08.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc8427fc700 1 -- 172.21.15.36:0/2617543348 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc83800d570 con 0x7fc8440b45c0 2022-06-17T12:56:08.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc8427fc700 1 -- 172.21.15.36:0/2617543348 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc838000e90 con 0x7fc8440b45c0 2022-06-17T12:56:08.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc8427fc700 1 -- 172.21.15.36:0/2617543348 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc83800d900 con 0x7fc8440b45c0 2022-06-17T12:56:08.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc84896d700 1 -- 172.21.15.36:0/2617543348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8440b45c0 msgr2=0x7fc8440b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:08.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc84896d700 1 --2- 172.21.15.36:0/2617543348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8440b45c0 0x7fc8440b4a10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fc838002f70 tx=0x7fc838006eb0).stop 2022-06-17T12:56:08.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc84896d700 1 -- 172.21.15.36:0/2617543348 shutdown_connections 2022-06-17T12:56:08.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc84896d700 1 --2- 172.21.15.36:0/2617543348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8440b45c0 0x7fc8440b4a10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:08.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc84896d700 1 --2- 172.21.15.36:0/2617543348 >> v2:172.21.15.70:3300/0 conn(0x7fc8440b9860 0x7fc844143840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:08.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc84896d700 1 -- 172.21.15.36:0/2617543348 >> 172.21.15.36:0/2617543348 conn(0x7fc84405c550 msgr2=0x7fc844060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:08.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc84896d700 1 -- 172.21.15.36:0/2617543348 shutdown_connections 2022-06-17T12:56:08.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc84896d700 1 -- 172.21.15.36:0/2617543348 wait complete. 2022-06-17T12:56:08.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc84896d700 1 Processor -- start 2022-06-17T12:56:08.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc84896d700 1 -- start start 2022-06-17T12:56:08.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc84896d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8440b45c0 0x7fc84412ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:08.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc84896d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8440b4f00 0x7fc84412bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:08.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc84896d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8440b9860 0x7fc84412c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:08.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc84896d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc844152b20 con 0x7fc8440b45c0 2022-06-17T12:56:08.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc84896d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc844152d90 con 0x7fc8440b4f00 2022-06-17T12:56:08.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc84896d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc844153030 con 0x7fc8440b9860 2022-06-17T12:56:08.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc8437fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8440b45c0 0x7fc84412ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:08.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc8437fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8440b45c0 0x7fc84412ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39178/0 (socket says 172.21.15.36:39178) 2022-06-17T12:56:08.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc843fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8440b9860 0x7fc84412c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:08.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc8437fe700 1 -- 172.21.15.36:0/1269570047 learned_addr learned my addr 172.21.15.36:0/1269570047 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:08.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc842ffd700 1 --2- 172.21.15.36:0/1269570047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8440b4f00 0x7fc84412bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:08.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc8437fe700 1 -- 172.21.15.36:0/1269570047 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8440b9860 msgr2=0x7fc84412c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:08.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc8437fe700 1 --2- 172.21.15.36:0/1269570047 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8440b9860 0x7fc84412c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:08.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc8437fe700 1 -- 172.21.15.36:0/1269570047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8440b4f00 msgr2=0x7fc84412bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:08.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc8437fe700 1 --2- 172.21.15.36:0/1269570047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8440b4f00 0x7fc84412bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:08.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.195+0000 7fc8437fe700 1 -- 172.21.15.36:0/1269570047 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83800b1e0 con 0x7fc8440b45c0 2022-06-17T12:56:08.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.199+0000 7fc8437fe700 1 --2- 172.21.15.36:0/1269570047 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8440b45c0 0x7fc84412ce90 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0x7fc83400c000 tx=0x7fc834006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:08.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.199+0000 7fc840ff9700 1 -- 172.21.15.36:0/1269570047 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc834011750 con 0x7fc8440b45c0 2022-06-17T12:56:08.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.199+0000 7fc84896d700 1 -- 172.21.15.36:0/1269570047 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84412c700 con 0x7fc8440b45c0 2022-06-17T12:56:08.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.199+0000 7fc840ff9700 1 -- 172.21.15.36:0/1269570047 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8340118b0 con 0x7fc8440b45c0 2022-06-17T12:56:08.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.199+0000 7fc840ff9700 1 -- 172.21.15.36:0/1269570047 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc834011c10 con 0x7fc8440b45c0 2022-06-17T12:56:08.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.199+0000 7fc84896d700 1 -- 172.21.15.36:0/1269570047 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8440b0010 con 0x7fc8440b45c0 2022-06-17T12:56:08.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.199+0000 7fc84896d700 1 -- 172.21.15.36:0/1269570047 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8440ad630 con 0x7fc8440b45c0 2022-06-17T12:56:08.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.199+0000 7fc840ff9700 1 -- 172.21.15.36:0/1269570047 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fc834011d70 con 0x7fc8440b45c0 2022-06-17T12:56:08.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.199+0000 7fc840ff9700 1 --2- 172.21.15.36:0/1269570047 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc824053a40 0x7fc824055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:08.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.199+0000 7fc840ff9700 1 -- 172.21.15.36:0/1269570047 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fc83406c110 con 0x7fc8440b45c0 2022-06-17T12:56:08.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.199+0000 7fc842ffd700 1 --2- 172.21.15.36:0/1269570047 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc824053a40 0x7fc824055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:08.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.199+0000 7fc842ffd700 1 --2- 172.21.15.36:0/1269570047 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc824053a40 0x7fc824055f00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fc83801c000 tx=0x7fc838015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:08.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.207+0000 7fc840ff9700 1 -- 172.21.15.36:0/1269570047 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc8440ad630 con 0x7fc8440b45c0 2022-06-17T12:56:08.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.331+0000 7fc84896d700 1 -- 172.21.15.36:0/1269570047 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc8440ad630 con 0x7fc8440b45c0 2022-06-17T12:56:08.321 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.331+0000 7fc840ff9700 1 -- 172.21.15.36:0/1269570047 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v127) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc8440ad630 con 0x7fc8440b45c0 2022-06-17T12:56:08.321 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:56:08.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.335+0000 7fc84896d700 1 -- 172.21.15.36:0/1269570047 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc824053a40 msgr2=0x7fc824055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:08.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.335+0000 7fc84896d700 1 --2- 172.21.15.36:0/1269570047 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc824053a40 0x7fc824055f00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fc83801c000 tx=0x7fc838015000).stop 2022-06-17T12:56:08.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.335+0000 7fc84896d700 1 -- 172.21.15.36:0/1269570047 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8440b45c0 msgr2=0x7fc84412ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:08.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.335+0000 7fc84896d700 1 --2- 172.21.15.36:0/1269570047 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8440b45c0 0x7fc84412ce90 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0x7fc83400c000 tx=0x7fc834006df0).stop 2022-06-17T12:56:08.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.335+0000 7fc84896d700 1 -- 172.21.15.36:0/1269570047 shutdown_connections 2022-06-17T12:56:08.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.335+0000 7fc84896d700 1 --2- 172.21.15.36:0/1269570047 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc824053a40 0x7fc824055f00 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:08.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.335+0000 7fc84896d700 1 --2- 172.21.15.36:0/1269570047 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8440b9860 0x7fc84412c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:08.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.335+0000 7fc84896d700 1 --2- 172.21.15.36:0/1269570047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8440b4f00 0x7fc84412bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:08.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.335+0000 7fc84896d700 1 --2- 172.21.15.36:0/1269570047 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8440b45c0 0x7fc84412ce90 secure :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 rx=0x7fc83400c000 tx=0x7fc834006df0).stop 2022-06-17T12:56:08.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.335+0000 7fc84896d700 1 -- 172.21.15.36:0/1269570047 >> 172.21.15.36:0/1269570047 conn(0x7fc84405c550 msgr2=0x7fc844146b90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:08.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.335+0000 7fc84896d700 1 -- 172.21.15.36:0/1269570047 shutdown_connections 2022-06-17T12:56:08.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.335+0000 7fc84896d700 1 -- 172.21.15.36:0/1269570047 wait complete. 2022-06-17T12:56:08.333 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-17T12:56:08.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.451+0000 7ff0668a8700 1 Processor -- start 2022-06-17T12:56:08.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.451+0000 7ff0668a8700 1 -- start start 2022-06-17T12:56:08.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.451+0000 7ff0668a8700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff0600aef60 0x7ff0600b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:08.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.451+0000 7ff0668a8700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff0600ba2e0 0x7ff0601442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:08.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.451+0000 7ff0668a8700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0600af4b0 con 0x7ff06013cb70 2022-06-17T12:56:08.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.451+0000 7ff0668a8700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff060143da0 con 0x7ff0600aef60 2022-06-17T12:56:08.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.451+0000 7ff0668a8700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff060142f50 con 0x7ff0600ba2e0 2022-06-17T12:56:08.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.451+0000 7ff0650a5700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff0600aef60 0x7ff0600b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:08.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.451+0000 7ff0650a5700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff0600aef60 0x7ff0600b9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39188/0 (socket says 172.21.15.36:39188) 2022-06-17T12:56:08.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.451+0000 7ff0650a5700 1 -- 172.21.15.36:0/1814394371 learned_addr learned my addr 172.21.15.36:0/1814394371 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:08.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.451+0000 7ff0660a7700 1 --2- 172.21.15.36:0/1814394371 >> v2:172.21.15.70:3300/0 conn(0x7ff0600ba2e0 0x7ff0601442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:08.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.451+0000 7ff0650a5700 1 -- 172.21.15.36:0/1814394371 >> v2:172.21.15.70:3300/0 conn(0x7ff0600ba2e0 msgr2=0x7ff0601442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:08.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.451+0000 7ff0650a5700 1 --2- 172.21.15.36:0/1814394371 >> v2:172.21.15.70:3300/0 conn(0x7ff0600ba2e0 0x7ff0601442a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:08.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.451+0000 7ff0650a5700 1 -- 172.21.15.36:0/1814394371 >> v1:172.21.15.70:6789/0 conn(0x7ff06013cb70 legacy=0x7ff060143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:08.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.451+0000 7ff0650a5700 1 -- 172.21.15.36:0/1814394371 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0601431f0 con 0x7ff0600aef60 2022-06-17T12:56:08.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.451+0000 7ff0650a5700 1 --2- 172.21.15.36:0/1814394371 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff0600aef60 0x7ff0600b9df0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 rx=0x7ff05c002f70 tx=0x7ff05c006eb0).ready entity=mon.0 client_cookie=4c8a40a75f0dbc07 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:08.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.451+0000 7ff0648a4700 1 -- 172.21.15.36:0/1814394371 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff05c00d5c0 con 0x7ff0600aef60 2022-06-17T12:56:08.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.451+0000 7ff0648a4700 1 -- 172.21.15.36:0/1814394371 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff05c000e90 con 0x7ff0600aef60 2022-06-17T12:56:08.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.451+0000 7ff0648a4700 1 -- 172.21.15.36:0/1814394371 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff05c00d950 con 0x7ff0600aef60 2022-06-17T12:56:08.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.455+0000 7ff0668a8700 1 -- 172.21.15.36:0/1814394371 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff0600aef60 msgr2=0x7ff0600b9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:08.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.455+0000 7ff0668a8700 1 --2- 172.21.15.36:0/1814394371 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff0600aef60 0x7ff0600b9df0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 rx=0x7ff05c002f70 tx=0x7ff05c006eb0).stop 2022-06-17T12:56:08.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.455+0000 7ff0668a8700 1 -- 172.21.15.36:0/1814394371 shutdown_connections 2022-06-17T12:56:08.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.455+0000 7ff0668a8700 1 --2- 172.21.15.36:0/1814394371 >> v2:172.21.15.70:3300/0 conn(0x7ff0600ba2e0 0x7ff0601442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:08.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.455+0000 7ff0668a8700 1 --2- 172.21.15.36:0/1814394371 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff0600aef60 0x7ff0600b9df0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:08.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.455+0000 7ff0668a8700 1 -- 172.21.15.36:0/1814394371 >> 172.21.15.36:0/1814394371 conn(0x7ff06005c550 msgr2=0x7ff0600b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:08.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.455+0000 7ff0668a8700 1 -- 172.21.15.36:0/1814394371 shutdown_connections 2022-06-17T12:56:08.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.455+0000 7ff0668a8700 1 -- 172.21.15.36:0/1814394371 wait complete. 2022-06-17T12:56:08.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.455+0000 7ff0668a8700 1 Processor -- start 2022-06-17T12:56:08.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.455+0000 7ff0668a8700 1 -- start start 2022-06-17T12:56:08.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.459+0000 7ff0668a8700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff0600aef60 0x7ff0601522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:08.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.459+0000 7ff0668a8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0600ba2e0 0x7ff0600b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:08.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.459+0000 7ff0668a8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff06013cb70 0x7ff0600b7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:08.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.459+0000 7ff0668a8700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff060155870 con 0x7ff06013cb70 2022-06-17T12:56:08.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.459+0000 7ff0668a8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff060140300 con 0x7ff0600ba2e0 2022-06-17T12:56:08.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.459+0000 7ff0668a8700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0601405a0 con 0x7ff0600aef60 2022-06-17T12:56:08.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.459+0000 7ff0660a7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff06013cb70 0x7ff0600b7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:08.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.459+0000 7ff0660a7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff06013cb70 0x7ff0600b7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39196/0 (socket says 172.21.15.36:39196) 2022-06-17T12:56:08.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.459+0000 7ff0658a6700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff0600aef60 0x7ff0601522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:08.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.459+0000 7ff0660a7700 1 -- 172.21.15.36:0/4244825936 learned_addr learned my addr 172.21.15.36:0/4244825936 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:08.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.459+0000 7ff0650a5700 1 --2- 172.21.15.36:0/4244825936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0600ba2e0 0x7ff0600b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:08.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.459+0000 7ff0660a7700 1 -- 172.21.15.36:0/4244825936 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff0600aef60 msgr2=0x7ff0601522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:08.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.459+0000 7ff0660a7700 1 --2- 172.21.15.36:0/4244825936 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff0600aef60 0x7ff0601522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:08.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.459+0000 7ff0660a7700 1 -- 172.21.15.36:0/4244825936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0600ba2e0 msgr2=0x7ff0600b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:08.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.459+0000 7ff0660a7700 1 --2- 172.21.15.36:0/4244825936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0600ba2e0 0x7ff0600b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:08.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.459+0000 7ff0660a7700 1 -- 172.21.15.36:0/4244825936 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05c00b1e0 con 0x7ff06013cb70 2022-06-17T12:56:08.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.459+0000 7ff0660a7700 1 --2- 172.21.15.36:0/4244825936 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff06013cb70 0x7ff0600b7ba0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 rx=0x7ff05000c000 tx=0x7ff050006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:08.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.459+0000 7ff056ffd700 1 -- 172.21.15.36:0/4244825936 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff050011750 con 0x7ff06013cb70 2022-06-17T12:56:08.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.459+0000 7ff056ffd700 1 -- 172.21.15.36:0/4244825936 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0500118b0 con 0x7ff06013cb70 2022-06-17T12:56:08.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.459+0000 7ff0668a8700 1 -- 172.21.15.36:0/4244825936 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0600b8090 con 0x7ff06013cb70 2022-06-17T12:56:08.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.459+0000 7ff056ffd700 1 -- 172.21.15.36:0/4244825936 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff050011c10 con 0x7ff06013cb70 2022-06-17T12:56:08.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.459+0000 7ff0668a8700 1 -- 172.21.15.36:0/4244825936 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06013ff80 con 0x7ff06013cb70 2022-06-17T12:56:08.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.459+0000 7ff0668a8700 1 -- 172.21.15.36:0/4244825936 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff060155a00 con 0x7ff06013cb70 2022-06-17T12:56:08.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.463+0000 7ff056ffd700 1 -- 172.21.15.36:0/4244825936 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff050011d70 con 0x7ff06013cb70 2022-06-17T12:56:08.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.463+0000 7ff056ffd700 1 --2- 172.21.15.36:0/4244825936 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff040053a40 0x7ff040055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:08.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.463+0000 7ff056ffd700 1 -- 172.21.15.36:0/4244825936 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5600+0+0 (secure 0 0 0) 0x7ff05006bfb0 con 0x7ff06013cb70 2022-06-17T12:56:08.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.463+0000 7ff0658a6700 1 --2- 172.21.15.36:0/4244825936 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff040053a40 0x7ff040055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:08.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.463+0000 7ff0658a6700 1 --2- 172.21.15.36:0/4244825936 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff040053a40 0x7ff040055f00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7ff05800d000 tx=0x7ff058007ef0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:08.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.467+0000 7ff056ffd700 1 -- 172.21.15.36:0/4244825936 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff060155a00 con 0x7ff06013cb70 2022-06-17T12:56:08.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.591+0000 7ff0668a8700 1 -- 172.21.15.36:0/4244825936 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff060155a00 con 0x7ff06013cb70 2022-06-17T12:56:08.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.591+0000 7ff056ffd700 1 -- 172.21.15.36:0/4244825936 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v127) v1 ==== 84+0+143 (secure 0 0 0) 0x7ff060155a00 con 0x7ff06013cb70 2022-06-17T12:56:08.580 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:08.581 INFO:teuthology.orchestra.run.smithi036.stdout:[ 2022-06-17T12:56:08.581 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:08.581 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 1, 2022-06-17T12:56:08.581 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "device_health_metrics" 2022-06-17T12:56:08.582 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:08.582 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:08.582 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 2, 2022-06-17T12:56:08.582 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "rbd" 2022-06-17T12:56:08.583 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:08.583 INFO:teuthology.orchestra.run.smithi036.stdout:] 2022-06-17T12:56:08.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.595+0000 7ff0668a8700 1 -- 172.21.15.36:0/4244825936 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff040053a40 msgr2=0x7ff040055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:08.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.595+0000 7ff0668a8700 1 --2- 172.21.15.36:0/4244825936 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff040053a40 0x7ff040055f00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7ff05800d000 tx=0x7ff058007ef0).stop 2022-06-17T12:56:08.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.595+0000 7ff0668a8700 1 -- 172.21.15.36:0/4244825936 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff06013cb70 msgr2=0x7ff0600b7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:08.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.595+0000 7ff0668a8700 1 --2- 172.21.15.36:0/4244825936 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff06013cb70 0x7ff0600b7ba0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 rx=0x7ff05000c000 tx=0x7ff050006df0).stop 2022-06-17T12:56:08.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.595+0000 7ff0668a8700 1 -- 172.21.15.36:0/4244825936 shutdown_connections 2022-06-17T12:56:08.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.595+0000 7ff0668a8700 1 --2- 172.21.15.36:0/4244825936 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff040053a40 0x7ff040055f00 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:08.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.595+0000 7ff0668a8700 1 --2- 172.21.15.36:0/4244825936 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff06013cb70 0x7ff0600b7ba0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:08.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.595+0000 7ff0668a8700 1 --2- 172.21.15.36:0/4244825936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0600ba2e0 0x7ff0600b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:08.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.595+0000 7ff0668a8700 1 --2- 172.21.15.36:0/4244825936 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff0600aef60 0x7ff0601522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:08.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.595+0000 7ff0668a8700 1 -- 172.21.15.36:0/4244825936 >> 172.21.15.36:0/4244825936 conn(0x7ff06005c550 msgr2=0x7ff0601457e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:08.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.595+0000 7ff0668a8700 1 -- 172.21.15.36:0/4244825936 shutdown_connections 2022-06-17T12:56:08.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.595+0000 7ff0668a8700 1 -- 172.21.15.36:0/4244825936 wait complete. 2022-06-17T12:56:08.601 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-17T12:56:08.603 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-17T12:56:08.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.723+0000 7f5157fff700 1 Processor -- start 2022-06-17T12:56:08.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.723+0000 7f5157fff700 1 -- start start 2022-06-17T12:56:08.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.727+0000 7f5157fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51580b1190 0x7f51580b2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:08.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.727+0000 7f5157fff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51580b3cf0 con 0x7f51580b6660 2022-06-17T12:56:08.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.727+0000 7f5157fff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5158144bf0 con 0x7f51580af540 2022-06-17T12:56:08.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.727+0000 7f5157fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51580670a0 con 0x7f51580b1190 2022-06-17T12:56:08.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.727+0000 7f5156ffd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f51580b6660 0x7f515813a790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56730/0 (socket says 172.21.15.36:56730) 2022-06-17T12:56:08.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.727+0000 7f5156ffd700 1 -- 172.21.15.36:0/4184618322 learned_addr learned my addr 172.21.15.36:0/4184618322 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:08.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.727+0000 7f51577fe700 1 --2- 172.21.15.36:0/4184618322 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51580b1190 0x7f51580b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:08.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.727+0000 7f51567fc700 1 -- 172.21.15.36:0/4184618322 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1358907064 0 0) 0x7f5148001110 con 0x7f51580b6660 2022-06-17T12:56:08.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.727+0000 7f51567fc700 1 -- 172.21.15.36:0/4184618322 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5138003260 con 0x7f51580b6660 2022-06-17T12:56:08.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.727+0000 7f51567fc700 1 -- 172.21.15.36:0/4184618322 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1189900358 0 0) 0x7f5158144bf0 con 0x7f51580af540 2022-06-17T12:56:08.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.727+0000 7f51577fe700 1 -- 172.21.15.36:0/4184618322 >> v1:172.21.15.70:6789/0 conn(0x7f51580af540 legacy=0x7f5158141010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:08.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.727+0000 7f51577fe700 1 -- 172.21.15.36:0/4184618322 >> v1:172.21.15.36:6789/0 conn(0x7f51580b6660 legacy=0x7f515813a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:08.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.727+0000 7f51577fe700 1 -- 172.21.15.36:0/4184618322 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5158144e90 con 0x7f51580b1190 2022-06-17T12:56:08.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.727+0000 7f51577fe700 1 --2- 172.21.15.36:0/4184618322 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51580b1190 0x7f51580b2860 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f51500032f0 tx=0x7f51500091d0).ready entity=mon.2 client_cookie=c2bf8257cb11cc0b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:08.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.727+0000 7f51567fc700 1 -- 172.21.15.36:0/4184618322 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f515000b6c0 con 0x7f51580b1190 2022-06-17T12:56:08.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.727+0000 7f51567fc700 1 -- 172.21.15.36:0/4184618322 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515000b820 con 0x7f51580b1190 2022-06-17T12:56:08.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.727+0000 7f51567fc700 1 -- 172.21.15.36:0/4184618322 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f515000b990 con 0x7f51580b1190 2022-06-17T12:56:08.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.727+0000 7f5157fff700 1 -- 172.21.15.36:0/4184618322 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51580b1190 msgr2=0x7f51580b2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:08.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.727+0000 7f5157fff700 1 --2- 172.21.15.36:0/4184618322 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51580b1190 0x7f51580b2860 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f51500032f0 tx=0x7f51500091d0).stop 2022-06-17T12:56:08.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.727+0000 7f5157fff700 1 -- 172.21.15.36:0/4184618322 shutdown_connections 2022-06-17T12:56:08.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.727+0000 7f5157fff700 1 --2- 172.21.15.36:0/4184618322 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51580b1190 0x7f51580b2860 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:08.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.727+0000 7f5157fff700 1 -- 172.21.15.36:0/4184618322 >> 172.21.15.36:0/4184618322 conn(0x7f51580614b0 msgr2=0x7f5158061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:08.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.727+0000 7f5157fff700 1 -- 172.21.15.36:0/4184618322 shutdown_connections 2022-06-17T12:56:08.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.727+0000 7f5157fff700 1 -- 172.21.15.36:0/4184618322 wait complete. 2022-06-17T12:56:08.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.731+0000 7f5157fff700 1 Processor -- start 2022-06-17T12:56:08.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.731+0000 7f5157fff700 1 -- start start 2022-06-17T12:56:08.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.731+0000 7f5157fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51580af540 0x7f515813b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:08.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.731+0000 7f5157fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f51580b1190 0x7f51580b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:08.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.731+0000 7f5157fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51580b6660 0x7f515813bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:08.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.731+0000 7f5157fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515813f930 con 0x7f51580b1190 2022-06-17T12:56:08.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.731+0000 7f5157fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515813d780 con 0x7f51580af540 2022-06-17T12:56:08.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.731+0000 7f5157fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f515810fa60 con 0x7f51580b6660 2022-06-17T12:56:08.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.731+0000 7f514ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f51580b1190 0x7f51580b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:08.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.731+0000 7f514ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f51580b1190 0x7f51580b4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39206/0 (socket says 172.21.15.36:39206) 2022-06-17T12:56:08.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.731+0000 7f514ffff700 1 -- 172.21.15.36:0/2461855403 learned_addr learned my addr 172.21.15.36:0/2461855403 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:08.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.731+0000 7f51577fe700 1 --2- 172.21.15.36:0/2461855403 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51580b6660 0x7f515813bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:08.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.731+0000 7f5156ffd700 1 --2- 172.21.15.36:0/2461855403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51580af540 0x7f515813b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:08.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.731+0000 7f514ffff700 1 -- 172.21.15.36:0/2461855403 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51580b6660 msgr2=0x7f515813bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:08.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.731+0000 7f514ffff700 1 --2- 172.21.15.36:0/2461855403 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51580b6660 0x7f515813bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:08.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.735+0000 7f514ffff700 1 -- 172.21.15.36:0/2461855403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51580af540 msgr2=0x7f515813b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:08.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.735+0000 7f514ffff700 1 --2- 172.21.15.36:0/2461855403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51580af540 0x7f515813b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:08.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.735+0000 7f514ffff700 1 -- 172.21.15.36:0/2461855403 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5150009470 con 0x7f51580b1190 2022-06-17T12:56:08.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.735+0000 7f51577fe700 1 --2- 172.21.15.36:0/2461855403 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51580b6660 0x7f515813bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:56:08.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.735+0000 7f5156ffd700 1 --2- 172.21.15.36:0/2461855403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51580af540 0x7f515813b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:56:08.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.735+0000 7f514ffff700 1 --2- 172.21.15.36:0/2461855403 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f51580b1190 0x7f51580b4e10 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7f515814db70 tx=0x7f5140007a30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:08.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.735+0000 7f5154ff9700 1 -- 172.21.15.36:0/2461855403 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f514000ce20 con 0x7f51580b1190 2022-06-17T12:56:08.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.735+0000 7f5154ff9700 1 -- 172.21.15.36:0/2461855403 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514000f470 con 0x7f51580b1190 2022-06-17T12:56:08.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.735+0000 7f5157fff700 1 -- 172.21.15.36:0/2461855403 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f515810fd00 con 0x7f51580b1190 2022-06-17T12:56:08.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.735+0000 7f5154ff9700 1 -- 172.21.15.36:0/2461855403 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f514000f7d0 con 0x7f51580b1190 2022-06-17T12:56:08.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.735+0000 7f5157fff700 1 -- 172.21.15.36:0/2461855403 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51580b5520 con 0x7f51580b1190 2022-06-17T12:56:08.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.735+0000 7f5154ff9700 1 -- 172.21.15.36:0/2461855403 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f5140019030 con 0x7f51580b1190 2022-06-17T12:56:08.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.735+0000 7f5157fff700 1 -- 172.21.15.36:0/2461855403 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f515810f8b0 con 0x7f51580b1190 2022-06-17T12:56:08.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.735+0000 7f5154ff9700 1 --2- 172.21.15.36:0/2461855403 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f513c053a40 0x7f513c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:08.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.735+0000 7f5154ff9700 1 -- 172.21.15.36:0/2461855403 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f514006af20 con 0x7f51580b1190 2022-06-17T12:56:08.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.735+0000 7f5156ffd700 1 --2- 172.21.15.36:0/2461855403 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f513c053a40 0x7f513c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:08.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.735+0000 7f5156ffd700 1 --2- 172.21.15.36:0/2461855403 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f513c053a40 0x7f513c055f00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f5148008000 tx=0x7f5148009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:08.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.739+0000 7f5154ff9700 1 -- 172.21.15.36:0/2461855403 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5140042110 con 0x7f51580b1190 2022-06-17T12:56:08.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:08.863+0000 7f5157fff700 1 -- 172.21.15.36:0/2461855403 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f51581500f0 con 0x7f51580b1190 2022-06-17T12:56:09.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.635+0000 7f5154ff9700 1 -- 172.21.15.36:0/2461855403 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v128) v1 ==== 133+0+0 (secure 0 0 0) 0x7f5140045730 con 0x7f51580b1190 2022-06-17T12:56:09.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.639+0000 7f5157fff700 1 -- 172.21.15.36:0/2461855403 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f513c053a40 msgr2=0x7f513c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:09.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.639+0000 7f5157fff700 1 --2- 172.21.15.36:0/2461855403 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f513c053a40 0x7f513c055f00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f5148008000 tx=0x7f5148009040).stop 2022-06-17T12:56:09.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.639+0000 7f5157fff700 1 -- 172.21.15.36:0/2461855403 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f51580b1190 msgr2=0x7f51580b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:09.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.639+0000 7f5157fff700 1 --2- 172.21.15.36:0/2461855403 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f51580b1190 0x7f51580b4e10 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7f515814db70 tx=0x7f5140007a30).stop 2022-06-17T12:56:09.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.639+0000 7f5157fff700 1 -- 172.21.15.36:0/2461855403 shutdown_connections 2022-06-17T12:56:09.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.639+0000 7f5157fff700 1 --2- 172.21.15.36:0/2461855403 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f513c053a40 0x7f513c055f00 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:09.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.639+0000 7f5157fff700 1 --2- 172.21.15.36:0/2461855403 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51580b6660 0x7f515813bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:09.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.639+0000 7f5157fff700 1 --2- 172.21.15.36:0/2461855403 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f51580b1190 0x7f51580b4e10 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:09.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.639+0000 7f5157fff700 1 --2- 172.21.15.36:0/2461855403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51580af540 0x7f515813b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:09.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.639+0000 7f5157fff700 1 -- 172.21.15.36:0/2461855403 >> 172.21.15.36:0/2461855403 conn(0x7f51580614b0 msgr2=0x7f5158061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:09.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.639+0000 7f5157fff700 1 -- 172.21.15.36:0/2461855403 shutdown_connections 2022-06-17T12:56:09.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.639+0000 7f5157fff700 1 -- 172.21.15.36:0/2461855403 wait complete. 2022-06-17T12:56:09.634 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' created 2022-06-17T12:56:09.642 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-06-17T12:56:09.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.783+0000 7f0b630b5700 1 Processor -- start 2022-06-17T12:56:09.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.783+0000 7f0b630b5700 1 -- start start 2022-06-17T12:56:09.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.783+0000 7f0b630b5700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0b5c0b45c0 0x7f0b5c0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:09.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.783+0000 7f0b630b5700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b5c0b4160 con 0x7f0b5c0b9860 2022-06-17T12:56:09.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.783+0000 7f0b630b5700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b5c0b9c40 con 0x7f0b5c0b4f00 2022-06-17T12:56:09.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.783+0000 7f0b630b5700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b5c0627d0 con 0x7f0b5c0b45c0 2022-06-17T12:56:09.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.787+0000 7f0b620b3700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f0b5c0b9860 0x7f0b5c143840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56744/0 (socket says 172.21.15.36:56744) 2022-06-17T12:56:09.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.787+0000 7f0b620b3700 1 -- 172.21.15.36:0/2560058593 learned_addr learned my addr 172.21.15.36:0/2560058593 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:09.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.787+0000 7f0b618b2700 1 --2- 172.21.15.36:0/2560058593 >> v2:172.21.15.70:3300/0 conn(0x7f0b5c0b45c0 0x7f0b5c0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:09.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.787+0000 7f0b610b1700 1 -- 172.21.15.36:0/2560058593 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4233360539 0 0) 0x7f0b5c0b4160 con 0x7f0b5c0b9860 2022-06-17T12:56:09.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.787+0000 7f0b610b1700 1 -- 172.21.15.36:0/2560058593 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b44003260 con 0x7f0b5c0b9860 2022-06-17T12:56:09.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.787+0000 7f0b618b2700 1 -- 172.21.15.36:0/2560058593 >> v1:172.21.15.70:6789/0 conn(0x7f0b5c0b4f00 legacy=0x7f0b5c0b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:09.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.787+0000 7f0b610b1700 1 -- 172.21.15.36:0/2560058593 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3448568653 0 0) 0x7f0b44003260 con 0x7f0b5c0b9860 2022-06-17T12:56:09.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.787+0000 7f0b618b2700 1 -- 172.21.15.36:0/2560058593 >> v1:172.21.15.36:6789/0 conn(0x7f0b5c0b9860 legacy=0x7f0b5c143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:09.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.787+0000 7f0b618b2700 1 -- 172.21.15.36:0/2560058593 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5c062a70 con 0x7f0b5c0b45c0 2022-06-17T12:56:09.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.787+0000 7f0b618b2700 1 --2- 172.21.15.36:0/2560058593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b5c0b45c0 0x7f0b5c0b4a10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f0b5c0ba5d0 tx=0x7f0b58006df0).ready entity=mon.1 client_cookie=df7a3509227aa6f8 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:09.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.787+0000 7f0b610b1700 1 -- 172.21.15.36:0/2560058593 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b5800d610 con 0x7f0b5c0b45c0 2022-06-17T12:56:09.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.787+0000 7f0b610b1700 1 -- 172.21.15.36:0/2560058593 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b58000e90 con 0x7f0b5c0b45c0 2022-06-17T12:56:09.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.787+0000 7f0b610b1700 1 -- 172.21.15.36:0/2560058593 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b5800d960 con 0x7f0b5c0b45c0 2022-06-17T12:56:09.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.787+0000 7f0b630b5700 1 -- 172.21.15.36:0/2560058593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b5c0b45c0 msgr2=0x7f0b5c0b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:09.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.787+0000 7f0b630b5700 1 --2- 172.21.15.36:0/2560058593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b5c0b45c0 0x7f0b5c0b4a10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f0b5c0ba5d0 tx=0x7f0b58006df0).stop 2022-06-17T12:56:09.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.787+0000 7f0b630b5700 1 -- 172.21.15.36:0/2560058593 shutdown_connections 2022-06-17T12:56:09.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.787+0000 7f0b630b5700 1 --2- 172.21.15.36:0/2560058593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b5c0b45c0 0x7f0b5c0b4a10 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:09.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.787+0000 7f0b630b5700 1 -- 172.21.15.36:0/2560058593 >> 172.21.15.36:0/2560058593 conn(0x7f0b5c05c550 msgr2=0x7f0b5c060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:09.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.787+0000 7f0b630b5700 1 -- 172.21.15.36:0/2560058593 shutdown_connections 2022-06-17T12:56:09.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.787+0000 7f0b630b5700 1 -- 172.21.15.36:0/2560058593 wait complete. 2022-06-17T12:56:09.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.791+0000 7f0b630b5700 1 Processor -- start 2022-06-17T12:56:09.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.791+0000 7f0b630b5700 1 -- start start 2022-06-17T12:56:09.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.791+0000 7f0b630b5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0b5c0b45c0 0x7f0b5c0ad880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:09.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.791+0000 7f0b630b5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b5c0b4f00 0x7f0b5c152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:09.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.791+0000 7f0b630b5700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0b5c0b9860 0x7f0b5c152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:09.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.791+0000 7f0b630b5700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c0af890 con 0x7f0b5c0b45c0 2022-06-17T12:56:09.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.791+0000 7f0b630b5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c1521b0 con 0x7f0b5c0b4f00 2022-06-17T12:56:09.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.791+0000 7f0b630b5700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c0ae0d0 con 0x7f0b5c0b9860 2022-06-17T12:56:09.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.791+0000 7f0b620b3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0b5c0b45c0 0x7f0b5c0ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:09.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.791+0000 7f0b620b3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0b5c0b45c0 0x7f0b5c0ad880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39218/0 (socket says 172.21.15.36:39218) 2022-06-17T12:56:09.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.791+0000 7f0b620b3700 1 -- 172.21.15.36:0/301421714 learned_addr learned my addr 172.21.15.36:0/301421714 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:09.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.791+0000 7f0b618b2700 1 --2- 172.21.15.36:0/301421714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b5c0b4f00 0x7f0b5c152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:09.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.791+0000 7f0b628b4700 1 --2- 172.21.15.36:0/301421714 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0b5c0b9860 0x7f0b5c152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:09.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.791+0000 7f0b620b3700 1 -- 172.21.15.36:0/301421714 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0b5c0b9860 msgr2=0x7f0b5c152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:09.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.791+0000 7f0b620b3700 1 --2- 172.21.15.36:0/301421714 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0b5c0b9860 0x7f0b5c152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:09.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.791+0000 7f0b620b3700 1 -- 172.21.15.36:0/301421714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b5c0b4f00 msgr2=0x7f0b5c152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:09.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.791+0000 7f0b620b3700 1 --2- 172.21.15.36:0/301421714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b5c0b4f00 0x7f0b5c152660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:09.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.791+0000 7f0b620b3700 1 -- 172.21.15.36:0/301421714 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5800b180 con 0x7f0b5c0b45c0 2022-06-17T12:56:09.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.791+0000 7f0b620b3700 1 --2- 172.21.15.36:0/301421714 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0b5c0b45c0 0x7f0b5c0ad880 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f0b4c000bc0 tx=0x7f0b4c0079e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:09.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.791+0000 7f0b4b7fe700 1 -- 172.21.15.36:0/301421714 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b4c00cda0 con 0x7f0b5c0b45c0 2022-06-17T12:56:09.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.791+0000 7f0b4b7fe700 1 -- 172.21.15.36:0/301421714 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4c007170 con 0x7f0b5c0b45c0 2022-06-17T12:56:09.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.791+0000 7f0b630b5700 1 -- 172.21.15.36:0/301421714 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5c140150 con 0x7f0b5c0b45c0 2022-06-17T12:56:09.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.791+0000 7f0b4b7fe700 1 -- 172.21.15.36:0/301421714 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b4c004520 con 0x7f0b5c0b45c0 2022-06-17T12:56:09.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.791+0000 7f0b630b5700 1 -- 172.21.15.36:0/301421714 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5c153040 con 0x7f0b5c0b45c0 2022-06-17T12:56:09.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.795+0000 7f0b4b7fe700 1 -- 172.21.15.36:0/301421714 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f0b4c004680 con 0x7f0b5c0b45c0 2022-06-17T12:56:09.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.795+0000 7f0b630b5700 1 -- 172.21.15.36:0/301421714 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b5c0adf10 con 0x7f0b5c0b45c0 2022-06-17T12:56:09.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.795+0000 7f0b4b7fe700 1 --2- 172.21.15.36:0/301421714 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0b3c053ad0 0x7f0b3c055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:09.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.795+0000 7f0b4b7fe700 1 -- 172.21.15.36:0/301421714 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f0b4c06b6d0 con 0x7f0b5c0b45c0 2022-06-17T12:56:09.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.795+0000 7f0b618b2700 1 --2- 172.21.15.36:0/301421714 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0b3c053ad0 0x7f0b3c055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:09.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.795+0000 7f0b618b2700 1 --2- 172.21.15.36:0/301421714 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0b3c053ad0 0x7f0b3c055f90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f0b58018000 tx=0x7f0b58014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:09.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.799+0000 7f0b4b7fe700 1 -- 172.21.15.36:0/301421714 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0b4c042530 con 0x7f0b5c0b45c0 2022-06-17T12:56:09.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:09.927+0000 7f0b630b5700 1 -- 172.21.15.36:0/301421714 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f0b5c057410 con 0x7f0b5c0b45c0 2022-06-17T12:56:10.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.643+0000 7f0b4b7fe700 1 -- 172.21.15.36:0/301421714 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v129) v1 ==== 125+0+0 (secure 0 0 0) 0x7f0b4c045b50 con 0x7f0b5c0b45c0 2022-06-17T12:56:10.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.655+0000 7f0b630b5700 1 -- 172.21.15.36:0/301421714 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0b3c053ad0 msgr2=0x7f0b3c055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:10.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.655+0000 7f0b630b5700 1 --2- 172.21.15.36:0/301421714 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0b3c053ad0 0x7f0b3c055f90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f0b58018000 tx=0x7f0b58014000).stop 2022-06-17T12:56:10.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.655+0000 7f0b630b5700 1 -- 172.21.15.36:0/301421714 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0b5c0b45c0 msgr2=0x7f0b5c0ad880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:10.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.655+0000 7f0b630b5700 1 --2- 172.21.15.36:0/301421714 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0b5c0b45c0 0x7f0b5c0ad880 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f0b4c000bc0 tx=0x7f0b4c0079e0).stop 2022-06-17T12:56:10.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.655+0000 7f0b630b5700 1 -- 172.21.15.36:0/301421714 shutdown_connections 2022-06-17T12:56:10.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.655+0000 7f0b630b5700 1 --2- 172.21.15.36:0/301421714 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0b3c053ad0 0x7f0b3c055f90 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:10.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.655+0000 7f0b630b5700 1 --2- 172.21.15.36:0/301421714 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0b5c0b9860 0x7f0b5c152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:10.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.655+0000 7f0b630b5700 1 --2- 172.21.15.36:0/301421714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b5c0b4f00 0x7f0b5c152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:10.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.655+0000 7f0b630b5700 1 --2- 172.21.15.36:0/301421714 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0b5c0b45c0 0x7f0b5c0ad880 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:10.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.655+0000 7f0b630b5700 1 -- 172.21.15.36:0/301421714 >> 172.21.15.36:0/301421714 conn(0x7f0b5c05c550 msgr2=0x7f0b5c0b6780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:10.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.655+0000 7f0b630b5700 1 -- 172.21.15.36:0/301421714 shutdown_connections 2022-06-17T12:56:10.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.655+0000 7f0b630b5700 1 -- 172.21.15.36:0/301421714 wait complete. 2022-06-17T12:56:10.676 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' created 2022-06-17T12:56:10.678 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-17T12:56:10.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.831+0000 7f3a586fd700 1 Processor -- start 2022-06-17T12:56:10.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.831+0000 7f3a586fd700 1 -- start start 2022-06-17T12:56:10.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a586fd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f3a50142a20 0x7f3a50060780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:10.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a586fd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a50060c70 0x7f3a5013fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:10.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a586fd700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a500b3520 con 0x7f3a500b2340 2022-06-17T12:56:10.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a586fd700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a500b37c0 con 0x7f3a50142a20 2022-06-17T12:56:10.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a586fd700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a50062f20 con 0x7f3a50060c70 2022-06-17T12:56:10.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a56499700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f3a50142a20 0x7f3a50060780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:10.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a56499700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f3a50142a20 0x7f3a50060780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39226/0 (socket says 172.21.15.36:39226) 2022-06-17T12:56:10.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a56499700 1 -- 172.21.15.36:0/3522141025 learned_addr learned my addr 172.21.15.36:0/3522141025 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:10.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a55c98700 1 --2- 172.21.15.36:0/3522141025 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a50060c70 0x7f3a5013fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:10.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a56499700 1 -- 172.21.15.36:0/3522141025 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a50060c70 msgr2=0x7f3a5013fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:10.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a56499700 1 --2- 172.21.15.36:0/3522141025 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a50060c70 0x7f3a5013fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:10.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a56499700 1 -- 172.21.15.36:0/3522141025 >> v1:172.21.15.70:6789/0 conn(0x7f3a500b2340 legacy=0x7f3a50140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:10.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a56499700 1 -- 172.21.15.36:0/3522141025 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a500acf00 con 0x7f3a50142a20 2022-06-17T12:56:10.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a56499700 1 --2- 172.21.15.36:0/3522141025 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3a50142a20 0x7f3a50060780 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7f3a4c002f70 tx=0x7f3a4c006eb0).ready entity=mon.0 client_cookie=53385ad5e4c71fe server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:10.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a55497700 1 -- 172.21.15.36:0/3522141025 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a4c00d5c0 con 0x7f3a50142a20 2022-06-17T12:56:10.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a55497700 1 -- 172.21.15.36:0/3522141025 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4c000e90 con 0x7f3a50142a20 2022-06-17T12:56:10.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a55497700 1 -- 172.21.15.36:0/3522141025 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a4c00d950 con 0x7f3a50142a20 2022-06-17T12:56:10.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a586fd700 1 -- 172.21.15.36:0/3522141025 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3a50142a20 msgr2=0x7f3a50060780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:10.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a586fd700 1 --2- 172.21.15.36:0/3522141025 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3a50142a20 0x7f3a50060780 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7f3a4c002f70 tx=0x7f3a4c006eb0).stop 2022-06-17T12:56:10.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a586fd700 1 -- 172.21.15.36:0/3522141025 shutdown_connections 2022-06-17T12:56:10.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a586fd700 1 --2- 172.21.15.36:0/3522141025 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a50060c70 0x7f3a5013fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:10.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a586fd700 1 --2- 172.21.15.36:0/3522141025 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3a50142a20 0x7f3a50060780 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:10.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a586fd700 1 -- 172.21.15.36:0/3522141025 >> 172.21.15.36:0/3522141025 conn(0x7f3a500b6870 msgr2=0x7f3a500b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:10.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a586fd700 1 -- 172.21.15.36:0/3522141025 shutdown_connections 2022-06-17T12:56:10.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.835+0000 7f3a586fd700 1 -- 172.21.15.36:0/3522141025 wait complete. 2022-06-17T12:56:10.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a586fd700 1 Processor -- start 2022-06-17T12:56:10.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a586fd700 1 -- start start 2022-06-17T12:56:10.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a586fd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a50060c70 0x7f3a50147500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:10.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a586fd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a500b2340 0x7f3a50144ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:10.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a586fd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3a50142a20 0x7f3a501453b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:10.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a586fd700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a50156670 con 0x7f3a50142a20 2022-06-17T12:56:10.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a586fd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a50147ce0 con 0x7f3a50060c70 2022-06-17T12:56:10.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a586fd700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a501568d0 con 0x7f3a500b2340 2022-06-17T12:56:10.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a56c9a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3a50142a20 0x7f3a501453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:10.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a56c9a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3a50142a20 0x7f3a501453b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39236/0 (socket says 172.21.15.36:39236) 2022-06-17T12:56:10.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a56c9a700 1 -- 172.21.15.36:0/1454683412 learned_addr learned my addr 172.21.15.36:0/1454683412 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:10.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a56499700 1 --2- 172.21.15.36:0/1454683412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a50060c70 0x7f3a50147500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:10.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a55c98700 1 --2- 172.21.15.36:0/1454683412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a500b2340 0x7f3a50144ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:10.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a56c9a700 1 -- 172.21.15.36:0/1454683412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a500b2340 msgr2=0x7f3a50144ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:10.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a56c9a700 1 --2- 172.21.15.36:0/1454683412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a500b2340 0x7f3a50144ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:10.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a56c9a700 1 -- 172.21.15.36:0/1454683412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a50060c70 msgr2=0x7f3a50147500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:10.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a56c9a700 1 --2- 172.21.15.36:0/1454683412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a50060c70 0x7f3a50147500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:10.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a56c9a700 1 -- 172.21.15.36:0/1454683412 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a4c00b1e0 con 0x7f3a50142a20 2022-06-17T12:56:10.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a55c98700 1 --2- 172.21.15.36:0/1454683412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a500b2340 0x7f3a50144ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:56:10.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a56c9a700 1 --2- 172.21.15.36:0/1454683412 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3a50142a20 0x7f3a501453b0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7f3a5005d080 tx=0x7f3a48007f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:10.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a3f7fe700 1 -- 172.21.15.36:0/1454683412 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a4800e4a0 con 0x7f3a50142a20 2022-06-17T12:56:10.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a3f7fe700 1 -- 172.21.15.36:0/1454683412 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4800e600 con 0x7f3a50142a20 2022-06-17T12:56:10.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a586fd700 1 -- 172.21.15.36:0/1454683412 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a501479f0 con 0x7f3a50142a20 2022-06-17T12:56:10.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a3f7fe700 1 -- 172.21.15.36:0/1454683412 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a4800e960 con 0x7f3a50142a20 2022-06-17T12:56:10.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.839+0000 7f3a586fd700 1 -- 172.21.15.36:0/1454683412 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a50144a20 con 0x7f3a50142a20 2022-06-17T12:56:10.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.843+0000 7f3a586fd700 1 -- 172.21.15.36:0/1454683412 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a5013d250 con 0x7f3a50142a20 2022-06-17T12:56:10.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.843+0000 7f3a3f7fe700 1 -- 172.21.15.36:0/1454683412 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f3a48019030 con 0x7f3a50142a20 2022-06-17T12:56:10.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.843+0000 7f3a3f7fe700 1 --2- 172.21.15.36:0/1454683412 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3a300539d0 0x7f3a30055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:10.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.843+0000 7f3a3f7fe700 1 -- 172.21.15.36:0/1454683412 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f3a4806b710 con 0x7f3a50142a20 2022-06-17T12:56:10.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.843+0000 7f3a56499700 1 --2- 172.21.15.36:0/1454683412 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3a300539d0 0x7f3a30055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:10.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.843+0000 7f3a56499700 1 --2- 172.21.15.36:0/1454683412 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3a300539d0 0x7f3a30055e90 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f3a4c01a000 tx=0x7f3a4c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:10.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:10.851+0000 7f3a3f7fe700 1 -- 172.21.15.36:0/1454683412 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3a5013d250 con 0x7f3a50142a20 2022-06-17T12:56:11.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.015+0000 7f3a586fd700 1 -- 172.21.15.36:0/1454683412 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f3a5013d250 con 0x7f3a50142a20 2022-06-17T12:56:11.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.023+0000 7f3a3f7fe700 1 -- 172.21.15.36:0/1454683412 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 25 and data pool 26 v69) v1 ==== 189+0+0 (secure 0 0 0) 0x7f3a5013d250 con 0x7f3a50142a20 2022-06-17T12:56:11.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.027+0000 7f3a586fd700 1 -- 172.21.15.36:0/1454683412 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3a300539d0 msgr2=0x7f3a30055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:11.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.027+0000 7f3a586fd700 1 --2- 172.21.15.36:0/1454683412 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3a300539d0 0x7f3a30055e90 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f3a4c01a000 tx=0x7f3a4c015000).stop 2022-06-17T12:56:11.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.027+0000 7f3a586fd700 1 -- 172.21.15.36:0/1454683412 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3a50142a20 msgr2=0x7f3a501453b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:11.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.027+0000 7f3a586fd700 1 --2- 172.21.15.36:0/1454683412 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3a50142a20 0x7f3a501453b0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7f3a5005d080 tx=0x7f3a48007f00).stop 2022-06-17T12:56:11.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.027+0000 7f3a586fd700 1 -- 172.21.15.36:0/1454683412 shutdown_connections 2022-06-17T12:56:11.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.027+0000 7f3a586fd700 1 --2- 172.21.15.36:0/1454683412 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3a300539d0 0x7f3a30055e90 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:11.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.027+0000 7f3a586fd700 1 --2- 172.21.15.36:0/1454683412 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3a50142a20 0x7f3a501453b0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:11.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.027+0000 7f3a586fd700 1 --2- 172.21.15.36:0/1454683412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a500b2340 0x7f3a50144ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:11.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.027+0000 7f3a586fd700 1 --2- 172.21.15.36:0/1454683412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a50060c70 0x7f3a50147500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:11.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.027+0000 7f3a586fd700 1 -- 172.21.15.36:0/1454683412 >> 172.21.15.36:0/1454683412 conn(0x7f3a500b6870 msgr2=0x7f3a500b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:11.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.027+0000 7f3a586fd700 1 -- 172.21.15.36:0/1454683412 shutdown_connections 2022-06-17T12:56:11.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.027+0000 7f3a586fd700 1 -- 172.21.15.36:0/1454683412 wait complete. 2022-06-17T12:56:11.080 INFO:teuthology.orchestra.run.smithi036.stderr:new fs with metadata pool 25 and data pool 26 2022-06-17T12:56:11.084 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:56:11.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.215+0000 7fce530f2700 1 Processor -- start 2022-06-17T12:56:11.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.215+0000 7fce530f2700 1 -- start start 2022-06-17T12:56:11.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.215+0000 7fce530f2700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fce4c0b2200 0x7fce4c0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:11.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.215+0000 7fce530f2700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4c0b25e0 con 0x7fce4c0b2cd0 2022-06-17T12:56:11.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.215+0000 7fce530f2700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4c14b6f0 con 0x7fce4c0af8d0 2022-06-17T12:56:11.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.215+0000 7fce530f2700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce4c0ad790 con 0x7fce4c0b2200 2022-06-17T12:56:11.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.215+0000 7fce528f1700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fce4c0b2cd0 0x7fce4c14afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56774/0 (socket says 172.21.15.36:56774) 2022-06-17T12:56:11.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.215+0000 7fce528f1700 1 -- 172.21.15.36:0/1724162142 learned_addr learned my addr 172.21.15.36:0/1724162142 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:11.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.215+0000 7fce520f0700 1 --2- 172.21.15.36:0/1724162142 >> v2:172.21.15.70:3300/0 conn(0x7fce4c0b2200 0x7fce4c0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:11.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.215+0000 7fce510ee700 1 -- 172.21.15.36:0/1724162142 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4159103070 0 0) 0x7fce4c0b25e0 con 0x7fce4c0b2cd0 2022-06-17T12:56:11.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.215+0000 7fce510ee700 1 -- 172.21.15.36:0/1724162142 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce34003260 con 0x7fce4c0b2cd0 2022-06-17T12:56:11.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.215+0000 7fce510ee700 1 -- 172.21.15.36:0/1724162142 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3679966024 0 0) 0x7fce4c14b6f0 con 0x7fce4c0af8d0 2022-06-17T12:56:11.366 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.215+0000 7fce510ee700 1 -- 172.21.15.36:0/1724162142 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce4c0b25e0 con 0x7fce4c0af8d0 2022-06-17T12:56:11.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.215+0000 7fce510ee700 1 -- 172.21.15.36:0/1724162142 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2510588976 0 0) 0x7fce34003260 con 0x7fce4c0b2cd0 2022-06-17T12:56:11.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.215+0000 7fce520f0700 1 -- 172.21.15.36:0/1724162142 >> v1:172.21.15.70:6789/0 conn(0x7fce4c0af8d0 legacy=0x7fce4c14a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:11.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.215+0000 7fce520f0700 1 -- 172.21.15.36:0/1724162142 >> v1:172.21.15.36:6789/0 conn(0x7fce4c0b2cd0 legacy=0x7fce4c14afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:11.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.215+0000 7fce520f0700 1 -- 172.21.15.36:0/1724162142 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c0aef40 con 0x7fce4c0b2200 2022-06-17T12:56:11.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.215+0000 7fce520f0700 1 --2- 172.21.15.36:0/1724162142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce4c0b2200 0x7fce4c0af3e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7fce44001490 tx=0x7fce40006df0).ready entity=mon.1 client_cookie=ecc7c116a992264 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:11.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.215+0000 7fce510ee700 1 -- 172.21.15.36:0/1724162142 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce440019e0 con 0x7fce4c0b2200 2022-06-17T12:56:11.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.215+0000 7fce510ee700 1 -- 172.21.15.36:0/1724162142 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce40000e90 con 0x7fce4c0b2200 2022-06-17T12:56:11.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.215+0000 7fce510ee700 1 -- 172.21.15.36:0/1724162142 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce40012460 con 0x7fce4c0b2200 2022-06-17T12:56:11.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.219+0000 7fce530f2700 1 -- 172.21.15.36:0/1724162142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce4c0b2200 msgr2=0x7fce4c0af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:11.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.219+0000 7fce530f2700 1 --2- 172.21.15.36:0/1724162142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce4c0b2200 0x7fce4c0af3e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7fce44001490 tx=0x7fce40006df0).stop 2022-06-17T12:56:11.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.219+0000 7fce530f2700 1 -- 172.21.15.36:0/1724162142 shutdown_connections 2022-06-17T12:56:11.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.219+0000 7fce530f2700 1 --2- 172.21.15.36:0/1724162142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce4c0b2200 0x7fce4c0af3e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:11.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.219+0000 7fce530f2700 1 -- 172.21.15.36:0/1724162142 >> 172.21.15.36:0/1724162142 conn(0x7fce4c05f770 msgr2=0x7fce4c05fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:11.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.219+0000 7fce530f2700 1 -- 172.21.15.36:0/1724162142 shutdown_connections 2022-06-17T12:56:11.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.219+0000 7fce530f2700 1 -- 172.21.15.36:0/1724162142 wait complete. 2022-06-17T12:56:11.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.219+0000 7fce530f2700 1 Processor -- start 2022-06-17T12:56:11.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.219+0000 7fce530f2700 1 -- start start 2022-06-17T12:56:11.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.223+0000 7fce530f2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce4c0af8d0 0x7fce4c1431e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:11.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.223+0000 7fce530f2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce4c0b2200 0x7fce4c1487d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:11.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.223+0000 7fce530f2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce4c0b2cd0 0x7fce4c148cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:11.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.223+0000 7fce530f2700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c149d00 con 0x7fce4c0b2200 2022-06-17T12:56:11.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.223+0000 7fce530f2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c14afe0 con 0x7fce4c0b2cd0 2022-06-17T12:56:11.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.223+0000 7fce530f2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c14a280 con 0x7fce4c0af8d0 2022-06-17T12:56:11.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.223+0000 7fce518ef700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce4c0b2200 0x7fce4c1487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:11.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.223+0000 7fce518ef700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce4c0b2200 0x7fce4c1487d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39248/0 (socket says 172.21.15.36:39248) 2022-06-17T12:56:11.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.223+0000 7fce518ef700 1 -- 172.21.15.36:0/823664376 learned_addr learned my addr 172.21.15.36:0/823664376 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:11.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.223+0000 7fce520f0700 1 --2- 172.21.15.36:0/823664376 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce4c0af8d0 0x7fce4c1431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:11.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.223+0000 7fce528f1700 1 --2- 172.21.15.36:0/823664376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce4c0b2cd0 0x7fce4c148cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:11.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.223+0000 7fce518ef700 1 -- 172.21.15.36:0/823664376 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce4c0af8d0 msgr2=0x7fce4c1431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:11.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.223+0000 7fce518ef700 1 --2- 172.21.15.36:0/823664376 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce4c0af8d0 0x7fce4c1431e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:11.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.223+0000 7fce518ef700 1 -- 172.21.15.36:0/823664376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce4c0b2cd0 msgr2=0x7fce4c148cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:11.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.223+0000 7fce518ef700 1 --2- 172.21.15.36:0/823664376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce4c0b2cd0 0x7fce4c148cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:11.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.223+0000 7fce518ef700 1 -- 172.21.15.36:0/823664376 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4000b180 con 0x7fce4c0b2200 2022-06-17T12:56:11.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.223+0000 7fce518ef700 1 --2- 172.21.15.36:0/823664376 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce4c0b2200 0x7fce4c1487d0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 rx=0x7fce48001110 tx=0x7fce48007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:11.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.223+0000 7fce3f7fe700 1 -- 172.21.15.36:0/823664376 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce4800ce50 con 0x7fce4c0b2200 2022-06-17T12:56:11.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.223+0000 7fce3f7fe700 1 -- 172.21.15.36:0/823664376 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4800f470 con 0x7fce4c0b2200 2022-06-17T12:56:11.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.223+0000 7fce530f2700 1 -- 172.21.15.36:0/823664376 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4c143970 con 0x7fce4c0b2200 2022-06-17T12:56:11.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.223+0000 7fce3f7fe700 1 -- 172.21.15.36:0/823664376 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce4800f7d0 con 0x7fce4c0b2200 2022-06-17T12:56:11.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.223+0000 7fce530f2700 1 -- 172.21.15.36:0/823664376 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4c142770 con 0x7fce4c0b2200 2022-06-17T12:56:11.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.223+0000 7fce530f2700 1 -- 172.21.15.36:0/823664376 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce4c14b150 con 0x7fce4c0b2200 2022-06-17T12:56:11.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.223+0000 7fce3f7fe700 1 -- 172.21.15.36:0/823664376 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fce48019030 con 0x7fce4c0b2200 2022-06-17T12:56:11.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.227+0000 7fce3f7fe700 1 --2- 172.21.15.36:0/823664376 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce2c053920 0x7fce2c055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:11.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.227+0000 7fce3f7fe700 1 -- 172.21.15.36:0/823664376 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fce4806b520 con 0x7fce4c0b2200 2022-06-17T12:56:11.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.227+0000 7fce520f0700 1 --2- 172.21.15.36:0/823664376 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce2c053920 0x7fce2c055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:11.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.227+0000 7fce520f0700 1 --2- 172.21.15.36:0/823664376 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce2c053920 0x7fce2c055de0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fce4001a000 tx=0x7fce40014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:11.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.231+0000 7fce3f7fe700 1 -- 172.21.15.36:0/823664376 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fce4c14b150 con 0x7fce4c0b2200 2022-06-17T12:56:11.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.351+0000 7fce530f2700 1 -- 172.21.15.36:0/823664376 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce4c14b150 con 0x7fce4c0b2200 2022-06-17T12:56:11.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.351+0000 7fce3f7fe700 1 -- 172.21.15.36:0/823664376 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+16298 (secure 0 0 0) 0x7fce4c14b150 con 0x7fce4c0b2200 2022-06-17T12:56:11.580 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:11.580 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":130,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:56:11.021555+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:56:08.924639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-06-17T12:56:09.989204+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:56:11.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.355+0000 7fce530f2700 1 -- 172.21.15.36:0/823664376 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce2c053920 msgr2=0x7fce2c055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:11.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.355+0000 7fce530f2700 1 --2- 172.21.15.36:0/823664376 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce2c053920 0x7fce2c055de0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fce4001a000 tx=0x7fce40014000).stop 2022-06-17T12:56:11.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.355+0000 7fce530f2700 1 -- 172.21.15.36:0/823664376 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce4c0b2200 msgr2=0x7fce4c1487d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:11.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.355+0000 7fce530f2700 1 --2- 172.21.15.36:0/823664376 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce4c0b2200 0x7fce4c1487d0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 rx=0x7fce48001110 tx=0x7fce48007ae0).stop 2022-06-17T12:56:11.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.355+0000 7fce530f2700 1 -- 172.21.15.36:0/823664376 shutdown_connections 2022-06-17T12:56:11.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.355+0000 7fce530f2700 1 --2- 172.21.15.36:0/823664376 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce2c053920 0x7fce2c055de0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:11.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.355+0000 7fce530f2700 1 --2- 172.21.15.36:0/823664376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce4c0b2cd0 0x7fce4c148cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:11.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.355+0000 7fce530f2700 1 --2- 172.21.15.36:0/823664376 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fce4c0b2200 0x7fce4c1487d0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:11.602 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.355+0000 7fce530f2700 1 --2- 172.21.15.36:0/823664376 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce4c0af8d0 0x7fce4c1431e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:11.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.355+0000 7fce530f2700 1 -- 172.21.15.36:0/823664376 >> 172.21.15.36:0/823664376 conn(0x7fce4c05f770 msgr2=0x7fce4c1459e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:11.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.355+0000 7fce530f2700 1 -- 172.21.15.36:0/823664376 shutdown_connections 2022-06-17T12:56:11.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.355+0000 7fce530f2700 1 -- 172.21.15.36:0/823664376 wait complete. 2022-06-17T12:56:11.604 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:56:11.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.743+0000 7f4593fff700 1 Processor -- start 2022-06-17T12:56:11.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.743+0000 7f4593fff700 1 -- start start 2022-06-17T12:56:11.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.743+0000 7f4593fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45940b1280 0x7f459413fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:11.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.743+0000 7f4593fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4594140410 0x7f45941436d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:11.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.743+0000 7f4593fff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45940b1660 con 0x7f45940abe30 2022-06-17T12:56:11.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.743+0000 7f4593fff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45940aef10 con 0x7f4594140410 2022-06-17T12:56:11.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.743+0000 7f4593fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4594140080 con 0x7f45940b1280 2022-06-17T12:56:11.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.743+0000 7f45927fc700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f45940abe30 0x7f4594062ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56784/0 (socket says 172.21.15.36:56784) 2022-06-17T12:56:11.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.743+0000 7f45927fc700 1 -- 172.21.15.36:0/3354358918 learned_addr learned my addr 172.21.15.36:0/3354358918 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:11.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.743+0000 7f4592ffd700 1 --2- 172.21.15.36:0/3354358918 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45940b1280 0x7f459413fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:11.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.743+0000 7f45937fe700 1 --2- 172.21.15.36:0/3354358918 >> v2:172.21.15.70:3300/0 conn(0x7f4594140410 0x7f45941436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:11.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.743+0000 7f4591ffb700 1 -- 172.21.15.36:0/3354358918 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3180733361 0 0) 0x7f45940b1660 con 0x7f45940abe30 2022-06-17T12:56:11.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.743+0000 7f4591ffb700 1 -- 172.21.15.36:0/3354358918 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4578003260 con 0x7f45940abe30 2022-06-17T12:56:11.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.747+0000 7f45937fe700 1 -- 172.21.15.36:0/3354358918 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45940b1280 msgr2=0x7f459413fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:11.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.747+0000 7f45937fe700 1 --2- 172.21.15.36:0/3354358918 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45940b1280 0x7f459413fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:11.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.747+0000 7f45937fe700 1 -- 172.21.15.36:0/3354358918 >> v1:172.21.15.36:6789/0 conn(0x7f45940abe30 legacy=0x7f4594062ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:11.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.747+0000 7f4591ffb700 1 -- 172.21.15.36:0/3354358918 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3999484539 0 0) 0x7f4578003260 con 0x7f45940abe30 2022-06-17T12:56:11.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.747+0000 7f45937fe700 1 -- 172.21.15.36:0/3354358918 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45940ac3b0 con 0x7f4594140410 2022-06-17T12:56:11.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.747+0000 7f4592ffd700 1 --2- 172.21.15.36:0/3354358918 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45940b1280 0x7f459413fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:11.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.747+0000 7f45937fe700 1 --2- 172.21.15.36:0/3354358918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4594140410 0x7f45941436d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f4588001800 tx=0x7f458c0091d0).ready entity=mon.1 client_cookie=43ca602c27b79cda server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:11.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.747+0000 7f4591ffb700 1 -- 172.21.15.36:0/3354358918 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4588001660 con 0x7f4594140410 2022-06-17T12:56:11.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.747+0000 7f4591ffb700 1 -- 172.21.15.36:0/3354358918 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458c00f510 con 0x7f4594140410 2022-06-17T12:56:11.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.747+0000 7f4591ffb700 1 -- 172.21.15.36:0/3354358918 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f458c00f680 con 0x7f4594140410 2022-06-17T12:56:11.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.747+0000 7f4593fff700 1 -- 172.21.15.36:0/3354358918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4594140410 msgr2=0x7f45941436d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:11.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.747+0000 7f4593fff700 1 --2- 172.21.15.36:0/3354358918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4594140410 0x7f45941436d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f4588001800 tx=0x7f458c0091d0).stop 2022-06-17T12:56:11.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.747+0000 7f4593fff700 1 -- 172.21.15.36:0/3354358918 shutdown_connections 2022-06-17T12:56:11.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.747+0000 7f4593fff700 1 --2- 172.21.15.36:0/3354358918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4594140410 0x7f45941436d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:11.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.747+0000 7f4593fff700 1 --2- 172.21.15.36:0/3354358918 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45940b1280 0x7f459413fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:11.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.747+0000 7f4593fff700 1 -- 172.21.15.36:0/3354358918 >> 172.21.15.36:0/3354358918 conn(0x7f459405f770 msgr2=0x7f459405fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:11.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.747+0000 7f4593fff700 1 -- 172.21.15.36:0/3354358918 shutdown_connections 2022-06-17T12:56:11.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.747+0000 7f4593fff700 1 -- 172.21.15.36:0/3354358918 wait complete. 2022-06-17T12:56:11.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.751+0000 7f4593fff700 1 Processor -- start 2022-06-17T12:56:11.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.751+0000 7f4593fff700 1 -- start start 2022-06-17T12:56:11.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.751+0000 7f4593fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45940abe30 0x7f4594139b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:11.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.751+0000 7f4593fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f45940b1280 0x7f459413c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:11.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.751+0000 7f4593fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4594140410 0x7f4594153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:11.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.751+0000 7f4593fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45940b3e60 con 0x7f45940b1280 2022-06-17T12:56:11.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.751+0000 7f4593fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45940b41d0 con 0x7f45940abe30 2022-06-17T12:56:11.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.751+0000 7f4593fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45940b4470 con 0x7f4594140410 2022-06-17T12:56:11.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.751+0000 7f45927fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f45940b1280 0x7f459413c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:11.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.751+0000 7f45927fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f45940b1280 0x7f459413c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39262/0 (socket says 172.21.15.36:39262) 2022-06-17T12:56:11.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.751+0000 7f45927fc700 1 -- 172.21.15.36:0/1415391892 learned_addr learned my addr 172.21.15.36:0/1415391892 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:11.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.751+0000 7f4592ffd700 1 --2- 172.21.15.36:0/1415391892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45940abe30 0x7f4594139b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:11.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.751+0000 7f45937fe700 1 --2- 172.21.15.36:0/1415391892 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4594140410 0x7f4594153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:11.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.751+0000 7f45927fc700 1 -- 172.21.15.36:0/1415391892 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4594140410 msgr2=0x7f4594153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:11.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.751+0000 7f45927fc700 1 --2- 172.21.15.36:0/1415391892 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4594140410 0x7f4594153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:11.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.751+0000 7f45927fc700 1 -- 172.21.15.36:0/1415391892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45940abe30 msgr2=0x7f4594139b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:11.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.751+0000 7f45927fc700 1 --2- 172.21.15.36:0/1415391892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45940abe30 0x7f4594139b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:11.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.751+0000 7f45927fc700 1 -- 172.21.15.36:0/1415391892 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458400b180 con 0x7f45940b1280 2022-06-17T12:56:11.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.751+0000 7f45927fc700 1 --2- 172.21.15.36:0/1415391892 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f45940b1280 0x7f459413c740 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 rx=0x7f45880018c0 tx=0x7f45880011a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:11.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.751+0000 7f4583fff700 1 -- 172.21.15.36:0/1415391892 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4588007170 con 0x7f45940b1280 2022-06-17T12:56:11.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.755+0000 7f4583fff700 1 -- 172.21.15.36:0/1415391892 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45880072d0 con 0x7f45940b1280 2022-06-17T12:56:11.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.755+0000 7f4583fff700 1 -- 172.21.15.36:0/1415391892 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4588004920 con 0x7f45940b1280 2022-06-17T12:56:11.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.755+0000 7f4593fff700 1 -- 172.21.15.36:0/1415391892 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458c009470 con 0x7f45940b1280 2022-06-17T12:56:11.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.755+0000 7f4593fff700 1 -- 172.21.15.36:0/1415391892 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f459412f4f0 con 0x7f45940b1280 2022-06-17T12:56:11.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.755+0000 7f4593fff700 1 -- 172.21.15.36:0/1415391892 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45940b4630 con 0x7f45940b1280 2022-06-17T12:56:11.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.755+0000 7f4583fff700 1 -- 172.21.15.36:0/1415391892 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f4588004a80 con 0x7f45940b1280 2022-06-17T12:56:11.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.755+0000 7f4583fff700 1 --2- 172.21.15.36:0/1415391892 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4570053af0 0x7f4570055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:11.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.755+0000 7f4592ffd700 1 --2- 172.21.15.36:0/1415391892 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4570053af0 0x7f4570055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:11.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.755+0000 7f4583fff700 1 -- 172.21.15.36:0/1415391892 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f458806bc50 con 0x7f45940b1280 2022-06-17T12:56:11.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.759+0000 7f4592ffd700 1 --2- 172.21.15.36:0/1415391892 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4570053af0 0x7f4570055fb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f458400b000 tx=0x7f4584006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:11.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.763+0000 7f4583fff700 1 -- 172.21.15.36:0/1415391892 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4588042940 con 0x7f45940b1280 2022-06-17T12:56:11.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.891+0000 7f4593fff700 1 -- 172.21.15.36:0/1415391892 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45940ac650 con 0x7f45940b1280 2022-06-17T12:56:11.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.895+0000 7f4583fff700 1 -- 172.21.15.36:0/1415391892 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f4588045f60 con 0x7f45940b1280 2022-06-17T12:56:11.889 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:11.892 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":130,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:56:11.021555+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:56:08.924639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-06-17T12:56:09.989204+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:56:11.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.895+0000 7f4593fff700 1 -- 172.21.15.36:0/1415391892 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4570053af0 msgr2=0x7f4570055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:11.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.895+0000 7f4593fff700 1 --2- 172.21.15.36:0/1415391892 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4570053af0 0x7f4570055fb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f458400b000 tx=0x7f4584006df0).stop 2022-06-17T12:56:11.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.895+0000 7f4593fff700 1 -- 172.21.15.36:0/1415391892 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f45940b1280 msgr2=0x7f459413c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:11.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.895+0000 7f4593fff700 1 --2- 172.21.15.36:0/1415391892 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f45940b1280 0x7f459413c740 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 rx=0x7f45880018c0 tx=0x7f45880011a0).stop 2022-06-17T12:56:11.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.895+0000 7f4593fff700 1 -- 172.21.15.36:0/1415391892 shutdown_connections 2022-06-17T12:56:11.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.895+0000 7f4593fff700 1 --2- 172.21.15.36:0/1415391892 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4570053af0 0x7f4570055fb0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:11.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.895+0000 7f4593fff700 1 --2- 172.21.15.36:0/1415391892 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4594140410 0x7f4594153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:11.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.895+0000 7f4593fff700 1 --2- 172.21.15.36:0/1415391892 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f45940b1280 0x7f459413c740 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:11.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.895+0000 7f4593fff700 1 --2- 172.21.15.36:0/1415391892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45940abe30 0x7f4594139b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:11.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.895+0000 7f4593fff700 1 -- 172.21.15.36:0/1415391892 >> 172.21.15.36:0/1415391892 conn(0x7f459405f770 msgr2=0x7f45940ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:11.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.899+0000 7f4593fff700 1 -- 172.21.15.36:0/1415391892 shutdown_connections 2022-06-17T12:56:11.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:11.899+0000 7f4593fff700 1 -- 172.21.15.36:0/1415391892 wait complete. 2022-06-17T12:56:11.915 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-17T12:56:12.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.063+0000 7fc482962700 1 Processor -- start 2022-06-17T12:56:12.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.063+0000 7fc482962700 1 -- start start 2022-06-17T12:56:12.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.063+0000 7fc482962700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc47c0b4190 0x7fc47c0b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:12.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.063+0000 7fc482962700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc47c143d40 0x7fc47c1471d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:12.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.063+0000 7fc482962700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc47c143130 con 0x7fc47c062270 2022-06-17T12:56:12.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.063+0000 7fc482962700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc47c0b6080 con 0x7fc47c0b4190 2022-06-17T12:56:12.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.063+0000 7fc482962700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c062650 con 0x7fc47c143d40 2022-06-17T12:56:12.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.063+0000 7fc47bfff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fc47c062270 0x7fc47c0b48b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56798/0 (socket says 172.21.15.36:56798) 2022-06-17T12:56:12.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.063+0000 7fc47bfff700 1 -- 172.21.15.36:0/607967159 learned_addr learned my addr 172.21.15.36:0/607967159 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:12.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.063+0000 7fc47b7fe700 1 --2- 172.21.15.36:0/607967159 >> v2:172.21.15.70:3300/0 conn(0x7fc47c0b4190 0x7fc47c0b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:12.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.063+0000 7fc480eff700 1 --2- 172.21.15.36:0/607967159 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc47c143d40 0x7fc47c1471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:12.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.063+0000 7fc47affd700 1 -- 172.21.15.36:0/607967159 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2386263674 0 0) 0x7fc47c143130 con 0x7fc47c062270 2022-06-17T12:56:12.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47affd700 1 -- 172.21.15.36:0/607967159 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc45c003260 con 0x7fc47c062270 2022-06-17T12:56:12.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47affd700 1 -- 172.21.15.36:0/607967159 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1560935760 0 0) 0x7fc45c003260 con 0x7fc47c062270 2022-06-17T12:56:12.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47affd700 1 -- 172.21.15.36:0/607967159 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc47c143d40 msgr2=0x7fc47c1471d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:12.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47affd700 1 --2- 172.21.15.36:0/607967159 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc47c143d40 0x7fc47c1471d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:12.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47affd700 1 -- 172.21.15.36:0/607967159 >> v2:172.21.15.70:3300/0 conn(0x7fc47c0b4190 msgr2=0x7fc47c0b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:12.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47affd700 1 --2- 172.21.15.36:0/607967159 >> v2:172.21.15.70:3300/0 conn(0x7fc47c0b4190 0x7fc47c0b4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:12.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47affd700 1 -- 172.21.15.36:0/607967159 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47c143360 con 0x7fc47c062270 2022-06-17T12:56:12.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47affd700 1 -- 172.21.15.36:0/607967159 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7fc46c001680 con 0x7fc47c062270 2022-06-17T12:56:12.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47affd700 1 -- 172.21.15.36:0/607967159 >> v1:172.21.15.36:6789/0 conn(0x7fc47c062270 legacy=0x7fc47c0b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:12.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47affd700 1 --2- 172.21.15.36:0/607967159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc45c0037d0 0x7fc45c003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:12.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47affd700 1 --2- 172.21.15.36:0/607967159 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc45c004500 0x7fc45c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:12.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47affd700 1 --2- 172.21.15.36:0/607967159 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc47c143d40 0x7fc45c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:12.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47affd700 1 -- 172.21.15.36:0/607967159 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c0b6080 con 0x7fc45c004500 2022-06-17T12:56:12.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47affd700 1 -- 172.21.15.36:0/607967159 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c062650 con 0x7fc45c0037d0 2022-06-17T12:56:12.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47affd700 1 -- 172.21.15.36:0/607967159 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c00b750 con 0x7fc47c143d40 2022-06-17T12:56:12.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47b7fe700 1 --2- 172.21.15.36:0/607967159 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc45c004500 0x7fc45c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:12.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47bfff700 1 --2- 172.21.15.36:0/607967159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc45c0037d0 0x7fc45c003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:12.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc480eff700 1 --2- 172.21.15.36:0/607967159 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc47c143d40 0x7fc45c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:12.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47b7fe700 1 -- 172.21.15.36:0/607967159 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc47c143d40 msgr2=0x7fc45c004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:12.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47b7fe700 1 --2- 172.21.15.36:0/607967159 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc47c143d40 0x7fc45c004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:12.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47b7fe700 1 -- 172.21.15.36:0/607967159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc45c0037d0 msgr2=0x7fc45c003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:12.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47b7fe700 1 --2- 172.21.15.36:0/607967159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc45c0037d0 0x7fc45c003bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:12.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47b7fe700 1 -- 172.21.15.36:0/607967159 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45c003260 con 0x7fc45c004500 2022-06-17T12:56:12.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47bfff700 1 --2- 172.21.15.36:0/607967159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc45c0037d0 0x7fc45c003bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:12.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47b7fe700 1 --2- 172.21.15.36:0/607967159 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc45c004500 0x7fc45c004980 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7fc46400c000 tx=0x7fc464006df0).ready entity=mon.0 client_cookie=2ff4ad6111737587 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:12.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47affd700 1 -- 172.21.15.36:0/607967159 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc46400e830 con 0x7fc45c004500 2022-06-17T12:56:12.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47affd700 1 -- 172.21.15.36:0/607967159 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46400e990 con 0x7fc45c004500 2022-06-17T12:56:12.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc47affd700 1 -- 172.21.15.36:0/607967159 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc46400ecf0 con 0x7fc45c004500 2022-06-17T12:56:12.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc482962700 1 -- 172.21.15.36:0/607967159 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc45c004500 msgr2=0x7fc45c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:12.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc482962700 1 --2- 172.21.15.36:0/607967159 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc45c004500 0x7fc45c004980 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7fc46400c000 tx=0x7fc464006df0).stop 2022-06-17T12:56:12.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc480eff700 1 -- 172.21.15.36:0/607967159 reap_dead start 2022-06-17T12:56:12.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc482962700 1 -- 172.21.15.36:0/607967159 shutdown_connections 2022-06-17T12:56:12.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc482962700 1 -- 172.21.15.36:0/607967159 >> 172.21.15.36:0/607967159 conn(0x7fc47c0b97f0 msgr2=0x7fc47c0b9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:12.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc482962700 1 -- 172.21.15.36:0/607967159 shutdown_connections 2022-06-17T12:56:12.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.067+0000 7fc482962700 1 -- 172.21.15.36:0/607967159 wait complete. 2022-06-17T12:56:12.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.071+0000 7fc482962700 1 Processor -- start 2022-06-17T12:56:12.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.071+0000 7fc482962700 1 -- start start 2022-06-17T12:56:12.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.071+0000 7fc482962700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc45c004500 0x7fc47c0b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:12.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.071+0000 7fc482962700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc47c12f8f0 0x7fc47c152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:12.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.071+0000 7fc482962700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc47c12cd90 0x7fc47c12d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:12.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.071+0000 7fc482962700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c130790 con 0x7fc47c12f8f0 2022-06-17T12:56:12.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.071+0000 7fc482962700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c1527a0 con 0x7fc45c004500 2022-06-17T12:56:12.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.071+0000 7fc482962700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c152a40 con 0x7fc47c12cd90 2022-06-17T12:56:12.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.071+0000 7fc47b7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc47c12f8f0 0x7fc47c152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:12.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.071+0000 7fc47bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc45c004500 0x7fc47c0b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:12.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.071+0000 7fc480eff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc47c12cd90 0x7fc47c12d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:12.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.071+0000 7fc47b7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc47c12f8f0 0x7fc47c152ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39282/0 (socket says 172.21.15.36:39282) 2022-06-17T12:56:12.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.071+0000 7fc47bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc45c004500 0x7fc47c0b32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:59662/0 (socket says 172.21.15.36:59662) 2022-06-17T12:56:12.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.071+0000 7fc480eff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc47c12cd90 0x7fc47c12d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:54074/0 (socket says 172.21.15.36:54074) 2022-06-17T12:56:12.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.071+0000 7fc47b7fe700 1 -- 172.21.15.36:0/599177385 learned_addr learned my addr 172.21.15.36:0/599177385 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:12.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.071+0000 7fc47b7fe700 1 -- 172.21.15.36:0/599177385 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc47c12cd90 msgr2=0x7fc47c12d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:12.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.071+0000 7fc47b7fe700 1 --2- 172.21.15.36:0/599177385 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc47c12cd90 0x7fc47c12d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:12.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.071+0000 7fc47b7fe700 1 -- 172.21.15.36:0/599177385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc45c004500 msgr2=0x7fc47c0b32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:12.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.071+0000 7fc47b7fe700 1 --2- 172.21.15.36:0/599177385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc45c004500 0x7fc47c0b32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:12.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.071+0000 7fc47b7fe700 1 -- 172.21.15.36:0/599177385 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47c0b3a70 con 0x7fc47c12f8f0 2022-06-17T12:56:12.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.071+0000 7fc480eff700 1 --2- 172.21.15.36:0/599177385 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc47c12cd90 0x7fc47c12d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:12.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.071+0000 7fc47bfff700 1 --2- 172.21.15.36:0/599177385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc45c004500 0x7fc47c0b32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:12.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.075+0000 7fc47b7fe700 1 --2- 172.21.15.36:0/599177385 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc47c12f8f0 0x7fc47c152ee0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7fc46400c000 tx=0x7fc464019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:12.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.075+0000 7fc4797fa700 1 -- 172.21.15.36:0/599177385 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc46401b820 con 0x7fc47c12f8f0 2022-06-17T12:56:12.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.075+0000 7fc4797fa700 1 -- 172.21.15.36:0/599177385 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc464000e90 con 0x7fc47c12f8f0 2022-06-17T12:56:12.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.075+0000 7fc482962700 1 -- 172.21.15.36:0/599177385 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc47c1533d0 con 0x7fc47c12f8f0 2022-06-17T12:56:12.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.075+0000 7fc4797fa700 1 -- 172.21.15.36:0/599177385 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc46401bb70 con 0x7fc47c12f8f0 2022-06-17T12:56:12.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.075+0000 7fc482962700 1 -- 172.21.15.36:0/599177385 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc47c12d770 con 0x7fc47c12f8f0 2022-06-17T12:56:12.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.075+0000 7fc4797fa700 1 -- 172.21.15.36:0/599177385 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fc46401bcd0 con 0x7fc47c12f8f0 2022-06-17T12:56:12.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.075+0000 7fc482962700 1 -- 172.21.15.36:0/599177385 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc47c152c00 con 0x7fc47c12f8f0 2022-06-17T12:56:12.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.075+0000 7fc4797fa700 1 --2- 172.21.15.36:0/599177385 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc4600539b0 0x7fc460055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:12.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.075+0000 7fc4797fa700 1 -- 172.21.15.36:0/599177385 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc46406d350 con 0x7fc47c12f8f0 2022-06-17T12:56:12.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.075+0000 7fc47bfff700 1 --2- 172.21.15.36:0/599177385 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc4600539b0 0x7fc460055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:12.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.079+0000 7fc47bfff700 1 --2- 172.21.15.36:0/599177385 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc4600539b0 0x7fc460055e70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fc46c005000 tx=0x7fc47c0ae860).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:12.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.083+0000 7fc4797fa700 1 -- 172.21.15.36:0/599177385 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc4640440b0 con 0x7fc47c12f8f0 2022-06-17T12:56:12.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:12.251+0000 7fc482962700 1 -- 172.21.15.36:0/599177385 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc47c143600 con 0x7fc47c12f8f0 2022-06-17T12:56:13.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.035+0000 7fc4797fa700 1 -- 172.21.15.36:0/599177385 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v72) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc4640476d0 con 0x7fc47c12f8f0 2022-06-17T12:56:13.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.039+0000 7fc482962700 1 -- 172.21.15.36:0/599177385 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc4600539b0 msgr2=0x7fc460055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:13.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.039+0000 7fc482962700 1 --2- 172.21.15.36:0/599177385 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc4600539b0 0x7fc460055e70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fc46c005000 tx=0x7fc47c0ae860).stop 2022-06-17T12:56:13.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.039+0000 7fc482962700 1 -- 172.21.15.36:0/599177385 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc47c12f8f0 msgr2=0x7fc47c152ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:13.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.039+0000 7fc482962700 1 --2- 172.21.15.36:0/599177385 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc47c12f8f0 0x7fc47c152ee0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7fc46400c000 tx=0x7fc464019000).stop 2022-06-17T12:56:13.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.039+0000 7fc482962700 1 -- 172.21.15.36:0/599177385 shutdown_connections 2022-06-17T12:56:13.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.039+0000 7fc482962700 1 --2- 172.21.15.36:0/599177385 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc4600539b0 0x7fc460055e70 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.039+0000 7fc482962700 1 --2- 172.21.15.36:0/599177385 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc47c12cd90 0x7fc47c12d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.039+0000 7fc482962700 1 --2- 172.21.15.36:0/599177385 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc47c12f8f0 0x7fc47c152ee0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.039+0000 7fc482962700 1 --2- 172.21.15.36:0/599177385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc45c004500 0x7fc47c0b32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.039+0000 7fc482962700 1 -- 172.21.15.36:0/599177385 >> 172.21.15.36:0/599177385 conn(0x7fc47c0b97f0 msgr2=0x7fc47c0ae8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:13.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.039+0000 7fc482962700 1 -- 172.21.15.36:0/599177385 shutdown_connections 2022-06-17T12:56:13.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.039+0000 7fc482962700 1 -- 172.21.15.36:0/599177385 wait complete. 2022-06-17T12:56:13.070 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:56:13.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.211+0000 7f2715a61700 1 Processor -- start 2022-06-17T12:56:13.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.211+0000 7f2715a61700 1 -- start start 2022-06-17T12:56:13.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.211+0000 7f2715a61700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2710062b50 0x7f27100ada40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:13.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.211+0000 7f2715a61700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27100b0ba0 con 0x7f2710144c00 2022-06-17T12:56:13.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.211+0000 7f2715a61700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2710147e90 con 0x7f2710062770 2022-06-17T12:56:13.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.211+0000 7f2715a61700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27100ae820 con 0x7f2710062b50 2022-06-17T12:56:13.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.211+0000 7f270e7fc700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f2710144c00 0x7f27100ad330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56820/0 (socket says 172.21.15.36:56820) 2022-06-17T12:56:13.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.211+0000 7f270e7fc700 1 -- 172.21.15.36:0/1105154358 learned_addr learned my addr 172.21.15.36:0/1105154358 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:13.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.211+0000 7f270f7fe700 1 --2- 172.21.15.36:0/1105154358 >> v2:172.21.15.36:3300/0 conn(0x7f2710062b50 0x7f27100ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:13.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.211+0000 7f270dffb700 1 -- 172.21.15.36:0/1105154358 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1691765858 0 0) 0x7f27100b0ba0 con 0x7f2710144c00 2022-06-17T12:56:13.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.211+0000 7f270dffb700 1 -- 172.21.15.36:0/1105154358 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26f0003260 con 0x7f2710144c00 2022-06-17T12:56:13.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.211+0000 7f270dffb700 1 -- 172.21.15.36:0/1105154358 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592244900 0 0) 0x7f2710147e90 con 0x7f2710062770 2022-06-17T12:56:13.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.211+0000 7f270dffb700 1 -- 172.21.15.36:0/1105154358 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27100b0ba0 con 0x7f2710062770 2022-06-17T12:56:13.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.211+0000 7f270dffb700 1 -- 172.21.15.36:0/1105154358 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 635812068 0 0) 0x7f26f0003260 con 0x7f2710144c00 2022-06-17T12:56:13.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.211+0000 7f270f7fe700 1 -- 172.21.15.36:0/1105154358 >> v1:172.21.15.70:6789/0 conn(0x7f2710062770 legacy=0x7f27101444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:13.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.211+0000 7f270f7fe700 1 -- 172.21.15.36:0/1105154358 >> v1:172.21.15.36:6789/0 conn(0x7f2710144c00 legacy=0x7f27100ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:13.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.211+0000 7f270f7fe700 1 -- 172.21.15.36:0/1105154358 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27100b0dd0 con 0x7f2710062b50 2022-06-17T12:56:13.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.211+0000 7f270f7fe700 1 --2- 172.21.15.36:0/1105154358 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2710062b50 0x7f27100ada40 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7f26f8001800 tx=0x7f27080091d0).ready entity=mon.0 client_cookie=e4f09971cc152c47 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:13.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.211+0000 7f270dffb700 1 -- 172.21.15.36:0/1105154358 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26f8001660 con 0x7f2710062b50 2022-06-17T12:56:13.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.211+0000 7f270dffb700 1 -- 172.21.15.36:0/1105154358 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270800f510 con 0x7f2710062b50 2022-06-17T12:56:13.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.211+0000 7f270dffb700 1 -- 172.21.15.36:0/1105154358 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f270800f680 con 0x7f2710062b50 2022-06-17T12:56:13.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.211+0000 7f2715a61700 1 -- 172.21.15.36:0/1105154358 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2710062b50 msgr2=0x7f27100ada40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:13.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.211+0000 7f2715a61700 1 --2- 172.21.15.36:0/1105154358 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2710062b50 0x7f27100ada40 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7f26f8001800 tx=0x7f27080091d0).stop 2022-06-17T12:56:13.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.215+0000 7f2715a61700 1 -- 172.21.15.36:0/1105154358 shutdown_connections 2022-06-17T12:56:13.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.215+0000 7f2715a61700 1 --2- 172.21.15.36:0/1105154358 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2710062b50 0x7f27100ada40 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.215+0000 7f2715a61700 1 -- 172.21.15.36:0/1105154358 >> 172.21.15.36:0/1105154358 conn(0x7f27100b53e0 msgr2=0x7f27100b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:13.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.215+0000 7f2715a61700 1 -- 172.21.15.36:0/1105154358 shutdown_connections 2022-06-17T12:56:13.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.215+0000 7f2715a61700 1 -- 172.21.15.36:0/1105154358 wait complete. 2022-06-17T12:56:13.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.215+0000 7f2715a61700 1 Processor -- start 2022-06-17T12:56:13.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.215+0000 7f2715a61700 1 -- start start 2022-06-17T12:56:13.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.215+0000 7f2715a61700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2710062770 0x7f2710156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:13.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.215+0000 7f2715a61700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2710062b50 0x7f2710066380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:13.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.215+0000 7f2715a61700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2710144c00 0x7f27101306b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:13.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.215+0000 7f2715a61700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2710066b90 con 0x7f2710062770 2022-06-17T12:56:13.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.215+0000 7f2715a61700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271014adf0 con 0x7f2710062b50 2022-06-17T12:56:13.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.215+0000 7f2715a61700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271014b090 con 0x7f2710144c00 2022-06-17T12:56:13.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.215+0000 7f270effd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2710062770 0x7f2710156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:13.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.215+0000 7f270effd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2710062770 0x7f2710156690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39294/0 (socket says 172.21.15.36:39294) 2022-06-17T12:56:13.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.215+0000 7f270effd700 1 -- 172.21.15.36:0/2091237801 learned_addr learned my addr 172.21.15.36:0/2091237801 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:13.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.215+0000 7f270e7fc700 1 --2- 172.21.15.36:0/2091237801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2710062b50 0x7f2710066380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:13.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.215+0000 7f270f7fe700 1 --2- 172.21.15.36:0/2091237801 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2710144c00 0x7f27101306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:13.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.215+0000 7f270effd700 1 -- 172.21.15.36:0/2091237801 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2710144c00 msgr2=0x7f27101306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:13.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.215+0000 7f270effd700 1 --2- 172.21.15.36:0/2091237801 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2710144c00 0x7f27101306b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.215+0000 7f270effd700 1 -- 172.21.15.36:0/2091237801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2710062b50 msgr2=0x7f2710066380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:13.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.215+0000 7f270effd700 1 --2- 172.21.15.36:0/2091237801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2710062b50 0x7f2710066380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.215+0000 7f270effd700 1 -- 172.21.15.36:0/2091237801 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2708009470 con 0x7f2710062770 2022-06-17T12:56:13.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.215+0000 7f270effd700 1 --2- 172.21.15.36:0/2091237801 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2710062770 0x7f2710156690 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7f2700001110 tx=0x7f2700007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:13.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.219+0000 7f2707fff700 1 -- 172.21.15.36:0/2091237801 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f270000f470 con 0x7f2710062770 2022-06-17T12:56:13.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.219+0000 7f2707fff700 1 -- 172.21.15.36:0/2091237801 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270000f5d0 con 0x7f2710062770 2022-06-17T12:56:13.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.219+0000 7f2715a61700 1 -- 172.21.15.36:0/2091237801 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2710130ba0 con 0x7f2710062770 2022-06-17T12:56:13.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.219+0000 7f2707fff700 1 -- 172.21.15.36:0/2091237801 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f270000f930 con 0x7f2710062770 2022-06-17T12:56:13.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.219+0000 7f2715a61700 1 -- 172.21.15.36:0/2091237801 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2710156dd0 con 0x7f2710062770 2022-06-17T12:56:13.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.219+0000 7f2707fff700 1 -- 172.21.15.36:0/2091237801 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2700019030 con 0x7f2710062770 2022-06-17T12:56:13.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.219+0000 7f2715a61700 1 -- 172.21.15.36:0/2091237801 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27100b1c20 con 0x7f2710062770 2022-06-17T12:56:13.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.219+0000 7f2707fff700 1 --2- 172.21.15.36:0/2091237801 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26f4053a40 0x7f26f4055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:13.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.219+0000 7f2707fff700 1 -- 172.21.15.36:0/2091237801 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f270006b510 con 0x7f2710062770 2022-06-17T12:56:13.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.219+0000 7f270e7fc700 1 --2- 172.21.15.36:0/2091237801 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26f4053a40 0x7f26f4055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:13.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.219+0000 7f270e7fc700 1 --2- 172.21.15.36:0/2091237801 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26f4053a40 0x7f26f4055f00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f26f800b000 tx=0x7f26f8001840).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:13.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.227+0000 7f2707fff700 1 -- 172.21.15.36:0/2091237801 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2700042310 con 0x7f2710062770 2022-06-17T12:56:13.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.395+0000 7f2715a61700 1 -- 172.21.15.36:0/2091237801 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f271014abe0 con 0x7f2710062770 2022-06-17T12:56:13.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.395+0000 7f2707fff700 1 -- 172.21.15.36:0/2091237801 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f2700045930 con 0x7f2710062770 2022-06-17T12:56:13.406 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:13.407 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":72,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/2357210652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2357210652},{"type":"v1","addr":"172.21.15.36:6835","nonce":2357210652}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:56:11.021928+0000","modified":"2022-06-17T12:56:13.034227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7942":{"gid":7942,"name":"b","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/197726269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":197726269},{"type":"v1","addr":"172.21.15.70:6833","nonce":197726269}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-06-17T12:56:13.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.395+0000 7f2715a61700 1 -- 172.21.15.36:0/2091237801 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26f4053a40 msgr2=0x7f26f4055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:13.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.395+0000 7f2715a61700 1 --2- 172.21.15.36:0/2091237801 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26f4053a40 0x7f26f4055f00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f26f800b000 tx=0x7f26f8001840).stop 2022-06-17T12:56:13.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.395+0000 7f2715a61700 1 -- 172.21.15.36:0/2091237801 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2710062770 msgr2=0x7f2710156690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:13.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.395+0000 7f2715a61700 1 --2- 172.21.15.36:0/2091237801 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2710062770 0x7f2710156690 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7f2700001110 tx=0x7f2700007ae0).stop 2022-06-17T12:56:13.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.399+0000 7f2715a61700 1 -- 172.21.15.36:0/2091237801 shutdown_connections 2022-06-17T12:56:13.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.399+0000 7f2715a61700 1 --2- 172.21.15.36:0/2091237801 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26f4053a40 0x7f26f4055f00 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.399+0000 7f2715a61700 1 --2- 172.21.15.36:0/2091237801 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2710144c00 0x7f27101306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.399+0000 7f2715a61700 1 --2- 172.21.15.36:0/2091237801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2710062b50 0x7f2710066380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.399+0000 7f2715a61700 1 --2- 172.21.15.36:0/2091237801 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2710062770 0x7f2710156690 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.399+0000 7f2715a61700 1 -- 172.21.15.36:0/2091237801 >> 172.21.15.36:0/2091237801 conn(0x7f27100b53e0 msgr2=0x7f27100b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:13.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.399+0000 7f2715a61700 1 -- 172.21.15.36:0/2091237801 shutdown_connections 2022-06-17T12:56:13.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.399+0000 7f2715a61700 1 -- 172.21.15.36:0/2091237801 wait complete. 2022-06-17T12:56:13.446 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 72 2022-06-17T12:56:13.447 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:56:13.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.571+0000 7f9020b08700 1 Processor -- start 2022-06-17T12:56:13.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.571+0000 7f9020b08700 1 -- start start 2022-06-17T12:56:13.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f9020b08700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f901c0b2200 0x7f901c0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:13.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f9020b08700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f901c0af8d0 0x7f901c14a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:13.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f9020b08700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f901c0b25e0 con 0x7f901c0b2cd0 2022-06-17T12:56:13.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f9020b08700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f901c0ad790 con 0x7f901c0b2200 2022-06-17T12:56:13.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f9020b08700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f901c0aef40 con 0x7f901c0af8d0 2022-06-17T12:56:13.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f901bfff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f901c0b2cd0 0x7f901c14adc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56836/0 (socket says 172.21.15.36:56836) 2022-06-17T12:56:13.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f901bfff700 1 -- 172.21.15.36:0/1800956857 learned_addr learned my addr 172.21.15.36:0/1800956857 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:13.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f901b7fe700 1 --2- 172.21.15.36:0/1800956857 >> v2:172.21.15.36:3300/0 conn(0x7f901c0b2200 0x7f901c0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:13.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f901affd700 1 --2- 172.21.15.36:0/1800956857 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f901c0af8d0 0x7f901c14a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:13.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f901a7fc700 1 -- 172.21.15.36:0/1800956857 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2641578483 0 0) 0x7f901c0b25e0 con 0x7f901c0b2cd0 2022-06-17T12:56:13.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f901a7fc700 1 -- 172.21.15.36:0/1800956857 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9008003260 con 0x7f901c0b2cd0 2022-06-17T12:56:13.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f901b7fe700 1 -- 172.21.15.36:0/1800956857 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f901c0af8d0 msgr2=0x7f901c14a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:13.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f901b7fe700 1 --2- 172.21.15.36:0/1800956857 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f901c0af8d0 0x7f901c14a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f901b7fe700 1 -- 172.21.15.36:0/1800956857 >> v1:172.21.15.36:6789/0 conn(0x7f901c0b2cd0 legacy=0x7f901c14adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:13.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f901b7fe700 1 -- 172.21.15.36:0/1800956857 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901c0b1950 con 0x7f901c0b2200 2022-06-17T12:56:13.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f901b7fe700 1 --2- 172.21.15.36:0/1800956857 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f901c0b2200 0x7f901c0af3e0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7f9010002f70 tx=0x7f9010006df0).ready entity=mon.0 client_cookie=b29efb2dfc7fe9b5 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:13.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f901a7fc700 1 -- 172.21.15.36:0/1800956857 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f901000d4a0 con 0x7f901c0b2200 2022-06-17T12:56:13.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f901a7fc700 1 -- 172.21.15.36:0/1800956857 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9010000e90 con 0x7f901c0b2200 2022-06-17T12:56:13.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f901a7fc700 1 -- 172.21.15.36:0/1800956857 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f901000d7f0 con 0x7f901c0b2200 2022-06-17T12:56:13.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f9020b08700 1 -- 172.21.15.36:0/1800956857 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f901c0b2200 msgr2=0x7f901c0af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:13.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f9020b08700 1 --2- 172.21.15.36:0/1800956857 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f901c0b2200 0x7f901c0af3e0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7f9010002f70 tx=0x7f9010006df0).stop 2022-06-17T12:56:13.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f9020b08700 1 -- 172.21.15.36:0/1800956857 shutdown_connections 2022-06-17T12:56:13.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f9020b08700 1 --2- 172.21.15.36:0/1800956857 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f901c0af8d0 0x7f901c14a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f9020b08700 1 --2- 172.21.15.36:0/1800956857 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f901c0b2200 0x7f901c0af3e0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f9020b08700 1 -- 172.21.15.36:0/1800956857 >> 172.21.15.36:0/1800956857 conn(0x7f901c05f770 msgr2=0x7f901c05fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:13.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f9020b08700 1 -- 172.21.15.36:0/1800956857 shutdown_connections 2022-06-17T12:56:13.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.575+0000 7f9020b08700 1 -- 172.21.15.36:0/1800956857 wait complete. 2022-06-17T12:56:13.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f9020b08700 1 Processor -- start 2022-06-17T12:56:13.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f9020b08700 1 -- start start 2022-06-17T12:56:13.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f9020b08700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f901c0af8d0 0x7f901c144430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:13.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f9020b08700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f901c0b2200 0x7f901c14cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:13.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f9020b08700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f901c0b2cd0 0x7f901c15a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:13.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f9020b08700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c142980 con 0x7f901c0af8d0 2022-06-17T12:56:13.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f9020b08700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c0a7520 con 0x7f901c0b2cd0 2022-06-17T12:56:13.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f9020b08700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f901c0a77c0 con 0x7f901c0b2200 2022-06-17T12:56:13.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f901b7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f901c0af8d0 0x7f901c144430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:13.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f901affd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f901c0b2200 0x7f901c14cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:13.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f901b7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f901c0af8d0 0x7f901c144430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39308/0 (socket says 172.21.15.36:39308) 2022-06-17T12:56:13.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f901b7fe700 1 -- 172.21.15.36:0/1068990622 learned_addr learned my addr 172.21.15.36:0/1068990622 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:13.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f901bfff700 1 --2- 172.21.15.36:0/1068990622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f901c0b2cd0 0x7f901c15a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:13.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f901b7fe700 1 -- 172.21.15.36:0/1068990622 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f901c0b2200 msgr2=0x7f901c14cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:13.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f901b7fe700 1 --2- 172.21.15.36:0/1068990622 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f901c0b2200 0x7f901c14cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f901b7fe700 1 -- 172.21.15.36:0/1068990622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f901c0b2cd0 msgr2=0x7f901c15a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:13.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f901b7fe700 1 --2- 172.21.15.36:0/1068990622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f901c0b2cd0 0x7f901c15a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f901b7fe700 1 -- 172.21.15.36:0/1068990622 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901000b180 con 0x7f901c0af8d0 2022-06-17T12:56:13.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f901bfff700 1 --2- 172.21.15.36:0/1068990622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f901c0b2cd0 0x7f901c15a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:13.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f901b7fe700 1 --2- 172.21.15.36:0/1068990622 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f901c0af8d0 0x7f901c144430 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 rx=0x7f9010002f70 tx=0x7f9010014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:13.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f9018ff9700 1 -- 172.21.15.36:0/1068990622 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9010016070 con 0x7f901c0af8d0 2022-06-17T12:56:13.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f9018ff9700 1 -- 172.21.15.36:0/1068990622 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90100138d0 con 0x7f901c0af8d0 2022-06-17T12:56:13.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f9020b08700 1 -- 172.21.15.36:0/1068990622 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901c145c50 con 0x7f901c0af8d0 2022-06-17T12:56:13.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f9018ff9700 1 -- 172.21.15.36:0/1068990622 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9010013c30 con 0x7f901c0af8d0 2022-06-17T12:56:13.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.579+0000 7f9020b08700 1 -- 172.21.15.36:0/1068990622 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901c144920 con 0x7f901c0af8d0 2022-06-17T12:56:13.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.583+0000 7f9020b08700 1 -- 172.21.15.36:0/1068990622 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f901c0ad330 con 0x7f901c0af8d0 2022-06-17T12:56:13.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.583+0000 7f9018ff9700 1 -- 172.21.15.36:0/1068990622 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f901000f030 con 0x7f901c0af8d0 2022-06-17T12:56:13.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.583+0000 7f9018ff9700 1 --2- 172.21.15.36:0/1068990622 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8ffc053a60 0x7f8ffc055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:13.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.583+0000 7f9018ff9700 1 -- 172.21.15.36:0/1068990622 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f901006cd70 con 0x7f901c0af8d0 2022-06-17T12:56:13.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.583+0000 7f901affd700 1 --2- 172.21.15.36:0/1068990622 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8ffc053a60 0x7f8ffc055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:13.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.583+0000 7f901affd700 1 --2- 172.21.15.36:0/1068990622 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8ffc053a60 0x7f8ffc055f20 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f900c00b000 tx=0x7f900c006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:13.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.591+0000 7f9018ff9700 1 -- 172.21.15.36:0/1068990622 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f90100439e0 con 0x7f901c0af8d0 2022-06-17T12:56:13.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.715+0000 7f9020b08700 1 -- 172.21.15.36:0/1068990622 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f901c057410 con 0x7f901c0af8d0 2022-06-17T12:56:13.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.715+0000 7f9018ff9700 1 -- 172.21.15.36:0/1068990622 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f9010023070 con 0x7f901c0af8d0 2022-06-17T12:56:13.727 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:13.728 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":131,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:56:12.025286+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:56:08.924639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-06-17T12:56:09.989204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:56:13.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.719+0000 7f9020b08700 1 -- 172.21.15.36:0/1068990622 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8ffc053a60 msgr2=0x7f8ffc055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:13.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.719+0000 7f9020b08700 1 --2- 172.21.15.36:0/1068990622 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8ffc053a60 0x7f8ffc055f20 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f900c00b000 tx=0x7f900c006df0).stop 2022-06-17T12:56:13.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.719+0000 7f9020b08700 1 -- 172.21.15.36:0/1068990622 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f901c0af8d0 msgr2=0x7f901c144430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:13.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.719+0000 7f9020b08700 1 --2- 172.21.15.36:0/1068990622 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f901c0af8d0 0x7f901c144430 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 rx=0x7f9010002f70 tx=0x7f9010014000).stop 2022-06-17T12:56:13.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.719+0000 7f9020b08700 1 -- 172.21.15.36:0/1068990622 shutdown_connections 2022-06-17T12:56:13.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.719+0000 7f9020b08700 1 --2- 172.21.15.36:0/1068990622 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8ffc053a60 0x7f8ffc055f20 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.719+0000 7f9020b08700 1 --2- 172.21.15.36:0/1068990622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f901c0b2cd0 0x7f901c15a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.719+0000 7f9020b08700 1 --2- 172.21.15.36:0/1068990622 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f901c0b2200 0x7f901c14cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.719+0000 7f9020b08700 1 --2- 172.21.15.36:0/1068990622 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f901c0af8d0 0x7f901c144430 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.719+0000 7f9020b08700 1 -- 172.21.15.36:0/1068990622 >> 172.21.15.36:0/1068990622 conn(0x7f901c05f770 msgr2=0x7f901c05fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:13.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.719+0000 7f9020b08700 1 -- 172.21.15.36:0/1068990622 shutdown_connections 2022-06-17T12:56:13.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.719+0000 7f9020b08700 1 -- 172.21.15.36:0/1068990622 wait complete. 2022-06-17T12:56:13.745 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:56:13.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.875+0000 7ff3eea36700 1 Processor -- start 2022-06-17T12:56:13.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.879+0000 7ff3eea36700 1 -- start start 2022-06-17T12:56:13.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.879+0000 7ff3eea36700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3e813cb70 0x7ff3e8143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:13.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.879+0000 7ff3eea36700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff3e80aef60 0x7ff3e80b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:13.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.879+0000 7ff3eea36700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e80af4b0 con 0x7ff3e80ba2e0 2022-06-17T12:56:13.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.879+0000 7ff3eea36700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3e8142f50 con 0x7ff3e80aef60 2022-06-17T12:56:13.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.879+0000 7ff3eea36700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e81431f0 con 0x7ff3e813cb70 2022-06-17T12:56:13.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.879+0000 7ff3ee235700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7ff3e80ba2e0 0x7ff3e81442a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56848/0 (socket says 172.21.15.36:56848) 2022-06-17T12:56:13.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.879+0000 7ff3ee235700 1 -- 172.21.15.36:0/1000736206 learned_addr learned my addr 172.21.15.36:0/1000736206 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:13.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.879+0000 7ff3eda34700 1 --2- 172.21.15.36:0/1000736206 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3e813cb70 0x7ff3e8143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:13.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.879+0000 7ff3ed233700 1 --2- 172.21.15.36:0/1000736206 >> v2:172.21.15.70:3300/0 conn(0x7ff3e80aef60 0x7ff3e80b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:13.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.879+0000 7ff3eca32700 1 -- 172.21.15.36:0/1000736206 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2967398404 0 0) 0x7ff3e80af4b0 con 0x7ff3e80ba2e0 2022-06-17T12:56:13.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.879+0000 7ff3eca32700 1 -- 172.21.15.36:0/1000736206 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3d4003260 con 0x7ff3e80ba2e0 2022-06-17T12:56:13.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.879+0000 7ff3eda34700 1 -- 172.21.15.36:0/1000736206 >> v2:172.21.15.70:3300/0 conn(0x7ff3e80aef60 msgr2=0x7ff3e80b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:13.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.879+0000 7ff3eda34700 1 --2- 172.21.15.36:0/1000736206 >> v2:172.21.15.70:3300/0 conn(0x7ff3e80aef60 0x7ff3e80b9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.879+0000 7ff3eda34700 1 -- 172.21.15.36:0/1000736206 >> v1:172.21.15.36:6789/0 conn(0x7ff3e80ba2e0 legacy=0x7ff3e81442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:13.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.879+0000 7ff3eda34700 1 -- 172.21.15.36:0/1000736206 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e81449b0 con 0x7ff3e813cb70 2022-06-17T12:56:13.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.879+0000 7ff3ed233700 1 --2- 172.21.15.36:0/1000736206 >> v2:172.21.15.70:3300/0 conn(0x7ff3e80aef60 0x7ff3e80b9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:13.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.879+0000 7ff3eda34700 1 --2- 172.21.15.36:0/1000736206 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3e813cb70 0x7ff3e8143690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7ff3d8002f70 tx=0x7ff3d8006df0).ready entity=mon.2 client_cookie=7371f55c2f80bf41 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:13.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.879+0000 7ff3eca32700 1 -- 172.21.15.36:0/1000736206 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3d800d4a0 con 0x7ff3e813cb70 2022-06-17T12:56:13.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.879+0000 7ff3eca32700 1 -- 172.21.15.36:0/1000736206 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d8000e90 con 0x7ff3e813cb70 2022-06-17T12:56:13.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.879+0000 7ff3eca32700 1 -- 172.21.15.36:0/1000736206 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3d800d7f0 con 0x7ff3e813cb70 2022-06-17T12:56:13.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3eea36700 1 -- 172.21.15.36:0/1000736206 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3e813cb70 msgr2=0x7ff3e8143690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:13.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3eea36700 1 --2- 172.21.15.36:0/1000736206 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3e813cb70 0x7ff3e8143690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7ff3d8002f70 tx=0x7ff3d8006df0).stop 2022-06-17T12:56:13.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3eea36700 1 -- 172.21.15.36:0/1000736206 shutdown_connections 2022-06-17T12:56:13.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3eea36700 1 --2- 172.21.15.36:0/1000736206 >> v2:172.21.15.70:3300/0 conn(0x7ff3e80aef60 0x7ff3e80b9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3eea36700 1 --2- 172.21.15.36:0/1000736206 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3e813cb70 0x7ff3e8143690 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3eea36700 1 -- 172.21.15.36:0/1000736206 >> 172.21.15.36:0/1000736206 conn(0x7ff3e805c550 msgr2=0x7ff3e80b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:13.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3eea36700 1 -- 172.21.15.36:0/1000736206 shutdown_connections 2022-06-17T12:56:13.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3eea36700 1 -- 172.21.15.36:0/1000736206 wait complete. 2022-06-17T12:56:13.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3eea36700 1 Processor -- start 2022-06-17T12:56:13.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3eea36700 1 -- start start 2022-06-17T12:56:13.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3eea36700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e80aef60 0x7ff3e80b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:13.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3eea36700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3e80ba2e0 0x7ff3e813b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:13.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3eea36700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff3e813cb70 0x7ff3e813b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:13.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3eea36700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e813b9f0 con 0x7ff3e813cb70 2022-06-17T12:56:13.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3eea36700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80b26a0 con 0x7ff3e80aef60 2022-06-17T12:56:13.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3eea36700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80b3bf0 con 0x7ff3e80ba2e0 2022-06-17T12:56:13.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3ee235700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff3e813cb70 0x7ff3e813b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:13.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3ed233700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3e80ba2e0 0x7ff3e813b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:13.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3ed233700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3e80ba2e0 0x7ff3e813b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:54114/0 (socket says 172.21.15.36:54114) 2022-06-17T12:56:13.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3ed233700 1 -- 172.21.15.36:0/3984987696 learned_addr learned my addr 172.21.15.36:0/3984987696 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:13.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3eda34700 1 --2- 172.21.15.36:0/3984987696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e80aef60 0x7ff3e80b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:13.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3ee235700 1 -- 172.21.15.36:0/3984987696 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3e80ba2e0 msgr2=0x7ff3e813b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:13.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3ee235700 1 --2- 172.21.15.36:0/3984987696 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3e80ba2e0 0x7ff3e813b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3ee235700 1 -- 172.21.15.36:0/3984987696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e80aef60 msgr2=0x7ff3e80b8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:13.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3ee235700 1 --2- 172.21.15.36:0/3984987696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e80aef60 0x7ff3e80b8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:13.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.883+0000 7ff3ee235700 1 -- 172.21.15.36:0/3984987696 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3dc00b180 con 0x7ff3e813cb70 2022-06-17T12:56:13.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.887+0000 7ff3ee235700 1 --2- 172.21.15.36:0/3984987696 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff3e813cb70 0x7ff3e813b500 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0x7ff3e80b27e0 tx=0x7ff3e0007d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:13.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.887+0000 7ff3e6ffd700 1 -- 172.21.15.36:0/3984987696 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3e0005ba0 con 0x7ff3e813cb70 2022-06-17T12:56:13.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.887+0000 7ff3e6ffd700 1 -- 172.21.15.36:0/3984987696 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e0005d00 con 0x7ff3e813cb70 2022-06-17T12:56:13.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.887+0000 7ff3eea36700 1 -- 172.21.15.36:0/3984987696 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d800b180 con 0x7ff3e813cb70 2022-06-17T12:56:13.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.887+0000 7ff3e6ffd700 1 -- 172.21.15.36:0/3984987696 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3e00072c0 con 0x7ff3e813cb70 2022-06-17T12:56:13.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.887+0000 7ff3eea36700 1 -- 172.21.15.36:0/3984987696 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e81401d0 con 0x7ff3e813cb70 2022-06-17T12:56:13.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.887+0000 7ff3eea36700 1 -- 172.21.15.36:0/3984987696 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e80b2390 con 0x7ff3e813cb70 2022-06-17T12:56:13.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.887+0000 7ff3e6ffd700 1 -- 172.21.15.36:0/3984987696 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff3e001a060 con 0x7ff3e813cb70 2022-06-17T12:56:13.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.887+0000 7ff3e6ffd700 1 --2- 172.21.15.36:0/3984987696 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff3c8053ad0 0x7ff3c8055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:13.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.887+0000 7ff3e6ffd700 1 -- 172.21.15.36:0/3984987696 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff3e006b9b0 con 0x7ff3e813cb70 2022-06-17T12:56:13.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.887+0000 7ff3eda34700 1 --2- 172.21.15.36:0/3984987696 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff3c8053ad0 0x7ff3c8055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:13.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.891+0000 7ff3eda34700 1 --2- 172.21.15.36:0/3984987696 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff3c8053ad0 0x7ff3c8055f90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7ff3d801a000 tx=0x7ff3d8014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:13.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:13.899+0000 7ff3e6ffd700 1 -- 172.21.15.36:0/3984987696 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff3e00427b0 con 0x7ff3e813cb70 2022-06-17T12:56:14.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.067+0000 7ff3eea36700 1 -- 172.21.15.36:0/3984987696 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7ff3e8057410 con 0x7ff3e813cb70 2022-06-17T12:56:14.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.075+0000 7ff3e6ffd700 1 -- 172.21.15.36:0/3984987696 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v41) v1 ==== 184+0+0 (secure 0 0 0) 0x7ff3e0018070 con 0x7ff3e813cb70 2022-06-17T12:56:14.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.075+0000 7ff3eea36700 1 -- 172.21.15.36:0/3984987696 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff3c8053ad0 msgr2=0x7ff3c8055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:14.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.075+0000 7ff3eea36700 1 --2- 172.21.15.36:0/3984987696 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff3c8053ad0 0x7ff3c8055f90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7ff3d801a000 tx=0x7ff3d8014000).stop 2022-06-17T12:56:14.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.075+0000 7ff3eea36700 1 -- 172.21.15.36:0/3984987696 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff3e813cb70 msgr2=0x7ff3e813b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:14.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.075+0000 7ff3eea36700 1 --2- 172.21.15.36:0/3984987696 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff3e813cb70 0x7ff3e813b500 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0x7ff3e80b27e0 tx=0x7ff3e0007d40).stop 2022-06-17T12:56:14.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.075+0000 7ff3eea36700 1 -- 172.21.15.36:0/3984987696 shutdown_connections 2022-06-17T12:56:14.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.075+0000 7ff3eea36700 1 --2- 172.21.15.36:0/3984987696 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff3c8053ad0 0x7ff3c8055f90 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:14.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.075+0000 7ff3eea36700 1 --2- 172.21.15.36:0/3984987696 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff3e813cb70 0x7ff3e813b500 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:14.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.075+0000 7ff3eea36700 1 --2- 172.21.15.36:0/3984987696 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3e80ba2e0 0x7ff3e813b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:14.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.075+0000 7ff3eea36700 1 --2- 172.21.15.36:0/3984987696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e80aef60 0x7ff3e80b8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:14.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.075+0000 7ff3eea36700 1 -- 172.21.15.36:0/3984987696 >> 172.21.15.36:0/3984987696 conn(0x7ff3e805c550 msgr2=0x7ff3e8145880 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:14.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.079+0000 7ff3eea36700 1 -- 172.21.15.36:0/3984987696 shutdown_connections 2022-06-17T12:56:14.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.079+0000 7ff3eea36700 1 -- 172.21.15.36:0/3984987696 wait complete. 2022-06-17T12:56:14.095 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.0 2022-06-17T12:56:14.100 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:56:14.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.279+0000 7fceae885700 1 Processor -- start 2022-06-17T12:56:14.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.279+0000 7fceae885700 1 -- start start 2022-06-17T12:56:14.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.279+0000 7fceae885700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea8063730 0x7fcea8065e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:14.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.279+0000 7fceae885700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcea80b2900 0x7fcea813eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:14.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.279+0000 7fceae885700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea8063b10 con 0x7fcea80b1930 2022-06-17T12:56:14.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.279+0000 7fceae885700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea8065430 con 0x7fcea80b2900 2022-06-17T12:56:14.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.279+0000 7fceae885700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea80656d0 con 0x7fcea8063730 2022-06-17T12:56:14.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.279+0000 7fcead082700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fcea80b1930 0x7fcea81431f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56862/0 (socket says 172.21.15.36:56862) 2022-06-17T12:56:14.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.279+0000 7fcead082700 1 -- 172.21.15.36:0/2524477252 learned_addr learned my addr 172.21.15.36:0/2524477252 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:14.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.279+0000 7fceae084700 1 --2- 172.21.15.36:0/2524477252 >> v2:172.21.15.70:3300/0 conn(0x7fcea80b2900 0x7fcea813eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:14.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.279+0000 7fcead883700 1 --2- 172.21.15.36:0/2524477252 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea8063730 0x7fcea8065e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:14.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.279+0000 7fceac881700 1 -- 172.21.15.36:0/2524477252 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335237681 0 0) 0x7fcea8063b10 con 0x7fcea80b1930 2022-06-17T12:56:14.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.279+0000 7fceac881700 1 -- 172.21.15.36:0/2524477252 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce90003260 con 0x7fcea80b1930 2022-06-17T12:56:14.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.279+0000 7fceae084700 1 -- 172.21.15.36:0/2524477252 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea8063730 msgr2=0x7fcea8065e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:14.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.279+0000 7fceac881700 1 -- 172.21.15.36:0/2524477252 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3347998155 0 0) 0x7fce90003260 con 0x7fcea80b1930 2022-06-17T12:56:14.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.279+0000 7fceae084700 1 --2- 172.21.15.36:0/2524477252 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea8063730 0x7fcea8065e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:14.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.279+0000 7fceae084700 1 -- 172.21.15.36:0/2524477252 >> v1:172.21.15.36:6789/0 conn(0x7fcea80b1930 legacy=0x7fcea81431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:14.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.279+0000 7fceae084700 1 -- 172.21.15.36:0/2524477252 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea8065970 con 0x7fcea80b2900 2022-06-17T12:56:14.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.279+0000 7fcead883700 1 --2- 172.21.15.36:0/2524477252 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea8063730 0x7fcea8065e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:14.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.279+0000 7fceae084700 1 --2- 172.21.15.36:0/2524477252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea80b2900 0x7fcea813eae0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7fcea4001800 tx=0x7fcea00091d0).ready entity=mon.1 client_cookie=893bfb1adcd7bf1e server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:14.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.283+0000 7fceac881700 1 -- 172.21.15.36:0/2524477252 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcea4001660 con 0x7fcea80b2900 2022-06-17T12:56:14.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.283+0000 7fceac881700 1 -- 172.21.15.36:0/2524477252 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea000f510 con 0x7fcea80b2900 2022-06-17T12:56:14.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.283+0000 7fceac881700 1 -- 172.21.15.36:0/2524477252 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcea000f680 con 0x7fcea80b2900 2022-06-17T12:56:14.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.283+0000 7fceae885700 1 -- 172.21.15.36:0/2524477252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea80b2900 msgr2=0x7fcea813eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:14.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.283+0000 7fceae885700 1 --2- 172.21.15.36:0/2524477252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea80b2900 0x7fcea813eae0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7fcea4001800 tx=0x7fcea00091d0).stop 2022-06-17T12:56:14.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.283+0000 7fceae885700 1 -- 172.21.15.36:0/2524477252 shutdown_connections 2022-06-17T12:56:14.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.283+0000 7fceae885700 1 --2- 172.21.15.36:0/2524477252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea80b2900 0x7fcea813eae0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:14.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.283+0000 7fceae885700 1 --2- 172.21.15.36:0/2524477252 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea8063730 0x7fcea8065e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:14.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.283+0000 7fceae885700 1 -- 172.21.15.36:0/2524477252 >> 172.21.15.36:0/2524477252 conn(0x7fcea80614b0 msgr2=0x7fcea80618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:14.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.283+0000 7fceae885700 1 -- 172.21.15.36:0/2524477252 shutdown_connections 2022-06-17T12:56:14.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.283+0000 7fceae885700 1 -- 172.21.15.36:0/2524477252 wait complete. 2022-06-17T12:56:14.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.283+0000 7fceae885700 1 Processor -- start 2022-06-17T12:56:14.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.283+0000 7fceae885700 1 -- start start 2022-06-17T12:56:14.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.287+0000 7fceae885700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcea8063730 0x7fcea805f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:14.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.287+0000 7fceae885700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea80b1930 0x7fcea8152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:14.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.287+0000 7fceae885700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea80b2900 0x7fcea8152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:14.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.287+0000 7fceae885700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea80b01d0 con 0x7fcea8063730 2022-06-17T12:56:14.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.287+0000 7fceae885700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea8117020 con 0x7fcea80b2900 2022-06-17T12:56:14.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.287+0000 7fceae885700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea81172c0 con 0x7fcea80b1930 2022-06-17T12:56:14.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.287+0000 7fcead883700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcea8063730 0x7fcea805f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:14.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.287+0000 7fcead883700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcea8063730 0x7fcea805f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39336/0 (socket says 172.21.15.36:39336) 2022-06-17T12:56:14.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.287+0000 7fcead883700 1 -- 172.21.15.36:0/1001577762 learned_addr learned my addr 172.21.15.36:0/1001577762 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:14.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.287+0000 7fceae084700 1 --2- 172.21.15.36:0/1001577762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea80b2900 0x7fcea8152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:14.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.287+0000 7fcead082700 1 --2- 172.21.15.36:0/1001577762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea80b1930 0x7fcea8152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:14.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.287+0000 7fcead883700 1 -- 172.21.15.36:0/1001577762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea80b1930 msgr2=0x7fcea8152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:14.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.287+0000 7fcead883700 1 --2- 172.21.15.36:0/1001577762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea80b1930 0x7fcea8152590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:14.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.287+0000 7fcead883700 1 -- 172.21.15.36:0/1001577762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea80b2900 msgr2=0x7fcea8152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:14.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.287+0000 7fcead883700 1 --2- 172.21.15.36:0/1001577762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea80b2900 0x7fcea8152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:14.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.287+0000 7fcead883700 1 -- 172.21.15.36:0/1001577762 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9800b180 con 0x7fcea8063730 2022-06-17T12:56:14.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.287+0000 7fcead883700 1 --2- 172.21.15.36:0/1001577762 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcea8063730 0x7fcea805f910 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7fce9800c000 tx=0x7fce98006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:14.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.287+0000 7fce96ffd700 1 -- 172.21.15.36:0/1001577762 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce9800c7e0 con 0x7fcea8063730 2022-06-17T12:56:14.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.287+0000 7fce96ffd700 1 -- 172.21.15.36:0/1001577762 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9800c940 con 0x7fcea8063730 2022-06-17T12:56:14.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.287+0000 7fce96ffd700 1 -- 172.21.15.36:0/1001577762 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce9800cca0 con 0x7fcea8063730 2022-06-17T12:56:14.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.287+0000 7fceae885700 1 -- 172.21.15.36:0/1001577762 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea0009470 con 0x7fcea8063730 2022-06-17T12:56:14.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.287+0000 7fceae885700 1 -- 172.21.15.36:0/1001577762 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea8152ff0 con 0x7fcea8063730 2022-06-17T12:56:14.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.291+0000 7fceae885700 1 -- 172.21.15.36:0/1001577762 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea80b3b30 con 0x7fcea8063730 2022-06-17T12:56:14.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.291+0000 7fce96ffd700 1 -- 172.21.15.36:0/1001577762 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fce9801c460 con 0x7fcea8063730 2022-06-17T12:56:14.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.291+0000 7fce96ffd700 1 --2- 172.21.15.36:0/1001577762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce88053a60 0x7fce88055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:14.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.291+0000 7fce96ffd700 1 -- 172.21.15.36:0/1001577762 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fce9806c560 con 0x7fcea8063730 2022-06-17T12:56:14.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.291+0000 7fcead082700 1 --2- 172.21.15.36:0/1001577762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce88053a60 0x7fce88055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:14.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.291+0000 7fcead082700 1 --2- 172.21.15.36:0/1001577762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce88053a60 0x7fce88055f20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7fcea400d000 tx=0x7fcea4001180).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:14.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.299+0000 7fce96ffd700 1 -- 172.21.15.36:0/1001577762 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fce98043300 con 0x7fcea8063730 2022-06-17T12:56:14.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.475+0000 7fceae885700 1 -- 172.21.15.36:0/1001577762 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fcea8152270 con 0x7fcea8063730 2022-06-17T12:56:14.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.483+0000 7fce96ffd700 1 -- 172.21.15.36:0/1001577762 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v42) v1 ==== 184+0+0 (secure 0 0 0) 0x7fce9800ce00 con 0x7fcea8063730 2022-06-17T12:56:14.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.487+0000 7fceae885700 1 -- 172.21.15.36:0/1001577762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce88053a60 msgr2=0x7fce88055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:14.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.487+0000 7fceae885700 1 --2- 172.21.15.36:0/1001577762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce88053a60 0x7fce88055f20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7fcea400d000 tx=0x7fcea4001180).stop 2022-06-17T12:56:14.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.487+0000 7fceae885700 1 -- 172.21.15.36:0/1001577762 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcea8063730 msgr2=0x7fcea805f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:14.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.487+0000 7fceae885700 1 --2- 172.21.15.36:0/1001577762 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcea8063730 0x7fcea805f910 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7fce9800c000 tx=0x7fce98006df0).stop 2022-06-17T12:56:14.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.487+0000 7fceae885700 1 -- 172.21.15.36:0/1001577762 shutdown_connections 2022-06-17T12:56:14.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.487+0000 7fceae885700 1 --2- 172.21.15.36:0/1001577762 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce88053a60 0x7fce88055f20 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:14.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.487+0000 7fceae885700 1 --2- 172.21.15.36:0/1001577762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea80b2900 0x7fcea8152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:14.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.487+0000 7fceae885700 1 --2- 172.21.15.36:0/1001577762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea80b1930 0x7fcea8152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:14.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.487+0000 7fceae885700 1 --2- 172.21.15.36:0/1001577762 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcea8063730 0x7fcea805f910 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:14.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.487+0000 7fceae885700 1 -- 172.21.15.36:0/1001577762 >> 172.21.15.36:0/1001577762 conn(0x7fcea80614b0 msgr2=0x7fcea80b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:14.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.487+0000 7fceae885700 1 -- 172.21.15.36:0/1001577762 shutdown_connections 2022-06-17T12:56:14.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.487+0000 7fceae885700 1 -- 172.21.15.36:0/1001577762 wait complete. 2022-06-17T12:56:14.492 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.1 2022-06-17T12:56:14.494 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:56:14.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.647+0000 7f88a0786700 1 Processor -- start 2022-06-17T12:56:14.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.647+0000 7f88a0786700 1 -- start start 2022-06-17T12:56:14.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.647+0000 7f88a0786700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f88980ae270 0x7f88980b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:14.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.647+0000 7f88a0786700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f889813fde0 0x7f8898140200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:14.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.647+0000 7f88a0786700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88980afc30 con 0x7f88980ad9c0 2022-06-17T12:56:14.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.647+0000 7f88a0786700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88980b0cb0 con 0x7f889813fde0 2022-06-17T12:56:14.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.647+0000 7f88a0786700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88980b3eb0 con 0x7f88980ae270 2022-06-17T12:56:14.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.647+0000 7f889ed23700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f889813fde0 0x7f8898140200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:14.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.647+0000 7f889ed23700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f889813fde0 0x7f8898140200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39346/0 (socket says 172.21.15.36:39346) 2022-06-17T12:56:14.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.647+0000 7f889ed23700 1 -- 172.21.15.36:0/1223874090 learned_addr learned my addr 172.21.15.36:0/1223874090 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:14.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.647+0000 7f889e522700 1 --2- 172.21.15.36:0/1223874090 >> v2:172.21.15.70:3300/0 conn(0x7f88980ae270 0x7f88980b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:14.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.647+0000 7f889ed23700 1 -- 172.21.15.36:0/1223874090 >> v2:172.21.15.70:3300/0 conn(0x7f88980ae270 msgr2=0x7f88980b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:14.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.647+0000 7f889ed23700 1 --2- 172.21.15.36:0/1223874090 >> v2:172.21.15.70:3300/0 conn(0x7f88980ae270 0x7f88980b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:14.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.647+0000 7f889ed23700 1 -- 172.21.15.36:0/1223874090 >> v1:172.21.15.36:6789/0 conn(0x7f88980ad9c0 legacy=0x7f88980b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:14.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.647+0000 7f889ed23700 1 -- 172.21.15.36:0/1223874090 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88980ae650 con 0x7f889813fde0 2022-06-17T12:56:14.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.647+0000 7f889ed23700 1 --2- 172.21.15.36:0/1223874090 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f889813fde0 0x7f8898140200 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 rx=0x7f88980b3ff0 tx=0x7f8890009290).ready entity=mon.0 client_cookie=58b24652dd862e92 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:14.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.647+0000 7f889d520700 1 -- 172.21.15.36:0/1223874090 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f889000ba20 con 0x7f889813fde0 2022-06-17T12:56:14.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.647+0000 7f889d520700 1 -- 172.21.15.36:0/1223874090 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889000bb80 con 0x7f889813fde0 2022-06-17T12:56:14.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.647+0000 7f889d520700 1 -- 172.21.15.36:0/1223874090 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8890010470 con 0x7f889813fde0 2022-06-17T12:56:14.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f88a0786700 1 -- 172.21.15.36:0/1223874090 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f889813fde0 msgr2=0x7f8898140200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:14.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f88a0786700 1 --2- 172.21.15.36:0/1223874090 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f889813fde0 0x7f8898140200 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 rx=0x7f88980b3ff0 tx=0x7f8890009290).stop 2022-06-17T12:56:14.652 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f88a0786700 1 -- 172.21.15.36:0/1223874090 shutdown_connections 2022-06-17T12:56:14.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f88a0786700 1 --2- 172.21.15.36:0/1223874090 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f889813fde0 0x7f8898140200 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:14.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f88a0786700 1 --2- 172.21.15.36:0/1223874090 >> v2:172.21.15.70:3300/0 conn(0x7f88980ae270 0x7f88980b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:14.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f88a0786700 1 -- 172.21.15.36:0/1223874090 >> 172.21.15.36:0/1223874090 conn(0x7f889805c550 msgr2=0x7f889805c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:14.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f88a0786700 1 -- 172.21.15.36:0/1223874090 shutdown_connections 2022-06-17T12:56:14.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f88a0786700 1 -- 172.21.15.36:0/1223874090 wait complete. 2022-06-17T12:56:14.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f88a0786700 1 Processor -- start 2022-06-17T12:56:14.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f88a0786700 1 -- start start 2022-06-17T12:56:14.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f88a0786700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88980ad9c0 0x7f889813b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:14.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f88a0786700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88980ae270 0x7f88980650f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:14.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f88a0786700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f889813fde0 0x7f889813a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:14.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f88a0786700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88980655e0 con 0x7f88980ad9c0 2022-06-17T12:56:14.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f88a0786700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8898065860 con 0x7f88980ae270 2022-06-17T12:56:14.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f88a0786700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8898065b00 con 0x7f889813fde0 2022-06-17T12:56:14.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f889e522700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88980ad9c0 0x7f889813b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:14.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f889e522700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88980ad9c0 0x7f889813b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39350/0 (socket says 172.21.15.36:39350) 2022-06-17T12:56:14.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f889e522700 1 -- 172.21.15.36:0/3413194304 learned_addr learned my addr 172.21.15.36:0/3413194304 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:14.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f889ed23700 1 --2- 172.21.15.36:0/3413194304 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f889813fde0 0x7f889813a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:14.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f889dd21700 1 --2- 172.21.15.36:0/3413194304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88980ae270 0x7f88980650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:14.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f889e522700 1 -- 172.21.15.36:0/3413194304 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f889813fde0 msgr2=0x7f889813a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:14.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f889e522700 1 --2- 172.21.15.36:0/3413194304 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f889813fde0 0x7f889813a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:14.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f889e522700 1 -- 172.21.15.36:0/3413194304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88980ae270 msgr2=0x7f88980650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:14.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f889e522700 1 --2- 172.21.15.36:0/3413194304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88980ae270 0x7f88980650f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:14.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f889e522700 1 -- 172.21.15.36:0/3413194304 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8890009530 con 0x7f88980ad9c0 2022-06-17T12:56:14.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.651+0000 7f889e522700 1 --2- 172.21.15.36:0/3413194304 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88980ad9c0 0x7f889813b5a0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0x7f888c0030b0 tx=0x7f888c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:14.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.655+0000 7f888b7fe700 1 -- 172.21.15.36:0/3413194304 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f888c011780 con 0x7f88980ad9c0 2022-06-17T12:56:14.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.655+0000 7f888b7fe700 1 -- 172.21.15.36:0/3413194304 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888c0118e0 con 0x7f88980ad9c0 2022-06-17T12:56:14.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.655+0000 7f88a0786700 1 -- 172.21.15.36:0/3413194304 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8898065da0 con 0x7f88980ad9c0 2022-06-17T12:56:14.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.655+0000 7f888b7fe700 1 -- 172.21.15.36:0/3413194304 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f888c011c40 con 0x7f88980ad9c0 2022-06-17T12:56:14.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.655+0000 7f88a0786700 1 -- 172.21.15.36:0/3413194304 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88981423e0 con 0x7f88980ad9c0 2022-06-17T12:56:14.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.655+0000 7f888b7fe700 1 -- 172.21.15.36:0/3413194304 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f888c01c460 con 0x7f88980ad9c0 2022-06-17T12:56:14.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.655+0000 7f88a0786700 1 -- 172.21.15.36:0/3413194304 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88980afe60 con 0x7f88980ad9c0 2022-06-17T12:56:14.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.655+0000 7f888b7fe700 1 --2- 172.21.15.36:0/3413194304 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8878053ad0 0x7f8878055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:14.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.655+0000 7f888b7fe700 1 -- 172.21.15.36:0/3413194304 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f888c06c310 con 0x7f88980ad9c0 2022-06-17T12:56:14.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.655+0000 7f889dd21700 1 --2- 172.21.15.36:0/3413194304 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8878053ad0 0x7f8878055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:14.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.655+0000 7f889dd21700 1 --2- 172.21.15.36:0/3413194304 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8878053ad0 0x7f8878055f90 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f889400b000 tx=0x7f8894009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:14.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.659+0000 7f888b7fe700 1 -- 172.21.15.36:0/3413194304 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f888c042fe0 con 0x7f88980ad9c0 2022-06-17T12:56:14.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.847+0000 7f88a0786700 1 -- 172.21.15.36:0/3413194304 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f8898057410 con 0x7f88980ad9c0 2022-06-17T12:56:14.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.855+0000 7f888b7fe700 1 -- 172.21.15.36:0/3413194304 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v43) v1 ==== 184+0+0 (secure 0 0 0) 0x7f8898057410 con 0x7f88980ad9c0 2022-06-17T12:56:14.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.855+0000 7f88a0786700 1 -- 172.21.15.36:0/3413194304 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8878053ad0 msgr2=0x7f8878055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:14.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.855+0000 7f88a0786700 1 --2- 172.21.15.36:0/3413194304 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8878053ad0 0x7f8878055f90 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f889400b000 tx=0x7f8894009040).stop 2022-06-17T12:56:14.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.855+0000 7f88a0786700 1 -- 172.21.15.36:0/3413194304 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88980ad9c0 msgr2=0x7f889813b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:14.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.855+0000 7f88a0786700 1 --2- 172.21.15.36:0/3413194304 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88980ad9c0 0x7f889813b5a0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0x7f888c0030b0 tx=0x7f888c008040).stop 2022-06-17T12:56:14.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.855+0000 7f88a0786700 1 -- 172.21.15.36:0/3413194304 shutdown_connections 2022-06-17T12:56:14.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.855+0000 7f88a0786700 1 --2- 172.21.15.36:0/3413194304 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8878053ad0 0x7f8878055f90 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:14.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.855+0000 7f88a0786700 1 --2- 172.21.15.36:0/3413194304 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f889813fde0 0x7f889813a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:14.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.855+0000 7f88a0786700 1 --2- 172.21.15.36:0/3413194304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88980ae270 0x7f88980650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:14.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.855+0000 7f88a0786700 1 --2- 172.21.15.36:0/3413194304 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f88980ad9c0 0x7f889813b5a0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:14.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.855+0000 7f88a0786700 1 -- 172.21.15.36:0/3413194304 >> 172.21.15.36:0/3413194304 conn(0x7f889805c550 msgr2=0x7f8898140780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:14.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.855+0000 7f88a0786700 1 -- 172.21.15.36:0/3413194304 shutdown_connections 2022-06-17T12:56:14.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:14.855+0000 7f88a0786700 1 -- 172.21.15.36:0/3413194304 wait complete. 2022-06-17T12:56:14.875 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.2 2022-06-17T12:56:14.884 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:56:15.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.015+0000 7fc425f2f700 1 Processor -- start 2022-06-17T12:56:15.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.015+0000 7fc425f2f700 1 -- start start 2022-06-17T12:56:15.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.015+0000 7fc425f2f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4200aef60 0x7fc4200b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:15.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.015+0000 7fc425f2f700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4200af4b0 con 0x7fc42013cb70 2022-06-17T12:56:15.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.015+0000 7fc425f2f700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc420142f50 con 0x7fc4200ba2e0 2022-06-17T12:56:15.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.015+0000 7fc425f2f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc420143da0 con 0x7fc4200aef60 2022-06-17T12:56:15.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.015+0000 7fc424f2d700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fc42013cb70 0x7fc420143690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56888/0 (socket says 172.21.15.36:56888) 2022-06-17T12:56:15.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.015+0000 7fc424f2d700 1 -- 172.21.15.36:0/2153136330 learned_addr learned my addr 172.21.15.36:0/2153136330 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:15.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.015+0000 7fc41ffff700 1 --2- 172.21.15.36:0/2153136330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4200aef60 0x7fc4200b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:15.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.015+0000 7fc41f7fe700 1 -- 172.21.15.36:0/2153136330 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2979754532 0 0) 0x7fc4200af4b0 con 0x7fc42013cb70 2022-06-17T12:56:15.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.015+0000 7fc41f7fe700 1 -- 172.21.15.36:0/2153136330 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc40c003260 con 0x7fc42013cb70 2022-06-17T12:56:15.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.019+0000 7fc41f7fe700 1 -- 172.21.15.36:0/2153136330 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2177250345 0 0) 0x7fc420142f50 con 0x7fc4200ba2e0 2022-06-17T12:56:15.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.019+0000 7fc41ffff700 1 -- 172.21.15.36:0/2153136330 >> v1:172.21.15.70:6789/0 conn(0x7fc4200ba2e0 legacy=0x7fc4201442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:15.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.019+0000 7fc41ffff700 1 -- 172.21.15.36:0/2153136330 >> v1:172.21.15.36:6789/0 conn(0x7fc42013cb70 legacy=0x7fc420143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:15.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.019+0000 7fc41ffff700 1 -- 172.21.15.36:0/2153136330 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4201431f0 con 0x7fc4200aef60 2022-06-17T12:56:15.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.019+0000 7fc41ffff700 1 --2- 172.21.15.36:0/2153136330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4200aef60 0x7fc4200b9df0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fc410002f70 tx=0x7fc410006df0).ready entity=mon.2 client_cookie=1cd0813c06676e8d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:15.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.019+0000 7fc41f7fe700 1 -- 172.21.15.36:0/2153136330 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc41000d570 con 0x7fc4200aef60 2022-06-17T12:56:15.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.019+0000 7fc41f7fe700 1 -- 172.21.15.36:0/2153136330 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc410000e90 con 0x7fc4200aef60 2022-06-17T12:56:15.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.019+0000 7fc41f7fe700 1 -- 172.21.15.36:0/2153136330 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc41000d8c0 con 0x7fc4200aef60 2022-06-17T12:56:15.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.019+0000 7fc425f2f700 1 -- 172.21.15.36:0/2153136330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4200aef60 msgr2=0x7fc4200b9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:15.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.019+0000 7fc425f2f700 1 --2- 172.21.15.36:0/2153136330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4200aef60 0x7fc4200b9df0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fc410002f70 tx=0x7fc410006df0).stop 2022-06-17T12:56:15.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.019+0000 7fc425f2f700 1 -- 172.21.15.36:0/2153136330 shutdown_connections 2022-06-17T12:56:15.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.019+0000 7fc425f2f700 1 --2- 172.21.15.36:0/2153136330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4200aef60 0x7fc4200b9df0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:15.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.019+0000 7fc425f2f700 1 -- 172.21.15.36:0/2153136330 >> 172.21.15.36:0/2153136330 conn(0x7fc42005c550 msgr2=0x7fc4200b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:15.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.019+0000 7fc425f2f700 1 -- 172.21.15.36:0/2153136330 shutdown_connections 2022-06-17T12:56:15.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.019+0000 7fc425f2f700 1 -- 172.21.15.36:0/2153136330 wait complete. 2022-06-17T12:56:15.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.019+0000 7fc425f2f700 1 Processor -- start 2022-06-17T12:56:15.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.019+0000 7fc425f2f700 1 -- start start 2022-06-17T12:56:15.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.023+0000 7fc425f2f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc4200aef60 0x7fc42013ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:15.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.023+0000 7fc425f2f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4200ba2e0 0x7fc4201451b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:15.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.023+0000 7fc425f2f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc42013cb70 0x7fc420155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:15.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.023+0000 7fc425f2f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc420140050 con 0x7fc4200aef60 2022-06-17T12:56:15.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.023+0000 7fc425f2f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4201402f0 con 0x7fc42013cb70 2022-06-17T12:56:15.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.023+0000 7fc425f2f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc420140590 con 0x7fc4200ba2e0 2022-06-17T12:56:15.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.023+0000 7fc424f2d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc4200aef60 0x7fc42013ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:15.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.023+0000 7fc424f2d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc4200aef60 0x7fc42013ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39364/0 (socket says 172.21.15.36:39364) 2022-06-17T12:56:15.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.023+0000 7fc424f2d700 1 -- 172.21.15.36:0/198997476 learned_addr learned my addr 172.21.15.36:0/198997476 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:15.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.023+0000 7fc41ffff700 1 --2- 172.21.15.36:0/198997476 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4200ba2e0 0x7fc4201451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:15.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.023+0000 7fc42572e700 1 --2- 172.21.15.36:0/198997476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc42013cb70 0x7fc420155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:15.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.023+0000 7fc424f2d700 1 -- 172.21.15.36:0/198997476 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4200ba2e0 msgr2=0x7fc4201451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:15.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.023+0000 7fc424f2d700 1 --2- 172.21.15.36:0/198997476 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4200ba2e0 0x7fc4201451b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:15.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.023+0000 7fc424f2d700 1 -- 172.21.15.36:0/198997476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc42013cb70 msgr2=0x7fc420155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:15.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.023+0000 7fc424f2d700 1 --2- 172.21.15.36:0/198997476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc42013cb70 0x7fc420155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:15.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.023+0000 7fc424f2d700 1 -- 172.21.15.36:0/198997476 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41000b180 con 0x7fc4200aef60 2022-06-17T12:56:15.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.023+0000 7fc424f2d700 1 --2- 172.21.15.36:0/198997476 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc4200aef60 0x7fc42013ef20 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7fc414001110 tx=0x7fc414007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:15.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.023+0000 7fc41dffb700 1 -- 172.21.15.36:0/198997476 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc41400cdd0 con 0x7fc4200aef60 2022-06-17T12:56:15.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.023+0000 7fc41dffb700 1 -- 172.21.15.36:0/198997476 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4140070d0 con 0x7fc4200aef60 2022-06-17T12:56:15.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.023+0000 7fc425f2f700 1 -- 172.21.15.36:0/198997476 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc42013eb40 con 0x7fc4200aef60 2022-06-17T12:56:15.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.023+0000 7fc41dffb700 1 -- 172.21.15.36:0/198997476 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc414007430 con 0x7fc4200aef60 2022-06-17T12:56:15.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.023+0000 7fc425f2f700 1 -- 172.21.15.36:0/198997476 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4201556a0 con 0x7fc4200aef60 2022-06-17T12:56:15.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.023+0000 7fc425f2f700 1 -- 172.21.15.36:0/198997476 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4201449b0 con 0x7fc4200aef60 2022-06-17T12:56:15.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.023+0000 7fc41dffb700 1 -- 172.21.15.36:0/198997476 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fc41401a070 con 0x7fc4200aef60 2022-06-17T12:56:15.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.027+0000 7fc41dffb700 1 --2- 172.21.15.36:0/198997476 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc400053ad0 0x7fc400055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:15.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.027+0000 7fc41dffb700 1 -- 172.21.15.36:0/198997476 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc41406b1b0 con 0x7fc4200aef60 2022-06-17T12:56:15.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.027+0000 7fc41ffff700 1 --2- 172.21.15.36:0/198997476 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc400053ad0 0x7fc400055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:15.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.027+0000 7fc41ffff700 1 --2- 172.21.15.36:0/198997476 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc400053ad0 0x7fc400055f90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7fc410018000 tx=0x7fc410014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:15.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.031+0000 7fc41dffb700 1 -- 172.21.15.36:0/198997476 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc414041f50 con 0x7fc4200aef60 2022-06-17T12:56:15.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.195+0000 7fc425f2f700 1 -- 172.21.15.36:0/198997476 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc420057410 con 0x7fc4200aef60 2022-06-17T12:56:15.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.199+0000 7fc41dffb700 1 -- 172.21.15.36:0/198997476 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+2139 (secure 0 0 0) 0x7fc414045570 con 0x7fc4200aef60 2022-06-17T12:56:15.192 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:15.193 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":72,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/2357210652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2357210652},{"type":"v1","addr":"172.21.15.36:6835","nonce":2357210652}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:56:11.021928+0000","modified":"2022-06-17T12:56:13.034227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7942":{"gid":7942,"name":"b","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/197726269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":197726269},{"type":"v1","addr":"172.21.15.70:6833","nonce":197726269}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-06-17T12:56:15.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.199+0000 7fc425f2f700 1 -- 172.21.15.36:0/198997476 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc400053ad0 msgr2=0x7fc400055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:15.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.199+0000 7fc425f2f700 1 --2- 172.21.15.36:0/198997476 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc400053ad0 0x7fc400055f90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7fc410018000 tx=0x7fc410014000).stop 2022-06-17T12:56:15.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.199+0000 7fc425f2f700 1 -- 172.21.15.36:0/198997476 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc4200aef60 msgr2=0x7fc42013ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:15.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.199+0000 7fc425f2f700 1 --2- 172.21.15.36:0/198997476 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc4200aef60 0x7fc42013ef20 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7fc414001110 tx=0x7fc414007aa0).stop 2022-06-17T12:56:15.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.199+0000 7fc425f2f700 1 -- 172.21.15.36:0/198997476 shutdown_connections 2022-06-17T12:56:15.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.199+0000 7fc425f2f700 1 --2- 172.21.15.36:0/198997476 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc400053ad0 0x7fc400055f90 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:15.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.199+0000 7fc425f2f700 1 --2- 172.21.15.36:0/198997476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc42013cb70 0x7fc420155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:15.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.199+0000 7fc425f2f700 1 --2- 172.21.15.36:0/198997476 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4200ba2e0 0x7fc4201451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:15.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.199+0000 7fc425f2f700 1 --2- 172.21.15.36:0/198997476 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc4200aef60 0x7fc42013ef20 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:15.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.199+0000 7fc425f2f700 1 -- 172.21.15.36:0/198997476 >> 172.21.15.36:0/198997476 conn(0x7fc42005c550 msgr2=0x7fc42013c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:15.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.199+0000 7fc425f2f700 1 -- 172.21.15.36:0/198997476 shutdown_connections 2022-06-17T12:56:15.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.199+0000 7fc425f2f700 1 -- 172.21.15.36:0/198997476 wait complete. 2022-06-17T12:56:15.272 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 72 2022-06-17T12:56:15.477 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:56:11.021928+0000', 'modified': '2022-06-17T12:56:13.034227+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7942}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7942': {'gid': 7942, 'name': 'b', 'rank': 0, 'incarnation': 69, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6833/197726269', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 197726269}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 197726269}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:56:15.478 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:56:15.479 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-17T12:56:15.619 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":7942,"want_state":"up:active","state":"up:active","rank_uptime":4.5938399570000001,"mdsmap_epoch":72,"osdmap_epoch":131,"osdmap_epoch_barrier":131,"uptime":9.5772046930000005} 2022-06-17T12:56:15.627 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 7942, "mdsmap_epoch": 72, "osdmap_epoch": 131, "osdmap_epoch_barrier": 131, "rank_uptime": 4.593839957, "state": "up:active", "uptime": 9.577204693, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:56:15.628 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:56:15.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.767+0000 7f3360fe1700 1 Processor -- start 2022-06-17T12:56:15.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.767+0000 7f3360fe1700 1 -- start start 2022-06-17T12:56:15.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f3360fe1700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f335c062770 0x7f335c062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:15.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f3360fe1700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f335c0b0ba0 con 0x7f335c1444f0 2022-06-17T12:56:15.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f3360fe1700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f335c147e90 con 0x7f335c0ad330 2022-06-17T12:56:15.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f3360fe1700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f335c0ae820 con 0x7f335c062770 2022-06-17T12:56:15.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f3359d9b700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f335c1444f0 0x7f335c144960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56902/0 (socket says 172.21.15.36:56902) 2022-06-17T12:56:15.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f3359d9b700 1 -- 172.21.15.36:0/1137088970 learned_addr learned my addr 172.21.15.36:0/1137088970 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:15.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f335a59c700 1 --2- 172.21.15.36:0/1137088970 >> v2:172.21.15.70:3300/0 conn(0x7f335c062770 0x7f335c062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:15.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f335959a700 1 -- 172.21.15.36:0/1137088970 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1300925099 0 0) 0x7f335c0b0ba0 con 0x7f335c1444f0 2022-06-17T12:56:15.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f335959a700 1 -- 172.21.15.36:0/1137088970 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f333c003260 con 0x7f335c1444f0 2022-06-17T12:56:15.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f335959a700 1 -- 172.21.15.36:0/1137088970 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3504804005 0 0) 0x7f333c003260 con 0x7f335c1444f0 2022-06-17T12:56:15.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f335959a700 1 -- 172.21.15.36:0/1137088970 >> v2:172.21.15.70:3300/0 conn(0x7f335c062770 msgr2=0x7f335c062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:15.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f335959a700 1 --2- 172.21.15.36:0/1137088970 >> v2:172.21.15.70:3300/0 conn(0x7f335c062770 0x7f335c062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:15.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f335959a700 1 -- 172.21.15.36:0/1137088970 >> v1:172.21.15.70:6789/0 conn(0x7f335c0ad330 legacy=0x7f335c0ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:15.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f335959a700 1 -- 172.21.15.36:0/1137088970 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335c0b0dd0 con 0x7f335c1444f0 2022-06-17T12:56:15.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f335959a700 1 -- 172.21.15.36:0/1137088970 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f334c001660 con 0x7f335c1444f0 2022-06-17T12:56:15.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f335959a700 1 -- 172.21.15.36:0/1137088970 >> v1:172.21.15.36:6789/0 conn(0x7f335c1444f0 legacy=0x7f335c144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:15.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f335959a700 1 --2- 172.21.15.36:0/1137088970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f333c003750 0x7f333c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:15.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f335959a700 1 --2- 172.21.15.36:0/1137088970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f333c004500 0x7f333c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:15.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f335959a700 1 --2- 172.21.15.36:0/1137088970 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f333c004e70 0x7f333c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:15.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f335959a700 1 -- 172.21.15.36:0/1137088970 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c0ae820 con 0x7f333c004e70 2022-06-17T12:56:15.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f335959a700 1 -- 172.21.15.36:0/1137088970 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c00bd20 con 0x7f333c003750 2022-06-17T12:56:15.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f335959a700 1 -- 172.21.15.36:0/1137088970 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333c00bee0 con 0x7f333c004500 2022-06-17T12:56:15.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f335ad9d700 1 --2- 172.21.15.36:0/1137088970 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f333c004e70 0x7f333c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:15.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f3359d9b700 1 --2- 172.21.15.36:0/1137088970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f333c004500 0x7f333c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:15.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f335a59c700 1 --2- 172.21.15.36:0/1137088970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f333c003750 0x7f333c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:15.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f3359d9b700 1 -- 172.21.15.36:0/1137088970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f333c003750 msgr2=0x7f333c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:15.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f3359d9b700 1 --2- 172.21.15.36:0/1137088970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f333c003750 0x7f333c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:15.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f3359d9b700 1 -- 172.21.15.36:0/1137088970 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f333c004e70 msgr2=0x7f333c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:15.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f3359d9b700 1 --2- 172.21.15.36:0/1137088970 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f333c004e70 0x7f333c00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:15.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f3359d9b700 1 -- 172.21.15.36:0/1137088970 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c003260 con 0x7f333c004500 2022-06-17T12:56:15.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f335ad9d700 1 -- 172.21.15.36:0/1137088970 reap_dead start 2022-06-17T12:56:15.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f3359d9b700 1 --2- 172.21.15.36:0/1137088970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f333c004500 0x7f333c004980 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f334c001370 tx=0x7f335c0af680).ready entity=mon.2 client_cookie=979d78bde85087fc server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:15.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f335959a700 1 -- 172.21.15.36:0/1137088970 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f334c0070d0 con 0x7f333c004500 2022-06-17T12:56:15.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f335959a700 1 -- 172.21.15.36:0/1137088970 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334c007230 con 0x7f333c004500 2022-06-17T12:56:15.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.771+0000 7f335959a700 1 -- 172.21.15.36:0/1137088970 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f334c00c8f0 con 0x7f333c004500 2022-06-17T12:56:15.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.775+0000 7f3360fe1700 1 -- 172.21.15.36:0/1137088970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f333c004500 msgr2=0x7f333c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:15.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.775+0000 7f3360fe1700 1 --2- 172.21.15.36:0/1137088970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f333c004500 0x7f333c004980 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f334c001370 tx=0x7f335c0af680).stop 2022-06-17T12:56:15.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.775+0000 7f3360fe1700 1 -- 172.21.15.36:0/1137088970 shutdown_connections 2022-06-17T12:56:15.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.775+0000 7f3360fe1700 1 --2- 172.21.15.36:0/1137088970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f333c004500 0x7f333c004980 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:15.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.775+0000 7f3360fe1700 1 -- 172.21.15.36:0/1137088970 >> 172.21.15.36:0/1137088970 conn(0x7f335c0b53e0 msgr2=0x7f335c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:15.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.775+0000 7f3360fe1700 1 -- 172.21.15.36:0/1137088970 shutdown_connections 2022-06-17T12:56:15.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.775+0000 7f3360fe1700 1 -- 172.21.15.36:0/1137088970 wait complete. 2022-06-17T12:56:15.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.775+0000 7f3360fe1700 1 Processor -- start 2022-06-17T12:56:15.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.775+0000 7f3360fe1700 1 -- start start 2022-06-17T12:56:15.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.779+0000 7f3360fe1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f333c004500 0x7f335c0b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:15.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.779+0000 7f3360fe1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f335c064630 0x7f335c156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:15.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.779+0000 7f3360fe1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f335c157230 0x7f335c131be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:15.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.779+0000 7f3360fe1700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c0657c0 con 0x7f335c157230 2022-06-17T12:56:15.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.779+0000 7f3360fe1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c0659f0 con 0x7f333c004500 2022-06-17T12:56:15.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.779+0000 7f3360fe1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335c065c90 con 0x7f335c064630 2022-06-17T12:56:15.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.779+0000 7f335ad9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f335c157230 0x7f335c131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:15.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.779+0000 7f3359d9b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f335c064630 0x7f335c156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:15.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.779+0000 7f3359d9b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f335c064630 0x7f335c156d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:54176/0 (socket says 172.21.15.36:54176) 2022-06-17T12:56:15.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.779+0000 7f335a59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f333c004500 0x7f335c0b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:15.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.779+0000 7f3359d9b700 1 -- 172.21.15.36:0/589382268 learned_addr learned my addr 172.21.15.36:0/589382268 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:15.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.779+0000 7f335ad9d700 1 -- 172.21.15.36:0/589382268 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f335c064630 msgr2=0x7f335c156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:15.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.779+0000 7f335ad9d700 1 --2- 172.21.15.36:0/589382268 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f335c064630 0x7f335c156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:15.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.779+0000 7f335ad9d700 1 -- 172.21.15.36:0/589382268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f333c004500 msgr2=0x7f335c0b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:15.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.779+0000 7f335ad9d700 1 --2- 172.21.15.36:0/589382268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f333c004500 0x7f335c0b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:15.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.779+0000 7f335ad9d700 1 -- 172.21.15.36:0/589382268 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335c146cc0 con 0x7f335c157230 2022-06-17T12:56:15.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.779+0000 7f3359d9b700 1 --2- 172.21.15.36:0/589382268 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f335c064630 0x7f335c156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:15.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.779+0000 7f335a59c700 1 --2- 172.21.15.36:0/589382268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f333c004500 0x7f335c0b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:15.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.779+0000 7f335ad9d700 1 --2- 172.21.15.36:0/589382268 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f335c157230 0x7f335c131be0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 rx=0x7f334c0018c0 tx=0x7f3354007d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:15.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.779+0000 7f33537fe700 1 -- 172.21.15.36:0/589382268 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3354006d60 con 0x7f335c157230 2022-06-17T12:56:15.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.779+0000 7f33537fe700 1 -- 172.21.15.36:0/589382268 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3354007030 con 0x7f335c157230 2022-06-17T12:56:15.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.779+0000 7f3360fe1700 1 -- 172.21.15.36:0/589382268 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f335c1567d0 con 0x7f335c157230 2022-06-17T12:56:15.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.779+0000 7f33537fe700 1 -- 172.21.15.36:0/589382268 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3354007390 con 0x7f335c157230 2022-06-17T12:56:15.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.779+0000 7f3360fe1700 1 -- 172.21.15.36:0/589382268 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f335c066cb0 con 0x7f335c157230 2022-06-17T12:56:15.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.783+0000 7f33537fe700 1 -- 172.21.15.36:0/589382268 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f33540074f0 con 0x7f335c157230 2022-06-17T12:56:15.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.783+0000 7f3360fe1700 1 -- 172.21.15.36:0/589382268 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f335c065e90 con 0x7f335c157230 2022-06-17T12:56:15.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.783+0000 7f33537fe700 1 --2- 172.21.15.36:0/589382268 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3340053ad0 0x7f3340055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:15.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.783+0000 7f33537fe700 1 -- 172.21.15.36:0/589382268 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f335406be50 con 0x7f335c157230 2022-06-17T12:56:15.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.783+0000 7f335a59c700 1 --2- 172.21.15.36:0/589382268 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3340053ad0 0x7f3340055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:15.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.783+0000 7f335a59c700 1 --2- 172.21.15.36:0/589382268 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3340053ad0 0x7f3340055f90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f334400b000 tx=0x7f3344006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:15.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.787+0000 7f33537fe700 1 -- 172.21.15.36:0/589382268 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f335c065e90 con 0x7f335c157230 2022-06-17T12:56:15.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.923+0000 7f3360fe1700 1 -- 172.21.15.36:0/589382268 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f335c065e90 con 0x7f335c157230 2022-06-17T12:56:15.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.923+0000 7f33537fe700 1 -- 172.21.15.36:0/589382268 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f335c065e90 con 0x7f335c157230 2022-06-17T12:56:15.920 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:15.920 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":131,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:56:12.025286+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:56:08.924639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-06-17T12:56:09.989204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:56:15.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.927+0000 7f3360fe1700 1 -- 172.21.15.36:0/589382268 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3340053ad0 msgr2=0x7f3340055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:15.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.927+0000 7f3360fe1700 1 --2- 172.21.15.36:0/589382268 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3340053ad0 0x7f3340055f90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f334400b000 tx=0x7f3344006df0).stop 2022-06-17T12:56:15.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.927+0000 7f3360fe1700 1 -- 172.21.15.36:0/589382268 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f335c157230 msgr2=0x7f335c131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:15.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.927+0000 7f3360fe1700 1 --2- 172.21.15.36:0/589382268 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f335c157230 0x7f335c131be0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 rx=0x7f334c0018c0 tx=0x7f3354007d80).stop 2022-06-17T12:56:15.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.927+0000 7f3360fe1700 1 -- 172.21.15.36:0/589382268 shutdown_connections 2022-06-17T12:56:15.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.927+0000 7f3360fe1700 1 --2- 172.21.15.36:0/589382268 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3340053ad0 0x7f3340055f90 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:15.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.927+0000 7f3360fe1700 1 --2- 172.21.15.36:0/589382268 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f335c157230 0x7f335c131be0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:15.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.927+0000 7f3360fe1700 1 --2- 172.21.15.36:0/589382268 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f335c064630 0x7f335c156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:15.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.927+0000 7f3360fe1700 1 --2- 172.21.15.36:0/589382268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f333c004500 0x7f335c0b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:15.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.927+0000 7f3360fe1700 1 -- 172.21.15.36:0/589382268 >> 172.21.15.36:0/589382268 conn(0x7f335c0b53e0 msgr2=0x7f335c12f860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:15.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.927+0000 7f3360fe1700 1 -- 172.21.15.36:0/589382268 shutdown_connections 2022-06-17T12:56:15.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:15.927+0000 7f3360fe1700 1 -- 172.21.15.36:0/589382268 wait complete. 2022-06-17T12:56:15.926 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:56:08.924639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '130', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:56:09.989204+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:56:15.927 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:56:16.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.063+0000 7f2341e28700 1 Processor -- start 2022-06-17T12:56:16.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.063+0000 7f2341e28700 1 -- start start 2022-06-17T12:56:16.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.063+0000 7f2341e28700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f233c0b0420 0x7f233c05f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:16.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.063+0000 7f2341e28700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f233c05fc60 0x7f233c146750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:16.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.063+0000 7f2341e28700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f233c0b10a0 con 0x7f233c060260 2022-06-17T12:56:16.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.063+0000 7f2341e28700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f233c144140 con 0x7f233c05fc60 2022-06-17T12:56:16.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.063+0000 7f2341e28700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233c143ea0 con 0x7f233c0b0420 2022-06-17T12:56:16.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.063+0000 7f233affd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f233c05fc60 0x7f233c146750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:16.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.067+0000 7f233affd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f233c05fc60 0x7f233c146750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39392/0 (socket says 172.21.15.36:39392) 2022-06-17T12:56:16.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.067+0000 7f233affd700 1 -- 172.21.15.36:0/963993898 learned_addr learned my addr 172.21.15.36:0/963993898 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:16.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.067+0000 7f233b7fe700 1 --2- 172.21.15.36:0/963993898 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f233c0b0420 0x7f233c05f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:16.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.067+0000 7f233affd700 1 -- 172.21.15.36:0/963993898 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f233c0b0420 msgr2=0x7f233c05f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:16.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.067+0000 7f233affd700 1 --2- 172.21.15.36:0/963993898 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f233c0b0420 0x7f233c05f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.067+0000 7f233affd700 1 -- 172.21.15.36:0/963993898 >> v1:172.21.15.70:6789/0 conn(0x7f233c060260 legacy=0x7f233c146c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:16.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.067+0000 7f233affd700 1 -- 172.21.15.36:0/963993898 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233c143bc0 con 0x7f233c05fc60 2022-06-17T12:56:16.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.067+0000 7f233affd700 1 --2- 172.21.15.36:0/963993898 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f233c05fc60 0x7f233c146750 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7f232c002f70 tx=0x7f232c006eb0).ready entity=mon.0 client_cookie=c5bfbcbc47041dac server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:16.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.067+0000 7f233a7fc700 1 -- 172.21.15.36:0/963993898 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f232c00d5c0 con 0x7f233c05fc60 2022-06-17T12:56:16.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.067+0000 7f233a7fc700 1 -- 172.21.15.36:0/963993898 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232c000e90 con 0x7f233c05fc60 2022-06-17T12:56:16.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.067+0000 7f233a7fc700 1 -- 172.21.15.36:0/963993898 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f232c00d950 con 0x7f233c05fc60 2022-06-17T12:56:16.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.067+0000 7f2341e28700 1 -- 172.21.15.36:0/963993898 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f233c05fc60 msgr2=0x7f233c146750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:16.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.067+0000 7f2341e28700 1 --2- 172.21.15.36:0/963993898 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f233c05fc60 0x7f233c146750 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7f232c002f70 tx=0x7f232c006eb0).stop 2022-06-17T12:56:16.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.067+0000 7f2341e28700 1 -- 172.21.15.36:0/963993898 shutdown_connections 2022-06-17T12:56:16.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.067+0000 7f2341e28700 1 --2- 172.21.15.36:0/963993898 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f233c05fc60 0x7f233c146750 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.067+0000 7f2341e28700 1 --2- 172.21.15.36:0/963993898 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f233c0b0420 0x7f233c05f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.067+0000 7f2341e28700 1 -- 172.21.15.36:0/963993898 >> 172.21.15.36:0/963993898 conn(0x7f233c05c550 msgr2=0x7f233c05c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:16.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.067+0000 7f2341e28700 1 -- 172.21.15.36:0/963993898 shutdown_connections 2022-06-17T12:56:16.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.067+0000 7f2341e28700 1 -- 172.21.15.36:0/963993898 wait complete. 2022-06-17T12:56:16.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.067+0000 7f2341e28700 1 Processor -- start 2022-06-17T12:56:16.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.067+0000 7f2341e28700 1 -- start start 2022-06-17T12:56:16.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.071+0000 7f2341e28700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f233c05fc60 0x7f233c13ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:16.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.071+0000 7f2341e28700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f233c060260 0x7f233c147870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:16.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.071+0000 7f2341e28700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f233c0b0420 0x7f233c13c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:16.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.071+0000 7f2341e28700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233c153b80 con 0x7f233c0b0420 2022-06-17T12:56:16.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.071+0000 7f2341e28700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233c153db0 con 0x7f233c060260 2022-06-17T12:56:16.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.071+0000 7f2341e28700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233c146160 con 0x7f233c05fc60 2022-06-17T12:56:16.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.071+0000 7f233bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f233c0b0420 0x7f233c13c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:16.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.071+0000 7f233b7fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f233c05fc60 0x7f233c13ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:16.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.071+0000 7f233bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f233c0b0420 0x7f233c13c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39402/0 (socket says 172.21.15.36:39402) 2022-06-17T12:56:16.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.071+0000 7f233bfff700 1 -- 172.21.15.36:0/770311120 learned_addr learned my addr 172.21.15.36:0/770311120 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:16.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.071+0000 7f233affd700 1 --2- 172.21.15.36:0/770311120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f233c060260 0x7f233c147870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:16.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.071+0000 7f233bfff700 1 -- 172.21.15.36:0/770311120 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f233c05fc60 msgr2=0x7f233c13ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:16.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.071+0000 7f233bfff700 1 --2- 172.21.15.36:0/770311120 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f233c05fc60 0x7f233c13ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.071+0000 7f233bfff700 1 -- 172.21.15.36:0/770311120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f233c060260 msgr2=0x7f233c147870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:16.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.071+0000 7f233bfff700 1 --2- 172.21.15.36:0/770311120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f233c060260 0x7f233c147870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.071+0000 7f233bfff700 1 -- 172.21.15.36:0/770311120 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232c00b1e0 con 0x7f233c0b0420 2022-06-17T12:56:16.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.071+0000 7f233bfff700 1 --2- 172.21.15.36:0/770311120 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f233c0b0420 0x7f233c13c2c0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7f233c0b2d70 tx=0x7f2334007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:16.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.071+0000 7f2338ff9700 1 -- 172.21.15.36:0/770311120 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f233400f470 con 0x7f233c0b0420 2022-06-17T12:56:16.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.071+0000 7f2338ff9700 1 -- 172.21.15.36:0/770311120 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233400f5d0 con 0x7f233c0b0420 2022-06-17T12:56:16.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.071+0000 7f2341e28700 1 -- 172.21.15.36:0/770311120 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f233c147d60 con 0x7f233c0b0420 2022-06-17T12:56:16.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.071+0000 7f2338ff9700 1 -- 172.21.15.36:0/770311120 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f233400f930 con 0x7f233c0b0420 2022-06-17T12:56:16.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.071+0000 7f2341e28700 1 -- 172.21.15.36:0/770311120 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f233c1475e0 con 0x7f233c0b0420 2022-06-17T12:56:16.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.071+0000 7f2341e28700 1 -- 172.21.15.36:0/770311120 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f233c153f70 con 0x7f233c0b0420 2022-06-17T12:56:16.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.075+0000 7f2338ff9700 1 -- 172.21.15.36:0/770311120 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2334019030 con 0x7f233c0b0420 2022-06-17T12:56:16.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.075+0000 7f2338ff9700 1 --2- 172.21.15.36:0/770311120 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f23200539d0 0x7f2320055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:16.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.075+0000 7f2338ff9700 1 -- 172.21.15.36:0/770311120 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f233406b5a0 con 0x7f233c0b0420 2022-06-17T12:56:16.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.075+0000 7f233b7fe700 1 --2- 172.21.15.36:0/770311120 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f23200539d0 0x7f2320055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:16.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.075+0000 7f233b7fe700 1 --2- 172.21.15.36:0/770311120 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f23200539d0 0x7f2320055e90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f232400c000 tx=0x7f2324008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:16.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.083+0000 7f2338ff9700 1 -- 172.21.15.36:0/770311120 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2334042340 con 0x7f233c0b0420 2022-06-17T12:56:16.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.207+0000 7f2341e28700 1 -- 172.21.15.36:0/770311120 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f233c057410 con 0x7f233c0b0420 2022-06-17T12:56:16.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.207+0000 7f2338ff9700 1 -- 172.21.15.36:0/770311120 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v131) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2334045960 con 0x7f233c0b0420 2022-06-17T12:56:16.198 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:56:16.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.211+0000 7f2341e28700 1 -- 172.21.15.36:0/770311120 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f23200539d0 msgr2=0x7f2320055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:16.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.211+0000 7f2341e28700 1 --2- 172.21.15.36:0/770311120 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f23200539d0 0x7f2320055e90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f232400c000 tx=0x7f2324008040).stop 2022-06-17T12:56:16.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.211+0000 7f2341e28700 1 -- 172.21.15.36:0/770311120 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f233c0b0420 msgr2=0x7f233c13c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:16.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.211+0000 7f2341e28700 1 --2- 172.21.15.36:0/770311120 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f233c0b0420 0x7f233c13c2c0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7f233c0b2d70 tx=0x7f2334007f40).stop 2022-06-17T12:56:16.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.211+0000 7f2341e28700 1 -- 172.21.15.36:0/770311120 shutdown_connections 2022-06-17T12:56:16.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.211+0000 7f2341e28700 1 --2- 172.21.15.36:0/770311120 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f23200539d0 0x7f2320055e90 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.211+0000 7f2341e28700 1 --2- 172.21.15.36:0/770311120 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f233c0b0420 0x7f233c13c2c0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.211+0000 7f2341e28700 1 --2- 172.21.15.36:0/770311120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f233c060260 0x7f233c147870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.211+0000 7f2341e28700 1 --2- 172.21.15.36:0/770311120 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f233c05fc60 0x7f233c13ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.211+0000 7f2341e28700 1 -- 172.21.15.36:0/770311120 >> 172.21.15.36:0/770311120 conn(0x7f233c05c550 msgr2=0x7f233c0b42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:16.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.211+0000 7f2341e28700 1 -- 172.21.15.36:0/770311120 shutdown_connections 2022-06-17T12:56:16.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.211+0000 7f2341e28700 1 -- 172.21.15.36:0/770311120 wait complete. 2022-06-17T12:56:16.210 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:56:16.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.335+0000 7f807bfff700 1 Processor -- start 2022-06-17T12:56:16.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.335+0000 7f807bfff700 1 -- start start 2022-06-17T12:56:16.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.335+0000 7f807bfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f807c0b9860 0x7f807c143840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:16.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.335+0000 7f807bfff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f807c0b5dc0 0x7f807c0b6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:16.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.335+0000 7f807bfff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f807c0b4160 con 0x7f807c0b45c0 2022-06-17T12:56:16.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.335+0000 7f807bfff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f807c0b9c40 con 0x7f807c0b5dc0 2022-06-17T12:56:16.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.335+0000 7f807bfff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807c0627d0 con 0x7f807c0b9860 2022-06-17T12:56:16.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.339+0000 7f807b7fe700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f807c0b5dc0 0x7f807c0b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:16.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.339+0000 7f807affd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f807c0b9860 0x7f807c143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:16.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.339+0000 7f807b7fe700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f807c0b5dc0 0x7f807c0b6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:59792/0 (socket says 172.21.15.36:59792) 2022-06-17T12:56:16.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.339+0000 7f80727fc700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f807c0b45c0 0x7f807c0b49e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:52672/0 (socket says 172.21.15.36:52672) 2022-06-17T12:56:16.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.339+0000 7f807b7fe700 1 -- 172.21.15.36:0/1227389403 learned_addr learned my addr 172.21.15.36:0/1227389403 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:16.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.339+0000 7f807b7fe700 1 -- 172.21.15.36:0/1227389403 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f807c0b9860 msgr2=0x7f807c143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:16.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.339+0000 7f807b7fe700 1 --2- 172.21.15.36:0/1227389403 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f807c0b9860 0x7f807c143840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.339+0000 7f807b7fe700 1 -- 172.21.15.36:0/1227389403 >> v1:172.21.15.70:6789/0 conn(0x7f807c0b45c0 legacy=0x7f807c0b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:16.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.339+0000 7f807b7fe700 1 -- 172.21.15.36:0/1227389403 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c062a70 con 0x7f807c0b5dc0 2022-06-17T12:56:16.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.339+0000 7f807b7fe700 1 --2- 172.21.15.36:0/1227389403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f807c0b5dc0 0x7f807c0b6210 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f80740032f0 tx=0x7f8074009290).ready entity=mon.1 client_cookie=5f18a5a499e4b8d7 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:16.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.339+0000 7f807a7fc700 1 -- 172.21.15.36:0/1227389403 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f807400b8a0 con 0x7f807c0b5dc0 2022-06-17T12:56:16.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.339+0000 7f807a7fc700 1 -- 172.21.15.36:0/1227389403 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807400ba00 con 0x7f807c0b5dc0 2022-06-17T12:56:16.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.339+0000 7f807a7fc700 1 -- 172.21.15.36:0/1227389403 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f807400bb70 con 0x7f807c0b5dc0 2022-06-17T12:56:16.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.339+0000 7f807bfff700 1 -- 172.21.15.36:0/1227389403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f807c0b5dc0 msgr2=0x7f807c0b6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:16.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.339+0000 7f807bfff700 1 --2- 172.21.15.36:0/1227389403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f807c0b5dc0 0x7f807c0b6210 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f80740032f0 tx=0x7f8074009290).stop 2022-06-17T12:56:16.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.339+0000 7f807bfff700 1 -- 172.21.15.36:0/1227389403 shutdown_connections 2022-06-17T12:56:16.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.339+0000 7f807bfff700 1 --2- 172.21.15.36:0/1227389403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f807c0b5dc0 0x7f807c0b6210 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.339+0000 7f807bfff700 1 --2- 172.21.15.36:0/1227389403 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f807c0b9860 0x7f807c143840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.339+0000 7f807bfff700 1 -- 172.21.15.36:0/1227389403 >> 172.21.15.36:0/1227389403 conn(0x7f807c05c550 msgr2=0x7f807c060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:16.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.339+0000 7f807bfff700 1 -- 172.21.15.36:0/1227389403 shutdown_connections 2022-06-17T12:56:16.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.339+0000 7f807bfff700 1 -- 172.21.15.36:0/1227389403 wait complete. 2022-06-17T12:56:16.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f807bfff700 1 Processor -- start 2022-06-17T12:56:16.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f807bfff700 1 -- start start 2022-06-17T12:56:16.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f807bfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f807c0b45c0 0x7f807c12fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:16.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f807bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f807c0b5dc0 0x7f807c1434f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:16.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f807bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f807c0b9860 0x7f807c155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:16.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f807bfff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c0afa20 con 0x7f807c0b9860 2022-06-17T12:56:16.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f807bfff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c1559e0 con 0x7f807c0b5dc0 2022-06-17T12:56:16.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f807bfff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807c0af4e0 con 0x7f807c0b45c0 2022-06-17T12:56:16.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f807b7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f807c0b9860 0x7f807c155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:16.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f807b7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f807c0b9860 0x7f807c155e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39416/0 (socket says 172.21.15.36:39416) 2022-06-17T12:56:16.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f807b7fe700 1 -- 172.21.15.36:0/1672787864 learned_addr learned my addr 172.21.15.36:0/1672787864 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:16.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f80727fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f807c0b5dc0 0x7f807c1434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:16.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f807affd700 1 --2- 172.21.15.36:0/1672787864 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f807c0b45c0 0x7f807c12fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:16.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f80727fc700 1 -- 172.21.15.36:0/1672787864 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f807c0b45c0 msgr2=0x7f807c12fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:16.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f80727fc700 1 --2- 172.21.15.36:0/1672787864 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f807c0b45c0 0x7f807c12fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f80727fc700 1 -- 172.21.15.36:0/1672787864 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f807c0b9860 msgr2=0x7f807c155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:16.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f80727fc700 1 --2- 172.21.15.36:0/1672787864 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f807c0b9860 0x7f807c155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f80727fc700 1 -- 172.21.15.36:0/1672787864 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8074009530 con 0x7f807c0b5dc0 2022-06-17T12:56:16.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f807b7fe700 1 --2- 172.21.15.36:0/1672787864 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f807c0b9860 0x7f807c155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:16.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f80727fc700 1 --2- 172.21.15.36:0/1672787864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f807c0b5dc0 0x7f807c1434f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f807c0b3e40 tx=0x7f8064007b80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:16.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f8078ff9700 1 -- 172.21.15.36:0/1672787864 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80640070d0 con 0x7f807c0b5dc0 2022-06-17T12:56:16.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f8078ff9700 1 -- 172.21.15.36:0/1672787864 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8064007230 con 0x7f807c0b5dc0 2022-06-17T12:56:16.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f807bfff700 1 -- 172.21.15.36:0/1672787864 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807c0af780 con 0x7f807c0b5dc0 2022-06-17T12:56:16.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f8078ff9700 1 -- 172.21.15.36:0/1672787864 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8064005550 con 0x7f807c0b5dc0 2022-06-17T12:56:16.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.343+0000 7f807bfff700 1 -- 172.21.15.36:0/1672787864 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f807c0b0120 con 0x7f807c0b5dc0 2022-06-17T12:56:16.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.347+0000 7f807bfff700 1 -- 172.21.15.36:0/1672787864 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f807c155ba0 con 0x7f807c0b5dc0 2022-06-17T12:56:16.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.347+0000 7f8078ff9700 1 -- 172.21.15.36:0/1672787864 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f806401a070 con 0x7f807c0b5dc0 2022-06-17T12:56:16.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.347+0000 7f8078ff9700 1 --2- 172.21.15.36:0/1672787864 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8060053a40 0x7f8060055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:16.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.347+0000 7f8078ff9700 1 -- 172.21.15.36:0/1672787864 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f806406ae80 con 0x7f807c0b5dc0 2022-06-17T12:56:16.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.347+0000 7f807affd700 1 --2- 172.21.15.36:0/1672787864 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8060053a40 0x7f8060055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:16.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.347+0000 7f807affd700 1 --2- 172.21.15.36:0/1672787864 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8060053a40 0x7f8060055f00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f806c00b000 tx=0x7f806c006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:16.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.359+0000 7f8078ff9700 1 -- 172.21.15.36:0/1672787864 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f807c155ba0 con 0x7f807c0b5dc0 2022-06-17T12:56:16.385 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:56:16.388 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:56:16.392 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:56:16.398 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:56:16.401 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:56:16.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.491+0000 7f807bfff700 1 -- 172.21.15.36:0/1672787864 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f807c0b50f0 con 0x7f807c0b5dc0 2022-06-17T12:56:16.483 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:56:16.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.491+0000 7f8078ff9700 1 -- 172.21.15.36:0/1672787864 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v131) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8064041c80 con 0x7f807c0b5dc0 2022-06-17T12:56:16.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.495+0000 7f807bfff700 1 -- 172.21.15.36:0/1672787864 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8060053a40 msgr2=0x7f8060055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:16.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.495+0000 7f807bfff700 1 --2- 172.21.15.36:0/1672787864 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8060053a40 0x7f8060055f00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f806c00b000 tx=0x7f806c006df0).stop 2022-06-17T12:56:16.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.495+0000 7f807bfff700 1 -- 172.21.15.36:0/1672787864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f807c0b5dc0 msgr2=0x7f807c1434f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:16.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.495+0000 7f807bfff700 1 --2- 172.21.15.36:0/1672787864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f807c0b5dc0 0x7f807c1434f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f807c0b3e40 tx=0x7f8064007b80).stop 2022-06-17T12:56:16.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.495+0000 7f807bfff700 1 -- 172.21.15.36:0/1672787864 shutdown_connections 2022-06-17T12:56:16.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.495+0000 7f807bfff700 1 --2- 172.21.15.36:0/1672787864 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8060053a40 0x7f8060055f00 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.495+0000 7f807bfff700 1 --2- 172.21.15.36:0/1672787864 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f807c0b9860 0x7f807c155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.495+0000 7f807bfff700 1 --2- 172.21.15.36:0/1672787864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f807c0b5dc0 0x7f807c1434f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.495+0000 7f807bfff700 1 --2- 172.21.15.36:0/1672787864 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f807c0b45c0 0x7f807c12fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.495+0000 7f807bfff700 1 -- 172.21.15.36:0/1672787864 >> 172.21.15.36:0/1672787864 conn(0x7f807c05c550 msgr2=0x7f807c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:16.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.495+0000 7f807bfff700 1 -- 172.21.15.36:0/1672787864 shutdown_connections 2022-06-17T12:56:16.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.495+0000 7f807bfff700 1 -- 172.21.15.36:0/1672787864 wait complete. 2022-06-17T12:56:16.496 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:56:16.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.699+0000 7fdd7574a700 1 Processor -- start 2022-06-17T12:56:16.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.699+0000 7fdd7574a700 1 -- start start 2022-06-17T12:56:16.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.699+0000 7fdd7574a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdd700b45c0 0x7fdd700b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:16.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.699+0000 7fdd7574a700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdd700b4f00 0x7fdd700b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:16.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.699+0000 7fdd7574a700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd700b4160 con 0x7fdd700b9860 2022-06-17T12:56:16.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.699+0000 7fdd7574a700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd700b9c40 con 0x7fdd700b4f00 2022-06-17T12:56:16.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.699+0000 7fdd7574a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd700627d0 con 0x7fdd700b45c0 2022-06-17T12:56:16.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.699+0000 7fdd6f7fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdd700b45c0 0x7fdd700b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:16.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.699+0000 7fdd74f49700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdd700b4f00 0x7fdd700b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:16.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.699+0000 7fdd6f7fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdd700b45c0 0x7fdd700b4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:54212/0 (socket says 172.21.15.36:54212) 2022-06-17T12:56:16.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.699+0000 7fdd6f7fe700 1 -- 172.21.15.36:0/3093166858 learned_addr learned my addr 172.21.15.36:0/3093166858 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:16.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.703+0000 7fdd6f7fe700 1 -- 172.21.15.36:0/3093166858 >> v2:172.21.15.70:3300/0 conn(0x7fdd700b4f00 msgr2=0x7fdd700b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:16.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.703+0000 7fdd6f7fe700 1 --2- 172.21.15.36:0/3093166858 >> v2:172.21.15.70:3300/0 conn(0x7fdd700b4f00 0x7fdd700b5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.703+0000 7fdd6f7fe700 1 -- 172.21.15.36:0/3093166858 >> v1:172.21.15.70:6789/0 conn(0x7fdd700b9860 legacy=0x7fdd70143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:16.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.703+0000 7fdd6f7fe700 1 -- 172.21.15.36:0/3093166858 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd70062a70 con 0x7fdd700b45c0 2022-06-17T12:56:16.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.703+0000 7fdd6f7fe700 1 --2- 172.21.15.36:0/3093166858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdd700b45c0 0x7fdd700b4a10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7fdd60002f70 tx=0x7fdd60006eb0).ready entity=mon.2 client_cookie=f2d126e7b761c245 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:16.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.703+0000 7fdd6effd700 1 -- 172.21.15.36:0/3093166858 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd6000d570 con 0x7fdd700b45c0 2022-06-17T12:56:16.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.703+0000 7fdd6effd700 1 -- 172.21.15.36:0/3093166858 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd60000e90 con 0x7fdd700b45c0 2022-06-17T12:56:16.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.703+0000 7fdd6effd700 1 -- 172.21.15.36:0/3093166858 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd6000d900 con 0x7fdd700b45c0 2022-06-17T12:56:16.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.703+0000 7fdd7574a700 1 -- 172.21.15.36:0/3093166858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdd700b45c0 msgr2=0x7fdd700b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:16.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.703+0000 7fdd7574a700 1 --2- 172.21.15.36:0/3093166858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdd700b45c0 0x7fdd700b4a10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7fdd60002f70 tx=0x7fdd60006eb0).stop 2022-06-17T12:56:16.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.703+0000 7fdd7574a700 1 -- 172.21.15.36:0/3093166858 shutdown_connections 2022-06-17T12:56:16.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.703+0000 7fdd7574a700 1 --2- 172.21.15.36:0/3093166858 >> v2:172.21.15.70:3300/0 conn(0x7fdd700b4f00 0x7fdd700b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.703+0000 7fdd7574a700 1 --2- 172.21.15.36:0/3093166858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdd700b45c0 0x7fdd700b4a10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.703+0000 7fdd7574a700 1 -- 172.21.15.36:0/3093166858 >> 172.21.15.36:0/3093166858 conn(0x7fdd7005c550 msgr2=0x7fdd70060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:16.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.703+0000 7fdd7574a700 1 -- 172.21.15.36:0/3093166858 shutdown_connections 2022-06-17T12:56:16.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.703+0000 7fdd7574a700 1 -- 172.21.15.36:0/3093166858 wait complete. 2022-06-17T12:56:16.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.703+0000 7fdd7574a700 1 Processor -- start 2022-06-17T12:56:16.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.703+0000 7fdd7574a700 1 -- start start 2022-06-17T12:56:16.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd7574a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdd700b45c0 0x7fdd701418a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:16.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd7574a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd700b4f00 0x7fdd700ad330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:16.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd7574a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdd700b9860 0x7fdd700ad820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:16.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd7574a700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd70146340 con 0x7fdd700b45c0 2022-06-17T12:56:16.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd7574a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd701465e0 con 0x7fdd700b4f00 2022-06-17T12:56:16.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd7574a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd70146880 con 0x7fdd700b9860 2022-06-17T12:56:16.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd6ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdd700b45c0 0x7fdd701418a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:16.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd74f49700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdd700b9860 0x7fdd700ad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:16.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd6f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd700b4f00 0x7fdd700ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:16.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd6ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdd700b45c0 0x7fdd701418a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39426/0 (socket says 172.21.15.36:39426) 2022-06-17T12:56:16.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd6f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd700b4f00 0x7fdd700ad330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:59810/0 (socket says 172.21.15.36:59810) 2022-06-17T12:56:16.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd74f49700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdd700b9860 0x7fdd700ad820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:54220/0 (socket says 172.21.15.36:54220) 2022-06-17T12:56:16.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd6ffff700 1 -- 172.21.15.36:0/113497379 learned_addr learned my addr 172.21.15.36:0/113497379 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:16.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd6ffff700 1 -- 172.21.15.36:0/113497379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdd700b9860 msgr2=0x7fdd700ad820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:16.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd6ffff700 1 --2- 172.21.15.36:0/113497379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdd700b9860 0x7fdd700ad820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd6ffff700 1 -- 172.21.15.36:0/113497379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd700b4f00 msgr2=0x7fdd700ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:16.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd6ffff700 1 --2- 172.21.15.36:0/113497379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd700b4f00 0x7fdd700ad330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd6ffff700 1 -- 172.21.15.36:0/113497379 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6000b1e0 con 0x7fdd700b45c0 2022-06-17T12:56:16.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd6f7fe700 1 --2- 172.21.15.36:0/113497379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd700b4f00 0x7fdd700ad330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:16.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd6ffff700 1 --2- 172.21.15.36:0/113497379 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdd700b45c0 0x7fdd701418a0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 rx=0x7fdd700b44a0 tx=0x7fdd58007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:16.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd6d7fa700 1 -- 172.21.15.36:0/113497379 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd58012400 con 0x7fdd700b45c0 2022-06-17T12:56:16.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd6d7fa700 1 -- 172.21.15.36:0/113497379 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd58012560 con 0x7fdd700b45c0 2022-06-17T12:56:16.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd7574a700 1 -- 172.21.15.36:0/113497379 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd70146d20 con 0x7fdd700b45c0 2022-06-17T12:56:16.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd6d7fa700 1 -- 172.21.15.36:0/113497379 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd580128c0 con 0x7fdd700b45c0 2022-06-17T12:56:16.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd7574a700 1 -- 172.21.15.36:0/113497379 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd70147040 con 0x7fdd700b45c0 2022-06-17T12:56:16.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.707+0000 7fdd7574a700 1 -- 172.21.15.36:0/113497379 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd70146a40 con 0x7fdd700b45c0 2022-06-17T12:56:16.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.711+0000 7fdd6d7fa700 1 -- 172.21.15.36:0/113497379 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fdd58019030 con 0x7fdd700b45c0 2022-06-17T12:56:16.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.711+0000 7fdd6d7fa700 1 --2- 172.21.15.36:0/113497379 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdd54053a40 0x7fdd54055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:16.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.711+0000 7fdd6d7fa700 1 -- 172.21.15.36:0/113497379 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdd5806b360 con 0x7fdd700b45c0 2022-06-17T12:56:16.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.711+0000 7fdd6f7fe700 1 --2- 172.21.15.36:0/113497379 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdd54053a40 0x7fdd54055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:16.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.711+0000 7fdd6f7fe700 1 --2- 172.21.15.36:0/113497379 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdd54053a40 0x7fdd54055f00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7fdd6001c000 tx=0x7fdd60015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:16.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.719+0000 7fdd6d7fa700 1 -- 172.21.15.36:0/113497379 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdd58042050 con 0x7fdd700b45c0 2022-06-17T12:56:16.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.847+0000 7fdd7574a700 1 -- 172.21.15.36:0/113497379 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd70062eb0 con 0x7fdd700b45c0 2022-06-17T12:56:16.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.847+0000 7fdd6d7fa700 1 -- 172.21.15.36:0/113497379 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v131) v1 ==== 104+0+10 (secure 0 0 0) 0x7fdd58045670 con 0x7fdd700b45c0 2022-06-17T12:56:16.871 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:56:16.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.851+0000 7fdd7574a700 1 -- 172.21.15.36:0/113497379 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdd54053a40 msgr2=0x7fdd54055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:16.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.851+0000 7fdd7574a700 1 --2- 172.21.15.36:0/113497379 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdd54053a40 0x7fdd54055f00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7fdd6001c000 tx=0x7fdd60015000).stop 2022-06-17T12:56:16.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.851+0000 7fdd7574a700 1 -- 172.21.15.36:0/113497379 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdd700b45c0 msgr2=0x7fdd701418a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:16.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.851+0000 7fdd7574a700 1 --2- 172.21.15.36:0/113497379 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdd700b45c0 0x7fdd701418a0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 rx=0x7fdd700b44a0 tx=0x7fdd58007c50).stop 2022-06-17T12:56:16.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.851+0000 7fdd7574a700 1 -- 172.21.15.36:0/113497379 shutdown_connections 2022-06-17T12:56:16.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.851+0000 7fdd7574a700 1 --2- 172.21.15.36:0/113497379 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdd54053a40 0x7fdd54055f00 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.851+0000 7fdd7574a700 1 --2- 172.21.15.36:0/113497379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdd700b9860 0x7fdd700ad820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.851+0000 7fdd7574a700 1 --2- 172.21.15.36:0/113497379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd700b4f00 0x7fdd700ad330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.851+0000 7fdd7574a700 1 --2- 172.21.15.36:0/113497379 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdd700b45c0 0x7fdd701418a0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:16.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.851+0000 7fdd7574a700 1 -- 172.21.15.36:0/113497379 >> 172.21.15.36:0/113497379 conn(0x7fdd7005c550 msgr2=0x7fdd700b5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:16.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.851+0000 7fdd7574a700 1 -- 172.21.15.36:0/113497379 shutdown_connections 2022-06-17T12:56:16.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:16.851+0000 7fdd7574a700 1 -- 172.21.15.36:0/113497379 wait complete. 2022-06-17T12:56:16.877 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:56:17.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.007+0000 7f11ca8fc700 1 Processor -- start 2022-06-17T12:56:17.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.007+0000 7f11ca8fc700 1 -- start start 2022-06-17T12:56:17.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11ca8fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f11c4060920 0x7f11c4067610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:17.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11ca8fc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11c40b1e00 0x7f11c40ad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:17.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11ca8fc700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f11c40adc50 0x7f11c4142920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:17.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11ca8fc700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c40aff80 con 0x7f11c4060920 2022-06-17T12:56:17.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11ca8fc700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c4067b00 con 0x7f11c40adc50 2022-06-17T12:56:17.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11ca8fc700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c40ad020 con 0x7f11c40b1e00 2022-06-17T12:56:17.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11c98fa700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f11c4060920 0x7f11c4067610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:17.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11c98fa700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f11c4060920 0x7f11c4067610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39434/0 (socket says 172.21.15.36:39434) 2022-06-17T12:56:17.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11c98fa700 1 -- 172.21.15.36:0/4226274003 learned_addr learned my addr 172.21.15.36:0/4226274003 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:17.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11c90f9700 1 --2- 172.21.15.36:0/4226274003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11c40b1e00 0x7f11c40ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:17.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11ca0fb700 1 --2- 172.21.15.36:0/4226274003 >> v2:172.21.15.70:3300/0 conn(0x7f11c40adc50 0x7f11c4142920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:17.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11c98fa700 1 -- 172.21.15.36:0/4226274003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11c40b1e00 msgr2=0x7f11c40ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:17.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11c98fa700 1 --2- 172.21.15.36:0/4226274003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11c40b1e00 0x7f11c40ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:17.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11c98fa700 1 -- 172.21.15.36:0/4226274003 >> v2:172.21.15.70:3300/0 conn(0x7f11c40adc50 msgr2=0x7f11c4142920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:17.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11c98fa700 1 --2- 172.21.15.36:0/4226274003 >> v2:172.21.15.70:3300/0 conn(0x7f11c40adc50 0x7f11c4142920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:17.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11c98fa700 1 -- 172.21.15.36:0/4226274003 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c40ad2c0 con 0x7f11c4060920 2022-06-17T12:56:17.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11ca0fb700 1 --2- 172.21.15.36:0/4226274003 >> v2:172.21.15.70:3300/0 conn(0x7f11c40adc50 0x7f11c4142920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:56:17.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11c98fa700 1 --2- 172.21.15.36:0/4226274003 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11c4060920 0x7f11c4067610 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0x7f11c0002f70 tx=0x7f11c0006eb0).ready entity=mon.0 client_cookie=67d9f6d10e4ffe2f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:17.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11c88f8700 1 -- 172.21.15.36:0/4226274003 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11c0014070 con 0x7f11c4060920 2022-06-17T12:56:17.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11c88f8700 1 -- 172.21.15.36:0/4226274003 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c0000e90 con 0x7f11c4060920 2022-06-17T12:56:17.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11c88f8700 1 -- 172.21.15.36:0/4226274003 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11c000eaa0 con 0x7f11c4060920 2022-06-17T12:56:17.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11ca8fc700 1 -- 172.21.15.36:0/4226274003 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11c4060920 msgr2=0x7f11c4067610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:17.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11ca8fc700 1 --2- 172.21.15.36:0/4226274003 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11c4060920 0x7f11c4067610 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0x7f11c0002f70 tx=0x7f11c0006eb0).stop 2022-06-17T12:56:17.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11ca8fc700 1 -- 172.21.15.36:0/4226274003 shutdown_connections 2022-06-17T12:56:17.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11ca8fc700 1 --2- 172.21.15.36:0/4226274003 >> v2:172.21.15.70:3300/0 conn(0x7f11c40adc50 0x7f11c4142920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:17.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11ca8fc700 1 --2- 172.21.15.36:0/4226274003 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11c40b1e00 0x7f11c40ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:17.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11ca8fc700 1 --2- 172.21.15.36:0/4226274003 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11c4060920 0x7f11c4067610 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:17.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11ca8fc700 1 -- 172.21.15.36:0/4226274003 >> 172.21.15.36:0/4226274003 conn(0x7f11c40b64f0 msgr2=0x7f11c40b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:17.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11ca8fc700 1 -- 172.21.15.36:0/4226274003 shutdown_connections 2022-06-17T12:56:17.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.011+0000 7f11ca8fc700 1 -- 172.21.15.36:0/4226274003 wait complete. 2022-06-17T12:56:17.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.015+0000 7f11ca8fc700 1 Processor -- start 2022-06-17T12:56:17.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.015+0000 7f11ca8fc700 1 -- start start 2022-06-17T12:56:17.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.019+0000 7f11ca8fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11c4060920 0x7f11c40a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:17.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.019+0000 7f11ca8fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11c40adc50 0x7f11c40abdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:17.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.019+0000 7f11ca8fc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11c40b1e00 0x7f11c40ac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:17.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.019+0000 7f11ca8fc700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c40a4a40 con 0x7f11c40adc50 2022-06-17T12:56:17.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.019+0000 7f11ca8fc700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c40a78a0 con 0x7f11c4060920 2022-06-17T12:56:17.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.019+0000 7f11ca8fc700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c40a7b40 con 0x7f11c40b1e00 2022-06-17T12:56:17.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.019+0000 7f11c90f9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11c40adc50 0x7f11c40abdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:17.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.019+0000 7f11c90f9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11c40adc50 0x7f11c40abdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39442/0 (socket says 172.21.15.36:39442) 2022-06-17T12:56:17.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.019+0000 7f11c90f9700 1 -- 172.21.15.36:0/3123497493 learned_addr learned my addr 172.21.15.36:0/3123497493 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:17.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.019+0000 7f11c98fa700 1 --2- 172.21.15.36:0/3123497493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11c4060920 0x7f11c40a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:17.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.019+0000 7f11ca0fb700 1 --2- 172.21.15.36:0/3123497493 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11c40b1e00 0x7f11c40ac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:17.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.019+0000 7f11c90f9700 1 -- 172.21.15.36:0/3123497493 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11c40b1e00 msgr2=0x7f11c40ac2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:17.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.019+0000 7f11c90f9700 1 --2- 172.21.15.36:0/3123497493 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11c40b1e00 0x7f11c40ac2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:17.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.019+0000 7f11c90f9700 1 -- 172.21.15.36:0/3123497493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11c4060920 msgr2=0x7f11c40a4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:17.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.019+0000 7f11c90f9700 1 --2- 172.21.15.36:0/3123497493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11c4060920 0x7f11c40a4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:17.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.019+0000 7f11c90f9700 1 -- 172.21.15.36:0/3123497493 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c000b1e0 con 0x7f11c40adc50 2022-06-17T12:56:17.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.019+0000 7f11c90f9700 1 --2- 172.21.15.36:0/3123497493 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11c40adc50 0x7f11c40abdd0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7f11b400c000 tx=0x7f11b4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:17.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.019+0000 7f11b2ffd700 1 -- 172.21.15.36:0/3123497493 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11b4011750 con 0x7f11c40adc50 2022-06-17T12:56:17.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.019+0000 7f11b2ffd700 1 -- 172.21.15.36:0/3123497493 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b40118b0 con 0x7f11c40adc50 2022-06-17T12:56:17.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.019+0000 7f11ca8fc700 1 -- 172.21.15.36:0/3123497493 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c40a46c0 con 0x7f11c40adc50 2022-06-17T12:56:17.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.019+0000 7f11b2ffd700 1 -- 172.21.15.36:0/3123497493 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11b4011c10 con 0x7f11c40adc50 2022-06-17T12:56:17.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.019+0000 7f11ca8fc700 1 -- 172.21.15.36:0/3123497493 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c40a3320 con 0x7f11c40adc50 2022-06-17T12:56:17.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.019+0000 7f11ca8fc700 1 -- 172.21.15.36:0/3123497493 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c40a76d0 con 0x7f11c40adc50 2022-06-17T12:56:17.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.023+0000 7f11b2ffd700 1 -- 172.21.15.36:0/3123497493 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f11b4011d70 con 0x7f11c40adc50 2022-06-17T12:56:17.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.023+0000 7f11b2ffd700 1 --2- 172.21.15.36:0/3123497493 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f11a4053940 0x7f11a4055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:17.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.023+0000 7f11b2ffd700 1 -- 172.21.15.36:0/3123497493 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f11b406c480 con 0x7f11c40adc50 2022-06-17T12:56:17.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.023+0000 7f11c98fa700 1 --2- 172.21.15.36:0/3123497493 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f11a4053940 0x7f11a4055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:17.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.023+0000 7f11c98fa700 1 --2- 172.21.15.36:0/3123497493 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f11a4053940 0x7f11a4055e00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f11c001c000 tx=0x7f11c0016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:17.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.031+0000 7f11b2ffd700 1 -- 172.21.15.36:0/3123497493 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f11b4043180 con 0x7f11c40adc50 2022-06-17T12:56:17.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.159+0000 7f11ca8fc700 1 -- 172.21.15.36:0/3123497493 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f11c4060d00 con 0x7f11c40adc50 2022-06-17T12:56:17.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.159+0000 7f11b2ffd700 1 -- 172.21.15.36:0/3123497493 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v131) v1 ==== 100+0+10 (secure 0 0 0) 0x7f11b40467a0 con 0x7f11c40adc50 2022-06-17T12:56:17.151 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:56:17.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.159+0000 7f11ca8fc700 1 -- 172.21.15.36:0/3123497493 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f11a4053940 msgr2=0x7f11a4055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:17.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.159+0000 7f11ca8fc700 1 --2- 172.21.15.36:0/3123497493 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f11a4053940 0x7f11a4055e00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f11c001c000 tx=0x7f11c0016000).stop 2022-06-17T12:56:17.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.159+0000 7f11ca8fc700 1 -- 172.21.15.36:0/3123497493 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11c40adc50 msgr2=0x7f11c40abdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:17.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.159+0000 7f11ca8fc700 1 --2- 172.21.15.36:0/3123497493 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11c40adc50 0x7f11c40abdd0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7f11b400c000 tx=0x7f11b4006df0).stop 2022-06-17T12:56:17.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.159+0000 7f11ca8fc700 1 -- 172.21.15.36:0/3123497493 shutdown_connections 2022-06-17T12:56:17.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.159+0000 7f11ca8fc700 1 --2- 172.21.15.36:0/3123497493 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f11a4053940 0x7f11a4055e00 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:17.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.159+0000 7f11ca8fc700 1 --2- 172.21.15.36:0/3123497493 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11c40b1e00 0x7f11c40ac2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:17.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.159+0000 7f11ca8fc700 1 --2- 172.21.15.36:0/3123497493 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11c40adc50 0x7f11c40abdd0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:17.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.159+0000 7f11ca8fc700 1 --2- 172.21.15.36:0/3123497493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11c4060920 0x7f11c40a4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:17.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.159+0000 7f11ca8fc700 1 -- 172.21.15.36:0/3123497493 >> 172.21.15.36:0/3123497493 conn(0x7f11c40b64f0 msgr2=0x7f11c40b0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:17.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.159+0000 7f11ca8fc700 1 -- 172.21.15.36:0/3123497493 shutdown_connections 2022-06-17T12:56:17.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.159+0000 7f11ca8fc700 1 -- 172.21.15.36:0/3123497493 wait complete. 2022-06-17T12:56:17.161 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:56:17.161 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:56:17.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.279+0000 7f9fd20bb700 1 Processor -- start 2022-06-17T12:56:17.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.279+0000 7f9fd20bb700 1 -- start start 2022-06-17T12:56:17.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.279+0000 7f9fd20bb700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9fcc0b20f0 0x7f9fcc0aee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:17.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.279+0000 7f9fd20bb700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9fcc0afe50 0x7f9fcc13c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:17.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.279+0000 7f9fd20bb700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fcc060ea0 con 0x7f9fcc060ac0 2022-06-17T12:56:17.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.279+0000 7f9fd20bb700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fcc0b3060 con 0x7f9fcc0afe50 2022-06-17T12:56:17.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.279+0000 7f9fd20bb700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fcc067a20 con 0x7f9fcc0b20f0 2022-06-17T12:56:17.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.279+0000 7f9fd18ba700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9fcc0afe50 0x7f9fcc13c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:17.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.279+0000 7f9fd18ba700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9fcc0afe50 0x7f9fcc13c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39452/0 (socket says 172.21.15.36:39452) 2022-06-17T12:56:17.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.279+0000 7f9fd18ba700 1 -- 172.21.15.36:0/2507785172 learned_addr learned my addr 172.21.15.36:0/2507785172 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:17.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.279+0000 7f9fd10b9700 1 --1- 172.21.15.36:0/2507785172 >> v1:172.21.15.36:6789/0 conn(0x7f9fcc060ac0 0x7f9fcc0ae720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56978/0 (socket says 172.21.15.36:56978) 2022-06-17T12:56:17.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.279+0000 7f9fd08b8700 1 --2- 172.21.15.36:0/2507785172 >> v2:172.21.15.70:3300/0 conn(0x7f9fcc0b20f0 0x7f9fcc0aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:17.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.283+0000 7f9fd18ba700 1 -- 172.21.15.36:0/2507785172 >> v2:172.21.15.70:3300/0 conn(0x7f9fcc0b20f0 msgr2=0x7f9fcc0aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:17.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.283+0000 7f9fd18ba700 1 --2- 172.21.15.36:0/2507785172 >> v2:172.21.15.70:3300/0 conn(0x7f9fcc0b20f0 0x7f9fcc0aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:17.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.283+0000 7f9fd18ba700 1 -- 172.21.15.36:0/2507785172 >> v1:172.21.15.36:6789/0 conn(0x7f9fcc060ac0 legacy=0x7f9fcc0ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:17.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.283+0000 7f9fd18ba700 1 -- 172.21.15.36:0/2507785172 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fcc143fb0 con 0x7f9fcc0afe50 2022-06-17T12:56:17.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.283+0000 7f9fd08b8700 1 --2- 172.21.15.36:0/2507785172 >> v2:172.21.15.70:3300/0 conn(0x7f9fcc0b20f0 0x7f9fcc0aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:56:17.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.283+0000 7f9fd18ba700 1 --2- 172.21.15.36:0/2507785172 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9fcc0afe50 0x7f9fcc13c7e0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 rx=0x7f9fc40032f0 tx=0x7f9fc4009290).ready entity=mon.0 client_cookie=90d2888a98bcb7a1 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:17.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.283+0000 7f9fbbfff700 1 -- 172.21.15.36:0/2507785172 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fc400b870 con 0x7f9fcc0afe50 2022-06-17T12:56:17.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.283+0000 7f9fbbfff700 1 -- 172.21.15.36:0/2507785172 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc400b9d0 con 0x7f9fcc0afe50 2022-06-17T12:56:17.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.283+0000 7f9fbbfff700 1 -- 172.21.15.36:0/2507785172 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fc400bb40 con 0x7f9fcc0afe50 2022-06-17T12:56:17.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.283+0000 7f9fd20bb700 1 -- 172.21.15.36:0/2507785172 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9fcc0afe50 msgr2=0x7f9fcc13c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:17.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.283+0000 7f9fd20bb700 1 --2- 172.21.15.36:0/2507785172 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9fcc0afe50 0x7f9fcc13c7e0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 rx=0x7f9fc40032f0 tx=0x7f9fc4009290).stop 2022-06-17T12:56:17.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.283+0000 7f9fd20bb700 1 -- 172.21.15.36:0/2507785172 shutdown_connections 2022-06-17T12:56:17.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.283+0000 7f9fd20bb700 1 --2- 172.21.15.36:0/2507785172 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9fcc0afe50 0x7f9fcc13c7e0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:17.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.283+0000 7f9fd20bb700 1 --2- 172.21.15.36:0/2507785172 >> v2:172.21.15.70:3300/0 conn(0x7f9fcc0b20f0 0x7f9fcc0aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:17.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.283+0000 7f9fd20bb700 1 -- 172.21.15.36:0/2507785172 >> 172.21.15.36:0/2507785172 conn(0x7f9fcc0b65f0 msgr2=0x7f9fcc0b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:17.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.283+0000 7f9fd20bb700 1 -- 172.21.15.36:0/2507785172 shutdown_connections 2022-06-17T12:56:17.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.283+0000 7f9fd20bb700 1 -- 172.21.15.36:0/2507785172 wait complete. 2022-06-17T12:56:17.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fd20bb700 1 Processor -- start 2022-06-17T12:56:17.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fd20bb700 1 -- start start 2022-06-17T12:56:17.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fd20bb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fcc060ac0 0x7f9fcc146c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:17.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fd20bb700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9fcc0afe50 0x7f9fcc13b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:17.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fd20bb700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fcc0b20f0 0x7f9fcc13bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:17.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fd20bb700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fcc13e7e0 con 0x7f9fcc0afe50 2022-06-17T12:56:17.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fd20bb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fcc140860 con 0x7f9fcc060ac0 2022-06-17T12:56:17.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fd20bb700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fcc13ea10 con 0x7f9fcc0b20f0 2022-06-17T12:56:17.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fd08b8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9fcc0afe50 0x7f9fcc13b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:17.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fd08b8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9fcc0afe50 0x7f9fcc13b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39456/0 (socket says 172.21.15.36:39456) 2022-06-17T12:56:17.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fd08b8700 1 -- 172.21.15.36:0/1183063754 learned_addr learned my addr 172.21.15.36:0/1183063754 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:17.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fd18ba700 1 --2- 172.21.15.36:0/1183063754 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fcc0b20f0 0x7f9fcc13bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:17.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fd10b9700 1 --2- 172.21.15.36:0/1183063754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fcc060ac0 0x7f9fcc146c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:17.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fd18ba700 1 -- 172.21.15.36:0/1183063754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fcc060ac0 msgr2=0x7f9fcc146c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:17.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fd18ba700 1 --2- 172.21.15.36:0/1183063754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fcc060ac0 0x7f9fcc146c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:17.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fd18ba700 1 -- 172.21.15.36:0/1183063754 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9fcc0afe50 msgr2=0x7f9fcc13b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:17.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fd18ba700 1 --2- 172.21.15.36:0/1183063754 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9fcc0afe50 0x7f9fcc13b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:17.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fd18ba700 1 -- 172.21.15.36:0/1183063754 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc4009530 con 0x7f9fcc0b20f0 2022-06-17T12:56:17.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fd08b8700 1 --2- 172.21.15.36:0/1183063754 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9fcc0afe50 0x7f9fcc13b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:17.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fd18ba700 1 --2- 172.21.15.36:0/1183063754 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fcc0b20f0 0x7f9fcc13bac0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f9fc40032f0 tx=0x7f9fc4016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:17.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fba7fc700 1 -- 172.21.15.36:0/1183063754 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fc400b870 con 0x7f9fcc0b20f0 2022-06-17T12:56:17.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fd20bb700 1 -- 172.21.15.36:0/1183063754 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fcc142bb0 con 0x7f9fcc0b20f0 2022-06-17T12:56:17.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fd20bb700 1 -- 172.21.15.36:0/1183063754 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fcc1481b0 con 0x7f9fcc0b20f0 2022-06-17T12:56:17.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fba7fc700 1 -- 172.21.15.36:0/1183063754 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc400bdb0 con 0x7f9fcc0b20f0 2022-06-17T12:56:17.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fba7fc700 1 -- 172.21.15.36:0/1183063754 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fc4020610 con 0x7f9fcc0b20f0 2022-06-17T12:56:17.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.287+0000 7f9fd20bb700 1 -- 172.21.15.36:0/1183063754 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa8003410 con 0x7f9fcc0b20f0 2022-06-17T12:56:17.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.291+0000 7f9fba7fc700 1 -- 172.21.15.36:0/1183063754 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f9fc4027070 con 0x7f9fcc0b20f0 2022-06-17T12:56:17.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.291+0000 7f9fba7fc700 1 --2- 172.21.15.36:0/1183063754 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9fac053af0 0x7f9fac055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:17.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.291+0000 7f9fba7fc700 1 -- 172.21.15.36:0/1183063754 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9fc406cbb0 con 0x7f9fcc0b20f0 2022-06-17T12:56:17.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.291+0000 7f9fd10b9700 1 --2- 172.21.15.36:0/1183063754 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9fac053af0 0x7f9fac055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:17.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.291+0000 7f9fd10b9700 1 --2- 172.21.15.36:0/1183063754 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9fac053af0 0x7f9fac055fb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f9fc800d000 tx=0x7f9fc8007b40).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:17.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.299+0000 7f9fba7fc700 1 -- 172.21.15.36:0/1183063754 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9fc40439f0 con 0x7f9fcc0b20f0 2022-06-17T12:56:17.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.467+0000 7f9fd20bb700 1 -- 172.21.15.36:0/1183063754 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fa8003990 con 0x7f9fcc0b20f0 2022-06-17T12:56:17.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.467+0000 7f9fba7fc700 1 -- 172.21.15.36:0/1183063754 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f9fc4025870 con 0x7f9fcc0b20f0 2022-06-17T12:56:17.457 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:17.457 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":72,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/2357210652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2357210652},{"type":"v1","addr":"172.21.15.36:6835","nonce":2357210652}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:56:11.021928+0000","modified":"2022-06-17T12:56:13.034227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7942":{"gid":7942,"name":"b","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/197726269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":197726269},{"type":"v1","addr":"172.21.15.70:6833","nonce":197726269}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-06-17T12:56:17.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.471+0000 7f9fd20bb700 1 -- 172.21.15.36:0/1183063754 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9fac053af0 msgr2=0x7f9fac055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:17.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.471+0000 7f9fd20bb700 1 --2- 172.21.15.36:0/1183063754 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9fac053af0 0x7f9fac055fb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f9fc800d000 tx=0x7f9fc8007b40).stop 2022-06-17T12:56:17.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.471+0000 7f9fd20bb700 1 -- 172.21.15.36:0/1183063754 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fcc0b20f0 msgr2=0x7f9fcc13bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:17.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.471+0000 7f9fd20bb700 1 --2- 172.21.15.36:0/1183063754 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fcc0b20f0 0x7f9fcc13bac0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f9fc40032f0 tx=0x7f9fc4016000).stop 2022-06-17T12:56:17.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.471+0000 7f9fd20bb700 1 -- 172.21.15.36:0/1183063754 shutdown_connections 2022-06-17T12:56:17.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.471+0000 7f9fd20bb700 1 --2- 172.21.15.36:0/1183063754 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9fac053af0 0x7f9fac055fb0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:17.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.471+0000 7f9fd20bb700 1 --2- 172.21.15.36:0/1183063754 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fcc0b20f0 0x7f9fcc13bac0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:17.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.471+0000 7f9fd20bb700 1 --2- 172.21.15.36:0/1183063754 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9fcc0afe50 0x7f9fcc13b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:17.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.471+0000 7f9fd20bb700 1 --2- 172.21.15.36:0/1183063754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fcc060ac0 0x7f9fcc146c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:17.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.471+0000 7f9fd20bb700 1 -- 172.21.15.36:0/1183063754 >> 172.21.15.36:0/1183063754 conn(0x7f9fcc0b65f0 msgr2=0x7f9fcc142e50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:17.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.471+0000 7f9fd20bb700 1 -- 172.21.15.36:0/1183063754 shutdown_connections 2022-06-17T12:56:17.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:17.471+0000 7f9fd20bb700 1 -- 172.21.15.36:0/1183063754 wait complete. 2022-06-17T12:56:17.485 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 72 2022-06-17T12:56:17.691 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:56:11.021928+0000', 'modified': '2022-06-17T12:56:13.034227+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7942}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7942': {'gid': 7942, 'name': 'b', 'rank': 0, 'incarnation': 69, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6833/197726269', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 197726269}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 197726269}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:56:17.693 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:56:17.693 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-17T12:56:17.831 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":7942,"want_state":"up:active","state":"up:active","rank_uptime":6.8058624559999998,"mdsmap_epoch":72,"osdmap_epoch":131,"osdmap_epoch_barrier":131,"uptime":11.789234962} 2022-06-17T12:56:17.831 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 7942, "mdsmap_epoch": 72, "osdmap_epoch": 131, "osdmap_epoch_barrier": 131, "rank_uptime": 6.805862456, "state": "up:active", "uptime": 11.789234962, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:56:17.832 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:56:17.833 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-17T12:56:17.834 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi107.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-17T12:56:17.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:17.835 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:56:17.867 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:56:17.887 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:56:17.888 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:56:17.951 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:56:17.952 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:56:17.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:17.956 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:56:17.973 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:56:17.974 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:56:18.033 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:56:18.050 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:56:18.051 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:56:18.108 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:56:18.109 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:56:18.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:18.122 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:56:18.124 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:2022-06-17T12:56:18.125+0000 7fc7e1339080 -1 init, newargv = 0x5628be616230 newargc=9 2022-06-17T12:56:18.124 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22894]: starting ceph client 2022-06-17T12:56:18.126 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22894]: starting fuse 2022-06-17T12:56:18.143 INFO:teuthology.orchestra.run.smithi107.stdout:52 2022-06-17T12:56:18.143 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:56:18.144 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:18.145 DEBUG:teuthology.orchestra.run.smithi107:> sudo python3 -c ' 2022-06-17T12:56:18.146 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2022-06-17T12:56:18.147 DEBUG:teuthology.orchestra.run.smithi107:> import re 2022-06-17T12:56:18.148 DEBUG:teuthology.orchestra.run.smithi107:> import os 2022-06-17T12:56:18.149 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2022-06-17T12:56:18.149 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:56:18.150 DEBUG:teuthology.orchestra.run.smithi107:> def find_socket(client_name): 2022-06-17T12:56:18.151 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-17T12:56:18.152 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2022-06-17T12:56:18.153 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:56:18.153 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2022-06-17T12:56:18.154 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2022-06-17T12:56:18.155 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2022-06-17T12:56:18.156 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2022-06-17T12:56:18.157 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:56:18.157 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2022-06-17T12:56:18.162 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:56:18.163 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:56:18.163 DEBUG:teuthology.orchestra.run.smithi107:> return f 2022-06-17T12:56:18.165 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:56:18.166 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:56:18.166 DEBUG:teuthology.orchestra.run.smithi107:> print(find_socket("client.0")) 2022-06-17T12:56:18.167 DEBUG:teuthology.orchestra.run.smithi107:> ' 2022-06-17T12:56:18.241 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.0.22894.asok 2022-06-17T12:56:18.260 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22894.asok 2022-06-17T12:56:18.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:18.262 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22894.asok status 2022-06-17T12:56:18.375 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2022-06-17T12:56:18.376 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2022-06-17T12:56:18.376 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:56:18.377 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:56:18.377 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "0", 2022-06-17T12:56:18.377 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2022-06-17T12:56:18.378 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-17T12:56:18.378 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "22894", 2022-06-17T12:56:18.379 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/" 2022-06-17T12:56:18.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:56:18.380 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2022-06-17T12:56:18.381 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2022-06-17T12:56:18.381 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 8104, 2022-06-17T12:56:18.382 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2022-06-17T12:56:18.382 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2022-06-17T12:56:18.383 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2022-06-17T12:56:18.401 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 8104 2022-06-17T12:56:18.402 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:56:18.403 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:56:18.403 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:56:18.404 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:56:18.404 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1621678664 2022-06-17T12:56:18.404 INFO:teuthology.orchestra.run.smithi107.stdout: } 2022-06-17T12:56:18.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:56:18.405 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:56:18.428 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:56:18.428 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:56:18.433 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1621678664 2022-06-17T12:56:18.433 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:56:18.434 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.8104 172.21.15.107:0/1621678664", 2022-06-17T12:56:18.435 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "172.21.15.107:0/1621678664", 2022-06-17T12:56:18.435 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 1, 2022-06-17T12:56:18.436 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 72, 2022-06-17T12:56:18.436 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 131, 2022-06-17T12:56:18.436 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:56:18.437 INFO:teuthology.orchestra.run.smithi107.stdout: "blacklisted": false, 2022-06-17T12:56:18.437 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2022-06-17T12:56:18.441 INFO:teuthology.orchestra.run.smithi107.stdout:} 2022-06-17T12:56:18.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:18.469 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:18.470 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:56:18.484 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:56:18.485 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:56:18.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:18.486 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:56:18.513 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:56:18.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.667+0000 7f3fd304c700 1 Processor -- start 2022-06-17T12:56:18.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.667+0000 7f3fd304c700 1 -- start start 2022-06-17T12:56:18.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.667+0000 7f3fd304c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f3fcc0ae270 0x7f3fcc0b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:18.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.667+0000 7f3fd304c700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fcc0afc30 con 0x7f3fcc13fde0 2022-06-17T12:56:18.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.667+0000 7f3fd304c700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fcc0ae650 con 0x7f3fcc0ad9c0 2022-06-17T12:56:18.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.667+0000 7f3fd304c700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fcc0b0cb0 con 0x7f3fcc0ae270 2022-06-17T12:56:18.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.667+0000 7f3fd0de8700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f3fcc0ae270 0x7f3fcc0b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:18.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.667+0000 7f3fd15e9700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f3fcc13fde0 0x7f3fcc13edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:56996/0 (socket says 172.21.15.36:56996) 2022-06-17T12:56:18.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.667+0000 7f3fd15e9700 1 -- 172.21.15.36:0/3121393824 learned_addr learned my addr 172.21.15.36:0/3121393824 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:18.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.667+0000 7f3fd0de8700 1 -- 172.21.15.36:0/3121393824 >> v1:172.21.15.70:6789/0 conn(0x7f3fcc0ad9c0 legacy=0x7f3fcc0b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:18.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.667+0000 7f3fd0de8700 1 -- 172.21.15.36:0/3121393824 >> v1:172.21.15.36:6789/0 conn(0x7f3fcc13fde0 legacy=0x7f3fcc13edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:18.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.667+0000 7f3fd0de8700 1 -- 172.21.15.36:0/3121393824 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fcc0afe60 con 0x7f3fcc0ae270 2022-06-17T12:56:18.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.667+0000 7f3fcb7fe700 1 -- 172.21.15.36:0/3121393824 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4094464974 0 0) 0x7f3fcc0afc30 con 0x7f3fcc13fde0 2022-06-17T12:56:18.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.667+0000 7f3fd0de8700 1 --2- 172.21.15.36:0/3121393824 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3fcc0ae270 0x7f3fcc0b39c0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 rx=0x7f3fbc002f70 tx=0x7f3fbc006df0).ready entity=mon.0 client_cookie=6324b966737eb912 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:18.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.667+0000 7f3fcb7fe700 1 -- 172.21.15.36:0/3121393824 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fbc00d610 con 0x7f3fcc0ae270 2022-06-17T12:56:18.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.667+0000 7f3fcb7fe700 1 -- 172.21.15.36:0/3121393824 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fbc000e90 con 0x7f3fcc0ae270 2022-06-17T12:56:18.674 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.667+0000 7f3fcb7fe700 1 -- 172.21.15.36:0/3121393824 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fbc00d960 con 0x7f3fcc0ae270 2022-06-17T12:56:18.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.667+0000 7f3fd304c700 1 -- 172.21.15.36:0/3121393824 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3fcc0ae270 msgr2=0x7f3fcc0b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:18.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.667+0000 7f3fd304c700 1 --2- 172.21.15.36:0/3121393824 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3fcc0ae270 0x7f3fcc0b39c0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 rx=0x7f3fbc002f70 tx=0x7f3fbc006df0).stop 2022-06-17T12:56:18.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.667+0000 7f3fd304c700 1 -- 172.21.15.36:0/3121393824 shutdown_connections 2022-06-17T12:56:18.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.667+0000 7f3fd304c700 1 --2- 172.21.15.36:0/3121393824 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3fcc0ae270 0x7f3fcc0b39c0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:18.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.667+0000 7f3fd304c700 1 -- 172.21.15.36:0/3121393824 >> 172.21.15.36:0/3121393824 conn(0x7f3fcc05c550 msgr2=0x7f3fcc05c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:18.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.667+0000 7f3fd304c700 1 -- 172.21.15.36:0/3121393824 shutdown_connections 2022-06-17T12:56:18.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.667+0000 7f3fd304c700 1 -- 172.21.15.36:0/3121393824 wait complete. 2022-06-17T12:56:18.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.671+0000 7f3fd304c700 1 Processor -- start 2022-06-17T12:56:18.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.671+0000 7f3fd304c700 1 -- start start 2022-06-17T12:56:18.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.671+0000 7f3fd304c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fcc0ad9c0 0x7f3fcc1436a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:18.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.671+0000 7f3fd304c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3fcc0ae270 0x7f3fcc1521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:18.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.671+0000 7f3fd304c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3fcc13fde0 0x7f3fcc0b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:18.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.671+0000 7f3fd304c700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc143b90 con 0x7f3fcc13fde0 2022-06-17T12:56:18.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.671+0000 7f3fd304c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc065030 con 0x7f3fcc0ad9c0 2022-06-17T12:56:18.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.671+0000 7f3fd304c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc1526b0 con 0x7f3fcc0ae270 2022-06-17T12:56:18.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.671+0000 7f3fd15e9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3fcc13fde0 0x7f3fcc0b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:18.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.671+0000 7f3fd0de8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fcc0ad9c0 0x7f3fcc1436a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:18.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.671+0000 7f3fcbfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3fcc0ae270 0x7f3fcc1521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:18.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.671+0000 7f3fd15e9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3fcc13fde0 0x7f3fcc0b4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39472/0 (socket says 172.21.15.36:39472) 2022-06-17T12:56:18.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.671+0000 7f3fd15e9700 1 -- 172.21.15.36:0/3776384549 learned_addr learned my addr 172.21.15.36:0/3776384549 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:18.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.671+0000 7f3fcbfff700 1 -- 172.21.15.36:0/3776384549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fcc0ad9c0 msgr2=0x7f3fcc1436a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:18.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.671+0000 7f3fcbfff700 1 --2- 172.21.15.36:0/3776384549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fcc0ad9c0 0x7f3fcc1436a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:18.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.671+0000 7f3fcbfff700 1 -- 172.21.15.36:0/3776384549 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3fcc13fde0 msgr2=0x7f3fcc0b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:18.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.671+0000 7f3fcbfff700 1 --2- 172.21.15.36:0/3776384549 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3fcc13fde0 0x7f3fcc0b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:18.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.671+0000 7f3fcbfff700 1 -- 172.21.15.36:0/3776384549 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fbc00b180 con 0x7f3fcc0ae270 2022-06-17T12:56:18.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.671+0000 7f3fd15e9700 1 --2- 172.21.15.36:0/3776384549 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3fcc13fde0 0x7f3fcc0b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:18.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.671+0000 7f3fcbfff700 1 --2- 172.21.15.36:0/3776384549 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3fcc0ae270 0x7f3fcc1521c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f3fcc143cd0 tx=0x7f3fb4007b90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:18.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.675+0000 7f3fc9ffb700 1 -- 172.21.15.36:0/3776384549 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fb4012400 con 0x7f3fcc0ae270 2022-06-17T12:56:18.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.675+0000 7f3fc9ffb700 1 -- 172.21.15.36:0/3776384549 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb4012560 con 0x7f3fcc0ae270 2022-06-17T12:56:18.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.675+0000 7f3fc9ffb700 1 -- 172.21.15.36:0/3776384549 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fb40128c0 con 0x7f3fcc0ae270 2022-06-17T12:56:18.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.675+0000 7f3fd304c700 1 -- 172.21.15.36:0/3776384549 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fcc143db0 con 0x7f3fcc0ae270 2022-06-17T12:56:18.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.675+0000 7f3fd304c700 1 -- 172.21.15.36:0/3776384549 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fcc0b5000 con 0x7f3fcc0ae270 2022-06-17T12:56:18.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.675+0000 7f3fd304c700 1 -- 172.21.15.36:0/3776384549 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fcc0b0100 con 0x7f3fcc0ae270 2022-06-17T12:56:18.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.675+0000 7f3fc9ffb700 1 -- 172.21.15.36:0/3776384549 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f3fb4019030 con 0x7f3fcc0ae270 2022-06-17T12:56:18.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.675+0000 7f3fc9ffb700 1 --2- 172.21.15.36:0/3776384549 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3fb0053ad0 0x7f3fb0055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:18.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.675+0000 7f3fc9ffb700 1 -- 172.21.15.36:0/3776384549 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3fb406b360 con 0x7f3fcc0ae270 2022-06-17T12:56:18.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.675+0000 7f3fd0de8700 1 --2- 172.21.15.36:0/3776384549 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3fb0053ad0 0x7f3fb0055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:18.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.675+0000 7f3fd0de8700 1 --2- 172.21.15.36:0/3776384549 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3fb0053ad0 0x7f3fb0055f90 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f3fbc01c000 tx=0x7f3fbc015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:18.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.683+0000 7f3fc9ffb700 1 -- 172.21.15.36:0/3776384549 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3fb4042050 con 0x7f3fcc0ae270 2022-06-17T12:56:18.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.815+0000 7f3fd304c700 1 -- 172.21.15.36:0/3776384549 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3fcc057410 con 0x7f3fcc0ae270 2022-06-17T12:56:18.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.815+0000 7f3fc9ffb700 1 -- 172.21.15.36:0/3776384549 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f3fb4045670 con 0x7f3fcc0ae270 2022-06-17T12:56:18.810 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:18.811 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":131,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:56:12.025286+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:56:08.924639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-06-17T12:56:09.989204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:56:18.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.819+0000 7f3fd304c700 1 -- 172.21.15.36:0/3776384549 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3fb0053ad0 msgr2=0x7f3fb0055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:18.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.819+0000 7f3fd304c700 1 --2- 172.21.15.36:0/3776384549 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3fb0053ad0 0x7f3fb0055f90 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f3fbc01c000 tx=0x7f3fbc015000).stop 2022-06-17T12:56:18.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.819+0000 7f3fd304c700 1 -- 172.21.15.36:0/3776384549 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3fcc0ae270 msgr2=0x7f3fcc1521c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:18.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.819+0000 7f3fd304c700 1 --2- 172.21.15.36:0/3776384549 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3fcc0ae270 0x7f3fcc1521c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f3fcc143cd0 tx=0x7f3fb4007b90).stop 2022-06-17T12:56:18.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.819+0000 7f3fd304c700 1 -- 172.21.15.36:0/3776384549 shutdown_connections 2022-06-17T12:56:18.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.819+0000 7f3fd304c700 1 --2- 172.21.15.36:0/3776384549 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3fb0053ad0 0x7f3fb0055f90 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:18.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.819+0000 7f3fd304c700 1 --2- 172.21.15.36:0/3776384549 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3fcc13fde0 0x7f3fcc0b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:18.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.819+0000 7f3fd304c700 1 --2- 172.21.15.36:0/3776384549 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3fcc0ae270 0x7f3fcc1521c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:18.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.819+0000 7f3fd304c700 1 --2- 172.21.15.36:0/3776384549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fcc0ad9c0 0x7f3fcc1436a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:18.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.819+0000 7f3fd304c700 1 -- 172.21.15.36:0/3776384549 >> 172.21.15.36:0/3776384549 conn(0x7f3fcc05c550 msgr2=0x7f3fcc140310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:18.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.819+0000 7f3fd304c700 1 -- 172.21.15.36:0/3776384549 shutdown_connections 2022-06-17T12:56:18.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.819+0000 7f3fd304c700 1 -- 172.21.15.36:0/3776384549 wait complete. 2022-06-17T12:56:18.824 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:56:08.924639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '130', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:56:09.989204+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:56:18.826 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:56:18.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.951+0000 7f1e92cc9700 1 Processor -- start 2022-06-17T12:56:18.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.955+0000 7f1e92cc9700 1 -- start start 2022-06-17T12:56:18.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.955+0000 7f1e92cc9700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e8c0b64f0 0x7f1e8c143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:18.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.955+0000 7f1e92cc9700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f1e8c0b4120 0x7f1e8c146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:18.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.955+0000 7f1e92cc9700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1e8c147270 0x7f1e8c140100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:18.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.955+0000 7f1e92cc9700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e8c0b68d0 con 0x7f1e8c0b4120 2022-06-17T12:56:18.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.955+0000 7f1e92cc9700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e8c1476d0 con 0x7f1e8c147270 2022-06-17T12:56:18.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.955+0000 7f1e92cc9700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c0b6e20 con 0x7f1e8c0b64f0 2022-06-17T12:56:18.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.955+0000 7f1e8bfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f1e8c0b4120 0x7f1e8c146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:18.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.955+0000 7f1e8bfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f1e8c0b4120 0x7f1e8c146d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39478/0 (socket says 172.21.15.36:39478) 2022-06-17T12:56:18.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.955+0000 7f1e8bfff700 1 -- 172.21.15.36:0/3103010930 learned_addr learned my addr 172.21.15.36:0/3103010930 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:18.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.955+0000 7f1e91266700 1 --2- 172.21.15.36:0/3103010930 >> v2:172.21.15.70:3300/0 conn(0x7f1e8c147270 0x7f1e8c140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:18.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.955+0000 7f1e90a65700 1 --2- 172.21.15.36:0/3103010930 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e8c0b64f0 0x7f1e8c143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:18.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.955+0000 7f1e8bfff700 1 -- 172.21.15.36:0/3103010930 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e8c0b64f0 msgr2=0x7f1e8c143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:18.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.955+0000 7f1e8bfff700 1 --2- 172.21.15.36:0/3103010930 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e8c0b64f0 0x7f1e8c143150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:18.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.955+0000 7f1e8bfff700 1 -- 172.21.15.36:0/3103010930 >> v2:172.21.15.70:3300/0 conn(0x7f1e8c147270 msgr2=0x7f1e8c140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:18.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.955+0000 7f1e8bfff700 1 --2- 172.21.15.36:0/3103010930 >> v2:172.21.15.70:3300/0 conn(0x7f1e8c147270 0x7f1e8c140100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:18.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.955+0000 7f1e8bfff700 1 -- 172.21.15.36:0/3103010930 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e8c1468e0 con 0x7f1e8c0b4120 2022-06-17T12:56:18.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.955+0000 7f1e91266700 1 --2- 172.21.15.36:0/3103010930 >> v2:172.21.15.70:3300/0 conn(0x7f1e8c147270 0x7f1e8c140100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:18.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.955+0000 7f1e8bfff700 1 --2- 172.21.15.36:0/3103010930 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1e8c0b4120 0x7f1e8c146d80 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7f1e7c002f70 tx=0x7f1e7c006eb0).ready entity=mon.0 client_cookie=cf84c1f46ba39387 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:18.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.955+0000 7f1e8b7fe700 1 -- 172.21.15.36:0/3103010930 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e7c00d570 con 0x7f1e8c0b4120 2022-06-17T12:56:18.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.955+0000 7f1e8b7fe700 1 -- 172.21.15.36:0/3103010930 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e7c000e90 con 0x7f1e8c0b4120 2022-06-17T12:56:18.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.955+0000 7f1e8b7fe700 1 -- 172.21.15.36:0/3103010930 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e7c00d900 con 0x7f1e8c0b4120 2022-06-17T12:56:18.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.955+0000 7f1e92cc9700 1 -- 172.21.15.36:0/3103010930 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1e8c0b4120 msgr2=0x7f1e8c146d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:18.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.955+0000 7f1e92cc9700 1 --2- 172.21.15.36:0/3103010930 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1e8c0b4120 0x7f1e8c146d80 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7f1e7c002f70 tx=0x7f1e7c006eb0).stop 2022-06-17T12:56:18.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.959+0000 7f1e92cc9700 1 -- 172.21.15.36:0/3103010930 shutdown_connections 2022-06-17T12:56:18.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.959+0000 7f1e92cc9700 1 --2- 172.21.15.36:0/3103010930 >> v2:172.21.15.70:3300/0 conn(0x7f1e8c147270 0x7f1e8c140100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:18.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.959+0000 7f1e92cc9700 1 --2- 172.21.15.36:0/3103010930 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1e8c0b4120 0x7f1e8c146d80 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:18.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.959+0000 7f1e92cc9700 1 --2- 172.21.15.36:0/3103010930 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e8c0b64f0 0x7f1e8c143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:18.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.959+0000 7f1e92cc9700 1 -- 172.21.15.36:0/3103010930 >> 172.21.15.36:0/3103010930 conn(0x7f1e8c0620f0 msgr2=0x7f1e8c062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:18.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.959+0000 7f1e92cc9700 1 -- 172.21.15.36:0/3103010930 shutdown_connections 2022-06-17T12:56:18.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.959+0000 7f1e92cc9700 1 -- 172.21.15.36:0/3103010930 wait complete. 2022-06-17T12:56:18.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.959+0000 7f1e92cc9700 1 Processor -- start 2022-06-17T12:56:18.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.959+0000 7f1e92cc9700 1 -- start start 2022-06-17T12:56:18.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.959+0000 7f1e92cc9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e8c0b4120 0x7f1e8c149e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:18.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.959+0000 7f1e92cc9700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e8c0b64f0 0x7f1e8c140bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:18.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.959+0000 7f1e92cc9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1e8c147270 0x7f1e8c1410e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:18.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.959+0000 7f1e92cc9700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c0b2fb0 con 0x7f1e8c147270 2022-06-17T12:56:18.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.959+0000 7f1e92cc9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c0b0eb0 con 0x7f1e8c0b4120 2022-06-17T12:56:18.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.959+0000 7f1e92cc9700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c0b10e0 con 0x7f1e8c0b64f0 2022-06-17T12:56:18.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.959+0000 7f1e91266700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1e8c147270 0x7f1e8c1410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:18.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.959+0000 7f1e91266700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1e8c147270 0x7f1e8c1410e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39486/0 (socket says 172.21.15.36:39486) 2022-06-17T12:56:18.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.959+0000 7f1e91266700 1 -- 172.21.15.36:0/532838885 learned_addr learned my addr 172.21.15.36:0/532838885 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:18.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.959+0000 7f1e8bfff700 1 --2- 172.21.15.36:0/532838885 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e8c0b64f0 0x7f1e8c140bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:18.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.959+0000 7f1e90a65700 1 --2- 172.21.15.36:0/532838885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e8c0b4120 0x7f1e8c149e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:18.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.963+0000 7f1e91266700 1 -- 172.21.15.36:0/532838885 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e8c0b64f0 msgr2=0x7f1e8c140bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:18.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.963+0000 7f1e91266700 1 --2- 172.21.15.36:0/532838885 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e8c0b64f0 0x7f1e8c140bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:18.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.963+0000 7f1e91266700 1 -- 172.21.15.36:0/532838885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e8c0b4120 msgr2=0x7f1e8c149e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:18.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.963+0000 7f1e91266700 1 --2- 172.21.15.36:0/532838885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e8c0b4120 0x7f1e8c149e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:18.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.963+0000 7f1e91266700 1 -- 172.21.15.36:0/532838885 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e84009470 con 0x7f1e8c147270 2022-06-17T12:56:18.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.963+0000 7f1e90a65700 1 --2- 172.21.15.36:0/532838885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e8c0b4120 0x7f1e8c149e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:56:18.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.963+0000 7f1e91266700 1 --2- 172.21.15.36:0/532838885 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1e8c147270 0x7f1e8c1410e0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0x7f1e8400c000 tx=0x7f1e840091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:18.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.963+0000 7f1e89ffb700 1 -- 172.21.15.36:0/532838885 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e8400c850 con 0x7f1e8c147270 2022-06-17T12:56:18.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.963+0000 7f1e89ffb700 1 -- 172.21.15.36:0/532838885 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e8400c9b0 con 0x7f1e8c147270 2022-06-17T12:56:18.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.963+0000 7f1e92cc9700 1 -- 172.21.15.36:0/532838885 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e7c00b1e0 con 0x7f1e8c147270 2022-06-17T12:56:18.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.963+0000 7f1e89ffb700 1 -- 172.21.15.36:0/532838885 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e8400cd10 con 0x7f1e8c147270 2022-06-17T12:56:18.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.963+0000 7f1e92cc9700 1 -- 172.21.15.36:0/532838885 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e8c14a730 con 0x7f1e8c147270 2022-06-17T12:56:18.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.963+0000 7f1e92cc9700 1 -- 172.21.15.36:0/532838885 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e8c143640 con 0x7f1e8c147270 2022-06-17T12:56:18.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.963+0000 7f1e89ffb700 1 -- 172.21.15.36:0/532838885 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f1e84022070 con 0x7f1e8c147270 2022-06-17T12:56:18.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.963+0000 7f1e89ffb700 1 --2- 172.21.15.36:0/532838885 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1e70053920 0x7f1e70055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:18.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.963+0000 7f1e89ffb700 1 -- 172.21.15.36:0/532838885 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1e8406c820 con 0x7f1e8c147270 2022-06-17T12:56:18.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.963+0000 7f1e90a65700 1 --2- 172.21.15.36:0/532838885 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1e70053920 0x7f1e70055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:18.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.967+0000 7f1e90a65700 1 --2- 172.21.15.36:0/532838885 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1e70053920 0x7f1e70055de0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f1e7400b000 tx=0x7f1e74008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:18.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:18.971+0000 7f1e89ffb700 1 -- 172.21.15.36:0/532838885 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1e84043510 con 0x7f1e8c147270 2022-06-17T12:56:19.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.103+0000 7f1e92cc9700 1 -- 172.21.15.36:0/532838885 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f1e8c057410 con 0x7f1e8c147270 2022-06-17T12:56:19.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.107+0000 7f1e89ffb700 1 -- 172.21.15.36:0/532838885 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v131) v1 ==== 110+0+10 (secure 0 0 0) 0x7f1e84016020 con 0x7f1e8c147270 2022-06-17T12:56:19.095 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:56:19.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.107+0000 7f1e92cc9700 1 -- 172.21.15.36:0/532838885 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1e70053920 msgr2=0x7f1e70055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:19.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.107+0000 7f1e92cc9700 1 --2- 172.21.15.36:0/532838885 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1e70053920 0x7f1e70055de0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f1e7400b000 tx=0x7f1e74008040).stop 2022-06-17T12:56:19.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.107+0000 7f1e92cc9700 1 -- 172.21.15.36:0/532838885 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1e8c147270 msgr2=0x7f1e8c1410e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:19.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.107+0000 7f1e92cc9700 1 --2- 172.21.15.36:0/532838885 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1e8c147270 0x7f1e8c1410e0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0x7f1e8400c000 tx=0x7f1e840091d0).stop 2022-06-17T12:56:19.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.107+0000 7f1e92cc9700 1 -- 172.21.15.36:0/532838885 shutdown_connections 2022-06-17T12:56:19.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.107+0000 7f1e92cc9700 1 --2- 172.21.15.36:0/532838885 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1e70053920 0x7f1e70055de0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.107+0000 7f1e92cc9700 1 --2- 172.21.15.36:0/532838885 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1e8c147270 0x7f1e8c1410e0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.107+0000 7f1e92cc9700 1 --2- 172.21.15.36:0/532838885 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e8c0b64f0 0x7f1e8c140bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.107+0000 7f1e92cc9700 1 --2- 172.21.15.36:0/532838885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e8c0b4120 0x7f1e8c149e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.107+0000 7f1e92cc9700 1 -- 172.21.15.36:0/532838885 >> 172.21.15.36:0/532838885 conn(0x7f1e8c0620f0 msgr2=0x7f1e8c143840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:19.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.107+0000 7f1e92cc9700 1 -- 172.21.15.36:0/532838885 shutdown_connections 2022-06-17T12:56:19.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.107+0000 7f1e92cc9700 1 -- 172.21.15.36:0/532838885 wait complete. 2022-06-17T12:56:19.112 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:56:19.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f425753a700 1 Processor -- start 2022-06-17T12:56:19.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f425753a700 1 -- start start 2022-06-17T12:56:19.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f425753a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f42500b45c0 0x7f42500b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:19.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f425753a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42500b4f00 0x7f42500b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:19.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f425753a700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42500b4160 con 0x7f42500b9860 2022-06-17T12:56:19.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f425753a700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42500b9c40 con 0x7f42500b45c0 2022-06-17T12:56:19.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f425753a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42500627d0 con 0x7f42500b4f00 2022-06-17T12:56:19.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f4255d37700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f42500b45c0 0x7f42500b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:19.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f4255d37700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f42500b45c0 0x7f42500b4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39492/0 (socket says 172.21.15.36:39492) 2022-06-17T12:56:19.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f4255d37700 1 -- 172.21.15.36:0/2907563481 learned_addr learned my addr 172.21.15.36:0/2907563481 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:19.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f4256d39700 1 --2- 172.21.15.36:0/2907563481 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42500b4f00 0x7f42500b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:19.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f4255d37700 1 -- 172.21.15.36:0/2907563481 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42500b4f00 msgr2=0x7f42500b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:19.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f4255d37700 1 --2- 172.21.15.36:0/2907563481 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42500b4f00 0x7f42500b5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f4255d37700 1 -- 172.21.15.36:0/2907563481 >> v1:172.21.15.70:6789/0 conn(0x7f42500b9860 legacy=0x7f4250143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:19.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f4255d37700 1 -- 172.21.15.36:0/2907563481 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4250062a70 con 0x7f42500b45c0 2022-06-17T12:56:19.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f4255d37700 1 --2- 172.21.15.36:0/2907563481 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f42500b45c0 0x7f42500b4a10 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 rx=0x7f4240002f70 tx=0x7f4240006eb0).ready entity=mon.0 client_cookie=27f19ae6ae1b6666 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:19.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f4255536700 1 -- 172.21.15.36:0/2907563481 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f424000d570 con 0x7f42500b45c0 2022-06-17T12:56:19.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f4255536700 1 -- 172.21.15.36:0/2907563481 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4240000e90 con 0x7f42500b45c0 2022-06-17T12:56:19.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f4255536700 1 -- 172.21.15.36:0/2907563481 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f424000d900 con 0x7f42500b45c0 2022-06-17T12:56:19.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f425753a700 1 -- 172.21.15.36:0/2907563481 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f42500b45c0 msgr2=0x7f42500b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:19.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f425753a700 1 --2- 172.21.15.36:0/2907563481 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f42500b45c0 0x7f42500b4a10 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 rx=0x7f4240002f70 tx=0x7f4240006eb0).stop 2022-06-17T12:56:19.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f425753a700 1 -- 172.21.15.36:0/2907563481 shutdown_connections 2022-06-17T12:56:19.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f425753a700 1 --2- 172.21.15.36:0/2907563481 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42500b4f00 0x7f42500b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f425753a700 1 --2- 172.21.15.36:0/2907563481 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f42500b45c0 0x7f42500b4a10 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.243+0000 7f425753a700 1 -- 172.21.15.36:0/2907563481 >> 172.21.15.36:0/2907563481 conn(0x7f425005c550 msgr2=0x7f4250060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:19.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f425753a700 1 -- 172.21.15.36:0/2907563481 shutdown_connections 2022-06-17T12:56:19.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f425753a700 1 -- 172.21.15.36:0/2907563481 wait complete. 2022-06-17T12:56:19.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f425753a700 1 Processor -- start 2022-06-17T12:56:19.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f425753a700 1 -- start start 2022-06-17T12:56:19.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f425753a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f42500b45c0 0x7f42500afe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:19.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f425753a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42500b4f00 0x7f4250155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:19.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f425753a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42500b9860 0x7f425013dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:19.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f425753a700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425013d590 con 0x7f42500b45c0 2022-06-17T12:56:19.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f425753a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425013d830 con 0x7f42500b9860 2022-06-17T12:56:19.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f425753a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4250156500 con 0x7f42500b4f00 2022-06-17T12:56:19.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f4256538700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f42500b45c0 0x7f42500afe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:19.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f4256538700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f42500b45c0 0x7f42500afe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39496/0 (socket says 172.21.15.36:39496) 2022-06-17T12:56:19.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f4256538700 1 -- 172.21.15.36:0/3953723744 learned_addr learned my addr 172.21.15.36:0/3953723744 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:19.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f4255d37700 1 --2- 172.21.15.36:0/3953723744 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42500b4f00 0x7f4250155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:19.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f4256d39700 1 --2- 172.21.15.36:0/3953723744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42500b9860 0x7f425013dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:19.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f4256538700 1 -- 172.21.15.36:0/3953723744 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42500b4f00 msgr2=0x7f4250155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:19.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f4256538700 1 --2- 172.21.15.36:0/3953723744 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42500b4f00 0x7f4250155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f4256538700 1 -- 172.21.15.36:0/3953723744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42500b9860 msgr2=0x7f425013dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:19.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f4256538700 1 --2- 172.21.15.36:0/3953723744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42500b9860 0x7f425013dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f4256538700 1 -- 172.21.15.36:0/3953723744 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f424000b1e0 con 0x7f42500b45c0 2022-06-17T12:56:19.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f4256d39700 1 --2- 172.21.15.36:0/3953723744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42500b9860 0x7f425013dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:56:19.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f4256538700 1 --2- 172.21.15.36:0/3953723744 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f42500b45c0 0x7f42500afe60 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7f4250062280 tx=0x7f424c007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:19.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f423f7fe700 1 -- 172.21.15.36:0/3953723744 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f424c013450 con 0x7f42500b45c0 2022-06-17T12:56:19.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f423f7fe700 1 -- 172.21.15.36:0/3953723744 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424c0135b0 con 0x7f42500b45c0 2022-06-17T12:56:19.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f425753a700 1 -- 172.21.15.36:0/3953723744 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42500aecc0 con 0x7f42500b45c0 2022-06-17T12:56:19.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f423f7fe700 1 -- 172.21.15.36:0/3953723744 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f424c013910 con 0x7f42500b45c0 2022-06-17T12:56:19.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.247+0000 7f425753a700 1 -- 172.21.15.36:0/3953723744 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4250152100 con 0x7f42500b45c0 2022-06-17T12:56:19.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.251+0000 7f423f7fe700 1 -- 172.21.15.36:0/3953723744 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f424c01a060 con 0x7f42500b45c0 2022-06-17T12:56:19.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.251+0000 7f425753a700 1 -- 172.21.15.36:0/3953723744 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f425013d9f0 con 0x7f42500b45c0 2022-06-17T12:56:19.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.251+0000 7f423f7fe700 1 --2- 172.21.15.36:0/3953723744 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4230053ad0 0x7f4230055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:19.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.251+0000 7f423f7fe700 1 -- 172.21.15.36:0/3953723744 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f424c06b830 con 0x7f42500b45c0 2022-06-17T12:56:19.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.251+0000 7f4255d37700 1 --2- 172.21.15.36:0/3953723744 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4230053ad0 0x7f4230055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:19.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.251+0000 7f4255d37700 1 --2- 172.21.15.36:0/3953723744 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4230053ad0 0x7f4230055f90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f424001c000 tx=0x7f4240015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:19.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.259+0000 7f423f7fe700 1 -- 172.21.15.36:0/3953723744 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f424c042520 con 0x7f42500b45c0 2022-06-17T12:56:19.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.383+0000 7f425753a700 1 -- 172.21.15.36:0/3953723744 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4250062eb0 con 0x7f42500b45c0 2022-06-17T12:56:19.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.383+0000 7f423f7fe700 1 -- 172.21.15.36:0/3953723744 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v131) v1 ==== 92+0+10 (secure 0 0 0) 0x7f424c018070 con 0x7f42500b45c0 2022-06-17T12:56:19.375 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:56:19.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.387+0000 7f425753a700 1 -- 172.21.15.36:0/3953723744 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4230053ad0 msgr2=0x7f4230055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:19.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.387+0000 7f425753a700 1 --2- 172.21.15.36:0/3953723744 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4230053ad0 0x7f4230055f90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f424001c000 tx=0x7f4240015000).stop 2022-06-17T12:56:19.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.387+0000 7f425753a700 1 -- 172.21.15.36:0/3953723744 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f42500b45c0 msgr2=0x7f42500afe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:19.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.387+0000 7f425753a700 1 --2- 172.21.15.36:0/3953723744 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f42500b45c0 0x7f42500afe60 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7f4250062280 tx=0x7f424c007c20).stop 2022-06-17T12:56:19.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.387+0000 7f425753a700 1 -- 172.21.15.36:0/3953723744 shutdown_connections 2022-06-17T12:56:19.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.387+0000 7f425753a700 1 --2- 172.21.15.36:0/3953723744 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4230053ad0 0x7f4230055f90 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.387+0000 7f425753a700 1 --2- 172.21.15.36:0/3953723744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42500b9860 0x7f425013dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.387+0000 7f425753a700 1 --2- 172.21.15.36:0/3953723744 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42500b4f00 0x7f4250155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.387+0000 7f425753a700 1 --2- 172.21.15.36:0/3953723744 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f42500b45c0 0x7f42500afe60 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.387+0000 7f425753a700 1 -- 172.21.15.36:0/3953723744 >> 172.21.15.36:0/3953723744 conn(0x7f425005c550 msgr2=0x7f42500b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:19.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.387+0000 7f425753a700 1 -- 172.21.15.36:0/3953723744 shutdown_connections 2022-06-17T12:56:19.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.387+0000 7f425753a700 1 -- 172.21.15.36:0/3953723744 wait complete. 2022-06-17T12:56:19.387 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:56:19.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.511+0000 7f1e8dea0700 1 Processor -- start 2022-06-17T12:56:19.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.511+0000 7f1e8dea0700 1 -- start start 2022-06-17T12:56:19.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.511+0000 7f1e8dea0700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f1e880601c0 0x7f1e881443d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:19.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.511+0000 7f1e8dea0700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e880b2b60 con 0x7f1e880ae360 2022-06-17T12:56:19.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.511+0000 7f1e8dea0700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e880b3230 con 0x7f1e88144fd0 2022-06-17T12:56:19.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.511+0000 7f1e8dea0700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e880605a0 con 0x7f1e880601c0 2022-06-17T12:56:19.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.511+0000 7f1e8ce9e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f1e880601c0 0x7f1e881443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:19.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.511+0000 7f1e87fff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f1e880ae360 0x7f1e881448c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57036/0 (socket says 172.21.15.36:57036) 2022-06-17T12:56:19.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.511+0000 7f1e87fff700 1 -- 172.21.15.36:0/1777264701 learned_addr learned my addr 172.21.15.36:0/1777264701 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:19.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.511+0000 7f1e877fe700 1 -- 172.21.15.36:0/1777264701 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1364839336 0 0) 0x7f1e880b2b60 con 0x7f1e880ae360 2022-06-17T12:56:19.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.511+0000 7f1e877fe700 1 -- 172.21.15.36:0/1777264701 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e74003260 con 0x7f1e880ae360 2022-06-17T12:56:19.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.511+0000 7f1e8ce9e700 1 -- 172.21.15.36:0/1777264701 >> v1:172.21.15.70:6789/0 conn(0x7f1e88144fd0 legacy=0x7f1e8814aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:19.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.511+0000 7f1e877fe700 1 -- 172.21.15.36:0/1777264701 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1690020743 0 0) 0x7f1e80001490 con 0x7f1e88144fd0 2022-06-17T12:56:19.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.511+0000 7f1e8ce9e700 1 -- 172.21.15.36:0/1777264701 >> v1:172.21.15.36:6789/0 conn(0x7f1e880ae360 legacy=0x7f1e881448c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:19.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.511+0000 7f1e8ce9e700 1 -- 172.21.15.36:0/1777264701 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e880662f0 con 0x7f1e880601c0 2022-06-17T12:56:19.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.511+0000 7f1e8ce9e700 1 --2- 172.21.15.36:0/1777264701 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1e880601c0 0x7f1e881443d0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7f1e78001800 tx=0x7f1e7c006df0).ready entity=mon.0 client_cookie=ae584d489515307b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:19.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.511+0000 7f1e877fe700 1 -- 172.21.15.36:0/1777264701 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e78001660 con 0x7f1e880601c0 2022-06-17T12:56:19.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.511+0000 7f1e877fe700 1 -- 172.21.15.36:0/1777264701 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e7c000e90 con 0x7f1e880601c0 2022-06-17T12:56:19.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.511+0000 7f1e877fe700 1 -- 172.21.15.36:0/1777264701 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e7c011400 con 0x7f1e880601c0 2022-06-17T12:56:19.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.515+0000 7f1e8dea0700 1 -- 172.21.15.36:0/1777264701 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1e880601c0 msgr2=0x7f1e881443d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:19.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.515+0000 7f1e8dea0700 1 --2- 172.21.15.36:0/1777264701 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1e880601c0 0x7f1e881443d0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7f1e78001800 tx=0x7f1e7c006df0).stop 2022-06-17T12:56:19.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.515+0000 7f1e8dea0700 1 -- 172.21.15.36:0/1777264701 shutdown_connections 2022-06-17T12:56:19.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.515+0000 7f1e8dea0700 1 --2- 172.21.15.36:0/1777264701 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1e880601c0 0x7f1e881443d0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.515+0000 7f1e8dea0700 1 -- 172.21.15.36:0/1777264701 >> 172.21.15.36:0/1777264701 conn(0x7f1e88060990 msgr2=0x7f1e8805f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:19.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.515+0000 7f1e8dea0700 1 -- 172.21.15.36:0/1777264701 shutdown_connections 2022-06-17T12:56:19.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.515+0000 7f1e8dea0700 1 -- 172.21.15.36:0/1777264701 wait complete. 2022-06-17T12:56:19.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.515+0000 7f1e8dea0700 1 Processor -- start 2022-06-17T12:56:19.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.515+0000 7f1e8dea0700 1 -- start start 2022-06-17T12:56:19.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.515+0000 7f1e8dea0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e880601c0 0x7f1e881436a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:19.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.515+0000 7f1e8dea0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1e880ae360 0x7f1e880afb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:19.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.515+0000 7f1e8dea0700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e88144fd0 0x7f1e88152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:19.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.515+0000 7f1e8dea0700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e88152380 con 0x7f1e880ae360 2022-06-17T12:56:19.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.515+0000 7f1e8dea0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e880b0c90 con 0x7f1e880601c0 2022-06-17T12:56:19.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.515+0000 7f1e8dea0700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e880b0f30 con 0x7f1e88144fd0 2022-06-17T12:56:19.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.515+0000 7f1e87fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1e880ae360 0x7f1e880afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:19.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.519+0000 7f1e87fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1e880ae360 0x7f1e880afb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39510/0 (socket says 172.21.15.36:39510) 2022-06-17T12:56:19.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.519+0000 7f1e87fff700 1 -- 172.21.15.36:0/1055128423 learned_addr learned my addr 172.21.15.36:0/1055128423 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:19.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.519+0000 7f1e8d69f700 1 --2- 172.21.15.36:0/1055128423 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e88144fd0 0x7f1e88152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:19.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.519+0000 7f1e8ce9e700 1 --2- 172.21.15.36:0/1055128423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e880601c0 0x7f1e881436a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:19.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.519+0000 7f1e87fff700 1 -- 172.21.15.36:0/1055128423 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e88144fd0 msgr2=0x7f1e88152910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:19.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.519+0000 7f1e87fff700 1 --2- 172.21.15.36:0/1055128423 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e88144fd0 0x7f1e88152910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.519+0000 7f1e87fff700 1 -- 172.21.15.36:0/1055128423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e880601c0 msgr2=0x7f1e881436a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:19.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.519+0000 7f1e87fff700 1 --2- 172.21.15.36:0/1055128423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e880601c0 0x7f1e881436a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.519+0000 7f1e87fff700 1 -- 172.21.15.36:0/1055128423 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7c00b180 con 0x7f1e880ae360 2022-06-17T12:56:19.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.519+0000 7f1e87fff700 1 --2- 172.21.15.36:0/1055128423 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1e880ae360 0x7f1e880afb50 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f1e780011d0 tx=0x7f1e78001ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:19.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.519+0000 7f1e85ffb700 1 -- 172.21.15.36:0/1055128423 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e78007170 con 0x7f1e880ae360 2022-06-17T12:56:19.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.519+0000 7f1e85ffb700 1 -- 172.21.15.36:0/1055128423 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e780072d0 con 0x7f1e880ae360 2022-06-17T12:56:19.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.519+0000 7f1e8dea0700 1 -- 172.21.15.36:0/1055128423 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e880b11d0 con 0x7f1e880ae360 2022-06-17T12:56:19.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.519+0000 7f1e85ffb700 1 -- 172.21.15.36:0/1055128423 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e78004920 con 0x7f1e880ae360 2022-06-17T12:56:19.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.519+0000 7f1e8dea0700 1 -- 172.21.15.36:0/1055128423 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e88143b90 con 0x7f1e880ae360 2022-06-17T12:56:19.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.519+0000 7f1e8dea0700 1 -- 172.21.15.36:0/1055128423 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e88066590 con 0x7f1e880ae360 2022-06-17T12:56:19.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.523+0000 7f1e85ffb700 1 -- 172.21.15.36:0/1055128423 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f1e7801b070 con 0x7f1e880ae360 2022-06-17T12:56:19.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.523+0000 7f1e85ffb700 1 --2- 172.21.15.36:0/1055128423 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1e68053ad0 0x7f1e68055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:19.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.523+0000 7f1e85ffb700 1 -- 172.21.15.36:0/1055128423 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1e7806b710 con 0x7f1e880ae360 2022-06-17T12:56:19.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.523+0000 7f1e8ce9e700 1 --2- 172.21.15.36:0/1055128423 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1e68053ad0 0x7f1e68055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:19.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.523+0000 7f1e8ce9e700 1 --2- 172.21.15.36:0/1055128423 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1e68053ad0 0x7f1e68055f90 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f1e7c019000 tx=0x7f1e88066480).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:19.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.527+0000 7f1e85ffb700 1 -- 172.21.15.36:0/1055128423 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1e78042510 con 0x7f1e880ae360 2022-06-17T12:56:19.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.647+0000 7f1e8dea0700 1 -- 172.21.15.36:0/1055128423 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1e88057410 con 0x7f1e880ae360 2022-06-17T12:56:19.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.651+0000 7f1e85ffb700 1 -- 172.21.15.36:0/1055128423 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v131) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1e78045b30 con 0x7f1e880ae360 2022-06-17T12:56:19.641 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:56:19.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.651+0000 7f1e8dea0700 1 -- 172.21.15.36:0/1055128423 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1e68053ad0 msgr2=0x7f1e68055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:19.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.651+0000 7f1e8dea0700 1 --2- 172.21.15.36:0/1055128423 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1e68053ad0 0x7f1e68055f90 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f1e7c019000 tx=0x7f1e88066480).stop 2022-06-17T12:56:19.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.651+0000 7f1e8dea0700 1 -- 172.21.15.36:0/1055128423 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1e880ae360 msgr2=0x7f1e880afb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:19.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.651+0000 7f1e8dea0700 1 --2- 172.21.15.36:0/1055128423 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1e880ae360 0x7f1e880afb50 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f1e780011d0 tx=0x7f1e78001ae0).stop 2022-06-17T12:56:19.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.651+0000 7f1e8dea0700 1 -- 172.21.15.36:0/1055128423 shutdown_connections 2022-06-17T12:56:19.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.651+0000 7f1e8dea0700 1 --2- 172.21.15.36:0/1055128423 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1e68053ad0 0x7f1e68055f90 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.651+0000 7f1e8dea0700 1 --2- 172.21.15.36:0/1055128423 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e88144fd0 0x7f1e88152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.651+0000 7f1e8dea0700 1 --2- 172.21.15.36:0/1055128423 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1e880ae360 0x7f1e880afb50 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.651+0000 7f1e8dea0700 1 --2- 172.21.15.36:0/1055128423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e880601c0 0x7f1e881436a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.651+0000 7f1e8dea0700 1 -- 172.21.15.36:0/1055128423 >> 172.21.15.36:0/1055128423 conn(0x7f1e88060990 msgr2=0x7f1e8813f300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:19.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.651+0000 7f1e8dea0700 1 -- 172.21.15.36:0/1055128423 shutdown_connections 2022-06-17T12:56:19.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.651+0000 7f1e8dea0700 1 -- 172.21.15.36:0/1055128423 wait complete. 2022-06-17T12:56:19.664 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:56:19.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.799+0000 7f22d7fff700 1 Processor -- start 2022-06-17T12:56:19.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.799+0000 7f22d7fff700 1 -- start start 2022-06-17T12:56:19.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.799+0000 7f22d7fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f22d814b2b0 0x7f22d8060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:19.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.799+0000 7f22d7fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22d8147bd0 0x7f22d81442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:19.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.799+0000 7f22d7fff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d814b870 con 0x7f22d80b27b0 2022-06-17T12:56:19.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.799+0000 7f22d7fff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d80ac500 con 0x7f22d814b2b0 2022-06-17T12:56:19.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.799+0000 7f22d7fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d80ace90 con 0x7f22d8147bd0 2022-06-17T12:56:19.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.799+0000 7f22d67fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f22d814b2b0 0x7f22d8060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:19.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.799+0000 7f22d67fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f22d814b2b0 0x7f22d8060ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39520/0 (socket says 172.21.15.36:39520) 2022-06-17T12:56:19.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.799+0000 7f22d67fc700 1 -- 172.21.15.36:0/2708875702 learned_addr learned my addr 172.21.15.36:0/2708875702 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:19.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.799+0000 7f22d6ffd700 1 --1- 172.21.15.36:0/2708875702 >> v1:172.21.15.36:6789/0 conn(0x7f22d80b27b0 0x7f22d80ac780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57048/0 (socket says 172.21.15.36:57048) 2022-06-17T12:56:19.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.799+0000 7f22d77fe700 1 --2- 172.21.15.36:0/2708875702 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22d8147bd0 0x7f22d81442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:19.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d67fc700 1 -- 172.21.15.36:0/2708875702 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22d8147bd0 msgr2=0x7f22d81442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:19.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d67fc700 1 --2- 172.21.15.36:0/2708875702 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22d8147bd0 0x7f22d81442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d67fc700 1 -- 172.21.15.36:0/2708875702 >> v1:172.21.15.36:6789/0 conn(0x7f22d80b27b0 legacy=0x7f22d80ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:19.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d5ffb700 1 -- 172.21.15.36:0/2708875702 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1541385820 0 0) 0x7f22d814b870 con 0x7f22d80b27b0 2022-06-17T12:56:19.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d67fc700 1 -- 172.21.15.36:0/2708875702 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d80b2b90 con 0x7f22d814b2b0 2022-06-17T12:56:19.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d67fc700 1 --2- 172.21.15.36:0/2708875702 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22d814b2b0 0x7f22d8060ca0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7f22c8002f70 tx=0x7f22c8006eb0).ready entity=mon.0 client_cookie=c0e714110213d646 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:19.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d5ffb700 1 -- 172.21.15.36:0/2708875702 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22c800d570 con 0x7f22d814b2b0 2022-06-17T12:56:19.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d5ffb700 1 -- 172.21.15.36:0/2708875702 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c8000e90 con 0x7f22d814b2b0 2022-06-17T12:56:19.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d5ffb700 1 -- 172.21.15.36:0/2708875702 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22c800d8c0 con 0x7f22d814b2b0 2022-06-17T12:56:19.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d7fff700 1 -- 172.21.15.36:0/2708875702 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22d814b2b0 msgr2=0x7f22d8060ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:19.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d7fff700 1 --2- 172.21.15.36:0/2708875702 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22d814b2b0 0x7f22d8060ca0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7f22c8002f70 tx=0x7f22c8006eb0).stop 2022-06-17T12:56:19.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d7fff700 1 -- 172.21.15.36:0/2708875702 shutdown_connections 2022-06-17T12:56:19.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d7fff700 1 --2- 172.21.15.36:0/2708875702 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22d8147bd0 0x7f22d81442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d7fff700 1 --2- 172.21.15.36:0/2708875702 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22d814b2b0 0x7f22d8060ca0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d7fff700 1 -- 172.21.15.36:0/2708875702 >> 172.21.15.36:0/2708875702 conn(0x7f22d80b5d10 msgr2=0x7f22d80b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:19.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d7fff700 1 -- 172.21.15.36:0/2708875702 shutdown_connections 2022-06-17T12:56:19.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d7fff700 1 -- 172.21.15.36:0/2708875702 wait complete. 2022-06-17T12:56:19.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d7fff700 1 Processor -- start 2022-06-17T12:56:19.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d7fff700 1 -- start start 2022-06-17T12:56:19.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d7fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22d80b27b0 0x7f22d81571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:19.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d7fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22d8147bd0 0x7f22d80aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:19.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d814b2b0 0x7f22d8155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:19.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d7fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d80aad90 con 0x7f22d80b27b0 2022-06-17T12:56:19.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d7fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d80ab030 con 0x7f22d814b2b0 2022-06-17T12:56:19.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d7fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d8066170 con 0x7f22d8147bd0 2022-06-17T12:56:19.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.803+0000 7f22d6ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22d80b27b0 0x7f22d81571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:19.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.807+0000 7f22d67fc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22d8147bd0 0x7f22d80aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:19.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.807+0000 7f22d6ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22d80b27b0 0x7f22d81571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39524/0 (socket says 172.21.15.36:39524) 2022-06-17T12:56:19.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.807+0000 7f22d67fc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22d8147bd0 0x7f22d80aa4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:54316/0 (socket says 172.21.15.36:54316) 2022-06-17T12:56:19.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.807+0000 7f22d6ffd700 1 -- 172.21.15.36:0/145314738 learned_addr learned my addr 172.21.15.36:0/145314738 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:19.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.807+0000 7f22d77fe700 1 --2- 172.21.15.36:0/145314738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d814b2b0 0x7f22d8155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:19.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.807+0000 7f22d6ffd700 1 -- 172.21.15.36:0/145314738 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22d8147bd0 msgr2=0x7f22d80aa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:19.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.807+0000 7f22d6ffd700 1 --2- 172.21.15.36:0/145314738 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22d8147bd0 0x7f22d80aa4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.807+0000 7f22d6ffd700 1 -- 172.21.15.36:0/145314738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d814b2b0 msgr2=0x7f22d8155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:19.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.807+0000 7f22d6ffd700 1 --2- 172.21.15.36:0/145314738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d814b2b0 0x7f22d8155aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.807+0000 7f22d6ffd700 1 -- 172.21.15.36:0/145314738 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c800b1e0 con 0x7f22d80b27b0 2022-06-17T12:56:19.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.807+0000 7f22d6ffd700 1 --2- 172.21.15.36:0/145314738 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22d80b27b0 0x7f22d81571c0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0x7f22cc000ed0 tx=0x7f22cc007ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:19.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.807+0000 7f22c7fff700 1 -- 172.21.15.36:0/145314738 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22cc007170 con 0x7f22d80b27b0 2022-06-17T12:56:19.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.807+0000 7f22c7fff700 1 -- 172.21.15.36:0/145314738 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22cc0072d0 con 0x7f22d80b27b0 2022-06-17T12:56:19.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.807+0000 7f22c7fff700 1 -- 172.21.15.36:0/145314738 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22cc004710 con 0x7f22d80b27b0 2022-06-17T12:56:19.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.807+0000 7f22d7fff700 1 -- 172.21.15.36:0/145314738 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d80aa9b0 con 0x7f22d80b27b0 2022-06-17T12:56:19.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.807+0000 7f22d7fff700 1 -- 172.21.15.36:0/145314738 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d80a70e0 con 0x7f22d80b27b0 2022-06-17T12:56:19.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.807+0000 7f22d7fff700 1 -- 172.21.15.36:0/145314738 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d80ab1a0 con 0x7f22d80b27b0 2022-06-17T12:56:19.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.811+0000 7f22c7fff700 1 -- 172.21.15.36:0/145314738 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f22cc004870 con 0x7f22d80b27b0 2022-06-17T12:56:19.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.811+0000 7f22c7fff700 1 --2- 172.21.15.36:0/145314738 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f22b4053a40 0x7f22b4055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:19.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.811+0000 7f22c7fff700 1 -- 172.21.15.36:0/145314738 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f22cc06b920 con 0x7f22d80b27b0 2022-06-17T12:56:19.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.811+0000 7f22d67fc700 1 --2- 172.21.15.36:0/145314738 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f22b4053a40 0x7f22b4055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:19.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.811+0000 7f22d67fc700 1 --2- 172.21.15.36:0/145314738 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f22b4053a40 0x7f22b4055f00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f22c801c000 tx=0x7f22c8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:19.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.819+0000 7f22c7fff700 1 -- 172.21.15.36:0/145314738 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f22cc042610 con 0x7f22d80b27b0 2022-06-17T12:56:19.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.943+0000 7f22d7fff700 1 -- 172.21.15.36:0/145314738 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f22d8057410 con 0x7f22d80b27b0 2022-06-17T12:56:19.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.943+0000 7f22c7fff700 1 -- 172.21.15.36:0/145314738 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v131) v1 ==== 100+0+10 (secure 0 0 0) 0x7f22cc045c30 con 0x7f22d80b27b0 2022-06-17T12:56:19.933 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:56:19.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.947+0000 7f22d7fff700 1 -- 172.21.15.36:0/145314738 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f22b4053a40 msgr2=0x7f22b4055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:19.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.947+0000 7f22d7fff700 1 --2- 172.21.15.36:0/145314738 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f22b4053a40 0x7f22b4055f00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f22c801c000 tx=0x7f22c8015000).stop 2022-06-17T12:56:19.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.947+0000 7f22d7fff700 1 -- 172.21.15.36:0/145314738 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22d80b27b0 msgr2=0x7f22d81571c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:19.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.947+0000 7f22d7fff700 1 --2- 172.21.15.36:0/145314738 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22d80b27b0 0x7f22d81571c0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0x7f22cc000ed0 tx=0x7f22cc007ab0).stop 2022-06-17T12:56:19.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.947+0000 7f22d7fff700 1 -- 172.21.15.36:0/145314738 shutdown_connections 2022-06-17T12:56:19.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.947+0000 7f22d7fff700 1 --2- 172.21.15.36:0/145314738 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f22b4053a40 0x7f22b4055f00 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.947+0000 7f22d7fff700 1 --2- 172.21.15.36:0/145314738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d814b2b0 0x7f22d8155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.947+0000 7f22d7fff700 1 --2- 172.21.15.36:0/145314738 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22d8147bd0 0x7f22d80aa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.947+0000 7f22d7fff700 1 --2- 172.21.15.36:0/145314738 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22d80b27b0 0x7f22d81571c0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:19.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.947+0000 7f22d7fff700 1 -- 172.21.15.36:0/145314738 >> 172.21.15.36:0/145314738 conn(0x7f22d80b5d10 msgr2=0x7f22d8145b50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:19.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.947+0000 7f22d7fff700 1 -- 172.21.15.36:0/145314738 shutdown_connections 2022-06-17T12:56:19.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:19.947+0000 7f22d7fff700 1 -- 172.21.15.36:0/145314738 wait complete. 2022-06-17T12:56:19.967 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:56:19.975 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:56:20.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.127+0000 7fb745672700 1 Processor -- start 2022-06-17T12:56:20.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.127+0000 7fb745672700 1 -- start start 2022-06-17T12:56:20.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.127+0000 7fb745672700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb740148060 0x7fb7400b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:20.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.127+0000 7fb745672700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb7400ae360 0x7fb7400b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:20.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.127+0000 7fb745672700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb740062e10 con 0x7fb7400b7e80 2022-06-17T12:56:20.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.127+0000 7fb745672700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7401472c0 con 0x7fb7400ae360 2022-06-17T12:56:20.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.127+0000 7fb745672700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7400630b0 con 0x7fb740148060 2022-06-17T12:56:20.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.127+0000 7fb73ffff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fb7400b7e80 0x7fb740147950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57062/0 (socket says 172.21.15.36:57062) 2022-06-17T12:56:20.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.127+0000 7fb73ffff700 1 -- 172.21.15.36:0/408867494 learned_addr learned my addr 172.21.15.36:0/408867494 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:20.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.127+0000 7fb744e71700 1 --2- 172.21.15.36:0/408867494 >> v2:172.21.15.36:3300/0 conn(0x7fb7400ae360 0x7fb7400b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:20.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.127+0000 7fb73f7fe700 1 --2- 172.21.15.36:0/408867494 >> v2:172.21.15.70:3300/0 conn(0x7fb740148060 0x7fb7400b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:20.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.127+0000 7fb73effd700 1 -- 172.21.15.36:0/408867494 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3947628765 0 0) 0x7fb740062e10 con 0x7fb7400b7e80 2022-06-17T12:56:20.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.127+0000 7fb73effd700 1 -- 172.21.15.36:0/408867494 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb720003260 con 0x7fb7400b7e80 2022-06-17T12:56:20.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.127+0000 7fb744e71700 1 -- 172.21.15.36:0/408867494 >> v2:172.21.15.70:3300/0 conn(0x7fb740148060 msgr2=0x7fb7400b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:20.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.127+0000 7fb744e71700 1 --2- 172.21.15.36:0/408867494 >> v2:172.21.15.70:3300/0 conn(0x7fb740148060 0x7fb7400b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:20.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.127+0000 7fb744e71700 1 -- 172.21.15.36:0/408867494 >> v1:172.21.15.36:6789/0 conn(0x7fb7400b7e80 legacy=0x7fb740147950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:20.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.127+0000 7fb744e71700 1 -- 172.21.15.36:0/408867494 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7401474b0 con 0x7fb7400ae360 2022-06-17T12:56:20.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.127+0000 7fb744e71700 1 --2- 172.21.15.36:0/408867494 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb7400ae360 0x7fb7400b29f0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7fb738002f70 tx=0x7fb738006df0).ready entity=mon.0 client_cookie=2a82af9ca9a487a4 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:20.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.127+0000 7fb73effd700 1 -- 172.21.15.36:0/408867494 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb73800d4f0 con 0x7fb7400ae360 2022-06-17T12:56:20.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.131+0000 7fb73effd700 1 -- 172.21.15.36:0/408867494 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb738000e90 con 0x7fb7400ae360 2022-06-17T12:56:20.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.131+0000 7fb73effd700 1 -- 172.21.15.36:0/408867494 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb73800d840 con 0x7fb7400ae360 2022-06-17T12:56:20.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.131+0000 7fb745672700 1 -- 172.21.15.36:0/408867494 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb7400ae360 msgr2=0x7fb7400b29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:20.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.131+0000 7fb745672700 1 --2- 172.21.15.36:0/408867494 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb7400ae360 0x7fb7400b29f0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7fb738002f70 tx=0x7fb738006df0).stop 2022-06-17T12:56:20.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.131+0000 7fb745672700 1 -- 172.21.15.36:0/408867494 shutdown_connections 2022-06-17T12:56:20.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.131+0000 7fb745672700 1 --2- 172.21.15.36:0/408867494 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb7400ae360 0x7fb7400b29f0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:20.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.131+0000 7fb745672700 1 --2- 172.21.15.36:0/408867494 >> v2:172.21.15.70:3300/0 conn(0x7fb740148060 0x7fb7400b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:20.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.131+0000 7fb745672700 1 -- 172.21.15.36:0/408867494 >> 172.21.15.36:0/408867494 conn(0x7fb74005c550 msgr2=0x7fb7400b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:20.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.131+0000 7fb745672700 1 -- 172.21.15.36:0/408867494 shutdown_connections 2022-06-17T12:56:20.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.131+0000 7fb745672700 1 -- 172.21.15.36:0/408867494 wait complete. 2022-06-17T12:56:20.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.131+0000 7fb745672700 1 Processor -- start 2022-06-17T12:56:20.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.131+0000 7fb745672700 1 -- start start 2022-06-17T12:56:20.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.131+0000 7fb745672700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7400ae360 0x7fb7400afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:20.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.131+0000 7fb745672700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7400b7e80 0x7fb74014e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:20.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.131+0000 7fb745672700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb740148060 0x7fb740159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:20.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.131+0000 7fb745672700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400649c0 con 0x7fb740148060 2022-06-17T12:56:20.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.131+0000 7fb745672700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400b0020 con 0x7fb7400ae360 2022-06-17T12:56:20.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.131+0000 7fb745672700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb740064720 con 0x7fb7400b7e80 2022-06-17T12:56:20.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.131+0000 7fb744e71700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb740148060 0x7fb740159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:20.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.131+0000 7fb744e71700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb740148060 0x7fb740159eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39540/0 (socket says 172.21.15.36:39540) 2022-06-17T12:56:20.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.131+0000 7fb744e71700 1 -- 172.21.15.36:0/687286230 learned_addr learned my addr 172.21.15.36:0/687286230 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:20.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.131+0000 7fb73ffff700 1 --2- 172.21.15.36:0/687286230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7400ae360 0x7fb7400afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:20.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.131+0000 7fb73f7fe700 1 --2- 172.21.15.36:0/687286230 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7400b7e80 0x7fb74014e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:20.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.135+0000 7fb744e71700 1 -- 172.21.15.36:0/687286230 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7400b7e80 msgr2=0x7fb74014e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:20.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.135+0000 7fb744e71700 1 --2- 172.21.15.36:0/687286230 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7400b7e80 0x7fb74014e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:20.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.135+0000 7fb744e71700 1 -- 172.21.15.36:0/687286230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7400ae360 msgr2=0x7fb7400afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:20.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.135+0000 7fb744e71700 1 --2- 172.21.15.36:0/687286230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7400ae360 0x7fb7400afb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:20.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.135+0000 7fb744e71700 1 -- 172.21.15.36:0/687286230 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb73800b180 con 0x7fb740148060 2022-06-17T12:56:20.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.135+0000 7fb744e71700 1 --2- 172.21.15.36:0/687286230 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb740148060 0x7fb740159eb0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7fb738002f70 tx=0x7fb738014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:20.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.135+0000 7fb73d7fa700 1 -- 172.21.15.36:0/687286230 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb738012930 con 0x7fb740148060 2022-06-17T12:56:20.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.135+0000 7fb73d7fa700 1 -- 172.21.15.36:0/687286230 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb738012a90 con 0x7fb740148060 2022-06-17T12:56:20.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.135+0000 7fb73d7fa700 1 -- 172.21.15.36:0/687286230 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb738012df0 con 0x7fb740148060 2022-06-17T12:56:20.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.135+0000 7fb745672700 1 -- 172.21.15.36:0/687286230 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb740159a10 con 0x7fb740148060 2022-06-17T12:56:20.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.135+0000 7fb745672700 1 -- 172.21.15.36:0/687286230 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb740146d10 con 0x7fb740148060 2022-06-17T12:56:20.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.135+0000 7fb745672700 1 -- 172.21.15.36:0/687286230 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb740159ba0 con 0x7fb740148060 2022-06-17T12:56:20.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.135+0000 7fb73d7fa700 1 -- 172.21.15.36:0/687286230 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fb73801c070 con 0x7fb740148060 2022-06-17T12:56:20.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.135+0000 7fb73d7fa700 1 --2- 172.21.15.36:0/687286230 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb724053af0 0x7fb724055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:20.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.135+0000 7fb73d7fa700 1 -- 172.21.15.36:0/687286230 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb73806d2a0 con 0x7fb740148060 2022-06-17T12:56:20.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.135+0000 7fb73ffff700 1 --2- 172.21.15.36:0/687286230 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb724053af0 0x7fb724055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:20.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.139+0000 7fb73ffff700 1 --2- 172.21.15.36:0/687286230 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb724053af0 0x7fb724055fb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7fb73000d000 tx=0x7fb730009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:20.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.143+0000 7fb73d7fa700 1 -- 172.21.15.36:0/687286230 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb7380440a0 con 0x7fb740148060 2022-06-17T12:56:20.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.303+0000 7fb745672700 1 -- 172.21.15.36:0/687286230 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7400b1ff0 con 0x7fb740148060 2022-06-17T12:56:20.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.303+0000 7fb73d7fa700 1 -- 172.21.15.36:0/687286230 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+2139 (secure 0 0 0) 0x7fb73802c870 con 0x7fb740148060 2022-06-17T12:56:20.292 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:20.292 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":72,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/2357210652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2357210652},{"type":"v1","addr":"172.21.15.36:6835","nonce":2357210652}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:56:11.021928+0000","modified":"2022-06-17T12:56:13.034227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7942":{"gid":7942,"name":"b","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/197726269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":197726269},{"type":"v1","addr":"172.21.15.70:6833","nonce":197726269}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-06-17T12:56:20.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.307+0000 7fb745672700 1 -- 172.21.15.36:0/687286230 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb724053af0 msgr2=0x7fb724055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:20.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.307+0000 7fb745672700 1 --2- 172.21.15.36:0/687286230 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb724053af0 0x7fb724055fb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7fb73000d000 tx=0x7fb730009040).stop 2022-06-17T12:56:20.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.307+0000 7fb745672700 1 -- 172.21.15.36:0/687286230 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb740148060 msgr2=0x7fb740159eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:20.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.307+0000 7fb745672700 1 --2- 172.21.15.36:0/687286230 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb740148060 0x7fb740159eb0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7fb738002f70 tx=0x7fb738014000).stop 2022-06-17T12:56:20.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.307+0000 7fb745672700 1 -- 172.21.15.36:0/687286230 shutdown_connections 2022-06-17T12:56:20.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.307+0000 7fb745672700 1 --2- 172.21.15.36:0/687286230 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb724053af0 0x7fb724055fb0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:20.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.307+0000 7fb745672700 1 --2- 172.21.15.36:0/687286230 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb740148060 0x7fb740159eb0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:20.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.307+0000 7fb745672700 1 --2- 172.21.15.36:0/687286230 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7400b7e80 0x7fb74014e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:20.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.307+0000 7fb745672700 1 --2- 172.21.15.36:0/687286230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7400ae360 0x7fb7400afb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:20.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.307+0000 7fb745672700 1 -- 172.21.15.36:0/687286230 >> 172.21.15.36:0/687286230 conn(0x7fb74005c550 msgr2=0x7fb7400b6630 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:20.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.307+0000 7fb745672700 1 -- 172.21.15.36:0/687286230 shutdown_connections 2022-06-17T12:56:20.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:20.307+0000 7fb745672700 1 -- 172.21.15.36:0/687286230 wait complete. 2022-06-17T12:56:20.307 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 72 2022-06-17T12:56:20.514 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:56:11.021928+0000', 'modified': '2022-06-17T12:56:13.034227+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7942}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7942': {'gid': 7942, 'name': 'b', 'rank': 0, 'incarnation': 69, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6833/197726269', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 197726269}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 197726269}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:56:20.515 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:56:20.516 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-17T12:56:20.652 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":7942,"want_state":"up:active","state":"up:active","rank_uptime":9.621676721,"mdsmap_epoch":72,"osdmap_epoch":131,"osdmap_epoch_barrier":131,"uptime":14.605054707000001} 2022-06-17T12:56:20.653 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 7942, "mdsmap_epoch": 72, "osdmap_epoch": 131, "osdmap_epoch_barrier": 131, "rank_uptime": 9.621676721, "state": "up:active", "uptime": 14.605054707, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:56:20.669 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:56:20.670 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-06-17T12:56:20.671 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi136.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-06-17T12:56:20.672 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:20.672 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:20.699 DEBUG:teuthology.orchestra.run.smithi136:> sudo modprobe fuse 2022-06-17T12:56:20.723 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:56:20.724 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:56:20.787 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:56:20.788 INFO:teuthology.orchestra.run.smithi136.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:56:20.800 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:20.801 DEBUG:teuthology.orchestra.run.smithi136:> ls /sys/fs/fuse/connections 2022-06-17T12:56:20.817 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:56:20.818 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:20.862 DEBUG:teuthology.orchestra.run.smithi136:> sudo modprobe fuse 2022-06-17T12:56:20.893 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:56:20.894 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:56:20.909 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:56:20.911 INFO:teuthology.orchestra.run.smithi136.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:56:20.911 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:20.912 DEBUG:teuthology.orchestra.run.smithi136:> ls /sys/fs/fuse/connections 2022-06-17T12:56:20.942 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:2022-06-17T12:56:20.949+0000 7f02cc5e5080 -1 init, newargv = 0x55d59c029640 newargc=9 2022-06-17T12:56:20.943 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:ceph-fuse[22320]: starting ceph client 2022-06-17T12:56:20.967 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:ceph-fuse[22320]: starting fuse 2022-06-17T12:56:20.971 INFO:teuthology.orchestra.run.smithi136.stdout:52 2022-06-17T12:56:20.972 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:56:20.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:20.974 DEBUG:teuthology.orchestra.run.smithi136:> sudo python3 -c ' 2022-06-17T12:56:20.975 DEBUG:teuthology.orchestra.run.smithi136:> import glob 2022-06-17T12:56:20.975 DEBUG:teuthology.orchestra.run.smithi136:> import re 2022-06-17T12:56:20.976 DEBUG:teuthology.orchestra.run.smithi136:> import os 2022-06-17T12:56:20.976 DEBUG:teuthology.orchestra.run.smithi136:> import subprocess 2022-06-17T12:56:20.977 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:56:20.978 DEBUG:teuthology.orchestra.run.smithi136:> def find_socket(client_name): 2022-06-17T12:56:20.978 DEBUG:teuthology.orchestra.run.smithi136:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-06-17T12:56:20.979 DEBUG:teuthology.orchestra.run.smithi136:> files = glob.glob(asok_path) 2022-06-17T12:56:20.980 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:56:21.011 DEBUG:teuthology.orchestra.run.smithi136:> # Given a non-glob path, it better be there 2022-06-17T12:56:21.016 DEBUG:teuthology.orchestra.run.smithi136:> if "*" not in asok_path: 2022-06-17T12:56:21.034 DEBUG:teuthology.orchestra.run.smithi136:> assert(len(files) == 1) 2022-06-17T12:56:21.035 DEBUG:teuthology.orchestra.run.smithi136:> return files[0] 2022-06-17T12:56:21.036 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:56:21.050 DEBUG:teuthology.orchestra.run.smithi136:> for f in files: 2022-06-17T12:56:21.051 DEBUG:teuthology.orchestra.run.smithi136:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:56:21.051 DEBUG:teuthology.orchestra.run.smithi136:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:56:21.052 DEBUG:teuthology.orchestra.run.smithi136:> return f 2022-06-17T12:56:21.053 DEBUG:teuthology.orchestra.run.smithi136:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:56:21.053 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:56:21.054 DEBUG:teuthology.orchestra.run.smithi136:> print(find_socket("client.1")) 2022-06-17T12:56:21.055 DEBUG:teuthology.orchestra.run.smithi136:> ' 2022-06-17T12:56:21.163 INFO:teuthology.orchestra.run.smithi136.stdout:/var/run/ceph/ceph-client.1.22320.asok 2022-06-17T12:56:21.164 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22320.asok 2022-06-17T12:56:21.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:21.178 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22320.asok status 2022-06-17T12:56:21.301 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2022-06-17T12:56:21.302 INFO:teuthology.orchestra.run.smithi136.stdout: "metadata": { 2022-06-17T12:56:21.302 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:56:21.302 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:56:21.303 INFO:teuthology.orchestra.run.smithi136.stdout: "entity_id": "1", 2022-06-17T12:56:21.303 INFO:teuthology.orchestra.run.smithi136.stdout: "hostname": "smithi136", 2022-06-17T12:56:21.304 INFO:teuthology.orchestra.run.smithi136.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-06-17T12:56:21.304 INFO:teuthology.orchestra.run.smithi136.stdout: "pid": "22320", 2022-06-17T12:56:21.304 INFO:teuthology.orchestra.run.smithi136.stdout: "root": "/" 2022-06-17T12:56:21.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:56:21.305 INFO:teuthology.orchestra.run.smithi136.stdout: "dentry_count": 0, 2022-06-17T12:56:21.305 INFO:teuthology.orchestra.run.smithi136.stdout: "dentry_pinned_count": 0, 2022-06-17T12:56:21.305 INFO:teuthology.orchestra.run.smithi136.stdout: "id": 7079, 2022-06-17T12:56:21.305 INFO:teuthology.orchestra.run.smithi136.stdout: "inst": { 2022-06-17T12:56:21.310 INFO:teuthology.orchestra.run.smithi136.stdout: "name": { 2022-06-17T12:56:21.310 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "client", 2022-06-17T12:56:21.311 INFO:teuthology.orchestra.run.smithi136.stdout: "num": 7079 2022-06-17T12:56:21.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:56:21.312 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": { 2022-06-17T12:56:21.313 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "v1", 2022-06-17T12:56:21.313 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": "172.21.15.136:0", 2022-06-17T12:56:21.313 INFO:teuthology.orchestra.run.smithi136.stdout: "nonce": 3336960382 2022-06-17T12:56:21.321 INFO:teuthology.orchestra.run.smithi136.stdout: } 2022-06-17T12:56:21.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:56:21.322 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": { 2022-06-17T12:56:21.323 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "v1", 2022-06-17T12:56:21.323 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": "172.21.15.136:0", 2022-06-17T12:56:21.324 INFO:teuthology.orchestra.run.smithi136.stdout: "nonce": 3336960382 2022-06-17T12:56:21.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:56:21.325 INFO:teuthology.orchestra.run.smithi136.stdout: "inst_str": "client.7079 172.21.15.136:0/3336960382", 2022-06-17T12:56:21.325 INFO:teuthology.orchestra.run.smithi136.stdout: "addr_str": "172.21.15.136:0/3336960382", 2022-06-17T12:56:21.333 INFO:teuthology.orchestra.run.smithi136.stdout: "inode_count": 1, 2022-06-17T12:56:21.333 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_epoch": 72, 2022-06-17T12:56:21.334 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_epoch": 131, 2022-06-17T12:56:21.334 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:56:21.334 INFO:teuthology.orchestra.run.smithi136.stdout: "blacklisted": false, 2022-06-17T12:56:21.335 INFO:teuthology.orchestra.run.smithi136.stdout: "fs_name": "cephfs" 2022-06-17T12:56:21.335 INFO:teuthology.orchestra.run.smithi136.stdout:} 2022-06-17T12:56:21.346 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:21.347 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:21.348 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:21.362 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:56:21.376 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:56:21.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:21.378 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:21.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:21.446 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:21.447 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:21.457 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:56:21.458 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:56:21.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:21.460 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:21.461 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:21.516 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:56:21.517 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:56:21.518 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi136.front.sepia.ceph.com... 2022-06-17T12:56:21.518 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:56:21.519 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:21.582 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:ceph-fuse[22320]: fuse finished with error 0 and tester_r 0 2022-06-17T12:56:21.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:21.585 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:21.586 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:21.617 INFO:teuthology.orchestra.run.smithi136.stdout:ext2/ext3 2022-06-17T12:56:21.618 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:56:21.619 INFO:teuthology.orchestra.run:waiting for 900 2022-06-17T12:56:27.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:56:27.738 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:27.800 DEBUG:teuthology.orchestra.run.smithi036:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-17T12:56:27.810 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-17T12:56:27.854 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-17T12:56:27.861 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-17T12:56:27.865 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-17T12:56:27.873 INFO:teuthology.orchestra.run.smithi036.stdout:[global] 2022-06-17T12:56:27.877 INFO:teuthology.orchestra.run.smithi036.stdout: chdir = "" 2022-06-17T12:56:27.878 INFO:teuthology.orchestra.run.smithi036.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-17T12:56:27.878 INFO:teuthology.orchestra.run.smithi036.stdout: auth supported = cephx 2022-06-17T12:56:27.878 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.879 INFO:teuthology.orchestra.run.smithi036.stdout: filestore xattr use omap = true 2022-06-17T12:56:27.879 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.879 INFO:teuthology.orchestra.run.smithi036.stdout: mon clock drift allowed = 1.000 2022-06-17T12:56:27.879 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.880 INFO:teuthology.orchestra.run.smithi036.stdout: osd crush chooseleaf type = 0 2022-06-17T12:56:27.880 INFO:teuthology.orchestra.run.smithi036.stdout: auth debug = true 2022-06-17T12:56:27.880 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.880 INFO:teuthology.orchestra.run.smithi036.stdout: ms die on old message = true 2022-06-17T12:56:27.881 INFO:teuthology.orchestra.run.smithi036.stdout: ms die on bug = true 2022-06-17T12:56:27.881 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.881 INFO:teuthology.orchestra.run.smithi036.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-17T12:56:27.881 INFO:teuthology.orchestra.run.smithi036.stdout: mon pg warn max object skew = 0 2022-06-17T12:56:27.882 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.882 INFO:teuthology.orchestra.run.smithi036.stdout: # disable pg_autoscaler by default for new pools 2022-06-17T12:56:27.882 INFO:teuthology.orchestra.run.smithi036.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-17T12:56:27.883 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.883 INFO:teuthology.orchestra.run.smithi036.stdout: osd pool default size = 2 2022-06-17T12:56:27.883 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.883 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd allow primary affinity = true 2022-06-17T12:56:27.884 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd allow pg remap = true 2022-06-17T12:56:27.887 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on legacy crush tunables = false 2022-06-17T12:56:27.888 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on crush straw calc version zero = false 2022-06-17T12:56:27.888 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on no sortbitwise = false 2022-06-17T12:56:27.890 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on osd down out interval zero = false 2022-06-17T12:56:27.890 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on too few osds = false 2022-06-17T12:56:27.890 INFO:teuthology.orchestra.run.smithi036.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-17T12:56:27.890 INFO:teuthology.orchestra.run.smithi036.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-17T12:56:27.891 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.891 INFO:teuthology.orchestra.run.smithi036.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-17T12:56:27.891 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.892 INFO:teuthology.orchestra.run.smithi036.stdout: osd default data pool replay window = 5 2022-06-17T12:56:27.892 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.893 INFO:teuthology.orchestra.run.smithi036.stdout: mon allow pool delete = true 2022-06-17T12:56:27.893 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.893 INFO:teuthology.orchestra.run.smithi036.stdout: mon cluster log file level = debug 2022-06-17T12:56:27.894 INFO:teuthology.orchestra.run.smithi036.stdout: debug asserts on shutdown = true 2022-06-17T12:56:27.894 INFO:teuthology.orchestra.run.smithi036.stdout: mon health detail to clog = false 2022-06-17T12:56:27.895 INFO:teuthology.orchestra.run.smithi036.stdout: mon host = "172.21.15.36,172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790]" 2022-06-17T12:56:27.898 INFO:teuthology.orchestra.run.smithi036.stdout: fsid = b68b9700-0c85-42ca-94d7-6cd53db91c14 2022-06-17T12:56:27.901 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.901 INFO:teuthology.orchestra.run.smithi036.stdout:[osd] 2022-06-17T12:56:27.902 INFO:teuthology.orchestra.run.smithi036.stdout: osd journal size = 100 2022-06-17T12:56:27.902 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.903 INFO:teuthology.orchestra.run.smithi036.stdout: osd scrub load threshold = 5.0 2022-06-17T12:56:27.903 INFO:teuthology.orchestra.run.smithi036.stdout: osd scrub max interval = 600 2022-06-17T12:56:27.903 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.904 INFO:teuthology.orchestra.run.smithi036.stdout: osd recover clone overlap = true 2022-06-17T12:56:27.904 INFO:teuthology.orchestra.run.smithi036.stdout: osd recovery max chunk = 1048576 2022-06-17T12:56:27.905 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.905 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug shutdown = true 2022-06-17T12:56:27.906 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug op order = true 2022-06-17T12:56:27.906 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug verify stray on activate = true 2022-06-17T12:56:27.907 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.909 INFO:teuthology.orchestra.run.smithi036.stdout: osd open classes on start = true 2022-06-17T12:56:27.912 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug pg log writeout = true 2022-06-17T12:56:27.913 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.913 INFO:teuthology.orchestra.run.smithi036.stdout: osd deep scrub update digest min age = 30 2022-06-17T12:56:27.914 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.914 INFO:teuthology.orchestra.run.smithi036.stdout: osd map max advance = 10 2022-06-17T12:56:27.915 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.915 INFO:teuthology.orchestra.run.smithi036.stdout: journal zero on create = true 2022-06-17T12:56:27.916 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.916 INFO:teuthology.orchestra.run.smithi036.stdout: filestore ondisk finisher threads = 3 2022-06-17T12:56:27.917 INFO:teuthology.orchestra.run.smithi036.stdout: filestore apply finisher threads = 3 2022-06-17T12:56:27.917 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.917 INFO:teuthology.orchestra.run.smithi036.stdout: bdev debug aio = true 2022-06-17T12:56:27.918 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug misdirected ops = true 2022-06-17T12:56:27.918 INFO:teuthology.orchestra.run.smithi036.stdout: bluestore block size = 96636764160 2022-06-17T12:56:27.920 INFO:teuthology.orchestra.run.smithi036.stdout: bluestore compression mode = aggressive 2022-06-17T12:56:27.921 INFO:teuthology.orchestra.run.smithi036.stdout: bluestore fsck on mount = True 2022-06-17T12:56:27.923 INFO:teuthology.orchestra.run.smithi036.stdout: debug bluefs = 20 2022-06-17T12:56:27.924 INFO:teuthology.orchestra.run.smithi036.stdout: debug bluestore = 20 2022-06-17T12:56:27.924 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2022-06-17T12:56:27.924 INFO:teuthology.orchestra.run.smithi036.stdout: debug osd = 20 2022-06-17T12:56:27.925 INFO:teuthology.orchestra.run.smithi036.stdout: debug rocksdb = 10 2022-06-17T12:56:27.925 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd backfillfull_ratio = 0.85 2022-06-17T12:56:27.926 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd full ratio = 0.9 2022-06-17T12:56:27.926 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd nearfull ratio = 0.8 2022-06-17T12:56:27.926 INFO:teuthology.orchestra.run.smithi036.stdout: osd failsafe full ratio = 0.95 2022-06-17T12:56:27.927 INFO:teuthology.orchestra.run.smithi036.stdout: osd objectstore = bluestore 2022-06-17T12:56:27.927 INFO:teuthology.orchestra.run.smithi036.stdout: osd op complaint time = 180 2022-06-17T12:56:27.927 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.928 INFO:teuthology.orchestra.run.smithi036.stdout:[mgr] 2022-06-17T12:56:27.928 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2022-06-17T12:56:27.929 INFO:teuthology.orchestra.run.smithi036.stdout: debug mgr = 20 2022-06-17T12:56:27.929 INFO:teuthology.orchestra.run.smithi036.stdout: debug mon = 20 2022-06-17T12:56:27.934 INFO:teuthology.orchestra.run.smithi036.stdout: debug auth = 20 2022-06-17T12:56:27.935 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:56:27.935 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:56:27.936 INFO:teuthology.orchestra.run.smithi036.stdout: mgr/telemetry/nag = false 2022-06-17T12:56:27.936 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.936 INFO:teuthology.orchestra.run.smithi036.stdout:[mon] 2022-06-17T12:56:27.937 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2022-06-17T12:56:27.937 INFO:teuthology.orchestra.run.smithi036.stdout: debug mon = 20 2022-06-17T12:56:27.938 INFO:teuthology.orchestra.run.smithi036.stdout: debug paxos = 20 2022-06-17T12:56:27.938 INFO:teuthology.orchestra.run.smithi036.stdout: debug auth = 20 2022-06-17T12:56:27.939 INFO:teuthology.orchestra.run.smithi036.stdout: mon data avail warn = 5 2022-06-17T12:56:27.939 INFO:teuthology.orchestra.run.smithi036.stdout: mon mgr mkfs grace = 240 2022-06-17T12:56:27.940 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:56:27.942 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd reporter subtree level = osd 2022-06-17T12:56:27.943 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd prime pg temp = true 2022-06-17T12:56:27.943 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:56:27.946 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.946 INFO:teuthology.orchestra.run.smithi036.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-17T12:56:27.947 INFO:teuthology.orchestra.run.smithi036.stdout: auth mon ticket ttl = 660 # 11m 2022-06-17T12:56:27.947 INFO:teuthology.orchestra.run.smithi036.stdout: auth service ticket ttl = 240 # 4m 2022-06-17T12:56:27.948 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.948 INFO:teuthology.orchestra.run.smithi036.stdout: # don't complain about insecure global_id in the test suite 2022-06-17T12:56:27.948 INFO:teuthology.orchestra.run.smithi036.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-17T12:56:27.949 INFO:teuthology.orchestra.run.smithi036.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-17T12:56:27.949 INFO:teuthology.orchestra.run.smithi036.stdout: mon op complaint time = 120 2022-06-17T12:56:27.968 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:27.969 INFO:teuthology.orchestra.run.smithi036.stdout:[client] 2022-06-17T12:56:27.973 INFO:teuthology.orchestra.run.smithi036.stdout: rgw cache enabled = true 2022-06-17T12:56:27.974 INFO:teuthology.orchestra.run.smithi036.stdout: rgw enable ops log = true 2022-06-17T12:56:27.975 INFO:teuthology.orchestra.run.smithi036.stdout: rgw enable usage log = true 2022-06-17T12:56:27.980 INFO:teuthology.orchestra.run.smithi036.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-17T12:56:27.981 INFO:teuthology.orchestra.run.smithi036.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-17T12:56:27.982 INFO:teuthology.orchestra.run.smithi036.stdout: client mount timeout = 600 2022-06-17T12:56:27.983 INFO:teuthology.orchestra.run.smithi036.stdout: debug client = 20 2022-06-17T12:56:27.983 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2022-06-17T12:56:27.984 INFO:teuthology.orchestra.run.smithi036.stdout: rados mon op timeout = 15m 2022-06-17T12:56:27.984 INFO:teuthology.orchestra.run.smithi036.stdout: rados osd op timeout = 15m 2022-06-17T12:56:27.985 INFO:teuthology.orchestra.run.smithi036.stdout: client debug getattr caps = true 2022-06-17T12:56:27.986 INFO:teuthology.orchestra.run.smithi036.stdout:[mon.a] 2022-06-17T12:56:27.986 INFO:teuthology.orchestra.run.smithi036.stdout:[mon.b] 2022-06-17T12:56:27.986 INFO:teuthology.orchestra.run.smithi036.stdout:[mon.c] 2022-06-17T12:56:27.987 INFO:teuthology.orchestra.run.smithi036.stdout:[mds.a] 2022-06-17T12:56:27.987 INFO:teuthology.orchestra.run.smithi036.stdout:[mds.b] 2022-06-17T12:56:27.988 INFO:teuthology.orchestra.run.smithi036.stdout:[mds] 2022-06-17T12:56:27.999 INFO:teuthology.orchestra.run.smithi036.stdout: debug mds = 20 2022-06-17T12:56:28.001 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2022-06-17T12:56:28.002 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal frag = True 2022-06-17T12:56:28.002 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal fragment size max = 10000 2022-06-17T12:56:28.007 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal merge size = 5 2022-06-17T12:56:28.007 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal split bits = 3 2022-06-17T12:56:28.011 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal split size = 100 2022-06-17T12:56:28.011 INFO:teuthology.orchestra.run.smithi036.stdout: mds debug frag = True 2022-06-17T12:56:28.011 INFO:teuthology.orchestra.run.smithi036.stdout: mds debug scatterstat = True 2022-06-17T12:56:28.012 INFO:teuthology.orchestra.run.smithi036.stdout: mds op complaint time = 180 2022-06-17T12:56:28.012 INFO:teuthology.orchestra.run.smithi036.stdout: mds verify scatter = True 2022-06-17T12:56:28.012 INFO:teuthology.orchestra.run.smithi036.stdout: osd op complaint time = 180 2022-06-17T12:56:28.012 INFO:teuthology.orchestra.run.smithi036.stdout: rados mon op timeout = 15m 2022-06-17T12:56:28.012 INFO:teuthology.orchestra.run.smithi036.stdout: rados osd op timeout = 15m 2022-06-17T12:56:28.021 INFO:teuthology.orchestra.run.smithi143.stdout:[global] 2022-06-17T12:56:28.022 INFO:teuthology.orchestra.run.smithi143.stdout: chdir = "" 2022-06-17T12:56:28.022 INFO:teuthology.orchestra.run.smithi143.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-17T12:56:28.023 INFO:teuthology.orchestra.run.smithi143.stdout: auth supported = cephx 2022-06-17T12:56:28.023 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.024 INFO:teuthology.orchestra.run.smithi143.stdout: filestore xattr use omap = true 2022-06-17T12:56:28.024 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.025 INFO:teuthology.orchestra.run.smithi143.stdout: mon clock drift allowed = 1.000 2022-06-17T12:56:28.026 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.032 INFO:teuthology.orchestra.run.smithi143.stdout: osd crush chooseleaf type = 0 2022-06-17T12:56:28.033 INFO:teuthology.orchestra.run.smithi143.stdout: auth debug = true 2022-06-17T12:56:28.033 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.034 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on old message = true 2022-06-17T12:56:28.034 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on bug = true 2022-06-17T12:56:28.035 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.035 INFO:teuthology.orchestra.run.smithi143.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-17T12:56:28.035 INFO:teuthology.orchestra.run.smithi143.stdout: mon pg warn max object skew = 0 2022-06-17T12:56:28.036 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.036 INFO:teuthology.orchestra.run.smithi143.stdout: # disable pg_autoscaler by default for new pools 2022-06-17T12:56:28.036 INFO:teuthology.orchestra.run.smithi143.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-17T12:56:28.037 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.037 INFO:teuthology.orchestra.run.smithi143.stdout: osd pool default size = 2 2022-06-17T12:56:28.037 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.055 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd allow primary affinity = true 2022-06-17T12:56:28.055 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd allow pg remap = true 2022-06-17T12:56:28.056 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on legacy crush tunables = false 2022-06-17T12:56:28.057 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on crush straw calc version zero = false 2022-06-17T12:56:28.058 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on no sortbitwise = false 2022-06-17T12:56:28.058 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on osd down out interval zero = false 2022-06-17T12:56:28.059 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on too few osds = false 2022-06-17T12:56:28.059 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-17T12:56:28.059 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-17T12:56:28.060 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.060 INFO:teuthology.orchestra.run.smithi143.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-17T12:56:28.061 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.061 INFO:teuthology.orchestra.run.smithi143.stdout: osd default data pool replay window = 5 2022-06-17T12:56:28.061 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.062 INFO:teuthology.orchestra.run.smithi143.stdout: mon allow pool delete = true 2022-06-17T12:56:28.062 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.063 INFO:teuthology.orchestra.run.smithi143.stdout: mon cluster log file level = debug 2022-06-17T12:56:28.063 INFO:teuthology.orchestra.run.smithi143.stdout: debug asserts on shutdown = true 2022-06-17T12:56:28.064 INFO:teuthology.orchestra.run.smithi143.stdout: mon health detail to clog = false 2022-06-17T12:56:28.064 INFO:teuthology.orchestra.run.smithi143.stdout: mon host = "172.21.15.36,172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790]" 2022-06-17T12:56:28.064 INFO:teuthology.orchestra.run.smithi143.stdout: fsid = b68b9700-0c85-42ca-94d7-6cd53db91c14 2022-06-17T12:56:28.065 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.065 INFO:teuthology.orchestra.run.smithi143.stdout:[osd] 2022-06-17T12:56:28.066 INFO:teuthology.orchestra.run.smithi143.stdout: osd journal size = 100 2022-06-17T12:56:28.066 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.067 INFO:teuthology.orchestra.run.smithi143.stdout: osd scrub load threshold = 5.0 2022-06-17T12:56:28.067 INFO:teuthology.orchestra.run.smithi143.stdout: osd scrub max interval = 600 2022-06-17T12:56:28.067 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.068 INFO:teuthology.orchestra.run.smithi143.stdout: osd recover clone overlap = true 2022-06-17T12:56:28.068 INFO:teuthology.orchestra.run.smithi143.stdout: osd recovery max chunk = 1048576 2022-06-17T12:56:28.069 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.070 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug shutdown = true 2022-06-17T12:56:28.070 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug op order = true 2022-06-17T12:56:28.071 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug verify stray on activate = true 2022-06-17T12:56:28.071 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.072 INFO:teuthology.orchestra.run.smithi143.stdout: osd open classes on start = true 2022-06-17T12:56:28.072 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug pg log writeout = true 2022-06-17T12:56:28.072 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.073 INFO:teuthology.orchestra.run.smithi143.stdout: osd deep scrub update digest min age = 30 2022-06-17T12:56:28.073 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.074 INFO:teuthology.orchestra.run.smithi143.stdout: osd map max advance = 10 2022-06-17T12:56:28.074 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.075 INFO:teuthology.orchestra.run.smithi143.stdout: journal zero on create = true 2022-06-17T12:56:28.075 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.076 INFO:teuthology.orchestra.run.smithi143.stdout: filestore ondisk finisher threads = 3 2022-06-17T12:56:28.087 INFO:teuthology.orchestra.run.smithi143.stdout: filestore apply finisher threads = 3 2022-06-17T12:56:28.087 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.088 INFO:teuthology.orchestra.run.smithi143.stdout: bdev debug aio = true 2022-06-17T12:56:28.088 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug misdirected ops = true 2022-06-17T12:56:28.089 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore block size = 96636764160 2022-06-17T12:56:28.089 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore compression mode = aggressive 2022-06-17T12:56:28.089 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore fsck on mount = True 2022-06-17T12:56:28.093 INFO:teuthology.orchestra.run.smithi143.stdout: debug bluefs = 20 2022-06-17T12:56:28.093 INFO:teuthology.orchestra.run.smithi143.stdout: debug bluestore = 20 2022-06-17T12:56:28.094 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2022-06-17T12:56:28.094 INFO:teuthology.orchestra.run.smithi143.stdout: debug osd = 20 2022-06-17T12:56:28.094 INFO:teuthology.orchestra.run.smithi143.stdout: debug rocksdb = 10 2022-06-17T12:56:28.095 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd backfillfull_ratio = 0.85 2022-06-17T12:56:28.095 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd full ratio = 0.9 2022-06-17T12:56:28.096 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd nearfull ratio = 0.8 2022-06-17T12:56:28.096 INFO:teuthology.orchestra.run.smithi143.stdout: osd failsafe full ratio = 0.95 2022-06-17T12:56:28.096 INFO:teuthology.orchestra.run.smithi143.stdout: osd objectstore = bluestore 2022-06-17T12:56:28.111 INFO:teuthology.orchestra.run.smithi143.stdout: osd op complaint time = 180 2022-06-17T12:56:28.111 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.112 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr] 2022-06-17T12:56:28.112 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2022-06-17T12:56:28.113 INFO:teuthology.orchestra.run.smithi143.stdout: debug mgr = 20 2022-06-17T12:56:28.113 INFO:teuthology.orchestra.run.smithi143.stdout: debug mon = 20 2022-06-17T12:56:28.113 INFO:teuthology.orchestra.run.smithi143.stdout: debug auth = 20 2022-06-17T12:56:28.114 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:56:28.114 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:56:28.115 INFO:teuthology.orchestra.run.smithi143.stdout: mgr/telemetry/nag = false 2022-06-17T12:56:28.116 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.117 INFO:teuthology.orchestra.run.smithi143.stdout:[mon] 2022-06-17T12:56:28.118 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2022-06-17T12:56:28.118 INFO:teuthology.orchestra.run.smithi143.stdout: debug mon = 20 2022-06-17T12:56:28.119 INFO:teuthology.orchestra.run.smithi143.stdout: debug paxos = 20 2022-06-17T12:56:28.119 INFO:teuthology.orchestra.run.smithi143.stdout: debug auth = 20 2022-06-17T12:56:28.120 INFO:teuthology.orchestra.run.smithi143.stdout: mon data avail warn = 5 2022-06-17T12:56:28.120 INFO:teuthology.orchestra.run.smithi143.stdout: mon mgr mkfs grace = 240 2022-06-17T12:56:28.121 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:56:28.121 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd reporter subtree level = osd 2022-06-17T12:56:28.121 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd prime pg temp = true 2022-06-17T12:56:28.122 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:56:28.122 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.123 INFO:teuthology.orchestra.run.smithi143.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-17T12:56:28.123 INFO:teuthology.orchestra.run.smithi143.stdout: auth mon ticket ttl = 660 # 11m 2022-06-17T12:56:28.124 INFO:teuthology.orchestra.run.smithi143.stdout: auth service ticket ttl = 240 # 4m 2022-06-17T12:56:28.124 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.124 INFO:teuthology.orchestra.run.smithi143.stdout: # don't complain about insecure global_id in the test suite 2022-06-17T12:56:28.125 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-17T12:56:28.125 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-17T12:56:28.126 INFO:teuthology.orchestra.run.smithi143.stdout: mon op complaint time = 120 2022-06-17T12:56:28.126 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:28.127 INFO:teuthology.orchestra.run.smithi143.stdout:[client] 2022-06-17T12:56:28.127 INFO:teuthology.orchestra.run.smithi143.stdout: rgw cache enabled = true 2022-06-17T12:56:28.127 INFO:teuthology.orchestra.run.smithi143.stdout: rgw enable ops log = true 2022-06-17T12:56:28.128 INFO:teuthology.orchestra.run.smithi143.stdout: rgw enable usage log = true 2022-06-17T12:56:28.128 INFO:teuthology.orchestra.run.smithi143.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-17T12:56:28.129 INFO:teuthology.orchestra.run.smithi143.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-17T12:56:28.129 INFO:teuthology.orchestra.run.smithi143.stdout: client mount timeout = 600 2022-06-17T12:56:28.130 INFO:teuthology.orchestra.run.smithi143.stdout: debug client = 20 2022-06-17T12:56:28.130 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2022-06-17T12:56:28.130 INFO:teuthology.orchestra.run.smithi143.stdout: rados mon op timeout = 15m 2022-06-17T12:56:28.131 INFO:teuthology.orchestra.run.smithi143.stdout: rados osd op timeout = 15m 2022-06-17T12:56:28.131 INFO:teuthology.orchestra.run.smithi143.stdout: client debug getattr caps = true 2022-06-17T12:56:28.131 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.a] 2022-06-17T12:56:28.132 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.b] 2022-06-17T12:56:28.132 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.c] 2022-06-17T12:56:28.133 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.a] 2022-06-17T12:56:28.133 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.b] 2022-06-17T12:56:28.133 INFO:teuthology.orchestra.run.smithi143.stdout:[mds] 2022-06-17T12:56:28.134 INFO:teuthology.orchestra.run.smithi143.stdout: debug mds = 20 2022-06-17T12:56:28.134 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2022-06-17T12:56:28.135 INFO:teuthology.orchestra.run.smithi143.stdout: mds bal frag = True 2022-06-17T12:56:28.135 INFO:teuthology.orchestra.run.smithi143.stdout: mds bal fragment size max = 10000 2022-06-17T12:56:28.136 INFO:teuthology.orchestra.run.smithi143.stdout: mds bal merge size = 5 2022-06-17T12:56:28.136 INFO:teuthology.orchestra.run.smithi143.stdout: mds bal split bits = 3 2022-06-17T12:56:28.137 INFO:teuthology.orchestra.run.smithi143.stdout: mds bal split size = 100 2022-06-17T12:56:28.137 INFO:teuthology.orchestra.run.smithi143.stdout: mds debug frag = True 2022-06-17T12:56:28.141 INFO:teuthology.orchestra.run.smithi143.stdout: mds debug scatterstat = True 2022-06-17T12:56:28.141 INFO:teuthology.orchestra.run.smithi143.stdout: mds op complaint time = 180 2022-06-17T12:56:28.142 INFO:teuthology.orchestra.run.smithi143.stdout: mds verify scatter = True 2022-06-17T12:56:28.142 INFO:teuthology.orchestra.run.smithi143.stdout: osd op complaint time = 180 2022-06-17T12:56:28.143 INFO:teuthology.orchestra.run.smithi143.stdout: rados mon op timeout = 15m 2022-06-17T12:56:28.143 INFO:teuthology.orchestra.run.smithi143.stdout: rados osd op timeout = 15m 2022-06-17T12:56:28.144 INFO:teuthology.orchestra.run.smithi136.stdout:[global] 2022-06-17T12:56:28.144 INFO:teuthology.orchestra.run.smithi136.stdout: chdir = "" 2022-06-17T12:56:28.145 INFO:teuthology.orchestra.run.smithi136.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-17T12:56:28.145 INFO:teuthology.orchestra.run.smithi136.stdout: auth supported = cephx 2022-06-17T12:56:28.146 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.146 INFO:teuthology.orchestra.run.smithi136.stdout: filestore xattr use omap = true 2022-06-17T12:56:28.146 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.147 INFO:teuthology.orchestra.run.smithi136.stdout: mon clock drift allowed = 1.000 2022-06-17T12:56:28.147 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.148 INFO:teuthology.orchestra.run.smithi136.stdout: osd crush chooseleaf type = 0 2022-06-17T12:56:28.148 INFO:teuthology.orchestra.run.smithi136.stdout: auth debug = true 2022-06-17T12:56:28.149 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.149 INFO:teuthology.orchestra.run.smithi136.stdout: ms die on old message = true 2022-06-17T12:56:28.149 INFO:teuthology.orchestra.run.smithi136.stdout: ms die on bug = true 2022-06-17T12:56:28.151 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.151 INFO:teuthology.orchestra.run.smithi136.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-17T12:56:28.152 INFO:teuthology.orchestra.run.smithi136.stdout: mon pg warn max object skew = 0 2022-06-17T12:56:28.153 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.154 INFO:teuthology.orchestra.run.smithi136.stdout: # disable pg_autoscaler by default for new pools 2022-06-17T12:56:28.155 INFO:teuthology.orchestra.run.smithi136.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-17T12:56:28.155 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.156 INFO:teuthology.orchestra.run.smithi136.stdout: osd pool default size = 2 2022-06-17T12:56:28.156 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.157 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd allow primary affinity = true 2022-06-17T12:56:28.157 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd allow pg remap = true 2022-06-17T12:56:28.158 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on legacy crush tunables = false 2022-06-17T12:56:28.158 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on crush straw calc version zero = false 2022-06-17T12:56:28.159 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on no sortbitwise = false 2022-06-17T12:56:28.159 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on osd down out interval zero = false 2022-06-17T12:56:28.159 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on too few osds = false 2022-06-17T12:56:28.160 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-17T12:56:28.160 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-17T12:56:28.161 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.161 INFO:teuthology.orchestra.run.smithi136.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-17T12:56:28.161 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.162 INFO:teuthology.orchestra.run.smithi136.stdout: osd default data pool replay window = 5 2022-06-17T12:56:28.163 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.163 INFO:teuthology.orchestra.run.smithi136.stdout: mon allow pool delete = true 2022-06-17T12:56:28.164 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.164 INFO:teuthology.orchestra.run.smithi136.stdout: mon cluster log file level = debug 2022-06-17T12:56:28.175 INFO:teuthology.orchestra.run.smithi136.stdout: debug asserts on shutdown = true 2022-06-17T12:56:28.176 INFO:teuthology.orchestra.run.smithi136.stdout: mon health detail to clog = false 2022-06-17T12:56:28.176 INFO:teuthology.orchestra.run.smithi136.stdout: mon host = "172.21.15.36,172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790]" 2022-06-17T12:56:28.176 INFO:teuthology.orchestra.run.smithi136.stdout: fsid = b68b9700-0c85-42ca-94d7-6cd53db91c14 2022-06-17T12:56:28.176 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.177 INFO:teuthology.orchestra.run.smithi136.stdout:[osd] 2022-06-17T12:56:28.177 INFO:teuthology.orchestra.run.smithi136.stdout: osd journal size = 100 2022-06-17T12:56:28.177 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.177 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub load threshold = 5.0 2022-06-17T12:56:28.178 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub max interval = 600 2022-06-17T12:56:28.178 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.178 INFO:teuthology.orchestra.run.smithi136.stdout: osd recover clone overlap = true 2022-06-17T12:56:28.178 INFO:teuthology.orchestra.run.smithi136.stdout: osd recovery max chunk = 1048576 2022-06-17T12:56:28.180 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.180 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug shutdown = true 2022-06-17T12:56:28.180 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug op order = true 2022-06-17T12:56:28.181 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug verify stray on activate = true 2022-06-17T12:56:28.181 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.181 INFO:teuthology.orchestra.run.smithi136.stdout: osd open classes on start = true 2022-06-17T12:56:28.181 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug pg log writeout = true 2022-06-17T12:56:28.182 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.182 INFO:teuthology.orchestra.run.smithi136.stdout: osd deep scrub update digest min age = 30 2022-06-17T12:56:28.182 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.182 INFO:teuthology.orchestra.run.smithi136.stdout: osd map max advance = 10 2022-06-17T12:56:28.183 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.183 INFO:teuthology.orchestra.run.smithi136.stdout: journal zero on create = true 2022-06-17T12:56:28.183 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.183 INFO:teuthology.orchestra.run.smithi136.stdout: filestore ondisk finisher threads = 3 2022-06-17T12:56:28.184 INFO:teuthology.orchestra.run.smithi136.stdout: filestore apply finisher threads = 3 2022-06-17T12:56:28.186 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.187 INFO:teuthology.orchestra.run.smithi136.stdout: bdev debug aio = true 2022-06-17T12:56:28.187 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug misdirected ops = true 2022-06-17T12:56:28.187 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore block size = 96636764160 2022-06-17T12:56:28.187 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore compression mode = aggressive 2022-06-17T12:56:28.188 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore fsck on mount = True 2022-06-17T12:56:28.188 INFO:teuthology.orchestra.run.smithi136.stdout: debug bluefs = 20 2022-06-17T12:56:28.188 INFO:teuthology.orchestra.run.smithi136.stdout: debug bluestore = 20 2022-06-17T12:56:28.188 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2022-06-17T12:56:28.189 INFO:teuthology.orchestra.run.smithi136.stdout: debug osd = 20 2022-06-17T12:56:28.189 INFO:teuthology.orchestra.run.smithi136.stdout: debug rocksdb = 10 2022-06-17T12:56:28.189 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd backfillfull_ratio = 0.85 2022-06-17T12:56:28.189 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd full ratio = 0.9 2022-06-17T12:56:28.189 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd nearfull ratio = 0.8 2022-06-17T12:56:28.190 INFO:teuthology.orchestra.run.smithi136.stdout: osd failsafe full ratio = 0.95 2022-06-17T12:56:28.190 INFO:teuthology.orchestra.run.smithi136.stdout: osd objectstore = bluestore 2022-06-17T12:56:28.191 INFO:teuthology.orchestra.run.smithi136.stdout: osd op complaint time = 180 2022-06-17T12:56:28.191 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.191 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr] 2022-06-17T12:56:28.191 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2022-06-17T12:56:28.192 INFO:teuthology.orchestra.run.smithi136.stdout: debug mgr = 20 2022-06-17T12:56:28.192 INFO:teuthology.orchestra.run.smithi136.stdout: debug mon = 20 2022-06-17T12:56:28.192 INFO:teuthology.orchestra.run.smithi136.stdout: debug auth = 20 2022-06-17T12:56:28.192 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:56:28.193 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:56:28.193 INFO:teuthology.orchestra.run.smithi136.stdout: mgr/telemetry/nag = false 2022-06-17T12:56:28.193 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.199 INFO:teuthology.orchestra.run.smithi136.stdout:[mon] 2022-06-17T12:56:28.199 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2022-06-17T12:56:28.199 INFO:teuthology.orchestra.run.smithi136.stdout: debug mon = 20 2022-06-17T12:56:28.200 INFO:teuthology.orchestra.run.smithi136.stdout: debug paxos = 20 2022-06-17T12:56:28.200 INFO:teuthology.orchestra.run.smithi136.stdout: debug auth = 20 2022-06-17T12:56:28.200 INFO:teuthology.orchestra.run.smithi136.stdout: mon data avail warn = 5 2022-06-17T12:56:28.201 INFO:teuthology.orchestra.run.smithi136.stdout: mon mgr mkfs grace = 240 2022-06-17T12:56:28.201 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:56:28.201 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd reporter subtree level = osd 2022-06-17T12:56:28.202 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd prime pg temp = true 2022-06-17T12:56:28.202 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:56:28.202 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.202 INFO:teuthology.orchestra.run.smithi136.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-17T12:56:28.203 INFO:teuthology.orchestra.run.smithi136.stdout: auth mon ticket ttl = 660 # 11m 2022-06-17T12:56:28.203 INFO:teuthology.orchestra.run.smithi136.stdout: auth service ticket ttl = 240 # 4m 2022-06-17T12:56:28.203 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.204 INFO:teuthology.orchestra.run.smithi136.stdout: # don't complain about insecure global_id in the test suite 2022-06-17T12:56:28.227 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-17T12:56:28.227 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-17T12:56:28.228 INFO:teuthology.orchestra.run.smithi136.stdout: mon op complaint time = 120 2022-06-17T12:56:28.228 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:28.237 INFO:teuthology.orchestra.run.smithi136.stdout:[client] 2022-06-17T12:56:28.237 INFO:teuthology.orchestra.run.smithi136.stdout: rgw cache enabled = true 2022-06-17T12:56:28.238 INFO:teuthology.orchestra.run.smithi136.stdout: rgw enable ops log = true 2022-06-17T12:56:28.238 INFO:teuthology.orchestra.run.smithi136.stdout: rgw enable usage log = true 2022-06-17T12:56:28.238 INFO:teuthology.orchestra.run.smithi136.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-17T12:56:28.238 INFO:teuthology.orchestra.run.smithi136.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-17T12:56:28.239 INFO:teuthology.orchestra.run.smithi136.stdout: client mount timeout = 600 2022-06-17T12:56:28.239 INFO:teuthology.orchestra.run.smithi136.stdout: debug client = 20 2022-06-17T12:56:28.239 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2022-06-17T12:56:28.240 INFO:teuthology.orchestra.run.smithi136.stdout: rados mon op timeout = 15m 2022-06-17T12:56:28.240 INFO:teuthology.orchestra.run.smithi136.stdout: rados osd op timeout = 15m 2022-06-17T12:56:28.240 INFO:teuthology.orchestra.run.smithi136.stdout: client debug getattr caps = true 2022-06-17T12:56:28.240 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.a] 2022-06-17T12:56:28.241 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.b] 2022-06-17T12:56:28.241 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.c] 2022-06-17T12:56:28.241 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.a] 2022-06-17T12:56:28.241 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.b] 2022-06-17T12:56:28.277 INFO:teuthology.orchestra.run.smithi136.stdout:[mds] 2022-06-17T12:56:28.277 INFO:teuthology.orchestra.run.smithi136.stdout: debug mds = 20 2022-06-17T12:56:28.291 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2022-06-17T12:56:28.292 INFO:teuthology.orchestra.run.smithi136.stdout: mds bal frag = True 2022-06-17T12:56:28.292 INFO:teuthology.orchestra.run.smithi136.stdout: mds bal fragment size max = 10000 2022-06-17T12:56:28.292 INFO:teuthology.orchestra.run.smithi136.stdout: mds bal merge size = 5 2022-06-17T12:56:28.293 INFO:teuthology.orchestra.run.smithi136.stdout: mds bal split bits = 3 2022-06-17T12:56:28.293 INFO:teuthology.orchestra.run.smithi136.stdout: mds bal split size = 100 2022-06-17T12:56:28.293 INFO:teuthology.orchestra.run.smithi136.stdout: mds debug frag = True 2022-06-17T12:56:28.293 INFO:teuthology.orchestra.run.smithi136.stdout: mds debug scatterstat = True 2022-06-17T12:56:28.294 INFO:teuthology.orchestra.run.smithi136.stdout: mds op complaint time = 180 2022-06-17T12:56:28.302 INFO:teuthology.orchestra.run.smithi136.stdout: mds verify scatter = True 2022-06-17T12:56:28.302 INFO:teuthology.orchestra.run.smithi136.stdout: osd op complaint time = 180 2022-06-17T12:56:28.303 INFO:teuthology.orchestra.run.smithi136.stdout: rados mon op timeout = 15m 2022-06-17T12:56:28.303 INFO:teuthology.orchestra.run.smithi136.stdout: rados osd op timeout = 15m 2022-06-17T12:56:28.305 INFO:teuthology.orchestra.run.smithi107.stdout:[global] 2022-06-17T12:56:28.305 INFO:teuthology.orchestra.run.smithi107.stdout: chdir = "" 2022-06-17T12:56:28.306 INFO:teuthology.orchestra.run.smithi107.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-17T12:56:28.331 INFO:teuthology.orchestra.run.smithi107.stdout: auth supported = cephx 2022-06-17T12:56:28.332 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.332 INFO:teuthology.orchestra.run.smithi107.stdout: filestore xattr use omap = true 2022-06-17T12:56:28.333 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.333 INFO:teuthology.orchestra.run.smithi107.stdout: mon clock drift allowed = 1.000 2022-06-17T12:56:28.333 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.334 INFO:teuthology.orchestra.run.smithi107.stdout: osd crush chooseleaf type = 0 2022-06-17T12:56:28.334 INFO:teuthology.orchestra.run.smithi107.stdout: auth debug = true 2022-06-17T12:56:28.335 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.335 INFO:teuthology.orchestra.run.smithi107.stdout: ms die on old message = true 2022-06-17T12:56:28.336 INFO:teuthology.orchestra.run.smithi107.stdout: ms die on bug = true 2022-06-17T12:56:28.336 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.336 INFO:teuthology.orchestra.run.smithi107.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-17T12:56:28.337 INFO:teuthology.orchestra.run.smithi107.stdout: mon pg warn max object skew = 0 2022-06-17T12:56:28.337 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.346 INFO:teuthology.orchestra.run.smithi107.stdout: # disable pg_autoscaler by default for new pools 2022-06-17T12:56:28.347 INFO:teuthology.orchestra.run.smithi107.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-17T12:56:28.347 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.347 INFO:teuthology.orchestra.run.smithi107.stdout: osd pool default size = 2 2022-06-17T12:56:28.348 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.348 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd allow primary affinity = true 2022-06-17T12:56:28.349 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd allow pg remap = true 2022-06-17T12:56:28.349 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on legacy crush tunables = false 2022-06-17T12:56:28.390 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on crush straw calc version zero = false 2022-06-17T12:56:28.390 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on no sortbitwise = false 2022-06-17T12:56:28.391 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on osd down out interval zero = false 2022-06-17T12:56:28.391 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on too few osds = false 2022-06-17T12:56:28.391 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-17T12:56:28.392 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-17T12:56:28.392 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.393 INFO:teuthology.orchestra.run.smithi107.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-17T12:56:28.393 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.393 INFO:teuthology.orchestra.run.smithi107.stdout: osd default data pool replay window = 5 2022-06-17T12:56:28.394 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.394 INFO:teuthology.orchestra.run.smithi107.stdout: mon allow pool delete = true 2022-06-17T12:56:28.395 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.395 INFO:teuthology.orchestra.run.smithi107.stdout: mon cluster log file level = debug 2022-06-17T12:56:28.395 INFO:teuthology.orchestra.run.smithi107.stdout: debug asserts on shutdown = true 2022-06-17T12:56:28.396 INFO:teuthology.orchestra.run.smithi107.stdout: mon health detail to clog = false 2022-06-17T12:56:28.396 INFO:teuthology.orchestra.run.smithi107.stdout: mon host = "172.21.15.36,172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790]" 2022-06-17T12:56:28.397 INFO:teuthology.orchestra.run.smithi107.stdout: fsid = b68b9700-0c85-42ca-94d7-6cd53db91c14 2022-06-17T12:56:28.397 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.398 INFO:teuthology.orchestra.run.smithi107.stdout:[osd] 2022-06-17T12:56:28.398 INFO:teuthology.orchestra.run.smithi107.stdout: osd journal size = 100 2022-06-17T12:56:28.398 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.399 INFO:teuthology.orchestra.run.smithi107.stdout: osd scrub load threshold = 5.0 2022-06-17T12:56:28.399 INFO:teuthology.orchestra.run.smithi107.stdout: osd scrub max interval = 600 2022-06-17T12:56:28.400 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.400 INFO:teuthology.orchestra.run.smithi107.stdout: osd recover clone overlap = true 2022-06-17T12:56:28.400 INFO:teuthology.orchestra.run.smithi107.stdout: osd recovery max chunk = 1048576 2022-06-17T12:56:28.401 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.401 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug shutdown = true 2022-06-17T12:56:28.401 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug op order = true 2022-06-17T12:56:28.402 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug verify stray on activate = true 2022-06-17T12:56:28.402 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.403 INFO:teuthology.orchestra.run.smithi107.stdout: osd open classes on start = true 2022-06-17T12:56:28.403 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug pg log writeout = true 2022-06-17T12:56:28.403 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.404 INFO:teuthology.orchestra.run.smithi107.stdout: osd deep scrub update digest min age = 30 2022-06-17T12:56:28.404 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.405 INFO:teuthology.orchestra.run.smithi107.stdout: osd map max advance = 10 2022-06-17T12:56:28.405 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.406 INFO:teuthology.orchestra.run.smithi107.stdout: journal zero on create = true 2022-06-17T12:56:28.406 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.407 INFO:teuthology.orchestra.run.smithi107.stdout: filestore ondisk finisher threads = 3 2022-06-17T12:56:28.417 INFO:teuthology.orchestra.run.smithi107.stdout: filestore apply finisher threads = 3 2022-06-17T12:56:28.417 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.418 INFO:teuthology.orchestra.run.smithi107.stdout: bdev debug aio = true 2022-06-17T12:56:28.418 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug misdirected ops = true 2022-06-17T12:56:28.418 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore block size = 96636764160 2022-06-17T12:56:28.418 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore compression mode = aggressive 2022-06-17T12:56:28.419 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore fsck on mount = True 2022-06-17T12:56:28.419 INFO:teuthology.orchestra.run.smithi107.stdout: debug bluefs = 20 2022-06-17T12:56:28.419 INFO:teuthology.orchestra.run.smithi107.stdout: debug bluestore = 20 2022-06-17T12:56:28.420 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2022-06-17T12:56:28.420 INFO:teuthology.orchestra.run.smithi107.stdout: debug osd = 20 2022-06-17T12:56:28.420 INFO:teuthology.orchestra.run.smithi107.stdout: debug rocksdb = 10 2022-06-17T12:56:28.420 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd backfillfull_ratio = 0.85 2022-06-17T12:56:28.421 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd full ratio = 0.9 2022-06-17T12:56:28.421 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd nearfull ratio = 0.8 2022-06-17T12:56:28.421 INFO:teuthology.orchestra.run.smithi107.stdout: osd failsafe full ratio = 0.95 2022-06-17T12:56:28.422 INFO:teuthology.orchestra.run.smithi107.stdout: osd objectstore = bluestore 2022-06-17T12:56:28.422 INFO:teuthology.orchestra.run.smithi107.stdout: osd op complaint time = 180 2022-06-17T12:56:28.422 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.422 INFO:teuthology.orchestra.run.smithi107.stdout:[mgr] 2022-06-17T12:56:28.422 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2022-06-17T12:56:28.423 INFO:teuthology.orchestra.run.smithi107.stdout: debug mgr = 20 2022-06-17T12:56:28.423 INFO:teuthology.orchestra.run.smithi107.stdout: debug mon = 20 2022-06-17T12:56:28.423 INFO:teuthology.orchestra.run.smithi107.stdout: debug auth = 20 2022-06-17T12:56:28.423 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:56:28.425 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:56:28.425 INFO:teuthology.orchestra.run.smithi107.stdout: mgr/telemetry/nag = false 2022-06-17T12:56:28.426 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.427 INFO:teuthology.orchestra.run.smithi107.stdout:[mon] 2022-06-17T12:56:28.427 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2022-06-17T12:56:28.432 INFO:teuthology.orchestra.run.smithi107.stdout: debug mon = 20 2022-06-17T12:56:28.432 INFO:teuthology.orchestra.run.smithi107.stdout: debug paxos = 20 2022-06-17T12:56:28.433 INFO:teuthology.orchestra.run.smithi107.stdout: debug auth = 20 2022-06-17T12:56:28.433 INFO:teuthology.orchestra.run.smithi107.stdout: mon data avail warn = 5 2022-06-17T12:56:28.433 INFO:teuthology.orchestra.run.smithi107.stdout: mon mgr mkfs grace = 240 2022-06-17T12:56:28.433 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:56:28.434 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd reporter subtree level = osd 2022-06-17T12:56:28.434 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd prime pg temp = true 2022-06-17T12:56:28.434 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:56:28.435 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.435 INFO:teuthology.orchestra.run.smithi107.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-17T12:56:28.435 INFO:teuthology.orchestra.run.smithi107.stdout: auth mon ticket ttl = 660 # 11m 2022-06-17T12:56:28.435 INFO:teuthology.orchestra.run.smithi107.stdout: auth service ticket ttl = 240 # 4m 2022-06-17T12:56:28.436 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.436 INFO:teuthology.orchestra.run.smithi107.stdout: # don't complain about insecure global_id in the test suite 2022-06-17T12:56:28.436 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-17T12:56:28.436 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-17T12:56:28.437 INFO:teuthology.orchestra.run.smithi107.stdout: mon op complaint time = 120 2022-06-17T12:56:28.437 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:28.437 INFO:teuthology.orchestra.run.smithi107.stdout:[client] 2022-06-17T12:56:28.437 INFO:teuthology.orchestra.run.smithi107.stdout: rgw cache enabled = true 2022-06-17T12:56:28.444 INFO:teuthology.orchestra.run.smithi107.stdout: rgw enable ops log = true 2022-06-17T12:56:28.445 INFO:teuthology.orchestra.run.smithi107.stdout: rgw enable usage log = true 2022-06-17T12:56:28.445 INFO:teuthology.orchestra.run.smithi107.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-17T12:56:28.445 INFO:teuthology.orchestra.run.smithi107.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-17T12:56:28.445 INFO:teuthology.orchestra.run.smithi107.stdout: client mount timeout = 600 2022-06-17T12:56:28.445 INFO:teuthology.orchestra.run.smithi107.stdout: debug client = 20 2022-06-17T12:56:28.446 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2022-06-17T12:56:28.446 INFO:teuthology.orchestra.run.smithi107.stdout: rados mon op timeout = 15m 2022-06-17T12:56:28.455 INFO:teuthology.orchestra.run.smithi107.stdout: rados osd op timeout = 15m 2022-06-17T12:56:28.456 INFO:teuthology.orchestra.run.smithi107.stdout: client debug getattr caps = true 2022-06-17T12:56:28.456 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.a] 2022-06-17T12:56:28.457 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.b] 2022-06-17T12:56:28.457 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.c] 2022-06-17T12:56:28.457 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.a] 2022-06-17T12:56:28.458 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.b] 2022-06-17T12:56:28.459 INFO:teuthology.orchestra.run.smithi107.stdout:[mds] 2022-06-17T12:56:28.459 INFO:teuthology.orchestra.run.smithi107.stdout: debug mds = 20 2022-06-17T12:56:28.459 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2022-06-17T12:56:28.460 INFO:teuthology.orchestra.run.smithi107.stdout: mds bal frag = True 2022-06-17T12:56:28.460 INFO:teuthology.orchestra.run.smithi107.stdout: mds bal fragment size max = 10000 2022-06-17T12:56:28.461 INFO:teuthology.orchestra.run.smithi107.stdout: mds bal merge size = 5 2022-06-17T12:56:28.461 INFO:teuthology.orchestra.run.smithi107.stdout: mds bal split bits = 3 2022-06-17T12:56:28.462 INFO:teuthology.orchestra.run.smithi107.stdout: mds bal split size = 100 2022-06-17T12:56:28.463 INFO:teuthology.orchestra.run.smithi107.stdout: mds debug frag = True 2022-06-17T12:56:28.463 INFO:teuthology.orchestra.run.smithi107.stdout: mds debug scatterstat = True 2022-06-17T12:56:28.464 INFO:teuthology.orchestra.run.smithi107.stdout: mds op complaint time = 180 2022-06-17T12:56:28.467 INFO:teuthology.orchestra.run.smithi107.stdout: mds verify scatter = True 2022-06-17T12:56:28.467 INFO:teuthology.orchestra.run.smithi107.stdout: osd op complaint time = 180 2022-06-17T12:56:28.468 INFO:teuthology.orchestra.run.smithi107.stdout: rados mon op timeout = 15m 2022-06-17T12:56:28.468 INFO:teuthology.orchestra.run.smithi107.stdout: rados osd op timeout = 15m 2022-06-17T12:56:28.469 INFO:teuthology.orchestra.run.smithi070.stdout:[global] 2022-06-17T12:56:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: chdir = "" 2022-06-17T12:56:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-17T12:56:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: auth supported = cephx 2022-06-17T12:56:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: filestore xattr use omap = true 2022-06-17T12:56:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: mon clock drift allowed = 1.000 2022-06-17T12:56:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: osd crush chooseleaf type = 0 2022-06-17T12:56:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: auth debug = true 2022-06-17T12:56:28.475 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: ms die on old message = true 2022-06-17T12:56:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: ms die on bug = true 2022-06-17T12:56:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-17T12:56:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: mon pg warn max object skew = 0 2022-06-17T12:56:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: # disable pg_autoscaler by default for new pools 2022-06-17T12:56:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-17T12:56:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: osd pool default size = 2 2022-06-17T12:56:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd allow primary affinity = true 2022-06-17T12:56:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd allow pg remap = true 2022-06-17T12:56:28.484 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on legacy crush tunables = false 2022-06-17T12:56:28.484 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on crush straw calc version zero = false 2022-06-17T12:56:28.485 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on no sortbitwise = false 2022-06-17T12:56:28.485 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on osd down out interval zero = false 2022-06-17T12:56:28.490 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on too few osds = false 2022-06-17T12:56:28.491 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-17T12:56:28.491 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-17T12:56:28.495 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.496 INFO:teuthology.orchestra.run.smithi070.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-17T12:56:28.496 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.497 INFO:teuthology.orchestra.run.smithi070.stdout: osd default data pool replay window = 5 2022-06-17T12:56:28.497 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.497 INFO:teuthology.orchestra.run.smithi070.stdout: mon allow pool delete = true 2022-06-17T12:56:28.498 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.498 INFO:teuthology.orchestra.run.smithi070.stdout: mon cluster log file level = debug 2022-06-17T12:56:28.498 INFO:teuthology.orchestra.run.smithi070.stdout: debug asserts on shutdown = true 2022-06-17T12:56:28.499 INFO:teuthology.orchestra.run.smithi070.stdout: mon health detail to clog = false 2022-06-17T12:56:28.499 INFO:teuthology.orchestra.run.smithi070.stdout: mon host = "172.21.15.36,172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790]" 2022-06-17T12:56:28.499 INFO:teuthology.orchestra.run.smithi070.stdout: fsid = b68b9700-0c85-42ca-94d7-6cd53db91c14 2022-06-17T12:56:28.499 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.499 INFO:teuthology.orchestra.run.smithi070.stdout:[osd] 2022-06-17T12:56:28.500 INFO:teuthology.orchestra.run.smithi070.stdout: osd journal size = 100 2022-06-17T12:56:28.500 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.500 INFO:teuthology.orchestra.run.smithi070.stdout: osd scrub load threshold = 5.0 2022-06-17T12:56:28.501 INFO:teuthology.orchestra.run.smithi070.stdout: osd scrub max interval = 600 2022-06-17T12:56:28.501 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.502 INFO:teuthology.orchestra.run.smithi070.stdout: osd recover clone overlap = true 2022-06-17T12:56:28.502 INFO:teuthology.orchestra.run.smithi070.stdout: osd recovery max chunk = 1048576 2022-06-17T12:56:28.502 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.503 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug shutdown = true 2022-06-17T12:56:28.503 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug op order = true 2022-06-17T12:56:28.504 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug verify stray on activate = true 2022-06-17T12:56:28.504 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.504 INFO:teuthology.orchestra.run.smithi070.stdout: osd open classes on start = true 2022-06-17T12:56:28.506 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug pg log writeout = true 2022-06-17T12:56:28.506 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.507 INFO:teuthology.orchestra.run.smithi070.stdout: osd deep scrub update digest min age = 30 2022-06-17T12:56:28.507 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.507 INFO:teuthology.orchestra.run.smithi070.stdout: osd map max advance = 10 2022-06-17T12:56:28.508 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.509 INFO:teuthology.orchestra.run.smithi070.stdout: journal zero on create = true 2022-06-17T12:56:28.509 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.510 INFO:teuthology.orchestra.run.smithi070.stdout: filestore ondisk finisher threads = 3 2022-06-17T12:56:28.510 INFO:teuthology.orchestra.run.smithi070.stdout: filestore apply finisher threads = 3 2022-06-17T12:56:28.510 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.511 INFO:teuthology.orchestra.run.smithi070.stdout: bdev debug aio = true 2022-06-17T12:56:28.511 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug misdirected ops = true 2022-06-17T12:56:28.512 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore block size = 96636764160 2022-06-17T12:56:28.512 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore compression mode = aggressive 2022-06-17T12:56:28.513 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore fsck on mount = True 2022-06-17T12:56:28.513 INFO:teuthology.orchestra.run.smithi070.stdout: debug bluefs = 20 2022-06-17T12:56:28.514 INFO:teuthology.orchestra.run.smithi070.stdout: debug bluestore = 20 2022-06-17T12:56:28.514 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2022-06-17T12:56:28.514 INFO:teuthology.orchestra.run.smithi070.stdout: debug osd = 20 2022-06-17T12:56:28.515 INFO:teuthology.orchestra.run.smithi070.stdout: debug rocksdb = 10 2022-06-17T12:56:28.515 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd backfillfull_ratio = 0.85 2022-06-17T12:56:28.515 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd full ratio = 0.9 2022-06-17T12:56:28.517 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd nearfull ratio = 0.8 2022-06-17T12:56:28.518 INFO:teuthology.orchestra.run.smithi070.stdout: osd failsafe full ratio = 0.95 2022-06-17T12:56:28.518 INFO:teuthology.orchestra.run.smithi070.stdout: osd objectstore = bluestore 2022-06-17T12:56:28.518 INFO:teuthology.orchestra.run.smithi070.stdout: osd op complaint time = 180 2022-06-17T12:56:28.519 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.519 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr] 2022-06-17T12:56:28.520 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2022-06-17T12:56:28.520 INFO:teuthology.orchestra.run.smithi070.stdout: debug mgr = 20 2022-06-17T12:56:28.521 INFO:teuthology.orchestra.run.smithi070.stdout: debug mon = 20 2022-06-17T12:56:28.521 INFO:teuthology.orchestra.run.smithi070.stdout: debug auth = 20 2022-06-17T12:56:28.521 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:56:28.522 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:56:28.522 INFO:teuthology.orchestra.run.smithi070.stdout: mgr/telemetry/nag = false 2022-06-17T12:56:28.523 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.523 INFO:teuthology.orchestra.run.smithi070.stdout:[mon] 2022-06-17T12:56:28.524 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2022-06-17T12:56:28.524 INFO:teuthology.orchestra.run.smithi070.stdout: debug mon = 20 2022-06-17T12:56:28.525 INFO:teuthology.orchestra.run.smithi070.stdout: debug paxos = 20 2022-06-17T12:56:28.525 INFO:teuthology.orchestra.run.smithi070.stdout: debug auth = 20 2022-06-17T12:56:28.525 INFO:teuthology.orchestra.run.smithi070.stdout: mon data avail warn = 5 2022-06-17T12:56:28.526 INFO:teuthology.orchestra.run.smithi070.stdout: mon mgr mkfs grace = 240 2022-06-17T12:56:28.527 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:56:28.527 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd reporter subtree level = osd 2022-06-17T12:56:28.528 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd prime pg temp = true 2022-06-17T12:56:28.528 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:56:28.528 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.529 INFO:teuthology.orchestra.run.smithi070.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-17T12:56:28.529 INFO:teuthology.orchestra.run.smithi070.stdout: auth mon ticket ttl = 660 # 11m 2022-06-17T12:56:28.529 INFO:teuthology.orchestra.run.smithi070.stdout: auth service ticket ttl = 240 # 4m 2022-06-17T12:56:28.530 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.530 INFO:teuthology.orchestra.run.smithi070.stdout: # don't complain about insecure global_id in the test suite 2022-06-17T12:56:28.531 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-17T12:56:28.531 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-17T12:56:28.531 INFO:teuthology.orchestra.run.smithi070.stdout: mon op complaint time = 120 2022-06-17T12:56:28.532 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:28.532 INFO:teuthology.orchestra.run.smithi070.stdout:[client] 2022-06-17T12:56:28.532 INFO:teuthology.orchestra.run.smithi070.stdout: rgw cache enabled = true 2022-06-17T12:56:28.533 INFO:teuthology.orchestra.run.smithi070.stdout: rgw enable ops log = true 2022-06-17T12:56:28.533 INFO:teuthology.orchestra.run.smithi070.stdout: rgw enable usage log = true 2022-06-17T12:56:28.533 INFO:teuthology.orchestra.run.smithi070.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-17T12:56:28.534 INFO:teuthology.orchestra.run.smithi070.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-17T12:56:28.546 INFO:teuthology.orchestra.run.smithi070.stdout: client mount timeout = 600 2022-06-17T12:56:28.547 INFO:teuthology.orchestra.run.smithi070.stdout: debug client = 20 2022-06-17T12:56:28.548 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2022-06-17T12:56:28.548 INFO:teuthology.orchestra.run.smithi070.stdout: rados mon op timeout = 15m 2022-06-17T12:56:28.548 INFO:teuthology.orchestra.run.smithi070.stdout: rados osd op timeout = 15m 2022-06-17T12:56:28.548 INFO:teuthology.orchestra.run.smithi070.stdout: client debug getattr caps = true 2022-06-17T12:56:28.549 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.a] 2022-06-17T12:56:28.550 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.b] 2022-06-17T12:56:28.551 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.c] 2022-06-17T12:56:28.552 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.a] 2022-06-17T12:56:28.552 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.b] 2022-06-17T12:56:28.552 INFO:teuthology.orchestra.run.smithi070.stdout:[mds] 2022-06-17T12:56:28.553 INFO:teuthology.orchestra.run.smithi070.stdout: debug mds = 20 2022-06-17T12:56:28.553 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2022-06-17T12:56:28.554 INFO:teuthology.orchestra.run.smithi070.stdout: mds bal frag = True 2022-06-17T12:56:28.554 INFO:teuthology.orchestra.run.smithi070.stdout: mds bal fragment size max = 10000 2022-06-17T12:56:28.554 INFO:teuthology.orchestra.run.smithi070.stdout: mds bal merge size = 5 2022-06-17T12:56:28.555 INFO:teuthology.orchestra.run.smithi070.stdout: mds bal split bits = 3 2022-06-17T12:56:28.555 INFO:teuthology.orchestra.run.smithi070.stdout: mds bal split size = 100 2022-06-17T12:56:28.555 INFO:teuthology.orchestra.run.smithi070.stdout: mds debug frag = True 2022-06-17T12:56:28.556 INFO:teuthology.orchestra.run.smithi070.stdout: mds debug scatterstat = True 2022-06-17T12:56:28.556 INFO:teuthology.orchestra.run.smithi070.stdout: mds op complaint time = 180 2022-06-17T12:56:28.557 INFO:teuthology.orchestra.run.smithi070.stdout: mds verify scatter = True 2022-06-17T12:56:28.557 INFO:teuthology.orchestra.run.smithi070.stdout: osd op complaint time = 180 2022-06-17T12:56:28.558 INFO:teuthology.orchestra.run.smithi070.stdout: rados mon op timeout = 15m 2022-06-17T12:56:28.558 INFO:teuthology.orchestra.run.smithi070.stdout: rados osd op timeout = 15m 2022-06-17T12:56:28.560 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:56:28.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.682+0000 7f14451fe700 1 Processor -- start 2022-06-17T12:56:28.675 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.682+0000 7f14451fe700 1 -- start start 2022-06-17T12:56:28.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f14451fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f14400b6530 0x7f14400b2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:28.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f14451fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14400b4160 0x7f14400b2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:28.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f14451fe700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f14400b2a40 0x7f144014bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:28.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f14451fe700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14400b6910 con 0x7f14400b6530 2022-06-17T12:56:28.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f14451fe700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14400b3570 con 0x7f14400b2a40 2022-06-17T12:56:28.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f14451fe700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14400b3810 con 0x7f14400b4160 2022-06-17T12:56:28.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f143ffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f14400b6530 0x7f14400b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:28.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f143ffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f14400b6530 0x7f14400b2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39548/0 (socket says 172.21.15.36:39548) 2022-06-17T12:56:28.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f143ffff700 1 -- 172.21.15.36:0/1071887970 learned_addr learned my addr 172.21.15.36:0/1071887970 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:28.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f14449fd700 1 --2- 172.21.15.36:0/1071887970 >> v2:172.21.15.70:3300/0 conn(0x7f14400b2a40 0x7f144014bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:28.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f143f7fe700 1 --2- 172.21.15.36:0/1071887970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14400b4160 0x7f14400b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:28.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f143ffff700 1 -- 172.21.15.36:0/1071887970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14400b4160 msgr2=0x7f14400b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:28.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f143ffff700 1 --2- 172.21.15.36:0/1071887970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14400b4160 0x7f14400b2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:28.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f143ffff700 1 -- 172.21.15.36:0/1071887970 >> v2:172.21.15.70:3300/0 conn(0x7f14400b2a40 msgr2=0x7f144014bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:28.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f143ffff700 1 --2- 172.21.15.36:0/1071887970 >> v2:172.21.15.70:3300/0 conn(0x7f14400b2a40 0x7f144014bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:28.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f143ffff700 1 -- 172.21.15.36:0/1071887970 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14400b7190 con 0x7f14400b6530 2022-06-17T12:56:28.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f143ffff700 1 --2- 172.21.15.36:0/1071887970 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14400b6530 0x7f14400b2060 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7f1428002f70 tx=0x7f1428006eb0).ready entity=mon.0 client_cookie=989bb7f6cc28fa7b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:28.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f143effd700 1 -- 172.21.15.36:0/1071887970 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f142800d570 con 0x7f14400b6530 2022-06-17T12:56:28.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f143effd700 1 -- 172.21.15.36:0/1071887970 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1428000e90 con 0x7f14400b6530 2022-06-17T12:56:28.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f143effd700 1 -- 172.21.15.36:0/1071887970 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f142800d900 con 0x7f14400b6530 2022-06-17T12:56:28.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f14451fe700 1 -- 172.21.15.36:0/1071887970 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14400b6530 msgr2=0x7f14400b2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:28.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f14451fe700 1 --2- 172.21.15.36:0/1071887970 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14400b6530 0x7f14400b2060 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7f1428002f70 tx=0x7f1428006eb0).stop 2022-06-17T12:56:28.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f14451fe700 1 -- 172.21.15.36:0/1071887970 shutdown_connections 2022-06-17T12:56:28.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f14451fe700 1 --2- 172.21.15.36:0/1071887970 >> v2:172.21.15.70:3300/0 conn(0x7f14400b2a40 0x7f144014bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:28.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f14451fe700 1 --2- 172.21.15.36:0/1071887970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14400b4160 0x7f14400b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:28.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f14451fe700 1 --2- 172.21.15.36:0/1071887970 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14400b6530 0x7f14400b2060 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:28.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.686+0000 7f14451fe700 1 -- 172.21.15.36:0/1071887970 >> 172.21.15.36:0/1071887970 conn(0x7f144005be50 msgr2=0x7f144005c260 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:28.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f14451fe700 1 -- 172.21.15.36:0/1071887970 shutdown_connections 2022-06-17T12:56:28.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f14451fe700 1 -- 172.21.15.36:0/1071887970 wait complete. 2022-06-17T12:56:28.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f14451fe700 1 Processor -- start 2022-06-17T12:56:28.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f14451fe700 1 -- start start 2022-06-17T12:56:28.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f14451fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14400b4160 0x7f14401459b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:28.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f14451fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14400b6530 0x7f144013f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:28.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f14451fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1440152240 0x7f144013f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:28.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f14451fe700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14401439e0 con 0x7f14400b4160 2022-06-17T12:56:28.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f14451fe700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1440142c80 con 0x7f14400b6530 2022-06-17T12:56:28.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f14451fe700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1440143c10 con 0x7f1440152240 2022-06-17T12:56:28.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f143ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14400b4160 0x7f14401459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:28.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f143ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14400b4160 0x7f14401459b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39552/0 (socket says 172.21.15.36:39552) 2022-06-17T12:56:28.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f143ffff700 1 -- 172.21.15.36:0/2471331705 learned_addr learned my addr 172.21.15.36:0/2471331705 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:28.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f14449fd700 1 --2- 172.21.15.36:0/2471331705 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1440152240 0x7f144013f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:28.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f143f7fe700 1 --2- 172.21.15.36:0/2471331705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14400b6530 0x7f144013f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:28.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f14449fd700 1 -- 172.21.15.36:0/2471331705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14400b6530 msgr2=0x7f144013f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:28.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f14449fd700 1 --2- 172.21.15.36:0/2471331705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14400b6530 0x7f144013f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:28.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f14449fd700 1 -- 172.21.15.36:0/2471331705 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14400b4160 msgr2=0x7f14401459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:28.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f14449fd700 1 --2- 172.21.15.36:0/2471331705 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14400b4160 0x7f14401459b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:28.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f14449fd700 1 -- 172.21.15.36:0/2471331705 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142800b1e0 con 0x7f1440152240 2022-06-17T12:56:28.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f14449fd700 1 --2- 172.21.15.36:0/2471331705 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1440152240 0x7f144013f530 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f1440142f50 tx=0x7f14380091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:28.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f143d7fa700 1 -- 172.21.15.36:0/2471331705 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f143800c670 con 0x7f1440152240 2022-06-17T12:56:28.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f143d7fa700 1 -- 172.21.15.36:0/2471331705 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143800c7d0 con 0x7f1440152240 2022-06-17T12:56:28.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f14451fe700 1 -- 172.21.15.36:0/2471331705 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f144013fa20 con 0x7f1440152240 2022-06-17T12:56:28.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f143d7fa700 1 -- 172.21.15.36:0/2471331705 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f143800cb30 con 0x7f1440152240 2022-06-17T12:56:28.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.690+0000 7f14451fe700 1 -- 172.21.15.36:0/2471331705 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14400ade80 con 0x7f1440152240 2022-06-17T12:56:28.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.694+0000 7f14451fe700 1 -- 172.21.15.36:0/2471331705 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1440057410 con 0x7f1440152240 2022-06-17T12:56:28.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.694+0000 7f143d7fa700 1 -- 172.21.15.36:0/2471331705 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f1438022070 con 0x7f1440152240 2022-06-17T12:56:28.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.694+0000 7f143d7fa700 1 --2- 172.21.15.36:0/2471331705 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1424053ad0 0x7f1424055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:28.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.694+0000 7f143d7fa700 1 -- 172.21.15.36:0/2471331705 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f143806c780 con 0x7f1440152240 2022-06-17T12:56:28.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.694+0000 7f143ffff700 1 --2- 172.21.15.36:0/2471331705 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1424053ad0 0x7f1424055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:28.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.694+0000 7f143ffff700 1 --2- 172.21.15.36:0/2471331705 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1424053ad0 0x7f1424055f90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f142801c000 tx=0x7f1428015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:28.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.702+0000 7f143d7fa700 1 -- 172.21.15.36:0/2471331705 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f14380433f0 con 0x7f1440152240 2022-06-17T12:56:28.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.830+0000 7f14451fe700 1 -- 172.21.15.36:0/2471331705 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14400af400 con 0x7f1440152240 2022-06-17T12:56:28.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.838+0000 7f143d7fa700 1 -- 172.21.15.36:0/2471331705 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f14400af400 con 0x7f1440152240 2022-06-17T12:56:28.844 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:28.845 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":131,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:56:12.025286+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:56:08.924639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-06-17T12:56:09.989204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:56:28.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.838+0000 7f14451fe700 1 -- 172.21.15.36:0/2471331705 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1424053ad0 msgr2=0x7f1424055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:28.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.838+0000 7f14451fe700 1 --2- 172.21.15.36:0/2471331705 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1424053ad0 0x7f1424055f90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f142801c000 tx=0x7f1428015000).stop 2022-06-17T12:56:28.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.838+0000 7f14451fe700 1 -- 172.21.15.36:0/2471331705 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1440152240 msgr2=0x7f144013f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:28.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.838+0000 7f14451fe700 1 --2- 172.21.15.36:0/2471331705 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1440152240 0x7f144013f530 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f1440142f50 tx=0x7f14380091d0).stop 2022-06-17T12:56:28.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.838+0000 7f14451fe700 1 -- 172.21.15.36:0/2471331705 shutdown_connections 2022-06-17T12:56:28.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.838+0000 7f14451fe700 1 --2- 172.21.15.36:0/2471331705 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1424053ad0 0x7f1424055f90 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:28.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.838+0000 7f14451fe700 1 --2- 172.21.15.36:0/2471331705 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1440152240 0x7f144013f530 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:28.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.838+0000 7f14451fe700 1 --2- 172.21.15.36:0/2471331705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14400b6530 0x7f144013f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:28.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.838+0000 7f14451fe700 1 --2- 172.21.15.36:0/2471331705 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f14400b4160 0x7f14401459b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:28.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.838+0000 7f14451fe700 1 -- 172.21.15.36:0/2471331705 >> 172.21.15.36:0/2471331705 conn(0x7f144005be50 msgr2=0x7f14400b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:28.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.838+0000 7f14451fe700 1 -- 172.21.15.36:0/2471331705 shutdown_connections 2022-06-17T12:56:28.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.838+0000 7f14451fe700 1 -- 172.21.15.36:0/2471331705 wait complete. 2022-06-17T12:56:28.858 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:56:08.924639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '130', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:56:09.989204+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:56:28.859 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:56:29.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.998+0000 7f0493fff700 1 Processor -- start 2022-06-17T12:56:29.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:28.998+0000 7f0493fff700 1 -- start start 2022-06-17T12:56:29.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f0493fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04940b22f0 0x7f049414a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:29.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f0493fff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0494061880 con 0x7f04940b3a20 2022-06-17T12:56:29.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f0493fff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0494140000 con 0x7f04940b1e80 2022-06-17T12:56:29.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f0493fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04940ae870 con 0x7f04940b22f0 2022-06-17T12:56:29.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f0492ffd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f04940b3a20 0x7f0494060fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57090/0 (socket says 172.21.15.36:57090) 2022-06-17T12:56:29.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f0492ffd700 1 -- 172.21.15.36:0/2041931120 learned_addr learned my addr 172.21.15.36:0/2041931120 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:29.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f04937fe700 1 --2- 172.21.15.36:0/2041931120 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04940b22f0 0x7f049414a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:29.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f0491ffb700 1 -- 172.21.15.36:0/2041931120 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1337256910 0 0) 0x7f0494061880 con 0x7f04940b3a20 2022-06-17T12:56:29.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f0491ffb700 1 -- 172.21.15.36:0/2041931120 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0478003260 con 0x7f04940b3a20 2022-06-17T12:56:29.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f0491ffb700 1 -- 172.21.15.36:0/2041931120 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2512304090 0 0) 0x7f0494140000 con 0x7f04940b1e80 2022-06-17T12:56:29.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f0491ffb700 1 -- 172.21.15.36:0/2041931120 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0494061880 con 0x7f04940b1e80 2022-06-17T12:56:29.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f04937fe700 1 -- 172.21.15.36:0/2041931120 >> v1:172.21.15.70:6789/0 conn(0x7f04940b1e80 legacy=0x7f04941404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:29.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f04937fe700 1 -- 172.21.15.36:0/2041931120 >> v1:172.21.15.36:6789/0 conn(0x7f04940b3a20 legacy=0x7f0494060fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:29.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f04937fe700 1 -- 172.21.15.36:0/2041931120 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0494140bb0 con 0x7f04940b22f0 2022-06-17T12:56:29.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f04937fe700 1 --2- 172.21.15.36:0/2041931120 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04940b22f0 0x7f049414a8f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f048c0032f0 tx=0x7f048c0091d0).ready entity=mon.2 client_cookie=6661a70b90b6dd16 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:29.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f0491ffb700 1 -- 172.21.15.36:0/2041931120 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f048c00b640 con 0x7f04940b22f0 2022-06-17T12:56:29.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f0491ffb700 1 -- 172.21.15.36:0/2041931120 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048c00b7a0 con 0x7f04940b22f0 2022-06-17T12:56:29.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f0491ffb700 1 -- 172.21.15.36:0/2041931120 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f048c00b910 con 0x7f04940b22f0 2022-06-17T12:56:29.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f0493fff700 1 -- 172.21.15.36:0/2041931120 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04940b22f0 msgr2=0x7f049414a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:29.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f0493fff700 1 --2- 172.21.15.36:0/2041931120 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04940b22f0 0x7f049414a8f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f048c0032f0 tx=0x7f048c0091d0).stop 2022-06-17T12:56:29.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f0493fff700 1 -- 172.21.15.36:0/2041931120 shutdown_connections 2022-06-17T12:56:29.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f0493fff700 1 --2- 172.21.15.36:0/2041931120 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04940b22f0 0x7f049414a8f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f0493fff700 1 -- 172.21.15.36:0/2041931120 >> 172.21.15.36:0/2041931120 conn(0x7f04940b6fa0 msgr2=0x7f04940b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:29.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f0493fff700 1 -- 172.21.15.36:0/2041931120 shutdown_connections 2022-06-17T12:56:29.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.002+0000 7f0493fff700 1 -- 172.21.15.36:0/2041931120 wait complete. 2022-06-17T12:56:29.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.006+0000 7f0493fff700 1 Processor -- start 2022-06-17T12:56:29.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.006+0000 7f0493fff700 1 -- start start 2022-06-17T12:56:29.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.006+0000 7f0493fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04940b1e80 0x7f0494066dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:29.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.006+0000 7f0493fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04940b22f0 0x7f0494141500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:29.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.006+0000 7f0493fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04940b3a20 0x7f0494138500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:29.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.006+0000 7f0493fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049413fa50 con 0x7f04940b22f0 2022-06-17T12:56:29.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.006+0000 7f0493fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049413cd10 con 0x7f04940b3a20 2022-06-17T12:56:29.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.006+0000 7f0493fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049413f7b0 con 0x7f04940b1e80 2022-06-17T12:56:29.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.006+0000 7f04927fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04940b22f0 0x7f0494141500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:29.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.006+0000 7f04927fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04940b22f0 0x7f0494141500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39568/0 (socket says 172.21.15.36:39568) 2022-06-17T12:56:29.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.006+0000 7f04927fc700 1 -- 172.21.15.36:0/2855284112 learned_addr learned my addr 172.21.15.36:0/2855284112 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:29.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.006+0000 7f0492ffd700 1 --2- 172.21.15.36:0/2855284112 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04940b1e80 0x7f0494066dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:29.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.006+0000 7f04937fe700 1 --2- 172.21.15.36:0/2855284112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04940b3a20 0x7f0494138500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:29.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.006+0000 7f04927fc700 1 -- 172.21.15.36:0/2855284112 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04940b1e80 msgr2=0x7f0494066dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:29.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.006+0000 7f04927fc700 1 --2- 172.21.15.36:0/2855284112 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04940b1e80 0x7f0494066dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.006+0000 7f04927fc700 1 -- 172.21.15.36:0/2855284112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04940b3a20 msgr2=0x7f0494138500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:29.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.006+0000 7f04927fc700 1 --2- 172.21.15.36:0/2855284112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04940b3a20 0x7f0494138500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.006+0000 7f04927fc700 1 -- 172.21.15.36:0/2855284112 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048c009470 con 0x7f04940b22f0 2022-06-17T12:56:29.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.006+0000 7f04927fc700 1 --2- 172.21.15.36:0/2855284112 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04940b22f0 0x7f0494141500 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 rx=0x7f0488001110 tx=0x7f0488007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:29.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.006+0000 7f0483fff700 1 -- 172.21.15.36:0/2855284112 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0488012400 con 0x7f04940b22f0 2022-06-17T12:56:29.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.006+0000 7f0493fff700 1 -- 172.21.15.36:0/2855284112 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04940642f0 con 0x7f04940b22f0 2022-06-17T12:56:29.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.006+0000 7f0483fff700 1 -- 172.21.15.36:0/2855284112 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0488012560 con 0x7f04940b22f0 2022-06-17T12:56:29.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.006+0000 7f0483fff700 1 -- 172.21.15.36:0/2855284112 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04880128c0 con 0x7f04940b22f0 2022-06-17T12:56:29.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.006+0000 7f0493fff700 1 -- 172.21.15.36:0/2855284112 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04940649b0 con 0x7f04940b22f0 2022-06-17T12:56:29.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.010+0000 7f0493fff700 1 -- 172.21.15.36:0/2855284112 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0494064480 con 0x7f04940b22f0 2022-06-17T12:56:29.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.010+0000 7f0483fff700 1 -- 172.21.15.36:0/2855284112 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f0488019030 con 0x7f04940b22f0 2022-06-17T12:56:29.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.010+0000 7f0483fff700 1 --2- 172.21.15.36:0/2855284112 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f04700539d0 0x7f0470055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:29.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.010+0000 7f0483fff700 1 -- 172.21.15.36:0/2855284112 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f048806b470 con 0x7f04940b22f0 2022-06-17T12:56:29.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.010+0000 7f0492ffd700 1 --2- 172.21.15.36:0/2855284112 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f04700539d0 0x7f0470055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:29.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.010+0000 7f0492ffd700 1 --2- 172.21.15.36:0/2855284112 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f04700539d0 0x7f0470055e90 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f048400d000 tx=0x7f0484009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:29.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.018+0000 7f0483fff700 1 -- 172.21.15.36:0/2855284112 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f04880421d0 con 0x7f04940b22f0 2022-06-17T12:56:29.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.142+0000 7f0493fff700 1 -- 172.21.15.36:0/2855284112 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0494057410 con 0x7f04940b22f0 2022-06-17T12:56:29.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.146+0000 7f0483fff700 1 -- 172.21.15.36:0/2855284112 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v131) v1 ==== 110+0+10 (secure 0 0 0) 0x7f04880457f0 con 0x7f04940b22f0 2022-06-17T12:56:29.147 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:56:29.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.146+0000 7f0493fff700 1 -- 172.21.15.36:0/2855284112 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f04700539d0 msgr2=0x7f0470055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:29.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.146+0000 7f0493fff700 1 --2- 172.21.15.36:0/2855284112 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f04700539d0 0x7f0470055e90 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f048400d000 tx=0x7f0484009040).stop 2022-06-17T12:56:29.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.146+0000 7f0493fff700 1 -- 172.21.15.36:0/2855284112 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04940b22f0 msgr2=0x7f0494141500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:29.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.146+0000 7f0493fff700 1 --2- 172.21.15.36:0/2855284112 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04940b22f0 0x7f0494141500 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 rx=0x7f0488001110 tx=0x7f0488007ae0).stop 2022-06-17T12:56:29.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.146+0000 7f0493fff700 1 -- 172.21.15.36:0/2855284112 shutdown_connections 2022-06-17T12:56:29.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.146+0000 7f0493fff700 1 --2- 172.21.15.36:0/2855284112 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f04700539d0 0x7f0470055e90 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.146+0000 7f0493fff700 1 --2- 172.21.15.36:0/2855284112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04940b3a20 0x7f0494138500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.146+0000 7f0493fff700 1 --2- 172.21.15.36:0/2855284112 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f04940b22f0 0x7f0494141500 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.146+0000 7f0493fff700 1 --2- 172.21.15.36:0/2855284112 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04940b1e80 0x7f0494066dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.146+0000 7f0493fff700 1 -- 172.21.15.36:0/2855284112 >> 172.21.15.36:0/2855284112 conn(0x7f04940b6fa0 msgr2=0x7f04940b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:29.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.146+0000 7f0493fff700 1 -- 172.21.15.36:0/2855284112 shutdown_connections 2022-06-17T12:56:29.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.146+0000 7f0493fff700 1 -- 172.21.15.36:0/2855284112 wait complete. 2022-06-17T12:56:29.161 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:56:29.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.290+0000 7f8524162700 1 Processor -- start 2022-06-17T12:56:29.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.290+0000 7f8524162700 1 -- start start 2022-06-17T12:56:29.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.290+0000 7f8524162700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f851c0ba3c0 0x7f851c0b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:29.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.290+0000 7f8524162700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f851c0b3dc0 0x7f851c142210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:29.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f8524162700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f851c0ba7a0 con 0x7f851c0b4310 2022-06-17T12:56:29.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f8524162700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f851c0b5610 con 0x7f851c0b3dc0 2022-06-17T12:56:29.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f8524162700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851c0b50d0 con 0x7f851c0ba3c0 2022-06-17T12:56:29.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f85226ff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f851c0b3dc0 0x7f851c142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:29.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f85226ff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f851c0b3dc0 0x7f851c142210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39576/0 (socket says 172.21.15.36:39576) 2022-06-17T12:56:29.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f85226ff700 1 -- 172.21.15.36:0/1649424591 learned_addr learned my addr 172.21.15.36:0/1649424591 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:29.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f8521efe700 1 --2- 172.21.15.36:0/1649424591 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f851c0ba3c0 0x7f851c0b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:29.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f85226ff700 1 -- 172.21.15.36:0/1649424591 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f851c0ba3c0 msgr2=0x7f851c0b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:29.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f85226ff700 1 --2- 172.21.15.36:0/1649424591 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f851c0ba3c0 0x7f851c0b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f85226ff700 1 -- 172.21.15.36:0/1649424591 >> v1:172.21.15.70:6789/0 conn(0x7f851c0b4310 legacy=0x7f851c0b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:29.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f85226ff700 1 -- 172.21.15.36:0/1649424591 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c0b5370 con 0x7f851c0b3dc0 2022-06-17T12:56:29.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f85226ff700 1 --2- 172.21.15.36:0/1649424591 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f851c0b3dc0 0x7f851c142210 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 rx=0x7f85140032f0 tx=0x7f8514009290).ready entity=mon.0 client_cookie=d31dabe7358e9076 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:29.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f8520efc700 1 -- 172.21.15.36:0/1649424591 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f851400b8a0 con 0x7f851c0b3dc0 2022-06-17T12:56:29.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f8520efc700 1 -- 172.21.15.36:0/1649424591 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851400ba00 con 0x7f851c0b3dc0 2022-06-17T12:56:29.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f8520efc700 1 -- 172.21.15.36:0/1649424591 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f851400bb70 con 0x7f851c0b3dc0 2022-06-17T12:56:29.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f8524162700 1 -- 172.21.15.36:0/1649424591 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f851c0b3dc0 msgr2=0x7f851c142210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:29.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f8524162700 1 --2- 172.21.15.36:0/1649424591 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f851c0b3dc0 0x7f851c142210 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 rx=0x7f85140032f0 tx=0x7f8514009290).stop 2022-06-17T12:56:29.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f8524162700 1 -- 172.21.15.36:0/1649424591 shutdown_connections 2022-06-17T12:56:29.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f8524162700 1 --2- 172.21.15.36:0/1649424591 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f851c0b3dc0 0x7f851c142210 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f8524162700 1 --2- 172.21.15.36:0/1649424591 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f851c0ba3c0 0x7f851c0b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f8524162700 1 -- 172.21.15.36:0/1649424591 >> 172.21.15.36:0/1649424591 conn(0x7f851c067650 msgr2=0x7f851c0b9970 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:29.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f8524162700 1 -- 172.21.15.36:0/1649424591 shutdown_connections 2022-06-17T12:56:29.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f8524162700 1 -- 172.21.15.36:0/1649424591 wait complete. 2022-06-17T12:56:29.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f8524162700 1 Processor -- start 2022-06-17T12:56:29.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.294+0000 7f8524162700 1 -- start start 2022-06-17T12:56:29.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.298+0000 7f8524162700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f851c0b3dc0 0x7f851c147c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:29.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.298+0000 7f8524162700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f851c0b4310 0x7f851c0aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:29.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.298+0000 7f8524162700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f851c0ba3c0 0x7f851c0af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:29.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.298+0000 7f8524162700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c0b8300 con 0x7f851c0b4310 2022-06-17T12:56:29.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.298+0000 7f8524162700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c0b85a0 con 0x7f851c0b3dc0 2022-06-17T12:56:29.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.298+0000 7f8524162700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851c0b8840 con 0x7f851c0ba3c0 2022-06-17T12:56:29.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.298+0000 7f85216fd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f851c0b4310 0x7f851c0aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:29.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.298+0000 7f85216fd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f851c0b4310 0x7f851c0aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39582/0 (socket says 172.21.15.36:39582) 2022-06-17T12:56:29.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.298+0000 7f85216fd700 1 -- 172.21.15.36:0/1494856322 learned_addr learned my addr 172.21.15.36:0/1494856322 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:29.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.298+0000 7f85226ff700 1 --2- 172.21.15.36:0/1494856322 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f851c0ba3c0 0x7f851c0af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:29.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.298+0000 7f8521efe700 1 --2- 172.21.15.36:0/1494856322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f851c0b3dc0 0x7f851c147c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:29.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.298+0000 7f85216fd700 1 -- 172.21.15.36:0/1494856322 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f851c0ba3c0 msgr2=0x7f851c0af250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:29.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.298+0000 7f85216fd700 1 --2- 172.21.15.36:0/1494856322 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f851c0ba3c0 0x7f851c0af250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.298+0000 7f85216fd700 1 -- 172.21.15.36:0/1494856322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f851c0b3dc0 msgr2=0x7f851c147c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:29.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.298+0000 7f85216fd700 1 --2- 172.21.15.36:0/1494856322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f851c0b3dc0 0x7f851c147c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.298+0000 7f85216fd700 1 -- 172.21.15.36:0/1494856322 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8514009530 con 0x7f851c0b4310 2022-06-17T12:56:29.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.298+0000 7f85216fd700 1 --2- 172.21.15.36:0/1494856322 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f851c0b4310 0x7f851c0aed60 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7f851c062f40 tx=0x7f8510007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:29.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.298+0000 7f850effd700 1 -- 172.21.15.36:0/1494856322 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8510007030 con 0x7f851c0b4310 2022-06-17T12:56:29.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.298+0000 7f850effd700 1 -- 172.21.15.36:0/1494856322 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8510007190 con 0x7f851c0b4310 2022-06-17T12:56:29.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.298+0000 7f8524162700 1 -- 172.21.15.36:0/1494856322 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851c0b7c90 con 0x7f851c0b4310 2022-06-17T12:56:29.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.298+0000 7f850effd700 1 -- 172.21.15.36:0/1494856322 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8510006510 con 0x7f851c0b4310 2022-06-17T12:56:29.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.298+0000 7f8524162700 1 -- 172.21.15.36:0/1494856322 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f851c0b91a0 con 0x7f851c0b4310 2022-06-17T12:56:29.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.302+0000 7f8524162700 1 -- 172.21.15.36:0/1494856322 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f851c0b8a00 con 0x7f851c0b4310 2022-06-17T12:56:29.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.302+0000 7f850effd700 1 -- 172.21.15.36:0/1494856322 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8510010030 con 0x7f851c0b4310 2022-06-17T12:56:29.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.302+0000 7f850effd700 1 --2- 172.21.15.36:0/1494856322 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f84fc053a60 0x7f84fc055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:29.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.302+0000 7f850effd700 1 -- 172.21.15.36:0/1494856322 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f851006bc40 con 0x7f851c0b4310 2022-06-17T12:56:29.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.302+0000 7f8521efe700 1 --2- 172.21.15.36:0/1494856322 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f84fc053a60 0x7f84fc055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:29.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.302+0000 7f8521efe700 1 --2- 172.21.15.36:0/1494856322 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f84fc053a60 0x7f84fc055f20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f851800c000 tx=0x7f8518008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:29.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.306+0000 7f850effd700 1 -- 172.21.15.36:0/1494856322 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f85100429e0 con 0x7f851c0b4310 2022-06-17T12:56:29.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.430+0000 7f8524162700 1 -- 172.21.15.36:0/1494856322 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f851c057410 con 0x7f851c0b4310 2022-06-17T12:56:29.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.434+0000 7f850effd700 1 -- 172.21.15.36:0/1494856322 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v131) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8510046000 con 0x7f851c0b4310 2022-06-17T12:56:29.486 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:56:29.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.434+0000 7f8524162700 1 -- 172.21.15.36:0/1494856322 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f84fc053a60 msgr2=0x7f84fc055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:29.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.434+0000 7f8524162700 1 --2- 172.21.15.36:0/1494856322 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f84fc053a60 0x7f84fc055f20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f851800c000 tx=0x7f8518008040).stop 2022-06-17T12:56:29.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.434+0000 7f8524162700 1 -- 172.21.15.36:0/1494856322 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f851c0b4310 msgr2=0x7f851c0aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:29.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.434+0000 7f8524162700 1 --2- 172.21.15.36:0/1494856322 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f851c0b4310 0x7f851c0aed60 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7f851c062f40 tx=0x7f8510007be0).stop 2022-06-17T12:56:29.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.434+0000 7f8524162700 1 -- 172.21.15.36:0/1494856322 shutdown_connections 2022-06-17T12:56:29.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.434+0000 7f8524162700 1 --2- 172.21.15.36:0/1494856322 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f84fc053a60 0x7f84fc055f20 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.434+0000 7f8524162700 1 --2- 172.21.15.36:0/1494856322 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f851c0ba3c0 0x7f851c0af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.434+0000 7f8524162700 1 --2- 172.21.15.36:0/1494856322 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f851c0b4310 0x7f851c0aed60 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.434+0000 7f8524162700 1 --2- 172.21.15.36:0/1494856322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f851c0b3dc0 0x7f851c147c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.434+0000 7f8524162700 1 -- 172.21.15.36:0/1494856322 >> 172.21.15.36:0/1494856322 conn(0x7f851c067650 msgr2=0x7f851c0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:29.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.434+0000 7f8524162700 1 -- 172.21.15.36:0/1494856322 shutdown_connections 2022-06-17T12:56:29.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.434+0000 7f8524162700 1 -- 172.21.15.36:0/1494856322 wait complete. 2022-06-17T12:56:29.512 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:56:29.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f2048c32700 1 Processor -- start 2022-06-17T12:56:29.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f2048c32700 1 -- start start 2022-06-17T12:56:29.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f2048c32700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f20440ba3c0 0x7f20440b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:29.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f2048c32700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f20440b4310 0x7f20440b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:29.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f2048c32700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20440ba7a0 con 0x7f20440b3ba0 2022-06-17T12:56:29.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f2048c32700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20440b5610 con 0x7f20440b4310 2022-06-17T12:56:29.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f2048c32700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20440b50d0 con 0x7f20440ba3c0 2022-06-17T12:56:29.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f2041d9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f20440b4310 0x7f20440b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:29.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f2041d9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f20440b4310 0x7f20440b36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39588/0 (socket says 172.21.15.36:39588) 2022-06-17T12:56:29.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f2041d9b700 1 -- 172.21.15.36:0/1575260 learned_addr learned my addr 172.21.15.36:0/1575260 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:29.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f204259c700 1 --2- 172.21.15.36:0/1575260 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f20440ba3c0 0x7f20440b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:29.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f2041d9b700 1 -- 172.21.15.36:0/1575260 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f20440ba3c0 msgr2=0x7f20440b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:29.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f2041d9b700 1 --2- 172.21.15.36:0/1575260 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f20440ba3c0 0x7f20440b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f2041d9b700 1 -- 172.21.15.36:0/1575260 >> v1:172.21.15.70:6789/0 conn(0x7f20440b3ba0 legacy=0x7f2044146860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:29.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f2041d9b700 1 -- 172.21.15.36:0/1575260 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20440b5370 con 0x7f20440b4310 2022-06-17T12:56:29.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f2041d9b700 1 --2- 172.21.15.36:0/1575260 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f20440b4310 0x7f20440b36b0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7f202c002f70 tx=0x7f202c006eb0).ready entity=mon.0 client_cookie=2e72987811262da1 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:29.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f204159a700 1 -- 172.21.15.36:0/1575260 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f202c00d5c0 con 0x7f20440b4310 2022-06-17T12:56:29.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f204159a700 1 -- 172.21.15.36:0/1575260 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c000e90 con 0x7f20440b4310 2022-06-17T12:56:29.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f204159a700 1 -- 172.21.15.36:0/1575260 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f202c00d950 con 0x7f20440b4310 2022-06-17T12:56:29.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f2048c32700 1 -- 172.21.15.36:0/1575260 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f20440b4310 msgr2=0x7f20440b36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:29.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f2048c32700 1 --2- 172.21.15.36:0/1575260 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f20440b4310 0x7f20440b36b0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7f202c002f70 tx=0x7f202c006eb0).stop 2022-06-17T12:56:29.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f2048c32700 1 -- 172.21.15.36:0/1575260 shutdown_connections 2022-06-17T12:56:29.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f2048c32700 1 --2- 172.21.15.36:0/1575260 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f20440b4310 0x7f20440b36b0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f2048c32700 1 --2- 172.21.15.36:0/1575260 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f20440ba3c0 0x7f20440b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.638+0000 7f2048c32700 1 -- 172.21.15.36:0/1575260 >> 172.21.15.36:0/1575260 conn(0x7f2044067650 msgr2=0x7f20440b9970 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:29.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.642+0000 7f2048c32700 1 -- 172.21.15.36:0/1575260 shutdown_connections 2022-06-17T12:56:29.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.642+0000 7f2048c32700 1 -- 172.21.15.36:0/1575260 wait complete. 2022-06-17T12:56:29.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.642+0000 7f2048c32700 1 Processor -- start 2022-06-17T12:56:29.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.642+0000 7f2048c32700 1 -- start start 2022-06-17T12:56:29.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.642+0000 7f2048c32700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20440b3ba0 0x7f20440b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:29.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.642+0000 7f2048c32700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f20440b4310 0x7f2044148f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:29.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.642+0000 7f2048c32700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f20440ba3c0 0x7f20440ad470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:29.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.642+0000 7f2048c32700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20440addf0 con 0x7f20440b4310 2022-06-17T12:56:29.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.642+0000 7f2048c32700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2044149430 con 0x7f20440b3ba0 2022-06-17T12:56:29.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.642+0000 7f2048c32700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2044148800 con 0x7f20440ba3c0 2022-06-17T12:56:29.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.642+0000 7f2041d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f20440b4310 0x7f2044148f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:29.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.642+0000 7f2041d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f20440b4310 0x7f2044148f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39596/0 (socket says 172.21.15.36:39596) 2022-06-17T12:56:29.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.642+0000 7f2041d9b700 1 -- 172.21.15.36:0/609429512 learned_addr learned my addr 172.21.15.36:0/609429512 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:29.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.642+0000 7f204259c700 1 --2- 172.21.15.36:0/609429512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20440b3ba0 0x7f20440b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:29.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.642+0000 7f2042d9d700 1 --2- 172.21.15.36:0/609429512 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f20440ba3c0 0x7f20440ad470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:29.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.642+0000 7f2041d9b700 1 -- 172.21.15.36:0/609429512 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f20440ba3c0 msgr2=0x7f20440ad470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:29.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.642+0000 7f2041d9b700 1 --2- 172.21.15.36:0/609429512 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f20440ba3c0 0x7f20440ad470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.642+0000 7f2041d9b700 1 -- 172.21.15.36:0/609429512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20440b3ba0 msgr2=0x7f20440b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:29.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.642+0000 7f2041d9b700 1 --2- 172.21.15.36:0/609429512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20440b3ba0 0x7f20440b80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.642+0000 7f2041d9b700 1 -- 172.21.15.36:0/609429512 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c00b1e0 con 0x7f20440b4310 2022-06-17T12:56:29.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.642+0000 7f204259c700 1 --2- 172.21.15.36:0/609429512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20440b3ba0 0x7f20440b80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:56:29.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.646+0000 7f2041d9b700 1 --2- 172.21.15.36:0/609429512 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f20440b4310 0x7f2044148f40 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0x7f202c002f70 tx=0x7f202c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:29.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.646+0000 7f203b7fe700 1 -- 172.21.15.36:0/609429512 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f202c011950 con 0x7f20440b4310 2022-06-17T12:56:29.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.646+0000 7f203b7fe700 1 -- 172.21.15.36:0/609429512 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c011ab0 con 0x7f20440b4310 2022-06-17T12:56:29.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.646+0000 7f2048c32700 1 -- 172.21.15.36:0/609429512 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2044148aa0 con 0x7f20440b4310 2022-06-17T12:56:29.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.646+0000 7f203b7fe700 1 -- 172.21.15.36:0/609429512 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f202c011e10 con 0x7f20440b4310 2022-06-17T12:56:29.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.646+0000 7f2048c32700 1 -- 172.21.15.36:0/609429512 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20440ad960 con 0x7f20440b4310 2022-06-17T12:56:29.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.646+0000 7f203b7fe700 1 -- 172.21.15.36:0/609429512 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f202c025030 con 0x7f20440b4310 2022-06-17T12:56:29.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.646+0000 7f2048c32700 1 -- 172.21.15.36:0/609429512 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20440adc10 con 0x7f20440b4310 2022-06-17T12:56:29.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.646+0000 7f203b7fe700 1 --2- 172.21.15.36:0/609429512 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2028053ad0 0x7f2028055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:29.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.646+0000 7f203b7fe700 1 -- 172.21.15.36:0/609429512 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f202c06d290 con 0x7f20440b4310 2022-06-17T12:56:29.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.646+0000 7f204259c700 1 --2- 172.21.15.36:0/609429512 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2028053ad0 0x7f2028055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:29.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.646+0000 7f204259c700 1 --2- 172.21.15.36:0/609429512 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2028053ad0 0x7f2028055f90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f203400c000 tx=0x7f2034008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:29.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.654+0000 7f203b7fe700 1 -- 172.21.15.36:0/609429512 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f202c044090 con 0x7f20440b4310 2022-06-17T12:56:29.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.782+0000 7f2048c32700 1 -- 172.21.15.36:0/609429512 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2044148c30 con 0x7f20440b4310 2022-06-17T12:56:29.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.782+0000 7f203b7fe700 1 -- 172.21.15.36:0/609429512 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v131) v1 ==== 104+0+10 (secure 0 0 0) 0x7f202c023020 con 0x7f20440b4310 2022-06-17T12:56:29.772 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:56:29.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.786+0000 7f2048c32700 1 -- 172.21.15.36:0/609429512 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2028053ad0 msgr2=0x7f2028055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:29.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.786+0000 7f2048c32700 1 --2- 172.21.15.36:0/609429512 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2028053ad0 0x7f2028055f90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f203400c000 tx=0x7f2034008040).stop 2022-06-17T12:56:29.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.786+0000 7f2048c32700 1 -- 172.21.15.36:0/609429512 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f20440b4310 msgr2=0x7f2044148f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:29.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.786+0000 7f2048c32700 1 --2- 172.21.15.36:0/609429512 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f20440b4310 0x7f2044148f40 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0x7f202c002f70 tx=0x7f202c015000).stop 2022-06-17T12:56:29.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.786+0000 7f2048c32700 1 -- 172.21.15.36:0/609429512 shutdown_connections 2022-06-17T12:56:29.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.786+0000 7f2048c32700 1 --2- 172.21.15.36:0/609429512 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2028053ad0 0x7f2028055f90 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.786+0000 7f2048c32700 1 --2- 172.21.15.36:0/609429512 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f20440ba3c0 0x7f20440ad470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.786+0000 7f2048c32700 1 --2- 172.21.15.36:0/609429512 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f20440b4310 0x7f2044148f40 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.786+0000 7f2048c32700 1 --2- 172.21.15.36:0/609429512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20440b3ba0 0x7f20440b80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.786+0000 7f2048c32700 1 -- 172.21.15.36:0/609429512 >> 172.21.15.36:0/609429512 conn(0x7f2044067650 msgr2=0x7f2044143380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:29.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.786+0000 7f2048c32700 1 -- 172.21.15.36:0/609429512 shutdown_connections 2022-06-17T12:56:29.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.786+0000 7f2048c32700 1 -- 172.21.15.36:0/609429512 wait complete. 2022-06-17T12:56:29.808 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:56:29.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4dbc954700 1 Processor -- start 2022-06-17T12:56:29.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4dbc954700 1 -- start start 2022-06-17T12:56:29.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4dbc954700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4db80b54c0 0x7f4db80b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:29.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4dbc954700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4db8147400 0x7f4db8141220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:29.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4dbc954700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4db8141710 0x7f4db814afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:29.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4dbc954700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db80b2500 con 0x7f4db8147400 2022-06-17T12:56:29.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4dbc954700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db81483d0 con 0x7f4db8141710 2022-06-17T12:56:29.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4dbc954700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db80ac460 con 0x7f4db80b54c0 2022-06-17T12:56:29.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4daeffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4db8147400 0x7f4db8141220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:29.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4daeffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4db8147400 0x7f4db8141220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39602/0 (socket says 172.21.15.36:39602) 2022-06-17T12:56:29.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4daeffd700 1 -- 172.21.15.36:0/1514700323 learned_addr learned my addr 172.21.15.36:0/1514700323 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:29.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4db7fff700 1 --2- 172.21.15.36:0/1514700323 >> v2:172.21.15.70:3300/0 conn(0x7f4db8141710 0x7f4db814afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:29.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4db77fe700 1 --2- 172.21.15.36:0/1514700323 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4db80b54c0 0x7f4db80b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:29.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4daeffd700 1 -- 172.21.15.36:0/1514700323 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4db80b54c0 msgr2=0x7f4db80b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:29.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4daeffd700 1 --2- 172.21.15.36:0/1514700323 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4db80b54c0 0x7f4db80b1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4daeffd700 1 -- 172.21.15.36:0/1514700323 >> v2:172.21.15.70:3300/0 conn(0x7f4db8141710 msgr2=0x7f4db814afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:29.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4daeffd700 1 --2- 172.21.15.36:0/1514700323 >> v2:172.21.15.70:3300/0 conn(0x7f4db8141710 0x7f4db814afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4daeffd700 1 -- 172.21.15.36:0/1514700323 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db80ac1c0 con 0x7f4db8147400 2022-06-17T12:56:29.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4daeffd700 1 --2- 172.21.15.36:0/1514700323 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4db8147400 0x7f4db8141220 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7f4db8148510 tx=0x7f4da0006eb0).ready entity=mon.0 client_cookie=26726d432afd891a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:29.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4db6ffd700 1 -- 172.21.15.36:0/1514700323 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4da000e610 con 0x7f4db8147400 2022-06-17T12:56:29.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4db6ffd700 1 -- 172.21.15.36:0/1514700323 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da0000e90 con 0x7f4db8147400 2022-06-17T12:56:29.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4db6ffd700 1 -- 172.21.15.36:0/1514700323 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4da000e9a0 con 0x7f4db8147400 2022-06-17T12:56:29.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4dbc954700 1 -- 172.21.15.36:0/1514700323 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4db8147400 msgr2=0x7f4db8141220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:29.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4dbc954700 1 --2- 172.21.15.36:0/1514700323 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4db8147400 0x7f4db8141220 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7f4db8148510 tx=0x7f4da0006eb0).stop 2022-06-17T12:56:29.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4dbc954700 1 -- 172.21.15.36:0/1514700323 shutdown_connections 2022-06-17T12:56:29.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4dbc954700 1 --2- 172.21.15.36:0/1514700323 >> v2:172.21.15.70:3300/0 conn(0x7f4db8141710 0x7f4db814afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4dbc954700 1 --2- 172.21.15.36:0/1514700323 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4db8147400 0x7f4db8141220 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4dbc954700 1 --2- 172.21.15.36:0/1514700323 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4db80b54c0 0x7f4db80b1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:29.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4dbc954700 1 -- 172.21.15.36:0/1514700323 >> 172.21.15.36:0/1514700323 conn(0x7f4db805be50 msgr2=0x7f4db805c230 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:29.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4dbc954700 1 -- 172.21.15.36:0/1514700323 shutdown_connections 2022-06-17T12:56:29.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.950+0000 7f4dbc954700 1 -- 172.21.15.36:0/1514700323 wait complete. 2022-06-17T12:56:29.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4dbc954700 1 Processor -- start 2022-06-17T12:56:29.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4dbc954700 1 -- start start 2022-06-17T12:56:29.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4dbc954700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4db80b54c0 0x7f4db814a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:29.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4dbc954700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4db8141710 0x7f4db8142e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:29.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4dbc954700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4db8147400 0x7f4db8156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:29.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4dbc954700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db814d570 con 0x7f4db8141710 2022-06-17T12:56:29.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4dbc954700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db80acb80 con 0x7f4db8147400 2022-06-17T12:56:29.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4dbc954700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db80ac8c0 con 0x7f4db80b54c0 2022-06-17T12:56:29.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4db7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4db8147400 0x7f4db8156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:29.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4daeffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4db8141710 0x7f4db8142e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:29.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4db77fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4db80b54c0 0x7f4db814a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:29.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4db7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4db8147400 0x7f4db8156cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:59990/0 (socket says 172.21.15.36:59990) 2022-06-17T12:56:29.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4db7fff700 1 -- 172.21.15.36:0/2170808305 learned_addr learned my addr 172.21.15.36:0/2170808305 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:29.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4db77fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4db80b54c0 0x7f4db814a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:54400/0 (socket says 172.21.15.36:54400) 2022-06-17T12:56:29.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4daeffd700 1 -- 172.21.15.36:0/2170808305 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4db80b54c0 msgr2=0x7f4db814a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:30.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4daeffd700 1 --2- 172.21.15.36:0/2170808305 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4db80b54c0 0x7f4db814a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:30.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4daeffd700 1 -- 172.21.15.36:0/2170808305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4db8147400 msgr2=0x7f4db8156cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:30.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4daeffd700 1 --2- 172.21.15.36:0/2170808305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4db8147400 0x7f4db8156cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:30.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4daeffd700 1 -- 172.21.15.36:0/2170808305 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da000b1e0 con 0x7f4db8141710 2022-06-17T12:56:30.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4db7fff700 1 --2- 172.21.15.36:0/2170808305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4db8147400 0x7f4db8156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:30.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4daeffd700 1 --2- 172.21.15.36:0/2170808305 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4db8141710 0x7f4db8142e20 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 rx=0x7f4db8148510 tx=0x7f4da0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:30.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4db57fa700 1 -- 172.21.15.36:0/2170808305 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4da0021070 con 0x7f4db8141710 2022-06-17T12:56:30.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4db57fa700 1 -- 172.21.15.36:0/2170808305 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da0002f70 con 0x7f4db8141710 2022-06-17T12:56:30.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4dbc954700 1 -- 172.21.15.36:0/2170808305 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db8156790 con 0x7f4db8141710 2022-06-17T12:56:30.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4db57fa700 1 -- 172.21.15.36:0/2170808305 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4da00068b0 con 0x7f4db8141710 2022-06-17T12:56:30.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.954+0000 7f4dbc954700 1 -- 172.21.15.36:0/2170808305 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db8144a70 con 0x7f4db8141710 2022-06-17T12:56:30.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.958+0000 7f4dbc954700 1 -- 172.21.15.36:0/2170808305 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4db8143310 con 0x7f4db8141710 2022-06-17T12:56:30.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.958+0000 7f4db57fa700 1 -- 172.21.15.36:0/2170808305 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f4da000f070 con 0x7f4db8141710 2022-06-17T12:56:30.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.958+0000 7f4db57fa700 1 --2- 172.21.15.36:0/2170808305 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d9c053a40 0x7f4d9c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:30.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.958+0000 7f4db57fa700 1 -- 172.21.15.36:0/2170808305 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4da006cd00 con 0x7f4db8141710 2022-06-17T12:56:30.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.958+0000 7f4db77fe700 1 --2- 172.21.15.36:0/2170808305 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d9c053a40 0x7f4d9c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:30.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.958+0000 7f4db77fe700 1 --2- 172.21.15.36:0/2170808305 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d9c053a40 0x7f4d9c055f00 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f4da800b000 tx=0x7f4da8006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:30.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:29.966+0000 7f4db57fa700 1 -- 172.21.15.36:0/2170808305 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4da0070030 con 0x7f4db8141710 2022-06-17T12:56:30.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.098+0000 7f4dbc954700 1 -- 172.21.15.36:0/2170808305 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4db80acf00 con 0x7f4db8141710 2022-06-17T12:56:30.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.098+0000 7f4db57fa700 1 -- 172.21.15.36:0/2170808305 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v131) v1 ==== 100+0+10 (secure 0 0 0) 0x7f4da0043b00 con 0x7f4db8141710 2022-06-17T12:56:30.089 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:56:30.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.098+0000 7f4dbc954700 1 -- 172.21.15.36:0/2170808305 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d9c053a40 msgr2=0x7f4d9c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:30.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.098+0000 7f4dbc954700 1 --2- 172.21.15.36:0/2170808305 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d9c053a40 0x7f4d9c055f00 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f4da800b000 tx=0x7f4da8006df0).stop 2022-06-17T12:56:30.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.098+0000 7f4dbc954700 1 -- 172.21.15.36:0/2170808305 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4db8141710 msgr2=0x7f4db8142e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:30.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.098+0000 7f4dbc954700 1 --2- 172.21.15.36:0/2170808305 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4db8141710 0x7f4db8142e20 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 rx=0x7f4db8148510 tx=0x7f4da0015000).stop 2022-06-17T12:56:30.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.098+0000 7f4dbc954700 1 -- 172.21.15.36:0/2170808305 shutdown_connections 2022-06-17T12:56:30.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.098+0000 7f4dbc954700 1 --2- 172.21.15.36:0/2170808305 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4d9c053a40 0x7f4d9c055f00 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:30.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.098+0000 7f4dbc954700 1 --2- 172.21.15.36:0/2170808305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4db8147400 0x7f4db8156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:30.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.098+0000 7f4dbc954700 1 --2- 172.21.15.36:0/2170808305 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4db8141710 0x7f4db8142e20 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:30.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.098+0000 7f4dbc954700 1 --2- 172.21.15.36:0/2170808305 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4db80b54c0 0x7f4db814a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:30.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.098+0000 7f4dbc954700 1 -- 172.21.15.36:0/2170808305 >> 172.21.15.36:0/2170808305 conn(0x7f4db805be50 msgr2=0x7f4db8148950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:30.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.098+0000 7f4dbc954700 1 -- 172.21.15.36:0/2170808305 shutdown_connections 2022-06-17T12:56:30.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.098+0000 7f4dbc954700 1 -- 172.21.15.36:0/2170808305 wait complete. 2022-06-17T12:56:30.104 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:56:30.104 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:56:30.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.254+0000 7f18ed66e700 1 Processor -- start 2022-06-17T12:56:30.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.254+0000 7f18ed66e700 1 -- start start 2022-06-17T12:56:30.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.254+0000 7f18ed66e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18e80677d0 0x7f18e80b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:30.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.254+0000 7f18ed66e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f18e80acda0 0x7f18e80ad210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:30.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.254+0000 7f18ed66e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18e80afe40 con 0x7f18e80b1220 2022-06-17T12:56:30.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.258+0000 7f18ed66e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18e80ae000 con 0x7f18e80acda0 2022-06-17T12:56:30.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.258+0000 7f18ed66e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e813f2f0 con 0x7f18e80677d0 2022-06-17T12:56:30.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.258+0000 7f18ece6d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f18e80acda0 0x7f18e80ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:30.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.258+0000 7f18e7fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18e80677d0 0x7f18e80b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:30.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.258+0000 7f18dffff700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f18e80b1220 0x7f18e8060820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:52880/0 (socket says 172.21.15.36:52880) 2022-06-17T12:56:30.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.258+0000 7f18dffff700 1 -- 172.21.15.36:0/218303718 learned_addr learned my addr 172.21.15.36:0/218303718 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:30.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.258+0000 7f18ece6d700 1 -- 172.21.15.36:0/218303718 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18e80677d0 msgr2=0x7f18e80b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:30.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.258+0000 7f18ece6d700 1 --2- 172.21.15.36:0/218303718 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18e80677d0 0x7f18e80b2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:30.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.258+0000 7f18ece6d700 1 -- 172.21.15.36:0/218303718 >> v1:172.21.15.70:6789/0 conn(0x7f18e80b1220 legacy=0x7f18e8060820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:30.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.258+0000 7f18e77fe700 1 -- 172.21.15.36:0/218303718 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538941895 0 0) 0x7f18e80afe40 con 0x7f18e80b1220 2022-06-17T12:56:30.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.258+0000 7f18ece6d700 1 -- 172.21.15.36:0/218303718 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e8067bb0 con 0x7f18e80acda0 2022-06-17T12:56:30.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.258+0000 7f18ece6d700 1 --2- 172.21.15.36:0/218303718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18e80acda0 0x7f18e80ad210 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f18e00032f0 tx=0x7f18e000b4b0).ready entity=mon.1 client_cookie=34f0ad14f62480ac server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:30.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.258+0000 7f18e77fe700 1 -- 172.21.15.36:0/218303718 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18e000de70 con 0x7f18e80acda0 2022-06-17T12:56:30.279 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.258+0000 7f18e77fe700 1 -- 172.21.15.36:0/218303718 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e00153f0 con 0x7f18e80acda0 2022-06-17T12:56:30.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.258+0000 7f18e77fe700 1 -- 172.21.15.36:0/218303718 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18e0015750 con 0x7f18e80acda0 2022-06-17T12:56:30.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.258+0000 7f18ed66e700 1 -- 172.21.15.36:0/218303718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18e80acda0 msgr2=0x7f18e80ad210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:30.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.258+0000 7f18ed66e700 1 --2- 172.21.15.36:0/218303718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18e80acda0 0x7f18e80ad210 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f18e00032f0 tx=0x7f18e000b4b0).stop 2022-06-17T12:56:30.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.258+0000 7f18ed66e700 1 -- 172.21.15.36:0/218303718 shutdown_connections 2022-06-17T12:56:30.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.258+0000 7f18ed66e700 1 --2- 172.21.15.36:0/218303718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18e80acda0 0x7f18e80ad210 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:30.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.258+0000 7f18ed66e700 1 --2- 172.21.15.36:0/218303718 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18e80677d0 0x7f18e80b2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:30.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.258+0000 7f18ed66e700 1 -- 172.21.15.36:0/218303718 >> 172.21.15.36:0/218303718 conn(0x7f18e805be50 msgr2=0x7f18e805c230 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:30.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.258+0000 7f18ed66e700 1 -- 172.21.15.36:0/218303718 shutdown_connections 2022-06-17T12:56:30.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.258+0000 7f18ed66e700 1 -- 172.21.15.36:0/218303718 wait complete. 2022-06-17T12:56:30.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.262+0000 7f18ed66e700 1 Processor -- start 2022-06-17T12:56:30.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.262+0000 7f18ed66e700 1 -- start start 2022-06-17T12:56:30.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.262+0000 7f18ed66e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18e80677d0 0x7f18e80646f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:30.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.262+0000 7f18ed66e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18e80acda0 0x7f18e8066600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:30.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.262+0000 7f18ed66e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18e80b1220 0x7f18e8066af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:30.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.262+0000 7f18ed66e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e813ce20 con 0x7f18e80acda0 2022-06-17T12:56:30.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.262+0000 7f18ed66e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e8064be0 con 0x7f18e80b1220 2022-06-17T12:56:30.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.262+0000 7f18ed66e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e813ac40 con 0x7f18e80677d0 2022-06-17T12:56:30.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.262+0000 7f18dffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18e80acda0 0x7f18e8066600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:30.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.262+0000 7f18dffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18e80acda0 0x7f18e8066600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39622/0 (socket says 172.21.15.36:39622) 2022-06-17T12:56:30.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.262+0000 7f18e7fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18e80677d0 0x7f18e80646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:30.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.262+0000 7f18dffff700 1 -- 172.21.15.36:0/1781748190 learned_addr learned my addr 172.21.15.36:0/1781748190 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:30.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.262+0000 7f18ece6d700 1 --2- 172.21.15.36:0/1781748190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18e80b1220 0x7f18e8066af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:30.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.262+0000 7f18dffff700 1 -- 172.21.15.36:0/1781748190 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18e80677d0 msgr2=0x7f18e80646f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:30.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.262+0000 7f18dffff700 1 --2- 172.21.15.36:0/1781748190 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18e80677d0 0x7f18e80646f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:30.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.262+0000 7f18dffff700 1 -- 172.21.15.36:0/1781748190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18e80b1220 msgr2=0x7f18e8066af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:30.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.262+0000 7f18dffff700 1 --2- 172.21.15.36:0/1781748190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18e80b1220 0x7f18e8066af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:30.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.262+0000 7f18dffff700 1 -- 172.21.15.36:0/1781748190 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e000b710 con 0x7f18e80acda0 2022-06-17T12:56:30.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.266+0000 7f18dffff700 1 --2- 172.21.15.36:0/1781748190 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18e80acda0 0x7f18e8066600 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 rx=0x7f18e813af50 tx=0x7f18d0007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:30.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.266+0000 7f18e5ffb700 1 -- 172.21.15.36:0/1781748190 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18d000cd90 con 0x7f18e80acda0 2022-06-17T12:56:30.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.266+0000 7f18e5ffb700 1 -- 172.21.15.36:0/1781748190 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d00070d0 con 0x7f18e80acda0 2022-06-17T12:56:30.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.266+0000 7f18ed66e700 1 -- 172.21.15.36:0/1781748190 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e8064e20 con 0x7f18e80acda0 2022-06-17T12:56:30.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.266+0000 7f18e5ffb700 1 -- 172.21.15.36:0/1781748190 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18d0007430 con 0x7f18e80acda0 2022-06-17T12:56:30.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.266+0000 7f18ed66e700 1 -- 172.21.15.36:0/1781748190 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e813dbd0 con 0x7f18e80acda0 2022-06-17T12:56:30.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.266+0000 7f18ed66e700 1 -- 172.21.15.36:0/1781748190 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e813cf60 con 0x7f18e80acda0 2022-06-17T12:56:30.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.266+0000 7f18e5ffb700 1 -- 172.21.15.36:0/1781748190 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f18d0010070 con 0x7f18e80acda0 2022-06-17T12:56:30.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.266+0000 7f18e5ffb700 1 --2- 172.21.15.36:0/1781748190 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f18cc053940 0x7f18cc055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:30.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.266+0000 7f18e5ffb700 1 -- 172.21.15.36:0/1781748190 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f18d006b590 con 0x7f18e80acda0 2022-06-17T12:56:30.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.266+0000 7f18e7fff700 1 --2- 172.21.15.36:0/1781748190 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f18cc053940 0x7f18cc055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:30.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.266+0000 7f18e7fff700 1 --2- 172.21.15.36:0/1781748190 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f18cc053940 0x7f18cc055e00 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f18d8006000 tx=0x7f18d8006d90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:30.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.274+0000 7f18e5ffb700 1 -- 172.21.15.36:0/1781748190 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f18e813cf60 con 0x7f18e80acda0 2022-06-17T12:56:30.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.438+0000 7f18ed66e700 1 -- 172.21.15.36:0/1781748190 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18e813cf60 con 0x7f18e80acda0 2022-06-17T12:56:30.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.438+0000 7f18e5ffb700 1 -- 172.21.15.36:0/1781748190 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f18e813cf60 con 0x7f18e80acda0 2022-06-17T12:56:30.432 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:30.432 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":72,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/2357210652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2357210652},{"type":"v1","addr":"172.21.15.36:6835","nonce":2357210652}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:56:11.021928+0000","modified":"2022-06-17T12:56:13.034227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7942":{"gid":7942,"name":"b","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/197726269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":197726269},{"type":"v1","addr":"172.21.15.70:6833","nonce":197726269}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-06-17T12:56:30.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.442+0000 7f18ed66e700 1 -- 172.21.15.36:0/1781748190 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f18cc053940 msgr2=0x7f18cc055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:30.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.442+0000 7f18ed66e700 1 --2- 172.21.15.36:0/1781748190 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f18cc053940 0x7f18cc055e00 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f18d8006000 tx=0x7f18d8006d90).stop 2022-06-17T12:56:30.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.442+0000 7f18ed66e700 1 -- 172.21.15.36:0/1781748190 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18e80acda0 msgr2=0x7f18e8066600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:30.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.442+0000 7f18ed66e700 1 --2- 172.21.15.36:0/1781748190 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18e80acda0 0x7f18e8066600 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 rx=0x7f18e813af50 tx=0x7f18d0007c90).stop 2022-06-17T12:56:30.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.442+0000 7f18ed66e700 1 -- 172.21.15.36:0/1781748190 shutdown_connections 2022-06-17T12:56:30.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.442+0000 7f18ed66e700 1 --2- 172.21.15.36:0/1781748190 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f18cc053940 0x7f18cc055e00 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:30.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.442+0000 7f18ed66e700 1 --2- 172.21.15.36:0/1781748190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18e80b1220 0x7f18e8066af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:30.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.442+0000 7f18ed66e700 1 --2- 172.21.15.36:0/1781748190 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18e80acda0 0x7f18e8066600 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:30.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.442+0000 7f18ed66e700 1 --2- 172.21.15.36:0/1781748190 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18e80677d0 0x7f18e80646f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:30.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.442+0000 7f18ed66e700 1 -- 172.21.15.36:0/1781748190 >> 172.21.15.36:0/1781748190 conn(0x7f18e805be50 msgr2=0x7f18e813e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:30.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.442+0000 7f18ed66e700 1 -- 172.21.15.36:0/1781748190 shutdown_connections 2022-06-17T12:56:30.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:30.442+0000 7f18ed66e700 1 -- 172.21.15.36:0/1781748190 wait complete. 2022-06-17T12:56:30.439 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 72 2022-06-17T12:56:30.685 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:56:11.021928+0000', 'modified': '2022-06-17T12:56:13.034227+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7942}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7942': {'gid': 7942, 'name': 'b', 'rank': 0, 'incarnation': 69, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6833/197726269', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 197726269}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 197726269}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:56:30.686 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:56:30.687 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-17T12:56:30.844 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":7942,"want_state":"up:active","state":"up:active","rank_uptime":19.824488508000002,"mdsmap_epoch":72,"osdmap_epoch":131,"osdmap_epoch_barrier":131,"uptime":24.807854116000001} 2022-06-17T12:56:30.852 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 7942, "mdsmap_epoch": 72, "osdmap_epoch": 131, "osdmap_epoch_barrier": 131, "rank_uptime": 19.824488508, "state": "up:active", "uptime": 24.807854116, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:56:30.865 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:56:30.866 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-06-17T12:56:30.884 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi136.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-06-17T12:56:30.885 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:30.915 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:30.979 DEBUG:teuthology.orchestra.run.smithi136:> sudo modprobe fuse 2022-06-17T12:56:31.015 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:56:31.017 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:56:31.085 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:56:31.087 INFO:teuthology.orchestra.run.smithi136.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:56:31.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:31.088 DEBUG:teuthology.orchestra.run.smithi136:> ls /sys/fs/fuse/connections 2022-06-17T12:56:31.117 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:56:31.119 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:31.165 DEBUG:teuthology.orchestra.run.smithi136:> sudo modprobe fuse 2022-06-17T12:56:31.201 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:56:31.202 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:56:31.239 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:2022-06-17T12:56:31.249+0000 7f01fb5af080 -1 init, newargv = 0x5573ff1119a0 newargc=9ceph-fuse[22395]: starting ceph client 2022-06-17T12:56:31.240 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr: 2022-06-17T12:56:31.249 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:ceph-fuse[22395]: starting fuse 2022-06-17T12:56:31.269 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:56:31.270 INFO:teuthology.orchestra.run.smithi136.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:56:31.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:31.272 DEBUG:teuthology.orchestra.run.smithi136:> ls /sys/fs/fuse/connections 2022-06-17T12:56:31.324 INFO:teuthology.orchestra.run.smithi136.stdout:52 2022-06-17T12:56:31.334 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:56:31.335 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:31.336 DEBUG:teuthology.orchestra.run.smithi136:> sudo python3 -c ' 2022-06-17T12:56:31.337 DEBUG:teuthology.orchestra.run.smithi136:> import glob 2022-06-17T12:56:31.337 DEBUG:teuthology.orchestra.run.smithi136:> import re 2022-06-17T12:56:31.339 DEBUG:teuthology.orchestra.run.smithi136:> import os 2022-06-17T12:56:31.341 DEBUG:teuthology.orchestra.run.smithi136:> import subprocess 2022-06-17T12:56:31.341 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:56:31.342 DEBUG:teuthology.orchestra.run.smithi136:> def find_socket(client_name): 2022-06-17T12:56:31.343 DEBUG:teuthology.orchestra.run.smithi136:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-06-17T12:56:31.344 DEBUG:teuthology.orchestra.run.smithi136:> files = glob.glob(asok_path) 2022-06-17T12:56:31.345 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:56:31.345 DEBUG:teuthology.orchestra.run.smithi136:> # Given a non-glob path, it better be there 2022-06-17T12:56:31.347 DEBUG:teuthology.orchestra.run.smithi136:> if "*" not in asok_path: 2022-06-17T12:56:31.348 DEBUG:teuthology.orchestra.run.smithi136:> assert(len(files) == 1) 2022-06-17T12:56:31.348 DEBUG:teuthology.orchestra.run.smithi136:> return files[0] 2022-06-17T12:56:31.349 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:56:31.351 DEBUG:teuthology.orchestra.run.smithi136:> for f in files: 2022-06-17T12:56:31.351 DEBUG:teuthology.orchestra.run.smithi136:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:56:31.352 DEBUG:teuthology.orchestra.run.smithi136:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:56:31.353 DEBUG:teuthology.orchestra.run.smithi136:> return f 2022-06-17T12:56:31.354 DEBUG:teuthology.orchestra.run.smithi136:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:56:31.354 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:56:31.355 DEBUG:teuthology.orchestra.run.smithi136:> print(find_socket("client.1")) 2022-06-17T12:56:31.356 DEBUG:teuthology.orchestra.run.smithi136:> ' 2022-06-17T12:56:31.406 INFO:teuthology.orchestra.run.smithi136.stdout:/var/run/ceph/ceph-client.1.22395.asok 2022-06-17T12:56:31.407 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22395.asok 2022-06-17T12:56:31.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:31.417 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22395.asok status 2022-06-17T12:56:31.534 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2022-06-17T12:56:31.535 INFO:teuthology.orchestra.run.smithi136.stdout: "metadata": { 2022-06-17T12:56:31.536 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:56:31.536 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:56:31.536 INFO:teuthology.orchestra.run.smithi136.stdout: "entity_id": "1", 2022-06-17T12:56:31.545 INFO:teuthology.orchestra.run.smithi136.stdout: "hostname": "smithi136", 2022-06-17T12:56:31.546 INFO:teuthology.orchestra.run.smithi136.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-06-17T12:56:31.546 INFO:teuthology.orchestra.run.smithi136.stdout: "pid": "22395", 2022-06-17T12:56:31.546 INFO:teuthology.orchestra.run.smithi136.stdout: "root": "/" 2022-06-17T12:56:31.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:56:31.547 INFO:teuthology.orchestra.run.smithi136.stdout: "dentry_count": 0, 2022-06-17T12:56:31.548 INFO:teuthology.orchestra.run.smithi136.stdout: "dentry_pinned_count": 0, 2022-06-17T12:56:31.548 INFO:teuthology.orchestra.run.smithi136.stdout: "id": 8370, 2022-06-17T12:56:31.548 INFO:teuthology.orchestra.run.smithi136.stdout: "inst": { 2022-06-17T12:56:31.549 INFO:teuthology.orchestra.run.smithi136.stdout: "name": { 2022-06-17T12:56:31.549 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "client", 2022-06-17T12:56:31.549 INFO:teuthology.orchestra.run.smithi136.stdout: "num": 8370 2022-06-17T12:56:31.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:56:31.552 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": { 2022-06-17T12:56:31.553 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "v1", 2022-06-17T12:56:31.553 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": "172.21.15.136:0", 2022-06-17T12:56:31.553 INFO:teuthology.orchestra.run.smithi136.stdout: "nonce": 3110096750 2022-06-17T12:56:31.554 INFO:teuthology.orchestra.run.smithi136.stdout: } 2022-06-17T12:56:31.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:56:31.554 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": { 2022-06-17T12:56:31.556 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "v1", 2022-06-17T12:56:31.556 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": "172.21.15.136:0", 2022-06-17T12:56:31.557 INFO:teuthology.orchestra.run.smithi136.stdout: "nonce": 3110096750 2022-06-17T12:56:31.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:56:31.561 INFO:teuthology.orchestra.run.smithi136.stdout: "inst_str": "client.8370 172.21.15.136:0/3110096750", 2022-06-17T12:56:31.561 INFO:teuthology.orchestra.run.smithi136.stdout: "addr_str": "172.21.15.136:0/3110096750", 2022-06-17T12:56:31.562 INFO:teuthology.orchestra.run.smithi136.stdout: "inode_count": 1, 2022-06-17T12:56:31.562 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_epoch": 72, 2022-06-17T12:56:31.562 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_epoch": 131, 2022-06-17T12:56:31.563 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:56:31.563 INFO:teuthology.orchestra.run.smithi136.stdout: "blacklisted": false, 2022-06-17T12:56:31.579 INFO:teuthology.orchestra.run.smithi136.stdout: "fs_name": "cephfs" 2022-06-17T12:56:31.580 INFO:teuthology.orchestra.run.smithi136.stdout:} 2022-06-17T12:56:31.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:31.590 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:31.591 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:31.608 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:56:31.609 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:56:31.613 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:31.614 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:31.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:31.669 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:31.670 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:56:31.691 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:56:31.692 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:56:31.693 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-06-17T12:56:31.693 DEBUG:teuthology.orchestra.run.smithi107:> import time 2022-06-17T12:56:31.694 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:56:31.695 DEBUG:teuthology.orchestra.run.smithi107:> with open("/home/ubuntu/cephtest/mnt.0/testfile", '"'"'w'"'"') as f: 2022-06-17T12:56:31.695 DEBUG:teuthology.orchestra.run.smithi107:> f.write('"'"'content'"'"') 2022-06-17T12:56:31.697 DEBUG:teuthology.orchestra.run.smithi107:> f.flush() 2022-06-17T12:56:31.698 DEBUG:teuthology.orchestra.run.smithi107:> f.write('"'"'content2'"'"') 2022-06-17T12:56:31.699 DEBUG:teuthology.orchestra.run.smithi107:> while True: 2022-06-17T12:56:31.699 DEBUG:teuthology.orchestra.run.smithi107:> time.sleep(1) 2022-06-17T12:56:31.700 DEBUG:teuthology.orchestra.run.smithi107:> ' 2022-06-17T12:56:31.751 DEBUG:teuthology.orchestra.run.smithi107:> sudo ls /home/ubuntu/cephtest/mnt.0/testfile 2022-06-17T12:56:31.794 DEBUG:teuthology.orchestra.run:got remote process result: 2 2022-06-17T12:56:31.795 INFO:teuthology.orchestra.run.smithi107.stderr:ls: cannot access '/home/ubuntu/cephtest/mnt.0/testfile': No such file or directory 2022-06-17T12:56:32.801 DEBUG:teuthology.orchestra.run.smithi107:> sudo ls /home/ubuntu/cephtest/mnt.0/testfile 2022-06-17T12:56:32.824 INFO:teuthology.orchestra.run.smithi107.stdout:/home/ubuntu/cephtest/mnt.0/testfile 2022-06-17T12:56:32.824 DEBUG:tasks.cephfs.mount:File testfile became visible from 0 after 1s 2022-06-17T12:56:32.825 DEBUG:teuthology.orchestra.run.smithi136:> sudo ls /home/ubuntu/cephtest/mnt.1/testfile 2022-06-17T12:56:32.880 INFO:teuthology.orchestra.run.smithi136.stdout:/home/ubuntu/cephtest/mnt.1/testfile 2022-06-17T12:56:32.881 DEBUG:tasks.cephfs.mount:File testfile became visible from 1 after 0s 2022-06-17T12:56:32.888 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-06-17T12:56:32.889 DEBUG:teuthology.orchestra.run.smithi136:> cd /home/ubuntu/cephtest/mnt.1 && sudo cat testfile 2022-06-17T12:56:32.952 INFO:teuthology.orchestra.run.smithi136.stdout:content 2022-06-17T12:56:32.977 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-17T12:56:32.978 DEBUG:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:56:33.000 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:56:33.038 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:56:33.062 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:56:33.084 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:56:33.107 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-17T12:56:33.109 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:33.110 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:33.111 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:56:33.125 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:56:33.126 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:56:33.127 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2022-06-17T12:56:33.128 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:56:33.129 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:56:33.146 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[22894]: fuse finished with error 0 and tester_r 0 2022-06-17T12:56:33.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:33.149 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:33.149 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:56:33.202 INFO:teuthology.orchestra.run.smithi107.stdout:ext2/ext3 2022-06-17T12:56:33.203 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:56:33.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:56:33.337 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:56:33.346 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:33.357 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:33.357 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:33.402 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:56:33.402 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:56:33.403 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi136.front.sepia.ceph.com... 2022-06-17T12:56:33.404 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:56:33.405 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:33.459 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:ceph-fuse[22395]: fuse finished with error 0 and tester_r 0 2022-06-17T12:56:33.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:33.472 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:33.474 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:33.482 INFO:teuthology.orchestra.run.smithi136.stdout:ext2/ext3 2022-06-17T12:56:33.485 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:56:33.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:56:33.622 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:33.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:33.681 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:33.682 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:56:33.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:56:33.718 INFO:teuthology.orchestra.run.smithi143.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:56:33.719 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-06-17T12:56:33.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:56:33.721 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:56:33.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:56:33.745 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:56:33.746 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-17T12:56:33.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f46a4700 1 Processor -- start 2022-06-17T12:56:33.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f46a4700 1 -- start start 2022-06-17T12:56:33.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f46a4700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f13ec0b7a00 0x7f13ec0b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:33.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f46a4700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f13ec0adfc0 0x7f13ec060f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:33.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f46a4700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13ec061410 0x7f13ec0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:33.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f46a4700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ec0b36b0 con 0x7f13ec0b7a00 2022-06-17T12:56:33.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f46a4700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ec061ab0 con 0x7f13ec0adfc0 2022-06-17T12:56:33.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f46a4700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec060ac0 con 0x7f13ec061410 2022-06-17T12:56:33.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f2440700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f13ec0b7a00 0x7f13ec0b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:33.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f2440700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f13ec0b7a00 0x7f13ec0b41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39628/0 (socket says 172.21.15.36:39628) 2022-06-17T12:56:33.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f2440700 1 -- 172.21.15.36:0/4155843442 learned_addr learned my addr 172.21.15.36:0/4155843442 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:33.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f1c3f700 1 --2- 172.21.15.36:0/4155843442 >> v2:172.21.15.70:3300/0 conn(0x7f13ec0adfc0 0x7f13ec060f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:33.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f2c41700 1 --2- 172.21.15.36:0/4155843442 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13ec061410 0x7f13ec0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:33.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f2440700 1 -- 172.21.15.36:0/4155843442 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13ec061410 msgr2=0x7f13ec0b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:33.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f2440700 1 --2- 172.21.15.36:0/4155843442 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13ec061410 0x7f13ec0b2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:33.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f2440700 1 -- 172.21.15.36:0/4155843442 >> v2:172.21.15.70:3300/0 conn(0x7f13ec0adfc0 msgr2=0x7f13ec060f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:33.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f2440700 1 --2- 172.21.15.36:0/4155843442 >> v2:172.21.15.70:3300/0 conn(0x7f13ec0adfc0 0x7f13ec060f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:33.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f2440700 1 -- 172.21.15.36:0/4155843442 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec0618c0 con 0x7f13ec0b7a00 2022-06-17T12:56:33.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f2440700 1 --2- 172.21.15.36:0/4155843442 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f13ec0b7a00 0x7f13ec0b41b0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0x7f13e0002f70 tx=0x7f13e0006eb0).ready entity=mon.0 client_cookie=56f65aa4b39a027f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:33.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f143e700 1 -- 172.21.15.36:0/4155843442 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13e000d570 con 0x7f13ec0b7a00 2022-06-17T12:56:33.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f143e700 1 -- 172.21.15.36:0/4155843442 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e0000e90 con 0x7f13ec0b7a00 2022-06-17T12:56:33.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f143e700 1 -- 172.21.15.36:0/4155843442 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13e000d900 con 0x7f13ec0b7a00 2022-06-17T12:56:33.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f46a4700 1 -- 172.21.15.36:0/4155843442 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f13ec0b7a00 msgr2=0x7f13ec0b41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:33.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f46a4700 1 --2- 172.21.15.36:0/4155843442 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f13ec0b7a00 0x7f13ec0b41b0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0x7f13e0002f70 tx=0x7f13e0006eb0).stop 2022-06-17T12:56:33.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f46a4700 1 -- 172.21.15.36:0/4155843442 shutdown_connections 2022-06-17T12:56:33.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f46a4700 1 --2- 172.21.15.36:0/4155843442 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13ec061410 0x7f13ec0b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:33.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f46a4700 1 --2- 172.21.15.36:0/4155843442 >> v2:172.21.15.70:3300/0 conn(0x7f13ec0adfc0 0x7f13ec060f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:33.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f46a4700 1 --2- 172.21.15.36:0/4155843442 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f13ec0b7a00 0x7f13ec0b41b0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:33.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f46a4700 1 -- 172.21.15.36:0/4155843442 >> 172.21.15.36:0/4155843442 conn(0x7f13ec05be50 msgr2=0x7f13ec064790 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:33.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f46a4700 1 -- 172.21.15.36:0/4155843442 shutdown_connections 2022-06-17T12:56:33.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.874+0000 7f13f46a4700 1 -- 172.21.15.36:0/4155843442 wait complete. 2022-06-17T12:56:33.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.878+0000 7f13f46a4700 1 Processor -- start 2022-06-17T12:56:33.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.878+0000 7f13f46a4700 1 -- start start 2022-06-17T12:56:33.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.878+0000 7f13f46a4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f13ec061410 0x7f13ec0aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:33.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.878+0000 7f13f46a4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13ec0adfc0 0x7f13ec149200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:33.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.878+0000 7f13f46a4700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13ec0b7a00 0x7f13ec0b1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:33.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.878+0000 7f13f46a4700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0b0900 con 0x7f13ec061410 2022-06-17T12:56:33.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.878+0000 7f13f46a4700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec156f00 con 0x7f13ec0adfc0 2022-06-17T12:56:33.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.878+0000 7f13f46a4700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0b0b60 con 0x7f13ec0b7a00 2022-06-17T12:56:33.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.878+0000 7f13f2440700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f13ec061410 0x7f13ec0aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:33.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.878+0000 7f13f2440700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f13ec061410 0x7f13ec0aeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39634/0 (socket says 172.21.15.36:39634) 2022-06-17T12:56:33.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.878+0000 7f13f2440700 1 -- 172.21.15.36:0/3975810198 learned_addr learned my addr 172.21.15.36:0/3975810198 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:33.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.878+0000 7f13f2c41700 1 --2- 172.21.15.36:0/3975810198 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13ec0b7a00 0x7f13ec0b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:33.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.878+0000 7f13f1c3f700 1 --2- 172.21.15.36:0/3975810198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13ec0adfc0 0x7f13ec149200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:33.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.878+0000 7f13f2440700 1 -- 172.21.15.36:0/3975810198 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13ec0b7a00 msgr2=0x7f13ec0b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:33.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.878+0000 7f13f2440700 1 --2- 172.21.15.36:0/3975810198 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13ec0b7a00 0x7f13ec0b1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:33.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.878+0000 7f13f2440700 1 -- 172.21.15.36:0/3975810198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13ec0adfc0 msgr2=0x7f13ec149200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:33.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.878+0000 7f13f2440700 1 --2- 172.21.15.36:0/3975810198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13ec0adfc0 0x7f13ec149200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:33.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.878+0000 7f13f2440700 1 -- 172.21.15.36:0/3975810198 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e000b1e0 con 0x7f13ec061410 2022-06-17T12:56:33.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.882+0000 7f13f2440700 1 --2- 172.21.15.36:0/3975810198 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f13ec061410 0x7f13ec0aeaf0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0x7f13e0002f70 tx=0x7f13e0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:33.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.882+0000 7f13eb7fe700 1 -- 172.21.15.36:0/3975810198 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13e0011760 con 0x7f13ec061410 2022-06-17T12:56:33.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.882+0000 7f13eb7fe700 1 -- 172.21.15.36:0/3975810198 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e00118c0 con 0x7f13ec061410 2022-06-17T12:56:33.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.882+0000 7f13f46a4700 1 -- 172.21.15.36:0/3975810198 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ec156c20 con 0x7f13ec061410 2022-06-17T12:56:33.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.882+0000 7f13eb7fe700 1 -- 172.21.15.36:0/3975810198 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13e0011c20 con 0x7f13ec061410 2022-06-17T12:56:33.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.882+0000 7f13f46a4700 1 -- 172.21.15.36:0/3975810198 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ec14aad0 con 0x7f13ec061410 2022-06-17T12:56:33.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.882+0000 7f13f46a4700 1 -- 172.21.15.36:0/3975810198 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13ec057410 con 0x7f13ec061410 2022-06-17T12:56:33.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.882+0000 7f13eb7fe700 1 -- 172.21.15.36:0/3975810198 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f13e0025030 con 0x7f13ec061410 2022-06-17T12:56:33.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.882+0000 7f13eb7fe700 1 --2- 172.21.15.36:0/3975810198 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f13cc053ad0 0x7f13cc055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:33.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.882+0000 7f13f1c3f700 1 --2- 172.21.15.36:0/3975810198 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f13cc053ad0 0x7f13cc055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:33.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.882+0000 7f13eb7fe700 1 -- 172.21.15.36:0/3975810198 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f13e006d260 con 0x7f13ec061410 2022-06-17T12:56:33.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.886+0000 7f13f1c3f700 1 --2- 172.21.15.36:0/3975810198 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f13cc053ad0 0x7f13cc055f90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f13dc00d000 tx=0x7f13dc008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:33.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:33.886+0000 7f13eb7fe700 1 -- 172.21.15.36:0/3975810198 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f13e0043f50 con 0x7f13ec061410 2022-06-17T12:56:33.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.010+0000 7f13f46a4700 1 -- 172.21.15.36:0/3975810198 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f13ec0adaf0 con 0x7f13ec061410 2022-06-17T12:56:33.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.014+0000 7f13eb7fe700 1 -- 172.21.15.36:0/3975810198 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v131) v1 ==== 81+0+34051 (secure 0 0 0) 0x7f13e0023090 con 0x7f13ec061410 2022-06-17T12:56:33.999 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:33.999 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:56:34.000 INFO:teuthology.orchestra.run.smithi036.stdout: "epoch": 131, 2022-06-17T12:56:34.000 INFO:teuthology.orchestra.run.smithi036.stdout: "fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", 2022-06-17T12:56:34.001 INFO:teuthology.orchestra.run.smithi036.stdout: "created": "2022-06-17T12:46:30.671408+0000", 2022-06-17T12:56:34.001 INFO:teuthology.orchestra.run.smithi036.stdout: "modified": "2022-06-17T12:56:12.025286+0000", 2022-06-17T12:56:34.002 INFO:teuthology.orchestra.run.smithi036.stdout: "last_up_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:56:34.002 INFO:teuthology.orchestra.run.smithi036.stdout: "last_in_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:56:34.002 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-17T12:56:34.003 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_num": 5799936, 2022-06-17T12:56:34.003 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_set": [ 2022-06-17T12:56:34.003 INFO:teuthology.orchestra.run.smithi036.stdout: "pglog_hardlimit", 2022-06-17T12:56:34.003 INFO:teuthology.orchestra.run.smithi036.stdout: "purged_snapdirs", 2022-06-17T12:56:34.004 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_deletes", 2022-06-17T12:56:34.004 INFO:teuthology.orchestra.run.smithi036.stdout: "sortbitwise" 2022-06-17T12:56:34.004 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:56:34.004 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_version": 6, 2022-06-17T12:56:34.004 INFO:teuthology.orchestra.run.smithi036.stdout: "full_ratio": 0.94999998807907104, 2022-06-17T12:56:34.005 INFO:teuthology.orchestra.run.smithi036.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-17T12:56:34.010 INFO:teuthology.orchestra.run.smithi036.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-17T12:56:34.010 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_snapshot": "", 2022-06-17T12:56:34.010 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_max": 26, 2022-06-17T12:56:34.011 INFO:teuthology.orchestra.run.smithi036.stdout: "max_osd": 8, 2022-06-17T12:56:34.011 INFO:teuthology.orchestra.run.smithi036.stdout: "require_min_compat_client": "jewel", 2022-06-17T12:56:34.012 INFO:teuthology.orchestra.run.smithi036.stdout: "min_compat_client": "jewel", 2022-06-17T12:56:34.012 INFO:teuthology.orchestra.run.smithi036.stdout: "require_osd_release": "octopus", 2022-06-17T12:56:34.013 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:56:34.013 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.013 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 1, 2022-06-17T12:56:34.013 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "device_health_metrics", 2022-06-17T12:56:34.014 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:33.881212+0000", 2022-06-17T12:56:34.014 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:56:34.014 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:56:34.014 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:56:34.015 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:56:34.015 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:56:34.015 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:56:34.015 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:56:34.015 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:56:34.017 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 1, 2022-06-17T12:56:34.018 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 1, 2022-06-17T12:56:34.018 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 1, 2022-06-17T12:56:34.019 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 1, 2022-06-17T12:56:34.019 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 1, 2022-06-17T12:56:34.020 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:56:34.020 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:56:34.021 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:56:34.021 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:56:34.021 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:56:34.022 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:56:34.023 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:56:34.023 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.024 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "16", 2022-06-17T12:56:34.024 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:56:34.025 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:56:34.025 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:56:34.026 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:56:34.026 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:56:34.027 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:56:34.027 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:56:34.030 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:56:34.030 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:56:34.030 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:56:34.031 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:56:34.031 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:56:34.031 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:56:34.031 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:56:34.043 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:56:34.043 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:56:34.044 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:56:34.044 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:56:34.044 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:56:34.044 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:56:34.045 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:56:34.045 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:56:34.045 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:56:34.045 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:56:34.047 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:56:34.047 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:56:34.047 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.047 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:56:34.048 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:56:34.049 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:56:34.049 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:56:34.049 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:56:34.050 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:56:34.050 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:56:34.050 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:56:34.050 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:56:34.051 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:56:34.051 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:56:34.051 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:56:34.051 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 1 2022-06-17T12:56:34.052 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.052 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:56:34.052 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr_devicehealth": {} 2022-06-17T12:56:34.053 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.053 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.053 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.053 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 2, 2022-06-17T12:56:34.054 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "rbd", 2022-06-17T12:56:34.054 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:44.485460+0000", 2022-06-17T12:56:34.054 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:56:34.054 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:56:34.055 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:56:34.055 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:56:34.055 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:56:34.055 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:56:34.056 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:56:34.056 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:56:34.056 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:56:34.057 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:56:34.057 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:56:34.058 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:56:34.058 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:56:34.058 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:56:34.058 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:56:34.059 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:56:34.059 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:56:34.059 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:56:34.059 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:56:34.060 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:56:34.060 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.060 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "19", 2022-06-17T12:56:34.060 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:56:34.060 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:56:34.061 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:56:34.061 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:56:34.061 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:56:34.061 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:56:34.062 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:56:34.062 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:56:34.062 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:56:34.062 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:56:34.063 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:56:34.063 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:56:34.064 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:56:34.064 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:56:34.064 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:56:34.065 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:56:34.065 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:56:34.065 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:56:34.065 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:56:34.065 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:56:34.066 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:56:34.068 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:56:34.070 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:56:34.070 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:56:34.071 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:56:34.071 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:56:34.071 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.072 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:56:34.072 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:56:34.072 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:56:34.072 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:56:34.072 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:56:34.073 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:56:34.073 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:56:34.074 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:56:34.074 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:56:34.075 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:56:34.075 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:56:34.075 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:56:34.075 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:56:34.076 INFO:teuthology.orchestra.run.smithi036.stdout: "rbd": {} 2022-06-17T12:56:34.076 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.076 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.076 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.077 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 25, 2022-06-17T12:56:34.077 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_metadata", 2022-06-17T12:56:34.077 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:56:08.924639+0000", 2022-06-17T12:56:34.078 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:56:34.078 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:56:34.079 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:56:34.079 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:56:34.080 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:56:34.081 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:56:34.081 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:56:34.082 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:56:34.083 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:56:34.084 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:56:34.095 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:56:34.096 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:56:34.097 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:56:34.097 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:56:34.098 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:56:34.098 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:56:34.099 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:56:34.099 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:56:34.100 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:56:34.100 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:56:34.101 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.101 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "130", 2022-06-17T12:56:34.101 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:56:34.102 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:56:34.102 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:56:34.123 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:56:34.124 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:56:34.124 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:56:34.124 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:56:34.125 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:56:34.126 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:56:34.126 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:56:34.127 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:56:34.127 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:56:34.127 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:56:34.127 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:56:34.128 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:56:34.128 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:56:34.128 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:56:34.128 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:56:34.129 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:56:34.129 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:56:34.129 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:56:34.129 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:56:34.129 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:56:34.130 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:56:34.130 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:56:34.130 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:56:34.131 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.131 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:56:34.131 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:56:34.131 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:56:34.131 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:56:34.132 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:56:34.132 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:56:34.132 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:56:34.132 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:56:34.133 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:56:34.133 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:56:34.133 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:56:34.133 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:56:34.134 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_bias": 4, 2022-06-17T12:56:34.134 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 16, 2022-06-17T12:56:34.134 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_priority": 5 2022-06-17T12:56:34.134 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.135 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:56:34.135 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:56:34.135 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": "cephfs" 2022-06-17T12:56:34.135 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.136 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.136 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.136 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.136 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 26, 2022-06-17T12:56:34.136 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_data", 2022-06-17T12:56:34.137 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:56:09.989204+0000", 2022-06-17T12:56:34.137 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:56:34.137 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:56:34.137 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:56:34.138 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:56:34.138 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:56:34.138 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:56:34.138 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:56:34.139 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:56:34.139 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:56:34.140 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:56:34.140 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:56:34.140 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:56:34.141 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:56:34.141 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:56:34.141 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:56:34.141 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:56:34.142 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:56:34.142 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:56:34.142 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:56:34.142 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:56:34.143 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.143 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "131", 2022-06-17T12:56:34.143 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:56:34.143 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:56:34.143 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:56:34.144 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:56:34.144 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:56:34.144 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:56:34.144 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:56:34.145 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:56:34.145 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:56:34.145 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:56:34.145 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:56:34.146 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:56:34.146 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:56:34.146 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:56:34.147 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:56:34.148 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:56:34.148 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:56:34.148 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:56:34.148 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:56:34.149 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:56:34.149 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:56:34.149 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:56:34.149 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:56:34.149 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:56:34.150 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:56:34.150 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:56:34.150 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.150 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:56:34.151 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:56:34.151 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:56:34.151 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:56:34.151 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:56:34.152 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:56:34.152 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:56:34.153 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:56:34.153 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:56:34.153 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:56:34.153 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:56:34.154 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:56:34.154 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:56:34.154 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:56:34.154 INFO:teuthology.orchestra.run.smithi036.stdout: "data": "cephfs" 2022-06-17T12:56:34.155 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.155 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.155 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.155 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:56:34.156 INFO:teuthology.orchestra.run.smithi036.stdout: "osds": [ 2022-06-17T12:56:34.156 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.156 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:56:34.156 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", 2022-06-17T12:56:34.157 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:56:34.157 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:56:34.157 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:56:34.157 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:56:34.158 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:56:34.158 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:56:34.158 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:56:34.158 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 128, 2022-06-17T12:56:34.159 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:56:34.159 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:56:34.159 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:56:34.159 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.160 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.160 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.160 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6802", 2022-06-17T12:56:34.160 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:56:34.161 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.161 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.161 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.161 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6803", 2022-06-17T12:56:34.162 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:56:34.162 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.162 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.162 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.163 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:56:34.163 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.163 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.163 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.163 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6804", 2022-06-17T12:56:34.164 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:56:34.164 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.164 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.164 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.165 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6805", 2022-06-17T12:56:34.165 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:56:34.165 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.165 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.166 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.166 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:56:34.166 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.167 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.167 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.167 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6808", 2022-06-17T12:56:34.168 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:56:34.168 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.168 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.169 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.169 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6809", 2022-06-17T12:56:34.169 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:56:34.169 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.170 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.170 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.175 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:56:34.175 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.175 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.176 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.176 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6806", 2022-06-17T12:56:34.177 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:56:34.177 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.178 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.179 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.179 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6807", 2022-06-17T12:56:34.180 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:56:34.181 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.181 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.181 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.181 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6803/22619", 2022-06-17T12:56:34.182 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6805/22619", 2022-06-17T12:56:34.182 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6809/22619", 2022-06-17T12:56:34.182 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6807/22619", 2022-06-17T12:56:34.183 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:56:34.183 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:56:34.183 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:56:34.183 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.184 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.184 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.185 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:56:34.185 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", 2022-06-17T12:56:34.185 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:56:34.185 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:56:34.186 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:56:34.186 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:56:34.186 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:56:34.186 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:56:34.187 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:56:34.187 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 129, 2022-06-17T12:56:34.187 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:56:34.187 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:56:34.188 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:56:34.188 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.188 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.188 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.189 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6810", 2022-06-17T12:56:34.189 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:56:34.189 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.189 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.190 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.190 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6811", 2022-06-17T12:56:34.190 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:56:34.190 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.191 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.191 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.191 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:56:34.191 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.192 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.192 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.192 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6812", 2022-06-17T12:56:34.192 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:56:34.192 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.193 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.193 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.193 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6813", 2022-06-17T12:56:34.194 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:56:34.194 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.194 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.194 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.195 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:56:34.195 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.195 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.196 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.196 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6816", 2022-06-17T12:56:34.196 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:56:34.197 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.197 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.197 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.197 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6817", 2022-06-17T12:56:34.198 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:56:34.198 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.198 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.198 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.199 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:56:34.199 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.214 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.215 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.215 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6814", 2022-06-17T12:56:34.215 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:56:34.215 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.216 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.216 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.216 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6815", 2022-06-17T12:56:34.217 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:56:34.217 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.217 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.217 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.223 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6811/22620", 2022-06-17T12:56:34.231 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6813/22620", 2022-06-17T12:56:34.237 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6817/22620", 2022-06-17T12:56:34.249 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6815/22620", 2022-06-17T12:56:34.249 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:56:34.249 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:56:34.250 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:56:34.254 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.255 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.255 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.255 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:56:34.256 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", 2022-06-17T12:56:34.256 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:56:34.256 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:56:34.256 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:56:34.256 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:56:34.257 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:56:34.257 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:56:34.257 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:56:34.257 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 129, 2022-06-17T12:56:34.262 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:56:34.262 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:56:34.262 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:56:34.262 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.263 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.263 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.270 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6826", 2022-06-17T12:56:34.271 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:56:34.272 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.272 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.273 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.273 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6827", 2022-06-17T12:56:34.274 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:56:34.274 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.275 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.275 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.276 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:56:34.276 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.277 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.277 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.278 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6828", 2022-06-17T12:56:34.278 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:56:34.279 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.280 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.280 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.281 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6829", 2022-06-17T12:56:34.281 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:56:34.282 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.306 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.307 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.307 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:56:34.308 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.308 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.309 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.309 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6832", 2022-06-17T12:56:34.343 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:56:34.344 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.345 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.345 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.346 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6833", 2022-06-17T12:56:34.346 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:56:34.353 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.354 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.368 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.369 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:56:34.369 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.369 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.369 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.370 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6830", 2022-06-17T12:56:34.370 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:56:34.370 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.370 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.371 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.371 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6831", 2022-06-17T12:56:34.371 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:56:34.371 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.372 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.372 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.372 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6827/22643", 2022-06-17T12:56:34.372 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6829/22643", 2022-06-17T12:56:34.373 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6833/22643", 2022-06-17T12:56:34.373 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6831/22643", 2022-06-17T12:56:34.373 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:56:34.399 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:56:34.400 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:56:34.400 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.400 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.401 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.401 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:56:34.401 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", 2022-06-17T12:56:34.401 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:56:34.402 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:56:34.402 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:56:34.402 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:56:34.403 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:56:34.403 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:56:34.404 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:56:34.404 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 128, 2022-06-17T12:56:34.404 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:56:34.404 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:56:34.414 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:56:34.414 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.415 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.415 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.415 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6818", 2022-06-17T12:56:34.416 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:56:34.428 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.428 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.429 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.429 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6819", 2022-06-17T12:56:34.430 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:56:34.430 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.430 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.430 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.431 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:56:34.431 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.431 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.431 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.432 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6820", 2022-06-17T12:56:34.432 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:56:34.432 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.432 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.432 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.433 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6821", 2022-06-17T12:56:34.433 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:56:34.434 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.434 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.434 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.434 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:56:34.435 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.435 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.435 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.435 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6824", 2022-06-17T12:56:34.436 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:56:34.436 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.436 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.436 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.437 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6825", 2022-06-17T12:56:34.437 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:56:34.437 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.450 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.451 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.451 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:56:34.452 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.452 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.453 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.456 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6822", 2022-06-17T12:56:34.456 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:56:34.457 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.457 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.458 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.459 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6823", 2022-06-17T12:56:34.459 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:56:34.459 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.460 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.460 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.461 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6819/22646", 2022-06-17T12:56:34.461 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6821/22646", 2022-06-17T12:56:34.462 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6825/22646", 2022-06-17T12:56:34.462 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6823/22646", 2022-06-17T12:56:34.463 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:56:34.463 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:56:34.463 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:56:34.464 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.464 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.465 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.466 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:56:34.466 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", 2022-06-17T12:56:34.467 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:56:34.467 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:56:34.468 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:56:34.468 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:56:34.469 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:56:34.469 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:56:34.469 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:56:34.470 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 129, 2022-06-17T12:56:34.470 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:56:34.471 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:56:34.472 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:56:34.472 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.473 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.473 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.473 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6808", 2022-06-17T12:56:34.474 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:56:34.474 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.475 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.475 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.475 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6809", 2022-06-17T12:56:34.476 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:56:34.476 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.477 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.477 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.478 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:56:34.478 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.479 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.479 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.479 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6810", 2022-06-17T12:56:34.480 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:56:34.480 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.481 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.481 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.481 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6811", 2022-06-17T12:56:34.493 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:56:34.493 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.494 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.494 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.495 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:56:34.510 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.510 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.511 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.511 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6814", 2022-06-17T12:56:34.512 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:56:34.512 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.512 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.513 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.513 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6815", 2022-06-17T12:56:34.514 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:56:34.514 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.515 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.515 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.516 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:56:34.516 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.522 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.523 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.523 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6812", 2022-06-17T12:56:34.527 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:56:34.527 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.528 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.528 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.529 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6813", 2022-06-17T12:56:34.529 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:56:34.530 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.530 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.531 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.531 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6809/22812", 2022-06-17T12:56:34.532 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6811/22812", 2022-06-17T12:56:34.532 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6815/22812", 2022-06-17T12:56:34.533 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6813/22812", 2022-06-17T12:56:34.534 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:56:34.534 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:56:34.534 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:56:34.535 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.535 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.536 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.536 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:56:34.536 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", 2022-06-17T12:56:34.537 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:56:34.537 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:56:34.538 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:56:34.538 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:56:34.539 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:56:34.539 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:56:34.539 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:56:34.540 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 122, 2022-06-17T12:56:34.540 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:56:34.541 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:56:34.562 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:56:34.563 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.563 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.563 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.564 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6800", 2022-06-17T12:56:34.564 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:56:34.564 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.564 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.564 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.565 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6801", 2022-06-17T12:56:34.565 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:56:34.565 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.565 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.566 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.566 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:56:34.566 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.572 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.572 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.573 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6802", 2022-06-17T12:56:34.573 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:56:34.575 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.576 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.576 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.576 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6803", 2022-06-17T12:56:34.577 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:56:34.578 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.579 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.579 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.579 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:56:34.580 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.580 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.581 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.581 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6806", 2022-06-17T12:56:34.581 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:56:34.582 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.582 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.583 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.584 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6807", 2022-06-17T12:56:34.585 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:56:34.585 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.586 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.588 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.588 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:56:34.589 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.589 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.589 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.590 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6804", 2022-06-17T12:56:34.590 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:56:34.591 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.591 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.592 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.592 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6805", 2022-06-17T12:56:34.592 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:56:34.593 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.593 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.594 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.595 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6801/22813", 2022-06-17T12:56:34.598 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6803/22813", 2022-06-17T12:56:34.599 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6807/22813", 2022-06-17T12:56:34.599 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6805/22813", 2022-06-17T12:56:34.600 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:56:34.601 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:56:34.601 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:56:34.602 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.603 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.603 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.604 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:56:34.613 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", 2022-06-17T12:56:34.614 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:56:34.614 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:56:34.615 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:56:34.620 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:56:34.620 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:56:34.621 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:56:34.621 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:56:34.621 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 129, 2022-06-17T12:56:34.622 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:56:34.622 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:56:34.623 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:56:34.625 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.625 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.625 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.626 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6816", 2022-06-17T12:56:34.628 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:56:34.628 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.628 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.629 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.629 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6817", 2022-06-17T12:56:34.630 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:56:34.630 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.630 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.631 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.631 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:56:34.632 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.632 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.633 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.633 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6818", 2022-06-17T12:56:34.633 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:56:34.639 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.640 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.640 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.641 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6819", 2022-06-17T12:56:34.641 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:56:34.642 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.642 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.645 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.645 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:56:34.646 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.647 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.659 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.659 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6822", 2022-06-17T12:56:34.659 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:56:34.668 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.668 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.670 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.670 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6823", 2022-06-17T12:56:34.670 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:56:34.671 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.671 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.671 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.671 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:56:34.672 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.672 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.672 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.672 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6820", 2022-06-17T12:56:34.672 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:56:34.673 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.673 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.673 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.673 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6821", 2022-06-17T12:56:34.674 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:56:34.674 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.674 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.674 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.675 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6817/22822", 2022-06-17T12:56:34.675 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6819/22822", 2022-06-17T12:56:34.675 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6823/22822", 2022-06-17T12:56:34.675 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6821/22822", 2022-06-17T12:56:34.676 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:56:34.676 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:56:34.676 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:56:34.677 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.677 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.677 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.677 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:56:34.677 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", 2022-06-17T12:56:34.678 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:56:34.678 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:56:34.678 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:56:34.679 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:56:34.679 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:56:34.679 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:56:34.680 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:56:34.680 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 129, 2022-06-17T12:56:34.680 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:56:34.681 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:56:34.682 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:56:34.682 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.682 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.683 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.683 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6824", 2022-06-17T12:56:34.683 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:56:34.683 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.683 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.684 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.684 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6825", 2022-06-17T12:56:34.684 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:56:34.684 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.685 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.685 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.685 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:56:34.685 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.686 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.686 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.686 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6826", 2022-06-17T12:56:34.686 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:56:34.687 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.687 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.687 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.687 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6827", 2022-06-17T12:56:34.688 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:56:34.688 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.688 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.688 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.689 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:56:34.689 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.689 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.689 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.691 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6830", 2022-06-17T12:56:34.691 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:56:34.692 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.692 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.692 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.692 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6831", 2022-06-17T12:56:34.693 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:56:34.693 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.693 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.694 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.694 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:56:34.694 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:34.695 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.695 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:34.695 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6828", 2022-06-17T12:56:34.695 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:56:34.695 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.696 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.696 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:34.696 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6829", 2022-06-17T12:56:34.696 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:56:34.697 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.697 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.697 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.697 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6825/22837", 2022-06-17T12:56:34.698 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6827/22837", 2022-06-17T12:56:34.698 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6831/22837", 2022-06-17T12:56:34.698 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6829/22837", 2022-06-17T12:56:34.698 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:56:34.699 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:56:34.699 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:56:34.699 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:34.699 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.700 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:56:34.700 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_xinfo": [ 2022-06-17T12:56:34.700 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.701 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:56:34.701 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:56:34.701 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:56:34.702 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:56:34.702 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:56:34.702 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:56:34.703 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.291461+0000", 2022-06-17T12:56:34.703 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:56:34.703 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.703 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.704 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:56:34.704 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:56:34.704 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:56:34.704 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:56:34.704 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:56:34.705 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:56:34.705 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.537912+0000", 2022-06-17T12:56:34.705 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:56:34.705 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.706 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.706 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:56:34.706 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:56:34.706 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:56:34.706 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:56:34.707 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:56:34.707 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:56:34.707 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.606947+0000", 2022-06-17T12:56:34.707 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:56:34.708 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.708 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.708 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:56:34.708 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:56:34.709 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:56:34.709 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:56:34.709 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:56:34.710 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:56:34.710 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.514412+0000", 2022-06-17T12:56:34.710 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:56:34.711 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.711 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.711 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:56:34.712 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:56:34.712 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:56:34.713 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:56:34.713 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:56:34.713 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:56:34.713 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.394645+0000", 2022-06-17T12:56:34.714 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:56:34.714 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.714 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.714 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:56:34.715 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:56:34.715 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:56:34.715 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:56:34.715 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:56:34.716 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:56:34.716 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.343806+0000", 2022-06-17T12:56:34.716 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:56:34.716 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.716 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.717 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:56:34.717 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:56:34.717 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:56:34.717 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:56:34.718 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:56:34.718 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:56:34.718 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.591864+0000", 2022-06-17T12:56:34.719 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:56:34.719 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.719 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:34.719 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:56:34.720 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:56:34.720 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:56:34.720 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:56:34.720 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:56:34.721 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:56:34.721 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.607045+0000", 2022-06-17T12:56:34.721 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:56:34.722 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.722 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:56:34.723 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap": [], 2022-06-17T12:56:34.723 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap_items": [], 2022-06-17T12:56:34.723 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_temp": [], 2022-06-17T12:56:34.724 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_temp": [], 2022-06-17T12:56:34.724 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklist": {}, 2022-06-17T12:56:34.724 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profiles": { 2022-06-17T12:56:34.724 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs_data": { 2022-06-17T12:56:34.725 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-device-class": "", 2022-06-17T12:56:34.725 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:56:34.725 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-root": "default", 2022-06-17T12:56:34.726 INFO:teuthology.orchestra.run.smithi036.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-17T12:56:34.726 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:56:34.726 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "2", 2022-06-17T12:56:34.726 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:56:34.726 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:56:34.727 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van", 2022-06-17T12:56:34.727 INFO:teuthology.orchestra.run.smithi036.stdout: "w": "8" 2022-06-17T12:56:34.727 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.727 INFO:teuthology.orchestra.run.smithi036.stdout: "default": { 2022-06-17T12:56:34.728 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:56:34.728 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:56:34.728 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "1", 2022-06-17T12:56:34.728 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:56:34.728 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:56:34.729 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van" 2022-06-17T12:56:34.729 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:34.729 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:34.729 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps_queue": [], 2022-06-17T12:56:34.730 INFO:teuthology.orchestra.run.smithi036.stdout: "new_removed_snaps": [], 2022-06-17T12:56:34.730 INFO:teuthology.orchestra.run.smithi036.stdout: "new_purged_snaps": [], 2022-06-17T12:56:34.730 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_node_flags": {}, 2022-06-17T12:56:34.730 INFO:teuthology.orchestra.run.smithi036.stdout: "device_class_flags": {} 2022-06-17T12:56:34.731 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:56:34.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.014+0000 7f13f46a4700 1 -- 172.21.15.36:0/3975810198 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f13cc053ad0 msgr2=0x7f13cc055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:34.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.014+0000 7f13f46a4700 1 --2- 172.21.15.36:0/3975810198 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f13cc053ad0 0x7f13cc055f90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f13dc00d000 tx=0x7f13dc008040).stop 2022-06-17T12:56:34.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.014+0000 7f13f46a4700 1 -- 172.21.15.36:0/3975810198 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f13ec061410 msgr2=0x7f13ec0aeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:34.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.014+0000 7f13f46a4700 1 --2- 172.21.15.36:0/3975810198 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f13ec061410 0x7f13ec0aeaf0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0x7f13e0002f70 tx=0x7f13e0015000).stop 2022-06-17T12:56:34.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.014+0000 7f13f46a4700 1 -- 172.21.15.36:0/3975810198 shutdown_connections 2022-06-17T12:56:34.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.014+0000 7f13f46a4700 1 --2- 172.21.15.36:0/3975810198 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f13cc053ad0 0x7f13cc055f90 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:34.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.014+0000 7f13f46a4700 1 --2- 172.21.15.36:0/3975810198 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13ec0b7a00 0x7f13ec0b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:34.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.014+0000 7f13f46a4700 1 --2- 172.21.15.36:0/3975810198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13ec0adfc0 0x7f13ec149200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:34.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.014+0000 7f13f46a4700 1 --2- 172.21.15.36:0/3975810198 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f13ec061410 0x7f13ec0aeaf0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:34.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.014+0000 7f13f46a4700 1 -- 172.21.15.36:0/3975810198 >> 172.21.15.36:0/3975810198 conn(0x7f13ec05be50 msgr2=0x7f13ec063180 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:34.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.014+0000 7f13f46a4700 1 -- 172.21.15.36:0/3975810198 shutdown_connections 2022-06-17T12:56:34.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.014+0000 7f13f46a4700 1 -- 172.21.15.36:0/3975810198 wait complete. 2022-06-17T12:56:34.738 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:56:34.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.866+0000 7f382202b700 1 Processor -- start 2022-06-17T12:56:34.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.866+0000 7f382202b700 1 -- start start 2022-06-17T12:56:34.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f382202b700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f381c0b7a80 0x7f381c0b2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:34.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f382202b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f381c0ad470 0x7f381c141060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:34.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f382202b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f381c144d00 0x7f381c145130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:34.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f382202b700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f381c0b81c0 con 0x7f381c144d00 2022-06-17T12:56:34.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f382202b700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f381c144630 con 0x7f381c0b7a80 2022-06-17T12:56:34.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f382202b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381c0b7e60 con 0x7f381c0ad470 2022-06-17T12:56:34.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f381bfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f381c144d00 0x7f381c145130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:34.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f381bfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f381c144d00 0x7f381c145130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39646/0 (socket says 172.21.15.36:39646) 2022-06-17T12:56:34.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f381bfff700 1 -- 172.21.15.36:0/3627457369 learned_addr learned my addr 172.21.15.36:0/3627457369 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:34.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f381b7fe700 1 --2- 172.21.15.36:0/3627457369 >> v2:172.21.15.70:3300/0 conn(0x7f381c0b7a80 0x7f381c0b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:34.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f381affd700 1 --2- 172.21.15.36:0/3627457369 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f381c0ad470 0x7f381c141060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:34.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f381bfff700 1 -- 172.21.15.36:0/3627457369 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f381c0ad470 msgr2=0x7f381c141060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:34.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f381bfff700 1 --2- 172.21.15.36:0/3627457369 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f381c0ad470 0x7f381c141060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:34.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f381bfff700 1 -- 172.21.15.36:0/3627457369 >> v2:172.21.15.70:3300/0 conn(0x7f381c0b7a80 msgr2=0x7f381c0b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:34.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f381bfff700 1 --2- 172.21.15.36:0/3627457369 >> v2:172.21.15.70:3300/0 conn(0x7f381c0b7a80 0x7f381c0b2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:34.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f381bfff700 1 -- 172.21.15.36:0/3627457369 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381c0b83b0 con 0x7f381c144d00 2022-06-17T12:56:34.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f381bfff700 1 --2- 172.21.15.36:0/3627457369 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f381c144d00 0x7f381c145130 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7f380c002f70 tx=0x7f380c006eb0).ready entity=mon.0 client_cookie=1ab43e6af40b28f9 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:34.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f381a7fc700 1 -- 172.21.15.36:0/3627457369 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f380c00d570 con 0x7f381c144d00 2022-06-17T12:56:34.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f381a7fc700 1 -- 172.21.15.36:0/3627457369 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380c000e90 con 0x7f381c144d00 2022-06-17T12:56:34.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f381a7fc700 1 -- 172.21.15.36:0/3627457369 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f380c00d900 con 0x7f381c144d00 2022-06-17T12:56:34.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f382202b700 1 -- 172.21.15.36:0/3627457369 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f381c144d00 msgr2=0x7f381c145130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:34.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f382202b700 1 --2- 172.21.15.36:0/3627457369 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f381c144d00 0x7f381c145130 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7f380c002f70 tx=0x7f380c006eb0).stop 2022-06-17T12:56:34.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f382202b700 1 -- 172.21.15.36:0/3627457369 shutdown_connections 2022-06-17T12:56:34.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f382202b700 1 --2- 172.21.15.36:0/3627457369 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f381c144d00 0x7f381c145130 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:34.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f382202b700 1 --2- 172.21.15.36:0/3627457369 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f381c0ad470 0x7f381c141060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:34.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f382202b700 1 --2- 172.21.15.36:0/3627457369 >> v2:172.21.15.70:3300/0 conn(0x7f381c0b7a80 0x7f381c0b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:34.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f382202b700 1 -- 172.21.15.36:0/3627457369 >> 172.21.15.36:0/3627457369 conn(0x7f381c064800 msgr2=0x7f381c064be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:34.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f382202b700 1 -- 172.21.15.36:0/3627457369 shutdown_connections 2022-06-17T12:56:34.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.870+0000 7f382202b700 1 -- 172.21.15.36:0/3627457369 wait complete. 2022-06-17T12:56:34.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f382202b700 1 Processor -- start 2022-06-17T12:56:34.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f382202b700 1 -- start start 2022-06-17T12:56:34.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f382202b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f381c0ad470 0x7f381c065f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:34.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f382202b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f381c0b7a80 0x7f381c13fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:34.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f382202b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f381c144d00 0x7f381c13ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:34.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f382202b700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c140410 con 0x7f381c144d00 2022-06-17T12:56:34.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f382202b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c13f5d0 con 0x7f381c0b7a80 2022-06-17T12:56:34.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f382202b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381c13e010 con 0x7f381c0ad470 2022-06-17T12:56:34.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f381bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f381c144d00 0x7f381c13ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:34.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f381b7fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f381c0ad470 0x7f381c065f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:34.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f381affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f381c0b7a80 0x7f381c13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:34.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f381bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f381c144d00 0x7f381c13ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39652/0 (socket says 172.21.15.36:39652) 2022-06-17T12:56:34.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f381b7fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f381c0ad470 0x7f381c065f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:54438/0 (socket says 172.21.15.36:54438) 2022-06-17T12:56:34.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f381bfff700 1 -- 172.21.15.36:0/15354418 learned_addr learned my addr 172.21.15.36:0/15354418 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:34.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f381bfff700 1 -- 172.21.15.36:0/15354418 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f381c0ad470 msgr2=0x7f381c065f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:34.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f381bfff700 1 --2- 172.21.15.36:0/15354418 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f381c0ad470 0x7f381c065f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:34.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f381bfff700 1 -- 172.21.15.36:0/15354418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f381c0b7a80 msgr2=0x7f381c13fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:34.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f381bfff700 1 --2- 172.21.15.36:0/15354418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f381c0b7a80 0x7f381c13fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:34.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f381bfff700 1 -- 172.21.15.36:0/15354418 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380c00b1e0 con 0x7f381c144d00 2022-06-17T12:56:34.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f381affd700 1 --2- 172.21.15.36:0/15354418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f381c0b7a80 0x7f381c13fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:34.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f381bfff700 1 --2- 172.21.15.36:0/15354418 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f381c144d00 0x7f381c13ff20 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7f380c002f70 tx=0x7f380c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:34.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f3818ff9700 1 -- 172.21.15.36:0/15354418 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f380c0117e0 con 0x7f381c144d00 2022-06-17T12:56:34.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f3818ff9700 1 -- 172.21.15.36:0/15354418 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380c011940 con 0x7f381c144d00 2022-06-17T12:56:34.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f382202b700 1 -- 172.21.15.36:0/15354418 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f381c13e2b0 con 0x7f381c144d00 2022-06-17T12:56:34.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f3818ff9700 1 -- 172.21.15.36:0/15354418 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f380c011ca0 con 0x7f381c144d00 2022-06-17T12:56:34.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.874+0000 7f382202b700 1 -- 172.21.15.36:0/15354418 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381c13e8b0 con 0x7f381c144d00 2022-06-17T12:56:34.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.878+0000 7f382202b700 1 -- 172.21.15.36:0/15354418 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f381c057410 con 0x7f381c144d00 2022-06-17T12:56:34.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.878+0000 7f3818ff9700 1 -- 172.21.15.36:0/15354418 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f380c025030 con 0x7f381c144d00 2022-06-17T12:56:34.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.878+0000 7f3818ff9700 1 --2- 172.21.15.36:0/15354418 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3800053920 0x7f3800055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:34.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.878+0000 7f3818ff9700 1 -- 172.21.15.36:0/15354418 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f380c06d080 con 0x7f381c144d00 2022-06-17T12:56:34.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.878+0000 7f381b7fe700 1 --2- 172.21.15.36:0/15354418 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3800053920 0x7f3800055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:34.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.878+0000 7f381b7fe700 1 --2- 172.21.15.36:0/15354418 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3800053920 0x7f3800055de0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f381400b000 tx=0x7f38140091a0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:34.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:34.890+0000 7f3818ff9700 1 -- 172.21.15.36:0/15354418 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f380c043de0 con 0x7f381c144d00 2022-06-17T12:56:35.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.054+0000 7f382202b700 1 -- 172.21.15.36:0/15354418 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f381c0b86d0 con 0x7f381c144d00 2022-06-17T12:56:35.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.054+0000 7f3818ff9700 1 -- 172.21.15.36:0/15354418 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f380c023090 con 0x7f381c144d00 2022-06-17T12:56:35.045 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:35.057 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":72,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/2357210652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2357210652},{"type":"v1","addr":"172.21.15.36:6835","nonce":2357210652}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:56:11.021928+0000","modified":"2022-06-17T12:56:13.034227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7942":{"gid":7942,"name":"b","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/197726269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":197726269},{"type":"v1","addr":"172.21.15.70:6833","nonce":197726269}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-06-17T12:56:35.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.058+0000 7f382202b700 1 -- 172.21.15.36:0/15354418 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3800053920 msgr2=0x7f3800055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:35.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.058+0000 7f382202b700 1 --2- 172.21.15.36:0/15354418 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3800053920 0x7f3800055de0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f381400b000 tx=0x7f38140091a0).stop 2022-06-17T12:56:35.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.058+0000 7f382202b700 1 -- 172.21.15.36:0/15354418 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f381c144d00 msgr2=0x7f381c13ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:35.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.058+0000 7f382202b700 1 --2- 172.21.15.36:0/15354418 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f381c144d00 0x7f381c13ff20 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7f380c002f70 tx=0x7f380c015000).stop 2022-06-17T12:56:35.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.058+0000 7f382202b700 1 -- 172.21.15.36:0/15354418 shutdown_connections 2022-06-17T12:56:35.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.058+0000 7f382202b700 1 --2- 172.21.15.36:0/15354418 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3800053920 0x7f3800055de0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:35.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.058+0000 7f382202b700 1 --2- 172.21.15.36:0/15354418 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f381c144d00 0x7f381c13ff20 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:35.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.058+0000 7f382202b700 1 --2- 172.21.15.36:0/15354418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f381c0b7a80 0x7f381c13fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:35.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.058+0000 7f382202b700 1 --2- 172.21.15.36:0/15354418 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f381c0ad470 0x7f381c065f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:35.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.058+0000 7f382202b700 1 -- 172.21.15.36:0/15354418 >> 172.21.15.36:0/15354418 conn(0x7f381c064800 msgr2=0x7f381c13bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:35.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.058+0000 7f382202b700 1 -- 172.21.15.36:0/15354418 shutdown_connections 2022-06-17T12:56:35.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.058+0000 7f382202b700 1 -- 172.21.15.36:0/15354418 wait complete. 2022-06-17T12:56:35.079 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 72 2022-06-17T12:56:35.080 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-17T12:56:35.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.210+0000 7fe3314b9700 1 Processor -- start 2022-06-17T12:56:35.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.210+0000 7fe3314b9700 1 -- start start 2022-06-17T12:56:35.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.210+0000 7fe3314b9700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe32c0b5c40 0x7fe32c14ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:35.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.210+0000 7fe3314b9700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe32c0ac1c0 0x7fe32c14b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:35.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.210+0000 7fe3314b9700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe32c0b6020 con 0x7fe32c14b830 2022-06-17T12:56:35.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.210+0000 7fe3314b9700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe32c0abf10 con 0x7fe32c0ac1c0 2022-06-17T12:56:35.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.210+0000 7fe3314b9700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c05c7f0 con 0x7fe32c0b5c40 2022-06-17T12:56:35.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.210+0000 7fe32affd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe32c0b5c40 0x7fe32c14ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:35.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.210+0000 7fe32a7fc700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe32c0ac1c0 0x7fe32c14b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:35.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.210+0000 7fe32affd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe32c0b5c40 0x7fe32c14ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:54446/0 (socket says 172.21.15.36:54446) 2022-06-17T12:56:35.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.210+0000 7fe32affd700 1 -- 172.21.15.36:0/1142774547 learned_addr learned my addr 172.21.15.36:0/1142774547 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:35.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.210+0000 7fe32affd700 1 -- 172.21.15.36:0/1142774547 >> v2:172.21.15.70:3300/0 conn(0x7fe32c0ac1c0 msgr2=0x7fe32c14b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:35.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.210+0000 7fe32affd700 1 --2- 172.21.15.36:0/1142774547 >> v2:172.21.15.70:3300/0 conn(0x7fe32c0ac1c0 0x7fe32c14b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:35.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.210+0000 7fe32affd700 1 -- 172.21.15.36:0/1142774547 >> v1:172.21.15.70:6789/0 conn(0x7fe32c14b830 legacy=0x7fe32c0b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:35.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.210+0000 7fe32affd700 1 -- 172.21.15.36:0/1142774547 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32c05ca90 con 0x7fe32c0b5c40 2022-06-17T12:56:35.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.214+0000 7fe32affd700 1 --2- 172.21.15.36:0/1142774547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe32c0b5c40 0x7fe32c14ae50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fe31c002f70 tx=0x7fe31c006eb0).ready entity=mon.2 client_cookie=34f0508fc6b1b33a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:35.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.214+0000 7fe329ffb700 1 -- 172.21.15.36:0/1142774547 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe31c00d570 con 0x7fe32c0b5c40 2022-06-17T12:56:35.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.214+0000 7fe329ffb700 1 -- 172.21.15.36:0/1142774547 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31c000e90 con 0x7fe32c0b5c40 2022-06-17T12:56:35.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.214+0000 7fe329ffb700 1 -- 172.21.15.36:0/1142774547 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe31c00d900 con 0x7fe32c0b5c40 2022-06-17T12:56:35.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.214+0000 7fe3314b9700 1 -- 172.21.15.36:0/1142774547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe32c0b5c40 msgr2=0x7fe32c14ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:35.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.214+0000 7fe3314b9700 1 --2- 172.21.15.36:0/1142774547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe32c0b5c40 0x7fe32c14ae50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fe31c002f70 tx=0x7fe31c006eb0).stop 2022-06-17T12:56:35.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.214+0000 7fe3314b9700 1 -- 172.21.15.36:0/1142774547 shutdown_connections 2022-06-17T12:56:35.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.214+0000 7fe3314b9700 1 --2- 172.21.15.36:0/1142774547 >> v2:172.21.15.70:3300/0 conn(0x7fe32c0ac1c0 0x7fe32c14b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:35.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.214+0000 7fe3314b9700 1 --2- 172.21.15.36:0/1142774547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe32c0b5c40 0x7fe32c14ae50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:35.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.214+0000 7fe3314b9700 1 -- 172.21.15.36:0/1142774547 >> 172.21.15.36:0/1142774547 conn(0x7fe32c0624e0 msgr2=0x7fe32c0628f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:35.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.214+0000 7fe3314b9700 1 -- 172.21.15.36:0/1142774547 shutdown_connections 2022-06-17T12:56:35.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.214+0000 7fe3314b9700 1 -- 172.21.15.36:0/1142774547 wait complete. 2022-06-17T12:56:35.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.214+0000 7fe3314b9700 1 Processor -- start 2022-06-17T12:56:35.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.214+0000 7fe3314b9700 1 -- start start 2022-06-17T12:56:35.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.218+0000 7fe3314b9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe32c0ac1c0 0x7fe32c09ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:35.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.218+0000 7fe3314b9700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe32c0b5c40 0x7fe32c0a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:35.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.218+0000 7fe3314b9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe32c14b830 0x7fe32c0a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:35.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.218+0000 7fe3314b9700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c065f50 con 0x7fe32c0ac1c0 2022-06-17T12:56:35.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.218+0000 7fe3314b9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c067220 con 0x7fe32c14b830 2022-06-17T12:56:35.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.218+0000 7fe3314b9700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c066180 con 0x7fe32c0b5c40 2022-06-17T12:56:35.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.218+0000 7fe32affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe32c0ac1c0 0x7fe32c09ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:35.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.218+0000 7fe32affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe32c0ac1c0 0x7fe32c09ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39662/0 (socket says 172.21.15.36:39662) 2022-06-17T12:56:35.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.218+0000 7fe32a7fc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe32c0b5c40 0x7fe32c0a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:35.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.218+0000 7fe32b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe32c14b830 0x7fe32c0a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:35.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.218+0000 7fe32affd700 1 -- 172.21.15.36:0/697070324 learned_addr learned my addr 172.21.15.36:0/697070324 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:35.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.218+0000 7fe32affd700 1 -- 172.21.15.36:0/697070324 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe32c0b5c40 msgr2=0x7fe32c0a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:35.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.218+0000 7fe32affd700 1 --2- 172.21.15.36:0/697070324 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe32c0b5c40 0x7fe32c0a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:35.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.218+0000 7fe32affd700 1 -- 172.21.15.36:0/697070324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe32c14b830 msgr2=0x7fe32c0a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:35.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.218+0000 7fe32affd700 1 --2- 172.21.15.36:0/697070324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe32c14b830 0x7fe32c0a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:35.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.218+0000 7fe32affd700 1 -- 172.21.15.36:0/697070324 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c00b1e0 con 0x7fe32c0ac1c0 2022-06-17T12:56:35.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.218+0000 7fe32affd700 1 --2- 172.21.15.36:0/697070324 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe32c0ac1c0 0x7fe32c09ee80 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7fe32c0673b0 tx=0x7fe31c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:35.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.218+0000 7fe323fff700 1 -- 172.21.15.36:0/697070324 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe31c00d8f0 con 0x7fe32c0ac1c0 2022-06-17T12:56:35.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.218+0000 7fe323fff700 1 -- 172.21.15.36:0/697070324 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31c011b40 con 0x7fe32c0ac1c0 2022-06-17T12:56:35.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.218+0000 7fe3314b9700 1 -- 172.21.15.36:0/697070324 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe32c066f80 con 0x7fe32c0ac1c0 2022-06-17T12:56:35.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.218+0000 7fe323fff700 1 -- 172.21.15.36:0/697070324 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe31c011ea0 con 0x7fe32c0ac1c0 2022-06-17T12:56:35.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.218+0000 7fe3314b9700 1 -- 172.21.15.36:0/697070324 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32c065000 con 0x7fe32c0ac1c0 2022-06-17T12:56:35.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.218+0000 7fe3314b9700 1 -- 172.21.15.36:0/697070324 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe32c0a1890 con 0x7fe32c0ac1c0 2022-06-17T12:56:35.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.222+0000 7fe323fff700 1 -- 172.21.15.36:0/697070324 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe31c025030 con 0x7fe32c0ac1c0 2022-06-17T12:56:35.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.222+0000 7fe323fff700 1 --2- 172.21.15.36:0/697070324 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe304053ad0 0x7fe304055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:35.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.222+0000 7fe323fff700 1 -- 172.21.15.36:0/697070324 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe31c06d2d0 con 0x7fe32c0ac1c0 2022-06-17T12:56:35.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.222+0000 7fe32a7fc700 1 --2- 172.21.15.36:0/697070324 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe304053ad0 0x7fe304055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:35.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.222+0000 7fe32a7fc700 1 --2- 172.21.15.36:0/697070324 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe304053ad0 0x7fe304055f90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7fe31400b000 tx=0x7fe314006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:35.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.230+0000 7fe323fff700 1 -- 172.21.15.36:0/697070324 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe31c043fc0 con 0x7fe32c0ac1c0 2022-06-17T12:56:35.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.390+0000 7fe3314b9700 1 -- 172.21.15.36:0/697070324 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe32c05cd30 con 0x7fe32c0ac1c0 2022-06-17T12:56:35.742 INFO:tasks.ceph.mon.a.smithi036.stderr:2022-06-17T12:56:35.750+0000 7fb7a2649700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-17T12:56:35.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.766+0000 7fe323fff700 1 -- 172.21.15.36:0/697070324 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v73) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe31c023090 con 0x7fe32c0ac1c0 2022-06-17T12:56:35.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.766+0000 7fe3314b9700 1 -- 172.21.15.36:0/697070324 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe304053ad0 msgr2=0x7fe304055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:35.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.766+0000 7fe3314b9700 1 --2- 172.21.15.36:0/697070324 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe304053ad0 0x7fe304055f90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7fe31400b000 tx=0x7fe314006df0).stop 2022-06-17T12:56:35.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.766+0000 7fe3314b9700 1 -- 172.21.15.36:0/697070324 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe32c0ac1c0 msgr2=0x7fe32c09ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:35.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.766+0000 7fe3314b9700 1 --2- 172.21.15.36:0/697070324 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe32c0ac1c0 0x7fe32c09ee80 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7fe32c0673b0 tx=0x7fe31c015000).stop 2022-06-17T12:56:35.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.770+0000 7fe3314b9700 1 -- 172.21.15.36:0/697070324 shutdown_connections 2022-06-17T12:56:35.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.770+0000 7fe3314b9700 1 --2- 172.21.15.36:0/697070324 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe304053ad0 0x7fe304055f90 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:35.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.770+0000 7fe3314b9700 1 --2- 172.21.15.36:0/697070324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe32c14b830 0x7fe32c0a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:35.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.770+0000 7fe3314b9700 1 --2- 172.21.15.36:0/697070324 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe32c0b5c40 0x7fe32c0a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:35.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.770+0000 7fe3314b9700 1 --2- 172.21.15.36:0/697070324 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe32c0ac1c0 0x7fe32c09ee80 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:35.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.770+0000 7fe3314b9700 1 -- 172.21.15.36:0/697070324 >> 172.21.15.36:0/697070324 conn(0x7fe32c0624e0 msgr2=0x7fe32c14a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:35.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.770+0000 7fe3314b9700 1 -- 172.21.15.36:0/697070324 shutdown_connections 2022-06-17T12:56:35.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.770+0000 7fe3314b9700 1 -- 172.21.15.36:0/697070324 wait complete. 2022-06-17T12:56:35.765 INFO:teuthology.orchestra.run.smithi036.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-17T12:56:35.767 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:56:35.790 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2022-06-17T12:56:35.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5cb25aa700 1 Processor -- start 2022-06-17T12:56:35.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5cb25aa700 1 -- start start 2022-06-17T12:56:35.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5cb25aa700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5cac0b26e0 0x7f5cac0618b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:35.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5cb25aa700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5cac140260 0x7f5cac1406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:35.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5cb25aa700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cac143e40 con 0x7f5cac0b3010 2022-06-17T12:56:35.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5cb25aa700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cac064760 con 0x7f5cac140260 2022-06-17T12:56:35.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5cb25aa700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cac1440e0 con 0x7f5cac0b26e0 2022-06-17T12:56:35.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5cb1da9700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5cac140260 0x7f5cac1406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:35.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5cb1da9700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5cac140260 0x7f5cac1406d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39672/0 (socket says 172.21.15.36:39672) 2022-06-17T12:56:35.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5cb1da9700 1 -- 172.21.15.36:0/1067014138 learned_addr learned my addr 172.21.15.36:0/1067014138 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:35.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5cb0da7700 1 --2- 172.21.15.36:0/1067014138 >> v2:172.21.15.70:3300/0 conn(0x7f5cac0b26e0 0x7f5cac0618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:35.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5cb1da9700 1 -- 172.21.15.36:0/1067014138 >> v2:172.21.15.70:3300/0 conn(0x7f5cac0b26e0 msgr2=0x7f5cac0618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:35.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5cb1da9700 1 --2- 172.21.15.36:0/1067014138 >> v2:172.21.15.70:3300/0 conn(0x7f5cac0b26e0 0x7f5cac0618b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:35.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5cb1da9700 1 -- 172.21.15.36:0/1067014138 >> v1:172.21.15.70:6789/0 conn(0x7f5cac0b3010 legacy=0x7f5cac139980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:35.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5cb1da9700 1 -- 172.21.15.36:0/1067014138 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac13fd50 con 0x7f5cac140260 2022-06-17T12:56:35.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5cb1da9700 1 --2- 172.21.15.36:0/1067014138 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cac140260 0x7f5cac1406d0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7f5cac0b3960 tx=0x7f5ca8006eb0).ready entity=mon.0 client_cookie=324c1f9c7e2eefa1 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:35.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5c9ffff700 1 -- 172.21.15.36:0/1067014138 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ca800d5c0 con 0x7f5cac140260 2022-06-17T12:56:35.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5c9ffff700 1 -- 172.21.15.36:0/1067014138 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca8000e90 con 0x7f5cac140260 2022-06-17T12:56:35.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5c9ffff700 1 -- 172.21.15.36:0/1067014138 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ca800d910 con 0x7f5cac140260 2022-06-17T12:56:35.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5cb25aa700 1 -- 172.21.15.36:0/1067014138 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cac140260 msgr2=0x7f5cac1406d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:35.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5cb25aa700 1 --2- 172.21.15.36:0/1067014138 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cac140260 0x7f5cac1406d0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7f5cac0b3960 tx=0x7f5ca8006eb0).stop 2022-06-17T12:56:35.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5cb25aa700 1 -- 172.21.15.36:0/1067014138 shutdown_connections 2022-06-17T12:56:35.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5cb25aa700 1 --2- 172.21.15.36:0/1067014138 >> v2:172.21.15.70:3300/0 conn(0x7f5cac0b26e0 0x7f5cac0618b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:35.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5cb25aa700 1 --2- 172.21.15.36:0/1067014138 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cac140260 0x7f5cac1406d0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:35.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5cb25aa700 1 -- 172.21.15.36:0/1067014138 >> 172.21.15.36:0/1067014138 conn(0x7f5cac063550 msgr2=0x7f5cac063930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:35.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5cb25aa700 1 -- 172.21.15.36:0/1067014138 shutdown_connections 2022-06-17T12:56:35.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.894+0000 7f5cb25aa700 1 -- 172.21.15.36:0/1067014138 wait complete. 2022-06-17T12:56:35.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.898+0000 7f5cb25aa700 1 Processor -- start 2022-06-17T12:56:35.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.898+0000 7f5cb25aa700 1 -- start start 2022-06-17T12:56:35.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.898+0000 7f5cb25aa700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cac0b26e0 0x7f5cac13ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:35.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.898+0000 7f5cb25aa700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5cac0b3010 0x7f5cac152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:35.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.898+0000 7f5cb25aa700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cac140260 0x7f5cac152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:35.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.898+0000 7f5cb25aa700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac138420 con 0x7f5cac140260 2022-06-17T12:56:35.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.898+0000 7f5cb25aa700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac10fcf0 con 0x7f5cac0b26e0 2022-06-17T12:56:35.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.898+0000 7f5cb25aa700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac152430 con 0x7f5cac0b3010 2022-06-17T12:56:35.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.898+0000 7f5cb1da9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cac140260 0x7f5cac152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:35.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.898+0000 7f5cb1da9700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cac140260 0x7f5cac152d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39680/0 (socket says 172.21.15.36:39680) 2022-06-17T12:56:35.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.898+0000 7f5cb1da9700 1 -- 172.21.15.36:0/3810796332 learned_addr learned my addr 172.21.15.36:0/3810796332 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:35.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.898+0000 7f5cb0da7700 1 --2- 172.21.15.36:0/3810796332 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5cac0b3010 0x7f5cac152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:35.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.898+0000 7f5cb15a8700 1 --2- 172.21.15.36:0/3810796332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cac0b26e0 0x7f5cac13ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:35.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.902+0000 7f5cb1da9700 1 -- 172.21.15.36:0/3810796332 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5cac0b3010 msgr2=0x7f5cac152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:35.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.902+0000 7f5cb1da9700 1 --2- 172.21.15.36:0/3810796332 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5cac0b3010 0x7f5cac152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:35.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.902+0000 7f5cb1da9700 1 -- 172.21.15.36:0/3810796332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cac0b26e0 msgr2=0x7f5cac13ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:35.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.902+0000 7f5cb1da9700 1 --2- 172.21.15.36:0/3810796332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cac0b26e0 0x7f5cac13ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:35.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.902+0000 7f5cb1da9700 1 -- 172.21.15.36:0/3810796332 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca800b1e0 con 0x7f5cac140260 2022-06-17T12:56:35.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.902+0000 7f5cb1da9700 1 --2- 172.21.15.36:0/3810796332 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cac140260 0x7f5cac152d60 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7f5cac0b3960 tx=0x7f5ca8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:35.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.902+0000 7f5c9e7fc700 1 -- 172.21.15.36:0/3810796332 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ca8002f70 con 0x7f5cac140260 2022-06-17T12:56:35.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.902+0000 7f5c9e7fc700 1 -- 172.21.15.36:0/3810796332 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca800bec0 con 0x7f5cac140260 2022-06-17T12:56:35.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.902+0000 7f5cb25aa700 1 -- 172.21.15.36:0/3810796332 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cac0afa90 con 0x7f5cac140260 2022-06-17T12:56:35.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.902+0000 7f5c9e7fc700 1 -- 172.21.15.36:0/3810796332 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ca8011d70 con 0x7f5cac140260 2022-06-17T12:56:35.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.902+0000 7f5cb25aa700 1 -- 172.21.15.36:0/3810796332 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cac1533f0 con 0x7f5cac140260 2022-06-17T12:56:36.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.902+0000 7f5cb25aa700 1 -- 172.21.15.36:0/3810796332 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cac060c40 con 0x7f5cac140260 2022-06-17T12:56:36.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.902+0000 7f5c9e7fc700 1 -- 172.21.15.36:0/3810796332 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f5ca8025030 con 0x7f5cac140260 2022-06-17T12:56:36.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.906+0000 7f5c9e7fc700 1 --2- 172.21.15.36:0/3810796332 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c94053af0 0x7f5c94055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:36.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.906+0000 7f5c9e7fc700 1 -- 172.21.15.36:0/3810796332 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f5ca806d080 con 0x7f5cac140260 2022-06-17T12:56:36.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.906+0000 7f5cb15a8700 1 --2- 172.21.15.36:0/3810796332 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c94053af0 0x7f5c94055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:36.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.906+0000 7f5cb15a8700 1 --2- 172.21.15.36:0/3810796332 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c94053af0 0x7f5c94055fb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f5ca400d000 tx=0x7f5ca4007ef0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:36.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:35.910+0000 7f5c9e7fc700 1 -- 172.21.15.36:0/3810796332 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5ca8043e30 con 0x7f5cac140260 2022-06-17T12:56:36.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.074+0000 7f5cb25aa700 1 -- 172.21.15.36:0/3810796332 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5cac1533f0 con 0x7f5cac140260 2022-06-17T12:56:36.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.074+0000 7f5c9e7fc700 1 -- 172.21.15.36:0/3810796332 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+1782 (secure 0 0 0) 0x7f5cac1533f0 con 0x7f5cac140260 2022-06-17T12:56:36.066 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:36.066 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":73,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/2357210652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2357210652},{"type":"v1","addr":"172.21.15.36:6835","nonce":2357210652}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:56:11.021928+0000","modified":"2022-06-17T12:56:35.753950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-06-17T12:56:36.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.074+0000 7f5cb25aa700 1 -- 172.21.15.36:0/3810796332 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c94053af0 msgr2=0x7f5c94055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:36.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.074+0000 7f5cb25aa700 1 --2- 172.21.15.36:0/3810796332 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c94053af0 0x7f5c94055fb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f5ca400d000 tx=0x7f5ca4007ef0).stop 2022-06-17T12:56:36.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.074+0000 7f5cb25aa700 1 -- 172.21.15.36:0/3810796332 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cac140260 msgr2=0x7f5cac152d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:36.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.074+0000 7f5cb25aa700 1 --2- 172.21.15.36:0/3810796332 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cac140260 0x7f5cac152d60 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7f5cac0b3960 tx=0x7f5ca8015000).stop 2022-06-17T12:56:36.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.074+0000 7f5cb25aa700 1 -- 172.21.15.36:0/3810796332 shutdown_connections 2022-06-17T12:56:36.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.074+0000 7f5cb25aa700 1 --2- 172.21.15.36:0/3810796332 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c94053af0 0x7f5c94055fb0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:36.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.074+0000 7f5cb25aa700 1 --2- 172.21.15.36:0/3810796332 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cac140260 0x7f5cac152d60 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:36.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.074+0000 7f5cb25aa700 1 --2- 172.21.15.36:0/3810796332 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5cac0b3010 0x7f5cac152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:36.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.074+0000 7f5cb25aa700 1 --2- 172.21.15.36:0/3810796332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cac0b26e0 0x7f5cac13ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:36.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.074+0000 7f5cb25aa700 1 -- 172.21.15.36:0/3810796332 >> 172.21.15.36:0/3810796332 conn(0x7f5cac063550 msgr2=0x7f5cac0b5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:36.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.074+0000 7f5cb25aa700 1 -- 172.21.15.36:0/3810796332 shutdown_connections 2022-06-17T12:56:36.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.074+0000 7f5cb25aa700 1 -- 172.21.15.36:0/3810796332 wait complete. 2022-06-17T12:56:36.076 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 73 2022-06-17T12:56:36.097 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-17T12:56:36.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.270+0000 7f8d9bb34700 1 Processor -- start 2022-06-17T12:56:36.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.270+0000 7f8d9bb34700 1 -- start start 2022-06-17T12:56:36.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.270+0000 7f8d9bb34700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8d940b0b30 0x7f8d940b2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:36.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.270+0000 7f8d9bb34700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d94064800 0x7f8d9413ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:36.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.270+0000 7f8d9bb34700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d9413fe30 con 0x7f8d940672f0 2022-06-17T12:56:36.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.270+0000 7f8d9bb34700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d940b13b0 con 0x7f8d940b0b30 2022-06-17T12:56:36.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.270+0000 7f8d9bb34700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d940b1110 con 0x7f8d94064800 2022-06-17T12:56:36.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.270+0000 7f8d990cf700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8d940b0b30 0x7f8d940b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:36.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.270+0000 7f8d998d0700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f8d940672f0 0x7f8d940b1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57214/0 (socket says 172.21.15.36:57214) 2022-06-17T12:56:36.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.270+0000 7f8d998d0700 1 -- 172.21.15.36:0/206710823 learned_addr learned my addr 172.21.15.36:0/206710823 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:36.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.270+0000 7f8d9a0d1700 1 --2- 172.21.15.36:0/206710823 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d94064800 0x7f8d9413ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:36.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.270+0000 7f8d988ce700 1 -- 172.21.15.36:0/206710823 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2855524116 0 0) 0x7f8d9413fe30 con 0x7f8d940672f0 2022-06-17T12:56:36.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.270+0000 7f8d988ce700 1 -- 172.21.15.36:0/206710823 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d78003260 con 0x7f8d940672f0 2022-06-17T12:56:36.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.270+0000 7f8d988ce700 1 -- 172.21.15.36:0/206710823 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2272597873 0 0) 0x7f8d78003260 con 0x7f8d940672f0 2022-06-17T12:56:36.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.270+0000 7f8d990cf700 1 -- 172.21.15.36:0/206710823 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d94064800 msgr2=0x7f8d9413ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:36.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.270+0000 7f8d990cf700 1 --2- 172.21.15.36:0/206710823 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d94064800 0x7f8d9413ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:36.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.270+0000 7f8d990cf700 1 -- 172.21.15.36:0/206710823 >> v1:172.21.15.36:6789/0 conn(0x7f8d940672f0 legacy=0x7f8d940b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:36.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.270+0000 7f8d990cf700 1 -- 172.21.15.36:0/206710823 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d940b2550 con 0x7f8d940b0b30 2022-06-17T12:56:36.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.270+0000 7f8d990cf700 1 --2- 172.21.15.36:0/206710823 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8d940b0b30 0x7f8d940b2060 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 rx=0x7f8d88001800 tx=0x7f8d90006df0).ready entity=mon.0 client_cookie=83af037d1f624bc0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:36.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.270+0000 7f8d988ce700 1 -- 172.21.15.36:0/206710823 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d88001660 con 0x7f8d940b0b30 2022-06-17T12:56:36.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.270+0000 7f8d988ce700 1 -- 172.21.15.36:0/206710823 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d90000e90 con 0x7f8d940b0b30 2022-06-17T12:56:36.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.270+0000 7f8d988ce700 1 -- 172.21.15.36:0/206710823 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d90012460 con 0x7f8d940b0b30 2022-06-17T12:56:36.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.274+0000 7f8d9bb34700 1 -- 172.21.15.36:0/206710823 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8d940b0b30 msgr2=0x7f8d940b2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:36.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.274+0000 7f8d9bb34700 1 --2- 172.21.15.36:0/206710823 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8d940b0b30 0x7f8d940b2060 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 rx=0x7f8d88001800 tx=0x7f8d90006df0).stop 2022-06-17T12:56:36.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.274+0000 7f8d9bb34700 1 -- 172.21.15.36:0/206710823 shutdown_connections 2022-06-17T12:56:36.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.274+0000 7f8d9bb34700 1 --2- 172.21.15.36:0/206710823 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d94064800 0x7f8d9413ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:36.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.274+0000 7f8d9bb34700 1 --2- 172.21.15.36:0/206710823 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8d940b0b30 0x7f8d940b2060 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:36.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.274+0000 7f8d9bb34700 1 -- 172.21.15.36:0/206710823 >> 172.21.15.36:0/206710823 conn(0x7f8d940b66d0 msgr2=0x7f8d940b6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:36.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.274+0000 7f8d9bb34700 1 -- 172.21.15.36:0/206710823 shutdown_connections 2022-06-17T12:56:36.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.274+0000 7f8d9bb34700 1 -- 172.21.15.36:0/206710823 wait complete. 2022-06-17T12:56:36.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.274+0000 7f8d9bb34700 1 Processor -- start 2022-06-17T12:56:36.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.274+0000 7f8d9bb34700 1 -- start start 2022-06-17T12:56:36.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.274+0000 7f8d9bb34700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8d94064800 0x7f8d94125330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:36.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.274+0000 7f8d9bb34700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d940672f0 0x7f8d941292c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:36.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.274+0000 7f8d9bb34700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d940b0b30 0x7f8d94132100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:36.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.274+0000 7f8d9bb34700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d940af0e0 con 0x7f8d94064800 2022-06-17T12:56:36.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.274+0000 7f8d9bb34700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d940aee40 con 0x7f8d940b0b30 2022-06-17T12:56:36.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.274+0000 7f8d9bb34700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d941325f0 con 0x7f8d940672f0 2022-06-17T12:56:36.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.274+0000 7f8d998d0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8d94064800 0x7f8d94125330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:36.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.274+0000 7f8d998d0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8d94064800 0x7f8d94125330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39690/0 (socket says 172.21.15.36:39690) 2022-06-17T12:56:36.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.274+0000 7f8d998d0700 1 -- 172.21.15.36:0/1967744573 learned_addr learned my addr 172.21.15.36:0/1967744573 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:36.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.274+0000 7f8d990cf700 1 --2- 172.21.15.36:0/1967744573 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d940672f0 0x7f8d941292c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:36.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.274+0000 7f8d9a0d1700 1 --2- 172.21.15.36:0/1967744573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d940b0b30 0x7f8d94132100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:36.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.278+0000 7f8d998d0700 1 -- 172.21.15.36:0/1967744573 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d940672f0 msgr2=0x7f8d941292c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:36.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.278+0000 7f8d998d0700 1 --2- 172.21.15.36:0/1967744573 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d940672f0 0x7f8d941292c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:36.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.278+0000 7f8d998d0700 1 -- 172.21.15.36:0/1967744573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d940b0b30 msgr2=0x7f8d94132100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:36.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.278+0000 7f8d998d0700 1 --2- 172.21.15.36:0/1967744573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d940b0b30 0x7f8d94132100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:36.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.278+0000 7f8d998d0700 1 -- 172.21.15.36:0/1967744573 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9000b180 con 0x7f8d94064800 2022-06-17T12:56:36.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.278+0000 7f8d998d0700 1 --2- 172.21.15.36:0/1967744573 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8d94064800 0x7f8d94125330 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7f8d880018c0 tx=0x7f8d88001230).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:36.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.278+0000 7f8d86ffd700 1 -- 172.21.15.36:0/1967744573 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d88007170 con 0x7f8d94064800 2022-06-17T12:56:36.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.278+0000 7f8d86ffd700 1 -- 172.21.15.36:0/1967744573 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d880072d0 con 0x7f8d94064800 2022-06-17T12:56:36.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.278+0000 7f8d9bb34700 1 -- 172.21.15.36:0/1967744573 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d94125d70 con 0x7f8d94064800 2022-06-17T12:56:36.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.278+0000 7f8d86ffd700 1 -- 172.21.15.36:0/1967744573 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d88004920 con 0x7f8d94064800 2022-06-17T12:56:36.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.278+0000 7f8d9bb34700 1 -- 172.21.15.36:0/1967744573 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d940b49c0 con 0x7f8d94064800 2022-06-17T12:56:36.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.278+0000 7f8d9bb34700 1 -- 172.21.15.36:0/1967744573 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d94126090 con 0x7f8d94064800 2022-06-17T12:56:36.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.278+0000 7f8d86ffd700 1 -- 172.21.15.36:0/1967744573 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8d88004a80 con 0x7f8d94064800 2022-06-17T12:56:36.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.278+0000 7f8d86ffd700 1 --2- 172.21.15.36:0/1967744573 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8d7c053af0 0x7f8d7c055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:36.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.278+0000 7f8d86ffd700 1 -- 172.21.15.36:0/1967744573 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f8d8806bc80 con 0x7f8d94064800 2022-06-17T12:56:36.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.278+0000 7f8d990cf700 1 --2- 172.21.15.36:0/1967744573 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8d7c053af0 0x7f8d7c055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:36.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.282+0000 7f8d990cf700 1 --2- 172.21.15.36:0/1967744573 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8d7c053af0 0x7f8d7c055fb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f8d90018000 tx=0x7f8d90014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:36.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.286+0000 7f8d86ffd700 1 -- 172.21.15.36:0/1967744573 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8d880429d0 con 0x7f8d94064800 2022-06-17T12:56:36.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.446+0000 7f8d9bb34700 1 -- 172.21.15.36:0/1967744573 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8d94067ad0 con 0x7f8d94064800 2022-06-17T12:56:36.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.770+0000 7f8d86ffd700 1 -- 172.21.15.36:0/1967744573 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v74) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8d88045ff0 con 0x7f8d94064800 2022-06-17T12:56:36.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.774+0000 7f8d9bb34700 1 -- 172.21.15.36:0/1967744573 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8d7c053af0 msgr2=0x7f8d7c055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:36.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.774+0000 7f8d9bb34700 1 --2- 172.21.15.36:0/1967744573 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8d7c053af0 0x7f8d7c055fb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f8d90018000 tx=0x7f8d90014000).stop 2022-06-17T12:56:36.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.774+0000 7f8d9bb34700 1 -- 172.21.15.36:0/1967744573 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8d94064800 msgr2=0x7f8d94125330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:36.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.774+0000 7f8d9bb34700 1 --2- 172.21.15.36:0/1967744573 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8d94064800 0x7f8d94125330 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7f8d880018c0 tx=0x7f8d88001230).stop 2022-06-17T12:56:36.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.774+0000 7f8d9bb34700 1 -- 172.21.15.36:0/1967744573 shutdown_connections 2022-06-17T12:56:36.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.774+0000 7f8d9bb34700 1 --2- 172.21.15.36:0/1967744573 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8d7c053af0 0x7f8d7c055fb0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:36.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.774+0000 7f8d9bb34700 1 --2- 172.21.15.36:0/1967744573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d940b0b30 0x7f8d94132100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:36.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.774+0000 7f8d9bb34700 1 --2- 172.21.15.36:0/1967744573 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d940672f0 0x7f8d941292c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:36.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.774+0000 7f8d9bb34700 1 --2- 172.21.15.36:0/1967744573 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8d94064800 0x7f8d94125330 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:36.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.774+0000 7f8d9bb34700 1 -- 172.21.15.36:0/1967744573 >> 172.21.15.36:0/1967744573 conn(0x7f8d940b66d0 msgr2=0x7f8d94060d10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:36.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.774+0000 7f8d9bb34700 1 -- 172.21.15.36:0/1967744573 shutdown_connections 2022-06-17T12:56:36.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.774+0000 7f8d9bb34700 1 -- 172.21.15.36:0/1967744573 wait complete. 2022-06-17T12:56:36.778 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-17T12:56:36.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.922+0000 7fc8e986d700 1 Processor -- start 2022-06-17T12:56:36.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.922+0000 7fc8e986d700 1 -- start start 2022-06-17T12:56:36.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.922+0000 7fc8e986d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc8e40b3dc0 0x7fc8e40b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:36.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.922+0000 7fc8e986d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc8e40b4310 0x7fc8e4142210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:36.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.922+0000 7fc8e986d700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e40ba7a0 con 0x7fc8e40ba3c0 2022-06-17T12:56:36.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.922+0000 7fc8e986d700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e40b5610 con 0x7fc8e40b3dc0 2022-06-17T12:56:36.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.922+0000 7fc8e986d700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e40b50d0 con 0x7fc8e40b4310 2022-06-17T12:56:36.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.922+0000 7fc8dbfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc8e40b3dc0 0x7fc8e40b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:36.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.922+0000 7fc8dbfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc8e40b3dc0 0x7fc8e40b2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39700/0 (socket says 172.21.15.36:39700) 2022-06-17T12:56:36.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.922+0000 7fc8dbfff700 1 -- 172.21.15.36:0/482453788 learned_addr learned my addr 172.21.15.36:0/482453788 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:36.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.922+0000 7fc8e37fe700 1 --2- 172.21.15.36:0/482453788 >> v2:172.21.15.70:3300/0 conn(0x7fc8e40b4310 0x7fc8e4142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:36.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.922+0000 7fc8dbfff700 1 -- 172.21.15.36:0/482453788 >> v2:172.21.15.70:3300/0 conn(0x7fc8e40b4310 msgr2=0x7fc8e4142210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:36.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.922+0000 7fc8dbfff700 1 --2- 172.21.15.36:0/482453788 >> v2:172.21.15.70:3300/0 conn(0x7fc8e40b4310 0x7fc8e4142210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:36.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.922+0000 7fc8dbfff700 1 -- 172.21.15.36:0/482453788 >> v1:172.21.15.70:6789/0 conn(0x7fc8e40ba3c0 legacy=0x7fc8e40b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:36.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.922+0000 7fc8dbfff700 1 -- 172.21.15.36:0/482453788 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e40b5370 con 0x7fc8e40b3dc0 2022-06-17T12:56:36.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.922+0000 7fc8dbfff700 1 --2- 172.21.15.36:0/482453788 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8e40b3dc0 0x7fc8e40b2360 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7fc8cc002f70 tx=0x7fc8cc006eb0).ready entity=mon.0 client_cookie=e2c24cdcf983099e server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:36.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.922+0000 7fc8e27fc700 1 -- 172.21.15.36:0/482453788 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8cc00d5c0 con 0x7fc8e40b3dc0 2022-06-17T12:56:36.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.922+0000 7fc8e27fc700 1 -- 172.21.15.36:0/482453788 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8cc000e90 con 0x7fc8e40b3dc0 2022-06-17T12:56:36.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.922+0000 7fc8e27fc700 1 -- 172.21.15.36:0/482453788 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8cc00d950 con 0x7fc8e40b3dc0 2022-06-17T12:56:36.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.926+0000 7fc8e986d700 1 -- 172.21.15.36:0/482453788 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8e40b3dc0 msgr2=0x7fc8e40b2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:36.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.926+0000 7fc8e986d700 1 --2- 172.21.15.36:0/482453788 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8e40b3dc0 0x7fc8e40b2360 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7fc8cc002f70 tx=0x7fc8cc006eb0).stop 2022-06-17T12:56:36.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.926+0000 7fc8e986d700 1 -- 172.21.15.36:0/482453788 shutdown_connections 2022-06-17T12:56:36.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.926+0000 7fc8e986d700 1 --2- 172.21.15.36:0/482453788 >> v2:172.21.15.70:3300/0 conn(0x7fc8e40b4310 0x7fc8e4142210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:36.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.926+0000 7fc8e986d700 1 --2- 172.21.15.36:0/482453788 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8e40b3dc0 0x7fc8e40b2360 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:36.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.926+0000 7fc8e986d700 1 -- 172.21.15.36:0/482453788 >> 172.21.15.36:0/482453788 conn(0x7fc8e4067650 msgr2=0x7fc8e40b9970 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:36.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.926+0000 7fc8e986d700 1 -- 172.21.15.36:0/482453788 shutdown_connections 2022-06-17T12:56:36.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.926+0000 7fc8e986d700 1 -- 172.21.15.36:0/482453788 wait complete. 2022-06-17T12:56:36.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.926+0000 7fc8e986d700 1 Processor -- start 2022-06-17T12:56:36.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.926+0000 7fc8e986d700 1 -- start start 2022-06-17T12:56:36.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.926+0000 7fc8e986d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8e40b3dc0 0x7fc8e4155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:36.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.926+0000 7fc8e986d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8e40b4310 0x7fc8e40b8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:36.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.926+0000 7fc8e986d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e40ba3c0 0x7fc8e4147010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:36.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.926+0000 7fc8e986d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e41458f0 con 0x7fc8e40b3dc0 2022-06-17T12:56:36.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.926+0000 7fc8e986d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e40af1b0 con 0x7fc8e40ba3c0 2022-06-17T12:56:36.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.926+0000 7fc8e986d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e40af450 con 0x7fc8e40b4310 2022-06-17T12:56:36.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.930+0000 7fc8e2ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8e40b3dc0 0x7fc8e4155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:36.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.930+0000 7fc8dbfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8e40b4310 0x7fc8e40b8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:36.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.930+0000 7fc8e2ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8e40b3dc0 0x7fc8e4155bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39704/0 (socket says 172.21.15.36:39704) 2022-06-17T12:56:36.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.930+0000 7fc8e2ffd700 1 -- 172.21.15.36:0/2417006508 learned_addr learned my addr 172.21.15.36:0/2417006508 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:36.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.930+0000 7fc8e37fe700 1 --2- 172.21.15.36:0/2417006508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e40ba3c0 0x7fc8e4147010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:36.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.930+0000 7fc8e2ffd700 1 -- 172.21.15.36:0/2417006508 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8e40b4310 msgr2=0x7fc8e40b8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:36.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.930+0000 7fc8e2ffd700 1 --2- 172.21.15.36:0/2417006508 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8e40b4310 0x7fc8e40b8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:36.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.930+0000 7fc8e2ffd700 1 -- 172.21.15.36:0/2417006508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e40ba3c0 msgr2=0x7fc8e4147010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:36.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.930+0000 7fc8e2ffd700 1 --2- 172.21.15.36:0/2417006508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e40ba3c0 0x7fc8e4147010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:36.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.930+0000 7fc8e2ffd700 1 -- 172.21.15.36:0/2417006508 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8cc00b1e0 con 0x7fc8e40b3dc0 2022-06-17T12:56:36.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.930+0000 7fc8dbfff700 1 --2- 172.21.15.36:0/2417006508 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8e40b4310 0x7fc8e40b8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:36.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.930+0000 7fc8e2ffd700 1 --2- 172.21.15.36:0/2417006508 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8e40b3dc0 0x7fc8e4155bf0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x7fc8e40b2080 tx=0x7fc8d4007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:36.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.930+0000 7fc8e0ff9700 1 -- 172.21.15.36:0/2417006508 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8d4007e70 con 0x7fc8e40b3dc0 2022-06-17T12:56:36.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.930+0000 7fc8e0ff9700 1 -- 172.21.15.36:0/2417006508 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d4007030 con 0x7fc8e40b3dc0 2022-06-17T12:56:36.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.930+0000 7fc8e986d700 1 -- 172.21.15.36:0/2417006508 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e40af6f0 con 0x7fc8e40b3dc0 2022-06-17T12:56:36.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.930+0000 7fc8e0ff9700 1 -- 172.21.15.36:0/2417006508 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8d4007390 con 0x7fc8e40b3dc0 2022-06-17T12:56:36.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.930+0000 7fc8e986d700 1 -- 172.21.15.36:0/2417006508 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e4144b10 con 0x7fc8e40b3dc0 2022-06-17T12:56:36.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.930+0000 7fc8e986d700 1 -- 172.21.15.36:0/2417006508 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e40aff80 con 0x7fc8e40b3dc0 2022-06-17T12:56:36.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.930+0000 7fc8e0ff9700 1 -- 172.21.15.36:0/2417006508 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fc8d4010030 con 0x7fc8e40b3dc0 2022-06-17T12:56:36.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.934+0000 7fc8e0ff9700 1 --2- 172.21.15.36:0/2417006508 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc8bc053920 0x7fc8bc055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:36.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.934+0000 7fc8e0ff9700 1 -- 172.21.15.36:0/2417006508 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fc8d406bc20 con 0x7fc8e40b3dc0 2022-06-17T12:56:36.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.934+0000 7fc8dbfff700 1 --2- 172.21.15.36:0/2417006508 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc8bc053920 0x7fc8bc055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:36.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.934+0000 7fc8dbfff700 1 --2- 172.21.15.36:0/2417006508 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc8bc053920 0x7fc8bc055de0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7fc8cc01c000 tx=0x7fc8cc015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:36.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:36.938+0000 7fc8e0ff9700 1 -- 172.21.15.36:0/2417006508 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc8e40aff80 con 0x7fc8e40b3dc0 2022-06-17T12:56:37.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.062+0000 7fc8e986d700 1 -- 172.21.15.36:0/2417006508 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc8e40aff80 con 0x7fc8e40b3dc0 2022-06-17T12:56:37.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.778+0000 7fc8e0ff9700 1 -- 172.21.15.36:0/2417006508 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v133) v1 ==== 185+0+0 (secure 0 0 0) 0x7fc8e40aff80 con 0x7fc8e40b3dc0 2022-06-17T12:56:37.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.782+0000 7fc8e986d700 1 -- 172.21.15.36:0/2417006508 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc8bc053920 msgr2=0x7fc8bc055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:37.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.782+0000 7fc8e986d700 1 --2- 172.21.15.36:0/2417006508 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc8bc053920 0x7fc8bc055de0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7fc8cc01c000 tx=0x7fc8cc015000).stop 2022-06-17T12:56:37.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.782+0000 7fc8e986d700 1 -- 172.21.15.36:0/2417006508 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8e40b3dc0 msgr2=0x7fc8e4155bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:37.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.782+0000 7fc8e986d700 1 --2- 172.21.15.36:0/2417006508 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8e40b3dc0 0x7fc8e4155bf0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x7fc8e40b2080 tx=0x7fc8d4007c00).stop 2022-06-17T12:56:37.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.782+0000 7fc8e986d700 1 -- 172.21.15.36:0/2417006508 shutdown_connections 2022-06-17T12:56:37.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.782+0000 7fc8e986d700 1 --2- 172.21.15.36:0/2417006508 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc8bc053920 0x7fc8bc055de0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:37.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.782+0000 7fc8e986d700 1 --2- 172.21.15.36:0/2417006508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e40ba3c0 0x7fc8e4147010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:37.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.782+0000 7fc8e986d700 1 --2- 172.21.15.36:0/2417006508 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc8e40b4310 0x7fc8e40b8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:37.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.782+0000 7fc8e986d700 1 --2- 172.21.15.36:0/2417006508 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc8e40b3dc0 0x7fc8e4155bf0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:37.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.782+0000 7fc8e986d700 1 -- 172.21.15.36:0/2417006508 >> 172.21.15.36:0/2417006508 conn(0x7fc8e4067650 msgr2=0x7fc8e40b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:37.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.782+0000 7fc8e986d700 1 -- 172.21.15.36:0/2417006508 shutdown_connections 2022-06-17T12:56:37.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.782+0000 7fc8e986d700 1 -- 172.21.15.36:0/2417006508 wait complete. 2022-06-17T12:56:37.780 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' removed 2022-06-17T12:56:37.783 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-17T12:56:37.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.918+0000 7f33ed6f5700 1 Processor -- start 2022-06-17T12:56:37.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.918+0000 7f33ed6f5700 1 -- start start 2022-06-17T12:56:37.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.918+0000 7f33ed6f5700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33e80b56e0 0x7f33e813ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:37.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.918+0000 7f33ed6f5700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f33e80affd0 0x7f33e813cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:37.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.918+0000 7f33ed6f5700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f33e81466e0 0x7f33e80aed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:37.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.918+0000 7f33ed6f5700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33e8144320 con 0x7f33e81466e0 2022-06-17T12:56:37.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.918+0000 7f33ed6f5700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33e80b1260 con 0x7f33e80affd0 2022-06-17T12:56:37.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.918+0000 7f33ed6f5700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e80b1500 con 0x7f33e80b56e0 2022-06-17T12:56:37.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.918+0000 7f33e77fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f33e81466e0 0x7f33e80aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:37.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.918+0000 7f33e77fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f33e81466e0 0x7f33e80aed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39716/0 (socket says 172.21.15.36:39716) 2022-06-17T12:56:37.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.918+0000 7f33e77fe700 1 -- 172.21.15.36:0/2900271345 learned_addr learned my addr 172.21.15.36:0/2900271345 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:37.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.918+0000 7f33e67fc700 1 --2- 172.21.15.36:0/2900271345 >> v2:172.21.15.70:3300/0 conn(0x7f33e80affd0 0x7f33e813cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:37.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.918+0000 7f33e6ffd700 1 --2- 172.21.15.36:0/2900271345 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33e80b56e0 0x7f33e813ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:37.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.918+0000 7f33e77fe700 1 -- 172.21.15.36:0/2900271345 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33e80b56e0 msgr2=0x7f33e813ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:37.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.918+0000 7f33e77fe700 1 --2- 172.21.15.36:0/2900271345 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33e80b56e0 0x7f33e813ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:37.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.918+0000 7f33e77fe700 1 -- 172.21.15.36:0/2900271345 >> v2:172.21.15.70:3300/0 conn(0x7f33e80affd0 msgr2=0x7f33e813cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:37.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.918+0000 7f33e77fe700 1 --2- 172.21.15.36:0/2900271345 >> v2:172.21.15.70:3300/0 conn(0x7f33e80affd0 0x7f33e813cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:37.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.918+0000 7f33e77fe700 1 -- 172.21.15.36:0/2900271345 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e80b17a0 con 0x7f33e81466e0 2022-06-17T12:56:37.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.918+0000 7f33e77fe700 1 --2- 172.21.15.36:0/2900271345 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33e81466e0 0x7f33e80aed30 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7f33e0002f70 tx=0x7f33e0006eb0).ready entity=mon.0 client_cookie=ff288a8f00a31233 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:37.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.918+0000 7f33e5ffb700 1 -- 172.21.15.36:0/2900271345 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33e000d570 con 0x7f33e81466e0 2022-06-17T12:56:37.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.918+0000 7f33e5ffb700 1 -- 172.21.15.36:0/2900271345 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e0000e90 con 0x7f33e81466e0 2022-06-17T12:56:37.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.918+0000 7f33e5ffb700 1 -- 172.21.15.36:0/2900271345 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33e000d900 con 0x7f33e81466e0 2022-06-17T12:56:37.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.922+0000 7f33ed6f5700 1 -- 172.21.15.36:0/2900271345 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33e81466e0 msgr2=0x7f33e80aed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:37.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.922+0000 7f33ed6f5700 1 --2- 172.21.15.36:0/2900271345 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33e81466e0 0x7f33e80aed30 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7f33e0002f70 tx=0x7f33e0006eb0).stop 2022-06-17T12:56:37.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.922+0000 7f33ed6f5700 1 -- 172.21.15.36:0/2900271345 shutdown_connections 2022-06-17T12:56:37.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.922+0000 7f33ed6f5700 1 --2- 172.21.15.36:0/2900271345 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33e81466e0 0x7f33e80aed30 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:37.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.922+0000 7f33ed6f5700 1 --2- 172.21.15.36:0/2900271345 >> v2:172.21.15.70:3300/0 conn(0x7f33e80affd0 0x7f33e813cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:37.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.922+0000 7f33ed6f5700 1 --2- 172.21.15.36:0/2900271345 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33e80b56e0 0x7f33e813ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:37.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.922+0000 7f33ed6f5700 1 -- 172.21.15.36:0/2900271345 >> 172.21.15.36:0/2900271345 conn(0x7f33e8062630 msgr2=0x7f33e8062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:37.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.922+0000 7f33ed6f5700 1 -- 172.21.15.36:0/2900271345 shutdown_connections 2022-06-17T12:56:37.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.922+0000 7f33ed6f5700 1 -- 172.21.15.36:0/2900271345 wait complete. 2022-06-17T12:56:37.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.922+0000 7f33ed6f5700 1 Processor -- start 2022-06-17T12:56:37.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.922+0000 7f33ed6f5700 1 -- start start 2022-06-17T12:56:37.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.922+0000 7f33ed6f5700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33e80affd0 0x7f33e8148650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:37.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.922+0000 7f33ed6f5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33e80b56e0 0x7f33e81380f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:37.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.926+0000 7f33ed6f5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33e81466e0 0x7f33e8155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:37.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.926+0000 7f33ed6f5700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e813da00 con 0x7f33e81466e0 2022-06-17T12:56:37.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.926+0000 7f33ed6f5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e8147920 con 0x7f33e80b56e0 2022-06-17T12:56:37.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.926+0000 7f33ed6f5700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e8147bc0 con 0x7f33e80affd0 2022-06-17T12:56:37.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.926+0000 7f33e77fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33e81466e0 0x7f33e8155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:37.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.926+0000 7f33e77fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33e81466e0 0x7f33e8155840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39722/0 (socket says 172.21.15.36:39722) 2022-06-17T12:56:37.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.926+0000 7f33e6ffd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33e80affd0 0x7f33e8148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:37.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.926+0000 7f33e77fe700 1 -- 172.21.15.36:0/1246693127 learned_addr learned my addr 172.21.15.36:0/1246693127 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:37.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.926+0000 7f33e67fc700 1 --2- 172.21.15.36:0/1246693127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33e80b56e0 0x7f33e81380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:37.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.926+0000 7f33e77fe700 1 -- 172.21.15.36:0/1246693127 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33e80affd0 msgr2=0x7f33e8148650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:37.942 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.926+0000 7f33e77fe700 1 --2- 172.21.15.36:0/1246693127 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33e80affd0 0x7f33e8148650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:37.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.926+0000 7f33e77fe700 1 -- 172.21.15.36:0/1246693127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33e80b56e0 msgr2=0x7f33e81380f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:37.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.926+0000 7f33e77fe700 1 --2- 172.21.15.36:0/1246693127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33e80b56e0 0x7f33e81380f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:37.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.926+0000 7f33e77fe700 1 -- 172.21.15.36:0/1246693127 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e000b1e0 con 0x7f33e81466e0 2022-06-17T12:56:37.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.926+0000 7f33e77fe700 1 --2- 172.21.15.36:0/1246693127 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33e81466e0 0x7f33e8155840 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 rx=0x7f33e0002f70 tx=0x7f33e0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:37.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.926+0000 7f33dffff700 1 -- 172.21.15.36:0/1246693127 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33e0011c30 con 0x7f33e81466e0 2022-06-17T12:56:37.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.926+0000 7f33dffff700 1 -- 172.21.15.36:0/1246693127 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e0011d90 con 0x7f33e81466e0 2022-06-17T12:56:37.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.926+0000 7f33ed6f5700 1 -- 172.21.15.36:0/1246693127 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e813aaf0 con 0x7f33e81466e0 2022-06-17T12:56:37.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.926+0000 7f33dffff700 1 -- 172.21.15.36:0/1246693127 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33e001e650 con 0x7f33e81466e0 2022-06-17T12:56:37.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.926+0000 7f33ed6f5700 1 -- 172.21.15.36:0/1246693127 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e813dfd0 con 0x7f33e81466e0 2022-06-17T12:56:37.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.926+0000 7f33ed6f5700 1 -- 172.21.15.36:0/1246693127 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e813db70 con 0x7f33e81466e0 2022-06-17T12:56:37.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.926+0000 7f33dffff700 1 -- 172.21.15.36:0/1246693127 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f33e0025030 con 0x7f33e81466e0 2022-06-17T12:56:37.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.930+0000 7f33dffff700 1 --2- 172.21.15.36:0/1246693127 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f33cc0539d0 0x7f33cc055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:37.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.930+0000 7f33e6ffd700 1 --2- 172.21.15.36:0/1246693127 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f33cc0539d0 0x7f33cc055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:37.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.930+0000 7f33dffff700 1 -- 172.21.15.36:0/1246693127 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f33e006cff0 con 0x7f33e81466e0 2022-06-17T12:56:37.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.930+0000 7f33e6ffd700 1 --2- 172.21.15.36:0/1246693127 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f33cc0539d0 0x7f33cc055e90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f33d000d000 tx=0x7f33d0008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:37.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:37.934+0000 7f33dffff700 1 -- 172.21.15.36:0/1246693127 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f33e813db70 con 0x7f33e81466e0 2022-06-17T12:56:38.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:38.062+0000 7f33ed6f5700 1 -- 172.21.15.36:0/1246693127 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f33e813db40 con 0x7f33e81466e0 2022-06-17T12:56:38.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:38.786+0000 7f33dffff700 1 -- 172.21.15.36:0/1246693127 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v134) v1 ==== 173+0+0 (secure 0 0 0) 0x7f33e0023070 con 0x7f33e81466e0 2022-06-17T12:56:38.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:38.790+0000 7f33ed6f5700 1 -- 172.21.15.36:0/1246693127 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f33cc0539d0 msgr2=0x7f33cc055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:38.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:38.790+0000 7f33ed6f5700 1 --2- 172.21.15.36:0/1246693127 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f33cc0539d0 0x7f33cc055e90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f33d000d000 tx=0x7f33d0008040).stop 2022-06-17T12:56:38.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:38.790+0000 7f33ed6f5700 1 -- 172.21.15.36:0/1246693127 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33e81466e0 msgr2=0x7f33e8155840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:38.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:38.790+0000 7f33ed6f5700 1 --2- 172.21.15.36:0/1246693127 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33e81466e0 0x7f33e8155840 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 rx=0x7f33e0002f70 tx=0x7f33e0015000).stop 2022-06-17T12:56:38.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:38.790+0000 7f33ed6f5700 1 -- 172.21.15.36:0/1246693127 shutdown_connections 2022-06-17T12:56:38.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:38.790+0000 7f33ed6f5700 1 --2- 172.21.15.36:0/1246693127 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f33cc0539d0 0x7f33cc055e90 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:38.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:38.790+0000 7f33ed6f5700 1 --2- 172.21.15.36:0/1246693127 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f33e81466e0 0x7f33e8155840 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:38.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:38.790+0000 7f33ed6f5700 1 --2- 172.21.15.36:0/1246693127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33e80b56e0 0x7f33e81380f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:38.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:38.790+0000 7f33ed6f5700 1 --2- 172.21.15.36:0/1246693127 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33e80affd0 0x7f33e8148650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:38.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:38.790+0000 7f33ed6f5700 1 -- 172.21.15.36:0/1246693127 >> 172.21.15.36:0/1246693127 conn(0x7f33e8062630 msgr2=0x7f33e813bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:38.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:38.790+0000 7f33ed6f5700 1 -- 172.21.15.36:0/1246693127 shutdown_connections 2022-06-17T12:56:38.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:38.790+0000 7f33ed6f5700 1 -- 172.21.15.36:0/1246693127 wait complete. 2022-06-17T12:56:38.790 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' removed 2022-06-17T12:56:38.795 DEBUG:teuthology.orchestra.run.smithi036:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-17T12:56:38.803 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-17T12:56:38.810 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-17T12:56:38.823 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-17T12:56:38.826 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-17T12:56:38.833 INFO:teuthology.orchestra.run.smithi036.stdout:[global] 2022-06-17T12:56:38.834 INFO:teuthology.orchestra.run.smithi036.stdout: chdir = "" 2022-06-17T12:56:38.834 INFO:teuthology.orchestra.run.smithi036.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-17T12:56:38.835 INFO:teuthology.orchestra.run.smithi036.stdout: auth supported = cephx 2022-06-17T12:56:38.835 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.835 INFO:teuthology.orchestra.run.smithi036.stdout: filestore xattr use omap = true 2022-06-17T12:56:38.836 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.836 INFO:teuthology.orchestra.run.smithi036.stdout: mon clock drift allowed = 1.000 2022-06-17T12:56:38.837 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.837 INFO:teuthology.orchestra.run.smithi036.stdout: osd crush chooseleaf type = 0 2022-06-17T12:56:38.837 INFO:teuthology.orchestra.run.smithi036.stdout: auth debug = true 2022-06-17T12:56:38.837 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.838 INFO:teuthology.orchestra.run.smithi036.stdout: ms die on old message = true 2022-06-17T12:56:38.838 INFO:teuthology.orchestra.run.smithi036.stdout: ms die on bug = true 2022-06-17T12:56:38.838 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.839 INFO:teuthology.orchestra.run.smithi036.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-17T12:56:38.839 INFO:teuthology.orchestra.run.smithi036.stdout: mon pg warn max object skew = 0 2022-06-17T12:56:38.840 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.840 INFO:teuthology.orchestra.run.smithi036.stdout: # disable pg_autoscaler by default for new pools 2022-06-17T12:56:38.841 INFO:teuthology.orchestra.run.smithi036.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-17T12:56:38.844 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.845 INFO:teuthology.orchestra.run.smithi036.stdout: osd pool default size = 2 2022-06-17T12:56:38.846 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.846 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd allow primary affinity = true 2022-06-17T12:56:38.847 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd allow pg remap = true 2022-06-17T12:56:38.847 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on legacy crush tunables = false 2022-06-17T12:56:38.847 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on crush straw calc version zero = false 2022-06-17T12:56:38.848 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on no sortbitwise = false 2022-06-17T12:56:38.848 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on osd down out interval zero = false 2022-06-17T12:56:38.848 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on too few osds = false 2022-06-17T12:56:38.849 INFO:teuthology.orchestra.run.smithi036.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-17T12:56:38.850 INFO:teuthology.orchestra.run.smithi036.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-17T12:56:38.850 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.851 INFO:teuthology.orchestra.run.smithi036.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-17T12:56:38.851 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.851 INFO:teuthology.orchestra.run.smithi036.stdout: osd default data pool replay window = 5 2022-06-17T12:56:38.852 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.852 INFO:teuthology.orchestra.run.smithi036.stdout: mon allow pool delete = true 2022-06-17T12:56:38.853 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.853 INFO:teuthology.orchestra.run.smithi036.stdout: mon cluster log file level = debug 2022-06-17T12:56:38.853 INFO:teuthology.orchestra.run.smithi036.stdout: debug asserts on shutdown = true 2022-06-17T12:56:38.854 INFO:teuthology.orchestra.run.smithi036.stdout: mon health detail to clog = false 2022-06-17T12:56:38.854 INFO:teuthology.orchestra.run.smithi036.stdout: mon host = "172.21.15.36,172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790]" 2022-06-17T12:56:38.854 INFO:teuthology.orchestra.run.smithi036.stdout: fsid = b68b9700-0c85-42ca-94d7-6cd53db91c14 2022-06-17T12:56:38.855 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.855 INFO:teuthology.orchestra.run.smithi036.stdout:[osd] 2022-06-17T12:56:38.855 INFO:teuthology.orchestra.run.smithi036.stdout: osd journal size = 100 2022-06-17T12:56:38.857 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.857 INFO:teuthology.orchestra.run.smithi036.stdout: osd scrub load threshold = 5.0 2022-06-17T12:56:38.857 INFO:teuthology.orchestra.run.smithi036.stdout: osd scrub max interval = 600 2022-06-17T12:56:38.858 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.859 INFO:teuthology.orchestra.run.smithi036.stdout: osd recover clone overlap = true 2022-06-17T12:56:38.859 INFO:teuthology.orchestra.run.smithi036.stdout: osd recovery max chunk = 1048576 2022-06-17T12:56:38.860 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.860 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug shutdown = true 2022-06-17T12:56:38.860 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug op order = true 2022-06-17T12:56:38.861 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug verify stray on activate = true 2022-06-17T12:56:38.861 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.862 INFO:teuthology.orchestra.run.smithi036.stdout: osd open classes on start = true 2022-06-17T12:56:38.862 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug pg log writeout = true 2022-06-17T12:56:38.863 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.864 INFO:teuthology.orchestra.run.smithi036.stdout: osd deep scrub update digest min age = 30 2022-06-17T12:56:38.864 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.873 INFO:teuthology.orchestra.run.smithi036.stdout: osd map max advance = 10 2022-06-17T12:56:38.874 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.874 INFO:teuthology.orchestra.run.smithi036.stdout: journal zero on create = true 2022-06-17T12:56:38.875 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.875 INFO:teuthology.orchestra.run.smithi036.stdout: filestore ondisk finisher threads = 3 2022-06-17T12:56:38.876 INFO:teuthology.orchestra.run.smithi036.stdout: filestore apply finisher threads = 3 2022-06-17T12:56:38.876 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.876 INFO:teuthology.orchestra.run.smithi036.stdout: bdev debug aio = true 2022-06-17T12:56:38.877 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug misdirected ops = true 2022-06-17T12:56:38.877 INFO:teuthology.orchestra.run.smithi036.stdout: bluestore block size = 96636764160 2022-06-17T12:56:38.877 INFO:teuthology.orchestra.run.smithi036.stdout: bluestore compression mode = aggressive 2022-06-17T12:56:38.878 INFO:teuthology.orchestra.run.smithi036.stdout: bluestore fsck on mount = True 2022-06-17T12:56:38.878 INFO:teuthology.orchestra.run.smithi036.stdout: debug bluefs = 20 2022-06-17T12:56:38.879 INFO:teuthology.orchestra.run.smithi036.stdout: debug bluestore = 20 2022-06-17T12:56:38.879 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2022-06-17T12:56:38.879 INFO:teuthology.orchestra.run.smithi036.stdout: debug osd = 20 2022-06-17T12:56:38.880 INFO:teuthology.orchestra.run.smithi036.stdout: debug rocksdb = 10 2022-06-17T12:56:38.880 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd backfillfull_ratio = 0.85 2022-06-17T12:56:38.881 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd full ratio = 0.9 2022-06-17T12:56:38.881 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd nearfull ratio = 0.8 2022-06-17T12:56:38.889 INFO:teuthology.orchestra.run.smithi036.stdout: osd failsafe full ratio = 0.95 2022-06-17T12:56:38.889 INFO:teuthology.orchestra.run.smithi036.stdout: osd objectstore = bluestore 2022-06-17T12:56:38.890 INFO:teuthology.orchestra.run.smithi036.stdout: osd op complaint time = 180 2022-06-17T12:56:38.890 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.891 INFO:teuthology.orchestra.run.smithi036.stdout:[mgr] 2022-06-17T12:56:38.891 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2022-06-17T12:56:38.892 INFO:teuthology.orchestra.run.smithi036.stdout: debug mgr = 20 2022-06-17T12:56:38.892 INFO:teuthology.orchestra.run.smithi036.stdout: debug mon = 20 2022-06-17T12:56:38.892 INFO:teuthology.orchestra.run.smithi036.stdout: debug auth = 20 2022-06-17T12:56:38.893 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:56:38.893 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:56:38.894 INFO:teuthology.orchestra.run.smithi036.stdout: mgr/telemetry/nag = false 2022-06-17T12:56:38.894 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.900 INFO:teuthology.orchestra.run.smithi036.stdout:[mon] 2022-06-17T12:56:38.900 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2022-06-17T12:56:38.901 INFO:teuthology.orchestra.run.smithi036.stdout: debug mon = 20 2022-06-17T12:56:38.902 INFO:teuthology.orchestra.run.smithi036.stdout: debug paxos = 20 2022-06-17T12:56:38.902 INFO:teuthology.orchestra.run.smithi036.stdout: debug auth = 20 2022-06-17T12:56:38.903 INFO:teuthology.orchestra.run.smithi036.stdout: mon data avail warn = 5 2022-06-17T12:56:38.903 INFO:teuthology.orchestra.run.smithi036.stdout: mon mgr mkfs grace = 240 2022-06-17T12:56:38.904 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:56:38.904 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd reporter subtree level = osd 2022-06-17T12:56:38.905 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd prime pg temp = true 2022-06-17T12:56:38.905 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:56:38.905 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.906 INFO:teuthology.orchestra.run.smithi036.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-17T12:56:38.907 INFO:teuthology.orchestra.run.smithi036.stdout: auth mon ticket ttl = 660 # 11m 2022-06-17T12:56:38.907 INFO:teuthology.orchestra.run.smithi036.stdout: auth service ticket ttl = 240 # 4m 2022-06-17T12:56:38.907 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.908 INFO:teuthology.orchestra.run.smithi036.stdout: # don't complain about insecure global_id in the test suite 2022-06-17T12:56:38.908 INFO:teuthology.orchestra.run.smithi036.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-17T12:56:38.909 INFO:teuthology.orchestra.run.smithi036.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-17T12:56:38.909 INFO:teuthology.orchestra.run.smithi036.stdout: mon op complaint time = 120 2022-06-17T12:56:38.909 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:38.910 INFO:teuthology.orchestra.run.smithi036.stdout:[client] 2022-06-17T12:56:38.910 INFO:teuthology.orchestra.run.smithi036.stdout: rgw cache enabled = true 2022-06-17T12:56:38.910 INFO:teuthology.orchestra.run.smithi036.stdout: rgw enable ops log = true 2022-06-17T12:56:38.911 INFO:teuthology.orchestra.run.smithi036.stdout: rgw enable usage log = true 2022-06-17T12:56:38.911 INFO:teuthology.orchestra.run.smithi036.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-17T12:56:38.926 INFO:teuthology.orchestra.run.smithi036.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-17T12:56:38.926 INFO:teuthology.orchestra.run.smithi036.stdout: client mount timeout = 600 2022-06-17T12:56:38.926 INFO:teuthology.orchestra.run.smithi036.stdout: debug client = 20 2022-06-17T12:56:38.927 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2022-06-17T12:56:38.927 INFO:teuthology.orchestra.run.smithi036.stdout: rados mon op timeout = 15m 2022-06-17T12:56:38.927 INFO:teuthology.orchestra.run.smithi036.stdout: rados osd op timeout = 15m 2022-06-17T12:56:38.927 INFO:teuthology.orchestra.run.smithi036.stdout:[mon.a] 2022-06-17T12:56:38.928 INFO:teuthology.orchestra.run.smithi036.stdout:[mon.b] 2022-06-17T12:56:38.928 INFO:teuthology.orchestra.run.smithi036.stdout:[mon.c] 2022-06-17T12:56:38.928 INFO:teuthology.orchestra.run.smithi036.stdout:[mds.a] 2022-06-17T12:56:38.928 INFO:teuthology.orchestra.run.smithi036.stdout:[mds.b] 2022-06-17T12:56:38.928 INFO:teuthology.orchestra.run.smithi036.stdout:[mds] 2022-06-17T12:56:38.929 INFO:teuthology.orchestra.run.smithi036.stdout: debug mds = 20 2022-06-17T12:56:38.929 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2022-06-17T12:56:38.929 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal frag = True 2022-06-17T12:56:38.929 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal fragment size max = 10000 2022-06-17T12:56:38.930 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal merge size = 5 2022-06-17T12:56:38.930 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal split bits = 3 2022-06-17T12:56:38.930 INFO:teuthology.orchestra.run.smithi036.stdout: mds bal split size = 100 2022-06-17T12:56:38.930 INFO:teuthology.orchestra.run.smithi036.stdout: mds debug frag = True 2022-06-17T12:56:38.931 INFO:teuthology.orchestra.run.smithi036.stdout: mds debug scatterstat = True 2022-06-17T12:56:38.931 INFO:teuthology.orchestra.run.smithi036.stdout: mds op complaint time = 180 2022-06-17T12:56:38.932 INFO:teuthology.orchestra.run.smithi036.stdout: mds verify scatter = True 2022-06-17T12:56:38.932 INFO:teuthology.orchestra.run.smithi036.stdout: osd op complaint time = 180 2022-06-17T12:56:38.932 INFO:teuthology.orchestra.run.smithi036.stdout: rados mon op timeout = 15m 2022-06-17T12:56:38.932 INFO:teuthology.orchestra.run.smithi036.stdout: rados osd op timeout = 15m 2022-06-17T12:56:38.945 INFO:teuthology.orchestra.run.smithi143.stdout:[global] 2022-06-17T12:56:38.945 INFO:teuthology.orchestra.run.smithi143.stdout: chdir = "" 2022-06-17T12:56:38.945 INFO:teuthology.orchestra.run.smithi143.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-17T12:56:38.946 INFO:teuthology.orchestra.run.smithi143.stdout: auth supported = cephx 2022-06-17T12:56:38.946 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.946 INFO:teuthology.orchestra.run.smithi143.stdout: filestore xattr use omap = true 2022-06-17T12:56:38.947 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.947 INFO:teuthology.orchestra.run.smithi143.stdout: mon clock drift allowed = 1.000 2022-06-17T12:56:38.947 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.947 INFO:teuthology.orchestra.run.smithi143.stdout: osd crush chooseleaf type = 0 2022-06-17T12:56:38.948 INFO:teuthology.orchestra.run.smithi143.stdout: auth debug = true 2022-06-17T12:56:38.948 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.948 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on old message = true 2022-06-17T12:56:38.948 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on bug = true 2022-06-17T12:56:38.949 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.949 INFO:teuthology.orchestra.run.smithi143.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-17T12:56:38.949 INFO:teuthology.orchestra.run.smithi143.stdout: mon pg warn max object skew = 0 2022-06-17T12:56:38.949 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.950 INFO:teuthology.orchestra.run.smithi143.stdout: # disable pg_autoscaler by default for new pools 2022-06-17T12:56:38.950 INFO:teuthology.orchestra.run.smithi143.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-17T12:56:38.950 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.951 INFO:teuthology.orchestra.run.smithi143.stdout: osd pool default size = 2 2022-06-17T12:56:38.951 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.951 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd allow primary affinity = true 2022-06-17T12:56:38.951 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd allow pg remap = true 2022-06-17T12:56:38.952 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on legacy crush tunables = false 2022-06-17T12:56:38.952 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on crush straw calc version zero = false 2022-06-17T12:56:38.952 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on no sortbitwise = false 2022-06-17T12:56:38.954 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on osd down out interval zero = false 2022-06-17T12:56:38.954 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on too few osds = false 2022-06-17T12:56:38.955 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-17T12:56:38.955 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-17T12:56:38.955 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.955 INFO:teuthology.orchestra.run.smithi143.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-17T12:56:38.956 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.956 INFO:teuthology.orchestra.run.smithi143.stdout: osd default data pool replay window = 5 2022-06-17T12:56:38.956 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.956 INFO:teuthology.orchestra.run.smithi143.stdout: mon allow pool delete = true 2022-06-17T12:56:38.957 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.957 INFO:teuthology.orchestra.run.smithi143.stdout: mon cluster log file level = debug 2022-06-17T12:56:38.957 INFO:teuthology.orchestra.run.smithi143.stdout: debug asserts on shutdown = true 2022-06-17T12:56:38.957 INFO:teuthology.orchestra.run.smithi143.stdout: mon health detail to clog = false 2022-06-17T12:56:38.957 INFO:teuthology.orchestra.run.smithi143.stdout: mon host = "172.21.15.36,172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790]" 2022-06-17T12:56:38.958 INFO:teuthology.orchestra.run.smithi143.stdout: fsid = b68b9700-0c85-42ca-94d7-6cd53db91c14 2022-06-17T12:56:38.958 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.958 INFO:teuthology.orchestra.run.smithi143.stdout:[osd] 2022-06-17T12:56:38.958 INFO:teuthology.orchestra.run.smithi143.stdout: osd journal size = 100 2022-06-17T12:56:38.959 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.959 INFO:teuthology.orchestra.run.smithi143.stdout: osd scrub load threshold = 5.0 2022-06-17T12:56:38.959 INFO:teuthology.orchestra.run.smithi143.stdout: osd scrub max interval = 600 2022-06-17T12:56:38.959 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.960 INFO:teuthology.orchestra.run.smithi143.stdout: osd recover clone overlap = true 2022-06-17T12:56:38.960 INFO:teuthology.orchestra.run.smithi143.stdout: osd recovery max chunk = 1048576 2022-06-17T12:56:38.960 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.960 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug shutdown = true 2022-06-17T12:56:38.960 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug op order = true 2022-06-17T12:56:38.961 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug verify stray on activate = true 2022-06-17T12:56:38.961 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.966 INFO:teuthology.orchestra.run.smithi143.stdout: osd open classes on start = true 2022-06-17T12:56:38.970 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug pg log writeout = true 2022-06-17T12:56:38.970 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.970 INFO:teuthology.orchestra.run.smithi143.stdout: osd deep scrub update digest min age = 30 2022-06-17T12:56:38.977 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.977 INFO:teuthology.orchestra.run.smithi143.stdout: osd map max advance = 10 2022-06-17T12:56:38.978 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.978 INFO:teuthology.orchestra.run.smithi143.stdout: journal zero on create = true 2022-06-17T12:56:38.980 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.981 INFO:teuthology.orchestra.run.smithi143.stdout: filestore ondisk finisher threads = 3 2022-06-17T12:56:38.981 INFO:teuthology.orchestra.run.smithi143.stdout: filestore apply finisher threads = 3 2022-06-17T12:56:38.982 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.982 INFO:teuthology.orchestra.run.smithi143.stdout: bdev debug aio = true 2022-06-17T12:56:38.983 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug misdirected ops = true 2022-06-17T12:56:38.983 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore block size = 96636764160 2022-06-17T12:56:38.983 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore compression mode = aggressive 2022-06-17T12:56:38.984 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore fsck on mount = True 2022-06-17T12:56:38.984 INFO:teuthology.orchestra.run.smithi143.stdout: debug bluefs = 20 2022-06-17T12:56:38.985 INFO:teuthology.orchestra.run.smithi143.stdout: debug bluestore = 20 2022-06-17T12:56:38.985 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2022-06-17T12:56:38.985 INFO:teuthology.orchestra.run.smithi143.stdout: debug osd = 20 2022-06-17T12:56:38.985 INFO:teuthology.orchestra.run.smithi143.stdout: debug rocksdb = 10 2022-06-17T12:56:38.986 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd backfillfull_ratio = 0.85 2022-06-17T12:56:38.986 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd full ratio = 0.9 2022-06-17T12:56:38.987 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd nearfull ratio = 0.8 2022-06-17T12:56:38.987 INFO:teuthology.orchestra.run.smithi143.stdout: osd failsafe full ratio = 0.95 2022-06-17T12:56:38.987 INFO:teuthology.orchestra.run.smithi143.stdout: osd objectstore = bluestore 2022-06-17T12:56:38.988 INFO:teuthology.orchestra.run.smithi143.stdout: osd op complaint time = 180 2022-06-17T12:56:38.988 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.988 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr] 2022-06-17T12:56:38.988 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2022-06-17T12:56:38.989 INFO:teuthology.orchestra.run.smithi143.stdout: debug mgr = 20 2022-06-17T12:56:38.989 INFO:teuthology.orchestra.run.smithi143.stdout: debug mon = 20 2022-06-17T12:56:38.989 INFO:teuthology.orchestra.run.smithi143.stdout: debug auth = 20 2022-06-17T12:56:38.989 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:56:38.990 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:56:38.991 INFO:teuthology.orchestra.run.smithi143.stdout: mgr/telemetry/nag = false 2022-06-17T12:56:38.991 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.991 INFO:teuthology.orchestra.run.smithi143.stdout:[mon] 2022-06-17T12:56:38.991 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2022-06-17T12:56:38.992 INFO:teuthology.orchestra.run.smithi143.stdout: debug mon = 20 2022-06-17T12:56:38.992 INFO:teuthology.orchestra.run.smithi143.stdout: debug paxos = 20 2022-06-17T12:56:38.992 INFO:teuthology.orchestra.run.smithi143.stdout: debug auth = 20 2022-06-17T12:56:38.992 INFO:teuthology.orchestra.run.smithi143.stdout: mon data avail warn = 5 2022-06-17T12:56:38.993 INFO:teuthology.orchestra.run.smithi143.stdout: mon mgr mkfs grace = 240 2022-06-17T12:56:38.993 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:56:38.994 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd reporter subtree level = osd 2022-06-17T12:56:38.995 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd prime pg temp = true 2022-06-17T12:56:38.995 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:56:38.996 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.996 INFO:teuthology.orchestra.run.smithi143.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-17T12:56:38.997 INFO:teuthology.orchestra.run.smithi143.stdout: auth mon ticket ttl = 660 # 11m 2022-06-17T12:56:38.997 INFO:teuthology.orchestra.run.smithi143.stdout: auth service ticket ttl = 240 # 4m 2022-06-17T12:56:38.997 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:38.998 INFO:teuthology.orchestra.run.smithi143.stdout: # don't complain about insecure global_id in the test suite 2022-06-17T12:56:38.998 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-17T12:56:38.999 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-17T12:56:38.999 INFO:teuthology.orchestra.run.smithi143.stdout: mon op complaint time = 120 2022-06-17T12:56:38.999 INFO:teuthology.orchestra.run.smithi143.stdout: 2022-06-17T12:56:39.000 INFO:teuthology.orchestra.run.smithi143.stdout:[client] 2022-06-17T12:56:39.000 INFO:teuthology.orchestra.run.smithi143.stdout: rgw cache enabled = true 2022-06-17T12:56:39.000 INFO:teuthology.orchestra.run.smithi143.stdout: rgw enable ops log = true 2022-06-17T12:56:39.001 INFO:teuthology.orchestra.run.smithi143.stdout: rgw enable usage log = true 2022-06-17T12:56:39.002 INFO:teuthology.orchestra.run.smithi143.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-17T12:56:39.002 INFO:teuthology.orchestra.run.smithi143.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-17T12:56:39.003 INFO:teuthology.orchestra.run.smithi143.stdout: client mount timeout = 600 2022-06-17T12:56:39.003 INFO:teuthology.orchestra.run.smithi143.stdout: debug client = 20 2022-06-17T12:56:39.003 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2022-06-17T12:56:39.004 INFO:teuthology.orchestra.run.smithi143.stdout: rados mon op timeout = 15m 2022-06-17T12:56:39.004 INFO:teuthology.orchestra.run.smithi143.stdout: rados osd op timeout = 15m 2022-06-17T12:56:39.005 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.a] 2022-06-17T12:56:39.005 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.b] 2022-06-17T12:56:39.005 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.c] 2022-06-17T12:56:39.005 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.a] 2022-06-17T12:56:39.006 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.b] 2022-06-17T12:56:39.006 INFO:teuthology.orchestra.run.smithi143.stdout:[mds] 2022-06-17T12:56:39.007 INFO:teuthology.orchestra.run.smithi143.stdout: debug mds = 20 2022-06-17T12:56:39.007 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2022-06-17T12:56:39.008 INFO:teuthology.orchestra.run.smithi143.stdout: mds bal frag = True 2022-06-17T12:56:39.008 INFO:teuthology.orchestra.run.smithi143.stdout: mds bal fragment size max = 10000 2022-06-17T12:56:39.008 INFO:teuthology.orchestra.run.smithi143.stdout: mds bal merge size = 5 2022-06-17T12:56:39.009 INFO:teuthology.orchestra.run.smithi143.stdout: mds bal split bits = 3 2022-06-17T12:56:39.009 INFO:teuthology.orchestra.run.smithi143.stdout: mds bal split size = 100 2022-06-17T12:56:39.010 INFO:teuthology.orchestra.run.smithi143.stdout: mds debug frag = True 2022-06-17T12:56:39.010 INFO:teuthology.orchestra.run.smithi143.stdout: mds debug scatterstat = True 2022-06-17T12:56:39.011 INFO:teuthology.orchestra.run.smithi143.stdout: mds op complaint time = 180 2022-06-17T12:56:39.011 INFO:teuthology.orchestra.run.smithi143.stdout: mds verify scatter = True 2022-06-17T12:56:39.019 INFO:teuthology.orchestra.run.smithi143.stdout: osd op complaint time = 180 2022-06-17T12:56:39.019 INFO:teuthology.orchestra.run.smithi143.stdout: rados mon op timeout = 15m 2022-06-17T12:56:39.020 INFO:teuthology.orchestra.run.smithi143.stdout: rados osd op timeout = 15m 2022-06-17T12:56:39.033 INFO:teuthology.orchestra.run.smithi136.stdout:[global] 2022-06-17T12:56:39.034 INFO:teuthology.orchestra.run.smithi136.stdout: chdir = "" 2022-06-17T12:56:39.045 INFO:teuthology.orchestra.run.smithi136.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-17T12:56:39.045 INFO:teuthology.orchestra.run.smithi136.stdout: auth supported = cephx 2022-06-17T12:56:39.045 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.045 INFO:teuthology.orchestra.run.smithi136.stdout: filestore xattr use omap = true 2022-06-17T12:56:39.046 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.046 INFO:teuthology.orchestra.run.smithi136.stdout: mon clock drift allowed = 1.000 2022-06-17T12:56:39.053 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.053 INFO:teuthology.orchestra.run.smithi136.stdout: osd crush chooseleaf type = 0 2022-06-17T12:56:39.054 INFO:teuthology.orchestra.run.smithi136.stdout: auth debug = true 2022-06-17T12:56:39.054 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.054 INFO:teuthology.orchestra.run.smithi136.stdout: ms die on old message = true 2022-06-17T12:56:39.055 INFO:teuthology.orchestra.run.smithi136.stdout: ms die on bug = true 2022-06-17T12:56:39.055 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.056 INFO:teuthology.orchestra.run.smithi136.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-17T12:56:39.056 INFO:teuthology.orchestra.run.smithi136.stdout: mon pg warn max object skew = 0 2022-06-17T12:56:39.057 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.057 INFO:teuthology.orchestra.run.smithi136.stdout: # disable pg_autoscaler by default for new pools 2022-06-17T12:56:39.057 INFO:teuthology.orchestra.run.smithi136.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-17T12:56:39.058 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.059 INFO:teuthology.orchestra.run.smithi136.stdout: osd pool default size = 2 2022-06-17T12:56:39.059 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.059 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd allow primary affinity = true 2022-06-17T12:56:39.060 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd allow pg remap = true 2022-06-17T12:56:39.060 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on legacy crush tunables = false 2022-06-17T12:56:39.060 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on crush straw calc version zero = false 2022-06-17T12:56:39.061 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on no sortbitwise = false 2022-06-17T12:56:39.062 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on osd down out interval zero = false 2022-06-17T12:56:39.062 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on too few osds = false 2022-06-17T12:56:39.063 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-17T12:56:39.063 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-17T12:56:39.063 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.064 INFO:teuthology.orchestra.run.smithi136.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-17T12:56:39.064 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.065 INFO:teuthology.orchestra.run.smithi136.stdout: osd default data pool replay window = 5 2022-06-17T12:56:39.065 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.065 INFO:teuthology.orchestra.run.smithi136.stdout: mon allow pool delete = true 2022-06-17T12:56:39.067 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.071 INFO:teuthology.orchestra.run.smithi136.stdout: mon cluster log file level = debug 2022-06-17T12:56:39.072 INFO:teuthology.orchestra.run.smithi136.stdout: debug asserts on shutdown = true 2022-06-17T12:56:39.072 INFO:teuthology.orchestra.run.smithi136.stdout: mon health detail to clog = false 2022-06-17T12:56:39.072 INFO:teuthology.orchestra.run.smithi136.stdout: mon host = "172.21.15.36,172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790]" 2022-06-17T12:56:39.073 INFO:teuthology.orchestra.run.smithi136.stdout: fsid = b68b9700-0c85-42ca-94d7-6cd53db91c14 2022-06-17T12:56:39.073 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.074 INFO:teuthology.orchestra.run.smithi136.stdout:[osd] 2022-06-17T12:56:39.074 INFO:teuthology.orchestra.run.smithi136.stdout: osd journal size = 100 2022-06-17T12:56:39.075 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.075 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub load threshold = 5.0 2022-06-17T12:56:39.075 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub max interval = 600 2022-06-17T12:56:39.087 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.088 INFO:teuthology.orchestra.run.smithi136.stdout: osd recover clone overlap = true 2022-06-17T12:56:39.089 INFO:teuthology.orchestra.run.smithi136.stdout: osd recovery max chunk = 1048576 2022-06-17T12:56:39.089 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.089 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug shutdown = true 2022-06-17T12:56:39.090 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug op order = true 2022-06-17T12:56:39.099 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug verify stray on activate = true 2022-06-17T12:56:39.100 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.100 INFO:teuthology.orchestra.run.smithi136.stdout: osd open classes on start = true 2022-06-17T12:56:39.100 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug pg log writeout = true 2022-06-17T12:56:39.100 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.101 INFO:teuthology.orchestra.run.smithi136.stdout: osd deep scrub update digest min age = 30 2022-06-17T12:56:39.101 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.101 INFO:teuthology.orchestra.run.smithi136.stdout: osd map max advance = 10 2022-06-17T12:56:39.105 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.105 INFO:teuthology.orchestra.run.smithi136.stdout: journal zero on create = true 2022-06-17T12:56:39.105 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.106 INFO:teuthology.orchestra.run.smithi136.stdout: filestore ondisk finisher threads = 3 2022-06-17T12:56:39.106 INFO:teuthology.orchestra.run.smithi136.stdout: filestore apply finisher threads = 3 2022-06-17T12:56:39.107 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.107 INFO:teuthology.orchestra.run.smithi136.stdout: bdev debug aio = true 2022-06-17T12:56:39.108 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug misdirected ops = true 2022-06-17T12:56:39.108 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore block size = 96636764160 2022-06-17T12:56:39.108 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore compression mode = aggressive 2022-06-17T12:56:39.109 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore fsck on mount = True 2022-06-17T12:56:39.109 INFO:teuthology.orchestra.run.smithi136.stdout: debug bluefs = 20 2022-06-17T12:56:39.110 INFO:teuthology.orchestra.run.smithi136.stdout: debug bluestore = 20 2022-06-17T12:56:39.110 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2022-06-17T12:56:39.111 INFO:teuthology.orchestra.run.smithi136.stdout: debug osd = 20 2022-06-17T12:56:39.111 INFO:teuthology.orchestra.run.smithi136.stdout: debug rocksdb = 10 2022-06-17T12:56:39.117 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd backfillfull_ratio = 0.85 2022-06-17T12:56:39.117 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd full ratio = 0.9 2022-06-17T12:56:39.118 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd nearfull ratio = 0.8 2022-06-17T12:56:39.118 INFO:teuthology.orchestra.run.smithi136.stdout: osd failsafe full ratio = 0.95 2022-06-17T12:56:39.118 INFO:teuthology.orchestra.run.smithi136.stdout: osd objectstore = bluestore 2022-06-17T12:56:39.119 INFO:teuthology.orchestra.run.smithi136.stdout: osd op complaint time = 180 2022-06-17T12:56:39.120 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.120 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr] 2022-06-17T12:56:39.120 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2022-06-17T12:56:39.121 INFO:teuthology.orchestra.run.smithi136.stdout: debug mgr = 20 2022-06-17T12:56:39.122 INFO:teuthology.orchestra.run.smithi136.stdout: debug mon = 20 2022-06-17T12:56:39.122 INFO:teuthology.orchestra.run.smithi136.stdout: debug auth = 20 2022-06-17T12:56:39.123 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:56:39.123 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:56:39.123 INFO:teuthology.orchestra.run.smithi136.stdout: mgr/telemetry/nag = false 2022-06-17T12:56:39.124 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.124 INFO:teuthology.orchestra.run.smithi136.stdout:[mon] 2022-06-17T12:56:39.125 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2022-06-17T12:56:39.125 INFO:teuthology.orchestra.run.smithi136.stdout: debug mon = 20 2022-06-17T12:56:39.126 INFO:teuthology.orchestra.run.smithi136.stdout: debug paxos = 20 2022-06-17T12:56:39.126 INFO:teuthology.orchestra.run.smithi136.stdout: debug auth = 20 2022-06-17T12:56:39.127 INFO:teuthology.orchestra.run.smithi136.stdout: mon data avail warn = 5 2022-06-17T12:56:39.127 INFO:teuthology.orchestra.run.smithi136.stdout: mon mgr mkfs grace = 240 2022-06-17T12:56:39.128 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:56:39.128 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd reporter subtree level = osd 2022-06-17T12:56:39.128 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd prime pg temp = true 2022-06-17T12:56:39.129 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:56:39.129 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.130 INFO:teuthology.orchestra.run.smithi136.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-17T12:56:39.130 INFO:teuthology.orchestra.run.smithi136.stdout: auth mon ticket ttl = 660 # 11m 2022-06-17T12:56:39.131 INFO:teuthology.orchestra.run.smithi136.stdout: auth service ticket ttl = 240 # 4m 2022-06-17T12:56:39.131 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.132 INFO:teuthology.orchestra.run.smithi136.stdout: # don't complain about insecure global_id in the test suite 2022-06-17T12:56:39.132 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-17T12:56:39.133 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-17T12:56:39.133 INFO:teuthology.orchestra.run.smithi136.stdout: mon op complaint time = 120 2022-06-17T12:56:39.133 INFO:teuthology.orchestra.run.smithi136.stdout: 2022-06-17T12:56:39.134 INFO:teuthology.orchestra.run.smithi136.stdout:[client] 2022-06-17T12:56:39.134 INFO:teuthology.orchestra.run.smithi136.stdout: rgw cache enabled = true 2022-06-17T12:56:39.135 INFO:teuthology.orchestra.run.smithi136.stdout: rgw enable ops log = true 2022-06-17T12:56:39.135 INFO:teuthology.orchestra.run.smithi136.stdout: rgw enable usage log = true 2022-06-17T12:56:39.136 INFO:teuthology.orchestra.run.smithi136.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-17T12:56:39.136 INFO:teuthology.orchestra.run.smithi136.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-17T12:56:39.137 INFO:teuthology.orchestra.run.smithi136.stdout: client mount timeout = 600 2022-06-17T12:56:39.137 INFO:teuthology.orchestra.run.smithi136.stdout: debug client = 20 2022-06-17T12:56:39.137 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2022-06-17T12:56:39.138 INFO:teuthology.orchestra.run.smithi136.stdout: rados mon op timeout = 15m 2022-06-17T12:56:39.138 INFO:teuthology.orchestra.run.smithi136.stdout: rados osd op timeout = 15m 2022-06-17T12:56:39.139 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.a] 2022-06-17T12:56:39.139 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.b] 2022-06-17T12:56:39.139 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.c] 2022-06-17T12:56:39.140 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.a] 2022-06-17T12:56:39.140 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.b] 2022-06-17T12:56:39.141 INFO:teuthology.orchestra.run.smithi136.stdout:[mds] 2022-06-17T12:56:39.141 INFO:teuthology.orchestra.run.smithi136.stdout: debug mds = 20 2022-06-17T12:56:39.141 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2022-06-17T12:56:39.142 INFO:teuthology.orchestra.run.smithi136.stdout: mds bal frag = True 2022-06-17T12:56:39.142 INFO:teuthology.orchestra.run.smithi136.stdout: mds bal fragment size max = 10000 2022-06-17T12:56:39.143 INFO:teuthology.orchestra.run.smithi136.stdout: mds bal merge size = 5 2022-06-17T12:56:39.143 INFO:teuthology.orchestra.run.smithi136.stdout: mds bal split bits = 3 2022-06-17T12:56:39.144 INFO:teuthology.orchestra.run.smithi136.stdout: mds bal split size = 100 2022-06-17T12:56:39.144 INFO:teuthology.orchestra.run.smithi136.stdout: mds debug frag = True 2022-06-17T12:56:39.144 INFO:teuthology.orchestra.run.smithi136.stdout: mds debug scatterstat = True 2022-06-17T12:56:39.146 INFO:teuthology.orchestra.run.smithi136.stdout: mds op complaint time = 180 2022-06-17T12:56:39.146 INFO:teuthology.orchestra.run.smithi136.stdout: mds verify scatter = True 2022-06-17T12:56:39.146 INFO:teuthology.orchestra.run.smithi136.stdout: osd op complaint time = 180 2022-06-17T12:56:39.147 INFO:teuthology.orchestra.run.smithi136.stdout: rados mon op timeout = 15m 2022-06-17T12:56:39.147 INFO:teuthology.orchestra.run.smithi136.stdout: rados osd op timeout = 15m 2022-06-17T12:56:39.148 INFO:teuthology.orchestra.run.smithi107.stdout:[global] 2022-06-17T12:56:39.148 INFO:teuthology.orchestra.run.smithi107.stdout: chdir = "" 2022-06-17T12:56:39.149 INFO:teuthology.orchestra.run.smithi107.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-17T12:56:39.149 INFO:teuthology.orchestra.run.smithi107.stdout: auth supported = cephx 2022-06-17T12:56:39.154 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.155 INFO:teuthology.orchestra.run.smithi107.stdout: filestore xattr use omap = true 2022-06-17T12:56:39.155 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.155 INFO:teuthology.orchestra.run.smithi107.stdout: mon clock drift allowed = 1.000 2022-06-17T12:56:39.156 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.156 INFO:teuthology.orchestra.run.smithi107.stdout: osd crush chooseleaf type = 0 2022-06-17T12:56:39.157 INFO:teuthology.orchestra.run.smithi107.stdout: auth debug = true 2022-06-17T12:56:39.157 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.158 INFO:teuthology.orchestra.run.smithi107.stdout: ms die on old message = true 2022-06-17T12:56:39.158 INFO:teuthology.orchestra.run.smithi107.stdout: ms die on bug = true 2022-06-17T12:56:39.159 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.159 INFO:teuthology.orchestra.run.smithi107.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-17T12:56:39.160 INFO:teuthology.orchestra.run.smithi107.stdout: mon pg warn max object skew = 0 2022-06-17T12:56:39.160 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.160 INFO:teuthology.orchestra.run.smithi107.stdout: # disable pg_autoscaler by default for new pools 2022-06-17T12:56:39.161 INFO:teuthology.orchestra.run.smithi107.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-17T12:56:39.161 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.162 INFO:teuthology.orchestra.run.smithi107.stdout: osd pool default size = 2 2022-06-17T12:56:39.162 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.162 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd allow primary affinity = true 2022-06-17T12:56:39.164 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd allow pg remap = true 2022-06-17T12:56:39.164 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on legacy crush tunables = false 2022-06-17T12:56:39.164 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on crush straw calc version zero = false 2022-06-17T12:56:39.165 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on no sortbitwise = false 2022-06-17T12:56:39.165 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on osd down out interval zero = false 2022-06-17T12:56:39.166 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on too few osds = false 2022-06-17T12:56:39.167 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-17T12:56:39.167 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-17T12:56:39.168 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.168 INFO:teuthology.orchestra.run.smithi107.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-17T12:56:39.169 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.169 INFO:teuthology.orchestra.run.smithi107.stdout: osd default data pool replay window = 5 2022-06-17T12:56:39.169 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.170 INFO:teuthology.orchestra.run.smithi107.stdout: mon allow pool delete = true 2022-06-17T12:56:39.171 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.171 INFO:teuthology.orchestra.run.smithi107.stdout: mon cluster log file level = debug 2022-06-17T12:56:39.171 INFO:teuthology.orchestra.run.smithi107.stdout: debug asserts on shutdown = true 2022-06-17T12:56:39.172 INFO:teuthology.orchestra.run.smithi107.stdout: mon health detail to clog = false 2022-06-17T12:56:39.172 INFO:teuthology.orchestra.run.smithi107.stdout: mon host = "172.21.15.36,172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790]" 2022-06-17T12:56:39.172 INFO:teuthology.orchestra.run.smithi107.stdout: fsid = b68b9700-0c85-42ca-94d7-6cd53db91c14 2022-06-17T12:56:39.173 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.173 INFO:teuthology.orchestra.run.smithi107.stdout:[osd] 2022-06-17T12:56:39.175 INFO:teuthology.orchestra.run.smithi107.stdout: osd journal size = 100 2022-06-17T12:56:39.175 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.176 INFO:teuthology.orchestra.run.smithi107.stdout: osd scrub load threshold = 5.0 2022-06-17T12:56:39.176 INFO:teuthology.orchestra.run.smithi107.stdout: osd scrub max interval = 600 2022-06-17T12:56:39.177 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.177 INFO:teuthology.orchestra.run.smithi107.stdout: osd recover clone overlap = true 2022-06-17T12:56:39.177 INFO:teuthology.orchestra.run.smithi107.stdout: osd recovery max chunk = 1048576 2022-06-17T12:56:39.178 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.179 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug shutdown = true 2022-06-17T12:56:39.179 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug op order = true 2022-06-17T12:56:39.179 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug verify stray on activate = true 2022-06-17T12:56:39.180 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.180 INFO:teuthology.orchestra.run.smithi107.stdout: osd open classes on start = true 2022-06-17T12:56:39.181 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug pg log writeout = true 2022-06-17T12:56:39.181 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.181 INFO:teuthology.orchestra.run.smithi107.stdout: osd deep scrub update digest min age = 30 2022-06-17T12:56:39.182 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.182 INFO:teuthology.orchestra.run.smithi107.stdout: osd map max advance = 10 2022-06-17T12:56:39.188 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.189 INFO:teuthology.orchestra.run.smithi107.stdout: journal zero on create = true 2022-06-17T12:56:39.189 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.190 INFO:teuthology.orchestra.run.smithi107.stdout: filestore ondisk finisher threads = 3 2022-06-17T12:56:39.190 INFO:teuthology.orchestra.run.smithi107.stdout: filestore apply finisher threads = 3 2022-06-17T12:56:39.198 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.202 INFO:teuthology.orchestra.run.smithi107.stdout: bdev debug aio = true 2022-06-17T12:56:39.202 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug misdirected ops = true 2022-06-17T12:56:39.203 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore block size = 96636764160 2022-06-17T12:56:39.203 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore compression mode = aggressive 2022-06-17T12:56:39.204 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore fsck on mount = True 2022-06-17T12:56:39.204 INFO:teuthology.orchestra.run.smithi107.stdout: debug bluefs = 20 2022-06-17T12:56:39.205 INFO:teuthology.orchestra.run.smithi107.stdout: debug bluestore = 20 2022-06-17T12:56:39.205 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2022-06-17T12:56:39.205 INFO:teuthology.orchestra.run.smithi107.stdout: debug osd = 20 2022-06-17T12:56:39.206 INFO:teuthology.orchestra.run.smithi107.stdout: debug rocksdb = 10 2022-06-17T12:56:39.206 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd backfillfull_ratio = 0.85 2022-06-17T12:56:39.207 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd full ratio = 0.9 2022-06-17T12:56:39.207 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd nearfull ratio = 0.8 2022-06-17T12:56:39.207 INFO:teuthology.orchestra.run.smithi107.stdout: osd failsafe full ratio = 0.95 2022-06-17T12:56:39.208 INFO:teuthology.orchestra.run.smithi107.stdout: osd objectstore = bluestore 2022-06-17T12:56:39.209 INFO:teuthology.orchestra.run.smithi107.stdout: osd op complaint time = 180 2022-06-17T12:56:39.209 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.210 INFO:teuthology.orchestra.run.smithi107.stdout:[mgr] 2022-06-17T12:56:39.211 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2022-06-17T12:56:39.211 INFO:teuthology.orchestra.run.smithi107.stdout: debug mgr = 20 2022-06-17T12:56:39.211 INFO:teuthology.orchestra.run.smithi107.stdout: debug mon = 20 2022-06-17T12:56:39.212 INFO:teuthology.orchestra.run.smithi107.stdout: debug auth = 20 2022-06-17T12:56:39.212 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:56:39.213 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:56:39.213 INFO:teuthology.orchestra.run.smithi107.stdout: mgr/telemetry/nag = false 2022-06-17T12:56:39.214 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.214 INFO:teuthology.orchestra.run.smithi107.stdout:[mon] 2022-06-17T12:56:39.215 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2022-06-17T12:56:39.215 INFO:teuthology.orchestra.run.smithi107.stdout: debug mon = 20 2022-06-17T12:56:39.215 INFO:teuthology.orchestra.run.smithi107.stdout: debug paxos = 20 2022-06-17T12:56:39.216 INFO:teuthology.orchestra.run.smithi107.stdout: debug auth = 20 2022-06-17T12:56:39.224 INFO:teuthology.orchestra.run.smithi107.stdout: mon data avail warn = 5 2022-06-17T12:56:39.225 INFO:teuthology.orchestra.run.smithi107.stdout: mon mgr mkfs grace = 240 2022-06-17T12:56:39.225 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:56:39.225 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd reporter subtree level = osd 2022-06-17T12:56:39.226 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd prime pg temp = true 2022-06-17T12:56:39.226 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:56:39.227 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.227 INFO:teuthology.orchestra.run.smithi107.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-17T12:56:39.228 INFO:teuthology.orchestra.run.smithi107.stdout: auth mon ticket ttl = 660 # 11m 2022-06-17T12:56:39.228 INFO:teuthology.orchestra.run.smithi107.stdout: auth service ticket ttl = 240 # 4m 2022-06-17T12:56:39.228 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.229 INFO:teuthology.orchestra.run.smithi107.stdout: # don't complain about insecure global_id in the test suite 2022-06-17T12:56:39.229 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-17T12:56:39.232 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-17T12:56:39.232 INFO:teuthology.orchestra.run.smithi107.stdout: mon op complaint time = 120 2022-06-17T12:56:39.232 INFO:teuthology.orchestra.run.smithi107.stdout: 2022-06-17T12:56:39.233 INFO:teuthology.orchestra.run.smithi107.stdout:[client] 2022-06-17T12:56:39.233 INFO:teuthology.orchestra.run.smithi107.stdout: rgw cache enabled = true 2022-06-17T12:56:39.234 INFO:teuthology.orchestra.run.smithi107.stdout: rgw enable ops log = true 2022-06-17T12:56:39.234 INFO:teuthology.orchestra.run.smithi107.stdout: rgw enable usage log = true 2022-06-17T12:56:39.234 INFO:teuthology.orchestra.run.smithi107.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-17T12:56:39.235 INFO:teuthology.orchestra.run.smithi107.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-17T12:56:39.241 INFO:teuthology.orchestra.run.smithi107.stdout: client mount timeout = 600 2022-06-17T12:56:39.242 INFO:teuthology.orchestra.run.smithi107.stdout: debug client = 20 2022-06-17T12:56:39.242 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2022-06-17T12:56:39.242 INFO:teuthology.orchestra.run.smithi107.stdout: rados mon op timeout = 15m 2022-06-17T12:56:39.243 INFO:teuthology.orchestra.run.smithi107.stdout: rados osd op timeout = 15m 2022-06-17T12:56:39.243 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.a] 2022-06-17T12:56:39.244 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.b] 2022-06-17T12:56:39.244 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.c] 2022-06-17T12:56:39.244 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.a] 2022-06-17T12:56:39.245 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.b] 2022-06-17T12:56:39.245 INFO:teuthology.orchestra.run.smithi107.stdout:[mds] 2022-06-17T12:56:39.245 INFO:teuthology.orchestra.run.smithi107.stdout: debug mds = 20 2022-06-17T12:56:39.254 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2022-06-17T12:56:39.255 INFO:teuthology.orchestra.run.smithi107.stdout: mds bal frag = True 2022-06-17T12:56:39.255 INFO:teuthology.orchestra.run.smithi107.stdout: mds bal fragment size max = 10000 2022-06-17T12:56:39.256 INFO:teuthology.orchestra.run.smithi107.stdout: mds bal merge size = 5 2022-06-17T12:56:39.256 INFO:teuthology.orchestra.run.smithi107.stdout: mds bal split bits = 3 2022-06-17T12:56:39.256 INFO:teuthology.orchestra.run.smithi107.stdout: mds bal split size = 100 2022-06-17T12:56:39.257 INFO:teuthology.orchestra.run.smithi107.stdout: mds debug frag = True 2022-06-17T12:56:39.257 INFO:teuthology.orchestra.run.smithi107.stdout: mds debug scatterstat = True 2022-06-17T12:56:39.258 INFO:teuthology.orchestra.run.smithi107.stdout: mds op complaint time = 180 2022-06-17T12:56:39.258 INFO:teuthology.orchestra.run.smithi107.stdout: mds verify scatter = True 2022-06-17T12:56:39.259 INFO:teuthology.orchestra.run.smithi107.stdout: osd op complaint time = 180 2022-06-17T12:56:39.260 INFO:teuthology.orchestra.run.smithi107.stdout: rados mon op timeout = 15m 2022-06-17T12:56:39.260 INFO:teuthology.orchestra.run.smithi107.stdout: rados osd op timeout = 15m 2022-06-17T12:56:39.261 INFO:teuthology.orchestra.run.smithi070.stdout:[global] 2022-06-17T12:56:39.261 INFO:teuthology.orchestra.run.smithi070.stdout: chdir = "" 2022-06-17T12:56:39.261 INFO:teuthology.orchestra.run.smithi070.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-17T12:56:39.261 INFO:teuthology.orchestra.run.smithi070.stdout: auth supported = cephx 2022-06-17T12:56:39.262 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.262 INFO:teuthology.orchestra.run.smithi070.stdout: filestore xattr use omap = true 2022-06-17T12:56:39.262 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.263 INFO:teuthology.orchestra.run.smithi070.stdout: mon clock drift allowed = 1.000 2022-06-17T12:56:39.264 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.264 INFO:teuthology.orchestra.run.smithi070.stdout: osd crush chooseleaf type = 0 2022-06-17T12:56:39.264 INFO:teuthology.orchestra.run.smithi070.stdout: auth debug = true 2022-06-17T12:56:39.265 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.265 INFO:teuthology.orchestra.run.smithi070.stdout: ms die on old message = true 2022-06-17T12:56:39.265 INFO:teuthology.orchestra.run.smithi070.stdout: ms die on bug = true 2022-06-17T12:56:39.265 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.266 INFO:teuthology.orchestra.run.smithi070.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-17T12:56:39.266 INFO:teuthology.orchestra.run.smithi070.stdout: mon pg warn max object skew = 0 2022-06-17T12:56:39.266 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.266 INFO:teuthology.orchestra.run.smithi070.stdout: # disable pg_autoscaler by default for new pools 2022-06-17T12:56:39.266 INFO:teuthology.orchestra.run.smithi070.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-17T12:56:39.267 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.267 INFO:teuthology.orchestra.run.smithi070.stdout: osd pool default size = 2 2022-06-17T12:56:39.267 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.267 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd allow primary affinity = true 2022-06-17T12:56:39.268 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd allow pg remap = true 2022-06-17T12:56:39.268 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on legacy crush tunables = false 2022-06-17T12:56:39.268 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on crush straw calc version zero = false 2022-06-17T12:56:39.268 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on no sortbitwise = false 2022-06-17T12:56:39.269 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on osd down out interval zero = false 2022-06-17T12:56:39.269 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on too few osds = false 2022-06-17T12:56:39.269 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-17T12:56:39.269 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-17T12:56:39.270 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.271 INFO:teuthology.orchestra.run.smithi070.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-17T12:56:39.271 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.271 INFO:teuthology.orchestra.run.smithi070.stdout: osd default data pool replay window = 5 2022-06-17T12:56:39.272 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.272 INFO:teuthology.orchestra.run.smithi070.stdout: mon allow pool delete = true 2022-06-17T12:56:39.272 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.272 INFO:teuthology.orchestra.run.smithi070.stdout: mon cluster log file level = debug 2022-06-17T12:56:39.273 INFO:teuthology.orchestra.run.smithi070.stdout: debug asserts on shutdown = true 2022-06-17T12:56:39.273 INFO:teuthology.orchestra.run.smithi070.stdout: mon health detail to clog = false 2022-06-17T12:56:39.274 INFO:teuthology.orchestra.run.smithi070.stdout: mon host = "172.21.15.36,172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790]" 2022-06-17T12:56:39.274 INFO:teuthology.orchestra.run.smithi070.stdout: fsid = b68b9700-0c85-42ca-94d7-6cd53db91c14 2022-06-17T12:56:39.274 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.275 INFO:teuthology.orchestra.run.smithi070.stdout:[osd] 2022-06-17T12:56:39.275 INFO:teuthology.orchestra.run.smithi070.stdout: osd journal size = 100 2022-06-17T12:56:39.275 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.275 INFO:teuthology.orchestra.run.smithi070.stdout: osd scrub load threshold = 5.0 2022-06-17T12:56:39.276 INFO:teuthology.orchestra.run.smithi070.stdout: osd scrub max interval = 600 2022-06-17T12:56:39.276 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.276 INFO:teuthology.orchestra.run.smithi070.stdout: osd recover clone overlap = true 2022-06-17T12:56:39.276 INFO:teuthology.orchestra.run.smithi070.stdout: osd recovery max chunk = 1048576 2022-06-17T12:56:39.276 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.277 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug shutdown = true 2022-06-17T12:56:39.277 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug op order = true 2022-06-17T12:56:39.277 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug verify stray on activate = true 2022-06-17T12:56:39.277 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.278 INFO:teuthology.orchestra.run.smithi070.stdout: osd open classes on start = true 2022-06-17T12:56:39.278 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug pg log writeout = true 2022-06-17T12:56:39.278 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.278 INFO:teuthology.orchestra.run.smithi070.stdout: osd deep scrub update digest min age = 30 2022-06-17T12:56:39.279 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.279 INFO:teuthology.orchestra.run.smithi070.stdout: osd map max advance = 10 2022-06-17T12:56:39.279 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.279 INFO:teuthology.orchestra.run.smithi070.stdout: journal zero on create = true 2022-06-17T12:56:39.279 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.280 INFO:teuthology.orchestra.run.smithi070.stdout: filestore ondisk finisher threads = 3 2022-06-17T12:56:39.280 INFO:teuthology.orchestra.run.smithi070.stdout: filestore apply finisher threads = 3 2022-06-17T12:56:39.280 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.280 INFO:teuthology.orchestra.run.smithi070.stdout: bdev debug aio = true 2022-06-17T12:56:39.281 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug misdirected ops = true 2022-06-17T12:56:39.281 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore block size = 96636764160 2022-06-17T12:56:39.282 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore compression mode = aggressive 2022-06-17T12:56:39.282 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore fsck on mount = True 2022-06-17T12:56:39.282 INFO:teuthology.orchestra.run.smithi070.stdout: debug bluefs = 20 2022-06-17T12:56:39.283 INFO:teuthology.orchestra.run.smithi070.stdout: debug bluestore = 20 2022-06-17T12:56:39.283 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2022-06-17T12:56:39.283 INFO:teuthology.orchestra.run.smithi070.stdout: debug osd = 20 2022-06-17T12:56:39.284 INFO:teuthology.orchestra.run.smithi070.stdout: debug rocksdb = 10 2022-06-17T12:56:39.284 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd backfillfull_ratio = 0.85 2022-06-17T12:56:39.285 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd full ratio = 0.9 2022-06-17T12:56:39.285 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd nearfull ratio = 0.8 2022-06-17T12:56:39.285 INFO:teuthology.orchestra.run.smithi070.stdout: osd failsafe full ratio = 0.95 2022-06-17T12:56:39.285 INFO:teuthology.orchestra.run.smithi070.stdout: osd objectstore = bluestore 2022-06-17T12:56:39.286 INFO:teuthology.orchestra.run.smithi070.stdout: osd op complaint time = 180 2022-06-17T12:56:39.286 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.286 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr] 2022-06-17T12:56:39.286 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2022-06-17T12:56:39.287 INFO:teuthology.orchestra.run.smithi070.stdout: debug mgr = 20 2022-06-17T12:56:39.287 INFO:teuthology.orchestra.run.smithi070.stdout: debug mon = 20 2022-06-17T12:56:39.287 INFO:teuthology.orchestra.run.smithi070.stdout: debug auth = 20 2022-06-17T12:56:39.287 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:56:39.288 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:56:39.288 INFO:teuthology.orchestra.run.smithi070.stdout: mgr/telemetry/nag = false 2022-06-17T12:56:39.288 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.288 INFO:teuthology.orchestra.run.smithi070.stdout:[mon] 2022-06-17T12:56:39.289 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2022-06-17T12:56:39.289 INFO:teuthology.orchestra.run.smithi070.stdout: debug mon = 20 2022-06-17T12:56:39.289 INFO:teuthology.orchestra.run.smithi070.stdout: debug paxos = 20 2022-06-17T12:56:39.289 INFO:teuthology.orchestra.run.smithi070.stdout: debug auth = 20 2022-06-17T12:56:39.290 INFO:teuthology.orchestra.run.smithi070.stdout: mon data avail warn = 5 2022-06-17T12:56:39.290 INFO:teuthology.orchestra.run.smithi070.stdout: mon mgr mkfs grace = 240 2022-06-17T12:56:39.290 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min pgs per osd = 4 2022-06-17T12:56:39.290 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd reporter subtree level = osd 2022-06-17T12:56:39.291 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd prime pg temp = true 2022-06-17T12:56:39.291 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min bytes per osd = 10 2022-06-17T12:56:39.292 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.292 INFO:teuthology.orchestra.run.smithi070.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-17T12:56:39.292 INFO:teuthology.orchestra.run.smithi070.stdout: auth mon ticket ttl = 660 # 11m 2022-06-17T12:56:39.292 INFO:teuthology.orchestra.run.smithi070.stdout: auth service ticket ttl = 240 # 4m 2022-06-17T12:56:39.293 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.293 INFO:teuthology.orchestra.run.smithi070.stdout: # don't complain about insecure global_id in the test suite 2022-06-17T12:56:39.293 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-17T12:56:39.293 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-17T12:56:39.294 INFO:teuthology.orchestra.run.smithi070.stdout: mon op complaint time = 120 2022-06-17T12:56:39.294 INFO:teuthology.orchestra.run.smithi070.stdout: 2022-06-17T12:56:39.295 INFO:teuthology.orchestra.run.smithi070.stdout:[client] 2022-06-17T12:56:39.295 INFO:teuthology.orchestra.run.smithi070.stdout: rgw cache enabled = true 2022-06-17T12:56:39.295 INFO:teuthology.orchestra.run.smithi070.stdout: rgw enable ops log = true 2022-06-17T12:56:39.296 INFO:teuthology.orchestra.run.smithi070.stdout: rgw enable usage log = true 2022-06-17T12:56:39.296 INFO:teuthology.orchestra.run.smithi070.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-17T12:56:39.296 INFO:teuthology.orchestra.run.smithi070.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-17T12:56:39.296 INFO:teuthology.orchestra.run.smithi070.stdout: client mount timeout = 600 2022-06-17T12:56:39.297 INFO:teuthology.orchestra.run.smithi070.stdout: debug client = 20 2022-06-17T12:56:39.297 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2022-06-17T12:56:39.297 INFO:teuthology.orchestra.run.smithi070.stdout: rados mon op timeout = 15m 2022-06-17T12:56:39.297 INFO:teuthology.orchestra.run.smithi070.stdout: rados osd op timeout = 15m 2022-06-17T12:56:39.306 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.a] 2022-06-17T12:56:39.306 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.b] 2022-06-17T12:56:39.307 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.c] 2022-06-17T12:56:39.307 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.a] 2022-06-17T12:56:39.307 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.b] 2022-06-17T12:56:39.308 INFO:teuthology.orchestra.run.smithi070.stdout:[mds] 2022-06-17T12:56:39.308 INFO:teuthology.orchestra.run.smithi070.stdout: debug mds = 20 2022-06-17T12:56:39.309 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2022-06-17T12:56:39.309 INFO:teuthology.orchestra.run.smithi070.stdout: mds bal frag = True 2022-06-17T12:56:39.310 INFO:teuthology.orchestra.run.smithi070.stdout: mds bal fragment size max = 10000 2022-06-17T12:56:39.310 INFO:teuthology.orchestra.run.smithi070.stdout: mds bal merge size = 5 2022-06-17T12:56:39.310 INFO:teuthology.orchestra.run.smithi070.stdout: mds bal split bits = 3 2022-06-17T12:56:39.311 INFO:teuthology.orchestra.run.smithi070.stdout: mds bal split size = 100 2022-06-17T12:56:39.311 INFO:teuthology.orchestra.run.smithi070.stdout: mds debug frag = True 2022-06-17T12:56:39.312 INFO:teuthology.orchestra.run.smithi070.stdout: mds debug scatterstat = True 2022-06-17T12:56:39.312 INFO:teuthology.orchestra.run.smithi070.stdout: mds op complaint time = 180 2022-06-17T12:56:39.313 INFO:teuthology.orchestra.run.smithi070.stdout: mds verify scatter = True 2022-06-17T12:56:39.313 INFO:teuthology.orchestra.run.smithi070.stdout: osd op complaint time = 180 2022-06-17T12:56:39.313 INFO:teuthology.orchestra.run.smithi070.stdout: rados mon op timeout = 15m 2022-06-17T12:56:39.314 INFO:teuthology.orchestra.run.smithi070.stdout: rados osd op timeout = 15m 2022-06-17T12:56:39.315 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-06-17T12:56:39.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.450+0000 7fc619d20700 1 Processor -- start 2022-06-17T12:56:39.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.450+0000 7fc619d20700 1 -- start start 2022-06-17T12:56:39.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.450+0000 7fc619d20700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc61414b2b0 0x7fc614060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:39.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.450+0000 7fc619d20700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc614147bd0 0x7fc6141442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:39.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.450+0000 7fc619d20700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc61414b870 con 0x7fc6140b27b0 2022-06-17T12:56:39.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.450+0000 7fc619d20700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6140ac500 con 0x7fc614147bd0 2022-06-17T12:56:39.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.450+0000 7fc619d20700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6140ace90 con 0x7fc61414b2b0 2022-06-17T12:56:39.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.450+0000 7fc61951f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc614147bd0 0x7fc6141442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:39.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.450+0000 7fc618d1e700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fc6140b27b0 0x7fc6140ac780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57256/0 (socket says 172.21.15.36:57256) 2022-06-17T12:56:39.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.450+0000 7fc618d1e700 1 -- 172.21.15.36:0/3101341885 learned_addr learned my addr 172.21.15.36:0/3101341885 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:39.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.450+0000 7fc613fff700 1 --2- 172.21.15.36:0/3101341885 >> v2:172.21.15.70:3300/0 conn(0x7fc61414b2b0 0x7fc614060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:39.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.450+0000 7fc61951f700 1 -- 172.21.15.36:0/3101341885 >> v2:172.21.15.70:3300/0 conn(0x7fc61414b2b0 msgr2=0x7fc614060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:39.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.450+0000 7fc61951f700 1 --2- 172.21.15.36:0/3101341885 >> v2:172.21.15.70:3300/0 conn(0x7fc61414b2b0 0x7fc614060ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:39.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.450+0000 7fc61951f700 1 -- 172.21.15.36:0/3101341885 >> v1:172.21.15.36:6789/0 conn(0x7fc6140b27b0 legacy=0x7fc6140ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:39.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.450+0000 7fc61951f700 1 -- 172.21.15.36:0/3101341885 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6140b2b90 con 0x7fc614147bd0 2022-06-17T12:56:39.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.450+0000 7fc6137fe700 1 -- 172.21.15.36:0/3101341885 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 357160793 0 0) 0x7fc61414b870 con 0x7fc6140b27b0 2022-06-17T12:56:39.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.450+0000 7fc61951f700 1 --2- 172.21.15.36:0/3101341885 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc614147bd0 0x7fc6141442d0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7fc60c0032f0 tx=0x7fc60c0091d0).ready entity=mon.0 client_cookie=ea54cac520492e72 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:39.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.450+0000 7fc6137fe700 1 -- 172.21.15.36:0/3101341885 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc60c00b680 con 0x7fc614147bd0 2022-06-17T12:56:39.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.450+0000 7fc6137fe700 1 -- 172.21.15.36:0/3101341885 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc60c00b7e0 con 0x7fc614147bd0 2022-06-17T12:56:39.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.450+0000 7fc6137fe700 1 -- 172.21.15.36:0/3101341885 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc60c00b950 con 0x7fc614147bd0 2022-06-17T12:56:39.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.454+0000 7fc619d20700 1 -- 172.21.15.36:0/3101341885 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc614147bd0 msgr2=0x7fc6141442d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:39.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.454+0000 7fc619d20700 1 --2- 172.21.15.36:0/3101341885 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc614147bd0 0x7fc6141442d0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7fc60c0032f0 tx=0x7fc60c0091d0).stop 2022-06-17T12:56:39.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.454+0000 7fc619d20700 1 -- 172.21.15.36:0/3101341885 shutdown_connections 2022-06-17T12:56:39.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.454+0000 7fc619d20700 1 --2- 172.21.15.36:0/3101341885 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc614147bd0 0x7fc6141442d0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:39.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.454+0000 7fc619d20700 1 --2- 172.21.15.36:0/3101341885 >> v2:172.21.15.70:3300/0 conn(0x7fc61414b2b0 0x7fc614060ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:39.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.454+0000 7fc619d20700 1 -- 172.21.15.36:0/3101341885 >> 172.21.15.36:0/3101341885 conn(0x7fc6140b5d10 msgr2=0x7fc6140b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:39.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.454+0000 7fc619d20700 1 -- 172.21.15.36:0/3101341885 shutdown_connections 2022-06-17T12:56:39.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.454+0000 7fc619d20700 1 -- 172.21.15.36:0/3101341885 wait complete. 2022-06-17T12:56:39.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.454+0000 7fc619d20700 1 Processor -- start 2022-06-17T12:56:39.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.454+0000 7fc619d20700 1 -- start start 2022-06-17T12:56:39.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.454+0000 7fc619d20700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc6140b27b0 0x7fc614064be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:39.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.454+0000 7fc619d20700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc614147bd0 0x7fc61415a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:39.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.454+0000 7fc619d20700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc61414b2b0 0x7fc6141568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:39.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.454+0000 7fc619d20700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6141564b0 con 0x7fc6140b27b0 2022-06-17T12:56:39.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.454+0000 7fc619d20700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6140646a0 con 0x7fc61414b2b0 2022-06-17T12:56:39.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.454+0000 7fc619d20700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc614065310 con 0x7fc614147bd0 2022-06-17T12:56:39.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.454+0000 7fc618d1e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc6140b27b0 0x7fc614064be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:39.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.454+0000 7fc618d1e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc6140b27b0 0x7fc614064be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39732/0 (socket says 172.21.15.36:39732) 2022-06-17T12:56:39.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.454+0000 7fc618d1e700 1 -- 172.21.15.36:0/2593159754 learned_addr learned my addr 172.21.15.36:0/2593159754 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:39.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.458+0000 7fc613fff700 1 --2- 172.21.15.36:0/2593159754 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc614147bd0 0x7fc61415a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:39.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.458+0000 7fc61951f700 1 --2- 172.21.15.36:0/2593159754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc61414b2b0 0x7fc6141568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:39.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.458+0000 7fc618d1e700 1 -- 172.21.15.36:0/2593159754 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc614147bd0 msgr2=0x7fc61415a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:39.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.458+0000 7fc618d1e700 1 --2- 172.21.15.36:0/2593159754 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc614147bd0 0x7fc61415a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:39.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.458+0000 7fc618d1e700 1 -- 172.21.15.36:0/2593159754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc61414b2b0 msgr2=0x7fc6141568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:39.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.458+0000 7fc618d1e700 1 --2- 172.21.15.36:0/2593159754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc61414b2b0 0x7fc6141568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:39.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.458+0000 7fc618d1e700 1 -- 172.21.15.36:0/2593159754 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60c009470 con 0x7fc6140b27b0 2022-06-17T12:56:39.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.458+0000 7fc61951f700 1 --2- 172.21.15.36:0/2593159754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc61414b2b0 0x7fc6141568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:56:39.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.458+0000 7fc618d1e700 1 --2- 172.21.15.36:0/2593159754 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc6140b27b0 0x7fc614064be0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 rx=0x7fc608000bc0 tx=0x7fc608007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:39.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.458+0000 7fc611ffb700 1 -- 172.21.15.36:0/2593159754 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6080070d0 con 0x7fc6140b27b0 2022-06-17T12:56:39.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.458+0000 7fc619d20700 1 -- 172.21.15.36:0/2593159754 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc61414a5a0 con 0x7fc6140b27b0 2022-06-17T12:56:39.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.458+0000 7fc611ffb700 1 -- 172.21.15.36:0/2593159754 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc608007230 con 0x7fc6140b27b0 2022-06-17T12:56:39.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.458+0000 7fc611ffb700 1 -- 172.21.15.36:0/2593159754 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc608005610 con 0x7fc6140b27b0 2022-06-17T12:56:39.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.458+0000 7fc619d20700 1 -- 172.21.15.36:0/2593159754 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc61415a8e0 con 0x7fc6140b27b0 2022-06-17T12:56:39.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.458+0000 7fc619d20700 1 -- 172.21.15.36:0/2593159754 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc614057410 con 0x7fc6140b27b0 2022-06-17T12:56:39.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.458+0000 7fc611ffb700 1 -- 172.21.15.36:0/2593159754 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fc608010070 con 0x7fc6140b27b0 2022-06-17T12:56:39.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.458+0000 7fc611ffb700 1 --2- 172.21.15.36:0/2593159754 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc5f4053a40 0x7fc5f4055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:39.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.462+0000 7fc611ffb700 1 -- 172.21.15.36:0/2593159754 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fc60806b600 con 0x7fc6140b27b0 2022-06-17T12:56:39.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.462+0000 7fc613fff700 1 --2- 172.21.15.36:0/2593159754 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc5f4053a40 0x7fc5f4055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:39.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.462+0000 7fc613fff700 1 --2- 172.21.15.36:0/2593159754 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc5f4053a40 0x7fc5f4055f00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7fc60000b000 tx=0x7fc600006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:39.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.470+0000 7fc611ffb700 1 -- 172.21.15.36:0/2593159754 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc608042550 con 0x7fc6140b27b0 2022-06-17T12:56:39.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.602+0000 7fc619d20700 1 -- 172.21.15.36:0/2593159754 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc6140b2160 con 0x7fc6140b27b0 2022-06-17T12:56:39.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.606+0000 7fc611ffb700 1 -- 172.21.15.36:0/2593159754 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc608045b70 con 0x7fc6140b27b0 2022-06-17T12:56:39.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.610+0000 7fc619d20700 1 -- 172.21.15.36:0/2593159754 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc5f4053a40 msgr2=0x7fc5f4055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:39.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.610+0000 7fc619d20700 1 --2- 172.21.15.36:0/2593159754 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc5f4053a40 0x7fc5f4055f00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7fc60000b000 tx=0x7fc600006df0).stop 2022-06-17T12:56:39.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.610+0000 7fc619d20700 1 -- 172.21.15.36:0/2593159754 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc6140b27b0 msgr2=0x7fc614064be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:39.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.610+0000 7fc619d20700 1 --2- 172.21.15.36:0/2593159754 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc6140b27b0 0x7fc614064be0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 rx=0x7fc608000bc0 tx=0x7fc608007a60).stop 2022-06-17T12:56:39.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.610+0000 7fc619d20700 1 -- 172.21.15.36:0/2593159754 shutdown_connections 2022-06-17T12:56:39.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.610+0000 7fc619d20700 1 --2- 172.21.15.36:0/2593159754 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc5f4053a40 0x7fc5f4055f00 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:39.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.610+0000 7fc619d20700 1 --2- 172.21.15.36:0/2593159754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc61414b2b0 0x7fc6141568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:39.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.610+0000 7fc619d20700 1 --2- 172.21.15.36:0/2593159754 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc614147bd0 0x7fc61415a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:39.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.610+0000 7fc619d20700 1 --2- 172.21.15.36:0/2593159754 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc6140b27b0 0x7fc614064be0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:39.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.610+0000 7fc619d20700 1 -- 172.21.15.36:0/2593159754 >> 172.21.15.36:0/2593159754 conn(0x7fc6140b5d10 msgr2=0x7fc614146920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:39.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.610+0000 7fc619d20700 1 -- 172.21.15.36:0/2593159754 shutdown_connections 2022-06-17T12:56:39.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.610+0000 7fc619d20700 1 -- 172.21.15.36:0/2593159754 wait complete. 2022-06-17T12:56:39.622 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps' 2022-06-17T12:56:39.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.754+0000 7fa40b0aa700 1 Processor -- start 2022-06-17T12:56:39.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.754+0000 7fa40b0aa700 1 -- start start 2022-06-17T12:56:39.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.754+0000 7fa40b0aa700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa4040abbb0 0x7fa4040b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:39.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.754+0000 7fa40b0aa700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4040aa060 con 0x7fa4040acf00 2022-06-17T12:56:39.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.754+0000 7fa40b0aa700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4040b66e0 con 0x7fa4040ad2e0 2022-06-17T12:56:39.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.754+0000 7fa40b0aa700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4040ad790 con 0x7fa4040abbb0 2022-06-17T12:56:39.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.754+0000 7fa403fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa4040abbb0 0x7fa4040b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:39.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.754+0000 7fa408e46700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fa4040acf00 0x7fa4040b1d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57270/0 (socket says 172.21.15.36:57270) 2022-06-17T12:56:39.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.754+0000 7fa408e46700 1 -- 172.21.15.36:0/3668746376 learned_addr learned my addr 172.21.15.36:0/3668746376 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:39.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.754+0000 7fa403fff700 1 -- 172.21.15.36:0/3668746376 >> v1:172.21.15.70:6789/0 conn(0x7fa4040ad2e0 legacy=0x7fa4041407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:39.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.754+0000 7fa403fff700 1 -- 172.21.15.36:0/3668746376 >> v1:172.21.15.36:6789/0 conn(0x7fa4040acf00 legacy=0x7fa4040b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:39.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.754+0000 7fa4037fe700 1 -- 172.21.15.36:0/3668746376 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507307789 0 0) 0x7fa4040aa060 con 0x7fa4040acf00 2022-06-17T12:56:39.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.754+0000 7fa403fff700 1 -- 172.21.15.36:0/3668746376 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4040b2480 con 0x7fa4040abbb0 2022-06-17T12:56:39.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.754+0000 7fa403fff700 1 --2- 172.21.15.36:0/3668746376 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4040abbb0 0x7fa4040b0f00 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 rx=0x7fa3f4002f70 tx=0x7fa3f4006df0).ready entity=mon.0 client_cookie=3ce15c156b446076 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:39.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.754+0000 7fa4037fe700 1 -- 172.21.15.36:0/3668746376 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3f400d610 con 0x7fa4040abbb0 2022-06-17T12:56:39.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.754+0000 7fa4037fe700 1 -- 172.21.15.36:0/3668746376 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f4000e90 con 0x7fa4040abbb0 2022-06-17T12:56:39.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.754+0000 7fa4037fe700 1 -- 172.21.15.36:0/3668746376 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3f400d960 con 0x7fa4040abbb0 2022-06-17T12:56:39.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.754+0000 7fa40b0aa700 1 -- 172.21.15.36:0/3668746376 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4040abbb0 msgr2=0x7fa4040b0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:39.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.754+0000 7fa40b0aa700 1 --2- 172.21.15.36:0/3668746376 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4040abbb0 0x7fa4040b0f00 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 rx=0x7fa3f4002f70 tx=0x7fa3f4006df0).stop 2022-06-17T12:56:39.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.754+0000 7fa40b0aa700 1 -- 172.21.15.36:0/3668746376 shutdown_connections 2022-06-17T12:56:39.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.754+0000 7fa40b0aa700 1 --2- 172.21.15.36:0/3668746376 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4040abbb0 0x7fa4040b0f00 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:39.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.754+0000 7fa40b0aa700 1 -- 172.21.15.36:0/3668746376 >> 172.21.15.36:0/3668746376 conn(0x7fa4040636d0 msgr2=0x7fa404063ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:39.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.754+0000 7fa40b0aa700 1 -- 172.21.15.36:0/3668746376 shutdown_connections 2022-06-17T12:56:39.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.754+0000 7fa40b0aa700 1 -- 172.21.15.36:0/3668746376 wait complete. 2022-06-17T12:56:39.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.758+0000 7fa40b0aa700 1 Processor -- start 2022-06-17T12:56:39.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.758+0000 7fa40b0aa700 1 -- start start 2022-06-17T12:56:39.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.758+0000 7fa40b0aa700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4040abbb0 0x7fa4040b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:39.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.758+0000 7fa40b0aa700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4040acf00 0x7fa404155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:39.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.758+0000 7fa40b0aa700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4040ad2e0 0x7fa4040a2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:39.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.758+0000 7fa40b0aa700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4040afb60 con 0x7fa4040ad2e0 2022-06-17T12:56:39.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.758+0000 7fa40b0aa700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4040645a0 con 0x7fa4040acf00 2022-06-17T12:56:39.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.758+0000 7fa40b0aa700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa404064840 con 0x7fa4040abbb0 2022-06-17T12:56:39.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.758+0000 7fa409647700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4040ad2e0 0x7fa4040a2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:39.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.758+0000 7fa409647700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4040ad2e0 0x7fa4040a2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39750/0 (socket says 172.21.15.36:39750) 2022-06-17T12:56:39.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.758+0000 7fa409647700 1 -- 172.21.15.36:0/906402712 learned_addr learned my addr 172.21.15.36:0/906402712 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:39.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.758+0000 7fa408e46700 1 --2- 172.21.15.36:0/906402712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4040abbb0 0x7fa4040b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:39.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.758+0000 7fa403fff700 1 --2- 172.21.15.36:0/906402712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4040acf00 0x7fa404155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:39.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.758+0000 7fa409647700 1 -- 172.21.15.36:0/906402712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4040abbb0 msgr2=0x7fa4040b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:39.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.758+0000 7fa409647700 1 --2- 172.21.15.36:0/906402712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4040abbb0 0x7fa4040b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:39.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.758+0000 7fa409647700 1 -- 172.21.15.36:0/906402712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4040acf00 msgr2=0x7fa404155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:39.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.758+0000 7fa409647700 1 --2- 172.21.15.36:0/906402712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4040acf00 0x7fa404155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:39.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.758+0000 7fa409647700 1 -- 172.21.15.36:0/906402712 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f400b180 con 0x7fa4040ad2e0 2022-06-17T12:56:39.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.758+0000 7fa409647700 1 --2- 172.21.15.36:0/906402712 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4040ad2e0 0x7fa4040a2ee0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 rx=0x7fa404067ab0 tx=0x7fa3fc0013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:39.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.758+0000 7fa401ffb700 1 -- 172.21.15.36:0/906402712 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3fc006aa0 con 0x7fa4040ad2e0 2022-06-17T12:56:39.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.758+0000 7fa401ffb700 1 -- 172.21.15.36:0/906402712 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3fc006c00 con 0x7fa4040ad2e0 2022-06-17T12:56:39.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.758+0000 7fa40b0aa700 1 -- 172.21.15.36:0/906402712 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa404064ae0 con 0x7fa4040ad2e0 2022-06-17T12:56:39.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.758+0000 7fa401ffb700 1 -- 172.21.15.36:0/906402712 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3fc007030 con 0x7fa4040ad2e0 2022-06-17T12:56:39.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.758+0000 7fa40b0aa700 1 -- 172.21.15.36:0/906402712 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4040b08b0 con 0x7fa4040ad2e0 2022-06-17T12:56:39.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.762+0000 7fa40b0aa700 1 -- 172.21.15.36:0/906402712 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa404065580 con 0x7fa4040ad2e0 2022-06-17T12:56:39.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.762+0000 7fa401ffb700 1 -- 172.21.15.36:0/906402712 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fa3fc010030 con 0x7fa4040ad2e0 2022-06-17T12:56:39.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.762+0000 7fa401ffb700 1 --2- 172.21.15.36:0/906402712 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa3e4053a40 0x7fa3e4055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:39.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.762+0000 7fa401ffb700 1 -- 172.21.15.36:0/906402712 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fa3fc06b960 con 0x7fa4040ad2e0 2022-06-17T12:56:39.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.762+0000 7fa408e46700 1 --2- 172.21.15.36:0/906402712 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa3e4053a40 0x7fa3e4055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:39.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.762+0000 7fa408e46700 1 --2- 172.21.15.36:0/906402712 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa3e4053a40 0x7fa3e4055f00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7fa3f800d000 tx=0x7fa3f80011a0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:39.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.766+0000 7fa401ffb700 1 -- 172.21.15.36:0/906402712 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa404065580 con 0x7fa4040ad2e0 2022-06-17T12:56:39.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:39.934+0000 7fa40b0aa700 1 -- 172.21.15.36:0/906402712 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]} v 0) v1 -- 0x7fa4040b08b0 con 0x7fa4040ad2e0 2022-06-17T12:56:40.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.818+0000 7fa401ffb700 1 -- 172.21.15.36:0/906402712 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]}]=0 v462) v1 ==== 128+0+0 (secure 0 0 0) 0x7fa4040b08b0 con 0x7fa4040ad2e0 2022-06-17T12:56:40.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.822+0000 7fa40b0aa700 1 -- 172.21.15.36:0/906402712 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa3e4053a40 msgr2=0x7fa3e4055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:40.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.822+0000 7fa40b0aa700 1 --2- 172.21.15.36:0/906402712 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa3e4053a40 0x7fa3e4055f00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7fa3f800d000 tx=0x7fa3f80011a0).stop 2022-06-17T12:56:40.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.822+0000 7fa40b0aa700 1 -- 172.21.15.36:0/906402712 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4040ad2e0 msgr2=0x7fa4040a2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:40.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.822+0000 7fa40b0aa700 1 --2- 172.21.15.36:0/906402712 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4040ad2e0 0x7fa4040a2ee0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 rx=0x7fa404067ab0 tx=0x7fa3fc0013a0).stop 2022-06-17T12:56:40.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.822+0000 7fa40b0aa700 1 -- 172.21.15.36:0/906402712 shutdown_connections 2022-06-17T12:56:40.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.822+0000 7fa40b0aa700 1 --2- 172.21.15.36:0/906402712 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa3e4053a40 0x7fa3e4055f00 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:40.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.822+0000 7fa40b0aa700 1 --2- 172.21.15.36:0/906402712 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4040ad2e0 0x7fa4040a2ee0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:40.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.822+0000 7fa40b0aa700 1 --2- 172.21.15.36:0/906402712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4040acf00 0x7fa404155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:40.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.822+0000 7fa40b0aa700 1 --2- 172.21.15.36:0/906402712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4040abbb0 0x7fa4040b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:40.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.822+0000 7fa40b0aa700 1 -- 172.21.15.36:0/906402712 >> 172.21.15.36:0/906402712 conn(0x7fa4040636d0 msgr2=0x7fa4041434c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:40.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.822+0000 7fa40b0aa700 1 -- 172.21.15.36:0/906402712 shutdown_connections 2022-06-17T12:56:40.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.822+0000 7fa40b0aa700 1 -- 172.21.15.36:0/906402712 wait complete. 2022-06-17T12:56:40.836 INFO:tasks.cephfs_test_runner:test_getattr_caps (tasks.cephfs.test_misc.TestMisc) ... ok 2022-06-17T12:56:40.837 INFO:tasks.cephfs_test_runner:Starting test: test_root_rctime (tasks.cephfs.test_misc.TestMisc) 2022-06-17T12:56:40.838 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime' 2022-06-17T12:56:40.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.990+0000 7f2af441a700 1 Processor -- start 2022-06-17T12:56:40.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.990+0000 7f2af441a700 1 -- start start 2022-06-17T12:56:40.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.990+0000 7f2af441a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2aec14b3b0 0x7f2aec0af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:40.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.990+0000 7f2af441a700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aec0ac870 con 0x7f2aec0ad140 2022-06-17T12:56:40.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.990+0000 7f2af441a700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aec0acaa0 con 0x7f2aec0ac340 2022-06-17T12:56:40.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.990+0000 7f2af441a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec148960 con 0x7f2aec14b3b0 2022-06-17T12:56:40.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.990+0000 7f2af21b6700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f2aec0ad140 0x7f2aec14aca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57284/0 (socket says 172.21.15.36:57284) 2022-06-17T12:56:40.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.990+0000 7f2af21b6700 1 -- 172.21.15.36:0/2443055823 learned_addr learned my addr 172.21.15.36:0/2443055823 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:40.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.990+0000 7f2af29b7700 1 --2- 172.21.15.36:0/2443055823 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2aec14b3b0 0x7f2aec0af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:40.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.990+0000 7f2af11b4700 1 -- 172.21.15.36:0/2443055823 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2869052171 0 0) 0x7f2aec0ac870 con 0x7f2aec0ad140 2022-06-17T12:56:40.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.990+0000 7f2af11b4700 1 -- 172.21.15.36:0/2443055823 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ad8003260 con 0x7f2aec0ad140 2022-06-17T12:56:40.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.990+0000 7f2af11b4700 1 -- 172.21.15.36:0/2443055823 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 85365934 0 0) 0x7f2aec0acaa0 con 0x7f2aec0ac340 2022-06-17T12:56:40.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.990+0000 7f2af11b4700 1 -- 172.21.15.36:0/2443055823 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2aec0ac870 con 0x7f2aec0ac340 2022-06-17T12:56:40.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.990+0000 7f2af29b7700 1 -- 172.21.15.36:0/2443055823 >> v1:172.21.15.70:6789/0 conn(0x7f2aec0ac340 legacy=0x7f2aec147b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:40.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.990+0000 7f2af29b7700 1 -- 172.21.15.36:0/2443055823 >> v1:172.21.15.36:6789/0 conn(0x7f2aec0ad140 legacy=0x7f2aec14aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:40.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.990+0000 7f2af29b7700 1 -- 172.21.15.36:0/2443055823 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aec0b0bf0 con 0x7f2aec14b3b0 2022-06-17T12:56:40.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.990+0000 7f2af29b7700 1 --2- 172.21.15.36:0/2443055823 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2aec14b3b0 0x7f2aec0af7f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f2ae4002f70 tx=0x7f2ae4006df0).ready entity=mon.2 client_cookie=57aac039b7ba9f4a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:40.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.990+0000 7f2af11b4700 1 -- 172.21.15.36:0/2443055823 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ae400d5c0 con 0x7f2aec14b3b0 2022-06-17T12:56:40.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.990+0000 7f2af11b4700 1 -- 172.21.15.36:0/2443055823 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae4000e90 con 0x7f2aec14b3b0 2022-06-17T12:56:41.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.990+0000 7f2af11b4700 1 -- 172.21.15.36:0/2443055823 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ae400d910 con 0x7f2aec14b3b0 2022-06-17T12:56:41.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af441a700 1 -- 172.21.15.36:0/2443055823 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2aec14b3b0 msgr2=0x7f2aec0af7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:41.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af441a700 1 --2- 172.21.15.36:0/2443055823 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2aec14b3b0 0x7f2aec0af7f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f2ae4002f70 tx=0x7f2ae4006df0).stop 2022-06-17T12:56:41.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af441a700 1 -- 172.21.15.36:0/2443055823 shutdown_connections 2022-06-17T12:56:41.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af441a700 1 --2- 172.21.15.36:0/2443055823 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2aec14b3b0 0x7f2aec0af7f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:41.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af441a700 1 -- 172.21.15.36:0/2443055823 >> 172.21.15.36:0/2443055823 conn(0x7f2aec0b5430 msgr2=0x7f2aec0b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:41.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af441a700 1 -- 172.21.15.36:0/2443055823 shutdown_connections 2022-06-17T12:56:41.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af441a700 1 -- 172.21.15.36:0/2443055823 wait complete. 2022-06-17T12:56:41.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af441a700 1 Processor -- start 2022-06-17T12:56:41.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af441a700 1 -- start start 2022-06-17T12:56:41.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af441a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2aec0ac340 0x7f2aec063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:41.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af441a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2aec0ad140 0x7f2aec156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:41.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af441a700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2aec14b3b0 0x7f2aec0a9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:41.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af441a700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec0a9650 con 0x7f2aec0ad140 2022-06-17T12:56:41.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af441a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec1572e0 con 0x7f2aec0ac340 2022-06-17T12:56:41.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af441a700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec157580 con 0x7f2aec14b3b0 2022-06-17T12:56:41.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af19b5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2aec0ad140 0x7f2aec156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:41.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af19b5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2aec0ad140 0x7f2aec156ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39762/0 (socket says 172.21.15.36:39762) 2022-06-17T12:56:41.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af19b5700 1 -- 172.21.15.36:0/3698640825 learned_addr learned my addr 172.21.15.36:0/3698640825 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:41.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af21b6700 1 --2- 172.21.15.36:0/3698640825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2aec0ac340 0x7f2aec063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:41.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af29b7700 1 --2- 172.21.15.36:0/3698640825 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2aec14b3b0 0x7f2aec0a9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:41.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af19b5700 1 -- 172.21.15.36:0/3698640825 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2aec14b3b0 msgr2=0x7f2aec0a9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:41.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af19b5700 1 --2- 172.21.15.36:0/3698640825 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2aec14b3b0 0x7f2aec0a9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:41.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af19b5700 1 -- 172.21.15.36:0/3698640825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2aec0ac340 msgr2=0x7f2aec063430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:41.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af19b5700 1 --2- 172.21.15.36:0/3698640825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2aec0ac340 0x7f2aec063430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:41.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af19b5700 1 -- 172.21.15.36:0/3698640825 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae400b180 con 0x7f2aec0ad140 2022-06-17T12:56:41.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.994+0000 7f2af19b5700 1 --2- 172.21.15.36:0/3698640825 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2aec0ad140 0x7f2aec156ca0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7f2ae8001490 tx=0x7f2ae8007e60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:41.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.998+0000 7f2ae37fe700 1 -- 172.21.15.36:0/3698640825 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ae8002310 con 0x7f2aec0ad140 2022-06-17T12:56:41.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.998+0000 7f2ae37fe700 1 -- 172.21.15.36:0/3698640825 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae8002470 con 0x7f2aec0ad140 2022-06-17T12:56:41.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.998+0000 7f2af441a700 1 -- 172.21.15.36:0/3698640825 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aec157820 con 0x7f2aec0ad140 2022-06-17T12:56:41.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.998+0000 7f2ae37fe700 1 -- 172.21.15.36:0/3698640825 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ae8001840 con 0x7f2aec0ad140 2022-06-17T12:56:41.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.998+0000 7f2af441a700 1 -- 172.21.15.36:0/3698640825 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aec0a8310 con 0x7f2aec0ad140 2022-06-17T12:56:41.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.998+0000 7f2ae37fe700 1 -- 172.21.15.36:0/3698640825 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2ae8010030 con 0x7f2aec0ad140 2022-06-17T12:56:41.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.998+0000 7f2af441a700 1 -- 172.21.15.36:0/3698640825 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aec1579b0 con 0x7f2aec0ad140 2022-06-17T12:56:41.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.998+0000 7f2ae37fe700 1 --2- 172.21.15.36:0/3698640825 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2acc053ad0 0x7f2acc055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:41.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.998+0000 7f2ae37fe700 1 -- 172.21.15.36:0/3698640825 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f2ae806bb80 con 0x7f2aec0ad140 2022-06-17T12:56:41.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.998+0000 7f2af21b6700 1 --2- 172.21.15.36:0/3698640825 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2acc053ad0 0x7f2acc055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:41.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:40.998+0000 7f2af21b6700 1 --2- 172.21.15.36:0/3698640825 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2acc053ad0 0x7f2acc055f90 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f2adc00d000 tx=0x7f2adc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:41.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.006+0000 7f2ae37fe700 1 -- 172.21.15.36:0/3698640825 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2ae8042bc0 con 0x7f2aec0ad140 2022-06-17T12:56:41.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.174+0000 7f2af441a700 1 -- 172.21.15.36:0/3698640825 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]} v 0) v1 -- 0x7f2aec0b1500 con 0x7f2aec0ad140 2022-06-17T12:56:41.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.826+0000 7f2ae37fe700 1 -- 172.21.15.36:0/3698640825 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]}]=0 v463) v1 ==== 130+0+0 (secure 0 0 0) 0x7f2ae80461e0 con 0x7f2aec0ad140 2022-06-17T12:56:41.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.830+0000 7f2af441a700 1 -- 172.21.15.36:0/3698640825 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2acc053ad0 msgr2=0x7f2acc055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:41.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.830+0000 7f2af441a700 1 --2- 172.21.15.36:0/3698640825 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2acc053ad0 0x7f2acc055f90 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f2adc00d000 tx=0x7f2adc009040).stop 2022-06-17T12:56:41.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.830+0000 7f2af441a700 1 -- 172.21.15.36:0/3698640825 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2aec0ad140 msgr2=0x7f2aec156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:41.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.830+0000 7f2af441a700 1 --2- 172.21.15.36:0/3698640825 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2aec0ad140 0x7f2aec156ca0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7f2ae8001490 tx=0x7f2ae8007e60).stop 2022-06-17T12:56:41.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.830+0000 7f2af441a700 1 -- 172.21.15.36:0/3698640825 shutdown_connections 2022-06-17T12:56:41.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.830+0000 7f2af441a700 1 --2- 172.21.15.36:0/3698640825 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2acc053ad0 0x7f2acc055f90 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:41.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.830+0000 7f2af441a700 1 --2- 172.21.15.36:0/3698640825 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2aec14b3b0 0x7f2aec0a9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:41.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.830+0000 7f2af441a700 1 --2- 172.21.15.36:0/3698640825 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2aec0ad140 0x7f2aec156ca0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:41.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.830+0000 7f2af441a700 1 --2- 172.21.15.36:0/3698640825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2aec0ac340 0x7f2aec063430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:41.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.830+0000 7f2af441a700 1 -- 172.21.15.36:0/3698640825 >> 172.21.15.36:0/3698640825 conn(0x7f2aec0b5430 msgr2=0x7f2aec0b3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:41.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.830+0000 7f2af441a700 1 -- 172.21.15.36:0/3698640825 shutdown_connections 2022-06-17T12:56:41.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.830+0000 7f2af441a700 1 -- 172.21.15.36:0/3698640825 wait complete. 2022-06-17T12:56:41.833 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-06-17T12:56:41.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.966+0000 7f6ce1638700 1 Processor -- start 2022-06-17T12:56:41.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.970+0000 7f6ce1638700 1 -- start start 2022-06-17T12:56:41.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.970+0000 7f6ce1638700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6cdc062270 0x7f6cdc143d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:41.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.970+0000 7f6ce1638700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6cdc0b4190 0x7f6cdc1471d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:41.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.970+0000 7f6ce1638700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cdc143130 con 0x7f6cdc0b48b0 2022-06-17T12:56:41.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.970+0000 7f6ce1638700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cdc0b6080 con 0x7f6cdc0b4190 2022-06-17T12:56:41.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.970+0000 7f6ce1638700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc062650 con 0x7f6cdc062270 2022-06-17T12:56:41.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.970+0000 7f6cdb7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f6cdc0b48b0 0x7f6cdc0b4d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57298/0 (socket says 172.21.15.36:57298) 2022-06-17T12:56:41.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.970+0000 7f6cdb7fe700 1 -- 172.21.15.36:0/931064609 learned_addr learned my addr 172.21.15.36:0/931064609 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:41.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.970+0000 7f6cda7fc700 1 --2- 172.21.15.36:0/931064609 >> v2:172.21.15.70:3300/0 conn(0x7f6cdc0b4190 0x7f6cdc1471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:41.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.970+0000 7f6cdaffd700 1 --2- 172.21.15.36:0/931064609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6cdc062270 0x7f6cdc143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:41.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.970+0000 7f6cd9ffb700 1 -- 172.21.15.36:0/931064609 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2475848047 0 0) 0x7f6cdc143130 con 0x7f6cdc0b48b0 2022-06-17T12:56:41.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.970+0000 7f6cd9ffb700 1 -- 172.21.15.36:0/931064609 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cbc003260 con 0x7f6cdc0b48b0 2022-06-17T12:56:41.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.970+0000 7f6cdaffd700 1 -- 172.21.15.36:0/931064609 >> v2:172.21.15.70:3300/0 conn(0x7f6cdc0b4190 msgr2=0x7f6cdc1471d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:41.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.970+0000 7f6cdaffd700 1 --2- 172.21.15.36:0/931064609 >> v2:172.21.15.70:3300/0 conn(0x7f6cdc0b4190 0x7f6cdc1471d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:41.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.970+0000 7f6cdaffd700 1 -- 172.21.15.36:0/931064609 >> v1:172.21.15.36:6789/0 conn(0x7f6cdc0b48b0 legacy=0x7f6cdc0b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:41.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.970+0000 7f6cdaffd700 1 -- 172.21.15.36:0/931064609 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc143360 con 0x7f6cdc062270 2022-06-17T12:56:41.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.970+0000 7f6cda7fc700 1 --2- 172.21.15.36:0/931064609 >> v2:172.21.15.70:3300/0 conn(0x7f6cdc0b4190 0x7f6cdc1471d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:41.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.970+0000 7f6cdaffd700 1 --2- 172.21.15.36:0/931064609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6cdc062270 0x7f6cdc143d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f6ccc002f70 tx=0x7f6ccc006df0).ready entity=mon.2 client_cookie=b85aca653fd33c22 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:41.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.970+0000 7f6cd9ffb700 1 -- 172.21.15.36:0/931064609 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ccc00d4a0 con 0x7f6cdc062270 2022-06-17T12:56:41.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.970+0000 7f6cd9ffb700 1 -- 172.21.15.36:0/931064609 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ccc000e90 con 0x7f6cdc062270 2022-06-17T12:56:41.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.970+0000 7f6cd9ffb700 1 -- 172.21.15.36:0/931064609 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ccc00d7f0 con 0x7f6cdc062270 2022-06-17T12:56:41.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.970+0000 7f6ce1638700 1 -- 172.21.15.36:0/931064609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6cdc062270 msgr2=0x7f6cdc143d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:41.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.970+0000 7f6ce1638700 1 --2- 172.21.15.36:0/931064609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6cdc062270 0x7f6cdc143d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f6ccc002f70 tx=0x7f6ccc006df0).stop 2022-06-17T12:56:41.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.974+0000 7f6ce1638700 1 -- 172.21.15.36:0/931064609 shutdown_connections 2022-06-17T12:56:41.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.974+0000 7f6ce1638700 1 --2- 172.21.15.36:0/931064609 >> v2:172.21.15.70:3300/0 conn(0x7f6cdc0b4190 0x7f6cdc1471d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:41.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.974+0000 7f6ce1638700 1 --2- 172.21.15.36:0/931064609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6cdc062270 0x7f6cdc143d40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:41.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.974+0000 7f6ce1638700 1 -- 172.21.15.36:0/931064609 >> 172.21.15.36:0/931064609 conn(0x7f6cdc0b97f0 msgr2=0x7f6cdc0b9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:41.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.974+0000 7f6ce1638700 1 -- 172.21.15.36:0/931064609 shutdown_connections 2022-06-17T12:56:41.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.974+0000 7f6ce1638700 1 -- 172.21.15.36:0/931064609 wait complete. 2022-06-17T12:56:41.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.974+0000 7f6ce1638700 1 Processor -- start 2022-06-17T12:56:41.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.974+0000 7f6ce1638700 1 -- start start 2022-06-17T12:56:41.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.974+0000 7f6ce1638700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6cdc062270 0x7f6cdc12d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:41.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.974+0000 7f6ce1638700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cdc0b4190 0x7f6cdc0b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:41.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.974+0000 7f6ce1638700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6cdc0b48b0 0x7f6cdc12bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:41.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.974+0000 7f6ce1638700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc1521f0 con 0x7f6cdc062270 2022-06-17T12:56:41.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.974+0000 7f6ce1638700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc12e2d0 con 0x7f6cdc0b4190 2022-06-17T12:56:41.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.974+0000 7f6ce1638700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc0b28b0 con 0x7f6cdc0b48b0 2022-06-17T12:56:41.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.974+0000 7f6cdaffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6cdc062270 0x7f6cdc12d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:41.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.974+0000 7f6cdaffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6cdc062270 0x7f6cdc12d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39772/0 (socket says 172.21.15.36:39772) 2022-06-17T12:56:41.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.974+0000 7f6cdaffd700 1 -- 172.21.15.36:0/621402576 learned_addr learned my addr 172.21.15.36:0/621402576 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:41.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.974+0000 7f6cdb7fe700 1 --2- 172.21.15.36:0/621402576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6cdc0b48b0 0x7f6cdc12bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:41.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.974+0000 7f6cda7fc700 1 --2- 172.21.15.36:0/621402576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cdc0b4190 0x7f6cdc0b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:41.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.978+0000 7f6cdb7fe700 1 -- 172.21.15.36:0/621402576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cdc0b4190 msgr2=0x7f6cdc0b8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:41.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.978+0000 7f6cdb7fe700 1 --2- 172.21.15.36:0/621402576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cdc0b4190 0x7f6cdc0b8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:41.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.978+0000 7f6cdb7fe700 1 -- 172.21.15.36:0/621402576 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6cdc062270 msgr2=0x7f6cdc12d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:41.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.978+0000 7f6cdb7fe700 1 --2- 172.21.15.36:0/621402576 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6cdc062270 0x7f6cdc12d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:41.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.978+0000 7f6cdb7fe700 1 -- 172.21.15.36:0/621402576 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc400b180 con 0x7f6cdc0b48b0 2022-06-17T12:56:41.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.978+0000 7f6cdb7fe700 1 --2- 172.21.15.36:0/621402576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6cdc0b48b0 0x7f6cdc12bed0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f6cdc0b3d60 tx=0x7f6cd4007d40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:41.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.978+0000 7f6cd3fff700 1 -- 172.21.15.36:0/621402576 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cd400ce60 con 0x7f6cdc0b48b0 2022-06-17T12:56:41.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.978+0000 7f6cd3fff700 1 -- 172.21.15.36:0/621402576 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd4005ba0 con 0x7f6cdc0b48b0 2022-06-17T12:56:41.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.978+0000 7f6ce1638700 1 -- 172.21.15.36:0/621402576 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ccc00b180 con 0x7f6cdc0b48b0 2022-06-17T12:56:41.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.978+0000 7f6cd3fff700 1 -- 172.21.15.36:0/621402576 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cd40070d0 con 0x7f6cdc0b48b0 2022-06-17T12:56:41.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.978+0000 7f6ce1638700 1 -- 172.21.15.36:0/621402576 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cdc0b2b60 con 0x7f6cdc0b48b0 2022-06-17T12:56:41.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.978+0000 7f6ce1638700 1 -- 172.21.15.36:0/621402576 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cdc143600 con 0x7f6cdc0b48b0 2022-06-17T12:56:41.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.978+0000 7f6cd3fff700 1 -- 172.21.15.36:0/621402576 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6cd401a060 con 0x7f6cdc0b48b0 2022-06-17T12:56:41.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.978+0000 7f6cd3fff700 1 --2- 172.21.15.36:0/621402576 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6cc00539b0 0x7f6cc0055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:41.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.982+0000 7f6cd3fff700 1 -- 172.21.15.36:0/621402576 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f6cd406b710 con 0x7f6cdc0b48b0 2022-06-17T12:56:41.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.982+0000 7f6cdaffd700 1 --2- 172.21.15.36:0/621402576 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6cc00539b0 0x7f6cc0055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:41.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.982+0000 7f6cdaffd700 1 --2- 172.21.15.36:0/621402576 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6cc00539b0 0x7f6cc0055e70 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f6ccc01a000 tx=0x7f6ccc014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:41.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:41.990+0000 7f6cd3fff700 1 -- 172.21.15.36:0/621402576 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6cd40426e0 con 0x7f6cdc0b48b0 2022-06-17T12:56:42.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.118+0000 7f6ce1638700 1 -- 172.21.15.36:0/621402576 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6cdc0b3b20 con 0x7f6cdc0b48b0 2022-06-17T12:56:42.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.126+0000 7f6cd3fff700 1 -- 172.21.15.36:0/621402576 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6cd4018020 con 0x7f6cdc0b48b0 2022-06-17T12:56:42.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.130+0000 7f6ce1638700 1 -- 172.21.15.36:0/621402576 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6cc00539b0 msgr2=0x7f6cc0055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:42.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.130+0000 7f6ce1638700 1 --2- 172.21.15.36:0/621402576 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6cc00539b0 0x7f6cc0055e70 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f6ccc01a000 tx=0x7f6ccc014000).stop 2022-06-17T12:56:42.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.130+0000 7f6ce1638700 1 -- 172.21.15.36:0/621402576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6cdc0b48b0 msgr2=0x7f6cdc12bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:42.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.130+0000 7f6ce1638700 1 --2- 172.21.15.36:0/621402576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6cdc0b48b0 0x7f6cdc12bed0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f6cdc0b3d60 tx=0x7f6cd4007d40).stop 2022-06-17T12:56:42.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.130+0000 7f6ce1638700 1 -- 172.21.15.36:0/621402576 shutdown_connections 2022-06-17T12:56:42.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.130+0000 7f6ce1638700 1 --2- 172.21.15.36:0/621402576 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6cc00539b0 0x7f6cc0055e70 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:42.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.130+0000 7f6ce1638700 1 --2- 172.21.15.36:0/621402576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6cdc0b48b0 0x7f6cdc12bed0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:42.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.130+0000 7f6ce1638700 1 --2- 172.21.15.36:0/621402576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cdc0b4190 0x7f6cdc0b8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:42.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.130+0000 7f6ce1638700 1 --2- 172.21.15.36:0/621402576 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6cdc062270 0x7f6cdc12d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:42.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.130+0000 7f6ce1638700 1 -- 172.21.15.36:0/621402576 >> 172.21.15.36:0/621402576 conn(0x7f6cdc0b97f0 msgr2=0x7f6cdc148880 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:42.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.130+0000 7f6ce1638700 1 -- 172.21.15.36:0/621402576 shutdown_connections 2022-06-17T12:56:42.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.130+0000 7f6ce1638700 1 -- 172.21.15.36:0/621402576 wait complete. 2022-06-17T12:56:42.139 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-17T12:56:42.140 DEBUG:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:56:42.160 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:56:42.188 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:56:42.224 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:56:42.254 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:56:42.311 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-17T12:56:42.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:42.313 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:42.314 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:56:42.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:56:42.329 INFO:teuthology.orchestra.run.smithi107.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-17T12:56:42.330 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-06-17T12:56:42.335 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:42.336 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:42.337 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:42.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:56:42.361 INFO:teuthology.orchestra.run.smithi136.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-17T12:56:42.361 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-17T12:56:42.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:42.371 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:42.372 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:56:42.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:56:42.423 INFO:teuthology.orchestra.run.smithi143.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:56:42.423 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-06-17T12:56:42.424 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-17T12:56:42.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.562+0000 7f5baf049700 1 Processor -- start 2022-06-17T12:56:42.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.562+0000 7f5baf049700 1 -- start start 2022-06-17T12:56:42.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.562+0000 7f5baf049700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ba80b27b0 0x7f5ba814b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:42.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.562+0000 7f5baf049700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5ba8060ca0 0x7f5ba80ac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:42.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.562+0000 7f5baf049700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5ba8147bd0 0x7f5ba81442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:42.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.562+0000 7f5baf049700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba814b870 con 0x7f5ba8147bd0 2022-06-17T12:56:42.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.562+0000 7f5baf049700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba80ac500 con 0x7f5ba8060ca0 2022-06-17T12:56:42.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.562+0000 7f5baf049700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba80b2b90 con 0x7f5ba80b27b0 2022-06-17T12:56:42.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5bae848700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5ba8147bd0 0x7f5ba81442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:42.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5bad846700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5ba8060ca0 0x7f5ba80ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:42.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5bae047700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ba80b27b0 0x7f5ba814b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:42.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5bae848700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5ba8147bd0 0x7f5ba81442d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39784/0 (socket says 172.21.15.36:39784) 2022-06-17T12:56:42.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5bad846700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5ba8060ca0 0x7f5ba80ac780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:60162/0 (socket says 172.21.15.36:60162) 2022-06-17T12:56:42.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5bae848700 1 -- 172.21.15.36:0/2120427651 learned_addr learned my addr 172.21.15.36:0/2120427651 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:42.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5bad846700 1 -- 172.21.15.36:0/2120427651 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ba80b27b0 msgr2=0x7f5ba814b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:42.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5bad846700 1 --2- 172.21.15.36:0/2120427651 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ba80b27b0 0x7f5ba814b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:42.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5bad846700 1 -- 172.21.15.36:0/2120427651 >> v2:172.21.15.36:3300/0 conn(0x7f5ba8147bd0 msgr2=0x7f5ba81442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:42.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5bad846700 1 --2- 172.21.15.36:0/2120427651 >> v2:172.21.15.36:3300/0 conn(0x7f5ba8147bd0 0x7f5ba81442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:42.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5bad846700 1 -- 172.21.15.36:0/2120427651 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba8147490 con 0x7f5ba8060ca0 2022-06-17T12:56:42.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5bae848700 1 --2- 172.21.15.36:0/2120427651 >> v2:172.21.15.36:3300/0 conn(0x7f5ba8147bd0 0x7f5ba81442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:42.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5bae047700 1 --2- 172.21.15.36:0/2120427651 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ba80b27b0 0x7f5ba814b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:42.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5bad846700 1 --2- 172.21.15.36:0/2120427651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ba8060ca0 0x7f5ba80ac780 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f5ba4002f70 tx=0x7f5ba4006df0).ready entity=mon.1 client_cookie=b2048b8648b8ec2c server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:42.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5bad045700 1 -- 172.21.15.36:0/2120427651 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ba400d4a0 con 0x7f5ba8060ca0 2022-06-17T12:56:42.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5bad045700 1 -- 172.21.15.36:0/2120427651 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba4000e90 con 0x7f5ba8060ca0 2022-06-17T12:56:42.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5bad045700 1 -- 172.21.15.36:0/2120427651 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ba400d7f0 con 0x7f5ba8060ca0 2022-06-17T12:56:42.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5baf049700 1 -- 172.21.15.36:0/2120427651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ba8060ca0 msgr2=0x7f5ba80ac780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:42.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5baf049700 1 --2- 172.21.15.36:0/2120427651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ba8060ca0 0x7f5ba80ac780 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f5ba4002f70 tx=0x7f5ba4006df0).stop 2022-06-17T12:56:42.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5baf049700 1 -- 172.21.15.36:0/2120427651 shutdown_connections 2022-06-17T12:56:42.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5baf049700 1 --2- 172.21.15.36:0/2120427651 >> v2:172.21.15.36:3300/0 conn(0x7f5ba8147bd0 0x7f5ba81442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:42.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5baf049700 1 --2- 172.21.15.36:0/2120427651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ba8060ca0 0x7f5ba80ac780 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:42.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5baf049700 1 --2- 172.21.15.36:0/2120427651 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ba80b27b0 0x7f5ba814b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:42.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5baf049700 1 -- 172.21.15.36:0/2120427651 >> 172.21.15.36:0/2120427651 conn(0x7f5ba80b5d10 msgr2=0x7f5ba80b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:42.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5baf049700 1 -- 172.21.15.36:0/2120427651 shutdown_connections 2022-06-17T12:56:42.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.566+0000 7f5baf049700 1 -- 172.21.15.36:0/2120427651 wait complete. 2022-06-17T12:56:42.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.570+0000 7f5baf049700 1 Processor -- start 2022-06-17T12:56:42.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.570+0000 7f5baf049700 1 -- start start 2022-06-17T12:56:42.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.570+0000 7f5baf049700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ba8060ca0 0x7f5ba814e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:42.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.570+0000 7f5baf049700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5ba80b27b0 0x7f5ba8062390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:42.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.570+0000 7f5baf049700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ba8147bd0 0x7f5ba81569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:42.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.570+0000 7f5baf049700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba80a8290 con 0x7f5ba80b27b0 2022-06-17T12:56:42.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.570+0000 7f5baf049700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba8062880 con 0x7f5ba8147bd0 2022-06-17T12:56:42.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.570+0000 7f5baf049700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba80a7ff0 con 0x7f5ba8060ca0 2022-06-17T12:56:42.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.570+0000 7f5bae848700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ba8147bd0 0x7f5ba81569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:42.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.570+0000 7f5bae047700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ba8060ca0 0x7f5ba814e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:42.670 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.570+0000 7f5bad846700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5ba80b27b0 0x7f5ba8062390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:42.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.570+0000 7f5bae047700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ba8060ca0 0x7f5ba814e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:54576/0 (socket says 172.21.15.36:54576) 2022-06-17T12:56:42.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.570+0000 7f5bae047700 1 -- 172.21.15.36:0/2785080085 learned_addr learned my addr 172.21.15.36:0/2785080085 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:42.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.570+0000 7f5bae047700 1 -- 172.21.15.36:0/2785080085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ba8147bd0 msgr2=0x7f5ba81569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:42.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.570+0000 7f5bae047700 1 --2- 172.21.15.36:0/2785080085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ba8147bd0 0x7f5ba81569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:42.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.570+0000 7f5bae047700 1 -- 172.21.15.36:0/2785080085 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5ba80b27b0 msgr2=0x7f5ba8062390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:42.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.570+0000 7f5bae047700 1 --2- 172.21.15.36:0/2785080085 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5ba80b27b0 0x7f5ba8062390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:42.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.570+0000 7f5bae047700 1 -- 172.21.15.36:0/2785080085 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9800b180 con 0x7f5ba8060ca0 2022-06-17T12:56:42.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.570+0000 7f5bae047700 1 --2- 172.21.15.36:0/2785080085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ba8060ca0 0x7f5ba814e4e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f5ba80a7300 tx=0x7f5b98006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:42.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.570+0000 7f5b977fe700 1 -- 172.21.15.36:0/2785080085 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b9800c7e0 con 0x7f5ba8060ca0 2022-06-17T12:56:42.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.570+0000 7f5b977fe700 1 -- 172.21.15.36:0/2785080085 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9800c940 con 0x7f5ba8060ca0 2022-06-17T12:56:42.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.570+0000 7f5baf049700 1 -- 172.21.15.36:0/2785080085 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba0009530 con 0x7f5ba8060ca0 2022-06-17T12:56:42.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.570+0000 7f5b977fe700 1 -- 172.21.15.36:0/2785080085 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b9800cca0 con 0x7f5ba8060ca0 2022-06-17T12:56:42.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.570+0000 7f5baf049700 1 -- 172.21.15.36:0/2785080085 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba400b180 con 0x7f5ba8060ca0 2022-06-17T12:56:42.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.574+0000 7f5baf049700 1 -- 172.21.15.36:0/2785080085 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba8057410 con 0x7f5ba8060ca0 2022-06-17T12:56:42.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.574+0000 7f5b977fe700 1 -- 172.21.15.36:0/2785080085 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f5b980334a0 con 0x7f5ba8060ca0 2022-06-17T12:56:42.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.574+0000 7f5b977fe700 1 --2- 172.21.15.36:0/2785080085 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5b88053920 0x7f5b88055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:42.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.574+0000 7f5b977fe700 1 -- 172.21.15.36:0/2785080085 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f5b9806c600 con 0x7f5ba8060ca0 2022-06-17T12:56:42.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.574+0000 7f5bad846700 1 --2- 172.21.15.36:0/2785080085 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5b88053920 0x7f5b88055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:42.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.574+0000 7f5bad846700 1 --2- 172.21.15.36:0/2785080085 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5b88053920 0x7f5b88055de0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f5ba401a000 tx=0x7f5ba4014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:42.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.582+0000 7f5b977fe700 1 -- 172.21.15.36:0/2785080085 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5b980435d0 con 0x7f5ba8060ca0 2022-06-17T12:56:42.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.710+0000 7f5baf049700 1 -- 172.21.15.36:0/2785080085 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f5ba8156730 con 0x7f5ba8060ca0 2022-06-17T12:56:42.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.714+0000 7f5b977fe700 1 -- 172.21.15.36:0/2785080085 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v134) v1 ==== 81+0+29227 (secure 0 0 0) 0x7f5b9800ce00 con 0x7f5ba8060ca0 2022-06-17T12:56:42.725 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:42.725 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:56:42.726 INFO:teuthology.orchestra.run.smithi036.stdout: "epoch": 134, 2022-06-17T12:56:42.727 INFO:teuthology.orchestra.run.smithi036.stdout: "fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", 2022-06-17T12:56:42.727 INFO:teuthology.orchestra.run.smithi036.stdout: "created": "2022-06-17T12:46:30.671408+0000", 2022-06-17T12:56:42.728 INFO:teuthology.orchestra.run.smithi036.stdout: "modified": "2022-06-17T12:56:38.777838+0000", 2022-06-17T12:56:42.728 INFO:teuthology.orchestra.run.smithi036.stdout: "last_up_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:56:42.729 INFO:teuthology.orchestra.run.smithi036.stdout: "last_in_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:56:42.729 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-17T12:56:42.730 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_num": 5799936, 2022-06-17T12:56:42.730 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_set": [ 2022-06-17T12:56:42.730 INFO:teuthology.orchestra.run.smithi036.stdout: "pglog_hardlimit", 2022-06-17T12:56:42.731 INFO:teuthology.orchestra.run.smithi036.stdout: "purged_snapdirs", 2022-06-17T12:56:42.732 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_deletes", 2022-06-17T12:56:42.732 INFO:teuthology.orchestra.run.smithi036.stdout: "sortbitwise" 2022-06-17T12:56:42.733 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:56:42.733 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_version": 6, 2022-06-17T12:56:42.733 INFO:teuthology.orchestra.run.smithi036.stdout: "full_ratio": 0.94999998807907104, 2022-06-17T12:56:42.734 INFO:teuthology.orchestra.run.smithi036.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-17T12:56:42.738 INFO:teuthology.orchestra.run.smithi036.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-17T12:56:42.743 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_snapshot": "", 2022-06-17T12:56:42.744 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_max": 26, 2022-06-17T12:56:42.746 INFO:teuthology.orchestra.run.smithi036.stdout: "max_osd": 8, 2022-06-17T12:56:42.746 INFO:teuthology.orchestra.run.smithi036.stdout: "require_min_compat_client": "jewel", 2022-06-17T12:56:42.758 INFO:teuthology.orchestra.run.smithi036.stdout: "min_compat_client": "jewel", 2022-06-17T12:56:42.762 INFO:teuthology.orchestra.run.smithi036.stdout: "require_osd_release": "octopus", 2022-06-17T12:56:42.762 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:56:42.763 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:42.763 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 1, 2022-06-17T12:56:42.764 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "device_health_metrics", 2022-06-17T12:56:42.764 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:33.881212+0000", 2022-06-17T12:56:42.772 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:56:42.772 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:56:42.773 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:56:42.773 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:56:42.774 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:56:42.774 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:56:42.775 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:56:42.775 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:56:42.775 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 1, 2022-06-17T12:56:42.776 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 1, 2022-06-17T12:56:42.776 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 1, 2022-06-17T12:56:42.777 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 1, 2022-06-17T12:56:42.777 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 1, 2022-06-17T12:56:42.778 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:56:42.778 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:56:42.785 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:56:42.787 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:56:42.787 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:56:42.788 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:56:42.788 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:56:42.789 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.792 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "16", 2022-06-17T12:56:42.792 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:56:42.792 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:56:42.793 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:56:42.793 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:56:42.793 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:56:42.794 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:56:42.794 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:56:42.794 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:56:42.794 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:56:42.795 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:56:42.795 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:56:42.795 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:56:42.795 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:56:42.796 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:56:42.796 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:56:42.796 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:56:42.796 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:56:42.797 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:56:42.797 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:56:42.797 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:56:42.797 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:56:42.798 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:56:42.798 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:56:42.798 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:56:42.798 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:56:42.799 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:56:42.799 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.799 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:56:42.799 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:56:42.800 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:56:42.800 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:56:42.801 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:56:42.801 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:56:42.801 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:56:42.801 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:56:42.802 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:56:42.802 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:56:42.802 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:56:42.802 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:56:42.803 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 1 2022-06-17T12:56:42.803 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.803 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:56:42.803 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr_devicehealth": {} 2022-06-17T12:56:42.804 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:42.804 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.804 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:42.804 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 2, 2022-06-17T12:56:42.805 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "rbd", 2022-06-17T12:56:42.805 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:44.485460+0000", 2022-06-17T12:56:42.805 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:56:42.806 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:56:42.807 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:56:42.808 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:56:42.808 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:56:42.808 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:56:42.809 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:56:42.809 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:56:42.810 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:56:42.810 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:56:42.811 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:56:42.812 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:56:42.813 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:56:42.813 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:56:42.818 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:56:42.818 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:56:42.819 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:56:42.819 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:56:42.820 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:56:42.820 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:56:42.821 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.821 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "19", 2022-06-17T12:56:42.822 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:56:42.822 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:56:42.823 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:56:42.823 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:56:42.824 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:56:42.824 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:56:42.824 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:56:42.827 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:56:42.827 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:56:42.827 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:56:42.827 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:56:42.828 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:56:42.828 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:56:42.829 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:56:42.829 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:56:42.829 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:56:42.836 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:56:42.837 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:56:42.837 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:56:42.837 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:56:42.837 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:56:42.838 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:56:42.838 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:56:42.838 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:56:42.839 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:56:42.839 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:56:42.839 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.839 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:56:42.840 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:56:42.840 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:56:42.840 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:56:42.840 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:56:42.841 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:56:42.841 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:56:42.841 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:56:42.841 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:56:42.842 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:56:42.842 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:56:42.842 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:56:42.842 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:56:42.843 INFO:teuthology.orchestra.run.smithi036.stdout: "rbd": {} 2022-06-17T12:56:42.843 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:42.843 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:42.844 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:56:42.844 INFO:teuthology.orchestra.run.smithi036.stdout: "osds": [ 2022-06-17T12:56:42.845 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:42.845 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:56:42.845 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", 2022-06-17T12:56:42.846 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:56:42.846 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:56:42.846 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:56:42.846 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:56:42.847 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:56:42.847 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:56:42.847 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:56:42.848 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 128, 2022-06-17T12:56:42.848 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:56:42.848 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:56:42.849 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:56:42.849 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:42.849 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:42.849 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:42.851 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6802", 2022-06-17T12:56:42.851 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:56:42.851 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.852 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:42.852 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:42.852 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6803", 2022-06-17T12:56:42.853 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:56:42.853 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:42.854 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:42.854 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.855 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:56:42.855 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:42.855 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:42.855 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:42.856 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6804", 2022-06-17T12:56:42.856 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:56:42.856 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.857 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:42.857 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:42.857 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6805", 2022-06-17T12:56:42.858 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:56:42.858 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:42.858 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:42.859 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.859 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:56:42.859 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:42.860 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:42.860 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:42.860 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6808", 2022-06-17T12:56:42.860 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:56:42.861 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.862 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:42.862 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:42.862 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6809", 2022-06-17T12:56:42.862 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:56:42.863 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:42.863 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:42.863 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.863 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:56:42.864 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:42.864 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:42.865 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:42.866 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6806", 2022-06-17T12:56:42.866 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:56:42.867 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.867 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:42.868 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:42.868 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6807", 2022-06-17T12:56:42.868 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:56:42.869 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:42.869 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:42.870 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.870 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6803/22619", 2022-06-17T12:56:42.919 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6805/22619", 2022-06-17T12:56:42.919 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6809/22619", 2022-06-17T12:56:42.919 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6807/22619", 2022-06-17T12:56:42.920 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:56:42.920 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:56:42.920 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:56:42.920 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:42.921 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.921 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:42.921 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:56:42.928 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", 2022-06-17T12:56:42.928 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:56:42.928 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:56:42.929 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:56:42.929 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:56:42.929 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:56:42.930 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:56:42.930 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:56:42.930 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 129, 2022-06-17T12:56:42.931 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:56:42.931 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:56:42.932 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:56:42.934 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:42.934 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:42.939 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:42.939 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6810", 2022-06-17T12:56:42.939 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:56:42.940 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.940 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:42.941 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:42.941 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6811", 2022-06-17T12:56:42.942 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:56:42.943 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:42.943 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:42.944 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.944 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:56:42.945 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:42.945 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:42.946 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:42.946 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6812", 2022-06-17T12:56:42.946 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:56:42.947 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.948 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:42.957 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:42.958 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6813", 2022-06-17T12:56:42.958 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:56:42.958 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:42.959 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:42.959 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.960 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:56:42.960 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:42.961 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:42.961 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:42.961 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6816", 2022-06-17T12:56:42.962 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:56:42.963 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.963 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:42.964 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:42.964 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6817", 2022-06-17T12:56:42.964 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:56:42.966 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:42.966 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:42.967 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.967 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:56:42.968 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:42.968 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:42.968 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:42.969 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6814", 2022-06-17T12:56:42.969 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:56:42.970 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.971 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:42.971 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:42.971 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6815", 2022-06-17T12:56:42.972 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:56:42.972 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:42.973 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:42.973 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.974 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6811/22620", 2022-06-17T12:56:42.974 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6813/22620", 2022-06-17T12:56:42.975 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6817/22620", 2022-06-17T12:56:42.975 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6815/22620", 2022-06-17T12:56:42.978 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:56:42.978 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:56:42.978 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:56:42.979 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:42.979 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.980 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:42.980 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:56:42.981 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", 2022-06-17T12:56:42.981 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:56:42.981 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:56:42.982 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:56:42.982 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:56:42.983 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:56:42.983 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:56:42.984 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:56:42.985 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 129, 2022-06-17T12:56:42.985 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:56:42.985 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:56:42.986 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:56:42.986 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:42.987 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:42.987 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:42.988 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6826", 2022-06-17T12:56:42.988 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:56:42.988 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.989 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:42.989 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:42.990 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6827", 2022-06-17T12:56:42.990 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:56:42.992 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:42.992 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:42.993 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:42.993 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:56:43.014 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:43.014 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.015 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:43.015 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6828", 2022-06-17T12:56:43.016 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:56:43.017 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.017 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.018 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:43.018 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6829", 2022-06-17T12:56:43.018 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:56:43.019 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.020 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.020 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.021 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:56:43.021 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:43.021 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.022 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:43.022 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6832", 2022-06-17T12:56:43.023 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:56:43.023 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.024 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.024 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:43.025 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6833", 2022-06-17T12:56:43.026 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:56:43.026 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.027 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.027 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.028 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:56:43.028 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:43.028 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.029 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:43.029 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6830", 2022-06-17T12:56:43.029 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:56:43.030 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.030 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.031 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:43.031 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6831", 2022-06-17T12:56:43.031 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:56:43.032 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.032 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.032 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.033 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6827/22643", 2022-06-17T12:56:43.033 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6829/22643", 2022-06-17T12:56:43.033 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6833/22643", 2022-06-17T12:56:43.034 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6831/22643", 2022-06-17T12:56:43.034 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:56:43.034 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:56:43.035 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:56:43.035 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.036 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.037 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.038 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:56:43.038 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", 2022-06-17T12:56:43.038 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:56:43.039 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:56:43.039 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:56:43.040 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:56:43.040 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:56:43.040 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:56:43.041 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:56:43.041 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 128, 2022-06-17T12:56:43.042 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:56:43.042 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:56:43.043 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:56:43.043 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:43.044 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.044 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:43.045 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6818", 2022-06-17T12:56:43.045 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:56:43.045 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.046 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.046 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:43.047 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6819", 2022-06-17T12:56:43.047 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:56:43.047 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.048 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.048 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.048 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:56:43.049 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:43.049 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.050 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:43.050 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6820", 2022-06-17T12:56:43.051 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:56:43.051 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.051 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.052 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:43.052 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6821", 2022-06-17T12:56:43.053 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:56:43.053 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.054 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.054 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.055 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:56:43.055 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:43.056 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.056 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:43.056 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6824", 2022-06-17T12:56:43.057 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:56:43.057 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.058 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.058 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:43.058 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6825", 2022-06-17T12:56:43.059 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:56:43.059 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.060 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.060 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.061 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:56:43.061 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:43.062 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.062 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:43.062 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6822", 2022-06-17T12:56:43.063 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:56:43.063 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.064 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.064 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:43.064 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6823", 2022-06-17T12:56:43.065 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:56:43.066 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.066 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.067 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.067 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6819/22646", 2022-06-17T12:56:43.068 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6821/22646", 2022-06-17T12:56:43.068 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6825/22646", 2022-06-17T12:56:43.069 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6823/22646", 2022-06-17T12:56:43.069 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:56:43.069 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:56:43.070 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:56:43.071 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.071 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.072 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.072 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:56:43.072 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", 2022-06-17T12:56:43.073 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:56:43.073 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:56:43.074 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:56:43.074 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:56:43.075 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:56:43.075 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:56:43.076 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:56:43.076 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 129, 2022-06-17T12:56:43.077 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:56:43.077 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:56:43.077 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:56:43.078 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:43.079 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.079 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:43.079 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6808", 2022-06-17T12:56:43.080 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:56:43.082 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.083 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.083 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:43.084 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6809", 2022-06-17T12:56:43.084 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:56:43.085 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.086 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.086 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.086 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:56:43.087 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:43.087 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.088 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:43.088 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6810", 2022-06-17T12:56:43.089 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:56:43.089 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.089 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.090 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:43.090 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6811", 2022-06-17T12:56:43.091 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:56:43.091 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.092 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.092 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.092 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:56:43.093 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:43.093 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.094 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:43.094 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6814", 2022-06-17T12:56:43.095 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:56:43.096 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.096 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.097 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:43.097 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6815", 2022-06-17T12:56:43.097 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:56:43.098 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.098 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.099 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.099 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:56:43.099 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:43.100 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.100 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:43.100 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6812", 2022-06-17T12:56:43.101 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:56:43.101 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.101 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.102 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:43.102 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6813", 2022-06-17T12:56:43.103 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:56:43.103 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.103 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.104 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.104 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6809/22812", 2022-06-17T12:56:43.105 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6811/22812", 2022-06-17T12:56:43.105 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6815/22812", 2022-06-17T12:56:43.106 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6813/22812", 2022-06-17T12:56:43.107 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:56:43.107 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:56:43.108 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:56:43.108 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.108 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.109 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.109 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:56:43.110 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", 2022-06-17T12:56:43.110 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:56:43.111 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:56:43.111 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:56:43.111 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:56:43.112 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:56:43.112 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:56:43.113 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:56:43.113 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 122, 2022-06-17T12:56:43.113 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:56:43.114 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:56:43.114 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:56:43.114 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:43.115 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.115 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:43.115 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6800", 2022-06-17T12:56:43.116 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:56:43.116 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.117 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.117 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:43.118 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6801", 2022-06-17T12:56:43.118 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:56:43.119 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.120 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.120 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.120 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:56:43.121 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:43.122 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.122 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:43.123 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6802", 2022-06-17T12:56:43.123 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:56:43.124 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.128 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.129 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:43.130 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6803", 2022-06-17T12:56:43.131 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:56:43.138 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.139 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.139 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.140 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:56:43.140 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:43.141 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.141 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:43.141 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6806", 2022-06-17T12:56:43.142 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:56:43.142 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.143 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.144 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:43.144 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6807", 2022-06-17T12:56:43.145 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:56:43.145 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.146 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.146 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.146 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:56:43.147 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:43.147 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.148 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:43.148 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6804", 2022-06-17T12:56:43.149 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:56:43.149 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.150 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.151 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:43.151 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6805", 2022-06-17T12:56:43.152 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:56:43.152 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.153 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.153 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.154 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6801/22813", 2022-06-17T12:56:43.154 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6803/22813", 2022-06-17T12:56:43.155 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6807/22813", 2022-06-17T12:56:43.155 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6805/22813", 2022-06-17T12:56:43.155 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:56:43.155 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:56:43.156 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:56:43.156 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.156 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.156 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.157 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:56:43.157 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", 2022-06-17T12:56:43.157 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:56:43.163 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:56:43.173 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:56:43.173 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:56:43.175 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:56:43.175 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:56:43.176 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:56:43.176 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 129, 2022-06-17T12:56:43.177 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:56:43.177 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:56:43.177 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:56:43.178 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:43.178 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.178 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:43.179 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6816", 2022-06-17T12:56:43.179 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:56:43.179 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.180 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.180 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:43.180 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6817", 2022-06-17T12:56:43.181 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:56:43.183 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.183 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.183 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.184 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:56:43.184 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:43.184 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.185 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:43.185 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6818", 2022-06-17T12:56:43.186 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:56:43.186 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.186 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.187 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:43.187 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6819", 2022-06-17T12:56:43.188 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:56:43.188 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.188 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.189 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.189 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:56:43.190 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:43.190 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.191 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:43.191 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6822", 2022-06-17T12:56:43.191 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:56:43.193 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.194 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.194 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:43.195 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6823", 2022-06-17T12:56:43.195 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:56:43.196 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.196 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.197 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.197 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:56:43.198 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:43.198 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.198 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:43.199 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6820", 2022-06-17T12:56:43.199 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:56:43.200 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.200 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.201 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:43.201 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6821", 2022-06-17T12:56:43.201 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:56:43.204 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.204 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.204 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.205 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6817/22822", 2022-06-17T12:56:43.205 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6819/22822", 2022-06-17T12:56:43.220 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6823/22822", 2022-06-17T12:56:43.221 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6821/22822", 2022-06-17T12:56:43.221 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:56:43.233 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:56:43.233 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:56:43.235 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.235 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.236 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.236 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:56:43.237 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", 2022-06-17T12:56:43.237 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:56:43.239 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:56:43.239 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:56:43.240 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:56:43.240 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:56:43.241 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:56:43.241 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:56:43.242 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 129, 2022-06-17T12:56:43.242 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:56:43.243 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:56:43.243 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:56:43.245 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:43.249 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.249 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:43.250 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6824", 2022-06-17T12:56:43.250 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:56:43.251 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.251 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.252 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:43.252 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6825", 2022-06-17T12:56:43.252 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:56:43.253 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.253 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.254 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.254 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:56:43.255 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:43.255 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.255 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:43.256 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6826", 2022-06-17T12:56:43.256 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:56:43.257 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.257 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.258 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:43.259 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6827", 2022-06-17T12:56:43.259 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:56:43.259 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.260 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.261 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.261 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:56:43.262 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:43.262 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.263 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:43.263 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6830", 2022-06-17T12:56:43.264 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:56:43.264 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.265 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.265 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:43.266 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6831", 2022-06-17T12:56:43.266 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:56:43.267 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.267 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.267 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.268 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:56:43.268 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:56:43.268 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.269 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:56:43.270 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6828", 2022-06-17T12:56:43.270 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:56:43.271 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.271 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.271 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:56:43.272 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6829", 2022-06-17T12:56:43.272 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:56:43.272 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.273 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.273 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.274 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6825/22837", 2022-06-17T12:56:43.274 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6827/22837", 2022-06-17T12:56:43.274 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6831/22837", 2022-06-17T12:56:43.275 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6829/22837", 2022-06-17T12:56:43.275 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:56:43.276 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:56:43.276 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:56:43.277 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:43.277 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.277 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:56:43.278 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_xinfo": [ 2022-06-17T12:56:43.278 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.279 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:56:43.279 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:56:43.280 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:56:43.280 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:56:43.280 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:56:43.281 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:56:43.281 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.291461+0000", 2022-06-17T12:56:43.282 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:56:43.282 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.283 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.283 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:56:43.283 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:56:43.284 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:56:43.284 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:56:43.285 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:56:43.285 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:56:43.286 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.537912+0000", 2022-06-17T12:56:43.286 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:56:43.287 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.287 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.287 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:56:43.288 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:56:43.288 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:56:43.289 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:56:43.289 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:56:43.290 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:56:43.296 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.606947+0000", 2022-06-17T12:56:43.296 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:56:43.298 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.298 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.299 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:56:43.299 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:56:43.299 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:56:43.299 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:56:43.300 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:56:43.300 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:56:43.300 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.514412+0000", 2022-06-17T12:56:43.300 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:56:43.300 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.301 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.301 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:56:43.301 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:56:43.301 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:56:43.302 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:56:43.302 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:56:43.302 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:56:43.303 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.394645+0000", 2022-06-17T12:56:43.303 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:56:43.303 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.304 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.304 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:56:43.304 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:56:43.304 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:56:43.305 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:56:43.305 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:56:43.305 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:56:43.305 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.343806+0000", 2022-06-17T12:56:43.306 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:56:43.306 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.306 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.307 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:56:43.307 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:56:43.308 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:56:43.308 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:56:43.308 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:56:43.308 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:56:43.309 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.591864+0000", 2022-06-17T12:56:43.309 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:56:43.310 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.311 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:43.311 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:56:43.311 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:56:43.312 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:56:43.312 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:56:43.312 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:56:43.312 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:56:43.313 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.607045+0000", 2022-06-17T12:56:43.313 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:56:43.314 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.314 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:56:43.315 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap": [], 2022-06-17T12:56:43.315 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap_items": [], 2022-06-17T12:56:43.315 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_temp": [], 2022-06-17T12:56:43.316 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_temp": [], 2022-06-17T12:56:43.318 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklist": { 2022-06-17T12:56:43.318 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.70:6833/197726269": "2022-06-18T12:56:35.394729+0000", 2022-06-17T12:56:43.319 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.70:6832/197726269": "2022-06-18T12:56:35.394729+0000" 2022-06-17T12:56:43.319 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.320 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profiles": { 2022-06-17T12:56:43.320 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs_data": { 2022-06-17T12:56:43.320 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-device-class": "", 2022-06-17T12:56:43.321 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:56:43.321 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-root": "default", 2022-06-17T12:56:43.321 INFO:teuthology.orchestra.run.smithi036.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-17T12:56:43.322 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:56:43.322 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "2", 2022-06-17T12:56:43.323 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:56:43.323 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:56:43.324 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van", 2022-06-17T12:56:43.324 INFO:teuthology.orchestra.run.smithi036.stdout: "w": "8" 2022-06-17T12:56:43.325 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.325 INFO:teuthology.orchestra.run.smithi036.stdout: "default": { 2022-06-17T12:56:43.325 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:56:43.326 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:56:43.326 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "1", 2022-06-17T12:56:43.327 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:56:43.327 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:56:43.328 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van" 2022-06-17T12:56:43.329 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:43.329 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:43.329 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps_queue": [], 2022-06-17T12:56:43.329 INFO:teuthology.orchestra.run.smithi036.stdout: "new_removed_snaps": [], 2022-06-17T12:56:43.330 INFO:teuthology.orchestra.run.smithi036.stdout: "new_purged_snaps": [], 2022-06-17T12:56:43.330 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_node_flags": {}, 2022-06-17T12:56:43.330 INFO:teuthology.orchestra.run.smithi036.stdout: "device_class_flags": {} 2022-06-17T12:56:43.330 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:56:43.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.714+0000 7f5baf049700 1 -- 172.21.15.36:0/2785080085 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5b88053920 msgr2=0x7f5b88055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:43.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.714+0000 7f5baf049700 1 --2- 172.21.15.36:0/2785080085 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5b88053920 0x7f5b88055de0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f5ba401a000 tx=0x7f5ba4014000).stop 2022-06-17T12:56:43.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.714+0000 7f5baf049700 1 -- 172.21.15.36:0/2785080085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ba8060ca0 msgr2=0x7f5ba814e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:43.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.714+0000 7f5baf049700 1 --2- 172.21.15.36:0/2785080085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ba8060ca0 0x7f5ba814e4e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f5ba80a7300 tx=0x7f5b98006df0).stop 2022-06-17T12:56:43.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.714+0000 7f5baf049700 1 -- 172.21.15.36:0/2785080085 shutdown_connections 2022-06-17T12:56:43.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.714+0000 7f5baf049700 1 --2- 172.21.15.36:0/2785080085 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5b88053920 0x7f5b88055de0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.714+0000 7f5baf049700 1 --2- 172.21.15.36:0/2785080085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ba8147bd0 0x7f5ba81569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.714+0000 7f5baf049700 1 --2- 172.21.15.36:0/2785080085 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5ba80b27b0 0x7f5ba8062390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.714+0000 7f5baf049700 1 --2- 172.21.15.36:0/2785080085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ba8060ca0 0x7f5ba814e4e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.714+0000 7f5baf049700 1 -- 172.21.15.36:0/2785080085 >> 172.21.15.36:0/2785080085 conn(0x7f5ba80b5d10 msgr2=0x7f5ba805f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:43.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.714+0000 7f5baf049700 1 -- 172.21.15.36:0/2785080085 shutdown_connections 2022-06-17T12:56:43.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:42.714+0000 7f5baf049700 1 -- 172.21.15.36:0/2785080085 wait complete. 2022-06-17T12:56:43.344 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:56:43.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.474+0000 7f08a3ba2700 1 Processor -- start 2022-06-17T12:56:43.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.474+0000 7f08a3ba2700 1 -- start start 2022-06-17T12:56:43.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.474+0000 7f08a3ba2700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f089c0afec0 0x7f089c060b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:43.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.474+0000 7f08a3ba2700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f089c0b15e0 con 0x7f089c060780 2022-06-17T12:56:43.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.474+0000 7f08a3ba2700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f089c143990 con 0x7f089c1434f0 2022-06-17T12:56:43.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.474+0000 7f08a3ba2700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f089c0b1a80 con 0x7f089c0afec0 2022-06-17T12:56:43.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.474+0000 7f08a193e700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f089c060780 0x7f089c0b07c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57324/0 (socket says 172.21.15.36:57324) 2022-06-17T12:56:43.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.474+0000 7f08a193e700 1 -- 172.21.15.36:0/2963566123 learned_addr learned my addr 172.21.15.36:0/2963566123 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:43.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a213f700 1 --2- 172.21.15.36:0/2963566123 >> v2:172.21.15.70:3300/0 conn(0x7f089c0afec0 0x7f089c060b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:43.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a093c700 1 -- 172.21.15.36:0/2963566123 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 41429290 0 0) 0x7f089c0b15e0 con 0x7f089c060780 2022-06-17T12:56:43.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a093c700 1 -- 172.21.15.36:0/2963566123 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0884003260 con 0x7f089c060780 2022-06-17T12:56:43.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a093c700 1 -- 172.21.15.36:0/2963566123 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1745693604 0 0) 0x7f0884003260 con 0x7f089c060780 2022-06-17T12:56:43.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a093c700 1 -- 172.21.15.36:0/2963566123 >> v2:172.21.15.70:3300/0 conn(0x7f089c0afec0 msgr2=0x7f089c060b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:43.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a093c700 1 --2- 172.21.15.36:0/2963566123 >> v2:172.21.15.70:3300/0 conn(0x7f089c0afec0 0x7f089c060b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a093c700 1 -- 172.21.15.36:0/2963566123 >> v1:172.21.15.70:6789/0 conn(0x7f089c1434f0 legacy=0x7f089c0b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:43.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a093c700 1 -- 172.21.15.36:0/2963566123 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089c0b1d20 con 0x7f089c060780 2022-06-17T12:56:43.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a093c700 1 -- 172.21.15.36:0/2963566123 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f0898001660 con 0x7f089c060780 2022-06-17T12:56:43.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a093c700 1 -- 172.21.15.36:0/2963566123 >> v1:172.21.15.36:6789/0 conn(0x7f089c060780 legacy=0x7f089c0b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:43.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a093c700 1 --2- 172.21.15.36:0/2963566123 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0884003750 0x7f0884003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:43.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a093c700 1 --2- 172.21.15.36:0/2963566123 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0884004500 0x7f0884004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:43.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a093c700 1 --2- 172.21.15.36:0/2963566123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0884004e70 0x7f088400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:43.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a093c700 1 -- 172.21.15.36:0/2963566123 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089c0b1a80 con 0x7f0884003750 2022-06-17T12:56:43.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a093c700 1 -- 172.21.15.36:0/2963566123 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088400bc70 con 0x7f0884004e70 2022-06-17T12:56:43.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a093c700 1 -- 172.21.15.36:0/2963566123 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088400be30 con 0x7f0884004500 2022-06-17T12:56:43.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a193e700 1 --2- 172.21.15.36:0/2963566123 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0884003750 0x7f0884003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:43.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a213f700 1 --2- 172.21.15.36:0/2963566123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0884004e70 0x7f088400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:43.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a113d700 1 --2- 172.21.15.36:0/2963566123 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0884004500 0x7f0884004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:43.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a193e700 1 -- 172.21.15.36:0/2963566123 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0884004500 msgr2=0x7f0884004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:43.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a193e700 1 --2- 172.21.15.36:0/2963566123 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0884004500 0x7f0884004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a193e700 1 -- 172.21.15.36:0/2963566123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0884004e70 msgr2=0x7f088400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:43.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a193e700 1 --2- 172.21.15.36:0/2963566123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0884004e70 0x7f088400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a193e700 1 -- 172.21.15.36:0/2963566123 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0884003260 con 0x7f0884003750 2022-06-17T12:56:43.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a213f700 1 -- 172.21.15.36:0/2963566123 reap_dead start 2022-06-17T12:56:43.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a193e700 1 --2- 172.21.15.36:0/2963566123 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0884003750 0x7f0884003b30 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 rx=0x7f0898001370 tx=0x7f089c143b20).ready entity=mon.0 client_cookie=291fb4301c5f535b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:43.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a093c700 1 -- 172.21.15.36:0/2963566123 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08980070d0 con 0x7f0884003750 2022-06-17T12:56:43.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a093c700 1 -- 172.21.15.36:0/2963566123 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0898007230 con 0x7f0884003750 2022-06-17T12:56:43.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a093c700 1 -- 172.21.15.36:0/2963566123 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f089800c870 con 0x7f0884003750 2022-06-17T12:56:43.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a3ba2700 1 -- 172.21.15.36:0/2963566123 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0884003750 msgr2=0x7f0884003b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:43.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a3ba2700 1 --2- 172.21.15.36:0/2963566123 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0884003750 0x7f0884003b30 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 rx=0x7f0898001370 tx=0x7f089c143b20).stop 2022-06-17T12:56:43.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a3ba2700 1 -- 172.21.15.36:0/2963566123 shutdown_connections 2022-06-17T12:56:43.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a3ba2700 1 --2- 172.21.15.36:0/2963566123 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0884003750 0x7f0884003b30 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a3ba2700 1 -- 172.21.15.36:0/2963566123 >> 172.21.15.36:0/2963566123 conn(0x7f089c0b6870 msgr2=0x7f089c0b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:43.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a3ba2700 1 -- 172.21.15.36:0/2963566123 shutdown_connections 2022-06-17T12:56:43.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.478+0000 7f08a3ba2700 1 -- 172.21.15.36:0/2963566123 wait complete. 2022-06-17T12:56:43.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f08a3ba2700 1 Processor -- start 2022-06-17T12:56:43.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f08a3ba2700 1 -- start start 2022-06-17T12:56:43.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f08a3ba2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0884003750 0x7f089c066800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:43.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f08a3ba2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f089c0ac730 0x7f089c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:43.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f08a3ba2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f089c13cb80 0x7f089c13d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:43.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f08a3ba2700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089c0a4f20 con 0x7f089c0ac730 2022-06-17T12:56:43.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f08a3ba2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089c0665c0 con 0x7f089c13cb80 2022-06-17T12:56:43.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f08a3ba2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f089c0a6790 con 0x7f0884003750 2022-06-17T12:56:43.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f08a113d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f089c0ac730 0x7f089c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:43.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f08a113d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f089c0ac730 0x7f089c0a4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39806/0 (socket says 172.21.15.36:39806) 2022-06-17T12:56:43.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f08a113d700 1 -- 172.21.15.36:0/2562547533 learned_addr learned my addr 172.21.15.36:0/2562547533 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:43.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f08a193e700 1 --2- 172.21.15.36:0/2562547533 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0884003750 0x7f089c066800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:43.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f08a213f700 1 --2- 172.21.15.36:0/2562547533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f089c13cb80 0x7f089c13d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:43.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f08a113d700 1 -- 172.21.15.36:0/2562547533 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0884003750 msgr2=0x7f089c066800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:43.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f08a113d700 1 --2- 172.21.15.36:0/2562547533 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0884003750 0x7f089c066800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f08a113d700 1 -- 172.21.15.36:0/2562547533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f089c13cb80 msgr2=0x7f089c13d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:43.504 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f08a113d700 1 --2- 172.21.15.36:0/2562547533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f089c13cb80 0x7f089c13d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f08a113d700 1 -- 172.21.15.36:0/2562547533 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089c0a6a30 con 0x7f089c0ac730 2022-06-17T12:56:43.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f08a213f700 1 --2- 172.21.15.36:0/2562547533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f089c13cb80 0x7f089c13d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:56:43.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f08a113d700 1 --2- 172.21.15.36:0/2562547533 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f089c0ac730 0x7f089c0a4580 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7f088c00d000 tx=0x7f088c007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:43.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f088affd700 1 -- 172.21.15.36:0/2562547533 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f088c00fac0 con 0x7f089c0ac730 2022-06-17T12:56:43.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f088affd700 1 -- 172.21.15.36:0/2562547533 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088c00fc20 con 0x7f089c0ac730 2022-06-17T12:56:43.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f08a3ba2700 1 -- 172.21.15.36:0/2562547533 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f089c0a40e0 con 0x7f089c0ac730 2022-06-17T12:56:43.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f088affd700 1 -- 172.21.15.36:0/2562547533 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f088c002e30 con 0x7f089c0ac730 2022-06-17T12:56:43.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.482+0000 7f08a3ba2700 1 -- 172.21.15.36:0/2562547533 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f089c0a5f80 con 0x7f089c0ac730 2022-06-17T12:56:43.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.486+0000 7f08a3ba2700 1 -- 172.21.15.36:0/2562547533 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f089c0a65d0 con 0x7f089c0ac730 2022-06-17T12:56:43.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.486+0000 7f088affd700 1 -- 172.21.15.36:0/2562547533 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f088c01b030 con 0x7f089c0ac730 2022-06-17T12:56:43.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.486+0000 7f088affd700 1 --2- 172.21.15.36:0/2562547533 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f087c0539b0 0x7f087c055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:43.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.486+0000 7f088affd700 1 -- 172.21.15.36:0/2562547533 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f088c06bf40 con 0x7f089c0ac730 2022-06-17T12:56:43.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.486+0000 7f08a193e700 1 --2- 172.21.15.36:0/2562547533 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f087c0539b0 0x7f087c055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:43.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.486+0000 7f08a193e700 1 --2- 172.21.15.36:0/2562547533 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f087c0539b0 0x7f087c055e70 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f089801a000 tx=0x7f0898015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:43.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.494+0000 7f088affd700 1 -- 172.21.15.36:0/2562547533 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f088c043020 con 0x7f089c0ac730 2022-06-17T12:56:43.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.658+0000 7f08a3ba2700 1 -- 172.21.15.36:0/2562547533 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f089c057410 con 0x7f089c0ac730 2022-06-17T12:56:43.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.658+0000 7f088affd700 1 -- 172.21.15.36:0/2562547533 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+1209 (secure 0 0 0) 0x7f088c046640 con 0x7f089c0ac730 2022-06-17T12:56:43.664 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:43.665 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":75,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/2357210652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2357210652},{"type":"v1","addr":"172.21.15.36:6835","nonce":2357210652}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68},{"gid":8218,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/3946304807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3946304807},{"type":"v1","addr":"172.21.15.70:6833","nonce":3946304807}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":75}],"filesystems":[]} 2022-06-17T12:56:43.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.658+0000 7f08a3ba2700 1 -- 172.21.15.36:0/2562547533 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f087c0539b0 msgr2=0x7f087c055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:43.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.658+0000 7f08a3ba2700 1 --2- 172.21.15.36:0/2562547533 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f087c0539b0 0x7f087c055e70 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f089801a000 tx=0x7f0898015000).stop 2022-06-17T12:56:43.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.658+0000 7f08a3ba2700 1 -- 172.21.15.36:0/2562547533 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f089c0ac730 msgr2=0x7f089c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:43.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.658+0000 7f08a3ba2700 1 --2- 172.21.15.36:0/2562547533 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f089c0ac730 0x7f089c0a4580 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7f088c00d000 tx=0x7f088c007a00).stop 2022-06-17T12:56:43.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.658+0000 7f08a3ba2700 1 -- 172.21.15.36:0/2562547533 shutdown_connections 2022-06-17T12:56:43.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.658+0000 7f08a3ba2700 1 --2- 172.21.15.36:0/2562547533 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f087c0539b0 0x7f087c055e70 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.658+0000 7f08a3ba2700 1 --2- 172.21.15.36:0/2562547533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f089c13cb80 0x7f089c13d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.658+0000 7f08a3ba2700 1 --2- 172.21.15.36:0/2562547533 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f089c0ac730 0x7f089c0a4580 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.658+0000 7f08a3ba2700 1 --2- 172.21.15.36:0/2562547533 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0884003750 0x7f089c066800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.658+0000 7f08a3ba2700 1 -- 172.21.15.36:0/2562547533 >> 172.21.15.36:0/2562547533 conn(0x7f089c0b6870 msgr2=0x7f089c0ad2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:43.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.658+0000 7f08a3ba2700 1 -- 172.21.15.36:0/2562547533 shutdown_connections 2022-06-17T12:56:43.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.658+0000 7f08a3ba2700 1 -- 172.21.15.36:0/2562547533 wait complete. 2022-06-17T12:56:43.682 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 75 2022-06-17T12:56:43.682 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:56:43.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a2bd0700 1 Processor -- start 2022-06-17T12:56:43.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a2bd0700 1 -- start start 2022-06-17T12:56:43.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a2bd0700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f299c0b2200 0x7f299c0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:43.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a2bd0700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f299c0af8d0 0x7f299c14a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:43.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a2bd0700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f299c0b2cd0 0x7f299c14adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:43.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a2bd0700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f299c0b25e0 con 0x7f299c0b2200 2022-06-17T12:56:43.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a2bd0700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f299c0ad790 con 0x7f299c0af8d0 2022-06-17T12:56:43.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a2bd0700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f299c0aef40 con 0x7f299c0b2cd0 2022-06-17T12:56:43.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a1bce700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f299c0b2200 0x7f299c0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:43.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a23cf700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f299c0b2cd0 0x7f299c14adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:43.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a1bce700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f299c0b2200 0x7f299c0af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39816/0 (socket says 172.21.15.36:39816) 2022-06-17T12:56:43.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a1bce700 1 -- 172.21.15.36:0/626505463 learned_addr learned my addr 172.21.15.36:0/626505463 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:43.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a13cd700 1 --2- 172.21.15.36:0/626505463 >> v2:172.21.15.70:3300/0 conn(0x7f299c0af8d0 0x7f299c14a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:43.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a1bce700 1 -- 172.21.15.36:0/626505463 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f299c0b2cd0 msgr2=0x7f299c14adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:43.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a1bce700 1 --2- 172.21.15.36:0/626505463 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f299c0b2cd0 0x7f299c14adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a1bce700 1 -- 172.21.15.36:0/626505463 >> v2:172.21.15.70:3300/0 conn(0x7f299c0af8d0 msgr2=0x7f299c14a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:43.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a1bce700 1 --2- 172.21.15.36:0/626505463 >> v2:172.21.15.70:3300/0 conn(0x7f299c0af8d0 0x7f299c14a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a1bce700 1 -- 172.21.15.36:0/626505463 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299c0b1950 con 0x7f299c0b2200 2022-06-17T12:56:43.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a23cf700 1 --2- 172.21.15.36:0/626505463 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f299c0b2cd0 0x7f299c14adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:43.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a1bce700 1 --2- 172.21.15.36:0/626505463 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f299c0b2200 0x7f299c0af3e0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 rx=0x7f298c002f70 tx=0x7f298c006eb0).ready entity=mon.0 client_cookie=5f54a77e602f8e73 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:43.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a0bcc700 1 -- 172.21.15.36:0/626505463 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f298c00d570 con 0x7f299c0b2200 2022-06-17T12:56:43.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a0bcc700 1 -- 172.21.15.36:0/626505463 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298c000e90 con 0x7f299c0b2200 2022-06-17T12:56:43.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a0bcc700 1 -- 172.21.15.36:0/626505463 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f298c00d900 con 0x7f299c0b2200 2022-06-17T12:56:43.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a2bd0700 1 -- 172.21.15.36:0/626505463 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f299c0b2200 msgr2=0x7f299c0af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:43.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a2bd0700 1 --2- 172.21.15.36:0/626505463 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f299c0b2200 0x7f299c0af3e0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 rx=0x7f298c002f70 tx=0x7f298c006eb0).stop 2022-06-17T12:56:43.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a2bd0700 1 -- 172.21.15.36:0/626505463 shutdown_connections 2022-06-17T12:56:43.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a2bd0700 1 --2- 172.21.15.36:0/626505463 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f299c0b2cd0 0x7f299c14adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a2bd0700 1 --2- 172.21.15.36:0/626505463 >> v2:172.21.15.70:3300/0 conn(0x7f299c0af8d0 0x7f299c14a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a2bd0700 1 --2- 172.21.15.36:0/626505463 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f299c0b2200 0x7f299c0af3e0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a2bd0700 1 -- 172.21.15.36:0/626505463 >> 172.21.15.36:0/626505463 conn(0x7f299c05f770 msgr2=0x7f299c05fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:43.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a2bd0700 1 -- 172.21.15.36:0/626505463 shutdown_connections 2022-06-17T12:56:43.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.806+0000 7f29a2bd0700 1 -- 172.21.15.36:0/626505463 wait complete. 2022-06-17T12:56:43.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.810+0000 7f29a2bd0700 1 Processor -- start 2022-06-17T12:56:43.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.810+0000 7f29a2bd0700 1 -- start start 2022-06-17T12:56:43.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.810+0000 7f29a2bd0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f299c0af8d0 0x7f299c0a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:43.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.810+0000 7f29a2bd0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f299c0b2200 0x7f299c145920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:43.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.810+0000 7f29a2bd0700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f299c0b2cd0 0x7f299c142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:43.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.810+0000 7f29a2bd0700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299c0a9330 con 0x7f299c0b2200 2022-06-17T12:56:43.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.810+0000 7f29a2bd0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299c0a9510 con 0x7f299c0af8d0 2022-06-17T12:56:43.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.810+0000 7f29a2bd0700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f299c0a97b0 con 0x7f299c0b2cd0 2022-06-17T12:56:43.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.810+0000 7f29a13cd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f299c0b2200 0x7f299c145920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:43.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.810+0000 7f29a13cd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f299c0b2200 0x7f299c145920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39820/0 (socket says 172.21.15.36:39820) 2022-06-17T12:56:43.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.810+0000 7f29a13cd700 1 -- 172.21.15.36:0/4161450111 learned_addr learned my addr 172.21.15.36:0/4161450111 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:43.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.810+0000 7f29a23cf700 1 --2- 172.21.15.36:0/4161450111 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f299c0b2cd0 0x7f299c142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:43.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.810+0000 7f29a1bce700 1 --2- 172.21.15.36:0/4161450111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f299c0af8d0 0x7f299c0a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:43.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.810+0000 7f29a13cd700 1 -- 172.21.15.36:0/4161450111 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f299c0b2cd0 msgr2=0x7f299c142250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:43.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.810+0000 7f29a13cd700 1 --2- 172.21.15.36:0/4161450111 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f299c0b2cd0 0x7f299c142250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.810+0000 7f29a13cd700 1 -- 172.21.15.36:0/4161450111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f299c0af8d0 msgr2=0x7f299c0a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:43.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.810+0000 7f29a13cd700 1 --2- 172.21.15.36:0/4161450111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f299c0af8d0 0x7f299c0a8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.810+0000 7f29a13cd700 1 -- 172.21.15.36:0/4161450111 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2994009470 con 0x7f299c0b2200 2022-06-17T12:56:43.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.810+0000 7f29a13cd700 1 --2- 172.21.15.36:0/4161450111 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f299c0b2200 0x7f299c145920 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0x7f299800c000 tx=0x7f2998006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:43.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.810+0000 7f298affd700 1 -- 172.21.15.36:0/4161450111 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2998010780 con 0x7f299c0b2200 2022-06-17T12:56:43.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.810+0000 7f298affd700 1 -- 172.21.15.36:0/4161450111 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29980108e0 con 0x7f299c0b2200 2022-06-17T12:56:43.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.810+0000 7f29a2bd0700 1 -- 172.21.15.36:0/4161450111 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f298c00b1e0 con 0x7f299c0b2200 2022-06-17T12:56:43.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.810+0000 7f298affd700 1 -- 172.21.15.36:0/4161450111 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2998010c40 con 0x7f299c0b2200 2022-06-17T12:56:43.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.810+0000 7f29a2bd0700 1 -- 172.21.15.36:0/4161450111 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f299c145e90 con 0x7f299c0b2200 2022-06-17T12:56:43.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.814+0000 7f298affd700 1 -- 172.21.15.36:0/4161450111 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f299801c460 con 0x7f299c0b2200 2022-06-17T12:56:43.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.814+0000 7f29a2bd0700 1 -- 172.21.15.36:0/4161450111 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f299c0a9c50 con 0x7f299c0b2200 2022-06-17T12:56:43.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.814+0000 7f298affd700 1 --2- 172.21.15.36:0/4161450111 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f297c053ad0 0x7f297c055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:43.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.814+0000 7f298affd700 1 -- 172.21.15.36:0/4161450111 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f299806c280 con 0x7f299c0b2200 2022-06-17T12:56:43.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.814+0000 7f29a1bce700 1 --2- 172.21.15.36:0/4161450111 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f297c053ad0 0x7f297c055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:43.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.814+0000 7f29a1bce700 1 --2- 172.21.15.36:0/4161450111 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f297c053ad0 0x7f297c055f90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f298c01a000 tx=0x7f298c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:43.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.818+0000 7f298affd700 1 -- 172.21.15.36:0/4161450111 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2998043260 con 0x7f299c0b2200 2022-06-17T12:56:43.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.978+0000 7f29a2bd0700 1 -- 172.21.15.36:0/4161450111 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f299c0ad330 con 0x7f299c0b2200 2022-06-17T12:56:43.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.978+0000 7f298affd700 1 -- 172.21.15.36:0/4161450111 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+1209 (secure 0 0 0) 0x7f2998046880 con 0x7f299c0b2200 2022-06-17T12:56:43.969 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:43.969 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":75,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/2357210652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2357210652},{"type":"v1","addr":"172.21.15.36:6835","nonce":2357210652}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68},{"gid":8218,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/3946304807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3946304807},{"type":"v1","addr":"172.21.15.70:6833","nonce":3946304807}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":75}],"filesystems":[]} 2022-06-17T12:56:43.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.982+0000 7f29a2bd0700 1 -- 172.21.15.36:0/4161450111 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f297c053ad0 msgr2=0x7f297c055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:43.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.982+0000 7f29a2bd0700 1 --2- 172.21.15.36:0/4161450111 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f297c053ad0 0x7f297c055f90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f298c01a000 tx=0x7f298c015000).stop 2022-06-17T12:56:43.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.982+0000 7f29a2bd0700 1 -- 172.21.15.36:0/4161450111 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f299c0b2200 msgr2=0x7f299c145920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:43.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.982+0000 7f29a2bd0700 1 --2- 172.21.15.36:0/4161450111 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f299c0b2200 0x7f299c145920 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0x7f299800c000 tx=0x7f2998006df0).stop 2022-06-17T12:56:43.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.982+0000 7f29a2bd0700 1 -- 172.21.15.36:0/4161450111 shutdown_connections 2022-06-17T12:56:43.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.982+0000 7f29a2bd0700 1 --2- 172.21.15.36:0/4161450111 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f297c053ad0 0x7f297c055f90 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.982+0000 7f29a2bd0700 1 --2- 172.21.15.36:0/4161450111 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f299c0b2cd0 0x7f299c142250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.982+0000 7f29a2bd0700 1 --2- 172.21.15.36:0/4161450111 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f299c0b2200 0x7f299c145920 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.982+0000 7f29a2bd0700 1 --2- 172.21.15.36:0/4161450111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f299c0af8d0 0x7f299c0a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:43.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.982+0000 7f29a2bd0700 1 -- 172.21.15.36:0/4161450111 >> 172.21.15.36:0/4161450111 conn(0x7f299c05f770 msgr2=0x7f299c0b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:43.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.982+0000 7f29a2bd0700 1 -- 172.21.15.36:0/4161450111 shutdown_connections 2022-06-17T12:56:43.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:43.982+0000 7f29a2bd0700 1 -- 172.21.15.36:0/4161450111 wait complete. 2022-06-17T12:56:43.979 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 75 2022-06-17T12:56:43.981 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-17T12:56:43.981 INFO:tasks.ceph.mds.a:Stopping old one... 2022-06-17T12:56:43.981 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-17T12:56:43.982 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T12:56:43.984 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-17T12:56:43.984 INFO:tasks.ceph.mds.b:Stopping old one... 2022-06-17T12:56:43.985 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-17T12:56:43.986 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T12:56:44.012 INFO:tasks.ceph.mds.b:Stopped 2022-06-17T12:56:44.013 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-17T12:56:44.013 INFO:tasks.ceph.mds.a:Stopped 2022-06-17T12:56:44.014 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-17T12:56:44.019 INFO:tasks.ceph.mds.b:Started 2022-06-17T12:56:44.021 DEBUG:teuthology.parallel:result is None 2022-06-17T12:56:44.023 INFO:tasks.ceph.mds.a:Started 2022-06-17T12:56:44.023 DEBUG:teuthology.parallel:result is None 2022-06-17T12:56:44.024 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-06-17T12:56:44.097 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2022-06-17T12:56:44.111 INFO:tasks.ceph.mds.a.smithi036.stdout:starting mds.a at 2022-06-17T12:56:44.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.142+0000 7fa5044d6700 1 Processor -- start 2022-06-17T12:56:44.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.142+0000 7fa5044d6700 1 -- start start 2022-06-17T12:56:44.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.142+0000 7fa5044d6700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4fc13afa0 0x7fa4fc064940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:44.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.142+0000 7fa5044d6700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4fc064e30 con 0x7fa4fc062bd0 2022-06-17T12:56:44.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.142+0000 7fa5044d6700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4fc141830 con 0x7fa4fc064560 2022-06-17T12:56:44.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.142+0000 7fa5044d6700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc0b1c30 con 0x7fa4fc13afa0 2022-06-17T12:56:44.130 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.142+0000 7fa502a73700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fa4fc062bd0 0x7fa4fc05fd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57374/0 (socket says 172.21.15.36:57374) 2022-06-17T12:56:44.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.142+0000 7fa502a73700 1 -- 172.21.15.36:0/417614430 learned_addr learned my addr 172.21.15.36:0/417614430 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:44.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.142+0000 7fa501a71700 1 --2- 172.21.15.36:0/417614430 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4fc13afa0 0x7fa4fc064940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:44.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.146+0000 7fa501270700 1 -- 172.21.15.36:0/417614430 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935019960 0 0) 0x7fa4fc064e30 con 0x7fa4fc062bd0 2022-06-17T12:56:44.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.146+0000 7fa501270700 1 -- 172.21.15.36:0/417614430 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4e4003260 con 0x7fa4fc062bd0 2022-06-17T12:56:44.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.146+0000 7fa501a71700 1 -- 172.21.15.36:0/417614430 >> v1:172.21.15.70:6789/0 conn(0x7fa4fc064560 legacy=0x7fa4fc13a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:44.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.146+0000 7fa501270700 1 -- 172.21.15.36:0/417614430 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3826224161 0 0) 0x7fa4fc141830 con 0x7fa4fc064560 2022-06-17T12:56:44.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.146+0000 7fa501a71700 1 -- 172.21.15.36:0/417614430 >> v1:172.21.15.36:6789/0 conn(0x7fa4fc062bd0 legacy=0x7fa4fc05fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:44.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.146+0000 7fa501a71700 1 -- 172.21.15.36:0/417614430 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4fc141ad0 con 0x7fa4fc13afa0 2022-06-17T12:56:44.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.146+0000 7fa501a71700 1 --2- 172.21.15.36:0/417614430 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4fc13afa0 0x7fa4fc064940 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fa4ec002f70 tx=0x7fa4ec006df0).ready entity=mon.2 client_cookie=7da1bd351494db14 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:44.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.146+0000 7fa501270700 1 -- 172.21.15.36:0/417614430 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4ec00d570 con 0x7fa4fc13afa0 2022-06-17T12:56:44.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.146+0000 7fa501270700 1 -- 172.21.15.36:0/417614430 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4ec000e90 con 0x7fa4fc13afa0 2022-06-17T12:56:44.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.146+0000 7fa501270700 1 -- 172.21.15.36:0/417614430 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4ec00d8c0 con 0x7fa4fc13afa0 2022-06-17T12:56:44.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.146+0000 7fa5044d6700 1 -- 172.21.15.36:0/417614430 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4fc13afa0 msgr2=0x7fa4fc064940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:44.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.146+0000 7fa5044d6700 1 --2- 172.21.15.36:0/417614430 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4fc13afa0 0x7fa4fc064940 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fa4ec002f70 tx=0x7fa4ec006df0).stop 2022-06-17T12:56:44.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.146+0000 7fa5044d6700 1 -- 172.21.15.36:0/417614430 shutdown_connections 2022-06-17T12:56:44.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.146+0000 7fa5044d6700 1 --2- 172.21.15.36:0/417614430 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4fc13afa0 0x7fa4fc064940 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:44.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.146+0000 7fa5044d6700 1 -- 172.21.15.36:0/417614430 >> 172.21.15.36:0/417614430 conn(0x7fa4fc0b53e0 msgr2=0x7fa4fc0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:44.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.146+0000 7fa5044d6700 1 -- 172.21.15.36:0/417614430 shutdown_connections 2022-06-17T12:56:44.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.146+0000 7fa5044d6700 1 -- 172.21.15.36:0/417614430 wait complete. 2022-06-17T12:56:44.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.146+0000 7fa5044d6700 1 Processor -- start 2022-06-17T12:56:44.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.146+0000 7fa5044d6700 1 -- start start 2022-06-17T12:56:44.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa5044d6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4fc062bd0 0x7fa4fc13f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:44.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa5044d6700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4fc064560 0x7fa4fc144a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:44.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa5044d6700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4fc13afa0 0x7fa4fc150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:44.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa5044d6700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc150100 con 0x7fa4fc064560 2022-06-17T12:56:44.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa5044d6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc143690 con 0x7fa4fc062bd0 2022-06-17T12:56:44.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa5044d6700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc143980 con 0x7fa4fc13afa0 2022-06-17T12:56:44.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa501a71700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4fc064560 0x7fa4fc144a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:44.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa501a71700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4fc064560 0x7fa4fc144a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39848/0 (socket says 172.21.15.36:39848) 2022-06-17T12:56:44.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa501a71700 1 -- 172.21.15.36:0/2384943719 learned_addr learned my addr 172.21.15.36:0/2384943719 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:44.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa502a73700 1 --2- 172.21.15.36:0/2384943719 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4fc13afa0 0x7fa4fc150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:44.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa502272700 1 --2- 172.21.15.36:0/2384943719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4fc062bd0 0x7fa4fc13f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:44.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa501a71700 1 -- 172.21.15.36:0/2384943719 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4fc13afa0 msgr2=0x7fa4fc150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:44.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa501a71700 1 --2- 172.21.15.36:0/2384943719 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4fc13afa0 0x7fa4fc150530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:44.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa501a71700 1 -- 172.21.15.36:0/2384943719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4fc062bd0 msgr2=0x7fa4fc13f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:44.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa501a71700 1 --2- 172.21.15.36:0/2384943719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4fc062bd0 0x7fa4fc13f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:44.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa501a71700 1 -- 172.21.15.36:0/2384943719 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ec00b180 con 0x7fa4fc064560 2022-06-17T12:56:44.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa501a71700 1 --2- 172.21.15.36:0/2384943719 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4fc064560 0x7fa4fc144a30 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 rx=0x7fa4ec002f70 tx=0x7fa4ec014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:44.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa4eb7fe700 1 -- 172.21.15.36:0/2384943719 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4ec016070 con 0x7fa4fc064560 2022-06-17T12:56:44.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa4eb7fe700 1 -- 172.21.15.36:0/2384943719 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4ec0137d0 con 0x7fa4fc064560 2022-06-17T12:56:44.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa5044d6700 1 -- 172.21.15.36:0/2384943719 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4fc13e7f0 con 0x7fa4fc064560 2022-06-17T12:56:44.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa4eb7fe700 1 -- 172.21.15.36:0/2384943719 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4ec013b30 con 0x7fa4fc064560 2022-06-17T12:56:44.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa5044d6700 1 -- 172.21.15.36:0/2384943719 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4fc13ed80 con 0x7fa4fc064560 2022-06-17T12:56:44.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa5044d6700 1 -- 172.21.15.36:0/2384943719 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4fc150270 con 0x7fa4fc064560 2022-06-17T12:56:44.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa4eb7fe700 1 -- 172.21.15.36:0/2384943719 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fa4ec025070 con 0x7fa4fc064560 2022-06-17T12:56:44.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa4eb7fe700 1 --2- 172.21.15.36:0/2384943719 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa4dc0539b0 0x7fa4dc055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:44.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.150+0000 7fa4eb7fe700 1 -- 172.21.15.36:0/2384943719 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fa4ec06cb20 con 0x7fa4fc064560 2022-06-17T12:56:44.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.154+0000 7fa502272700 1 --2- 172.21.15.36:0/2384943719 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa4dc0539b0 0x7fa4dc055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:44.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.154+0000 7fa502272700 1 --2- 172.21.15.36:0/2384943719 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa4dc0539b0 0x7fa4dc055e70 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7fa4f800d000 tx=0x7fa4f80079b0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:44.166 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.158+0000 7fa4eb7fe700 1 -- 172.21.15.36:0/2384943719 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa4fc150270 con 0x7fa4fc064560 2022-06-17T12:56:44.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:44.294+0000 7fa5044d6700 1 -- 172.21.15.36:0/2384943719 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fa4fc150240 con 0x7fa4fc064560 2022-06-17T12:56:45.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.146+0000 7fa4eb7fe700 1 -- 172.21.15.36:0/2384943719 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v137) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa4ec023030 con 0x7fa4fc064560 2022-06-17T12:56:45.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.150+0000 7fa5044d6700 1 -- 172.21.15.36:0/2384943719 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa4dc0539b0 msgr2=0x7fa4dc055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:45.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.150+0000 7fa5044d6700 1 --2- 172.21.15.36:0/2384943719 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa4dc0539b0 0x7fa4dc055e70 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7fa4f800d000 tx=0x7fa4f80079b0).stop 2022-06-17T12:56:45.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.150+0000 7fa5044d6700 1 -- 172.21.15.36:0/2384943719 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4fc064560 msgr2=0x7fa4fc144a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:45.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.150+0000 7fa5044d6700 1 --2- 172.21.15.36:0/2384943719 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4fc064560 0x7fa4fc144a30 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 rx=0x7fa4ec002f70 tx=0x7fa4ec014000).stop 2022-06-17T12:56:45.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.150+0000 7fa5044d6700 1 -- 172.21.15.36:0/2384943719 shutdown_connections 2022-06-17T12:56:45.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.150+0000 7fa5044d6700 1 --2- 172.21.15.36:0/2384943719 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa4dc0539b0 0x7fa4dc055e70 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:45.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.150+0000 7fa5044d6700 1 --2- 172.21.15.36:0/2384943719 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4fc13afa0 0x7fa4fc150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:45.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.150+0000 7fa5044d6700 1 --2- 172.21.15.36:0/2384943719 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa4fc064560 0x7fa4fc144a30 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:45.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.150+0000 7fa5044d6700 1 --2- 172.21.15.36:0/2384943719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4fc062bd0 0x7fa4fc13f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:45.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.150+0000 7fa5044d6700 1 -- 172.21.15.36:0/2384943719 >> 172.21.15.36:0/2384943719 conn(0x7fa4fc0b53e0 msgr2=0x7fa4fc05fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:45.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.150+0000 7fa5044d6700 1 -- 172.21.15.36:0/2384943719 shutdown_connections 2022-06-17T12:56:45.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.150+0000 7fa5044d6700 1 -- 172.21.15.36:0/2384943719 wait complete. 2022-06-17T12:56:45.145 INFO:teuthology.orchestra.run.smithi036.stderr: removed all blacklist entries 2022-06-17T12:56:45.149 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2022-06-17T12:56:45.150 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-06-17T12:56:45.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250ede7700 1 Processor -- start 2022-06-17T12:56:45.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250ede7700 1 -- start start 2022-06-17T12:56:45.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250ede7700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f25080b39a0 0x7f25081441e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:45.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250ede7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25080b2890 0x7f250813faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:45.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250ede7700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25080b6be0 con 0x7f25080b6800 2022-06-17T12:56:45.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250ede7700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2508140b30 con 0x7f25080b39a0 2022-06-17T12:56:45.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250ede7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25081446d0 con 0x7f25080b2890 2022-06-17T12:56:45.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250d5e4700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f25080b39a0 0x7f25081441e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:45.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250d5e4700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f25080b39a0 0x7f25081441e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39856/0 (socket says 172.21.15.36:39856) 2022-06-17T12:56:45.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250d5e4700 1 -- 172.21.15.36:0/2798630210 learned_addr learned my addr 172.21.15.36:0/2798630210 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:45.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250e5e6700 1 --2- 172.21.15.36:0/2798630210 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25080b2890 0x7f250813faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:45.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250d5e4700 1 -- 172.21.15.36:0/2798630210 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25080b2890 msgr2=0x7f250813faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:45.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250d5e4700 1 --2- 172.21.15.36:0/2798630210 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25080b2890 0x7f250813faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:45.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250d5e4700 1 -- 172.21.15.36:0/2798630210 >> v1:172.21.15.70:6789/0 conn(0x7f25080b6800 legacy=0x7f25080b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:45.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250d5e4700 1 -- 172.21.15.36:0/2798630210 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25080af6c0 con 0x7f25080b39a0 2022-06-17T12:56:45.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250e5e6700 1 --2- 172.21.15.36:0/2798630210 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25080b2890 0x7f250813faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:56:45.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250d5e4700 1 --2- 172.21.15.36:0/2798630210 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f25080b39a0 0x7f25081441e0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7f24fc002f70 tx=0x7f24fc006e50).ready entity=mon.0 client_cookie=b268316f636d1cec server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:45.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250cde3700 1 -- 172.21.15.36:0/2798630210 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24fc00d5c0 con 0x7f25080b39a0 2022-06-17T12:56:45.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250cde3700 1 -- 172.21.15.36:0/2798630210 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24fc000e90 con 0x7f25080b39a0 2022-06-17T12:56:45.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250cde3700 1 -- 172.21.15.36:0/2798630210 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24fc00d950 con 0x7f25080b39a0 2022-06-17T12:56:45.278 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250ede7700 1 -- 172.21.15.36:0/2798630210 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f25080b39a0 msgr2=0x7f25081441e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:45.278 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250ede7700 1 --2- 172.21.15.36:0/2798630210 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f25080b39a0 0x7f25081441e0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7f24fc002f70 tx=0x7f24fc006e50).stop 2022-06-17T12:56:45.278 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250ede7700 1 -- 172.21.15.36:0/2798630210 shutdown_connections 2022-06-17T12:56:45.278 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250ede7700 1 --2- 172.21.15.36:0/2798630210 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25080b2890 0x7f250813faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:45.278 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250ede7700 1 --2- 172.21.15.36:0/2798630210 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f25080b39a0 0x7f25081441e0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:45.279 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.286+0000 7f250ede7700 1 -- 172.21.15.36:0/2798630210 >> 172.21.15.36:0/2798630210 conn(0x7f25080614b0 msgr2=0x7f2508061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:45.279 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.290+0000 7f250ede7700 1 -- 172.21.15.36:0/2798630210 shutdown_connections 2022-06-17T12:56:45.279 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.290+0000 7f250ede7700 1 -- 172.21.15.36:0/2798630210 wait complete. 2022-06-17T12:56:45.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.290+0000 7f250ede7700 1 Processor -- start 2022-06-17T12:56:45.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.290+0000 7f250ede7700 1 -- start start 2022-06-17T12:56:45.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.294+0000 7f250ede7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25080b2890 0x7f2508147da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:45.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.294+0000 7f250ede7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f25080b39a0 0x7f2508153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:45.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.294+0000 7f250ede7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25080b6800 0x7f2508153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:45.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.294+0000 7f250ede7700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25080669e0 con 0x7f25080b39a0 2022-06-17T12:56:45.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.294+0000 7f250ede7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2508066c10 con 0x7f25080b2890 2022-06-17T12:56:45.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.294+0000 7f250ede7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2508066eb0 con 0x7f25080b6800 2022-06-17T12:56:45.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.294+0000 7f250d5e4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f25080b39a0 0x7f2508153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:45.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.294+0000 7f250d5e4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f25080b39a0 0x7f2508153630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39862/0 (socket says 172.21.15.36:39862) 2022-06-17T12:56:45.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.294+0000 7f250d5e4700 1 -- 172.21.15.36:0/4242891427 learned_addr learned my addr 172.21.15.36:0/4242891427 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:45.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.294+0000 7f250dde5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25080b2890 0x7f2508147da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:45.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.294+0000 7f250e5e6700 1 --2- 172.21.15.36:0/4242891427 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25080b6800 0x7f2508153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:45.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.294+0000 7f250d5e4700 1 -- 172.21.15.36:0/4242891427 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25080b6800 msgr2=0x7f2508153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:45.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.294+0000 7f250d5e4700 1 --2- 172.21.15.36:0/4242891427 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25080b6800 0x7f2508153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:45.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.294+0000 7f250d5e4700 1 -- 172.21.15.36:0/4242891427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25080b2890 msgr2=0x7f2508147da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:45.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.294+0000 7f250d5e4700 1 --2- 172.21.15.36:0/4242891427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25080b2890 0x7f2508147da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:45.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.294+0000 7f250d5e4700 1 -- 172.21.15.36:0/4242891427 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24fc00b1e0 con 0x7f25080b39a0 2022-06-17T12:56:45.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.294+0000 7f250d5e4700 1 --2- 172.21.15.36:0/4242891427 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f25080b39a0 0x7f2508153630 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0x7f24fc002f70 tx=0x7f24fc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:45.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.294+0000 7f24faffd700 1 -- 172.21.15.36:0/4242891427 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24fc011960 con 0x7f25080b39a0 2022-06-17T12:56:45.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.294+0000 7f24faffd700 1 -- 172.21.15.36:0/4242891427 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24fc011ac0 con 0x7f25080b39a0 2022-06-17T12:56:45.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.294+0000 7f250ede7700 1 -- 172.21.15.36:0/4242891427 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25080648e0 con 0x7f25080b39a0 2022-06-17T12:56:45.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.294+0000 7f24faffd700 1 -- 172.21.15.36:0/4242891427 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24fc011e20 con 0x7f25080b39a0 2022-06-17T12:56:45.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.294+0000 7f250ede7700 1 -- 172.21.15.36:0/4242891427 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25080652a0 con 0x7f25080b39a0 2022-06-17T12:56:45.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.298+0000 7f250ede7700 1 -- 172.21.15.36:0/4242891427 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25080af960 con 0x7f25080b39a0 2022-06-17T12:56:45.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.298+0000 7f24faffd700 1 -- 172.21.15.36:0/4242891427 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f24fc01a040 con 0x7f25080b39a0 2022-06-17T12:56:45.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.298+0000 7f24faffd700 1 --2- 172.21.15.36:0/4242891427 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f24e8053a40 0x7f24e8055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:45.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.298+0000 7f24faffd700 1 -- 172.21.15.36:0/4242891427 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f24fc06cb00 con 0x7f25080b39a0 2022-06-17T12:56:45.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.298+0000 7f250dde5700 1 --2- 172.21.15.36:0/4242891427 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f24e8053a40 0x7f24e8055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:45.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.298+0000 7f250dde5700 1 --2- 172.21.15.36:0/4242891427 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f24e8053a40 0x7f24e8055f00 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f250400d000 tx=0x7f2504007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:45.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.302+0000 7f24faffd700 1 -- 172.21.15.36:0/4242891427 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f24fc043c40 con 0x7f25080b39a0 2022-06-17T12:56:45.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.474+0000 7f250ede7700 1 -- 172.21.15.36:0/4242891427 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2508057410 con 0x7f25080b39a0 2022-06-17T12:56:45.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.474+0000 7f24faffd700 1 -- 172.21.15.36:0/4242891427 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v43) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f24fc023020 con 0x7f25080b39a0 2022-06-17T12:56:45.462 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:45.463 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:56:45.464 INFO:teuthology.orchestra.run.smithi036.stdout: "auth_dump": [ 2022-06-17T12:56:45.464 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:45.464 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mds.a", 2022-06-17T12:56:45.465 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiyaUgARAAxp7bzOl4uVo2EsaoRBeMRw==", 2022-06-17T12:56:45.465 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:45.466 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:56:45.466 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:56:45.467 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:56:45.468 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:45.469 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:45.471 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:45.472 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:45.472 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mds.b", 2022-06-17T12:56:45.473 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xifyxLCRAABI389jmz1zboeDanzraT8Q==", 2022-06-17T12:56:45.473 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:45.474 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:56:45.474 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:56:45.474 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:56:45.475 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:45.475 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:45.476 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:45.476 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:45.476 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.0", 2022-06-17T12:56:45.477 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB+d6xibhXGNBAA6AuUT9ITWAZjDFpNsgA4Wg==", 2022-06-17T12:56:45.477 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:45.478 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:56:45.478 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:56:45.478 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:45.479 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:45.479 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:45.495 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:45.495 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.1", 2022-06-17T12:56:45.496 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCBd6xiOIPvGxAADkrBoJCy9x91zan6TWU0tQ==", 2022-06-17T12:56:45.496 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:45.497 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:56:45.497 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:56:45.497 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:45.498 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:45.498 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:45.499 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:45.499 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.2", 2022-06-17T12:56:45.500 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCEd6xiP4/9AhAAaP3cWXV6vbfGBPfqSLKqUg==", 2022-06-17T12:56:45.500 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:45.501 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:56:45.501 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:56:45.502 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:45.502 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:45.503 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:45.503 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:45.505 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.3", 2022-06-17T12:56:45.505 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCGd6xi+LZgJxAAJjGwP5qW6TFDZfR+2rz6ig==", 2022-06-17T12:56:45.506 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:45.506 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:56:45.506 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:56:45.507 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:45.507 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:45.508 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:45.508 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:45.509 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.4", 2022-06-17T12:56:45.509 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCKd6xikvrTChAAXIyWSETswDv+tb3zsGyYpA==", 2022-06-17T12:56:45.510 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:45.511 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:56:45.523 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:56:45.523 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:45.524 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:45.524 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:45.525 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:45.525 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.5", 2022-06-17T12:56:45.526 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCMd6xiqr2QLhAArhimHuNsVWrDGGreFYxj6A==", 2022-06-17T12:56:45.526 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:45.527 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:56:45.527 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:56:45.528 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:45.528 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:45.529 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:45.530 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:45.530 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.6", 2022-06-17T12:56:45.531 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCPd6xiUXcXGxAAdrYz8txdBWMIqF5DF4Aimw==", 2022-06-17T12:56:45.531 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:45.532 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:56:45.536 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:56:45.536 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:45.537 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:45.537 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:45.538 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:45.538 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.7", 2022-06-17T12:56:45.539 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCSd6xiBJ2IAhAAUkNfgmlxxWY7+VouPLAvyw==", 2022-06-17T12:56:45.539 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:45.540 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:56:45.540 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:56:45.540 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:45.541 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:45.541 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:45.542 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:45.542 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.0", 2022-06-17T12:56:45.544 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiXO3QFBAAsSLtQ0yKKZxhqUT/eYKdtg==", 2022-06-17T12:56:45.544 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:45.545 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:56:45.545 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:56:45.546 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:56:45.546 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:45.547 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:45.575 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:45.581 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.1", 2022-06-17T12:56:45.582 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xi51KdGBAAi7fGZ+Qepg6j3DAhlzoWxw==", 2022-06-17T12:56:45.582 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:45.583 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:56:45.583 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:56:45.584 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:56:45.584 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:45.585 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:45.585 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:45.589 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.2", 2022-06-17T12:56:45.590 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiug2hHBAAqHpLgwMdthSWtJ8Vwu+m1Q==", 2022-06-17T12:56:45.590 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:45.590 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:56:45.593 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:56:45.593 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:56:45.593 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:45.599 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:45.600 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:45.600 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.admin", 2022-06-17T12:56:45.600 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB5d6xiGDODKhAAPZMG/M2nVssdr4FJHcS6XQ==", 2022-06-17T12:56:45.600 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:45.601 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:56:45.601 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:56:45.601 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:56:45.601 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:45.602 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:45.602 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:45.602 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:45.602 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mgr.x", 2022-06-17T12:56:45.603 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB6d6xiyH9uORAATJb0chHE1CH0X02w7edVtw==", 2022-06-17T12:56:45.603 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:45.603 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:56:45.604 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile mgr", 2022-06-17T12:56:45.604 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:45.604 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:45.604 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:45.604 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:45.605 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mgr.y", 2022-06-17T12:56:45.605 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB6d6xiHXghMxAAzpsVRCbLWHWZPtrPk1jFlA==", 2022-06-17T12:56:45.605 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:56:45.605 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:56:45.609 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile mgr", 2022-06-17T12:56:45.609 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:56:45.609 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:45.614 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:45.614 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:56:45.614 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:56:45.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.478+0000 7f250ede7700 1 -- 172.21.15.36:0/4242891427 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f24e8053a40 msgr2=0x7f24e8055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:45.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.478+0000 7f250ede7700 1 --2- 172.21.15.36:0/4242891427 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f24e8053a40 0x7f24e8055f00 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f250400d000 tx=0x7f2504007b90).stop 2022-06-17T12:56:45.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.478+0000 7f250ede7700 1 -- 172.21.15.36:0/4242891427 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f25080b39a0 msgr2=0x7f2508153630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:45.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.478+0000 7f250ede7700 1 --2- 172.21.15.36:0/4242891427 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f25080b39a0 0x7f2508153630 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0x7f24fc002f70 tx=0x7f24fc015000).stop 2022-06-17T12:56:45.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.478+0000 7f250ede7700 1 -- 172.21.15.36:0/4242891427 shutdown_connections 2022-06-17T12:56:45.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.478+0000 7f250ede7700 1 --2- 172.21.15.36:0/4242891427 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f24e8053a40 0x7f24e8055f00 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:45.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.478+0000 7f250ede7700 1 --2- 172.21.15.36:0/4242891427 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25080b6800 0x7f2508153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:45.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.478+0000 7f250ede7700 1 --2- 172.21.15.36:0/4242891427 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f25080b39a0 0x7f2508153630 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:45.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.478+0000 7f250ede7700 1 --2- 172.21.15.36:0/4242891427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25080b2890 0x7f2508147da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:45.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.478+0000 7f250ede7700 1 -- 172.21.15.36:0/4242891427 >> 172.21.15.36:0/4242891427 conn(0x7f25080614b0 msgr2=0x7f2508144810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:45.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.478+0000 7f250ede7700 1 -- 172.21.15.36:0/4242891427 shutdown_connections 2022-06-17T12:56:45.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.478+0000 7f250ede7700 1 -- 172.21.15.36:0/4242891427 wait complete. 2022-06-17T12:56:45.620 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:56:45.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.746+0000 7f1d61936700 1 Processor -- start 2022-06-17T12:56:45.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.746+0000 7f1d61936700 1 -- start start 2022-06-17T12:56:45.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d61936700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d5c0af540 0x7f1d5c14b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:45.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d61936700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d5c0b3140 con 0x7f1d5c060ac0 2022-06-17T12:56:45.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d61936700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d5c060ea0 con 0x7f1d5c0afd80 2022-06-17T12:56:45.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d61936700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c1441e0 con 0x7f1d5c0af540 2022-06-17T12:56:45.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d60934700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f1d5c060ac0 0x7f1d5c144b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57400/0 (socket says 172.21.15.36:57400) 2022-06-17T12:56:45.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d60934700 1 -- 172.21.15.36:0/867956113 learned_addr learned my addr 172.21.15.36:0/867956113 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:45.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d61135700 1 --2- 172.21.15.36:0/867956113 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d5c0af540 0x7f1d5c14b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:45.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d5b7fe700 1 -- 172.21.15.36:0/867956113 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 567887018 0 0) 0x7f1d5c0b3140 con 0x7f1d5c060ac0 2022-06-17T12:56:45.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d5b7fe700 1 -- 172.21.15.36:0/867956113 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d40003260 con 0x7f1d5c060ac0 2022-06-17T12:56:45.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d5b7fe700 1 -- 172.21.15.36:0/867956113 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2563978123 0 0) 0x7f1d5c060ea0 con 0x7f1d5c0afd80 2022-06-17T12:56:45.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d5b7fe700 1 -- 172.21.15.36:0/867956113 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d5c0b3140 con 0x7f1d5c0afd80 2022-06-17T12:56:45.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d61135700 1 -- 172.21.15.36:0/867956113 >> v1:172.21.15.70:6789/0 conn(0x7f1d5c0afd80 legacy=0x7f1d5c140bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:45.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d61135700 1 -- 172.21.15.36:0/867956113 >> v1:172.21.15.36:6789/0 conn(0x7f1d5c060ac0 legacy=0x7f1d5c144b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:45.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d61135700 1 -- 172.21.15.36:0/867956113 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5c1443d0 con 0x7f1d5c0af540 2022-06-17T12:56:45.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d61135700 1 --2- 172.21.15.36:0/867956113 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d5c0af540 0x7f1d5c14b790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f1d540032f0 tx=0x7f1d540091d0).ready entity=mon.2 client_cookie=77a9e8e8094dce0d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:45.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d5b7fe700 1 -- 172.21.15.36:0/867956113 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d5400b640 con 0x7f1d5c0af540 2022-06-17T12:56:45.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d5b7fe700 1 -- 172.21.15.36:0/867956113 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5400b7a0 con 0x7f1d5c0af540 2022-06-17T12:56:45.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d5b7fe700 1 -- 172.21.15.36:0/867956113 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d5400b910 con 0x7f1d5c0af540 2022-06-17T12:56:45.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d61936700 1 -- 172.21.15.36:0/867956113 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d5c0af540 msgr2=0x7f1d5c14b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:45.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d61936700 1 --2- 172.21.15.36:0/867956113 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d5c0af540 0x7f1d5c14b790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f1d540032f0 tx=0x7f1d540091d0).stop 2022-06-17T12:56:45.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d61936700 1 -- 172.21.15.36:0/867956113 shutdown_connections 2022-06-17T12:56:45.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d61936700 1 --2- 172.21.15.36:0/867956113 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d5c0af540 0x7f1d5c14b790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:45.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d61936700 1 -- 172.21.15.36:0/867956113 >> 172.21.15.36:0/867956113 conn(0x7f1d5c0b65f0 msgr2=0x7f1d5c0b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:45.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d61936700 1 -- 172.21.15.36:0/867956113 shutdown_connections 2022-06-17T12:56:45.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.750+0000 7f1d61936700 1 -- 172.21.15.36:0/867956113 wait complete. 2022-06-17T12:56:45.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.754+0000 7f1d61936700 1 Processor -- start 2022-06-17T12:56:45.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.754+0000 7f1d61936700 1 -- start start 2022-06-17T12:56:45.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.754+0000 7f1d61936700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d5c060ac0 0x7f1d5c13fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:45.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.754+0000 7f1d61936700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d5c0af540 0x7f1d5c138300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:45.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.754+0000 7f1d61936700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d5c0afd80 0x7f1d5c1387f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:45.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.754+0000 7f1d61936700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c13a700 con 0x7f1d5c0afd80 2022-06-17T12:56:45.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.754+0000 7f1d61936700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c13a460 con 0x7f1d5c060ac0 2022-06-17T12:56:45.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.754+0000 7f1d61936700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c1501a0 con 0x7f1d5c0af540 2022-06-17T12:56:45.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.754+0000 7f1d61135700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d5c0afd80 0x7f1d5c1387f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:45.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.754+0000 7f1d61135700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d5c0afd80 0x7f1d5c1387f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39878/0 (socket says 172.21.15.36:39878) 2022-06-17T12:56:45.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.754+0000 7f1d60934700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d5c060ac0 0x7f1d5c13fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:45.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.754+0000 7f1d61135700 1 -- 172.21.15.36:0/3514329200 learned_addr learned my addr 172.21.15.36:0/3514329200 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:45.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.754+0000 7f1d5bfff700 1 --2- 172.21.15.36:0/3514329200 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d5c0af540 0x7f1d5c138300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:45.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.754+0000 7f1d61135700 1 -- 172.21.15.36:0/3514329200 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d5c0af540 msgr2=0x7f1d5c138300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:45.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.754+0000 7f1d61135700 1 --2- 172.21.15.36:0/3514329200 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d5c0af540 0x7f1d5c138300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:45.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.754+0000 7f1d61135700 1 -- 172.21.15.36:0/3514329200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d5c060ac0 msgr2=0x7f1d5c13fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:45.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.754+0000 7f1d61135700 1 --2- 172.21.15.36:0/3514329200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d5c060ac0 0x7f1d5c13fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:45.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.754+0000 7f1d61135700 1 -- 172.21.15.36:0/3514329200 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d54009470 con 0x7f1d5c0afd80 2022-06-17T12:56:45.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.754+0000 7f1d61135700 1 --2- 172.21.15.36:0/3514329200 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d5c0afd80 0x7f1d5c1387f0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7f1d540032f0 tx=0x7f1d54016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:45.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.754+0000 7f1d59ffb700 1 -- 172.21.15.36:0/3514329200 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d5400b910 con 0x7f1d5c0afd80 2022-06-17T12:56:45.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.754+0000 7f1d59ffb700 1 -- 172.21.15.36:0/3514329200 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d54011de0 con 0x7f1d5c0afd80 2022-06-17T12:56:45.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.754+0000 7f1d61936700 1 -- 172.21.15.36:0/3514329200 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d5c138ce0 con 0x7f1d5c0afd80 2022-06-17T12:56:45.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.754+0000 7f1d59ffb700 1 -- 172.21.15.36:0/3514329200 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d5400b640 con 0x7f1d5c0afd80 2022-06-17T12:56:45.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.754+0000 7f1d61936700 1 -- 172.21.15.36:0/3514329200 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d5c13cc50 con 0x7f1d5c0afd80 2022-06-17T12:56:45.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.758+0000 7f1d61936700 1 -- 172.21.15.36:0/3514329200 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d5c0658d0 con 0x7f1d5c0afd80 2022-06-17T12:56:45.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.758+0000 7f1d59ffb700 1 -- 172.21.15.36:0/3514329200 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f1d5400f070 con 0x7f1d5c0afd80 2022-06-17T12:56:45.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.758+0000 7f1d59ffb700 1 --2- 172.21.15.36:0/3514329200 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1d3c053a60 0x7f1d3c055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:45.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.758+0000 7f1d59ffb700 1 -- 172.21.15.36:0/3514329200 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f1d5406c680 con 0x7f1d5c0afd80 2022-06-17T12:56:45.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.758+0000 7f1d60934700 1 --2- 172.21.15.36:0/3514329200 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1d3c053a60 0x7f1d3c055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:45.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.758+0000 7f1d60934700 1 --2- 172.21.15.36:0/3514329200 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1d3c053a60 0x7f1d3c055f20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f1d5000d000 tx=0x7f1d50009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:45.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.766+0000 7f1d59ffb700 1 -- 172.21.15.36:0/3514329200 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1d5c0658d0 con 0x7f1d5c0afd80 2022-06-17T12:56:45.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.894+0000 7f1d61936700 1 -- 172.21.15.36:0/3514329200 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d5c0658d0 con 0x7f1d5c0afd80 2022-06-17T12:56:45.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.894+0000 7f1d59ffb700 1 -- 172.21.15.36:0/3514329200 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+13424 (secure 0 0 0) 0x7f1d54030480 con 0x7f1d5c0afd80 2022-06-17T12:56:45.932 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:45.933 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":137,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:56:45.139797+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:56:45.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.898+0000 7f1d61936700 1 -- 172.21.15.36:0/3514329200 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1d3c053a60 msgr2=0x7f1d3c055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:45.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.898+0000 7f1d61936700 1 --2- 172.21.15.36:0/3514329200 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1d3c053a60 0x7f1d3c055f20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f1d5000d000 tx=0x7f1d50009040).stop 2022-06-17T12:56:45.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.898+0000 7f1d61936700 1 -- 172.21.15.36:0/3514329200 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d5c0afd80 msgr2=0x7f1d5c1387f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:45.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.898+0000 7f1d61936700 1 --2- 172.21.15.36:0/3514329200 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d5c0afd80 0x7f1d5c1387f0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7f1d540032f0 tx=0x7f1d54016000).stop 2022-06-17T12:56:45.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.898+0000 7f1d61936700 1 -- 172.21.15.36:0/3514329200 shutdown_connections 2022-06-17T12:56:45.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.898+0000 7f1d61936700 1 --2- 172.21.15.36:0/3514329200 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1d3c053a60 0x7f1d3c055f20 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:45.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.898+0000 7f1d61936700 1 --2- 172.21.15.36:0/3514329200 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1d5c0afd80 0x7f1d5c1387f0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:45.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.898+0000 7f1d61936700 1 --2- 172.21.15.36:0/3514329200 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1d5c0af540 0x7f1d5c138300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:45.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.898+0000 7f1d61936700 1 --2- 172.21.15.36:0/3514329200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d5c060ac0 0x7f1d5c13fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:45.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.898+0000 7f1d61936700 1 -- 172.21.15.36:0/3514329200 >> 172.21.15.36:0/3514329200 conn(0x7f1d5c0b65f0 msgr2=0x7f1d5c0b4840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:45.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.898+0000 7f1d61936700 1 -- 172.21.15.36:0/3514329200 shutdown_connections 2022-06-17T12:56:45.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:45.898+0000 7f1d61936700 1 -- 172.21.15.36:0/3514329200 wait complete. 2022-06-17T12:56:45.961 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-17T12:56:45.964 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:56:46.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.094+0000 7f8fcc0d8700 1 Processor -- start 2022-06-17T12:56:46.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.094+0000 7f8fcc0d8700 1 -- start start 2022-06-17T12:56:46.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.094+0000 7f8fcc0d8700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8fc40b64f0 0x7f8fc4143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:46.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.094+0000 7f8fcc0d8700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8fc40b4120 0x7f8fc4146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:46.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.094+0000 7f8fcc0d8700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc41476e0 con 0x7f8fc4147270 2022-06-17T12:56:46.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.094+0000 7f8fcc0d8700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc40b68d0 con 0x7f8fc40b64f0 2022-06-17T12:56:46.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.094+0000 7f8fcc0d8700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc40b6e20 con 0x7f8fc40b4120 2022-06-17T12:56:46.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.098+0000 7f8fc9e74700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8fc40b64f0 0x7f8fc4143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:46.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.098+0000 7f8fca675700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f8fc4147270 0x7f8fc4140100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57416/0 (socket says 172.21.15.36:57416) 2022-06-17T12:56:46.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.098+0000 7f8fca675700 1 -- 172.21.15.36:0/3109227033 learned_addr learned my addr 172.21.15.36:0/3109227033 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:46.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.098+0000 7f8fc9673700 1 --2- 172.21.15.36:0/3109227033 >> v2:172.21.15.70:3300/0 conn(0x7f8fc40b4120 0x7f8fc4146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:46.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.098+0000 7f8fc9e74700 1 -- 172.21.15.36:0/3109227033 >> v2:172.21.15.70:3300/0 conn(0x7f8fc40b4120 msgr2=0x7f8fc4146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:46.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.098+0000 7f8fc9e74700 1 --2- 172.21.15.36:0/3109227033 >> v2:172.21.15.70:3300/0 conn(0x7f8fc40b4120 0x7f8fc4146d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.122 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.098+0000 7f8fc9e74700 1 -- 172.21.15.36:0/3109227033 >> v1:172.21.15.36:6789/0 conn(0x7f8fc4147270 legacy=0x7f8fc4140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:46.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.098+0000 7f8fc9e74700 1 -- 172.21.15.36:0/3109227033 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc41468e0 con 0x7f8fc40b64f0 2022-06-17T12:56:46.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.098+0000 7f8fc8e72700 1 -- 172.21.15.36:0/3109227033 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351116018 0 0) 0x7f8fc41476e0 con 0x7f8fc4147270 2022-06-17T12:56:46.124 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.098+0000 7f8fc9e74700 1 --2- 172.21.15.36:0/3109227033 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fc40b64f0 0x7f8fc4143150 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0x7f8fc00032f0 tx=0x7f8fc00091d0).ready entity=mon.0 client_cookie=532064039a8c0f60 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:46.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.098+0000 7f8fc8e72700 1 -- 172.21.15.36:0/3109227033 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fc0009ea0 con 0x7f8fc40b64f0 2022-06-17T12:56:46.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.098+0000 7f8fc8e72700 1 -- 172.21.15.36:0/3109227033 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc000b870 con 0x7f8fc40b64f0 2022-06-17T12:56:46.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.098+0000 7f8fc8e72700 1 -- 172.21.15.36:0/3109227033 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fc000b9e0 con 0x7f8fc40b64f0 2022-06-17T12:56:46.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.098+0000 7f8fcc0d8700 1 -- 172.21.15.36:0/3109227033 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fc40b64f0 msgr2=0x7f8fc4143150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:46.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.098+0000 7f8fcc0d8700 1 --2- 172.21.15.36:0/3109227033 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fc40b64f0 0x7f8fc4143150 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0x7f8fc00032f0 tx=0x7f8fc00091d0).stop 2022-06-17T12:56:46.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.098+0000 7f8fcc0d8700 1 -- 172.21.15.36:0/3109227033 shutdown_connections 2022-06-17T12:56:46.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.098+0000 7f8fcc0d8700 1 --2- 172.21.15.36:0/3109227033 >> v2:172.21.15.70:3300/0 conn(0x7f8fc40b4120 0x7f8fc4146d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.098+0000 7f8fcc0d8700 1 --2- 172.21.15.36:0/3109227033 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fc40b64f0 0x7f8fc4143150 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.098+0000 7f8fcc0d8700 1 -- 172.21.15.36:0/3109227033 >> 172.21.15.36:0/3109227033 conn(0x7f8fc40620f0 msgr2=0x7f8fc4062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:46.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.098+0000 7f8fcc0d8700 1 -- 172.21.15.36:0/3109227033 shutdown_connections 2022-06-17T12:56:46.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.098+0000 7f8fcc0d8700 1 -- 172.21.15.36:0/3109227033 wait complete. 2022-06-17T12:56:46.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.102+0000 7f8fcc0d8700 1 Processor -- start 2022-06-17T12:56:46.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.102+0000 7f8fcc0d8700 1 -- start start 2022-06-17T12:56:46.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.102+0000 7f8fcc0d8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fc40b4120 0x7f8fc40b1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:46.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.102+0000 7f8fcc0d8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc40b64f0 0x7f8fc4159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:46.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.102+0000 7f8fcc0d8700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fc4147270 0x7f8fc415a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:46.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.102+0000 7f8fcc0d8700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc40b29d0 con 0x7f8fc40b4120 2022-06-17T12:56:46.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.102+0000 7f8fcc0d8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc40b0b40 con 0x7f8fc40b64f0 2022-06-17T12:56:46.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.102+0000 7f8fcc0d8700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc40b0580 con 0x7f8fc4147270 2022-06-17T12:56:46.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.102+0000 7f8fc9e74700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fc40b4120 0x7f8fc40b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:46.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.102+0000 7f8fc9e74700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fc40b4120 0x7f8fc40b1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39888/0 (socket says 172.21.15.36:39888) 2022-06-17T12:56:46.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.102+0000 7f8fc9e74700 1 -- 172.21.15.36:0/4252465043 learned_addr learned my addr 172.21.15.36:0/4252465043 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:46.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.102+0000 7f8fc9673700 1 --2- 172.21.15.36:0/4252465043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc40b64f0 0x7f8fc4159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:46.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.102+0000 7f8fca675700 1 --2- 172.21.15.36:0/4252465043 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fc4147270 0x7f8fc415a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:46.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.102+0000 7f8fc9e74700 1 -- 172.21.15.36:0/4252465043 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fc4147270 msgr2=0x7f8fc415a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:46.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.102+0000 7f8fc9e74700 1 --2- 172.21.15.36:0/4252465043 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fc4147270 0x7f8fc415a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.102+0000 7f8fc9e74700 1 -- 172.21.15.36:0/4252465043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc40b64f0 msgr2=0x7f8fc4159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:46.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.102+0000 7f8fc9e74700 1 --2- 172.21.15.36:0/4252465043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc40b64f0 0x7f8fc4159e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.102+0000 7f8fc9e74700 1 -- 172.21.15.36:0/4252465043 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc0009470 con 0x7f8fc40b4120 2022-06-17T12:56:46.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.102+0000 7f8fc9e74700 1 --2- 172.21.15.36:0/4252465043 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fc40b4120 0x7f8fc40b1850 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0x7f8fc00032f0 tx=0x7f8fc0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:46.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.102+0000 7f8fb2ffd700 1 -- 172.21.15.36:0/4252465043 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fc0011ea0 con 0x7f8fc40b4120 2022-06-17T12:56:46.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.102+0000 7f8fb2ffd700 1 -- 172.21.15.36:0/4252465043 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc000b9e0 con 0x7f8fc40b4120 2022-06-17T12:56:46.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.102+0000 7f8fcc0d8700 1 -- 172.21.15.36:0/4252465043 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc40b10b0 con 0x7f8fc40b4120 2022-06-17T12:56:46.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.102+0000 7f8fb2ffd700 1 -- 172.21.15.36:0/4252465043 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fc001f5e0 con 0x7f8fc40b4120 2022-06-17T12:56:46.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.102+0000 7f8fcc0d8700 1 -- 172.21.15.36:0/4252465043 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc40b1d40 con 0x7f8fc40b4120 2022-06-17T12:56:46.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.106+0000 7f8fb2ffd700 1 -- 172.21.15.36:0/4252465043 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f8fc0026030 con 0x7f8fc40b4120 2022-06-17T12:56:46.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.106+0000 7f8fcc0d8700 1 -- 172.21.15.36:0/4252465043 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc40b1240 con 0x7f8fc40b4120 2022-06-17T12:56:46.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.106+0000 7f8fb2ffd700 1 --2- 172.21.15.36:0/4252465043 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8fa4053a40 0x7f8fa4055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:46.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.106+0000 7f8fb2ffd700 1 -- 172.21.15.36:0/4252465043 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f8fc006ca00 con 0x7f8fc40b4120 2022-06-17T12:56:46.165 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.106+0000 7f8fc9673700 1 --2- 172.21.15.36:0/4252465043 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8fa4053a40 0x7f8fa4055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:46.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.106+0000 7f8fc9673700 1 --2- 172.21.15.36:0/4252465043 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8fa4053a40 0x7f8fa4055f00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f8fb400c000 tx=0x7f8fb4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:46.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.114+0000 7f8fb2ffd700 1 -- 172.21.15.36:0/4252465043 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8fc0043b40 con 0x7f8fc40b4120 2022-06-17T12:56:46.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.234+0000 7f8fcc0d8700 1 -- 172.21.15.36:0/4252465043 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f8fc4143640 con 0x7f8fc40b4120 2022-06-17T12:56:46.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.234+0000 7f8fb2ffd700 1 -- 172.21.15.36:0/4252465043 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v137) v1 ==== 110+0+10 (secure 0 0 0) 0x7f8fc00240b0 con 0x7f8fc40b4120 2022-06-17T12:56:46.223 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:56:46.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.234+0000 7f8fcc0d8700 1 -- 172.21.15.36:0/4252465043 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8fa4053a40 msgr2=0x7f8fa4055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:46.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.234+0000 7f8fcc0d8700 1 --2- 172.21.15.36:0/4252465043 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8fa4053a40 0x7f8fa4055f00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f8fb400c000 tx=0x7f8fb4008040).stop 2022-06-17T12:56:46.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.234+0000 7f8fcc0d8700 1 -- 172.21.15.36:0/4252465043 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fc40b4120 msgr2=0x7f8fc40b1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:46.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.234+0000 7f8fcc0d8700 1 --2- 172.21.15.36:0/4252465043 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fc40b4120 0x7f8fc40b1850 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0x7f8fc00032f0 tx=0x7f8fc0016000).stop 2022-06-17T12:56:46.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.238+0000 7f8fcc0d8700 1 -- 172.21.15.36:0/4252465043 shutdown_connections 2022-06-17T12:56:46.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.238+0000 7f8fcc0d8700 1 --2- 172.21.15.36:0/4252465043 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f8fa4053a40 0x7f8fa4055f00 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.238+0000 7f8fcc0d8700 1 --2- 172.21.15.36:0/4252465043 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fc4147270 0x7f8fc415a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.238+0000 7f8fcc0d8700 1 --2- 172.21.15.36:0/4252465043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc40b64f0 0x7f8fc4159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.238+0000 7f8fcc0d8700 1 --2- 172.21.15.36:0/4252465043 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8fc40b4120 0x7f8fc40b1850 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.238+0000 7f8fcc0d8700 1 -- 172.21.15.36:0/4252465043 >> 172.21.15.36:0/4252465043 conn(0x7f8fc40620f0 msgr2=0x7f8fc40624d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:46.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.238+0000 7f8fcc0d8700 1 -- 172.21.15.36:0/4252465043 shutdown_connections 2022-06-17T12:56:46.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.238+0000 7f8fcc0d8700 1 -- 172.21.15.36:0/4252465043 wait complete. 2022-06-17T12:56:46.239 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:56:46.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.366+0000 7fca4f59e700 1 Processor -- start 2022-06-17T12:56:46.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.366+0000 7fca4f59e700 1 -- start start 2022-06-17T12:56:46.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.366+0000 7fca4f59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fca500b3f80 0x7fca500b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:46.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.366+0000 7fca4f59e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fca500b4b90 0x7fca5013ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:46.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.366+0000 7fca4f59e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca501432c0 con 0x7fca50060020 2022-06-17T12:56:46.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.366+0000 7fca4f59e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca50062db0 con 0x7fca500b4b90 2022-06-17T12:56:46.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.366+0000 7fca4f59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca50143df0 con 0x7fca500b3f80 2022-06-17T12:56:46.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.366+0000 7fca4dd9b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fca500b3f80 0x7fca500b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:46.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.366+0000 7fca4dd9b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fca500b3f80 0x7fca500b46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:54688/0 (socket says 172.21.15.36:54688) 2022-06-17T12:56:46.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.366+0000 7fca4dd9b700 1 -- 172.21.15.36:0/972042265 learned_addr learned my addr 172.21.15.36:0/972042265 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:46.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.366+0000 7fca4ed9d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fca500b4b90 0x7fca5013ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:46.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.366+0000 7fca4dd9b700 1 -- 172.21.15.36:0/972042265 >> v2:172.21.15.70:3300/0 conn(0x7fca500b4b90 msgr2=0x7fca5013ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:46.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.366+0000 7fca4dd9b700 1 --2- 172.21.15.36:0/972042265 >> v2:172.21.15.70:3300/0 conn(0x7fca500b4b90 0x7fca5013ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.366+0000 7fca4dd9b700 1 -- 172.21.15.36:0/972042265 >> v1:172.21.15.70:6789/0 conn(0x7fca50060020 legacy=0x7fca500626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:46.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.366+0000 7fca4dd9b700 1 -- 172.21.15.36:0/972042265 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca50061f60 con 0x7fca500b3f80 2022-06-17T12:56:46.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.366+0000 7fca4dd9b700 1 --2- 172.21.15.36:0/972042265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fca500b3f80 0x7fca500b46a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fca44002f70 tx=0x7fca44006eb0).ready entity=mon.2 client_cookie=2868c2be49f031ab server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:46.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.366+0000 7fca4d59a700 1 -- 172.21.15.36:0/972042265 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca4400d570 con 0x7fca500b3f80 2022-06-17T12:56:46.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.366+0000 7fca4d59a700 1 -- 172.21.15.36:0/972042265 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca44000e90 con 0x7fca500b3f80 2022-06-17T12:56:46.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.366+0000 7fca4d59a700 1 -- 172.21.15.36:0/972042265 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca4400d900 con 0x7fca500b3f80 2022-06-17T12:56:46.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.370+0000 7fca4f59e700 1 -- 172.21.15.36:0/972042265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fca500b3f80 msgr2=0x7fca500b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:46.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.370+0000 7fca4f59e700 1 --2- 172.21.15.36:0/972042265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fca500b3f80 0x7fca500b46a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fca44002f70 tx=0x7fca44006eb0).stop 2022-06-17T12:56:46.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.370+0000 7fca4f59e700 1 -- 172.21.15.36:0/972042265 shutdown_connections 2022-06-17T12:56:46.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.370+0000 7fca4f59e700 1 --2- 172.21.15.36:0/972042265 >> v2:172.21.15.70:3300/0 conn(0x7fca500b4b90 0x7fca5013ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.370+0000 7fca4f59e700 1 --2- 172.21.15.36:0/972042265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fca500b3f80 0x7fca500b46a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.370+0000 7fca4f59e700 1 -- 172.21.15.36:0/972042265 >> 172.21.15.36:0/972042265 conn(0x7fca5005c550 msgr2=0x7fca5005f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:46.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.370+0000 7fca4f59e700 1 -- 172.21.15.36:0/972042265 shutdown_connections 2022-06-17T12:56:46.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.370+0000 7fca4f59e700 1 -- 172.21.15.36:0/972042265 wait complete. 2022-06-17T12:56:46.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.370+0000 7fca4f59e700 1 Processor -- start 2022-06-17T12:56:46.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.370+0000 7fca4f59e700 1 -- start start 2022-06-17T12:56:46.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.370+0000 7fca4f59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fca50060020 0x7fca50141aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:46.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.370+0000 7fca4f59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fca500b3f80 0x7fca5013edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:46.366 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.370+0000 7fca4f59e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca500b4b90 0x7fca5013f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:46.366 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.370+0000 7fca4f59e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca500af4c0 con 0x7fca500b3f80 2022-06-17T12:56:46.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.370+0000 7fca4f59e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca500af760 con 0x7fca500b4b90 2022-06-17T12:56:46.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.370+0000 7fca4f59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca500afa00 con 0x7fca50060020 2022-06-17T12:56:46.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.370+0000 7fca4dd9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fca500b3f80 0x7fca5013edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:46.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.370+0000 7fca4dd9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fca500b3f80 0x7fca5013edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39904/0 (socket says 172.21.15.36:39904) 2022-06-17T12:56:46.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.370+0000 7fca4dd9b700 1 -- 172.21.15.36:0/117012646 learned_addr learned my addr 172.21.15.36:0/117012646 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:46.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.374+0000 7fca4e59c700 1 --2- 172.21.15.36:0/117012646 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fca50060020 0x7fca50141aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:46.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.374+0000 7fca4ed9d700 1 --2- 172.21.15.36:0/117012646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca500b4b90 0x7fca5013f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:46.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.374+0000 7fca4dd9b700 1 -- 172.21.15.36:0/117012646 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fca50060020 msgr2=0x7fca50141aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:46.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.374+0000 7fca4dd9b700 1 --2- 172.21.15.36:0/117012646 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fca50060020 0x7fca50141aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.374+0000 7fca4dd9b700 1 -- 172.21.15.36:0/117012646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca500b4b90 msgr2=0x7fca5013f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:46.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.374+0000 7fca4dd9b700 1 --2- 172.21.15.36:0/117012646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca500b4b90 0x7fca5013f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.374+0000 7fca4dd9b700 1 -- 172.21.15.36:0/117012646 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4400b1e0 con 0x7fca500b3f80 2022-06-17T12:56:46.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.374+0000 7fca4ed9d700 1 --2- 172.21.15.36:0/117012646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca500b4b90 0x7fca5013f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:56:46.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.374+0000 7fca4dd9b700 1 --2- 172.21.15.36:0/117012646 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fca500b3f80 0x7fca5013edf0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7fca44002f70 tx=0x7fca44015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:46.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.374+0000 7fca3f7fe700 1 -- 172.21.15.36:0/117012646 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca440117e0 con 0x7fca500b3f80 2022-06-17T12:56:46.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.374+0000 7fca3f7fe700 1 -- 172.21.15.36:0/117012646 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca44011940 con 0x7fca500b3f80 2022-06-17T12:56:46.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.374+0000 7fca4f59e700 1 -- 172.21.15.36:0/117012646 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca5013f7d0 con 0x7fca500b3f80 2022-06-17T12:56:46.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.374+0000 7fca3f7fe700 1 -- 172.21.15.36:0/117012646 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca44011ca0 con 0x7fca500b3f80 2022-06-17T12:56:46.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.374+0000 7fca4f59e700 1 -- 172.21.15.36:0/117012646 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca500adda0 con 0x7fca500b3f80 2022-06-17T12:56:46.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.374+0000 7fca4f59e700 1 -- 172.21.15.36:0/117012646 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca500afbc0 con 0x7fca500b3f80 2022-06-17T12:56:46.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.374+0000 7fca3f7fe700 1 -- 172.21.15.36:0/117012646 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fca4401a040 con 0x7fca500b3f80 2022-06-17T12:56:46.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.374+0000 7fca3f7fe700 1 --2- 172.21.15.36:0/117012646 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fca2c0539d0 0x7fca2c055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:46.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.374+0000 7fca3f7fe700 1 -- 172.21.15.36:0/117012646 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fca4406cca0 con 0x7fca500b3f80 2022-06-17T12:56:46.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.374+0000 7fca4e59c700 1 --2- 172.21.15.36:0/117012646 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fca2c0539d0 0x7fca2c055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:46.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.374+0000 7fca4e59c700 1 --2- 172.21.15.36:0/117012646 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fca2c0539d0 0x7fca2c055e90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7fca4000d000 tx=0x7fca40007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:46.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.382+0000 7fca3f7fe700 1 -- 172.21.15.36:0/117012646 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fca44071070 con 0x7fca500b3f80 2022-06-17T12:56:46.469 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:56:46.486 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:56:46.490 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:56:46.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.506+0000 7fca4f59e700 1 -- 172.21.15.36:0/117012646 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fca50057410 con 0x7fca500b3f80 2022-06-17T12:56:46.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.506+0000 7fca3f7fe700 1 -- 172.21.15.36:0/117012646 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v137) v1 ==== 92+0+10 (secure 0 0 0) 0x7fca44023070 con 0x7fca500b3f80 2022-06-17T12:56:46.495 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:56:46.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.510+0000 7fca4f59e700 1 -- 172.21.15.36:0/117012646 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fca2c0539d0 msgr2=0x7fca2c055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:46.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.510+0000 7fca4f59e700 1 --2- 172.21.15.36:0/117012646 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fca2c0539d0 0x7fca2c055e90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7fca4000d000 tx=0x7fca40007b90).stop 2022-06-17T12:56:46.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.510+0000 7fca4f59e700 1 -- 172.21.15.36:0/117012646 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fca500b3f80 msgr2=0x7fca5013edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:46.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.510+0000 7fca4f59e700 1 --2- 172.21.15.36:0/117012646 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fca500b3f80 0x7fca5013edf0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7fca44002f70 tx=0x7fca44015000).stop 2022-06-17T12:56:46.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.510+0000 7fca4f59e700 1 -- 172.21.15.36:0/117012646 shutdown_connections 2022-06-17T12:56:46.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.510+0000 7fca4f59e700 1 --2- 172.21.15.36:0/117012646 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fca2c0539d0 0x7fca2c055e90 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.510+0000 7fca4f59e700 1 --2- 172.21.15.36:0/117012646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca500b4b90 0x7fca5013f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.510+0000 7fca4f59e700 1 --2- 172.21.15.36:0/117012646 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fca500b3f80 0x7fca5013edf0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.510+0000 7fca4f59e700 1 --2- 172.21.15.36:0/117012646 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fca50060020 0x7fca50141aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.510+0000 7fca4f59e700 1 -- 172.21.15.36:0/117012646 >> 172.21.15.36:0/117012646 conn(0x7fca5005c550 msgr2=0x7fca500b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:46.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.510+0000 7fca4f59e700 1 -- 172.21.15.36:0/117012646 shutdown_connections 2022-06-17T12:56:46.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.510+0000 7fca4f59e700 1 -- 172.21.15.36:0/117012646 wait complete. 2022-06-17T12:56:46.503 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:56:46.507 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-17T12:56:46.510 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:56:46.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.686+0000 7f44d150c700 1 Processor -- start 2022-06-17T12:56:46.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44d150c700 1 -- start start 2022-06-17T12:56:46.676 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44d150c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f44cc144c00 0x7f44cc062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:46.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44d150c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44cc0ad330 0x7f44cc0ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:46.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44d150c700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44cc0b0ba0 con 0x7f44cc062770 2022-06-17T12:56:46.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44d150c700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44cc0ae820 con 0x7f44cc144c00 2022-06-17T12:56:46.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44d150c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc061030 con 0x7f44cc0ad330 2022-06-17T12:56:46.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44ca7fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f44cc144c00 0x7f44cc062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:46.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44ca7fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f44cc144c00 0x7f44cc062b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39910/0 (socket says 172.21.15.36:39910) 2022-06-17T12:56:46.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44ca7fc700 1 -- 172.21.15.36:0/1776786008 learned_addr learned my addr 172.21.15.36:0/1776786008 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:46.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44cb7fe700 1 --2- 172.21.15.36:0/1776786008 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44cc0ad330 0x7f44cc0ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:46.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44ca7fc700 1 -- 172.21.15.36:0/1776786008 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44cc0ad330 msgr2=0x7f44cc0ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:46.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44ca7fc700 1 --2- 172.21.15.36:0/1776786008 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44cc0ad330 0x7f44cc0ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44ca7fc700 1 -- 172.21.15.36:0/1776786008 >> v1:172.21.15.70:6789/0 conn(0x7f44cc062770 legacy=0x7f44cc1444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:46.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44ca7fc700 1 -- 172.21.15.36:0/1776786008 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44cc147e90 con 0x7f44cc144c00 2022-06-17T12:56:46.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44ca7fc700 1 --2- 172.21.15.36:0/1776786008 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f44cc144c00 0x7f44cc062b50 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7f44cc061170 tx=0x7f44b4006eb0).ready entity=mon.0 client_cookie=b45fa5a9156fae0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:46.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44c9ffb700 1 -- 172.21.15.36:0/1776786008 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44b400d5c0 con 0x7f44cc144c00 2022-06-17T12:56:46.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44c9ffb700 1 -- 172.21.15.36:0/1776786008 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b4000e90 con 0x7f44cc144c00 2022-06-17T12:56:46.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44c9ffb700 1 -- 172.21.15.36:0/1776786008 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44b400d950 con 0x7f44cc144c00 2022-06-17T12:56:46.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44d150c700 1 -- 172.21.15.36:0/1776786008 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f44cc144c00 msgr2=0x7f44cc062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:46.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44d150c700 1 --2- 172.21.15.36:0/1776786008 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f44cc144c00 0x7f44cc062b50 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7f44cc061170 tx=0x7f44b4006eb0).stop 2022-06-17T12:56:46.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44d150c700 1 -- 172.21.15.36:0/1776786008 shutdown_connections 2022-06-17T12:56:46.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44d150c700 1 --2- 172.21.15.36:0/1776786008 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44cc0ad330 0x7f44cc0ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44d150c700 1 --2- 172.21.15.36:0/1776786008 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f44cc144c00 0x7f44cc062b50 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44d150c700 1 -- 172.21.15.36:0/1776786008 >> 172.21.15.36:0/1776786008 conn(0x7f44cc0b53e0 msgr2=0x7f44cc0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:46.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44d150c700 1 -- 172.21.15.36:0/1776786008 shutdown_connections 2022-06-17T12:56:46.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.690+0000 7f44d150c700 1 -- 172.21.15.36:0/1776786008 wait complete. 2022-06-17T12:56:46.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44d150c700 1 Processor -- start 2022-06-17T12:56:46.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44d150c700 1 -- start start 2022-06-17T12:56:46.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44d150c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44cc062770 0x7f44cc0b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:46.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44d150c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44cc0ad330 0x7f44cc1334c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:46.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44d150c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f44cc144c00 0x7f44cc1339b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:46.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44d150c700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc131190 con 0x7f44cc144c00 2022-06-17T12:56:46.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44d150c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc131430 con 0x7f44cc062770 2022-06-17T12:56:46.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44d150c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc1316d0 con 0x7f44cc0ad330 2022-06-17T12:56:46.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44cb7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f44cc144c00 0x7f44cc1339b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:46.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44caffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44cc062770 0x7f44cc0b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:46.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44cb7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f44cc144c00 0x7f44cc1339b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39918/0 (socket says 172.21.15.36:39918) 2022-06-17T12:56:46.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44cb7fe700 1 -- 172.21.15.36:0/3200638977 learned_addr learned my addr 172.21.15.36:0/3200638977 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:46.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44ca7fc700 1 --2- 172.21.15.36:0/3200638977 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44cc0ad330 0x7f44cc1334c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:46.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44cb7fe700 1 -- 172.21.15.36:0/3200638977 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44cc0ad330 msgr2=0x7f44cc1334c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:46.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44cb7fe700 1 --2- 172.21.15.36:0/3200638977 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44cc0ad330 0x7f44cc1334c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44cb7fe700 1 -- 172.21.15.36:0/3200638977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44cc062770 msgr2=0x7f44cc0b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:46.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44cb7fe700 1 --2- 172.21.15.36:0/3200638977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44cc062770 0x7f44cc0b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44cb7fe700 1 -- 172.21.15.36:0/3200638977 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b400b1e0 con 0x7f44cc144c00 2022-06-17T12:56:46.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44ca7fc700 1 --2- 172.21.15.36:0/3200638977 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44cc0ad330 0x7f44cc1334c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:46.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44cb7fe700 1 --2- 172.21.15.36:0/3200638977 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f44cc144c00 0x7f44cc1339b0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 rx=0x7f44c4006e00 tx=0x7f44c40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:46.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44c3fff700 1 -- 172.21.15.36:0/3200638977 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44c400c640 con 0x7f44cc144c00 2022-06-17T12:56:46.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44c3fff700 1 -- 172.21.15.36:0/3200638977 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c400c7a0 con 0x7f44cc144c00 2022-06-17T12:56:46.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44d150c700 1 -- 172.21.15.36:0/3200638977 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44cc131970 con 0x7f44cc144c00 2022-06-17T12:56:46.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44c3fff700 1 -- 172.21.15.36:0/3200638977 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44c400cb00 con 0x7f44cc144c00 2022-06-17T12:56:46.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.694+0000 7f44d150c700 1 -- 172.21.15.36:0/3200638977 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44cc066000 con 0x7f44cc144c00 2022-06-17T12:56:46.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.698+0000 7f44d150c700 1 -- 172.21.15.36:0/3200638977 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44cc066250 con 0x7f44cc144c00 2022-06-17T12:56:46.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.698+0000 7f44c3fff700 1 -- 172.21.15.36:0/3200638977 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f44c400cc60 con 0x7f44cc144c00 2022-06-17T12:56:46.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.698+0000 7f44c3fff700 1 --2- 172.21.15.36:0/3200638977 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f44b0053ad0 0x7f44b0055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:46.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.698+0000 7f44c3fff700 1 -- 172.21.15.36:0/3200638977 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f44c406bea0 con 0x7f44cc144c00 2022-06-17T12:56:46.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.698+0000 7f44caffd700 1 --2- 172.21.15.36:0/3200638977 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f44b0053ad0 0x7f44b0055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:46.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.698+0000 7f44caffd700 1 --2- 172.21.15.36:0/3200638977 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f44b0053ad0 0x7f44b0055f90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f44bc00d000 tx=0x7f44bc000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:46.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.706+0000 7f44c3fff700 1 -- 172.21.15.36:0/3200638977 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f44c4042fe0 con 0x7f44cc144c00 2022-06-17T12:56:46.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.838+0000 7f44d150c700 1 -- 172.21.15.36:0/3200638977 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f44cc066720 con 0x7f44cc144c00 2022-06-17T12:56:46.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.838+0000 7f44c3fff700 1 -- 172.21.15.36:0/3200638977 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v137) v1 ==== 84+0+143 (secure 0 0 0) 0x7f44cc066720 con 0x7f44cc144c00 2022-06-17T12:56:46.825 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:46.829 INFO:teuthology.orchestra.run.smithi036.stdout:[ 2022-06-17T12:56:46.830 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:46.830 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 1, 2022-06-17T12:56:46.832 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "device_health_metrics" 2022-06-17T12:56:46.832 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:56:46.833 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:56:46.834 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 2, 2022-06-17T12:56:46.834 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "rbd" 2022-06-17T12:56:46.834 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:56:46.835 INFO:teuthology.orchestra.run.smithi036.stdout:] 2022-06-17T12:56:46.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.838+0000 7f44d150c700 1 -- 172.21.15.36:0/3200638977 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f44b0053ad0 msgr2=0x7f44b0055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:46.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.838+0000 7f44d150c700 1 --2- 172.21.15.36:0/3200638977 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f44b0053ad0 0x7f44b0055f90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f44bc00d000 tx=0x7f44bc000bc0).stop 2022-06-17T12:56:46.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.838+0000 7f44d150c700 1 -- 172.21.15.36:0/3200638977 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f44cc144c00 msgr2=0x7f44cc1339b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:46.839 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.838+0000 7f44d150c700 1 --2- 172.21.15.36:0/3200638977 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f44cc144c00 0x7f44cc1339b0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 rx=0x7f44c4006e00 tx=0x7f44c40091d0).stop 2022-06-17T12:56:46.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.838+0000 7f44d150c700 1 -- 172.21.15.36:0/3200638977 shutdown_connections 2022-06-17T12:56:46.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.838+0000 7f44d150c700 1 --2- 172.21.15.36:0/3200638977 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f44b0053ad0 0x7f44b0055f90 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.840 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.838+0000 7f44d150c700 1 --2- 172.21.15.36:0/3200638977 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f44cc144c00 0x7f44cc1339b0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.838+0000 7f44d150c700 1 --2- 172.21.15.36:0/3200638977 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44cc0ad330 0x7f44cc1334c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.838+0000 7f44d150c700 1 --2- 172.21.15.36:0/3200638977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44cc062770 0x7f44cc0b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.838+0000 7f44d150c700 1 -- 172.21.15.36:0/3200638977 >> 172.21.15.36:0/3200638977 conn(0x7f44cc0b53e0 msgr2=0x7f44cc1307e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:46.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.838+0000 7f44d150c700 1 -- 172.21.15.36:0/3200638977 shutdown_connections 2022-06-17T12:56:46.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.838+0000 7f44d150c700 1 -- 172.21.15.36:0/3200638977 wait complete. 2022-06-17T12:56:46.843 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-17T12:56:46.845 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-17T12:56:46.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb6b4ad700 1 Processor -- start 2022-06-17T12:56:46.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb6b4ad700 1 -- start start 2022-06-17T12:56:46.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb6b4ad700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbb640aeea0 0x7fbb6414a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:46.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb6b4ad700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb6414acc0 0x7fbb641463a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:46.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb6b4ad700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb640af360 con 0x7fbb640b6800 2022-06-17T12:56:46.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb6b4ad700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb640b6be0 con 0x7fbb640aeea0 2022-06-17T12:56:46.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb6b4ad700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb641473a0 con 0x7fbb6414acc0 2022-06-17T12:56:46.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb6a4ab700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fbb640b6800 0x7fbb64147ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57454/0 (socket says 172.21.15.36:57454) 2022-06-17T12:56:46.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb6a4ab700 1 -- 172.21.15.36:0/1813975021 learned_addr learned my addr 172.21.15.36:0/1813975021 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:46.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb6acac700 1 --2- 172.21.15.36:0/1813975021 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb6414acc0 0x7fbb641463a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:46.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb69caa700 1 --2- 172.21.15.36:0/1813975021 >> v2:172.21.15.70:3300/0 conn(0x7fbb640aeea0 0x7fbb6414a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:46.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb694a9700 1 -- 172.21.15.36:0/1813975021 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3292149133 0 0) 0x7fbb640af360 con 0x7fbb640b6800 2022-06-17T12:56:46.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb694a9700 1 -- 172.21.15.36:0/1813975021 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb4c003260 con 0x7fbb640b6800 2022-06-17T12:56:46.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb694a9700 1 -- 172.21.15.36:0/1813975021 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 879552529 0 0) 0x7fbb4c003260 con 0x7fbb640b6800 2022-06-17T12:56:46.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb694a9700 1 -- 172.21.15.36:0/1813975021 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb6414acc0 msgr2=0x7fbb641463a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:46.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb694a9700 1 --2- 172.21.15.36:0/1813975021 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb6414acc0 0x7fbb641463a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb694a9700 1 -- 172.21.15.36:0/1813975021 >> v2:172.21.15.70:3300/0 conn(0x7fbb640aeea0 msgr2=0x7fbb6414a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:46.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb694a9700 1 --2- 172.21.15.36:0/1813975021 >> v2:172.21.15.70:3300/0 conn(0x7fbb640aeea0 0x7fbb6414a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb694a9700 1 -- 172.21.15.36:0/1813975021 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb64147640 con 0x7fbb640b6800 2022-06-17T12:56:46.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb694a9700 1 -- 172.21.15.36:0/1813975021 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7fbb600019e0 con 0x7fbb640b6800 2022-06-17T12:56:46.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb694a9700 1 -- 172.21.15.36:0/1813975021 >> v1:172.21.15.36:6789/0 conn(0x7fbb640b6800 legacy=0x7fbb64147ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:46.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb694a9700 1 --2- 172.21.15.36:0/1813975021 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb6414acc0 0x7fbb4c003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:46.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb694a9700 1 --2- 172.21.15.36:0/1813975021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb4c003c80 0x7fbb4c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:46.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb694a9700 1 --2- 172.21.15.36:0/1813975021 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb4c0049f0 0x7fbb4c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:46.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb694a9700 1 -- 172.21.15.36:0/1813975021 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb640b6be0 con 0x7fbb4c0049f0 2022-06-17T12:56:46.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb694a9700 1 -- 172.21.15.36:0/1813975021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb641473a0 con 0x7fbb4c003c80 2022-06-17T12:56:46.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb694a9700 1 -- 172.21.15.36:0/1813975021 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c00b750 con 0x7fbb6414acc0 2022-06-17T12:56:46.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb6acac700 1 --2- 172.21.15.36:0/1813975021 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb4c0049f0 0x7fbb4c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:46.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb6a4ab700 1 --2- 172.21.15.36:0/1813975021 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb6414acc0 0x7fbb4c003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:46.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb69caa700 1 --2- 172.21.15.36:0/1813975021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb4c003c80 0x7fbb4c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:46.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb6acac700 1 -- 172.21.15.36:0/1813975021 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb6414acc0 msgr2=0x7fbb4c003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:46.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb6acac700 1 --2- 172.21.15.36:0/1813975021 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb6414acc0 0x7fbb4c003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb6acac700 1 -- 172.21.15.36:0/1813975021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb4c003c80 msgr2=0x7fbb4c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:46.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb6acac700 1 --2- 172.21.15.36:0/1813975021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb4c003c80 0x7fbb4c004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:46.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb6acac700 1 -- 172.21.15.36:0/1813975021 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4c003260 con 0x7fbb4c0049f0 2022-06-17T12:56:46.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb6acac700 1 --2- 172.21.15.36:0/1813975021 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb4c0049f0 0x7fbb4c004e70 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0x7fbb4c0054d0 tx=0x7fbb5c006df0).ready entity=mon.0 client_cookie=9aa899ae7b632220 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:46.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb694a9700 1 -- 172.21.15.36:0/1813975021 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb5c0125e0 con 0x7fbb4c0049f0 2022-06-17T12:56:46.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb694a9700 1 -- 172.21.15.36:0/1813975021 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5c012740 con 0x7fbb4c0049f0 2022-06-17T12:56:46.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.974+0000 7fbb694a9700 1 -- 172.21.15.36:0/1813975021 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb5c012aa0 con 0x7fbb4c0049f0 2022-06-17T12:56:46.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.978+0000 7fbb6b4ad700 1 -- 172.21.15.36:0/1813975021 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb4c0049f0 msgr2=0x7fbb4c004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:46.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.978+0000 7fbb6b4ad700 1 --2- 172.21.15.36:0/1813975021 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb4c0049f0 0x7fbb4c004e70 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0x7fbb4c0054d0 tx=0x7fbb5c006df0).stop 2022-06-17T12:56:46.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.978+0000 7fbb6acac700 1 -- 172.21.15.36:0/1813975021 reap_dead start 2022-06-17T12:56:46.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.978+0000 7fbb6b4ad700 1 -- 172.21.15.36:0/1813975021 shutdown_connections 2022-06-17T12:56:46.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.978+0000 7fbb6b4ad700 1 -- 172.21.15.36:0/1813975021 >> 172.21.15.36:0/1813975021 conn(0x7fbb640614b0 msgr2=0x7fbb64061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:46.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.978+0000 7fbb6b4ad700 1 -- 172.21.15.36:0/1813975021 shutdown_connections 2022-06-17T12:56:46.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.978+0000 7fbb6b4ad700 1 -- 172.21.15.36:0/1813975021 wait complete. 2022-06-17T12:56:46.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.978+0000 7fbb6b4ad700 1 Processor -- start 2022-06-17T12:56:47.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.978+0000 7fbb6b4ad700 1 -- start start 2022-06-17T12:56:47.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.982+0000 7fbb6b4ad700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb64064100 0x7fbb64112b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:47.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.982+0000 7fbb6b4ad700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb6410fb00 0x7fbb64113020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:47.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.982+0000 7fbb6b4ad700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb640652b0 0x7fbb64156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:47.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.982+0000 7fbb6b4ad700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb64063560 con 0x7fbb6410fb00 2022-06-17T12:56:47.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.982+0000 7fbb6b4ad700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb64113510 con 0x7fbb640652b0 2022-06-17T12:56:47.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.982+0000 7fbb6b4ad700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb64065830 con 0x7fbb64064100 2022-06-17T12:56:47.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.982+0000 7fbb69caa700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb6410fb00 0x7fbb64113020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:47.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.982+0000 7fbb6a4ab700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb64064100 0x7fbb64112b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:47.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.982+0000 7fbb69caa700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb6410fb00 0x7fbb64113020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39938/0 (socket says 172.21.15.36:39938) 2022-06-17T12:56:47.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.982+0000 7fbb69caa700 1 -- 172.21.15.36:0/160094781 learned_addr learned my addr 172.21.15.36:0/160094781 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:47.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.982+0000 7fbb6acac700 1 --2- 172.21.15.36:0/160094781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb640652b0 0x7fbb64156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:47.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.982+0000 7fbb69caa700 1 -- 172.21.15.36:0/160094781 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb64064100 msgr2=0x7fbb64112b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:47.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.982+0000 7fbb69caa700 1 --2- 172.21.15.36:0/160094781 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb64064100 0x7fbb64112b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:47.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.982+0000 7fbb69caa700 1 -- 172.21.15.36:0/160094781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb640652b0 msgr2=0x7fbb64156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:47.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.982+0000 7fbb69caa700 1 --2- 172.21.15.36:0/160094781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb640652b0 0x7fbb64156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:47.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.982+0000 7fbb69caa700 1 -- 172.21.15.36:0/160094781 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb64065ad0 con 0x7fbb6410fb00 2022-06-17T12:56:47.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.982+0000 7fbb69caa700 1 --2- 172.21.15.36:0/160094781 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb6410fb00 0x7fbb64113020 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7fbb58006000 tx=0x7fbb58006d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:47.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.982+0000 7fbb577fe700 1 -- 172.21.15.36:0/160094781 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb58014650 con 0x7fbb6410fb00 2022-06-17T12:56:47.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.982+0000 7fbb577fe700 1 -- 172.21.15.36:0/160094781 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb580147b0 con 0x7fbb6410fb00 2022-06-17T12:56:47.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.982+0000 7fbb6b4ad700 1 -- 172.21.15.36:0/160094781 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb64065d70 con 0x7fbb6410fb00 2022-06-17T12:56:47.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.982+0000 7fbb577fe700 1 -- 172.21.15.36:0/160094781 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb58014b10 con 0x7fbb6410fb00 2022-06-17T12:56:47.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.982+0000 7fbb6b4ad700 1 -- 172.21.15.36:0/160094781 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb640644e0 con 0x7fbb6410fb00 2022-06-17T12:56:47.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.982+0000 7fbb6b4ad700 1 -- 172.21.15.36:0/160094781 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb64065f00 con 0x7fbb6410fb00 2022-06-17T12:56:47.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.986+0000 7fbb577fe700 1 -- 172.21.15.36:0/160094781 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fbb58014c70 con 0x7fbb6410fb00 2022-06-17T12:56:47.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.986+0000 7fbb577fe700 1 --2- 172.21.15.36:0/160094781 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb440539b0 0x7fbb44055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:47.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.986+0000 7fbb577fe700 1 -- 172.21.15.36:0/160094781 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fbb5806c3c0 con 0x7fbb6410fb00 2022-06-17T12:56:47.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.986+0000 7fbb6a4ab700 1 --2- 172.21.15.36:0/160094781 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb440539b0 0x7fbb44055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:47.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.986+0000 7fbb6a4ab700 1 --2- 172.21.15.36:0/160094781 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb440539b0 0x7fbb44055e70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7fbb60005000 tx=0x7fbb641477d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:47.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:46.994+0000 7fbb577fe700 1 -- 172.21.15.36:0/160094781 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbb58043500 con 0x7fbb6410fb00 2022-06-17T12:56:47.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:47.122+0000 7fbb6b4ad700 1 -- 172.21.15.36:0/160094781 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fbb64057410 con 0x7fbb6410fb00 2022-06-17T12:56:47.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:47.874+0000 7fbb577fe700 1 -- 172.21.15.36:0/160094781 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v138) v1 ==== 133+0+0 (secure 0 0 0) 0x7fbb58018020 con 0x7fbb6410fb00 2022-06-17T12:56:47.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:47.882+0000 7fbb6b4ad700 1 -- 172.21.15.36:0/160094781 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb440539b0 msgr2=0x7fbb44055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:47.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:47.882+0000 7fbb6b4ad700 1 --2- 172.21.15.36:0/160094781 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb440539b0 0x7fbb44055e70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7fbb60005000 tx=0x7fbb641477d0).stop 2022-06-17T12:56:47.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:47.882+0000 7fbb6b4ad700 1 -- 172.21.15.36:0/160094781 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb6410fb00 msgr2=0x7fbb64113020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:47.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:47.882+0000 7fbb6b4ad700 1 --2- 172.21.15.36:0/160094781 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb6410fb00 0x7fbb64113020 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7fbb58006000 tx=0x7fbb58006d90).stop 2022-06-17T12:56:47.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:47.882+0000 7fbb6b4ad700 1 -- 172.21.15.36:0/160094781 shutdown_connections 2022-06-17T12:56:47.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:47.882+0000 7fbb6b4ad700 1 --2- 172.21.15.36:0/160094781 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fbb440539b0 0x7fbb44055e70 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:47.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:47.882+0000 7fbb6b4ad700 1 --2- 172.21.15.36:0/160094781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb640652b0 0x7fbb64156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:47.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:47.882+0000 7fbb6b4ad700 1 --2- 172.21.15.36:0/160094781 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbb6410fb00 0x7fbb64113020 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:47.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:47.882+0000 7fbb6b4ad700 1 --2- 172.21.15.36:0/160094781 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb64064100 0x7fbb64112b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:47.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:47.882+0000 7fbb6b4ad700 1 -- 172.21.15.36:0/160094781 >> 172.21.15.36:0/160094781 conn(0x7fbb640614b0 msgr2=0x7fbb640b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:47.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:47.882+0000 7fbb6b4ad700 1 -- 172.21.15.36:0/160094781 shutdown_connections 2022-06-17T12:56:47.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:47.882+0000 7fbb6b4ad700 1 -- 172.21.15.36:0/160094781 wait complete. 2022-06-17T12:56:47.883 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' created 2022-06-17T12:56:47.884 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-06-17T12:56:48.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.050+0000 7ff81e1d2700 1 Processor -- start 2022-06-17T12:56:48.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.050+0000 7ff81e1d2700 1 -- start start 2022-06-17T12:56:48.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff81e1d2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8180ae360 0x7ff818060260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:48.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff81e1d2700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff8180b1db0 0x7ff8181437d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:48.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff81e1d2700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8180633c0 con 0x7ff81805fe80 2022-06-17T12:56:48.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff81e1d2700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff818143330 con 0x7ff8180b1db0 2022-06-17T12:56:48.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff81e1d2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff818063100 con 0x7ff8180ae360 2022-06-17T12:56:48.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff81d9d1700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff8180b1db0 0x7ff8181437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:48.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff81d1d0700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7ff81805fe80 0x7ff8181444f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57474/0 (socket says 172.21.15.36:57474) 2022-06-17T12:56:48.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff81d1d0700 1 -- 172.21.15.36:0/425774796 learned_addr learned my addr 172.21.15.36:0/425774796 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:48.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff81c9cf700 1 --2- 172.21.15.36:0/425774796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8180ae360 0x7ff818060260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:48.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff81d9d1700 1 -- 172.21.15.36:0/425774796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8180ae360 msgr2=0x7ff818060260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:48.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff81d9d1700 1 --2- 172.21.15.36:0/425774796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8180ae360 0x7ff818060260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:48.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff81d9d1700 1 -- 172.21.15.36:0/425774796 >> v1:172.21.15.36:6789/0 conn(0x7ff81805fe80 legacy=0x7ff8181444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:48.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff81d9d1700 1 -- 172.21.15.36:0/425774796 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff818143020 con 0x7ff8180b1db0 2022-06-17T12:56:48.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff80ffff700 1 -- 172.21.15.36:0/425774796 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4033451390 0 0) 0x7ff8180633c0 con 0x7ff81805fe80 2022-06-17T12:56:48.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff81c9cf700 1 --2- 172.21.15.36:0/425774796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8180ae360 0x7ff818060260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:56:48.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff81d9d1700 1 --2- 172.21.15.36:0/425774796 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8180b1db0 0x7ff8181437d0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7ff8100032f0 tx=0x7ff8100091d0).ready entity=mon.0 client_cookie=2242d6569b00def2 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:48.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff80ffff700 1 -- 172.21.15.36:0/425774796 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff810009ea0 con 0x7ff8180b1db0 2022-06-17T12:56:48.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff80ffff700 1 -- 172.21.15.36:0/425774796 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81000b8a0 con 0x7ff8180b1db0 2022-06-17T12:56:48.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff80ffff700 1 -- 172.21.15.36:0/425774796 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff81000ba10 con 0x7ff8180b1db0 2022-06-17T12:56:48.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff81e1d2700 1 -- 172.21.15.36:0/425774796 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8180b1db0 msgr2=0x7ff8181437d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:48.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff81e1d2700 1 --2- 172.21.15.36:0/425774796 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8180b1db0 0x7ff8181437d0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7ff8100032f0 tx=0x7ff8100091d0).stop 2022-06-17T12:56:48.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff81e1d2700 1 -- 172.21.15.36:0/425774796 shutdown_connections 2022-06-17T12:56:48.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff81e1d2700 1 --2- 172.21.15.36:0/425774796 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8180b1db0 0x7ff8181437d0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:48.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff81e1d2700 1 --2- 172.21.15.36:0/425774796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8180ae360 0x7ff818060260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:48.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff81e1d2700 1 -- 172.21.15.36:0/425774796 >> 172.21.15.36:0/425774796 conn(0x7ff818060990 msgr2=0x7ff81805f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:48.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff81e1d2700 1 -- 172.21.15.36:0/425774796 shutdown_connections 2022-06-17T12:56:48.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.054+0000 7ff81e1d2700 1 -- 172.21.15.36:0/425774796 wait complete. 2022-06-17T12:56:48.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.058+0000 7ff81e1d2700 1 Processor -- start 2022-06-17T12:56:48.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.058+0000 7ff81e1d2700 1 -- start start 2022-06-17T12:56:48.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.062+0000 7ff81e1d2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff81805fe80 0x7ff8181525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:48.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.062+0000 7ff81e1d2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8180ae360 0x7ff81813b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:48.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.062+0000 7ff81e1d2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8180b1db0 0x7ff81813dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:48.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.062+0000 7ff81e1d2700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8180b06a0 con 0x7ff8180b1db0 2022-06-17T12:56:48.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.062+0000 7ff81e1d2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8180672a0 con 0x7ff81805fe80 2022-06-17T12:56:48.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.062+0000 7ff81e1d2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff818067000 con 0x7ff8180ae360 2022-06-17T12:56:48.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.062+0000 7ff81d9d1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8180b1db0 0x7ff81813dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:48.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.062+0000 7ff81d9d1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8180b1db0 0x7ff81813dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39954/0 (socket says 172.21.15.36:39954) 2022-06-17T12:56:48.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.062+0000 7ff81d9d1700 1 -- 172.21.15.36:0/292225138 learned_addr learned my addr 172.21.15.36:0/292225138 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:48.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.062+0000 7ff81c9cf700 1 --2- 172.21.15.36:0/292225138 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8180ae360 0x7ff81813b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:48.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.062+0000 7ff81d1d0700 1 --2- 172.21.15.36:0/292225138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff81805fe80 0x7ff8181525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:48.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.062+0000 7ff81d9d1700 1 -- 172.21.15.36:0/292225138 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8180ae360 msgr2=0x7ff81813b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:48.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.062+0000 7ff81d9d1700 1 --2- 172.21.15.36:0/292225138 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8180ae360 0x7ff81813b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:48.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.062+0000 7ff81d9d1700 1 -- 172.21.15.36:0/292225138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff81805fe80 msgr2=0x7ff8181525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:48.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.062+0000 7ff81d9d1700 1 --2- 172.21.15.36:0/292225138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff81805fe80 0x7ff8181525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:48.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.062+0000 7ff81d9d1700 1 -- 172.21.15.36:0/292225138 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff810009470 con 0x7ff8180b1db0 2022-06-17T12:56:48.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.062+0000 7ff81d9d1700 1 --2- 172.21.15.36:0/292225138 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8180b1db0 0x7ff81813dc70 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7ff8100032f0 tx=0x7ff810016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:48.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.062+0000 7ff80e7fc700 1 -- 172.21.15.36:0/292225138 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff810009ea0 con 0x7ff8180b1db0 2022-06-17T12:56:48.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.062+0000 7ff80e7fc700 1 -- 172.21.15.36:0/292225138 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8100205f0 con 0x7ff8180b1db0 2022-06-17T12:56:48.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.062+0000 7ff81e1d2700 1 -- 172.21.15.36:0/292225138 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8180b0320 con 0x7ff8180b1db0 2022-06-17T12:56:48.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.062+0000 7ff80e7fc700 1 -- 172.21.15.36:0/292225138 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff810020950 con 0x7ff8180b1db0 2022-06-17T12:56:48.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.062+0000 7ff81e1d2700 1 -- 172.21.15.36:0/292225138 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8180aeb20 con 0x7ff8180b1db0 2022-06-17T12:56:48.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.062+0000 7ff80e7fc700 1 -- 172.21.15.36:0/292225138 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff810027030 con 0x7ff8180b1db0 2022-06-17T12:56:48.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.062+0000 7ff81e1d2700 1 -- 172.21.15.36:0/292225138 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff818057410 con 0x7ff8180b1db0 2022-06-17T12:56:48.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.066+0000 7ff80e7fc700 1 --2- 172.21.15.36:0/292225138 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff800053940 0x7ff800055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:48.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.066+0000 7ff80e7fc700 1 -- 172.21.15.36:0/292225138 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5954+0+0 (secure 0 0 0) 0x7ff81006cc10 con 0x7ff8180b1db0 2022-06-17T12:56:48.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.066+0000 7ff81d1d0700 1 --2- 172.21.15.36:0/292225138 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff800053940 0x7ff800055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:48.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.066+0000 7ff81d1d0700 1 --2- 172.21.15.36:0/292225138 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff800053940 0x7ff800055e00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7ff81400d000 tx=0x7ff814007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:48.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.070+0000 7ff80e7fc700 1 -- 172.21.15.36:0/292225138 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff810047590 con 0x7ff8180b1db0 2022-06-17T12:56:48.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.202+0000 7ff81e1d2700 1 -- 172.21.15.36:0/292225138 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7ff8180aeb20 con 0x7ff8180b1db0 2022-06-17T12:56:48.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.878+0000 7ff80e7fc700 1 -- 172.21.15.36:0/292225138 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v139) v1 ==== 125+0+0 (secure 0 0 0) 0x7ff8180aeb20 con 0x7ff8180b1db0 2022-06-17T12:56:48.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.886+0000 7ff81e1d2700 1 -- 172.21.15.36:0/292225138 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff800053940 msgr2=0x7ff800055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:48.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.886+0000 7ff81e1d2700 1 --2- 172.21.15.36:0/292225138 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff800053940 0x7ff800055e00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7ff81400d000 tx=0x7ff814007a00).stop 2022-06-17T12:56:48.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.886+0000 7ff81e1d2700 1 -- 172.21.15.36:0/292225138 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8180b1db0 msgr2=0x7ff81813dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:48.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.886+0000 7ff81e1d2700 1 --2- 172.21.15.36:0/292225138 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8180b1db0 0x7ff81813dc70 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7ff8100032f0 tx=0x7ff810016000).stop 2022-06-17T12:56:48.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.890+0000 7ff81e1d2700 1 -- 172.21.15.36:0/292225138 shutdown_connections 2022-06-17T12:56:48.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.890+0000 7ff81e1d2700 1 --2- 172.21.15.36:0/292225138 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff800053940 0x7ff800055e00 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:48.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.890+0000 7ff81e1d2700 1 --2- 172.21.15.36:0/292225138 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff8180b1db0 0x7ff81813dc70 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:48.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.890+0000 7ff81e1d2700 1 --2- 172.21.15.36:0/292225138 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8180ae360 0x7ff81813b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:48.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.890+0000 7ff81e1d2700 1 --2- 172.21.15.36:0/292225138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff81805fe80 0x7ff8181525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:48.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.890+0000 7ff81e1d2700 1 -- 172.21.15.36:0/292225138 >> 172.21.15.36:0/292225138 conn(0x7ff818060990 msgr2=0x7ff818144c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:48.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.890+0000 7ff81e1d2700 1 -- 172.21.15.36:0/292225138 shutdown_connections 2022-06-17T12:56:48.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:48.890+0000 7ff81e1d2700 1 -- 172.21.15.36:0/292225138 wait complete. 2022-06-17T12:56:48.880 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' created 2022-06-17T12:56:48.889 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-17T12:56:49.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.018+0000 7f37a68a1700 1 Processor -- start 2022-06-17T12:56:49.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.018+0000 7f37a68a1700 1 -- start start 2022-06-17T12:56:49.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.018+0000 7f37a68a1700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f37a00af930 0x7f37a0064230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:49.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.018+0000 7f37a68a1700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a00af280 con 0x7f37a00aeea0 2022-06-17T12:56:49.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.018+0000 7f37a68a1700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a00b14f0 con 0x7f37a00afe30 2022-06-17T12:56:49.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.018+0000 7f37a68a1700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37a0067560 con 0x7f37a00af930 2022-06-17T12:56:49.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.018+0000 7f37a589f700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f37a00aeea0 0x7f37a00b19e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57488/0 (socket says 172.21.15.36:57488) 2022-06-17T12:56:49.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.018+0000 7f37a589f700 1 -- 172.21.15.36:0/2441579211 learned_addr learned my addr 172.21.15.36:0/2441579211 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:49.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.018+0000 7f37a509e700 1 --2- 172.21.15.36:0/2441579211 >> v2:172.21.15.70:3300/0 conn(0x7f37a00af930 0x7f37a0064230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:49.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.022+0000 7f37a489d700 1 -- 172.21.15.36:0/2441579211 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3638342131 0 0) 0x7f37a00af280 con 0x7f37a00aeea0 2022-06-17T12:56:49.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.022+0000 7f37a489d700 1 -- 172.21.15.36:0/2441579211 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3788003260 con 0x7f37a00aeea0 2022-06-17T12:56:49.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.022+0000 7f37a509e700 1 -- 172.21.15.36:0/2441579211 >> v1:172.21.15.70:6789/0 conn(0x7f37a00afe30 legacy=0x7f37a00ab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:49.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.022+0000 7f37a509e700 1 -- 172.21.15.36:0/2441579211 >> v1:172.21.15.36:6789/0 conn(0x7f37a00aeea0 legacy=0x7f37a00b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:49.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.022+0000 7f37a509e700 1 -- 172.21.15.36:0/2441579211 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a00b20f0 con 0x7f37a00af930 2022-06-17T12:56:49.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.022+0000 7f37a509e700 1 --2- 172.21.15.36:0/2441579211 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37a00af930 0x7f37a0064230 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f37a00b0700 tx=0x7f379c006df0).ready entity=mon.1 client_cookie=b6ddb0c84d9047d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:49.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.022+0000 7f37a489d700 1 -- 172.21.15.36:0/2441579211 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f379c00d4f0 con 0x7f37a00af930 2022-06-17T12:56:49.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.022+0000 7f37a489d700 1 -- 172.21.15.36:0/2441579211 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c000e90 con 0x7f37a00af930 2022-06-17T12:56:49.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.022+0000 7f37a489d700 1 -- 172.21.15.36:0/2441579211 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f379c00d840 con 0x7f37a00af930 2022-06-17T12:56:49.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.022+0000 7f37a68a1700 1 -- 172.21.15.36:0/2441579211 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37a00af930 msgr2=0x7f37a0064230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:49.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.022+0000 7f37a68a1700 1 --2- 172.21.15.36:0/2441579211 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37a00af930 0x7f37a0064230 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f37a00b0700 tx=0x7f379c006df0).stop 2022-06-17T12:56:49.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.022+0000 7f37a68a1700 1 -- 172.21.15.36:0/2441579211 shutdown_connections 2022-06-17T12:56:49.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.022+0000 7f37a68a1700 1 --2- 172.21.15.36:0/2441579211 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37a00af930 0x7f37a0064230 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.022+0000 7f37a68a1700 1 -- 172.21.15.36:0/2441579211 >> 172.21.15.36:0/2441579211 conn(0x7f37a005f770 msgr2=0x7f37a005fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:49.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.022+0000 7f37a68a1700 1 -- 172.21.15.36:0/2441579211 shutdown_connections 2022-06-17T12:56:49.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.022+0000 7f37a68a1700 1 -- 172.21.15.36:0/2441579211 wait complete. 2022-06-17T12:56:49.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.022+0000 7f37a68a1700 1 Processor -- start 2022-06-17T12:56:49.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.022+0000 7f37a68a1700 1 -- start start 2022-06-17T12:56:49.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.026+0000 7f37a68a1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37a00aeea0 0x7f37a00a3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:49.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.026+0000 7f37a68a1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37a00af930 0x7f37a0153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:49.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.026+0000 7f37a68a1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37a00afe30 0x7f37a0153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:49.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.026+0000 7f37a68a1700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a00a54c0 con 0x7f37a00af930 2022-06-17T12:56:49.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.026+0000 7f37a68a1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a00a5760 con 0x7f37a00afe30 2022-06-17T12:56:49.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.026+0000 7f37a68a1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a00a5140 con 0x7f37a00aeea0 2022-06-17T12:56:49.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.026+0000 7f37a509e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37a00af930 0x7f37a0153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:49.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.026+0000 7f37a509e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37a00af930 0x7f37a0153800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39966/0 (socket says 172.21.15.36:39966) 2022-06-17T12:56:49.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.026+0000 7f37a509e700 1 -- 172.21.15.36:0/3950900546 learned_addr learned my addr 172.21.15.36:0/3950900546 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:49.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.026+0000 7f37a60a0700 1 --2- 172.21.15.36:0/3950900546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37a00afe30 0x7f37a0153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:49.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.026+0000 7f37a589f700 1 --2- 172.21.15.36:0/3950900546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37a00aeea0 0x7f37a00a3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:49.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.026+0000 7f37a509e700 1 -- 172.21.15.36:0/3950900546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37a00aeea0 msgr2=0x7f37a00a3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:49.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.026+0000 7f37a509e700 1 --2- 172.21.15.36:0/3950900546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37a00aeea0 0x7f37a00a3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.026+0000 7f37a509e700 1 -- 172.21.15.36:0/3950900546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37a00afe30 msgr2=0x7f37a0153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:49.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.026+0000 7f37a509e700 1 --2- 172.21.15.36:0/3950900546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37a00afe30 0x7f37a0153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.026+0000 7f37a509e700 1 -- 172.21.15.36:0/3950900546 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c00b180 con 0x7f37a00af930 2022-06-17T12:56:49.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.026+0000 7f37a589f700 1 --2- 172.21.15.36:0/3950900546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37a00aeea0 0x7f37a00a3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:56:49.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.026+0000 7f37a509e700 1 --2- 172.21.15.36:0/3950900546 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37a00af930 0x7f37a0153800 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7f37a00b0700 tx=0x7f379c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:49.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.026+0000 7f378effd700 1 -- 172.21.15.36:0/3950900546 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f379c0138f0 con 0x7f37a00af930 2022-06-17T12:56:49.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.026+0000 7f378effd700 1 -- 172.21.15.36:0/3950900546 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c013a50 con 0x7f37a00af930 2022-06-17T12:56:49.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.026+0000 7f37a68a1700 1 -- 172.21.15.36:0/3950900546 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a00a5ea0 con 0x7f37a00af930 2022-06-17T12:56:49.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.026+0000 7f378effd700 1 -- 172.21.15.36:0/3950900546 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f379c013db0 con 0x7f37a00af930 2022-06-17T12:56:49.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.026+0000 7f37a68a1700 1 -- 172.21.15.36:0/3950900546 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a00a8250 con 0x7f37a00af930 2022-06-17T12:56:49.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.030+0000 7f378effd700 1 -- 172.21.15.36:0/3950900546 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f379c025030 con 0x7f37a00af930 2022-06-17T12:56:49.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.030+0000 7f37a68a1700 1 -- 172.21.15.36:0/3950900546 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a0057410 con 0x7f37a00af930 2022-06-17T12:56:49.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.030+0000 7f378effd700 1 --2- 172.21.15.36:0/3950900546 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f37800539b0 0x7f3780055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:49.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.030+0000 7f378effd700 1 -- 172.21.15.36:0/3950900546 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f379c06cc60 con 0x7f37a00af930 2022-06-17T12:56:49.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.030+0000 7f37a589f700 1 --2- 172.21.15.36:0/3950900546 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f37800539b0 0x7f3780055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:49.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.030+0000 7f37a589f700 1 --2- 172.21.15.36:0/3950900546 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f37800539b0 0x7f3780055e70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f379000b000 tx=0x7f3790009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:49.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.034+0000 7f378effd700 1 -- 172.21.15.36:0/3950900546 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f379c0439d0 con 0x7f37a00af930 2022-06-17T12:56:49.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.198+0000 7f37a68a1700 1 -- 172.21.15.36:0/3950900546 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f37a0069cb0 con 0x7f37a00af930 2022-06-17T12:56:49.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.210+0000 7f378effd700 1 -- 172.21.15.36:0/3950900546 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 27 and data pool 28 v77) v1 ==== 189+0+0 (secure 0 0 0) 0x7f379c023020 con 0x7f37a00af930 2022-06-17T12:56:49.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.214+0000 7f37a68a1700 1 -- 172.21.15.36:0/3950900546 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f37800539b0 msgr2=0x7f3780055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:49.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.214+0000 7f37a68a1700 1 --2- 172.21.15.36:0/3950900546 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f37800539b0 0x7f3780055e70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f379000b000 tx=0x7f3790009040).stop 2022-06-17T12:56:49.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.214+0000 7f37a68a1700 1 -- 172.21.15.36:0/3950900546 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37a00af930 msgr2=0x7f37a0153800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:49.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.214+0000 7f37a68a1700 1 --2- 172.21.15.36:0/3950900546 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37a00af930 0x7f37a0153800 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7f37a00b0700 tx=0x7f379c014000).stop 2022-06-17T12:56:49.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.222+0000 7f37a68a1700 1 -- 172.21.15.36:0/3950900546 shutdown_connections 2022-06-17T12:56:49.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.222+0000 7f37a68a1700 1 --2- 172.21.15.36:0/3950900546 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f37800539b0 0x7f3780055e70 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.222+0000 7f37a68a1700 1 --2- 172.21.15.36:0/3950900546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37a00afe30 0x7f37a0153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.222+0000 7f37a68a1700 1 --2- 172.21.15.36:0/3950900546 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37a00af930 0x7f37a0153800 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.222+0000 7f37a68a1700 1 --2- 172.21.15.36:0/3950900546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37a00aeea0 0x7f37a00a3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.222+0000 7f37a68a1700 1 -- 172.21.15.36:0/3950900546 >> 172.21.15.36:0/3950900546 conn(0x7f37a005f770 msgr2=0x7f37a00b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:49.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.222+0000 7f37a68a1700 1 -- 172.21.15.36:0/3950900546 shutdown_connections 2022-06-17T12:56:49.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.222+0000 7f37a68a1700 1 -- 172.21.15.36:0/3950900546 wait complete. 2022-06-17T12:56:49.221 INFO:teuthology.orchestra.run.smithi036.stderr:new fs with metadata pool 27 and data pool 28 2022-06-17T12:56:49.223 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:56:49.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd7a4def700 1 Processor -- start 2022-06-17T12:56:49.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd7a4def700 1 -- start start 2022-06-17T12:56:49.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd7a4def700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7a0062770 0x7fd7a0062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:49.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd7a4def700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a00b0ba0 con 0x7fd7a01444f0 2022-06-17T12:56:49.372 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd7a4def700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a0147e90 con 0x7fd7a00ad330 2022-06-17T12:56:49.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd7a4def700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a00ae820 con 0x7fd7a0062770 2022-06-17T12:56:49.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79dd9b700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fd7a01444f0 0x7fd7a0144930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57512/0 (socket says 172.21.15.36:57512) 2022-06-17T12:56:49.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79dd9b700 1 -- 172.21.15.36:0/4263286197 learned_addr learned my addr 172.21.15.36:0/4263286197 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:49.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79e59c700 1 --2- 172.21.15.36:0/4263286197 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7a0062770 0x7fd7a0062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:49.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79d59a700 1 -- 172.21.15.36:0/4263286197 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3865914241 0 0) 0x7fd7a00b0ba0 con 0x7fd7a01444f0 2022-06-17T12:56:49.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79d59a700 1 -- 172.21.15.36:0/4263286197 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd780003260 con 0x7fd7a01444f0 2022-06-17T12:56:49.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79d59a700 1 -- 172.21.15.36:0/4263286197 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995407079 0 0) 0x7fd7a0147e90 con 0x7fd7a00ad330 2022-06-17T12:56:49.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79d59a700 1 -- 172.21.15.36:0/4263286197 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7a00b0ba0 con 0x7fd7a00ad330 2022-06-17T12:56:49.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79d59a700 1 -- 172.21.15.36:0/4263286197 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2455754467 0 0) 0x7fd780003260 con 0x7fd7a01444f0 2022-06-17T12:56:49.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79d59a700 1 -- 172.21.15.36:0/4263286197 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7a0062770 msgr2=0x7fd7a0062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:49.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79d59a700 1 --2- 172.21.15.36:0/4263286197 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7a0062770 0x7fd7a0062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79d59a700 1 -- 172.21.15.36:0/4263286197 >> v1:172.21.15.70:6789/0 conn(0x7fd7a00ad330 legacy=0x7fd7a00ad780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:49.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79d59a700 1 -- 172.21.15.36:0/4263286197 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a00b0dd0 con 0x7fd7a01444f0 2022-06-17T12:56:49.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79d59a700 1 -- 172.21.15.36:0/4263286197 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7fd788001660 con 0x7fd7a01444f0 2022-06-17T12:56:49.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79d59a700 1 -- 172.21.15.36:0/4263286197 >> v1:172.21.15.36:6789/0 conn(0x7fd7a01444f0 legacy=0x7fd7a0144930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:49.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79d59a700 1 --2- 172.21.15.36:0/4263286197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd780003910 0x7fd780003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:49.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79d59a700 1 --2- 172.21.15.36:0/4263286197 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7a0062770 0x7fd780004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:49.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79d59a700 1 --2- 172.21.15.36:0/4263286197 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7800049f0 0x7fd780004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:49.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79d59a700 1 -- 172.21.15.36:0/4263286197 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a00ae820 con 0x7fd7800049f0 2022-06-17T12:56:49.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79d59a700 1 -- 172.21.15.36:0/4263286197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78000b780 con 0x7fd780003910 2022-06-17T12:56:49.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79d59a700 1 -- 172.21.15.36:0/4263286197 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd78000b940 con 0x7fd7a0062770 2022-06-17T12:56:49.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79ed9d700 1 --2- 172.21.15.36:0/4263286197 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7800049f0 0x7fd780004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:49.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79e59c700 1 --2- 172.21.15.36:0/4263286197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd780003910 0x7fd780003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:49.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79dd9b700 1 --2- 172.21.15.36:0/4263286197 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7a0062770 0x7fd780004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:49.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79ed9d700 1 -- 172.21.15.36:0/4263286197 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7a0062770 msgr2=0x7fd780004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:49.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79ed9d700 1 --2- 172.21.15.36:0/4263286197 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7a0062770 0x7fd780004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79ed9d700 1 -- 172.21.15.36:0/4263286197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd780003910 msgr2=0x7fd780003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:49.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79ed9d700 1 --2- 172.21.15.36:0/4263286197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd780003910 0x7fd780003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79ed9d700 1 -- 172.21.15.36:0/4263286197 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd780003260 con 0x7fd7800049f0 2022-06-17T12:56:49.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79ed9d700 1 --2- 172.21.15.36:0/4263286197 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7800049f0 0x7fd780004e70 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0x7fd798001490 tx=0x7fd798007d80).ready entity=mon.0 client_cookie=9045f660814ef8ed server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:49.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79d59a700 1 -- 172.21.15.36:0/4263286197 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd798001780 con 0x7fd7800049f0 2022-06-17T12:56:49.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79d59a700 1 -- 172.21.15.36:0/4263286197 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd798006890 con 0x7fd7800049f0 2022-06-17T12:56:49.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.362+0000 7fd79d59a700 1 -- 172.21.15.36:0/4263286197 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd798006bf0 con 0x7fd7800049f0 2022-06-17T12:56:49.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.366+0000 7fd7a4def700 1 -- 172.21.15.36:0/4263286197 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7800049f0 msgr2=0x7fd780004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:49.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.366+0000 7fd7a4def700 1 --2- 172.21.15.36:0/4263286197 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7800049f0 0x7fd780004e70 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0x7fd798001490 tx=0x7fd798007d80).stop 2022-06-17T12:56:49.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.366+0000 7fd79ed9d700 1 -- 172.21.15.36:0/4263286197 reap_dead start 2022-06-17T12:56:49.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.366+0000 7fd7a4def700 1 -- 172.21.15.36:0/4263286197 shutdown_connections 2022-06-17T12:56:49.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.366+0000 7fd7a4def700 1 -- 172.21.15.36:0/4263286197 >> 172.21.15.36:0/4263286197 conn(0x7fd7a00b53e0 msgr2=0x7fd7a00b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:49.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.366+0000 7fd7a4def700 1 -- 172.21.15.36:0/4263286197 shutdown_connections 2022-06-17T12:56:49.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.366+0000 7fd7a4def700 1 -- 172.21.15.36:0/4263286197 wait complete. 2022-06-17T12:56:49.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.366+0000 7fd7a4def700 1 Processor -- start 2022-06-17T12:56:49.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.366+0000 7fd7a4def700 1 -- start start 2022-06-17T12:56:49.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.366+0000 7fd7a4def700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7800049f0 0x7fd7a00b2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:49.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.366+0000 7fd7a4def700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7a0065ab0 0x7fd7a012e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:49.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.366+0000 7fd7a4def700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7a014acb0 0x7fd7a012e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:49.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.366+0000 7fd7a4def700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a0066910 con 0x7fd7800049f0 2022-06-17T12:56:49.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.366+0000 7fd7a4def700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a0065f30 con 0x7fd7a0065ab0 2022-06-17T12:56:49.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.366+0000 7fd7a4def700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a00661d0 con 0x7fd7a014acb0 2022-06-17T12:56:49.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.366+0000 7fd79e59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7800049f0 0x7fd7a00b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:49.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.366+0000 7fd79e59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7800049f0 0x7fd7a00b2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:39994/0 (socket says 172.21.15.36:39994) 2022-06-17T12:56:49.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.366+0000 7fd79e59c700 1 -- 172.21.15.36:0/3159810911 learned_addr learned my addr 172.21.15.36:0/3159810911 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:49.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.366+0000 7fd79dd9b700 1 --2- 172.21.15.36:0/3159810911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7a0065ab0 0x7fd7a012e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:49.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.366+0000 7fd79ed9d700 1 --2- 172.21.15.36:0/3159810911 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7a014acb0 0x7fd7a012e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:49.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.370+0000 7fd79e59c700 1 -- 172.21.15.36:0/3159810911 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7a014acb0 msgr2=0x7fd7a012e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:49.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.370+0000 7fd79e59c700 1 --2- 172.21.15.36:0/3159810911 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7a014acb0 0x7fd7a012e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.370+0000 7fd79e59c700 1 -- 172.21.15.36:0/3159810911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7a0065ab0 msgr2=0x7fd7a012e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:49.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.370+0000 7fd79e59c700 1 --2- 172.21.15.36:0/3159810911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7a0065ab0 0x7fd7a012e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.370+0000 7fd79e59c700 1 -- 172.21.15.36:0/3159810911 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a0066470 con 0x7fd7800049f0 2022-06-17T12:56:49.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.370+0000 7fd79e59c700 1 --2- 172.21.15.36:0/3159810911 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7800049f0 0x7fd7a00b2590 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7fd79000b000 tx=0x7fd790006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:49.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.370+0000 7fd7977fe700 1 -- 172.21.15.36:0/3159810911 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd79000f760 con 0x7fd7800049f0 2022-06-17T12:56:49.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.370+0000 7fd7977fe700 1 -- 172.21.15.36:0/3159810911 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79000f8c0 con 0x7fd7800049f0 2022-06-17T12:56:49.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.370+0000 7fd7a4def700 1 -- 172.21.15.36:0/3159810911 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a012ec90 con 0x7fd7800049f0 2022-06-17T12:56:49.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.370+0000 7fd7977fe700 1 -- 172.21.15.36:0/3159810911 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd79000fc20 con 0x7fd7800049f0 2022-06-17T12:56:49.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.370+0000 7fd7a4def700 1 -- 172.21.15.36:0/3159810911 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a0159cb0 con 0x7fd7800049f0 2022-06-17T12:56:49.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.370+0000 7fd7a4def700 1 -- 172.21.15.36:0/3159810911 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a0066600 con 0x7fd7800049f0 2022-06-17T12:56:49.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.370+0000 7fd7977fe700 1 -- 172.21.15.36:0/3159810911 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd790014440 con 0x7fd7800049f0 2022-06-17T12:56:49.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.370+0000 7fd7977fe700 1 --2- 172.21.15.36:0/3159810911 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7840539d0 0x7fd784055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:49.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.370+0000 7fd7977fe700 1 -- 172.21.15.36:0/3159810911 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd79006c8e0 con 0x7fd7800049f0 2022-06-17T12:56:49.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.370+0000 7fd79dd9b700 1 --2- 172.21.15.36:0/3159810911 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7840539d0 0x7fd784055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:49.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.374+0000 7fd79dd9b700 1 --2- 172.21.15.36:0/3159810911 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7840539d0 0x7fd784055e90 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7fd788005000 tx=0x7fd7a01487d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:49.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.378+0000 7fd7977fe700 1 -- 172.21.15.36:0/3159810911 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd790043680 con 0x7fd7800049f0 2022-06-17T12:56:49.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.506+0000 7fd7a4def700 1 -- 172.21.15.36:0/3159810911 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7a00b1c20 con 0x7fd7800049f0 2022-06-17T12:56:49.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.510+0000 7fd7977fe700 1 -- 172.21.15.36:0/3159810911 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16298 (secure 0 0 0) 0x7fd790018020 con 0x7fd7800049f0 2022-06-17T12:56:49.497 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:49.497 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":140,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:56:49.200405+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:56:47.177576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-06-17T12:56:48.257688+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:56:49.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.510+0000 7fd7a4def700 1 -- 172.21.15.36:0/3159810911 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7840539d0 msgr2=0x7fd784055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:49.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.510+0000 7fd7a4def700 1 --2- 172.21.15.36:0/3159810911 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7840539d0 0x7fd784055e90 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7fd788005000 tx=0x7fd7a01487d0).stop 2022-06-17T12:56:49.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.510+0000 7fd7a4def700 1 -- 172.21.15.36:0/3159810911 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7800049f0 msgr2=0x7fd7a00b2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:49.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.510+0000 7fd7a4def700 1 --2- 172.21.15.36:0/3159810911 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7800049f0 0x7fd7a00b2590 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7fd79000b000 tx=0x7fd790006df0).stop 2022-06-17T12:56:49.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.510+0000 7fd7a4def700 1 -- 172.21.15.36:0/3159810911 shutdown_connections 2022-06-17T12:56:49.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.510+0000 7fd7a4def700 1 --2- 172.21.15.36:0/3159810911 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd7840539d0 0x7fd784055e90 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.510+0000 7fd7a4def700 1 --2- 172.21.15.36:0/3159810911 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7a014acb0 0x7fd7a012e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.510+0000 7fd7a4def700 1 --2- 172.21.15.36:0/3159810911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7a0065ab0 0x7fd7a012e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.510+0000 7fd7a4def700 1 --2- 172.21.15.36:0/3159810911 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7800049f0 0x7fd7a00b2590 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.510+0000 7fd7a4def700 1 -- 172.21.15.36:0/3159810911 >> 172.21.15.36:0/3159810911 conn(0x7fd7a00b53e0 msgr2=0x7fd7a0147480 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:49.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.510+0000 7fd7a4def700 1 -- 172.21.15.36:0/3159810911 shutdown_connections 2022-06-17T12:56:49.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.510+0000 7fd7a4def700 1 -- 172.21.15.36:0/3159810911 wait complete. 2022-06-17T12:56:49.557 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:56:49.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.686+0000 7f52b18f4700 1 Processor -- start 2022-06-17T12:56:49.672 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.686+0000 7f52b18f4700 1 -- start start 2022-06-17T12:56:49.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.686+0000 7f52b18f4700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f52ac068410 0x7f52ac0af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:49.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.686+0000 7f52b18f4700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52ac0b1140 0x7f52ac13e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:49.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.686+0000 7f52b18f4700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52ac067510 con 0x7f52ac142500 2022-06-17T12:56:49.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.686+0000 7f52b18f4700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52ac0672b0 con 0x7f52ac068410 2022-06-17T12:56:49.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.686+0000 7f52b18f4700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac0b1670 con 0x7f52ac0b1140 2022-06-17T12:56:49.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.686+0000 7f52abfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f52ac068410 0x7f52ac0af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:49.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.686+0000 7f52abfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f52ac068410 0x7f52ac0af5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40002/0 (socket says 172.21.15.36:40002) 2022-06-17T12:56:49.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.686+0000 7f52abfff700 1 -- 172.21.15.36:0/2353968569 learned_addr learned my addr 172.21.15.36:0/2353968569 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:49.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.686+0000 7f52b08f2700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f52ac142500 0x7f52ac13e210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57532/0 (socket says 172.21.15.36:57532) 2022-06-17T12:56:49.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.686+0000 7f52b10f3700 1 --2- 172.21.15.36:0/2353968569 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52ac0b1140 0x7f52ac13e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:49.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.686+0000 7f52abfff700 1 -- 172.21.15.36:0/2353968569 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52ac0b1140 msgr2=0x7f52ac13e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:49.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.686+0000 7f52abfff700 1 --2- 172.21.15.36:0/2353968569 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52ac0b1140 0x7f52ac13e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.686+0000 7f52abfff700 1 -- 172.21.15.36:0/2353968569 >> v1:172.21.15.36:6789/0 conn(0x7f52ac142500 legacy=0x7f52ac13e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:49.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.686+0000 7f52abfff700 1 -- 172.21.15.36:0/2353968569 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac13ee10 con 0x7f52ac068410 2022-06-17T12:56:49.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.686+0000 7f52b10f3700 1 --2- 172.21.15.36:0/2353968569 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52ac0b1140 0x7f52ac13e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:56:49.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.690+0000 7f52abfff700 1 --2- 172.21.15.36:0/2353968569 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52ac068410 0x7f52ac0af5e0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0x7f529c002f70 tx=0x7f529c006eb0).ready entity=mon.0 client_cookie=39326d68d6911952 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:49.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.690+0000 7f52ab7fe700 1 -- 172.21.15.36:0/2353968569 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f529c00d5c0 con 0x7f52ac068410 2022-06-17T12:56:49.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.690+0000 7f52ab7fe700 1 -- 172.21.15.36:0/2353968569 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529c000e90 con 0x7f52ac068410 2022-06-17T12:56:49.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.690+0000 7f52ab7fe700 1 -- 172.21.15.36:0/2353968569 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f529c00d950 con 0x7f52ac068410 2022-06-17T12:56:49.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.690+0000 7f52b18f4700 1 -- 172.21.15.36:0/2353968569 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52ac068410 msgr2=0x7f52ac0af5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:49.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.690+0000 7f52b18f4700 1 --2- 172.21.15.36:0/2353968569 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52ac068410 0x7f52ac0af5e0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0x7f529c002f70 tx=0x7f529c006eb0).stop 2022-06-17T12:56:49.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.690+0000 7f52b18f4700 1 -- 172.21.15.36:0/2353968569 shutdown_connections 2022-06-17T12:56:49.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.690+0000 7f52b18f4700 1 --2- 172.21.15.36:0/2353968569 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52ac0b1140 0x7f52ac13e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.690+0000 7f52b18f4700 1 --2- 172.21.15.36:0/2353968569 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52ac068410 0x7f52ac0af5e0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.690+0000 7f52b18f4700 1 -- 172.21.15.36:0/2353968569 >> 172.21.15.36:0/2353968569 conn(0x7f52ac05f770 msgr2=0x7f52ac05fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:49.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.690+0000 7f52b18f4700 1 -- 172.21.15.36:0/2353968569 shutdown_connections 2022-06-17T12:56:49.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.690+0000 7f52b18f4700 1 -- 172.21.15.36:0/2353968569 wait complete. 2022-06-17T12:56:49.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.690+0000 7f52b18f4700 1 Processor -- start 2022-06-17T12:56:49.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.690+0000 7f52b18f4700 1 -- start start 2022-06-17T12:56:49.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.694+0000 7f52b18f4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52ac068410 0x7f52ac065f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:49.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.694+0000 7f52b18f4700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52ac0b1140 0x7f52ac0b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:49.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.694+0000 7f52b18f4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52ac142500 0x7f52ac09fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:49.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.694+0000 7f52b18f4700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac09ebd0 con 0x7f52ac068410 2022-06-17T12:56:49.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.694+0000 7f52b18f4700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac064120 con 0x7f52ac142500 2022-06-17T12:56:49.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.694+0000 7f52b18f4700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac0643c0 con 0x7f52ac0b1140 2022-06-17T12:56:49.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.694+0000 7f52b08f2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52ac068410 0x7f52ac065f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:49.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.694+0000 7f52b08f2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52ac068410 0x7f52ac065f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40008/0 (socket says 172.21.15.36:40008) 2022-06-17T12:56:49.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.694+0000 7f52b08f2700 1 -- 172.21.15.36:0/849700806 learned_addr learned my addr 172.21.15.36:0/849700806 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:49.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.694+0000 7f52b10f3700 1 --2- 172.21.15.36:0/849700806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52ac142500 0x7f52ac09fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:49.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.694+0000 7f52abfff700 1 --2- 172.21.15.36:0/849700806 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52ac0b1140 0x7f52ac0b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:49.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.694+0000 7f52b10f3700 1 -- 172.21.15.36:0/849700806 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52ac0b1140 msgr2=0x7f52ac0b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:49.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.694+0000 7f52b10f3700 1 --2- 172.21.15.36:0/849700806 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52ac0b1140 0x7f52ac0b57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.694+0000 7f52b10f3700 1 -- 172.21.15.36:0/849700806 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52ac068410 msgr2=0x7f52ac065f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:49.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.694+0000 7f52b10f3700 1 --2- 172.21.15.36:0/849700806 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52ac068410 0x7f52ac065f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.694+0000 7f52b10f3700 1 -- 172.21.15.36:0/849700806 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c00b1e0 con 0x7f52ac142500 2022-06-17T12:56:49.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.694+0000 7f52b08f2700 1 --2- 172.21.15.36:0/849700806 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52ac068410 0x7f52ac065f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:49.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.694+0000 7f52b10f3700 1 --2- 172.21.15.36:0/849700806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52ac142500 0x7f52ac09fbd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7f52a400a000 tx=0x7f52a4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:49.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.694+0000 7f52a9ffb700 1 -- 172.21.15.36:0/849700806 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52a400c070 con 0x7f52ac142500 2022-06-17T12:56:49.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.694+0000 7f52a9ffb700 1 -- 172.21.15.36:0/849700806 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a4011920 con 0x7f52ac142500 2022-06-17T12:56:49.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.694+0000 7f52b18f4700 1 -- 172.21.15.36:0/849700806 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ac064660 con 0x7f52ac142500 2022-06-17T12:56:49.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.694+0000 7f52a9ffb700 1 -- 172.21.15.36:0/849700806 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52a4011c80 con 0x7f52ac142500 2022-06-17T12:56:49.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.694+0000 7f52b18f4700 1 -- 172.21.15.36:0/849700806 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ac13e6b0 con 0x7f52ac142500 2022-06-17T12:56:49.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.698+0000 7f52b18f4700 1 -- 172.21.15.36:0/849700806 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ac057410 con 0x7f52ac142500 2022-06-17T12:56:49.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.698+0000 7f52a9ffb700 1 -- 172.21.15.36:0/849700806 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f52a401a420 con 0x7f52ac142500 2022-06-17T12:56:49.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.698+0000 7f52a9ffb700 1 --2- 172.21.15.36:0/849700806 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f528c053a40 0x7f528c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:49.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.698+0000 7f52a9ffb700 1 -- 172.21.15.36:0/849700806 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f52a406e3f0 con 0x7f52ac142500 2022-06-17T12:56:49.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.698+0000 7f52b08f2700 1 --2- 172.21.15.36:0/849700806 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f528c053a40 0x7f528c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:49.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.698+0000 7f52b08f2700 1 --2- 172.21.15.36:0/849700806 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f528c053a40 0x7f528c055f00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f52a000d000 tx=0x7f52a0007c40).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:49.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.706+0000 7f52a9ffb700 1 -- 172.21.15.36:0/849700806 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f52a4045000 con 0x7f52ac142500 2022-06-17T12:56:49.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.834+0000 7f52b18f4700 1 -- 172.21.15.36:0/849700806 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52ac146620 con 0x7f52ac142500 2022-06-17T12:56:49.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.838+0000 7f52a9ffb700 1 -- 172.21.15.36:0/849700806 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f52a400f040 con 0x7f52ac142500 2022-06-17T12:56:49.824 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:49.824 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":140,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:56:49.200405+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:56:47.177576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-06-17T12:56:48.257688+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:56:49.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.838+0000 7f52b18f4700 1 -- 172.21.15.36:0/849700806 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f528c053a40 msgr2=0x7f528c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:49.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.838+0000 7f52b18f4700 1 --2- 172.21.15.36:0/849700806 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f528c053a40 0x7f528c055f00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f52a000d000 tx=0x7f52a0007c40).stop 2022-06-17T12:56:49.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.838+0000 7f52b18f4700 1 -- 172.21.15.36:0/849700806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52ac142500 msgr2=0x7f52ac09fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:49.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.838+0000 7f52b18f4700 1 --2- 172.21.15.36:0/849700806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52ac142500 0x7f52ac09fbd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7f52a400a000 tx=0x7f52a4008040).stop 2022-06-17T12:56:49.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.838+0000 7f52b18f4700 1 -- 172.21.15.36:0/849700806 shutdown_connections 2022-06-17T12:56:49.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.838+0000 7f52b18f4700 1 --2- 172.21.15.36:0/849700806 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f528c053a40 0x7f528c055f00 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.838+0000 7f52b18f4700 1 --2- 172.21.15.36:0/849700806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52ac142500 0x7f52ac09fbd0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.838+0000 7f52b18f4700 1 --2- 172.21.15.36:0/849700806 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52ac0b1140 0x7f52ac0b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.838+0000 7f52b18f4700 1 --2- 172.21.15.36:0/849700806 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f52ac068410 0x7f52ac065f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:49.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.838+0000 7f52b18f4700 1 -- 172.21.15.36:0/849700806 >> 172.21.15.36:0/849700806 conn(0x7f52ac05f770 msgr2=0x7f52ac0b41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:49.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.838+0000 7f52b18f4700 1 -- 172.21.15.36:0/849700806 shutdown_connections 2022-06-17T12:56:49.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:49.838+0000 7f52b18f4700 1 -- 172.21.15.36:0/849700806 wait complete. 2022-06-17T12:56:49.876 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-17T12:56:50.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.002+0000 7f02983f7700 1 Processor -- start 2022-06-17T12:56:50.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.002+0000 7f02983f7700 1 -- start start 2022-06-17T12:56:50.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.002+0000 7f02983f7700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0290060260 0x7f02900b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:50.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.002+0000 7f02983f7700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f02900a9b90 0x7f02900b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:50.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.002+0000 7f02983f7700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02900b10a0 con 0x7f02900b0420 2022-06-17T12:56:50.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.002+0000 7f02983f7700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f029005fe80 con 0x7f0290060260 2022-06-17T12:56:50.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.002+0000 7f02983f7700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0290144140 con 0x7f02900a9b90 2022-06-17T12:56:50.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.002+0000 7f0295992700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0290060260 0x7f02900b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:50.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.002+0000 7f0296193700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f02900b0420 0x7f029005f770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57546/0 (socket says 172.21.15.36:57546) 2022-06-17T12:56:50.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.002+0000 7f0296193700 1 -- 172.21.15.36:0/3041739796 learned_addr learned my addr 172.21.15.36:0/3041739796 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:50.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.002+0000 7f0296994700 1 --2- 172.21.15.36:0/3041739796 >> v2:172.21.15.70:3300/0 conn(0x7f02900a9b90 0x7f02900b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:50.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.002+0000 7f0295992700 1 -- 172.21.15.36:0/3041739796 >> v2:172.21.15.70:3300/0 conn(0x7f02900a9b90 msgr2=0x7f02900b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:50.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.002+0000 7f0295992700 1 --2- 172.21.15.36:0/3041739796 >> v2:172.21.15.70:3300/0 conn(0x7f02900a9b90 0x7f02900b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.002+0000 7f0295992700 1 -- 172.21.15.36:0/3041739796 >> v1:172.21.15.36:6789/0 conn(0x7f02900b0420 legacy=0x7f029005f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:50.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.002+0000 7f0295992700 1 -- 172.21.15.36:0/3041739796 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0290143ea0 con 0x7f0290060260 2022-06-17T12:56:50.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.002+0000 7f0295191700 1 -- 172.21.15.36:0/3041739796 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1231073428 0 0) 0x7f02900b10a0 con 0x7f02900b0420 2022-06-17T12:56:50.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.002+0000 7f0295992700 1 --2- 172.21.15.36:0/3041739796 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0290060260 0x7f02900b2e90 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 rx=0x7f0280002f70 tx=0x7f0280006df0).ready entity=mon.0 client_cookie=4691c528fe0ad86 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:50.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.006+0000 7f0295191700 1 -- 172.21.15.36:0/3041739796 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f028000d570 con 0x7f0290060260 2022-06-17T12:56:50.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.006+0000 7f0295191700 1 -- 172.21.15.36:0/3041739796 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0280000e90 con 0x7f0290060260 2022-06-17T12:56:50.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.006+0000 7f0295191700 1 -- 172.21.15.36:0/3041739796 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f028000d8c0 con 0x7f0290060260 2022-06-17T12:56:50.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.006+0000 7f02983f7700 1 -- 172.21.15.36:0/3041739796 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0290060260 msgr2=0x7f02900b2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:50.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.006+0000 7f02983f7700 1 --2- 172.21.15.36:0/3041739796 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0290060260 0x7f02900b2e90 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 rx=0x7f0280002f70 tx=0x7f0280006df0).stop 2022-06-17T12:56:50.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.006+0000 7f02983f7700 1 -- 172.21.15.36:0/3041739796 shutdown_connections 2022-06-17T12:56:50.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.006+0000 7f02983f7700 1 --2- 172.21.15.36:0/3041739796 >> v2:172.21.15.70:3300/0 conn(0x7f02900a9b90 0x7f02900b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.006+0000 7f02983f7700 1 --2- 172.21.15.36:0/3041739796 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0290060260 0x7f02900b2e90 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.006+0000 7f02983f7700 1 -- 172.21.15.36:0/3041739796 >> 172.21.15.36:0/3041739796 conn(0x7f029005c550 msgr2=0x7f029005c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:50.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.006+0000 7f02983f7700 1 -- 172.21.15.36:0/3041739796 shutdown_connections 2022-06-17T12:56:50.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.006+0000 7f02983f7700 1 -- 172.21.15.36:0/3041739796 wait complete. 2022-06-17T12:56:50.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.006+0000 7f02983f7700 1 Processor -- start 2022-06-17T12:56:50.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.006+0000 7f02983f7700 1 -- start start 2022-06-17T12:56:50.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.006+0000 7f02983f7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0290060260 0x7f02900ab850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:50.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.006+0000 7f02983f7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f02900a9b90 0x7f0290137b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:50.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.006+0000 7f02983f7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f02900b0420 0x7f0290137ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:50.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.006+0000 7f02983f7700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0290152f60 con 0x7f02900b0420 2022-06-17T12:56:50.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.006+0000 7f02983f7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0290145af0 con 0x7f0290060260 2022-06-17T12:56:50.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.006+0000 7f02983f7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02900acaf0 con 0x7f02900a9b90 2022-06-17T12:56:50.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.010+0000 7f0296994700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f02900b0420 0x7f0290137ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:50.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.010+0000 7f0296994700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f02900b0420 0x7f0290137ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40026/0 (socket says 172.21.15.36:40026) 2022-06-17T12:56:50.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.010+0000 7f0296994700 1 -- 172.21.15.36:0/2841590467 learned_addr learned my addr 172.21.15.36:0/2841590467 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:50.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.010+0000 7f0296193700 1 --2- 172.21.15.36:0/2841590467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0290060260 0x7f02900ab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:50.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.010+0000 7f0295992700 1 --2- 172.21.15.36:0/2841590467 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f02900a9b90 0x7f0290137b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:50.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.010+0000 7f0296994700 1 -- 172.21.15.36:0/2841590467 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f02900a9b90 msgr2=0x7f0290137b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:50.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.010+0000 7f0296994700 1 --2- 172.21.15.36:0/2841590467 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f02900a9b90 0x7f0290137b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.010+0000 7f0296994700 1 -- 172.21.15.36:0/2841590467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0290060260 msgr2=0x7f02900ab850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:50.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.010+0000 7f0296994700 1 --2- 172.21.15.36:0/2841590467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0290060260 0x7f02900ab850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.010+0000 7f0296994700 1 -- 172.21.15.36:0/2841590467 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028000b180 con 0x7f02900b0420 2022-06-17T12:56:50.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.010+0000 7f0296994700 1 --2- 172.21.15.36:0/2841590467 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f02900b0420 0x7f0290137ff0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 rx=0x7f0288006e00 tx=0x7f02880091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:50.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.010+0000 7f027f7fe700 1 -- 172.21.15.36:0/2841590467 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f028800c640 con 0x7f02900b0420 2022-06-17T12:56:50.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.010+0000 7f02983f7700 1 -- 172.21.15.36:0/2841590467 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02900acd90 con 0x7f02900b0420 2022-06-17T12:56:50.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.010+0000 7f02983f7700 1 -- 172.21.15.36:0/2841590467 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02900ab110 con 0x7f02900b0420 2022-06-17T12:56:50.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.010+0000 7f027f7fe700 1 -- 172.21.15.36:0/2841590467 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028800c7a0 con 0x7f02900b0420 2022-06-17T12:56:50.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.010+0000 7f027f7fe700 1 -- 172.21.15.36:0/2841590467 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f028800cb00 con 0x7f02900b0420 2022-06-17T12:56:50.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.010+0000 7f027f7fe700 1 -- 172.21.15.36:0/2841590467 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f0288022070 con 0x7f02900b0420 2022-06-17T12:56:50.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.010+0000 7f02983f7700 1 -- 172.21.15.36:0/2841590467 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f026c003410 con 0x7f02900b0420 2022-06-17T12:56:50.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.010+0000 7f027f7fe700 1 --2- 172.21.15.36:0/2841590467 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f02700539b0 0x7f0270055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:50.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.010+0000 7f027f7fe700 1 -- 172.21.15.36:0/2841590467 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f028806c6e0 con 0x7f02900b0420 2022-06-17T12:56:50.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.010+0000 7f0296193700 1 --2- 172.21.15.36:0/2841590467 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f02700539b0 0x7f0270055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:50.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.014+0000 7f0296193700 1 --2- 172.21.15.36:0/2841590467 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f02700539b0 0x7f0270055e70 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f028c00d000 tx=0x7f028c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:50.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.018+0000 7f027f7fe700 1 -- 172.21.15.36:0/2841590467 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f02880433d0 con 0x7f02900b0420 2022-06-17T12:56:50.167 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.182+0000 7f02983f7700 1 -- 172.21.15.36:0/2841590467 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f026c003c40 con 0x7f02900b0420 2022-06-17T12:56:50.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.214+0000 7f027f7fe700 1 -- 172.21.15.36:0/2841590467 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v78) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0288016020 con 0x7f02900b0420 2022-06-17T12:56:50.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.218+0000 7f02983f7700 1 -- 172.21.15.36:0/2841590467 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f02700539b0 msgr2=0x7f0270055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:50.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.218+0000 7f02983f7700 1 --2- 172.21.15.36:0/2841590467 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f02700539b0 0x7f0270055e70 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f028c00d000 tx=0x7f028c009040).stop 2022-06-17T12:56:50.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.218+0000 7f02983f7700 1 -- 172.21.15.36:0/2841590467 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f02900b0420 msgr2=0x7f0290137ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:50.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.218+0000 7f02983f7700 1 --2- 172.21.15.36:0/2841590467 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f02900b0420 0x7f0290137ff0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 rx=0x7f0288006e00 tx=0x7f02880091d0).stop 2022-06-17T12:56:50.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.218+0000 7f02983f7700 1 -- 172.21.15.36:0/2841590467 shutdown_connections 2022-06-17T12:56:50.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.218+0000 7f02983f7700 1 --2- 172.21.15.36:0/2841590467 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f02700539b0 0x7f0270055e70 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.218+0000 7f02983f7700 1 --2- 172.21.15.36:0/2841590467 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f02900b0420 0x7f0290137ff0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.218+0000 7f02983f7700 1 --2- 172.21.15.36:0/2841590467 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f02900a9b90 0x7f0290137b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.218+0000 7f02983f7700 1 --2- 172.21.15.36:0/2841590467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0290060260 0x7f02900ab850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.218+0000 7f02983f7700 1 -- 172.21.15.36:0/2841590467 >> 172.21.15.36:0/2841590467 conn(0x7f029005c550 msgr2=0x7f029005f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:50.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.218+0000 7f02983f7700 1 -- 172.21.15.36:0/2841590467 shutdown_connections 2022-06-17T12:56:50.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.218+0000 7f02983f7700 1 -- 172.21.15.36:0/2841590467 wait complete. 2022-06-17T12:56:50.224 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:56:50.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.342+0000 7f229164f700 1 Processor -- start 2022-06-17T12:56:50.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.342+0000 7f229164f700 1 -- start start 2022-06-17T12:56:50.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f229164f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f228c144c00 0x7f228c062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:50.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f229164f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f228c0ad330 0x7f228c0ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:50.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f229164f700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f228c0b0ba0 con 0x7f228c062770 2022-06-17T12:56:50.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f229164f700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f228c0ae820 con 0x7f228c144c00 2022-06-17T12:56:50.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f229164f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228c061030 con 0x7f228c0ad330 2022-06-17T12:56:50.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f228a7fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f228c144c00 0x7f228c062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:50.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f228a7fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f228c144c00 0x7f228c062b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40030/0 (socket says 172.21.15.36:40030) 2022-06-17T12:56:50.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f228a7fc700 1 -- 172.21.15.36:0/1585812547 learned_addr learned my addr 172.21.15.36:0/1585812547 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:50.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f228affd700 1 --1- 172.21.15.36:0/1585812547 >> v1:172.21.15.36:6789/0 conn(0x7f228c062770 0x7f228c1444f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57562/0 (socket says 172.21.15.36:57562) 2022-06-17T12:56:50.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f228b7fe700 1 --2- 172.21.15.36:0/1585812547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f228c0ad330 0x7f228c0ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:50.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f228a7fc700 1 -- 172.21.15.36:0/1585812547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f228c0ad330 msgr2=0x7f228c0ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:50.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f228a7fc700 1 --2- 172.21.15.36:0/1585812547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f228c0ad330 0x7f228c0ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f228a7fc700 1 -- 172.21.15.36:0/1585812547 >> v1:172.21.15.36:6789/0 conn(0x7f228c062770 legacy=0x7f228c1444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:50.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f228a7fc700 1 -- 172.21.15.36:0/1585812547 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f228c147e90 con 0x7f228c144c00 2022-06-17T12:56:50.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f228b7fe700 1 --2- 172.21.15.36:0/1585812547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f228c0ad330 0x7f228c0ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:56:50.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f228a7fc700 1 --2- 172.21.15.36:0/1585812547 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f228c144c00 0x7f228c062b50 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x7f228c061170 tx=0x7f2274009290).ready entity=mon.0 client_cookie=68195a7e90a49717 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:50.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f2289ffb700 1 -- 172.21.15.36:0/1585812547 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f227400b860 con 0x7f228c144c00 2022-06-17T12:56:50.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f2289ffb700 1 -- 172.21.15.36:0/1585812547 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227400b9c0 con 0x7f228c144c00 2022-06-17T12:56:50.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f2289ffb700 1 -- 172.21.15.36:0/1585812547 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f227400bb30 con 0x7f228c144c00 2022-06-17T12:56:50.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f229164f700 1 -- 172.21.15.36:0/1585812547 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f228c144c00 msgr2=0x7f228c062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:50.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f229164f700 1 --2- 172.21.15.36:0/1585812547 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f228c144c00 0x7f228c062b50 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x7f228c061170 tx=0x7f2274009290).stop 2022-06-17T12:56:50.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f229164f700 1 -- 172.21.15.36:0/1585812547 shutdown_connections 2022-06-17T12:56:50.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f229164f700 1 --2- 172.21.15.36:0/1585812547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f228c0ad330 0x7f228c0ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f229164f700 1 --2- 172.21.15.36:0/1585812547 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f228c144c00 0x7f228c062b50 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f229164f700 1 -- 172.21.15.36:0/1585812547 >> 172.21.15.36:0/1585812547 conn(0x7f228c0b53e0 msgr2=0x7f228c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:50.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f229164f700 1 -- 172.21.15.36:0/1585812547 shutdown_connections 2022-06-17T12:56:50.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f229164f700 1 -- 172.21.15.36:0/1585812547 wait complete. 2022-06-17T12:56:50.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f229164f700 1 Processor -- start 2022-06-17T12:56:50.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.346+0000 7f229164f700 1 -- start start 2022-06-17T12:56:50.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.350+0000 7f229164f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f228c062770 0x7f228c0633d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:50.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.350+0000 7f229164f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f228c0ad330 0x7f228c1302d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:50.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.350+0000 7f229164f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f228c144c00 0x7f228c1307c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:50.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.350+0000 7f229164f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c066dc0 con 0x7f228c0ad330 2022-06-17T12:56:50.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.350+0000 7f229164f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c131230 con 0x7f228c062770 2022-06-17T12:56:50.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.350+0000 7f229164f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228c1314d0 con 0x7f228c144c00 2022-06-17T12:56:50.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.350+0000 7f228a7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f228c0ad330 0x7f228c1302d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:50.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.350+0000 7f228a7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f228c0ad330 0x7f228c1302d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40040/0 (socket says 172.21.15.36:40040) 2022-06-17T12:56:50.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.350+0000 7f228a7fc700 1 -- 172.21.15.36:0/2159618282 learned_addr learned my addr 172.21.15.36:0/2159618282 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:50.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.350+0000 7f228b7fe700 1 --2- 172.21.15.36:0/2159618282 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f228c144c00 0x7f228c1307c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:50.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.350+0000 7f228affd700 1 --2- 172.21.15.36:0/2159618282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f228c062770 0x7f228c0633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:50.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.350+0000 7f228a7fc700 1 -- 172.21.15.36:0/2159618282 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f228c144c00 msgr2=0x7f228c1307c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:50.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.350+0000 7f228a7fc700 1 --2- 172.21.15.36:0/2159618282 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f228c144c00 0x7f228c1307c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.350+0000 7f228a7fc700 1 -- 172.21.15.36:0/2159618282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f228c062770 msgr2=0x7f228c0633d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:50.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.350+0000 7f228a7fc700 1 --2- 172.21.15.36:0/2159618282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f228c062770 0x7f228c0633d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.350+0000 7f228a7fc700 1 -- 172.21.15.36:0/2159618282 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2274009530 con 0x7f228c0ad330 2022-06-17T12:56:50.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.350+0000 7f228a7fc700 1 --2- 172.21.15.36:0/2159618282 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f228c0ad330 0x7f228c1302d0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7f228c061170 tx=0x7f2274016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:50.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.350+0000 7f2283fff700 1 -- 172.21.15.36:0/2159618282 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2274021440 con 0x7f228c0ad330 2022-06-17T12:56:50.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.350+0000 7f2283fff700 1 -- 172.21.15.36:0/2159618282 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22740215a0 con 0x7f228c0ad330 2022-06-17T12:56:50.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.350+0000 7f229164f700 1 -- 172.21.15.36:0/2159618282 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228c064cd0 con 0x7f228c0ad330 2022-06-17T12:56:50.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.350+0000 7f2283fff700 1 -- 172.21.15.36:0/2159618282 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2274021900 con 0x7f228c0ad330 2022-06-17T12:56:50.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.350+0000 7f229164f700 1 -- 172.21.15.36:0/2159618282 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228c0668f0 con 0x7f228c0ad330 2022-06-17T12:56:50.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.354+0000 7f2283fff700 1 -- 172.21.15.36:0/2159618282 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2274029070 con 0x7f228c0ad330 2022-06-17T12:56:50.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.354+0000 7f229164f700 1 -- 172.21.15.36:0/2159618282 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f228c0652f0 con 0x7f228c0ad330 2022-06-17T12:56:50.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.354+0000 7f2283fff700 1 --2- 172.21.15.36:0/2159618282 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2270053920 0x7f2270055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:50.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.354+0000 7f2283fff700 1 -- 172.21.15.36:0/2159618282 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f227406d0d0 con 0x7f228c0ad330 2022-06-17T12:56:50.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.354+0000 7f228affd700 1 --2- 172.21.15.36:0/2159618282 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2270053920 0x7f2270055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:50.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.354+0000 7f228affd700 1 --2- 172.21.15.36:0/2159618282 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2270053920 0x7f2270055de0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f227c00d000 tx=0x7f227c007b40).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:50.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.358+0000 7f2283fff700 1 -- 172.21.15.36:0/2159618282 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f228c0652f0 con 0x7f228c0ad330 2022-06-17T12:56:50.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.530+0000 7f229164f700 1 -- 172.21.15.36:0/2159618282 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f228c0652f0 con 0x7f228c0ad330 2022-06-17T12:56:50.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.530+0000 7f2283fff700 1 -- 172.21.15.36:0/2159618282 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f228c0652f0 con 0x7f228c0ad330 2022-06-17T12:56:50.518 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:50.518 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":78,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1081450840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1081450840},{"type":"v1","addr":"172.21.15.70:6833","nonce":1081450840}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":76}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:56:49.200774+0000","modified":"2022-06-17T12:56:50.204509+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8478":{"gid":8478,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/3169088536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3169088536},{"type":"v1","addr":"172.21.15.36:6835","nonce":3169088536}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-06-17T12:56:50.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.530+0000 7f229164f700 1 -- 172.21.15.36:0/2159618282 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2270053920 msgr2=0x7f2270055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:50.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.530+0000 7f229164f700 1 --2- 172.21.15.36:0/2159618282 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2270053920 0x7f2270055de0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f227c00d000 tx=0x7f227c007b40).stop 2022-06-17T12:56:50.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.530+0000 7f229164f700 1 -- 172.21.15.36:0/2159618282 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f228c0ad330 msgr2=0x7f228c1302d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:50.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.530+0000 7f229164f700 1 --2- 172.21.15.36:0/2159618282 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f228c0ad330 0x7f228c1302d0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7f228c061170 tx=0x7f2274016000).stop 2022-06-17T12:56:50.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.530+0000 7f229164f700 1 -- 172.21.15.36:0/2159618282 shutdown_connections 2022-06-17T12:56:50.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.530+0000 7f229164f700 1 --2- 172.21.15.36:0/2159618282 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2270053920 0x7f2270055de0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.530+0000 7f229164f700 1 --2- 172.21.15.36:0/2159618282 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f228c144c00 0x7f228c1307c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.530+0000 7f229164f700 1 --2- 172.21.15.36:0/2159618282 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f228c0ad330 0x7f228c1302d0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.530+0000 7f229164f700 1 --2- 172.21.15.36:0/2159618282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f228c062770 0x7f228c0633d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.530+0000 7f229164f700 1 -- 172.21.15.36:0/2159618282 >> 172.21.15.36:0/2159618282 conn(0x7f228c0b53e0 msgr2=0x7f228c14dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:50.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.530+0000 7f229164f700 1 -- 172.21.15.36:0/2159618282 shutdown_connections 2022-06-17T12:56:50.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.530+0000 7f229164f700 1 -- 172.21.15.36:0/2159618282 wait complete. 2022-06-17T12:56:50.527 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 78 2022-06-17T12:56:50.529 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:56:50.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.662+0000 7fb98a44d700 1 Processor -- start 2022-06-17T12:56:50.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.662+0000 7fb98a44d700 1 -- start start 2022-06-17T12:56:50.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.662+0000 7fb98a44d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb9840b2340 0x7fb98413fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:50.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.662+0000 7fb98a44d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb984142e00 0x7fb984140340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:50.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.662+0000 7fb98a44d700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb984060e90 con 0x7fb984142a20 2022-06-17T12:56:50.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.662+0000 7fb98a44d700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9840b3520 con 0x7fb9840b2340 2022-06-17T12:56:50.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.662+0000 7fb98a44d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9840b37c0 con 0x7fb984142e00 2022-06-17T12:56:50.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.662+0000 7fb9837fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb9840b2340 0x7fb98413fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:50.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.662+0000 7fb9889ea700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb984142e00 0x7fb984140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:50.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.662+0000 7fb9837fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb9840b2340 0x7fb98413fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40046/0 (socket says 172.21.15.36:40046) 2022-06-17T12:56:50.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.662+0000 7fb9837fe700 1 -- 172.21.15.36:0/2852641964 learned_addr learned my addr 172.21.15.36:0/2852641964 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:50.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.662+0000 7fb983fff700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb984142a20 0x7fb984060780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:53308/0 (socket says 172.21.15.36:53308) 2022-06-17T12:56:50.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.662+0000 7fb9837fe700 1 -- 172.21.15.36:0/2852641964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb984142e00 msgr2=0x7fb984140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:50.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.662+0000 7fb9837fe700 1 --2- 172.21.15.36:0/2852641964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb984142e00 0x7fb984140340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.662+0000 7fb9837fe700 1 -- 172.21.15.36:0/2852641964 >> v1:172.21.15.70:6789/0 conn(0x7fb984142a20 legacy=0x7fb984060780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:50.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.662+0000 7fb982ffd700 1 -- 172.21.15.36:0/2852641964 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 145401096 0 0) 0x7fb984060e90 con 0x7fb984142a20 2022-06-17T12:56:50.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.662+0000 7fb9837fe700 1 -- 172.21.15.36:0/2852641964 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb984062f20 con 0x7fb9840b2340 2022-06-17T12:56:50.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.662+0000 7fb9889ea700 1 --2- 172.21.15.36:0/2852641964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb984142e00 0x7fb984140340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:50.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.666+0000 7fb9837fe700 1 --2- 172.21.15.36:0/2852641964 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb9840b2340 0x7fb98413fe50 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7fb978002f70 tx=0x7fb978006f40).ready entity=mon.0 client_cookie=135507ed19a2e6fd server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:50.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.666+0000 7fb982ffd700 1 -- 172.21.15.36:0/2852641964 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb97800da80 con 0x7fb9840b2340 2022-06-17T12:56:50.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.666+0000 7fb982ffd700 1 -- 172.21.15.36:0/2852641964 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb978000e90 con 0x7fb9840b2340 2022-06-17T12:56:50.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.666+0000 7fb982ffd700 1 -- 172.21.15.36:0/2852641964 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb97800ddd0 con 0x7fb9840b2340 2022-06-17T12:56:50.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.666+0000 7fb98a44d700 1 -- 172.21.15.36:0/2852641964 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb9840b2340 msgr2=0x7fb98413fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:50.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.666+0000 7fb98a44d700 1 --2- 172.21.15.36:0/2852641964 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb9840b2340 0x7fb98413fe50 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7fb978002f70 tx=0x7fb978006f40).stop 2022-06-17T12:56:50.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.666+0000 7fb98a44d700 1 -- 172.21.15.36:0/2852641964 shutdown_connections 2022-06-17T12:56:50.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.666+0000 7fb98a44d700 1 --2- 172.21.15.36:0/2852641964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb984142e00 0x7fb984140340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.666+0000 7fb98a44d700 1 --2- 172.21.15.36:0/2852641964 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb9840b2340 0x7fb98413fe50 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.666+0000 7fb98a44d700 1 -- 172.21.15.36:0/2852641964 >> 172.21.15.36:0/2852641964 conn(0x7fb9840b6870 msgr2=0x7fb9840b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:50.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.666+0000 7fb98a44d700 1 -- 172.21.15.36:0/2852641964 shutdown_connections 2022-06-17T12:56:50.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.666+0000 7fb98a44d700 1 -- 172.21.15.36:0/2852641964 wait complete. 2022-06-17T12:56:50.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.670+0000 7fb98a44d700 1 Processor -- start 2022-06-17T12:56:50.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.670+0000 7fb98a44d700 1 -- start start 2022-06-17T12:56:50.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.670+0000 7fb98a44d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9840b2340 0x7fb984066a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:50.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.670+0000 7fb98a44d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb984142a20 0x7fb98413dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:50.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.670+0000 7fb98a44d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb984142e00 0x7fb98413e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:50.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.670+0000 7fb98a44d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98413a560 con 0x7fb984142e00 2022-06-17T12:56:50.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.670+0000 7fb98a44d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98413fb50 con 0x7fb984142a20 2022-06-17T12:56:50.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.670+0000 7fb98a44d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9841525b0 con 0x7fb9840b2340 2022-06-17T12:56:50.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.670+0000 7fb9889ea700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb984142e00 0x7fb98413e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:50.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.670+0000 7fb9889ea700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb984142e00 0x7fb98413e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40052/0 (socket says 172.21.15.36:40052) 2022-06-17T12:56:50.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.670+0000 7fb9889ea700 1 -- 172.21.15.36:0/4054455520 learned_addr learned my addr 172.21.15.36:0/4054455520 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:50.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.670+0000 7fb983fff700 1 --2- 172.21.15.36:0/4054455520 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9840b2340 0x7fb984066a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:50.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.670+0000 7fb9837fe700 1 --2- 172.21.15.36:0/4054455520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb984142a20 0x7fb98413dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:50.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.670+0000 7fb9889ea700 1 -- 172.21.15.36:0/4054455520 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9840b2340 msgr2=0x7fb984066a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:50.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.670+0000 7fb9889ea700 1 --2- 172.21.15.36:0/4054455520 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9840b2340 0x7fb984066a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.670+0000 7fb9889ea700 1 -- 172.21.15.36:0/4054455520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb984142a20 msgr2=0x7fb98413dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:50.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.670+0000 7fb9889ea700 1 --2- 172.21.15.36:0/4054455520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb984142a20 0x7fb98413dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.670+0000 7fb9889ea700 1 -- 172.21.15.36:0/4054455520 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97c009470 con 0x7fb984142e00 2022-06-17T12:56:50.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.670+0000 7fb9889ea700 1 --2- 172.21.15.36:0/4054455520 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb984142e00 0x7fb98413e0d0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7fb97c00c000 tx=0x7fb97c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:50.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.670+0000 7fb9817fa700 1 -- 172.21.15.36:0/4054455520 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb97c00ee60 con 0x7fb984142e00 2022-06-17T12:56:50.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.670+0000 7fb9817fa700 1 -- 172.21.15.36:0/4054455520 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97c00c850 con 0x7fb984142e00 2022-06-17T12:56:50.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.670+0000 7fb98a44d700 1 -- 172.21.15.36:0/4054455520 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97800b460 con 0x7fb984142e00 2022-06-17T12:56:50.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.670+0000 7fb9817fa700 1 -- 172.21.15.36:0/4054455520 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb97c00cbb0 con 0x7fb984142e00 2022-06-17T12:56:50.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.674+0000 7fb98a44d700 1 -- 172.21.15.36:0/4054455520 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb984066f40 con 0x7fb984142e00 2022-06-17T12:56:50.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.674+0000 7fb98a44d700 1 -- 172.21.15.36:0/4054455520 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb984064a80 con 0x7fb984142e00 2022-06-17T12:56:50.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.674+0000 7fb9817fa700 1 -- 172.21.15.36:0/4054455520 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fb97c022070 con 0x7fb984142e00 2022-06-17T12:56:50.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.674+0000 7fb9817fa700 1 --2- 172.21.15.36:0/4054455520 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb964053ad0 0x7fb964055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:50.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.674+0000 7fb9817fa700 1 -- 172.21.15.36:0/4054455520 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb97c06c720 con 0x7fb984142e00 2022-06-17T12:56:50.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.674+0000 7fb983fff700 1 --2- 172.21.15.36:0/4054455520 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb964053ad0 0x7fb964055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:50.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.674+0000 7fb983fff700 1 --2- 172.21.15.36:0/4054455520 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb964053ad0 0x7fb964055f90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7fb97400d000 tx=0x7fb9740079e0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:50.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.682+0000 7fb9817fa700 1 -- 172.21.15.36:0/4054455520 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb984064a80 con 0x7fb984142e00 2022-06-17T12:56:50.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.806+0000 7fb98a44d700 1 -- 172.21.15.36:0/4054455520 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb984064a80 con 0x7fb984142e00 2022-06-17T12:56:50.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.810+0000 7fb9817fa700 1 -- 172.21.15.36:0/4054455520 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+16285 (secure 0 0 0) 0x7fb984064a80 con 0x7fb984142e00 2022-06-17T12:56:50.821 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:50.822 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":141,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:56:50.204603+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:56:47.177576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-06-17T12:56:48.257688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:56:50.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.810+0000 7fb98a44d700 1 -- 172.21.15.36:0/4054455520 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb964053ad0 msgr2=0x7fb964055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:50.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.810+0000 7fb98a44d700 1 --2- 172.21.15.36:0/4054455520 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb964053ad0 0x7fb964055f90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7fb97400d000 tx=0x7fb9740079e0).stop 2022-06-17T12:56:50.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.810+0000 7fb98a44d700 1 -- 172.21.15.36:0/4054455520 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb984142e00 msgr2=0x7fb98413e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:50.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.810+0000 7fb98a44d700 1 --2- 172.21.15.36:0/4054455520 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb984142e00 0x7fb98413e0d0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7fb97c00c000 tx=0x7fb97c0091d0).stop 2022-06-17T12:56:50.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.810+0000 7fb98a44d700 1 -- 172.21.15.36:0/4054455520 shutdown_connections 2022-06-17T12:56:50.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.810+0000 7fb98a44d700 1 --2- 172.21.15.36:0/4054455520 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb964053ad0 0x7fb964055f90 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.810+0000 7fb98a44d700 1 --2- 172.21.15.36:0/4054455520 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb984142e00 0x7fb98413e0d0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.810+0000 7fb98a44d700 1 --2- 172.21.15.36:0/4054455520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb984142a20 0x7fb98413dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.810+0000 7fb98a44d700 1 --2- 172.21.15.36:0/4054455520 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9840b2340 0x7fb984066a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:50.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.810+0000 7fb98a44d700 1 -- 172.21.15.36:0/4054455520 >> 172.21.15.36:0/4054455520 conn(0x7fb9840b6870 msgr2=0x7fb9841393d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:50.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.810+0000 7fb98a44d700 1 -- 172.21.15.36:0/4054455520 shutdown_connections 2022-06-17T12:56:50.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.810+0000 7fb98a44d700 1 -- 172.21.15.36:0/4054455520 wait complete. 2022-06-17T12:56:50.832 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:56:50.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.982+0000 7f9aafd99700 1 Processor -- start 2022-06-17T12:56:50.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.982+0000 7f9aafd99700 1 -- start start 2022-06-17T12:56:50.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aafd99700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9aa8142f20 0x7f9aa8143e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:50.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aafd99700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aa80b44b0 con 0x7f9aa8062270 2022-06-17T12:56:50.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aafd99700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aa8062650 con 0x7f9aa80b0cb0 2022-06-17T12:56:50.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aafd99700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa80b6650 con 0x7f9aa8142f20 2022-06-17T12:56:50.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aadb35700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f9aa8062270 0x7f9aa8143770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57588/0 (socket says 172.21.15.36:57588) 2022-06-17T12:56:50.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aadb35700 1 -- 172.21.15.36:0/136949902 learned_addr learned my addr 172.21.15.36:0/136949902 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:50.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aae336700 1 --2- 172.21.15.36:0/136949902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9aa8142f20 0x7f9aa8143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:50.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aacb33700 1 -- 172.21.15.36:0/136949902 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4193862282 0 0) 0x7f9aa80b44b0 con 0x7f9aa8062270 2022-06-17T12:56:50.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aacb33700 1 -- 172.21.15.36:0/136949902 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a90003260 con 0x7f9aa8062270 2022-06-17T12:56:50.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aacb33700 1 -- 172.21.15.36:0/136949902 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 925159157 0 0) 0x7f9aa8062650 con 0x7f9aa80b0cb0 2022-06-17T12:56:50.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aacb33700 1 -- 172.21.15.36:0/136949902 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9aa80b44b0 con 0x7f9aa80b0cb0 2022-06-17T12:56:50.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aae336700 1 -- 172.21.15.36:0/136949902 >> v1:172.21.15.70:6789/0 conn(0x7f9aa80b0cb0 legacy=0x7f9aa8146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:50.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aae336700 1 -- 172.21.15.36:0/136949902 >> v1:172.21.15.36:6789/0 conn(0x7f9aa8062270 legacy=0x7f9aa8143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:50.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aae336700 1 -- 172.21.15.36:0/136949902 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa80b4210 con 0x7f9aa8142f20 2022-06-17T12:56:50.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aae336700 1 --2- 172.21.15.36:0/136949902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9aa8142f20 0x7f9aa8143e80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f9aa00032f0 tx=0x7f9aa00091d0).ready entity=mon.2 client_cookie=3e1d3e536b5981a6 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:50.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aacb33700 1 -- 172.21.15.36:0/136949902 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9aa000b640 con 0x7f9aa8142f20 2022-06-17T12:56:50.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aacb33700 1 -- 172.21.15.36:0/136949902 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa000b7a0 con 0x7f9aa8142f20 2022-06-17T12:56:50.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aacb33700 1 -- 172.21.15.36:0/136949902 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9aa000b910 con 0x7f9aa8142f20 2022-06-17T12:56:50.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aafd99700 1 -- 172.21.15.36:0/136949902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9aa8142f20 msgr2=0x7f9aa8143e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:50.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aafd99700 1 --2- 172.21.15.36:0/136949902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9aa8142f20 0x7f9aa8143e80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f9aa00032f0 tx=0x7f9aa00091d0).stop 2022-06-17T12:56:50.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aafd99700 1 -- 172.21.15.36:0/136949902 shutdown_connections 2022-06-17T12:56:51.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aafd99700 1 --2- 172.21.15.36:0/136949902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9aa8142f20 0x7f9aa8143e80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:51.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aafd99700 1 -- 172.21.15.36:0/136949902 >> 172.21.15.36:0/136949902 conn(0x7f9aa80b9870 msgr2=0x7f9aa80b9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:51.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aafd99700 1 -- 172.21.15.36:0/136949902 shutdown_connections 2022-06-17T12:56:51.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.986+0000 7f9aafd99700 1 -- 172.21.15.36:0/136949902 wait complete. 2022-06-17T12:56:51.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.990+0000 7f9aafd99700 1 Processor -- start 2022-06-17T12:56:51.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.990+0000 7f9aafd99700 1 -- start start 2022-06-17T12:56:51.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.990+0000 7f9aafd99700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9aa8062270 0x7f9aa8146710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:51.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.990+0000 7f9aafd99700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9aa8142f20 0x7f9aa8147fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:51.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.990+0000 7f9aafd99700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9aa80afa00 0x7f9aa81404b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:51.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.990+0000 7f9aafd99700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa8146db0 con 0x7f9aa8142f20 2022-06-17T12:56:51.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.990+0000 7f9aafd99700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa8147050 con 0x7f9aa8062270 2022-06-17T12:56:51.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.990+0000 7f9aafd99700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa8152430 con 0x7f9aa80afa00 2022-06-17T12:56:51.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.990+0000 7f9aad334700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9aa8142f20 0x7f9aa8147fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:51.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.990+0000 7f9aad334700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9aa8142f20 0x7f9aa8147fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40066/0 (socket says 172.21.15.36:40066) 2022-06-17T12:56:51.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.990+0000 7f9aad334700 1 -- 172.21.15.36:0/5194851 learned_addr learned my addr 172.21.15.36:0/5194851 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:51.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.990+0000 7f9aae336700 1 --2- 172.21.15.36:0/5194851 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9aa80afa00 0x7f9aa81404b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:51.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.990+0000 7f9aadb35700 1 --2- 172.21.15.36:0/5194851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9aa8062270 0x7f9aa8146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:51.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.990+0000 7f9aad334700 1 -- 172.21.15.36:0/5194851 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9aa80afa00 msgr2=0x7f9aa81404b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:51.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.990+0000 7f9aad334700 1 --2- 172.21.15.36:0/5194851 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9aa80afa00 0x7f9aa81404b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:51.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.990+0000 7f9aad334700 1 -- 172.21.15.36:0/5194851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9aa8062270 msgr2=0x7f9aa8146710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:51.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.990+0000 7f9aad334700 1 --2- 172.21.15.36:0/5194851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9aa8062270 0x7f9aa8146710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:51.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.990+0000 7f9aad334700 1 -- 172.21.15.36:0/5194851 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa0009470 con 0x7f9aa8142f20 2022-06-17T12:56:51.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.990+0000 7f9aad334700 1 --2- 172.21.15.36:0/5194851 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9aa8142f20 0x7f9aa8147fd0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0x7f9aa4001110 tx=0x7f9aa4007a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:51.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.994+0000 7f9a96ffd700 1 -- 172.21.15.36:0/5194851 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9aa4012400 con 0x7f9aa8142f20 2022-06-17T12:56:51.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.994+0000 7f9a96ffd700 1 -- 172.21.15.36:0/5194851 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa4012560 con 0x7f9aa8142f20 2022-06-17T12:56:51.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.994+0000 7f9aafd99700 1 -- 172.21.15.36:0/5194851 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aa81409a0 con 0x7f9aa8142f20 2022-06-17T12:56:51.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.994+0000 7f9a96ffd700 1 -- 172.21.15.36:0/5194851 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9aa40128c0 con 0x7f9aa8142f20 2022-06-17T12:56:51.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.994+0000 7f9aafd99700 1 -- 172.21.15.36:0/5194851 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa8152a70 con 0x7f9aa8142f20 2022-06-17T12:56:51.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.994+0000 7f9aafd99700 1 -- 172.21.15.36:0/5194851 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aa8146c00 con 0x7f9aa8142f20 2022-06-17T12:56:51.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.994+0000 7f9a96ffd700 1 -- 172.21.15.36:0/5194851 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f9aa4019030 con 0x7f9aa8142f20 2022-06-17T12:56:51.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.994+0000 7f9a96ffd700 1 --2- 172.21.15.36:0/5194851 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9a88053920 0x7f9a88055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:51.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.994+0000 7f9a96ffd700 1 -- 172.21.15.36:0/5194851 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9aa406b430 con 0x7f9aa8142f20 2022-06-17T12:56:51.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.994+0000 7f9aadb35700 1 --2- 172.21.15.36:0/5194851 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9a88053920 0x7f9a88055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:51.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:50.998+0000 7f9aadb35700 1 --2- 172.21.15.36:0/5194851 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9a88053920 0x7f9a88055de0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f9a9800d000 tx=0x7f9a98009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:51.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.006+0000 7f9a96ffd700 1 -- 172.21.15.36:0/5194851 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9aa4042190 con 0x7f9aa8142f20 2022-06-17T12:56:51.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.174+0000 7f9aafd99700 1 -- 172.21.15.36:0/5194851 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f9aa81525f0 con 0x7f9aa8142f20 2022-06-17T12:56:51.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.186+0000 7f9a96ffd700 1 -- 172.21.15.36:0/5194851 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v44) v1 ==== 184+0+0 (secure 0 0 0) 0x7f9aa40457b0 con 0x7f9aa8142f20 2022-06-17T12:56:51.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.186+0000 7f9aafd99700 1 -- 172.21.15.36:0/5194851 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9a88053920 msgr2=0x7f9a88055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:51.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.186+0000 7f9aafd99700 1 --2- 172.21.15.36:0/5194851 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9a88053920 0x7f9a88055de0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f9a9800d000 tx=0x7f9a98009040).stop 2022-06-17T12:56:51.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.186+0000 7f9aafd99700 1 -- 172.21.15.36:0/5194851 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9aa8142f20 msgr2=0x7f9aa8147fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:51.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.186+0000 7f9aafd99700 1 --2- 172.21.15.36:0/5194851 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9aa8142f20 0x7f9aa8147fd0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0x7f9aa4001110 tx=0x7f9aa4007a90).stop 2022-06-17T12:56:51.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.186+0000 7f9aafd99700 1 -- 172.21.15.36:0/5194851 shutdown_connections 2022-06-17T12:56:51.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.186+0000 7f9aafd99700 1 --2- 172.21.15.36:0/5194851 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9a88053920 0x7f9a88055de0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:51.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.186+0000 7f9aafd99700 1 --2- 172.21.15.36:0/5194851 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9aa80afa00 0x7f9aa81404b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:51.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.186+0000 7f9aafd99700 1 --2- 172.21.15.36:0/5194851 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9aa8142f20 0x7f9aa8147fd0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:51.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.186+0000 7f9aafd99700 1 --2- 172.21.15.36:0/5194851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9aa8062270 0x7f9aa8146710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:51.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.186+0000 7f9aafd99700 1 -- 172.21.15.36:0/5194851 >> 172.21.15.36:0/5194851 conn(0x7f9aa80b9870 msgr2=0x7f9aa80b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:51.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.186+0000 7f9aafd99700 1 -- 172.21.15.36:0/5194851 shutdown_connections 2022-06-17T12:56:51.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.186+0000 7f9aafd99700 1 -- 172.21.15.36:0/5194851 wait complete. 2022-06-17T12:56:51.187 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.0 2022-06-17T12:56:51.192 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:56:51.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.330+0000 7fb7ebfff700 1 Processor -- start 2022-06-17T12:56:51.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.330+0000 7fb7ebfff700 1 -- start start 2022-06-17T12:56:51.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.330+0000 7fb7ebfff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb7ec13cb70 0x7fb7ec143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:51.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.330+0000 7fb7ebfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb7ec13cf50 0x7fb7ec1442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:51.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.330+0000 7fb7ebfff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7ec0af4b0 con 0x7fb7ec0aef60 2022-06-17T12:56:51.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.330+0000 7fb7ebfff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7ec0ba500 con 0x7fb7ec13cf50 2022-06-17T12:56:51.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.330+0000 7fb7ebfff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7ec142f50 con 0x7fb7ec13cb70 2022-06-17T12:56:51.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.330+0000 7fb7eb7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fb7ec13cf50 0x7fb7ec1442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:51.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.330+0000 7fb7ea7fc700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fb7ec0aef60 0x7fb7ec0b9df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57602/0 (socket says 172.21.15.36:57602) 2022-06-17T12:56:51.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.330+0000 7fb7ea7fc700 1 -- 172.21.15.36:0/4101530516 learned_addr learned my addr 172.21.15.36:0/4101530516 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:51.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.330+0000 7fb7eaffd700 1 --2- 172.21.15.36:0/4101530516 >> v2:172.21.15.70:3300/0 conn(0x7fb7ec13cb70 0x7fb7ec143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:51.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.330+0000 7fb7e9ffb700 1 -- 172.21.15.36:0/4101530516 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2281975906 0 0) 0x7fb7ec0af4b0 con 0x7fb7ec0aef60 2022-06-17T12:56:51.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.330+0000 7fb7e9ffb700 1 -- 172.21.15.36:0/4101530516 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7cc003260 con 0x7fb7ec0aef60 2022-06-17T12:56:51.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.330+0000 7fb7e9ffb700 1 -- 172.21.15.36:0/4101530516 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2943625536 0 0) 0x7fb7cc003260 con 0x7fb7ec0aef60 2022-06-17T12:56:51.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.330+0000 7fb7eb7fe700 1 -- 172.21.15.36:0/4101530516 >> v2:172.21.15.70:3300/0 conn(0x7fb7ec13cb70 msgr2=0x7fb7ec143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:51.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.330+0000 7fb7eb7fe700 1 --2- 172.21.15.36:0/4101530516 >> v2:172.21.15.70:3300/0 conn(0x7fb7ec13cb70 0x7fb7ec143690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:51.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.330+0000 7fb7eb7fe700 1 -- 172.21.15.36:0/4101530516 >> v1:172.21.15.36:6789/0 conn(0x7fb7ec0aef60 legacy=0x7fb7ec0b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:51.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.330+0000 7fb7eb7fe700 1 -- 172.21.15.36:0/4101530516 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ec1431f0 con 0x7fb7ec13cf50 2022-06-17T12:56:51.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.330+0000 7fb7eb7fe700 1 --2- 172.21.15.36:0/4101530516 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb7ec13cf50 0x7fb7ec1442a0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7fb7d4001800 tx=0x7fb7dc006df0).ready entity=mon.0 client_cookie=4a336903419a3693 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:51.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.330+0000 7fb7e9ffb700 1 -- 172.21.15.36:0/4101530516 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7d4001660 con 0x7fb7ec13cf50 2022-06-17T12:56:51.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.330+0000 7fb7e9ffb700 1 -- 172.21.15.36:0/4101530516 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7dc000e90 con 0x7fb7ec13cf50 2022-06-17T12:56:51.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.330+0000 7fb7e9ffb700 1 -- 172.21.15.36:0/4101530516 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7dc012460 con 0x7fb7ec13cf50 2022-06-17T12:56:51.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.334+0000 7fb7ebfff700 1 -- 172.21.15.36:0/4101530516 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb7ec13cf50 msgr2=0x7fb7ec1442a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:51.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.334+0000 7fb7ebfff700 1 --2- 172.21.15.36:0/4101530516 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb7ec13cf50 0x7fb7ec1442a0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7fb7d4001800 tx=0x7fb7dc006df0).stop 2022-06-17T12:56:51.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.334+0000 7fb7ebfff700 1 -- 172.21.15.36:0/4101530516 shutdown_connections 2022-06-17T12:56:51.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.334+0000 7fb7ebfff700 1 --2- 172.21.15.36:0/4101530516 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb7ec13cf50 0x7fb7ec1442a0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:51.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.334+0000 7fb7ebfff700 1 --2- 172.21.15.36:0/4101530516 >> v2:172.21.15.70:3300/0 conn(0x7fb7ec13cb70 0x7fb7ec143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:51.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.334+0000 7fb7ebfff700 1 -- 172.21.15.36:0/4101530516 >> 172.21.15.36:0/4101530516 conn(0x7fb7ec05c550 msgr2=0x7fb7ec0b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:51.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.334+0000 7fb7ebfff700 1 -- 172.21.15.36:0/4101530516 shutdown_connections 2022-06-17T12:56:51.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.334+0000 7fb7ebfff700 1 -- 172.21.15.36:0/4101530516 wait complete. 2022-06-17T12:56:51.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.334+0000 7fb7ebfff700 1 Processor -- start 2022-06-17T12:56:51.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.334+0000 7fb7ebfff700 1 -- start start 2022-06-17T12:56:51.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.334+0000 7fb7ebfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7ec0aef60 0x7fb7ec1408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:51.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.334+0000 7fb7ebfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7ec13cb70 0x7fb7ec140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:51.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.334+0000 7fb7ebfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb7ec13cf50 0x7fb7ec13e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:51.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.334+0000 7fb7ebfff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec1417c0 con 0x7fb7ec13cf50 2022-06-17T12:56:51.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.338+0000 7fb7ebfff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec141a60 con 0x7fb7ec0aef60 2022-06-17T12:56:51.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.338+0000 7fb7ebfff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec0b2540 con 0x7fb7ec13cb70 2022-06-17T12:56:51.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.338+0000 7fb7eb7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb7ec13cf50 0x7fb7ec13e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:51.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.338+0000 7fb7eb7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb7ec13cf50 0x7fb7ec13e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40082/0 (socket says 172.21.15.36:40082) 2022-06-17T12:56:51.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.338+0000 7fb7eb7fe700 1 -- 172.21.15.36:0/972139773 learned_addr learned my addr 172.21.15.36:0/972139773 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:51.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.338+0000 7fb7ea7fc700 1 --2- 172.21.15.36:0/972139773 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7ec13cb70 0x7fb7ec140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:51.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.338+0000 7fb7eaffd700 1 --2- 172.21.15.36:0/972139773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7ec0aef60 0x7fb7ec1408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:51.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.338+0000 7fb7eb7fe700 1 -- 172.21.15.36:0/972139773 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7ec13cb70 msgr2=0x7fb7ec140da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:51.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.338+0000 7fb7eb7fe700 1 --2- 172.21.15.36:0/972139773 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7ec13cb70 0x7fb7ec140da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:51.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.338+0000 7fb7eb7fe700 1 -- 172.21.15.36:0/972139773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7ec0aef60 msgr2=0x7fb7ec1408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:51.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.338+0000 7fb7eb7fe700 1 --2- 172.21.15.36:0/972139773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7ec0aef60 0x7fb7ec1408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:51.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.338+0000 7fb7eb7fe700 1 -- 172.21.15.36:0/972139773 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7dc00b180 con 0x7fb7ec13cf50 2022-06-17T12:56:51.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.338+0000 7fb7eb7fe700 1 --2- 172.21.15.36:0/972139773 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb7ec13cf50 0x7fb7ec13e5a0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 rx=0x7fb7d4001800 tx=0x7fb7dc014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:51.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.338+0000 7fb7e3fff700 1 -- 172.21.15.36:0/972139773 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7dc00d8a0 con 0x7fb7ec13cf50 2022-06-17T12:56:51.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.338+0000 7fb7e3fff700 1 -- 172.21.15.36:0/972139773 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7dc00da00 con 0x7fb7ec13cf50 2022-06-17T12:56:51.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.338+0000 7fb7e3fff700 1 -- 172.21.15.36:0/972139773 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7dc00dd60 con 0x7fb7ec13cf50 2022-06-17T12:56:51.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.338+0000 7fb7ebfff700 1 -- 172.21.15.36:0/972139773 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7ec141290 con 0x7fb7ec13cf50 2022-06-17T12:56:51.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.338+0000 7fb7ebfff700 1 -- 172.21.15.36:0/972139773 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7ec0b0610 con 0x7fb7ec13cf50 2022-06-17T12:56:51.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.338+0000 7fb7ebfff700 1 -- 172.21.15.36:0/972139773 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7ec057410 con 0x7fb7ec13cf50 2022-06-17T12:56:51.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.338+0000 7fb7e3fff700 1 -- 172.21.15.36:0/972139773 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fb7dc025030 con 0x7fb7ec13cf50 2022-06-17T12:56:51.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.338+0000 7fb7e3fff700 1 --2- 172.21.15.36:0/972139773 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb7d00539b0 0x7fb7d0055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:51.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.342+0000 7fb7eaffd700 1 --2- 172.21.15.36:0/972139773 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb7d00539b0 0x7fb7d0055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:51.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.342+0000 7fb7e3fff700 1 -- 172.21.15.36:0/972139773 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb7dc06cbe0 con 0x7fb7ec13cf50 2022-06-17T12:56:51.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.342+0000 7fb7eaffd700 1 --2- 172.21.15.36:0/972139773 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb7d00539b0 0x7fb7d0055e70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7fb7e400b000 tx=0x7fb7e40091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:51.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.346+0000 7fb7e3fff700 1 -- 172.21.15.36:0/972139773 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb7dc043a20 con 0x7fb7ec13cf50 2022-06-17T12:56:51.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.510+0000 7fb7ebfff700 1 -- 172.21.15.36:0/972139773 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fb7ec141bd0 con 0x7fb7ec13cf50 2022-06-17T12:56:51.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.522+0000 7fb7e3fff700 1 -- 172.21.15.36:0/972139773 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v45) v1 ==== 184+0+0 (secure 0 0 0) 0x7fb7dc023070 con 0x7fb7ec13cf50 2022-06-17T12:56:51.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.522+0000 7fb7ebfff700 1 -- 172.21.15.36:0/972139773 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb7d00539b0 msgr2=0x7fb7d0055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:51.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.522+0000 7fb7ebfff700 1 --2- 172.21.15.36:0/972139773 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb7d00539b0 0x7fb7d0055e70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7fb7e400b000 tx=0x7fb7e40091d0).stop 2022-06-17T12:56:51.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.522+0000 7fb7ebfff700 1 -- 172.21.15.36:0/972139773 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb7ec13cf50 msgr2=0x7fb7ec13e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:51.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.522+0000 7fb7ebfff700 1 --2- 172.21.15.36:0/972139773 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb7ec13cf50 0x7fb7ec13e5a0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 rx=0x7fb7d4001800 tx=0x7fb7dc014000).stop 2022-06-17T12:56:51.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.522+0000 7fb7ebfff700 1 -- 172.21.15.36:0/972139773 shutdown_connections 2022-06-17T12:56:51.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.522+0000 7fb7ebfff700 1 --2- 172.21.15.36:0/972139773 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb7d00539b0 0x7fb7d0055e70 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:51.566 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.522+0000 7fb7ebfff700 1 --2- 172.21.15.36:0/972139773 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb7ec13cf50 0x7fb7ec13e5a0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:51.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.522+0000 7fb7ebfff700 1 --2- 172.21.15.36:0/972139773 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7ec13cb70 0x7fb7ec140da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:51.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.522+0000 7fb7ebfff700 1 --2- 172.21.15.36:0/972139773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7ec0aef60 0x7fb7ec1408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:51.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.522+0000 7fb7ebfff700 1 -- 172.21.15.36:0/972139773 >> 172.21.15.36:0/972139773 conn(0x7fb7ec05c550 msgr2=0x7fb7ec13b680 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:51.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.522+0000 7fb7ebfff700 1 -- 172.21.15.36:0/972139773 shutdown_connections 2022-06-17T12:56:51.568 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.522+0000 7fb7ebfff700 1 -- 172.21.15.36:0/972139773 wait complete. 2022-06-17T12:56:51.569 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.1 2022-06-17T12:56:51.570 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:56:51.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.706+0000 7fd8ca21c700 1 Processor -- start 2022-06-17T12:56:51.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.706+0000 7fd8ca21c700 1 -- start start 2022-06-17T12:56:51.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.710+0000 7fd8ca21c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd8c4062270 0x7fd8c4142f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:51.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.710+0000 7fd8ca21c700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c40b44b0 con 0x7fd8c4143e80 2022-06-17T12:56:51.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.710+0000 7fd8ca21c700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c4062650 con 0x7fd8c40b0cb0 2022-06-17T12:56:51.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.710+0000 7fd8ca21c700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c40b6650 con 0x7fd8c4062270 2022-06-17T12:56:51.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.710+0000 7fd8c37fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd8c4062270 0x7fd8c4142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:51.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.710+0000 7fd8c3fff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fd8c4143e80 0x7fd8c4146970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57618/0 (socket says 172.21.15.36:57618) 2022-06-17T12:56:51.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.710+0000 7fd8c3fff700 1 -- 172.21.15.36:0/2189620475 learned_addr learned my addr 172.21.15.36:0/2189620475 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:51.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.710+0000 7fd8c27fc700 1 -- 172.21.15.36:0/2189620475 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2929930927 0 0) 0x7fd8c40b44b0 con 0x7fd8c4143e80 2022-06-17T12:56:51.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.710+0000 7fd8c27fc700 1 -- 172.21.15.36:0/2189620475 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8a4003260 con 0x7fd8c4143e80 2022-06-17T12:56:51.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.710+0000 7fd8c37fe700 1 -- 172.21.15.36:0/2189620475 >> v1:172.21.15.70:6789/0 conn(0x7fd8c40b0cb0 legacy=0x7fd8c4143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:51.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.710+0000 7fd8c37fe700 1 -- 172.21.15.36:0/2189620475 >> v1:172.21.15.36:6789/0 conn(0x7fd8c4143e80 legacy=0x7fd8c4146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:51.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.710+0000 7fd8c37fe700 1 -- 172.21.15.36:0/2189620475 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c40b4210 con 0x7fd8c4062270 2022-06-17T12:56:51.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.710+0000 7fd8c37fe700 1 --2- 172.21.15.36:0/2189620475 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8c4062270 0x7fd8c4142f20 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0x7fd8b4002f70 tx=0x7fd8b4006df0).ready entity=mon.0 client_cookie=723e62097234f0ad server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:51.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.710+0000 7fd8c27fc700 1 -- 172.21.15.36:0/2189620475 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8b400d610 con 0x7fd8c4062270 2022-06-17T12:56:51.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.710+0000 7fd8c27fc700 1 -- 172.21.15.36:0/2189620475 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b4000e90 con 0x7fd8c4062270 2022-06-17T12:56:51.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.710+0000 7fd8c27fc700 1 -- 172.21.15.36:0/2189620475 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8b400d960 con 0x7fd8c4062270 2022-06-17T12:56:51.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.710+0000 7fd8ca21c700 1 -- 172.21.15.36:0/2189620475 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8c4062270 msgr2=0x7fd8c4142f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:51.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.710+0000 7fd8ca21c700 1 --2- 172.21.15.36:0/2189620475 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8c4062270 0x7fd8c4142f20 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0x7fd8b4002f70 tx=0x7fd8b4006df0).stop 2022-06-17T12:56:51.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.710+0000 7fd8ca21c700 1 -- 172.21.15.36:0/2189620475 shutdown_connections 2022-06-17T12:56:51.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.710+0000 7fd8ca21c700 1 --2- 172.21.15.36:0/2189620475 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8c4062270 0x7fd8c4142f20 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:51.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.710+0000 7fd8ca21c700 1 -- 172.21.15.36:0/2189620475 >> 172.21.15.36:0/2189620475 conn(0x7fd8c40b9870 msgr2=0x7fd8c40b9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:51.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.710+0000 7fd8ca21c700 1 -- 172.21.15.36:0/2189620475 shutdown_connections 2022-06-17T12:56:51.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.710+0000 7fd8ca21c700 1 -- 172.21.15.36:0/2189620475 wait complete. 2022-06-17T12:56:51.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.714+0000 7fd8ca21c700 1 Processor -- start 2022-06-17T12:56:51.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.714+0000 7fd8ca21c700 1 -- start start 2022-06-17T12:56:51.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.714+0000 7fd8ca21c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8c4062270 0x7fd8c413e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:51.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.714+0000 7fd8ca21c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8c4143e80 0x7fd8c413d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:51.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.714+0000 7fd8ca21c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8c40ad330 0x7fd8c413d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:51.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.714+0000 7fd8ca21c700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c4147010 con 0x7fd8c4062270 2022-06-17T12:56:51.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.714+0000 7fd8ca21c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c41472b0 con 0x7fd8c4143e80 2022-06-17T12:56:51.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.714+0000 7fd8ca21c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c4147550 con 0x7fd8c40ad330 2022-06-17T12:56:51.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.714+0000 7fd8c37fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8c4062270 0x7fd8c413e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:51.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.714+0000 7fd8c37fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8c4062270 0x7fd8c413e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40090/0 (socket says 172.21.15.36:40090) 2022-06-17T12:56:51.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.714+0000 7fd8c37fe700 1 -- 172.21.15.36:0/2642265455 learned_addr learned my addr 172.21.15.36:0/2642265455 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:51.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.714+0000 7fd8c2ffd700 1 --2- 172.21.15.36:0/2642265455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8c4143e80 0x7fd8c413d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:51.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.714+0000 7fd8c3fff700 1 --2- 172.21.15.36:0/2642265455 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8c40ad330 0x7fd8c413d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:51.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.714+0000 7fd8c37fe700 1 -- 172.21.15.36:0/2642265455 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8c40ad330 msgr2=0x7fd8c413d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:51.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.714+0000 7fd8c37fe700 1 --2- 172.21.15.36:0/2642265455 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8c40ad330 0x7fd8c413d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:51.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.714+0000 7fd8c37fe700 1 -- 172.21.15.36:0/2642265455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8c4143e80 msgr2=0x7fd8c413d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:51.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.714+0000 7fd8c37fe700 1 --2- 172.21.15.36:0/2642265455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8c4143e80 0x7fd8c413d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:51.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.714+0000 7fd8c37fe700 1 -- 172.21.15.36:0/2642265455 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b400b180 con 0x7fd8c4062270 2022-06-17T12:56:51.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.714+0000 7fd8c37fe700 1 --2- 172.21.15.36:0/2642265455 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8c4062270 0x7fd8c413e760 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 rx=0x7fd8b4002f70 tx=0x7fd8b4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:51.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.714+0000 7fd8c0ff9700 1 -- 172.21.15.36:0/2642265455 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8b4016070 con 0x7fd8c4062270 2022-06-17T12:56:51.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.714+0000 7fd8c0ff9700 1 -- 172.21.15.36:0/2642265455 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b400de90 con 0x7fd8c4062270 2022-06-17T12:56:51.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.714+0000 7fd8ca21c700 1 -- 172.21.15.36:0/2642265455 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c413de60 con 0x7fd8c4062270 2022-06-17T12:56:51.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.714+0000 7fd8c0ff9700 1 -- 172.21.15.36:0/2642265455 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8b4013e50 con 0x7fd8c4062270 2022-06-17T12:56:51.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.714+0000 7fd8ca21c700 1 -- 172.21.15.36:0/2642265455 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c4146c00 con 0x7fd8c4062270 2022-06-17T12:56:51.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.718+0000 7fd8ca21c700 1 -- 172.21.15.36:0/2642265455 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c4147710 con 0x7fd8c4062270 2022-06-17T12:56:51.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.718+0000 7fd8c0ff9700 1 -- 172.21.15.36:0/2642265455 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd8b4025030 con 0x7fd8c4062270 2022-06-17T12:56:51.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.718+0000 7fd8c0ff9700 1 --2- 172.21.15.36:0/2642265455 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd8a8053920 0x7fd8a8055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:51.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.718+0000 7fd8c0ff9700 1 -- 172.21.15.36:0/2642265455 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd8b406ce30 con 0x7fd8c4062270 2022-06-17T12:56:51.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.718+0000 7fd8c2ffd700 1 --2- 172.21.15.36:0/2642265455 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd8a8053920 0x7fd8a8055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:51.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.718+0000 7fd8c2ffd700 1 --2- 172.21.15.36:0/2642265455 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd8a8053920 0x7fd8a8055de0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fd8ac00d000 tx=0x7fd8ac000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:51.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.722+0000 7fd8c0ff9700 1 -- 172.21.15.36:0/2642265455 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd8b4043c70 con 0x7fd8c4062270 2022-06-17T12:56:51.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.890+0000 7fd8ca21c700 1 -- 172.21.15.36:0/2642265455 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fd8c40b63b0 con 0x7fd8c4062270 2022-06-17T12:56:51.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.906+0000 7fd8c0ff9700 1 -- 172.21.15.36:0/2642265455 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v46) v1 ==== 184+0+0 (secure 0 0 0) 0x7fd8b4023070 con 0x7fd8c4062270 2022-06-17T12:56:51.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.910+0000 7fd8ca21c700 1 -- 172.21.15.36:0/2642265455 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd8a8053920 msgr2=0x7fd8a8055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:51.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.910+0000 7fd8ca21c700 1 --2- 172.21.15.36:0/2642265455 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd8a8053920 0x7fd8a8055de0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fd8ac00d000 tx=0x7fd8ac000bc0).stop 2022-06-17T12:56:51.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.910+0000 7fd8ca21c700 1 -- 172.21.15.36:0/2642265455 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8c4062270 msgr2=0x7fd8c413e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:51.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.910+0000 7fd8ca21c700 1 --2- 172.21.15.36:0/2642265455 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8c4062270 0x7fd8c413e760 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 rx=0x7fd8b4002f70 tx=0x7fd8b4014000).stop 2022-06-17T12:56:51.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.910+0000 7fd8ca21c700 1 -- 172.21.15.36:0/2642265455 shutdown_connections 2022-06-17T12:56:51.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.910+0000 7fd8ca21c700 1 --2- 172.21.15.36:0/2642265455 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd8a8053920 0x7fd8a8055de0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:51.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.910+0000 7fd8ca21c700 1 --2- 172.21.15.36:0/2642265455 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8c40ad330 0x7fd8c413d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:51.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.910+0000 7fd8ca21c700 1 --2- 172.21.15.36:0/2642265455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8c4143e80 0x7fd8c413d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:51.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.910+0000 7fd8ca21c700 1 --2- 172.21.15.36:0/2642265455 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd8c4062270 0x7fd8c413e760 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:51.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.910+0000 7fd8ca21c700 1 -- 172.21.15.36:0/2642265455 >> 172.21.15.36:0/2642265455 conn(0x7fd8c40b9870 msgr2=0x7fd8c40b9060 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:51.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.910+0000 7fd8ca21c700 1 -- 172.21.15.36:0/2642265455 shutdown_connections 2022-06-17T12:56:51.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:51.910+0000 7fd8ca21c700 1 -- 172.21.15.36:0/2642265455 wait complete. 2022-06-17T12:56:51.932 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.2 2022-06-17T12:56:51.936 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:56:52.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.082+0000 7efbf5508700 1 Processor -- start 2022-06-17T12:56:52.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.082+0000 7efbf5508700 1 -- start start 2022-06-17T12:56:52.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.082+0000 7efbf5508700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7efbf00b2200 0x7efbf00af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:52.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.082+0000 7efbf5508700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efbf00b2cd0 0x7efbf0147600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:52.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.082+0000 7efbf5508700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbf00b25e0 con 0x7efbf00af8d0 2022-06-17T12:56:52.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.082+0000 7efbf5508700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbf00ad790 con 0x7efbf00b2200 2022-06-17T12:56:52.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.082+0000 7efbf5508700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbf00aef40 con 0x7efbf00b2cd0 2022-06-17T12:56:52.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.082+0000 7efbeffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7efbf00b2200 0x7efbf00af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:52.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.082+0000 7efbeffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7efbf00b2200 0x7efbf00af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40100/0 (socket says 172.21.15.36:40100) 2022-06-17T12:56:52.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.082+0000 7efbeffff700 1 -- 172.21.15.36:0/2954513148 learned_addr learned my addr 172.21.15.36:0/2954513148 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:52.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.082+0000 7efbf4d07700 1 --2- 172.21.15.36:0/2954513148 >> v2:172.21.15.70:3300/0 conn(0x7efbf00b2cd0 0x7efbf0147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:52.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.082+0000 7efbeffff700 1 -- 172.21.15.36:0/2954513148 >> v2:172.21.15.70:3300/0 conn(0x7efbf00b2cd0 msgr2=0x7efbf0147600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:52.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.082+0000 7efbeffff700 1 --2- 172.21.15.36:0/2954513148 >> v2:172.21.15.70:3300/0 conn(0x7efbf00b2cd0 0x7efbf0147600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:52.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.082+0000 7efbeffff700 1 -- 172.21.15.36:0/2954513148 >> v1:172.21.15.70:6789/0 conn(0x7efbf00af8d0 legacy=0x7efbf014a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:52.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.082+0000 7efbeffff700 1 -- 172.21.15.36:0/2954513148 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf00b1950 con 0x7efbf00b2200 2022-06-17T12:56:52.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbeffff700 1 --2- 172.21.15.36:0/2954513148 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efbf00b2200 0x7efbf00af3e0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7efbe4002f70 tx=0x7efbe4006eb0).ready entity=mon.0 client_cookie=798cfebac91d17fb server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:52.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbeeffd700 1 -- 172.21.15.36:0/2954513148 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efbe400d610 con 0x7efbf00b2200 2022-06-17T12:56:52.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbeeffd700 1 -- 172.21.15.36:0/2954513148 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe4000e90 con 0x7efbf00b2200 2022-06-17T12:56:52.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbeeffd700 1 -- 172.21.15.36:0/2954513148 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efbe400d9a0 con 0x7efbf00b2200 2022-06-17T12:56:52.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbf5508700 1 -- 172.21.15.36:0/2954513148 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efbf00b2200 msgr2=0x7efbf00af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:52.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbf5508700 1 --2- 172.21.15.36:0/2954513148 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efbf00b2200 0x7efbf00af3e0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7efbe4002f70 tx=0x7efbe4006eb0).stop 2022-06-17T12:56:52.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbf5508700 1 -- 172.21.15.36:0/2954513148 shutdown_connections 2022-06-17T12:56:52.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbf5508700 1 --2- 172.21.15.36:0/2954513148 >> v2:172.21.15.70:3300/0 conn(0x7efbf00b2cd0 0x7efbf0147600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:52.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbf5508700 1 --2- 172.21.15.36:0/2954513148 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efbf00b2200 0x7efbf00af3e0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:52.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbf5508700 1 -- 172.21.15.36:0/2954513148 >> 172.21.15.36:0/2954513148 conn(0x7efbf005f770 msgr2=0x7efbf005fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:52.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbf5508700 1 -- 172.21.15.36:0/2954513148 shutdown_connections 2022-06-17T12:56:52.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbf5508700 1 -- 172.21.15.36:0/2954513148 wait complete. 2022-06-17T12:56:52.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbf5508700 1 Processor -- start 2022-06-17T12:56:52.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbf5508700 1 -- start start 2022-06-17T12:56:52.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbf5508700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efbf00af8d0 0x7efbf014e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:52.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbf5508700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efbf00b2200 0x7efbf00a70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:52.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbf5508700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efbf00b2cd0 0x7efbf00aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:52.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbf5508700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf00a75d0 con 0x7efbf00b2200 2022-06-17T12:56:52.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbf5508700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf00aa680 con 0x7efbf00af8d0 2022-06-17T12:56:52.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbf5508700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf00aa920 con 0x7efbf00b2cd0 2022-06-17T12:56:52.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbef7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efbf00b2200 0x7efbf00a70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:52.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbeffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efbf00af8d0 0x7efbf014e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:52.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbef7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efbf00b2200 0x7efbf00a70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40106/0 (socket says 172.21.15.36:40106) 2022-06-17T12:56:52.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbef7fe700 1 -- 172.21.15.36:0/1171511141 learned_addr learned my addr 172.21.15.36:0/1171511141 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:52.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.086+0000 7efbf4d07700 1 --2- 172.21.15.36:0/1171511141 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efbf00b2cd0 0x7efbf00aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:52.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.090+0000 7efbef7fe700 1 -- 172.21.15.36:0/1171511141 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efbf00b2cd0 msgr2=0x7efbf00aadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:52.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.090+0000 7efbef7fe700 1 --2- 172.21.15.36:0/1171511141 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efbf00b2cd0 0x7efbf00aadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:52.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.090+0000 7efbef7fe700 1 -- 172.21.15.36:0/1171511141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efbf00af8d0 msgr2=0x7efbf014e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:52.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.090+0000 7efbef7fe700 1 --2- 172.21.15.36:0/1171511141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efbf00af8d0 0x7efbf014e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:52.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.090+0000 7efbef7fe700 1 -- 172.21.15.36:0/1171511141 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbe400b1e0 con 0x7efbf00b2200 2022-06-17T12:56:52.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.090+0000 7efbef7fe700 1 --2- 172.21.15.36:0/1171511141 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efbf00b2200 0x7efbf00a70e0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7efbf006ad20 tx=0x7efbe00014a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:52.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.090+0000 7efbed7fa700 1 -- 172.21.15.36:0/1171511141 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efbe000e4a0 con 0x7efbf00b2200 2022-06-17T12:56:52.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.090+0000 7efbed7fa700 1 -- 172.21.15.36:0/1171511141 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe000e600 con 0x7efbf00b2200 2022-06-17T12:56:52.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.090+0000 7efbed7fa700 1 -- 172.21.15.36:0/1171511141 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efbe000e960 con 0x7efbf00b2200 2022-06-17T12:56:52.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.090+0000 7efbf5508700 1 -- 172.21.15.36:0/1171511141 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf00ab2b0 con 0x7efbf00b2200 2022-06-17T12:56:52.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.090+0000 7efbf5508700 1 -- 172.21.15.36:0/1171511141 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf014cb50 con 0x7efbf00b2200 2022-06-17T12:56:52.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.090+0000 7efbf5508700 1 -- 172.21.15.36:0/1171511141 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbf00ad330 con 0x7efbf00b2200 2022-06-17T12:56:52.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.090+0000 7efbed7fa700 1 -- 172.21.15.36:0/1171511141 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7efbe0019030 con 0x7efbf00b2200 2022-06-17T12:56:52.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.090+0000 7efbed7fa700 1 --2- 172.21.15.36:0/1171511141 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efbd0053a40 0x7efbd0055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:52.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.090+0000 7efbeffff700 1 --2- 172.21.15.36:0/1171511141 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efbd0053a40 0x7efbd0055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:52.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.094+0000 7efbed7fa700 1 -- 172.21.15.36:0/1171511141 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6420+0+0 (secure 0 0 0) 0x7efbe006b790 con 0x7efbf00b2200 2022-06-17T12:56:52.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.094+0000 7efbeffff700 1 --2- 172.21.15.36:0/1171511141 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efbd0053a40 0x7efbd0055f00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7efbe401c000 tx=0x7efbe4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:52.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.098+0000 7efbed7fa700 1 -- 172.21.15.36:0/1171511141 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efbe0042420 con 0x7efbf00b2200 2022-06-17T12:56:52.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.266+0000 7efbf5508700 1 -- 172.21.15.36:0/1171511141 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efbf0057410 con 0x7efbf00b2200 2022-06-17T12:56:52.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.266+0000 7efbed7fa700 1 -- 172.21.15.36:0/1171511141 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+2142 (secure 0 0 0) 0x7efbe0045a40 con 0x7efbf00b2200 2022-06-17T12:56:52.256 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:52.257 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":78,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1081450840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1081450840},{"type":"v1","addr":"172.21.15.70:6833","nonce":1081450840}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":76}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:56:49.200774+0000","modified":"2022-06-17T12:56:50.204509+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8478":{"gid":8478,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/3169088536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3169088536},{"type":"v1","addr":"172.21.15.36:6835","nonce":3169088536}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-06-17T12:56:52.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.266+0000 7efbf5508700 1 -- 172.21.15.36:0/1171511141 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efbd0053a40 msgr2=0x7efbd0055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:52.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.266+0000 7efbf5508700 1 --2- 172.21.15.36:0/1171511141 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efbd0053a40 0x7efbd0055f00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7efbe401c000 tx=0x7efbe4015000).stop 2022-06-17T12:56:52.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.266+0000 7efbf5508700 1 -- 172.21.15.36:0/1171511141 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efbf00b2200 msgr2=0x7efbf00a70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:52.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.266+0000 7efbf5508700 1 --2- 172.21.15.36:0/1171511141 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efbf00b2200 0x7efbf00a70e0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7efbf006ad20 tx=0x7efbe00014a0).stop 2022-06-17T12:56:52.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.266+0000 7efbf5508700 1 -- 172.21.15.36:0/1171511141 shutdown_connections 2022-06-17T12:56:52.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.266+0000 7efbf5508700 1 --2- 172.21.15.36:0/1171511141 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efbd0053a40 0x7efbd0055f00 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:52.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.266+0000 7efbf5508700 1 --2- 172.21.15.36:0/1171511141 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efbf00b2cd0 0x7efbf00aadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:52.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.266+0000 7efbf5508700 1 --2- 172.21.15.36:0/1171511141 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efbf00b2200 0x7efbf00a70e0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:52.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.266+0000 7efbf5508700 1 --2- 172.21.15.36:0/1171511141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efbf00af8d0 0x7efbf014e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:52.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.266+0000 7efbf5508700 1 -- 172.21.15.36:0/1171511141 >> 172.21.15.36:0/1171511141 conn(0x7efbf005f770 msgr2=0x7efbf00b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:52.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.266+0000 7efbf5508700 1 -- 172.21.15.36:0/1171511141 shutdown_connections 2022-06-17T12:56:52.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.266+0000 7efbf5508700 1 -- 172.21.15.36:0/1171511141 wait complete. 2022-06-17T12:56:52.273 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 78 2022-06-17T12:56:52.492 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 78, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:56:49.200774+0000', 'modified': '2022-06-17T12:56:50.204509+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8478}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8478': {'gid': 8478, 'name': 'a', 'rank': 0, 'incarnation': 77, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.36:6835/3169088536', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 3169088536}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 3169088536}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:56:52.493 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:56:52.495 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:56:52.627 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":8478,"want_state":"up:active","state":"up:active","rank_uptime":3.4308915510000002,"mdsmap_epoch":78,"osdmap_epoch":141,"osdmap_epoch_barrier":140,"uptime":8.5041082980000002} 2022-06-17T12:56:52.628 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 8478, "mdsmap_epoch": 78, "osdmap_epoch": 141, "osdmap_epoch_barrier": 140, "rank_uptime": 3.430891551, "state": "up:active", "uptime": 8.504108298, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:56:52.634 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:56:52.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdafe2fa700 1 Processor -- start 2022-06-17T12:56:52.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdafe2fa700 1 -- start start 2022-06-17T12:56:52.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdafe2fa700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdaf8144c00 0x7fdaf8062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:52.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdafe2fa700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdaf80ad330 0x7fdaf80ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:52.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdafe2fa700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf80b0ba0 con 0x7fdaf8062770 2022-06-17T12:56:52.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdafe2fa700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf80ae820 con 0x7fdaf8144c00 2022-06-17T12:56:52.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdafe2fa700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf8061030 con 0x7fdaf80ad330 2022-06-17T12:56:52.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdafc897700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdaf80ad330 0x7fdaf80ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:52.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdaf77fe700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdaf8144c00 0x7fdaf8062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:52.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdafc897700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdaf80ad330 0x7fdaf80ad7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:54906/0 (socket says 172.21.15.36:54906) 2022-06-17T12:56:52.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdafc897700 1 -- 172.21.15.36:0/1897797423 learned_addr learned my addr 172.21.15.36:0/1897797423 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:52.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdaf77fe700 1 -- 172.21.15.36:0/1897797423 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdaf80ad330 msgr2=0x7fdaf80ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:52.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdaf77fe700 1 --2- 172.21.15.36:0/1897797423 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdaf80ad330 0x7fdaf80ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:52.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdaf77fe700 1 -- 172.21.15.36:0/1897797423 >> v1:172.21.15.70:6789/0 conn(0x7fdaf8062770 legacy=0x7fdaf81444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:52.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdaf77fe700 1 -- 172.21.15.36:0/1897797423 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf8147e90 con 0x7fdaf8144c00 2022-06-17T12:56:52.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdafc897700 1 --2- 172.21.15.36:0/1897797423 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdaf80ad330 0x7fdaf80ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:52.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdaf77fe700 1 --2- 172.21.15.36:0/1897797423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaf8144c00 0x7fdaf8062b50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7fdaf8061170 tx=0x7fdaec0091d0).ready entity=mon.1 client_cookie=6b42e34a3ba8134 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:52.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdaf6ffd700 1 -- 172.21.15.36:0/1897797423 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdaec00b680 con 0x7fdaf8144c00 2022-06-17T12:56:52.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdaf6ffd700 1 -- 172.21.15.36:0/1897797423 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaec00b7e0 con 0x7fdaf8144c00 2022-06-17T12:56:52.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdaf6ffd700 1 -- 172.21.15.36:0/1897797423 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdaec00b950 con 0x7fdaf8144c00 2022-06-17T12:56:52.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdafe2fa700 1 -- 172.21.15.36:0/1897797423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaf8144c00 msgr2=0x7fdaf8062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:52.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdafe2fa700 1 --2- 172.21.15.36:0/1897797423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaf8144c00 0x7fdaf8062b50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7fdaf8061170 tx=0x7fdaec0091d0).stop 2022-06-17T12:56:52.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdafe2fa700 1 -- 172.21.15.36:0/1897797423 shutdown_connections 2022-06-17T12:56:52.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdafe2fa700 1 --2- 172.21.15.36:0/1897797423 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdaf80ad330 0x7fdaf80ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:52.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdafe2fa700 1 --2- 172.21.15.36:0/1897797423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaf8144c00 0x7fdaf8062b50 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:52.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdafe2fa700 1 -- 172.21.15.36:0/1897797423 >> 172.21.15.36:0/1897797423 conn(0x7fdaf80b53e0 msgr2=0x7fdaf80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:52.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdafe2fa700 1 -- 172.21.15.36:0/1897797423 shutdown_connections 2022-06-17T12:56:52.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.758+0000 7fdafe2fa700 1 -- 172.21.15.36:0/1897797423 wait complete. 2022-06-17T12:56:52.837 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdafe2fa700 1 Processor -- start 2022-06-17T12:56:52.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdafe2fa700 1 -- start start 2022-06-17T12:56:52.838 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdafe2fa700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaf8062770 0x7fdaf81572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:52.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdafe2fa700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdaf80ad330 0x7fdaf8063230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:52.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdafe2fa700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdaf8144c00 0x7fdaf8063720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:52.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdafe2fa700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf8066c50 con 0x7fdaf80ad330 2022-06-17T12:56:52.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdafe2fa700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf8066ef0 con 0x7fdaf8062770 2022-06-17T12:56:52.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdafe2fa700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf80b4790 con 0x7fdaf8144c00 2022-06-17T12:56:52.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdaf77fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdaf80ad330 0x7fdaf8063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:52.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdaf77fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdaf80ad330 0x7fdaf8063230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40120/0 (socket says 172.21.15.36:40120) 2022-06-17T12:56:52.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdaf77fe700 1 -- 172.21.15.36:0/1622656085 learned_addr learned my addr 172.21.15.36:0/1622656085 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:52.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdafc897700 1 --2- 172.21.15.36:0/1622656085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdaf8144c00 0x7fdaf8063720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:52.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdaf7fff700 1 --2- 172.21.15.36:0/1622656085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaf8062770 0x7fdaf81572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:52.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdafc897700 1 -- 172.21.15.36:0/1622656085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaf8062770 msgr2=0x7fdaf81572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:52.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdafc897700 1 --2- 172.21.15.36:0/1622656085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaf8062770 0x7fdaf81572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:52.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdafc897700 1 -- 172.21.15.36:0/1622656085 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdaf80ad330 msgr2=0x7fdaf8063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:52.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdafc897700 1 --2- 172.21.15.36:0/1622656085 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdaf80ad330 0x7fdaf8063230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:52.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdafc897700 1 -- 172.21.15.36:0/1622656085 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf000b1e0 con 0x7fdaf8144c00 2022-06-17T12:56:52.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdaf77fe700 1 --2- 172.21.15.36:0/1622656085 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdaf80ad330 0x7fdaf8063230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:52.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdafc897700 1 --2- 172.21.15.36:0/1622656085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdaf8144c00 0x7fdaf8063720 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fdaf000c000 tx=0x7fdaf0006eb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:52.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdaf57fa700 1 -- 172.21.15.36:0/1622656085 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdaf0012600 con 0x7fdaf8144c00 2022-06-17T12:56:52.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdaf57fa700 1 -- 172.21.15.36:0/1622656085 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf0012760 con 0x7fdaf8144c00 2022-06-17T12:56:52.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdafe2fa700 1 -- 172.21.15.36:0/1622656085 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaec009470 con 0x7fdaf8144c00 2022-06-17T12:56:52.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdaf57fa700 1 -- 172.21.15.36:0/1622656085 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdaf0012ac0 con 0x7fdaf8144c00 2022-06-17T12:56:52.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.762+0000 7fdafe2fa700 1 -- 172.21.15.36:0/1622656085 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf8067390 con 0x7fdaf8144c00 2022-06-17T12:56:52.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.766+0000 7fdafe2fa700 1 -- 172.21.15.36:0/1622656085 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf81577a0 con 0x7fdaf8144c00 2022-06-17T12:56:52.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.766+0000 7fdaf57fa700 1 -- 172.21.15.36:0/1622656085 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fdaf0022070 con 0x7fdaf8144c00 2022-06-17T12:56:52.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.766+0000 7fdaf57fa700 1 --2- 172.21.15.36:0/1622656085 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdad8053a40 0x7fdad8055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:52.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.766+0000 7fdaf57fa700 1 -- 172.21.15.36:0/1622656085 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdaf006c520 con 0x7fdaf8144c00 2022-06-17T12:56:52.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.766+0000 7fdaf7fff700 1 --2- 172.21.15.36:0/1622656085 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdad8053a40 0x7fdad8055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:52.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.766+0000 7fdaf7fff700 1 --2- 172.21.15.36:0/1622656085 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdad8053a40 0x7fdad8055f00 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fdae800d000 tx=0x7fdae8007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:52.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.774+0000 7fdaf57fa700 1 -- 172.21.15.36:0/1622656085 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdaf0043190 con 0x7fdaf8144c00 2022-06-17T12:56:52.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.906+0000 7fdafe2fa700 1 -- 172.21.15.36:0/1622656085 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdaf80b1c20 con 0x7fdaf8144c00 2022-06-17T12:56:52.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.906+0000 7fdaf57fa700 1 -- 172.21.15.36:0/1622656085 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+16285 (secure 0 0 0) 0x7fdaf0016020 con 0x7fdaf8144c00 2022-06-17T12:56:52.909 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:52.924 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":141,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:56:50.204603+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:56:47.177576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-06-17T12:56:48.257688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:56:52.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.910+0000 7fdafe2fa700 1 -- 172.21.15.36:0/1622656085 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdad8053a40 msgr2=0x7fdad8055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:52.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.910+0000 7fdafe2fa700 1 --2- 172.21.15.36:0/1622656085 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdad8053a40 0x7fdad8055f00 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fdae800d000 tx=0x7fdae8007b90).stop 2022-06-17T12:56:52.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.910+0000 7fdafe2fa700 1 -- 172.21.15.36:0/1622656085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdaf8144c00 msgr2=0x7fdaf8063720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:52.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.910+0000 7fdafe2fa700 1 --2- 172.21.15.36:0/1622656085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdaf8144c00 0x7fdaf8063720 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fdaf000c000 tx=0x7fdaf0006eb0).stop 2022-06-17T12:56:52.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.910+0000 7fdafe2fa700 1 -- 172.21.15.36:0/1622656085 shutdown_connections 2022-06-17T12:56:52.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.910+0000 7fdafe2fa700 1 --2- 172.21.15.36:0/1622656085 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fdad8053a40 0x7fdad8055f00 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:52.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.910+0000 7fdafe2fa700 1 --2- 172.21.15.36:0/1622656085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdaf8144c00 0x7fdaf8063720 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:52.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.910+0000 7fdafe2fa700 1 --2- 172.21.15.36:0/1622656085 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fdaf80ad330 0x7fdaf8063230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:52.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.910+0000 7fdafe2fa700 1 --2- 172.21.15.36:0/1622656085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaf8062770 0x7fdaf81572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:52.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.910+0000 7fdafe2fa700 1 -- 172.21.15.36:0/1622656085 >> 172.21.15.36:0/1622656085 conn(0x7fdaf80b53e0 msgr2=0x7fdaf805f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:52.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.910+0000 7fdafe2fa700 1 -- 172.21.15.36:0/1622656085 shutdown_connections 2022-06-17T12:56:52.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:52.910+0000 7fdafe2fa700 1 -- 172.21.15.36:0/1622656085 wait complete. 2022-06-17T12:56:52.939 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:56:47.177576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:56:48.257688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:56:52.941 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:56:53.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.070+0000 7f488581e700 1 Processor -- start 2022-06-17T12:56:53.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.070+0000 7f488581e700 1 -- start start 2022-06-17T12:56:53.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.070+0000 7f488581e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f48800b27b0 0x7f488014b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:53.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.070+0000 7f488581e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4880060ca0 0x7f48800ac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:53.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.070+0000 7f488581e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4880147bd0 0x7f48801442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:53.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.070+0000 7f488581e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f488014b870 con 0x7f48800b27b0 2022-06-17T12:56:53.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.070+0000 7f488581e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48800ac500 con 0x7f4880060ca0 2022-06-17T12:56:53.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.070+0000 7f488581e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48800b2b90 con 0x7f4880147bd0 2022-06-17T12:56:53.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.070+0000 7f488481c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f48800b27b0 0x7f488014b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:53.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.070+0000 7f488481c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f48800b27b0 0x7f488014b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40126/0 (socket says 172.21.15.36:40126) 2022-06-17T12:56:53.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.070+0000 7f488481c700 1 -- 172.21.15.36:0/3093594412 learned_addr learned my addr 172.21.15.36:0/3093594412 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:53.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.070+0000 7f488501d700 1 --2- 172.21.15.36:0/3093594412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4880147bd0 0x7f48801442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:53.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.070+0000 7f487ffff700 1 --2- 172.21.15.36:0/3093594412 >> v2:172.21.15.70:3300/0 conn(0x7f4880060ca0 0x7f48800ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:53.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.070+0000 7f488481c700 1 -- 172.21.15.36:0/3093594412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4880147bd0 msgr2=0x7f48801442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:53.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.070+0000 7f488481c700 1 --2- 172.21.15.36:0/3093594412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4880147bd0 0x7f48801442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.070+0000 7f488481c700 1 -- 172.21.15.36:0/3093594412 >> v2:172.21.15.70:3300/0 conn(0x7f4880060ca0 msgr2=0x7f48800ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:53.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.070+0000 7f488481c700 1 --2- 172.21.15.36:0/3093594412 >> v2:172.21.15.70:3300/0 conn(0x7f4880060ca0 0x7f48800ac780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.070+0000 7f488481c700 1 -- 172.21.15.36:0/3093594412 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4880147490 con 0x7f48800b27b0 2022-06-17T12:56:53.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.070+0000 7f488501d700 1 --2- 172.21.15.36:0/3093594412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4880147bd0 0x7f48801442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:53.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.070+0000 7f487ffff700 1 --2- 172.21.15.36:0/3093594412 >> v2:172.21.15.70:3300/0 conn(0x7f4880060ca0 0x7f48800ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:53.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.070+0000 7f488481c700 1 --2- 172.21.15.36:0/3093594412 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f48800b27b0 0x7f488014b2b0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 rx=0x7f4874002f70 tx=0x7f4874006eb0).ready entity=mon.0 client_cookie=991f34d83b1278b1 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:53.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.074+0000 7f487f7fe700 1 -- 172.21.15.36:0/3093594412 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f487400d570 con 0x7f48800b27b0 2022-06-17T12:56:53.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.074+0000 7f487f7fe700 1 -- 172.21.15.36:0/3093594412 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4874000e90 con 0x7f48800b27b0 2022-06-17T12:56:53.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.074+0000 7f487f7fe700 1 -- 172.21.15.36:0/3093594412 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f487400d900 con 0x7f48800b27b0 2022-06-17T12:56:53.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.074+0000 7f488581e700 1 -- 172.21.15.36:0/3093594412 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f48800b27b0 msgr2=0x7f488014b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:53.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.074+0000 7f488581e700 1 --2- 172.21.15.36:0/3093594412 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f48800b27b0 0x7f488014b2b0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 rx=0x7f4874002f70 tx=0x7f4874006eb0).stop 2022-06-17T12:56:53.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.074+0000 7f488581e700 1 -- 172.21.15.36:0/3093594412 shutdown_connections 2022-06-17T12:56:53.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.074+0000 7f488581e700 1 --2- 172.21.15.36:0/3093594412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4880147bd0 0x7f48801442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.074+0000 7f488581e700 1 --2- 172.21.15.36:0/3093594412 >> v2:172.21.15.70:3300/0 conn(0x7f4880060ca0 0x7f48800ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.074+0000 7f488581e700 1 --2- 172.21.15.36:0/3093594412 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f48800b27b0 0x7f488014b2b0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.074+0000 7f488581e700 1 -- 172.21.15.36:0/3093594412 >> 172.21.15.36:0/3093594412 conn(0x7f48800b5d10 msgr2=0x7f48800b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:53.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.074+0000 7f488581e700 1 -- 172.21.15.36:0/3093594412 shutdown_connections 2022-06-17T12:56:53.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.074+0000 7f488581e700 1 -- 172.21.15.36:0/3093594412 wait complete. 2022-06-17T12:56:53.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.074+0000 7f488581e700 1 Processor -- start 2022-06-17T12:56:53.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.074+0000 7f488581e700 1 -- start start 2022-06-17T12:56:53.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f488581e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4880060ca0 0x7f4880149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:53.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f488581e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f48800b27b0 0x7f4880156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:53.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f488581e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4880147bd0 0x7f4880156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:53.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f488581e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48801563d0 con 0x7f48800b27b0 2022-06-17T12:56:53.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f488581e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4880064750 con 0x7f4880147bd0 2022-06-17T12:56:53.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f488581e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48801565c0 con 0x7f4880060ca0 2022-06-17T12:56:53.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f487ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f48800b27b0 0x7f4880156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:53.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f487ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f48800b27b0 0x7f4880156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40134/0 (socket says 172.21.15.36:40134) 2022-06-17T12:56:53.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f487ffff700 1 -- 172.21.15.36:0/3954676731 learned_addr learned my addr 172.21.15.36:0/3954676731 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:53.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f488481c700 1 --2- 172.21.15.36:0/3954676731 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4880060ca0 0x7f4880149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:53.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f488501d700 1 --2- 172.21.15.36:0/3954676731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4880147bd0 0x7f4880156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:53.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f487ffff700 1 -- 172.21.15.36:0/3954676731 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4880060ca0 msgr2=0x7f4880149fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:53.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f487ffff700 1 --2- 172.21.15.36:0/3954676731 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4880060ca0 0x7f4880149fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f487ffff700 1 -- 172.21.15.36:0/3954676731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4880147bd0 msgr2=0x7f4880156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:53.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f487ffff700 1 --2- 172.21.15.36:0/3954676731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4880147bd0 0x7f4880156f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f487ffff700 1 -- 172.21.15.36:0/3954676731 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4870009470 con 0x7f48800b27b0 2022-06-17T12:56:53.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f487ffff700 1 --2- 172.21.15.36:0/3954676731 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f48800b27b0 0x7f4880156a60 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7f487000c000 tx=0x7f48700091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:53.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f487dffb700 1 -- 172.21.15.36:0/3954676731 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f487000c850 con 0x7f48800b27b0 2022-06-17T12:56:53.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f487dffb700 1 -- 172.21.15.36:0/3954676731 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487000c9b0 con 0x7f48800b27b0 2022-06-17T12:56:53.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f487dffb700 1 -- 172.21.15.36:0/3954676731 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f487000cd10 con 0x7f48800b27b0 2022-06-17T12:56:53.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f488581e700 1 -- 172.21.15.36:0/3954676731 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f487800b180 con 0x7f48800b27b0 2022-06-17T12:56:53.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f488581e700 1 -- 172.21.15.36:0/3954676731 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f487400b1e0 con 0x7f48800b27b0 2022-06-17T12:56:53.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f487dffb700 1 -- 172.21.15.36:0/3954676731 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f4870022070 con 0x7f48800b27b0 2022-06-17T12:56:53.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f488581e700 1 -- 172.21.15.36:0/3954676731 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4880057410 con 0x7f48800b27b0 2022-06-17T12:56:53.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.078+0000 7f487dffb700 1 --2- 172.21.15.36:0/3954676731 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4860053a40 0x7f4860055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:53.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.082+0000 7f487dffb700 1 -- 172.21.15.36:0/3954676731 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f487006c8b0 con 0x7f48800b27b0 2022-06-17T12:56:53.105 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.082+0000 7f488481c700 1 --2- 172.21.15.36:0/3954676731 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4860053a40 0x7f4860055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:53.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.082+0000 7f488481c700 1 --2- 172.21.15.36:0/3954676731 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4860053a40 0x7f4860055f00 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f487401c000 tx=0x7f4874015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:53.106 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.086+0000 7f487dffb700 1 -- 172.21.15.36:0/3954676731 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f48700435a0 con 0x7f48800b27b0 2022-06-17T12:56:53.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.210+0000 7f488581e700 1 -- 172.21.15.36:0/3954676731 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f4880156780 con 0x7f48800b27b0 2022-06-17T12:56:53.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.214+0000 7f487dffb700 1 -- 172.21.15.36:0/3954676731 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v141) v1 ==== 110+0+10 (secure 0 0 0) 0x7f4870046bc0 con 0x7f48800b27b0 2022-06-17T12:56:53.198 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:56:53.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.214+0000 7f488581e700 1 -- 172.21.15.36:0/3954676731 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4860053a40 msgr2=0x7f4860055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:53.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.214+0000 7f488581e700 1 --2- 172.21.15.36:0/3954676731 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4860053a40 0x7f4860055f00 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f487401c000 tx=0x7f4874015000).stop 2022-06-17T12:56:53.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.214+0000 7f488581e700 1 -- 172.21.15.36:0/3954676731 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f48800b27b0 msgr2=0x7f4880156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:53.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.214+0000 7f488581e700 1 --2- 172.21.15.36:0/3954676731 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f48800b27b0 0x7f4880156a60 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7f487000c000 tx=0x7f48700091d0).stop 2022-06-17T12:56:53.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.214+0000 7f488581e700 1 -- 172.21.15.36:0/3954676731 shutdown_connections 2022-06-17T12:56:53.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.214+0000 7f488581e700 1 --2- 172.21.15.36:0/3954676731 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4860053a40 0x7f4860055f00 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.214+0000 7f488581e700 1 --2- 172.21.15.36:0/3954676731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4880147bd0 0x7f4880156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.214+0000 7f488581e700 1 --2- 172.21.15.36:0/3954676731 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f48800b27b0 0x7f4880156a60 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.214+0000 7f488581e700 1 --2- 172.21.15.36:0/3954676731 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4880060ca0 0x7f4880149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.214+0000 7f488581e700 1 -- 172.21.15.36:0/3954676731 >> 172.21.15.36:0/3954676731 conn(0x7f48800b5d10 msgr2=0x7f4880146710 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:53.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.214+0000 7f488581e700 1 -- 172.21.15.36:0/3954676731 shutdown_connections 2022-06-17T12:56:53.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.214+0000 7f488581e700 1 -- 172.21.15.36:0/3954676731 wait complete. 2022-06-17T12:56:53.217 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:56:53.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f328f7700 1 Processor -- start 2022-06-17T12:56:53.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f328f7700 1 -- start start 2022-06-17T12:56:53.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f328f7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f2c14b2b0 0x7f2f2c060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:53.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f328f7700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2f2c147bd0 0x7f2f2c1442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:53.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f328f7700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2c14b870 con 0x7f2f2c0b27b0 2022-06-17T12:56:53.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f328f7700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f2c0ac500 con 0x7f2f2c147bd0 2022-06-17T12:56:53.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f328f7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c0ace90 con 0x7f2f2c14b2b0 2022-06-17T12:56:53.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f318f5700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f2f2c0b27b0 0x7f2f2c0ac780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57670/0 (socket says 172.21.15.36:57670) 2022-06-17T12:56:53.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f318f5700 1 -- 172.21.15.36:0/3230953600 learned_addr learned my addr 172.21.15.36:0/3230953600 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:53.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f320f6700 1 --2- 172.21.15.36:0/3230953600 >> v2:172.21.15.36:3300/0 conn(0x7f2f2c147bd0 0x7f2f2c1442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:53.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f310f4700 1 --2- 172.21.15.36:0/3230953600 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f2c14b2b0 0x7f2f2c060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:53.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f308f3700 1 -- 172.21.15.36:0/3230953600 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4075749014 0 0) 0x7f2f2c14b870 con 0x7f2f2c0b27b0 2022-06-17T12:56:53.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f308f3700 1 -- 172.21.15.36:0/3230953600 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f10003260 con 0x7f2f2c0b27b0 2022-06-17T12:56:53.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f308f3700 1 -- 172.21.15.36:0/3230953600 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 398736111 0 0) 0x7f2f10003260 con 0x7f2f2c0b27b0 2022-06-17T12:56:53.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f308f3700 1 -- 172.21.15.36:0/3230953600 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f2c14b2b0 msgr2=0x7f2f2c060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:53.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f308f3700 1 --2- 172.21.15.36:0/3230953600 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f2c14b2b0 0x7f2f2c060ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f308f3700 1 -- 172.21.15.36:0/3230953600 >> v2:172.21.15.36:3300/0 conn(0x7f2f2c147bd0 msgr2=0x7f2f2c1442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:53.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f308f3700 1 --2- 172.21.15.36:0/3230953600 >> v2:172.21.15.36:3300/0 conn(0x7f2f2c147bd0 0x7f2f2c1442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f308f3700 1 -- 172.21.15.36:0/3230953600 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2c0b2b90 con 0x7f2f2c0b27b0 2022-06-17T12:56:53.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f308f3700 1 -- 172.21.15.36:0/3230953600 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f2f28001680 con 0x7f2f2c0b27b0 2022-06-17T12:56:53.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f308f3700 1 -- 172.21.15.36:0/3230953600 >> v1:172.21.15.36:6789/0 conn(0x7f2f2c0b27b0 legacy=0x7f2f2c0ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:53.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f308f3700 1 --2- 172.21.15.36:0/3230953600 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f10003750 0x7f2f10003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:53.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f308f3700 1 --2- 172.21.15.36:0/3230953600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f10004500 0x7f2f10004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:53.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f308f3700 1 --2- 172.21.15.36:0/3230953600 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f2c14b2b0 0x7f2f10004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:53.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f308f3700 1 -- 172.21.15.36:0/3230953600 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c0ac500 con 0x7f2f10003750 2022-06-17T12:56:53.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f308f3700 1 -- 172.21.15.36:0/3230953600 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c0ace90 con 0x7f2f10004500 2022-06-17T12:56:53.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f308f3700 1 -- 172.21.15.36:0/3230953600 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1000b750 con 0x7f2f2c14b2b0 2022-06-17T12:56:53.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f318f5700 1 --2- 172.21.15.36:0/3230953600 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f10003750 0x7f2f10003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:53.505 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f310f4700 1 --2- 172.21.15.36:0/3230953600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f10004500 0x7f2f10004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:53.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f320f6700 1 --2- 172.21.15.36:0/3230953600 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f2c14b2b0 0x7f2f10004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:53.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f318f5700 1 -- 172.21.15.36:0/3230953600 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f2c14b2b0 msgr2=0x7f2f10004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:53.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f318f5700 1 --2- 172.21.15.36:0/3230953600 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f2c14b2b0 0x7f2f10004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f318f5700 1 -- 172.21.15.36:0/3230953600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f10004500 msgr2=0x7f2f10004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:53.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f318f5700 1 --2- 172.21.15.36:0/3230953600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f10004500 0x7f2f10004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f318f5700 1 -- 172.21.15.36:0/3230953600 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f10003260 con 0x7f2f10003750 2022-06-17T12:56:53.508 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f320f6700 1 --2- 172.21.15.36:0/3230953600 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f2c14b2b0 0x7f2f10004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:56:53.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f318f5700 1 --2- 172.21.15.36:0/3230953600 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f10003750 0x7f2f10003b70 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 rx=0x7f2f28001390 tx=0x7f2f2c068680).ready entity=mon.0 client_cookie=abbd44237e96dbd server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:53.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f308f3700 1 -- 172.21.15.36:0/3230953600 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f28007170 con 0x7f2f10003750 2022-06-17T12:56:53.509 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f308f3700 1 -- 172.21.15.36:0/3230953600 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f280072d0 con 0x7f2f10003750 2022-06-17T12:56:53.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.374+0000 7f2f308f3700 1 -- 172.21.15.36:0/3230953600 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f2800c9c0 con 0x7f2f10003750 2022-06-17T12:56:53.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f328f7700 1 -- 172.21.15.36:0/3230953600 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f10003750 msgr2=0x7f2f10003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:53.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f328f7700 1 --2- 172.21.15.36:0/3230953600 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f10003750 0x7f2f10003b70 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 rx=0x7f2f28001390 tx=0x7f2f2c068680).stop 2022-06-17T12:56:53.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f320f6700 1 -- 172.21.15.36:0/3230953600 reap_dead start 2022-06-17T12:56:53.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f328f7700 1 -- 172.21.15.36:0/3230953600 shutdown_connections 2022-06-17T12:56:53.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f328f7700 1 -- 172.21.15.36:0/3230953600 >> 172.21.15.36:0/3230953600 conn(0x7f2f2c0b5d10 msgr2=0x7f2f2c0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:53.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f328f7700 1 -- 172.21.15.36:0/3230953600 shutdown_connections 2022-06-17T12:56:53.522 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f328f7700 1 -- 172.21.15.36:0/3230953600 wait complete. 2022-06-17T12:56:53.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f328f7700 1 Processor -- start 2022-06-17T12:56:53.523 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f328f7700 1 -- start start 2022-06-17T12:56:53.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f328f7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f2c0b05d0 0x7f2f2c0b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:53.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f328f7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f2c066970 0x7f2f2c156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:53.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f328f7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f2c15a200 0x7f2f2c15a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:53.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f328f7700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c065be0 con 0x7f2f2c0b05d0 2022-06-17T12:56:53.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f328f7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c159b30 con 0x7f2f2c066970 2022-06-17T12:56:53.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f328f7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c0aa9c0 con 0x7f2f2c15a200 2022-06-17T12:56:53.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f318f5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f2c0b05d0 0x7f2f2c0b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:53.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f318f5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f2c0b05d0 0x7f2f2c0b09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40152/0 (socket says 172.21.15.36:40152) 2022-06-17T12:56:53.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f318f5700 1 -- 172.21.15.36:0/1413874345 learned_addr learned my addr 172.21.15.36:0/1413874345 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:53.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f320f6700 1 --2- 172.21.15.36:0/1413874345 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f2c15a200 0x7f2f2c15a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:53.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f310f4700 1 --2- 172.21.15.36:0/1413874345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f2c066970 0x7f2f2c156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:53.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f318f5700 1 -- 172.21.15.36:0/1413874345 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f2c15a200 msgr2=0x7f2f2c15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:53.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f318f5700 1 --2- 172.21.15.36:0/1413874345 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f2c15a200 0x7f2f2c15a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f318f5700 1 -- 172.21.15.36:0/1413874345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f2c066970 msgr2=0x7f2f2c156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:53.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f318f5700 1 --2- 172.21.15.36:0/1413874345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f2c066970 0x7f2f2c156520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.378+0000 7f2f318f5700 1 -- 172.21.15.36:0/1413874345 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2c159d60 con 0x7f2f2c0b05d0 2022-06-17T12:56:53.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.382+0000 7f2f318f5700 1 --2- 172.21.15.36:0/1413874345 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f2c0b05d0 0x7f2f2c0b09b0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7f2f28001390 tx=0x7f2f28013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:53.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.382+0000 7f2f1effd700 1 -- 172.21.15.36:0/1413874345 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f28007440 con 0x7f2f2c0b05d0 2022-06-17T12:56:53.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.382+0000 7f2f1effd700 1 -- 172.21.15.36:0/1413874345 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f280047f0 con 0x7f2f2c0b05d0 2022-06-17T12:56:53.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.382+0000 7f2f328f7700 1 -- 172.21.15.36:0/1413874345 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2c066570 con 0x7f2f2c0b05d0 2022-06-17T12:56:53.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.382+0000 7f2f1effd700 1 -- 172.21.15.36:0/1413874345 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f28004b50 con 0x7f2f2c0b05d0 2022-06-17T12:56:53.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.382+0000 7f2f328f7700 1 -- 172.21.15.36:0/1413874345 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2c156bb0 con 0x7f2f2c0b05d0 2022-06-17T12:56:53.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.382+0000 7f2f1effd700 1 -- 172.21.15.36:0/1413874345 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2f28027070 con 0x7f2f2c0b05d0 2022-06-17T12:56:53.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.382+0000 7f2f328f7700 1 -- 172.21.15.36:0/1413874345 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f2c159ef0 con 0x7f2f2c0b05d0 2022-06-17T12:56:53.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.382+0000 7f2f1effd700 1 --2- 172.21.15.36:0/1413874345 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2f14053a40 0x7f2f14055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:53.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.382+0000 7f2f310f4700 1 --2- 172.21.15.36:0/1413874345 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2f14053a40 0x7f2f14055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:53.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.382+0000 7f2f1effd700 1 -- 172.21.15.36:0/1413874345 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2f2806c660 con 0x7f2f2c0b05d0 2022-06-17T12:56:53.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.382+0000 7f2f310f4700 1 --2- 172.21.15.36:0/1413874345 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2f14053a40 0x7f2f14055f00 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f2f2400b000 tx=0x7f2f24008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:53.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.390+0000 7f2f1effd700 1 -- 172.21.15.36:0/1413874345 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2f280433e0 con 0x7f2f2c0b05d0 2022-06-17T12:56:53.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.514+0000 7f2f328f7700 1 -- 172.21.15.36:0/1413874345 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2f2c057410 con 0x7f2f2c0b05d0 2022-06-17T12:56:53.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.518+0000 7f2f1effd700 1 -- 172.21.15.36:0/1413874345 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v141) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2f28046a00 con 0x7f2f2c0b05d0 2022-06-17T12:56:53.558 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:56:53.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.518+0000 7f2f328f7700 1 -- 172.21.15.36:0/1413874345 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2f14053a40 msgr2=0x7f2f14055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:53.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.518+0000 7f2f328f7700 1 --2- 172.21.15.36:0/1413874345 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2f14053a40 0x7f2f14055f00 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f2f2400b000 tx=0x7f2f24008040).stop 2022-06-17T12:56:53.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.518+0000 7f2f328f7700 1 -- 172.21.15.36:0/1413874345 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f2c0b05d0 msgr2=0x7f2f2c0b09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:53.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.518+0000 7f2f328f7700 1 --2- 172.21.15.36:0/1413874345 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f2c0b05d0 0x7f2f2c0b09b0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7f2f28001390 tx=0x7f2f28013000).stop 2022-06-17T12:56:53.571 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.518+0000 7f2f328f7700 1 -- 172.21.15.36:0/1413874345 shutdown_connections 2022-06-17T12:56:53.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.518+0000 7f2f328f7700 1 --2- 172.21.15.36:0/1413874345 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2f14053a40 0x7f2f14055f00 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.518+0000 7f2f328f7700 1 --2- 172.21.15.36:0/1413874345 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f2c15a200 0x7f2f2c15a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.518+0000 7f2f328f7700 1 --2- 172.21.15.36:0/1413874345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f2c066970 0x7f2f2c156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.518+0000 7f2f328f7700 1 --2- 172.21.15.36:0/1413874345 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f2c0b05d0 0x7f2f2c0b09b0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.518+0000 7f2f328f7700 1 -- 172.21.15.36:0/1413874345 >> 172.21.15.36:0/1413874345 conn(0x7f2f2c0b5d10 msgr2=0x7f2f2c0b3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:53.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.518+0000 7f2f328f7700 1 -- 172.21.15.36:0/1413874345 shutdown_connections 2022-06-17T12:56:53.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.518+0000 7f2f328f7700 1 -- 172.21.15.36:0/1413874345 wait complete. 2022-06-17T12:56:53.575 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:56:53.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.738+0000 7fceaf01c700 1 Processor -- start 2022-06-17T12:56:53.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.738+0000 7fceaf01c700 1 -- start start 2022-06-17T12:56:53.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.738+0000 7fceaf01c700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcea8060020 0x7fcea80626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:53.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.738+0000 7fceaf01c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea813ca70 0x7fcea813cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:53.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.738+0000 7fceaf01c700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea81432c0 con 0x7fcea80b3f80 2022-06-17T12:56:53.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.738+0000 7fceaf01c700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea8143df0 con 0x7fcea8060020 2022-06-17T12:56:53.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.738+0000 7fceaf01c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea8061f60 con 0x7fcea813ca70 2022-06-17T12:56:53.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.738+0000 7fceae81b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea813ca70 0x7fcea813cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:53.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.738+0000 7fceae81b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea813ca70 0x7fcea813cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:54952/0 (socket says 172.21.15.36:54952) 2022-06-17T12:56:53.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.738+0000 7fceae01a700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcea8060020 0x7fcea80626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:53.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.738+0000 7fceae81b700 1 -- 172.21.15.36:0/3400298949 learned_addr learned my addr 172.21.15.36:0/3400298949 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:53.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.738+0000 7fceae81b700 1 -- 172.21.15.36:0/3400298949 >> v2:172.21.15.70:3300/0 conn(0x7fcea8060020 msgr2=0x7fcea80626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:53.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.738+0000 7fceae81b700 1 --2- 172.21.15.36:0/3400298949 >> v2:172.21.15.70:3300/0 conn(0x7fcea8060020 0x7fcea80626a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.738+0000 7fceae81b700 1 -- 172.21.15.36:0/3400298949 >> v1:172.21.15.70:6789/0 conn(0x7fcea80b3f80 legacy=0x7fcea80b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:53.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.738+0000 7fcead018700 1 -- 172.21.15.36:0/3400298949 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3011474666 0 0) 0x7fcea81432c0 con 0x7fcea80b3f80 2022-06-17T12:56:53.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.738+0000 7fceae81b700 1 -- 172.21.15.36:0/3400298949 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea8062200 con 0x7fcea813ca70 2022-06-17T12:56:53.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.738+0000 7fceae81b700 1 --2- 172.21.15.36:0/3400298949 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea813ca70 0x7fcea813cf00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fcea00032f0 tx=0x7fcea0009290).ready entity=mon.2 client_cookie=e6447af89abc73aa server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:53.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.738+0000 7fcead018700 1 -- 172.21.15.36:0/3400298949 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcea000b8a0 con 0x7fcea813ca70 2022-06-17T12:56:53.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.738+0000 7fcead018700 1 -- 172.21.15.36:0/3400298949 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea000ba00 con 0x7fcea813ca70 2022-06-17T12:56:53.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.738+0000 7fcead018700 1 -- 172.21.15.36:0/3400298949 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcea000bb70 con 0x7fcea813ca70 2022-06-17T12:56:53.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.738+0000 7fceaf01c700 1 -- 172.21.15.36:0/3400298949 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea813ca70 msgr2=0x7fcea813cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:53.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.738+0000 7fceaf01c700 1 --2- 172.21.15.36:0/3400298949 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea813ca70 0x7fcea813cf00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fcea00032f0 tx=0x7fcea0009290).stop 2022-06-17T12:56:53.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.742+0000 7fceaf01c700 1 -- 172.21.15.36:0/3400298949 shutdown_connections 2022-06-17T12:56:53.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.742+0000 7fceaf01c700 1 --2- 172.21.15.36:0/3400298949 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea813ca70 0x7fcea813cf00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.742+0000 7fceaf01c700 1 --2- 172.21.15.36:0/3400298949 >> v2:172.21.15.70:3300/0 conn(0x7fcea8060020 0x7fcea80626a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.742+0000 7fceaf01c700 1 -- 172.21.15.36:0/3400298949 >> 172.21.15.36:0/3400298949 conn(0x7fcea805c550 msgr2=0x7fcea805f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:53.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.742+0000 7fceaf01c700 1 -- 172.21.15.36:0/3400298949 shutdown_connections 2022-06-17T12:56:53.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.742+0000 7fceaf01c700 1 -- 172.21.15.36:0/3400298949 wait complete. 2022-06-17T12:56:53.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.742+0000 7fceaf01c700 1 Processor -- start 2022-06-17T12:56:53.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.742+0000 7fceaf01c700 1 -- start start 2022-06-17T12:56:53.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.746+0000 7fceaf01c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcea8060020 0x7fcea8149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:53.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.746+0000 7fceaf01c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea80b3f80 0x7fcea8148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:53.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.746+0000 7fceaf01c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea813ca70 0x7fcea813ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:53.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.746+0000 7fceaf01c700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea8147000 con 0x7fcea8060020 2022-06-17T12:56:53.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.746+0000 7fceaf01c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea8149450 con 0x7fcea813ca70 2022-06-17T12:56:53.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.746+0000 7fceaf01c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea8147230 con 0x7fcea80b3f80 2022-06-17T12:56:53.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.746+0000 7fceae01a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcea8060020 0x7fcea8149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:53.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.746+0000 7fceae81b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea813ca70 0x7fcea813ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:53.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.746+0000 7fceae01a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcea8060020 0x7fcea8149f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40166/0 (socket says 172.21.15.36:40166) 2022-06-17T12:56:53.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.746+0000 7fceae01a700 1 -- 172.21.15.36:0/3975666528 learned_addr learned my addr 172.21.15.36:0/3975666528 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:53.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.746+0000 7fcead819700 1 --2- 172.21.15.36:0/3975666528 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea80b3f80 0x7fcea8148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:53.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.746+0000 7fceae81b700 1 -- 172.21.15.36:0/3975666528 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea80b3f80 msgr2=0x7fcea8148f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:53.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.746+0000 7fceae81b700 1 --2- 172.21.15.36:0/3975666528 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea80b3f80 0x7fcea8148f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.746+0000 7fceae81b700 1 -- 172.21.15.36:0/3975666528 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcea8060020 msgr2=0x7fcea8149f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:53.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.746+0000 7fceae81b700 1 --2- 172.21.15.36:0/3975666528 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcea8060020 0x7fcea8149f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.746+0000 7fceae81b700 1 -- 172.21.15.36:0/3975666528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea0009530 con 0x7fcea813ca70 2022-06-17T12:56:53.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.746+0000 7fceae01a700 1 --2- 172.21.15.36:0/3975666528 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcea8060020 0x7fcea8149f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:53.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.746+0000 7fceae81b700 1 --2- 172.21.15.36:0/3975666528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea813ca70 0x7fcea813ec10 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7fcea00032f0 tx=0x7fcea0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:53.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.746+0000 7fce977fe700 1 -- 172.21.15.36:0/3975666528 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcea000bb70 con 0x7fcea813ca70 2022-06-17T12:56:53.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.746+0000 7fce977fe700 1 -- 172.21.15.36:0/3975666528 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea0021630 con 0x7fcea813ca70 2022-06-17T12:56:53.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.746+0000 7fceaf01c700 1 -- 172.21.15.36:0/3975666528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea8145460 con 0x7fcea813ca70 2022-06-17T12:56:53.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.746+0000 7fce977fe700 1 -- 172.21.15.36:0/3975666528 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcea000b8a0 con 0x7fcea813ca70 2022-06-17T12:56:53.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.746+0000 7fceaf01c700 1 -- 172.21.15.36:0/3975666528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea8140300 con 0x7fcea813ca70 2022-06-17T12:56:53.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.746+0000 7fceaf01c700 1 -- 172.21.15.36:0/3975666528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea8145790 con 0x7fcea813ca70 2022-06-17T12:56:53.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.750+0000 7fce977fe700 1 -- 172.21.15.36:0/3975666528 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fcea0027020 con 0x7fcea813ca70 2022-06-17T12:56:53.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.750+0000 7fce977fe700 1 --2- 172.21.15.36:0/3975666528 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce880539b0 0x7fce88055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:53.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.750+0000 7fce977fe700 1 -- 172.21.15.36:0/3975666528 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcea006d160 con 0x7fcea813ca70 2022-06-17T12:56:53.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.750+0000 7fceae01a700 1 --2- 172.21.15.36:0/3975666528 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce880539b0 0x7fce88055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:53.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.750+0000 7fceae01a700 1 --2- 172.21.15.36:0/3975666528 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce880539b0 0x7fce88055e70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7fce9800b000 tx=0x7fce98006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:53.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.758+0000 7fce977fe700 1 -- 172.21.15.36:0/3975666528 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcea0044010 con 0x7fcea813ca70 2022-06-17T12:56:53.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.886+0000 7fceaf01c700 1 -- 172.21.15.36:0/3975666528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcea80b4db0 con 0x7fcea813ca70 2022-06-17T12:56:53.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.890+0000 7fce977fe700 1 -- 172.21.15.36:0/3975666528 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v141) v1 ==== 104+0+10 (secure 0 0 0) 0x7fcea00150a0 con 0x7fcea813ca70 2022-06-17T12:56:53.891 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:56:53.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.890+0000 7fceaf01c700 1 -- 172.21.15.36:0/3975666528 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce880539b0 msgr2=0x7fce88055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:53.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.890+0000 7fceaf01c700 1 --2- 172.21.15.36:0/3975666528 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce880539b0 0x7fce88055e70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7fce9800b000 tx=0x7fce98006df0).stop 2022-06-17T12:56:53.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.890+0000 7fceaf01c700 1 -- 172.21.15.36:0/3975666528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea813ca70 msgr2=0x7fcea813ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:53.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.890+0000 7fceaf01c700 1 --2- 172.21.15.36:0/3975666528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea813ca70 0x7fcea813ec10 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7fcea00032f0 tx=0x7fcea0016000).stop 2022-06-17T12:56:53.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.890+0000 7fceaf01c700 1 -- 172.21.15.36:0/3975666528 shutdown_connections 2022-06-17T12:56:53.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.890+0000 7fceaf01c700 1 --2- 172.21.15.36:0/3975666528 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce880539b0 0x7fce88055e70 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.890+0000 7fceaf01c700 1 --2- 172.21.15.36:0/3975666528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea813ca70 0x7fcea813ec10 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.890+0000 7fceaf01c700 1 --2- 172.21.15.36:0/3975666528 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea80b3f80 0x7fcea8148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.890+0000 7fceaf01c700 1 --2- 172.21.15.36:0/3975666528 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcea8060020 0x7fcea8149f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:53.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.890+0000 7fceaf01c700 1 -- 172.21.15.36:0/3975666528 >> 172.21.15.36:0/3975666528 conn(0x7fcea805c550 msgr2=0x7fcea80b5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:53.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.890+0000 7fceaf01c700 1 -- 172.21.15.36:0/3975666528 shutdown_connections 2022-06-17T12:56:53.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:53.890+0000 7fceaf01c700 1 -- 172.21.15.36:0/3975666528 wait complete. 2022-06-17T12:56:53.942 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:56:54.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.074+0000 7facca770700 1 Processor -- start 2022-06-17T12:56:54.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.074+0000 7facca770700 1 -- start start 2022-06-17T12:56:54.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.074+0000 7facca770700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7facc40b3f80 0x7facc40b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:54.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facca770700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc41432c0 con 0x7facc40b4b90 2022-06-17T12:56:54.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facca770700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc4143df0 con 0x7facc4060020 2022-06-17T12:56:54.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facca770700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc4062db0 con 0x7facc40b3f80 2022-06-17T12:56:54.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facc9f6f700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7facc40b4b90 0x7facc413ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57708/0 (socket says 172.21.15.36:57708) 2022-06-17T12:56:54.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facc9f6f700 1 -- 172.21.15.36:0/790625750 learned_addr learned my addr 172.21.15.36:0/790625750 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:54.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facc8f6d700 1 --2- 172.21.15.36:0/790625750 >> v2:172.21.15.70:3300/0 conn(0x7facc40b3f80 0x7facc40b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:54.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facaffff700 1 -- 172.21.15.36:0/790625750 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 139874608 0 0) 0x7facc41432c0 con 0x7facc40b4b90 2022-06-17T12:56:54.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facaffff700 1 -- 172.21.15.36:0/790625750 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faca8003260 con 0x7facc40b4b90 2022-06-17T12:56:54.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facaffff700 1 -- 172.21.15.36:0/790625750 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 462883624 0 0) 0x7facc4143df0 con 0x7facc4060020 2022-06-17T12:56:54.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facaffff700 1 -- 172.21.15.36:0/790625750 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facc41432c0 con 0x7facc4060020 2022-06-17T12:56:54.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facaffff700 1 -- 172.21.15.36:0/790625750 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2432964595 0 0) 0x7faca8003260 con 0x7facc40b4b90 2022-06-17T12:56:54.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facaffff700 1 -- 172.21.15.36:0/790625750 >> v2:172.21.15.70:3300/0 conn(0x7facc40b3f80 msgr2=0x7facc40b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:54.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facaffff700 1 --2- 172.21.15.36:0/790625750 >> v2:172.21.15.70:3300/0 conn(0x7facc40b3f80 0x7facc40b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:54.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facaffff700 1 -- 172.21.15.36:0/790625750 >> v1:172.21.15.70:6789/0 conn(0x7facc4060020 legacy=0x7facc40626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:54.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facaffff700 1 -- 172.21.15.36:0/790625750 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc4061f60 con 0x7facc40b4b90 2022-06-17T12:56:54.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facaffff700 1 -- 172.21.15.36:0/790625750 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7facbc0019e0 con 0x7facc40b4b90 2022-06-17T12:56:54.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facaffff700 1 -- 172.21.15.36:0/790625750 >> v1:172.21.15.36:6789/0 conn(0x7facc40b4b90 legacy=0x7facc413ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:54.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facaffff700 1 --2- 172.21.15.36:0/790625750 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faca8003930 0x7faca8003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:54.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facaffff700 1 --2- 172.21.15.36:0/790625750 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faca8004500 0x7faca8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:54.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facaffff700 1 --2- 172.21.15.36:0/790625750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faca8004e70 0x7faca800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:54.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facaffff700 1 -- 172.21.15.36:0/790625750 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc4062db0 con 0x7faca8004500 2022-06-17T12:56:54.117 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facaffff700 1 -- 172.21.15.36:0/790625750 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca800bd20 con 0x7faca8004e70 2022-06-17T12:56:54.123 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facaffff700 1 -- 172.21.15.36:0/790625750 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca800bee0 con 0x7faca8003930 2022-06-17T12:56:54.125 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facc8f6d700 1 --2- 172.21.15.36:0/790625750 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faca8004500 0x7faca8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:54.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facc9f6f700 1 --2- 172.21.15.36:0/790625750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faca8004e70 0x7faca800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:54.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facc976e700 1 --2- 172.21.15.36:0/790625750 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faca8003930 0x7faca8003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:54.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facc9f6f700 1 -- 172.21.15.36:0/790625750 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faca8003930 msgr2=0x7faca8003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:54.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facc9f6f700 1 --2- 172.21.15.36:0/790625750 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faca8003930 0x7faca8003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:54.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facc9f6f700 1 -- 172.21.15.36:0/790625750 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faca8004500 msgr2=0x7faca8004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:54.129 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facc9f6f700 1 --2- 172.21.15.36:0/790625750 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faca8004500 0x7faca8004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:54.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facc9f6f700 1 -- 172.21.15.36:0/790625750 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca8003260 con 0x7faca8004e70 2022-06-17T12:56:54.131 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facc8f6d700 1 --2- 172.21.15.36:0/790625750 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7faca8004500 0x7faca8004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:54.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facc9f6f700 1 -- 172.21.15.36:0/790625750 reap_dead start 2022-06-17T12:56:54.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facc9f6f700 1 --2- 172.21.15.36:0/790625750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faca8004e70 0x7faca800b750 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7facbc0017b0 tx=0x7facc40620f0).ready entity=mon.1 client_cookie=c36a04e9bd2c40ee server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:54.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facaffff700 1 -- 172.21.15.36:0/790625750 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facbc00cb90 con 0x7faca8004e70 2022-06-17T12:56:54.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facaffff700 1 -- 172.21.15.36:0/790625750 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facbc00ccf0 con 0x7faca8004e70 2022-06-17T12:56:54.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facaffff700 1 -- 172.21.15.36:0/790625750 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facbc005b80 con 0x7faca8004e70 2022-06-17T12:56:54.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facca770700 1 -- 172.21.15.36:0/790625750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faca8004e70 msgr2=0x7faca800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:54.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facca770700 1 --2- 172.21.15.36:0/790625750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faca8004e70 0x7faca800b750 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7facbc0017b0 tx=0x7facc40620f0).stop 2022-06-17T12:56:54.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facca770700 1 -- 172.21.15.36:0/790625750 shutdown_connections 2022-06-17T12:56:54.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facca770700 1 --2- 172.21.15.36:0/790625750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faca8004e70 0x7faca800b750 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:54.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facca770700 1 -- 172.21.15.36:0/790625750 >> 172.21.15.36:0/790625750 conn(0x7facc405c550 msgr2=0x7facc405f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:54.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facca770700 1 -- 172.21.15.36:0/790625750 shutdown_connections 2022-06-17T12:56:54.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.078+0000 7facca770700 1 -- 172.21.15.36:0/790625750 wait complete. 2022-06-17T12:56:54.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facca770700 1 Processor -- start 2022-06-17T12:56:54.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facca770700 1 -- start start 2022-06-17T12:56:54.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facca770700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faca8004e70 0x7facc40b9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:54.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facca770700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7facc40ad540 0x7facc413eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:54.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facca770700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7facc413f190 0x7facc40adac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:54.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facca770700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc4146910 con 0x7facc413f190 2022-06-17T12:56:54.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facca770700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc4146670 con 0x7faca8004e70 2022-06-17T12:56:54.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facca770700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc4147050 con 0x7facc40ad540 2022-06-17T12:56:54.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facc9f6f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7facc413f190 0x7facc40adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:54.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facc9f6f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7facc413f190 0x7facc40adac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40190/0 (socket says 172.21.15.36:40190) 2022-06-17T12:56:54.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facc9f6f700 1 -- 172.21.15.36:0/2732395945 learned_addr learned my addr 172.21.15.36:0/2732395945 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:54.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facc8f6d700 1 --2- 172.21.15.36:0/2732395945 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7facc40ad540 0x7facc413eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:54.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facc976e700 1 --2- 172.21.15.36:0/2732395945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faca8004e70 0x7facc40b9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:54.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facc9f6f700 1 -- 172.21.15.36:0/2732395945 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7facc40ad540 msgr2=0x7facc413eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:54.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facc9f6f700 1 --2- 172.21.15.36:0/2732395945 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7facc40ad540 0x7facc413eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:54.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facc9f6f700 1 -- 172.21.15.36:0/2732395945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faca8004e70 msgr2=0x7facc40b9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:54.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facc9f6f700 1 --2- 172.21.15.36:0/2732395945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faca8004e70 0x7facc40b9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:54.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facc9f6f700 1 -- 172.21.15.36:0/2732395945 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc41462d0 con 0x7facc413f190 2022-06-17T12:56:54.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facc8f6d700 1 --2- 172.21.15.36:0/2732395945 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7facc40ad540 0x7facc413eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:54.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facc9f6f700 1 --2- 172.21.15.36:0/2732395945 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7facc413f190 0x7facc40adac0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7facbc0017b0 tx=0x7facbc013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:54.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facb6ffd700 1 -- 172.21.15.36:0/2732395945 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facbc00ce60 con 0x7facc413f190 2022-06-17T12:56:54.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facb6ffd700 1 -- 172.21.15.36:0/2732395945 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facbc0070d0 con 0x7facc413f190 2022-06-17T12:56:54.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facb6ffd700 1 -- 172.21.15.36:0/2732395945 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facbc007430 con 0x7facc413f190 2022-06-17T12:56:54.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facca770700 1 -- 172.21.15.36:0/2732395945 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc4155670 con 0x7facc413f190 2022-06-17T12:56:54.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.082+0000 7facca770700 1 -- 172.21.15.36:0/2732395945 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc40b0710 con 0x7facc413f190 2022-06-17T12:56:54.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.086+0000 7facca770700 1 -- 172.21.15.36:0/2732395945 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc4147210 con 0x7facc413f190 2022-06-17T12:56:54.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.086+0000 7facb6ffd700 1 -- 172.21.15.36:0/2732395945 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7facbc027070 con 0x7facc413f190 2022-06-17T12:56:54.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.086+0000 7facb6ffd700 1 --2- 172.21.15.36:0/2732395945 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faca0053ad0 0x7faca0055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:54.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.086+0000 7facb6ffd700 1 -- 172.21.15.36:0/2732395945 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6420+0+0 (secure 0 0 0) 0x7facbc06c6b0 con 0x7facc413f190 2022-06-17T12:56:54.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.086+0000 7facc976e700 1 --2- 172.21.15.36:0/2732395945 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faca0053ad0 0x7faca0055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:54.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.086+0000 7facc976e700 1 --2- 172.21.15.36:0/2732395945 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faca0053ad0 0x7faca0055f90 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7facc0003000 tx=0x7facc0007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:54.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.090+0000 7facb6ffd700 1 -- 172.21.15.36:0/2732395945 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7facbc0434b0 con 0x7facc413f190 2022-06-17T12:56:54.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.218+0000 7facca770700 1 -- 172.21.15.36:0/2732395945 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7facc4057410 con 0x7facc413f190 2022-06-17T12:56:54.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.218+0000 7facb6ffd700 1 -- 172.21.15.36:0/2732395945 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v141) v1 ==== 100+0+10 (secure 0 0 0) 0x7facbc046ad0 con 0x7facc413f190 2022-06-17T12:56:54.209 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:56:54.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.222+0000 7facca770700 1 -- 172.21.15.36:0/2732395945 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faca0053ad0 msgr2=0x7faca0055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:54.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.222+0000 7facca770700 1 --2- 172.21.15.36:0/2732395945 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faca0053ad0 0x7faca0055f90 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7facc0003000 tx=0x7facc0007a00).stop 2022-06-17T12:56:54.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.222+0000 7facca770700 1 -- 172.21.15.36:0/2732395945 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7facc413f190 msgr2=0x7facc40adac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:54.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.222+0000 7facca770700 1 --2- 172.21.15.36:0/2732395945 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7facc413f190 0x7facc40adac0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7facbc0017b0 tx=0x7facbc013000).stop 2022-06-17T12:56:54.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.222+0000 7facca770700 1 -- 172.21.15.36:0/2732395945 shutdown_connections 2022-06-17T12:56:54.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.222+0000 7facca770700 1 --2- 172.21.15.36:0/2732395945 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7faca0053ad0 0x7faca0055f90 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:54.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.222+0000 7facca770700 1 --2- 172.21.15.36:0/2732395945 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7facc413f190 0x7facc40adac0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:54.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.222+0000 7facca770700 1 --2- 172.21.15.36:0/2732395945 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7facc40ad540 0x7facc413eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:54.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.222+0000 7facca770700 1 --2- 172.21.15.36:0/2732395945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faca8004e70 0x7facc40b9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:54.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.222+0000 7facca770700 1 -- 172.21.15.36:0/2732395945 >> 172.21.15.36:0/2732395945 conn(0x7facc405c550 msgr2=0x7facc41420a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:54.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.222+0000 7facca770700 1 -- 172.21.15.36:0/2732395945 shutdown_connections 2022-06-17T12:56:54.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.222+0000 7facca770700 1 -- 172.21.15.36:0/2732395945 wait complete. 2022-06-17T12:56:54.226 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:56:54.226 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:56:54.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.346+0000 7f6e19db7700 1 Processor -- start 2022-06-17T12:56:54.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.346+0000 7f6e19db7700 1 -- start start 2022-06-17T12:56:54.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e19db7700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6e14062770 0x7f6e14062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:54.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e19db7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e140ad330 0x7f6e140ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:54.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e19db7700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e140b0ba0 con 0x7f6e141444f0 2022-06-17T12:56:54.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e19db7700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e140ae820 con 0x7f6e14062770 2022-06-17T12:56:54.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e19db7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e14061030 con 0x7f6e140ad330 2022-06-17T12:56:54.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e13fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e140ad330 0x7f6e140ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:54.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e137fe700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6e14062770 0x7f6e14062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:54.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e13fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e140ad330 0x7f6e140ad7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:54986/0 (socket says 172.21.15.36:54986) 2022-06-17T12:56:54.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e0affd700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6e141444f0 0x7f6e14144960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:53460/0 (socket says 172.21.15.36:53460) 2022-06-17T12:56:54.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e13fff700 1 -- 172.21.15.36:0/3962480242 learned_addr learned my addr 172.21.15.36:0/3962480242 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:54.373 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e12ffd700 1 -- 172.21.15.36:0/3962480242 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1590930666 0 0) 0x7f6e140b0ba0 con 0x7f6e141444f0 2022-06-17T12:56:54.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e13fff700 1 -- 172.21.15.36:0/3962480242 >> v2:172.21.15.70:3300/0 conn(0x7f6e14062770 msgr2=0x7f6e14062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:54.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e13fff700 1 --2- 172.21.15.36:0/3962480242 >> v2:172.21.15.70:3300/0 conn(0x7f6e14062770 0x7f6e14062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:54.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e13fff700 1 -- 172.21.15.36:0/3962480242 >> v1:172.21.15.70:6789/0 conn(0x7f6e141444f0 legacy=0x7f6e14144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:54.374 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e13fff700 1 -- 172.21.15.36:0/3962480242 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e14147e90 con 0x7f6e140ad330 2022-06-17T12:56:54.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e13fff700 1 --2- 172.21.15.36:0/3962480242 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e140ad330 0x7f6e140ad7c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f6e0c0032f0 tx=0x7f6e0c009290).ready entity=mon.2 client_cookie=ed7af58611325077 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:54.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e12ffd700 1 -- 172.21.15.36:0/3962480242 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e0c00b8a0 con 0x7f6e140ad330 2022-06-17T12:56:54.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e12ffd700 1 -- 172.21.15.36:0/3962480242 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e0c00ba00 con 0x7f6e140ad330 2022-06-17T12:56:54.375 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e12ffd700 1 -- 172.21.15.36:0/3962480242 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e0c00bb70 con 0x7f6e140ad330 2022-06-17T12:56:54.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e19db7700 1 -- 172.21.15.36:0/3962480242 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e140ad330 msgr2=0x7f6e140ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:54.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e19db7700 1 --2- 172.21.15.36:0/3962480242 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e140ad330 0x7f6e140ad7c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f6e0c0032f0 tx=0x7f6e0c009290).stop 2022-06-17T12:56:54.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e19db7700 1 -- 172.21.15.36:0/3962480242 shutdown_connections 2022-06-17T12:56:54.376 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e19db7700 1 --2- 172.21.15.36:0/3962480242 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e140ad330 0x7f6e140ad7c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:54.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e19db7700 1 --2- 172.21.15.36:0/3962480242 >> v2:172.21.15.70:3300/0 conn(0x7f6e14062770 0x7f6e14062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:54.377 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e19db7700 1 -- 172.21.15.36:0/3962480242 >> 172.21.15.36:0/3962480242 conn(0x7f6e140b53e0 msgr2=0x7f6e140b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:54.378 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e19db7700 1 -- 172.21.15.36:0/3962480242 shutdown_connections 2022-06-17T12:56:54.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.350+0000 7f6e19db7700 1 -- 172.21.15.36:0/3962480242 wait complete. 2022-06-17T12:56:54.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e19db7700 1 Processor -- start 2022-06-17T12:56:54.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e19db7700 1 -- start start 2022-06-17T12:56:54.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e19db7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e14062770 0x7f6e1414c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:54.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e19db7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6e140ad330 0x7f6e1414aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:54.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e19db7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e141444f0 0x7f6e1414af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:54.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e19db7700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e14131e80 con 0x7f6e140ad330 2022-06-17T12:56:54.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e19db7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e141567d0 con 0x7f6e141444f0 2022-06-17T12:56:54.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e19db7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e14143f70 con 0x7f6e14062770 2022-06-17T12:56:54.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e0affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6e140ad330 0x7f6e1414aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:54.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e0affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6e140ad330 0x7f6e1414aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40204/0 (socket says 172.21.15.36:40204) 2022-06-17T12:56:54.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e0affd700 1 -- 172.21.15.36:0/1802029100 learned_addr learned my addr 172.21.15.36:0/1802029100 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:54.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e137fe700 1 --2- 172.21.15.36:0/1802029100 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e14062770 0x7f6e1414c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:54.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e13fff700 1 --2- 172.21.15.36:0/1802029100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e141444f0 0x7f6e1414af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:54.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e0affd700 1 -- 172.21.15.36:0/1802029100 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e14062770 msgr2=0x7f6e1414c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:54.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e0affd700 1 --2- 172.21.15.36:0/1802029100 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e14062770 0x7f6e1414c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:54.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e0affd700 1 -- 172.21.15.36:0/1802029100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e141444f0 msgr2=0x7f6e1414af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:54.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e0affd700 1 --2- 172.21.15.36:0/1802029100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e141444f0 0x7f6e1414af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:54.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e0affd700 1 -- 172.21.15.36:0/1802029100 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c009530 con 0x7f6e140ad330 2022-06-17T12:56:54.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e13fff700 1 --2- 172.21.15.36:0/1802029100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e141444f0 0x7f6e1414af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:56:54.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e0affd700 1 --2- 172.21.15.36:0/1802029100 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6e140ad330 0x7f6e1414aa20 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0x7f6dfc000ed0 tx=0x7f6dfc007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:54.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e117fa700 1 -- 172.21.15.36:0/1802029100 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6dfc0070d0 con 0x7f6e140ad330 2022-06-17T12:56:54.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e117fa700 1 -- 172.21.15.36:0/1802029100 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dfc007230 con 0x7f6e140ad330 2022-06-17T12:56:54.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e19db7700 1 -- 172.21.15.36:0/1802029100 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e14132060 con 0x7f6e140ad330 2022-06-17T12:56:54.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e117fa700 1 -- 172.21.15.36:0/1802029100 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6dfc005550 con 0x7f6e140ad330 2022-06-17T12:56:54.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.354+0000 7f6e19db7700 1 -- 172.21.15.36:0/1802029100 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e14064590 con 0x7f6e140ad330 2022-06-17T12:56:54.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.358+0000 7f6e19db7700 1 -- 172.21.15.36:0/1802029100 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e14064c40 con 0x7f6e140ad330 2022-06-17T12:56:54.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.358+0000 7f6e117fa700 1 -- 172.21.15.36:0/1802029100 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6dfc01a060 con 0x7f6e140ad330 2022-06-17T12:56:54.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.358+0000 7f6e117fa700 1 --2- 172.21.15.36:0/1802029100 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6df8053a60 0x7f6df8055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:54.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.358+0000 7f6e117fa700 1 -- 172.21.15.36:0/1802029100 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6dfc06b3d0 con 0x7f6e140ad330 2022-06-17T12:56:54.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.358+0000 7f6e137fe700 1 --2- 172.21.15.36:0/1802029100 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6df8053a60 0x7f6df8055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:54.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.358+0000 7f6e137fe700 1 --2- 172.21.15.36:0/1802029100 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6df8053a60 0x7f6df8055f20 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f6e0400b000 tx=0x7f6e04006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:54.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.362+0000 7f6e117fa700 1 -- 172.21.15.36:0/1802029100 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6e14064c40 con 0x7f6e140ad330 2022-06-17T12:56:54.506 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.522+0000 7f6e19db7700 1 -- 172.21.15.36:0/1802029100 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e14064c40 con 0x7f6e140ad330 2022-06-17T12:56:54.507 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.522+0000 7f6e117fa700 1 -- 172.21.15.36:0/1802029100 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f6e14064c40 con 0x7f6e140ad330 2022-06-17T12:56:54.508 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:54.509 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":78,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1081450840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1081450840},{"type":"v1","addr":"172.21.15.70:6833","nonce":1081450840}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":76}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:56:49.200774+0000","modified":"2022-06-17T12:56:50.204509+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8478":{"gid":8478,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/3169088536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3169088536},{"type":"v1","addr":"172.21.15.36:6835","nonce":3169088536}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-06-17T12:56:54.510 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.526+0000 7f6e19db7700 1 -- 172.21.15.36:0/1802029100 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6df8053a60 msgr2=0x7f6df8055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:54.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.526+0000 7f6e19db7700 1 --2- 172.21.15.36:0/1802029100 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6df8053a60 0x7f6df8055f20 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f6e0400b000 tx=0x7f6e04006df0).stop 2022-06-17T12:56:54.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.526+0000 7f6e19db7700 1 -- 172.21.15.36:0/1802029100 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6e140ad330 msgr2=0x7f6e1414aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:54.511 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.526+0000 7f6e19db7700 1 --2- 172.21.15.36:0/1802029100 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6e140ad330 0x7f6e1414aa20 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0x7f6dfc000ed0 tx=0x7f6dfc007c80).stop 2022-06-17T12:56:54.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.526+0000 7f6e19db7700 1 -- 172.21.15.36:0/1802029100 shutdown_connections 2022-06-17T12:56:54.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.526+0000 7f6e19db7700 1 --2- 172.21.15.36:0/1802029100 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6df8053a60 0x7f6df8055f20 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:54.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.526+0000 7f6e19db7700 1 --2- 172.21.15.36:0/1802029100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e141444f0 0x7f6e1414af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:54.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.526+0000 7f6e19db7700 1 --2- 172.21.15.36:0/1802029100 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6e140ad330 0x7f6e1414aa20 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:54.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.526+0000 7f6e19db7700 1 --2- 172.21.15.36:0/1802029100 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e14062770 0x7f6e1414c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:54.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.526+0000 7f6e19db7700 1 -- 172.21.15.36:0/1802029100 >> 172.21.15.36:0/1802029100 conn(0x7f6e140b53e0 msgr2=0x7f6e14144960 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:54.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.526+0000 7f6e19db7700 1 -- 172.21.15.36:0/1802029100 shutdown_connections 2022-06-17T12:56:54.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:54.526+0000 7f6e19db7700 1 -- 172.21.15.36:0/1802029100 wait complete. 2022-06-17T12:56:54.513 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 78 2022-06-17T12:56:54.751 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 78, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:56:49.200774+0000', 'modified': '2022-06-17T12:56:50.204509+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8478}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8478': {'gid': 8478, 'name': 'a', 'rank': 0, 'incarnation': 77, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.36:6835/3169088536', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 3169088536}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 3169088536}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:56:54.752 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:56:54.753 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:56:54.902 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":8478,"want_state":"up:active","state":"up:active","rank_uptime":5.7049656669999997,"mdsmap_epoch":78,"osdmap_epoch":141,"osdmap_epoch_barrier":140,"uptime":10.778172254999999} 2022-06-17T12:56:54.903 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 8478, "mdsmap_epoch": 78, "osdmap_epoch": 141, "osdmap_epoch_barrier": 140, "rank_uptime": 5.704965667, "state": "up:active", "uptime": 10.778172255, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:56:54.905 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:56:54.905 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-17T12:56:54.906 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi107.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-17T12:56:54.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:54.908 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:56:54.918 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:56:54.989 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:56:54.990 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:56:55.028 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:56:55.035 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:56:55.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:55.038 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:56:55.104 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:56:55.106 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:56:55.111 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:56:55.150 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:56:55.152 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:56:55.194 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:2022-06-17T12:56:55.196+0000 7f95d7713080 -1 init, newargv = 0x555df38a0760 newargc=9ceph-fuse[23004]: starting ceph client 2022-06-17T12:56:55.195 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr: 2022-06-17T12:56:55.196 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[23004]: starting fuse 2022-06-17T12:56:55.214 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:56:55.233 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:56:55.248 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:55.249 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:56:55.299 INFO:teuthology.orchestra.run.smithi107.stdout:52 2022-06-17T12:56:55.300 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:56:55.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:55.302 DEBUG:teuthology.orchestra.run.smithi107:> sudo python3 -c ' 2022-06-17T12:56:55.303 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2022-06-17T12:56:55.303 DEBUG:teuthology.orchestra.run.smithi107:> import re 2022-06-17T12:56:55.304 DEBUG:teuthology.orchestra.run.smithi107:> import os 2022-06-17T12:56:55.313 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2022-06-17T12:56:55.314 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:56:55.315 DEBUG:teuthology.orchestra.run.smithi107:> def find_socket(client_name): 2022-06-17T12:56:55.315 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-17T12:56:55.316 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2022-06-17T12:56:55.316 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:56:55.317 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2022-06-17T12:56:55.325 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2022-06-17T12:56:55.325 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2022-06-17T12:56:55.342 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2022-06-17T12:56:55.343 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:56:55.343 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2022-06-17T12:56:55.344 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:56:55.344 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:56:55.354 DEBUG:teuthology.orchestra.run.smithi107:> return f 2022-06-17T12:56:55.355 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:56:55.360 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:56:55.361 DEBUG:teuthology.orchestra.run.smithi107:> print(find_socket("client.0")) 2022-06-17T12:56:55.361 DEBUG:teuthology.orchestra.run.smithi107:> ' 2022-06-17T12:56:55.428 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.0.23004.asok 2022-06-17T12:56:55.429 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23004.asok 2022-06-17T12:56:55.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:55.436 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23004.asok status 2022-06-17T12:56:55.587 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2022-06-17T12:56:55.587 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2022-06-17T12:56:55.588 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:56:55.588 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:56:55.589 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "0", 2022-06-17T12:56:55.591 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2022-06-17T12:56:55.591 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-17T12:56:55.591 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "23004", 2022-06-17T12:56:55.591 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/" 2022-06-17T12:56:55.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:56:55.592 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2022-06-17T12:56:55.592 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2022-06-17T12:56:55.593 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 7322, 2022-06-17T12:56:55.593 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2022-06-17T12:56:55.593 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2022-06-17T12:56:55.593 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2022-06-17T12:56:55.594 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 7322 2022-06-17T12:56:55.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:56:55.594 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:56:55.594 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:56:55.594 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:56:55.595 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4242276788 2022-06-17T12:56:55.595 INFO:teuthology.orchestra.run.smithi107.stdout: } 2022-06-17T12:56:55.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:56:55.595 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:56:55.602 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:56:55.607 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:56:55.614 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4242276788 2022-06-17T12:56:55.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:56:55.615 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.7322 172.21.15.107:0/4242276788", 2022-06-17T12:56:55.624 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "172.21.15.107:0/4242276788", 2022-06-17T12:56:55.625 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 1, 2022-06-17T12:56:55.625 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 78, 2022-06-17T12:56:55.625 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 141, 2022-06-17T12:56:55.626 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:56:55.626 INFO:teuthology.orchestra.run.smithi107.stdout: "blacklisted": false, 2022-06-17T12:56:55.628 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2022-06-17T12:56:55.628 INFO:teuthology.orchestra.run.smithi107.stdout:} 2022-06-17T12:56:55.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:55.645 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:55.646 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:56:55.668 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:56:55.669 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:56:55.669 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:55.670 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:56:55.747 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:56:55.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.886+0000 7f6efb59e700 1 Processor -- start 2022-06-17T12:56:55.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.886+0000 7f6efb59e700 1 -- start start 2022-06-17T12:56:55.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.886+0000 7f6efb59e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f6efc0b20f0 0x7f6efc0aee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:55.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.886+0000 7f6efb59e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6efc0afe50 0x7f6efc13c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:55.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.886+0000 7f6efb59e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6efc060ea0 con 0x7f6efc060ac0 2022-06-17T12:56:55.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.886+0000 7f6efb59e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6efc0b3060 con 0x7f6efc0b20f0 2022-06-17T12:56:55.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.886+0000 7f6efb59e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6efc067a20 con 0x7f6efc0afe50 2022-06-17T12:56:55.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.886+0000 7f6ef9d9b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f6efc0b20f0 0x7f6efc0aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:55.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.886+0000 7f6efa59c700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f6efc060ac0 0x7f6efc0ae720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57738/0 (socket says 172.21.15.36:57738) 2022-06-17T12:56:55.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.886+0000 7f6efa59c700 1 -- 172.21.15.36:0/2678615788 learned_addr learned my addr 172.21.15.36:0/2678615788 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:55.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.886+0000 7f6efad9d700 1 --2- 172.21.15.36:0/2678615788 >> v2:172.21.15.70:3300/0 conn(0x7f6efc0afe50 0x7f6efc13c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:55.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.886+0000 7f6ef9d9b700 1 -- 172.21.15.36:0/2678615788 >> v2:172.21.15.70:3300/0 conn(0x7f6efc0afe50 msgr2=0x7f6efc13c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:55.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.886+0000 7f6ef9d9b700 1 --2- 172.21.15.36:0/2678615788 >> v2:172.21.15.70:3300/0 conn(0x7f6efc0afe50 0x7f6efc13c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:55.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.886+0000 7f6ef959a700 1 -- 172.21.15.36:0/2678615788 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 420980229 0 0) 0x7f6efc060ea0 con 0x7f6efc060ac0 2022-06-17T12:56:55.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.886+0000 7f6ef9d9b700 1 -- 172.21.15.36:0/2678615788 >> v1:172.21.15.36:6789/0 conn(0x7f6efc060ac0 legacy=0x7f6efc0ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:55.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.886+0000 7f6ef9d9b700 1 -- 172.21.15.36:0/2678615788 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6efc143fb0 con 0x7f6efc0b20f0 2022-06-17T12:56:55.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.886+0000 7f6ef9d9b700 1 --2- 172.21.15.36:0/2678615788 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6efc0b20f0 0x7f6efc0aee30 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7f6ef0002f70 tx=0x7f6ef0006df0).ready entity=mon.0 client_cookie=fea688a85415918d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:55.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.886+0000 7f6ef959a700 1 -- 172.21.15.36:0/2678615788 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ef000d570 con 0x7f6efc0b20f0 2022-06-17T12:56:55.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.886+0000 7f6ef959a700 1 -- 172.21.15.36:0/2678615788 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef0000e90 con 0x7f6efc0b20f0 2022-06-17T12:56:55.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.886+0000 7f6ef959a700 1 -- 172.21.15.36:0/2678615788 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ef000d8c0 con 0x7f6efc0b20f0 2022-06-17T12:56:55.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.890+0000 7f6efb59e700 1 -- 172.21.15.36:0/2678615788 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6efc0b20f0 msgr2=0x7f6efc0aee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:55.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.890+0000 7f6efb59e700 1 --2- 172.21.15.36:0/2678615788 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6efc0b20f0 0x7f6efc0aee30 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7f6ef0002f70 tx=0x7f6ef0006df0).stop 2022-06-17T12:56:55.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.890+0000 7f6efb59e700 1 -- 172.21.15.36:0/2678615788 shutdown_connections 2022-06-17T12:56:55.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.890+0000 7f6efb59e700 1 --2- 172.21.15.36:0/2678615788 >> v2:172.21.15.70:3300/0 conn(0x7f6efc0afe50 0x7f6efc13c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:55.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.890+0000 7f6efb59e700 1 --2- 172.21.15.36:0/2678615788 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6efc0b20f0 0x7f6efc0aee30 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:55.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.890+0000 7f6efb59e700 1 -- 172.21.15.36:0/2678615788 >> 172.21.15.36:0/2678615788 conn(0x7f6efc0b65f0 msgr2=0x7f6efc0b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:55.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.890+0000 7f6efb59e700 1 -- 172.21.15.36:0/2678615788 shutdown_connections 2022-06-17T12:56:55.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.890+0000 7f6efb59e700 1 -- 172.21.15.36:0/2678615788 wait complete. 2022-06-17T12:56:55.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.890+0000 7f6efb59e700 1 Processor -- start 2022-06-17T12:56:55.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.890+0000 7f6efb59e700 1 -- start start 2022-06-17T12:56:55.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.890+0000 7f6efb59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6efc060ac0 0x7f6efc1442b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:55.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.890+0000 7f6efb59e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6efc0afe50 0x7f6efc0b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:55.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.890+0000 7f6efb59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6efc0b20f0 0x7f6efc13e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:55.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.890+0000 7f6efb59e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc13e3a0 con 0x7f6efc0b20f0 2022-06-17T12:56:55.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.890+0000 7f6efb59e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc0b51f0 con 0x7f6efc0afe50 2022-06-17T12:56:55.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.890+0000 7f6efb59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc1456c0 con 0x7f6efc060ac0 2022-06-17T12:56:55.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.890+0000 7f6efad9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6efc0b20f0 0x7f6efc13e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:55.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.890+0000 7f6efad9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6efc0b20f0 0x7f6efc13e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40216/0 (socket says 172.21.15.36:40216) 2022-06-17T12:56:55.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.894+0000 7f6efad9d700 1 -- 172.21.15.36:0/3137857911 learned_addr learned my addr 172.21.15.36:0/3137857911 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:55.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.894+0000 7f6ef9d9b700 1 --2- 172.21.15.36:0/3137857911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6efc0afe50 0x7f6efc0b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:55.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.894+0000 7f6efa59c700 1 --2- 172.21.15.36:0/3137857911 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6efc060ac0 0x7f6efc1442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:56.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.894+0000 7f6efad9d700 1 -- 172.21.15.36:0/3137857911 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6efc060ac0 msgr2=0x7f6efc1442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:56.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.894+0000 7f6efad9d700 1 --2- 172.21.15.36:0/3137857911 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6efc060ac0 0x7f6efc1442b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.894+0000 7f6efad9d700 1 -- 172.21.15.36:0/3137857911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6efc0afe50 msgr2=0x7f6efc0b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:56.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.894+0000 7f6efad9d700 1 --2- 172.21.15.36:0/3137857911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6efc0afe50 0x7f6efc0b49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.894+0000 7f6efad9d700 1 -- 172.21.15.36:0/3137857911 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef000b180 con 0x7f6efc0b20f0 2022-06-17T12:56:56.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.894+0000 7f6efad9d700 1 --2- 172.21.15.36:0/3137857911 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6efc0b20f0 0x7f6efc13e790 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7f6efc13e240 tx=0x7f6ef40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:56.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.894+0000 7f6eeb7fe700 1 -- 172.21.15.36:0/3137857911 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ef400c640 con 0x7f6efc0b20f0 2022-06-17T12:56:56.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.894+0000 7f6eeb7fe700 1 -- 172.21.15.36:0/3137857911 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef400c7a0 con 0x7f6efc0b20f0 2022-06-17T12:56:56.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.894+0000 7f6efb59e700 1 -- 172.21.15.36:0/3137857911 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6efc145960 con 0x7f6efc0b20f0 2022-06-17T12:56:56.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.894+0000 7f6eeb7fe700 1 -- 172.21.15.36:0/3137857911 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ef400cb00 con 0x7f6efc0b20f0 2022-06-17T12:56:56.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.894+0000 7f6efb59e700 1 -- 172.21.15.36:0/3137857911 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6efc147a30 con 0x7f6efc0b20f0 2022-06-17T12:56:56.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.894+0000 7f6efb59e700 1 -- 172.21.15.36:0/3137857911 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6efc146cc0 con 0x7f6efc0b20f0 2022-06-17T12:56:56.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.894+0000 7f6eeb7fe700 1 -- 172.21.15.36:0/3137857911 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6ef400cc60 con 0x7f6efc0b20f0 2022-06-17T12:56:56.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.898+0000 7f6eeb7fe700 1 --2- 172.21.15.36:0/3137857911 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6ed8053a40 0x7f6ed8055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:56.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.898+0000 7f6eeb7fe700 1 -- 172.21.15.36:0/3137857911 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6ef406c1b0 con 0x7f6efc0b20f0 2022-06-17T12:56:56.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.898+0000 7f6efa59c700 1 --2- 172.21.15.36:0/3137857911 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6ed8053a40 0x7f6ed8055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:56.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.898+0000 7f6efa59c700 1 --2- 172.21.15.36:0/3137857911 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6ed8053a40 0x7f6ed8055f00 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f6eec00d000 tx=0x7f6eec007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:56.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:55.906+0000 7f6eeb7fe700 1 -- 172.21.15.36:0/3137857911 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6efc146cc0 con 0x7f6efc0b20f0 2022-06-17T12:56:56.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.034+0000 7f6efb59e700 1 -- 172.21.15.36:0/3137857911 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6efc146cc0 con 0x7f6efc0b20f0 2022-06-17T12:56:56.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.038+0000 7f6eeb7fe700 1 -- 172.21.15.36:0/3137857911 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f6efc146cc0 con 0x7f6efc0b20f0 2022-06-17T12:56:56.057 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:56.057 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":141,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:56:50.204603+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:56:47.177576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-06-17T12:56:48.257688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:56:56.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.038+0000 7f6efb59e700 1 -- 172.21.15.36:0/3137857911 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6ed8053a40 msgr2=0x7f6ed8055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:56.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.038+0000 7f6efb59e700 1 --2- 172.21.15.36:0/3137857911 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6ed8053a40 0x7f6ed8055f00 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f6eec00d000 tx=0x7f6eec007a00).stop 2022-06-17T12:56:56.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.038+0000 7f6efb59e700 1 -- 172.21.15.36:0/3137857911 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6efc0b20f0 msgr2=0x7f6efc13e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:56.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.038+0000 7f6efb59e700 1 --2- 172.21.15.36:0/3137857911 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6efc0b20f0 0x7f6efc13e790 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7f6efc13e240 tx=0x7f6ef40091d0).stop 2022-06-17T12:56:56.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.038+0000 7f6efb59e700 1 -- 172.21.15.36:0/3137857911 shutdown_connections 2022-06-17T12:56:56.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.038+0000 7f6efb59e700 1 --2- 172.21.15.36:0/3137857911 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6ed8053a40 0x7f6ed8055f00 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.038+0000 7f6efb59e700 1 --2- 172.21.15.36:0/3137857911 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6efc0b20f0 0x7f6efc13e790 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.038+0000 7f6efb59e700 1 --2- 172.21.15.36:0/3137857911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6efc0afe50 0x7f6efc0b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.038+0000 7f6efb59e700 1 --2- 172.21.15.36:0/3137857911 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6efc060ac0 0x7f6efc1442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.038+0000 7f6efb59e700 1 -- 172.21.15.36:0/3137857911 >> 172.21.15.36:0/3137857911 conn(0x7f6efc0b65f0 msgr2=0x7f6efc0ae700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:56.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.038+0000 7f6efb59e700 1 -- 172.21.15.36:0/3137857911 shutdown_connections 2022-06-17T12:56:56.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.038+0000 7f6efb59e700 1 -- 172.21.15.36:0/3137857911 wait complete. 2022-06-17T12:56:56.090 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:56:47.177576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:56:48.257688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:56:56.092 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:56:56.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.234+0000 7f56013fd700 1 Processor -- start 2022-06-17T12:56:56.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.234+0000 7f56013fd700 1 -- start start 2022-06-17T12:56:56.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.234+0000 7f56013fd700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f55fc0b05e0 0x7f55fc144950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:56.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.234+0000 7f56013fd700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55fc0b1900 con 0x7f55fc0acb00 2022-06-17T12:56:56.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.234+0000 7f56013fd700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55fc0af370 con 0x7f55fc0acee0 2022-06-17T12:56:56.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.234+0000 7f56013fd700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55fc0b0a50 con 0x7f55fc0b05e0 2022-06-17T12:56:56.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.234+0000 7f55faffd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f55fc0acb00 0x7f55fc147480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57752/0 (socket says 172.21.15.36:57752) 2022-06-17T12:56:56.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.234+0000 7f55faffd700 1 -- 172.21.15.36:0/909172790 learned_addr learned my addr 172.21.15.36:0/909172790 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:56.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.234+0000 7f55fb7fe700 1 --2- 172.21.15.36:0/909172790 >> v2:172.21.15.70:3300/0 conn(0x7f55fc0b05e0 0x7f55fc144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:56.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.234+0000 7f55f9ffb700 1 -- 172.21.15.36:0/909172790 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3453818767 0 0) 0x7f55fc0b1900 con 0x7f55fc0acb00 2022-06-17T12:56:56.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.234+0000 7f55f9ffb700 1 -- 172.21.15.36:0/909172790 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55dc003260 con 0x7f55fc0acb00 2022-06-17T12:56:56.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.234+0000 7f55fb7fe700 1 -- 172.21.15.36:0/909172790 >> v1:172.21.15.70:6789/0 conn(0x7f55fc0acee0 legacy=0x7f55fc144240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:56.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.234+0000 7f55fb7fe700 1 -- 172.21.15.36:0/909172790 >> v1:172.21.15.36:6789/0 conn(0x7f55fc0acb00 legacy=0x7f55fc147480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:56.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.234+0000 7f55fb7fe700 1 -- 172.21.15.36:0/909172790 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55fc144e40 con 0x7f55fc0b05e0 2022-06-17T12:56:56.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.234+0000 7f55fb7fe700 1 --2- 172.21.15.36:0/909172790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55fc0b05e0 0x7f55fc144950 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f55f4002f70 tx=0x7f55f4006df0).ready entity=mon.1 client_cookie=432519ece4fecf55 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:56.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.234+0000 7f55f9ffb700 1 -- 172.21.15.36:0/909172790 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55f400d610 con 0x7f55fc0b05e0 2022-06-17T12:56:56.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.234+0000 7f55f9ffb700 1 -- 172.21.15.36:0/909172790 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f4000e90 con 0x7f55fc0b05e0 2022-06-17T12:56:56.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.234+0000 7f55f9ffb700 1 -- 172.21.15.36:0/909172790 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55f400d960 con 0x7f55fc0b05e0 2022-06-17T12:56:56.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.238+0000 7f56013fd700 1 -- 172.21.15.36:0/909172790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55fc0b05e0 msgr2=0x7f55fc144950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:56.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.238+0000 7f56013fd700 1 --2- 172.21.15.36:0/909172790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55fc0b05e0 0x7f55fc144950 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f55f4002f70 tx=0x7f55f4006df0).stop 2022-06-17T12:56:56.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.238+0000 7f56013fd700 1 -- 172.21.15.36:0/909172790 shutdown_connections 2022-06-17T12:56:56.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.238+0000 7f56013fd700 1 --2- 172.21.15.36:0/909172790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55fc0b05e0 0x7f55fc144950 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.238+0000 7f56013fd700 1 -- 172.21.15.36:0/909172790 >> 172.21.15.36:0/909172790 conn(0x7f55fc05c550 msgr2=0x7f55fc0b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:56.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.238+0000 7f56013fd700 1 -- 172.21.15.36:0/909172790 shutdown_connections 2022-06-17T12:56:56.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.238+0000 7f56013fd700 1 -- 172.21.15.36:0/909172790 wait complete. 2022-06-17T12:56:56.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.238+0000 7f56013fd700 1 Processor -- start 2022-06-17T12:56:56.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.238+0000 7f56013fd700 1 -- start start 2022-06-17T12:56:56.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.238+0000 7f56013fd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55fc0acb00 0x7f55fc148170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:56.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.238+0000 7f56013fd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f55fc0acee0 0x7f55fc063150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:56.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.238+0000 7f56013fd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f55fc0b05e0 0x7f55fc066650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:56.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.238+0000 7f56013fd700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc145a50 con 0x7f55fc0acee0 2022-06-17T12:56:56.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.238+0000 7f56013fd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc0663f0 con 0x7f55fc0acb00 2022-06-17T12:56:56.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.238+0000 7f56013fd700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc145310 con 0x7f55fc0b05e0 2022-06-17T12:56:56.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.238+0000 7f55fa7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f55fc0acee0 0x7f55fc063150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:56.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.238+0000 7f55fa7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f55fc0acee0 0x7f55fc063150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40230/0 (socket says 172.21.15.36:40230) 2022-06-17T12:56:56.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.238+0000 7f55fa7fc700 1 -- 172.21.15.36:0/875521875 learned_addr learned my addr 172.21.15.36:0/875521875 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:56.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.238+0000 7f55fb7fe700 1 --2- 172.21.15.36:0/875521875 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f55fc0b05e0 0x7f55fc066650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:56.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.238+0000 7f55faffd700 1 --2- 172.21.15.36:0/875521875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55fc0acb00 0x7f55fc148170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:56.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.242+0000 7f55fa7fc700 1 -- 172.21.15.36:0/875521875 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f55fc0b05e0 msgr2=0x7f55fc066650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:56.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.242+0000 7f55fa7fc700 1 --2- 172.21.15.36:0/875521875 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f55fc0b05e0 0x7f55fc066650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.242+0000 7f55fa7fc700 1 -- 172.21.15.36:0/875521875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55fc0acb00 msgr2=0x7f55fc148170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:56.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.242+0000 7f55fa7fc700 1 --2- 172.21.15.36:0/875521875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55fc0acb00 0x7f55fc148170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.242+0000 7f55fa7fc700 1 -- 172.21.15.36:0/875521875 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f400b180 con 0x7f55fc0acee0 2022-06-17T12:56:56.366 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.242+0000 7f55faffd700 1 --2- 172.21.15.36:0/875521875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55fc0acb00 0x7f55fc148170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:56:56.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.242+0000 7f55fa7fc700 1 --2- 172.21.15.36:0/875521875 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f55fc0acee0 0x7f55fc063150 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7f55fc0612a0 tx=0x7f55ec0014a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:56.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.242+0000 7f55f3fff700 1 -- 172.21.15.36:0/875521875 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55ec00e4a0 con 0x7f55fc0acee0 2022-06-17T12:56:56.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.242+0000 7f55f3fff700 1 -- 172.21.15.36:0/875521875 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ec00e600 con 0x7f55fc0acee0 2022-06-17T12:56:56.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.242+0000 7f56013fd700 1 -- 172.21.15.36:0/875521875 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55fc1455b0 con 0x7f55fc0acee0 2022-06-17T12:56:56.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.242+0000 7f55f3fff700 1 -- 172.21.15.36:0/875521875 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55ec00e960 con 0x7f55fc0acee0 2022-06-17T12:56:56.380 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.242+0000 7f56013fd700 1 -- 172.21.15.36:0/875521875 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55fc062550 con 0x7f55fc0acee0 2022-06-17T12:56:56.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.242+0000 7f56013fd700 1 -- 172.21.15.36:0/875521875 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55fc066260 con 0x7f55fc0acee0 2022-06-17T12:56:56.381 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.242+0000 7f55f3fff700 1 -- 172.21.15.36:0/875521875 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f55ec019030 con 0x7f55fc0acee0 2022-06-17T12:56:56.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.242+0000 7f55f3fff700 1 --2- 172.21.15.36:0/875521875 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f55e0053a40 0x7f55e0055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:56.382 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.246+0000 7f55f3fff700 1 -- 172.21.15.36:0/875521875 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f55ec06b9c0 con 0x7f55fc0acee0 2022-06-17T12:56:56.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.246+0000 7f55faffd700 1 --2- 172.21.15.36:0/875521875 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f55e0053a40 0x7f55e0055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:56.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.246+0000 7f55faffd700 1 --2- 172.21.15.36:0/875521875 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f55e0053a40 0x7f55e0055f00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f55e400b000 tx=0x7f55e4009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:56.383 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.250+0000 7f55f3fff700 1 -- 172.21.15.36:0/875521875 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f55ec0425d0 con 0x7f55fc0acee0 2022-06-17T12:56:56.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.378+0000 7f56013fd700 1 -- 172.21.15.36:0/875521875 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f55fc145740 con 0x7f55fc0acee0 2022-06-17T12:56:56.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.378+0000 7f55f3fff700 1 -- 172.21.15.36:0/875521875 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v141) v1 ==== 110+0+10 (secure 0 0 0) 0x7f55ec017020 con 0x7f55fc0acee0 2022-06-17T12:56:56.389 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:56:56.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.378+0000 7f56013fd700 1 -- 172.21.15.36:0/875521875 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f55e0053a40 msgr2=0x7f55e0055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:56.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.378+0000 7f56013fd700 1 --2- 172.21.15.36:0/875521875 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f55e0053a40 0x7f55e0055f00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f55e400b000 tx=0x7f55e4009040).stop 2022-06-17T12:56:56.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.378+0000 7f56013fd700 1 -- 172.21.15.36:0/875521875 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f55fc0acee0 msgr2=0x7f55fc063150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:56.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.378+0000 7f56013fd700 1 --2- 172.21.15.36:0/875521875 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f55fc0acee0 0x7f55fc063150 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7f55fc0612a0 tx=0x7f55ec0014a0).stop 2022-06-17T12:56:56.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.378+0000 7f56013fd700 1 -- 172.21.15.36:0/875521875 shutdown_connections 2022-06-17T12:56:56.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.378+0000 7f56013fd700 1 --2- 172.21.15.36:0/875521875 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f55e0053a40 0x7f55e0055f00 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.378+0000 7f56013fd700 1 --2- 172.21.15.36:0/875521875 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f55fc0b05e0 0x7f55fc066650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.378+0000 7f56013fd700 1 --2- 172.21.15.36:0/875521875 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f55fc0acee0 0x7f55fc063150 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.378+0000 7f56013fd700 1 --2- 172.21.15.36:0/875521875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55fc0acb00 0x7f55fc148170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.378+0000 7f56013fd700 1 -- 172.21.15.36:0/875521875 >> 172.21.15.36:0/875521875 conn(0x7f55fc05c550 msgr2=0x7f55fc0b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:56.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.382+0000 7f56013fd700 1 -- 172.21.15.36:0/875521875 shutdown_connections 2022-06-17T12:56:56.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.382+0000 7f56013fd700 1 -- 172.21.15.36:0/875521875 wait complete. 2022-06-17T12:56:56.400 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:56:56.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.542+0000 7f95a4922700 1 Processor -- start 2022-06-17T12:56:56.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.542+0000 7f95a4922700 1 -- start start 2022-06-17T12:56:56.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.542+0000 7f95a4922700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f95a00b2d80 0x7f95a00b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:56.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.542+0000 7f95a4922700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95a0143e60 0x7f95a0140300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:56.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.542+0000 7f95a4922700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95a00aa170 con 0x7f95a00b2350 2022-06-17T12:56:56.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.542+0000 7f95a4922700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95a00a9c30 con 0x7f95a00b2d80 2022-06-17T12:56:56.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.542+0000 7f95a4922700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a00a9ed0 con 0x7f95a0143e60 2022-06-17T12:56:56.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.542+0000 7f959e59c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f95a00b2d80 0x7f95a00b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:56.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.542+0000 7f959e59c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f95a00b2d80 0x7f95a00b3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40236/0 (socket says 172.21.15.36:40236) 2022-06-17T12:56:56.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.542+0000 7f959e59c700 1 -- 172.21.15.36:0/3808402956 learned_addr learned my addr 172.21.15.36:0/3808402956 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:56.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.542+0000 7f959ed9d700 1 --2- 172.21.15.36:0/3808402956 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95a0143e60 0x7f95a0140300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:56.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.542+0000 7f959e59c700 1 -- 172.21.15.36:0/3808402956 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95a0143e60 msgr2=0x7f95a0140300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:56.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.542+0000 7f959e59c700 1 --2- 172.21.15.36:0/3808402956 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95a0143e60 0x7f95a0140300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.542+0000 7f959e59c700 1 -- 172.21.15.36:0/3808402956 >> v1:172.21.15.70:6789/0 conn(0x7f95a00b2350 legacy=0x7f95a0143750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:56.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.542+0000 7f959e59c700 1 -- 172.21.15.36:0/3808402956 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a00b08c0 con 0x7f95a00b2d80 2022-06-17T12:56:56.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.546+0000 7f959e59c700 1 --2- 172.21.15.36:0/3808402956 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95a00b2d80 0x7f95a00b3160 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0x7f9590002f70 tx=0x7f9590006eb0).ready entity=mon.0 client_cookie=732f26e96c8c7a64 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:56.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.546+0000 7f959d59a700 1 -- 172.21.15.36:0/3808402956 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f959000d570 con 0x7f95a00b2d80 2022-06-17T12:56:56.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.546+0000 7f959d59a700 1 -- 172.21.15.36:0/3808402956 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9590000e90 con 0x7f95a00b2d80 2022-06-17T12:56:56.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.546+0000 7f959d59a700 1 -- 172.21.15.36:0/3808402956 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f959000d900 con 0x7f95a00b2d80 2022-06-17T12:56:56.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.546+0000 7f95a4922700 1 -- 172.21.15.36:0/3808402956 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95a00b2d80 msgr2=0x7f95a00b3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:56.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.546+0000 7f95a4922700 1 --2- 172.21.15.36:0/3808402956 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95a00b2d80 0x7f95a00b3160 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0x7f9590002f70 tx=0x7f9590006eb0).stop 2022-06-17T12:56:56.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.546+0000 7f95a4922700 1 -- 172.21.15.36:0/3808402956 shutdown_connections 2022-06-17T12:56:56.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.546+0000 7f95a4922700 1 --2- 172.21.15.36:0/3808402956 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95a0143e60 0x7f95a0140300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.546+0000 7f95a4922700 1 --2- 172.21.15.36:0/3808402956 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95a00b2d80 0x7f95a00b3160 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.546+0000 7f95a4922700 1 -- 172.21.15.36:0/3808402956 >> 172.21.15.36:0/3808402956 conn(0x7f95a005c550 msgr2=0x7f95a005c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:56.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.546+0000 7f95a4922700 1 -- 172.21.15.36:0/3808402956 shutdown_connections 2022-06-17T12:56:56.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.546+0000 7f95a4922700 1 -- 172.21.15.36:0/3808402956 wait complete. 2022-06-17T12:56:56.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.546+0000 7f95a4922700 1 Processor -- start 2022-06-17T12:56:56.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.546+0000 7f95a4922700 1 -- start start 2022-06-17T12:56:56.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.550+0000 7f95a4922700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95a00b2350 0x7f95a014f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:56.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.550+0000 7f95a4922700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95a00b2d80 0x7f95a00aba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:56.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.550+0000 7f95a4922700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95a0143e60 0x7f95a0063bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:56.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.550+0000 7f95a4922700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a0065c80 con 0x7f95a00b2350 2022-06-17T12:56:56.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.550+0000 7f95a4922700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a0066ba0 con 0x7f95a00b2d80 2022-06-17T12:56:56.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.550+0000 7f95a4922700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a0063770 con 0x7f95a0143e60 2022-06-17T12:56:56.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.550+0000 7f959e59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95a00b2350 0x7f95a014f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:56.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.550+0000 7f959e59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95a00b2350 0x7f95a014f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40242/0 (socket says 172.21.15.36:40242) 2022-06-17T12:56:56.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.550+0000 7f959e59c700 1 -- 172.21.15.36:0/2624562908 learned_addr learned my addr 172.21.15.36:0/2624562908 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:56.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.550+0000 7f959ed9d700 1 --2- 172.21.15.36:0/2624562908 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95a0143e60 0x7f95a0063bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:56.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.550+0000 7f959dd9b700 1 --2- 172.21.15.36:0/2624562908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95a00b2d80 0x7f95a00aba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:56.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.550+0000 7f959e59c700 1 -- 172.21.15.36:0/2624562908 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95a0143e60 msgr2=0x7f95a0063bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:56.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.550+0000 7f959e59c700 1 --2- 172.21.15.36:0/2624562908 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95a0143e60 0x7f95a0063bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.550+0000 7f959e59c700 1 -- 172.21.15.36:0/2624562908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95a00b2d80 msgr2=0x7f95a00aba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:56.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.550+0000 7f959e59c700 1 --2- 172.21.15.36:0/2624562908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95a00b2d80 0x7f95a00aba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.550+0000 7f959e59c700 1 -- 172.21.15.36:0/2624562908 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959000b1e0 con 0x7f95a00b2350 2022-06-17T12:56:56.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.550+0000 7f959ed9d700 1 --2- 172.21.15.36:0/2624562908 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95a0143e60 0x7f95a0063bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:56.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.550+0000 7f959e59c700 1 --2- 172.21.15.36:0/2624562908 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95a00b2350 0x7f95a014f820 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7f95a00acc40 tx=0x7f9590015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:56.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.550+0000 7f958f7fe700 1 -- 172.21.15.36:0/2624562908 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95900116e0 con 0x7f95a00b2350 2022-06-17T12:56:56.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.550+0000 7f958f7fe700 1 -- 172.21.15.36:0/2624562908 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9590011840 con 0x7f95a00b2350 2022-06-17T12:56:56.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.550+0000 7f958f7fe700 1 -- 172.21.15.36:0/2624562908 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9590011ba0 con 0x7f95a00b2350 2022-06-17T12:56:56.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.550+0000 7f95a4922700 1 -- 172.21.15.36:0/2624562908 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a00ac700 con 0x7f95a00b2350 2022-06-17T12:56:56.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.550+0000 7f95a4922700 1 -- 172.21.15.36:0/2624562908 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a0064dc0 con 0x7f95a00b2350 2022-06-17T12:56:56.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.554+0000 7f958f7fe700 1 -- 172.21.15.36:0/2624562908 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f959001a040 con 0x7f95a00b2350 2022-06-17T12:56:56.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.554+0000 7f95a4922700 1 -- 172.21.15.36:0/2624562908 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a01442d0 con 0x7f95a00b2350 2022-06-17T12:56:56.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.554+0000 7f958f7fe700 1 --2- 172.21.15.36:0/2624562908 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f957c055c50 0x7f957c058110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:56.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.554+0000 7f958f7fe700 1 -- 172.21.15.36:0/2624562908 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f959006ce40 con 0x7f95a00b2350 2022-06-17T12:56:56.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.554+0000 7f959dd9b700 1 --2- 172.21.15.36:0/2624562908 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f957c055c50 0x7f957c058110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:56.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.554+0000 7f959dd9b700 1 --2- 172.21.15.36:0/2624562908 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f957c055c50 0x7f957c058110 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f959400d000 tx=0x7f9594007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:56.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.562+0000 7f958f7fe700 1 -- 172.21.15.36:0/2624562908 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9590043c80 con 0x7f95a00b2350 2022-06-17T12:56:56.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.686+0000 7f95a4922700 1 -- 172.21.15.36:0/2624562908 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f95a0057410 con 0x7f95a00b2350 2022-06-17T12:56:56.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.686+0000 7f958f7fe700 1 -- 172.21.15.36:0/2624562908 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v141) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9590023070 con 0x7f95a00b2350 2022-06-17T12:56:56.696 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:56:56.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.690+0000 7f95a4922700 1 -- 172.21.15.36:0/2624562908 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f957c055c50 msgr2=0x7f957c058110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:56.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.690+0000 7f95a4922700 1 --2- 172.21.15.36:0/2624562908 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f957c055c50 0x7f957c058110 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f959400d000 tx=0x7f9594007b90).stop 2022-06-17T12:56:56.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.690+0000 7f95a4922700 1 -- 172.21.15.36:0/2624562908 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95a00b2350 msgr2=0x7f95a014f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:56.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.690+0000 7f95a4922700 1 --2- 172.21.15.36:0/2624562908 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95a00b2350 0x7f95a014f820 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7f95a00acc40 tx=0x7f9590015000).stop 2022-06-17T12:56:56.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.690+0000 7f95a4922700 1 -- 172.21.15.36:0/2624562908 shutdown_connections 2022-06-17T12:56:56.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.690+0000 7f95a4922700 1 --2- 172.21.15.36:0/2624562908 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f957c055c50 0x7f957c058110 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.690+0000 7f95a4922700 1 --2- 172.21.15.36:0/2624562908 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95a0143e60 0x7f95a0063bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.690+0000 7f95a4922700 1 --2- 172.21.15.36:0/2624562908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95a00b2d80 0x7f95a00aba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.690+0000 7f95a4922700 1 --2- 172.21.15.36:0/2624562908 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f95a00b2350 0x7f95a014f820 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.690+0000 7f95a4922700 1 -- 172.21.15.36:0/2624562908 >> 172.21.15.36:0/2624562908 conn(0x7f95a005c550 msgr2=0x7f95a01428a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:56.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.690+0000 7f95a4922700 1 -- 172.21.15.36:0/2624562908 shutdown_connections 2022-06-17T12:56:56.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.690+0000 7f95a4922700 1 -- 172.21.15.36:0/2624562908 wait complete. 2022-06-17T12:56:56.739 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:56:56.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.882+0000 7f4981d19700 1 Processor -- start 2022-06-17T12:56:56.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.882+0000 7f4981d19700 1 -- start start 2022-06-17T12:56:56.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4981d19700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f497c05fe80 0x7f497c060260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:56.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4981d19700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f497c0ae360 0x7f497c1444f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:56.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4981d19700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f497c0b1db0 0x7f497c1437d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:56.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4981d19700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f497c0633c0 con 0x7f497c05fe80 2022-06-17T12:56:56.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4981d19700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f497c143330 con 0x7f497c0b1db0 2022-06-17T12:56:56.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4981d19700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497c063100 con 0x7f497c0ae360 2022-06-17T12:56:56.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4980d17700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f497c05fe80 0x7f497c060260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:56.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4980d17700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f497c05fe80 0x7f497c060260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40250/0 (socket says 172.21.15.36:40250) 2022-06-17T12:56:56.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4980d17700 1 -- 172.21.15.36:0/1581003939 learned_addr learned my addr 172.21.15.36:0/1581003939 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:56.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4981518700 1 --2- 172.21.15.36:0/1581003939 >> v2:172.21.15.70:3300/0 conn(0x7f497c0b1db0 0x7f497c1437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:56.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f497bfff700 1 --2- 172.21.15.36:0/1581003939 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f497c0ae360 0x7f497c1444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:56.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4980d17700 1 -- 172.21.15.36:0/1581003939 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f497c0ae360 msgr2=0x7f497c1444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:56.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4980d17700 1 --2- 172.21.15.36:0/1581003939 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f497c0ae360 0x7f497c1444f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4980d17700 1 -- 172.21.15.36:0/1581003939 >> v2:172.21.15.70:3300/0 conn(0x7f497c0b1db0 msgr2=0x7f497c1437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:56.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4980d17700 1 --2- 172.21.15.36:0/1581003939 >> v2:172.21.15.70:3300/0 conn(0x7f497c0b1db0 0x7f497c1437d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4980d17700 1 -- 172.21.15.36:0/1581003939 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497c143020 con 0x7f497c05fe80 2022-06-17T12:56:56.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4980d17700 1 --2- 172.21.15.36:0/1581003939 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f497c05fe80 0x7f497c060260 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 rx=0x7f4968002f70 tx=0x7f4968006eb0).ready entity=mon.0 client_cookie=9195ad892d13a57a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:56.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f497b7fe700 1 -- 172.21.15.36:0/1581003939 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f496800d570 con 0x7f497c05fe80 2022-06-17T12:56:56.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f497b7fe700 1 -- 172.21.15.36:0/1581003939 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4968000e90 con 0x7f497c05fe80 2022-06-17T12:56:56.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f497b7fe700 1 -- 172.21.15.36:0/1581003939 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f496800d900 con 0x7f497c05fe80 2022-06-17T12:56:56.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4981d19700 1 -- 172.21.15.36:0/1581003939 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f497c05fe80 msgr2=0x7f497c060260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:56.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4981d19700 1 --2- 172.21.15.36:0/1581003939 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f497c05fe80 0x7f497c060260 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 rx=0x7f4968002f70 tx=0x7f4968006eb0).stop 2022-06-17T12:56:56.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4981d19700 1 -- 172.21.15.36:0/1581003939 shutdown_connections 2022-06-17T12:56:56.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4981d19700 1 --2- 172.21.15.36:0/1581003939 >> v2:172.21.15.70:3300/0 conn(0x7f497c0b1db0 0x7f497c1437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4981d19700 1 --2- 172.21.15.36:0/1581003939 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f497c0ae360 0x7f497c1444f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.936 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4981d19700 1 --2- 172.21.15.36:0/1581003939 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f497c05fe80 0x7f497c060260 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:56.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4981d19700 1 -- 172.21.15.36:0/1581003939 >> 172.21.15.36:0/1581003939 conn(0x7f497c060990 msgr2=0x7f497c05f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:56.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4981d19700 1 -- 172.21.15.36:0/1581003939 shutdown_connections 2022-06-17T12:56:56.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.886+0000 7f4981d19700 1 -- 172.21.15.36:0/1581003939 wait complete. 2022-06-17T12:56:56.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f4981d19700 1 Processor -- start 2022-06-17T12:56:56.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f4981d19700 1 -- start start 2022-06-17T12:56:57.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f4981d19700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f497c05fe80 0x7f497c0b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:57.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f4981d19700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f497c0ae360 0x7f497c0669b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:57.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f4981d19700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f497c0b1db0 0x7f497c13f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:57.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f4981d19700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497c066550 con 0x7f497c0ae360 2022-06-17T12:56:57.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f4981d19700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497c065a20 con 0x7f497c0b1db0 2022-06-17T12:56:57.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f4981d19700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497c065cc0 con 0x7f497c05fe80 2022-06-17T12:56:57.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f497bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f497c0ae360 0x7f497c0669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:57.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f497bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f497c0ae360 0x7f497c0669b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40258/0 (socket says 172.21.15.36:40258) 2022-06-17T12:56:57.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f497bfff700 1 -- 172.21.15.36:0/3071920396 learned_addr learned my addr 172.21.15.36:0/3071920396 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:57.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f4980d17700 1 --2- 172.21.15.36:0/3071920396 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f497c05fe80 0x7f497c0b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:57.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f4981518700 1 --2- 172.21.15.36:0/3071920396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f497c0b1db0 0x7f497c13f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:57.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f497bfff700 1 -- 172.21.15.36:0/3071920396 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f497c05fe80 msgr2=0x7f497c0b0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:57.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f497bfff700 1 --2- 172.21.15.36:0/3071920396 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f497c05fe80 0x7f497c0b0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:57.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f497bfff700 1 -- 172.21.15.36:0/3071920396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f497c0b1db0 msgr2=0x7f497c13f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:57.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f497bfff700 1 --2- 172.21.15.36:0/3071920396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f497c0b1db0 0x7f497c13f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:57.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f497bfff700 1 -- 172.21.15.36:0/3071920396 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496800b1e0 con 0x7f497c0ae360 2022-06-17T12:56:57.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f4981518700 1 --2- 172.21.15.36:0/3071920396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f497c0b1db0 0x7f497c13f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:56:57.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f497bfff700 1 --2- 172.21.15.36:0/3071920396 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f497c0ae360 0x7f497c0669b0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0x7f497000c000 tx=0x7f4970006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:57.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f4979ffb700 1 -- 172.21.15.36:0/3071920396 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4970010780 con 0x7f497c0ae360 2022-06-17T12:56:57.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f4979ffb700 1 -- 172.21.15.36:0/3071920396 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49700108e0 con 0x7f497c0ae360 2022-06-17T12:56:57.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f4981d19700 1 -- 172.21.15.36:0/3071920396 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f497c0670c0 con 0x7f497c0ae360 2022-06-17T12:56:57.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f4979ffb700 1 -- 172.21.15.36:0/3071920396 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4970010c40 con 0x7f497c0ae360 2022-06-17T12:56:57.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.890+0000 7f4981d19700 1 -- 172.21.15.36:0/3071920396 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f497c13dbd0 con 0x7f497c0ae360 2022-06-17T12:56:57.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.894+0000 7f4979ffb700 1 -- 172.21.15.36:0/3071920396 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f497001c460 con 0x7f497c0ae360 2022-06-17T12:56:57.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.894+0000 7f4981d19700 1 -- 172.21.15.36:0/3071920396 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f497c0666c0 con 0x7f497c0ae360 2022-06-17T12:56:57.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.894+0000 7f4979ffb700 1 --2- 172.21.15.36:0/3071920396 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f495c053a60 0x7f495c055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:57.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.894+0000 7f4979ffb700 1 -- 172.21.15.36:0/3071920396 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f497006c4b0 con 0x7f497c0ae360 2022-06-17T12:56:57.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.894+0000 7f4980d17700 1 --2- 172.21.15.36:0/3071920396 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f495c053a60 0x7f495c055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:57.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.894+0000 7f4980d17700 1 --2- 172.21.15.36:0/3071920396 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f495c053a60 0x7f495c055f20 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f496801c000 tx=0x7f4968015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:57.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:56.898+0000 7f4979ffb700 1 -- 172.21.15.36:0/3071920396 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f49700431b0 con 0x7f497c0ae360 2022-06-17T12:56:57.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.022+0000 7f4981d19700 1 -- 172.21.15.36:0/3071920396 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f497c143be0 con 0x7f497c0ae360 2022-06-17T12:56:57.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.022+0000 7f4979ffb700 1 -- 172.21.15.36:0/3071920396 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v141) v1 ==== 104+0+10 (secure 0 0 0) 0x7f49700467d0 con 0x7f497c0ae360 2022-06-17T12:56:57.087 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:56:57.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.026+0000 7f4981d19700 1 -- 172.21.15.36:0/3071920396 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f495c053a60 msgr2=0x7f495c055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:57.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.026+0000 7f4981d19700 1 --2- 172.21.15.36:0/3071920396 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f495c053a60 0x7f495c055f20 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f496801c000 tx=0x7f4968015000).stop 2022-06-17T12:56:57.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.026+0000 7f4981d19700 1 -- 172.21.15.36:0/3071920396 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f497c0ae360 msgr2=0x7f497c0669b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:57.093 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.026+0000 7f4981d19700 1 --2- 172.21.15.36:0/3071920396 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f497c0ae360 0x7f497c0669b0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0x7f497000c000 tx=0x7f4970006df0).stop 2022-06-17T12:56:57.094 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.026+0000 7f4981d19700 1 -- 172.21.15.36:0/3071920396 shutdown_connections 2022-06-17T12:56:57.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.026+0000 7f4981d19700 1 --2- 172.21.15.36:0/3071920396 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f495c053a60 0x7f495c055f20 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:57.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.026+0000 7f4981d19700 1 --2- 172.21.15.36:0/3071920396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f497c0b1db0 0x7f497c13f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:57.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.026+0000 7f4981d19700 1 --2- 172.21.15.36:0/3071920396 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f497c0ae360 0x7f497c0669b0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:57.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.026+0000 7f4981d19700 1 --2- 172.21.15.36:0/3071920396 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f497c05fe80 0x7f497c0b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:57.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.026+0000 7f4981d19700 1 -- 172.21.15.36:0/3071920396 >> 172.21.15.36:0/3071920396 conn(0x7f497c060990 msgr2=0x7f497c141570 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:57.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.026+0000 7f4981d19700 1 -- 172.21.15.36:0/3071920396 shutdown_connections 2022-06-17T12:56:57.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.026+0000 7f4981d19700 1 -- 172.21.15.36:0/3071920396 wait complete. 2022-06-17T12:56:57.111 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:56:57.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.282+0000 7f11cb521700 1 Processor -- start 2022-06-17T12:56:57.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.282+0000 7f11cb521700 1 -- start start 2022-06-17T12:56:57.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.282+0000 7f11cb521700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11c40b3030 0x7f11c40b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:57.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.282+0000 7f11cb521700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f11c4143080 0x7f11c4062400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:57.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.282+0000 7f11cb521700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c40b2650 con 0x7f11c40b7790 2022-06-17T12:56:57.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.282+0000 7f11cb521700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c40b28f0 con 0x7f11c4143080 2022-06-17T12:56:57.278 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.282+0000 7f11cb521700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c40b2b90 con 0x7f11c40b3030 2022-06-17T12:56:57.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.282+0000 7f11c9abe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f11c4143080 0x7f11c4062400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:57.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.282+0000 7f11c92bd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f11c40b7790 0x7f11c413ce10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57794/0 (socket says 172.21.15.36:57794) 2022-06-17T12:56:57.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.282+0000 7f11c9abe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f11c4143080 0x7f11c4062400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40268/0 (socket says 172.21.15.36:40268) 2022-06-17T12:56:57.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.282+0000 7f11c92bd700 1 -- 172.21.15.36:0/26457053 learned_addr learned my addr 172.21.15.36:0/26457053 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:57.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.286+0000 7f11c8abc700 1 --2- 172.21.15.36:0/26457053 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11c40b3030 0x7f11c40b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:57.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.286+0000 7f11c9abe700 1 -- 172.21.15.36:0/26457053 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11c40b3030 msgr2=0x7f11c40b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:56:57.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.286+0000 7f11c9abe700 1 --2- 172.21.15.36:0/26457053 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11c40b3030 0x7f11c40b7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:57.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.286+0000 7f11c9abe700 1 -- 172.21.15.36:0/26457053 >> v1:172.21.15.36:6789/0 conn(0x7f11c40b7790 legacy=0x7f11c413ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:57.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.286+0000 7f11c9abe700 1 -- 172.21.15.36:0/26457053 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c413d520 con 0x7f11c4143080 2022-06-17T12:56:57.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.286+0000 7f11c8abc700 1 --2- 172.21.15.36:0/26457053 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11c40b3030 0x7f11c40b7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:56:57.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.286+0000 7f11c9abe700 1 --2- 172.21.15.36:0/26457053 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11c4143080 0x7f11c4062400 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7f11bc0032f0 tx=0x7f11bc0091d0).ready entity=mon.0 client_cookie=6db268d08136b689 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:57.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.286+0000 7f11b3fff700 1 -- 172.21.15.36:0/26457053 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11bc00b6c0 con 0x7f11c4143080 2022-06-17T12:56:57.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.286+0000 7f11b3fff700 1 -- 172.21.15.36:0/26457053 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11bc00b820 con 0x7f11c4143080 2022-06-17T12:56:57.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.286+0000 7f11b3fff700 1 -- 172.21.15.36:0/26457053 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11bc00b990 con 0x7f11c4143080 2022-06-17T12:56:57.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.286+0000 7f11cb521700 1 -- 172.21.15.36:0/26457053 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11c4143080 msgr2=0x7f11c4062400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:57.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.286+0000 7f11cb521700 1 --2- 172.21.15.36:0/26457053 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11c4143080 0x7f11c4062400 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7f11bc0032f0 tx=0x7f11bc0091d0).stop 2022-06-17T12:56:57.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.286+0000 7f11cb521700 1 -- 172.21.15.36:0/26457053 shutdown_connections 2022-06-17T12:56:57.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.286+0000 7f11cb521700 1 --2- 172.21.15.36:0/26457053 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11c4143080 0x7f11c4062400 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:57.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.286+0000 7f11cb521700 1 --2- 172.21.15.36:0/26457053 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11c40b3030 0x7f11c40b7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:57.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.286+0000 7f11cb521700 1 -- 172.21.15.36:0/26457053 >> 172.21.15.36:0/26457053 conn(0x7f11c4060990 msgr2=0x7f11c405c550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:57.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.286+0000 7f11cb521700 1 -- 172.21.15.36:0/26457053 shutdown_connections 2022-06-17T12:56:57.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.286+0000 7f11cb521700 1 -- 172.21.15.36:0/26457053 wait complete. 2022-06-17T12:56:57.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.290+0000 7f11cb521700 1 Processor -- start 2022-06-17T12:56:57.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.290+0000 7f11cb521700 1 -- start start 2022-06-17T12:56:57.320 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.290+0000 7f11cb521700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11c40b3030 0x7f11c4147b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:57.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.290+0000 7f11cb521700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11c40b7790 0x7f11c40658e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:57.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.290+0000 7f11cb521700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11c4143080 0x7f11c4142500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:57.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.290+0000 7f11cb521700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c41420d0 con 0x7f11c4143080 2022-06-17T12:56:57.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.290+0000 7f11cb521700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c41412f0 con 0x7f11c40b3030 2022-06-17T12:56:57.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.290+0000 7f11cb521700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c4141590 con 0x7f11c40b7790 2022-06-17T12:56:57.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.290+0000 7f11c9abe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11c4143080 0x7f11c4142500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:57.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.290+0000 7f11c9abe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11c4143080 0x7f11c4142500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40272/0 (socket says 172.21.15.36:40272) 2022-06-17T12:56:57.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.290+0000 7f11c9abe700 1 -- 172.21.15.36:0/3270111098 learned_addr learned my addr 172.21.15.36:0/3270111098 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:57.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.290+0000 7f11c92bd700 1 --2- 172.21.15.36:0/3270111098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11c40b3030 0x7f11c4147b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:57.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.290+0000 7f11c8abc700 1 --2- 172.21.15.36:0/3270111098 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11c40b7790 0x7f11c40658e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:57.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.290+0000 7f11c9abe700 1 -- 172.21.15.36:0/3270111098 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11c40b7790 msgr2=0x7f11c40658e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:57.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.290+0000 7f11c9abe700 1 --2- 172.21.15.36:0/3270111098 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11c40b7790 0x7f11c40658e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:57.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.290+0000 7f11c9abe700 1 -- 172.21.15.36:0/3270111098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11c40b3030 msgr2=0x7f11c4147b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:57.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.290+0000 7f11c9abe700 1 --2- 172.21.15.36:0/3270111098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11c40b3030 0x7f11c4147b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:57.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.290+0000 7f11c9abe700 1 -- 172.21.15.36:0/3270111098 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b400b180 con 0x7f11c4143080 2022-06-17T12:56:57.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.294+0000 7f11c9abe700 1 --2- 172.21.15.36:0/3270111098 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11c4143080 0x7f11c4142500 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7f11bc0032f0 tx=0x7f11bc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:57.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.294+0000 7f11b27fc700 1 -- 172.21.15.36:0/3270111098 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11bc011d70 con 0x7f11c4143080 2022-06-17T12:56:57.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.294+0000 7f11b27fc700 1 -- 172.21.15.36:0/3270111098 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11bc021440 con 0x7f11c4143080 2022-06-17T12:56:57.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.294+0000 7f11cb521700 1 -- 172.21.15.36:0/3270111098 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11bc009470 con 0x7f11c4143080 2022-06-17T12:56:57.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.294+0000 7f11b27fc700 1 -- 172.21.15.36:0/3270111098 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11bc0217a0 con 0x7f11c4143080 2022-06-17T12:56:57.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.294+0000 7f11cb521700 1 -- 172.21.15.36:0/3270111098 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c40672d0 con 0x7f11c4143080 2022-06-17T12:56:57.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.294+0000 7f11cb521700 1 -- 172.21.15.36:0/3270111098 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c4141e10 con 0x7f11c4143080 2022-06-17T12:56:57.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.294+0000 7f11b27fc700 1 -- 172.21.15.36:0/3270111098 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f11bc00f070 con 0x7f11c4143080 2022-06-17T12:56:57.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.294+0000 7f11b27fc700 1 --2- 172.21.15.36:0/3270111098 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f11a4053af0 0x7f11a4055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:57.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.294+0000 7f11c92bd700 1 --2- 172.21.15.36:0/3270111098 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f11a4053af0 0x7f11a4055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:57.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.294+0000 7f11b27fc700 1 -- 172.21.15.36:0/3270111098 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f11bc06d2e0 con 0x7f11c4143080 2022-06-17T12:56:57.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.298+0000 7f11c92bd700 1 --2- 172.21.15.36:0/3270111098 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f11a4053af0 0x7f11a4055fb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f11c000d000 tx=0x7f11c0009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:57.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.298+0000 7f11b27fc700 1 -- 172.21.15.36:0/3270111098 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f11c4141e10 con 0x7f11c4143080 2022-06-17T12:56:57.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.426+0000 7f11cb521700 1 -- 172.21.15.36:0/3270111098 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f11c4141e10 con 0x7f11c4143080 2022-06-17T12:56:57.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.426+0000 7f11b27fc700 1 -- 172.21.15.36:0/3270111098 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v141) v1 ==== 100+0+10 (secure 0 0 0) 0x7f11c4141e10 con 0x7f11c4143080 2022-06-17T12:56:57.435 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:56:57.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.426+0000 7f11cb521700 1 -- 172.21.15.36:0/3270111098 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f11a4053af0 msgr2=0x7f11a4055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:57.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.426+0000 7f11cb521700 1 --2- 172.21.15.36:0/3270111098 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f11a4053af0 0x7f11a4055fb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f11c000d000 tx=0x7f11c0009040).stop 2022-06-17T12:56:57.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.426+0000 7f11cb521700 1 -- 172.21.15.36:0/3270111098 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11c4143080 msgr2=0x7f11c4142500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:57.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.426+0000 7f11cb521700 1 --2- 172.21.15.36:0/3270111098 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11c4143080 0x7f11c4142500 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7f11bc0032f0 tx=0x7f11bc016000).stop 2022-06-17T12:56:57.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.426+0000 7f11cb521700 1 -- 172.21.15.36:0/3270111098 shutdown_connections 2022-06-17T12:56:57.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.426+0000 7f11cb521700 1 --2- 172.21.15.36:0/3270111098 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f11a4053af0 0x7f11a4055fb0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:57.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.426+0000 7f11cb521700 1 --2- 172.21.15.36:0/3270111098 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f11c4143080 0x7f11c4142500 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:57.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.426+0000 7f11cb521700 1 --2- 172.21.15.36:0/3270111098 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11c40b7790 0x7f11c40658e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:57.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.426+0000 7f11cb521700 1 --2- 172.21.15.36:0/3270111098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11c40b3030 0x7f11c4147b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:57.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.426+0000 7f11cb521700 1 -- 172.21.15.36:0/3270111098 >> 172.21.15.36:0/3270111098 conn(0x7f11c4060990 msgr2=0x7f11c41490d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:57.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.426+0000 7f11cb521700 1 -- 172.21.15.36:0/3270111098 shutdown_connections 2022-06-17T12:56:57.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.426+0000 7f11cb521700 1 -- 172.21.15.36:0/3270111098 wait complete. 2022-06-17T12:56:57.486 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:56:57.488 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:56:57.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.626+0000 7f2101848700 1 Processor -- start 2022-06-17T12:56:57.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.626+0000 7f2101848700 1 -- start start 2022-06-17T12:56:57.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.626+0000 7f2101848700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f20fc062b50 0x7f20fc0ada40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:57.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.626+0000 7f2101848700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20fc0b0ba0 con 0x7f20fc144c00 2022-06-17T12:56:57.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.626+0000 7f2101848700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20fc147e90 con 0x7f20fc062770 2022-06-17T12:56:57.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.626+0000 7f2101848700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20fc0ae820 con 0x7f20fc062b50 2022-06-17T12:56:57.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.626+0000 7f20fa7fc700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f20fc144c00 0x7f20fc0ad330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57810/0 (socket says 172.21.15.36:57810) 2022-06-17T12:56:57.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.626+0000 7f20fa7fc700 1 -- 172.21.15.36:0/1678284142 learned_addr learned my addr 172.21.15.36:0/1678284142 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:57.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.626+0000 7f20fb7fe700 1 --2- 172.21.15.36:0/1678284142 >> v2:172.21.15.70:3300/0 conn(0x7f20fc062b50 0x7f20fc0ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:57.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.626+0000 7f20f9ffb700 1 -- 172.21.15.36:0/1678284142 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2782233401 0 0) 0x7f20fc0b0ba0 con 0x7f20fc144c00 2022-06-17T12:56:57.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.626+0000 7f20f9ffb700 1 -- 172.21.15.36:0/1678284142 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20e0003260 con 0x7f20fc144c00 2022-06-17T12:56:57.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.626+0000 7f20f9ffb700 1 -- 172.21.15.36:0/1678284142 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724014138 0 0) 0x7f20fc147e90 con 0x7f20fc062770 2022-06-17T12:56:57.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.626+0000 7f20f9ffb700 1 -- 172.21.15.36:0/1678284142 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20fc0b0ba0 con 0x7f20fc062770 2022-06-17T12:56:57.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.626+0000 7f20fb7fe700 1 -- 172.21.15.36:0/1678284142 >> v1:172.21.15.70:6789/0 conn(0x7f20fc062770 legacy=0x7f20fc1444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:57.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.626+0000 7f20fb7fe700 1 -- 172.21.15.36:0/1678284142 >> v1:172.21.15.36:6789/0 conn(0x7f20fc144c00 legacy=0x7f20fc0ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:57.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.626+0000 7f20fb7fe700 1 -- 172.21.15.36:0/1678284142 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc0b0dd0 con 0x7f20fc062b50 2022-06-17T12:56:57.647 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.626+0000 7f20fb7fe700 1 --2- 172.21.15.36:0/1678284142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20fc062b50 0x7f20fc0ada40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f20ec002f70 tx=0x7f20ec006df0).ready entity=mon.1 client_cookie=e92931977a4c9d98 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:57.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f20f9ffb700 1 -- 172.21.15.36:0/1678284142 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20ec00d540 con 0x7f20fc062b50 2022-06-17T12:56:57.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f20f9ffb700 1 -- 172.21.15.36:0/1678284142 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ec000e90 con 0x7f20fc062b50 2022-06-17T12:56:57.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f20f9ffb700 1 -- 172.21.15.36:0/1678284142 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20ec00d890 con 0x7f20fc062b50 2022-06-17T12:56:57.648 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f2101848700 1 -- 172.21.15.36:0/1678284142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20fc062b50 msgr2=0x7f20fc0ada40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:57.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f2101848700 1 --2- 172.21.15.36:0/1678284142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20fc062b50 0x7f20fc0ada40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f20ec002f70 tx=0x7f20ec006df0).stop 2022-06-17T12:56:57.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f2101848700 1 -- 172.21.15.36:0/1678284142 shutdown_connections 2022-06-17T12:56:57.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f2101848700 1 --2- 172.21.15.36:0/1678284142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20fc062b50 0x7f20fc0ada40 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:57.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f2101848700 1 -- 172.21.15.36:0/1678284142 >> 172.21.15.36:0/1678284142 conn(0x7f20fc0b53e0 msgr2=0x7f20fc0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:57.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f2101848700 1 -- 172.21.15.36:0/1678284142 shutdown_connections 2022-06-17T12:56:57.653 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f2101848700 1 -- 172.21.15.36:0/1678284142 wait complete. 2022-06-17T12:56:57.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f2101848700 1 Processor -- start 2022-06-17T12:56:57.654 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f2101848700 1 -- start start 2022-06-17T12:56:57.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f2101848700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20fc062770 0x7f20fc1314d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:57.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f2101848700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f20fc062b50 0x7f20fc05fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:57.655 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f2101848700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f20fc144c00 0x7f20fc063650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:57.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f2101848700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc066200 con 0x7f20fc062b50 2022-06-17T12:56:57.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f2101848700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc1443a0 con 0x7f20fc062770 2022-06-17T12:56:57.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f2101848700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc15ab80 con 0x7f20fc144c00 2022-06-17T12:56:57.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f20fa7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f20fc062b50 0x7f20fc05fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:57.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f20fa7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f20fc062b50 0x7f20fc05fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40286/0 (socket says 172.21.15.36:40286) 2022-06-17T12:56:57.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f20fa7fc700 1 -- 172.21.15.36:0/729101317 learned_addr learned my addr 172.21.15.36:0/729101317 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:57.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f20faffd700 1 --2- 172.21.15.36:0/729101317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20fc062770 0x7f20fc1314d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:57.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f20fb7fe700 1 --2- 172.21.15.36:0/729101317 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f20fc144c00 0x7f20fc063650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:57.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f20fa7fc700 1 -- 172.21.15.36:0/729101317 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f20fc144c00 msgr2=0x7f20fc063650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:57.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f20fa7fc700 1 --2- 172.21.15.36:0/729101317 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f20fc144c00 0x7f20fc063650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:57.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.630+0000 7f20fa7fc700 1 -- 172.21.15.36:0/729101317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20fc062770 msgr2=0x7f20fc1314d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:57.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.634+0000 7f20fa7fc700 1 --2- 172.21.15.36:0/729101317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20fc062770 0x7f20fc1314d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:57.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.634+0000 7f20fa7fc700 1 -- 172.21.15.36:0/729101317 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec00b180 con 0x7f20fc062b50 2022-06-17T12:56:57.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.634+0000 7f20fa7fc700 1 --2- 172.21.15.36:0/729101317 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f20fc062b50 0x7f20fc05fc80 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7f20f00011d0 tx=0x7f20f0007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:57.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.634+0000 7f20ebfff700 1 -- 172.21.15.36:0/729101317 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20f000ce70 con 0x7f20fc062b50 2022-06-17T12:56:57.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.634+0000 7f20ebfff700 1 -- 172.21.15.36:0/729101317 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f00070d0 con 0x7f20fc062b50 2022-06-17T12:56:57.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.634+0000 7f20ebfff700 1 -- 172.21.15.36:0/729101317 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20f0007430 con 0x7f20fc062b50 2022-06-17T12:56:57.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.634+0000 7f2101848700 1 -- 172.21.15.36:0/729101317 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20fc060170 con 0x7f20fc062b50 2022-06-17T12:56:57.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.634+0000 7f2101848700 1 -- 172.21.15.36:0/729101317 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20fc063370 con 0x7f20fc062b50 2022-06-17T12:56:57.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.634+0000 7f2101848700 1 -- 172.21.15.36:0/729101317 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20fc0b1c20 con 0x7f20fc062b50 2022-06-17T12:56:57.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.634+0000 7f20ebfff700 1 -- 172.21.15.36:0/729101317 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f20f0010070 con 0x7f20fc062b50 2022-06-17T12:56:57.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.634+0000 7f20ebfff700 1 --2- 172.21.15.36:0/729101317 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f20d8053920 0x7f20d8055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:57.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.638+0000 7f20ebfff700 1 -- 172.21.15.36:0/729101317 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f20f006b670 con 0x7f20fc062b50 2022-06-17T12:56:57.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.638+0000 7f20faffd700 1 --2- 172.21.15.36:0/729101317 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f20d8053920 0x7f20d8055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:57.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.638+0000 7f20faffd700 1 --2- 172.21.15.36:0/729101317 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f20d8053920 0x7f20d8055de0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f20f400d000 tx=0x7f20f4007da0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:57.671 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.642+0000 7f20ebfff700 1 -- 172.21.15.36:0/729101317 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f20f00422e0 con 0x7f20fc062b50 2022-06-17T12:56:57.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.806+0000 7f2101848700 1 -- 172.21.15.36:0/729101317 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20fc057410 con 0x7f20fc062b50 2022-06-17T12:56:57.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.806+0000 7f20ebfff700 1 -- 172.21.15.36:0/729101317 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f20f0018020 con 0x7f20fc062b50 2022-06-17T12:56:57.807 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:56:57.807 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":78,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1081450840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1081450840},{"type":"v1","addr":"172.21.15.70:6833","nonce":1081450840}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":76}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:56:49.200774+0000","modified":"2022-06-17T12:56:50.204509+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8478":{"gid":8478,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/3169088536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3169088536},{"type":"v1","addr":"172.21.15.36:6835","nonce":3169088536}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-06-17T12:56:57.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.806+0000 7f2101848700 1 -- 172.21.15.36:0/729101317 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f20d8053920 msgr2=0x7f20d8055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:57.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.806+0000 7f2101848700 1 --2- 172.21.15.36:0/729101317 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f20d8053920 0x7f20d8055de0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f20f400d000 tx=0x7f20f4007da0).stop 2022-06-17T12:56:57.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.806+0000 7f2101848700 1 -- 172.21.15.36:0/729101317 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f20fc062b50 msgr2=0x7f20fc05fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:57.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.806+0000 7f2101848700 1 --2- 172.21.15.36:0/729101317 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f20fc062b50 0x7f20fc05fc80 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7f20f00011d0 tx=0x7f20f0007b50).stop 2022-06-17T12:56:57.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.806+0000 7f2101848700 1 -- 172.21.15.36:0/729101317 shutdown_connections 2022-06-17T12:56:57.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.806+0000 7f2101848700 1 --2- 172.21.15.36:0/729101317 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f20d8053920 0x7f20d8055de0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:57.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.806+0000 7f2101848700 1 --2- 172.21.15.36:0/729101317 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f20fc144c00 0x7f20fc063650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:57.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.806+0000 7f2101848700 1 --2- 172.21.15.36:0/729101317 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f20fc062b50 0x7f20fc05fc80 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:57.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.806+0000 7f2101848700 1 --2- 172.21.15.36:0/729101317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20fc062770 0x7f20fc1314d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:57.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.806+0000 7f2101848700 1 -- 172.21.15.36:0/729101317 >> 172.21.15.36:0/729101317 conn(0x7f20fc0b53e0 msgr2=0x7f20fc0ad330 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:57.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.810+0000 7f2101848700 1 -- 172.21.15.36:0/729101317 shutdown_connections 2022-06-17T12:56:57.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:57.810+0000 7f2101848700 1 -- 172.21.15.36:0/729101317 wait complete. 2022-06-17T12:56:57.822 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 78 2022-06-17T12:56:58.034 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 78, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:56:49.200774+0000', 'modified': '2022-06-17T12:56:50.204509+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8478}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8478': {'gid': 8478, 'name': 'a', 'rank': 0, 'incarnation': 77, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.36:6835/3169088536', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 3169088536}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 3169088536}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:56:58.035 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:56:58.036 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:56:58.168 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":8478,"want_state":"up:active","state":"up:active","rank_uptime":8.9590620259999998,"mdsmap_epoch":78,"osdmap_epoch":141,"osdmap_epoch_barrier":140,"uptime":14.032260393} 2022-06-17T12:56:58.179 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 8478, "mdsmap_epoch": 78, "osdmap_epoch": 141, "osdmap_epoch_barrier": 140, "rank_uptime": 8.959062026, "state": "up:active", "uptime": 14.032260393, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:56:58.188 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:56:58.189 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-06-17T12:56:58.189 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi136.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-06-17T12:56:58.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:58.191 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:58.213 DEBUG:teuthology.orchestra.run.smithi136:> sudo modprobe fuse 2022-06-17T12:56:58.258 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:56:58.259 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:56:58.341 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:56:58.344 INFO:teuthology.orchestra.run.smithi136.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:56:58.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:58.346 DEBUG:teuthology.orchestra.run.smithi136:> ls /sys/fs/fuse/connections 2022-06-17T12:56:58.373 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:56:58.374 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:58.413 DEBUG:teuthology.orchestra.run.smithi136:> sudo modprobe fuse 2022-06-17T12:56:58.483 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:56:58.488 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:56:58.490 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:2022-06-17T12:56:58.497+0000 7f279abc5080 -1 init, newargv = 0x55c280b1e2b0 newargc=9ceph-fuse[22488]: starting ceph client 2022-06-17T12:56:58.491 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr: 2022-06-17T12:56:58.503 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:ceph-fuse[22488]: starting fuse 2022-06-17T12:56:58.546 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:56:58.548 INFO:teuthology.orchestra.run.smithi136.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:56:58.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:58.550 DEBUG:teuthology.orchestra.run.smithi136:> ls /sys/fs/fuse/connections 2022-06-17T12:56:58.624 INFO:teuthology.orchestra.run.smithi136.stdout:52 2022-06-17T12:56:58.625 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:56:58.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:58.627 DEBUG:teuthology.orchestra.run.smithi136:> sudo python3 -c ' 2022-06-17T12:56:58.627 DEBUG:teuthology.orchestra.run.smithi136:> import glob 2022-06-17T12:56:58.628 DEBUG:teuthology.orchestra.run.smithi136:> import re 2022-06-17T12:56:58.628 DEBUG:teuthology.orchestra.run.smithi136:> import os 2022-06-17T12:56:58.629 DEBUG:teuthology.orchestra.run.smithi136:> import subprocess 2022-06-17T12:56:58.632 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:56:58.632 DEBUG:teuthology.orchestra.run.smithi136:> def find_socket(client_name): 2022-06-17T12:56:58.633 DEBUG:teuthology.orchestra.run.smithi136:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-06-17T12:56:58.633 DEBUG:teuthology.orchestra.run.smithi136:> files = glob.glob(asok_path) 2022-06-17T12:56:58.634 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:56:58.647 DEBUG:teuthology.orchestra.run.smithi136:> # Given a non-glob path, it better be there 2022-06-17T12:56:58.647 DEBUG:teuthology.orchestra.run.smithi136:> if "*" not in asok_path: 2022-06-17T12:56:58.648 DEBUG:teuthology.orchestra.run.smithi136:> assert(len(files) == 1) 2022-06-17T12:56:58.649 DEBUG:teuthology.orchestra.run.smithi136:> return files[0] 2022-06-17T12:56:58.649 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:56:58.655 DEBUG:teuthology.orchestra.run.smithi136:> for f in files: 2022-06-17T12:56:58.655 DEBUG:teuthology.orchestra.run.smithi136:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:56:58.656 DEBUG:teuthology.orchestra.run.smithi136:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:56:58.662 DEBUG:teuthology.orchestra.run.smithi136:> return f 2022-06-17T12:56:58.662 DEBUG:teuthology.orchestra.run.smithi136:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:56:58.663 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:56:58.663 DEBUG:teuthology.orchestra.run.smithi136:> print(find_socket("client.1")) 2022-06-17T12:56:58.664 DEBUG:teuthology.orchestra.run.smithi136:> ' 2022-06-17T12:56:58.717 INFO:teuthology.orchestra.run.smithi136.stdout:/var/run/ceph/ceph-client.1.22488.asok 2022-06-17T12:56:58.721 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22488.asok 2022-06-17T12:56:58.722 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:58.723 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22488.asok status 2022-06-17T12:56:58.866 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2022-06-17T12:56:58.866 INFO:teuthology.orchestra.run.smithi136.stdout: "metadata": { 2022-06-17T12:56:58.867 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:56:58.868 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:56:58.868 INFO:teuthology.orchestra.run.smithi136.stdout: "entity_id": "1", 2022-06-17T12:56:58.869 INFO:teuthology.orchestra.run.smithi136.stdout: "hostname": "smithi136", 2022-06-17T12:56:58.869 INFO:teuthology.orchestra.run.smithi136.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-06-17T12:56:58.871 INFO:teuthology.orchestra.run.smithi136.stdout: "pid": "22488", 2022-06-17T12:56:58.872 INFO:teuthology.orchestra.run.smithi136.stdout: "root": "/" 2022-06-17T12:56:58.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:56:58.873 INFO:teuthology.orchestra.run.smithi136.stdout: "dentry_count": 0, 2022-06-17T12:56:58.874 INFO:teuthology.orchestra.run.smithi136.stdout: "dentry_pinned_count": 0, 2022-06-17T12:56:58.874 INFO:teuthology.orchestra.run.smithi136.stdout: "id": 7355, 2022-06-17T12:56:58.875 INFO:teuthology.orchestra.run.smithi136.stdout: "inst": { 2022-06-17T12:56:58.875 INFO:teuthology.orchestra.run.smithi136.stdout: "name": { 2022-06-17T12:56:58.876 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "client", 2022-06-17T12:56:58.876 INFO:teuthology.orchestra.run.smithi136.stdout: "num": 7355 2022-06-17T12:56:58.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:56:58.878 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": { 2022-06-17T12:56:58.878 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "v1", 2022-06-17T12:56:58.878 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": "172.21.15.136:0", 2022-06-17T12:56:58.879 INFO:teuthology.orchestra.run.smithi136.stdout: "nonce": 3865758298 2022-06-17T12:56:58.879 INFO:teuthology.orchestra.run.smithi136.stdout: } 2022-06-17T12:56:58.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:56:58.888 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": { 2022-06-17T12:56:58.888 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "v1", 2022-06-17T12:56:58.889 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": "172.21.15.136:0", 2022-06-17T12:56:58.889 INFO:teuthology.orchestra.run.smithi136.stdout: "nonce": 3865758298 2022-06-17T12:56:58.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:56:58.891 INFO:teuthology.orchestra.run.smithi136.stdout: "inst_str": "client.7355 172.21.15.136:0/3865758298", 2022-06-17T12:56:58.893 INFO:teuthology.orchestra.run.smithi136.stdout: "addr_str": "172.21.15.136:0/3865758298", 2022-06-17T12:56:58.894 INFO:teuthology.orchestra.run.smithi136.stdout: "inode_count": 1, 2022-06-17T12:56:58.894 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_epoch": 78, 2022-06-17T12:56:58.895 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_epoch": 141, 2022-06-17T12:56:58.895 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:56:58.898 INFO:teuthology.orchestra.run.smithi136.stdout: "blacklisted": false, 2022-06-17T12:56:58.898 INFO:teuthology.orchestra.run.smithi136.stdout: "fs_name": "cephfs" 2022-06-17T12:56:58.899 INFO:teuthology.orchestra.run.smithi136.stdout:} 2022-06-17T12:56:58.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:58.909 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:58.910 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:58.983 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:56:58.984 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:56:58.984 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:58.985 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:59.057 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-06-17T12:56:59.058 DEBUG:teuthology.orchestra.run.smithi107:> cd /home/ubuntu/cephtest/mnt.0 && sudo getfattr --only-values -n ceph.dir.rctime . 2022-06-17T12:56:59.125 INFO:teuthology.orchestra.run.smithi107.stdout:1655470619.063656310 2022-06-17T12:56:59.126 INFO:tasks.cephfs.test_misc:rctime = 1655470619.063656310 2022-06-17T12:56:59.127 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-17T12:56:59.128 DEBUG:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:56:59.167 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:56:59.201 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:56:59.223 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:56:59.244 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:56:59.275 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-17T12:56:59.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:59.277 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:59.278 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:56:59.289 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:56:59.290 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:56:59.290 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2022-06-17T12:56:59.291 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:56:59.291 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:56:59.353 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[23004]: fuse finished with error 0 and tester_r 0 2022-06-17T12:56:59.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:59.356 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:59.387 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:56:59.447 INFO:teuthology.orchestra.run.smithi107.stdout:ext2/ext3 2022-06-17T12:56:59.448 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:56:59.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:56:59.458 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:56:59.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:59.494 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:59.495 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:59.519 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:56:59.520 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:56:59.536 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi136.front.sepia.ceph.com... 2022-06-17T12:56:59.537 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:56:59.548 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:59.572 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:ceph-fuse[22488]: fuse finished with error 0 and tester_r 0 2022-06-17T12:56:59.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:59.589 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:59.599 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:59.672 INFO:teuthology.orchestra.run.smithi136.stdout:ext2/ext3 2022-06-17T12:56:59.672 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:56:59.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:56:59.685 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:56:59.752 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:56:59.753 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:56:59.753 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:56:59.776 INFO:teuthology.orchestra.run.smithi143.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:56:59.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:56:59.779 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-06-17T12:56:59.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:56:59.789 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:56:59.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:56:59.812 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:56:59.813 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-17T12:56:59.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.942+0000 7f38654ac700 1 Processor -- start 2022-06-17T12:56:59.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.942+0000 7f38654ac700 1 -- start start 2022-06-17T12:56:59.949 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.946+0000 7f38654ac700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f38600ae360 0x7f3860060260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:59.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.946+0000 7f38654ac700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38600633c0 con 0x7f38600b1db0 2022-06-17T12:56:59.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.946+0000 7f38654ac700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3860143330 con 0x7f386005fe80 2022-06-17T12:56:59.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.946+0000 7f38654ac700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3860063100 con 0x7f38600ae360 2022-06-17T12:56:59.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.946+0000 7f385f7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f38600ae360 0x7f3860060260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:59.950 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.946+0000 7f3864cab700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f38600b1db0 0x7f38601437d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57826/0 (socket says 172.21.15.36:57826) 2022-06-17T12:56:59.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.946+0000 7f3864cab700 1 -- 172.21.15.36:0/1561106205 learned_addr learned my addr 172.21.15.36:0/1561106205 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:56:59.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.946+0000 7f385f7fe700 1 -- 172.21.15.36:0/1561106205 >> v1:172.21.15.70:6789/0 conn(0x7f386005fe80 legacy=0x7f38601444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:59.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.946+0000 7f385f7fe700 1 -- 172.21.15.36:0/1561106205 >> v1:172.21.15.36:6789/0 conn(0x7f38600b1db0 legacy=0x7f38601437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:59.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.946+0000 7f385f7fe700 1 -- 172.21.15.36:0/1561106205 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3860143020 con 0x7f38600ae360 2022-06-17T12:56:59.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.946+0000 7f385f7fe700 1 --2- 172.21.15.36:0/1561106205 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f38600ae360 0x7f3860060260 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 rx=0x7f3848002f70 tx=0x7f3848006df0).ready entity=mon.0 client_cookie=b7c3a923be1f1a18 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:59.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.946+0000 7f385effd700 1 -- 172.21.15.36:0/1561106205 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f384800d610 con 0x7f38600ae360 2022-06-17T12:56:59.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.946+0000 7f385effd700 1 -- 172.21.15.36:0/1561106205 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3848000e90 con 0x7f38600ae360 2022-06-17T12:56:59.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.946+0000 7f385effd700 1 -- 172.21.15.36:0/1561106205 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f384800d960 con 0x7f38600ae360 2022-06-17T12:56:59.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.946+0000 7f38654ac700 1 -- 172.21.15.36:0/1561106205 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f38600ae360 msgr2=0x7f3860060260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:59.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.946+0000 7f38654ac700 1 --2- 172.21.15.36:0/1561106205 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f38600ae360 0x7f3860060260 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 rx=0x7f3848002f70 tx=0x7f3848006df0).stop 2022-06-17T12:56:59.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.946+0000 7f38654ac700 1 -- 172.21.15.36:0/1561106205 shutdown_connections 2022-06-17T12:56:59.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.946+0000 7f38654ac700 1 --2- 172.21.15.36:0/1561106205 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f38600ae360 0x7f3860060260 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:59.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.946+0000 7f38654ac700 1 -- 172.21.15.36:0/1561106205 >> 172.21.15.36:0/1561106205 conn(0x7f3860060990 msgr2=0x7f386005f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:56:59.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.946+0000 7f38654ac700 1 -- 172.21.15.36:0/1561106205 shutdown_connections 2022-06-17T12:56:59.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.946+0000 7f38654ac700 1 -- 172.21.15.36:0/1561106205 wait complete. 2022-06-17T12:56:59.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f38654ac700 1 Processor -- start 2022-06-17T12:56:59.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f38654ac700 1 -- start start 2022-06-17T12:56:59.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f38654ac700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f386005fe80 0x7f38601415d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:59.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f38654ac700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38600ae360 0x7f386013ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:59.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f38654ac700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38600b1db0 0x7f3860064790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:59.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f38654ac700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3860142620 con 0x7f386005fe80 2022-06-17T12:56:59.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f38654ac700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3860155c60 con 0x7f38600ae360 2022-06-17T12:56:59.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f38654ac700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3860140900 con 0x7f38600b1db0 2022-06-17T12:56:59.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f385ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f386005fe80 0x7f38601415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:59.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f385ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f386005fe80 0x7f38601415d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40298/0 (socket says 172.21.15.36:40298) 2022-06-17T12:56:59.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f385ffff700 1 -- 172.21.15.36:0/3321177537 learned_addr learned my addr 172.21.15.36:0/3321177537 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:56:59.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f3864cab700 1 --2- 172.21.15.36:0/3321177537 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38600b1db0 0x7f3860064790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:59.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f385f7fe700 1 --2- 172.21.15.36:0/3321177537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38600ae360 0x7f386013ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:59.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f385ffff700 1 -- 172.21.15.36:0/3321177537 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38600b1db0 msgr2=0x7f3860064790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:59.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f385ffff700 1 --2- 172.21.15.36:0/3321177537 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38600b1db0 0x7f3860064790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:59.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f385ffff700 1 -- 172.21.15.36:0/3321177537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38600ae360 msgr2=0x7f386013ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:56:59.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f385ffff700 1 --2- 172.21.15.36:0/3321177537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38600ae360 0x7f386013ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:56:59.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f385ffff700 1 -- 172.21.15.36:0/3321177537 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384800b180 con 0x7f386005fe80 2022-06-17T12:56:59.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f385ffff700 1 --2- 172.21.15.36:0/3321177537 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f386005fe80 0x7f38601415d0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7f3860140a90 tx=0x7f3850007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:59.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f385d7fa700 1 -- 172.21.15.36:0/3321177537 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3850007030 con 0x7f386005fe80 2022-06-17T12:56:59.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f385d7fa700 1 -- 172.21.15.36:0/3321177537 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3850007190 con 0x7f386005fe80 2022-06-17T12:56:59.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f38654ac700 1 -- 172.21.15.36:0/3321177537 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3860140660 con 0x7f386005fe80 2022-06-17T12:56:59.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f385d7fa700 1 -- 172.21.15.36:0/3321177537 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3850006510 con 0x7f386005fe80 2022-06-17T12:56:59.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f38654ac700 1 -- 172.21.15.36:0/3321177537 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386013b160 con 0x7f386005fe80 2022-06-17T12:56:59.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.950+0000 7f38654ac700 1 -- 172.21.15.36:0/3321177537 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3860143ee0 con 0x7f386005fe80 2022-06-17T12:56:59.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.954+0000 7f385d7fa700 1 -- 172.21.15.36:0/3321177537 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f3850010030 con 0x7f386005fe80 2022-06-17T12:56:59.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.954+0000 7f385d7fa700 1 --2- 172.21.15.36:0/3321177537 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3844053a40 0x7f3844055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:56:59.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.954+0000 7f385d7fa700 1 -- 172.21.15.36:0/3321177537 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f385006bb00 con 0x7f386005fe80 2022-06-17T12:56:59.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.954+0000 7f385f7fe700 1 --2- 172.21.15.36:0/3321177537 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3844053a40 0x7f3844055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:56:59.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.954+0000 7f385f7fe700 1 --2- 172.21.15.36:0/3321177537 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3844053a40 0x7f3844055f00 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f384801c000 tx=0x7f3848015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:56:59.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:56:59.958+0000 7f385d7fa700 1 -- 172.21.15.36:0/3321177537 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3850042900 con 0x7f386005fe80 2022-06-17T12:57:00.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:00.086+0000 7f38654ac700 1 -- 172.21.15.36:0/3321177537 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f3860057410 con 0x7f386005fe80 2022-06-17T12:57:00.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:00.086+0000 7f385d7fa700 1 -- 172.21.15.36:0/3321177537 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v141) v1 ==== 81+0+34051 (secure 0 0 0) 0x7f3850045f20 con 0x7f386005fe80 2022-06-17T12:57:00.072 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:57:00.073 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:57:00.074 INFO:teuthology.orchestra.run.smithi036.stdout: "epoch": 141, 2022-06-17T12:57:00.074 INFO:teuthology.orchestra.run.smithi036.stdout: "fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", 2022-06-17T12:57:00.075 INFO:teuthology.orchestra.run.smithi036.stdout: "created": "2022-06-17T12:46:30.671408+0000", 2022-06-17T12:57:00.075 INFO:teuthology.orchestra.run.smithi036.stdout: "modified": "2022-06-17T12:56:50.204603+0000", 2022-06-17T12:57:00.076 INFO:teuthology.orchestra.run.smithi036.stdout: "last_up_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:57:00.076 INFO:teuthology.orchestra.run.smithi036.stdout: "last_in_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:57:00.077 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-17T12:57:00.077 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_num": 5799936, 2022-06-17T12:57:00.078 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_set": [ 2022-06-17T12:57:00.078 INFO:teuthology.orchestra.run.smithi036.stdout: "pglog_hardlimit", 2022-06-17T12:57:00.079 INFO:teuthology.orchestra.run.smithi036.stdout: "purged_snapdirs", 2022-06-17T12:57:00.079 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_deletes", 2022-06-17T12:57:00.079 INFO:teuthology.orchestra.run.smithi036.stdout: "sortbitwise" 2022-06-17T12:57:00.081 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:57:00.081 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_version": 6, 2022-06-17T12:57:00.081 INFO:teuthology.orchestra.run.smithi036.stdout: "full_ratio": 0.94999998807907104, 2022-06-17T12:57:00.082 INFO:teuthology.orchestra.run.smithi036.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-17T12:57:00.082 INFO:teuthology.orchestra.run.smithi036.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-17T12:57:00.092 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_snapshot": "", 2022-06-17T12:57:00.092 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_max": 28, 2022-06-17T12:57:00.110 INFO:teuthology.orchestra.run.smithi036.stdout: "max_osd": 8, 2022-06-17T12:57:00.122 INFO:teuthology.orchestra.run.smithi036.stdout: "require_min_compat_client": "jewel", 2022-06-17T12:57:00.123 INFO:teuthology.orchestra.run.smithi036.stdout: "min_compat_client": "jewel", 2022-06-17T12:57:00.124 INFO:teuthology.orchestra.run.smithi036.stdout: "require_osd_release": "octopus", 2022-06-17T12:57:00.125 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:57:00.125 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.133 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 1, 2022-06-17T12:57:00.141 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "device_health_metrics", 2022-06-17T12:57:00.142 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:33.881212+0000", 2022-06-17T12:57:00.142 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:57:00.146 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:57:00.146 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:57:00.147 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:57:00.148 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:57:00.148 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:57:00.158 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:57:00.159 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:57:00.159 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 1, 2022-06-17T12:57:00.160 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 1, 2022-06-17T12:57:00.160 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 1, 2022-06-17T12:57:00.160 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 1, 2022-06-17T12:57:00.161 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 1, 2022-06-17T12:57:00.163 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:57:00.164 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:57:00.164 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:57:00.164 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:57:00.165 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:57:00.170 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:57:00.170 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:57:00.171 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.171 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "16", 2022-06-17T12:57:00.172 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:57:00.172 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:57:00.173 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:57:00.173 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:57:00.202 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:57:00.203 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:57:00.203 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:57:00.203 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:57:00.204 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:57:00.204 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:57:00.205 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:57:00.205 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:57:00.205 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:57:00.226 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:57:00.226 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:57:00.227 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:57:00.228 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:57:00.228 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:57:00.229 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:57:00.229 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:57:00.230 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:57:00.230 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:57:00.231 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:57:00.231 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:57:00.232 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:57:00.232 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:57:00.232 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.233 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:57:00.233 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:57:00.234 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:57:00.235 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:57:00.236 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:57:00.236 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:57:00.237 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:57:00.237 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:57:00.237 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:57:00.238 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:57:00.239 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:57:00.239 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:57:00.240 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 1 2022-06-17T12:57:00.240 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.242 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:57:00.249 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr_devicehealth": {} 2022-06-17T12:57:00.250 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.282 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.283 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.293 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 2, 2022-06-17T12:57:00.294 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "rbd", 2022-06-17T12:57:00.294 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:44.485460+0000", 2022-06-17T12:57:00.295 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:57:00.295 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:57:00.296 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:57:00.296 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:57:00.296 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:57:00.297 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:57:00.297 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:57:00.298 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:57:00.299 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:57:00.299 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:57:00.299 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:57:00.309 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:57:00.309 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:57:00.310 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:57:00.310 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:57:00.311 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:57:00.311 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:57:00.312 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:57:00.312 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:57:00.313 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:57:00.313 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.314 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "19", 2022-06-17T12:57:00.314 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:57:00.315 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:57:00.315 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:57:00.316 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:57:00.316 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:57:00.334 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:57:00.335 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:57:00.335 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:57:00.336 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:57:00.336 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:57:00.336 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:57:00.337 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:57:00.337 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:57:00.338 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:57:00.339 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:57:00.339 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:57:00.340 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:57:00.341 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:57:00.341 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:57:00.341 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:57:00.342 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:57:00.342 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:57:00.343 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:57:00.344 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:57:00.344 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:57:00.345 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:57:00.345 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.346 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:57:00.346 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:57:00.346 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:57:00.347 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:57:00.347 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:57:00.348 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:57:00.348 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:57:00.349 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:57:00.349 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:57:00.350 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:57:00.350 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:57:00.351 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:57:00.351 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:57:00.351 INFO:teuthology.orchestra.run.smithi036.stdout: "rbd": {} 2022-06-17T12:57:00.352 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.352 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.353 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.353 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 27, 2022-06-17T12:57:00.355 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_metadata", 2022-06-17T12:57:00.355 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:56:47.177576+0000", 2022-06-17T12:57:00.356 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:57:00.356 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:57:00.356 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:57:00.357 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:57:00.357 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:57:00.358 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:57:00.358 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:57:00.359 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:57:00.359 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:57:00.360 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:57:00.360 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:57:00.361 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:57:00.361 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:57:00.362 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:57:00.362 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:57:00.363 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:57:00.363 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:57:00.363 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:57:00.364 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:57:00.364 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:57:00.365 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.365 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "140", 2022-06-17T12:57:00.366 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:57:00.366 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:57:00.367 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:57:00.367 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:57:00.368 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:57:00.368 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:57:00.368 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:57:00.369 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:57:00.369 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:57:00.371 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:57:00.372 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:57:00.372 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:57:00.373 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:57:00.373 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:57:00.373 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:57:00.374 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:57:00.374 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:57:00.375 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:57:00.375 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:57:00.376 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:57:00.377 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:57:00.377 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:57:00.378 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:57:00.378 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:57:00.389 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:57:00.389 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:57:00.390 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.390 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:57:00.391 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:57:00.391 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:57:00.391 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:57:00.392 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:57:00.392 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:57:00.392 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:57:00.393 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:57:00.393 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:57:00.394 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:57:00.394 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:57:00.395 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:57:00.395 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_bias": 4, 2022-06-17T12:57:00.396 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 16, 2022-06-17T12:57:00.396 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_priority": 5 2022-06-17T12:57:00.397 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.397 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:57:00.397 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:57:00.398 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": "cephfs" 2022-06-17T12:57:00.399 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.399 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.399 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.400 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.400 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 28, 2022-06-17T12:57:00.401 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_data", 2022-06-17T12:57:00.401 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:56:48.257688+0000", 2022-06-17T12:57:00.401 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:57:00.402 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:57:00.402 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:57:00.403 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:57:00.403 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:57:00.403 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:57:00.404 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:57:00.404 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:57:00.405 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:57:00.405 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:57:00.406 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:57:00.406 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:57:00.406 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:57:00.407 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:57:00.410 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:57:00.410 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:57:00.411 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:57:00.411 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:57:00.412 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:57:00.412 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:57:00.412 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.416 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "141", 2022-06-17T12:57:00.416 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:57:00.416 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:57:00.416 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:57:00.417 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:57:00.417 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:57:00.417 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:57:00.417 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:57:00.418 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:57:00.418 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:57:00.418 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:57:00.418 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:57:00.418 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:57:00.419 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:57:00.419 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:57:00.419 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:57:00.419 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:57:00.419 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:57:00.420 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:57:00.420 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:57:00.420 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:57:00.420 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:57:00.421 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:57:00.421 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:57:00.421 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:57:00.421 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:57:00.421 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:57:00.429 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.430 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:57:00.430 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:57:00.430 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:57:00.430 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:57:00.431 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:57:00.431 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:57:00.431 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:57:00.431 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:57:00.432 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:57:00.432 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:57:00.432 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:57:00.432 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:57:00.433 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:57:00.433 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:57:00.433 INFO:teuthology.orchestra.run.smithi036.stdout: "data": "cephfs" 2022-06-17T12:57:00.434 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.434 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.434 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.434 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:57:00.435 INFO:teuthology.orchestra.run.smithi036.stdout: "osds": [ 2022-06-17T12:57:00.435 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.435 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:57:00.435 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", 2022-06-17T12:57:00.436 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:57:00.436 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:57:00.437 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:57:00.437 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:57:00.438 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:57:00.438 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:57:00.439 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:57:00.439 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 139, 2022-06-17T12:57:00.439 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:57:00.440 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:57:00.440 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:57:00.441 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:00.441 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.448 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:00.449 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6802", 2022-06-17T12:57:00.450 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:57:00.450 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.451 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.451 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:00.452 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6803", 2022-06-17T12:57:00.452 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:57:00.452 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.453 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.453 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.453 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:57:00.454 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:00.454 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.454 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:00.455 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6804", 2022-06-17T12:57:00.455 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:57:00.455 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.456 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.456 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:00.456 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6805", 2022-06-17T12:57:00.457 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:57:00.457 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.457 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.458 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.458 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:57:00.459 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:00.459 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.460 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:00.461 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6808", 2022-06-17T12:57:00.461 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:57:00.462 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.462 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.463 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:00.463 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6809", 2022-06-17T12:57:00.463 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:57:00.464 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.464 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.464 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.465 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:57:00.465 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:00.466 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.466 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:00.466 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6806", 2022-06-17T12:57:00.467 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:57:00.467 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.468 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.468 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:00.480 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6807", 2022-06-17T12:57:00.481 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:57:00.481 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.482 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.482 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.483 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6803/22619", 2022-06-17T12:57:00.483 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6805/22619", 2022-06-17T12:57:00.483 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6809/22619", 2022-06-17T12:57:00.484 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6807/22619", 2022-06-17T12:57:00.484 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:57:00.485 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:57:00.485 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:57:00.495 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.498 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.499 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.513 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:57:00.515 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", 2022-06-17T12:57:00.516 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:57:00.516 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:57:00.517 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:57:00.517 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:57:00.527 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:57:00.528 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:57:00.528 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:57:00.528 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 139, 2022-06-17T12:57:00.529 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:57:00.529 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:57:00.530 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:57:00.547 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:00.547 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.548 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:00.548 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6810", 2022-06-17T12:57:00.548 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:57:00.549 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.549 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.569 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:00.570 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6811", 2022-06-17T12:57:00.570 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:57:00.571 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.571 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.572 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.572 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:57:00.572 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:00.573 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.573 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:00.593 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6812", 2022-06-17T12:57:00.594 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:57:00.594 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.594 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.595 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:00.595 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6813", 2022-06-17T12:57:00.596 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:57:00.596 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.597 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.597 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.598 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:57:00.598 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:00.599 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.599 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:00.600 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6816", 2022-06-17T12:57:00.600 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:57:00.601 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.601 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.602 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:00.607 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6817", 2022-06-17T12:57:00.608 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:57:00.608 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.608 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.609 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.609 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:57:00.610 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:00.610 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.611 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:00.611 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6814", 2022-06-17T12:57:00.611 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:57:00.612 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.613 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.614 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:00.614 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6815", 2022-06-17T12:57:00.614 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:57:00.615 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.615 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.616 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.631 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6811/22620", 2022-06-17T12:57:00.631 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6813/22620", 2022-06-17T12:57:00.632 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6817/22620", 2022-06-17T12:57:00.633 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6815/22620", 2022-06-17T12:57:00.633 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:57:00.633 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:57:00.634 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:57:00.634 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.635 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.635 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.636 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:57:00.636 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", 2022-06-17T12:57:00.636 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:57:00.637 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:57:00.637 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:57:00.638 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:57:00.638 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:57:00.638 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:57:00.639 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:57:00.639 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 138, 2022-06-17T12:57:00.640 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:57:00.640 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:57:00.641 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:57:00.641 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:00.641 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.642 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:00.643 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6826", 2022-06-17T12:57:00.643 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:57:00.643 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.644 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.644 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:00.645 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6827", 2022-06-17T12:57:00.645 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:57:00.646 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.646 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.647 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.647 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:57:00.648 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:00.648 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.648 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:00.649 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6828", 2022-06-17T12:57:00.649 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:57:00.650 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.651 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.651 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:00.651 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6829", 2022-06-17T12:57:00.652 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:57:00.652 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.653 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.653 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.654 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:57:00.654 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:00.654 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.655 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:00.655 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6832", 2022-06-17T12:57:00.656 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:57:00.656 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.657 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.657 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:00.657 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6833", 2022-06-17T12:57:00.658 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:57:00.658 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.659 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.659 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.660 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:57:00.673 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:00.674 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.674 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:00.675 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6830", 2022-06-17T12:57:00.675 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:57:00.675 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.676 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.676 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:00.677 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6831", 2022-06-17T12:57:00.677 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:57:00.687 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.688 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.688 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.689 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6827/22643", 2022-06-17T12:57:00.689 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6829/22643", 2022-06-17T12:57:00.689 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6833/22643", 2022-06-17T12:57:00.690 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6831/22643", 2022-06-17T12:57:00.690 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:57:00.691 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:57:00.691 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:57:00.691 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.692 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.692 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.693 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:57:00.693 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", 2022-06-17T12:57:00.694 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:57:00.694 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:57:00.694 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:57:00.695 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:57:00.695 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:57:00.696 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:57:00.696 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:57:00.697 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 128, 2022-06-17T12:57:00.697 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:57:00.697 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:57:00.698 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:57:00.698 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:00.698 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.725 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:00.726 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6818", 2022-06-17T12:57:00.726 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:57:00.727 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.727 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.728 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:00.728 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6819", 2022-06-17T12:57:00.729 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:57:00.730 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.730 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.731 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.733 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:57:00.733 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:00.733 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.734 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:00.734 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6820", 2022-06-17T12:57:00.735 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:57:00.737 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.737 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.738 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:00.738 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6821", 2022-06-17T12:57:00.739 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:57:00.739 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.739 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.744 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.745 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:57:00.745 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:00.746 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.746 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:00.747 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6824", 2022-06-17T12:57:00.747 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:57:00.748 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.748 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.749 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:00.749 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6825", 2022-06-17T12:57:00.750 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:57:00.750 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.751 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.751 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.752 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:57:00.752 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:00.752 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.755 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:00.756 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6822", 2022-06-17T12:57:00.756 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:57:00.756 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.757 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.757 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:00.758 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6823", 2022-06-17T12:57:00.758 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:57:00.759 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.759 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.760 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.760 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6819/22646", 2022-06-17T12:57:00.761 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6821/22646", 2022-06-17T12:57:00.761 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6825/22646", 2022-06-17T12:57:00.761 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6823/22646", 2022-06-17T12:57:00.788 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:57:00.788 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:57:00.789 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:57:00.790 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.790 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.871 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.872 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:57:00.874 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", 2022-06-17T12:57:00.875 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:57:00.875 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:57:00.876 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:57:00.876 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:57:00.890 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:57:00.896 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:57:00.905 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:57:00.905 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 138, 2022-06-17T12:57:00.907 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:57:00.908 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:57:00.908 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:57:00.909 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:00.912 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.913 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:00.913 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6808", 2022-06-17T12:57:00.914 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:57:00.914 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.916 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.930 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:00.931 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6809", 2022-06-17T12:57:00.931 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:57:00.931 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.932 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.932 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.932 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:57:00.932 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:00.933 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.933 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:00.933 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6810", 2022-06-17T12:57:00.933 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:57:00.934 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.934 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.934 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:00.935 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6811", 2022-06-17T12:57:00.935 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:57:00.936 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.936 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.937 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.937 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:57:00.938 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:00.939 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.939 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:00.940 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6814", 2022-06-17T12:57:00.941 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:57:00.941 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.942 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.942 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:00.943 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6815", 2022-06-17T12:57:00.943 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:57:00.943 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.944 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.944 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.945 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:57:00.945 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:00.945 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.946 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:00.946 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6812", 2022-06-17T12:57:00.947 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:57:00.948 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.948 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.949 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:00.949 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6813", 2022-06-17T12:57:00.950 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:57:00.950 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.950 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.951 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.952 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6809/22812", 2022-06-17T12:57:00.952 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6811/22812", 2022-06-17T12:57:00.953 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6815/22812", 2022-06-17T12:57:00.954 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6813/22812", 2022-06-17T12:57:00.955 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:57:00.955 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:57:00.956 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:57:00.957 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.957 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.957 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.958 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:57:00.958 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", 2022-06-17T12:57:00.959 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:57:00.959 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:57:00.960 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:57:00.960 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:57:00.960 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:57:00.961 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:57:00.961 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:57:00.961 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 139, 2022-06-17T12:57:00.962 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:57:00.963 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:57:00.963 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:57:00.964 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:00.964 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.964 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:00.978 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6800", 2022-06-17T12:57:00.979 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:57:00.979 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.979 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.980 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:00.980 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6801", 2022-06-17T12:57:00.981 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:57:00.981 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:00.981 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:00.991 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:00.991 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:57:00.991 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:00.992 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:00.992 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:00.992 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6802", 2022-06-17T12:57:00.993 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:57:00.993 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.004 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.005 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:01.005 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6803", 2022-06-17T12:57:01.006 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:57:01.006 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:01.007 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:01.007 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.008 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:57:01.008 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:01.009 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.009 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:01.010 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6806", 2022-06-17T12:57:01.010 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:57:01.011 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.011 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.012 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:01.012 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6807", 2022-06-17T12:57:01.012 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:57:01.013 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:01.013 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:01.015 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.015 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:57:01.015 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:01.016 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.016 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:01.017 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6804", 2022-06-17T12:57:01.017 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:57:01.018 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.018 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.019 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:01.019 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6805", 2022-06-17T12:57:01.020 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:57:01.020 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:01.020 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:01.021 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.021 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6801/22813", 2022-06-17T12:57:01.022 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6803/22813", 2022-06-17T12:57:01.022 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6807/22813", 2022-06-17T12:57:01.023 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6805/22813", 2022-06-17T12:57:01.023 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:57:01.024 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:57:01.024 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:57:01.025 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:01.026 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.027 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.027 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:57:01.027 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", 2022-06-17T12:57:01.028 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:57:01.028 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:57:01.029 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:57:01.029 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:57:01.029 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:57:01.037 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:57:01.037 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:57:01.038 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 139, 2022-06-17T12:57:01.047 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:57:01.047 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:57:01.048 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:57:01.048 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:01.049 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.049 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:01.049 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6816", 2022-06-17T12:57:01.050 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:57:01.051 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.051 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.052 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:01.052 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6817", 2022-06-17T12:57:01.053 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:57:01.053 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:01.053 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:01.063 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.063 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:57:01.064 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:01.064 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.064 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:01.067 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6818", 2022-06-17T12:57:01.067 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:57:01.068 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.079 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.080 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:01.080 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6819", 2022-06-17T12:57:01.081 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:57:01.081 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:01.082 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:01.082 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.083 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:57:01.083 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:01.084 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.084 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:01.085 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6822", 2022-06-17T12:57:01.085 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:57:01.085 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.096 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.096 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:01.097 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6823", 2022-06-17T12:57:01.097 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:57:01.097 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:01.098 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:01.099 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.099 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:57:01.100 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:01.100 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.100 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:01.101 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6820", 2022-06-17T12:57:01.101 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:57:01.102 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.102 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.102 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:01.103 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6821", 2022-06-17T12:57:01.105 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:57:01.105 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:01.106 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:01.106 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.107 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6817/22822", 2022-06-17T12:57:01.108 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6819/22822", 2022-06-17T12:57:01.109 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6823/22822", 2022-06-17T12:57:01.109 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6821/22822", 2022-06-17T12:57:01.110 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:57:01.114 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:57:01.114 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:57:01.115 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:01.115 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.115 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.116 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:57:01.116 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", 2022-06-17T12:57:01.117 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:57:01.117 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:57:01.117 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:57:01.119 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:57:01.119 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:57:01.120 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:57:01.120 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:57:01.120 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 139, 2022-06-17T12:57:01.120 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:57:01.121 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:57:01.121 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:57:01.121 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:01.121 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.122 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:01.122 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6824", 2022-06-17T12:57:01.122 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:57:01.122 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.123 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.123 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:01.123 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6825", 2022-06-17T12:57:01.129 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:57:01.129 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:01.129 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:01.130 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.130 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:57:01.130 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:01.130 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.131 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:01.131 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6826", 2022-06-17T12:57:01.131 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:57:01.132 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.133 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.133 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:01.134 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6827", 2022-06-17T12:57:01.134 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:57:01.134 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:01.134 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:01.135 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.135 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:57:01.135 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:01.135 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.135 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:01.136 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6830", 2022-06-17T12:57:01.136 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:57:01.136 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.136 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.137 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:01.137 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6831", 2022-06-17T12:57:01.137 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:57:01.137 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:01.137 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:01.142 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.142 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:57:01.142 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:01.142 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.143 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:01.143 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6828", 2022-06-17T12:57:01.143 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:57:01.143 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.144 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.144 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:01.144 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6829", 2022-06-17T12:57:01.144 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:57:01.145 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:01.145 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:01.145 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.145 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6825/22837", 2022-06-17T12:57:01.145 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6827/22837", 2022-06-17T12:57:01.146 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6831/22837", 2022-06-17T12:57:01.146 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6829/22837", 2022-06-17T12:57:01.146 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:57:01.146 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:57:01.147 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:57:01.147 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:01.148 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:01.148 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:57:01.149 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_xinfo": [ 2022-06-17T12:57:01.149 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.149 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:57:01.150 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:57:01.150 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:57:01.151 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:57:01.151 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:57:01.151 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:57:01.152 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.291461+0000", 2022-06-17T12:57:01.152 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:57:01.153 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.153 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.153 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:57:01.156 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:57:01.156 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:57:01.157 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:57:01.158 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:57:01.158 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:57:01.159 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.537912+0000", 2022-06-17T12:57:01.159 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:57:01.160 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.170 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.171 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:57:01.171 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:57:01.172 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:57:01.172 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:57:01.172 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:57:01.173 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:57:01.173 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.606947+0000", 2022-06-17T12:57:01.173 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:57:01.174 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.175 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.175 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:57:01.176 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:57:01.176 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:57:01.177 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:57:01.177 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:57:01.178 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:57:01.178 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.514412+0000", 2022-06-17T12:57:01.178 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:57:01.181 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.181 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.182 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:57:01.182 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:57:01.183 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:57:01.183 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:57:01.184 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:57:01.184 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:57:01.185 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.394645+0000", 2022-06-17T12:57:01.185 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:57:01.194 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.210 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.211 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:57:01.211 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:57:01.214 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:57:01.214 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:57:01.214 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:57:01.214 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:57:01.215 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.343806+0000", 2022-06-17T12:57:01.215 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:57:01.215 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.215 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.216 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:57:01.216 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:57:01.217 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:57:01.217 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:57:01.218 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:57:01.218 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:57:01.218 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.591864+0000", 2022-06-17T12:57:01.219 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:57:01.221 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.221 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:01.221 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:57:01.222 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:57:01.222 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:57:01.222 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:57:01.223 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:57:01.223 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:57:01.224 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.607045+0000", 2022-06-17T12:57:01.224 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:57:01.224 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:01.224 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:57:01.225 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap": [], 2022-06-17T12:57:01.225 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap_items": [], 2022-06-17T12:57:01.225 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_temp": [], 2022-06-17T12:57:01.226 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_temp": [], 2022-06-17T12:57:01.226 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklist": {}, 2022-06-17T12:57:01.226 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profiles": { 2022-06-17T12:57:01.227 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs_data": { 2022-06-17T12:57:01.228 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-device-class": "", 2022-06-17T12:57:01.228 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:57:01.228 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-root": "default", 2022-06-17T12:57:01.229 INFO:teuthology.orchestra.run.smithi036.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-17T12:57:01.229 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:57:01.229 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "2", 2022-06-17T12:57:01.229 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:57:01.230 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:57:01.230 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van", 2022-06-17T12:57:01.230 INFO:teuthology.orchestra.run.smithi036.stdout: "w": "8" 2022-06-17T12:57:01.230 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.231 INFO:teuthology.orchestra.run.smithi036.stdout: "default": { 2022-06-17T12:57:01.231 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:57:01.231 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:57:01.231 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "1", 2022-06-17T12:57:01.232 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:57:01.232 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:57:01.232 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van" 2022-06-17T12:57:01.233 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:01.234 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:01.234 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps_queue": [], 2022-06-17T12:57:01.235 INFO:teuthology.orchestra.run.smithi036.stdout: "new_removed_snaps": [], 2022-06-17T12:57:01.235 INFO:teuthology.orchestra.run.smithi036.stdout: "new_purged_snaps": [], 2022-06-17T12:57:01.235 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_node_flags": {}, 2022-06-17T12:57:01.235 INFO:teuthology.orchestra.run.smithi036.stdout: "device_class_flags": {} 2022-06-17T12:57:01.236 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:57:01.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:00.090+0000 7f38654ac700 1 -- 172.21.15.36:0/3321177537 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3844053a40 msgr2=0x7f3844055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:01.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:00.090+0000 7f38654ac700 1 --2- 172.21.15.36:0/3321177537 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3844053a40 0x7f3844055f00 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f384801c000 tx=0x7f3848015000).stop 2022-06-17T12:57:01.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:00.090+0000 7f38654ac700 1 -- 172.21.15.36:0/3321177537 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f386005fe80 msgr2=0x7f38601415d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:01.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:00.090+0000 7f38654ac700 1 --2- 172.21.15.36:0/3321177537 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f386005fe80 0x7f38601415d0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7f3860140a90 tx=0x7f3850007c10).stop 2022-06-17T12:57:01.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:00.090+0000 7f38654ac700 1 -- 172.21.15.36:0/3321177537 shutdown_connections 2022-06-17T12:57:01.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:00.090+0000 7f38654ac700 1 --2- 172.21.15.36:0/3321177537 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3844053a40 0x7f3844055f00 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:01.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:00.090+0000 7f38654ac700 1 --2- 172.21.15.36:0/3321177537 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38600b1db0 0x7f3860064790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:01.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:00.090+0000 7f38654ac700 1 --2- 172.21.15.36:0/3321177537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38600ae360 0x7f386013ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:01.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:00.090+0000 7f38654ac700 1 --2- 172.21.15.36:0/3321177537 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f386005fe80 0x7f38601415d0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:01.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:00.090+0000 7f38654ac700 1 -- 172.21.15.36:0/3321177537 >> 172.21.15.36:0/3321177537 conn(0x7f3860060990 msgr2=0x7f38600b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:01.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:00.090+0000 7f38654ac700 1 -- 172.21.15.36:0/3321177537 shutdown_connections 2022-06-17T12:57:01.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:00.090+0000 7f38654ac700 1 -- 172.21.15.36:0/3321177537 wait complete. 2022-06-17T12:57:01.244 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:57:01.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba3667e700 1 Processor -- start 2022-06-17T12:57:01.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba3667e700 1 -- start start 2022-06-17T12:57:01.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba3667e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba30062bd0 0x7fba300b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:01.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba3667e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fba30140e30 0x7fba30141280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:01.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba3667e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba30065090 con 0x7fba300b12f0 2022-06-17T12:57:01.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba3667e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba30063510 con 0x7fba30140e30 2022-06-17T12:57:01.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba3667e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba300b0b40 con 0x7fba30062bd0 2022-06-17T12:57:01.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba34c1b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fba30140e30 0x7fba30141280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:01.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba2ffff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fba300b12f0 0x7fba3013aac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57838/0 (socket says 172.21.15.36:57838) 2022-06-17T12:57:01.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba2ffff700 1 -- 172.21.15.36:0/2575036948 learned_addr learned my addr 172.21.15.36:0/2575036948 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:01.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba2f7fe700 1 --2- 172.21.15.36:0/2575036948 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba30062bd0 0x7fba300b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:01.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba34c1b700 1 -- 172.21.15.36:0/2575036948 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba30062bd0 msgr2=0x7fba300b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:01.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba34c1b700 1 --2- 172.21.15.36:0/2575036948 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba30062bd0 0x7fba300b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:01.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba34c1b700 1 -- 172.21.15.36:0/2575036948 >> v1:172.21.15.36:6789/0 conn(0x7fba300b12f0 legacy=0x7fba3013aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:01.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba2effd700 1 -- 172.21.15.36:0/2575036948 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1566480040 0 0) 0x7fba30065090 con 0x7fba300b12f0 2022-06-17T12:57:01.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba34c1b700 1 -- 172.21.15.36:0/2575036948 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba300b1bc0 con 0x7fba30140e30 2022-06-17T12:57:01.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba34c1b700 1 --2- 172.21.15.36:0/2575036948 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba30140e30 0x7fba30141280 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 rx=0x7fba280032f0 tx=0x7fba280091d0).ready entity=mon.0 client_cookie=d68dd378983a7f3 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:01.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba2effd700 1 -- 172.21.15.36:0/2575036948 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba2800b6c0 con 0x7fba30140e30 2022-06-17T12:57:01.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba2effd700 1 -- 172.21.15.36:0/2575036948 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2800b820 con 0x7fba30140e30 2022-06-17T12:57:01.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba2effd700 1 -- 172.21.15.36:0/2575036948 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba2800b990 con 0x7fba30140e30 2022-06-17T12:57:01.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba3667e700 1 -- 172.21.15.36:0/2575036948 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba30140e30 msgr2=0x7fba30141280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:01.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba3667e700 1 --2- 172.21.15.36:0/2575036948 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba30140e30 0x7fba30141280 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 rx=0x7fba280032f0 tx=0x7fba280091d0).stop 2022-06-17T12:57:01.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba3667e700 1 -- 172.21.15.36:0/2575036948 shutdown_connections 2022-06-17T12:57:01.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba3667e700 1 --2- 172.21.15.36:0/2575036948 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba30140e30 0x7fba30141280 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:01.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba3667e700 1 --2- 172.21.15.36:0/2575036948 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba30062bd0 0x7fba300b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:01.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba3667e700 1 -- 172.21.15.36:0/2575036948 >> 172.21.15.36:0/2575036948 conn(0x7fba300b53e0 msgr2=0x7fba300b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:01.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.374+0000 7fba3667e700 1 -- 172.21.15.36:0/2575036948 shutdown_connections 2022-06-17T12:57:01.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.378+0000 7fba3667e700 1 -- 172.21.15.36:0/2575036948 wait complete. 2022-06-17T12:57:01.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.378+0000 7fba3667e700 1 Processor -- start 2022-06-17T12:57:01.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.378+0000 7fba3667e700 1 -- start start 2022-06-17T12:57:01.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.378+0000 7fba3667e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba30062bd0 0x7fba3012f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:01.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.378+0000 7fba3667e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba300b12f0 0x7fba30155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:01.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.378+0000 7fba3667e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba30140e30 0x7fba30156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:01.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.378+0000 7fba3667e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3013de70 con 0x7fba300b12f0 2022-06-17T12:57:01.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.378+0000 7fba3667e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3012e240 con 0x7fba30062bd0 2022-06-17T12:57:01.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.378+0000 7fba3667e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba300af2f0 con 0x7fba30140e30 2022-06-17T12:57:01.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.378+0000 7fba2f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba300b12f0 0x7fba30155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:01.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.378+0000 7fba2f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba300b12f0 0x7fba30155c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40314/0 (socket says 172.21.15.36:40314) 2022-06-17T12:57:01.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.378+0000 7fba2f7fe700 1 -- 172.21.15.36:0/1054068855 learned_addr learned my addr 172.21.15.36:0/1054068855 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:01.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.378+0000 7fba34c1b700 1 --2- 172.21.15.36:0/1054068855 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba30140e30 0x7fba30156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:01.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.378+0000 7fba2ffff700 1 --2- 172.21.15.36:0/1054068855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba30062bd0 0x7fba3012f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:01.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.378+0000 7fba2f7fe700 1 -- 172.21.15.36:0/1054068855 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba30140e30 msgr2=0x7fba30156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:01.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.378+0000 7fba2f7fe700 1 --2- 172.21.15.36:0/1054068855 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba30140e30 0x7fba30156170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:01.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.378+0000 7fba2f7fe700 1 -- 172.21.15.36:0/1054068855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba30062bd0 msgr2=0x7fba3012f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:01.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.378+0000 7fba2f7fe700 1 --2- 172.21.15.36:0/1054068855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba30062bd0 0x7fba3012f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:01.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.378+0000 7fba2f7fe700 1 -- 172.21.15.36:0/1054068855 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba28009470 con 0x7fba300b12f0 2022-06-17T12:57:01.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.378+0000 7fba34c1b700 1 --2- 172.21.15.36:0/1054068855 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba30140e30 0x7fba30156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:57:01.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.382+0000 7fba2f7fe700 1 --2- 172.21.15.36:0/1054068855 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba300b12f0 0x7fba30155c80 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7fba2000c000 tx=0x7fba20006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:01.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.382+0000 7fba2d7fa700 1 -- 172.21.15.36:0/1054068855 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba20010780 con 0x7fba300b12f0 2022-06-17T12:57:01.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.382+0000 7fba2d7fa700 1 -- 172.21.15.36:0/1054068855 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba200108e0 con 0x7fba300b12f0 2022-06-17T12:57:01.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.382+0000 7fba3667e700 1 -- 172.21.15.36:0/1054068855 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba3012f920 con 0x7fba300b12f0 2022-06-17T12:57:01.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.382+0000 7fba2d7fa700 1 -- 172.21.15.36:0/1054068855 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba20010c40 con 0x7fba300b12f0 2022-06-17T12:57:01.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.382+0000 7fba3667e700 1 -- 172.21.15.36:0/1054068855 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba3012e670 con 0x7fba300b12f0 2022-06-17T12:57:01.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.382+0000 7fba3667e700 1 -- 172.21.15.36:0/1054068855 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba300b03c0 con 0x7fba300b12f0 2022-06-17T12:57:01.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.382+0000 7fba2d7fa700 1 -- 172.21.15.36:0/1054068855 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fba2001c460 con 0x7fba300b12f0 2022-06-17T12:57:01.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.382+0000 7fba2d7fa700 1 --2- 172.21.15.36:0/1054068855 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fba14053af0 0x7fba14055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:01.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.382+0000 7fba2d7fa700 1 -- 172.21.15.36:0/1054068855 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fba2006c4a0 con 0x7fba300b12f0 2022-06-17T12:57:01.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.382+0000 7fba2ffff700 1 --2- 172.21.15.36:0/1054068855 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fba14053af0 0x7fba14055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:01.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.386+0000 7fba2ffff700 1 --2- 172.21.15.36:0/1054068855 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fba14053af0 0x7fba14055fb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fba1800d000 tx=0x7fba18007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:01.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.390+0000 7fba2d7fa700 1 -- 172.21.15.36:0/1054068855 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fba200431a0 con 0x7fba300b12f0 2022-06-17T12:57:01.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.558+0000 7fba3667e700 1 -- 172.21.15.36:0/1054068855 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba30057410 con 0x7fba300b12f0 2022-06-17T12:57:01.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.558+0000 7fba2d7fa700 1 -- 172.21.15.36:0/1054068855 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+2142 (secure 0 0 0) 0x7fba200467c0 con 0x7fba300b12f0 2022-06-17T12:57:01.550 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:57:01.551 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":78,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1081450840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1081450840},{"type":"v1","addr":"172.21.15.70:6833","nonce":1081450840}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":76}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:56:49.200774+0000","modified":"2022-06-17T12:56:50.204509+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8478":{"gid":8478,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/3169088536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3169088536},{"type":"v1","addr":"172.21.15.36:6835","nonce":3169088536}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-06-17T12:57:01.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.562+0000 7fba3667e700 1 -- 172.21.15.36:0/1054068855 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fba14053af0 msgr2=0x7fba14055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:01.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.562+0000 7fba3667e700 1 --2- 172.21.15.36:0/1054068855 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fba14053af0 0x7fba14055fb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fba1800d000 tx=0x7fba18007a00).stop 2022-06-17T12:57:01.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.562+0000 7fba3667e700 1 -- 172.21.15.36:0/1054068855 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba300b12f0 msgr2=0x7fba30155c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:01.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.562+0000 7fba3667e700 1 --2- 172.21.15.36:0/1054068855 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba300b12f0 0x7fba30155c80 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7fba2000c000 tx=0x7fba20006df0).stop 2022-06-17T12:57:01.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.562+0000 7fba3667e700 1 -- 172.21.15.36:0/1054068855 shutdown_connections 2022-06-17T12:57:01.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.562+0000 7fba3667e700 1 --2- 172.21.15.36:0/1054068855 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fba14053af0 0x7fba14055fb0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:01.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.562+0000 7fba3667e700 1 --2- 172.21.15.36:0/1054068855 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba30140e30 0x7fba30156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:01.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.562+0000 7fba3667e700 1 --2- 172.21.15.36:0/1054068855 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fba300b12f0 0x7fba30155c80 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:01.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.562+0000 7fba3667e700 1 --2- 172.21.15.36:0/1054068855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba30062bd0 0x7fba3012f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:01.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.562+0000 7fba3667e700 1 -- 172.21.15.36:0/1054068855 >> 172.21.15.36:0/1054068855 conn(0x7fba300b53e0 msgr2=0x7fba30133a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:01.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.562+0000 7fba3667e700 1 -- 172.21.15.36:0/1054068855 shutdown_connections 2022-06-17T12:57:01.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.562+0000 7fba3667e700 1 -- 172.21.15.36:0/1054068855 wait complete. 2022-06-17T12:57:01.560 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 78 2022-06-17T12:57:01.562 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-17T12:57:01.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.714+0000 7f669708c700 1 Processor -- start 2022-06-17T12:57:01.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.714+0000 7f669708c700 1 -- start start 2022-06-17T12:57:01.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f669708c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66900b27c0 0x7f66900acf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:01.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f669708c700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66900af6c0 con 0x7f66900b14d0 2022-06-17T12:57:01.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f669708c700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66900af920 con 0x7f66900b2ba0 2022-06-17T12:57:01.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f669708c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66900ad890 con 0x7f66900b27c0 2022-06-17T12:57:01.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f669688b700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f66900b14d0 0x7f6690149d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57850/0 (socket says 172.21.15.36:57850) 2022-06-17T12:57:01.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f669688b700 1 -- 172.21.15.36:0/497332626 learned_addr learned my addr 172.21.15.36:0/497332626 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:01.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f669608a700 1 --2- 172.21.15.36:0/497332626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66900b27c0 0x7f66900acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:01.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f6695088700 1 -- 172.21.15.36:0/497332626 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 729313065 0 0) 0x7f66900af6c0 con 0x7f66900b14d0 2022-06-17T12:57:01.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f6695088700 1 -- 172.21.15.36:0/497332626 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6678003260 con 0x7f66900b14d0 2022-06-17T12:57:01.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f6695088700 1 -- 172.21.15.36:0/497332626 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1993221867 0 0) 0x7f6678003260 con 0x7f66900b14d0 2022-06-17T12:57:01.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f6695088700 1 -- 172.21.15.36:0/497332626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66900b27c0 msgr2=0x7f66900acf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:01.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f6695088700 1 --2- 172.21.15.36:0/497332626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66900b27c0 0x7f66900acf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:01.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f6695088700 1 -- 172.21.15.36:0/497332626 >> v1:172.21.15.70:6789/0 conn(0x7f66900b2ba0 legacy=0x7f66900afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:01.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f6695088700 1 -- 172.21.15.36:0/497332626 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66900ac4b0 con 0x7f66900b14d0 2022-06-17T12:57:01.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f6695088700 1 -- 172.21.15.36:0/497332626 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f66880019e0 con 0x7f66900b14d0 2022-06-17T12:57:01.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f669608a700 1 --2- 172.21.15.36:0/497332626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66900b27c0 0x7f66900acf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:57:01.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f6695088700 1 -- 172.21.15.36:0/497332626 >> v1:172.21.15.36:6789/0 conn(0x7f66900b14d0 legacy=0x7f6690149d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:01.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f6695088700 1 --2- 172.21.15.36:0/497332626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6678003780 0x7f6678003b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:01.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f6695088700 1 --2- 172.21.15.36:0/497332626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66900b27c0 0x7f6678004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:01.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f6695088700 1 --2- 172.21.15.36:0/497332626 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f66780049f0 0x7f6678004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:01.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f6695088700 1 -- 172.21.15.36:0/497332626 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66900ad890 con 0x7f66780049f0 2022-06-17T12:57:01.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f6695088700 1 -- 172.21.15.36:0/497332626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667800b750 con 0x7f6678003780 2022-06-17T12:57:01.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f6695088700 1 -- 172.21.15.36:0/497332626 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f667800daa0 con 0x7f66900b27c0 2022-06-17T12:57:01.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f669688b700 1 --2- 172.21.15.36:0/497332626 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f66780049f0 0x7f6678004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:01.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f669608a700 1 --2- 172.21.15.36:0/497332626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6678003780 0x7f6678003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:01.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f6695889700 1 --2- 172.21.15.36:0/497332626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66900b27c0 0x7f6678004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:01.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f669688b700 1 -- 172.21.15.36:0/497332626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66900b27c0 msgr2=0x7f6678004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:01.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f669688b700 1 --2- 172.21.15.36:0/497332626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66900b27c0 0x7f6678004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:01.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f669688b700 1 -- 172.21.15.36:0/497332626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6678003780 msgr2=0x7f6678003b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:01.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f669688b700 1 --2- 172.21.15.36:0/497332626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6678003780 0x7f6678003b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:01.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f669688b700 1 -- 172.21.15.36:0/497332626 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c00b180 con 0x7f66780049f0 2022-06-17T12:57:01.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f669608a700 1 --2- 172.21.15.36:0/497332626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6678003780 0x7f6678003b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:57:01.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f6695889700 1 --2- 172.21.15.36:0/497332626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66900b27c0 0x7f6678004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:57:01.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f669688b700 1 --2- 172.21.15.36:0/497332626 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f66780049f0 0x7f6678004e70 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0x7f66880016f0 tx=0x7f66880021b0).ready entity=mon.0 client_cookie=ce5811a636807021 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:01.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f6695088700 1 -- 172.21.15.36:0/497332626 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f668800cca0 con 0x7f66780049f0 2022-06-17T12:57:01.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f6695088700 1 -- 172.21.15.36:0/497332626 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668800ce00 con 0x7f66780049f0 2022-06-17T12:57:01.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.718+0000 7f6695088700 1 -- 172.21.15.36:0/497332626 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6688005b80 con 0x7f66780049f0 2022-06-17T12:57:01.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.722+0000 7f669708c700 1 -- 172.21.15.36:0/497332626 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f66780049f0 msgr2=0x7f6678004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:01.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.722+0000 7f669708c700 1 --2- 172.21.15.36:0/497332626 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f66780049f0 0x7f6678004e70 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0x7f66880016f0 tx=0x7f66880021b0).stop 2022-06-17T12:57:01.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.722+0000 7f669688b700 1 -- 172.21.15.36:0/497332626 reap_dead start 2022-06-17T12:57:01.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.722+0000 7f669708c700 1 -- 172.21.15.36:0/497332626 shutdown_connections 2022-06-17T12:57:01.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.722+0000 7f669708c700 1 -- 172.21.15.36:0/497332626 >> 172.21.15.36:0/497332626 conn(0x7f669005f770 msgr2=0x7f669005fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:01.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.722+0000 7f669708c700 1 -- 172.21.15.36:0/497332626 shutdown_connections 2022-06-17T12:57:01.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.722+0000 7f669708c700 1 -- 172.21.15.36:0/497332626 wait complete. 2022-06-17T12:57:01.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.722+0000 7f669708c700 1 Processor -- start 2022-06-17T12:57:01.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.722+0000 7f669708c700 1 -- start start 2022-06-17T12:57:01.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.722+0000 7f669708c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f66901441d0 0x7f6690151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:01.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.722+0000 7f669708c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6690065630 0x7f66900a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:01.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.722+0000 7f669708c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6690065100 0x7f66900a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:01.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.722+0000 7f669708c700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66900a0b70 con 0x7f66901441d0 2022-06-17T12:57:01.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.722+0000 7f669708c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669009fd60 con 0x7f6690065100 2022-06-17T12:57:01.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.722+0000 7f669708c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6690066340 con 0x7f6690065630 2022-06-17T12:57:01.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.726+0000 7f669608a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f66901441d0 0x7f6690151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:01.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.726+0000 7f669608a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f66901441d0 0x7f6690151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40332/0 (socket says 172.21.15.36:40332) 2022-06-17T12:57:01.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.726+0000 7f669608a700 1 -- 172.21.15.36:0/3801514728 learned_addr learned my addr 172.21.15.36:0/3801514728 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:01.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.726+0000 7f6695889700 1 --2- 172.21.15.36:0/3801514728 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6690065630 0x7f66900a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:01.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.726+0000 7f669688b700 1 --2- 172.21.15.36:0/3801514728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6690065100 0x7f66900a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:01.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.726+0000 7f669608a700 1 -- 172.21.15.36:0/3801514728 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6690065630 msgr2=0x7f66900a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:01.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.726+0000 7f669608a700 1 --2- 172.21.15.36:0/3801514728 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6690065630 0x7f66900a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:01.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.726+0000 7f669608a700 1 -- 172.21.15.36:0/3801514728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6690065100 msgr2=0x7f66900a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:01.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.726+0000 7f669608a700 1 --2- 172.21.15.36:0/3801514728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6690065100 0x7f66900a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:01.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.726+0000 7f669608a700 1 -- 172.21.15.36:0/3801514728 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66901445b0 con 0x7f66901441d0 2022-06-17T12:57:01.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.726+0000 7f669608a700 1 --2- 172.21.15.36:0/3801514728 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f66901441d0 0x7f6690151c20 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7f6690064ae0 tx=0x7f668c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:01.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.726+0000 7f667f7fe700 1 -- 172.21.15.36:0/3801514728 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f668c011e40 con 0x7f66901441d0 2022-06-17T12:57:01.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.726+0000 7f667f7fe700 1 -- 172.21.15.36:0/3801514728 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668c00fc20 con 0x7f66901441d0 2022-06-17T12:57:01.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.726+0000 7f669708c700 1 -- 172.21.15.36:0/3801514728 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6690064e40 con 0x7f66901441d0 2022-06-17T12:57:01.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.726+0000 7f667f7fe700 1 -- 172.21.15.36:0/3801514728 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f668c014440 con 0x7f66901441d0 2022-06-17T12:57:01.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.726+0000 7f669708c700 1 -- 172.21.15.36:0/3801514728 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66900aafa0 con 0x7f66901441d0 2022-06-17T12:57:01.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.726+0000 7f667f7fe700 1 -- 172.21.15.36:0/3801514728 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f668c0145a0 con 0x7f66901441d0 2022-06-17T12:57:01.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.726+0000 7f669708c700 1 -- 172.21.15.36:0/3801514728 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f669009fea0 con 0x7f66901441d0 2022-06-17T12:57:01.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.726+0000 7f667f7fe700 1 --2- 172.21.15.36:0/3801514728 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6670053a40 0x7f6670055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:01.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.726+0000 7f667f7fe700 1 -- 172.21.15.36:0/3801514728 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f668c06ce40 con 0x7f66901441d0 2022-06-17T12:57:01.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.726+0000 7f6695889700 1 --2- 172.21.15.36:0/3801514728 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6670053a40 0x7f6670055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:01.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.730+0000 7f6695889700 1 --2- 172.21.15.36:0/3801514728 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6670053a40 0x7f6670055f00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f668000d000 tx=0x7f668000c040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:01.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.734+0000 7f667f7fe700 1 -- 172.21.15.36:0/3801514728 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f668c043be0 con 0x7f66901441d0 2022-06-17T12:57:01.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:01.902+0000 7f669708c700 1 -- 172.21.15.36:0/3801514728 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f66900ac640 con 0x7f66901441d0 2022-06-17T12:57:02.739 INFO:tasks.ceph.mon.a.smithi036.stderr:2022-06-17T12:57:02.746+0000 7fb7a2649700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-17T12:57:02.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.754+0000 7f667f7fe700 1 -- 172.21.15.36:0/3801514728 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v79) v1 ==== 161+0+0 (secure 0 0 0) 0x7f668c047200 con 0x7f66901441d0 2022-06-17T12:57:02.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.758+0000 7f669708c700 1 -- 172.21.15.36:0/3801514728 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6670053a40 msgr2=0x7f6670055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:02.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.758+0000 7f669708c700 1 --2- 172.21.15.36:0/3801514728 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6670053a40 0x7f6670055f00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f668000d000 tx=0x7f668000c040).stop 2022-06-17T12:57:02.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.758+0000 7f669708c700 1 -- 172.21.15.36:0/3801514728 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f66901441d0 msgr2=0x7f6690151c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:02.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.758+0000 7f669708c700 1 --2- 172.21.15.36:0/3801514728 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f66901441d0 0x7f6690151c20 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7f6690064ae0 tx=0x7f668c006df0).stop 2022-06-17T12:57:02.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.758+0000 7f669708c700 1 -- 172.21.15.36:0/3801514728 shutdown_connections 2022-06-17T12:57:02.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.758+0000 7f669708c700 1 --2- 172.21.15.36:0/3801514728 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6670053a40 0x7f6670055f00 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:02.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.758+0000 7f669708c700 1 --2- 172.21.15.36:0/3801514728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6690065100 0x7f66900a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:02.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.758+0000 7f669708c700 1 --2- 172.21.15.36:0/3801514728 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6690065630 0x7f66900a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:02.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.758+0000 7f669708c700 1 --2- 172.21.15.36:0/3801514728 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f66901441d0 0x7f6690151c20 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:02.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.758+0000 7f669708c700 1 -- 172.21.15.36:0/3801514728 >> 172.21.15.36:0/3801514728 conn(0x7f669005f770 msgr2=0x7f66900b1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:02.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.758+0000 7f669708c700 1 -- 172.21.15.36:0/3801514728 shutdown_connections 2022-06-17T12:57:02.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.758+0000 7f669708c700 1 -- 172.21.15.36:0/3801514728 wait complete. 2022-06-17T12:57:02.773 INFO:teuthology.orchestra.run.smithi036.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-17T12:57:02.774 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:57:02.779 INFO:tasks.ceph.mds.a.smithi036.stdout:starting mds.a at 2022-06-17T12:57:02.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.898+0000 7f24fd0b1700 1 Processor -- start 2022-06-17T12:57:02.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.898+0000 7f24fd0b1700 1 -- start start 2022-06-17T12:57:02.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.898+0000 7f24fd0b1700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f24f8060780 0x7f24f8060b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:02.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.898+0000 7f24fd0b1700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f24f81434f0 0x7f24f80b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:02.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.898+0000 7f24fd0b1700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f8143990 con 0x7f24f80afec0 2022-06-17T12:57:02.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.898+0000 7f24fd0b1700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f80b1a80 con 0x7f24f81434f0 2022-06-17T12:57:02.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.898+0000 7f24fd0b1700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f80b1d20 con 0x7f24f8060780 2022-06-17T12:57:02.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.898+0000 7f24f759e700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f24f80afec0 0x7f24f80b0cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57886/0 (socket says 172.21.15.36:57886) 2022-06-17T12:57:02.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.898+0000 7f24f759e700 1 -- 172.21.15.36:0/1767114820 learned_addr learned my addr 172.21.15.36:0/1767114820 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:02.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.898+0000 7f24f659c700 1 --2- 172.21.15.36:0/1767114820 >> v2:172.21.15.36:3300/0 conn(0x7f24f81434f0 0x7f24f80b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:02.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.898+0000 7f24f6d9d700 1 --2- 172.21.15.36:0/1767114820 >> v2:172.21.15.70:3300/0 conn(0x7f24f8060780 0x7f24f8060b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:02.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.898+0000 7f24f5d9b700 1 -- 172.21.15.36:0/1767114820 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2075143812 0 0) 0x7f24f8143990 con 0x7f24f80afec0 2022-06-17T12:57:02.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.898+0000 7f24f659c700 1 -- 172.21.15.36:0/1767114820 >> v2:172.21.15.70:3300/0 conn(0x7f24f8060780 msgr2=0x7f24f8060b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:02.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.898+0000 7f24f659c700 1 --2- 172.21.15.36:0/1767114820 >> v2:172.21.15.70:3300/0 conn(0x7f24f8060780 0x7f24f8060b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:02.937 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.898+0000 7f24f659c700 1 -- 172.21.15.36:0/1767114820 >> v1:172.21.15.36:6789/0 conn(0x7f24f80afec0 legacy=0x7f24f80b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:02.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.898+0000 7f24f659c700 1 -- 172.21.15.36:0/1767114820 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f80b2640 con 0x7f24f81434f0 2022-06-17T12:57:02.951 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.902+0000 7f24f659c700 1 --2- 172.21.15.36:0/1767114820 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f24f81434f0 0x7f24f80b07c0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0x7f24ec002f70 tx=0x7f24ec006df0).ready entity=mon.0 client_cookie=b5256fa504f67bb2 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:02.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.902+0000 7f24f5d9b700 1 -- 172.21.15.36:0/1767114820 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24ec00d570 con 0x7f24f81434f0 2022-06-17T12:57:02.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.902+0000 7f24f5d9b700 1 -- 172.21.15.36:0/1767114820 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ec000e90 con 0x7f24f81434f0 2022-06-17T12:57:02.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.902+0000 7f24f5d9b700 1 -- 172.21.15.36:0/1767114820 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24ec00d8c0 con 0x7f24f81434f0 2022-06-17T12:57:02.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.902+0000 7f24fd0b1700 1 -- 172.21.15.36:0/1767114820 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f24f81434f0 msgr2=0x7f24f80b07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:02.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.902+0000 7f24fd0b1700 1 --2- 172.21.15.36:0/1767114820 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f24f81434f0 0x7f24f80b07c0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0x7f24ec002f70 tx=0x7f24ec006df0).stop 2022-06-17T12:57:02.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.902+0000 7f24fd0b1700 1 -- 172.21.15.36:0/1767114820 shutdown_connections 2022-06-17T12:57:02.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.902+0000 7f24fd0b1700 1 --2- 172.21.15.36:0/1767114820 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f24f81434f0 0x7f24f80b07c0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:02.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.902+0000 7f24fd0b1700 1 --2- 172.21.15.36:0/1767114820 >> v2:172.21.15.70:3300/0 conn(0x7f24f8060780 0x7f24f8060b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:02.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.902+0000 7f24fd0b1700 1 -- 172.21.15.36:0/1767114820 >> 172.21.15.36:0/1767114820 conn(0x7f24f80b6870 msgr2=0x7f24f80b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:02.957 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.902+0000 7f24fd0b1700 1 -- 172.21.15.36:0/1767114820 shutdown_connections 2022-06-17T12:57:02.958 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.902+0000 7f24fd0b1700 1 -- 172.21.15.36:0/1767114820 wait complete. 2022-06-17T12:57:02.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.906+0000 7f24fd0b1700 1 Processor -- start 2022-06-17T12:57:02.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.906+0000 7f24fd0b1700 1 -- start start 2022-06-17T12:57:02.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.906+0000 7f24fd0b1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24f8060780 0x7f24f80646b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:02.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.906+0000 7f24fd0b1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f24f80afec0 0x7f24f80636e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:02.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.906+0000 7f24fd0b1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f24f81434f0 0x7f24f8152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:02.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.906+0000 7f24fd0b1700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f8152560 con 0x7f24f80afec0 2022-06-17T12:57:02.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.906+0000 7f24fd0b1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f8152800 con 0x7f24f8060780 2022-06-17T12:57:02.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.906+0000 7f24fd0b1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f8153190 con 0x7f24f81434f0 2022-06-17T12:57:02.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.906+0000 7f24f659c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f24f80afec0 0x7f24f80636e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:02.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.906+0000 7f24f6d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24f8060780 0x7f24f80646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:02.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.906+0000 7f24f659c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f24f80afec0 0x7f24f80636e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40358/0 (socket says 172.21.15.36:40358) 2022-06-17T12:57:02.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.906+0000 7f24f6d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24f8060780 0x7f24f80646b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:60740/0 (socket says 172.21.15.36:60740) 2022-06-17T12:57:02.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.906+0000 7f24f659c700 1 -- 172.21.15.36:0/2980892455 learned_addr learned my addr 172.21.15.36:0/2980892455 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:02.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.906+0000 7f24f759e700 1 --2- 172.21.15.36:0/2980892455 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f24f81434f0 0x7f24f8152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:02.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.906+0000 7f24f659c700 1 -- 172.21.15.36:0/2980892455 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f24f81434f0 msgr2=0x7f24f8152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:02.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.906+0000 7f24f659c700 1 --2- 172.21.15.36:0/2980892455 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f24f81434f0 0x7f24f8152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:02.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.906+0000 7f24f659c700 1 -- 172.21.15.36:0/2980892455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24f8060780 msgr2=0x7f24f80646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:02.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.906+0000 7f24f659c700 1 --2- 172.21.15.36:0/2980892455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24f8060780 0x7f24f80646b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:02.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.906+0000 7f24f659c700 1 -- 172.21.15.36:0/2980892455 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ec00b180 con 0x7f24f80afec0 2022-06-17T12:57:02.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.906+0000 7f24f659c700 1 --2- 172.21.15.36:0/2980892455 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f24f80afec0 0x7f24f80636e0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0x7f24ec002f70 tx=0x7f24ec014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:02.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.906+0000 7f24e7fff700 1 -- 172.21.15.36:0/2980892455 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24ec00dea0 con 0x7f24f80afec0 2022-06-17T12:57:02.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.906+0000 7f24e7fff700 1 -- 172.21.15.36:0/2980892455 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ec013e90 con 0x7f24f80afec0 2022-06-17T12:57:02.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.906+0000 7f24fd0b1700 1 -- 172.21.15.36:0/2980892455 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f8152250 con 0x7f24f80afec0 2022-06-17T12:57:02.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.906+0000 7f24e7fff700 1 -- 172.21.15.36:0/2980892455 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24ec01e5f0 con 0x7f24f80afec0 2022-06-17T12:57:02.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.910+0000 7f24fd0b1700 1 -- 172.21.15.36:0/2980892455 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f8065870 con 0x7f24f80afec0 2022-06-17T12:57:02.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.910+0000 7f24fd0b1700 1 -- 172.21.15.36:0/2980892455 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f80b13c0 con 0x7f24f80afec0 2022-06-17T12:57:02.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.910+0000 7f24e7fff700 1 -- 172.21.15.36:0/2980892455 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f24ec01c070 con 0x7f24f80afec0 2022-06-17T12:57:02.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.910+0000 7f24e7fff700 1 --2- 172.21.15.36:0/2980892455 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f24d4053a60 0x7f24d4055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:02.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.910+0000 7f24e7fff700 1 -- 172.21.15.36:0/2980892455 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f24ec06ca80 con 0x7f24f80afec0 2022-06-17T12:57:03.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.910+0000 7f24f6d9d700 1 --2- 172.21.15.36:0/2980892455 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f24d4053a60 0x7f24d4055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:03.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.910+0000 7f24f6d9d700 1 --2- 172.21.15.36:0/2980892455 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f24d4053a60 0x7f24d4055f20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f24e800b000 tx=0x7f24e8006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:03.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:02.918+0000 7f24e7fff700 1 -- 172.21.15.36:0/2980892455 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f24ec043790 con 0x7f24f80afec0 2022-06-17T12:57:03.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.082+0000 7f24fd0b1700 1 -- 172.21.15.36:0/2980892455 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24f8057410 con 0x7f24f80afec0 2022-06-17T12:57:03.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.082+0000 7f24e7fff700 1 -- 172.21.15.36:0/2980892455 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+1782 (secure 0 0 0) 0x7f24ec023070 con 0x7f24f80afec0 2022-06-17T12:57:03.066 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:57:03.066 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":79,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1081450840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1081450840},{"type":"v1","addr":"172.21.15.70:6833","nonce":1081450840}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":76}],"filesystems":[{"mdsmap":{"epoch":79,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:56:49.200774+0000","modified":"2022-06-17T12:57:02.747068+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":142,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-06-17T12:57:03.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.082+0000 7f24fd0b1700 1 -- 172.21.15.36:0/2980892455 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f24d4053a60 msgr2=0x7f24d4055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:03.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.082+0000 7f24fd0b1700 1 --2- 172.21.15.36:0/2980892455 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f24d4053a60 0x7f24d4055f20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f24e800b000 tx=0x7f24e8006df0).stop 2022-06-17T12:57:03.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.082+0000 7f24fd0b1700 1 -- 172.21.15.36:0/2980892455 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f24f80afec0 msgr2=0x7f24f80636e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:03.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.082+0000 7f24fd0b1700 1 --2- 172.21.15.36:0/2980892455 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f24f80afec0 0x7f24f80636e0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0x7f24ec002f70 tx=0x7f24ec014000).stop 2022-06-17T12:57:03.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.082+0000 7f24fd0b1700 1 -- 172.21.15.36:0/2980892455 shutdown_connections 2022-06-17T12:57:03.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.082+0000 7f24fd0b1700 1 --2- 172.21.15.36:0/2980892455 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f24d4053a60 0x7f24d4055f20 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:03.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.082+0000 7f24fd0b1700 1 --2- 172.21.15.36:0/2980892455 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f24f81434f0 0x7f24f8152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:03.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.082+0000 7f24fd0b1700 1 --2- 172.21.15.36:0/2980892455 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f24f80afec0 0x7f24f80636e0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:03.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.082+0000 7f24fd0b1700 1 --2- 172.21.15.36:0/2980892455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24f8060780 0x7f24f80646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:03.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.082+0000 7f24fd0b1700 1 -- 172.21.15.36:0/2980892455 >> 172.21.15.36:0/2980892455 conn(0x7f24f80b6870 msgr2=0x7f24f80b4990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:03.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.082+0000 7f24fd0b1700 1 -- 172.21.15.36:0/2980892455 shutdown_connections 2022-06-17T12:57:03.100 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.082+0000 7f24fd0b1700 1 -- 172.21.15.36:0/2980892455 wait complete. 2022-06-17T12:57:03.101 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 79 2022-06-17T12:57:03.102 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-17T12:57:03.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6ca4d83700 1 Processor -- start 2022-06-17T12:57:03.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6ca4d83700 1 -- start start 2022-06-17T12:57:03.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6ca4d83700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ca00b27b0 0x7f6ca014b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:03.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6ca4d83700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f6ca0060ca0 0x7f6ca00ac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:03.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6ca4d83700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6ca0147bd0 0x7f6ca01442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:03.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6ca4d83700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca014b870 con 0x7f6ca0060ca0 2022-06-17T12:57:03.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6ca4d83700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca00ac500 con 0x7f6ca0147bd0 2022-06-17T12:57:03.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6ca4d83700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca00b2b90 con 0x7f6ca00b27b0 2022-06-17T12:57:03.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6c9effd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f6ca0060ca0 0x7f6ca00ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:03.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6c9effd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f6ca0060ca0 0x7f6ca00ac780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40368/0 (socket says 172.21.15.36:40368) 2022-06-17T12:57:03.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6c9effd700 1 -- 172.21.15.36:0/2776006951 learned_addr learned my addr 172.21.15.36:0/2776006951 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:03.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6c9f7fe700 1 --2- 172.21.15.36:0/2776006951 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ca00b27b0 0x7f6ca014b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:03.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6c9ffff700 1 --2- 172.21.15.36:0/2776006951 >> v2:172.21.15.70:3300/0 conn(0x7f6ca0147bd0 0x7f6ca01442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:03.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6c9effd700 1 -- 172.21.15.36:0/2776006951 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ca00b27b0 msgr2=0x7f6ca014b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:03.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6c9effd700 1 --2- 172.21.15.36:0/2776006951 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ca00b27b0 0x7f6ca014b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:03.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6c9effd700 1 -- 172.21.15.36:0/2776006951 >> v2:172.21.15.70:3300/0 conn(0x7f6ca0147bd0 msgr2=0x7f6ca01442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:03.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6c9effd700 1 --2- 172.21.15.36:0/2776006951 >> v2:172.21.15.70:3300/0 conn(0x7f6ca0147bd0 0x7f6ca01442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:03.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6c9effd700 1 -- 172.21.15.36:0/2776006951 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca0147490 con 0x7f6ca0060ca0 2022-06-17T12:57:03.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6c9effd700 1 --2- 172.21.15.36:0/2776006951 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca0060ca0 0x7f6ca00ac780 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 rx=0x7f6c900032f0 tx=0x7f6c90009290).ready entity=mon.0 client_cookie=bba095ad6905da4d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:03.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6c9e7fc700 1 -- 172.21.15.36:0/2776006951 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c9000b870 con 0x7f6ca0060ca0 2022-06-17T12:57:03.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6c9e7fc700 1 -- 172.21.15.36:0/2776006951 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9000b9d0 con 0x7f6ca0060ca0 2022-06-17T12:57:03.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6c9e7fc700 1 -- 172.21.15.36:0/2776006951 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c9000bb40 con 0x7f6ca0060ca0 2022-06-17T12:57:03.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6ca4d83700 1 -- 172.21.15.36:0/2776006951 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca0060ca0 msgr2=0x7f6ca00ac780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:03.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6ca4d83700 1 --2- 172.21.15.36:0/2776006951 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca0060ca0 0x7f6ca00ac780 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 rx=0x7f6c900032f0 tx=0x7f6c90009290).stop 2022-06-17T12:57:03.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6ca4d83700 1 -- 172.21.15.36:0/2776006951 shutdown_connections 2022-06-17T12:57:03.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6ca4d83700 1 --2- 172.21.15.36:0/2776006951 >> v2:172.21.15.70:3300/0 conn(0x7f6ca0147bd0 0x7f6ca01442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:03.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6ca4d83700 1 --2- 172.21.15.36:0/2776006951 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca0060ca0 0x7f6ca00ac780 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:03.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6ca4d83700 1 --2- 172.21.15.36:0/2776006951 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ca00b27b0 0x7f6ca014b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:03.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6ca4d83700 1 -- 172.21.15.36:0/2776006951 >> 172.21.15.36:0/2776006951 conn(0x7f6ca00b5d10 msgr2=0x7f6ca00b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:03.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6ca4d83700 1 -- 172.21.15.36:0/2776006951 shutdown_connections 2022-06-17T12:57:03.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.234+0000 7f6ca4d83700 1 -- 172.21.15.36:0/2776006951 wait complete. 2022-06-17T12:57:03.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.238+0000 7f6ca4d83700 1 Processor -- start 2022-06-17T12:57:03.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.238+0000 7f6ca4d83700 1 -- start start 2022-06-17T12:57:03.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.238+0000 7f6ca4d83700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca0060ca0 0x7f6ca0065430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:03.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.238+0000 7f6ca4d83700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ca00b27b0 0x7f6ca01454a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:03.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.238+0000 7f6ca4d83700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ca0147bd0 0x7f6ca0145990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:03.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.238+0000 7f6ca4d83700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca0064c10 con 0x7f6ca0060ca0 2022-06-17T12:57:03.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.238+0000 7f6ca4d83700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca0064020 con 0x7f6ca00b27b0 2022-06-17T12:57:03.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.238+0000 7f6ca4d83700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca0156660 con 0x7f6ca0147bd0 2022-06-17T12:57:03.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.238+0000 7f6c9f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca0060ca0 0x7f6ca0065430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:03.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.238+0000 7f6c9f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca0060ca0 0x7f6ca0065430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40372/0 (socket says 172.21.15.36:40372) 2022-06-17T12:57:03.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.238+0000 7f6c9f7fe700 1 -- 172.21.15.36:0/2297707850 learned_addr learned my addr 172.21.15.36:0/2297707850 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:03.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.238+0000 7f6c9ffff700 1 --2- 172.21.15.36:0/2297707850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ca0147bd0 0x7f6ca0145990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:03.278 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.238+0000 7f6c9effd700 1 --2- 172.21.15.36:0/2297707850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ca00b27b0 0x7f6ca01454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:03.279 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.238+0000 7f6c9f7fe700 1 -- 172.21.15.36:0/2297707850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ca0147bd0 msgr2=0x7f6ca0145990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:03.279 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.238+0000 7f6c9f7fe700 1 --2- 172.21.15.36:0/2297707850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ca0147bd0 0x7f6ca0145990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:03.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.238+0000 7f6c9f7fe700 1 -- 172.21.15.36:0/2297707850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ca00b27b0 msgr2=0x7f6ca01454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:03.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.238+0000 7f6c9f7fe700 1 --2- 172.21.15.36:0/2297707850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ca00b27b0 0x7f6ca01454a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:03.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.238+0000 7f6c9f7fe700 1 -- 172.21.15.36:0/2297707850 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c90009530 con 0x7f6ca0060ca0 2022-06-17T12:57:03.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.238+0000 7f6c9f7fe700 1 --2- 172.21.15.36:0/2297707850 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca0060ca0 0x7f6ca0065430 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 rx=0x7f6ca014e8d0 tx=0x7f6c88008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:03.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.238+0000 7f6c9cff9700 1 -- 172.21.15.36:0/2297707850 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c880106e0 con 0x7f6ca0060ca0 2022-06-17T12:57:03.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.238+0000 7f6c9cff9700 1 -- 172.21.15.36:0/2297707850 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c88010840 con 0x7f6ca0060ca0 2022-06-17T12:57:03.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.238+0000 7f6ca4d83700 1 -- 172.21.15.36:0/2297707850 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca0062390 con 0x7f6ca0060ca0 2022-06-17T12:57:03.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.238+0000 7f6c9cff9700 1 -- 172.21.15.36:0/2297707850 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c88010ba0 con 0x7f6ca0060ca0 2022-06-17T12:57:03.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.238+0000 7f6ca4d83700 1 -- 172.21.15.36:0/2297707850 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca00644f0 con 0x7f6ca0060ca0 2022-06-17T12:57:03.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.242+0000 7f6ca4d83700 1 -- 172.21.15.36:0/2297707850 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca01567a0 con 0x7f6ca0060ca0 2022-06-17T12:57:03.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.242+0000 7f6c9cff9700 1 -- 172.21.15.36:0/2297707850 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6c8801a030 con 0x7f6ca0060ca0 2022-06-17T12:57:03.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.242+0000 7f6c9cff9700 1 --2- 172.21.15.36:0/2297707850 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c84053a40 0x7f6c84055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:03.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.242+0000 7f6c9effd700 1 --2- 172.21.15.36:0/2297707850 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c84053a40 0x7f6c84055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:03.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.242+0000 7f6c9cff9700 1 -- 172.21.15.36:0/2297707850 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f6c8806bdd0 con 0x7f6ca0060ca0 2022-06-17T12:57:03.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.242+0000 7f6c9effd700 1 --2- 172.21.15.36:0/2297707850 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c84053a40 0x7f6c84055f00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f6c90019000 tx=0x7f6c90016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:03.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.254+0000 7f6c9cff9700 1 -- 172.21.15.36:0/2297707850 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6c88042a10 con 0x7f6ca0060ca0 2022-06-17T12:57:03.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.414+0000 7f6ca4d83700 1 -- 172.21.15.36:0/2297707850 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6ca0057410 con 0x7f6ca0060ca0 2022-06-17T12:57:03.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.770+0000 7f6c9cff9700 1 -- 172.21.15.36:0/2297707850 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v80) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6c88046030 con 0x7f6ca0060ca0 2022-06-17T12:57:03.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.774+0000 7f6ca4d83700 1 -- 172.21.15.36:0/2297707850 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c84053a40 msgr2=0x7f6c84055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:03.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.774+0000 7f6ca4d83700 1 --2- 172.21.15.36:0/2297707850 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c84053a40 0x7f6c84055f00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f6c90019000 tx=0x7f6c90016000).stop 2022-06-17T12:57:03.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.774+0000 7f6ca4d83700 1 -- 172.21.15.36:0/2297707850 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca0060ca0 msgr2=0x7f6ca0065430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:03.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.774+0000 7f6ca4d83700 1 --2- 172.21.15.36:0/2297707850 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca0060ca0 0x7f6ca0065430 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 rx=0x7f6ca014e8d0 tx=0x7f6c88008040).stop 2022-06-17T12:57:03.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.774+0000 7f6ca4d83700 1 -- 172.21.15.36:0/2297707850 shutdown_connections 2022-06-17T12:57:03.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.774+0000 7f6ca4d83700 1 --2- 172.21.15.36:0/2297707850 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6c84053a40 0x7f6c84055f00 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:03.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.774+0000 7f6ca4d83700 1 --2- 172.21.15.36:0/2297707850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ca0147bd0 0x7f6ca0145990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:03.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.774+0000 7f6ca4d83700 1 --2- 172.21.15.36:0/2297707850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ca00b27b0 0x7f6ca01454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:03.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.774+0000 7f6ca4d83700 1 --2- 172.21.15.36:0/2297707850 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6ca0060ca0 0x7f6ca0065430 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:03.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.774+0000 7f6ca4d83700 1 -- 172.21.15.36:0/2297707850 >> 172.21.15.36:0/2297707850 conn(0x7f6ca00b5d10 msgr2=0x7f6ca00af930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:03.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.774+0000 7f6ca4d83700 1 -- 172.21.15.36:0/2297707850 shutdown_connections 2022-06-17T12:57:03.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.774+0000 7f6ca4d83700 1 -- 172.21.15.36:0/2297707850 wait complete. 2022-06-17T12:57:03.784 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-17T12:57:03.943 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.950+0000 7f5c6cd88700 1 Processor -- start 2022-06-17T12:57:03.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.950+0000 7f5c6cd88700 1 -- start start 2022-06-17T12:57:03.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.950+0000 7f5c6cd88700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5c680601c0 0x7f5c680b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:03.944 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.950+0000 7f5c6cd88700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c680adf40 0x7f5c68143840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:03.945 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c6cd88700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c680605a0 con 0x7f5c68143d30 2022-06-17T12:57:03.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c6cd88700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c68143100 con 0x7f5c680601c0 2022-06-17T12:57:03.946 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c6cd88700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c681433a0 con 0x7f5c680adf40 2022-06-17T12:57:03.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c66d9d700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f5c68143d30 0x7f5c68140300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57916/0 (socket says 172.21.15.36:57916) 2022-06-17T12:57:03.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c66d9d700 1 -- 172.21.15.36:0/3773571227 learned_addr learned my addr 172.21.15.36:0/3773571227 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:03.948 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c6659c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5c680601c0 0x7f5c680b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:03.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c65d9b700 1 --2- 172.21.15.36:0/3773571227 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c680adf40 0x7f5c68143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:03.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c6559a700 1 -- 172.21.15.36:0/3773571227 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2390053198 0 0) 0x7f5c680605a0 con 0x7f5c68143d30 2022-06-17T12:57:03.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c6559a700 1 -- 172.21.15.36:0/3773571227 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c48003260 con 0x7f5c68143d30 2022-06-17T12:57:03.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c6659c700 1 -- 172.21.15.36:0/3773571227 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c680adf40 msgr2=0x7f5c68143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:03.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c6659c700 1 --2- 172.21.15.36:0/3773571227 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c680adf40 0x7f5c68143840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:03.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c6659c700 1 -- 172.21.15.36:0/3773571227 >> v1:172.21.15.36:6789/0 conn(0x7f5c68143d30 legacy=0x7f5c68140300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:03.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c6659c700 1 -- 172.21.15.36:0/3773571227 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c68140a10 con 0x7f5c680601c0 2022-06-17T12:57:03.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c6659c700 1 --2- 172.21.15.36:0/3773571227 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c680601c0 0x7f5c680b2a90 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7f5c58002f70 tx=0x7f5c58006df0).ready entity=mon.0 client_cookie=fab4aba86c14ab81 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:03.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c6559a700 1 -- 172.21.15.36:0/3773571227 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c5800d4a0 con 0x7f5c680601c0 2022-06-17T12:57:03.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c6559a700 1 -- 172.21.15.36:0/3773571227 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c58000e90 con 0x7f5c680601c0 2022-06-17T12:57:03.982 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c6559a700 1 -- 172.21.15.36:0/3773571227 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c5800d7f0 con 0x7f5c680601c0 2022-06-17T12:57:03.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c6cd88700 1 -- 172.21.15.36:0/3773571227 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c680601c0 msgr2=0x7f5c680b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:03.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c6cd88700 1 --2- 172.21.15.36:0/3773571227 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c680601c0 0x7f5c680b2a90 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7f5c58002f70 tx=0x7f5c58006df0).stop 2022-06-17T12:57:03.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c6cd88700 1 -- 172.21.15.36:0/3773571227 shutdown_connections 2022-06-17T12:57:03.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c6cd88700 1 --2- 172.21.15.36:0/3773571227 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c680adf40 0x7f5c68143840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:03.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c6cd88700 1 --2- 172.21.15.36:0/3773571227 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c680601c0 0x7f5c680b2a90 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:03.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c6cd88700 1 -- 172.21.15.36:0/3773571227 >> 172.21.15.36:0/3773571227 conn(0x7f5c68060990 msgr2=0x7f5c6805f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:04.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c6cd88700 1 -- 172.21.15.36:0/3773571227 shutdown_connections 2022-06-17T12:57:04.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c6cd88700 1 -- 172.21.15.36:0/3773571227 wait complete. 2022-06-17T12:57:04.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c6cd88700 1 Processor -- start 2022-06-17T12:57:04.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.954+0000 7f5c6cd88700 1 -- start start 2022-06-17T12:57:04.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.958+0000 7f5c6cd88700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c680601c0 0x7f5c68146590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:04.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.958+0000 7f5c6cd88700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c680adf40 0x7f5c68066a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:04.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.958+0000 7f5c6cd88700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c68143d30 0x7f5c68152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:04.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.958+0000 7f5c6cd88700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c68146bc0 con 0x7f5c680601c0 2022-06-17T12:57:04.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.958+0000 7f5c6cd88700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c68146e60 con 0x7f5c68143d30 2022-06-17T12:57:04.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.958+0000 7f5c6cd88700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c68147100 con 0x7f5c680adf40 2022-06-17T12:57:04.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.958+0000 7f5c6659c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c680601c0 0x7f5c68146590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:04.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.958+0000 7f5c6659c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c680601c0 0x7f5c68146590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40388/0 (socket says 172.21.15.36:40388) 2022-06-17T12:57:04.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.958+0000 7f5c6659c700 1 -- 172.21.15.36:0/1103816507 learned_addr learned my addr 172.21.15.36:0/1103816507 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:04.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.958+0000 7f5c66d9d700 1 --2- 172.21.15.36:0/1103816507 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c68143d30 0x7f5c68152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:04.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.958+0000 7f5c65d9b700 1 --2- 172.21.15.36:0/1103816507 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c680adf40 0x7f5c68066a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:04.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.958+0000 7f5c6659c700 1 -- 172.21.15.36:0/1103816507 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c680adf40 msgr2=0x7f5c68066a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:04.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.958+0000 7f5c6659c700 1 --2- 172.21.15.36:0/1103816507 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c680adf40 0x7f5c68066a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:04.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.958+0000 7f5c6659c700 1 -- 172.21.15.36:0/1103816507 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c68143d30 msgr2=0x7f5c68152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:04.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.958+0000 7f5c6659c700 1 --2- 172.21.15.36:0/1103816507 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c68143d30 0x7f5c68152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:04.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.958+0000 7f5c6659c700 1 -- 172.21.15.36:0/1103816507 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5800b180 con 0x7f5c680601c0 2022-06-17T12:57:04.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.958+0000 7f5c65d9b700 1 --2- 172.21.15.36:0/1103816507 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c680adf40 0x7f5c68066a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:57:04.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.958+0000 7f5c6659c700 1 --2- 172.21.15.36:0/1103816507 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c680601c0 0x7f5c68146590 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7f5c58002f70 tx=0x7f5c58014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:04.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.958+0000 7f5c5f7fe700 1 -- 172.21.15.36:0/1103816507 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c580139c0 con 0x7f5c680601c0 2022-06-17T12:57:04.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.958+0000 7f5c5f7fe700 1 -- 172.21.15.36:0/1103816507 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c58013b20 con 0x7f5c680601c0 2022-06-17T12:57:04.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.958+0000 7f5c6cd88700 1 -- 172.21.15.36:0/1103816507 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c68152220 con 0x7f5c680601c0 2022-06-17T12:57:04.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.958+0000 7f5c6cd88700 1 -- 172.21.15.36:0/1103816507 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c68145ba0 con 0x7f5c680601c0 2022-06-17T12:57:04.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.958+0000 7f5c5f7fe700 1 -- 172.21.15.36:0/1103816507 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c58013e80 con 0x7f5c680601c0 2022-06-17T12:57:04.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.962+0000 7f5c6cd88700 1 -- 172.21.15.36:0/1103816507 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c68063320 con 0x7f5c680601c0 2022-06-17T12:57:04.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.962+0000 7f5c5f7fe700 1 -- 172.21.15.36:0/1103816507 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f5c58025030 con 0x7f5c680601c0 2022-06-17T12:57:04.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.962+0000 7f5c5f7fe700 1 --2- 172.21.15.36:0/1103816507 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c4c0539d0 0x7f5c4c055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:04.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.962+0000 7f5c5f7fe700 1 -- 172.21.15.36:0/1103816507 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f5c5806cd00 con 0x7f5c680601c0 2022-06-17T12:57:04.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.962+0000 7f5c65d9b700 1 --2- 172.21.15.36:0/1103816507 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c4c0539d0 0x7f5c4c055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:04.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.962+0000 7f5c65d9b700 1 --2- 172.21.15.36:0/1103816507 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c4c0539d0 0x7f5c4c055e90 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f5c5000b000 tx=0x7f5c50008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:04.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:03.966+0000 7f5c5f7fe700 1 -- 172.21.15.36:0/1103816507 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5c58070030 con 0x7f5c680601c0 2022-06-17T12:57:04.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.094+0000 7f5c6cd88700 1 -- 172.21.15.36:0/1103816507 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5c68057410 con 0x7f5c680601c0 2022-06-17T12:57:04.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.822+0000 7f5c5f7fe700 1 -- 172.21.15.36:0/1103816507 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v143) v1 ==== 185+0+0 (secure 0 0 0) 0x7f5c58023070 con 0x7f5c680601c0 2022-06-17T12:57:04.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.826+0000 7f5c6cd88700 1 -- 172.21.15.36:0/1103816507 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c4c0539d0 msgr2=0x7f5c4c055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:04.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.826+0000 7f5c6cd88700 1 --2- 172.21.15.36:0/1103816507 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c4c0539d0 0x7f5c4c055e90 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f5c5000b000 tx=0x7f5c50008040).stop 2022-06-17T12:57:04.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.826+0000 7f5c6cd88700 1 -- 172.21.15.36:0/1103816507 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c680601c0 msgr2=0x7f5c68146590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:04.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.826+0000 7f5c6cd88700 1 --2- 172.21.15.36:0/1103816507 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c680601c0 0x7f5c68146590 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7f5c58002f70 tx=0x7f5c58014000).stop 2022-06-17T12:57:04.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.826+0000 7f5c6cd88700 1 -- 172.21.15.36:0/1103816507 shutdown_connections 2022-06-17T12:57:04.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.826+0000 7f5c6cd88700 1 --2- 172.21.15.36:0/1103816507 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c4c0539d0 0x7f5c4c055e90 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:04.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.826+0000 7f5c6cd88700 1 --2- 172.21.15.36:0/1103816507 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c68143d30 0x7f5c68152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:04.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.826+0000 7f5c6cd88700 1 --2- 172.21.15.36:0/1103816507 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c680adf40 0x7f5c68066a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:04.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.826+0000 7f5c6cd88700 1 --2- 172.21.15.36:0/1103816507 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c680601c0 0x7f5c68146590 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:04.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.826+0000 7f5c6cd88700 1 -- 172.21.15.36:0/1103816507 >> 172.21.15.36:0/1103816507 conn(0x7f5c68060990 msgr2=0x7f5c6813ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:04.842 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.826+0000 7f5c6cd88700 1 -- 172.21.15.36:0/1103816507 shutdown_connections 2022-06-17T12:57:04.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.826+0000 7f5c6cd88700 1 -- 172.21.15.36:0/1103816507 wait complete. 2022-06-17T12:57:04.843 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' removed 2022-06-17T12:57:04.844 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-17T12:57:04.979 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.994+0000 7f18ffe79700 1 Processor -- start 2022-06-17T12:57:04.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.994+0000 7f18ffe79700 1 -- start start 2022-06-17T12:57:04.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18ffe79700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f18f80b4e10 0x7f18f80b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:04.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18ffe79700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18f80b21d0 con 0x7f18f80b31c0 2022-06-17T12:57:04.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18ffe79700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18f80b2430 con 0x7f18f80620d0 2022-06-17T12:57:04.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18ffe79700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f80b26d0 con 0x7f18f80b4e10 2022-06-17T12:57:04.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18fe416700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f18f80b31c0 0x7f18f8141ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57930/0 (socket says 172.21.15.36:57930) 2022-06-17T12:57:04.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18fe416700 1 -- 172.21.15.36:0/607014088 learned_addr learned my addr 172.21.15.36:0/607014088 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:04.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18fd414700 1 --2- 172.21.15.36:0/607014088 >> v2:172.21.15.36:3300/0 conn(0x7f18f80b4e10 0x7f18f80b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:04.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18fcc13700 1 -- 172.21.15.36:0/607014088 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2124867166 0 0) 0x7f18f80b21d0 con 0x7f18f80b31c0 2022-06-17T12:57:04.991 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18fcc13700 1 -- 172.21.15.36:0/607014088 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18e0003260 con 0x7f18f80b31c0 2022-06-17T12:57:04.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18fcc13700 1 -- 172.21.15.36:0/607014088 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1280951726 0 0) 0x7f18f80b2430 con 0x7f18f80620d0 2022-06-17T12:57:04.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18fcc13700 1 -- 172.21.15.36:0/607014088 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18f80b21d0 con 0x7f18f80620d0 2022-06-17T12:57:04.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18fcc13700 1 -- 172.21.15.36:0/607014088 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3621569335 0 0) 0x7f18e0003260 con 0x7f18f80b31c0 2022-06-17T12:57:04.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18fd414700 1 -- 172.21.15.36:0/607014088 >> v1:172.21.15.70:6789/0 conn(0x7f18f80620d0 legacy=0x7f18f80b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:04.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18fd414700 1 -- 172.21.15.36:0/607014088 >> v1:172.21.15.36:6789/0 conn(0x7f18f80b31c0 legacy=0x7f18f8141ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:04.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18fd414700 1 -- 172.21.15.36:0/607014088 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f80b43f0 con 0x7f18f80b4e10 2022-06-17T12:57:04.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18fd414700 1 --2- 172.21.15.36:0/607014088 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18f80b4e10 0x7f18f80b5bc0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 rx=0x7f18f0001b00 tx=0x7f18ec006df0).ready entity=mon.0 client_cookie=fdecb06984b4d198 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:04.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18fcc13700 1 -- 172.21.15.36:0/607014088 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18f0001960 con 0x7f18f80b4e10 2022-06-17T12:57:04.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18fcc13700 1 -- 172.21.15.36:0/607014088 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ec000e90 con 0x7f18f80b4e10 2022-06-17T12:57:04.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18fcc13700 1 -- 172.21.15.36:0/607014088 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18ec012460 con 0x7f18f80b4e10 2022-06-17T12:57:04.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18ffe79700 1 -- 172.21.15.36:0/607014088 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18f80b4e10 msgr2=0x7f18f80b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:04.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18ffe79700 1 --2- 172.21.15.36:0/607014088 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18f80b4e10 0x7f18f80b5bc0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 rx=0x7f18f0001b00 tx=0x7f18ec006df0).stop 2022-06-17T12:57:04.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18ffe79700 1 -- 172.21.15.36:0/607014088 shutdown_connections 2022-06-17T12:57:04.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18ffe79700 1 --2- 172.21.15.36:0/607014088 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18f80b4e10 0x7f18f80b5bc0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:04.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18ffe79700 1 -- 172.21.15.36:0/607014088 >> 172.21.15.36:0/607014088 conn(0x7f18f80b9850 msgr2=0x7f18f80b9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:04.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18ffe79700 1 -- 172.21.15.36:0/607014088 shutdown_connections 2022-06-17T12:57:04.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:04.998+0000 7f18ffe79700 1 -- 172.21.15.36:0/607014088 wait complete. 2022-06-17T12:57:04.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.002+0000 7f18ffe79700 1 Processor -- start 2022-06-17T12:57:05.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.002+0000 7f18ffe79700 1 -- start start 2022-06-17T12:57:05.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.002+0000 7f18ffe79700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18f80620d0 0x7f18f8112e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:05.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.002+0000 7f18ffe79700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18f80b31c0 0x7f18f8155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:05.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.002+0000 7f18ffe79700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18f80b4e10 0x7f18f80b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:05.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.002+0000 7f18ffe79700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f81561d0 con 0x7f18f80b31c0 2022-06-17T12:57:05.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.002+0000 7f18ffe79700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f8156470 con 0x7f18f80b4e10 2022-06-17T12:57:05.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.002+0000 7f18ffe79700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f8156710 con 0x7f18f80620d0 2022-06-17T12:57:05.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.006+0000 7f18fd414700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18f80b31c0 0x7f18f8155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:05.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.006+0000 7f18fe416700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18f80b4e10 0x7f18f80b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:05.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.006+0000 7f18fd414700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18f80b31c0 0x7f18f8155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40404/0 (socket says 172.21.15.36:40404) 2022-06-17T12:57:05.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.006+0000 7f18fd414700 1 -- 172.21.15.36:0/2161511702 learned_addr learned my addr 172.21.15.36:0/2161511702 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:05.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.006+0000 7f18fdc15700 1 --2- 172.21.15.36:0/2161511702 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18f80620d0 0x7f18f8112e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:05.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.006+0000 7f18fd414700 1 -- 172.21.15.36:0/2161511702 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18f80620d0 msgr2=0x7f18f8112e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:05.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.006+0000 7f18fd414700 1 --2- 172.21.15.36:0/2161511702 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18f80620d0 0x7f18f8112e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:05.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.006+0000 7f18fd414700 1 -- 172.21.15.36:0/2161511702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18f80b4e10 msgr2=0x7f18f80b8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:05.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.006+0000 7f18fd414700 1 --2- 172.21.15.36:0/2161511702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18f80b4e10 0x7f18f80b8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:05.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.006+0000 7f18fd414700 1 -- 172.21.15.36:0/2161511702 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec00b180 con 0x7f18f80b31c0 2022-06-17T12:57:05.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.006+0000 7f18fd414700 1 --2- 172.21.15.36:0/2161511702 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18f80b31c0 0x7f18f8155ba0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0x7f18f0001b00 tx=0x7f18ec014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:05.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.006+0000 7f18eaffd700 1 -- 172.21.15.36:0/2161511702 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18ec00d8a0 con 0x7f18f80b31c0 2022-06-17T12:57:05.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.006+0000 7f18eaffd700 1 -- 172.21.15.36:0/2161511702 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ec00da00 con 0x7f18f80b31c0 2022-06-17T12:57:05.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.006+0000 7f18ffe79700 1 -- 172.21.15.36:0/2161511702 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f8140290 con 0x7f18f80b31c0 2022-06-17T12:57:05.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.006+0000 7f18eaffd700 1 -- 172.21.15.36:0/2161511702 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18ec00dd60 con 0x7f18f80b31c0 2022-06-17T12:57:05.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.006+0000 7f18ffe79700 1 -- 172.21.15.36:0/2161511702 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f81407e0 con 0x7f18f80b31c0 2022-06-17T12:57:05.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.006+0000 7f18eaffd700 1 -- 172.21.15.36:0/2161511702 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f18ec025030 con 0x7f18f80b31c0 2022-06-17T12:57:05.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.010+0000 7f18ffe79700 1 -- 172.21.15.36:0/2161511702 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f8057410 con 0x7f18f80b31c0 2022-06-17T12:57:05.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.010+0000 7f18eaffd700 1 --2- 172.21.15.36:0/2161511702 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f18d8053a40 0x7f18d8055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:05.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.010+0000 7f18eaffd700 1 -- 172.21.15.36:0/2161511702 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f18ec06c940 con 0x7f18f80b31c0 2022-06-17T12:57:05.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.010+0000 7f18fdc15700 1 --2- 172.21.15.36:0/2161511702 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f18d8053a40 0x7f18d8055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:05.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.010+0000 7f18fdc15700 1 --2- 172.21.15.36:0/2161511702 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f18d8053a40 0x7f18d8055f00 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f18f400d000 tx=0x7f18f4007ab0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:05.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.014+0000 7f18eaffd700 1 -- 172.21.15.36:0/2161511702 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f18ec0438e0 con 0x7f18f80b31c0 2022-06-17T12:57:05.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.146+0000 7f18ffe79700 1 -- 172.21.15.36:0/2161511702 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f18f80b0640 con 0x7f18f80b31c0 2022-06-17T12:57:05.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.830+0000 7f18eaffd700 1 -- 172.21.15.36:0/2161511702 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v144) v1 ==== 173+0+0 (secure 0 0 0) 0x7f18ec023070 con 0x7f18f80b31c0 2022-06-17T12:57:05.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.838+0000 7f18ffe79700 1 -- 172.21.15.36:0/2161511702 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f18d8053a40 msgr2=0x7f18d8055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:05.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.838+0000 7f18ffe79700 1 --2- 172.21.15.36:0/2161511702 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f18d8053a40 0x7f18d8055f00 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f18f400d000 tx=0x7f18f4007ab0).stop 2022-06-17T12:57:05.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.838+0000 7f18ffe79700 1 -- 172.21.15.36:0/2161511702 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18f80b31c0 msgr2=0x7f18f8155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:05.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.838+0000 7f18ffe79700 1 --2- 172.21.15.36:0/2161511702 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18f80b31c0 0x7f18f8155ba0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0x7f18f0001b00 tx=0x7f18ec014000).stop 2022-06-17T12:57:05.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.838+0000 7f18ffe79700 1 -- 172.21.15.36:0/2161511702 shutdown_connections 2022-06-17T12:57:05.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.838+0000 7f18ffe79700 1 --2- 172.21.15.36:0/2161511702 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f18d8053a40 0x7f18d8055f00 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:05.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.838+0000 7f18ffe79700 1 --2- 172.21.15.36:0/2161511702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18f80b4e10 0x7f18f80b8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:05.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.838+0000 7f18ffe79700 1 --2- 172.21.15.36:0/2161511702 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18f80b31c0 0x7f18f8155ba0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:05.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.838+0000 7f18ffe79700 1 --2- 172.21.15.36:0/2161511702 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18f80620d0 0x7f18f8112e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:05.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.838+0000 7f18ffe79700 1 -- 172.21.15.36:0/2161511702 >> 172.21.15.36:0/2161511702 conn(0x7f18f80b9850 msgr2=0x7f18f8141e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:05.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.838+0000 7f18ffe79700 1 -- 172.21.15.36:0/2161511702 shutdown_connections 2022-06-17T12:57:05.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.838+0000 7f18ffe79700 1 -- 172.21.15.36:0/2161511702 wait complete. 2022-06-17T12:57:05.859 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' removed 2022-06-17T12:57:05.861 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-06-17T12:57:06.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.994+0000 7f50dc71d700 1 Processor -- start 2022-06-17T12:57:06.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.994+0000 7f50dc71d700 1 -- start start 2022-06-17T12:57:06.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.994+0000 7f50dc71d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f50d40b4120 0x7f50d4146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:06.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.994+0000 7f50dc71d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f50d4147270 0x7f50d4140100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:06.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.994+0000 7f50dc71d700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d40b68d0 con 0x7f50d40b64f0 2022-06-17T12:57:06.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.994+0000 7f50dc71d700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d41476b0 con 0x7f50d4147270 2022-06-17T12:57:06.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.994+0000 7f50dc71d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d40b6e20 con 0x7f50d40b4120 2022-06-17T12:57:06.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.994+0000 7f50d9cb8700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f50d40b4120 0x7f50d4146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:06.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.994+0000 7f50dacba700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f50d4147270 0x7f50d4140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:06.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.994+0000 7f50da4b9700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f50d40b64f0 0x7f50d4143150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:53674/0 (socket says 172.21.15.36:53674) 2022-06-17T12:57:06.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.994+0000 7f50da4b9700 1 -- 172.21.15.36:0/43922356 learned_addr learned my addr 172.21.15.36:0/43922356 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:06.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.998+0000 7f50dacba700 1 -- 172.21.15.36:0/43922356 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f50d40b4120 msgr2=0x7f50d4146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:06.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.998+0000 7f50dacba700 1 --2- 172.21.15.36:0/43922356 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f50d40b4120 0x7f50d4146d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:06.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.998+0000 7f50dacba700 1 -- 172.21.15.36:0/43922356 >> v1:172.21.15.70:6789/0 conn(0x7f50d40b64f0 legacy=0x7f50d4143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:06.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.998+0000 7f50dacba700 1 -- 172.21.15.36:0/43922356 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d41468e0 con 0x7f50d4147270 2022-06-17T12:57:06.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.998+0000 7f50dacba700 1 --2- 172.21.15.36:0/43922356 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50d4147270 0x7f50d4140100 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f50cc002f70 tx=0x7f50cc006df0).ready entity=mon.1 client_cookie=d72feef8bb27ec32 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:06.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.998+0000 7f50d94b7700 1 -- 172.21.15.36:0/43922356 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50cc00d570 con 0x7f50d4147270 2022-06-17T12:57:06.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.998+0000 7f50d94b7700 1 -- 172.21.15.36:0/43922356 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50cc000e90 con 0x7f50d4147270 2022-06-17T12:57:06.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.998+0000 7f50d94b7700 1 -- 172.21.15.36:0/43922356 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50cc00d8c0 con 0x7f50d4147270 2022-06-17T12:57:06.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.998+0000 7f50dc71d700 1 -- 172.21.15.36:0/43922356 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50d4147270 msgr2=0x7f50d4140100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:06.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.998+0000 7f50dc71d700 1 --2- 172.21.15.36:0/43922356 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50d4147270 0x7f50d4140100 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f50cc002f70 tx=0x7f50cc006df0).stop 2022-06-17T12:57:06.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.998+0000 7f50dc71d700 1 -- 172.21.15.36:0/43922356 shutdown_connections 2022-06-17T12:57:06.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.998+0000 7f50dc71d700 1 --2- 172.21.15.36:0/43922356 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50d4147270 0x7f50d4140100 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:06.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.998+0000 7f50dc71d700 1 --2- 172.21.15.36:0/43922356 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f50d40b4120 0x7f50d4146d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:06.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.998+0000 7f50dc71d700 1 -- 172.21.15.36:0/43922356 >> 172.21.15.36:0/43922356 conn(0x7f50d40620f0 msgr2=0x7f50d4062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:06.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.998+0000 7f50dc71d700 1 -- 172.21.15.36:0/43922356 shutdown_connections 2022-06-17T12:57:06.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.998+0000 7f50dc71d700 1 -- 172.21.15.36:0/43922356 wait complete. 2022-06-17T12:57:06.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.998+0000 7f50dc71d700 1 Processor -- start 2022-06-17T12:57:06.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.998+0000 7f50dc71d700 1 -- start start 2022-06-17T12:57:06.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.998+0000 7f50dc71d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50d40b4120 0x7f50d40b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:06.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:05.998+0000 7f50dc71d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f50d40b64f0 0x7f50d4143140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:06.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.002+0000 7f50dc71d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f50d4147270 0x7f50d4143630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:06.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.002+0000 7f50dc71d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d4143b20 con 0x7f50d4147270 2022-06-17T12:57:06.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.002+0000 7f50dc71d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d4113110 con 0x7f50d40b4120 2022-06-17T12:57:06.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.002+0000 7f50dc71d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d41567d0 con 0x7f50d40b64f0 2022-06-17T12:57:06.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.002+0000 7f50da4b9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50d40b4120 0x7f50d40b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:06.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.002+0000 7f50da4b9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50d40b4120 0x7f50d40b4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:60798/0 (socket says 172.21.15.36:60798) 2022-06-17T12:57:06.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.002+0000 7f50da4b9700 1 -- 172.21.15.36:0/860908434 learned_addr learned my addr 172.21.15.36:0/860908434 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:06.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.002+0000 7f50dacba700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f50d4147270 0x7f50d4143630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:06.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.002+0000 7f50d9cb8700 1 --2- 172.21.15.36:0/860908434 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f50d40b64f0 0x7f50d4143140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:06.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.002+0000 7f50da4b9700 1 -- 172.21.15.36:0/860908434 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f50d40b64f0 msgr2=0x7f50d4143140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:06.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.002+0000 7f50da4b9700 1 --2- 172.21.15.36:0/860908434 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f50d40b64f0 0x7f50d4143140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:06.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.002+0000 7f50da4b9700 1 -- 172.21.15.36:0/860908434 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f50d4147270 msgr2=0x7f50d4143630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:06.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.002+0000 7f50da4b9700 1 --2- 172.21.15.36:0/860908434 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f50d4147270 0x7f50d4143630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:06.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.002+0000 7f50da4b9700 1 -- 172.21.15.36:0/860908434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50cc00b180 con 0x7f50d40b4120 2022-06-17T12:57:06.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.002+0000 7f50da4b9700 1 --2- 172.21.15.36:0/860908434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50d40b4120 0x7f50d40b4d80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f50d4061d20 tx=0x7f50c40014a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:06.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.002+0000 7f50d37fe700 1 -- 172.21.15.36:0/860908434 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50c4005ba0 con 0x7f50d40b4120 2022-06-17T12:57:06.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.002+0000 7f50d37fe700 1 -- 172.21.15.36:0/860908434 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c4005d00 con 0x7f50d40b4120 2022-06-17T12:57:06.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.002+0000 7f50dc71d700 1 -- 172.21.15.36:0/860908434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d4156a70 con 0x7f50d40b4120 2022-06-17T12:57:06.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.002+0000 7f50d37fe700 1 -- 172.21.15.36:0/860908434 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50c40072c0 con 0x7f50d40b4120 2022-06-17T12:57:06.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.002+0000 7f50dc71d700 1 -- 172.21.15.36:0/860908434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d40b5270 con 0x7f50d40b4120 2022-06-17T12:57:06.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.002+0000 7f50dc71d700 1 -- 172.21.15.36:0/860908434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d4156c50 con 0x7f50d40b4120 2022-06-17T12:57:06.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.002+0000 7f50d37fe700 1 -- 172.21.15.36:0/860908434 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f50c4010070 con 0x7f50d40b4120 2022-06-17T12:57:06.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.002+0000 7f50d37fe700 1 --2- 172.21.15.36:0/860908434 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f50b4053ad0 0x7f50b4055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:06.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.006+0000 7f50d37fe700 1 -- 172.21.15.36:0/860908434 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f50c406b700 con 0x7f50d40b4120 2022-06-17T12:57:06.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.006+0000 7f50d9cb8700 1 --2- 172.21.15.36:0/860908434 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f50b4053ad0 0x7f50b4055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:06.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.006+0000 7f50d9cb8700 1 --2- 172.21.15.36:0/860908434 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f50b4053ad0 0x7f50b4055f90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f50c800b000 tx=0x7f50c8006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:06.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.014+0000 7f50d37fe700 1 -- 172.21.15.36:0/860908434 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f50c40426d0 con 0x7f50d40b4120 2022-06-17T12:57:06.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.138+0000 7f50dc71d700 1 -- 172.21.15.36:0/860908434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f50d4057410 con 0x7f50d40b4120 2022-06-17T12:57:06.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.150+0000 7f50d37fe700 1 -- 172.21.15.36:0/860908434 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f50c4045cf0 con 0x7f50d40b4120 2022-06-17T12:57:06.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.150+0000 7f50dc71d700 1 -- 172.21.15.36:0/860908434 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f50b4053ad0 msgr2=0x7f50b4055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:06.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.150+0000 7f50dc71d700 1 --2- 172.21.15.36:0/860908434 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f50b4053ad0 0x7f50b4055f90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f50c800b000 tx=0x7f50c8006df0).stop 2022-06-17T12:57:06.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.150+0000 7f50dc71d700 1 -- 172.21.15.36:0/860908434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50d40b4120 msgr2=0x7f50d40b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:06.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.150+0000 7f50dc71d700 1 --2- 172.21.15.36:0/860908434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50d40b4120 0x7f50d40b4d80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f50d4061d20 tx=0x7f50c40014a0).stop 2022-06-17T12:57:06.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.150+0000 7f50dc71d700 1 -- 172.21.15.36:0/860908434 shutdown_connections 2022-06-17T12:57:06.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.150+0000 7f50dc71d700 1 --2- 172.21.15.36:0/860908434 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f50b4053ad0 0x7f50b4055f90 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:06.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.150+0000 7f50dc71d700 1 --2- 172.21.15.36:0/860908434 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f50d4147270 0x7f50d4143630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:06.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.150+0000 7f50dc71d700 1 --2- 172.21.15.36:0/860908434 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f50d40b64f0 0x7f50d4143140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:06.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.150+0000 7f50dc71d700 1 --2- 172.21.15.36:0/860908434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50d40b4120 0x7f50d40b4d80 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:06.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.150+0000 7f50dc71d700 1 -- 172.21.15.36:0/860908434 >> 172.21.15.36:0/860908434 conn(0x7f50d40620f0 msgr2=0x7f50d4146050 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:06.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.154+0000 7f50dc71d700 1 -- 172.21.15.36:0/860908434 shutdown_connections 2022-06-17T12:57:06.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.154+0000 7f50dc71d700 1 -- 172.21.15.36:0/860908434 wait complete. 2022-06-17T12:57:06.158 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime' 2022-06-17T12:57:06.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.290+0000 7f7736ef5700 1 Processor -- start 2022-06-17T12:57:06.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.290+0000 7f7736ef5700 1 -- start start 2022-06-17T12:57:06.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.290+0000 7f7736ef5700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f77300620d0 0x7f77300b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:06.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.290+0000 7f7736ef5700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f77300b31c0 0x7f77300b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:06.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.290+0000 7f7736ef5700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77300b21d0 con 0x7f77300b4700 2022-06-17T12:57:06.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.290+0000 7f7736ef5700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77300b2430 con 0x7f77300620d0 2022-06-17T12:57:06.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.290+0000 7f7736ef5700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77300b26d0 con 0x7f77300b31c0 2022-06-17T12:57:06.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.290+0000 7f7734c91700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f77300620d0 0x7f77300b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:06.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.290+0000 7f7734c91700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f77300620d0 0x7f77300b5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40424/0 (socket says 172.21.15.36:40424) 2022-06-17T12:57:06.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.290+0000 7f7735492700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f77300b4700 0x7f7730146660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57956/0 (socket says 172.21.15.36:57956) 2022-06-17T12:57:06.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.290+0000 7f7734c91700 1 -- 172.21.15.36:0/58375836 learned_addr learned my addr 172.21.15.36:0/58375836 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:06.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.290+0000 7f772ffff700 1 --2- 172.21.15.36:0/58375836 >> v2:172.21.15.70:3300/0 conn(0x7f77300b31c0 0x7f77300b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:06.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.290+0000 7f7734c91700 1 -- 172.21.15.36:0/58375836 >> v2:172.21.15.70:3300/0 conn(0x7f77300b31c0 msgr2=0x7f77300b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:06.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.290+0000 7f7734c91700 1 --2- 172.21.15.36:0/58375836 >> v2:172.21.15.70:3300/0 conn(0x7f77300b31c0 0x7f77300b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:06.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.290+0000 7f7734c91700 1 -- 172.21.15.36:0/58375836 >> v1:172.21.15.36:6789/0 conn(0x7f77300b4700 legacy=0x7f7730146660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:06.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.290+0000 7f772f7fe700 1 -- 172.21.15.36:0/58375836 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3422972369 0 0) 0x7f77300b21d0 con 0x7f77300b4700 2022-06-17T12:57:06.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.290+0000 7f7734c91700 1 -- 172.21.15.36:0/58375836 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77300b43f0 con 0x7f77300620d0 2022-06-17T12:57:06.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.290+0000 7f7734c91700 1 --2- 172.21.15.36:0/58375836 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77300620d0 0x7f77300b5bc0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 rx=0x7f77200032f0 tx=0x7f7720009290).ready entity=mon.0 client_cookie=5c6583bc27c2ef5d server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:06.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.290+0000 7f772f7fe700 1 -- 172.21.15.36:0/58375836 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f772000b870 con 0x7f77300620d0 2022-06-17T12:57:06.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.290+0000 7f772f7fe700 1 -- 172.21.15.36:0/58375836 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772000b9d0 con 0x7f77300620d0 2022-06-17T12:57:06.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.290+0000 7f772f7fe700 1 -- 172.21.15.36:0/58375836 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f772000bb40 con 0x7f77300620d0 2022-06-17T12:57:06.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.294+0000 7f7736ef5700 1 -- 172.21.15.36:0/58375836 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77300620d0 msgr2=0x7f77300b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:06.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.294+0000 7f7736ef5700 1 --2- 172.21.15.36:0/58375836 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77300620d0 0x7f77300b5bc0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 rx=0x7f77200032f0 tx=0x7f7720009290).stop 2022-06-17T12:57:06.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.294+0000 7f7736ef5700 1 -- 172.21.15.36:0/58375836 shutdown_connections 2022-06-17T12:57:06.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.294+0000 7f7736ef5700 1 --2- 172.21.15.36:0/58375836 >> v2:172.21.15.70:3300/0 conn(0x7f77300b31c0 0x7f77300b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:06.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.294+0000 7f7736ef5700 1 --2- 172.21.15.36:0/58375836 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77300620d0 0x7f77300b5bc0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:06.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.294+0000 7f7736ef5700 1 -- 172.21.15.36:0/58375836 >> 172.21.15.36:0/58375836 conn(0x7f77300b9850 msgr2=0x7f77300b9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:06.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.294+0000 7f7736ef5700 1 -- 172.21.15.36:0/58375836 shutdown_connections 2022-06-17T12:57:06.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.294+0000 7f7736ef5700 1 -- 172.21.15.36:0/58375836 wait complete. 2022-06-17T12:57:06.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.294+0000 7f7736ef5700 1 Processor -- start 2022-06-17T12:57:06.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.294+0000 7f7736ef5700 1 -- start start 2022-06-17T12:57:06.395 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.298+0000 7f7736ef5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77300620d0 0x7f7730146c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:06.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.298+0000 7f7736ef5700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77300b31c0 0x7f77301456b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:06.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.298+0000 7f7736ef5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77300b4700 0x7f7730145ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:06.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.298+0000 7f7736ef5700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7730114870 con 0x7f77300620d0 2022-06-17T12:57:06.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.298+0000 7f7736ef5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77301529b0 con 0x7f77300b4700 2022-06-17T12:57:06.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.298+0000 7f7736ef5700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7730152c50 con 0x7f77300b31c0 2022-06-17T12:57:06.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.298+0000 7f7734c91700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77300620d0 0x7f7730146c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:06.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.298+0000 7f7734c91700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77300620d0 0x7f7730146c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40430/0 (socket says 172.21.15.36:40430) 2022-06-17T12:57:06.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.298+0000 7f7734c91700 1 -- 172.21.15.36:0/3845520250 learned_addr learned my addr 172.21.15.36:0/3845520250 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:06.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.298+0000 7f772ffff700 1 --2- 172.21.15.36:0/3845520250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77300b31c0 0x7f77301456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:06.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.298+0000 7f7735492700 1 --2- 172.21.15.36:0/3845520250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77300b4700 0x7f7730145ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:06.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.298+0000 7f7734c91700 1 -- 172.21.15.36:0/3845520250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77300b31c0 msgr2=0x7f77301456b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:06.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.298+0000 7f7734c91700 1 --2- 172.21.15.36:0/3845520250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77300b31c0 0x7f77301456b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:06.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.298+0000 7f7734c91700 1 -- 172.21.15.36:0/3845520250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77300b4700 msgr2=0x7f7730145ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:06.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.298+0000 7f7734c91700 1 --2- 172.21.15.36:0/3845520250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77300b4700 0x7f7730145ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:06.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.298+0000 7f7734c91700 1 -- 172.21.15.36:0/3845520250 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7720009530 con 0x7f77300620d0 2022-06-17T12:57:06.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.298+0000 7f7734c91700 1 --2- 172.21.15.36:0/3845520250 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77300620d0 0x7f7730146c60 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0x7f77200032f0 tx=0x7f7720016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:06.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.298+0000 7f772dffb700 1 -- 172.21.15.36:0/3845520250 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f772000b870 con 0x7f77300620d0 2022-06-17T12:57:06.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.298+0000 7f772dffb700 1 -- 172.21.15.36:0/3845520250 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7720021630 con 0x7f77300620d0 2022-06-17T12:57:06.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.298+0000 7f7736ef5700 1 -- 172.21.15.36:0/3845520250 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7730152ef0 con 0x7f77300620d0 2022-06-17T12:57:06.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.298+0000 7f772dffb700 1 -- 172.21.15.36:0/3845520250 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7720021990 con 0x7f77300620d0 2022-06-17T12:57:06.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.298+0000 7f7736ef5700 1 -- 172.21.15.36:0/3845520250 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7730146910 con 0x7f77300620d0 2022-06-17T12:57:06.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.302+0000 7f7736ef5700 1 -- 172.21.15.36:0/3845520250 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77300624b0 con 0x7f77300620d0 2022-06-17T12:57:06.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.302+0000 7f772dffb700 1 -- 172.21.15.36:0/3845520250 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f7720027020 con 0x7f77300620d0 2022-06-17T12:57:06.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.302+0000 7f772dffb700 1 --2- 172.21.15.36:0/3845520250 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7708053a40 0x7f7708055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:06.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.302+0000 7f772dffb700 1 -- 172.21.15.36:0/3845520250 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f772006d0a0 con 0x7f77300620d0 2022-06-17T12:57:06.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.302+0000 7f772ffff700 1 --2- 172.21.15.36:0/3845520250 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7708053a40 0x7f7708055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:06.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.302+0000 7f772ffff700 1 --2- 172.21.15.36:0/3845520250 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7708053a40 0x7f7708055f00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f771800b000 tx=0x7f7718006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:06.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.306+0000 7f772dffb700 1 -- 172.21.15.36:0/3845520250 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7720044180 con 0x7f77300620d0 2022-06-17T12:57:06.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.474+0000 7f7736ef5700 1 -- 172.21.15.36:0/3845520250 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]} v 0) v1 -- 0x7f7730057410 con 0x7f77300620d0 2022-06-17T12:57:06.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.834+0000 7f772dffb700 1 -- 172.21.15.36:0/3845520250 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]}]=0 v486) v1 ==== 127+0+0 (secure 0 0 0) 0x7f7720015070 con 0x7f77300620d0 2022-06-17T12:57:06.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.834+0000 7f7736ef5700 1 -- 172.21.15.36:0/3845520250 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7708053a40 msgr2=0x7f7708055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:06.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.834+0000 7f7736ef5700 1 --2- 172.21.15.36:0/3845520250 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7708053a40 0x7f7708055f00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f771800b000 tx=0x7f7718006df0).stop 2022-06-17T12:57:06.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.834+0000 7f7736ef5700 1 -- 172.21.15.36:0/3845520250 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77300620d0 msgr2=0x7f7730146c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:06.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.834+0000 7f7736ef5700 1 --2- 172.21.15.36:0/3845520250 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77300620d0 0x7f7730146c60 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0x7f77200032f0 tx=0x7f7720016000).stop 2022-06-17T12:57:06.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.838+0000 7f7736ef5700 1 -- 172.21.15.36:0/3845520250 shutdown_connections 2022-06-17T12:57:06.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.838+0000 7f7736ef5700 1 --2- 172.21.15.36:0/3845520250 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7708053a40 0x7f7708055f00 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:06.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.838+0000 7f7736ef5700 1 --2- 172.21.15.36:0/3845520250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77300b4700 0x7f7730145ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:06.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.838+0000 7f7736ef5700 1 --2- 172.21.15.36:0/3845520250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77300b31c0 0x7f77301456b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:06.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.838+0000 7f7736ef5700 1 --2- 172.21.15.36:0/3845520250 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f77300620d0 0x7f7730146c60 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:06.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.838+0000 7f7736ef5700 1 -- 172.21.15.36:0/3845520250 >> 172.21.15.36:0/3845520250 conn(0x7f77300b9850 msgr2=0x7f77300af340 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:06.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.838+0000 7f7736ef5700 1 -- 172.21.15.36:0/3845520250 shutdown_connections 2022-06-17T12:57:06.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.838+0000 7f7736ef5700 1 -- 172.21.15.36:0/3845520250 wait complete. 2022-06-17T12:57:06.858 INFO:tasks.cephfs_test_runner:test_root_rctime (tasks.cephfs.test_misc.TestMisc) ... ok 2022-06-17T12:57:06.859 INFO:tasks.cephfs_test_runner:Starting test: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-06-17T12:57:06.859 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs' 2022-06-17T12:57:07.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd58a17700 1 Processor -- start 2022-06-17T12:57:07.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd58a17700 1 -- start start 2022-06-17T12:57:07.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd58a17700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcd540ac340 0x7fcd5414b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:07.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd58a17700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcd540af7f0 0x7fcd540ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:07.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd58a17700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd540ac870 con 0x7fcd540ad140 2022-06-17T12:57:07.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd58a17700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd54148960 con 0x7fcd540af7f0 2022-06-17T12:57:07.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd58a17700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd540acaa0 con 0x7fcd540ac340 2022-06-17T12:57:07.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd52d9d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcd540af7f0 0x7fcd540ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:07.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd5259c700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fcd540ad140 0x7fcd5414aca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57968/0 (socket says 172.21.15.36:57968) 2022-06-17T12:57:07.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd5259c700 1 -- 172.21.15.36:0/1885615209 learned_addr learned my addr 172.21.15.36:0/1885615209 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:07.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd51d9b700 1 --2- 172.21.15.36:0/1885615209 >> v2:172.21.15.70:3300/0 conn(0x7fcd540ac340 0x7fcd5414b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:07.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd5159a700 1 -- 172.21.15.36:0/1885615209 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1881419628 0 0) 0x7fcd540ac870 con 0x7fcd540ad140 2022-06-17T12:57:07.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd5159a700 1 -- 172.21.15.36:0/1885615209 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd34003260 con 0x7fcd540ad140 2022-06-17T12:57:07.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd52d9d700 1 -- 172.21.15.36:0/1885615209 >> v2:172.21.15.70:3300/0 conn(0x7fcd540ac340 msgr2=0x7fcd5414b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:07.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd52d9d700 1 --2- 172.21.15.36:0/1885615209 >> v2:172.21.15.70:3300/0 conn(0x7fcd540ac340 0x7fcd5414b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:07.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd52d9d700 1 -- 172.21.15.36:0/1885615209 >> v1:172.21.15.36:6789/0 conn(0x7fcd540ad140 legacy=0x7fcd5414aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:07.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd52d9d700 1 -- 172.21.15.36:0/1885615209 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd540b0bf0 con 0x7fcd540af7f0 2022-06-17T12:57:07.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd52d9d700 1 --2- 172.21.15.36:0/1885615209 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd540af7f0 0x7fcd540ad520 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 rx=0x7fcd540b10c0 tx=0x7fcd4c0091d0).ready entity=mon.0 client_cookie=adf71706ba37bb03 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:07.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd5159a700 1 -- 172.21.15.36:0/1885615209 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd4c00b5a0 con 0x7fcd540af7f0 2022-06-17T12:57:07.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd5159a700 1 -- 172.21.15.36:0/1885615209 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4c00b700 con 0x7fcd540af7f0 2022-06-17T12:57:07.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd5159a700 1 -- 172.21.15.36:0/1885615209 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd4c00b870 con 0x7fcd540af7f0 2022-06-17T12:57:07.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd58a17700 1 -- 172.21.15.36:0/1885615209 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd540af7f0 msgr2=0x7fcd540ad520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:07.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd58a17700 1 --2- 172.21.15.36:0/1885615209 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd540af7f0 0x7fcd540ad520 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 rx=0x7fcd540b10c0 tx=0x7fcd4c0091d0).stop 2022-06-17T12:57:07.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd58a17700 1 -- 172.21.15.36:0/1885615209 shutdown_connections 2022-06-17T12:57:07.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd58a17700 1 --2- 172.21.15.36:0/1885615209 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd540af7f0 0x7fcd540ad520 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:07.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd58a17700 1 --2- 172.21.15.36:0/1885615209 >> v2:172.21.15.70:3300/0 conn(0x7fcd540ac340 0x7fcd5414b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:07.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd58a17700 1 -- 172.21.15.36:0/1885615209 >> 172.21.15.36:0/1885615209 conn(0x7fcd540b5430 msgr2=0x7fcd540b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:07.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd58a17700 1 -- 172.21.15.36:0/1885615209 shutdown_connections 2022-06-17T12:57:07.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.994+0000 7fcd58a17700 1 -- 172.21.15.36:0/1885615209 wait complete. 2022-06-17T12:57:07.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.998+0000 7fcd58a17700 1 Processor -- start 2022-06-17T12:57:07.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.998+0000 7fcd58a17700 1 -- start start 2022-06-17T12:57:07.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.998+0000 7fcd58a17700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd540ac340 0x7fcd54060320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:07.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.998+0000 7fcd58a17700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd540ad140 0x7fcd540a9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:07.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.998+0000 7fcd58a17700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd540af7f0 0x7fcd540a9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:07.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.998+0000 7fcd58a17700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5414b780 con 0x7fcd540ad140 2022-06-17T12:57:07.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.998+0000 7fcd58a17700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5414a5b0 con 0x7fcd540ac340 2022-06-17T12:57:07.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.998+0000 7fcd58a17700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5414b9b0 con 0x7fcd540af7f0 2022-06-17T12:57:07.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.998+0000 7fcd51d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd540ad140 0x7fcd540a9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:07.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.998+0000 7fcd5259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd540ac340 0x7fcd54060320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:07.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.998+0000 7fcd51d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd540ad140 0x7fcd540a9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40446/0 (socket says 172.21.15.36:40446) 2022-06-17T12:57:07.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.998+0000 7fcd51d9b700 1 -- 172.21.15.36:0/473823743 learned_addr learned my addr 172.21.15.36:0/473823743 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:07.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.998+0000 7fcd52d9d700 1 --2- 172.21.15.36:0/473823743 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd540af7f0 0x7fcd540a9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:07.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.998+0000 7fcd51d9b700 1 -- 172.21.15.36:0/473823743 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd540af7f0 msgr2=0x7fcd540a9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:07.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.998+0000 7fcd51d9b700 1 --2- 172.21.15.36:0/473823743 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd540af7f0 0x7fcd540a9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:07.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.998+0000 7fcd51d9b700 1 -- 172.21.15.36:0/473823743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd540ac340 msgr2=0x7fcd54060320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:07.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.998+0000 7fcd51d9b700 1 --2- 172.21.15.36:0/473823743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd540ac340 0x7fcd54060320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:07.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.998+0000 7fcd51d9b700 1 -- 172.21.15.36:0/473823743 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4c009470 con 0x7fcd540ad140 2022-06-17T12:57:07.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.998+0000 7fcd5259c700 1 --2- 172.21.15.36:0/473823743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd540ac340 0x7fcd54060320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:57:07.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:06.998+0000 7fcd51d9b700 1 --2- 172.21.15.36:0/473823743 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd540ad140 0x7fcd540a9390 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7fcd3c00c000 tx=0x7fcd3c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:07.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.002+0000 7fcd4b7fe700 1 -- 172.21.15.36:0/473823743 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd3c010780 con 0x7fcd540ad140 2022-06-17T12:57:07.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.002+0000 7fcd4b7fe700 1 -- 172.21.15.36:0/473823743 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3c0108e0 con 0x7fcd540ad140 2022-06-17T12:57:07.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.002+0000 7fcd58a17700 1 -- 172.21.15.36:0/473823743 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5414a810 con 0x7fcd540ad140 2022-06-17T12:57:07.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.002+0000 7fcd4b7fe700 1 -- 172.21.15.36:0/473823743 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd3c010c40 con 0x7fcd540ad140 2022-06-17T12:57:07.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.002+0000 7fcd58a17700 1 -- 172.21.15.36:0/473823743 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd541429a0 con 0x7fcd540ad140 2022-06-17T12:57:07.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.002+0000 7fcd58a17700 1 -- 172.21.15.36:0/473823743 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd540b1500 con 0x7fcd540ad140 2022-06-17T12:57:07.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.002+0000 7fcd4b7fe700 1 -- 172.21.15.36:0/473823743 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fcd3c01c460 con 0x7fcd540ad140 2022-06-17T12:57:07.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.002+0000 7fcd4b7fe700 1 --2- 172.21.15.36:0/473823743 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcd38053ad0 0x7fcd38055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:07.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.002+0000 7fcd4b7fe700 1 -- 172.21.15.36:0/473823743 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fcd3c06c0c0 con 0x7fcd540ad140 2022-06-17T12:57:07.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.002+0000 7fcd5259c700 1 --2- 172.21.15.36:0/473823743 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcd38053ad0 0x7fcd38055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:07.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.002+0000 7fcd5259c700 1 --2- 172.21.15.36:0/473823743 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcd38053ad0 0x7fcd38055f90 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fcd4400d000 tx=0x7fcd44009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:07.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.014+0000 7fcd4b7fe700 1 -- 172.21.15.36:0/473823743 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcd3c043140 con 0x7fcd540ad140 2022-06-17T12:57:07.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.186+0000 7fcd58a17700 1 -- 172.21.15.36:0/473823743 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs"]} v 0) v1 -- 0x7fcd54057410 con 0x7fcd540ad140 2022-06-17T12:57:07.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.834+0000 7fcd4b7fe700 1 -- 172.21.15.36:0/473823743 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs"]}]=0 v487) v1 ==== 139+0+0 (secure 0 0 0) 0x7fcd3c046760 con 0x7fcd540ad140 2022-06-17T12:57:07.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.838+0000 7fcd58a17700 1 -- 172.21.15.36:0/473823743 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcd38053ad0 msgr2=0x7fcd38055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:07.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.838+0000 7fcd58a17700 1 --2- 172.21.15.36:0/473823743 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcd38053ad0 0x7fcd38055f90 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fcd4400d000 tx=0x7fcd44009040).stop 2022-06-17T12:57:07.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.838+0000 7fcd58a17700 1 -- 172.21.15.36:0/473823743 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd540ad140 msgr2=0x7fcd540a9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:07.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.838+0000 7fcd58a17700 1 --2- 172.21.15.36:0/473823743 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd540ad140 0x7fcd540a9390 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7fcd3c00c000 tx=0x7fcd3c006df0).stop 2022-06-17T12:57:07.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.842+0000 7fcd58a17700 1 -- 172.21.15.36:0/473823743 shutdown_connections 2022-06-17T12:57:07.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.842+0000 7fcd58a17700 1 --2- 172.21.15.36:0/473823743 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fcd38053ad0 0x7fcd38055f90 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:07.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.842+0000 7fcd58a17700 1 --2- 172.21.15.36:0/473823743 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd540af7f0 0x7fcd540a9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:07.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.842+0000 7fcd58a17700 1 --2- 172.21.15.36:0/473823743 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd540ad140 0x7fcd540a9390 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:07.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.842+0000 7fcd58a17700 1 --2- 172.21.15.36:0/473823743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd540ac340 0x7fcd54060320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:07.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.842+0000 7fcd58a17700 1 -- 172.21.15.36:0/473823743 >> 172.21.15.36:0/473823743 conn(0x7fcd540b5430 msgr2=0x7fcd540b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:07.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.842+0000 7fcd58a17700 1 -- 172.21.15.36:0/473823743 shutdown_connections 2022-06-17T12:57:07.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.842+0000 7fcd58a17700 1 -- 172.21.15.36:0/473823743 wait complete. 2022-06-17T12:57:07.856 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-06-17T12:57:08.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.990+0000 7f267ec9d700 1 Processor -- start 2022-06-17T12:57:08.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.990+0000 7f267ec9d700 1 -- start start 2022-06-17T12:57:08.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f267ec9d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f26780aed10 0x7f26781451e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:08.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f267ec9d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2678060e40 0x7f26781425f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:08.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f267ec9d700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26780b1830 con 0x7f26780b1360 2022-06-17T12:57:08.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f267ec9d700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26781423e0 con 0x7f26780aed10 2022-06-17T12:57:08.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f267ec9d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2678062b30 con 0x7f2678060e40 2022-06-17T12:57:08.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f267d23a700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f26780b1360 0x7f2678066a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:57986/0 (socket says 172.21.15.36:57986) 2022-06-17T12:57:08.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f267d23a700 1 -- 172.21.15.36:0/4103665921 learned_addr learned my addr 172.21.15.36:0/4103665921 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:08.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f2677fff700 1 --2- 172.21.15.36:0/4103665921 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2678060e40 0x7f26781425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:08.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f267ca39700 1 --2- 172.21.15.36:0/4103665921 >> v2:172.21.15.70:3300/0 conn(0x7f26780aed10 0x7f26781451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:08.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f26777fe700 1 -- 172.21.15.36:0/4103665921 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621364499 0 0) 0x7f26780b1830 con 0x7f26780b1360 2022-06-17T12:57:08.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f26777fe700 1 -- 172.21.15.36:0/4103665921 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2664003260 con 0x7f26780b1360 2022-06-17T12:57:08.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f26777fe700 1 -- 172.21.15.36:0/4103665921 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1138418594 0 0) 0x7f2664003260 con 0x7f26780b1360 2022-06-17T12:57:08.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f26777fe700 1 -- 172.21.15.36:0/4103665921 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2678060e40 msgr2=0x7f26781425f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:08.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f26777fe700 1 --2- 172.21.15.36:0/4103665921 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2678060e40 0x7f26781425f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:08.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f26777fe700 1 -- 172.21.15.36:0/4103665921 >> v2:172.21.15.70:3300/0 conn(0x7f26780aed10 msgr2=0x7f26781451e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:08.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f26777fe700 1 --2- 172.21.15.36:0/4103665921 >> v2:172.21.15.70:3300/0 conn(0x7f26780aed10 0x7f26781451e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:08.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f26777fe700 1 -- 172.21.15.36:0/4103665921 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26780b0440 con 0x7f26780b1360 2022-06-17T12:57:08.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f26777fe700 1 -- 172.21.15.36:0/4103665921 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f26700019e0 con 0x7f26780b1360 2022-06-17T12:57:08.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f26777fe700 1 -- 172.21.15.36:0/4103665921 >> v1:172.21.15.36:6789/0 conn(0x7f26780b1360 legacy=0x7f2678066a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:08.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f26777fe700 1 --2- 172.21.15.36:0/4103665921 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2678060e40 0x7f26640037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:08.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f26777fe700 1 --2- 172.21.15.36:0/4103665921 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2664003d50 0x7f2664004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:08.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f26777fe700 1 --2- 172.21.15.36:0/4103665921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26640049f0 0x7f2664004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:08.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f26777fe700 1 -- 172.21.15.36:0/4103665921 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26781423e0 con 0x7f2664003d50 2022-06-17T12:57:08.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f26777fe700 1 -- 172.21.15.36:0/4103665921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266400b750 con 0x7f26640049f0 2022-06-17T12:57:08.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f26777fe700 1 -- 172.21.15.36:0/4103665921 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266400b8c0 con 0x7f2678060e40 2022-06-17T12:57:08.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f2677fff700 1 --2- 172.21.15.36:0/4103665921 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2664003d50 0x7f2664004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:08.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f267ca39700 1 --2- 172.21.15.36:0/4103665921 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2678060e40 0x7f26640037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:08.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f267d23a700 1 --2- 172.21.15.36:0/4103665921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26640049f0 0x7f2664004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:08.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f2677fff700 1 -- 172.21.15.36:0/4103665921 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2678060e40 msgr2=0x7f26640037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:08.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f2677fff700 1 --2- 172.21.15.36:0/4103665921 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2678060e40 0x7f26640037d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:08.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f2677fff700 1 -- 172.21.15.36:0/4103665921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26640049f0 msgr2=0x7f2664004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:08.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f2677fff700 1 --2- 172.21.15.36:0/4103665921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26640049f0 0x7f2664004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:08.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f2677fff700 1 -- 172.21.15.36:0/4103665921 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2664003260 con 0x7f2664003d50 2022-06-17T12:57:08.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f2677fff700 1 --2- 172.21.15.36:0/4103665921 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2664003d50 0x7f2664004500 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7f266800c000 tx=0x7f2668006df0).ready entity=mon.0 client_cookie=4eb2cc353e2f7da4 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:08.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f26777fe700 1 -- 172.21.15.36:0/4103665921 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f266800e830 con 0x7f2664003d50 2022-06-17T12:57:08.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f26777fe700 1 -- 172.21.15.36:0/4103665921 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266800e990 con 0x7f2664003d50 2022-06-17T12:57:08.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f26777fe700 1 -- 172.21.15.36:0/4103665921 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f266800ecf0 con 0x7f2664003d50 2022-06-17T12:57:08.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f267ec9d700 1 -- 172.21.15.36:0/4103665921 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2664003d50 msgr2=0x7f2664004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:08.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f267ec9d700 1 --2- 172.21.15.36:0/4103665921 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2664003d50 0x7f2664004500 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7f266800c000 tx=0x7f2668006df0).stop 2022-06-17T12:57:08.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f267d23a700 1 -- 172.21.15.36:0/4103665921 reap_dead start 2022-06-17T12:57:08.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f267ec9d700 1 -- 172.21.15.36:0/4103665921 shutdown_connections 2022-06-17T12:57:08.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f267ec9d700 1 -- 172.21.15.36:0/4103665921 >> 172.21.15.36:0/4103665921 conn(0x7f267805c550 msgr2=0x7f26780b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:08.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f267ec9d700 1 -- 172.21.15.36:0/4103665921 shutdown_connections 2022-06-17T12:57:08.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.994+0000 7f267ec9d700 1 -- 172.21.15.36:0/4103665921 wait complete. 2022-06-17T12:57:08.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.998+0000 7f267ec9d700 1 Processor -- start 2022-06-17T12:57:08.099 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.998+0000 7f267ec9d700 1 -- start start 2022-06-17T12:57:08.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.998+0000 7f267ec9d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f267809f590 0x7f26780a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:08.101 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.998+0000 7f267ec9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26780ab6f0 0x7f26780abb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:08.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.998+0000 7f267ec9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2678150360 0x7f26781507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:08.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.998+0000 7f267ec9d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2678151630 con 0x7f2678150360 2022-06-17T12:57:08.102 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.998+0000 7f267ec9d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26780a06b0 con 0x7f26780ab6f0 2022-06-17T12:57:08.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.998+0000 7f267ec9d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26780a1ca0 con 0x7f267809f590 2022-06-17T12:57:08.103 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.998+0000 7f267d23a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2678150360 0x7f26781507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:08.104 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.998+0000 7f267d23a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2678150360 0x7f26781507e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40468/0 (socket says 172.21.15.36:40468) 2022-06-17T12:57:08.107 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.998+0000 7f267d23a700 1 -- 172.21.15.36:0/3999830270 learned_addr learned my addr 172.21.15.36:0/3999830270 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:08.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.998+0000 7f2677fff700 1 --2- 172.21.15.36:0/3999830270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26780ab6f0 0x7f26780abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:08.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.998+0000 7f267ca39700 1 --2- 172.21.15.36:0/3999830270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f267809f590 0x7f26780a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:08.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.998+0000 7f267d23a700 1 -- 172.21.15.36:0/3999830270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f267809f590 msgr2=0x7f26780a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:08.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.998+0000 7f267d23a700 1 --2- 172.21.15.36:0/3999830270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f267809f590 0x7f26780a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:08.109 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.998+0000 7f267d23a700 1 -- 172.21.15.36:0/3999830270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26780ab6f0 msgr2=0x7f26780abb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:08.110 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.998+0000 7f267d23a700 1 --2- 172.21.15.36:0/3999830270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26780ab6f0 0x7f26780abb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:08.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:07.998+0000 7f267d23a700 1 -- 172.21.15.36:0/3999830270 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267809f970 con 0x7f2678150360 2022-06-17T12:57:08.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.002+0000 7f267d23a700 1 --2- 172.21.15.36:0/3999830270 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2678150360 0x7f26781507e0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7f267809ff60 tx=0x7f2678063700).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:08.111 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.002+0000 7f2675ffb700 1 -- 172.21.15.36:0/3999830270 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2670010710 con 0x7f2678150360 2022-06-17T12:57:08.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.002+0000 7f2675ffb700 1 -- 172.21.15.36:0/3999830270 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2670010870 con 0x7f2678150360 2022-06-17T12:57:08.112 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.002+0000 7f267ec9d700 1 -- 172.21.15.36:0/3999830270 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267809ed60 con 0x7f2678150360 2022-06-17T12:57:08.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.002+0000 7f2675ffb700 1 -- 172.21.15.36:0/3999830270 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2670010bd0 con 0x7f2678150360 2022-06-17T12:57:08.113 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.002+0000 7f267ec9d700 1 -- 172.21.15.36:0/3999830270 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26780ac060 con 0x7f2678150360 2022-06-17T12:57:08.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.002+0000 7f267ec9d700 1 -- 172.21.15.36:0/3999830270 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26780b1a60 con 0x7f2678150360 2022-06-17T12:57:08.114 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.002+0000 7f2675ffb700 1 -- 172.21.15.36:0/3999830270 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f267001a020 con 0x7f2678150360 2022-06-17T12:57:08.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.002+0000 7f2675ffb700 1 --2- 172.21.15.36:0/3999830270 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2658053ad0 0x7f2658055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:08.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.002+0000 7f2675ffb700 1 -- 172.21.15.36:0/3999830270 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f267006b900 con 0x7f2678150360 2022-06-17T12:57:08.115 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.002+0000 7f267ca39700 1 --2- 172.21.15.36:0/3999830270 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2658053ad0 0x7f2658055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:08.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.006+0000 7f267ca39700 1 --2- 172.21.15.36:0/3999830270 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2658053ad0 0x7f2658055f90 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f266c00b000 tx=0x7f266c006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:08.116 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.010+0000 7f2675ffb700 1 -- 172.21.15.36:0/3999830270 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f26700429e0 con 0x7f2678150360 2022-06-17T12:57:08.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.134+0000 7f267ec9d700 1 -- 172.21.15.36:0/3999830270 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2678057410 con 0x7f2678150360 2022-06-17T12:57:08.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.142+0000 7f2675ffb700 1 -- 172.21.15.36:0/3999830270 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2670046000 con 0x7f2678150360 2022-06-17T12:57:08.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.142+0000 7f267ec9d700 1 -- 172.21.15.36:0/3999830270 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2658053ad0 msgr2=0x7f2658055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:08.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.142+0000 7f267ec9d700 1 --2- 172.21.15.36:0/3999830270 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2658053ad0 0x7f2658055f90 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f266c00b000 tx=0x7f266c006df0).stop 2022-06-17T12:57:08.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.142+0000 7f267ec9d700 1 -- 172.21.15.36:0/3999830270 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2678150360 msgr2=0x7f26781507e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:08.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.142+0000 7f267ec9d700 1 --2- 172.21.15.36:0/3999830270 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2678150360 0x7f26781507e0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7f267809ff60 tx=0x7f2678063700).stop 2022-06-17T12:57:08.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.146+0000 7f267ec9d700 1 -- 172.21.15.36:0/3999830270 shutdown_connections 2022-06-17T12:57:08.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.146+0000 7f267ec9d700 1 --2- 172.21.15.36:0/3999830270 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2658053ad0 0x7f2658055f90 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:08.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.146+0000 7f267ec9d700 1 --2- 172.21.15.36:0/3999830270 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2678150360 0x7f26781507e0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:08.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.146+0000 7f267ec9d700 1 --2- 172.21.15.36:0/3999830270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26780ab6f0 0x7f26780abb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:08.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.146+0000 7f267ec9d700 1 --2- 172.21.15.36:0/3999830270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f267809f590 0x7f26780a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:08.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.146+0000 7f267ec9d700 1 -- 172.21.15.36:0/3999830270 >> 172.21.15.36:0/3999830270 conn(0x7f267805c550 msgr2=0x7f26781419b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:08.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.146+0000 7f267ec9d700 1 -- 172.21.15.36:0/3999830270 shutdown_connections 2022-06-17T12:57:08.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.146+0000 7f267ec9d700 1 -- 172.21.15.36:0/3999830270 wait complete. 2022-06-17T12:57:08.198 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-17T12:57:08.199 DEBUG:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:57:08.233 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:57:08.273 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:57:08.303 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:57:08.337 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T12:57:08.378 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-17T12:57:08.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:57:08.379 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:57:08.379 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:57:08.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:57:08.455 INFO:teuthology.orchestra.run.smithi107.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-17T12:57:08.457 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-06-17T12:57:08.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:57:08.476 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:57:08.477 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:57:08.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:57:08.535 INFO:teuthology.orchestra.run.smithi136.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-17T12:57:08.536 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-17T12:57:08.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:57:08.548 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:57:08.548 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-06-17T12:57:08.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T12:57:08.559 INFO:teuthology.orchestra.run.smithi143.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-06-17T12:57:08.561 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-06-17T12:57:08.562 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-17T12:57:08.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.690+0000 7fc9a46a3700 1 Processor -- start 2022-06-17T12:57:08.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.690+0000 7fc9a46a3700 1 -- start start 2022-06-17T12:57:08.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.690+0000 7fc9a46a3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc99c0620d0 0x7fc99c0b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:08.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.690+0000 7fc9a46a3700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc99c0b4e10 0x7fc99c0b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:08.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.690+0000 7fc9a46a3700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99c0b21d0 con 0x7fc99c0b31c0 2022-06-17T12:57:08.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.690+0000 7fc9a46a3700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc99c0b2430 con 0x7fc99c0b4e10 2022-06-17T12:57:08.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.690+0000 7fc9a46a3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c0b26d0 con 0x7fc99c0620d0 2022-06-17T12:57:08.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.690+0000 7fc9a243f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc99c0620d0 0x7fc99c0b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:08.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.690+0000 7fc9a2c40700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc99c0b4e10 0x7fc99c0b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:08.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.690+0000 7fc9a1c3e700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc99c0b31c0 0x7fc99c0b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:53736/0 (socket says 172.21.15.36:53736) 2022-06-17T12:57:08.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.690+0000 7fc9a1c3e700 1 -- 172.21.15.36:0/2207187539 learned_addr learned my addr 172.21.15.36:0/2207187539 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:08.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.694+0000 7fc9a143d700 1 -- 172.21.15.36:0/2207187539 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4062850615 0 0) 0x7fc99c0b21d0 con 0x7fc99c0b31c0 2022-06-17T12:57:08.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.694+0000 7fc9a143d700 1 -- 172.21.15.36:0/2207187539 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc984003260 con 0x7fc99c0b31c0 2022-06-17T12:57:08.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.694+0000 7fc9a243f700 1 -- 172.21.15.36:0/2207187539 >> v2:172.21.15.70:3300/0 conn(0x7fc99c0b4e10 msgr2=0x7fc99c0b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:08.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.694+0000 7fc9a243f700 1 --2- 172.21.15.36:0/2207187539 >> v2:172.21.15.70:3300/0 conn(0x7fc99c0b4e10 0x7fc99c0b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:08.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.694+0000 7fc9a243f700 1 -- 172.21.15.36:0/2207187539 >> v1:172.21.15.70:6789/0 conn(0x7fc99c0b31c0 legacy=0x7fc99c0b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:08.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.694+0000 7fc9a243f700 1 -- 172.21.15.36:0/2207187539 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99c0b43f0 con 0x7fc99c0620d0 2022-06-17T12:57:08.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.694+0000 7fc9a243f700 1 --2- 172.21.15.36:0/2207187539 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc99c0620d0 0x7fc99c0b5bc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fc998002f70 tx=0x7fc998006df0).ready entity=mon.2 client_cookie=cf50e4ce527b3456 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:08.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.694+0000 7fc9a143d700 1 -- 172.21.15.36:0/2207187539 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc99800d4a0 con 0x7fc99c0620d0 2022-06-17T12:57:08.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.694+0000 7fc9a143d700 1 -- 172.21.15.36:0/2207187539 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc998000e90 con 0x7fc99c0620d0 2022-06-17T12:57:08.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.694+0000 7fc9a143d700 1 -- 172.21.15.36:0/2207187539 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc99800d7f0 con 0x7fc99c0620d0 2022-06-17T12:57:08.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.694+0000 7fc9a46a3700 1 -- 172.21.15.36:0/2207187539 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc99c0620d0 msgr2=0x7fc99c0b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:08.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.694+0000 7fc9a46a3700 1 --2- 172.21.15.36:0/2207187539 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc99c0620d0 0x7fc99c0b5bc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fc998002f70 tx=0x7fc998006df0).stop 2022-06-17T12:57:08.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.694+0000 7fc9a46a3700 1 -- 172.21.15.36:0/2207187539 shutdown_connections 2022-06-17T12:57:08.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.694+0000 7fc9a46a3700 1 --2- 172.21.15.36:0/2207187539 >> v2:172.21.15.70:3300/0 conn(0x7fc99c0b4e10 0x7fc99c0b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:08.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.694+0000 7fc9a46a3700 1 --2- 172.21.15.36:0/2207187539 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc99c0620d0 0x7fc99c0b5bc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:08.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.694+0000 7fc9a46a3700 1 -- 172.21.15.36:0/2207187539 >> 172.21.15.36:0/2207187539 conn(0x7fc99c0b9850 msgr2=0x7fc99c0b9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:08.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.694+0000 7fc9a46a3700 1 -- 172.21.15.36:0/2207187539 shutdown_connections 2022-06-17T12:57:08.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.694+0000 7fc9a46a3700 1 -- 172.21.15.36:0/2207187539 wait complete. 2022-06-17T12:57:08.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.698+0000 7fc9a46a3700 1 Processor -- start 2022-06-17T12:57:08.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.698+0000 7fc9a46a3700 1 -- start start 2022-06-17T12:57:08.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.698+0000 7fc9a46a3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc99c0620d0 0x7fc99c115ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:08.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.698+0000 7fc9a46a3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc99c0b31c0 0x7fc99c115fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:08.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.698+0000 7fc9a46a3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc99c0b4e10 0x7fc99c155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:08.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.698+0000 7fc9a46a3700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c146b50 con 0x7fc99c0620d0 2022-06-17T12:57:08.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.698+0000 7fc9a46a3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c147330 con 0x7fc99c0b4e10 2022-06-17T12:57:08.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.698+0000 7fc9a46a3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c113d30 con 0x7fc99c0b31c0 2022-06-17T12:57:08.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.698+0000 7fc9a243f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc99c0620d0 0x7fc99c115ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:08.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.698+0000 7fc9a243f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc99c0620d0 0x7fc99c115ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40478/0 (socket says 172.21.15.36:40478) 2022-06-17T12:57:08.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.698+0000 7fc9a243f700 1 -- 172.21.15.36:0/2458553513 learned_addr learned my addr 172.21.15.36:0/2458553513 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:08.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.698+0000 7fc9a1c3e700 1 --2- 172.21.15.36:0/2458553513 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc99c0b31c0 0x7fc99c115fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:08.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.698+0000 7fc9a2c40700 1 --2- 172.21.15.36:0/2458553513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc99c0b4e10 0x7fc99c155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:08.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.698+0000 7fc9a243f700 1 -- 172.21.15.36:0/2458553513 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc99c0b31c0 msgr2=0x7fc99c115fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:08.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.698+0000 7fc9a243f700 1 --2- 172.21.15.36:0/2458553513 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc99c0b31c0 0x7fc99c115fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:08.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.698+0000 7fc9a243f700 1 -- 172.21.15.36:0/2458553513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc99c0b4e10 msgr2=0x7fc99c155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:08.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.698+0000 7fc9a243f700 1 --2- 172.21.15.36:0/2458553513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc99c0b4e10 0x7fc99c155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:08.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.698+0000 7fc9a243f700 1 -- 172.21.15.36:0/2458553513 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99800b180 con 0x7fc99c0620d0 2022-06-17T12:57:08.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.698+0000 7fc9a243f700 1 --2- 172.21.15.36:0/2458553513 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc99c0620d0 0x7fc99c115ae0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0x7fc998002f70 tx=0x7fc998014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:08.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.702+0000 7fc98b7fe700 1 -- 172.21.15.36:0/2458553513 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc998016070 con 0x7fc99c0620d0 2022-06-17T12:57:08.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.702+0000 7fc98b7fe700 1 -- 172.21.15.36:0/2458553513 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9980137d0 con 0x7fc99c0620d0 2022-06-17T12:57:08.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.702+0000 7fc9a46a3700 1 -- 172.21.15.36:0/2458553513 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc99c113fd0 con 0x7fc99c0620d0 2022-06-17T12:57:08.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.702+0000 7fc98b7fe700 1 -- 172.21.15.36:0/2458553513 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc998013b30 con 0x7fc99c0620d0 2022-06-17T12:57:08.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.702+0000 7fc9a46a3700 1 -- 172.21.15.36:0/2458553513 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99c146fd0 con 0x7fc99c0620d0 2022-06-17T12:57:08.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.702+0000 7fc9a46a3700 1 -- 172.21.15.36:0/2458553513 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99c0624b0 con 0x7fc99c0620d0 2022-06-17T12:57:08.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.702+0000 7fc98b7fe700 1 -- 172.21.15.36:0/2458553513 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fc998025030 con 0x7fc99c0620d0 2022-06-17T12:57:08.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.702+0000 7fc98b7fe700 1 --2- 172.21.15.36:0/2458553513 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc97c053a40 0x7fc97c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:08.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.702+0000 7fc98b7fe700 1 -- 172.21.15.36:0/2458553513 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fc99806cbc0 con 0x7fc99c0620d0 2022-06-17T12:57:08.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.702+0000 7fc9a1c3e700 1 --2- 172.21.15.36:0/2458553513 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc97c053a40 0x7fc97c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:08.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.702+0000 7fc9a1c3e700 1 --2- 172.21.15.36:0/2458553513 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc97c053a40 0x7fc97c055f00 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7fc98c00d000 tx=0x7fc98c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:08.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.706+0000 7fc98b7fe700 1 -- 172.21.15.36:0/2458553513 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc998043c20 con 0x7fc99c0620d0 2022-06-17T12:57:08.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.834+0000 7fc9a46a3700 1 -- 172.21.15.36:0/2458553513 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fc99c057410 con 0x7fc99c0620d0 2022-06-17T12:57:08.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.834+0000 7fc98b7fe700 1 -- 172.21.15.36:0/2458553513 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v144) v1 ==== 81+0+29229 (secure 0 0 0) 0x7fc998023090 con 0x7fc99c0620d0 2022-06-17T12:57:08.822 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:57:08.823 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:57:08.831 INFO:teuthology.orchestra.run.smithi036.stdout: "epoch": 144, 2022-06-17T12:57:08.831 INFO:teuthology.orchestra.run.smithi036.stdout: "fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", 2022-06-17T12:57:08.833 INFO:teuthology.orchestra.run.smithi036.stdout: "created": "2022-06-17T12:46:30.671408+0000", 2022-06-17T12:57:08.833 INFO:teuthology.orchestra.run.smithi036.stdout: "modified": "2022-06-17T12:57:05.817738+0000", 2022-06-17T12:57:08.833 INFO:teuthology.orchestra.run.smithi036.stdout: "last_up_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:57:08.834 INFO:teuthology.orchestra.run.smithi036.stdout: "last_in_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:57:08.835 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-17T12:57:08.835 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_num": 5799936, 2022-06-17T12:57:08.836 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_set": [ 2022-06-17T12:57:08.836 INFO:teuthology.orchestra.run.smithi036.stdout: "pglog_hardlimit", 2022-06-17T12:57:08.836 INFO:teuthology.orchestra.run.smithi036.stdout: "purged_snapdirs", 2022-06-17T12:57:08.837 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_deletes", 2022-06-17T12:57:08.837 INFO:teuthology.orchestra.run.smithi036.stdout: "sortbitwise" 2022-06-17T12:57:08.838 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:57:08.838 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_version": 6, 2022-06-17T12:57:08.839 INFO:teuthology.orchestra.run.smithi036.stdout: "full_ratio": 0.94999998807907104, 2022-06-17T12:57:08.840 INFO:teuthology.orchestra.run.smithi036.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-17T12:57:08.840 INFO:teuthology.orchestra.run.smithi036.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-17T12:57:08.841 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_snapshot": "", 2022-06-17T12:57:08.842 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_max": 28, 2022-06-17T12:57:08.842 INFO:teuthology.orchestra.run.smithi036.stdout: "max_osd": 8, 2022-06-17T12:57:08.843 INFO:teuthology.orchestra.run.smithi036.stdout: "require_min_compat_client": "jewel", 2022-06-17T12:57:08.843 INFO:teuthology.orchestra.run.smithi036.stdout: "min_compat_client": "jewel", 2022-06-17T12:57:08.844 INFO:teuthology.orchestra.run.smithi036.stdout: "require_osd_release": "octopus", 2022-06-17T12:57:08.844 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:57:08.845 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:08.845 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 1, 2022-06-17T12:57:08.846 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "device_health_metrics", 2022-06-17T12:57:08.847 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:33.881212+0000", 2022-06-17T12:57:08.867 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:57:08.868 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:57:08.868 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:57:08.869 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:57:08.869 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:57:08.869 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:57:08.869 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:57:08.870 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:57:08.870 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 1, 2022-06-17T12:57:08.870 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 1, 2022-06-17T12:57:08.870 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 1, 2022-06-17T12:57:08.871 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 1, 2022-06-17T12:57:08.871 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 1, 2022-06-17T12:57:08.871 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:57:08.872 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:57:08.872 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:57:08.872 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:57:08.872 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:57:08.873 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:57:08.873 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:57:08.873 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:08.873 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "16", 2022-06-17T12:57:08.874 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:57:08.874 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:57:08.874 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:57:08.874 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:57:08.875 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:57:08.875 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:57:08.877 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:57:08.877 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:57:08.877 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:57:08.888 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:57:08.889 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:57:08.889 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:57:08.890 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:57:08.890 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:57:08.890 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:57:08.893 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:57:08.899 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:57:08.899 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:57:08.900 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:57:08.900 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:57:08.901 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:57:08.901 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:57:08.902 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:57:08.902 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:57:08.903 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:57:08.903 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:57:08.904 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:08.904 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:57:08.904 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:57:08.905 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:57:08.905 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:57:08.906 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:57:08.906 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:57:08.907 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:57:08.908 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:57:08.908 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:57:08.909 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:57:08.909 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:57:08.911 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:57:08.911 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 1 2022-06-17T12:57:08.911 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:08.912 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:57:08.912 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr_devicehealth": {} 2022-06-17T12:57:08.912 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:08.913 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:08.913 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:08.913 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 2, 2022-06-17T12:57:08.914 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "rbd", 2022-06-17T12:57:08.914 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:44.485460+0000", 2022-06-17T12:57:08.914 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:57:08.914 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:57:08.915 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:57:08.915 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:57:08.915 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:57:08.915 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:57:08.916 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:57:08.916 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:57:08.916 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:57:08.916 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:57:08.917 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:57:08.917 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:57:08.917 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:57:08.917 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:57:08.918 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:57:08.919 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:57:08.919 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:57:08.919 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:57:08.919 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:57:08.920 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:57:08.920 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:08.920 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "19", 2022-06-17T12:57:08.921 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:57:08.921 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:57:08.921 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:57:08.922 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:57:08.922 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:57:08.922 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:57:08.923 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:57:08.923 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:57:08.924 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:57:08.924 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:57:08.924 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:57:08.924 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:57:08.925 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:57:08.925 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:57:08.925 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:57:08.925 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:57:08.926 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:57:08.926 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:57:08.926 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:57:08.926 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:57:08.927 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:57:08.930 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:57:08.930 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:57:08.931 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:57:08.931 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:57:08.931 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:57:08.932 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:08.932 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:57:08.933 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:57:08.933 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:57:08.935 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:57:08.936 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:57:08.936 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:57:08.937 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:57:08.937 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:57:08.937 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:57:08.938 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:57:08.938 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:57:08.938 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:57:08.939 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:57:08.939 INFO:teuthology.orchestra.run.smithi036.stdout: "rbd": {} 2022-06-17T12:57:08.940 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:08.940 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:08.941 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:57:08.942 INFO:teuthology.orchestra.run.smithi036.stdout: "osds": [ 2022-06-17T12:57:08.943 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:08.944 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:57:08.945 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", 2022-06-17T12:57:08.946 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:57:08.946 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:57:08.947 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:57:08.947 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:57:08.947 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:57:08.948 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:57:08.948 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:57:08.948 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 139, 2022-06-17T12:57:08.948 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:57:08.949 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:57:08.949 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:57:08.949 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:08.950 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:08.951 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:08.951 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6802", 2022-06-17T12:57:08.951 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:57:08.952 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:08.953 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:08.953 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:08.953 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6803", 2022-06-17T12:57:08.954 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:57:08.955 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:08.956 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:08.956 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:08.956 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:57:08.957 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:08.957 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:08.957 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:08.958 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6804", 2022-06-17T12:57:08.959 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:57:08.959 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:08.960 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:08.960 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:08.960 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6805", 2022-06-17T12:57:08.960 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:57:08.961 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:08.961 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:08.961 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:08.961 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:57:08.962 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:08.967 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:08.967 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:08.968 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6808", 2022-06-17T12:57:08.968 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:57:08.969 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:08.969 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:08.971 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:08.971 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6809", 2022-06-17T12:57:08.971 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:57:08.972 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:08.972 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:08.973 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:08.973 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:57:08.974 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:08.974 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:08.975 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:08.975 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6806", 2022-06-17T12:57:08.976 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:57:08.976 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:08.977 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:08.977 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:08.978 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6807", 2022-06-17T12:57:08.978 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:57:08.979 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:08.979 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:08.980 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:08.980 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6803/22619", 2022-06-17T12:57:08.981 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6805/22619", 2022-06-17T12:57:08.981 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6809/22619", 2022-06-17T12:57:08.982 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6807/22619", 2022-06-17T12:57:08.982 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:57:08.983 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:57:08.983 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:57:08.984 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:08.984 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:08.984 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:08.985 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:57:08.985 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", 2022-06-17T12:57:08.986 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:57:08.986 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:57:08.987 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:57:08.987 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:57:08.988 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:57:08.988 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:57:08.989 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:57:08.989 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 139, 2022-06-17T12:57:08.990 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:57:08.990 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:57:08.990 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:57:08.991 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:08.991 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:08.992 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:08.992 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6810", 2022-06-17T12:57:08.993 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:57:08.993 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:08.994 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:08.994 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:08.995 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6811", 2022-06-17T12:57:08.995 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:57:08.996 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:08.996 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:08.996 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:08.997 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:57:08.997 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:08.998 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:08.998 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:08.999 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6812", 2022-06-17T12:57:08.999 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:57:09.000 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.000 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.001 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.001 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6813", 2022-06-17T12:57:09.001 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:57:09.003 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.011 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.017 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.023 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:57:09.023 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.024 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.024 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.025 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6816", 2022-06-17T12:57:09.025 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:57:09.026 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.026 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.027 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.027 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6817", 2022-06-17T12:57:09.028 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:57:09.028 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.028 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.029 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.029 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:57:09.030 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.030 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.031 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.031 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6814", 2022-06-17T12:57:09.031 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:57:09.032 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.032 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.033 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.033 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6815", 2022-06-17T12:57:09.034 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:57:09.035 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.035 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.036 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.036 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6811/22620", 2022-06-17T12:57:09.037 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6813/22620", 2022-06-17T12:57:09.038 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6817/22620", 2022-06-17T12:57:09.038 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6815/22620", 2022-06-17T12:57:09.039 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:57:09.039 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:57:09.039 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:57:09.040 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.040 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.041 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.041 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:57:09.041 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", 2022-06-17T12:57:09.042 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:57:09.043 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:57:09.044 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:57:09.044 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:57:09.045 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:57:09.045 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:57:09.046 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:57:09.046 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 138, 2022-06-17T12:57:09.048 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:57:09.049 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:57:09.049 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:57:09.050 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.050 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.051 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.053 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6826", 2022-06-17T12:57:09.053 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:57:09.061 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.062 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.062 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.062 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6827", 2022-06-17T12:57:09.063 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:57:09.063 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.063 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.063 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.064 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:57:09.064 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.064 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.064 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.065 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6828", 2022-06-17T12:57:09.065 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:57:09.065 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.065 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.065 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.066 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6829", 2022-06-17T12:57:09.066 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:57:09.066 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.066 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.067 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.067 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:57:09.067 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.067 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.068 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.068 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6832", 2022-06-17T12:57:09.068 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:57:09.068 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.068 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.069 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.069 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6833", 2022-06-17T12:57:09.069 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:57:09.069 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.070 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.070 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.070 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:57:09.070 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.071 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.071 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.071 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6830", 2022-06-17T12:57:09.072 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:57:09.079 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.080 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.080 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.080 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6831", 2022-06-17T12:57:09.080 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:57:09.081 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.081 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.082 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.082 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6827/22643", 2022-06-17T12:57:09.082 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6829/22643", 2022-06-17T12:57:09.083 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6833/22643", 2022-06-17T12:57:09.083 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6831/22643", 2022-06-17T12:57:09.083 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:57:09.083 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:57:09.083 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:57:09.084 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.084 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.084 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.084 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:57:09.085 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", 2022-06-17T12:57:09.085 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:57:09.085 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:57:09.086 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:57:09.086 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:57:09.086 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:57:09.086 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:57:09.087 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:57:09.087 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 128, 2022-06-17T12:57:09.087 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:57:09.087 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:57:09.087 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:57:09.088 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.088 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.088 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.088 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6818", 2022-06-17T12:57:09.089 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:57:09.089 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.089 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.089 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.090 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6819", 2022-06-17T12:57:09.090 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:57:09.090 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.090 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.091 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.091 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:57:09.091 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.091 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.091 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.092 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6820", 2022-06-17T12:57:09.092 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:57:09.092 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.092 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.093 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.093 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6821", 2022-06-17T12:57:09.093 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:57:09.093 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.093 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.094 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.094 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:57:09.094 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.094 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.095 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.095 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6824", 2022-06-17T12:57:09.095 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:57:09.095 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.095 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.096 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.096 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6825", 2022-06-17T12:57:09.096 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:57:09.096 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.097 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.097 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.097 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:57:09.097 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.097 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.098 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.098 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6822", 2022-06-17T12:57:09.098 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:57:09.099 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.099 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.099 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.099 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6823", 2022-06-17T12:57:09.099 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:57:09.100 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.100 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.100 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.100 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6819/22646", 2022-06-17T12:57:09.100 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6821/22646", 2022-06-17T12:57:09.101 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6825/22646", 2022-06-17T12:57:09.101 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6823/22646", 2022-06-17T12:57:09.102 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:57:09.102 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:57:09.103 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:57:09.103 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.103 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.104 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.104 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:57:09.105 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", 2022-06-17T12:57:09.105 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:57:09.105 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:57:09.106 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:57:09.106 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:57:09.107 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:57:09.107 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:57:09.108 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:57:09.108 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 138, 2022-06-17T12:57:09.109 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:57:09.109 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:57:09.109 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:57:09.110 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.110 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.111 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.111 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6808", 2022-06-17T12:57:09.112 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:57:09.112 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.112 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.113 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.113 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6809", 2022-06-17T12:57:09.114 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:57:09.114 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.116 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.116 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.117 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:57:09.117 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.117 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.119 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.119 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6810", 2022-06-17T12:57:09.119 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:57:09.120 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.120 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.121 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.121 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6811", 2022-06-17T12:57:09.121 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:57:09.122 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.122 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.123 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.123 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:57:09.123 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.124 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.124 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.125 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6814", 2022-06-17T12:57:09.125 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:57:09.125 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.125 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.126 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.126 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6815", 2022-06-17T12:57:09.126 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:57:09.126 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.127 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.127 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.127 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:57:09.128 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.128 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.128 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.128 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6812", 2022-06-17T12:57:09.130 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:57:09.130 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.131 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.131 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.132 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6813", 2022-06-17T12:57:09.132 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:57:09.133 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.133 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.134 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.134 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6809/22812", 2022-06-17T12:57:09.134 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6811/22812", 2022-06-17T12:57:09.135 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6815/22812", 2022-06-17T12:57:09.135 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6813/22812", 2022-06-17T12:57:09.136 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:57:09.136 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:57:09.137 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:57:09.137 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.138 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.139 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.139 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:57:09.139 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", 2022-06-17T12:57:09.139 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:57:09.140 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:57:09.140 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:57:09.140 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:57:09.140 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:57:09.141 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:57:09.141 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:57:09.141 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 139, 2022-06-17T12:57:09.141 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:57:09.142 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:57:09.142 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:57:09.142 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.142 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.143 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.143 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6800", 2022-06-17T12:57:09.143 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:57:09.144 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.144 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.144 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.144 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6801", 2022-06-17T12:57:09.145 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:57:09.145 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.145 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.145 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.147 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:57:09.147 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.148 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.148 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.149 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6802", 2022-06-17T12:57:09.149 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:57:09.149 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.150 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.150 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.151 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6803", 2022-06-17T12:57:09.151 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:57:09.152 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.152 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.153 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.153 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:57:09.153 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.154 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.162 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.162 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6806", 2022-06-17T12:57:09.163 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:57:09.163 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.164 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.164 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.165 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6807", 2022-06-17T12:57:09.165 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:57:09.165 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.166 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.187 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.187 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:57:09.188 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.189 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.189 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.198 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6804", 2022-06-17T12:57:09.198 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:57:09.199 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.199 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.200 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.200 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6805", 2022-06-17T12:57:09.201 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:57:09.201 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.202 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.202 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.203 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6801/22813", 2022-06-17T12:57:09.203 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6803/22813", 2022-06-17T12:57:09.203 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6807/22813", 2022-06-17T12:57:09.204 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6805/22813", 2022-06-17T12:57:09.204 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:57:09.205 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:57:09.206 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:57:09.207 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.216 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.217 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.217 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:57:09.217 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", 2022-06-17T12:57:09.218 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:57:09.219 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:57:09.219 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:57:09.220 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:57:09.220 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:57:09.220 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:57:09.221 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:57:09.221 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 139, 2022-06-17T12:57:09.222 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:57:09.222 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:57:09.223 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:57:09.223 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.224 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.224 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.224 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6816", 2022-06-17T12:57:09.225 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:57:09.225 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.226 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.226 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.227 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6817", 2022-06-17T12:57:09.227 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:57:09.227 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.228 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.228 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.229 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:57:09.229 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.230 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.230 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.230 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6818", 2022-06-17T12:57:09.231 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:57:09.231 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.232 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.232 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.233 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6819", 2022-06-17T12:57:09.233 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:57:09.234 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.234 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.234 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.235 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:57:09.235 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.236 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.236 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.237 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6822", 2022-06-17T12:57:09.237 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:57:09.238 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.238 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.239 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.239 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6823", 2022-06-17T12:57:09.240 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:57:09.240 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.240 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.241 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.241 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:57:09.242 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.242 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.243 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.243 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6820", 2022-06-17T12:57:09.243 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:57:09.244 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.245 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.245 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.245 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6821", 2022-06-17T12:57:09.246 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:57:09.246 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.247 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.247 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.247 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6817/22822", 2022-06-17T12:57:09.248 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6819/22822", 2022-06-17T12:57:09.248 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6823/22822", 2022-06-17T12:57:09.249 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6821/22822", 2022-06-17T12:57:09.249 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:57:09.250 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:57:09.250 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:57:09.250 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.251 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.251 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.252 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:57:09.252 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", 2022-06-17T12:57:09.252 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:57:09.253 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:57:09.253 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:57:09.253 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:57:09.254 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:57:09.254 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:57:09.255 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:57:09.255 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 139, 2022-06-17T12:57:09.255 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:57:09.256 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:57:09.256 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:57:09.257 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.257 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.257 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.258 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6824", 2022-06-17T12:57:09.258 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:57:09.259 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.259 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.260 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.260 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6825", 2022-06-17T12:57:09.260 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:57:09.261 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.261 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.262 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.262 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:57:09.263 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.263 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.263 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.264 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6826", 2022-06-17T12:57:09.264 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:57:09.265 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.265 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.265 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.272 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6827", 2022-06-17T12:57:09.273 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:57:09.274 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.274 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.275 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.275 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:57:09.276 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.276 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.276 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.277 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6830", 2022-06-17T12:57:09.277 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:57:09.278 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.278 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.279 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.279 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6831", 2022-06-17T12:57:09.280 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:57:09.280 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.280 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.281 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.281 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:57:09.282 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:09.291 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.292 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:09.292 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6828", 2022-06-17T12:57:09.293 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:57:09.293 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.294 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.294 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:09.294 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6829", 2022-06-17T12:57:09.295 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:57:09.295 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.295 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.296 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.296 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6825/22837", 2022-06-17T12:57:09.302 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6827/22837", 2022-06-17T12:57:09.303 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6831/22837", 2022-06-17T12:57:09.303 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6829/22837", 2022-06-17T12:57:09.304 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:57:09.304 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:57:09.305 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:57:09.305 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:09.305 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.306 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:57:09.306 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_xinfo": [ 2022-06-17T12:57:09.307 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.307 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:57:09.308 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:57:09.308 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:57:09.309 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:57:09.309 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:57:09.309 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:57:09.311 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.291461+0000", 2022-06-17T12:57:09.311 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:57:09.312 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.312 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.313 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:57:09.313 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:57:09.314 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:57:09.314 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:57:09.315 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:57:09.315 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:57:09.316 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.537912+0000", 2022-06-17T12:57:09.316 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:57:09.317 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.317 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.317 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:57:09.318 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:57:09.318 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:57:09.319 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:57:09.320 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:57:09.320 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:57:09.320 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.606947+0000", 2022-06-17T12:57:09.321 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:57:09.321 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.322 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.322 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:57:09.322 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:57:09.329 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:57:09.338 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:57:09.339 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:57:09.339 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:57:09.340 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.514412+0000", 2022-06-17T12:57:09.340 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:57:09.340 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.341 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.341 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:57:09.342 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:57:09.342 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:57:09.342 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:57:09.343 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:57:09.343 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:57:09.344 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.394645+0000", 2022-06-17T12:57:09.344 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:57:09.350 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.351 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.351 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:57:09.352 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:57:09.353 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:57:09.353 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:57:09.353 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:57:09.354 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:57:09.354 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.343806+0000", 2022-06-17T12:57:09.355 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:57:09.355 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.356 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.357 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:57:09.358 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:57:09.358 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:57:09.359 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:57:09.359 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:57:09.360 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:57:09.360 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.591864+0000", 2022-06-17T12:57:09.360 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:57:09.361 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.361 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:09.362 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:57:09.363 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:57:09.363 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:57:09.364 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:57:09.364 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:57:09.365 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:57:09.365 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.607045+0000", 2022-06-17T12:57:09.366 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:57:09.367 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.367 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:57:09.368 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap": [], 2022-06-17T12:57:09.368 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap_items": [], 2022-06-17T12:57:09.369 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_temp": [], 2022-06-17T12:57:09.369 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_temp": [], 2022-06-17T12:57:09.370 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklist": { 2022-06-17T12:57:09.370 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.36:6835/3169088536": "2022-06-18T12:57:01.906118+0000", 2022-06-17T12:57:09.371 INFO:teuthology.orchestra.run.smithi036.stdout: "172.21.15.36:6834/3169088536": "2022-06-18T12:57:01.906118+0000" 2022-06-17T12:57:09.371 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.372 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profiles": { 2022-06-17T12:57:09.372 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs_data": { 2022-06-17T12:57:09.373 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-device-class": "", 2022-06-17T12:57:09.373 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:57:09.373 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-root": "default", 2022-06-17T12:57:09.374 INFO:teuthology.orchestra.run.smithi036.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-17T12:57:09.374 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:57:09.375 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "2", 2022-06-17T12:57:09.375 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:57:09.375 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:57:09.376 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van", 2022-06-17T12:57:09.376 INFO:teuthology.orchestra.run.smithi036.stdout: "w": "8" 2022-06-17T12:57:09.376 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.377 INFO:teuthology.orchestra.run.smithi036.stdout: "default": { 2022-06-17T12:57:09.377 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:57:09.391 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:57:09.391 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "1", 2022-06-17T12:57:09.392 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:57:09.392 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:57:09.392 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van" 2022-06-17T12:57:09.393 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:09.393 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:09.400 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps_queue": [], 2022-06-17T12:57:09.400 INFO:teuthology.orchestra.run.smithi036.stdout: "new_removed_snaps": [], 2022-06-17T12:57:09.401 INFO:teuthology.orchestra.run.smithi036.stdout: "new_purged_snaps": [], 2022-06-17T12:57:09.401 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_node_flags": {}, 2022-06-17T12:57:09.401 INFO:teuthology.orchestra.run.smithi036.stdout: "device_class_flags": {} 2022-06-17T12:57:09.401 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:57:09.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.834+0000 7fc9a46a3700 1 -- 172.21.15.36:0/2458553513 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc97c053a40 msgr2=0x7fc97c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:09.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.834+0000 7fc9a46a3700 1 --2- 172.21.15.36:0/2458553513 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc97c053a40 0x7fc97c055f00 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7fc98c00d000 tx=0x7fc98c009040).stop 2022-06-17T12:57:09.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.838+0000 7fc9a46a3700 1 -- 172.21.15.36:0/2458553513 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc99c0620d0 msgr2=0x7fc99c115ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:09.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.838+0000 7fc9a46a3700 1 --2- 172.21.15.36:0/2458553513 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc99c0620d0 0x7fc99c115ae0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0x7fc998002f70 tx=0x7fc998014000).stop 2022-06-17T12:57:09.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.838+0000 7fc9a46a3700 1 -- 172.21.15.36:0/2458553513 shutdown_connections 2022-06-17T12:57:09.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.838+0000 7fc9a46a3700 1 --2- 172.21.15.36:0/2458553513 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc97c053a40 0x7fc97c055f00 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:09.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.838+0000 7fc9a46a3700 1 --2- 172.21.15.36:0/2458553513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc99c0b4e10 0x7fc99c155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:09.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.838+0000 7fc9a46a3700 1 --2- 172.21.15.36:0/2458553513 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc99c0b31c0 0x7fc99c115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:09.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.838+0000 7fc9a46a3700 1 --2- 172.21.15.36:0/2458553513 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc99c0620d0 0x7fc99c115ae0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:09.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.838+0000 7fc9a46a3700 1 -- 172.21.15.36:0/2458553513 >> 172.21.15.36:0/2458553513 conn(0x7fc99c0b9850 msgr2=0x7fc99c140680 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:09.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.838+0000 7fc9a46a3700 1 -- 172.21.15.36:0/2458553513 shutdown_connections 2022-06-17T12:57:09.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:08.838+0000 7fc9a46a3700 1 -- 172.21.15.36:0/2458553513 wait complete. 2022-06-17T12:57:09.408 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:57:09.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.534+0000 7fe4b4036700 1 Processor -- start 2022-06-17T12:57:09.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b4036700 1 -- start start 2022-06-17T12:57:09.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b4036700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4ac144c00 0x7fe4ac062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:09.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b4036700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4ac0b0ba0 con 0x7fe4ac062770 2022-06-17T12:57:09.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b4036700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4ac147e90 con 0x7fe4ac0ad330 2022-06-17T12:57:09.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b4036700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac0ae820 con 0x7fe4ac144c00 2022-06-17T12:57:09.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b1dd2700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fe4ac062770 0x7fe4ac1444f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58018/0 (socket says 172.21.15.36:58018) 2022-06-17T12:57:09.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b1dd2700 1 -- 172.21.15.36:0/4061728378 learned_addr learned my addr 172.21.15.36:0/4061728378 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:09.535 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b15d1700 1 --2- 172.21.15.36:0/4061728378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4ac144c00 0x7fe4ac062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:09.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b0dd0700 1 -- 172.21.15.36:0/4061728378 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3631992908 0 0) 0x7fe4ac0b0ba0 con 0x7fe4ac062770 2022-06-17T12:57:09.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b0dd0700 1 -- 172.21.15.36:0/4061728378 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe494003260 con 0x7fe4ac062770 2022-06-17T12:57:09.536 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b0dd0700 1 -- 172.21.15.36:0/4061728378 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1575284972 0 0) 0x7fe4ac147e90 con 0x7fe4ac0ad330 2022-06-17T12:57:09.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b0dd0700 1 -- 172.21.15.36:0/4061728378 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4ac0b0ba0 con 0x7fe4ac0ad330 2022-06-17T12:57:09.537 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b15d1700 1 -- 172.21.15.36:0/4061728378 >> v1:172.21.15.70:6789/0 conn(0x7fe4ac0ad330 legacy=0x7fe4ac0ad780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:09.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b15d1700 1 -- 172.21.15.36:0/4061728378 >> v1:172.21.15.36:6789/0 conn(0x7fe4ac062770 legacy=0x7fe4ac1444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:09.538 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b15d1700 1 -- 172.21.15.36:0/4061728378 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ac0b0dd0 con 0x7fe4ac144c00 2022-06-17T12:57:09.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b15d1700 1 --2- 172.21.15.36:0/4061728378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4ac144c00 0x7fe4ac062b50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fe4a0002f70 tx=0x7fe4a0006df0).ready entity=mon.2 client_cookie=9f31a0a7e990d766 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:09.539 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b0dd0700 1 -- 172.21.15.36:0/4061728378 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4a000d5c0 con 0x7fe4ac144c00 2022-06-17T12:57:09.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b0dd0700 1 -- 172.21.15.36:0/4061728378 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a0000e90 con 0x7fe4ac144c00 2022-06-17T12:57:09.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b0dd0700 1 -- 172.21.15.36:0/4061728378 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4a000d910 con 0x7fe4ac144c00 2022-06-17T12:57:09.540 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b4036700 1 -- 172.21.15.36:0/4061728378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4ac144c00 msgr2=0x7fe4ac062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:09.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b4036700 1 --2- 172.21.15.36:0/4061728378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4ac144c00 0x7fe4ac062b50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fe4a0002f70 tx=0x7fe4a0006df0).stop 2022-06-17T12:57:09.541 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b4036700 1 -- 172.21.15.36:0/4061728378 shutdown_connections 2022-06-17T12:57:09.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b4036700 1 --2- 172.21.15.36:0/4061728378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4ac144c00 0x7fe4ac062b50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:09.542 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b4036700 1 -- 172.21.15.36:0/4061728378 >> 172.21.15.36:0/4061728378 conn(0x7fe4ac0b53e0 msgr2=0x7fe4ac0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:09.543 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b4036700 1 -- 172.21.15.36:0/4061728378 shutdown_connections 2022-06-17T12:57:09.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.538+0000 7fe4b4036700 1 -- 172.21.15.36:0/4061728378 wait complete. 2022-06-17T12:57:09.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.542+0000 7fe4b4036700 1 Processor -- start 2022-06-17T12:57:09.544 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.542+0000 7fe4b4036700 1 -- start start 2022-06-17T12:57:09.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.542+0000 7fe4b4036700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4ac062770 0x7fe4ac148c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:09.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.542+0000 7fe4b4036700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4ac0ad330 0x7fe4ac064b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:09.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.542+0000 7fe4b4036700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4ac144c00 0x7fe4ac066480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:09.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.542+0000 7fe4b4036700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac144690 con 0x7fe4ac144c00 2022-06-17T12:57:09.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.542+0000 7fe4b4036700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac0644a0 con 0x7fe4ac0ad330 2022-06-17T12:57:09.546 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.542+0000 7fe4b4036700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac1448c0 con 0x7fe4ac062770 2022-06-17T12:57:09.547 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.542+0000 7fe4b25d3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4ac144c00 0x7fe4ac066480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:09.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.542+0000 7fe4b1dd2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4ac062770 0x7fe4ac148c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:09.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.542+0000 7fe4b25d3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4ac144c00 0x7fe4ac066480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40496/0 (socket says 172.21.15.36:40496) 2022-06-17T12:57:09.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.542+0000 7fe4b25d3700 1 -- 172.21.15.36:0/940946719 learned_addr learned my addr 172.21.15.36:0/940946719 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:09.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.542+0000 7fe4b15d1700 1 --2- 172.21.15.36:0/940946719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4ac0ad330 0x7fe4ac064b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:09.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.542+0000 7fe4b25d3700 1 -- 172.21.15.36:0/940946719 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4ac062770 msgr2=0x7fe4ac148c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:09.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.542+0000 7fe4b25d3700 1 --2- 172.21.15.36:0/940946719 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4ac062770 0x7fe4ac148c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:09.555 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.542+0000 7fe4b25d3700 1 -- 172.21.15.36:0/940946719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4ac0ad330 msgr2=0x7fe4ac064b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:09.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.542+0000 7fe4b25d3700 1 --2- 172.21.15.36:0/940946719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4ac0ad330 0x7fe4ac064b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:09.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.542+0000 7fe4b25d3700 1 -- 172.21.15.36:0/940946719 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a000b180 con 0x7fe4ac144c00 2022-06-17T12:57:09.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.542+0000 7fe4b25d3700 1 --2- 172.21.15.36:0/940946719 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4ac144c00 0x7fe4ac066480 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 rx=0x7fe4a4001490 tx=0x7fe4a4007d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:09.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.542+0000 7fe49effd700 1 -- 172.21.15.36:0/940946719 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4a4002310 con 0x7fe4ac144c00 2022-06-17T12:57:09.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.542+0000 7fe49effd700 1 -- 172.21.15.36:0/940946719 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a4002470 con 0x7fe4ac144c00 2022-06-17T12:57:09.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.542+0000 7fe4b4036700 1 -- 172.21.15.36:0/940946719 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4ac0646d0 con 0x7fe4ac144c00 2022-06-17T12:57:09.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.542+0000 7fe49effd700 1 -- 172.21.15.36:0/940946719 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4a4001840 con 0x7fe4ac144c00 2022-06-17T12:57:09.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.542+0000 7fe4b4036700 1 -- 172.21.15.36:0/940946719 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4ac063f80 con 0x7fe4ac144c00 2022-06-17T12:57:09.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.546+0000 7fe4b4036700 1 -- 172.21.15.36:0/940946719 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4ac0b1c20 con 0x7fe4ac144c00 2022-06-17T12:57:09.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.546+0000 7fe49effd700 1 -- 172.21.15.36:0/940946719 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe4a4010030 con 0x7fe4ac144c00 2022-06-17T12:57:09.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.546+0000 7fe49effd700 1 --2- 172.21.15.36:0/940946719 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe48c053af0 0x7fe48c055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:09.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.546+0000 7fe49effd700 1 -- 172.21.15.36:0/940946719 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fe4a406bb60 con 0x7fe4ac144c00 2022-06-17T12:57:09.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.546+0000 7fe4b1dd2700 1 --2- 172.21.15.36:0/940946719 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe48c053af0 0x7fe48c055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:09.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.546+0000 7fe4b1dd2700 1 --2- 172.21.15.36:0/940946719 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe48c053af0 0x7fe48c055fb0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7fe4a800d000 tx=0x7fe4a8009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:09.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.550+0000 7fe49effd700 1 -- 172.21.15.36:0/940946719 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe4a4042be0 con 0x7fe4ac144c00 2022-06-17T12:57:09.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.714+0000 7fe4b4036700 1 -- 172.21.15.36:0/940946719 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4ac057410 con 0x7fe4ac144c00 2022-06-17T12:57:09.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.714+0000 7fe49effd700 1 -- 172.21.15.36:0/940946719 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+1209 (secure 0 0 0) 0x7fe4a4046200 con 0x7fe4ac144c00 2022-06-17T12:57:09.704 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:57:09.705 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":80,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1081450840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1081450840},{"type":"v1","addr":"172.21.15.70:6833","nonce":1081450840}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":76},{"gid":8715,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/3373137980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3373137980},{"type":"v1","addr":"172.21.15.36:6835","nonce":3373137980}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":80}],"filesystems":[]} 2022-06-17T12:57:09.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.714+0000 7fe4b4036700 1 -- 172.21.15.36:0/940946719 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe48c053af0 msgr2=0x7fe48c055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:09.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.714+0000 7fe4b4036700 1 --2- 172.21.15.36:0/940946719 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe48c053af0 0x7fe48c055fb0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7fe4a800d000 tx=0x7fe4a8009040).stop 2022-06-17T12:57:09.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.714+0000 7fe4b4036700 1 -- 172.21.15.36:0/940946719 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4ac144c00 msgr2=0x7fe4ac066480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:09.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.714+0000 7fe4b4036700 1 --2- 172.21.15.36:0/940946719 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4ac144c00 0x7fe4ac066480 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 rx=0x7fe4a4001490 tx=0x7fe4a4007d90).stop 2022-06-17T12:57:09.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.714+0000 7fe4b4036700 1 -- 172.21.15.36:0/940946719 shutdown_connections 2022-06-17T12:57:09.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.714+0000 7fe4b4036700 1 --2- 172.21.15.36:0/940946719 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe48c053af0 0x7fe48c055fb0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:09.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.714+0000 7fe4b4036700 1 --2- 172.21.15.36:0/940946719 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4ac144c00 0x7fe4ac066480 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:09.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.714+0000 7fe4b4036700 1 --2- 172.21.15.36:0/940946719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4ac0ad330 0x7fe4ac064b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:09.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.714+0000 7fe4b4036700 1 --2- 172.21.15.36:0/940946719 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4ac062770 0x7fe4ac148c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:09.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.714+0000 7fe4b4036700 1 -- 172.21.15.36:0/940946719 >> 172.21.15.36:0/940946719 conn(0x7fe4ac0b53e0 msgr2=0x7fe4ac0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:09.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.714+0000 7fe4b4036700 1 -- 172.21.15.36:0/940946719 shutdown_connections 2022-06-17T12:57:09.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.714+0000 7fe4b4036700 1 -- 172.21.15.36:0/940946719 wait complete. 2022-06-17T12:57:09.721 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 80 2022-06-17T12:57:09.722 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:57:09.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.846+0000 7ff2f3fff700 1 Processor -- start 2022-06-17T12:57:09.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.846+0000 7ff2f3fff700 1 -- start start 2022-06-17T12:57:09.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.846+0000 7ff2f3fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7ff2f4063730 0x7ff2f40b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:09.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.846+0000 7ff2f3fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff2f4140610 0x7ff2f40b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:09.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.846+0000 7ff2f3fff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2f4063b10 con 0x7ff2f4064eb0 2022-06-17T12:57:09.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.846+0000 7ff2f3fff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2f4147430 con 0x7ff2f4063730 2022-06-17T12:57:09.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.846+0000 7ff2f3fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f40b1ca0 con 0x7ff2f4140610 2022-06-17T12:57:09.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.846+0000 7ff2f27fc700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7ff2f4064eb0 0x7ff2f413ff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58032/0 (socket says 172.21.15.36:58032) 2022-06-17T12:57:09.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.846+0000 7ff2f27fc700 1 -- 172.21.15.36:0/1949291316 learned_addr learned my addr 172.21.15.36:0/1949291316 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:09.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.846+0000 7ff2f2ffd700 1 --2- 172.21.15.36:0/1949291316 >> v2:172.21.15.36:3300/0 conn(0x7ff2f4063730 0x7ff2f40b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:09.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.850+0000 7ff2f37fe700 1 --2- 172.21.15.36:0/1949291316 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff2f4140610 0x7ff2f40b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:09.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.850+0000 7ff2f2ffd700 1 -- 172.21.15.36:0/1949291316 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff2f4140610 msgr2=0x7ff2f40b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:09.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.850+0000 7ff2f2ffd700 1 --2- 172.21.15.36:0/1949291316 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff2f4140610 0x7ff2f40b33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:09.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.850+0000 7ff2f2ffd700 1 -- 172.21.15.36:0/1949291316 >> v1:172.21.15.36:6789/0 conn(0x7ff2f4064eb0 legacy=0x7ff2f413ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:09.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.850+0000 7ff2f2ffd700 1 -- 172.21.15.36:0/1949291316 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f41464d0 con 0x7ff2f4063730 2022-06-17T12:57:09.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.850+0000 7ff2f2ffd700 1 --2- 172.21.15.36:0/1949291316 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff2f4063730 0x7ff2f40b2e10 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7ff2f4063ca0 tx=0x7ff2ec0091d0).ready entity=mon.0 client_cookie=52da97054e78d3ad server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:09.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.850+0000 7ff2f1ffb700 1 -- 172.21.15.36:0/1949291316 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2ec00b860 con 0x7ff2f4063730 2022-06-17T12:57:09.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.850+0000 7ff2f1ffb700 1 -- 172.21.15.36:0/1949291316 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ec00b9c0 con 0x7ff2f4063730 2022-06-17T12:57:09.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.850+0000 7ff2f1ffb700 1 -- 172.21.15.36:0/1949291316 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2ec00bb30 con 0x7ff2f4063730 2022-06-17T12:57:09.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.850+0000 7ff2f3fff700 1 -- 172.21.15.36:0/1949291316 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff2f4063730 msgr2=0x7ff2f40b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:09.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.850+0000 7ff2f3fff700 1 --2- 172.21.15.36:0/1949291316 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff2f4063730 0x7ff2f40b2e10 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7ff2f4063ca0 tx=0x7ff2ec0091d0).stop 2022-06-17T12:57:09.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.850+0000 7ff2f3fff700 1 -- 172.21.15.36:0/1949291316 shutdown_connections 2022-06-17T12:57:09.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.850+0000 7ff2f3fff700 1 --2- 172.21.15.36:0/1949291316 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff2f4140610 0x7ff2f40b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:09.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.850+0000 7ff2f3fff700 1 --2- 172.21.15.36:0/1949291316 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff2f4063730 0x7ff2f40b2e10 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:09.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.850+0000 7ff2f3fff700 1 -- 172.21.15.36:0/1949291316 >> 172.21.15.36:0/1949291316 conn(0x7ff2f40614b0 msgr2=0x7ff2f40618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:09.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.850+0000 7ff2f3fff700 1 -- 172.21.15.36:0/1949291316 shutdown_connections 2022-06-17T12:57:09.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.850+0000 7ff2f3fff700 1 -- 172.21.15.36:0/1949291316 wait complete. 2022-06-17T12:57:09.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.850+0000 7ff2f3fff700 1 Processor -- start 2022-06-17T12:57:09.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.850+0000 7ff2f3fff700 1 -- start start 2022-06-17T12:57:09.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.854+0000 7ff2f3fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff2f4063730 0x7ff2f40aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:09.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.854+0000 7ff2f3fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff2f4064eb0 0x7ff2f40b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:09.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.854+0000 7ff2f3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f4140610 0x7ff2f4152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:09.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.854+0000 7ff2f3fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f4152550 con 0x7ff2f4064eb0 2022-06-17T12:57:09.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.854+0000 7ff2f3fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f41527f0 con 0x7ff2f4140610 2022-06-17T12:57:09.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.854+0000 7ff2f3fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f40af0e0 con 0x7ff2f4063730 2022-06-17T12:57:09.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.854+0000 7ff2f27fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff2f4064eb0 0x7ff2f40b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:09.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.854+0000 7ff2f37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f4140610 0x7ff2f4152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:09.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.854+0000 7ff2f27fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff2f4064eb0 0x7ff2f40b6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40510/0 (socket says 172.21.15.36:40510) 2022-06-17T12:57:09.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.854+0000 7ff2f27fc700 1 -- 172.21.15.36:0/3733507840 learned_addr learned my addr 172.21.15.36:0/3733507840 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:09.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.854+0000 7ff2f2ffd700 1 --2- 172.21.15.36:0/3733507840 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff2f4063730 0x7ff2f40aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:09.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.854+0000 7ff2f37fe700 1 -- 172.21.15.36:0/3733507840 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff2f4063730 msgr2=0x7ff2f40aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:09.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.854+0000 7ff2f37fe700 1 --2- 172.21.15.36:0/3733507840 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff2f4063730 0x7ff2f40aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:09.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.854+0000 7ff2f37fe700 1 -- 172.21.15.36:0/3733507840 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff2f4064eb0 msgr2=0x7ff2f40b6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:09.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.854+0000 7ff2f37fe700 1 --2- 172.21.15.36:0/3733507840 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff2f4064eb0 0x7ff2f40b6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:09.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.854+0000 7ff2f37fe700 1 -- 172.21.15.36:0/3733507840 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ec009470 con 0x7ff2f4140610 2022-06-17T12:57:09.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.854+0000 7ff2f37fe700 1 --2- 172.21.15.36:0/3733507840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f4140610 0x7ff2f4152c90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7ff2e800a000 tx=0x7ff2e8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:09.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.854+0000 7ff2e3fff700 1 -- 172.21.15.36:0/3733507840 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2e80104f0 con 0x7ff2f4140610 2022-06-17T12:57:09.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.854+0000 7ff2e3fff700 1 -- 172.21.15.36:0/3733507840 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e8010650 con 0x7ff2f4140610 2022-06-17T12:57:09.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.854+0000 7ff2e3fff700 1 -- 172.21.15.36:0/3733507840 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2e80109b0 con 0x7ff2f4140610 2022-06-17T12:57:09.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.854+0000 7ff2f3fff700 1 -- 172.21.15.36:0/3733507840 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f40b5760 con 0x7ff2f4140610 2022-06-17T12:57:09.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.854+0000 7ff2f3fff700 1 -- 172.21.15.36:0/3733507840 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f4114e40 con 0x7ff2f4140610 2022-06-17T12:57:09.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.854+0000 7ff2f3fff700 1 -- 172.21.15.36:0/3733507840 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2f4057410 con 0x7ff2f4140610 2022-06-17T12:57:09.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.858+0000 7ff2e3fff700 1 -- 172.21.15.36:0/3733507840 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff2e801a030 con 0x7ff2f4140610 2022-06-17T12:57:09.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.858+0000 7ff2e3fff700 1 --2- 172.21.15.36:0/3733507840 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff2d00539d0 0x7ff2d0055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:09.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.858+0000 7ff2e3fff700 1 -- 172.21.15.36:0/3733507840 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5686+0+0 (secure 0 0 0) 0x7ff2e806b760 con 0x7ff2f4140610 2022-06-17T12:57:09.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.858+0000 7ff2f2ffd700 1 --2- 172.21.15.36:0/3733507840 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff2d00539d0 0x7ff2d0055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:09.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.858+0000 7ff2f2ffd700 1 --2- 172.21.15.36:0/3733507840 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff2d00539d0 0x7ff2d0055e90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7ff2ec01b000 tx=0x7ff2ec016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:09.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:09.862+0000 7ff2e3fff700 1 -- 172.21.15.36:0/3733507840 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff2e8042840 con 0x7ff2f4140610 2022-06-17T12:57:10.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.030+0000 7ff2f3fff700 1 -- 172.21.15.36:0/3733507840 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2f405fb30 con 0x7ff2f4140610 2022-06-17T12:57:10.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.030+0000 7ff2e3fff700 1 -- 172.21.15.36:0/3733507840 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+1209 (secure 0 0 0) 0x7ff2e8045e60 con 0x7ff2f4140610 2022-06-17T12:57:10.017 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:57:10.017 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":80,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1081450840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1081450840},{"type":"v1","addr":"172.21.15.70:6833","nonce":1081450840}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":76},{"gid":8715,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/3373137980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":3373137980},{"type":"v1","addr":"172.21.15.36:6835","nonce":3373137980}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":80}],"filesystems":[]} 2022-06-17T12:57:10.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.030+0000 7ff2f3fff700 1 -- 172.21.15.36:0/3733507840 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff2d00539d0 msgr2=0x7ff2d0055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:10.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.030+0000 7ff2f3fff700 1 --2- 172.21.15.36:0/3733507840 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff2d00539d0 0x7ff2d0055e90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7ff2ec01b000 tx=0x7ff2ec016000).stop 2022-06-17T12:57:10.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.030+0000 7ff2f3fff700 1 -- 172.21.15.36:0/3733507840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f4140610 msgr2=0x7ff2f4152c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:10.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.030+0000 7ff2f3fff700 1 --2- 172.21.15.36:0/3733507840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f4140610 0x7ff2f4152c90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7ff2e800a000 tx=0x7ff2e8008040).stop 2022-06-17T12:57:10.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.030+0000 7ff2f3fff700 1 -- 172.21.15.36:0/3733507840 shutdown_connections 2022-06-17T12:57:10.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.030+0000 7ff2f3fff700 1 --2- 172.21.15.36:0/3733507840 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff2d00539d0 0x7ff2d0055e90 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:10.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.030+0000 7ff2f3fff700 1 --2- 172.21.15.36:0/3733507840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f4140610 0x7ff2f4152c90 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:10.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.030+0000 7ff2f3fff700 1 --2- 172.21.15.36:0/3733507840 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff2f4064eb0 0x7ff2f40b6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:10.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.030+0000 7ff2f3fff700 1 --2- 172.21.15.36:0/3733507840 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff2f4063730 0x7ff2f40aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:10.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.030+0000 7ff2f3fff700 1 -- 172.21.15.36:0/3733507840 >> 172.21.15.36:0/3733507840 conn(0x7ff2f40614b0 msgr2=0x7ff2f41443d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:10.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.034+0000 7ff2f3fff700 1 -- 172.21.15.36:0/3733507840 shutdown_connections 2022-06-17T12:57:10.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.034+0000 7ff2f3fff700 1 -- 172.21.15.36:0/3733507840 wait complete. 2022-06-17T12:57:10.043 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 80 2022-06-17T12:57:10.044 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-17T12:57:10.045 INFO:tasks.ceph.mds.a:Stopping old one... 2022-06-17T12:57:10.046 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-17T12:57:10.054 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T12:57:10.057 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-17T12:57:10.057 INFO:tasks.ceph.mds.b:Stopping old one... 2022-06-17T12:57:10.058 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-17T12:57:10.059 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T12:57:10.063 INFO:tasks.ceph.mds.a:Stopped 2022-06-17T12:57:10.064 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-17T12:57:10.081 INFO:tasks.ceph.mds.b:Stopped 2022-06-17T12:57:10.082 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-17T12:57:10.092 INFO:tasks.ceph.mds.b:Started 2022-06-17T12:57:10.092 DEBUG:teuthology.parallel:result is None 2022-06-17T12:57:10.128 INFO:tasks.ceph.mds.a:Started 2022-06-17T12:57:10.129 DEBUG:teuthology.parallel:result is None 2022-06-17T12:57:10.130 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-06-17T12:57:10.160 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2022-06-17T12:57:10.198 INFO:tasks.ceph.mds.a.smithi036.stdout:starting mds.a at 2022-06-17T12:57:10.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.250+0000 7fefe6786700 1 Processor -- start 2022-06-17T12:57:10.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.250+0000 7fefe6786700 1 -- start start 2022-06-17T12:57:10.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.250+0000 7fefe6786700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fefe013cb70 0x7fefe0143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:10.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.250+0000 7fefe6786700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fefe00aef60 0x7fefe00b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:10.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.250+0000 7fefe6786700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe00af4b0 con 0x7fefe00ba2e0 2022-06-17T12:57:10.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.250+0000 7fefe6786700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe0142f50 con 0x7fefe013cb70 2022-06-17T12:57:10.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.250+0000 7fefe6786700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe01431f0 con 0x7fefe00aef60 2022-06-17T12:57:10.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.250+0000 7fefe5784700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fefe013cb70 0x7fefe0143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:10.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.250+0000 7fefe5784700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fefe013cb70 0x7fefe0143690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40526/0 (socket says 172.21.15.36:40526) 2022-06-17T12:57:10.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.250+0000 7fefe5784700 1 -- 172.21.15.36:0/3558781846 learned_addr learned my addr 172.21.15.36:0/3558781846 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:10.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.250+0000 7fefe4f83700 1 --2- 172.21.15.36:0/3558781846 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fefe00aef60 0x7fefe00b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:10.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.250+0000 7fefe5784700 1 -- 172.21.15.36:0/3558781846 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fefe00aef60 msgr2=0x7fefe00b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:10.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.250+0000 7fefe5784700 1 --2- 172.21.15.36:0/3558781846 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fefe00aef60 0x7fefe00b9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:10.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.250+0000 7fefe5784700 1 -- 172.21.15.36:0/3558781846 >> v1:172.21.15.70:6789/0 conn(0x7fefe00ba2e0 legacy=0x7fefe01442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:10.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.250+0000 7fefe5784700 1 -- 172.21.15.36:0/3558781846 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe01449b0 con 0x7fefe013cb70 2022-06-17T12:57:10.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.250+0000 7fefe4f83700 1 --2- 172.21.15.36:0/3558781846 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fefe00aef60 0x7fefe00b9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:57:10.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.254+0000 7fefe5784700 1 --2- 172.21.15.36:0/3558781846 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fefe013cb70 0x7fefe0143690 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7fefd80032f0 tx=0x7fefd8009290).ready entity=mon.0 client_cookie=5febf4077815c9a8 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:10.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.254+0000 7fefdffff700 1 -- 172.21.15.36:0/3558781846 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefd800b8a0 con 0x7fefe013cb70 2022-06-17T12:57:10.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.254+0000 7fefdffff700 1 -- 172.21.15.36:0/3558781846 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd800ba00 con 0x7fefe013cb70 2022-06-17T12:57:10.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.254+0000 7fefdffff700 1 -- 172.21.15.36:0/3558781846 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefd800bb70 con 0x7fefe013cb70 2022-06-17T12:57:10.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.254+0000 7fefe6786700 1 -- 172.21.15.36:0/3558781846 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fefe013cb70 msgr2=0x7fefe0143690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:10.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.254+0000 7fefe6786700 1 --2- 172.21.15.36:0/3558781846 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fefe013cb70 0x7fefe0143690 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7fefd80032f0 tx=0x7fefd8009290).stop 2022-06-17T12:57:10.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.254+0000 7fefe6786700 1 -- 172.21.15.36:0/3558781846 shutdown_connections 2022-06-17T12:57:10.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.254+0000 7fefe6786700 1 --2- 172.21.15.36:0/3558781846 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fefe00aef60 0x7fefe00b9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:10.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.254+0000 7fefe6786700 1 --2- 172.21.15.36:0/3558781846 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fefe013cb70 0x7fefe0143690 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:10.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.254+0000 7fefe6786700 1 -- 172.21.15.36:0/3558781846 >> 172.21.15.36:0/3558781846 conn(0x7fefe005c550 msgr2=0x7fefe00b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:10.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.254+0000 7fefe6786700 1 -- 172.21.15.36:0/3558781846 shutdown_connections 2022-06-17T12:57:10.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.254+0000 7fefe6786700 1 -- 172.21.15.36:0/3558781846 wait complete. 2022-06-17T12:57:10.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.254+0000 7fefe6786700 1 Processor -- start 2022-06-17T12:57:10.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.254+0000 7fefe6786700 1 -- start start 2022-06-17T12:57:10.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefe6786700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fefe00aef60 0x7fefe0063c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:10.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefe6786700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefe00ba2e0 0x7fefe00673d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:10.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefe6786700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fefe013cb70 0x7fefe013b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:10.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefe6786700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe0066610 con 0x7fefe00aef60 2022-06-17T12:57:10.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefe6786700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe0065380 con 0x7fefe00ba2e0 2022-06-17T12:57:10.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefe6786700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe00655e0 con 0x7fefe013cb70 2022-06-17T12:57:10.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefe5784700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fefe00aef60 0x7fefe0063c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:10.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefe4f83700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefe00ba2e0 0x7fefe00673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:10.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefe5784700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fefe00aef60 0x7fefe0063c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40532/0 (socket says 172.21.15.36:40532) 2022-06-17T12:57:10.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefe4f83700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefe00ba2e0 0x7fefe00673d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:60916/0 (socket says 172.21.15.36:60916) 2022-06-17T12:57:10.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefe4f83700 1 -- 172.21.15.36:0/2249914616 learned_addr learned my addr 172.21.15.36:0/2249914616 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:10.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefe5f85700 1 --2- 172.21.15.36:0/2249914616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fefe013cb70 0x7fefe013b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:10.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefe5784700 1 -- 172.21.15.36:0/2249914616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fefe013cb70 msgr2=0x7fefe013b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:10.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefe5784700 1 --2- 172.21.15.36:0/2249914616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fefe013cb70 0x7fefe013b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:10.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefe5784700 1 -- 172.21.15.36:0/2249914616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefe00ba2e0 msgr2=0x7fefe00673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:10.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefe5784700 1 --2- 172.21.15.36:0/2249914616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefe00ba2e0 0x7fefe00673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:10.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefe5784700 1 -- 172.21.15.36:0/2249914616 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd8009530 con 0x7fefe00aef60 2022-06-17T12:57:10.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefe4f83700 1 --2- 172.21.15.36:0/2249914616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefe00ba2e0 0x7fefe00673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:57:10.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefe5f85700 1 --2- 172.21.15.36:0/2249914616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fefe013cb70 0x7fefe013b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:57:10.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefe5784700 1 --2- 172.21.15.36:0/2249914616 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fefe00aef60 0x7fefe0063c80 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 rx=0x7fefd80032f0 tx=0x7fefd8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:10.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefde7fc700 1 -- 172.21.15.36:0/2249914616 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefd8021480 con 0x7fefe00aef60 2022-06-17T12:57:10.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefde7fc700 1 -- 172.21.15.36:0/2249914616 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd80215e0 con 0x7fefe00aef60 2022-06-17T12:57:10.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefde7fc700 1 -- 172.21.15.36:0/2249914616 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefd8021940 con 0x7fefe00aef60 2022-06-17T12:57:10.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefe6786700 1 -- 172.21.15.36:0/2249914616 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe0065880 con 0x7fefe00aef60 2022-06-17T12:57:10.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefe6786700 1 -- 172.21.15.36:0/2249914616 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe00636c0 con 0x7fefe00aef60 2022-06-17T12:57:10.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.258+0000 7fefde7fc700 1 -- 172.21.15.36:0/2249914616 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fefd800f070 con 0x7fefe00aef60 2022-06-17T12:57:10.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.262+0000 7fefe6786700 1 -- 172.21.15.36:0/2249914616 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe0057410 con 0x7fefe00aef60 2022-06-17T12:57:10.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.262+0000 7fefde7fc700 1 --2- 172.21.15.36:0/2249914616 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fefc0053a40 0x7fefc0055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:10.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.262+0000 7fefde7fc700 1 -- 172.21.15.36:0/2249914616 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fefd806cbe0 con 0x7fefe00aef60 2022-06-17T12:57:10.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.262+0000 7fefe4f83700 1 --2- 172.21.15.36:0/2249914616 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fefc0053a40 0x7fefc0055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:10.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.262+0000 7fefe4f83700 1 --2- 172.21.15.36:0/2249914616 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fefc0053a40 0x7fefc0055f00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7fefd4008000 tx=0x7fefd4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:10.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.266+0000 7fefde7fc700 1 -- 172.21.15.36:0/2249914616 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fefd8043f80 con 0x7fefe00aef60 2022-06-17T12:57:10.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:10.402+0000 7fefe6786700 1 -- 172.21.15.36:0/2249914616 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fefe0067bf0 con 0x7fefe00aef60 2022-06-17T12:57:11.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.242+0000 7fefde7fc700 1 -- 172.21.15.36:0/2249914616 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v146) v1 ==== 97+0+0 (secure 0 0 0) 0x7fefd8015030 con 0x7fefe00aef60 2022-06-17T12:57:11.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.242+0000 7fefe6786700 1 -- 172.21.15.36:0/2249914616 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fefc0053a40 msgr2=0x7fefc0055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:11.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.242+0000 7fefe6786700 1 --2- 172.21.15.36:0/2249914616 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fefc0053a40 0x7fefc0055f00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7fefd4008000 tx=0x7fefd4008040).stop 2022-06-17T12:57:11.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.242+0000 7fefe6786700 1 -- 172.21.15.36:0/2249914616 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fefe00aef60 msgr2=0x7fefe0063c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:11.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.242+0000 7fefe6786700 1 --2- 172.21.15.36:0/2249914616 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fefe00aef60 0x7fefe0063c80 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 rx=0x7fefd80032f0 tx=0x7fefd8016000).stop 2022-06-17T12:57:11.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.246+0000 7fefe6786700 1 -- 172.21.15.36:0/2249914616 shutdown_connections 2022-06-17T12:57:11.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.246+0000 7fefe6786700 1 --2- 172.21.15.36:0/2249914616 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fefc0053a40 0x7fefc0055f00 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:11.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.246+0000 7fefe6786700 1 --2- 172.21.15.36:0/2249914616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fefe013cb70 0x7fefe013b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:11.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.246+0000 7fefe6786700 1 --2- 172.21.15.36:0/2249914616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefe00ba2e0 0x7fefe00673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:11.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.246+0000 7fefe6786700 1 --2- 172.21.15.36:0/2249914616 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fefe00aef60 0x7fefe0063c80 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:11.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.246+0000 7fefe6786700 1 -- 172.21.15.36:0/2249914616 >> 172.21.15.36:0/2249914616 conn(0x7fefe005c550 msgr2=0x7fefe00b96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:11.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.246+0000 7fefe6786700 1 -- 172.21.15.36:0/2249914616 shutdown_connections 2022-06-17T12:57:11.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.246+0000 7fefe6786700 1 -- 172.21.15.36:0/2249914616 wait complete. 2022-06-17T12:57:11.249 INFO:teuthology.orchestra.run.smithi036.stderr: removed all blacklist entries 2022-06-17T12:57:11.250 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2022-06-17T12:57:11.251 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-06-17T12:57:11.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a9a6c700 1 Processor -- start 2022-06-17T12:57:11.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a9a6c700 1 -- start start 2022-06-17T12:57:11.384 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a9a6c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f67a40620d0 0x7f67a40b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:11.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a9a6c700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f67a40b31c0 0x7f67a40b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:11.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a9a6c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67a40b4700 0x7f67a40b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:11.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a9a6c700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a40b21d0 con 0x7f67a40620d0 2022-06-17T12:57:11.385 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a9a6c700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a40b5080 con 0x7f67a40b31c0 2022-06-17T12:57:11.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a9a6c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a40b2430 con 0x7f67a40b4700 2022-06-17T12:57:11.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a37fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f67a40620d0 0x7f67a40b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:11.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a3fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67a40b4700 0x7f67a40b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:11.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a2ffd700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f67a40b31c0 0x7f67a40b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:11.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a37fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f67a40620d0 0x7f67a40b5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40542/0 (socket says 172.21.15.36:40542) 2022-06-17T12:57:11.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a3fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67a40b4700 0x7f67a40b4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:55332/0 (socket says 172.21.15.36:55332) 2022-06-17T12:57:11.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a37fe700 1 -- 172.21.15.36:0/390344505 learned_addr learned my addr 172.21.15.36:0/390344505 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:11.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a37fe700 1 -- 172.21.15.36:0/390344505 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67a40b4700 msgr2=0x7f67a40b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:11.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a37fe700 1 --2- 172.21.15.36:0/390344505 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67a40b4700 0x7f67a40b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:11.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a37fe700 1 -- 172.21.15.36:0/390344505 >> v2:172.21.15.70:3300/0 conn(0x7f67a40b31c0 msgr2=0x7f67a40b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:11.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a37fe700 1 --2- 172.21.15.36:0/390344505 >> v2:172.21.15.70:3300/0 conn(0x7f67a40b31c0 0x7f67a40b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:11.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a37fe700 1 -- 172.21.15.36:0/390344505 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a40b26d0 con 0x7f67a40620d0 2022-06-17T12:57:11.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a3fff700 1 --2- 172.21.15.36:0/390344505 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67a40b4700 0x7f67a40b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:57:11.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a2ffd700 1 --2- 172.21.15.36:0/390344505 >> v2:172.21.15.70:3300/0 conn(0x7f67a40b31c0 0x7f67a40b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:57:11.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a37fe700 1 --2- 172.21.15.36:0/390344505 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67a40620d0 0x7f67a40b5bc0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7f67940032f0 tx=0x7f6794009290).ready entity=mon.0 client_cookie=8605c2042ff4762c server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:11.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a27fc700 1 -- 172.21.15.36:0/390344505 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f679400b680 con 0x7f67a40620d0 2022-06-17T12:57:11.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a27fc700 1 -- 172.21.15.36:0/390344505 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679400b7e0 con 0x7f67a40620d0 2022-06-17T12:57:11.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.386+0000 7f67a27fc700 1 -- 172.21.15.36:0/390344505 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f679400b950 con 0x7f67a40620d0 2022-06-17T12:57:11.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.390+0000 7f67a9a6c700 1 -- 172.21.15.36:0/390344505 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67a40620d0 msgr2=0x7f67a40b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:11.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.390+0000 7f67a9a6c700 1 --2- 172.21.15.36:0/390344505 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67a40620d0 0x7f67a40b5bc0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7f67940032f0 tx=0x7f6794009290).stop 2022-06-17T12:57:11.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.390+0000 7f67a9a6c700 1 -- 172.21.15.36:0/390344505 shutdown_connections 2022-06-17T12:57:11.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.390+0000 7f67a9a6c700 1 --2- 172.21.15.36:0/390344505 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67a40b4700 0x7f67a40b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:11.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.390+0000 7f67a9a6c700 1 --2- 172.21.15.36:0/390344505 >> v2:172.21.15.70:3300/0 conn(0x7f67a40b31c0 0x7f67a40b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:11.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.390+0000 7f67a9a6c700 1 --2- 172.21.15.36:0/390344505 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67a40620d0 0x7f67a40b5bc0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:11.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.390+0000 7f67a9a6c700 1 -- 172.21.15.36:0/390344505 >> 172.21.15.36:0/390344505 conn(0x7f67a40b9850 msgr2=0x7f67a40b9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:11.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.390+0000 7f67a9a6c700 1 -- 172.21.15.36:0/390344505 shutdown_connections 2022-06-17T12:57:11.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.390+0000 7f67a9a6c700 1 -- 172.21.15.36:0/390344505 wait complete. 2022-06-17T12:57:11.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.390+0000 7f67a9a6c700 1 Processor -- start 2022-06-17T12:57:11.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.390+0000 7f67a9a6c700 1 -- start start 2022-06-17T12:57:11.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.390+0000 7f67a9a6c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67a40620d0 0x7f67a4112d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:11.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.390+0000 7f67a9a6c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67a40b31c0 0x7f67a413e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:11.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.390+0000 7f67a9a6c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67a40b4700 0x7f67a413e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:11.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.390+0000 7f67a9a6c700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a40b5490 con 0x7f67a40b31c0 2022-06-17T12:57:11.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.390+0000 7f67a9a6c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a41468c0 con 0x7f67a40620d0 2022-06-17T12:57:11.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.390+0000 7f67a9a6c700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a4145010 con 0x7f67a40b4700 2022-06-17T12:57:11.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.390+0000 7f67a2ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67a40b31c0 0x7f67a413e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:11.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.390+0000 7f67a2ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67a40b31c0 0x7f67a413e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40548/0 (socket says 172.21.15.36:40548) 2022-06-17T12:57:11.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.390+0000 7f67a2ffd700 1 -- 172.21.15.36:0/1378462439 learned_addr learned my addr 172.21.15.36:0/1378462439 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:11.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.390+0000 7f67a37fe700 1 --2- 172.21.15.36:0/1378462439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67a40620d0 0x7f67a4112d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:11.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.390+0000 7f67a3fff700 1 --2- 172.21.15.36:0/1378462439 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67a40b4700 0x7f67a413e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:11.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.394+0000 7f67a2ffd700 1 -- 172.21.15.36:0/1378462439 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67a40b4700 msgr2=0x7f67a413e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:11.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.394+0000 7f67a2ffd700 1 --2- 172.21.15.36:0/1378462439 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67a40b4700 0x7f67a413e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:11.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.394+0000 7f67a2ffd700 1 -- 172.21.15.36:0/1378462439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67a40620d0 msgr2=0x7f67a4112d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:11.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.394+0000 7f67a2ffd700 1 --2- 172.21.15.36:0/1378462439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67a40620d0 0x7f67a4112d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:11.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.394+0000 7f67a2ffd700 1 -- 172.21.15.36:0/1378462439 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c00b180 con 0x7f67a40b31c0 2022-06-17T12:57:11.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.394+0000 7f67a2ffd700 1 --2- 172.21.15.36:0/1378462439 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67a40b31c0 0x7f67a413e350 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0x7f678c00c000 tx=0x7f678c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:11.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.394+0000 7f67a0ff9700 1 -- 172.21.15.36:0/1378462439 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f678c00c5f0 con 0x7f67a40b31c0 2022-06-17T12:57:11.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.394+0000 7f67a0ff9700 1 -- 172.21.15.36:0/1378462439 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678c00c750 con 0x7f67a40b31c0 2022-06-17T12:57:11.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.394+0000 7f67a9a6c700 1 -- 172.21.15.36:0/1378462439 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679c00b180 con 0x7f67a40b31c0 2022-06-17T12:57:11.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.394+0000 7f67a0ff9700 1 -- 172.21.15.36:0/1378462439 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f678c00cab0 con 0x7f67a40b31c0 2022-06-17T12:57:11.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.394+0000 7f67a9a6c700 1 -- 172.21.15.36:0/1378462439 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6794009530 con 0x7f67a40b31c0 2022-06-17T12:57:11.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.394+0000 7f67a9a6c700 1 -- 172.21.15.36:0/1378462439 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a40b43f0 con 0x7f67a40b31c0 2022-06-17T12:57:11.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.394+0000 7f67a0ff9700 1 -- 172.21.15.36:0/1378462439 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f678c00cc10 con 0x7f67a40b31c0 2022-06-17T12:57:11.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.394+0000 7f67a0ff9700 1 --2- 172.21.15.36:0/1378462439 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f677c053940 0x7f677c055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:11.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.394+0000 7f67a0ff9700 1 -- 172.21.15.36:0/1378462439 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f678c06c180 con 0x7f67a40b31c0 2022-06-17T12:57:11.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.394+0000 7f67a37fe700 1 --2- 172.21.15.36:0/1378462439 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f677c053940 0x7f677c055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:11.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.398+0000 7f67a37fe700 1 --2- 172.21.15.36:0/1378462439 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f677c053940 0x7f677c055e00 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f679401a000 tx=0x7f6794017000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:11.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.402+0000 7f67a0ff9700 1 -- 172.21.15.36:0/1378462439 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f678c0432c0 con 0x7f67a40b31c0 2022-06-17T12:57:11.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.570+0000 7f67a9a6c700 1 -- 172.21.15.36:0/1378462439 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f67a4057410 con 0x7f67a40b31c0 2022-06-17T12:57:11.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.570+0000 7f67a0ff9700 1 -- 172.21.15.36:0/1378462439 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v46) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f678c0468e0 con 0x7f67a40b31c0 2022-06-17T12:57:11.566 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:57:11.567 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:57:11.567 INFO:teuthology.orchestra.run.smithi036.stdout: "auth_dump": [ 2022-06-17T12:57:11.568 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:11.569 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mds.a", 2022-06-17T12:57:11.569 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiyaUgARAAxp7bzOl4uVo2EsaoRBeMRw==", 2022-06-17T12:57:11.570 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:57:11.570 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:57:11.571 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:57:11.572 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:57:11.573 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:57:11.574 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:11.574 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:11.575 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:11.575 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mds.b", 2022-06-17T12:57:11.576 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xifyxLCRAABI389jmz1zboeDanzraT8Q==", 2022-06-17T12:57:11.576 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:57:11.598 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:57:11.599 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:57:11.600 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:57:11.600 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:57:11.600 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:11.601 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:11.601 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:11.602 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.0", 2022-06-17T12:57:11.602 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB+d6xibhXGNBAA6AuUT9ITWAZjDFpNsgA4Wg==", 2022-06-17T12:57:11.602 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:57:11.603 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:57:11.603 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:57:11.604 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:57:11.604 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:11.604 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:11.605 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:11.605 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.1", 2022-06-17T12:57:11.605 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCBd6xiOIPvGxAADkrBoJCy9x91zan6TWU0tQ==", 2022-06-17T12:57:11.606 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:57:11.606 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:57:11.607 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:57:11.607 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:57:11.607 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:11.608 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:11.608 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:11.608 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.2", 2022-06-17T12:57:11.609 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCEd6xiP4/9AhAAaP3cWXV6vbfGBPfqSLKqUg==", 2022-06-17T12:57:11.611 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:57:11.611 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:57:11.612 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:57:11.612 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:57:11.613 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:11.613 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:11.613 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:11.614 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.3", 2022-06-17T12:57:11.614 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCGd6xi+LZgJxAAJjGwP5qW6TFDZfR+2rz6ig==", 2022-06-17T12:57:11.614 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:57:11.615 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:57:11.615 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:57:11.616 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:57:11.617 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:11.617 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:11.618 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:11.618 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.4", 2022-06-17T12:57:11.619 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCKd6xikvrTChAAXIyWSETswDv+tb3zsGyYpA==", 2022-06-17T12:57:11.619 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:57:11.619 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:57:11.621 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:57:11.622 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:57:11.622 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:11.623 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:11.623 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:11.624 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.5", 2022-06-17T12:57:11.624 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCMd6xiqr2QLhAArhimHuNsVWrDGGreFYxj6A==", 2022-06-17T12:57:11.625 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:57:11.625 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:57:11.625 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:57:11.626 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:57:11.626 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:11.627 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:11.627 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:11.628 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.6", 2022-06-17T12:57:11.628 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCPd6xiUXcXGxAAdrYz8txdBWMIqF5DF4Aimw==", 2022-06-17T12:57:11.628 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:57:11.629 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:57:11.630 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:57:11.630 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:57:11.632 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:11.633 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:11.633 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:11.634 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "osd.7", 2022-06-17T12:57:11.634 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQCSd6xiBJ2IAhAAUkNfgmlxxWY7+VouPLAvyw==", 2022-06-17T12:57:11.634 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:57:11.635 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:57:11.636 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:57:11.636 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:57:11.636 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:11.637 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:11.637 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:11.638 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.0", 2022-06-17T12:57:11.638 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiXO3QFBAAsSLtQ0yKKZxhqUT/eYKdtg==", 2022-06-17T12:57:11.639 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:57:11.639 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:57:11.639 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:57:11.640 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:57:11.640 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:11.641 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:11.641 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:11.653 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.1", 2022-06-17T12:57:11.653 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xi51KdGBAAi7fGZ+Qepg6j3DAhlzoWxw==", 2022-06-17T12:57:11.654 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:57:11.654 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:57:11.654 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:57:11.655 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:57:11.655 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:11.656 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:11.657 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:11.657 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.2", 2022-06-17T12:57:11.657 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB7d6xiug2hHBAAqHpLgwMdthSWtJ8Vwu+m1Q==", 2022-06-17T12:57:11.658 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:57:11.658 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow", 2022-06-17T12:57:11.659 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow r", 2022-06-17T12:57:11.659 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-17T12:57:11.660 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:11.660 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:11.661 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:11.661 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "client.admin", 2022-06-17T12:57:11.661 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB5d6xiGDODKhAAPZMG/M2nVssdr4FJHcS6XQ==", 2022-06-17T12:57:11.662 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:57:11.662 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:57:11.663 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr": "allow *", 2022-06-17T12:57:11.664 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow *", 2022-06-17T12:57:11.664 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:57:11.670 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:11.671 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:11.671 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:11.671 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mgr.x", 2022-06-17T12:57:11.672 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB6d6xiyH9uORAATJb0chHE1CH0X02w7edVtw==", 2022-06-17T12:57:11.672 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:57:11.672 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:57:11.673 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile mgr", 2022-06-17T12:57:11.679 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:57:11.680 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:11.681 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:11.681 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:11.682 INFO:teuthology.orchestra.run.smithi036.stdout: "entity": "mgr.y", 2022-06-17T12:57:11.682 INFO:teuthology.orchestra.run.smithi036.stdout: "key": "AQB6d6xiHXghMxAAzpsVRCbLWHWZPtrPk1jFlA==", 2022-06-17T12:57:11.683 INFO:teuthology.orchestra.run.smithi036.stdout: "caps": { 2022-06-17T12:57:11.683 INFO:teuthology.orchestra.run.smithi036.stdout: "mds": "allow *", 2022-06-17T12:57:11.684 INFO:teuthology.orchestra.run.smithi036.stdout: "mon": "allow profile mgr", 2022-06-17T12:57:11.685 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": "allow *" 2022-06-17T12:57:11.685 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:11.686 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:11.686 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:11.687 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:57:11.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.570+0000 7f67a9a6c700 1 -- 172.21.15.36:0/1378462439 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f677c053940 msgr2=0x7f677c055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:11.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.570+0000 7f67a9a6c700 1 --2- 172.21.15.36:0/1378462439 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f677c053940 0x7f677c055e00 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f679401a000 tx=0x7f6794017000).stop 2022-06-17T12:57:11.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.570+0000 7f67a9a6c700 1 -- 172.21.15.36:0/1378462439 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67a40b31c0 msgr2=0x7f67a413e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:11.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.570+0000 7f67a9a6c700 1 --2- 172.21.15.36:0/1378462439 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67a40b31c0 0x7f67a413e350 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0x7f678c00c000 tx=0x7f678c006df0).stop 2022-06-17T12:57:11.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.570+0000 7f67a9a6c700 1 -- 172.21.15.36:0/1378462439 shutdown_connections 2022-06-17T12:57:11.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.570+0000 7f67a9a6c700 1 --2- 172.21.15.36:0/1378462439 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f677c053940 0x7f677c055e00 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:11.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.570+0000 7f67a9a6c700 1 --2- 172.21.15.36:0/1378462439 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f67a40b4700 0x7f67a413e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:11.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.570+0000 7f67a9a6c700 1 --2- 172.21.15.36:0/1378462439 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f67a40b31c0 0x7f67a413e350 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:11.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.570+0000 7f67a9a6c700 1 --2- 172.21.15.36:0/1378462439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67a40620d0 0x7f67a4112d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:11.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.570+0000 7f67a9a6c700 1 -- 172.21.15.36:0/1378462439 >> 172.21.15.36:0/1378462439 conn(0x7f67a40b9850 msgr2=0x7f67a4141690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:11.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.570+0000 7f67a9a6c700 1 -- 172.21.15.36:0/1378462439 shutdown_connections 2022-06-17T12:57:11.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.570+0000 7f67a9a6c700 1 -- 172.21.15.36:0/1378462439 wait complete. 2022-06-17T12:57:11.698 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:57:11.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.830+0000 7fcec5777700 1 Processor -- start 2022-06-17T12:57:11.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.830+0000 7fcec5777700 1 -- start start 2022-06-17T12:57:11.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.830+0000 7fcec5777700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcec00ad330 0x7fcec0145070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:11.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.830+0000 7fcec5777700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcec0066fd0 0x7fcec013df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:11.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.830+0000 7fcec5777700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec01417a0 con 0x7fcec00b7c10 2022-06-17T12:57:11.886 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.830+0000 7fcec5777700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcec01422e0 con 0x7fcec0066fd0 2022-06-17T12:57:11.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.830+0000 7fcec5777700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcec00b7ff0 con 0x7fcec00ad330 2022-06-17T12:57:11.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.830+0000 7fcebf7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcec0066fd0 0x7fcec013df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:11.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.830+0000 7fcebf7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcec0066fd0 0x7fcec013df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40558/0 (socket says 172.21.15.36:40558) 2022-06-17T12:57:11.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.830+0000 7fcebf7fe700 1 -- 172.21.15.36:0/2382286632 learned_addr learned my addr 172.21.15.36:0/2382286632 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:11.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.830+0000 7fcebe7fc700 1 --2- 172.21.15.36:0/2382286632 >> v2:172.21.15.70:3300/0 conn(0x7fcec00ad330 0x7fcec0145070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:11.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.830+0000 7fcebf7fe700 1 -- 172.21.15.36:0/2382286632 >> v2:172.21.15.70:3300/0 conn(0x7fcec00ad330 msgr2=0x7fcec0145070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:11.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.830+0000 7fcebf7fe700 1 --2- 172.21.15.36:0/2382286632 >> v2:172.21.15.70:3300/0 conn(0x7fcec00ad330 0x7fcec0145070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:11.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.830+0000 7fcebf7fe700 1 -- 172.21.15.36:0/2382286632 >> v1:172.21.15.70:6789/0 conn(0x7fcec00b7c10 legacy=0x7fcec0141090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:11.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.830+0000 7fcebf7fe700 1 -- 172.21.15.36:0/2382286632 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec00b30a0 con 0x7fcec0066fd0 2022-06-17T12:57:11.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.830+0000 7fcebf7fe700 1 --2- 172.21.15.36:0/2382286632 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcec0066fd0 0x7fcec013df70 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7fceb8002f70 tx=0x7fceb8006eb0).ready entity=mon.0 client_cookie=74875db3f415f1ce server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:11.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.830+0000 7fcebdffb700 1 -- 172.21.15.36:0/2382286632 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fceb800d5c0 con 0x7fcec0066fd0 2022-06-17T12:57:11.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.830+0000 7fcebdffb700 1 -- 172.21.15.36:0/2382286632 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb8000e90 con 0x7fcec0066fd0 2022-06-17T12:57:11.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.830+0000 7fcebdffb700 1 -- 172.21.15.36:0/2382286632 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fceb800d950 con 0x7fcec0066fd0 2022-06-17T12:57:11.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.830+0000 7fcec5777700 1 -- 172.21.15.36:0/2382286632 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcec0066fd0 msgr2=0x7fcec013df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:11.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.830+0000 7fcec5777700 1 --2- 172.21.15.36:0/2382286632 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcec0066fd0 0x7fcec013df70 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7fceb8002f70 tx=0x7fceb8006eb0).stop 2022-06-17T12:57:11.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.834+0000 7fcec5777700 1 -- 172.21.15.36:0/2382286632 shutdown_connections 2022-06-17T12:57:11.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.834+0000 7fcec5777700 1 --2- 172.21.15.36:0/2382286632 >> v2:172.21.15.70:3300/0 conn(0x7fcec00ad330 0x7fcec0145070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:11.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.834+0000 7fcec5777700 1 --2- 172.21.15.36:0/2382286632 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcec0066fd0 0x7fcec013df70 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:11.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.834+0000 7fcec5777700 1 -- 172.21.15.36:0/2382286632 >> 172.21.15.36:0/2382286632 conn(0x7fcec0060c60 msgr2=0x7fcec005c550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:11.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.834+0000 7fcec5777700 1 -- 172.21.15.36:0/2382286632 shutdown_connections 2022-06-17T12:57:11.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.834+0000 7fcec5777700 1 -- 172.21.15.36:0/2382286632 wait complete. 2022-06-17T12:57:11.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.834+0000 7fcec5777700 1 Processor -- start 2022-06-17T12:57:11.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.834+0000 7fcec5777700 1 -- start start 2022-06-17T12:57:11.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.838+0000 7fcec5777700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcec0066fd0 0x7fcec00b4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:11.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.838+0000 7fcec5777700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcec00ad330 0x7fcec0152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:11.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.838+0000 7fcec5777700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcec00b7c10 0x7fcec013cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:11.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.838+0000 7fcec5777700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec01536e0 con 0x7fcec0066fd0 2022-06-17T12:57:11.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.838+0000 7fcec5777700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec0155cf0 con 0x7fcec00ad330 2022-06-17T12:57:11.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.838+0000 7fcec5777700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec0110c40 con 0x7fcec00b7c10 2022-06-17T12:57:11.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.838+0000 7fcebeffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcec0066fd0 0x7fcec00b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:11.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.838+0000 7fcebeffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcec0066fd0 0x7fcec00b4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40560/0 (socket says 172.21.15.36:40560) 2022-06-17T12:57:11.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.838+0000 7fcebeffd700 1 -- 172.21.15.36:0/1860718546 learned_addr learned my addr 172.21.15.36:0/1860718546 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:11.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.838+0000 7fcebe7fc700 1 --2- 172.21.15.36:0/1860718546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcec00ad330 0x7fcec0152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:11.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.838+0000 7fcebf7fe700 1 --2- 172.21.15.36:0/1860718546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcec00b7c10 0x7fcec013cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:11.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.838+0000 7fcebeffd700 1 -- 172.21.15.36:0/1860718546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcec00b7c10 msgr2=0x7fcec013cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:11.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.838+0000 7fcebeffd700 1 --2- 172.21.15.36:0/1860718546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcec00b7c10 0x7fcec013cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:11.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.838+0000 7fcebeffd700 1 -- 172.21.15.36:0/1860718546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcec00ad330 msgr2=0x7fcec0152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:11.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.838+0000 7fcebeffd700 1 --2- 172.21.15.36:0/1860718546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcec00ad330 0x7fcec0152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:11.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.838+0000 7fcebeffd700 1 -- 172.21.15.36:0/1860718546 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb800b1e0 con 0x7fcec0066fd0 2022-06-17T12:57:11.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.838+0000 7fcebeffd700 1 --2- 172.21.15.36:0/1860718546 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcec0066fd0 0x7fcec00b4230 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7fcec00ad7a0 tx=0x7fceb0001400).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:11.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.838+0000 7fceaffff700 1 -- 172.21.15.36:0/1860718546 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fceb000e4a0 con 0x7fcec0066fd0 2022-06-17T12:57:11.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.838+0000 7fceaffff700 1 -- 172.21.15.36:0/1860718546 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb000e600 con 0x7fcec0066fd0 2022-06-17T12:57:11.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.838+0000 7fcec5777700 1 -- 172.21.15.36:0/1860718546 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec0110f10 con 0x7fcec0066fd0 2022-06-17T12:57:11.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.838+0000 7fceaffff700 1 -- 172.21.15.36:0/1860718546 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fceb000e960 con 0x7fcec0066fd0 2022-06-17T12:57:11.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.838+0000 7fcec5777700 1 -- 172.21.15.36:0/1860718546 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec013e090 con 0x7fcec0066fd0 2022-06-17T12:57:11.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.838+0000 7fceaffff700 1 -- 172.21.15.36:0/1860718546 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fceb0019030 con 0x7fcec0066fd0 2022-06-17T12:57:11.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.842+0000 7fcec5777700 1 -- 172.21.15.36:0/1860718546 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec00b3340 con 0x7fcec0066fd0 2022-06-17T12:57:11.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.842+0000 7fceaffff700 1 --2- 172.21.15.36:0/1860718546 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce9c0539d0 0x7fce9c055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:11.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.842+0000 7fceaffff700 1 -- 172.21.15.36:0/1860718546 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fceb006b530 con 0x7fcec0066fd0 2022-06-17T12:57:11.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.842+0000 7fcebe7fc700 1 --2- 172.21.15.36:0/1860718546 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce9c0539d0 0x7fce9c055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:11.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.842+0000 7fcebe7fc700 1 --2- 172.21.15.36:0/1860718546 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce9c0539d0 0x7fce9c055e90 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7fceb400b000 tx=0x7fceb4006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:11.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.850+0000 7fceaffff700 1 -- 172.21.15.36:0/1860718546 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fceb00425d0 con 0x7fcec0066fd0 2022-06-17T12:57:12.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.978+0000 7fcec5777700 1 -- 172.21.15.36:0/1860718546 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcec0057410 con 0x7fcec0066fd0 2022-06-17T12:57:12.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.978+0000 7fceaffff700 1 -- 172.21.15.36:0/1860718546 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) v1 ==== 74+0+13424 (secure 0 0 0) 0x7fceb0045bf0 con 0x7fcec0066fd0 2022-06-17T12:57:12.012 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:57:12.012 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":146,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:57:11.234112+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:57:12.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.978+0000 7fcec5777700 1 -- 172.21.15.36:0/1860718546 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce9c0539d0 msgr2=0x7fce9c055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:12.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.978+0000 7fcec5777700 1 --2- 172.21.15.36:0/1860718546 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce9c0539d0 0x7fce9c055e90 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7fceb400b000 tx=0x7fceb4006df0).stop 2022-06-17T12:57:12.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.978+0000 7fcec5777700 1 -- 172.21.15.36:0/1860718546 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcec0066fd0 msgr2=0x7fcec00b4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:12.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.978+0000 7fcec5777700 1 --2- 172.21.15.36:0/1860718546 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcec0066fd0 0x7fcec00b4230 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7fcec00ad7a0 tx=0x7fceb0001400).stop 2022-06-17T12:57:12.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.978+0000 7fcec5777700 1 -- 172.21.15.36:0/1860718546 shutdown_connections 2022-06-17T12:57:12.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.978+0000 7fcec5777700 1 --2- 172.21.15.36:0/1860718546 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fce9c0539d0 0x7fce9c055e90 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.982+0000 7fcec5777700 1 --2- 172.21.15.36:0/1860718546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcec00b7c10 0x7fcec013cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.982+0000 7fcec5777700 1 --2- 172.21.15.36:0/1860718546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcec00ad330 0x7fcec0152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.982+0000 7fcec5777700 1 --2- 172.21.15.36:0/1860718546 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcec0066fd0 0x7fcec00b4230 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.982+0000 7fcec5777700 1 -- 172.21.15.36:0/1860718546 >> 172.21.15.36:0/1860718546 conn(0x7fcec0060c60 msgr2=0x7fcec00b6990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:12.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.982+0000 7fcec5777700 1 -- 172.21.15.36:0/1860718546 shutdown_connections 2022-06-17T12:57:12.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:11.982+0000 7fcec5777700 1 -- 172.21.15.36:0/1860718546 wait complete. 2022-06-17T12:57:12.042 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-17T12:57:12.047 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:57:12.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.178+0000 7f6443fff700 1 Processor -- start 2022-06-17T12:57:12.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.178+0000 7f6443fff700 1 -- start start 2022-06-17T12:57:12.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.178+0000 7f6443fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64440601c0 0x7f6444144270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:12.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.178+0000 7f6443fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f64440b5be0 0x7f6444144760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:12.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.178+0000 7f6443fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6444147270 0x7f644414ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:12.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.178+0000 7f6443fff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64440b72d0 con 0x7f64440b5be0 2022-06-17T12:57:12.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.178+0000 7f6443fff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64440b0e50 con 0x7f6444147270 2022-06-17T12:57:12.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.178+0000 7f6443fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64440b2c80 con 0x7f64440601c0 2022-06-17T12:57:12.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.178+0000 7f64427fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f64440b5be0 0x7f6444144760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:12.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.178+0000 7f64427fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f64440b5be0 0x7f6444144760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40570/0 (socket says 172.21.15.36:40570) 2022-06-17T12:57:12.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.178+0000 7f64427fc700 1 -- 172.21.15.36:0/2608688266 learned_addr learned my addr 172.21.15.36:0/2608688266 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:12.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.178+0000 7f6442ffd700 1 --2- 172.21.15.36:0/2608688266 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64440601c0 0x7f6444144270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:12.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.178+0000 7f64437fe700 1 --2- 172.21.15.36:0/2608688266 >> v2:172.21.15.70:3300/0 conn(0x7f6444147270 0x7f644414ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:12.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.178+0000 7f64427fc700 1 -- 172.21.15.36:0/2608688266 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64440601c0 msgr2=0x7f6444144270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:12.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.178+0000 7f64427fc700 1 --2- 172.21.15.36:0/2608688266 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64440601c0 0x7f6444144270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.178+0000 7f64427fc700 1 -- 172.21.15.36:0/2608688266 >> v2:172.21.15.70:3300/0 conn(0x7f6444147270 msgr2=0x7f644414ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:12.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.178+0000 7f64427fc700 1 --2- 172.21.15.36:0/2608688266 >> v2:172.21.15.70:3300/0 conn(0x7f6444147270 0x7f644414ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.178+0000 7f64427fc700 1 -- 172.21.15.36:0/2608688266 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64440605a0 con 0x7f64440b5be0 2022-06-17T12:57:12.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.178+0000 7f64427fc700 1 --2- 172.21.15.36:0/2608688266 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64440b5be0 0x7f6444144760 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0x7f6438002f70 tx=0x7f6438006eb0).ready entity=mon.0 client_cookie=e646aa94b9bb0ac4 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:12.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.178+0000 7f6441ffb700 1 -- 172.21.15.36:0/2608688266 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f643800d570 con 0x7f64440b5be0 2022-06-17T12:57:12.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.178+0000 7f6441ffb700 1 -- 172.21.15.36:0/2608688266 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6438000e90 con 0x7f64440b5be0 2022-06-17T12:57:12.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.178+0000 7f6441ffb700 1 -- 172.21.15.36:0/2608688266 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f643800d900 con 0x7f64440b5be0 2022-06-17T12:57:12.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.182+0000 7f6443fff700 1 -- 172.21.15.36:0/2608688266 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64440b5be0 msgr2=0x7f6444144760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:12.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.182+0000 7f6443fff700 1 --2- 172.21.15.36:0/2608688266 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64440b5be0 0x7f6444144760 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0x7f6438002f70 tx=0x7f6438006eb0).stop 2022-06-17T12:57:12.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.182+0000 7f6443fff700 1 -- 172.21.15.36:0/2608688266 shutdown_connections 2022-06-17T12:57:12.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.182+0000 7f6443fff700 1 --2- 172.21.15.36:0/2608688266 >> v2:172.21.15.70:3300/0 conn(0x7f6444147270 0x7f644414ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.182+0000 7f6443fff700 1 --2- 172.21.15.36:0/2608688266 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64440b5be0 0x7f6444144760 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.182+0000 7f6443fff700 1 --2- 172.21.15.36:0/2608688266 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64440601c0 0x7f6444144270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.182+0000 7f6443fff700 1 -- 172.21.15.36:0/2608688266 >> 172.21.15.36:0/2608688266 conn(0x7f644405c550 msgr2=0x7f644405f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:12.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.182+0000 7f6443fff700 1 -- 172.21.15.36:0/2608688266 shutdown_connections 2022-06-17T12:57:12.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.182+0000 7f6443fff700 1 -- 172.21.15.36:0/2608688266 wait complete. 2022-06-17T12:57:12.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.182+0000 7f6443fff700 1 Processor -- start 2022-06-17T12:57:12.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.182+0000 7f6443fff700 1 -- start start 2022-06-17T12:57:12.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.182+0000 7f6443fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64440601c0 0x7f64440b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:12.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.182+0000 7f6443fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64440b5be0 0x7f6444146900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:12.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.182+0000 7f6443fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6444147270 0x7f64440af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:12.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.182+0000 7f6443fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6444146df0 con 0x7f64440601c0 2022-06-17T12:57:12.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.182+0000 7f6443fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6444144f10 con 0x7f64440b5be0 2022-06-17T12:57:12.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.182+0000 7f6443fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64441451b0 con 0x7f6444147270 2022-06-17T12:57:12.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.182+0000 7f6442ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64440601c0 0x7f64440b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:12.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.182+0000 7f6442ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64440601c0 0x7f64440b89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40574/0 (socket says 172.21.15.36:40574) 2022-06-17T12:57:12.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.182+0000 7f6442ffd700 1 -- 172.21.15.36:0/1010426668 learned_addr learned my addr 172.21.15.36:0/1010426668 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:12.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.182+0000 7f64427fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64440b5be0 0x7f6444146900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:12.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.182+0000 7f64437fe700 1 --2- 172.21.15.36:0/1010426668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6444147270 0x7f64440af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:12.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.186+0000 7f6442ffd700 1 -- 172.21.15.36:0/1010426668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6444147270 msgr2=0x7f64440af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:12.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.186+0000 7f6442ffd700 1 --2- 172.21.15.36:0/1010426668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6444147270 0x7f64440af720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.186+0000 7f6442ffd700 1 -- 172.21.15.36:0/1010426668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64440b5be0 msgr2=0x7f6444146900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:12.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.186+0000 7f6442ffd700 1 --2- 172.21.15.36:0/1010426668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64440b5be0 0x7f6444146900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.186+0000 7f6442ffd700 1 -- 172.21.15.36:0/1010426668 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643800b1e0 con 0x7f64440601c0 2022-06-17T12:57:12.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.186+0000 7f64427fc700 1 --2- 172.21.15.36:0/1010426668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64440b5be0 0x7f6444146900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:57:12.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.186+0000 7f64437fe700 1 --2- 172.21.15.36:0/1010426668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6444147270 0x7f64440af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:57:12.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.186+0000 7f6442ffd700 1 --2- 172.21.15.36:0/1010426668 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64440601c0 0x7f64440b89e0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7f643400c000 tx=0x7f6434006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:12.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.186+0000 7f6433fff700 1 -- 172.21.15.36:0/1010426668 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6434010780 con 0x7f64440601c0 2022-06-17T12:57:12.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.186+0000 7f6433fff700 1 -- 172.21.15.36:0/1010426668 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64340108e0 con 0x7f64440601c0 2022-06-17T12:57:12.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.186+0000 7f6443fff700 1 -- 172.21.15.36:0/1010426668 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6444145450 con 0x7f64440601c0 2022-06-17T12:57:12.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.186+0000 7f6433fff700 1 -- 172.21.15.36:0/1010426668 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6434010c40 con 0x7f64440601c0 2022-06-17T12:57:12.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.186+0000 7f6443fff700 1 -- 172.21.15.36:0/1010426668 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6444156570 con 0x7f64440601c0 2022-06-17T12:57:12.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.186+0000 7f6443fff700 1 -- 172.21.15.36:0/1010426668 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6444062bc0 con 0x7f64440601c0 2022-06-17T12:57:12.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.186+0000 7f6433fff700 1 -- 172.21.15.36:0/1010426668 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f643401c460 con 0x7f64440601c0 2022-06-17T12:57:12.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.190+0000 7f6433fff700 1 --2- 172.21.15.36:0/1010426668 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6420053ad0 0x7f6420055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:12.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.190+0000 7f6433fff700 1 -- 172.21.15.36:0/1010426668 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f643406c0f0 con 0x7f64440601c0 2022-06-17T12:57:12.278 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.190+0000 7f64427fc700 1 --2- 172.21.15.36:0/1010426668 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6420053ad0 0x7f6420055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:12.278 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.190+0000 7f64427fc700 1 --2- 172.21.15.36:0/1010426668 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6420053ad0 0x7f6420055f90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7f643801c000 tx=0x7f6438015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:12.279 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.194+0000 7f6433fff700 1 -- 172.21.15.36:0/1010426668 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6444062bc0 con 0x7f64440601c0 2022-06-17T12:57:12.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.318+0000 7f6443fff700 1 -- 172.21.15.36:0/1010426668 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6444062bc0 con 0x7f64440601c0 2022-06-17T12:57:12.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.318+0000 7f6433fff700 1 -- 172.21.15.36:0/1010426668 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v146) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6444062bc0 con 0x7f64440601c0 2022-06-17T12:57:12.405 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:57:12.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.322+0000 7f6443fff700 1 -- 172.21.15.36:0/1010426668 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6420053ad0 msgr2=0x7f6420055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:12.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.322+0000 7f6443fff700 1 --2- 172.21.15.36:0/1010426668 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6420053ad0 0x7f6420055f90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7f643801c000 tx=0x7f6438015000).stop 2022-06-17T12:57:12.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.322+0000 7f6443fff700 1 -- 172.21.15.36:0/1010426668 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64440601c0 msgr2=0x7f64440b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:12.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.322+0000 7f6443fff700 1 --2- 172.21.15.36:0/1010426668 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64440601c0 0x7f64440b89e0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7f643400c000 tx=0x7f6434006df0).stop 2022-06-17T12:57:12.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.322+0000 7f6443fff700 1 -- 172.21.15.36:0/1010426668 shutdown_connections 2022-06-17T12:57:12.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.322+0000 7f6443fff700 1 --2- 172.21.15.36:0/1010426668 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6420053ad0 0x7f6420055f90 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.322+0000 7f6443fff700 1 --2- 172.21.15.36:0/1010426668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6444147270 0x7f64440af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.322+0000 7f6443fff700 1 --2- 172.21.15.36:0/1010426668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64440b5be0 0x7f6444146900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.322+0000 7f6443fff700 1 --2- 172.21.15.36:0/1010426668 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f64440601c0 0x7f64440b89e0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.322+0000 7f6443fff700 1 -- 172.21.15.36:0/1010426668 >> 172.21.15.36:0/1010426668 conn(0x7f644405c550 msgr2=0x7f6444143ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:12.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.322+0000 7f6443fff700 1 -- 172.21.15.36:0/1010426668 shutdown_connections 2022-06-17T12:57:12.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.322+0000 7f6443fff700 1 -- 172.21.15.36:0/1010426668 wait complete. 2022-06-17T12:57:12.428 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:57:12.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.614+0000 7f2ce5108700 1 Processor -- start 2022-06-17T12:57:12.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.614+0000 7f2ce5108700 1 -- start start 2022-06-17T12:57:12.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.614+0000 7f2ce5108700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2ce0062270 0x7f2ce0143d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:12.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.614+0000 7f2ce5108700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce00b4190 0x7f2ce01471d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:12.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.614+0000 7f2ce5108700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce0143130 con 0x7f2ce00b48b0 2022-06-17T12:57:12.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.614+0000 7f2ce5108700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce00b6080 con 0x7f2ce0062270 2022-06-17T12:57:12.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.614+0000 7f2ce5108700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce0062650 con 0x7f2ce00b4190 2022-06-17T12:57:12.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.618+0000 7f2cde59c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce00b4190 0x7f2ce01471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:12.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.618+0000 7f2cded9d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2ce0062270 0x7f2ce0143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:12.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.618+0000 7f2cde59c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce00b4190 0x7f2ce01471d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:55376/0 (socket says 172.21.15.36:55376) 2022-06-17T12:57:12.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.618+0000 7f2cde59c700 1 -- 172.21.15.36:0/3345369816 learned_addr learned my addr 172.21.15.36:0/3345369816 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:12.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.618+0000 7f2cdf59e700 1 --1- 172.21.15.36:0/3345369816 >> v1:172.21.15.70:6789/0 conn(0x7f2ce00b48b0 0x7f2ce00b4d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:53848/0 (socket says 172.21.15.36:53848) 2022-06-17T12:57:12.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.618+0000 7f2cded9d700 1 -- 172.21.15.36:0/3345369816 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce00b4190 msgr2=0x7f2ce01471d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:12.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.618+0000 7f2cded9d700 1 --2- 172.21.15.36:0/3345369816 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce00b4190 0x7f2ce01471d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.618+0000 7f2cddd9b700 1 -- 172.21.15.36:0/3345369816 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 863558586 0 0) 0x7f2ce0143130 con 0x7f2ce00b48b0 2022-06-17T12:57:12.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.618+0000 7f2cded9d700 1 -- 172.21.15.36:0/3345369816 >> v1:172.21.15.70:6789/0 conn(0x7f2ce00b48b0 legacy=0x7f2ce00b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:12.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.618+0000 7f2cded9d700 1 -- 172.21.15.36:0/3345369816 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce0143360 con 0x7f2ce0062270 2022-06-17T12:57:12.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.618+0000 7f2cde59c700 1 --2- 172.21.15.36:0/3345369816 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce00b4190 0x7f2ce01471d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:57:12.636 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.618+0000 7f2cded9d700 1 --2- 172.21.15.36:0/3345369816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce0062270 0x7f2ce0143d40 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f2cd0002f70 tx=0x7f2cd0006df0).ready entity=mon.1 client_cookie=7ee4f53fd7b06ee8 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:12.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.618+0000 7f2cddd9b700 1 -- 172.21.15.36:0/3345369816 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cd000d570 con 0x7f2ce0062270 2022-06-17T12:57:12.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.618+0000 7f2cddd9b700 1 -- 172.21.15.36:0/3345369816 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd0000e90 con 0x7f2ce0062270 2022-06-17T12:57:12.637 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.618+0000 7f2cddd9b700 1 -- 172.21.15.36:0/3345369816 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cd000d8c0 con 0x7f2ce0062270 2022-06-17T12:57:12.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.618+0000 7f2ce5108700 1 -- 172.21.15.36:0/3345369816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce0062270 msgr2=0x7f2ce0143d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:12.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.618+0000 7f2ce5108700 1 --2- 172.21.15.36:0/3345369816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce0062270 0x7f2ce0143d40 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f2cd0002f70 tx=0x7f2cd0006df0).stop 2022-06-17T12:57:12.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.618+0000 7f2ce5108700 1 -- 172.21.15.36:0/3345369816 shutdown_connections 2022-06-17T12:57:12.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.618+0000 7f2ce5108700 1 --2- 172.21.15.36:0/3345369816 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce00b4190 0x7f2ce01471d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.638 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.618+0000 7f2ce5108700 1 --2- 172.21.15.36:0/3345369816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce0062270 0x7f2ce0143d40 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.618+0000 7f2ce5108700 1 -- 172.21.15.36:0/3345369816 >> 172.21.15.36:0/3345369816 conn(0x7f2ce00b97f0 msgr2=0x7f2ce00b9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:12.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.618+0000 7f2ce5108700 1 -- 172.21.15.36:0/3345369816 shutdown_connections 2022-06-17T12:57:12.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.618+0000 7f2ce5108700 1 -- 172.21.15.36:0/3345369816 wait complete. 2022-06-17T12:57:12.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.622+0000 7f2ce5108700 1 Processor -- start 2022-06-17T12:57:12.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.622+0000 7f2ce5108700 1 -- start start 2022-06-17T12:57:12.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.622+0000 7f2ce5108700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce0062270 0x7f2ce0149530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:12.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.622+0000 7f2ce5108700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce00b4190 0x7f2ce00b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:12.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.622+0000 7f2ce5108700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2ce00b48b0 0x7f2ce012e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:12.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.622+0000 7f2ce5108700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce00b1560 con 0x7f2ce00b48b0 2022-06-17T12:57:12.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.622+0000 7f2ce5108700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce012ddc0 con 0x7f2ce0062270 2022-06-17T12:57:12.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.622+0000 7f2ce5108700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce012e060 con 0x7f2ce00b4190 2022-06-17T12:57:12.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.622+0000 7f2cdf59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2ce00b48b0 0x7f2ce012e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:12.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.622+0000 7f2cde59c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce00b4190 0x7f2ce00b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:12.643 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.622+0000 7f2cdf59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2ce00b48b0 0x7f2ce012e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40592/0 (socket says 172.21.15.36:40592) 2022-06-17T12:57:12.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.622+0000 7f2cded9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce0062270 0x7f2ce0149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:12.644 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.622+0000 7f2cdf59e700 1 -- 172.21.15.36:0/3463794831 learned_addr learned my addr 172.21.15.36:0/3463794831 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:12.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.622+0000 7f2cde59c700 1 -- 172.21.15.36:0/3463794831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce0062270 msgr2=0x7f2ce0149530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:12.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.622+0000 7f2cde59c700 1 --2- 172.21.15.36:0/3463794831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce0062270 0x7f2ce0149530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.622+0000 7f2cde59c700 1 -- 172.21.15.36:0/3463794831 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2ce00b48b0 msgr2=0x7f2ce012e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:12.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.622+0000 7f2cde59c700 1 --2- 172.21.15.36:0/3463794831 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2ce00b48b0 0x7f2ce012e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.622+0000 7f2cde59c700 1 -- 172.21.15.36:0/3463794831 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd400b1e0 con 0x7f2ce00b4190 2022-06-17T12:57:12.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.622+0000 7f2cde59c700 1 --2- 172.21.15.36:0/3463794831 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce00b4190 0x7f2ce00b1070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f2cd400c000 tx=0x7f2cd4006eb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:12.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.622+0000 7f2ccffff700 1 -- 172.21.15.36:0/3463794831 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cd4012600 con 0x7f2ce00b4190 2022-06-17T12:57:12.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.626+0000 7f2ccffff700 1 -- 172.21.15.36:0/3463794831 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd4012760 con 0x7f2ce00b4190 2022-06-17T12:57:12.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.626+0000 7f2ce5108700 1 -- 172.21.15.36:0/3463794831 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd000b180 con 0x7f2ce00b4190 2022-06-17T12:57:12.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.626+0000 7f2ccffff700 1 -- 172.21.15.36:0/3463794831 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cd4012ac0 con 0x7f2ce00b4190 2022-06-17T12:57:12.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.626+0000 7f2ce5108700 1 -- 172.21.15.36:0/3463794831 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce00b1980 con 0x7f2ce00b4190 2022-06-17T12:57:12.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.626+0000 7f2ce5108700 1 -- 172.21.15.36:0/3463794831 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce00b1730 con 0x7f2ce00b4190 2022-06-17T12:57:12.681 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.626+0000 7f2ccffff700 1 -- 172.21.15.36:0/3463794831 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2cd4012c20 con 0x7f2ce00b4190 2022-06-17T12:57:12.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.626+0000 7f2ccffff700 1 --2- 172.21.15.36:0/3463794831 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2cbc053a40 0x7f2cbc055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:12.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.626+0000 7f2ccffff700 1 -- 172.21.15.36:0/3463794831 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f2cd406bd40 con 0x7f2ce00b4190 2022-06-17T12:57:12.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.626+0000 7f2cded9d700 1 --2- 172.21.15.36:0/3463794831 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2cbc053a40 0x7f2cbc055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:12.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.626+0000 7f2cded9d700 1 --2- 172.21.15.36:0/3463794831 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2cbc053a40 0x7f2cbc055f00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f2cd001a000 tx=0x7f2cd0014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:12.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.634+0000 7f2ccffff700 1 -- 172.21.15.36:0/3463794831 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2ce00b1730 con 0x7f2ce00b4190 2022-06-17T12:57:12.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.766+0000 7f2ce5108700 1 -- 172.21.15.36:0/3463794831 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2ce00b1730 con 0x7f2ce00b4190 2022-06-17T12:57:12.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.766+0000 7f2ccffff700 1 -- 172.21.15.36:0/3463794831 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v146) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2ce00b1730 con 0x7f2ce00b4190 2022-06-17T12:57:12.763 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:57:12.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.770+0000 7f2ce5108700 1 -- 172.21.15.36:0/3463794831 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2cbc053a40 msgr2=0x7f2cbc055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:12.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.770+0000 7f2ce5108700 1 --2- 172.21.15.36:0/3463794831 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2cbc053a40 0x7f2cbc055f00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f2cd001a000 tx=0x7f2cd0014000).stop 2022-06-17T12:57:12.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.770+0000 7f2ce5108700 1 -- 172.21.15.36:0/3463794831 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce00b4190 msgr2=0x7f2ce00b1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:12.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.770+0000 7f2ce5108700 1 --2- 172.21.15.36:0/3463794831 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce00b4190 0x7f2ce00b1070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f2cd400c000 tx=0x7f2cd4006eb0).stop 2022-06-17T12:57:12.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.770+0000 7f2ce5108700 1 -- 172.21.15.36:0/3463794831 shutdown_connections 2022-06-17T12:57:12.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.770+0000 7f2ce5108700 1 --2- 172.21.15.36:0/3463794831 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2cbc053a40 0x7f2cbc055f00 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.770+0000 7f2ce5108700 1 --2- 172.21.15.36:0/3463794831 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2ce00b48b0 0x7f2ce012e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.770+0000 7f2ce5108700 1 --2- 172.21.15.36:0/3463794831 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce00b4190 0x7f2ce00b1070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.770+0000 7f2ce5108700 1 --2- 172.21.15.36:0/3463794831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce0062270 0x7f2ce0149530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.770+0000 7f2ce5108700 1 -- 172.21.15.36:0/3463794831 >> 172.21.15.36:0/3463794831 conn(0x7f2ce00b97f0 msgr2=0x7f2ce00b0150 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:12.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.770+0000 7f2ce5108700 1 -- 172.21.15.36:0/3463794831 shutdown_connections 2022-06-17T12:57:12.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.770+0000 7f2ce5108700 1 -- 172.21.15.36:0/3463794831 wait complete. 2022-06-17T12:57:12.809 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-17T12:57:12.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.930+0000 7fa611183700 1 Processor -- start 2022-06-17T12:57:12.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.930+0000 7fa611183700 1 -- start start 2022-06-17T12:57:12.935 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.930+0000 7fa611183700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa60c0b7ce0 0x7fa60c0b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:12.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.930+0000 7fa611183700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa60c142160 con 0x7fa60c140c90 2022-06-17T12:57:12.938 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.930+0000 7fa611183700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa60c0b4f30 con 0x7fa60c142d70 2022-06-17T12:57:12.939 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.930+0000 7fa611183700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa60c142390 con 0x7fa60c0b7ce0 2022-06-17T12:57:12.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.930+0000 7fa60bfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa60c0b7ce0 0x7fa60c0b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:12.940 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.930+0000 7fa60b7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fa60c140c90 0x7fa60c13aa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58128/0 (socket says 172.21.15.36:58128) 2022-06-17T12:57:12.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.930+0000 7fa60b7fe700 1 -- 172.21.15.36:0/1097112423 learned_addr learned my addr 172.21.15.36:0/1097112423 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:12.947 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.934+0000 7fa60affd700 1 -- 172.21.15.36:0/1097112423 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1070376685 0 0) 0x7fa60c142160 con 0x7fa60c140c90 2022-06-17T12:57:12.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.934+0000 7fa60affd700 1 -- 172.21.15.36:0/1097112423 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5f8003260 con 0x7fa60c140c90 2022-06-17T12:57:12.952 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.934+0000 7fa60affd700 1 -- 172.21.15.36:0/1097112423 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2432057248 0 0) 0x7fa60c0b4f30 con 0x7fa60c142d70 2022-06-17T12:57:12.953 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.934+0000 7fa60affd700 1 -- 172.21.15.36:0/1097112423 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa60c142160 con 0x7fa60c142d70 2022-06-17T12:57:12.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.934+0000 7fa60bfff700 1 -- 172.21.15.36:0/1097112423 >> v1:172.21.15.70:6789/0 conn(0x7fa60c142d70 legacy=0x7fa60c143220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:12.954 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.934+0000 7fa60bfff700 1 -- 172.21.15.36:0/1097112423 >> v1:172.21.15.36:6789/0 conn(0x7fa60c140c90 legacy=0x7fa60c13aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:12.955 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.934+0000 7fa60bfff700 1 -- 172.21.15.36:0/1097112423 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60c142630 con 0x7fa60c0b7ce0 2022-06-17T12:57:12.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.934+0000 7fa60bfff700 1 --2- 172.21.15.36:0/1097112423 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa60c0b7ce0 0x7fa60c0b80c0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 rx=0x7fa600002f70 tx=0x7fa600006df0).ready entity=mon.0 client_cookie=4cb095fd87cd690a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:12.956 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.934+0000 7fa60affd700 1 -- 172.21.15.36:0/1097112423 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa60000d540 con 0x7fa60c0b7ce0 2022-06-17T12:57:12.959 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.934+0000 7fa60affd700 1 -- 172.21.15.36:0/1097112423 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa600000e90 con 0x7fa60c0b7ce0 2022-06-17T12:57:12.960 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.934+0000 7fa60affd700 1 -- 172.21.15.36:0/1097112423 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa60000d890 con 0x7fa60c0b7ce0 2022-06-17T12:57:12.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.934+0000 7fa611183700 1 -- 172.21.15.36:0/1097112423 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa60c0b7ce0 msgr2=0x7fa60c0b80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:12.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.934+0000 7fa611183700 1 --2- 172.21.15.36:0/1097112423 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa60c0b7ce0 0x7fa60c0b80c0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 rx=0x7fa600002f70 tx=0x7fa600006df0).stop 2022-06-17T12:57:12.961 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.934+0000 7fa611183700 1 -- 172.21.15.36:0/1097112423 shutdown_connections 2022-06-17T12:57:12.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.934+0000 7fa611183700 1 --2- 172.21.15.36:0/1097112423 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa60c0b7ce0 0x7fa60c0b80c0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.962 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.934+0000 7fa611183700 1 -- 172.21.15.36:0/1097112423 >> 172.21.15.36:0/1097112423 conn(0x7fa60c05c550 msgr2=0x7fa60c0b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:12.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.934+0000 7fa611183700 1 -- 172.21.15.36:0/1097112423 shutdown_connections 2022-06-17T12:57:12.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.934+0000 7fa611183700 1 -- 172.21.15.36:0/1097112423 wait complete. 2022-06-17T12:57:12.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.934+0000 7fa611183700 1 Processor -- start 2022-06-17T12:57:12.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.934+0000 7fa611183700 1 -- start start 2022-06-17T12:57:12.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.938+0000 7fa611183700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa60c0b7ce0 0x7fa60c1541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:12.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.938+0000 7fa611183700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa60c140c90 0x7fa60c0645a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:12.965 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.938+0000 7fa611183700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa60c142d70 0x7fa60c150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:12.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.938+0000 7fa611183700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c153a70 con 0x7fa60c0b7ce0 2022-06-17T12:57:12.966 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.938+0000 7fa611183700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c153d10 con 0x7fa60c142d70 2022-06-17T12:57:12.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.938+0000 7fa611183700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c13b8b0 con 0x7fa60c140c90 2022-06-17T12:57:12.967 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.938+0000 7fa60bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa60c0b7ce0 0x7fa60c1541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:12.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.938+0000 7fa60bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa60c0b7ce0 0x7fa60c1541b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40602/0 (socket says 172.21.15.36:40602) 2022-06-17T12:57:12.968 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.938+0000 7fa60bfff700 1 -- 172.21.15.36:0/186964162 learned_addr learned my addr 172.21.15.36:0/186964162 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:12.969 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.938+0000 7fa60b7fe700 1 --2- 172.21.15.36:0/186964162 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa60c140c90 0x7fa60c0645a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:12.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.938+0000 7fa610982700 1 --2- 172.21.15.36:0/186964162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa60c142d70 0x7fa60c150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:12.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.938+0000 7fa60bfff700 1 -- 172.21.15.36:0/186964162 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa60c140c90 msgr2=0x7fa60c0645a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:12.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.938+0000 7fa60bfff700 1 --2- 172.21.15.36:0/186964162 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa60c140c90 0x7fa60c0645a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.938+0000 7fa60bfff700 1 -- 172.21.15.36:0/186964162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa60c142d70 msgr2=0x7fa60c150320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:12.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.938+0000 7fa60bfff700 1 --2- 172.21.15.36:0/186964162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa60c142d70 0x7fa60c150320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:12.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.938+0000 7fa60bfff700 1 -- 172.21.15.36:0/186964162 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60000b180 con 0x7fa60c0b7ce0 2022-06-17T12:57:12.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.938+0000 7fa60bfff700 1 --2- 172.21.15.36:0/186964162 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa60c0b7ce0 0x7fa60c1541b0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7fa600002f70 tx=0x7fa600014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:12.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.938+0000 7fa6097fa700 1 -- 172.21.15.36:0/186964162 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa60000d1c0 con 0x7fa60c0b7ce0 2022-06-17T12:57:12.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.938+0000 7fa6097fa700 1 -- 172.21.15.36:0/186964162 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa600013d50 con 0x7fa60c0b7ce0 2022-06-17T12:57:12.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.938+0000 7fa611183700 1 -- 172.21.15.36:0/186964162 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa60c1546a0 con 0x7fa60c0b7ce0 2022-06-17T12:57:12.974 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.938+0000 7fa6097fa700 1 -- 172.21.15.36:0/186964162 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa60001e5f0 con 0x7fa60c0b7ce0 2022-06-17T12:57:12.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.938+0000 7fa611183700 1 -- 172.21.15.36:0/186964162 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa60c13e8c0 con 0x7fa60c0b7ce0 2022-06-17T12:57:12.975 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.938+0000 7fa611183700 1 -- 172.21.15.36:0/186964162 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa60c153ed0 con 0x7fa60c0b7ce0 2022-06-17T12:57:12.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.942+0000 7fa6097fa700 1 -- 172.21.15.36:0/186964162 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fa600025030 con 0x7fa60c0b7ce0 2022-06-17T12:57:12.976 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.942+0000 7fa6097fa700 1 --2- 172.21.15.36:0/186964162 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa5ec053a40 0x7fa5ec055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:12.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.942+0000 7fa6097fa700 1 -- 172.21.15.36:0/186964162 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fa60006cc20 con 0x7fa60c0b7ce0 2022-06-17T12:57:12.977 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.942+0000 7fa60b7fe700 1 --2- 172.21.15.36:0/186964162 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa5ec053a40 0x7fa5ec055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:12.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.942+0000 7fa60b7fe700 1 --2- 172.21.15.36:0/186964162 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa5ec053a40 0x7fa5ec055f00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7fa5fc00b000 tx=0x7fa5fc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:12.978 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:12.950+0000 7fa6097fa700 1 -- 172.21.15.36:0/186964162 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa600043c50 con 0x7fa60c0b7ce0 2022-06-17T12:57:13.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.078+0000 7fa611183700 1 -- 172.21.15.36:0/186964162 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa60c1428d0 con 0x7fa60c0b7ce0 2022-06-17T12:57:13.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.078+0000 7fa6097fa700 1 -- 172.21.15.36:0/186964162 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v146) v1 ==== 84+0+143 (secure 0 0 0) 0x7fa600047270 con 0x7fa60c0b7ce0 2022-06-17T12:57:13.072 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:57:13.072 INFO:teuthology.orchestra.run.smithi036.stdout:[ 2022-06-17T12:57:13.073 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:13.073 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 1, 2022-06-17T12:57:13.074 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "device_health_metrics" 2022-06-17T12:57:13.074 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:13.074 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:13.075 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 2, 2022-06-17T12:57:13.075 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "rbd" 2022-06-17T12:57:13.076 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:13.076 INFO:teuthology.orchestra.run.smithi036.stdout:] 2022-06-17T12:57:13.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.082+0000 7fa611183700 1 -- 172.21.15.36:0/186964162 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa5ec053a40 msgr2=0x7fa5ec055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:13.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.082+0000 7fa611183700 1 --2- 172.21.15.36:0/186964162 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa5ec053a40 0x7fa5ec055f00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7fa5fc00b000 tx=0x7fa5fc009040).stop 2022-06-17T12:57:13.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.082+0000 7fa611183700 1 -- 172.21.15.36:0/186964162 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa60c0b7ce0 msgr2=0x7fa60c1541b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:13.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.082+0000 7fa611183700 1 --2- 172.21.15.36:0/186964162 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa60c0b7ce0 0x7fa60c1541b0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7fa600002f70 tx=0x7fa600014000).stop 2022-06-17T12:57:13.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.082+0000 7fa611183700 1 -- 172.21.15.36:0/186964162 shutdown_connections 2022-06-17T12:57:13.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.082+0000 7fa611183700 1 --2- 172.21.15.36:0/186964162 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa5ec053a40 0x7fa5ec055f00 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:13.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.082+0000 7fa611183700 1 --2- 172.21.15.36:0/186964162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa60c142d70 0x7fa60c150320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:13.088 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.082+0000 7fa611183700 1 --2- 172.21.15.36:0/186964162 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa60c140c90 0x7fa60c0645a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:13.089 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.082+0000 7fa611183700 1 --2- 172.21.15.36:0/186964162 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa60c0b7ce0 0x7fa60c1541b0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:13.090 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.082+0000 7fa611183700 1 -- 172.21.15.36:0/186964162 >> 172.21.15.36:0/186964162 conn(0x7fa60c05c550 msgr2=0x7fa60c0b6c90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:13.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.082+0000 7fa611183700 1 -- 172.21.15.36:0/186964162 shutdown_connections 2022-06-17T12:57:13.092 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.082+0000 7fa611183700 1 -- 172.21.15.36:0/186964162 wait complete. 2022-06-17T12:57:13.092 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-17T12:57:13.093 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-17T12:57:13.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f5711e85700 1 Processor -- start 2022-06-17T12:57:13.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f5711e85700 1 -- start start 2022-06-17T12:57:13.247 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f5711e85700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f570c0b3300 0x7f570c147110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:13.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f5711e85700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f570c143b50 0x7f570c0b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:13.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f5711e85700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f570c1477f0 con 0x7f570c0b7910 2022-06-17T12:57:13.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f5711e85700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f570c062e90 con 0x7f570c0b3300 2022-06-17T12:57:13.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f5711e85700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f570c147600 con 0x7f570c143b50 2022-06-17T12:57:13.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f570b7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f570c0b3300 0x7f570c147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:13.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f570b7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f570c0b3300 0x7f570c147110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40610/0 (socket says 172.21.15.36:40610) 2022-06-17T12:57:13.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f570b7fe700 1 -- 172.21.15.36:0/1446157514 learned_addr learned my addr 172.21.15.36:0/1446157514 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:13.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f570bfff700 1 --2- 172.21.15.36:0/1446157514 >> v2:172.21.15.70:3300/0 conn(0x7f570c143b50 0x7f570c0b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:13.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f570b7fe700 1 -- 172.21.15.36:0/1446157514 >> v2:172.21.15.70:3300/0 conn(0x7f570c143b50 msgr2=0x7f570c0b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:13.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f570b7fe700 1 --2- 172.21.15.36:0/1446157514 >> v2:172.21.15.70:3300/0 conn(0x7f570c143b50 0x7f570c0b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:13.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f570b7fe700 1 -- 172.21.15.36:0/1446157514 >> v1:172.21.15.70:6789/0 conn(0x7f570c0b7910 legacy=0x7f570c0b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:13.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f570b7fe700 1 -- 172.21.15.36:0/1446157514 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570c0b4170 con 0x7f570c0b3300 2022-06-17T12:57:13.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f570bfff700 1 --2- 172.21.15.36:0/1446157514 >> v2:172.21.15.70:3300/0 conn(0x7f570c143b50 0x7f570c0b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:57:13.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f570b7fe700 1 --2- 172.21.15.36:0/1446157514 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f570c0b3300 0x7f570c147110 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0x7f56f4002f70 tx=0x7f56f4006eb0).ready entity=mon.0 client_cookie=2bdbe091b17a0ed8 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:13.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f570a7fc700 1 -- 172.21.15.36:0/1446157514 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56f400d610 con 0x7f570c0b3300 2022-06-17T12:57:13.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f570a7fc700 1 -- 172.21.15.36:0/1446157514 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f4000e90 con 0x7f570c0b3300 2022-06-17T12:57:13.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f570a7fc700 1 -- 172.21.15.36:0/1446157514 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56f400d9a0 con 0x7f570c0b3300 2022-06-17T12:57:13.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f5711e85700 1 -- 172.21.15.36:0/1446157514 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f570c0b3300 msgr2=0x7f570c147110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:13.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f5711e85700 1 --2- 172.21.15.36:0/1446157514 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f570c0b3300 0x7f570c147110 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0x7f56f4002f70 tx=0x7f56f4006eb0).stop 2022-06-17T12:57:13.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f5711e85700 1 -- 172.21.15.36:0/1446157514 shutdown_connections 2022-06-17T12:57:13.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f5711e85700 1 --2- 172.21.15.36:0/1446157514 >> v2:172.21.15.70:3300/0 conn(0x7f570c143b50 0x7f570c0b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:13.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f5711e85700 1 --2- 172.21.15.36:0/1446157514 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f570c0b3300 0x7f570c147110 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:13.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f5711e85700 1 -- 172.21.15.36:0/1446157514 >> 172.21.15.36:0/1446157514 conn(0x7f570c05c920 msgr2=0x7f570c060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:13.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f5711e85700 1 -- 172.21.15.36:0/1446157514 shutdown_connections 2022-06-17T12:57:13.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.222+0000 7f5711e85700 1 -- 172.21.15.36:0/1446157514 wait complete. 2022-06-17T12:57:13.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.226+0000 7f5711e85700 1 Processor -- start 2022-06-17T12:57:13.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.226+0000 7f5711e85700 1 -- start start 2022-06-17T12:57:13.265 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.230+0000 7f5711e85700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f570c0b3300 0x7f570c12d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:13.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.230+0000 7f5711e85700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f570c0b7910 0x7f570c0b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:13.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.230+0000 7f5711e85700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f570c143b50 0x7f570c12bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:13.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.230+0000 7f5711e85700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570c142150 con 0x7f570c0b7910 2022-06-17T12:57:13.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.230+0000 7f5711e85700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570c1423f0 con 0x7f570c0b3300 2022-06-17T12:57:13.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.230+0000 7f5711e85700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570c141570 con 0x7f570c143b50 2022-06-17T12:57:13.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.230+0000 7f570affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f570c0b7910 0x7f570c0b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:13.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.230+0000 7f570b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f570c0b3300 0x7f570c12d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:13.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.230+0000 7f570affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f570c0b7910 0x7f570c0b5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40618/0 (socket says 172.21.15.36:40618) 2022-06-17T12:57:13.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.230+0000 7f570affd700 1 -- 172.21.15.36:0/3291473237 learned_addr learned my addr 172.21.15.36:0/3291473237 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:13.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.230+0000 7f570bfff700 1 --2- 172.21.15.36:0/3291473237 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f570c143b50 0x7f570c12bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:13.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.230+0000 7f570affd700 1 -- 172.21.15.36:0/3291473237 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f570c143b50 msgr2=0x7f570c12bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:13.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.230+0000 7f570affd700 1 --2- 172.21.15.36:0/3291473237 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f570c143b50 0x7f570c12bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:13.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.230+0000 7f570affd700 1 -- 172.21.15.36:0/3291473237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f570c0b3300 msgr2=0x7f570c12d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:13.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.230+0000 7f570affd700 1 --2- 172.21.15.36:0/3291473237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f570c0b3300 0x7f570c12d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:13.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.230+0000 7f570affd700 1 -- 172.21.15.36:0/3291473237 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f400b1e0 con 0x7f570c0b7910 2022-06-17T12:57:13.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.230+0000 7f570affd700 1 --2- 172.21.15.36:0/3291473237 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f570c0b7910 0x7f570c0b5f80 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0x7f570c0b1740 tx=0x7f56fc007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:13.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.230+0000 7f5708ff9700 1 -- 172.21.15.36:0/3291473237 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56fc012400 con 0x7f570c0b7910 2022-06-17T12:57:13.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.230+0000 7f5708ff9700 1 -- 172.21.15.36:0/3291473237 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56fc012560 con 0x7f570c0b7910 2022-06-17T12:57:13.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.230+0000 7f5711e85700 1 -- 172.21.15.36:0/3291473237 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f570c142890 con 0x7f570c0b7910 2022-06-17T12:57:13.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.230+0000 7f5708ff9700 1 -- 172.21.15.36:0/3291473237 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56fc0128c0 con 0x7f570c0b7910 2022-06-17T12:57:13.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.230+0000 7f5711e85700 1 -- 172.21.15.36:0/3291473237 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f570c12c260 con 0x7f570c0b7910 2022-06-17T12:57:13.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.230+0000 7f5711e85700 1 -- 172.21.15.36:0/3291473237 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f570c057410 con 0x7f570c0b7910 2022-06-17T12:57:13.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.234+0000 7f5708ff9700 1 -- 172.21.15.36:0/3291473237 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f56fc019030 con 0x7f570c0b7910 2022-06-17T12:57:13.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.234+0000 7f5708ff9700 1 --2- 172.21.15.36:0/3291473237 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f56f0053a40 0x7f56f0055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:13.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.234+0000 7f5708ff9700 1 -- 172.21.15.36:0/3291473237 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f56fc06b0e0 con 0x7f570c0b7910 2022-06-17T12:57:13.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.234+0000 7f570b7fe700 1 --2- 172.21.15.36:0/3291473237 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f56f0053a40 0x7f56f0055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:13.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.234+0000 7f570b7fe700 1 --2- 172.21.15.36:0/3291473237 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f56f0053a40 0x7f56f0055f00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f56f401c000 tx=0x7f56f4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:13.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.238+0000 7f5708ff9700 1 -- 172.21.15.36:0/3291473237 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f56fc0420b0 con 0x7f570c0b7910 2022-06-17T12:57:13.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.362+0000 7f5711e85700 1 -- 172.21.15.36:0/3291473237 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f570c12c9f0 con 0x7f570c0b7910 2022-06-17T12:57:13.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.878+0000 7f5708ff9700 1 -- 172.21.15.36:0/3291473237 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v147) v1 ==== 133+0+0 (secure 0 0 0) 0x7f56fc0456d0 con 0x7f570c0b7910 2022-06-17T12:57:13.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.882+0000 7f5711e85700 1 -- 172.21.15.36:0/3291473237 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f56f0053a40 msgr2=0x7f56f0055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:13.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.882+0000 7f5711e85700 1 --2- 172.21.15.36:0/3291473237 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f56f0053a40 0x7f56f0055f00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f56f401c000 tx=0x7f56f4015000).stop 2022-06-17T12:57:13.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.882+0000 7f5711e85700 1 -- 172.21.15.36:0/3291473237 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f570c0b7910 msgr2=0x7f570c0b5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:13.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.882+0000 7f5711e85700 1 --2- 172.21.15.36:0/3291473237 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f570c0b7910 0x7f570c0b5f80 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0x7f570c0b1740 tx=0x7f56fc007c50).stop 2022-06-17T12:57:13.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.882+0000 7f5711e85700 1 -- 172.21.15.36:0/3291473237 shutdown_connections 2022-06-17T12:57:13.878 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.882+0000 7f5711e85700 1 --2- 172.21.15.36:0/3291473237 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f56f0053a40 0x7f56f0055f00 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:13.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.882+0000 7f5711e85700 1 --2- 172.21.15.36:0/3291473237 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f570c143b50 0x7f570c12bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:13.879 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.882+0000 7f5711e85700 1 --2- 172.21.15.36:0/3291473237 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f570c0b7910 0x7f570c0b5f80 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:13.880 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.882+0000 7f5711e85700 1 --2- 172.21.15.36:0/3291473237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f570c0b3300 0x7f570c12d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:13.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.882+0000 7f5711e85700 1 -- 172.21.15.36:0/3291473237 >> 172.21.15.36:0/3291473237 conn(0x7f570c05c920 msgr2=0x7f570c0614b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:13.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.882+0000 7f5711e85700 1 -- 172.21.15.36:0/3291473237 shutdown_connections 2022-06-17T12:57:13.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:13.882+0000 7f5711e85700 1 -- 172.21.15.36:0/3291473237 wait complete. 2022-06-17T12:57:13.882 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' created 2022-06-17T12:57:13.892 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-06-17T12:57:14.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.026+0000 7fb3b277b700 1 Processor -- start 2022-06-17T12:57:14.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.026+0000 7fb3b277b700 1 -- start start 2022-06-17T12:57:14.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.026+0000 7fb3b277b700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb3ac0b45c0 0x7fb3ac0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:14.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.026+0000 7fb3b277b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3ac0b4f00 0x7fb3ac0b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:14.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.026+0000 7fb3b277b700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3ac0b4160 con 0x7fb3ac0b9860 2022-06-17T12:57:14.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.026+0000 7fb3b277b700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3ac0b9c40 con 0x7fb3ac0b45c0 2022-06-17T12:57:14.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.026+0000 7fb3b277b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac0627d0 con 0x7fb3ac0b4f00 2022-06-17T12:57:14.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.026+0000 7fb3b1779700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fb3ac0b9860 0x7fb3ac143840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58154/0 (socket says 172.21.15.36:58154) 2022-06-17T12:57:14.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.026+0000 7fb3b1779700 1 -- 172.21.15.36:0/3448055942 learned_addr learned my addr 172.21.15.36:0/3448055942 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:14.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.026+0000 7fb3b0f78700 1 --2- 172.21.15.36:0/3448055942 >> v2:172.21.15.70:3300/0 conn(0x7fb3ac0b45c0 0x7fb3ac0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:14.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.026+0000 7fb3b1f7a700 1 --2- 172.21.15.36:0/3448055942 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3ac0b4f00 0x7fb3ac0b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:14.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.026+0000 7fb3a3fff700 1 -- 172.21.15.36:0/3448055942 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3066891461 0 0) 0x7fb3ac0b4160 con 0x7fb3ac0b9860 2022-06-17T12:57:14.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.026+0000 7fb3a3fff700 1 -- 172.21.15.36:0/3448055942 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb390003260 con 0x7fb3ac0b9860 2022-06-17T12:57:14.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.026+0000 7fb3b0f78700 1 -- 172.21.15.36:0/3448055942 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3ac0b4f00 msgr2=0x7fb3ac0b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:14.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.026+0000 7fb3b0f78700 1 --2- 172.21.15.36:0/3448055942 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3ac0b4f00 0x7fb3ac0b5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.026+0000 7fb3b0f78700 1 -- 172.21.15.36:0/3448055942 >> v1:172.21.15.36:6789/0 conn(0x7fb3ac0b9860 legacy=0x7fb3ac143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:14.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.026+0000 7fb3b0f78700 1 -- 172.21.15.36:0/3448055942 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ac062a70 con 0x7fb3ac0b45c0 2022-06-17T12:57:14.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.026+0000 7fb3b0f78700 1 --2- 172.21.15.36:0/3448055942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3ac0b45c0 0x7fb3ac0b4a10 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7fb39c002f70 tx=0x7fb39c006df0).ready entity=mon.1 client_cookie=d663e62917aaef40 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:14.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.030+0000 7fb3a3fff700 1 -- 172.21.15.36:0/3448055942 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb39c00d4a0 con 0x7fb3ac0b45c0 2022-06-17T12:57:14.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.030+0000 7fb3a3fff700 1 -- 172.21.15.36:0/3448055942 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39c000e90 con 0x7fb3ac0b45c0 2022-06-17T12:57:14.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.030+0000 7fb3a3fff700 1 -- 172.21.15.36:0/3448055942 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb39c00d7f0 con 0x7fb3ac0b45c0 2022-06-17T12:57:14.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.030+0000 7fb3b277b700 1 -- 172.21.15.36:0/3448055942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3ac0b45c0 msgr2=0x7fb3ac0b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:14.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.030+0000 7fb3b277b700 1 --2- 172.21.15.36:0/3448055942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3ac0b45c0 0x7fb3ac0b4a10 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7fb39c002f70 tx=0x7fb39c006df0).stop 2022-06-17T12:57:14.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.030+0000 7fb3b277b700 1 -- 172.21.15.36:0/3448055942 shutdown_connections 2022-06-17T12:57:14.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.030+0000 7fb3b277b700 1 --2- 172.21.15.36:0/3448055942 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3ac0b4f00 0x7fb3ac0b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.030+0000 7fb3b277b700 1 --2- 172.21.15.36:0/3448055942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3ac0b45c0 0x7fb3ac0b4a10 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.030+0000 7fb3b277b700 1 -- 172.21.15.36:0/3448055942 >> 172.21.15.36:0/3448055942 conn(0x7fb3ac05c550 msgr2=0x7fb3ac060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:14.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.030+0000 7fb3b277b700 1 -- 172.21.15.36:0/3448055942 shutdown_connections 2022-06-17T12:57:14.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.030+0000 7fb3b277b700 1 -- 172.21.15.36:0/3448055942 wait complete. 2022-06-17T12:57:14.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.034+0000 7fb3b277b700 1 Processor -- start 2022-06-17T12:57:14.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.034+0000 7fb3b277b700 1 -- start start 2022-06-17T12:57:14.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.034+0000 7fb3b277b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3ac0b45c0 0x7fb3ac13de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:14.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.034+0000 7fb3b277b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3ac0b4f00 0x7fb3ac0ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:14.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.034+0000 7fb3b277b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3ac0b9860 0x7fb3ac0ad800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:14.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.034+0000 7fb3b277b700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac0aece0 con 0x7fb3ac0b9860 2022-06-17T12:57:14.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.034+0000 7fb3b277b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac0aef20 con 0x7fb3ac0b4f00 2022-06-17T12:57:14.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.034+0000 7fb3b277b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac0af1c0 con 0x7fb3ac0b45c0 2022-06-17T12:57:14.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.034+0000 7fb3b0f78700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3ac0b4f00 0x7fb3ac0ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:14.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.034+0000 7fb3b1779700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3ac0b45c0 0x7fb3ac13de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:14.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.034+0000 7fb3b1f7a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3ac0b9860 0x7fb3ac0ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:14.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.034+0000 7fb3b0f78700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3ac0b4f00 0x7fb3ac0ae7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:32782/0 (socket says 172.21.15.36:32782) 2022-06-17T12:57:14.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.034+0000 7fb3b0f78700 1 -- 172.21.15.36:0/1577526116 learned_addr learned my addr 172.21.15.36:0/1577526116 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:14.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.034+0000 7fb3b0f78700 1 -- 172.21.15.36:0/1577526116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3ac0b45c0 msgr2=0x7fb3ac13de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:14.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.034+0000 7fb3b0f78700 1 --2- 172.21.15.36:0/1577526116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3ac0b45c0 0x7fb3ac13de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.034+0000 7fb3b0f78700 1 -- 172.21.15.36:0/1577526116 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3ac0b9860 msgr2=0x7fb3ac0ad800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:14.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.034+0000 7fb3b0f78700 1 --2- 172.21.15.36:0/1577526116 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3ac0b9860 0x7fb3ac0ad800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.034+0000 7fb3b0f78700 1 -- 172.21.15.36:0/1577526116 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39c00b180 con 0x7fb3ac0b4f00 2022-06-17T12:57:14.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.034+0000 7fb3b1f7a700 1 --2- 172.21.15.36:0/1577526116 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3ac0b9860 0x7fb3ac0ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:57:14.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.034+0000 7fb3b0f78700 1 --2- 172.21.15.36:0/1577526116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3ac0b4f00 0x7fb3ac0ae7f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7fb39c002f70 tx=0x7fb39c014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:14.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.034+0000 7fb3a27fc700 1 -- 172.21.15.36:0/1577526116 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb39c0139c0 con 0x7fb3ac0b4f00 2022-06-17T12:57:14.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.034+0000 7fb3a27fc700 1 -- 172.21.15.36:0/1577526116 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39c013b20 con 0x7fb3ac0b4f00 2022-06-17T12:57:14.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.034+0000 7fb3b277b700 1 -- 172.21.15.36:0/1577526116 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3ac1462a0 con 0x7fb3ac0b4f00 2022-06-17T12:57:14.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.034+0000 7fb3a27fc700 1 -- 172.21.15.36:0/1577526116 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb39c013e80 con 0x7fb3ac0b4f00 2022-06-17T12:57:14.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.038+0000 7fb3b277b700 1 -- 172.21.15.36:0/1577526116 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3ac13ce60 con 0x7fb3ac0b4f00 2022-06-17T12:57:14.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.038+0000 7fb3b277b700 1 -- 172.21.15.36:0/1577526116 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3ac062eb0 con 0x7fb3ac0b4f00 2022-06-17T12:57:14.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.038+0000 7fb3a27fc700 1 -- 172.21.15.36:0/1577526116 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fb39c025030 con 0x7fb3ac0b4f00 2022-06-17T12:57:14.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.038+0000 7fb3a27fc700 1 --2- 172.21.15.36:0/1577526116 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb394053a60 0x7fb394055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:14.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.038+0000 7fb3a27fc700 1 -- 172.21.15.36:0/1577526116 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5954+0+0 (secure 0 0 0) 0x7fb39c06cc50 con 0x7fb3ac0b4f00 2022-06-17T12:57:14.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.038+0000 7fb3b1779700 1 --2- 172.21.15.36:0/1577526116 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb394053a60 0x7fb394055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:14.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.038+0000 7fb3b1779700 1 --2- 172.21.15.36:0/1577526116 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb394053a60 0x7fb394055f20 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7fb3a800d000 tx=0x7fb3a8009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:14.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.046+0000 7fb3a27fc700 1 -- 172.21.15.36:0/1577526116 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb39c043b10 con 0x7fb3ac0b4f00 2022-06-17T12:57:14.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.178+0000 7fb3b277b700 1 -- 172.21.15.36:0/1577526116 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fb3ac057410 con 0x7fb3ac0b4f00 2022-06-17T12:57:14.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.242+0000 7fb3a27fc700 1 -- 172.21.15.36:0/1577526116 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v148) v1 ==== 125+0+0 (secure 0 0 0) 0x7fb39c023070 con 0x7fb3ac0b4f00 2022-06-17T12:57:14.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.250+0000 7fb3b277b700 1 -- 172.21.15.36:0/1577526116 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb394053a60 msgr2=0x7fb394055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:14.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.250+0000 7fb3b277b700 1 --2- 172.21.15.36:0/1577526116 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb394053a60 0x7fb394055f20 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7fb3a800d000 tx=0x7fb3a8009040).stop 2022-06-17T12:57:14.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.250+0000 7fb3b277b700 1 -- 172.21.15.36:0/1577526116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3ac0b4f00 msgr2=0x7fb3ac0ae7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:14.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.250+0000 7fb3b277b700 1 --2- 172.21.15.36:0/1577526116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3ac0b4f00 0x7fb3ac0ae7f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7fb39c002f70 tx=0x7fb39c014000).stop 2022-06-17T12:57:14.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.250+0000 7fb3b277b700 1 -- 172.21.15.36:0/1577526116 shutdown_connections 2022-06-17T12:57:14.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.250+0000 7fb3b277b700 1 --2- 172.21.15.36:0/1577526116 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fb394053a60 0x7fb394055f20 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.250+0000 7fb3b277b700 1 --2- 172.21.15.36:0/1577526116 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fb3ac0b9860 0x7fb3ac0ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.250+0000 7fb3b277b700 1 --2- 172.21.15.36:0/1577526116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3ac0b4f00 0x7fb3ac0ae7f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.250+0000 7fb3b277b700 1 --2- 172.21.15.36:0/1577526116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3ac0b45c0 0x7fb3ac13de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.250+0000 7fb3b277b700 1 -- 172.21.15.36:0/1577526116 >> 172.21.15.36:0/1577526116 conn(0x7fb3ac05c550 msgr2=0x7fb3ac0b26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:14.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.250+0000 7fb3b277b700 1 -- 172.21.15.36:0/1577526116 shutdown_connections 2022-06-17T12:57:14.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.250+0000 7fb3b277b700 1 -- 172.21.15.36:0/1577526116 wait complete. 2022-06-17T12:57:14.290 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' created 2022-06-17T12:57:14.292 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-17T12:57:14.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.422+0000 7ff319276700 1 Processor -- start 2022-06-17T12:57:14.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.422+0000 7ff319276700 1 -- start start 2022-06-17T12:57:14.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.422+0000 7ff319276700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3140ae970 0x7ff314142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:14.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.422+0000 7ff319276700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff314060ea0 con 0x7ff314060ac0 2022-06-17T12:57:14.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.422+0000 7ff319276700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3140b36a0 con 0x7ff314067290 2022-06-17T12:57:14.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.422+0000 7ff319276700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff314143060 con 0x7ff3140ae970 2022-06-17T12:57:14.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.422+0000 7ff312ffd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7ff314060ac0 0x7ff3140ad640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58168/0 (socket says 172.21.15.36:58168) 2022-06-17T12:57:14.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.422+0000 7ff312ffd700 1 -- 172.21.15.36:0/3640716726 learned_addr learned my addr 172.21.15.36:0/3640716726 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:14.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.422+0000 7ff3137fe700 1 --2- 172.21.15.36:0/3640716726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3140ae970 0x7ff314142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:14.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.422+0000 7ff311ffb700 1 -- 172.21.15.36:0/3640716726 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 400767000 0 0) 0x7ff314060ea0 con 0x7ff314060ac0 2022-06-17T12:57:14.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.422+0000 7ff311ffb700 1 -- 172.21.15.36:0/3640716726 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2f8003260 con 0x7ff314060ac0 2022-06-17T12:57:14.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.422+0000 7ff311ffb700 1 -- 172.21.15.36:0/3640716726 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3272969274 0 0) 0x7ff2f8003260 con 0x7ff314060ac0 2022-06-17T12:57:14.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.422+0000 7ff311ffb700 1 -- 172.21.15.36:0/3640716726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3140ae970 msgr2=0x7ff314142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:14.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.422+0000 7ff311ffb700 1 --2- 172.21.15.36:0/3640716726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3140ae970 0x7ff314142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.422+0000 7ff311ffb700 1 -- 172.21.15.36:0/3640716726 >> v1:172.21.15.70:6789/0 conn(0x7ff314067290 legacy=0x7ff314141f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:14.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.422+0000 7ff311ffb700 1 -- 172.21.15.36:0/3640716726 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3140b1b60 con 0x7ff314060ac0 2022-06-17T12:57:14.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff311ffb700 1 -- 172.21.15.36:0/3640716726 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7ff3040019e0 con 0x7ff314060ac0 2022-06-17T12:57:14.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff311ffb700 1 -- 172.21.15.36:0/3640716726 >> v1:172.21.15.36:6789/0 conn(0x7ff314060ac0 legacy=0x7ff3140ad640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:14.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff311ffb700 1 --2- 172.21.15.36:0/3640716726 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff2f8003780 0x7ff2f8003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:14.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff311ffb700 1 --2- 172.21.15.36:0/3640716726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3140ae970 0x7ff2f8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:14.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff311ffb700 1 --2- 172.21.15.36:0/3640716726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f80049f0 0x7ff2f8004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:14.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff311ffb700 1 -- 172.21.15.36:0/3640716726 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff314143060 con 0x7ff2f8003780 2022-06-17T12:57:14.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff311ffb700 1 -- 172.21.15.36:0/3640716726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f800b7b0 con 0x7ff2f80049f0 2022-06-17T12:57:14.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff311ffb700 1 -- 172.21.15.36:0/3640716726 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f800b970 con 0x7ff3140ae970 2022-06-17T12:57:14.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff312ffd700 1 --2- 172.21.15.36:0/3640716726 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff2f8003780 0x7ff2f8003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:14.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff3127fc700 1 --2- 172.21.15.36:0/3640716726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3140ae970 0x7ff2f8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:14.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff3137fe700 1 --2- 172.21.15.36:0/3640716726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f80049f0 0x7ff2f8004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:14.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff312ffd700 1 -- 172.21.15.36:0/3640716726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3140ae970 msgr2=0x7ff2f8004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:14.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff312ffd700 1 --2- 172.21.15.36:0/3640716726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff3140ae970 0x7ff2f8004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff312ffd700 1 -- 172.21.15.36:0/3640716726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f80049f0 msgr2=0x7ff2f8004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:14.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff312ffd700 1 --2- 172.21.15.36:0/3640716726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f80049f0 0x7ff2f8004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff312ffd700 1 -- 172.21.15.36:0/3640716726 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f8003260 con 0x7ff2f8003780 2022-06-17T12:57:14.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff3137fe700 1 --2- 172.21.15.36:0/3640716726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f80049f0 0x7ff2f8004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:57:14.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff312ffd700 1 --2- 172.21.15.36:0/3640716726 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff2f8003780 0x7ff2f8003ba0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7ff3040016f0 tx=0x7ff3140b3830).ready entity=mon.0 client_cookie=4615d831eff362f1 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:14.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff311ffb700 1 -- 172.21.15.36:0/3640716726 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff30400cc70 con 0x7ff2f8003780 2022-06-17T12:57:14.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff311ffb700 1 -- 172.21.15.36:0/3640716726 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30400cdd0 con 0x7ff2f8003780 2022-06-17T12:57:14.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff311ffb700 1 -- 172.21.15.36:0/3640716726 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff304005b80 con 0x7ff2f8003780 2022-06-17T12:57:14.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff319276700 1 -- 172.21.15.36:0/3640716726 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff2f8003780 msgr2=0x7ff2f8003ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:14.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff319276700 1 --2- 172.21.15.36:0/3640716726 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff2f8003780 0x7ff2f8003ba0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7ff3040016f0 tx=0x7ff3140b3830).stop 2022-06-17T12:57:14.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff3137fe700 1 -- 172.21.15.36:0/3640716726 reap_dead start 2022-06-17T12:57:14.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff319276700 1 -- 172.21.15.36:0/3640716726 shutdown_connections 2022-06-17T12:57:14.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff319276700 1 -- 172.21.15.36:0/3640716726 >> 172.21.15.36:0/3640716726 conn(0x7ff3140b6870 msgr2=0x7ff3140b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:14.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff319276700 1 -- 172.21.15.36:0/3640716726 shutdown_connections 2022-06-17T12:57:14.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff319276700 1 -- 172.21.15.36:0/3640716726 wait complete. 2022-06-17T12:57:14.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff319276700 1 Processor -- start 2022-06-17T12:57:14.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.426+0000 7ff319276700 1 -- start start 2022-06-17T12:57:14.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.430+0000 7ff319276700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f8003780 0x7ff31413fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:14.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.430+0000 7ff319276700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff314141050 0x7ff3140b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:14.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.430+0000 7ff319276700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff3140b4bd0 0x7ff314152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:14.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.430+0000 7ff319276700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3140a01c0 con 0x7ff3140b4bd0 2022-06-17T12:57:14.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.430+0000 7ff319276700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff314152600 con 0x7ff2f8003780 2022-06-17T12:57:14.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.430+0000 7ff319276700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3141528a0 con 0x7ff314141050 2022-06-17T12:57:14.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.430+0000 7ff3137fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff3140b4bd0 0x7ff314152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:14.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.430+0000 7ff3127fc700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff314141050 0x7ff3140b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:14.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.430+0000 7ff3137fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff3140b4bd0 0x7ff314152d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40654/0 (socket says 172.21.15.36:40654) 2022-06-17T12:57:14.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.430+0000 7ff3137fe700 1 -- 172.21.15.36:0/555910202 learned_addr learned my addr 172.21.15.36:0/555910202 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:14.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.430+0000 7ff312ffd700 1 --2- 172.21.15.36:0/555910202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f8003780 0x7ff31413fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:14.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.430+0000 7ff3137fe700 1 -- 172.21.15.36:0/555910202 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff314141050 msgr2=0x7ff3140b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:14.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.430+0000 7ff3137fe700 1 --2- 172.21.15.36:0/555910202 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff314141050 0x7ff3140b46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.430+0000 7ff3137fe700 1 -- 172.21.15.36:0/555910202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f8003780 msgr2=0x7ff31413fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:14.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.430+0000 7ff3137fe700 1 --2- 172.21.15.36:0/555910202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f8003780 0x7ff31413fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.430+0000 7ff3137fe700 1 -- 172.21.15.36:0/555910202 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31409fdf0 con 0x7ff3140b4bd0 2022-06-17T12:57:14.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.430+0000 7ff3137fe700 1 --2- 172.21.15.36:0/555910202 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff3140b4bd0 0x7ff314152d40 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7ff30c00b000 tx=0x7ff30c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:14.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.430+0000 7ff303fff700 1 -- 172.21.15.36:0/555910202 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff30c017070 con 0x7ff3140b4bd0 2022-06-17T12:57:14.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.430+0000 7ff303fff700 1 -- 172.21.15.36:0/555910202 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30c000e90 con 0x7ff3140b4bd0 2022-06-17T12:57:14.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.430+0000 7ff319276700 1 -- 172.21.15.36:0/555910202 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3141416b0 con 0x7ff3140b4bd0 2022-06-17T12:57:14.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.430+0000 7ff303fff700 1 -- 172.21.15.36:0/555910202 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff30c0135f0 con 0x7ff3140b4bd0 2022-06-17T12:57:14.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.430+0000 7ff319276700 1 -- 172.21.15.36:0/555910202 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3141400c0 con 0x7ff3140b4bd0 2022-06-17T12:57:14.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.430+0000 7ff319276700 1 -- 172.21.15.36:0/555910202 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff314152a60 con 0x7ff3140b4bd0 2022-06-17T12:57:14.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.434+0000 7ff303fff700 1 -- 172.21.15.36:0/555910202 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff30c013750 con 0x7ff3140b4bd0 2022-06-17T12:57:14.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.434+0000 7ff303fff700 1 --2- 172.21.15.36:0/555910202 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff2f00539b0 0x7ff2f0055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:14.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.434+0000 7ff303fff700 1 -- 172.21.15.36:0/555910202 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6304+0+0 (secure 0 0 0) 0x7ff30c06bba0 con 0x7ff3140b4bd0 2022-06-17T12:57:14.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.434+0000 7ff312ffd700 1 --2- 172.21.15.36:0/555910202 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff2f00539b0 0x7ff2f0055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:14.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.434+0000 7ff312ffd700 1 --2- 172.21.15.36:0/555910202 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff2f00539b0 0x7ff2f0055e70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7ff30401a000 tx=0x7ff304013000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:14.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.438+0000 7ff303fff700 1 -- 172.21.15.36:0/555910202 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff30c0428b0 con 0x7ff3140b4bd0 2022-06-17T12:57:14.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.602+0000 7ff319276700 1 -- 172.21.15.36:0/555910202 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7ff3140aedc0 con 0x7ff3140b4bd0 2022-06-17T12:57:14.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.614+0000 7ff303fff700 1 -- 172.21.15.36:0/555910202 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 29 and data pool 30 v82) v1 ==== 189+0+0 (secure 0 0 0) 0x7ff30c045ed0 con 0x7ff3140b4bd0 2022-06-17T12:57:14.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.618+0000 7ff319276700 1 -- 172.21.15.36:0/555910202 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff2f00539b0 msgr2=0x7ff2f0055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:14.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.618+0000 7ff319276700 1 --2- 172.21.15.36:0/555910202 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff2f00539b0 0x7ff2f0055e70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7ff30401a000 tx=0x7ff304013000).stop 2022-06-17T12:57:14.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.618+0000 7ff319276700 1 -- 172.21.15.36:0/555910202 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff3140b4bd0 msgr2=0x7ff314152d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:14.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.618+0000 7ff319276700 1 --2- 172.21.15.36:0/555910202 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff3140b4bd0 0x7ff314152d40 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7ff30c00b000 tx=0x7ff30c008040).stop 2022-06-17T12:57:14.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.618+0000 7ff319276700 1 -- 172.21.15.36:0/555910202 shutdown_connections 2022-06-17T12:57:14.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.618+0000 7ff319276700 1 --2- 172.21.15.36:0/555910202 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff2f00539b0 0x7ff2f0055e70 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.618+0000 7ff319276700 1 --2- 172.21.15.36:0/555910202 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff3140b4bd0 0x7ff314152d40 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.618+0000 7ff319276700 1 --2- 172.21.15.36:0/555910202 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff314141050 0x7ff3140b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.618+0000 7ff319276700 1 --2- 172.21.15.36:0/555910202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f8003780 0x7ff31413fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.618+0000 7ff319276700 1 -- 172.21.15.36:0/555910202 >> 172.21.15.36:0/555910202 conn(0x7ff3140b6870 msgr2=0x7ff314142610 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:14.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.618+0000 7ff319276700 1 -- 172.21.15.36:0/555910202 shutdown_connections 2022-06-17T12:57:14.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.618+0000 7ff319276700 1 -- 172.21.15.36:0/555910202 wait complete. 2022-06-17T12:57:14.619 INFO:teuthology.orchestra.run.smithi036.stderr:new fs with metadata pool 29 and data pool 30 2022-06-17T12:57:14.621 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:57:14.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.730+0000 7fa1dd3fa700 1 Processor -- start 2022-06-17T12:57:14.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.730+0000 7fa1dd3fa700 1 -- start start 2022-06-17T12:57:14.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.730+0000 7fa1dd3fa700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa1d80b7ce0 0x7fa1d80b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:14.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1dd3fa700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa1d8142d70 0x7fa1d81431e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:14.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1dd3fa700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1d8142160 con 0x7fa1d8140c90 2022-06-17T12:57:14.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1dd3fa700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1d80b4f30 con 0x7fa1d8142d70 2022-06-17T12:57:14.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1dd3fa700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d8142390 con 0x7fa1d80b7ce0 2022-06-17T12:57:14.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1d77fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fa1d8140c90 0x7fa1d813aa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58198/0 (socket says 172.21.15.36:58198) 2022-06-17T12:57:14.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1d77fe700 1 -- 172.21.15.36:0/831358406 learned_addr learned my addr 172.21.15.36:0/831358406 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:14.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1d7fff700 1 --2- 172.21.15.36:0/831358406 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa1d80b7ce0 0x7fa1d80b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:14.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1dcbf9700 1 --2- 172.21.15.36:0/831358406 >> v2:172.21.15.70:3300/0 conn(0x7fa1d8142d70 0x7fa1d81431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:14.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1d6ffd700 1 -- 172.21.15.36:0/831358406 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568808924 0 0) 0x7fa1d8142160 con 0x7fa1d8140c90 2022-06-17T12:57:14.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1d6ffd700 1 -- 172.21.15.36:0/831358406 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1c4003260 con 0x7fa1d8140c90 2022-06-17T12:57:14.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1d7fff700 1 -- 172.21.15.36:0/831358406 >> v2:172.21.15.70:3300/0 conn(0x7fa1d8142d70 msgr2=0x7fa1d81431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:14.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1d7fff700 1 --2- 172.21.15.36:0/831358406 >> v2:172.21.15.70:3300/0 conn(0x7fa1d8142d70 0x7fa1d81431e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.733 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1d7fff700 1 -- 172.21.15.36:0/831358406 >> v1:172.21.15.36:6789/0 conn(0x7fa1d8140c90 legacy=0x7fa1d813aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:14.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1d7fff700 1 -- 172.21.15.36:0/831358406 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d8142630 con 0x7fa1d80b7ce0 2022-06-17T12:57:14.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1d7fff700 1 --2- 172.21.15.36:0/831358406 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa1d80b7ce0 0x7fa1d80b80c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7fa1c8002f70 tx=0x7fa1c8006df0).ready entity=mon.2 client_cookie=a228b7173578652c server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:14.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1d6ffd700 1 -- 172.21.15.36:0/831358406 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1c800d4a0 con 0x7fa1d80b7ce0 2022-06-17T12:57:14.735 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1d6ffd700 1 -- 172.21.15.36:0/831358406 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c8000e90 con 0x7fa1d80b7ce0 2022-06-17T12:57:14.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1d6ffd700 1 -- 172.21.15.36:0/831358406 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1c800d7f0 con 0x7fa1d80b7ce0 2022-06-17T12:57:14.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1dd3fa700 1 -- 172.21.15.36:0/831358406 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa1d80b7ce0 msgr2=0x7fa1d80b80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:14.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1dd3fa700 1 --2- 172.21.15.36:0/831358406 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa1d80b7ce0 0x7fa1d80b80c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7fa1c8002f70 tx=0x7fa1c8006df0).stop 2022-06-17T12:57:14.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1dd3fa700 1 -- 172.21.15.36:0/831358406 shutdown_connections 2022-06-17T12:57:14.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1dd3fa700 1 --2- 172.21.15.36:0/831358406 >> v2:172.21.15.70:3300/0 conn(0x7fa1d8142d70 0x7fa1d81431e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1dd3fa700 1 --2- 172.21.15.36:0/831358406 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa1d80b7ce0 0x7fa1d80b80c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1dd3fa700 1 -- 172.21.15.36:0/831358406 >> 172.21.15.36:0/831358406 conn(0x7fa1d805c550 msgr2=0x7fa1d80b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:14.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1dd3fa700 1 -- 172.21.15.36:0/831358406 shutdown_connections 2022-06-17T12:57:14.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.734+0000 7fa1dd3fa700 1 -- 172.21.15.36:0/831358406 wait complete. 2022-06-17T12:57:14.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.738+0000 7fa1dd3fa700 1 Processor -- start 2022-06-17T12:57:14.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.738+0000 7fa1dd3fa700 1 -- start start 2022-06-17T12:57:14.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.738+0000 7fa1dd3fa700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1d80b7ce0 0x7fa1d814fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:14.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.738+0000 7fa1dd3fa700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa1d8140c90 0x7fa1d81455a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:14.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.738+0000 7fa1dd3fa700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa1d8142d70 0x7fa1d813b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:14.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.738+0000 7fa1dd3fa700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d81460d0 con 0x7fa1d8140c90 2022-06-17T12:57:14.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.738+0000 7fa1dd3fa700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d8145b90 con 0x7fa1d80b7ce0 2022-06-17T12:57:14.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.738+0000 7fa1dd3fa700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d8145e30 con 0x7fa1d8142d70 2022-06-17T12:57:14.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.738+0000 7fa1d77fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa1d8140c90 0x7fa1d81455a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:14.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.738+0000 7fa1d77fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa1d8140c90 0x7fa1d81455a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40672/0 (socket says 172.21.15.36:40672) 2022-06-17T12:57:14.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.738+0000 7fa1d77fe700 1 -- 172.21.15.36:0/1533235900 learned_addr learned my addr 172.21.15.36:0/1533235900 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:14.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.738+0000 7fa1dcbf9700 1 --2- 172.21.15.36:0/1533235900 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa1d8142d70 0x7fa1d813b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:14.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.738+0000 7fa1d7fff700 1 --2- 172.21.15.36:0/1533235900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1d80b7ce0 0x7fa1d814fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:14.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.738+0000 7fa1d77fe700 1 -- 172.21.15.36:0/1533235900 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa1d8142d70 msgr2=0x7fa1d813b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:14.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.738+0000 7fa1d77fe700 1 --2- 172.21.15.36:0/1533235900 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa1d8142d70 0x7fa1d813b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.738+0000 7fa1d77fe700 1 -- 172.21.15.36:0/1533235900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1d80b7ce0 msgr2=0x7fa1d814fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:14.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.738+0000 7fa1d77fe700 1 --2- 172.21.15.36:0/1533235900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1d80b7ce0 0x7fa1d814fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.738+0000 7fa1d77fe700 1 -- 172.21.15.36:0/1533235900 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c800b180 con 0x7fa1d8140c90 2022-06-17T12:57:14.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.738+0000 7fa1d77fe700 1 --2- 172.21.15.36:0/1533235900 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa1d8140c90 0x7fa1d81455a0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0x7fa1cc001110 tx=0x7fa1cc007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:14.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.738+0000 7fa1d57fa700 1 -- 172.21.15.36:0/1533235900 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1cc00ce80 con 0x7fa1d8140c90 2022-06-17T12:57:14.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.738+0000 7fa1d57fa700 1 -- 172.21.15.36:0/1533235900 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1cc0070d0 con 0x7fa1d8140c90 2022-06-17T12:57:14.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.738+0000 7fa1dd3fa700 1 -- 172.21.15.36:0/1533235900 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1d81443f0 con 0x7fa1d8140c90 2022-06-17T12:57:14.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.738+0000 7fa1d57fa700 1 -- 172.21.15.36:0/1533235900 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1cc007430 con 0x7fa1d8140c90 2022-06-17T12:57:14.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.738+0000 7fa1dd3fa700 1 -- 172.21.15.36:0/1533235900 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d813ffa0 con 0x7fa1d8140c90 2022-06-17T12:57:14.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.742+0000 7fa1d57fa700 1 -- 172.21.15.36:0/1533235900 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fa1cc010070 con 0x7fa1d8140c90 2022-06-17T12:57:14.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.742+0000 7fa1dd3fa700 1 -- 172.21.15.36:0/1533235900 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1d813cfc0 con 0x7fa1d8140c90 2022-06-17T12:57:14.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.742+0000 7fa1d57fa700 1 --2- 172.21.15.36:0/1533235900 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa1b8053a40 0x7fa1b8055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:14.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.742+0000 7fa1d7fff700 1 --2- 172.21.15.36:0/1533235900 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa1b8053a40 0x7fa1b8055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:14.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.742+0000 7fa1d57fa700 1 -- 172.21.15.36:0/1533235900 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa1cc06b500 con 0x7fa1d8140c90 2022-06-17T12:57:14.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.742+0000 7fa1d7fff700 1 --2- 172.21.15.36:0/1533235900 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa1b8053a40 0x7fa1b8055f00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7fa1c801a000 tx=0x7fa1c8014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:14.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.746+0000 7fa1d57fa700 1 -- 172.21.15.36:0/1533235900 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa1d813cfc0 con 0x7fa1d8140c90 2022-06-17T12:57:14.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.874+0000 7fa1dd3fa700 1 -- 172.21.15.36:0/1533235900 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1d813cfc0 con 0x7fa1d8140c90 2022-06-17T12:57:14.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.874+0000 7fa1d57fa700 1 -- 172.21.15.36:0/1533235900 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+16298 (secure 0 0 0) 0x7fa1d813cfc0 con 0x7fa1d8140c90 2022-06-17T12:57:14.864 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:57:14.864 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":149,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:57:14.606005+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:57:13.415000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-06-17T12:57:14.234187+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:57:14.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.878+0000 7fa1dd3fa700 1 -- 172.21.15.36:0/1533235900 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa1b8053a40 msgr2=0x7fa1b8055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:14.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.878+0000 7fa1dd3fa700 1 --2- 172.21.15.36:0/1533235900 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa1b8053a40 0x7fa1b8055f00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7fa1c801a000 tx=0x7fa1c8014000).stop 2022-06-17T12:57:14.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.878+0000 7fa1dd3fa700 1 -- 172.21.15.36:0/1533235900 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa1d8140c90 msgr2=0x7fa1d81455a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:14.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.878+0000 7fa1dd3fa700 1 --2- 172.21.15.36:0/1533235900 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa1d8140c90 0x7fa1d81455a0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0x7fa1cc001110 tx=0x7fa1cc007aa0).stop 2022-06-17T12:57:14.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.878+0000 7fa1dd3fa700 1 -- 172.21.15.36:0/1533235900 shutdown_connections 2022-06-17T12:57:14.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.878+0000 7fa1dd3fa700 1 --2- 172.21.15.36:0/1533235900 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa1b8053a40 0x7fa1b8055f00 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.878+0000 7fa1dd3fa700 1 --2- 172.21.15.36:0/1533235900 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa1d8142d70 0x7fa1d813b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.878+0000 7fa1dd3fa700 1 --2- 172.21.15.36:0/1533235900 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa1d8140c90 0x7fa1d81455a0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.878+0000 7fa1dd3fa700 1 --2- 172.21.15.36:0/1533235900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1d80b7ce0 0x7fa1d814fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:14.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.878+0000 7fa1dd3fa700 1 -- 172.21.15.36:0/1533235900 >> 172.21.15.36:0/1533235900 conn(0x7fa1d805c550 msgr2=0x7fa1d80af9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:14.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.878+0000 7fa1dd3fa700 1 -- 172.21.15.36:0/1533235900 shutdown_connections 2022-06-17T12:57:14.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:14.878+0000 7fa1dd3fa700 1 -- 172.21.15.36:0/1533235900 wait complete. 2022-06-17T12:57:14.884 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:57:15.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.030+0000 7f3303565700 1 Processor -- start 2022-06-17T12:57:15.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.030+0000 7f3303565700 1 -- start start 2022-06-17T12:57:15.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.030+0000 7f3303565700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32fc0afe30 0x7f32fc064230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:15.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.030+0000 7f3303565700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32fc0af280 con 0x7f32fc0aeea0 2022-06-17T12:57:15.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.030+0000 7f3303565700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32fc0b14f0 con 0x7f32fc0af930 2022-06-17T12:57:15.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.030+0000 7f3303565700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc067560 con 0x7f32fc0afe30 2022-06-17T12:57:15.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.030+0000 7f3302563700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f32fc0aeea0 0x7f32fc0b19e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58208/0 (socket says 172.21.15.36:58208) 2022-06-17T12:57:15.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.030+0000 7f3302563700 1 -- 172.21.15.36:0/4058063958 learned_addr learned my addr 172.21.15.36:0/4058063958 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:15.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3302d64700 1 --2- 172.21.15.36:0/4058063958 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32fc0afe30 0x7f32fc064230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:15.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301561700 1 -- 172.21.15.36:0/4058063958 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3244914194 0 0) 0x7f32fc0af280 con 0x7f32fc0aeea0 2022-06-17T12:57:15.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301561700 1 -- 172.21.15.36:0/4058063958 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32e4003260 con 0x7f32fc0aeea0 2022-06-17T12:57:15.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301561700 1 -- 172.21.15.36:0/4058063958 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3589849870 0 0) 0x7f32fc0b14f0 con 0x7f32fc0af930 2022-06-17T12:57:15.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301561700 1 -- 172.21.15.36:0/4058063958 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32fc0af280 con 0x7f32fc0af930 2022-06-17T12:57:15.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301561700 1 -- 172.21.15.36:0/4058063958 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3412614350 0 0) 0x7f32e4003260 con 0x7f32fc0aeea0 2022-06-17T12:57:15.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301561700 1 -- 172.21.15.36:0/4058063958 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32fc0afe30 msgr2=0x7f32fc064230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:15.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301561700 1 --2- 172.21.15.36:0/4058063958 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32fc0afe30 0x7f32fc064230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301561700 1 -- 172.21.15.36:0/4058063958 >> v1:172.21.15.70:6789/0 conn(0x7f32fc0af930 legacy=0x7f32fc0ab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:15.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301561700 1 -- 172.21.15.36:0/4058063958 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32fc0b20f0 con 0x7f32fc0aeea0 2022-06-17T12:57:15.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301561700 1 -- 172.21.15.36:0/4058063958 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f32f40019e0 con 0x7f32fc0aeea0 2022-06-17T12:57:15.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3302d64700 1 --2- 172.21.15.36:0/4058063958 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32fc0afe30 0x7f32fc064230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:57:15.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301561700 1 -- 172.21.15.36:0/4058063958 >> v1:172.21.15.36:6789/0 conn(0x7f32fc0aeea0 legacy=0x7f32fc0b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:15.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301561700 1 --2- 172.21.15.36:0/4058063958 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32e4003910 0x7f32e4003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:15.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301561700 1 --2- 172.21.15.36:0/4058063958 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f32e4004500 0x7f32e4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:15.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301561700 1 --2- 172.21.15.36:0/4058063958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e4004e70 0x7f32e400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:15.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301561700 1 -- 172.21.15.36:0/4058063958 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc067560 con 0x7f32e4004500 2022-06-17T12:57:15.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301561700 1 -- 172.21.15.36:0/4058063958 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e400bc40 con 0x7f32e4004e70 2022-06-17T12:57:15.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301561700 1 -- 172.21.15.36:0/4058063958 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e400be00 con 0x7f32e4003910 2022-06-17T12:57:15.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3302563700 1 --2- 172.21.15.36:0/4058063958 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32e4003910 0x7f32e4003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:15.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301d62700 1 --2- 172.21.15.36:0/4058063958 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f32e4004500 0x7f32e4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:15.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3302d64700 1 --2- 172.21.15.36:0/4058063958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e4004e70 0x7f32e400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:15.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301d62700 1 -- 172.21.15.36:0/4058063958 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32e4003910 msgr2=0x7f32e4003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:15.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301d62700 1 --2- 172.21.15.36:0/4058063958 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32e4003910 0x7f32e4003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301d62700 1 -- 172.21.15.36:0/4058063958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e4004e70 msgr2=0x7f32e400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:15.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301d62700 1 --2- 172.21.15.36:0/4058063958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e4004e70 0x7f32e400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301d62700 1 -- 172.21.15.36:0/4058063958 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e4003260 con 0x7f32e4004500 2022-06-17T12:57:15.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301d62700 1 --2- 172.21.15.36:0/4058063958 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f32e4004500 0x7f32e4004980 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 rx=0x7f32ec001110 tx=0x7f32ec007a00).ready entity=mon.0 client_cookie=dd93e4e63ae591e7 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:15.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301561700 1 -- 172.21.15.36:0/4058063958 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32ec007d10 con 0x7f32e4004500 2022-06-17T12:57:15.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301561700 1 -- 172.21.15.36:0/4058063958 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ec007030 con 0x7f32e4004500 2022-06-17T12:57:15.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3301561700 1 -- 172.21.15.36:0/4058063958 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32ec007390 con 0x7f32e4004500 2022-06-17T12:57:15.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3303565700 1 -- 172.21.15.36:0/4058063958 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f32e4004500 msgr2=0x7f32e4004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:15.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3303565700 1 --2- 172.21.15.36:0/4058063958 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f32e4004500 0x7f32e4004980 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 rx=0x7f32ec001110 tx=0x7f32ec007a00).stop 2022-06-17T12:57:15.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3302d64700 1 -- 172.21.15.36:0/4058063958 reap_dead start 2022-06-17T12:57:15.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3303565700 1 -- 172.21.15.36:0/4058063958 shutdown_connections 2022-06-17T12:57:15.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.034+0000 7f3303565700 1 -- 172.21.15.36:0/4058063958 >> 172.21.15.36:0/4058063958 conn(0x7f32fc05f770 msgr2=0x7f32fc05fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:15.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.038+0000 7f3303565700 1 -- 172.21.15.36:0/4058063958 shutdown_connections 2022-06-17T12:57:15.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.038+0000 7f3303565700 1 -- 172.21.15.36:0/4058063958 wait complete. 2022-06-17T12:57:15.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.038+0000 7f3303565700 1 Processor -- start 2022-06-17T12:57:15.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.038+0000 7f3303565700 1 -- start start 2022-06-17T12:57:15.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f3303565700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e4004500 0x7f32fc0ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:15.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f3303565700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32fc0adf90 0x7f32fc0b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:15.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f3303565700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f32fc0b3240 0x7f32fc140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:15.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f3303565700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc0add60 con 0x7f32fc0b3240 2022-06-17T12:57:15.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f3303565700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc0b4ee0 con 0x7f32e4004500 2022-06-17T12:57:15.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f3303565700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc0b5180 con 0x7f32fc0adf90 2022-06-17T12:57:15.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f3302d64700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f32fc0b3240 0x7f32fc140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:15.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f3302d64700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f32fc0b3240 0x7f32fc140230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40694/0 (socket says 172.21.15.36:40694) 2022-06-17T12:57:15.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f3302d64700 1 -- 172.21.15.36:0/345708884 learned_addr learned my addr 172.21.15.36:0/345708884 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:15.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f3302563700 1 --2- 172.21.15.36:0/345708884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e4004500 0x7f32fc0ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:15.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f3301d62700 1 --2- 172.21.15.36:0/345708884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32fc0adf90 0x7f32fc0b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:15.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f3302d64700 1 -- 172.21.15.36:0/345708884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32fc0adf90 msgr2=0x7f32fc0b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:15.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f3302d64700 1 --2- 172.21.15.36:0/345708884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32fc0adf90 0x7f32fc0b58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f3302d64700 1 -- 172.21.15.36:0/345708884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e4004500 msgr2=0x7f32fc0ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:15.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f3302d64700 1 --2- 172.21.15.36:0/345708884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e4004500 0x7f32fc0ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f3302d64700 1 -- 172.21.15.36:0/345708884 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32fc0b5420 con 0x7f32fc0b3240 2022-06-17T12:57:15.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f3302d64700 1 --2- 172.21.15.36:0/345708884 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f32fc0b3240 0x7f32fc140230 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7f32f800b000 tx=0x7f32f8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:15.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f32eb7fe700 1 -- 172.21.15.36:0/345708884 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32f8018070 con 0x7f32fc0b3240 2022-06-17T12:57:15.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f32eb7fe700 1 -- 172.21.15.36:0/345708884 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f8000e90 con 0x7f32fc0b3240 2022-06-17T12:57:15.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f3303565700 1 -- 172.21.15.36:0/345708884 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32fc1439e0 con 0x7f32fc0b3240 2022-06-17T12:57:15.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f32eb7fe700 1 -- 172.21.15.36:0/345708884 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32f8013940 con 0x7f32fc0b3240 2022-06-17T12:57:15.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f3303565700 1 -- 172.21.15.36:0/345708884 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32fc140720 con 0x7f32fc0b3240 2022-06-17T12:57:15.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f32eb7fe700 1 -- 172.21.15.36:0/345708884 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f32f8013aa0 con 0x7f32fc0b3240 2022-06-17T12:57:15.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f3303565700 1 -- 172.21.15.36:0/345708884 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32fc0b55b0 con 0x7f32fc0b3240 2022-06-17T12:57:15.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f32eb7fe700 1 --2- 172.21.15.36:0/345708884 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f32dc053940 0x7f32dc055e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:15.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f32eb7fe700 1 -- 172.21.15.36:0/345708884 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f32f806c320 con 0x7f32fc0b3240 2022-06-17T12:57:15.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.042+0000 7f3302563700 1 --2- 172.21.15.36:0/345708884 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f32dc053940 0x7f32dc055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:15.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.046+0000 7f3302563700 1 --2- 172.21.15.36:0/345708884 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f32dc053940 0x7f32dc055e00 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f32f4005000 tx=0x7f32f4002130).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:15.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.046+0000 7f32eb7fe700 1 -- 172.21.15.36:0/345708884 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f32f8043080 con 0x7f32fc0b3240 2022-06-17T12:57:15.170 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.178+0000 7f3303565700 1 -- 172.21.15.36:0/345708884 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32fc057410 con 0x7f32fc0b3240 2022-06-17T12:57:15.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.178+0000 7f32eb7fe700 1 -- 172.21.15.36:0/345708884 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f32f80466a0 con 0x7f32fc0b3240 2022-06-17T12:57:15.181 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:57:15.181 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":149,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:57:14.606005+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:57:13.415000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-06-17T12:57:14.234187+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:57:15.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.182+0000 7f3303565700 1 -- 172.21.15.36:0/345708884 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f32dc053940 msgr2=0x7f32dc055e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:15.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.182+0000 7f3303565700 1 --2- 172.21.15.36:0/345708884 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f32dc053940 0x7f32dc055e00 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f32f4005000 tx=0x7f32f4002130).stop 2022-06-17T12:57:15.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.182+0000 7f3303565700 1 -- 172.21.15.36:0/345708884 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f32fc0b3240 msgr2=0x7f32fc140230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:15.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.182+0000 7f3303565700 1 --2- 172.21.15.36:0/345708884 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f32fc0b3240 0x7f32fc140230 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7f32f800b000 tx=0x7f32f8006df0).stop 2022-06-17T12:57:15.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.182+0000 7f3303565700 1 -- 172.21.15.36:0/345708884 shutdown_connections 2022-06-17T12:57:15.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.182+0000 7f3303565700 1 --2- 172.21.15.36:0/345708884 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f32dc053940 0x7f32dc055e00 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.182+0000 7f3303565700 1 --2- 172.21.15.36:0/345708884 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f32fc0b3240 0x7f32fc140230 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.182+0000 7f3303565700 1 --2- 172.21.15.36:0/345708884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32fc0adf90 0x7f32fc0b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.182+0000 7f3303565700 1 --2- 172.21.15.36:0/345708884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e4004500 0x7f32fc0ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.182+0000 7f3303565700 1 -- 172.21.15.36:0/345708884 >> 172.21.15.36:0/345708884 conn(0x7f32fc05f770 msgr2=0x7f32fc0abea0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:15.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.182+0000 7f3303565700 1 -- 172.21.15.36:0/345708884 shutdown_connections 2022-06-17T12:57:15.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.182+0000 7f3303565700 1 -- 172.21.15.36:0/345708884 wait complete. 2022-06-17T12:57:15.188 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-17T12:57:15.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.318+0000 7f93a556f700 1 Processor -- start 2022-06-17T12:57:15.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.318+0000 7f93a556f700 1 -- start start 2022-06-17T12:57:15.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.318+0000 7f93a556f700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f93a00b27b0 0x7f93a00b1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:15.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.318+0000 7f93a556f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93a0144d20 0x7f93a0140230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:15.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.318+0000 7f93a556f700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a00ac0b0 con 0x7f93a0140720 2022-06-17T12:57:15.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.318+0000 7f93a556f700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a00b2b90 con 0x7f93a00b27b0 2022-06-17T12:57:15.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.318+0000 7f93a556f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a00afc60 con 0x7f93a0144d20 2022-06-17T12:57:15.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.318+0000 7f93a4d6e700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f93a0140720 0x7f93a0141180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58228/0 (socket says 172.21.15.36:58228) 2022-06-17T12:57:15.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.318+0000 7f93a4d6e700 1 -- 172.21.15.36:0/1162635919 learned_addr learned my addr 172.21.15.36:0/1162635919 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:15.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.318+0000 7f939f7fe700 1 --2- 172.21.15.36:0/1162635919 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93a0144d20 0x7f93a0140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:15.334 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.318+0000 7f939ffff700 1 --2- 172.21.15.36:0/1162635919 >> v2:172.21.15.70:3300/0 conn(0x7f93a00b27b0 0x7f93a00b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:15.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.318+0000 7f939effd700 1 -- 172.21.15.36:0/1162635919 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 348500676 0 0) 0x7f93a00ac0b0 con 0x7f93a0140720 2022-06-17T12:57:15.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.318+0000 7f939effd700 1 -- 172.21.15.36:0/1162635919 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f938c003260 con 0x7f93a0140720 2022-06-17T12:57:15.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.318+0000 7f939f7fe700 1 -- 172.21.15.36:0/1162635919 >> v2:172.21.15.70:3300/0 conn(0x7f93a00b27b0 msgr2=0x7f93a00b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:15.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.318+0000 7f939f7fe700 1 --2- 172.21.15.36:0/1162635919 >> v2:172.21.15.70:3300/0 conn(0x7f93a00b27b0 0x7f93a00b1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.318+0000 7f939f7fe700 1 -- 172.21.15.36:0/1162635919 >> v1:172.21.15.36:6789/0 conn(0x7f93a0140720 legacy=0x7f93a0141180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:15.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.318+0000 7f939f7fe700 1 -- 172.21.15.36:0/1162635919 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a00ac310 con 0x7f93a0144d20 2022-06-17T12:57:15.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.318+0000 7f939f7fe700 1 --2- 172.21.15.36:0/1162635919 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93a0144d20 0x7f93a0140230 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f9390002f70 tx=0x7f9390006df0).ready entity=mon.2 client_cookie=85e9a004426ca20f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:15.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.322+0000 7f939effd700 1 -- 172.21.15.36:0/1162635919 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f939000d4a0 con 0x7f93a0144d20 2022-06-17T12:57:15.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.322+0000 7f939effd700 1 -- 172.21.15.36:0/1162635919 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9390000e90 con 0x7f93a0144d20 2022-06-17T12:57:15.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.322+0000 7f939effd700 1 -- 172.21.15.36:0/1162635919 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f939000d7f0 con 0x7f93a0144d20 2022-06-17T12:57:15.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.322+0000 7f93a556f700 1 -- 172.21.15.36:0/1162635919 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93a0144d20 msgr2=0x7f93a0140230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:15.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.322+0000 7f93a556f700 1 --2- 172.21.15.36:0/1162635919 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93a0144d20 0x7f93a0140230 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f9390002f70 tx=0x7f9390006df0).stop 2022-06-17T12:57:15.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.322+0000 7f93a556f700 1 -- 172.21.15.36:0/1162635919 shutdown_connections 2022-06-17T12:57:15.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.322+0000 7f93a556f700 1 --2- 172.21.15.36:0/1162635919 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93a0144d20 0x7f93a0140230 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.322+0000 7f93a556f700 1 --2- 172.21.15.36:0/1162635919 >> v2:172.21.15.70:3300/0 conn(0x7f93a00b27b0 0x7f93a00b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.322+0000 7f93a556f700 1 -- 172.21.15.36:0/1162635919 >> 172.21.15.36:0/1162635919 conn(0x7f93a00b5d10 msgr2=0x7f93a00b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:15.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.322+0000 7f93a556f700 1 -- 172.21.15.36:0/1162635919 shutdown_connections 2022-06-17T12:57:15.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.322+0000 7f93a556f700 1 -- 172.21.15.36:0/1162635919 wait complete. 2022-06-17T12:57:15.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.322+0000 7f93a556f700 1 Processor -- start 2022-06-17T12:57:15.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.322+0000 7f93a556f700 1 -- start start 2022-06-17T12:57:15.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f93a556f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93a00b27b0 0x7f93a00af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:15.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f93a556f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93a0140720 0x7f93a013ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:15.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f93a556f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93a0144d20 0x7f93a0066d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:15.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f93a556f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a013eca0 con 0x7f93a00b27b0 2022-06-17T12:57:15.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f93a556f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a00664d0 con 0x7f93a0140720 2022-06-17T12:57:15.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f93a556f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a0066770 con 0x7f93a0144d20 2022-06-17T12:57:15.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f939ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93a00b27b0 0x7f93a00af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:15.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f939ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93a00b27b0 0x7f93a00af160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40700/0 (socket says 172.21.15.36:40700) 2022-06-17T12:57:15.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f939ffff700 1 -- 172.21.15.36:0/3945858883 learned_addr learned my addr 172.21.15.36:0/3945858883 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:15.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f939f7fe700 1 --2- 172.21.15.36:0/3945858883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93a0140720 0x7f93a013ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:15.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f93a4d6e700 1 --2- 172.21.15.36:0/3945858883 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93a0144d20 0x7f93a0066d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:15.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f939ffff700 1 -- 172.21.15.36:0/3945858883 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93a0144d20 msgr2=0x7f93a0066d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:15.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f939ffff700 1 --2- 172.21.15.36:0/3945858883 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93a0144d20 0x7f93a0066d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f939ffff700 1 -- 172.21.15.36:0/3945858883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93a0140720 msgr2=0x7f93a013ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:15.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f939ffff700 1 --2- 172.21.15.36:0/3945858883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93a0140720 0x7f93a013ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f939ffff700 1 -- 172.21.15.36:0/3945858883 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939000b180 con 0x7f93a00b27b0 2022-06-17T12:57:15.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f939f7fe700 1 --2- 172.21.15.36:0/3945858883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93a0140720 0x7f93a013ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:57:15.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f93a4d6e700 1 --2- 172.21.15.36:0/3945858883 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93a0144d20 0x7f93a0066d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:57:15.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f939ffff700 1 --2- 172.21.15.36:0/3945858883 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93a00b27b0 0x7f93a00af160 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7f939400c000 tx=0x7f9394006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:15.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f939d7fa700 1 -- 172.21.15.36:0/3945858883 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9394010780 con 0x7f93a00b27b0 2022-06-17T12:57:15.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f939d7fa700 1 -- 172.21.15.36:0/3945858883 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93940108e0 con 0x7f93a00b27b0 2022-06-17T12:57:15.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f93a556f700 1 -- 172.21.15.36:0/3945858883 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a00aecc0 con 0x7f93a00b27b0 2022-06-17T12:57:15.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f939d7fa700 1 -- 172.21.15.36:0/3945858883 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9394010c40 con 0x7f93a00b27b0 2022-06-17T12:57:15.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f93a556f700 1 -- 172.21.15.36:0/3945858883 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a013f760 con 0x7f93a00b27b0 2022-06-17T12:57:15.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f93a556f700 1 -- 172.21.15.36:0/3945858883 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a0145160 con 0x7f93a00b27b0 2022-06-17T12:57:15.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.326+0000 7f939d7fa700 1 -- 172.21.15.36:0/3945858883 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f939401c460 con 0x7f93a00b27b0 2022-06-17T12:57:15.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.330+0000 7f939d7fa700 1 --2- 172.21.15.36:0/3945858883 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9380053a60 0x7f9380055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:15.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.330+0000 7f939d7fa700 1 -- 172.21.15.36:0/3945858883 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f939406c510 con 0x7f93a00b27b0 2022-06-17T12:57:15.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.330+0000 7f939f7fe700 1 --2- 172.21.15.36:0/3945858883 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9380053a60 0x7f9380055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:15.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.330+0000 7f939f7fe700 1 --2- 172.21.15.36:0/3945858883 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9380053a60 0x7f9380055f20 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f9390018000 tx=0x7f9390014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:15.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.338+0000 7f939d7fa700 1 -- 172.21.15.36:0/3945858883 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f93940432b0 con 0x7f93a00b27b0 2022-06-17T12:57:15.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.502+0000 7f93a556f700 1 -- 172.21.15.36:0/3945858883 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f93a0057410 con 0x7f93a00b27b0 2022-06-17T12:57:15.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.614+0000 7f939d7fa700 1 -- 172.21.15.36:0/3945858883 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v83) v1 ==== 118+0+0 (secure 0 0 0) 0x7f93940468d0 con 0x7f93a00b27b0 2022-06-17T12:57:15.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.618+0000 7f93a556f700 1 -- 172.21.15.36:0/3945858883 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9380053a60 msgr2=0x7f9380055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:15.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.618+0000 7f93a556f700 1 --2- 172.21.15.36:0/3945858883 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9380053a60 0x7f9380055f20 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f9390018000 tx=0x7f9390014000).stop 2022-06-17T12:57:15.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.618+0000 7f93a556f700 1 -- 172.21.15.36:0/3945858883 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93a00b27b0 msgr2=0x7f93a00af160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:15.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.618+0000 7f93a556f700 1 --2- 172.21.15.36:0/3945858883 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93a00b27b0 0x7f93a00af160 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7f939400c000 tx=0x7f9394006df0).stop 2022-06-17T12:57:15.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.618+0000 7f93a556f700 1 -- 172.21.15.36:0/3945858883 shutdown_connections 2022-06-17T12:57:15.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.618+0000 7f93a556f700 1 --2- 172.21.15.36:0/3945858883 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f9380053a60 0x7f9380055f20 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.618+0000 7f93a556f700 1 --2- 172.21.15.36:0/3945858883 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93a0144d20 0x7f93a0066d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.618+0000 7f93a556f700 1 --2- 172.21.15.36:0/3945858883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93a0140720 0x7f93a013ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.618+0000 7f93a556f700 1 --2- 172.21.15.36:0/3945858883 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f93a00b27b0 0x7f93a00af160 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.618+0000 7f93a556f700 1 -- 172.21.15.36:0/3945858883 >> 172.21.15.36:0/3945858883 conn(0x7f93a00b5d10 msgr2=0x7f93a00b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:15.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.618+0000 7f93a556f700 1 -- 172.21.15.36:0/3945858883 shutdown_connections 2022-06-17T12:57:15.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.618+0000 7f93a556f700 1 -- 172.21.15.36:0/3945858883 wait complete. 2022-06-17T12:57:15.620 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:57:15.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.742+0000 7f5f4d730700 1 Processor -- start 2022-06-17T12:57:15.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.742+0000 7f5f4d730700 1 -- start start 2022-06-17T12:57:15.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.742+0000 7f5f4d730700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f48063730 0x7f5f48065e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:15.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.742+0000 7f5f4d730700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f48063b10 con 0x7f5f480b1930 2022-06-17T12:57:15.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.742+0000 7f5f4d730700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f48065430 con 0x7f5f480b2900 2022-06-17T12:57:15.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.742+0000 7f5f4d730700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f480656d0 con 0x7f5f48063730 2022-06-17T12:57:15.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.742+0000 7f5f3ffff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f5f480b1930 0x7f5f481431f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58238/0 (socket says 172.21.15.36:58238) 2022-06-17T12:57:15.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.742+0000 7f5f3ffff700 1 -- 172.21.15.36:0/2955009126 learned_addr learned my addr 172.21.15.36:0/2955009126 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:15.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f47fff700 1 --2- 172.21.15.36:0/2955009126 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f48063730 0x7f5f48065e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:15.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f477fe700 1 -- 172.21.15.36:0/2955009126 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 618796203 0 0) 0x7f5f48063b10 con 0x7f5f480b1930 2022-06-17T12:57:15.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f477fe700 1 -- 172.21.15.36:0/2955009126 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f28003260 con 0x7f5f480b1930 2022-06-17T12:57:15.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f477fe700 1 -- 172.21.15.36:0/2955009126 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1871621048 0 0) 0x7f5f48065430 con 0x7f5f480b2900 2022-06-17T12:57:15.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f477fe700 1 -- 172.21.15.36:0/2955009126 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f48063b10 con 0x7f5f480b2900 2022-06-17T12:57:15.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f477fe700 1 -- 172.21.15.36:0/2955009126 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 575372438 0 0) 0x7f5f28003260 con 0x7f5f480b1930 2022-06-17T12:57:15.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f477fe700 1 -- 172.21.15.36:0/2955009126 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f48063730 msgr2=0x7f5f48065e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:15.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f477fe700 1 --2- 172.21.15.36:0/2955009126 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f48063730 0x7f5f48065e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f477fe700 1 -- 172.21.15.36:0/2955009126 >> v1:172.21.15.70:6789/0 conn(0x7f5f480b2900 legacy=0x7f5f4813eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:15.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f477fe700 1 -- 172.21.15.36:0/2955009126 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f48065970 con 0x7f5f480b1930 2022-06-17T12:57:15.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f477fe700 1 -- 172.21.15.36:0/2955009126 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f5f30001660 con 0x7f5f480b1930 2022-06-17T12:57:15.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f477fe700 1 -- 172.21.15.36:0/2955009126 >> v1:172.21.15.36:6789/0 conn(0x7f5f480b1930 legacy=0x7f5f481431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:15.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f477fe700 1 --2- 172.21.15.36:0/2955009126 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f48063730 0x7f5f28003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:15.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f477fe700 1 --2- 172.21.15.36:0/2955009126 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5f28003e20 0x7f5f28004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:15.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f477fe700 1 --2- 172.21.15.36:0/2955009126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f280049f0 0x7f5f28004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:15.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f477fe700 1 -- 172.21.15.36:0/2955009126 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f480656d0 con 0x7f5f28003e20 2022-06-17T12:57:15.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f477fe700 1 -- 172.21.15.36:0/2955009126 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2800b800 con 0x7f5f280049f0 2022-06-17T12:57:15.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f477fe700 1 -- 172.21.15.36:0/2955009126 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2800b9c0 con 0x7f5f48063730 2022-06-17T12:57:15.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f3ffff700 1 --2- 172.21.15.36:0/2955009126 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5f28003e20 0x7f5f28004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:15.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f4cf2f700 1 --2- 172.21.15.36:0/2955009126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f280049f0 0x7f5f28004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:15.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f47fff700 1 --2- 172.21.15.36:0/2955009126 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f48063730 0x7f5f28003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:15.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f3ffff700 1 -- 172.21.15.36:0/2955009126 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f48063730 msgr2=0x7f5f28003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:15.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f3ffff700 1 --2- 172.21.15.36:0/2955009126 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f48063730 0x7f5f28003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f3ffff700 1 -- 172.21.15.36:0/2955009126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f280049f0 msgr2=0x7f5f28004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:15.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f3ffff700 1 --2- 172.21.15.36:0/2955009126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f280049f0 0x7f5f28004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f3ffff700 1 -- 172.21.15.36:0/2955009126 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f28003260 con 0x7f5f28003e20 2022-06-17T12:57:15.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f4cf2f700 1 --2- 172.21.15.36:0/2955009126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f280049f0 0x7f5f28004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:57:15.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f3ffff700 1 --2- 172.21.15.36:0/2955009126 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5f28003e20 0x7f5f28004500 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7f5f30001430 tx=0x7f5f48065b00).ready entity=mon.0 client_cookie=d5b2c590c0c9ba9b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:15.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f477fe700 1 -- 172.21.15.36:0/2955009126 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f300070d0 con 0x7f5f28003e20 2022-06-17T12:57:15.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f477fe700 1 -- 172.21.15.36:0/2955009126 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f30007230 con 0x7f5f28003e20 2022-06-17T12:57:15.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f477fe700 1 -- 172.21.15.36:0/2955009126 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f3000c8f0 con 0x7f5f28003e20 2022-06-17T12:57:15.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f4d730700 1 -- 172.21.15.36:0/2955009126 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5f28003e20 msgr2=0x7f5f28004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:15.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f4d730700 1 --2- 172.21.15.36:0/2955009126 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5f28003e20 0x7f5f28004500 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7f5f30001430 tx=0x7f5f48065b00).stop 2022-06-17T12:57:15.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f4cf2f700 1 -- 172.21.15.36:0/2955009126 reap_dead start 2022-06-17T12:57:15.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f4d730700 1 -- 172.21.15.36:0/2955009126 shutdown_connections 2022-06-17T12:57:15.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f4d730700 1 -- 172.21.15.36:0/2955009126 >> 172.21.15.36:0/2955009126 conn(0x7f5f480614b0 msgr2=0x7f5f480618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:15.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f4d730700 1 -- 172.21.15.36:0/2955009126 shutdown_connections 2022-06-17T12:57:15.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.746+0000 7f5f4d730700 1 -- 172.21.15.36:0/2955009126 wait complete. 2022-06-17T12:57:15.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.750+0000 7f5f4d730700 1 Processor -- start 2022-06-17T12:57:15.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.750+0000 7f5f4d730700 1 -- start start 2022-06-17T12:57:15.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f4d730700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5f28003e20 0x7f5f48110050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:15.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f4d730700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f48152150 0x7f5f480adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:15.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f4d730700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f480afda0 0x7f5f4814d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:15.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f4d730700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f481315f0 con 0x7f5f28003e20 2022-06-17T12:57:15.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f4d730700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f481514d0 con 0x7f5f48152150 2022-06-17T12:57:15.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f4d730700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f48151770 con 0x7f5f480afda0 2022-06-17T12:57:15.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f47fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5f28003e20 0x7f5f48110050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:15.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f47fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5f28003e20 0x7f5f48110050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40718/0 (socket says 172.21.15.36:40718) 2022-06-17T12:57:15.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f47fff700 1 -- 172.21.15.36:0/872766796 learned_addr learned my addr 172.21.15.36:0/872766796 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:15.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f4cf2f700 1 --2- 172.21.15.36:0/872766796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f480afda0 0x7f5f4814d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:15.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f3ffff700 1 --2- 172.21.15.36:0/872766796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f48152150 0x7f5f480adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:15.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f47fff700 1 -- 172.21.15.36:0/872766796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f480afda0 msgr2=0x7f5f4814d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:15.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f47fff700 1 --2- 172.21.15.36:0/872766796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f480afda0 0x7f5f4814d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f47fff700 1 -- 172.21.15.36:0/872766796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f48152150 msgr2=0x7f5f480adcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:15.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f47fff700 1 --2- 172.21.15.36:0/872766796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f48152150 0x7f5f480adcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f47fff700 1 -- 172.21.15.36:0/872766796 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f48151a10 con 0x7f5f28003e20 2022-06-17T12:57:15.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f3ffff700 1 --2- 172.21.15.36:0/872766796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f48152150 0x7f5f480adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:57:15.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f47fff700 1 --2- 172.21.15.36:0/872766796 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5f28003e20 0x7f5f48110050 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 rx=0x7f5f3800b000 tx=0x7f5f38008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:15.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f45ffb700 1 -- 172.21.15.36:0/872766796 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f38014680 con 0x7f5f28003e20 2022-06-17T12:57:15.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f45ffb700 1 -- 172.21.15.36:0/872766796 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f380147e0 con 0x7f5f28003e20 2022-06-17T12:57:15.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f4d730700 1 -- 172.21.15.36:0/872766796 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f48151cb0 con 0x7f5f28003e20 2022-06-17T12:57:15.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f45ffb700 1 -- 172.21.15.36:0/872766796 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f38014b40 con 0x7f5f28003e20 2022-06-17T12:57:15.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f4d730700 1 -- 172.21.15.36:0/872766796 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4814dad0 con 0x7f5f28003e20 2022-06-17T12:57:15.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f4d730700 1 -- 172.21.15.36:0/872766796 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f48151e90 con 0x7f5f28003e20 2022-06-17T12:57:15.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.754+0000 7f5f45ffb700 1 -- 172.21.15.36:0/872766796 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f5f38024070 con 0x7f5f28003e20 2022-06-17T12:57:15.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.758+0000 7f5f45ffb700 1 --2- 172.21.15.36:0/872766796 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5f2c053a60 0x7f5f2c055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:15.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.758+0000 7f5f45ffb700 1 -- 172.21.15.36:0/872766796 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5f3806c7d0 con 0x7f5f28003e20 2022-06-17T12:57:15.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.758+0000 7f5f3ffff700 1 --2- 172.21.15.36:0/872766796 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5f2c053a60 0x7f5f2c055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:15.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.758+0000 7f5f3ffff700 1 --2- 172.21.15.36:0/872766796 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5f2c053a60 0x7f5f2c055f20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f5f30015000 tx=0x7f5f30013000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:15.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.762+0000 7f5f45ffb700 1 -- 172.21.15.36:0/872766796 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5f48151e90 con 0x7f5f28003e20 2022-06-17T12:57:15.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.926+0000 7f5f4d730700 1 -- 172.21.15.36:0/872766796 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f48151e40 con 0x7f5f28003e20 2022-06-17T12:57:15.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.926+0000 7f5f45ffb700 1 -- 172.21.15.36:0/872766796 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f5f380434c0 con 0x7f5f28003e20 2022-06-17T12:57:15.911 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:57:15.911 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":83,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8599,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/4216085219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4216085219},{"type":"v1","addr":"172.21.15.70:6833","nonce":4216085219}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":82}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:57:14.606323+0000","modified":"2022-06-17T12:57:15.610154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8805":{"gid":8805,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/2534551587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2534551587},{"type":"v1","addr":"172.21.15.36:6835","nonce":2534551587}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-06-17T12:57:15.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.926+0000 7f5f4d730700 1 -- 172.21.15.36:0/872766796 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5f2c053a60 msgr2=0x7f5f2c055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:15.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.926+0000 7f5f4d730700 1 --2- 172.21.15.36:0/872766796 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5f2c053a60 0x7f5f2c055f20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f5f30015000 tx=0x7f5f30013000).stop 2022-06-17T12:57:15.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.926+0000 7f5f4d730700 1 -- 172.21.15.36:0/872766796 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5f28003e20 msgr2=0x7f5f48110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:15.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.926+0000 7f5f4d730700 1 --2- 172.21.15.36:0/872766796 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5f28003e20 0x7f5f48110050 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 rx=0x7f5f3800b000 tx=0x7f5f38008040).stop 2022-06-17T12:57:15.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.926+0000 7f5f4d730700 1 -- 172.21.15.36:0/872766796 shutdown_connections 2022-06-17T12:57:15.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.926+0000 7f5f4d730700 1 --2- 172.21.15.36:0/872766796 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5f2c053a60 0x7f5f2c055f20 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.926+0000 7f5f4d730700 1 --2- 172.21.15.36:0/872766796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f480afda0 0x7f5f4814d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.926+0000 7f5f4d730700 1 --2- 172.21.15.36:0/872766796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f48152150 0x7f5f480adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.926+0000 7f5f4d730700 1 --2- 172.21.15.36:0/872766796 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5f28003e20 0x7f5f48110050 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:15.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.926+0000 7f5f4d730700 1 -- 172.21.15.36:0/872766796 >> 172.21.15.36:0/872766796 conn(0x7f5f480614b0 msgr2=0x7f5f48140830 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:15.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.926+0000 7f5f4d730700 1 -- 172.21.15.36:0/872766796 shutdown_connections 2022-06-17T12:57:15.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:15.926+0000 7f5f4d730700 1 -- 172.21.15.36:0/872766796 wait complete. 2022-06-17T12:57:15.916 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 83 2022-06-17T12:57:15.924 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:57:16.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.042+0000 7f7d449b8700 1 Processor -- start 2022-06-17T12:57:16.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.042+0000 7f7d449b8700 1 -- start start 2022-06-17T12:57:16.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.042+0000 7f7d449b8700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d400b4820 0x7f7d400b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:16.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.042+0000 7f7d449b8700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7d40062470 0x7f7d40143a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:16.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.042+0000 7f7d449b8700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d400b1ff0 con 0x7f7d400b7ce0 2022-06-17T12:57:16.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.042+0000 7f7d449b8700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d40142ec0 con 0x7f7d40062470 2022-06-17T12:57:16.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.042+0000 7f7d449b8700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d401430b0 con 0x7f7d400b4820 2022-06-17T12:57:16.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.042+0000 7f7d3e59c700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7d40062470 0x7f7d40143a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:16.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.042+0000 7f7d3ed9d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d400b4820 0x7f7d400b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:16.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.042+0000 7f7d3f59e700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7d400b7ce0 0x7f7d40144390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:53992/0 (socket says 172.21.15.36:53992) 2022-06-17T12:57:16.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.042+0000 7f7d3e59c700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7d40062470 0x7f7d40143a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:32876/0 (socket says 172.21.15.36:32876) 2022-06-17T12:57:16.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.042+0000 7f7d3f59e700 1 -- 172.21.15.36:0/1333537827 learned_addr learned my addr 172.21.15.36:0/1333537827 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:16.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.042+0000 7f7d3e59c700 1 -- 172.21.15.36:0/1333537827 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d400b4820 msgr2=0x7f7d400b4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:16.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.042+0000 7f7d3e59c700 1 --2- 172.21.15.36:0/1333537827 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d400b4820 0x7f7d400b4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.042+0000 7f7d3e59c700 1 -- 172.21.15.36:0/1333537827 >> v1:172.21.15.70:6789/0 conn(0x7f7d400b7ce0 legacy=0x7f7d40144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:16.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.046+0000 7f7d3e59c700 1 -- 172.21.15.36:0/1333537827 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d40143350 con 0x7f7d40062470 2022-06-17T12:57:16.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.046+0000 7f7d3e59c700 1 --2- 172.21.15.36:0/1333537827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d40062470 0x7f7d40143a90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f7d30002f70 tx=0x7f7d30006df0).ready entity=mon.1 client_cookie=3a74df0233e77d7c server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:16.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.046+0000 7f7d3dd9b700 1 -- 172.21.15.36:0/1333537827 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d3000d570 con 0x7f7d40062470 2022-06-17T12:57:16.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.046+0000 7f7d3dd9b700 1 -- 172.21.15.36:0/1333537827 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d30000e90 con 0x7f7d40062470 2022-06-17T12:57:16.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.046+0000 7f7d3dd9b700 1 -- 172.21.15.36:0/1333537827 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d3000d8c0 con 0x7f7d40062470 2022-06-17T12:57:16.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.046+0000 7f7d449b8700 1 -- 172.21.15.36:0/1333537827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d40062470 msgr2=0x7f7d40143a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:16.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.046+0000 7f7d449b8700 1 --2- 172.21.15.36:0/1333537827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d40062470 0x7f7d40143a90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f7d30002f70 tx=0x7f7d30006df0).stop 2022-06-17T12:57:16.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.046+0000 7f7d449b8700 1 -- 172.21.15.36:0/1333537827 shutdown_connections 2022-06-17T12:57:16.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.046+0000 7f7d449b8700 1 --2- 172.21.15.36:0/1333537827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d40062470 0x7f7d40143a90 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.046+0000 7f7d449b8700 1 --2- 172.21.15.36:0/1333537827 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d400b4820 0x7f7d400b4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.046+0000 7f7d449b8700 1 -- 172.21.15.36:0/1333537827 >> 172.21.15.36:0/1333537827 conn(0x7f7d4005c550 msgr2=0x7f7d400b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:16.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.046+0000 7f7d449b8700 1 -- 172.21.15.36:0/1333537827 shutdown_connections 2022-06-17T12:57:16.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.046+0000 7f7d449b8700 1 -- 172.21.15.36:0/1333537827 wait complete. 2022-06-17T12:57:16.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.046+0000 7f7d449b8700 1 Processor -- start 2022-06-17T12:57:16.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.046+0000 7f7d449b8700 1 -- start start 2022-06-17T12:57:16.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d449b8700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7d40062470 0x7f7d400b2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:16.070 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d449b8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d400b4820 0x7f7d400afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:16.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d449b8700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d400b7ce0 0x7f7d400aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:16.071 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d449b8700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d40064f60 con 0x7f7d40062470 2022-06-17T12:57:16.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d449b8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d401523b0 con 0x7f7d400b4820 2022-06-17T12:57:16.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d449b8700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d40064850 con 0x7f7d400b7ce0 2022-06-17T12:57:16.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d3ed9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7d40062470 0x7f7d400b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:16.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d3e59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d400b4820 0x7f7d400afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:16.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d3f59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d400b7ce0 0x7f7d400aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:16.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d3e59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d400b4820 0x7f7d400afaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:32884/0 (socket says 172.21.15.36:32884) 2022-06-17T12:57:16.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d3e59c700 1 -- 172.21.15.36:0/253276236 learned_addr learned my addr 172.21.15.36:0/253276236 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:16.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d3ed9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7d40062470 0x7f7d400b2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40732/0 (socket says 172.21.15.36:40732) 2022-06-17T12:57:16.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d3e59c700 1 -- 172.21.15.36:0/253276236 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d400b7ce0 msgr2=0x7f7d400aff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:16.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d3e59c700 1 --2- 172.21.15.36:0/253276236 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d400b7ce0 0x7f7d400aff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d3e59c700 1 -- 172.21.15.36:0/253276236 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7d40062470 msgr2=0x7f7d400b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:16.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d3e59c700 1 --2- 172.21.15.36:0/253276236 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7d40062470 0x7f7d400b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d3e59c700 1 -- 172.21.15.36:0/253276236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3000b180 con 0x7f7d400b4820 2022-06-17T12:57:16.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d3f59e700 1 --2- 172.21.15.36:0/253276236 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d400b7ce0 0x7f7d400aff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:57:16.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d3ed9d700 1 --2- 172.21.15.36:0/253276236 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7d40062470 0x7f7d400b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:57:16.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d3e59c700 1 --2- 172.21.15.36:0/253276236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d400b4820 0x7f7d400afaa0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f7d400b70a0 tx=0x7f7d30014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:16.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d37fff700 1 -- 172.21.15.36:0/253276236 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d3000dea0 con 0x7f7d400b4820 2022-06-17T12:57:16.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d449b8700 1 -- 172.21.15.36:0/253276236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d4013ec60 con 0x7f7d400b4820 2022-06-17T12:57:16.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d37fff700 1 -- 172.21.15.36:0/253276236 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d30013e50 con 0x7f7d400b4820 2022-06-17T12:57:16.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d37fff700 1 -- 172.21.15.36:0/253276236 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d3001e5f0 con 0x7f7d400b4820 2022-06-17T12:57:16.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d449b8700 1 -- 172.21.15.36:0/253276236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d40142a20 con 0x7f7d400b4820 2022-06-17T12:57:16.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d449b8700 1 -- 172.21.15.36:0/253276236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d401435f0 con 0x7f7d400b4820 2022-06-17T12:57:16.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.050+0000 7f7d37fff700 1 -- 172.21.15.36:0/253276236 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f7d30025030 con 0x7f7d400b4820 2022-06-17T12:57:16.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.054+0000 7f7d37fff700 1 --2- 172.21.15.36:0/253276236 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7d24053a40 0x7f7d24055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:16.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.054+0000 7f7d37fff700 1 -- 172.21.15.36:0/253276236 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7d3006cfd0 con 0x7f7d400b4820 2022-06-17T12:57:16.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.054+0000 7f7d3ed9d700 1 --2- 172.21.15.36:0/253276236 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7d24053a40 0x7f7d24055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:16.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.054+0000 7f7d3ed9d700 1 --2- 172.21.15.36:0/253276236 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7d24053a40 0x7f7d24055f00 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f7d2800b000 tx=0x7f7d28006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:16.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.062+0000 7f7d37fff700 1 -- 172.21.15.36:0/253276236 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7d30043d00 con 0x7f7d400b4820 2022-06-17T12:57:16.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.186+0000 7f7d449b8700 1 -- 172.21.15.36:0/253276236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d40057410 con 0x7f7d400b4820 2022-06-17T12:57:16.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.190+0000 7f7d37fff700 1 -- 172.21.15.36:0/253276236 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f7d30047260 con 0x7f7d400b4820 2022-06-17T12:57:16.177 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:57:16.178 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":150,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:57:15.610840+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:57:13.415000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-06-17T12:57:14.234187+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:57:16.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.194+0000 7f7d449b8700 1 -- 172.21.15.36:0/253276236 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7d24053a40 msgr2=0x7f7d24055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:16.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.194+0000 7f7d449b8700 1 --2- 172.21.15.36:0/253276236 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7d24053a40 0x7f7d24055f00 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f7d2800b000 tx=0x7f7d28006df0).stop 2022-06-17T12:57:16.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.194+0000 7f7d449b8700 1 -- 172.21.15.36:0/253276236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d400b4820 msgr2=0x7f7d400afaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:16.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.194+0000 7f7d449b8700 1 --2- 172.21.15.36:0/253276236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d400b4820 0x7f7d400afaa0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f7d400b70a0 tx=0x7f7d30014000).stop 2022-06-17T12:57:16.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.194+0000 7f7d449b8700 1 -- 172.21.15.36:0/253276236 shutdown_connections 2022-06-17T12:57:16.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.194+0000 7f7d449b8700 1 --2- 172.21.15.36:0/253276236 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7d24053a40 0x7f7d24055f00 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.194+0000 7f7d449b8700 1 --2- 172.21.15.36:0/253276236 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d400b7ce0 0x7f7d400aff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.194+0000 7f7d449b8700 1 --2- 172.21.15.36:0/253276236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d400b4820 0x7f7d400afaa0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.194+0000 7f7d449b8700 1 --2- 172.21.15.36:0/253276236 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7d40062470 0x7f7d400b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.194+0000 7f7d449b8700 1 -- 172.21.15.36:0/253276236 >> 172.21.15.36:0/253276236 conn(0x7f7d4005c550 msgr2=0x7f7d400b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:16.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.194+0000 7f7d449b8700 1 -- 172.21.15.36:0/253276236 shutdown_connections 2022-06-17T12:57:16.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.194+0000 7f7d449b8700 1 -- 172.21.15.36:0/253276236 wait complete. 2022-06-17T12:57:16.196 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:57:16.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.346+0000 7f7bac703700 1 Processor -- start 2022-06-17T12:57:16.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.346+0000 7f7bac703700 1 -- start start 2022-06-17T12:57:16.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.346+0000 7f7bac703700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ba4062270 0x7f7ba4142f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:16.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.346+0000 7f7bac703700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7ba4143e80 0x7f7ba4146970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:16.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.346+0000 7f7bac703700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba40b44b0 con 0x7f7ba40b0cb0 2022-06-17T12:57:16.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.346+0000 7f7bac703700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba40b6650 con 0x7f7ba4143e80 2022-06-17T12:57:16.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.346+0000 7f7bac703700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba4144300 con 0x7f7ba4062270 2022-06-17T12:57:16.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.346+0000 7f7baaca0700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7ba4143e80 0x7f7ba4146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:16.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.346+0000 7f7baa49f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ba4062270 0x7f7ba4142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:16.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.346+0000 7f7ba9c9e700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7ba40b0cb0 0x7f7ba4143770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:54006/0 (socket says 172.21.15.36:54006) 2022-06-17T12:57:16.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.346+0000 7f7ba9c9e700 1 -- 172.21.15.36:0/573031429 learned_addr learned my addr 172.21.15.36:0/573031429 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:16.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.346+0000 7f7baaca0700 1 -- 172.21.15.36:0/573031429 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ba4062270 msgr2=0x7f7ba4142f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:16.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.346+0000 7f7baaca0700 1 --2- 172.21.15.36:0/573031429 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ba4062270 0x7f7ba4142f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.346+0000 7f7baaca0700 1 -- 172.21.15.36:0/573031429 >> v1:172.21.15.70:6789/0 conn(0x7f7ba40b0cb0 legacy=0x7f7ba4143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:16.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.346+0000 7f7baaca0700 1 -- 172.21.15.36:0/573031429 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba4062650 con 0x7f7ba4143e80 2022-06-17T12:57:16.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.346+0000 7f7baaca0700 1 --2- 172.21.15.36:0/573031429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ba4143e80 0x7f7ba4146970 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f7b9c0032f0 tx=0x7f7b9c0091d0).ready entity=mon.1 client_cookie=2fd6012d56775405 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:16.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.350+0000 7f7ba949d700 1 -- 172.21.15.36:0/573031429 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b9c00b6c0 con 0x7f7ba4143e80 2022-06-17T12:57:16.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.350+0000 7f7ba949d700 1 -- 172.21.15.36:0/573031429 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9c00b820 con 0x7f7ba4143e80 2022-06-17T12:57:16.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.350+0000 7f7ba949d700 1 -- 172.21.15.36:0/573031429 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b9c00b990 con 0x7f7ba4143e80 2022-06-17T12:57:16.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.350+0000 7f7bac703700 1 -- 172.21.15.36:0/573031429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ba4143e80 msgr2=0x7f7ba4146970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:16.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.350+0000 7f7bac703700 1 --2- 172.21.15.36:0/573031429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ba4143e80 0x7f7ba4146970 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f7b9c0032f0 tx=0x7f7b9c0091d0).stop 2022-06-17T12:57:16.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.350+0000 7f7bac703700 1 -- 172.21.15.36:0/573031429 shutdown_connections 2022-06-17T12:57:16.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.350+0000 7f7bac703700 1 --2- 172.21.15.36:0/573031429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ba4143e80 0x7f7ba4146970 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.350+0000 7f7bac703700 1 --2- 172.21.15.36:0/573031429 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ba4062270 0x7f7ba4142f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.350+0000 7f7bac703700 1 -- 172.21.15.36:0/573031429 >> 172.21.15.36:0/573031429 conn(0x7f7ba40b9870 msgr2=0x7f7ba40b9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:16.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.350+0000 7f7bac703700 1 -- 172.21.15.36:0/573031429 shutdown_connections 2022-06-17T12:57:16.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.350+0000 7f7bac703700 1 -- 172.21.15.36:0/573031429 wait complete. 2022-06-17T12:57:16.379 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.350+0000 7f7bac703700 1 Processor -- start 2022-06-17T12:57:16.386 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.350+0000 7f7bac703700 1 -- start start 2022-06-17T12:57:16.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.354+0000 7f7bac703700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ba4062270 0x7f7ba4152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:16.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.354+0000 7f7bac703700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ba4143e80 0x7f7ba405fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:16.387 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.354+0000 7f7bac703700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ba4147ad0 0x7f7ba4152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:16.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.354+0000 7f7bac703700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba41558c0 con 0x7f7ba4143e80 2022-06-17T12:57:16.388 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.354+0000 7f7bac703700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba41491d0 con 0x7f7ba4062270 2022-06-17T12:57:16.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.354+0000 7f7bac703700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba41467b0 con 0x7f7ba4147ad0 2022-06-17T12:57:16.389 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.354+0000 7f7ba9c9e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ba4143e80 0x7f7ba405fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:16.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.354+0000 7f7ba9c9e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ba4143e80 0x7f7ba405fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40748/0 (socket says 172.21.15.36:40748) 2022-06-17T12:57:16.390 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.354+0000 7f7ba9c9e700 1 -- 172.21.15.36:0/1630796290 learned_addr learned my addr 172.21.15.36:0/1630796290 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:16.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.354+0000 7f7baa49f700 1 --2- 172.21.15.36:0/1630796290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ba4062270 0x7f7ba4152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:16.391 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.354+0000 7f7baaca0700 1 --2- 172.21.15.36:0/1630796290 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ba4147ad0 0x7f7ba4152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:16.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.354+0000 7f7ba9c9e700 1 -- 172.21.15.36:0/1630796290 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ba4147ad0 msgr2=0x7f7ba4152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:16.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.354+0000 7f7ba9c9e700 1 --2- 172.21.15.36:0/1630796290 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ba4147ad0 0x7f7ba4152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.392 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.354+0000 7f7ba9c9e700 1 -- 172.21.15.36:0/1630796290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ba4062270 msgr2=0x7f7ba4152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:16.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.354+0000 7f7ba9c9e700 1 --2- 172.21.15.36:0/1630796290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ba4062270 0x7f7ba4152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.354+0000 7f7ba9c9e700 1 -- 172.21.15.36:0/1630796290 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9c009470 con 0x7f7ba4143e80 2022-06-17T12:57:16.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.354+0000 7f7ba9c9e700 1 --2- 172.21.15.36:0/1630796290 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ba4143e80 0x7f7ba405fc10 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7f7ba4062b00 tx=0x7f7b94007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:16.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.354+0000 7f7b937fe700 1 -- 172.21.15.36:0/1630796290 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b940070d0 con 0x7f7ba4143e80 2022-06-17T12:57:16.396 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.354+0000 7f7b937fe700 1 -- 172.21.15.36:0/1630796290 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b94007230 con 0x7f7ba4143e80 2022-06-17T12:57:16.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.354+0000 7f7bac703700 1 -- 172.21.15.36:0/1630796290 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba4146a50 con 0x7f7ba4143e80 2022-06-17T12:57:16.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.354+0000 7f7b937fe700 1 -- 172.21.15.36:0/1630796290 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b94005610 con 0x7f7ba4143e80 2022-06-17T12:57:16.397 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.354+0000 7f7bac703700 1 -- 172.21.15.36:0/1630796290 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba4146e20 con 0x7f7ba4143e80 2022-06-17T12:57:16.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.358+0000 7f7b937fe700 1 -- 172.21.15.36:0/1630796290 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f7b94010070 con 0x7f7ba4143e80 2022-06-17T12:57:16.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.358+0000 7f7bac703700 1 -- 172.21.15.36:0/1630796290 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba40b4210 con 0x7f7ba4143e80 2022-06-17T12:57:16.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.358+0000 7f7b937fe700 1 --2- 172.21.15.36:0/1630796290 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7b84053a40 0x7f7b84055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:16.398 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.358+0000 7f7b937fe700 1 -- 172.21.15.36:0/1630796290 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7b9406b5d0 con 0x7f7ba4143e80 2022-06-17T12:57:16.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.358+0000 7f7baa49f700 1 --2- 172.21.15.36:0/1630796290 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7b84053a40 0x7f7b84055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:16.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.358+0000 7f7baa49f700 1 --2- 172.21.15.36:0/1630796290 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7b84053a40 0x7f7b84055f00 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f7ba000b000 tx=0x7f7ba0006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:16.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.366+0000 7f7b937fe700 1 -- 172.21.15.36:0/1630796290 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7b940423d0 con 0x7f7ba4143e80 2022-06-17T12:57:16.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.530+0000 7f7bac703700 1 -- 172.21.15.36:0/1630796290 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f7ba4057410 con 0x7f7ba4143e80 2022-06-17T12:57:16.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.538+0000 7f7b937fe700 1 -- 172.21.15.36:0/1630796290 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v47) v1 ==== 184+0+0 (secure 0 0 0) 0x7f7b940459f0 con 0x7f7ba4143e80 2022-06-17T12:57:16.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.538+0000 7f7bac703700 1 -- 172.21.15.36:0/1630796290 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7b84053a40 msgr2=0x7f7b84055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:16.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.538+0000 7f7bac703700 1 --2- 172.21.15.36:0/1630796290 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7b84053a40 0x7f7b84055f00 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f7ba000b000 tx=0x7f7ba0006df0).stop 2022-06-17T12:57:16.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.538+0000 7f7bac703700 1 -- 172.21.15.36:0/1630796290 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ba4143e80 msgr2=0x7f7ba405fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:16.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.538+0000 7f7bac703700 1 --2- 172.21.15.36:0/1630796290 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ba4143e80 0x7f7ba405fc10 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7f7ba4062b00 tx=0x7f7b94007c40).stop 2022-06-17T12:57:16.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.538+0000 7f7bac703700 1 -- 172.21.15.36:0/1630796290 shutdown_connections 2022-06-17T12:57:16.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.538+0000 7f7bac703700 1 --2- 172.21.15.36:0/1630796290 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7b84053a40 0x7f7b84055f00 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.538+0000 7f7bac703700 1 --2- 172.21.15.36:0/1630796290 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ba4147ad0 0x7f7ba4152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.538+0000 7f7bac703700 1 --2- 172.21.15.36:0/1630796290 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7ba4143e80 0x7f7ba405fc10 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.532 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.538+0000 7f7bac703700 1 --2- 172.21.15.36:0/1630796290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ba4062270 0x7f7ba4152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.538+0000 7f7bac703700 1 -- 172.21.15.36:0/1630796290 >> 172.21.15.36:0/1630796290 conn(0x7f7ba40b9870 msgr2=0x7f7ba40b2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:16.533 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.538+0000 7f7bac703700 1 -- 172.21.15.36:0/1630796290 shutdown_connections 2022-06-17T12:57:16.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.538+0000 7f7bac703700 1 -- 172.21.15.36:0/1630796290 wait complete. 2022-06-17T12:57:16.534 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.0 2022-06-17T12:57:16.537 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:57:16.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.694+0000 7f1c92888700 1 Processor -- start 2022-06-17T12:57:16.677 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.694+0000 7f1c92888700 1 -- start start 2022-06-17T12:57:16.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.694+0000 7f1c92888700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f1c8c062270 0x7f1c8c142f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:16.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.694+0000 7f1c92888700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1c8c143e80 0x7f1c8c146970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:16.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.694+0000 7f1c92888700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c8c0b44b0 con 0x7f1c8c0b0cb0 2022-06-17T12:57:16.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.694+0000 7f1c92888700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c8c0b6650 con 0x7f1c8c062270 2022-06-17T12:57:16.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.694+0000 7f1c92888700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c062650 con 0x7f1c8c143e80 2022-06-17T12:57:16.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.694+0000 7f1c8bfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f1c8c062270 0x7f1c8c142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:16.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.694+0000 7f1c8bfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f1c8c062270 0x7f1c8c142f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40754/0 (socket says 172.21.15.36:40754) 2022-06-17T12:57:16.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.694+0000 7f1c8bfff700 1 -- 172.21.15.36:0/2912765756 learned_addr learned my addr 172.21.15.36:0/2912765756 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:16.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.694+0000 7f1c8b7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f1c8c0b0cb0 0x7f1c8c143770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58286/0 (socket says 172.21.15.36:58286) 2022-06-17T12:57:16.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.694+0000 7f1c90e25700 1 --2- 172.21.15.36:0/2912765756 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1c8c143e80 0x7f1c8c146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:16.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.694+0000 7f1c8bfff700 1 -- 172.21.15.36:0/2912765756 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1c8c143e80 msgr2=0x7f1c8c146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:16.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.694+0000 7f1c8affd700 1 -- 172.21.15.36:0/2912765756 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1231195594 0 0) 0x7f1c8c0b44b0 con 0x7f1c8c0b0cb0 2022-06-17T12:57:16.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.694+0000 7f1c8bfff700 1 --2- 172.21.15.36:0/2912765756 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1c8c143e80 0x7f1c8c146970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.694+0000 7f1c8bfff700 1 -- 172.21.15.36:0/2912765756 >> v1:172.21.15.36:6789/0 conn(0x7f1c8c0b0cb0 legacy=0x7f1c8c143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:16.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.694+0000 7f1c8bfff700 1 -- 172.21.15.36:0/2912765756 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c8c0b4210 con 0x7f1c8c062270 2022-06-17T12:57:16.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.694+0000 7f1c8bfff700 1 --2- 172.21.15.36:0/2912765756 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1c8c062270 0x7f1c8c142f20 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0x7f1c80002f70 tx=0x7f1c80006eb0).ready entity=mon.0 client_cookie=5f8b7e5e7ffe4b6b server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:16.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.694+0000 7f1c8affd700 1 -- 172.21.15.36:0/2912765756 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c8000d570 con 0x7f1c8c062270 2022-06-17T12:57:16.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.694+0000 7f1c8affd700 1 -- 172.21.15.36:0/2912765756 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c80000e90 con 0x7f1c8c062270 2022-06-17T12:57:16.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.694+0000 7f1c8affd700 1 -- 172.21.15.36:0/2912765756 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c8000d8c0 con 0x7f1c8c062270 2022-06-17T12:57:16.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.698+0000 7f1c92888700 1 -- 172.21.15.36:0/2912765756 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1c8c062270 msgr2=0x7f1c8c142f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:16.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.698+0000 7f1c92888700 1 --2- 172.21.15.36:0/2912765756 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1c8c062270 0x7f1c8c142f20 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0x7f1c80002f70 tx=0x7f1c80006eb0).stop 2022-06-17T12:57:16.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.698+0000 7f1c92888700 1 -- 172.21.15.36:0/2912765756 shutdown_connections 2022-06-17T12:57:16.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.698+0000 7f1c92888700 1 --2- 172.21.15.36:0/2912765756 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1c8c143e80 0x7f1c8c146970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.698+0000 7f1c92888700 1 --2- 172.21.15.36:0/2912765756 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1c8c062270 0x7f1c8c142f20 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.698+0000 7f1c92888700 1 -- 172.21.15.36:0/2912765756 >> 172.21.15.36:0/2912765756 conn(0x7f1c8c0b9870 msgr2=0x7f1c8c0b9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:16.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.698+0000 7f1c92888700 1 -- 172.21.15.36:0/2912765756 shutdown_connections 2022-06-17T12:57:16.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.698+0000 7f1c92888700 1 -- 172.21.15.36:0/2912765756 wait complete. 2022-06-17T12:57:16.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.698+0000 7f1c92888700 1 Processor -- start 2022-06-17T12:57:16.692 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.698+0000 7f1c92888700 1 -- start start 2022-06-17T12:57:16.696 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.702+0000 7f1c92888700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1c8c062270 0x7f1c8c144870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:16.697 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.702+0000 7f1c92888700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1c8c143e80 0x7f1c8c1491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:16.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.702+0000 7f1c92888700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c8c0ae850 0x7f1c8c147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:16.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.702+0000 7f1c92888700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c148d70 con 0x7f1c8c062270 2022-06-17T12:57:16.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.702+0000 7f1c92888700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c145580 con 0x7f1c8c0ae850 2022-06-17T12:57:16.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.702+0000 7f1c92888700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c0af010 con 0x7f1c8c143e80 2022-06-17T12:57:16.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.702+0000 7f1c8bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1c8c062270 0x7f1c8c144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:16.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.702+0000 7f1c8bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1c8c062270 0x7f1c8c144870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40760/0 (socket says 172.21.15.36:40760) 2022-06-17T12:57:16.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.702+0000 7f1c8bfff700 1 -- 172.21.15.36:0/2591926243 learned_addr learned my addr 172.21.15.36:0/2591926243 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:16.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.702+0000 7f1c8b7fe700 1 --2- 172.21.15.36:0/2591926243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1c8c143e80 0x7f1c8c1491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:16.700 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.702+0000 7f1c90e25700 1 --2- 172.21.15.36:0/2591926243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c8c0ae850 0x7f1c8c147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:16.701 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.702+0000 7f1c8bfff700 1 -- 172.21.15.36:0/2591926243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1c8c143e80 msgr2=0x7f1c8c1491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:16.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.702+0000 7f1c8bfff700 1 --2- 172.21.15.36:0/2591926243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1c8c143e80 0x7f1c8c1491a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.702+0000 7f1c8bfff700 1 -- 172.21.15.36:0/2591926243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c8c0ae850 msgr2=0x7f1c8c147cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:16.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.702+0000 7f1c8bfff700 1 --2- 172.21.15.36:0/2591926243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c8c0ae850 0x7f1c8c147cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.702+0000 7f1c8bfff700 1 -- 172.21.15.36:0/2591926243 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c8000b1e0 con 0x7f1c8c062270 2022-06-17T12:57:16.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.702+0000 7f1c90e25700 1 --2- 172.21.15.36:0/2591926243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c8c0ae850 0x7f1c8c147cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:57:16.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.702+0000 7f1c8bfff700 1 --2- 172.21.15.36:0/2591926243 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1c8c062270 0x7f1c8c144870 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7f1c80002f70 tx=0x7f1c80015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:16.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.702+0000 7f1c897fa700 1 -- 172.21.15.36:0/2591926243 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c800068b0 con 0x7f1c8c062270 2022-06-17T12:57:16.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.702+0000 7f1c897fa700 1 -- 172.21.15.36:0/2591926243 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8000d8f0 con 0x7f1c8c062270 2022-06-17T12:57:16.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.702+0000 7f1c92888700 1 -- 172.21.15.36:0/2591926243 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c8c0af2b0 con 0x7f1c8c062270 2022-06-17T12:57:16.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.702+0000 7f1c897fa700 1 -- 172.21.15.36:0/2591926243 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c80011e20 con 0x7f1c8c062270 2022-06-17T12:57:16.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.702+0000 7f1c92888700 1 -- 172.21.15.36:0/2591926243 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c8c149690 con 0x7f1c8c062270 2022-06-17T12:57:16.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.706+0000 7f1c92888700 1 -- 172.21.15.36:0/2591926243 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c8c0b63b0 con 0x7f1c8c062270 2022-06-17T12:57:16.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.706+0000 7f1c897fa700 1 -- 172.21.15.36:0/2591926243 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f1c80025030 con 0x7f1c8c062270 2022-06-17T12:57:16.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.706+0000 7f1c897fa700 1 --2- 172.21.15.36:0/2591926243 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1c6c053af0 0x7f1c6c055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:16.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.706+0000 7f1c897fa700 1 -- 172.21.15.36:0/2591926243 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1c8006d0e0 con 0x7f1c8c062270 2022-06-17T12:57:16.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.706+0000 7f1c8b7fe700 1 --2- 172.21.15.36:0/2591926243 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1c6c053af0 0x7f1c6c055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:16.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.706+0000 7f1c8b7fe700 1 --2- 172.21.15.36:0/2591926243 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1c6c053af0 0x7f1c6c055fb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7f1c7c00d000 tx=0x7f1c7c007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:16.712 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:57:16.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.714+0000 7f1c897fa700 1 -- 172.21.15.36:0/2591926243 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1c80043ee0 con 0x7f1c8c062270 2022-06-17T12:57:16.716 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:57:16.721 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:57:16.725 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:57:16.729 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:57:16.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.882+0000 7f1c92888700 1 -- 172.21.15.36:0/2591926243 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f1c8c057410 con 0x7f1c8c062270 2022-06-17T12:57:16.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.898+0000 7f1c897fa700 1 -- 172.21.15.36:0/2591926243 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v48) v1 ==== 184+0+0 (secure 0 0 0) 0x7f1c80023020 con 0x7f1c8c062270 2022-06-17T12:57:16.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.902+0000 7f1c92888700 1 -- 172.21.15.36:0/2591926243 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1c6c053af0 msgr2=0x7f1c6c055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:16.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.902+0000 7f1c92888700 1 --2- 172.21.15.36:0/2591926243 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1c6c053af0 0x7f1c6c055fb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7f1c7c00d000 tx=0x7f1c7c007a00).stop 2022-06-17T12:57:16.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.902+0000 7f1c92888700 1 -- 172.21.15.36:0/2591926243 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1c8c062270 msgr2=0x7f1c8c144870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:16.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.902+0000 7f1c92888700 1 --2- 172.21.15.36:0/2591926243 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1c8c062270 0x7f1c8c144870 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7f1c80002f70 tx=0x7f1c80015000).stop 2022-06-17T12:57:16.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.902+0000 7f1c92888700 1 -- 172.21.15.36:0/2591926243 shutdown_connections 2022-06-17T12:57:16.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.902+0000 7f1c92888700 1 --2- 172.21.15.36:0/2591926243 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f1c6c053af0 0x7f1c6c055fb0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.902+0000 7f1c92888700 1 --2- 172.21.15.36:0/2591926243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c8c0ae850 0x7f1c8c147cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.902+0000 7f1c92888700 1 --2- 172.21.15.36:0/2591926243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1c8c143e80 0x7f1c8c1491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.902+0000 7f1c92888700 1 --2- 172.21.15.36:0/2591926243 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f1c8c062270 0x7f1c8c144870 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:16.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.902+0000 7f1c92888700 1 -- 172.21.15.36:0/2591926243 >> 172.21.15.36:0/2591926243 conn(0x7f1c8c0b9870 msgr2=0x7f1c8c0b7640 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:16.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.902+0000 7f1c92888700 1 -- 172.21.15.36:0/2591926243 shutdown_connections 2022-06-17T12:57:16.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:16.902+0000 7f1c92888700 1 -- 172.21.15.36:0/2591926243 wait complete. 2022-06-17T12:57:16.904 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.1 2022-06-17T12:57:16.907 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-17T12:57:17.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.046+0000 7f7fb49e1700 1 Processor -- start 2022-06-17T12:57:17.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.046+0000 7f7fb49e1700 1 -- start start 2022-06-17T12:57:17.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.046+0000 7f7fb49e1700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7fb0060ac0 0x7f7fb00afe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:17.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.046+0000 7f7fb49e1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fb00aee30 0x7f7fb013c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:17.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.046+0000 7f7fb49e1700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb0060ea0 con 0x7f7fb00b20f0 2022-06-17T12:57:17.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.046+0000 7f7fb49e1700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb00b3060 con 0x7f7fb0060ac0 2022-06-17T12:57:17.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.046+0000 7f7fb49e1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb0067a20 con 0x7f7fb00aee30 2022-06-17T12:57:17.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.046+0000 7f7faf59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fb00aee30 0x7f7fb013c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:17.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.046+0000 7f7faf59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fb00aee30 0x7f7fb013c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:55560/0 (socket says 172.21.15.36:55560) 2022-06-17T12:57:17.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.046+0000 7f7fa7fff700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7fb00b20f0 0x7f7fb00ae720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:54036/0 (socket says 172.21.15.36:54036) 2022-06-17T12:57:17.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.046+0000 7f7fa7fff700 1 -- 172.21.15.36:0/1474894884 learned_addr learned my addr 172.21.15.36:0/1474894884 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:17.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.046+0000 7f7faed9d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7fb0060ac0 0x7f7fb00afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:17.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.046+0000 7f7fae59c700 1 -- 172.21.15.36:0/1474894884 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697458849 0 0) 0x7f7fb0060ea0 con 0x7f7fb00b20f0 2022-06-17T12:57:17.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.046+0000 7f7fae59c700 1 -- 172.21.15.36:0/1474894884 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f90003260 con 0x7f7fb00b20f0 2022-06-17T12:57:17.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.046+0000 7f7faf59e700 1 -- 172.21.15.36:0/1474894884 >> v2:172.21.15.70:3300/0 conn(0x7f7fb0060ac0 msgr2=0x7f7fb00afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:17.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.046+0000 7f7faf59e700 1 --2- 172.21.15.36:0/1474894884 >> v2:172.21.15.70:3300/0 conn(0x7f7fb0060ac0 0x7f7fb00afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:17.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.046+0000 7f7faf59e700 1 -- 172.21.15.36:0/1474894884 >> v1:172.21.15.70:6789/0 conn(0x7f7fb00b20f0 legacy=0x7f7fb00ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:17.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.046+0000 7f7faf59e700 1 -- 172.21.15.36:0/1474894884 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb0143fb0 con 0x7f7fb00aee30 2022-06-17T12:57:17.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.046+0000 7f7faf59e700 1 --2- 172.21.15.36:0/1474894884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fb00aee30 0x7f7fb013c7e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f7fa80032f0 tx=0x7f7fa80091d0).ready entity=mon.2 client_cookie=ea5fe0db4fcad2d9 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:17.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.046+0000 7f7fae59c700 1 -- 172.21.15.36:0/1474894884 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fa800b600 con 0x7f7fb00aee30 2022-06-17T12:57:17.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.046+0000 7f7fae59c700 1 -- 172.21.15.36:0/1474894884 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa800b760 con 0x7f7fb00aee30 2022-06-17T12:57:17.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.046+0000 7f7fae59c700 1 -- 172.21.15.36:0/1474894884 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fa800b8d0 con 0x7f7fb00aee30 2022-06-17T12:57:17.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.050+0000 7f7fb49e1700 1 -- 172.21.15.36:0/1474894884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fb00aee30 msgr2=0x7f7fb013c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:17.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.050+0000 7f7fb49e1700 1 --2- 172.21.15.36:0/1474894884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fb00aee30 0x7f7fb013c7e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f7fa80032f0 tx=0x7f7fa80091d0).stop 2022-06-17T12:57:17.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.050+0000 7f7fb49e1700 1 -- 172.21.15.36:0/1474894884 shutdown_connections 2022-06-17T12:57:17.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.050+0000 7f7fb49e1700 1 --2- 172.21.15.36:0/1474894884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fb00aee30 0x7f7fb013c7e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:17.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.050+0000 7f7fb49e1700 1 --2- 172.21.15.36:0/1474894884 >> v2:172.21.15.70:3300/0 conn(0x7f7fb0060ac0 0x7f7fb00afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:17.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.050+0000 7f7fb49e1700 1 -- 172.21.15.36:0/1474894884 >> 172.21.15.36:0/1474894884 conn(0x7f7fb00b65f0 msgr2=0x7f7fb00b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:17.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.050+0000 7f7fb49e1700 1 -- 172.21.15.36:0/1474894884 shutdown_connections 2022-06-17T12:57:17.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.050+0000 7f7fb49e1700 1 -- 172.21.15.36:0/1474894884 wait complete. 2022-06-17T12:57:17.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.050+0000 7f7fb49e1700 1 Processor -- start 2022-06-17T12:57:17.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.050+0000 7f7fb49e1700 1 -- start start 2022-06-17T12:57:17.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.050+0000 7f7fb49e1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fb0060ac0 0x7f7fb010f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:17.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.050+0000 7f7fb49e1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fb00aee30 0x7f7fb013b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:17.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.050+0000 7f7fb49e1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fb00b20f0 0x7f7fb013b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:17.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.050+0000 7f7fb49e1700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb010f8c0 con 0x7f7fb00aee30 2022-06-17T12:57:17.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.050+0000 7f7fb49e1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb0153230 con 0x7f7fb0060ac0 2022-06-17T12:57:17.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.050+0000 7f7fb49e1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb010fb00 con 0x7f7fb00b20f0 2022-06-17T12:57:17.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.054+0000 7f7fa7fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fb00aee30 0x7f7fb013b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:17.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.054+0000 7f7fa7fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fb00aee30 0x7f7fb013b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40774/0 (socket says 172.21.15.36:40774) 2022-06-17T12:57:17.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.054+0000 7f7faf59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fb00b20f0 0x7f7fb013b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:17.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.054+0000 7f7fa7fff700 1 -- 172.21.15.36:0/538158428 learned_addr learned my addr 172.21.15.36:0/538158428 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:17.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.054+0000 7f7faed9d700 1 --2- 172.21.15.36:0/538158428 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fb0060ac0 0x7f7fb010f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:17.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.054+0000 7f7faf59e700 1 -- 172.21.15.36:0/538158428 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fb0060ac0 msgr2=0x7f7fb010f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:17.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.054+0000 7f7faf59e700 1 --2- 172.21.15.36:0/538158428 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fb0060ac0 0x7f7fb010f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:17.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.054+0000 7f7faf59e700 1 -- 172.21.15.36:0/538158428 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fb00aee30 msgr2=0x7f7fb013b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:17.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.054+0000 7f7faf59e700 1 --2- 172.21.15.36:0/538158428 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fb00aee30 0x7f7fb013b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:17.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.054+0000 7f7faf59e700 1 -- 172.21.15.36:0/538158428 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa8009470 con 0x7f7fb00b20f0 2022-06-17T12:57:17.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.054+0000 7f7faf59e700 1 --2- 172.21.15.36:0/538158428 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fb00b20f0 0x7f7fb013b840 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f7fa80032f0 tx=0x7f7fa8016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:17.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.054+0000 7f7facd99700 1 -- 172.21.15.36:0/538158428 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fa800b8d0 con 0x7f7fb00b20f0 2022-06-17T12:57:17.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.054+0000 7f7fb49e1700 1 -- 172.21.15.36:0/538158428 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb013faa0 con 0x7f7fb00b20f0 2022-06-17T12:57:17.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.054+0000 7f7facd99700 1 -- 172.21.15.36:0/538158428 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa801f3f0 con 0x7f7fb00b20f0 2022-06-17T12:57:17.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.054+0000 7f7facd99700 1 -- 172.21.15.36:0/538158428 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fa800b600 con 0x7f7fb00b20f0 2022-06-17T12:57:17.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.054+0000 7f7fb49e1700 1 -- 172.21.15.36:0/538158428 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb0148a10 con 0x7f7fb00b20f0 2022-06-17T12:57:17.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.054+0000 7f7fb49e1700 1 -- 172.21.15.36:0/538158428 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb013f840 con 0x7f7fb00b20f0 2022-06-17T12:57:17.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.058+0000 7f7facd99700 1 -- 172.21.15.36:0/538158428 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f7fa800f070 con 0x7f7fb00b20f0 2022-06-17T12:57:17.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.058+0000 7f7facd99700 1 --2- 172.21.15.36:0/538158428 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7f94053a40 0x7f7f94055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:17.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.058+0000 7f7facd99700 1 -- 172.21.15.36:0/538158428 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7fa806c7c0 con 0x7f7fb00b20f0 2022-06-17T12:57:17.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.058+0000 7f7faed9d700 1 --2- 172.21.15.36:0/538158428 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7f94053a40 0x7f7f94055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:17.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.058+0000 7f7faed9d700 1 --2- 172.21.15.36:0/538158428 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7f94053a40 0x7f7f94055f00 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f7fa000b000 tx=0x7f7fa0006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:17.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.066+0000 7f7facd99700 1 -- 172.21.15.36:0/538158428 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7fa8043520 con 0x7f7fb00b20f0 2022-06-17T12:57:17.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.234+0000 7f7fb49e1700 1 -- 172.21.15.36:0/538158428 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f7fb0057410 con 0x7f7fb00b20f0 2022-06-17T12:57:17.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.246+0000 7f7facd99700 1 -- 172.21.15.36:0/538158428 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v49) v1 ==== 184+0+0 (secure 0 0 0) 0x7f7fa80150e0 con 0x7f7fb00b20f0 2022-06-17T12:57:17.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.250+0000 7f7fb49e1700 1 -- 172.21.15.36:0/538158428 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7f94053a40 msgr2=0x7f7f94055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:17.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.250+0000 7f7fb49e1700 1 --2- 172.21.15.36:0/538158428 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7f94053a40 0x7f7f94055f00 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f7fa000b000 tx=0x7f7fa0006df0).stop 2022-06-17T12:57:17.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.250+0000 7f7fb49e1700 1 -- 172.21.15.36:0/538158428 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fb00b20f0 msgr2=0x7f7fb013b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:17.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.250+0000 7f7fb49e1700 1 --2- 172.21.15.36:0/538158428 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fb00b20f0 0x7f7fb013b840 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f7fa80032f0 tx=0x7f7fa8016000).stop 2022-06-17T12:57:17.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.250+0000 7f7fb49e1700 1 -- 172.21.15.36:0/538158428 shutdown_connections 2022-06-17T12:57:17.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.250+0000 7f7fb49e1700 1 --2- 172.21.15.36:0/538158428 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7f94053a40 0x7f7f94055f00 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:17.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.250+0000 7f7fb49e1700 1 --2- 172.21.15.36:0/538158428 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fb00b20f0 0x7f7fb013b840 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:17.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.250+0000 7f7fb49e1700 1 --2- 172.21.15.36:0/538158428 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7fb00aee30 0x7f7fb013b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:17.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.250+0000 7f7fb49e1700 1 --2- 172.21.15.36:0/538158428 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fb0060ac0 0x7f7fb010f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:17.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.250+0000 7f7fb49e1700 1 -- 172.21.15.36:0/538158428 >> 172.21.15.36:0/538158428 conn(0x7f7fb00b65f0 msgr2=0x7f7fb00b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:17.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.250+0000 7f7fb49e1700 1 -- 172.21.15.36:0/538158428 shutdown_connections 2022-06-17T12:57:17.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.250+0000 7f7fb49e1700 1 -- 172.21.15.36:0/538158428 wait complete. 2022-06-17T12:57:17.253 INFO:teuthology.orchestra.run.smithi036.stderr:updated caps for client.2 2022-06-17T12:57:17.254 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:57:17.393 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc98f7d3700 1 Processor -- start 2022-06-17T12:57:17.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc98f7d3700 1 -- start start 2022-06-17T12:57:17.394 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc98f7d3700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc988062b50 0x7fc9880ada40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:17.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc98f7d3700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9880b0ba0 con 0x7fc988144c00 2022-06-17T12:57:17.400 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc98f7d3700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc988147e90 con 0x7fc988062770 2022-06-17T12:57:17.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc98f7d3700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9880ae820 con 0x7fc988062b50 2022-06-17T12:57:17.401 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc98cd6e700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fc988144c00 0x7fc9880ad330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58314/0 (socket says 172.21.15.36:58314) 2022-06-17T12:57:17.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc98cd6e700 1 -- 172.21.15.36:0/508888775 learned_addr learned my addr 172.21.15.36:0/508888775 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:17.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc98dd70700 1 --2- 172.21.15.36:0/508888775 >> v2:172.21.15.70:3300/0 conn(0x7fc988062b50 0x7fc9880ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:17.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc987fff700 1 -- 172.21.15.36:0/508888775 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1054650680 0 0) 0x7fc9880b0ba0 con 0x7fc988144c00 2022-06-17T12:57:17.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc987fff700 1 -- 172.21.15.36:0/508888775 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc974003260 con 0x7fc988144c00 2022-06-17T12:57:17.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc98dd70700 1 -- 172.21.15.36:0/508888775 >> v1:172.21.15.70:6789/0 conn(0x7fc988062770 legacy=0x7fc9881444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:17.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc98dd70700 1 -- 172.21.15.36:0/508888775 >> v1:172.21.15.36:6789/0 conn(0x7fc988144c00 legacy=0x7fc9880ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:17.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc98dd70700 1 -- 172.21.15.36:0/508888775 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9880b0dd0 con 0x7fc988062b50 2022-06-17T12:57:17.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc98dd70700 1 --2- 172.21.15.36:0/508888775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc988062b50 0x7fc9880ada40 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7fc980002f70 tx=0x7fc980006df0).ready entity=mon.1 client_cookie=f0ef4f609acf1cab server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:17.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc987fff700 1 -- 172.21.15.36:0/508888775 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc98000d610 con 0x7fc988062b50 2022-06-17T12:57:17.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc987fff700 1 -- 172.21.15.36:0/508888775 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc980000e90 con 0x7fc988062b50 2022-06-17T12:57:17.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc987fff700 1 -- 172.21.15.36:0/508888775 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc98000d960 con 0x7fc988062b50 2022-06-17T12:57:17.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc98f7d3700 1 -- 172.21.15.36:0/508888775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc988062b50 msgr2=0x7fc9880ada40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:17.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc98f7d3700 1 --2- 172.21.15.36:0/508888775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc988062b50 0x7fc9880ada40 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7fc980002f70 tx=0x7fc980006df0).stop 2022-06-17T12:57:17.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc98f7d3700 1 -- 172.21.15.36:0/508888775 shutdown_connections 2022-06-17T12:57:17.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc98f7d3700 1 --2- 172.21.15.36:0/508888775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc988062b50 0x7fc9880ada40 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:17.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc98f7d3700 1 -- 172.21.15.36:0/508888775 >> 172.21.15.36:0/508888775 conn(0x7fc9880b53e0 msgr2=0x7fc9880b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:17.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc98f7d3700 1 -- 172.21.15.36:0/508888775 shutdown_connections 2022-06-17T12:57:17.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.382+0000 7fc98f7d3700 1 -- 172.21.15.36:0/508888775 wait complete. 2022-06-17T12:57:17.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.386+0000 7fc98f7d3700 1 Processor -- start 2022-06-17T12:57:17.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.386+0000 7fc98f7d3700 1 -- start start 2022-06-17T12:57:17.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.386+0000 7fc98f7d3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc988062770 0x7fc9880664b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:17.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.386+0000 7fc98f7d3700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc988062b50 0x7fc9881571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:17.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.386+0000 7fc98f7d3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc988144c00 0x7fc988148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:17.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.386+0000 7fc98f7d3700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc988156fb0 con 0x7fc988062770 2022-06-17T12:57:17.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.386+0000 7fc98f7d3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9881576e0 con 0x7fc988144c00 2022-06-17T12:57:17.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.386+0000 7fc98f7d3700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98814b770 con 0x7fc988062b50 2022-06-17T12:57:17.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.386+0000 7fc98d56f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc988062770 0x7fc9880664b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:17.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.386+0000 7fc98d56f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc988062770 0x7fc9880664b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40788/0 (socket says 172.21.15.36:40788) 2022-06-17T12:57:17.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.386+0000 7fc98d56f700 1 -- 172.21.15.36:0/2709434682 learned_addr learned my addr 172.21.15.36:0/2709434682 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:17.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.386+0000 7fc98dd70700 1 --2- 172.21.15.36:0/2709434682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc988144c00 0x7fc988148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:17.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.386+0000 7fc98cd6e700 1 --2- 172.21.15.36:0/2709434682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc988062b50 0x7fc9881571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:17.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.386+0000 7fc98d56f700 1 -- 172.21.15.36:0/2709434682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc988062b50 msgr2=0x7fc9881571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:17.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.386+0000 7fc98d56f700 1 --2- 172.21.15.36:0/2709434682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc988062b50 0x7fc9881571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:17.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.386+0000 7fc98d56f700 1 -- 172.21.15.36:0/2709434682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc988144c00 msgr2=0x7fc988148910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:17.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.386+0000 7fc98d56f700 1 --2- 172.21.15.36:0/2709434682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc988144c00 0x7fc988148910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:17.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.386+0000 7fc98d56f700 1 -- 172.21.15.36:0/2709434682 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98000b180 con 0x7fc988062770 2022-06-17T12:57:17.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.386+0000 7fc98d56f700 1 --2- 172.21.15.36:0/2709434682 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc988062770 0x7fc9880664b0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7fc98805d2b0 tx=0x7fc9780014a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:17.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.390+0000 7fc9867fc700 1 -- 172.21.15.36:0/2709434682 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc97800e4a0 con 0x7fc988062770 2022-06-17T12:57:17.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.390+0000 7fc9867fc700 1 -- 172.21.15.36:0/2709434682 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97800e600 con 0x7fc988062770 2022-06-17T12:57:17.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.390+0000 7fc98f7d3700 1 -- 172.21.15.36:0/2709434682 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc98814a6a0 con 0x7fc988062770 2022-06-17T12:57:17.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.390+0000 7fc9867fc700 1 -- 172.21.15.36:0/2709434682 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc97800e960 con 0x7fc988062770 2022-06-17T12:57:17.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.390+0000 7fc98f7d3700 1 -- 172.21.15.36:0/2709434682 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc988144810 con 0x7fc988062770 2022-06-17T12:57:17.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.390+0000 7fc98f7d3700 1 -- 172.21.15.36:0/2709434682 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9880b1c20 con 0x7fc988062770 2022-06-17T12:57:17.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.390+0000 7fc9867fc700 1 -- 172.21.15.36:0/2709434682 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fc978019030 con 0x7fc988062770 2022-06-17T12:57:17.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.390+0000 7fc9867fc700 1 --2- 172.21.15.36:0/2709434682 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc9680539b0 0x7fc968055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:17.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.390+0000 7fc9867fc700 1 -- 172.21.15.36:0/2709434682 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc97806b640 con 0x7fc988062770 2022-06-17T12:57:17.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.390+0000 7fc98cd6e700 1 --2- 172.21.15.36:0/2709434682 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc9680539b0 0x7fc968055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:17.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.394+0000 7fc98cd6e700 1 --2- 172.21.15.36:0/2709434682 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc9680539b0 0x7fc968055e70 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7fc97c00d000 tx=0x7fc97c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:17.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.398+0000 7fc9867fc700 1 -- 172.21.15.36:0/2709434682 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc9780423e0 con 0x7fc988062770 2022-06-17T12:57:17.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.562+0000 7fc98f7d3700 1 -- 172.21.15.36:0/2709434682 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc988067300 con 0x7fc988062770 2022-06-17T12:57:17.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.562+0000 7fc9867fc700 1 -- 172.21.15.36:0/2709434682 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+2142 (secure 0 0 0) 0x7fc978045a00 con 0x7fc988062770 2022-06-17T12:57:17.553 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:57:17.554 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":83,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8599,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/4216085219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4216085219},{"type":"v1","addr":"172.21.15.70:6833","nonce":4216085219}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":82}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:57:14.606323+0000","modified":"2022-06-17T12:57:15.610154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8805":{"gid":8805,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/2534551587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2534551587},{"type":"v1","addr":"172.21.15.36:6835","nonce":2534551587}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-06-17T12:57:17.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.566+0000 7fc98f7d3700 1 -- 172.21.15.36:0/2709434682 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc9680539b0 msgr2=0x7fc968055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:17.556 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.566+0000 7fc98f7d3700 1 --2- 172.21.15.36:0/2709434682 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc9680539b0 0x7fc968055e70 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7fc97c00d000 tx=0x7fc97c009040).stop 2022-06-17T12:57:17.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.566+0000 7fc98f7d3700 1 -- 172.21.15.36:0/2709434682 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc988062770 msgr2=0x7fc9880664b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:17.557 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.566+0000 7fc98f7d3700 1 --2- 172.21.15.36:0/2709434682 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc988062770 0x7fc9880664b0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7fc98805d2b0 tx=0x7fc9780014a0).stop 2022-06-17T12:57:17.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.566+0000 7fc98f7d3700 1 -- 172.21.15.36:0/2709434682 shutdown_connections 2022-06-17T12:57:17.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.566+0000 7fc98f7d3700 1 --2- 172.21.15.36:0/2709434682 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc9680539b0 0x7fc968055e70 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:17.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.566+0000 7fc98f7d3700 1 --2- 172.21.15.36:0/2709434682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc988144c00 0x7fc988148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:17.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.566+0000 7fc98f7d3700 1 --2- 172.21.15.36:0/2709434682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc988062b50 0x7fc9881571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:17.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.566+0000 7fc98f7d3700 1 --2- 172.21.15.36:0/2709434682 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc988062770 0x7fc9880664b0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:17.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.566+0000 7fc98f7d3700 1 -- 172.21.15.36:0/2709434682 >> 172.21.15.36:0/2709434682 conn(0x7fc9880b53e0 msgr2=0x7fc9880b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:17.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.566+0000 7fc98f7d3700 1 -- 172.21.15.36:0/2709434682 shutdown_connections 2022-06-17T12:57:17.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:17.566+0000 7fc98f7d3700 1 -- 172.21.15.36:0/2709434682 wait complete. 2022-06-17T12:57:17.562 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 83 2022-06-17T12:57:17.768 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:57:14.606323+0000', 'modified': '2022-06-17T12:57:15.610154+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8805}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8805': {'gid': 8805, 'name': 'a', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.36:6835/2534551587', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 2534551587}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 2534551587}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:57:17.769 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:57:17.770 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:57:17.903 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":8805,"want_state":"up:active","state":"up:active","rank_uptime":3.2900540390000002,"mdsmap_epoch":83,"osdmap_epoch":151,"osdmap_epoch_barrier":149,"uptime":7.6836034379999996} 2022-06-17T12:57:17.904 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 8805, "mdsmap_epoch": 83, "osdmap_epoch": 151, "osdmap_epoch_barrier": 149, "rank_uptime": 3.290054039, "state": "up:active", "uptime": 7.683603438, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:57:17.905 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:57:18.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.022+0000 7ff17efe5700 1 Processor -- start 2022-06-17T12:57:18.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.022+0000 7ff17efe5700 1 -- start start 2022-06-17T12:57:18.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.022+0000 7ff17efe5700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1780aef60 0x7ff1780b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:18.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.022+0000 7ff17efe5700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1780af4b0 con 0x7ff17813cb70 2022-06-17T12:57:18.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.022+0000 7ff17efe5700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff178142f50 con 0x7ff1780ba2e0 2022-06-17T12:57:18.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.022+0000 7ff17efe5700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff178143da0 con 0x7ff1780aef60 2022-06-17T12:57:18.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.022+0000 7ff17dfe3700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7ff17813cb70 0x7ff178143690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58326/0 (socket says 172.21.15.36:58326) 2022-06-17T12:57:18.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.022+0000 7ff17dfe3700 1 -- 172.21.15.36:0/2439878035 learned_addr learned my addr 172.21.15.36:0/2439878035 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:18.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.022+0000 7ff17d7e2700 1 --2- 172.21.15.36:0/2439878035 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1780aef60 0x7ff1780b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:18.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.022+0000 7ff17cfe1700 1 -- 172.21.15.36:0/2439878035 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2219140737 0 0) 0x7ff1780af4b0 con 0x7ff17813cb70 2022-06-17T12:57:18.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.022+0000 7ff17cfe1700 1 -- 172.21.15.36:0/2439878035 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff160003260 con 0x7ff17813cb70 2022-06-17T12:57:18.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.022+0000 7ff17cfe1700 1 -- 172.21.15.36:0/2439878035 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 892139598 0 0) 0x7ff160003260 con 0x7ff17813cb70 2022-06-17T12:57:18.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.022+0000 7ff17d7e2700 1 -- 172.21.15.36:0/2439878035 >> v1:172.21.15.70:6789/0 conn(0x7ff1780ba2e0 legacy=0x7ff1781442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:18.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.022+0000 7ff17d7e2700 1 -- 172.21.15.36:0/2439878035 >> v1:172.21.15.36:6789/0 conn(0x7ff17813cb70 legacy=0x7ff178143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:18.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.022+0000 7ff17d7e2700 1 -- 172.21.15.36:0/2439878035 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1781431f0 con 0x7ff1780aef60 2022-06-17T12:57:18.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.022+0000 7ff17d7e2700 1 --2- 172.21.15.36:0/2439878035 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1780aef60 0x7ff1780b9df0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7ff174001800 tx=0x7ff168006df0).ready entity=mon.2 client_cookie=fb4d824fce1a6f09 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:18.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.022+0000 7ff17cfe1700 1 -- 172.21.15.36:0/2439878035 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff174001660 con 0x7ff1780aef60 2022-06-17T12:57:18.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.022+0000 7ff17cfe1700 1 -- 172.21.15.36:0/2439878035 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff168000e90 con 0x7ff1780aef60 2022-06-17T12:57:18.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.022+0000 7ff17cfe1700 1 -- 172.21.15.36:0/2439878035 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff16800d4a0 con 0x7ff1780aef60 2022-06-17T12:57:18.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.026+0000 7ff17efe5700 1 -- 172.21.15.36:0/2439878035 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1780aef60 msgr2=0x7ff1780b9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:18.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.026+0000 7ff17efe5700 1 --2- 172.21.15.36:0/2439878035 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1780aef60 0x7ff1780b9df0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7ff174001800 tx=0x7ff168006df0).stop 2022-06-17T12:57:18.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.026+0000 7ff17efe5700 1 -- 172.21.15.36:0/2439878035 shutdown_connections 2022-06-17T12:57:18.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.026+0000 7ff17efe5700 1 --2- 172.21.15.36:0/2439878035 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1780aef60 0x7ff1780b9df0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.026+0000 7ff17efe5700 1 -- 172.21.15.36:0/2439878035 >> 172.21.15.36:0/2439878035 conn(0x7ff17805c550 msgr2=0x7ff1780b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:18.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.026+0000 7ff17efe5700 1 -- 172.21.15.36:0/2439878035 shutdown_connections 2022-06-17T12:57:18.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.026+0000 7ff17efe5700 1 -- 172.21.15.36:0/2439878035 wait complete. 2022-06-17T12:57:18.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.026+0000 7ff17efe5700 1 Processor -- start 2022-06-17T12:57:18.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.026+0000 7ff17efe5700 1 -- start start 2022-06-17T12:57:18.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.026+0000 7ff17efe5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1780aef60 0x7ff1780b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:18.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.026+0000 7ff17efe5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1780ba2e0 0x7ff1780b6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:18.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.026+0000 7ff17efe5700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff17813cb70 0x7ff1780b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:18.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.026+0000 7ff17efe5700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1780b3b80 con 0x7ff1780ba2e0 2022-06-17T12:57:18.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.026+0000 7ff17efe5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1780b1730 con 0x7ff1780aef60 2022-06-17T12:57:18.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.026+0000 7ff17efe5700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1780b19d0 con 0x7ff17813cb70 2022-06-17T12:57:18.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.026+0000 7ff17d7e2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1780ba2e0 0x7ff1780b6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:18.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.026+0000 7ff17d7e2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1780ba2e0 0x7ff1780b6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40806/0 (socket says 172.21.15.36:40806) 2022-06-17T12:57:18.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.026+0000 7ff17d7e2700 1 -- 172.21.15.36:0/1043555924 learned_addr learned my addr 172.21.15.36:0/1043555924 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:18.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.026+0000 7ff17e7e4700 1 --2- 172.21.15.36:0/1043555924 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff17813cb70 0x7ff1780b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:18.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.026+0000 7ff17dfe3700 1 --2- 172.21.15.36:0/1043555924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1780aef60 0x7ff1780b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:18.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.030+0000 7ff17d7e2700 1 -- 172.21.15.36:0/1043555924 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff17813cb70 msgr2=0x7ff1780b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:18.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.030+0000 7ff17d7e2700 1 --2- 172.21.15.36:0/1043555924 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff17813cb70 0x7ff1780b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.030+0000 7ff17d7e2700 1 -- 172.21.15.36:0/1043555924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1780aef60 msgr2=0x7ff1780b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:18.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.030+0000 7ff17d7e2700 1 --2- 172.21.15.36:0/1043555924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1780aef60 0x7ff1780b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.030+0000 7ff17d7e2700 1 -- 172.21.15.36:0/1043555924 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16800b180 con 0x7ff1780ba2e0 2022-06-17T12:57:18.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.030+0000 7ff17d7e2700 1 --2- 172.21.15.36:0/1043555924 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1780ba2e0 0x7ff1780b6b60 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 rx=0x7ff174001800 tx=0x7ff168006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:18.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.030+0000 7ff166ffd700 1 -- 172.21.15.36:0/1043555924 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff168012a60 con 0x7ff1780ba2e0 2022-06-17T12:57:18.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.030+0000 7ff166ffd700 1 -- 172.21.15.36:0/1043555924 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff168012bc0 con 0x7ff1780ba2e0 2022-06-17T12:57:18.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.030+0000 7ff17efe5700 1 -- 172.21.15.36:0/1043555924 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1780b1e70 con 0x7ff1780ba2e0 2022-06-17T12:57:18.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.030+0000 7ff166ffd700 1 -- 172.21.15.36:0/1043555924 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff16801d400 con 0x7ff1780ba2e0 2022-06-17T12:57:18.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.030+0000 7ff17efe5700 1 -- 172.21.15.36:0/1043555924 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff17813c1e0 con 0x7ff1780ba2e0 2022-06-17T12:57:18.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.030+0000 7ff17efe5700 1 -- 172.21.15.36:0/1043555924 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1780b1b90 con 0x7ff1780ba2e0 2022-06-17T12:57:18.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.030+0000 7ff166ffd700 1 -- 172.21.15.36:0/1043555924 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7ff168024030 con 0x7ff1780ba2e0 2022-06-17T12:57:18.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.030+0000 7ff166ffd700 1 --2- 172.21.15.36:0/1043555924 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff158053ad0 0x7ff158055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:18.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.030+0000 7ff166ffd700 1 -- 172.21.15.36:0/1043555924 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff16806c740 con 0x7ff1780ba2e0 2022-06-17T12:57:18.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.030+0000 7ff17dfe3700 1 --2- 172.21.15.36:0/1043555924 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff158053ad0 0x7ff158055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:18.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.034+0000 7ff17dfe3700 1 --2- 172.21.15.36:0/1043555924 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff158053ad0 0x7ff158055f90 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7ff17400d000 tx=0x7ff174000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:18.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.038+0000 7ff166ffd700 1 -- 172.21.15.36:0/1043555924 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff168043540 con 0x7ff1780ba2e0 2022-06-17T12:57:18.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.162+0000 7ff17efe5700 1 -- 172.21.15.36:0/1043555924 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1781449b0 con 0x7ff1780ba2e0 2022-06-17T12:57:18.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.162+0000 7ff166ffd700 1 -- 172.21.15.36:0/1043555924 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+16285 (secure 0 0 0) 0x7ff168022070 con 0x7ff1780ba2e0 2022-06-17T12:57:18.153 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:57:18.154 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":151,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:57:16.881090+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:57:13.415000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-06-17T12:57:14.234187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:57:18.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.166+0000 7ff17efe5700 1 -- 172.21.15.36:0/1043555924 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff158053ad0 msgr2=0x7ff158055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:18.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.166+0000 7ff17efe5700 1 --2- 172.21.15.36:0/1043555924 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff158053ad0 0x7ff158055f90 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7ff17400d000 tx=0x7ff174000bc0).stop 2022-06-17T12:57:18.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.166+0000 7ff17efe5700 1 -- 172.21.15.36:0/1043555924 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1780ba2e0 msgr2=0x7ff1780b6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:18.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.166+0000 7ff17efe5700 1 --2- 172.21.15.36:0/1043555924 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1780ba2e0 0x7ff1780b6b60 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 rx=0x7ff174001800 tx=0x7ff168006df0).stop 2022-06-17T12:57:18.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.166+0000 7ff17efe5700 1 -- 172.21.15.36:0/1043555924 shutdown_connections 2022-06-17T12:57:18.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.166+0000 7ff17efe5700 1 --2- 172.21.15.36:0/1043555924 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7ff158053ad0 0x7ff158055f90 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.166+0000 7ff17efe5700 1 --2- 172.21.15.36:0/1043555924 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff17813cb70 0x7ff1780b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.166+0000 7ff17efe5700 1 --2- 172.21.15.36:0/1043555924 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff1780ba2e0 0x7ff1780b6b60 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.166+0000 7ff17efe5700 1 --2- 172.21.15.36:0/1043555924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1780aef60 0x7ff1780b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.166+0000 7ff17efe5700 1 -- 172.21.15.36:0/1043555924 >> 172.21.15.36:0/1043555924 conn(0x7ff17805c550 msgr2=0x7ff1780b5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:18.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.166+0000 7ff17efe5700 1 -- 172.21.15.36:0/1043555924 shutdown_connections 2022-06-17T12:57:18.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.166+0000 7ff17efe5700 1 -- 172.21.15.36:0/1043555924 wait complete. 2022-06-17T12:57:18.165 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:57:13.415000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '149', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:57:14.234187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:57:18.167 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:57:18.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.306+0000 7f6169792700 1 Processor -- start 2022-06-17T12:57:18.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.306+0000 7f6169792700 1 -- start start 2022-06-17T12:57:18.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f6169792700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f61640b0420 0x7f616405f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:18.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f6169792700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f616405fc60 0x7f6164146750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:18.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f6169792700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61640b10a0 con 0x7f6164060260 2022-06-17T12:57:18.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f6169792700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6164144140 con 0x7f61640b0420 2022-06-17T12:57:18.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f6169792700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6164143ea0 con 0x7f616405fc60 2022-06-17T12:57:18.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f61637fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f6164060260 0x7f6164146c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58340/0 (socket says 172.21.15.36:58340) 2022-06-17T12:57:18.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f61637fe700 1 -- 172.21.15.36:0/228495740 learned_addr learned my addr 172.21.15.36:0/228495740 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:18.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f6162ffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f61640b0420 0x7f616405f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:18.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f61627fc700 1 --2- 172.21.15.36:0/228495740 >> v2:172.21.15.70:3300/0 conn(0x7f616405fc60 0x7f6164146750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:18.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f6161ffb700 1 -- 172.21.15.36:0/228495740 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2544736619 0 0) 0x7f61640b10a0 con 0x7f6164060260 2022-06-17T12:57:18.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f6161ffb700 1 -- 172.21.15.36:0/228495740 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6144003260 con 0x7f6164060260 2022-06-17T12:57:18.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f6162ffd700 1 -- 172.21.15.36:0/228495740 >> v2:172.21.15.70:3300/0 conn(0x7f616405fc60 msgr2=0x7f6164146750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:18.351 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f6161ffb700 1 -- 172.21.15.36:0/228495740 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3872921521 0 0) 0x7f6144003260 con 0x7f6164060260 2022-06-17T12:57:18.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f6162ffd700 1 --2- 172.21.15.36:0/228495740 >> v2:172.21.15.70:3300/0 conn(0x7f616405fc60 0x7f6164146750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f6162ffd700 1 -- 172.21.15.36:0/228495740 >> v1:172.21.15.36:6789/0 conn(0x7f6164060260 legacy=0x7f6164146c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:18.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f6162ffd700 1 -- 172.21.15.36:0/228495740 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6164143bc0 con 0x7f61640b0420 2022-06-17T12:57:18.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f6162ffd700 1 --2- 172.21.15.36:0/228495740 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61640b0420 0x7f616405f770 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7f615c001b80 tx=0x7f6154006df0).ready entity=mon.0 client_cookie=4eff7c9092994ae6 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:18.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f6161ffb700 1 -- 172.21.15.36:0/228495740 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f615c0019e0 con 0x7f61640b0420 2022-06-17T12:57:18.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f6161ffb700 1 -- 172.21.15.36:0/228495740 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6154000e90 con 0x7f61640b0420 2022-06-17T12:57:18.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f6161ffb700 1 -- 172.21.15.36:0/228495740 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6154012460 con 0x7f61640b0420 2022-06-17T12:57:18.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f6169792700 1 -- 172.21.15.36:0/228495740 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61640b0420 msgr2=0x7f616405f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:18.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f6169792700 1 --2- 172.21.15.36:0/228495740 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61640b0420 0x7f616405f770 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7f615c001b80 tx=0x7f6154006df0).stop 2022-06-17T12:57:18.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f6169792700 1 -- 172.21.15.36:0/228495740 shutdown_connections 2022-06-17T12:57:18.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f6169792700 1 --2- 172.21.15.36:0/228495740 >> v2:172.21.15.70:3300/0 conn(0x7f616405fc60 0x7f6164146750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f6169792700 1 --2- 172.21.15.36:0/228495740 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f61640b0420 0x7f616405f770 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f6169792700 1 -- 172.21.15.36:0/228495740 >> 172.21.15.36:0/228495740 conn(0x7f616405c550 msgr2=0x7f616405c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:18.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.310+0000 7f6169792700 1 -- 172.21.15.36:0/228495740 shutdown_connections 2022-06-17T12:57:18.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.314+0000 7f6169792700 1 -- 172.21.15.36:0/228495740 wait complete. 2022-06-17T12:57:18.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.314+0000 7f6169792700 1 Processor -- start 2022-06-17T12:57:18.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.314+0000 7f6169792700 1 -- start start 2022-06-17T12:57:18.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.314+0000 7f6169792700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f616405fc60 0x7f616413fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:18.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.314+0000 7f6169792700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6164060260 0x7f6164140330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:18.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.314+0000 7f6169792700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61640b0420 0x7f6164156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:18.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.314+0000 7f6169792700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616413dd90 con 0x7f6164060260 2022-06-17T12:57:18.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.314+0000 7f6169792700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6164145410 con 0x7f61640b0420 2022-06-17T12:57:18.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.314+0000 7f6169792700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61641456b0 con 0x7f616405fc60 2022-06-17T12:57:18.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.314+0000 7f61627fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6164060260 0x7f6164140330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:18.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.314+0000 7f61627fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6164060260 0x7f6164140330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40818/0 (socket says 172.21.15.36:40818) 2022-06-17T12:57:18.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.314+0000 7f61627fc700 1 -- 172.21.15.36:0/3145775638 learned_addr learned my addr 172.21.15.36:0/3145775638 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:18.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.314+0000 7f61637fe700 1 --2- 172.21.15.36:0/3145775638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61640b0420 0x7f6164156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:18.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.314+0000 7f6162ffd700 1 --2- 172.21.15.36:0/3145775638 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f616405fc60 0x7f616413fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:18.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.314+0000 7f61627fc700 1 -- 172.21.15.36:0/3145775638 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f616405fc60 msgr2=0x7f616413fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:18.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.314+0000 7f61627fc700 1 --2- 172.21.15.36:0/3145775638 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f616405fc60 0x7f616413fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.314+0000 7f61627fc700 1 -- 172.21.15.36:0/3145775638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61640b0420 msgr2=0x7f6164156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:18.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.318+0000 7f61627fc700 1 --2- 172.21.15.36:0/3145775638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61640b0420 0x7f6164156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.318+0000 7f61627fc700 1 -- 172.21.15.36:0/3145775638 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615400b180 con 0x7f6164060260 2022-06-17T12:57:18.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.318+0000 7f61627fc700 1 --2- 172.21.15.36:0/3145775638 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6164060260 0x7f6164140330 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 rx=0x7f614c00c000 tx=0x7f614c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:18.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.318+0000 7f615bfff700 1 -- 172.21.15.36:0/3145775638 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f614c010780 con 0x7f6164060260 2022-06-17T12:57:18.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.318+0000 7f615bfff700 1 -- 172.21.15.36:0/3145775638 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614c0108e0 con 0x7f6164060260 2022-06-17T12:57:18.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.318+0000 7f615bfff700 1 -- 172.21.15.36:0/3145775638 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f614c010c40 con 0x7f6164060260 2022-06-17T12:57:18.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.318+0000 7f6169792700 1 -- 172.21.15.36:0/3145775638 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6164144fe0 con 0x7f6164060260 2022-06-17T12:57:18.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.318+0000 7f6169792700 1 -- 172.21.15.36:0/3145775638 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616413f720 con 0x7f6164060260 2022-06-17T12:57:18.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.318+0000 7f6169792700 1 -- 172.21.15.36:0/3145775638 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6164057410 con 0x7f6164060260 2022-06-17T12:57:18.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.318+0000 7f615bfff700 1 -- 172.21.15.36:0/3145775638 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f614c01c460 con 0x7f6164060260 2022-06-17T12:57:18.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.318+0000 7f615bfff700 1 --2- 172.21.15.36:0/3145775638 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6148053a40 0x7f6148055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:18.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.318+0000 7f615bfff700 1 -- 172.21.15.36:0/3145775638 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f614c06c370 con 0x7f6164060260 2022-06-17T12:57:18.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.318+0000 7f6162ffd700 1 --2- 172.21.15.36:0/3145775638 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6148053a40 0x7f6148055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:18.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.318+0000 7f6162ffd700 1 --2- 172.21.15.36:0/3145775638 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6148053a40 0x7f6148055f00 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f615401a000 tx=0x7f6154014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:18.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.330+0000 7f615bfff700 1 -- 172.21.15.36:0/3145775638 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f614c043110 con 0x7f6164060260 2022-06-17T12:57:18.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.454+0000 7f6169792700 1 -- 172.21.15.36:0/3145775638 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6164145870 con 0x7f6164060260 2022-06-17T12:57:18.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.458+0000 7f615bfff700 1 -- 172.21.15.36:0/3145775638 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v151) v1 ==== 110+0+10 (secure 0 0 0) 0x7f614c046730 con 0x7f6164060260 2022-06-17T12:57:18.455 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:57:18.462 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.458+0000 7f6169792700 1 -- 172.21.15.36:0/3145775638 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6148053a40 msgr2=0x7f6148055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:18.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.458+0000 7f6169792700 1 --2- 172.21.15.36:0/3145775638 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6148053a40 0x7f6148055f00 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f615401a000 tx=0x7f6154014000).stop 2022-06-17T12:57:18.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.458+0000 7f6169792700 1 -- 172.21.15.36:0/3145775638 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6164060260 msgr2=0x7f6164140330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:18.463 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.458+0000 7f6169792700 1 --2- 172.21.15.36:0/3145775638 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6164060260 0x7f6164140330 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 rx=0x7f614c00c000 tx=0x7f614c006df0).stop 2022-06-17T12:57:18.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.458+0000 7f6169792700 1 -- 172.21.15.36:0/3145775638 shutdown_connections 2022-06-17T12:57:18.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.458+0000 7f6169792700 1 --2- 172.21.15.36:0/3145775638 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6148053a40 0x7f6148055f00 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.458+0000 7f6169792700 1 --2- 172.21.15.36:0/3145775638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61640b0420 0x7f6164156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.458+0000 7f6169792700 1 --2- 172.21.15.36:0/3145775638 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6164060260 0x7f6164140330 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.464 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.458+0000 7f6169792700 1 --2- 172.21.15.36:0/3145775638 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f616405fc60 0x7f616413fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.458+0000 7f6169792700 1 -- 172.21.15.36:0/3145775638 >> 172.21.15.36:0/3145775638 conn(0x7f616405c550 msgr2=0x7f6164142de0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:18.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.458+0000 7f6169792700 1 -- 172.21.15.36:0/3145775638 shutdown_connections 2022-06-17T12:57:18.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.458+0000 7f6169792700 1 -- 172.21.15.36:0/3145775638 wait complete. 2022-06-17T12:57:18.466 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:57:18.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.594+0000 7f3043fb5700 1 Processor -- start 2022-06-17T12:57:18.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.594+0000 7f3043fb5700 1 -- start start 2022-06-17T12:57:18.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.594+0000 7f3043fb5700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f303c0b4120 0x7f303c146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:18.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.594+0000 7f3043fb5700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f303c147270 0x7f303c140100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:18.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.594+0000 7f3043fb5700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f303c0b68d0 con 0x7f303c0b64f0 2022-06-17T12:57:18.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.594+0000 7f3043fb5700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f303c1476b0 con 0x7f303c147270 2022-06-17T12:57:18.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.594+0000 7f3043fb5700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f303c0b6e20 con 0x7f303c0b4120 2022-06-17T12:57:18.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.594+0000 7f3042552700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f303c147270 0x7f303c140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:18.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.594+0000 7f3041550700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f303c0b4120 0x7f303c146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:18.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.594+0000 7f3042552700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f303c147270 0x7f303c140100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:32978/0 (socket says 172.21.15.36:32978) 2022-06-17T12:57:18.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.594+0000 7f3042552700 1 -- 172.21.15.36:0/1070945967 learned_addr learned my addr 172.21.15.36:0/1070945967 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:18.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.598+0000 7f3042552700 1 -- 172.21.15.36:0/1070945967 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f303c0b4120 msgr2=0x7f303c146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:18.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.598+0000 7f3042552700 1 --2- 172.21.15.36:0/1070945967 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f303c0b4120 0x7f303c146d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.598+0000 7f3042552700 1 -- 172.21.15.36:0/1070945967 >> v1:172.21.15.70:6789/0 conn(0x7f303c0b64f0 legacy=0x7f303c143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:18.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.598+0000 7f3042552700 1 -- 172.21.15.36:0/1070945967 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303c1468e0 con 0x7f303c147270 2022-06-17T12:57:18.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.598+0000 7f3042552700 1 --2- 172.21.15.36:0/1070945967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f303c147270 0x7f303c140100 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f30340032f0 tx=0x7f3034009290).ready entity=mon.1 client_cookie=c5d0a537f2dec596 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:18.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.598+0000 7f3040d4f700 1 -- 172.21.15.36:0/1070945967 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f303400b8a0 con 0x7f303c147270 2022-06-17T12:57:18.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.598+0000 7f3040d4f700 1 -- 172.21.15.36:0/1070945967 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303400ba00 con 0x7f303c147270 2022-06-17T12:57:18.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.598+0000 7f3040d4f700 1 -- 172.21.15.36:0/1070945967 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f303400bb70 con 0x7f303c147270 2022-06-17T12:57:18.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.598+0000 7f3043fb5700 1 -- 172.21.15.36:0/1070945967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f303c147270 msgr2=0x7f303c140100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:18.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.598+0000 7f3043fb5700 1 --2- 172.21.15.36:0/1070945967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f303c147270 0x7f303c140100 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f30340032f0 tx=0x7f3034009290).stop 2022-06-17T12:57:18.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.598+0000 7f3043fb5700 1 -- 172.21.15.36:0/1070945967 shutdown_connections 2022-06-17T12:57:18.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.598+0000 7f3043fb5700 1 --2- 172.21.15.36:0/1070945967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f303c147270 0x7f303c140100 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.598+0000 7f3043fb5700 1 --2- 172.21.15.36:0/1070945967 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f303c0b4120 0x7f303c146d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.598+0000 7f3043fb5700 1 -- 172.21.15.36:0/1070945967 >> 172.21.15.36:0/1070945967 conn(0x7f303c0620f0 msgr2=0x7f303c062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:18.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.598+0000 7f3043fb5700 1 -- 172.21.15.36:0/1070945967 shutdown_connections 2022-06-17T12:57:18.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.598+0000 7f3043fb5700 1 -- 172.21.15.36:0/1070945967 wait complete. 2022-06-17T12:57:18.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.602+0000 7f3043fb5700 1 Processor -- start 2022-06-17T12:57:18.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.602+0000 7f3043fb5700 1 -- start start 2022-06-17T12:57:18.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.602+0000 7f3043fb5700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f303c0b4120 0x7f303c142c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:18.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.602+0000 7f3043fb5700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f303c0b64f0 0x7f303c153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:18.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.602+0000 7f3043fb5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f303c147270 0x7f303c0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:18.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.602+0000 7f3043fb5700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303c1131c0 con 0x7f303c0b64f0 2022-06-17T12:57:18.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.602+0000 7f3043fb5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303c141a60 con 0x7f303c147270 2022-06-17T12:57:18.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.602+0000 7f3043fb5700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f303c141d00 con 0x7f303c0b4120 2022-06-17T12:57:18.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.602+0000 7f3041550700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f303c0b64f0 0x7f303c153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:18.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.602+0000 7f3042552700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f303c147270 0x7f303c0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:18.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.602+0000 7f3041550700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f303c0b64f0 0x7f303c153080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40832/0 (socket says 172.21.15.36:40832) 2022-06-17T12:57:18.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.602+0000 7f3041d51700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f303c0b4120 0x7f303c142c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:18.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.602+0000 7f3041550700 1 -- 172.21.15.36:0/1145726691 learned_addr learned my addr 172.21.15.36:0/1145726691 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:18.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.602+0000 7f3041550700 1 -- 172.21.15.36:0/1145726691 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f303c0b4120 msgr2=0x7f303c142c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:18.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.602+0000 7f3041550700 1 --2- 172.21.15.36:0/1145726691 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f303c0b4120 0x7f303c142c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.602+0000 7f3041550700 1 -- 172.21.15.36:0/1145726691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f303c147270 msgr2=0x7f303c0b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:18.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.602+0000 7f3041550700 1 --2- 172.21.15.36:0/1145726691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f303c147270 0x7f303c0b4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.602+0000 7f3041550700 1 -- 172.21.15.36:0/1145726691 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3034009530 con 0x7f303c0b64f0 2022-06-17T12:57:18.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.606+0000 7f3041550700 1 --2- 172.21.15.36:0/1145726691 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f303c0b64f0 0x7f303c153080 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7f302c00c000 tx=0x7f302c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:18.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.606+0000 7f302affd700 1 -- 172.21.15.36:0/1145726691 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f302c010780 con 0x7f303c0b64f0 2022-06-17T12:57:18.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.606+0000 7f302affd700 1 -- 172.21.15.36:0/1145726691 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302c0108e0 con 0x7f303c0b64f0 2022-06-17T12:57:18.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.606+0000 7f3043fb5700 1 -- 172.21.15.36:0/1145726691 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303c13fed0 con 0x7f303c0b64f0 2022-06-17T12:57:18.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.606+0000 7f302affd700 1 -- 172.21.15.36:0/1145726691 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f302c010c40 con 0x7f303c0b64f0 2022-06-17T12:57:18.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.606+0000 7f3043fb5700 1 -- 172.21.15.36:0/1145726691 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f303c112c60 con 0x7f303c0b64f0 2022-06-17T12:57:18.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.606+0000 7f3043fb5700 1 -- 172.21.15.36:0/1145726691 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f303c140060 con 0x7f303c0b64f0 2022-06-17T12:57:18.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.606+0000 7f302affd700 1 -- 172.21.15.36:0/1145726691 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f302c01c460 con 0x7f303c0b64f0 2022-06-17T12:57:18.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.610+0000 7f302affd700 1 --2- 172.21.15.36:0/1145726691 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f301c0539b0 0x7f301c055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:18.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.610+0000 7f302affd700 1 -- 172.21.15.36:0/1145726691 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f302c06c3a0 con 0x7f303c0b64f0 2022-06-17T12:57:18.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.610+0000 7f3041d51700 1 --2- 172.21.15.36:0/1145726691 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f301c0539b0 0x7f301c055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:18.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.610+0000 7f3041d51700 1 --2- 172.21.15.36:0/1145726691 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f301c0539b0 0x7f301c055e70 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f303800d000 tx=0x7f3038000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:18.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.614+0000 7f302affd700 1 -- 172.21.15.36:0/1145726691 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f302c043140 con 0x7f303c0b64f0 2022-06-17T12:57:18.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.742+0000 7f3043fb5700 1 -- 172.21.15.36:0/1145726691 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f303c0b3a60 con 0x7f303c0b64f0 2022-06-17T12:57:18.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.746+0000 7f302affd700 1 -- 172.21.15.36:0/1145726691 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v151) v1 ==== 92+0+10 (secure 0 0 0) 0x7f302c046760 con 0x7f303c0b64f0 2022-06-17T12:57:18.732 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:57:18.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.746+0000 7f3043fb5700 1 -- 172.21.15.36:0/1145726691 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f301c0539b0 msgr2=0x7f301c055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:18.734 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.746+0000 7f3043fb5700 1 --2- 172.21.15.36:0/1145726691 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f301c0539b0 0x7f301c055e70 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f303800d000 tx=0x7f3038000bc0).stop 2022-06-17T12:57:18.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.746+0000 7f3043fb5700 1 -- 172.21.15.36:0/1145726691 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f303c0b64f0 msgr2=0x7f303c153080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:18.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.746+0000 7f3043fb5700 1 --2- 172.21.15.36:0/1145726691 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f303c0b64f0 0x7f303c153080 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7f302c00c000 tx=0x7f302c006df0).stop 2022-06-17T12:57:18.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.746+0000 7f3043fb5700 1 -- 172.21.15.36:0/1145726691 shutdown_connections 2022-06-17T12:57:18.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.746+0000 7f3043fb5700 1 --2- 172.21.15.36:0/1145726691 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f301c0539b0 0x7f301c055e70 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.746+0000 7f3043fb5700 1 --2- 172.21.15.36:0/1145726691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f303c147270 0x7f303c0b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.746+0000 7f3043fb5700 1 --2- 172.21.15.36:0/1145726691 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f303c0b64f0 0x7f303c153080 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.746+0000 7f3043fb5700 1 --2- 172.21.15.36:0/1145726691 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f303c0b4120 0x7f303c142c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.746+0000 7f3043fb5700 1 -- 172.21.15.36:0/1145726691 >> 172.21.15.36:0/1145726691 conn(0x7f303c0620f0 msgr2=0x7f303c0b7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:18.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.746+0000 7f3043fb5700 1 -- 172.21.15.36:0/1145726691 shutdown_connections 2022-06-17T12:57:18.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.746+0000 7f3043fb5700 1 -- 172.21.15.36:0/1145726691 wait complete. 2022-06-17T12:57:18.757 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:57:18.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.890+0000 7f60ddb21700 1 Processor -- start 2022-06-17T12:57:18.887 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.890+0000 7f60ddb21700 1 -- start start 2022-06-17T12:57:18.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.890+0000 7f60ddb21700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f60d80b45c0 0x7f60d80b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:18.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.890+0000 7f60ddb21700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60d80b4160 con 0x7f60d80b4f00 2022-06-17T12:57:18.888 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.890+0000 7f60ddb21700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60d80b9c40 con 0x7f60d80b9860 2022-06-17T12:57:18.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.890+0000 7f60ddb21700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60d80627d0 con 0x7f60d80b45c0 2022-06-17T12:57:18.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.890+0000 7f60d7fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f60d80b45c0 0x7f60d80b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:18.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.890+0000 7f60dd320700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f60d80b4f00 0x7f60d80b5dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58372/0 (socket says 172.21.15.36:58372) 2022-06-17T12:57:18.889 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.890+0000 7f60dd320700 1 -- 172.21.15.36:0/1064532649 learned_addr learned my addr 172.21.15.36:0/1064532649 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:18.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.890+0000 7f60d7fff700 1 -- 172.21.15.36:0/1064532649 >> v1:172.21.15.70:6789/0 conn(0x7f60d80b9860 legacy=0x7f60d8143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:18.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.890+0000 7f60d7fff700 1 -- 172.21.15.36:0/1064532649 >> v1:172.21.15.36:6789/0 conn(0x7f60d80b4f00 legacy=0x7f60d80b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:18.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.890+0000 7f60d77fe700 1 -- 172.21.15.36:0/1064532649 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252837373 0 0) 0x7f60d80b4160 con 0x7f60d80b4f00 2022-06-17T12:57:18.890 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.890+0000 7f60d7fff700 1 -- 172.21.15.36:0/1064532649 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d8062a70 con 0x7f60d80b45c0 2022-06-17T12:57:18.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.890+0000 7f60d7fff700 1 --2- 172.21.15.36:0/1064532649 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f60d80b45c0 0x7f60d80b4a10 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0x7f60d80ba5d0 tx=0x7f60c00091d0).ready entity=mon.0 client_cookie=17e00eee8e68b695 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:18.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.890+0000 7f60d77fe700 1 -- 172.21.15.36:0/1064532649 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60c000b860 con 0x7f60d80b45c0 2022-06-17T12:57:18.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.890+0000 7f60d77fe700 1 -- 172.21.15.36:0/1064532649 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c000b9c0 con 0x7f60d80b45c0 2022-06-17T12:57:18.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.890+0000 7f60d77fe700 1 -- 172.21.15.36:0/1064532649 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60c000bb30 con 0x7f60d80b45c0 2022-06-17T12:57:18.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.890+0000 7f60ddb21700 1 -- 172.21.15.36:0/1064532649 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f60d80b45c0 msgr2=0x7f60d80b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:18.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.890+0000 7f60ddb21700 1 --2- 172.21.15.36:0/1064532649 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f60d80b45c0 0x7f60d80b4a10 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0x7f60d80ba5d0 tx=0x7f60c00091d0).stop 2022-06-17T12:57:18.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.894+0000 7f60ddb21700 1 -- 172.21.15.36:0/1064532649 shutdown_connections 2022-06-17T12:57:18.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.894+0000 7f60ddb21700 1 --2- 172.21.15.36:0/1064532649 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f60d80b45c0 0x7f60d80b4a10 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.894+0000 7f60ddb21700 1 -- 172.21.15.36:0/1064532649 >> 172.21.15.36:0/1064532649 conn(0x7f60d805c550 msgr2=0x7f60d8060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:18.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.894+0000 7f60ddb21700 1 -- 172.21.15.36:0/1064532649 shutdown_connections 2022-06-17T12:57:18.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.894+0000 7f60ddb21700 1 -- 172.21.15.36:0/1064532649 wait complete. 2022-06-17T12:57:18.893 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.894+0000 7f60ddb21700 1 Processor -- start 2022-06-17T12:57:18.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.894+0000 7f60ddb21700 1 -- start start 2022-06-17T12:57:18.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.894+0000 7f60ddb21700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f60d80b45c0 0x7f60d81404b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:18.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.894+0000 7f60ddb21700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f60d80b4f00 0x7f60d813d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:18.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.894+0000 7f60ddb21700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60d80b9860 0x7f60d80af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:18.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.894+0000 7f60ddb21700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d8145e90 con 0x7f60d80b4f00 2022-06-17T12:57:18.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.894+0000 7f60ddb21700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d8152f00 con 0x7f60d80b9860 2022-06-17T12:57:18.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.894+0000 7f60ddb21700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d8147000 con 0x7f60d80b45c0 2022-06-17T12:57:18.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.894+0000 7f60d7fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f60d80b4f00 0x7f60d813d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:18.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.894+0000 7f60dcb1f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f60d80b45c0 0x7f60d81404b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:18.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.894+0000 7f60d7fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f60d80b4f00 0x7f60d813d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40846/0 (socket says 172.21.15.36:40846) 2022-06-17T12:57:18.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.894+0000 7f60d7fff700 1 -- 172.21.15.36:0/2078680960 learned_addr learned my addr 172.21.15.36:0/2078680960 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:18.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.894+0000 7f60dd320700 1 --2- 172.21.15.36:0/2078680960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60d80b9860 0x7f60d80af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:18.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.894+0000 7f60d7fff700 1 -- 172.21.15.36:0/2078680960 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f60d80b45c0 msgr2=0x7f60d81404b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:18.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.894+0000 7f60d7fff700 1 --2- 172.21.15.36:0/2078680960 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f60d80b45c0 0x7f60d81404b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.894+0000 7f60d7fff700 1 -- 172.21.15.36:0/2078680960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60d80b9860 msgr2=0x7f60d80af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:18.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.894+0000 7f60d7fff700 1 --2- 172.21.15.36:0/2078680960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60d80b9860 0x7f60d80af7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:18.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.894+0000 7f60d7fff700 1 -- 172.21.15.36:0/2078680960 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c0009470 con 0x7f60d80b4f00 2022-06-17T12:57:18.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.898+0000 7f60d7fff700 1 --2- 172.21.15.36:0/2078680960 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f60d80b4f00 0x7f60d813d850 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7f60d80ba5d0 tx=0x7f60c0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:18.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.898+0000 7f60d5ffb700 1 -- 172.21.15.36:0/2078680960 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60c00032f0 con 0x7f60d80b4f00 2022-06-17T12:57:18.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.898+0000 7f60d5ffb700 1 -- 172.21.15.36:0/2078680960 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c001f3f0 con 0x7f60d80b4f00 2022-06-17T12:57:18.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.898+0000 7f60d5ffb700 1 -- 172.21.15.36:0/2078680960 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60c001f750 con 0x7f60d80b4f00 2022-06-17T12:57:18.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.898+0000 7f60ddb21700 1 -- 172.21.15.36:0/2078680960 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d8152bf0 con 0x7f60d80b4f00 2022-06-17T12:57:18.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.898+0000 7f60ddb21700 1 -- 172.21.15.36:0/2078680960 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d80af3c0 con 0x7f60d80b4f00 2022-06-17T12:57:18.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.902+0000 7f60ddb21700 1 -- 172.21.15.36:0/2078680960 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d8146e50 con 0x7f60d80b4f00 2022-06-17T12:57:18.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.902+0000 7f60d5ffb700 1 -- 172.21.15.36:0/2078680960 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f60c000f070 con 0x7f60d80b4f00 2022-06-17T12:57:18.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.902+0000 7f60d5ffb700 1 --2- 172.21.15.36:0/2078680960 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f60bc053a40 0x7f60bc055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:18.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.902+0000 7f60d5ffb700 1 -- 172.21.15.36:0/2078680960 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f60c006ca80 con 0x7f60d80b4f00 2022-06-17T12:57:18.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.902+0000 7f60dcb1f700 1 --2- 172.21.15.36:0/2078680960 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f60bc053a40 0x7f60bc055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:18.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.902+0000 7f60dcb1f700 1 --2- 172.21.15.36:0/2078680960 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f60bc053a40 0x7f60bc055f00 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f60c800d000 tx=0x7f60c8007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:18.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:18.906+0000 7f60d5ffb700 1 -- 172.21.15.36:0/2078680960 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f60c00437e0 con 0x7f60d80b4f00 2022-06-17T12:57:19.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.038+0000 7f60ddb21700 1 -- 172.21.15.36:0/2078680960 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f60d81471c0 con 0x7f60d80b4f00 2022-06-17T12:57:19.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.038+0000 7f60d5ffb700 1 -- 172.21.15.36:0/2078680960 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v151) v1 ==== 104+0+10 (secure 0 0 0) 0x7f60c00150a0 con 0x7f60d80b4f00 2022-06-17T12:57:19.022 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:57:19.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.038+0000 7f60ddb21700 1 -- 172.21.15.36:0/2078680960 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f60bc053a40 msgr2=0x7f60bc055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:19.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.038+0000 7f60ddb21700 1 --2- 172.21.15.36:0/2078680960 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f60bc053a40 0x7f60bc055f00 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f60c800d000 tx=0x7f60c8007b90).stop 2022-06-17T12:57:19.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.038+0000 7f60ddb21700 1 -- 172.21.15.36:0/2078680960 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f60d80b4f00 msgr2=0x7f60d813d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:19.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.038+0000 7f60ddb21700 1 --2- 172.21.15.36:0/2078680960 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f60d80b4f00 0x7f60d813d850 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7f60d80ba5d0 tx=0x7f60c0016000).stop 2022-06-17T12:57:19.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.038+0000 7f60ddb21700 1 -- 172.21.15.36:0/2078680960 shutdown_connections 2022-06-17T12:57:19.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.038+0000 7f60ddb21700 1 --2- 172.21.15.36:0/2078680960 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f60bc053a40 0x7f60bc055f00 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:19.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.038+0000 7f60ddb21700 1 --2- 172.21.15.36:0/2078680960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60d80b9860 0x7f60d80af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:19.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.038+0000 7f60ddb21700 1 --2- 172.21.15.36:0/2078680960 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f60d80b4f00 0x7f60d813d850 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:19.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.038+0000 7f60ddb21700 1 --2- 172.21.15.36:0/2078680960 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f60d80b45c0 0x7f60d81404b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:19.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.038+0000 7f60ddb21700 1 -- 172.21.15.36:0/2078680960 >> 172.21.15.36:0/2078680960 conn(0x7f60d805c550 msgr2=0x7f60d813cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:19.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.038+0000 7f60ddb21700 1 -- 172.21.15.36:0/2078680960 shutdown_connections 2022-06-17T12:57:19.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.038+0000 7f60ddb21700 1 -- 172.21.15.36:0/2078680960 wait complete. 2022-06-17T12:57:19.036 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:57:19.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.182+0000 7fa67a624700 1 Processor -- start 2022-06-17T12:57:19.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.182+0000 7fa67a624700 1 -- start start 2022-06-17T12:57:19.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.182+0000 7fa67a624700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6740acee0 0x7fa6740b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:19.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.182+0000 7fa67a624700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa674144240 0x7fa6741446d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:19.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.182+0000 7fa67a624700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6740b1900 con 0x7fa6740acb00 2022-06-17T12:57:19.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.182+0000 7fa67a624700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6740ad360 con 0x7fa674144240 2022-06-17T12:57:19.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.182+0000 7fa67a624700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6740b0ad0 con 0x7fa6740acee0 2022-06-17T12:57:19.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.182+0000 7fa678bc1700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa674144240 0x7fa6741446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:19.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.182+0000 7fa678bc1700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa674144240 0x7fa6741446d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40856/0 (socket says 172.21.15.36:40856) 2022-06-17T12:57:19.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.182+0000 7fa678bc1700 1 -- 172.21.15.36:0/1594954058 learned_addr learned my addr 172.21.15.36:0/1594954058 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:19.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.182+0000 7fa6737fe700 1 --2- 172.21.15.36:0/1594954058 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6740acee0 0x7fa6740b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:19.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.182+0000 7fa678bc1700 1 -- 172.21.15.36:0/1594954058 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6740acee0 msgr2=0x7fa6740b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:19.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.182+0000 7fa678bc1700 1 --2- 172.21.15.36:0/1594954058 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6740acee0 0x7fa6740b05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:19.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.182+0000 7fa678bc1700 1 -- 172.21.15.36:0/1594954058 >> v1:172.21.15.70:6789/0 conn(0x7fa6740acb00 legacy=0x7fa674147480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:19.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.182+0000 7fa678bc1700 1 -- 172.21.15.36:0/1594954058 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6740af370 con 0x7fa674144240 2022-06-17T12:57:19.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.182+0000 7fa678bc1700 1 --2- 172.21.15.36:0/1594954058 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa674144240 0x7fa6741446d0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 rx=0x7fa66c002f70 tx=0x7fa66c006eb0).ready entity=mon.0 client_cookie=60bfdf0826993bcc server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:19.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.182+0000 7fa672ffd700 1 -- 172.21.15.36:0/1594954058 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa66c00d570 con 0x7fa674144240 2022-06-17T12:57:19.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.182+0000 7fa672ffd700 1 -- 172.21.15.36:0/1594954058 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66c000e90 con 0x7fa674144240 2022-06-17T12:57:19.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.182+0000 7fa672ffd700 1 -- 172.21.15.36:0/1594954058 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa66c00d900 con 0x7fa674144240 2022-06-17T12:57:19.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.186+0000 7fa67a624700 1 -- 172.21.15.36:0/1594954058 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa674144240 msgr2=0x7fa6741446d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:19.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.186+0000 7fa67a624700 1 --2- 172.21.15.36:0/1594954058 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa674144240 0x7fa6741446d0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 rx=0x7fa66c002f70 tx=0x7fa66c006eb0).stop 2022-06-17T12:57:19.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.186+0000 7fa67a624700 1 -- 172.21.15.36:0/1594954058 shutdown_connections 2022-06-17T12:57:19.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.186+0000 7fa67a624700 1 --2- 172.21.15.36:0/1594954058 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6740acee0 0x7fa6740b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:19.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.186+0000 7fa67a624700 1 --2- 172.21.15.36:0/1594954058 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa674144240 0x7fa6741446d0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:19.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.186+0000 7fa67a624700 1 -- 172.21.15.36:0/1594954058 >> 172.21.15.36:0/1594954058 conn(0x7fa67405c550 msgr2=0x7fa6740b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:19.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.186+0000 7fa67a624700 1 -- 172.21.15.36:0/1594954058 shutdown_connections 2022-06-17T12:57:19.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.186+0000 7fa67a624700 1 -- 172.21.15.36:0/1594954058 wait complete. 2022-06-17T12:57:19.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.186+0000 7fa67a624700 1 Processor -- start 2022-06-17T12:57:19.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.186+0000 7fa67a624700 1 -- start start 2022-06-17T12:57:19.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.190+0000 7fa67a624700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6740acb00 0x7fa67414aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:19.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.190+0000 7fa67a624700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6740acee0 0x7fa6741455e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:19.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.190+0000 7fa67a624700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa674144240 0x7fa674145ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:19.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.190+0000 7fa67a624700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa674147640 con 0x7fa674144240 2022-06-17T12:57:19.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.190+0000 7fa67a624700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6741478e0 con 0x7fa6740acb00 2022-06-17T12:57:19.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.190+0000 7fa67a624700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6740ac490 con 0x7fa6740acee0 2022-06-17T12:57:19.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.190+0000 7fa678bc1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa674144240 0x7fa674145ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:19.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.190+0000 7fa678bc1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa674144240 0x7fa674145ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40862/0 (socket says 172.21.15.36:40862) 2022-06-17T12:57:19.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.190+0000 7fa678bc1700 1 -- 172.21.15.36:0/3911099001 learned_addr learned my addr 172.21.15.36:0/3911099001 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:19.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.190+0000 7fa6737fe700 1 --2- 172.21.15.36:0/3911099001 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6740acee0 0x7fa6741455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:19.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.190+0000 7fa673fff700 1 --2- 172.21.15.36:0/3911099001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6740acb00 0x7fa67414aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:19.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.190+0000 7fa678bc1700 1 -- 172.21.15.36:0/3911099001 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6740acee0 msgr2=0x7fa6741455e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:19.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.190+0000 7fa678bc1700 1 --2- 172.21.15.36:0/3911099001 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6740acee0 0x7fa6741455e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:19.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.190+0000 7fa678bc1700 1 -- 172.21.15.36:0/3911099001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6740acb00 msgr2=0x7fa67414aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:19.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.190+0000 7fa678bc1700 1 --2- 172.21.15.36:0/3911099001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6740acb00 0x7fa67414aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:19.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.190+0000 7fa678bc1700 1 -- 172.21.15.36:0/3911099001 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66c00b1e0 con 0x7fa674144240 2022-06-17T12:57:19.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.190+0000 7fa673fff700 1 --2- 172.21.15.36:0/3911099001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6740acb00 0x7fa67414aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:57:19.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.190+0000 7fa678bc1700 1 --2- 172.21.15.36:0/3911099001 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa674144240 0x7fa674145ad0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 rx=0x7fa66c002f70 tx=0x7fa66c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:19.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.190+0000 7fa6717fa700 1 -- 172.21.15.36:0/3911099001 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa66c0117e0 con 0x7fa674144240 2022-06-17T12:57:19.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.190+0000 7fa6717fa700 1 -- 172.21.15.36:0/3911099001 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66c011940 con 0x7fa674144240 2022-06-17T12:57:19.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.190+0000 7fa67a624700 1 -- 172.21.15.36:0/3911099001 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6740656f0 con 0x7fa674144240 2022-06-17T12:57:19.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.190+0000 7fa6717fa700 1 -- 172.21.15.36:0/3911099001 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa66c011ca0 con 0x7fa674144240 2022-06-17T12:57:19.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.190+0000 7fa67a624700 1 -- 172.21.15.36:0/3911099001 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6740659d0 con 0x7fa674144240 2022-06-17T12:57:19.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.190+0000 7fa67a624700 1 -- 172.21.15.36:0/3911099001 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa674065b60 con 0x7fa674144240 2022-06-17T12:57:19.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.194+0000 7fa6717fa700 1 -- 172.21.15.36:0/3911099001 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fa66c025030 con 0x7fa674144240 2022-06-17T12:57:19.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.194+0000 7fa6717fa700 1 --2- 172.21.15.36:0/3911099001 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa658053a40 0x7fa658055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:19.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.194+0000 7fa6717fa700 1 -- 172.21.15.36:0/3911099001 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa66c06d3f0 con 0x7fa674144240 2022-06-17T12:57:19.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.194+0000 7fa673fff700 1 --2- 172.21.15.36:0/3911099001 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa658053a40 0x7fa658055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:19.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.194+0000 7fa673fff700 1 --2- 172.21.15.36:0/3911099001 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa658053a40 0x7fa658055f00 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7fa66400b000 tx=0x7fa664009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:19.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.198+0000 7fa6717fa700 1 -- 172.21.15.36:0/3911099001 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa66c044060 con 0x7fa674144240 2022-06-17T12:57:19.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.326+0000 7fa67a624700 1 -- 172.21.15.36:0/3911099001 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa674147aa0 con 0x7fa674144240 2022-06-17T12:57:19.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.330+0000 7fa6717fa700 1 -- 172.21.15.36:0/3911099001 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v151) v1 ==== 100+0+10 (secure 0 0 0) 0x7fa66c0230a0 con 0x7fa674144240 2022-06-17T12:57:19.315 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:57:19.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.330+0000 7fa67a624700 1 -- 172.21.15.36:0/3911099001 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa658053a40 msgr2=0x7fa658055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:19.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.330+0000 7fa67a624700 1 --2- 172.21.15.36:0/3911099001 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa658053a40 0x7fa658055f00 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7fa66400b000 tx=0x7fa664009040).stop 2022-06-17T12:57:19.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.330+0000 7fa67a624700 1 -- 172.21.15.36:0/3911099001 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa674144240 msgr2=0x7fa674145ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:19.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.330+0000 7fa67a624700 1 --2- 172.21.15.36:0/3911099001 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa674144240 0x7fa674145ad0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 rx=0x7fa66c002f70 tx=0x7fa66c015000).stop 2022-06-17T12:57:19.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.330+0000 7fa67a624700 1 -- 172.21.15.36:0/3911099001 shutdown_connections 2022-06-17T12:57:19.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.330+0000 7fa67a624700 1 --2- 172.21.15.36:0/3911099001 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa658053a40 0x7fa658055f00 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:19.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.330+0000 7fa67a624700 1 --2- 172.21.15.36:0/3911099001 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa674144240 0x7fa674145ad0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:19.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.330+0000 7fa67a624700 1 --2- 172.21.15.36:0/3911099001 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6740acee0 0x7fa6741455e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:19.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.330+0000 7fa67a624700 1 --2- 172.21.15.36:0/3911099001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6740acb00 0x7fa67414aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:19.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.330+0000 7fa67a624700 1 -- 172.21.15.36:0/3911099001 >> 172.21.15.36:0/3911099001 conn(0x7fa67405c550 msgr2=0x7fa674148950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:19.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.330+0000 7fa67a624700 1 -- 172.21.15.36:0/3911099001 shutdown_connections 2022-06-17T12:57:19.319 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.330+0000 7fa67a624700 1 -- 172.21.15.36:0/3911099001 wait complete. 2022-06-17T12:57:19.334 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:57:19.335 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:57:19.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.458+0000 7f7561d58700 1 Processor -- start 2022-06-17T12:57:19.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.458+0000 7f7561d58700 1 -- start start 2022-06-17T12:57:19.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f7561d58700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f755c066c60 0x7f755c0ad880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:19.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f7561d58700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f755c0ad330 0x7f755c05f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:19.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f7561d58700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f755c0add70 con 0x7f755c0afd80 2022-06-17T12:57:19.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f7561d58700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f755c146d50 con 0x7f755c066c60 2022-06-17T12:57:19.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f7561d58700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f755c067040 con 0x7f755c0ad330 2022-06-17T12:57:19.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f755b7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f755c066c60 0x7f755c0ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:19.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f755affd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f755c0afd80 0x7f755c146640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58396/0 (socket says 172.21.15.36:58396) 2022-06-17T12:57:19.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f755affd700 1 -- 172.21.15.36:0/1598293373 learned_addr learned my addr 172.21.15.36:0/1598293373 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:19.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f755bfff700 1 --2- 172.21.15.36:0/1598293373 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f755c0ad330 0x7f755c05f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:19.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f755a7fc700 1 -- 172.21.15.36:0/1598293373 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4080177607 0 0) 0x7f7544001490 con 0x7f755c0afd80 2022-06-17T12:57:19.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f755b7fe700 1 -- 172.21.15.36:0/1598293373 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f755c0ad330 msgr2=0x7f755c05f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:19.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f755b7fe700 1 --2- 172.21.15.36:0/1598293373 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f755c0ad330 0x7f755c05f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:19.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f755b7fe700 1 -- 172.21.15.36:0/1598293373 >> v1:172.21.15.36:6789/0 conn(0x7f755c0afd80 legacy=0x7f755c146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:19.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f755b7fe700 1 -- 172.21.15.36:0/1598293373 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c0b0940 con 0x7f755c066c60 2022-06-17T12:57:19.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f755bfff700 1 --2- 172.21.15.36:0/1598293373 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f755c0ad330 0x7f755c05f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:57:19.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f755b7fe700 1 --2- 172.21.15.36:0/1598293373 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f755c066c60 0x7f755c0ad880 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7f754c002f70 tx=0x7f754c006df0).ready entity=mon.0 client_cookie=a5f760778fe1c079 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:19.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f755a7fc700 1 -- 172.21.15.36:0/1598293373 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f754c00d570 con 0x7f755c066c60 2022-06-17T12:57:19.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f755a7fc700 1 -- 172.21.15.36:0/1598293373 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754c000e90 con 0x7f755c066c60 2022-06-17T12:57:19.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f755a7fc700 1 -- 172.21.15.36:0/1598293373 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f754c00d8c0 con 0x7f755c066c60 2022-06-17T12:57:19.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f7561d58700 1 -- 172.21.15.36:0/1598293373 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f755c066c60 msgr2=0x7f755c0ad880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:19.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f7561d58700 1 --2- 172.21.15.36:0/1598293373 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f755c066c60 0x7f755c0ad880 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7f754c002f70 tx=0x7f754c006df0).stop 2022-06-17T12:57:19.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f7561d58700 1 -- 172.21.15.36:0/1598293373 shutdown_connections 2022-06-17T12:57:19.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f7561d58700 1 --2- 172.21.15.36:0/1598293373 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f755c0ad330 0x7f755c05f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:19.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f7561d58700 1 --2- 172.21.15.36:0/1598293373 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f755c066c60 0x7f755c0ad880 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:19.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f7561d58700 1 -- 172.21.15.36:0/1598293373 >> 172.21.15.36:0/1598293373 conn(0x7f755c0b53e0 msgr2=0x7f755c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:19.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f7561d58700 1 -- 172.21.15.36:0/1598293373 shutdown_connections 2022-06-17T12:57:19.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.462+0000 7f7561d58700 1 -- 172.21.15.36:0/1598293373 wait complete. 2022-06-17T12:57:19.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f7561d58700 1 Processor -- start 2022-06-17T12:57:19.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f7561d58700 1 -- start start 2022-06-17T12:57:19.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f7561d58700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f755c066c60 0x7f755c1525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:19.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f7561d58700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f755c0ad330 0x7f755c152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:19.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f7561d58700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f755c0afd80 0x7f755c1122e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:19.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f7561d58700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c10ec80 con 0x7f755c0afd80 2022-06-17T12:57:19.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f7561d58700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c153130 con 0x7f755c0ad330 2022-06-17T12:57:19.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f7561d58700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f755c1533d0 con 0x7f755c066c60 2022-06-17T12:57:19.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f755bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f755c0afd80 0x7f755c1122e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:19.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f755bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f755c0afd80 0x7f755c1122e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40876/0 (socket says 172.21.15.36:40876) 2022-06-17T12:57:19.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f755bfff700 1 -- 172.21.15.36:0/3294711263 learned_addr learned my addr 172.21.15.36:0/3294711263 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:19.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f755affd700 1 --2- 172.21.15.36:0/3294711263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f755c0ad330 0x7f755c152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:19.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f755b7fe700 1 --2- 172.21.15.36:0/3294711263 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f755c066c60 0x7f755c1525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:19.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f755bfff700 1 -- 172.21.15.36:0/3294711263 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f755c066c60 msgr2=0x7f755c1525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:19.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f755bfff700 1 --2- 172.21.15.36:0/3294711263 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f755c066c60 0x7f755c1525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:19.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f755bfff700 1 -- 172.21.15.36:0/3294711263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f755c0ad330 msgr2=0x7f755c152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:19.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f755bfff700 1 --2- 172.21.15.36:0/3294711263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f755c0ad330 0x7f755c152ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:19.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f755bfff700 1 -- 172.21.15.36:0/3294711263 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c00b180 con 0x7f755c0afd80 2022-06-17T12:57:19.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f755b7fe700 1 --2- 172.21.15.36:0/3294711263 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f755c066c60 0x7f755c1525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:57:19.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f755affd700 1 --2- 172.21.15.36:0/3294711263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f755c0ad330 0x7f755c152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:57:19.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f755bfff700 1 --2- 172.21.15.36:0/3294711263 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f755c0afd80 0x7f755c1122e0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7f7554006940 tx=0x7f7554008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:19.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f7558ff9700 1 -- 172.21.15.36:0/3294711263 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7554011780 con 0x7f755c0afd80 2022-06-17T12:57:19.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f7558ff9700 1 -- 172.21.15.36:0/3294711263 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75540118e0 con 0x7f755c0afd80 2022-06-17T12:57:19.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f7561d58700 1 -- 172.21.15.36:0/3294711263 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f755c1410f0 con 0x7f755c0afd80 2022-06-17T12:57:19.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f7558ff9700 1 -- 172.21.15.36:0/3294711263 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7554011c40 con 0x7f755c0afd80 2022-06-17T12:57:19.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.466+0000 7f7561d58700 1 -- 172.21.15.36:0/3294711263 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755c10e880 con 0x7f755c0afd80 2022-06-17T12:57:19.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.470+0000 7f7561d58700 1 -- 172.21.15.36:0/3294711263 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f755c057410 con 0x7f755c0afd80 2022-06-17T12:57:19.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.470+0000 7f7558ff9700 1 -- 172.21.15.36:0/3294711263 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f755401c460 con 0x7f755c0afd80 2022-06-17T12:57:19.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.470+0000 7f7558ff9700 1 --2- 172.21.15.36:0/3294711263 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7540053ad0 0x7f7540055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:19.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.470+0000 7f755b7fe700 1 --2- 172.21.15.36:0/3294711263 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7540053ad0 0x7f7540055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:19.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.470+0000 7f7558ff9700 1 -- 172.21.15.36:0/3294711263 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f755406c3a0 con 0x7f755c0afd80 2022-06-17T12:57:19.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.470+0000 7f755b7fe700 1 --2- 172.21.15.36:0/3294711263 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7540053ad0 0x7f7540055f90 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f754c01a000 tx=0x7f754c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:19.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.474+0000 7f7558ff9700 1 -- 172.21.15.36:0/3294711263 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7554043030 con 0x7f755c0afd80 2022-06-17T12:57:19.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.634+0000 7f7561d58700 1 -- 172.21.15.36:0/3294711263 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f755c0b0cd0 con 0x7f755c0afd80 2022-06-17T12:57:19.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.634+0000 7f7558ff9700 1 -- 172.21.15.36:0/3294711263 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f7554046650 con 0x7f755c0afd80 2022-06-17T12:57:19.623 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:57:19.624 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":83,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8599,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/4216085219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4216085219},{"type":"v1","addr":"172.21.15.70:6833","nonce":4216085219}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":82}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:57:14.606323+0000","modified":"2022-06-17T12:57:15.610154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8805":{"gid":8805,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/2534551587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2534551587},{"type":"v1","addr":"172.21.15.36:6835","nonce":2534551587}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-06-17T12:57:19.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.634+0000 7f7561d58700 1 -- 172.21.15.36:0/3294711263 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7540053ad0 msgr2=0x7f7540055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:19.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.634+0000 7f7561d58700 1 --2- 172.21.15.36:0/3294711263 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7540053ad0 0x7f7540055f90 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f754c01a000 tx=0x7f754c015000).stop 2022-06-17T12:57:19.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.634+0000 7f7561d58700 1 -- 172.21.15.36:0/3294711263 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f755c0afd80 msgr2=0x7f755c1122e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:19.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.634+0000 7f7561d58700 1 --2- 172.21.15.36:0/3294711263 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f755c0afd80 0x7f755c1122e0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7f7554006940 tx=0x7f7554008040).stop 2022-06-17T12:57:19.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.634+0000 7f7561d58700 1 -- 172.21.15.36:0/3294711263 shutdown_connections 2022-06-17T12:57:19.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.634+0000 7f7561d58700 1 --2- 172.21.15.36:0/3294711263 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f7540053ad0 0x7f7540055f90 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:19.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.634+0000 7f7561d58700 1 --2- 172.21.15.36:0/3294711263 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f755c0afd80 0x7f755c1122e0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:19.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.634+0000 7f7561d58700 1 --2- 172.21.15.36:0/3294711263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f755c0ad330 0x7f755c152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:19.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.634+0000 7f7561d58700 1 --2- 172.21.15.36:0/3294711263 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f755c066c60 0x7f755c1525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:19.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.634+0000 7f7561d58700 1 -- 172.21.15.36:0/3294711263 >> 172.21.15.36:0/3294711263 conn(0x7f755c0b53e0 msgr2=0x7f755c13d120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:19.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.634+0000 7f7561d58700 1 -- 172.21.15.36:0/3294711263 shutdown_connections 2022-06-17T12:57:19.634 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:19.634+0000 7f7561d58700 1 -- 172.21.15.36:0/3294711263 wait complete. 2022-06-17T12:57:19.635 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 83 2022-06-17T12:57:19.839 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:57:14.606323+0000', 'modified': '2022-06-17T12:57:15.610154+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8805}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8805': {'gid': 8805, 'name': 'a', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.36:6835/2534551587', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 2534551587}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 2534551587}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:57:19.840 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:57:19.840 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:57:19.981 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":8805,"want_state":"up:active","state":"up:active","rank_uptime":5.358318895,"mdsmap_epoch":83,"osdmap_epoch":151,"osdmap_epoch_barrier":149,"uptime":9.7518576980000002} 2022-06-17T12:57:19.982 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 8805, "mdsmap_epoch": 83, "osdmap_epoch": 151, "osdmap_epoch_barrier": 149, "rank_uptime": 5.358318895, "state": "up:active", "uptime": 9.751857698, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:57:19.985 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:57:19.986 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-17T12:57:19.987 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi107.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-17T12:57:19.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:57:19.990 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:57:20.003 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:57:20.068 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:57:20.069 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:57:20.084 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:57:20.086 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:57:20.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:57:20.087 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:57:20.097 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:57:20.098 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:57:20.142 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2022-06-17T12:57:20.159 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:57:20.160 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:57:20.221 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:57:20.223 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:2022-06-17T12:57:20.236+0000 7faf6f4f2080 -1 init, newargv = 0x55e72fe3d5a0 newargc=9 2022-06-17T12:57:20.223 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[23087]: starting ceph client 2022-06-17T12:57:20.224 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:57:20.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:57:20.227 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2022-06-17T12:57:20.230 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[23087]: starting fuse 2022-06-17T12:57:20.256 INFO:teuthology.orchestra.run.smithi107.stdout:52 2022-06-17T12:57:20.257 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:57:20.257 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:57:20.258 DEBUG:teuthology.orchestra.run.smithi107:> sudo python3 -c ' 2022-06-17T12:57:20.259 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2022-06-17T12:57:20.260 DEBUG:teuthology.orchestra.run.smithi107:> import re 2022-06-17T12:57:20.262 DEBUG:teuthology.orchestra.run.smithi107:> import os 2022-06-17T12:57:20.265 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2022-06-17T12:57:20.265 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:57:20.266 DEBUG:teuthology.orchestra.run.smithi107:> def find_socket(client_name): 2022-06-17T12:57:20.266 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-17T12:57:20.267 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2022-06-17T12:57:20.267 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:57:20.268 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2022-06-17T12:57:20.268 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2022-06-17T12:57:20.268 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2022-06-17T12:57:20.269 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2022-06-17T12:57:20.269 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:57:20.277 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2022-06-17T12:57:20.277 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:57:20.278 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:57:20.279 DEBUG:teuthology.orchestra.run.smithi107:> return f 2022-06-17T12:57:20.280 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:57:20.281 DEBUG:teuthology.orchestra.run.smithi107:> 2022-06-17T12:57:20.281 DEBUG:teuthology.orchestra.run.smithi107:> print(find_socket("client.0")) 2022-06-17T12:57:20.282 DEBUG:teuthology.orchestra.run.smithi107:> ' 2022-06-17T12:57:20.341 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.0.23087.asok 2022-06-17T12:57:20.352 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23087.asok 2022-06-17T12:57:20.353 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:57:20.354 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23087.asok status 2022-06-17T12:57:20.446 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2022-06-17T12:57:20.447 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2022-06-17T12:57:20.447 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:57:20.448 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:57:20.448 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "0", 2022-06-17T12:57:20.449 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2022-06-17T12:57:20.449 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-17T12:57:20.449 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "23087", 2022-06-17T12:57:20.450 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/" 2022-06-17T12:57:20.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:57:20.451 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2022-06-17T12:57:20.452 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2022-06-17T12:57:20.452 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 8958, 2022-06-17T12:57:20.454 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2022-06-17T12:57:20.454 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2022-06-17T12:57:20.454 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2022-06-17T12:57:20.455 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 8958 2022-06-17T12:57:20.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:57:20.456 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:57:20.456 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:57:20.457 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:57:20.457 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 2423802705 2022-06-17T12:57:20.458 INFO:teuthology.orchestra.run.smithi107.stdout: } 2022-06-17T12:57:20.458 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:57:20.458 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2022-06-17T12:57:20.459 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2022-06-17T12:57:20.459 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2022-06-17T12:57:20.460 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 2423802705 2022-06-17T12:57:20.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2022-06-17T12:57:20.460 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.8958 172.21.15.107:0/2423802705", 2022-06-17T12:57:20.461 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "172.21.15.107:0/2423802705", 2022-06-17T12:57:20.461 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 1, 2022-06-17T12:57:20.462 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 83, 2022-06-17T12:57:20.462 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 151, 2022-06-17T12:57:20.463 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:57:20.463 INFO:teuthology.orchestra.run.smithi107.stdout: "blacklisted": false, 2022-06-17T12:57:20.463 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2022-06-17T12:57:20.464 INFO:teuthology.orchestra.run.smithi107.stdout:} 2022-06-17T12:57:20.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:57:20.467 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:57:20.468 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:57:20.496 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2022-06-17T12:57:20.497 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-17T12:57:20.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:57:20.500 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:57:20.523 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T12:57:20.656 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.654+0000 7fe2fe3dd700 1 Processor -- start 2022-06-17T12:57:20.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.654+0000 7fe2fe3dd700 1 -- start start 2022-06-17T12:57:20.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.654+0000 7fe2fe3dd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe2f813cb70 0x7fe2f8143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:20.657 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.654+0000 7fe2fe3dd700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe2f813cf50 0x7fe2f81442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:20.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.654+0000 7fe2fe3dd700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2f80af4b0 con 0x7fe2f80aef60 2022-06-17T12:57:20.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.654+0000 7fe2fe3dd700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f80ba500 con 0x7fe2f813cb70 2022-06-17T12:57:20.658 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.654+0000 7fe2fe3dd700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f8142f50 con 0x7fe2f813cf50 2022-06-17T12:57:20.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.658+0000 7fe2fcbda700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fe2f80aef60 0x7fe2f80b9df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58410/0 (socket says 172.21.15.36:58410) 2022-06-17T12:57:20.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.658+0000 7fe2fcbda700 1 -- 172.21.15.36:0/2755290994 learned_addr learned my addr 172.21.15.36:0/2755290994 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:20.659 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.658+0000 7fe2fd3db700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe2f813cb70 0x7fe2f8143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:20.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.658+0000 7fe2fdbdc700 1 --2- 172.21.15.36:0/2755290994 >> v2:172.21.15.70:3300/0 conn(0x7fe2f813cf50 0x7fe2f81442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:20.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.658+0000 7fe2effff700 1 -- 172.21.15.36:0/2755290994 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3483413668 0 0) 0x7fe2f80af4b0 con 0x7fe2f80aef60 2022-06-17T12:57:20.660 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.658+0000 7fe2effff700 1 -- 172.21.15.36:0/2755290994 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2dc003260 con 0x7fe2f80aef60 2022-06-17T12:57:20.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.658+0000 7fe2fdbdc700 1 -- 172.21.15.36:0/2755290994 >> v2:172.21.15.36:3300/0 conn(0x7fe2f813cb70 msgr2=0x7fe2f8143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:20.661 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.658+0000 7fe2fdbdc700 1 --2- 172.21.15.36:0/2755290994 >> v2:172.21.15.36:3300/0 conn(0x7fe2f813cb70 0x7fe2f8143690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:20.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.658+0000 7fe2fdbdc700 1 -- 172.21.15.36:0/2755290994 >> v1:172.21.15.36:6789/0 conn(0x7fe2f80aef60 legacy=0x7fe2f80b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:20.662 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.658+0000 7fe2fdbdc700 1 -- 172.21.15.36:0/2755290994 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f81431f0 con 0x7fe2f813cf50 2022-06-17T12:57:20.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.658+0000 7fe2fd3db700 1 --2- 172.21.15.36:0/2755290994 >> v2:172.21.15.36:3300/0 conn(0x7fe2f813cb70 0x7fe2f8143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:57:20.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.658+0000 7fe2fdbdc700 1 --2- 172.21.15.36:0/2755290994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f813cf50 0x7fe2f81442a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7fe2f00032f0 tx=0x7fe2f00091d0).ready entity=mon.1 client_cookie=9bc80d15e68875b2 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:20.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.658+0000 7fe2effff700 1 -- 172.21.15.36:0/2755290994 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2f000b600 con 0x7fe2f813cf50 2022-06-17T12:57:20.663 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.658+0000 7fe2effff700 1 -- 172.21.15.36:0/2755290994 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f000b760 con 0x7fe2f813cf50 2022-06-17T12:57:20.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.658+0000 7fe2effff700 1 -- 172.21.15.36:0/2755290994 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2f000b8d0 con 0x7fe2f813cf50 2022-06-17T12:57:20.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.658+0000 7fe2fe3dd700 1 -- 172.21.15.36:0/2755290994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f813cf50 msgr2=0x7fe2f81442a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:20.664 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.658+0000 7fe2fe3dd700 1 --2- 172.21.15.36:0/2755290994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f813cf50 0x7fe2f81442a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7fe2f00032f0 tx=0x7fe2f00091d0).stop 2022-06-17T12:57:20.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.658+0000 7fe2fe3dd700 1 -- 172.21.15.36:0/2755290994 shutdown_connections 2022-06-17T12:57:20.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.658+0000 7fe2fe3dd700 1 --2- 172.21.15.36:0/2755290994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f813cf50 0x7fe2f81442a0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:20.665 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.658+0000 7fe2fe3dd700 1 --2- 172.21.15.36:0/2755290994 >> v2:172.21.15.36:3300/0 conn(0x7fe2f813cb70 0x7fe2f8143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:20.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.658+0000 7fe2fe3dd700 1 -- 172.21.15.36:0/2755290994 >> 172.21.15.36:0/2755290994 conn(0x7fe2f805c550 msgr2=0x7fe2f80b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:20.666 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.658+0000 7fe2fe3dd700 1 -- 172.21.15.36:0/2755290994 shutdown_connections 2022-06-17T12:57:20.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.658+0000 7fe2fe3dd700 1 -- 172.21.15.36:0/2755290994 wait complete. 2022-06-17T12:57:20.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.658+0000 7fe2fe3dd700 1 Processor -- start 2022-06-17T12:57:20.667 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.662+0000 7fe2fe3dd700 1 -- start start 2022-06-17T12:57:20.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.662+0000 7fe2fe3dd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2f80aef60 0x7fe2f81426a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:20.668 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.662+0000 7fe2fe3dd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2f813cb70 0x7fe2f81420c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:20.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.662+0000 7fe2fe3dd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f813cf50 0x7fe2f81455c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:20.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.662+0000 7fe2fe3dd700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f813dd50 con 0x7fe2f813cb70 2022-06-17T12:57:20.669 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.662+0000 7fe2fe3dd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f813dff0 con 0x7fe2f813cf50 2022-06-17T12:57:20.673 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.662+0000 7fe2fe3dd700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f813e290 con 0x7fe2f80aef60 2022-06-17T12:57:20.678 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.662+0000 7fe2fcbda700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2f813cb70 0x7fe2f81420c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:20.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.662+0000 7fe2fcbda700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2f813cb70 0x7fe2f81420c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40886/0 (socket says 172.21.15.36:40886) 2022-06-17T12:57:20.679 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.662+0000 7fe2fcbda700 1 -- 172.21.15.36:0/1468811934 learned_addr learned my addr 172.21.15.36:0/1468811934 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:20.680 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.662+0000 7fe2fd3db700 1 --2- 172.21.15.36:0/1468811934 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2f80aef60 0x7fe2f81426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:20.682 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.662+0000 7fe2fdbdc700 1 --2- 172.21.15.36:0/1468811934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f813cf50 0x7fe2f81455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:20.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.662+0000 7fe2fcbda700 1 -- 172.21.15.36:0/1468811934 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2f80aef60 msgr2=0x7fe2f81426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:20.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.662+0000 7fe2fcbda700 1 --2- 172.21.15.36:0/1468811934 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2f80aef60 0x7fe2f81426a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:20.683 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.662+0000 7fe2fcbda700 1 -- 172.21.15.36:0/1468811934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f813cf50 msgr2=0x7fe2f81455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:20.684 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.662+0000 7fe2fcbda700 1 --2- 172.21.15.36:0/1468811934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f813cf50 0x7fe2f81455c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:20.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.662+0000 7fe2fcbda700 1 -- 172.21.15.36:0/1468811934 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f400b180 con 0x7fe2f813cb70 2022-06-17T12:57:20.685 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.662+0000 7fe2fcbda700 1 --2- 172.21.15.36:0/1468811934 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2f813cb70 0x7fe2f81420c0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7fe2e8001110 tx=0x7fe2e8007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:20.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.662+0000 7fe2ee7fc700 1 -- 172.21.15.36:0/1468811934 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2e80070d0 con 0x7fe2f813cb70 2022-06-17T12:57:20.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.662+0000 7fe2ee7fc700 1 -- 172.21.15.36:0/1468811934 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e8007230 con 0x7fe2f813cb70 2022-06-17T12:57:20.686 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.662+0000 7fe2fe3dd700 1 -- 172.21.15.36:0/1468811934 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f0009470 con 0x7fe2f813cb70 2022-06-17T12:57:20.687 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.662+0000 7fe2ee7fc700 1 -- 172.21.15.36:0/1468811934 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2e8005610 con 0x7fe2f813cb70 2022-06-17T12:57:20.688 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.662+0000 7fe2fe3dd700 1 -- 172.21.15.36:0/1468811934 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f8152290 con 0x7fe2f813cb70 2022-06-17T12:57:20.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.662+0000 7fe2fe3dd700 1 -- 172.21.15.36:0/1468811934 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f8057410 con 0x7fe2f813cb70 2022-06-17T12:57:20.689 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.666+0000 7fe2ee7fc700 1 -- 172.21.15.36:0/1468811934 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe2e8010070 con 0x7fe2f813cb70 2022-06-17T12:57:20.690 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.666+0000 7fe2ee7fc700 1 --2- 172.21.15.36:0/1468811934 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe2e0053b20 0x7fe2e0055fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:20.691 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.666+0000 7fe2fd3db700 1 --2- 172.21.15.36:0/1468811934 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe2e0053b20 0x7fe2e0055fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:20.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.666+0000 7fe2ee7fc700 1 -- 172.21.15.36:0/1468811934 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe2e806b820 con 0x7fe2f813cb70 2022-06-17T12:57:20.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.666+0000 7fe2fd3db700 1 --2- 172.21.15.36:0/1468811934 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe2e0053b20 0x7fe2e0055fe0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7fe2f400b000 tx=0x7fe2f4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:20.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.670+0000 7fe2ee7fc700 1 -- 172.21.15.36:0/1468811934 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe2e8042510 con 0x7fe2f813cb70 2022-06-17T12:57:20.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.798+0000 7fe2fe3dd700 1 -- 172.21.15.36:0/1468811934 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2f813c950 con 0x7fe2f813cb70 2022-06-17T12:57:20.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.798+0000 7fe2ee7fc700 1 -- 172.21.15.36:0/1468811934 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+16285 (secure 0 0 0) 0x7fe2e8045b30 con 0x7fe2f813cb70 2022-06-17T12:57:20.787 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:57:20.787 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":151,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:57:16.881090+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-06-17T12:57:13.415000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-06-17T12:57:14.234187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T12:57:20.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.802+0000 7fe2fe3dd700 1 -- 172.21.15.36:0/1468811934 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe2e0053b20 msgr2=0x7fe2e0055fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:20.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.802+0000 7fe2fe3dd700 1 --2- 172.21.15.36:0/1468811934 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe2e0053b20 0x7fe2e0055fe0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7fe2f400b000 tx=0x7fe2f4008040).stop 2022-06-17T12:57:20.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.802+0000 7fe2fe3dd700 1 -- 172.21.15.36:0/1468811934 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2f813cb70 msgr2=0x7fe2f81420c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:20.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.802+0000 7fe2fe3dd700 1 --2- 172.21.15.36:0/1468811934 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2f813cb70 0x7fe2f81420c0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7fe2e8001110 tx=0x7fe2e8007aa0).stop 2022-06-17T12:57:20.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.802+0000 7fe2fe3dd700 1 -- 172.21.15.36:0/1468811934 shutdown_connections 2022-06-17T12:57:20.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.802+0000 7fe2fe3dd700 1 --2- 172.21.15.36:0/1468811934 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe2e0053b20 0x7fe2e0055fe0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:20.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.802+0000 7fe2fe3dd700 1 --2- 172.21.15.36:0/1468811934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f813cf50 0x7fe2f81455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:20.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.802+0000 7fe2fe3dd700 1 --2- 172.21.15.36:0/1468811934 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2f813cb70 0x7fe2f81420c0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:20.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.802+0000 7fe2fe3dd700 1 --2- 172.21.15.36:0/1468811934 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2f80aef60 0x7fe2f81426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:20.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.802+0000 7fe2fe3dd700 1 -- 172.21.15.36:0/1468811934 >> 172.21.15.36:0/1468811934 conn(0x7fe2f805c550 msgr2=0x7fe2f805f770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:20.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.802+0000 7fe2fe3dd700 1 -- 172.21.15.36:0/1468811934 shutdown_connections 2022-06-17T12:57:20.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.802+0000 7fe2fe3dd700 1 -- 172.21.15.36:0/1468811934 wait complete. 2022-06-17T12:57:20.817 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-17T12:46:33.881212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-17T12:46:44.485460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-17T12:57:13.415000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '149', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2022-06-17T12:57:14.234187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-17T12:57:20.821 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-17T12:57:20.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.966+0000 7f765f173700 1 Processor -- start 2022-06-17T12:57:20.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.966+0000 7f765f173700 1 -- start start 2022-06-17T12:57:20.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f765f173700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76580b64f0 0x7f7658143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:20.963 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f765f173700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f76580b4120 0x7f7658146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:20.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f765f173700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76580b68d0 con 0x7f7658147270 2022-06-17T12:57:20.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f765f173700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76581476d0 con 0x7f76580b4120 2022-06-17T12:57:20.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f765f173700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76580b6e20 con 0x7f76580b64f0 2022-06-17T12:57:20.964 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f7657fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f76580b4120 0x7f7658146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:20.970 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f765cf0f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76580b64f0 0x7f7658143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:20.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f765d710700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7658147270 0x7f7658140100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:54160/0 (socket says 172.21.15.36:54160) 2022-06-17T12:57:20.971 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f765d710700 1 -- 172.21.15.36:0/4277461331 learned_addr learned my addr 172.21.15.36:0/4277461331 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:20.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f7657fff700 1 -- 172.21.15.36:0/4277461331 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76580b64f0 msgr2=0x7f7658143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:20.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f7657fff700 1 --2- 172.21.15.36:0/4277461331 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76580b64f0 0x7f7658143150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:20.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f7657fff700 1 -- 172.21.15.36:0/4277461331 >> v1:172.21.15.70:6789/0 conn(0x7f7658147270 legacy=0x7f7658140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:20.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f7657fff700 1 -- 172.21.15.36:0/4277461331 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76581468e0 con 0x7f76580b4120 2022-06-17T12:57:20.972 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f7657fff700 1 --2- 172.21.15.36:0/4277461331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76580b4120 0x7f7658146d80 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f7648002f70 tx=0x7f7648006df0).ready entity=mon.1 client_cookie=e1d54ea66eeb67c4 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:20.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f76577fe700 1 -- 172.21.15.36:0/4277461331 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f764800d570 con 0x7f76580b4120 2022-06-17T12:57:20.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f76577fe700 1 -- 172.21.15.36:0/4277461331 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7648000e90 con 0x7f76580b4120 2022-06-17T12:57:20.973 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f76577fe700 1 -- 172.21.15.36:0/4277461331 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f764800d8c0 con 0x7f76580b4120 2022-06-17T12:57:20.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f765f173700 1 -- 172.21.15.36:0/4277461331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76580b4120 msgr2=0x7f7658146d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:20.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f765f173700 1 --2- 172.21.15.36:0/4277461331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76580b4120 0x7f7658146d80 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f7648002f70 tx=0x7f7648006df0).stop 2022-06-17T12:57:20.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f765f173700 1 -- 172.21.15.36:0/4277461331 shutdown_connections 2022-06-17T12:57:20.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f765f173700 1 --2- 172.21.15.36:0/4277461331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76580b4120 0x7f7658146d80 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:20.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f765f173700 1 --2- 172.21.15.36:0/4277461331 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76580b64f0 0x7f7658143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:20.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f765f173700 1 -- 172.21.15.36:0/4277461331 >> 172.21.15.36:0/4277461331 conn(0x7f76580620f0 msgr2=0x7f7658062500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:21.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f765f173700 1 -- 172.21.15.36:0/4277461331 shutdown_connections 2022-06-17T12:57:21.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.970+0000 7f765f173700 1 -- 172.21.15.36:0/4277461331 wait complete. 2022-06-17T12:57:21.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.974+0000 7f765f173700 1 Processor -- start 2022-06-17T12:57:21.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.974+0000 7f765f173700 1 -- start start 2022-06-17T12:57:21.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.974+0000 7f765f173700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76580b4120 0x7f7658153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:21.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.974+0000 7f765f173700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76580b64f0 0x7f7658113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:21.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.974+0000 7f765f173700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7658147270 0x7f7658113f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:21.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.974+0000 7f765f173700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7658152640 con 0x7f7658147270 2022-06-17T12:57:21.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.974+0000 7f765f173700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7658152830 con 0x7f76580b64f0 2022-06-17T12:57:21.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.974+0000 7f765f173700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7658152ad0 con 0x7f76580b4120 2022-06-17T12:57:21.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.974+0000 7f765d710700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7658147270 0x7f7658113f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:21.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.974+0000 7f765d710700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7658147270 0x7f7658113f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40902/0 (socket says 172.21.15.36:40902) 2022-06-17T12:57:21.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.974+0000 7f7657fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76580b64f0 0x7f7658113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:21.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.974+0000 7f765d710700 1 -- 172.21.15.36:0/2560042008 learned_addr learned my addr 172.21.15.36:0/2560042008 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:21.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.974+0000 7f765cf0f700 1 --2- 172.21.15.36:0/2560042008 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76580b4120 0x7f7658153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:21.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.974+0000 7f7657fff700 1 -- 172.21.15.36:0/2560042008 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76580b4120 msgr2=0x7f7658153210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:21.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.974+0000 7f7657fff700 1 --2- 172.21.15.36:0/2560042008 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76580b4120 0x7f7658153210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.974+0000 7f7657fff700 1 -- 172.21.15.36:0/2560042008 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7658147270 msgr2=0x7f7658113f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:21.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.974+0000 7f7657fff700 1 --2- 172.21.15.36:0/2560042008 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7658147270 0x7f7658113f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.974+0000 7f7657fff700 1 -- 172.21.15.36:0/2560042008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764800b180 con 0x7f76580b64f0 2022-06-17T12:57:21.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.978+0000 7f765d710700 1 --2- 172.21.15.36:0/2560042008 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7658147270 0x7f7658113f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:57:21.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.978+0000 7f765cf0f700 1 --2- 172.21.15.36:0/2560042008 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76580b4120 0x7f7658153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:57:21.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.978+0000 7f7657fff700 1 --2- 172.21.15.36:0/2560042008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76580b64f0 0x7f7658113aa0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f7648002f70 tx=0x7f7648014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:21.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.978+0000 7f7655ffb700 1 -- 172.21.15.36:0/2560042008 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7648013e50 con 0x7f76580b64f0 2022-06-17T12:57:21.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.978+0000 7f7655ffb700 1 -- 172.21.15.36:0/2560042008 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764801e400 con 0x7f76580b64f0 2022-06-17T12:57:21.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.978+0000 7f765f173700 1 -- 172.21.15.36:0/2560042008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7658152d70 con 0x7f76580b64f0 2022-06-17T12:57:21.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.978+0000 7f7655ffb700 1 -- 172.21.15.36:0/2560042008 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f764801e760 con 0x7f76580b64f0 2022-06-17T12:57:21.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.978+0000 7f765f173700 1 -- 172.21.15.36:0/2560042008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7658113550 con 0x7f76580b64f0 2022-06-17T12:57:21.012 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.978+0000 7f765f173700 1 -- 172.21.15.36:0/2560042008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7658152f00 con 0x7f76580b64f0 2022-06-17T12:57:21.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.978+0000 7f7655ffb700 1 -- 172.21.15.36:0/2560042008 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f7648025030 con 0x7f76580b64f0 2022-06-17T12:57:21.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.978+0000 7f7655ffb700 1 --2- 172.21.15.36:0/2560042008 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f76380539d0 0x7f7638055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:21.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.978+0000 7f765cf0f700 1 --2- 172.21.15.36:0/2560042008 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f76380539d0 0x7f7638055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:21.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.978+0000 7f7655ffb700 1 -- 172.21.15.36:0/2560042008 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f764806cde0 con 0x7f76580b64f0 2022-06-17T12:57:21.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.982+0000 7f765cf0f700 1 --2- 172.21.15.36:0/2560042008 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f76380539d0 0x7f7638055e90 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f764c00b000 tx=0x7f764c006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:21.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:20.986+0000 7f7655ffb700 1 -- 172.21.15.36:0/2560042008 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7648043ad0 con 0x7f76580b64f0 2022-06-17T12:57:21.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.118+0000 7f765f173700 1 -- 172.21.15.36:0/2560042008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7658143640 con 0x7f76580b64f0 2022-06-17T12:57:21.108 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.118+0000 7f7655ffb700 1 -- 172.21.15.36:0/2560042008 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v151) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7648023020 con 0x7f76580b64f0 2022-06-17T12:57:21.115 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-06-17T12:57:21.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.122+0000 7f765f173700 1 -- 172.21.15.36:0/2560042008 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f76380539d0 msgr2=0x7f7638055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:21.118 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.122+0000 7f765f173700 1 --2- 172.21.15.36:0/2560042008 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f76380539d0 0x7f7638055e90 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f764c00b000 tx=0x7f764c006df0).stop 2022-06-17T12:57:21.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.122+0000 7f765f173700 1 -- 172.21.15.36:0/2560042008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76580b64f0 msgr2=0x7f7658113aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:21.119 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.122+0000 7f765f173700 1 --2- 172.21.15.36:0/2560042008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76580b64f0 0x7f7658113aa0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f7648002f70 tx=0x7f7648014000).stop 2022-06-17T12:57:21.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.122+0000 7f765f173700 1 -- 172.21.15.36:0/2560042008 shutdown_connections 2022-06-17T12:57:21.120 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.122+0000 7f765f173700 1 --2- 172.21.15.36:0/2560042008 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f76380539d0 0x7f7638055e90 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.122+0000 7f765f173700 1 --2- 172.21.15.36:0/2560042008 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7658147270 0x7f7658113f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.121 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.122+0000 7f765f173700 1 --2- 172.21.15.36:0/2560042008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76580b64f0 0x7f7658113aa0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.122+0000 7f765f173700 1 --2- 172.21.15.36:0/2560042008 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76580b4120 0x7f7658153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.122+0000 7f765f173700 1 -- 172.21.15.36:0/2560042008 >> 172.21.15.36:0/2560042008 conn(0x7f76580620f0 msgr2=0x7f765813e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:21.127 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.122+0000 7f765f173700 1 -- 172.21.15.36:0/2560042008 shutdown_connections 2022-06-17T12:57:21.128 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.122+0000 7f765f173700 1 -- 172.21.15.36:0/2560042008 wait complete. 2022-06-17T12:57:21.141 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-17T12:57:21.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.274+0000 7fcd1c212700 1 Processor -- start 2022-06-17T12:57:21.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd1c212700 1 -- start start 2022-06-17T12:57:21.268 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd1c212700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcd14060e40 0x7fcd141451e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:21.269 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd1c212700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd140b1360 0x7fcd14066a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:21.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd1c212700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd140b1830 con 0x7fcd140aed10 2022-06-17T12:57:21.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd1c212700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd141423e0 con 0x7fcd14060e40 2022-06-17T12:57:21.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd1c212700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd14062b30 con 0x7fcd140b1360 2022-06-17T12:57:21.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd197ad700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcd14060e40 0x7fcd141451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:21.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd19fae700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fcd140aed10 0x7fcd141425f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58436/0 (socket says 172.21.15.36:58436) 2022-06-17T12:57:21.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd19fae700 1 -- 172.21.15.36:0/2424864899 learned_addr learned my addr 172.21.15.36:0/2424864899 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:21.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd1a7af700 1 --2- 172.21.15.36:0/2424864899 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd140b1360 0x7fcd14066a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:21.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd197ad700 1 -- 172.21.15.36:0/2424864899 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd140b1360 msgr2=0x7fcd14066a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:21.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd197ad700 1 --2- 172.21.15.36:0/2424864899 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd140b1360 0x7fcd14066a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd197ad700 1 -- 172.21.15.36:0/2424864899 >> v1:172.21.15.36:6789/0 conn(0x7fcd140aed10 legacy=0x7fcd141425f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:21.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd18fac700 1 -- 172.21.15.36:0/2424864899 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2301726808 0 0) 0x7fcd140b1830 con 0x7fcd140aed10 2022-06-17T12:57:21.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd197ad700 1 -- 172.21.15.36:0/2424864899 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd140b0440 con 0x7fcd14060e40 2022-06-17T12:57:21.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd197ad700 1 --2- 172.21.15.36:0/2424864899 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd14060e40 0x7fcd141451e0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 rx=0x7fcd14068870 tx=0x7fcd100091d0).ready entity=mon.0 client_cookie=c29a064111bc69e4 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:21.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd18fac700 1 -- 172.21.15.36:0/2424864899 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd1000b680 con 0x7fcd14060e40 2022-06-17T12:57:21.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd18fac700 1 -- 172.21.15.36:0/2424864899 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1000b7e0 con 0x7fcd14060e40 2022-06-17T12:57:21.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd18fac700 1 -- 172.21.15.36:0/2424864899 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd1000b950 con 0x7fcd14060e40 2022-06-17T12:57:21.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd1c212700 1 -- 172.21.15.36:0/2424864899 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd14060e40 msgr2=0x7fcd141451e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:21.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd1c212700 1 --2- 172.21.15.36:0/2424864899 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd14060e40 0x7fcd141451e0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 rx=0x7fcd14068870 tx=0x7fcd100091d0).stop 2022-06-17T12:57:21.277 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd1c212700 1 -- 172.21.15.36:0/2424864899 shutdown_connections 2022-06-17T12:57:21.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd1c212700 1 --2- 172.21.15.36:0/2424864899 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd140b1360 0x7fcd14066a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.280 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd1c212700 1 --2- 172.21.15.36:0/2424864899 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd14060e40 0x7fcd141451e0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd1c212700 1 -- 172.21.15.36:0/2424864899 >> 172.21.15.36:0/2424864899 conn(0x7fcd1405c550 msgr2=0x7fcd140b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:21.281 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd1c212700 1 -- 172.21.15.36:0/2424864899 shutdown_connections 2022-06-17T12:57:21.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.278+0000 7fcd1c212700 1 -- 172.21.15.36:0/2424864899 wait complete. 2022-06-17T12:57:21.282 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.282+0000 7fcd1c212700 1 Processor -- start 2022-06-17T12:57:21.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.282+0000 7fcd1c212700 1 -- start start 2022-06-17T12:57:21.283 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.282+0000 7fcd1c212700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd14060e40 0x7fcd1413a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:21.284 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.282+0000 7fcd1c212700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd140aed10 0x7fcd14066eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:21.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.282+0000 7fcd1c212700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd140b1360 0x7fcd1413d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:21.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.282+0000 7fcd1c212700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd14065580 con 0x7fcd14060e40 2022-06-17T12:57:21.285 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.282+0000 7fcd1c212700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1413ac80 con 0x7fcd140aed10 2022-06-17T12:57:21.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.282+0000 7fcd1c212700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd14066770 con 0x7fcd140b1360 2022-06-17T12:57:21.286 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.282+0000 7fcd19fae700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd14060e40 0x7fcd1413a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:21.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.282+0000 7fcd19fae700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd14060e40 0x7fcd1413a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40912/0 (socket says 172.21.15.36:40912) 2022-06-17T12:57:21.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.282+0000 7fcd19fae700 1 -- 172.21.15.36:0/4196702478 learned_addr learned my addr 172.21.15.36:0/4196702478 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:21.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.282+0000 7fcd197ad700 1 --2- 172.21.15.36:0/4196702478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd140aed10 0x7fcd14066eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:21.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.282+0000 7fcd1a7af700 1 --2- 172.21.15.36:0/4196702478 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd140b1360 0x7fcd1413d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:21.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.282+0000 7fcd19fae700 1 -- 172.21.15.36:0/4196702478 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd140b1360 msgr2=0x7fcd1413d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:21.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.282+0000 7fcd19fae700 1 --2- 172.21.15.36:0/4196702478 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd140b1360 0x7fcd1413d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.282+0000 7fcd19fae700 1 -- 172.21.15.36:0/4196702478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd140aed10 msgr2=0x7fcd14066eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:21.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.282+0000 7fcd19fae700 1 --2- 172.21.15.36:0/4196702478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd140aed10 0x7fcd14066eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.282+0000 7fcd19fae700 1 -- 172.21.15.36:0/4196702478 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd10009470 con 0x7fcd14060e40 2022-06-17T12:57:21.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.282+0000 7fcd19fae700 1 --2- 172.21.15.36:0/4196702478 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd14060e40 0x7fcd1413a0c0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7fcd04000ed0 tx=0x7fcd04007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:21.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.282+0000 7fcd02ffd700 1 -- 172.21.15.36:0/4196702478 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd040070d0 con 0x7fcd14060e40 2022-06-17T12:57:21.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.282+0000 7fcd02ffd700 1 -- 172.21.15.36:0/4196702478 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd04007230 con 0x7fcd14060e40 2022-06-17T12:57:21.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.282+0000 7fcd1c212700 1 -- 172.21.15.36:0/4196702478 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd14066a10 con 0x7fcd14060e40 2022-06-17T12:57:21.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.282+0000 7fcd02ffd700 1 -- 172.21.15.36:0/4196702478 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd04005610 con 0x7fcd14060e40 2022-06-17T12:57:21.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.282+0000 7fcd1c212700 1 -- 172.21.15.36:0/4196702478 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd14065200 con 0x7fcd14060e40 2022-06-17T12:57:21.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.286+0000 7fcd1c212700 1 -- 172.21.15.36:0/4196702478 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd14066ba0 con 0x7fcd14060e40 2022-06-17T12:57:21.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.286+0000 7fcd02ffd700 1 -- 172.21.15.36:0/4196702478 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fcd0401a060 con 0x7fcd14060e40 2022-06-17T12:57:21.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.286+0000 7fcd02ffd700 1 --2- 172.21.15.36:0/4196702478 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fccf40539d0 0x7fccf4055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:21.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.286+0000 7fcd197ad700 1 --2- 172.21.15.36:0/4196702478 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fccf40539d0 0x7fccf4055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:21.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.286+0000 7fcd02ffd700 1 -- 172.21.15.36:0/4196702478 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcd0406b3f0 con 0x7fcd14060e40 2022-06-17T12:57:21.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.286+0000 7fcd197ad700 1 --2- 172.21.15.36:0/4196702478 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fccf40539d0 0x7fccf4055e90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7fcd1001b000 tx=0x7fcd10016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:21.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.294+0000 7fcd02ffd700 1 -- 172.21.15.36:0/4196702478 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcd040421f0 con 0x7fcd14060e40 2022-06-17T12:57:21.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.422+0000 7fcd1c212700 1 -- 172.21.15.36:0/4196702478 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcd14057410 con 0x7fcd14060e40 2022-06-17T12:57:21.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.422+0000 7fcd02ffd700 1 -- 172.21.15.36:0/4196702478 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v151) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcd04018020 con 0x7fcd14060e40 2022-06-17T12:57:21.408 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:57:21.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.426+0000 7fcd1c212700 1 -- 172.21.15.36:0/4196702478 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fccf40539d0 msgr2=0x7fccf4055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:21.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.426+0000 7fcd1c212700 1 --2- 172.21.15.36:0/4196702478 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fccf40539d0 0x7fccf4055e90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7fcd1001b000 tx=0x7fcd10016000).stop 2022-06-17T12:57:21.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.426+0000 7fcd1c212700 1 -- 172.21.15.36:0/4196702478 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd14060e40 msgr2=0x7fcd1413a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:21.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.426+0000 7fcd1c212700 1 --2- 172.21.15.36:0/4196702478 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd14060e40 0x7fcd1413a0c0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7fcd04000ed0 tx=0x7fcd04007ac0).stop 2022-06-17T12:57:21.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.426+0000 7fcd1c212700 1 -- 172.21.15.36:0/4196702478 shutdown_connections 2022-06-17T12:57:21.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.426+0000 7fcd1c212700 1 --2- 172.21.15.36:0/4196702478 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fccf40539d0 0x7fccf4055e90 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.426+0000 7fcd1c212700 1 --2- 172.21.15.36:0/4196702478 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd140b1360 0x7fcd1413d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.426+0000 7fcd1c212700 1 --2- 172.21.15.36:0/4196702478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd140aed10 0x7fcd14066eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.426+0000 7fcd1c212700 1 --2- 172.21.15.36:0/4196702478 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd14060e40 0x7fcd1413a0c0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.412 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.426+0000 7fcd1c212700 1 -- 172.21.15.36:0/4196702478 >> 172.21.15.36:0/4196702478 conn(0x7fcd1405c550 msgr2=0x7fcd140ad910 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:21.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.426+0000 7fcd1c212700 1 -- 172.21.15.36:0/4196702478 shutdown_connections 2022-06-17T12:57:21.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.426+0000 7fcd1c212700 1 -- 172.21.15.36:0/4196702478 wait complete. 2022-06-17T12:57:21.424 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-17T12:57:21.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.562+0000 7fdea3445700 1 Processor -- start 2022-06-17T12:57:21.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.562+0000 7fdea3445700 1 -- start start 2022-06-17T12:57:21.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.562+0000 7fdea3445700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fde9c14b2b0 0x7fde9c060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:21.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.562+0000 7fdea3445700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fde9c147bd0 0x7fde9c1442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:21.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.562+0000 7fdea3445700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde9c14b870 con 0x7fde9c0b27b0 2022-06-17T12:57:21.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.562+0000 7fdea3445700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde9c0ac500 con 0x7fde9c147bd0 2022-06-17T12:57:21.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.562+0000 7fdea3445700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde9c0ace90 con 0x7fde9c14b2b0 2022-06-17T12:57:21.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.562+0000 7fdea2c44700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fde9c147bd0 0x7fde9c1442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:21.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.562+0000 7fdea2c44700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fde9c147bd0 0x7fde9c1442d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40922/0 (socket says 172.21.15.36:40922) 2022-06-17T12:57:21.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.562+0000 7fdea2c44700 1 -- 172.21.15.36:0/2454372767 learned_addr learned my addr 172.21.15.36:0/2454372767 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:21.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.562+0000 7fdea1c42700 1 --2- 172.21.15.36:0/2454372767 >> v2:172.21.15.70:3300/0 conn(0x7fde9c14b2b0 0x7fde9c060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:21.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.562+0000 7fdea2c44700 1 -- 172.21.15.36:0/2454372767 >> v2:172.21.15.70:3300/0 conn(0x7fde9c14b2b0 msgr2=0x7fde9c060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:21.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.562+0000 7fdea2c44700 1 --2- 172.21.15.36:0/2454372767 >> v2:172.21.15.70:3300/0 conn(0x7fde9c14b2b0 0x7fde9c060ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.562+0000 7fdea2c44700 1 -- 172.21.15.36:0/2454372767 >> v1:172.21.15.70:6789/0 conn(0x7fde9c0b27b0 legacy=0x7fde9c0ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:21.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.562+0000 7fdea2c44700 1 -- 172.21.15.36:0/2454372767 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c0b2b90 con 0x7fde9c147bd0 2022-06-17T12:57:21.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.562+0000 7fdea1c42700 1 --2- 172.21.15.36:0/2454372767 >> v2:172.21.15.70:3300/0 conn(0x7fde9c14b2b0 0x7fde9c060ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:57:21.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.562+0000 7fdea2c44700 1 --2- 172.21.15.36:0/2454372767 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde9c147bd0 0x7fde9c1442d0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7fde94002f70 tx=0x7fde94006eb0).ready entity=mon.0 client_cookie=a298722a827f73c2 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:21.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.566+0000 7fdea1441700 1 -- 172.21.15.36:0/2454372767 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde9400d5c0 con 0x7fde9c147bd0 2022-06-17T12:57:21.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.566+0000 7fdea1441700 1 -- 172.21.15.36:0/2454372767 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde94000e90 con 0x7fde9c147bd0 2022-06-17T12:57:21.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.566+0000 7fdea1441700 1 -- 172.21.15.36:0/2454372767 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde9400d950 con 0x7fde9c147bd0 2022-06-17T12:57:21.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.566+0000 7fdea3445700 1 -- 172.21.15.36:0/2454372767 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde9c147bd0 msgr2=0x7fde9c1442d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:21.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.566+0000 7fdea3445700 1 --2- 172.21.15.36:0/2454372767 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde9c147bd0 0x7fde9c1442d0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7fde94002f70 tx=0x7fde94006eb0).stop 2022-06-17T12:57:21.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.566+0000 7fdea3445700 1 -- 172.21.15.36:0/2454372767 shutdown_connections 2022-06-17T12:57:21.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.566+0000 7fdea3445700 1 --2- 172.21.15.36:0/2454372767 >> v2:172.21.15.70:3300/0 conn(0x7fde9c14b2b0 0x7fde9c060ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.566+0000 7fdea3445700 1 --2- 172.21.15.36:0/2454372767 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde9c147bd0 0x7fde9c1442d0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.566+0000 7fdea3445700 1 -- 172.21.15.36:0/2454372767 >> 172.21.15.36:0/2454372767 conn(0x7fde9c0b5d10 msgr2=0x7fde9c0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:21.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.566+0000 7fdea3445700 1 -- 172.21.15.36:0/2454372767 shutdown_connections 2022-06-17T12:57:21.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.566+0000 7fdea3445700 1 -- 172.21.15.36:0/2454372767 wait complete. 2022-06-17T12:57:21.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.566+0000 7fdea3445700 1 Processor -- start 2022-06-17T12:57:21.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.566+0000 7fdea3445700 1 -- start start 2022-06-17T12:57:21.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.566+0000 7fdea3445700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde9c0b27b0 0x7fde9c063120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:21.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.566+0000 7fdea3445700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde9c147bd0 0x7fde9c064180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:21.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.566+0000 7fdea3445700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde9c14b2b0 0x7fde9c064e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:21.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.566+0000 7fdea3445700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c067340 con 0x7fde9c14b2b0 2022-06-17T12:57:21.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.566+0000 7fdea3445700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c065330 con 0x7fde9c0b27b0 2022-06-17T12:57:21.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.566+0000 7fdea3445700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c0a71c0 con 0x7fde9c147bd0 2022-06-17T12:57:21.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.566+0000 7fdea2c44700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde9c14b2b0 0x7fde9c064e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:21.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.566+0000 7fdea2443700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde9c0b27b0 0x7fde9c063120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:21.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.566+0000 7fdea2c44700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde9c14b2b0 0x7fde9c064e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40928/0 (socket says 172.21.15.36:40928) 2022-06-17T12:57:21.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.566+0000 7fdea2c44700 1 -- 172.21.15.36:0/2328069142 learned_addr learned my addr 172.21.15.36:0/2328069142 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:21.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.566+0000 7fdea1c42700 1 --2- 172.21.15.36:0/2328069142 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde9c147bd0 0x7fde9c064180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:21.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.570+0000 7fdea2443700 1 -- 172.21.15.36:0/2328069142 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde9c147bd0 msgr2=0x7fde9c064180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:21.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.570+0000 7fdea2443700 1 --2- 172.21.15.36:0/2328069142 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde9c147bd0 0x7fde9c064180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.570+0000 7fdea2443700 1 -- 172.21.15.36:0/2328069142 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde9c14b2b0 msgr2=0x7fde9c064e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:21.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.570+0000 7fdea2443700 1 --2- 172.21.15.36:0/2328069142 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde9c14b2b0 0x7fde9c064e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.570+0000 7fdea2443700 1 -- 172.21.15.36:0/2328069142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde98009470 con 0x7fde9c0b27b0 2022-06-17T12:57:21.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.570+0000 7fdea2c44700 1 --2- 172.21.15.36:0/2328069142 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde9c14b2b0 0x7fde9c064e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:57:21.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.570+0000 7fdea2443700 1 --2- 172.21.15.36:0/2328069142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde9c0b27b0 0x7fde9c063120 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7fde9c0b5190 tx=0x7fde8c007b50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:21.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.570+0000 7fde8b7fe700 1 -- 172.21.15.36:0/2328069142 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde8c007030 con 0x7fde9c0b27b0 2022-06-17T12:57:21.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.570+0000 7fde8b7fe700 1 -- 172.21.15.36:0/2328069142 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8c007190 con 0x7fde9c0b27b0 2022-06-17T12:57:21.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.570+0000 7fdea3445700 1 -- 172.21.15.36:0/2328069142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde9400b1e0 con 0x7fde9c0b27b0 2022-06-17T12:57:21.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.570+0000 7fde8b7fe700 1 -- 172.21.15.36:0/2328069142 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde8c006510 con 0x7fde9c0b27b0 2022-06-17T12:57:21.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.570+0000 7fdea3445700 1 -- 172.21.15.36:0/2328069142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9c144d10 con 0x7fde9c0b27b0 2022-06-17T12:57:21.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.570+0000 7fdea3445700 1 -- 172.21.15.36:0/2328069142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde9c057410 con 0x7fde9c0b27b0 2022-06-17T12:57:21.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.570+0000 7fde8b7fe700 1 -- 172.21.15.36:0/2328069142 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fde8c010030 con 0x7fde9c0b27b0 2022-06-17T12:57:21.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.570+0000 7fde8b7fe700 1 --2- 172.21.15.36:0/2328069142 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde7c0539b0 0x7fde7c055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:21.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.570+0000 7fde8b7fe700 1 -- 172.21.15.36:0/2328069142 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fde8c06bb50 con 0x7fde9c0b27b0 2022-06-17T12:57:21.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.570+0000 7fdea1c42700 1 --2- 172.21.15.36:0/2328069142 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde7c0539b0 0x7fde7c055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:21.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.574+0000 7fdea1c42700 1 --2- 172.21.15.36:0/2328069142 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde7c0539b0 0x7fde7c055e70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7fde9800b000 tx=0x7fde980091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:21.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.582+0000 7fde8b7fe700 1 -- 172.21.15.36:0/2328069142 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fde8c042840 con 0x7fde9c0b27b0 2022-06-17T12:57:21.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.710+0000 7fdea3445700 1 -- 172.21.15.36:0/2328069142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fde9c05d9b0 con 0x7fde9c0b27b0 2022-06-17T12:57:21.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.710+0000 7fde8b7fe700 1 -- 172.21.15.36:0/2328069142 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v151) v1 ==== 104+0+10 (secure 0 0 0) 0x7fde8c045e60 con 0x7fde9c0b27b0 2022-06-17T12:57:21.705 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:57:21.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.714+0000 7fdea3445700 1 -- 172.21.15.36:0/2328069142 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde7c0539b0 msgr2=0x7fde7c055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:21.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.714+0000 7fdea3445700 1 --2- 172.21.15.36:0/2328069142 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde7c0539b0 0x7fde7c055e70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7fde9800b000 tx=0x7fde980091d0).stop 2022-06-17T12:57:21.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.714+0000 7fdea3445700 1 -- 172.21.15.36:0/2328069142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde9c0b27b0 msgr2=0x7fde9c063120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:21.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.714+0000 7fdea3445700 1 --2- 172.21.15.36:0/2328069142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde9c0b27b0 0x7fde9c063120 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7fde9c0b5190 tx=0x7fde8c007b50).stop 2022-06-17T12:57:21.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.714+0000 7fdea3445700 1 -- 172.21.15.36:0/2328069142 shutdown_connections 2022-06-17T12:57:21.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.714+0000 7fdea3445700 1 --2- 172.21.15.36:0/2328069142 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde7c0539b0 0x7fde7c055e70 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.714+0000 7fdea3445700 1 --2- 172.21.15.36:0/2328069142 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde9c14b2b0 0x7fde9c064e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.714+0000 7fdea3445700 1 --2- 172.21.15.36:0/2328069142 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde9c147bd0 0x7fde9c064180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.714+0000 7fdea3445700 1 --2- 172.21.15.36:0/2328069142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde9c0b27b0 0x7fde9c063120 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.714+0000 7fdea3445700 1 -- 172.21.15.36:0/2328069142 >> 172.21.15.36:0/2328069142 conn(0x7fde9c0b5d10 msgr2=0x7fde9c0aff00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:21.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.714+0000 7fdea3445700 1 -- 172.21.15.36:0/2328069142 shutdown_connections 2022-06-17T12:57:21.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.714+0000 7fdea3445700 1 -- 172.21.15.36:0/2328069142 wait complete. 2022-06-17T12:57:21.719 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-17T12:57:21.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.858+0000 7fe84ffff700 1 Processor -- start 2022-06-17T12:57:21.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.858+0000 7fe84ffff700 1 -- start start 2022-06-17T12:57:21.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe84ffff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe8500b6860 0x7fe850146d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:21.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe84ffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8500b18e0 0x7fe850143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:21.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe84ffff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8500b2b70 con 0x7fe8500b39a0 2022-06-17T12:57:21.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe84ffff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8500b6c40 con 0x7fe8500b6860 2022-06-17T12:57:21.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe84ffff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe850142d30 con 0x7fe8500b18e0 2022-06-17T12:57:21.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe84f7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fe8500b39a0 0x7fe850143640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58466/0 (socket says 172.21.15.36:58466) 2022-06-17T12:57:21.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe84f7fe700 1 -- 172.21.15.36:0/1413099895 learned_addr learned my addr 172.21.15.36:0/1413099895 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:21.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe84effd700 1 --2- 172.21.15.36:0/1413099895 >> v2:172.21.15.36:3300/0 conn(0x7fe8500b6860 0x7fe850146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:21.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe8467fc700 1 --2- 172.21.15.36:0/1413099895 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8500b18e0 0x7fe850143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:21.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe84e7fc700 1 -- 172.21.15.36:0/1413099895 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1893002689 0 0) 0x7fe8500b2b70 con 0x7fe8500b39a0 2022-06-17T12:57:21.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe84e7fc700 1 -- 172.21.15.36:0/1413099895 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe830003260 con 0x7fe8500b39a0 2022-06-17T12:57:21.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe8467fc700 1 -- 172.21.15.36:0/1413099895 >> v2:172.21.15.36:3300/0 conn(0x7fe8500b6860 msgr2=0x7fe850146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:21.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe8467fc700 1 --2- 172.21.15.36:0/1413099895 >> v2:172.21.15.36:3300/0 conn(0x7fe8500b6860 0x7fe850146d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe8467fc700 1 -- 172.21.15.36:0/1413099895 >> v1:172.21.15.36:6789/0 conn(0x7fe8500b39a0 legacy=0x7fe850143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:21.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe8467fc700 1 -- 172.21.15.36:0/1413099895 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8500b0d30 con 0x7fe8500b18e0 2022-06-17T12:57:21.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe8467fc700 1 --2- 172.21.15.36:0/1413099895 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8500b18e0 0x7fe850143150 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7fe838002f70 tx=0x7fe838006d90).ready entity=mon.2 client_cookie=bcb80229b64eece9 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:21.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe84e7fc700 1 -- 172.21.15.36:0/1413099895 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe83800e590 con 0x7fe8500b18e0 2022-06-17T12:57:21.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe84e7fc700 1 -- 172.21.15.36:0/1413099895 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe838000e90 con 0x7fe8500b18e0 2022-06-17T12:57:21.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe84e7fc700 1 -- 172.21.15.36:0/1413099895 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe83800e8e0 con 0x7fe8500b18e0 2022-06-17T12:57:21.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe84ffff700 1 -- 172.21.15.36:0/1413099895 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8500b18e0 msgr2=0x7fe850143150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:21.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe84ffff700 1 --2- 172.21.15.36:0/1413099895 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8500b18e0 0x7fe850143150 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7fe838002f70 tx=0x7fe838006d90).stop 2022-06-17T12:57:21.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe84ffff700 1 -- 172.21.15.36:0/1413099895 shutdown_connections 2022-06-17T12:57:21.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe84ffff700 1 --2- 172.21.15.36:0/1413099895 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8500b18e0 0x7fe850143150 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe84ffff700 1 --2- 172.21.15.36:0/1413099895 >> v2:172.21.15.36:3300/0 conn(0x7fe8500b6860 0x7fe850146d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe84ffff700 1 -- 172.21.15.36:0/1413099895 >> 172.21.15.36:0/1413099895 conn(0x7fe8500614b0 msgr2=0x7fe850061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:21.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe84ffff700 1 -- 172.21.15.36:0/1413099895 shutdown_connections 2022-06-17T12:57:21.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.862+0000 7fe84ffff700 1 -- 172.21.15.36:0/1413099895 wait complete. 2022-06-17T12:57:21.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.866+0000 7fe84ffff700 1 Processor -- start 2022-06-17T12:57:21.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.866+0000 7fe84ffff700 1 -- start start 2022-06-17T12:57:21.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.866+0000 7fe84ffff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8500b18e0 0x7fe8501465b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:21.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.866+0000 7fe84ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8500b39a0 0x7fe850140320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:21.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.866+0000 7fe84ffff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8500b6860 0x7fe850140810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:21.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.866+0000 7fe84ffff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8501564e0 con 0x7fe8500b6860 2022-06-17T12:57:21.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.866+0000 7fe84ffff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe850156240 con 0x7fe8500b39a0 2022-06-17T12:57:21.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.866+0000 7fe84ffff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe850155f60 con 0x7fe8500b18e0 2022-06-17T12:57:21.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.866+0000 7fe84f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8500b6860 0x7fe850140810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:21.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.866+0000 7fe84f7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8500b6860 0x7fe850140810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40942/0 (socket says 172.21.15.36:40942) 2022-06-17T12:57:21.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.866+0000 7fe84f7fe700 1 -- 172.21.15.36:0/2503977465 learned_addr learned my addr 172.21.15.36:0/2503977465 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:21.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.866+0000 7fe8467fc700 1 --2- 172.21.15.36:0/2503977465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8500b39a0 0x7fe850140320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:21.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.866+0000 7fe84effd700 1 --2- 172.21.15.36:0/2503977465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8500b18e0 0x7fe8501465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:21.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.870+0000 7fe84f7fe700 1 -- 172.21.15.36:0/2503977465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8500b18e0 msgr2=0x7fe8501465b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:21.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.870+0000 7fe84f7fe700 1 --2- 172.21.15.36:0/2503977465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8500b18e0 0x7fe8501465b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.870+0000 7fe84f7fe700 1 -- 172.21.15.36:0/2503977465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8500b39a0 msgr2=0x7fe850140320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:21.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.870+0000 7fe84f7fe700 1 --2- 172.21.15.36:0/2503977465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8500b39a0 0x7fe850140320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:21.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.870+0000 7fe84f7fe700 1 -- 172.21.15.36:0/2503977465 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83800b120 con 0x7fe8500b6860 2022-06-17T12:57:21.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.870+0000 7fe8467fc700 1 --2- 172.21.15.36:0/2503977465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8500b39a0 0x7fe850140320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:57:21.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.870+0000 7fe84effd700 1 --2- 172.21.15.36:0/2503977465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8500b18e0 0x7fe8501465b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T12:57:21.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.870+0000 7fe84f7fe700 1 --2- 172.21.15.36:0/2503977465 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8500b6860 0x7fe850140810 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 rx=0x7fe848001490 tx=0x7fe848007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:21.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.870+0000 7fe84cff9700 1 -- 172.21.15.36:0/2503977465 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe848005ba0 con 0x7fe8500b6860 2022-06-17T12:57:21.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.870+0000 7fe84cff9700 1 -- 172.21.15.36:0/2503977465 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe848005d00 con 0x7fe8500b6860 2022-06-17T12:57:21.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.870+0000 7fe84ffff700 1 -- 172.21.15.36:0/2503977465 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8500667a0 con 0x7fe8500b6860 2022-06-17T12:57:21.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.870+0000 7fe84cff9700 1 -- 172.21.15.36:0/2503977465 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8480072c0 con 0x7fe8500b6860 2022-06-17T12:57:21.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.870+0000 7fe84ffff700 1 -- 172.21.15.36:0/2503977465 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe850146aa0 con 0x7fe8500b6860 2022-06-17T12:57:21.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.870+0000 7fe84ffff700 1 -- 172.21.15.36:0/2503977465 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe850057410 con 0x7fe8500b6860 2022-06-17T12:57:21.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.870+0000 7fe84cff9700 1 -- 172.21.15.36:0/2503977465 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe84801a060 con 0x7fe8500b6860 2022-06-17T12:57:21.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.870+0000 7fe84cff9700 1 --2- 172.21.15.36:0/2503977465 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe834053920 0x7fe834055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:21.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.870+0000 7fe84effd700 1 --2- 172.21.15.36:0/2503977465 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe834053920 0x7fe834055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:21.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.870+0000 7fe84cff9700 1 -- 172.21.15.36:0/2503977465 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe84806bba0 con 0x7fe8500b6860 2022-06-17T12:57:21.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.874+0000 7fe84effd700 1 --2- 172.21.15.36:0/2503977465 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe834053920 0x7fe834055de0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fe84000b000 tx=0x7fe840008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:21.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:21.878+0000 7fe84cff9700 1 -- 172.21.15.36:0/2503977465 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe848042900 con 0x7fe8500b6860 2022-06-17T12:57:21.994 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.006+0000 7fe84ffff700 1 -- 172.21.15.36:0/2503977465 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe8500b0960 con 0x7fe8500b6860 2022-06-17T12:57:21.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.010+0000 7fe84cff9700 1 -- 172.21.15.36:0/2503977465 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v151) v1 ==== 100+0+10 (secure 0 0 0) 0x7fe848018030 con 0x7fe8500b6860 2022-06-17T12:57:21.996 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-06-17T12:57:22.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.010+0000 7fe84ffff700 1 -- 172.21.15.36:0/2503977465 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe834053920 msgr2=0x7fe834055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:22.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.010+0000 7fe84ffff700 1 --2- 172.21.15.36:0/2503977465 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe834053920 0x7fe834055de0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fe84000b000 tx=0x7fe840008040).stop 2022-06-17T12:57:22.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.010+0000 7fe84ffff700 1 -- 172.21.15.36:0/2503977465 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8500b6860 msgr2=0x7fe850140810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:22.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.010+0000 7fe84ffff700 1 --2- 172.21.15.36:0/2503977465 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8500b6860 0x7fe850140810 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 rx=0x7fe848001490 tx=0x7fe848007e00).stop 2022-06-17T12:57:22.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.010+0000 7fe84ffff700 1 -- 172.21.15.36:0/2503977465 shutdown_connections 2022-06-17T12:57:22.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.010+0000 7fe84ffff700 1 --2- 172.21.15.36:0/2503977465 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe834053920 0x7fe834055de0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:22.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.010+0000 7fe84ffff700 1 --2- 172.21.15.36:0/2503977465 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe8500b6860 0x7fe850140810 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:22.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.010+0000 7fe84ffff700 1 --2- 172.21.15.36:0/2503977465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8500b39a0 0x7fe850140320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:22.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.010+0000 7fe84ffff700 1 --2- 172.21.15.36:0/2503977465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe8500b18e0 0x7fe8501465b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:22.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.010+0000 7fe84ffff700 1 -- 172.21.15.36:0/2503977465 >> 172.21.15.36:0/2503977465 conn(0x7fe8500614b0 msgr2=0x7fe850141600 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:22.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.010+0000 7fe84ffff700 1 -- 172.21.15.36:0/2503977465 shutdown_connections 2022-06-17T12:57:22.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.010+0000 7fe84ffff700 1 -- 172.21.15.36:0/2503977465 wait complete. 2022-06-17T12:57:22.036 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-17T12:57:22.037 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:57:22.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.158+0000 7f121bfff700 1 Processor -- start 2022-06-17T12:57:22.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.158+0000 7f121bfff700 1 -- start start 2022-06-17T12:57:22.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.158+0000 7f121bfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f121c0b80c0 0x7f121c140c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:22.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.158+0000 7f121bfff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f121c142d70 0x7f121c143220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:22.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.158+0000 7f121bfff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f121c142160 con 0x7f121c0b7ce0 2022-06-17T12:57:22.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.158+0000 7f121bfff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f121c0b4f30 con 0x7f121c0b80c0 2022-06-17T12:57:22.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.158+0000 7f121bfff700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f121c142390 con 0x7f121c142d70 2022-06-17T12:57:22.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.162+0000 7f121a7fc700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f121c0b80c0 0x7f121c140c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:22.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.162+0000 7f121affd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f121c0b7ce0 0x7f121c13aa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58476/0 (socket says 172.21.15.36:58476) 2022-06-17T12:57:22.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.162+0000 7f121affd700 1 -- 172.21.15.36:0/2493127415 learned_addr learned my addr 172.21.15.36:0/2493127415 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:22.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.162+0000 7f121b7fe700 1 --2- 172.21.15.36:0/2493127415 >> v2:172.21.15.70:3300/0 conn(0x7f121c142d70 0x7f121c143220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:22.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.162+0000 7f121a7fc700 1 -- 172.21.15.36:0/2493127415 >> v2:172.21.15.70:3300/0 conn(0x7f121c142d70 msgr2=0x7f121c143220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:22.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.162+0000 7f121a7fc700 1 --2- 172.21.15.36:0/2493127415 >> v2:172.21.15.70:3300/0 conn(0x7f121c142d70 0x7f121c143220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:22.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.162+0000 7f121a7fc700 1 -- 172.21.15.36:0/2493127415 >> v1:172.21.15.36:6789/0 conn(0x7f121c0b7ce0 legacy=0x7f121c13aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:22.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.162+0000 7f121a7fc700 1 -- 172.21.15.36:0/2493127415 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f121c142630 con 0x7f121c0b80c0 2022-06-17T12:57:22.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.162+0000 7f121a7fc700 1 --2- 172.21.15.36:0/2493127415 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f121c0b80c0 0x7f121c140c90 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f1204002f70 tx=0x7f1204006df0).ready entity=mon.0 client_cookie=262270602576462a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:22.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.162+0000 7f1219ffb700 1 -- 172.21.15.36:0/2493127415 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f120400d570 con 0x7f121c0b80c0 2022-06-17T12:57:22.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.162+0000 7f1219ffb700 1 -- 172.21.15.36:0/2493127415 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1204000e90 con 0x7f121c0b80c0 2022-06-17T12:57:22.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.162+0000 7f1219ffb700 1 -- 172.21.15.36:0/2493127415 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f120400d8c0 con 0x7f121c0b80c0 2022-06-17T12:57:22.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.162+0000 7f121bfff700 1 -- 172.21.15.36:0/2493127415 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f121c0b80c0 msgr2=0x7f121c140c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:22.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.162+0000 7f121bfff700 1 --2- 172.21.15.36:0/2493127415 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f121c0b80c0 0x7f121c140c90 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f1204002f70 tx=0x7f1204006df0).stop 2022-06-17T12:57:22.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.162+0000 7f121bfff700 1 -- 172.21.15.36:0/2493127415 shutdown_connections 2022-06-17T12:57:22.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.162+0000 7f121bfff700 1 --2- 172.21.15.36:0/2493127415 >> v2:172.21.15.70:3300/0 conn(0x7f121c142d70 0x7f121c143220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:22.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.162+0000 7f121bfff700 1 --2- 172.21.15.36:0/2493127415 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f121c0b80c0 0x7f121c140c90 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:22.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.162+0000 7f121bfff700 1 -- 172.21.15.36:0/2493127415 >> 172.21.15.36:0/2493127415 conn(0x7f121c05c550 msgr2=0x7f121c0b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:22.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.162+0000 7f121bfff700 1 -- 172.21.15.36:0/2493127415 shutdown_connections 2022-06-17T12:57:22.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.162+0000 7f121bfff700 1 -- 172.21.15.36:0/2493127415 wait complete. 2022-06-17T12:57:22.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.166+0000 7f121bfff700 1 Processor -- start 2022-06-17T12:57:22.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.166+0000 7f121bfff700 1 -- start start 2022-06-17T12:57:22.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.166+0000 7f121bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f121c0b7ce0 0x7f121c066d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:22.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.166+0000 7f121bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f121c0b80c0 0x7f121c13e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:22.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.166+0000 7f121bfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f121c142d70 0x7f121c13e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:22.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.166+0000 7f121bfff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121c13ee20 con 0x7f121c0b7ce0 2022-06-17T12:57:22.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.166+0000 7f121bfff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121c150aa0 con 0x7f121c0b80c0 2022-06-17T12:57:22.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.166+0000 7f121bfff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f121c150d40 con 0x7f121c142d70 2022-06-17T12:57:22.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.166+0000 7f121affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f121c0b7ce0 0x7f121c066d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:22.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.166+0000 7f121affd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f121c0b7ce0 0x7f121c066d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40952/0 (socket says 172.21.15.36:40952) 2022-06-17T12:57:22.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.166+0000 7f121affd700 1 -- 172.21.15.36:0/3037796395 learned_addr learned my addr 172.21.15.36:0/3037796395 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:22.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.166+0000 7f121a7fc700 1 --2- 172.21.15.36:0/3037796395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f121c0b80c0 0x7f121c13e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:22.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.166+0000 7f121b7fe700 1 --2- 172.21.15.36:0/3037796395 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f121c142d70 0x7f121c13e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:22.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.166+0000 7f121affd700 1 -- 172.21.15.36:0/3037796395 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f121c142d70 msgr2=0x7f121c13e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:22.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.166+0000 7f121affd700 1 --2- 172.21.15.36:0/3037796395 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f121c142d70 0x7f121c13e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:22.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.166+0000 7f121affd700 1 -- 172.21.15.36:0/3037796395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f121c0b80c0 msgr2=0x7f121c13e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:22.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.166+0000 7f121affd700 1 --2- 172.21.15.36:0/3037796395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f121c0b80c0 0x7f121c13e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:22.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.166+0000 7f121affd700 1 -- 172.21.15.36:0/3037796395 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120400b180 con 0x7f121c0b7ce0 2022-06-17T12:57:22.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.166+0000 7f121affd700 1 --2- 172.21.15.36:0/3037796395 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f121c0b7ce0 0x7f121c066d90 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 rx=0x7f121c141da0 tx=0x7f120c007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:22.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.170+0000 7f1213fff700 1 -- 172.21.15.36:0/3037796395 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f120c00ce30 con 0x7f121c0b7ce0 2022-06-17T12:57:22.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.170+0000 7f1213fff700 1 -- 172.21.15.36:0/3037796395 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120c0072e0 con 0x7f121c0b7ce0 2022-06-17T12:57:22.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.170+0000 7f121bfff700 1 -- 172.21.15.36:0/3037796395 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f121c0660c0 con 0x7f121c0b7ce0 2022-06-17T12:57:22.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.170+0000 7f1213fff700 1 -- 172.21.15.36:0/3037796395 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f120c005800 con 0x7f121c0b7ce0 2022-06-17T12:57:22.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.170+0000 7f121bfff700 1 -- 172.21.15.36:0/3037796395 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f121c14f630 con 0x7f121c0b7ce0 2022-06-17T12:57:22.245 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.170+0000 7f121bfff700 1 -- 172.21.15.36:0/3037796395 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f121c150910 con 0x7f121c0b7ce0 2022-06-17T12:57:22.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.170+0000 7f1213fff700 1 -- 172.21.15.36:0/3037796395 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f120c010070 con 0x7f121c0b7ce0 2022-06-17T12:57:22.246 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.170+0000 7f1213fff700 1 --2- 172.21.15.36:0/3037796395 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f12000539b0 0x7f1200055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:22.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.170+0000 7f1213fff700 1 -- 172.21.15.36:0/3037796395 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f120c06b6e0 con 0x7f121c0b7ce0 2022-06-17T12:57:22.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.170+0000 7f121a7fc700 1 --2- 172.21.15.36:0/3037796395 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f12000539b0 0x7f1200055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:22.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.174+0000 7f121a7fc700 1 --2- 172.21.15.36:0/3037796395 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f12000539b0 0x7f1200055e70 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f1204018000 tx=0x7f1204014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:22.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.178+0000 7f1213fff700 1 -- 172.21.15.36:0/3037796395 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f120c0423d0 con 0x7f121c0b7ce0 2022-06-17T12:57:22.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.338+0000 7f121bfff700 1 -- 172.21.15.36:0/3037796395 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f121c150f00 con 0x7f121c0b7ce0 2022-06-17T12:57:22.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.338+0000 7f1213fff700 1 -- 172.21.15.36:0/3037796395 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f120c0459f0 con 0x7f121c0b7ce0 2022-06-17T12:57:22.327 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:57:22.328 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":83,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8599,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/4216085219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4216085219},{"type":"v1","addr":"172.21.15.70:6833","nonce":4216085219}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":82}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:57:14.606323+0000","modified":"2022-06-17T12:57:15.610154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8805":{"gid":8805,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/2534551587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2534551587},{"type":"v1","addr":"172.21.15.36:6835","nonce":2534551587}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-06-17T12:57:22.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.342+0000 7f121bfff700 1 -- 172.21.15.36:0/3037796395 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f12000539b0 msgr2=0x7f1200055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:22.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.342+0000 7f121bfff700 1 --2- 172.21.15.36:0/3037796395 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f12000539b0 0x7f1200055e70 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f1204018000 tx=0x7f1204014000).stop 2022-06-17T12:57:22.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.342+0000 7f121bfff700 1 -- 172.21.15.36:0/3037796395 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f121c0b7ce0 msgr2=0x7f121c066d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:22.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.342+0000 7f121bfff700 1 --2- 172.21.15.36:0/3037796395 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f121c0b7ce0 0x7f121c066d90 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 rx=0x7f121c141da0 tx=0x7f120c007c40).stop 2022-06-17T12:57:22.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.342+0000 7f121bfff700 1 -- 172.21.15.36:0/3037796395 shutdown_connections 2022-06-17T12:57:22.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.342+0000 7f121bfff700 1 --2- 172.21.15.36:0/3037796395 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f12000539b0 0x7f1200055e70 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:22.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.342+0000 7f121bfff700 1 --2- 172.21.15.36:0/3037796395 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f121c142d70 0x7f121c13e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:22.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.342+0000 7f121bfff700 1 --2- 172.21.15.36:0/3037796395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f121c0b80c0 0x7f121c13e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:22.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.342+0000 7f121bfff700 1 --2- 172.21.15.36:0/3037796395 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f121c0b7ce0 0x7f121c066d90 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:22.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.342+0000 7f121bfff700 1 -- 172.21.15.36:0/3037796395 >> 172.21.15.36:0/3037796395 conn(0x7f121c05c550 msgr2=0x7f121c1408d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:22.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.342+0000 7f121bfff700 1 -- 172.21.15.36:0/3037796395 shutdown_connections 2022-06-17T12:57:22.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:22.342+0000 7f121bfff700 1 -- 172.21.15.36:0/3037796395 wait complete. 2022-06-17T12:57:22.354 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 83 2022-06-17T12:57:22.559 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-17T12:57:14.606323+0000', 'modified': '2022-06-17T12:57:15.610154+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8805}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8805': {'gid': 8805, 'name': 'a', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.36:6835/2534551587', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.36:6834', 'nonce': 2534551587}, {'type': 'v1', 'addr': '172.21.15.36:6835', 'nonce': 2534551587}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-17T12:57:22.562 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-17T12:57:22.566 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-17T12:57:22.693 INFO:teuthology.orchestra.run.smithi036.stdout:{"cluster_fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","whoami":0,"id":8805,"want_state":"up:active","state":"up:active","rank_uptime":8.0852105329999997,"mdsmap_epoch":83,"osdmap_epoch":151,"osdmap_epoch_barrier":149,"uptime":12.478754124} 2022-06-17T12:57:22.694 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", "id": 8805, "mdsmap_epoch": 83, "osdmap_epoch": 151, "osdmap_epoch_barrier": 149, "rank_uptime": 8.085210533, "state": "up:active", "uptime": 12.478754124, "want_state": "up:active", "whoami": 0 } 2022-06-17T12:57:22.695 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-17T12:57:22.696 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-06-17T12:57:22.697 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi136.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-06-17T12:57:22.697 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:57:22.698 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:57:22.717 DEBUG:teuthology.orchestra.run.smithi136:> sudo modprobe fuse 2022-06-17T12:57:22.801 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:57:22.802 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:57:22.859 INFO:teuthology.orchestra.run.smithi136.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:57:22.861 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:57:22.862 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:57:22.862 DEBUG:teuthology.orchestra.run.smithi136:> ls /sys/fs/fuse/connections 2022-06-17T12:57:22.869 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-17T12:57:22.869 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:57:22.873 DEBUG:teuthology.orchestra.run.smithi136:> sudo modprobe fuse 2022-06-17T12:57:22.893 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-17T12:57:22.899 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-17T12:57:22.950 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:2022-06-17T12:57:22.960+0000 7f34ce49d080 -1 init, newargv = 0x5580323ad4f0 newargc=9 2022-06-17T12:57:22.950 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:ceph-fuse[22568]: starting ceph client 2022-06-17T12:57:22.960 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-17T12:57:22.963 INFO:teuthology.orchestra.run.smithi136.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-17T12:57:22.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:57:22.967 DEBUG:teuthology.orchestra.run.smithi136:> ls /sys/fs/fuse/connections 2022-06-17T12:57:22.971 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:ceph-fuse[22568]: starting fuse 2022-06-17T12:57:23.044 INFO:teuthology.orchestra.run.smithi136.stdout:52 2022-06-17T12:57:23.045 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-17T12:57:23.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:57:23.048 DEBUG:teuthology.orchestra.run.smithi136:> sudo python3 -c ' 2022-06-17T12:57:23.049 DEBUG:teuthology.orchestra.run.smithi136:> import glob 2022-06-17T12:57:23.050 DEBUG:teuthology.orchestra.run.smithi136:> import re 2022-06-17T12:57:23.058 DEBUG:teuthology.orchestra.run.smithi136:> import os 2022-06-17T12:57:23.059 DEBUG:teuthology.orchestra.run.smithi136:> import subprocess 2022-06-17T12:57:23.060 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:57:23.060 DEBUG:teuthology.orchestra.run.smithi136:> def find_socket(client_name): 2022-06-17T12:57:23.061 DEBUG:teuthology.orchestra.run.smithi136:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-06-17T12:57:23.065 DEBUG:teuthology.orchestra.run.smithi136:> files = glob.glob(asok_path) 2022-06-17T12:57:23.065 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:57:23.066 DEBUG:teuthology.orchestra.run.smithi136:> # Given a non-glob path, it better be there 2022-06-17T12:57:23.066 DEBUG:teuthology.orchestra.run.smithi136:> if "*" not in asok_path: 2022-06-17T12:57:23.066 DEBUG:teuthology.orchestra.run.smithi136:> assert(len(files) == 1) 2022-06-17T12:57:23.067 DEBUG:teuthology.orchestra.run.smithi136:> return files[0] 2022-06-17T12:57:23.067 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:57:23.068 DEBUG:teuthology.orchestra.run.smithi136:> for f in files: 2022-06-17T12:57:23.068 DEBUG:teuthology.orchestra.run.smithi136:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-17T12:57:23.069 DEBUG:teuthology.orchestra.run.smithi136:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-17T12:57:23.070 DEBUG:teuthology.orchestra.run.smithi136:> return f 2022-06-17T12:57:23.071 DEBUG:teuthology.orchestra.run.smithi136:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-17T12:57:23.071 DEBUG:teuthology.orchestra.run.smithi136:> 2022-06-17T12:57:23.071 DEBUG:teuthology.orchestra.run.smithi136:> print(find_socket("client.1")) 2022-06-17T12:57:23.072 DEBUG:teuthology.orchestra.run.smithi136:> ' 2022-06-17T12:57:23.140 INFO:teuthology.orchestra.run.smithi136.stdout:/var/run/ceph/ceph-client.1.22568.asok 2022-06-17T12:57:23.147 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22568.asok 2022-06-17T12:57:23.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:57:23.165 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22568.asok status 2022-06-17T12:57:23.262 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2022-06-17T12:57:23.263 INFO:teuthology.orchestra.run.smithi136.stdout: "metadata": { 2022-06-17T12:57:23.263 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_sha1": "b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6", 2022-06-17T12:57:23.264 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_version": "ceph version 15.2.16-260-gb3641a55 (b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6) octopus (stable)", 2022-06-17T12:57:23.280 INFO:teuthology.orchestra.run.smithi136.stdout: "entity_id": "1", 2022-06-17T12:57:23.281 INFO:teuthology.orchestra.run.smithi136.stdout: "hostname": "smithi136", 2022-06-17T12:57:23.281 INFO:teuthology.orchestra.run.smithi136.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-06-17T12:57:23.282 INFO:teuthology.orchestra.run.smithi136.stdout: "pid": "22568", 2022-06-17T12:57:23.282 INFO:teuthology.orchestra.run.smithi136.stdout: "root": "/" 2022-06-17T12:57:23.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:57:23.284 INFO:teuthology.orchestra.run.smithi136.stdout: "dentry_count": 0, 2022-06-17T12:57:23.285 INFO:teuthology.orchestra.run.smithi136.stdout: "dentry_pinned_count": 0, 2022-06-17T12:57:23.285 INFO:teuthology.orchestra.run.smithi136.stdout: "id": 9009, 2022-06-17T12:57:23.286 INFO:teuthology.orchestra.run.smithi136.stdout: "inst": { 2022-06-17T12:57:23.288 INFO:teuthology.orchestra.run.smithi136.stdout: "name": { 2022-06-17T12:57:23.288 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "client", 2022-06-17T12:57:23.289 INFO:teuthology.orchestra.run.smithi136.stdout: "num": 9009 2022-06-17T12:57:23.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:57:23.290 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": { 2022-06-17T12:57:23.290 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "v1", 2022-06-17T12:57:23.291 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": "172.21.15.136:0", 2022-06-17T12:57:23.291 INFO:teuthology.orchestra.run.smithi136.stdout: "nonce": 3912096541 2022-06-17T12:57:23.292 INFO:teuthology.orchestra.run.smithi136.stdout: } 2022-06-17T12:57:23.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:57:23.293 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": { 2022-06-17T12:57:23.293 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "v1", 2022-06-17T12:57:23.293 INFO:teuthology.orchestra.run.smithi136.stdout: "addr": "172.21.15.136:0", 2022-06-17T12:57:23.294 INFO:teuthology.orchestra.run.smithi136.stdout: "nonce": 3912096541 2022-06-17T12:57:23.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2022-06-17T12:57:23.295 INFO:teuthology.orchestra.run.smithi136.stdout: "inst_str": "client.9009 172.21.15.136:0/3912096541", 2022-06-17T12:57:23.295 INFO:teuthology.orchestra.run.smithi136.stdout: "addr_str": "172.21.15.136:0/3912096541", 2022-06-17T12:57:23.296 INFO:teuthology.orchestra.run.smithi136.stdout: "inode_count": 1, 2022-06-17T12:57:23.296 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_epoch": 83, 2022-06-17T12:57:23.297 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_epoch": 151, 2022-06-17T12:57:23.299 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_epoch_barrier": 0, 2022-06-17T12:57:23.299 INFO:teuthology.orchestra.run.smithi136.stdout: "blacklisted": false, 2022-06-17T12:57:23.300 INFO:teuthology.orchestra.run.smithi136.stdout: "fs_name": "cephfs" 2022-06-17T12:57:23.300 INFO:teuthology.orchestra.run.smithi136.stdout:} 2022-06-17T12:57:23.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:57:23.305 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:57:23.326 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:57:23.337 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:57:23.338 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:57:23.339 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:57:23.348 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:57:23.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:57:23.378 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:57:23.378 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:57:23.436 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:57:23.437 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:57:23.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:57:23.449 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:57:23.449 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:57:23.463 INFO:teuthology.orchestra.run.smithi136.stdout:fuseblk 2022-06-17T12:57:23.463 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-17T12:57:23.464 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi136.front.sepia.ceph.com... 2022-06-17T12:57:23.480 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-17T12:57:23.480 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:57:23.543 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi136.stderr:ceph-fuse[22568]: fuse finished with error 0 and tester_r 0 2022-06-17T12:57:23.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:57:23.565 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:57:23.569 DEBUG:teuthology.orchestra.run.smithi136:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:57:23.619 INFO:teuthology.orchestra.run.smithi136.stdout:ext2/ext3 2022-06-17T12:57:23.619 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-17T12:57:23.620 INFO:teuthology.orchestra.run:waiting for 900 2022-06-17T12:57:29.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-17T12:57:29.726 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-17T12:57:29.738 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-06-17T12:57:29.738 DEBUG:teuthology.orchestra.run.smithi107:> cd /home/ubuntu/cephtest/mnt.0 && sudo bash -c 'stat -f .' 2022-06-17T12:57:29.768 INFO:teuthology.orchestra.run.smithi107.stdout: File: "." 2022-06-17T12:57:29.769 INFO:teuthology.orchestra.run.smithi107.stdout: ID: 0 Namelen: 255 Type: fuseblk 2022-06-17T12:57:29.769 INFO:teuthology.orchestra.run.smithi107.stdout:Block size: 4194304 Fundamental block size: 4194304 2022-06-17T12:57:29.770 INFO:teuthology.orchestra.run.smithi107.stdout:Blocks: Total: 86526 Free: 86526 Available: 86526 2022-06-17T12:57:29.770 INFO:teuthology.orchestra.run.smithi107.stdout:Inodes: Total: 1 Free: 0 2022-06-17T12:57:29.771 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-17T12:57:29.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.914+0000 7f6a9f59e700 1 Processor -- start 2022-06-17T12:57:29.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.914+0000 7f6a9f59e700 1 -- start start 2022-06-17T12:57:29.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.914+0000 7f6a9f59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6aa00b27b0 0x7f6aa0146520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:29.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.914+0000 7f6a9f59e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6aa0060ca0 0x7f6aa0146a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:29.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.914+0000 7f6a9f59e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa00b2b90 con 0x7f6aa00aeac0 2022-06-17T12:57:29.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.914+0000 7f6a9f59e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa0066b90 con 0x7f6aa0060ca0 2022-06-17T12:57:29.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.914+0000 7f6a9f59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa00af610 con 0x7f6aa00b27b0 2022-06-17T12:57:29.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.918+0000 7f6a97fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6aa0060ca0 0x7f6aa0146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:29.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.918+0000 7f6a9e59c700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6aa00b27b0 0x7f6aa0146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:29.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.918+0000 7f6a97fff700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6aa0060ca0 0x7f6aa0146a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:33110/0 (socket says 172.21.15.36:33110) 2022-06-17T12:57:29.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.918+0000 7f6a97fff700 1 -- 172.21.15.36:0/365358398 learned_addr learned my addr 172.21.15.36:0/365358398 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:29.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.918+0000 7f6a97fff700 1 -- 172.21.15.36:0/365358398 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6aa00b27b0 msgr2=0x7f6aa0146520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:29.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.918+0000 7f6a97fff700 1 --2- 172.21.15.36:0/365358398 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6aa00b27b0 0x7f6aa0146520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:29.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.918+0000 7f6a97fff700 1 -- 172.21.15.36:0/365358398 >> v1:172.21.15.70:6789/0 conn(0x7f6aa00aeac0 legacy=0x7f6aa0142350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:29.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.918+0000 7f6a97fff700 1 -- 172.21.15.36:0/365358398 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa00620f0 con 0x7f6aa0060ca0 2022-06-17T12:57:29.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.918+0000 7f6a97fff700 1 --2- 172.21.15.36:0/365358398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6aa0060ca0 0x7f6aa0146a10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f6a88002f70 tx=0x7f6a88006eb0).ready entity=mon.1 client_cookie=fa58204e7605551e server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:29.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.918+0000 7f6a9dd9b700 1 -- 172.21.15.36:0/365358398 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a8800d570 con 0x7f6aa0060ca0 2022-06-17T12:57:29.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.918+0000 7f6a9dd9b700 1 -- 172.21.15.36:0/365358398 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a88000e90 con 0x7f6aa0060ca0 2022-06-17T12:57:29.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.918+0000 7f6a9dd9b700 1 -- 172.21.15.36:0/365358398 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a8800d900 con 0x7f6aa0060ca0 2022-06-17T12:57:29.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.918+0000 7f6a9f59e700 1 -- 172.21.15.36:0/365358398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6aa0060ca0 msgr2=0x7f6aa0146a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:29.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.918+0000 7f6a9f59e700 1 --2- 172.21.15.36:0/365358398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6aa0060ca0 0x7f6aa0146a10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f6a88002f70 tx=0x7f6a88006eb0).stop 2022-06-17T12:57:29.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.918+0000 7f6a9f59e700 1 -- 172.21.15.36:0/365358398 shutdown_connections 2022-06-17T12:57:29.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.918+0000 7f6a9f59e700 1 --2- 172.21.15.36:0/365358398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6aa0060ca0 0x7f6aa0146a10 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:29.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.918+0000 7f6a9f59e700 1 --2- 172.21.15.36:0/365358398 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6aa00b27b0 0x7f6aa0146520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:29.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.918+0000 7f6a9f59e700 1 -- 172.21.15.36:0/365358398 >> 172.21.15.36:0/365358398 conn(0x7f6aa00b5d10 msgr2=0x7f6aa00b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:29.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.918+0000 7f6a9f59e700 1 -- 172.21.15.36:0/365358398 shutdown_connections 2022-06-17T12:57:29.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.918+0000 7f6a9f59e700 1 -- 172.21.15.36:0/365358398 wait complete. 2022-06-17T12:57:29.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.922+0000 7f6a9f59e700 1 Processor -- start 2022-06-17T12:57:29.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.922+0000 7f6a9f59e700 1 -- start start 2022-06-17T12:57:29.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.922+0000 7f6a9f59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6aa0060ca0 0x7f6aa00a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:29.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.922+0000 7f6a9f59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6aa00b27b0 0x7f6aa00a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:29.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.922+0000 7f6a9f59e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6aa013f050 0x7f6aa013f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:29.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.922+0000 7f6a9f59e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa00a0cc0 con 0x7f6aa00b27b0 2022-06-17T12:57:29.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.922+0000 7f6a9f59e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa0146cd0 con 0x7f6aa013f050 2022-06-17T12:57:29.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.922+0000 7f6a9f59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa013f9b0 con 0x7f6aa0060ca0 2022-06-17T12:57:29.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.922+0000 7f6a97fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6aa00b27b0 0x7f6aa00a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:29.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.922+0000 7f6a97fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6aa00b27b0 0x7f6aa00a1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40968/0 (socket says 172.21.15.36:40968) 2022-06-17T12:57:29.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.922+0000 7f6a97fff700 1 -- 172.21.15.36:0/605259383 learned_addr learned my addr 172.21.15.36:0/605259383 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:29.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.922+0000 7f6a9e59c700 1 --2- 172.21.15.36:0/605259383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6aa0060ca0 0x7f6aa00a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:29.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.922+0000 7f6a9ed9d700 1 --2- 172.21.15.36:0/605259383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6aa013f050 0x7f6aa013f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:29.926 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.926+0000 7f6a97fff700 1 -- 172.21.15.36:0/605259383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6aa0060ca0 msgr2=0x7f6aa00a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:29.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.926+0000 7f6a97fff700 1 --2- 172.21.15.36:0/605259383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6aa0060ca0 0x7f6aa00a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:29.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.926+0000 7f6a97fff700 1 -- 172.21.15.36:0/605259383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6aa013f050 msgr2=0x7f6aa013f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:29.927 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.926+0000 7f6a97fff700 1 --2- 172.21.15.36:0/605259383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6aa013f050 0x7f6aa013f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:29.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.926+0000 7f6a97fff700 1 -- 172.21.15.36:0/605259383 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8800b1e0 con 0x7f6aa00b27b0 2022-06-17T12:57:29.928 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.926+0000 7f6a9e59c700 1 --2- 172.21.15.36:0/605259383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6aa0060ca0 0x7f6aa00a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:57:29.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.926+0000 7f6a97fff700 1 --2- 172.21.15.36:0/605259383 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6aa00b27b0 0x7f6aa00a1f70 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 rx=0x7f6a88002f70 tx=0x7f6a88015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:29.929 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.926+0000 7f6a977fe700 1 -- 172.21.15.36:0/605259383 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a880068b0 con 0x7f6aa00b27b0 2022-06-17T12:57:29.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.926+0000 7f6a977fe700 1 -- 172.21.15.36:0/605259383 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8800d8f0 con 0x7f6aa00b27b0 2022-06-17T12:57:29.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.926+0000 7f6a9f59e700 1 -- 172.21.15.36:0/605259383 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa0152f90 con 0x7f6aa00b27b0 2022-06-17T12:57:29.930 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.926+0000 7f6a977fe700 1 -- 172.21.15.36:0/605259383 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a88011e20 con 0x7f6aa00b27b0 2022-06-17T12:57:29.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.926+0000 7f6a9f59e700 1 -- 172.21.15.36:0/605259383 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa009e9b0 con 0x7f6aa00b27b0 2022-06-17T12:57:29.931 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.926+0000 7f6a9f59e700 1 -- 172.21.15.36:0/605259383 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa0066880 con 0x7f6aa00b27b0 2022-06-17T12:57:29.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.926+0000 7f6a977fe700 1 -- 172.21.15.36:0/605259383 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f6a88025030 con 0x7f6aa00b27b0 2022-06-17T12:57:29.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.926+0000 7f6a977fe700 1 --2- 172.21.15.36:0/605259383 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6a84053a60 0x7f6a84055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:29.932 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.926+0000 7f6a977fe700 1 -- 172.21.15.36:0/605259383 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6a8806d100 con 0x7f6aa00b27b0 2022-06-17T12:57:29.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.926+0000 7f6a9e59c700 1 --2- 172.21.15.36:0/605259383 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6a84053a60 0x7f6a84055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:29.933 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.930+0000 7f6a9e59c700 1 --2- 172.21.15.36:0/605259383 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6a84053a60 0x7f6a84055f20 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f6a9000b000 tx=0x7f6a90008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:29.934 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:29.934+0000 7f6a977fe700 1 -- 172.21.15.36:0/605259383 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6a88043f00 con 0x7f6aa00b27b0 2022-06-17T12:57:30.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.066+0000 7f6a9f59e700 1 -- 172.21.15.36:0/605259383 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f6aa0057410 con 0x7f6aa00b27b0 2022-06-17T12:57:30.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.070+0000 7f6a977fe700 1 -- 172.21.15.36:0/605259383 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v151) v1 ==== 81+0+34051 (secure 0 0 0) 0x7f6a88047520 con 0x7f6aa00b27b0 2022-06-17T12:57:30.054 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:57:30.057 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T12:57:30.057 INFO:teuthology.orchestra.run.smithi036.stdout: "epoch": 151, 2022-06-17T12:57:30.058 INFO:teuthology.orchestra.run.smithi036.stdout: "fsid": "b68b9700-0c85-42ca-94d7-6cd53db91c14", 2022-06-17T12:57:30.058 INFO:teuthology.orchestra.run.smithi036.stdout: "created": "2022-06-17T12:46:30.671408+0000", 2022-06-17T12:57:30.058 INFO:teuthology.orchestra.run.smithi036.stdout: "modified": "2022-06-17T12:57:16.881090+0000", 2022-06-17T12:57:30.058 INFO:teuthology.orchestra.run.smithi036.stdout: "last_up_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:57:30.059 INFO:teuthology.orchestra.run.smithi036.stdout: "last_in_change": "2022-06-17T12:46:40.587632+0000", 2022-06-17T12:57:30.059 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-17T12:57:30.059 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_num": 5799936, 2022-06-17T12:57:30.059 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_set": [ 2022-06-17T12:57:30.059 INFO:teuthology.orchestra.run.smithi036.stdout: "pglog_hardlimit", 2022-06-17T12:57:30.060 INFO:teuthology.orchestra.run.smithi036.stdout: "purged_snapdirs", 2022-06-17T12:57:30.061 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_deletes", 2022-06-17T12:57:30.061 INFO:teuthology.orchestra.run.smithi036.stdout: "sortbitwise" 2022-06-17T12:57:30.061 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:57:30.061 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_version": 6, 2022-06-17T12:57:30.067 INFO:teuthology.orchestra.run.smithi036.stdout: "full_ratio": 0.94999998807907104, 2022-06-17T12:57:30.068 INFO:teuthology.orchestra.run.smithi036.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-17T12:57:30.069 INFO:teuthology.orchestra.run.smithi036.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-17T12:57:30.069 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_snapshot": "", 2022-06-17T12:57:30.071 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_max": 30, 2022-06-17T12:57:30.072 INFO:teuthology.orchestra.run.smithi036.stdout: "max_osd": 8, 2022-06-17T12:57:30.072 INFO:teuthology.orchestra.run.smithi036.stdout: "require_min_compat_client": "jewel", 2022-06-17T12:57:30.072 INFO:teuthology.orchestra.run.smithi036.stdout: "min_compat_client": "jewel", 2022-06-17T12:57:30.073 INFO:teuthology.orchestra.run.smithi036.stdout: "require_osd_release": "octopus", 2022-06-17T12:57:30.073 INFO:teuthology.orchestra.run.smithi036.stdout: "pools": [ 2022-06-17T12:57:30.073 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.074 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 1, 2022-06-17T12:57:30.074 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "device_health_metrics", 2022-06-17T12:57:30.074 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:33.881212+0000", 2022-06-17T12:57:30.075 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:57:30.075 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:57:30.075 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:57:30.076 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:57:30.076 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:57:30.076 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:57:30.077 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:57:30.077 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:57:30.084 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 1, 2022-06-17T12:57:30.084 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 1, 2022-06-17T12:57:30.085 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 1, 2022-06-17T12:57:30.085 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 1, 2022-06-17T12:57:30.085 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 1, 2022-06-17T12:57:30.085 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:57:30.086 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:57:30.086 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:57:30.086 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:57:30.086 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:57:30.087 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:57:30.087 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:57:30.087 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.087 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "16", 2022-06-17T12:57:30.087 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:57:30.088 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:57:30.088 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:57:30.088 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:57:30.088 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:57:30.089 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:57:30.089 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:57:30.089 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:57:30.089 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:57:30.089 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:57:30.091 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:57:30.091 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:57:30.093 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:57:30.093 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:57:30.094 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:57:30.094 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:57:30.094 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:57:30.094 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:57:30.095 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:57:30.095 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:57:30.095 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:57:30.095 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:57:30.095 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:57:30.096 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:57:30.096 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:57:30.096 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:57:30.097 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.097 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:57:30.097 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:57:30.097 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:57:30.098 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:57:30.098 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:57:30.099 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:57:30.099 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:57:30.099 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:57:30.099 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:57:30.100 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:57:30.100 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:57:30.100 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:57:30.100 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 1 2022-06-17T12:57:30.100 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.101 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:57:30.101 INFO:teuthology.orchestra.run.smithi036.stdout: "mgr_devicehealth": {} 2022-06-17T12:57:30.101 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.101 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.102 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.102 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 2, 2022-06-17T12:57:30.104 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "rbd", 2022-06-17T12:57:30.105 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:46:44.485460+0000", 2022-06-17T12:57:30.105 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:57:30.105 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:57:30.106 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:57:30.106 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:57:30.107 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:57:30.107 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:57:30.108 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:57:30.108 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:57:30.108 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:57:30.108 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:57:30.109 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:57:30.109 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:57:30.110 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:57:30.111 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:57:30.111 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:57:30.112 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:57:30.112 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:57:30.112 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:57:30.112 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:57:30.113 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:57:30.113 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.113 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "19", 2022-06-17T12:57:30.113 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:57:30.115 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:57:30.116 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:57:30.116 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:57:30.116 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:57:30.117 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:57:30.117 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:57:30.117 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:57:30.119 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:57:30.119 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:57:30.119 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:57:30.120 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:57:30.121 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:57:30.121 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:57:30.121 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:57:30.128 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:57:30.128 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:57:30.129 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:57:30.129 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:57:30.129 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:57:30.129 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:57:30.129 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:57:30.130 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:57:30.130 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:57:30.130 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:57:30.130 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:57:30.132 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.132 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:57:30.132 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:57:30.132 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:57:30.132 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:57:30.133 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:57:30.133 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:57:30.133 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:57:30.133 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:57:30.134 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:57:30.134 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:57:30.134 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:57:30.134 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:57:30.134 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:57:30.135 INFO:teuthology.orchestra.run.smithi036.stdout: "rbd": {} 2022-06-17T12:57:30.135 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.135 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.135 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.135 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 29, 2022-06-17T12:57:30.136 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_metadata", 2022-06-17T12:57:30.136 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:57:13.415000+0000", 2022-06-17T12:57:30.136 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:57:30.136 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:57:30.137 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:57:30.137 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:57:30.137 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:57:30.137 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:57:30.137 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:57:30.138 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:57:30.138 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:57:30.139 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:57:30.139 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:57:30.139 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:57:30.139 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:57:30.139 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:57:30.140 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:57:30.140 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:57:30.140 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:57:30.140 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:57:30.141 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:57:30.141 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:57:30.141 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.141 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "149", 2022-06-17T12:57:30.143 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:57:30.144 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:57:30.144 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:57:30.144 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:57:30.145 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:57:30.145 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:57:30.145 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:57:30.145 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:57:30.145 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:57:30.146 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:57:30.146 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:57:30.146 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:57:30.147 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:57:30.147 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:57:30.147 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:57:30.147 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:57:30.148 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:57:30.148 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:57:30.148 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:57:30.149 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:57:30.150 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:57:30.150 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:57:30.151 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:57:30.151 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:57:30.152 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:57:30.152 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:57:30.153 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.154 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:57:30.154 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:57:30.154 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:57:30.155 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:57:30.155 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:57:30.156 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:57:30.156 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:57:30.157 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:57:30.157 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:57:30.157 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:57:30.158 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:57:30.159 INFO:teuthology.orchestra.run.smithi036.stdout: "options": { 2022-06-17T12:57:30.160 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_bias": 4, 2022-06-17T12:57:30.161 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_min": 16, 2022-06-17T12:57:30.162 INFO:teuthology.orchestra.run.smithi036.stdout: "recovery_priority": 5 2022-06-17T12:57:30.162 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.163 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:57:30.163 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:57:30.164 INFO:teuthology.orchestra.run.smithi036.stdout: "metadata": "cephfs" 2022-06-17T12:57:30.164 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.165 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.165 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.166 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.166 INFO:teuthology.orchestra.run.smithi036.stdout: "pool": 30, 2022-06-17T12:57:30.167 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_name": "cephfs_data", 2022-06-17T12:57:30.167 INFO:teuthology.orchestra.run.smithi036.stdout: "create_time": "2022-06-17T12:57:14.234187+0000", 2022-06-17T12:57:30.168 INFO:teuthology.orchestra.run.smithi036.stdout: "flags": 1, 2022-06-17T12:57:30.169 INFO:teuthology.orchestra.run.smithi036.stdout: "flags_names": "hashpspool", 2022-06-17T12:57:30.171 INFO:teuthology.orchestra.run.smithi036.stdout: "type": 1, 2022-06-17T12:57:30.172 INFO:teuthology.orchestra.run.smithi036.stdout: "size": 2, 2022-06-17T12:57:30.172 INFO:teuthology.orchestra.run.smithi036.stdout: "min_size": 1, 2022-06-17T12:57:30.173 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_rule": 0, 2022-06-17T12:57:30.173 INFO:teuthology.orchestra.run.smithi036.stdout: "object_hash": 2, 2022-06-17T12:57:30.173 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_autoscale_mode": "off", 2022-06-17T12:57:30.174 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num": 8, 2022-06-17T12:57:30.174 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num": 8, 2022-06-17T12:57:30.175 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_placement_num_target": 8, 2022-06-17T12:57:30.175 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_target": 8, 2022-06-17T12:57:30.175 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_num_pending": 8, 2022-06-17T12:57:30.176 INFO:teuthology.orchestra.run.smithi036.stdout: "last_pg_merge_meta": { 2022-06-17T12:57:30.176 INFO:teuthology.orchestra.run.smithi036.stdout: "source_pgid": "0.0", 2022-06-17T12:57:30.177 INFO:teuthology.orchestra.run.smithi036.stdout: "ready_epoch": 0, 2022-06-17T12:57:30.177 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_started": 0, 2022-06-17T12:57:30.177 INFO:teuthology.orchestra.run.smithi036.stdout: "last_epoch_clean": 0, 2022-06-17T12:57:30.178 INFO:teuthology.orchestra.run.smithi036.stdout: "source_version": "0'0", 2022-06-17T12:57:30.178 INFO:teuthology.orchestra.run.smithi036.stdout: "target_version": "0'0" 2022-06-17T12:57:30.180 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.181 INFO:teuthology.orchestra.run.smithi036.stdout: "last_change": "151", 2022-06-17T12:57:30.182 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend": "0", 2022-06-17T12:57:30.183 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-17T12:57:30.183 INFO:teuthology.orchestra.run.smithi036.stdout: "last_force_op_resend_preluminous": "0", 2022-06-17T12:57:30.184 INFO:teuthology.orchestra.run.smithi036.stdout: "auid": 0, 2022-06-17T12:57:30.184 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_mode": "selfmanaged", 2022-06-17T12:57:30.184 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_seq": 0, 2022-06-17T12:57:30.185 INFO:teuthology.orchestra.run.smithi036.stdout: "snap_epoch": 0, 2022-06-17T12:57:30.185 INFO:teuthology.orchestra.run.smithi036.stdout: "pool_snaps": [], 2022-06-17T12:57:30.185 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps": "[]", 2022-06-17T12:57:30.186 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_bytes": 0, 2022-06-17T12:57:30.186 INFO:teuthology.orchestra.run.smithi036.stdout: "quota_max_objects": 0, 2022-06-17T12:57:30.187 INFO:teuthology.orchestra.run.smithi036.stdout: "tiers": [], 2022-06-17T12:57:30.187 INFO:teuthology.orchestra.run.smithi036.stdout: "tier_of": -1, 2022-06-17T12:57:30.188 INFO:teuthology.orchestra.run.smithi036.stdout: "read_tier": -1, 2022-06-17T12:57:30.188 INFO:teuthology.orchestra.run.smithi036.stdout: "write_tier": -1, 2022-06-17T12:57:30.189 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_mode": "none", 2022-06-17T12:57:30.189 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_bytes": 0, 2022-06-17T12:57:30.190 INFO:teuthology.orchestra.run.smithi036.stdout: "target_max_objects": 0, 2022-06-17T12:57:30.193 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-17T12:57:30.193 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-17T12:57:30.194 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-17T12:57:30.195 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_flush_age": 0, 2022-06-17T12:57:30.195 INFO:teuthology.orchestra.run.smithi036.stdout: "cache_min_evict_age": 0, 2022-06-17T12:57:30.196 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profile": "", 2022-06-17T12:57:30.196 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_params": { 2022-06-17T12:57:30.196 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "none" 2022-06-17T12:57:30.197 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.197 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_period": 0, 2022-06-17T12:57:30.198 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_count": 0, 2022-06-17T12:57:30.198 INFO:teuthology.orchestra.run.smithi036.stdout: "use_gmt_hitset": true, 2022-06-17T12:57:30.199 INFO:teuthology.orchestra.run.smithi036.stdout: "min_read_recency_for_promote": 0, 2022-06-17T12:57:30.199 INFO:teuthology.orchestra.run.smithi036.stdout: "min_write_recency_for_promote": 0, 2022-06-17T12:57:30.200 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_grade_decay_rate": 0, 2022-06-17T12:57:30.201 INFO:teuthology.orchestra.run.smithi036.stdout: "hit_set_search_last_n": 0, 2022-06-17T12:57:30.204 INFO:teuthology.orchestra.run.smithi036.stdout: "grade_table": [], 2022-06-17T12:57:30.205 INFO:teuthology.orchestra.run.smithi036.stdout: "stripe_width": 0, 2022-06-17T12:57:30.206 INFO:teuthology.orchestra.run.smithi036.stdout: "expected_num_objects": 0, 2022-06-17T12:57:30.206 INFO:teuthology.orchestra.run.smithi036.stdout: "fast_read": false, 2022-06-17T12:57:30.207 INFO:teuthology.orchestra.run.smithi036.stdout: "options": {}, 2022-06-17T12:57:30.207 INFO:teuthology.orchestra.run.smithi036.stdout: "application_metadata": { 2022-06-17T12:57:30.208 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs": { 2022-06-17T12:57:30.208 INFO:teuthology.orchestra.run.smithi036.stdout: "data": "cephfs" 2022-06-17T12:57:30.209 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.209 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.210 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.214 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:57:30.215 INFO:teuthology.orchestra.run.smithi036.stdout: "osds": [ 2022-06-17T12:57:30.216 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.216 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:57:30.217 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "eb8ddf14-4f6d-4045-9340-acf01d2935ad", 2022-06-17T12:57:30.217 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:57:30.217 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:57:30.218 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:57:30.218 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:57:30.219 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:57:30.219 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:57:30.220 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:57:30.220 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 148, 2022-06-17T12:57:30.221 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:57:30.221 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:57:30.222 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:57:30.222 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.223 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.225 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.225 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6802", 2022-06-17T12:57:30.226 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:57:30.226 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.227 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.227 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.227 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6803", 2022-06-17T12:57:30.228 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:57:30.228 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.229 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.229 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.229 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:57:30.230 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.230 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.231 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.231 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6804", 2022-06-17T12:57:30.232 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:57:30.232 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.232 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.233 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.234 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6805", 2022-06-17T12:57:30.235 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:57:30.235 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.236 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.236 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.237 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:57:30.237 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.237 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.238 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.238 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6808", 2022-06-17T12:57:30.239 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:57:30.239 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.239 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.240 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.240 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6809", 2022-06-17T12:57:30.241 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:57:30.241 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.241 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.242 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.242 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:57:30.242 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.243 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.243 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.244 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6806", 2022-06-17T12:57:30.246 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:57:30.246 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.247 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.250 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.250 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6807", 2022-06-17T12:57:30.251 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22619 2022-06-17T12:57:30.251 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.251 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.252 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.252 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6803/22619", 2022-06-17T12:57:30.253 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6805/22619", 2022-06-17T12:57:30.253 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6809/22619", 2022-06-17T12:57:30.253 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6807/22619", 2022-06-17T12:57:30.255 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:57:30.255 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:57:30.256 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:57:30.257 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.257 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.257 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.259 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:57:30.260 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "07042677-5771-4de0-bfef-a3e2a4540735", 2022-06-17T12:57:30.260 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:57:30.261 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:57:30.261 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:57:30.262 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:57:30.262 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:57:30.263 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:57:30.264 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:57:30.264 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 147, 2022-06-17T12:57:30.264 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:57:30.265 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:57:30.265 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:57:30.265 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.266 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.267 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.267 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6810", 2022-06-17T12:57:30.267 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:57:30.267 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.268 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.268 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.268 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6811", 2022-06-17T12:57:30.269 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:57:30.270 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.270 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.271 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.271 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:57:30.272 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.272 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.273 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.273 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6812", 2022-06-17T12:57:30.274 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:57:30.275 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.275 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.276 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.276 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6813", 2022-06-17T12:57:30.277 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:57:30.277 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.278 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.278 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.279 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:57:30.279 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.279 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.280 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.280 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6816", 2022-06-17T12:57:30.281 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:57:30.281 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.281 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.282 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.282 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6817", 2022-06-17T12:57:30.283 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:57:30.283 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.284 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.284 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.285 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:57:30.285 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.286 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.287 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.287 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6814", 2022-06-17T12:57:30.287 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:57:30.288 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.288 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.288 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.288 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6815", 2022-06-17T12:57:30.289 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22620 2022-06-17T12:57:30.289 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.289 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.290 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.290 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6811/22620", 2022-06-17T12:57:30.290 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6813/22620", 2022-06-17T12:57:30.291 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6817/22620", 2022-06-17T12:57:30.291 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6815/22620", 2022-06-17T12:57:30.291 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:57:30.291 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:57:30.291 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:57:30.292 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.292 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.292 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.292 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:57:30.293 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "97d0097b-fa05-483c-808c-44feaa212a1c", 2022-06-17T12:57:30.293 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:57:30.293 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:57:30.293 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:57:30.294 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:57:30.294 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:57:30.294 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:57:30.294 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:57:30.295 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 148, 2022-06-17T12:57:30.295 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:57:30.295 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:57:30.295 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:57:30.296 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.296 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.296 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.297 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6826", 2022-06-17T12:57:30.297 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:57:30.297 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.298 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.298 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.298 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6827", 2022-06-17T12:57:30.298 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:57:30.299 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.299 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.299 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.300 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:57:30.300 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.300 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.300 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.301 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6828", 2022-06-17T12:57:30.301 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:57:30.301 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.301 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.301 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.302 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6829", 2022-06-17T12:57:30.302 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:57:30.302 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.303 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.303 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.303 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:57:30.304 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.304 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.304 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.304 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6832", 2022-06-17T12:57:30.305 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:57:30.305 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.305 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.305 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.306 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6833", 2022-06-17T12:57:30.306 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:57:30.306 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.306 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.307 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.307 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:57:30.307 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.307 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.308 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.308 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6830", 2022-06-17T12:57:30.308 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:57:30.308 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.308 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.309 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.309 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6831", 2022-06-17T12:57:30.309 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22643 2022-06-17T12:57:30.309 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.310 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.310 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.310 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6827/22643", 2022-06-17T12:57:30.310 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6829/22643", 2022-06-17T12:57:30.311 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6833/22643", 2022-06-17T12:57:30.311 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6831/22643", 2022-06-17T12:57:30.311 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:57:30.311 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:57:30.312 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:57:30.312 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.312 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.312 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.312 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:57:30.313 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "fb471fe2-5427-441e-b6d7-47c004564c69", 2022-06-17T12:57:30.313 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:57:30.313 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:57:30.313 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:57:30.314 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:57:30.314 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:57:30.314 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:57:30.314 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:57:30.315 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 148, 2022-06-17T12:57:30.315 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:57:30.315 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:57:30.315 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:57:30.315 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.316 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.316 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.316 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6818", 2022-06-17T12:57:30.317 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:57:30.317 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.317 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.317 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.318 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6819", 2022-06-17T12:57:30.318 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:57:30.318 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.318 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.318 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.319 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:57:30.319 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.319 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.319 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.320 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6820", 2022-06-17T12:57:30.320 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:57:30.320 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.320 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.320 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.321 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6821", 2022-06-17T12:57:30.321 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:57:30.321 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.321 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.322 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.322 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:57:30.322 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.322 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.323 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.323 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6824", 2022-06-17T12:57:30.323 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:57:30.323 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.323 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.324 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.324 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6825", 2022-06-17T12:57:30.324 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:57:30.324 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.325 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.325 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.325 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:57:30.325 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.325 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.326 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.326 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6822", 2022-06-17T12:57:30.327 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:57:30.327 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.327 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.327 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.328 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.36:6823", 2022-06-17T12:57:30.328 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22646 2022-06-17T12:57:30.328 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.328 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.329 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.329 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.36:6819/22646", 2022-06-17T12:57:30.329 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.36:6821/22646", 2022-06-17T12:57:30.329 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.36:6825/22646", 2022-06-17T12:57:30.330 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.36:6823/22646", 2022-06-17T12:57:30.330 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:57:30.330 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:57:30.331 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:57:30.331 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.332 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.332 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.332 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:57:30.332 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "4894aebf-e4b3-4436-a3ff-00a1241dbfcc", 2022-06-17T12:57:30.333 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:57:30.333 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:57:30.333 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:57:30.333 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:57:30.334 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:57:30.334 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:57:30.334 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:57:30.334 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 147, 2022-06-17T12:57:30.335 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:57:30.335 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:57:30.335 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:57:30.335 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.336 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.336 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.336 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6808", 2022-06-17T12:57:30.336 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:57:30.336 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.337 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.337 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.337 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6809", 2022-06-17T12:57:30.337 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:57:30.338 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.338 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.338 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.338 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:57:30.338 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.339 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.339 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.339 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6810", 2022-06-17T12:57:30.339 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:57:30.340 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.340 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.340 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.340 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6811", 2022-06-17T12:57:30.341 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:57:30.341 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.341 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.341 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.342 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:57:30.342 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.342 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.342 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.343 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6814", 2022-06-17T12:57:30.343 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:57:30.343 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.343 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.344 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.344 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6815", 2022-06-17T12:57:30.344 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:57:30.345 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.345 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.345 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.345 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:57:30.346 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.346 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.346 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.346 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6812", 2022-06-17T12:57:30.347 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:57:30.347 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.347 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.347 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.348 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6813", 2022-06-17T12:57:30.348 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22812 2022-06-17T12:57:30.348 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.348 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.349 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.349 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6809/22812", 2022-06-17T12:57:30.350 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6811/22812", 2022-06-17T12:57:30.350 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6815/22812", 2022-06-17T12:57:30.350 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6813/22812", 2022-06-17T12:57:30.350 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:57:30.351 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:57:30.351 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:57:30.351 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.351 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.351 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.352 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:57:30.352 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "74e2041b-4756-47c2-bd3c-defaa046e936", 2022-06-17T12:57:30.352 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:57:30.352 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:57:30.353 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:57:30.353 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:57:30.353 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:57:30.354 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:57:30.354 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 13, 2022-06-17T12:57:30.355 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 149, 2022-06-17T12:57:30.355 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:57:30.355 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:57:30.356 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:57:30.356 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.357 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.358 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.358 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6800", 2022-06-17T12:57:30.358 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:57:30.359 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.359 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.360 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.360 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6801", 2022-06-17T12:57:30.361 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:57:30.361 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.362 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.362 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.362 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:57:30.363 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.363 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.364 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.364 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6802", 2022-06-17T12:57:30.365 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:57:30.365 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.365 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.366 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.366 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6803", 2022-06-17T12:57:30.367 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:57:30.367 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.368 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.368 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.369 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:57:30.369 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.370 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.370 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.371 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6806", 2022-06-17T12:57:30.371 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:57:30.372 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.372 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.373 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.373 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6807", 2022-06-17T12:57:30.374 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:57:30.374 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.375 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.375 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.376 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:57:30.376 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.376 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.377 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.377 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6804", 2022-06-17T12:57:30.378 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:57:30.378 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.379 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.379 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.380 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6805", 2022-06-17T12:57:30.380 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22813 2022-06-17T12:57:30.380 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.381 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.381 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.382 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6801/22813", 2022-06-17T12:57:30.382 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6803/22813", 2022-06-17T12:57:30.383 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6807/22813", 2022-06-17T12:57:30.383 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6805/22813", 2022-06-17T12:57:30.384 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:57:30.384 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:57:30.384 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:57:30.385 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.385 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.386 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.386 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:57:30.387 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "f55cd87d-673d-4717-bde0-103a9204f75b", 2022-06-17T12:57:30.387 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:57:30.387 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:57:30.388 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:57:30.388 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:57:30.394 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:57:30.394 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:57:30.394 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:57:30.395 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 148, 2022-06-17T12:57:30.395 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:57:30.396 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:57:30.396 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:57:30.397 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.397 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.398 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.398 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6816", 2022-06-17T12:57:30.398 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:57:30.399 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.400 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.400 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.400 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6817", 2022-06-17T12:57:30.401 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:57:30.401 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.401 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.402 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.402 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:57:30.403 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.403 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.404 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.404 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6818", 2022-06-17T12:57:30.405 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:57:30.405 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.409 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.409 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.409 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6819", 2022-06-17T12:57:30.410 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:57:30.410 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.411 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.411 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.412 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:57:30.412 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.413 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.413 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.413 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6822", 2022-06-17T12:57:30.414 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:57:30.414 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.415 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.415 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.415 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6823", 2022-06-17T12:57:30.416 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:57:30.419 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.419 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.420 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.420 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:57:30.420 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.421 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.421 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.422 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6820", 2022-06-17T12:57:30.422 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:57:30.422 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.423 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.423 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.424 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6821", 2022-06-17T12:57:30.424 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22822 2022-06-17T12:57:30.425 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.425 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.425 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.426 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6817/22822", 2022-06-17T12:57:30.426 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6819/22822", 2022-06-17T12:57:30.427 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6823/22822", 2022-06-17T12:57:30.427 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6821/22822", 2022-06-17T12:57:30.428 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:57:30.428 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:57:30.428 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:57:30.429 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.429 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.430 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.431 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:57:30.431 INFO:teuthology.orchestra.run.smithi036.stdout: "uuid": "cbf2a5fd-464a-4b16-872b-d86fbe56f84c", 2022-06-17T12:57:30.432 INFO:teuthology.orchestra.run.smithi036.stdout: "up": 1, 2022-06-17T12:57:30.432 INFO:teuthology.orchestra.run.smithi036.stdout: "in": 1, 2022-06-17T12:57:30.432 INFO:teuthology.orchestra.run.smithi036.stdout: "weight": 1, 2022-06-17T12:57:30.433 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_affinity": 1, 2022-06-17T12:57:30.433 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_begin": 0, 2022-06-17T12:57:30.434 INFO:teuthology.orchestra.run.smithi036.stdout: "last_clean_end": 0, 2022-06-17T12:57:30.434 INFO:teuthology.orchestra.run.smithi036.stdout: "up_from": 14, 2022-06-17T12:57:30.434 INFO:teuthology.orchestra.run.smithi036.stdout: "up_thru": 148, 2022-06-17T12:57:30.435 INFO:teuthology.orchestra.run.smithi036.stdout: "down_at": 0, 2022-06-17T12:57:30.435 INFO:teuthology.orchestra.run.smithi036.stdout: "lost_at": 0, 2022-06-17T12:57:30.436 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addrs": { 2022-06-17T12:57:30.436 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.437 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.437 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.438 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6824", 2022-06-17T12:57:30.438 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:57:30.438 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.439 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.439 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.440 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6825", 2022-06-17T12:57:30.440 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:57:30.441 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.441 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.442 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.442 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addrs": { 2022-06-17T12:57:30.443 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.444 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.444 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.445 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6826", 2022-06-17T12:57:30.445 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:57:30.445 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.451 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.451 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.451 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6827", 2022-06-17T12:57:30.452 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:57:30.452 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.453 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.453 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.457 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addrs": { 2022-06-17T12:57:30.458 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.458 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.459 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.459 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6830", 2022-06-17T12:57:30.460 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:57:30.460 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.461 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.461 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.461 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6831", 2022-06-17T12:57:30.462 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:57:30.462 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.463 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.463 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.464 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addrs": { 2022-06-17T12:57:30.464 INFO:teuthology.orchestra.run.smithi036.stdout: "addrvec": [ 2022-06-17T12:57:30.465 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.465 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v2", 2022-06-17T12:57:30.466 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6828", 2022-06-17T12:57:30.466 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:57:30.467 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.468 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.468 INFO:teuthology.orchestra.run.smithi036.stdout: "type": "v1", 2022-06-17T12:57:30.469 INFO:teuthology.orchestra.run.smithi036.stdout: "addr": "172.21.15.70:6829", 2022-06-17T12:57:30.469 INFO:teuthology.orchestra.run.smithi036.stdout: "nonce": 22837 2022-06-17T12:57:30.470 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.470 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.471 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.471 INFO:teuthology.orchestra.run.smithi036.stdout: "public_addr": "172.21.15.70:6825/22837", 2022-06-17T12:57:30.472 INFO:teuthology.orchestra.run.smithi036.stdout: "cluster_addr": "172.21.15.70:6827/22837", 2022-06-17T12:57:30.472 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_back_addr": "172.21.15.70:6831/22837", 2022-06-17T12:57:30.472 INFO:teuthology.orchestra.run.smithi036.stdout: "heartbeat_front_addr": "172.21.15.70:6829/22837", 2022-06-17T12:57:30.473 INFO:teuthology.orchestra.run.smithi036.stdout: "state": [ 2022-06-17T12:57:30.473 INFO:teuthology.orchestra.run.smithi036.stdout: "exists", 2022-06-17T12:57:30.474 INFO:teuthology.orchestra.run.smithi036.stdout: "up" 2022-06-17T12:57:30.474 INFO:teuthology.orchestra.run.smithi036.stdout: ] 2022-06-17T12:57:30.475 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.475 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:57:30.476 INFO:teuthology.orchestra.run.smithi036.stdout: "osd_xinfo": [ 2022-06-17T12:57:30.476 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.476 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 0, 2022-06-17T12:57:30.477 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:57:30.478 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:57:30.479 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:57:30.479 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:57:30.480 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:57:30.480 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.291461+0000", 2022-06-17T12:57:30.481 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:57:30.481 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.482 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.482 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 1, 2022-06-17T12:57:30.483 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:57:30.483 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:57:30.484 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:57:30.484 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:57:30.485 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:57:30.485 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.537912+0000", 2022-06-17T12:57:30.485 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:57:30.486 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.487 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.487 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 2, 2022-06-17T12:57:30.488 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:57:30.488 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:57:30.490 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:57:30.490 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:57:30.491 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:57:30.491 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.606947+0000", 2022-06-17T12:57:30.491 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:57:30.492 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.492 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.492 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 3, 2022-06-17T12:57:30.492 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:57:30.504 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:57:30.504 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:57:30.505 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:57:30.505 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:57:30.506 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.514412+0000", 2022-06-17T12:57:30.506 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:57:30.507 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.507 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.508 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 4, 2022-06-17T12:57:30.508 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:57:30.509 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:57:30.509 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:57:30.510 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:57:30.510 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:57:30.511 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.394645+0000", 2022-06-17T12:57:30.512 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:57:30.512 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.530 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.530 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 5, 2022-06-17T12:57:30.530 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:57:30.531 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:57:30.531 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:57:30.531 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:57:30.531 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:57:30.532 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.343806+0000", 2022-06-17T12:57:30.532 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:57:30.532 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.532 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.532 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 6, 2022-06-17T12:57:30.533 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:57:30.533 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:57:30.533 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:57:30.534 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:57:30.534 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:57:30.536 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.591864+0000", 2022-06-17T12:57:30.537 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:57:30.537 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.537 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-06-17T12:57:30.538 INFO:teuthology.orchestra.run.smithi036.stdout: "osd": 7, 2022-06-17T12:57:30.538 INFO:teuthology.orchestra.run.smithi036.stdout: "down_stamp": "0.000000", 2022-06-17T12:57:30.538 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_probability": 0, 2022-06-17T12:57:30.539 INFO:teuthology.orchestra.run.smithi036.stdout: "laggy_interval": 0, 2022-06-17T12:57:30.539 INFO:teuthology.orchestra.run.smithi036.stdout: "features": 4540138292840890367, 2022-06-17T12:57:30.539 INFO:teuthology.orchestra.run.smithi036.stdout: "old_weight": 0, 2022-06-17T12:57:30.539 INFO:teuthology.orchestra.run.smithi036.stdout: "last_purged_snaps_scrub": "2022-06-17T12:46:38.607045+0000", 2022-06-17T12:57:30.540 INFO:teuthology.orchestra.run.smithi036.stdout: "dead_epoch": 0 2022-06-17T12:57:30.540 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.540 INFO:teuthology.orchestra.run.smithi036.stdout: ], 2022-06-17T12:57:30.540 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap": [], 2022-06-17T12:57:30.541 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_upmap_items": [], 2022-06-17T12:57:30.541 INFO:teuthology.orchestra.run.smithi036.stdout: "pg_temp": [], 2022-06-17T12:57:30.541 INFO:teuthology.orchestra.run.smithi036.stdout: "primary_temp": [], 2022-06-17T12:57:30.541 INFO:teuthology.orchestra.run.smithi036.stdout: "blacklist": {}, 2022-06-17T12:57:30.541 INFO:teuthology.orchestra.run.smithi036.stdout: "erasure_code_profiles": { 2022-06-17T12:57:30.542 INFO:teuthology.orchestra.run.smithi036.stdout: "cephfs_data": { 2022-06-17T12:57:30.542 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-device-class": "", 2022-06-17T12:57:30.542 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:57:30.542 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-root": "default", 2022-06-17T12:57:30.542 INFO:teuthology.orchestra.run.smithi036.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-17T12:57:30.543 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:57:30.543 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "2", 2022-06-17T12:57:30.543 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:57:30.543 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:57:30.544 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van", 2022-06-17T12:57:30.544 INFO:teuthology.orchestra.run.smithi036.stdout: "w": "8" 2022-06-17T12:57:30.544 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.545 INFO:teuthology.orchestra.run.smithi036.stdout: "default": { 2022-06-17T12:57:30.545 INFO:teuthology.orchestra.run.smithi036.stdout: "crush-failure-domain": "osd", 2022-06-17T12:57:30.545 INFO:teuthology.orchestra.run.smithi036.stdout: "k": "2", 2022-06-17T12:57:30.548 INFO:teuthology.orchestra.run.smithi036.stdout: "m": "1", 2022-06-17T12:57:30.548 INFO:teuthology.orchestra.run.smithi036.stdout: "plugin": "jerasure", 2022-06-17T12:57:30.549 INFO:teuthology.orchestra.run.smithi036.stdout: "ruleset-failure-domain": "osd", 2022-06-17T12:57:30.549 INFO:teuthology.orchestra.run.smithi036.stdout: "technique": "reed_sol_van" 2022-06-17T12:57:30.549 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-06-17T12:57:30.550 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-06-17T12:57:30.550 INFO:teuthology.orchestra.run.smithi036.stdout: "removed_snaps_queue": [], 2022-06-17T12:57:30.550 INFO:teuthology.orchestra.run.smithi036.stdout: "new_removed_snaps": [], 2022-06-17T12:57:30.550 INFO:teuthology.orchestra.run.smithi036.stdout: "new_purged_snaps": [], 2022-06-17T12:57:30.551 INFO:teuthology.orchestra.run.smithi036.stdout: "crush_node_flags": {}, 2022-06-17T12:57:30.551 INFO:teuthology.orchestra.run.smithi036.stdout: "device_class_flags": {} 2022-06-17T12:57:30.551 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T12:57:30.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.070+0000 7f6a9f59e700 1 -- 172.21.15.36:0/605259383 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6a84053a60 msgr2=0x7f6a84055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:30.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.070+0000 7f6a9f59e700 1 --2- 172.21.15.36:0/605259383 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6a84053a60 0x7f6a84055f20 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f6a9000b000 tx=0x7f6a90008040).stop 2022-06-17T12:57:30.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.070+0000 7f6a9f59e700 1 -- 172.21.15.36:0/605259383 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6aa00b27b0 msgr2=0x7f6aa00a1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:30.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.070+0000 7f6a9f59e700 1 --2- 172.21.15.36:0/605259383 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6aa00b27b0 0x7f6aa00a1f70 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 rx=0x7f6a88002f70 tx=0x7f6a88015000).stop 2022-06-17T12:57:30.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.074+0000 7f6a9f59e700 1 -- 172.21.15.36:0/605259383 shutdown_connections 2022-06-17T12:57:30.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.074+0000 7f6a9f59e700 1 --2- 172.21.15.36:0/605259383 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f6a84053a60 0x7f6a84055f20 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:30.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.074+0000 7f6a9f59e700 1 --2- 172.21.15.36:0/605259383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6aa013f050 0x7f6aa013f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:30.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.074+0000 7f6a9f59e700 1 --2- 172.21.15.36:0/605259383 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f6aa00b27b0 0x7f6aa00a1f70 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:30.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.074+0000 7f6a9f59e700 1 --2- 172.21.15.36:0/605259383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6aa0060ca0 0x7f6aa00a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:30.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.074+0000 7f6a9f59e700 1 -- 172.21.15.36:0/605259383 >> 172.21.15.36:0/605259383 conn(0x7f6aa00b5d10 msgr2=0x7f6aa013fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:30.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.074+0000 7f6a9f59e700 1 -- 172.21.15.36:0/605259383 shutdown_connections 2022-06-17T12:57:30.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.074+0000 7f6a9f59e700 1 -- 172.21.15.36:0/605259383 wait complete. 2022-06-17T12:57:30.563 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:57:30.693 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f3350707700 1 Processor -- start 2022-06-17T12:57:30.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f3350707700 1 -- start start 2022-06-17T12:57:30.694 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f3350707700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f334805f770 0x7f334814b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:30.695 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f3350707700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3348147f50 0x7f3348141210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:30.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f3350707700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334814b930 con 0x7f3348141700 2022-06-17T12:57:30.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f3350707700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33481475b0 con 0x7f334805f770 2022-06-17T12:57:30.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f3350707700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3348147810 con 0x7f3348147f50 2022-06-17T12:57:30.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f334e4a3700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f334805f770 0x7f334814b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:30.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f334e4a3700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f334805f770 0x7f334814b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40974/0 (socket says 172.21.15.36:40974) 2022-06-17T12:57:30.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f334e4a3700 1 -- 172.21.15.36:0/1956755043 learned_addr learned my addr 172.21.15.36:0/1956755043 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:30.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f334dca2700 1 --2- 172.21.15.36:0/1956755043 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3348147f50 0x7f3348141210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:30.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f334e4a3700 1 -- 172.21.15.36:0/1956755043 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3348147f50 msgr2=0x7f3348141210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:30.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f334e4a3700 1 --2- 172.21.15.36:0/1956755043 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3348147f50 0x7f3348141210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:30.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f334e4a3700 1 -- 172.21.15.36:0/1956755043 >> v1:172.21.15.36:6789/0 conn(0x7f3348141700 legacy=0x7f334805fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:30.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f334e4a3700 1 -- 172.21.15.36:0/1956755043 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3348147ab0 con 0x7f334805f770 2022-06-17T12:57:30.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f334dca2700 1 --2- 172.21.15.36:0/1956755043 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3348147f50 0x7f3348141210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:57:30.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f334e4a3700 1 --2- 172.21.15.36:0/1956755043 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f334805f770 0x7f334814b440 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7f33400032f0 tx=0x7f3340009290).ready entity=mon.0 client_cookie=5142fbf4c44c8dc6 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:30.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f334d4a1700 1 -- 172.21.15.36:0/1956755043 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f334000b8a0 con 0x7f334805f770 2022-06-17T12:57:30.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f334d4a1700 1 -- 172.21.15.36:0/1956755043 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334000ba00 con 0x7f334805f770 2022-06-17T12:57:30.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f334d4a1700 1 -- 172.21.15.36:0/1956755043 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f334000bb70 con 0x7f334805f770 2022-06-17T12:57:30.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f3350707700 1 -- 172.21.15.36:0/1956755043 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f334805f770 msgr2=0x7f334814b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:30.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f3350707700 1 --2- 172.21.15.36:0/1956755043 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f334805f770 0x7f334814b440 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7f33400032f0 tx=0x7f3340009290).stop 2022-06-17T12:57:30.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f3350707700 1 -- 172.21.15.36:0/1956755043 shutdown_connections 2022-06-17T12:57:30.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f3350707700 1 --2- 172.21.15.36:0/1956755043 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3348147f50 0x7f3348141210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:30.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f3350707700 1 --2- 172.21.15.36:0/1956755043 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f334805f770 0x7f334814b440 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:30.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.694+0000 7f3350707700 1 -- 172.21.15.36:0/1956755043 >> 172.21.15.36:0/1956755043 conn(0x7f334805c550 msgr2=0x7f334805c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:30.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f3350707700 1 -- 172.21.15.36:0/1956755043 shutdown_connections 2022-06-17T12:57:30.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f3350707700 1 -- 172.21.15.36:0/1956755043 wait complete. 2022-06-17T12:57:30.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f3350707700 1 Processor -- start 2022-06-17T12:57:30.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f3350707700 1 -- start start 2022-06-17T12:57:30.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f3350707700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f334805f770 0x7f3348156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:30.716 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f3350707700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3348141700 0x7f3348141ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:30.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f3350707700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3348147f50 0x7f334815a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:30.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f3350707700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33480b51e0 con 0x7f3348141700 2022-06-17T12:57:30.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f3350707700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33480b4f40 con 0x7f334805f770 2022-06-17T12:57:30.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f3350707700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3348159fb0 con 0x7f3348147f50 2022-06-17T12:57:30.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f334dca2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3348141700 0x7f3348141ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:30.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f334dca2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3348141700 0x7f3348141ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40982/0 (socket says 172.21.15.36:40982) 2022-06-17T12:57:30.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f334dca2700 1 -- 172.21.15.36:0/406173263 learned_addr learned my addr 172.21.15.36:0/406173263 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:30.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f334e4a3700 1 --2- 172.21.15.36:0/406173263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f334805f770 0x7f3348156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:30.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f334eca4700 1 --2- 172.21.15.36:0/406173263 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3348147f50 0x7f334815a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:30.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f334dca2700 1 -- 172.21.15.36:0/406173263 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3348147f50 msgr2=0x7f334815a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:30.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f334dca2700 1 --2- 172.21.15.36:0/406173263 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3348147f50 0x7f334815a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:30.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f334dca2700 1 -- 172.21.15.36:0/406173263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f334805f770 msgr2=0x7f3348156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:30.741 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f334dca2700 1 --2- 172.21.15.36:0/406173263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f334805f770 0x7f3348156890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:30.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f334dca2700 1 -- 172.21.15.36:0/406173263 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3340009530 con 0x7f3348141700 2022-06-17T12:57:30.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f334eca4700 1 --2- 172.21.15.36:0/406173263 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3348147f50 0x7f334815a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:57:30.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f334e4a3700 1 --2- 172.21.15.36:0/406173263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f334805f770 0x7f3348156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:57:30.743 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f334dca2700 1 --2- 172.21.15.36:0/406173263 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3348141700 0x7f3348141ff0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 rx=0x7f3338006940 tx=0x7f3338008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:30.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f33377fe700 1 -- 172.21.15.36:0/406173263 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3338011780 con 0x7f3348141700 2022-06-17T12:57:30.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f33377fe700 1 -- 172.21.15.36:0/406173263 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33380118e0 con 0x7f3348141700 2022-06-17T12:57:30.745 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f3350707700 1 -- 172.21.15.36:0/406173263 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3348156d80 con 0x7f3348141700 2022-06-17T12:57:30.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f33377fe700 1 -- 172.21.15.36:0/406173263 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3338011c40 con 0x7f3348141700 2022-06-17T12:57:30.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.698+0000 7f3350707700 1 -- 172.21.15.36:0/406173263 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3348142700 con 0x7f3348141700 2022-06-17T12:57:30.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.702+0000 7f3350707700 1 -- 172.21.15.36:0/406173263 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33481483c0 con 0x7f3348141700 2022-06-17T12:57:30.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.702+0000 7f33377fe700 1 -- 172.21.15.36:0/406173263 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f3338012420 con 0x7f3348141700 2022-06-17T12:57:30.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.702+0000 7f33377fe700 1 --2- 172.21.15.36:0/406173263 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3328053a40 0x7f3328055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:30.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.702+0000 7f33377fe700 1 -- 172.21.15.36:0/406173263 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f333806be60 con 0x7f3348141700 2022-06-17T12:57:30.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.702+0000 7f334e4a3700 1 --2- 172.21.15.36:0/406173263 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3328053a40 0x7f3328055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:30.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.706+0000 7f334e4a3700 1 --2- 172.21.15.36:0/406173263 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3328053a40 0x7f3328055f00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f3340018000 tx=0x7f3340016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:30.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.706+0000 7f33377fe700 1 -- 172.21.15.36:0/406173263 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3338042c00 con 0x7f3348141700 2022-06-17T12:57:30.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.870+0000 7f3350707700 1 -- 172.21.15.36:0/406173263 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f334815a170 con 0x7f3348141700 2022-06-17T12:57:30.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.870+0000 7f33377fe700 1 -- 172.21.15.36:0/406173263 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f3338046220 con 0x7f3348141700 2022-06-17T12:57:30.864 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:57:30.864 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":83,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8599,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/4216085219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4216085219},{"type":"v1","addr":"172.21.15.70:6833","nonce":4216085219}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":82}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:57:14.606323+0000","modified":"2022-06-17T12:57:15.610154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8805":{"gid":8805,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.36:6835/2534551587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":2534551587},{"type":"v1","addr":"172.21.15.36:6835","nonce":2534551587}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-06-17T12:57:30.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.874+0000 7f3350707700 1 -- 172.21.15.36:0/406173263 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3328053a40 msgr2=0x7f3328055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:30.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.874+0000 7f3350707700 1 --2- 172.21.15.36:0/406173263 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3328053a40 0x7f3328055f00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f3340018000 tx=0x7f3340016000).stop 2022-06-17T12:57:30.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.874+0000 7f3350707700 1 -- 172.21.15.36:0/406173263 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3348141700 msgr2=0x7f3348141ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:30.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.874+0000 7f3350707700 1 --2- 172.21.15.36:0/406173263 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3348141700 0x7f3348141ff0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 rx=0x7f3338006940 tx=0x7f3338008040).stop 2022-06-17T12:57:30.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.874+0000 7f3350707700 1 -- 172.21.15.36:0/406173263 shutdown_connections 2022-06-17T12:57:30.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.874+0000 7f3350707700 1 --2- 172.21.15.36:0/406173263 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3328053a40 0x7f3328055f00 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:30.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.874+0000 7f3350707700 1 --2- 172.21.15.36:0/406173263 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3348147f50 0x7f334815a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:30.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.874+0000 7f3350707700 1 --2- 172.21.15.36:0/406173263 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3348141700 0x7f3348141ff0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:30.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.874+0000 7f3350707700 1 --2- 172.21.15.36:0/406173263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f334805f770 0x7f3348156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:30.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.874+0000 7f3350707700 1 -- 172.21.15.36:0/406173263 >> 172.21.15.36:0/406173263 conn(0x7f334805c550 msgr2=0x7f33480b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:30.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.874+0000 7f3350707700 1 -- 172.21.15.36:0/406173263 shutdown_connections 2022-06-17T12:57:30.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:30.874+0000 7f3350707700 1 -- 172.21.15.36:0/406173263 wait complete. 2022-06-17T12:57:30.875 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 83 2022-06-17T12:57:30.901 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-17T12:57:31.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd76caa2700 1 Processor -- start 2022-06-17T12:57:31.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd76caa2700 1 -- start start 2022-06-17T12:57:31.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd76caa2700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd7680b2200 0x7fd7680af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:31.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd76caa2700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd7680b2cd0 0x7fd768147600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:31.031 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd76caa2700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7680b25e0 con 0x7fd7680af8d0 2022-06-17T12:57:31.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd76caa2700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7680ad790 con 0x7fd7680b2cd0 2022-06-17T12:57:31.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd76caa2700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7680aef40 con 0x7fd7680b2200 2022-06-17T12:57:31.032 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd766ffd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fd7680af8d0 0x7fd76814a8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58518/0 (socket says 172.21.15.36:58518) 2022-06-17T12:57:31.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd766ffd700 1 -- 172.21.15.36:0/2202371186 learned_addr learned my addr 172.21.15.36:0/2202371186 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:31.033 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd767fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fd7680b2cd0 0x7fd768147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:31.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd7677fe700 1 --2- 172.21.15.36:0/2202371186 >> v2:172.21.15.70:3300/0 conn(0x7fd7680b2200 0x7fd7680af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:31.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd7667fc700 1 -- 172.21.15.36:0/2202371186 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2199748137 0 0) 0x7fd7680b25e0 con 0x7fd7680af8d0 2022-06-17T12:57:31.034 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd7667fc700 1 -- 172.21.15.36:0/2202371186 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd748003260 con 0x7fd7680af8d0 2022-06-17T12:57:31.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd767fff700 1 -- 172.21.15.36:0/2202371186 >> v2:172.21.15.70:3300/0 conn(0x7fd7680b2200 msgr2=0x7fd7680af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:31.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd767fff700 1 --2- 172.21.15.36:0/2202371186 >> v2:172.21.15.70:3300/0 conn(0x7fd7680b2200 0x7fd7680af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:31.035 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd767fff700 1 -- 172.21.15.36:0/2202371186 >> v1:172.21.15.36:6789/0 conn(0x7fd7680af8d0 legacy=0x7fd76814a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:31.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd767fff700 1 -- 172.21.15.36:0/2202371186 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7680b1950 con 0x7fd7680b2cd0 2022-06-17T12:57:31.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd767fff700 1 --2- 172.21.15.36:0/2202371186 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7680b2cd0 0x7fd768147600 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 rx=0x7fd758002f70 tx=0x7fd758006df0).ready entity=mon.0 client_cookie=d8a7ac2d6d967c60 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:31.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd7667fc700 1 -- 172.21.15.36:0/2202371186 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd75800d4f0 con 0x7fd7680b2cd0 2022-06-17T12:57:31.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd7667fc700 1 -- 172.21.15.36:0/2202371186 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd758000e90 con 0x7fd7680b2cd0 2022-06-17T12:57:31.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd7667fc700 1 -- 172.21.15.36:0/2202371186 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd75800d840 con 0x7fd7680b2cd0 2022-06-17T12:57:31.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd76caa2700 1 -- 172.21.15.36:0/2202371186 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7680b2cd0 msgr2=0x7fd768147600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:31.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd76caa2700 1 --2- 172.21.15.36:0/2202371186 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7680b2cd0 0x7fd768147600 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 rx=0x7fd758002f70 tx=0x7fd758006df0).stop 2022-06-17T12:57:31.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd76caa2700 1 -- 172.21.15.36:0/2202371186 shutdown_connections 2022-06-17T12:57:31.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd76caa2700 1 --2- 172.21.15.36:0/2202371186 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7680b2cd0 0x7fd768147600 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:31.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd76caa2700 1 --2- 172.21.15.36:0/2202371186 >> v2:172.21.15.70:3300/0 conn(0x7fd7680b2200 0x7fd7680af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:31.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd76caa2700 1 -- 172.21.15.36:0/2202371186 >> 172.21.15.36:0/2202371186 conn(0x7fd76805f770 msgr2=0x7fd76805fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:31.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd76caa2700 1 -- 172.21.15.36:0/2202371186 shutdown_connections 2022-06-17T12:57:31.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.042+0000 7fd76caa2700 1 -- 172.21.15.36:0/2202371186 wait complete. 2022-06-17T12:57:31.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.046+0000 7fd76caa2700 1 Processor -- start 2022-06-17T12:57:31.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.046+0000 7fd76caa2700 1 -- start start 2022-06-17T12:57:31.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.046+0000 7fd76caa2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7680af8d0 0x7fd768063f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:31.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.046+0000 7fd76caa2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7680b2200 0x7fd768062390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:31.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.046+0000 7fd76caa2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7680b2cd0 0x7fd768142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:31.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.046+0000 7fd76caa2700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd768064450 con 0x7fd7680b2cd0 2022-06-17T12:57:31.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.046+0000 7fd76caa2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7680646a0 con 0x7fd7680b2200 2022-06-17T12:57:31.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.046+0000 7fd76caa2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd768064940 con 0x7fd7680af8d0 2022-06-17T12:57:31.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.046+0000 7fd767fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7680b2cd0 0x7fd768142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:31.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.046+0000 7fd767fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7680b2cd0 0x7fd768142250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:40998/0 (socket says 172.21.15.36:40998) 2022-06-17T12:57:31.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.046+0000 7fd767fff700 1 -- 172.21.15.36:0/2842777624 learned_addr learned my addr 172.21.15.36:0/2842777624 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:31.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.050+0000 7fd7677fe700 1 --2- 172.21.15.36:0/2842777624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7680af8d0 0x7fd768063f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:31.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.050+0000 7fd766ffd700 1 --2- 172.21.15.36:0/2842777624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7680b2200 0x7fd768062390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:31.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.050+0000 7fd767fff700 1 -- 172.21.15.36:0/2842777624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7680af8d0 msgr2=0x7fd768063f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:31.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.050+0000 7fd767fff700 1 --2- 172.21.15.36:0/2842777624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7680af8d0 0x7fd768063f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:31.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.050+0000 7fd767fff700 1 -- 172.21.15.36:0/2842777624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7680b2200 msgr2=0x7fd768062390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:31.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.050+0000 7fd767fff700 1 --2- 172.21.15.36:0/2842777624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7680b2200 0x7fd768062390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:31.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.050+0000 7fd767fff700 1 -- 172.21.15.36:0/2842777624 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75800b180 con 0x7fd7680b2cd0 2022-06-17T12:57:31.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.050+0000 7fd767fff700 1 --2- 172.21.15.36:0/2842777624 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7680b2cd0 0x7fd768142250 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 rx=0x7fd758002f70 tx=0x7fd758014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:31.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.050+0000 7fd764ff9700 1 -- 172.21.15.36:0/2842777624 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd758013ac0 con 0x7fd7680b2cd0 2022-06-17T12:57:31.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.050+0000 7fd764ff9700 1 -- 172.21.15.36:0/2842777624 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd758013c20 con 0x7fd7680b2cd0 2022-06-17T12:57:31.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.050+0000 7fd76caa2700 1 -- 172.21.15.36:0/2842777624 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd768064be0 con 0x7fd7680b2cd0 2022-06-17T12:57:31.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.050+0000 7fd764ff9700 1 -- 172.21.15.36:0/2842777624 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd75801e400 con 0x7fd7680b2cd0 2022-06-17T12:57:31.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.050+0000 7fd76caa2700 1 -- 172.21.15.36:0/2842777624 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd768062aa0 con 0x7fd7680b2cd0 2022-06-17T12:57:31.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.050+0000 7fd76caa2700 1 -- 172.21.15.36:0/2842777624 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd768064d70 con 0x7fd7680b2cd0 2022-06-17T12:57:31.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.050+0000 7fd764ff9700 1 -- 172.21.15.36:0/2842777624 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd758025030 con 0x7fd7680b2cd0 2022-06-17T12:57:31.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.050+0000 7fd764ff9700 1 --2- 172.21.15.36:0/2842777624 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd74c053a60 0x7fd74c055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:31.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.050+0000 7fd764ff9700 1 -- 172.21.15.36:0/2842777624 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd75806cc40 con 0x7fd7680b2cd0 2022-06-17T12:57:31.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.050+0000 7fd7677fe700 1 --2- 172.21.15.36:0/2842777624 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd74c053a60 0x7fd74c055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:31.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.054+0000 7fd7677fe700 1 --2- 172.21.15.36:0/2842777624 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd74c053a60 0x7fd74c055f20 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7fd76000b000 tx=0x7fd760008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:31.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.054+0000 7fd764ff9700 1 -- 172.21.15.36:0/2842777624 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd758043a80 con 0x7fd7680b2cd0 2022-06-17T12:57:31.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.222+0000 7fd76caa2700 1 -- 172.21.15.36:0/2842777624 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd7680ad330 con 0x7fd7680b2cd0 2022-06-17T12:57:31.728 INFO:tasks.ceph.mon.a.smithi036.stderr:2022-06-17T12:57:31.742+0000 7fb7a2649700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-17T12:57:31.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.754+0000 7fd764ff9700 1 -- 172.21.15.36:0/2842777624 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd758023070 con 0x7fd7680b2cd0 2022-06-17T12:57:31.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.758+0000 7fd76caa2700 1 -- 172.21.15.36:0/2842777624 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd74c053a60 msgr2=0x7fd74c055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:31.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.758+0000 7fd76caa2700 1 --2- 172.21.15.36:0/2842777624 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd74c053a60 0x7fd74c055f20 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7fd76000b000 tx=0x7fd760008040).stop 2022-06-17T12:57:31.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.758+0000 7fd76caa2700 1 -- 172.21.15.36:0/2842777624 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7680b2cd0 msgr2=0x7fd768142250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:31.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.758+0000 7fd76caa2700 1 --2- 172.21.15.36:0/2842777624 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7680b2cd0 0x7fd768142250 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 rx=0x7fd758002f70 tx=0x7fd758014000).stop 2022-06-17T12:57:31.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.758+0000 7fd76caa2700 1 -- 172.21.15.36:0/2842777624 shutdown_connections 2022-06-17T12:57:31.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.758+0000 7fd76caa2700 1 --2- 172.21.15.36:0/2842777624 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd74c053a60 0x7fd74c055f20 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:31.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.758+0000 7fd76caa2700 1 --2- 172.21.15.36:0/2842777624 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd7680b2cd0 0x7fd768142250 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:31.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.758+0000 7fd76caa2700 1 --2- 172.21.15.36:0/2842777624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7680b2200 0x7fd768062390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:31.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.758+0000 7fd76caa2700 1 --2- 172.21.15.36:0/2842777624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7680af8d0 0x7fd768063f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:31.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.758+0000 7fd76caa2700 1 -- 172.21.15.36:0/2842777624 >> 172.21.15.36:0/2842777624 conn(0x7fd76805f770 msgr2=0x7fd7680b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:31.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.758+0000 7fd76caa2700 1 -- 172.21.15.36:0/2842777624 shutdown_connections 2022-06-17T12:57:31.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.758+0000 7fd76caa2700 1 -- 172.21.15.36:0/2842777624 wait complete. 2022-06-17T12:57:31.751 INFO:teuthology.orchestra.run.smithi036.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-17T12:57:31.764 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-17T12:57:31.783 INFO:tasks.ceph.mds.a.smithi036.stdout:starting mds.a at 2022-06-17T12:57:31.891 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.906+0000 7fcd07fff700 1 Processor -- start 2022-06-17T12:57:31.892 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.906+0000 7fcd07fff700 1 -- start start 2022-06-17T12:57:31.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.910+0000 7fcd07fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcd08063810 0x7fcd081446c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:31.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.910+0000 7fcd07fff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd08144290 con 0x7fcd0805f770 2022-06-17T12:57:31.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.910+0000 7fcd07fff700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd08063bf0 con 0x7fcd080b6cc0 2022-06-17T12:57:31.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.910+0000 7fcd07fff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd080b25c0 con 0x7fcd08063810 2022-06-17T12:57:31.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.910+0000 7fcd06ffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcd08063810 0x7fcd081446c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:31.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.910+0000 7fcd077fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fcd0805f770 0x7fcd0813fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58548/0 (socket says 172.21.15.36:58548) 2022-06-17T12:57:31.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.910+0000 7fcd077fe700 1 -- 172.21.15.36:0/2068699436 learned_addr learned my addr 172.21.15.36:0/2068699436 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:31.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.910+0000 7fcd06ffd700 1 -- 172.21.15.36:0/2068699436 >> v1:172.21.15.70:6789/0 conn(0x7fcd080b6cc0 legacy=0x7fcd08144bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:31.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.910+0000 7fcd06ffd700 1 -- 172.21.15.36:0/2068699436 >> v1:172.21.15.36:6789/0 conn(0x7fcd0805f770 legacy=0x7fcd0813fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:31.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.910+0000 7fcd067fc700 1 -- 172.21.15.36:0/2068699436 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 677271580 0 0) 0x7fcd08063bf0 con 0x7fcd080b6cc0 2022-06-17T12:57:31.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.910+0000 7fcd06ffd700 1 -- 172.21.15.36:0/2068699436 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd080b2860 con 0x7fcd08063810 2022-06-17T12:57:31.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.910+0000 7fcd06ffd700 1 --2- 172.21.15.36:0/2068699436 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd08063810 0x7fcd081446c0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 rx=0x7fcd00001490 tx=0x7fccf8006df0).ready entity=mon.0 client_cookie=de7d8c148efcb1c8 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:31.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.910+0000 7fcd067fc700 1 -- 172.21.15.36:0/2068699436 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccf800d5c0 con 0x7fcd08063810 2022-06-17T12:57:31.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.910+0000 7fcd067fc700 1 -- 172.21.15.36:0/2068699436 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf8000e90 con 0x7fcd08063810 2022-06-17T12:57:31.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.910+0000 7fcd067fc700 1 -- 172.21.15.36:0/2068699436 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccf800d910 con 0x7fcd08063810 2022-06-17T12:57:31.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.910+0000 7fcd07fff700 1 -- 172.21.15.36:0/2068699436 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd08063810 msgr2=0x7fcd081446c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:31.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.910+0000 7fcd07fff700 1 --2- 172.21.15.36:0/2068699436 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd08063810 0x7fcd081446c0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 rx=0x7fcd00001490 tx=0x7fccf8006df0).stop 2022-06-17T12:57:31.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.910+0000 7fcd07fff700 1 -- 172.21.15.36:0/2068699436 shutdown_connections 2022-06-17T12:57:31.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.910+0000 7fcd07fff700 1 --2- 172.21.15.36:0/2068699436 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd08063810 0x7fcd081446c0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:31.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.910+0000 7fcd07fff700 1 -- 172.21.15.36:0/2068699436 >> 172.21.15.36:0/2068699436 conn(0x7fcd080614b0 msgr2=0x7fcd080618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:31.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.910+0000 7fcd07fff700 1 -- 172.21.15.36:0/2068699436 shutdown_connections 2022-06-17T12:57:31.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.910+0000 7fcd07fff700 1 -- 172.21.15.36:0/2068699436 wait complete. 2022-06-17T12:57:31.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.914+0000 7fcd07fff700 1 Processor -- start 2022-06-17T12:57:31.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.914+0000 7fcd07fff700 1 -- start start 2022-06-17T12:57:31.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.918+0000 7fcd07fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd0805f770 0x7fcd081500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:31.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.918+0000 7fcd07fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd08063810 0x7fcd081480a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:31.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.918+0000 7fcd07fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd080b6cc0 0x7fcd080b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:31.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.918+0000 7fcd07fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd080af910 con 0x7fcd080b6cc0 2022-06-17T12:57:31.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.918+0000 7fcd07fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd080b42e0 con 0x7fcd08063810 2022-06-17T12:57:31.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.918+0000 7fcd07fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd080ae710 con 0x7fcd0805f770 2022-06-17T12:57:31.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.918+0000 7fcd077fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd080b6cc0 0x7fcd080b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:31.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.918+0000 7fcd077fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd080b6cc0 0x7fcd080b4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41024/0 (socket says 172.21.15.36:41024) 2022-06-17T12:57:31.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.918+0000 7fcd077fe700 1 -- 172.21.15.36:0/2759492897 learned_addr learned my addr 172.21.15.36:0/2759492897 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:31.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.918+0000 7fccfffff700 1 --2- 172.21.15.36:0/2759492897 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd08063810 0x7fcd081480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:31.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.918+0000 7fcd06ffd700 1 --2- 172.21.15.36:0/2759492897 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd0805f770 0x7fcd081500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:31.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.918+0000 7fcd077fe700 1 -- 172.21.15.36:0/2759492897 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd0805f770 msgr2=0x7fcd081500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:31.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.918+0000 7fcd077fe700 1 --2- 172.21.15.36:0/2759492897 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd0805f770 0x7fcd081500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:31.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.918+0000 7fcd077fe700 1 -- 172.21.15.36:0/2759492897 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd08063810 msgr2=0x7fcd081480a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:31.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.918+0000 7fcd077fe700 1 --2- 172.21.15.36:0/2759492897 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd08063810 0x7fcd081480a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:31.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.918+0000 7fcd077fe700 1 -- 172.21.15.36:0/2759492897 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf800b180 con 0x7fcd080b6cc0 2022-06-17T12:57:31.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.918+0000 7fcd077fe700 1 --2- 172.21.15.36:0/2759492897 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd080b6cc0 0x7fcd080b4740 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7fcd080ae8a0 tx=0x7fcd00007f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:31.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.918+0000 7fcd04ff9700 1 -- 172.21.15.36:0/2759492897 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd00005ba0 con 0x7fcd080b6cc0 2022-06-17T12:57:31.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.918+0000 7fcd04ff9700 1 -- 172.21.15.36:0/2759492897 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd00005d00 con 0x7fcd080b6cc0 2022-06-17T12:57:31.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.918+0000 7fcd07fff700 1 -- 172.21.15.36:0/2759492897 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd080ae470 con 0x7fcd080b6cc0 2022-06-17T12:57:31.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.918+0000 7fcd04ff9700 1 -- 172.21.15.36:0/2759492897 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd00007300 con 0x7fcd080b6cc0 2022-06-17T12:57:31.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.918+0000 7fcd07fff700 1 -- 172.21.15.36:0/2759492897 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd080ad850 con 0x7fcd080b6cc0 2022-06-17T12:57:31.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.918+0000 7fcd07fff700 1 -- 172.21.15.36:0/2759492897 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd080b7240 con 0x7fcd080b6cc0 2022-06-17T12:57:31.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.922+0000 7fcd04ff9700 1 -- 172.21.15.36:0/2759492897 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fcd00010030 con 0x7fcd080b6cc0 2022-06-17T12:57:31.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.922+0000 7fcd04ff9700 1 --2- 172.21.15.36:0/2759492897 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fccec053a40 0x7fccec055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:31.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.922+0000 7fcd04ff9700 1 -- 172.21.15.36:0/2759492897 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fcd0006c0e0 con 0x7fcd080b6cc0 2022-06-17T12:57:31.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.922+0000 7fcd06ffd700 1 --2- 172.21.15.36:0/2759492897 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fccec053a40 0x7fccec055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:31.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.922+0000 7fcd06ffd700 1 --2- 172.21.15.36:0/2759492897 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fccec053a40 0x7fccec055f00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7fccf801a000 tx=0x7fccf8014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:31.925 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:31.926+0000 7fcd04ff9700 1 -- 172.21.15.36:0/2759492897 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcd00042df0 con 0x7fcd080b6cc0 2022-06-17T12:57:32.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.090+0000 7fcd07fff700 1 -- 172.21.15.36:0/2759492897 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd08057410 con 0x7fcd080b6cc0 2022-06-17T12:57:32.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.094+0000 7fcd04ff9700 1 -- 172.21.15.36:0/2759492897 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+1782 (secure 0 0 0) 0x7fcd00046410 con 0x7fcd080b6cc0 2022-06-17T12:57:32.079 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T12:57:32.079 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":84,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8599,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6833/4216085219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4216085219},{"type":"v1","addr":"172.21.15.70:6833","nonce":4216085219}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":82}],"filesystems":[{"mdsmap":{"epoch":84,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-17T12:57:14.606323+0000","modified":"2022-06-17T12:57:31.745644+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":152,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-06-17T12:57:32.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.094+0000 7fcd07fff700 1 -- 172.21.15.36:0/2759492897 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fccec053a40 msgr2=0x7fccec055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:32.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.094+0000 7fcd07fff700 1 --2- 172.21.15.36:0/2759492897 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fccec053a40 0x7fccec055f00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7fccf801a000 tx=0x7fccf8014000).stop 2022-06-17T12:57:32.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.094+0000 7fcd07fff700 1 -- 172.21.15.36:0/2759492897 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd080b6cc0 msgr2=0x7fcd080b4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:32.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.094+0000 7fcd07fff700 1 --2- 172.21.15.36:0/2759492897 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd080b6cc0 0x7fcd080b4740 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7fcd080ae8a0 tx=0x7fcd00007f00).stop 2022-06-17T12:57:32.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.098+0000 7fcd07fff700 1 -- 172.21.15.36:0/2759492897 shutdown_connections 2022-06-17T12:57:32.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.098+0000 7fcd07fff700 1 --2- 172.21.15.36:0/2759492897 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fccec053a40 0x7fccec055f00 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:32.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.098+0000 7fcd07fff700 1 --2- 172.21.15.36:0/2759492897 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd080b6cc0 0x7fcd080b4740 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:32.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.098+0000 7fcd07fff700 1 --2- 172.21.15.36:0/2759492897 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd08063810 0x7fcd081480a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:32.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.098+0000 7fcd07fff700 1 --2- 172.21.15.36:0/2759492897 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd0805f770 0x7fcd081500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:32.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.098+0000 7fcd07fff700 1 -- 172.21.15.36:0/2759492897 >> 172.21.15.36:0/2759492897 conn(0x7fcd080614b0 msgr2=0x7fcd08066df0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:32.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.098+0000 7fcd07fff700 1 -- 172.21.15.36:0/2759492897 shutdown_connections 2022-06-17T12:57:32.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.098+0000 7fcd07fff700 1 -- 172.21.15.36:0/2759492897 wait complete. 2022-06-17T12:57:32.087 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 84 2022-06-17T12:57:32.111 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-17T12:57:32.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.282+0000 7f446bfff700 1 Processor -- start 2022-06-17T12:57:32.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.282+0000 7f446bfff700 1 -- start start 2022-06-17T12:57:32.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.282+0000 7f446bfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f446c14b2b0 0x7f446c060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:32.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.282+0000 7f446bfff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f446c147bd0 0x7f446c1442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:32.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.282+0000 7f446bfff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f446c14b870 con 0x7f446c0b27b0 2022-06-17T12:57:32.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.282+0000 7f446bfff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f446c0ac500 con 0x7f446c147bd0 2022-06-17T12:57:32.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.282+0000 7f446bfff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446c0ace90 con 0x7f446c14b2b0 2022-06-17T12:57:32.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.282+0000 7f446b7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f446c147bd0 0x7f446c1442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:32.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.282+0000 7f446affd700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f446c0b27b0 0x7f446c0ac780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58558/0 (socket says 172.21.15.36:58558) 2022-06-17T12:57:32.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.282+0000 7f446affd700 1 -- 172.21.15.36:0/833127497 learned_addr learned my addr 172.21.15.36:0/833127497 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:32.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.282+0000 7f446a7fc700 1 --2- 172.21.15.36:0/833127497 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f446c14b2b0 0x7f446c060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:32.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.282+0000 7f446b7fe700 1 -- 172.21.15.36:0/833127497 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f446c14b2b0 msgr2=0x7f446c060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:32.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.282+0000 7f446b7fe700 1 --2- 172.21.15.36:0/833127497 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f446c14b2b0 0x7f446c060ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:32.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.282+0000 7f446b7fe700 1 -- 172.21.15.36:0/833127497 >> v1:172.21.15.36:6789/0 conn(0x7f446c0b27b0 legacy=0x7f446c0ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:32.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.282+0000 7f4469ffb700 1 -- 172.21.15.36:0/833127497 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1297325151 0 0) 0x7f446c14b870 con 0x7f446c0b27b0 2022-06-17T12:57:32.318 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.282+0000 7f446b7fe700 1 -- 172.21.15.36:0/833127497 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c0b2b90 con 0x7f446c147bd0 2022-06-17T12:57:32.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.282+0000 7f446a7fc700 1 --2- 172.21.15.36:0/833127497 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f446c14b2b0 0x7f446c060ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T12:57:32.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.282+0000 7f446b7fe700 1 --2- 172.21.15.36:0/833127497 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f446c147bd0 0x7f446c1442d0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 rx=0x7f44640032f0 tx=0x7f44640091d0).ready entity=mon.0 client_cookie=3d976504c67584ec server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:32.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.282+0000 7f4469ffb700 1 -- 172.21.15.36:0/833127497 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f446400b6c0 con 0x7f446c147bd0 2022-06-17T12:57:32.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.282+0000 7f4469ffb700 1 -- 172.21.15.36:0/833127497 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446400b820 con 0x7f446c147bd0 2022-06-17T12:57:32.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.282+0000 7f4469ffb700 1 -- 172.21.15.36:0/833127497 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f446400b990 con 0x7f446c147bd0 2022-06-17T12:57:32.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.286+0000 7f446bfff700 1 -- 172.21.15.36:0/833127497 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f446c147bd0 msgr2=0x7f446c1442d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:32.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.286+0000 7f446bfff700 1 --2- 172.21.15.36:0/833127497 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f446c147bd0 0x7f446c1442d0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 rx=0x7f44640032f0 tx=0x7f44640091d0).stop 2022-06-17T12:57:32.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.286+0000 7f446bfff700 1 -- 172.21.15.36:0/833127497 shutdown_connections 2022-06-17T12:57:32.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.286+0000 7f446bfff700 1 --2- 172.21.15.36:0/833127497 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f446c147bd0 0x7f446c1442d0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:32.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.286+0000 7f446bfff700 1 --2- 172.21.15.36:0/833127497 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f446c14b2b0 0x7f446c060ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:32.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.286+0000 7f446bfff700 1 -- 172.21.15.36:0/833127497 >> 172.21.15.36:0/833127497 conn(0x7f446c0b5d10 msgr2=0x7f446c0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:32.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.286+0000 7f446bfff700 1 -- 172.21.15.36:0/833127497 shutdown_connections 2022-06-17T12:57:32.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.286+0000 7f446bfff700 1 -- 172.21.15.36:0/833127497 wait complete. 2022-06-17T12:57:32.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.286+0000 7f446bfff700 1 Processor -- start 2022-06-17T12:57:32.330 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.286+0000 7f446bfff700 1 -- start start 2022-06-17T12:57:32.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.290+0000 7f446bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f446c0b27b0 0x7f446c0b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:32.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.290+0000 7f446bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f446c147bd0 0x7f446c144630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:32.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.290+0000 7f446bfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f446c14b2b0 0x7f446c144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:32.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.290+0000 7f446a7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f446c147bd0 0x7f446c144630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:32.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.290+0000 7f446bfff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446c0aa980 con 0x7f446c147bd0 2022-06-17T12:57:32.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.290+0000 7f446a7fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f446c147bd0 0x7f446c144630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41036/0 (socket says 172.21.15.36:41036) 2022-06-17T12:57:32.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.290+0000 7f446bfff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446c066890 con 0x7f446c0b27b0 2022-06-17T12:57:32.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.290+0000 7f446a7fc700 1 -- 172.21.15.36:0/524358303 learned_addr learned my addr 172.21.15.36:0/524358303 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:32.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.290+0000 7f446bfff700 1 -- 172.21.15.36:0/524358303 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446c0aa520 con 0x7f446c14b2b0 2022-06-17T12:57:32.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.290+0000 7f446affd700 1 --2- 172.21.15.36:0/524358303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f446c0b27b0 0x7f446c0b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:32.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.290+0000 7f446b7fe700 1 --2- 172.21.15.36:0/524358303 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f446c14b2b0 0x7f446c144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:32.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.290+0000 7f446a7fc700 1 -- 172.21.15.36:0/524358303 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f446c14b2b0 msgr2=0x7f446c144b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:32.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.290+0000 7f446a7fc700 1 --2- 172.21.15.36:0/524358303 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f446c14b2b0 0x7f446c144b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:32.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.290+0000 7f446a7fc700 1 -- 172.21.15.36:0/524358303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f446c0b27b0 msgr2=0x7f446c0b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:32.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.290+0000 7f446a7fc700 1 --2- 172.21.15.36:0/524358303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f446c0b27b0 0x7f446c0b3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:32.366 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.290+0000 7f446a7fc700 1 -- 172.21.15.36:0/524358303 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445c00b180 con 0x7f446c147bd0 2022-06-17T12:57:32.366 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.290+0000 7f446a7fc700 1 --2- 172.21.15.36:0/524358303 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f446c147bd0 0x7f446c144630 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 rx=0x7f445c00c000 tx=0x7f445c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:32.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.290+0000 7f445bfff700 1 -- 172.21.15.36:0/524358303 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f445c00c800 con 0x7f446c147bd0 2022-06-17T12:57:32.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.290+0000 7f445bfff700 1 -- 172.21.15.36:0/524358303 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445c00c960 con 0x7f446c147bd0 2022-06-17T12:57:32.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.290+0000 7f446bfff700 1 -- 172.21.15.36:0/524358303 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4464009470 con 0x7f446c147bd0 2022-06-17T12:57:32.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.290+0000 7f445bfff700 1 -- 172.21.15.36:0/524358303 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f445c00ccc0 con 0x7f446c147bd0 2022-06-17T12:57:32.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.290+0000 7f446bfff700 1 -- 172.21.15.36:0/524358303 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f446c0a70e0 con 0x7f446c147bd0 2022-06-17T12:57:32.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.290+0000 7f446bfff700 1 -- 172.21.15.36:0/524358303 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f446c057410 con 0x7f446c147bd0 2022-06-17T12:57:32.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.290+0000 7f445bfff700 1 -- 172.21.15.36:0/524358303 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f445c01c460 con 0x7f446c147bd0 2022-06-17T12:57:32.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.294+0000 7f445bfff700 1 --2- 172.21.15.36:0/524358303 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4448053920 0x7f4448055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:32.370 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.294+0000 7f445bfff700 1 -- 172.21.15.36:0/524358303 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f445c06c600 con 0x7f446c147bd0 2022-06-17T12:57:32.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.294+0000 7f446affd700 1 --2- 172.21.15.36:0/524358303 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4448053920 0x7f4448055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:32.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.294+0000 7f446affd700 1 --2- 172.21.15.36:0/524358303 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4448053920 0x7f4448055de0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7f446000d000 tx=0x7f4460009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:32.371 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.298+0000 7f445bfff700 1 -- 172.21.15.36:0/524358303 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f445c043350 con 0x7f446c147bd0 2022-06-17T12:57:32.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:32.470+0000 7f446bfff700 1 -- 172.21.15.36:0/524358303 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f446c0a72d0 con 0x7f446c147bd0 2022-06-17T12:57:33.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.234+0000 7f445bfff700 1 -- 172.21.15.36:0/524358303 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v85) v1 ==== 104+0+0 (secure 0 0 0) 0x7f445c046970 con 0x7f446c147bd0 2022-06-17T12:57:33.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.238+0000 7f446bfff700 1 -- 172.21.15.36:0/524358303 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4448053920 msgr2=0x7f4448055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:33.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.238+0000 7f446bfff700 1 --2- 172.21.15.36:0/524358303 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4448053920 0x7f4448055de0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7f446000d000 tx=0x7f4460009040).stop 2022-06-17T12:57:33.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.238+0000 7f446bfff700 1 -- 172.21.15.36:0/524358303 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f446c147bd0 msgr2=0x7f446c144630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:33.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.238+0000 7f446bfff700 1 --2- 172.21.15.36:0/524358303 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f446c147bd0 0x7f446c144630 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 rx=0x7f445c00c000 tx=0x7f445c006df0).stop 2022-06-17T12:57:33.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.238+0000 7f446bfff700 1 -- 172.21.15.36:0/524358303 shutdown_connections 2022-06-17T12:57:33.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.238+0000 7f446bfff700 1 --2- 172.21.15.36:0/524358303 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4448053920 0x7f4448055de0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:33.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.238+0000 7f446bfff700 1 --2- 172.21.15.36:0/524358303 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f446c14b2b0 0x7f446c144b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:33.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.238+0000 7f446bfff700 1 --2- 172.21.15.36:0/524358303 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f446c147bd0 0x7f446c144630 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:33.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.238+0000 7f446bfff700 1 --2- 172.21.15.36:0/524358303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f446c0b27b0 0x7f446c0b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:33.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.238+0000 7f446bfff700 1 -- 172.21.15.36:0/524358303 >> 172.21.15.36:0/524358303 conn(0x7f446c0b5d10 msgr2=0x7f446c145be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:33.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.238+0000 7f446bfff700 1 -- 172.21.15.36:0/524358303 shutdown_connections 2022-06-17T12:57:33.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.238+0000 7f446bfff700 1 -- 172.21.15.36:0/524358303 wait complete. 2022-06-17T12:57:33.256 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-17T12:57:33.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.398+0000 7fd35d701700 1 Processor -- start 2022-06-17T12:57:33.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.398+0000 7fd35d701700 1 -- start start 2022-06-17T12:57:33.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd35d701700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3580b27b0 0x7fd35813f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:33.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd35d701700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3580b2b90 con 0x7fd3580b15f0 2022-06-17T12:57:33.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd35d701700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3580610e0 con 0x7fd3580b0d70 2022-06-17T12:57:33.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd35d701700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3580ac190 con 0x7fd3580b27b0 2022-06-17T12:57:33.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd35cf00700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fd3580b15f0 0x7fd35814a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58576/0 (socket says 172.21.15.36:58576) 2022-06-17T12:57:33.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd35cf00700 1 -- 172.21.15.36:0/4212213686 learned_addr learned my addr 172.21.15.36:0/4212213686 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T12:57:33.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd357fff700 1 --2- 172.21.15.36:0/4212213686 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3580b27b0 0x7fd35813f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:33.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd356ffd700 1 -- 172.21.15.36:0/4212213686 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1514277543 0 0) 0x7fd3580b2b90 con 0x7fd3580b15f0 2022-06-17T12:57:33.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd356ffd700 1 -- 172.21.15.36:0/4212213686 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd344003260 con 0x7fd3580b15f0 2022-06-17T12:57:33.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd356ffd700 1 -- 172.21.15.36:0/4212213686 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2591256917 0 0) 0x7fd3580610e0 con 0x7fd3580b0d70 2022-06-17T12:57:33.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd356ffd700 1 -- 172.21.15.36:0/4212213686 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3580b2b90 con 0x7fd3580b0d70 2022-06-17T12:57:33.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd356ffd700 1 -- 172.21.15.36:0/4212213686 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 421909797 0 0) 0x7fd344003260 con 0x7fd3580b15f0 2022-06-17T12:57:33.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd356ffd700 1 -- 172.21.15.36:0/4212213686 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3580b27b0 msgr2=0x7fd35813f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:33.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd356ffd700 1 --2- 172.21.15.36:0/4212213686 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3580b27b0 0x7fd35813f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:33.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd356ffd700 1 -- 172.21.15.36:0/4212213686 >> v1:172.21.15.70:6789/0 conn(0x7fd3580b0d70 legacy=0x7fd3580ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:33.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd356ffd700 1 -- 172.21.15.36:0/4212213686 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3580ac430 con 0x7fd3580b15f0 2022-06-17T12:57:33.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd356ffd700 1 -- 172.21.15.36:0/4212213686 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7fd3500019e0 con 0x7fd3580b15f0 2022-06-17T12:57:33.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd356ffd700 1 -- 172.21.15.36:0/4212213686 >> v1:172.21.15.36:6789/0 conn(0x7fd3580b15f0 legacy=0x7fd35814a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:33.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd356ffd700 1 --2- 172.21.15.36:0/4212213686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd344003910 0x7fd344003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:33.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd356ffd700 1 --2- 172.21.15.36:0/4212213686 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd344004500 0x7fd344004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:33.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd356ffd700 1 --2- 172.21.15.36:0/4212213686 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3580b27b0 0x7fd344004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:33.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd356ffd700 1 -- 172.21.15.36:0/4212213686 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3580ac190 con 0x7fd344004500 2022-06-17T12:57:33.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd356ffd700 1 -- 172.21.15.36:0/4212213686 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34400b780 con 0x7fd344003910 2022-06-17T12:57:33.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd356ffd700 1 -- 172.21.15.36:0/4212213686 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd34400b940 con 0x7fd3580b27b0 2022-06-17T12:57:33.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd3577fe700 1 --2- 172.21.15.36:0/4212213686 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd344004500 0x7fd344004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:33.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd357fff700 1 --2- 172.21.15.36:0/4212213686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd344003910 0x7fd344003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:33.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd35cf00700 1 --2- 172.21.15.36:0/4212213686 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3580b27b0 0x7fd344004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:33.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd3577fe700 1 -- 172.21.15.36:0/4212213686 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3580b27b0 msgr2=0x7fd344004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:33.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd3577fe700 1 --2- 172.21.15.36:0/4212213686 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3580b27b0 0x7fd344004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:33.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd3577fe700 1 -- 172.21.15.36:0/4212213686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd344003910 msgr2=0x7fd344003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:33.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd3577fe700 1 --2- 172.21.15.36:0/4212213686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd344003910 0x7fd344003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:33.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd3577fe700 1 -- 172.21.15.36:0/4212213686 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd344003260 con 0x7fd344004500 2022-06-17T12:57:33.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd3577fe700 1 --2- 172.21.15.36:0/4212213686 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd344004500 0x7fd344004980 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7fd348001110 tx=0x7fd348007a00).ready entity=mon.0 client_cookie=20a1f5fccfb1f542 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:33.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd356ffd700 1 -- 172.21.15.36:0/4212213686 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd348007d10 con 0x7fd344004500 2022-06-17T12:57:33.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd356ffd700 1 -- 172.21.15.36:0/4212213686 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd348007030 con 0x7fd344004500 2022-06-17T12:57:33.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.402+0000 7fd356ffd700 1 -- 172.21.15.36:0/4212213686 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd348007390 con 0x7fd344004500 2022-06-17T12:57:33.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.406+0000 7fd35d701700 1 -- 172.21.15.36:0/4212213686 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd344004500 msgr2=0x7fd344004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:33.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.406+0000 7fd35d701700 1 --2- 172.21.15.36:0/4212213686 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd344004500 0x7fd344004980 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7fd348001110 tx=0x7fd348007a00).stop 2022-06-17T12:57:33.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.406+0000 7fd35cf00700 1 -- 172.21.15.36:0/4212213686 reap_dead start 2022-06-17T12:57:33.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.406+0000 7fd35d701700 1 -- 172.21.15.36:0/4212213686 shutdown_connections 2022-06-17T12:57:33.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.406+0000 7fd35d701700 1 -- 172.21.15.36:0/4212213686 >> 172.21.15.36:0/4212213686 conn(0x7fd3580b5d10 msgr2=0x7fd3580b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:33.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.406+0000 7fd35d701700 1 -- 172.21.15.36:0/4212213686 shutdown_connections 2022-06-17T12:57:33.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.406+0000 7fd35d701700 1 -- 172.21.15.36:0/4212213686 wait complete. 2022-06-17T12:57:33.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.406+0000 7fd35d701700 1 Processor -- start 2022-06-17T12:57:33.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.406+0000 7fd35d701700 1 -- start start 2022-06-17T12:57:33.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.406+0000 7fd35d701700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd344004500 0x7fd358066b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:33.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.406+0000 7fd35d701700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd35813dd00 0x7fd3581418b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:33.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.406+0000 7fd35d701700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd358141da0 0x7fd35813b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:33.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.406+0000 7fd35d701700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35813e580 con 0x7fd344004500 2022-06-17T12:57:33.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.406+0000 7fd35d701700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd358141100 con 0x7fd358141da0 2022-06-17T12:57:33.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.406+0000 7fd35d701700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3581377d0 con 0x7fd35813dd00 2022-06-17T12:57:33.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.406+0000 7fd357fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd344004500 0x7fd358066b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:33.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.406+0000 7fd357fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd344004500 0x7fd358066b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41054/0 (socket says 172.21.15.36:41054) 2022-06-17T12:57:33.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.406+0000 7fd357fff700 1 -- 172.21.15.36:0/2356954689 learned_addr learned my addr 172.21.15.36:0/2356954689 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:33.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.406+0000 7fd3577fe700 1 --2- 172.21.15.36:0/2356954689 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd35813dd00 0x7fd3581418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:33.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.406+0000 7fd35cf00700 1 --2- 172.21.15.36:0/2356954689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd358141da0 0x7fd35813b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:33.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.410+0000 7fd357fff700 1 -- 172.21.15.36:0/2356954689 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd35813dd00 msgr2=0x7fd3581418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:33.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.410+0000 7fd357fff700 1 --2- 172.21.15.36:0/2356954689 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd35813dd00 0x7fd3581418b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:33.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.410+0000 7fd357fff700 1 -- 172.21.15.36:0/2356954689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd358141da0 msgr2=0x7fd35813b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:33.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.410+0000 7fd357fff700 1 --2- 172.21.15.36:0/2356954689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd358141da0 0x7fd35813b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:33.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.410+0000 7fd357fff700 1 -- 172.21.15.36:0/2356954689 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35813e770 con 0x7fd344004500 2022-06-17T12:57:33.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.410+0000 7fd3577fe700 1 --2- 172.21.15.36:0/2356954689 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd35813dd00 0x7fd3581418b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:57:33.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.410+0000 7fd35cf00700 1 --2- 172.21.15.36:0/2356954689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd358141da0 0x7fd35813b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T12:57:33.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.410+0000 7fd357fff700 1 --2- 172.21.15.36:0/2356954689 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd344004500 0x7fd358066b10 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 rx=0x7fd34c00b000 tx=0x7fd34c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:33.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.410+0000 7fd3557fa700 1 -- 172.21.15.36:0/2356954689 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd34c00f940 con 0x7fd344004500 2022-06-17T12:57:33.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.410+0000 7fd3557fa700 1 -- 172.21.15.36:0/2356954689 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34c00faa0 con 0x7fd344004500 2022-06-17T12:57:33.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.410+0000 7fd35d701700 1 -- 172.21.15.36:0/2356954689 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd358141330 con 0x7fd344004500 2022-06-17T12:57:33.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.410+0000 7fd3557fa700 1 -- 172.21.15.36:0/2356954689 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd34c00fe00 con 0x7fd344004500 2022-06-17T12:57:33.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.410+0000 7fd35d701700 1 -- 172.21.15.36:0/2356954689 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35813d940 con 0x7fd344004500 2022-06-17T12:57:33.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.410+0000 7fd35d701700 1 -- 172.21.15.36:0/2356954689 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd35813e900 con 0x7fd344004500 2022-06-17T12:57:33.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.410+0000 7fd3557fa700 1 -- 172.21.15.36:0/2356954689 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fd34c014440 con 0x7fd344004500 2022-06-17T12:57:33.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.410+0000 7fd3557fa700 1 --2- 172.21.15.36:0/2356954689 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd3380539b0 0x7fd338055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:33.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.410+0000 7fd3557fa700 1 -- 172.21.15.36:0/2356954689 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fd34c06c9b0 con 0x7fd344004500 2022-06-17T12:57:33.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.410+0000 7fd3577fe700 1 --2- 172.21.15.36:0/2356954689 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd3380539b0 0x7fd338055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:33.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.410+0000 7fd3577fe700 1 --2- 172.21.15.36:0/2356954689 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd3380539b0 0x7fd338055e70 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fd348018000 tx=0x7fd348016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:33.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.418+0000 7fd3557fa700 1 -- 172.21.15.36:0/2356954689 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd34c043700 con 0x7fd344004500 2022-06-17T12:57:33.534 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:33.546+0000 7fd35d701700 1 -- 172.21.15.36:0/2356954689 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd3581414c0 con 0x7fd344004500 2022-06-17T12:57:34.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.238+0000 7fd3557fa700 1 -- 172.21.15.36:0/2356954689 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v153) v1 ==== 185+0+0 (secure 0 0 0) 0x7fd34c018020 con 0x7fd344004500 2022-06-17T12:57:34.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.242+0000 7fd35d701700 1 -- 172.21.15.36:0/2356954689 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd3380539b0 msgr2=0x7fd338055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:34.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.242+0000 7fd35d701700 1 --2- 172.21.15.36:0/2356954689 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd3380539b0 0x7fd338055e70 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fd348018000 tx=0x7fd348016000).stop 2022-06-17T12:57:34.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.242+0000 7fd35d701700 1 -- 172.21.15.36:0/2356954689 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd344004500 msgr2=0x7fd358066b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:34.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.242+0000 7fd35d701700 1 --2- 172.21.15.36:0/2356954689 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd344004500 0x7fd358066b10 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 rx=0x7fd34c00b000 tx=0x7fd34c006df0).stop 2022-06-17T12:57:34.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.242+0000 7fd35d701700 1 -- 172.21.15.36:0/2356954689 shutdown_connections 2022-06-17T12:57:34.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.242+0000 7fd35d701700 1 --2- 172.21.15.36:0/2356954689 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fd3380539b0 0x7fd338055e70 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:34.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.242+0000 7fd35d701700 1 --2- 172.21.15.36:0/2356954689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd358141da0 0x7fd35813b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:34.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.242+0000 7fd35d701700 1 --2- 172.21.15.36:0/2356954689 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd35813dd00 0x7fd3581418b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:34.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.242+0000 7fd35d701700 1 --2- 172.21.15.36:0/2356954689 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fd344004500 0x7fd358066b10 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:34.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.242+0000 7fd35d701700 1 -- 172.21.15.36:0/2356954689 >> 172.21.15.36:0/2356954689 conn(0x7fd3580b5d10 msgr2=0x7fd3580afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:34.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.242+0000 7fd35d701700 1 -- 172.21.15.36:0/2356954689 shutdown_connections 2022-06-17T12:57:34.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.242+0000 7fd35d701700 1 -- 172.21.15.36:0/2356954689 wait complete. 2022-06-17T12:57:34.244 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' removed 2022-06-17T12:57:34.244 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-17T12:57:34.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.382+0000 7f0e5e744700 1 Processor -- start 2022-06-17T12:57:34.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.382+0000 7f0e5e744700 1 -- start start 2022-06-17T12:57:34.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.382+0000 7f0e5e744700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e580b6780 0x7f0e5805f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:34.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.382+0000 7f0e5e744700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0e58060370 0x7f0e5813c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:34.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e5e744700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e58146ff0 con 0x7f0e58060c40 2022-06-17T12:57:34.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e5e744700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e58147420 con 0x7f0e58060370 2022-06-17T12:57:34.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e5e744700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e580b6b60 con 0x7f0e580b6780 2022-06-17T12:57:34.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e5cce1700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0e58060370 0x7f0e5813c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:34.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e5cce1700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0e58060370 0x7f0e5813c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41066/0 (socket says 172.21.15.36:41066) 2022-06-17T12:57:34.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e5cce1700 1 -- 172.21.15.36:0/3059793419 learned_addr learned my addr 172.21.15.36:0/3059793419 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:34.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e57fff700 1 --2- 172.21.15.36:0/3059793419 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e580b6780 0x7f0e5805f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:34.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e5cce1700 1 -- 172.21.15.36:0/3059793419 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e580b6780 msgr2=0x7f0e5805f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T12:57:34.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e5cce1700 1 --2- 172.21.15.36:0/3059793419 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e580b6780 0x7f0e5805f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:34.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e5cce1700 1 -- 172.21.15.36:0/3059793419 >> v1:172.21.15.70:6789/0 conn(0x7f0e58060c40 legacy=0x7f0e5805fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:34.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e5cce1700 1 -- 172.21.15.36:0/3059793419 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e580afd40 con 0x7f0e58060370 2022-06-17T12:57:34.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e5cce1700 1 --2- 172.21.15.36:0/3059793419 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e58060370 0x7f0e5813c5f0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 rx=0x7f0e48002f70 tx=0x7f0e48006eb0).ready entity=mon.0 client_cookie=6be356ae0513cf35 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:34.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e56ffd700 1 -- 172.21.15.36:0/3059793419 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e4800d5c0 con 0x7f0e58060370 2022-06-17T12:57:34.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e56ffd700 1 -- 172.21.15.36:0/3059793419 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e48000e90 con 0x7f0e58060370 2022-06-17T12:57:34.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e56ffd700 1 -- 172.21.15.36:0/3059793419 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e4800d950 con 0x7f0e58060370 2022-06-17T12:57:34.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e5e744700 1 -- 172.21.15.36:0/3059793419 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e58060370 msgr2=0x7f0e5813c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:34.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e5e744700 1 --2- 172.21.15.36:0/3059793419 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e58060370 0x7f0e5813c5f0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 rx=0x7f0e48002f70 tx=0x7f0e48006eb0).stop 2022-06-17T12:57:34.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e5e744700 1 -- 172.21.15.36:0/3059793419 shutdown_connections 2022-06-17T12:57:34.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e5e744700 1 --2- 172.21.15.36:0/3059793419 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e58060370 0x7f0e5813c5f0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:34.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e5e744700 1 --2- 172.21.15.36:0/3059793419 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e580b6780 0x7f0e5805f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:34.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e5e744700 1 -- 172.21.15.36:0/3059793419 >> 172.21.15.36:0/3059793419 conn(0x7f0e58063890 msgr2=0x7f0e580b5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:34.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e5e744700 1 -- 172.21.15.36:0/3059793419 shutdown_connections 2022-06-17T12:57:34.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e5e744700 1 -- 172.21.15.36:0/3059793419 wait complete. 2022-06-17T12:57:34.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e5e744700 1 Processor -- start 2022-06-17T12:57:34.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.386+0000 7f0e5e744700 1 -- start start 2022-06-17T12:57:34.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.390+0000 7f0e5e744700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e58060370 0x7f0e5813e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:34.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.390+0000 7f0e5e744700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e58060c40 0x7f0e5813ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:34.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.390+0000 7f0e5e744700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e580b6780 0x7f0e58143670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:34.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.390+0000 7f0e5e744700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e581431d0 con 0x7f0e580b6780 2022-06-17T12:57:34.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.390+0000 7f0e5e744700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e58065b80 con 0x7f0e58060370 2022-06-17T12:57:34.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.390+0000 7f0e5e744700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e58142f30 con 0x7f0e58060c40 2022-06-17T12:57:34.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.390+0000 7f0e5cce1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e580b6780 0x7f0e58143670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:34.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.390+0000 7f0e57fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e58060370 0x7f0e5813e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:34.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.390+0000 7f0e577fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e58060c40 0x7f0e5813ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:34.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.390+0000 7f0e5cce1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e580b6780 0x7f0e58143670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41072/0 (socket says 172.21.15.36:41072) 2022-06-17T12:57:34.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.390+0000 7f0e5cce1700 1 -- 172.21.15.36:0/3360583424 learned_addr learned my addr 172.21.15.36:0/3360583424 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T12:57:34.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.390+0000 7f0e5cce1700 1 -- 172.21.15.36:0/3360583424 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e58060c40 msgr2=0x7f0e5813ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:34.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.390+0000 7f0e5cce1700 1 --2- 172.21.15.36:0/3360583424 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e58060c40 0x7f0e5813ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:34.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.390+0000 7f0e5cce1700 1 -- 172.21.15.36:0/3360583424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e58060370 msgr2=0x7f0e5813e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:34.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.390+0000 7f0e5cce1700 1 --2- 172.21.15.36:0/3360583424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e58060370 0x7f0e5813e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:34.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.390+0000 7f0e5cce1700 1 -- 172.21.15.36:0/3360583424 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4800b1e0 con 0x7f0e580b6780 2022-06-17T12:57:34.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.390+0000 7f0e5cce1700 1 --2- 172.21.15.36:0/3360583424 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e580b6780 0x7f0e58143670 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 rx=0x7f0e48002f70 tx=0x7f0e48015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:34.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.390+0000 7f0e557fa700 1 -- 172.21.15.36:0/3360583424 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e4800d940 con 0x7f0e580b6780 2022-06-17T12:57:34.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.390+0000 7f0e5e744700 1 -- 172.21.15.36:0/3360583424 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e58143410 con 0x7f0e580b6780 2022-06-17T12:57:34.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.390+0000 7f0e557fa700 1 -- 172.21.15.36:0/3360583424 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e48011b40 con 0x7f0e580b6780 2022-06-17T12:57:34.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.390+0000 7f0e557fa700 1 -- 172.21.15.36:0/3360583424 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e48011ea0 con 0x7f0e580b6780 2022-06-17T12:57:34.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.390+0000 7f0e5e744700 1 -- 172.21.15.36:0/3360583424 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e58067b00 con 0x7f0e580b6780 2022-06-17T12:57:34.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.390+0000 7f0e5e744700 1 -- 172.21.15.36:0/3360583424 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e58067de0 con 0x7f0e580b6780 2022-06-17T12:57:34.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.394+0000 7f0e557fa700 1 -- 172.21.15.36:0/3360583424 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f0e4801a040 con 0x7f0e580b6780 2022-06-17T12:57:34.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.394+0000 7f0e557fa700 1 --2- 172.21.15.36:0/3360583424 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0e3c053a40 0x7f0e3c055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T12:57:34.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.394+0000 7f0e557fa700 1 -- 172.21.15.36:0/3360583424 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f0e4806cdb0 con 0x7f0e580b6780 2022-06-17T12:57:34.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.394+0000 7f0e57fff700 1 --2- 172.21.15.36:0/3360583424 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0e3c053a40 0x7f0e3c055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T12:57:34.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.394+0000 7f0e57fff700 1 --2- 172.21.15.36:0/3360583424 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0e3c053a40 0x7f0e3c055f00 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7f0e5000c000 tx=0x7f0e50008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T12:57:34.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.402+0000 7f0e557fa700 1 -- 172.21.15.36:0/3360583424 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0e58067de0 con 0x7f0e580b6780 2022-06-17T12:57:34.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:34.526+0000 7f0e5e744700 1 -- 172.21.15.36:0/3360583424 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0e58067de0 con 0x7f0e580b6780 2022-06-17T12:57:35.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:35.250+0000 7f0e557fa700 1 -- 172.21.15.36:0/3360583424 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v154) v1 ==== 173+0+0 (secure 0 0 0) 0x7f0e58067de0 con 0x7f0e580b6780 2022-06-17T12:57:35.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:35.254+0000 7f0e5e744700 1 -- 172.21.15.36:0/3360583424 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0e3c053a40 msgr2=0x7f0e3c055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:35.248 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:35.254+0000 7f0e5e744700 1 --2- 172.21.15.36:0/3360583424 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0e3c053a40 0x7f0e3c055f00 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7f0e5000c000 tx=0x7f0e50008040).stop 2022-06-17T12:57:35.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:35.254+0000 7f0e5e744700 1 -- 172.21.15.36:0/3360583424 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e580b6780 msgr2=0x7f0e58143670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T12:57:35.249 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:35.254+0000 7f0e5e744700 1 --2- 172.21.15.36:0/3360583424 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e580b6780 0x7f0e58143670 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 rx=0x7f0e48002f70 tx=0x7f0e48015000).stop 2022-06-17T12:57:35.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:35.254+0000 7f0e5e744700 1 -- 172.21.15.36:0/3360583424 shutdown_connections 2022-06-17T12:57:35.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:35.254+0000 7f0e5e744700 1 --2- 172.21.15.36:0/3360583424 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0e3c053a40 0x7f0e3c055f00 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:35.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:35.254+0000 7f0e5e744700 1 --2- 172.21.15.36:0/3360583424 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e580b6780 0x7f0e58143670 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:35.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:35.254+0000 7f0e5e744700 1 --2- 172.21.15.36:0/3360583424 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e58060c40 0x7f0e5813ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:35.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:35.254+0000 7f0e5e744700 1 --2- 172.21.15.36:0/3360583424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e58060370 0x7f0e5813e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T12:57:35.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:35.254+0000 7f0e5e744700 1 -- 172.21.15.36:0/3360583424 >> 172.21.15.36:0/3360583424 conn(0x7f0e58063890 msgr2=0x7f0e580b4920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T12:57:35.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:35.254+0000 7f0e5e744700 1 -- 172.21.15.36:0/3360583424 shutdown_connections 2022-06-17T12:57:35.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T12:57:35.254+0000 7f0e5e744700 1 -- 172.21.15.36:0/3360583424 wait complete. 2022-06-17T12:57:35.253 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' removed 2022-06-17T12:57:35.257 INFO:teuthology.orchestra.run:Running command with timeout 60 2022-06-17T12:57:35.258 DEBUG:teuthology.orchestra.run.smithi107:> cd /home/ubuntu/cephtest/mnt.0 && sudo bash -c 'stat -f .' 2022-06-17T12:57:46.794 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:57:46.804 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:57:46.809 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:57:46.818 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:57:46.824 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:58:05.378 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T12:58:05.379 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T12:58:05.380 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T12:58:17.211 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:58:17.216 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:58:17.253 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:58:17.263 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:58:17.271 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:58:35.281 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-06-17T12:58:35.312 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2022-06-17T12:58:47.324 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:58:47.331 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:58:47.335 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:58:47.339 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:58:47.345 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:59:17.531 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:59:17.551 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:59:17.565 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:59:17.584 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:59:17.593 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:59:47.699 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:59:47.704 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:59:47.708 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:59:47.713 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T12:59:47.716 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:00:17.779 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:00:17.837 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:00:17.874 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:00:17.880 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:00:17.898 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:00:47.950 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:00:47.956 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:00:47.962 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:00:47.968 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:00:47.976 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:01:18.051 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:01:18.063 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:01:18.072 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:01:18.077 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:01:18.080 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:01:48.338 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:01:48.347 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:01:48.380 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:01:48.385 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:01:48.394 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:02:18.490 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:02:18.508 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:02:18.513 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:02:18.520 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:02:18.525 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:02:48.592 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:02:48.627 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:02:48.635 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:02:48.642 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:02:48.646 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:03:18.737 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:03:18.742 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:03:18.750 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:03:18.761 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:03:18.775 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:03:49.069 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:03:49.081 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:03:49.107 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:03:49.131 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:03:49.148 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:04:19.248 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:04:19.542 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:04:19.555 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:04:19.565 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:04:19.574 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:04:49.641 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:04:49.652 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:04:49.655 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:04:49.660 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:04:49.723 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:05:19.818 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:05:19.829 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:05:19.851 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:05:19.857 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:05:19.862 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:05:49.974 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:05:50.017 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:05:50.042 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:05:50.079 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:05:50.123 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:06:20.176 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:06:20.206 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:06:20.233 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:06:20.246 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:06:20.255 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:06:50.352 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:06:50.380 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:06:50.412 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:06:50.427 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:06:50.449 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:07:20.542 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:07:20.549 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:07:20.553 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:07:20.557 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:07:20.561 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:07:50.636 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:07:50.641 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:07:50.645 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:07:50.648 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:07:50.651 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:08:20.763 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:08:20.767 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:08:20.772 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:08:20.776 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:08:20.796 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:08:50.877 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:08:50.896 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:08:50.900 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:08:50.906 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:08:50.910 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:09:20.989 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:09:20.993 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:09:20.997 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:09:21.023 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:09:21.066 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:09:51.290 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:09:51.295 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:09:51.299 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:09:51.303 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:09:51.307 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:10:21.387 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:10:21.392 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:10:21.396 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:10:21.421 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:10:21.437 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:10:51.725 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:10:51.729 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:10:51.733 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:10:51.736 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:10:51.739 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:11:21.808 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:11:21.843 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:11:21.848 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:11:21.854 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:11:21.862 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:11:51.979 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:11:51.984 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:11:51.988 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:11:51.992 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:11:51.996 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:12:22.125 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:12:22.129 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:12:22.133 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:12:22.137 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:12:22.140 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:12:52.225 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:12:52.228 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:12:52.231 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:12:52.234 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:12:52.236 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:13:05.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T13:13:05.388 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2022-06-17T13:13:05.390 INFO:teuthology.orchestra.run.smithi107.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Connection timed out 2022-06-17T13:13:05.393 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-17T13:13:05.394 DEBUG:teuthology.orchestra.run.smithi036:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T13:13:05.415 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T13:13:05.437 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T13:13:05.461 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T13:13:05.483 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-17T13:13:05.506 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-17T13:13:05.507 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T13:13:05.508 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T13:13:05.508 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T13:13:22.361 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:13:22.364 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:13:22.367 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:13:22.370 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:13:22.372 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:13:52.532 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:13:52.536 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:13:52.540 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:13:52.543 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:13:52.546 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:14:22.803 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:14:22.807 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:14:22.810 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:14:22.813 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:14:22.816 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:14:52.901 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:14:52.904 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:14:52.907 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:14:52.910 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:14:52.914 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:15:23.089 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:15:23.094 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:15:23.097 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:15:23.100 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:15:23.104 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:15:53.221 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:15:53.225 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:15:53.228 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:15:53.231 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:15:53.235 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:16:23.373 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:16:23.377 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:16:23.380 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:16:23.383 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:16:23.387 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:16:53.496 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:16:53.500 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:16:53.506 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:16:53.509 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:16:53.512 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:17:23.646 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:17:23.650 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:17:23.654 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:17:23.657 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:17:23.662 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:17:53.789 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:17:53.793 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:17:53.796 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:17:53.799 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:17:53.801 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:18:23.925 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:18:23.928 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:18:23.932 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:18:23.934 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:18:23.937 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:18:54.035 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:18:54.040 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:18:54.044 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:18:54.047 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:18:54.049 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:19:24.196 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:19:24.201 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:19:24.204 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:19:24.208 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:19:24.212 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:19:54.332 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:19:54.337 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:19:54.340 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:19:54.346 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:19:54.349 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:20:24.491 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:20:24.502 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:20:24.506 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:20:24.510 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:20:24.516 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:20:54.650 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:20:54.657 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:20:54.674 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:20:54.688 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:20:54.707 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:21:24.838 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:21:24.845 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:21:24.855 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:21:24.857 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:21:24.860 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:21:55.009 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:21:55.012 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:21:55.018 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:21:55.021 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:21:55.025 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:22:25.143 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:22:25.152 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:22:25.162 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:22:25.166 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:22:25.170 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:22:55.285 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:22:55.290 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:22:55.294 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:22:55.298 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:22:55.300 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:23:25.418 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:23:25.421 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:23:25.424 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:23:25.427 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:23:25.430 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:23:55.753 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:23:55.757 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:23:55.767 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:23:55.775 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:23:55.784 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:24:25.931 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:24:25.938 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:24:25.944 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:24:25.948 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:24:25.953 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:24:56.062 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:24:56.065 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:24:56.069 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:24:56.072 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:24:56.075 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:25:26.189 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:25:26.196 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:25:26.198 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:25:26.201 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:25:26.215 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:25:56.334 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:25:56.359 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:25:56.384 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:25:56.389 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:25:56.398 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:26:26.528 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:26:26.535 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:26:26.563 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:26:26.581 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:26:26.585 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:26:56.713 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:26:56.784 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:26:56.826 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:26:56.835 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:26:56.848 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:27:27.075 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:27:27.085 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:27:27.090 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:27:27.094 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:27:27.098 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:27:57.214 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:27:57.220 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:27:57.226 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:27:57.230 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:27:57.238 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:28:05.515 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-06-17T13:28:05.517 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2022-06-17T13:28:05.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T13:28:05.523 INFO:tasks.cephfs_test_runner:test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) ... ERROR 2022-06-17T13:28:05.524 INFO:tasks.cephfs_test_runner:ERROR 2022-06-17T13:28:05.525 INFO:tasks.cephfs_test_runner: 2022-06-17T13:28:05.527 INFO:tasks.cephfs_test_runner:====================================================================== 2022-06-17T13:28:05.527 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-06-17T13:28:05.528 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-06-17T13:28:05.528 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-06-17T13:28:05.529 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv 2022-06-17T13:28:05.530 INFO:tasks.cephfs_test_runner: out = self.in_buffer.read(nbytes, self.timeout) 2022-06-17T13:28:05.531 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2022-06-17T13:28:05.531 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2022-06-17T13:28:05.532 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2022-06-17T13:28:05.532 INFO:tasks.cephfs_test_runner: 2022-06-17T13:28:05.532 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2022-06-17T13:28:05.533 INFO:tasks.cephfs_test_runner: 2022-06-17T13:28:05.534 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-06-17T13:28:05.534 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/cephfs/test_misc.py", line 28, in test_statfs_on_deleted_fs 2022-06-17T13:28:05.535 INFO:tasks.cephfs_test_runner: self.mount_a.umount_wait(force=True) 2022-06-17T13:28:05.535 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait 2022-06-17T13:28:05.536 INFO:tasks.cephfs_test_runner: if not (self.is_mounted() and self.fuse_daemon): 2022-06-17T13:28:05.536 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2022-06-17T13:28:05.537 INFO:tasks.cephfs_test_runner: proc.wait() 2022-06-17T13:28:05.538 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 148, in wait 2022-06-17T13:28:05.539 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2022-06-17T13:28:05.539 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2022-06-17T13:28:05.540 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2022-06-17T13:28:05.542 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2022-06-17T13:28:05.543 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2022-06-17T13:28:05.543 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2022-06-17T13:28:05.544 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 323, in copy_file_to 2022-06-17T13:28:05.544 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2022-06-17T13:28:05.545 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 276, in copy_to_log 2022-06-17T13:28:05.545 INFO:tasks.cephfs_test_runner: for line in f: 2022-06-17T13:28:05.546 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2022-06-17T13:28:05.546 INFO:tasks.cephfs_test_runner: line = self.readline() 2022-06-17T13:28:05.547 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2022-06-17T13:28:05.548 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2022-06-17T13:28:05.548 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read 2022-06-17T13:28:05.549 INFO:tasks.cephfs_test_runner: return self.channel.recv(size) 2022-06-17T13:28:05.550 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv 2022-06-17T13:28:05.551 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2022-06-17T13:28:05.551 INFO:tasks.cephfs_test_runner:socket.timeout 2022-06-17T13:28:05.552 INFO:tasks.cephfs_test_runner: 2022-06-17T13:28:05.552 INFO:tasks.cephfs_test_runner:====================================================================== 2022-06-17T13:28:05.553 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-06-17T13:28:05.554 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-06-17T13:28:05.554 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-06-17T13:28:05.555 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2022-06-17T13:28:05.555 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2022-06-17T13:28:05.556 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2022-06-17T13:28:05.556 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2022-06-17T13:28:05.557 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2022-06-17T13:28:05.558 INFO:tasks.cephfs_test_runner: 2022-06-17T13:28:05.559 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2022-06-17T13:28:05.563 INFO:tasks.cephfs_test_runner: 2022-06-17T13:28:05.564 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-06-17T13:28:05.564 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/cephfs/cephfs_test_case.py", line 175, in tearDown 2022-06-17T13:28:05.565 INFO:tasks.cephfs_test_runner: m.teardown() 2022-06-17T13:28:05.566 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/cephfs/fuse_mount.py", line 408, in teardown 2022-06-17T13:28:05.566 INFO:tasks.cephfs_test_runner: self.umount() 2022-06-17T13:28:05.567 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/cephfs/fuse_mount.py", line 261, in umount 2022-06-17T13:28:05.567 INFO:tasks.cephfs_test_runner: if not self.is_mounted(): 2022-06-17T13:28:05.568 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2022-06-17T13:28:05.570 INFO:tasks.cephfs_test_runner: proc.wait() 2022-06-17T13:28:05.570 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 148, in wait 2022-06-17T13:28:05.571 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2022-06-17T13:28:05.571 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2022-06-17T13:28:05.571 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2022-06-17T13:28:05.572 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2022-06-17T13:28:05.572 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2022-06-17T13:28:05.572 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2022-06-17T13:28:05.573 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 323, in copy_file_to 2022-06-17T13:28:05.573 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2022-06-17T13:28:05.573 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 276, in copy_to_log 2022-06-17T13:28:05.574 INFO:tasks.cephfs_test_runner: for line in f: 2022-06-17T13:28:05.574 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2022-06-17T13:28:05.574 INFO:tasks.cephfs_test_runner: line = self.readline() 2022-06-17T13:28:05.575 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2022-06-17T13:28:05.575 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2022-06-17T13:28:05.575 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2022-06-17T13:28:05.576 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2022-06-17T13:28:05.576 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2022-06-17T13:28:05.576 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2022-06-17T13:28:05.577 INFO:tasks.cephfs_test_runner:socket.timeout 2022-06-17T13:28:05.577 INFO:tasks.cephfs_test_runner: 2022-06-17T13:28:05.577 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-06-17T13:28:05.578 INFO:tasks.cephfs_test_runner:Ran 12 tests in 2436.703s 2022-06-17T13:28:05.578 INFO:tasks.cephfs_test_runner: 2022-06-17T13:28:05.578 INFO:tasks.cephfs_test_runner:FAILED (errors=2) 2022-06-17T13:28:05.579 INFO:tasks.cephfs_test_runner: 2022-06-17T13:28:05.579 INFO:tasks.cephfs_test_runner:====================================================================== 2022-06-17T13:28:05.579 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-06-17T13:28:05.580 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-06-17T13:28:05.580 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-06-17T13:28:05.580 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv 2022-06-17T13:28:05.581 INFO:tasks.cephfs_test_runner: out = self.in_buffer.read(nbytes, self.timeout) 2022-06-17T13:28:05.581 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2022-06-17T13:28:05.581 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2022-06-17T13:28:05.582 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2022-06-17T13:28:05.582 INFO:tasks.cephfs_test_runner: 2022-06-17T13:28:05.582 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2022-06-17T13:28:05.583 INFO:tasks.cephfs_test_runner: 2022-06-17T13:28:05.583 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-06-17T13:28:05.583 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/cephfs/test_misc.py", line 28, in test_statfs_on_deleted_fs 2022-06-17T13:28:05.584 INFO:tasks.cephfs_test_runner: self.mount_a.umount_wait(force=True) 2022-06-17T13:28:05.584 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait 2022-06-17T13:28:05.584 INFO:tasks.cephfs_test_runner: if not (self.is_mounted() and self.fuse_daemon): 2022-06-17T13:28:05.585 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2022-06-17T13:28:05.585 INFO:tasks.cephfs_test_runner: proc.wait() 2022-06-17T13:28:05.585 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 148, in wait 2022-06-17T13:28:05.585 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2022-06-17T13:28:05.586 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2022-06-17T13:28:05.586 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2022-06-17T13:28:05.586 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2022-06-17T13:28:05.587 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2022-06-17T13:28:05.587 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2022-06-17T13:28:05.587 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 323, in copy_file_to 2022-06-17T13:28:05.588 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2022-06-17T13:28:05.588 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 276, in copy_to_log 2022-06-17T13:28:05.588 INFO:tasks.cephfs_test_runner: for line in f: 2022-06-17T13:28:05.588 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2022-06-17T13:28:05.589 INFO:tasks.cephfs_test_runner: line = self.readline() 2022-06-17T13:28:05.589 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2022-06-17T13:28:05.589 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2022-06-17T13:28:05.590 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read 2022-06-17T13:28:05.590 INFO:tasks.cephfs_test_runner: return self.channel.recv(size) 2022-06-17T13:28:05.590 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv 2022-06-17T13:28:05.591 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2022-06-17T13:28:05.591 INFO:tasks.cephfs_test_runner:socket.timeout 2022-06-17T13:28:05.591 INFO:tasks.cephfs_test_runner: 2022-06-17T13:28:05.592 INFO:tasks.cephfs_test_runner:====================================================================== 2022-06-17T13:28:05.592 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-06-17T13:28:05.593 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-06-17T13:28:05.593 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-06-17T13:28:05.593 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2022-06-17T13:28:05.594 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2022-06-17T13:28:05.594 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2022-06-17T13:28:05.594 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2022-06-17T13:28:05.595 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2022-06-17T13:28:05.595 INFO:tasks.cephfs_test_runner: 2022-06-17T13:28:05.595 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2022-06-17T13:28:05.596 INFO:tasks.cephfs_test_runner: 2022-06-17T13:28:05.596 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-06-17T13:28:05.596 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/cephfs/cephfs_test_case.py", line 175, in tearDown 2022-06-17T13:28:05.596 INFO:tasks.cephfs_test_runner: m.teardown() 2022-06-17T13:28:05.597 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/cephfs/fuse_mount.py", line 408, in teardown 2022-06-17T13:28:05.597 INFO:tasks.cephfs_test_runner: self.umount() 2022-06-17T13:28:05.597 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/cephfs/fuse_mount.py", line 261, in umount 2022-06-17T13:28:05.598 INFO:tasks.cephfs_test_runner: if not self.is_mounted(): 2022-06-17T13:28:05.598 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2022-06-17T13:28:05.598 INFO:tasks.cephfs_test_runner: proc.wait() 2022-06-17T13:28:05.599 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 148, in wait 2022-06-17T13:28:05.599 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2022-06-17T13:28:05.599 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2022-06-17T13:28:05.599 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2022-06-17T13:28:05.600 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2022-06-17T13:28:05.600 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2022-06-17T13:28:05.600 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2022-06-17T13:28:05.601 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 323, in copy_file_to 2022-06-17T13:28:05.601 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2022-06-17T13:28:05.601 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 276, in copy_to_log 2022-06-17T13:28:05.602 INFO:tasks.cephfs_test_runner: for line in f: 2022-06-17T13:28:05.606 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2022-06-17T13:28:05.606 INFO:tasks.cephfs_test_runner: line = self.readline() 2022-06-17T13:28:05.607 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2022-06-17T13:28:05.607 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2022-06-17T13:28:05.607 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2022-06-17T13:28:05.607 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2022-06-17T13:28:05.608 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2022-06-17T13:28:05.608 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2022-06-17T13:28:05.608 INFO:tasks.cephfs_test_runner:socket.timeout 2022-06-17T13:28:05.609 INFO:tasks.cephfs_test_runner: 2022-06-17T13:28:05.609 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/run_tasks.py", line 106, in run_tasks manager.__enter__() File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc), test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-06-17T13:28:05.819 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=08bd1b4e45894cdabf781d51313eccc8 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/run_tasks.py", line 106, in run_tasks manager.__enter__() File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc), test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-06-17T13:28:05.823 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2022-06-17T13:28:05.837 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2022-06-17T13:28:05.872 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2022-06-17T13:28:05.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T13:28:05.875 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T13:28:05.875 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T13:28:27.351 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:28:27.354 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:28:27.363 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:28:27.369 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:28:27.380 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:28:57.494 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:28:57.502 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:28:57.507 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:28:57.511 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:28:57.516 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:29:27.653 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:29:27.658 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:29:27.663 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:29:27.673 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:29:27.678 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:29:57.815 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:29:57.819 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:29:57.831 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:29:57.846 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:29:57.850 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:30:27.975 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:30:27.979 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:30:27.982 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:30:27.986 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:30:28.000 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:30:58.112 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:30:58.123 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:30:58.127 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:30:58.131 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:30:58.134 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:31:28.362 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:31:28.371 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:31:28.381 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:31:28.384 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:31:28.391 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:31:58.518 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:31:58.524 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:31:58.530 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:31:58.535 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:31:58.538 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:32:28.646 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:32:28.652 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:32:28.656 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:32:28.659 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:32:28.662 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:32:58.779 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:32:58.784 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:32:58.788 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:32:58.792 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:32:58.796 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:33:28.902 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:33:28.906 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:33:28.909 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:33:28.911 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:33:28.914 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:33:59.054 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:33:59.057 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:33:59.060 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:33:59.063 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:33:59.067 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:34:29.354 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:34:29.358 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:34:29.361 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:34:29.364 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:34:29.367 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:34:59.475 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:34:59.479 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:34:59.482 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:34:59.484 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:34:59.487 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:35:29.610 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:35:29.614 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:35:29.617 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:35:29.620 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:35:29.623 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:35:59.747 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:35:59.751 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:35:59.754 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:35:59.756 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:35:59.759 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:36:30.123 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:36:30.127 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:36:30.130 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:36:30.133 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:36:30.136 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:37:00.252 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:37:00.255 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:37:00.258 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:37:00.261 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:37:00.263 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:37:30.382 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:37:30.386 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:37:30.389 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:37:30.391 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:37:30.394 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:38:00.508 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:38:00.512 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:38:00.515 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:38:00.518 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:38:00.521 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:38:30.894 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:38:30.899 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:38:30.902 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:38:30.905 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:38:30.907 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:39:01.020 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:39:01.024 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:39:01.027 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:39:01.029 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:39:01.032 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:39:31.318 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:39:31.323 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:39:31.326 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:39:31.329 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:39:31.332 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:40:01.434 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:40:01.438 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:40:01.441 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:40:01.443 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:40:01.446 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:40:31.636 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:40:31.639 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:40:31.642 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:40:31.646 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:40:31.649 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:41:01.772 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:41:01.776 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:41:01.780 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:41:01.783 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:41:01.787 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:41:32.090 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:41:32.094 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:41:32.097 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:41:32.100 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:41:32.103 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:42:02.227 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:42:02.231 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:42:02.235 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:42:02.237 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:42:02.241 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:42:32.623 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:42:32.627 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:42:32.629 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:42:32.632 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:42:32.635 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:43:02.739 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:43:02.744 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:43:02.748 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:43:02.751 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:43:02.753 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:43:05.925 INFO:teuthology.orchestra.run.smithi107.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Connection timed out 2022-06-17T13:43:05.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T13:43:05.929 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2022-06-17T13:43:05.929 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-17T13:43:05.930 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-17T13:43:05.930 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-17T13:43:32.858 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:43:32.861 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:43:32.864 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:43:32.867 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:43:32.870 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:44:02.994 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:44:02.998 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:44:03.001 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:44:03.004 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:44:03.007 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:44:33.163 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:44:33.167 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:44:33.170 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:44:33.172 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:44:33.175 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:45:03.501 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:45:03.505 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:45:03.508 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:45:03.510 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:45:03.513 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:45:33.618 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:45:33.622 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:45:33.625 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:45:33.628 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:45:33.631 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:46:03.793 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:46:03.797 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:46:03.799 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:46:03.802 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:46:03.813 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:46:34.058 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:46:34.062 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:46:34.065 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:46:34.068 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:46:34.070 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:47:04.375 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:47:04.379 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:47:04.382 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:47:04.385 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:47:04.387 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:47:34.561 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:47:34.565 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:47:34.568 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:47:34.571 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:47:34.574 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:48:04.743 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:48:04.747 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:48:04.750 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:48:04.753 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:48:04.756 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:48:35.037 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:48:35.041 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:48:35.044 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:48:35.047 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:48:35.050 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:49:05.227 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:49:05.231 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:49:05.236 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:49:05.239 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:49:05.253 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:49:35.380 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:49:35.385 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:49:35.389 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:49:35.392 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:49:35.395 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:50:05.763 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:50:05.768 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:50:05.772 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:50:05.776 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:50:05.779 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:50:35.989 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:50:35.994 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:50:35.998 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:50:36.003 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:50:36.009 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:51:06.239 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:51:06.243 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:51:06.245 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:51:06.248 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:51:06.251 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:51:36.336 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:51:36.341 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:51:36.344 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:51:36.346 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:51:36.349 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:52:06.612 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:52:06.615 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:52:06.618 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:52:06.621 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:52:06.624 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:52:36.726 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:52:36.730 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:52:36.734 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:52:36.738 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:52:36.741 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:53:07.086 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:53:07.090 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:53:07.092 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:53:07.095 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:53:07.098 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:53:37.171 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:53:37.175 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:53:37.178 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:53:37.181 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:53:37.184 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:54:07.449 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:54:07.452 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:54:07.455 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:54:07.458 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:54:07.461 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:54:37.619 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:54:37.623 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:54:37.626 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:54:37.630 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:54:37.634 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:55:07.994 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:55:08.007 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:55:08.011 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:55:08.039 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:55:08.043 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:55:38.183 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:55:38.188 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:55:38.191 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:55:38.193 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:55:38.196 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:56:08.342 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:56:08.347 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:56:08.355 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:56:08.359 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:56:08.363 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:56:38.487 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:56:38.492 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:56:38.495 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:56:38.498 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:56:38.503 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:57:08.625 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:57:08.629 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:57:08.633 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:57:08.636 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:57:08.639 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:57:38.750 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:57:38.754 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:57:38.760 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:57:38.763 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:57:38.766 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:58:05.935 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-06-17T13:58:05.973 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2022-06-17T13:58:05.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T13:58:05.988 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/run_tasks.py", line 188, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/ceph_fuse.py", line 160, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait if not (self.is_mounted() and self.fuse_daemon): File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 148, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-06-17T13:58:06.002 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2022-06-17T13:58:06.021 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2022-06-17T13:58:06.022 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T13:58:06.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.152+0000 7f5c927fd700 1 Processor -- start 2022-06-17T13:58:06.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.152+0000 7f5c927fd700 1 -- start start 2022-06-17T13:58:06.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.152+0000 7f5c927fd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8c0ae270 0x7f5c8c0b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:06.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.152+0000 7f5c927fd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5c8c13fde0 0x7f5c8c140230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:06.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.152+0000 7f5c927fd700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c8c0afc30 con 0x7f5c8c0ad9c0 2022-06-17T13:58:06.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.152+0000 7f5c927fd700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c8c0b0cb0 con 0x7f5c8c13fde0 2022-06-17T13:58:06.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.152+0000 7f5c927fd700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8c0b3eb0 con 0x7f5c8c0ae270 2022-06-17T13:58:06.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.152+0000 7f5c90d9a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5c8c13fde0 0x7f5c8c140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:06.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.152+0000 7f5c8bfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8c0ae270 0x7f5c8c0b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:06.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.152+0000 7f5c8b7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f5c8c0ad9c0 0x7f5c8c0b05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58678/0 (socket says 172.21.15.36:58678) 2022-06-17T13:58:06.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.152+0000 7f5c8bfff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8c0ae270 0x7f5c8c0b39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:55936/0 (socket says 172.21.15.36:55936) 2022-06-17T13:58:06.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.152+0000 7f5c8b7fe700 1 -- 172.21.15.36:0/784592728 learned_addr learned my addr 172.21.15.36:0/784592728 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T13:58:06.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.152+0000 7f5c8bfff700 1 -- 172.21.15.36:0/784592728 >> v2:172.21.15.36:3300/0 conn(0x7f5c8c13fde0 msgr2=0x7f5c8c140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T13:58:06.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.152+0000 7f5c8bfff700 1 --2- 172.21.15.36:0/784592728 >> v2:172.21.15.36:3300/0 conn(0x7f5c8c13fde0 0x7f5c8c140230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.152+0000 7f5c8bfff700 1 -- 172.21.15.36:0/784592728 >> v1:172.21.15.36:6789/0 conn(0x7f5c8c0ad9c0 legacy=0x7f5c8c0b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:06.163 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.152+0000 7f5c8bfff700 1 -- 172.21.15.36:0/784592728 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8c0ae650 con 0x7f5c8c0ae270 2022-06-17T13:58:06.164 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.152+0000 7f5c8bfff700 1 --2- 172.21.15.36:0/784592728 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8c0ae270 0x7f5c8c0b39c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f5c74002f70 tx=0x7f5c740090f0).ready entity=mon.2 client_cookie=c1c8208e6c856a4e server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:06.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.152+0000 7f5c8affd700 1 -- 172.21.15.36:0/784592728 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c7400e890 con 0x7f5c8c0ae270 2022-06-17T13:58:06.171 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.152+0000 7f5c8affd700 1 -- 172.21.15.36:0/784592728 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c74000e90 con 0x7f5c8c0ae270 2022-06-17T13:58:06.172 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.152+0000 7f5c8affd700 1 -- 172.21.15.36:0/784592728 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c7400ebe0 con 0x7f5c8c0ae270 2022-06-17T13:58:06.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.156+0000 7f5c927fd700 1 -- 172.21.15.36:0/784592728 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8c0ae270 msgr2=0x7f5c8c0b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:06.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.156+0000 7f5c927fd700 1 --2- 172.21.15.36:0/784592728 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8c0ae270 0x7f5c8c0b39c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f5c74002f70 tx=0x7f5c740090f0).stop 2022-06-17T13:58:06.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.156+0000 7f5c927fd700 1 -- 172.21.15.36:0/784592728 shutdown_connections 2022-06-17T13:58:06.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.156+0000 7f5c927fd700 1 --2- 172.21.15.36:0/784592728 >> v2:172.21.15.36:3300/0 conn(0x7f5c8c13fde0 0x7f5c8c140230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.175 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.156+0000 7f5c927fd700 1 --2- 172.21.15.36:0/784592728 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8c0ae270 0x7f5c8c0b39c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.156+0000 7f5c927fd700 1 -- 172.21.15.36:0/784592728 >> 172.21.15.36:0/784592728 conn(0x7f5c8c05c550 msgr2=0x7f5c8c05c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:06.176 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.156+0000 7f5c927fd700 1 -- 172.21.15.36:0/784592728 shutdown_connections 2022-06-17T13:58:06.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.156+0000 7f5c927fd700 1 -- 172.21.15.36:0/784592728 wait complete. 2022-06-17T13:58:06.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.156+0000 7f5c927fd700 1 Processor -- start 2022-06-17T13:58:06.177 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.156+0000 7f5c927fd700 1 -- start start 2022-06-17T13:58:06.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.160+0000 7f5c927fd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c8c0ad9c0 0x7f5c8c13e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:06.178 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.160+0000 7f5c927fd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c8c0ae270 0x7f5c8c13cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:06.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.160+0000 7f5c927fd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8c13fde0 0x7f5c8c13d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:06.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.160+0000 7f5c927fd700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8c152660 con 0x7f5c8c0ae270 2022-06-17T13:58:06.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.160+0000 7f5c927fd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8c067130 con 0x7f5c8c0ad9c0 2022-06-17T13:58:06.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.160+0000 7f5c927fd700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8c066ad0 con 0x7f5c8c13fde0 2022-06-17T13:58:06.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.160+0000 7f5c8b7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c8c0ae270 0x7f5c8c13cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:06.180 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.160+0000 7f5c8bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c8c0ad9c0 0x7f5c8c13e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:06.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.160+0000 7f5c8b7fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c8c0ae270 0x7f5c8c13cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41154/0 (socket says 172.21.15.36:41154) 2022-06-17T13:58:06.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.160+0000 7f5c8bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c8c0ad9c0 0x7f5c8c13e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:33302/0 (socket says 172.21.15.36:33302) 2022-06-17T13:58:06.181 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.160+0000 7f5c8bfff700 1 -- 172.21.15.36:0/497941698 learned_addr learned my addr 172.21.15.36:0/497941698 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:06.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.160+0000 7f5c90d9a700 1 --2- 172.21.15.36:0/497941698 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8c13fde0 0x7f5c8c13d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:06.182 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.160+0000 7f5c8bfff700 1 -- 172.21.15.36:0/497941698 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8c13fde0 msgr2=0x7f5c8c13d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:06.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.160+0000 7f5c8bfff700 1 --2- 172.21.15.36:0/497941698 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8c13fde0 0x7f5c8c13d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.160+0000 7f5c8bfff700 1 -- 172.21.15.36:0/497941698 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c8c0ae270 msgr2=0x7f5c8c13cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:06.183 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.160+0000 7f5c8bfff700 1 --2- 172.21.15.36:0/497941698 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c8c0ae270 0x7f5c8c13cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.160+0000 7f5c8bfff700 1 -- 172.21.15.36:0/497941698 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7400c2e0 con 0x7f5c8c0ad9c0 2022-06-17T13:58:06.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.160+0000 7f5c8bfff700 1 --2- 172.21.15.36:0/497941698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c8c0ad9c0 0x7f5c8c13e030 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f5c8c1527a0 tx=0x7f5c74016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:06.184 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.160+0000 7f5c897fa700 1 -- 172.21.15.36:0/497941698 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c7401e070 con 0x7f5c8c0ad9c0 2022-06-17T13:58:06.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.160+0000 7f5c897fa700 1 -- 172.21.15.36:0/497941698 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c74018470 con 0x7f5c8c0ad9c0 2022-06-17T13:58:06.185 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.160+0000 7f5c927fd700 1 -- 172.21.15.36:0/497941698 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c8c13d950 con 0x7f5c8c0ad9c0 2022-06-17T13:58:06.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.160+0000 7f5c897fa700 1 -- 172.21.15.36:0/497941698 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c740185e0 con 0x7f5c8c0ad9c0 2022-06-17T13:58:06.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.160+0000 7f5c927fd700 1 -- 172.21.15.36:0/497941698 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c8c13e520 con 0x7f5c8c0ad9c0 2022-06-17T13:58:06.186 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.164+0000 7f5c927fd700 1 -- 172.21.15.36:0/497941698 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c8c0afe60 con 0x7f5c8c0ad9c0 2022-06-17T13:58:06.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.164+0000 7f5c897fa700 1 -- 172.21.15.36:0/497941698 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f5c74018740 con 0x7f5c8c0ad9c0 2022-06-17T13:58:06.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.164+0000 7f5c897fa700 1 --2- 172.21.15.36:0/497941698 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c700539b0 0x7f5c70055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:06.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.164+0000 7f5c897fa700 1 -- 172.21.15.36:0/497941698 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f5c7406d070 con 0x7f5c8c0ad9c0 2022-06-17T13:58:06.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.164+0000 7f5c8b7fe700 1 --2- 172.21.15.36:0/497941698 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c700539b0 0x7f5c70055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:06.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.164+0000 7f5c8b7fe700 1 --2- 172.21.15.36:0/497941698 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c700539b0 0x7f5c70055e70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f5c7c00d000 tx=0x7f5c7c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:06.189 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.172+0000 7f5c897fa700 1 -- 172.21.15.36:0/497941698 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5c74043fe0 con 0x7f5c8c0ad9c0 2022-06-17T13:58:06.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.300+0000 7f5c927fd700 1 -- 172.21.15.36:0/497941698 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c8c057410 con 0x7f5c8c0ad9c0 2022-06-17T13:58:06.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.304+0000 7f5c897fa700 1 -- 172.21.15.36:0/497941698 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+13553 (secure 0 0 0) 0x7f5c74047600 con 0x7f5c8c0ad9c0 2022-06-17T13:58:06.304 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T13:58:06.305 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":154,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:57:35.244125+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.36:6835/2534551587":"2022-06-18T12:57:31.224715+0000","172.21.15.36:6834/2534551587":"2022-06-18T12:57:31.224715+0000"},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T13:58:06.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.308+0000 7f5c927fd700 1 -- 172.21.15.36:0/497941698 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c700539b0 msgr2=0x7f5c70055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:06.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.308+0000 7f5c927fd700 1 --2- 172.21.15.36:0/497941698 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c700539b0 0x7f5c70055e70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f5c7c00d000 tx=0x7f5c7c009040).stop 2022-06-17T13:58:06.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.308+0000 7f5c927fd700 1 -- 172.21.15.36:0/497941698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c8c0ad9c0 msgr2=0x7f5c8c13e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:06.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.308+0000 7f5c927fd700 1 --2- 172.21.15.36:0/497941698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c8c0ad9c0 0x7f5c8c13e030 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f5c8c1527a0 tx=0x7f5c74016000).stop 2022-06-17T13:58:06.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.308+0000 7f5c927fd700 1 -- 172.21.15.36:0/497941698 shutdown_connections 2022-06-17T13:58:06.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.308+0000 7f5c927fd700 1 --2- 172.21.15.36:0/497941698 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c700539b0 0x7f5c70055e70 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.308+0000 7f5c927fd700 1 --2- 172.21.15.36:0/497941698 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8c13fde0 0x7f5c8c13d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.308+0000 7f5c927fd700 1 --2- 172.21.15.36:0/497941698 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c8c0ae270 0x7f5c8c13cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.308+0000 7f5c927fd700 1 --2- 172.21.15.36:0/497941698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c8c0ad9c0 0x7f5c8c13e030 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.308+0000 7f5c927fd700 1 -- 172.21.15.36:0/497941698 >> 172.21.15.36:0/497941698 conn(0x7f5c8c05c550 msgr2=0x7f5c8c0b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:06.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.308+0000 7f5c927fd700 1 -- 172.21.15.36:0/497941698 shutdown_connections 2022-06-17T13:58:06.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.308+0000 7f5c927fd700 1 -- 172.21.15.36:0/497941698 wait complete. 2022-06-17T13:58:06.323 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-06-17T13:58:06.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912ef1b700 1 Processor -- start 2022-06-17T13:58:06.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912ef1b700 1 -- start start 2022-06-17T13:58:06.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912ef1b700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f91280abe30 0x7f912813fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:06.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912ef1b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9128140410 0x7f9128140860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:06.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912ef1b700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91280b1660 con 0x7f91280b1280 2022-06-17T13:58:06.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912ef1b700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91280aef10 con 0x7f9128140410 2022-06-17T13:58:06.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912ef1b700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9128140080 con 0x7f91280abe30 2022-06-17T13:58:06.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912e71a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9128140410 0x7f9128140860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:06.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912e71a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9128140410 0x7f9128140860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41164/0 (socket says 172.21.15.36:41164) 2022-06-17T13:58:06.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912e71a700 1 -- 172.21.15.36:0/629724869 learned_addr learned my addr 172.21.15.36:0/629724869 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:06.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912d718700 1 --2- 172.21.15.36:0/629724869 >> v2:172.21.15.70:3300/0 conn(0x7f91280abe30 0x7f912813fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:06.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912e71a700 1 -- 172.21.15.36:0/629724869 >> v2:172.21.15.70:3300/0 conn(0x7f91280abe30 msgr2=0x7f912813fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T13:58:06.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912e71a700 1 --2- 172.21.15.36:0/629724869 >> v2:172.21.15.70:3300/0 conn(0x7f91280abe30 0x7f912813fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912e71a700 1 -- 172.21.15.36:0/629724869 >> v1:172.21.15.70:6789/0 conn(0x7f91280b1280 legacy=0x7f9128062ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:06.434 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912e71a700 1 -- 172.21.15.36:0/629724869 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91280ac3b0 con 0x7f9128140410 2022-06-17T13:58:06.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912e71a700 1 --2- 172.21.15.36:0/629724869 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9128140410 0x7f9128140860 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 rx=0x7f91280b17f0 tx=0x7f9120009290).ready entity=mon.0 client_cookie=f15ea55833fb9d28 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:06.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912cf17700 1 -- 172.21.15.36:0/629724869 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f912000b830 con 0x7f9128140410 2022-06-17T13:58:06.435 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912cf17700 1 -- 172.21.15.36:0/629724869 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912000b990 con 0x7f9128140410 2022-06-17T13:58:06.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912cf17700 1 -- 172.21.15.36:0/629724869 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f912000bb00 con 0x7f9128140410 2022-06-17T13:58:06.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912ef1b700 1 -- 172.21.15.36:0/629724869 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9128140410 msgr2=0x7f9128140860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:06.436 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912ef1b700 1 --2- 172.21.15.36:0/629724869 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9128140410 0x7f9128140860 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 rx=0x7f91280b17f0 tx=0x7f9120009290).stop 2022-06-17T13:58:06.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912ef1b700 1 -- 172.21.15.36:0/629724869 shutdown_connections 2022-06-17T13:58:06.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912ef1b700 1 --2- 172.21.15.36:0/629724869 >> v2:172.21.15.70:3300/0 conn(0x7f91280abe30 0x7f912813fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912ef1b700 1 --2- 172.21.15.36:0/629724869 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9128140410 0x7f9128140860 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912ef1b700 1 -- 172.21.15.36:0/629724869 >> 172.21.15.36:0/629724869 conn(0x7f912805f770 msgr2=0x7f912805fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:06.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912ef1b700 1 -- 172.21.15.36:0/629724869 shutdown_connections 2022-06-17T13:58:06.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.428+0000 7f912ef1b700 1 -- 172.21.15.36:0/629724869 wait complete. 2022-06-17T13:58:06.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.432+0000 7f912ef1b700 1 Processor -- start 2022-06-17T13:58:06.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.432+0000 7f912ef1b700 1 -- start start 2022-06-17T13:58:06.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.432+0000 7f912ef1b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91280abe30 0x7f91280b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:06.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.432+0000 7f912ef1b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91280b1280 0x7f91280aad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:06.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.432+0000 7f912ef1b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9128140410 0x7f91280ab250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:06.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.432+0000 7f912ef1b700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91281363c0 con 0x7f91280abe30 2022-06-17T13:58:06.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.432+0000 7f912ef1b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912813a780 con 0x7f91280b1280 2022-06-17T13:58:06.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.432+0000 7f912ef1b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912813a9b0 con 0x7f9128140410 2022-06-17T13:58:06.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.432+0000 7f912df19700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91280abe30 0x7f91280b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:06.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.432+0000 7f912df19700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91280abe30 0x7f91280b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41166/0 (socket says 172.21.15.36:41166) 2022-06-17T13:58:06.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.432+0000 7f912df19700 1 -- 172.21.15.36:0/2054058624 learned_addr learned my addr 172.21.15.36:0/2054058624 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:06.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.432+0000 7f912e71a700 1 --2- 172.21.15.36:0/2054058624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9128140410 0x7f91280ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:06.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.432+0000 7f912d718700 1 --2- 172.21.15.36:0/2054058624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91280b1280 0x7f91280aad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:06.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.432+0000 7f912df19700 1 -- 172.21.15.36:0/2054058624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9128140410 msgr2=0x7f91280ab250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:06.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.432+0000 7f912df19700 1 --2- 172.21.15.36:0/2054058624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9128140410 0x7f91280ab250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.432+0000 7f912df19700 1 -- 172.21.15.36:0/2054058624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91280b1280 msgr2=0x7f91280aad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:06.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.432+0000 7f912df19700 1 --2- 172.21.15.36:0/2054058624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91280b1280 0x7f91280aad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.432+0000 7f912df19700 1 -- 172.21.15.36:0/2054058624 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9120009530 con 0x7f91280abe30 2022-06-17T13:58:06.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.432+0000 7f912df19700 1 --2- 172.21.15.36:0/2054058624 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91280abe30 0x7f91280b40d0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 rx=0x7f912806ad20 tx=0x7f9124007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:06.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.432+0000 7f9116ffd700 1 -- 172.21.15.36:0/2054058624 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f912400ce30 con 0x7f91280abe30 2022-06-17T13:58:06.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.432+0000 7f9116ffd700 1 -- 172.21.15.36:0/2054058624 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9124007240 con 0x7f91280abe30 2022-06-17T13:58:06.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.432+0000 7f912ef1b700 1 -- 172.21.15.36:0/2054058624 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912813ac50 con 0x7f91280abe30 2022-06-17T13:58:06.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.432+0000 7f9116ffd700 1 -- 172.21.15.36:0/2054058624 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9124006700 con 0x7f91280abe30 2022-06-17T13:58:06.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.432+0000 7f912ef1b700 1 -- 172.21.15.36:0/2054058624 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912813b170 con 0x7f91280abe30 2022-06-17T13:58:06.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.436+0000 7f912ef1b700 1 -- 172.21.15.36:0/2054058624 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f912813ade0 con 0x7f91280abe30 2022-06-17T13:58:06.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.436+0000 7f9116ffd700 1 -- 172.21.15.36:0/2054058624 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f9124010030 con 0x7f91280abe30 2022-06-17T13:58:06.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.436+0000 7f9116ffd700 1 --2- 172.21.15.36:0/2054058624 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f91080539b0 0x7f9108055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:06.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.436+0000 7f912d718700 1 --2- 172.21.15.36:0/2054058624 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f91080539b0 0x7f9108055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:06.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.436+0000 7f9116ffd700 1 -- 172.21.15.36:0/2054058624 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f912406b9f0 con 0x7f91280abe30 2022-06-17T13:58:06.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.436+0000 7f912d718700 1 --2- 172.21.15.36:0/2054058624 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f91080539b0 0x7f9108055e70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f9118002000 tx=0x7f9118006e80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:06.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.444+0000 7f9116ffd700 1 -- 172.21.15.36:0/2054058624 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9124042a50 con 0x7f91280abe30 2022-06-17T13:58:06.567 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.568+0000 7f912ef1b700 1 -- 172.21.15.36:0/2054058624 --> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f91280ac650 con 0x7f91080539b0 2022-06-17T13:58:06.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.572+0000 7f9116ffd700 1 -- 172.21.15.36:0/2054058624 <== mgr.4104 v2:172.21.15.36:6800/22419 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+67983 (secure 0 0 0) 0x7f91280ac650 con 0x7f91080539b0 2022-06-17T13:58:06.575 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T13:58:06.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.576+0000 7f912ef1b700 1 -- 172.21.15.36:0/2054058624 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f91080539b0 msgr2=0x7f9108055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:06.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.576+0000 7f912ef1b700 1 --2- 172.21.15.36:0/2054058624 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f91080539b0 0x7f9108055e70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f9118002000 tx=0x7f9118006e80).stop 2022-06-17T13:58:06.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.576+0000 7f912ef1b700 1 -- 172.21.15.36:0/2054058624 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91280abe30 msgr2=0x7f91280b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:06.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.576+0000 7f912ef1b700 1 --2- 172.21.15.36:0/2054058624 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91280abe30 0x7f91280b40d0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 rx=0x7f912806ad20 tx=0x7f9124007c10).stop 2022-06-17T13:58:06.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.576+0000 7f912ef1b700 1 -- 172.21.15.36:0/2054058624 shutdown_connections 2022-06-17T13:58:06.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.576+0000 7f912ef1b700 1 --2- 172.21.15.36:0/2054058624 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f91080539b0 0x7f9108055e70 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.576+0000 7f912ef1b700 1 --2- 172.21.15.36:0/2054058624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9128140410 0x7f91280ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.576+0000 7f912ef1b700 1 --2- 172.21.15.36:0/2054058624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91280b1280 0x7f91280aad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.576+0000 7f912ef1b700 1 --2- 172.21.15.36:0/2054058624 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f91280abe30 0x7f91280b40d0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.576+0000 7f912ef1b700 1 -- 172.21.15.36:0/2054058624 >> 172.21.15.36:0/2054058624 conn(0x7f912805f770 msgr2=0x7f912805fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:06.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.576+0000 7f912ef1b700 1 -- 172.21.15.36:0/2054058624 shutdown_connections 2022-06-17T13:58:06.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.576+0000 7f912ef1b700 1 -- 172.21.15.36:0/2054058624 wait complete. 2022-06-17T13:58:06.582 INFO:teuthology.orchestra.run.smithi036.stderr:dumped all 2022-06-17T13:58:06.588 INFO:teuthology.orchestra.run.smithi036.stdout:{"pg_ready":true,"pg_map":{"version":2297,"stamp":"2022-06-17T13:58:04.689133+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":8404000,"kb_used_data":15328,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746570720,"statfs":{"total":773094113280,"available":764488417280,"internally_reserved":8589934592,"allocated":15695872,"data_stored":7700384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2742,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-46,"num_write_kb":-14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.554850"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":138,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-06-17T12:57:34.240894+0000","last_change":"2022-06-17T12:46:41.622624+0000","last_active":"2022-06-17T12:57:34.240894+0000","last_peered":"2022-06-17T12:57:34.240894+0000","last_clean":"2022-06-17T12:57:34.240894+0000","last_became_active":"2022-06-17T12:46:41.622138+0000","last_became_peered":"2022-06-17T12:46:41.622138+0000","last_unstale":"2022-06-17T12:57:34.240894+0000","last_undegraded":"2022-06-17T12:57:34.240894+0000","last_fullsized":"2022-06-17T12:57:34.240894+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":4,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:34.095313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:34.095313+0000","last_clean_scrub_stamp":"2022-06-17T12:46:34.095313+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":138,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-06-17T12:57:34.240542+0000","last_change":"2022-06-17T12:46:45.658148+0000","last_active":"2022-06-17T12:57:34.240542+0000","last_peered":"2022-06-17T12:57:34.240542+0000","last_clean":"2022-06-17T12:57:34.240542+0000","last_became_active":"2022-06-17T12:46:45.657682+0000","last_became_peered":"2022-06-17T12:46:45.657682+0000","last_unstale":"2022-06-17T12:57:34.240542+0000","last_undegraded":"2022-06-17T12:57:34.240542+0000","last_fullsized":"2022-06-17T12:57:34.240542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":135,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-06-17T12:57:34.240570+0000","last_change":"2022-06-17T12:46:45.662621+0000","last_active":"2022-06-17T12:57:34.240570+0000","last_peered":"2022-06-17T12:57:34.240570+0000","last_clean":"2022-06-17T12:57:34.240570+0000","last_became_active":"2022-06-17T12:46:45.662293+0000","last_became_peered":"2022-06-17T12:46:45.662293+0000","last_unstale":"2022-06-17T12:57:34.240570+0000","last_undegraded":"2022-06-17T12:57:34.240570+0000","last_fullsized":"2022-06-17T12:57:34.240570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":136,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-06-17T12:57:34.240237+0000","last_change":"2022-06-17T12:46:46.357931+0000","last_active":"2022-06-17T12:57:34.240237+0000","last_peered":"2022-06-17T12:57:34.240237+0000","last_clean":"2022-06-17T12:57:34.240237+0000","last_became_active":"2022-06-17T12:46:46.356789+0000","last_became_peered":"2022-06-17T12:46:46.356789+0000","last_unstale":"2022-06-17T12:57:34.240237+0000","last_undegraded":"2022-06-17T12:57:34.240237+0000","last_fullsized":"2022-06-17T12:57:34.240237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":138,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-06-17T12:57:34.240588+0000","last_change":"2022-06-17T12:46:45.662903+0000","last_active":"2022-06-17T12:57:34.240588+0000","last_peered":"2022-06-17T12:57:34.240588+0000","last_clean":"2022-06-17T12:57:34.240588+0000","last_became_active":"2022-06-17T12:46:45.662460+0000","last_became_peered":"2022-06-17T12:46:45.662460+0000","last_unstale":"2022-06-17T12:57:34.240588+0000","last_undegraded":"2022-06-17T12:57:34.240588+0000","last_fullsized":"2022-06-17T12:57:34.240588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":134,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-06-17T12:57:34.262357+0000","last_change":"2022-06-17T12:46:46.261176+0000","last_active":"2022-06-17T12:57:34.262357+0000","last_peered":"2022-06-17T12:57:34.262357+0000","last_clean":"2022-06-17T12:57:34.262357+0000","last_became_active":"2022-06-17T12:46:46.260143+0000","last_became_peered":"2022-06-17T12:46:46.260143+0000","last_unstale":"2022-06-17T12:57:34.262357+0000","last_undegraded":"2022-06-17T12:57:34.262357+0000","last_fullsized":"2022-06-17T12:57:34.262357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":135,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-06-17T12:57:34.242051+0000","last_change":"2022-06-17T12:46:46.265378+0000","last_active":"2022-06-17T12:57:34.242051+0000","last_peered":"2022-06-17T12:57:34.242051+0000","last_clean":"2022-06-17T12:57:34.242051+0000","last_became_active":"2022-06-17T12:46:46.264499+0000","last_became_peered":"2022-06-17T12:46:46.264499+0000","last_unstale":"2022-06-17T12:57:34.242051+0000","last_undegraded":"2022-06-17T12:57:34.242051+0000","last_fullsized":"2022-06-17T12:57:34.242051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":134,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-06-17T12:57:34.262466+0000","last_change":"2022-06-17T12:46:45.658907+0000","last_active":"2022-06-17T12:57:34.262466+0000","last_peered":"2022-06-17T12:57:34.262466+0000","last_clean":"2022-06-17T12:57:34.262466+0000","last_became_active":"2022-06-17T12:46:45.658569+0000","last_became_peered":"2022-06-17T12:46:45.658569+0000","last_unstale":"2022-06-17T12:57:34.262466+0000","last_undegraded":"2022-06-17T12:57:34.262466+0000","last_fullsized":"2022-06-17T12:57:34.262466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":136,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-06-17T12:57:34.410767+0000","last_change":"2022-06-17T12:46:45.660735+0000","last_active":"2022-06-17T12:57:34.410767+0000","last_peered":"2022-06-17T12:57:34.410767+0000","last_clean":"2022-06-17T12:57:34.410767+0000","last_became_active":"2022-06-17T12:46:45.660343+0000","last_became_peered":"2022-06-17T12:46:45.660343+0000","last_unstale":"2022-06-17T12:57:34.410767+0000","last_undegraded":"2022-06-17T12:57:34.410767+0000","last_fullsized":"2022-06-17T12:57:34.410767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543002,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050500,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321340,"statfs":{"total":96636764160,"available":95561052160,"internally_reserved":1073741824,"allocated":1961984,"data_stored":962548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 17 13:57:03 2022","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.88400000000000001,"15min":0.88800000000000001},"min":{"1min":0.73199999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.006,"5min":1.1639999999999999,"15min":1.2270000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.92400000000000004,"15min":0.91400000000000003},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.57499999999999996},"max":{"1min":1.056,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.77300000000000002}]},{"osd":1,"last update":"Fri Jun 17 13:57:03 2022","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81399999999999995,"15min":0.81799999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0409999999999999,"5min":1.2110000000000001,"15min":1.2350000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.84099999999999997,"15min":0.81899999999999995},"min":{"1min":0.73899999999999999,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1100000000000001,"5min":1.1519999999999999,"15min":1.1559999999999999},"last":1.046}]},{"osd":2,"last update":"Fri Jun 17 13:57:03 2022","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.86099999999999999,"15min":0.85199999999999998},"min":{"1min":0.60599999999999998,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.95099999999999996,"5min":1.1220000000000001,"15min":1.1759999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.83399999999999996,"15min":0.85999999999999999},"min":{"1min":0.64000000000000001,"5min":0.62,"15min":0.61799999999999999},"max":{"1min":1.0449999999999999,"5min":1.2010000000000001,"15min":1.23},"last":1.0169999999999999}]},{"osd":3,"last update":"Fri Jun 17 13:57:03 2022","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.86099999999999999,"15min":0.86499999999999999},"min":{"1min":0.64500000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.99299999999999999,"5min":1.496,"15min":1.496},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.877,"5min":0.89700000000000002,"15min":0.90900000000000003},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.61699999999999999},"max":{"1min":1.1259999999999999,"5min":1.28,"15min":1.28},"last":0.93300000000000005}]},{"osd":4,"last update":"Fri Jun 17 13:57:03 2022","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78300000000000003,"15min":0.79000000000000004},"min":{"1min":0.58099999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.2170000000000001,"5min":1.597,"15min":1.597},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.85799999999999998,"15min":0.88200000000000001},"min":{"1min":0.56899999999999995,"5min":0.56499999999999995,"15min":0.54600000000000004},"max":{"1min":1.119,"5min":1.623,"15min":1.623},"last":0.99199999999999999}]},{"osd":5,"last update":"Fri Jun 17 13:57:03 2022","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.88500000000000001,"15min":0.89600000000000002},"min":{"1min":0.59799999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.2,"5min":1.611,"15min":1.611},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93300000000000005,"15min":0.94099999999999995},"min":{"1min":0.65800000000000003,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.155,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.90000000000000002}]},{"osd":6,"last update":"Fri Jun 17 13:57:03 2022","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.97599999999999998,"15min":0.98499999999999999},"min":{"1min":0.76000000000000001,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.278,"5min":1.696,"15min":1.696},"last":1.127},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.96799999999999997,"15min":1.002},"min":{"1min":0.69499999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0840000000000001,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.88100000000000001}]}]},{"osd":6,"up_from":14,"seq":60129543003,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050500,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321340,"statfs":{"total":96636764160,"available":95561052160,"internally_reserved":1073741824,"allocated":1961984,"data_stored":962548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 17 13:56:58 2022","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81699999999999995,"15min":0.80800000000000005},"min":{"1min":0.56299999999999994,"5min":0.52800000000000002,"15min":0.48099999999999998},"max":{"1min":1.026,"5min":1.1220000000000001,"15min":1.5549999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.59599999999999997,"5min":0.56000000000000005,"15min":0.52800000000000002},"max":{"1min":1.0820000000000001,"5min":1.1359999999999999,"15min":1.302},"last":0.66900000000000004}]},{"osd":1,"last update":"Fri Jun 17 13:56:58 2022","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.82899999999999996,"15min":0.83599999999999997},"min":{"1min":0.66200000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.02,"5min":1.1419999999999999,"15min":1.1699999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.82899999999999996,"15min":0.84399999999999997},"min":{"1min":0.63,"5min":0.59999999999999998,"15min":0.57299999999999995},"max":{"1min":1.0449999999999999,"5min":1.121,"15min":1.4370000000000001},"last":0.70199999999999996}]},{"osd":2,"last update":"Fri Jun 17 13:56:58 2022","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.84799999999999998,"15min":0.83199999999999996},"min":{"1min":0.68300000000000005,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0249999999999999,"5min":1.179,"15min":1.3879999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.88,"5min":0.86699999999999999,"15min":0.86299999999999999},"min":{"1min":0.71799999999999997,"5min":0.64900000000000002,"15min":0.62},"max":{"1min":1.1060000000000001,"5min":1.224,"15min":1.2430000000000001},"last":0.83599999999999997}]},{"osd":3,"last update":"Fri Jun 17 13:56:58 2022","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.86899999999999999,"15min":0.878},"min":{"1min":0.71799999999999997,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1890000000000001,"5min":1.2170000000000001,"15min":1.296},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.91900000000000004,"15min":0.91100000000000003},"min":{"1min":0.70699999999999996,"5min":0.65400000000000003,"15min":0.63800000000000001},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.339},"last":0.95199999999999996}]},{"osd":4,"last update":"Fri Jun 17 13:56:58 2022","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.56799999999999995,"5min":0.56399999999999995,"15min":0.53800000000000003},"max":{"1min":1.1399999999999999,"5min":1.2010000000000001,"15min":1.3580000000000001},"last":0.999},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.90000000000000002,"15min":0.90200000000000002},"min":{"1min":0.66100000000000003,"5min":0.60399999999999998,"15min":0.57999999999999996},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.238},"last":0.93700000000000006}]},{"osd":5,"last update":"Fri Jun 17 13:56:58 2022","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.879,"15min":0.88400000000000001},"min":{"1min":0.67500000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.141,"5min":1.1799999999999999,"15min":1.5009999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.94599999999999995,"15min":0.94799999999999995},"min":{"1min":0.69899999999999995,"5min":0.63400000000000001,"15min":0.59999999999999998},"max":{"1min":1.222,"5min":1.222,"15min":1.2869999999999999},"last":0.88900000000000001}]},{"osd":7,"last update":"Fri Jun 17 13:56:58 2022","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.95899999999999996,"15min":0.97099999999999997},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.61299999999999999},"max":{"1min":1.377,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0309999999999999,"15min":1.0289999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.66000000000000003},"max":{"1min":1.2809999999999999,"5min":1.714,"15min":1.714},"last":1.0249999999999999}]}]},{"osd":3,"up_from":14,"seq":60129543004,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050500,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321340,"statfs":{"total":96636764160,"available":95561052160,"internally_reserved":1073741824,"allocated":1961984,"data_stored":962548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 17 13:57:57 2022","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.52600000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93999999999999995,"5min":1.0369999999999999,"15min":1.0900000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.72599999999999998,"15min":0.71399999999999997},"min":{"1min":0.52800000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93899999999999995,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.67400000000000004}]},{"osd":1,"last update":"Fri Jun 17 13:57:57 2022","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65900000000000003,"15min":0.65300000000000002},"min":{"1min":0.55000000000000004,"5min":0.48299999999999998,"15min":0.44900000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.2669999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76700000000000002,"15min":0.755},"min":{"1min":0.59799999999999998,"5min":0.54400000000000004,"15min":0.48599999999999999},"max":{"1min":1.0089999999999999,"5min":1.0629999999999999,"15min":1.266},"last":0.59799999999999998}]},{"osd":2,"last update":"Fri Jun 17 13:57:57 2022","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.80500000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0149999999999999,"5min":1.2250000000000001,"15min":1.7589999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.90700000000000003,"15min":0.90400000000000003},"min":{"1min":0.73799999999999999,"5min":0.73499999999999999,"15min":0.66600000000000004},"max":{"1min":1.157,"5min":1.157,"15min":1.704},"last":0.78600000000000003}]},{"osd":4,"last update":"Fri Jun 17 13:57:57 2022","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92700000000000005,"15min":0.93100000000000005},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.67200000000000004},"max":{"1min":1.234,"5min":1.3160000000000001,"15min":1.401},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.88600000000000001,"15min":0.90100000000000002},"min":{"1min":0.70599999999999996,"5min":0.68400000000000005,"15min":0.66500000000000004},"max":{"1min":1.1910000000000001,"5min":1.302,"15min":1.3979999999999999},"last":0.88300000000000001}]},{"osd":5,"last update":"Fri Jun 17 13:57:57 2022","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.97299999999999998,"15min":0.97499999999999998},"min":{"1min":0.80100000000000005,"5min":0.69799999999999995,"15min":0.68700000000000006},"max":{"1min":1.353,"5min":1.353,"15min":1.4790000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.01,"15min":1.0129999999999999},"min":{"1min":0.78500000000000003,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.218,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.96199999999999997}]},{"osd":6,"last update":"Fri Jun 17 13:57:57 2022","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.98899999999999999,"15min":0.996},"min":{"1min":0.77700000000000002,"5min":0.69299999999999995,"15min":0.68300000000000005},"max":{"1min":1.1899999999999999,"5min":1.3979999999999999,"15min":1.8500000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0940000000000001,"15min":1.079},"min":{"1min":0.89500000000000002,"5min":0.88700000000000001,"15min":0.72899999999999998},"max":{"1min":1.482,"5min":1.554,"15min":1.74},"last":0.93000000000000005}]},{"osd":7,"last update":"Fri Jun 17 13:57:57 2022","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.0920000000000001,"15min":1.0980000000000001},"min":{"1min":0.878,"5min":0.72099999999999997,"15min":0.71299999999999997},"max":{"1min":1.3100000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.087,"5min":1.0600000000000001,"15min":1.079},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.27,"5min":1.355,"15min":1.7450000000000001},"last":1.1200000000000001}]}]},{"osd":2,"up_from":14,"seq":60129543003,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050500,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321340,"statfs":{"total":96636764160,"available":95561052160,"internally_reserved":1073741824,"allocated":1961984,"data_stored":962548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 17 13:57:44 2022","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58699999999999997,"15min":0.60299999999999998},"min":{"1min":0.45900000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.143},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70499999999999996,"15min":0.71199999999999997},"min":{"1min":0.499,"5min":0.495,"15min":0.46200000000000002},"max":{"1min":1.028,"5min":1.0800000000000001,"15min":1.204},"last":0.61699999999999999}]},{"osd":1,"last update":"Fri Jun 17 13:57:44 2022","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.68200000000000005,"15min":0.69099999999999995},"min":{"1min":0.57099999999999995,"5min":0.499,"15min":0.48099999999999998},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.526},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72999999999999998,"15min":0.73399999999999999},"min":{"1min":0.57299999999999995,"5min":0.496,"15min":0.47699999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.228},"last":0.76500000000000001}]},{"osd":3,"last update":"Fri Jun 17 13:57:44 2022","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70399999999999996,"15min":0.71599999999999997},"min":{"1min":0.52500000000000002,"5min":0.503,"15min":0.48699999999999999},"max":{"1min":1.0489999999999999,"5min":1.056,"15min":1.2929999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.85899999999999999,"15min":0.85999999999999999},"min":{"1min":0.70499999999999996,"5min":0.64200000000000002,"15min":0.60799999999999998},"max":{"1min":1.167,"5min":1.3109999999999999,"15min":1.3560000000000001},"last":1.006}]},{"osd":4,"last update":"Fri Jun 17 13:57:44 2022","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91700000000000004,"15min":0.92600000000000005},"min":{"1min":0.71699999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.2070000000000001,"5min":1.2689999999999999,"15min":1.4419999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.91700000000000004,"15min":0.94199999999999995},"min":{"1min":0.71799999999999997,"5min":0.71499999999999997,"15min":0.65900000000000003},"max":{"1min":1.0600000000000001,"5min":1.3660000000000001,"15min":1.4299999999999999},"last":0.88200000000000001}]},{"osd":5,"last update":"Fri Jun 17 13:57:44 2022","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.93899999999999995,"15min":0.93300000000000005},"min":{"1min":0.76500000000000001,"5min":0.64200000000000002,"15min":0.629},"max":{"1min":1.329,"5min":1.3560000000000001,"15min":1.508},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97399999999999998,"15min":0.99199999999999999},"min":{"1min":0.79400000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.94899999999999995}]},{"osd":6,"last update":"Fri Jun 17 13:57:44 2022","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.98699999999999999,"15min":1.002},"min":{"1min":0.76900000000000002,"5min":0.71699999999999997,"15min":0.69299999999999995},"max":{"1min":1.1259999999999999,"5min":1.4870000000000001,"15min":1.514},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.012,"5min":1.0069999999999999,"15min":1},"min":{"1min":0.79500000000000004,"5min":0.71599999999999997,"15min":0.68999999999999995},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.79700000000000004}]},{"osd":7,"last update":"Fri Jun 17 13:57:44 2022","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.038,"15min":1.0609999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.365,"5min":1.464,"15min":1.589},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.054,"15min":1.0529999999999999},"min":{"1min":0.88,"5min":0.78000000000000003,"15min":0.77600000000000002},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.90700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575707,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050500,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321340,"statfs":{"total":96636764160,"available":95561052160,"internally_reserved":1073741824,"allocated":1961984,"data_stored":962548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 17 13:57:14 2022","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81699999999999995,"15min":0.83799999999999997},"min":{"1min":0.61699999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0169999999999999,"5min":1.1579999999999999,"15min":1.167},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.83199999999999996,"15min":0.82599999999999996},"min":{"1min":0.64300000000000002,"5min":0.60499999999999998,"15min":0.57099999999999995},"max":{"1min":1.0309999999999999,"5min":1.0920000000000001,"15min":1.1879999999999999},"last":0.67400000000000004}]},{"osd":1,"last update":"Fri Jun 17 13:57:14 2022","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.83399999999999996,"15min":0.84099999999999997},"min":{"1min":0.63400000000000001,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0029999999999999,"5min":1.1000000000000001,"15min":1.232},"last":0.87},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.80400000000000005,"15min":0.81299999999999994},"min":{"1min":0.65800000000000003,"5min":0.621,"15min":0.58699999999999997},"max":{"1min":0.95299999999999996,"5min":1.0920000000000001,"15min":1.254},"last":0.92600000000000005}]},{"osd":2,"last update":"Fri Jun 17 13:57:14 2022","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.83099999999999996,"15min":0.86499999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.008,"5min":1.093,"15min":1.236},"last":1.038},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83999999999999997,"15min":0.83899999999999997},"min":{"1min":0.67600000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.0580000000000001,"5min":1.167,"15min":1.3100000000000001},"last":0.99299999999999999}]},{"osd":3,"last update":"Fri Jun 17 13:57:14 2022","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84799999999999998,"15min":0.88100000000000001},"min":{"1min":0.65500000000000003,"5min":0.61799999999999999,"15min":0.55000000000000004},"max":{"1min":1.151,"5min":1.151,"15min":1.3149999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0049999999999999,"15min":1.006},"min":{"1min":0.89100000000000001,"5min":0.81699999999999995,"15min":0.61499999999999999},"max":{"1min":1.143,"5min":1.161,"15min":1.244},"last":1.1000000000000001}]},{"osd":4,"last update":"Fri Jun 17 13:57:14 2022","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.76400000000000001,"15min":0.79200000000000004},"min":{"1min":0.60599999999999998,"5min":0.53300000000000003,"15min":0.52300000000000002},"max":{"1min":1.236,"5min":1.236,"15min":1.2549999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.88200000000000001,"15min":0.89700000000000002},"min":{"1min":0.65100000000000002,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.077,"5min":1.2110000000000001,"15min":1.2190000000000001},"last":0.96499999999999997}]},{"osd":6,"last update":"Fri Jun 17 13:57:14 2022","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.86199999999999999,"15min":0.88500000000000001},"min":{"1min":0.68100000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.95699999999999996,"15min":0.97399999999999998},"min":{"1min":0.70499999999999996,"5min":0.624,"15min":0.60499999999999998},"max":{"1min":1.095,"5min":1.208,"15min":1.236},"last":1.1659999999999999}]},{"osd":7,"last update":"Fri Jun 17 13:57:14 2022","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.872,"15min":0.88500000000000001},"min":{"1min":0.63600000000000001,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.361},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.98299999999999998,"15min":0.99099999999999999},"min":{"1min":0.82899999999999996,"5min":0.68999999999999995,"15min":0.68400000000000005},"max":{"1min":1.3420000000000001,"5min":1.361,"15min":1.397},"last":0.95399999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575707,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1050500,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321340,"statfs":{"total":96636764160,"available":95561052160,"internally_reserved":1073741824,"allocated":1961984,"data_stored":962548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 17 13:57:32 2022","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.91300000000000003,"15min":0.89200000000000002},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.496},"max":{"1min":1.0840000000000001,"5min":1.288,"15min":1.288},"last":1.085},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.72899999999999998,"5min":0.66900000000000004,"15min":0.622},"max":{"1min":1.099,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.98899999999999999}]},{"osd":1,"last update":"Fri Jun 17 13:57:32 2022","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80800000000000005,"15min":0.79500000000000004},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.52500000000000002},"max":{"1min":1.046,"5min":1.1499999999999999,"15min":1.165},"last":1.002},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.2,"5min":1.2,"15min":1.3100000000000001},"last":0.96799999999999997}]},{"osd":2,"last update":"Fri Jun 17 13:57:32 2022","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.85699999999999998,"15min":0.84999999999999998},"min":{"1min":0.71299999999999997,"5min":0.60399999999999998,"15min":0.58399999999999996},"max":{"1min":1.099,"5min":1.238,"15min":1.238},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.83699999999999997,"15min":0.84599999999999997},"min":{"1min":0.627,"5min":0.627,"15min":0.621},"max":{"1min":0.99099999999999999,"5min":1.1459999999999999,"15min":1.218},"last":0.83099999999999996}]},{"osd":3,"last update":"Fri Jun 17 13:57:32 2022","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85699999999999998,"15min":0.874},"min":{"1min":0.66000000000000003,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.26},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.92300000000000004,"15min":0.90400000000000003},"min":{"1min":0.68999999999999995,"5min":0.63600000000000001,"15min":0.629},"max":{"1min":1.131,"5min":1.2350000000000001,"15min":1.2430000000000001},"last":1.0609999999999999}]},{"osd":5,"last update":"Fri Jun 17 13:57:32 2022","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.86199999999999999,"15min":0.86299999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.54900000000000004},"max":{"1min":1.038,"5min":1.1890000000000001,"15min":1.2729999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.871,"5min":0.89900000000000002,"15min":0.89600000000000002},"min":{"1min":0.56399999999999995,"5min":0.55600000000000005,"15min":0.51800000000000002},"max":{"1min":1.0389999999999999,"5min":1.1879999999999999,"15min":1.234},"last":0.86399999999999999}]},{"osd":6,"last update":"Fri Jun 17 13:57:32 2022","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.85799999999999998,"15min":0.85499999999999998},"min":{"1min":0.64600000000000002,"5min":0.58899999999999997,"15min":0.55800000000000005},"max":{"1min":1.1339999999999999,"5min":1.137,"15min":1.502},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.88400000000000001,"15min":0.90300000000000002},"min":{"1min":0.57699999999999996,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.256,"5min":1.276,"15min":1.5629999999999999},"last":1.024}]},{"osd":7,"last update":"Fri Jun 17 13:57:32 2022","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.91100000000000003,"15min":0.92000000000000004},"min":{"1min":0.63100000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.1839999999999999,"5min":1.2529999999999999,"15min":1.2849999999999999},"last":0.75},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.93200000000000005,"15min":0.93700000000000006},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.59399999999999997},"max":{"1min":1.103,"5min":1.1930000000000001,"15min":1.268},"last":0.93700000000000006}]}]},{"osd":1,"up_from":13,"seq":55834575707,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050500,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321340,"statfs":{"total":96636764160,"available":95561052160,"internally_reserved":1073741824,"allocated":1961984,"data_stored":962548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 17 13:57:38 2022","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.51200000000000001,"5min":0.47899999999999998,"15min":0.46999999999999997},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.75,"15min":0.755},"min":{"1min":0.65700000000000003,"5min":0.59299999999999997,"15min":0.56100000000000005},"max":{"1min":0.94899999999999995,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.93300000000000005}]},{"osd":2,"last update":"Fri Jun 17 13:57:18 2022","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.499,"5min":0.47599999999999998,"15min":0.45900000000000002},"max":{"1min":1.0369999999999999,"5min":1.105,"15min":1.784},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76000000000000001,"15min":0.77500000000000002},"min":{"1min":0.55800000000000005,"5min":0.5,"15min":0.47599999999999998},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.4830000000000001},"last":0.86099999999999999}]},{"osd":3,"last update":"Fri Jun 17 13:57:18 2022","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.66600000000000004,"15min":0.68400000000000005},"min":{"1min":0.52300000000000002,"5min":0.504,"15min":0.504},"max":{"1min":0.85199999999999998,"5min":1.0149999999999999,"15min":1.2050000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.79000000000000004,"15min":0.79800000000000004},"min":{"1min":0.58399999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.871,"5min":1.1910000000000001,"15min":1.27},"last":0.81399999999999995}]},{"osd":4,"last update":"Fri Jun 17 13:57:38 2022","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94599999999999995,"15min":0.94199999999999995},"min":{"1min":0.77500000000000002,"5min":0.70599999999999996,"15min":0.66300000000000003},"max":{"1min":1.206,"5min":1.2729999999999999,"15min":1.3140000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.01,"5min":0.96899999999999997,"15min":0.97099999999999997},"min":{"1min":0.76600000000000001,"5min":0.65600000000000003,"15min":0.64900000000000002},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":1.085}]},{"osd":5,"last update":"Fri Jun 17 13:57:38 2022","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94799999999999995,"15min":0.94999999999999996},"min":{"1min":0.81999999999999995,"5min":0.73699999999999999,"15min":0.68200000000000005},"max":{"1min":1.1060000000000001,"5min":1.371,"15min":1.371},"last":1.1339999999999999},{"interface":"front","average":{"1min":0.995,"5min":1,"15min":0.999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.74199999999999999},"max":{"1min":1.1879999999999999,"5min":1.623,"15min":1.623},"last":0.96699999999999997}]},{"osd":6,"last update":"Fri Jun 17 13:57:18 2022","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0469999999999999,"15min":1.044},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.71499999999999997},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.603},"last":0.998},{"interface":"front","average":{"1min":0.995,"5min":1.0249999999999999,"15min":1.0369999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.74399999999999999},"max":{"1min":1.2090000000000001,"5min":1.6679999999999999,"15min":1.75},"last":1.298}]},{"osd":7,"last update":"Fri Jun 17 13:57:18 2022","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.052,"15min":1.0720000000000001},"min":{"1min":0.75900000000000001,"5min":0.748,"15min":0.748},"max":{"1min":1.24,"5min":1.6799999999999999,"15min":1.7929999999999999},"last":1.167},{"interface":"front","average":{"1min":1.085,"5min":1.1040000000000001,"15min":1.115},"min":{"1min":0.84899999999999998,"5min":0.78300000000000003,"15min":0.76300000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.6619999999999999},"last":1.1970000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575707,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050500,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321340,"statfs":{"total":96636764160,"available":95561052160,"internally_reserved":1073741824,"allocated":1961984,"data_stored":962548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 17 13:57:54 2022","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.61499999999999999,"15min":0.61799999999999999},"min":{"1min":0.47999999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.873,"5min":0.94699999999999995,"15min":1.208},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73899999999999999,"15min":0.72699999999999998},"min":{"1min":0.56299999999999994,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.097,"5min":1.1819999999999999,"15min":1.258},"last":0.79200000000000004}]},{"osd":2,"last update":"Fri Jun 17 13:57:54 2022","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.625,"15min":0.63100000000000001},"min":{"1min":0.495,"5min":0.46700000000000003,"15min":0.45100000000000001},"max":{"1min":0.85899999999999999,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.77600000000000002,"15min":0.76100000000000001},"min":{"1min":0.56299999999999994,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.96399999999999997,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.65800000000000003}]},{"osd":3,"last update":"Fri Jun 17 13:57:54 2022","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.82199999999999995,"15min":0.81100000000000005},"min":{"1min":0.623,"5min":0.58499999999999996,"15min":0.51000000000000001},"max":{"1min":1.036,"5min":1.1000000000000001,"15min":1.349},"last":0.873},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81899999999999995,"15min":0.82299999999999995},"min":{"1min":0.65200000000000002,"5min":0.627,"15min":0.627},"max":{"1min":1.036,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.76400000000000001}]},{"osd":4,"last update":"Fri Jun 17 13:57:54 2022","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.86399999999999999,"15min":0.879},"min":{"1min":0.66500000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.97499999999999998,"5min":1.145,"15min":1.2809999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.95299999999999996,"15min":0.95199999999999996},"min":{"1min":0.70699999999999996,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.2350000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.90100000000000002}]},{"osd":5,"last update":"Fri Jun 17 13:57:54 2022","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.99299999999999999,"15min":0.996},"min":{"1min":0.71699999999999997,"5min":0.69799999999999995,"15min":0.69099999999999995},"max":{"1min":1.464,"5min":1.5129999999999999,"15min":1.623},"last":1.325},{"interface":"front","average":{"1min":1.006,"5min":0.96199999999999997,"15min":0.95499999999999996},"min":{"1min":0.77000000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.85199999999999998}]},{"osd":6,"last update":"Fri Jun 17 13:57:54 2022","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0529999999999999,"15min":1.0449999999999999},"min":{"1min":0.77300000000000002,"5min":0.748,"15min":0.69499999999999995},"max":{"1min":1.468,"5min":1.4910000000000001,"15min":1.51},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0920000000000001,"15min":1.1000000000000001},"min":{"1min":0.79900000000000004,"5min":0.74199999999999999,"15min":0.71999999999999997},"max":{"1min":1.3600000000000001,"5min":1.5740000000000001,"15min":1.6819999999999999},"last":1.2609999999999999}]},{"osd":7,"last update":"Fri Jun 17 13:57:54 2022","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.115,"15min":1.099},"min":{"1min":0.93300000000000005,"5min":0.79500000000000004,"15min":0.73999999999999999},"max":{"1min":1.411,"5min":1.536,"15min":1.536},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.145,"5min":1.139,"15min":1.1499999999999999},"min":{"1min":0.91600000000000004,"5min":0.85199999999999998,"15min":0.78400000000000003},"max":{"1min":1.431,"5min":1.524,"15min":1.619},"last":1.365}]}]}]}} 2022-06-17T13:58:06.592 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-17T13:58:06.698 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f0029ea0700 1 Processor -- start 2022-06-17T13:58:06.699 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f0029ea0700 1 -- start start 2022-06-17T13:58:06.702 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f0029ea0700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0024060780 0x7f0024060b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:06.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f0029ea0700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f00241434f0 0x7f002405c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:06.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f0029ea0700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0024143990 con 0x7f00240afec0 2022-06-17T13:58:06.703 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f0029ea0700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00240b1a80 con 0x7f0024060780 2022-06-17T13:58:06.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f0029ea0700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00240b1d20 con 0x7f00241434f0 2022-06-17T13:58:06.704 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f00237fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0024060780 0x7f0024060b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:06.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f00237fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0024060780 0x7f0024060b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41174/0 (socket says 172.21.15.36:41174) 2022-06-17T13:58:06.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f00237fe700 1 -- 172.21.15.36:0/3679063301 learned_addr learned my addr 172.21.15.36:0/3679063301 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:06.705 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f001affd700 1 --2- 172.21.15.36:0/3679063301 >> v2:172.21.15.70:3300/0 conn(0x7f00241434f0 0x7f002405c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:06.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f00237fe700 1 -- 172.21.15.36:0/3679063301 >> v2:172.21.15.70:3300/0 conn(0x7f00241434f0 msgr2=0x7f002405c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T13:58:06.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f00237fe700 1 --2- 172.21.15.36:0/3679063301 >> v2:172.21.15.70:3300/0 conn(0x7f00241434f0 0x7f002405c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.706 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f00237fe700 1 -- 172.21.15.36:0/3679063301 >> v1:172.21.15.70:6789/0 conn(0x7f00240afec0 legacy=0x7f002413fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T13:58:06.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f00237fe700 1 -- 172.21.15.36:0/3679063301 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00240b2640 con 0x7f0024060780 2022-06-17T13:58:06.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f001affd700 1 --2- 172.21.15.36:0/3679063301 >> v2:172.21.15.70:3300/0 conn(0x7f00241434f0 0x7f002405c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T13:58:06.707 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f00237fe700 1 --2- 172.21.15.36:0/3679063301 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0024060780 0x7f0024060b60 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 rx=0x7f00240afd60 tx=0x7f0014006eb0).ready entity=mon.0 client_cookie=45aa6c6e58c0efda server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:06.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f0022ffd700 1 -- 172.21.15.36:0/3679063301 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f001400d5c0 con 0x7f0024060780 2022-06-17T13:58:06.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f0022ffd700 1 -- 172.21.15.36:0/3679063301 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0014000e90 con 0x7f0024060780 2022-06-17T13:58:06.708 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f0022ffd700 1 -- 172.21.15.36:0/3679063301 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f001400d910 con 0x7f0024060780 2022-06-17T13:58:06.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f0029ea0700 1 -- 172.21.15.36:0/3679063301 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0024060780 msgr2=0x7f0024060b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:06.709 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f0029ea0700 1 --2- 172.21.15.36:0/3679063301 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0024060780 0x7f0024060b60 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 rx=0x7f00240afd60 tx=0x7f0014006eb0).stop 2022-06-17T13:58:06.710 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f0029ea0700 1 -- 172.21.15.36:0/3679063301 shutdown_connections 2022-06-17T13:58:06.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f0029ea0700 1 --2- 172.21.15.36:0/3679063301 >> v2:172.21.15.70:3300/0 conn(0x7f00241434f0 0x7f002405c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.711 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f0029ea0700 1 --2- 172.21.15.36:0/3679063301 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0024060780 0x7f0024060b60 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f0029ea0700 1 -- 172.21.15.36:0/3679063301 >> 172.21.15.36:0/3679063301 conn(0x7f00240b6870 msgr2=0x7f00240b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:06.712 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f0029ea0700 1 -- 172.21.15.36:0/3679063301 shutdown_connections 2022-06-17T13:58:06.713 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.700+0000 7f0029ea0700 1 -- 172.21.15.36:0/3679063301 wait complete. 2022-06-17T13:58:06.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f0029ea0700 1 Processor -- start 2022-06-17T13:58:06.714 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f0029ea0700 1 -- start start 2022-06-17T13:58:06.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f0029ea0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0024060780 0x7f002413eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:06.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f0029ea0700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00240afec0 0x7f00240a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:06.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f0029ea0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f00241434f0 0x7f00240a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:06.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f0029ea0700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002409f810 con 0x7f00241434f0 2022-06-17T13:58:06.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f0029ea0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0024146bc0 con 0x7f0024060780 2022-06-17T13:58:06.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f0029ea0700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0024146e60 con 0x7f00240afec0 2022-06-17T13:58:06.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f0023fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f00241434f0 0x7f00240a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:06.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f00237fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0024060780 0x7f002413eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:06.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f001affd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00240afec0 0x7f00240a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:06.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f0023fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f00241434f0 0x7f00240a1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41184/0 (socket says 172.21.15.36:41184) 2022-06-17T13:58:06.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f00237fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0024060780 0x7f002413eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.36:33330/0 (socket says 172.21.15.36:33330) 2022-06-17T13:58:06.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f001affd700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00240afec0 0x7f00240a1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:55972/0 (socket says 172.21.15.36:55972) 2022-06-17T13:58:06.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f0023fff700 1 -- 172.21.15.36:0/1391544766 learned_addr learned my addr 172.21.15.36:0/1391544766 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:06.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f0023fff700 1 -- 172.21.15.36:0/1391544766 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00240afec0 msgr2=0x7f00240a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:06.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f0023fff700 1 --2- 172.21.15.36:0/1391544766 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00240afec0 0x7f00240a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f0023fff700 1 -- 172.21.15.36:0/1391544766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0024060780 msgr2=0x7f002413eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:06.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f0023fff700 1 --2- 172.21.15.36:0/1391544766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0024060780 0x7f002413eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f0023fff700 1 -- 172.21.15.36:0/1391544766 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001400b1e0 con 0x7f00241434f0 2022-06-17T13:58:06.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f0023fff700 1 --2- 172.21.15.36:0/1391544766 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f00241434f0 0x7f00240a1620 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 rx=0x7f001c002320 tx=0x7f001c0023f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:06.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f00217fa700 1 -- 172.21.15.36:0/1391544766 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f001c006aa0 con 0x7f00241434f0 2022-06-17T13:58:06.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f00217fa700 1 -- 172.21.15.36:0/1391544766 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001c006c00 con 0x7f00241434f0 2022-06-17T13:58:06.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f00217fa700 1 -- 172.21.15.36:0/1391544766 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f001c007030 con 0x7f00241434f0 2022-06-17T13:58:06.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f0029ea0700 1 -- 172.21.15.36:0/1391544766 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0024147950 con 0x7f00241434f0 2022-06-17T13:58:06.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.704+0000 7f0029ea0700 1 -- 172.21.15.36:0/1391544766 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00241472e0 con 0x7f00241434f0 2022-06-17T13:58:06.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.708+0000 7f0029ea0700 1 -- 172.21.15.36:0/1391544766 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f002409f650 con 0x7f00241434f0 2022-06-17T13:58:06.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.708+0000 7f00217fa700 1 -- 172.21.15.36:0/1391544766 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f001c010030 con 0x7f00241434f0 2022-06-17T13:58:06.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.708+0000 7f00217fa700 1 --2- 172.21.15.36:0/1391544766 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0008053a40 0x7f0008055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:06.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.708+0000 7f00217fa700 1 -- 172.21.15.36:0/1391544766 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f001c06bd10 con 0x7f00241434f0 2022-06-17T13:58:06.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.708+0000 7f00237fe700 1 --2- 172.21.15.36:0/1391544766 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0008053a40 0x7f0008055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:06.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.708+0000 7f00237fe700 1 --2- 172.21.15.36:0/1391544766 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0008053a40 0x7f0008055f00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7f001401c000 tx=0x7f0014015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:06.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.716+0000 7f00217fa700 1 -- 172.21.15.36:0/1391544766 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f001c071020 con 0x7f00241434f0 2022-06-17T13:58:06.841 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.840+0000 7f0029ea0700 1 -- 172.21.15.36:0/1391544766 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f002405ca40 con 0x7f00241434f0 2022-06-17T13:58:06.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.844+0000 7f00217fa700 1 -- 172.21.15.36:0/1391544766 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+13553 (secure 0 0 0) 0x7f001c042ce0 con 0x7f00241434f0 2022-06-17T13:58:06.843 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T13:58:06.844 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":154,"fsid":"b68b9700-0c85-42ca-94d7-6cd53db91c14","created":"2022-06-17T12:46:30.671408+0000","modified":"2022-06-17T12:57:35.244125+0000","last_up_change":"2022-06-17T12:46:40.587632+0000","last_in_change":"2022-06-17T12:46:40.587632+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-17T12:46:33.881212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-17T12:46:44.485460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"eb8ddf14-4f6d-4045-9340-acf01d2935ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":22619},{"type":"v1","addr":"172.21.15.36:6803","nonce":22619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":22619},{"type":"v1","addr":"172.21.15.36:6805","nonce":22619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":22619},{"type":"v1","addr":"172.21.15.36:6809","nonce":22619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":22619},{"type":"v1","addr":"172.21.15.36:6807","nonce":22619}]},"public_addr":"172.21.15.36:6803/22619","cluster_addr":"172.21.15.36:6805/22619","heartbeat_back_addr":"172.21.15.36:6809/22619","heartbeat_front_addr":"172.21.15.36:6807/22619","state":["exists","up"]},{"osd":1,"uuid":"07042677-5771-4de0-bfef-a3e2a4540735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":22620},{"type":"v1","addr":"172.21.15.36:6811","nonce":22620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":22620},{"type":"v1","addr":"172.21.15.36:6813","nonce":22620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":22620},{"type":"v1","addr":"172.21.15.36:6817","nonce":22620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":22620},{"type":"v1","addr":"172.21.15.36:6815","nonce":22620}]},"public_addr":"172.21.15.36:6811/22620","cluster_addr":"172.21.15.36:6813/22620","heartbeat_back_addr":"172.21.15.36:6817/22620","heartbeat_front_addr":"172.21.15.36:6815/22620","state":["exists","up"]},{"osd":2,"uuid":"97d0097b-fa05-483c-808c-44feaa212a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":22643},{"type":"v1","addr":"172.21.15.36:6827","nonce":22643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":22643},{"type":"v1","addr":"172.21.15.36:6829","nonce":22643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":22643},{"type":"v1","addr":"172.21.15.36:6833","nonce":22643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":22643},{"type":"v1","addr":"172.21.15.36:6831","nonce":22643}]},"public_addr":"172.21.15.36:6827/22643","cluster_addr":"172.21.15.36:6829/22643","heartbeat_back_addr":"172.21.15.36:6833/22643","heartbeat_front_addr":"172.21.15.36:6831/22643","state":["exists","up"]},{"osd":3,"uuid":"fb471fe2-5427-441e-b6d7-47c004564c69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":22646},{"type":"v1","addr":"172.21.15.36:6819","nonce":22646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":22646},{"type":"v1","addr":"172.21.15.36:6821","nonce":22646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":22646},{"type":"v1","addr":"172.21.15.36:6825","nonce":22646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":22646},{"type":"v1","addr":"172.21.15.36:6823","nonce":22646}]},"public_addr":"172.21.15.36:6819/22646","cluster_addr":"172.21.15.36:6821/22646","heartbeat_back_addr":"172.21.15.36:6825/22646","heartbeat_front_addr":"172.21.15.36:6823/22646","state":["exists","up"]},{"osd":4,"uuid":"4894aebf-e4b3-4436-a3ff-00a1241dbfcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":22812},{"type":"v1","addr":"172.21.15.70:6809","nonce":22812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":22812},{"type":"v1","addr":"172.21.15.70:6811","nonce":22812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":22812},{"type":"v1","addr":"172.21.15.70:6815","nonce":22812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":22812},{"type":"v1","addr":"172.21.15.70:6813","nonce":22812}]},"public_addr":"172.21.15.70:6809/22812","cluster_addr":"172.21.15.70:6811/22812","heartbeat_back_addr":"172.21.15.70:6815/22812","heartbeat_front_addr":"172.21.15.70:6813/22812","state":["exists","up"]},{"osd":5,"uuid":"74e2041b-4756-47c2-bd3c-defaa046e936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":22813},{"type":"v1","addr":"172.21.15.70:6801","nonce":22813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":22813},{"type":"v1","addr":"172.21.15.70:6803","nonce":22813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":22813},{"type":"v1","addr":"172.21.15.70:6807","nonce":22813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":22813},{"type":"v1","addr":"172.21.15.70:6805","nonce":22813}]},"public_addr":"172.21.15.70:6801/22813","cluster_addr":"172.21.15.70:6803/22813","heartbeat_back_addr":"172.21.15.70:6807/22813","heartbeat_front_addr":"172.21.15.70:6805/22813","state":["exists","up"]},{"osd":6,"uuid":"f55cd87d-673d-4717-bde0-103a9204f75b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":22822},{"type":"v1","addr":"172.21.15.70:6817","nonce":22822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":22822},{"type":"v1","addr":"172.21.15.70:6819","nonce":22822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":22822},{"type":"v1","addr":"172.21.15.70:6823","nonce":22822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":22822},{"type":"v1","addr":"172.21.15.70:6821","nonce":22822}]},"public_addr":"172.21.15.70:6817/22822","cluster_addr":"172.21.15.70:6819/22822","heartbeat_back_addr":"172.21.15.70:6823/22822","heartbeat_front_addr":"172.21.15.70:6821/22822","state":["exists","up"]},{"osd":7,"uuid":"cbf2a5fd-464a-4b16-872b-d86fbe56f84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":22837},{"type":"v1","addr":"172.21.15.70:6825","nonce":22837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":22837},{"type":"v1","addr":"172.21.15.70:6827","nonce":22837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":22837},{"type":"v1","addr":"172.21.15.70:6831","nonce":22837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":22837},{"type":"v1","addr":"172.21.15.70:6829","nonce":22837}]},"public_addr":"172.21.15.70:6825/22837","cluster_addr":"172.21.15.70:6827/22837","heartbeat_back_addr":"172.21.15.70:6831/22837","heartbeat_front_addr":"172.21.15.70:6829/22837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.291461+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.537912+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.606947+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.514412+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.394645+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.343806+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.591864+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-17T12:46:38.607045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.36:6835/2534551587":"2022-06-18T12:57:31.224715+0000","172.21.15.36:6834/2534551587":"2022-06-18T12:57:31.224715+0000"},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-17T13:58:06.845 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.844+0000 7f0029ea0700 1 -- 172.21.15.36:0/1391544766 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0008053a40 msgr2=0x7f0008055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:06.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.844+0000 7f0029ea0700 1 --2- 172.21.15.36:0/1391544766 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0008053a40 0x7f0008055f00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7f001401c000 tx=0x7f0014015000).stop 2022-06-17T13:58:06.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.844+0000 7f0029ea0700 1 -- 172.21.15.36:0/1391544766 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f00241434f0 msgr2=0x7f00240a1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:06.846 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.844+0000 7f0029ea0700 1 --2- 172.21.15.36:0/1391544766 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f00241434f0 0x7f00240a1620 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 rx=0x7f001c002320 tx=0x7f001c0023f0).stop 2022-06-17T13:58:06.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.844+0000 7f0029ea0700 1 -- 172.21.15.36:0/1391544766 shutdown_connections 2022-06-17T13:58:06.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.844+0000 7f0029ea0700 1 --2- 172.21.15.36:0/1391544766 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0008053a40 0x7f0008055f00 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.844+0000 7f0029ea0700 1 --2- 172.21.15.36:0/1391544766 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f00241434f0 0x7f00240a1620 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.844+0000 7f0029ea0700 1 --2- 172.21.15.36:0/1391544766 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00240afec0 0x7f00240a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.844+0000 7f0029ea0700 1 --2- 172.21.15.36:0/1391544766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0024060780 0x7f002413eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:06.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.844+0000 7f0029ea0700 1 -- 172.21.15.36:0/1391544766 >> 172.21.15.36:0/1391544766 conn(0x7f00240b6870 msgr2=0x7f00241429f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:06.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.844+0000 7f0029ea0700 1 -- 172.21.15.36:0/1391544766 shutdown_connections 2022-06-17T13:58:06.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:06.844+0000 7f0029ea0700 1 -- 172.21.15.36:0/1391544766 wait complete. 2022-06-17T13:58:07.858 INFO:tasks.ceph:Scrubbing osd.0 2022-06-17T13:58:07.859 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2022-06-17T13:58:07.980 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.980+0000 7fc911de1700 1 Processor -- start 2022-06-17T13:58:07.981 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.980+0000 7fc911de1700 1 -- start start 2022-06-17T13:58:07.983 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.980+0000 7fc911de1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc90c0b4e10 0x7fc90c0b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:07.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.980+0000 7fc911de1700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc90c0b21d0 con 0x7fc90c0620d0 2022-06-17T13:58:07.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.980+0000 7fc911de1700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc90c0b2430 con 0x7fc90c0b31c0 2022-06-17T13:58:07.984 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.980+0000 7fc911de1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c0b26d0 con 0x7fc90c0b4e10 2022-06-17T13:58:07.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.980+0000 7fc90b7fe700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fc90c0620d0 0x7fc90c0b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58718/0 (socket says 172.21.15.36:58718) 2022-06-17T13:58:07.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.980+0000 7fc90b7fe700 1 -- 172.21.15.36:0/642264235 learned_addr learned my addr 172.21.15.36:0/642264235 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T13:58:07.985 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.980+0000 7fc90affd700 1 --2- 172.21.15.36:0/642264235 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc90c0b4e10 0x7fc90c0b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:07.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.984+0000 7fc90a7fc700 1 -- 172.21.15.36:0/642264235 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3624315223 0 0) 0x7fc90c0b21d0 con 0x7fc90c0620d0 2022-06-17T13:58:07.986 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.984+0000 7fc90a7fc700 1 -- 172.21.15.36:0/642264235 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8ec003260 con 0x7fc90c0620d0 2022-06-17T13:58:07.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.984+0000 7fc90a7fc700 1 -- 172.21.15.36:0/642264235 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3605172553 0 0) 0x7fc90c0b2430 con 0x7fc90c0b31c0 2022-06-17T13:58:07.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.984+0000 7fc90a7fc700 1 -- 172.21.15.36:0/642264235 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc90c0b21d0 con 0x7fc90c0b31c0 2022-06-17T13:58:07.987 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.984+0000 7fc90affd700 1 -- 172.21.15.36:0/642264235 >> v1:172.21.15.70:6789/0 conn(0x7fc90c0b31c0 legacy=0x7fc90c141ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:07.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.984+0000 7fc90a7fc700 1 -- 172.21.15.36:0/642264235 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1606758231 0 0) 0x7fc8ec003260 con 0x7fc90c0620d0 2022-06-17T13:58:07.988 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.984+0000 7fc90affd700 1 -- 172.21.15.36:0/642264235 >> v1:172.21.15.36:6789/0 conn(0x7fc90c0620d0 legacy=0x7fc90c0b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:07.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.984+0000 7fc90affd700 1 -- 172.21.15.36:0/642264235 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90c0b43f0 con 0x7fc90c0b4e10 2022-06-17T13:58:07.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.984+0000 7fc90affd700 1 --2- 172.21.15.36:0/642264235 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc90c0b4e10 0x7fc90c0b5bc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fc8fc001490 tx=0x7fc8f4006df0).ready entity=mon.2 client_cookie=cd780f41548650af server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:07.989 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.984+0000 7fc90a7fc700 1 -- 172.21.15.36:0/642264235 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8fc001a00 con 0x7fc90c0b4e10 2022-06-17T13:58:07.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.984+0000 7fc90a7fc700 1 -- 172.21.15.36:0/642264235 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f4000e90 con 0x7fc90c0b4e10 2022-06-17T13:58:07.990 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.984+0000 7fc90a7fc700 1 -- 172.21.15.36:0/642264235 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8f4012460 con 0x7fc90c0b4e10 2022-06-17T13:58:07.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.984+0000 7fc911de1700 1 -- 172.21.15.36:0/642264235 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc90c0b4e10 msgr2=0x7fc90c0b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:07.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.984+0000 7fc911de1700 1 --2- 172.21.15.36:0/642264235 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc90c0b4e10 0x7fc90c0b5bc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fc8fc001490 tx=0x7fc8f4006df0).stop 2022-06-17T13:58:07.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.984+0000 7fc911de1700 1 -- 172.21.15.36:0/642264235 shutdown_connections 2022-06-17T13:58:07.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.984+0000 7fc911de1700 1 --2- 172.21.15.36:0/642264235 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc90c0b4e10 0x7fc90c0b5bc0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:07.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.984+0000 7fc911de1700 1 -- 172.21.15.36:0/642264235 >> 172.21.15.36:0/642264235 conn(0x7fc90c0b9850 msgr2=0x7fc90c0b9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:07.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.984+0000 7fc911de1700 1 -- 172.21.15.36:0/642264235 shutdown_connections 2022-06-17T13:58:07.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.984+0000 7fc911de1700 1 -- 172.21.15.36:0/642264235 wait complete. 2022-06-17T13:58:07.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.984+0000 7fc911de1700 1 Processor -- start 2022-06-17T13:58:07.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.988+0000 7fc911de1700 1 -- start start 2022-06-17T13:58:07.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.988+0000 7fc911de1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc90c0620d0 0x7fc90c0b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:07.999 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.988+0000 7fc911de1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc90c0b31c0 0x7fc90c146dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.988+0000 7fc911de1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc90c0b4e10 0x7fc90c152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.988+0000 7fc911de1700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c112f80 con 0x7fc90c0b4e10 2022-06-17T13:58:08.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.988+0000 7fc911de1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c113220 con 0x7fc90c0b31c0 2022-06-17T13:58:08.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.988+0000 7fc911de1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c1134c0 con 0x7fc90c0620d0 2022-06-17T13:58:08.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.988+0000 7fc90bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc90c0b4e10 0x7fc90c152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.988+0000 7fc90b7fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc90c0620d0 0x7fc90c0b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.988+0000 7fc90bfff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc90c0b4e10 0x7fc90c152ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41196/0 (socket says 172.21.15.36:41196) 2022-06-17T13:58:08.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.988+0000 7fc90b7fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc90c0620d0 0x7fc90c0b0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.36:55984/0 (socket says 172.21.15.36:55984) 2022-06-17T13:58:08.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.988+0000 7fc90bfff700 1 -- 172.21.15.36:0/2050214614 learned_addr learned my addr 172.21.15.36:0/2050214614 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:08.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.988+0000 7fc90affd700 1 --2- 172.21.15.36:0/2050214614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc90c0b31c0 0x7fc90c146dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.988+0000 7fc90bfff700 1 -- 172.21.15.36:0/2050214614 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc90c0620d0 msgr2=0x7fc90c0b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.988+0000 7fc90bfff700 1 --2- 172.21.15.36:0/2050214614 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc90c0620d0 0x7fc90c0b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.988+0000 7fc90bfff700 1 -- 172.21.15.36:0/2050214614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc90c0b31c0 msgr2=0x7fc90c146dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.988+0000 7fc90bfff700 1 --2- 172.21.15.36:0/2050214614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc90c0b31c0 0x7fc90c146dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.988+0000 7fc90bfff700 1 -- 172.21.15.36:0/2050214614 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f400b180 con 0x7fc90c0b4e10 2022-06-17T13:58:08.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.988+0000 7fc90bfff700 1 --2- 172.21.15.36:0/2050214614 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc90c0b4e10 0x7fc90c152ce0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 rx=0x7fc904001110 tx=0x7fc904007a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:08.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.988+0000 7fc908ff9700 1 -- 172.21.15.36:0/2050214614 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc904012400 con 0x7fc90c0b4e10 2022-06-17T13:58:08.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.988+0000 7fc908ff9700 1 -- 172.21.15.36:0/2050214614 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc904012560 con 0x7fc90c0b4e10 2022-06-17T13:58:08.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.988+0000 7fc911de1700 1 -- 172.21.15.36:0/2050214614 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90c152760 con 0x7fc90c0b4e10 2022-06-17T13:58:08.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.988+0000 7fc908ff9700 1 -- 172.21.15.36:0/2050214614 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9040128c0 con 0x7fc90c0b4e10 2022-06-17T13:58:08.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.988+0000 7fc911de1700 1 -- 172.21.15.36:0/2050214614 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90c112be0 con 0x7fc90c0b4e10 2022-06-17T13:58:08.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.992+0000 7fc911de1700 1 -- 172.21.15.36:0/2050214614 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc90c0af730 con 0x7fc90c0b4e10 2022-06-17T13:58:08.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.992+0000 7fc908ff9700 1 -- 172.21.15.36:0/2050214614 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fc904019030 con 0x7fc90c0b4e10 2022-06-17T13:58:08.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.992+0000 7fc908ff9700 1 --2- 172.21.15.36:0/2050214614 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc8f00539b0 0x7fc8f0055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.992+0000 7fc908ff9700 1 -- 172.21.15.36:0/2050214614 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fc90406b120 con 0x7fc90c0b4e10 2022-06-17T13:58:08.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.992+0000 7fc90b7fe700 1 --2- 172.21.15.36:0/2050214614 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc8f00539b0 0x7fc8f0055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.992+0000 7fc908ff9700 1 --2- 172.21.15.36:0/2050214614 >> [v2:172.21.15.36:6802/22619,v1:172.21.15.36:6803/22619] conn(0x7fc8f005a640 0x7fc8f005cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.992+0000 7fc908ff9700 1 -- 172.21.15.36:0/2050214614 --> [v2:172.21.15.36:6802/22619,v1:172.21.15.36:6803/22619] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc8f005d160 con 0x7fc8f005a640 2022-06-17T13:58:08.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.992+0000 7fc908ff9700 1 -- 172.21.15.36:0/2050214614 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc90406d6a0 con 0x7fc90c0b4e10 2022-06-17T13:58:08.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.992+0000 7fc90affd700 1 --2- 172.21.15.36:0/2050214614 >> [v2:172.21.15.36:6802/22619,v1:172.21.15.36:6803/22619] conn(0x7fc8f005a640 0x7fc8f005cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.992+0000 7fc90b7fe700 1 --2- 172.21.15.36:0/2050214614 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc8f00539b0 0x7fc8f0055e70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7fc8fc00d000 tx=0x7fc8fc000f40).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:08.010 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.992+0000 7fc90affd700 1 --2- 172.21.15.36:0/2050214614 >> [v2:172.21.15.36:6802/22619,v1:172.21.15.36:6803/22619] conn(0x7fc8f005a640 0x7fc8f005cae0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:08.011 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:07.996+0000 7fc908ff9700 1 -- 172.21.15.36:0/2050214614 <== osd.0 v2:172.21.15.36:6802/22619 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7fc8f005d160 con 0x7fc8f005a640 2022-06-17T13:58:08.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.012+0000 7fc911de1700 1 -- 172.21.15.36:0/2050214614 --> [v2:172.21.15.36:6802/22619,v1:172.21.15.36:6803/22619] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc90c113680 con 0x7fc8f005a640 2022-06-17T13:58:08.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.016+0000 7fc908ff9700 1 -- 172.21.15.36:0/2050214614 <== osd.0 v2:172.21.15.36:6802/22619 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fc8f4017080 con 0x7fc8f005a640 2022-06-17T13:58:08.015 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T13:58:08.016 INFO:teuthology.orchestra.run.smithi036.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-06-17T13:58:08.016 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T13:58:08.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.016+0000 7fc911de1700 1 -- 172.21.15.36:0/2050214614 >> [v2:172.21.15.36:6802/22619,v1:172.21.15.36:6803/22619] conn(0x7fc8f005a640 msgr2=0x7fc8f005cae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.016+0000 7fc911de1700 1 --2- 172.21.15.36:0/2050214614 >> [v2:172.21.15.36:6802/22619,v1:172.21.15.36:6803/22619] conn(0x7fc8f005a640 0x7fc8f005cae0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.016+0000 7fc911de1700 1 -- 172.21.15.36:0/2050214614 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc8f00539b0 msgr2=0x7fc8f0055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.016+0000 7fc911de1700 1 --2- 172.21.15.36:0/2050214614 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fc8f00539b0 0x7fc8f0055e70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7fc8fc00d000 tx=0x7fc8fc000f40).stop 2022-06-17T13:58:08.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.016+0000 7fc911de1700 1 -- 172.21.15.36:0/2050214614 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc90c0b4e10 msgr2=0x7fc90c152ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.016+0000 7fc911de1700 1 --2- 172.21.15.36:0/2050214614 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc90c0b4e10 0x7fc90c152ce0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 rx=0x7fc904001110 tx=0x7fc904007a50).stop 2022-06-17T13:58:08.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.016+0000 7fc90bfff700 1 -- 172.21.15.36:0/2050214614 reap_dead start 2022-06-17T13:58:08.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.016+0000 7fc911de1700 1 -- 172.21.15.36:0/2050214614 shutdown_connections 2022-06-17T13:58:08.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.016+0000 7fc911de1700 1 -- 172.21.15.36:0/2050214614 >> 172.21.15.36:0/2050214614 conn(0x7fc90c0b9850 msgr2=0x7fc90c0b4680 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:08.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.016+0000 7fc911de1700 1 -- 172.21.15.36:0/2050214614 shutdown_connections 2022-06-17T13:58:08.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.016+0000 7fc911de1700 1 -- 172.21.15.36:0/2050214614 wait complete. 2022-06-17T13:58:08.026 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2022-06-17T13:58:08.134 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.132+0000 7efdf9a78700 1 Processor -- start 2022-06-17T13:58:08.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.136+0000 7efdf9a78700 1 -- start start 2022-06-17T13:58:08.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.136+0000 7efdf9a78700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efdf40aee30 0x7efdf40ada40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.136+0000 7efdf9a78700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdf4060ea0 con 0x7efdf4063940 2022-06-17T13:58:08.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.136+0000 7efdf9a78700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdf4062f60 con 0x7efdf4060ac0 2022-06-17T13:58:08.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.136+0000 7efdf9a78700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdf413fa00 con 0x7efdf40aee30 2022-06-17T13:58:08.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.136+0000 7efdf3fff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7efdf4063940 0x7efdf413e950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58736/0 (socket says 172.21.15.36:58736) 2022-06-17T13:58:08.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.136+0000 7efdf3fff700 1 -- 172.21.15.36:0/2269187236 learned_addr learned my addr 172.21.15.36:0/2269187236 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T13:58:08.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.136+0000 7efdf9277700 1 --2- 172.21.15.36:0/2269187236 >> v2:172.21.15.70:3300/0 conn(0x7efdf40aee30 0x7efdf40ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.136+0000 7efdf37fe700 1 -- 172.21.15.36:0/2269187236 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2317954883 0 0) 0x7efdf4062f60 con 0x7efdf4060ac0 2022-06-17T13:58:08.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.136+0000 7efdf37fe700 1 -- 172.21.15.36:0/2269187236 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdd8003260 con 0x7efdf4060ac0 2022-06-17T13:58:08.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.136+0000 7efdf37fe700 1 -- 172.21.15.36:0/2269187236 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1641211279 0 0) 0x7efdf4060ea0 con 0x7efdf4063940 2022-06-17T13:58:08.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.136+0000 7efdf37fe700 1 -- 172.21.15.36:0/2269187236 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdf4062f60 con 0x7efdf4063940 2022-06-17T13:58:08.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.136+0000 7efdf9277700 1 -- 172.21.15.36:0/2269187236 >> v1:172.21.15.70:6789/0 conn(0x7efdf4060ac0 legacy=0x7efdf40ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.136+0000 7efdf9277700 1 -- 172.21.15.36:0/2269187236 >> v1:172.21.15.36:6789/0 conn(0x7efdf4063940 legacy=0x7efdf413e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.136+0000 7efdf9277700 1 -- 172.21.15.36:0/2269187236 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf4063200 con 0x7efdf40aee30 2022-06-17T13:58:08.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.136+0000 7efdf9277700 1 --2- 172.21.15.36:0/2269187236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdf40aee30 0x7efdf40ada40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7efdec0032f0 tx=0x7efdec0091d0).ready entity=mon.1 client_cookie=7cc829c57f3debd4 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:08.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.136+0000 7efdf37fe700 1 -- 172.21.15.36:0/2269187236 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efdec00b600 con 0x7efdf40aee30 2022-06-17T13:58:08.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.136+0000 7efdf37fe700 1 -- 172.21.15.36:0/2269187236 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdec00b760 con 0x7efdf40aee30 2022-06-17T13:58:08.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf37fe700 1 -- 172.21.15.36:0/2269187236 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efdec00b8d0 con 0x7efdf40aee30 2022-06-17T13:58:08.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf9a78700 1 -- 172.21.15.36:0/2269187236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdf40aee30 msgr2=0x7efdf40ada40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf9a78700 1 --2- 172.21.15.36:0/2269187236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdf40aee30 0x7efdf40ada40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7efdec0032f0 tx=0x7efdec0091d0).stop 2022-06-17T13:58:08.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf9a78700 1 -- 172.21.15.36:0/2269187236 shutdown_connections 2022-06-17T13:58:08.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf9a78700 1 --2- 172.21.15.36:0/2269187236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdf40aee30 0x7efdf40ada40 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf9a78700 1 -- 172.21.15.36:0/2269187236 >> 172.21.15.36:0/2269187236 conn(0x7efdf40b65f0 msgr2=0x7efdf40b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:08.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf9a78700 1 -- 172.21.15.36:0/2269187236 shutdown_connections 2022-06-17T13:58:08.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf9a78700 1 -- 172.21.15.36:0/2269187236 wait complete. 2022-06-17T13:58:08.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf9a78700 1 Processor -- start 2022-06-17T13:58:08.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf9a78700 1 -- start start 2022-06-17T13:58:08.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf9a78700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efdf4060ac0 0x7efdf413dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf9a78700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efdf4063940 0x7efdf40677e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf9a78700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdf40aee30 0x7efdf40643f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf9a78700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf413ca80 con 0x7efdf4063940 2022-06-17T13:58:08.152 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf9a78700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf4064ab0 con 0x7efdf40aee30 2022-06-17T13:58:08.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf9a78700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf4064d50 con 0x7efdf4060ac0 2022-06-17T13:58:08.153 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf3fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efdf4063940 0x7efdf40677e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf3fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efdf4063940 0x7efdf40677e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41212/0 (socket says 172.21.15.36:41212) 2022-06-17T13:58:08.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf3fff700 1 -- 172.21.15.36:0/525182320 learned_addr learned my addr 172.21.15.36:0/525182320 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:08.154 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf9277700 1 --2- 172.21.15.36:0/525182320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdf40aee30 0x7efdf40643f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf8a76700 1 --2- 172.21.15.36:0/525182320 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efdf4060ac0 0x7efdf413dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf3fff700 1 -- 172.21.15.36:0/525182320 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efdf4060ac0 msgr2=0x7efdf413dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.155 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf3fff700 1 --2- 172.21.15.36:0/525182320 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efdf4060ac0 0x7efdf413dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf3fff700 1 -- 172.21.15.36:0/525182320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdf40aee30 msgr2=0x7efdf40643f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf3fff700 1 --2- 172.21.15.36:0/525182320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdf40aee30 0x7efdf40643f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.156 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf3fff700 1 -- 172.21.15.36:0/525182320 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec009470 con 0x7efdf4063940 2022-06-17T13:58:08.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf3fff700 1 --2- 172.21.15.36:0/525182320 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efdf4063940 0x7efdf40677e0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 rx=0x7efde0001110 tx=0x7efde0007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:08.157 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.140+0000 7efdf1ffb700 1 -- 172.21.15.36:0/525182320 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efde00070d0 con 0x7efdf4063940 2022-06-17T13:58:08.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.144+0000 7efdf1ffb700 1 -- 172.21.15.36:0/525182320 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde0007230 con 0x7efdf4063940 2022-06-17T13:58:08.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.144+0000 7efdf9a78700 1 -- 172.21.15.36:0/525182320 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf4064ff0 con 0x7efdf4063940 2022-06-17T13:58:08.158 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.144+0000 7efdf1ffb700 1 -- 172.21.15.36:0/525182320 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efde0005610 con 0x7efdf4063940 2022-06-17T13:58:08.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.144+0000 7efdf9a78700 1 -- 172.21.15.36:0/525182320 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf41525e0 con 0x7efdf4063940 2022-06-17T13:58:08.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.144+0000 7efdf9a78700 1 -- 172.21.15.36:0/525182320 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdf4066490 con 0x7efdf4063940 2022-06-17T13:58:08.159 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.144+0000 7efdf1ffb700 1 -- 172.21.15.36:0/525182320 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7efde0010070 con 0x7efdf4063940 2022-06-17T13:58:08.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.148+0000 7efdf1ffb700 1 --2- 172.21.15.36:0/525182320 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efdd4053a40 0x7efdd4055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.160 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.148+0000 7efdf8a76700 1 --2- 172.21.15.36:0/525182320 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efdd4053a40 0x7efdd4055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.148+0000 7efdf1ffb700 1 -- 172.21.15.36:0/525182320 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7efde006b340 con 0x7efdf4063940 2022-06-17T13:58:08.161 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.148+0000 7efdf8a76700 1 --2- 172.21.15.36:0/525182320 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efdd4053a40 0x7efdd4055f00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7efde800d000 tx=0x7efde8007a20).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:08.162 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.152+0000 7efdf1ffb700 1 -- 172.21.15.36:0/525182320 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efde0042420 con 0x7efdf4063940 2022-06-17T13:58:08.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.288+0000 7efdf9a78700 1 -- 172.21.15.36:0/525182320 --> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7efdf40634a0 con 0x7efdd4053a40 2022-06-17T13:58:08.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.288+0000 7efdf1ffb700 1 -- 172.21.15.36:0/525182320 <== mgr.4104 v2:172.21.15.36:6800/22419 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7efdf40634a0 con 0x7efdd4053a40 2022-06-17T13:58:08.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.292+0000 7efdf9a78700 1 -- 172.21.15.36:0/525182320 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efdd4053a40 msgr2=0x7efdd4055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.292+0000 7efdf9a78700 1 --2- 172.21.15.36:0/525182320 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efdd4053a40 0x7efdd4055f00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7efde800d000 tx=0x7efde8007a20).stop 2022-06-17T13:58:08.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.292+0000 7efdf9a78700 1 -- 172.21.15.36:0/525182320 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efdf4063940 msgr2=0x7efdf40677e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.292+0000 7efdf9a78700 1 --2- 172.21.15.36:0/525182320 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efdf4063940 0x7efdf40677e0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 rx=0x7efde0001110 tx=0x7efde0007b50).stop 2022-06-17T13:58:08.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.292+0000 7efdf9a78700 1 -- 172.21.15.36:0/525182320 shutdown_connections 2022-06-17T13:58:08.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.292+0000 7efdf9a78700 1 --2- 172.21.15.36:0/525182320 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7efdd4053a40 0x7efdd4055f00 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.292+0000 7efdf9a78700 1 --2- 172.21.15.36:0/525182320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdf40aee30 0x7efdf40643f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.292+0000 7efdf9a78700 1 --2- 172.21.15.36:0/525182320 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7efdf4063940 0x7efdf40677e0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.292+0000 7efdf9a78700 1 --2- 172.21.15.36:0/525182320 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efdf4060ac0 0x7efdf413dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.292+0000 7efdf9a78700 1 -- 172.21.15.36:0/525182320 >> 172.21.15.36:0/525182320 conn(0x7efdf40b65f0 msgr2=0x7efdf413e950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:08.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.292+0000 7efdf9a78700 1 -- 172.21.15.36:0/525182320 shutdown_connections 2022-06-17T13:58:08.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.292+0000 7efdf9a78700 1 -- 172.21.15.36:0/525182320 wait complete. 2022-06-17T13:58:08.296 INFO:teuthology.orchestra.run.smithi036.stderr:instructed osd(s) 0 to deep-scrub 2022-06-17T13:58:08.303 INFO:tasks.ceph:Scrubbing osd.1 2022-06-17T13:58:08.304 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2022-06-17T13:58:08.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.420+0000 7f0df5c65700 1 Processor -- start 2022-06-17T13:58:08.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.420+0000 7f0df5c65700 1 -- start start 2022-06-17T13:58:08.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.420+0000 7f0df5c65700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0df00b4e10 0x7f0df00b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.420+0000 7f0df5c65700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0df00b31c0 0x7f0df00b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.420+0000 7f0df5c65700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0df00b21d0 con 0x7f0df00620d0 2022-06-17T13:58:08.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.420+0000 7f0df5c65700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0df00b2430 con 0x7f0df00b4e10 2022-06-17T13:58:08.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.420+0000 7f0df5c65700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0df00b26d0 con 0x7f0df00b31c0 2022-06-17T13:58:08.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.420+0000 7f0deeffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0df00b4e10 0x7f0df00b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.420+0000 7f0deeffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f0df00b4e10 0x7f0df00b5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41218/0 (socket says 172.21.15.36:41218) 2022-06-17T13:58:08.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.420+0000 7f0deeffd700 1 -- 172.21.15.36:0/1895696714 learned_addr learned my addr 172.21.15.36:0/1895696714 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:08.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.424+0000 7f0deffff700 1 --2- 172.21.15.36:0/1895696714 >> v2:172.21.15.70:3300/0 conn(0x7f0df00b31c0 0x7f0df00b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.424+0000 7f0deeffd700 1 -- 172.21.15.36:0/1895696714 >> v2:172.21.15.70:3300/0 conn(0x7f0df00b31c0 msgr2=0x7f0df00b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T13:58:08.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.424+0000 7f0deeffd700 1 --2- 172.21.15.36:0/1895696714 >> v2:172.21.15.70:3300/0 conn(0x7f0df00b31c0 0x7f0df00b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.424+0000 7f0deeffd700 1 -- 172.21.15.36:0/1895696714 >> v1:172.21.15.36:6789/0 conn(0x7f0df00620d0 legacy=0x7f0df00b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T13:58:08.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.424+0000 7f0deeffd700 1 -- 172.21.15.36:0/1895696714 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df00b43f0 con 0x7f0df00b4e10 2022-06-17T13:58:08.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.424+0000 7f0deeffd700 1 --2- 172.21.15.36:0/1895696714 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0df00b4e10 0x7f0df00b5bc0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 rx=0x7f0de0002f70 tx=0x7f0de0006eb0).ready entity=mon.0 client_cookie=459d1b59846a0260 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:08.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.424+0000 7f0dee7fc700 1 -- 172.21.15.36:0/1895696714 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0de000d610 con 0x7f0df00b4e10 2022-06-17T13:58:08.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.424+0000 7f0dee7fc700 1 -- 172.21.15.36:0/1895696714 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de0000e90 con 0x7f0df00b4e10 2022-06-17T13:58:08.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.424+0000 7f0dee7fc700 1 -- 172.21.15.36:0/1895696714 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0de000d9a0 con 0x7f0df00b4e10 2022-06-17T13:58:08.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.424+0000 7f0df5c65700 1 -- 172.21.15.36:0/1895696714 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0df00b4e10 msgr2=0x7f0df00b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.424+0000 7f0df5c65700 1 --2- 172.21.15.36:0/1895696714 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0df00b4e10 0x7f0df00b5bc0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 rx=0x7f0de0002f70 tx=0x7f0de0006eb0).stop 2022-06-17T13:58:08.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.424+0000 7f0df5c65700 1 -- 172.21.15.36:0/1895696714 shutdown_connections 2022-06-17T13:58:08.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.424+0000 7f0df5c65700 1 --2- 172.21.15.36:0/1895696714 >> v2:172.21.15.70:3300/0 conn(0x7f0df00b31c0 0x7f0df00b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.432 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.424+0000 7f0df5c65700 1 --2- 172.21.15.36:0/1895696714 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0df00b4e10 0x7f0df00b5bc0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.424+0000 7f0df5c65700 1 -- 172.21.15.36:0/1895696714 >> 172.21.15.36:0/1895696714 conn(0x7f0df00b9850 msgr2=0x7f0df00b9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:08.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.424+0000 7f0df5c65700 1 -- 172.21.15.36:0/1895696714 shutdown_connections 2022-06-17T13:58:08.433 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.424+0000 7f0df5c65700 1 -- 172.21.15.36:0/1895696714 wait complete. 2022-06-17T13:58:08.438 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.424+0000 7f0df5c65700 1 Processor -- start 2022-06-17T13:58:08.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.424+0000 7f0df5c65700 1 -- start start 2022-06-17T13:58:08.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.428+0000 7f0df5c65700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0df00620d0 0x7f0df01467d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.428+0000 7f0df5c65700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0df00b31c0 0x7f0df013fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.428+0000 7f0df5c65700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0df00b4e10 0x7f0df0156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.440 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.428+0000 7f0df5c65700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df0146ef0 con 0x7f0df00b31c0 2022-06-17T13:58:08.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.428+0000 7f0df5c65700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df0147120 con 0x7f0df00620d0 2022-06-17T13:58:08.441 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.428+0000 7f0df5c65700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df01473c0 con 0x7f0df00b4e10 2022-06-17T13:58:08.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.428+0000 7f0deeffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0df00b31c0 0x7f0df013fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.428+0000 7f0deeffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0df00b31c0 0x7f0df013fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41226/0 (socket says 172.21.15.36:41226) 2022-06-17T13:58:08.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.428+0000 7f0deeffd700 1 -- 172.21.15.36:0/1835722959 learned_addr learned my addr 172.21.15.36:0/1835722959 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:08.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.428+0000 7f0def7fe700 1 --2- 172.21.15.36:0/1835722959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0df00620d0 0x7f0df01467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.428+0000 7f0deffff700 1 --2- 172.21.15.36:0/1835722959 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0df00b4e10 0x7f0df0156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.428+0000 7f0deeffd700 1 -- 172.21.15.36:0/1835722959 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0df00b4e10 msgr2=0x7f0df0156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.428+0000 7f0deeffd700 1 --2- 172.21.15.36:0/1835722959 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0df00b4e10 0x7f0df0156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.428+0000 7f0deeffd700 1 -- 172.21.15.36:0/1835722959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0df00620d0 msgr2=0x7f0df01467d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.428+0000 7f0deeffd700 1 --2- 172.21.15.36:0/1835722959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0df00620d0 0x7f0df01467d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.428+0000 7f0deeffd700 1 -- 172.21.15.36:0/1835722959 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de000b1e0 con 0x7f0df00b31c0 2022-06-17T13:58:08.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.428+0000 7f0deeffd700 1 --2- 172.21.15.36:0/1835722959 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0df00b31c0 0x7f0df013fff0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 rx=0x7f0de0002f70 tx=0x7f0de0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:08.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.428+0000 7f0decff9700 1 -- 172.21.15.36:0/1835722959 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0de0011860 con 0x7f0df00b31c0 2022-06-17T13:58:08.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.428+0000 7f0decff9700 1 -- 172.21.15.36:0/1835722959 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de00119c0 con 0x7f0df00b31c0 2022-06-17T13:58:08.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.428+0000 7f0df5c65700 1 -- 172.21.15.36:0/1835722959 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df01404e0 con 0x7f0df00b31c0 2022-06-17T13:58:08.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.428+0000 7f0decff9700 1 -- 172.21.15.36:0/1835722959 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0de0011d20 con 0x7f0df00b31c0 2022-06-17T13:58:08.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.428+0000 7f0df5c65700 1 -- 172.21.15.36:0/1835722959 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df0155cf0 con 0x7f0df00b31c0 2022-06-17T13:58:08.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.428+0000 7f0df5c65700 1 -- 172.21.15.36:0/1835722959 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0df0156a70 con 0x7f0df00b31c0 2022-06-17T13:58:08.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.432+0000 7f0decff9700 1 -- 172.21.15.36:0/1835722959 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f0de0025030 con 0x7f0df00b31c0 2022-06-17T13:58:08.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.432+0000 7f0decff9700 1 --2- 172.21.15.36:0/1835722959 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0dd40539b0 0x7f0dd4055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.432+0000 7f0decff9700 1 -- 172.21.15.36:0/1835722959 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f0de006ce50 con 0x7f0df00b31c0 2022-06-17T13:58:08.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.432+0000 7f0def7fe700 1 --2- 172.21.15.36:0/1835722959 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0dd40539b0 0x7f0dd4055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.432+0000 7f0decff9700 1 --2- 172.21.15.36:0/1835722959 >> [v2:172.21.15.36:6810/22620,v1:172.21.15.36:6811/22620] conn(0x7f0dd405a640 0x7f0dd405cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.432+0000 7f0decff9700 1 -- 172.21.15.36:0/1835722959 --> [v2:172.21.15.36:6810/22620,v1:172.21.15.36:6811/22620] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0dd405d160 con 0x7f0dd405a640 2022-06-17T13:58:08.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.432+0000 7f0decff9700 1 -- 172.21.15.36:0/1835722959 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0de006f4e0 con 0x7f0df00b31c0 2022-06-17T13:58:08.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.432+0000 7f0def7fe700 1 --2- 172.21.15.36:0/1835722959 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0dd40539b0 0x7f0dd4055e70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f0dd800d000 tx=0x7f0dd8007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:08.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.432+0000 7f0deffff700 1 --2- 172.21.15.36:0/1835722959 >> [v2:172.21.15.36:6810/22620,v1:172.21.15.36:6811/22620] conn(0x7f0dd405a640 0x7f0dd405cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.432+0000 7f0deffff700 1 --2- 172.21.15.36:0/1835722959 >> [v2:172.21.15.36:6810/22620,v1:172.21.15.36:6811/22620] conn(0x7f0dd405a640 0x7f0dd405cae0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:08.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.432+0000 7f0decff9700 1 -- 172.21.15.36:0/1835722959 <== osd.1 v2:172.21.15.36:6810/22620 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f0dd405d160 con 0x7f0dd405a640 2022-06-17T13:58:08.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.452+0000 7f0df5c65700 1 -- 172.21.15.36:0/1835722959 --> [v2:172.21.15.36:6810/22620,v1:172.21.15.36:6811/22620] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0df0145860 con 0x7f0dd405a640 2022-06-17T13:58:08.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.452+0000 7f0decff9700 1 -- 172.21.15.36:0/1835722959 <== osd.1 v2:172.21.15.36:6810/22620 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f0df0145860 con 0x7f0dd405a640 2022-06-17T13:58:08.454 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T13:58:08.455 INFO:teuthology.orchestra.run.smithi036.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-06-17T13:58:08.455 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T13:58:08.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.456+0000 7f0df5c65700 1 -- 172.21.15.36:0/1835722959 >> [v2:172.21.15.36:6810/22620,v1:172.21.15.36:6811/22620] conn(0x7f0dd405a640 msgr2=0x7f0dd405cae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.456+0000 7f0df5c65700 1 --2- 172.21.15.36:0/1835722959 >> [v2:172.21.15.36:6810/22620,v1:172.21.15.36:6811/22620] conn(0x7f0dd405a640 0x7f0dd405cae0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.456+0000 7f0df5c65700 1 -- 172.21.15.36:0/1835722959 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0dd40539b0 msgr2=0x7f0dd4055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.456+0000 7f0df5c65700 1 --2- 172.21.15.36:0/1835722959 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0dd40539b0 0x7f0dd4055e70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f0dd800d000 tx=0x7f0dd8007b90).stop 2022-06-17T13:58:08.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.456+0000 7f0df5c65700 1 -- 172.21.15.36:0/1835722959 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0df00b31c0 msgr2=0x7f0df013fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.456+0000 7f0df5c65700 1 --2- 172.21.15.36:0/1835722959 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0df00b31c0 0x7f0df013fff0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 rx=0x7f0de0002f70 tx=0x7f0de0015000).stop 2022-06-17T13:58:08.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.456+0000 7f0deffff700 1 -- 172.21.15.36:0/1835722959 reap_dead start 2022-06-17T13:58:08.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.456+0000 7f0df5c65700 1 -- 172.21.15.36:0/1835722959 shutdown_connections 2022-06-17T13:58:08.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.456+0000 7f0df5c65700 1 -- 172.21.15.36:0/1835722959 >> 172.21.15.36:0/1835722959 conn(0x7f0df00b9850 msgr2=0x7f0df01416f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:08.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.456+0000 7f0df5c65700 1 -- 172.21.15.36:0/1835722959 shutdown_connections 2022-06-17T13:58:08.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.456+0000 7f0df5c65700 1 -- 172.21.15.36:0/1835722959 wait complete. 2022-06-17T13:58:08.466 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2022-06-17T13:58:08.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.576+0000 7f3ef3be7700 1 Processor -- start 2022-06-17T13:58:08.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.576+0000 7f3ef3be7700 1 -- start start 2022-06-17T13:58:08.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.576+0000 7f3ef3be7700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f3eec060340 0x7f3eec13ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.576+0000 7f3ef3be7700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eec060720 con 0x7f3eec0ad330 2022-06-17T13:58:08.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.576+0000 7f3ef3be7700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eec062d30 con 0x7f3eec13f100 2022-06-17T13:58:08.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.576+0000 7f3ef3be7700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eec062fd0 con 0x7f3eec060340 2022-06-17T13:58:08.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.576+0000 7f3ef1983700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f3eec060340 0x7f3eec13ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.576+0000 7f3ef2184700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f3eec0ad330 0x7f3eec1431b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58768/0 (socket says 172.21.15.36:58768) 2022-06-17T13:58:08.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.576+0000 7f3ef2184700 1 -- 172.21.15.36:0/1654426020 learned_addr learned my addr 172.21.15.36:0/1654426020 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T13:58:08.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.576+0000 7f3ef1182700 1 --1- 172.21.15.36:0/1654426020 >> v1:172.21.15.70:6789/0 conn(0x7f3eec13f100 0x7f3eec142aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:54498/0 (socket says 172.21.15.36:54498) 2022-06-17T13:58:08.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.580+0000 7f3ef0981700 1 -- 172.21.15.36:0/1654426020 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1786115142 0 0) 0x7f3eec062d30 con 0x7f3eec13f100 2022-06-17T13:58:08.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.580+0000 7f3ef1983700 1 -- 172.21.15.36:0/1654426020 >> v1:172.21.15.70:6789/0 conn(0x7f3eec13f100 legacy=0x7f3eec142aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.580+0000 7f3ef1983700 1 -- 172.21.15.36:0/1654426020 >> v1:172.21.15.36:6789/0 conn(0x7f3eec0ad330 legacy=0x7f3eec1431b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.580+0000 7f3ef1983700 1 -- 172.21.15.36:0/1654426020 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec063710 con 0x7f3eec060340 2022-06-17T13:58:08.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.580+0000 7f3ef1983700 1 --2- 172.21.15.36:0/1654426020 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3eec060340 0x7f3eec13ec10 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 rx=0x7f3ee8002f70 tx=0x7f3ee8006df0).ready entity=mon.0 client_cookie=175124d8c3500b4c server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:08.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.580+0000 7f3ef0981700 1 -- 172.21.15.36:0/1654426020 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ee800d610 con 0x7f3eec060340 2022-06-17T13:58:08.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.580+0000 7f3ef0981700 1 -- 172.21.15.36:0/1654426020 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee8000e90 con 0x7f3eec060340 2022-06-17T13:58:08.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.580+0000 7f3ef0981700 1 -- 172.21.15.36:0/1654426020 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ee800d960 con 0x7f3eec060340 2022-06-17T13:58:08.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.580+0000 7f3ef3be7700 1 -- 172.21.15.36:0/1654426020 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3eec060340 msgr2=0x7f3eec13ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.580+0000 7f3ef3be7700 1 --2- 172.21.15.36:0/1654426020 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3eec060340 0x7f3eec13ec10 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 rx=0x7f3ee8002f70 tx=0x7f3ee8006df0).stop 2022-06-17T13:58:08.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.580+0000 7f3ef3be7700 1 -- 172.21.15.36:0/1654426020 shutdown_connections 2022-06-17T13:58:08.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.580+0000 7f3ef3be7700 1 --2- 172.21.15.36:0/1654426020 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3eec060340 0x7f3eec13ec10 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.580+0000 7f3ef3be7700 1 -- 172.21.15.36:0/1654426020 >> 172.21.15.36:0/1654426020 conn(0x7f3eec0b53e0 msgr2=0x7f3eec0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:08.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.580+0000 7f3ef3be7700 1 -- 172.21.15.36:0/1654426020 shutdown_connections 2022-06-17T13:58:08.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.580+0000 7f3ef3be7700 1 -- 172.21.15.36:0/1654426020 wait complete. 2022-06-17T13:58:08.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3ef3be7700 1 Processor -- start 2022-06-17T13:58:08.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3ef3be7700 1 -- start start 2022-06-17T13:58:08.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3ef3be7700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3eec060340 0x7f3eec13b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3ef3be7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3eec0ad330 0x7f3eec065a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3ef3be7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eec13f100 0x7f3eec13cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3ef3be7700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec137450 con 0x7f3eec0ad330 2022-06-17T13:58:08.595 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3ef3be7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec1376f0 con 0x7f3eec13f100 2022-06-17T13:58:08.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3ef3be7700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec13a3b0 con 0x7f3eec060340 2022-06-17T13:58:08.596 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3ef1182700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3eec0ad330 0x7f3eec065a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3ef1182700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3eec0ad330 0x7f3eec065a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41242/0 (socket says 172.21.15.36:41242) 2022-06-17T13:58:08.597 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3ef1182700 1 -- 172.21.15.36:0/3389347559 learned_addr learned my addr 172.21.15.36:0/3389347559 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:08.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3ef1983700 1 --2- 172.21.15.36:0/3389347559 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3eec060340 0x7f3eec13b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3ef2184700 1 --2- 172.21.15.36:0/3389347559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eec13f100 0x7f3eec13cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.599 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3ef1182700 1 -- 172.21.15.36:0/3389347559 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3eec060340 msgr2=0x7f3eec13b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.600 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3ef1182700 1 --2- 172.21.15.36:0/3389347559 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3eec060340 0x7f3eec13b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3ef1182700 1 -- 172.21.15.36:0/3389347559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eec13f100 msgr2=0x7f3eec13cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.601 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3ef1182700 1 --2- 172.21.15.36:0/3389347559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eec13f100 0x7f3eec13cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.602 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3ef1182700 1 -- 172.21.15.36:0/3389347559 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee800b180 con 0x7f3eec0ad330 2022-06-17T13:58:08.602 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3ef1983700 1 --2- 172.21.15.36:0/3389347559 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3eec060340 0x7f3eec13b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T13:58:08.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3ef2184700 1 --2- 172.21.15.36:0/3389347559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eec13f100 0x7f3eec13cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T13:58:08.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3ef1182700 1 --2- 172.21.15.36:0/3389347559 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3eec0ad330 0x7f3eec065a10 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 rx=0x7f3edc000ed0 tx=0x7f3edc007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:08.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3edaffd700 1 -- 172.21.15.36:0/3389347559 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3edc0070d0 con 0x7f3eec0ad330 2022-06-17T13:58:08.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3edaffd700 1 -- 172.21.15.36:0/3389347559 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3edc007230 con 0x7f3eec0ad330 2022-06-17T13:58:08.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3ef3be7700 1 -- 172.21.15.36:0/3389347559 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eec13a850 con 0x7f3eec0ad330 2022-06-17T13:58:08.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3edaffd700 1 -- 172.21.15.36:0/3389347559 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3edc005550 con 0x7f3eec0ad330 2022-06-17T13:58:08.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.584+0000 7f3ef3be7700 1 -- 172.21.15.36:0/3389347559 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eec139400 con 0x7f3eec0ad330 2022-06-17T13:58:08.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.588+0000 7f3ef3be7700 1 -- 172.21.15.36:0/3389347559 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eec13a570 con 0x7f3eec0ad330 2022-06-17T13:58:08.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.588+0000 7f3edaffd700 1 -- 172.21.15.36:0/3389347559 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f3edc01a060 con 0x7f3eec0ad330 2022-06-17T13:58:08.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.588+0000 7f3edaffd700 1 --2- 172.21.15.36:0/3389347559 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3ecc0539d0 0x7f3ecc055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.588+0000 7f3edaffd700 1 -- 172.21.15.36:0/3389347559 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f3edc06b3f0 con 0x7f3eec0ad330 2022-06-17T13:58:08.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.588+0000 7f3ef1983700 1 --2- 172.21.15.36:0/3389347559 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3ecc0539d0 0x7f3ecc055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.588+0000 7f3ef1983700 1 --2- 172.21.15.36:0/3389347559 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3ecc0539d0 0x7f3ecc055e90 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f3ee801c000 tx=0x7f3ee8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:08.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.592+0000 7f3edaffd700 1 -- 172.21.15.36:0/3389347559 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3edc0423c0 con 0x7f3eec0ad330 2022-06-17T13:58:08.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.720+0000 7f3ef3be7700 1 -- 172.21.15.36:0/3389347559 --> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3eec057410 con 0x7f3ecc0539d0 2022-06-17T13:58:08.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.720+0000 7f3edaffd700 1 -- 172.21.15.36:0/3389347559 <== mgr.4104 v2:172.21.15.36:6800/22419 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3eec057410 con 0x7f3ecc0539d0 2022-06-17T13:58:08.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.724+0000 7f3ef3be7700 1 -- 172.21.15.36:0/3389347559 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3ecc0539d0 msgr2=0x7f3ecc055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.724+0000 7f3ef3be7700 1 --2- 172.21.15.36:0/3389347559 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3ecc0539d0 0x7f3ecc055e90 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f3ee801c000 tx=0x7f3ee8015000).stop 2022-06-17T13:58:08.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.724+0000 7f3ef3be7700 1 -- 172.21.15.36:0/3389347559 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3eec0ad330 msgr2=0x7f3eec065a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.724+0000 7f3ef3be7700 1 --2- 172.21.15.36:0/3389347559 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3eec0ad330 0x7f3eec065a10 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 rx=0x7f3edc000ed0 tx=0x7f3edc007c50).stop 2022-06-17T13:58:08.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.724+0000 7f3ef3be7700 1 -- 172.21.15.36:0/3389347559 shutdown_connections 2022-06-17T13:58:08.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.724+0000 7f3ef3be7700 1 --2- 172.21.15.36:0/3389347559 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3ecc0539d0 0x7f3ecc055e90 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.724+0000 7f3ef3be7700 1 --2- 172.21.15.36:0/3389347559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eec13f100 0x7f3eec13cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.724+0000 7f3ef3be7700 1 --2- 172.21.15.36:0/3389347559 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3eec0ad330 0x7f3eec065a10 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.724+0000 7f3ef3be7700 1 --2- 172.21.15.36:0/3389347559 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3eec060340 0x7f3eec13b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.724+0000 7f3ef3be7700 1 -- 172.21.15.36:0/3389347559 >> 172.21.15.36:0/3389347559 conn(0x7f3eec0b53e0 msgr2=0x7f3eec13f580 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:08.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.724+0000 7f3ef3be7700 1 -- 172.21.15.36:0/3389347559 shutdown_connections 2022-06-17T13:58:08.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.724+0000 7f3ef3be7700 1 -- 172.21.15.36:0/3389347559 wait complete. 2022-06-17T13:58:08.729 INFO:teuthology.orchestra.run.smithi036.stderr:instructed osd(s) 1 to deep-scrub 2022-06-17T13:58:08.734 INFO:tasks.ceph:Scrubbing osd.2 2022-06-17T13:58:08.735 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2022-06-17T13:58:08.843 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.844+0000 7f2f89e80700 1 Processor -- start 2022-06-17T13:58:08.844 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.844+0000 7f2f89e80700 1 -- start start 2022-06-17T13:58:08.847 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.844+0000 7f2f89e80700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f8413ae90 0x7f2f84060780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.844+0000 7f2f89e80700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2f84060c70 0x7f2f840aee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.844+0000 7f2f89e80700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f84064880 con 0x7f2f840b2ec0 2022-06-17T13:58:08.848 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.844+0000 7f2f89e80700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f84140f60 con 0x7f2f84060c70 2022-06-17T13:58:08.849 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.844+0000 7f2f89e80700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f84141200 con 0x7f2f8413ae90 2022-06-17T13:58:08.850 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.844+0000 7f2f83fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2f84060c70 0x7f2f840aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.844+0000 7f2f83fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2f84060c70 0x7f2f840aee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41248/0 (socket says 172.21.15.36:41248) 2022-06-17T13:58:08.851 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.844+0000 7f2f83fff700 1 -- 172.21.15.36:0/444044849 learned_addr learned my addr 172.21.15.36:0/444044849 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:08.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.844+0000 7f2f88e7e700 1 --2- 172.21.15.36:0/444044849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f8413ae90 0x7f2f84060780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.852 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.844+0000 7f2f83fff700 1 -- 172.21.15.36:0/444044849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f8413ae90 msgr2=0x7f2f84060780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T13:58:08.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.844+0000 7f2f83fff700 1 --2- 172.21.15.36:0/444044849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f8413ae90 0x7f2f84060780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.844+0000 7f2f83fff700 1 -- 172.21.15.36:0/444044849 >> v1:172.21.15.36:6789/0 conn(0x7f2f840b2ec0 legacy=0x7f2f840af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.853 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.844+0000 7f2f83fff700 1 -- 172.21.15.36:0/444044849 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f841414a0 con 0x7f2f84060c70 2022-06-17T13:58:08.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.844+0000 7f2f88e7e700 1 --2- 172.21.15.36:0/444044849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f8413ae90 0x7f2f84060780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T13:58:08.854 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.844+0000 7f2f83fff700 1 --2- 172.21.15.36:0/444044849 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f84060c70 0x7f2f840aee50 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 rx=0x7f2f6c002f70 tx=0x7f2f6c006eb0).ready entity=mon.0 client_cookie=3d6a7e233f55630f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:08.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.844+0000 7f2f837fe700 1 -- 172.21.15.36:0/444044849 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f6c00d5c0 con 0x7f2f84060c70 2022-06-17T13:58:08.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.844+0000 7f2f837fe700 1 -- 172.21.15.36:0/444044849 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6c000e90 con 0x7f2f84060c70 2022-06-17T13:58:08.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.844+0000 7f2f837fe700 1 -- 172.21.15.36:0/444044849 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f6c00d950 con 0x7f2f84060c70 2022-06-17T13:58:08.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.848+0000 7f2f89e80700 1 -- 172.21.15.36:0/444044849 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f84060c70 msgr2=0x7f2f840aee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.848+0000 7f2f89e80700 1 --2- 172.21.15.36:0/444044849 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f84060c70 0x7f2f840aee50 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 rx=0x7f2f6c002f70 tx=0x7f2f6c006eb0).stop 2022-06-17T13:58:08.856 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.848+0000 7f2f89e80700 1 -- 172.21.15.36:0/444044849 shutdown_connections 2022-06-17T13:58:08.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.848+0000 7f2f89e80700 1 --2- 172.21.15.36:0/444044849 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f84060c70 0x7f2f840aee50 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.848+0000 7f2f89e80700 1 --2- 172.21.15.36:0/444044849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f8413ae90 0x7f2f84060780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.857 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.848+0000 7f2f89e80700 1 -- 172.21.15.36:0/444044849 >> 172.21.15.36:0/444044849 conn(0x7f2f840b65d0 msgr2=0x7f2f840b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:08.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.848+0000 7f2f89e80700 1 -- 172.21.15.36:0/444044849 shutdown_connections 2022-06-17T13:58:08.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.848+0000 7f2f89e80700 1 -- 172.21.15.36:0/444044849 wait complete. 2022-06-17T13:58:08.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.848+0000 7f2f89e80700 1 Processor -- start 2022-06-17T13:58:08.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.848+0000 7f2f89e80700 1 -- start start 2022-06-17T13:58:08.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.852+0000 7f2f89e80700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f84060c70 0x7f2f8413e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.852+0000 7f2f89e80700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f840b2ec0 0x7f2f8405ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.852+0000 7f2f89e80700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f8413ae90 0x7f2f841539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.852+0000 7f2f89e80700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f840a26e0 con 0x7f2f840b2ec0 2022-06-17T13:58:08.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.852+0000 7f2f89e80700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f840a2910 con 0x7f2f84060c70 2022-06-17T13:58:08.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.852+0000 7f2f89e80700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8409ea50 con 0x7f2f8413ae90 2022-06-17T13:58:08.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.852+0000 7f2f83fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f840b2ec0 0x7f2f8405ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.852+0000 7f2f83fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f840b2ec0 0x7f2f8405ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41256/0 (socket says 172.21.15.36:41256) 2022-06-17T13:58:08.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.852+0000 7f2f83fff700 1 -- 172.21.15.36:0/4050221459 learned_addr learned my addr 172.21.15.36:0/4050221459 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:08.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.852+0000 7f2f88e7e700 1 --2- 172.21.15.36:0/4050221459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f84060c70 0x7f2f8413e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.852+0000 7f2f8967f700 1 --2- 172.21.15.36:0/4050221459 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f8413ae90 0x7f2f841539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.866 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.852+0000 7f2f83fff700 1 -- 172.21.15.36:0/4050221459 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f8413ae90 msgr2=0x7f2f841539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.852+0000 7f2f83fff700 1 --2- 172.21.15.36:0/4050221459 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f8413ae90 0x7f2f841539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.852+0000 7f2f83fff700 1 -- 172.21.15.36:0/4050221459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f84060c70 msgr2=0x7f2f8413e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.867 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.852+0000 7f2f83fff700 1 --2- 172.21.15.36:0/4050221459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f84060c70 0x7f2f8413e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.852+0000 7f2f83fff700 1 -- 172.21.15.36:0/4050221459 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6c00b1e0 con 0x7f2f840b2ec0 2022-06-17T13:58:08.868 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.852+0000 7f2f8967f700 1 --2- 172.21.15.36:0/4050221459 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f8413ae90 0x7f2f841539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T13:58:08.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.852+0000 7f2f83fff700 1 --2- 172.21.15.36:0/4050221459 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f840b2ec0 0x7f2f8405ffc0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 rx=0x7f2f6c002f70 tx=0x7f2f6c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:08.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.852+0000 7f2f81ffb700 1 -- 172.21.15.36:0/4050221459 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f6c0119d0 con 0x7f2f840b2ec0 2022-06-17T13:58:08.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.852+0000 7f2f81ffb700 1 -- 172.21.15.36:0/4050221459 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6c011b30 con 0x7f2f840b2ec0 2022-06-17T13:58:08.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.852+0000 7f2f81ffb700 1 -- 172.21.15.36:0/4050221459 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f6c011e90 con 0x7f2f840b2ec0 2022-06-17T13:58:08.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.852+0000 7f2f89e80700 1 -- 172.21.15.36:0/4050221459 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8409eef0 con 0x7f2f840b2ec0 2022-06-17T13:58:08.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.852+0000 7f2f89e80700 1 -- 172.21.15.36:0/4050221459 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f840ac790 con 0x7f2f840b2ec0 2022-06-17T13:58:08.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.856+0000 7f2f89e80700 1 -- 172.21.15.36:0/4050221459 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2f840a2b10 con 0x7f2f840b2ec0 2022-06-17T13:58:08.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.856+0000 7f2f81ffb700 1 -- 172.21.15.36:0/4050221459 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f2f6c025030 con 0x7f2f840b2ec0 2022-06-17T13:58:08.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.856+0000 7f2f81ffb700 1 --2- 172.21.15.36:0/4050221459 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2f68053920 0x7f2f68055de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.856+0000 7f2f81ffb700 1 -- 172.21.15.36:0/4050221459 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f2f6c06d1e0 con 0x7f2f840b2ec0 2022-06-17T13:58:08.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.856+0000 7f2f88e7e700 1 --2- 172.21.15.36:0/4050221459 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2f68053920 0x7f2f68055de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.856+0000 7f2f81ffb700 1 --2- 172.21.15.36:0/4050221459 >> [v2:172.21.15.36:6826/22643,v1:172.21.15.36:6827/22643] conn(0x7f2f6805a4e0 0x7f2f6805c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:08.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.856+0000 7f2f81ffb700 1 -- 172.21.15.36:0/4050221459 --> [v2:172.21.15.36:6826/22643,v1:172.21.15.36:6827/22643] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f6805d000 con 0x7f2f6805a4e0 2022-06-17T13:58:08.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.856+0000 7f2f81ffb700 1 -- 172.21.15.36:0/4050221459 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f6c06f760 con 0x7f2f840b2ec0 2022-06-17T13:58:08.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.856+0000 7f2f8967f700 1 --2- 172.21.15.36:0/4050221459 >> [v2:172.21.15.36:6826/22643,v1:172.21.15.36:6827/22643] conn(0x7f2f6805a4e0 0x7f2f6805c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:08.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.856+0000 7f2f88e7e700 1 --2- 172.21.15.36:0/4050221459 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2f68053920 0x7f2f68055de0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f2f7400d000 tx=0x7f2f74008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:08.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.856+0000 7f2f8967f700 1 --2- 172.21.15.36:0/4050221459 >> [v2:172.21.15.36:6826/22643,v1:172.21.15.36:6827/22643] conn(0x7f2f6805a4e0 0x7f2f6805c980 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:08.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.856+0000 7f2f81ffb700 1 -- 172.21.15.36:0/4050221459 <== osd.2 v2:172.21.15.36:6826/22643 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f2f6805d000 con 0x7f2f6805a4e0 2022-06-17T13:58:08.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.876+0000 7f2f89e80700 1 -- 172.21.15.36:0/4050221459 --> [v2:172.21.15.36:6826/22643,v1:172.21.15.36:6827/22643] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2f841544c0 con 0x7f2f6805a4e0 2022-06-17T13:58:08.877 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.876+0000 7f2f81ffb700 1 -- 172.21.15.36:0/4050221459 <== osd.2 v2:172.21.15.36:6826/22643 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2f841544c0 con 0x7f2f6805a4e0 2022-06-17T13:58:08.878 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T13:58:08.879 INFO:teuthology.orchestra.run.smithi036.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-06-17T13:58:08.879 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T13:58:08.881 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.876+0000 7f2f89e80700 1 -- 172.21.15.36:0/4050221459 >> [v2:172.21.15.36:6826/22643,v1:172.21.15.36:6827/22643] conn(0x7f2f6805a4e0 msgr2=0x7f2f6805c980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.876+0000 7f2f89e80700 1 --2- 172.21.15.36:0/4050221459 >> [v2:172.21.15.36:6826/22643,v1:172.21.15.36:6827/22643] conn(0x7f2f6805a4e0 0x7f2f6805c980 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:08.882 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.876+0000 7f2f89e80700 1 -- 172.21.15.36:0/4050221459 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2f68053920 msgr2=0x7f2f68055de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.876+0000 7f2f89e80700 1 --2- 172.21.15.36:0/4050221459 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f2f68053920 0x7f2f68055de0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f2f7400d000 tx=0x7f2f74008040).stop 2022-06-17T13:58:08.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.876+0000 7f2f89e80700 1 -- 172.21.15.36:0/4050221459 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f840b2ec0 msgr2=0x7f2f8405ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:08.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.876+0000 7f2f89e80700 1 --2- 172.21.15.36:0/4050221459 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2f840b2ec0 0x7f2f8405ffc0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 rx=0x7f2f6c002f70 tx=0x7f2f6c015000).stop 2022-06-17T13:58:08.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.876+0000 7f2f8967f700 1 -- 172.21.15.36:0/4050221459 reap_dead start 2022-06-17T13:58:08.884 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.876+0000 7f2f89e80700 1 -- 172.21.15.36:0/4050221459 shutdown_connections 2022-06-17T13:58:08.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.876+0000 7f2f89e80700 1 -- 172.21.15.36:0/4050221459 >> 172.21.15.36:0/4050221459 conn(0x7f2f840b65d0 msgr2=0x7f2f840aee50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:08.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.876+0000 7f2f89e80700 1 -- 172.21.15.36:0/4050221459 shutdown_connections 2022-06-17T13:58:08.885 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.876+0000 7f2f89e80700 1 -- 172.21.15.36:0/4050221459 wait complete. 2022-06-17T13:58:08.887 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2022-06-17T13:58:08.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.996+0000 7f3c62ca1700 1 Processor -- start 2022-06-17T13:58:08.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.996+0000 7f3c62ca1700 1 -- start start 2022-06-17T13:58:09.000 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.996+0000 7f3c62ca1700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f3c5c062770 0x7f3c5c062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.996+0000 7f3c62ca1700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3c5c1444f0 0x7f3c5c144960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.996+0000 7f3c62ca1700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c5c0b0ba0 con 0x7f3c5c0ad330 2022-06-17T13:58:09.001 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.996+0000 7f3c62ca1700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c5c0ae820 con 0x7f3c5c062770 2022-06-17T13:58:09.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.996+0000 7f3c62ca1700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c5c061030 con 0x7f3c5c1444f0 2022-06-17T13:58:09.002 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.996+0000 7f3c60a3d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f3c5c062770 0x7f3c5c062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.996+0000 7f3c6123e700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f3c5c0ad330 0x7f3c5c0ad7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58798/0 (socket says 172.21.15.36:58798) 2022-06-17T13:58:09.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:08.996+0000 7f3c6123e700 1 -- 172.21.15.36:0/2244248242 learned_addr learned my addr 172.21.15.36:0/2244248242 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T13:58:09.003 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.000+0000 7f3c5bfff700 1 --2- 172.21.15.36:0/2244248242 >> v2:172.21.15.70:3300/0 conn(0x7f3c5c1444f0 0x7f3c5c144960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.000+0000 7f3c5b7fe700 1 -- 172.21.15.36:0/2244248242 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807405011 0 0) 0x7f3c5c0b0ba0 con 0x7f3c5c0ad330 2022-06-17T13:58:09.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.000+0000 7f3c5b7fe700 1 -- 172.21.15.36:0/2244248242 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c48003260 con 0x7f3c5c0ad330 2022-06-17T13:58:09.004 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.000+0000 7f3c60a3d700 1 -- 172.21.15.36:0/2244248242 >> v2:172.21.15.70:3300/0 conn(0x7f3c5c1444f0 msgr2=0x7f3c5c144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T13:58:09.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.000+0000 7f3c60a3d700 1 --2- 172.21.15.36:0/2244248242 >> v2:172.21.15.70:3300/0 conn(0x7f3c5c1444f0 0x7f3c5c144960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.000+0000 7f3c60a3d700 1 -- 172.21.15.36:0/2244248242 >> v1:172.21.15.36:6789/0 conn(0x7f3c5c0ad330 legacy=0x7f3c5c0ad7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.005 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.000+0000 7f3c60a3d700 1 -- 172.21.15.36:0/2244248242 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5c147e90 con 0x7f3c5c062770 2022-06-17T13:58:09.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.000+0000 7f3c60a3d700 1 --2- 172.21.15.36:0/2244248242 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3c5c062770 0x7f3c5c062b50 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 rx=0x7f3c5c061170 tx=0x7f3c50006df0).ready entity=mon.0 client_cookie=ef3e54dd0d75666c server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:09.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.000+0000 7f3c5b7fe700 1 -- 172.21.15.36:0/2244248242 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c5000d4f0 con 0x7f3c5c062770 2022-06-17T13:58:09.006 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.000+0000 7f3c5b7fe700 1 -- 172.21.15.36:0/2244248242 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c50000e90 con 0x7f3c5c062770 2022-06-17T13:58:09.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.000+0000 7f3c5b7fe700 1 -- 172.21.15.36:0/2244248242 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c5000d840 con 0x7f3c5c062770 2022-06-17T13:58:09.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.000+0000 7f3c62ca1700 1 -- 172.21.15.36:0/2244248242 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3c5c062770 msgr2=0x7f3c5c062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.007 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.000+0000 7f3c62ca1700 1 --2- 172.21.15.36:0/2244248242 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3c5c062770 0x7f3c5c062b50 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 rx=0x7f3c5c061170 tx=0x7f3c50006df0).stop 2022-06-17T13:58:09.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.000+0000 7f3c62ca1700 1 -- 172.21.15.36:0/2244248242 shutdown_connections 2022-06-17T13:58:09.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.000+0000 7f3c62ca1700 1 --2- 172.21.15.36:0/2244248242 >> v2:172.21.15.70:3300/0 conn(0x7f3c5c1444f0 0x7f3c5c144960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.008 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.000+0000 7f3c62ca1700 1 --2- 172.21.15.36:0/2244248242 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3c5c062770 0x7f3c5c062b50 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.000+0000 7f3c62ca1700 1 -- 172.21.15.36:0/2244248242 >> 172.21.15.36:0/2244248242 conn(0x7f3c5c0b53e0 msgr2=0x7f3c5c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:09.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.000+0000 7f3c62ca1700 1 -- 172.21.15.36:0/2244248242 shutdown_connections 2022-06-17T13:58:09.009 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.000+0000 7f3c62ca1700 1 -- 172.21.15.36:0/2244248242 wait complete. 2022-06-17T13:58:09.013 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.000+0000 7f3c62ca1700 1 Processor -- start 2022-06-17T13:58:09.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.000+0000 7f3c62ca1700 1 -- start start 2022-06-17T13:58:09.014 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.004+0000 7f3c62ca1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3c5c062770 0x7f3c5c066ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.004+0000 7f3c62ca1700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c5c0ad330 0x7f3c5c05fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.004+0000 7f3c62ca1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c5c1444f0 0x7f3c5c0637e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.015 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.004+0000 7f3c62ca1700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c064c70 con 0x7f3c5c062770 2022-06-17T13:58:09.016 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.004+0000 7f3c62ca1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c14b610 con 0x7f3c5c1444f0 2022-06-17T13:58:09.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.004+0000 7f3c62ca1700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c14b0d0 con 0x7f3c5c0ad330 2022-06-17T13:58:09.017 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.004+0000 7f3c60a3d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3c5c062770 0x7f3c5c066ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.018 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.004+0000 7f3c60a3d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3c5c062770 0x7f3c5c066ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41270/0 (socket says 172.21.15.36:41270) 2022-06-17T13:58:09.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.004+0000 7f3c60a3d700 1 -- 172.21.15.36:0/3538982337 learned_addr learned my addr 172.21.15.36:0/3538982337 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:09.019 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.004+0000 7f3c6123e700 1 --2- 172.21.15.36:0/3538982337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c5c1444f0 0x7f3c5c0637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.004+0000 7f3c5bfff700 1 --2- 172.21.15.36:0/3538982337 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c5c0ad330 0x7f3c5c05fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.020 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.004+0000 7f3c60a3d700 1 -- 172.21.15.36:0/3538982337 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c5c0ad330 msgr2=0x7f3c5c05fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.021 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.004+0000 7f3c60a3d700 1 --2- 172.21.15.36:0/3538982337 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c5c0ad330 0x7f3c5c05fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.004+0000 7f3c60a3d700 1 -- 172.21.15.36:0/3538982337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c5c1444f0 msgr2=0x7f3c5c0637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.022 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.004+0000 7f3c60a3d700 1 --2- 172.21.15.36:0/3538982337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c5c1444f0 0x7f3c5c0637e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.004+0000 7f3c60a3d700 1 -- 172.21.15.36:0/3538982337 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5000b180 con 0x7f3c5c062770 2022-06-17T13:58:09.023 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.004+0000 7f3c60a3d700 1 --2- 172.21.15.36:0/3538982337 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3c5c062770 0x7f3c5c066ad0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 rx=0x7f3c5c061170 tx=0x7f3c50014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:09.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.004+0000 7f3c59ffb700 1 -- 172.21.15.36:0/3538982337 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c50016070 con 0x7f3c5c062770 2022-06-17T13:58:09.024 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.004+0000 7f3c59ffb700 1 -- 172.21.15.36:0/3538982337 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c50013a00 con 0x7f3c5c062770 2022-06-17T13:58:09.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.004+0000 7f3c62ca1700 1 -- 172.21.15.36:0/3538982337 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c5c14b370 con 0x7f3c5c062770 2022-06-17T13:58:09.025 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.004+0000 7f3c59ffb700 1 -- 172.21.15.36:0/3538982337 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c50013d60 con 0x7f3c5c062770 2022-06-17T13:58:09.026 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.004+0000 7f3c62ca1700 1 -- 172.21.15.36:0/3538982337 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c5c1564e0 con 0x7f3c5c062770 2022-06-17T13:58:09.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.004+0000 7f3c62ca1700 1 -- 172.21.15.36:0/3538982337 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c5c144e50 con 0x7f3c5c062770 2022-06-17T13:58:09.027 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.008+0000 7f3c59ffb700 1 -- 172.21.15.36:0/3538982337 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f3c5000f030 con 0x7f3c5c062770 2022-06-17T13:58:09.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.008+0000 7f3c59ffb700 1 --2- 172.21.15.36:0/3538982337 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3c3c053a60 0x7f3c3c055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.028 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.008+0000 7f3c59ffb700 1 -- 172.21.15.36:0/3538982337 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f3c5006c980 con 0x7f3c5c062770 2022-06-17T13:58:09.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.008+0000 7f3c5bfff700 1 --2- 172.21.15.36:0/3538982337 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3c3c053a60 0x7f3c3c055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.029 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.008+0000 7f3c5bfff700 1 --2- 172.21.15.36:0/3538982337 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3c3c053a60 0x7f3c3c055f20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f3c4c00b000 tx=0x7f3c4c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:09.030 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.012+0000 7f3c59ffb700 1 -- 172.21.15.36:0/3538982337 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3c50043a60 con 0x7f3c5c062770 2022-06-17T13:58:09.113 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:58:09.116 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:58:09.119 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:58:09.122 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:58:09.125 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:58:09.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.140+0000 7f3c62ca1700 1 -- 172.21.15.36:0/3538982337 --> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c5c14af50 con 0x7f3c3c053a60 2022-06-17T13:58:09.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.140+0000 7f3c59ffb700 1 -- 172.21.15.36:0/3538982337 <== mgr.4104 v2:172.21.15.36:6800/22419 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3c4c010620 con 0x7f3c3c053a60 2022-06-17T13:58:09.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.140+0000 7f3c62ca1700 1 -- 172.21.15.36:0/3538982337 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3c3c053a60 msgr2=0x7f3c3c055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.140+0000 7f3c62ca1700 1 --2- 172.21.15.36:0/3538982337 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3c3c053a60 0x7f3c3c055f20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f3c4c00b000 tx=0x7f3c4c008040).stop 2022-06-17T13:58:09.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.140+0000 7f3c62ca1700 1 -- 172.21.15.36:0/3538982337 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3c5c062770 msgr2=0x7f3c5c066ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.140+0000 7f3c62ca1700 1 --2- 172.21.15.36:0/3538982337 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3c5c062770 0x7f3c5c066ad0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 rx=0x7f3c5c061170 tx=0x7f3c50014000).stop 2022-06-17T13:58:09.142 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.140+0000 7f3c62ca1700 1 -- 172.21.15.36:0/3538982337 shutdown_connections 2022-06-17T13:58:09.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.140+0000 7f3c62ca1700 1 --2- 172.21.15.36:0/3538982337 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f3c3c053a60 0x7f3c3c055f20 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.140+0000 7f3c62ca1700 1 --2- 172.21.15.36:0/3538982337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c5c1444f0 0x7f3c5c0637e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.143 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.140+0000 7f3c62ca1700 1 --2- 172.21.15.36:0/3538982337 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c5c0ad330 0x7f3c5c05fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.140+0000 7f3c62ca1700 1 --2- 172.21.15.36:0/3538982337 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f3c5c062770 0x7f3c5c066ad0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.140+0000 7f3c62ca1700 1 -- 172.21.15.36:0/3538982337 >> 172.21.15.36:0/3538982337 conn(0x7f3c5c0b53e0 msgr2=0x7f3c5c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:09.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.140+0000 7f3c62ca1700 1 -- 172.21.15.36:0/3538982337 shutdown_connections 2022-06-17T13:58:09.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.140+0000 7f3c62ca1700 1 -- 172.21.15.36:0/3538982337 wait complete. 2022-06-17T13:58:09.145 INFO:teuthology.orchestra.run.smithi036.stderr:instructed osd(s) 2 to deep-scrub 2022-06-17T13:58:09.152 INFO:tasks.ceph:Scrubbing osd.3 2022-06-17T13:58:09.153 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2022-06-17T13:58:09.266 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.264+0000 7f26be42b700 1 Processor -- start 2022-06-17T13:58:09.267 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.264+0000 7f26be42b700 1 -- start start 2022-06-17T13:58:09.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.268+0000 7f26be42b700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f26b8063730 0x7f26b80af380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.270 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.268+0000 7f26be42b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26b80b2430 0x7f26b8143760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.268+0000 7f26be42b700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b8063b10 con 0x7f26b80b3290 2022-06-17T13:58:09.271 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.268+0000 7f26be42b700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b80ada90 con 0x7f26b8063730 2022-06-17T13:58:09.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.268+0000 7f26be42b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b80b3d50 con 0x7f26b80b2430 2022-06-17T13:58:09.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.268+0000 7f26bd429700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f26b8063730 0x7f26b80af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.272 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.268+0000 7f26bcc28700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26b80b2430 0x7f26b8143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.268+0000 7f26bdc2a700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f26b80b3290 0x7f26b8128b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:54544/0 (socket says 172.21.15.36:54544) 2022-06-17T13:58:09.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.268+0000 7f26bdc2a700 1 -- 172.21.15.36:0/61323793 learned_addr learned my addr 172.21.15.36:0/61323793 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T13:58:09.273 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.268+0000 7f26bd429700 1 -- 172.21.15.36:0/61323793 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26b80b2430 msgr2=0x7f26b8143760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T13:58:09.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.268+0000 7f26bd429700 1 --2- 172.21.15.36:0/61323793 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26b80b2430 0x7f26b8143760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.268+0000 7f26bd429700 1 -- 172.21.15.36:0/61323793 >> v1:172.21.15.70:6789/0 conn(0x7f26b80b3290 legacy=0x7f26b8128b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.274 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.268+0000 7f26bd429700 1 -- 172.21.15.36:0/61323793 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b80af0e0 con 0x7f26b8063730 2022-06-17T13:58:09.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.268+0000 7f26bcc28700 1 --2- 172.21.15.36:0/61323793 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26b80b2430 0x7f26b8143760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T13:58:09.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.268+0000 7f26bd429700 1 --2- 172.21.15.36:0/61323793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26b8063730 0x7f26b80af380 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f26b4002f70 tx=0x7f26b4006df0).ready entity=mon.1 client_cookie=c1104fcb278f484 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:09.275 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.268+0000 7f26a7fff700 1 -- 172.21.15.36:0/61323793 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26b400d570 con 0x7f26b8063730 2022-06-17T13:58:09.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.272+0000 7f26a7fff700 1 -- 172.21.15.36:0/61323793 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b4000e90 con 0x7f26b8063730 2022-06-17T13:58:09.276 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.272+0000 7f26a7fff700 1 -- 172.21.15.36:0/61323793 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26b400d8c0 con 0x7f26b8063730 2022-06-17T13:58:09.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.272+0000 7f26be42b700 1 -- 172.21.15.36:0/61323793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26b8063730 msgr2=0x7f26b80af380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.287 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.272+0000 7f26be42b700 1 --2- 172.21.15.36:0/61323793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26b8063730 0x7f26b80af380 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f26b4002f70 tx=0x7f26b4006df0).stop 2022-06-17T13:58:09.288 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.272+0000 7f26be42b700 1 -- 172.21.15.36:0/61323793 shutdown_connections 2022-06-17T13:58:09.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.272+0000 7f26be42b700 1 --2- 172.21.15.36:0/61323793 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26b80b2430 0x7f26b8143760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.289 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.272+0000 7f26be42b700 1 --2- 172.21.15.36:0/61323793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26b8063730 0x7f26b80af380 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.290 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.272+0000 7f26be42b700 1 -- 172.21.15.36:0/61323793 >> 172.21.15.36:0/61323793 conn(0x7f26b8060880 msgr2=0x7f26b80614b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:09.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.272+0000 7f26be42b700 1 -- 172.21.15.36:0/61323793 shutdown_connections 2022-06-17T13:58:09.291 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.272+0000 7f26be42b700 1 -- 172.21.15.36:0/61323793 wait complete. 2022-06-17T13:58:09.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.272+0000 7f26be42b700 1 Processor -- start 2022-06-17T13:58:09.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.276+0000 7f26be42b700 1 -- start start 2022-06-17T13:58:09.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.276+0000 7f26be42b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26b8063730 0x7f26b8066350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.294 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.276+0000 7f26be42b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26b80b2430 0x7f26b805fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.276+0000 7f26be42b700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26b80b3290 0x7f26b81533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.276+0000 7f26be42b700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b8143f00 con 0x7f26b8063730 2022-06-17T13:58:09.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.276+0000 7f26be42b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b8065360 con 0x7f26b80b2430 2022-06-17T13:58:09.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.276+0000 7f26be42b700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b8065590 con 0x7f26b80b3290 2022-06-17T13:58:09.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.276+0000 7f26bd429700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26b8063730 0x7f26b8066350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.276+0000 7f26bd429700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26b8063730 0x7f26b8066350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41284/0 (socket says 172.21.15.36:41284) 2022-06-17T13:58:09.299 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.276+0000 7f26bd429700 1 -- 172.21.15.36:0/3891097136 learned_addr learned my addr 172.21.15.36:0/3891097136 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:09.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.276+0000 7f26bdc2a700 1 --2- 172.21.15.36:0/3891097136 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26b80b3290 0x7f26b81533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.300 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.276+0000 7f26bcc28700 1 --2- 172.21.15.36:0/3891097136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26b80b2430 0x7f26b805fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.301 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.276+0000 7f26bd429700 1 -- 172.21.15.36:0/3891097136 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26b80b3290 msgr2=0x7f26b81533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.302 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.276+0000 7f26bd429700 1 --2- 172.21.15.36:0/3891097136 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26b80b3290 0x7f26b81533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.276+0000 7f26bd429700 1 -- 172.21.15.36:0/3891097136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26b80b2430 msgr2=0x7f26b805fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.303 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.276+0000 7f26bd429700 1 --2- 172.21.15.36:0/3891097136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26b80b2430 0x7f26b805fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.276+0000 7f26bd429700 1 -- 172.21.15.36:0/3891097136 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a800b180 con 0x7f26b8063730 2022-06-17T13:58:09.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.276+0000 7f26bcc28700 1 --2- 172.21.15.36:0/3891097136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26b80b2430 0x7f26b805fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T13:58:09.304 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.276+0000 7f26bd429700 1 --2- 172.21.15.36:0/3891097136 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26b8063730 0x7f26b8066350 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 rx=0x7f26b4002f70 tx=0x7f26b4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:09.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.276+0000 7f26a67fc700 1 -- 172.21.15.36:0/3891097136 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26b400dea0 con 0x7f26b8063730 2022-06-17T13:58:09.305 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.276+0000 7f26a67fc700 1 -- 172.21.15.36:0/3891097136 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b401e400 con 0x7f26b8063730 2022-06-17T13:58:09.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.276+0000 7f26be42b700 1 -- 172.21.15.36:0/3891097136 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b400b180 con 0x7f26b8063730 2022-06-17T13:58:09.306 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.276+0000 7f26a67fc700 1 -- 172.21.15.36:0/3891097136 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26b401e760 con 0x7f26b8063730 2022-06-17T13:58:09.307 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.276+0000 7f26be42b700 1 -- 172.21.15.36:0/3891097136 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b81440a0 con 0x7f26b8063730 2022-06-17T13:58:09.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.280+0000 7f26be42b700 1 -- 172.21.15.36:0/3891097136 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f26b8065cd0 con 0x7f26b8063730 2022-06-17T13:58:09.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.280+0000 7f26a67fc700 1 -- 172.21.15.36:0/3891097136 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f26b4025030 con 0x7f26b8063730 2022-06-17T13:58:09.308 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.280+0000 7f26a67fc700 1 --2- 172.21.15.36:0/3891097136 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26980539d0 0x7f2698055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.280+0000 7f26a67fc700 1 -- 172.21.15.36:0/3891097136 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f26b406cdd0 con 0x7f26b8063730 2022-06-17T13:58:09.309 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.280+0000 7f26a67fc700 1 --2- 172.21.15.36:0/3891097136 >> [v2:172.21.15.36:6818/22646,v1:172.21.15.36:6819/22646] conn(0x7f269805a660 0x7f269805cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.280+0000 7f26a67fc700 1 -- 172.21.15.36:0/3891097136 --> [v2:172.21.15.36:6818/22646,v1:172.21.15.36:6819/22646] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f269805d180 con 0x7f269805a660 2022-06-17T13:58:09.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.280+0000 7f26a67fc700 1 -- 172.21.15.36:0/3891097136 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26b4070020 con 0x7f26b8063730 2022-06-17T13:58:09.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.280+0000 7f26bcc28700 1 --2- 172.21.15.36:0/3891097136 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26980539d0 0x7f2698055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.280+0000 7f26bdc2a700 1 --2- 172.21.15.36:0/3891097136 >> [v2:172.21.15.36:6818/22646,v1:172.21.15.36:6819/22646] conn(0x7f269805a660 0x7f269805cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.280+0000 7f26bcc28700 1 --2- 172.21.15.36:0/3891097136 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26980539d0 0x7f2698055e90 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f26a800b000 tx=0x7f26a8008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:09.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.280+0000 7f26bdc2a700 1 --2- 172.21.15.36:0/3891097136 >> [v2:172.21.15.36:6818/22646,v1:172.21.15.36:6819/22646] conn(0x7f269805a660 0x7f269805cb00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:09.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.284+0000 7f26a67fc700 1 -- 172.21.15.36:0/3891097136 <== osd.3 v2:172.21.15.36:6818/22646 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f269805d180 con 0x7f269805a660 2022-06-17T13:58:09.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.300+0000 7f26be42b700 1 -- 172.21.15.36:0/3891097136 --> [v2:172.21.15.36:6818/22646,v1:172.21.15.36:6819/22646] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f26b81432c0 con 0x7f269805a660 2022-06-17T13:58:09.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.304+0000 7f26a67fc700 1 -- 172.21.15.36:0/3891097136 <== osd.3 v2:172.21.15.36:6818/22646 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f26b81432c0 con 0x7f269805a660 2022-06-17T13:58:09.317 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T13:58:09.318 INFO:teuthology.orchestra.run.smithi036.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-06-17T13:58:09.318 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T13:58:09.322 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.304+0000 7f26be42b700 1 -- 172.21.15.36:0/3891097136 >> [v2:172.21.15.36:6818/22646,v1:172.21.15.36:6819/22646] conn(0x7f269805a660 msgr2=0x7f269805cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.304+0000 7f26be42b700 1 --2- 172.21.15.36:0/3891097136 >> [v2:172.21.15.36:6818/22646,v1:172.21.15.36:6819/22646] conn(0x7f269805a660 0x7f269805cb00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.323 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.304+0000 7f26be42b700 1 -- 172.21.15.36:0/3891097136 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26980539d0 msgr2=0x7f2698055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.324 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.304+0000 7f26be42b700 1 --2- 172.21.15.36:0/3891097136 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f26980539d0 0x7f2698055e90 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f26a800b000 tx=0x7f26a8008040).stop 2022-06-17T13:58:09.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.304+0000 7f26be42b700 1 -- 172.21.15.36:0/3891097136 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26b8063730 msgr2=0x7f26b8066350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.325 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.304+0000 7f26be42b700 1 --2- 172.21.15.36:0/3891097136 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f26b8063730 0x7f26b8066350 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 rx=0x7f26b4002f70 tx=0x7f26b4014000).stop 2022-06-17T13:58:09.326 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.304+0000 7f26bdc2a700 1 -- 172.21.15.36:0/3891097136 reap_dead start 2022-06-17T13:58:09.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.304+0000 7f26be42b700 1 -- 172.21.15.36:0/3891097136 shutdown_connections 2022-06-17T13:58:09.327 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.304+0000 7f26be42b700 1 -- 172.21.15.36:0/3891097136 >> 172.21.15.36:0/3891097136 conn(0x7f26b8060880 msgr2=0x7f26b805c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:09.328 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.304+0000 7f26be42b700 1 -- 172.21.15.36:0/3891097136 shutdown_connections 2022-06-17T13:58:09.329 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.304+0000 7f26be42b700 1 -- 172.21.15.36:0/3891097136 wait complete. 2022-06-17T13:58:09.330 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2022-06-17T13:58:09.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.440+0000 7f25ada1f700 1 Processor -- start 2022-06-17T13:58:09.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.440+0000 7f25ada1f700 1 -- start start 2022-06-17T13:58:09.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25ada1f700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f25a8060780 0x7f25a8060b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25ada1f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25a80afec0 0x7f25a80b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25ada1f700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a8143990 con 0x7f25a81434f0 2022-06-17T13:58:09.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25ada1f700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a80b1a80 con 0x7f25a8060780 2022-06-17T13:58:09.449 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25ada1f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a80b1d20 con 0x7f25a80afec0 2022-06-17T13:58:09.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f259ffff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f25a81434f0 0x7f25a80b07c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58824/0 (socket says 172.21.15.36:58824) 2022-06-17T13:58:09.450 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f259ffff700 1 -- 172.21.15.36:0/2764328205 learned_addr learned my addr 172.21.15.36:0/2764328205 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T13:58:09.451 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25a77fe700 1 --2- 172.21.15.36:0/2764328205 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25a80afec0 0x7f25a80b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25a6ffd700 1 --2- 172.21.15.36:0/2764328205 >> v2:172.21.15.70:3300/0 conn(0x7f25a8060780 0x7f25a8060b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25a67fc700 1 -- 172.21.15.36:0/2764328205 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3164778870 0 0) 0x7f25a8143990 con 0x7f25a81434f0 2022-06-17T13:58:09.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25a67fc700 1 -- 172.21.15.36:0/2764328205 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2588003260 con 0x7f25a81434f0 2022-06-17T13:58:09.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25a77fe700 1 -- 172.21.15.36:0/2764328205 >> v2:172.21.15.70:3300/0 conn(0x7f25a8060780 msgr2=0x7f25a8060b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T13:58:09.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25a77fe700 1 --2- 172.21.15.36:0/2764328205 >> v2:172.21.15.70:3300/0 conn(0x7f25a8060780 0x7f25a8060b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25a77fe700 1 -- 172.21.15.36:0/2764328205 >> v1:172.21.15.36:6789/0 conn(0x7f25a81434f0 legacy=0x7f25a80b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25a77fe700 1 -- 172.21.15.36:0/2764328205 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a80b2640 con 0x7f25a80afec0 2022-06-17T13:58:09.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25a77fe700 1 --2- 172.21.15.36:0/2764328205 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25a80afec0 0x7f25a80b0ed0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f25a00032f0 tx=0x7f25a00091d0).ready entity=mon.2 client_cookie=4a9999e52a27fe50 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:09.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25a67fc700 1 -- 172.21.15.36:0/2764328205 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25a000f8a0 con 0x7f25a80afec0 2022-06-17T13:58:09.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25a67fc700 1 -- 172.21.15.36:0/2764328205 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a000fa00 con 0x7f25a80afec0 2022-06-17T13:58:09.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25a67fc700 1 -- 172.21.15.36:0/2764328205 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25a000fd60 con 0x7f25a80afec0 2022-06-17T13:58:09.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25ada1f700 1 -- 172.21.15.36:0/2764328205 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25a80afec0 msgr2=0x7f25a80b0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25ada1f700 1 --2- 172.21.15.36:0/2764328205 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25a80afec0 0x7f25a80b0ed0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f25a00032f0 tx=0x7f25a00091d0).stop 2022-06-17T13:58:09.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25ada1f700 1 -- 172.21.15.36:0/2764328205 shutdown_connections 2022-06-17T13:58:09.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25ada1f700 1 --2- 172.21.15.36:0/2764328205 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25a80afec0 0x7f25a80b0ed0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25ada1f700 1 --2- 172.21.15.36:0/2764328205 >> v2:172.21.15.70:3300/0 conn(0x7f25a8060780 0x7f25a8060b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25ada1f700 1 -- 172.21.15.36:0/2764328205 >> 172.21.15.36:0/2764328205 conn(0x7f25a80b6870 msgr2=0x7f25a80b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:09.460 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25ada1f700 1 -- 172.21.15.36:0/2764328205 shutdown_connections 2022-06-17T13:58:09.461 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.444+0000 7f25ada1f700 1 -- 172.21.15.36:0/2764328205 wait complete. 2022-06-17T13:58:09.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.448+0000 7f25ada1f700 1 Processor -- start 2022-06-17T13:58:09.465 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.448+0000 7f25ada1f700 1 -- start start 2022-06-17T13:58:09.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.448+0000 7f25ada1f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f25a8060780 0x7f25a8063130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.448+0000 7f25ada1f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25a80afec0 0x7f25a8144880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.466 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.448+0000 7f25ada1f700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25a81434f0 0x7f25a8144d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.448+0000 7f25ada1f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a8063f20 con 0x7f25a8060780 2022-06-17T13:58:09.467 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.448+0000 7f25ada1f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a8065480 con 0x7f25a80afec0 2022-06-17T13:58:09.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.448+0000 7f25ada1f700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a81563b0 con 0x7f25a81434f0 2022-06-17T13:58:09.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.448+0000 7f25a6ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f25a8060780 0x7f25a8063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.468 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.448+0000 7f25a6ffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f25a8060780 0x7f25a8063130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41300/0 (socket says 172.21.15.36:41300) 2022-06-17T13:58:09.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.448+0000 7f25a6ffd700 1 -- 172.21.15.36:0/3560018995 learned_addr learned my addr 172.21.15.36:0/3560018995 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:09.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.448+0000 7f259ffff700 1 --2- 172.21.15.36:0/3560018995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25a80afec0 0x7f25a8144880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.469 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.448+0000 7f25a77fe700 1 --2- 172.21.15.36:0/3560018995 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25a81434f0 0x7f25a8144d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.448+0000 7f25a6ffd700 1 -- 172.21.15.36:0/3560018995 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25a81434f0 msgr2=0x7f25a8144d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.448+0000 7f25a6ffd700 1 --2- 172.21.15.36:0/3560018995 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25a81434f0 0x7f25a8144d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.470 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.448+0000 7f25a6ffd700 1 -- 172.21.15.36:0/3560018995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25a80afec0 msgr2=0x7f25a8144880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.448+0000 7f25a6ffd700 1 --2- 172.21.15.36:0/3560018995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25a80afec0 0x7f25a8144880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.448+0000 7f25a6ffd700 1 -- 172.21.15.36:0/3560018995 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a0009470 con 0x7f25a8060780 2022-06-17T13:58:09.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.448+0000 7f25a6ffd700 1 --2- 172.21.15.36:0/3560018995 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f25a8060780 0x7f25a8063130 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 rx=0x7f259800c000 tx=0x7f2598006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:09.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.448+0000 7f25a4ff9700 1 -- 172.21.15.36:0/3560018995 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2598010780 con 0x7f25a8060780 2022-06-17T13:58:09.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.448+0000 7f25a4ff9700 1 -- 172.21.15.36:0/3560018995 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25980108e0 con 0x7f25a8060780 2022-06-17T13:58:09.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.448+0000 7f25a4ff9700 1 -- 172.21.15.36:0/3560018995 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2598010c40 con 0x7f25a8060780 2022-06-17T13:58:09.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.448+0000 7f25ada1f700 1 -- 172.21.15.36:0/3560018995 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a8156650 con 0x7f25a8060780 2022-06-17T13:58:09.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.448+0000 7f25ada1f700 1 -- 172.21.15.36:0/3560018995 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a8065170 con 0x7f25a8060780 2022-06-17T13:58:09.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.452+0000 7f25ada1f700 1 -- 172.21.15.36:0/3560018995 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a8066c00 con 0x7f25a8060780 2022-06-17T13:58:09.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.452+0000 7f25a4ff9700 1 -- 172.21.15.36:0/3560018995 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f259801c460 con 0x7f25a8060780 2022-06-17T13:58:09.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.452+0000 7f25a4ff9700 1 --2- 172.21.15.36:0/3560018995 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f258c053ad0 0x7f258c055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.452+0000 7f25a4ff9700 1 -- 172.21.15.36:0/3560018995 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f259806c250 con 0x7f25a8060780 2022-06-17T13:58:09.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.452+0000 7f259ffff700 1 --2- 172.21.15.36:0/3560018995 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f258c053ad0 0x7f258c055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.452+0000 7f259ffff700 1 --2- 172.21.15.36:0/3560018995 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f258c053ad0 0x7f258c055f90 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7f259000d000 tx=0x7f2590009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:09.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.460+0000 7f25a4ff9700 1 -- 172.21.15.36:0/3560018995 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2598043230 con 0x7f25a8060780 2022-06-17T13:58:09.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.588+0000 7f25ada1f700 1 -- 172.21.15.36:0/3560018995 --> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f25a8142c00 con 0x7f258c053ad0 2022-06-17T13:58:09.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.588+0000 7f25a4ff9700 1 -- 172.21.15.36:0/3560018995 <== mgr.4104 v2:172.21.15.36:6800/22419 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f25a8142c00 con 0x7f258c053ad0 2022-06-17T13:58:09.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.588+0000 7f25ada1f700 1 -- 172.21.15.36:0/3560018995 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f258c053ad0 msgr2=0x7f258c055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.588+0000 7f25ada1f700 1 --2- 172.21.15.36:0/3560018995 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f258c053ad0 0x7f258c055f90 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7f259000d000 tx=0x7f2590009040).stop 2022-06-17T13:58:09.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.588+0000 7f25ada1f700 1 -- 172.21.15.36:0/3560018995 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f25a8060780 msgr2=0x7f25a8063130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.588+0000 7f25ada1f700 1 --2- 172.21.15.36:0/3560018995 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f25a8060780 0x7f25a8063130 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 rx=0x7f259800c000 tx=0x7f2598006df0).stop 2022-06-17T13:58:09.591 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.588+0000 7f25ada1f700 1 -- 172.21.15.36:0/3560018995 shutdown_connections 2022-06-17T13:58:09.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.588+0000 7f25ada1f700 1 --2- 172.21.15.36:0/3560018995 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f258c053ad0 0x7f258c055f90 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.588+0000 7f25ada1f700 1 --2- 172.21.15.36:0/3560018995 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25a81434f0 0x7f25a8144d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.592 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.588+0000 7f25ada1f700 1 --2- 172.21.15.36:0/3560018995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25a80afec0 0x7f25a8144880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.588+0000 7f25ada1f700 1 --2- 172.21.15.36:0/3560018995 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f25a8060780 0x7f25a8063130 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.593 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.588+0000 7f25ada1f700 1 -- 172.21.15.36:0/3560018995 >> 172.21.15.36:0/3560018995 conn(0x7f25a80b6870 msgr2=0x7f25a80b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:09.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.588+0000 7f25ada1f700 1 -- 172.21.15.36:0/3560018995 shutdown_connections 2022-06-17T13:58:09.594 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.588+0000 7f25ada1f700 1 -- 172.21.15.36:0/3560018995 wait complete. 2022-06-17T13:58:09.594 INFO:teuthology.orchestra.run.smithi036.stderr:instructed osd(s) 3 to deep-scrub 2022-06-17T13:58:09.606 INFO:tasks.ceph:Scrubbing osd.4 2022-06-17T13:58:09.607 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2022-06-17T13:58:09.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.716+0000 7f449b59e700 1 Processor -- start 2022-06-17T13:58:09.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.716+0000 7f449b59e700 1 -- start start 2022-06-17T13:58:09.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.716+0000 7f449b59e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f449c0b45c0 0x7f449c0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.721 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.716+0000 7f449b59e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f449c0b4160 con 0x7f449c0b4f00 2022-06-17T13:58:09.722 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.716+0000 7f449b59e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f449c0b9c40 con 0x7f449c0b9860 2022-06-17T13:58:09.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.720+0000 7f449b59e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f449c0627d0 con 0x7f449c0b45c0 2022-06-17T13:58:09.723 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.720+0000 7f449ad9d700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f449c0b4f00 0x7f449c0b5dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58842/0 (socket says 172.21.15.36:58842) 2022-06-17T13:58:09.724 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.720+0000 7f449ad9d700 1 -- 172.21.15.36:0/1090188928 learned_addr learned my addr 172.21.15.36:0/1090188928 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T13:58:09.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.720+0000 7f4499d9b700 1 --2- 172.21.15.36:0/1090188928 >> v2:172.21.15.36:3300/0 conn(0x7f449c0b45c0 0x7f449c0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.720+0000 7f449959a700 1 -- 172.21.15.36:0/1090188928 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3072965406 0 0) 0x7f449c0b4160 con 0x7f449c0b4f00 2022-06-17T13:58:09.725 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.720+0000 7f449959a700 1 -- 172.21.15.36:0/1090188928 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f447c003260 con 0x7f449c0b4f00 2022-06-17T13:58:09.726 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.720+0000 7f449959a700 1 -- 172.21.15.36:0/1090188928 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1243959774 0 0) 0x7f447c003260 con 0x7f449c0b4f00 2022-06-17T13:58:09.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.720+0000 7f4499d9b700 1 -- 172.21.15.36:0/1090188928 >> v1:172.21.15.70:6789/0 conn(0x7f449c0b9860 legacy=0x7f449c143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.720+0000 7f4499d9b700 1 -- 172.21.15.36:0/1090188928 >> v1:172.21.15.36:6789/0 conn(0x7f449c0b4f00 legacy=0x7f449c0b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.727 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.720+0000 7f4499d9b700 1 -- 172.21.15.36:0/1090188928 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449c062a70 con 0x7f449c0b45c0 2022-06-17T13:58:09.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.720+0000 7f4499d9b700 1 --2- 172.21.15.36:0/1090188928 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f449c0b45c0 0x7f449c0b4a10 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x7f449c0ba5d0 tx=0x7f448c006df0).ready entity=mon.0 client_cookie=c209b6e30f685e24 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:09.728 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.720+0000 7f449959a700 1 -- 172.21.15.36:0/1090188928 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4494001a00 con 0x7f449c0b45c0 2022-06-17T13:58:09.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.720+0000 7f449959a700 1 -- 172.21.15.36:0/1090188928 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448c000e90 con 0x7f449c0b45c0 2022-06-17T13:58:09.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.720+0000 7f449959a700 1 -- 172.21.15.36:0/1090188928 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f448c00d4f0 con 0x7f449c0b45c0 2022-06-17T13:58:09.729 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.720+0000 7f449b59e700 1 -- 172.21.15.36:0/1090188928 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f449c0b45c0 msgr2=0x7f449c0b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.720+0000 7f449b59e700 1 --2- 172.21.15.36:0/1090188928 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f449c0b45c0 0x7f449c0b4a10 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x7f449c0ba5d0 tx=0x7f448c006df0).stop 2022-06-17T13:58:09.730 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.720+0000 7f449b59e700 1 -- 172.21.15.36:0/1090188928 shutdown_connections 2022-06-17T13:58:09.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.720+0000 7f449b59e700 1 --2- 172.21.15.36:0/1090188928 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f449c0b45c0 0x7f449c0b4a10 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.720+0000 7f449b59e700 1 -- 172.21.15.36:0/1090188928 >> 172.21.15.36:0/1090188928 conn(0x7f449c05c550 msgr2=0x7f449c060990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:09.731 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.720+0000 7f449b59e700 1 -- 172.21.15.36:0/1090188928 shutdown_connections 2022-06-17T13:58:09.732 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.720+0000 7f449b59e700 1 -- 172.21.15.36:0/1090188928 wait complete. 2022-06-17T13:58:09.736 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.720+0000 7f449b59e700 1 Processor -- start 2022-06-17T13:58:09.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.724+0000 7f449b59e700 1 -- start start 2022-06-17T13:58:09.737 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.724+0000 7f449b59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f449c0b45c0 0x7f449c146a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.724+0000 7f449b59e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f449c0b4f00 0x7f449c155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.738 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.724+0000 7f449b59e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f449c0b9860 0x7f449c156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.724+0000 7f449b59e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449c13eeb0 con 0x7f449c0b45c0 2022-06-17T13:58:09.739 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.724+0000 7f449b59e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449c1556a0 con 0x7f449c0b9860 2022-06-17T13:58:09.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.724+0000 7f449b59e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449c0afc00 con 0x7f449c0b4f00 2022-06-17T13:58:09.740 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.724+0000 7f449a59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f449c0b45c0 0x7f449c146a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.724+0000 7f449a59c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f449c0b45c0 0x7f449c146a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41314/0 (socket says 172.21.15.36:41314) 2022-06-17T13:58:09.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.724+0000 7f449a59c700 1 -- 172.21.15.36:0/2068283367 learned_addr learned my addr 172.21.15.36:0/2068283367 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:09.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.724+0000 7f4499d9b700 1 --2- 172.21.15.36:0/2068283367 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f449c0b4f00 0x7f449c155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.724+0000 7f449ad9d700 1 --2- 172.21.15.36:0/2068283367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f449c0b9860 0x7f449c156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.724+0000 7f449a59c700 1 -- 172.21.15.36:0/2068283367 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f449c0b4f00 msgr2=0x7f449c155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.754 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.724+0000 7f449a59c700 1 --2- 172.21.15.36:0/2068283367 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f449c0b4f00 0x7f449c155b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.724+0000 7f449a59c700 1 -- 172.21.15.36:0/2068283367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f449c0b9860 msgr2=0x7f449c156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.755 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.724+0000 7f449a59c700 1 --2- 172.21.15.36:0/2068283367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f449c0b9860 0x7f449c156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.724+0000 7f449a59c700 1 -- 172.21.15.36:0/2068283367 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448c00b180 con 0x7f449c0b45c0 2022-06-17T13:58:09.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.724+0000 7f449a59c700 1 --2- 172.21.15.36:0/2068283367 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f449c0b45c0 0x7f449c146a40 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 rx=0x7f449c062280 tx=0x7f4484007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:09.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.724+0000 7f44937fe700 1 -- 172.21.15.36:0/2068283367 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4484012400 con 0x7f449c0b45c0 2022-06-17T13:58:09.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.724+0000 7f44937fe700 1 -- 172.21.15.36:0/2068283367 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4484012560 con 0x7f449c0b45c0 2022-06-17T13:58:09.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.724+0000 7f449b59e700 1 -- 172.21.15.36:0/2068283367 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f449c0aff00 con 0x7f449c0b45c0 2022-06-17T13:58:09.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.724+0000 7f44937fe700 1 -- 172.21.15.36:0/2068283367 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44840128c0 con 0x7f449c0b45c0 2022-06-17T13:58:09.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.724+0000 7f449b59e700 1 -- 172.21.15.36:0/2068283367 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f449c0af840 con 0x7f449c0b45c0 2022-06-17T13:58:09.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.728+0000 7f449b59e700 1 -- 172.21.15.36:0/2068283367 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f449c0b0340 con 0x7f449c0b45c0 2022-06-17T13:58:09.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.728+0000 7f44937fe700 1 -- 172.21.15.36:0/2068283367 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f4484019030 con 0x7f449c0b45c0 2022-06-17T13:58:09.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.728+0000 7f44937fe700 1 --2- 172.21.15.36:0/2068283367 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4480053a60 0x7f4480055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.728+0000 7f44937fe700 1 -- 172.21.15.36:0/2068283367 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f448406b610 con 0x7f449c0b45c0 2022-06-17T13:58:09.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.728+0000 7f4499d9b700 1 --2- 172.21.15.36:0/2068283367 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4480053a60 0x7f4480055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.728+0000 7f44937fe700 1 --2- 172.21.15.36:0/2068283367 >> [v2:172.21.15.70:6808/22812,v1:172.21.15.70:6809/22812] conn(0x7f448005a6f0 0x7f448005cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.728+0000 7f44937fe700 1 -- 172.21.15.36:0/2068283367 --> [v2:172.21.15.70:6808/22812,v1:172.21.15.70:6809/22812] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f448005d170 con 0x7f448005a6f0 2022-06-17T13:58:09.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.728+0000 7f44937fe700 1 -- 172.21.15.36:0/2068283367 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) v2 ==== 24+0+0 (secure 0 0 0) 0x7f448406b8e0 con 0x7f449c0b45c0 2022-06-17T13:58:09.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.728+0000 7f449ad9d700 1 --2- 172.21.15.36:0/2068283367 >> [v2:172.21.15.70:6808/22812,v1:172.21.15.70:6809/22812] conn(0x7f448005a6f0 0x7f448005cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.728+0000 7f4499d9b700 1 --2- 172.21.15.36:0/2068283367 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4480053a60 0x7f4480055f20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7f448c01a000 tx=0x7f448c014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:09.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.728+0000 7f449ad9d700 1 --2- 172.21.15.36:0/2068283367 >> [v2:172.21.15.70:6808/22812,v1:172.21.15.70:6809/22812] conn(0x7f448005a6f0 0x7f448005cb90 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:09.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.732+0000 7f44937fe700 1 -- 172.21.15.36:0/2068283367 <== osd.4 v2:172.21.15.70:6808/22812 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f448005d170 con 0x7f448005a6f0 2022-06-17T13:58:09.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.748+0000 7f449b59e700 1 -- 172.21.15.36:0/2068283367 --> [v2:172.21.15.70:6808/22812,v1:172.21.15.70:6809/22812] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f449c062ee0 con 0x7f448005a6f0 2022-06-17T13:58:09.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.748+0000 7f44937fe700 1 -- 172.21.15.36:0/2068283367 <== osd.4 v2:172.21.15.70:6808/22812 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f449c062ee0 con 0x7f448005a6f0 2022-06-17T13:58:09.772 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T13:58:09.772 INFO:teuthology.orchestra.run.smithi036.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-06-17T13:58:09.773 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T13:58:09.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.748+0000 7f449b59e700 1 -- 172.21.15.36:0/2068283367 >> [v2:172.21.15.70:6808/22812,v1:172.21.15.70:6809/22812] conn(0x7f448005a6f0 msgr2=0x7f448005cb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.748+0000 7f449b59e700 1 --2- 172.21.15.36:0/2068283367 >> [v2:172.21.15.70:6808/22812,v1:172.21.15.70:6809/22812] conn(0x7f448005a6f0 0x7f448005cb90 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.748+0000 7f449b59e700 1 -- 172.21.15.36:0/2068283367 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4480053a60 msgr2=0x7f4480055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.748+0000 7f449b59e700 1 --2- 172.21.15.36:0/2068283367 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f4480053a60 0x7f4480055f20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7f448c01a000 tx=0x7f448c014000).stop 2022-06-17T13:58:09.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.748+0000 7f449b59e700 1 -- 172.21.15.36:0/2068283367 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f449c0b45c0 msgr2=0x7f449c146a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.748+0000 7f449b59e700 1 --2- 172.21.15.36:0/2068283367 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f449c0b45c0 0x7f449c146a40 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 rx=0x7f449c062280 tx=0x7f4484007c50).stop 2022-06-17T13:58:09.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.748+0000 7f449ad9d700 1 -- 172.21.15.36:0/2068283367 reap_dead start 2022-06-17T13:58:09.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.748+0000 7f449b59e700 1 -- 172.21.15.36:0/2068283367 shutdown_connections 2022-06-17T13:58:09.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.748+0000 7f449b59e700 1 -- 172.21.15.36:0/2068283367 >> 172.21.15.36:0/2068283367 conn(0x7f449c05c550 msgr2=0x7f449c12c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:09.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.748+0000 7f449b59e700 1 -- 172.21.15.36:0/2068283367 shutdown_connections 2022-06-17T13:58:09.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.748+0000 7f449b59e700 1 -- 172.21.15.36:0/2068283367 wait complete. 2022-06-17T13:58:09.783 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2022-06-17T13:58:09.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.896+0000 7fe1983e4700 1 Processor -- start 2022-06-17T13:58:09.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.896+0000 7fe1983e4700 1 -- start start 2022-06-17T13:58:09.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe1983e4700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe1900ae270 0x7fe1900b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe1983e4700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe19013fde0 0x7fe190140200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe1983e4700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1900afc30 con 0x7fe1900ad9c0 2022-06-17T13:58:09.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe1983e4700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1900b0cb0 con 0x7fe19013fde0 2022-06-17T13:58:09.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe1983e4700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1900b3eb0 con 0x7fe1900ae270 2022-06-17T13:58:09.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe196981700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe19013fde0 0x7fe190140200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe196981700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe19013fde0 0x7fe190140200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41328/0 (socket says 172.21.15.36:41328) 2022-06-17T13:58:09.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe196981700 1 -- 172.21.15.36:0/2728533295 learned_addr learned my addr 172.21.15.36:0/2728533295 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:09.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe196180700 1 --2- 172.21.15.36:0/2728533295 >> v2:172.21.15.70:3300/0 conn(0x7fe1900ae270 0x7fe1900b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe196981700 1 -- 172.21.15.36:0/2728533295 >> v2:172.21.15.70:3300/0 conn(0x7fe1900ae270 msgr2=0x7fe1900b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T13:58:09.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe196981700 1 --2- 172.21.15.36:0/2728533295 >> v2:172.21.15.70:3300/0 conn(0x7fe1900ae270 0x7fe1900b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe196981700 1 -- 172.21.15.36:0/2728533295 >> v1:172.21.15.70:6789/0 conn(0x7fe1900ad9c0 legacy=0x7fe1900b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe196981700 1 -- 172.21.15.36:0/2728533295 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1900ae650 con 0x7fe19013fde0 2022-06-17T13:58:09.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe196981700 1 --2- 172.21.15.36:0/2728533295 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe19013fde0 0x7fe190140200 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 rx=0x7fe1900b3ff0 tx=0x7fe188009290).ready entity=mon.0 client_cookie=d247e928151c45bf server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:09.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe19517e700 1 -- 172.21.15.36:0/2728533295 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe18800b830 con 0x7fe19013fde0 2022-06-17T13:58:09.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe19517e700 1 -- 172.21.15.36:0/2728533295 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18800b990 con 0x7fe19013fde0 2022-06-17T13:58:09.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe19517e700 1 -- 172.21.15.36:0/2728533295 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe18800bb00 con 0x7fe19013fde0 2022-06-17T13:58:09.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe1983e4700 1 -- 172.21.15.36:0/2728533295 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe19013fde0 msgr2=0x7fe190140200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe1983e4700 1 --2- 172.21.15.36:0/2728533295 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe19013fde0 0x7fe190140200 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 rx=0x7fe1900b3ff0 tx=0x7fe188009290).stop 2022-06-17T13:58:09.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe1983e4700 1 -- 172.21.15.36:0/2728533295 shutdown_connections 2022-06-17T13:58:09.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe1983e4700 1 --2- 172.21.15.36:0/2728533295 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe19013fde0 0x7fe190140200 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe1983e4700 1 --2- 172.21.15.36:0/2728533295 >> v2:172.21.15.70:3300/0 conn(0x7fe1900ae270 0x7fe1900b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe1983e4700 1 -- 172.21.15.36:0/2728533295 >> 172.21.15.36:0/2728533295 conn(0x7fe19005c550 msgr2=0x7fe19005c930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:09.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe1983e4700 1 -- 172.21.15.36:0/2728533295 shutdown_connections 2022-06-17T13:58:09.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.900+0000 7fe1983e4700 1 -- 172.21.15.36:0/2728533295 wait complete. 2022-06-17T13:58:09.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.904+0000 7fe1983e4700 1 Processor -- start 2022-06-17T13:58:09.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.904+0000 7fe1983e4700 1 -- start start 2022-06-17T13:58:09.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.904+0000 7fe1983e4700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1900ad9c0 0x7fe190138480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.904+0000 7fe1983e4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1900ae270 0x7fe190063630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.904+0000 7fe1983e4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe19013fde0 0x7fe190063b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.904+0000 7fe1983e4700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1901394f0 con 0x7fe19013fde0 2022-06-17T13:58:09.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.904+0000 7fe1983e4700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe190139790 con 0x7fe1900ae270 2022-06-17T13:58:09.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.904+0000 7fe1983e4700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe190136ba0 con 0x7fe1900ad9c0 2022-06-17T13:58:09.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.904+0000 7fe196981700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe19013fde0 0x7fe190063b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.916 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.904+0000 7fe196981700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe19013fde0 0x7fe190063b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41334/0 (socket says 172.21.15.36:41334) 2022-06-17T13:58:09.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.904+0000 7fe196981700 1 -- 172.21.15.36:0/656125054 learned_addr learned my addr 172.21.15.36:0/656125054 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:09.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.904+0000 7fe19597f700 1 --2- 172.21.15.36:0/656125054 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1900ae270 0x7fe190063630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.917 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.904+0000 7fe196180700 1 --2- 172.21.15.36:0/656125054 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1900ad9c0 0x7fe190138480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.904+0000 7fe196981700 1 -- 172.21.15.36:0/656125054 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1900ad9c0 msgr2=0x7fe190138480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.904+0000 7fe196981700 1 --2- 172.21.15.36:0/656125054 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1900ad9c0 0x7fe190138480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.918 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.904+0000 7fe196981700 1 -- 172.21.15.36:0/656125054 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1900ae270 msgr2=0x7fe190063630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:09.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.904+0000 7fe196981700 1 --2- 172.21.15.36:0/656125054 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1900ae270 0x7fe190063630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:09.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.904+0000 7fe196981700 1 -- 172.21.15.36:0/656125054 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe188009530 con 0x7fe19013fde0 2022-06-17T13:58:09.919 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.904+0000 7fe19597f700 1 --2- 172.21.15.36:0/656125054 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1900ae270 0x7fe190063630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T13:58:09.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.904+0000 7fe196981700 1 --2- 172.21.15.36:0/656125054 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe19013fde0 0x7fe190063b20 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 rx=0x7fe1900b3ff0 tx=0x7fe188016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:09.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.904+0000 7fe1837fe700 1 -- 172.21.15.36:0/656125054 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe188021630 con 0x7fe19013fde0 2022-06-17T13:58:09.920 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.904+0000 7fe1837fe700 1 -- 172.21.15.36:0/656125054 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe188021790 con 0x7fe19013fde0 2022-06-17T13:58:09.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.904+0000 7fe1983e4700 1 -- 172.21.15.36:0/656125054 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1901375f0 con 0x7fe19013fde0 2022-06-17T13:58:09.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.904+0000 7fe1837fe700 1 -- 172.21.15.36:0/656125054 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe188021af0 con 0x7fe19013fde0 2022-06-17T13:58:09.921 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.908+0000 7fe1983e4700 1 -- 172.21.15.36:0/656125054 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1900641b0 con 0x7fe19013fde0 2022-06-17T13:58:09.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.908+0000 7fe1983e4700 1 -- 172.21.15.36:0/656125054 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe190136d60 con 0x7fe19013fde0 2022-06-17T13:58:09.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.908+0000 7fe1837fe700 1 -- 172.21.15.36:0/656125054 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe188027020 con 0x7fe19013fde0 2022-06-17T13:58:09.922 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.908+0000 7fe1837fe700 1 --2- 172.21.15.36:0/656125054 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe170053ad0 0x7fe170055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:09.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.908+0000 7fe1837fe700 1 -- 172.21.15.36:0/656125054 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fe18806d2a0 con 0x7fe19013fde0 2022-06-17T13:58:09.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.908+0000 7fe196180700 1 --2- 172.21.15.36:0/656125054 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe170053ad0 0x7fe170055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:09.923 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.908+0000 7fe196180700 1 --2- 172.21.15.36:0/656125054 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe170053ad0 0x7fe170055f90 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7fe18c00c000 tx=0x7fe18c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:09.924 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:09.920+0000 7fe1837fe700 1 -- 172.21.15.36:0/656125054 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe1880443c0 con 0x7fe19013fde0 2022-06-17T13:58:10.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.044+0000 7fe1983e4700 1 -- 172.21.15.36:0/656125054 --> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1900afe60 con 0x7fe170053ad0 2022-06-17T13:58:10.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.048+0000 7fe1837fe700 1 -- 172.21.15.36:0/656125054 <== mgr.4104 v2:172.21.15.36:6800/22419 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe1900afe60 con 0x7fe170053ad0 2022-06-17T13:58:10.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.048+0000 7fe1983e4700 1 -- 172.21.15.36:0/656125054 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe170053ad0 msgr2=0x7fe170055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.048+0000 7fe1983e4700 1 --2- 172.21.15.36:0/656125054 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe170053ad0 0x7fe170055f90 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7fe18c00c000 tx=0x7fe18c008040).stop 2022-06-17T13:58:10.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.048+0000 7fe1983e4700 1 -- 172.21.15.36:0/656125054 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe19013fde0 msgr2=0x7fe190063b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.048+0000 7fe1983e4700 1 --2- 172.21.15.36:0/656125054 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe19013fde0 0x7fe190063b20 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 rx=0x7fe1900b3ff0 tx=0x7fe188016000).stop 2022-06-17T13:58:10.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.048+0000 7fe1983e4700 1 -- 172.21.15.36:0/656125054 shutdown_connections 2022-06-17T13:58:10.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.048+0000 7fe1983e4700 1 --2- 172.21.15.36:0/656125054 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe170053ad0 0x7fe170055f90 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.048+0000 7fe1983e4700 1 --2- 172.21.15.36:0/656125054 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe19013fde0 0x7fe190063b20 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.048+0000 7fe1983e4700 1 --2- 172.21.15.36:0/656125054 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1900ae270 0x7fe190063630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.048+0000 7fe1983e4700 1 --2- 172.21.15.36:0/656125054 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe1900ad9c0 0x7fe190138480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.048+0000 7fe1983e4700 1 -- 172.21.15.36:0/656125054 >> 172.21.15.36:0/656125054 conn(0x7fe19005c550 msgr2=0x7fe190140780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:10.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.048+0000 7fe1983e4700 1 -- 172.21.15.36:0/656125054 shutdown_connections 2022-06-17T13:58:10.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.048+0000 7fe1983e4700 1 -- 172.21.15.36:0/656125054 wait complete. 2022-06-17T13:58:10.054 INFO:teuthology.orchestra.run.smithi036.stderr:instructed osd(s) 4 to deep-scrub 2022-06-17T13:58:10.061 INFO:tasks.ceph:Scrubbing osd.5 2022-06-17T13:58:10.062 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2022-06-17T13:58:10.173 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.172+0000 7fecb23e6700 1 Processor -- start 2022-06-17T13:58:10.174 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.172+0000 7fecb23e6700 1 -- start start 2022-06-17T13:58:10.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fecb23e6700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecac063730 0x7fecac0af380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.190 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fecb23e6700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fecac0b2430 0x7fecac143760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fecb23e6700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecac063b10 con 0x7fecac0b3290 2022-06-17T13:58:10.191 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fecb23e6700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecac0ada90 con 0x7fecac0b2430 2022-06-17T13:58:10.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fecb23e6700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecac0b3d50 con 0x7fecac063730 2022-06-17T13:58:10.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fecb1be5700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fecac0b3290 0x7fecac128b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58872/0 (socket says 172.21.15.36:58872) 2022-06-17T13:58:10.192 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fecb1be5700 1 -- 172.21.15.36:0/1276322555 learned_addr learned my addr 172.21.15.36:0/1276322555 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T13:58:10.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fecb0be3700 1 --2- 172.21.15.36:0/1276322555 >> v2:172.21.15.70:3300/0 conn(0x7fecac0b2430 0x7fecac143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fecb13e4700 1 --2- 172.21.15.36:0/1276322555 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecac063730 0x7fecac0af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fec97fff700 1 -- 172.21.15.36:0/1276322555 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309799562 0 0) 0x7fecac063b10 con 0x7fecac0b3290 2022-06-17T13:58:10.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fec97fff700 1 -- 172.21.15.36:0/1276322555 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec90003260 con 0x7fecac0b3290 2022-06-17T13:58:10.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fec97fff700 1 -- 172.21.15.36:0/1276322555 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2128824323 0 0) 0x7fec90003260 con 0x7fecac0b3290 2022-06-17T13:58:10.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fec97fff700 1 -- 172.21.15.36:0/1276322555 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecac063730 msgr2=0x7fecac0af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T13:58:10.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fec97fff700 1 --2- 172.21.15.36:0/1276322555 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecac063730 0x7fecac0af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fec97fff700 1 -- 172.21.15.36:0/1276322555 >> v2:172.21.15.70:3300/0 conn(0x7fecac0b2430 msgr2=0x7fecac143760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T13:58:10.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fec97fff700 1 --2- 172.21.15.36:0/1276322555 >> v2:172.21.15.70:3300/0 conn(0x7fecac0b2430 0x7fecac143760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fec97fff700 1 -- 172.21.15.36:0/1276322555 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecac0af0e0 con 0x7fecac0b3290 2022-06-17T13:58:10.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fec97fff700 1 -- 172.21.15.36:0/1276322555 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7feca40019e0 con 0x7fecac0b3290 2022-06-17T13:58:10.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fec97fff700 1 -- 172.21.15.36:0/1276322555 >> v1:172.21.15.36:6789/0 conn(0x7fecac0b3290 legacy=0x7fecac128b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fec97fff700 1 --2- 172.21.15.36:0/1276322555 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecac063730 0x7fec90003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fec97fff700 1 --2- 172.21.15.36:0/1276322555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec90003c80 0x7fec90004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fec97fff700 1 --2- 172.21.15.36:0/1276322555 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fec900049f0 0x7fec90004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.198 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fec97fff700 1 -- 172.21.15.36:0/1276322555 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac0b3d50 con 0x7fec900049f0 2022-06-17T13:58:10.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fec97fff700 1 -- 172.21.15.36:0/1276322555 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9000b750 con 0x7fec90003c80 2022-06-17T13:58:10.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fec97fff700 1 -- 172.21.15.36:0/1276322555 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9000b890 con 0x7fecac063730 2022-06-17T13:58:10.199 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fecb0be3700 1 --2- 172.21.15.36:0/1276322555 >> v2:172.21.15.70:3300/0 conn(0x7fecac0b2430 0x7fecac143760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T13:58:10.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fecb1be5700 1 --2- 172.21.15.36:0/1276322555 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fec900049f0 0x7fec90004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fecb13e4700 1 --2- 172.21.15.36:0/1276322555 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecac063730 0x7fec90003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.200 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fecb0be3700 1 --2- 172.21.15.36:0/1276322555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec90003c80 0x7fec90004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fecb1be5700 1 -- 172.21.15.36:0/1276322555 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecac063730 msgr2=0x7fec90003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T13:58:10.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fecb1be5700 1 --2- 172.21.15.36:0/1276322555 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecac063730 0x7fec90003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.201 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fecb1be5700 1 -- 172.21.15.36:0/1276322555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec90003c80 msgr2=0x7fec90004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T13:58:10.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fecb1be5700 1 --2- 172.21.15.36:0/1276322555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec90003c80 0x7fec90004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fecb1be5700 1 -- 172.21.15.36:0/1276322555 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec90003260 con 0x7fec900049f0 2022-06-17T13:58:10.202 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fecb0be3700 1 --2- 172.21.15.36:0/1276322555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec90003c80 0x7fec90004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T13:58:10.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.176+0000 7fecb1be5700 1 --2- 172.21.15.36:0/1276322555 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fec900049f0 0x7fec90004e70 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 rx=0x7fec900054d0 tx=0x7fecac0b2db0).ready entity=mon.0 client_cookie=104560bbb48e3b0a server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:10.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.180+0000 7fec97fff700 1 -- 172.21.15.36:0/1276322555 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feca400cc40 con 0x7fec900049f0 2022-06-17T13:58:10.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.180+0000 7fec97fff700 1 -- 172.21.15.36:0/1276322555 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca400cda0 con 0x7fec900049f0 2022-06-17T13:58:10.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.180+0000 7fec97fff700 1 -- 172.21.15.36:0/1276322555 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feca4005b80 con 0x7fec900049f0 2022-06-17T13:58:10.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb23e6700 1 -- 172.21.15.36:0/1276322555 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fec900049f0 msgr2=0x7fec90004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb23e6700 1 --2- 172.21.15.36:0/1276322555 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fec900049f0 0x7fec90004e70 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 rx=0x7fec900054d0 tx=0x7fecac0b2db0).stop 2022-06-17T13:58:10.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb1be5700 1 -- 172.21.15.36:0/1276322555 reap_dead start 2022-06-17T13:58:10.205 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb23e6700 1 -- 172.21.15.36:0/1276322555 shutdown_connections 2022-06-17T13:58:10.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb23e6700 1 -- 172.21.15.36:0/1276322555 >> 172.21.15.36:0/1276322555 conn(0x7fecac060880 msgr2=0x7fecac0614b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:10.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb23e6700 1 -- 172.21.15.36:0/1276322555 shutdown_connections 2022-06-17T13:58:10.206 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb23e6700 1 -- 172.21.15.36:0/1276322555 wait complete. 2022-06-17T13:58:10.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb23e6700 1 Processor -- start 2022-06-17T13:58:10.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb23e6700 1 -- start start 2022-06-17T13:58:10.207 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb23e6700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fec900049f0 0x7fecac121010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb23e6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecac155560 0x7fecac152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb23e6700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecac1412c0 0x7fecac141740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.208 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb23e6700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac141c30 con 0x7fec900049f0 2022-06-17T13:58:10.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb23e6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac152f70 con 0x7fecac155560 2022-06-17T13:58:10.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb23e6700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecac1520a0 con 0x7fecac1412c0 2022-06-17T13:58:10.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb13e4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fec900049f0 0x7fecac121010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb13e4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fec900049f0 0x7fecac121010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41350/0 (socket says 172.21.15.36:41350) 2022-06-17T13:58:10.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb13e4700 1 -- 172.21.15.36:0/4035736543 learned_addr learned my addr 172.21.15.36:0/4035736543 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:10.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb0be3700 1 --2- 172.21.15.36:0/4035736543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecac155560 0x7fecac152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb1be5700 1 --2- 172.21.15.36:0/4035736543 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecac1412c0 0x7fecac141740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb13e4700 1 -- 172.21.15.36:0/4035736543 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecac1412c0 msgr2=0x7fecac141740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb13e4700 1 --2- 172.21.15.36:0/4035736543 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecac1412c0 0x7fecac141740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb13e4700 1 -- 172.21.15.36:0/4035736543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecac155560 msgr2=0x7fecac152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb13e4700 1 --2- 172.21.15.36:0/4035736543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecac155560 0x7fecac152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb13e4700 1 -- 172.21.15.36:0/4035736543 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecac152340 con 0x7fec900049f0 2022-06-17T13:58:10.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb0be3700 1 --2- 172.21.15.36:0/4035736543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecac155560 0x7fecac152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T13:58:10.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb13e4700 1 --2- 172.21.15.36:0/4035736543 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fec900049f0 0x7fecac121010 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7fecac152960 tx=0x7feca8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:10.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fec9effd700 1 -- 172.21.15.36:0/4035736543 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feca800f940 con 0x7fec900049f0 2022-06-17T13:58:10.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fec9effd700 1 -- 172.21.15.36:0/4035736543 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca800faa0 con 0x7fec900049f0 2022-06-17T13:58:10.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fecb23e6700 1 -- 172.21.15.36:0/4035736543 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecac1525e0 con 0x7fec900049f0 2022-06-17T13:58:10.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.184+0000 7fec9effd700 1 -- 172.21.15.36:0/4035736543 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feca800fe00 con 0x7fec900049f0 2022-06-17T13:58:10.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.188+0000 7fecb23e6700 1 -- 172.21.15.36:0/4035736543 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecac127df0 con 0x7fec900049f0 2022-06-17T13:58:10.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.188+0000 7fecb23e6700 1 -- 172.21.15.36:0/4035736543 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fecac152770 con 0x7fec900049f0 2022-06-17T13:58:10.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.188+0000 7fec9effd700 1 -- 172.21.15.36:0/4035736543 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7feca8014440 con 0x7fec900049f0 2022-06-17T13:58:10.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.188+0000 7fec9effd700 1 --2- 172.21.15.36:0/4035736543 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fec88053af0 0x7fec88055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.188+0000 7fec9effd700 1 -- 172.21.15.36:0/4035736543 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7feca806c760 con 0x7fec900049f0 2022-06-17T13:58:10.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.188+0000 7fecb0be3700 1 --2- 172.21.15.36:0/4035736543 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fec88053af0 0x7fec88055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.188+0000 7fec9effd700 1 --2- 172.21.15.36:0/4035736543 >> [v2:172.21.15.70:6800/22813,v1:172.21.15.70:6801/22813] conn(0x7fec8805a780 0x7fec8805cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.188+0000 7fec9effd700 1 -- 172.21.15.36:0/4035736543 --> [v2:172.21.15.70:6800/22813,v1:172.21.15.70:6801/22813] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fec8805d200 con 0x7fec8805a780 2022-06-17T13:58:10.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.188+0000 7fec9effd700 1 -- 172.21.15.36:0/4035736543 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) v2 ==== 24+0+0 (secure 0 0 0) 0x7feca806edf0 con 0x7fec900049f0 2022-06-17T13:58:10.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.188+0000 7fecb1be5700 1 --2- 172.21.15.36:0/4035736543 >> [v2:172.21.15.70:6800/22813,v1:172.21.15.70:6801/22813] conn(0x7fec8805a780 0x7fec8805cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.188+0000 7fecb0be3700 1 --2- 172.21.15.36:0/4035736543 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fec88053af0 0x7fec88055fb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7feca000b000 tx=0x7feca0008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:10.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.188+0000 7fecb1be5700 1 --2- 172.21.15.36:0/4035736543 >> [v2:172.21.15.70:6800/22813,v1:172.21.15.70:6801/22813] conn(0x7fec8805a780 0x7fec8805cc20 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:10.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.192+0000 7fec9effd700 1 -- 172.21.15.36:0/4035736543 <== osd.5 v2:172.21.15.70:6800/22813 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7fec8805d200 con 0x7fec8805a780 2022-06-17T13:58:10.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.208+0000 7fecb23e6700 1 -- 172.21.15.36:0/4035736543 --> [v2:172.21.15.70:6800/22813,v1:172.21.15.70:6801/22813] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fecac0b1030 con 0x7fec8805a780 2022-06-17T13:58:10.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.208+0000 7fec9effd700 1 -- 172.21.15.36:0/4035736543 <== osd.5 v2:172.21.15.70:6800/22813 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fecac0b1030 con 0x7fec8805a780 2022-06-17T13:58:10.222 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T13:58:10.223 INFO:teuthology.orchestra.run.smithi036.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-06-17T13:58:10.223 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T13:58:10.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.212+0000 7fecb23e6700 1 -- 172.21.15.36:0/4035736543 >> [v2:172.21.15.70:6800/22813,v1:172.21.15.70:6801/22813] conn(0x7fec8805a780 msgr2=0x7fec8805cc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.212+0000 7fecb23e6700 1 --2- 172.21.15.36:0/4035736543 >> [v2:172.21.15.70:6800/22813,v1:172.21.15.70:6801/22813] conn(0x7fec8805a780 0x7fec8805cc20 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.212+0000 7fecb23e6700 1 -- 172.21.15.36:0/4035736543 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fec88053af0 msgr2=0x7fec88055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.212+0000 7fecb23e6700 1 --2- 172.21.15.36:0/4035736543 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fec88053af0 0x7fec88055fb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7feca000b000 tx=0x7feca0008040).stop 2022-06-17T13:58:10.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.212+0000 7fecb23e6700 1 -- 172.21.15.36:0/4035736543 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fec900049f0 msgr2=0x7fecac121010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.212+0000 7fecb23e6700 1 --2- 172.21.15.36:0/4035736543 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fec900049f0 0x7fecac121010 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7fecac152960 tx=0x7feca8006df0).stop 2022-06-17T13:58:10.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.212+0000 7fecb1be5700 1 -- 172.21.15.36:0/4035736543 reap_dead start 2022-06-17T13:58:10.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.212+0000 7fecb23e6700 1 -- 172.21.15.36:0/4035736543 shutdown_connections 2022-06-17T13:58:10.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.212+0000 7fecb23e6700 1 -- 172.21.15.36:0/4035736543 >> 172.21.15.36:0/4035736543 conn(0x7fecac060880 msgr2=0x7fecac05bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:10.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.212+0000 7fecb23e6700 1 -- 172.21.15.36:0/4035736543 shutdown_connections 2022-06-17T13:58:10.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.212+0000 7fecb23e6700 1 -- 172.21.15.36:0/4035736543 wait complete. 2022-06-17T13:58:10.229 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2022-06-17T13:58:10.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.332+0000 7fde7fa67700 1 Processor -- start 2022-06-17T13:58:10.333 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.332+0000 7fde7fa67700 1 -- start start 2022-06-17T13:58:10.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.332+0000 7fde7fa67700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fde780ad330 0x7fde78129150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.332+0000 7fde7fa67700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fde780ad840 0x7fde780adc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.332+0000 7fde7fa67700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde78140930 con 0x7fde78063570 2022-06-17T13:58:10.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.332+0000 7fde7fa67700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde78140d90 con 0x7fde780ad330 2022-06-17T13:58:10.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.332+0000 7fde7fa67700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde780676f0 con 0x7fde780ad840 2022-06-17T13:58:10.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.336+0000 7fde7d002700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fde780ad330 0x7fde78129150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.336+0000 7fde7d803700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fde78063570 0x7fde78128a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58890/0 (socket says 172.21.15.36:58890) 2022-06-17T13:58:10.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.336+0000 7fde7d803700 1 -- 172.21.15.36:0/4051559588 learned_addr learned my addr 172.21.15.36:0/4051559588 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T13:58:10.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.336+0000 7fde7e004700 1 --2- 172.21.15.36:0/4051559588 >> v2:172.21.15.70:3300/0 conn(0x7fde780ad840 0x7fde780adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.336+0000 7fde7c801700 1 -- 172.21.15.36:0/4051559588 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2693456609 0 0) 0x7fde78140930 con 0x7fde78063570 2022-06-17T13:58:10.340 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.336+0000 7fde7c801700 1 -- 172.21.15.36:0/4051559588 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde60003260 con 0x7fde78063570 2022-06-17T13:58:10.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.336+0000 7fde7e004700 1 -- 172.21.15.36:0/4051559588 >> v2:172.21.15.36:3300/0 conn(0x7fde780ad330 msgr2=0x7fde78129150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T13:58:10.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.336+0000 7fde7e004700 1 --2- 172.21.15.36:0/4051559588 >> v2:172.21.15.36:3300/0 conn(0x7fde780ad330 0x7fde78129150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.341 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.336+0000 7fde7e004700 1 -- 172.21.15.36:0/4051559588 >> v1:172.21.15.36:6789/0 conn(0x7fde78063570 legacy=0x7fde78128a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.336+0000 7fde7e004700 1 -- 172.21.15.36:0/4051559588 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde7805fd80 con 0x7fde780ad840 2022-06-17T13:58:10.342 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.336+0000 7fde7e004700 1 --2- 172.21.15.36:0/4051559588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde780ad840 0x7fde780adc70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7fde700032f0 tx=0x7fde700091d0).ready entity=mon.1 client_cookie=afc88e3170bf304f server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:10.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.336+0000 7fde7c801700 1 -- 172.21.15.36:0/4051559588 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde7000b600 con 0x7fde780ad840 2022-06-17T13:58:10.343 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.336+0000 7fde7c801700 1 -- 172.21.15.36:0/4051559588 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7000b760 con 0x7fde780ad840 2022-06-17T13:58:10.344 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.336+0000 7fde7c801700 1 -- 172.21.15.36:0/4051559588 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde7000b8d0 con 0x7fde780ad840 2022-06-17T13:58:10.345 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.336+0000 7fde7fa67700 1 -- 172.21.15.36:0/4051559588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde780ad840 msgr2=0x7fde780adc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.336+0000 7fde7fa67700 1 --2- 172.21.15.36:0/4051559588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde780ad840 0x7fde780adc70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7fde700032f0 tx=0x7fde700091d0).stop 2022-06-17T13:58:10.346 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.336+0000 7fde7fa67700 1 -- 172.21.15.36:0/4051559588 shutdown_connections 2022-06-17T13:58:10.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.336+0000 7fde7fa67700 1 --2- 172.21.15.36:0/4051559588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde780ad840 0x7fde780adc70 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.347 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.336+0000 7fde7fa67700 1 --2- 172.21.15.36:0/4051559588 >> v2:172.21.15.36:3300/0 conn(0x7fde780ad330 0x7fde78129150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.336+0000 7fde7fa67700 1 -- 172.21.15.36:0/4051559588 >> 172.21.15.36:0/4051559588 conn(0x7fde780b53e0 msgr2=0x7fde780b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:10.348 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.336+0000 7fde7fa67700 1 -- 172.21.15.36:0/4051559588 shutdown_connections 2022-06-17T13:58:10.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.336+0000 7fde7fa67700 1 -- 172.21.15.36:0/4051559588 wait complete. 2022-06-17T13:58:10.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.340+0000 7fde7fa67700 1 Processor -- start 2022-06-17T13:58:10.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.340+0000 7fde7fa67700 1 -- start start 2022-06-17T13:58:10.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.340+0000 7fde7fa67700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde78063570 0x7fde78122a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.340+0000 7fde7fa67700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde780ad330 0x7fde78122060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.340+0000 7fde7fa67700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde780ad840 0x7fde78066420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.358 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.340+0000 7fde7fa67700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde78123350 con 0x7fde780ad840 2022-06-17T13:58:10.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.340+0000 7fde7fa67700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde78122550 con 0x7fde78063570 2022-06-17T13:58:10.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.340+0000 7fde7fa67700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde78140140 con 0x7fde780ad330 2022-06-17T13:58:10.359 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.340+0000 7fde7e004700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde780ad840 0x7fde78066420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.340+0000 7fde7d002700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde780ad330 0x7fde78122060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.340+0000 7fde7e004700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde780ad840 0x7fde78066420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41370/0 (socket says 172.21.15.36:41370) 2022-06-17T13:58:10.360 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.340+0000 7fde7e004700 1 -- 172.21.15.36:0/3851363589 learned_addr learned my addr 172.21.15.36:0/3851363589 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:10.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.340+0000 7fde7d803700 1 --2- 172.21.15.36:0/3851363589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde78063570 0x7fde78122a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.340+0000 7fde7e004700 1 -- 172.21.15.36:0/3851363589 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde780ad330 msgr2=0x7fde78122060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.361 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.340+0000 7fde7e004700 1 --2- 172.21.15.36:0/3851363589 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde780ad330 0x7fde78122060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.340+0000 7fde7e004700 1 -- 172.21.15.36:0/3851363589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde78063570 msgr2=0x7fde78122a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.340+0000 7fde7e004700 1 --2- 172.21.15.36:0/3851363589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde78063570 0x7fde78122a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.362 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.340+0000 7fde7e004700 1 -- 172.21.15.36:0/3851363589 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde70009470 con 0x7fde780ad840 2022-06-17T13:58:10.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.340+0000 7fde7e004700 1 --2- 172.21.15.36:0/3851363589 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde780ad840 0x7fde78066420 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 rx=0x7fde700032f0 tx=0x7fde70016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:10.363 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.340+0000 7fde66ffd700 1 -- 172.21.15.36:0/3851363589 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde70011c10 con 0x7fde780ad840 2022-06-17T13:58:10.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.340+0000 7fde66ffd700 1 -- 172.21.15.36:0/3851363589 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde70011d70 con 0x7fde780ad840 2022-06-17T13:58:10.364 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.340+0000 7fde7fa67700 1 -- 172.21.15.36:0/3851363589 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde781230b0 con 0x7fde780ad840 2022-06-17T13:58:10.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.340+0000 7fde66ffd700 1 -- 172.21.15.36:0/3851363589 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde7001f5e0 con 0x7fde780ad840 2022-06-17T13:58:10.365 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.340+0000 7fde7fa67700 1 -- 172.21.15.36:0/3851363589 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde78124740 con 0x7fde780ad840 2022-06-17T13:58:10.366 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.344+0000 7fde7fa67700 1 -- 172.21.15.36:0/3851363589 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde78121d50 con 0x7fde780ad840 2022-06-17T13:58:10.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.344+0000 7fde66ffd700 1 -- 172.21.15.36:0/3851363589 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fde70026030 con 0x7fde780ad840 2022-06-17T13:58:10.367 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.344+0000 7fde66ffd700 1 --2- 172.21.15.36:0/3851363589 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde58053a40 0x7fde58055f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.344+0000 7fde66ffd700 1 -- 172.21.15.36:0/3851363589 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fde7006c670 con 0x7fde780ad840 2022-06-17T13:58:10.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.344+0000 7fde7d803700 1 --2- 172.21.15.36:0/3851363589 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde58053a40 0x7fde58055f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.368 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.344+0000 7fde7d803700 1 --2- 172.21.15.36:0/3851363589 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde58053a40 0x7fde58055f00 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7fde7400d000 tx=0x7fde74009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:10.369 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.348+0000 7fde66ffd700 1 -- 172.21.15.36:0/3851363589 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fde70043790 con 0x7fde780ad840 2022-06-17T13:58:10.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.472+0000 7fde7fa67700 1 -- 172.21.15.36:0/3851363589 --> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fde7811cbd0 con 0x7fde58053a40 2022-06-17T13:58:10.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.476+0000 7fde66ffd700 1 -- 172.21.15.36:0/3851363589 <== mgr.4104 v2:172.21.15.36:6800/22419 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fde740133f0 con 0x7fde58053a40 2022-06-17T13:58:10.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.476+0000 7fde7fa67700 1 -- 172.21.15.36:0/3851363589 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde58053a40 msgr2=0x7fde58055f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.476+0000 7fde7fa67700 1 --2- 172.21.15.36:0/3851363589 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde58053a40 0x7fde58055f00 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7fde7400d000 tx=0x7fde74009040).stop 2022-06-17T13:58:10.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.476+0000 7fde7fa67700 1 -- 172.21.15.36:0/3851363589 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde780ad840 msgr2=0x7fde78066420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.476+0000 7fde7fa67700 1 --2- 172.21.15.36:0/3851363589 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde780ad840 0x7fde78066420 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 rx=0x7fde700032f0 tx=0x7fde70016000).stop 2022-06-17T13:58:10.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.476+0000 7fde7fa67700 1 -- 172.21.15.36:0/3851363589 shutdown_connections 2022-06-17T13:58:10.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.476+0000 7fde7fa67700 1 --2- 172.21.15.36:0/3851363589 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fde58053a40 0x7fde58055f00 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.476+0000 7fde7fa67700 1 --2- 172.21.15.36:0/3851363589 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fde780ad840 0x7fde78066420 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.476+0000 7fde7fa67700 1 --2- 172.21.15.36:0/3851363589 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde780ad330 0x7fde78122060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.476+0000 7fde7fa67700 1 --2- 172.21.15.36:0/3851363589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde78063570 0x7fde78122a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.476+0000 7fde7fa67700 1 -- 172.21.15.36:0/3851363589 >> 172.21.15.36:0/3851363589 conn(0x7fde780b53e0 msgr2=0x7fde780b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:10.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.476+0000 7fde7fa67700 1 -- 172.21.15.36:0/3851363589 shutdown_connections 2022-06-17T13:58:10.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.476+0000 7fde7fa67700 1 -- 172.21.15.36:0/3851363589 wait complete. 2022-06-17T13:58:10.481 INFO:teuthology.orchestra.run.smithi036.stderr:instructed osd(s) 5 to deep-scrub 2022-06-17T13:58:10.487 INFO:tasks.ceph:Scrubbing osd.6 2022-06-17T13:58:10.489 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2022-06-17T13:58:10.602 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.600+0000 7f5c92caa700 1 Processor -- start 2022-06-17T13:58:10.602 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.600+0000 7f5c92caa700 1 -- start start 2022-06-17T13:58:10.605 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.604+0000 7f5c92caa700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5c8c0b2430 0x7f5c8c143760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.604+0000 7f5c92caa700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8c0b3290 0x7f5c8c143c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.604+0000 7f5c92caa700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c8c063b10 con 0x7f5c8c063730 2022-06-17T13:58:10.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.604+0000 7f5c92caa700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c8c0ada90 con 0x7f5c8c0b2430 2022-06-17T13:58:10.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.604+0000 7f5c92caa700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8c144140 con 0x7f5c8c0b3290 2022-06-17T13:58:10.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.604+0000 7f5c914a7700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5c8c0b2430 0x7f5c8c143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.604+0000 7f5c914a7700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5c8c0b2430 0x7f5c8c143760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41376/0 (socket says 172.21.15.36:41376) 2022-06-17T13:58:10.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.604+0000 7f5c914a7700 1 -- 172.21.15.36:0/408335345 learned_addr learned my addr 172.21.15.36:0/408335345 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:10.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.604+0000 7f5c924a9700 1 --2- 172.21.15.36:0/408335345 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8c0b3290 0x7f5c8c143c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.604+0000 7f5c914a7700 1 -- 172.21.15.36:0/408335345 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8c0b3290 msgr2=0x7f5c8c143c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T13:58:10.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.604+0000 7f5c914a7700 1 --2- 172.21.15.36:0/408335345 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8c0b3290 0x7f5c8c143c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.604+0000 7f5c914a7700 1 -- 172.21.15.36:0/408335345 >> v1:172.21.15.70:6789/0 conn(0x7f5c8c063730 legacy=0x7f5c8c0af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.611 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.604+0000 7f5c914a7700 1 -- 172.21.15.36:0/408335345 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8c0b3d50 con 0x7f5c8c0b2430 2022-06-17T13:58:10.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.604+0000 7f5c90ca6700 1 -- 172.21.15.36:0/408335345 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 280070480 0 0) 0x7f5c8c063b10 con 0x7f5c8c063730 2022-06-17T13:58:10.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.604+0000 7f5c914a7700 1 --2- 172.21.15.36:0/408335345 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c8c0b2430 0x7f5c8c143760 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 rx=0x7f5c8c144280 tx=0x7f5c80006f40).ready entity=mon.0 client_cookie=8e4e5ff8bdc644d8 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:10.612 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.604+0000 7f5c90ca6700 1 -- 172.21.15.36:0/408335345 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c8000da40 con 0x7f5c8c0b2430 2022-06-17T13:58:10.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.604+0000 7f5c90ca6700 1 -- 172.21.15.36:0/408335345 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c80000e90 con 0x7f5c8c0b2430 2022-06-17T13:58:10.613 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.604+0000 7f5c90ca6700 1 -- 172.21.15.36:0/408335345 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c8000dd90 con 0x7f5c8c0b2430 2022-06-17T13:58:10.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c92caa700 1 -- 172.21.15.36:0/408335345 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c8c0b2430 msgr2=0x7f5c8c143760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c92caa700 1 --2- 172.21.15.36:0/408335345 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c8c0b2430 0x7f5c8c143760 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 rx=0x7f5c8c144280 tx=0x7f5c80006f40).stop 2022-06-17T13:58:10.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c92caa700 1 -- 172.21.15.36:0/408335345 shutdown_connections 2022-06-17T13:58:10.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c92caa700 1 --2- 172.21.15.36:0/408335345 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8c0b3290 0x7f5c8c143c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c92caa700 1 --2- 172.21.15.36:0/408335345 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c8c0b2430 0x7f5c8c143760 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.619 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c92caa700 1 -- 172.21.15.36:0/408335345 >> 172.21.15.36:0/408335345 conn(0x7f5c8c060880 msgr2=0x7f5c8c0614b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:10.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c92caa700 1 -- 172.21.15.36:0/408335345 shutdown_connections 2022-06-17T13:58:10.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c92caa700 1 -- 172.21.15.36:0/408335345 wait complete. 2022-06-17T13:58:10.620 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c92caa700 1 Processor -- start 2022-06-17T13:58:10.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c92caa700 1 -- start start 2022-06-17T13:58:10.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c92caa700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c8c063730 0x7f5c8c127f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.621 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c92caa700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c8c0b2430 0x7f5c8c064280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c92caa700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8c0b3290 0x7f5c8c064770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c92caa700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8c066e50 con 0x7f5c8c0b2430 2022-06-17T13:58:10.622 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c92caa700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8c0661d0 con 0x7f5c8c063730 2022-06-17T13:58:10.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c92caa700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8c066470 con 0x7f5c8c0b3290 2022-06-17T13:58:10.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c914a7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c8c0b2430 0x7f5c8c064280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.623 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c914a7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c8c0b2430 0x7f5c8c064280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41382/0 (socket says 172.21.15.36:41382) 2022-06-17T13:58:10.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c914a7700 1 -- 172.21.15.36:0/63477133 learned_addr learned my addr 172.21.15.36:0/63477133 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:10.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c924a9700 1 --2- 172.21.15.36:0/63477133 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8c0b3290 0x7f5c8c064770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.624 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c91ca8700 1 --2- 172.21.15.36:0/63477133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c8c063730 0x7f5c8c127f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c914a7700 1 -- 172.21.15.36:0/63477133 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8c0b3290 msgr2=0x7f5c8c064770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c914a7700 1 --2- 172.21.15.36:0/63477133 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8c0b3290 0x7f5c8c064770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.625 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c914a7700 1 -- 172.21.15.36:0/63477133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c8c063730 msgr2=0x7f5c8c127f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.608+0000 7f5c914a7700 1 --2- 172.21.15.36:0/63477133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c8c063730 0x7f5c8c127f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.626 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.612+0000 7f5c914a7700 1 -- 172.21.15.36:0/63477133 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8000b460 con 0x7f5c8c0b2430 2022-06-17T13:58:10.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.612+0000 7f5c924a9700 1 --2- 172.21.15.36:0/63477133 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8c0b3290 0x7f5c8c064770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-17T13:58:10.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.612+0000 7f5c914a7700 1 --2- 172.21.15.36:0/63477133 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c8c0b2430 0x7f5c8c064280 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 rx=0x7f5c8c144280 tx=0x7f5c8c0b2db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:10.627 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.612+0000 7f5c7effd700 1 -- 172.21.15.36:0/63477133 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c8001d070 con 0x7f5c8c0b2430 2022-06-17T13:58:10.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.612+0000 7f5c7effd700 1 -- 172.21.15.36:0/63477133 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c80018400 con 0x7f5c8c0b2430 2022-06-17T13:58:10.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.612+0000 7f5c92caa700 1 -- 172.21.15.36:0/63477133 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c8c12ea70 con 0x7f5c8c0b2430 2022-06-17T13:58:10.628 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.612+0000 7f5c7effd700 1 -- 172.21.15.36:0/63477133 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c80018760 con 0x7f5c8c0b2430 2022-06-17T13:58:10.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.612+0000 7f5c92caa700 1 -- 172.21.15.36:0/63477133 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c8c13f6e0 con 0x7f5c8c0b2430 2022-06-17T13:58:10.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.612+0000 7f5c7effd700 1 -- 172.21.15.36:0/63477133 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f5c80029070 con 0x7f5c8c0b2430 2022-06-17T13:58:10.629 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.612+0000 7f5c92caa700 1 -- 172.21.15.36:0/63477133 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5c8c12ef50 con 0x7f5c8c0b2430 2022-06-17T13:58:10.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.612+0000 7f5c7effd700 1 --2- 172.21.15.36:0/63477133 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c6c0539b0 0x7f5c6c055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.630 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.612+0000 7f5c7effd700 1 -- 172.21.15.36:0/63477133 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f5c8006d5a0 con 0x7f5c8c0b2430 2022-06-17T13:58:10.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.612+0000 7f5c7effd700 1 --2- 172.21.15.36:0/63477133 >> [v2:172.21.15.70:6816/22822,v1:172.21.15.70:6817/22822] conn(0x7f5c6c05a640 0x7f5c6c05cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.612+0000 7f5c7effd700 1 -- 172.21.15.36:0/63477133 --> [v2:172.21.15.70:6816/22822,v1:172.21.15.70:6817/22822] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5c6c05d160 con 0x7f5c6c05a640 2022-06-17T13:58:10.631 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.612+0000 7f5c7effd700 1 -- 172.21.15.36:0/63477133 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c8006fb20 con 0x7f5c8c0b2430 2022-06-17T13:58:10.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.612+0000 7f5c91ca8700 1 --2- 172.21.15.36:0/63477133 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c6c0539b0 0x7f5c6c055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.612+0000 7f5c924a9700 1 --2- 172.21.15.36:0/63477133 >> [v2:172.21.15.70:6816/22822,v1:172.21.15.70:6817/22822] conn(0x7f5c6c05a640 0x7f5c6c05cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.632 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.616+0000 7f5c91ca8700 1 --2- 172.21.15.36:0/63477133 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c6c0539b0 0x7f5c6c055e70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7f5c8800b000 tx=0x7f5c88009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:10.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.616+0000 7f5c924a9700 1 --2- 172.21.15.36:0/63477133 >> [v2:172.21.15.70:6816/22822,v1:172.21.15.70:6817/22822] conn(0x7f5c6c05a640 0x7f5c6c05cae0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:10.633 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.616+0000 7f5c7effd700 1 -- 172.21.15.36:0/63477133 <== osd.6 v2:172.21.15.70:6816/22822 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f5c6c05d160 con 0x7f5c6c05a640 2022-06-17T13:58:10.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.632+0000 7f5c92caa700 1 -- 172.21.15.36:0/63477133 --> [v2:172.21.15.70:6816/22822,v1:172.21.15.70:6817/22822] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5c8c0650c0 con 0x7f5c6c05a640 2022-06-17T13:58:10.635 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.632+0000 7f5c7effd700 1 -- 172.21.15.36:0/63477133 <== osd.6 v2:172.21.15.70:6816/22822 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f5c8c0650c0 con 0x7f5c6c05a640 2022-06-17T13:58:10.636 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T13:58:10.636 INFO:teuthology.orchestra.run.smithi036.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-06-17T13:58:10.637 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T13:58:10.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.632+0000 7f5c92caa700 1 -- 172.21.15.36:0/63477133 >> [v2:172.21.15.70:6816/22822,v1:172.21.15.70:6817/22822] conn(0x7f5c6c05a640 msgr2=0x7f5c6c05cae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.636+0000 7f5c92caa700 1 --2- 172.21.15.36:0/63477133 >> [v2:172.21.15.70:6816/22822,v1:172.21.15.70:6817/22822] conn(0x7f5c6c05a640 0x7f5c6c05cae0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.639 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.636+0000 7f5c92caa700 1 -- 172.21.15.36:0/63477133 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c6c0539b0 msgr2=0x7f5c6c055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.636+0000 7f5c92caa700 1 --2- 172.21.15.36:0/63477133 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5c6c0539b0 0x7f5c6c055e70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7f5c8800b000 tx=0x7f5c88009040).stop 2022-06-17T13:58:10.640 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.636+0000 7f5c92caa700 1 -- 172.21.15.36:0/63477133 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c8c0b2430 msgr2=0x7f5c8c064280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.636+0000 7f5c92caa700 1 --2- 172.21.15.36:0/63477133 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5c8c0b2430 0x7f5c8c064280 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 rx=0x7f5c8c144280 tx=0x7f5c8c0b2db0).stop 2022-06-17T13:58:10.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.636+0000 7f5c924a9700 1 -- 172.21.15.36:0/63477133 reap_dead start 2022-06-17T13:58:10.641 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.636+0000 7f5c92caa700 1 -- 172.21.15.36:0/63477133 shutdown_connections 2022-06-17T13:58:10.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.636+0000 7f5c92caa700 1 -- 172.21.15.36:0/63477133 >> 172.21.15.36:0/63477133 conn(0x7f5c8c060880 msgr2=0x7f5c8c0614b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:10.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.636+0000 7f5c92caa700 1 -- 172.21.15.36:0/63477133 shutdown_connections 2022-06-17T13:58:10.642 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.636+0000 7f5c92caa700 1 -- 172.21.15.36:0/63477133 wait complete. 2022-06-17T13:58:10.650 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2022-06-17T13:58:10.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe28359e700 1 Processor -- start 2022-06-17T13:58:10.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe28359e700 1 -- start start 2022-06-17T13:58:10.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe28359e700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe2840b2200 0x7fe2840af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe28359e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe2840b2cd0 0x7fe284147600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe28359e700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2840b25e0 con 0x7fe2840af8d0 2022-06-17T13:58:10.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe28359e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2840ad790 con 0x7fe2840b2cd0 2022-06-17T13:58:10.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe28359e700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2840aef40 con 0x7fe2840b2200 2022-06-17T13:58:10.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe279d9b700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fe2840af8d0 0x7fe28414a8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58922/0 (socket says 172.21.15.36:58922) 2022-06-17T13:58:10.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe279d9b700 1 -- 172.21.15.36:0/3667973406 learned_addr learned my addr 172.21.15.36:0/3667973406 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T13:58:10.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe282d9d700 1 --2- 172.21.15.36:0/3667973406 >> v2:172.21.15.36:3300/0 conn(0x7fe2840b2cd0 0x7fe284147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe28259c700 1 --2- 172.21.15.36:0/3667973406 >> v2:172.21.15.70:3300/0 conn(0x7fe2840b2200 0x7fe2840af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe281d9b700 1 -- 172.21.15.36:0/3667973406 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2459427869 0 0) 0x7fe2840b25e0 con 0x7fe2840af8d0 2022-06-17T13:58:10.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe281d9b700 1 -- 172.21.15.36:0/3667973406 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe264003260 con 0x7fe2840af8d0 2022-06-17T13:58:10.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe282d9d700 1 -- 172.21.15.36:0/3667973406 >> v2:172.21.15.70:3300/0 conn(0x7fe2840b2200 msgr2=0x7fe2840af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T13:58:10.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe281d9b700 1 -- 172.21.15.36:0/3667973406 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1147779599 0 0) 0x7fe264003260 con 0x7fe2840af8d0 2022-06-17T13:58:10.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe282d9d700 1 --2- 172.21.15.36:0/3667973406 >> v2:172.21.15.70:3300/0 conn(0x7fe2840b2200 0x7fe2840af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe282d9d700 1 -- 172.21.15.36:0/3667973406 >> v1:172.21.15.36:6789/0 conn(0x7fe2840af8d0 legacy=0x7fe28414a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe282d9d700 1 -- 172.21.15.36:0/3667973406 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2840b1950 con 0x7fe2840b2cd0 2022-06-17T13:58:10.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe282d9d700 1 --2- 172.21.15.36:0/3667973406 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2840b2cd0 0x7fe284147600 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 rx=0x7fe26c001800 tx=0x7fe27c0091d0).ready entity=mon.0 client_cookie=2cb5691041343fc8 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:10.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe281d9b700 1 -- 172.21.15.36:0/3667973406 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe26c001660 con 0x7fe2840b2cd0 2022-06-17T13:58:10.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe281d9b700 1 -- 172.21.15.36:0/3667973406 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27c00f510 con 0x7fe2840b2cd0 2022-06-17T13:58:10.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe281d9b700 1 -- 172.21.15.36:0/3667973406 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe27c00f680 con 0x7fe2840b2cd0 2022-06-17T13:58:10.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe28359e700 1 -- 172.21.15.36:0/3667973406 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2840b2cd0 msgr2=0x7fe284147600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe28359e700 1 --2- 172.21.15.36:0/3667973406 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2840b2cd0 0x7fe284147600 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 rx=0x7fe26c001800 tx=0x7fe27c0091d0).stop 2022-06-17T13:58:10.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe28359e700 1 -- 172.21.15.36:0/3667973406 shutdown_connections 2022-06-17T13:58:10.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe28359e700 1 --2- 172.21.15.36:0/3667973406 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2840b2cd0 0x7fe284147600 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe28359e700 1 --2- 172.21.15.36:0/3667973406 >> v2:172.21.15.70:3300/0 conn(0x7fe2840b2200 0x7fe2840af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe28359e700 1 -- 172.21.15.36:0/3667973406 >> 172.21.15.36:0/3667973406 conn(0x7fe28405f770 msgr2=0x7fe28405fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:10.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe28359e700 1 -- 172.21.15.36:0/3667973406 shutdown_connections 2022-06-17T13:58:10.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.756+0000 7fe28359e700 1 -- 172.21.15.36:0/3667973406 wait complete. 2022-06-17T13:58:10.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.760+0000 7fe28359e700 1 Processor -- start 2022-06-17T13:58:10.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.760+0000 7fe28359e700 1 -- start start 2022-06-17T13:58:10.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.760+0000 7fe28359e700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2840af8d0 0x7fe2840b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.760+0000 7fe28359e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2840b2200 0x7fe28415a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.760+0000 7fe28359e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2840b2cd0 0x7fe284066a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.760+0000 7fe28359e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28415abe0 con 0x7fe2840b2cd0 2022-06-17T13:58:10.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.760+0000 7fe28359e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2840a9900 con 0x7fe2840b2200 2022-06-17T13:58:10.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.760+0000 7fe28359e700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2840a93c0 con 0x7fe2840af8d0 2022-06-17T13:58:10.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.760+0000 7fe282d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2840b2cd0 0x7fe284066a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.760+0000 7fe282d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2840b2cd0 0x7fe284066a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41400/0 (socket says 172.21.15.36:41400) 2022-06-17T13:58:10.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.760+0000 7fe282d9d700 1 -- 172.21.15.36:0/2000992791 learned_addr learned my addr 172.21.15.36:0/2000992791 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:10.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.760+0000 7fe279d9b700 1 --2- 172.21.15.36:0/2000992791 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2840b2200 0x7fe28415a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.760+0000 7fe28259c700 1 --2- 172.21.15.36:0/2000992791 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2840af8d0 0x7fe2840b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.760+0000 7fe282d9d700 1 -- 172.21.15.36:0/2000992791 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2840af8d0 msgr2=0x7fe2840b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.760+0000 7fe282d9d700 1 --2- 172.21.15.36:0/2000992791 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2840af8d0 0x7fe2840b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.760+0000 7fe282d9d700 1 -- 172.21.15.36:0/2000992791 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2840b2200 msgr2=0x7fe28415a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.760+0000 7fe282d9d700 1 --2- 172.21.15.36:0/2000992791 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2840b2200 0x7fe28415a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.760+0000 7fe282d9d700 1 -- 172.21.15.36:0/2000992791 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27c009470 con 0x7fe2840b2cd0 2022-06-17T13:58:10.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.760+0000 7fe282d9d700 1 --2- 172.21.15.36:0/2000992791 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2840b2cd0 0x7fe284066a60 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 rx=0x7fe26c001800 tx=0x7fe27c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:10.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.760+0000 7fe27bfff700 1 -- 172.21.15.36:0/2000992791 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe27c0032d0 con 0x7fe2840b2cd0 2022-06-17T13:58:10.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.760+0000 7fe27bfff700 1 -- 172.21.15.36:0/2000992791 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27c011ad0 con 0x7fe2840b2cd0 2022-06-17T13:58:10.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.760+0000 7fe28359e700 1 -- 172.21.15.36:0/2000992791 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2840a9660 con 0x7fe2840b2cd0 2022-06-17T13:58:10.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.760+0000 7fe27bfff700 1 -- 172.21.15.36:0/2000992791 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe27c011e30 con 0x7fe2840b2cd0 2022-06-17T13:58:10.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.760+0000 7fe28359e700 1 -- 172.21.15.36:0/2000992791 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2840aa200 con 0x7fe2840b2cd0 2022-06-17T13:58:10.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.764+0000 7fe28359e700 1 -- 172.21.15.36:0/2000992791 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2840ad330 con 0x7fe2840b2cd0 2022-06-17T13:58:10.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.764+0000 7fe27bfff700 1 -- 172.21.15.36:0/2000992791 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe27c026030 con 0x7fe2840b2cd0 2022-06-17T13:58:10.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.764+0000 7fe27bfff700 1 --2- 172.21.15.36:0/2000992791 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe268053ad0 0x7fe268055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:10.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.764+0000 7fe28259c700 1 --2- 172.21.15.36:0/2000992791 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe268053ad0 0x7fe268055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:10.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.764+0000 7fe27bfff700 1 -- 172.21.15.36:0/2000992791 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fe27c06c650 con 0x7fe2840b2cd0 2022-06-17T13:58:10.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.764+0000 7fe28259c700 1 --2- 172.21.15.36:0/2000992791 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe268053ad0 0x7fe268055f90 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7fe27400b000 tx=0x7fe274008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:10.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.768+0000 7fe27bfff700 1 -- 172.21.15.36:0/2000992791 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe27c043540 con 0x7fe2840b2cd0 2022-06-17T13:58:10.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.892+0000 7fe28359e700 1 -- 172.21.15.36:0/2000992791 --> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2840a9240 con 0x7fe268053ad0 2022-06-17T13:58:10.894 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.896+0000 7fe27bfff700 1 -- 172.21.15.36:0/2000992791 <== mgr.4104 v2:172.21.15.36:6800/22419 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe274010680 con 0x7fe268053ad0 2022-06-17T13:58:10.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.896+0000 7fe28359e700 1 -- 172.21.15.36:0/2000992791 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe268053ad0 msgr2=0x7fe268055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.896+0000 7fe28359e700 1 --2- 172.21.15.36:0/2000992791 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe268053ad0 0x7fe268055f90 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7fe27400b000 tx=0x7fe274008040).stop 2022-06-17T13:58:10.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.896+0000 7fe28359e700 1 -- 172.21.15.36:0/2000992791 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2840b2cd0 msgr2=0x7fe284066a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:10.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.896+0000 7fe28359e700 1 --2- 172.21.15.36:0/2000992791 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2840b2cd0 0x7fe284066a60 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 rx=0x7fe26c001800 tx=0x7fe27c016000).stop 2022-06-17T13:58:10.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.896+0000 7fe28359e700 1 -- 172.21.15.36:0/2000992791 shutdown_connections 2022-06-17T13:58:10.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.896+0000 7fe28359e700 1 --2- 172.21.15.36:0/2000992791 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe268053ad0 0x7fe268055f90 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.896+0000 7fe28359e700 1 --2- 172.21.15.36:0/2000992791 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe2840b2cd0 0x7fe284066a60 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.896+0000 7fe28359e700 1 --2- 172.21.15.36:0/2000992791 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2840b2200 0x7fe28415a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.896+0000 7fe28359e700 1 --2- 172.21.15.36:0/2000992791 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2840af8d0 0x7fe2840b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:10.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.896+0000 7fe28359e700 1 -- 172.21.15.36:0/2000992791 >> 172.21.15.36:0/2000992791 conn(0x7fe28405f770 msgr2=0x7fe284147e10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:10.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.896+0000 7fe28359e700 1 -- 172.21.15.36:0/2000992791 shutdown_connections 2022-06-17T13:58:10.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:10.896+0000 7fe28359e700 1 -- 172.21.15.36:0/2000992791 wait complete. 2022-06-17T13:58:10.902 INFO:teuthology.orchestra.run.smithi036.stderr:instructed osd(s) 6 to deep-scrub 2022-06-17T13:58:10.914 INFO:tasks.ceph:Scrubbing osd.7 2022-06-17T13:58:10.915 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2022-06-17T13:58:11.038 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.040+0000 7fe439468700 1 Processor -- start 2022-06-17T13:58:11.039 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.040+0000 7fe439468700 1 -- start start 2022-06-17T13:58:11.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.040+0000 7fe439468700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4340b0d70 0x7fe4340ac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:11.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.040+0000 7fe439468700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4340b2b90 con 0x7fe4340b15f0 2022-06-17T13:58:11.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.040+0000 7fe439468700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4340610e0 con 0x7fe4340b27b0 2022-06-17T13:58:11.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.040+0000 7fe439468700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4340ad380 con 0x7fe4340b0d70 2022-06-17T13:58:11.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.040+0000 7fe433fff700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe4340b27b0 0x7fe43413f110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.36:54668/0 (socket says 172.21.15.36:54668) 2022-06-17T13:58:11.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.040+0000 7fe433fff700 1 -- 172.21.15.36:0/4211021535 learned_addr learned my addr 172.21.15.36:0/4211021535 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T13:58:11.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.040+0000 7fe438c67700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fe4340b15f0 0x7fe4340acc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58938/0 (socket says 172.21.15.36:58938) 2022-06-17T13:58:11.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.040+0000 7fe42b7fe700 1 --2- 172.21.15.36:0/4211021535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4340b0d70 0x7fe4340ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:11.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.040+0000 7fe4337fe700 1 -- 172.21.15.36:0/4211021535 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 118515760 0 0) 0x7fe4340b2b90 con 0x7fe4340b15f0 2022-06-17T13:58:11.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.040+0000 7fe42b7fe700 1 -- 172.21.15.36:0/4211021535 >> v1:172.21.15.70:6789/0 conn(0x7fe4340b27b0 legacy=0x7fe43413f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:11.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.040+0000 7fe42b7fe700 1 -- 172.21.15.36:0/4211021535 >> v1:172.21.15.36:6789/0 conn(0x7fe4340b15f0 legacy=0x7fe4340acc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:11.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.040+0000 7fe42b7fe700 1 -- 172.21.15.36:0/4211021535 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4340ac190 con 0x7fe4340b0d70 2022-06-17T13:58:11.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.040+0000 7fe42b7fe700 1 --2- 172.21.15.36:0/4211021535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4340b0d70 0x7fe4340ac780 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fe424001110 tx=0x7fe41c006df0).ready entity=mon.2 client_cookie=2d1fd13d5345a5dd server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:11.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.040+0000 7fe4337fe700 1 -- 172.21.15.36:0/4211021535 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe41c014070 con 0x7fe4340b0d70 2022-06-17T13:58:11.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.040+0000 7fe4337fe700 1 -- 172.21.15.36:0/4211021535 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe41c000e90 con 0x7fe4340b0d70 2022-06-17T13:58:11.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.040+0000 7fe4337fe700 1 -- 172.21.15.36:0/4211021535 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe41c00fb60 con 0x7fe4340b0d70 2022-06-17T13:58:11.051 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.044+0000 7fe439468700 1 -- 172.21.15.36:0/4211021535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4340b0d70 msgr2=0x7fe4340ac780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:11.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.044+0000 7fe439468700 1 --2- 172.21.15.36:0/4211021535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4340b0d70 0x7fe4340ac780 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fe424001110 tx=0x7fe41c006df0).stop 2022-06-17T13:58:11.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.044+0000 7fe439468700 1 -- 172.21.15.36:0/4211021535 shutdown_connections 2022-06-17T13:58:11.052 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.044+0000 7fe439468700 1 --2- 172.21.15.36:0/4211021535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4340b0d70 0x7fe4340ac780 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:11.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.044+0000 7fe439468700 1 -- 172.21.15.36:0/4211021535 >> 172.21.15.36:0/4211021535 conn(0x7fe4340b5d10 msgr2=0x7fe4340b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:11.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.044+0000 7fe439468700 1 -- 172.21.15.36:0/4211021535 shutdown_connections 2022-06-17T13:58:11.053 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.044+0000 7fe439468700 1 -- 172.21.15.36:0/4211021535 wait complete. 2022-06-17T13:58:11.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.044+0000 7fe439468700 1 Processor -- start 2022-06-17T13:58:11.054 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.044+0000 7fe439468700 1 -- start start 2022-06-17T13:58:11.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.044+0000 7fe439468700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4340b0d70 0x7fe4341419b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:11.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.044+0000 7fe439468700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4340b15f0 0x7fe43413e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:11.055 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.044+0000 7fe439468700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4340b27b0 0x7fe434149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:11.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.044+0000 7fe439468700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4341420b0 con 0x7fe4340b27b0 2022-06-17T13:58:11.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.044+0000 7fe439468700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe434141300 con 0x7fe4340b0d70 2022-06-17T13:58:11.056 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.044+0000 7fe439468700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe43413db90 con 0x7fe4340b15f0 2022-06-17T13:58:11.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.048+0000 7fe438c67700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4340b27b0 0x7fe434149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:11.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.048+0000 7fe433fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4340b0d70 0x7fe4341419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:11.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.048+0000 7fe42b7fe700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4340b15f0 0x7fe43413e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:11.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.048+0000 7fe438c67700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4340b27b0 0x7fe434149f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41414/0 (socket says 172.21.15.36:41414) 2022-06-17T13:58:11.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.048+0000 7fe438c67700 1 -- 172.21.15.36:0/3557595556 learned_addr learned my addr 172.21.15.36:0/3557595556 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:11.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.048+0000 7fe438c67700 1 -- 172.21.15.36:0/3557595556 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4340b15f0 msgr2=0x7fe43413e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:11.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.048+0000 7fe438c67700 1 --2- 172.21.15.36:0/3557595556 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4340b15f0 0x7fe43413e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:11.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.048+0000 7fe438c67700 1 -- 172.21.15.36:0/3557595556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4340b0d70 msgr2=0x7fe4341419b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:11.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.048+0000 7fe438c67700 1 --2- 172.21.15.36:0/3557595556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4340b0d70 0x7fe4341419b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:11.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.048+0000 7fe438c67700 1 -- 172.21.15.36:0/3557595556 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe41c00b180 con 0x7fe4340b27b0 2022-06-17T13:58:11.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.048+0000 7fe433fff700 1 --2- 172.21.15.36:0/3557595556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4340b0d70 0x7fe4341419b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-17T13:58:11.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.048+0000 7fe438c67700 1 --2- 172.21.15.36:0/3557595556 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4340b27b0 0x7fe434149f60 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 rx=0x7fe42c000f40 tx=0x7fe42c007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:11.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.048+0000 7fe431ffb700 1 -- 172.21.15.36:0/3557595556 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe42c005ae0 con 0x7fe4340b27b0 2022-06-17T13:58:11.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.048+0000 7fe431ffb700 1 -- 172.21.15.36:0/3557595556 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42c005c40 con 0x7fe4340b27b0 2022-06-17T13:58:11.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.048+0000 7fe439468700 1 -- 172.21.15.36:0/3557595556 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe434141510 con 0x7fe4340b27b0 2022-06-17T13:58:11.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.048+0000 7fe431ffb700 1 -- 172.21.15.36:0/3557595556 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe42c0070d0 con 0x7fe4340b27b0 2022-06-17T13:58:11.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.048+0000 7fe439468700 1 -- 172.21.15.36:0/3557595556 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe43413d870 con 0x7fe4340b27b0 2022-06-17T13:58:11.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.048+0000 7fe431ffb700 1 -- 172.21.15.36:0/3557595556 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fe42c01a060 con 0x7fe4340b27b0 2022-06-17T13:58:11.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.048+0000 7fe439468700 1 -- 172.21.15.36:0/3557595556 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe4340ac430 con 0x7fe4340b27b0 2022-06-17T13:58:11.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.052+0000 7fe431ffb700 1 --2- 172.21.15.36:0/3557595556 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe418053a60 0x7fe418055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:11.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.052+0000 7fe433fff700 1 --2- 172.21.15.36:0/3557595556 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe418053a60 0x7fe418055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:11.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.052+0000 7fe431ffb700 1 -- 172.21.15.36:0/3557595556 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fe42c06b6a0 con 0x7fe4340b27b0 2022-06-17T13:58:11.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.052+0000 7fe431ffb700 1 --2- 172.21.15.36:0/3557595556 >> [v2:172.21.15.70:6824/22837,v1:172.21.15.70:6825/22837] conn(0x7fe41805a6f0 0x7fe41805cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:11.065 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.052+0000 7fe431ffb700 1 -- 172.21.15.36:0/3557595556 --> [v2:172.21.15.70:6824/22837,v1:172.21.15.70:6825/22837] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe41805d170 con 0x7fe41805a6f0 2022-06-17T13:58:11.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.052+0000 7fe431ffb700 1 -- 172.21.15.36:0/3557595556 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe42c06dcb0 con 0x7fe4340b27b0 2022-06-17T13:58:11.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.052+0000 7fe433fff700 1 --2- 172.21.15.36:0/3557595556 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe418053a60 0x7fe418055f20 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7fe42400d000 tx=0x7fe424009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:11.066 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.052+0000 7fe42b7fe700 1 --2- 172.21.15.36:0/3557595556 >> [v2:172.21.15.70:6824/22837,v1:172.21.15.70:6825/22837] conn(0x7fe41805a6f0 0x7fe41805cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:11.067 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.052+0000 7fe42b7fe700 1 --2- 172.21.15.36:0/3557595556 >> [v2:172.21.15.70:6824/22837,v1:172.21.15.70:6825/22837] conn(0x7fe41805a6f0 0x7fe41805cb90 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:11.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.052+0000 7fe431ffb700 1 -- 172.21.15.36:0/3557595556 <== osd.7 v2:172.21.15.70:6824/22837 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7fe41805d170 con 0x7fe41805a6f0 2022-06-17T13:58:11.068 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.068+0000 7fe439468700 1 -- 172.21.15.36:0/3557595556 --> [v2:172.21.15.70:6824/22837,v1:172.21.15.70:6825/22837] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe434057410 con 0x7fe41805a6f0 2022-06-17T13:58:11.069 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.068+0000 7fe431ffb700 1 -- 172.21.15.36:0/3557595556 <== osd.7 v2:172.21.15.70:6824/22837 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe434057410 con 0x7fe41805a6f0 2022-06-17T13:58:11.069 INFO:teuthology.orchestra.run.smithi036.stdout:{ 2022-06-17T13:58:11.070 INFO:teuthology.orchestra.run.smithi036.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-06-17T13:58:11.070 INFO:teuthology.orchestra.run.smithi036.stdout:} 2022-06-17T13:58:11.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.068+0000 7fe439468700 1 -- 172.21.15.36:0/3557595556 >> [v2:172.21.15.70:6824/22837,v1:172.21.15.70:6825/22837] conn(0x7fe41805a6f0 msgr2=0x7fe41805cb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:11.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.068+0000 7fe439468700 1 --2- 172.21.15.36:0/3557595556 >> [v2:172.21.15.70:6824/22837,v1:172.21.15.70:6825/22837] conn(0x7fe41805a6f0 0x7fe41805cb90 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:11.072 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.068+0000 7fe439468700 1 -- 172.21.15.36:0/3557595556 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe418053a60 msgr2=0x7fe418055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:11.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.068+0000 7fe439468700 1 --2- 172.21.15.36:0/3557595556 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fe418053a60 0x7fe418055f20 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7fe42400d000 tx=0x7fe424009040).stop 2022-06-17T13:58:11.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.068+0000 7fe439468700 1 -- 172.21.15.36:0/3557595556 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4340b27b0 msgr2=0x7fe434149f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:11.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.068+0000 7fe439468700 1 --2- 172.21.15.36:0/3557595556 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe4340b27b0 0x7fe434149f60 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 rx=0x7fe42c000f40 tx=0x7fe42c007f90).stop 2022-06-17T13:58:11.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.068+0000 7fe438c67700 1 -- 172.21.15.36:0/3557595556 reap_dead start 2022-06-17T13:58:11.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.068+0000 7fe439468700 1 -- 172.21.15.36:0/3557595556 shutdown_connections 2022-06-17T13:58:11.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.068+0000 7fe439468700 1 -- 172.21.15.36:0/3557595556 >> 172.21.15.36:0/3557595556 conn(0x7fe4340b5d10 msgr2=0x7fe4340b4820 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:11.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.068+0000 7fe439468700 1 -- 172.21.15.36:0/3557595556 shutdown_connections 2022-06-17T13:58:11.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.068+0000 7fe439468700 1 -- 172.21.15.36:0/3557595556 wait complete. 2022-06-17T13:58:11.086 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2022-06-17T13:58:11.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.204+0000 7fa970949700 1 Processor -- start 2022-06-17T13:58:11.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa970949700 1 -- start start 2022-06-17T13:58:11.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa970949700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa96c0b2200 0x7fa96c0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:11.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa970949700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa96c0b2cd0 0x7fa96c147600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:11.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa970949700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa96c0b25e0 con 0x7fa96c0af8d0 2022-06-17T13:58:11.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa970949700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa96c0ad790 con 0x7fa96c0b2200 2022-06-17T13:58:11.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa970949700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c0aef40 con 0x7fa96c0b2cd0 2022-06-17T13:58:11.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa96ad9d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa96c0b2200 0x7fa96c0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:11.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa96ad9d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa96c0b2200 0x7fa96c0af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41422/0 (socket says 172.21.15.36:41422) 2022-06-17T13:58:11.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa96ad9d700 1 -- 172.21.15.36:0/3510737220 learned_addr learned my addr 172.21.15.36:0/3510737220 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:11.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa96b59e700 1 --2- 172.21.15.36:0/3510737220 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa96c0b2cd0 0x7fa96c147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:11.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa96ad9d700 1 -- 172.21.15.36:0/3510737220 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa96c0b2cd0 msgr2=0x7fa96c147600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T13:58:11.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa96ad9d700 1 --2- 172.21.15.36:0/3510737220 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa96c0b2cd0 0x7fa96c147600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:11.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa96ad9d700 1 -- 172.21.15.36:0/3510737220 >> v1:172.21.15.70:6789/0 conn(0x7fa96c0af8d0 legacy=0x7fa96c14a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:11.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa96ad9d700 1 -- 172.21.15.36:0/3510737220 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c0b1950 con 0x7fa96c0b2200 2022-06-17T13:58:11.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa96a59c700 1 -- 172.21.15.36:0/3510737220 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1655577674 0 0) 0x7fa96c0b25e0 con 0x7fa96c0af8d0 2022-06-17T13:58:11.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa96ad9d700 1 --2- 172.21.15.36:0/3510737220 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa96c0b2200 0x7fa96c0af3e0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 rx=0x7fa95c002f70 tx=0x7fa95c006eb0).ready entity=mon.0 client_cookie=bce17b967b6e8bf0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:11.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa96a59c700 1 -- 172.21.15.36:0/3510737220 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa95c00d570 con 0x7fa96c0b2200 2022-06-17T13:58:11.225 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa96a59c700 1 -- 172.21.15.36:0/3510737220 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95c000e90 con 0x7fa96c0b2200 2022-06-17T13:58:11.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa96a59c700 1 -- 172.21.15.36:0/3510737220 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa95c00d8c0 con 0x7fa96c0b2200 2022-06-17T13:58:11.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa970949700 1 -- 172.21.15.36:0/3510737220 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa96c0b2200 msgr2=0x7fa96c0af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:11.226 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa970949700 1 --2- 172.21.15.36:0/3510737220 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa96c0b2200 0x7fa96c0af3e0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 rx=0x7fa95c002f70 tx=0x7fa95c006eb0).stop 2022-06-17T13:58:11.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa970949700 1 -- 172.21.15.36:0/3510737220 shutdown_connections 2022-06-17T13:58:11.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa970949700 1 --2- 172.21.15.36:0/3510737220 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa96c0b2cd0 0x7fa96c147600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:11.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa970949700 1 --2- 172.21.15.36:0/3510737220 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa96c0b2200 0x7fa96c0af3e0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:11.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa970949700 1 -- 172.21.15.36:0/3510737220 >> 172.21.15.36:0/3510737220 conn(0x7fa96c05f770 msgr2=0x7fa96c05fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:11.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa970949700 1 -- 172.21.15.36:0/3510737220 shutdown_connections 2022-06-17T13:58:11.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.208+0000 7fa970949700 1 -- 172.21.15.36:0/3510737220 wait complete. 2022-06-17T13:58:11.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.212+0000 7fa970949700 1 Processor -- start 2022-06-17T13:58:11.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.212+0000 7fa970949700 1 -- start start 2022-06-17T13:58:11.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.212+0000 7fa970949700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa96c0af8d0 0x7fa96c1566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:11.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.212+0000 7fa970949700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa96c0b2200 0x7fa96c14abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:11.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.212+0000 7fa970949700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa96c0b2cd0 0x7fa96c142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:11.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.212+0000 7fa970949700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c156bd0 con 0x7fa96c0b2cd0 2022-06-17T13:58:11.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.212+0000 7fa970949700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c0a7580 con 0x7fa96c0af8d0 2022-06-17T13:58:11.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.212+0000 7fa970949700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c0a7820 con 0x7fa96c0b2200 2022-06-17T13:58:11.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.212+0000 7fa96b59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa96c0b2cd0 0x7fa96c142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:11.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.212+0000 7fa96ad9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa96c0af8d0 0x7fa96c1566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:11.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.212+0000 7fa96b59e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa96c0b2cd0 0x7fa96c142250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41430/0 (socket says 172.21.15.36:41430) 2022-06-17T13:58:11.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.212+0000 7fa96b59e700 1 -- 172.21.15.36:0/2561800648 learned_addr learned my addr 172.21.15.36:0/2561800648 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:11.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.212+0000 7fa963fff700 1 --2- 172.21.15.36:0/2561800648 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa96c0b2200 0x7fa96c14abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:11.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.212+0000 7fa96b59e700 1 -- 172.21.15.36:0/2561800648 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa96c0b2200 msgr2=0x7fa96c14abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:11.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.212+0000 7fa96b59e700 1 --2- 172.21.15.36:0/2561800648 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa96c0b2200 0x7fa96c14abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:11.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.212+0000 7fa96b59e700 1 -- 172.21.15.36:0/2561800648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa96c0af8d0 msgr2=0x7fa96c1566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:11.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.212+0000 7fa96b59e700 1 --2- 172.21.15.36:0/2561800648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa96c0af8d0 0x7fa96c1566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:11.235 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.212+0000 7fa96b59e700 1 -- 172.21.15.36:0/2561800648 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95c00b1e0 con 0x7fa96c0b2cd0 2022-06-17T13:58:11.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.212+0000 7fa96b59e700 1 --2- 172.21.15.36:0/2561800648 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa96c0b2cd0 0x7fa96c142250 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 rx=0x7fa964006e00 tx=0x7fa9640091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:11.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.212+0000 7fa968d99700 1 -- 172.21.15.36:0/2561800648 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa964012600 con 0x7fa96c0b2cd0 2022-06-17T13:58:11.236 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.212+0000 7fa968d99700 1 -- 172.21.15.36:0/2561800648 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa964012760 con 0x7fa96c0b2cd0 2022-06-17T13:58:11.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.212+0000 7fa968d99700 1 -- 172.21.15.36:0/2561800648 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa964012ac0 con 0x7fa96c0b2cd0 2022-06-17T13:58:11.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.212+0000 7fa970949700 1 -- 172.21.15.36:0/2561800648 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa96c159c70 con 0x7fa96c0b2cd0 2022-06-17T13:58:11.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.212+0000 7fa970949700 1 -- 172.21.15.36:0/2561800648 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa96c145d40 con 0x7fa96c0b2cd0 2022-06-17T13:58:11.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.216+0000 7fa970949700 1 -- 172.21.15.36:0/2561800648 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa96c159e00 con 0x7fa96c0b2cd0 2022-06-17T13:58:11.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.216+0000 7fa968d99700 1 -- 172.21.15.36:0/2561800648 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7fa964022070 con 0x7fa96c0b2cd0 2022-06-17T13:58:11.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.216+0000 7fa968d99700 1 --2- 172.21.15.36:0/2561800648 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa950053af0 0x7fa950055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:11.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.216+0000 7fa968d99700 1 -- 172.21.15.36:0/2561800648 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fa96406c2f0 con 0x7fa96c0b2cd0 2022-06-17T13:58:11.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.216+0000 7fa96ad9d700 1 --2- 172.21.15.36:0/2561800648 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa950053af0 0x7fa950055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:11.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.216+0000 7fa96ad9d700 1 --2- 172.21.15.36:0/2561800648 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa950053af0 0x7fa950055fb0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7fa95c01c000 tx=0x7fa95c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:11.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.224+0000 7fa968d99700 1 -- 172.21.15.36:0/2561800648 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa964043330 con 0x7fa96c0b2cd0 2022-06-17T13:58:11.349 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.348+0000 7fa970949700 1 -- 172.21.15.36:0/2561800648 --> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fa96c0ad330 con 0x7fa950053af0 2022-06-17T13:58:11.350 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.352+0000 7fa968d99700 1 -- 172.21.15.36:0/2561800648 <== mgr.4104 v2:172.21.15.36:6800/22419 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa96c0ad330 con 0x7fa950053af0 2022-06-17T13:58:11.352 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.352+0000 7fa970949700 1 -- 172.21.15.36:0/2561800648 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa950053af0 msgr2=0x7fa950055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:11.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.352+0000 7fa970949700 1 --2- 172.21.15.36:0/2561800648 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa950053af0 0x7fa950055fb0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7fa95c01c000 tx=0x7fa95c015000).stop 2022-06-17T13:58:11.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.352+0000 7fa970949700 1 -- 172.21.15.36:0/2561800648 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa96c0b2cd0 msgr2=0x7fa96c142250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:11.353 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.352+0000 7fa970949700 1 --2- 172.21.15.36:0/2561800648 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa96c0b2cd0 0x7fa96c142250 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 rx=0x7fa964006e00 tx=0x7fa9640091d0).stop 2022-06-17T13:58:11.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.352+0000 7fa970949700 1 -- 172.21.15.36:0/2561800648 shutdown_connections 2022-06-17T13:58:11.354 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.352+0000 7fa970949700 1 --2- 172.21.15.36:0/2561800648 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fa950053af0 0x7fa950055fb0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:11.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.352+0000 7fa970949700 1 --2- 172.21.15.36:0/2561800648 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa96c0b2cd0 0x7fa96c142250 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:11.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.352+0000 7fa970949700 1 --2- 172.21.15.36:0/2561800648 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa96c0b2200 0x7fa96c14abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:11.355 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.352+0000 7fa970949700 1 --2- 172.21.15.36:0/2561800648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa96c0af8d0 0x7fa96c1566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:11.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.352+0000 7fa970949700 1 -- 172.21.15.36:0/2561800648 >> 172.21.15.36:0/2561800648 conn(0x7fa96c05f770 msgr2=0x7fa96c0b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:11.356 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.352+0000 7fa970949700 1 -- 172.21.15.36:0/2561800648 shutdown_connections 2022-06-17T13:58:11.357 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.352+0000 7fa970949700 1 -- 172.21.15.36:0/2561800648 wait complete. 2022-06-17T13:58:11.357 INFO:teuthology.orchestra.run.smithi036.stderr:instructed osd(s) 7 to deep-scrub 2022-06-17T13:58:11.363 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-06-17T13:58:11.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.468+0000 7f5ff7fff700 1 Processor -- start 2022-06-17T13:58:11.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.468+0000 7f5ff7fff700 1 -- start start 2022-06-17T13:58:11.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.468+0000 7f5ff7fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ff80b2200 0x7f5ff80af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:11.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.468+0000 7f5ff7fff700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5ff80b2cd0 0x7f5ff8147600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:11.473 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.468+0000 7f5ff7fff700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff80b25e0 con 0x7f5ff80af8d0 2022-06-17T13:58:11.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.468+0000 7f5ff7fff700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff80ad790 con 0x7f5ff80b2cd0 2022-06-17T13:58:11.474 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.468+0000 7f5ff7fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff80aef40 con 0x7f5ff80b2200 2022-06-17T13:58:11.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.468+0000 7f5ff77fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5ff80b2cd0 0x7f5ff8147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:11.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.468+0000 7f5ff67fc700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f5ff80af8d0 0x7f5ff814a8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58964/0 (socket says 172.21.15.36:58964) 2022-06-17T13:58:11.475 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.468+0000 7f5ff67fc700 1 -- 172.21.15.36:0/2548071821 learned_addr learned my addr 172.21.15.36:0/2548071821 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T13:58:11.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.468+0000 7f5ff6ffd700 1 --2- 172.21.15.36:0/2548071821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ff80b2200 0x7f5ff80af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:11.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.468+0000 7f5ff5ffb700 1 -- 172.21.15.36:0/2548071821 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 15224181 0 0) 0x7f5ff80b25e0 con 0x7f5ff80af8d0 2022-06-17T13:58:11.476 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.468+0000 7f5ff5ffb700 1 -- 172.21.15.36:0/2548071821 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fdc003260 con 0x7f5ff80af8d0 2022-06-17T13:58:11.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.468+0000 7f5ff77fe700 1 -- 172.21.15.36:0/2548071821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ff80b2200 msgr2=0x7f5ff80af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T13:58:11.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.468+0000 7f5ff77fe700 1 --2- 172.21.15.36:0/2548071821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ff80b2200 0x7f5ff80af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:11.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.468+0000 7f5ff77fe700 1 -- 172.21.15.36:0/2548071821 >> v1:172.21.15.36:6789/0 conn(0x7f5ff80af8d0 legacy=0x7f5ff814a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:11.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.468+0000 7f5ff77fe700 1 -- 172.21.15.36:0/2548071821 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff80b1950 con 0x7f5ff80b2cd0 2022-06-17T13:58:11.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.468+0000 7f5ff77fe700 1 --2- 172.21.15.36:0/2548071821 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5ff80b2cd0 0x7f5ff8147600 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 rx=0x7f5ff00032f0 tx=0x7f5ff00091d0).ready entity=mon.0 client_cookie=983352e9c2461d4e server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:11.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.472+0000 7f5ff5ffb700 1 -- 172.21.15.36:0/2548071821 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ff000b600 con 0x7f5ff80b2cd0 2022-06-17T13:58:11.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.472+0000 7f5ff5ffb700 1 -- 172.21.15.36:0/2548071821 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff000b760 con 0x7f5ff80b2cd0 2022-06-17T13:58:11.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.472+0000 7f5ff5ffb700 1 -- 172.21.15.36:0/2548071821 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ff000b8d0 con 0x7f5ff80b2cd0 2022-06-17T13:58:11.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.472+0000 7f5ff7fff700 1 -- 172.21.15.36:0/2548071821 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5ff80b2cd0 msgr2=0x7f5ff8147600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:11.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.472+0000 7f5ff7fff700 1 --2- 172.21.15.36:0/2548071821 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5ff80b2cd0 0x7f5ff8147600 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 rx=0x7f5ff00032f0 tx=0x7f5ff00091d0).stop 2022-06-17T13:58:11.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.472+0000 7f5ff7fff700 1 -- 172.21.15.36:0/2548071821 shutdown_connections 2022-06-17T13:58:11.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.472+0000 7f5ff7fff700 1 --2- 172.21.15.36:0/2548071821 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5ff80b2cd0 0x7f5ff8147600 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:11.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.472+0000 7f5ff7fff700 1 --2- 172.21.15.36:0/2548071821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ff80b2200 0x7f5ff80af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:11.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.472+0000 7f5ff7fff700 1 -- 172.21.15.36:0/2548071821 >> 172.21.15.36:0/2548071821 conn(0x7f5ff805f770 msgr2=0x7f5ff805fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:11.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.472+0000 7f5ff7fff700 1 -- 172.21.15.36:0/2548071821 shutdown_connections 2022-06-17T13:58:11.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.472+0000 7f5ff7fff700 1 -- 172.21.15.36:0/2548071821 wait complete. 2022-06-17T13:58:11.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.472+0000 7f5ff7fff700 1 Processor -- start 2022-06-17T13:58:11.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.472+0000 7f5ff7fff700 1 -- start start 2022-06-17T13:58:11.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.472+0000 7f5ff7fff700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ff80af8d0 0x7f5ff80a95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:11.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.472+0000 7f5ff7fff700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5ff80b2200 0x7f5ff814d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:11.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.472+0000 7f5ff7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ff80b2cd0 0x7f5ff8142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:11.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.472+0000 7f5ff7fff700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff80a7460 con 0x7f5ff80b2200 2022-06-17T13:58:11.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.472+0000 7f5ff7fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff80a7700 con 0x7f5ff80b2cd0 2022-06-17T13:58:11.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.472+0000 7f5ff7fff700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff80a79a0 con 0x7f5ff80af8d0 2022-06-17T13:58:11.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.472+0000 7f5ff67fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5ff80b2200 0x7f5ff814d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:11.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.476+0000 7f5ff67fc700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5ff80b2200 0x7f5ff814d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41442/0 (socket says 172.21.15.36:41442) 2022-06-17T13:58:11.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.476+0000 7f5ff67fc700 1 -- 172.21.15.36:0/4184823302 learned_addr learned my addr 172.21.15.36:0/4184823302 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:11.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.476+0000 7f5ff6ffd700 1 --2- 172.21.15.36:0/4184823302 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ff80af8d0 0x7f5ff80a95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:11.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.476+0000 7f5ff77fe700 1 --2- 172.21.15.36:0/4184823302 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ff80b2cd0 0x7f5ff8142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:11.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.476+0000 7f5ff67fc700 1 -- 172.21.15.36:0/4184823302 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ff80af8d0 msgr2=0x7f5ff80a95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:11.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.476+0000 7f5ff67fc700 1 --2- 172.21.15.36:0/4184823302 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ff80af8d0 0x7f5ff80a95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:11.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.476+0000 7f5ff67fc700 1 -- 172.21.15.36:0/4184823302 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ff80b2cd0 msgr2=0x7f5ff8142250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:11.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.476+0000 7f5ff67fc700 1 --2- 172.21.15.36:0/4184823302 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ff80b2cd0 0x7f5ff8142250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:11.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.476+0000 7f5ff67fc700 1 -- 172.21.15.36:0/4184823302 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff0009470 con 0x7f5ff80b2200 2022-06-17T13:58:11.494 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.476+0000 7f5ff67fc700 1 --2- 172.21.15.36:0/4184823302 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5ff80b2200 0x7f5ff814d210 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 rx=0x7f5fe8001110 tx=0x7f5fe8007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:11.495 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.476+0000 7f5fe7fff700 1 -- 172.21.15.36:0/4184823302 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fe80070d0 con 0x7f5ff80b2200 2022-06-17T13:58:11.496 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.476+0000 7f5fe7fff700 1 -- 172.21.15.36:0/4184823302 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe8007230 con 0x7f5ff80b2200 2022-06-17T13:58:11.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.476+0000 7f5fe7fff700 1 -- 172.21.15.36:0/4184823302 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fe8005610 con 0x7f5ff80b2200 2022-06-17T13:58:11.497 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.476+0000 7f5ff7fff700 1 -- 172.21.15.36:0/4184823302 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff8144150 con 0x7f5ff80b2200 2022-06-17T13:58:11.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.476+0000 7f5ff7fff700 1 -- 172.21.15.36:0/4184823302 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff8142740 con 0x7f5ff80b2200 2022-06-17T13:58:11.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.476+0000 7f5ff7fff700 1 -- 172.21.15.36:0/4184823302 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff80ad330 con 0x7f5ff80b2200 2022-06-17T13:58:11.498 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.476+0000 7f5fe7fff700 1 -- 172.21.15.36:0/4184823302 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f5fe8010070 con 0x7f5ff80b2200 2022-06-17T13:58:11.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.480+0000 7f5fe7fff700 1 --2- 172.21.15.36:0/4184823302 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5fd40539b0 0x7f5fd4055e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:11.499 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.480+0000 7f5fe7fff700 1 -- 172.21.15.36:0/4184823302 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f5fe806b430 con 0x7f5ff80b2200 2022-06-17T13:58:11.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.480+0000 7f5ff6ffd700 1 --2- 172.21.15.36:0/4184823302 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5fd40539b0 0x7f5fd4055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:11.500 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.480+0000 7f5ff6ffd700 1 --2- 172.21.15.36:0/4184823302 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5fd40539b0 0x7f5fd4055e70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7f5fec00b000 tx=0x7f5fec006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:11.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.484+0000 7f5fe7fff700 1 -- 172.21.15.36:0/4184823302 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5fe8042510 con 0x7f5ff80b2200 2022-06-17T13:58:11.604 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.604+0000 7f5ff7fff700 1 -- 172.21.15.36:0/4184823302 --> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ff81429c0 con 0x7f5fd40539b0 2022-06-17T13:58:11.610 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.612+0000 7f5fe7fff700 1 -- 172.21.15.36:0/4184823302 <== mgr.4104 v2:172.21.15.36:6800/22419 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+67937 (secure 0 0 0) 0x7f5fec0116f0 con 0x7f5fd40539b0 2022-06-17T13:58:11.612 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T13:58:11.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.612+0000 7f5ff7fff700 1 -- 172.21.15.36:0/4184823302 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5fd40539b0 msgr2=0x7f5fd4055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:11.614 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.612+0000 7f5ff7fff700 1 --2- 172.21.15.36:0/4184823302 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5fd40539b0 0x7f5fd4055e70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7f5fec00b000 tx=0x7f5fec006df0).stop 2022-06-17T13:58:11.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.612+0000 7f5ff7fff700 1 -- 172.21.15.36:0/4184823302 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5ff80b2200 msgr2=0x7f5ff814d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:11.615 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.612+0000 7f5ff7fff700 1 --2- 172.21.15.36:0/4184823302 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5ff80b2200 0x7f5ff814d210 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 rx=0x7f5fe8001110 tx=0x7f5fe8007aa0).stop 2022-06-17T13:58:11.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.612+0000 7f5ff7fff700 1 -- 172.21.15.36:0/4184823302 shutdown_connections 2022-06-17T13:58:11.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.612+0000 7f5ff7fff700 1 --2- 172.21.15.36:0/4184823302 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f5fd40539b0 0x7f5fd4055e70 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:11.616 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.612+0000 7f5ff7fff700 1 --2- 172.21.15.36:0/4184823302 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ff80b2cd0 0x7f5ff8142250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:11.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.612+0000 7f5ff7fff700 1 --2- 172.21.15.36:0/4184823302 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5ff80b2200 0x7f5ff814d210 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:11.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.612+0000 7f5ff7fff700 1 --2- 172.21.15.36:0/4184823302 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ff80af8d0 0x7f5ff80a95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:11.617 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.612+0000 7f5ff7fff700 1 -- 172.21.15.36:0/4184823302 >> 172.21.15.36:0/4184823302 conn(0x7f5ff805f770 msgr2=0x7f5ff8147d20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:11.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.612+0000 7f5ff7fff700 1 -- 172.21.15.36:0/4184823302 shutdown_connections 2022-06-17T13:58:11.618 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:11.612+0000 7f5ff7fff700 1 -- 172.21.15.36:0/4184823302 wait complete. 2022-06-17T13:58:11.619 INFO:teuthology.orchestra.run.smithi036.stderr:dumped all 2022-06-17T13:58:11.626 INFO:teuthology.orchestra.run.smithi036.stdout:{"pg_ready":true,"pg_map":{"version":2300,"stamp":"2022-06-17T13:58:10.690595+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":8404000,"kb_used_data":15328,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746570720,"statfs":{"total":773094113280,"available":764488417280,"internally_reserved":8589934592,"allocated":15695872,"data_stored":7700384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2742,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-46,"num_write_kb":-14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.554850"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":138,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-06-17T12:57:34.240894+0000","last_change":"2022-06-17T12:46:41.622624+0000","last_active":"2022-06-17T12:57:34.240894+0000","last_peered":"2022-06-17T12:57:34.240894+0000","last_clean":"2022-06-17T12:57:34.240894+0000","last_became_active":"2022-06-17T12:46:41.622138+0000","last_became_peered":"2022-06-17T12:46:41.622138+0000","last_unstale":"2022-06-17T12:57:34.240894+0000","last_undegraded":"2022-06-17T12:57:34.240894+0000","last_fullsized":"2022-06-17T12:57:34.240894+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":4,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:34.095313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:34.095313+0000","last_clean_scrub_stamp":"2022-06-17T12:46:34.095313+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":138,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-06-17T12:57:34.240542+0000","last_change":"2022-06-17T12:46:45.658148+0000","last_active":"2022-06-17T12:57:34.240542+0000","last_peered":"2022-06-17T12:57:34.240542+0000","last_clean":"2022-06-17T12:57:34.240542+0000","last_became_active":"2022-06-17T12:46:45.657682+0000","last_became_peered":"2022-06-17T12:46:45.657682+0000","last_unstale":"2022-06-17T12:57:34.240542+0000","last_undegraded":"2022-06-17T12:57:34.240542+0000","last_fullsized":"2022-06-17T12:57:34.240542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":135,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-06-17T12:57:34.240570+0000","last_change":"2022-06-17T12:46:45.662621+0000","last_active":"2022-06-17T12:57:34.240570+0000","last_peered":"2022-06-17T12:57:34.240570+0000","last_clean":"2022-06-17T12:57:34.240570+0000","last_became_active":"2022-06-17T12:46:45.662293+0000","last_became_peered":"2022-06-17T12:46:45.662293+0000","last_unstale":"2022-06-17T12:57:34.240570+0000","last_undegraded":"2022-06-17T12:57:34.240570+0000","last_fullsized":"2022-06-17T12:57:34.240570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":136,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-06-17T12:57:34.240237+0000","last_change":"2022-06-17T12:46:46.357931+0000","last_active":"2022-06-17T12:57:34.240237+0000","last_peered":"2022-06-17T12:57:34.240237+0000","last_clean":"2022-06-17T12:57:34.240237+0000","last_became_active":"2022-06-17T12:46:46.356789+0000","last_became_peered":"2022-06-17T12:46:46.356789+0000","last_unstale":"2022-06-17T12:57:34.240237+0000","last_undegraded":"2022-06-17T12:57:34.240237+0000","last_fullsized":"2022-06-17T12:57:34.240237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":138,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-06-17T12:57:34.240588+0000","last_change":"2022-06-17T12:46:45.662903+0000","last_active":"2022-06-17T12:57:34.240588+0000","last_peered":"2022-06-17T12:57:34.240588+0000","last_clean":"2022-06-17T12:57:34.240588+0000","last_became_active":"2022-06-17T12:46:45.662460+0000","last_became_peered":"2022-06-17T12:46:45.662460+0000","last_unstale":"2022-06-17T12:57:34.240588+0000","last_undegraded":"2022-06-17T12:57:34.240588+0000","last_fullsized":"2022-06-17T12:57:34.240588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":137,"reported_epoch":154,"state":"active+clean","last_fresh":"2022-06-17T13:58:09.566370+0000","last_change":"2022-06-17T13:58:09.566558+0000","last_active":"2022-06-17T13:58:09.566370+0000","last_peered":"2022-06-17T13:58:09.566370+0000","last_clean":"2022-06-17T13:58:09.566370+0000","last_became_active":"2022-06-17T12:46:46.260143+0000","last_became_peered":"2022-06-17T12:46:46.260143+0000","last_unstale":"2022-06-17T13:58:09.566370+0000","last_undegraded":"2022-06-17T13:58:09.566370+0000","last_fullsized":"2022-06-17T13:58:09.566370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T13:58:09.566369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T13:58:09.566369+0000","last_clean_scrub_stamp":"2022-06-17T13:58:09.566369+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":135,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-06-17T12:57:34.242051+0000","last_change":"2022-06-17T12:46:46.265378+0000","last_active":"2022-06-17T12:57:34.242051+0000","last_peered":"2022-06-17T12:57:34.242051+0000","last_clean":"2022-06-17T12:57:34.242051+0000","last_became_active":"2022-06-17T12:46:46.264499+0000","last_became_peered":"2022-06-17T12:46:46.264499+0000","last_unstale":"2022-06-17T12:57:34.242051+0000","last_undegraded":"2022-06-17T12:57:34.242051+0000","last_fullsized":"2022-06-17T12:57:34.242051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":134,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-06-17T12:57:34.262466+0000","last_change":"2022-06-17T12:46:45.658907+0000","last_active":"2022-06-17T12:57:34.262466+0000","last_peered":"2022-06-17T12:57:34.262466+0000","last_clean":"2022-06-17T12:57:34.262466+0000","last_became_active":"2022-06-17T12:46:45.658569+0000","last_became_peered":"2022-06-17T12:46:45.658569+0000","last_unstale":"2022-06-17T12:57:34.262466+0000","last_undegraded":"2022-06-17T12:57:34.262466+0000","last_fullsized":"2022-06-17T12:57:34.262466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":136,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-06-17T12:57:34.410767+0000","last_change":"2022-06-17T12:46:45.660735+0000","last_active":"2022-06-17T12:57:34.410767+0000","last_peered":"2022-06-17T12:57:34.410767+0000","last_clean":"2022-06-17T12:57:34.410767+0000","last_became_active":"2022-06-17T12:46:45.660343+0000","last_became_peered":"2022-06-17T12:46:45.660343+0000","last_unstale":"2022-06-17T12:57:34.410767+0000","last_undegraded":"2022-06-17T12:57:34.410767+0000","last_fullsized":"2022-06-17T12:57:34.410767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T12:46:44.630167+0000","last_clean_scrub_stamp":"2022-06-17T12:46:44.630167+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543003,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050500,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321340,"statfs":{"total":96636764160,"available":95561052160,"internally_reserved":1073741824,"allocated":1961984,"data_stored":962548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 17 13:57:03 2022","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.88400000000000001,"15min":0.88800000000000001},"min":{"1min":0.73199999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.006,"5min":1.1639999999999999,"15min":1.2270000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.92400000000000004,"15min":0.91400000000000003},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.57499999999999996},"max":{"1min":1.056,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.72099999999999997}]},{"osd":1,"last update":"Fri Jun 17 13:57:03 2022","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81399999999999995,"15min":0.81799999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0409999999999999,"5min":1.2110000000000001,"15min":1.2350000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.84099999999999997,"15min":0.81899999999999995},"min":{"1min":0.73899999999999999,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1100000000000001,"5min":1.1519999999999999,"15min":1.1559999999999999},"last":0.68500000000000005}]},{"osd":2,"last update":"Fri Jun 17 13:57:03 2022","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.86099999999999999,"15min":0.85199999999999998},"min":{"1min":0.60599999999999998,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.95099999999999996,"5min":1.1220000000000001,"15min":1.1759999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.83399999999999996,"15min":0.85999999999999999},"min":{"1min":0.64000000000000001,"5min":0.62,"15min":0.61799999999999999},"max":{"1min":1.0449999999999999,"5min":1.2010000000000001,"15min":1.23},"last":0.66300000000000003}]},{"osd":3,"last update":"Fri Jun 17 13:57:03 2022","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.86099999999999999,"15min":0.86499999999999999},"min":{"1min":0.64500000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.99299999999999999,"5min":1.496,"15min":1.496},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.877,"5min":0.89700000000000002,"15min":0.90900000000000003},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.61699999999999999},"max":{"1min":1.1259999999999999,"5min":1.28,"15min":1.28},"last":1.0549999999999999}]},{"osd":4,"last update":"Fri Jun 17 13:57:03 2022","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78300000000000003,"15min":0.79000000000000004},"min":{"1min":0.58099999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.2170000000000001,"5min":1.597,"15min":1.597},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.85799999999999998,"15min":0.88200000000000001},"min":{"1min":0.56899999999999995,"5min":0.56499999999999995,"15min":0.54600000000000004},"max":{"1min":1.119,"5min":1.623,"15min":1.623},"last":1.081}]},{"osd":5,"last update":"Fri Jun 17 13:57:03 2022","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.88500000000000001,"15min":0.89600000000000002},"min":{"1min":0.59799999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.2,"5min":1.611,"15min":1.611},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93300000000000005,"15min":0.94099999999999995},"min":{"1min":0.65800000000000003,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.155,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.99099999999999999}]},{"osd":6,"last update":"Fri Jun 17 13:57:03 2022","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.97599999999999998,"15min":0.98499999999999999},"min":{"1min":0.76000000000000001,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.278,"5min":1.696,"15min":1.696},"last":0.873},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.96799999999999997,"15min":1.002},"min":{"1min":0.69499999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0840000000000001,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.034}]}]},{"osd":6,"up_from":14,"seq":60129543004,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050500,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321340,"statfs":{"total":96636764160,"available":95561052160,"internally_reserved":1073741824,"allocated":1961984,"data_stored":962548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 17 13:58:02 2022","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.81399999999999995,"15min":0.81000000000000005},"min":{"1min":0.623,"5min":0.52800000000000002,"15min":0.48099999999999998},"max":{"1min":1.105,"5min":1.1220000000000001,"15min":1.5549999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85199999999999998,"15min":0.83799999999999997},"min":{"1min":0.63500000000000001,"5min":0.56000000000000005,"15min":0.52800000000000002},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.302},"last":1.036}]},{"osd":1,"last update":"Fri Jun 17 13:58:02 2022","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.81599999999999995,"15min":0.83299999999999996},"min":{"1min":0.61599999999999999,"5min":0.60499999999999998,"15min":0.56200000000000006},"max":{"1min":1.0780000000000001,"5min":1.095,"15min":1.1699999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.82899999999999996,"15min":0.84599999999999997},"min":{"1min":0.63600000000000001,"5min":0.59999999999999998,"15min":0.57299999999999995},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.4370000000000001},"last":0.89100000000000001}]},{"osd":2,"last update":"Fri Jun 17 13:58:02 2022","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.86099999999999999,"15min":0.83999999999999997},"min":{"1min":0.65300000000000002,"5min":0.64700000000000002,"15min":0.59399999999999997},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.3879999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.88,"5min":0.86799999999999999,"15min":0.86499999999999999},"min":{"1min":0.67700000000000005,"5min":0.64900000000000002,"15min":0.62},"max":{"1min":1.1120000000000001,"5min":1.224,"15min":1.2430000000000001},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Jun 17 13:58:02 2022","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.86899999999999999,"15min":0.878},"min":{"1min":0.71399999999999997,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.21,"5min":1.2170000000000001,"15min":1.296},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.92000000000000004,"15min":0.91300000000000003},"min":{"1min":0.754,"5min":0.65400000000000003,"15min":0.63800000000000001},"max":{"1min":1.1779999999999999,"5min":1.1799999999999999,"15min":1.339},"last":0.75600000000000001}]},{"osd":4,"last update":"Fri Jun 17 13:58:02 2022","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.80600000000000005,"15min":0.81299999999999994},"min":{"1min":0.57299999999999995,"5min":0.56399999999999995,"15min":0.53800000000000003},"max":{"1min":1.0289999999999999,"5min":1.2010000000000001,"15min":1.3580000000000001},"last":0.62},{"interface":"front","average":{"1min":1.04,"5min":0.92300000000000004,"15min":0.91100000000000003},"min":{"1min":0.80000000000000004,"5min":0.60399999999999998,"15min":0.59499999999999997},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.86799999999999999}]},{"osd":5,"last update":"Fri Jun 17 13:58:02 2022","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.879,"15min":0.88700000000000001},"min":{"1min":0.625,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.5009999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.94899999999999995,"15min":0.94799999999999995},"min":{"1min":0.63800000000000001,"5min":0.63400000000000001,"15min":0.59999999999999998},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.91100000000000003}]},{"osd":7,"last update":"Fri Jun 17 13:58:02 2022","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.97799999999999998,"15min":0.97599999999999998},"min":{"1min":0.71799999999999997,"5min":0.64700000000000002,"15min":0.61299999999999999},"max":{"1min":1.339,"5min":1.377,"15min":1.6910000000000001},"last":1.091},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.034,"15min":1.0329999999999999},"min":{"1min":0.83099999999999996,"5min":0.77400000000000002,"15min":0.70599999999999996},"max":{"1min":1.454,"5min":1.454,"15min":1.714},"last":1.0169999999999999}]}]},{"osd":3,"up_from":14,"seq":60129543005,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050500,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321340,"statfs":{"total":96636764160,"available":95561052160,"internally_reserved":1073741824,"allocated":1961984,"data_stored":962548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 17 13:57:57 2022","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.52600000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93999999999999995,"5min":1.0369999999999999,"15min":1.0900000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.72599999999999998,"15min":0.71399999999999997},"min":{"1min":0.52800000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93899999999999995,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.52100000000000002}]},{"osd":1,"last update":"Fri Jun 17 13:57:57 2022","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65900000000000003,"15min":0.65300000000000002},"min":{"1min":0.55000000000000004,"5min":0.48299999999999998,"15min":0.44900000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.2669999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76700000000000002,"15min":0.755},"min":{"1min":0.59799999999999998,"5min":0.54400000000000004,"15min":0.48599999999999999},"max":{"1min":1.0089999999999999,"5min":1.0629999999999999,"15min":1.266},"last":0.90600000000000003}]},{"osd":2,"last update":"Fri Jun 17 13:57:57 2022","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.80500000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0149999999999999,"5min":1.2250000000000001,"15min":1.7589999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.90700000000000003,"15min":0.90400000000000003},"min":{"1min":0.73799999999999999,"5min":0.73499999999999999,"15min":0.66600000000000004},"max":{"1min":1.157,"5min":1.157,"15min":1.704},"last":0.94799999999999995}]},{"osd":4,"last update":"Fri Jun 17 13:57:57 2022","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92700000000000005,"15min":0.93100000000000005},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.67200000000000004},"max":{"1min":1.234,"5min":1.3160000000000001,"15min":1.401},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.88600000000000001,"15min":0.90100000000000002},"min":{"1min":0.70599999999999996,"5min":0.68400000000000005,"15min":0.66500000000000004},"max":{"1min":1.1910000000000001,"5min":1.302,"15min":1.3979999999999999},"last":0.96899999999999997}]},{"osd":5,"last update":"Fri Jun 17 13:57:57 2022","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.97299999999999998,"15min":0.97499999999999998},"min":{"1min":0.80100000000000005,"5min":0.69799999999999995,"15min":0.68700000000000006},"max":{"1min":1.353,"5min":1.353,"15min":1.4790000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.01,"15min":1.0129999999999999},"min":{"1min":0.78500000000000003,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.218,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.93000000000000005}]},{"osd":6,"last update":"Fri Jun 17 13:57:57 2022","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.98899999999999999,"15min":0.996},"min":{"1min":0.77700000000000002,"5min":0.69299999999999995,"15min":0.68300000000000005},"max":{"1min":1.1899999999999999,"5min":1.3979999999999999,"15min":1.8500000000000001},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0940000000000001,"15min":1.079},"min":{"1min":0.89500000000000002,"5min":0.88700000000000001,"15min":0.72899999999999998},"max":{"1min":1.482,"5min":1.554,"15min":1.74},"last":0.99299999999999999}]},{"osd":7,"last update":"Fri Jun 17 13:57:57 2022","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.0920000000000001,"15min":1.0980000000000001},"min":{"1min":0.878,"5min":0.72099999999999997,"15min":0.71299999999999997},"max":{"1min":1.3100000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.087,"5min":1.0600000000000001,"15min":1.079},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.27,"5min":1.355,"15min":1.7450000000000001},"last":1.0569999999999999}]}]},{"osd":2,"up_from":14,"seq":60129543004,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050500,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321340,"statfs":{"total":96636764160,"available":95561052160,"internally_reserved":1073741824,"allocated":1961984,"data_stored":962548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 17 13:57:44 2022","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58699999999999997,"15min":0.60299999999999998},"min":{"1min":0.45900000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.143},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70499999999999996,"15min":0.71199999999999997},"min":{"1min":0.499,"5min":0.495,"15min":0.46200000000000002},"max":{"1min":1.028,"5min":1.0800000000000001,"15min":1.204},"last":0.76600000000000001}]},{"osd":1,"last update":"Fri Jun 17 13:57:44 2022","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.68200000000000005,"15min":0.69099999999999995},"min":{"1min":0.57099999999999995,"5min":0.499,"15min":0.48099999999999998},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.526},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72999999999999998,"15min":0.73399999999999999},"min":{"1min":0.57299999999999995,"5min":0.496,"15min":0.47699999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.228},"last":0.71699999999999997}]},{"osd":3,"last update":"Fri Jun 17 13:57:44 2022","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70399999999999996,"15min":0.71599999999999997},"min":{"1min":0.52500000000000002,"5min":0.503,"15min":0.48699999999999999},"max":{"1min":1.0489999999999999,"5min":1.056,"15min":1.2929999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.85899999999999999,"15min":0.85999999999999999},"min":{"1min":0.70499999999999996,"5min":0.64200000000000002,"15min":0.60799999999999998},"max":{"1min":1.167,"5min":1.3109999999999999,"15min":1.3560000000000001},"last":0.82299999999999995}]},{"osd":4,"last update":"Fri Jun 17 13:57:44 2022","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91700000000000004,"15min":0.92600000000000005},"min":{"1min":0.71699999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.2070000000000001,"5min":1.2689999999999999,"15min":1.4419999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.91700000000000004,"15min":0.94199999999999995},"min":{"1min":0.71799999999999997,"5min":0.71499999999999997,"15min":0.65900000000000003},"max":{"1min":1.0600000000000001,"5min":1.3660000000000001,"15min":1.4299999999999999},"last":1.171}]},{"osd":5,"last update":"Fri Jun 17 13:57:44 2022","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.93899999999999995,"15min":0.93300000000000005},"min":{"1min":0.76500000000000001,"5min":0.64200000000000002,"15min":0.629},"max":{"1min":1.329,"5min":1.3560000000000001,"15min":1.508},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97399999999999998,"15min":0.99199999999999999},"min":{"1min":0.79400000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.014}]},{"osd":6,"last update":"Fri Jun 17 13:57:44 2022","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.98699999999999999,"15min":1.002},"min":{"1min":0.76900000000000002,"5min":0.71699999999999997,"15min":0.69299999999999995},"max":{"1min":1.1259999999999999,"5min":1.4870000000000001,"15min":1.514},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.012,"5min":1.0069999999999999,"15min":1},"min":{"1min":0.79500000000000004,"5min":0.71599999999999997,"15min":0.68999999999999995},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.90200000000000002}]},{"osd":7,"last update":"Fri Jun 17 13:57:44 2022","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.038,"15min":1.0609999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.365,"5min":1.464,"15min":1.589},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.054,"15min":1.0529999999999999},"min":{"1min":0.88,"5min":0.78000000000000003,"15min":0.77600000000000002},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.073}]}]},{"osd":5,"up_from":13,"seq":55834575709,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050500,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321340,"statfs":{"total":96636764160,"available":95561052160,"internally_reserved":1073741824,"allocated":1961984,"data_stored":962548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 17 13:57:14 2022","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81699999999999995,"15min":0.83799999999999997},"min":{"1min":0.61699999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0169999999999999,"5min":1.1579999999999999,"15min":1.167},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.83199999999999996,"15min":0.82599999999999996},"min":{"1min":0.64300000000000002,"5min":0.60499999999999998,"15min":0.57099999999999995},"max":{"1min":1.0309999999999999,"5min":1.0920000000000001,"15min":1.1879999999999999},"last":0.80100000000000005}]},{"osd":1,"last update":"Fri Jun 17 13:57:14 2022","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.83399999999999996,"15min":0.84099999999999997},"min":{"1min":0.63400000000000001,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0029999999999999,"5min":1.1000000000000001,"15min":1.232},"last":0.745},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.80400000000000005,"15min":0.81299999999999994},"min":{"1min":0.65800000000000003,"5min":0.621,"15min":0.58699999999999997},"max":{"1min":0.95299999999999996,"5min":1.0920000000000001,"15min":1.254},"last":0.60499999999999998}]},{"osd":2,"last update":"Fri Jun 17 13:57:14 2022","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.83099999999999996,"15min":0.86499999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.008,"5min":1.093,"15min":1.236},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83999999999999997,"15min":0.83899999999999997},"min":{"1min":0.67600000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.0580000000000001,"5min":1.167,"15min":1.3100000000000001},"last":0.72099999999999997}]},{"osd":3,"last update":"Fri Jun 17 13:57:14 2022","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84799999999999998,"15min":0.88100000000000001},"min":{"1min":0.65500000000000003,"5min":0.61799999999999999,"15min":0.55000000000000004},"max":{"1min":1.151,"5min":1.151,"15min":1.3149999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0049999999999999,"15min":1.006},"min":{"1min":0.89100000000000001,"5min":0.81699999999999995,"15min":0.61499999999999999},"max":{"1min":1.143,"5min":1.161,"15min":1.244},"last":0.91900000000000004}]},{"osd":4,"last update":"Fri Jun 17 13:57:14 2022","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.76400000000000001,"15min":0.79200000000000004},"min":{"1min":0.60599999999999998,"5min":0.53300000000000003,"15min":0.52300000000000002},"max":{"1min":1.236,"5min":1.236,"15min":1.2549999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.88200000000000001,"15min":0.89700000000000002},"min":{"1min":0.65100000000000002,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.077,"5min":1.2110000000000001,"15min":1.2190000000000001},"last":0.95399999999999996}]},{"osd":6,"last update":"Fri Jun 17 13:57:14 2022","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.86199999999999999,"15min":0.88500000000000001},"min":{"1min":0.68100000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.95699999999999996,"15min":0.97399999999999998},"min":{"1min":0.70499999999999996,"5min":0.624,"15min":0.60499999999999998},"max":{"1min":1.095,"5min":1.208,"15min":1.236},"last":0.98799999999999999}]},{"osd":7,"last update":"Fri Jun 17 13:57:14 2022","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.872,"15min":0.88500000000000001},"min":{"1min":0.63600000000000001,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.361},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.98299999999999998,"15min":0.99099999999999999},"min":{"1min":0.82899999999999996,"5min":0.68999999999999995,"15min":0.68400000000000005},"max":{"1min":1.3420000000000001,"5min":1.361,"15min":1.397},"last":1.04}]}]},{"osd":4,"up_from":13,"seq":55834575708,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1050500,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321340,"statfs":{"total":96636764160,"available":95561052160,"internally_reserved":1073741824,"allocated":1961984,"data_stored":962548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 17 13:57:32 2022","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.91300000000000003,"15min":0.89200000000000002},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.496},"max":{"1min":1.0840000000000001,"5min":1.288,"15min":1.288},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.72899999999999998,"5min":0.66900000000000004,"15min":0.622},"max":{"1min":1.099,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.90300000000000002}]},{"osd":1,"last update":"Fri Jun 17 13:57:32 2022","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80800000000000005,"15min":0.79500000000000004},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.52500000000000002},"max":{"1min":1.046,"5min":1.1499999999999999,"15min":1.165},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.2,"5min":1.2,"15min":1.3100000000000001},"last":0.96099999999999997}]},{"osd":2,"last update":"Fri Jun 17 13:57:32 2022","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.85699999999999998,"15min":0.84999999999999998},"min":{"1min":0.71299999999999997,"5min":0.60399999999999998,"15min":0.58399999999999996},"max":{"1min":1.099,"5min":1.238,"15min":1.238},"last":0.75},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.83699999999999997,"15min":0.84599999999999997},"min":{"1min":0.627,"5min":0.627,"15min":0.621},"max":{"1min":0.99099999999999999,"5min":1.1459999999999999,"15min":1.218},"last":0.76900000000000002}]},{"osd":3,"last update":"Fri Jun 17 13:57:32 2022","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85699999999999998,"15min":0.874},"min":{"1min":0.66000000000000003,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.26},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.92300000000000004,"15min":0.90400000000000003},"min":{"1min":0.68999999999999995,"5min":0.63600000000000001,"15min":0.629},"max":{"1min":1.131,"5min":1.2350000000000001,"15min":1.2430000000000001},"last":0.84099999999999997}]},{"osd":5,"last update":"Fri Jun 17 13:57:32 2022","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.86199999999999999,"15min":0.86299999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.54900000000000004},"max":{"1min":1.038,"5min":1.1890000000000001,"15min":1.2729999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.871,"5min":0.89900000000000002,"15min":0.89600000000000002},"min":{"1min":0.56399999999999995,"5min":0.55600000000000005,"15min":0.51800000000000002},"max":{"1min":1.0389999999999999,"5min":1.1879999999999999,"15min":1.234},"last":0.92100000000000004}]},{"osd":6,"last update":"Fri Jun 17 13:57:32 2022","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.85799999999999998,"15min":0.85499999999999998},"min":{"1min":0.64600000000000002,"5min":0.58899999999999997,"15min":0.55800000000000005},"max":{"1min":1.1339999999999999,"5min":1.137,"15min":1.502},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.88400000000000001,"15min":0.90300000000000002},"min":{"1min":0.57699999999999996,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.256,"5min":1.276,"15min":1.5629999999999999},"last":0.872}]},{"osd":7,"last update":"Fri Jun 17 13:57:32 2022","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.91100000000000003,"15min":0.92000000000000004},"min":{"1min":0.63100000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.1839999999999999,"5min":1.2529999999999999,"15min":1.2849999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.93200000000000005,"15min":0.93700000000000006},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.59399999999999997},"max":{"1min":1.103,"5min":1.1930000000000001,"15min":1.268},"last":1.012}]}]},{"osd":1,"up_from":13,"seq":55834575709,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050500,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321340,"statfs":{"total":96636764160,"available":95561052160,"internally_reserved":1073741824,"allocated":1961984,"data_stored":962548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 17 13:57:38 2022","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.51200000000000001,"5min":0.47899999999999998,"15min":0.46999999999999997},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.75,"15min":0.755},"min":{"1min":0.65700000000000003,"5min":0.59299999999999997,"15min":0.56100000000000005},"max":{"1min":0.94899999999999995,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.60299999999999998}]},{"osd":2,"last update":"Fri Jun 17 13:57:18 2022","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.499,"5min":0.47599999999999998,"15min":0.45900000000000002},"max":{"1min":1.0369999999999999,"5min":1.105,"15min":1.784},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76000000000000001,"15min":0.77500000000000002},"min":{"1min":0.55800000000000005,"5min":0.5,"15min":0.47599999999999998},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.4830000000000001},"last":0.66800000000000004}]},{"osd":3,"last update":"Fri Jun 17 13:57:18 2022","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.66600000000000004,"15min":0.68400000000000005},"min":{"1min":0.52300000000000002,"5min":0.504,"15min":0.504},"max":{"1min":0.85199999999999998,"5min":1.0149999999999999,"15min":1.2050000000000001},"last":0.752},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.79000000000000004,"15min":0.79800000000000004},"min":{"1min":0.58399999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.871,"5min":1.1910000000000001,"15min":1.27},"last":0.83899999999999997}]},{"osd":4,"last update":"Fri Jun 17 13:57:38 2022","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94599999999999995,"15min":0.94199999999999995},"min":{"1min":0.77500000000000002,"5min":0.70599999999999996,"15min":0.66300000000000003},"max":{"1min":1.206,"5min":1.2729999999999999,"15min":1.3140000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.01,"5min":0.96899999999999997,"15min":0.97099999999999997},"min":{"1min":0.76600000000000001,"5min":0.65600000000000003,"15min":0.64900000000000002},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":1.1819999999999999}]},{"osd":5,"last update":"Fri Jun 17 13:57:38 2022","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94799999999999995,"15min":0.94999999999999996},"min":{"1min":0.81999999999999995,"5min":0.73699999999999999,"15min":0.68200000000000005},"max":{"1min":1.1060000000000001,"5min":1.371,"15min":1.371},"last":0.878},{"interface":"front","average":{"1min":0.995,"5min":1,"15min":0.999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.74199999999999999},"max":{"1min":1.1879999999999999,"5min":1.623,"15min":1.623},"last":1.0820000000000001}]},{"osd":6,"last update":"Fri Jun 17 13:57:18 2022","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0469999999999999,"15min":1.044},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.71499999999999997},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.603},"last":1.2749999999999999},{"interface":"front","average":{"1min":0.995,"5min":1.0249999999999999,"15min":1.0369999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.74399999999999999},"max":{"1min":1.2090000000000001,"5min":1.6679999999999999,"15min":1.75},"last":1.0580000000000001}]},{"osd":7,"last update":"Fri Jun 17 13:57:18 2022","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.052,"15min":1.0720000000000001},"min":{"1min":0.75900000000000001,"5min":0.748,"15min":0.748},"max":{"1min":1.24,"5min":1.6799999999999999,"15min":1.7929999999999999},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.1040000000000001,"15min":1.115},"min":{"1min":0.84899999999999998,"5min":0.78300000000000003,"15min":0.76300000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.6619999999999999},"last":1.321}]}]},{"osd":0,"up_from":13,"seq":55834575709,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050500,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321340,"statfs":{"total":96636764160,"available":95561052160,"internally_reserved":1073741824,"allocated":1961984,"data_stored":962548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 17 13:57:54 2022","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.61499999999999999,"15min":0.61799999999999999},"min":{"1min":0.47999999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.873,"5min":0.94699999999999995,"15min":1.208},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73899999999999999,"15min":0.72699999999999998},"min":{"1min":0.56299999999999994,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.097,"5min":1.1819999999999999,"15min":1.258},"last":0.64000000000000001}]},{"osd":2,"last update":"Fri Jun 17 13:57:54 2022","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.625,"15min":0.63100000000000001},"min":{"1min":0.495,"5min":0.46700000000000003,"15min":0.45100000000000001},"max":{"1min":0.85899999999999999,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.77600000000000002,"15min":0.76100000000000001},"min":{"1min":0.56299999999999994,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.96399999999999997,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.88400000000000001}]},{"osd":3,"last update":"Fri Jun 17 13:57:54 2022","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.82199999999999995,"15min":0.81100000000000005},"min":{"1min":0.623,"5min":0.58499999999999996,"15min":0.51000000000000001},"max":{"1min":1.036,"5min":1.1000000000000001,"15min":1.349},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81899999999999995,"15min":0.82299999999999995},"min":{"1min":0.65200000000000002,"5min":0.627,"15min":0.627},"max":{"1min":1.036,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.747}]},{"osd":4,"last update":"Fri Jun 17 13:57:54 2022","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.86399999999999999,"15min":0.879},"min":{"1min":0.66500000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.97499999999999998,"5min":1.145,"15min":1.2809999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.95299999999999996,"15min":0.95199999999999996},"min":{"1min":0.70699999999999996,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.2350000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.81999999999999995}]},{"osd":5,"last update":"Fri Jun 17 13:57:54 2022","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.99299999999999999,"15min":0.996},"min":{"1min":0.71699999999999997,"5min":0.69799999999999995,"15min":0.69099999999999995},"max":{"1min":1.464,"5min":1.5129999999999999,"15min":1.623},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.006,"5min":0.96199999999999997,"15min":0.95499999999999996},"min":{"1min":0.77000000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":1.091}]},{"osd":6,"last update":"Fri Jun 17 13:57:54 2022","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0529999999999999,"15min":1.0449999999999999},"min":{"1min":0.77300000000000002,"5min":0.748,"15min":0.69499999999999995},"max":{"1min":1.468,"5min":1.4910000000000001,"15min":1.51},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0920000000000001,"15min":1.1000000000000001},"min":{"1min":0.79900000000000004,"5min":0.74199999999999999,"15min":0.71999999999999997},"max":{"1min":1.3600000000000001,"5min":1.5740000000000001,"15min":1.6819999999999999},"last":0.97699999999999998}]},{"osd":7,"last update":"Fri Jun 17 13:57:54 2022","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.115,"15min":1.099},"min":{"1min":0.93300000000000005,"5min":0.79500000000000004,"15min":0.73999999999999999},"max":{"1min":1.411,"5min":1.536,"15min":1.536},"last":1.03},{"interface":"front","average":{"1min":1.145,"5min":1.139,"15min":1.1499999999999999},"min":{"1min":0.91600000000000004,"5min":0.85199999999999998,"15min":0.78400000000000003},"max":{"1min":1.431,"5min":1.524,"15min":1.619},"last":0.93500000000000005}]}]}]}} 2022-06-17T13:58:11.632 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2022-06-17T12:46:34.095313+0000 time.struct_time(tm_year=2022, tm_mon=6, tm_mday=17, tm_hour=12, tm_min=46, tm_sec=34, tm_wday=4, tm_yday=168, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=6, tm_mday=17, tm_hour=13, tm_min=58, tm_sec=6, tm_wday=4, tm_yday=168, tm_isdst=0) 2022-06-17T13:58:11.633 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2022-06-17T12:46:44.630167+0000 time.struct_time(tm_year=2022, tm_mon=6, tm_mday=17, tm_hour=12, tm_min=46, tm_sec=44, tm_wday=4, tm_yday=168, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=6, tm_mday=17, tm_hour=13, tm_min=58, tm_sec=6, tm_wday=4, tm_yday=168, tm_isdst=0) 2022-06-17T13:58:11.634 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2022-06-17T12:46:44.630167+0000 time.struct_time(tm_year=2022, tm_mon=6, tm_mday=17, tm_hour=12, tm_min=46, tm_sec=44, tm_wday=4, tm_yday=168, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=6, tm_mday=17, tm_hour=13, tm_min=58, tm_sec=6, tm_wday=4, tm_yday=168, tm_isdst=0) 2022-06-17T13:58:11.635 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2022-06-17T12:46:44.630167+0000 time.struct_time(tm_year=2022, tm_mon=6, tm_mday=17, tm_hour=12, tm_min=46, tm_sec=44, tm_wday=4, tm_yday=168, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=6, tm_mday=17, tm_hour=13, tm_min=58, tm_sec=6, tm_wday=4, tm_yday=168, tm_isdst=0) 2022-06-17T13:58:11.636 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2022-06-17T12:46:44.630167+0000 time.struct_time(tm_year=2022, tm_mon=6, tm_mday=17, tm_hour=12, tm_min=46, tm_sec=44, tm_wday=4, tm_yday=168, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=6, tm_mday=17, tm_hour=13, tm_min=58, tm_sec=6, tm_wday=4, tm_yday=168, tm_isdst=0) 2022-06-17T13:58:11.636 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2022-06-17T12:46:44.630167+0000 time.struct_time(tm_year=2022, tm_mon=6, tm_mday=17, tm_hour=12, tm_min=46, tm_sec=44, tm_wday=4, tm_yday=168, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=6, tm_mday=17, tm_hour=13, tm_min=58, tm_sec=6, tm_wday=4, tm_yday=168, tm_isdst=0) 2022-06-17T13:58:11.637 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2022-06-17T12:46:44.630167+0000 time.struct_time(tm_year=2022, tm_mon=6, tm_mday=17, tm_hour=12, tm_min=46, tm_sec=44, tm_wday=4, tm_yday=168, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=6, tm_mday=17, tm_hour=13, tm_min=58, tm_sec=6, tm_wday=4, tm_yday=168, tm_isdst=0) 2022-06-17T13:58:11.638 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2022-06-17T12:46:44.630167+0000 time.struct_time(tm_year=2022, tm_mon=6, tm_mday=17, tm_hour=12, tm_min=46, tm_sec=44, tm_wday=4, tm_yday=168, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=6, tm_mday=17, tm_hour=13, tm_min=58, tm_sec=6, tm_wday=4, tm_yday=168, tm_isdst=0) 2022-06-17T13:58:11.639 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2022-06-17T13:58:31.641 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-06-17T13:58:31.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f4afe2700 1 Processor -- start 2022-06-17T13:58:31.756 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f4afe2700 1 -- start start 2022-06-17T13:58:31.765 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f4afe2700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0f440b2200 0x7f0f440af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:31.766 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f4afe2700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f440b25e0 con 0x7f0f440af8d0 2022-06-17T13:58:31.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f4afe2700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4414b6f0 con 0x7f0f440b2cd0 2022-06-17T13:58:31.767 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f4afe2700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f440ad790 con 0x7f0f440b2200 2022-06-17T13:58:31.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f497df700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f0f440af8d0 0x7f0f4414a8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:58978/0 (socket says 172.21.15.36:58978) 2022-06-17T13:58:31.768 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f497df700 1 -- 172.21.15.36:0/1840234521 learned_addr learned my addr 172.21.15.36:0/1840234521 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T13:58:31.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f48fde700 1 -- 172.21.15.36:0/1840234521 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4137550001 0 0) 0x7f0f440b25e0 con 0x7f0f440af8d0 2022-06-17T13:58:31.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f48fde700 1 -- 172.21.15.36:0/1840234521 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f2c003260 con 0x7f0f440af8d0 2022-06-17T13:58:31.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f49fe0700 1 --2- 172.21.15.36:0/1840234521 >> v2:172.21.15.70:3300/0 conn(0x7f0f440b2200 0x7f0f440af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:31.770 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f48fde700 1 -- 172.21.15.36:0/1840234521 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2175585625 0 0) 0x7f0f2c003260 con 0x7f0f440af8d0 2022-06-17T13:58:31.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f48fde700 1 -- 172.21.15.36:0/1840234521 >> v2:172.21.15.70:3300/0 conn(0x7f0f440b2200 msgr2=0x7f0f440af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T13:58:31.771 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f48fde700 1 --2- 172.21.15.36:0/1840234521 >> v2:172.21.15.70:3300/0 conn(0x7f0f440b2200 0x7f0f440af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:31.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f48fde700 1 -- 172.21.15.36:0/1840234521 >> v1:172.21.15.70:6789/0 conn(0x7f0f440b2cd0 legacy=0x7f0f4414afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:31.772 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f48fde700 1 -- 172.21.15.36:0/1840234521 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f440aef40 con 0x7f0f440af8d0 2022-06-17T13:58:31.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f48fde700 1 -- 172.21.15.36:0/1840234521 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1391850993 0 0) 0x7f0f40001660 con 0x7f0f440af8d0 2022-06-17T13:58:31.773 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f48fde700 1 -- 172.21.15.36:0/1840234521 >> v1:172.21.15.36:6789/0 conn(0x7f0f440af8d0 legacy=0x7f0f4414a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:31.774 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f48fde700 1 --2- 172.21.15.36:0/1840234521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f2c003750 0x7f0f2c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:31.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f48fde700 1 --2- 172.21.15.36:0/1840234521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f2c004500 0x7f0f2c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:31.775 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f48fde700 1 --2- 172.21.15.36:0/1840234521 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0f2c004ec0 0x7f0f2c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:31.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f48fde700 1 -- 172.21.15.36:0/1840234521 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f440ad790 con 0x7f0f2c004ec0 2022-06-17T13:58:31.776 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f48fde700 1 -- 172.21.15.36:0/1840234521 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c00bda0 con 0x7f0f2c003750 2022-06-17T13:58:31.777 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f48fde700 1 -- 172.21.15.36:0/1840234521 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c00bf60 con 0x7f0f2c004500 2022-06-17T13:58:31.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f4a7e1700 1 --2- 172.21.15.36:0/1840234521 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0f2c004ec0 0x7f0f2c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:31.778 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.756+0000 7f0f497df700 1 --2- 172.21.15.36:0/1840234521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f2c004500 0x7f0f2c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:31.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.760+0000 7f0f4a7e1700 1 -- 172.21.15.36:0/1840234521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f2c004500 msgr2=0x7f0f2c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T13:58:31.779 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.760+0000 7f0f4a7e1700 1 --2- 172.21.15.36:0/1840234521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f2c004500 0x7f0f2c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:31.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.760+0000 7f0f4a7e1700 1 -- 172.21.15.36:0/1840234521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f2c003750 msgr2=0x7f0f2c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T13:58:31.780 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.760+0000 7f0f49fe0700 1 --2- 172.21.15.36:0/1840234521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f2c003750 0x7f0f2c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:31.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.760+0000 7f0f4a7e1700 1 --2- 172.21.15.36:0/1840234521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f2c003750 0x7f0f2c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:31.781 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.760+0000 7f0f4a7e1700 1 -- 172.21.15.36:0/1840234521 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2c003260 con 0x7f0f2c004ec0 2022-06-17T13:58:31.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.760+0000 7f0f49fe0700 1 --2- 172.21.15.36:0/1840234521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f2c003750 0x7f0f2c003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T13:58:31.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.760+0000 7f0f4a7e1700 1 -- 172.21.15.36:0/1840234521 reap_dead start 2022-06-17T13:58:31.782 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.760+0000 7f0f4a7e1700 1 --2- 172.21.15.36:0/1840234521 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0f2c004ec0 0x7f0f2c00b750 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 rx=0x7f0f4406ad60 tx=0x7f0f3c007d80).ready entity=mon.0 client_cookie=f61c899fbeb076d6 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:31.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.760+0000 7f0f48fde700 1 -- 172.21.15.36:0/1840234521 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f3c001620 con 0x7f0f2c004ec0 2022-06-17T13:58:31.783 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.760+0000 7f0f48fde700 1 -- 172.21.15.36:0/1840234521 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3c006890 con 0x7f0f2c004ec0 2022-06-17T13:58:31.784 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.760+0000 7f0f48fde700 1 -- 172.21.15.36:0/1840234521 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f3c006bf0 con 0x7f0f2c004ec0 2022-06-17T13:58:31.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.760+0000 7f0f4afe2700 1 -- 172.21.15.36:0/1840234521 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0f2c004ec0 msgr2=0x7f0f2c00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:31.785 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.760+0000 7f0f4afe2700 1 --2- 172.21.15.36:0/1840234521 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0f2c004ec0 0x7f0f2c00b750 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 rx=0x7f0f4406ad60 tx=0x7f0f3c007d80).stop 2022-06-17T13:58:31.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.760+0000 7f0f4afe2700 1 -- 172.21.15.36:0/1840234521 shutdown_connections 2022-06-17T13:58:31.786 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.760+0000 7f0f4afe2700 1 --2- 172.21.15.36:0/1840234521 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0f2c004ec0 0x7f0f2c00b750 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:31.787 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.760+0000 7f0f4afe2700 1 -- 172.21.15.36:0/1840234521 >> 172.21.15.36:0/1840234521 conn(0x7f0f4405f770 msgr2=0x7f0f4405fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:31.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.760+0000 7f0f4afe2700 1 -- 172.21.15.36:0/1840234521 shutdown_connections 2022-06-17T13:58:31.788 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.760+0000 7f0f4afe2700 1 -- 172.21.15.36:0/1840234521 wait complete. 2022-06-17T13:58:31.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.760+0000 7f0f4afe2700 1 Processor -- start 2022-06-17T13:58:31.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.760+0000 7f0f4afe2700 1 -- start start 2022-06-17T13:58:31.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.764+0000 7f0f4afe2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0f2c004ec0 0x7f0f44064a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:31.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.764+0000 7f0f4afe2700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f440a91f0 0x7f0f440b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:31.791 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.764+0000 7f0f4afe2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f440aa840 0x7f0f44155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:31.792 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.764+0000 7f0f4afe2700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f44062720 con 0x7f0f2c004ec0 2022-06-17T13:58:31.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.764+0000 7f0f4afe2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f440629c0 con 0x7f0f440aa840 2022-06-17T13:58:31.793 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.764+0000 7f0f4afe2700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f44156100 con 0x7f0f440a91f0 2022-06-17T13:58:31.794 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.764+0000 7f0f49fe0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0f2c004ec0 0x7f0f44064a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:31.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.764+0000 7f0f49fe0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0f2c004ec0 0x7f0f44064a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41460/0 (socket says 172.21.15.36:41460) 2022-06-17T13:58:31.795 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.764+0000 7f0f49fe0700 1 -- 172.21.15.36:0/978783100 learned_addr learned my addr 172.21.15.36:0/978783100 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:31.796 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.764+0000 7f0f497df700 1 --2- 172.21.15.36:0/978783100 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f440a91f0 0x7f0f440b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:31.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.764+0000 7f0f4a7e1700 1 --2- 172.21.15.36:0/978783100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f440aa840 0x7f0f44155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:31.797 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.764+0000 7f0f49fe0700 1 -- 172.21.15.36:0/978783100 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f440a91f0 msgr2=0x7f0f440b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:31.798 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.764+0000 7f0f49fe0700 1 --2- 172.21.15.36:0/978783100 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f440a91f0 0x7f0f440b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:31.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.764+0000 7f0f49fe0700 1 -- 172.21.15.36:0/978783100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f440aa840 msgr2=0x7f0f44155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:31.799 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.764+0000 7f0f49fe0700 1 --2- 172.21.15.36:0/978783100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f440aa840 0x7f0f44155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:31.800 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.764+0000 7f0f49fe0700 1 -- 172.21.15.36:0/978783100 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f441562c0 con 0x7f0f2c004ec0 2022-06-17T13:58:31.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.764+0000 7f0f49fe0700 1 --2- 172.21.15.36:0/978783100 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0f2c004ec0 0x7f0f44064a10 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 rx=0x7f0f34008000 tx=0x7f0f34008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:31.801 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.764+0000 7f0f32ffd700 1 -- 172.21.15.36:0/978783100 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f34014400 con 0x7f0f2c004ec0 2022-06-17T13:58:31.802 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.764+0000 7f0f32ffd700 1 -- 172.21.15.36:0/978783100 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f34000e90 con 0x7f0f2c004ec0 2022-06-17T13:58:31.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.764+0000 7f0f4afe2700 1 -- 172.21.15.36:0/978783100 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f441564b0 con 0x7f0f2c004ec0 2022-06-17T13:58:31.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.764+0000 7f0f32ffd700 1 -- 172.21.15.36:0/978783100 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f34014750 con 0x7f0f2c004ec0 2022-06-17T13:58:31.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.764+0000 7f0f4afe2700 1 -- 172.21.15.36:0/978783100 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f4415a3a0 con 0x7f0f2c004ec0 2022-06-17T13:58:31.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.768+0000 7f0f4afe2700 1 -- 172.21.15.36:0/978783100 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f440b1950 con 0x7f0f2c004ec0 2022-06-17T13:58:31.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.768+0000 7f0f32ffd700 1 -- 172.21.15.36:0/978783100 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7f0f3401a020 con 0x7f0f2c004ec0 2022-06-17T13:58:31.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.768+0000 7f0f32ffd700 1 --2- 172.21.15.36:0/978783100 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0f24053a60 0x7f0f24055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:31.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.768+0000 7f0f32ffd700 1 -- 172.21.15.36:0/978783100 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f0f3406af40 con 0x7f0f2c004ec0 2022-06-17T13:58:31.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.768+0000 7f0f497df700 1 --2- 172.21.15.36:0/978783100 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0f24053a60 0x7f0f24055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:31.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.768+0000 7f0f497df700 1 --2- 172.21.15.36:0/978783100 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0f24053a60 0x7f0f24055f20 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7f0f4000d000 tx=0x7f0f40007040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:31.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.776+0000 7f0f32ffd700 1 -- 172.21.15.36:0/978783100 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0f34041fa0 con 0x7f0f2c004ec0 2022-06-17T13:58:31.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.900+0000 7f0f4afe2700 1 -- 172.21.15.36:0/978783100 --> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0f440625a0 con 0x7f0f24053a60 2022-06-17T13:58:31.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.904+0000 7f0f32ffd700 1 -- 172.21.15.36:0/978783100 <== mgr.4104 v2:172.21.15.36:6800/22419 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+67999 (secure 0 0 0) 0x7f0f40010040 con 0x7f0f24053a60 2022-06-17T13:58:31.905 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-06-17T13:58:31.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.908+0000 7f0f4afe2700 1 -- 172.21.15.36:0/978783100 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0f24053a60 msgr2=0x7f0f24055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:31.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.908+0000 7f0f4afe2700 1 --2- 172.21.15.36:0/978783100 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0f24053a60 0x7f0f24055f20 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7f0f4000d000 tx=0x7f0f40007040).stop 2022-06-17T13:58:31.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.908+0000 7f0f4afe2700 1 -- 172.21.15.36:0/978783100 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0f2c004ec0 msgr2=0x7f0f44064a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:31.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.908+0000 7f0f4afe2700 1 --2- 172.21.15.36:0/978783100 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0f2c004ec0 0x7f0f44064a10 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 rx=0x7f0f34008000 tx=0x7f0f34008040).stop 2022-06-17T13:58:31.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.908+0000 7f0f4afe2700 1 -- 172.21.15.36:0/978783100 shutdown_connections 2022-06-17T13:58:31.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.908+0000 7f0f4afe2700 1 --2- 172.21.15.36:0/978783100 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7f0f24053a60 0x7f0f24055f20 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:31.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.908+0000 7f0f4afe2700 1 --2- 172.21.15.36:0/978783100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f440aa840 0x7f0f44155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:31.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.908+0000 7f0f4afe2700 1 --2- 172.21.15.36:0/978783100 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f440a91f0 0x7f0f440b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:31.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.908+0000 7f0f4afe2700 1 --2- 172.21.15.36:0/978783100 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0f2c004ec0 0x7f0f44064a10 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:31.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.908+0000 7f0f4afe2700 1 -- 172.21.15.36:0/978783100 >> 172.21.15.36:0/978783100 conn(0x7f0f4405f770 msgr2=0x7f0f44147660 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:31.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.908+0000 7f0f4afe2700 1 -- 172.21.15.36:0/978783100 shutdown_connections 2022-06-17T13:58:31.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:31.908+0000 7f0f4afe2700 1 -- 172.21.15.36:0/978783100 wait complete. 2022-06-17T13:58:31.911 INFO:teuthology.orchestra.run.smithi036.stderr:dumped all 2022-06-17T13:58:31.922 INFO:teuthology.orchestra.run.smithi036.stdout:{"pg_ready":true,"pg_map":{"version":2310,"stamp":"2022-06-17T13:58:30.696179+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":8404000,"kb_used_data":15328,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746570720,"statfs":{"total":773094113280,"available":764488417280,"internally_reserved":8589934592,"allocated":15695872,"data_stored":7700384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2742,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-46,"num_write_kb":-14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.554850"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":141,"reported_epoch":154,"state":"active+clean","last_fresh":"2022-06-17T13:58:12.008015+0000","last_change":"2022-06-17T13:58:12.008196+0000","last_active":"2022-06-17T13:58:12.008015+0000","last_peered":"2022-06-17T13:58:12.008015+0000","last_clean":"2022-06-17T13:58:12.008015+0000","last_became_active":"2022-06-17T12:46:41.622138+0000","last_became_peered":"2022-06-17T12:46:41.622138+0000","last_unstale":"2022-06-17T13:58:12.008015+0000","last_undegraded":"2022-06-17T13:58:12.008015+0000","last_fullsized":"2022-06-17T13:58:12.008015+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":4,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T13:58:12.008014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T13:58:12.008014+0000","last_clean_scrub_stamp":"2022-06-17T13:58:12.008014+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":141,"reported_epoch":154,"state":"active+clean","last_fresh":"2022-06-17T13:58:21.136829+0000","last_change":"2022-06-17T13:58:21.137019+0000","last_active":"2022-06-17T13:58:21.136829+0000","last_peered":"2022-06-17T13:58:21.136829+0000","last_clean":"2022-06-17T13:58:21.136829+0000","last_became_active":"2022-06-17T12:46:45.657682+0000","last_became_peered":"2022-06-17T12:46:45.657682+0000","last_unstale":"2022-06-17T13:58:21.136829+0000","last_undegraded":"2022-06-17T13:58:21.136829+0000","last_fullsized":"2022-06-17T13:58:21.136829+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T13:58:21.136829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T13:58:21.136829+0000","last_clean_scrub_stamp":"2022-06-17T13:58:21.136829+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":138,"reported_epoch":154,"state":"active+clean","last_fresh":"2022-06-17T13:58:12.964527+0000","last_change":"2022-06-17T13:58:12.964708+0000","last_active":"2022-06-17T13:58:12.964527+0000","last_peered":"2022-06-17T13:58:12.964527+0000","last_clean":"2022-06-17T13:58:12.964527+0000","last_became_active":"2022-06-17T12:46:45.662293+0000","last_became_peered":"2022-06-17T12:46:45.662293+0000","last_unstale":"2022-06-17T13:58:12.964527+0000","last_undegraded":"2022-06-17T13:58:12.964527+0000","last_fullsized":"2022-06-17T13:58:12.964527+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T13:58:12.964526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T13:58:12.964526+0000","last_clean_scrub_stamp":"2022-06-17T13:58:12.964526+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":139,"reported_epoch":154,"state":"active+clean","last_fresh":"2022-06-17T13:58:10.256113+0000","last_change":"2022-06-17T13:58:10.256267+0000","last_active":"2022-06-17T13:58:10.256113+0000","last_peered":"2022-06-17T13:58:10.256113+0000","last_clean":"2022-06-17T13:58:10.256113+0000","last_became_active":"2022-06-17T12:46:46.356789+0000","last_became_peered":"2022-06-17T12:46:46.356789+0000","last_unstale":"2022-06-17T13:58:10.256113+0000","last_undegraded":"2022-06-17T13:58:10.256113+0000","last_fullsized":"2022-06-17T13:58:10.256113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T13:58:10.256113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T13:58:10.256113+0000","last_clean_scrub_stamp":"2022-06-17T13:58:10.256113+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":141,"reported_epoch":154,"state":"active+clean","last_fresh":"2022-06-17T13:58:12.231158+0000","last_change":"2022-06-17T13:58:12.231339+0000","last_active":"2022-06-17T13:58:12.231158+0000","last_peered":"2022-06-17T13:58:12.231158+0000","last_clean":"2022-06-17T13:58:12.231158+0000","last_became_active":"2022-06-17T12:46:45.662460+0000","last_became_peered":"2022-06-17T12:46:45.662460+0000","last_unstale":"2022-06-17T13:58:12.231158+0000","last_undegraded":"2022-06-17T13:58:12.231158+0000","last_fullsized":"2022-06-17T13:58:12.231158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T13:58:12.231157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T13:58:12.231157+0000","last_clean_scrub_stamp":"2022-06-17T13:58:12.231157+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":137,"reported_epoch":154,"state":"active+clean","last_fresh":"2022-06-17T13:58:09.566370+0000","last_change":"2022-06-17T13:58:09.566558+0000","last_active":"2022-06-17T13:58:09.566370+0000","last_peered":"2022-06-17T13:58:09.566370+0000","last_clean":"2022-06-17T13:58:09.566370+0000","last_became_active":"2022-06-17T12:46:46.260143+0000","last_became_peered":"2022-06-17T12:46:46.260143+0000","last_unstale":"2022-06-17T13:58:09.566370+0000","last_undegraded":"2022-06-17T13:58:09.566370+0000","last_fullsized":"2022-06-17T13:58:09.566370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T13:58:09.566369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T13:58:09.566369+0000","last_clean_scrub_stamp":"2022-06-17T13:58:09.566369+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":138,"reported_epoch":154,"state":"active+clean","last_fresh":"2022-06-17T13:58:24.900385+0000","last_change":"2022-06-17T13:58:24.900491+0000","last_active":"2022-06-17T13:58:24.900385+0000","last_peered":"2022-06-17T13:58:24.900385+0000","last_clean":"2022-06-17T13:58:24.900385+0000","last_became_active":"2022-06-17T12:46:46.264499+0000","last_became_peered":"2022-06-17T12:46:46.264499+0000","last_unstale":"2022-06-17T13:58:24.900385+0000","last_undegraded":"2022-06-17T13:58:24.900385+0000","last_fullsized":"2022-06-17T13:58:24.900385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T13:58:24.900385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T13:58:24.900385+0000","last_clean_scrub_stamp":"2022-06-17T13:58:24.900385+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":137,"reported_epoch":154,"state":"active+clean","last_fresh":"2022-06-17T13:58:17.428053+0000","last_change":"2022-06-17T13:58:17.428134+0000","last_active":"2022-06-17T13:58:17.428053+0000","last_peered":"2022-06-17T13:58:17.428053+0000","last_clean":"2022-06-17T13:58:17.428053+0000","last_became_active":"2022-06-17T12:46:45.658569+0000","last_became_peered":"2022-06-17T12:46:45.658569+0000","last_unstale":"2022-06-17T13:58:17.428053+0000","last_undegraded":"2022-06-17T13:58:17.428053+0000","last_fullsized":"2022-06-17T13:58:17.428053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T13:58:17.428052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T13:58:17.428052+0000","last_clean_scrub_stamp":"2022-06-17T13:58:17.428052+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":139,"reported_epoch":154,"state":"active+clean","last_fresh":"2022-06-17T13:58:13.857844+0000","last_change":"2022-06-17T13:58:13.858120+0000","last_active":"2022-06-17T13:58:13.857844+0000","last_peered":"2022-06-17T13:58:13.857844+0000","last_clean":"2022-06-17T13:58:13.857844+0000","last_became_active":"2022-06-17T12:46:45.660343+0000","last_became_peered":"2022-06-17T12:46:45.660343+0000","last_unstale":"2022-06-17T13:58:13.857844+0000","last_undegraded":"2022-06-17T13:58:13.857844+0000","last_fullsized":"2022-06-17T13:58:13.857844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-17T13:58:13.857843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-17T13:58:13.857843+0000","last_clean_scrub_stamp":"2022-06-17T13:58:13.857843+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543007,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050500,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321340,"statfs":{"total":96636764160,"available":95561052160,"internally_reserved":1073741824,"allocated":1961984,"data_stored":962548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 17 13:58:08 2022","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.86599999999999999,"15min":0.88700000000000001},"min":{"1min":0.51600000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1220000000000001,"5min":1.1639999999999999,"15min":1.2270000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.88,"5min":0.90500000000000003,"15min":0.91300000000000003},"min":{"1min":0.69799999999999995,"5min":0.66300000000000003,"15min":0.57499999999999996},"max":{"1min":1.175,"5min":1.175,"15min":1.4359999999999999},"last":0.94099999999999995}]},{"osd":1,"last update":"Fri Jun 17 13:58:08 2022","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.83099999999999996,"15min":0.81799999999999995},"min":{"1min":0.57199999999999995,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0800000000000001,"5min":1.2110000000000001,"15min":1.2350000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84599999999999997,"15min":0.81899999999999995},"min":{"1min":0.67300000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.111,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.80400000000000005}]},{"osd":2,"last update":"Fri Jun 17 13:58:08 2022","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.84099999999999997,"15min":0.85299999999999998},"min":{"1min":0.65500000000000003,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0289999999999999,"5min":1.0780000000000001,"15min":1.1759999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.875,"5min":0.84699999999999998,"15min":0.85599999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.095,"5min":1.2010000000000001,"15min":1.23},"last":0.92400000000000004}]},{"osd":3,"last update":"Fri Jun 17 13:58:08 2022","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.83699999999999997,"15min":0.85999999999999999},"min":{"1min":0.63100000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0169999999999999,"5min":1.1120000000000001,"15min":1.496},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.90200000000000002,"15min":0.91000000000000003},"min":{"1min":0.69799999999999995,"5min":0.67600000000000005,"15min":0.61699999999999999},"max":{"1min":1.0980000000000001,"5min":1.264,"15min":1.28},"last":0.86099999999999999}]},{"osd":4,"last update":"Fri Jun 17 13:58:08 2022","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.78000000000000003,"15min":0.79100000000000004},"min":{"1min":0.61199999999999999,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0600000000000001,"5min":1.2170000000000001,"15min":1.597},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.85399999999999998,"15min":0.88300000000000001},"min":{"1min":0.69099999999999995,"5min":0.56899999999999995,"15min":0.54600000000000004},"max":{"1min":1.0940000000000001,"5min":1.2,"15min":1.623},"last":0.61199999999999999}]},{"osd":5,"last update":"Fri Jun 17 13:58:08 2022","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.89300000000000002,"15min":0.89800000000000002},"min":{"1min":0.70299999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.153,"5min":1.2929999999999999,"15min":1.611},"last":0.755},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.93100000000000005,"15min":0.94299999999999995},"min":{"1min":0.66800000000000004,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.1719999999999999,"5min":1.206,"15min":1.6459999999999999},"last":0.71999999999999997}]},{"osd":6,"last update":"Fri Jun 17 13:58:08 2022","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.96999999999999997,"15min":0.98799999999999999},"min":{"1min":0.748,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.2170000000000001,"5min":1.278,"15min":1.696},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.96099999999999997,"15min":1.0009999999999999},"min":{"1min":0.879,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.256,"5min":1.2949999999999999,"15min":1.6819999999999999},"last":0.85099999999999998}]}]},{"osd":6,"up_from":14,"seq":60129543008,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050500,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321340,"statfs":{"total":96636764160,"available":95561052160,"internally_reserved":1073741824,"allocated":1961984,"data_stored":962548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 17 13:58:02 2022","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.81399999999999995,"15min":0.81000000000000005},"min":{"1min":0.623,"5min":0.52800000000000002,"15min":0.48099999999999998},"max":{"1min":1.105,"5min":1.1220000000000001,"15min":1.5549999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85199999999999998,"15min":0.83799999999999997},"min":{"1min":0.63500000000000001,"5min":0.56000000000000005,"15min":0.52800000000000002},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.302},"last":0.96099999999999997}]},{"osd":1,"last update":"Fri Jun 17 13:58:02 2022","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.81599999999999995,"15min":0.83299999999999996},"min":{"1min":0.61599999999999999,"5min":0.60499999999999998,"15min":0.56200000000000006},"max":{"1min":1.0780000000000001,"5min":1.095,"15min":1.1699999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.82899999999999996,"15min":0.84599999999999997},"min":{"1min":0.63600000000000001,"5min":0.59999999999999998,"15min":0.57299999999999995},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.4370000000000001},"last":0.99099999999999999}]},{"osd":2,"last update":"Fri Jun 17 13:58:02 2022","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.86099999999999999,"15min":0.83999999999999997},"min":{"1min":0.65300000000000002,"5min":0.64700000000000002,"15min":0.59399999999999997},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.3879999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.88,"5min":0.86799999999999999,"15min":0.86499999999999999},"min":{"1min":0.67700000000000005,"5min":0.64900000000000002,"15min":0.62},"max":{"1min":1.1120000000000001,"5min":1.224,"15min":1.2430000000000001},"last":0.88500000000000001}]},{"osd":3,"last update":"Fri Jun 17 13:58:02 2022","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.86899999999999999,"15min":0.878},"min":{"1min":0.71399999999999997,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.21,"5min":1.2170000000000001,"15min":1.296},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.92000000000000004,"15min":0.91300000000000003},"min":{"1min":0.754,"5min":0.65400000000000003,"15min":0.63800000000000001},"max":{"1min":1.1779999999999999,"5min":1.1799999999999999,"15min":1.339},"last":0.71899999999999997}]},{"osd":4,"last update":"Fri Jun 17 13:58:02 2022","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.80600000000000005,"15min":0.81299999999999994},"min":{"1min":0.57299999999999995,"5min":0.56399999999999995,"15min":0.53800000000000003},"max":{"1min":1.0289999999999999,"5min":1.2010000000000001,"15min":1.3580000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.04,"5min":0.92300000000000004,"15min":0.91100000000000003},"min":{"1min":0.80000000000000004,"5min":0.60399999999999998,"15min":0.59499999999999997},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.85599999999999998}]},{"osd":5,"last update":"Fri Jun 17 13:58:02 2022","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.879,"15min":0.88700000000000001},"min":{"1min":0.625,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.5009999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.94899999999999995,"15min":0.94799999999999995},"min":{"1min":0.63800000000000001,"5min":0.63400000000000001,"15min":0.59999999999999998},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.81100000000000005}]},{"osd":7,"last update":"Fri Jun 17 13:58:02 2022","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.97799999999999998,"15min":0.97599999999999998},"min":{"1min":0.71799999999999997,"5min":0.64700000000000002,"15min":0.61299999999999999},"max":{"1min":1.339,"5min":1.377,"15min":1.6910000000000001},"last":1.04},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.034,"15min":1.0329999999999999},"min":{"1min":0.83099999999999996,"5min":0.77400000000000002,"15min":0.70599999999999996},"max":{"1min":1.454,"5min":1.454,"15min":1.714},"last":0.94099999999999995}]}]},{"osd":3,"up_from":14,"seq":60129543009,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050500,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321340,"statfs":{"total":96636764160,"available":95561052160,"internally_reserved":1073741824,"allocated":1961984,"data_stored":962548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 17 13:57:57 2022","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.52600000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93999999999999995,"5min":1.0369999999999999,"15min":1.0900000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.72599999999999998,"15min":0.71399999999999997},"min":{"1min":0.52800000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93899999999999995,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.60299999999999998}]},{"osd":1,"last update":"Fri Jun 17 13:57:57 2022","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65900000000000003,"15min":0.65300000000000002},"min":{"1min":0.55000000000000004,"5min":0.48299999999999998,"15min":0.44900000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.2669999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76700000000000002,"15min":0.755},"min":{"1min":0.59799999999999998,"5min":0.54400000000000004,"15min":0.48599999999999999},"max":{"1min":1.0089999999999999,"5min":1.0629999999999999,"15min":1.266},"last":0.69699999999999995}]},{"osd":2,"last update":"Fri Jun 17 13:57:57 2022","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.80500000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0149999999999999,"5min":1.2250000000000001,"15min":1.7589999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.90700000000000003,"15min":0.90400000000000003},"min":{"1min":0.73799999999999999,"5min":0.73499999999999999,"15min":0.66600000000000004},"max":{"1min":1.157,"5min":1.157,"15min":1.704},"last":0.81599999999999995}]},{"osd":4,"last update":"Fri Jun 17 13:57:57 2022","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92700000000000005,"15min":0.93100000000000005},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.67200000000000004},"max":{"1min":1.234,"5min":1.3160000000000001,"15min":1.401},"last":1.1910000000000001},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.88600000000000001,"15min":0.90100000000000002},"min":{"1min":0.70599999999999996,"5min":0.68400000000000005,"15min":0.66500000000000004},"max":{"1min":1.1910000000000001,"5min":1.302,"15min":1.3979999999999999},"last":1.0800000000000001}]},{"osd":5,"last update":"Fri Jun 17 13:57:57 2022","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.97299999999999998,"15min":0.97499999999999998},"min":{"1min":0.80100000000000005,"5min":0.69799999999999995,"15min":0.68700000000000006},"max":{"1min":1.353,"5min":1.353,"15min":1.4790000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.01,"15min":1.0129999999999999},"min":{"1min":0.78500000000000003,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.218,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":1.2549999999999999}]},{"osd":6,"last update":"Fri Jun 17 13:57:57 2022","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.98899999999999999,"15min":0.996},"min":{"1min":0.77700000000000002,"5min":0.69299999999999995,"15min":0.68300000000000005},"max":{"1min":1.1899999999999999,"5min":1.3979999999999999,"15min":1.8500000000000001},"last":1.276},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0940000000000001,"15min":1.079},"min":{"1min":0.89500000000000002,"5min":0.88700000000000001,"15min":0.72899999999999998},"max":{"1min":1.482,"5min":1.554,"15min":1.74},"last":1.179}]},{"osd":7,"last update":"Fri Jun 17 13:57:57 2022","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.0920000000000001,"15min":1.0980000000000001},"min":{"1min":0.878,"5min":0.72099999999999997,"15min":0.71299999999999997},"max":{"1min":1.3100000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.087,"5min":1.0600000000000001,"15min":1.079},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.27,"5min":1.355,"15min":1.7450000000000001},"last":1.159}]}]},{"osd":2,"up_from":14,"seq":60129543008,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050500,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321340,"statfs":{"total":96636764160,"available":95561052160,"internally_reserved":1073741824,"allocated":1961984,"data_stored":962548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 17 13:57:44 2022","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58699999999999997,"15min":0.60299999999999998},"min":{"1min":0.45900000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.143},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70499999999999996,"15min":0.71199999999999997},"min":{"1min":0.499,"5min":0.495,"15min":0.46200000000000002},"max":{"1min":1.028,"5min":1.0800000000000001,"15min":1.204},"last":0.57199999999999995}]},{"osd":1,"last update":"Fri Jun 17 13:57:44 2022","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.68200000000000005,"15min":0.69099999999999995},"min":{"1min":0.57099999999999995,"5min":0.499,"15min":0.48099999999999998},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.526},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72999999999999998,"15min":0.73399999999999999},"min":{"1min":0.57299999999999995,"5min":0.496,"15min":0.47699999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.228},"last":0.76300000000000001}]},{"osd":3,"last update":"Fri Jun 17 13:57:44 2022","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70399999999999996,"15min":0.71599999999999997},"min":{"1min":0.52500000000000002,"5min":0.503,"15min":0.48699999999999999},"max":{"1min":1.0489999999999999,"5min":1.056,"15min":1.2929999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.85899999999999999,"15min":0.85999999999999999},"min":{"1min":0.70499999999999996,"5min":0.64200000000000002,"15min":0.60799999999999998},"max":{"1min":1.167,"5min":1.3109999999999999,"15min":1.3560000000000001},"last":0.89300000000000002}]},{"osd":4,"last update":"Fri Jun 17 13:57:44 2022","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91700000000000004,"15min":0.92600000000000005},"min":{"1min":0.71699999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.2070000000000001,"5min":1.2689999999999999,"15min":1.4419999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.91700000000000004,"15min":0.94199999999999995},"min":{"1min":0.71799999999999997,"5min":0.71499999999999997,"15min":0.65900000000000003},"max":{"1min":1.0600000000000001,"5min":1.3660000000000001,"15min":1.4299999999999999},"last":1.278}]},{"osd":5,"last update":"Fri Jun 17 13:57:44 2022","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.93899999999999995,"15min":0.93300000000000005},"min":{"1min":0.76500000000000001,"5min":0.64200000000000002,"15min":0.629},"max":{"1min":1.329,"5min":1.3560000000000001,"15min":1.508},"last":1.236},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97399999999999998,"15min":0.99199999999999999},"min":{"1min":0.79400000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.95799999999999996}]},{"osd":6,"last update":"Fri Jun 17 13:57:44 2022","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.98699999999999999,"15min":1.002},"min":{"1min":0.76900000000000002,"5min":0.71699999999999997,"15min":0.69299999999999995},"max":{"1min":1.1259999999999999,"5min":1.4870000000000001,"15min":1.514},"last":1.456},{"interface":"front","average":{"1min":1.012,"5min":1.0069999999999999,"15min":1},"min":{"1min":0.79500000000000004,"5min":0.71599999999999997,"15min":0.68999999999999995},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":1.341}]},{"osd":7,"last update":"Fri Jun 17 13:57:44 2022","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.038,"15min":1.0609999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.365,"5min":1.464,"15min":1.589},"last":1.365},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.054,"15min":1.0529999999999999},"min":{"1min":0.88,"5min":0.78000000000000003,"15min":0.77600000000000002},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.425}]}]},{"osd":5,"up_from":13,"seq":55834575713,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050500,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321340,"statfs":{"total":96636764160,"available":95561052160,"internally_reserved":1073741824,"allocated":1961984,"data_stored":962548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 17 13:58:14 2022","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.82499999999999996,"15min":0.83799999999999997},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.48799999999999999},"max":{"1min":1.026,"5min":1.1579999999999999,"15min":1.167},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.82999999999999996,"15min":0.82699999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.008,"5min":1.0920000000000001,"15min":1.1879999999999999},"last":1.0409999999999999}]},{"osd":1,"last update":"Fri Jun 17 13:58:14 2022","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83399999999999996,"15min":0.84399999999999997},"min":{"1min":0.64400000000000002,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.028,"5min":1.1000000000000001,"15min":1.232},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.80400000000000005,"15min":0.81200000000000006},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.58699999999999997},"max":{"1min":1.008,"5min":1.0920000000000001,"15min":1.254},"last":0.66400000000000003}]},{"osd":2,"last update":"Fri Jun 17 13:58:14 2022","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.83999999999999997,"15min":0.86499999999999999},"min":{"1min":0.629,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.0820000000000001,"5min":1.093,"15min":1.236},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.83999999999999997,"15min":0.83899999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.087,"5min":1.087,"15min":1.3100000000000001},"last":0.77200000000000002}]},{"osd":3,"last update":"Fri Jun 17 13:58:14 2022","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.86299999999999999,"15min":0.88300000000000001},"min":{"1min":0.72399999999999998,"5min":0.61799999999999999,"15min":0.57799999999999996},"max":{"1min":1.0840000000000001,"5min":1.151,"15min":1.3149999999999999},"last":1.111},{"interface":"front","average":{"1min":1.018,"5min":1.0149999999999999,"15min":1.0089999999999999},"min":{"1min":0.86899999999999999,"5min":0.84299999999999997,"15min":0.61499999999999999},"max":{"1min":1.1080000000000001,"5min":1.161,"15min":1.244},"last":1.0649999999999999}]},{"osd":4,"last update":"Fri Jun 17 13:58:14 2022","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76000000000000001,"15min":0.78900000000000003},"min":{"1min":0.60099999999999998,"5min":0.53300000000000003,"15min":0.52300000000000002},"max":{"1min":1.0229999999999999,"5min":1.236,"15min":1.2549999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.89000000000000001,"15min":0.89600000000000002},"min":{"1min":0.63200000000000001,"5min":0.61799999999999999,"15min":0.56499999999999995},"max":{"1min":1.1519999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":1.0129999999999999}]},{"osd":6,"last update":"Fri Jun 17 13:58:14 2022","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.86699999999999999,"15min":0.88300000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.034,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.2350000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.96399999999999997,"15min":0.97199999999999998},"min":{"1min":0.751,"5min":0.624,"15min":0.60499999999999998},"max":{"1min":1.1659999999999999,"5min":1.208,"15min":1.236},"last":0.90500000000000003}]},{"osd":7,"last update":"Fri Jun 17 13:58:14 2022","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.85699999999999998,"15min":0.88},"min":{"1min":0.57999999999999996,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0629999999999999,"5min":1.1379999999999999,"15min":1.361},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.96999999999999997,"15min":0.98499999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.145,"5min":1.361,"15min":1.397},"last":1.1719999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575712,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1050500,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321340,"statfs":{"total":96636764160,"available":95561052160,"internally_reserved":1073741824,"allocated":1961984,"data_stored":962548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 17 13:57:32 2022","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.91300000000000003,"15min":0.89200000000000002},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.496},"max":{"1min":1.0840000000000001,"5min":1.288,"15min":1.288},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.72899999999999998,"5min":0.66900000000000004,"15min":0.622},"max":{"1min":1.099,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":1.1180000000000001}]},{"osd":1,"last update":"Fri Jun 17 13:57:32 2022","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80800000000000005,"15min":0.79500000000000004},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.52500000000000002},"max":{"1min":1.046,"5min":1.1499999999999999,"15min":1.165},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.2,"5min":1.2,"15min":1.3100000000000001},"last":0.64600000000000002}]},{"osd":2,"last update":"Fri Jun 17 13:57:32 2022","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.85699999999999998,"15min":0.84999999999999998},"min":{"1min":0.71299999999999997,"5min":0.60399999999999998,"15min":0.58399999999999996},"max":{"1min":1.099,"5min":1.238,"15min":1.238},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.83699999999999997,"15min":0.84599999999999997},"min":{"1min":0.627,"5min":0.627,"15min":0.621},"max":{"1min":0.99099999999999999,"5min":1.1459999999999999,"15min":1.218},"last":1.022}]},{"osd":3,"last update":"Fri Jun 17 13:57:32 2022","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85699999999999998,"15min":0.874},"min":{"1min":0.66000000000000003,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.26},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.92300000000000004,"15min":0.90400000000000003},"min":{"1min":0.68999999999999995,"5min":0.63600000000000001,"15min":0.629},"max":{"1min":1.131,"5min":1.2350000000000001,"15min":1.2430000000000001},"last":1.0840000000000001}]},{"osd":5,"last update":"Fri Jun 17 13:57:32 2022","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.86199999999999999,"15min":0.86299999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.54900000000000004},"max":{"1min":1.038,"5min":1.1890000000000001,"15min":1.2729999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.871,"5min":0.89900000000000002,"15min":0.89600000000000002},"min":{"1min":0.56399999999999995,"5min":0.55600000000000005,"15min":0.51800000000000002},"max":{"1min":1.0389999999999999,"5min":1.1879999999999999,"15min":1.234},"last":1.0569999999999999}]},{"osd":6,"last update":"Fri Jun 17 13:57:32 2022","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.85799999999999998,"15min":0.85499999999999998},"min":{"1min":0.64600000000000002,"5min":0.58899999999999997,"15min":0.55800000000000005},"max":{"1min":1.1339999999999999,"5min":1.137,"15min":1.502},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.88400000000000001,"15min":0.90300000000000002},"min":{"1min":0.57699999999999996,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.256,"5min":1.276,"15min":1.5629999999999999},"last":1.0009999999999999}]},{"osd":7,"last update":"Fri Jun 17 13:57:32 2022","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.91100000000000003,"15min":0.92000000000000004},"min":{"1min":0.63100000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.1839999999999999,"5min":1.2529999999999999,"15min":1.2849999999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.93200000000000005,"15min":0.93700000000000006},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.59399999999999997},"max":{"1min":1.103,"5min":1.1930000000000001,"15min":1.268},"last":1.143}]}]},{"osd":1,"up_from":13,"seq":55834575713,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050500,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321340,"statfs":{"total":96636764160,"available":95561052160,"internally_reserved":1073741824,"allocated":1961984,"data_stored":962548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 17 13:57:38 2022","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.51200000000000001,"5min":0.47899999999999998,"15min":0.46999999999999997},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.75,"15min":0.755},"min":{"1min":0.65700000000000003,"5min":0.59299999999999997,"15min":0.56100000000000005},"max":{"1min":0.94899999999999995,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.751}]},{"osd":2,"last update":"Fri Jun 17 13:58:23 2022","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65900000000000003,"15min":0.66700000000000004},"min":{"1min":0.504,"5min":0.47599999999999998,"15min":0.45900000000000002},"max":{"1min":0.98999999999999999,"5min":1.105,"15min":1.784},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76500000000000001,"15min":0.77400000000000002},"min":{"1min":0.58599999999999997,"5min":0.5,"15min":0.47599999999999998},"max":{"1min":0.91100000000000003,"5min":1.2170000000000001,"15min":1.4830000000000001},"last":0.79300000000000004}]},{"osd":3,"last update":"Fri Jun 17 13:58:23 2022","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65700000000000003,"15min":0.68300000000000005},"min":{"1min":0.55300000000000005,"5min":0.504,"15min":0.504},"max":{"1min":0.82099999999999995,"5min":1.0149999999999999,"15min":1.2050000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.79700000000000004,"15min":0.79800000000000004},"min":{"1min":0.64700000000000002,"5min":0.58199999999999996,"15min":0.54600000000000004},"max":{"1min":1.1859999999999999,"5min":1.1910000000000001,"15min":1.27},"last":0.71399999999999997}]},{"osd":4,"last update":"Fri Jun 17 13:57:38 2022","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94599999999999995,"15min":0.94199999999999995},"min":{"1min":0.77500000000000002,"5min":0.70599999999999996,"15min":0.66300000000000003},"max":{"1min":1.206,"5min":1.2729999999999999,"15min":1.3140000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.01,"5min":0.96899999999999997,"15min":0.97099999999999997},"min":{"1min":0.76600000000000001,"5min":0.65600000000000003,"15min":0.64900000000000002},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.93899999999999995}]},{"osd":5,"last update":"Fri Jun 17 13:57:38 2022","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94799999999999995,"15min":0.94999999999999996},"min":{"1min":0.81999999999999995,"5min":0.73699999999999999,"15min":0.68200000000000005},"max":{"1min":1.1060000000000001,"5min":1.371,"15min":1.371},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.995,"5min":1,"15min":0.999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.74199999999999999},"max":{"1min":1.1879999999999999,"5min":1.623,"15min":1.623},"last":0.98599999999999999}]},{"osd":6,"last update":"Fri Jun 17 13:58:23 2022","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0589999999999999,"15min":1.044},"min":{"1min":0.80400000000000005,"5min":0.73599999999999999,"15min":0.71499999999999997},"max":{"1min":1.3759999999999999,"5min":1.4630000000000001,"15min":1.603},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.0169999999999999,"15min":1.038},"min":{"1min":0.82299999999999995,"5min":0.75900000000000001,"15min":0.74399999999999999},"max":{"1min":1.298,"5min":1.3740000000000001,"15min":1.75},"last":0.97299999999999998}]},{"osd":7,"last update":"Fri Jun 17 13:58:23 2022","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.0509999999999999,"15min":1.0700000000000001},"min":{"1min":0.78700000000000003,"5min":0.748,"15min":0.748},"max":{"1min":1.2869999999999999,"5min":1.6799999999999999,"15min":1.7929999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.1060000000000001,"15min":1.111},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.76300000000000001},"max":{"1min":1.321,"5min":1.506,"15min":1.5209999999999999},"last":1.0129999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575713,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050500,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321340,"statfs":{"total":96636764160,"available":95561052160,"internally_reserved":1073741824,"allocated":1961984,"data_stored":962548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 17 13:57:54 2022","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.61499999999999999,"15min":0.61799999999999999},"min":{"1min":0.47999999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.873,"5min":0.94699999999999995,"15min":1.208},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73899999999999999,"15min":0.72699999999999998},"min":{"1min":0.56299999999999994,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.097,"5min":1.1819999999999999,"15min":1.258},"last":0.92600000000000005}]},{"osd":2,"last update":"Fri Jun 17 13:57:54 2022","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.625,"15min":0.63100000000000001},"min":{"1min":0.495,"5min":0.46700000000000003,"15min":0.45100000000000001},"max":{"1min":0.85899999999999999,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.77600000000000002,"15min":0.76100000000000001},"min":{"1min":0.56299999999999994,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.96399999999999997,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.59999999999999998}]},{"osd":3,"last update":"Fri Jun 17 13:57:54 2022","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.82199999999999995,"15min":0.81100000000000005},"min":{"1min":0.623,"5min":0.58499999999999996,"15min":0.51000000000000001},"max":{"1min":1.036,"5min":1.1000000000000001,"15min":1.349},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81899999999999995,"15min":0.82299999999999995},"min":{"1min":0.65200000000000002,"5min":0.627,"15min":0.627},"max":{"1min":1.036,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.81299999999999994}]},{"osd":4,"last update":"Fri Jun 17 13:57:54 2022","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.86399999999999999,"15min":0.879},"min":{"1min":0.66500000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.97499999999999998,"5min":1.145,"15min":1.2809999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.95299999999999996,"15min":0.95199999999999996},"min":{"1min":0.70699999999999996,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.2350000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.876}]},{"osd":5,"last update":"Fri Jun 17 13:57:54 2022","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.99299999999999999,"15min":0.996},"min":{"1min":0.71699999999999997,"5min":0.69799999999999995,"15min":0.69099999999999995},"max":{"1min":1.464,"5min":1.5129999999999999,"15min":1.623},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.006,"5min":0.96199999999999997,"15min":0.95499999999999996},"min":{"1min":0.77000000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":1.0589999999999999}]},{"osd":6,"last update":"Fri Jun 17 13:57:54 2022","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0529999999999999,"15min":1.0449999999999999},"min":{"1min":0.77300000000000002,"5min":0.748,"15min":0.69499999999999995},"max":{"1min":1.468,"5min":1.4910000000000001,"15min":1.51},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0920000000000001,"15min":1.1000000000000001},"min":{"1min":0.79900000000000004,"5min":0.74199999999999999,"15min":0.71999999999999997},"max":{"1min":1.3600000000000001,"5min":1.5740000000000001,"15min":1.6819999999999999},"last":1.044}]},{"osd":7,"last update":"Fri Jun 17 13:57:54 2022","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.115,"15min":1.099},"min":{"1min":0.93300000000000005,"5min":0.79500000000000004,"15min":0.73999999999999999},"max":{"1min":1.411,"5min":1.536,"15min":1.536},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.145,"5min":1.139,"15min":1.1499999999999999},"min":{"1min":0.91600000000000004,"5min":0.85199999999999998,"15min":0.78400000000000003},"max":{"1min":1.431,"5min":1.524,"15min":1.619},"last":1.1180000000000001}]}]}]}} 2022-06-17T13:58:31.927 DEBUG:teuthology.orchestra.run.smithi036:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2022-06-17T13:58:32.036 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.036+0000 7feafa06d700 1 Processor -- start 2022-06-17T13:58:32.037 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.036+0000 7feafa06d700 1 -- start start 2022-06-17T13:58:32.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.036+0000 7feafa06d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7feaf40b65f0 0x7feaf40b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:32.040 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.036+0000 7feafa06d700 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feaf40adf10 0x7feaf4140530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:32.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.036+0000 7feafa06d700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf413fdf0 con 0x7feaf40b15b0 2022-06-17T13:58:32.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.036+0000 7feafa06d700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf40ae8e0 con 0x7feaf40b65f0 2022-06-17T13:58:32.041 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.036+0000 7feafa06d700 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf4140090 con 0x7feaf40adf10 2022-06-17T13:58:32.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.036+0000 7feaf906b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7feaf40b65f0 0x7feaf40b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:32.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.036+0000 7feaf986c700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7feaf40b15b0 0x7feaf40af630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:59000/0 (socket says 172.21.15.36:59000) 2022-06-17T13:58:32.042 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.036+0000 7feaf986c700 1 -- 172.21.15.36:0/228561598 learned_addr learned my addr 172.21.15.36:0/228561598 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-06-17T13:58:32.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.036+0000 7feaf886a700 1 --2- 172.21.15.36:0/228561598 >> v2:172.21.15.70:3300/0 conn(0x7feaf40adf10 0x7feaf4140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:32.043 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.040+0000 7feaf906b700 1 -- 172.21.15.36:0/228561598 >> v2:172.21.15.70:3300/0 conn(0x7feaf40adf10 msgr2=0x7feaf4140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-17T13:58:32.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.040+0000 7feaf906b700 1 --2- 172.21.15.36:0/228561598 >> v2:172.21.15.70:3300/0 conn(0x7feaf40adf10 0x7feaf4140530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:32.044 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.040+0000 7feaf906b700 1 -- 172.21.15.36:0/228561598 >> v1:172.21.15.36:6789/0 conn(0x7feaf40b15b0 legacy=0x7feaf40af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:32.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.040+0000 7feaf3fff700 1 -- 172.21.15.36:0/228561598 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4239488699 0 0) 0x7feaf413fdf0 con 0x7feaf40b15b0 2022-06-17T13:58:32.045 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.040+0000 7feaf906b700 1 -- 172.21.15.36:0/228561598 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf4140a20 con 0x7feaf40b65f0 2022-06-17T13:58:32.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.040+0000 7feaf906b700 1 --2- 172.21.15.36:0/228561598 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaf40b65f0 0x7feaf40b69d0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 rx=0x7feae40032f0 tx=0x7feae40091d0).ready entity=mon.0 client_cookie=53523e562bad62ec server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:32.046 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.040+0000 7feaf3fff700 1 -- 172.21.15.36:0/228561598 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feae4009ea0 con 0x7feaf40b65f0 2022-06-17T13:58:32.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.040+0000 7feaf3fff700 1 -- 172.21.15.36:0/228561598 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae400b870 con 0x7feaf40b65f0 2022-06-17T13:58:32.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.040+0000 7feaf3fff700 1 -- 172.21.15.36:0/228561598 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feae400b9e0 con 0x7feaf40b65f0 2022-06-17T13:58:32.047 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.040+0000 7feafa06d700 1 -- 172.21.15.36:0/228561598 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaf40b65f0 msgr2=0x7feaf40b69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:32.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.040+0000 7feafa06d700 1 --2- 172.21.15.36:0/228561598 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaf40b65f0 0x7feaf40b69d0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 rx=0x7feae40032f0 tx=0x7feae40091d0).stop 2022-06-17T13:58:32.048 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.040+0000 7feafa06d700 1 -- 172.21.15.36:0/228561598 shutdown_connections 2022-06-17T13:58:32.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.040+0000 7feafa06d700 1 --2- 172.21.15.36:0/228561598 >> v2:172.21.15.70:3300/0 conn(0x7feaf40adf10 0x7feaf4140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:32.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.040+0000 7feafa06d700 1 --2- 172.21.15.36:0/228561598 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaf40b65f0 0x7feaf40b69d0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:32.049 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.040+0000 7feafa06d700 1 -- 172.21.15.36:0/228561598 >> 172.21.15.36:0/228561598 conn(0x7feaf40614b0 msgr2=0x7feaf4061890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:32.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.040+0000 7feafa06d700 1 -- 172.21.15.36:0/228561598 shutdown_connections 2022-06-17T13:58:32.050 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.040+0000 7feafa06d700 1 -- 172.21.15.36:0/228561598 wait complete. 2022-06-17T13:58:32.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.040+0000 7feafa06d700 1 Processor -- start 2022-06-17T13:58:32.057 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.040+0000 7feafa06d700 1 -- start start 2022-06-17T13:58:32.058 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feafa06d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaf40adf10 0x7feaf40667c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:32.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feafa06d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf40b15b0 0x7feaf413a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:32.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feafa06d700 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaf40b65f0 0x7feaf413f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:32.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feafa06d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf4067030 con 0x7feaf40adf10 2022-06-17T13:58:32.060 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feafa06d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf413c950 con 0x7feaf40b15b0 2022-06-17T13:58:32.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feafa06d700 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf4065530 con 0x7feaf40b65f0 2022-06-17T13:58:32.061 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feaf906b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaf40adf10 0x7feaf40667c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:32.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feaf906b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaf40adf10 0x7feaf40667c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:41474/0 (socket says 172.21.15.36:41474) 2022-06-17T13:58:32.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feaf906b700 1 -- 172.21.15.36:0/258073971 learned_addr learned my addr 172.21.15.36:0/258073971 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-06-17T13:58:32.073 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feaf886a700 1 --2- 172.21.15.36:0/258073971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf40b15b0 0x7feaf413a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:32.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feaf986c700 1 --2- 172.21.15.36:0/258073971 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaf40b65f0 0x7feaf413f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:32.074 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feaf906b700 1 -- 172.21.15.36:0/258073971 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaf40b65f0 msgr2=0x7feaf413f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:32.075 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feaf906b700 1 --2- 172.21.15.36:0/258073971 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaf40b65f0 0x7feaf413f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:32.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feaf906b700 1 -- 172.21.15.36:0/258073971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf40b15b0 msgr2=0x7feaf413a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:32.076 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feaf906b700 1 --2- 172.21.15.36:0/258073971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf40b15b0 0x7feaf413a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:32.077 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feaf906b700 1 -- 172.21.15.36:0/258073971 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae4009470 con 0x7feaf40adf10 2022-06-17T13:58:32.078 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feaf886a700 1 --2- 172.21.15.36:0/258073971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf40b15b0 0x7feaf413a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T13:58:32.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feaf986c700 1 --2- 172.21.15.36:0/258073971 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaf40b65f0 0x7feaf413f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-17T13:58:32.079 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feaf906b700 1 --2- 172.21.15.36:0/258073971 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaf40adf10 0x7feaf40667c0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 rx=0x7feae40032f0 tx=0x7feae4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:32.080 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feaf27fc700 1 -- 172.21.15.36:0/258073971 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feae400b9e0 con 0x7feaf40adf10 2022-06-17T13:58:32.081 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feaf27fc700 1 -- 172.21.15.36:0/258073971 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae401d600 con 0x7feaf40adf10 2022-06-17T13:58:32.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feafa06d700 1 -- 172.21.15.36:0/258073971 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf4065930 con 0x7feaf40adf10 2022-06-17T13:58:32.082 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feaf27fc700 1 -- 172.21.15.36:0/258073971 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feae401d960 con 0x7feaf40adf10 2022-06-17T13:58:32.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feafa06d700 1 -- 172.21.15.36:0/258073971 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf413abc0 con 0x7feaf40adf10 2022-06-17T13:58:32.083 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feafa06d700 1 -- 172.21.15.36:0/258073971 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf4066e90 con 0x7feaf40adf10 2022-06-17T13:58:32.084 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feaf27fc700 1 -- 172.21.15.36:0/258073971 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 65964+0+0 (secure 0 0 0) 0x7feae402d070 con 0x7feaf40adf10 2022-06-17T13:58:32.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feaf27fc700 1 --2- 172.21.15.36:0/258073971 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fead4053ad0 0x7fead4055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-17T13:58:32.085 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feaf27fc700 1 -- 172.21.15.36:0/258073971 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7feae406f880 con 0x7feaf40adf10 2022-06-17T13:58:32.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.044+0000 7feaf886a700 1 --2- 172.21.15.36:0/258073971 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fead4053ad0 0x7fead4055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-17T13:58:32.086 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.048+0000 7feaf886a700 1 --2- 172.21.15.36:0/258073971 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fead4053ad0 0x7fead4055f90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7feae8008000 tx=0x7feae8008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-17T13:58:32.087 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.052+0000 7feaf27fc700 1 -- 172.21.15.36:0/258073971 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7feaf4066e90 con 0x7feaf40adf10 2022-06-17T13:58:32.179 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.180+0000 7feafa06d700 1 -- 172.21.15.36:0/258073971 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7feaf4066e90 con 0x7feaf40adf10 2022-06-17T13:58:32.187 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.188+0000 7feaf27fc700 1 -- 172.21.15.36:0/258073971 <== mon.0 v2:172.21.15.36:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7feae4046960 con 0x7feaf40adf10 2022-06-17T13:58:32.188 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.188+0000 7feaf27fc700 1 -- 172.21.15.36:0/258073971 <== mon.0 v2:172.21.15.36:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v25)=0 v25) v1 ==== 123+0+0 (secure 0 0 0) 0x7feaf4066e90 con 0x7feaf40adf10 2022-06-17T13:58:32.193 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.192+0000 7feafa06d700 1 -- 172.21.15.36:0/258073971 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fead4053ad0 msgr2=0x7fead4055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:32.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.192+0000 7feafa06d700 1 --2- 172.21.15.36:0/258073971 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fead4053ad0 0x7fead4055f90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7feae8008000 tx=0x7feae8008040).stop 2022-06-17T13:58:32.194 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.192+0000 7feafa06d700 1 -- 172.21.15.36:0/258073971 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaf40adf10 msgr2=0x7feaf40667c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-17T13:58:32.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.192+0000 7feafa06d700 1 --2- 172.21.15.36:0/258073971 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaf40adf10 0x7feaf40667c0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 rx=0x7feae40032f0 tx=0x7feae4016000).stop 2022-06-17T13:58:32.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.192+0000 7feafa06d700 1 -- 172.21.15.36:0/258073971 shutdown_connections 2022-06-17T13:58:32.195 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.192+0000 7feafa06d700 1 --2- 172.21.15.36:0/258073971 >> [v2:172.21.15.36:6800/22419,v1:172.21.15.36:6801/22419] conn(0x7fead4053ad0 0x7fead4055f90 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:32.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.192+0000 7feafa06d700 1 --2- 172.21.15.36:0/258073971 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaf40b65f0 0x7feaf413f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:32.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.192+0000 7feafa06d700 1 --2- 172.21.15.36:0/258073971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf40b15b0 0x7feaf413a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:32.196 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.192+0000 7feafa06d700 1 --2- 172.21.15.36:0/258073971 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7feaf40adf10 0x7feaf40667c0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-17T13:58:32.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.192+0000 7feafa06d700 1 -- 172.21.15.36:0/258073971 >> 172.21.15.36:0/258073971 conn(0x7feaf40614b0 msgr2=0x7feaf405bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-17T13:58:32.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.192+0000 7feafa06d700 1 -- 172.21.15.36:0/258073971 shutdown_connections 2022-06-17T13:58:32.197 INFO:teuthology.orchestra.run.smithi036.stderr:2022-06-17T13:58:32.192+0000 7feafa06d700 1 -- 172.21.15.36:0/258073971 wait complete. 2022-06-17T13:58:32.208 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/ceph.py", line 1839, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/run_tasks.py", line 188, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/ceph_fuse.py", line 160, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait if not (self.is_mounted() and self.fuse_daemon): File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 148, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-06-17T13:58:32.212 INFO:teuthology.misc:Shutting down mds daemons... 2022-06-17T13:58:32.213 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-17T13:58:32.214 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T13:58:32.228 INFO:tasks.ceph.mds.a:Stopped 2022-06-17T13:58:32.230 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-17T13:58:32.231 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T13:58:32.248 INFO:tasks.ceph.mds.b:Stopped 2022-06-17T13:58:32.249 INFO:teuthology.misc:Shutting down osd daemons... 2022-06-17T13:58:32.249 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2022-06-17T13:58:32.250 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T13:58:32.321 INFO:tasks.ceph.osd.0:Stopped 2022-06-17T13:58:32.322 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2022-06-17T13:58:32.324 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T13:58:32.402 INFO:tasks.ceph.osd.1:Stopped 2022-06-17T13:58:32.404 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2022-06-17T13:58:32.406 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T13:58:32.488 INFO:tasks.ceph.osd.2:Stopped 2022-06-17T13:58:32.489 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2022-06-17T13:58:32.490 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T13:58:32.556 INFO:tasks.ceph.osd.3:Stopped 2022-06-17T13:58:32.557 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2022-06-17T13:58:32.558 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T13:58:32.634 INFO:tasks.ceph.osd.4:Stopped 2022-06-17T13:58:32.635 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2022-06-17T13:58:32.636 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T13:58:32.716 INFO:tasks.ceph.osd.5:Stopped 2022-06-17T13:58:32.718 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2022-06-17T13:58:32.719 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T13:58:35.553 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2022-06-17T13:58:38.823 INFO:tasks.ceph.osd.6:Stopped 2022-06-17T13:58:38.825 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2022-06-17T13:58:38.826 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T13:58:38.885 INFO:tasks.ceph.osd.7:Stopped 2022-06-17T13:58:38.886 INFO:teuthology.misc:Shutting down mgr daemons... 2022-06-17T13:58:38.887 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2022-06-17T13:58:38.888 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T13:58:38.918 INFO:tasks.ceph.mgr.y:Stopped 2022-06-17T13:58:38.919 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2022-06-17T13:58:38.920 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T13:58:38.947 INFO:tasks.ceph.mgr.x:Stopped 2022-06-17T13:58:38.948 INFO:teuthology.misc:Shutting down mon daemons... 2022-06-17T13:58:38.950 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2022-06-17T13:58:38.951 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T13:58:38.978 INFO:tasks.ceph.mon.a:Stopped 2022-06-17T13:58:38.979 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2022-06-17T13:58:38.980 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T13:58:39.013 INFO:tasks.ceph.mon.b:Stopped 2022-06-17T13:58:39.014 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2022-06-17T13:58:39.015 INFO:teuthology.orchestra.run:waiting for 300 2022-06-17T13:58:39.036 INFO:tasks.ceph.mon.c:Stopped 2022-06-17T13:58:39.038 INFO:tasks.ceph:Checking cluster log for badness... 2022-06-17T13:58:39.040 DEBUG:teuthology.orchestra.run.smithi036:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'evicting unresponsive client' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'has not responded to cap revoke by MDS for over' | egrep -v MDS_CLIENT_LATE_RELEASE | egrep -v 'responding to mclientcaps' | head -n 1 2022-06-17T13:58:39.061 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi036.front.sepia.ceph.com 2022-06-17T13:58:39.063 DEBUG:teuthology.orchestra.run.smithi036:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2022-06-17T13:58:39.280 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:58:39.283 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:58:39.287 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:58:39.291 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:58:39.294 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:58:39.495 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi036.front.sepia.ceph.com 2022-06-17T13:58:39.497 DEBUG:teuthology.orchestra.run.smithi036:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2022-06-17T13:58:39.714 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi036.front.sepia.ceph.com 2022-06-17T13:58:39.716 DEBUG:teuthology.orchestra.run.smithi036:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2022-06-17T13:58:39.880 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi036.front.sepia.ceph.com 2022-06-17T13:58:39.881 DEBUG:teuthology.orchestra.run.smithi036:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2022-06-17T13:58:40.030 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi070.front.sepia.ceph.com 2022-06-17T13:58:40.032 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2022-06-17T13:58:40.263 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi070.front.sepia.ceph.com 2022-06-17T13:58:40.264 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2022-06-17T13:58:40.478 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi070.front.sepia.ceph.com 2022-06-17T13:58:40.480 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2022-06-17T13:58:40.554 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2022-06-17T13:58:40.710 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi070.front.sepia.ceph.com 2022-06-17T13:58:40.712 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2022-06-17T13:58:40.897 INFO:tasks.ceph:Archiving mon data... 2022-06-17T13:58:40.901 DEBUG:teuthology.misc:Transferring archived files from smithi036:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821/data/mon.a.tgz 2022-06-17T13:58:40.902 DEBUG:teuthology.orchestra.run.smithi036:> mktemp 2022-06-17T13:58:40.911 INFO:teuthology.orchestra.run.smithi036.stdout:/tmp/tmp.P2Op1b5sS8 2022-06-17T13:58:40.912 DEBUG:teuthology.orchestra.run.smithi036:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.P2Op1b5sS8 2022-06-17T13:58:41.156 DEBUG:teuthology.orchestra.run.smithi036:> sudo chmod 0666 /tmp/tmp.P2Op1b5sS8 2022-06-17T13:58:41.184 DEBUG:teuthology.orchestra.remote:smithi036:/tmp/tmp.P2Op1b5sS8 is 471KB 2022-06-17T13:58:41.297 DEBUG:teuthology.orchestra.run.smithi036:> rm -fr /tmp/tmp.P2Op1b5sS8 2022-06-17T13:58:41.307 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821/data/mon.b.tgz 2022-06-17T13:58:41.308 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2022-06-17T13:58:41.315 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.OXywig1E6e 2022-06-17T13:58:41.316 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.OXywig1E6e 2022-06-17T13:58:41.510 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.OXywig1E6e 2022-06-17T13:58:41.537 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.OXywig1E6e is 471KB 2022-06-17T13:58:41.615 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.OXywig1E6e 2022-06-17T13:58:41.623 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821/data/mon.c.tgz 2022-06-17T13:58:41.624 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2022-06-17T13:58:41.671 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.vKpYQBNhHV 2022-06-17T13:58:41.672 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.vKpYQBNhHV 2022-06-17T13:58:41.866 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.vKpYQBNhHV 2022-06-17T13:58:41.941 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.vKpYQBNhHV is 471KB 2022-06-17T13:58:42.013 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.vKpYQBNhHV 2022-06-17T13:58:42.023 INFO:tasks.ceph:Cleaning ceph cluster... 2022-06-17T13:58:42.026 DEBUG:teuthology.orchestra.run.smithi036:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2022-06-17T13:58:42.031 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2022-06-17T13:58:42.067 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2022-06-17T13:58:42.072 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2022-06-17T13:58:42.076 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2022-06-17T13:58:42.095 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2022-06-17T13:58:42.096 DEBUG:teuthology.orchestra.run.smithi036:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2022-06-17T13:58:42.100 DEBUG:teuthology.orchestra.run.smithi070:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2022-06-17T13:58:42.104 DEBUG:teuthology.orchestra.run.smithi107:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2022-06-17T13:58:42.126 INFO:teuthology.orchestra.run.smithi070.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2022-06-17T13:58:42.130 DEBUG:teuthology.orchestra.run.smithi136:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2022-06-17T13:58:42.132 INFO:teuthology.orchestra.run.smithi036.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2022-06-17T13:58:42.137 DEBUG:teuthology.orchestra.run.smithi143:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2022-06-17T13:58:42.156 INFO:teuthology.orchestra.run.smithi107.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2022-06-17T13:58:42.168 INFO:teuthology.orchestra.run.smithi143.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2022-06-17T13:58:42.224 INFO:teuthology.orchestra.run.smithi136.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2022-06-17T13:58:42.231 INFO:tasks.ceph:Archiving crash dumps... 2022-06-17T13:58:42.234 DEBUG:teuthology.misc:Transferring archived files from smithi036:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821/remote/smithi036/crash 2022-06-17T13:58:42.238 DEBUG:teuthology.orchestra.run.smithi036:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2022-06-17T13:58:42.263 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821/remote/smithi070/crash 2022-06-17T13:58:42.265 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2022-06-17T13:58:42.288 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821/remote/smithi107/crash 2022-06-17T13:58:42.294 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2022-06-17T13:58:42.318 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821/remote/smithi136/crash 2022-06-17T13:58:42.320 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2022-06-17T13:58:42.343 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821/remote/smithi143/crash 2022-06-17T13:58:42.346 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2022-06-17T13:58:42.367 INFO:tasks.ceph:Shutting down logrotate 2022-06-17T13:58:42.368 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:58:42.373 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:58:42.376 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:58:42.380 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:58:42.384 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-17T13:58:42.463 DEBUG:teuthology.orchestra.run.smithi036:> sudo rm /etc/logrotate.d/ceph-test.conf 2022-06-17T13:58:42.480 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm /etc/logrotate.d/ceph-test.conf 2022-06-17T13:58:42.497 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm /etc/logrotate.d/ceph-test.conf 2022-06-17T13:58:42.514 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm /etc/logrotate.d/ceph-test.conf 2022-06-17T13:58:42.531 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm /etc/logrotate.d/ceph-test.conf 2022-06-17T13:58:42.549 INFO:tasks.ceph:Compressing logs... 2022-06-17T13:58:42.550 DEBUG:teuthology.orchestra.run.smithi036:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-06-17T13:58:42.555 DEBUG:teuthology.orchestra.run.smithi070:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-06-17T13:58:42.558 DEBUG:teuthology.orchestra.run.smithi107:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-06-17T13:58:42.561 DEBUG:teuthology.orchestra.run.smithi136:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-06-17T13:58:42.564 DEBUG:teuthology.orchestra.run.smithi143:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-06-17T13:58:55.796 INFO:tasks.ceph:Archiving logs... 2022-06-17T13:58:55.798 DEBUG:teuthology.misc:Transferring archived files from smithi036:/var/log/ceph to /home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821/remote/smithi036/log 2022-06-17T13:58:55.801 DEBUG:teuthology.orchestra.run.smithi036:> sudo tar cz -f - -C /var/log/ceph -- . 2022-06-17T13:58:59.705 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/log/ceph to /home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821/remote/smithi070/log 2022-06-17T13:58:59.706 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar cz -f - -C /var/log/ceph -- . 2022-06-17T13:59:02.596 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/log/ceph to /home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821/remote/smithi107/log 2022-06-17T13:59:02.599 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar cz -f - -C /var/log/ceph -- . 2022-06-17T13:59:03.026 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/log/ceph to /home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821/remote/smithi136/log 2022-06-17T13:59:03.028 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar cz -f - -C /var/log/ceph -- . 2022-06-17T13:59:03.069 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/log/ceph to /home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821/remote/smithi143/log 2022-06-17T13:59:03.071 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar cz -f - -C /var/log/ceph -- . 2022-06-17T13:59:03.096 DEBUG:teuthology.run_tasks:Unwinding manager install 2022-06-17T13:59:03.115 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/task/install/__init__.py", line 619, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/run_tasks.py", line 188, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/ceph_fuse.py", line 160, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait if not (self.is_mounted() and self.fuse_daemon): File "/home/teuthworker/src/git.ceph.com_ceph_b3641a55c6b9ed8f813eab4ef0cde6337b2b7ee6/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 148, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_31a356e0c737f1781079045bb8dc366982be8713/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-06-17T13:59:03.122 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits... 2022-06-17T13:59:03.124 DEBUG:teuthology.orchestra.run.smithi036:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2022-06-17T13:59:03.129 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2022-06-17T13:59:03.135 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2022-06-17T13:59:03.140 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2022-06-17T13:59:03.144 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2022-06-17T13:59:03.298 DEBUG:teuthology.run_tasks:Unwinding manager clock 2022-06-17T13:59:03.321 INFO:teuthology.task.clock:Checking final clock skew... 2022-06-17T13:59:03.324 DEBUG:teuthology.orchestra.run.smithi036:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-06-17T13:59:03.332 DEBUG:teuthology.orchestra.run.smithi070:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-06-17T13:59:03.336 DEBUG:teuthology.orchestra.run.smithi107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-06-17T13:59:03.341 DEBUG:teuthology.orchestra.run.smithi136:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-06-17T13:59:03.345 DEBUG:teuthology.orchestra.run.smithi143:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-06-17T13:59:03.356 INFO:teuthology.orchestra.run.smithi036.stdout: remote refid st t when poll reach delay offset jitter 2022-06-17T13:59:03.356 INFO:teuthology.orchestra.run.smithi036.stdout:============================================================================== 2022-06-17T13:59:03.357 INFO:teuthology.orchestra.run.smithi036.stdout:+hv01.front.sepi 162.248.241.94 3 u 211 256 377 0.120 3.100 0.225 2022-06-17T13:59:03.357 INFO:teuthology.orchestra.run.smithi036.stdout:*hv02.front.sepi 163.237.218.18 2 u 224 256 377 0.196 -0.629 0.164 2022-06-17T13:59:03.358 INFO:teuthology.orchestra.run.smithi036.stdout:+hv03.front.sepi 216.229.4.69 3 u 204 256 377 0.191 -3.709 0.262 2022-06-17T13:59:03.358 INFO:teuthology.orchestra.run.smithi036.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2022-06-17T13:59:03.360 INFO:teuthology.orchestra.run.smithi070.stdout: remote refid st t when poll reach delay offset jitter 2022-06-17T13:59:03.360 INFO:teuthology.orchestra.run.smithi070.stdout:============================================================================== 2022-06-17T13:59:03.361 INFO:teuthology.orchestra.run.smithi070.stdout:+hv01.front.sepi 162.248.241.94 3 u 72 128 377 0.169 12.638 1.340 2022-06-17T13:59:03.361 INFO:teuthology.orchestra.run.smithi070.stdout:*hv02.front.sepi 163.237.218.18 2 u 11 128 377 0.139 8.859 1.183 2022-06-17T13:59:03.362 INFO:teuthology.orchestra.run.smithi070.stdout:+hv03.front.sepi 216.229.4.69 3 u 51 128 377 0.171 4.494 0.758 2022-06-17T13:59:03.362 INFO:teuthology.orchestra.run.smithi070.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2022-06-17T13:59:03.364 INFO:teuthology.orchestra.run.smithi107.stdout: remote refid st t when poll reach delay offset jitter 2022-06-17T13:59:03.364 INFO:teuthology.orchestra.run.smithi107.stdout:============================================================================== 2022-06-17T13:59:03.365 INFO:teuthology.orchestra.run.smithi107.stdout:+hv01.front.sepi 162.248.241.94 3 u 120 256 377 0.182 2.845 0.164 2022-06-17T13:59:03.365 INFO:teuthology.orchestra.run.smithi107.stdout:*hv02.front.sepi 163.237.218.18 2 u 141 256 377 0.192 -0.191 0.542 2022-06-17T13:59:03.366 INFO:teuthology.orchestra.run.smithi107.stdout:+hv03.front.sepi 216.229.4.69 3 u 135 256 377 0.174 -3.669 0.070 2022-06-17T13:59:03.366 INFO:teuthology.orchestra.run.smithi107.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2022-06-17T13:59:03.367 INFO:teuthology.orchestra.run.smithi136.stdout: remote refid st t when poll reach delay offset jitter 2022-06-17T13:59:03.368 INFO:teuthology.orchestra.run.smithi136.stdout:============================================================================== 2022-06-17T13:59:03.368 INFO:teuthology.orchestra.run.smithi136.stdout:+hv01.front.sepi 162.248.241.94 3 u 88 256 377 0.153 11.320 1.734 2022-06-17T13:59:03.368 INFO:teuthology.orchestra.run.smithi136.stdout:*hv02.front.sepi 163.237.218.18 2 u 154 256 377 0.180 7.739 1.725 2022-06-17T13:59:03.369 INFO:teuthology.orchestra.run.smithi136.stdout:+hv03.front.sepi 216.229.4.69 3 u 140 256 377 0.176 4.776 1.502 2022-06-17T13:59:03.369 INFO:teuthology.orchestra.run.smithi136.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2022-06-17T13:59:03.380 INFO:teuthology.orchestra.run.smithi143.stdout: remote refid st t when poll reach delay offset jitter 2022-06-17T13:59:03.381 INFO:teuthology.orchestra.run.smithi143.stdout:============================================================================== 2022-06-17T13:59:03.383 INFO:teuthology.orchestra.run.smithi143.stdout:+hv01.front.sepi 162.248.241.94 3 u 80 256 377 0.198 1.940 0.946 2022-06-17T13:59:03.384 INFO:teuthology.orchestra.run.smithi143.stdout:*hv02.front.sepi 163.237.218.18 2 u 150 256 377 0.194 -2.326 0.662 2022-06-17T13:59:03.384 INFO:teuthology.orchestra.run.smithi143.stdout:+hv03.front.sepi 216.229.4.69 3 u 64 256 377 0.176 -5.740 0.581 2022-06-17T13:59:03.384 INFO:teuthology.orchestra.run.smithi143.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2022-06-17T13:59:03.385 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2022-06-17T13:59:03.396 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2022-06-17T13:59:03.398 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2022-06-17T13:59:03.415 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2022-06-17T13:59:03.435 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2022-06-17T13:59:03.455 INFO:teuthology.task.internal:Duration was 5122.142166 seconds 2022-06-17T13:59:03.459 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2022-06-17T13:59:03.481 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2022-06-17T13:59:03.483 DEBUG:teuthology.orchestra.run.smithi036:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2022-06-17T13:59:03.500 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2022-06-17T13:59:03.505 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2022-06-17T13:59:03.510 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2022-06-17T13:59:03.515 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2022-06-17T13:59:03.569 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2022-06-17T13:59:03.570 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi036.front.sepia.ceph.com 2022-06-17T13:59:03.571 DEBUG:teuthology.orchestra.run.smithi036:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2022-06-17T13:59:03.593 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi070.front.sepia.ceph.com 2022-06-17T13:59:03.594 DEBUG:teuthology.orchestra.run.smithi070:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2022-06-17T13:59:03.611 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi107.front.sepia.ceph.com 2022-06-17T13:59:03.613 DEBUG:teuthology.orchestra.run.smithi107:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2022-06-17T13:59:03.629 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi136.front.sepia.ceph.com 2022-06-17T13:59:03.630 DEBUG:teuthology.orchestra.run.smithi136:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2022-06-17T13:59:03.659 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi143.front.sepia.ceph.com 2022-06-17T13:59:03.660 DEBUG:teuthology.orchestra.run.smithi143:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2022-06-17T13:59:03.693 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2022-06-17T13:59:03.694 DEBUG:teuthology.orchestra.run.smithi036:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-06-17T13:59:03.701 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-06-17T13:59:03.709 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-06-17T13:59:03.719 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-06-17T13:59:03.742 DEBUG:teuthology.orchestra.run.smithi143:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-06-17T13:59:03.779 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2022-06-17T13:59:03.780 DEBUG:teuthology.orchestra.run.smithi036:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2022-06-17T13:59:03.795 DEBUG:teuthology.orchestra.run.smithi070:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2022-06-17T13:59:03.805 DEBUG:teuthology.orchestra.run.smithi107:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2022-06-17T13:59:03.814 DEBUG:teuthology.orchestra.run.smithi136:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2022-06-17T13:59:03.823 DEBUG:teuthology.orchestra.run.smithi143:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2022-06-17T13:59:03.989 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2022-06-17T13:59:04.005 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2022-06-17T13:59:04.006 DEBUG:teuthology.orchestra.run.smithi036:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2022-06-17T13:59:04.027 DEBUG:teuthology.orchestra.run.smithi070:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2022-06-17T13:59:04.046 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2022-06-17T13:59:04.063 DEBUG:teuthology.orchestra.run.smithi136:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2022-06-17T13:59:04.087 DEBUG:teuthology.orchestra.run.smithi143:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2022-06-17T13:59:04.107 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2022-06-17T13:59:04.127 DEBUG:teuthology.orchestra.run.smithi036:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2022-06-17T13:59:04.132 DEBUG:teuthology.orchestra.run.smithi070:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2022-06-17T13:59:04.136 DEBUG:teuthology.orchestra.run.smithi107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2022-06-17T13:59:04.141 DEBUG:teuthology.orchestra.run.smithi136:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2022-06-17T13:59:04.143 INFO:teuthology.orchestra.run.smithi036.stdout:kernel.core_pattern = core 2022-06-17T13:59:04.145 DEBUG:teuthology.orchestra.run.smithi143:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2022-06-17T13:59:04.149 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = core 2022-06-17T13:59:04.153 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = core 2022-06-17T13:59:04.157 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = core 2022-06-17T13:59:04.160 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = core 2022-06-17T13:59:04.176 DEBUG:teuthology.orchestra.run.smithi036:> test -e /home/ubuntu/cephtest/archive/coredump 2022-06-17T13:59:04.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T13:59:04.222 DEBUG:teuthology.orchestra.run.smithi070:> test -e /home/ubuntu/cephtest/archive/coredump 2022-06-17T13:59:04.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T13:59:04.231 DEBUG:teuthology.orchestra.run.smithi107:> test -e /home/ubuntu/cephtest/archive/coredump 2022-06-17T13:59:04.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T13:59:04.240 DEBUG:teuthology.orchestra.run.smithi136:> test -e /home/ubuntu/cephtest/archive/coredump 2022-06-17T13:59:04.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T13:59:04.248 DEBUG:teuthology.orchestra.run.smithi143:> test -e /home/ubuntu/cephtest/archive/coredump 2022-06-17T13:59:04.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-17T13:59:04.256 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2022-06-17T13:59:04.276 INFO:teuthology.task.internal:Transferring archived files... 2022-06-17T13:59:04.278 DEBUG:teuthology.misc:Transferring archived files from smithi036:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821/remote/smithi036 2022-06-17T13:59:04.280 DEBUG:teuthology.orchestra.run.smithi036:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2022-06-17T13:59:04.323 DEBUG:teuthology.misc:Transferring archived files from smithi070:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821/remote/smithi070 2022-06-17T13:59:04.324 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2022-06-17T13:59:04.367 DEBUG:teuthology.misc:Transferring archived files from smithi107:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821/remote/smithi107 2022-06-17T13:59:04.368 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2022-06-17T13:59:04.414 DEBUG:teuthology.misc:Transferring archived files from smithi136:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821/remote/smithi136 2022-06-17T13:59:04.415 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2022-06-17T13:59:04.457 DEBUG:teuthology.misc:Transferring archived files from smithi143:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2022-06-17_04:10:03-fs-octopus-distro-default-smithi/6883821/remote/smithi143 2022-06-17T13:59:04.459 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2022-06-17T13:59:04.501 INFO:teuthology.task.internal:Removing archive directory... 2022-06-17T13:59:04.502 DEBUG:teuthology.orchestra.run.smithi036:> rm -rf -- /home/ubuntu/cephtest/archive 2022-06-17T13:59:04.506 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf -- /home/ubuntu/cephtest/archive 2022-06-17T13:59:04.509 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf -- /home/ubuntu/cephtest/archive 2022-06-17T13:59:04.512 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf -- /home/ubuntu/cephtest/archive 2022-06-17T13:59:04.516 DEBUG:teuthology.orchestra.run.smithi143:> rm -rf -- /home/ubuntu/cephtest/archive 2022-06-17T13:59:04.523 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2022-06-17T13:59:04.550 INFO:teuthology.task.internal:Not uploading archives. 2022-06-17T13:59:04.551 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2022-06-17T13:59:04.575 INFO:teuthology.task.internal:Tidying up after the test... 2022-06-17T13:59:04.577 DEBUG:teuthology.orchestra.run.smithi036:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2022-06-17T13:59:04.581 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2022-06-17T13:59:04.585 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2022-06-17T13:59:04.588 INFO:teuthology.orchestra.run.smithi036.stdout: 524315 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 17 13:59 /home/ubuntu/cephtest 2022-06-17T13:59:04.596 INFO:teuthology.orchestra.run.smithi070.stdout: 524315 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 17 13:59 /home/ubuntu/cephtest 2022-06-17T13:59:04.603 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2022-06-17T13:59:04.606 DEBUG:teuthology.orchestra.run.smithi143:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2022-06-17T13:59:04.612 INFO:teuthology.orchestra.run.smithi136.stdout: 524315 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 17 13:59 /home/ubuntu/cephtest 2022-06-17T13:59:04.615 INFO:teuthology.orchestra.run.smithi143.stdout: 524315 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 17 13:59 /home/ubuntu/cephtest 2022-06-18T00:31:59.743 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2022-06-18T00:32:00.365 DEBUG:teuthology.task.console_log:Killing console logger for smithi036 2022-06-18T00:32:00.529 DEBUG:teuthology.task.console_log:Killing console logger for smithi070 2022-06-18T00:32:00.535 DEBUG:teuthology.task.console_log:Killing console logger for smithi107 2022-06-18T00:32:00.664 DEBUG:teuthology.task.console_log:Killing console logger for smithi136 2022-06-18T00:32:00.665 DEBUG:teuthology.task.console_log:Killing console logger for smithi143 2022-06-18T00:32:00.691 DEBUG:teuthology.exit:Finished running handlers